[PATCH v3 00/24] modules: add meta-data database

[PATCH v3 00/24] modules: add meta-data database

[Gerd Hoffmann]

This patch series adds support for module meta-data.  Today this is
either hard-coded in qemu (see qemu_load_module_for_opts) or handled
with manually maintained lists in util/module (see module_deps[] and
qom_modules[]).  This series replaced that scheme with annotation
macros, so the meta-data can go into the module source code and -- for
example -- the module_obj() annotations can go next to the TypeInfo
struct for the object class.

Patches 1-3 put the infrastructure in place:  Add the annotation macros,
add a script to collect the meta-data, add a script to compile the
meta-data into C source code which we can then add to qemu.

Patches 4-12 add annotations macros to the modules we have.

Patches 13-15 put the modinfo database into use and remove the
module_deps[] and qom_modules[] lists.

Patch 16 adds two tracepoints for easier trouble-shooting.

Patches 17-19 add support for target-specific modules.

Patches 20-14 start building accelerators modular.  So far it is
only qtest (all archs) and a small fraction of tcg (x86 only).

take care,
  Gerd

Gerd Hoffmann (24):
  modules: add modinfo macros
  modules: collect module meta-data
  modules: generate modinfo.c
  modules: add qxl module annotations
  modules: add virtio-gpu module annotations
  modules: add chardev module annotations
  modules: add audio module annotations
  modules: add usb-redir module annotations
  modules: add ccid module annotations
  modules: add ui module annotations
  modules: add s390x module annotations
  modules: add block module annotations
  modules: use modinfo for dependencies
  modules: use modinfo for qom load
  modules: use modinfo for qemu opts load
  modules: add tracepoints
  modules: check arch and block load on mismatch
  modules: check arch on qom lookup
  modules: target-specific module build infrastructure
  accel: autoload modules
  accel: add qtest module annotations
  accel: build qtest modular
  accel: add tcg module annotations
  accel: build tcg modular

 scripts/modinfo-collect.py      |  67 +++++++++++
 scripts/modinfo-generate.py     |  84 ++++++++++++++
 include/qemu/module.h           |  43 +++++++
 accel/accel-common.c            |   2 +-
 accel/accel-softmmu.c           |   2 +-
 accel/qtest/qtest.c             |   2 +
 accel/tcg/tcg-accel-ops.c       |   1 +
 accel/tcg/tcg-all.c             |   1 +
 audio/spiceaudio.c              |   2 +
 block/iscsi-opts.c              |   1 +
 chardev/baum.c                  |   1 +
 chardev/spice.c                 |   4 +
 hw/display/qxl.c                |   4 +
 hw/display/vhost-user-gpu-pci.c |   1 +
 hw/display/vhost-user-gpu.c     |   1 +
 hw/display/vhost-user-vga.c     |   1 +
 hw/display/virtio-gpu-base.c    |   1 +
 hw/display/virtio-gpu-gl.c      |   3 +
 hw/display/virtio-gpu-pci-gl.c  |   3 +
 hw/display/virtio-gpu-pci.c     |   2 +
 hw/display/virtio-gpu.c         |   1 +
 hw/display/virtio-vga-gl.c      |   3 +
 hw/display/virtio-vga.c         |   2 +
 hw/s390x/virtio-ccw-gpu.c       |   3 +
 hw/usb/ccid-card-emulated.c     |   1 +
 hw/usb/ccid-card-passthru.c     |   1 +
 hw/usb/redirect.c               |   1 +
 softmmu/vl.c                    |  24 ++--
 stubs/module-opts.c             |   4 -
 ui/egl-headless.c               |   4 +
 ui/gtk.c                        |   4 +
 ui/sdl2.c                       |   4 +
 ui/spice-app.c                  |   3 +
 ui/spice-core.c                 |   5 +
 util/module.c                   | 200 ++++++++++++++++++--------------
 accel/qtest/meson.build         |   8 +-
 accel/tcg/meson.build           |   5 +-
 meson.build                     |  76 ++++++++++++
 util/trace-events               |   4 +
 39 files changed, 462 insertions(+), 117 deletions(-)
 create mode 100755 scripts/modinfo-collect.py
 create mode 100755 scripts/modinfo-generate.py

-- 
2.31.1

[PATCH v3 01/24] modules: add modinfo macros

[Gerd Hoffmann]

Add macros for module info annotations.

Instead of having that module meta-data stored in lists in util/module.c
place directly in the module source code.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 include/qemu/module.h | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/include/qemu/module.h b/include/qemu/module.h
index 944d403cbd15..81ef086da023 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -73,4 +73,29 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail);
 void module_load_qom_one(const char *type);
 void module_load_qom_all(void);
 
+/*
+ * module info annotation macros
+ *
+ * scripts/modinfo-collect.py will collect module info,
+ * using the preprocessor and -DQEMU_MODINFO
+ */
+#ifdef QEMU_MODINFO
+# define modinfo(kind, value) \
+    MODINFO_START kind value MODINFO_END
+#else
+# define modinfo(kind, value)
+#endif
+
+/* module implements QOM type <name> */
+#define module_obj(name) modinfo(obj, name)
+
+/* module has a dependency on <name> */
+#define module_dep(name) modinfo(dep, name)
+
+/* module is for target architecture <name> */
+#define module_arch(name) modinfo(arch, name)
+
+/* module registers QemuOpts <name> */
+#define module_opts(name) modinfo(opts, name)
+
 #endif
-- 
2.31.1

[PATCH v3 02/24] modules: collect module meta-data

[Gerd Hoffmann]

Add script to collect the module meta-data from the source code,
store the results in *.modinfo files.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 scripts/modinfo-collect.py | 67 ++++++++++++++++++++++++++++++++++++++
 meson.build                | 11 +++++++
 2 files changed, 78 insertions(+)
 create mode 100755 scripts/modinfo-collect.py

diff --git a/scripts/modinfo-collect.py b/scripts/modinfo-collect.py
new file mode 100755
index 000000000000..4acb188c3e89
--- /dev/null
+++ b/scripts/modinfo-collect.py
@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+import os
+import sys
+import json
+import shlex
+import subprocess
+
+def find_command(src, target, compile_commands):
+    for command in compile_commands:
+        if command['file'] != src:
+            continue
+        if target != '' and command['command'].find(target) == -1:
+            continue
+        return command['command']
+    return 'false'
+
+def process_command(src, command):
+    skip = False
+    arg = False
+    out = []
+    for item in shlex.split(command):
+        if arg:
+            out.append(x)
+            arg = False
+            continue
+        if skip:
+            skip = False
+            continue
+        if item == '-MF' or item == '-MQ' or item == '-o':
+            skip = True
+            continue
+        if item == '-c':
+            skip = True
+            continue
+        out.append(item)
+    out.append('-DQEMU_MODINFO')
+    out.append('-E')
+    out.append(src)
+    return out
+
+def main(args):
+    target = ''
+    if args[0] == '--target':
+        args.pop(0)
+        target = args.pop(0)
+        print("MODINFO_DEBUG target %s" % target)
+        arch = target[:-8] # cut '-softmmu'
+        print("MODINFO_START arch \"%s\" MODINFO_END" % arch)
+    with open('compile_commands.json') as f:
+        compile_commands = json.load(f)
+    for src in args:
+        print("MODINFO_DEBUG src %s" % src)
+        command = find_command(src, target, compile_commands)
+        cmdline = process_command(src, command)
+        print("MODINFO_DEBUG cmd", cmdline)
+        result = subprocess.run(cmdline, stdout = subprocess.PIPE,
+                                universal_newlines = True)
+        if result.returncode != 0:
+            sys.exit(result.returncode)
+        for line in result.stdout.split('\n'):
+            if line.find('MODINFO') != -1:
+                print(line)
+
+if __name__ == "__main__":
+    main(sys.argv[1:])
diff --git a/meson.build b/meson.build
index a2311eda6ec5..bb99619257d5 100644
--- a/meson.build
+++ b/meson.build
@@ -2021,6 +2021,9 @@ subdir('tests/qtest/fuzz')
 # Library dependencies #
 ########################
 
+modinfo_collect = find_program('scripts/modinfo-collect.py')
+modinfo_files = []
+
 block_mods = []
 softmmu_mods = []
 foreach d, list : modules
@@ -2034,6 +2037,14 @@ foreach d, list : modules
       else
         softmmu_mods += sl
       endif
+      if module_ss.sources() != []
+        modinfo_files += custom_target(d + '-' + m + '.modinfo',
+                                       output: d + '-' + m + '.modinfo',
+                                       input: module_ss.sources(),
+                                       capture: true,
+                                       build_by_default: true, # to be removed when added to a target
+                                       command: [modinfo_collect, '@INPUT@'])
+      endif
     else
       if d == 'block'
         block_ss.add_all(module_ss)
-- 
2.31.1

[PATCH v3 03/24] modules: generate modinfo.c

[Gerd Hoffmann]

Add script to generate C source with a small
database containing the module meta-data.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 scripts/modinfo-generate.py | 84 +++++++++++++++++++++++++++++++++++++
 include/qemu/module.h       | 17 ++++++++
 softmmu/vl.c                |  4 ++
 util/module.c               | 11 +++++
 meson.build                 | 13 +++++-
 5 files changed, 128 insertions(+), 1 deletion(-)
 create mode 100755 scripts/modinfo-generate.py

diff --git a/scripts/modinfo-generate.py b/scripts/modinfo-generate.py
new file mode 100755
index 000000000000..2b925432655a
--- /dev/null
+++ b/scripts/modinfo-generate.py
@@ -0,0 +1,84 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+import os
+import sys
+
+def print_array(name, values):
+    if len(values) == 0:
+        return
+    list = ", ".join(values)
+    print("    .%s = ((const char*[]){ %s, NULL })," % (name, list))
+
+def parse_line(line):
+    kind = ""
+    data = ""
+    get_kind = False
+    get_data = False
+    for item in line.split():
+        if item == "MODINFO_START":
+            get_kind = True
+            continue
+        if item.startswith("MODINFO_END"):
+            get_data = False
+            continue
+        if get_kind:
+            kind = item
+            get_kind = False
+            get_data = True
+            continue
+        if get_data:
+            data += " " + item
+            continue
+    return (kind, data)
+
+def generate(name, lines):
+    arch = ""
+    objs = []
+    deps = []
+    opts = []
+    for line in lines:
+        if line.find("MODINFO_START") != -1:
+            (kind, data) = parse_line(line)
+            if kind == 'obj':
+                objs.append(data)
+            elif kind == 'dep':
+                deps.append(data)
+            elif kind == 'opts':
+                opts.append(data)
+            elif kind == 'arch':
+                arch = data;
+            else:
+                print("unknown:", kind)
+                exit(1)
+
+    print("    .name = \"%s\"," % name)
+    if arch != "":
+        print("    .arch = %s," % arch)
+    print_array("objs", objs)
+    print_array("deps", deps)
+    print_array("opts", opts)
+    print("},{");
+
+def print_pre():
+    print("/* generated by scripts/modinfo.py */")
+    print("#include \"qemu/osdep.h\"")
+    print("#include \"qemu/module.h\"")
+    print("const QemuModinfo qemu_modinfo[] = {{")
+
+def print_post():
+    print("    /* end of list */")
+    print("}};")
+
+def main(args):
+    print_pre()
+    for modinfo in args:
+        with open(modinfo) as f:
+            lines = f.readlines()
+        print("    /* %s */" % modinfo)
+        (basename, ext) = os.path.splitext(modinfo)
+        generate(basename, lines)
+    print_post()
+
+if __name__ == "__main__":
+    main(sys.argv[1:])
diff --git a/include/qemu/module.h b/include/qemu/module.h
index 81ef086da023..a98748d501d3 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -98,4 +98,21 @@ void module_load_qom_all(void);
 /* module registers QemuOpts <name> */
 #define module_opts(name) modinfo(opts, name)
 
+/*
+ * module info database
+ *
+ * scripts/modinfo-generate.c will build this using the data collected
+ * by scripts/modinfo-collect.py
+ */
+typedef struct QemuModinfo QemuModinfo;
+struct QemuModinfo {
+    const char *name;
+    const char *arch;
+    const char **objs;
+    const char **deps;
+    const char **opts;
+};
+extern const QemuModinfo qemu_modinfo[];
+void module_init_info(const QemuModinfo *info);
+
 #endif
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 326c1e908008..a4857ec43ff3 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2740,6 +2740,10 @@ void qemu_init(int argc, char **argv, char **envp)
     error_init(argv[0]);
     qemu_init_exec_dir(argv[0]);
 
+#ifdef CONFIG_MODULES
+    module_init_info(qemu_modinfo);
+#endif
+
     qemu_init_subsystems();
 
     /* first pass of option parsing */
diff --git a/util/module.c b/util/module.c
index eee8ff2de136..8d3e8275b9f7 100644
--- a/util/module.c
+++ b/util/module.c
@@ -110,6 +110,17 @@ void module_call_init(module_init_type type)
 }
 
 #ifdef CONFIG_MODULES
+
+static const QemuModinfo module_info_stub[] = { {
+    /* end of list */
+} };
+static const QemuModinfo *module_info = module_info_stub;
+
+void module_init_info(const QemuModinfo *info)
+{
+    module_info = info;
+}
+
 static int module_load_file(const char *fname, bool mayfail, bool export_symbols)
 {
     GModule *g_module;
diff --git a/meson.build b/meson.build
index bb99619257d5..9cf50a50d39a 100644
--- a/meson.build
+++ b/meson.build
@@ -2022,6 +2022,7 @@ subdir('tests/qtest/fuzz')
 ########################
 
 modinfo_collect = find_program('scripts/modinfo-collect.py')
+modinfo_generate = find_program('scripts/modinfo-generate.py')
 modinfo_files = []
 
 block_mods = []
@@ -2042,7 +2043,6 @@ foreach d, list : modules
                                        output: d + '-' + m + '.modinfo',
                                        input: module_ss.sources(),
                                        capture: true,
-                                       build_by_default: true, # to be removed when added to a target
                                        command: [modinfo_collect, '@INPUT@'])
       endif
     else
@@ -2055,6 +2055,17 @@ foreach d, list : modules
   endforeach
 endforeach
 
+if enable_modules
+  modinfo_src = custom_target('modinfo.c',
+                              output: 'modinfo.c',
+                              input: modinfo_files,
+                              command: [modinfo_generate, '@INPUT@'],
+                              capture: true)
+  modinfo_lib = static_library('modinfo', modinfo_src)
+  modinfo_dep = declare_dependency(link_whole: modinfo_lib)
+  softmmu_ss.add(modinfo_dep)
+endif
+
 nm = find_program('nm')
 undefsym = find_program('scripts/undefsym.py')
 block_syms = custom_target('block.syms', output: 'block.syms',
-- 
2.31.1

[PATCH v3 04/24] modules: add qxl module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/display/qxl.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/hw/display/qxl.c b/hw/display/qxl.c
index 6e1f8ff1b2a7..84f99088e0a0 100644
--- a/hw/display/qxl.c
+++ b/hw/display/qxl.c
@@ -2522,6 +2522,7 @@ static const TypeInfo qxl_primary_info = {
     .parent        = TYPE_PCI_QXL,
     .class_init    = qxl_primary_class_init,
 };
+module_obj("qxl-vga");
 
 static void qxl_secondary_class_init(ObjectClass *klass, void *data)
 {
@@ -2538,6 +2539,7 @@ static const TypeInfo qxl_secondary_info = {
     .parent        = TYPE_PCI_QXL,
     .class_init    = qxl_secondary_class_init,
 };
+module_obj("qxl");
 
 static void qxl_register_types(void)
 {
@@ -2547,3 +2549,5 @@ static void qxl_register_types(void)
 }
 
 type_init(qxl_register_types)
+
+module_dep("ui-spice-core");
-- 
2.31.1

[PATCH v3 05/24] modules: add virtio-gpu module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/display/vhost-user-gpu-pci.c | 1 +
 hw/display/vhost-user-gpu.c     | 1 +
 hw/display/vhost-user-vga.c     | 1 +
 hw/display/virtio-gpu-base.c    | 1 +
 hw/display/virtio-gpu-gl.c      | 3 +++
 hw/display/virtio-gpu-pci-gl.c  | 3 +++
 hw/display/virtio-gpu-pci.c     | 2 ++
 hw/display/virtio-gpu.c         | 1 +
 hw/display/virtio-vga-gl.c      | 3 +++
 hw/display/virtio-vga.c         | 2 ++
 10 files changed, 18 insertions(+)

diff --git a/hw/display/vhost-user-gpu-pci.c b/hw/display/vhost-user-gpu-pci.c
index a02b23ecaf11..daefcf710159 100644
--- a/hw/display/vhost-user-gpu-pci.c
+++ b/hw/display/vhost-user-gpu-pci.c
@@ -43,6 +43,7 @@ static const VirtioPCIDeviceTypeInfo vhost_user_gpu_pci_info = {
     .instance_size = sizeof(VhostUserGPUPCI),
     .instance_init = vhost_user_gpu_pci_initfn,
 };
+module_obj(TYPE_VHOST_USER_GPU_PCI);
 
 static void vhost_user_gpu_pci_register_types(void)
 {
diff --git a/hw/display/vhost-user-gpu.c b/hw/display/vhost-user-gpu.c
index 6cdaa1c73b9b..32ef0061f924 100644
--- a/hw/display/vhost-user-gpu.c
+++ b/hw/display/vhost-user-gpu.c
@@ -596,6 +596,7 @@ static const TypeInfo vhost_user_gpu_info = {
     .instance_finalize = vhost_user_gpu_instance_finalize,
     .class_init = vhost_user_gpu_class_init,
 };
+module_obj(TYPE_VHOST_USER_GPU);
 
 static void vhost_user_gpu_register_types(void)
 {
diff --git a/hw/display/vhost-user-vga.c b/hw/display/vhost-user-vga.c
index a34a99856d73..072c9c65bc75 100644
--- a/hw/display/vhost-user-vga.c
+++ b/hw/display/vhost-user-vga.c
@@ -44,6 +44,7 @@ static const VirtioPCIDeviceTypeInfo vhost_user_vga_info = {
     .instance_size = sizeof(VhostUserVGA),
     .instance_init = vhost_user_vga_inst_initfn,
 };
+module_obj(TYPE_VHOST_USER_VGA);
 
 static void vhost_user_vga_register_types(void)
 {
diff --git a/hw/display/virtio-gpu-base.c b/hw/display/virtio-gpu-base.c
index dd294276cb38..c8da4806e0bb 100644
--- a/hw/display/virtio-gpu-base.c
+++ b/hw/display/virtio-gpu-base.c
@@ -256,6 +256,7 @@ static const TypeInfo virtio_gpu_base_info = {
     .class_init = virtio_gpu_base_class_init,
     .abstract = true
 };
+module_obj(TYPE_VIRTIO_GPU_BASE);
 
 static void
 virtio_register_types(void)
diff --git a/hw/display/virtio-gpu-gl.c b/hw/display/virtio-gpu-gl.c
index d971b480806a..7ab93bf8c829 100644
--- a/hw/display/virtio-gpu-gl.c
+++ b/hw/display/virtio-gpu-gl.c
@@ -154,6 +154,7 @@ static const TypeInfo virtio_gpu_gl_info = {
     .instance_size = sizeof(VirtIOGPUGL),
     .class_init = virtio_gpu_gl_class_init,
 };
+module_obj(TYPE_VIRTIO_GPU_GL);
 
 static void virtio_register_types(void)
 {
@@ -161,3 +162,5 @@ static void virtio_register_types(void)
 }
 
 type_init(virtio_register_types)
+
+module_dep("hw-display-virtio-gpu");
diff --git a/hw/display/virtio-gpu-pci-gl.c b/hw/display/virtio-gpu-pci-gl.c
index 902dda345275..99b14a07185e 100644
--- a/hw/display/virtio-gpu-pci-gl.c
+++ b/hw/display/virtio-gpu-pci-gl.c
@@ -46,6 +46,7 @@ static const VirtioPCIDeviceTypeInfo virtio_gpu_gl_pci_info = {
     .instance_size = sizeof(VirtIOGPUGLPCI),
     .instance_init = virtio_gpu_gl_initfn,
 };
+module_obj(TYPE_VIRTIO_GPU_GL_PCI);
 
 static void virtio_gpu_gl_pci_register_types(void)
 {
@@ -53,3 +54,5 @@ static void virtio_gpu_gl_pci_register_types(void)
 }
 
 type_init(virtio_gpu_gl_pci_register_types)
+
+module_dep("hw-display-virtio-gpu-pci");
diff --git a/hw/display/virtio-gpu-pci.c b/hw/display/virtio-gpu-pci.c
index d742a30aecf7..e36eee0c409b 100644
--- a/hw/display/virtio-gpu-pci.c
+++ b/hw/display/virtio-gpu-pci.c
@@ -64,6 +64,7 @@ static const TypeInfo virtio_gpu_pci_base_info = {
     .class_init = virtio_gpu_pci_base_class_init,
     .abstract = true
 };
+module_obj(TYPE_VIRTIO_GPU_PCI_BASE);
 
 #define TYPE_VIRTIO_GPU_PCI "virtio-gpu-pci"
 typedef struct VirtIOGPUPCI VirtIOGPUPCI;
@@ -90,6 +91,7 @@ static const VirtioPCIDeviceTypeInfo virtio_gpu_pci_info = {
     .instance_size = sizeof(VirtIOGPUPCI),
     .instance_init = virtio_gpu_initfn,
 };
+module_obj(TYPE_VIRTIO_GPU_PCI);
 
 static void virtio_gpu_pci_register_types(void)
 {
diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index e183f4ecdaa5..6b7f643951fe 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -1427,6 +1427,7 @@ static const TypeInfo virtio_gpu_info = {
     .class_size = sizeof(VirtIOGPUClass),
     .class_init = virtio_gpu_class_init,
 };
+module_obj(TYPE_VIRTIO_GPU);
 
 static void virtio_register_types(void)
 {
diff --git a/hw/display/virtio-vga-gl.c b/hw/display/virtio-vga-gl.c
index c971340ebb1a..f22549097c5e 100644
--- a/hw/display/virtio-vga-gl.c
+++ b/hw/display/virtio-vga-gl.c
@@ -36,6 +36,7 @@ static VirtioPCIDeviceTypeInfo virtio_vga_gl_info = {
     .instance_size = sizeof(VirtIOVGAGL),
     .instance_init = virtio_vga_gl_inst_initfn,
 };
+module_obj(TYPE_VIRTIO_VGA_GL);
 
 static void virtio_vga_register_types(void)
 {
@@ -45,3 +46,5 @@ static void virtio_vga_register_types(void)
 }
 
 type_init(virtio_vga_register_types)
+
+module_dep("hw-display-virtio-vga");
diff --git a/hw/display/virtio-vga.c b/hw/display/virtio-vga.c
index d3c640406152..9e57f61e9edb 100644
--- a/hw/display/virtio-vga.c
+++ b/hw/display/virtio-vga.c
@@ -239,6 +239,7 @@ static TypeInfo virtio_vga_base_info = {
     .class_init    = virtio_vga_base_class_init,
     .abstract      = true,
 };
+module_obj(TYPE_VIRTIO_VGA_BASE);
 
 #define TYPE_VIRTIO_VGA "virtio-vga"
 
@@ -268,6 +269,7 @@ static VirtioPCIDeviceTypeInfo virtio_vga_info = {
     .instance_size = sizeof(VirtIOVGA),
     .instance_init = virtio_vga_inst_initfn,
 };
+module_obj(TYPE_VIRTIO_VGA);
 
 static void virtio_vga_register_types(void)
 {
-- 
2.31.1

[PATCH v3 06/24] modules: add chardev module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 chardev/baum.c  | 1 +
 chardev/spice.c | 4 ++++
 2 files changed, 5 insertions(+)

diff --git a/chardev/baum.c b/chardev/baum.c
index 5deca778bc44..79d618e35045 100644
--- a/chardev/baum.c
+++ b/chardev/baum.c
@@ -680,6 +680,7 @@ static const TypeInfo char_braille_type_info = {
     .instance_finalize = char_braille_finalize,
     .class_init = char_braille_class_init,
 };
+module_obj(TYPE_CHARDEV_BRAILLE);
 
 static void register_types(void)
 {
diff --git a/chardev/spice.c b/chardev/spice.c
index 1104426e3a11..3ffb3fdc0dac 100644
--- a/chardev/spice.c
+++ b/chardev/spice.c
@@ -366,6 +366,7 @@ static const TypeInfo char_spice_type_info = {
     .class_init = char_spice_class_init,
     .abstract = true,
 };
+module_obj(TYPE_CHARDEV_SPICE);
 
 static void char_spicevmc_class_init(ObjectClass *oc, void *data)
 {
@@ -396,6 +397,7 @@ static const TypeInfo char_spiceport_type_info = {
     .parent = TYPE_CHARDEV_SPICE,
     .class_init = char_spiceport_class_init,
 };
+module_obj(TYPE_CHARDEV_SPICEPORT);
 
 static void register_types(void)
 {
@@ -405,3 +407,5 @@ static void register_types(void)
 }
 
 type_init(register_types);
+
+module_dep("ui-spice-core");
-- 
2.31.1

[PATCH v3 07/24] modules: add audio module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 audio/spiceaudio.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/audio/spiceaudio.c b/audio/spiceaudio.c
index 999bfbde47c5..a8d370fe6f31 100644
--- a/audio/spiceaudio.c
+++ b/audio/spiceaudio.c
@@ -317,3 +317,5 @@ static void register_audio_spice(void)
     audio_driver_register(&spice_audio_driver);
 }
 type_init(register_audio_spice);
+
+module_dep("ui-spice-core");
-- 
2.31.1

[PATCH v3 08/24] modules: add usb-redir module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/usb/redirect.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/usb/redirect.c b/hw/usb/redirect.c
index 6a75b0dc4ab2..4ec9326e0582 100644
--- a/hw/usb/redirect.c
+++ b/hw/usb/redirect.c
@@ -2608,6 +2608,7 @@ static const TypeInfo usbredir_dev_info = {
     .class_init    = usbredir_class_initfn,
     .instance_init = usbredir_instance_init,
 };
+module_obj(TYPE_USB_REDIR);
 
 static void usbredir_register_types(void)
 {
-- 
2.31.1

[PATCH v3 09/24] modules: add ccid module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/usb/ccid-card-emulated.c | 1 +
 hw/usb/ccid-card-passthru.c | 1 +
 2 files changed, 2 insertions(+)

diff --git a/hw/usb/ccid-card-emulated.c b/hw/usb/ccid-card-emulated.c
index 5c76bed77aa0..6c8c0355e099 100644
--- a/hw/usb/ccid-card-emulated.c
+++ b/hw/usb/ccid-card-emulated.c
@@ -612,6 +612,7 @@ static const TypeInfo emulated_card_info = {
     .instance_size = sizeof(EmulatedState),
     .class_init    = emulated_class_initfn,
 };
+module_obj(TYPE_EMULATED_CCID);
 
 static void ccid_card_emulated_register_types(void)
 {
diff --git a/hw/usb/ccid-card-passthru.c b/hw/usb/ccid-card-passthru.c
index 7212d0d7fb5e..fa3040fb7154 100644
--- a/hw/usb/ccid-card-passthru.c
+++ b/hw/usb/ccid-card-passthru.c
@@ -414,6 +414,7 @@ static const TypeInfo passthru_card_info = {
     .instance_size = sizeof(PassthruState),
     .class_init    = passthru_class_initfn,
 };
+module_obj(TYPE_CCID_PASSTHRU);
 
 static void ccid_card_passthru_register_types(void)
 {
-- 
2.31.1

[PATCH v3 10/24] modules: add ui module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 ui/egl-headless.c | 4 ++++
 ui/gtk.c          | 4 ++++
 ui/sdl2.c         | 4 ++++
 ui/spice-app.c    | 3 +++
 ui/spice-core.c   | 5 +++++
 5 files changed, 20 insertions(+)

diff --git a/ui/egl-headless.c b/ui/egl-headless.c
index da377a74af69..75404e0e8700 100644
--- a/ui/egl-headless.c
+++ b/ui/egl-headless.c
@@ -213,3 +213,7 @@ static void register_egl(void)
 }
 
 type_init(register_egl);
+
+#ifdef CONFIG_OPENGL
+module_dep("ui-opengl");
+#endif
diff --git a/ui/gtk.c b/ui/gtk.c
index 98046f577b9d..376b4d528daa 100644
--- a/ui/gtk.c
+++ b/ui/gtk.c
@@ -2333,3 +2333,7 @@ static void register_gtk(void)
 }
 
 type_init(register_gtk);
+
+#ifdef CONFIG_OPENGL
+module_dep("ui-opengl");
+#endif
diff --git a/ui/sdl2.c b/ui/sdl2.c
index a203cb0239e1..36d9010cb6c1 100644
--- a/ui/sdl2.c
+++ b/ui/sdl2.c
@@ -918,3 +918,7 @@ static void register_sdl1(void)
 }
 
 type_init(register_sdl1);
+
+#ifdef CONFIG_OPENGL
+module_dep("ui-opengl");
+#endif
diff --git a/ui/spice-app.c b/ui/spice-app.c
index 4325ac2d9c54..641f4a9d53e3 100644
--- a/ui/spice-app.c
+++ b/ui/spice-app.c
@@ -221,3 +221,6 @@ static void register_spice_app(void)
 }
 
 type_init(register_spice_app);
+
+module_dep("ui-spice-core");
+module_dep("chardev-spice");
diff --git a/ui/spice-core.c b/ui/spice-core.c
index 272d19b0c152..86d43783acac 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -1037,3 +1037,8 @@ static void spice_register_config(void)
     qemu_add_opts(&qemu_spice_opts);
 }
 opts_init(spice_register_config);
+module_opts("spice");
+
+#ifdef CONFIG_OPENGL
+module_dep("ui-opengl");
+#endif
-- 
2.31.1

[PATCH v3 11/24] modules: add s390x module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/s390x/virtio-ccw-gpu.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/hw/s390x/virtio-ccw-gpu.c b/hw/s390x/virtio-ccw-gpu.c
index 75a9e4bb3908..5868a2a07093 100644
--- a/hw/s390x/virtio-ccw-gpu.c
+++ b/hw/s390x/virtio-ccw-gpu.c
@@ -59,6 +59,7 @@ static const TypeInfo virtio_ccw_gpu = {
     .instance_init = virtio_ccw_gpu_instance_init,
     .class_init    = virtio_ccw_gpu_class_init,
 };
+module_obj(TYPE_VIRTIO_GPU_CCW);
 
 static void virtio_ccw_gpu_register(void)
 {
@@ -68,3 +69,5 @@ static void virtio_ccw_gpu_register(void)
 }
 
 type_init(virtio_ccw_gpu_register)
+
+module_arch("s390x");
-- 
2.31.1

[PATCH v3 12/24] modules: add block module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 block/iscsi-opts.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/block/iscsi-opts.c b/block/iscsi-opts.c
index afaf8837d6c1..4f2da405e645 100644
--- a/block/iscsi-opts.c
+++ b/block/iscsi-opts.c
@@ -68,3 +68,4 @@ static void iscsi_block_opts_init(void)
 }
 
 block_init(iscsi_block_opts_init);
+module_opts("iscsi");
-- 
2.31.1

[PATCH v3 13/24] modules: use modinfo for dependencies

[Gerd Hoffmann]

Use module database for module dependencies.
Drop hard-coded dependency list.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 util/module.c | 55 ++++++++++++++++++++-------------------------------
 1 file changed, 21 insertions(+), 34 deletions(-)

diff --git a/util/module.c b/util/module.c
index 8d3e8275b9f7..7d7b69cbdaca 100644
--- a/util/module.c
+++ b/util/module.c
@@ -182,28 +182,6 @@ static int module_load_file(const char *fname, bool mayfail, bool export_symbols
 out:
     return ret;
 }
-
-static const struct {
-    const char *name;
-    const char *dep;
-} module_deps[] = {
-    { "audio-spice",    "ui-spice-core" },
-    { "chardev-spice",  "ui-spice-core" },
-    { "hw-display-qxl", "ui-spice-core" },
-    { "ui-spice-app",   "ui-spice-core" },
-    { "ui-spice-app",   "chardev-spice" },
-
-    { "hw-display-virtio-gpu-gl", "hw-display-virtio-gpu" },
-    { "hw-display-virtio-gpu-pci-gl", "hw-display-virtio-gpu-pci" },
-    { "hw-display-virtio-vga-gl", "hw-display-virtio-vga" },
-
-#ifdef CONFIG_OPENGL
-    { "ui-egl-headless", "ui-opengl"    },
-    { "ui-gtk",          "ui-opengl"    },
-    { "ui-sdl",          "ui-opengl"    },
-    { "ui-spice-core",   "ui-opengl"    },
-#endif
-};
 #endif
 
 bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
@@ -219,9 +197,11 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
     char *dirs[5];
     char *module_name;
     int i = 0, n_dirs = 0;
-    int ret, dep;
+    int ret;
     bool export_symbols = false;
     static GHashTable *loaded_modules;
+    const QemuModinfo *modinfo;
+    const char **sl;
 
     if (!g_module_supported()) {
         fprintf(stderr, "Module is not supported by system.\n");
@@ -234,23 +214,30 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
 
     module_name = g_strdup_printf("%s%s", prefix, lib_name);
 
-    for (dep = 0; dep < ARRAY_SIZE(module_deps); dep++) {
-        if (strcmp(module_name, module_deps[dep].name) == 0) {
-            /* we depend on another module */
-            module_load_one("", module_deps[dep].dep, false);
-        }
-        if (strcmp(module_name, module_deps[dep].dep) == 0) {
-            /* another module depends on us */
-            export_symbols = true;
-        }
-    }
-
     if (g_hash_table_contains(loaded_modules, module_name)) {
         g_free(module_name);
         return true;
     }
     g_hash_table_add(loaded_modules, module_name);
 
+    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (modinfo->deps) {
+            if (strcmp(modinfo->name, module_name) == 0) {
+                /* we depend on other module(s) */
+                for (sl = modinfo->deps; *sl != NULL; sl++) {
+                    module_load_one("", *sl, false);
+                }
+            } else {
+                for (sl = modinfo->deps; *sl != NULL; sl++) {
+                    if (strcmp(module_name, *sl) == 0) {
+                        /* another module depends on us */
+                        export_symbols = true;
+                    }
+                }
+            }
+        }
+    }
+
     search_dir = getenv("QEMU_MODULE_DIR");
     if (search_dir != NULL) {
         dirs[n_dirs++] = g_strdup_printf("%s", search_dir);
-- 
2.31.1

[PATCH v3 14/24] modules: use modinfo for qom load

[Gerd Hoffmann]

Use module database to figure which module implements a given QOM type.
Drop hard-coded object list.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 util/module.c | 77 ++++++++++++++++-----------------------------------
 1 file changed, 24 insertions(+), 53 deletions(-)

diff --git a/util/module.c b/util/module.c
index 7d7b69cbdaca..745ae0fb20ed 100644
--- a/util/module.c
+++ b/util/module.c
@@ -280,80 +280,51 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
     return success;
 }
 
-/*
- * Building devices and other qom objects modular is mostly useful in
- * case they have dependencies to external shared libraries, so we can
- * cut down the core qemu library dependencies.  Which is the case for
- * only a very few devices & objects.
- *
- * So with the expectation that this will be rather the exception than
- * the rule and the list will not gain that many entries, go with a
- * simple manually maintained list for now.
- *
- * The list must be sorted by module (module_load_qom_all() needs this).
- */
-static struct {
-    const char *type;
-    const char *prefix;
-    const char *module;
-} const qom_modules[] = {
-    { "ccid-card-passthru",    "hw-", "usb-smartcard"         },
-    { "ccid-card-emulated",    "hw-", "usb-smartcard"         },
-    { "usb-redir",             "hw-", "usb-redirect"          },
-    { "qxl-vga",               "hw-", "display-qxl"           },
-    { "qxl",                   "hw-", "display-qxl"           },
-    { "virtio-gpu-device",     "hw-", "display-virtio-gpu"    },
-    { "virtio-gpu-gl-device",  "hw-", "display-virtio-gpu-gl" },
-    { "vhost-user-gpu",        "hw-", "display-virtio-gpu"    },
-    { "virtio-gpu-pci-base",   "hw-", "display-virtio-gpu-pci" },
-    { "virtio-gpu-pci",        "hw-", "display-virtio-gpu-pci" },
-    { "virtio-gpu-gl-pci",     "hw-", "display-virtio-gpu-pci-gl" },
-    { "vhost-user-gpu-pci",    "hw-", "display-virtio-gpu-pci" },
-    { "virtio-gpu-ccw",        "hw-", "s390x-virtio-gpu-ccw"   },
-    { "virtio-vga-base",       "hw-", "display-virtio-vga"    },
-    { "virtio-vga",            "hw-", "display-virtio-vga"    },
-    { "virtio-vga-gl",         "hw-", "display-virtio-vga-gl" },
-    { "vhost-user-vga",        "hw-", "display-virtio-vga"    },
-    { "chardev-braille",       "chardev-", "baum"             },
-    { "chardev-spicevmc",      "chardev-", "spice"            },
-    { "chardev-spiceport",     "chardev-", "spice"            },
-};
+#ifdef CONFIG_MODULES
 
 static bool module_loaded_qom_all;
 
 void module_load_qom_one(const char *type)
 {
-    int i;
+    const QemuModinfo *modinfo;
+    const char **sl;
 
     if (!type) {
         return;
     }
-    for (i = 0; i < ARRAY_SIZE(qom_modules); i++) {
-        if (strcmp(qom_modules[i].type, type) == 0) {
-            module_load_one(qom_modules[i].prefix,
-                            qom_modules[i].module,
-                            false);
-            return;
+
+    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (!modinfo->objs) {
+            continue;
+        }
+        for (sl = modinfo->objs; *sl != NULL; sl++) {
+            if (strcmp(type, *sl) == 0) {
+                module_load_one("", modinfo->name, false);
+            }
         }
     }
 }
 
 void module_load_qom_all(void)
 {
-    int i;
+    const QemuModinfo *modinfo;
 
     if (module_loaded_qom_all) {
         return;
     }
-    for (i = 0; i < ARRAY_SIZE(qom_modules); i++) {
-        if (i > 0 && (strcmp(qom_modules[i - 1].module,
-                             qom_modules[i].module) == 0 &&
-                      strcmp(qom_modules[i - 1].prefix,
-                             qom_modules[i].prefix) == 0)) {
-            /* one module implementing multiple types -> load only once */
+
+    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (!modinfo->objs) {
             continue;
         }
-        module_load_one(qom_modules[i].prefix, qom_modules[i].module, true);
+        module_load_one("", modinfo->name, false);
     }
     module_loaded_qom_all = true;
 }
+
+#else
+
+void module_load_qom_one(const char *type) {}
+void module_load_qom_all(void) {}
+
+#endif
-- 
2.31.1

[PATCH v3 15/24] modules: use modinfo for qemu opts load

[Gerd Hoffmann]

Use module database to figure which module adds given QemuOpts group.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 softmmu/vl.c        | 17 -----------------
 stubs/module-opts.c |  4 ----
 util/module.c       | 19 +++++++++++++++++++
 3 files changed, 19 insertions(+), 21 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index a4857ec43ff3..c91d63e3cc02 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2675,23 +2675,6 @@ void qmp_x_exit_preconfig(Error **errp)
     }
 }
 
-#ifdef CONFIG_MODULES
-void qemu_load_module_for_opts(const char *group)
-{
-    static bool spice_tried;
-    if (g_str_equal(group, "spice") && !spice_tried) {
-        ui_module_load_one("spice-core");
-        spice_tried = true;
-    }
-
-    static bool iscsi_tried;
-    if (g_str_equal(group, "iscsi") && !iscsi_tried) {
-        block_module_load_one("iscsi");
-        iscsi_tried = true;
-    }
-}
-#endif
-
 void qemu_init(int argc, char **argv, char **envp)
 {
     QemuOpts *opts;
diff --git a/stubs/module-opts.c b/stubs/module-opts.c
index a7d0e4ad6ead..5412429ea869 100644
--- a/stubs/module-opts.c
+++ b/stubs/module-opts.c
@@ -1,6 +1,2 @@
 #include "qemu/osdep.h"
 #include "qemu/config-file.h"
-
-void qemu_load_module_for_opts(const char *group)
-{
-}
diff --git a/util/module.c b/util/module.c
index 745ae0fb20ed..a9ec2da9972e 100644
--- a/util/module.c
+++ b/util/module.c
@@ -20,6 +20,7 @@
 #include "qemu/queue.h"
 #include "qemu/module.h"
 #include "qemu/cutils.h"
+#include "qemu/config-file.h"
 #ifdef CONFIG_MODULE_UPGRADES
 #include "qemu-version.h"
 #endif
@@ -322,8 +323,26 @@ void module_load_qom_all(void)
     module_loaded_qom_all = true;
 }
 
+void qemu_load_module_for_opts(const char *group)
+{
+    const QemuModinfo *modinfo;
+    const char **sl;
+
+    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (!modinfo->opts) {
+            continue;
+        }
+        for (sl = modinfo->opts; *sl != NULL; sl++) {
+            if (strcmp(group, *sl) == 0) {
+                module_load_one("", modinfo->name, false);
+            }
+        }
+    }
+}
+
 #else
 
+void qemu_load_module_for_opts(const char *group) {}
 void module_load_qom_one(const char *type) {}
 void module_load_qom_all(void) {}
 
-- 
2.31.1

[PATCH v3 16/24] modules: add tracepoints

[Gerd Hoffmann]

One for module load and one for qom type lookup.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 util/module.c     | 3 +++
 util/trace-events | 4 ++++
 2 files changed, 7 insertions(+)

diff --git a/util/module.c b/util/module.c
index a9ec2da9972e..acaaecad56c9 100644
--- a/util/module.c
+++ b/util/module.c
@@ -24,6 +24,7 @@
 #ifdef CONFIG_MODULE_UPGRADES
 #include "qemu-version.h"
 #endif
+#include "trace.h"
 
 typedef struct ModuleEntry
 {
@@ -176,6 +177,7 @@ static int module_load_file(const char *fname, bool mayfail, bool export_symbols
         ret = 0;
     }
 
+    trace_module_load_module(fname);
     QTAILQ_FOREACH_SAFE(e, &dso_init_list, node, next) {
         QTAILQ_REMOVE(&dso_init_list, e, node);
         g_free(e);
@@ -294,6 +296,7 @@ void module_load_qom_one(const char *type)
         return;
     }
 
+    trace_module_lookup_object_type(type);
     for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
         if (!modinfo->objs) {
             continue;
diff --git a/util/trace-events b/util/trace-events
index 806cac14a762..c8f53d7d9fc3 100644
--- a/util/trace-events
+++ b/util/trace-events
@@ -100,3 +100,7 @@ uffd_create_fd_api_failed(int err) "errno: %i"
 uffd_create_fd_api_noioctl(uint64_t ioctl_req, uint64_t ioctl_supp) "ioctl_req: 0x%" PRIx64 "ioctl_supp: 0x%" PRIx64
 uffd_register_memory_failed(void *addr, uint64_t length, uint64_t mode, int err) "addr: %p length: %" PRIu64 " mode: 0x%" PRIx64 " errno: %i"
 uffd_unregister_memory_failed(void *addr, uint64_t length, int err) "addr: %p length: %" PRIu64 " errno: %i"
+
+# module.c
+module_load_module(const char *name) "file %s"
+module_lookup_object_type(const char *name) "name %s"
-- 
2.31.1

[PATCH v3 17/24] modules: check arch and block load on mismatch

[Gerd Hoffmann]

Add module_allow_arch() to set the target architecture.
In case a module is limited to some arch verify arches
match and ignore the module if not.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 include/qemu/module.h |  1 +
 softmmu/vl.c          |  3 +++
 util/module.c         | 29 +++++++++++++++++++++++++++++
 3 files changed, 33 insertions(+)

diff --git a/include/qemu/module.h b/include/qemu/module.h
index a98748d501d3..7f4b1af8198c 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -72,6 +72,7 @@ void module_call_init(module_init_type type);
 bool module_load_one(const char *prefix, const char *lib_name, bool mayfail);
 void module_load_qom_one(const char *type);
 void module_load_qom_all(void);
+void module_allow_arch(const char *arch);
 
 /*
  * module info annotation macros
diff --git a/softmmu/vl.c b/softmmu/vl.c
index c91d63e3cc02..866ea310a411 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -126,6 +126,8 @@
 #include "sysemu/iothread.h"
 #include "qemu/guest-random.h"
 
+#include "config-host.h"
+
 #define MAX_VIRTIO_CONSOLES 1
 
 typedef struct BlockdevOptionsQueueEntry {
@@ -2725,6 +2727,7 @@ void qemu_init(int argc, char **argv, char **envp)
 
 #ifdef CONFIG_MODULES
     module_init_info(qemu_modinfo);
+    module_allow_arch(TARGET_NAME);
 #endif
 
     qemu_init_subsystems();
diff --git a/util/module.c b/util/module.c
index acaaecad56c9..065aed09ffef 100644
--- a/util/module.c
+++ b/util/module.c
@@ -117,12 +117,33 @@ static const QemuModinfo module_info_stub[] = { {
     /* end of list */
 } };
 static const QemuModinfo *module_info = module_info_stub;
+static const char *module_arch;
 
 void module_init_info(const QemuModinfo *info)
 {
     module_info = info;
 }
 
+void module_allow_arch(const char *arch)
+{
+    module_arch = arch;
+}
+
+static bool module_check_arch(const QemuModinfo *modinfo)
+{
+    if (modinfo->arch) {
+        if (!module_arch) {
+            /* no arch set -> ignore all */
+            return false;
+        }
+        if (strcmp(module_arch, modinfo->arch) != 0) {
+            /* mismatch */
+            return false;
+        }
+    }
+    return true;
+}
+
 static int module_load_file(const char *fname, bool mayfail, bool export_symbols)
 {
     GModule *g_module;
@@ -224,6 +245,13 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
     g_hash_table_add(loaded_modules, module_name);
 
     for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (modinfo->arch) {
+            if (strcmp(modinfo->name, module_name) == 0) {
+                if (!module_check_arch(modinfo)) {
+                    return false;
+                }
+            }
+        }
         if (modinfo->deps) {
             if (strcmp(modinfo->name, module_name) == 0) {
                 /* we depend on other module(s) */
@@ -345,6 +373,7 @@ void qemu_load_module_for_opts(const char *group)
 
 #else
 
+void module_allow_arch(const char *arch) {}
 void qemu_load_module_for_opts(const char *group) {}
 void module_load_qom_one(const char *type) {}
 void module_load_qom_all(void) {}
-- 
2.31.1

[PATCH v3 18/24] modules: check arch on qom lookup

[Gerd Hoffmann]

With target-specific modules we can have multiple modules implementing
the same object.  Therefore we have to check the target arch on lookup
to find the correct module.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 util/module.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/util/module.c b/util/module.c
index 065aed09ffef..6bb4ad915a1c 100644
--- a/util/module.c
+++ b/util/module.c
@@ -329,6 +329,9 @@ void module_load_qom_one(const char *type)
         if (!modinfo->objs) {
             continue;
         }
+        if (!module_check_arch(modinfo)) {
+            continue;
+        }
         for (sl = modinfo->objs; *sl != NULL; sl++) {
             if (strcmp(type, *sl) == 0) {
                 module_load_one("", modinfo->name, false);
@@ -349,6 +352,9 @@ void module_load_qom_all(void)
         if (!modinfo->objs) {
             continue;
         }
+        if (!module_check_arch(modinfo)) {
+            continue;
+        }
         module_load_one("", modinfo->name, false);
     }
     module_loaded_qom_all = true;
-- 
2.31.1

[PATCH v3 19/24] modules: target-specific module build infrastructure

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 meson.build | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/meson.build b/meson.build
index 9cf50a50d39a..e822477a231a 100644
--- a/meson.build
+++ b/meson.build
@@ -1781,6 +1781,7 @@ user_ss = ss.source_set()
 util_ss = ss.source_set()
 
 modules = {}
+target_modules = {}
 hw_arch = {}
 target_arch = {}
 target_softmmu_arch = {}
@@ -2055,6 +2056,41 @@ foreach d, list : modules
   endforeach
 endforeach
 
+foreach d, list : target_modules
+  foreach m, module_ss : list
+    if enable_modules and targetos != 'windows'
+      foreach target : target_dirs
+        if target.endswith('-softmmu')
+          config_target = config_target_mak[target]
+          config_target += config_host
+          target_inc = [include_directories('target' / config_target['TARGET_BASE_ARCH'])]
+          c_args = ['-DNEED_CPU_H',
+                    '-DCONFIG_TARGET="@0@-config-target.h"'.format(target),
+                    '-DCONFIG_DEVICES="@0@-config-devices.h"'.format(target)]
+          target_module_ss = module_ss.apply(config_target, strict: false)
+          if target_module_ss.sources() != []
+            module_name = d + '-' + m + '-' + config_target['TARGET_NAME']
+            sl = static_library(module_name,
+                                [genh, target_module_ss.sources()],
+                                dependencies: [modulecommon, target_module_ss.dependencies()],
+                                include_directories: target_inc,
+                                c_args: c_args,
+                                pic: true)
+            softmmu_mods += sl
+            modinfo_files += custom_target(module_name + '.modinfo',
+                                           output: module_name + '.modinfo',
+                                           input: target_module_ss.sources(),
+                                           capture: true,
+                                           command: [modinfo_collect, '--target', target, '@INPUT@'])
+          endif
+        endif
+      endforeach
+    else
+      specific_ss.add_all(module_ss)
+    endif
+  endforeach
+endforeach
+
 if enable_modules
   modinfo_src = custom_target('modinfo.c',
                               output: 'modinfo.c',
-- 
2.31.1

[PATCH v3 20/24] accel: autoload modules

[Gerd Hoffmann]

Call module_object_class_by_name() instead of object_class_by_name()
for objects possibly implemented as module

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 accel/accel-common.c  | 2 +-
 accel/accel-softmmu.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/accel/accel-common.c b/accel/accel-common.c
index cf07f78421d6..7b8ec7e0f72a 100644
--- a/accel/accel-common.c
+++ b/accel/accel-common.c
@@ -44,7 +44,7 @@ static const TypeInfo accel_type = {
 AccelClass *accel_find(const char *opt_name)
 {
     char *class_name = g_strdup_printf(ACCEL_CLASS_NAME("%s"), opt_name);
-    AccelClass *ac = ACCEL_CLASS(object_class_by_name(class_name));
+    AccelClass *ac = ACCEL_CLASS(module_object_class_by_name(class_name));
     g_free(class_name);
     return ac;
 }
diff --git a/accel/accel-softmmu.c b/accel/accel-softmmu.c
index 50fa5acaa401..67276e4f5222 100644
--- a/accel/accel-softmmu.c
+++ b/accel/accel-softmmu.c
@@ -72,7 +72,7 @@ void accel_init_ops_interfaces(AccelClass *ac)
     g_assert(ac_name != NULL);
 
     ops_name = g_strdup_printf("%s" ACCEL_OPS_SUFFIX, ac_name);
-    ops = ACCEL_OPS_CLASS(object_class_by_name(ops_name));
+    ops = ACCEL_OPS_CLASS(module_object_class_by_name(ops_name));
     g_free(ops_name);
 
     /*
-- 
2.31.1

[PATCH v3 21/24] accel: add qtest module annotations

[Gerd Hoffmann]

Add module annotations for qtest so autoloading works.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 accel/qtest/qtest.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/accel/qtest/qtest.c b/accel/qtest/qtest.c
index edb29f6fa4c0..7e6b8110d52b 100644
--- a/accel/qtest/qtest.c
+++ b/accel/qtest/qtest.c
@@ -45,6 +45,7 @@ static const TypeInfo qtest_accel_type = {
     .parent = TYPE_ACCEL,
     .class_init = qtest_accel_class_init,
 };
+module_obj(TYPE_QTEST_ACCEL);
 
 static void qtest_accel_ops_class_init(ObjectClass *oc, void *data)
 {
@@ -61,6 +62,7 @@ static const TypeInfo qtest_accel_ops_type = {
     .class_init = qtest_accel_ops_class_init,
     .abstract = true,
 };
+module_obj(ACCEL_OPS_NAME("qtest"));
 
 static void qtest_type_init(void)
 {
-- 
2.31.1

[PATCH v3 22/24] accel: build qtest modular

[Gerd Hoffmann]

Allow building accelerators as module.
Start with qtest as first user.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 accel/qtest/meson.build | 8 ++------
 meson.build             | 6 ++++++
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/accel/qtest/meson.build b/accel/qtest/meson.build
index a2f327645980..4c6560029336 100644
--- a/accel/qtest/meson.build
+++ b/accel/qtest/meson.build
@@ -1,6 +1,2 @@
-qtest_ss = ss.source_set()
-qtest_ss.add(files(
-  'qtest.c',
-))
-
-specific_ss.add_all(when: ['CONFIG_SOFTMMU', 'CONFIG_POSIX'], if_true: qtest_ss)
+qtest_module_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_POSIX'],
+                    if_true: files('qtest.c'))
diff --git a/meson.build b/meson.build
index e822477a231a..8f55f563eefe 100644
--- a/meson.build
+++ b/meson.build
@@ -1780,6 +1780,9 @@ trace_ss = ss.source_set()
 user_ss = ss.source_set()
 util_ss = ss.source_set()
 
+# accel modules
+qtest_module_ss = ss.source_set()
+
 modules = {}
 target_modules = {}
 hw_arch = {}
@@ -2018,6 +2021,9 @@ specific_ss.add_all(when: 'CONFIG_LINUX_USER', if_true: linux_user_ss)
 subdir('tests/qtest/libqos')
 subdir('tests/qtest/fuzz')
 
+# accel modules
+target_modules += { 'accel' : { 'qtest': qtest_module_ss }}
+
 ########################
 # Library dependencies #
 ########################
-- 
2.31.1

[PATCH v3 23/24] accel: add tcg module annotations

[Gerd Hoffmann]

Add module annotations for tcg so autoloading works.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 accel/tcg/tcg-accel-ops.c | 1 +
 accel/tcg/tcg-all.c       | 1 +
 2 files changed, 2 insertions(+)

diff --git a/accel/tcg/tcg-accel-ops.c b/accel/tcg/tcg-accel-ops.c
index 7191315aeed4..1a8e8390bd60 100644
--- a/accel/tcg/tcg-accel-ops.c
+++ b/accel/tcg/tcg-accel-ops.c
@@ -124,6 +124,7 @@ static const TypeInfo tcg_accel_ops_type = {
     .class_init = tcg_accel_ops_class_init,
     .abstract = true,
 };
+module_obj(ACCEL_OPS_NAME("tcg"));
 
 static void tcg_accel_ops_register_types(void)
 {
diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
index 00803f76d870..d6336a9c966d 100644
--- a/accel/tcg/tcg-all.c
+++ b/accel/tcg/tcg-all.c
@@ -238,6 +238,7 @@ static const TypeInfo tcg_accel_type = {
     .class_init = tcg_accel_class_init,
     .instance_size = sizeof(TCGState),
 };
+module_obj(TYPE_TCG_ACCEL);
 
 static void register_accel_types(void)
 {
-- 
2.31.1

[PATCH v3 24/24] accel: build tcg modular

[Gerd Hoffmann]

Build tcg accel ops as module.
Which is only a small fraction of tcg.
Also only x86 for now.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 accel/tcg/meson.build |  5 ++++-
 meson.build           | 14 +++++++++++++-
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/accel/tcg/meson.build b/accel/tcg/meson.build
index 1236ac7b910b..0ae9180282e3 100644
--- a/accel/tcg/meson.build
+++ b/accel/tcg/meson.build
@@ -15,8 +15,11 @@ specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
 
 specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
   'cputlb.c',
+))
+
+tcg_module_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
   'tcg-accel-ops.c',
   'tcg-accel-ops-mttcg.c',
   'tcg-accel-ops-icount.c',
-  'tcg-accel-ops-rr.c'
+  'tcg-accel-ops-rr.c',
 ))
diff --git a/meson.build b/meson.build
index 8f55f563eefe..03bacca7cddb 100644
--- a/meson.build
+++ b/meson.build
@@ -92,6 +92,8 @@ if cpu in ['x86', 'x86_64']
   }
 endif
 
+modular_tcg = ['i386-softmmu', 'x86_64-softmmu']
+
 edk2_targets = [ 'arm-softmmu', 'aarch64-softmmu', 'i386-softmmu', 'x86_64-softmmu' ]
 install_edk2_blobs = false
 if get_option('install_blobs')
@@ -1311,6 +1313,11 @@ foreach target : target_dirs
       elif sym == 'CONFIG_XEN' and have_xen_pci_passthrough
         config_target += { 'CONFIG_XEN_PCI_PASSTHROUGH': 'y' }
       endif
+      if target in modular_tcg
+        config_target += { 'CONFIG_TCG_MODULAR': 'y' }
+      else
+        config_target += { 'CONFIG_TCG_BUILTIN': 'y' }
+      endif
       accel_kconfig += [ sym + '=y' ]
     endif
   endforeach
@@ -1782,6 +1789,7 @@ util_ss = ss.source_set()
 
 # accel modules
 qtest_module_ss = ss.source_set()
+tcg_module_ss = ss.source_set()
 
 modules = {}
 target_modules = {}
@@ -2022,7 +2030,11 @@ subdir('tests/qtest/libqos')
 subdir('tests/qtest/fuzz')
 
 # accel modules
-target_modules += { 'accel' : { 'qtest': qtest_module_ss }}
+tcg_real_module_ss = ss.source_set()
+tcg_real_module_ss.add_all(when: 'CONFIG_TCG_MODULAR', if_true: tcg_module_ss)
+specific_ss.add_all(when: 'CONFIG_TCG_BUILTIN', if_true: tcg_module_ss)
+target_modules += { 'accel' : { 'qtest': qtest_module_ss,
+                                'tcg': tcg_real_module_ss }}
 
 ########################
 # Library dependencies #
-- 
2.31.1

Re: [PATCH v3 02/24] modules: collect module meta-data

[Paolo Bonzini]

On 18/06/21 06:53, Gerd Hoffmann wrote:
> +def find_command(src, target, compile_commands):
> +    for command in compile_commands:
> +        if command['file'] != src:
> +            continue
> +        if target != '' and command['command'].find(target) == -1:
> +            continue


Did you look into using extract_objects for this instead of looking for 
the target (which works, but yuck :))?

Paolo

Re: [PATCH v3 01/24] modules: add modinfo macros

[Paolo Bonzini]

On 18/06/21 06:53, Gerd Hoffmann wrote:
> Add macros for module info annotations.
> 
> Instead of having that module meta-data stored in lists in util/module.c
> place directly in the module source code.
> 
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>   include/qemu/module.h | 25 +++++++++++++++++++++++++
>   1 file changed, 25 insertions(+)
> 
> diff --git a/include/qemu/module.h b/include/qemu/module.h
> index 944d403cbd15..81ef086da023 100644
> --- a/include/qemu/module.h
> +++ b/include/qemu/module.h
> @@ -73,4 +73,29 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail);
>   void module_load_qom_one(const char *type);
>   void module_load_qom_all(void);
>   
> +/*
> + * module info annotation macros
> + *
> + * scripts/modinfo-collect.py will collect module info,
> + * using the preprocessor and -DQEMU_MODINFO
> + */
> +#ifdef QEMU_MODINFO
> +# define modinfo(kind, value) \
> +    MODINFO_START kind value MODINFO_END
> +#else
> +# define modinfo(kind, value)
> +#endif
> +
> +/* module implements QOM type <name> */
> +#define module_obj(name) modinfo(obj, name)
> +
> +/* module has a dependency on <name> */
> +#define module_dep(name) modinfo(dep, name)
> +
> +/* module is for target architecture <name> */
> +#define module_arch(name) modinfo(arch, name)
> +
> +/* module registers QemuOpts <name> */
> +#define module_opts(name) modinfo(opts, name)
> +
>   #endif
> 

Let's ensure that the module system is documented in some file of 
docs/devel.

Paolo

Re: [PATCH v3 13/24] modules: use modinfo for dependencies

[Paolo Bonzini]

On 18/06/21 06:53, Gerd Hoffmann wrote:
> Use module database for module dependencies.
> Drop hard-coded dependency list.
> 
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>   util/module.c | 55 ++++++++++++++++++++-------------------------------
>   1 file changed, 21 insertions(+), 34 deletions(-)
> 
> diff --git a/util/module.c b/util/module.c
> index 8d3e8275b9f7..7d7b69cbdaca 100644
> --- a/util/module.c
> +++ b/util/module.c
> @@ -182,28 +182,6 @@ static int module_load_file(const char *fname, bool mayfail, bool export_symbols
>   out:
>       return ret;
>   }
> -
> -static const struct {
> -    const char *name;
> -    const char *dep;
> -} module_deps[] = {
> -    { "audio-spice",    "ui-spice-core" },
> -    { "chardev-spice",  "ui-spice-core" },
> -    { "hw-display-qxl", "ui-spice-core" },
> -    { "ui-spice-app",   "ui-spice-core" },
> -    { "ui-spice-app",   "chardev-spice" },
> -
> -    { "hw-display-virtio-gpu-gl", "hw-display-virtio-gpu" },
> -    { "hw-display-virtio-gpu-pci-gl", "hw-display-virtio-gpu-pci" },
> -    { "hw-display-virtio-vga-gl", "hw-display-virtio-vga" },
> -
> -#ifdef CONFIG_OPENGL
> -    { "ui-egl-headless", "ui-opengl"    },
> -    { "ui-gtk",          "ui-opengl"    },
> -    { "ui-sdl",          "ui-opengl"    },
> -    { "ui-spice-core",   "ui-opengl"    },
> -#endif
> -};
>   #endif
>   
>   bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
> @@ -219,9 +197,11 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
>       char *dirs[5];
>       char *module_name;
>       int i = 0, n_dirs = 0;
> -    int ret, dep;
> +    int ret;
>       bool export_symbols = false;
>       static GHashTable *loaded_modules;
> +    const QemuModinfo *modinfo;
> +    const char **sl;
>   
>       if (!g_module_supported()) {
>           fprintf(stderr, "Module is not supported by system.\n");
> @@ -234,23 +214,30 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
>   
>       module_name = g_strdup_printf("%s%s", prefix, lib_name);
>   
> -    for (dep = 0; dep < ARRAY_SIZE(module_deps); dep++) {
> -        if (strcmp(module_name, module_deps[dep].name) == 0) {
> -            /* we depend on another module */
> -            module_load_one("", module_deps[dep].dep, false);
> -        }
> -        if (strcmp(module_name, module_deps[dep].dep) == 0) {
> -            /* another module depends on us */
> -            export_symbols = true;
> -        }
> -    }
> -
>       if (g_hash_table_contains(loaded_modules, module_name)) {
>           g_free(module_name);
>           return true;
>       }
>       g_hash_table_add(loaded_modules, module_name);
>   
> +    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
> +        if (modinfo->deps) {
> +            if (strcmp(modinfo->name, module_name) == 0) {
> +                /* we depend on other module(s) */
> +                for (sl = modinfo->deps; *sl != NULL; sl++) {
> +                    module_load_one("", *sl, false);
> +                }
> +            } else {
> +                for (sl = modinfo->deps; *sl != NULL; sl++) {
> +                    if (strcmp(module_name, *sl) == 0) {
> +                        /* another module depends on us */
> +                        export_symbols = true;
> +                    }
> +                }
> +            }
> +        }
> +    }
> +
>       search_dir = getenv("QEMU_MODULE_DIR");
>       if (search_dir != NULL) {
>           dirs[n_dirs++] = g_strdup_printf("%s", search_dir);
> 

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

Re: [PATCH v3 14/24] modules: use modinfo for qom load

[Paolo Bonzini]

On 18/06/21 06:53, Gerd Hoffmann wrote:
> Use module database to figure which module implements a given QOM type.
> Drop hard-coded object list.
> 
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>   util/module.c | 77 ++++++++++++++++-----------------------------------
>   1 file changed, 24 insertions(+), 53 deletions(-)
> 
> diff --git a/util/module.c b/util/module.c
> index 7d7b69cbdaca..745ae0fb20ed 100644
> --- a/util/module.c
> +++ b/util/module.c
> @@ -280,80 +280,51 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
>       return success;
>   }
>   
> -/*
> - * Building devices and other qom objects modular is mostly useful in
> - * case they have dependencies to external shared libraries, so we can
> - * cut down the core qemu library dependencies.  Which is the case for
> - * only a very few devices & objects.
> - *
> - * So with the expectation that this will be rather the exception than
> - * the rule and the list will not gain that many entries, go with a
> - * simple manually maintained list for now.
> - *
> - * The list must be sorted by module (module_load_qom_all() needs this).
> - */
> -static struct {
> -    const char *type;
> -    const char *prefix;
> -    const char *module;
> -} const qom_modules[] = {
> -    { "ccid-card-passthru",    "hw-", "usb-smartcard"         },
> -    { "ccid-card-emulated",    "hw-", "usb-smartcard"         },
> -    { "usb-redir",             "hw-", "usb-redirect"          },
> -    { "qxl-vga",               "hw-", "display-qxl"           },
> -    { "qxl",                   "hw-", "display-qxl"           },
> -    { "virtio-gpu-device",     "hw-", "display-virtio-gpu"    },
> -    { "virtio-gpu-gl-device",  "hw-", "display-virtio-gpu-gl" },
> -    { "vhost-user-gpu",        "hw-", "display-virtio-gpu"    },
> -    { "virtio-gpu-pci-base",   "hw-", "display-virtio-gpu-pci" },
> -    { "virtio-gpu-pci",        "hw-", "display-virtio-gpu-pci" },
> -    { "virtio-gpu-gl-pci",     "hw-", "display-virtio-gpu-pci-gl" },
> -    { "vhost-user-gpu-pci",    "hw-", "display-virtio-gpu-pci" },
> -    { "virtio-gpu-ccw",        "hw-", "s390x-virtio-gpu-ccw"   },
> -    { "virtio-vga-base",       "hw-", "display-virtio-vga"    },
> -    { "virtio-vga",            "hw-", "display-virtio-vga"    },
> -    { "virtio-vga-gl",         "hw-", "display-virtio-vga-gl" },
> -    { "vhost-user-vga",        "hw-", "display-virtio-vga"    },
> -    { "chardev-braille",       "chardev-", "baum"             },
> -    { "chardev-spicevmc",      "chardev-", "spice"            },
> -    { "chardev-spiceport",     "chardev-", "spice"            },
> -};
> +#ifdef CONFIG_MODULES
>   
>   static bool module_loaded_qom_all;
>   
>   void module_load_qom_one(const char *type)
>   {
> -    int i;
> +    const QemuModinfo *modinfo;
> +    const char **sl;
>   
>       if (!type) {
>           return;
>       }
> -    for (i = 0; i < ARRAY_SIZE(qom_modules); i++) {
> -        if (strcmp(qom_modules[i].type, type) == 0) {
> -            module_load_one(qom_modules[i].prefix,
> -                            qom_modules[i].module,
> -                            false);
> -            return;
> +
> +    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
> +        if (!modinfo->objs) {
> +            continue;
> +        }
> +        for (sl = modinfo->objs; *sl != NULL; sl++) {
> +            if (strcmp(type, *sl) == 0) {
> +                module_load_one("", modinfo->name, false);
> +            }
>           }
>       }
>   }
>   
>   void module_load_qom_all(void)
>   {
> -    int i;
> +    const QemuModinfo *modinfo;
>   
>       if (module_loaded_qom_all) {
>           return;
>       }
> -    for (i = 0; i < ARRAY_SIZE(qom_modules); i++) {
> -        if (i > 0 && (strcmp(qom_modules[i - 1].module,
> -                             qom_modules[i].module) == 0 &&
> -                      strcmp(qom_modules[i - 1].prefix,
> -                             qom_modules[i].prefix) == 0)) {
> -            /* one module implementing multiple types -> load only once */
> +
> +    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
> +        if (!modinfo->objs) {
>               continue;
>           }
> -        module_load_one(qom_modules[i].prefix, qom_modules[i].module, true);
> +        module_load_one("", modinfo->name, false);
>       }
>       module_loaded_qom_all = true;
>   }
> +
> +#else
> +
> +void module_load_qom_one(const char *type) {}
> +void module_load_qom_all(void) {}
> +
> +#endif
> 

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

Re: [PATCH v3 15/24] modules: use modinfo for qemu opts load

[Paolo Bonzini]

On 18/06/21 06:53, Gerd Hoffmann wrote:
> Use module database to figure which module adds given QemuOpts group.
> 
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>   softmmu/vl.c        | 17 -----------------
>   stubs/module-opts.c |  4 ----
>   util/module.c       | 19 +++++++++++++++++++
>   3 files changed, 19 insertions(+), 21 deletions(-)
> 
> diff --git a/softmmu/vl.c b/softmmu/vl.c
> index a4857ec43ff3..c91d63e3cc02 100644
> --- a/softmmu/vl.c
> +++ b/softmmu/vl.c
> @@ -2675,23 +2675,6 @@ void qmp_x_exit_preconfig(Error **errp)
>       }
>   }
>   
> -#ifdef CONFIG_MODULES
> -void qemu_load_module_for_opts(const char *group)
> -{
> -    static bool spice_tried;
> -    if (g_str_equal(group, "spice") && !spice_tried) {
> -        ui_module_load_one("spice-core");
> -        spice_tried = true;
> -    }
> -
> -    static bool iscsi_tried;
> -    if (g_str_equal(group, "iscsi") && !iscsi_tried) {
> -        block_module_load_one("iscsi");
> -        iscsi_tried = true;
> -    }
> -}
> -#endif
> -
>   void qemu_init(int argc, char **argv, char **envp)
>   {
>       QemuOpts *opts;
> diff --git a/stubs/module-opts.c b/stubs/module-opts.c
> index a7d0e4ad6ead..5412429ea869 100644
> --- a/stubs/module-opts.c
> +++ b/stubs/module-opts.c
> @@ -1,6 +1,2 @@
>   #include "qemu/osdep.h"
>   #include "qemu/config-file.h"
> -
> -void qemu_load_module_for_opts(const char *group)
> -{
> -}
> diff --git a/util/module.c b/util/module.c
> index 745ae0fb20ed..a9ec2da9972e 100644
> --- a/util/module.c
> +++ b/util/module.c
> @@ -20,6 +20,7 @@
>   #include "qemu/queue.h"
>   #include "qemu/module.h"
>   #include "qemu/cutils.h"
> +#include "qemu/config-file.h"
>   #ifdef CONFIG_MODULE_UPGRADES
>   #include "qemu-version.h"
>   #endif
> @@ -322,8 +323,26 @@ void module_load_qom_all(void)
>       module_loaded_qom_all = true;
>   }
>   
> +void qemu_load_module_for_opts(const char *group)
> +{
> +    const QemuModinfo *modinfo;
> +    const char **sl;
> +
> +    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
> +        if (!modinfo->opts) {
> +            continue;
> +        }
> +        for (sl = modinfo->opts; *sl != NULL; sl++) {
> +            if (strcmp(group, *sl) == 0) {
> +                module_load_one("", modinfo->name, false);
> +            }
> +        }
> +    }
> +}
> +
>   #else
>   
> +void qemu_load_module_for_opts(const char *group) {}
>   void module_load_qom_one(const char *type) {}
>   void module_load_qom_all(void) {}
>   
> 

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

Re: [PATCH v3 02/24] modules: collect module meta-data

[Gerd Hoffmann]

On Fri, Jun 18, 2021 at 06:09:55PM +0200, Paolo Bonzini wrote:
> On 18/06/21 06:53, Gerd Hoffmann wrote:
> > +def find_command(src, target, compile_commands):
> > +    for command in compile_commands:
> > +        if command['file'] != src:
> > +            continue
> > +        if target != '' and command['command'].find(target) == -1:
> > +            continue
> 
> 
> Did you look into using extract_objects for this instead of looking for the
> target (which works, but yuck :))?

ninja: error: 'libui-curses.a.p/meson-generated_.._config-host.h.o', needed by 'ui-curses.modinfo.test', missing and no known rule to make it

Hmm, not sure where this comes from.  meson doesn't try to link
config-host.h.o into libui-curses.a, so why does extract_all_objects()
return it?

Test patch (incremental to this series) below.

take care,
  Gerd

From 5453683429d7b08b959e2cd63ee00fdccfb0c7b7 Mon Sep 17 00:00:00 2001
From: Gerd Hoffmann <kraxel@redhat.com>
Date: Mon, 21 Jun 2021 14:45:14 +0200
Subject: [PATCH] [wip] extract_all_objects experiments

---
 meson.build             | 7 +++++++
 scripts/modinfo-test.sh | 8 ++++++++
 2 files changed, 15 insertions(+)
 create mode 100755 scripts/modinfo-test.sh

diff --git a/meson.build b/meson.build
index 03bacca7cddb..8e7ccccf176c 100644
--- a/meson.build
+++ b/meson.build
@@ -2042,6 +2042,7 @@ target_modules += { 'accel' : { 'qtest': qtest_module_ss,
 
 modinfo_collect = find_program('scripts/modinfo-collect.py')
 modinfo_generate = find_program('scripts/modinfo-generate.py')
+modinfo_test = find_program('scripts/modinfo-test.sh')
 modinfo_files = []
 
 block_mods = []
@@ -2063,6 +2064,12 @@ foreach d, list : modules
                                        input: module_ss.sources(),
                                        capture: true,
                                        command: [modinfo_collect, '@INPUT@'])
+        custom_target(d + '-' + m + '.modinfo.test',
+                      output: d + '-' + m + '.modinfo.test',
+                      input: sl.extract_all_objects(recursive: true),
+                      capture: true,
+                      build_by_default: true, # to be removed when added to a target
+                      command: [modinfo_test, '@INPUT@'])
       endif
     else
       if d == 'block'
diff --git a/scripts/modinfo-test.sh b/scripts/modinfo-test.sh
new file mode 100755
index 000000000000..979c9cc9aeef
--- /dev/null
+++ b/scripts/modinfo-test.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+if test "$1" = "--target"; then
+    echo "# target $2"
+    shift;shift
+fi
+for item in "$@"; do
+    echo "# input $item"
+done
-- 
2.31.1

Re: [PATCH v3 03/24] modules: generate modinfo.c

[Jose R. Ziviani]

[-- Attachment #1: Type: text/plain, Size: 6758 bytes --]

Hello,

Just a small change.

On Fri, Jun 18, 2021 at 06:53:32AM +0200, Gerd Hoffmann wrote:
> Add script to generate C source with a small
> database containing the module meta-data.
> 
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  scripts/modinfo-generate.py | 84 +++++++++++++++++++++++++++++++++++++
>  include/qemu/module.h       | 17 ++++++++
>  softmmu/vl.c                |  4 ++
>  util/module.c               | 11 +++++
>  meson.build                 | 13 +++++-
>  5 files changed, 128 insertions(+), 1 deletion(-)
>  create mode 100755 scripts/modinfo-generate.py
> 
> diff --git a/scripts/modinfo-generate.py b/scripts/modinfo-generate.py
> new file mode 100755
> index 000000000000..2b925432655a
> --- /dev/null
> +++ b/scripts/modinfo-generate.py
> @@ -0,0 +1,84 @@
> +#!/usr/bin/env python3
> +# -*- coding: utf-8 -*-
> +
> +import os
> +import sys
> +
> +def print_array(name, values):
> +    if len(values) == 0:
> +        return
> +    list = ", ".join(values)
> +    print("    .%s = ((const char*[]){ %s, NULL })," % (name, list))
> +
> +def parse_line(line):
> +    kind = ""
> +    data = ""
> +    get_kind = False
> +    get_data = False
> +    for item in line.split():
> +        if item == "MODINFO_START":
> +            get_kind = True
> +            continue
> +        if item.startswith("MODINFO_END"):
> +            get_data = False
> +            continue
> +        if get_kind:
> +            kind = item
> +            get_kind = False
> +            get_data = True
> +            continue
> +        if get_data:
> +            data += " " + item
> +            continue
> +    return (kind, data)
> +
> +def generate(name, lines):
> +    arch = ""
> +    objs = []
> +    deps = []
> +    opts = []
> +    for line in lines:
> +        if line.find("MODINFO_START") != -1:
> +            (kind, data) = parse_line(line)
> +            if kind == 'obj':
> +                objs.append(data)
> +            elif kind == 'dep':
> +                deps.append(data)
> +            elif kind == 'opts':
> +                opts.append(data)
> +            elif kind == 'arch':
> +                arch = data;
> +            else:
> +                print("unknown:", kind)
> +                exit(1)
> +
> +    print("    .name = \"%s\"," % name)
> +    if arch != "":
> +        print("    .arch = %s," % arch)
> +    print_array("objs", objs)
> +    print_array("deps", deps)
> +    print_array("opts", opts)
> +    print("},{");
> +
> +def print_pre():
> +    print("/* generated by scripts/modinfo.py */")

generated by scripts/modinfo-generate.py

> +    print("#include \"qemu/osdep.h\"")
> +    print("#include \"qemu/module.h\"")
> +    print("const QemuModinfo qemu_modinfo[] = {{")
> +
> +def print_post():
> +    print("    /* end of list */")
> +    print("}};")
> +
> +def main(args):
> +    print_pre()
> +    for modinfo in args:
> +        with open(modinfo) as f:
> +            lines = f.readlines()
> +        print("    /* %s */" % modinfo)
> +        (basename, ext) = os.path.splitext(modinfo)
> +        generate(basename, lines)
> +    print_post()
> +
> +if __name__ == "__main__":
> +    main(sys.argv[1:])
> diff --git a/include/qemu/module.h b/include/qemu/module.h
> index 81ef086da023..a98748d501d3 100644
> --- a/include/qemu/module.h
> +++ b/include/qemu/module.h
> @@ -98,4 +98,21 @@ void module_load_qom_all(void);
>  /* module registers QemuOpts <name> */
>  #define module_opts(name) modinfo(opts, name)
>  
> +/*
> + * module info database
> + *
> + * scripts/modinfo-generate.c will build this using the data collected
> + * by scripts/modinfo-collect.py
> + */
> +typedef struct QemuModinfo QemuModinfo;
> +struct QemuModinfo {
> +    const char *name;
> +    const char *arch;
> +    const char **objs;
> +    const char **deps;
> +    const char **opts;
> +};
> +extern const QemuModinfo qemu_modinfo[];
> +void module_init_info(const QemuModinfo *info);
> +
>  #endif
> diff --git a/softmmu/vl.c b/softmmu/vl.c
> index 326c1e908008..a4857ec43ff3 100644
> --- a/softmmu/vl.c
> +++ b/softmmu/vl.c
> @@ -2740,6 +2740,10 @@ void qemu_init(int argc, char **argv, char **envp)
>      error_init(argv[0]);
>      qemu_init_exec_dir(argv[0]);
>  
> +#ifdef CONFIG_MODULES
> +    module_init_info(qemu_modinfo);
> +#endif
> +
>      qemu_init_subsystems();
>  
>      /* first pass of option parsing */
> diff --git a/util/module.c b/util/module.c
> index eee8ff2de136..8d3e8275b9f7 100644
> --- a/util/module.c
> +++ b/util/module.c
> @@ -110,6 +110,17 @@ void module_call_init(module_init_type type)
>  }
>  
>  #ifdef CONFIG_MODULES
> +
> +static const QemuModinfo module_info_stub[] = { {
> +    /* end of list */
> +} };
> +static const QemuModinfo *module_info = module_info_stub;
> +
> +void module_init_info(const QemuModinfo *info)
> +{
> +    module_info = info;
> +}
> +
>  static int module_load_file(const char *fname, bool mayfail, bool export_symbols)
>  {
>      GModule *g_module;
> diff --git a/meson.build b/meson.build
> index bb99619257d5..9cf50a50d39a 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -2022,6 +2022,7 @@ subdir('tests/qtest/fuzz')
>  ########################
>  
>  modinfo_collect = find_program('scripts/modinfo-collect.py')
> +modinfo_generate = find_program('scripts/modinfo-generate.py')
>  modinfo_files = []
>  
>  block_mods = []
> @@ -2042,7 +2043,6 @@ foreach d, list : modules
>                                         output: d + '-' + m + '.modinfo',
>                                         input: module_ss.sources(),
>                                         capture: true,
> -                                       build_by_default: true, # to be removed when added to a target
>                                         command: [modinfo_collect, '@INPUT@'])
>        endif
>      else
> @@ -2055,6 +2055,17 @@ foreach d, list : modules
>    endforeach
>  endforeach
>  
> +if enable_modules
> +  modinfo_src = custom_target('modinfo.c',
> +                              output: 'modinfo.c',
> +                              input: modinfo_files,
> +                              command: [modinfo_generate, '@INPUT@'],
> +                              capture: true)
> +  modinfo_lib = static_library('modinfo', modinfo_src)
> +  modinfo_dep = declare_dependency(link_whole: modinfo_lib)
> +  softmmu_ss.add(modinfo_dep)
> +endif
> +
>  nm = find_program('nm')
>  undefsym = find_program('scripts/undefsym.py')
>  block_syms = custom_target('block.syms', output: 'block.syms',
> -- 
> 2.31.1
> 
> 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3 03/24] modules: generate modinfo.c

[Jose R. Ziviani]

[-- Attachment #1.1: Type: text/plain, Size: 7679 bytes --]

On Fri, Jun 18, 2021 at 06:53:32AM +0200, Gerd Hoffmann wrote:
> Add script to generate C source with a small
> database containing the module meta-data.
> 
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  scripts/modinfo-generate.py | 84 +++++++++++++++++++++++++++++++++++++
>  include/qemu/module.h       | 17 ++++++++
>  softmmu/vl.c                |  4 ++
>  util/module.c               | 11 +++++
>  meson.build                 | 13 +++++-
>  5 files changed, 128 insertions(+), 1 deletion(-)
>  create mode 100755 scripts/modinfo-generate.py
> 
> diff --git a/scripts/modinfo-generate.py b/scripts/modinfo-generate.py
> new file mode 100755
> index 000000000000..2b925432655a
> --- /dev/null
> +++ b/scripts/modinfo-generate.py
> @@ -0,0 +1,84 @@
> +#!/usr/bin/env python3
> +# -*- coding: utf-8 -*-
> +
> +import os
> +import sys
> +
> +def print_array(name, values):
> +    if len(values) == 0:
> +        return
> +    list = ", ".join(values)
> +    print("    .%s = ((const char*[]){ %s, NULL })," % (name, list))
> +
> +def parse_line(line):
> +    kind = ""
> +    data = ""
> +    get_kind = False
> +    get_data = False
> +    for item in line.split():
> +        if item == "MODINFO_START":
> +            get_kind = True
> +            continue
> +        if item.startswith("MODINFO_END"):
> +            get_data = False
> +            continue
> +        if get_kind:
> +            kind = item
> +            get_kind = False
> +            get_data = True
> +            continue
> +        if get_data:
> +            data += " " + item
> +            continue
> +    return (kind, data)
> +
> +def generate(name, lines):
> +    arch = ""
> +    objs = []
> +    deps = []
> +    opts = []
> +    for line in lines:
> +        if line.find("MODINFO_START") != -1:
> +            (kind, data) = parse_line(line)
> +            if kind == 'obj':
> +                objs.append(data)
> +            elif kind == 'dep':
> +                deps.append(data)
> +            elif kind == 'opts':
> +                opts.append(data)
> +            elif kind == 'arch':
> +                arch = data;
> +            else:
> +                print("unknown:", kind)
> +                exit(1)
> +
> +    print("    .name = \"%s\"," % name)
> +    if arch != "":
> +        print("    .arch = %s," % arch)
> +    print_array("objs", objs)
> +    print_array("deps", deps)
> +    print_array("opts", opts)
> +    print("},{");
> +
> +def print_pre():
> +    print("/* generated by scripts/modinfo.py */")
> +    print("#include \"qemu/osdep.h\"")
> +    print("#include \"qemu/module.h\"")
> +    print("const QemuModinfo qemu_modinfo[] = {{")
> +
> +def print_post():
> +    print("    /* end of list */")
> +    print("}};")
> +
> +def main(args):
> +    print_pre()
> +    for modinfo in args:
> +        with open(modinfo) as f:
> +            lines = f.readlines()
> +        print("    /* %s */" % modinfo)
> +        (basename, ext) = os.path.splitext(modinfo)
> +        generate(basename, lines)
> +    print_post()

I attached a patch in this message to check if a dependency can be satisfied. It will detect the following case:
(in any module)
module_dep("blabla");

[2/5] Generating modinfo.c with a custom command (wrapped by meson to capture output)
FAILED: modinfo.c 
/usr/bin/meson --internal exe --capture modinfo.c -- /home/...
Dependency blabla cannot be satisfied
/* generated by scripts/modinfo.py */
#include "qemu/osdep.h"
#include "qemu/module.h"
const QemuModinfo qemu_modinfo[] = {{
    /* ui-curses.modinfo */
    .name = "ui-curses",
...
},{
    /* accel-tcg-x86_64.modinfo */
    .name = "accel-tcg-x86_64",
    .arch =  "x86_64",
    .objs = ((const char*[]){  ("tcg" "-" "accel" "-ops"), NULL }),
    .deps = ((const char*[]){  "blabla", NULL }),
},{
    /* end of list */
}};
ninja: build stopped: subcommand failed.
make: *** [Makefile:154: run-ninja] Error 1

It can help developers to know early they have a typo somewhere.
You can add this code if you like.

> +
> +if __name__ == "__main__":
> +    main(sys.argv[1:])
> diff --git a/include/qemu/module.h b/include/qemu/module.h
> index 81ef086da023..a98748d501d3 100644
> --- a/include/qemu/module.h
> +++ b/include/qemu/module.h
> @@ -98,4 +98,21 @@ void module_load_qom_all(void);
>  /* module registers QemuOpts <name> */
>  #define module_opts(name) modinfo(opts, name)
>  
> +/*
> + * module info database
> + *
> + * scripts/modinfo-generate.c will build this using the data collected
> + * by scripts/modinfo-collect.py
> + */
> +typedef struct QemuModinfo QemuModinfo;
> +struct QemuModinfo {
> +    const char *name;
> +    const char *arch;
> +    const char **objs;
> +    const char **deps;
> +    const char **opts;
> +};
> +extern const QemuModinfo qemu_modinfo[];
> +void module_init_info(const QemuModinfo *info);
> +
>  #endif
> diff --git a/softmmu/vl.c b/softmmu/vl.c
> index 326c1e908008..a4857ec43ff3 100644
> --- a/softmmu/vl.c
> +++ b/softmmu/vl.c
> @@ -2740,6 +2740,10 @@ void qemu_init(int argc, char **argv, char **envp)
>      error_init(argv[0]);
>      qemu_init_exec_dir(argv[0]);
>  
> +#ifdef CONFIG_MODULES
> +    module_init_info(qemu_modinfo);
> +#endif
> +
>      qemu_init_subsystems();
>  
>      /* first pass of option parsing */
> diff --git a/util/module.c b/util/module.c
> index eee8ff2de136..8d3e8275b9f7 100644
> --- a/util/module.c
> +++ b/util/module.c
> @@ -110,6 +110,17 @@ void module_call_init(module_init_type type)
>  }
>  
>  #ifdef CONFIG_MODULES
> +
> +static const QemuModinfo module_info_stub[] = { {
> +    /* end of list */
> +} };
> +static const QemuModinfo *module_info = module_info_stub;
> +
> +void module_init_info(const QemuModinfo *info)
> +{
> +    module_info = info;
> +}
> +
>  static int module_load_file(const char *fname, bool mayfail, bool export_symbols)
>  {
>      GModule *g_module;
> diff --git a/meson.build b/meson.build
> index bb99619257d5..9cf50a50d39a 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -2022,6 +2022,7 @@ subdir('tests/qtest/fuzz')
>  ########################
>  
>  modinfo_collect = find_program('scripts/modinfo-collect.py')
> +modinfo_generate = find_program('scripts/modinfo-generate.py')
>  modinfo_files = []
>  
>  block_mods = []
> @@ -2042,7 +2043,6 @@ foreach d, list : modules
>                                         output: d + '-' + m + '.modinfo',
>                                         input: module_ss.sources(),
>                                         capture: true,
> -                                       build_by_default: true, # to be removed when added to a target
>                                         command: [modinfo_collect, '@INPUT@'])
>        endif
>      else
> @@ -2055,6 +2055,17 @@ foreach d, list : modules
>    endforeach
>  endforeach
>  
> +if enable_modules
> +  modinfo_src = custom_target('modinfo.c',
> +                              output: 'modinfo.c',
> +                              input: modinfo_files,
> +                              command: [modinfo_generate, '@INPUT@'],
> +                              capture: true)
> +  modinfo_lib = static_library('modinfo', modinfo_src)
> +  modinfo_dep = declare_dependency(link_whole: modinfo_lib)
> +  softmmu_ss.add(modinfo_dep)
> +endif
> +
>  nm = find_program('nm')
>  undefsym = find_program('scripts/undefsym.py')
>  block_syms = custom_target('block.syms', output: 'block.syms',
> -- 
> 2.31.1
> 
> 

[-- Attachment #1.2: 0001-modules-check-if-all-dependencies-can-be-satisfied.patch --]
[-- Type: text/x-patch, Size: 1732 bytes --]

From 0e6866bcae2d6576e7dbc08d8bbd2837f655d5a3 Mon Sep 17 00:00:00 2001
From: "Jose R. Ziviani" <jziviani@suse.de>
Date: Tue, 22 Jun 2021 17:37:42 -0300
Subject: [PATCH] modules: check if all dependencies can be satisfied

Verifies if all dependencies are correctly listed in the modinfo.c too
and stop the builds if they're not.

Signed-off-by: Jose R. Ziviani <jziviani@suse.de>
---
 scripts/modinfo-generate.py | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/scripts/modinfo-generate.py b/scripts/modinfo-generate.py
index 2b92543265..a36ddb77dd 100755
--- a/scripts/modinfo-generate.py
+++ b/scripts/modinfo-generate.py
@@ -59,6 +59,7 @@ def generate(name, lines):
     print_array("deps", deps)
     print_array("opts", opts)
     print("},{");
+    return deps
 
 def print_pre():
     print("/* generated by scripts/modinfo.py */")
@@ -71,14 +72,26 @@ def print_post():
     print("}};")
 
 def main(args):
+    deps = {}
     print_pre()
     for modinfo in args:
         with open(modinfo) as f:
             lines = f.readlines()
         print("    /* %s */" % modinfo)
         (basename, ext) = os.path.splitext(modinfo)
-        generate(basename, lines)
+        deps[basename] = generate(basename, lines)
     print_post()
 
+    flattened_deps = {flat.strip('" ') for dep in deps.values() for flat in dep}
+    error = False
+    for dep in flattened_deps:
+        if dep not in deps.keys():
+            print("Dependency {} cannot be satisfied".format(dep),
+                  file=sys.stderr)
+            error = True
+
+    if error:
+        exit(1)
+
 if __name__ == "__main__":
     main(sys.argv[1:])
-- 
2.32.0


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]