[PATCH v4 00/34] modules: add meta-data database

[PATCH v4 00/34] modules: add meta-data database

[Gerd Hoffmann]

This patch series adds support for module meta-data.  Today this is
either hard-coded in qemu (see qemu_load_module_for_opts) or handled
with manually maintained lists in util/module (see module_deps[] and
qom_modules[]).  This series replaced that scheme with annotation
macros, so the meta-data can go into the module source code and -- for
example -- the module_obj() annotations can go next to the TypeInfo
struct for the object class.

Patches 1-3 put the infrastructure in place:  Add the annotation macros,
add a script to collect the meta-data, add a script to compile the
meta-data into C source code which we can then add to qemu.

Patch 4 - check module dependencies (Jose, new in v4).

Patches 5-13 add annotations macros to the modules we have.

Patches 14-16 put the modinfo database into use and remove the
module_deps[] and qom_modules[] lists.

Patch 16 adds two tracepoints for easier trouble-shooting.

Patches 18-20 add support for target-specific modules.

Patches 21-24 add documentation for all of the above (new in v4, was
separate series).

Patches 25-29 start building accelerators modular.  So far it is
only qtest (all archs) and a small fraction of tcg (x86 only).

Patches 30-34 add support for registering hmp commands so they can
be implemented as module (new in v4, was separate series).

take care,
  Gerd

Gerd Hoffmann (33):
  modules: add modinfo macros
  modules: collect module meta-data
  modules: generate modinfo.c
  modules: add qxl module annotations
  modules: add virtio-gpu module annotations
  modules: add chardev module annotations
  modules: add audio module annotations
  modules: add usb-redir module annotations
  modules: add ccid module annotations
  modules: add ui module annotations
  modules: add s390x module annotations
  modules: add block module annotations
  modules: use modinfo for dependencies
  modules: use modinfo for qom load
  modules: use modinfo for qemu opts load
  modules: add tracepoints
  modules: check arch and block load on mismatch
  modules: check arch on qom lookup
  modules: target-specific module build infrastructure
  modules: add documentation for module sourcesets
  modules: add module_obj() note to QOM docs
  modules: module.h kerneldoc annotations
  modules: hook up modules.h to docs build
  accel: autoload modules
  accel: add qtest module annotations
  accel: build qtest modular
  accel: add tcg module annotations
  accel: build tcg modular
  monitor: allow register hmp commands
  usb: drop usb_host_dev_is_scsi_storage hook
  monitor/usb: register 'info usbhost' dynamically
  usb: build usb-host as module
  monitor/tcg: move tcg hmp commands to accel/tcg, register them
    dynamically

Jose R. Ziviani (1):
  modules: check if all dependencies can be satisfied

 scripts/modinfo-collect.py      |  67 +++++++++++
 scripts/modinfo-generate.py     |  97 ++++++++++++++++
 include/hw/usb.h                |   7 +-
 include/monitor/monitor.h       |   3 +
 include/qemu/module.h           |  74 ++++++++++++
 accel/accel-common.c            |   2 +-
 accel/accel-softmmu.c           |   2 +-
 accel/qtest/qtest.c             |   2 +
 accel/tcg/hmp.c                 |  29 +++++
 accel/tcg/tcg-accel-ops.c       |   1 +
 accel/tcg/tcg-all.c             |   1 +
 audio/spiceaudio.c              |   2 +
 block/iscsi-opts.c              |   1 +
 chardev/baum.c                  |   1 +
 chardev/spice.c                 |   4 +
 hw/display/qxl.c                |   4 +
 hw/display/vhost-user-gpu-pci.c |   1 +
 hw/display/vhost-user-gpu.c     |   1 +
 hw/display/vhost-user-vga.c     |   1 +
 hw/display/virtio-gpu-base.c    |   1 +
 hw/display/virtio-gpu-gl.c      |   3 +
 hw/display/virtio-gpu-pci-gl.c  |   3 +
 hw/display/virtio-gpu-pci.c     |   2 +
 hw/display/virtio-gpu.c         |   1 +
 hw/display/virtio-vga-gl.c      |   3 +
 hw/display/virtio-vga.c         |   2 +
 hw/ppc/spapr.c                  |   2 +-
 hw/s390x/virtio-ccw-gpu.c       |   3 +
 hw/usb/ccid-card-emulated.c     |   1 +
 hw/usb/ccid-card-passthru.c     |   1 +
 hw/usb/dev-storage-bot.c        |   1 +
 hw/usb/dev-storage-classic.c    |   1 +
 hw/usb/dev-uas.c                |   1 +
 hw/usb/host-libusb.c            |  38 ++----
 hw/usb/host-stub.c              |  45 -------
 hw/usb/redirect.c               |   1 +
 monitor/hmp.c                   |   7 ++
 monitor/misc.c                  |  34 +++---
 softmmu/vl.c                    |  24 ++--
 stubs/module-opts.c             |   4 -
 ui/egl-headless.c               |   4 +
 ui/gtk.c                        |   4 +
 ui/sdl2.c                       |   4 +
 ui/spice-app.c                  |   3 +
 ui/spice-core.c                 |   5 +
 util/module.c                   | 200 ++++++++++++++++++--------------
 accel/qtest/meson.build         |   8 +-
 accel/tcg/meson.build           |   6 +-
 docs/devel/build-system.rst     |  17 +++
 docs/devel/index.rst            |   1 +
 docs/devel/modules.rst          |   5 +
 docs/devel/qom.rst              |   8 ++
 hmp-commands-info.hx            |   3 -
 hw/usb/meson.build              |  10 +-
 meson.build                     |  82 +++++++++++++
 util/trace-events               |   4 +
 56 files changed, 624 insertions(+), 218 deletions(-)
 create mode 100755 scripts/modinfo-collect.py
 create mode 100755 scripts/modinfo-generate.py
 create mode 100644 accel/tcg/hmp.c
 delete mode 100644 hw/usb/host-stub.c
 create mode 100644 docs/devel/modules.rst

-- 
2.31.1

[PATCH v4 01/34] modules: add modinfo macros

[Gerd Hoffmann]

Add macros for module info annotations.

Instead of having that module meta-data stored in lists in util/module.c
place directly in the module source code.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 include/qemu/module.h | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/include/qemu/module.h b/include/qemu/module.h
index 944d403cbd15..81ef086da023 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -73,4 +73,29 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail);
 void module_load_qom_one(const char *type);
 void module_load_qom_all(void);
 
+/*
+ * module info annotation macros
+ *
+ * scripts/modinfo-collect.py will collect module info,
+ * using the preprocessor and -DQEMU_MODINFO
+ */
+#ifdef QEMU_MODINFO
+# define modinfo(kind, value) \
+    MODINFO_START kind value MODINFO_END
+#else
+# define modinfo(kind, value)
+#endif
+
+/* module implements QOM type <name> */
+#define module_obj(name) modinfo(obj, name)
+
+/* module has a dependency on <name> */
+#define module_dep(name) modinfo(dep, name)
+
+/* module is for target architecture <name> */
+#define module_arch(name) modinfo(arch, name)
+
+/* module registers QemuOpts <name> */
+#define module_opts(name) modinfo(opts, name)
+
 #endif
-- 
2.31.1

Re: [PATCH v4 01/34] modules: add modinfo macros

[Eduardo Habkost]

On Thu, Jun 24, 2021 at 12:38:03PM +0200, Gerd Hoffmann wrote:
> Add macros for module info annotations.
> 
> Instead of having that module meta-data stored in lists in util/module.c
> place directly in the module source code.
> 
[...]
> +/* module implements QOM type <name> */
> +#define module_obj(name) modinfo(obj, name)

Can we make OBJECT_DEFINE_TYPE*() use this macro automatically?

-- 
Eduardo

Re: [PATCH v4 01/34] modules: add modinfo macros

[Paolo Bonzini]

On 24/06/21 22:37, Eduardo Habkost wrote:
> On Thu, Jun 24, 2021 at 12:38:03PM +0200, Gerd Hoffmann wrote:
>> Add macros for module info annotations.
>>
>> Instead of having that module meta-data stored in lists in util/module.c
>> place directly in the module source code.
>>
> [...]
>> +/* module implements QOM type <name> */
>> +#define module_obj(name) modinfo(obj, name)
> 
> Can we make OBJECT_DEFINE_TYPE*() use this macro automatically?
> 

Yeah, that's possible.  I would do it as a separate patch though, 
because Gerd is on vacation and he asked me to include it in a pull 
request before soft freeze.

Thanks,

Paolo

[PATCH v4 02/34] modules: collect module meta-data

[Gerd Hoffmann]

Add script to collect the module meta-data from the source code,
store the results in *.modinfo files.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 scripts/modinfo-collect.py | 67 ++++++++++++++++++++++++++++++++++++++
 meson.build                | 16 +++++++++
 2 files changed, 83 insertions(+)
 create mode 100755 scripts/modinfo-collect.py

diff --git a/scripts/modinfo-collect.py b/scripts/modinfo-collect.py
new file mode 100755
index 000000000000..4acb188c3e89
--- /dev/null
+++ b/scripts/modinfo-collect.py
@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+import os
+import sys
+import json
+import shlex
+import subprocess
+
+def find_command(src, target, compile_commands):
+    for command in compile_commands:
+        if command['file'] != src:
+            continue
+        if target != '' and command['command'].find(target) == -1:
+            continue
+        return command['command']
+    return 'false'
+
+def process_command(src, command):
+    skip = False
+    arg = False
+    out = []
+    for item in shlex.split(command):
+        if arg:
+            out.append(x)
+            arg = False
+            continue
+        if skip:
+            skip = False
+            continue
+        if item == '-MF' or item == '-MQ' or item == '-o':
+            skip = True
+            continue
+        if item == '-c':
+            skip = True
+            continue
+        out.append(item)
+    out.append('-DQEMU_MODINFO')
+    out.append('-E')
+    out.append(src)
+    return out
+
+def main(args):
+    target = ''
+    if args[0] == '--target':
+        args.pop(0)
+        target = args.pop(0)
+        print("MODINFO_DEBUG target %s" % target)
+        arch = target[:-8] # cut '-softmmu'
+        print("MODINFO_START arch \"%s\" MODINFO_END" % arch)
+    with open('compile_commands.json') as f:
+        compile_commands = json.load(f)
+    for src in args:
+        print("MODINFO_DEBUG src %s" % src)
+        command = find_command(src, target, compile_commands)
+        cmdline = process_command(src, command)
+        print("MODINFO_DEBUG cmd", cmdline)
+        result = subprocess.run(cmdline, stdout = subprocess.PIPE,
+                                universal_newlines = True)
+        if result.returncode != 0:
+            sys.exit(result.returncode)
+        for line in result.stdout.split('\n'):
+            if line.find('MODINFO') != -1:
+                print(line)
+
+if __name__ == "__main__":
+    main(sys.argv[1:])
diff --git a/meson.build b/meson.build
index d8a92666fbcd..30ca06c6c2d4 100644
--- a/meson.build
+++ b/meson.build
@@ -2021,6 +2021,9 @@ subdir('tests/qtest/fuzz')
 # Library dependencies #
 ########################
 
+modinfo_collect = find_program('scripts/modinfo-collect.py')
+modinfo_files = []
+
 block_mods = []
 softmmu_mods = []
 foreach d, list : modules
@@ -2034,6 +2037,19 @@ foreach d, list : modules
       else
         softmmu_mods += sl
       endif
+      if module_ss.sources() != []
+        # FIXME: Should use sl.extract_all_objects(recursive: true) as
+        # input. Sources can be used multiple times but objects are
+        # unique when it comes to lookup in compile_commands.json.
+        # Depnds on a mesion version with
+        # https://github.com/mesonbuild/meson/pull/8900
+        modinfo_files += custom_target(d + '-' + m + '.modinfo',
+                                       output: d + '-' + m + '.modinfo',
+                                       input: module_ss.sources(),
+                                       capture: true,
+                                       build_by_default: true, # to be removed when added to a target
+                                       command: [modinfo_collect, '@INPUT@'])
+      endif
     else
       if d == 'block'
         block_ss.add_all(module_ss)
-- 
2.31.1

[PATCH v4 03/34] modules: generate modinfo.c

[Gerd Hoffmann]

Add script to generate C source with a small
database containing the module meta-data.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 scripts/modinfo-generate.py | 84 +++++++++++++++++++++++++++++++++++++
 include/qemu/module.h       | 17 ++++++++
 softmmu/vl.c                |  4 ++
 util/module.c               | 11 +++++
 meson.build                 | 13 +++++-
 5 files changed, 128 insertions(+), 1 deletion(-)
 create mode 100755 scripts/modinfo-generate.py

diff --git a/scripts/modinfo-generate.py b/scripts/modinfo-generate.py
new file mode 100755
index 000000000000..a6d98a6bc4cc
--- /dev/null
+++ b/scripts/modinfo-generate.py
@@ -0,0 +1,84 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+import os
+import sys
+
+def print_array(name, values):
+    if len(values) == 0:
+        return
+    list = ", ".join(values)
+    print("    .%s = ((const char*[]){ %s, NULL })," % (name, list))
+
+def parse_line(line):
+    kind = ""
+    data = ""
+    get_kind = False
+    get_data = False
+    for item in line.split():
+        if item == "MODINFO_START":
+            get_kind = True
+            continue
+        if item.startswith("MODINFO_END"):
+            get_data = False
+            continue
+        if get_kind:
+            kind = item
+            get_kind = False
+            get_data = True
+            continue
+        if get_data:
+            data += " " + item
+            continue
+    return (kind, data)
+
+def generate(name, lines):
+    arch = ""
+    objs = []
+    deps = []
+    opts = []
+    for line in lines:
+        if line.find("MODINFO_START") != -1:
+            (kind, data) = parse_line(line)
+            if kind == 'obj':
+                objs.append(data)
+            elif kind == 'dep':
+                deps.append(data)
+            elif kind == 'opts':
+                opts.append(data)
+            elif kind == 'arch':
+                arch = data;
+            else:
+                print("unknown:", kind)
+                exit(1)
+
+    print("    .name = \"%s\"," % name)
+    if arch != "":
+        print("    .arch = %s," % arch)
+    print_array("objs", objs)
+    print_array("deps", deps)
+    print_array("opts", opts)
+    print("},{");
+
+def print_pre():
+    print("/* generated by scripts/modinfo-generate.py */")
+    print("#include \"qemu/osdep.h\"")
+    print("#include \"qemu/module.h\"")
+    print("const QemuModinfo qemu_modinfo[] = {{")
+
+def print_post():
+    print("    /* end of list */")
+    print("}};")
+
+def main(args):
+    print_pre()
+    for modinfo in args:
+        with open(modinfo) as f:
+            lines = f.readlines()
+        print("    /* %s */" % modinfo)
+        (basename, ext) = os.path.splitext(modinfo)
+        generate(basename, lines)
+    print_post()
+
+if __name__ == "__main__":
+    main(sys.argv[1:])
diff --git a/include/qemu/module.h b/include/qemu/module.h
index 81ef086da023..a98748d501d3 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -98,4 +98,21 @@ void module_load_qom_all(void);
 /* module registers QemuOpts <name> */
 #define module_opts(name) modinfo(opts, name)
 
+/*
+ * module info database
+ *
+ * scripts/modinfo-generate.c will build this using the data collected
+ * by scripts/modinfo-collect.py
+ */
+typedef struct QemuModinfo QemuModinfo;
+struct QemuModinfo {
+    const char *name;
+    const char *arch;
+    const char **objs;
+    const char **deps;
+    const char **opts;
+};
+extern const QemuModinfo qemu_modinfo[];
+void module_init_info(const QemuModinfo *info);
+
 #endif
diff --git a/softmmu/vl.c b/softmmu/vl.c
index feb4d201f30f..e001505d79f7 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2740,6 +2740,10 @@ void qemu_init(int argc, char **argv, char **envp)
     error_init(argv[0]);
     qemu_init_exec_dir(argv[0]);
 
+#ifdef CONFIG_MODULES
+    module_init_info(qemu_modinfo);
+#endif
+
     qemu_init_subsystems();
 
     /* first pass of option parsing */
diff --git a/util/module.c b/util/module.c
index eee8ff2de136..8d3e8275b9f7 100644
--- a/util/module.c
+++ b/util/module.c
@@ -110,6 +110,17 @@ void module_call_init(module_init_type type)
 }
 
 #ifdef CONFIG_MODULES
+
+static const QemuModinfo module_info_stub[] = { {
+    /* end of list */
+} };
+static const QemuModinfo *module_info = module_info_stub;
+
+void module_init_info(const QemuModinfo *info)
+{
+    module_info = info;
+}
+
 static int module_load_file(const char *fname, bool mayfail, bool export_symbols)
 {
     GModule *g_module;
diff --git a/meson.build b/meson.build
index 30ca06c6c2d4..6ae2b9ab9443 100644
--- a/meson.build
+++ b/meson.build
@@ -2022,6 +2022,7 @@ subdir('tests/qtest/fuzz')
 ########################
 
 modinfo_collect = find_program('scripts/modinfo-collect.py')
+modinfo_generate = find_program('scripts/modinfo-generate.py')
 modinfo_files = []
 
 block_mods = []
@@ -2047,7 +2048,6 @@ foreach d, list : modules
                                        output: d + '-' + m + '.modinfo',
                                        input: module_ss.sources(),
                                        capture: true,
-                                       build_by_default: true, # to be removed when added to a target
                                        command: [modinfo_collect, '@INPUT@'])
       endif
     else
@@ -2060,6 +2060,17 @@ foreach d, list : modules
   endforeach
 endforeach
 
+if enable_modules
+  modinfo_src = custom_target('modinfo.c',
+                              output: 'modinfo.c',
+                              input: modinfo_files,
+                              command: [modinfo_generate, '@INPUT@'],
+                              capture: true)
+  modinfo_lib = static_library('modinfo', modinfo_src)
+  modinfo_dep = declare_dependency(link_whole: modinfo_lib)
+  softmmu_ss.add(modinfo_dep)
+endif
+
 nm = find_program('nm')
 undefsym = find_program('scripts/undefsym.py')
 block_syms = custom_target('block.syms', output: 'block.syms',
-- 
2.31.1

[PATCH v4 04/34] modules: check if all dependencies can be satisfied

[Gerd Hoffmann]

From: "Jose R. Ziviani" <jziviani@suse.de>

Verifies if all dependencies are correctly listed in the modinfo.c too
and stop the builds if they're not.

Signed-off-by: Jose R. Ziviani <jziviani@suse.de>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 scripts/modinfo-generate.py | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/scripts/modinfo-generate.py b/scripts/modinfo-generate.py
index a6d98a6bc4cc..f559eed0077a 100755
--- a/scripts/modinfo-generate.py
+++ b/scripts/modinfo-generate.py
@@ -59,6 +59,7 @@ def generate(name, lines):
     print_array("deps", deps)
     print_array("opts", opts)
     print("},{");
+    return deps
 
 def print_pre():
     print("/* generated by scripts/modinfo-generate.py */")
@@ -71,14 +72,26 @@ def print_post():
     print("}};")
 
 def main(args):
+    deps = {}
     print_pre()
     for modinfo in args:
         with open(modinfo) as f:
             lines = f.readlines()
         print("    /* %s */" % modinfo)
         (basename, ext) = os.path.splitext(modinfo)
-        generate(basename, lines)
+        deps[basename] = generate(basename, lines)
     print_post()
 
+    flattened_deps = {flat.strip('" ') for dep in deps.values() for flat in dep}
+    error = False
+    for dep in flattened_deps:
+        if dep not in deps.keys():
+            print("Dependency {} cannot be satisfied".format(dep),
+                  file=sys.stderr)
+            error = True
+
+    if error:
+        exit(1)
+
 if __name__ == "__main__":
     main(sys.argv[1:])
-- 
2.31.1

[PATCH v4 05/34] modules: add qxl module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/display/qxl.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/hw/display/qxl.c b/hw/display/qxl.c
index 6e1f8ff1b2a7..84f99088e0a0 100644
--- a/hw/display/qxl.c
+++ b/hw/display/qxl.c
@@ -2522,6 +2522,7 @@ static const TypeInfo qxl_primary_info = {
     .parent        = TYPE_PCI_QXL,
     .class_init    = qxl_primary_class_init,
 };
+module_obj("qxl-vga");
 
 static void qxl_secondary_class_init(ObjectClass *klass, void *data)
 {
@@ -2538,6 +2539,7 @@ static const TypeInfo qxl_secondary_info = {
     .parent        = TYPE_PCI_QXL,
     .class_init    = qxl_secondary_class_init,
 };
+module_obj("qxl");
 
 static void qxl_register_types(void)
 {
@@ -2547,3 +2549,5 @@ static void qxl_register_types(void)
 }
 
 type_init(qxl_register_types)
+
+module_dep("ui-spice-core");
-- 
2.31.1

[PATCH v4 06/34] modules: add virtio-gpu module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/display/vhost-user-gpu-pci.c | 1 +
 hw/display/vhost-user-gpu.c     | 1 +
 hw/display/vhost-user-vga.c     | 1 +
 hw/display/virtio-gpu-base.c    | 1 +
 hw/display/virtio-gpu-gl.c      | 3 +++
 hw/display/virtio-gpu-pci-gl.c  | 3 +++
 hw/display/virtio-gpu-pci.c     | 2 ++
 hw/display/virtio-gpu.c         | 1 +
 hw/display/virtio-vga-gl.c      | 3 +++
 hw/display/virtio-vga.c         | 2 ++
 10 files changed, 18 insertions(+)

diff --git a/hw/display/vhost-user-gpu-pci.c b/hw/display/vhost-user-gpu-pci.c
index a02b23ecaf11..daefcf710159 100644
--- a/hw/display/vhost-user-gpu-pci.c
+++ b/hw/display/vhost-user-gpu-pci.c
@@ -43,6 +43,7 @@ static const VirtioPCIDeviceTypeInfo vhost_user_gpu_pci_info = {
     .instance_size = sizeof(VhostUserGPUPCI),
     .instance_init = vhost_user_gpu_pci_initfn,
 };
+module_obj(TYPE_VHOST_USER_GPU_PCI);
 
 static void vhost_user_gpu_pci_register_types(void)
 {
diff --git a/hw/display/vhost-user-gpu.c b/hw/display/vhost-user-gpu.c
index 6cdaa1c73b9b..32ef0061f924 100644
--- a/hw/display/vhost-user-gpu.c
+++ b/hw/display/vhost-user-gpu.c
@@ -596,6 +596,7 @@ static const TypeInfo vhost_user_gpu_info = {
     .instance_finalize = vhost_user_gpu_instance_finalize,
     .class_init = vhost_user_gpu_class_init,
 };
+module_obj(TYPE_VHOST_USER_GPU);
 
 static void vhost_user_gpu_register_types(void)
 {
diff --git a/hw/display/vhost-user-vga.c b/hw/display/vhost-user-vga.c
index a34a99856d73..072c9c65bc75 100644
--- a/hw/display/vhost-user-vga.c
+++ b/hw/display/vhost-user-vga.c
@@ -44,6 +44,7 @@ static const VirtioPCIDeviceTypeInfo vhost_user_vga_info = {
     .instance_size = sizeof(VhostUserVGA),
     .instance_init = vhost_user_vga_inst_initfn,
 };
+module_obj(TYPE_VHOST_USER_VGA);
 
 static void vhost_user_vga_register_types(void)
 {
diff --git a/hw/display/virtio-gpu-base.c b/hw/display/virtio-gpu-base.c
index dd294276cb38..c8da4806e0bb 100644
--- a/hw/display/virtio-gpu-base.c
+++ b/hw/display/virtio-gpu-base.c
@@ -256,6 +256,7 @@ static const TypeInfo virtio_gpu_base_info = {
     .class_init = virtio_gpu_base_class_init,
     .abstract = true
 };
+module_obj(TYPE_VIRTIO_GPU_BASE);
 
 static void
 virtio_register_types(void)
diff --git a/hw/display/virtio-gpu-gl.c b/hw/display/virtio-gpu-gl.c
index d971b480806a..7ab93bf8c829 100644
--- a/hw/display/virtio-gpu-gl.c
+++ b/hw/display/virtio-gpu-gl.c
@@ -154,6 +154,7 @@ static const TypeInfo virtio_gpu_gl_info = {
     .instance_size = sizeof(VirtIOGPUGL),
     .class_init = virtio_gpu_gl_class_init,
 };
+module_obj(TYPE_VIRTIO_GPU_GL);
 
 static void virtio_register_types(void)
 {
@@ -161,3 +162,5 @@ static void virtio_register_types(void)
 }
 
 type_init(virtio_register_types)
+
+module_dep("hw-display-virtio-gpu");
diff --git a/hw/display/virtio-gpu-pci-gl.c b/hw/display/virtio-gpu-pci-gl.c
index 902dda345275..99b14a07185e 100644
--- a/hw/display/virtio-gpu-pci-gl.c
+++ b/hw/display/virtio-gpu-pci-gl.c
@@ -46,6 +46,7 @@ static const VirtioPCIDeviceTypeInfo virtio_gpu_gl_pci_info = {
     .instance_size = sizeof(VirtIOGPUGLPCI),
     .instance_init = virtio_gpu_gl_initfn,
 };
+module_obj(TYPE_VIRTIO_GPU_GL_PCI);
 
 static void virtio_gpu_gl_pci_register_types(void)
 {
@@ -53,3 +54,5 @@ static void virtio_gpu_gl_pci_register_types(void)
 }
 
 type_init(virtio_gpu_gl_pci_register_types)
+
+module_dep("hw-display-virtio-gpu-pci");
diff --git a/hw/display/virtio-gpu-pci.c b/hw/display/virtio-gpu-pci.c
index d742a30aecf7..e36eee0c409b 100644
--- a/hw/display/virtio-gpu-pci.c
+++ b/hw/display/virtio-gpu-pci.c
@@ -64,6 +64,7 @@ static const TypeInfo virtio_gpu_pci_base_info = {
     .class_init = virtio_gpu_pci_base_class_init,
     .abstract = true
 };
+module_obj(TYPE_VIRTIO_GPU_PCI_BASE);
 
 #define TYPE_VIRTIO_GPU_PCI "virtio-gpu-pci"
 typedef struct VirtIOGPUPCI VirtIOGPUPCI;
@@ -90,6 +91,7 @@ static const VirtioPCIDeviceTypeInfo virtio_gpu_pci_info = {
     .instance_size = sizeof(VirtIOGPUPCI),
     .instance_init = virtio_gpu_initfn,
 };
+module_obj(TYPE_VIRTIO_GPU_PCI);
 
 static void virtio_gpu_pci_register_types(void)
 {
diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index e183f4ecdaa5..6b7f643951fe 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -1427,6 +1427,7 @@ static const TypeInfo virtio_gpu_info = {
     .class_size = sizeof(VirtIOGPUClass),
     .class_init = virtio_gpu_class_init,
 };
+module_obj(TYPE_VIRTIO_GPU);
 
 static void virtio_register_types(void)
 {
diff --git a/hw/display/virtio-vga-gl.c b/hw/display/virtio-vga-gl.c
index c971340ebb1a..f22549097c5e 100644
--- a/hw/display/virtio-vga-gl.c
+++ b/hw/display/virtio-vga-gl.c
@@ -36,6 +36,7 @@ static VirtioPCIDeviceTypeInfo virtio_vga_gl_info = {
     .instance_size = sizeof(VirtIOVGAGL),
     .instance_init = virtio_vga_gl_inst_initfn,
 };
+module_obj(TYPE_VIRTIO_VGA_GL);
 
 static void virtio_vga_register_types(void)
 {
@@ -45,3 +46,5 @@ static void virtio_vga_register_types(void)
 }
 
 type_init(virtio_vga_register_types)
+
+module_dep("hw-display-virtio-vga");
diff --git a/hw/display/virtio-vga.c b/hw/display/virtio-vga.c
index d3c640406152..9e57f61e9edb 100644
--- a/hw/display/virtio-vga.c
+++ b/hw/display/virtio-vga.c
@@ -239,6 +239,7 @@ static TypeInfo virtio_vga_base_info = {
     .class_init    = virtio_vga_base_class_init,
     .abstract      = true,
 };
+module_obj(TYPE_VIRTIO_VGA_BASE);
 
 #define TYPE_VIRTIO_VGA "virtio-vga"
 
@@ -268,6 +269,7 @@ static VirtioPCIDeviceTypeInfo virtio_vga_info = {
     .instance_size = sizeof(VirtIOVGA),
     .instance_init = virtio_vga_inst_initfn,
 };
+module_obj(TYPE_VIRTIO_VGA);
 
 static void virtio_vga_register_types(void)
 {
-- 
2.31.1

[PATCH v4 07/34] modules: add chardev module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 chardev/baum.c  | 1 +
 chardev/spice.c | 4 ++++
 2 files changed, 5 insertions(+)

diff --git a/chardev/baum.c b/chardev/baum.c
index 5deca778bc44..79d618e35045 100644
--- a/chardev/baum.c
+++ b/chardev/baum.c
@@ -680,6 +680,7 @@ static const TypeInfo char_braille_type_info = {
     .instance_finalize = char_braille_finalize,
     .class_init = char_braille_class_init,
 };
+module_obj(TYPE_CHARDEV_BRAILLE);
 
 static void register_types(void)
 {
diff --git a/chardev/spice.c b/chardev/spice.c
index 1104426e3a11..3ffb3fdc0dac 100644
--- a/chardev/spice.c
+++ b/chardev/spice.c
@@ -366,6 +366,7 @@ static const TypeInfo char_spice_type_info = {
     .class_init = char_spice_class_init,
     .abstract = true,
 };
+module_obj(TYPE_CHARDEV_SPICE);
 
 static void char_spicevmc_class_init(ObjectClass *oc, void *data)
 {
@@ -396,6 +397,7 @@ static const TypeInfo char_spiceport_type_info = {
     .parent = TYPE_CHARDEV_SPICE,
     .class_init = char_spiceport_class_init,
 };
+module_obj(TYPE_CHARDEV_SPICEPORT);
 
 static void register_types(void)
 {
@@ -405,3 +407,5 @@ static void register_types(void)
 }
 
 type_init(register_types);
+
+module_dep("ui-spice-core");
-- 
2.31.1

[PATCH v4 08/34] modules: add audio module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 audio/spiceaudio.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/audio/spiceaudio.c b/audio/spiceaudio.c
index 999bfbde47c5..a8d370fe6f31 100644
--- a/audio/spiceaudio.c
+++ b/audio/spiceaudio.c
@@ -317,3 +317,5 @@ static void register_audio_spice(void)
     audio_driver_register(&spice_audio_driver);
 }
 type_init(register_audio_spice);
+
+module_dep("ui-spice-core");
-- 
2.31.1

[PATCH v4 09/34] modules: add usb-redir module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/usb/redirect.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/usb/redirect.c b/hw/usb/redirect.c
index 6a75b0dc4ab2..4ec9326e0582 100644
--- a/hw/usb/redirect.c
+++ b/hw/usb/redirect.c
@@ -2608,6 +2608,7 @@ static const TypeInfo usbredir_dev_info = {
     .class_init    = usbredir_class_initfn,
     .instance_init = usbredir_instance_init,
 };
+module_obj(TYPE_USB_REDIR);
 
 static void usbredir_register_types(void)
 {
-- 
2.31.1

[PATCH v4 10/34] modules: add ccid module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/usb/ccid-card-emulated.c | 1 +
 hw/usb/ccid-card-passthru.c | 1 +
 2 files changed, 2 insertions(+)

diff --git a/hw/usb/ccid-card-emulated.c b/hw/usb/ccid-card-emulated.c
index 5c76bed77aa0..6c8c0355e099 100644
--- a/hw/usb/ccid-card-emulated.c
+++ b/hw/usb/ccid-card-emulated.c
@@ -612,6 +612,7 @@ static const TypeInfo emulated_card_info = {
     .instance_size = sizeof(EmulatedState),
     .class_init    = emulated_class_initfn,
 };
+module_obj(TYPE_EMULATED_CCID);
 
 static void ccid_card_emulated_register_types(void)
 {
diff --git a/hw/usb/ccid-card-passthru.c b/hw/usb/ccid-card-passthru.c
index 7212d0d7fb5e..fa3040fb7154 100644
--- a/hw/usb/ccid-card-passthru.c
+++ b/hw/usb/ccid-card-passthru.c
@@ -414,6 +414,7 @@ static const TypeInfo passthru_card_info = {
     .instance_size = sizeof(PassthruState),
     .class_init    = passthru_class_initfn,
 };
+module_obj(TYPE_CCID_PASSTHRU);
 
 static void ccid_card_passthru_register_types(void)
 {
-- 
2.31.1

[PATCH v4 11/34] modules: add ui module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 ui/egl-headless.c | 4 ++++
 ui/gtk.c          | 4 ++++
 ui/sdl2.c         | 4 ++++
 ui/spice-app.c    | 3 +++
 ui/spice-core.c   | 5 +++++
 5 files changed, 20 insertions(+)

diff --git a/ui/egl-headless.c b/ui/egl-headless.c
index da377a74af69..75404e0e8700 100644
--- a/ui/egl-headless.c
+++ b/ui/egl-headless.c
@@ -213,3 +213,7 @@ static void register_egl(void)
 }
 
 type_init(register_egl);
+
+#ifdef CONFIG_OPENGL
+module_dep("ui-opengl");
+#endif
diff --git a/ui/gtk.c b/ui/gtk.c
index 98046f577b9d..376b4d528daa 100644
--- a/ui/gtk.c
+++ b/ui/gtk.c
@@ -2333,3 +2333,7 @@ static void register_gtk(void)
 }
 
 type_init(register_gtk);
+
+#ifdef CONFIG_OPENGL
+module_dep("ui-opengl");
+#endif
diff --git a/ui/sdl2.c b/ui/sdl2.c
index a203cb0239e1..36d9010cb6c1 100644
--- a/ui/sdl2.c
+++ b/ui/sdl2.c
@@ -918,3 +918,7 @@ static void register_sdl1(void)
 }
 
 type_init(register_sdl1);
+
+#ifdef CONFIG_OPENGL
+module_dep("ui-opengl");
+#endif
diff --git a/ui/spice-app.c b/ui/spice-app.c
index 4325ac2d9c54..641f4a9d53e3 100644
--- a/ui/spice-app.c
+++ b/ui/spice-app.c
@@ -221,3 +221,6 @@ static void register_spice_app(void)
 }
 
 type_init(register_spice_app);
+
+module_dep("ui-spice-core");
+module_dep("chardev-spice");
diff --git a/ui/spice-core.c b/ui/spice-core.c
index 272d19b0c152..86d43783acac 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -1037,3 +1037,8 @@ static void spice_register_config(void)
     qemu_add_opts(&qemu_spice_opts);
 }
 opts_init(spice_register_config);
+module_opts("spice");
+
+#ifdef CONFIG_OPENGL
+module_dep("ui-opengl");
+#endif
-- 
2.31.1

[PATCH v4 12/34] modules: add s390x module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/s390x/virtio-ccw-gpu.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/hw/s390x/virtio-ccw-gpu.c b/hw/s390x/virtio-ccw-gpu.c
index 75a9e4bb3908..5868a2a07093 100644
--- a/hw/s390x/virtio-ccw-gpu.c
+++ b/hw/s390x/virtio-ccw-gpu.c
@@ -59,6 +59,7 @@ static const TypeInfo virtio_ccw_gpu = {
     .instance_init = virtio_ccw_gpu_instance_init,
     .class_init    = virtio_ccw_gpu_class_init,
 };
+module_obj(TYPE_VIRTIO_GPU_CCW);
 
 static void virtio_ccw_gpu_register(void)
 {
@@ -68,3 +69,5 @@ static void virtio_ccw_gpu_register(void)
 }
 
 type_init(virtio_ccw_gpu_register)
+
+module_arch("s390x");
-- 
2.31.1

[PATCH v4 13/34] modules: add block module annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 block/iscsi-opts.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/block/iscsi-opts.c b/block/iscsi-opts.c
index afaf8837d6c1..4f2da405e645 100644
--- a/block/iscsi-opts.c
+++ b/block/iscsi-opts.c
@@ -68,3 +68,4 @@ static void iscsi_block_opts_init(void)
 }
 
 block_init(iscsi_block_opts_init);
+module_opts("iscsi");
-- 
2.31.1

[PATCH v4 14/34] modules: use modinfo for dependencies

[Gerd Hoffmann]

Use module database for module dependencies.
Drop hard-coded dependency list.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
 util/module.c | 55 ++++++++++++++++++++-------------------------------
 1 file changed, 21 insertions(+), 34 deletions(-)

diff --git a/util/module.c b/util/module.c
index 8d3e8275b9f7..7d7b69cbdaca 100644
--- a/util/module.c
+++ b/util/module.c
@@ -182,28 +182,6 @@ static int module_load_file(const char *fname, bool mayfail, bool export_symbols
 out:
     return ret;
 }
-
-static const struct {
-    const char *name;
-    const char *dep;
-} module_deps[] = {
-    { "audio-spice",    "ui-spice-core" },
-    { "chardev-spice",  "ui-spice-core" },
-    { "hw-display-qxl", "ui-spice-core" },
-    { "ui-spice-app",   "ui-spice-core" },
-    { "ui-spice-app",   "chardev-spice" },
-
-    { "hw-display-virtio-gpu-gl", "hw-display-virtio-gpu" },
-    { "hw-display-virtio-gpu-pci-gl", "hw-display-virtio-gpu-pci" },
-    { "hw-display-virtio-vga-gl", "hw-display-virtio-vga" },
-
-#ifdef CONFIG_OPENGL
-    { "ui-egl-headless", "ui-opengl"    },
-    { "ui-gtk",          "ui-opengl"    },
-    { "ui-sdl",          "ui-opengl"    },
-    { "ui-spice-core",   "ui-opengl"    },
-#endif
-};
 #endif
 
 bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
@@ -219,9 +197,11 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
     char *dirs[5];
     char *module_name;
     int i = 0, n_dirs = 0;
-    int ret, dep;
+    int ret;
     bool export_symbols = false;
     static GHashTable *loaded_modules;
+    const QemuModinfo *modinfo;
+    const char **sl;
 
     if (!g_module_supported()) {
         fprintf(stderr, "Module is not supported by system.\n");
@@ -234,23 +214,30 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
 
     module_name = g_strdup_printf("%s%s", prefix, lib_name);
 
-    for (dep = 0; dep < ARRAY_SIZE(module_deps); dep++) {
-        if (strcmp(module_name, module_deps[dep].name) == 0) {
-            /* we depend on another module */
-            module_load_one("", module_deps[dep].dep, false);
-        }
-        if (strcmp(module_name, module_deps[dep].dep) == 0) {
-            /* another module depends on us */
-            export_symbols = true;
-        }
-    }
-
     if (g_hash_table_contains(loaded_modules, module_name)) {
         g_free(module_name);
         return true;
     }
     g_hash_table_add(loaded_modules, module_name);
 
+    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (modinfo->deps) {
+            if (strcmp(modinfo->name, module_name) == 0) {
+                /* we depend on other module(s) */
+                for (sl = modinfo->deps; *sl != NULL; sl++) {
+                    module_load_one("", *sl, false);
+                }
+            } else {
+                for (sl = modinfo->deps; *sl != NULL; sl++) {
+                    if (strcmp(module_name, *sl) == 0) {
+                        /* another module depends on us */
+                        export_symbols = true;
+                    }
+                }
+            }
+        }
+    }
+
     search_dir = getenv("QEMU_MODULE_DIR");
     if (search_dir != NULL) {
         dirs[n_dirs++] = g_strdup_printf("%s", search_dir);
-- 
2.31.1

[PATCH v4 15/34] modules: use modinfo for qom load

[Gerd Hoffmann]

Use module database to figure which module implements a given QOM type.
Drop hard-coded object list.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
 util/module.c | 77 ++++++++++++++++-----------------------------------
 1 file changed, 24 insertions(+), 53 deletions(-)

diff --git a/util/module.c b/util/module.c
index 7d7b69cbdaca..745ae0fb20ed 100644
--- a/util/module.c
+++ b/util/module.c
@@ -280,80 +280,51 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
     return success;
 }
 
-/*
- * Building devices and other qom objects modular is mostly useful in
- * case they have dependencies to external shared libraries, so we can
- * cut down the core qemu library dependencies.  Which is the case for
- * only a very few devices & objects.
- *
- * So with the expectation that this will be rather the exception than
- * the rule and the list will not gain that many entries, go with a
- * simple manually maintained list for now.
- *
- * The list must be sorted by module (module_load_qom_all() needs this).
- */
-static struct {
-    const char *type;
-    const char *prefix;
-    const char *module;
-} const qom_modules[] = {
-    { "ccid-card-passthru",    "hw-", "usb-smartcard"         },
-    { "ccid-card-emulated",    "hw-", "usb-smartcard"         },
-    { "usb-redir",             "hw-", "usb-redirect"          },
-    { "qxl-vga",               "hw-", "display-qxl"           },
-    { "qxl",                   "hw-", "display-qxl"           },
-    { "virtio-gpu-device",     "hw-", "display-virtio-gpu"    },
-    { "virtio-gpu-gl-device",  "hw-", "display-virtio-gpu-gl" },
-    { "vhost-user-gpu",        "hw-", "display-virtio-gpu"    },
-    { "virtio-gpu-pci-base",   "hw-", "display-virtio-gpu-pci" },
-    { "virtio-gpu-pci",        "hw-", "display-virtio-gpu-pci" },
-    { "virtio-gpu-gl-pci",     "hw-", "display-virtio-gpu-pci-gl" },
-    { "vhost-user-gpu-pci",    "hw-", "display-virtio-gpu-pci" },
-    { "virtio-gpu-ccw",        "hw-", "s390x-virtio-gpu-ccw"   },
-    { "virtio-vga-base",       "hw-", "display-virtio-vga"    },
-    { "virtio-vga",            "hw-", "display-virtio-vga"    },
-    { "virtio-vga-gl",         "hw-", "display-virtio-vga-gl" },
-    { "vhost-user-vga",        "hw-", "display-virtio-vga"    },
-    { "chardev-braille",       "chardev-", "baum"             },
-    { "chardev-spicevmc",      "chardev-", "spice"            },
-    { "chardev-spiceport",     "chardev-", "spice"            },
-};
+#ifdef CONFIG_MODULES
 
 static bool module_loaded_qom_all;
 
 void module_load_qom_one(const char *type)
 {
-    int i;
+    const QemuModinfo *modinfo;
+    const char **sl;
 
     if (!type) {
         return;
     }
-    for (i = 0; i < ARRAY_SIZE(qom_modules); i++) {
-        if (strcmp(qom_modules[i].type, type) == 0) {
-            module_load_one(qom_modules[i].prefix,
-                            qom_modules[i].module,
-                            false);
-            return;
+
+    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (!modinfo->objs) {
+            continue;
+        }
+        for (sl = modinfo->objs; *sl != NULL; sl++) {
+            if (strcmp(type, *sl) == 0) {
+                module_load_one("", modinfo->name, false);
+            }
         }
     }
 }
 
 void module_load_qom_all(void)
 {
-    int i;
+    const QemuModinfo *modinfo;
 
     if (module_loaded_qom_all) {
         return;
     }
-    for (i = 0; i < ARRAY_SIZE(qom_modules); i++) {
-        if (i > 0 && (strcmp(qom_modules[i - 1].module,
-                             qom_modules[i].module) == 0 &&
-                      strcmp(qom_modules[i - 1].prefix,
-                             qom_modules[i].prefix) == 0)) {
-            /* one module implementing multiple types -> load only once */
+
+    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (!modinfo->objs) {
             continue;
         }
-        module_load_one(qom_modules[i].prefix, qom_modules[i].module, true);
+        module_load_one("", modinfo->name, false);
     }
     module_loaded_qom_all = true;
 }
+
+#else
+
+void module_load_qom_one(const char *type) {}
+void module_load_qom_all(void) {}
+
+#endif
-- 
2.31.1

[PATCH v4 16/34] modules: use modinfo for qemu opts load

[Gerd Hoffmann]

Use module database to figure which module adds given QemuOpts group.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
 softmmu/vl.c        | 17 -----------------
 stubs/module-opts.c |  4 ----
 util/module.c       | 19 +++++++++++++++++++
 3 files changed, 19 insertions(+), 21 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index e001505d79f7..5c26e80126db 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2675,23 +2675,6 @@ void qmp_x_exit_preconfig(Error **errp)
     }
 }
 
-#ifdef CONFIG_MODULES
-void qemu_load_module_for_opts(const char *group)
-{
-    static bool spice_tried;
-    if (g_str_equal(group, "spice") && !spice_tried) {
-        ui_module_load_one("spice-core");
-        spice_tried = true;
-    }
-
-    static bool iscsi_tried;
-    if (g_str_equal(group, "iscsi") && !iscsi_tried) {
-        block_module_load_one("iscsi");
-        iscsi_tried = true;
-    }
-}
-#endif
-
 void qemu_init(int argc, char **argv, char **envp)
 {
     QemuOpts *opts;
diff --git a/stubs/module-opts.c b/stubs/module-opts.c
index a7d0e4ad6ead..5412429ea869 100644
--- a/stubs/module-opts.c
+++ b/stubs/module-opts.c
@@ -1,6 +1,2 @@
 #include "qemu/osdep.h"
 #include "qemu/config-file.h"
-
-void qemu_load_module_for_opts(const char *group)
-{
-}
diff --git a/util/module.c b/util/module.c
index 745ae0fb20ed..a9ec2da9972e 100644
--- a/util/module.c
+++ b/util/module.c
@@ -20,6 +20,7 @@
 #include "qemu/queue.h"
 #include "qemu/module.h"
 #include "qemu/cutils.h"
+#include "qemu/config-file.h"
 #ifdef CONFIG_MODULE_UPGRADES
 #include "qemu-version.h"
 #endif
@@ -322,8 +323,26 @@ void module_load_qom_all(void)
     module_loaded_qom_all = true;
 }
 
+void qemu_load_module_for_opts(const char *group)
+{
+    const QemuModinfo *modinfo;
+    const char **sl;
+
+    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (!modinfo->opts) {
+            continue;
+        }
+        for (sl = modinfo->opts; *sl != NULL; sl++) {
+            if (strcmp(group, *sl) == 0) {
+                module_load_one("", modinfo->name, false);
+            }
+        }
+    }
+}
+
 #else
 
+void qemu_load_module_for_opts(const char *group) {}
 void module_load_qom_one(const char *type) {}
 void module_load_qom_all(void) {}
 
-- 
2.31.1

[PATCH v4 17/34] modules: add tracepoints

[Gerd Hoffmann]

One for module load and one for qom type lookup.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 util/module.c     | 3 +++
 util/trace-events | 4 ++++
 2 files changed, 7 insertions(+)

diff --git a/util/module.c b/util/module.c
index a9ec2da9972e..acaaecad56c9 100644
--- a/util/module.c
+++ b/util/module.c
@@ -24,6 +24,7 @@
 #ifdef CONFIG_MODULE_UPGRADES
 #include "qemu-version.h"
 #endif
+#include "trace.h"
 
 typedef struct ModuleEntry
 {
@@ -176,6 +177,7 @@ static int module_load_file(const char *fname, bool mayfail, bool export_symbols
         ret = 0;
     }
 
+    trace_module_load_module(fname);
     QTAILQ_FOREACH_SAFE(e, &dso_init_list, node, next) {
         QTAILQ_REMOVE(&dso_init_list, e, node);
         g_free(e);
@@ -294,6 +296,7 @@ void module_load_qom_one(const char *type)
         return;
     }
 
+    trace_module_lookup_object_type(type);
     for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
         if (!modinfo->objs) {
             continue;
diff --git a/util/trace-events b/util/trace-events
index 806cac14a762..c8f53d7d9fc3 100644
--- a/util/trace-events
+++ b/util/trace-events
@@ -100,3 +100,7 @@ uffd_create_fd_api_failed(int err) "errno: %i"
 uffd_create_fd_api_noioctl(uint64_t ioctl_req, uint64_t ioctl_supp) "ioctl_req: 0x%" PRIx64 "ioctl_supp: 0x%" PRIx64
 uffd_register_memory_failed(void *addr, uint64_t length, uint64_t mode, int err) "addr: %p length: %" PRIu64 " mode: 0x%" PRIx64 " errno: %i"
 uffd_unregister_memory_failed(void *addr, uint64_t length, int err) "addr: %p length: %" PRIu64 " errno: %i"
+
+# module.c
+module_load_module(const char *name) "file %s"
+module_lookup_object_type(const char *name) "name %s"
-- 
2.31.1

[PATCH v4 18/34] modules: check arch and block load on mismatch

[Gerd Hoffmann]

Add module_allow_arch() to set the target architecture.
In case a module is limited to some arch verify arches
match and ignore the module if not.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 include/qemu/module.h |  1 +
 softmmu/vl.c          |  3 +++
 util/module.c         | 29 +++++++++++++++++++++++++++++
 3 files changed, 33 insertions(+)

diff --git a/include/qemu/module.h b/include/qemu/module.h
index a98748d501d3..7f4b1af8198c 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -72,6 +72,7 @@ void module_call_init(module_init_type type);
 bool module_load_one(const char *prefix, const char *lib_name, bool mayfail);
 void module_load_qom_one(const char *type);
 void module_load_qom_all(void);
+void module_allow_arch(const char *arch);
 
 /*
  * module info annotation macros
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 5c26e80126db..a70e7b4658e8 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -126,6 +126,8 @@
 #include "sysemu/iothread.h"
 #include "qemu/guest-random.h"
 
+#include "config-host.h"
+
 #define MAX_VIRTIO_CONSOLES 1
 
 typedef struct BlockdevOptionsQueueEntry {
@@ -2725,6 +2727,7 @@ void qemu_init(int argc, char **argv, char **envp)
 
 #ifdef CONFIG_MODULES
     module_init_info(qemu_modinfo);
+    module_allow_arch(TARGET_NAME);
 #endif
 
     qemu_init_subsystems();
diff --git a/util/module.c b/util/module.c
index acaaecad56c9..065aed09ffef 100644
--- a/util/module.c
+++ b/util/module.c
@@ -117,12 +117,33 @@ static const QemuModinfo module_info_stub[] = { {
     /* end of list */
 } };
 static const QemuModinfo *module_info = module_info_stub;
+static const char *module_arch;
 
 void module_init_info(const QemuModinfo *info)
 {
     module_info = info;
 }
 
+void module_allow_arch(const char *arch)
+{
+    module_arch = arch;
+}
+
+static bool module_check_arch(const QemuModinfo *modinfo)
+{
+    if (modinfo->arch) {
+        if (!module_arch) {
+            /* no arch set -> ignore all */
+            return false;
+        }
+        if (strcmp(module_arch, modinfo->arch) != 0) {
+            /* mismatch */
+            return false;
+        }
+    }
+    return true;
+}
+
 static int module_load_file(const char *fname, bool mayfail, bool export_symbols)
 {
     GModule *g_module;
@@ -224,6 +245,13 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
     g_hash_table_add(loaded_modules, module_name);
 
     for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (modinfo->arch) {
+            if (strcmp(modinfo->name, module_name) == 0) {
+                if (!module_check_arch(modinfo)) {
+                    return false;
+                }
+            }
+        }
         if (modinfo->deps) {
             if (strcmp(modinfo->name, module_name) == 0) {
                 /* we depend on other module(s) */
@@ -345,6 +373,7 @@ void qemu_load_module_for_opts(const char *group)
 
 #else
 
+void module_allow_arch(const char *arch) {}
 void qemu_load_module_for_opts(const char *group) {}
 void module_load_qom_one(const char *type) {}
 void module_load_qom_all(void) {}
-- 
2.31.1

[PATCH v4 19/34] modules: check arch on qom lookup

[Gerd Hoffmann]

With target-specific modules we can have multiple modules implementing
the same object.  Therefore we have to check the target arch on lookup
to find the correct module.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 util/module.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/util/module.c b/util/module.c
index 065aed09ffef..6bb4ad915a1c 100644
--- a/util/module.c
+++ b/util/module.c
@@ -329,6 +329,9 @@ void module_load_qom_one(const char *type)
         if (!modinfo->objs) {
             continue;
         }
+        if (!module_check_arch(modinfo)) {
+            continue;
+        }
         for (sl = modinfo->objs; *sl != NULL; sl++) {
             if (strcmp(type, *sl) == 0) {
                 module_load_one("", modinfo->name, false);
@@ -349,6 +352,9 @@ void module_load_qom_all(void)
         if (!modinfo->objs) {
             continue;
         }
+        if (!module_check_arch(modinfo)) {
+            continue;
+        }
         module_load_one("", modinfo->name, false);
     }
     module_loaded_qom_all = true;
-- 
2.31.1

[PATCH v4 20/34] modules: target-specific module build infrastructure

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 meson.build | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/meson.build b/meson.build
index 6ae2b9ab9443..76020b43fb32 100644
--- a/meson.build
+++ b/meson.build
@@ -1781,6 +1781,7 @@ user_ss = ss.source_set()
 util_ss = ss.source_set()
 
 modules = {}
+target_modules = {}
 hw_arch = {}
 target_arch = {}
 target_softmmu_arch = {}
@@ -2060,6 +2061,42 @@ foreach d, list : modules
   endforeach
 endforeach
 
+foreach d, list : target_modules
+  foreach m, module_ss : list
+    if enable_modules and targetos != 'windows'
+      foreach target : target_dirs
+        if target.endswith('-softmmu')
+          config_target = config_target_mak[target]
+          config_target += config_host
+          target_inc = [include_directories('target' / config_target['TARGET_BASE_ARCH'])]
+          c_args = ['-DNEED_CPU_H',
+                    '-DCONFIG_TARGET="@0@-config-target.h"'.format(target),
+                    '-DCONFIG_DEVICES="@0@-config-devices.h"'.format(target)]
+          target_module_ss = module_ss.apply(config_target, strict: false)
+          if target_module_ss.sources() != []
+            module_name = d + '-' + m + '-' + config_target['TARGET_NAME']
+            sl = static_library(module_name,
+                                [genh, target_module_ss.sources()],
+                                dependencies: [modulecommon, target_module_ss.dependencies()],
+                                include_directories: target_inc,
+                                c_args: c_args,
+                                pic: true)
+            softmmu_mods += sl
+            # FIXME: Should use sl.extract_all_objects(recursive: true) too.
+            modinfo_files += custom_target(module_name + '.modinfo',
+                                           output: module_name + '.modinfo',
+                                           input: target_module_ss.sources(),
+                                           capture: true,
+                                           command: [modinfo_collect, '--target', target, '@INPUT@'])
+          endif
+        endif
+      endforeach
+    else
+      specific_ss.add_all(module_ss)
+    endif
+  endforeach
+endforeach
+
 if enable_modules
   modinfo_src = custom_target('modinfo.c',
                               output: 'modinfo.c',
-- 
2.31.1

[PATCH v4 21/34] modules: add documentation for module sourcesets

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/devel/build-system.rst | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst
index 7ef36f42d0f5..fd1650442ecc 100644
--- a/docs/devel/build-system.rst
+++ b/docs/devel/build-system.rst
@@ -272,6 +272,23 @@ Target-dependent emulator sourcesets:
     target_arch += {'arm': arm_ss}
     target_softmmu_arch += {'arm': arm_softmmu_ss}
 
+Module sourcesets:
+  There are two dictionaries for modules: `modules` is used for
+  target-independent modules and `target_modules` is used for
+  target-dependent modules.  When modules are disabled the `module`
+  source sets are added to `softmmu_ss` and the `target_modules`
+  source sets are added to `specific_ss`.
+
+  Both dictionaries are nested.  One dictionary is created per
+  subdirectory, and these per-subdirectory dictionaries are added to
+  the toplevel dictionaries.  For example::
+
+    hw_display_modules = {}
+    qxl_ss = ss.source_set()
+    ...
+    hw_display_modules += { 'qxl': qxl_ss }
+    modules += { 'hw-display': hw_display_modules }
+
 Utility sourcesets:
   All binaries link with a static library `libqemuutil.a`.  This library
   is built from several sourcesets; most of them however host generated
-- 
2.31.1

[PATCH v4 22/34] modules: add module_obj() note to QOM docs

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/devel/qom.rst | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/docs/devel/qom.rst b/docs/devel/qom.rst
index 42d0dc4f4da8..e5fe3597cd82 100644
--- a/docs/devel/qom.rst
+++ b/docs/devel/qom.rst
@@ -87,6 +87,14 @@ specific type:
    #define MY_DEVICE(obj) \
       OBJECT_CHECK(MyDevice, obj, TYPE_MY_DEVICE)
 
+In case the ObjectClass implementation can be built as module a
+module_obj() line must be added to make sure qemu loads the module
+when the object is needed.
+
+.. code-block:: c
+
+   module_obj(TYPE_MY_DEVICE);
+
 Class Initialization
 ====================
 
-- 
2.31.1

[PATCH v4 23/34] modules: module.h kerneldoc annotations

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
 include/qemu/module.h | 59 +++++++++++++++++++++++++++++++++----------
 1 file changed, 45 insertions(+), 14 deletions(-)

diff --git a/include/qemu/module.h b/include/qemu/module.h
index 7f4b1af8198c..8bc80535a4d4 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -74,11 +74,18 @@ void module_load_qom_one(const char *type);
 void module_load_qom_all(void);
 void module_allow_arch(const char *arch);
 
-/*
- * module info annotation macros
+/**
+ * DOC: module info annotation macros
  *
- * scripts/modinfo-collect.py will collect module info,
- * using the preprocessor and -DQEMU_MODINFO
+ * `scripts/modinfo-collect.py` will collect module info,
+ * using the preprocessor and -DQEMU_MODINFO.
+ *
+ * `scripts/modinfo-generate.py` will create a module meta-data database
+ * from the collected information so qemu knows about module
+ * dependencies and QOM objects implemented by modules.
+ *
+ * See `*.modinfo` and `modinfo.c` in the build directory to check the
+ * script results.
  */
 #ifdef QEMU_MODINFO
 # define modinfo(kind, value) \
@@ -87,24 +94,48 @@ void module_allow_arch(const char *arch);
 # define modinfo(kind, value)
 #endif
 
-/* module implements QOM type <name> */
+/**
+ * module_obj
+ *
+ * @name: QOM type.
+ *
+ * This module implements QOM type @name.
+ */
 #define module_obj(name) modinfo(obj, name)
 
-/* module has a dependency on <name> */
+/**
+ * module_dep
+ *
+ * @name: module name
+ *
+ * This module depends on module @name.
+ */
 #define module_dep(name) modinfo(dep, name)
 
-/* module is for target architecture <name> */
+/**
+ * module_arch
+ *
+ * @arch: target architecture
+ *
+ * This module is for target architecture @arch.
+ *
+ * Note that target-dependent modules are tagged automatically, so
+ * this is only needed in case target-independent modules should be
+ * restricted.  Use case example: the ccw bus is implemented by s390x
+ * only.
+ */
 #define module_arch(name) modinfo(arch, name)
 
-/* module registers QemuOpts <name> */
+/**
+ * module_opts
+ *
+ * @name: QemuOpts name
+ *
+ * This module registers QemuOpts @name.
+ */
 #define module_opts(name) modinfo(opts, name)
 
-/*
- * module info database
- *
- * scripts/modinfo-generate.c will build this using the data collected
- * by scripts/modinfo-collect.py
- */
+/* module info database (created by scripts/modinfo-generate.py) */
 typedef struct QemuModinfo QemuModinfo;
 struct QemuModinfo {
     const char *name;
-- 
2.31.1

[PATCH v4 24/34] modules: hook up modules.h to docs build

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/devel/index.rst   | 1 +
 docs/devel/modules.rst | 5 +++++
 2 files changed, 6 insertions(+)
 create mode 100644 docs/devel/modules.rst

diff --git a/docs/devel/index.rst b/docs/devel/index.rst
index 977c3893bdaf..ba90badbbd6d 100644
--- a/docs/devel/index.rst
+++ b/docs/devel/index.rst
@@ -41,6 +41,7 @@ Contents:
    s390-dasd-ipl
    clocks
    qom
+   modules
    block-coroutine-wrapper
    multi-process
    ebpf_rss
diff --git a/docs/devel/modules.rst b/docs/devel/modules.rst
new file mode 100644
index 000000000000..066f347b89ba
--- /dev/null
+++ b/docs/devel/modules.rst
@@ -0,0 +1,5 @@
+============
+Qemu modules
+============
+
+.. kernel-doc:: include/qemu/module.h
-- 
2.31.1

[PATCH v4 25/34] accel: autoload modules

[Gerd Hoffmann]

Call module_object_class_by_name() instead of object_class_by_name()
for objects possibly implemented as module

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 accel/accel-common.c  | 2 +-
 accel/accel-softmmu.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/accel/accel-common.c b/accel/accel-common.c
index cf07f78421d6..7b8ec7e0f72a 100644
--- a/accel/accel-common.c
+++ b/accel/accel-common.c
@@ -44,7 +44,7 @@ static const TypeInfo accel_type = {
 AccelClass *accel_find(const char *opt_name)
 {
     char *class_name = g_strdup_printf(ACCEL_CLASS_NAME("%s"), opt_name);
-    AccelClass *ac = ACCEL_CLASS(object_class_by_name(class_name));
+    AccelClass *ac = ACCEL_CLASS(module_object_class_by_name(class_name));
     g_free(class_name);
     return ac;
 }
diff --git a/accel/accel-softmmu.c b/accel/accel-softmmu.c
index 50fa5acaa401..67276e4f5222 100644
--- a/accel/accel-softmmu.c
+++ b/accel/accel-softmmu.c
@@ -72,7 +72,7 @@ void accel_init_ops_interfaces(AccelClass *ac)
     g_assert(ac_name != NULL);
 
     ops_name = g_strdup_printf("%s" ACCEL_OPS_SUFFIX, ac_name);
-    ops = ACCEL_OPS_CLASS(object_class_by_name(ops_name));
+    ops = ACCEL_OPS_CLASS(module_object_class_by_name(ops_name));
     g_free(ops_name);
 
     /*
-- 
2.31.1

[PATCH v4 26/34] accel: add qtest module annotations

[Gerd Hoffmann]

Add module annotations for qtest so autoloading works.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 accel/qtest/qtest.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/accel/qtest/qtest.c b/accel/qtest/qtest.c
index edb29f6fa4c0..7e6b8110d52b 100644
--- a/accel/qtest/qtest.c
+++ b/accel/qtest/qtest.c
@@ -45,6 +45,7 @@ static const TypeInfo qtest_accel_type = {
     .parent = TYPE_ACCEL,
     .class_init = qtest_accel_class_init,
 };
+module_obj(TYPE_QTEST_ACCEL);
 
 static void qtest_accel_ops_class_init(ObjectClass *oc, void *data)
 {
@@ -61,6 +62,7 @@ static const TypeInfo qtest_accel_ops_type = {
     .class_init = qtest_accel_ops_class_init,
     .abstract = true,
 };
+module_obj(ACCEL_OPS_NAME("qtest"));
 
 static void qtest_type_init(void)
 {
-- 
2.31.1

[PATCH v4 27/34] accel: build qtest modular

[Gerd Hoffmann]

Allow building accelerators as module.
Start with qtest as first user.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 accel/qtest/meson.build | 8 ++------
 meson.build             | 6 ++++++
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/accel/qtest/meson.build b/accel/qtest/meson.build
index a2f327645980..4c6560029336 100644
--- a/accel/qtest/meson.build
+++ b/accel/qtest/meson.build
@@ -1,6 +1,2 @@
-qtest_ss = ss.source_set()
-qtest_ss.add(files(
-  'qtest.c',
-))
-
-specific_ss.add_all(when: ['CONFIG_SOFTMMU', 'CONFIG_POSIX'], if_true: qtest_ss)
+qtest_module_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_POSIX'],
+                    if_true: files('qtest.c'))
diff --git a/meson.build b/meson.build
index 76020b43fb32..d2cacc145a91 100644
--- a/meson.build
+++ b/meson.build
@@ -1780,6 +1780,9 @@ trace_ss = ss.source_set()
 user_ss = ss.source_set()
 util_ss = ss.source_set()
 
+# accel modules
+qtest_module_ss = ss.source_set()
+
 modules = {}
 target_modules = {}
 hw_arch = {}
@@ -2018,6 +2021,9 @@ specific_ss.add_all(when: 'CONFIG_LINUX_USER', if_true: linux_user_ss)
 subdir('tests/qtest/libqos')
 subdir('tests/qtest/fuzz')
 
+# accel modules
+target_modules += { 'accel' : { 'qtest': qtest_module_ss }}
+
 ########################
 # Library dependencies #
 ########################
-- 
2.31.1

[PATCH v4 28/34] accel: add tcg module annotations

[Gerd Hoffmann]

Add module annotations for tcg so autoloading works.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 accel/tcg/tcg-accel-ops.c | 1 +
 accel/tcg/tcg-all.c       | 1 +
 2 files changed, 2 insertions(+)

diff --git a/accel/tcg/tcg-accel-ops.c b/accel/tcg/tcg-accel-ops.c
index 7191315aeed4..1a8e8390bd60 100644
--- a/accel/tcg/tcg-accel-ops.c
+++ b/accel/tcg/tcg-accel-ops.c
@@ -124,6 +124,7 @@ static const TypeInfo tcg_accel_ops_type = {
     .class_init = tcg_accel_ops_class_init,
     .abstract = true,
 };
+module_obj(ACCEL_OPS_NAME("tcg"));
 
 static void tcg_accel_ops_register_types(void)
 {
diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
index 00803f76d870..d6336a9c966d 100644
--- a/accel/tcg/tcg-all.c
+++ b/accel/tcg/tcg-all.c
@@ -238,6 +238,7 @@ static const TypeInfo tcg_accel_type = {
     .class_init = tcg_accel_class_init,
     .instance_size = sizeof(TCGState),
 };
+module_obj(TYPE_TCG_ACCEL);
 
 static void register_accel_types(void)
 {
-- 
2.31.1

[PATCH v4 29/34] accel: build tcg modular

[Gerd Hoffmann]

Build tcg accel ops as module.
Which is only a small fraction of tcg.
Also only x86 for now.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 accel/tcg/meson.build |  5 ++++-
 meson.build           | 14 +++++++++++++-
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/accel/tcg/meson.build b/accel/tcg/meson.build
index 1236ac7b910b..0ae9180282e3 100644
--- a/accel/tcg/meson.build
+++ b/accel/tcg/meson.build
@@ -15,8 +15,11 @@ specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
 
 specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
   'cputlb.c',
+))
+
+tcg_module_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
   'tcg-accel-ops.c',
   'tcg-accel-ops-mttcg.c',
   'tcg-accel-ops-icount.c',
-  'tcg-accel-ops-rr.c'
+  'tcg-accel-ops-rr.c',
 ))
diff --git a/meson.build b/meson.build
index d2cacc145a91..9f1ca4177073 100644
--- a/meson.build
+++ b/meson.build
@@ -92,6 +92,8 @@ if cpu in ['x86', 'x86_64']
   }
 endif
 
+modular_tcg = ['i386-softmmu', 'x86_64-softmmu']
+
 edk2_targets = [ 'arm-softmmu', 'aarch64-softmmu', 'i386-softmmu', 'x86_64-softmmu' ]
 install_edk2_blobs = false
 if get_option('install_blobs')
@@ -1311,6 +1313,11 @@ foreach target : target_dirs
       elif sym == 'CONFIG_XEN' and have_xen_pci_passthrough
         config_target += { 'CONFIG_XEN_PCI_PASSTHROUGH': 'y' }
       endif
+      if target in modular_tcg
+        config_target += { 'CONFIG_TCG_MODULAR': 'y' }
+      else
+        config_target += { 'CONFIG_TCG_BUILTIN': 'y' }
+      endif
       accel_kconfig += [ sym + '=y' ]
     endif
   endforeach
@@ -1782,6 +1789,7 @@ util_ss = ss.source_set()
 
 # accel modules
 qtest_module_ss = ss.source_set()
+tcg_module_ss = ss.source_set()
 
 modules = {}
 target_modules = {}
@@ -2022,7 +2030,11 @@ subdir('tests/qtest/libqos')
 subdir('tests/qtest/fuzz')
 
 # accel modules
-target_modules += { 'accel' : { 'qtest': qtest_module_ss }}
+tcg_real_module_ss = ss.source_set()
+tcg_real_module_ss.add_all(when: 'CONFIG_TCG_MODULAR', if_true: tcg_module_ss)
+specific_ss.add_all(when: 'CONFIG_TCG_BUILTIN', if_true: tcg_module_ss)
+target_modules += { 'accel' : { 'qtest': qtest_module_ss,
+                                'tcg': tcg_real_module_ss }}
 
 ########################
 # Library dependencies #
-- 
2.31.1

[PATCH v4 30/34] monitor: allow register hmp commands

[Gerd Hoffmann]

Allow commands having a NULL cmd pointer, add a function to set the
pointer later.  Use case: allow modules implement hmp commands.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 include/monitor/monitor.h |  3 +++
 monitor/hmp.c             |  7 +++++++
 monitor/misc.c            | 16 ++++++++++++++++
 3 files changed, 26 insertions(+)

diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 1211d6e6d69f..1a8a369b50b2 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -51,4 +51,7 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags);
 void monitor_fdset_dup_fd_remove(int dup_fd);
 int64_t monitor_fdset_dup_fd_find(int dup_fd);
 
+void monitor_register_hmp(const char *name, bool info,
+                          void (*cmd)(Monitor *mon, const QDict *qdict));
+
 #endif /* MONITOR_H */
diff --git a/monitor/hmp.c b/monitor/hmp.c
index 6c0b33a0b19d..d50c3124e1e1 100644
--- a/monitor/hmp.c
+++ b/monitor/hmp.c
@@ -1089,6 +1089,13 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
         return;
     }
 
+    if (!cmd->cmd) {
+        /* FIXME: is it useful to try autoload modules here ??? */
+        monitor_printf(&mon->common, "Command \"%.*s\" is not available.\n",
+                       (int)(cmdline - cmd_start), cmd_start);
+        return;
+    }
+
     qdict = monitor_parse_arguments(&mon->common, &cmdline, cmd);
     if (!qdict) {
         while (cmdline > cmd_start && qemu_isspace(cmdline[-1])) {
diff --git a/monitor/misc.c b/monitor/misc.c
index 1539e18557f0..ad476c6e51ea 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -1974,6 +1974,22 @@ static void sortcmdlist(void)
           compare_mon_cmd);
 }
 
+void monitor_register_hmp(const char *name, bool info,
+                          void (*cmd)(Monitor *mon, const QDict *qdict))
+{
+    HMPCommand *table = info ? hmp_info_cmds : hmp_cmds;
+
+    while (table->name != NULL) {
+        if (strcmp(table->name, name) == 0) {
+            g_assert(table->cmd == NULL);
+            table->cmd = cmd;
+            return;
+        }
+        table++;
+    }
+    g_assert_not_reached();
+}
+
 void monitor_init_globals(void)
 {
     monitor_init_globals_core();
-- 
2.31.1

Re: [PATCH v4 30/34] monitor: allow register hmp commands

[Dr. David Alan Gilbert]

* Gerd Hoffmann (kraxel@redhat.com) wrote:
> Allow commands having a NULL cmd pointer, add a function to set the
> pointer later.  Use case: allow modules implement hmp commands.
> 
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

So this is OK, so

Acked-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

however, I can imagine:
  a) Auto load as you suggest
  b) Making the error better 'Command ... needs module ....'
  c) Don't actually define the command in the tables at all; make
     the module actually add the command to the table.

Dave

> ---
>  include/monitor/monitor.h |  3 +++
>  monitor/hmp.c             |  7 +++++++
>  monitor/misc.c            | 16 ++++++++++++++++
>  3 files changed, 26 insertions(+)
> 
> diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
> index 1211d6e6d69f..1a8a369b50b2 100644
> --- a/include/monitor/monitor.h
> +++ b/include/monitor/monitor.h
> @@ -51,4 +51,7 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags);
>  void monitor_fdset_dup_fd_remove(int dup_fd);
>  int64_t monitor_fdset_dup_fd_find(int dup_fd);
>  
> +void monitor_register_hmp(const char *name, bool info,
> +                          void (*cmd)(Monitor *mon, const QDict *qdict));
> +
>  #endif /* MONITOR_H */
> diff --git a/monitor/hmp.c b/monitor/hmp.c
> index 6c0b33a0b19d..d50c3124e1e1 100644
> --- a/monitor/hmp.c
> +++ b/monitor/hmp.c
> @@ -1089,6 +1089,13 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
>          return;
>      }
>  
> +    if (!cmd->cmd) {
> +        /* FIXME: is it useful to try autoload modules here ??? */
> +        monitor_printf(&mon->common, "Command \"%.*s\" is not available.\n",
> +                       (int)(cmdline - cmd_start), cmd_start);
> +        return;
> +    }
> +
>      qdict = monitor_parse_arguments(&mon->common, &cmdline, cmd);
>      if (!qdict) {
>          while (cmdline > cmd_start && qemu_isspace(cmdline[-1])) {
> diff --git a/monitor/misc.c b/monitor/misc.c
> index 1539e18557f0..ad476c6e51ea 100644
> --- a/monitor/misc.c
> +++ b/monitor/misc.c
> @@ -1974,6 +1974,22 @@ static void sortcmdlist(void)
>            compare_mon_cmd);
>  }
>  
> +void monitor_register_hmp(const char *name, bool info,
> +                          void (*cmd)(Monitor *mon, const QDict *qdict))
> +{
> +    HMPCommand *table = info ? hmp_info_cmds : hmp_cmds;
> +
> +    while (table->name != NULL) {
> +        if (strcmp(table->name, name) == 0) {
> +            g_assert(table->cmd == NULL);
> +            table->cmd = cmd;
> +            return;
> +        }
> +        table++;
> +    }
> +    g_assert_not_reached();
> +}
> +
>  void monitor_init_globals(void)
>  {
>      monitor_init_globals_core();
> -- 
> 2.31.1
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [PATCH v4 30/34] monitor: allow register hmp commands

[Gerd Hoffmann]

On Thu, Jun 24, 2021 at 03:55:29PM +0100, Dr. David Alan Gilbert wrote:
> * Gerd Hoffmann (kraxel@redhat.com) wrote:
> > Allow commands having a NULL cmd pointer, add a function to set the
> > pointer later.  Use case: allow modules implement hmp commands.
> > 
> > Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> 
> So this is OK, so
> 
> Acked-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> 
> however, I can imagine:
>   a) Auto load as you suggest

Not sure about that.  The tcg monitor commands are pointless when you
picked another accelerator, and "info usbhost" would probably also be
most useful when trouble-shooting usb-host issues.  That's why I left
it as FIXME question for now.  But can certainly be done, we can add
something along the lines of 'module_hmp("info usbhost");' to the
meta-data database and autoload based on that (or use it for more
verbose error messages).

>   c) Don't actually define the command in the tables at all; make
>      the module actually add the command to the table.

Another possible approach.  I don't see a need for modules to expand the
list of commands though, so I only set the function pointer for existing
table entries ...

take care,
  Gerd

[PATCH v4 31/34] usb: drop usb_host_dev_is_scsi_storage hook

[Gerd Hoffmann]

Introduce an usb device flag instead, set it when usb-host looks at the
device descriptors anyway.  Also set it for emulated storage devices,
for consistency.  Add an inline helper function to check the flag.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: David Gibson <david@gibson.dropbear.id.au>
---
 include/hw/usb.h             |  7 ++++++-
 hw/ppc/spapr.c               |  2 +-
 hw/usb/dev-storage-bot.c     |  1 +
 hw/usb/dev-storage-classic.c |  1 +
 hw/usb/dev-uas.c             |  1 +
 hw/usb/host-libusb.c         | 36 +++++++-----------------------------
 hw/usb/host-stub.c           |  5 -----
 7 files changed, 17 insertions(+), 36 deletions(-)

diff --git a/include/hw/usb.h b/include/hw/usb.h
index 436e07b30404..33668dd0a99a 100644
--- a/include/hw/usb.h
+++ b/include/hw/usb.h
@@ -219,6 +219,7 @@ enum USBDeviceFlags {
     USB_DEV_FLAG_IS_HOST,
     USB_DEV_FLAG_MSOS_DESC_ENABLE,
     USB_DEV_FLAG_MSOS_DESC_IN_USE,
+    USB_DEV_FLAG_IS_SCSI_STORAGE,
 };
 
 /* definition of a USB device */
@@ -465,7 +466,6 @@ void usb_generic_async_ctrl_complete(USBDevice *s, USBPacket *p);
 
 /* usb-linux.c */
 void hmp_info_usbhost(Monitor *mon, const QDict *qdict);
-bool usb_host_dev_is_scsi_storage(USBDevice *usbdev);
 
 /* usb ports of the VM */
 
@@ -561,6 +561,11 @@ const char *usb_device_get_product_desc(USBDevice *dev);
 
 const USBDesc *usb_device_get_usb_desc(USBDevice *dev);
 
+static inline bool usb_device_is_scsi_storage(USBDevice *dev)
+{
+    return dev->flags & (1 << USB_DEV_FLAG_IS_SCSI_STORAGE);
+}
+
 /* quirks.c */
 
 /* In bulk endpoints are streaming data sources (iow behave like isoc eps) */
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 4dd90b75cc52..f83a081af0f1 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -3106,7 +3106,7 @@ static char *spapr_get_fw_dev_path(FWPathProvider *p, BusState *bus,
      */
     if (strcmp("usb-host", qdev_fw_name(dev)) == 0) {
         USBDevice *usbdev = CAST(USBDevice, dev, TYPE_USB_DEVICE);
-        if (usb_host_dev_is_scsi_storage(usbdev)) {
+        if (usb_device_is_scsi_storage(usbdev)) {
             return g_strdup_printf("storage@%s/disk", usbdev->port->path);
         }
     }
diff --git a/hw/usb/dev-storage-bot.c b/hw/usb/dev-storage-bot.c
index 6aad026d1133..68ebaca10c66 100644
--- a/hw/usb/dev-storage-bot.c
+++ b/hw/usb/dev-storage-bot.c
@@ -32,6 +32,7 @@ static void usb_msd_bot_realize(USBDevice *dev, Error **errp)
 
     usb_desc_create_serial(dev);
     usb_desc_init(dev);
+    dev->flags |= (1 << USB_DEV_FLAG_IS_SCSI_STORAGE);
     if (d->hotplugged) {
         s->dev.auto_attach = 0;
     }
diff --git a/hw/usb/dev-storage-classic.c b/hw/usb/dev-storage-classic.c
index 00cb34b22f02..3d017a4e6791 100644
--- a/hw/usb/dev-storage-classic.c
+++ b/hw/usb/dev-storage-classic.c
@@ -64,6 +64,7 @@ static void usb_msd_storage_realize(USBDevice *dev, Error **errp)
 
     usb_desc_create_serial(dev);
     usb_desc_init(dev);
+    dev->flags |= (1 << USB_DEV_FLAG_IS_SCSI_STORAGE);
     scsi_bus_new(&s->bus, sizeof(s->bus), DEVICE(dev),
                  &usb_msd_scsi_info_storage, NULL);
     scsi_dev = scsi_bus_legacy_add_drive(&s->bus, blk, 0, !!s->removable,
diff --git a/hw/usb/dev-uas.c b/hw/usb/dev-uas.c
index d2bd85d3f6bb..263056231c79 100644
--- a/hw/usb/dev-uas.c
+++ b/hw/usb/dev-uas.c
@@ -926,6 +926,7 @@ static void usb_uas_realize(USBDevice *dev, Error **errp)
     QTAILQ_INIT(&uas->requests);
     uas->status_bh = qemu_bh_new(usb_uas_send_status_bh, uas);
 
+    dev->flags |= (1 << USB_DEV_FLAG_IS_SCSI_STORAGE);
     scsi_bus_new(&uas->bus, sizeof(uas->bus), DEVICE(dev),
                  &usb_uas_scsi_info, NULL);
 }
diff --git a/hw/usb/host-libusb.c b/hw/usb/host-libusb.c
index 2518306f527f..e6d21aa8e1d3 100644
--- a/hw/usb/host-libusb.c
+++ b/hw/usb/host-libusb.c
@@ -770,6 +770,13 @@ static void usb_host_speed_compat(USBHostDevice *s)
         for (i = 0; i < conf->bNumInterfaces; i++) {
             for (a = 0; a < conf->interface[i].num_altsetting; a++) {
                 intf = &conf->interface[i].altsetting[a];
+
+                if (intf->bInterfaceClass == LIBUSB_CLASS_MASS_STORAGE &&
+                    intf->bInterfaceSubClass == 6) { /* SCSI */
+                    udev->flags |= (1 << USB_DEV_FLAG_IS_SCSI_STORAGE);
+                    break;
+                }
+
                 for (e = 0; e < intf->bNumEndpoints; e++) {
                     endp = &intf->endpoint[e];
                     type = endp->bmAttributes & 0x3;
@@ -1893,35 +1900,6 @@ static void usb_host_auto_check(void *unused)
     timer_mod(usb_auto_timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + 2000);
 }
 
-/**
- * Check whether USB host device has a USB mass storage SCSI interface
- */
-bool usb_host_dev_is_scsi_storage(USBDevice *ud)
-{
-    USBHostDevice *uhd = USB_HOST_DEVICE(ud);
-    struct libusb_config_descriptor *conf;
-    const struct libusb_interface_descriptor *intf;
-    bool is_scsi_storage = false;
-    int i;
-
-    if (!uhd || libusb_get_active_config_descriptor(uhd->dev, &conf) != 0) {
-        return false;
-    }
-
-    for (i = 0; i < conf->bNumInterfaces; i++) {
-        intf = &conf->interface[i].altsetting[ud->altsetting[i]];
-        if (intf->bInterfaceClass == LIBUSB_CLASS_MASS_STORAGE &&
-            intf->bInterfaceSubClass == 6) {                 /* 6 means SCSI */
-            is_scsi_storage = true;
-            break;
-        }
-    }
-
-    libusb_free_config_descriptor(conf);
-
-    return is_scsi_storage;
-}
-
 void hmp_info_usbhost(Monitor *mon, const QDict *qdict)
 {
     libusb_device **devs = NULL;
diff --git a/hw/usb/host-stub.c b/hw/usb/host-stub.c
index 80809ceba542..bbe69baa390f 100644
--- a/hw/usb/host-stub.c
+++ b/hw/usb/host-stub.c
@@ -38,8 +38,3 @@ void hmp_info_usbhost(Monitor *mon, const QDict *qdict)
 {
     monitor_printf(mon, "USB host devices not supported\n");
 }
-
-bool usb_host_dev_is_scsi_storage(USBDevice *ud)
-{
-    return false;
-}
-- 
2.31.1

[PATCH v4 32/34] monitor/usb: register 'info usbhost' dynamically

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/usb/host-libusb.c |  1 +
 hw/usb/host-stub.c   | 40 ----------------------------------------
 hmp-commands-info.hx |  1 -
 hw/usb/meson.build   |  4 +---
 4 files changed, 2 insertions(+), 44 deletions(-)
 delete mode 100644 hw/usb/host-stub.c

diff --git a/hw/usb/host-libusb.c b/hw/usb/host-libusb.c
index e6d21aa8e1d3..2b7f87872ce3 100644
--- a/hw/usb/host-libusb.c
+++ b/hw/usb/host-libusb.c
@@ -1781,6 +1781,7 @@ static TypeInfo usb_host_dev_info = {
 static void usb_host_register_types(void)
 {
     type_register_static(&usb_host_dev_info);
+    monitor_register_hmp("usbhost", true, hmp_info_usbhost);
 }
 
 type_init(usb_host_register_types)
diff --git a/hw/usb/host-stub.c b/hw/usb/host-stub.c
deleted file mode 100644
index bbe69baa390f..000000000000
--- a/hw/usb/host-stub.c
+++ /dev/null
@@ -1,40 +0,0 @@
-/*
- * Stub host USB redirector
- *
- * Copyright (c) 2005 Fabrice Bellard
- *
- * Copyright (c) 2008 Max Krasnyansky
- *      Support for host device auto connect & disconnect
- *      Major rewrite to support fully async operation
- *
- * Copyright 2008 TJ <linux@tjworld.net>
- *      Added flexible support for /dev/bus/usb /sys/bus/usb/devices in addition
- *      to the legacy /proc/bus/usb USB device discovery and handling
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to deal
- * in the Software without restriction, including without limitation the rights
- * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
- * copies of the Software, and to permit persons to whom the Software is
- * furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
- * THE SOFTWARE.
- */
-
-#include "qemu/osdep.h"
-#include "hw/usb.h"
-#include "monitor/monitor.h"
-
-void hmp_info_usbhost(Monitor *mon, const QDict *qdict)
-{
-    monitor_printf(mon, "USB host devices not supported\n");
-}
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index fb59c27200cb..ce42aef47acb 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -368,7 +368,6 @@ ERST
         .args_type  = "",
         .params     = "",
         .help       = "show host USB devices",
-        .cmd        = hmp_info_usbhost,
     },
 
 SRST
diff --git a/hw/usb/meson.build b/hw/usb/meson.build
index f357270d0b6b..3d8f2ae99302 100644
--- a/hw/usb/meson.build
+++ b/hw/usb/meson.build
@@ -73,9 +73,7 @@ endif
 
 # usb pass-through
 softmmu_ss.add(when: ['CONFIG_USB', 'CONFIG_USB_LIBUSB', libusb],
-               if_true: files('host-libusb.c'),
-               if_false: files('host-stub.c'))
-softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('host-stub.c'))
+               if_true: files('host-libusb.c'))
 
 softmmu_ss.add(when: ['CONFIG_USB', 'CONFIG_XEN', libusb], if_true: files('xen-usb.c'))
 
-- 
2.31.1

[PATCH v4 33/34] usb: build usb-host as module

[Gerd Hoffmann]

Drop one more shared library dependency (libusb) from core qemu.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/usb/host-libusb.c | 1 +
 hw/usb/meson.build   | 8 ++++++--
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/hw/usb/host-libusb.c b/hw/usb/host-libusb.c
index 2b7f87872ce3..c0f314462aaf 100644
--- a/hw/usb/host-libusb.c
+++ b/hw/usb/host-libusb.c
@@ -1777,6 +1777,7 @@ static TypeInfo usb_host_dev_info = {
     .class_init    = usb_host_class_initfn,
     .instance_init = usb_host_instance_init,
 };
+module_obj(TYPE_USB_HOST_DEVICE);
 
 static void usb_host_register_types(void)
 {
diff --git a/hw/usb/meson.build b/hw/usb/meson.build
index 3d8f2ae99302..0a6029ec9797 100644
--- a/hw/usb/meson.build
+++ b/hw/usb/meson.build
@@ -72,8 +72,12 @@ if config_host.has_key('CONFIG_USB_REDIR')
 endif
 
 # usb pass-through
-softmmu_ss.add(when: ['CONFIG_USB', 'CONFIG_USB_LIBUSB', libusb],
-               if_true: files('host-libusb.c'))
+if config_host.has_key('CONFIG_USB_LIBUSB')
+  usbhost_ss = ss.source_set()
+  usbhost_ss.add(when: ['CONFIG_USB', libusb],
+                 if_true: files('host-libusb.c'))
+  hw_usb_modules += {'host': usbhost_ss}
+endif
 
 softmmu_ss.add(when: ['CONFIG_USB', 'CONFIG_XEN', libusb], if_true: files('xen-usb.c'))
 
-- 
2.31.1

[PATCH v4 34/34] monitor/tcg: move tcg hmp commands to accel/tcg, register them dynamically

[Gerd Hoffmann]

One more little step towards modular tcg ...

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 accel/tcg/hmp.c       | 29 +++++++++++++++++++++++++++++
 monitor/misc.c        | 18 ------------------
 accel/tcg/meson.build |  1 +
 hmp-commands-info.hx  |  2 --
 4 files changed, 30 insertions(+), 20 deletions(-)
 create mode 100644 accel/tcg/hmp.c

diff --git a/accel/tcg/hmp.c b/accel/tcg/hmp.c
new file mode 100644
index 000000000000..a6e72fdb3ed6
--- /dev/null
+++ b/accel/tcg/hmp.c
@@ -0,0 +1,29 @@
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "exec/exec-all.h"
+#include "monitor/monitor.h"
+#include "sysemu/tcg.h"
+
+static void hmp_info_jit(Monitor *mon, const QDict *qdict)
+{
+    if (!tcg_enabled()) {
+        error_report("JIT information is only available with accel=tcg");
+        return;
+    }
+
+    dump_exec_info();
+    dump_drift_info();
+}
+
+static void hmp_info_opcount(Monitor *mon, const QDict *qdict)
+{
+    dump_opcount_info();
+}
+
+static void hmp_tcg_register(void)
+{
+    monitor_register_hmp("jit", true, hmp_info_jit);
+    monitor_register_hmp("opcount", true, hmp_info_opcount);
+}
+
+type_init(hmp_tcg_register);
diff --git a/monitor/misc.c b/monitor/misc.c
index ad476c6e51ea..b28874d6dc59 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -320,24 +320,6 @@ static void hmp_info_registers(Monitor *mon, const QDict *qdict)
     }
 }
 
-#ifdef CONFIG_TCG
-static void hmp_info_jit(Monitor *mon, const QDict *qdict)
-{
-    if (!tcg_enabled()) {
-        error_report("JIT information is only available with accel=tcg");
-        return;
-    }
-
-    dump_exec_info();
-    dump_drift_info();
-}
-
-static void hmp_info_opcount(Monitor *mon, const QDict *qdict)
-{
-    dump_opcount_info();
-}
-#endif
-
 static void hmp_info_sync_profile(Monitor *mon, const QDict *qdict)
 {
     int64_t max = qdict_get_try_int(qdict, "max", 10);
diff --git a/accel/tcg/meson.build b/accel/tcg/meson.build
index 0ae9180282e3..137a1a44cc0a 100644
--- a/accel/tcg/meson.build
+++ b/accel/tcg/meson.build
@@ -15,6 +15,7 @@ specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
 
 specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
   'cputlb.c',
+  'hmp.c',
 ))
 
 tcg_module_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index ce42aef47acb..27206ac049df 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -274,7 +274,6 @@ ERST
         .args_type  = "",
         .params     = "",
         .help       = "show dynamic compiler info",
-        .cmd        = hmp_info_jit,
     },
 #endif
 
@@ -289,7 +288,6 @@ ERST
         .args_type  = "",
         .params     = "",
         .help       = "show dynamic compiler opcode counters",
-        .cmd        = hmp_info_opcount,
     },
 #endif
 
-- 
2.31.1

Re: [PATCH v4 34/34] monitor/tcg: move tcg hmp commands to accel/tcg, register them dynamically

[Dr. David Alan Gilbert]

* Gerd Hoffmann (kraxel@redhat.com) wrote:
> One more little step towards modular tcg ...
> 
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

Acked-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> ---
>  accel/tcg/hmp.c       | 29 +++++++++++++++++++++++++++++
>  monitor/misc.c        | 18 ------------------
>  accel/tcg/meson.build |  1 +
>  hmp-commands-info.hx  |  2 --
>  4 files changed, 30 insertions(+), 20 deletions(-)
>  create mode 100644 accel/tcg/hmp.c
> 
> diff --git a/accel/tcg/hmp.c b/accel/tcg/hmp.c
> new file mode 100644
> index 000000000000..a6e72fdb3ed6
> --- /dev/null
> +++ b/accel/tcg/hmp.c
> @@ -0,0 +1,29 @@
> +#include "qemu/osdep.h"
> +#include "qemu/error-report.h"
> +#include "exec/exec-all.h"
> +#include "monitor/monitor.h"
> +#include "sysemu/tcg.h"
> +
> +static void hmp_info_jit(Monitor *mon, const QDict *qdict)
> +{
> +    if (!tcg_enabled()) {
> +        error_report("JIT information is only available with accel=tcg");
> +        return;
> +    }
> +
> +    dump_exec_info();
> +    dump_drift_info();
> +}
> +
> +static void hmp_info_opcount(Monitor *mon, const QDict *qdict)
> +{
> +    dump_opcount_info();
> +}
> +
> +static void hmp_tcg_register(void)
> +{
> +    monitor_register_hmp("jit", true, hmp_info_jit);
> +    monitor_register_hmp("opcount", true, hmp_info_opcount);
> +}
> +
> +type_init(hmp_tcg_register);
> diff --git a/monitor/misc.c b/monitor/misc.c
> index ad476c6e51ea..b28874d6dc59 100644
> --- a/monitor/misc.c
> +++ b/monitor/misc.c
> @@ -320,24 +320,6 @@ static void hmp_info_registers(Monitor *mon, const QDict *qdict)
>      }
>  }
>  
> -#ifdef CONFIG_TCG
> -static void hmp_info_jit(Monitor *mon, const QDict *qdict)
> -{
> -    if (!tcg_enabled()) {
> -        error_report("JIT information is only available with accel=tcg");
> -        return;
> -    }
> -
> -    dump_exec_info();
> -    dump_drift_info();
> -}
> -
> -static void hmp_info_opcount(Monitor *mon, const QDict *qdict)
> -{
> -    dump_opcount_info();
> -}
> -#endif
> -
>  static void hmp_info_sync_profile(Monitor *mon, const QDict *qdict)
>  {
>      int64_t max = qdict_get_try_int(qdict, "max", 10);
> diff --git a/accel/tcg/meson.build b/accel/tcg/meson.build
> index 0ae9180282e3..137a1a44cc0a 100644
> --- a/accel/tcg/meson.build
> +++ b/accel/tcg/meson.build
> @@ -15,6 +15,7 @@ specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
>  
>  specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
>    'cputlb.c',
> +  'hmp.c',
>  ))
>  
>  tcg_module_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
> diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
> index ce42aef47acb..27206ac049df 100644
> --- a/hmp-commands-info.hx
> +++ b/hmp-commands-info.hx
> @@ -274,7 +274,6 @@ ERST
>          .args_type  = "",
>          .params     = "",
>          .help       = "show dynamic compiler info",
> -        .cmd        = hmp_info_jit,
>      },
>  #endif
>  
> @@ -289,7 +288,6 @@ ERST
>          .args_type  = "",
>          .params     = "",
>          .help       = "show dynamic compiler opcode counters",
> -        .cmd        = hmp_info_opcount,
>      },
>  #endif
>  
> -- 
> 2.31.1
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [PATCH v4 00/34] modules: add meta-data database

[Jose R. Ziviani]

[-- Attachment #1: Type: text/plain, Size: 6073 bytes --]

Hello Gerd,

Reviewed and tested successfully here. Thank you!

Reviewed-by: Jose R. Ziviani <jziviani@suse.de>


On Thu, Jun 24, 2021 at 12:38:02PM +0200, Gerd Hoffmann wrote:
> This patch series adds support for module meta-data.  Today this is
> either hard-coded in qemu (see qemu_load_module_for_opts) or handled
> with manually maintained lists in util/module (see module_deps[] and
> qom_modules[]).  This series replaced that scheme with annotation
> macros, so the meta-data can go into the module source code and -- for
> example -- the module_obj() annotations can go next to the TypeInfo
> struct for the object class.
> 
> Patches 1-3 put the infrastructure in place:  Add the annotation macros,
> add a script to collect the meta-data, add a script to compile the
> meta-data into C source code which we can then add to qemu.
> 
> Patch 4 - check module dependencies (Jose, new in v4).
> 
> Patches 5-13 add annotations macros to the modules we have.
> 
> Patches 14-16 put the modinfo database into use and remove the
> module_deps[] and qom_modules[] lists.
> 
> Patch 16 adds two tracepoints for easier trouble-shooting.
> 
> Patches 18-20 add support for target-specific modules.
> 
> Patches 21-24 add documentation for all of the above (new in v4, was
> separate series).
> 
> Patches 25-29 start building accelerators modular.  So far it is
> only qtest (all archs) and a small fraction of tcg (x86 only).
> 
> Patches 30-34 add support for registering hmp commands so they can
> be implemented as module (new in v4, was separate series).
> 
> take care,
>   Gerd
> 
> Gerd Hoffmann (33):
>   modules: add modinfo macros
>   modules: collect module meta-data
>   modules: generate modinfo.c
>   modules: add qxl module annotations
>   modules: add virtio-gpu module annotations
>   modules: add chardev module annotations
>   modules: add audio module annotations
>   modules: add usb-redir module annotations
>   modules: add ccid module annotations
>   modules: add ui module annotations
>   modules: add s390x module annotations
>   modules: add block module annotations
>   modules: use modinfo for dependencies
>   modules: use modinfo for qom load
>   modules: use modinfo for qemu opts load
>   modules: add tracepoints
>   modules: check arch and block load on mismatch
>   modules: check arch on qom lookup
>   modules: target-specific module build infrastructure
>   modules: add documentation for module sourcesets
>   modules: add module_obj() note to QOM docs
>   modules: module.h kerneldoc annotations
>   modules: hook up modules.h to docs build
>   accel: autoload modules
>   accel: add qtest module annotations
>   accel: build qtest modular
>   accel: add tcg module annotations
>   accel: build tcg modular
>   monitor: allow register hmp commands
>   usb: drop usb_host_dev_is_scsi_storage hook
>   monitor/usb: register 'info usbhost' dynamically
>   usb: build usb-host as module
>   monitor/tcg: move tcg hmp commands to accel/tcg, register them
>     dynamically
> 
> Jose R. Ziviani (1):
>   modules: check if all dependencies can be satisfied
> 
>  scripts/modinfo-collect.py      |  67 +++++++++++
>  scripts/modinfo-generate.py     |  97 ++++++++++++++++
>  include/hw/usb.h                |   7 +-
>  include/monitor/monitor.h       |   3 +
>  include/qemu/module.h           |  74 ++++++++++++
>  accel/accel-common.c            |   2 +-
>  accel/accel-softmmu.c           |   2 +-
>  accel/qtest/qtest.c             |   2 +
>  accel/tcg/hmp.c                 |  29 +++++
>  accel/tcg/tcg-accel-ops.c       |   1 +
>  accel/tcg/tcg-all.c             |   1 +
>  audio/spiceaudio.c              |   2 +
>  block/iscsi-opts.c              |   1 +
>  chardev/baum.c                  |   1 +
>  chardev/spice.c                 |   4 +
>  hw/display/qxl.c                |   4 +
>  hw/display/vhost-user-gpu-pci.c |   1 +
>  hw/display/vhost-user-gpu.c     |   1 +
>  hw/display/vhost-user-vga.c     |   1 +
>  hw/display/virtio-gpu-base.c    |   1 +
>  hw/display/virtio-gpu-gl.c      |   3 +
>  hw/display/virtio-gpu-pci-gl.c  |   3 +
>  hw/display/virtio-gpu-pci.c     |   2 +
>  hw/display/virtio-gpu.c         |   1 +
>  hw/display/virtio-vga-gl.c      |   3 +
>  hw/display/virtio-vga.c         |   2 +
>  hw/ppc/spapr.c                  |   2 +-
>  hw/s390x/virtio-ccw-gpu.c       |   3 +
>  hw/usb/ccid-card-emulated.c     |   1 +
>  hw/usb/ccid-card-passthru.c     |   1 +
>  hw/usb/dev-storage-bot.c        |   1 +
>  hw/usb/dev-storage-classic.c    |   1 +
>  hw/usb/dev-uas.c                |   1 +
>  hw/usb/host-libusb.c            |  38 ++----
>  hw/usb/host-stub.c              |  45 -------
>  hw/usb/redirect.c               |   1 +
>  monitor/hmp.c                   |   7 ++
>  monitor/misc.c                  |  34 +++---
>  softmmu/vl.c                    |  24 ++--
>  stubs/module-opts.c             |   4 -
>  ui/egl-headless.c               |   4 +
>  ui/gtk.c                        |   4 +
>  ui/sdl2.c                       |   4 +
>  ui/spice-app.c                  |   3 +
>  ui/spice-core.c                 |   5 +
>  util/module.c                   | 200 ++++++++++++++++++--------------
>  accel/qtest/meson.build         |   8 +-
>  accel/tcg/meson.build           |   6 +-
>  docs/devel/build-system.rst     |  17 +++
>  docs/devel/index.rst            |   1 +
>  docs/devel/modules.rst          |   5 +
>  docs/devel/qom.rst              |   8 ++
>  hmp-commands-info.hx            |   3 -
>  hw/usb/meson.build              |  10 +-
>  meson.build                     |  82 +++++++++++++
>  util/trace-events               |   4 +
>  56 files changed, 624 insertions(+), 218 deletions(-)
>  create mode 100755 scripts/modinfo-collect.py
>  create mode 100755 scripts/modinfo-generate.py
>  create mode 100644 accel/tcg/hmp.c
>  delete mode 100644 hw/usb/host-stub.c
>  create mode 100644 docs/devel/modules.rst
> 
> -- 
> 2.31.1
> 
> 
> 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v4 00/34] modules: add meta-data database

[Dr. David Alan Gilbert]

* Gerd Hoffmann (kraxel@redhat.com) wrote:
> This patch series adds support for module meta-data.  Today this is
> either hard-coded in qemu (see qemu_load_module_for_opts) or handled
> with manually maintained lists in util/module (see module_deps[] and
> qom_modules[]).  This series replaced that scheme with annotation
> macros, so the meta-data can go into the module source code and -- for
> example -- the module_obj() annotations can go next to the TypeInfo
> struct for the object class.

So this is slightly off-topic for the series; but kind of relevant,
but...
Is there a way to inhibit module loading after a given point?

I ask, because there's a fairly well known security escalation that
takes advantage of NSS loading of PAM modules; typically you have
your nice sandboxed application, you write out your nasty .so into the
sandbox and then somehow get your application to trigger the PAM module
load.
Now, what stops the same attack here?

Dave

> Patches 1-3 put the infrastructure in place:  Add the annotation macros,
> add a script to collect the meta-data, add a script to compile the
> meta-data into C source code which we can then add to qemu.
> 
> Patch 4 - check module dependencies (Jose, new in v4).
> 
> Patches 5-13 add annotations macros to the modules we have.
> 
> Patches 14-16 put the modinfo database into use and remove the
> module_deps[] and qom_modules[] lists.
> 
> Patch 16 adds two tracepoints for easier trouble-shooting.
> 
> Patches 18-20 add support for target-specific modules.
> 
> Patches 21-24 add documentation for all of the above (new in v4, was
> separate series).
> 
> Patches 25-29 start building accelerators modular.  So far it is
> only qtest (all archs) and a small fraction of tcg (x86 only).
> 
> Patches 30-34 add support for registering hmp commands so they can
> be implemented as module (new in v4, was separate series).
> 
> take care,
>   Gerd
> 
> Gerd Hoffmann (33):
>   modules: add modinfo macros
>   modules: collect module meta-data
>   modules: generate modinfo.c
>   modules: add qxl module annotations
>   modules: add virtio-gpu module annotations
>   modules: add chardev module annotations
>   modules: add audio module annotations
>   modules: add usb-redir module annotations
>   modules: add ccid module annotations
>   modules: add ui module annotations
>   modules: add s390x module annotations
>   modules: add block module annotations
>   modules: use modinfo for dependencies
>   modules: use modinfo for qom load
>   modules: use modinfo for qemu opts load
>   modules: add tracepoints
>   modules: check arch and block load on mismatch
>   modules: check arch on qom lookup
>   modules: target-specific module build infrastructure
>   modules: add documentation for module sourcesets
>   modules: add module_obj() note to QOM docs
>   modules: module.h kerneldoc annotations
>   modules: hook up modules.h to docs build
>   accel: autoload modules
>   accel: add qtest module annotations
>   accel: build qtest modular
>   accel: add tcg module annotations
>   accel: build tcg modular
>   monitor: allow register hmp commands
>   usb: drop usb_host_dev_is_scsi_storage hook
>   monitor/usb: register 'info usbhost' dynamically
>   usb: build usb-host as module
>   monitor/tcg: move tcg hmp commands to accel/tcg, register them
>     dynamically
> 
> Jose R. Ziviani (1):
>   modules: check if all dependencies can be satisfied
> 
>  scripts/modinfo-collect.py      |  67 +++++++++++
>  scripts/modinfo-generate.py     |  97 ++++++++++++++++
>  include/hw/usb.h                |   7 +-
>  include/monitor/monitor.h       |   3 +
>  include/qemu/module.h           |  74 ++++++++++++
>  accel/accel-common.c            |   2 +-
>  accel/accel-softmmu.c           |   2 +-
>  accel/qtest/qtest.c             |   2 +
>  accel/tcg/hmp.c                 |  29 +++++
>  accel/tcg/tcg-accel-ops.c       |   1 +
>  accel/tcg/tcg-all.c             |   1 +
>  audio/spiceaudio.c              |   2 +
>  block/iscsi-opts.c              |   1 +
>  chardev/baum.c                  |   1 +
>  chardev/spice.c                 |   4 +
>  hw/display/qxl.c                |   4 +
>  hw/display/vhost-user-gpu-pci.c |   1 +
>  hw/display/vhost-user-gpu.c     |   1 +
>  hw/display/vhost-user-vga.c     |   1 +
>  hw/display/virtio-gpu-base.c    |   1 +
>  hw/display/virtio-gpu-gl.c      |   3 +
>  hw/display/virtio-gpu-pci-gl.c  |   3 +
>  hw/display/virtio-gpu-pci.c     |   2 +
>  hw/display/virtio-gpu.c         |   1 +
>  hw/display/virtio-vga-gl.c      |   3 +
>  hw/display/virtio-vga.c         |   2 +
>  hw/ppc/spapr.c                  |   2 +-
>  hw/s390x/virtio-ccw-gpu.c       |   3 +
>  hw/usb/ccid-card-emulated.c     |   1 +
>  hw/usb/ccid-card-passthru.c     |   1 +
>  hw/usb/dev-storage-bot.c        |   1 +
>  hw/usb/dev-storage-classic.c    |   1 +
>  hw/usb/dev-uas.c                |   1 +
>  hw/usb/host-libusb.c            |  38 ++----
>  hw/usb/host-stub.c              |  45 -------
>  hw/usb/redirect.c               |   1 +
>  monitor/hmp.c                   |   7 ++
>  monitor/misc.c                  |  34 +++---
>  softmmu/vl.c                    |  24 ++--
>  stubs/module-opts.c             |   4 -
>  ui/egl-headless.c               |   4 +
>  ui/gtk.c                        |   4 +
>  ui/sdl2.c                       |   4 +
>  ui/spice-app.c                  |   3 +
>  ui/spice-core.c                 |   5 +
>  util/module.c                   | 200 ++++++++++++++++++--------------
>  accel/qtest/meson.build         |   8 +-
>  accel/tcg/meson.build           |   6 +-
>  docs/devel/build-system.rst     |  17 +++
>  docs/devel/index.rst            |   1 +
>  docs/devel/modules.rst          |   5 +
>  docs/devel/qom.rst              |   8 ++
>  hmp-commands-info.hx            |   3 -
>  hw/usb/meson.build              |  10 +-
>  meson.build                     |  82 +++++++++++++
>  util/trace-events               |   4 +
>  56 files changed, 624 insertions(+), 218 deletions(-)
>  create mode 100755 scripts/modinfo-collect.py
>  create mode 100755 scripts/modinfo-generate.py
>  create mode 100644 accel/tcg/hmp.c
>  delete mode 100644 hw/usb/host-stub.c
>  create mode 100644 docs/devel/modules.rst
> 
> -- 
> 2.31.1
> 
> 
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [PATCH v4 00/34] modules: add meta-data database

[Gerd Hoffmann]

On Thu, Jun 24, 2021 at 04:01:25PM +0100, Dr. David Alan Gilbert wrote:
> * Gerd Hoffmann (kraxel@redhat.com) wrote:
> > This patch series adds support for module meta-data.  Today this is
> > either hard-coded in qemu (see qemu_load_module_for_opts) or handled
> > with manually maintained lists in util/module (see module_deps[] and
> > qom_modules[]).  This series replaced that scheme with annotation
> > macros, so the meta-data can go into the module source code and -- for
> > example -- the module_obj() annotations can go next to the TypeInfo
> > struct for the object class.
> 
> So this is slightly off-topic for the series; but kind of relevant,
> but...
> Is there a way to inhibit module loading after a given point?

We could block loading after machine initialization.
Has implications for hotplug though.

> I ask, because there's a fairly well known security escalation that
> takes advantage of NSS loading of PAM modules; typically you have
> your nice sandboxed application, you write out your nasty .so into the
> sandbox and then somehow get your application to trigger the PAM module
> load.
> Now, what stops the same attack here?

Placing a new .so at some random directory wouldn't work, qemu only
loads modules from the search path (but I guess the same is true for
pam).

With this patch series applied all modules are listed the in modinfo.c
database (even if we don't have any metadata about them), so we could
easily limit loading to modules known at compile time.  Not sure how
much that alone would improve security though, when the attacker is able
to write to the qemu module directory it isn't much of a problem to just
overwrite one of the existing modules.

We could try work with hashes or signatures stored in modinfo ...

take care,
  Gerd

Re: [PATCH v4 00/34] modules: add meta-data database

[Dr. David Alan Gilbert]

* Gerd Hoffmann (kraxel@redhat.com) wrote:
> On Thu, Jun 24, 2021 at 04:01:25PM +0100, Dr. David Alan Gilbert wrote:
> > * Gerd Hoffmann (kraxel@redhat.com) wrote:
> > > This patch series adds support for module meta-data.  Today this is
> > > either hard-coded in qemu (see qemu_load_module_for_opts) or handled
> > > with manually maintained lists in util/module (see module_deps[] and
> > > qom_modules[]).  This series replaced that scheme with annotation
> > > macros, so the meta-data can go into the module source code and -- for
> > > example -- the module_obj() annotations can go next to the TypeInfo
> > > struct for the object class.
> > 
> > So this is slightly off-topic for the series; but kind of relevant,
> > but...
> > Is there a way to inhibit module loading after a given point?
> 
> We could block loading after machine initialization.
> Has implications for hotplug though.

Yes; I was thinking perhaps a command to explicitly disable autoloading
if people worried about it.

> > I ask, because there's a fairly well known security escalation that
> > takes advantage of NSS loading of PAM modules; typically you have
> > your nice sandboxed application, you write out your nasty .so into the
> > sandbox and then somehow get your application to trigger the PAM module
> > load.
> > Now, what stops the same attack here?
> 
> Placing a new .so at some random directory wouldn't work, qemu only
> loads modules from the search path (but I guess the same is true for
> pam).

Yes, I'm failing to find the CVE I vaguely remember about the details of
how it was messed up.

Dave

> With this patch series applied all modules are listed the in modinfo.c
> database (even if we don't have any metadata about them), so we could
> easily limit loading to modules known at compile time.  Not sure how
> much that alone would improve security though, when the attacker is able
> to write to the qemu module directory it isn't much of a problem to just
> overwrite one of the existing modules.
> 
> We could try work with hashes or signatures stored in modinfo ...
> 
> take care,
>   Gerd
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK