Re: [PATCH v3 02/24] modules: collect module meta-data

Re: [PATCH v3 02/24] modules: collect module meta-data

[Paolo Bonzini]

On 21/06/21 14:52, Gerd Hoffmann wrote:
> ninja: error: 'libui-curses.a.p/meson-generated_.._config-host.h.o', needed by 'ui-curses.modinfo.test', missing and no known rule to make it
> 
> Hmm, not sure where this comes from.  meson doesn't try to link
> config-host.h.o into libui-curses.a, so why does extract_all_objects()
> return it?
> 
> Test patch (incremental to this series) below.

Bug in Meson, fix at https://github.com/mesonbuild/meson/pull/8900.  You 
can just ignore missing files.

Paolo

Re: [PATCH v3 02/24] modules: collect module meta-data

[Gerd Hoffmann]

On Tue, Jun 22, 2021 at 06:03:45PM +0200, Paolo Bonzini wrote:
> On 21/06/21 14:52, Gerd Hoffmann wrote:
> > ninja: error: 'libui-curses.a.p/meson-generated_.._config-host.h.o', needed by 'ui-curses.modinfo.test', missing and no known rule to make it
> > 
> > Hmm, not sure where this comes from.  meson doesn't try to link
> > config-host.h.o into libui-curses.a, so why does extract_all_objects()
> > return it?
> > 
> > Test patch (incremental to this series) below.
> 
> Bug in Meson, fix at https://github.com/mesonbuild/meson/pull/8900.  You can
> just ignore missing files.

Well, it's ninja throwing the error not the modinfo script, the script
doesn't even run ...

take care,
  Gerd

Re: [PATCH v3 02/24] modules: collect module meta-data

[Paolo Bonzini]

[-- Attachment #1: Type: text/plain, Size: 847 bytes --]

Oh well. Let's add a to-do marker.

Paolo

Il mer 23 giu 2021, 09:36 Gerd Hoffmann <kraxel@redhat.com> ha scritto:

> On Tue, Jun 22, 2021 at 06:03:45PM +0200, Paolo Bonzini wrote:
> > On 21/06/21 14:52, Gerd Hoffmann wrote:
> > > ninja: error: 'libui-curses.a.p/meson-generated_.._config-host.h.o',
> needed by 'ui-curses.modinfo.test', missing and no known rule to make it
> > >
> > > Hmm, not sure where this comes from.  meson doesn't try to link
> > > config-host.h.o into libui-curses.a, so why does extract_all_objects()
> > > return it?
> > >
> > > Test patch (incremental to this series) below.
> >
> > Bug in Meson, fix at https://github.com/mesonbuild/meson/pull/8900.
> You can
> > just ignore missing files.
>
> Well, it's ninja throwing the error not the modinfo script, the script
> doesn't even run ...
>
> take care,
>   Gerd
>
>

[-- Attachment #2: Type: text/html, Size: 1403 bytes --]

Re: [PATCH v3 02/24] modules: collect module meta-data

[Gerd Hoffmann]

On Fri, Jun 18, 2021 at 06:09:55PM +0200, Paolo Bonzini wrote:
> On 18/06/21 06:53, Gerd Hoffmann wrote:
> > +def find_command(src, target, compile_commands):
> > +    for command in compile_commands:
> > +        if command['file'] != src:
> > +            continue
> > +        if target != '' and command['command'].find(target) == -1:
> > +            continue
> 
> 
> Did you look into using extract_objects for this instead of looking for the
> target (which works, but yuck :))?

ninja: error: 'libui-curses.a.p/meson-generated_.._config-host.h.o', needed by 'ui-curses.modinfo.test', missing and no known rule to make it

Hmm, not sure where this comes from.  meson doesn't try to link
config-host.h.o into libui-curses.a, so why does extract_all_objects()
return it?

Test patch (incremental to this series) below.

take care,
  Gerd

From 5453683429d7b08b959e2cd63ee00fdccfb0c7b7 Mon Sep 17 00:00:00 2001
From: Gerd Hoffmann <kraxel@redhat.com>
Date: Mon, 21 Jun 2021 14:45:14 +0200
Subject: [PATCH] [wip] extract_all_objects experiments

---
 meson.build             | 7 +++++++
 scripts/modinfo-test.sh | 8 ++++++++
 2 files changed, 15 insertions(+)
 create mode 100755 scripts/modinfo-test.sh

diff --git a/meson.build b/meson.build
index 03bacca7cddb..8e7ccccf176c 100644
--- a/meson.build
+++ b/meson.build
@@ -2042,6 +2042,7 @@ target_modules += { 'accel' : { 'qtest': qtest_module_ss,
 
 modinfo_collect = find_program('scripts/modinfo-collect.py')
 modinfo_generate = find_program('scripts/modinfo-generate.py')
+modinfo_test = find_program('scripts/modinfo-test.sh')
 modinfo_files = []
 
 block_mods = []
@@ -2063,6 +2064,12 @@ foreach d, list : modules
                                        input: module_ss.sources(),
                                        capture: true,
                                        command: [modinfo_collect, '@INPUT@'])
+        custom_target(d + '-' + m + '.modinfo.test',
+                      output: d + '-' + m + '.modinfo.test',
+                      input: sl.extract_all_objects(recursive: true),
+                      capture: true,
+                      build_by_default: true, # to be removed when added to a target
+                      command: [modinfo_test, '@INPUT@'])
       endif
     else
       if d == 'block'
diff --git a/scripts/modinfo-test.sh b/scripts/modinfo-test.sh
new file mode 100755
index 000000000000..979c9cc9aeef
--- /dev/null
+++ b/scripts/modinfo-test.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+if test "$1" = "--target"; then
+    echo "# target $2"
+    shift;shift
+fi
+for item in "$@"; do
+    echo "# input $item"
+done
-- 
2.31.1

Re: [PATCH v3 02/24] modules: collect module meta-data

[Paolo Bonzini]

On 18/06/21 06:53, Gerd Hoffmann wrote:
> +def find_command(src, target, compile_commands):
> +    for command in compile_commands:
> +        if command['file'] != src:
> +            continue
> +        if target != '' and command['command'].find(target) == -1:
> +            continue


Did you look into using extract_objects for this instead of looking for 
the target (which works, but yuck :))?

Paolo

[PATCH v3 02/24] modules: collect module meta-data

[Gerd Hoffmann]

Add script to collect the module meta-data from the source code,
store the results in *.modinfo files.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 scripts/modinfo-collect.py | 67 ++++++++++++++++++++++++++++++++++++++
 meson.build                | 11 +++++++
 2 files changed, 78 insertions(+)
 create mode 100755 scripts/modinfo-collect.py

diff --git a/scripts/modinfo-collect.py b/scripts/modinfo-collect.py
new file mode 100755
index 000000000000..4acb188c3e89
--- /dev/null
+++ b/scripts/modinfo-collect.py
@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+import os
+import sys
+import json
+import shlex
+import subprocess
+
+def find_command(src, target, compile_commands):
+    for command in compile_commands:
+        if command['file'] != src:
+            continue
+        if target != '' and command['command'].find(target) == -1:
+            continue
+        return command['command']
+    return 'false'
+
+def process_command(src, command):
+    skip = False
+    arg = False
+    out = []
+    for item in shlex.split(command):
+        if arg:
+            out.append(x)
+            arg = False
+            continue
+        if skip:
+            skip = False
+            continue
+        if item == '-MF' or item == '-MQ' or item == '-o':
+            skip = True
+            continue
+        if item == '-c':
+            skip = True
+            continue
+        out.append(item)
+    out.append('-DQEMU_MODINFO')
+    out.append('-E')
+    out.append(src)
+    return out
+
+def main(args):
+    target = ''
+    if args[0] == '--target':
+        args.pop(0)
+        target = args.pop(0)
+        print("MODINFO_DEBUG target %s" % target)
+        arch = target[:-8] # cut '-softmmu'
+        print("MODINFO_START arch \"%s\" MODINFO_END" % arch)
+    with open('compile_commands.json') as f:
+        compile_commands = json.load(f)
+    for src in args:
+        print("MODINFO_DEBUG src %s" % src)
+        command = find_command(src, target, compile_commands)
+        cmdline = process_command(src, command)
+        print("MODINFO_DEBUG cmd", cmdline)
+        result = subprocess.run(cmdline, stdout = subprocess.PIPE,
+                                universal_newlines = True)
+        if result.returncode != 0:
+            sys.exit(result.returncode)
+        for line in result.stdout.split('\n'):
+            if line.find('MODINFO') != -1:
+                print(line)
+
+if __name__ == "__main__":
+    main(sys.argv[1:])
diff --git a/meson.build b/meson.build
index a2311eda6ec5..bb99619257d5 100644
--- a/meson.build
+++ b/meson.build
@@ -2021,6 +2021,9 @@ subdir('tests/qtest/fuzz')
 # Library dependencies #
 ########################
 
+modinfo_collect = find_program('scripts/modinfo-collect.py')
+modinfo_files = []
+
 block_mods = []
 softmmu_mods = []
 foreach d, list : modules
@@ -2034,6 +2037,14 @@ foreach d, list : modules
       else
         softmmu_mods += sl
       endif
+      if module_ss.sources() != []
+        modinfo_files += custom_target(d + '-' + m + '.modinfo',
+                                       output: d + '-' + m + '.modinfo',
+                                       input: module_ss.sources(),
+                                       capture: true,
+                                       build_by_default: true, # to be removed when added to a target
+                                       command: [modinfo_collect, '@INPUT@'])
+      endif
     else
       if d == 'block'
         block_ss.add_all(module_ss)
-- 
2.31.1