[Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration

[Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration

[Paolo Bonzini]

Hi all,

this is just a simple example of introducing Meson into the build system,
converting a few contrib/ subprojects to use it instead of Makefile.objs.
There are certainly many rough edges, but I wanted to throw this out to
show what it could look like.

I chose contrib/ because it's a simple example of how the current build
system causes clutter in the toplevel Makefiles and how interrelated the
Makefile, Makefile.objs and contrib/*/Makefile.objs file are even for
such simple executables.

Paolo

Paolo Bonzini (7):
  configure: do not include $(...) variables in config-host.mak
  configure: set $PYTHON to a full path
  configure: integrate Meson in the build system
  libvhost-user: convert to Meson
  vhost-user-blk: convert to Meson
  vhost-user-scsi: convert to Meson
  rdmacm-mux: convert to Meson

 .gitignore                            |   5 +-
 Makefile                              |  26 +-
 Makefile.objs                         |   6 -
 configure                             |  66 ++-
 contrib/libvhost-user/Makefile.objs   |   1 -
 contrib/libvhost-user/meson.build     |   7 +
 contrib/rdmacm-mux/Makefile.objs      |   3 -
 contrib/rdmacm-mux/meson.build        |   6 +
 contrib/vhost-user-blk/Makefile.objs  |   1 -
 contrib/vhost-user-blk/meson.build    |   3 +
 contrib/vhost-user-scsi/Makefile.objs |   1 -
 contrib/vhost-user-scsi/meson.build   |   3 +
 meson.build                           |  20 +
 scripts/ninjatool.py                  | 964 ++++++++++++++++++++++++++++++++++
 14 files changed, 1066 insertions(+), 46 deletions(-)
 delete mode 100644 contrib/libvhost-user/Makefile.objs
 create mode 100644 contrib/libvhost-user/meson.build
 delete mode 100644 contrib/rdmacm-mux/Makefile.objs
 create mode 100644 contrib/rdmacm-mux/meson.build
 delete mode 100644 contrib/vhost-user-blk/Makefile.objs
 create mode 100644 contrib/vhost-user-blk/meson.build
 delete mode 100644 contrib/vhost-user-scsi/Makefile.objs
 create mode 100644 contrib/vhost-user-scsi/meson.build
 create mode 100644 meson.build
 create mode 100644 scripts/ninjatool.py

-- 
1.8.3.1

[Qemu-devel] [PATCH 1/7] configure: do not include $(...) variables in config-host.mak

[Paolo Bonzini]

This ensures that Meson will be able to reuse the results of
the tests that are performed in the configure script.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/configure b/configure
index 6cdcfb2..203dece 100755
--- a/configure
+++ b/configure
@@ -597,7 +597,7 @@ QEMU_CFLAGS="-fno-strict-aliasing -fno-common -fwrapv -std=gnu99 $QEMU_CFLAGS"
 QEMU_CFLAGS="-Wall -Wundef -Wwrite-strings -Wmissing-prototypes $QEMU_CFLAGS"
 QEMU_CFLAGS="-Wstrict-prototypes -Wredundant-decls $QEMU_CFLAGS"
 QEMU_CFLAGS="-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE $QEMU_CFLAGS"
-QEMU_INCLUDES="-iquote . -iquote \$(SRC_PATH) -iquote \$(SRC_PATH)/accel/tcg -iquote \$(SRC_PATH)/include"
+QEMU_INCLUDES="-iquote . -iquote ${source_path} -iquote ${source_path}/accel/tcg -iquote ${source_path}/include"
 if test "$debug_info" = "yes"; then
     CFLAGS="-g $CFLAGS"
     LDFLAGS="-g $LDFLAGS"
@@ -896,7 +896,7 @@ Linux)
   linux="yes"
   linux_user="yes"
   kvm="yes"
-  QEMU_INCLUDES="-I\$(SRC_PATH)/linux-headers -I$PWD/linux-headers $QEMU_INCLUDES"
+  QEMU_INCLUDES="-I${source_path}/linux-headers -I$PWD/linux-headers $QEMU_INCLUDES"
   supported_os="yes"
   libudev="yes"
 ;;
@@ -4090,8 +4090,8 @@ EOF
               symlink "$source_path/dtc/Makefile" "dtc/Makefile"
               symlink "$source_path/dtc/scripts" "dtc/scripts"
           fi
-          fdt_cflags="-I\$(SRC_PATH)/dtc/libfdt"
-          fdt_ldflags="-L\$(BUILD_DIR)/dtc/libfdt"
+          fdt_cflags="-I${source_path}/dtc/libfdt"
+          fdt_ldflags="-L$PWD/dtc/libfdt"
           fdt_libs="$fdt_libs"
       elif test "$fdt" = "yes" ; then
           # Not a git build & no libfdt found, prompt for system install
@@ -5040,13 +5040,13 @@ case "$capstone" in
       git_submodules="${git_submodules} capstone"
     fi
     mkdir -p capstone
-    QEMU_CFLAGS="$QEMU_CFLAGS -I\$(SRC_PATH)/capstone/include"
+    QEMU_CFLAGS="$QEMU_CFLAGS -I${source_path}/capstone/include"
     if test "$mingw32" = "yes"; then
       LIBCAPSTONE=capstone.lib
     else
       LIBCAPSTONE=libcapstone.a
     fi
-    LIBS="-L\$(BUILD_DIR)/capstone -lcapstone $LIBS"
+    LIBS="-L$PWD/capstone -lcapstone $LIBS"
     ;;
 
   system)
@@ -5951,8 +5951,8 @@ case "$slirp" in
       git_submodules="${git_submodules} slirp"
     fi
     mkdir -p slirp
-    slirp_cflags="-I\$(SRC_PATH)/slirp/src -I\$(BUILD_DIR)/slirp/src"
-    slirp_libs="-L\$(BUILD_DIR)/slirp -lslirp"
+    slirp_cflags="-I${source_path}/slirp/src -I$PWD/slirp/src"
+    slirp_libs="-L$PWD/slirp -lslirp"
     ;;
 
   system)
@@ -7307,21 +7307,21 @@ if test "$sheepdog" = "yes" ; then
 fi
 
 if test "$tcg_interpreter" = "yes"; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/tci $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/tci $QEMU_INCLUDES"
 elif test "$ARCH" = "sparc64" ; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/sparc $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/sparc $QEMU_INCLUDES"
 elif test "$ARCH" = "s390x" ; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/s390 $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/s390 $QEMU_INCLUDES"
 elif test "$ARCH" = "x86_64" || test "$ARCH" = "x32" ; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/i386 $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/i386 $QEMU_INCLUDES"
 elif test "$ARCH" = "ppc64" ; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/ppc $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/ppc $QEMU_INCLUDES"
 elif test "$ARCH" = "riscv32" || test "$ARCH" = "riscv64" ; then
-  QEMU_INCLUDES="-I\$(SRC_PATH)/tcg/riscv $QEMU_INCLUDES"
+  QEMU_INCLUDES="-I${source_path}/tcg/riscv $QEMU_INCLUDES"
 else
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/\$(ARCH) $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/\$(ARCH) $QEMU_INCLUDES"
 fi
-QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg $QEMU_INCLUDES"
+QEMU_INCLUDES="-iquote ${source_path}/tcg $QEMU_INCLUDES"
 
 echo "TOOLS=$tools" >> $config_host_mak
 echo "ROMS=$roms" >> $config_host_mak
-- 
1.8.3.1

[Qemu-devel] [PATCH 2/7] configure: set $PYTHON to a full path

[Paolo Bonzini]

This will make it possible to replace it in a shebang line.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure b/configure
index 203dece..0814a5f 100755
--- a/configure
+++ b/configure
@@ -918,7 +918,7 @@ for binary in "${PYTHON-python3}" python python2
 do
     if has "$binary"
     then
-        python="$binary"
+        python=$(type -p "$binary")
         break
     fi
 done
-- 
1.8.3.1

[Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Paolo Bonzini]

The Meson build system is integrated in the existing configure/make steps
by invoking Meson from the configure script and converting Meson's build.ninja
rules to an included Makefile.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Makefile             |   9 +
 configure            |  30 ++
 meson.build          |   9 +
 scripts/ninjatool.py | 964 +++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 1012 insertions(+)
 create mode 100644 meson.build
 create mode 100644 scripts/ninjatool.py

diff --git a/Makefile b/Makefile
index 8e2fc66..b8f802c 100644
--- a/Makefile
+++ b/Makefile
@@ -48,6 +48,15 @@ git-submodule-update:
 endif
 endif
 
+export NINJA=./ninjatool
+Makefile.ninja: build.ninja ninjatool
+	./ninjatool -t ninja2make --omit dist uninstall < $< > $@
+-include Makefile.ninja
+
+ninjatool: $(SRC_PATH)/scripts/ninjatool.py
+	sed -e '1c\' -e '#! $(PYTHON)' $< > $@
+	chmod +x $@
+
 .git-submodule-status: git-submodule-update config-host.mak
 
 # Check that we're not trying to do an out-of-tree build from
diff --git a/configure b/configure
index 0814a5f..b8c3c58 100755
--- a/configure
+++ b/configure
@@ -493,6 +493,7 @@ docker="no"
 debug_mutex="no"
 libpmem=""
 default_devices="yes"
+meson=meson
 
 # cross compilers defaults, can be overridden with --cross-cc-ARCH
 cross_cc_aarch64="aarch64-linux-gnu-gcc"
@@ -983,6 +984,8 @@ for opt do
   ;;
   --python=*) python="$optarg"
   ;;
+  --meson=*) meson="$optarg"
+  ;;
   --gcov=*) gcov_tool="$optarg"
   ;;
   --smbd=*) smbd="$optarg"
@@ -1685,6 +1688,7 @@ Advanced options (experts only):
   --make=MAKE              use specified make [$make]
   --install=INSTALL        use specified install [$install]
   --python=PYTHON          use specified python [$python]
+  --meson=PYTHON           use specified meson [$meson]
   --smbd=SMBD              use specified smbd [$smbd]
   --with-git=GIT           use specified git [$git]
   --static                 enable static build [$static]
@@ -1850,6 +1854,11 @@ then
     error_exit "Python not found. Use --python=/path/to/python"
 fi
 
+if ! has "$meson"
+then
+    error_exit "Meson not found. Use --meson=/path/to/meson"
+fi
+
 # Note that if the Python conditional here evaluates True we will exit
 # with status 1 which is a shell 'false' value.
 if ! $python -c 'import sys; sys.exit(sys.version_info < (2,7))'; then
@@ -7983,6 +7992,27 @@ echo "# Automatically generated by configure - do not modify" > "$iotests_common
 echo >> "$iotests_common_env"
 echo "export PYTHON='$python'" >> "$iotests_common_env"
 
+# bootstrap ninjatool, we need it before Make runs
+if ! test -x ninjatool; then
+  sed -e '1c\' -e "#! $python" ${source_path}/scripts/ninjatool.py > ninjatool
+  chmod +x ninjatool
+fi
+rm -rf meson-private meson-info meson-logs
+NINJA=$PWD/ninjatool $python $meson setup \
+	--prefix "$prefix" \
+	--libdir "$libdir" \
+	--libexecdir "$libexecdir" \
+	--bindir "$bindir" \
+	--includedir "$includedir" \
+	--datadir "$datadir" \
+	--mandir "$mandir" \
+	--sysconfdir "$sysconfdir" \
+	--localstatedir "$local_statedir" \
+	$(test "$strip_opt" = yes && echo --strip) \
+	--buildtype $(if test "$debug" = yes; then echo debug; else echo release; fi) \
+	"$PWD" "$source_path"
+
+
 # Save the configure command line for later reuse.
 cat <<EOD >config.status
 #!/bin/sh
diff --git a/meson.build b/meson.build
new file mode 100644
index 0000000..b683d70
--- /dev/null
+++ b/meson.build
@@ -0,0 +1,9 @@
+project('qemu', 'c', meson_version: '>=0.50.999')
+
+kconfig = import('unstable-kconfig')
+config_host = kconfig.load(meson.current_build_dir() / 'config-host.mak')
+
+add_project_arguments(config_host['QEMU_CFLAGS'].split(),
+                      language: 'c')
+add_project_arguments(config_host['QEMU_INCLUDES'].split(),
+                      language: 'c')
diff --git a/scripts/ninjatool.py b/scripts/ninjatool.py
new file mode 100644
index 0000000..6d90919
--- /dev/null
+++ b/scripts/ninjatool.py
@@ -0,0 +1,964 @@
+#! /usr/bin/env python3
+
+"""Python module for parsing and processing .ninja files.
+
+   Author: Paolo Bonzini
+
+   Copyright (C) 2019 Red Hat, Inc."""
+
+from collections import namedtuple, defaultdict
+import sys
+import os
+import re
+import json
+import argparse
+
+
+class InvalidArgumentError(Exception):
+    pass
+
+
+# ---- lexer and parser ----
+
+BUILD = 1
+POOL = 2
+RULE = 3
+DEFAULT = 4
+EQUALS = 5
+COLON = 6
+PIPE = 7
+PIPE2 = 8
+IDENT = 9
+INCLUDE = 10
+INDENT = 11
+EOL = 12
+
+
+class LexerError(Exception):
+    pass
+
+
+class ParseError(Exception):
+    pass
+
+
+class NinjaParserEvents(object):
+    def __init__(self, parser):
+        self.parser = parser
+
+    def literal_token(self, word, in_path=False):
+        word = word.replace('$', '$$')
+        if in_path:
+            word = word.replace(' ', '$ ').replace(':', '$:')
+        return word
+
+    def variable_expansion_token(self, varname):
+        return '${%s}' % varname
+
+    def tokens_to_arg(self, tokens):
+        return ''.join(tokens or [])
+
+    def variable(self, name, arg):
+        pass
+
+    def begin_file(self):
+        pass
+
+    def end_file(self):
+        pass
+
+    def end_scope(self):
+        pass
+
+    def begin_pool(self, name):
+        pass
+
+    def begin_rule(self, name):
+        pass
+
+    def begin_build(self, out, iout, rule, in_, iin, orderdep):
+        pass
+
+    def default(self, targets):
+        pass
+
+
+class NinjaParser(object):
+    PATH_RE = r"[^$\s:|]+|\$[$ :]|\$[a-zA-Z0-9_-]+|\$\{[a-zA-Z0-9_.-]+\}"
+    STRING_RE = r"(" + PATH_RE + r"|[\s:|])(?:\r?\n)?"
+    TOPLEVEL_RE = r"([=:#]|\|\|?|^ +|(?:" + PATH_RE + r")+)\s*"
+    IDENT_RE = r"[a-zA-Z0-9_.-]+$"
+
+    InputFile = namedtuple('InputFile', 'filename iter lineno')
+
+    def __init__(self, filename, input):
+        self.stack = []
+        self.lineno = None
+        self.match_keyword = True
+        self.push(filename, input)
+
+    def __iter__(self):
+        return self
+
+    def __next__(self):
+        while len(self.stack):
+            try:
+                return next(self.stack[-1].iter)
+            except StopIteration:
+                self.pop()
+        raise StopIteration
+
+    def push(self, filename, input):
+        entered = self.InputFile(filename=filename or 'stdin',
+                                 iter=self._tokens(input),
+                                 lineno=self.lineno)
+        self.stack.append(entered)
+        self.lineno = 0
+        if filename is not None:
+            os.chdir(os.path.dirname(filename) or '.')
+
+    def pop(self):
+        left = self.stack.pop()
+        self.lineno = left.lineno
+        if len(self.stack):
+            os.chdir(os.path.dirname(self.stack[-1].filename) or '.')
+
+    def next_line(self, input):
+        line = next(input).rstrip()
+        self.lineno += 1
+        while len(line) and line[-1] == '$':
+            line = line[0:-1] + next(input).strip()
+            self.lineno += 1
+        return line
+
+    def print_token(self, tok):
+        if tok == EOL:
+            return "end of line"
+        if tok == BUILD:
+            return '"build"'
+        if tok == POOL:
+            return '"pool"'
+        if tok == RULE:
+            return '"rule"'
+        if tok == DEFAULT:
+            return '"default"'
+        if tok == EQUALS:
+            return '"="'
+        if tok == COLON:
+            return '":"'
+        if tok == PIPE:
+            return '"|"'
+        if tok == PIPE2:
+            return '"||"'
+        if tok == INCLUDE:
+            return '"include"'
+        if tok == IDENT:
+            return 'identifier'
+        return '"%s"' % tok
+
+    def error(self, msg):
+        raise LexerError("%s:%d: %s" % (self.stack[-1].filename, self.lineno, msg))
+
+    def parse_error(self, msg):
+        raise ParseError("%s:%d: %s" % (self.stack[-1].filename, self.lineno, msg))
+
+    def expected(self, expected, tok):
+        msg = "found %s, expected " % (self.print_token(tok), )
+        for i, exp_tok in enumerate(expected):
+            if i > 0:
+                msg = msg + (' or ' if i == len(expected) - 1 else ', ')
+            msg = msg + self.print_token(exp_tok)
+        self.parse_error(msg)
+
+    def find_tokens(self, regex, line):
+        # group 1 is used to distinguish expected input from bad characters
+        # group 2, which is really group 1 in regex, is returned
+        for m in re.finditer("(" + regex + ")|.", line):
+            if not m.group(1):
+                self.error("unexpected '%s'" % (m.group(0), ))
+            if m.group(2):
+                yield m.span(), m.group(2)
+
+    def _variable_tokens(self, value):
+        for span, match in self.find_tokens(self.STRING_RE, value):
+            yield match
+
+    def _tokens(self, input):
+        while True:
+            try:
+                line = self.next_line(input)
+            except StopIteration:
+                return
+            for span, match in self.find_tokens(self.TOPLEVEL_RE, line):
+                if match == ':':
+                    yield COLON
+                    continue
+                if match == '|':
+                    yield PIPE
+                    continue
+                if match == '||':
+                    yield PIPE2
+                    continue
+                if match[0] == ' ':
+                    yield INDENT
+                    continue
+                if match[0] == '=':
+                    value = line[span[0] + 1:].lstrip()
+                    yield EQUALS
+                    yield from self._variable_tokens(value)
+                    break
+                if match[0] == '#':
+                    break
+
+                # identifier
+                if self.match_keyword:
+                    if match == 'build':
+                        yield BUILD
+                        continue
+                    if match == 'pool':
+                        yield POOL
+                        continue
+                    if match == 'rule':
+                        yield RULE
+                        continue
+                    if match == 'default':
+                        yield DEFAULT
+                        continue
+                    if match == 'include':
+                        filename = line[span[0] + 8:].strip()
+                        self.push(filename, open(filename, 'r'))
+                        break
+                    if match == 'subninja':
+                        self.error('subninja is not supported')
+                yield match
+            yield EOL
+
+    def read_ident(self):
+        """Read an identifier, where the only allowed characters are
+           letter, numbers, dash, period and underscore."""
+        self.match_keyword = False
+        token = self.__next__()
+        self.match_keyword = True
+        if isinstance(token, str) and not re.match(self.IDENT_RE, token):
+            self.parse_error('variable expansion not allowed')
+        return token
+
+    def read_path(self):
+        """Read a space-delimited path or a non-alphanumeric token, and
+           return a generator that produces either the token alone, or
+           the components of the path (identifiers and variable expansions)."""
+        self.match_keyword = False
+        token = self.__next__()
+        self.match_keyword = True
+        if isinstance(token, str):
+            yield from self._variable_tokens(token)
+        else:
+            yield token
+
+    def parse(self, events):
+        global_var = True
+
+        def advance():
+            try:
+                return next(self)
+            except StopIteration:
+                return None
+
+        def look_for(*expected):
+            tok = advance()
+            if tok in expected:
+                return tok
+            self.expected(expected, tok)
+
+        def look_for_ident(*expected):
+            try:
+                tok = self.read_ident()
+            except StopIteration:
+                tok = None
+            if isinstance(tok, str) or tok in expected:
+                return tok
+            self.expected(expected + (IDENT,), tok)
+
+        def parse_assignment_rhs(gen, expected, in_path):
+            value = []
+            for tok in gen:
+                if not isinstance(tok, str):
+                    if tok in expected:
+                        break
+                    self.expected(expected + (IDENT,), tok)
+                if tok[0] != '$':
+                    value.append(events.literal_token(tok, in_path))
+                elif tok == '$ ' or tok == '$$' or tok == '$:':
+                    value.append(events.literal_token(tok[1], in_path))
+                else:
+                    var = tok[2:-1] if tok[1] == '{' else tok[1:]
+                    value.append(events.variable_expansion_token(var))
+            else:
+                # gen must have raised StopIteration, return None
+                # for consistency with advance()
+                tok = None
+            value = value if len(value) else None
+            return value, tok
+
+        def look_for_path(*expected):
+            # paths in build rules are parsed one space-separated token
+            # at a time and expanded
+            gen = self.read_path()
+            value, tok = parse_assignment_rhs(gen, expected, True)
+            return value, tok
+
+        def parse_assignment(tok):
+            assert isinstance(tok, str)
+            look_for(EQUALS)
+            name = tok
+            value, tok = parse_assignment_rhs(self, (EOL,), False)
+            assert tok == EOL
+            events.variable(name, events.tokens_to_arg(value))
+
+        def parse_build():
+            # parse outputs
+            out = []
+            iout = []
+            while True:
+                value, tok = look_for_path(COLON, PIPE)
+                if value is None:
+                    break
+                out.append(events.tokens_to_arg(value))
+            if tok == PIPE:
+                while True:
+                    value, tok = look_for_path(COLON)
+                    if value is None:
+                        break
+                    iout.append(events.tokens_to_arg(value))
+
+            # parse rule
+            assert tok == COLON
+            rule = look_for_ident()
+
+            # parse inputs and dependencies
+            in_ = []
+            iin = []
+            orderdep = []
+            while True:
+                value, tok = look_for_path(PIPE, PIPE2, EOL)
+                if value is None:
+                    break
+                in_.append(events.tokens_to_arg(value))
+            if tok == PIPE:
+                while True:
+                    value, tok = look_for_path(PIPE2, EOL)
+                    if value is None:
+                        break
+                    iin.append(events.tokens_to_arg(value))
+            if tok == PIPE2:
+                while True:
+                    value, tok = look_for_path(EOL)
+                    if value is None:
+                        break
+                    orderdep.append(events.tokens_to_arg(value))
+            assert tok == EOL
+            events.begin_build(out, iout, rule, in_, iin, orderdep)
+            nonlocal global_var
+            global_var = False
+
+        def parse_pool():
+            # pool declarations are ignored.  Just gobble all the variables
+            ident = look_for_ident()
+            look_for(EOL)
+            events.begin_pool(ident)
+            nonlocal global_var
+            global_var = False
+
+        def parse_rule():
+            ident = look_for_ident()
+            look_for(EOL)
+            events.begin_rule(ident)
+            nonlocal global_var
+            global_var = False
+
+        def parse_default():
+            idents = []
+            while True:
+                ident = look_for_ident(EOL)
+                if ident == EOL:
+                    break
+                idents.append(ident)
+            events.default(idents)
+
+        def parse_declaration(tok):
+            if tok == EOL:
+                return
+
+            nonlocal global_var
+            if tok == INDENT:
+                if global_var:
+                    self.parse_error('indented line outside rule or edge')
+                tok = look_for_ident(EOL)
+                if tok == EOL:
+                    return
+                parse_assignment(tok)
+                return
+
+            if not global_var:
+                events.end_scope()
+                global_var = True
+            if not tok:
+                return
+            if tok == POOL:
+                parse_pool()
+            elif tok == BUILD:
+                parse_build()
+            elif tok == RULE:
+                parse_rule()
+            elif tok == DEFAULT:
+                parse_default()
+            elif isinstance(tok, str):
+                parse_assignment(tok)
+            else:
+                self.expected((POOL, BUILD, RULE, INCLUDE, DEFAULT, IDENT), tok)
+
+        events.begin_file()
+        while True:
+            tok = advance()
+            parse_declaration(tok)
+            if not tok:
+                break
+        events.end_file()
+
+
+# ---- variable handling ----
+
+def expand(x, rule_vars=None, build_vars=None, global_vars=None):
+    if x is None:
+        return None
+    changed = True
+    while changed:
+        changed = False
+        matches = list(re.finditer(r'\$\$|\$\{([^}]*)\}', x))
+        if not matches:
+            break
+
+        # Reverse the match so that expanding later matches does not
+        # invalidate m.span() for earlier ones.  Do not reduce $$ to $
+        # until all variables are dealt with.
+        matches.reverse()
+        for m in matches:
+            if not m.group(1):
+                continue
+            changed = True
+            name = m.group(1)
+            if build_vars and name in build_vars:
+                value = build_vars[name]
+            elif rule_vars and name in rule_vars:
+                value = rule_vars[name]
+            elif name in global_vars:
+                value = global_vars[name]
+            else:
+                value = ''
+            start, end = m.span()
+            x = x[:start] + value + x[end:]
+    return x.replace('$$', '$')
+
+
+class Scope(object):
+    def __init__(self, events):
+        self.events = events
+
+    def on_left_scope(self):
+        pass
+
+    def on_variable(self, key, value):
+        pass
+
+
+class BuildScope(Scope):
+    def __init__(self, events, out, iout, rule, in_, iin, orderdep, rule_vars):
+        super().__init__(events)
+        self.rule = rule
+        self.out = [events.expand_and_normalize(x) for x in out]
+        self.in_ = [events.expand_and_normalize(x) for x in in_]
+        self.iin = [events.expand_and_normalize(x) for x in iin]
+        self.orderdep = [events.expand_and_normalize(x) for x in orderdep]
+        self.iout = [events.expand_and_normalize(x) for x in iout]
+        self.rule_vars = rule_vars
+        self.build_vars = dict()
+        self._define_variable('out', ' '.join(self.out))
+        self._define_variable('in', ' '.join(self.in_))
+
+    def expand(self, x):
+        return self.events.expand(x, self.rule_vars, self.build_vars)
+
+    def on_left_scope(self):
+        self.events.variable('out', self.build_vars['out'])
+        self.events.variable('in', self.build_vars['in'])
+        self.events.end_build(self, self.out, self.iout, self.rule, self.in_,
+                              self.iin, self.orderdep)
+
+    def _define_variable(self, key, value):
+        # The value has been expanded already, quote it for further
+        # expansion from rule variables
+        value = value.replace('$', '$$')
+        self.build_vars[key] = value
+
+    def on_variable(self, key, value):
+        # in and out are at the top of the lookup order and cannot
+        # be overridden.  Also, unlike what the manual says, build
+        # variables only lookup global variables.  They never lookup
+        # rule variables, earlier build variables, or in/out.
+        if key not in ('in', 'out'):
+            self._define_variable(key, self.events.expand(value))
+
+
+class RuleScope(Scope):
+    def __init__(self, events, name, vars_dict):
+        super().__init__(events)
+        self.name = name
+        self.vars_dict = vars_dict
+        self.generator = False
+
+    def on_left_scope(self):
+        self.events.end_rule(self, self.name)
+
+    def on_variable(self, key, value):
+        self.vars_dict[key] = value
+        if key == 'generator':
+            self.generator = True
+
+
+class NinjaParserEventsWithVars(NinjaParserEvents):
+    def __init__(self, parser):
+        super().__init__(parser)
+        self.rule_vars = defaultdict(lambda: dict())
+        self.global_vars = dict()
+        self.scope = None
+
+    def variable(self, name, value):
+        if self.scope:
+            self.scope.on_variable(name, value)
+        else:
+            self.global_vars[name] = self.expand(value)
+
+    def begin_build(self, out, iout, rule, in_, iin, orderdep):
+        if rule != 'phony' and rule not in self.rule_vars:
+            self.parser.parse_error("undefined rule '%s'" % rule)
+
+        self.scope = BuildScope(self, out, iout, rule, in_, iin, orderdep, self.rule_vars[rule])
+
+    def begin_pool(self, name):
+        # pool declarations are ignored.  Just gobble all the variables
+        self.scope = Scope(self)
+
+    def begin_rule(self, name):
+        if name in self.rule_vars:
+            self.parser.parse_error("duplicate rule '%s'" % name)
+        self.scope = RuleScope(self, name, self.rule_vars[name])
+
+    def end_scope(self):
+        self.scope.on_left_scope()
+        self.scope = None
+
+    # utility functions:
+
+    def expand(self, x, rule_vars=None, build_vars=None):
+        return expand(x, rule_vars, build_vars, self.global_vars)
+
+    def expand_and_normalize(self, x):
+        return os.path.normpath(self.expand(x))
+
+    # extra events not present in the superclass:
+
+    def end_build(self, scope, out, iout, rule, in_, iin, orderdep):
+        pass
+
+    def end_rule(self, scope, name):
+        pass
+
+
+# ---- test client that just prints back whatever it parsed  ----
+
+class Writer(NinjaParserEvents):
+    ARGS = argparse.ArgumentParser(description='Rewrite input build.ninja to stdout.')
+
+    def __init__(self, output, parser, args):
+        super().__init__(parser)
+        self.output = output
+        self.indent = ''
+        self.had_vars = False
+
+    def print(self, *args, **kwargs):
+        if len(args):
+            self.output.write(self.indent)
+        print(*args, **kwargs, file=self.output)
+
+    def variable(self, name, value):
+        self.print('%s = %s' % (name, value))
+        self.had_vars = True
+
+    def begin_scope(self):
+        self.indent = '  '
+        self.had_vars = False
+
+    def end_scope(self):
+        if self.had_vars:
+            self.print()
+        self.indent = ''
+        self.had_vars = False
+
+    def begin_pool(self, name):
+        self.print('pool %s' % name)
+        self.begin_scope()
+
+    def begin_rule(self, name):
+        self.print('rule %s' % name)
+        self.begin_scope()
+
+    def begin_build(self, outputs, implicit_outputs, rule, inputs, implicit, order_only):
+        all_outputs = list(outputs)
+        all_inputs = list(inputs)
+
+        if implicit:
+            all_inputs.append('|')
+            all_inputs.extend(implicit)
+        if order_only:
+            all_inputs.append('||')
+            all_inputs.extend(order_only)
+        if implicit_outputs:
+            all_outputs.append('|')
+            all_outputs.extend(implicit_outputs)
+
+        self.print('build %s: %s' % (' '.join(all_outputs),
+                                     ' '.join([rule] + all_inputs)))
+        self.begin_scope()
+
+    def default(self, targets):
+        self.print('default %s' % ' '.join(targets))
+
+
+# ---- emit compile_commands.json ----
+
+class Compdb(NinjaParserEventsWithVars):
+    ARGS = argparse.ArgumentParser(description='Emit compile_commands.json.')
+    ARGS.add_argument('rules', nargs='*',
+                      help='The ninja rules to emit compilation commands for.')
+
+    def __init__(self, output, parser, args):
+        super().__init__(parser)
+        self.output = output
+        self.rules = args.rules
+        self.sep = ''
+
+    def begin_file(self):
+        self.output.write('[')
+        self.directory = os.getcwd()
+
+    def print_entry(self, **entry):
+        entry['directory'] = self.directory
+        self.output.write(self.sep + json.dumps(entry))
+        self.sep = ',\n'
+
+    def begin_build(self, out, iout, rule, in_, iin, orderdep):
+        if in_ and rule in self.rules:
+            super().begin_build(out, iout, rule, in_, iin, orderdep)
+        else:
+            self.scope = Scope(self)
+
+    def end_build(self, scope, out, iout, rule, in_, iin, orderdep):
+        self.print_entry(command=scope.expand('${command}'), file=in_[0])
+
+    def end_file(self):
+        self.output.write(']\n')
+
+
+# ---- clean output files ----
+
+class Clean(NinjaParserEventsWithVars):
+    ARGS = argparse.ArgumentParser(description='Remove output build files.')
+    ARGS.add_argument('-g', dest='generator', action='store_true',
+                      help='clean generated files too')
+
+    def __init__(self, output, parser, args):
+        super().__init__(parser)
+        self.dry_run = args.dry_run
+        self.verbose = args.verbose or args.dry_run
+        self.generator = args.generator
+
+    def begin_file(self):
+        print('Cleaning... ', end=(None if self.verbose else ''), flush=True)
+        self.cnt = 0
+
+    def end_file(self):
+        print('%d files' % self.cnt)
+
+    def do_clean(self, *files):
+        for f in files:
+            if self.dry_run:
+                if os.path.exists(f):
+                    self.cnt += 1
+                    print('Would remove ' + f)
+                    continue
+            else:
+                try:
+                    os.unlink(f)
+                    self.cnt += 1
+                    if self.verbose:
+                        print('Removed ' + f)
+                except FileNotFoundError:
+                    pass
+
+    def end_build(self, scope, out, iout, rule, in_, iin, orderdep):
+        if not self.generator and scope.expand('${generator}'):
+            return
+        self.do_clean(*out, *iout)
+        depfile = scope.expand('${depfile}')
+        if depfile:
+            self.do_clean(depfile)
+
+
+# ---- convert build.ninja to makefile ----
+
+class Ninja2Make(NinjaParserEventsWithVars):
+    ARGS = argparse.ArgumentParser(description='Convert build.ninja to a Makefile.')
+    ARGS.add_argument('--clean', dest='emit_clean', action='store_true',
+                      help='Emit clean/distclean rules.')
+    ARGS.add_argument('--doublecolon', action='store_true',
+                      help='Emit double-colon rules for phony targets.')
+    ARGS.add_argument('--omit', metavar='TARGET', nargs='+',
+                      help='Targets to omit.')
+
+    def __init__(self, output, parser, args):
+        super().__init__(parser)
+        self.output = output
+
+        self.emit_clean = args.emit_clean
+        self.doublecolon = args.doublecolon
+        self.omit = set(args.omit)
+
+        if self.emit_clean:
+            self.omit.update(['clean', 'distclean'])
+
+        # Lists of targets are kept in memory and emitted only at the
+        # end because appending is really inefficient in GNU make.
+        # We only do it when it's O(#rules) or O(#variables), but
+        # never when it could be O(#targets).
+        self.depfiles = list()
+        self.build_vars = defaultdict(lambda: dict())
+        self.rule_targets = defaultdict(lambda: list())
+        self.all_outs = set()
+        self.all_ins = set()
+        self.all_phony = set()
+        self.seen_default = False
+
+    def print(self, *args, **kwargs):
+        print(*args, **kwargs, file=self.output)
+
+    def literal_token(self, word, in_path=False):
+        if in_path and ' ' in word:
+            self.parser.parse_error('Make does not support spaces in filenames')
+        return word.replace('$', '$$')
+
+    def print_phony(self, outs, ins):
+        targets = ' '.join(outs).replace('$', '$$')
+        deps = ' '.join(ins).replace('$', '$$')
+        deps = deps.strip()
+        if self.doublecolon:
+            self.print(targets + '::' + (' ' if deps else '') + deps + ';@:')
+        else:
+            self.print(targets + ':' + (' ' if deps else '') + deps)
+        self.all_phony.update(outs)
+
+    def begin_file(self):
+        self.print(r'# This is an automatically generated file, and it shows.')
+        self.print(r'ninja-default:')
+        self.print(r'.PHONY: ninja-default')
+        if self.emit_clean:
+            self.print(r'ninja-clean:: ninja-clean-start; $(if $V,,@)rm -f ${ninja-depfiles}')
+            self.print(r'ninja-clean-start:; $(if $V,,@echo Cleaning...)')
+            self.print(r'ninja-distclean:: clean')
+            self.print(r'.PHONY: ninja-clean ninja-clean-start ninja-distclean')
+            self.print_phony(['clean'], ['ninja-clean'])
+            self.print_phony(['distclean'], ['ninja-distclean'])
+        self.print(r'vpath')
+        self.print(r'NULL :=')
+        self.print(r'SPACE := ${NULL} #')
+        self.print(r'MAKEFLAGS += -rR')
+        self.print(r'define NEWLINE')
+        self.print(r'')
+        self.print(r'endef')
+        self.print(r'.var.in_newline = $(subst $(SPACE),$(NEWLINE),${.var.in})')
+        self.print(r"ninja-command = $(if $V,,@printf '%s\n' '$(subst ','\'',${.var.description})' && )${.var.command}")
+
+    def mangle_vars_for_make(self, x):
+        matches = list(re.finditer(r'\$\$|\$\{([^}]*)\}', x))
+
+        # Reverse the match so that expanding later matches does not
+        # invalidate m.span() for earlier ones.
+        matches.reverse()
+        for m in matches:
+            if not m.group(1):
+                continue
+            value = '${.var.' + m.group(1) + '}'
+            start, end = m.span()
+            x = x[:start] + value + x[end:]
+        return x
+
+    def end_file(self):
+        def natural_sort_key(s, _nsre=re.compile('([0-9]+)')):
+            return [int(text) if text.isdigit() else text.lower()
+                    for text in _nsre.split(s)]
+
+        self.print()
+        self.print('ninja-outputdirs :=')
+        for rule in self.rule_vars:
+            if rule == 'phony':
+                continue
+            self.print('ninja-targets-%s := %s' % (rule, ' '.join(self.rule_targets[rule])))
+            for name, value in self.rule_vars[rule].items():
+                self.print('$(ninja-targets-%s): .var.%s = %s' % (rule, name, self.mangle_vars_for_make(value)))
+            self.print('ninja-outputdirs += $(sort $(dir ${ninja-targets-%s}))' % rule)
+            self.print()
+        self.print('dummy := $(shell mkdir -p . $(sort $(ninja-outputdirs)))')
+        self.print('ninja-depfiles :=' + ' '.join(self.depfiles))
+        self.print('-include ${ninja-depfiles}')
+        self.print()
+        for targets in self.build_vars:
+            for name, value in self.build_vars[targets].items():
+                self.print('%s: .var.%s := %s' % (targets, name, value))
+            self.print()
+        if not self.seen_default:
+            default_targets = sorted(self.all_outs - self.all_ins, key=natural_sort_key)
+            self.print('ninja-default: ' + ' '.join(default_targets))
+
+        # This is a hack...  Meson declares input meson.build files as
+        # phony, because Ninja does not have an equivalent of Make's
+        # "path/to/file:" declaration that ignores "path/to/file" even
+        # if it is absent.  However, Makefile.ninja wants to depend on
+        # build.ninja, which in turn depends on these phony targets which
+        # would cause Makefile.ninja to be rebuilt in a loop.
+        phony_targets = sorted(self.all_phony - self.all_ins, key=natural_sort_key)
+        self.print('.PHONY: ' + ' '.join(phony_targets))
+
+    def variable(self, name, value):
+        super().variable(name, value)
+        if self.scope is None:
+            self.global_vars[name] = self.expand(value)
+            self.print('.var.%s := %s' % (name, self.global_vars[name]))
+        elif isinstance(self.scope, BuildScope):
+            # These have to be printed last, because they override rule variables
+            # but we cannot print rule variables until we know the list of targets
+            # for each rule.
+            self.build_vars[self.current_targets][name] = self.scope.build_vars[name]
+
+    def begin_build(self, out, iout, rule, in_, iin, orderdep):
+        if any(x in self.omit for x in out):
+            self.scope = Scope(self)
+            return
+
+        super().begin_build(out, iout, rule, in_, iin, orderdep)
+        self.rule_targets[rule] += self.scope.out
+        self.rule_targets[rule] += self.scope.iout
+
+        self.all_outs.update(self.scope.iout)
+        self.all_outs.update(self.scope.out)
+        self.all_ins.update(self.scope.in_)
+        self.all_ins.update(self.scope.iin)
+
+        if rule == 'phony':
+            # Phony rules treat order-only dependencies as normal deps
+            self.print_phony(self.scope.out + self.scope.iout,
+                             self.scope.in_ + self.scope.iin + self.scope.orderdep)
+            self.scope = Scope(self)
+            return
+
+        targets = ' '.join(self.scope.out + self.scope.iout).replace('$', '$$')
+        inputs = ' '.join(self.scope.in_ + self.scope.iin).replace('$', '$$')
+        orderonly = ' '.join(self.scope.orderdep).replace('$', '$$')
+        self.print('%s: %s | %s; ${ninja-command}' % (targets, inputs, orderonly))
+        self.current_targets = targets
+
+    def end_build(self, scope, out, iout, rule, in_, iin, orderdep):
+        depfile = scope.expand('${depfile}')
+        if depfile:
+            self.depfiles.append(depfile)
+        self.current_targets = None
+
+    def end_rule(self, scope, name):
+        if self.emit_clean:
+            self.print('ninja-%s:: ; $(if $V,,@)rm -f ${ninja-targets-%s}' % (
+                'distclean' if scope.generator else 'clean',
+                name))
+
+    def default(self, targets):
+        self.print("ninja-default: " + ' '.join(targets))
+        self.seen_default = True
+
+
+# ---- command line parsing ----
+
+# we cannot use subparsers because tools are chosen through the "-t"
+# option.
+
+class ToolAction(argparse.Action):
+    def __init__(self, option_strings, dest, choices, metavar='TOOL', nargs=None, **kwargs):
+        if nargs is not None:
+            raise ValueError("nargs not allowed")
+        super().__init__(option_strings, dest, required=True, choices=choices,
+                         metavar=metavar, **kwargs)
+
+    def __call__(self, parser, namespace, value, option_string):
+        tool = self.choices[value]
+        setattr(namespace, self.dest, tool)
+        tool.ARGS.prog = '%s %s %s' % (parser.prog, option_string, value)
+
+
+class ToolHelpAction(argparse.Action):
+    def __init__(self, option_strings, dest, nargs=None, **kwargs):
+        if nargs is not None:
+            raise ValueError("nargs not allowed")
+        super().__init__(option_strings, dest, nargs=0, **kwargs)
+
+    def __call__(self, parser, namespace, values, option_string=None):
+        if namespace.tool:
+            namespace.tool.ARGS.print_help()
+        else:
+            parser.print_help()
+        parser.exit()
+
+
+tools = {
+    'test': Writer,
+    'ninja2make': Ninja2Make,
+    'compdb': Compdb,
+    'clean': Clean,
+}
+
+parser = argparse.ArgumentParser(description='Process and transform build.ninja files.',
+                                 add_help=False)
+parser.add_argument('-C', metavar='DIR', dest='dir', default='.',
+                    help='change to DIR before doing anything else')
+parser.add_argument('-f', metavar='FILE', dest='file', default='build.ninja',
+                    help='specify input build file [default=build.ninja]')
+parser.add_argument('-n', dest='dry_run', action='store_true',
+                    help='do not actually do anything')
+parser.add_argument('-v', dest='verbose', action='store_true',
+                    help='be more verbose')
+
+parser.add_argument('-t', dest='tool', choices=tools, action=ToolAction,
+                    help='choose the tool to run')
+parser.add_argument('-h', '--help', action=ToolHelpAction,
+                    help='show this help message and exit')
+
+if len(sys.argv) >= 2 and sys.argv[1] == '--version':
+    print('1.8')
+    sys.exit(0)
+
+args, tool_args = parser.parse_known_args()
+args.tool.ARGS.parse_args(tool_args, args)
+
+os.chdir(args.dir)
+with open(args.file, 'r') as f:
+    parser = NinjaParser(args.file, f)
+    try:
+        events = args.tool(sys.stdout, parser, args)
+    except InvalidArgumentError as e:
+        parser.error(str(e))
+    parser.parse(events)
-- 
1.8.3.1

[Qemu-devel] [PATCH 4/7] libvhost-user: convert to Meson

[Paolo Bonzini]

Because libqemuutil.a is not converted yet, selected files have to be
compiled twice, once with Meson and once with Makefile.objs.  Apart
from this the conversion is straightforward.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Makefile                            | 10 ++++------
 Makefile.objs                       |  1 -
 contrib/libvhost-user/Makefile.objs |  1 -
 contrib/libvhost-user/meson.build   |  7 +++++++
 meson.build                         |  2 ++
 5 files changed, 13 insertions(+), 8 deletions(-)
 delete mode 100644 contrib/libvhost-user/Makefile.objs
 create mode 100644 contrib/libvhost-user/meson.build

diff --git a/Makefile b/Makefile
index b8f802c..6a3461e 100644
--- a/Makefile
+++ b/Makefile
@@ -426,7 +426,6 @@ dummy := $(call unnest-vars,, \
                 ivshmem-client-obj-y \
                 ivshmem-server-obj-y \
                 rdmacm-mux-obj-y \
-                libvhost-user-obj-y \
                 vhost-user-scsi-obj-y \
                 vhost-user-blk-obj-y \
                 vhost-user-input-obj-y \
@@ -529,7 +528,6 @@ Makefile: $(version-obj-y)
 # Build libraries
 
 libqemuutil.a: $(util-obj-y) $(trace-obj-y) $(stub-obj-y)
-libvhost-user.a: $(libvhost-user-obj-y) $(util-obj-y) $(stub-obj-y)
 
 ######################################################################
 
@@ -632,21 +630,21 @@ ivshmem-client$(EXESUF): $(ivshmem-client-obj-y) $(COMMON_LDADDS)
 ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS)
 	$(call LINK, $^)
 endif
-vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) libvhost-user.a
+vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) contrib/libvhost-user/libvhost-user.a
 	$(call LINK, $^)
-vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) libvhost-user.a
+vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) contrib/libvhost-user/libvhost-user.a
 	$(call LINK, $^)
 
 rdmacm-mux$(EXESUF): LIBS += "-libumad"
 rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
 	$(call LINK, $^)
 
-vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) $(libvhost-user-obj-y) libqemuutil.a libqemustub.a
+vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) contrib/libvhost-user/libvhost-user.a
 	$(call LINK, $^)
 
 ifdef CONFIG_VHOST_USER_INPUT
 ifdef CONFIG_LINUX
-vhost-user-input$(EXESUF): $(vhost-user-input-obj-y) libvhost-user.a libqemuutil.a
+vhost-user-input$(EXESUF): $(vhost-user-input-obj-y) contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
 	$(call LINK, $^)
 
 # build by default, do not install
diff --git a/Makefile.objs b/Makefile.objs
index c8337fa..4f2fa6a 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -116,7 +116,6 @@ qga-vss-dll-obj-y = qga/
 elf2dmp-obj-y = contrib/elf2dmp/
 ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
 ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
-libvhost-user-obj-y = contrib/libvhost-user/
 vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS)
 vhost-user-scsi.o-libs := $(LIBISCSI_LIBS)
 vhost-user-scsi-obj-y = contrib/vhost-user-scsi/
diff --git a/contrib/libvhost-user/Makefile.objs b/contrib/libvhost-user/Makefile.objs
deleted file mode 100644
index ef3778e..0000000
--- a/contrib/libvhost-user/Makefile.objs
+++ /dev/null
@@ -1 +0,0 @@
-libvhost-user-obj-y += libvhost-user.o libvhost-user-glib.o
diff --git a/contrib/libvhost-user/meson.build b/contrib/libvhost-user/meson.build
new file mode 100644
index 0000000..7ef610a
--- /dev/null
+++ b/contrib/libvhost-user/meson.build
@@ -0,0 +1,7 @@
+libvhost_user = static_library('vhost-user',
+               [files('libvhost-user.c', 'libvhost-user-glib.c'),
+                meson.source_root() / 'stubs/error-printf.c',
+                meson.source_root() / 'stubs/monitor.c',
+                meson.source_root() / 'util/error.c',
+                meson.source_root() / 'util/qemu-error.c',
+                meson.source_root() / 'util/memfd.c'])
diff --git a/meson.build b/meson.build
index b683d70..a6748f9 100644
--- a/meson.build
+++ b/meson.build
@@ -7,3 +7,5 @@ add_project_arguments(config_host['QEMU_CFLAGS'].split(),
                       language: 'c')
 add_project_arguments(config_host['QEMU_INCLUDES'].split(),
                       language: 'c')
+
+subdir('contrib/libvhost-user')
-- 
1.8.3.1

[Qemu-devel] [PATCH 5/7] vhost-user-blk: convert to Meson

[Paolo Bonzini]

The GLib pkg-config information is extracted from config-host.mak and
used to link vhost-user-blk.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitignore                           | 2 +-
 Makefile                             | 3 ---
 Makefile.objs                        | 1 -
 configure                            | 2 ++
 contrib/vhost-user-blk/Makefile.objs | 1 -
 contrib/vhost-user-blk/meson.build   | 3 +++
 meson.build                          | 4 ++++
 7 files changed, 10 insertions(+), 6 deletions(-)
 delete mode 100644 contrib/vhost-user-blk/Makefile.objs
 create mode 100644 contrib/vhost-user-blk/meson.build

diff --git a/.gitignore b/.gitignore
index fd6e6c3..20637a1 100644
--- a/.gitignore
+++ b/.gitignore
@@ -64,7 +64,7 @@
 /module_block.h
 /scsi/qemu-pr-helper
 /vhost-user-scsi
-/vhost-user-blk
+/contrib/vhost-user-blk/vhost-user-blk
 /fsdev/virtfs-proxy-helper
 *.tmp
 *.[1-9]
diff --git a/Makefile b/Makefile
index 6a3461e..da290fa 100644
--- a/Makefile
+++ b/Makefile
@@ -427,7 +427,6 @@ dummy := $(call unnest-vars,, \
                 ivshmem-server-obj-y \
                 rdmacm-mux-obj-y \
                 vhost-user-scsi-obj-y \
-                vhost-user-blk-obj-y \
                 vhost-user-input-obj-y \
                 vhost-user-gpu-obj-y \
                 qga-vss-dll-obj-y \
@@ -632,8 +631,6 @@ ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS)
 endif
 vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) contrib/libvhost-user/libvhost-user.a
 	$(call LINK, $^)
-vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) contrib/libvhost-user/libvhost-user.a
-	$(call LINK, $^)
 
 rdmacm-mux$(EXESUF): LIBS += "-libumad"
 rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
diff --git a/Makefile.objs b/Makefile.objs
index 4f2fa6a..644e2bd 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -119,7 +119,6 @@ ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
 vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS)
 vhost-user-scsi.o-libs := $(LIBISCSI_LIBS)
 vhost-user-scsi-obj-y = contrib/vhost-user-scsi/
-vhost-user-blk-obj-y = contrib/vhost-user-blk/
 rdmacm-mux-obj-y = contrib/rdmacm-mux/
 vhost-user-input-obj-y = contrib/vhost-user-input/
 vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
diff --git a/configure b/configure
index b8c3c58..f00a9e4 100755
--- a/configure
+++ b/configure
@@ -7364,6 +7364,8 @@ echo "CFLAGS_NOPIE=$CFLAGS_NOPIE" >> $config_host_mak
 echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak
 echo "QEMU_CXXFLAGS=$QEMU_CXXFLAGS" >> $config_host_mak
 echo "QEMU_INCLUDES=$QEMU_INCLUDES" >> $config_host_mak
+echo "GLIB_CFLAGS=$glib_cflags" >> $config_host_mak
+echo "GLIB_LIBS=$glib_libs" >> $config_host_mak
 if test "$sparse" = "yes" ; then
   echo "CC           := REAL_CC=\"\$(CC)\" cgcc"       >> $config_host_mak
   echo "CPP          := REAL_CC=\"\$(CPP)\" cgcc"      >> $config_host_mak
diff --git a/contrib/vhost-user-blk/Makefile.objs b/contrib/vhost-user-blk/Makefile.objs
deleted file mode 100644
index 72e2cdc..0000000
--- a/contrib/vhost-user-blk/Makefile.objs
+++ /dev/null
@@ -1 +0,0 @@
-vhost-user-blk-obj-y = vhost-user-blk.o
diff --git a/contrib/vhost-user-blk/meson.build b/contrib/vhost-user-blk/meson.build
new file mode 100644
index 0000000..6c00b3b
--- /dev/null
+++ b/contrib/vhost-user-blk/meson.build
@@ -0,0 +1,3 @@
+executable('vhost-user-blk', files('vhost-user-blk.c'),
+           link_with: libvhost_user,
+           dependencies: glib)
diff --git a/meson.build b/meson.build
index a6748f9..c4cad8f 100644
--- a/meson.build
+++ b/meson.build
@@ -8,4 +8,8 @@ add_project_arguments(config_host['QEMU_CFLAGS'].split(),
 add_project_arguments(config_host['QEMU_INCLUDES'].split(),
                       language: 'c')
 
+glib = declare_dependency(compile_args: config_host['GLIB_CFLAGS'].split(),
+                          link_args: config_host['GLIB_LIBS'].split())
+
 subdir('contrib/libvhost-user')
+subdir('contrib/vhost-user-blk')
-- 
1.8.3.1

[Qemu-devel] [PATCH 6/7] vhost-user-scsi: convert to Meson

[Paolo Bonzini]

The libiscsi pkg-config information is extracted from config-host.mak and
used to link vhost-user-blk.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitignore                            | 2 +-
 Makefile                              | 3 ---
 Makefile.objs                         | 3 ---
 contrib/vhost-user-scsi/Makefile.objs | 1 -
 contrib/vhost-user-scsi/meson.build   | 3 +++
 meson.build                           | 3 +++
 6 files changed, 7 insertions(+), 8 deletions(-)
 delete mode 100644 contrib/vhost-user-scsi/Makefile.objs
 create mode 100644 contrib/vhost-user-scsi/meson.build

diff --git a/.gitignore b/.gitignore
index 20637a1..3934eff 100644
--- a/.gitignore
+++ b/.gitignore
@@ -63,7 +63,7 @@
 /qemu-version.h.tmp
 /module_block.h
 /scsi/qemu-pr-helper
-/vhost-user-scsi
+/contrib/vhost-user-scsi/vhost-user-scsi
 /contrib/vhost-user-blk/vhost-user-blk
 /fsdev/virtfs-proxy-helper
 *.tmp
diff --git a/Makefile b/Makefile
index da290fa..bff097c 100644
--- a/Makefile
+++ b/Makefile
@@ -426,7 +426,6 @@ dummy := $(call unnest-vars,, \
                 ivshmem-client-obj-y \
                 ivshmem-server-obj-y \
                 rdmacm-mux-obj-y \
-                vhost-user-scsi-obj-y \
                 vhost-user-input-obj-y \
                 vhost-user-gpu-obj-y \
                 qga-vss-dll-obj-y \
@@ -629,8 +628,6 @@ ivshmem-client$(EXESUF): $(ivshmem-client-obj-y) $(COMMON_LDADDS)
 ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS)
 	$(call LINK, $^)
 endif
-vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) contrib/libvhost-user/libvhost-user.a
-	$(call LINK, $^)
 
 rdmacm-mux$(EXESUF): LIBS += "-libumad"
 rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
diff --git a/Makefile.objs b/Makefile.objs
index 644e2bd..adf2f92 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -116,9 +116,6 @@ qga-vss-dll-obj-y = qga/
 elf2dmp-obj-y = contrib/elf2dmp/
 ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
 ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
-vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS)
-vhost-user-scsi.o-libs := $(LIBISCSI_LIBS)
-vhost-user-scsi-obj-y = contrib/vhost-user-scsi/
 rdmacm-mux-obj-y = contrib/rdmacm-mux/
 vhost-user-input-obj-y = contrib/vhost-user-input/
 vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
diff --git a/contrib/vhost-user-scsi/Makefile.objs b/contrib/vhost-user-scsi/Makefile.objs
deleted file mode 100644
index e83a38a..0000000
--- a/contrib/vhost-user-scsi/Makefile.objs
+++ /dev/null
@@ -1 +0,0 @@
-vhost-user-scsi-obj-y = vhost-user-scsi.o
diff --git a/contrib/vhost-user-scsi/meson.build b/contrib/vhost-user-scsi/meson.build
new file mode 100644
index 0000000..b80398c
--- /dev/null
+++ b/contrib/vhost-user-scsi/meson.build
@@ -0,0 +1,3 @@
+executable('vhost-user-scsi', files('vhost-user-scsi.c'),
+           link_with: libvhost_user,
+           dependencies: [glib, libiscsi])
diff --git a/meson.build b/meson.build
index c4cad8f..bc7fbea 100644
--- a/meson.build
+++ b/meson.build
@@ -10,6 +10,9 @@ add_project_arguments(config_host['QEMU_INCLUDES'].split(),
 
 glib = declare_dependency(compile_args: config_host['GLIB_CFLAGS'].split(),
                           link_args: config_host['GLIB_LIBS'].split())
+libiscsi = declare_dependency(compile_args: config_host['LIBISCSI_CFLAGS'].split(),
+                          link_args: config_host['LIBISCSI_LIBS'].split())
 
 subdir('contrib/libvhost-user')
 subdir('contrib/vhost-user-blk')
+subdir('contrib/vhost-user-scsi')
-- 
1.8.3.1

[Qemu-devel] [PATCH 7/7] rdmacm-mux: convert to Meson

[Paolo Bonzini]

We can use config-host.mak to decide whether the tool has to be built,
apart from that the conversion is straightforward.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitignore                       | 1 +
 Makefile                         | 5 -----
 Makefile.objs                    | 1 -
 contrib/rdmacm-mux/Makefile.objs | 3 ---
 contrib/rdmacm-mux/meson.build   | 6 ++++++
 meson.build                      | 2 ++
 6 files changed, 9 insertions(+), 9 deletions(-)
 delete mode 100644 contrib/rdmacm-mux/Makefile.objs
 create mode 100644 contrib/rdmacm-mux/meson.build

diff --git a/.gitignore b/.gitignore
index 3934eff..b8d38a8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -63,6 +63,7 @@
 /qemu-version.h.tmp
 /module_block.h
 /scsi/qemu-pr-helper
+/contrib/rdmacm-mux/rdmacm-mux
 /contrib/vhost-user-scsi/vhost-user-scsi
 /contrib/vhost-user-blk/vhost-user-blk
 /fsdev/virtfs-proxy-helper
diff --git a/Makefile b/Makefile
index bff097c..713f301 100644
--- a/Makefile
+++ b/Makefile
@@ -425,7 +425,6 @@ dummy := $(call unnest-vars,, \
                 elf2dmp-obj-y \
                 ivshmem-client-obj-y \
                 ivshmem-server-obj-y \
-                rdmacm-mux-obj-y \
                 vhost-user-input-obj-y \
                 vhost-user-gpu-obj-y \
                 qga-vss-dll-obj-y \
@@ -629,10 +628,6 @@ ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS)
 	$(call LINK, $^)
 endif
 
-rdmacm-mux$(EXESUF): LIBS += "-libumad"
-rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
-	$(call LINK, $^)
-
 vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) contrib/libvhost-user/libvhost-user.a
 	$(call LINK, $^)
 
diff --git a/Makefile.objs b/Makefile.objs
index adf2f92..cf02f63 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -116,7 +116,6 @@ qga-vss-dll-obj-y = qga/
 elf2dmp-obj-y = contrib/elf2dmp/
 ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
 ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
-rdmacm-mux-obj-y = contrib/rdmacm-mux/
 vhost-user-input-obj-y = contrib/vhost-user-input/
 vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
 
diff --git a/contrib/rdmacm-mux/Makefile.objs b/contrib/rdmacm-mux/Makefile.objs
deleted file mode 100644
index 3df744a..0000000
--- a/contrib/rdmacm-mux/Makefile.objs
+++ /dev/null
@@ -1,3 +0,0 @@
-ifdef CONFIG_PVRDMA
-rdmacm-mux-obj-y = main.o
-endif
diff --git a/contrib/rdmacm-mux/meson.build b/contrib/rdmacm-mux/meson.build
new file mode 100644
index 0000000..8451756
--- /dev/null
+++ b/contrib/rdmacm-mux/meson.build
@@ -0,0 +1,6 @@
+if config_host['CONFIG_PVRDMA'] == 'y'
+  # if not found, CONFIG_PVRDMA should not be set
+  libumad = cc.find_library('ibumad', required: true)
+  executable('rdmacm-mux', files('main.c'),
+             dependencies: [glib, libumad])
+endif
diff --git a/meson.build b/meson.build
index bc7fbea..2fc6111 100644
--- a/meson.build
+++ b/meson.build
@@ -1,4 +1,5 @@
 project('qemu', 'c', meson_version: '>=0.50.999')
+cc = meson.get_compiler('c')
 
 kconfig = import('unstable-kconfig')
 config_host = kconfig.load(meson.current_build_dir() / 'config-host.mak')
@@ -14,5 +15,6 @@ libiscsi = declare_dependency(compile_args: config_host['LIBISCSI_CFLAGS'].split
                           link_args: config_host['LIBISCSI_LIBS'].split())
 
 subdir('contrib/libvhost-user')
+subdir('contrib/rdmacm-mux')
 subdir('contrib/vhost-user-blk')
 subdir('contrib/vhost-user-scsi')
-- 
1.8.3.1

Re: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration

[no-reply]

Patchew URL: https://patchew.org/QEMU/1560165301-39026-1-git-send-email-pbonzini@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration
Type: series
Message-id: 1560165301-39026-1-git-send-email-pbonzini@redhat.com

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

From https://github.com/patchew-project/qemu
   6960005212..19735c837a  master     -> master
From https://github.com/patchew-project/qemu
 * [new tag]               patchew/1560165301-39026-1-git-send-email-pbonzini@redhat.com -> patchew/1560165301-39026-1-git-send-email-pbonzini@redhat.com
Switched to a new branch 'test'
d681fb0695 rdmacm-mux: convert to Meson
2c12fcafd1 vhost-user-scsi: convert to Meson
7b87823a4c vhost-user-blk: convert to Meson
45233abe03 libvhost-user: convert to Meson
090f670ef8 configure: integrate Meson in the build system
9d9a4b1926 configure: set $PYTHON to a full path
d47d1b7f13 configure: do not include $(...) variables in config-host.mak

=== OUTPUT BEGIN ===
1/7 Checking commit d47d1b7f1366 (configure: do not include $(...) variables in config-host.mak)
2/7 Checking commit 9d9a4b192614 (configure: set $PYTHON to a full path)
3/7 Checking commit 090f670ef84c (configure: integrate Meson in the build system)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#104: 
new file mode 100644

WARNING: line over 80 characters
#283: FILE: scripts/ninjatool.py:160:
+        raise LexerError("%s:%d: %s" % (self.stack[-1].filename, self.lineno, msg))

WARNING: line over 80 characters
#286: FILE: scripts/ninjatool.py:163:
+        raise ParseError("%s:%d: %s" % (self.stack[-1].filename, self.lineno, msg))

ERROR: line over 90 characters
#668: FILE: scripts/ninjatool.py:545:
+        self.scope = BuildScope(self, out, iout, rule, in_, iin, orderdep, self.rule_vars[rule])

WARNING: line over 80 characters
#703: FILE: scripts/ninjatool.py:580:
+    ARGS = argparse.ArgumentParser(description='Rewrite input build.ninja to stdout.')

WARNING: line over 80 characters
#738: FILE: scripts/ninjatool.py:615:
+    def begin_build(self, outputs, implicit_outputs, rule, inputs, implicit, order_only):

WARNING: line over 80 characters
#843: FILE: scripts/ninjatool.py:720:
+    ARGS = argparse.ArgumentParser(description='Convert build.ninja to a Makefile.')

ERROR: line over 90 characters
#897: FILE: scripts/ninjatool.py:774:
+            self.print(r'ninja-clean:: ninja-clean-start; $(if $V,,@)rm -f ${ninja-depfiles}')

ERROR: line over 90 characters
#911: FILE: scripts/ninjatool.py:788:
+        self.print(r"ninja-command = $(if $V,,@printf '%s\n' '$(subst ','\'',${.var.description})' && )${.var.command}")

ERROR: line over 90 characters
#937: FILE: scripts/ninjatool.py:814:
+            self.print('ninja-targets-%s := %s' % (rule, ' '.join(self.rule_targets[rule])))

ERROR: line over 90 characters
#939: FILE: scripts/ninjatool.py:816:
+                self.print('$(ninja-targets-%s): .var.%s = %s' % (rule, name, self.mangle_vars_for_make(value)))

WARNING: line over 80 characters
#940: FILE: scripts/ninjatool.py:817:
+            self.print('ninja-outputdirs += $(sort $(dir ${ninja-targets-%s}))' % rule)

WARNING: line over 80 characters
#951: FILE: scripts/ninjatool.py:828:
+            default_targets = sorted(self.all_outs - self.all_ins, key=natural_sort_key)

WARNING: line over 80 characters
#960: FILE: scripts/ninjatool.py:837:
+        phony_targets = sorted(self.all_phony - self.all_ins, key=natural_sort_key)

WARNING: line over 80 characters
#969: FILE: scripts/ninjatool.py:846:
+            # These have to be printed last, because they override rule variables

WARNING: line over 80 characters
#970: FILE: scripts/ninjatool.py:847:
+            # but we cannot print rule variables until we know the list of targets

WARNING: line over 80 characters
#972: FILE: scripts/ninjatool.py:849:
+            self.build_vars[self.current_targets][name] = self.scope.build_vars[name]

WARNING: line over 80 characters
#991: FILE: scripts/ninjatool.py:868:
+                             self.scope.in_ + self.scope.iin + self.scope.orderdep)

WARNING: line over 80 characters
#998: FILE: scripts/ninjatool.py:875:
+        self.print('%s: %s | %s; ${ninja-command}' % (targets, inputs, orderonly))

ERROR: line over 90 characters
#1024: FILE: scripts/ninjatool.py:901:
+    def __init__(self, option_strings, dest, choices, metavar='TOOL', nargs=None, **kwargs):

WARNING: line over 80 characters
#1057: FILE: scripts/ninjatool.py:934:
+parser = argparse.ArgumentParser(description='Process and transform build.ninja files.',

total: 6 errors, 15 warnings, 1048 lines checked

Patch 3/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/7 Checking commit 45233abe03bb (libvhost-user: convert to Meson)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#73: 
deleted file mode 100644

total: 0 errors, 1 warnings, 58 lines checked

Patch 4/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
5/7 Checking commit 7b87823a4cbe (vhost-user-blk: convert to Meson)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#73: 
deleted file mode 100644

total: 0 errors, 1 warnings, 49 lines checked

Patch 5/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
6/7 Checking commit 2c12fcafd119 (vhost-user-scsi: convert to Meson)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#62: 
deleted file mode 100644

total: 0 errors, 1 warnings, 44 lines checked

Patch 6/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
7/7 Checking commit d681fb0695f1 (rdmacm-mux: convert to Meson)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#61: 
deleted file mode 100644

total: 0 errors, 1 warnings, 48 lines checked

Patch 7/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/1560165301-39026-1-git-send-email-pbonzini@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration

[no-reply]

Patchew URL: https://patchew.org/QEMU/1560165301-39026-1-git-send-email-pbonzini@redhat.com/



Hi,

This series failed build test on s390x host. Please find the details below.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
# Testing script will be invoked under the git checkout with
# HEAD pointing to a commit that has the patches applied on top of "base"
# branch
set -e
CC=$HOME/bin/cc
INSTALL=$PWD/install
BUILD=$PWD/build
mkdir -p $BUILD $INSTALL
SRC=$PWD
cd $BUILD
$SRC/configure --cc=$CC --prefix=$INSTALL
make -j4
# XXX: we need reliable clean up
# make check -j4 V=1
make install

echo
echo "=== ENV ==="
env

echo
echo "=== PACKAGES ==="
rpm -qa
=== TEST SCRIPT END ===

=== OUTPUT BEGIN ===

ERROR: Meson not found. Use --meson=/path/to/meson

=== OUTPUT END ===


The full log is available at
http://patchew.org/logs/1560165301-39026-1-git-send-email-pbonzini@redhat.com/testing.s390x/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration

[no-reply]

Patchew URL: https://patchew.org/QEMU/1560165301-39026-1-git-send-email-pbonzini@redhat.com/



Hi,

This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===

Configure options:
--enable-werror --target-list=x86_64-softmmu --prefix=/tmp/qemu-test/install --python=/usr/bin/python3 --enable-debug --enable-sanitizers --cxx=clang++ --cc=clang --host-cc=clang

ERROR: Meson not found. Use --meson=/path/to/meson

# QEMU configure log Mon Jun 10 12:40:26 UTC 2019
# Configured with: '/tmp/qemu-test/src/configure' '--enable-werror' '--target-list=x86_64-softmmu' '--prefix=/tmp/qemu-test/install' '--python=/usr/bin/python3' '--enable-debug' '--enable-sanitizers' '--cxx=clang++' '--cc=clang' '--host-cc=clang'
---
funcs: do_compiler do_cc compile_object check_define main
lines: 92 122 622 691 0
clang -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99 -c -o config-temp/qemu-conf.o config-temp/qemu-conf.c
config-temp/qemu-conf.c:2:2: error: __i386__ not defined
#error __i386__ not defined
 ^
1 error generated.
---
funcs: do_compiler do_cc compile_object check_define main
lines: 92 122 622 694 0
clang -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99 -c -o config-temp/qemu-conf.o config-temp/qemu-conf.c
config-temp/qemu-conf.c:2:2: error: __ILP32__ not defined
#error __ILP32__ not defined
 ^
1 error generated.


The full log is available at
http://patchew.org/logs/1560165301-39026-1-git-send-email-pbonzini@redhat.com/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Markus Armbruster]

Paolo Bonzini <pbonzini@redhat.com> writes:

> The Meson build system is integrated in the existing configure/make steps
> by invoking Meson from the configure script and converting Meson's build.ninja
> rules to an included Makefile.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  Makefile             |   9 +
>  configure            |  30 ++
>  meson.build          |   9 +
>  scripts/ninjatool.py | 964 +++++++++++++++++++++++++++++++++++++++++++++++++++

Uff.

>  4 files changed, 1012 insertions(+)
>  create mode 100644 meson.build
>  create mode 100644 scripts/ninjatool.py
>
> diff --git a/Makefile b/Makefile
> index 8e2fc66..b8f802c 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -48,6 +48,15 @@ git-submodule-update:
>  endif
>  endif
>  
> +export NINJA=./ninjatool
> +Makefile.ninja: build.ninja ninjatool
> +	./ninjatool -t ninja2make --omit dist uninstall < $< > $@
> +-include Makefile.ninja
> +
> +ninjatool: $(SRC_PATH)/scripts/ninjatool.py
> +	sed -e '1c\' -e '#! $(PYTHON)' $< > $@
> +	chmod +x $@

Why do we need this here, but not for other Python scripts?

We have 39 Python scripts with #!/usr/bin/env python, one with
#!/usr/bin/env python2, and 12 with #!/usr/bin/python.  The Makefiles
generally use $(PYTHON) SCRIPT ARGS...

> +
>  .git-submodule-status: git-submodule-update config-host.mak
>  
>  # Check that we're not trying to do an out-of-tree build from
> diff --git a/configure b/configure
> index 0814a5f..b8c3c58 100755
> --- a/configure
> +++ b/configure
> @@ -493,6 +493,7 @@ docker="no"
>  debug_mutex="no"
>  libpmem=""
>  default_devices="yes"
> +meson=meson
>  
>  # cross compilers defaults, can be overridden with --cross-cc-ARCH
>  cross_cc_aarch64="aarch64-linux-gnu-gcc"
> @@ -983,6 +984,8 @@ for opt do
>    ;;
>    --python=*) python="$optarg"
>    ;;
> +  --meson=*) meson="$optarg"
> +  ;;
>    --gcov=*) gcov_tool="$optarg"
>    ;;
>    --smbd=*) smbd="$optarg"
> @@ -1685,6 +1688,7 @@ Advanced options (experts only):
>    --make=MAKE              use specified make [$make]
>    --install=INSTALL        use specified install [$install]
>    --python=PYTHON          use specified python [$python]
> +  --meson=PYTHON           use specified meson [$meson]
>    --smbd=SMBD              use specified smbd [$smbd]
>    --with-git=GIT           use specified git [$git]
>    --static                 enable static build [$static]
> @@ -1850,6 +1854,11 @@ then
>      error_exit "Python not found. Use --python=/path/to/python"
>  fi
>  
> +if ! has "$meson"
> +then
> +    error_exit "Meson not found. Use --meson=/path/to/meson"
> +fi
> +
>  # Note that if the Python conditional here evaluates True we will exit
>  # with status 1 which is a shell 'false' value.
>  if ! $python -c 'import sys; sys.exit(sys.version_info < (2,7))'; then
> @@ -7983,6 +7992,27 @@ echo "# Automatically generated by configure - do not modify" > "$iotests_common
>  echo >> "$iotests_common_env"
>  echo "export PYTHON='$python'" >> "$iotests_common_env"
>  
> +# bootstrap ninjatool, we need it before Make runs
> +if ! test -x ninjatool; then
> +  sed -e '1c\' -e "#! $python" ${source_path}/scripts/ninjatool.py > ninjatool
> +  chmod +x ninjatool
> +fi
> +rm -rf meson-private meson-info meson-logs

Ignorant question: why do we need configure remove this stuff?

> +NINJA=$PWD/ninjatool $python $meson setup \

This prints

    /usr/bin/python3: can't open file 'meson': [Errno 2] No such file or directory

for me, then goes on happily.

For what it's worth:

    $ type meson
    meson is /usr/bin/meson

Are you sure you want to override /usr/bin/meson's #! line?

If I drop $python, I get

    meson.build:1:0: ERROR: Meson version is 0.50.1 but project requires >=0.50.999.

which is expected.

It's too hot right for me now to figure out how to obtain a suitable
version.

> +	--prefix "$prefix" \
> +	--libdir "$libdir" \
> +	--libexecdir "$libexecdir" \
> +	--bindir "$bindir" \
> +	--includedir "$includedir" \
> +	--datadir "$datadir" \
> +	--mandir "$mandir" \
> +	--sysconfdir "$sysconfdir" \
> +	--localstatedir "$local_statedir" \
> +	$(test "$strip_opt" = yes && echo --strip) \
> +	--buildtype $(if test "$debug" = yes; then echo debug; else echo release; fi) \
> +	"$PWD" "$source_path"
> +
> +
>  # Save the configure command line for later reuse.
>  cat <<EOD >config.status
>  #!/bin/sh
> diff --git a/meson.build b/meson.build
> new file mode 100644
> index 0000000..b683d70
> --- /dev/null
> +++ b/meson.build
> @@ -0,0 +1,9 @@
> +project('qemu', 'c', meson_version: '>=0.50.999')
> +
> +kconfig = import('unstable-kconfig')
> +config_host = kconfig.load(meson.current_build_dir() / 'config-host.mak')
> +
> +add_project_arguments(config_host['QEMU_CFLAGS'].split(),
> +                      language: 'c')
> +add_project_arguments(config_host['QEMU_INCLUDES'].split(),
> +                      language: 'c')
> diff --git a/scripts/ninjatool.py b/scripts/ninjatool.py
> new file mode 100644
> index 0000000..6d90919
> --- /dev/null
> +++ b/scripts/ninjatool.py
[Lots of code...]

Did you write ninjatool.py specifically for QEMU, or did you steal it
(or parts) somewhere?

Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Paolo Bonzini]

On 26/06/19 19:34, Markus Armbruster wrote:
> Paolo Bonzini <pbonzini@redhat.com> writes:
> 
>> The Meson build system is integrated in the existing configure/make steps
>> by invoking Meson from the configure script and converting Meson's build.ninja
>> rules to an included Makefile.
>>
>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>> ---
>>  Makefile             |   9 +
>>  configure            |  30 ++
>>  meson.build          |   9 +
>>  scripts/ninjatool.py | 964 +++++++++++++++++++++++++++++++++++++++++++++++++++
> 
> Uff.

What, the 9+30+9 or the 964? :)

>> +export NINJA=./ninjatool
>> +Makefile.ninja: build.ninja ninjatool
>> +	./ninjatool -t ninja2make --omit dist uninstall < $< > $@
>> +-include Makefile.ninja
>> +
>> +ninjatool: $(SRC_PATH)/scripts/ninjatool.py
>> +	sed -e '1c\' -e '#! $(PYTHON)' $< > $@
>> +	chmod +x $@
> 
> Why do we need this here, but not for other Python scripts?
> 
> We have 39 Python scripts with #!/usr/bin/env python, one with
> #!/usr/bin/env python2, and 12 with #!/usr/bin/python.  The Makefiles
> generally use $(PYTHON) SCRIPT ARGS...

Because this script is invoked by meson too.  While my patch to support
a NINJA environment variable was accepted (and we point it to
ninjatool), meson invokes it as "$(NINJA)" --foo with no splitting, so
we need the above change.

But I can do better than this.  I can generate the file directly from
meson.build:

   configure_file(input: files('scripts/ninjatool.py'),
                  output: 'ninjatool',
                  configuration: config_host)

and avoid code duplication between configure and Makefile.

>> +rm -rf meson-private meson-info meson-logs
> 
> Ignorant question: why do we need configure remove this stuff?

By default, reinvoking meson complains that you probably want to
reconfigure (a la config.status) because there is already a build tree
here.  Our configure script always reconfigures the tree from scratch.
 So I just cause meson to fail that check.

>> +NINJA=$PWD/ninjatool $python $meson setup \
> 
> This prints
> 
>     /usr/bin/python3: can't open file 'meson': [Errno 2] No such file or directory
> 
> for me, then goes on happily.
> 
> For what it's worth:
> 
>     $ type meson
>     meson is /usr/bin/meson
> 
> Are you sure you want to override /usr/bin/meson's #! line?

No I'm not.  It should be just $meson, as you do below, though it could
also be $python "$(type -p meson)" if I wanted to override---but
certainly not $python $meson.  It worked for me because I was pointing
$meson to an absolute path under my home directory.

There is actually one reason to override it, which is why I did it in
the patch.  Meson has an automagic way to invoke all Python programs
with the same interpreter that was used to run meson itself (you make
the script non-executable and add "#! /usr/bin/env python3" as the first
line).  If we want to use that feature, we should invoke meson with $python.

> If I drop $python, I get
> 
>     meson.build:1:0: ERROR: Meson version is 0.50.1 but project requires >=0.50.999.
> 
> which is expected.
> 
> It's too hot right for me now to figure out how to obtain a suitable
> version.

FWIW 0.51.0 has now been released so you could grab it from Koji.

>> diff --git a/scripts/ninjatool.py b/scripts/ninjatool.py
>> new file mode 100644
>> index 0000000..6d90919
>> --- /dev/null
>> +++ b/scripts/ninjatool.py
> [Lots of code...]
> 
> Did you write ninjatool.py specifically for QEMU, or did you steal it
> (or parts) somewhere?

I wrote it specifically for QEMU.

Paolo

Re: [Qemu-devel] [PATCH 4/7] libvhost-user: convert to Meson

[Markus Armbruster]

Paolo Bonzini <pbonzini@redhat.com> writes:

> Because libqemuutil.a is not converted yet, selected files have to be
> compiled twice, once with Meson and once with Makefile.objs.  Apart
> from this the conversion is straightforward.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  Makefile                            | 10 ++++------
>  Makefile.objs                       |  1 -
>  contrib/libvhost-user/Makefile.objs |  1 -
>  contrib/libvhost-user/meson.build   |  7 +++++++
>  meson.build                         |  2 ++
>  5 files changed, 13 insertions(+), 8 deletions(-)
>  delete mode 100644 contrib/libvhost-user/Makefile.objs
>  create mode 100644 contrib/libvhost-user/meson.build
>
> diff --git a/Makefile b/Makefile
> index b8f802c..6a3461e 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -426,7 +426,6 @@ dummy := $(call unnest-vars,, \
>                  ivshmem-client-obj-y \
>                  ivshmem-server-obj-y \
>                  rdmacm-mux-obj-y \
> -                libvhost-user-obj-y \
>                  vhost-user-scsi-obj-y \
>                  vhost-user-blk-obj-y \
>                  vhost-user-input-obj-y \
> @@ -529,7 +528,6 @@ Makefile: $(version-obj-y)
>  # Build libraries
>  
>  libqemuutil.a: $(util-obj-y) $(trace-obj-y) $(stub-obj-y)
> -libvhost-user.a: $(libvhost-user-obj-y) $(util-obj-y) $(stub-obj-y)
>  
>  ######################################################################
>  
> @@ -632,21 +630,21 @@ ivshmem-client$(EXESUF): $(ivshmem-client-obj-y) $(COMMON_LDADDS)
>  ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS)
>  	$(call LINK, $^)
>  endif
> -vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) libvhost-user.a
> +vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) contrib/libvhost-user/libvhost-user.a
>  	$(call LINK, $^)
> -vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) libvhost-user.a
> +vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) contrib/libvhost-user/libvhost-user.a
>  	$(call LINK, $^)
>  
>  rdmacm-mux$(EXESUF): LIBS += "-libumad"
>  rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
>  	$(call LINK, $^)
>  
> -vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) $(libvhost-user-obj-y) libqemuutil.a libqemustub.a
> +vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) contrib/libvhost-user/libvhost-user.a
>  	$(call LINK, $^)
>  
>  ifdef CONFIG_VHOST_USER_INPUT
>  ifdef CONFIG_LINUX
> -vhost-user-input$(EXESUF): $(vhost-user-input-obj-y) libvhost-user.a libqemuutil.a
> +vhost-user-input$(EXESUF): $(vhost-user-input-obj-y) contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
>  	$(call LINK, $^)
>  
>  # build by default, do not install

This hunk is only due to the move of libvhost-user.a from the root of
the build tree to contrib/libvhost-user/.  I'm not counting it as "meson
churn".

> diff --git a/Makefile.objs b/Makefile.objs
> index c8337fa..4f2fa6a 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -116,7 +116,6 @@ qga-vss-dll-obj-y = qga/
>  elf2dmp-obj-y = contrib/elf2dmp/
>  ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
>  ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
> -libvhost-user-obj-y = contrib/libvhost-user/
>  vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS)
>  vhost-user-scsi.o-libs := $(LIBISCSI_LIBS)
>  vhost-user-scsi-obj-y = contrib/vhost-user-scsi/
> diff --git a/contrib/libvhost-user/Makefile.objs b/contrib/libvhost-user/Makefile.objs
> deleted file mode 100644
> index ef3778e..0000000
> --- a/contrib/libvhost-user/Makefile.objs
> +++ /dev/null
> @@ -1 +0,0 @@
> -libvhost-user-obj-y += libvhost-user.o libvhost-user-glib.o
> diff --git a/contrib/libvhost-user/meson.build b/contrib/libvhost-user/meson.build
> new file mode 100644
> index 0000000..7ef610a
> --- /dev/null
> +++ b/contrib/libvhost-user/meson.build
> @@ -0,0 +1,7 @@
> +libvhost_user = static_library('vhost-user',
> +               [files('libvhost-user.c', 'libvhost-user-glib.c'),

These are the "selected files [that] have to be compiled twice, once
with Meson and once with Makefile.objs":

> +                meson.source_root() / 'stubs/error-printf.c',
> +                meson.source_root() / 'stubs/monitor.c',
> +                meson.source_root() / 'util/error.c',
> +                meson.source_root() / 'util/qemu-error.c',
> +                meson.source_root() / 'util/memfd.c'])

Quite okay for a PoC.

> diff --git a/meson.build b/meson.build
> index b683d70..a6748f9 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -7,3 +7,5 @@ add_project_arguments(config_host['QEMU_CFLAGS'].split(),
>                        language: 'c')
>  add_project_arguments(config_host['QEMU_INCLUDES'].split(),
>                        language: 'c')
> +
> +subdir('contrib/libvhost-user')

In your cover letter, you wrote "it's a simple example of how the
current build system causes clutter in the toplevel Makefiles and how
interrelated the Makefile, Makefile.objs and contrib/*/Makefile.objs
file are even for such simple executables."  True.  Before this patch,
the logic for building libvhost-user.a is split over Makefile (two
lines), Makefile.obj (one) and contrib/libvhost-user/Makefile.objs
(one).  Afterwards, its all in one place.

Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Daniel P. Berrangé]

On Mon, Jun 10, 2019 at 01:14:57PM +0200, Paolo Bonzini wrote:
> The Meson build system is integrated in the existing configure/make steps
> by invoking Meson from the configure script and converting Meson's build.ninja
> rules to an included Makefile.

Why did you take the route of converting ninja rules into makefile
rules, as opposed to just having some stub makefile rules which
directly invoke ninja where needed ?

Obviously this series is just some initial integration, but eventually
when we get a complete conversion, I think it will look pretty wierd
if we're still converting ninja to make.

Part of the downside of our current build system is that although it
uses make, the usage is wierd and QEMU specific structure. It would
be a shame to pick meson and then again use it is a way that is wierd
and QEMU specific.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Christophe de Dinechin]

Markus Armbruster writes:

> Paolo Bonzini <pbonzini@redhat.com> writes:
>
>> The Meson build system is integrated in the existing configure/make steps
>> by invoking Meson from the configure script and converting Meson's build.ninja
>> rules to an included Makefile.
>>
>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>> ---
>>  Makefile             |   9 +
>>  configure            |  30 ++
>>  meson.build          |   9 +
>>  scripts/ninjatool.py | 964 +++++++++++++++++++++++++++++++++++++++++++++++++++
>
> Uff.
>
>>  4 files changed, 1012 insertions(+)
>>  create mode 100644 meson.build
>>  create mode 100644 scripts/ninjatool.py
>>
>> diff --git a/Makefile b/Makefile
>> index 8e2fc66..b8f802c 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -48,6 +48,15 @@ git-submodule-update:
>>  endif
>>  endif
>>
>> +export NINJA=./ninjatool
>> +Makefile.ninja: build.ninja ninjatool
>> +	./ninjatool -t ninja2make --omit dist uninstall < $< > $@
>> +-include Makefile.ninja
>> +
>> +ninjatool: $(SRC_PATH)/scripts/ninjatool.py
>> +	sed -e '1c\' -e '#! $(PYTHON)' $< > $@
>> +	chmod +x $@
>
> Why do we need this here, but not for other Python scripts?
>
> We have 39 Python scripts with #!/usr/bin/env python, one with
> #!/usr/bin/env python2, and 12 with #!/usr/bin/python.  The Makefiles
> generally use $(PYTHON) SCRIPT ARGS...
>
>> +
>>  .git-submodule-status: git-submodule-update config-host.mak
>>
>>  # Check that we're not trying to do an out-of-tree build from
>> diff --git a/configure b/configure
>> index 0814a5f..b8c3c58 100755
>> --- a/configure
>> +++ b/configure
>> @@ -493,6 +493,7 @@ docker="no"
>>  debug_mutex="no"
>>  libpmem=""
>>  default_devices="yes"
>> +meson=meson
>>
>>  # cross compilers defaults, can be overridden with --cross-cc-ARCH
>>  cross_cc_aarch64="aarch64-linux-gnu-gcc"
>> @@ -983,6 +984,8 @@ for opt do
>>    ;;
>>    --python=*) python="$optarg"
>>    ;;
>> +  --meson=*) meson="$optarg"
>> +  ;;
>>    --gcov=*) gcov_tool="$optarg"
>>    ;;
>>    --smbd=*) smbd="$optarg"
>> @@ -1685,6 +1688,7 @@ Advanced options (experts only):
>>    --make=MAKE              use specified make [$make]
>>    --install=INSTALL        use specified install [$install]
>>    --python=PYTHON          use specified python [$python]
>> +  --meson=PYTHON           use specified meson [$meson]

Typo: --meson=MESON

>>    --smbd=SMBD              use specified smbd [$smbd]
>>    --with-git=GIT           use specified git [$git]
>>    --static                 enable static build [$static]
>> @@ -1850,6 +1854,11 @@ then
>>      error_exit "Python not found. Use --python=/path/to/python"
>>  fi
>>
>> +if ! has "$meson"
>> +then
>> +    error_exit "Meson not found. Use --meson=/path/to/meson"
>> +fi
>> +
>>  # Note that if the Python conditional here evaluates True we will exit
>>  # with status 1 which is a shell 'false' value.
>>  if ! $python -c 'import sys; sys.exit(sys.version_info < (2,7))'; then
>> @@ -7983,6 +7992,27 @@ echo "# Automatically generated by configure - do not modify" > "$iotests_common
>>  echo >> "$iotests_common_env"
>>  echo "export PYTHON='$python'" >> "$iotests_common_env"
>>
>> +# bootstrap ninjatool, we need it before Make runs
>> +if ! test -x ninjatool; then
>> +  sed -e '1c\' -e "#! $python" ${source_path}/scripts/ninjatool.py > ninjatool
>> +  chmod +x ninjatool
>> +fi
>> +rm -rf meson-private meson-info meson-logs
>
> Ignorant question: why do we need configure remove this stuff?
>
>> +NINJA=$PWD/ninjatool $python $meson setup \
>
> This prints
>
>     /usr/bin/python3: can't open file 'meson': [Errno 2] No such file or directory
>
> for me, then goes on happily.
>
> For what it's worth:
>
>     $ type meson
>     meson is /usr/bin/meson
>
> Are you sure you want to override /usr/bin/meson's #! line?
>
> If I drop $python, I get
>
>     meson.build:1:0: ERROR: Meson version is 0.50.1 but project requires >=0.50.999.
>
> which is expected.
>
> It's too hot right for me now to figure out how to obtain a suitable
> version.
>
>> +	--prefix "$prefix" \
>> +	--libdir "$libdir" \
>> +	--libexecdir "$libexecdir" \
>> +	--bindir "$bindir" \
>> +	--includedir "$includedir" \
>> +	--datadir "$datadir" \
>> +	--mandir "$mandir" \
>> +	--sysconfdir "$sysconfdir" \
>> +	--localstatedir "$local_statedir" \
>> +	$(test "$strip_opt" = yes && echo --strip) \
>> +	--buildtype $(if test "$debug" = yes; then echo debug; else echo release; fi) \
>> +	"$PWD" "$source_path"
>> +
>> +
>>  # Save the configure command line for later reuse.
>>  cat <<EOD >config.status
>>  #!/bin/sh
>> diff --git a/meson.build b/meson.build
>> new file mode 100644
>> index 0000000..b683d70
>> --- /dev/null
>> +++ b/meson.build
>> @@ -0,0 +1,9 @@
>> +project('qemu', 'c', meson_version: '>=0.50.999')
>> +
>> +kconfig = import('unstable-kconfig')
>> +config_host = kconfig.load(meson.current_build_dir() / 'config-host.mak')
>> +
>> +add_project_arguments(config_host['QEMU_CFLAGS'].split(),
>> +                      language: 'c')
>> +add_project_arguments(config_host['QEMU_INCLUDES'].split(),
>> +                      language: 'c')
>> diff --git a/scripts/ninjatool.py b/scripts/ninjatool.py
>> new file mode 100644
>> index 0000000..6d90919
>> --- /dev/null
>> +++ b/scripts/ninjatool.py
> [Lots of code...]
>
> Did you write ninjatool.py specifically for QEMU, or did you steal it
> (or parts) somewhere?


--
Cheers,
Christophe de Dinechin (IRC c3d)

Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Paolo Bonzini]

On 27/06/19 11:03, Daniel P. Berrangé wrote:
> On Mon, Jun 10, 2019 at 01:14:57PM +0200, Paolo Bonzini wrote:
>> The Meson build system is integrated in the existing configure/make steps
>> by invoking Meson from the configure script and converting Meson's build.ninja
>> rules to an included Makefile.
> 
> Why did you take the route of converting ninja rules into makefile
> rules, as opposed to just having some stub makefile rules which
> directly invoke ninja where needed ?

There are two parts of this.  One is practical and has to do with
supporting a step-by-step transition.  Using ninja2make makes it trivial
to have make build products that depend on meson build products, and
this way bottom up is a natural direction to do the conversion, which is
bottom up.  You'd start from libqemuutil.a and code generators (tracing
+ QAPI), then go to the tools and the emulators.

The second is a design decision that simplifying the Make/meson
integration is *not* a goal.  Rather the goals are: 1) making the
transition easier on developers; 2) avoiding magic in meson.build at all
costs.  More specifically:

- it should remain trivial to do things that used to be trivial, and
most "make" invocations should be kept the same at least until
everything is converted and we can perhaps declare a flag day.  People
are used to "make check" or "make subdir-x86_64-softmmu", those should
continue to work while the transition is in progress.

- it should be possible to modify meson.build without knowing QEMU
specific details, and that should be _already_ possible now at the
beginning of the transition (to avoid creating technical debt).  This
means keeping the magic confined in Makefile rules and external scripts,
while having a pretty plain meson.build.

I expect testing might also require some hand-holding, because "meson
test" does not integrate with "make -j" and to keep supporting our "make
check-*" targets.  However, until the make->ninja flag day we could
generate tap-driver Makefile rules from "meson introspect --tests"
output.  Basically I'm dropping Makefile magic in favor of build rule
generators are written in high-level languages.

> Obviously this series is just some initial integration, but eventually
> when we get a complete conversion, I think it will look pretty wierd
> if we're still converting ninja to make.

I agree; once all the build rules are converted the Makefile could be as
simple as

	all:
	include config.mak
	include tests/docker/Makefile.include
	include tests/vm/Makefile.include
	.NOTPARALLEL:
	%:
		ninja $@

	subdir-%-softmmu:
		ninja qemu-system-$*

	subdir-%-linux-user:
		ninja qemu-$*

	check:
		$(MESON) test

	check-%:
		$(MESON) test --suite $*

etc. (and likewise the configure script could just translate the command
line options before invoking meson).  But for now, since rules are
written half in meson and half in make, ninja2make seems the most
transparent way to integrate the two.

> Part of the downside of our current build system is that although it
> uses make, the usage is wierd and QEMU specific structure. It would
> be a shame to pick meson and then again use it is a way that is wierd
> and QEMU specific.

I agree, this is why it's important to have at least a standard meson.build.

Some knowledge of config-host.mak is needed, because meson.build uses
declare_dependency() instead of dependency() to link with libraries that
were already found by configure, but that's it.

Paolo

Re: [Qemu-devel] [PATCH 6/7] vhost-user-scsi: convert to Meson

[Markus Armbruster]

Paolo Bonzini <pbonzini@redhat.com> writes:

> The libiscsi pkg-config information is extracted from config-host.mak and
> used to link vhost-user-blk.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  .gitignore                            | 2 +-
>  Makefile                              | 3 ---
>  Makefile.objs                         | 3 ---
>  contrib/vhost-user-scsi/Makefile.objs | 1 -
>  contrib/vhost-user-scsi/meson.build   | 3 +++
>  meson.build                           | 3 +++
>  6 files changed, 7 insertions(+), 8 deletions(-)
>  delete mode 100644 contrib/vhost-user-scsi/Makefile.objs
>  create mode 100644 contrib/vhost-user-scsi/meson.build
>
> diff --git a/.gitignore b/.gitignore
> index 20637a1..3934eff 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -63,7 +63,7 @@
>  /qemu-version.h.tmp
>  /module_block.h
>  /scsi/qemu-pr-helper
> -/vhost-user-scsi
> +/contrib/vhost-user-scsi/vhost-user-scsi
>  /contrib/vhost-user-blk/vhost-user-blk
>  /fsdev/virtfs-proxy-helper
>  *.tmp
> diff --git a/Makefile b/Makefile
> index da290fa..bff097c 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -426,7 +426,6 @@ dummy := $(call unnest-vars,, \
>                  ivshmem-client-obj-y \
>                  ivshmem-server-obj-y \
>                  rdmacm-mux-obj-y \
> -                vhost-user-scsi-obj-y \
>                  vhost-user-input-obj-y \
>                  vhost-user-gpu-obj-y \
>                  qga-vss-dll-obj-y \
> @@ -629,8 +628,6 @@ ivshmem-client$(EXESUF): $(ivshmem-client-obj-y) $(COMMON_LDADDS)
>  ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS)
>  	$(call LINK, $^)
>  endif
> -vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) contrib/libvhost-user/libvhost-user.a
> -	$(call LINK, $^)
>  
>  rdmacm-mux$(EXESUF): LIBS += "-libumad"
>  rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
> diff --git a/Makefile.objs b/Makefile.objs
> index 644e2bd..adf2f92 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -116,9 +116,6 @@ qga-vss-dll-obj-y = qga/
>  elf2dmp-obj-y = contrib/elf2dmp/
>  ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
>  ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
> -vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS)
> -vhost-user-scsi.o-libs := $(LIBISCSI_LIBS)
> -vhost-user-scsi-obj-y = contrib/vhost-user-scsi/

The casual reader may wonder what these variables do.  The %.o: %.c rule
in rules.mak passes $($@-cflags) to $(CC).

With meson, ...

>  rdmacm-mux-obj-y = contrib/rdmacm-mux/
>  vhost-user-input-obj-y = contrib/vhost-user-input/
>  vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
> diff --git a/contrib/vhost-user-scsi/Makefile.objs b/contrib/vhost-user-scsi/Makefile.objs
> deleted file mode 100644
> index e83a38a..0000000
> --- a/contrib/vhost-user-scsi/Makefile.objs
> +++ /dev/null
> @@ -1 +0,0 @@
> -vhost-user-scsi-obj-y = vhost-user-scsi.o
> diff --git a/contrib/vhost-user-scsi/meson.build b/contrib/vhost-user-scsi/meson.build
> new file mode 100644
> index 0000000..b80398c
> --- /dev/null
> +++ b/contrib/vhost-user-scsi/meson.build
> @@ -0,0 +1,3 @@
> +executable('vhost-user-scsi', files('vhost-user-scsi.c'),
> +           link_with: libvhost_user,
> +           dependencies: [glib, libiscsi])

... we instead name dependencies, which ...

> diff --git a/meson.build b/meson.build
> index c4cad8f..bc7fbea 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -10,6 +10,9 @@ add_project_arguments(config_host['QEMU_INCLUDES'].split(),
>  
>  glib = declare_dependency(compile_args: config_host['GLIB_CFLAGS'].split(),
>                            link_args: config_host['GLIB_LIBS'].split())
> +libiscsi = declare_dependency(compile_args: config_host['LIBISCSI_CFLAGS'].split(),
> +                          link_args: config_host['LIBISCSI_LIBS'].split())

... we define here.  Less magical.

Note that there's also glib dependency, which works the same (see
previous patch).  With Make, we add GLib flags to QEMU_CFLAGS and LIBS,
so they get used for everything unless overridden.

>  
>  subdir('contrib/libvhost-user')
>  subdir('contrib/vhost-user-blk')
> +subdir('contrib/vhost-user-scsi')

Re: [Qemu-devel] [PATCH 7/7] rdmacm-mux: convert to Meson

[Markus Armbruster]

Paolo Bonzini <pbonzini@redhat.com> writes:

> We can use config-host.mak to decide whether the tool has to be built,
> apart from that the conversion is straightforward.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  .gitignore                       | 1 +
>  Makefile                         | 5 -----
>  Makefile.objs                    | 1 -
>  contrib/rdmacm-mux/Makefile.objs | 3 ---
>  contrib/rdmacm-mux/meson.build   | 6 ++++++
>  meson.build                      | 2 ++
>  6 files changed, 9 insertions(+), 9 deletions(-)
>  delete mode 100644 contrib/rdmacm-mux/Makefile.objs
>  create mode 100644 contrib/rdmacm-mux/meson.build
>
> diff --git a/.gitignore b/.gitignore
> index 3934eff..b8d38a8 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -63,6 +63,7 @@
>  /qemu-version.h.tmp
>  /module_block.h
>  /scsi/qemu-pr-helper
> +/contrib/rdmacm-mux/rdmacm-mux
>  /contrib/vhost-user-scsi/vhost-user-scsi
>  /contrib/vhost-user-blk/vhost-user-blk
>  /fsdev/virtfs-proxy-helper
> diff --git a/Makefile b/Makefile
> index bff097c..713f301 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -425,7 +425,6 @@ dummy := $(call unnest-vars,, \
>                  elf2dmp-obj-y \
>                  ivshmem-client-obj-y \
>                  ivshmem-server-obj-y \
> -                rdmacm-mux-obj-y \
>                  vhost-user-input-obj-y \
>                  vhost-user-gpu-obj-y \
>                  qga-vss-dll-obj-y \
> @@ -629,10 +628,6 @@ ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS)
>  	$(call LINK, $^)
>  endif
>  
> -rdmacm-mux$(EXESUF): LIBS += "-libumad"
> -rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
> -	$(call LINK, $^)
> -

Another way to pass extra flags to the compiler: target-specific
variables.  More direct than the funny variables we saw in the previous
patch.

The fine manual describes a way how to shoot yourself in the foot with
target-specific variabes.  Been there, done that, had "fun" debugging.

>  vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) contrib/libvhost-user/libvhost-user.a
>  	$(call LINK, $^)
>  
> diff --git a/Makefile.objs b/Makefile.objs
> index adf2f92..cf02f63 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -116,7 +116,6 @@ qga-vss-dll-obj-y = qga/
>  elf2dmp-obj-y = contrib/elf2dmp/
>  ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
>  ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
> -rdmacm-mux-obj-y = contrib/rdmacm-mux/
>  vhost-user-input-obj-y = contrib/vhost-user-input/
>  vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
>  
> diff --git a/contrib/rdmacm-mux/Makefile.objs b/contrib/rdmacm-mux/Makefile.objs
> deleted file mode 100644
> index 3df744a..0000000
> --- a/contrib/rdmacm-mux/Makefile.objs
> +++ /dev/null
> @@ -1,3 +0,0 @@
> -ifdef CONFIG_PVRDMA
> -rdmacm-mux-obj-y = main.o
> -endif
> diff --git a/contrib/rdmacm-mux/meson.build b/contrib/rdmacm-mux/meson.build
> new file mode 100644
> index 0000000..8451756
> --- /dev/null
> +++ b/contrib/rdmacm-mux/meson.build
> @@ -0,0 +1,6 @@
> +if config_host['CONFIG_PVRDMA'] == 'y'
> +  # if not found, CONFIG_PVRDMA should not be set

Our Makefiles use "either unset or 'y'" booleans pretty pervasively.  Is
it idiomatic meson?

> +  libumad = cc.find_library('ibumad', required: true)

With configure & make, we have configure check -libumad works, and make
use it.

Ignorant question: cc.find_library() looks like it checks.  Could this
replace checking in configure?

> +  executable('rdmacm-mux', files('main.c'),
> +             dependencies: [glib, libumad])
> +endif
> diff --git a/meson.build b/meson.build
> index bc7fbea..2fc6111 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1,4 +1,5 @@
>  project('qemu', 'c', meson_version: '>=0.50.999')
> +cc = meson.get_compiler('c')
>  
>  kconfig = import('unstable-kconfig')
>  config_host = kconfig.load(meson.current_build_dir() / 'config-host.mak')
> @@ -14,5 +15,6 @@ libiscsi = declare_dependency(compile_args: config_host['LIBISCSI_CFLAGS'].split
>                            link_args: config_host['LIBISCSI_LIBS'].split())
>  
>  subdir('contrib/libvhost-user')
> +subdir('contrib/rdmacm-mux')
>  subdir('contrib/vhost-user-blk')
>  subdir('contrib/vhost-user-scsi')

Re: [Qemu-devel] [PATCH 7/7] rdmacm-mux: convert to Meson

[Paolo Bonzini]

On 27/06/19 13:38, Markus Armbruster wrote:
>> +if config_host['CONFIG_PVRDMA'] == 'y'
>> +  # if not found, CONFIG_PVRDMA should not be set
> Our Makefiles use "either unset or 'y'" booleans pretty pervasively.  Is
> it idiomatic meson?

No, the idiomatic way would be to use the found() method.  Something like:

# in toplevel meson.build:
ibverbs = cc.find_library('ibverbs')
ibumad = cc.find_library('ibumad')
rdmacm = cc.find_library('rdmacm')
if ibverbs.found() and ibumad.found() and rdmacm.found()
  pvrdma = declare_dependency(dependencies: [ibverbs, ibumad, rdmacm]
else
  # magic "not found" dependency
  pvrdma = dependency('', required: false)
endif

# in contrib/rdmacm-mux/meson.build
if pvrdma.found() then
  executable('rdmacm-mux', files('main.c'),
             dependencies: [glib, ibumad])
endif

>> +  libumad = cc.find_library('ibumad', required: true)
> With configure & make, we have configure check -libumad works, and make
> use it.
> 
> Ignorant question: cc.find_library() looks like it checks.  Could this
> replace checking in configure?

Yes, see above.  However, you would have to remove all uses of
$(RDMA_LIBS) and in the Makefiles first.

Paolo

Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Markus Armbruster]

Paolo Bonzini <pbonzini@redhat.com> writes:

> On 27/06/19 11:03, Daniel P. Berrangé wrote:
>> On Mon, Jun 10, 2019 at 01:14:57PM +0200, Paolo Bonzini wrote:
>>> The Meson build system is integrated in the existing configure/make steps
>>> by invoking Meson from the configure script and converting Meson's build.ninja
>>> rules to an included Makefile.
>> 
>> Why did you take the route of converting ninja rules into makefile
>> rules, as opposed to just having some stub makefile rules which
>> directly invoke ninja where needed ?
>
> There are two parts of this.  One is practical and has to do with
> supporting a step-by-step transition.  Using ninja2make makes it trivial
> to have make build products that depend on meson build products, and
> this way bottom up is a natural direction to do the conversion, which is
> bottom up.  You'd start from libqemuutil.a and code generators (tracing
> + QAPI), then go to the tools and the emulators.

*If* the conversion is too big a task to permit doing it all at once,
then the step by step strategy you describe makes sense to me.

The trouble with step by step is running out of steam before the final
step.  That would leave us worse off.  Even an overly protracted
conversion would be bad.

Thus, my standard question on any proposed step-by-step conversion:
commitment to finishing it?  I'd be quite happy to take your word for
it.

> The second is a design decision that simplifying the Make/meson
> integration is *not* a goal.  Rather the goals are: 1) making the
> transition easier on developers; 2) avoiding magic in meson.build at all
> costs.  More specifically:
>
> - it should remain trivial to do things that used to be trivial, and
> most "make" invocations should be kept the same at least until
> everything is converted and we can perhaps declare a flag day.  People
> are used to "make check" or "make subdir-x86_64-softmmu", those should
> continue to work while the transition is in progress.
>
> - it should be possible to modify meson.build without knowing QEMU
> specific details, and that should be _already_ possible now at the
> beginning of the transition (to avoid creating technical debt).  This
> means keeping the magic confined in Makefile rules and external scripts,
> while having a pretty plain meson.build.

I wholeheartedly agree with (2).  Reducing magic is the whole point.

Your plan confines new magic to "Makefile rules and external scripts".
We'll get actual reduction only if we can retire or at least radically
simplify them at some point.

I'm more ambivalent on (1).  Yes, making the transition easier for
developers is worth hiding a certain amount of magic out of their way.
But not any amount.  We should try to preserve the targets printed by
"make help".

> I expect testing might also require some hand-holding, because "meson
> test" does not integrate with "make -j" and to keep supporting our "make
> check-*" targets.  However, until the make->ninja flag day we could
> generate tap-driver Makefile rules from "meson introspect --tests"
> output.  Basically I'm dropping Makefile magic in favor of build rule
> generators are written in high-level languages.

A PoC for selected tests would be nice.

>> Obviously this series is just some initial integration, but eventually
>> when we get a complete conversion, I think it will look pretty wierd
>> if we're still converting ninja to make.
>
> I agree; once all the build rules are converted the Makefile could be as
> simple as
>
> 	all:
> 	include config.mak
> 	include tests/docker/Makefile.include
> 	include tests/vm/Makefile.include
> 	.NOTPARALLEL:
> 	%:
> 		ninja $@
>
> 	subdir-%-softmmu:
> 		ninja qemu-system-$*
>
> 	subdir-%-linux-user:
> 		ninja qemu-$*
>
> 	check:
> 		$(MESON) test
>
> 	check-%:
> 		$(MESON) test --suite $*
>
> etc. (and likewise the configure script could just translate the command
> line options before invoking meson).

I wouldn't bother keeping around such an empty shell of a Makefile.

Keeping configure as a wrapper could be marginally more useful, mostly
for people trying to build QEMU for the first time.

>                                       But for now, since rules are
> written half in meson and half in make, ninja2make seems the most
> transparent way to integrate the two.
>
>> Part of the downside of our current build system is that although it
>> uses make, the usage is wierd and QEMU specific structure. It would
>> be a shame to pick meson and then again use it is a way that is wierd
>> and QEMU specific.
>
> I agree, this is why it's important to have at least a standard meson.build.
>
> Some knowledge of config-host.mak is needed, because meson.build uses
> declare_dependency() instead of dependency() to link with libraries that
> were already found by configure, but that's it.

Ignorant question: could the switch to Meson enable doing less in
configure?  It's big and sloooow.

Re: [Qemu-devel] [PATCH 6/7] vhost-user-scsi: convert to Meson

[Paolo Bonzini]

On 27/06/19 13:23, Markus Armbruster wrote:
> Note that there's also glib dependency, which works the same (see
> previous patch).  With Make, we add GLib flags to QEMU_CFLAGS and LIBS,
> so they get used for everything unless overridden.

Having to add glib everywhere is mildly annoying.  We can avoid that
once we convert libqemuutil.a to meson, via

libqemuutil = static_library('qemuutil', ...)
libqemuutil = declare_dependency(link_with: libqemuutil,
                                 dependencies: glib)

>> +libiscsi = declare_dependency(compile_args: config_host['LIBISCSI_CFLAGS'].split(),
>> +                          link_args: config_host['LIBISCSI_LIBS'].split())
> ... we define here.  Less magical.

A couple notes here.  First, later on this declare_dependency() will be
replaced by dependency(), so that meson itself will run pkg-config
rather than our configure script.

Second, some magic will reappear once we start using sourcesets[1],
which allow dependencies will be attached to sources rather than
executables.  However, there will still be no need to track cflags and
ldflags separately.  It will look like this:

# in block/meson.build
block.add(when: libiscsi, if_true: files('iscsi.c'))

# in toplevel meson.build
block_obj = block.apply(config_host)
executable('qemu-img',
           sources: ['qemu-img.c', block_obj.sources()],
           dependencies: [libqemutil, block_obj.dependencies()])

Paolo

[1] https://mesonbuild.com/SourceSet-module.html

Re: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration

[Markus Armbruster]

Paolo Bonzini <pbonzini@redhat.com> writes:

> Hi all,
>
> this is just a simple example of introducing Meson into the build system,
> converting a few contrib/ subprojects to use it instead of Makefile.objs.
> There are certainly many rough edges, but I wanted to throw this out to
> show what it could look like.
>
> I chose contrib/ because it's a simple example of how the current build
> system causes clutter in the toplevel Makefiles and how interrelated the
> Makefile, Makefile.objs and contrib/*/Makefile.objs file are even for
> such simple executables.

This left a pleasant first impression, apart from the rather large
ninjatools.py (which I didn't actually review).  I think I'm at the
point where I go "okay, now show me something complicated".  Something
going beyond "build this from that", where both "this" and "that" are
basically literals.  Show us how to *compute* in the brave new Meson
world.

To avoid misunderstandings: I think the less computing we need in the
build system, the better.  But I doubt we can banish it entirely there.

Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Daniel P. Berrangé]

On Thu, Jun 27, 2019 at 02:23:52PM +0200, Markus Armbruster wrote:
> Paolo Bonzini <pbonzini@redhat.com> writes:


> >                                       But for now, since rules are
> > written half in meson and half in make, ninja2make seems the most
> > transparent way to integrate the two.
> >
> >> Part of the downside of our current build system is that although it
> >> uses make, the usage is wierd and QEMU specific structure. It would
> >> be a shame to pick meson and then again use it is a way that is wierd
> >> and QEMU specific.
> >
> > I agree, this is why it's important to have at least a standard meson.build.
> >
> > Some knowledge of config-host.mak is needed, because meson.build uses
> > declare_dependency() instead of dependency() to link with libraries that
> > were already found by configure, but that's it.
> 
> Ignorant question: could the switch to Meson enable doing less in
> configure?  It's big and sloooow.

At a conceptual level meson is supposed to take over the role of both
configure and automake, so you have a single domain specific language
covering all aspects of your build system (though sometimes needs some
external python helper scripts to cover missing features in meson).

IOW, in a proper conversion, I'd expect configure to go away. I can
imagine that being one of the last steps in QEMU's conversion though

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Daniel P. Berrangé]

On Thu, Jun 27, 2019 at 12:16:05PM +0200, Paolo Bonzini wrote:
> On 27/06/19 11:03, Daniel P. Berrangé wrote:
> > On Mon, Jun 10, 2019 at 01:14:57PM +0200, Paolo Bonzini wrote:
> >> The Meson build system is integrated in the existing configure/make steps
> >> by invoking Meson from the configure script and converting Meson's build.ninja
> >> rules to an included Makefile.
> > 
> > Why did you take the route of converting ninja rules into makefile
> > rules, as opposed to just having some stub makefile rules which
> > directly invoke ninja where needed ?
> 
> There are two parts of this.  One is practical and has to do with
> supporting a step-by-step transition.  Using ninja2make makes it trivial
> to have make build products that depend on meson build products, and
> this way bottom up is a natural direction to do the conversion, which is
> bottom up.  You'd start from libqemuutil.a and code generators (tracing
> + QAPI), then go to the tools and the emulators.

Ok, I can understand that. I've been thinking about how we can switch
libvirt to use meson too, and trying to decide between meson being the
owner, calling out to make  vs keeping make as the owner and calling
out to meson. Ultimately to entirely banish make, autoconf, automake,
libtool, m4 & shell from our build system :-)

Despite thinking about an incremental conversion though, I was still
hoping libvirt would just have a single (largish) patch series to
do a complete conversion at a specific point in time.

> > Obviously this series is just some initial integration, but eventually
> > when we get a complete conversion, I think it will look pretty wierd
> > if we're still converting ninja to make.
> 
> I agree; once all the build rules are converted the Makefile could be as
> simple as
> 
> 	all:
> 	include config.mak
> 	include tests/docker/Makefile.include
> 	include tests/vm/Makefile.include
> 	.NOTPARALLEL:
> 	%:
> 		ninja $@
> 
> 	subdir-%-softmmu:
> 		ninja qemu-system-$*
> 
> 	subdir-%-linux-user:
> 		ninja qemu-$*
> 
> 	check:
> 		$(MESON) test
> 
> 	check-%:
> 		$(MESON) test --suite $*
> 
> etc. (and likewise the configure script could just translate the command
> line options before invoking meson).  But for now, since rules are
> written half in meson and half in make, ninja2make seems the most
> transparent way to integrate the two.

Ok, I think it understand the motiviation better now that its obviously
just a short term expedient step to convert ninja to make.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Paolo Bonzini]

On 27/06/19 14:23, Markus Armbruster wrote:
> Paolo Bonzini <pbonzini@redhat.com> writes:
>> On 27/06/19 11:03, Daniel P. Berrangé wrote:
>> There are two parts of this.  One is practical and has to do with
>> supporting a step-by-step transition.  Using ninja2make makes it trivial
>> to have make build products that depend on meson build products, and
>> this way bottom up is a natural direction to do the conversion, which is
>> bottom up.  You'd start from libqemuutil.a and code generators (tracing
>> + QAPI), then go to the tools and the emulators.
> 
> *If* the conversion is too big a task to permit doing it all at once,
> then the step by step strategy you describe makes sense to me.
> 
> The trouble with step by step is running out of steam before the final
> step.  That would leave us worse off.  Even an overly protracted
> conversion would be bad.

Agreed.  But hey, Makefile.objs is 2000 lines of lists of files, it is
boring but it cannot take that long to convert.  So it's not really
about being _possible_ to do it all at once, it's mostly about the
manageability of conflicts.

> Thus, my standard question on any proposed step-by-step conversion:
> commitment to finishing it?  I'd be quite happy to take your word for
> it.

I cannot really make such a commitment now, but perhaps I can (or I and
someone else can) once a little more of the conversion is ready.
Especially QAPI and trace generators, since those are where most of the
magic resides.  I can try that since your reaction wasn't of total
disgust. :)

>> The second is a design decision that simplifying the Make/meson
>> integration is *not* a goal.  Rather the goals are: 1) making the
>> transition easier on developers; 2) avoiding magic in meson.build at all
>> costs.  More specifically:
>>
> Your plan confines new magic to "Makefile rules and external scripts".
> We'll get actual reduction only if we can retire or at least radically
> simplify them at some point.

But even before that, we'll get *practical* reduction if it hacking the
Makefile rules and external scripts becomes rare enough.  See for
example kconfig, where # of people hacking minikconf.py is much less
than # of people hacking default-configs.

> I'm more ambivalent on (1).  Yes, making the transition easier for
> developers is worth hiding a certain amount of magic out of their way.

It's especially worth during the transition.  Once Makefile.objs is gone
I agree we can afford more radical changes, but let's cross that bridge
when we get there.

>> I expect testing might also require some hand-holding, because "meson
>> test" does not integrate with "make -j" and to keep supporting our "make
>> check-*" targets.  However, until the make->ninja flag day we could
>> generate tap-driver Makefile rules from "meson introspect --tests"
>> output.  Basically I'm dropping Makefile magic in favor of build rule
>> generators are written in high-level languages.
> 
> A PoC for selected tests would be nice.

Fair enough.

> Ignorant question: could the switch to Meson enable doing less in
> configure?  It's big and sloooow.

It would be smaller since the DSL is more compact than shell scripts,
probably not much faster as the same tasks still have to be done.  But
perhaps in the future Meson can parallelize them, and then we'd get that
for free.

Paolo

Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Gerd Hoffmann]

 Hi,

> Ok, I can understand that. I've been thinking about how we can switch
> libvirt to use meson too, and trying to decide between meson being the
> owner, calling out to make  vs keeping make as the owner and calling
> out to meson. Ultimately to entirely banish make, autoconf, automake,
> libtool, m4 & shell from our build system :-)
> 
> Despite thinking about an incremental conversion though, I was still
> hoping libvirt would just have a single (largish) patch series to
> do a complete conversion at a specific point in time.

Another possible approach would be to have two build systems.
The traditional configure & make and the new meson & ninja.

Advantage is we don't have to worry about the transition and mixing &
make + meson at all.

Disadvantage is the duplication.  That wouldn't be forever though.
I'd expect we'll have one or maybe two releases with both build systems,
then delete the make & configure.

cheers,
  Gerd

Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Paolo Bonzini]

On 27/06/19 15:16, Gerd Hoffmann wrote:
>  Hi,
> 
>> Ok, I can understand that. I've been thinking about how we can switch
>> libvirt to use meson too, and trying to decide between meson being the
>> owner, calling out to make  vs keeping make as the owner and calling
>> out to meson. Ultimately to entirely banish make, autoconf, automake,
>> libtool, m4 & shell from our build system :-)
>>
>> Despite thinking about an incremental conversion though, I was still
>> hoping libvirt would just have a single (largish) patch series to
>> do a complete conversion at a specific point in time.
> 
> Another possible approach would be to have two build systems.
> The traditional configure & make and the new meson & ninja.
> 
> Advantage is we don't have to worry about the transition and mixing &
> make + meson at all.
> 
> Disadvantage is the duplication.  That wouldn't be forever though.
> I'd expect we'll have one or maybe two releases with both build systems,
> then delete the make & configure.

Yeah, I have seen other projects do this.  On the other hand I'm not
sure how you avoid bitrot in one of the two, and having to convert the
huge configure script from the beginning seems like a daunting task.

Paolo

Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Daniel P. Berrangé]

On Thu, Jun 27, 2019 at 03:16:01PM +0200, Gerd Hoffmann wrote:
>  Hi,
> 
> > Ok, I can understand that. I've been thinking about how we can switch
> > libvirt to use meson too, and trying to decide between meson being the
> > owner, calling out to make  vs keeping make as the owner and calling
> > out to meson. Ultimately to entirely banish make, autoconf, automake,
> > libtool, m4 & shell from our build system :-)
> > 
> > Despite thinking about an incremental conversion though, I was still
> > hoping libvirt would just have a single (largish) patch series to
> > do a complete conversion at a specific point in time.
> 
> Another possible approach would be to have two build systems.
> The traditional configure & make and the new meson & ninja.
> 
> Advantage is we don't have to worry about the transition and mixing &
> make + meson at all.
> 
> Disadvantage is the duplication.  That wouldn't be forever though.
> I'd expect we'll have one or maybe two releases with both build systems,
> then delete the make & configure.

Personally I strongly wish to avoid having two parallel build systems.
In several projects I've encountered with that I've ended up hitting
bugs where each build system produced different binary results  due to
the developers not putting the same changes into both build systems.

Prefer to only merge a new build system once there is confidence that
it is working, and then immediately delete the old one so as to avoid
bitrot & concentrate any bug fixing on the modern stuff.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|