* [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration @ 2019-06-10 11:14 Paolo Bonzini 2019-06-10 11:14 ` [Qemu-devel] [PATCH 1/7] configure: do not include $(...) variables in config-host.mak Paolo Bonzini ` (10 more replies) 0 siblings, 11 replies; 29+ messages in thread From: Paolo Bonzini @ 2019-06-10 11:14 UTC (permalink / raw) To: qemu-devel Hi all, this is just a simple example of introducing Meson into the build system, converting a few contrib/ subprojects to use it instead of Makefile.objs. There are certainly many rough edges, but I wanted to throw this out to show what it could look like. I chose contrib/ because it's a simple example of how the current build system causes clutter in the toplevel Makefiles and how interrelated the Makefile, Makefile.objs and contrib/*/Makefile.objs file are even for such simple executables. Paolo Paolo Bonzini (7): configure: do not include $(...) variables in config-host.mak configure: set $PYTHON to a full path configure: integrate Meson in the build system libvhost-user: convert to Meson vhost-user-blk: convert to Meson vhost-user-scsi: convert to Meson rdmacm-mux: convert to Meson .gitignore | 5 +- Makefile | 26 +- Makefile.objs | 6 - configure | 66 ++- contrib/libvhost-user/Makefile.objs | 1 - contrib/libvhost-user/meson.build | 7 + contrib/rdmacm-mux/Makefile.objs | 3 - contrib/rdmacm-mux/meson.build | 6 + contrib/vhost-user-blk/Makefile.objs | 1 - contrib/vhost-user-blk/meson.build | 3 + contrib/vhost-user-scsi/Makefile.objs | 1 - contrib/vhost-user-scsi/meson.build | 3 + meson.build | 20 + scripts/ninjatool.py | 964 ++++++++++++++++++++++++++++++++++ 14 files changed, 1066 insertions(+), 46 deletions(-) delete mode 100644 contrib/libvhost-user/Makefile.objs create mode 100644 contrib/libvhost-user/meson.build delete mode 100644 contrib/rdmacm-mux/Makefile.objs create mode 100644 contrib/rdmacm-mux/meson.build delete mode 100644 contrib/vhost-user-blk/Makefile.objs create mode 100644 contrib/vhost-user-blk/meson.build delete mode 100644 contrib/vhost-user-scsi/Makefile.objs create mode 100644 contrib/vhost-user-scsi/meson.build create mode 100644 meson.build create mode 100644 scripts/ninjatool.py -- 1.8.3.1 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH 1/7] configure: do not include $(...) variables in config-host.mak 2019-06-10 11:14 [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration Paolo Bonzini @ 2019-06-10 11:14 ` Paolo Bonzini 2019-06-10 11:14 ` [Qemu-devel] [PATCH 2/7] configure: set $PYTHON to a full path Paolo Bonzini ` (9 subsequent siblings) 10 siblings, 0 replies; 29+ messages in thread From: Paolo Bonzini @ 2019-06-10 11:14 UTC (permalink / raw) To: qemu-devel This ensures that Meson will be able to reuse the results of the tests that are performed in the configure script. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- configure | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/configure b/configure index 6cdcfb2..203dece 100755 --- a/configure +++ b/configure @@ -597,7 +597,7 @@ QEMU_CFLAGS="-fno-strict-aliasing -fno-common -fwrapv -std=gnu99 $QEMU_CFLAGS" QEMU_CFLAGS="-Wall -Wundef -Wwrite-strings -Wmissing-prototypes $QEMU_CFLAGS" QEMU_CFLAGS="-Wstrict-prototypes -Wredundant-decls $QEMU_CFLAGS" QEMU_CFLAGS="-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE $QEMU_CFLAGS" -QEMU_INCLUDES="-iquote . -iquote \$(SRC_PATH) -iquote \$(SRC_PATH)/accel/tcg -iquote \$(SRC_PATH)/include" +QEMU_INCLUDES="-iquote . -iquote ${source_path} -iquote ${source_path}/accel/tcg -iquote ${source_path}/include" if test "$debug_info" = "yes"; then CFLAGS="-g $CFLAGS" LDFLAGS="-g $LDFLAGS" @@ -896,7 +896,7 @@ Linux) linux="yes" linux_user="yes" kvm="yes" - QEMU_INCLUDES="-I\$(SRC_PATH)/linux-headers -I$PWD/linux-headers $QEMU_INCLUDES" + QEMU_INCLUDES="-I${source_path}/linux-headers -I$PWD/linux-headers $QEMU_INCLUDES" supported_os="yes" libudev="yes" ;; @@ -4090,8 +4090,8 @@ EOF symlink "$source_path/dtc/Makefile" "dtc/Makefile" symlink "$source_path/dtc/scripts" "dtc/scripts" fi - fdt_cflags="-I\$(SRC_PATH)/dtc/libfdt" - fdt_ldflags="-L\$(BUILD_DIR)/dtc/libfdt" + fdt_cflags="-I${source_path}/dtc/libfdt" + fdt_ldflags="-L$PWD/dtc/libfdt" fdt_libs="$fdt_libs" elif test "$fdt" = "yes" ; then # Not a git build & no libfdt found, prompt for system install @@ -5040,13 +5040,13 @@ case "$capstone" in git_submodules="${git_submodules} capstone" fi mkdir -p capstone - QEMU_CFLAGS="$QEMU_CFLAGS -I\$(SRC_PATH)/capstone/include" + QEMU_CFLAGS="$QEMU_CFLAGS -I${source_path}/capstone/include" if test "$mingw32" = "yes"; then LIBCAPSTONE=capstone.lib else LIBCAPSTONE=libcapstone.a fi - LIBS="-L\$(BUILD_DIR)/capstone -lcapstone $LIBS" + LIBS="-L$PWD/capstone -lcapstone $LIBS" ;; system) @@ -5951,8 +5951,8 @@ case "$slirp" in git_submodules="${git_submodules} slirp" fi mkdir -p slirp - slirp_cflags="-I\$(SRC_PATH)/slirp/src -I\$(BUILD_DIR)/slirp/src" - slirp_libs="-L\$(BUILD_DIR)/slirp -lslirp" + slirp_cflags="-I${source_path}/slirp/src -I$PWD/slirp/src" + slirp_libs="-L$PWD/slirp -lslirp" ;; system) @@ -7307,21 +7307,21 @@ if test "$sheepdog" = "yes" ; then fi if test "$tcg_interpreter" = "yes"; then - QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/tci $QEMU_INCLUDES" + QEMU_INCLUDES="-iquote ${source_path}/tcg/tci $QEMU_INCLUDES" elif test "$ARCH" = "sparc64" ; then - QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/sparc $QEMU_INCLUDES" + QEMU_INCLUDES="-iquote ${source_path}/tcg/sparc $QEMU_INCLUDES" elif test "$ARCH" = "s390x" ; then - QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/s390 $QEMU_INCLUDES" + QEMU_INCLUDES="-iquote ${source_path}/tcg/s390 $QEMU_INCLUDES" elif test "$ARCH" = "x86_64" || test "$ARCH" = "x32" ; then - QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/i386 $QEMU_INCLUDES" + QEMU_INCLUDES="-iquote ${source_path}/tcg/i386 $QEMU_INCLUDES" elif test "$ARCH" = "ppc64" ; then - QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/ppc $QEMU_INCLUDES" + QEMU_INCLUDES="-iquote ${source_path}/tcg/ppc $QEMU_INCLUDES" elif test "$ARCH" = "riscv32" || test "$ARCH" = "riscv64" ; then - QEMU_INCLUDES="-I\$(SRC_PATH)/tcg/riscv $QEMU_INCLUDES" + QEMU_INCLUDES="-I${source_path}/tcg/riscv $QEMU_INCLUDES" else - QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/\$(ARCH) $QEMU_INCLUDES" + QEMU_INCLUDES="-iquote ${source_path}/tcg/\$(ARCH) $QEMU_INCLUDES" fi -QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg $QEMU_INCLUDES" +QEMU_INCLUDES="-iquote ${source_path}/tcg $QEMU_INCLUDES" echo "TOOLS=$tools" >> $config_host_mak echo "ROMS=$roms" >> $config_host_mak -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH 2/7] configure: set $PYTHON to a full path 2019-06-10 11:14 [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration Paolo Bonzini 2019-06-10 11:14 ` [Qemu-devel] [PATCH 1/7] configure: do not include $(...) variables in config-host.mak Paolo Bonzini @ 2019-06-10 11:14 ` Paolo Bonzini 2019-06-10 11:14 ` [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system Paolo Bonzini ` (8 subsequent siblings) 10 siblings, 0 replies; 29+ messages in thread From: Paolo Bonzini @ 2019-06-10 11:14 UTC (permalink / raw) To: qemu-devel This will make it possible to replace it in a shebang line. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- configure | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure b/configure index 203dece..0814a5f 100755 --- a/configure +++ b/configure @@ -918,7 +918,7 @@ for binary in "${PYTHON-python3}" python python2 do if has "$binary" then - python="$binary" + python=$(type -p "$binary") break fi done -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system 2019-06-10 11:14 [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration Paolo Bonzini 2019-06-10 11:14 ` [Qemu-devel] [PATCH 1/7] configure: do not include $(...) variables in config-host.mak Paolo Bonzini 2019-06-10 11:14 ` [Qemu-devel] [PATCH 2/7] configure: set $PYTHON to a full path Paolo Bonzini @ 2019-06-10 11:14 ` Paolo Bonzini 2019-06-26 17:34 ` Markus Armbruster 2019-06-27 9:03 ` Daniel P. Berrangé 2019-06-10 11:14 ` [Qemu-devel] [PATCH 4/7] libvhost-user: convert to Meson Paolo Bonzini ` (7 subsequent siblings) 10 siblings, 2 replies; 29+ messages in thread From: Paolo Bonzini @ 2019-06-10 11:14 UTC (permalink / raw) To: qemu-devel The Meson build system is integrated in the existing configure/make steps by invoking Meson from the configure script and converting Meson's build.ninja rules to an included Makefile. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- Makefile | 9 + configure | 30 ++ meson.build | 9 + scripts/ninjatool.py | 964 +++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 1012 insertions(+) create mode 100644 meson.build create mode 100644 scripts/ninjatool.py diff --git a/Makefile b/Makefile index 8e2fc66..b8f802c 100644 --- a/Makefile +++ b/Makefile @@ -48,6 +48,15 @@ git-submodule-update: endif endif +export NINJA=./ninjatool +Makefile.ninja: build.ninja ninjatool + ./ninjatool -t ninja2make --omit dist uninstall < $< > $@ +-include Makefile.ninja + +ninjatool: $(SRC_PATH)/scripts/ninjatool.py + sed -e '1c\' -e '#! $(PYTHON)' $< > $@ + chmod +x $@ + .git-submodule-status: git-submodule-update config-host.mak # Check that we're not trying to do an out-of-tree build from diff --git a/configure b/configure index 0814a5f..b8c3c58 100755 --- a/configure +++ b/configure @@ -493,6 +493,7 @@ docker="no" debug_mutex="no" libpmem="" default_devices="yes" +meson=meson # cross compilers defaults, can be overridden with --cross-cc-ARCH cross_cc_aarch64="aarch64-linux-gnu-gcc" @@ -983,6 +984,8 @@ for opt do ;; --python=*) python="$optarg" ;; + --meson=*) meson="$optarg" + ;; --gcov=*) gcov_tool="$optarg" ;; --smbd=*) smbd="$optarg" @@ -1685,6 +1688,7 @@ Advanced options (experts only): --make=MAKE use specified make [$make] --install=INSTALL use specified install [$install] --python=PYTHON use specified python [$python] + --meson=PYTHON use specified meson [$meson] --smbd=SMBD use specified smbd [$smbd] --with-git=GIT use specified git [$git] --static enable static build [$static] @@ -1850,6 +1854,11 @@ then error_exit "Python not found. Use --python=/path/to/python" fi +if ! has "$meson" +then + error_exit "Meson not found. Use --meson=/path/to/meson" +fi + # Note that if the Python conditional here evaluates True we will exit # with status 1 which is a shell 'false' value. if ! $python -c 'import sys; sys.exit(sys.version_info < (2,7))'; then @@ -7983,6 +7992,27 @@ echo "# Automatically generated by configure - do not modify" > "$iotests_common echo >> "$iotests_common_env" echo "export PYTHON='$python'" >> "$iotests_common_env" +# bootstrap ninjatool, we need it before Make runs +if ! test -x ninjatool; then + sed -e '1c\' -e "#! $python" ${source_path}/scripts/ninjatool.py > ninjatool + chmod +x ninjatool +fi +rm -rf meson-private meson-info meson-logs +NINJA=$PWD/ninjatool $python $meson setup \ + --prefix "$prefix" \ + --libdir "$libdir" \ + --libexecdir "$libexecdir" \ + --bindir "$bindir" \ + --includedir "$includedir" \ + --datadir "$datadir" \ + --mandir "$mandir" \ + --sysconfdir "$sysconfdir" \ + --localstatedir "$local_statedir" \ + $(test "$strip_opt" = yes && echo --strip) \ + --buildtype $(if test "$debug" = yes; then echo debug; else echo release; fi) \ + "$PWD" "$source_path" + + # Save the configure command line for later reuse. cat <<EOD >config.status #!/bin/sh diff --git a/meson.build b/meson.build new file mode 100644 index 0000000..b683d70 --- /dev/null +++ b/meson.build @@ -0,0 +1,9 @@ +project('qemu', 'c', meson_version: '>=0.50.999') + +kconfig = import('unstable-kconfig') +config_host = kconfig.load(meson.current_build_dir() / 'config-host.mak') + +add_project_arguments(config_host['QEMU_CFLAGS'].split(), + language: 'c') +add_project_arguments(config_host['QEMU_INCLUDES'].split(), + language: 'c') diff --git a/scripts/ninjatool.py b/scripts/ninjatool.py new file mode 100644 index 0000000..6d90919 --- /dev/null +++ b/scripts/ninjatool.py @@ -0,0 +1,964 @@ +#! /usr/bin/env python3 + +"""Python module for parsing and processing .ninja files. + + Author: Paolo Bonzini + + Copyright (C) 2019 Red Hat, Inc.""" + +from collections import namedtuple, defaultdict +import sys +import os +import re +import json +import argparse + + +class InvalidArgumentError(Exception): + pass + + +# ---- lexer and parser ---- + +BUILD = 1 +POOL = 2 +RULE = 3 +DEFAULT = 4 +EQUALS = 5 +COLON = 6 +PIPE = 7 +PIPE2 = 8 +IDENT = 9 +INCLUDE = 10 +INDENT = 11 +EOL = 12 + + +class LexerError(Exception): + pass + + +class ParseError(Exception): + pass + + +class NinjaParserEvents(object): + def __init__(self, parser): + self.parser = parser + + def literal_token(self, word, in_path=False): + word = word.replace('$', '$$') + if in_path: + word = word.replace(' ', '$ ').replace(':', '$:') + return word + + def variable_expansion_token(self, varname): + return '${%s}' % varname + + def tokens_to_arg(self, tokens): + return ''.join(tokens or []) + + def variable(self, name, arg): + pass + + def begin_file(self): + pass + + def end_file(self): + pass + + def end_scope(self): + pass + + def begin_pool(self, name): + pass + + def begin_rule(self, name): + pass + + def begin_build(self, out, iout, rule, in_, iin, orderdep): + pass + + def default(self, targets): + pass + + +class NinjaParser(object): + PATH_RE = r"[^$\s:|]+|\$[$ :]|\$[a-zA-Z0-9_-]+|\$\{[a-zA-Z0-9_.-]+\}" + STRING_RE = r"(" + PATH_RE + r"|[\s:|])(?:\r?\n)?" + TOPLEVEL_RE = r"([=:#]|\|\|?|^ +|(?:" + PATH_RE + r")+)\s*" + IDENT_RE = r"[a-zA-Z0-9_.-]+$" + + InputFile = namedtuple('InputFile', 'filename iter lineno') + + def __init__(self, filename, input): + self.stack = [] + self.lineno = None + self.match_keyword = True + self.push(filename, input) + + def __iter__(self): + return self + + def __next__(self): + while len(self.stack): + try: + return next(self.stack[-1].iter) + except StopIteration: + self.pop() + raise StopIteration + + def push(self, filename, input): + entered = self.InputFile(filename=filename or 'stdin', + iter=self._tokens(input), + lineno=self.lineno) + self.stack.append(entered) + self.lineno = 0 + if filename is not None: + os.chdir(os.path.dirname(filename) or '.') + + def pop(self): + left = self.stack.pop() + self.lineno = left.lineno + if len(self.stack): + os.chdir(os.path.dirname(self.stack[-1].filename) or '.') + + def next_line(self, input): + line = next(input).rstrip() + self.lineno += 1 + while len(line) and line[-1] == '$': + line = line[0:-1] + next(input).strip() + self.lineno += 1 + return line + + def print_token(self, tok): + if tok == EOL: + return "end of line" + if tok == BUILD: + return '"build"' + if tok == POOL: + return '"pool"' + if tok == RULE: + return '"rule"' + if tok == DEFAULT: + return '"default"' + if tok == EQUALS: + return '"="' + if tok == COLON: + return '":"' + if tok == PIPE: + return '"|"' + if tok == PIPE2: + return '"||"' + if tok == INCLUDE: + return '"include"' + if tok == IDENT: + return 'identifier' + return '"%s"' % tok + + def error(self, msg): + raise LexerError("%s:%d: %s" % (self.stack[-1].filename, self.lineno, msg)) + + def parse_error(self, msg): + raise ParseError("%s:%d: %s" % (self.stack[-1].filename, self.lineno, msg)) + + def expected(self, expected, tok): + msg = "found %s, expected " % (self.print_token(tok), ) + for i, exp_tok in enumerate(expected): + if i > 0: + msg = msg + (' or ' if i == len(expected) - 1 else ', ') + msg = msg + self.print_token(exp_tok) + self.parse_error(msg) + + def find_tokens(self, regex, line): + # group 1 is used to distinguish expected input from bad characters + # group 2, which is really group 1 in regex, is returned + for m in re.finditer("(" + regex + ")|.", line): + if not m.group(1): + self.error("unexpected '%s'" % (m.group(0), )) + if m.group(2): + yield m.span(), m.group(2) + + def _variable_tokens(self, value): + for span, match in self.find_tokens(self.STRING_RE, value): + yield match + + def _tokens(self, input): + while True: + try: + line = self.next_line(input) + except StopIteration: + return + for span, match in self.find_tokens(self.TOPLEVEL_RE, line): + if match == ':': + yield COLON + continue + if match == '|': + yield PIPE + continue + if match == '||': + yield PIPE2 + continue + if match[0] == ' ': + yield INDENT + continue + if match[0] == '=': + value = line[span[0] + 1:].lstrip() + yield EQUALS + yield from self._variable_tokens(value) + break + if match[0] == '#': + break + + # identifier + if self.match_keyword: + if match == 'build': + yield BUILD + continue + if match == 'pool': + yield POOL + continue + if match == 'rule': + yield RULE + continue + if match == 'default': + yield DEFAULT + continue + if match == 'include': + filename = line[span[0] + 8:].strip() + self.push(filename, open(filename, 'r')) + break + if match == 'subninja': + self.error('subninja is not supported') + yield match + yield EOL + + def read_ident(self): + """Read an identifier, where the only allowed characters are + letter, numbers, dash, period and underscore.""" + self.match_keyword = False + token = self.__next__() + self.match_keyword = True + if isinstance(token, str) and not re.match(self.IDENT_RE, token): + self.parse_error('variable expansion not allowed') + return token + + def read_path(self): + """Read a space-delimited path or a non-alphanumeric token, and + return a generator that produces either the token alone, or + the components of the path (identifiers and variable expansions).""" + self.match_keyword = False + token = self.__next__() + self.match_keyword = True + if isinstance(token, str): + yield from self._variable_tokens(token) + else: + yield token + + def parse(self, events): + global_var = True + + def advance(): + try: + return next(self) + except StopIteration: + return None + + def look_for(*expected): + tok = advance() + if tok in expected: + return tok + self.expected(expected, tok) + + def look_for_ident(*expected): + try: + tok = self.read_ident() + except StopIteration: + tok = None + if isinstance(tok, str) or tok in expected: + return tok + self.expected(expected + (IDENT,), tok) + + def parse_assignment_rhs(gen, expected, in_path): + value = [] + for tok in gen: + if not isinstance(tok, str): + if tok in expected: + break + self.expected(expected + (IDENT,), tok) + if tok[0] != '$': + value.append(events.literal_token(tok, in_path)) + elif tok == '$ ' or tok == '$$' or tok == '$:': + value.append(events.literal_token(tok[1], in_path)) + else: + var = tok[2:-1] if tok[1] == '{' else tok[1:] + value.append(events.variable_expansion_token(var)) + else: + # gen must have raised StopIteration, return None + # for consistency with advance() + tok = None + value = value if len(value) else None + return value, tok + + def look_for_path(*expected): + # paths in build rules are parsed one space-separated token + # at a time and expanded + gen = self.read_path() + value, tok = parse_assignment_rhs(gen, expected, True) + return value, tok + + def parse_assignment(tok): + assert isinstance(tok, str) + look_for(EQUALS) + name = tok + value, tok = parse_assignment_rhs(self, (EOL,), False) + assert tok == EOL + events.variable(name, events.tokens_to_arg(value)) + + def parse_build(): + # parse outputs + out = [] + iout = [] + while True: + value, tok = look_for_path(COLON, PIPE) + if value is None: + break + out.append(events.tokens_to_arg(value)) + if tok == PIPE: + while True: + value, tok = look_for_path(COLON) + if value is None: + break + iout.append(events.tokens_to_arg(value)) + + # parse rule + assert tok == COLON + rule = look_for_ident() + + # parse inputs and dependencies + in_ = [] + iin = [] + orderdep = [] + while True: + value, tok = look_for_path(PIPE, PIPE2, EOL) + if value is None: + break + in_.append(events.tokens_to_arg(value)) + if tok == PIPE: + while True: + value, tok = look_for_path(PIPE2, EOL) + if value is None: + break + iin.append(events.tokens_to_arg(value)) + if tok == PIPE2: + while True: + value, tok = look_for_path(EOL) + if value is None: + break + orderdep.append(events.tokens_to_arg(value)) + assert tok == EOL + events.begin_build(out, iout, rule, in_, iin, orderdep) + nonlocal global_var + global_var = False + + def parse_pool(): + # pool declarations are ignored. Just gobble all the variables + ident = look_for_ident() + look_for(EOL) + events.begin_pool(ident) + nonlocal global_var + global_var = False + + def parse_rule(): + ident = look_for_ident() + look_for(EOL) + events.begin_rule(ident) + nonlocal global_var + global_var = False + + def parse_default(): + idents = [] + while True: + ident = look_for_ident(EOL) + if ident == EOL: + break + idents.append(ident) + events.default(idents) + + def parse_declaration(tok): + if tok == EOL: + return + + nonlocal global_var + if tok == INDENT: + if global_var: + self.parse_error('indented line outside rule or edge') + tok = look_for_ident(EOL) + if tok == EOL: + return + parse_assignment(tok) + return + + if not global_var: + events.end_scope() + global_var = True + if not tok: + return + if tok == POOL: + parse_pool() + elif tok == BUILD: + parse_build() + elif tok == RULE: + parse_rule() + elif tok == DEFAULT: + parse_default() + elif isinstance(tok, str): + parse_assignment(tok) + else: + self.expected((POOL, BUILD, RULE, INCLUDE, DEFAULT, IDENT), tok) + + events.begin_file() + while True: + tok = advance() + parse_declaration(tok) + if not tok: + break + events.end_file() + + +# ---- variable handling ---- + +def expand(x, rule_vars=None, build_vars=None, global_vars=None): + if x is None: + return None + changed = True + while changed: + changed = False + matches = list(re.finditer(r'\$\$|\$\{([^}]*)\}', x)) + if not matches: + break + + # Reverse the match so that expanding later matches does not + # invalidate m.span() for earlier ones. Do not reduce $$ to $ + # until all variables are dealt with. + matches.reverse() + for m in matches: + if not m.group(1): + continue + changed = True + name = m.group(1) + if build_vars and name in build_vars: + value = build_vars[name] + elif rule_vars and name in rule_vars: + value = rule_vars[name] + elif name in global_vars: + value = global_vars[name] + else: + value = '' + start, end = m.span() + x = x[:start] + value + x[end:] + return x.replace('$$', '$') + + +class Scope(object): + def __init__(self, events): + self.events = events + + def on_left_scope(self): + pass + + def on_variable(self, key, value): + pass + + +class BuildScope(Scope): + def __init__(self, events, out, iout, rule, in_, iin, orderdep, rule_vars): + super().__init__(events) + self.rule = rule + self.out = [events.expand_and_normalize(x) for x in out] + self.in_ = [events.expand_and_normalize(x) for x in in_] + self.iin = [events.expand_and_normalize(x) for x in iin] + self.orderdep = [events.expand_and_normalize(x) for x in orderdep] + self.iout = [events.expand_and_normalize(x) for x in iout] + self.rule_vars = rule_vars + self.build_vars = dict() + self._define_variable('out', ' '.join(self.out)) + self._define_variable('in', ' '.join(self.in_)) + + def expand(self, x): + return self.events.expand(x, self.rule_vars, self.build_vars) + + def on_left_scope(self): + self.events.variable('out', self.build_vars['out']) + self.events.variable('in', self.build_vars['in']) + self.events.end_build(self, self.out, self.iout, self.rule, self.in_, + self.iin, self.orderdep) + + def _define_variable(self, key, value): + # The value has been expanded already, quote it for further + # expansion from rule variables + value = value.replace('$', '$$') + self.build_vars[key] = value + + def on_variable(self, key, value): + # in and out are at the top of the lookup order and cannot + # be overridden. Also, unlike what the manual says, build + # variables only lookup global variables. They never lookup + # rule variables, earlier build variables, or in/out. + if key not in ('in', 'out'): + self._define_variable(key, self.events.expand(value)) + + +class RuleScope(Scope): + def __init__(self, events, name, vars_dict): + super().__init__(events) + self.name = name + self.vars_dict = vars_dict + self.generator = False + + def on_left_scope(self): + self.events.end_rule(self, self.name) + + def on_variable(self, key, value): + self.vars_dict[key] = value + if key == 'generator': + self.generator = True + + +class NinjaParserEventsWithVars(NinjaParserEvents): + def __init__(self, parser): + super().__init__(parser) + self.rule_vars = defaultdict(lambda: dict()) + self.global_vars = dict() + self.scope = None + + def variable(self, name, value): + if self.scope: + self.scope.on_variable(name, value) + else: + self.global_vars[name] = self.expand(value) + + def begin_build(self, out, iout, rule, in_, iin, orderdep): + if rule != 'phony' and rule not in self.rule_vars: + self.parser.parse_error("undefined rule '%s'" % rule) + + self.scope = BuildScope(self, out, iout, rule, in_, iin, orderdep, self.rule_vars[rule]) + + def begin_pool(self, name): + # pool declarations are ignored. Just gobble all the variables + self.scope = Scope(self) + + def begin_rule(self, name): + if name in self.rule_vars: + self.parser.parse_error("duplicate rule '%s'" % name) + self.scope = RuleScope(self, name, self.rule_vars[name]) + + def end_scope(self): + self.scope.on_left_scope() + self.scope = None + + # utility functions: + + def expand(self, x, rule_vars=None, build_vars=None): + return expand(x, rule_vars, build_vars, self.global_vars) + + def expand_and_normalize(self, x): + return os.path.normpath(self.expand(x)) + + # extra events not present in the superclass: + + def end_build(self, scope, out, iout, rule, in_, iin, orderdep): + pass + + def end_rule(self, scope, name): + pass + + +# ---- test client that just prints back whatever it parsed ---- + +class Writer(NinjaParserEvents): + ARGS = argparse.ArgumentParser(description='Rewrite input build.ninja to stdout.') + + def __init__(self, output, parser, args): + super().__init__(parser) + self.output = output + self.indent = '' + self.had_vars = False + + def print(self, *args, **kwargs): + if len(args): + self.output.write(self.indent) + print(*args, **kwargs, file=self.output) + + def variable(self, name, value): + self.print('%s = %s' % (name, value)) + self.had_vars = True + + def begin_scope(self): + self.indent = ' ' + self.had_vars = False + + def end_scope(self): + if self.had_vars: + self.print() + self.indent = '' + self.had_vars = False + + def begin_pool(self, name): + self.print('pool %s' % name) + self.begin_scope() + + def begin_rule(self, name): + self.print('rule %s' % name) + self.begin_scope() + + def begin_build(self, outputs, implicit_outputs, rule, inputs, implicit, order_only): + all_outputs = list(outputs) + all_inputs = list(inputs) + + if implicit: + all_inputs.append('|') + all_inputs.extend(implicit) + if order_only: + all_inputs.append('||') + all_inputs.extend(order_only) + if implicit_outputs: + all_outputs.append('|') + all_outputs.extend(implicit_outputs) + + self.print('build %s: %s' % (' '.join(all_outputs), + ' '.join([rule] + all_inputs))) + self.begin_scope() + + def default(self, targets): + self.print('default %s' % ' '.join(targets)) + + +# ---- emit compile_commands.json ---- + +class Compdb(NinjaParserEventsWithVars): + ARGS = argparse.ArgumentParser(description='Emit compile_commands.json.') + ARGS.add_argument('rules', nargs='*', + help='The ninja rules to emit compilation commands for.') + + def __init__(self, output, parser, args): + super().__init__(parser) + self.output = output + self.rules = args.rules + self.sep = '' + + def begin_file(self): + self.output.write('[') + self.directory = os.getcwd() + + def print_entry(self, **entry): + entry['directory'] = self.directory + self.output.write(self.sep + json.dumps(entry)) + self.sep = ',\n' + + def begin_build(self, out, iout, rule, in_, iin, orderdep): + if in_ and rule in self.rules: + super().begin_build(out, iout, rule, in_, iin, orderdep) + else: + self.scope = Scope(self) + + def end_build(self, scope, out, iout, rule, in_, iin, orderdep): + self.print_entry(command=scope.expand('${command}'), file=in_[0]) + + def end_file(self): + self.output.write(']\n') + + +# ---- clean output files ---- + +class Clean(NinjaParserEventsWithVars): + ARGS = argparse.ArgumentParser(description='Remove output build files.') + ARGS.add_argument('-g', dest='generator', action='store_true', + help='clean generated files too') + + def __init__(self, output, parser, args): + super().__init__(parser) + self.dry_run = args.dry_run + self.verbose = args.verbose or args.dry_run + self.generator = args.generator + + def begin_file(self): + print('Cleaning... ', end=(None if self.verbose else ''), flush=True) + self.cnt = 0 + + def end_file(self): + print('%d files' % self.cnt) + + def do_clean(self, *files): + for f in files: + if self.dry_run: + if os.path.exists(f): + self.cnt += 1 + print('Would remove ' + f) + continue + else: + try: + os.unlink(f) + self.cnt += 1 + if self.verbose: + print('Removed ' + f) + except FileNotFoundError: + pass + + def end_build(self, scope, out, iout, rule, in_, iin, orderdep): + if not self.generator and scope.expand('${generator}'): + return + self.do_clean(*out, *iout) + depfile = scope.expand('${depfile}') + if depfile: + self.do_clean(depfile) + + +# ---- convert build.ninja to makefile ---- + +class Ninja2Make(NinjaParserEventsWithVars): + ARGS = argparse.ArgumentParser(description='Convert build.ninja to a Makefile.') + ARGS.add_argument('--clean', dest='emit_clean', action='store_true', + help='Emit clean/distclean rules.') + ARGS.add_argument('--doublecolon', action='store_true', + help='Emit double-colon rules for phony targets.') + ARGS.add_argument('--omit', metavar='TARGET', nargs='+', + help='Targets to omit.') + + def __init__(self, output, parser, args): + super().__init__(parser) + self.output = output + + self.emit_clean = args.emit_clean + self.doublecolon = args.doublecolon + self.omit = set(args.omit) + + if self.emit_clean: + self.omit.update(['clean', 'distclean']) + + # Lists of targets are kept in memory and emitted only at the + # end because appending is really inefficient in GNU make. + # We only do it when it's O(#rules) or O(#variables), but + # never when it could be O(#targets). + self.depfiles = list() + self.build_vars = defaultdict(lambda: dict()) + self.rule_targets = defaultdict(lambda: list()) + self.all_outs = set() + self.all_ins = set() + self.all_phony = set() + self.seen_default = False + + def print(self, *args, **kwargs): + print(*args, **kwargs, file=self.output) + + def literal_token(self, word, in_path=False): + if in_path and ' ' in word: + self.parser.parse_error('Make does not support spaces in filenames') + return word.replace('$', '$$') + + def print_phony(self, outs, ins): + targets = ' '.join(outs).replace('$', '$$') + deps = ' '.join(ins).replace('$', '$$') + deps = deps.strip() + if self.doublecolon: + self.print(targets + '::' + (' ' if deps else '') + deps + ';@:') + else: + self.print(targets + ':' + (' ' if deps else '') + deps) + self.all_phony.update(outs) + + def begin_file(self): + self.print(r'# This is an automatically generated file, and it shows.') + self.print(r'ninja-default:') + self.print(r'.PHONY: ninja-default') + if self.emit_clean: + self.print(r'ninja-clean:: ninja-clean-start; $(if $V,,@)rm -f ${ninja-depfiles}') + self.print(r'ninja-clean-start:; $(if $V,,@echo Cleaning...)') + self.print(r'ninja-distclean:: clean') + self.print(r'.PHONY: ninja-clean ninja-clean-start ninja-distclean') + self.print_phony(['clean'], ['ninja-clean']) + self.print_phony(['distclean'], ['ninja-distclean']) + self.print(r'vpath') + self.print(r'NULL :=') + self.print(r'SPACE := ${NULL} #') + self.print(r'MAKEFLAGS += -rR') + self.print(r'define NEWLINE') + self.print(r'') + self.print(r'endef') + self.print(r'.var.in_newline = $(subst $(SPACE),$(NEWLINE),${.var.in})') + self.print(r"ninja-command = $(if $V,,@printf '%s\n' '$(subst ','\'',${.var.description})' && )${.var.command}") + + def mangle_vars_for_make(self, x): + matches = list(re.finditer(r'\$\$|\$\{([^}]*)\}', x)) + + # Reverse the match so that expanding later matches does not + # invalidate m.span() for earlier ones. + matches.reverse() + for m in matches: + if not m.group(1): + continue + value = '${.var.' + m.group(1) + '}' + start, end = m.span() + x = x[:start] + value + x[end:] + return x + + def end_file(self): + def natural_sort_key(s, _nsre=re.compile('([0-9]+)')): + return [int(text) if text.isdigit() else text.lower() + for text in _nsre.split(s)] + + self.print() + self.print('ninja-outputdirs :=') + for rule in self.rule_vars: + if rule == 'phony': + continue + self.print('ninja-targets-%s := %s' % (rule, ' '.join(self.rule_targets[rule]))) + for name, value in self.rule_vars[rule].items(): + self.print('$(ninja-targets-%s): .var.%s = %s' % (rule, name, self.mangle_vars_for_make(value))) + self.print('ninja-outputdirs += $(sort $(dir ${ninja-targets-%s}))' % rule) + self.print() + self.print('dummy := $(shell mkdir -p . $(sort $(ninja-outputdirs)))') + self.print('ninja-depfiles :=' + ' '.join(self.depfiles)) + self.print('-include ${ninja-depfiles}') + self.print() + for targets in self.build_vars: + for name, value in self.build_vars[targets].items(): + self.print('%s: .var.%s := %s' % (targets, name, value)) + self.print() + if not self.seen_default: + default_targets = sorted(self.all_outs - self.all_ins, key=natural_sort_key) + self.print('ninja-default: ' + ' '.join(default_targets)) + + # This is a hack... Meson declares input meson.build files as + # phony, because Ninja does not have an equivalent of Make's + # "path/to/file:" declaration that ignores "path/to/file" even + # if it is absent. However, Makefile.ninja wants to depend on + # build.ninja, which in turn depends on these phony targets which + # would cause Makefile.ninja to be rebuilt in a loop. + phony_targets = sorted(self.all_phony - self.all_ins, key=natural_sort_key) + self.print('.PHONY: ' + ' '.join(phony_targets)) + + def variable(self, name, value): + super().variable(name, value) + if self.scope is None: + self.global_vars[name] = self.expand(value) + self.print('.var.%s := %s' % (name, self.global_vars[name])) + elif isinstance(self.scope, BuildScope): + # These have to be printed last, because they override rule variables + # but we cannot print rule variables until we know the list of targets + # for each rule. + self.build_vars[self.current_targets][name] = self.scope.build_vars[name] + + def begin_build(self, out, iout, rule, in_, iin, orderdep): + if any(x in self.omit for x in out): + self.scope = Scope(self) + return + + super().begin_build(out, iout, rule, in_, iin, orderdep) + self.rule_targets[rule] += self.scope.out + self.rule_targets[rule] += self.scope.iout + + self.all_outs.update(self.scope.iout) + self.all_outs.update(self.scope.out) + self.all_ins.update(self.scope.in_) + self.all_ins.update(self.scope.iin) + + if rule == 'phony': + # Phony rules treat order-only dependencies as normal deps + self.print_phony(self.scope.out + self.scope.iout, + self.scope.in_ + self.scope.iin + self.scope.orderdep) + self.scope = Scope(self) + return + + targets = ' '.join(self.scope.out + self.scope.iout).replace('$', '$$') + inputs = ' '.join(self.scope.in_ + self.scope.iin).replace('$', '$$') + orderonly = ' '.join(self.scope.orderdep).replace('$', '$$') + self.print('%s: %s | %s; ${ninja-command}' % (targets, inputs, orderonly)) + self.current_targets = targets + + def end_build(self, scope, out, iout, rule, in_, iin, orderdep): + depfile = scope.expand('${depfile}') + if depfile: + self.depfiles.append(depfile) + self.current_targets = None + + def end_rule(self, scope, name): + if self.emit_clean: + self.print('ninja-%s:: ; $(if $V,,@)rm -f ${ninja-targets-%s}' % ( + 'distclean' if scope.generator else 'clean', + name)) + + def default(self, targets): + self.print("ninja-default: " + ' '.join(targets)) + self.seen_default = True + + +# ---- command line parsing ---- + +# we cannot use subparsers because tools are chosen through the "-t" +# option. + +class ToolAction(argparse.Action): + def __init__(self, option_strings, dest, choices, metavar='TOOL', nargs=None, **kwargs): + if nargs is not None: + raise ValueError("nargs not allowed") + super().__init__(option_strings, dest, required=True, choices=choices, + metavar=metavar, **kwargs) + + def __call__(self, parser, namespace, value, option_string): + tool = self.choices[value] + setattr(namespace, self.dest, tool) + tool.ARGS.prog = '%s %s %s' % (parser.prog, option_string, value) + + +class ToolHelpAction(argparse.Action): + def __init__(self, option_strings, dest, nargs=None, **kwargs): + if nargs is not None: + raise ValueError("nargs not allowed") + super().__init__(option_strings, dest, nargs=0, **kwargs) + + def __call__(self, parser, namespace, values, option_string=None): + if namespace.tool: + namespace.tool.ARGS.print_help() + else: + parser.print_help() + parser.exit() + + +tools = { + 'test': Writer, + 'ninja2make': Ninja2Make, + 'compdb': Compdb, + 'clean': Clean, +} + +parser = argparse.ArgumentParser(description='Process and transform build.ninja files.', + add_help=False) +parser.add_argument('-C', metavar='DIR', dest='dir', default='.', + help='change to DIR before doing anything else') +parser.add_argument('-f', metavar='FILE', dest='file', default='build.ninja', + help='specify input build file [default=build.ninja]') +parser.add_argument('-n', dest='dry_run', action='store_true', + help='do not actually do anything') +parser.add_argument('-v', dest='verbose', action='store_true', + help='be more verbose') + +parser.add_argument('-t', dest='tool', choices=tools, action=ToolAction, + help='choose the tool to run') +parser.add_argument('-h', '--help', action=ToolHelpAction, + help='show this help message and exit') + +if len(sys.argv) >= 2 and sys.argv[1] == '--version': + print('1.8') + sys.exit(0) + +args, tool_args = parser.parse_known_args() +args.tool.ARGS.parse_args(tool_args, args) + +os.chdir(args.dir) +with open(args.file, 'r') as f: + parser = NinjaParser(args.file, f) + try: + events = args.tool(sys.stdout, parser, args) + except InvalidArgumentError as e: + parser.error(str(e)) + parser.parse(events) -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system 2019-06-10 11:14 ` [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system Paolo Bonzini @ 2019-06-26 17:34 ` Markus Armbruster 2019-06-27 8:52 ` Paolo Bonzini 2019-06-27 9:46 ` Christophe de Dinechin 2019-06-27 9:03 ` Daniel P. Berrangé 1 sibling, 2 replies; 29+ messages in thread From: Markus Armbruster @ 2019-06-26 17:34 UTC (permalink / raw) To: Paolo Bonzini; +Cc: qemu-devel Paolo Bonzini <pbonzini@redhat.com> writes: > The Meson build system is integrated in the existing configure/make steps > by invoking Meson from the configure script and converting Meson's build.ninja > rules to an included Makefile. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > Makefile | 9 + > configure | 30 ++ > meson.build | 9 + > scripts/ninjatool.py | 964 +++++++++++++++++++++++++++++++++++++++++++++++++++ Uff. > 4 files changed, 1012 insertions(+) > create mode 100644 meson.build > create mode 100644 scripts/ninjatool.py > > diff --git a/Makefile b/Makefile > index 8e2fc66..b8f802c 100644 > --- a/Makefile > +++ b/Makefile > @@ -48,6 +48,15 @@ git-submodule-update: > endif > endif > > +export NINJA=./ninjatool > +Makefile.ninja: build.ninja ninjatool > + ./ninjatool -t ninja2make --omit dist uninstall < $< > $@ > +-include Makefile.ninja > + > +ninjatool: $(SRC_PATH)/scripts/ninjatool.py > + sed -e '1c\' -e '#! $(PYTHON)' $< > $@ > + chmod +x $@ Why do we need this here, but not for other Python scripts? We have 39 Python scripts with #!/usr/bin/env python, one with #!/usr/bin/env python2, and 12 with #!/usr/bin/python. The Makefiles generally use $(PYTHON) SCRIPT ARGS... > + > .git-submodule-status: git-submodule-update config-host.mak > > # Check that we're not trying to do an out-of-tree build from > diff --git a/configure b/configure > index 0814a5f..b8c3c58 100755 > --- a/configure > +++ b/configure > @@ -493,6 +493,7 @@ docker="no" > debug_mutex="no" > libpmem="" > default_devices="yes" > +meson=meson > > # cross compilers defaults, can be overridden with --cross-cc-ARCH > cross_cc_aarch64="aarch64-linux-gnu-gcc" > @@ -983,6 +984,8 @@ for opt do > ;; > --python=*) python="$optarg" > ;; > + --meson=*) meson="$optarg" > + ;; > --gcov=*) gcov_tool="$optarg" > ;; > --smbd=*) smbd="$optarg" > @@ -1685,6 +1688,7 @@ Advanced options (experts only): > --make=MAKE use specified make [$make] > --install=INSTALL use specified install [$install] > --python=PYTHON use specified python [$python] > + --meson=PYTHON use specified meson [$meson] > --smbd=SMBD use specified smbd [$smbd] > --with-git=GIT use specified git [$git] > --static enable static build [$static] > @@ -1850,6 +1854,11 @@ then > error_exit "Python not found. Use --python=/path/to/python" > fi > > +if ! has "$meson" > +then > + error_exit "Meson not found. Use --meson=/path/to/meson" > +fi > + > # Note that if the Python conditional here evaluates True we will exit > # with status 1 which is a shell 'false' value. > if ! $python -c 'import sys; sys.exit(sys.version_info < (2,7))'; then > @@ -7983,6 +7992,27 @@ echo "# Automatically generated by configure - do not modify" > "$iotests_common > echo >> "$iotests_common_env" > echo "export PYTHON='$python'" >> "$iotests_common_env" > > +# bootstrap ninjatool, we need it before Make runs > +if ! test -x ninjatool; then > + sed -e '1c\' -e "#! $python" ${source_path}/scripts/ninjatool.py > ninjatool > + chmod +x ninjatool > +fi > +rm -rf meson-private meson-info meson-logs Ignorant question: why do we need configure remove this stuff? > +NINJA=$PWD/ninjatool $python $meson setup \ This prints /usr/bin/python3: can't open file 'meson': [Errno 2] No such file or directory for me, then goes on happily. For what it's worth: $ type meson meson is /usr/bin/meson Are you sure you want to override /usr/bin/meson's #! line? If I drop $python, I get meson.build:1:0: ERROR: Meson version is 0.50.1 but project requires >=0.50.999. which is expected. It's too hot right for me now to figure out how to obtain a suitable version. > + --prefix "$prefix" \ > + --libdir "$libdir" \ > + --libexecdir "$libexecdir" \ > + --bindir "$bindir" \ > + --includedir "$includedir" \ > + --datadir "$datadir" \ > + --mandir "$mandir" \ > + --sysconfdir "$sysconfdir" \ > + --localstatedir "$local_statedir" \ > + $(test "$strip_opt" = yes && echo --strip) \ > + --buildtype $(if test "$debug" = yes; then echo debug; else echo release; fi) \ > + "$PWD" "$source_path" > + > + > # Save the configure command line for later reuse. > cat <<EOD >config.status > #!/bin/sh > diff --git a/meson.build b/meson.build > new file mode 100644 > index 0000000..b683d70 > --- /dev/null > +++ b/meson.build > @@ -0,0 +1,9 @@ > +project('qemu', 'c', meson_version: '>=0.50.999') > + > +kconfig = import('unstable-kconfig') > +config_host = kconfig.load(meson.current_build_dir() / 'config-host.mak') > + > +add_project_arguments(config_host['QEMU_CFLAGS'].split(), > + language: 'c') > +add_project_arguments(config_host['QEMU_INCLUDES'].split(), > + language: 'c') > diff --git a/scripts/ninjatool.py b/scripts/ninjatool.py > new file mode 100644 > index 0000000..6d90919 > --- /dev/null > +++ b/scripts/ninjatool.py [Lots of code...] Did you write ninjatool.py specifically for QEMU, or did you steal it (or parts) somewhere? ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system 2019-06-26 17:34 ` Markus Armbruster @ 2019-06-27 8:52 ` Paolo Bonzini 2019-06-27 9:46 ` Christophe de Dinechin 1 sibling, 0 replies; 29+ messages in thread From: Paolo Bonzini @ 2019-06-27 8:52 UTC (permalink / raw) To: Markus Armbruster; +Cc: qemu-devel On 26/06/19 19:34, Markus Armbruster wrote: > Paolo Bonzini <pbonzini@redhat.com> writes: > >> The Meson build system is integrated in the existing configure/make steps >> by invoking Meson from the configure script and converting Meson's build.ninja >> rules to an included Makefile. >> >> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> >> --- >> Makefile | 9 + >> configure | 30 ++ >> meson.build | 9 + >> scripts/ninjatool.py | 964 +++++++++++++++++++++++++++++++++++++++++++++++++++ > > Uff. What, the 9+30+9 or the 964? :) >> +export NINJA=./ninjatool >> +Makefile.ninja: build.ninja ninjatool >> + ./ninjatool -t ninja2make --omit dist uninstall < $< > $@ >> +-include Makefile.ninja >> + >> +ninjatool: $(SRC_PATH)/scripts/ninjatool.py >> + sed -e '1c\' -e '#! $(PYTHON)' $< > $@ >> + chmod +x $@ > > Why do we need this here, but not for other Python scripts? > > We have 39 Python scripts with #!/usr/bin/env python, one with > #!/usr/bin/env python2, and 12 with #!/usr/bin/python. The Makefiles > generally use $(PYTHON) SCRIPT ARGS... Because this script is invoked by meson too. While my patch to support a NINJA environment variable was accepted (and we point it to ninjatool), meson invokes it as "$(NINJA)" --foo with no splitting, so we need the above change. But I can do better than this. I can generate the file directly from meson.build: configure_file(input: files('scripts/ninjatool.py'), output: 'ninjatool', configuration: config_host) and avoid code duplication between configure and Makefile. >> +rm -rf meson-private meson-info meson-logs > > Ignorant question: why do we need configure remove this stuff? By default, reinvoking meson complains that you probably want to reconfigure (a la config.status) because there is already a build tree here. Our configure script always reconfigures the tree from scratch. So I just cause meson to fail that check. >> +NINJA=$PWD/ninjatool $python $meson setup \ > > This prints > > /usr/bin/python3: can't open file 'meson': [Errno 2] No such file or directory > > for me, then goes on happily. > > For what it's worth: > > $ type meson > meson is /usr/bin/meson > > Are you sure you want to override /usr/bin/meson's #! line? No I'm not. It should be just $meson, as you do below, though it could also be $python "$(type -p meson)" if I wanted to override---but certainly not $python $meson. It worked for me because I was pointing $meson to an absolute path under my home directory. There is actually one reason to override it, which is why I did it in the patch. Meson has an automagic way to invoke all Python programs with the same interpreter that was used to run meson itself (you make the script non-executable and add "#! /usr/bin/env python3" as the first line). If we want to use that feature, we should invoke meson with $python. > If I drop $python, I get > > meson.build:1:0: ERROR: Meson version is 0.50.1 but project requires >=0.50.999. > > which is expected. > > It's too hot right for me now to figure out how to obtain a suitable > version. FWIW 0.51.0 has now been released so you could grab it from Koji. >> diff --git a/scripts/ninjatool.py b/scripts/ninjatool.py >> new file mode 100644 >> index 0000000..6d90919 >> --- /dev/null >> +++ b/scripts/ninjatool.py > [Lots of code...] > > Did you write ninjatool.py specifically for QEMU, or did you steal it > (or parts) somewhere? I wrote it specifically for QEMU. Paolo ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system 2019-06-26 17:34 ` Markus Armbruster 2019-06-27 8:52 ` Paolo Bonzini @ 2019-06-27 9:46 ` Christophe de Dinechin 1 sibling, 0 replies; 29+ messages in thread From: Christophe de Dinechin @ 2019-06-27 9:46 UTC (permalink / raw) To: qemu-devel; +Cc: Paolo Bonzini Markus Armbruster writes: > Paolo Bonzini <pbonzini@redhat.com> writes: > >> The Meson build system is integrated in the existing configure/make steps >> by invoking Meson from the configure script and converting Meson's build.ninja >> rules to an included Makefile. >> >> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> >> --- >> Makefile | 9 + >> configure | 30 ++ >> meson.build | 9 + >> scripts/ninjatool.py | 964 +++++++++++++++++++++++++++++++++++++++++++++++++++ > > Uff. > >> 4 files changed, 1012 insertions(+) >> create mode 100644 meson.build >> create mode 100644 scripts/ninjatool.py >> >> diff --git a/Makefile b/Makefile >> index 8e2fc66..b8f802c 100644 >> --- a/Makefile >> +++ b/Makefile >> @@ -48,6 +48,15 @@ git-submodule-update: >> endif >> endif >> >> +export NINJA=./ninjatool >> +Makefile.ninja: build.ninja ninjatool >> + ./ninjatool -t ninja2make --omit dist uninstall < $< > $@ >> +-include Makefile.ninja >> + >> +ninjatool: $(SRC_PATH)/scripts/ninjatool.py >> + sed -e '1c\' -e '#! $(PYTHON)' $< > $@ >> + chmod +x $@ > > Why do we need this here, but not for other Python scripts? > > We have 39 Python scripts with #!/usr/bin/env python, one with > #!/usr/bin/env python2, and 12 with #!/usr/bin/python. The Makefiles > generally use $(PYTHON) SCRIPT ARGS... > >> + >> .git-submodule-status: git-submodule-update config-host.mak >> >> # Check that we're not trying to do an out-of-tree build from >> diff --git a/configure b/configure >> index 0814a5f..b8c3c58 100755 >> --- a/configure >> +++ b/configure >> @@ -493,6 +493,7 @@ docker="no" >> debug_mutex="no" >> libpmem="" >> default_devices="yes" >> +meson=meson >> >> # cross compilers defaults, can be overridden with --cross-cc-ARCH >> cross_cc_aarch64="aarch64-linux-gnu-gcc" >> @@ -983,6 +984,8 @@ for opt do >> ;; >> --python=*) python="$optarg" >> ;; >> + --meson=*) meson="$optarg" >> + ;; >> --gcov=*) gcov_tool="$optarg" >> ;; >> --smbd=*) smbd="$optarg" >> @@ -1685,6 +1688,7 @@ Advanced options (experts only): >> --make=MAKE use specified make [$make] >> --install=INSTALL use specified install [$install] >> --python=PYTHON use specified python [$python] >> + --meson=PYTHON use specified meson [$meson] Typo: --meson=MESON >> --smbd=SMBD use specified smbd [$smbd] >> --with-git=GIT use specified git [$git] >> --static enable static build [$static] >> @@ -1850,6 +1854,11 @@ then >> error_exit "Python not found. Use --python=/path/to/python" >> fi >> >> +if ! has "$meson" >> +then >> + error_exit "Meson not found. Use --meson=/path/to/meson" >> +fi >> + >> # Note that if the Python conditional here evaluates True we will exit >> # with status 1 which is a shell 'false' value. >> if ! $python -c 'import sys; sys.exit(sys.version_info < (2,7))'; then >> @@ -7983,6 +7992,27 @@ echo "# Automatically generated by configure - do not modify" > "$iotests_common >> echo >> "$iotests_common_env" >> echo "export PYTHON='$python'" >> "$iotests_common_env" >> >> +# bootstrap ninjatool, we need it before Make runs >> +if ! test -x ninjatool; then >> + sed -e '1c\' -e "#! $python" ${source_path}/scripts/ninjatool.py > ninjatool >> + chmod +x ninjatool >> +fi >> +rm -rf meson-private meson-info meson-logs > > Ignorant question: why do we need configure remove this stuff? > >> +NINJA=$PWD/ninjatool $python $meson setup \ > > This prints > > /usr/bin/python3: can't open file 'meson': [Errno 2] No such file or directory > > for me, then goes on happily. > > For what it's worth: > > $ type meson > meson is /usr/bin/meson > > Are you sure you want to override /usr/bin/meson's #! line? > > If I drop $python, I get > > meson.build:1:0: ERROR: Meson version is 0.50.1 but project requires >=0.50.999. > > which is expected. > > It's too hot right for me now to figure out how to obtain a suitable > version. > >> + --prefix "$prefix" \ >> + --libdir "$libdir" \ >> + --libexecdir "$libexecdir" \ >> + --bindir "$bindir" \ >> + --includedir "$includedir" \ >> + --datadir "$datadir" \ >> + --mandir "$mandir" \ >> + --sysconfdir "$sysconfdir" \ >> + --localstatedir "$local_statedir" \ >> + $(test "$strip_opt" = yes && echo --strip) \ >> + --buildtype $(if test "$debug" = yes; then echo debug; else echo release; fi) \ >> + "$PWD" "$source_path" >> + >> + >> # Save the configure command line for later reuse. >> cat <<EOD >config.status >> #!/bin/sh >> diff --git a/meson.build b/meson.build >> new file mode 100644 >> index 0000000..b683d70 >> --- /dev/null >> +++ b/meson.build >> @@ -0,0 +1,9 @@ >> +project('qemu', 'c', meson_version: '>=0.50.999') >> + >> +kconfig = import('unstable-kconfig') >> +config_host = kconfig.load(meson.current_build_dir() / 'config-host.mak') >> + >> +add_project_arguments(config_host['QEMU_CFLAGS'].split(), >> + language: 'c') >> +add_project_arguments(config_host['QEMU_INCLUDES'].split(), >> + language: 'c') >> diff --git a/scripts/ninjatool.py b/scripts/ninjatool.py >> new file mode 100644 >> index 0000000..6d90919 >> --- /dev/null >> +++ b/scripts/ninjatool.py > [Lots of code...] > > Did you write ninjatool.py specifically for QEMU, or did you steal it > (or parts) somewhere? -- Cheers, Christophe de Dinechin (IRC c3d) ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system 2019-06-10 11:14 ` [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system Paolo Bonzini 2019-06-26 17:34 ` Markus Armbruster @ 2019-06-27 9:03 ` Daniel P. Berrangé 2019-06-27 10:16 ` Paolo Bonzini 1 sibling, 1 reply; 29+ messages in thread From: Daniel P. Berrangé @ 2019-06-27 9:03 UTC (permalink / raw) To: Paolo Bonzini; +Cc: qemu-devel On Mon, Jun 10, 2019 at 01:14:57PM +0200, Paolo Bonzini wrote: > The Meson build system is integrated in the existing configure/make steps > by invoking Meson from the configure script and converting Meson's build.ninja > rules to an included Makefile. Why did you take the route of converting ninja rules into makefile rules, as opposed to just having some stub makefile rules which directly invoke ninja where needed ? Obviously this series is just some initial integration, but eventually when we get a complete conversion, I think it will look pretty wierd if we're still converting ninja to make. Part of the downside of our current build system is that although it uses make, the usage is wierd and QEMU specific structure. It would be a shame to pick meson and then again use it is a way that is wierd and QEMU specific. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system 2019-06-27 9:03 ` Daniel P. Berrangé @ 2019-06-27 10:16 ` Paolo Bonzini 2019-06-27 12:23 ` Markus Armbruster 2019-06-27 12:55 ` Daniel P. Berrangé 0 siblings, 2 replies; 29+ messages in thread From: Paolo Bonzini @ 2019-06-27 10:16 UTC (permalink / raw) To: Daniel P. Berrangé; +Cc: qemu-devel On 27/06/19 11:03, Daniel P. Berrangé wrote: > On Mon, Jun 10, 2019 at 01:14:57PM +0200, Paolo Bonzini wrote: >> The Meson build system is integrated in the existing configure/make steps >> by invoking Meson from the configure script and converting Meson's build.ninja >> rules to an included Makefile. > > Why did you take the route of converting ninja rules into makefile > rules, as opposed to just having some stub makefile rules which > directly invoke ninja where needed ? There are two parts of this. One is practical and has to do with supporting a step-by-step transition. Using ninja2make makes it trivial to have make build products that depend on meson build products, and this way bottom up is a natural direction to do the conversion, which is bottom up. You'd start from libqemuutil.a and code generators (tracing + QAPI), then go to the tools and the emulators. The second is a design decision that simplifying the Make/meson integration is *not* a goal. Rather the goals are: 1) making the transition easier on developers; 2) avoiding magic in meson.build at all costs. More specifically: - it should remain trivial to do things that used to be trivial, and most "make" invocations should be kept the same at least until everything is converted and we can perhaps declare a flag day. People are used to "make check" or "make subdir-x86_64-softmmu", those should continue to work while the transition is in progress. - it should be possible to modify meson.build without knowing QEMU specific details, and that should be _already_ possible now at the beginning of the transition (to avoid creating technical debt). This means keeping the magic confined in Makefile rules and external scripts, while having a pretty plain meson.build. I expect testing might also require some hand-holding, because "meson test" does not integrate with "make -j" and to keep supporting our "make check-*" targets. However, until the make->ninja flag day we could generate tap-driver Makefile rules from "meson introspect --tests" output. Basically I'm dropping Makefile magic in favor of build rule generators are written in high-level languages. > Obviously this series is just some initial integration, but eventually > when we get a complete conversion, I think it will look pretty wierd > if we're still converting ninja to make. I agree; once all the build rules are converted the Makefile could be as simple as all: include config.mak include tests/docker/Makefile.include include tests/vm/Makefile.include .NOTPARALLEL: %: ninja $@ subdir-%-softmmu: ninja qemu-system-$* subdir-%-linux-user: ninja qemu-$* check: $(MESON) test check-%: $(MESON) test --suite $* etc. (and likewise the configure script could just translate the command line options before invoking meson). But for now, since rules are written half in meson and half in make, ninja2make seems the most transparent way to integrate the two. > Part of the downside of our current build system is that although it > uses make, the usage is wierd and QEMU specific structure. It would > be a shame to pick meson and then again use it is a way that is wierd > and QEMU specific. I agree, this is why it's important to have at least a standard meson.build. Some knowledge of config-host.mak is needed, because meson.build uses declare_dependency() instead of dependency() to link with libraries that were already found by configure, but that's it. Paolo ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system 2019-06-27 10:16 ` Paolo Bonzini @ 2019-06-27 12:23 ` Markus Armbruster 2019-06-27 12:50 ` Daniel P. Berrangé 2019-06-27 12:57 ` Paolo Bonzini 2019-06-27 12:55 ` Daniel P. Berrangé 1 sibling, 2 replies; 29+ messages in thread From: Markus Armbruster @ 2019-06-27 12:23 UTC (permalink / raw) To: Paolo Bonzini; +Cc: Daniel P. Berrangé, qemu-devel Paolo Bonzini <pbonzini@redhat.com> writes: > On 27/06/19 11:03, Daniel P. Berrangé wrote: >> On Mon, Jun 10, 2019 at 01:14:57PM +0200, Paolo Bonzini wrote: >>> The Meson build system is integrated in the existing configure/make steps >>> by invoking Meson from the configure script and converting Meson's build.ninja >>> rules to an included Makefile. >> >> Why did you take the route of converting ninja rules into makefile >> rules, as opposed to just having some stub makefile rules which >> directly invoke ninja where needed ? > > There are two parts of this. One is practical and has to do with > supporting a step-by-step transition. Using ninja2make makes it trivial > to have make build products that depend on meson build products, and > this way bottom up is a natural direction to do the conversion, which is > bottom up. You'd start from libqemuutil.a and code generators (tracing > + QAPI), then go to the tools and the emulators. *If* the conversion is too big a task to permit doing it all at once, then the step by step strategy you describe makes sense to me. The trouble with step by step is running out of steam before the final step. That would leave us worse off. Even an overly protracted conversion would be bad. Thus, my standard question on any proposed step-by-step conversion: commitment to finishing it? I'd be quite happy to take your word for it. > The second is a design decision that simplifying the Make/meson > integration is *not* a goal. Rather the goals are: 1) making the > transition easier on developers; 2) avoiding magic in meson.build at all > costs. More specifically: > > - it should remain trivial to do things that used to be trivial, and > most "make" invocations should be kept the same at least until > everything is converted and we can perhaps declare a flag day. People > are used to "make check" or "make subdir-x86_64-softmmu", those should > continue to work while the transition is in progress. > > - it should be possible to modify meson.build without knowing QEMU > specific details, and that should be _already_ possible now at the > beginning of the transition (to avoid creating technical debt). This > means keeping the magic confined in Makefile rules and external scripts, > while having a pretty plain meson.build. I wholeheartedly agree with (2). Reducing magic is the whole point. Your plan confines new magic to "Makefile rules and external scripts". We'll get actual reduction only if we can retire or at least radically simplify them at some point. I'm more ambivalent on (1). Yes, making the transition easier for developers is worth hiding a certain amount of magic out of their way. But not any amount. We should try to preserve the targets printed by "make help". > I expect testing might also require some hand-holding, because "meson > test" does not integrate with "make -j" and to keep supporting our "make > check-*" targets. However, until the make->ninja flag day we could > generate tap-driver Makefile rules from "meson introspect --tests" > output. Basically I'm dropping Makefile magic in favor of build rule > generators are written in high-level languages. A PoC for selected tests would be nice. >> Obviously this series is just some initial integration, but eventually >> when we get a complete conversion, I think it will look pretty wierd >> if we're still converting ninja to make. > > I agree; once all the build rules are converted the Makefile could be as > simple as > > all: > include config.mak > include tests/docker/Makefile.include > include tests/vm/Makefile.include > .NOTPARALLEL: > %: > ninja $@ > > subdir-%-softmmu: > ninja qemu-system-$* > > subdir-%-linux-user: > ninja qemu-$* > > check: > $(MESON) test > > check-%: > $(MESON) test --suite $* > > etc. (and likewise the configure script could just translate the command > line options before invoking meson). I wouldn't bother keeping around such an empty shell of a Makefile. Keeping configure as a wrapper could be marginally more useful, mostly for people trying to build QEMU for the first time. > But for now, since rules are > written half in meson and half in make, ninja2make seems the most > transparent way to integrate the two. > >> Part of the downside of our current build system is that although it >> uses make, the usage is wierd and QEMU specific structure. It would >> be a shame to pick meson and then again use it is a way that is wierd >> and QEMU specific. > > I agree, this is why it's important to have at least a standard meson.build. > > Some knowledge of config-host.mak is needed, because meson.build uses > declare_dependency() instead of dependency() to link with libraries that > were already found by configure, but that's it. Ignorant question: could the switch to Meson enable doing less in configure? It's big and sloooow. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system 2019-06-27 12:23 ` Markus Armbruster @ 2019-06-27 12:50 ` Daniel P. Berrangé 2019-06-27 12:57 ` Paolo Bonzini 1 sibling, 0 replies; 29+ messages in thread From: Daniel P. Berrangé @ 2019-06-27 12:50 UTC (permalink / raw) To: Markus Armbruster; +Cc: Paolo Bonzini, qemu-devel On Thu, Jun 27, 2019 at 02:23:52PM +0200, Markus Armbruster wrote: > Paolo Bonzini <pbonzini@redhat.com> writes: > > But for now, since rules are > > written half in meson and half in make, ninja2make seems the most > > transparent way to integrate the two. > > > >> Part of the downside of our current build system is that although it > >> uses make, the usage is wierd and QEMU specific structure. It would > >> be a shame to pick meson and then again use it is a way that is wierd > >> and QEMU specific. > > > > I agree, this is why it's important to have at least a standard meson.build. > > > > Some knowledge of config-host.mak is needed, because meson.build uses > > declare_dependency() instead of dependency() to link with libraries that > > were already found by configure, but that's it. > > Ignorant question: could the switch to Meson enable doing less in > configure? It's big and sloooow. At a conceptual level meson is supposed to take over the role of both configure and automake, so you have a single domain specific language covering all aspects of your build system (though sometimes needs some external python helper scripts to cover missing features in meson). IOW, in a proper conversion, I'd expect configure to go away. I can imagine that being one of the last steps in QEMU's conversion though Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system 2019-06-27 12:23 ` Markus Armbruster 2019-06-27 12:50 ` Daniel P. Berrangé @ 2019-06-27 12:57 ` Paolo Bonzini 1 sibling, 0 replies; 29+ messages in thread From: Paolo Bonzini @ 2019-06-27 12:57 UTC (permalink / raw) To: Markus Armbruster; +Cc: Daniel P. Berrangé, qemu-devel On 27/06/19 14:23, Markus Armbruster wrote: > Paolo Bonzini <pbonzini@redhat.com> writes: >> On 27/06/19 11:03, Daniel P. Berrangé wrote: >> There are two parts of this. One is practical and has to do with >> supporting a step-by-step transition. Using ninja2make makes it trivial >> to have make build products that depend on meson build products, and >> this way bottom up is a natural direction to do the conversion, which is >> bottom up. You'd start from libqemuutil.a and code generators (tracing >> + QAPI), then go to the tools and the emulators. > > *If* the conversion is too big a task to permit doing it all at once, > then the step by step strategy you describe makes sense to me. > > The trouble with step by step is running out of steam before the final > step. That would leave us worse off. Even an overly protracted > conversion would be bad. Agreed. But hey, Makefile.objs is 2000 lines of lists of files, it is boring but it cannot take that long to convert. So it's not really about being _possible_ to do it all at once, it's mostly about the manageability of conflicts. > Thus, my standard question on any proposed step-by-step conversion: > commitment to finishing it? I'd be quite happy to take your word for > it. I cannot really make such a commitment now, but perhaps I can (or I and someone else can) once a little more of the conversion is ready. Especially QAPI and trace generators, since those are where most of the magic resides. I can try that since your reaction wasn't of total disgust. :) >> The second is a design decision that simplifying the Make/meson >> integration is *not* a goal. Rather the goals are: 1) making the >> transition easier on developers; 2) avoiding magic in meson.build at all >> costs. More specifically: >> > Your plan confines new magic to "Makefile rules and external scripts". > We'll get actual reduction only if we can retire or at least radically > simplify them at some point. But even before that, we'll get *practical* reduction if it hacking the Makefile rules and external scripts becomes rare enough. See for example kconfig, where # of people hacking minikconf.py is much less than # of people hacking default-configs. > I'm more ambivalent on (1). Yes, making the transition easier for > developers is worth hiding a certain amount of magic out of their way. It's especially worth during the transition. Once Makefile.objs is gone I agree we can afford more radical changes, but let's cross that bridge when we get there. >> I expect testing might also require some hand-holding, because "meson >> test" does not integrate with "make -j" and to keep supporting our "make >> check-*" targets. However, until the make->ninja flag day we could >> generate tap-driver Makefile rules from "meson introspect --tests" >> output. Basically I'm dropping Makefile magic in favor of build rule >> generators are written in high-level languages. > > A PoC for selected tests would be nice. Fair enough. > Ignorant question: could the switch to Meson enable doing less in > configure? It's big and sloooow. It would be smaller since the DSL is more compact than shell scripts, probably not much faster as the same tasks still have to be done. But perhaps in the future Meson can parallelize them, and then we'd get that for free. Paolo ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system 2019-06-27 10:16 ` Paolo Bonzini 2019-06-27 12:23 ` Markus Armbruster @ 2019-06-27 12:55 ` Daniel P. Berrangé 2019-06-27 13:16 ` Gerd Hoffmann 1 sibling, 1 reply; 29+ messages in thread From: Daniel P. Berrangé @ 2019-06-27 12:55 UTC (permalink / raw) To: Paolo Bonzini; +Cc: qemu-devel On Thu, Jun 27, 2019 at 12:16:05PM +0200, Paolo Bonzini wrote: > On 27/06/19 11:03, Daniel P. Berrangé wrote: > > On Mon, Jun 10, 2019 at 01:14:57PM +0200, Paolo Bonzini wrote: > >> The Meson build system is integrated in the existing configure/make steps > >> by invoking Meson from the configure script and converting Meson's build.ninja > >> rules to an included Makefile. > > > > Why did you take the route of converting ninja rules into makefile > > rules, as opposed to just having some stub makefile rules which > > directly invoke ninja where needed ? > > There are two parts of this. One is practical and has to do with > supporting a step-by-step transition. Using ninja2make makes it trivial > to have make build products that depend on meson build products, and > this way bottom up is a natural direction to do the conversion, which is > bottom up. You'd start from libqemuutil.a and code generators (tracing > + QAPI), then go to the tools and the emulators. Ok, I can understand that. I've been thinking about how we can switch libvirt to use meson too, and trying to decide between meson being the owner, calling out to make vs keeping make as the owner and calling out to meson. Ultimately to entirely banish make, autoconf, automake, libtool, m4 & shell from our build system :-) Despite thinking about an incremental conversion though, I was still hoping libvirt would just have a single (largish) patch series to do a complete conversion at a specific point in time. > > Obviously this series is just some initial integration, but eventually > > when we get a complete conversion, I think it will look pretty wierd > > if we're still converting ninja to make. > > I agree; once all the build rules are converted the Makefile could be as > simple as > > all: > include config.mak > include tests/docker/Makefile.include > include tests/vm/Makefile.include > .NOTPARALLEL: > %: > ninja $@ > > subdir-%-softmmu: > ninja qemu-system-$* > > subdir-%-linux-user: > ninja qemu-$* > > check: > $(MESON) test > > check-%: > $(MESON) test --suite $* > > etc. (and likewise the configure script could just translate the command > line options before invoking meson). But for now, since rules are > written half in meson and half in make, ninja2make seems the most > transparent way to integrate the two. Ok, I think it understand the motiviation better now that its obviously just a short term expedient step to convert ninja to make. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system 2019-06-27 12:55 ` Daniel P. Berrangé @ 2019-06-27 13:16 ` Gerd Hoffmann 2019-06-27 13:20 ` Paolo Bonzini 2019-06-27 13:27 ` Daniel P. Berrangé 0 siblings, 2 replies; 29+ messages in thread From: Gerd Hoffmann @ 2019-06-27 13:16 UTC (permalink / raw) To: Daniel P. Berrangé; +Cc: Paolo Bonzini, qemu-devel Hi, > Ok, I can understand that. I've been thinking about how we can switch > libvirt to use meson too, and trying to decide between meson being the > owner, calling out to make vs keeping make as the owner and calling > out to meson. Ultimately to entirely banish make, autoconf, automake, > libtool, m4 & shell from our build system :-) > > Despite thinking about an incremental conversion though, I was still > hoping libvirt would just have a single (largish) patch series to > do a complete conversion at a specific point in time. Another possible approach would be to have two build systems. The traditional configure & make and the new meson & ninja. Advantage is we don't have to worry about the transition and mixing & make + meson at all. Disadvantage is the duplication. That wouldn't be forever though. I'd expect we'll have one or maybe two releases with both build systems, then delete the make & configure. cheers, Gerd ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system 2019-06-27 13:16 ` Gerd Hoffmann @ 2019-06-27 13:20 ` Paolo Bonzini 2019-06-27 13:27 ` Daniel P. Berrangé 1 sibling, 0 replies; 29+ messages in thread From: Paolo Bonzini @ 2019-06-27 13:20 UTC (permalink / raw) To: Gerd Hoffmann, Daniel P. Berrangé; +Cc: qemu-devel On 27/06/19 15:16, Gerd Hoffmann wrote: > Hi, > >> Ok, I can understand that. I've been thinking about how we can switch >> libvirt to use meson too, and trying to decide between meson being the >> owner, calling out to make vs keeping make as the owner and calling >> out to meson. Ultimately to entirely banish make, autoconf, automake, >> libtool, m4 & shell from our build system :-) >> >> Despite thinking about an incremental conversion though, I was still >> hoping libvirt would just have a single (largish) patch series to >> do a complete conversion at a specific point in time. > > Another possible approach would be to have two build systems. > The traditional configure & make and the new meson & ninja. > > Advantage is we don't have to worry about the transition and mixing & > make + meson at all. > > Disadvantage is the duplication. That wouldn't be forever though. > I'd expect we'll have one or maybe two releases with both build systems, > then delete the make & configure. Yeah, I have seen other projects do this. On the other hand I'm not sure how you avoid bitrot in one of the two, and having to convert the huge configure script from the beginning seems like a daunting task. Paolo ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system 2019-06-27 13:16 ` Gerd Hoffmann 2019-06-27 13:20 ` Paolo Bonzini @ 2019-06-27 13:27 ` Daniel P. Berrangé 1 sibling, 0 replies; 29+ messages in thread From: Daniel P. Berrangé @ 2019-06-27 13:27 UTC (permalink / raw) To: Gerd Hoffmann; +Cc: Paolo Bonzini, qemu-devel On Thu, Jun 27, 2019 at 03:16:01PM +0200, Gerd Hoffmann wrote: > Hi, > > > Ok, I can understand that. I've been thinking about how we can switch > > libvirt to use meson too, and trying to decide between meson being the > > owner, calling out to make vs keeping make as the owner and calling > > out to meson. Ultimately to entirely banish make, autoconf, automake, > > libtool, m4 & shell from our build system :-) > > > > Despite thinking about an incremental conversion though, I was still > > hoping libvirt would just have a single (largish) patch series to > > do a complete conversion at a specific point in time. > > Another possible approach would be to have two build systems. > The traditional configure & make and the new meson & ninja. > > Advantage is we don't have to worry about the transition and mixing & > make + meson at all. > > Disadvantage is the duplication. That wouldn't be forever though. > I'd expect we'll have one or maybe two releases with both build systems, > then delete the make & configure. Personally I strongly wish to avoid having two parallel build systems. In several projects I've encountered with that I've ended up hitting bugs where each build system produced different binary results due to the developers not putting the same changes into both build systems. Prefer to only merge a new build system once there is confidence that it is working, and then immediately delete the old one so as to avoid bitrot & concentrate any bug fixing on the modern stuff. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH 4/7] libvhost-user: convert to Meson 2019-06-10 11:14 [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration Paolo Bonzini ` (2 preceding siblings ...) 2019-06-10 11:14 ` [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system Paolo Bonzini @ 2019-06-10 11:14 ` Paolo Bonzini 2019-06-27 9:03 ` Markus Armbruster 2019-06-10 11:14 ` [Qemu-devel] [PATCH 5/7] vhost-user-blk: " Paolo Bonzini ` (6 subsequent siblings) 10 siblings, 1 reply; 29+ messages in thread From: Paolo Bonzini @ 2019-06-10 11:14 UTC (permalink / raw) To: qemu-devel Because libqemuutil.a is not converted yet, selected files have to be compiled twice, once with Meson and once with Makefile.objs. Apart from this the conversion is straightforward. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- Makefile | 10 ++++------ Makefile.objs | 1 - contrib/libvhost-user/Makefile.objs | 1 - contrib/libvhost-user/meson.build | 7 +++++++ meson.build | 2 ++ 5 files changed, 13 insertions(+), 8 deletions(-) delete mode 100644 contrib/libvhost-user/Makefile.objs create mode 100644 contrib/libvhost-user/meson.build diff --git a/Makefile b/Makefile index b8f802c..6a3461e 100644 --- a/Makefile +++ b/Makefile @@ -426,7 +426,6 @@ dummy := $(call unnest-vars,, \ ivshmem-client-obj-y \ ivshmem-server-obj-y \ rdmacm-mux-obj-y \ - libvhost-user-obj-y \ vhost-user-scsi-obj-y \ vhost-user-blk-obj-y \ vhost-user-input-obj-y \ @@ -529,7 +528,6 @@ Makefile: $(version-obj-y) # Build libraries libqemuutil.a: $(util-obj-y) $(trace-obj-y) $(stub-obj-y) -libvhost-user.a: $(libvhost-user-obj-y) $(util-obj-y) $(stub-obj-y) ###################################################################### @@ -632,21 +630,21 @@ ivshmem-client$(EXESUF): $(ivshmem-client-obj-y) $(COMMON_LDADDS) ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS) $(call LINK, $^) endif -vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) libvhost-user.a +vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) contrib/libvhost-user/libvhost-user.a $(call LINK, $^) -vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) libvhost-user.a +vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) contrib/libvhost-user/libvhost-user.a $(call LINK, $^) rdmacm-mux$(EXESUF): LIBS += "-libumad" rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS) $(call LINK, $^) -vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) $(libvhost-user-obj-y) libqemuutil.a libqemustub.a +vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) contrib/libvhost-user/libvhost-user.a $(call LINK, $^) ifdef CONFIG_VHOST_USER_INPUT ifdef CONFIG_LINUX -vhost-user-input$(EXESUF): $(vhost-user-input-obj-y) libvhost-user.a libqemuutil.a +vhost-user-input$(EXESUF): $(vhost-user-input-obj-y) contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS) $(call LINK, $^) # build by default, do not install diff --git a/Makefile.objs b/Makefile.objs index c8337fa..4f2fa6a 100644 --- a/Makefile.objs +++ b/Makefile.objs @@ -116,7 +116,6 @@ qga-vss-dll-obj-y = qga/ elf2dmp-obj-y = contrib/elf2dmp/ ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/ ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/ -libvhost-user-obj-y = contrib/libvhost-user/ vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS) vhost-user-scsi.o-libs := $(LIBISCSI_LIBS) vhost-user-scsi-obj-y = contrib/vhost-user-scsi/ diff --git a/contrib/libvhost-user/Makefile.objs b/contrib/libvhost-user/Makefile.objs deleted file mode 100644 index ef3778e..0000000 --- a/contrib/libvhost-user/Makefile.objs +++ /dev/null @@ -1 +0,0 @@ -libvhost-user-obj-y += libvhost-user.o libvhost-user-glib.o diff --git a/contrib/libvhost-user/meson.build b/contrib/libvhost-user/meson.build new file mode 100644 index 0000000..7ef610a --- /dev/null +++ b/contrib/libvhost-user/meson.build @@ -0,0 +1,7 @@ +libvhost_user = static_library('vhost-user', + [files('libvhost-user.c', 'libvhost-user-glib.c'), + meson.source_root() / 'stubs/error-printf.c', + meson.source_root() / 'stubs/monitor.c', + meson.source_root() / 'util/error.c', + meson.source_root() / 'util/qemu-error.c', + meson.source_root() / 'util/memfd.c']) diff --git a/meson.build b/meson.build index b683d70..a6748f9 100644 --- a/meson.build +++ b/meson.build @@ -7,3 +7,5 @@ add_project_arguments(config_host['QEMU_CFLAGS'].split(), language: 'c') add_project_arguments(config_host['QEMU_INCLUDES'].split(), language: 'c') + +subdir('contrib/libvhost-user') -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 4/7] libvhost-user: convert to Meson 2019-06-10 11:14 ` [Qemu-devel] [PATCH 4/7] libvhost-user: convert to Meson Paolo Bonzini @ 2019-06-27 9:03 ` Markus Armbruster 0 siblings, 0 replies; 29+ messages in thread From: Markus Armbruster @ 2019-06-27 9:03 UTC (permalink / raw) To: Paolo Bonzini; +Cc: qemu-devel Paolo Bonzini <pbonzini@redhat.com> writes: > Because libqemuutil.a is not converted yet, selected files have to be > compiled twice, once with Meson and once with Makefile.objs. Apart > from this the conversion is straightforward. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > Makefile | 10 ++++------ > Makefile.objs | 1 - > contrib/libvhost-user/Makefile.objs | 1 - > contrib/libvhost-user/meson.build | 7 +++++++ > meson.build | 2 ++ > 5 files changed, 13 insertions(+), 8 deletions(-) > delete mode 100644 contrib/libvhost-user/Makefile.objs > create mode 100644 contrib/libvhost-user/meson.build > > diff --git a/Makefile b/Makefile > index b8f802c..6a3461e 100644 > --- a/Makefile > +++ b/Makefile > @@ -426,7 +426,6 @@ dummy := $(call unnest-vars,, \ > ivshmem-client-obj-y \ > ivshmem-server-obj-y \ > rdmacm-mux-obj-y \ > - libvhost-user-obj-y \ > vhost-user-scsi-obj-y \ > vhost-user-blk-obj-y \ > vhost-user-input-obj-y \ > @@ -529,7 +528,6 @@ Makefile: $(version-obj-y) > # Build libraries > > libqemuutil.a: $(util-obj-y) $(trace-obj-y) $(stub-obj-y) > -libvhost-user.a: $(libvhost-user-obj-y) $(util-obj-y) $(stub-obj-y) > > ###################################################################### > > @@ -632,21 +630,21 @@ ivshmem-client$(EXESUF): $(ivshmem-client-obj-y) $(COMMON_LDADDS) > ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS) > $(call LINK, $^) > endif > -vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) libvhost-user.a > +vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) contrib/libvhost-user/libvhost-user.a > $(call LINK, $^) > -vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) libvhost-user.a > +vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) contrib/libvhost-user/libvhost-user.a > $(call LINK, $^) > > rdmacm-mux$(EXESUF): LIBS += "-libumad" > rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS) > $(call LINK, $^) > > -vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) $(libvhost-user-obj-y) libqemuutil.a libqemustub.a > +vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) contrib/libvhost-user/libvhost-user.a > $(call LINK, $^) > > ifdef CONFIG_VHOST_USER_INPUT > ifdef CONFIG_LINUX > -vhost-user-input$(EXESUF): $(vhost-user-input-obj-y) libvhost-user.a libqemuutil.a > +vhost-user-input$(EXESUF): $(vhost-user-input-obj-y) contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS) > $(call LINK, $^) > > # build by default, do not install This hunk is only due to the move of libvhost-user.a from the root of the build tree to contrib/libvhost-user/. I'm not counting it as "meson churn". > diff --git a/Makefile.objs b/Makefile.objs > index c8337fa..4f2fa6a 100644 > --- a/Makefile.objs > +++ b/Makefile.objs > @@ -116,7 +116,6 @@ qga-vss-dll-obj-y = qga/ > elf2dmp-obj-y = contrib/elf2dmp/ > ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/ > ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/ > -libvhost-user-obj-y = contrib/libvhost-user/ > vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS) > vhost-user-scsi.o-libs := $(LIBISCSI_LIBS) > vhost-user-scsi-obj-y = contrib/vhost-user-scsi/ > diff --git a/contrib/libvhost-user/Makefile.objs b/contrib/libvhost-user/Makefile.objs > deleted file mode 100644 > index ef3778e..0000000 > --- a/contrib/libvhost-user/Makefile.objs > +++ /dev/null > @@ -1 +0,0 @@ > -libvhost-user-obj-y += libvhost-user.o libvhost-user-glib.o > diff --git a/contrib/libvhost-user/meson.build b/contrib/libvhost-user/meson.build > new file mode 100644 > index 0000000..7ef610a > --- /dev/null > +++ b/contrib/libvhost-user/meson.build > @@ -0,0 +1,7 @@ > +libvhost_user = static_library('vhost-user', > + [files('libvhost-user.c', 'libvhost-user-glib.c'), These are the "selected files [that] have to be compiled twice, once with Meson and once with Makefile.objs": > + meson.source_root() / 'stubs/error-printf.c', > + meson.source_root() / 'stubs/monitor.c', > + meson.source_root() / 'util/error.c', > + meson.source_root() / 'util/qemu-error.c', > + meson.source_root() / 'util/memfd.c']) Quite okay for a PoC. > diff --git a/meson.build b/meson.build > index b683d70..a6748f9 100644 > --- a/meson.build > +++ b/meson.build > @@ -7,3 +7,5 @@ add_project_arguments(config_host['QEMU_CFLAGS'].split(), > language: 'c') > add_project_arguments(config_host['QEMU_INCLUDES'].split(), > language: 'c') > + > +subdir('contrib/libvhost-user') In your cover letter, you wrote "it's a simple example of how the current build system causes clutter in the toplevel Makefiles and how interrelated the Makefile, Makefile.objs and contrib/*/Makefile.objs file are even for such simple executables." True. Before this patch, the logic for building libvhost-user.a is split over Makefile (two lines), Makefile.obj (one) and contrib/libvhost-user/Makefile.objs (one). Afterwards, its all in one place. ^ permalink raw reply [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH 5/7] vhost-user-blk: convert to Meson 2019-06-10 11:14 [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration Paolo Bonzini ` (3 preceding siblings ...) 2019-06-10 11:14 ` [Qemu-devel] [PATCH 4/7] libvhost-user: convert to Meson Paolo Bonzini @ 2019-06-10 11:14 ` Paolo Bonzini 2019-06-10 11:15 ` [Qemu-devel] [PATCH 6/7] vhost-user-scsi: " Paolo Bonzini ` (5 subsequent siblings) 10 siblings, 0 replies; 29+ messages in thread From: Paolo Bonzini @ 2019-06-10 11:14 UTC (permalink / raw) To: qemu-devel The GLib pkg-config information is extracted from config-host.mak and used to link vhost-user-blk. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- .gitignore | 2 +- Makefile | 3 --- Makefile.objs | 1 - configure | 2 ++ contrib/vhost-user-blk/Makefile.objs | 1 - contrib/vhost-user-blk/meson.build | 3 +++ meson.build | 4 ++++ 7 files changed, 10 insertions(+), 6 deletions(-) delete mode 100644 contrib/vhost-user-blk/Makefile.objs create mode 100644 contrib/vhost-user-blk/meson.build diff --git a/.gitignore b/.gitignore index fd6e6c3..20637a1 100644 --- a/.gitignore +++ b/.gitignore @@ -64,7 +64,7 @@ /module_block.h /scsi/qemu-pr-helper /vhost-user-scsi -/vhost-user-blk +/contrib/vhost-user-blk/vhost-user-blk /fsdev/virtfs-proxy-helper *.tmp *.[1-9] diff --git a/Makefile b/Makefile index 6a3461e..da290fa 100644 --- a/Makefile +++ b/Makefile @@ -427,7 +427,6 @@ dummy := $(call unnest-vars,, \ ivshmem-server-obj-y \ rdmacm-mux-obj-y \ vhost-user-scsi-obj-y \ - vhost-user-blk-obj-y \ vhost-user-input-obj-y \ vhost-user-gpu-obj-y \ qga-vss-dll-obj-y \ @@ -632,8 +631,6 @@ ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS) endif vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) contrib/libvhost-user/libvhost-user.a $(call LINK, $^) -vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) contrib/libvhost-user/libvhost-user.a - $(call LINK, $^) rdmacm-mux$(EXESUF): LIBS += "-libumad" rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS) diff --git a/Makefile.objs b/Makefile.objs index 4f2fa6a..644e2bd 100644 --- a/Makefile.objs +++ b/Makefile.objs @@ -119,7 +119,6 @@ ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/ vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS) vhost-user-scsi.o-libs := $(LIBISCSI_LIBS) vhost-user-scsi-obj-y = contrib/vhost-user-scsi/ -vhost-user-blk-obj-y = contrib/vhost-user-blk/ rdmacm-mux-obj-y = contrib/rdmacm-mux/ vhost-user-input-obj-y = contrib/vhost-user-input/ vhost-user-gpu-obj-y = contrib/vhost-user-gpu/ diff --git a/configure b/configure index b8c3c58..f00a9e4 100755 --- a/configure +++ b/configure @@ -7364,6 +7364,8 @@ echo "CFLAGS_NOPIE=$CFLAGS_NOPIE" >> $config_host_mak echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak echo "QEMU_CXXFLAGS=$QEMU_CXXFLAGS" >> $config_host_mak echo "QEMU_INCLUDES=$QEMU_INCLUDES" >> $config_host_mak +echo "GLIB_CFLAGS=$glib_cflags" >> $config_host_mak +echo "GLIB_LIBS=$glib_libs" >> $config_host_mak if test "$sparse" = "yes" ; then echo "CC := REAL_CC=\"\$(CC)\" cgcc" >> $config_host_mak echo "CPP := REAL_CC=\"\$(CPP)\" cgcc" >> $config_host_mak diff --git a/contrib/vhost-user-blk/Makefile.objs b/contrib/vhost-user-blk/Makefile.objs deleted file mode 100644 index 72e2cdc..0000000 --- a/contrib/vhost-user-blk/Makefile.objs +++ /dev/null @@ -1 +0,0 @@ -vhost-user-blk-obj-y = vhost-user-blk.o diff --git a/contrib/vhost-user-blk/meson.build b/contrib/vhost-user-blk/meson.build new file mode 100644 index 0000000..6c00b3b --- /dev/null +++ b/contrib/vhost-user-blk/meson.build @@ -0,0 +1,3 @@ +executable('vhost-user-blk', files('vhost-user-blk.c'), + link_with: libvhost_user, + dependencies: glib) diff --git a/meson.build b/meson.build index a6748f9..c4cad8f 100644 --- a/meson.build +++ b/meson.build @@ -8,4 +8,8 @@ add_project_arguments(config_host['QEMU_CFLAGS'].split(), add_project_arguments(config_host['QEMU_INCLUDES'].split(), language: 'c') +glib = declare_dependency(compile_args: config_host['GLIB_CFLAGS'].split(), + link_args: config_host['GLIB_LIBS'].split()) + subdir('contrib/libvhost-user') +subdir('contrib/vhost-user-blk') -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH 6/7] vhost-user-scsi: convert to Meson 2019-06-10 11:14 [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration Paolo Bonzini ` (4 preceding siblings ...) 2019-06-10 11:14 ` [Qemu-devel] [PATCH 5/7] vhost-user-blk: " Paolo Bonzini @ 2019-06-10 11:15 ` Paolo Bonzini 2019-06-27 11:23 ` Markus Armbruster 2019-06-10 11:15 ` [Qemu-devel] [PATCH 7/7] rdmacm-mux: " Paolo Bonzini ` (4 subsequent siblings) 10 siblings, 1 reply; 29+ messages in thread From: Paolo Bonzini @ 2019-06-10 11:15 UTC (permalink / raw) To: qemu-devel The libiscsi pkg-config information is extracted from config-host.mak and used to link vhost-user-blk. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- .gitignore | 2 +- Makefile | 3 --- Makefile.objs | 3 --- contrib/vhost-user-scsi/Makefile.objs | 1 - contrib/vhost-user-scsi/meson.build | 3 +++ meson.build | 3 +++ 6 files changed, 7 insertions(+), 8 deletions(-) delete mode 100644 contrib/vhost-user-scsi/Makefile.objs create mode 100644 contrib/vhost-user-scsi/meson.build diff --git a/.gitignore b/.gitignore index 20637a1..3934eff 100644 --- a/.gitignore +++ b/.gitignore @@ -63,7 +63,7 @@ /qemu-version.h.tmp /module_block.h /scsi/qemu-pr-helper -/vhost-user-scsi +/contrib/vhost-user-scsi/vhost-user-scsi /contrib/vhost-user-blk/vhost-user-blk /fsdev/virtfs-proxy-helper *.tmp diff --git a/Makefile b/Makefile index da290fa..bff097c 100644 --- a/Makefile +++ b/Makefile @@ -426,7 +426,6 @@ dummy := $(call unnest-vars,, \ ivshmem-client-obj-y \ ivshmem-server-obj-y \ rdmacm-mux-obj-y \ - vhost-user-scsi-obj-y \ vhost-user-input-obj-y \ vhost-user-gpu-obj-y \ qga-vss-dll-obj-y \ @@ -629,8 +628,6 @@ ivshmem-client$(EXESUF): $(ivshmem-client-obj-y) $(COMMON_LDADDS) ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS) $(call LINK, $^) endif -vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) contrib/libvhost-user/libvhost-user.a - $(call LINK, $^) rdmacm-mux$(EXESUF): LIBS += "-libumad" rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS) diff --git a/Makefile.objs b/Makefile.objs index 644e2bd..adf2f92 100644 --- a/Makefile.objs +++ b/Makefile.objs @@ -116,9 +116,6 @@ qga-vss-dll-obj-y = qga/ elf2dmp-obj-y = contrib/elf2dmp/ ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/ ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/ -vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS) -vhost-user-scsi.o-libs := $(LIBISCSI_LIBS) -vhost-user-scsi-obj-y = contrib/vhost-user-scsi/ rdmacm-mux-obj-y = contrib/rdmacm-mux/ vhost-user-input-obj-y = contrib/vhost-user-input/ vhost-user-gpu-obj-y = contrib/vhost-user-gpu/ diff --git a/contrib/vhost-user-scsi/Makefile.objs b/contrib/vhost-user-scsi/Makefile.objs deleted file mode 100644 index e83a38a..0000000 --- a/contrib/vhost-user-scsi/Makefile.objs +++ /dev/null @@ -1 +0,0 @@ -vhost-user-scsi-obj-y = vhost-user-scsi.o diff --git a/contrib/vhost-user-scsi/meson.build b/contrib/vhost-user-scsi/meson.build new file mode 100644 index 0000000..b80398c --- /dev/null +++ b/contrib/vhost-user-scsi/meson.build @@ -0,0 +1,3 @@ +executable('vhost-user-scsi', files('vhost-user-scsi.c'), + link_with: libvhost_user, + dependencies: [glib, libiscsi]) diff --git a/meson.build b/meson.build index c4cad8f..bc7fbea 100644 --- a/meson.build +++ b/meson.build @@ -10,6 +10,9 @@ add_project_arguments(config_host['QEMU_INCLUDES'].split(), glib = declare_dependency(compile_args: config_host['GLIB_CFLAGS'].split(), link_args: config_host['GLIB_LIBS'].split()) +libiscsi = declare_dependency(compile_args: config_host['LIBISCSI_CFLAGS'].split(), + link_args: config_host['LIBISCSI_LIBS'].split()) subdir('contrib/libvhost-user') subdir('contrib/vhost-user-blk') +subdir('contrib/vhost-user-scsi') -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 6/7] vhost-user-scsi: convert to Meson 2019-06-10 11:15 ` [Qemu-devel] [PATCH 6/7] vhost-user-scsi: " Paolo Bonzini @ 2019-06-27 11:23 ` Markus Armbruster 2019-06-27 12:31 ` Paolo Bonzini 0 siblings, 1 reply; 29+ messages in thread From: Markus Armbruster @ 2019-06-27 11:23 UTC (permalink / raw) To: Paolo Bonzini; +Cc: qemu-devel Paolo Bonzini <pbonzini@redhat.com> writes: > The libiscsi pkg-config information is extracted from config-host.mak and > used to link vhost-user-blk. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > .gitignore | 2 +- > Makefile | 3 --- > Makefile.objs | 3 --- > contrib/vhost-user-scsi/Makefile.objs | 1 - > contrib/vhost-user-scsi/meson.build | 3 +++ > meson.build | 3 +++ > 6 files changed, 7 insertions(+), 8 deletions(-) > delete mode 100644 contrib/vhost-user-scsi/Makefile.objs > create mode 100644 contrib/vhost-user-scsi/meson.build > > diff --git a/.gitignore b/.gitignore > index 20637a1..3934eff 100644 > --- a/.gitignore > +++ b/.gitignore > @@ -63,7 +63,7 @@ > /qemu-version.h.tmp > /module_block.h > /scsi/qemu-pr-helper > -/vhost-user-scsi > +/contrib/vhost-user-scsi/vhost-user-scsi > /contrib/vhost-user-blk/vhost-user-blk > /fsdev/virtfs-proxy-helper > *.tmp > diff --git a/Makefile b/Makefile > index da290fa..bff097c 100644 > --- a/Makefile > +++ b/Makefile > @@ -426,7 +426,6 @@ dummy := $(call unnest-vars,, \ > ivshmem-client-obj-y \ > ivshmem-server-obj-y \ > rdmacm-mux-obj-y \ > - vhost-user-scsi-obj-y \ > vhost-user-input-obj-y \ > vhost-user-gpu-obj-y \ > qga-vss-dll-obj-y \ > @@ -629,8 +628,6 @@ ivshmem-client$(EXESUF): $(ivshmem-client-obj-y) $(COMMON_LDADDS) > ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS) > $(call LINK, $^) > endif > -vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) contrib/libvhost-user/libvhost-user.a > - $(call LINK, $^) > > rdmacm-mux$(EXESUF): LIBS += "-libumad" > rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS) > diff --git a/Makefile.objs b/Makefile.objs > index 644e2bd..adf2f92 100644 > --- a/Makefile.objs > +++ b/Makefile.objs > @@ -116,9 +116,6 @@ qga-vss-dll-obj-y = qga/ > elf2dmp-obj-y = contrib/elf2dmp/ > ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/ > ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/ > -vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS) > -vhost-user-scsi.o-libs := $(LIBISCSI_LIBS) > -vhost-user-scsi-obj-y = contrib/vhost-user-scsi/ The casual reader may wonder what these variables do. The %.o: %.c rule in rules.mak passes $($@-cflags) to $(CC). With meson, ... > rdmacm-mux-obj-y = contrib/rdmacm-mux/ > vhost-user-input-obj-y = contrib/vhost-user-input/ > vhost-user-gpu-obj-y = contrib/vhost-user-gpu/ > diff --git a/contrib/vhost-user-scsi/Makefile.objs b/contrib/vhost-user-scsi/Makefile.objs > deleted file mode 100644 > index e83a38a..0000000 > --- a/contrib/vhost-user-scsi/Makefile.objs > +++ /dev/null > @@ -1 +0,0 @@ > -vhost-user-scsi-obj-y = vhost-user-scsi.o > diff --git a/contrib/vhost-user-scsi/meson.build b/contrib/vhost-user-scsi/meson.build > new file mode 100644 > index 0000000..b80398c > --- /dev/null > +++ b/contrib/vhost-user-scsi/meson.build > @@ -0,0 +1,3 @@ > +executable('vhost-user-scsi', files('vhost-user-scsi.c'), > + link_with: libvhost_user, > + dependencies: [glib, libiscsi]) ... we instead name dependencies, which ... > diff --git a/meson.build b/meson.build > index c4cad8f..bc7fbea 100644 > --- a/meson.build > +++ b/meson.build > @@ -10,6 +10,9 @@ add_project_arguments(config_host['QEMU_INCLUDES'].split(), > > glib = declare_dependency(compile_args: config_host['GLIB_CFLAGS'].split(), > link_args: config_host['GLIB_LIBS'].split()) > +libiscsi = declare_dependency(compile_args: config_host['LIBISCSI_CFLAGS'].split(), > + link_args: config_host['LIBISCSI_LIBS'].split()) ... we define here. Less magical. Note that there's also glib dependency, which works the same (see previous patch). With Make, we add GLib flags to QEMU_CFLAGS and LIBS, so they get used for everything unless overridden. > > subdir('contrib/libvhost-user') > subdir('contrib/vhost-user-blk') > +subdir('contrib/vhost-user-scsi') ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 6/7] vhost-user-scsi: convert to Meson 2019-06-27 11:23 ` Markus Armbruster @ 2019-06-27 12:31 ` Paolo Bonzini 0 siblings, 0 replies; 29+ messages in thread From: Paolo Bonzini @ 2019-06-27 12:31 UTC (permalink / raw) To: Markus Armbruster; +Cc: qemu-devel On 27/06/19 13:23, Markus Armbruster wrote: > Note that there's also glib dependency, which works the same (see > previous patch). With Make, we add GLib flags to QEMU_CFLAGS and LIBS, > so they get used for everything unless overridden. Having to add glib everywhere is mildly annoying. We can avoid that once we convert libqemuutil.a to meson, via libqemuutil = static_library('qemuutil', ...) libqemuutil = declare_dependency(link_with: libqemuutil, dependencies: glib) >> +libiscsi = declare_dependency(compile_args: config_host['LIBISCSI_CFLAGS'].split(), >> + link_args: config_host['LIBISCSI_LIBS'].split()) > ... we define here. Less magical. A couple notes here. First, later on this declare_dependency() will be replaced by dependency(), so that meson itself will run pkg-config rather than our configure script. Second, some magic will reappear once we start using sourcesets[1], which allow dependencies will be attached to sources rather than executables. However, there will still be no need to track cflags and ldflags separately. It will look like this: # in block/meson.build block.add(when: libiscsi, if_true: files('iscsi.c')) # in toplevel meson.build block_obj = block.apply(config_host) executable('qemu-img', sources: ['qemu-img.c', block_obj.sources()], dependencies: [libqemutil, block_obj.dependencies()]) Paolo [1] https://mesonbuild.com/SourceSet-module.html ^ permalink raw reply [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH 7/7] rdmacm-mux: convert to Meson 2019-06-10 11:14 [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration Paolo Bonzini ` (5 preceding siblings ...) 2019-06-10 11:15 ` [Qemu-devel] [PATCH 6/7] vhost-user-scsi: " Paolo Bonzini @ 2019-06-10 11:15 ` Paolo Bonzini 2019-06-27 11:38 ` Markus Armbruster 2019-06-10 12:33 ` [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration no-reply ` (3 subsequent siblings) 10 siblings, 1 reply; 29+ messages in thread From: Paolo Bonzini @ 2019-06-10 11:15 UTC (permalink / raw) To: qemu-devel We can use config-host.mak to decide whether the tool has to be built, apart from that the conversion is straightforward. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- .gitignore | 1 + Makefile | 5 ----- Makefile.objs | 1 - contrib/rdmacm-mux/Makefile.objs | 3 --- contrib/rdmacm-mux/meson.build | 6 ++++++ meson.build | 2 ++ 6 files changed, 9 insertions(+), 9 deletions(-) delete mode 100644 contrib/rdmacm-mux/Makefile.objs create mode 100644 contrib/rdmacm-mux/meson.build diff --git a/.gitignore b/.gitignore index 3934eff..b8d38a8 100644 --- a/.gitignore +++ b/.gitignore @@ -63,6 +63,7 @@ /qemu-version.h.tmp /module_block.h /scsi/qemu-pr-helper +/contrib/rdmacm-mux/rdmacm-mux /contrib/vhost-user-scsi/vhost-user-scsi /contrib/vhost-user-blk/vhost-user-blk /fsdev/virtfs-proxy-helper diff --git a/Makefile b/Makefile index bff097c..713f301 100644 --- a/Makefile +++ b/Makefile @@ -425,7 +425,6 @@ dummy := $(call unnest-vars,, \ elf2dmp-obj-y \ ivshmem-client-obj-y \ ivshmem-server-obj-y \ - rdmacm-mux-obj-y \ vhost-user-input-obj-y \ vhost-user-gpu-obj-y \ qga-vss-dll-obj-y \ @@ -629,10 +628,6 @@ ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS) $(call LINK, $^) endif -rdmacm-mux$(EXESUF): LIBS += "-libumad" -rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS) - $(call LINK, $^) - vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) contrib/libvhost-user/libvhost-user.a $(call LINK, $^) diff --git a/Makefile.objs b/Makefile.objs index adf2f92..cf02f63 100644 --- a/Makefile.objs +++ b/Makefile.objs @@ -116,7 +116,6 @@ qga-vss-dll-obj-y = qga/ elf2dmp-obj-y = contrib/elf2dmp/ ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/ ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/ -rdmacm-mux-obj-y = contrib/rdmacm-mux/ vhost-user-input-obj-y = contrib/vhost-user-input/ vhost-user-gpu-obj-y = contrib/vhost-user-gpu/ diff --git a/contrib/rdmacm-mux/Makefile.objs b/contrib/rdmacm-mux/Makefile.objs deleted file mode 100644 index 3df744a..0000000 --- a/contrib/rdmacm-mux/Makefile.objs +++ /dev/null @@ -1,3 +0,0 @@ -ifdef CONFIG_PVRDMA -rdmacm-mux-obj-y = main.o -endif diff --git a/contrib/rdmacm-mux/meson.build b/contrib/rdmacm-mux/meson.build new file mode 100644 index 0000000..8451756 --- /dev/null +++ b/contrib/rdmacm-mux/meson.build @@ -0,0 +1,6 @@ +if config_host['CONFIG_PVRDMA'] == 'y' + # if not found, CONFIG_PVRDMA should not be set + libumad = cc.find_library('ibumad', required: true) + executable('rdmacm-mux', files('main.c'), + dependencies: [glib, libumad]) +endif diff --git a/meson.build b/meson.build index bc7fbea..2fc6111 100644 --- a/meson.build +++ b/meson.build @@ -1,4 +1,5 @@ project('qemu', 'c', meson_version: '>=0.50.999') +cc = meson.get_compiler('c') kconfig = import('unstable-kconfig') config_host = kconfig.load(meson.current_build_dir() / 'config-host.mak') @@ -14,5 +15,6 @@ libiscsi = declare_dependency(compile_args: config_host['LIBISCSI_CFLAGS'].split link_args: config_host['LIBISCSI_LIBS'].split()) subdir('contrib/libvhost-user') +subdir('contrib/rdmacm-mux') subdir('contrib/vhost-user-blk') subdir('contrib/vhost-user-scsi') -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 7/7] rdmacm-mux: convert to Meson 2019-06-10 11:15 ` [Qemu-devel] [PATCH 7/7] rdmacm-mux: " Paolo Bonzini @ 2019-06-27 11:38 ` Markus Armbruster 2019-06-27 12:21 ` Paolo Bonzini 0 siblings, 1 reply; 29+ messages in thread From: Markus Armbruster @ 2019-06-27 11:38 UTC (permalink / raw) To: Paolo Bonzini; +Cc: qemu-devel Paolo Bonzini <pbonzini@redhat.com> writes: > We can use config-host.mak to decide whether the tool has to be built, > apart from that the conversion is straightforward. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > .gitignore | 1 + > Makefile | 5 ----- > Makefile.objs | 1 - > contrib/rdmacm-mux/Makefile.objs | 3 --- > contrib/rdmacm-mux/meson.build | 6 ++++++ > meson.build | 2 ++ > 6 files changed, 9 insertions(+), 9 deletions(-) > delete mode 100644 contrib/rdmacm-mux/Makefile.objs > create mode 100644 contrib/rdmacm-mux/meson.build > > diff --git a/.gitignore b/.gitignore > index 3934eff..b8d38a8 100644 > --- a/.gitignore > +++ b/.gitignore > @@ -63,6 +63,7 @@ > /qemu-version.h.tmp > /module_block.h > /scsi/qemu-pr-helper > +/contrib/rdmacm-mux/rdmacm-mux > /contrib/vhost-user-scsi/vhost-user-scsi > /contrib/vhost-user-blk/vhost-user-blk > /fsdev/virtfs-proxy-helper > diff --git a/Makefile b/Makefile > index bff097c..713f301 100644 > --- a/Makefile > +++ b/Makefile > @@ -425,7 +425,6 @@ dummy := $(call unnest-vars,, \ > elf2dmp-obj-y \ > ivshmem-client-obj-y \ > ivshmem-server-obj-y \ > - rdmacm-mux-obj-y \ > vhost-user-input-obj-y \ > vhost-user-gpu-obj-y \ > qga-vss-dll-obj-y \ > @@ -629,10 +628,6 @@ ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS) > $(call LINK, $^) > endif > > -rdmacm-mux$(EXESUF): LIBS += "-libumad" > -rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS) > - $(call LINK, $^) > - Another way to pass extra flags to the compiler: target-specific variables. More direct than the funny variables we saw in the previous patch. The fine manual describes a way how to shoot yourself in the foot with target-specific variabes. Been there, done that, had "fun" debugging. > vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) contrib/libvhost-user/libvhost-user.a > $(call LINK, $^) > > diff --git a/Makefile.objs b/Makefile.objs > index adf2f92..cf02f63 100644 > --- a/Makefile.objs > +++ b/Makefile.objs > @@ -116,7 +116,6 @@ qga-vss-dll-obj-y = qga/ > elf2dmp-obj-y = contrib/elf2dmp/ > ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/ > ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/ > -rdmacm-mux-obj-y = contrib/rdmacm-mux/ > vhost-user-input-obj-y = contrib/vhost-user-input/ > vhost-user-gpu-obj-y = contrib/vhost-user-gpu/ > > diff --git a/contrib/rdmacm-mux/Makefile.objs b/contrib/rdmacm-mux/Makefile.objs > deleted file mode 100644 > index 3df744a..0000000 > --- a/contrib/rdmacm-mux/Makefile.objs > +++ /dev/null > @@ -1,3 +0,0 @@ > -ifdef CONFIG_PVRDMA > -rdmacm-mux-obj-y = main.o > -endif > diff --git a/contrib/rdmacm-mux/meson.build b/contrib/rdmacm-mux/meson.build > new file mode 100644 > index 0000000..8451756 > --- /dev/null > +++ b/contrib/rdmacm-mux/meson.build > @@ -0,0 +1,6 @@ > +if config_host['CONFIG_PVRDMA'] == 'y' > + # if not found, CONFIG_PVRDMA should not be set Our Makefiles use "either unset or 'y'" booleans pretty pervasively. Is it idiomatic meson? > + libumad = cc.find_library('ibumad', required: true) With configure & make, we have configure check -libumad works, and make use it. Ignorant question: cc.find_library() looks like it checks. Could this replace checking in configure? > + executable('rdmacm-mux', files('main.c'), > + dependencies: [glib, libumad]) > +endif > diff --git a/meson.build b/meson.build > index bc7fbea..2fc6111 100644 > --- a/meson.build > +++ b/meson.build > @@ -1,4 +1,5 @@ > project('qemu', 'c', meson_version: '>=0.50.999') > +cc = meson.get_compiler('c') > > kconfig = import('unstable-kconfig') > config_host = kconfig.load(meson.current_build_dir() / 'config-host.mak') > @@ -14,5 +15,6 @@ libiscsi = declare_dependency(compile_args: config_host['LIBISCSI_CFLAGS'].split > link_args: config_host['LIBISCSI_LIBS'].split()) > > subdir('contrib/libvhost-user') > +subdir('contrib/rdmacm-mux') > subdir('contrib/vhost-user-blk') > subdir('contrib/vhost-user-scsi') ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH 7/7] rdmacm-mux: convert to Meson 2019-06-27 11:38 ` Markus Armbruster @ 2019-06-27 12:21 ` Paolo Bonzini 0 siblings, 0 replies; 29+ messages in thread From: Paolo Bonzini @ 2019-06-27 12:21 UTC (permalink / raw) To: Markus Armbruster; +Cc: qemu-devel On 27/06/19 13:38, Markus Armbruster wrote: >> +if config_host['CONFIG_PVRDMA'] == 'y' >> + # if not found, CONFIG_PVRDMA should not be set > Our Makefiles use "either unset or 'y'" booleans pretty pervasively. Is > it idiomatic meson? No, the idiomatic way would be to use the found() method. Something like: # in toplevel meson.build: ibverbs = cc.find_library('ibverbs') ibumad = cc.find_library('ibumad') rdmacm = cc.find_library('rdmacm') if ibverbs.found() and ibumad.found() and rdmacm.found() pvrdma = declare_dependency(dependencies: [ibverbs, ibumad, rdmacm] else # magic "not found" dependency pvrdma = dependency('', required: false) endif # in contrib/rdmacm-mux/meson.build if pvrdma.found() then executable('rdmacm-mux', files('main.c'), dependencies: [glib, ibumad]) endif >> + libumad = cc.find_library('ibumad', required: true) > With configure & make, we have configure check -libumad works, and make > use it. > > Ignorant question: cc.find_library() looks like it checks. Could this > replace checking in configure? Yes, see above. However, you would have to remove all uses of $(RDMA_LIBS) and in the Makefiles first. Paolo ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration 2019-06-10 11:14 [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration Paolo Bonzini ` (6 preceding siblings ...) 2019-06-10 11:15 ` [Qemu-devel] [PATCH 7/7] rdmacm-mux: " Paolo Bonzini @ 2019-06-10 12:33 ` no-reply 2019-06-10 12:36 ` no-reply ` (2 subsequent siblings) 10 siblings, 0 replies; 29+ messages in thread From: no-reply @ 2019-06-10 12:33 UTC (permalink / raw) To: pbonzini; +Cc: qemu-devel Patchew URL: https://patchew.org/QEMU/1560165301-39026-1-git-send-email-pbonzini@redhat.com/ Hi, This series seems to have some coding style problems. See output below for more information: Subject: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration Type: series Message-id: 1560165301-39026-1-git-send-email-pbonzini@redhat.com === TEST SCRIPT BEGIN === #!/bin/bash git rev-parse base > /dev/null || exit 0 git config --local diff.renamelimit 0 git config --local diff.renames True git config --local diff.algorithm histogram ./scripts/checkpatch.pl --mailback base.. === TEST SCRIPT END === From https://github.com/patchew-project/qemu 6960005212..19735c837a master -> master From https://github.com/patchew-project/qemu * [new tag] patchew/1560165301-39026-1-git-send-email-pbonzini@redhat.com -> patchew/1560165301-39026-1-git-send-email-pbonzini@redhat.com Switched to a new branch 'test' d681fb0695 rdmacm-mux: convert to Meson 2c12fcafd1 vhost-user-scsi: convert to Meson 7b87823a4c vhost-user-blk: convert to Meson 45233abe03 libvhost-user: convert to Meson 090f670ef8 configure: integrate Meson in the build system 9d9a4b1926 configure: set $PYTHON to a full path d47d1b7f13 configure: do not include $(...) variables in config-host.mak === OUTPUT BEGIN === 1/7 Checking commit d47d1b7f1366 (configure: do not include $(...) variables in config-host.mak) 2/7 Checking commit 9d9a4b192614 (configure: set $PYTHON to a full path) 3/7 Checking commit 090f670ef84c (configure: integrate Meson in the build system) WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #104: new file mode 100644 WARNING: line over 80 characters #283: FILE: scripts/ninjatool.py:160: + raise LexerError("%s:%d: %s" % (self.stack[-1].filename, self.lineno, msg)) WARNING: line over 80 characters #286: FILE: scripts/ninjatool.py:163: + raise ParseError("%s:%d: %s" % (self.stack[-1].filename, self.lineno, msg)) ERROR: line over 90 characters #668: FILE: scripts/ninjatool.py:545: + self.scope = BuildScope(self, out, iout, rule, in_, iin, orderdep, self.rule_vars[rule]) WARNING: line over 80 characters #703: FILE: scripts/ninjatool.py:580: + ARGS = argparse.ArgumentParser(description='Rewrite input build.ninja to stdout.') WARNING: line over 80 characters #738: FILE: scripts/ninjatool.py:615: + def begin_build(self, outputs, implicit_outputs, rule, inputs, implicit, order_only): WARNING: line over 80 characters #843: FILE: scripts/ninjatool.py:720: + ARGS = argparse.ArgumentParser(description='Convert build.ninja to a Makefile.') ERROR: line over 90 characters #897: FILE: scripts/ninjatool.py:774: + self.print(r'ninja-clean:: ninja-clean-start; $(if $V,,@)rm -f ${ninja-depfiles}') ERROR: line over 90 characters #911: FILE: scripts/ninjatool.py:788: + self.print(r"ninja-command = $(if $V,,@printf '%s\n' '$(subst ','\'',${.var.description})' && )${.var.command}") ERROR: line over 90 characters #937: FILE: scripts/ninjatool.py:814: + self.print('ninja-targets-%s := %s' % (rule, ' '.join(self.rule_targets[rule]))) ERROR: line over 90 characters #939: FILE: scripts/ninjatool.py:816: + self.print('$(ninja-targets-%s): .var.%s = %s' % (rule, name, self.mangle_vars_for_make(value))) WARNING: line over 80 characters #940: FILE: scripts/ninjatool.py:817: + self.print('ninja-outputdirs += $(sort $(dir ${ninja-targets-%s}))' % rule) WARNING: line over 80 characters #951: FILE: scripts/ninjatool.py:828: + default_targets = sorted(self.all_outs - self.all_ins, key=natural_sort_key) WARNING: line over 80 characters #960: FILE: scripts/ninjatool.py:837: + phony_targets = sorted(self.all_phony - self.all_ins, key=natural_sort_key) WARNING: line over 80 characters #969: FILE: scripts/ninjatool.py:846: + # These have to be printed last, because they override rule variables WARNING: line over 80 characters #970: FILE: scripts/ninjatool.py:847: + # but we cannot print rule variables until we know the list of targets WARNING: line over 80 characters #972: FILE: scripts/ninjatool.py:849: + self.build_vars[self.current_targets][name] = self.scope.build_vars[name] WARNING: line over 80 characters #991: FILE: scripts/ninjatool.py:868: + self.scope.in_ + self.scope.iin + self.scope.orderdep) WARNING: line over 80 characters #998: FILE: scripts/ninjatool.py:875: + self.print('%s: %s | %s; ${ninja-command}' % (targets, inputs, orderonly)) ERROR: line over 90 characters #1024: FILE: scripts/ninjatool.py:901: + def __init__(self, option_strings, dest, choices, metavar='TOOL', nargs=None, **kwargs): WARNING: line over 80 characters #1057: FILE: scripts/ninjatool.py:934: +parser = argparse.ArgumentParser(description='Process and transform build.ninja files.', total: 6 errors, 15 warnings, 1048 lines checked Patch 3/7 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. 4/7 Checking commit 45233abe03bb (libvhost-user: convert to Meson) WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #73: deleted file mode 100644 total: 0 errors, 1 warnings, 58 lines checked Patch 4/7 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. 5/7 Checking commit 7b87823a4cbe (vhost-user-blk: convert to Meson) WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #73: deleted file mode 100644 total: 0 errors, 1 warnings, 49 lines checked Patch 5/7 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. 6/7 Checking commit 2c12fcafd119 (vhost-user-scsi: convert to Meson) WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #62: deleted file mode 100644 total: 0 errors, 1 warnings, 44 lines checked Patch 6/7 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. 7/7 Checking commit d681fb0695f1 (rdmacm-mux: convert to Meson) WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #61: deleted file mode 100644 total: 0 errors, 1 warnings, 48 lines checked Patch 7/7 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. === OUTPUT END === Test command exited with code: 1 The full log is available at http://patchew.org/logs/1560165301-39026-1-git-send-email-pbonzini@redhat.com/testing.checkpatch/?type=message. --- Email generated automatically by Patchew [https://patchew.org/]. Please send your feedback to patchew-devel@redhat.com ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration 2019-06-10 11:14 [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration Paolo Bonzini ` (7 preceding siblings ...) 2019-06-10 12:33 ` [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration no-reply @ 2019-06-10 12:36 ` no-reply 2019-06-10 12:40 ` no-reply 2019-06-27 12:39 ` Markus Armbruster 10 siblings, 0 replies; 29+ messages in thread From: no-reply @ 2019-06-10 12:36 UTC (permalink / raw) To: pbonzini; +Cc: qemu-devel Patchew URL: https://patchew.org/QEMU/1560165301-39026-1-git-send-email-pbonzini@redhat.com/ Hi, This series failed build test on s390x host. Please find the details below. === TEST SCRIPT BEGIN === #!/bin/bash # Testing script will be invoked under the git checkout with # HEAD pointing to a commit that has the patches applied on top of "base" # branch set -e CC=$HOME/bin/cc INSTALL=$PWD/install BUILD=$PWD/build mkdir -p $BUILD $INSTALL SRC=$PWD cd $BUILD $SRC/configure --cc=$CC --prefix=$INSTALL make -j4 # XXX: we need reliable clean up # make check -j4 V=1 make install echo echo "=== ENV ===" env echo echo "=== PACKAGES ===" rpm -qa === TEST SCRIPT END === === OUTPUT BEGIN === ERROR: Meson not found. Use --meson=/path/to/meson === OUTPUT END === The full log is available at http://patchew.org/logs/1560165301-39026-1-git-send-email-pbonzini@redhat.com/testing.s390x/?type=message. --- Email generated automatically by Patchew [https://patchew.org/]. Please send your feedback to patchew-devel@redhat.com ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration 2019-06-10 11:14 [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration Paolo Bonzini ` (8 preceding siblings ...) 2019-06-10 12:36 ` no-reply @ 2019-06-10 12:40 ` no-reply 2019-06-27 12:39 ` Markus Armbruster 10 siblings, 0 replies; 29+ messages in thread From: no-reply @ 2019-06-10 12:40 UTC (permalink / raw) To: pbonzini; +Cc: qemu-devel Patchew URL: https://patchew.org/QEMU/1560165301-39026-1-git-send-email-pbonzini@redhat.com/ Hi, This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN === #!/bin/bash time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1 === TEST SCRIPT END === Configure options: --enable-werror --target-list=x86_64-softmmu --prefix=/tmp/qemu-test/install --python=/usr/bin/python3 --enable-debug --enable-sanitizers --cxx=clang++ --cc=clang --host-cc=clang ERROR: Meson not found. Use --meson=/path/to/meson # QEMU configure log Mon Jun 10 12:40:26 UTC 2019 # Configured with: '/tmp/qemu-test/src/configure' '--enable-werror' '--target-list=x86_64-softmmu' '--prefix=/tmp/qemu-test/install' '--python=/usr/bin/python3' '--enable-debug' '--enable-sanitizers' '--cxx=clang++' '--cc=clang' '--host-cc=clang' --- funcs: do_compiler do_cc compile_object check_define main lines: 92 122 622 691 0 clang -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99 -c -o config-temp/qemu-conf.o config-temp/qemu-conf.c config-temp/qemu-conf.c:2:2: error: __i386__ not defined #error __i386__ not defined ^ 1 error generated. --- funcs: do_compiler do_cc compile_object check_define main lines: 92 122 622 694 0 clang -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99 -c -o config-temp/qemu-conf.o config-temp/qemu-conf.c config-temp/qemu-conf.c:2:2: error: __ILP32__ not defined #error __ILP32__ not defined ^ 1 error generated. The full log is available at http://patchew.org/logs/1560165301-39026-1-git-send-email-pbonzini@redhat.com/testing.asan/?type=message. --- Email generated automatically by Patchew [https://patchew.org/]. Please send your feedback to patchew-devel@redhat.com ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration 2019-06-10 11:14 [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration Paolo Bonzini ` (9 preceding siblings ...) 2019-06-10 12:40 ` no-reply @ 2019-06-27 12:39 ` Markus Armbruster 10 siblings, 0 replies; 29+ messages in thread From: Markus Armbruster @ 2019-06-27 12:39 UTC (permalink / raw) To: Paolo Bonzini; +Cc: qemu-devel Paolo Bonzini <pbonzini@redhat.com> writes: > Hi all, > > this is just a simple example of introducing Meson into the build system, > converting a few contrib/ subprojects to use it instead of Makefile.objs. > There are certainly many rough edges, but I wanted to throw this out to > show what it could look like. > > I chose contrib/ because it's a simple example of how the current build > system causes clutter in the toplevel Makefiles and how interrelated the > Makefile, Makefile.objs and contrib/*/Makefile.objs file are even for > such simple executables. This left a pleasant first impression, apart from the rather large ninjatools.py (which I didn't actually review). I think I'm at the point where I go "okay, now show me something complicated". Something going beyond "build this from that", where both "this" and "that" are basically literals. Show us how to *compute* in the brave new Meson world. To avoid misunderstandings: I think the less computing we need in the build system, the better. But I doubt we can banish it entirely there. ^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2019-06-27 13:29 UTC | newest] Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-06-10 11:14 [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration Paolo Bonzini 2019-06-10 11:14 ` [Qemu-devel] [PATCH 1/7] configure: do not include $(...) variables in config-host.mak Paolo Bonzini 2019-06-10 11:14 ` [Qemu-devel] [PATCH 2/7] configure: set $PYTHON to a full path Paolo Bonzini 2019-06-10 11:14 ` [Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system Paolo Bonzini 2019-06-26 17:34 ` Markus Armbruster 2019-06-27 8:52 ` Paolo Bonzini 2019-06-27 9:46 ` Christophe de Dinechin 2019-06-27 9:03 ` Daniel P. Berrangé 2019-06-27 10:16 ` Paolo Bonzini 2019-06-27 12:23 ` Markus Armbruster 2019-06-27 12:50 ` Daniel P. Berrangé 2019-06-27 12:57 ` Paolo Bonzini 2019-06-27 12:55 ` Daniel P. Berrangé 2019-06-27 13:16 ` Gerd Hoffmann 2019-06-27 13:20 ` Paolo Bonzini 2019-06-27 13:27 ` Daniel P. Berrangé 2019-06-10 11:14 ` [Qemu-devel] [PATCH 4/7] libvhost-user: convert to Meson Paolo Bonzini 2019-06-27 9:03 ` Markus Armbruster 2019-06-10 11:14 ` [Qemu-devel] [PATCH 5/7] vhost-user-blk: " Paolo Bonzini 2019-06-10 11:15 ` [Qemu-devel] [PATCH 6/7] vhost-user-scsi: " Paolo Bonzini 2019-06-27 11:23 ` Markus Armbruster 2019-06-27 12:31 ` Paolo Bonzini 2019-06-10 11:15 ` [Qemu-devel] [PATCH 7/7] rdmacm-mux: " Paolo Bonzini 2019-06-27 11:38 ` Markus Armbruster 2019-06-27 12:21 ` Paolo Bonzini 2019-06-10 12:33 ` [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration no-reply 2019-06-10 12:36 ` no-reply 2019-06-10 12:40 ` no-reply 2019-06-27 12:39 ` Markus Armbruster
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).