* [PATCH 0/7] build: replace ninjatool with ninja @ 2020-10-14 13:54 Paolo Bonzini 2020-10-14 13:54 ` [PATCH 1/7] tests/Makefile.include: unbreak non-tcg builds Paolo Bonzini ` (9 more replies) 0 siblings, 10 replies; 29+ messages in thread From: Paolo Bonzini @ 2020-10-14 13:54 UTC (permalink / raw) To: qemu-devel; +Cc: berrange, alex.bennee, peter.maydell This pull request is the last build system change for 5.2 from me, and it is simple: similar to how we are invoking ROM or tests/tcg "make" from the main Makefile, we now invoke ninja to build QEMU. Unlike those cases, however, build.ninja targets are forwarded transparently. The advantages cover various areas: - maintainability: we drop scripts/ninjatool.py, which is a large and hairy piece of code, and generally remove one thing that can go wrong and one thing that is unique to QEMU; - platform support: we remove the requirement for GNU make 3.82, which was annoying on Mac. We also avoid bugs on Windows due to meson emitting Windows rather than POSIX escapes (as expected by Ninja) and ninjatool ignoring the difference; - speed: invoking "configure" does not have to generate 44k lines of rules, while invoking "Make" does not anymore have to parse 44k lines of rules. - ease of use: Ninja tracks command lines, hence the problem of static library changing the objects they hold goes away Paolo Paolo Bonzini (7): tests/Makefile.include: unbreak non-tcg builds make: run shell with pipefail tests: add missing generated sources to testqapi configure: move QEMU_INCLUDES to meson dockerfiles: enable Centos 8 PowerTools add ninja to dockerfiles, CI configurations and test VMs build: replace ninjatool with ninja .cirrus.yml | 6 +- .travis.yml | 13 + Makefile | 42 +- configure | 29 +- docs/devel/build-system.rst | 6 +- meson.build | 34 +- scripts/mtest2make.py | 4 +- scripts/ninjatool.py | 1008 -------------------- tests/Makefile.include | 2 +- tests/docker/dockerfiles/centos7.docker | 1 + tests/docker/dockerfiles/centos8.docker | 4 + tests/docker/dockerfiles/debian10.docker | 1 + tests/docker/dockerfiles/fedora.docker | 1 + tests/docker/dockerfiles/travis.docker | 2 +- tests/docker/dockerfiles/ubuntu.docker | 1 + tests/docker/dockerfiles/ubuntu1804.docker | 1 + tests/docker/dockerfiles/ubuntu2004.docker | 1 + tests/include/meson.build | 8 +- tests/meson.build | 14 +- tests/vm/centos | 2 +- tests/vm/centos.aarch64 | 2 +- tests/vm/fedora | 2 +- tests/vm/freebsd | 1 + tests/vm/netbsd | 1 + tests/vm/openbsd | 1 + tests/vm/ubuntu.aarch64 | 2 +- tests/vm/ubuntu.i386 | 2 +- 27 files changed, 121 insertions(+), 1070 deletions(-) delete mode 100755 scripts/ninjatool.py -- 2.26.2 ^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 1/7] tests/Makefile.include: unbreak non-tcg builds 2020-10-14 13:54 [PATCH 0/7] build: replace ninjatool with ninja Paolo Bonzini @ 2020-10-14 13:54 ` Paolo Bonzini 2020-10-14 16:26 ` Daniel P. Berrangé 2020-10-14 13:54 ` [PATCH 2/7] make: run shell with pipefail Paolo Bonzini ` (8 subsequent siblings) 9 siblings, 1 reply; 29+ messages in thread From: Paolo Bonzini @ 2020-10-14 13:54 UTC (permalink / raw) To: qemu-devel; +Cc: berrange, alex.bennee, peter.maydell Remove from check-block the requirement that all TARGET_DIRS are built. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- tests/Makefile.include | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/Makefile.include b/tests/Makefile.include index 5aca98e60c..4037490b69 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -140,7 +140,7 @@ QEMU_IOTESTS_HELPERS-$(CONFIG_LINUX) = tests/qemu-iotests/socket_scm_helper$(EXE check: check-block check-block: $(SRC_PATH)/tests/check-block.sh qemu-img$(EXESUF) \ qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) \ - $(patsubst %-softmmu,qemu-system-%,$(filter %-softmmu,$(TARGET_DIRS))) + $(filter qemu-system-%, $(ninja-targets-c_LINKER) $(ninja-targets-cpp_LINKER)) @$< endif -- 2.26.2 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH 1/7] tests/Makefile.include: unbreak non-tcg builds 2020-10-14 13:54 ` [PATCH 1/7] tests/Makefile.include: unbreak non-tcg builds Paolo Bonzini @ 2020-10-14 16:26 ` Daniel P. Berrangé 0 siblings, 0 replies; 29+ messages in thread From: Daniel P. Berrangé @ 2020-10-14 16:26 UTC (permalink / raw) To: Paolo Bonzini; +Cc: peter.maydell, alex.bennee, qemu-devel On Wed, Oct 14, 2020 at 09:54:10AM -0400, Paolo Bonzini wrote: > Remove from check-block the requirement that all TARGET_DIRS are built. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > tests/Makefile.include | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 2/7] make: run shell with pipefail 2020-10-14 13:54 [PATCH 0/7] build: replace ninjatool with ninja Paolo Bonzini 2020-10-14 13:54 ` [PATCH 1/7] tests/Makefile.include: unbreak non-tcg builds Paolo Bonzini @ 2020-10-14 13:54 ` Paolo Bonzini 2020-10-14 16:27 ` Daniel P. Berrangé 2020-10-14 13:54 ` [PATCH 3/7] tests: add missing generated sources to testqapi Paolo Bonzini ` (7 subsequent siblings) 9 siblings, 1 reply; 29+ messages in thread From: Paolo Bonzini @ 2020-10-14 13:54 UTC (permalink / raw) To: qemu-devel; +Cc: berrange, alex.bennee, peter.maydell Without pipefail, it is possible to miss failures if the recipes include pipes. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Makefile b/Makefile index c37e513431..5e51e61c3b 100644 --- a/Makefile +++ b/Makefile @@ -14,6 +14,8 @@ SRC_PATH=. # we have explicit rules for everything MAKEFLAGS += -rR +SHELL = /usr/bin/env bash -o pipefail + # Usage: $(call quiet-command,command and args,"NAME","args to print") # This will run "command and args", and either: # if V=1 just print the whole command and args -- 2.26.2 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH 2/7] make: run shell with pipefail 2020-10-14 13:54 ` [PATCH 2/7] make: run shell with pipefail Paolo Bonzini @ 2020-10-14 16:27 ` Daniel P. Berrangé 0 siblings, 0 replies; 29+ messages in thread From: Daniel P. Berrangé @ 2020-10-14 16:27 UTC (permalink / raw) To: Paolo Bonzini; +Cc: peter.maydell, alex.bennee, qemu-devel On Wed, Oct 14, 2020 at 09:54:11AM -0400, Paolo Bonzini wrote: > Without pipefail, it is possible to miss failures if the recipes > include pipes. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > Makefile | 2 ++ > 1 file changed, 2 insertions(+) Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 3/7] tests: add missing generated sources to testqapi 2020-10-14 13:54 [PATCH 0/7] build: replace ninjatool with ninja Paolo Bonzini 2020-10-14 13:54 ` [PATCH 1/7] tests/Makefile.include: unbreak non-tcg builds Paolo Bonzini 2020-10-14 13:54 ` [PATCH 2/7] make: run shell with pipefail Paolo Bonzini @ 2020-10-14 13:54 ` Paolo Bonzini 2020-10-14 16:27 ` Daniel P. Berrangé 2020-10-14 13:54 ` [PATCH 4/7] configure: move QEMU_INCLUDES to meson Paolo Bonzini ` (6 subsequent siblings) 9 siblings, 1 reply; 29+ messages in thread From: Paolo Bonzini @ 2020-10-14 13:54 UTC (permalink / raw) To: qemu-devel; +Cc: berrange, alex.bennee, peter.maydell Ninja notices them due to a different order in visiting the graph. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- tests/include/meson.build | 8 ++++---- tests/meson.build | 14 ++++++++++++-- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/tests/include/meson.build b/tests/include/meson.build index fea3a6342f..9abba308fa 100644 --- a/tests/include/meson.build +++ b/tests/include/meson.build @@ -10,7 +10,7 @@ test_qapi_outputs_extra = [ 'test-qapi-visit-sub-module.h', ] -test_qapi_outputs_extra = custom_target('QAPI test (include)', - output: test_qapi_outputs_extra, - input: test_qapi_files, - command: 'true') +test_qapi_files_extra = custom_target('QAPI test (include)', + output: test_qapi_outputs_extra, + input: test_qapi_files, + command: 'true') diff --git a/tests/meson.build b/tests/meson.build index bf47a38c74..afeb6be689 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -56,8 +56,18 @@ test_qapi_files = custom_target('Test QAPI files', # perhaps change qapi_gen to replace / with _, like Meson itself does? subdir('include') -libtestqapi = static_library('testqapi', sources: [test_qapi_files, genh, test_qapi_outputs_extra]) -testqapi = declare_dependency(link_with: libtestqapi) +test_qapi_sources = [] +test_qapi_headers = [] +i = 0 +foreach o: test_qapi_files.to_list() + test_qapi_files_extra.to_list() + if o.full_path().endswith('.h') + test_qapi_headers += o + endif + test_qapi_sources += o +endforeach + +libtestqapi = static_library('testqapi', sources: [genh, test_qapi_sources]) +testqapi = declare_dependency(link_with: libtestqapi, sources: [genh, test_qapi_headers]) testblock = declare_dependency(dependencies: [block], sources: 'iothread.c') -- 2.26.2 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH 3/7] tests: add missing generated sources to testqapi 2020-10-14 13:54 ` [PATCH 3/7] tests: add missing generated sources to testqapi Paolo Bonzini @ 2020-10-14 16:27 ` Daniel P. Berrangé 0 siblings, 0 replies; 29+ messages in thread From: Daniel P. Berrangé @ 2020-10-14 16:27 UTC (permalink / raw) To: Paolo Bonzini; +Cc: peter.maydell, alex.bennee, qemu-devel On Wed, Oct 14, 2020 at 09:54:12AM -0400, Paolo Bonzini wrote: > Ninja notices them due to a different order in visiting the graph. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > tests/include/meson.build | 8 ++++---- > tests/meson.build | 14 ++++++++++++-- > 2 files changed, 16 insertions(+), 6 deletions(-) Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 4/7] configure: move QEMU_INCLUDES to meson 2020-10-14 13:54 [PATCH 0/7] build: replace ninjatool with ninja Paolo Bonzini ` (2 preceding siblings ...) 2020-10-14 13:54 ` [PATCH 3/7] tests: add missing generated sources to testqapi Paolo Bonzini @ 2020-10-14 13:54 ` Paolo Bonzini 2020-10-14 16:28 ` Daniel P. Berrangé 2020-10-14 13:54 ` [PATCH 5/7] dockerfiles: enable Centos 8 PowerTools Paolo Bonzini ` (5 subsequent siblings) 9 siblings, 1 reply; 29+ messages in thread From: Paolo Bonzini @ 2020-10-14 13:54 UTC (permalink / raw) To: qemu-devel; +Cc: berrange, alex.bennee, peter.maydell Confusingly, QEMU_INCLUDES is not used by configure tests. Moving it to meson.build ensures that Windows paths are specified instead of the msys paths like /c/Users/... Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- configure | 20 -------------------- meson.build | 30 ++++++++++++++++++++++++++++-- 2 files changed, 28 insertions(+), 22 deletions(-) diff --git a/configure b/configure index b553288c5e..804b7115ab 100755 --- a/configure +++ b/configure @@ -536,8 +536,6 @@ QEMU_CFLAGS="-fno-strict-aliasing -fno-common -fwrapv $QEMU_CFLAGS" QEMU_CFLAGS="-Wundef -Wwrite-strings -Wmissing-prototypes $QEMU_CFLAGS" QEMU_CFLAGS="-Wstrict-prototypes -Wredundant-decls $QEMU_CFLAGS" QEMU_CFLAGS="-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE $QEMU_CFLAGS" -QEMU_INCLUDES="-iquote . -iquote ${source_path} -iquote ${source_path}/accel/tcg -iquote ${source_path}/include" -QEMU_INCLUDES="$QEMU_INCLUDES -iquote ${source_path}/disas/libvixl" # Flags that are needed during configure but later taken care of by Meson CONFIGURE_CFLAGS="-std=gnu99 -Wall" @@ -795,7 +793,6 @@ Linux) audio_possible_drivers="oss alsa sdl pa" linux="yes" linux_user="yes" - QEMU_INCLUDES="-isystem ${source_path}/linux-headers -Ilinux-headers $QEMU_INCLUDES" ;; esac @@ -6918,22 +6915,6 @@ if test "$secret_keyring" = "yes" ; then echo "CONFIG_SECRET_KEYRING=y" >> $config_host_mak fi -if test "$tcg_interpreter" = "yes"; then - QEMU_INCLUDES="-iquote ${source_path}/tcg/tci $QEMU_INCLUDES" -elif test "$ARCH" = "sparc64" ; then - QEMU_INCLUDES="-iquote ${source_path}/tcg/sparc $QEMU_INCLUDES" -elif test "$ARCH" = "s390x" ; then - QEMU_INCLUDES="-iquote ${source_path}/tcg/s390 $QEMU_INCLUDES" -elif test "$ARCH" = "x86_64" || test "$ARCH" = "x32" ; then - QEMU_INCLUDES="-iquote ${source_path}/tcg/i386 $QEMU_INCLUDES" -elif test "$ARCH" = "ppc64" ; then - QEMU_INCLUDES="-iquote ${source_path}/tcg/ppc $QEMU_INCLUDES" -elif test "$ARCH" = "riscv32" || test "$ARCH" = "riscv64" ; then - QEMU_INCLUDES="-I${source_path}/tcg/riscv $QEMU_INCLUDES" -else - QEMU_INCLUDES="-iquote ${source_path}/tcg/${ARCH} $QEMU_INCLUDES" -fi - echo "ROMS=$roms" >> $config_host_mak echo "MAKE=$make" >> $config_host_mak echo "PYTHON=$python" >> $config_host_mak @@ -6960,7 +6941,6 @@ echo "WINDRES=$windres" >> $config_host_mak echo "CFLAGS_NOPIE=$CFLAGS_NOPIE" >> $config_host_mak echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak echo "QEMU_CXXFLAGS=$QEMU_CXXFLAGS" >> $config_host_mak -echo "QEMU_INCLUDES=$QEMU_INCLUDES" >> $config_host_mak echo "GLIB_CFLAGS=$glib_cflags" >> $config_host_mak echo "GLIB_LIBS=$glib_libs" >> $config_host_mak echo "QEMU_LDFLAGS=$QEMU_LDFLAGS" >> $config_host_mak diff --git a/meson.build b/meson.build index 652c37cceb..a1146522e2 100644 --- a/meson.build +++ b/meson.build @@ -93,9 +93,35 @@ add_project_arguments(config_host['QEMU_CXXFLAGS'].split(), native: false, language: 'cpp') add_project_link_arguments(config_host['QEMU_LDFLAGS'].split(), native: false, language: ['c', 'cpp', 'objc']) -add_project_arguments(config_host['QEMU_INCLUDES'].split(), - language: ['c', 'cpp', 'objc']) +if targetos == 'linux' + add_project_arguments('-isystem', meson.current_source_dir() / 'linux-headers', + '-isystem', 'linux-headers', + language: ['c', 'cpp']) +endif + +if 'CONFIG_TCG_INTERPRETER' in config_host + tcg_arch = 'tci' +elif config_host['ARCH'] == 'sparc64' + tcg_arch = 'sparc' +elif config_host['ARCH'] == 's390x' + tcg_arch = 's390' +elif config_host['ARCH'] in ['x86_64', 'x32'] + tcg_arch = 'i386' +elif config_host['ARCH'] == 'ppc64' + tcg_arch = 'ppc' +elif config_host['ARCH'] in ['riscv32', 'riscv64'] + tcg_arch = 'riscv' +else + tcg_arch = config_host['ARCH'] +endif +add_project_arguments('-iquote', meson.current_source_dir() / 'tcg' / tcg_arch, + '-iquote', '.', + '-iquote', meson.current_source_dir(), + '-iquote', meson.current_source_dir() / 'accel/tcg', + '-iquote', meson.current_source_dir() / 'include', + '-iquote', meson.current_source_dir() / 'disas/libvixl', + language: ['c', 'cpp', 'objc']) link_language = meson.get_external_property('link_language', 'cpp') if link_language == 'cpp' -- 2.26.2 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH 4/7] configure: move QEMU_INCLUDES to meson 2020-10-14 13:54 ` [PATCH 4/7] configure: move QEMU_INCLUDES to meson Paolo Bonzini @ 2020-10-14 16:28 ` Daniel P. Berrangé 0 siblings, 0 replies; 29+ messages in thread From: Daniel P. Berrangé @ 2020-10-14 16:28 UTC (permalink / raw) To: Paolo Bonzini; +Cc: peter.maydell, alex.bennee, qemu-devel On Wed, Oct 14, 2020 at 09:54:13AM -0400, Paolo Bonzini wrote: > Confusingly, QEMU_INCLUDES is not used by configure tests. Moving > it to meson.build ensures that Windows paths are specified instead of > the msys paths like /c/Users/... > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > configure | 20 -------------------- > meson.build | 30 ++++++++++++++++++++++++++++-- > 2 files changed, 28 insertions(+), 22 deletions(-) Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 5/7] dockerfiles: enable Centos 8 PowerTools 2020-10-14 13:54 [PATCH 0/7] build: replace ninjatool with ninja Paolo Bonzini ` (3 preceding siblings ...) 2020-10-14 13:54 ` [PATCH 4/7] configure: move QEMU_INCLUDES to meson Paolo Bonzini @ 2020-10-14 13:54 ` Paolo Bonzini 2020-10-14 16:33 ` Daniel P. Berrangé 2020-10-14 20:49 ` Cleber Rosa 2020-10-14 13:54 ` [PATCH 6/7] add ninja to dockerfiles, CI configurations and test VMs Paolo Bonzini ` (4 subsequent siblings) 9 siblings, 2 replies; 29+ messages in thread From: Paolo Bonzini @ 2020-10-14 13:54 UTC (permalink / raw) To: qemu-devel; +Cc: berrange, alex.bennee, peter.maydell ninja is included in the CentOS PowerTools repository. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- tests/docker/dockerfiles/centos8.docker | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker index 0fc2697491..e31d366fc1 100644 --- a/tests/docker/dockerfiles/centos8.docker +++ b/tests/docker/dockerfiles/centos8.docker @@ -1,6 +1,9 @@ FROM centos:8.1.1911 +RUN dnf -y install dnf-plugins-core +RUN dnf config-manager --set-enabled PowerTools RUN dnf -y update + ENV PACKAGES \ SDL-devel \ bzip2 \ -- 2.26.2 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH 5/7] dockerfiles: enable Centos 8 PowerTools 2020-10-14 13:54 ` [PATCH 5/7] dockerfiles: enable Centos 8 PowerTools Paolo Bonzini @ 2020-10-14 16:33 ` Daniel P. Berrangé 2020-10-14 20:49 ` Cleber Rosa 1 sibling, 0 replies; 29+ messages in thread From: Daniel P. Berrangé @ 2020-10-14 16:33 UTC (permalink / raw) To: Paolo Bonzini; +Cc: peter.maydell, alex.bennee, qemu-devel On Wed, Oct 14, 2020 at 09:54:14AM -0400, Paolo Bonzini wrote: > ninja is included in the CentOS PowerTools repository. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > tests/docker/dockerfiles/centos8.docker | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker > index 0fc2697491..e31d366fc1 100644 > --- a/tests/docker/dockerfiles/centos8.docker > +++ b/tests/docker/dockerfiles/centos8.docker > @@ -1,6 +1,9 @@ > FROM centos:8.1.1911 > > +RUN dnf -y install dnf-plugins-core > +RUN dnf config-manager --set-enabled PowerTools > RUN dnf -y update This results in multiple image layers which is undesirable. Instead do RUN dnf -y install dnf-plugins-core && \ dnf config-manager --set-enabled PowerTools && \ dnf -y update > + > ENV PACKAGES \ > SDL-devel \ > bzip2 \ Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 5/7] dockerfiles: enable Centos 8 PowerTools 2020-10-14 13:54 ` [PATCH 5/7] dockerfiles: enable Centos 8 PowerTools Paolo Bonzini 2020-10-14 16:33 ` Daniel P. Berrangé @ 2020-10-14 20:49 ` Cleber Rosa 2020-10-15 6:36 ` Paolo Bonzini 1 sibling, 1 reply; 29+ messages in thread From: Cleber Rosa @ 2020-10-14 20:49 UTC (permalink / raw) To: Paolo Bonzini; +Cc: alex.bennee, berrange, qemu-devel, peter.maydell [-- Attachment #1: Type: text/plain, Size: 1026 bytes --] On Wed, Oct 14, 2020 at 09:54:14AM -0400, Paolo Bonzini wrote: > ninja is included in the CentOS PowerTools repository. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > tests/docker/dockerfiles/centos8.docker | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker > index 0fc2697491..e31d366fc1 100644 > --- a/tests/docker/dockerfiles/centos8.docker > +++ b/tests/docker/dockerfiles/centos8.docker > @@ -1,6 +1,9 @@ > FROM centos:8.1.1911 > > +RUN dnf -y install dnf-plugins-core > +RUN dnf config-manager --set-enabled PowerTools Since config-manager won't be used besides here, I think it's safe and cheaper to simply do: RUN sed -i /etc/yum.repos.d/CentOS-PowerTools.repo -e 's/^enabled=0/enabled=1/' It's also a single command, reducing the number of layers. - Cleber. > RUN dnf -y update > + > ENV PACKAGES \ > SDL-devel \ > bzip2 \ > -- > 2.26.2 > > > [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 5/7] dockerfiles: enable Centos 8 PowerTools 2020-10-14 20:49 ` Cleber Rosa @ 2020-10-15 6:36 ` Paolo Bonzini 0 siblings, 0 replies; 29+ messages in thread From: Paolo Bonzini @ 2020-10-15 6:36 UTC (permalink / raw) To: Cleber Rosa; +Cc: alex.bennee, berrange, qemu-devel, peter.maydell On 14/10/20 22:49, Cleber Rosa wrote: > On Wed, Oct 14, 2020 at 09:54:14AM -0400, Paolo Bonzini wrote: >> ninja is included in the CentOS PowerTools repository. >> >> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> >> --- >> tests/docker/dockerfiles/centos8.docker | 3 +++ >> 1 file changed, 3 insertions(+) >> >> diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker >> index 0fc2697491..e31d366fc1 100644 >> --- a/tests/docker/dockerfiles/centos8.docker >> +++ b/tests/docker/dockerfiles/centos8.docker >> @@ -1,6 +1,9 @@ >> FROM centos:8.1.1911 >> >> +RUN dnf -y install dnf-plugins-core >> +RUN dnf config-manager --set-enabled PowerTools > > Since config-manager won't be used besides here, I think it's safe and > cheaper to simply do: > > RUN sed -i /etc/yum.repos.d/CentOS-PowerTools.repo -e 's/^enabled=0/enabled=1/' > > It's also a single command, reducing the number of layers. Yeah, I'm lazy. :) I'll use Dan's suggestion. Paolo ^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 6/7] add ninja to dockerfiles, CI configurations and test VMs 2020-10-14 13:54 [PATCH 0/7] build: replace ninjatool with ninja Paolo Bonzini ` (4 preceding siblings ...) 2020-10-14 13:54 ` [PATCH 5/7] dockerfiles: enable Centos 8 PowerTools Paolo Bonzini @ 2020-10-14 13:54 ` Paolo Bonzini 2020-10-14 16:36 ` Daniel P. Berrangé 2020-10-15 6:50 ` Alex Bennée 2020-10-14 13:54 ` [PATCH 7/7] build: replace ninjatool with ninja Paolo Bonzini ` (3 subsequent siblings) 9 siblings, 2 replies; 29+ messages in thread From: Paolo Bonzini @ 2020-10-14 13:54 UTC (permalink / raw) To: qemu-devel; +Cc: berrange, alex.bennee, peter.maydell Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- .cirrus.yml | 6 +++--- .travis.yml | 13 +++++++++++++ tests/docker/dockerfiles/centos7.docker | 1 + tests/docker/dockerfiles/centos8.docker | 1 + tests/docker/dockerfiles/debian10.docker | 1 + tests/docker/dockerfiles/fedora.docker | 1 + tests/docker/dockerfiles/travis.docker | 2 +- tests/docker/dockerfiles/ubuntu.docker | 1 + tests/docker/dockerfiles/ubuntu1804.docker | 1 + tests/docker/dockerfiles/ubuntu2004.docker | 1 + tests/vm/centos | 2 +- tests/vm/centos.aarch64 | 2 +- tests/vm/fedora | 2 +- tests/vm/freebsd | 1 + tests/vm/netbsd | 1 + tests/vm/openbsd | 1 + tests/vm/ubuntu.aarch64 | 2 +- tests/vm/ubuntu.i386 | 2 +- 18 files changed, 32 insertions(+), 9 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index 99d118239c..87f7b047d7 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -9,7 +9,7 @@ freebsd_12_task: install_script: - ASSUME_ALWAYS_YES=yes pkg bootstrap -f ; - pkg install -y bash curl cyrus-sasl git glib gmake gnutls gsed - nettle perl5 pixman pkgconf png usbredir + nettle perl5 pixman pkgconf png usbredir ninja script: - mkdir build - cd build @@ -21,7 +21,7 @@ macos_task: osx_instance: image: catalina-base install_script: - - brew install pkg-config python gnu-sed glib pixman make sdl2 bash + - brew install pkg-config python gnu-sed glib pixman make sdl2 bash ninja script: - mkdir build - cd build @@ -36,7 +36,7 @@ macos_xcode_task: # this is an alias for the latest Xcode image: catalina-xcode install_script: - - brew install pkg-config gnu-sed glib pixman make sdl2 bash + - brew install pkg-config gnu-sed glib pixman make sdl2 bash ninja script: - mkdir build - cd build diff --git a/.travis.yml b/.travis.yml index 1054ec5d29..d7bfbb8bfe 100644 --- a/.travis.yml +++ b/.travis.yml @@ -49,6 +49,7 @@ addons: - libvdeplug-dev - libvte-2.91-dev - libzstd-dev + - ninja-build - sparse - uuid-dev - gcovr @@ -177,6 +178,7 @@ jobs: addons: apt: packages: + - ninja-build - python3-sphinx - perl @@ -211,6 +213,10 @@ jobs: # gprof/gcov are GCC features - name: "GCC gprof/gcov" dist: bionic + addons: + apt: + packages: + - ninja-build env: - CONFIG="--enable-gprof --enable-gcov --disable-libssh --target-list=${MAIN_SOFTMMU_TARGETS}" @@ -281,6 +287,7 @@ jobs: - liburcu-dev - libusb-1.0-0-dev - libvte-2.91-dev + - ninja-build - sparse - uuid-dev language: generic @@ -346,6 +353,7 @@ jobs: - libusb-1.0-0-dev - libvdeplug-dev - libvte-2.91-dev + - ninja-build # Tests dependencies - genisoimage env: @@ -379,6 +387,7 @@ jobs: - libusb-1.0-0-dev - libvdeplug-dev - libvte-2.91-dev + - ninja-build # Tests dependencies - genisoimage env: @@ -411,6 +420,7 @@ jobs: - libusb-1.0-0-dev - libvdeplug-dev - libvte-2.91-dev + - ninja-build # Tests dependencies - genisoimage env: @@ -450,6 +460,7 @@ jobs: - libzstd-dev - nettle-dev - xfslibs-dev + - ninja-build # Tests dependencies - genisoimage env: @@ -463,6 +474,7 @@ jobs: apt_packages: - libgcrypt20-dev - libgnutls28-dev + - ninja-build env: - CONFIG="--disable-containers --disable-system" @@ -493,6 +505,7 @@ jobs: - libusb-1.0-0-dev - libvdeplug-dev - libvte-2.91-dev + - ninja-build env: - TEST_CMD="make check-unit" - CONFIG="--disable-containers --disable-tcg --enable-kvm diff --git a/tests/docker/dockerfiles/centos7.docker b/tests/docker/dockerfiles/centos7.docker index 46277773bf..8b273725ee 100644 --- a/tests/docker/dockerfiles/centos7.docker +++ b/tests/docker/dockerfiles/centos7.docker @@ -27,6 +27,7 @@ ENV PACKAGES \ mesa-libEGL-devel \ mesa-libgbm-devel \ nettle-devel \ + ninja-build \ perl-Test-Harness \ pixman-devel \ python3 \ diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker index e31d366fc1..b0eef13987 100644 --- a/tests/docker/dockerfiles/centos8.docker +++ b/tests/docker/dockerfiles/centos8.docker @@ -22,6 +22,7 @@ ENV PACKAGES \ make \ mesa-libEGL-devel \ nettle-devel \ + ninja-build \ perl-Test-Harness \ pixman-devel \ python36 \ diff --git a/tests/docker/dockerfiles/debian10.docker b/tests/docker/dockerfiles/debian10.docker index 1e4188ba22..21cc671d71 100644 --- a/tests/docker/dockerfiles/debian10.docker +++ b/tests/docker/dockerfiles/debian10.docker @@ -26,6 +26,7 @@ RUN apt update && \ gettext \ git \ libncurses5-dev \ + ninja-build \ pkg-config \ psmisc \ python3 \ diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker index 85c975543d..ac79d95418 100644 --- a/tests/docker/dockerfiles/fedora.docker +++ b/tests/docker/dockerfiles/fedora.docker @@ -75,6 +75,7 @@ ENV PACKAGES \ mingw64-SDL2 \ ncurses-devel \ nettle-devel \ + ninja-build \ nss-devel \ numactl-devel \ perl \ diff --git a/tests/docker/dockerfiles/travis.docker b/tests/docker/dockerfiles/travis.docker index 591282561b..cd1435a7e9 100644 --- a/tests/docker/dockerfiles/travis.docker +++ b/tests/docker/dockerfiles/travis.docker @@ -9,7 +9,7 @@ ENV LC_ALL en_US.UTF-8 RUN sed -i "s/# deb-src/deb-src/" /etc/apt/sources.list RUN apt-get update RUN apt-get -y build-dep qemu -RUN apt-get -y install device-tree-compiler python3 python3-yaml dh-autoreconf gdb strace lsof net-tools gcovr +RUN apt-get -y install device-tree-compiler python3 python3-yaml dh-autoreconf gdb strace lsof net-tools gcovr ninja-build # Travis tools require PhantomJS / Neo4j / Maven accessible # in their PATH (QEMU build won't access them). ENV PATH /usr/local/phantomjs/bin:/usr/local/phantomjs:/usr/local/neo4j-3.2.7/bin:/usr/local/maven-3.5.2/bin:/usr/local/cmake-3.9.2/bin:/usr/local/clang-5.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker index b556ed17d2..b5ef7a8198 100644 --- a/tests/docker/dockerfiles/ubuntu.docker +++ b/tests/docker/dockerfiles/ubuntu.docker @@ -60,6 +60,7 @@ ENV PACKAGES \ libxen-dev \ libzstd-dev \ make \ + ninja-build \ python3-yaml \ python3-sphinx \ sparse \ diff --git a/tests/docker/dockerfiles/ubuntu1804.docker b/tests/docker/dockerfiles/ubuntu1804.docker index a6a7617da6..9b0a19ba5e 100644 --- a/tests/docker/dockerfiles/ubuntu1804.docker +++ b/tests/docker/dockerfiles/ubuntu1804.docker @@ -48,6 +48,7 @@ ENV PACKAGES \ make \ python3-yaml \ python3-sphinx \ + ninja-build \ sparse \ xfslibs-dev RUN apt-get update && \ diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker index f4b9556b9e..17b37cda38 100644 --- a/tests/docker/dockerfiles/ubuntu2004.docker +++ b/tests/docker/dockerfiles/ubuntu2004.docker @@ -47,6 +47,7 @@ ENV PACKAGES flex bison \ libxen-dev \ libzstd-dev \ make \ + ninja-build \ python3-numpy \ python3-opencv \ python3-pil \ diff --git a/tests/vm/centos b/tests/vm/centos index 0ad4ecf419..efe3dbbb36 100755 --- a/tests/vm/centos +++ b/tests/vm/centos @@ -42,7 +42,7 @@ class CentosVM(basevm.BaseVM): self.wait_ssh() self.ssh_root_check("touch /etc/cloud/cloud-init.disabled") self.ssh_root_check("yum update -y") - self.ssh_root_check("yum install -y docker make git python3") + self.ssh_root_check("yum install -y docker make ninja-build git python3") self.ssh_root_check("systemctl enable docker") self.ssh_root("poweroff") self.wait() diff --git a/tests/vm/centos.aarch64 b/tests/vm/centos.aarch64 index d5232ecdb8..e687b93e52 100755 --- a/tests/vm/centos.aarch64 +++ b/tests/vm/centos.aarch64 @@ -23,7 +23,7 @@ import aarch64vm DEFAULT_CONFIG = { 'cpu' : "max", 'machine' : "virt,gic-version=max", - 'install_cmds' : "yum install -y make git python3 gcc gcc-c++ flex bison, "\ + 'install_cmds' : "yum install -y make ninja-build git python3 gcc gcc-c++ flex bison, "\ "yum install -y glib2-devel pixman-devel zlib-devel, "\ "yum install -y perl-Test-Harness, "\ "alternatives --set python /usr/bin/python3, "\ diff --git a/tests/vm/fedora b/tests/vm/fedora index b2b478fdbc..b977efe4a2 100755 --- a/tests/vm/fedora +++ b/tests/vm/fedora @@ -32,7 +32,7 @@ class FedoraVM(basevm.BaseVM): pkgs = [ # tools 'git-core', - 'gcc', 'binutils', 'make', + 'gcc', 'binutils', 'make', 'ninja-build', # perl 'perl-Test-Harness', diff --git a/tests/vm/freebsd b/tests/vm/freebsd index 5f866e09c4..04ee793381 100755 --- a/tests/vm/freebsd +++ b/tests/vm/freebsd @@ -34,6 +34,7 @@ class FreeBSDVM(basevm.BaseVM): "bzip2", "python37", "py37-setuptools", + "ninja", # gnu tools "bash", diff --git a/tests/vm/netbsd b/tests/vm/netbsd index ffb65a89be..a9da255c5a 100755 --- a/tests/vm/netbsd +++ b/tests/vm/netbsd @@ -32,6 +32,7 @@ class NetBSDVM(basevm.BaseVM): "xz", "python37", "py37-setuptools", + "ninja", # gnu tools "bash", diff --git a/tests/vm/openbsd b/tests/vm/openbsd index 8356646f21..ad882a76a2 100755 --- a/tests/vm/openbsd +++ b/tests/vm/openbsd @@ -31,6 +31,7 @@ class OpenBSDVM(basevm.BaseVM): "pkgconf", "bzip2", "xz", "py3-setuptools", + "ninja", # gnu tools "bash", diff --git a/tests/vm/ubuntu.aarch64 b/tests/vm/ubuntu.aarch64 index 21d454c27f..b291945a7e 100755 --- a/tests/vm/ubuntu.aarch64 +++ b/tests/vm/ubuntu.aarch64 @@ -22,7 +22,7 @@ DEFAULT_CONFIG = { 'machine' : "virt,gic-version=3", 'install_cmds' : "apt-get update,"\ "apt-get build-dep -y --arch-only qemu,"\ - "apt-get install -y libfdt-dev pkg-config language-pack-en", + "apt-get install -y libfdt-dev pkg-config language-pack-en ninja-build", # We increase beyond the default time since during boot # it can take some time (many seconds) to log into the VM # especially using softmmu. diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386 index 5ce72610a6..47681b6f87 100755 --- a/tests/vm/ubuntu.i386 +++ b/tests/vm/ubuntu.i386 @@ -18,7 +18,7 @@ import ubuntuvm DEFAULT_CONFIG = { 'install_cmds' : "apt-get update,"\ "apt-get build-dep -y qemu,"\ - "apt-get install -y libfdt-dev language-pack-en", + "apt-get install -y libfdt-dev language-pack-en ninja-build", } class UbuntuX86VM(ubuntuvm.UbuntuVM): -- 2.26.2 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH 6/7] add ninja to dockerfiles, CI configurations and test VMs 2020-10-14 13:54 ` [PATCH 6/7] add ninja to dockerfiles, CI configurations and test VMs Paolo Bonzini @ 2020-10-14 16:36 ` Daniel P. Berrangé 2020-10-15 6:50 ` Alex Bennée 1 sibling, 0 replies; 29+ messages in thread From: Daniel P. Berrangé @ 2020-10-14 16:36 UTC (permalink / raw) To: Paolo Bonzini; +Cc: peter.maydell, alex.bennee, qemu-devel On Wed, Oct 14, 2020 at 09:54:15AM -0400, Paolo Bonzini wrote: > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > .cirrus.yml | 6 +++--- > .travis.yml | 13 +++++++++++++ > tests/docker/dockerfiles/centos7.docker | 1 + > tests/docker/dockerfiles/centos8.docker | 1 + > tests/docker/dockerfiles/debian10.docker | 1 + > tests/docker/dockerfiles/fedora.docker | 1 + > tests/docker/dockerfiles/travis.docker | 2 +- > tests/docker/dockerfiles/ubuntu.docker | 1 + > tests/docker/dockerfiles/ubuntu1804.docker | 1 + > tests/docker/dockerfiles/ubuntu2004.docker | 1 + > tests/vm/centos | 2 +- > tests/vm/centos.aarch64 | 2 +- > tests/vm/fedora | 2 +- > tests/vm/freebsd | 1 + > tests/vm/netbsd | 1 + > tests/vm/openbsd | 1 + > tests/vm/ubuntu.aarch64 | 2 +- > tests/vm/ubuntu.i386 | 2 +- > 18 files changed, 32 insertions(+), 9 deletions(-) Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 6/7] add ninja to dockerfiles, CI configurations and test VMs 2020-10-14 13:54 ` [PATCH 6/7] add ninja to dockerfiles, CI configurations and test VMs Paolo Bonzini 2020-10-14 16:36 ` Daniel P. Berrangé @ 2020-10-15 6:50 ` Alex Bennée 1 sibling, 0 replies; 29+ messages in thread From: Alex Bennée @ 2020-10-15 6:50 UTC (permalink / raw) To: Paolo Bonzini; +Cc: peter.maydell, berrange, qemu-devel Paolo Bonzini <pbonzini@redhat.com> writes: > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Acked-by: Alex Bennée <alex.bennee@linaro.org> > --- > .cirrus.yml | 6 +++--- > .travis.yml | 13 +++++++++++++ > tests/docker/dockerfiles/centos7.docker | 1 + > tests/docker/dockerfiles/centos8.docker | 1 + > tests/docker/dockerfiles/debian10.docker | 1 + > tests/docker/dockerfiles/fedora.docker | 1 + > tests/docker/dockerfiles/travis.docker | 2 +- > tests/docker/dockerfiles/ubuntu.docker | 1 + > tests/docker/dockerfiles/ubuntu1804.docker | 1 + > tests/docker/dockerfiles/ubuntu2004.docker | 1 + > tests/vm/centos | 2 +- > tests/vm/centos.aarch64 | 2 +- > tests/vm/fedora | 2 +- > tests/vm/freebsd | 1 + > tests/vm/netbsd | 1 + > tests/vm/openbsd | 1 + > tests/vm/ubuntu.aarch64 | 2 +- > tests/vm/ubuntu.i386 | 2 +- > 18 files changed, 32 insertions(+), 9 deletions(-) > > diff --git a/.cirrus.yml b/.cirrus.yml > index 99d118239c..87f7b047d7 100644 > --- a/.cirrus.yml > +++ b/.cirrus.yml > @@ -9,7 +9,7 @@ freebsd_12_task: > install_script: > - ASSUME_ALWAYS_YES=yes pkg bootstrap -f ; > - pkg install -y bash curl cyrus-sasl git glib gmake gnutls gsed > - nettle perl5 pixman pkgconf png usbredir > + nettle perl5 pixman pkgconf png usbredir ninja > script: > - mkdir build > - cd build > @@ -21,7 +21,7 @@ macos_task: > osx_instance: > image: catalina-base > install_script: > - - brew install pkg-config python gnu-sed glib pixman make sdl2 bash > + - brew install pkg-config python gnu-sed glib pixman make sdl2 bash ninja > script: > - mkdir build > - cd build > @@ -36,7 +36,7 @@ macos_xcode_task: > # this is an alias for the latest Xcode > image: catalina-xcode > install_script: > - - brew install pkg-config gnu-sed glib pixman make sdl2 bash > + - brew install pkg-config gnu-sed glib pixman make sdl2 bash ninja > script: > - mkdir build > - cd build > diff --git a/.travis.yml b/.travis.yml > index 1054ec5d29..d7bfbb8bfe 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -49,6 +49,7 @@ addons: > - libvdeplug-dev > - libvte-2.91-dev > - libzstd-dev > + - ninja-build > - sparse > - uuid-dev > - gcovr > @@ -177,6 +178,7 @@ jobs: > addons: > apt: > packages: > + - ninja-build > - python3-sphinx > - perl > > @@ -211,6 +213,10 @@ jobs: > # gprof/gcov are GCC features > - name: "GCC gprof/gcov" > dist: bionic > + addons: > + apt: > + packages: > + - ninja-build > env: > - CONFIG="--enable-gprof --enable-gcov --disable-libssh > --target-list=${MAIN_SOFTMMU_TARGETS}" > @@ -281,6 +287,7 @@ jobs: > - liburcu-dev > - libusb-1.0-0-dev > - libvte-2.91-dev > + - ninja-build > - sparse > - uuid-dev > language: generic > @@ -346,6 +353,7 @@ jobs: > - libusb-1.0-0-dev > - libvdeplug-dev > - libvte-2.91-dev > + - ninja-build > # Tests dependencies > - genisoimage > env: > @@ -379,6 +387,7 @@ jobs: > - libusb-1.0-0-dev > - libvdeplug-dev > - libvte-2.91-dev > + - ninja-build > # Tests dependencies > - genisoimage > env: > @@ -411,6 +420,7 @@ jobs: > - libusb-1.0-0-dev > - libvdeplug-dev > - libvte-2.91-dev > + - ninja-build > # Tests dependencies > - genisoimage > env: > @@ -450,6 +460,7 @@ jobs: > - libzstd-dev > - nettle-dev > - xfslibs-dev > + - ninja-build > # Tests dependencies > - genisoimage > env: > @@ -463,6 +474,7 @@ jobs: > apt_packages: > - libgcrypt20-dev > - libgnutls28-dev > + - ninja-build > env: > - CONFIG="--disable-containers --disable-system" > > @@ -493,6 +505,7 @@ jobs: > - libusb-1.0-0-dev > - libvdeplug-dev > - libvte-2.91-dev > + - ninja-build > env: > - TEST_CMD="make check-unit" > - CONFIG="--disable-containers --disable-tcg --enable-kvm > diff --git a/tests/docker/dockerfiles/centos7.docker b/tests/docker/dockerfiles/centos7.docker > index 46277773bf..8b273725ee 100644 > --- a/tests/docker/dockerfiles/centos7.docker > +++ b/tests/docker/dockerfiles/centos7.docker > @@ -27,6 +27,7 @@ ENV PACKAGES \ > mesa-libEGL-devel \ > mesa-libgbm-devel \ > nettle-devel \ > + ninja-build \ > perl-Test-Harness \ > pixman-devel \ > python3 \ > diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker > index e31d366fc1..b0eef13987 100644 > --- a/tests/docker/dockerfiles/centos8.docker > +++ b/tests/docker/dockerfiles/centos8.docker > @@ -22,6 +22,7 @@ ENV PACKAGES \ > make \ > mesa-libEGL-devel \ > nettle-devel \ > + ninja-build \ > perl-Test-Harness \ > pixman-devel \ > python36 \ > diff --git a/tests/docker/dockerfiles/debian10.docker b/tests/docker/dockerfiles/debian10.docker > index 1e4188ba22..21cc671d71 100644 > --- a/tests/docker/dockerfiles/debian10.docker > +++ b/tests/docker/dockerfiles/debian10.docker > @@ -26,6 +26,7 @@ RUN apt update && \ > gettext \ > git \ > libncurses5-dev \ > + ninja-build \ > pkg-config \ > psmisc \ > python3 \ > diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker > index 85c975543d..ac79d95418 100644 > --- a/tests/docker/dockerfiles/fedora.docker > +++ b/tests/docker/dockerfiles/fedora.docker > @@ -75,6 +75,7 @@ ENV PACKAGES \ > mingw64-SDL2 \ > ncurses-devel \ > nettle-devel \ > + ninja-build \ > nss-devel \ > numactl-devel \ > perl \ > diff --git a/tests/docker/dockerfiles/travis.docker b/tests/docker/dockerfiles/travis.docker > index 591282561b..cd1435a7e9 100644 > --- a/tests/docker/dockerfiles/travis.docker > +++ b/tests/docker/dockerfiles/travis.docker > @@ -9,7 +9,7 @@ ENV LC_ALL en_US.UTF-8 > RUN sed -i "s/# deb-src/deb-src/" /etc/apt/sources.list > RUN apt-get update > RUN apt-get -y build-dep qemu > -RUN apt-get -y install device-tree-compiler python3 python3-yaml dh-autoreconf gdb strace lsof net-tools gcovr > +RUN apt-get -y install device-tree-compiler python3 python3-yaml dh-autoreconf gdb strace lsof net-tools gcovr ninja-build > # Travis tools require PhantomJS / Neo4j / Maven accessible > # in their PATH (QEMU build won't access them). > ENV PATH /usr/local/phantomjs/bin:/usr/local/phantomjs:/usr/local/neo4j-3.2.7/bin:/usr/local/maven-3.5.2/bin:/usr/local/cmake-3.9.2/bin:/usr/local/clang-5.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin > diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker > index b556ed17d2..b5ef7a8198 100644 > --- a/tests/docker/dockerfiles/ubuntu.docker > +++ b/tests/docker/dockerfiles/ubuntu.docker > @@ -60,6 +60,7 @@ ENV PACKAGES \ > libxen-dev \ > libzstd-dev \ > make \ > + ninja-build \ > python3-yaml \ > python3-sphinx \ > sparse \ > diff --git a/tests/docker/dockerfiles/ubuntu1804.docker b/tests/docker/dockerfiles/ubuntu1804.docker > index a6a7617da6..9b0a19ba5e 100644 > --- a/tests/docker/dockerfiles/ubuntu1804.docker > +++ b/tests/docker/dockerfiles/ubuntu1804.docker > @@ -48,6 +48,7 @@ ENV PACKAGES \ > make \ > python3-yaml \ > python3-sphinx \ > + ninja-build \ > sparse \ > xfslibs-dev > RUN apt-get update && \ > diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker > index f4b9556b9e..17b37cda38 100644 > --- a/tests/docker/dockerfiles/ubuntu2004.docker > +++ b/tests/docker/dockerfiles/ubuntu2004.docker > @@ -47,6 +47,7 @@ ENV PACKAGES flex bison \ > libxen-dev \ > libzstd-dev \ > make \ > + ninja-build \ > python3-numpy \ > python3-opencv \ > python3-pil \ > diff --git a/tests/vm/centos b/tests/vm/centos > index 0ad4ecf419..efe3dbbb36 100755 > --- a/tests/vm/centos > +++ b/tests/vm/centos > @@ -42,7 +42,7 @@ class CentosVM(basevm.BaseVM): > self.wait_ssh() > self.ssh_root_check("touch /etc/cloud/cloud-init.disabled") > self.ssh_root_check("yum update -y") > - self.ssh_root_check("yum install -y docker make git python3") > + self.ssh_root_check("yum install -y docker make ninja-build git python3") > self.ssh_root_check("systemctl enable docker") > self.ssh_root("poweroff") > self.wait() > diff --git a/tests/vm/centos.aarch64 b/tests/vm/centos.aarch64 > index d5232ecdb8..e687b93e52 100755 > --- a/tests/vm/centos.aarch64 > +++ b/tests/vm/centos.aarch64 > @@ -23,7 +23,7 @@ import aarch64vm > DEFAULT_CONFIG = { > 'cpu' : "max", > 'machine' : "virt,gic-version=max", > - 'install_cmds' : "yum install -y make git python3 gcc gcc-c++ flex bison, "\ > + 'install_cmds' : "yum install -y make ninja-build git python3 gcc gcc-c++ flex bison, "\ > "yum install -y glib2-devel pixman-devel zlib-devel, "\ > "yum install -y perl-Test-Harness, "\ > "alternatives --set python /usr/bin/python3, "\ > diff --git a/tests/vm/fedora b/tests/vm/fedora > index b2b478fdbc..b977efe4a2 100755 > --- a/tests/vm/fedora > +++ b/tests/vm/fedora > @@ -32,7 +32,7 @@ class FedoraVM(basevm.BaseVM): > pkgs = [ > # tools > 'git-core', > - 'gcc', 'binutils', 'make', > + 'gcc', 'binutils', 'make', 'ninja-build', > > # perl > 'perl-Test-Harness', > diff --git a/tests/vm/freebsd b/tests/vm/freebsd > index 5f866e09c4..04ee793381 100755 > --- a/tests/vm/freebsd > +++ b/tests/vm/freebsd > @@ -34,6 +34,7 @@ class FreeBSDVM(basevm.BaseVM): > "bzip2", > "python37", > "py37-setuptools", > + "ninja", > > # gnu tools > "bash", > diff --git a/tests/vm/netbsd b/tests/vm/netbsd > index ffb65a89be..a9da255c5a 100755 > --- a/tests/vm/netbsd > +++ b/tests/vm/netbsd > @@ -32,6 +32,7 @@ class NetBSDVM(basevm.BaseVM): > "xz", > "python37", > "py37-setuptools", > + "ninja", > > # gnu tools > "bash", > diff --git a/tests/vm/openbsd b/tests/vm/openbsd > index 8356646f21..ad882a76a2 100755 > --- a/tests/vm/openbsd > +++ b/tests/vm/openbsd > @@ -31,6 +31,7 @@ class OpenBSDVM(basevm.BaseVM): > "pkgconf", > "bzip2", "xz", > "py3-setuptools", > + "ninja", > > # gnu tools > "bash", > diff --git a/tests/vm/ubuntu.aarch64 b/tests/vm/ubuntu.aarch64 > index 21d454c27f..b291945a7e 100755 > --- a/tests/vm/ubuntu.aarch64 > +++ b/tests/vm/ubuntu.aarch64 > @@ -22,7 +22,7 @@ DEFAULT_CONFIG = { > 'machine' : "virt,gic-version=3", > 'install_cmds' : "apt-get update,"\ > "apt-get build-dep -y --arch-only qemu,"\ > - "apt-get install -y libfdt-dev pkg-config language-pack-en", > + "apt-get install -y libfdt-dev pkg-config language-pack-en ninja-build", > # We increase beyond the default time since during boot > # it can take some time (many seconds) to log into the VM > # especially using softmmu. > diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386 > index 5ce72610a6..47681b6f87 100755 > --- a/tests/vm/ubuntu.i386 > +++ b/tests/vm/ubuntu.i386 > @@ -18,7 +18,7 @@ import ubuntuvm > DEFAULT_CONFIG = { > 'install_cmds' : "apt-get update,"\ > "apt-get build-dep -y qemu,"\ > - "apt-get install -y libfdt-dev language-pack-en", > + "apt-get install -y libfdt-dev language-pack-en ninja-build", > } > > class UbuntuX86VM(ubuntuvm.UbuntuVM): -- Alex Bennée ^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 7/7] build: replace ninjatool with ninja 2020-10-14 13:54 [PATCH 0/7] build: replace ninjatool with ninja Paolo Bonzini ` (5 preceding siblings ...) 2020-10-14 13:54 ` [PATCH 6/7] add ninja to dockerfiles, CI configurations and test VMs Paolo Bonzini @ 2020-10-14 13:54 ` Paolo Bonzini 2020-10-14 16:38 ` Daniel P. Berrangé 2020-10-14 13:57 ` [PATCH 0/7] " Peter Maydell ` (2 subsequent siblings) 9 siblings, 1 reply; 29+ messages in thread From: Paolo Bonzini @ 2020-10-14 13:54 UTC (permalink / raw) To: qemu-devel; +Cc: berrange, alex.bennee, peter.maydell Now that the build is done entirely by Meson, there is no need to keep the Makefile conversion. Instead, we can ask Ninja about the targets it exposes and forward them. The main advantages are, from smallest to largest: - reducing the possible namespace pollution within the Makefile - removal of a relatively large Python program - faster build because parsing Makefile.ninja is slower than parsing build.ninja; and faster build after Meson runs because we do not have to generate Makefile.ninja. - tracking of command lines, which provides more accurate rebuilds In addition the change removes the requirement for GNU make 3.82, which was annoying on Mac, and avoids bugs on Windows due to ninjatool not knowing how to convert Windows escapes to POSIX escapes. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- Makefile | 40 +- configure | 9 +- docs/devel/build-system.rst | 6 +- meson.build | 4 - scripts/mtest2make.py | 4 +- scripts/ninjatool.py | 1008 ----------------------------------- tests/Makefile.include | 2 +- 7 files changed, 40 insertions(+), 1033 deletions(-) delete mode 100755 scripts/ninjatool.py diff --git a/Makefile b/Makefile index 5e51e61c3b..2e01d29d36 100644 --- a/Makefile +++ b/Makefile @@ -64,17 +64,34 @@ git-submodule-update: endif endif -export NINJA=./ninjatool +.PHONY: ninja-clean ninja-distclean +ninja-clean ninja-distclean:: +ifneq ($(NINJA),) +export NINJA +NINJAFLAGS = $(if $V,-v,) \ + $(filter-out -j, $(lastword -j1 $(filter -l% -j%, $(MAKEFLAGS)))) \ + $(subst -k, -k0, $(filter -n -k,$(MAKEFLAGS))) -# Running meson regenerates both build.ninja and ninjatool, and that is -# enough to prime the rest of the build. -ninjatool: build.ninja - -Makefile.ninja: build.ninja ninjatool - ./ninjatool -t ninja2make --omit clean dist uninstall cscope TAGS ctags < $< > $@ -include Makefile.ninja +Makefile.ninja: build.ninja + { echo 'ninja-targets = \'; $(NINJA) -t targets all | sed 's/:.*//; $$!s/$$/ \\/'; } > $@ + +ninja-targets := $(filter-out build.ninja dist clean uninstall, $(ninja-targets)) +ninja-cmd-goals = $(or $(MAKECMDGOALS), all) +.PHONY: $(ninja-targets) run-ninja +$(ninja-targets): run-ninja -${ninja-targets-c_COMPILER} ${ninja-targets-cpp_COMPILER}: .var.command += -MP +# Use "| cat" to give Ninja a more "make-y" output. Use "+" to bypass the +# --output-sync line. +run-ninja: + +$(NINJA) $(NINJAFLAGS) $(sort $(filter $(ninja-targets), $(ninja-cmd-goals))) | cat + +ninja-clean:: + +$(quiet-@)$(NINJA) $(NINJAFLAGS) -t clean + +ninja-distclean:: + +$(quiet-@)$(NINJA) $(NINJAFLAGS) -t clean -g +endif # If MESON is empty, the rule will be re-evaluated after Makefiles are # reread (and MESON won't be empty anymore). @@ -159,8 +176,8 @@ recurse-clean: $(addsuffix /clean, $(ROM_DIRS)) ###################################################################### -clean: recurse-clean ninja-clean clean-ctlist - if test -f ninjatool; then ./ninjatool $(if $(V),-v,) -t clean; fi +clean: recurse-clean ninja-clean + -test -f build.ninja && @$(NINJA) $(NINJAFLAGS) clean-ctlist || : # avoid old build problems by removing potentially incorrect old files rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h find . \( -name '*.so' -o -name '*.dll' -o -name '*.[oda]' \) -type f \ @@ -178,7 +195,6 @@ qemu-%.tar.bz2: $(SRC_PATH)/scripts/make-release "$(SRC_PATH)" "$(patsubst qemu-%.tar.bz2,%,$@)" distclean: clean ninja-distclean - -test -f ninjatool && ./ninjatool $(if $(V),-v,) -t clean -g rm -f config-host.mak config-host.h* rm -f tests/tcg/config-*.mak rm -f config-all-disas.mak config.status @@ -187,7 +203,7 @@ distclean: clean ninja-distclean rm -f qemu-plugins-ld.symbols qemu-plugins-ld64.symbols rm -f *-config-target.h *-config-devices.mak *-config-devices.h rm -rf meson-private meson-logs meson-info compile_commands.json - rm -f Makefile.ninja ninjatool ninjatool.stamp Makefile.mtest + rm -f Makefile.ninja Makefile.mtest rm -f config.log rm -f linux-headers/asm rm -Rf .sdk diff --git a/configure b/configure index 804b7115ab..5f5375c378 100755 --- a/configure +++ b/configure @@ -1904,7 +1904,7 @@ case "$meson" in *) meson=$(command -v "$meson") ;; esac -# Probe for ninja (used for compdb) +# Probe for ninja if test -z "$ninja"; then for c in ninja ninja-build samu; do @@ -1913,6 +1913,9 @@ if test -z "$ninja"; then break fi done + if test -z "$ninja"; then + error_exit "Cannot find Ninja" + fi fi # Check that the C compiler works. Doing this here before testing @@ -6921,6 +6924,7 @@ echo "PYTHON=$python" >> $config_host_mak echo "SPHINX_BUILD=$sphinx_build" >> $config_host_mak echo "GENISOIMAGE=$genisoimage" >> $config_host_mak echo "MESON=$meson" >> $config_host_mak +echo "NINJA=$ninja" >> $config_host_mak echo "CC=$cc" >> $config_host_mak if $iasl -h > /dev/null 2>&1; then echo "CONFIG_IASL=$iasl" >> $config_host_mak @@ -7172,7 +7176,7 @@ fi mv $cross config-meson.cross rm -rf meson-private meson-info meson-logs -NINJA=${ninja:-$PWD/ninjatool} $meson setup \ +NINJA=$ninja $meson setup \ --prefix "$prefix" \ --libdir "$libdir" \ --libexecdir "$libexecdir" \ @@ -7204,7 +7208,6 @@ NINJA=${ninja:-$PWD/ninjatool} $meson setup \ if test "$?" -ne 0 ; then error_exit "meson setup failed" fi -touch ninjatool.stamp fi if test -n "${deprecated_features}"; then diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst index 2ee368fad6..9c92a43b43 100644 --- a/docs/devel/build-system.rst +++ b/docs/devel/build-system.rst @@ -404,10 +404,8 @@ Built by Meson: Built by Makefile: `Makefile.ninja` - A Makefile conversion of the build rules in build.ninja. The conversion - is straightforward and, were it necessary to debug the rules produced - by Meson, it should be enough to look at build.ninja. The conversion - is performed by scripts/ninjatool.py. + A Makefile include that bridges to ninja for the actual build. The + Makefile is just a list of targets included by Meson in build.ninja. `Makefile.mtest` The Makefile definitions that let "make check" run tests defined in diff --git a/meson.build b/meson.build index a1146522e2..f7aafd2308 100644 --- a/meson.build +++ b/meson.build @@ -47,10 +47,6 @@ supported_cpus = ['ppc', 'ppc64', 's390x', 'sparc64', 'riscv32', 'riscv64', 'x86 cpu = host_machine.cpu_family() targetos = host_machine.system() -configure_file(input: files('scripts/ninjatool.py'), - output: 'ninjatool', - configuration: config_host) - if cpu in ['x86', 'x86_64'] kvm_targets = ['i386-softmmu', 'x86_64-softmmu'] elif cpu == 'aarch64' diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py index c3489a4605..c4c4d725f8 100644 --- a/scripts/mtest2make.py +++ b/scripts/mtest2make.py @@ -70,8 +70,9 @@ def process_tests(test, targets, suites): print('.test.driver.%d := %s' % (i, driver)) print('.test.env.%d := $(.test.env) %s' % (i, env)) print('.test.cmd.%d := %s' % (i, cmd)) + print('.test.deps.%d := %s' % (i, ' '.join(deps))) print('.PHONY: run-test-%d' % (i,)) - print('run-test-%d: %s' % (i, ' '.join(deps))) + print('run-test-%d: $(.test.deps.%d)' % (i,i)) print('\t@$(call .test.run,%d,$(.test.output-format))' % (i,)) test_suites = test['suite'] or ['default'] @@ -128,3 +129,4 @@ for name, suite in benchsuites.items(): emit_suite(name, suite, 'bench') print('run-tests: $(patsubst %, run-test-%, $(.tests))') +print('ninja-cmd-goals += $(foreach t, $(.tests), $(.test.deps.$t))') diff --git a/scripts/ninjatool.py b/scripts/ninjatool.py deleted file mode 100755 index 6f0e35c727..0000000000 --- a/scripts/ninjatool.py +++ /dev/null @@ -1,1008 +0,0 @@ -#! /bin/sh - -# Python module for parsing and processing .ninja files. -# -# Author: Paolo Bonzini -# -# Copyright (C) 2019 Red Hat, Inc. - - -# We don't want to put "#! @PYTHON@" as the shebang and -# make the file executable, so instead we make this a -# Python/shell polyglot. The first line below starts a -# multiline string literal for Python, while it is just -# ":" for bash. The closing of the multiline string literal -# is never parsed by bash since it exits before. - -'''': -case "$0" in - /*) me=$0 ;; - *) me=$(command -v "$0") ;; -esac -python="@PYTHON@" -case $python in - @*) python=python3 ;; -esac -exec $python "$me" "$@" -exit 1 -''' - - -from collections import namedtuple, defaultdict -import sys -import os -import re -import json -import argparse -import hashlib -import shutil - - -class InvalidArgumentError(Exception): - pass - -# faster version of os.path.normpath: do nothing unless there is a double -# slash or a "." or ".." component. The filter does not have to be super -# precise, but it has to be fast. os.path.normpath is the hottest function -# for ninja2make without this optimization! -if os.path.sep == '/': - def normpath(path, _slow_re=re.compile('/[./]')): - return os.path.normpath(path) if _slow_re.search(path) or path[0] == '.' else path -else: - normpath = os.path.normpath - - -def sha1_text(text): - return hashlib.sha1(text.encode()).hexdigest() - -# ---- lexer and parser ---- - -PATH_RE = r"[^$\s:|]+|\$[$ :]|\$[a-zA-Z0-9_-]+|\$\{[a-zA-Z0-9_.-]+\}" - -SIMPLE_PATH_RE = re.compile(r"^[^$\s:|]+$") -IDENT_RE = re.compile(r"[a-zA-Z0-9_.-]+$") -STRING_RE = re.compile(r"(" + PATH_RE + r"|[\s:|])(?:\r?\n)?|.") -TOPLEVEL_RE = re.compile(r"([=:#]|\|\|?|^ +|(?:" + PATH_RE + r")+)\s*|.") -VAR_RE=re.compile(r'\$\$|\$\{([^}]*)\}') - -BUILD = 1 -POOL = 2 -RULE = 3 -DEFAULT = 4 -EQUALS = 5 -COLON = 6 -PIPE = 7 -PIPE2 = 8 -IDENT = 9 -INCLUDE = 10 -INDENT = 11 -EOL = 12 - - -class LexerError(Exception): - pass - - -class ParseError(Exception): - pass - - -class NinjaParserEvents(object): - def __init__(self, parser): - self.parser = parser - - def dollar_token(self, word, in_path=False): - return '$$' if word == '$' else word - - def variable_expansion_token(self, varname): - return '${%s}' % varname - - def variable(self, name, arg): - pass - - def begin_file(self): - pass - - def end_file(self): - pass - - def end_scope(self): - pass - - def begin_pool(self, name): - pass - - def begin_rule(self, name): - pass - - def begin_build(self, out, iout, rule, in_, iin, orderdep): - pass - - def default(self, targets): - pass - - -class NinjaParser(object): - - InputFile = namedtuple('InputFile', 'filename iter lineno') - - def __init__(self, filename, input): - self.stack = [] - self.top = None - self.iter = None - self.lineno = None - self.match_keyword = False - self.push(filename, input) - - def file_changed(self): - self.iter = self.top.iter - self.lineno = self.top.lineno - if self.top.filename is not None: - os.chdir(os.path.dirname(self.top.filename) or '.') - - def push(self, filename, input): - if self.top: - self.top.lineno = self.lineno - self.top.iter = self.iter - self.stack.append(self.top) - self.top = self.InputFile(filename=filename or 'stdin', - iter=self._tokens(input), lineno=0) - self.file_changed() - - def pop(self): - if len(self.stack): - self.top = self.stack[-1] - self.stack.pop() - self.file_changed() - else: - self.top = self.iter = None - - def next_line(self, input): - line = next(input).rstrip() - self.lineno += 1 - while len(line) and line[-1] == '$': - line = line[0:-1] + next(input).strip() - self.lineno += 1 - return line - - def print_token(self, tok): - if tok == EOL: - return "end of line" - if tok == BUILD: - return '"build"' - if tok == POOL: - return '"pool"' - if tok == RULE: - return '"rule"' - if tok == DEFAULT: - return '"default"' - if tok == EQUALS: - return '"="' - if tok == COLON: - return '":"' - if tok == PIPE: - return '"|"' - if tok == PIPE2: - return '"||"' - if tok == INCLUDE: - return '"include"' - if tok == IDENT: - return 'identifier' - return '"%s"' % tok - - def error(self, msg): - raise LexerError("%s:%d: %s" % (self.stack[-1].filename, self.lineno, msg)) - - def parse_error(self, msg): - raise ParseError("%s:%d: %s" % (self.stack[-1].filename, self.lineno, msg)) - - def expected(self, expected, tok): - msg = "found %s, expected " % (self.print_token(tok), ) - for i, exp_tok in enumerate(expected): - if i > 0: - msg = msg + (' or ' if i == len(expected) - 1 else ', ') - msg = msg + self.print_token(exp_tok) - self.parse_error(msg) - - def _variable_tokens(self, value): - for m in STRING_RE.finditer(value): - match = m.group(1) - if not match: - self.error("unexpected '%s'" % (m.group(0), )) - yield match - - def _tokens(self, input): - while True: - try: - line = self.next_line(input) - except StopIteration: - return - for m in TOPLEVEL_RE.finditer(line): - match = m.group(1) - if not match: - self.error("unexpected '%s'" % (m.group(0), )) - if match == ':': - yield COLON - continue - if match == '|': - yield PIPE - continue - if match == '||': - yield PIPE2 - continue - if match[0] == ' ': - yield INDENT - continue - if match[0] == '=': - yield EQUALS - value = line[m.start() + 1:].lstrip() - yield from self._variable_tokens(value) - break - if match[0] == '#': - break - - # identifier - if self.match_keyword: - if match == 'build': - yield BUILD - continue - if match == 'pool': - yield POOL - continue - if match == 'rule': - yield RULE - continue - if match == 'default': - yield DEFAULT - continue - if match == 'include': - filename = line[m.start() + 8:].strip() - self.push(filename, open(filename, 'r')) - break - if match == 'subninja': - self.error('subninja is not supported') - yield match - yield EOL - - def parse(self, events): - global_var = True - - def look_for(*expected): - # The last token in the token stream is always EOL. This - # is exploited to avoid catching StopIteration everywhere. - tok = next(self.iter) - if tok not in expected: - self.expected(expected, tok) - return tok - - def look_for_ident(*expected): - tok = next(self.iter) - if isinstance(tok, str): - if not IDENT_RE.match(tok): - self.parse_error('variable expansion not allowed') - elif tok not in expected: - self.expected(expected + (IDENT,), tok) - return tok - - def parse_assignment_rhs(gen, expected, in_path): - tokens = [] - for tok in gen: - if not isinstance(tok, str): - if tok in expected: - break - self.expected(expected + (IDENT,), tok) - if tok[0] != '$': - tokens.append(tok) - elif tok == '$ ' or tok == '$$' or tok == '$:': - tokens.append(events.dollar_token(tok[1], in_path)) - else: - var = tok[2:-1] if tok[1] == '{' else tok[1:] - tokens.append(events.variable_expansion_token(var)) - else: - # gen must have raised StopIteration - tok = None - - if tokens: - # Fast path avoiding str.join() - value = tokens[0] if len(tokens) == 1 else ''.join(tokens) - else: - value = None - return value, tok - - def look_for_path(*expected): - # paths in build rules are parsed one space-separated token - # at a time and expanded - token = next(self.iter) - if not isinstance(token, str): - return None, token - # Fast path if there are no dollar and variable expansion - if SIMPLE_PATH_RE.match(token): - return token, None - gen = self._variable_tokens(token) - return parse_assignment_rhs(gen, expected, True) - - def parse_assignment(tok): - name = tok - assert isinstance(name, str) - look_for(EQUALS) - value, tok = parse_assignment_rhs(self.iter, (EOL,), False) - assert tok == EOL - events.variable(name, value) - - def parse_build(): - # parse outputs - out = [] - iout = [] - while True: - value, tok = look_for_path(COLON, PIPE) - if value is None: - break - out.append(value) - if tok == PIPE: - while True: - value, tok = look_for_path(COLON) - if value is None: - break - iout.append(value) - - # parse rule - assert tok == COLON - rule = look_for_ident() - - # parse inputs and dependencies - in_ = [] - iin = [] - orderdep = [] - while True: - value, tok = look_for_path(PIPE, PIPE2, EOL) - if value is None: - break - in_.append(value) - if tok == PIPE: - while True: - value, tok = look_for_path(PIPE2, EOL) - if value is None: - break - iin.append(value) - if tok == PIPE2: - while True: - value, tok = look_for_path(EOL) - if value is None: - break - orderdep.append(value) - assert tok == EOL - events.begin_build(out, iout, rule, in_, iin, orderdep) - nonlocal global_var - global_var = False - - def parse_pool(): - # pool declarations are ignored. Just gobble all the variables - ident = look_for_ident() - look_for(EOL) - events.begin_pool(ident) - nonlocal global_var - global_var = False - - def parse_rule(): - ident = look_for_ident() - look_for(EOL) - events.begin_rule(ident) - nonlocal global_var - global_var = False - - def parse_default(): - idents = [] - while True: - ident = look_for_ident(EOL) - if ident == EOL: - break - idents.append(ident) - events.default(idents) - - def parse_declaration(tok): - if tok == EOL: - return - - nonlocal global_var - if tok == INDENT: - if global_var: - self.parse_error('indented line outside rule or edge') - tok = look_for_ident(EOL) - if tok == EOL: - return - parse_assignment(tok) - return - - if not global_var: - events.end_scope() - global_var = True - if tok == POOL: - parse_pool() - elif tok == BUILD: - parse_build() - elif tok == RULE: - parse_rule() - elif tok == DEFAULT: - parse_default() - elif isinstance(tok, str): - parse_assignment(tok) - else: - self.expected((POOL, BUILD, RULE, INCLUDE, DEFAULT, IDENT), tok) - - events.begin_file() - while self.iter: - try: - self.match_keyword = True - token = next(self.iter) - self.match_keyword = False - parse_declaration(token) - except StopIteration: - self.pop() - events.end_file() - - -# ---- variable handling ---- - -def expand(x, rule_vars=None, build_vars=None, global_vars=None): - if x is None: - return None - changed = True - have_dollar_replacement = False - while changed: - changed = False - matches = list(VAR_RE.finditer(x)) - if not matches: - break - - # Reverse the match so that expanding later matches does not - # invalidate m.start()/m.end() for earlier ones. Do not reduce $$ to $ - # until all variables are dealt with. - for m in reversed(matches): - name = m.group(1) - if not name: - have_dollar_replacement = True - continue - changed = True - if build_vars and name in build_vars: - value = build_vars[name] - elif rule_vars and name in rule_vars: - value = rule_vars[name] - elif name in global_vars: - value = global_vars[name] - else: - value = '' - x = x[:m.start()] + value + x[m.end():] - return x.replace('$$', '$') if have_dollar_replacement else x - - -class Scope(object): - def __init__(self, events): - self.events = events - - def on_left_scope(self): - pass - - def on_variable(self, key, value): - pass - - -class BuildScope(Scope): - def __init__(self, events, out, iout, rule, in_, iin, orderdep, rule_vars): - super().__init__(events) - self.rule = rule - self.out = [events.expand_and_normalize(x) for x in out] - self.in_ = [events.expand_and_normalize(x) for x in in_] - self.iin = [events.expand_and_normalize(x) for x in iin] - self.orderdep = [events.expand_and_normalize(x) for x in orderdep] - self.iout = [events.expand_and_normalize(x) for x in iout] - self.rule_vars = rule_vars - self.build_vars = dict() - self._define_variable('out', ' '.join(self.out)) - self._define_variable('in', ' '.join(self.in_)) - - def expand(self, x): - return self.events.expand(x, self.rule_vars, self.build_vars) - - def on_left_scope(self): - self.events.variable('out', self.build_vars['out']) - self.events.variable('in', self.build_vars['in']) - self.events.end_build(self, self.out, self.iout, self.rule, self.in_, - self.iin, self.orderdep) - - def _define_variable(self, key, value): - # The value has been expanded already, quote it for further - # expansion from rule variables - value = value.replace('$', '$$') - self.build_vars[key] = value - - def on_variable(self, key, value): - # in and out are at the top of the lookup order and cannot - # be overridden. Also, unlike what the manual says, build - # variables only lookup global variables. They never lookup - # rule variables, earlier build variables, or in/out. - if key not in ('in', 'in_newline', 'out'): - self._define_variable(key, self.events.expand(value)) - - -class RuleScope(Scope): - def __init__(self, events, name, vars_dict): - super().__init__(events) - self.name = name - self.vars_dict = vars_dict - self.generator = False - - def on_left_scope(self): - self.events.end_rule(self, self.name) - - def on_variable(self, key, value): - self.vars_dict[key] = value - if key == 'generator': - self.generator = True - - -class NinjaParserEventsWithVars(NinjaParserEvents): - def __init__(self, parser): - super().__init__(parser) - self.rule_vars = defaultdict(lambda: dict()) - self.global_vars = dict() - self.scope = None - - def variable(self, name, value): - if self.scope: - self.scope.on_variable(name, value) - else: - self.global_vars[name] = self.expand(value) - - def begin_build(self, out, iout, rule, in_, iin, orderdep): - if rule != 'phony' and rule not in self.rule_vars: - self.parser.parse_error("undefined rule '%s'" % rule) - - self.scope = BuildScope(self, out, iout, rule, in_, iin, orderdep, self.rule_vars[rule]) - - def begin_pool(self, name): - # pool declarations are ignored. Just gobble all the variables - self.scope = Scope(self) - - def begin_rule(self, name): - if name in self.rule_vars: - self.parser.parse_error("duplicate rule '%s'" % name) - self.scope = RuleScope(self, name, self.rule_vars[name]) - - def end_scope(self): - self.scope.on_left_scope() - self.scope = None - - # utility functions: - - def expand(self, x, rule_vars=None, build_vars=None): - return expand(x, rule_vars, build_vars, self.global_vars) - - def expand_and_normalize(self, x): - return normpath(self.expand(x)) - - # extra events not present in the superclass: - - def end_build(self, scope, out, iout, rule, in_, iin, orderdep): - pass - - def end_rule(self, scope, name): - pass - - -# ---- test client that just prints back whatever it parsed ---- - -class Writer(NinjaParserEvents): - ARGS = argparse.ArgumentParser(description='Rewrite input build.ninja to stdout.') - - def __init__(self, output, parser, args): - super().__init__(parser) - self.output = output - self.indent = '' - self.had_vars = False - - def dollar_token(self, word, in_path=False): - return '$' + word - - def print(self, *args, **kwargs): - if len(args): - self.output.write(self.indent) - print(*args, **kwargs, file=self.output) - - def variable(self, name, value): - self.print('%s = %s' % (name, value)) - self.had_vars = True - - def begin_scope(self): - self.indent = ' ' - self.had_vars = False - - def end_scope(self): - if self.had_vars: - self.print() - self.indent = '' - self.had_vars = False - - def begin_pool(self, name): - self.print('pool %s' % name) - self.begin_scope() - - def begin_rule(self, name): - self.print('rule %s' % name) - self.begin_scope() - - def begin_build(self, outputs, implicit_outputs, rule, inputs, implicit, order_only): - all_outputs = list(outputs) - all_inputs = list(inputs) - - if implicit: - all_inputs.append('|') - all_inputs.extend(implicit) - if order_only: - all_inputs.append('||') - all_inputs.extend(order_only) - if implicit_outputs: - all_outputs.append('|') - all_outputs.extend(implicit_outputs) - - self.print('build %s: %s' % (' '.join(all_outputs), - ' '.join([rule] + all_inputs))) - self.begin_scope() - - def default(self, targets): - self.print('default %s' % ' '.join(targets)) - - -# ---- emit compile_commands.json ---- - -class Compdb(NinjaParserEventsWithVars): - ARGS = argparse.ArgumentParser(description='Emit compile_commands.json.') - ARGS.add_argument('rules', nargs='*', - help='The ninja rules to emit compilation commands for.') - - def __init__(self, output, parser, args): - super().__init__(parser) - self.output = output - self.rules = args.rules - self.sep = '' - - def begin_file(self): - self.output.write('[') - self.directory = os.getcwd() - - def print_entry(self, **entry): - entry['directory'] = self.directory - self.output.write(self.sep + json.dumps(entry)) - self.sep = ',\n' - - def begin_build(self, out, iout, rule, in_, iin, orderdep): - if in_ and rule in self.rules: - super().begin_build(out, iout, rule, in_, iin, orderdep) - else: - self.scope = Scope(self) - - def end_build(self, scope, out, iout, rule, in_, iin, orderdep): - self.print_entry(command=scope.expand('${command}'), file=in_[0]) - - def end_file(self): - self.output.write(']\n') - - -# ---- clean output files ---- - -class Clean(NinjaParserEventsWithVars): - ARGS = argparse.ArgumentParser(description='Remove output build files.') - ARGS.add_argument('-g', dest='generator', action='store_true', - help='clean generated files too') - - def __init__(self, output, parser, args): - super().__init__(parser) - self.dry_run = args.dry_run - self.verbose = args.verbose or args.dry_run - self.generator = args.generator - - def begin_file(self): - print('Cleaning... ', end=(None if self.verbose else ''), flush=True) - self.cnt = 0 - - def end_file(self): - print('%d files' % self.cnt) - - def do_clean(self, *files): - for f in files: - if self.dry_run: - if os.path.exists(f): - self.cnt += 1 - print('Would remove ' + f) - continue - else: - try: - if os.path.isdir(f): - shutil.rmtree(f) - else: - os.unlink(f) - self.cnt += 1 - if self.verbose: - print('Removed ' + f) - except FileNotFoundError: - pass - - def end_build(self, scope, out, iout, rule, in_, iin, orderdep): - if rule == 'phony': - return - if self.generator: - rspfile = scope.expand('${rspfile}') - if rspfile: - self.do_clean(rspfile) - if self.generator or not scope.expand('${generator}'): - self.do_clean(*out, *iout) - depfile = scope.expand('${depfile}') - if depfile: - self.do_clean(depfile) - - -# ---- convert build.ninja to makefile ---- - -class Ninja2Make(NinjaParserEventsWithVars): - ARGS = argparse.ArgumentParser(description='Convert build.ninja to a Makefile.') - ARGS.add_argument('--clean', dest='emit_clean', action='store_true', - help='Emit clean/distclean rules.') - ARGS.add_argument('--doublecolon', action='store_true', - help='Emit double-colon rules for phony targets.') - ARGS.add_argument('--omit', metavar='TARGET', nargs='+', - help='Targets to omit.') - - def __init__(self, output, parser, args): - super().__init__(parser) - self.output = output - - self.emit_clean = args.emit_clean - self.doublecolon = args.doublecolon - self.omit = set(args.omit) - - if self.emit_clean: - self.omit.update(['clean', 'distclean']) - - # Lists of targets are kept in memory and emitted only at the - # end because appending is really inefficient in GNU make. - # We only do it when it's O(#rules) or O(#variables), but - # never when it could be O(#targets). - self.depfiles = list() - self.rspfiles = list() - self.build_vars = defaultdict(lambda: dict()) - self.rule_targets = defaultdict(lambda: list()) - self.stamp_targets = defaultdict(lambda: list()) - self.all_outs = set() - self.all_ins = set() - self.all_phony = set() - self.seen_default = False - - def print(self, *args, **kwargs): - print(*args, **kwargs, file=self.output) - - def dollar_token(self, word, in_path=False): - if in_path and word == ' ': - self.parser.parse_error('Make does not support spaces in filenames') - return '$$' if word == '$' else word - - def print_phony(self, outs, ins): - targets = ' '.join(outs).replace('$', '$$') - deps = ' '.join(ins).replace('$', '$$') - deps = deps.strip() - if self.doublecolon: - self.print(targets + '::' + (' ' if deps else '') + deps + ';@:') - else: - self.print(targets + ':' + (' ' if deps else '') + deps) - self.all_phony.update(outs) - - def begin_file(self): - self.print(r'# This is an automatically generated file, and it shows.') - self.print(r'ninja-default:') - self.print(r'.PHONY: ninja-default ninja-clean ninja-distclean') - if self.emit_clean: - self.print(r'ninja-clean:: ninja-clean-start; $(if $V,,@)rm -f ${ninja-depfiles}') - self.print(r'ninja-clean-start:; $(if $V,,@echo Cleaning...)') - self.print(r'ninja-distclean:: clean; $(if $V,,@)rm -f ${ninja-rspfiles}') - self.print(r'.PHONY: ninja-clean-start') - self.print_phony(['clean'], ['ninja-clean']) - self.print_phony(['distclean'], ['ninja-distclean']) - self.print(r'vpath') - self.print(r'NULL :=') - self.print(r'SPACE := ${NULL} #') - self.print(r'MAKEFLAGS += -rR') - self.print(r'define NEWLINE') - self.print(r'') - self.print(r'endef') - self.print(r'.var.in_newline = $(subst $(SPACE),$(NEWLINE),${.var.in})') - self.print(r"ninja-command = $(if $V,,$(if ${.var.description},@printf '%s\n' '$(subst ','\'',${.var.description})' && ))${.var.command}") - self.print(r"ninja-command-restat = $(if $V,,$(if ${.var.description},@printf '%s\n' '$(subst ','\'',${.var.description})' && ))${.var.command} && if test -e $(firstword ${.var.out}); then printf '%s\n' ${.var.out} > $@; fi") - - def end_file(self): - def natural_sort_key(s, _nsre=re.compile('([0-9]+)')): - return [int(text) if text.isdigit() else text.lower() - for text in _nsre.split(s)] - - self.print() - self.print('ninja-outputdirs :=') - for rule in self.rule_vars: - if rule == 'phony': - continue - self.print('ninja-targets-%s := %s' % (rule, ' '.join(self.rule_targets[rule]))) - self.print('ninja-stamp-%s := %s' % (rule, ' '.join(self.stamp_targets[rule]))) - self.print('ninja-outputdirs += $(sort $(dir ${ninja-targets-%s}))' % rule) - self.print() - self.print('dummy := $(shell mkdir -p . $(sort $(ninja-outputdirs)))') - self.print('ninja-depfiles :=' + ' '.join(self.depfiles)) - self.print('ninja-rspfiles :=' + ' '.join(self.rspfiles)) - self.print('-include ${ninja-depfiles}') - self.print() - for targets in self.build_vars: - for name, value in self.build_vars[targets].items(): - self.print('%s: private .var.%s := %s' % - (targets, name, value.replace('$', '$$'))) - self.print() - if not self.seen_default: - default_targets = sorted(self.all_outs - self.all_ins, key=natural_sort_key) - self.print('ninja-default: ' + ' '.join(default_targets)) - - # This is a hack... Meson declares input meson.build files as - # phony, because Ninja does not have an equivalent of Make's - # "path/to/file:" declaration that ignores "path/to/file" even - # if it is absent. However, Makefile.ninja wants to depend on - # build.ninja, which in turn depends on these phony targets which - # would cause Makefile.ninja to be rebuilt in a loop. - phony_targets = sorted(self.all_phony - self.all_ins, key=natural_sort_key) - self.print('.PHONY: ' + ' '.join(phony_targets)) - - def variable(self, name, value): - super().variable(name, value) - if self.scope is None: - self.global_vars[name] = self.expand(value) - self.print('.var.%s := %s' % (name, self.global_vars[name])) - - def begin_build(self, out, iout, rule, in_, iin, orderdep): - if any(x in self.omit for x in out): - self.scope = Scope(self) - return - - super().begin_build(out, iout, rule, in_, iin, orderdep) - self.current_targets = ' '.join(self.scope.out + self.scope.iout).replace('$', '$$') - - def end_build(self, scope, out, iout, rule, in_, iin, orderdep): - self.rule_targets[rule] += self.scope.out - self.rule_targets[rule] += self.scope.iout - - self.all_outs.update(self.scope.iout) - self.all_outs.update(self.scope.out) - self.all_ins.update(self.scope.in_) - self.all_ins.update(self.scope.iin) - - targets = self.current_targets - self.current_targets = None - if rule == 'phony': - # Phony rules treat order-only dependencies as normal deps - self.print_phony(out + iout, in_ + iin + orderdep) - return - - inputs = ' '.join(in_ + iin).replace('$', '$$') - orderonly = ' '.join(orderdep).replace('$', '$$') - - rspfile = scope.expand('${rspfile}') - if rspfile: - rspfile_content = scope.expand('${rspfile_content}') - with open(rspfile, 'w') as f: - f.write(rspfile_content) - inputs += ' ' + rspfile - self.rspfiles.append(rspfile) - - restat = 'restat' in self.scope.build_vars or 'restat' in self.rule_vars[rule] - depfile = scope.expand('${depfile}') - build_vars = { - 'command': scope.expand('${command}'), - 'description': scope.expand('${description}'), - 'out': scope.expand('${out}') - } - - if restat and not depfile: - if len(out) == 1: - stamp = out[0] + '.stamp' - else: - stamp = '%s@%s.stamp' % (rule, sha1_text(targets)[0:11]) - self.print('%s: %s; @:' % (targets, stamp)) - self.print('ifneq (%s, $(wildcard %s))' % (targets, targets)) - self.print('.PHONY: %s' % (stamp, )) - self.print('endif') - self.print('%s: %s | %s; ${ninja-command-restat}' % (stamp, inputs, orderonly)) - self.rule_targets[rule].append(stamp) - self.stamp_targets[rule].append(stamp) - self.build_vars[stamp] = build_vars - else: - self.print('%s: %s | %s; ${ninja-command}' % (targets, inputs, orderonly)) - self.build_vars[targets] = build_vars - if depfile: - self.depfiles.append(depfile) - - def end_rule(self, scope, name): - # Note that the generator pseudo-variable could also be attached - # to a build block rather than a rule. This is not handled here - # in order to reduce the number of "rm" invocations. However, - # "ninjatool.py -t clean" does that correctly. - target = 'distclean' if scope.generator else 'clean' - self.print('ninja-%s:: ; $(if $V,,@)rm -f ${ninja-stamp-%s}' % (target, name)) - if self.emit_clean: - self.print('ninja-%s:: ; $(if $V,,@)rm -rf ${ninja-targets-%s}' % (target, name)) - - def default(self, targets): - self.print("ninja-default: " + ' '.join(targets)) - self.seen_default = True - - -# ---- command line parsing ---- - -# we cannot use subparsers because tools are chosen through the "-t" -# option. - -class ToolAction(argparse.Action): - def __init__(self, option_strings, dest, choices, metavar='TOOL', nargs=None, **kwargs): - if nargs is not None: - raise ValueError("nargs not allowed") - super().__init__(option_strings, dest, required=True, choices=choices, - metavar=metavar, **kwargs) - - def __call__(self, parser, namespace, value, option_string): - tool = self.choices[value] - setattr(namespace, self.dest, tool) - tool.ARGS.prog = '%s %s %s' % (parser.prog, option_string, value) - - -class ToolHelpAction(argparse.Action): - def __init__(self, option_strings, dest, nargs=None, **kwargs): - if nargs is not None: - raise ValueError("nargs not allowed") - super().__init__(option_strings, dest, nargs=0, **kwargs) - - def __call__(self, parser, namespace, values, option_string=None): - if namespace.tool: - namespace.tool.ARGS.print_help() - else: - parser.print_help() - parser.exit() - - -tools = { - 'test': Writer, - 'ninja2make': Ninja2Make, - 'compdb': Compdb, - 'clean': Clean, -} - -parser = argparse.ArgumentParser(description='Process and transform build.ninja files.', - add_help=False) -parser.add_argument('-C', metavar='DIR', dest='dir', default='.', - help='change to DIR before doing anything else') -parser.add_argument('-f', metavar='FILE', dest='file', default='build.ninja', - help='specify input build file [default=build.ninja]') -parser.add_argument('-n', dest='dry_run', action='store_true', - help='do not actually do anything') -parser.add_argument('-v', dest='verbose', action='store_true', - help='be more verbose') - -parser.add_argument('-t', dest='tool', choices=tools, action=ToolAction, - help='choose the tool to run') -parser.add_argument('-h', '--help', action=ToolHelpAction, - help='show this help message and exit') - -if len(sys.argv) >= 2 and sys.argv[1] == '--version': - print('1.8') - sys.exit(0) - -args, tool_args = parser.parse_known_args() -args.tool.ARGS.parse_args(tool_args, args) - -os.chdir(args.dir) -with open(args.file, 'r') as f: - parser = NinjaParser(args.file, f) - try: - events = args.tool(sys.stdout, parser, args) - except InvalidArgumentError as e: - parser.error(str(e)) - parser.parse(events) diff --git a/tests/Makefile.include b/tests/Makefile.include index 4037490b69..3a0524ce74 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -140,7 +140,7 @@ QEMU_IOTESTS_HELPERS-$(CONFIG_LINUX) = tests/qemu-iotests/socket_scm_helper$(EXE check: check-block check-block: $(SRC_PATH)/tests/check-block.sh qemu-img$(EXESUF) \ qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) \ - $(filter qemu-system-%, $(ninja-targets-c_LINKER) $(ninja-targets-cpp_LINKER)) + $(filter qemu-system-%, $(ninja-targets)) @$< endif -- 2.26.2 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH 7/7] build: replace ninjatool with ninja 2020-10-14 13:54 ` [PATCH 7/7] build: replace ninjatool with ninja Paolo Bonzini @ 2020-10-14 16:38 ` Daniel P. Berrangé 0 siblings, 0 replies; 29+ messages in thread From: Daniel P. Berrangé @ 2020-10-14 16:38 UTC (permalink / raw) To: Paolo Bonzini; +Cc: peter.maydell, alex.bennee, qemu-devel On Wed, Oct 14, 2020 at 09:54:16AM -0400, Paolo Bonzini wrote: > Now that the build is done entirely by Meson, there is no need > to keep the Makefile conversion. Instead, we can ask Ninja about > the targets it exposes and forward them. > > The main advantages are, from smallest to largest: > > - reducing the possible namespace pollution within the Makefile > > - removal of a relatively large Python program > > - faster build because parsing Makefile.ninja is slower than > parsing build.ninja; and faster build after Meson runs because > we do not have to generate Makefile.ninja. > > - tracking of command lines, which provides more accurate rebuilds > > In addition the change removes the requirement for GNU make 3.82, which > was annoying on Mac, and avoids bugs on Windows due to ninjatool not > knowing how to convert Windows escapes to POSIX escapes. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > Makefile | 40 +- > configure | 9 +- > docs/devel/build-system.rst | 6 +- > meson.build | 4 - > scripts/mtest2make.py | 4 +- > scripts/ninjatool.py | 1008 ----------------------------------- > tests/Makefile.include | 2 +- > 7 files changed, 40 insertions(+), 1033 deletions(-) > delete mode 100755 scripts/ninjatool.py Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/7] build: replace ninjatool with ninja 2020-10-14 13:54 [PATCH 0/7] build: replace ninjatool with ninja Paolo Bonzini ` (6 preceding siblings ...) 2020-10-14 13:54 ` [PATCH 7/7] build: replace ninjatool with ninja Paolo Bonzini @ 2020-10-14 13:57 ` Peter Maydell 2020-10-15 6:15 ` Howard Spoelstra 2020-10-15 18:21 ` 罗勇刚(Yonggang Luo) 9 siblings, 0 replies; 29+ messages in thread From: Peter Maydell @ 2020-10-14 13:57 UTC (permalink / raw) To: Paolo Bonzini; +Cc: Daniel P. Berrange, Alex Bennée, QEMU Developers On Wed, 14 Oct 2020 at 14:54, Paolo Bonzini <pbonzini@redhat.com> wrote: > > This pull request is the last build system change for 5.2 from > me, and it is simple: similar to how we are invoking > ROM or tests/tcg "make" from the main Makefile, we now invoke ninja > to build QEMU. Unlike those cases, however, build.ninja targets are > forwarded transparently. > 27 files changed, 121 insertions(+), 1070 deletions(-) The diffstat is certainly persuasive :-) -- PMM ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/7] build: replace ninjatool with ninja 2020-10-14 13:54 [PATCH 0/7] build: replace ninjatool with ninja Paolo Bonzini ` (7 preceding siblings ...) 2020-10-14 13:57 ` [PATCH 0/7] " Peter Maydell @ 2020-10-15 6:15 ` Howard Spoelstra 2020-10-15 6:34 ` Paolo Bonzini 2020-10-15 18:21 ` 罗勇刚(Yonggang Luo) 9 siblings, 1 reply; 29+ messages in thread From: Howard Spoelstra @ 2020-10-15 6:15 UTC (permalink / raw) To: Paolo Bonzini Cc: Alex Bennée, Daniel P. Berrangé, qemu-devel qemu-devel, Peter Maydell [-- Attachment #1: Type: text/plain, Size: 3375 bytes --] On Wed, Oct 14, 2020 at 3:57 PM Paolo Bonzini <pbonzini@redhat.com> wrote: > This pull request is the last build system change for 5.2 from > me, and it is simple: similar to how we are invoking > ROM or tests/tcg "make" from the main Makefile, we now invoke ninja > to build QEMU. Unlike those cases, however, build.ninja targets are > forwarded transparently. > > The advantages cover various areas: > > - maintainability: we drop scripts/ninjatool.py, which is > a large and hairy piece of code, and generally remove one > thing that can go wrong and one thing that is unique to QEMU; > > - platform support: we remove the requirement for GNU make > 3.82, which was annoying on Mac. We also avoid bugs on Windows > due to meson emitting Windows rather than POSIX escapes (as > expected by Ninja) and ninjatool ignoring the difference; > > - speed: invoking "configure" does not have to generate > 44k lines of rules, while invoking "Make" does not anymore have > to parse 44k lines of rules. > > - ease of use: Ninja tracks command lines, hence the problem > of static library changing the objects they hold goes away > > Paolo > > Hi, I still have some issues when compiling on Windows with msys2/mingw64 after this patch set. Command line: mkdir build && cd build && ../configure --cross-prefix=x86_64-w64-mingw32- --target-list=ppc-softmmu --enable-gtk --enable-sdl Traceback (most recent call last): File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/mesonmain.py", line 131, in run return options.run_func(options) File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/msetup.py", line 245, in run app.generate() File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/msetup.py", line 159, in generate self._generate(env) File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/msetup.py", line 215, in _generate intr.backend.generate() File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/backend/ninjabackend.py", line 483, in generate ninja = environment.detect_ninja_command_and_version(log=True) File "C:/msys64/home/hsp/src/qemu-master/meson/mesonbuild/environment.py", line 177, in detect_ninja_command_and_version name = os.path.basename(n) File "C:/msys64/mingw64/lib/python3.8/ntpath.py", line 231, in basename return split(p)[1] File "C:/msys64/mingw64/lib/python3.8/ntpath.py", line 200, in split p = os.fspath(p) TypeError: expected str, bytes or os.PathLike object, not NoneType ERROR: meson setup failed When I set --ninja=ninja explicitly, this error does not occur: Found ninja.EXE-1.10.1 at C:\msys64\mingw64\bin/ninja.EXE However, compiling then runs into a problem: $ make -j8 /mingw64/bin/python3 -B /home/hsp/src/qemu-master/meson/meson.py introspect --targets --tests --benchmarks | /mingw64/bin/python3 -B scripts/mtest2make.py > Makefile.mtest { echo 'ninja-targets = \'; ninja -t targets all | sed 's/:.*//; $!s/$/ \\/'; } > Makefile.ninja ninja -j8 all | cat make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. Stop. make: *** [Makefile:171: pc-bios/optionrom/all] Error 2 make: *** Waiting for unfinished jobs.... The problem with optionrom was already present earlier and could be fixed by applying: https://patchwork.ozlabs.org/project/qemu-devel/list/?series=202798 That patch set does by now not apply cleanly anymore. Best, Howard [-- Attachment #2: Type: text/html, Size: 4438 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/7] build: replace ninjatool with ninja 2020-10-15 6:15 ` Howard Spoelstra @ 2020-10-15 6:34 ` Paolo Bonzini 2020-10-15 7:39 ` Howard Spoelstra 0 siblings, 1 reply; 29+ messages in thread From: Paolo Bonzini @ 2020-10-15 6:34 UTC (permalink / raw) To: Howard Spoelstra Cc: Alex Bennée, Daniel P. Berrangé, qemu-devel qemu-devel, Peter Maydell On 15/10/20 08:15, Howard Spoelstra wrote: > ERROR: meson setup failed > > When I set --ninja=ninja explicitly, this error does not occur: > Found ninja.EXE-1.10.1 at C:\msys64\mingw64\bin/ninja.EXE That is fixed by Meson 0.55.2, I'll add a submodule update. Thanks for the reminder! Paolo ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/7] build: replace ninjatool with ninja 2020-10-15 6:34 ` Paolo Bonzini @ 2020-10-15 7:39 ` Howard Spoelstra 2020-10-15 9:44 ` Paolo Bonzini 2020-10-15 17:39 ` Volker Rümelin 0 siblings, 2 replies; 29+ messages in thread From: Howard Spoelstra @ 2020-10-15 7:39 UTC (permalink / raw) To: Paolo Bonzini Cc: Alex Bennée, Daniel P. Berrangé, qemu-devel qemu-devel, Peter Maydell [-- Attachment #1: Type: text/plain, Size: 665 bytes --] On Thu, Oct 15, 2020 at 8:34 AM Paolo Bonzini <pbonzini@redhat.com> wrote: > On 15/10/20 08:15, Howard Spoelstra wrote: > > ERROR: meson setup failed > > > > When I set --ninja=ninja explicitly, this error does not occur: > > Found ninja.EXE-1.10.1 at C:\msys64\mingw64\bin/ninja.EXE > > That is fixed by Meson 0.55.2, I'll add a submodule update. Thanks for > the reminder! > > Paolo > > Thanks Paolo, Then only the issue regarding the pcbios/optionrom stuff remains ;-) make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. Stop. make: *** [Makefile:171: pc-bios/optionrom/all] Error 2 make: *** Waiting for unfinished jobs.... Best, Howard [-- Attachment #2: Type: text/html, Size: 1197 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/7] build: replace ninjatool with ninja 2020-10-15 7:39 ` Howard Spoelstra @ 2020-10-15 9:44 ` Paolo Bonzini 2020-10-15 17:39 ` Volker Rümelin 1 sibling, 0 replies; 29+ messages in thread From: Paolo Bonzini @ 2020-10-15 9:44 UTC (permalink / raw) To: Howard Spoelstra Cc: Alex Bennée, Daniel P. Berrangé, qemu-devel qemu-devel, Peter Maydell On 15/10/20 09:39, Howard Spoelstra wrote: > > > Thanks Paolo, > > Then only the issue regarding the pcbios/optionrom stuff remains ;-) > > make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. Stop. > make: *** [Makefile:171: pc-bios/optionrom/all] Error 2 > make: *** Waiting for unfinished jobs.... It seems to work on cirrus CI: https://cirrus-ci.com/task/5383432727429120 Paolo ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/7] build: replace ninjatool with ninja 2020-10-15 7:39 ` Howard Spoelstra 2020-10-15 9:44 ` Paolo Bonzini @ 2020-10-15 17:39 ` Volker Rümelin 2020-10-15 18:49 ` Mark Cave-Ayland 2020-10-15 23:07 ` Howard Spoelstra 1 sibling, 2 replies; 29+ messages in thread From: Volker Rümelin @ 2020-10-15 17:39 UTC (permalink / raw) To: Howard Spoelstra, Paolo Bonzini Cc: Alex Bennée, Daniel P. Berrangé, qemu-devel, Peter Maydell > > Thanks Paolo, > > Then only the issue regarding the pcbios/optionrom stuff remains ;-) > > make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. Stop. > make: *** [Makefile:171: pc-bios/optionrom/all] Error 2 > make: *** Waiting for unfinished jobs.... > > Best, > Howard > Hi Howard, one solution for this issue is to uncomment the following line in msys2_shell.cmd. rem To activate windows native symlinks uncomment next line set MSYS=winsymlinks:nativestrict Then tell Windows 10 it's okay to create symlinks without elevated rights. Here is a link with a description how to do this. https://www.joshkel.com/2018/01/18/symlinks-in-windows/ I think since commit bf708f3c4a "optionrom: simplify Makefile" pc-bios/optionrom/Makefile in your build directory has to be a symbolic link. Without 'set MSYS=winsymlinks:nativestrict' msys2 ln -s copies the Makefile instead of creating a symbolic link. With best regards, Volker ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/7] build: replace ninjatool with ninja 2020-10-15 17:39 ` Volker Rümelin @ 2020-10-15 18:49 ` Mark Cave-Ayland 2020-10-15 21:41 ` Paolo Bonzini 2020-10-15 23:07 ` Howard Spoelstra 1 sibling, 1 reply; 29+ messages in thread From: Mark Cave-Ayland @ 2020-10-15 18:49 UTC (permalink / raw) To: Volker Rümelin, Howard Spoelstra, Paolo Bonzini Cc: Daniel P. Berrangé, Alex Bennée, qemu-devel, Peter Maydell On 15/10/2020 18:39, Volker Rümelin wrote: >> Thanks Paolo, >> >> Then only the issue regarding the pcbios/optionrom stuff remains ;-) >> >> make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. Stop. >> make: *** [Makefile:171: pc-bios/optionrom/all] Error 2 >> make: *** Waiting for unfinished jobs.... >> >> Best, >> Howard >> > > Hi Howard, > > one solution for this issue is to uncomment the following line in msys2_shell.cmd. > > rem To activate windows native symlinks uncomment next line > set MSYS=winsymlinks:nativestrict > > Then tell Windows 10 it's okay to create symlinks without elevated rights. Here is a link with a description how to do this. > https://www.joshkel.com/2018/01/18/symlinks-in-windows/ > > I think since commit bf708f3c4a "optionrom: simplify Makefile" pc-bios/optionrom/Makefile in your build directory has to be a symbolic link. Without 'set MSYS=winsymlinks:nativestrict' msys2 ln -s copies the Makefile instead of creating a symbolic link. Wow thanks for this. I had the same issue as Howard, and I can confirm that this fixes the problem for me. Having said that it was quite fiddly to get this working correctly - is there any reason why https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg06997.html still can't be merged? ATB, Mark. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/7] build: replace ninjatool with ninja 2020-10-15 18:49 ` Mark Cave-Ayland @ 2020-10-15 21:41 ` Paolo Bonzini 2020-10-16 9:35 ` Mark Cave-Ayland 0 siblings, 1 reply; 29+ messages in thread From: Paolo Bonzini @ 2020-10-15 21:41 UTC (permalink / raw) To: Mark Cave-Ayland Cc: Peter Maydell, Daniel P. Berrangé, Volker Rümelin, qemu-devel, Howard Spoelstra, Alex Bennée [-- Attachment #1: Type: text/plain, Size: 1445 bytes --] Il gio 15 ott 2020, 20:49 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> ha scritto: > Is there any reason why > https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg06997.html still > can't be > merged? > Because it's not the right approach. There is no reason why building firmware cannot be done with cross compilers, so moving those directories to Meson (not because Meson can't handle them; more specifically, the issue is tying the firmware build to the QEMU build system) is going in the wrong direction. The "Canadian cross" scenario, where you build on Linux a mingw GCC but the compiler is s390, is not even enough to describe the complexity in the case of QEMU, because there are multiple firmware for different machines. However we already have all the infrastructure to do such builds, we just don't use it for the firmware. So, instead of the patch you recalled above, the tests/tcg machinery should be extended into something that can be reused for firmware. As an aside, orchestrating this multi-compiler part of the build is what the Makefiles will keep on handling for the foreseeable future. As an aside to the aside, tests/tcg is more than underdocumented and I forget everything about it 5 minutes after looking at it. This is not something that I will be able to work on anytime soon. But still I don't think that going in the wrong direction is a good idea, even if temporarily. Paolo > > ATB, > > Mark. > > [-- Attachment #2: Type: text/html, Size: 2340 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/7] build: replace ninjatool with ninja 2020-10-15 21:41 ` Paolo Bonzini @ 2020-10-16 9:35 ` Mark Cave-Ayland 0 siblings, 0 replies; 29+ messages in thread From: Mark Cave-Ayland @ 2020-10-16 9:35 UTC (permalink / raw) To: Paolo Bonzini Cc: Peter Maydell, Daniel P. Berrangé, Volker Rümelin, qemu-devel, Howard Spoelstra, Alex Bennée On 15/10/2020 22:41, Paolo Bonzini wrote: > Il gio 15 ott 2020, 20:49 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk > <mailto:mark.cave-ayland@ilande.co.uk>> ha scritto: > > Is there any reason why > https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg06997.html > <https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg06997.html> still can't be > merged? > > > Because it's not the right approach. There is no reason why building firmware cannot > be done with cross compilers, so moving those directories to Meson (not because Meson > can't handle them; more specifically, the issue is tying the firmware build to the > QEMU build system) is going in the wrong direction. > > The "Canadian cross" scenario, where you build on Linux a mingw GCC but the compiler > is s390, is not even enough to describe the complexity in the case of QEMU, because > there are multiple firmware for different machines. > > However we already have all the infrastructure to do such builds, we just don't use > it for the firmware. So, instead of the patch you recalled above, the tests/tcg > machinery should be extended into something that can be reused for firmware. As an > aside, orchestrating this multi-compiler part of the build is what the Makefiles will > keep on handling for the foreseeable future. As an aside to the aside, tests/tcg is > more than underdocumented and I forget everything about it 5 minutes after looking at it. > > This is not something that I will be able to work on anytime soon. But still I don't > think that going in the wrong direction is a good idea, even if temporarily. That's a shame, although I do appreciate the huge amount of time and effort that you've put into this release in order to get the Meson build up and running, and so why taking on another large task is going to be lower down the list :) At the moment OpenBIOS doesn't have a docker image capable of building the required binaries: I did experiment with trying to use the QEMU docker images for openbios-ppc but whilst the binary built successfully, it did not run compared to my hand-rolled compilers. So there's still some debugging to be done there... ATB, Mark. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/7] build: replace ninjatool with ninja 2020-10-15 17:39 ` Volker Rümelin 2020-10-15 18:49 ` Mark Cave-Ayland @ 2020-10-15 23:07 ` Howard Spoelstra 1 sibling, 0 replies; 29+ messages in thread From: Howard Spoelstra @ 2020-10-15 23:07 UTC (permalink / raw) To: Volker Rümelin Cc: Alex Bennée, Paolo Bonzini, Daniel P. Berrangé, qemu-devel, Peter Maydell [-- Attachment #1: Type: text/plain, Size: 1596 bytes --] On Thu, Oct 15, 2020 at 7:39 PM Volker Rümelin <vr_qemu@t-online.de> wrote: > > > > > Thanks Paolo, > > > > Then only the issue regarding the pcbios/optionrom stuff remains ;-) > > > > make[1]: *** No rule to make target 'multiboot.bin', needed by 'all'. > Stop. > > make: *** [Makefile:171: pc-bios/optionrom/all] Error 2 > > make: *** Waiting for unfinished jobs.... > > > > Best, > > Howard > > > > Hi Howard, > > one solution for this issue is to uncomment the following line in > msys2_shell.cmd. > > rem To activate windows native symlinks uncomment next line > set MSYS=winsymlinks:nativestrict > > Then tell Windows 10 it's okay to create symlinks without elevated rights. > Here is a link with a description how to do this. > https://www.joshkel.com/2018/01/18/symlinks-in-windows/ > > I think since commit bf708f3c4a "optionrom: simplify Makefile" > pc-bios/optionrom/Makefile in your build directory has to be a symbolic > link. Without 'set MSYS=winsymlinks:nativestrict' msys2 ln -s copies the > Makefile instead of creating a symbolic link. > > Thanks Volker! I changed the Windows policy and setting in msys2_shell.cmd. However, I had to edit ming64.ini to uncomment the MSYS=winsymlinks:nativestrict there as well to get things going. While it is great to have this fix, I can't say I'm really happy with the need to change the Windows policy and to have to fix msys2. The patches Mark referred to and to which I also pointed earlier did fix this problem without changing Windows/Msys2/Mingw64 settings. Best regards, Howard [-- Attachment #2: Type: text/html, Size: 2297 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/7] build: replace ninjatool with ninja 2020-10-14 13:54 [PATCH 0/7] build: replace ninjatool with ninja Paolo Bonzini ` (8 preceding siblings ...) 2020-10-15 6:15 ` Howard Spoelstra @ 2020-10-15 18:21 ` 罗勇刚(Yonggang Luo) 9 siblings, 0 replies; 29+ messages in thread From: 罗勇刚(Yonggang Luo) @ 2020-10-15 18:21 UTC (permalink / raw) To: Paolo Bonzini Cc: Alex Bennée, Daniel P. Berrangé, qemu-level, Peter Maydell [-- Attachment #1: Type: text/plain, Size: 7008 bytes --] On Wed, Oct 14, 2020 at 9:58 PM Paolo Bonzini <pbonzini@redhat.com> wrote: > > This pull request is the last build system change for 5.2 from > me, and it is simple: similar to how we are invoking > ROM or tests/tcg "make" from the main Makefile, we now invoke ninja > to build QEMU. Unlike those cases, however, build.ninja targets are > forwarded transparently. > > The advantages cover various areas: > > - maintainability: we drop scripts/ninjatool.py, which is > a large and hairy piece of code, and generally remove one > thing that can go wrong and one thing that is unique to QEMU; > > - platform support: we remove the requirement for GNU make > 3.82, which was annoying on Mac. We also avoid bugs on Windows > due to meson emitting Windows rather than POSIX escapes (as > expected by Ninja) and ninjatool ignoring the difference; > > - speed: invoking "configure" does not have to generate > 44k lines of rules, while invoking "Make" does not anymore have > to parse 44k lines of rules. > > - ease of use: Ninja tracks command lines, hence the problem > of static library changing the objects they hold goes away > > Paolo > > Paolo Bonzini (7): > tests/Makefile.include: unbreak non-tcg builds > make: run shell with pipefail > tests: add missing generated sources to testqapi > configure: move QEMU_INCLUDES to meson > dockerfiles: enable Centos 8 PowerTools > add ninja to dockerfiles, CI configurations and test VMs > build: replace ninjatool with ninja > > .cirrus.yml | 6 +- > .travis.yml | 13 + > Makefile | 42 +- > configure | 29 +- > docs/devel/build-system.rst | 6 +- > meson.build | 34 +- > scripts/mtest2make.py | 4 +- > scripts/ninjatool.py | 1008 -------------------- > tests/Makefile.include | 2 +- > tests/docker/dockerfiles/centos7.docker | 1 + > tests/docker/dockerfiles/centos8.docker | 4 + > tests/docker/dockerfiles/debian10.docker | 1 + > tests/docker/dockerfiles/fedora.docker | 1 + > tests/docker/dockerfiles/travis.docker | 2 +- > tests/docker/dockerfiles/ubuntu.docker | 1 + > tests/docker/dockerfiles/ubuntu1804.docker | 1 + > tests/docker/dockerfiles/ubuntu2004.docker | 1 + > tests/include/meson.build | 8 +- > tests/meson.build | 14 +- > tests/vm/centos | 2 +- > tests/vm/centos.aarch64 | 2 +- > tests/vm/fedora | 2 +- > tests/vm/freebsd | 1 + > tests/vm/netbsd | 1 + > tests/vm/openbsd | 1 + > tests/vm/ubuntu.aarch64 | 2 +- > tests/vm/ubuntu.i386 | 2 +- > 27 files changed, 121 insertions(+), 1070 deletions(-) > delete mode 100755 scripts/ninjatool.py > > -- > 2.26.2 > > That's good, it's broken my locally rST doc build ``` [226/1564] Generating input-keymap-qnum-to-qcode.c.inc with a custom command (wrapped by meson to capture output) [227/1564] Generating input-keymap-qcode-to-qnum.c.inc with a custom command (wrapped by meson to capture output) [228/1564] Generating shared QAPI source files with a custom command [229/1564] Generating input-keymap-qcode-to-linux.c.inc with a custom command (wrapped by meson to capture output) [230/1564] Generating input-keymap-win32-to-qcode.c.inc with a custom command (wrapped by meson to capture output) [231/1564] Generating QAPI rST doc with a custom command FAILED: tests/qapi-schema/doc-good.txt "/mingw64/bin/sphinx-build" "-Dversion=5.1.50" "-Drelease=" "-W" "-b" "text" "-E" "-c" "C:/work/xemu/qemu/docs" "-D" "master_doc=doc-good" "C:/work/xemu/qemu/tests/qapi-schema" "C:/work/xemu/qemu/build/tests/qapi-schema" CreateProcess failed: The system cannot find the file specified. [232/1564] Generating input-keymap-qcode-to-atset1.c.inc with a custom command (wrapped by meson to capture output) [233/1564] Generating generated-helpers-wrappers.h with a custom command (wrapped by meson to capture output) ninja: fatal: GetOverlappedResult: �����Ч�� make: *** [Makefile:87:run-ninja] 错误 1 /mingw64/bin/ninja -v -j1 tests/check-block-qdict.exe tests/check-qdict.exe tests/check-qjson.exe tests/check-qlist.exe tests/check-qlit.exe tests/check-qnull.exe tests/check-qnum.exe tests/check-qobject.exe tests/check-qom-interface.exe tests/check-qom-proplist.exe tests/check-qstring.exe tests/ptimer-test.exe tests/qapi-schema/doc-good.ref.nocr tests/qapi-schema/doc-good.txt.nocr tests/rcutorture.exe tests/test-aio-multithread.exe tests/test-aio.exe tests/test-authz-list.exe tests/test-authz-listfile.exe tests/test-authz-simple.exe tests/test-base64.exe tests/test-bdrv-drain.exe tests/test-bdrv-graph-mod.exe tests/test-bitcnt.exe tests/test-bitmap.exe tests/test-bitops.exe tests/test-block-backend.exe tests/test-block-iothread.exe tests/test-blockjob-txn.exe tests/test-blockjob.exe tests/test-bufferiszero.exe tests/test-char.exe tests/test-clone-visitor.exe tests/test-coroutine.exe tests/test-crypto-afsplit.exe tests/test-crypto-block.exe tests/test-crypto-cipher.exe tests/test-crypto-hash.exe tests/test-crypto-hmac.exe tests/test-crypto-ivgen.exe tests/test-crypto-pbkdf.exe tests/test-crypto-secret.exe tests/test-cutils.exe tests/test-hbitmap.exe tests/test-int128.exe tests/test-io-channel-buffer.exe tests/test-io-channel-command.exe tests/test-io-channel-file.exe tests/test-io-channel-socket.exe tests/test-io-task.exe tests/test-iov.exe tests/test-keyval.exe tests/test-logging.exe tests/test-mul64.exe tests/test-opts-visitor.exe tests/test-qapi-util.exe tests/test-qdev-global-props.exe tests/test-qdist.exe tests/test-qemu-opts.exe tests/test-qgraph.exe tests/test-qht.exe tests/test-qmp-cmds.exe tests/test-qmp-event.exe tests/test-qobject-input-visitor.exe tests/test-qobject-output-visitor.exe tests/test-rcu-list.exe tests/test-rcu-simpleq.exe tests/test-rcu-slist.exe tests/test-rcu-tailq.exe tests/test-replication.exe tests/test-shift128.exe tests/test-string-input-visitor.exe tests/test-string-output-visitor.exe tests/test-thread-pool.exe tests/test-throttle.exe tests/test-timed-average.exe tests/test-util-sockets.exe tests/test-uuid.exe tests/test-visitor-serialization.exe tests/test-vmstate.exe tests/test-write-threshold.exe tests/test-x86-cpuid.exe tests/test-xbzrle.exe | cat ``` That's because currently sphinx-build are found in configure not in meson, maybe it's worth to move the sphinx related thing into meson totally. -- 此致 礼 罗勇刚 Yours sincerely, Yonggang Luo [-- Attachment #2: Type: text/html, Size: 8008 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2020-10-16 9:36 UTC | newest] Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-10-14 13:54 [PATCH 0/7] build: replace ninjatool with ninja Paolo Bonzini 2020-10-14 13:54 ` [PATCH 1/7] tests/Makefile.include: unbreak non-tcg builds Paolo Bonzini 2020-10-14 16:26 ` Daniel P. Berrangé 2020-10-14 13:54 ` [PATCH 2/7] make: run shell with pipefail Paolo Bonzini 2020-10-14 16:27 ` Daniel P. Berrangé 2020-10-14 13:54 ` [PATCH 3/7] tests: add missing generated sources to testqapi Paolo Bonzini 2020-10-14 16:27 ` Daniel P. Berrangé 2020-10-14 13:54 ` [PATCH 4/7] configure: move QEMU_INCLUDES to meson Paolo Bonzini 2020-10-14 16:28 ` Daniel P. Berrangé 2020-10-14 13:54 ` [PATCH 5/7] dockerfiles: enable Centos 8 PowerTools Paolo Bonzini 2020-10-14 16:33 ` Daniel P. Berrangé 2020-10-14 20:49 ` Cleber Rosa 2020-10-15 6:36 ` Paolo Bonzini 2020-10-14 13:54 ` [PATCH 6/7] add ninja to dockerfiles, CI configurations and test VMs Paolo Bonzini 2020-10-14 16:36 ` Daniel P. Berrangé 2020-10-15 6:50 ` Alex Bennée 2020-10-14 13:54 ` [PATCH 7/7] build: replace ninjatool with ninja Paolo Bonzini 2020-10-14 16:38 ` Daniel P. Berrangé 2020-10-14 13:57 ` [PATCH 0/7] " Peter Maydell 2020-10-15 6:15 ` Howard Spoelstra 2020-10-15 6:34 ` Paolo Bonzini 2020-10-15 7:39 ` Howard Spoelstra 2020-10-15 9:44 ` Paolo Bonzini 2020-10-15 17:39 ` Volker Rümelin 2020-10-15 18:49 ` Mark Cave-Ayland 2020-10-15 21:41 ` Paolo Bonzini 2020-10-16 9:35 ` Mark Cave-Ayland 2020-10-15 23:07 ` Howard Spoelstra 2020-10-15 18:21 ` 罗勇刚(Yonggang Luo)
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.