All of lore.kernel.org
 help / color / mirror / Atom feed
* [XEN PATCH 00/57] Toolstack build system improvement, toward non-recursive makefiles
@ 2021-12-06 17:01 Anthony PERARD
  2021-12-06 17:01 ` [XEN PATCH 01/57] .gitignore: Non existing toolcore/include files Anthony PERARD
                   ` (56 more replies)
  0 siblings, 57 replies; 115+ messages in thread
From: Anthony PERARD @ 2021-12-06 17:01 UTC (permalink / raw)
  To: xen-devel
  Cc: Anthony PERARD, Anthony PERARD, David Scott, Ian Jackson,
	Jan Beulich, Christian Lindig, Daniel P. Smith,
	Stefano Stabellini, Samuel Thibault, Daniel De Graaf,
	Andrew Cooper, Elena Ufimtseva, Juergen Gross, George Dunlap,
	Roger Pau Monné,
	Julien Grall, Wei Liu

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v1

Hi everyone,

I've been looking at reworking the build system we have for the "tools/", and
transforming it to something that suit it better. There are a lot of
dependencies between different sub-directories so it would be nice if GNU make
could actually handle them. This is possible with "non-recursive makefiles".

With non-recursive makefiles, make will have to load/include all the makefiles
and thus will have complete overview of all the dependencies. This will allow
make to build the necessary targets in other directory, and we won't need to
build sub-directories one by one.

To help with this transformation, I've chosen to go with a recent project
called "subdirmk". It help to deal with the fact that all makefiles will share
the same namespace, it is hooked into autoconf, we can easily run `make` from
any subdirectory. Together "autoconf" and "subdirmk" will also help to get
closer to be able to do out-of-tree build of the tools, but I'm mainly looking
to have non-recursive makefile.

Link to the project:
    https://www.chiark.greenend.org.uk/ucgi/~ian/git/subdirmk.git/

But before getting to the main course, I've got quite a few cleanup and some
changes to the makefiles. I start the patch series with patches that remove old
left over stuff, then start reworking makefiles. They are some common changes like
removing the "build" targets in many places as "all" would be the more common
way to spell it and "all" is the default target anyway. They are other changes
related to the conversion to "subdirmk", I start to use the variable $(TARGETS)
in several makefiles, this variable will have a special meaning in subdirmk
which will build those target by default.

stubdom has some changes also as it won't be able to use tools/ Makefiles
as-is anymore.

As for the conversion to non-recursive makefile, with subdirmk, I have this WIP
branch, it contains some changes that I'm trying out, some notes, and the
conversion, one Makefile per commit. Cleanup are still needed, some makefile
not converted yet, but it's otherwise mostly done.

    https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v1-wip-extra

With that branch, you could tried something like:
    ./configure; cd tools/xl; make
and `xl` should be built as well as all the xen library needed.
Also, things like `make clean` or rebuild should be faster in the all tools/
directory.

Cheers,

Anthony PERARD (57):
  .gitignore: Non existing toolcore/include files.
  tools/configure.ac: Remove left over system_aio
  tools/Rules.mk: Remove $(CFLAGS.opic) from %.opic: %.S
  libs: Remove _paths.h from libs ctrl and util.
  libs: Remove -Iinclude from CFLAGS
  libs/libs.mk: Remove generic variable that already exist
  libs/guest: Drop spurious include in Makefile
  libs/light: Remove non-existing CFLAGS_libxl.o var
  libs/light: Remove -I.
  libs/stat: Remove duplicated CFLAGS from deps
  libs/store: Remove PKG_CONFIG_REMOVE
  tools: remove some unneeded subdir-distclean-* targets
  tools/firmware/hvmloader: remove "subdirs-*" prerequisite
  tools/flask/utils: remove unused variables/targets from Makefile
  tools/libacpi: cleanup Makefile, don't check for iasl binary
  tools/ocaml: Remove generation of _paths.h
  tools/xl: Remove unnecessary -I. from CFLAGS
  tools: Use config.h from autoconf instead of "buildmakevars2header"
  tools/configure.ac: Create ZLIB_LIBS and ZLIB_CFLAGS
  tools/Rules.mk: introduce FORCE target
  tools/include/xen-foreign: avoid to rely on default .SUFFIXES
  tools/console: have one Makefile per program/directory
  tools/debugger: Allow make to recurse into debugger/
  tools/debugger/gdbsx: Fix and cleanup makefiles
  tools/examples: cleanup Makefile
  tools/firmware/hvmloader: rework Makefile
  tools/fuzz/libelf: rework makefile
  tools/fuzz/x86_instruction_emulator: rework makefile
  tools/helper: Cleanup Makefile
  tools/hotplug: cleanup Makefiles
  tools/libfsimage: Cleanup makefiles
  tools/misc: rework Makefile
  tools/vchan: Collect targets in TARGETS
  tools/xcutils: rework Makefile
  tools/xenpaging: Rework makefile
  tools/xenstore: Cleanup makefile
  tools/xentop: rework makefile
  tools/xenstrace: rework Makefile
  libs: Remove both "libs" and "build" target
  libs: rename LDUSELIBS to LDLIBS and use it instead of APPEND_LDFLAGS
  libs: Remove need for *installlocal targets
  libs,tools/include: Clean "clean" targets
  libs: Rename $(SRCS-y) to $(OBJS-y)
  libs/guest: rename ELF_OBJS to LIBELF_OBJS
  libs/guest: rework CFLAGS
  libs/store: use of -iquote instead of -I
  libs/stat: Fix and rework python-bindings build
  libs/stat: Fix and rework perl-binding build
  libs/toolcore: don't install xentoolcore_internal.h anymore
  tools/Rules.mk: Cleanup %.pc rules
  .gitignore: Cleanup ignores of tools/libs/*/{headers.chk,*.pc}
  stubdom: only build libxen*.a from tools/libs/
  stubdom: introduce xenlibs.mk
  tools/libs: create Makefile.common to be used by stubdom build system
  tools/xenstore: introduce Makefile.common to be used by stubdom
  stubdom: build xenstore*-stubdom using new Makefile.common
  stubdom: xenlibs linkfarm, ignore non-regular files

 tools/configure.ac                           |  20 +-
 Config.mk                                    |   9 -
 stubdom/xenlibs.mk                           |  13 ++
 tools/Rules.mk                               |   9 +-
 tools/libfsimage/Rules.mk                    |  26 +--
 tools/libfsimage/common.mk                   |  11 +
 tools/libs/libs.mk                           |  36 +--
 .gitignore                                   |  44 +---
 config/Tools.mk.in                           |   3 +-
 configure                                    |  45 ++++
 docs/configure                               |  45 ++++
 m4/paths.m4                                  |   9 +
 stubdom/Makefile                             |  26 ++-
 tools/Makefile                               |  32 +--
 tools/config.h.in                            |  27 +++
 tools/configure                              |  67 +++++-
 tools/console/Makefile                       |  57 +----
 tools/console/client/Makefile                |  39 ++++
 tools/console/client/main.c                  |   1 -
 tools/console/daemon/Makefile                |  50 +++++
 tools/console/daemon/main.c                  |   1 -
 tools/debugger/Makefile                      |   8 +
 tools/debugger/gdbsx/Makefile                |  20 +-
 tools/debugger/gdbsx/gx/Makefile             |  15 +-
 tools/debugger/gdbsx/xg/Makefile             |  25 +--
 tools/examples/Makefile                      |  21 +-
 tools/firmware/Makefile                      |   3 -
 tools/firmware/hvmloader/Makefile            |  18 +-
 tools/flask/utils/Makefile                   |  16 +-
 tools/fuzz/libelf/Makefile                   |  21 +-
 tools/fuzz/x86_instruction_emulator/Makefile |  32 ++-
 tools/helpers/Makefile                       |  31 +--
 tools/helpers/init-xenstore-domain.c         |   1 -
 tools/hotplug/FreeBSD/Makefile               |  11 +-
 tools/hotplug/Linux/Makefile                 |  16 +-
 tools/hotplug/Linux/systemd/Makefile         |  16 +-
 tools/hotplug/NetBSD/Makefile                |   9 +-
 tools/hotplug/common/Makefile                |  16 +-
 tools/include/Makefile                       |   1 +
 tools/include/xen-foreign/Makefile           |   9 +-
 tools/libacpi/Makefile                       |  17 +-
 tools/libacpi/README                         |   6 -
 tools/libfsimage/common/Makefile             |  11 +-
 tools/libfsimage/ext2fs-lib/Makefile         |   9 -
 tools/libfsimage/ext2fs/Makefile             |   9 -
 tools/libfsimage/fat/Makefile                |   9 -
 tools/libfsimage/iso9660/Makefile            |  11 -
 tools/libfsimage/reiserfs/Makefile           |   9 -
 tools/libfsimage/ufs/Makefile                |   9 -
 tools/libfsimage/xfs/Makefile                |   9 -
 tools/libfsimage/zfs/Makefile                |   9 -
 tools/libs/call/Makefile                     |   7 +-
 tools/libs/call/Makefile.common              |   6 +
 tools/libs/ctrl/Makefile                     |  55 +----
 tools/libs/ctrl/Makefile.common              |  44 ++++
 tools/libs/ctrl/xc_private.h                 |   2 -
 tools/libs/devicemodel/Makefile              |   7 +-
 tools/libs/devicemodel/Makefile.common       |   6 +
 tools/libs/evtchn/Makefile                   |   7 +-
 tools/libs/evtchn/Makefile.common            |   6 +
 tools/libs/foreignmemory/Makefile            |   7 +-
 tools/libs/foreignmemory/Makefile.common     |   6 +
 tools/libs/gnttab/Makefile                   |   9 +-
 tools/libs/gnttab/Makefile.common            |   8 +
 tools/libs/guest/Makefile                    | 109 +--------
 tools/libs/guest/Makefile.common             |  84 +++++++
 tools/libs/guest/xg_dom_core.c               |   1 -
 tools/libs/guest/xg_offline_page.c           |   2 +-
 tools/libs/hypfs/Makefile                    |   4 +-
 tools/libs/light/Makefile                    | 222 +++++++++----------
 tools/libs/light/libxl_dom.c                 |   2 -
 tools/libs/light/libxl_internal.h            |   1 -
 tools/libs/light/libxl_utils.c               |   1 -
 tools/libs/stat/Makefile                     |  70 +++---
 tools/libs/stat/bindings/swig/perl/.empty    |   1 -
 tools/libs/stat/bindings/swig/python/.empty  |   1 -
 tools/libs/stat/xenstat_qmp.c                |   1 -
 tools/libs/store/Makefile                    |  20 +-
 tools/libs/toolcore/Makefile                 |  10 +-
 tools/libs/toolcore/Makefile.common          |   9 +
 tools/libs/toollog/Makefile                  |   3 +-
 tools/libs/toollog/Makefile.common           |   2 +
 tools/libs/util/Makefile                     |  27 +--
 tools/libs/vchan/Makefile                    |   9 +-
 tools/misc/Makefile                          |  16 +-
 tools/ocaml/xenstored/Makefile               |   7 -
 tools/ocaml/xenstored/systemd_stubs.c        |   2 -
 tools/vchan/Makefile                         |   6 +-
 tools/xcutils/Makefile                       |  15 +-
 tools/xenpaging/Makefile                     |  24 +-
 tools/xenstore/Makefile                      |  83 +++----
 tools/xenstore/Makefile.common               |  34 +++
 tools/xentop/Makefile                        |  19 +-
 tools/xentrace/Makefile                      |  11 +-
 tools/xl/Makefile                            |   9 +-
 tools/xl/xl.h                                |   1 -
 96 files changed, 953 insertions(+), 990 deletions(-)
 create mode 100644 stubdom/xenlibs.mk
 create mode 100644 tools/libfsimage/common.mk
 create mode 100644 tools/console/client/Makefile
 create mode 100644 tools/console/daemon/Makefile
 create mode 100644 tools/debugger/Makefile
 create mode 100644 tools/libs/call/Makefile.common
 create mode 100644 tools/libs/ctrl/Makefile.common
 create mode 100644 tools/libs/devicemodel/Makefile.common
 create mode 100644 tools/libs/evtchn/Makefile.common
 create mode 100644 tools/libs/foreignmemory/Makefile.common
 create mode 100644 tools/libs/gnttab/Makefile.common
 create mode 100644 tools/libs/guest/Makefile.common
 delete mode 100644 tools/libs/stat/bindings/swig/perl/.empty
 delete mode 100644 tools/libs/stat/bindings/swig/python/.empty
 create mode 100644 tools/libs/toolcore/Makefile.common
 create mode 100644 tools/libs/toollog/Makefile.common
 create mode 100644 tools/xenstore/Makefile.common

-- 
Anthony PERARD



^ permalink raw reply	[flat|nested] 115+ messages in thread

end of thread, other threads:[~2021-12-21 18:04 UTC | newest]

Thread overview: 115+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-12-06 17:01 [XEN PATCH 00/57] Toolstack build system improvement, toward non-recursive makefiles Anthony PERARD
2021-12-06 17:01 ` [XEN PATCH 01/57] .gitignore: Non existing toolcore/include files Anthony PERARD
2021-12-06 17:01 ` [XEN PATCH 02/57] tools/configure.ac: Remove left over system_aio Anthony PERARD
2021-12-06 17:01 ` [XEN PATCH 03/57] tools/Rules.mk: Remove $(CFLAGS.opic) from %.opic: %.S Anthony PERARD
2021-12-06 17:01 ` [XEN PATCH 04/57] libs: Remove _paths.h from libs ctrl and util Anthony PERARD
2021-12-07  7:43   ` Juergen Gross
2021-12-06 17:01 ` [XEN PATCH 05/57] libs: Remove -Iinclude from CFLAGS Anthony PERARD
2021-12-07  7:44   ` Juergen Gross
2021-12-06 17:01 ` [XEN PATCH 06/57] libs/libs.mk: Remove generic variable that already exist Anthony PERARD
2021-12-07  7:46   ` Juergen Gross
2021-12-06 17:01 ` [XEN PATCH 07/57] libs/guest: Drop spurious include in Makefile Anthony PERARD
2021-12-07  7:48   ` Juergen Gross
2021-12-06 17:01 ` [XEN PATCH 08/57] libs/light: Remove non-existing CFLAGS_libxl.o var Anthony PERARD
2021-12-07  7:49   ` Juergen Gross
2021-12-06 17:01 ` [XEN PATCH 09/57] libs/light: Remove -I Anthony PERARD
2021-12-07  7:59   ` Juergen Gross
2021-12-06 17:01 ` [XEN PATCH 10/57] libs/stat: Remove duplicated CFLAGS from deps Anthony PERARD
2021-12-07  8:01   ` Juergen Gross
2021-12-06 17:01 ` [XEN PATCH 11/57] libs/store: Remove PKG_CONFIG_REMOVE Anthony PERARD
2021-12-07  8:05   ` Juergen Gross
2021-12-06 17:01 ` [XEN PATCH 12/57] tools: remove some unneeded subdir-distclean-* targets Anthony PERARD
2021-12-06 17:01 ` [XEN PATCH 13/57] tools/firmware/hvmloader: remove "subdirs-*" prerequisite Anthony PERARD
2021-12-06 17:01 ` [XEN PATCH 14/57] tools/flask/utils: remove unused variables/targets from Makefile Anthony PERARD
2021-12-16 12:35   ` Daniel P. Smith
2021-12-16 12:36   ` Andrew Cooper
2021-12-21 17:00     ` Anthony PERARD
2021-12-06 17:01 ` [XEN PATCH 15/57] tools/libacpi: cleanup Makefile, don't check for iasl binary Anthony PERARD
2021-12-16 11:47   ` Andrew Cooper
2021-12-06 17:01 ` [XEN PATCH 16/57] tools/ocaml: Remove generation of _paths.h Anthony PERARD
2021-12-16 14:09   ` Andrew Cooper
2021-12-06 17:02 ` [XEN PATCH 17/57] tools/xl: Remove unnecessary -I. from CFLAGS Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 18/57] tools: Use config.h from autoconf instead of "buildmakevars2header" Anthony PERARD
2021-12-07 10:08   ` Juergen Gross
2021-12-07 10:49     ` Anthony PERARD
2021-12-07 10:53       ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 19/57] tools/configure.ac: Create ZLIB_LIBS and ZLIB_CFLAGS Anthony PERARD
2021-12-07 10:20   ` Juergen Gross
2021-12-16 17:16   ` Andrew Cooper
2021-12-21 17:17     ` Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 20/57] tools/Rules.mk: introduce FORCE target Anthony PERARD
2021-12-07 10:54   ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 21/57] tools/include/xen-foreign: avoid to rely on default .SUFFIXES Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 22/57] tools/console: have one Makefile per program/directory Anthony PERARD
2021-12-16 17:26   ` Andrew Cooper
2021-12-21 17:25     ` Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 23/57] tools/debugger: Allow make to recurse into debugger/ Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 24/57] tools/debugger/gdbsx: Fix and cleanup makefiles Anthony PERARD
2021-12-16 17:55   ` Andrew Cooper
2021-12-21 17:30     ` Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 25/57] tools/examples: cleanup Makefile Anthony PERARD
2021-12-16 17:57   ` Andrew Cooper
2021-12-21 17:38     ` Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 26/57] tools/firmware/hvmloader: rework Makefile Anthony PERARD
2021-12-16 18:03   ` Andrew Cooper
2021-12-21 17:45     ` Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 27/57] tools/fuzz/libelf: rework makefile Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 28/57] tools/fuzz/x86_instruction_emulator: " Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 29/57] tools/helper: Cleanup Makefile Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 30/57] tools/hotplug: cleanup Makefiles Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 31/57] tools/libfsimage: Cleanup makefiles Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 32/57] tools/misc: rework Makefile Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 33/57] tools/vchan: Collect targets in TARGETS Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 34/57] tools/xcutils: rework Makefile Anthony PERARD
2021-12-16 18:20   ` Andrew Cooper
2021-12-06 17:02 ` [XEN PATCH 35/57] tools/xenpaging: Rework makefile Anthony PERARD
2021-12-16 18:25   ` Andrew Cooper
2021-12-06 17:02 ` [XEN PATCH 36/57] tools/xenstore: Cleanup makefile Anthony PERARD
2021-12-07 11:21   ` Juergen Gross
2021-12-16 17:19   ` Andrew Cooper
2021-12-16 18:34   ` Andrew Cooper
2021-12-17  5:24     ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 37/57] tools/xentop: rework makefile Anthony PERARD
2021-12-16 18:36   ` Andrew Cooper
2021-12-06 17:02 ` [XEN PATCH 38/57] tools/xenstrace: rework Makefile Anthony PERARD
2021-12-16 18:38   ` Andrew Cooper
2021-12-06 17:02 ` [XEN PATCH 39/57] libs: Remove both "libs" and "build" target Anthony PERARD
2021-12-07 12:14   ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 40/57] libs: rename LDUSELIBS to LDLIBS and use it instead of APPEND_LDFLAGS Anthony PERARD
2021-12-07 12:17   ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 41/57] libs: Remove need for *installlocal targets Anthony PERARD
2021-12-07 13:20   ` Juergen Gross
2021-12-21 17:59     ` Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 42/57] libs,tools/include: Clean "clean" targets Anthony PERARD
2021-12-07 13:21   ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 43/57] libs: Rename $(SRCS-y) to $(OBJS-y) Anthony PERARD
2021-12-07 13:24   ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 44/57] libs/guest: rename ELF_OBJS to LIBELF_OBJS Anthony PERARD
2021-12-07 13:26   ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 45/57] libs/guest: rework CFLAGS Anthony PERARD
2021-12-07 13:29   ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 46/57] libs/store: use of -iquote instead of -I Anthony PERARD
2021-12-07 13:30   ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 47/57] libs/stat: Fix and rework python-bindings build Anthony PERARD
2021-12-07 14:45   ` Juergen Gross
2021-12-16 18:47   ` Andrew Cooper
2021-12-21 18:03     ` Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 48/57] libs/stat: Fix and rework perl-binding build Anthony PERARD
2021-12-07 14:46   ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 49/57] libs/toolcore: don't install xentoolcore_internal.h anymore Anthony PERARD
2021-12-07 14:47   ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 50/57] tools/Rules.mk: Cleanup %.pc rules Anthony PERARD
2021-12-16 18:51   ` Andrew Cooper
2021-12-06 17:02 ` [XEN PATCH 51/57] .gitignore: Cleanup ignores of tools/libs/*/{headers.chk,*.pc} Anthony PERARD
2021-12-06 17:02 ` [XEN PATCH 52/57] stubdom: only build libxen*.a from tools/libs/ Anthony PERARD
2021-12-11 23:47   ` Samuel Thibault
2021-12-06 17:02 ` [XEN PATCH 53/57] stubdom: introduce xenlibs.mk Anthony PERARD
2021-12-11 23:49   ` Samuel Thibault
2021-12-06 17:02 ` [XEN PATCH 54/57] tools/libs: create Makefile.common to be used by stubdom build system Anthony PERARD
2021-12-07 14:57   ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 55/57] tools/xenstore: introduce Makefile.common to be used by stubdom Anthony PERARD
2021-12-07 14:58   ` Juergen Gross
2021-12-06 17:02 ` [XEN PATCH 56/57] stubdom: build xenstore*-stubdom using new Makefile.common Anthony PERARD
2021-12-11 23:51   ` Samuel Thibault
2021-12-06 17:02 ` [XEN PATCH 57/57] stubdom: xenlibs linkfarm, ignore non-regular files Anthony PERARD
2021-12-11 23:53   ` Samuel Thibault

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.