[XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

[XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

[Anthony PERARD]

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v5

Changes in v5:
- rebased on staging
- added "tools: Rework linking options for ocaml binding libraries"

Changes in v4:
- several new patches
- some changes to other patches listed in their changelogs

Changes in v3:
- rebased
- several new patches, starting with 13/25 "tools/libs/util: cleanup Makefile"
- introducing macros to deal with linking with in-tree xen libraries
- Add -Werror to CFLAGS for all builds in tools/

Changes in v2:
- one new patch
- other changes described in patch notes

Hi everyone,

I've been looking at reworking the build system we have for the "tools/", and
transforming it to something that suit it better. There are a lot of
dependencies between different sub-directories so it would be nice if GNU make
could actually handle them. This is possible with "non-recursive makefiles".

With non-recursive makefiles, make will have to load/include all the makefiles
and thus will have complete overview of all the dependencies. This will allow
make to build the necessary targets in other directory, and we won't need to
build sub-directories one by one.

To help with this transformation, I've chosen to go with a recent project
called "subdirmk". It help to deal with the fact that all makefiles will share
the same namespace, it is hooked into autoconf, we can easily run `make` from
any subdirectory. Together "autoconf" and "subdirmk" will also help to get
closer to be able to do out-of-tree build of the tools, but I'm mainly looking
to have non-recursive makefile.

Link to the project:
    https://www.chiark.greenend.org.uk/ucgi/~ian/git/subdirmk.git/

But before getting to the main course, I've got quite a few cleanup and some
changes to the makefiles. I start the patch series with patches that remove old
left over stuff, then start reworking makefiles. They are some common changes like
removing the "build" targets in many places as "all" would be the more common
way to spell it and "all" is the default target anyway. They are other changes
related to the conversion to "subdirmk", I start to use the variable $(TARGETS)
in several makefiles, this variable will have a special meaning in subdirmk
which will build those target by default.

As for the conversion to non-recursive makefile, with subdirmk, I have this WIP
branch, it contains some changes that I'm trying out, some notes, and the
conversion, one Makefile per commit. Cleanup are still needed, some makefile
not converted yet, but it's otherwise mostly done.

    https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v1-wip-extra

With that branch, you could tried something like:
    ./configure; cd tools/xl; make
and `xl` should be built as well as all the xen library needed.
Also, things like `make clean` or rebuild should be faster in the all tools/
directory.

Cheers,

Anthony PERARD (17):
  tools/debugger/gdbsx: Fix and cleanup makefiles
  tools/xentrace: rework Makefile
  tools: Introduce $(xenlibs-ldflags, ) macro
  tools: Add -Werror by default to all tools/
  tools: Remove -Werror everywhere else
  tools/hotplug: Generate "hotplugpath.sh" with configure
  libs/light/gentypes.py: allow to generate headers in subdirectory
  git-checkout.sh: handle running git-checkout from a different
    directory
  libs: Avoid exposing -Wl,--version-script to other built library
  libs: Fix auto-generation of version-script for unstable libs
  tools/include: Rework Makefile
  libs/light: Rework acpi table build targets
  libs/light: Rework generation of include/_libxl_*.h
  libs/light: Rework targets prerequisites
  libs/light: Makefile cleanup
  tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go
  tools: Rework linking options for ocaml binding libraries

 tools/configure.ac                     |  2 +
 tools/console/client/Makefile          |  1 -
 tools/console/daemon/Makefile          |  1 -
 tools/debugger/gdbsx/Makefile          | 20 +++----
 tools/debugger/gdbsx/gx/Makefile       | 15 +++---
 tools/debugger/gdbsx/xg/Makefile       | 25 +++------
 tools/debugger/kdd/Makefile            |  1 -
 tools/flask/utils/Makefile             |  1 -
 tools/fuzz/cpu-policy/Makefile         |  2 +-
 tools/golang/xenlight/Makefile         |  8 +--
 tools/hotplug/common/Makefile          | 10 +---
 tools/include/Makefile                 | 28 +++++-----
 tools/libs/call/Makefile               |  1 +
 tools/libs/ctrl/Makefile               |  3 --
 tools/libs/devicemodel/Makefile        |  1 +
 tools/libs/evtchn/Makefile             |  1 +
 tools/libs/foreignmemory/Makefile      |  1 +
 tools/libs/gnttab/Makefile             |  1 +
 tools/libs/guest/Makefile              |  3 --
 tools/libs/hypfs/Makefile              |  1 +
 tools/libs/light/Makefile              | 72 +++++++++++++++-----------
 tools/libs/stat/Makefile               |  2 +-
 tools/libs/store/Makefile              |  1 +
 tools/libs/toolcore/Makefile           |  1 +
 tools/libs/toollog/Makefile            |  1 +
 tools/libs/util/Makefile               |  3 --
 tools/libs/vchan/Makefile              |  3 --
 tools/misc/Makefile                    |  1 -
 tools/ocaml/libs/eventchn/Makefile     |  2 +-
 tools/ocaml/libs/xc/Makefile           |  2 +-
 tools/ocaml/libs/xentoollog/Makefile   |  2 +-
 tools/ocaml/libs/xl/Makefile           |  2 +-
 tools/tests/cpu-policy/Makefile        |  2 +-
 tools/tests/depriv/Makefile            |  2 +-
 tools/tests/resource/Makefile          |  1 -
 tools/tests/tsx/Makefile               |  1 -
 tools/tests/xenstore/Makefile          |  1 -
 tools/xcutils/Makefile                 |  2 -
 tools/xenmon/Makefile                  |  1 -
 tools/xenpaging/Makefile               |  1 -
 tools/xenpmd/Makefile                  |  1 -
 tools/xentop/Makefile                  |  2 +-
 tools/xentrace/Makefile                | 29 ++++-------
 tools/xl/Makefile                      |  2 +-
 tools/Rules.mk                         | 20 +++++++
 tools/debugger/gdbsx/Rules.mk          |  2 +-
 tools/firmware/Rules.mk                |  2 -
 tools/libfsimage/common.mk             |  2 +-
 tools/libs/libs.mk                     | 21 +++++---
 tools/libs/light/libxl_x86_acpi.c      |  2 +-
 tools/ocaml/common.make                |  2 +-
 .gitignore                             |  6 ---
 config/Tools.mk.in                     |  1 +
 scripts/git-checkout.sh                |  4 +-
 tools/configure                        | 29 ++++++++++-
 tools/golang/xenlight/gengotypes.py    | 10 +++-
 tools/hotplug/common/hotplugpath.sh.in | 16 ++++++
 tools/libs/light/gentypes.py           |  9 ++--
 tools/xenstore/Makefile.common         |  1 -
 59 files changed, 218 insertions(+), 172 deletions(-)
 create mode 100644 tools/hotplug/common/hotplugpath.sh.in

-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 01/17] tools/debugger/gdbsx: Fix and cleanup makefiles

[Anthony PERARD]

gdbsx/:
  - Make use of subdir facility for the "clean" target.
  - No need to remove the *.a, they aren't in this dir.
  - Avoid calling "distclean" in subdirs as "distclean" targets do only
    call "clean", and the "clean" also runs "clean" in subdirs.
  - Avoid the need to make "gx_all.a" and "xg_all.a" in the "all"
    recipe by forcing make to check for update of "xg/xg_all.a" and
    "gx/gx_all.a" by having "FORCE" as prerequisite. Now, when making
    "gdbsx", make will recurse even when both *.a already exist.
  - List target in $(TARGETS).

gdbsx/*/:
  - Fix dependency on *.h.
  - Remove some dead code.
  - List targets in $(TARGETS).
  - Remove "build" target.
  - Cleanup "clean" targets.
  - remove comments about the choice of "ar" instead of "ld"
  - Use "$(AR)" instead of plain "ar".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
---

Notes:
    missing-ack: GDBSX DEBUGGER
    
    v2:
    - also replace plain "ar" by "$(AR)"

 tools/debugger/gdbsx/Makefile    | 20 ++++++++++----------
 tools/debugger/gdbsx/gx/Makefile | 15 +++++++--------
 tools/debugger/gdbsx/xg/Makefile | 25 +++++++------------------
 3 files changed, 24 insertions(+), 36 deletions(-)

diff --git a/tools/debugger/gdbsx/Makefile b/tools/debugger/gdbsx/Makefile
index 5571450a89..4aaf427c45 100644
--- a/tools/debugger/gdbsx/Makefile
+++ b/tools/debugger/gdbsx/Makefile
@@ -1,20 +1,20 @@
 XEN_ROOT = $(CURDIR)/../../..
 include ./Rules.mk
 
+SUBDIRS-y += gx
+SUBDIRS-y += xg
+
+TARGETS := gdbsx
+
 .PHONY: all
-all:
-	$(MAKE) -C gx
-	$(MAKE) -C xg
-	$(MAKE) gdbsx
+all: $(TARGETS)
 
 .PHONY: clean
-clean:
-	rm -f xg_all.a gx_all.a gdbsx
-	set -e; for d in xg gx; do $(MAKE) -C $$d clean; done
+clean: subdirs-clean
+	rm -f $(TARGETS)
 
 .PHONY: distclean
 distclean: clean
-	set -e; for d in xg gx; do $(MAKE) -C $$d distclean; done
 
 .PHONY: install
 install: all
@@ -28,7 +28,7 @@ uninstall:
 gdbsx: gx/gx_all.a xg/xg_all.a 
 	$(CC) $(LDFLAGS) -o $@ $^
 
-xg/xg_all.a:
+xg/xg_all.a: FORCE
 	$(MAKE) -C xg
-gx/gx_all.a:
+gx/gx_all.a: FORCE
 	$(MAKE) -C gx
diff --git a/tools/debugger/gdbsx/gx/Makefile b/tools/debugger/gdbsx/gx/Makefile
index 3b8467f799..e9859aea9c 100644
--- a/tools/debugger/gdbsx/gx/Makefile
+++ b/tools/debugger/gdbsx/gx/Makefile
@@ -2,21 +2,20 @@ XEN_ROOT = $(CURDIR)/../../../..
 include ../Rules.mk
 
 GX_OBJS := gx_comm.o gx_main.o gx_utils.o gx_local.o
-GX_HDRS := $(wildcard *.h)
+
+TARGETS := gx_all.a
 
 .PHONY: all
-all: gx_all.a
+all: $(TARGETS)
 
 .PHONY: clean
 clean:
-	rm -rf gx_all.a *.o .*.d
+	rm -f *.o $(TARGETS) $(DEPS_RM)
 
 .PHONY: distclean
 distclean: clean
 
-#%.o: %.c $(GX_HDRS) Makefile
-#	$(CC) -c $(CFLAGS) -o $@ $<
-
-gx_all.a: $(GX_OBJS) Makefile $(GX_HDRS)
-	ar cr $@ $(GX_OBJS)        # problem with ld using -m32 
+gx_all.a: $(GX_OBJS) Makefile
+	$(AR) cr $@ $(GX_OBJS)
 
+-include $(DEPS_INCLUDE)
diff --git a/tools/debugger/gdbsx/xg/Makefile b/tools/debugger/gdbsx/xg/Makefile
index acdcddf0d5..05325d6d81 100644
--- a/tools/debugger/gdbsx/xg/Makefile
+++ b/tools/debugger/gdbsx/xg/Makefile
@@ -1,35 +1,24 @@
 XEN_ROOT = $(CURDIR)/../../../..
 include ../Rules.mk
 
-XG_HDRS := xg_public.h 
 XG_OBJS := xg_main.o 
 
 CFLAGS += -D__XEN_TOOLS__
 CFLAGS += $(CFLAGS_xeninclude)
 
+TARGETS := xg_all.a
 
 .PHONY: all
-all: build
+all: $(TARGETS)
 
-.PHONY: build
-build: xg_all.a $(XG_HDRS) $(XG_OBJS) Makefile
-# build: mk-symlinks xg_all.a $(XG_HDRS) $(XG_OBJS) Makefile
-# build: mk-symlinks xg_all.a
-
-xg_all.a: $(XG_OBJS) Makefile $(XG_HDRS)
-	ar cr $@ $(XG_OBJS)    # problems using -m32 in ld 
-#	$(LD) -b elf32-i386 $(LDFLAGS) -r -o $@ $^
-#	$(CC) -m32 -c -o $@ $^
-
-# xg_main.o: xg_main.c Makefile $(XG_HDRS)
-#$(CC) -c $(CFLAGS) -o $@ $<
-
-# %.o: %.c $(XG_HDRS) Makefile  -- doesn't work as it won't overwrite Rules.mk
-#%.o: %.c       -- doesn't recompile when .c changed
+xg_all.a: $(XG_OBJS) Makefile
+	$(AR) cr $@ $(XG_OBJS)
 
 .PHONY: clean
 clean:
-	rm -rf xen xg_all.a $(XG_OBJS)  .*.d
+	rm -f $(TARGETS) $(XG_OBJS) $(DEPS_RM)
 
 .PHONY: distclean
 distclean: clean
+
+-include $(DEPS_INCLUDE)
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 02/17] tools/xentrace: rework Makefile

[Anthony PERARD]

Remove "build" targets.

Use "$(TARGETS)" to list binary to be built.

Cleanup "clean" rule.

Also drop conditional install of $(BIN) and $(LIBBIN) as those two
variables are now always populated.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    missing-ack: XENTRACE
    
    v4:
    - also use := for BIN, SBIN, LIBBIN, SCRIPTS
    
    v2:
    - fix typo in title
    - drop conditional install of $(BIN) and $(LIBBIN)

 tools/xentrace/Makefile | 27 +++++++++++----------------
 1 file changed, 11 insertions(+), 16 deletions(-)

diff --git a/tools/xentrace/Makefile b/tools/xentrace/Makefile
index 9fb7fc96e7..63f2f6532d 100644
--- a/tools/xentrace/Makefile
+++ b/tools/xentrace/Makefile
@@ -9,41 +9,36 @@ LDLIBS += $(LDLIBS_libxenevtchn)
 LDLIBS += $(LDLIBS_libxenctrl)
 LDLIBS += $(ARGP_LDFLAGS)
 
-BIN      = xenalyze
-SBIN     = xentrace xentrace_setsize
-LIBBIN   = xenctx
-SCRIPTS  = xentrace_format
+BIN     := xenalyze
+SBIN    := xentrace xentrace_setsize
+LIBBIN  := xenctx
+SCRIPTS := xentrace_format
 
-.PHONY: all
-all: build
+TARGETS := $(BIN) $(SBIN) $(LIBBIN)
 
-.PHONY: build
-build: $(BIN) $(SBIN) $(LIBBIN)
+.PHONY: all
+all: $(TARGETS)
 
 .PHONY: install
-install: build
+install: all
 	$(INSTALL_DIR) $(DESTDIR)$(bindir)
 	$(INSTALL_DIR) $(DESTDIR)$(sbindir)
-	[ -z "$(LIBBIN)" ] || $(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
-ifneq ($(BIN),)
+	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
 	$(INSTALL_PROG) $(BIN) $(DESTDIR)$(bindir)
-endif
 	$(INSTALL_PROG) $(SBIN) $(DESTDIR)$(sbindir)
 	$(INSTALL_PYTHON_PROG) $(SCRIPTS) $(DESTDIR)$(bindir)
-	[ -z "$(LIBBIN)" ] || $(INSTALL_PROG) $(LIBBIN) $(DESTDIR)$(LIBEXEC_BIN)
+	$(INSTALL_PROG) $(LIBBIN) $(DESTDIR)$(LIBEXEC_BIN)
 
 .PHONY: uninstall
 uninstall:
 	rm -f $(addprefix $(DESTDIR)$(LIBEXEC_BIN)/, $(LIBBIN))
 	rm -f $(addprefix $(DESTDIR)$(bindir)/, $(SCRIPTS))
 	rm -f $(addprefix $(DESTDIR)$(sbindir)/, $(SBIN))
-ifneq ($(BIN),)
 	rm -f $(addprefix $(DESTDIR)$(bindir)/, $(BIN))
-endif
 
 .PHONY: clean
 clean:
-	$(RM) *.a *.so *.o *.rpm $(BIN) $(SBIN) $(LIBBIN) $(DEPS_RM)
+	$(RM) *.o $(TARGETS) $(DEPS_RM)
 
 .PHONY: distclean
 distclean: clean
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 03/17] tools: Introduce $(xenlibs-ldflags, ) macro

[Anthony PERARD]

This avoid the need to open-coding the list of flags needed to link
with an in-tree Xen library when using -lxen*.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Henry Wang <Henry.Wang@arm.com>
---

Notes:
    missing-ack: GOLANG BINDINGS

 tools/golang/xenlight/Makefile | 2 +-
 tools/Rules.mk                 | 8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/tools/golang/xenlight/Makefile b/tools/golang/xenlight/Makefile
index 64671f246c..00e6d17f2b 100644
--- a/tools/golang/xenlight/Makefile
+++ b/tools/golang/xenlight/Makefile
@@ -27,7 +27,7 @@ GOXL_GEN_FILES = types.gen.go helpers.gen.go
 # so that it can find the actual library.
 .PHONY: build
 build: xenlight.go $(GOXL_GEN_FILES)
-	CGO_CFLAGS="$(CFLAGS_libxenlight) $(CFLAGS_libxentoollog) $(APPEND_CFLAGS)" CGO_LDFLAGS="$(LDLIBS_libxenlight) $(LDLIBS_libxentoollog) -L$(XEN_libxenlight) -L$(XEN_libxentoollog) $(APPEND_LDFLAGS)" $(GO) build -x
+	CGO_CFLAGS="$(CFLAGS_libxenlight) $(CFLAGS_libxentoollog) $(APPEND_CFLAGS)" CGO_LDFLAGS="$(call xenlibs-ldflags,light toollog) $(APPEND_LDFLAGS)" $(GO) build -x
 
 .PHONY: install
 install: build
diff --git a/tools/Rules.mk b/tools/Rules.mk
index ce77dd2eb1..26958b2948 100644
--- a/tools/Rules.mk
+++ b/tools/Rules.mk
@@ -105,6 +105,14 @@ define xenlibs-ldlibs
     $(foreach lib,$(1),$(xenlibs-ldlibs-$(lib)))
 endef
 
+# Provide needed flags for linking an in-tree Xen library by an external
+# project (or when it is necessary to link with "-lxen$(1)" instead of using
+# the full path to the library).
+define xenlibs-ldflags
+    $(call xenlibs-rpath,$(1)) \
+    $(foreach lib,$(1),-L$(XEN_ROOT)/tools/libs/$(lib))
+endef
+
 define LIB_defs
  FILENAME_$(1) ?= xen$(1)
  XEN_libxen$(1) = $$(XEN_ROOT)/tools/libs/$(1)
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 04/17] tools: Add -Werror by default to all tools/

[Anthony PERARD]

And provide an option to ./configure to disable it.

A follow-up patch will remove -Werror from every other Makefile in
tools/. ("tools: Remove -Werror everywhere else")

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
---
 tools/configure.ac |  1 +
 tools/Rules.mk     |  4 ++++
 config/Tools.mk.in |  1 +
 tools/configure    | 26 ++++++++++++++++++++++++++
 4 files changed, 32 insertions(+)

diff --git a/tools/configure.ac b/tools/configure.ac
index 32cbe6bd3c..09059bc569 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -81,6 +81,7 @@ m4_include([../m4/header.m4])
 AX_XEN_EXPAND_CONFIG()
 
 # Enable/disable options
+AX_ARG_DEFAULT_ENABLE([werror], [Build tools without -Werror])
 AX_ARG_DEFAULT_DISABLE([rpath], [Build tools with -Wl,-rpath,LIBDIR])
 AX_ARG_DEFAULT_DISABLE([githttp], [Download GIT repositories via HTTP])
 AX_ARG_DEFAULT_ENABLE([monitors], [Disable xenstat and xentop monitoring tools])
diff --git a/tools/Rules.mk b/tools/Rules.mk
index 26958b2948..a165dc4bda 100644
--- a/tools/Rules.mk
+++ b/tools/Rules.mk
@@ -133,6 +133,10 @@ endif
 
 CFLAGS_libxenlight += $(CFLAGS_libxenctrl)
 
+ifeq ($(CONFIG_WERROR),y)
+CFLAGS += -Werror
+endif
+
 ifeq ($(debug),y)
 # Use -Og if available, -O0 otherwise
 dbg_opt_level := $(call cc-option,$(CC),-Og,-O0)
diff --git a/config/Tools.mk.in b/config/Tools.mk.in
index 6c1a0a676f..d0d460f922 100644
--- a/config/Tools.mk.in
+++ b/config/Tools.mk.in
@@ -1,5 +1,6 @@
 -include $(XEN_ROOT)/config/Paths.mk
 
+CONFIG_WERROR       := @werror@
 CONFIG_RUMP         := @CONFIG_RUMP@
 ifeq ($(CONFIG_RUMP),y)
 XEN_OS              := NetBSDRump
diff --git a/tools/configure b/tools/configure
index 41deb7fb96..acd9a04c3b 100755
--- a/tools/configure
+++ b/tools/configure
@@ -716,6 +716,7 @@ ocamltools
 monitors
 githttp
 rpath
+werror
 DEBUG_DIR
 XEN_DUMP_DIR
 XEN_PAGING_DIR
@@ -805,6 +806,7 @@ with_xen_scriptdir
 with_xen_dumpdir
 with_rundir
 with_debugdir
+enable_werror
 enable_rpath
 enable_githttp
 enable_monitors
@@ -1490,6 +1492,7 @@ Optional Features:
   --disable-FEATURE       do not include FEATURE (same as --enable-FEATURE=no)
   --enable-FEATURE[=ARG]  include FEATURE [ARG=yes]
   --disable-largefile     omit support for large files
+  --disable-werror        Build tools without -Werror (default is ENABLED)
   --enable-rpath          Build tools with -Wl,-rpath,LIBDIR (default is
                           DISABLED)
   --enable-githttp        Download GIT repositories via HTTP (default is
@@ -4111,6 +4114,29 @@ DEBUG_DIR=$debugdir_path
 
 # Enable/disable options
 
+# Check whether --enable-werror was given.
+if test "${enable_werror+set}" = set; then :
+  enableval=$enable_werror;
+fi
+
+
+if test "x$enable_werror" = "xno"; then :
+
+    ax_cv_werror="n"
+
+elif test "x$enable_werror" = "xyes"; then :
+
+    ax_cv_werror="y"
+
+elif test -z $ax_cv_werror; then :
+
+    ax_cv_werror="y"
+
+fi
+werror=$ax_cv_werror
+
+
+
 # Check whether --enable-rpath was given.
 if test "${enable_rpath+set}" = set; then :
   enableval=$enable_rpath;
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 05/17] tools: Remove -Werror everywhere else

[Anthony PERARD]

Patch "tools: Add -Werror by default to all tools/" have added
"-Werror" to CFLAGS in tools/Rules.mk, remove it from every other
makefiles as it is now duplicated.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Christian Lindig <christian.lindig@citrix.com>
Acked-by: Daniel P. Smith <dpsmith@apertussolutions.com> # tools/flask
Acked-by: Jan Beulich <jbeulich@suse.com> # tools/*/cpu-policy
---

Notes:
    missing-ack:
    - GDBSX DEBUGGER
    - KDD DEBUGGER
    - XENTRACE

 tools/console/client/Makefile   | 1 -
 tools/console/daemon/Makefile   | 1 -
 tools/debugger/kdd/Makefile     | 1 -
 tools/flask/utils/Makefile      | 1 -
 tools/fuzz/cpu-policy/Makefile  | 2 +-
 tools/misc/Makefile             | 1 -
 tools/tests/cpu-policy/Makefile | 2 +-
 tools/tests/depriv/Makefile     | 2 +-
 tools/tests/resource/Makefile   | 1 -
 tools/tests/tsx/Makefile        | 1 -
 tools/tests/xenstore/Makefile   | 1 -
 tools/xcutils/Makefile          | 2 --
 tools/xenmon/Makefile           | 1 -
 tools/xenpaging/Makefile        | 1 -
 tools/xenpmd/Makefile           | 1 -
 tools/xentop/Makefile           | 2 +-
 tools/xentrace/Makefile         | 2 --
 tools/xl/Makefile               | 2 +-
 tools/debugger/gdbsx/Rules.mk   | 2 +-
 tools/firmware/Rules.mk         | 2 --
 tools/libfsimage/common.mk      | 2 +-
 tools/libs/libs.mk              | 2 +-
 tools/ocaml/common.make         | 2 +-
 tools/xenstore/Makefile.common  | 1 -
 24 files changed, 9 insertions(+), 27 deletions(-)

diff --git a/tools/console/client/Makefile b/tools/console/client/Makefile
index e2f2554f92..62d89fdeb9 100644
--- a/tools/console/client/Makefile
+++ b/tools/console/client/Makefile
@@ -1,7 +1,6 @@
 XEN_ROOT=$(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror
 CFLAGS += $(CFLAGS_libxenctrl)
 CFLAGS += $(CFLAGS_libxenstore)
 CFLAGS += -include $(XEN_ROOT)/tools/config.h
diff --git a/tools/console/daemon/Makefile b/tools/console/daemon/Makefile
index 99bb33b6a2..9fc3b6711f 100644
--- a/tools/console/daemon/Makefile
+++ b/tools/console/daemon/Makefile
@@ -1,7 +1,6 @@
 XEN_ROOT=$(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror
 CFLAGS += $(CFLAGS_libxenctrl)
 CFLAGS += $(CFLAGS_libxenstore)
 CFLAGS += $(CFLAGS_libxenevtchn)
diff --git a/tools/debugger/kdd/Makefile b/tools/debugger/kdd/Makefile
index 26116949d4..a72ad3b1e0 100644
--- a/tools/debugger/kdd/Makefile
+++ b/tools/debugger/kdd/Makefile
@@ -1,7 +1,6 @@
 XEN_ROOT = $(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS  += -Werror
 CFLAGS  += $(CFLAGS_libxenctrl)
 CFLAGS  += -DXC_WANT_COMPAT_MAP_FOREIGN_API
 LDLIBS  += $(LDLIBS_libxenctrl)
diff --git a/tools/flask/utils/Makefile b/tools/flask/utils/Makefile
index 6be134142a..88d7edb6b1 100644
--- a/tools/flask/utils/Makefile
+++ b/tools/flask/utils/Makefile
@@ -1,7 +1,6 @@
 XEN_ROOT=$(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror
 CFLAGS += $(CFLAGS_libxenctrl)
 
 TARGETS := flask-loadpolicy flask-setenforce flask-getenforce flask-label-pci flask-get-bool flask-set-bool
diff --git a/tools/fuzz/cpu-policy/Makefile b/tools/fuzz/cpu-policy/Makefile
index 41a2230408..6e7743e0aa 100644
--- a/tools/fuzz/cpu-policy/Makefile
+++ b/tools/fuzz/cpu-policy/Makefile
@@ -17,7 +17,7 @@ install: all
 
 .PHONY: uninstall
 
-CFLAGS += -Werror $(CFLAGS_xeninclude) -D__XEN_TOOLS__
+CFLAGS += $(CFLAGS_xeninclude) -D__XEN_TOOLS__
 CFLAGS += $(APPEND_CFLAGS) -Og
 
 vpath %.c ../../../xen/lib/x86
diff --git a/tools/misc/Makefile b/tools/misc/Makefile
index 0e02401227..1c6e1d6a04 100644
--- a/tools/misc/Makefile
+++ b/tools/misc/Makefile
@@ -1,7 +1,6 @@
 XEN_ROOT=$(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror
 # Include configure output (config.h)
 CFLAGS += -include $(XEN_ROOT)/tools/config.h
 CFLAGS += $(CFLAGS_libxenevtchn)
diff --git a/tools/tests/cpu-policy/Makefile b/tools/tests/cpu-policy/Makefile
index 93af9d76fa..c5b81afc71 100644
--- a/tools/tests/cpu-policy/Makefile
+++ b/tools/tests/cpu-policy/Makefile
@@ -36,7 +36,7 @@ install: all
 uninstall:
 	$(RM) -- $(addprefix $(DESTDIR)$(LIBEXEC_BIN)/,$(TARGETS))
 
-CFLAGS += -Werror -D__XEN_TOOLS__
+CFLAGS += -D__XEN_TOOLS__
 CFLAGS += $(CFLAGS_xeninclude)
 CFLAGS += $(APPEND_CFLAGS)
 
diff --git a/tools/tests/depriv/Makefile b/tools/tests/depriv/Makefile
index 3cba28da25..7d9e3b01bb 100644
--- a/tools/tests/depriv/Makefile
+++ b/tools/tests/depriv/Makefile
@@ -1,7 +1,7 @@
 XEN_ROOT=$(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror -Wno-declaration-after-statement
+CFLAGS += -Wno-declaration-after-statement
 
 CFLAGS += $(CFLAGS_xeninclude)
 CFLAGS += $(CFLAGS_libxenctrl)
diff --git a/tools/tests/resource/Makefile b/tools/tests/resource/Makefile
index b3cd70c06d..a5856bf095 100644
--- a/tools/tests/resource/Makefile
+++ b/tools/tests/resource/Makefile
@@ -27,7 +27,6 @@ install: all
 uninstall:
 	$(RM) -- $(DESTDIR)$(LIBEXEC_BIN)/$(TARGET)
 
-CFLAGS += -Werror
 CFLAGS += $(CFLAGS_xeninclude)
 CFLAGS += $(CFLAGS_libxenctrl)
 CFLAGS += $(CFLAGS_libxenforeginmemory)
diff --git a/tools/tests/tsx/Makefile b/tools/tests/tsx/Makefile
index d7d2a5d95e..a4f516b725 100644
--- a/tools/tests/tsx/Makefile
+++ b/tools/tests/tsx/Makefile
@@ -26,7 +26,6 @@ uninstall:
 .PHONY: uninstall
 uninstall:
 
-CFLAGS += -Werror
 CFLAGS += -I$(XEN_ROOT)/tools/libs/ctrl -I$(XEN_ROOT)/tools/libs/guest
 CFLAGS += $(CFLAGS_xeninclude)
 CFLAGS += $(CFLAGS_libxenctrl)
diff --git a/tools/tests/xenstore/Makefile b/tools/tests/xenstore/Makefile
index 239e1dce47..202dda0d3c 100644
--- a/tools/tests/xenstore/Makefile
+++ b/tools/tests/xenstore/Makefile
@@ -27,7 +27,6 @@ install: all
 uninstall:
 	$(RM) -- $(addprefix $(DESTDIR)$(LIBEXEC_BIN)/,$(TARGETS))
 
-CFLAGS += -Werror
 CFLAGS += $(CFLAGS_libxenstore)
 CFLAGS += $(APPEND_CFLAGS)
 
diff --git a/tools/xcutils/Makefile b/tools/xcutils/Makefile
index e40a2c4bfa..3687f6cd8f 100644
--- a/tools/xcutils/Makefile
+++ b/tools/xcutils/Makefile
@@ -13,8 +13,6 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 TARGETS := readnotes lsevtchn
 
-CFLAGS += -Werror
-
 CFLAGS_readnotes.o  := $(CFLAGS_libxenevtchn) $(CFLAGS_libxenctrl) $(CFLAGS_libxenguest)
 CFLAGS_lsevtchn.o   := $(CFLAGS_libxenevtchn) $(CFLAGS_libxenctrl)
 
diff --git a/tools/xenmon/Makefile b/tools/xenmon/Makefile
index 3e150b0659..679c4b41a3 100644
--- a/tools/xenmon/Makefile
+++ b/tools/xenmon/Makefile
@@ -13,7 +13,6 @@
 XEN_ROOT=$(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS  += -Werror
 CFLAGS  += $(CFLAGS_libxenevtchn)
 CFLAGS  += $(CFLAGS_libxenctrl)
 LDLIBS  += $(LDLIBS_libxenctrl)
diff --git a/tools/xenpaging/Makefile b/tools/xenpaging/Makefile
index e2ed9eaa3f..835cf2b965 100644
--- a/tools/xenpaging/Makefile
+++ b/tools/xenpaging/Makefile
@@ -12,7 +12,6 @@ OBJS-y   += xenpaging.o
 OBJS-y   += policy_$(POLICY).o
 OBJS-y   += pagein.o
 
-CFLAGS   += -Werror
 CFLAGS   += -Wno-unused
 
 TARGETS := xenpaging
diff --git a/tools/xenpmd/Makefile b/tools/xenpmd/Makefile
index e0d3f06ab2..8da20510b5 100644
--- a/tools/xenpmd/Makefile
+++ b/tools/xenpmd/Makefile
@@ -1,7 +1,6 @@
 XEN_ROOT=$(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror
 CFLAGS += $(CFLAGS_libxenstore)
 
 LDLIBS += $(LDLIBS_libxenstore)
diff --git a/tools/xentop/Makefile b/tools/xentop/Makefile
index 7bd96f34d5..70cc2211c5 100644
--- a/tools/xentop/Makefile
+++ b/tools/xentop/Makefile
@@ -13,7 +13,7 @@
 XEN_ROOT=$(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -DGCC_PRINTF -Werror $(CFLAGS_libxenstat)
+CFLAGS += -DGCC_PRINTF $(CFLAGS_libxenstat)
 LDLIBS += $(LDLIBS_libxenstat) $(CURSES_LIBS) $(TINFO_LIBS) $(SOCKET_LIBS) -lm
 CFLAGS += -DHOST_$(XEN_OS)
 
diff --git a/tools/xentrace/Makefile b/tools/xentrace/Makefile
index 63f2f6532d..d50d400472 100644
--- a/tools/xentrace/Makefile
+++ b/tools/xentrace/Makefile
@@ -1,8 +1,6 @@
 XEN_ROOT=$(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror
-
 CFLAGS += $(CFLAGS_libxenevtchn)
 CFLAGS += $(CFLAGS_libxenctrl)
 LDLIBS += $(LDLIBS_libxenevtchn)
diff --git a/tools/xl/Makefile b/tools/xl/Makefile
index b7f439121a..5f7aa5f46c 100644
--- a/tools/xl/Makefile
+++ b/tools/xl/Makefile
@@ -5,7 +5,7 @@
 XEN_ROOT = $(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS += -Werror -Wno-format-zero-length -Wmissing-declarations \
+CFLAGS += -Wno-format-zero-length -Wmissing-declarations \
 	-Wno-declaration-after-statement -Wformat-nonliteral
 CFLAGS += -fPIC
 
diff --git a/tools/debugger/gdbsx/Rules.mk b/tools/debugger/gdbsx/Rules.mk
index 920f1c87fb..0610db873b 100644
--- a/tools/debugger/gdbsx/Rules.mk
+++ b/tools/debugger/gdbsx/Rules.mk
@@ -1,6 +1,6 @@
 include $(XEN_ROOT)/tools/Rules.mk
 
-CFLAGS   += -Werror -Wmissing-prototypes 
+CFLAGS   += -Wmissing-prototypes 
 # (gcc 4.3x and later)   -Wconversion -Wno-sign-conversion
 
 CFLAGS-$(clang) += -Wno-ignored-attributes
diff --git a/tools/firmware/Rules.mk b/tools/firmware/Rules.mk
index 278cca01e4..d3482c9ec4 100644
--- a/tools/firmware/Rules.mk
+++ b/tools/firmware/Rules.mk
@@ -11,8 +11,6 @@ ifneq ($(debug),y)
 CFLAGS += -DNDEBUG
 endif
 
-CFLAGS += -Werror
-
 $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
 
 $(call cc-option-add,CFLAGS,CC,-fcf-protection=none)
diff --git a/tools/libfsimage/common.mk b/tools/libfsimage/common.mk
index 77bc957f27..4fc8c66795 100644
--- a/tools/libfsimage/common.mk
+++ b/tools/libfsimage/common.mk
@@ -2,7 +2,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 FSDIR := $(libdir)/xenfsimage
 CFLAGS += -Wno-unknown-pragmas -I$(XEN_ROOT)/tools/libfsimage/common/ -DFSIMAGE_FSDIR=\"$(FSDIR)\"
-CFLAGS += -Werror -D_GNU_SOURCE
+CFLAGS += -D_GNU_SOURCE
 LDFLAGS += -L../common/
 
 PIC_OBJS = $(patsubst %.c,%.opic,$(LIB_SRCS-y))
diff --git a/tools/libs/libs.mk b/tools/libs/libs.mk
index 2b8e7a6128..e47fb30ed4 100644
--- a/tools/libs/libs.mk
+++ b/tools/libs/libs.mk
@@ -14,7 +14,7 @@ MINOR ?= 0
 
 SHLIB_LDFLAGS += -Wl,--version-script=libxen$(LIBNAME).map
 
-CFLAGS   += -Werror -Wmissing-prototypes
+CFLAGS   += -Wmissing-prototypes
 CFLAGS   += $(CFLAGS_xeninclude)
 CFLAGS   += $(foreach lib, $(USELIBS_$(LIBNAME)), $(CFLAGS_libxen$(lib)))
 
diff --git a/tools/ocaml/common.make b/tools/ocaml/common.make
index d5478f626f..0c8a597d5b 100644
--- a/tools/ocaml/common.make
+++ b/tools/ocaml/common.make
@@ -9,7 +9,7 @@ OCAMLLEX ?= ocamllex
 OCAMLYACC ?= ocamlyacc
 OCAMLFIND ?= ocamlfind
 
-CFLAGS += -fPIC -Werror -I$(shell ocamlc -where)
+CFLAGS += -fPIC -I$(shell ocamlc -where)
 
 OCAMLOPTFLAG_G := $(shell $(OCAMLOPT) -h 2>&1 | sed -n 's/^  *\(-g\) .*/\1/p')
 OCAMLOPTFLAGS = $(OCAMLOPTFLAG_G) -ccopt "$(LDFLAGS)" -dtypes $(OCAMLINCLUDE) -cc $(CC) -w F -warn-error F
diff --git a/tools/xenstore/Makefile.common b/tools/xenstore/Makefile.common
index 21b78b0538..ddbac052ac 100644
--- a/tools/xenstore/Makefile.common
+++ b/tools/xenstore/Makefile.common
@@ -9,7 +9,6 @@ XENSTORED_OBJS-$(CONFIG_NetBSD) += xenstored_posix.o
 XENSTORED_OBJS-$(CONFIG_FreeBSD) += xenstored_posix.o
 XENSTORED_OBJS-$(CONFIG_MiniOS) += xenstored_minios.o
 
-CFLAGS += -Werror
 # Include configure output (config.h)
 CFLAGS += -include $(XEN_ROOT)/tools/config.h
 CFLAGS += -I./include
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 06/17] tools/hotplug: Generate "hotplugpath.sh" with configure

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v4:
    - new patch

 tools/configure.ac                     |  1 +
 tools/hotplug/common/Makefile          | 10 ++--------
 tools/configure                        |  3 ++-
 tools/hotplug/common/hotplugpath.sh.in | 16 ++++++++++++++++
 4 files changed, 21 insertions(+), 9 deletions(-)
 create mode 100644 tools/hotplug/common/hotplugpath.sh.in

diff --git a/tools/configure.ac b/tools/configure.ac
index 09059bc569..18e481d77e 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -7,6 +7,7 @@ AC_INIT([Xen Hypervisor Tools], m4_esyscmd([../version.sh ../xen/Makefile]),
 AC_CONFIG_SRCDIR([libs/light/libxl.c])
 AC_CONFIG_FILES([
 ../config/Tools.mk
+hotplug/common/hotplugpath.sh
 hotplug/FreeBSD/rc.d/xencommons
 hotplug/FreeBSD/rc.d/xendriverdomain
 hotplug/Linux/init.d/sysconfig.xencommons
diff --git a/tools/hotplug/common/Makefile b/tools/hotplug/common/Makefile
index e8a8dbea6c..62afe1019e 100644
--- a/tools/hotplug/common/Makefile
+++ b/tools/hotplug/common/Makefile
@@ -1,19 +1,14 @@
 XEN_ROOT = $(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-HOTPLUGPATH := hotplugpath.sh
-
 # OS-independent hotplug scripts go in this directory
 
 # Xen scripts to go there.
 XEN_SCRIPTS :=
-XEN_SCRIPT_DATA := $(HOTPLUGPATH)
-
-genpath-target = $(call buildmakevars2file,$(HOTPLUGPATH))
-$(eval $(genpath-target))
+XEN_SCRIPT_DATA := hotplugpath.sh
 
 .PHONY: all
-all: $(HOTPLUGPATH)
+all:
 
 .PHONY: install
 install: install-scripts
@@ -40,7 +35,6 @@ uninstall-scripts:
 
 .PHONY: clean
 clean:
-	rm -f $(HOTPLUGPATH)
 
 .PHONY: distclean
 distclean: clean
diff --git a/tools/configure b/tools/configure
index acd9a04c3b..6199823f5a 100755
--- a/tools/configure
+++ b/tools/configure
@@ -2456,7 +2456,7 @@ ac_compiler_gnu=$ac_cv_c_compiler_gnu
 
 
 
-ac_config_files="$ac_config_files ../config/Tools.mk hotplug/FreeBSD/rc.d/xencommons hotplug/FreeBSD/rc.d/xendriverdomain hotplug/Linux/init.d/sysconfig.xencommons hotplug/Linux/init.d/sysconfig.xendomains hotplug/Linux/init.d/xen-watchdog hotplug/Linux/init.d/xencommons hotplug/Linux/init.d/xendomains hotplug/Linux/init.d/xendriverdomain hotplug/Linux/launch-xenstore hotplug/Linux/vif-setup hotplug/Linux/xen-hotplug-common.sh hotplug/Linux/xendomains hotplug/NetBSD/rc.d/xencommons hotplug/NetBSD/rc.d/xendriverdomain ocaml/libs/xs/paths.ml ocaml/xenstored/paths.ml ocaml/xenstored/oxenstored.conf"
+ac_config_files="$ac_config_files ../config/Tools.mk hotplug/common/hotplugpath.sh hotplug/FreeBSD/rc.d/xencommons hotplug/FreeBSD/rc.d/xendriverdomain hotplug/Linux/init.d/sysconfig.xencommons hotplug/Linux/init.d/sysconfig.xendomains hotplug/Linux/init.d/xen-watchdog hotplug/Linux/init.d/xencommons hotplug/Linux/init.d/xendomains hotplug/Linux/init.d/xendriverdomain hotplug/Linux/launch-xenstore hotplug/Linux/vif-setup hotplug/Linux/xen-hotplug-common.sh hotplug/Linux/xendomains hotplug/NetBSD/rc.d/xencommons hotplug/NetBSD/rc.d/xendriverdomain ocaml/libs/xs/paths.ml ocaml/xenstored/paths.ml ocaml/xenstored/oxenstored.conf"
 
 ac_config_headers="$ac_config_headers config.h"
 
@@ -10947,6 +10947,7 @@ for ac_config_target in $ac_config_targets
 do
   case $ac_config_target in
     "../config/Tools.mk") CONFIG_FILES="$CONFIG_FILES ../config/Tools.mk" ;;
+    "hotplug/common/hotplugpath.sh") CONFIG_FILES="$CONFIG_FILES hotplug/common/hotplugpath.sh" ;;
     "hotplug/FreeBSD/rc.d/xencommons") CONFIG_FILES="$CONFIG_FILES hotplug/FreeBSD/rc.d/xencommons" ;;
     "hotplug/FreeBSD/rc.d/xendriverdomain") CONFIG_FILES="$CONFIG_FILES hotplug/FreeBSD/rc.d/xendriverdomain" ;;
     "hotplug/Linux/init.d/sysconfig.xencommons") CONFIG_FILES="$CONFIG_FILES hotplug/Linux/init.d/sysconfig.xencommons" ;;
diff --git a/tools/hotplug/common/hotplugpath.sh.in b/tools/hotplug/common/hotplugpath.sh.in
new file mode 100644
index 0000000000..1036b884b8
--- /dev/null
+++ b/tools/hotplug/common/hotplugpath.sh.in
@@ -0,0 +1,16 @@
+sbindir="@sbindir@"
+bindir="@bindir@"
+LIBEXEC="@LIBEXEC@"
+LIBEXEC_BIN="@LIBEXEC_BIN@"
+libdir="@libdir@"
+SHAREDIR="@SHAREDIR@"
+XENFIRMWAREDIR="@XENFIRMWAREDIR@"
+XEN_CONFIG_DIR="@XEN_CONFIG_DIR@"
+XEN_SCRIPT_DIR="@XEN_SCRIPT_DIR@"
+XEN_LOCK_DIR="@XEN_LOCK_DIR@"
+XEN_RUN_DIR="@XEN_RUN_DIR@"
+XEN_PAGING_DIR="@XEN_PAGING_DIR@"
+XEN_DUMP_DIR="@XEN_DUMP_DIR@"
+XEN_LOG_DIR="@XEN_LOG_DIR@"
+XEN_LIB_DIR="@XEN_LIB_DIR@"
+XEN_RUN_STORED="@XEN_RUN_STORED@"
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 07/17] libs/light/gentypes.py: allow to generate headers in subdirectory

[Anthony PERARD]

This doesn't matter yet but it will when for example the script will
be run from tools/ to generate files tools/libs/light/.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v4:
    - new patch

 tools/libs/light/gentypes.py | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/tools/libs/light/gentypes.py b/tools/libs/light/gentypes.py
index 9a45e45acc..3fe3873242 100644
--- a/tools/libs/light/gentypes.py
+++ b/tools/libs/light/gentypes.py
@@ -584,6 +584,9 @@ def libxl_C_enum_from_string(ty, str, e, indent = "    "):
         s = indent + s
     return s.replace("\n", "\n%s" % indent).rstrip(indent)
 
+def clean_header_define(header_path):
+    return header_path.split('/')[-1].upper().replace('.','_')
+
 
 if __name__ == '__main__':
     if len(sys.argv) != 6:
@@ -598,7 +601,7 @@ if __name__ == '__main__':
 
     f = open(header, "w")
 
-    header_define = header.upper().replace('.','_')
+    header_define = clean_header_define(header)
     f.write("""#ifndef %s
 #define %s
 
@@ -648,7 +651,7 @@ if __name__ == '__main__':
 
     f = open(header_json, "w")
 
-    header_json_define = header_json.upper().replace('.','_')
+    header_json_define = clean_header_define(header_json)
     f.write("""#ifndef %s
 #define %s
 
@@ -672,7 +675,7 @@ if __name__ == '__main__':
 
     f = open(header_private, "w")
 
-    header_private_define = header_private.upper().replace('.','_')
+    header_private_define = clean_header_define(header_private)
     f.write("""#ifndef %s
 #define %s
 
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 08/17] git-checkout.sh: handle running git-checkout from a different directory

[Anthony PERARD]

"$DIR" might not be a full path and it might not have `pwd` as ".."
directory. So use `cd -` to undo the first `cd` command.

Also, use `basename` to make a symbolic link with a relative path.

This doesn't matter yet but it will when for example the commands to
clone OVMF is been run from tools/ rather than tools/firmware/.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v4:
    - new patch

 scripts/git-checkout.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/git-checkout.sh b/scripts/git-checkout.sh
index 20ae31ff23..fd4425ac4e 100755
--- a/scripts/git-checkout.sh
+++ b/scripts/git-checkout.sh
@@ -19,9 +19,9 @@ if test \! -d $DIR-remote; then
 		cd $DIR-remote.tmp
 		$GIT branch -D dummy >/dev/null 2>&1 ||:
 		$GIT checkout -b dummy $TAG
-		cd ..
+		cd -
 	fi
 	mv $DIR-remote.tmp $DIR-remote
 fi
 rm -f $DIR
-ln -sf $DIR-remote $DIR
+ln -sf $(basename $DIR-remote) $DIR
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 09/17] libs: Avoid exposing -Wl,--version-script to other built library

[Anthony PERARD]

$(SHLIB_LDFLAGS) is used by more targets that the single targets that
except it (libxenfoo.so.X.Y). There is also some dynamic libraries in
stats/ that uses $(SHLIB_LDFLAGS) (even if those are never built), and
there's libxenlight_test.so which doesn't needs a version script.

Also, libxenlight_test.so might failed to build if the version script
doesn't exist yet.

For these reasons, avoid changing the generic $(SHLIB_LDFLAGS) flags,
and add the flag directly on the command line.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v4:
    - new patch

 tools/libs/libs.mk | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/tools/libs/libs.mk b/tools/libs/libs.mk
index e47fb30ed4..3eb91fc8f3 100644
--- a/tools/libs/libs.mk
+++ b/tools/libs/libs.mk
@@ -12,8 +12,6 @@ MAJOR := $(shell $(XEN_ROOT)/version.sh $(XEN_ROOT)/xen/Makefile)
 endif
 MINOR ?= 0
 
-SHLIB_LDFLAGS += -Wl,--version-script=libxen$(LIBNAME).map
-
 CFLAGS   += -Wmissing-prototypes
 CFLAGS   += $(CFLAGS_xeninclude)
 CFLAGS   += $(foreach lib, $(USELIBS_$(LIBNAME)), $(CFLAGS_libxen$(lib)))
@@ -85,7 +83,7 @@ lib$(LIB_FILE_NAME).so.$(MAJOR): lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR)
 	$(SYMLINK_SHLIB) $< $@
 
 lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR): $(PIC_OBJS) libxen$(LIBNAME).map
-	$(CC) $(LDFLAGS) $(PTHREAD_LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,lib$(LIB_FILE_NAME).so.$(MAJOR) $(SHLIB_LDFLAGS) -o $@ $(PIC_OBJS) $(LDLIBS) $(APPEND_LDFLAGS)
+	$(CC) $(LDFLAGS) $(PTHREAD_LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,lib$(LIB_FILE_NAME).so.$(MAJOR) -Wl,--version-script=libxen$(LIBNAME).map $(SHLIB_LDFLAGS) -o $@ $(PIC_OBJS) $(LDLIBS) $(APPEND_LDFLAGS)
 
 # If abi-dumper is available, write out the ABI analysis
 ifneq ($(ABI_DUMPER),)
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 10/17] libs: Fix auto-generation of version-script for unstable libs

[Anthony PERARD]

When there isn't a version-script for a shared library (like for
unstable libs), we create one based on the current Xen version. But
that version-script became out-of-date as soon as Xen's version
changes and make as no way to regenerate the version-script on
rebuild.

In order to allow regenerating the script, we'll have a different
filename. In order to check if the content is up-to-date, we'll always
generated it and compare.

But we also need to know if there's already an existing version script
or if we need to generate one, for that we introduce $(version-script)
to be used to point to the path of the existing script. (Guessing if a
version script exist for a stable library with for example $(wildcard)
won't work as a file will exist when building the library without this
patch.)

We don't need the version-script unless we are making the shared
library so it is removed from the "all" target.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Alternatively, we could remove the need for to generate a
version-script, but that would mean the exported symbols won't have a
version anymore (like before, a few versions of Xen ago). Or, we could
use --default-symver, but the symboled would be versionned with the
soname (e.g. "libxenctrl.so.4.16.0") instead of "VERS_4.16.0".
---

Notes:
    v4:
    - new patch

 tools/libs/call/Makefile          |  1 +
 tools/libs/ctrl/Makefile          |  3 ---
 tools/libs/devicemodel/Makefile   |  1 +
 tools/libs/evtchn/Makefile        |  1 +
 tools/libs/foreignmemory/Makefile |  1 +
 tools/libs/gnttab/Makefile        |  1 +
 tools/libs/guest/Makefile         |  3 ---
 tools/libs/hypfs/Makefile         |  1 +
 tools/libs/light/Makefile         |  1 -
 tools/libs/stat/Makefile          |  2 +-
 tools/libs/store/Makefile         |  1 +
 tools/libs/toolcore/Makefile      |  1 +
 tools/libs/toollog/Makefile       |  1 +
 tools/libs/util/Makefile          |  3 ---
 tools/libs/vchan/Makefile         |  3 ---
 tools/libs/libs.mk                | 19 +++++++++++++------
 .gitignore                        |  6 ------
 17 files changed, 23 insertions(+), 26 deletions(-)

diff --git a/tools/libs/call/Makefile b/tools/libs/call/Makefile
index 103f5ad360..56a964b517 100644
--- a/tools/libs/call/Makefile
+++ b/tools/libs/call/Makefile
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR    = 1
 MINOR    = 3
+version-script := libxencall.map
 
 include Makefile.common
 
diff --git a/tools/libs/ctrl/Makefile b/tools/libs/ctrl/Makefile
index 93442ab389..094e84b8d8 100644
--- a/tools/libs/ctrl/Makefile
+++ b/tools/libs/ctrl/Makefile
@@ -10,6 +10,3 @@ PKG_CONFIG_NAME := Xencontrol
 NO_HEADERS_CHK := y
 
 include $(XEN_ROOT)/tools/libs/libs.mk
-
-clean::
-	rm -f libxenctrl.map
diff --git a/tools/libs/devicemodel/Makefile b/tools/libs/devicemodel/Makefile
index b70dd774e4..20d1d112e7 100644
--- a/tools/libs/devicemodel/Makefile
+++ b/tools/libs/devicemodel/Makefile
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR    = 1
 MINOR    = 4
+version-script := libxendevicemodel.map
 
 include Makefile.common
 
diff --git a/tools/libs/evtchn/Makefile b/tools/libs/evtchn/Makefile
index 3dad3840c6..18cdaab89e 100644
--- a/tools/libs/evtchn/Makefile
+++ b/tools/libs/evtchn/Makefile
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR    = 1
 MINOR    = 2
+version-script := libxenevtchn.map
 
 include Makefile.common
 
diff --git a/tools/libs/foreignmemory/Makefile b/tools/libs/foreignmemory/Makefile
index b70dd774e4..81398e88b1 100644
--- a/tools/libs/foreignmemory/Makefile
+++ b/tools/libs/foreignmemory/Makefile
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR    = 1
 MINOR    = 4
+version-script := libxenforeignmemory.map
 
 include Makefile.common
 
diff --git a/tools/libs/gnttab/Makefile b/tools/libs/gnttab/Makefile
index 3dad3840c6..4528830bdc 100644
--- a/tools/libs/gnttab/Makefile
+++ b/tools/libs/gnttab/Makefile
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR    = 1
 MINOR    = 2
+version-script := libxengnttab.map
 
 include Makefile.common
 
diff --git a/tools/libs/guest/Makefile b/tools/libs/guest/Makefile
index 19d3ff2fdb..93338a9301 100644
--- a/tools/libs/guest/Makefile
+++ b/tools/libs/guest/Makefile
@@ -14,6 +14,3 @@ NO_HEADERS_CHK := y
 include $(XEN_ROOT)/tools/libs/libs.mk
 
 libxenguest.so.$(MAJOR).$(MINOR): LDLIBS += $(ZLIB_LIBS) -lz
-
-clean::
-	rm -f libxenguest.map
diff --git a/tools/libs/hypfs/Makefile b/tools/libs/hypfs/Makefile
index 630e1e6f3e..7fae5c750d 100644
--- a/tools/libs/hypfs/Makefile
+++ b/tools/libs/hypfs/Makefile
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR    = 1
 MINOR    = 0
+version-script := libxenhypfs.map
 
 LDLIBS += -lz
 
diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
index 13545654c2..1d5716dbba 100644
--- a/tools/libs/light/Makefile
+++ b/tools/libs/light/Makefile
@@ -252,6 +252,5 @@ clean::
 	$(RM) testidl.c.new testidl.c *.api-ok
 	$(RM) $(TEST_PROGS) libxenlight_test.so libxl_test_*.opic
 	$(RM) -r __pycache__
-	$(RM) libxenlight.map
 	$(RM) $(AUTOSRCS) $(AUTOINCS)
 	$(MAKE) -C $(ACPI_PATH) ACPI_BUILD_DIR=$(CURDIR) clean
diff --git a/tools/libs/stat/Makefile b/tools/libs/stat/Makefile
index 7eaf50e91e..ee5c42bf7b 100644
--- a/tools/libs/stat/Makefile
+++ b/tools/libs/stat/Makefile
@@ -134,4 +134,4 @@ uninstall:: uninstall-perl-bindings
 endif
 
 clean::
-	$(RM) libxenstat.map $(BINDINGS) $(BINDINGSRC)
+	$(RM) $(BINDINGS) $(BINDINGSRC)
diff --git a/tools/libs/store/Makefile b/tools/libs/store/Makefile
index 2334c953bb..300c271eb0 100644
--- a/tools/libs/store/Makefile
+++ b/tools/libs/store/Makefile
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR = 4
 MINOR = 0
+version-script := libxenstore.map
 
 ifeq ($(CONFIG_Linux),y)
 LDLIBS += -ldl
diff --git a/tools/libs/toolcore/Makefile b/tools/libs/toolcore/Makefile
index 0d92b68b3b..20671dadd0 100644
--- a/tools/libs/toolcore/Makefile
+++ b/tools/libs/toolcore/Makefile
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR	= 1
 MINOR	= 0
+version-script := libxentoolcore.map
 
 LIBHEADER := xentoolcore.h
 
diff --git a/tools/libs/toollog/Makefile b/tools/libs/toollog/Makefile
index 2361b8cbf1..d612227c85 100644
--- a/tools/libs/toollog/Makefile
+++ b/tools/libs/toollog/Makefile
@@ -3,6 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR	= 1
 MINOR	= 0
+version-script := libxentoollog.map
 
 include Makefile.common
 
diff --git a/tools/libs/util/Makefile b/tools/libs/util/Makefile
index 493d2e00be..e016baf888 100644
--- a/tools/libs/util/Makefile
+++ b/tools/libs/util/Makefile
@@ -47,6 +47,3 @@ $(OBJS-y) $(PIC_OBJS): $(AUTOINCS)
 %.c %.h:: %.l
 	@rm -f $*.[ch]
 	$(FLEX) --header-file=$*.h --outfile=$*.c $<
-
-clean::
-	$(RM) libxenutil.map
diff --git a/tools/libs/vchan/Makefile b/tools/libs/vchan/Makefile
index ac2bff66f5..a1ef60ac8e 100644
--- a/tools/libs/vchan/Makefile
+++ b/tools/libs/vchan/Makefile
@@ -11,6 +11,3 @@ OBJS-y += io.o
 NO_HEADERS_CHK := y
 
 include $(XEN_ROOT)/tools/libs/libs.mk
-
-clean::
-	rm -f libxenvchan.map
diff --git a/tools/libs/libs.mk b/tools/libs/libs.mk
index 3eb91fc8f3..cab8e9704a 100644
--- a/tools/libs/libs.mk
+++ b/tools/libs/libs.mk
@@ -4,6 +4,8 @@
 #   PKG_CONFIG: name of pkg-config file (xen$(LIBNAME).pc if empty)
 #   MAJOR:   major version of lib (Xen version if empty)
 #   MINOR:   minor version of lib (0 if empty)
+#   version-script: Specify the name of a version script to the linker.
+#     (If empty, a temporary one for unstable library is created)
 
 LIBNAME := $(notdir $(CURDIR))
 
@@ -12,6 +14,10 @@ MAJOR := $(shell $(XEN_ROOT)/version.sh $(XEN_ROOT)/xen/Makefile)
 endif
 MINOR ?= 0
 
+ifeq ($(origin version-script), undefined)
+version-script := libxen$(LIBNAME).map.tmp
+endif
+
 CFLAGS   += -Wmissing-prototypes
 CFLAGS   += $(CFLAGS_xeninclude)
 CFLAGS   += $(foreach lib, $(USELIBS_$(LIBNAME)), $(CFLAGS_libxen$(lib)))
@@ -53,7 +59,7 @@ $(PKG_CONFIG_LOCAL): PKG_CONFIG_INCDIR = $(XEN_INCLUDE)
 $(PKG_CONFIG_LOCAL): PKG_CONFIG_LIBDIR = $(CURDIR)
 
 .PHONY: all
-all: $(TARGETS) $(PKG_CONFIG_LOCAL) libxen$(LIBNAME).map $(LIBHEADERS)
+all: $(TARGETS) $(PKG_CONFIG_LOCAL) $(LIBHEADERS)
 
 ifneq ($(NO_HEADERS_CHK),y)
 all: headers.chk
@@ -71,8 +77,9 @@ headers.lst: FORCE
 	@{ set -e; $(foreach h,$(LIBHEADERS),echo $(h);) } > $@.tmp
 	@$(call move-if-changed,$@.tmp,$@)
 
-libxen$(LIBNAME).map:
-	echo 'VERS_$(MAJOR).$(MINOR) { global: *; };' >$@
+libxen$(LIBNAME).map.tmp: FORCE
+	echo 'VERS_$(MAJOR).$(MINOR) { global: *; };' >.$@.tmp
+	$(call move-if-changed,.$@.tmp,$@)
 
 lib$(LIB_FILE_NAME).a: $(OBJS-y)
 	$(AR) rc $@ $^
@@ -82,8 +89,8 @@ lib$(LIB_FILE_NAME).so: lib$(LIB_FILE_NAME).so.$(MAJOR)
 lib$(LIB_FILE_NAME).so.$(MAJOR): lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR)
 	$(SYMLINK_SHLIB) $< $@
 
-lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR): $(PIC_OBJS) libxen$(LIBNAME).map
-	$(CC) $(LDFLAGS) $(PTHREAD_LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,lib$(LIB_FILE_NAME).so.$(MAJOR) -Wl,--version-script=libxen$(LIBNAME).map $(SHLIB_LDFLAGS) -o $@ $(PIC_OBJS) $(LDLIBS) $(APPEND_LDFLAGS)
+lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR): $(PIC_OBJS) $(version-script)
+	$(CC) $(LDFLAGS) $(PTHREAD_LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,lib$(LIB_FILE_NAME).so.$(MAJOR) -Wl,--version-script=$(version-script) $(SHLIB_LDFLAGS) -o $@ $(PIC_OBJS) $(LDLIBS) $(APPEND_LDFLAGS)
 
 # If abi-dumper is available, write out the ABI analysis
 ifneq ($(ABI_DUMPER),)
@@ -122,7 +129,7 @@ TAGS:
 clean::
 	rm -rf $(TARGETS) *~ $(DEPS_RM) $(OBJS-y) $(PIC_OBJS)
 	rm -f lib$(LIB_FILE_NAME).so.$(MAJOR).$(MINOR) lib$(LIB_FILE_NAME).so.$(MAJOR)
-	rm -f headers.chk headers.lst
+	rm -f headers.chk headers.lst libxen*.map.tmp
 
 .PHONY: distclean
 distclean: clean
diff --git a/.gitignore b/.gitignore
index 418bdfaebf..7febfc224d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -106,8 +106,6 @@ tools/config.cache
 config/Tools.mk
 config/Stubdom.mk
 config/Docs.mk
-tools/libs/ctrl/libxenctrl.map
-tools/libs/guest/libxenguest.map
 tools/libs/guest/xc_bitops.h
 tools/libs/guest/xc_core.h
 tools/libs/guest/xc_core_arm.h
@@ -117,7 +115,6 @@ tools/libs/light/_*.[ch]
 tools/libs/light/*.pyc
 tools/libs/light/_libxl.api-for-check
 tools/libs/light/*.api-ok
-tools/libs/light/libxenlight.map
 tools/libs/light/libxl-save-helper
 tools/libs/light/dsdt*
 tools/libs/light/mk_dsdt
@@ -127,13 +124,10 @@ tools/libs/light/testidl.c
 tools/libs/light/test_timedereg
 tools/libs/light/test_fdderegrace
 tools/libs/light/tmp.*
-tools/libs/stat/libxenstat.map
 tools/libs/store/list.h
 tools/libs/store/utils.h
 tools/libs/store/xs_lib.c
 tools/libs/util/libxlu_cfg_y.output
-tools/libs/util/libxenutil.map
-tools/libs/vchan/libxenvchan.map
 tools/debugger/gdb/gdb-6.2.1-linux-i386-xen/*
 tools/debugger/gdb/gdb-6.2.1/*
 tools/debugger/gdb/gdb-6.2.1.tar.bz2
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 11/17] tools/include: Rework Makefile

[Anthony PERARD]

Rework "xen-xsm" rules to not have to change directory to run
mkflask.sh, and store mkflask.sh path in a var, and use a full path
for FLASK_H_DEPEND, and output directory is made relative.

Rename "all-y" target to a more descriptive "xen/lib/x86/all".

Removed the "dist" target which was the only one existing in tools/.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v4:
    - new patch

 tools/include/Makefile | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)

diff --git a/tools/include/Makefile b/tools/include/Makefile
index b488f7ca9f..81c3d09039 100644
--- a/tools/include/Makefile
+++ b/tools/include/Makefile
@@ -7,17 +7,20 @@ include $(XEN_ROOT)/tools/Rules.mk
 # taken into account, i.e. there should be no rules added here for generating
 # any tools/include/*.h files.
 
-# Relative to $(XEN_ROOT)/xen/xsm/flask
-FLASK_H_DEPEND := policy/initial_sids
+.PHONY: all
+all: xen-foreign xen-dir xen-xsm/.dir
+ifeq ($(CONFIG_X86),y)
+all: xen/lib/x86/all
+endif
 
-.PHONY: all all-y build xen-dir
-all build: all-y xen-foreign xen-dir xen-xsm/.dir
-all-y:
+.PHONY: build
+build: all
 
 .PHONY: xen-foreign
 xen-foreign:
 	$(MAKE) -C xen-foreign
 
+.PHONY: xen-dir
 xen-dir:
 	mkdir -p xen/libelf acpi
 	find xen/ acpi/ -type l -exec rm '{}' +
@@ -36,16 +39,18 @@ ifeq ($(CONFIG_X86),y)
 	ln -s $(XEN_ROOT)/xen/include/xen/lib/x86/Makefile xen/lib/x86/
 endif
 
-all-$(CONFIG_X86): xen-dir
+.PHONY: xen/lib/x86/all
+xen/lib/x86/all: xen-dir
 	$(MAKE) -C xen/lib/x86 all XEN_ROOT=$(XEN_ROOT) PYTHON=$(PYTHON)
 
+MKFLASK := $(XEN_ROOT)/xen/xsm/flask/policy/mkflask.sh
+FLASK_H_DEPEND := $(XEN_ROOT)/xen/xsm/flask/policy/initial_sids
+
 # Not xen/xsm as that clashes with link to
 # $(XEN_ROOT)/xen/include/public/xsm above.
-xen-xsm/.dir: $(XEN_ROOT)/xen/xsm/flask/policy/mkflask.sh \
-	      $(patsubst %,$(XEN_ROOT)/xen/xsm/flask/%,$(FLASK_H_DEPEND))
+xen-xsm/.dir: $(MKFLASK) $(FLASK_H_DEPEND)
 	mkdir -p xen-xsm/flask
-	cd $(XEN_ROOT)/xen/xsm/flask/ && \
-		$(SHELL) policy/mkflask.sh $(AWK) $(CURDIR)/xen-xsm/flask $(FLASK_H_DEPEND)
+	$(SHELL) $(MKFLASK) $(AWK) xen-xsm/flask $(FLASK_H_DEPEND)
 	touch $@
 
 .PHONY: install
@@ -84,8 +89,5 @@ clean:
 	$(MAKE) -C xen-foreign clean
 	rm -f _*.h
 
-.PHONY: dist
-dist: install
-
 .PHONY: distclean
 distclean: clean
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 12/17] libs/light: Rework acpi table build targets

[Anthony PERARD]

Currently, a rebuild of libxl will always rebuild for example
"build.o". This is because the target depends on "acpi" which never
exist. So instead we will have "build.o" have as prerequisites targets
that are actually generated by "acpi", that is $(DSDT_FILES-y).

While "dsdt_*.c" isn't really a dependency for "build.o", a side
effect of building that dsdt_*.c is to also generate the "ssdt_*.h"
that "build.o" needs, but I don't want to list all the headers needed
by "build.o" and duplicate the information available in
"libacpi/Makefile" at this time.

Also avoid duplicating the "acpi" target for Arm, and unique one for
both architecture. And move the "acpi" target to be with other targets
rather than in the middle of the source listing. For the same reason,
move the prerequisites listing for both $(DSDT_FILES-y) and "build.o".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v4:
    - new patch

 tools/libs/light/Makefile | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
index 1d5716dbba..4a8b0f7bcf 100644
--- a/tools/libs/light/Makefile
+++ b/tools/libs/light/Makefile
@@ -32,14 +32,10 @@ ACPI_PATH  = $(XEN_ROOT)/tools/libacpi
 DSDT_FILES-$(CONFIG_X86) = dsdt_pvh.c
 ACPI_OBJS  = $(patsubst %.c,%.o,$(DSDT_FILES-y)) build.o static_tables.o
 ACPI_PIC_OBJS = $(patsubst %.o,%.opic,$(ACPI_OBJS))
-$(DSDT_FILES-y) build.o build.opic: acpi
+
 vpath build.c $(ACPI_PATH)/
 vpath static_tables.c $(ACPI_PATH)/
 
-.PHONY: acpi
-acpi:
-	$(MAKE) -C $(ACPI_PATH) ACPI_BUILD_DIR=$(CURDIR) DSDT_FILES="$(DSDT_FILES-y)"
-
 OBJS-$(CONFIG_X86) += $(ACPI_OBJS)
 
 CFLAGS += -Wno-format-zero-length -Wmissing-declarations \
@@ -58,8 +54,6 @@ ifeq ($(CONFIG_ARM_64),y)
 DSDT_FILES-y = dsdt_anycpu_arm.c
 OBJS-y += libxl_arm_acpi.o
 OBJS-y += $(DSDT_FILES-y:.c=.o)
-dsdt_anycpu_arm.c:
-	$(MAKE) -C $(ACPI_PATH) ACPI_BUILD_DIR=$(CURDIR) DSDT_FILES="$(DSDT_FILES-y)"
 else
 OBJS-$(CONFIG_ARM) += libxl_arm_no_acpi.o
 endif
@@ -191,6 +185,12 @@ all: $(CLIENTS) $(TEST_PROGS) $(AUTOSRCS) $(AUTOINCS)
 
 $(OBJS-y) $(PIC_OBJS) $(SAVE_HELPER_OBJS) $(LIBXL_TEST_OBJS) $(TEST_PROG_OBJS): $(AUTOINCS) libxl.api-ok
 
+$(DSDT_FILES-y): acpi
+
+# Depends on the source files generated by the "acpi" target even if "build.o"
+# don't needs them but do need the headers that are also generated by "acpi".
+build.o build.opic: $(DSDT_FILES-y)
+
 libxl.api-ok: check-libxl-api-rules _libxl.api-for-check
 	$(PERL) $^
 	touch $@
@@ -227,6 +227,10 @@ _libxl_type%.h _libxl_type%_json.h _libxl_type%_private.h _libxl_type%.c: libxl_
 $(XEN_INCLUDE)/_%.h: _%.h
 	$(call move-if-changed,_$*.h,$(XEN_INCLUDE)/_$*.h)
 
+.PHONY: acpi
+acpi:
+	$(MAKE) -C $(ACPI_PATH) ACPI_BUILD_DIR=$(CURDIR) DSDT_FILES="$(DSDT_FILES-y)"
+
 libxenlight_test.so: $(PIC_OBJS) $(LIBXL_TEST_OBJS)
 	$(CC) $(LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,libxenlight.so.$(MAJOR) $(SHLIB_LDFLAGS) -o $@ $^ $(LDLIBS) $(APPEND_LDFLAGS)
 
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 13/17] libs/light: Rework generation of include/_libxl_*.h

[Anthony PERARD]

Instead of moving the public "_libxl_*.h" headers, we make a copy to
the destination so that make doesn't try to remake the targets
"_libxl_*.h" in libs/light/ again.

A new .PRECIOUS target is added to tell make to not deletes the
intermediate targets generated by "gentypes.py".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v4:
    - new patch

 tools/libs/light/Makefile | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
index 4a8b0f7bcf..9329055c98 100644
--- a/tools/libs/light/Makefile
+++ b/tools/libs/light/Makefile
@@ -215,6 +215,8 @@ libxl_internal_json.h: _libxl_types_internal_json.h
 $(OBJS-y) $(PIC_OBJS) $(LIBXL_TEST_OBJS) $(TEST_PROG_OBJS) $(SAVE_HELPER_OBJS): $(XEN_INCLUDE)/libxl.h
 $(OBJS-y) $(PIC_OBJS) $(LIBXL_TEST_OBJS): libxl_internal.h
 
+# This exploits the 'multi-target pattern rule' trick.
+# gentypes.py should be executed only once to make all the targets.
 _libxl_type%.h _libxl_type%_json.h _libxl_type%_private.h _libxl_type%.c: libxl_type%.idl gentypes.py idl.py
 	$(eval stem = $(notdir $*))
 	$(PYTHON) gentypes.py libxl_type$(stem).idl __libxl_type$(stem).h __libxl_type$(stem)_private.h \
@@ -224,8 +226,10 @@ _libxl_type%.h _libxl_type%_json.h _libxl_type%_private.h _libxl_type%.c: libxl_
 	$(call move-if-changed,__libxl_type$(stem)_json.h,_libxl_type$(stem)_json.h)
 	$(call move-if-changed,__libxl_type$(stem).c,_libxl_type$(stem).c)
 
-$(XEN_INCLUDE)/_%.h: _%.h
-	$(call move-if-changed,_$*.h,$(XEN_INCLUDE)/_$*.h)
+.PRECIOUS: _libxl_type%.h _libxl_type%.c
+
+$(XEN_INCLUDE)/_libxl_%.h: _libxl_%.h
+	cp -f $< $@
 
 .PHONY: acpi
 acpi:
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 14/17] libs/light: Rework targets prerequisites

[Anthony PERARD]

No need for $(AUTOSRCS), GNU make can generate them as needed when
trying to build them as needed when trying to build the object. Also,
those two AUTOSRCS don't need to be a prerequisite of "all". As for
the "clean" target, those two files are already removed via "_*.c".

We don't need $(AUTOINCS) either:
- As for both _libxl_savm_msgs*.h headers, we are adding more
  selective dependencies so the headers will still be generated as
  needed.
- "clean" rule already delete the _*.h files, so AUTOINCS aren't needed
  there.

"libxl_internal_json.h" doesn't seems to have ever existed, so the
dependency is removed.

Add few prerequisite for "libxl_internal.h" so all headers that it
depends on should be generated. And have $(SAVE_HELPER_OBJS) depends
on "libxl_internal.h".

Rework objects prerequisites, to have them dependents on either
"libxl.h" or "libxl_internal.h". "libxl.h" is not normally included
directly in the source code as "libxl_internal.h" is used instead. But
we are adding "libxl.h" as prerequisite of "libxl_internal.h", so
generated headers will still be generated as needed.

"testidl.c" doesn't depends on "libxl.h" but "testidl.o" does. Also
use automatic variables $< and $@.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v4:
    - new patch

 tools/libs/light/Makefile | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)

diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
index 9329055c98..274e8350bb 100644
--- a/tools/libs/light/Makefile
+++ b/tools/libs/light/Makefile
@@ -147,9 +147,6 @@ LIBXL_TEST_OBJS += $(foreach t, $(LIBXL_TESTS_INSIDE),libxl_test_$t.opic)
 TEST_PROG_OBJS += $(foreach t, $(LIBXL_TESTS_PROGS),test_$t.o) test_common.o
 TEST_PROGS += $(foreach t, $(LIBXL_TESTS_PROGS),test_$t)
 
-AUTOINCS = _libxl_save_msgs_callout.h _libxl_save_msgs_helper.h
-AUTOSRCS = _libxl_save_msgs_callout.c _libxl_save_msgs_helper.c
-
 CLIENTS = testidl libxl-save-helper
 
 SAVE_HELPER_OBJS = libxl_save_helper.o _libxl_save_msgs_helper.o
@@ -177,13 +174,13 @@ libxl_x86_acpi.o libxl_x86_acpi.opic: CFLAGS += -I$(XEN_ROOT)/tools
 $(SAVE_HELPER_OBJS): CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenevtchn) $(CFLAGS_libxenguest)
 
 testidl.o: CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenlight)
-testidl.c: libxl_types.idl gentest.py $(XEN_INCLUDE)/libxl.h $(AUTOINCS)
-	$(PYTHON) gentest.py libxl_types.idl testidl.c.new
-	mv testidl.c.new testidl.c
+testidl.c: libxl_types.idl gentest.py
+	$(PYTHON) gentest.py $< $@.new
+	mv -f $@.new $@
 
-all: $(CLIENTS) $(TEST_PROGS) $(AUTOSRCS) $(AUTOINCS)
+all: $(CLIENTS) $(TEST_PROGS)
 
-$(OBJS-y) $(PIC_OBJS) $(SAVE_HELPER_OBJS) $(LIBXL_TEST_OBJS) $(TEST_PROG_OBJS): $(AUTOINCS) libxl.api-ok
+$(OBJS-y) $(PIC_OBJS) $(SAVE_HELPER_OBJS) $(LIBXL_TEST_OBJS) $(TEST_PROG_OBJS): libxl.api-ok
 
 $(DSDT_FILES-y): acpi
 
@@ -195,7 +192,7 @@ libxl.api-ok: check-libxl-api-rules _libxl.api-for-check
 	$(PERL) $^
 	touch $@
 
-_libxl.api-for-check: $(XEN_INCLUDE)/libxl.h $(AUTOINCS)
+_libxl.api-for-check: $(XEN_INCLUDE)/libxl.h
 	$(CC) $(CPPFLAGS) $(CFLAGS) -c -E $< $(APPEND_CFLAGS) \
 		-DLIBXL_EXTERNAL_CALLERS_ONLY=LIBXL_EXTERNAL_CALLERS_ONLY \
 		>$@.new
@@ -207,13 +204,22 @@ _libxl_save_msgs_helper.h _libxl_save_msgs_callout.h: \
 	$(PERL) -w $< $@ >$@.new
 	$(call move-if-changed,$@.new,$@)
 
+#
+# headers dependencies on generated headers
+#
 $(XEN_INCLUDE)/libxl.h: $(XEN_INCLUDE)/_libxl_types.h
 $(XEN_INCLUDE)/libxl_json.h: $(XEN_INCLUDE)/_libxl_types_json.h
+libxl_internal.h: $(XEN_INCLUDE)/libxl.h $(XEN_INCLUDE)/libxl_json.h
 libxl_internal.h: _libxl_types_internal.h _libxl_types_private.h _libxl_types_internal_private.h
-libxl_internal_json.h: _libxl_types_internal_json.h
+libxl_internal.h: _libxl_save_msgs_callout.h
 
-$(OBJS-y) $(PIC_OBJS) $(LIBXL_TEST_OBJS) $(TEST_PROG_OBJS) $(SAVE_HELPER_OBJS): $(XEN_INCLUDE)/libxl.h
+#
+# objects dependencies on headers that depends on generated headers
+#
+$(TEST_PROG_OBJS): $(XEN_INCLUDE)/libxl.h
 $(OBJS-y) $(PIC_OBJS) $(LIBXL_TEST_OBJS): libxl_internal.h
+$(SAVE_HELPER_OBJS): $(XEN_INCLUDE)/libxl.h _libxl_save_msgs_helper.h
+testidl.o: $(XEN_INCLUDE)/libxl.h
 
 # This exploits the 'multi-target pattern rule' trick.
 # gentypes.py should be executed only once to make all the targets.
@@ -260,5 +266,4 @@ clean::
 	$(RM) testidl.c.new testidl.c *.api-ok
 	$(RM) $(TEST_PROGS) libxenlight_test.so libxl_test_*.opic
 	$(RM) -r __pycache__
-	$(RM) $(AUTOSRCS) $(AUTOINCS)
 	$(MAKE) -C $(ACPI_PATH) ACPI_BUILD_DIR=$(CURDIR) clean
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 15/17] libs/light: Makefile cleanup

[Anthony PERARD]

Rework "libacpi.h" include in "libxl_x86_acpi.c" as to be more
selective about the include path and only add "tools/libacpi/". Also
"libxl_dom.c" don't use "libacpi.h" anymore. Use "-iquote" for libacpi
headers.

Get rid of the weird "$(eval stem =" in the middle of a recipe and use
a make automatic variable "$(*F)" instead.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v4:
    - new patch

 tools/libs/light/Makefile         | 16 +++++++---------
 tools/libs/light/libxl_x86_acpi.c |  2 +-
 2 files changed, 8 insertions(+), 10 deletions(-)

diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
index 274e8350bb..250cc3bd2c 100644
--- a/tools/libs/light/Makefile
+++ b/tools/libs/light/Makefile
@@ -169,8 +169,7 @@ LDLIBS += $(LDLIBS-y)
 $(OBJS-y) $(PIC_OBJS) $(LIBXL_TEST_OBJS): CFLAGS += $(CFLAGS_LIBXL) -include $(XEN_ROOT)/tools/config.h
 $(ACPI_OBJS) $(ACPI_PIC_OBJS): CFLAGS += -I. -DLIBACPI_STDUTILS=\"$(CURDIR)/libxl_x86_acpi.h\"
 $(TEST_PROG_OBJS) _libxl.api-for-check: CFLAGS += $(CFLAGS_libxentoollog) $(CFLAGS_libxentoolcore)
-libxl_dom.o libxl_dom.opic: CFLAGS += -I$(XEN_ROOT)/tools  # include libacpi/x86.h
-libxl_x86_acpi.o libxl_x86_acpi.opic: CFLAGS += -I$(XEN_ROOT)/tools
+libxl_x86_acpi.o libxl_x86_acpi.opic: CFLAGS += -iquote $(ACPI_PATH)
 $(SAVE_HELPER_OBJS): CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenevtchn) $(CFLAGS_libxenguest)
 
 testidl.o: CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenlight)
@@ -224,13 +223,12 @@ testidl.o: $(XEN_INCLUDE)/libxl.h
 # This exploits the 'multi-target pattern rule' trick.
 # gentypes.py should be executed only once to make all the targets.
 _libxl_type%.h _libxl_type%_json.h _libxl_type%_private.h _libxl_type%.c: libxl_type%.idl gentypes.py idl.py
-	$(eval stem = $(notdir $*))
-	$(PYTHON) gentypes.py libxl_type$(stem).idl __libxl_type$(stem).h __libxl_type$(stem)_private.h \
-		__libxl_type$(stem)_json.h  __libxl_type$(stem).c
-	$(call move-if-changed,__libxl_type$(stem).h,_libxl_type$(stem).h)
-	$(call move-if-changed,__libxl_type$(stem)_private.h,_libxl_type$(stem)_private.h)
-	$(call move-if-changed,__libxl_type$(stem)_json.h,_libxl_type$(stem)_json.h)
-	$(call move-if-changed,__libxl_type$(stem).c,_libxl_type$(stem).c)
+	$(PYTHON) gentypes.py libxl_type$(*F).idl __libxl_type$(*F).h __libxl_type$(*F)_private.h \
+		__libxl_type$(*F)_json.h  __libxl_type$(*F).c
+	$(call move-if-changed,__libxl_type$(*F).h,_libxl_type$(*F).h)
+	$(call move-if-changed,__libxl_type$(*F)_private.h,_libxl_type$(*F)_private.h)
+	$(call move-if-changed,__libxl_type$(*F)_json.h,_libxl_type$(*F)_json.h)
+	$(call move-if-changed,__libxl_type$(*F).c,_libxl_type$(*F).c)
 
 .PRECIOUS: _libxl_type%.h _libxl_type%.c
 
diff --git a/tools/libs/light/libxl_x86_acpi.c b/tools/libs/light/libxl_x86_acpi.c
index 57a6b63790..22eb160659 100644
--- a/tools/libs/light/libxl_x86_acpi.c
+++ b/tools/libs/light/libxl_x86_acpi.c
@@ -16,7 +16,7 @@
 #include "libxl_arch.h"
 #include <xen/hvm/hvm_info_table.h>
 #include <xen/hvm/e820.h>
-#include "libacpi/libacpi.h"
+#include "libacpi.h"
 
  /* Number of pages holding ACPI tables */
 #define NUM_ACPI_PAGES 16
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 16/17] tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

[Anthony PERARD]

gengotypes.py creates both "types.gen.go" and "helpers.gen.go", but
make can start gengotypes.py twice. Rework the rules so that
gengotypes.py is executed only once.

Also, add the ability to provide a path to tell gengotypes.py where to
put the files. This doesn't matter yet but it will when for example
the script will be run from tools/ to generate the targets.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Release-acked-by: Henry Wang <Henry.Wang@arm.com>
---

Notes:
    v5:
    - released-acked to fix occasional CI build issue
    
    v4:
    - new patch

 tools/golang/xenlight/Makefile      |  6 ++++--
 tools/golang/xenlight/gengotypes.py | 10 +++++++++-
 2 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/tools/golang/xenlight/Makefile b/tools/golang/xenlight/Makefile
index 00e6d17f2b..c5bb6b94a8 100644
--- a/tools/golang/xenlight/Makefile
+++ b/tools/golang/xenlight/Makefile
@@ -15,8 +15,10 @@ all: build
 
 GOXL_GEN_FILES = types.gen.go helpers.gen.go
 
-%.gen.go: gengotypes.py $(LIBXL_SRC_DIR)/libxl_types.idl $(LIBXL_SRC_DIR)/idl.py
-	LIBXL_SRC_DIR=$(LIBXL_SRC_DIR) $(PYTHON) gengotypes.py $(LIBXL_SRC_DIR)/libxl_types.idl
+# This exploits the 'multi-target pattern rule' trick.
+# gentypes.py should be executed only once to make all the targets.
+$(subst .gen.,.%.,$(GOXL_GEN_FILES)): gengotypes.py $(LIBXL_SRC_DIR)/libxl_types.idl $(LIBXL_SRC_DIR)/idl.py
+	LIBXL_SRC_DIR=$(LIBXL_SRC_DIR) $(PYTHON) gengotypes.py $(LIBXL_SRC_DIR)/libxl_types.idl $(@D)/types.gen.go $(@D)/helpers.gen.go
 
 # Go will do its own dependency checking, and not actuall go through
 # with the build if none of the input files have changed.
diff --git a/tools/golang/xenlight/gengotypes.py b/tools/golang/xenlight/gengotypes.py
index ac1cf060dd..ff4c2ad216 100644
--- a/tools/golang/xenlight/gengotypes.py
+++ b/tools/golang/xenlight/gengotypes.py
@@ -723,7 +723,13 @@ def xenlight_golang_fmt_name(name, exported = True):
     return words[0] + ''.join(x.title() for x in words[1:])
 
 if __name__ == '__main__':
+    if len(sys.argv) != 4:
+        print("Usage: gengotypes.py <idl> <types.gen.go> <helpers.gen.go>", file=sys.stderr)
+        sys.exit(1)
+
     idlname = sys.argv[1]
+    path_types = sys.argv[2]
+    path_helpers = sys.argv[3]
 
     (builtins, types) = idl.parse(idlname)
 
@@ -735,9 +741,11 @@ if __name__ == '__main__':
 // source: {}
 
 """.format(os.path.basename(sys.argv[0]),
-           ' '.join([os.path.basename(a) for a in sys.argv[1:]]))
+           os.path.basename(sys.argv[1]))
 
     xenlight_golang_generate_types(types=types,
+                                   path=path_types,
                                    comment=header_comment)
     xenlight_golang_generate_helpers(types=types,
+                                     path=path_helpers,
                                      comment=header_comment)
-- 
Anthony PERARD

[XEN PATCH for-4.17 v5 17/17] tools: Rework linking options for ocaml binding libraries

[Anthony PERARD]

Using a full path to the C libraries when preparing one of the ocaml
binding for those libraries make the binding unusable by external
project. The full path is somehow embedded and reused by the external
project when linking against the binding.

Instead, we will use the proper way to link a library, by using '-l'.
For in-tree build, we also need to provide the search directory via
'-L'.

(The search path -L are still be embedded, but at least that doesn't
prevent the ocaml binding from been used.)

Related-to: xen-project/xen#96
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    This patch only partially resolve xen-project/xen#96. At least with this
    patch, the ocaml binding can be used by external project, but those
    project will still "see" and try to use the rpath and lib search path as
    the -L and -W,-rpath-link are still embedded in the ocaml binding.
    
    I'm working on a more involve series of patch which would remove the
    unnecessary flags. (This would mean that oxenstored and the ocaml test
    will need those ldflags, instead of the binding.)
    
    v5:
    - new patch

 tools/ocaml/libs/eventchn/Makefile   | 2 +-
 tools/ocaml/libs/xc/Makefile         | 2 +-
 tools/ocaml/libs/xentoollog/Makefile | 2 +-
 tools/ocaml/libs/xl/Makefile         | 2 +-
 tools/Rules.mk                       | 8 ++++++++
 5 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/tools/ocaml/libs/eventchn/Makefile b/tools/ocaml/libs/eventchn/Makefile
index 7362a28d9e..dc560ba49b 100644
--- a/tools/ocaml/libs/eventchn/Makefile
+++ b/tools/ocaml/libs/eventchn/Makefile
@@ -8,7 +8,7 @@ OBJS = xeneventchn
 INTF = $(foreach obj, $(OBJS),$(obj).cmi)
 LIBS = xeneventchn.cma xeneventchn.cmxa
 
-LIBS_xeneventchn = $(LDLIBS_libxenevtchn)
+LIBS_xeneventchn = $(call xenlibs-ldflags-ldlibs,evtchn)
 
 all: $(INTF) $(LIBS) $(PROGRAMS)
 
diff --git a/tools/ocaml/libs/xc/Makefile b/tools/ocaml/libs/xc/Makefile
index 67acc46bee..3b76e9ad7b 100644
--- a/tools/ocaml/libs/xc/Makefile
+++ b/tools/ocaml/libs/xc/Makefile
@@ -10,7 +10,7 @@ OBJS = xenctrl
 INTF = xenctrl.cmi
 LIBS = xenctrl.cma xenctrl.cmxa
 
-LIBS_xenctrl = $(LDLIBS_libxenctrl) $(LDLIBS_libxenguest)
+LIBS_xenctrl = $(call xenlibs-ldflags-ldlibs,ctrl guest)
 
 xenctrl_OBJS = $(OBJS)
 xenctrl_C_OBJS = xenctrl_stubs
diff --git a/tools/ocaml/libs/xentoollog/Makefile b/tools/ocaml/libs/xentoollog/Makefile
index 9ede2fd124..1645b40faf 100644
--- a/tools/ocaml/libs/xentoollog/Makefile
+++ b/tools/ocaml/libs/xentoollog/Makefile
@@ -13,7 +13,7 @@ OBJS = xentoollog
 INTF = xentoollog.cmi
 LIBS = xentoollog.cma xentoollog.cmxa
 
-LIBS_xentoollog = $(LDLIBS_libxentoollog)
+LIBS_xentoollog = $(call xenlibs-ldflags-ldlibs,toollog)
 
 xentoollog_OBJS = $(OBJS)
 xentoollog_C_OBJS = xentoollog_stubs
diff --git a/tools/ocaml/libs/xl/Makefile b/tools/ocaml/libs/xl/Makefile
index 7c1c4edced..22d6c93aae 100644
--- a/tools/ocaml/libs/xl/Makefile
+++ b/tools/ocaml/libs/xl/Makefile
@@ -15,7 +15,7 @@ LIBS = xenlight.cma xenlight.cmxa
 
 OCAMLINCLUDE += -I ../xentoollog
 
-LIBS_xenlight = $(LDLIBS_libxenlight)
+LIBS_xenlight = $(call xenlibs-ldflags-ldlibs,light)
 
 xenlight_OBJS = $(OBJS)
 xenlight_C_OBJS = xenlight_stubs
diff --git a/tools/Rules.mk b/tools/Rules.mk
index a165dc4bda..34d495fff7 100644
--- a/tools/Rules.mk
+++ b/tools/Rules.mk
@@ -113,6 +113,14 @@ define xenlibs-ldflags
     $(foreach lib,$(1),-L$(XEN_ROOT)/tools/libs/$(lib))
 endef
 
+# Flags for linking against all Xen libraries listed in $(1) but by making use
+# of -L and -l instead of providing a path to the shared library.
+define xenlibs-ldflags-ldlibs
+    $(call xenlibs-ldflags,$(1)) \
+    $(foreach lib,$(1), -l$(FILENAME_$(lib))) \
+    $(foreach lib,$(1),$(xenlibs-ldlibs-$(lib)))
+endef
+
 define LIB_defs
  FILENAME_$(1) ?= xen$(1)
  XEN_libxen$(1) = $$(XEN_ROOT)/tools/libs/$(1)
-- 
Anthony PERARD

Re: [XEN PATCH for-4.17 v5 16/17] tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

[Andrew Cooper]

On 13/10/2022 14:05, Anthony Perard wrote:
> diff --git a/tools/golang/xenlight/gengotypes.py b/tools/golang/xenlight/gengotypes.py
> index ac1cf060dd..ff4c2ad216 100644
> --- a/tools/golang/xenlight/gengotypes.py
> +++ b/tools/golang/xenlight/gengotypes.py
> @@ -723,7 +723,13 @@ def xenlight_golang_fmt_name(name, exported = True):
>      return words[0] + ''.join(x.title() for x in words[1:])
>  
>  if __name__ == '__main__':
> +    if len(sys.argv) != 4:
> +        print("Usage: gengotypes.py <idl> <types.gen.go> <helpers.gen.go>", file=sys.stderr)

This breaks with Py2.7.  Needs a

from __future__ import print_function

inserting at the top.

~Andrew

Re: [XEN PATCH for-4.17 v5 05/17] tools: Remove -Werror everywhere else

[Andrew Cooper]

On 13/10/2022 14:05, Anthony Perard wrote:
> diff --git a/tools/debugger/gdbsx/Rules.mk b/tools/debugger/gdbsx/Rules.mk
> index 920f1c87fb..0610db873b 100644
> --- a/tools/debugger/gdbsx/Rules.mk
> +++ b/tools/debugger/gdbsx/Rules.mk
> @@ -1,6 +1,6 @@
>  include $(XEN_ROOT)/tools/Rules.mk
>  
> -CFLAGS   += -Werror -Wmissing-prototypes 
> +CFLAGS   += -Wmissing-prototypes 
>  # (gcc 4.3x and later)   -Wconversion -Wno-sign-conversion
>  
>  CFLAGS-$(clang) += -Wno-ignored-attributes
>

This hunk introduces trailing whitespace.  All other hunks are fine.

~Andrew

Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

[Andrew Cooper]

On 13/10/2022 14:04, Anthony PERARD wrote:
> Patch series available in this git branch:
> https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v5

I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and
it does resolve the Ocaml library linking issues.  Thanks.

So T-by.  If an allowance for 4.17 appears, I can see about getting some
of the more obviously-trivial areas reviewed and committed.

~Andrew

RE: [XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

[Henry Wang]

Hi Andrew,

> -----Original Message-----
> From: Andrew Cooper <Andrew.Cooper3@citrix.com>
> Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system
> improvement, toward non-recursive makefiles
> 
> On 13/10/2022 14:04, Anthony PERARD wrote:
> > Patch series available in this git branch:
> > https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git
> br.toolstack-build-system-v5
> 
> I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and
> it does resolve the Ocaml library linking issues.  Thanks.
> 
> So T-by.  If an allowance for 4.17 appears, I can see about getting some
> of the more obviously-trivial areas reviewed and committed.

Release-acked-by: Henry Wang <Henry.Wang@arm.com>

Since this series is quite large and just out of curiosity, do we have a
pipeline or CI jobs to test all the modules that affected by the makefile
touched by this series? Would be good to have some test results to
boost our confidence of committing the series.

Kind regards,
Henry

> 
> ~Andrew

Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

[Anthony PERARD]

On Fri, Oct 14, 2022 at 06:26:09AM +0000, Henry Wang wrote:
> > -----Original Message-----
> > From: Andrew Cooper <Andrew.Cooper3@citrix.com>
> > Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system
> > improvement, toward non-recursive makefiles
> > 
> > On 13/10/2022 14:04, Anthony PERARD wrote:
> > > Patch series available in this git branch:
> > > https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git
> > br.toolstack-build-system-v5
> > 
> > I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and
> > it does resolve the Ocaml library linking issues.  Thanks.
> > 
> > So T-by.  If an allowance for 4.17 appears, I can see about getting some
> > of the more obviously-trivial areas reviewed and committed.
> 
> Release-acked-by: Henry Wang <Henry.Wang@arm.com>

Thanks.

> Since this series is quite large and just out of curiosity, do we have a
> pipeline or CI jobs to test all the modules that affected by the makefile
> touched by this series? Would be good to have some test results to
> boost our confidence of committing the series.

I did push my branch to Gitlab, and that the pipeline:
    https://gitlab.com/xen-project/people/anthonyper/xen/-/pipelines/665987083
    (private to xen-project members)
Only failures are I think the same one as on staging, test failure on
arm, but no build issues.

I'm not sure that everything is tested, but most of it is.

Cheers,

-- 
Anthony PERARD

Re: [XEN PATCH for-4.17 v5 05/17] tools: Remove -Werror everywhere else

[Anthony PERARD]

On Thu, Oct 13, 2022 at 03:13:32PM +0000, Andrew Cooper wrote:
> On 13/10/2022 14:05, Anthony Perard wrote:
> > diff --git a/tools/debugger/gdbsx/Rules.mk b/tools/debugger/gdbsx/Rules.mk
> > index 920f1c87fb..0610db873b 100644
> > --- a/tools/debugger/gdbsx/Rules.mk
> > +++ b/tools/debugger/gdbsx/Rules.mk
> > @@ -1,6 +1,6 @@
> >  include $(XEN_ROOT)/tools/Rules.mk
> >  
> > -CFLAGS   += -Werror -Wmissing-prototypes 
> > +CFLAGS   += -Wmissing-prototypes 
> >  # (gcc 4.3x and later)   -Wconversion -Wno-sign-conversion
> >  
> >  CFLAGS-$(clang) += -Wno-ignored-attributes
> >
> 
> This hunk introduces trailing whitespace.

No, but it would be a good opportunity to remove it, I agree.

Cheers,

-- 
Anthony PERARD

RE: [XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

[Henry Wang]

Hi Anthony,

> -----Original Message-----
> From: Anthony PERARD <anthony.perard@citrix.com>
> Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system
> improvement, toward non-recursive makefiles
> 
> On Fri, Oct 14, 2022 at 06:26:09AM +0000, Henry Wang wrote:
> > > -----Original Message-----
> > > From: Andrew Cooper <Andrew.Cooper3@citrix.com>
> > > Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system
> > > improvement, toward non-recursive makefiles
> > >
> > > On 13/10/2022 14:04, Anthony PERARD wrote:
> > > > Patch series available in this git branch:
> > > > https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git
> > > br.toolstack-build-system-v5
> > >
> > > I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and
> > > it does resolve the Ocaml library linking issues.� Thanks.
> > >
> > > So T-by.� If an allowance for 4.17 appears, I can see about getting some
> > > of the more obviously-trivial areas reviewed and committed.
> >
> > Release-acked-by: Henry Wang <Henry.Wang@arm.com>
> 
> Thanks.

No problem, my pleasure.

> 
> > Since this series is quite large and just out of curiosity, do we have a
> > pipeline or CI jobs to test all the modules that affected by the makefile
> > touched by this series? Would be good to have some test results to
> > boost our confidence of committing the series.
> 
> I did push my branch to Gitlab, and that the pipeline:
>     https://gitlab.com/xen-project/people/anthonyper/xen/-
> /pipelines/665987083
>     (private to xen-project members)
> Only failures are I think the same one as on staging, test failure on
> arm, but no build issues.

Sorry for the arm failure, I will try to respin the patch ASAP and hopefully
the CI will be unblocked soon.

Kind regards,
Henry

> 
> I'm not sure that everything is tested, but most of it is.
> 
> Cheers,
> 
> --
> Anthony PERARD

Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

[Andrew Cooper]

On 14/10/2022 11:14, Anthony PERARD wrote:
> On Fri, Oct 14, 2022 at 06:26:09AM +0000, Henry Wang wrote:
>>> -----Original Message-----
>>> From: Andrew Cooper <Andrew.Cooper3@citrix.com>
>>> Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system
>>> improvement, toward non-recursive makefiles
>>>
>>> On 13/10/2022 14:04, Anthony PERARD wrote:
>>>> Patch series available in this git branch:
>>>> https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git
>>> br.toolstack-build-system-v5
>>>
>>> I've done a full rebuild of this (with the 2 noted fixes) with Xapi, and
>>> it does resolve the Ocaml library linking issues.  Thanks.
>>>
>>> So T-by.  If an allowance for 4.17 appears, I can see about getting some
>>> of the more obviously-trivial areas reviewed and committed.
>> Release-acked-by: Henry Wang <Henry.Wang@arm.com>
> Thanks.
>
>> Since this series is quite large and just out of curiosity, do we have a
>> pipeline or CI jobs to test all the modules that affected by the makefile
>> touched by this series? Would be good to have some test results to
>> boost our confidence of committing the series.
> I did push my branch to Gitlab, and that the pipeline:
>     https://gitlab.com/xen-project/people/anthonyper/xen/-/pipelines/665987083
>     (private to xen-project members)
> Only failures are I think the same one as on staging, test failure on
> arm, but no build issues.
>
> I'm not sure that everything is tested, but most of it is.

The XenServer specfile lists all build artefacts specifically, because
we've had too many bugs where things have silently disappeared.

My testing confirms that nothing has appeared or disappeared as a result
of these changes, and the functional looks pretty good.  (except for the
latest shadow assertion found, which was a regression in XSA-410.  Fix
is in review elsewhere.)

~Andrew

RE: [XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

[Henry Wang]

Hi Andrew,

> -----Original Message-----
> From: Andrew Cooper <Andrew.Cooper3@citrix.com>
> Subject: Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system
> improvement, toward non-recursive makefiles
> The XenServer specfile lists all build artefacts specifically, because
> we've had too many bugs where things have silently disappeared.
> 
> My testing confirms that nothing has appeared or disappeared as a result
> of these changes, and the functional looks pretty good.  (except for the
> latest shadow assertion found, which was a regression in XSA-410.  Fix
> is in review elsewhere.)

This is good to know. Thanks for the confirmation. No problem from
my side then.

Kind regards,
Henry

> 
> ~Andrew

Re: [XEN PATCH for-4.17 v5 16/17] tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

[George Dunlap]

[-- Attachment #1: Type: text/plain, Size: 968 bytes --]



> On 13 Oct 2022, at 16:00, Andrew Cooper <Andrew.Cooper3@citrix.com> wrote:
> 
> On 13/10/2022 14:05, Anthony Perard wrote:
>> diff --git a/tools/golang/xenlight/gengotypes.py b/tools/golang/xenlight/gengotypes.py
>> index ac1cf060dd..ff4c2ad216 100644
>> --- a/tools/golang/xenlight/gengotypes.py
>> +++ b/tools/golang/xenlight/gengotypes.py
>> @@ -723,7 +723,13 @@ def xenlight_golang_fmt_name(name, exported = True):
>>     return words[0] + ''.join(x.title() for x in words[1:])
>> 
>> if __name__ == '__main__':
>> +    if len(sys.argv) != 4:
>> +        print("Usage: gengotypes.py <idl> <types.gen.go> <helpers.gen.go>", file=sys.stderr)
> 
> This breaks with Py2.7.  Needs a
> 
> from __future__ import print_function
> 
> inserting at the top.

Out of curiosity, did you notice this by inspection, or  because you specifically tested Python 2.7, or because a system you were using is still actually using Python 2.7?

 -George



[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [XEN PATCH for-4.17 v5 16/17] tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

[George Dunlap]

[-- Attachment #1: Type: text/plain, Size: 708 bytes --]



> On 13 Oct 2022, at 14:05, Anthony PERARD <anthony.perard@citrix.com> wrote:
> 
> gengotypes.py creates both "types.gen.go" and "helpers.gen.go", but
> make can start gengotypes.py twice. Rework the rules so that
> gengotypes.py is executed only once.
> 
> Also, add the ability to provide a path to tell gengotypes.py where to
> put the files. This doesn't matter yet but it will when for example
> the script will be run from tools/ to generate the targets.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Release-acked-by: Henry Wang <Henry.Wang@arm.com>

Acked-by: George Dunlap <george.dunlap@citrix.com>

And you can keep the ack when you address the Python 2.7 problem.

 -George


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [XEN PATCH for-4.17 v5 16/17] tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

[Andrew Cooper]

On 14/10/2022 12:24, George Dunlap wrote:
>> On 13 Oct 2022, at 16:00, Andrew Cooper <Andrew.Cooper3@citrix.com> wrote:
>>
>> On 13/10/2022 14:05, Anthony Perard wrote:
>>> diff --git a/tools/golang/xenlight/gengotypes.py b/tools/golang/xenlight/gengotypes.py
>>> index ac1cf060dd..ff4c2ad216 100644
>>> --- a/tools/golang/xenlight/gengotypes.py
>>> +++ b/tools/golang/xenlight/gengotypes.py
>>> @@ -723,7 +723,13 @@ def xenlight_golang_fmt_name(name, exported = True):
>>>     return words[0] + ''.join(x.title() for x in words[1:])
>>>
>>> if __name__ == '__main__':
>>> +    if len(sys.argv) != 4:
>>> +        print("Usage: gengotypes.py <idl> <types.gen.go> <helpers.gen.go>", file=sys.stderr)
>> This breaks with Py2.7.  Needs a
>>
>> from __future__ import print_function
>>
>> inserting at the top.
> Out of curiosity, did you notice this by inspection, or  because you specifically tested Python 2.7, or because a system you were using is still actually using Python 2.7?

Xen's build system can't actually create a build which supports Py2 and
Py3, because xen.lowlevel.{xc,xs} only get built once.  It would be nice
to fix this, but -ETUITS, so we state a specific version in the specfile
and mock ensures there is no trace of the other one.

XenServer is in the process of trying to retire Py2, but it turns out
that Xen isn't actually fully Py3 clean yet, so we use Py2 for Xen.

The build breaks because the libxl build writes the .go files even when
we don't actually want go bindings in the end.

~Andrew

Re: [XEN PATCH for-4.17 v5 16/17] tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

[George Dunlap]

[-- Attachment #1.1: Type: text/plain, Size: 2073 bytes --]



> On 14 Oct 2022, at 12:37, Andrew Cooper <Andrew.Cooper3@citrix.com> wrote:
> 
> On 14/10/2022 12:24, George Dunlap wrote:
>>> On 13 Oct 2022, at 16:00, Andrew Cooper <Andrew.Cooper3@citrix.com> wrote:
>>> 
>>> On 13/10/2022 14:05, Anthony Perard wrote:
>>>> diff --git a/tools/golang/xenlight/gengotypes.py b/tools/golang/xenlight/gengotypes.py
>>>> index ac1cf060dd..ff4c2ad216 100644
>>>> --- a/tools/golang/xenlight/gengotypes.py
>>>> +++ b/tools/golang/xenlight/gengotypes.py
>>>> @@ -723,7 +723,13 @@ def xenlight_golang_fmt_name(name, exported = True):
>>>> return words[0] + ''.join(x.title() for x in words[1:])
>>>> 
>>>> if __name__ == '__main__':
>>>> + if len(sys.argv) != 4:
>>>> + print("Usage: gengotypes.py <idl> <types.gen.go> <helpers.gen.go>", file=sys.stderr)
>>> This breaks with Py2.7. Needs a
>>> 
>>> from __future__ import print_function
>>> 
>>> inserting at the top.
>> Out of curiosity, did you notice this by inspection, or because you specifically tested Python 2.7, or because a system you were using is still actually using Python 2.7?
> 
> Xen's build system can't actually create a build which supports Py2 and
> Py3, because xen.lowlevel.{xc,xs} only get built once.  It would be nice
> to fix this, but -ETUITS, so we state a specific version in the specfile
> and mock ensures there is no trace of the other one.
> 
> XenServer is in the process of trying to retire Py2, but it turns out
> that Xen isn't actually fully Py3 clean yet, so we use Py2 for Xen.
> 
> The build breaks because the libxl build writes the .go files even when
> we don't actually want go bindings in the end.

I think the generation code is looped in even when golang is disabled so that we can detect IDL changes during development, even on systems which don’t have golang installed.  In theory if libxl.idl doesn’t change, it shouldn’t trigger the build?  Alternately we could consider skipping the code generation on non-debug builds, since we only really need to detect changes during development.

 -George

[-- Attachment #1.2: Type: text/html, Size: 10993 bytes --]

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [XEN PATCH for-4.17 v5 16/17] tools/golang/xenlight: Rework gengotypes.py and generation of *.gen.go

[Andrew Cooper]

[-- Attachment #1: Type: text/plain, Size: 2287 bytes --]

On 14/10/2022 12:42, George Dunlap wrote:


On 14 Oct 2022, at 12:37, Andrew Cooper <Andrew.Cooper3@citrix.com<mailto:Andrew.Cooper3@citrix.com>> wrote:

On 14/10/2022 12:24, George Dunlap wrote:
On 13 Oct 2022, at 16:00, Andrew Cooper <Andrew.Cooper3@citrix.com<mailto:Andrew.Cooper3@citrix.com>> wrote:

On 13/10/2022 14:05, Anthony Perard wrote:
diff --git a/tools/golang/xenlight/gengotypes.py b/tools/golang/xenlight/gengotypes.py
index ac1cf060dd..ff4c2ad216 100644
--- a/tools/golang/xenlight/gengotypes.py
+++ b/tools/golang/xenlight/gengotypes.py
@@ -723,7 +723,13 @@ def xenlight_golang_fmt_name(name, exported = True):
return words[0] + ''.join(x.title() for x in words[1:])

if __name__ == '__main__':
+ if len(sys.argv) != 4:
+ print("Usage: gengotypes.py <idl> <types.gen.go> <helpers.gen.go>", file=sys.stderr)
This breaks with Py2.7. Needs a

from __future__ import print_function

inserting at the top.
Out of curiosity, did you notice this by inspection, or because you specifically tested Python 2.7, or because a system you were using is still actually using Python 2.7?

Xen's build system can't actually create a build which supports Py2 and
Py3, because xen.lowlevel.{xc,xs} only get built once.  It would be nice
to fix this, but -ETUITS, so we state a specific version in the specfile
and mock ensures there is no trace of the other one.

XenServer is in the process of trying to retire Py2, but it turns out
that Xen isn't actually fully Py3 clean yet, so we use Py2 for Xen.

The build breaks because the libxl build writes the .go files even when
we don't actually want go bindings in the end.

I think the generation code is looped in even when golang is disabled so that we can detect IDL changes during development, even on systems which don’t have golang installed.  In theory if libxl.idl doesn’t change, it shouldn’t trigger the build?  Alternately we could consider skipping the code generation on non-debug builds, since we only really need to detect changes during development.

This is a "clean" mock build, from a git archive'd tarball.  Any logic like this is going to trigger unconditionally.

It's not a problem IMO - writing out one extra text file is not the slow bit of a Xen build.

~Andrew

[-- Attachment #2: Type: text/html, Size: 13602 bytes --]

Re: [XEN PATCH for-4.17 v5 00/17] Toolstack build system improvement, toward non-recursive makefiles

[Christian Lindig]

[-- Attachment #1: Type: text/plain, Size: 870 bytes --]



On 13 Oct 2022, at 14:04, Anthony PERARD <anthony.perard@citrix.com<mailto:anthony.perard@citrix.com>> wrote:

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.toolstack-build-system-v5

Changes in v5:
- rebased on staging
- added "tools: Rework linking options for ocaml binding libraries"

Changes in v4:
- several new patches
- some changes to other patches listed in their changelogs

Changes in v3:
- rebased
- several new patches, starting with 13/25 "tools/libs/util: cleanup Makefile"
- introducing macros to deal with linking with in-tree xen libraries
- Add -Werror to CFLAGS for all builds in tools/

Changes in v2:
- one new patch
- other changes described in patch notes

Acked-by: Christian Lindig <christian.lindig@citrix.com<mailto:christian.lindig@citrix.com>>


[-- Attachment #2: Type: text/html, Size: 14736 bytes --]

Re: [XEN PATCH for-4.17 v5 05/17] tools: Remove -Werror everywhere else

[Anthony PERARD]

On Thu, Oct 13, 2022 at 02:05:01PM +0100, Anthony PERARD wrote:
> Patch "tools: Add -Werror by default to all tools/" have added
> "-Werror" to CFLAGS in tools/Rules.mk, remove it from every other
> makefiles as it is now duplicated.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Acked-by: Christian Lindig <christian.lindig@citrix.com>
> Acked-by: Daniel P. Smith <dpsmith@apertussolutions.com> # tools/flask
> Acked-by: Jan Beulich <jbeulich@suse.com> # tools/*/cpu-policy
> ---
> 
> Notes:
>     missing-ack:
>     - GDBSX DEBUGGER
>     - KDD DEBUGGER
>     - XENTRACE
> 
>  tools/console/client/Makefile   | 1 -
>  tools/console/daemon/Makefile   | 1 -
>  tools/debugger/kdd/Makefile     | 1 -
>  tools/flask/utils/Makefile      | 1 -
>  tools/fuzz/cpu-policy/Makefile  | 2 +-
>  tools/misc/Makefile             | 1 -
>  tools/tests/cpu-policy/Makefile | 2 +-
>  tools/tests/depriv/Makefile     | 2 +-
>  tools/tests/resource/Makefile   | 1 -
>  tools/tests/tsx/Makefile        | 1 -
>  tools/tests/xenstore/Makefile   | 1 -
>  tools/xcutils/Makefile          | 2 --
>  tools/xenmon/Makefile           | 1 -
>  tools/xenpaging/Makefile        | 1 -
>  tools/xenpmd/Makefile           | 1 -
>  tools/xentop/Makefile           | 2 +-
>  tools/xentrace/Makefile         | 2 --
>  tools/xl/Makefile               | 2 +-
>  tools/debugger/gdbsx/Rules.mk   | 2 +-
>  tools/firmware/Rules.mk         | 2 --
>  tools/libfsimage/common.mk      | 2 +-
>  tools/libs/libs.mk              | 2 +-
>  tools/ocaml/common.make         | 2 +-
>  tools/xenstore/Makefile.common  | 1 -
>  24 files changed, 9 insertions(+), 27 deletions(-)

Missing hunk as pointed out by Andrew:

diff --git a/tools/pygrub/setup.py b/tools/pygrub/setup.py
index b8f1dc4590..0e4e3d02d3 100644
--- a/tools/pygrub/setup.py
+++ b/tools/pygrub/setup.py
@@ -3,7 +3,7 @@ from distutils.ccompiler import new_compiler
 import os
 import sys

-extra_compile_args  = [ "-fno-strict-aliasing", "-Werror" ]
+extra_compile_args  = [ "-fno-strict-aliasing" ]

 XEN_ROOT = "../.."

diff --git a/tools/python/setup.py b/tools/python/setup.py
index 8c95db7769..721a3141d7 100644
--- a/tools/python/setup.py
+++ b/tools/python/setup.py
@@ -8,7 +8,7 @@ SHLIB_libxenctrl = os.environ['SHLIB_libxenctrl'].split()
 SHLIB_libxenguest = os.environ['SHLIB_libxenguest'].split()
 SHLIB_libxenstore = os.environ['SHLIB_libxenstore'].split()

-extra_compile_args  = [ "-fno-strict-aliasing", "-Werror" ]
+extra_compile_args  = [ "-fno-strict-aliasing" ]

 PATH_XEN      = XEN_ROOT + "/tools/include"
 PATH_LIBXENTOOLLOG = XEN_ROOT + "/tools/libs/toollog"


We can see the change, or duplicate of "-Werror" in commands run by
setup.py by running `setup.py build --verbose`.

-- 
Anthony PERARD

Re: [XEN PATCH for-4.17 v5 06/17] tools/hotplug: Generate "hotplugpath.sh" with configure

[Andrew Cooper]

On 13/10/2022 14:05, Anthony Perard wrote:
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

It occurs to me that with this sorted, there's only a single remaining
user of buildmakevars2file, and that's for stubdompath.sh.  It would be
great if we could convert that final user too.

~Andrew

Re: [XEN PATCH for-4.17 v5 10/17] libs: Fix auto-generation of version-script for unstable libs

[Andrew Cooper]

On 13/10/2022 14:05, Anthony Perard wrote:
> When there isn't a version-script for a shared library (like for
> unstable libs), we create one based on the current Xen version. But
> that version-script became out-of-date as soon as Xen's version
> changes and make as no way to regenerate the version-script on
> rebuild.
>
> In order to allow regenerating the script, we'll have a different
> filename. In order to check if the content is up-to-date, we'll always
> generated it and compare.
>
> But we also need to know if there's already an existing version script
> or if we need to generate one, for that we introduce $(version-script)
> to be used to point to the path of the existing script. (Guessing if a
> version script exist for a stable library with for example $(wildcard)
> won't work as a file will exist when building the library without this
> patch.)
>
> We don't need the version-script unless we are making the shared
> library so it is removed from the "all" target.
>
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
>
> Alternatively, we could remove the need for to generate a
> version-script, but that would mean the exported symbols won't have a
> version anymore (like before, a few versions of Xen ago). Or, we could
> use --default-symver, but the symboled would be versionned with the
> soname (e.g. "libxenctrl.so.4.16.0") instead of "VERS_4.16.0".

It turns out that the use of VERS_* everywhere is buggy.  We should have
had 'XEN$FOO' in the soname for libraries.  But it's too late for the
stable ones.

As far as the unstable libraries go, it really doesn't matter.  The only
thing we care about (rightly or wrongly) is that you can't mistake the
libraries from different versions of Xen.

--default-symver sounds like it would be ideal, and avoid writing out
temporary version scripts.

~Andrew

Re: [XEN PATCH for-4.17 v5 14/17] libs/light: Rework targets prerequisites

[Andrew Cooper]

On 13/10/2022 14:05, Anthony Perard wrote:
> No need for $(AUTOSRCS), GNU make can generate them as needed when
> trying to build them as needed when trying to build the object. Also,
> those two AUTOSRCS don't need to be a prerequisite of "all". As for
> the "clean" target, those two files are already removed via "_*.c".
>
> We don't need $(AUTOINCS) either:
> - As for both _libxl_savm_msgs*.h headers, we are adding more
>   selective dependencies so the headers will still be generated as
>   needed.
> - "clean" rule already delete the _*.h files, so AUTOINCS aren't needed
>   there.
>
> "libxl_internal_json.h" doesn't seems to have ever existed, so the
> dependency is removed.
>
> Add few prerequisite for "libxl_internal.h" so all headers that it
> depends on should be generated. And have $(SAVE_HELPER_OBJS) depends
> on "libxl_internal.h".
>
> Rework objects prerequisites, to have them dependents on either
> "libxl.h" or "libxl_internal.h". "libxl.h" is not normally included
> directly in the source code as "libxl_internal.h" is used instead. But
> we are adding "libxl.h" as prerequisite of "libxl_internal.h", so
> generated headers will still be generated as needed.
>
> "testidl.c" doesn't depends on "libxl.h" but "testidl.o" does.

I'm afraid I don't follow here.  How can this be true?

~Andrew

Re: [XEN PATCH for-4.17 v5 10/17] libs: Fix auto-generation of version-script for unstable libs

[Anthony PERARD]

On Fri, Oct 14, 2022 at 06:16:27PM +0000, Andrew Cooper wrote:
> On 13/10/2022 14:05, Anthony Perard wrote:
> > Alternatively, we could remove the need for to generate a
> > version-script, but that would mean the exported symbols won't have a
> > version anymore (like before, a few versions of Xen ago). Or, we could
> > use --default-symver, but the symboled would be versionned with the
> > soname (e.g. "libxenctrl.so.4.16.0") instead of "VERS_4.16.0".
> 
> It turns out that the use of VERS_* everywhere is buggy.  We should have
> had 'XEN$FOO' in the soname for libraries.  But it's too late for the
> stable ones.
> 
> As far as the unstable libraries go, it really doesn't matter.  The only
> thing we care about (rightly or wrongly) is that you can't mistake the
> libraries from different versions of Xen.
> 
> --default-symver sounds like it would be ideal, and avoid writing out
> temporary version scripts.

Sounds good to me, I'll rewrite the patch to use --default-symver.
Hopefully, that's an old enough option.

Thanks,

-- 
Anthony PERARD

Re: [XEN PATCH for-4.17 v5 14/17] libs/light: Rework targets prerequisites

[Anthony PERARD]

On Fri, Oct 14, 2022 at 07:35:20PM +0000, Andrew Cooper wrote:
> On 13/10/2022 14:05, Anthony Perard wrote:
> > No need for $(AUTOSRCS), GNU make can generate them as needed when
> > trying to build them as needed when trying to build the object. Also,
> > those two AUTOSRCS don't need to be a prerequisite of "all". As for
> > the "clean" target, those two files are already removed via "_*.c".
> >
> > We don't need $(AUTOINCS) either:
> > - As for both _libxl_savm_msgs*.h headers, we are adding more
> >   selective dependencies so the headers will still be generated as
> >   needed.
> > - "clean" rule already delete the _*.h files, so AUTOINCS aren't needed
> >   there.
> >
> > "libxl_internal_json.h" doesn't seems to have ever existed, so the
> > dependency is removed.
> >
> > Add few prerequisite for "libxl_internal.h" so all headers that it
> > depends on should be generated. And have $(SAVE_HELPER_OBJS) depends
> > on "libxl_internal.h".
> >
> > Rework objects prerequisites, to have them dependents on either
> > "libxl.h" or "libxl_internal.h". "libxl.h" is not normally included
> > directly in the source code as "libxl_internal.h" is used instead. But
> > we are adding "libxl.h" as prerequisite of "libxl_internal.h", so
> > generated headers will still be generated as needed.
> >
> > "testidl.c" doesn't depends on "libxl.h" but "testidl.o" does.
> 
> I'm afraid I don't follow here.  How can this be true?

From make point-of-view, in order to generate "testidl.c", we only need
to execute "gentest.py" which takes "libxl_types.idl" as input. It
doesn't even matter if "libxl.h" exist when generating "testidl.c" via
`make testidl.c`.

"libxl.h" is only used later when compiling "testidl.c" into
"testidl.o".

I can probably expand the commit message to better explain this.

Thanks,

-- 
Anthony PERARD