[XEN PATCH v9 00/30] xen: Build system improvements, now with out-of-tree build!

[XEN PATCH v9 00/30] xen: Build system improvements, now with out-of-tree build!

[Anthony PERARD]

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-v9

v9:
    One new patch (patch 3).
    Otherwise, detailed change logs in patches.

    Removed the rfc part about doing out-of-tree build without setting VPATH
    (which would have allowed mixed in-tree / out-of-tree builds).

v8:
    Mostly rework of v7. With many patch already applied.
    Some detail changes that are spread through many patches:
    - `make cloc` recipe should now work throughout the series, update of it is
      done in 3 patches.
    - new patch "build: fix enforce unique symbols for recent clang version"
      to fix an issue with clang.
    - introducing $(srctree) and $(objtree) earlier
    - introducing $(srcdir) as shortcut for $(srctree)/$(src)
    - introduce usage of -iquote instead of -I in some cases
    More detail change log can be found in patches notes.

    Also this v8 present a work-in-progress of the ability to do out-of-tree
    build without setting VPATH. This is presented as an alternative to force
    use of out-of-tree build. As the last patch show, it allows to build the
    xen-shim without the linkfarm and we don't need to make any other changes
    to any thing that build xen (osstest, distribution packages, xen.git, ...,
    and developers finger macros). The patches are only there as WIP / RFC as
    they were some concern about the usefulness and extra changes needed.
    We can decide whether those changes are good or if this is too much and we
    should force out-of-tree build for the hypervisor.

v7:
    Out-of-tree build!

    This mean many more patches. Everything after patch 27 is new.

    There's a few new patch before that, but otherwise are rework of v6.

Hi,

I have work toward building Xen (the hypervisor) with Linux's build system,
Kbuild.

The main reason for that is to be able to have out-of-tree build. It's annoying
when a build fail because of the pvshim. Other benefit is a much faster
rebuild, and `make clean` doesn't take ages, and better dependencies to figure
out what needs to be rebuild.

Cheers,

Anthony PERARD (30):
  build: set XEN_BUILD_EFI earlier
  build: avoid re-executing the main Makefile by introducing build.mk
  build: fix exported variable name CFLAGS_stack_boundary
  build: set ALL_OBJS in main Makefile; move prelink.o to main Makefile
  build: prepare to always invoke $(MAKE) from xen/, use $(obj)
  build: rework test/livepatch/Makefile
  build: rework cloc recipe
  build: fix enforce unique symbols for recent clang version
  build: build everything from the root dir, use obj=$subdir
  build: introduce if_changed_deps
  build: rename __LINKER__ to LINKER_SCRIPT
  build: hook kconfig into xen build system
  xen/tools/kconfig: fix build with -Wdeclaration-after-statement
  build: remove KBUILD_ specific from Makefile.host
  build: handle always-y and hostprogs-always-y
  build: start building the tools with the main makefiles
  build: add headers path to CFLAGS once for all archs
  build: generate x86's asm-macros.h with filechk
  build: clean-up "clean" rules of duplication
  build: rework "clean" to clean from the root dir
  build: use main rune to build host binary x86's mkelf32 and mkreloc
  build: rework coverage and ubsan CFLAGS handling
  build,x86: remove the need for build32.mk
  build: grab common EFI source files in arch specific dir
  build: replace $(BASEDIR) by $(objtree)
  build: replace $(BASEDIR) and use $(srctree)
  build: rework "headers*.chk" prerequisite in include/
  build: specify source tree in include/ for prerequisite
  build: shuffle main Makefile
  build: adding out-of-tree support to the xen build

 .gitignore                                   |   2 +
 xen/Kconfig                                  |   4 +-
 xen/Makefile                                 | 367 ++++++++++-------
 xen/Rules.mk                                 | 227 +++++++----
 xen/arch/arm/Makefile                        |  53 +--
 xen/arch/arm/arch.mk                         |   5 +-
 xen/arch/arm/efi/Makefile                    |   4 +-
 xen/arch/arm/include/asm/config.h            |   2 +-
 xen/arch/riscv/arch.mk                       |   2 -
 xen/arch/x86/Makefile                        | 204 ++++------
 xen/arch/x86/Rules.mk                        |   8 +-
 xen/arch/x86/arch.mk                         |  59 ++-
 xen/arch/x86/boot/Makefile                   |  57 ++-
 xen/arch/x86/boot/build32.mk                 |  40 --
 xen/arch/x86/efi/Makefile                    |  11 +-
 xen/build.mk                                 |  90 +++++
 xen/common/Makefile                          |  13 +-
 xen/common/efi/efi-common.mk                 |  16 +
 xen/common/libelf/Makefile                   |   4 +-
 xen/common/libfdt/Makefile                   |   8 +-
 xen/include/Makefile                         |  76 ++--
 xen/scripts/Kbuild.include                   |  69 +++-
 xen/scripts/Kconfig.include                  |   2 +-
 xen/scripts/Makefile.clean                   |  32 +-
 xen/{tools/kconfig => scripts}/Makefile.host |  37 +-
 xen/test/Makefile                            |   7 +-
 xen/test/livepatch/Makefile                  | 213 ++++------
 xen/tools/Makefile                           |  14 +-
 xen/tools/fixdep.c                           | 404 +++++++++++++++++++
 xen/tools/kconfig/Makefile                   |   3 +
 xen/tools/kconfig/Makefile.kconfig           | 106 -----
 xen/tools/kconfig/confdata.c                 |   2 +-
 xen/xsm/flask/Makefile                       |  43 +-
 xen/xsm/flask/policy/mkaccess_vector.sh      |   7 +-
 xen/xsm/flask/ss/Makefile                    |   3 +-
 35 files changed, 1371 insertions(+), 823 deletions(-)
 delete mode 100644 xen/arch/x86/boot/build32.mk
 create mode 100644 xen/build.mk
 create mode 100644 xen/common/efi/efi-common.mk
 rename xen/{tools/kconfig => scripts}/Makefile.host (85%)
 create mode 100644 xen/tools/fixdep.c
 delete mode 100644 xen/tools/kconfig/Makefile.kconfig

-- 
Anthony PERARD

[XEN PATCH v9 01/30] build: set XEN_BUILD_EFI earlier

[Anthony PERARD]

We are going to need the variable XEN_BUILD_EFI earlier.

But a side effect of calculating the value of $(XEN_BUILD_EFI) is to
also to generate "efi/check.o" which is used for further checks.
Thus the whole chain that check for EFI support is moved to
"arch.mk".

Some other changes are made to avoid too much duplication:
    - $(efi-check): Used to avoid repeating "efi/check.*". We don't
      set it to the path to the source as it would be wrong as soon
      as we support out-of-tree build.
    - $(LD_PE_check_cmd): As it is called twice, with an updated
      $(EFI_LDFLAGS).

$(nr-fixups) is renamed to $(efi-nr-fixups) as the former might be
a bit too generic.

In order to avoid exporting MKRELOC, the variable is added to $(MAKE)
command line. The only modification needed is in target "build", the
modification target "$(TARGET)" will be needed with a following patch
"build: avoid re-executing the main Makefile by introducing build.mk".

We can now revert 24b0ce9a5da2, we don't need to override efi-y on
recursion anymore.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v9:
    - change $(efi-check) to hold the $(basename ) of check.c, and rename
      the variable from $(efi-check-0).
    - unexport MKRELOC, pass it down via command line instead.
    - remove efi-y override which reverts 24b0ce9a5da2
    
    v8:
    - rename to efi-nr-fixups rather than efi-check-relocs
    - use := when assigning variable in makefile when recursive expansion
      isn't needed.
    - no more check of $(efi-y) value for "CFLAGS-$(XEN_BUILD_EFI) +=
      -DXEN_BUILD_EFI".
    
    v7:
    - Do the whole check for EFI support in arch.mk. So efi/check.o is
      produce there and used there, and produce efi/check.efi and use it there.
      Thus avoid the need to repeat the test done for XEN_BUILD_EFI.

 xen/Makefile          |  4 ++--
 xen/arch/x86/Makefile | 46 ++++++-------------------------------------
 xen/arch/x86/arch.mk  | 42 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 50 insertions(+), 42 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index 1e10d9f68080..f3ff03a7170e 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -296,7 +296,7 @@ endif # need-config
 .PHONY: build install uninstall clean distclean MAP
 build install uninstall debug clean distclean MAP::
 ifneq ($(XEN_TARGET_ARCH),x86_32)
-	$(MAKE) -f Rules.mk _$@
+	$(MAKE) -f Rules.mk MKRELOC=$(MKRELOC) _$@
 else
 	echo "*** Xen x86/32 target no longer supported!"
 endif
@@ -404,7 +404,7 @@ $(TARGET): FORCE
 	$(MAKE) -f $(BASEDIR)/Rules.mk -C include
 	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) include
 	$(MAKE) -f $(BASEDIR)/Rules.mk arch/$(TARGET_ARCH)/include/asm/asm-offsets.h
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) $@
+	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) MKRELOC=$(MKRELOC) $@
 
 quiet_cmd_banner = BANNER  $@
 define cmd_banner
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 8db4cb98edbb..9fc884813cb5 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -121,44 +121,8 @@ $(TARGET): $(TARGET)-syms $(efi-y) boot/mkelf32
 		{ echo "No Multiboot2 header found" >&2; false; }
 	mv $(TMP) $(TARGET)
 
-ifneq ($(efi-y),)
-
-# Check if the compiler supports the MS ABI.
-export XEN_BUILD_EFI := $(shell $(CC) $(XEN_CFLAGS) -c efi/check.c -o efi/check.o 2>/dev/null && echo y)
 CFLAGS-$(XEN_BUILD_EFI) += -DXEN_BUILD_EFI
 
-# Check if the linker supports PE.
-EFI_LDFLAGS = $(patsubst -m%,-mi386pep,$(XEN_LDFLAGS)) --subsystem=10
-XEN_BUILD_PE := $(if $(XEN_BUILD_EFI),$(call ld-option,$(EFI_LDFLAGS) --image-base=0x100000000 -o efi/check.efi efi/check.o))
-# If the above failed, it may be merely because of the linker not dealing well
-# with debug info. Try again with stripping it.
-ifeq ($(CONFIG_DEBUG_INFO)-$(XEN_BUILD_PE),y-n)
-EFI_LDFLAGS += --strip-debug
-XEN_BUILD_PE := $(call ld-option,$(EFI_LDFLAGS) --image-base=0x100000000 -o efi/check.efi efi/check.o)
-endif
-
-ifeq ($(XEN_BUILD_PE),y)
-
-# Check if the linker produces fixups in PE by default
-nr-fixups := $(shell $(OBJDUMP) -p efi/check.efi | grep '^[[:blank:]]*reloc[[:blank:]]*[0-9][[:blank:]].*DIR64$$' | wc -l)
-ifeq ($(nr-fixups),2)
-MKRELOC := :
-relocs-dummy :=
-else
-MKRELOC := efi/mkreloc
-relocs-dummy := efi/relocs-dummy.o
-# If the linker produced fixups but not precisely two of them, we need to
-# disable it doing so.  But if it didn't produce any fixups, it also wouldn't
-# recognize the option.
-ifneq ($(nr-fixups),0)
-EFI_LDFLAGS += --disable-reloc-section
-endif
-endif
-
-endif # $(XEN_BUILD_PE)
-
-endif # $(efi-y)
-
 ALL_OBJS := $(BASEDIR)/arch/x86/boot/built_in.o $(BASEDIR)/arch/x86/efi/built_in.o $(ALL_OBJS)
 
 ifeq ($(CONFIG_LTO),y)
@@ -182,13 +146,13 @@ $(TARGET)-syms: prelink.o xen.lds
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort \
 		>$(@D)/.$(@F).0.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk efi-y= $(@D)/.$(@F).0.o
+	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
 	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort $(syms-warn-dup-y) \
 		>$(@D)/.$(@F).1.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk efi-y= $(@D)/.$(@F).1.o
+	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
 	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
@@ -217,8 +181,10 @@ endif
 
 $(TARGET).efi: VIRT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A VIRT_START$$,,p')
 ifeq ($(MKRELOC),:)
+relocs-dummy :=
 $(TARGET).efi: ALT_BASE :=
 else
+relocs-dummy := efi/relocs-dummy.o
 $(TARGET).efi: ALT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A ALT_START$$,,p')
 endif
 
@@ -250,14 +216,14 @@ endif
 	$(MKRELOC) $(foreach base,$(VIRT_BASE) $(ALT_BASE),$(@D)/.$(@F).$(base).0) >$(@D)/.$(@F).0r.S
 	$(NM) -pa --format=sysv $(@D)/.$(@F).$(VIRT_BASE).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0s.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk efi-y= $(@D)/.$(@F).0r.o $(@D)/.$(@F).0s.o
+	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0r.o $(@D)/.$(@F).0s.o
 	$(foreach base, $(VIRT_BASE) $(ALT_BASE), \
 	          $(LD) $(call EFI_LDFLAGS,$(base)) -T efi.lds -N $< \
 	                $(@D)/.$(@F).0r.o $(@D)/.$(@F).0s.o $(note_file_option) -o $(@D)/.$(@F).$(base).1 &&) :
 	$(MKRELOC) $(foreach base,$(VIRT_BASE) $(ALT_BASE),$(@D)/.$(@F).$(base).1) >$(@D)/.$(@F).1r.S
 	$(NM) -pa --format=sysv $(@D)/.$(@F).$(VIRT_BASE).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1s.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk efi-y= $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o
+	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o
 	$(LD) $(call EFI_LDFLAGS,$(VIRT_BASE)) -T efi.lds -N $< \
 	                $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o $(note_file_option) -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
index eea320e618b9..a93fa6d2e4c9 100644
--- a/xen/arch/x86/arch.mk
+++ b/xen/arch/x86/arch.mk
@@ -60,5 +60,47 @@ ifeq ($(CONFIG_UBSAN),y)
 $(call cc-option-add,CFLAGS_UBSAN,CC,-fno-sanitize=alignment)
 endif
 
+ifneq ($(CONFIG_PV_SHIM_EXCLUSIVE),y)
+
+efi-check := arch/x86/efi/check
+
+# Check if the compiler supports the MS ABI.
+XEN_BUILD_EFI := $(call if-success,$(CC) $(CFLAGS) -c $(efi-check).c -o $(efi-check).o,y)
+
+# Check if the linker supports PE.
+EFI_LDFLAGS := $(patsubst -m%,-mi386pep,$(LDFLAGS)) --subsystem=10
+LD_PE_check_cmd = $(call ld-option,$(EFI_LDFLAGS) --image-base=0x100000000 -o $(efi-check).efi $(efi-check).o)
+XEN_BUILD_PE := $(LD_PE_check_cmd)
+
+# If the above failed, it may be merely because of the linker not dealing well
+# with debug info. Try again with stripping it.
+ifeq ($(CONFIG_DEBUG_INFO)-$(XEN_BUILD_PE),y-n)
+EFI_LDFLAGS += --strip-debug
+XEN_BUILD_PE := $(LD_PE_check_cmd)
+endif
+
+ifeq ($(XEN_BUILD_PE),y)
+
+# Check if the linker produces fixups in PE by default
+efi-nr-fixups := $(shell $(OBJDUMP) -p $(efi-check).efi | grep '^[[:blank:]]*reloc[[:blank:]]*[0-9][[:blank:]].*DIR64$$' | wc -l)
+
+ifeq ($(efi-nr-fixups),2)
+MKRELOC := :
+else
+MKRELOC := efi/mkreloc
+# If the linker produced fixups but not precisely two of them, we need to
+# disable it doing so.  But if it didn't produce any fixups, it also wouldn't
+# recognize the option.
+ifneq ($(efi-nr-fixups),0)
+EFI_LDFLAGS += --disable-reloc-section
+endif
+endif
+
+endif # $(XEN_BUILD_PE)
+
+export XEN_BUILD_EFI XEN_BUILD_PE
+export EFI_LDFLAGS
+endif
+
 # Set up the assembler include path properly for older toolchains.
 CFLAGS += -Wa,-I$(BASEDIR)/include
-- 
Anthony PERARD

[XEN PATCH v9 02/30] build: avoid re-executing the main Makefile by introducing build.mk

[Anthony PERARD]

Currently, the xen/Makefile is re-parsed several times: once to start
the build process, and several more time with Rules.mk including it.
This makes it difficult to work with a Makefile used for several
purpose, and it actually slow down the build process.

So this patch introduce "build.mk" which Rules.mk will use when
present instead of the "Makefile" of a directory. (Linux's Kbuild
named that file "Kbuild".)

We have a few targets to move to "build.mk" identified by them been
build via "make -f Rules.mk" without changing directory.

As for the main targets like "build", we can have them depends on
there underscore-prefix targets like "_build" without having to use
"Rules.mk" while still retaining the check for unsupported
architecture. (Those main rules are changed to be single-colon as
there should only be a single recipe for them.)

With nearly everything needed to move to "build.mk" moved, there is a
single dependency left from "Rules.mk": the variable $(TARGET), so its
assignement is moved to the main Makefile.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v9:
    - reviewed
    
    v8:
    - for $(main-targets) rules, use ; on a single line instead of adding a
      recipe with only @:
    - To include build.mk instead of Makefile, use a simpler expresion with
      $(firstword) and remove the use of $(mk-dir) var
    - removed an extra blank line, and one when updating $(targets)
    - reword patch description
    
    v7:
    - new patch

 xen/Makefile | 70 +++++-----------------------------------------------
 xen/Rules.mk |  6 ++---
 xen/build.mk | 61 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 69 insertions(+), 68 deletions(-)
 create mode 100644 xen/build.mk

diff --git a/xen/Makefile b/xen/Makefile
index f3ff03a7170e..fb37043d08e0 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -47,6 +47,8 @@ export KCONFIG_CONFIG ?= .config
 
 export CC CXX LD
 
+export TARGET := $(BASEDIR)/xen
+
 .PHONY: default
 default: build
 
@@ -293,11 +295,12 @@ export CFLAGS_UBSAN
 
 endif # need-config
 
-.PHONY: build install uninstall clean distclean MAP
-build install uninstall debug clean distclean MAP::
+main-targets := build install uninstall clean distclean MAP
+.PHONY: $(main-targets)
 ifneq ($(XEN_TARGET_ARCH),x86_32)
-	$(MAKE) -f Rules.mk MKRELOC=$(MKRELOC) _$@
+$(main-targets): %: _% ;
 else
+$(main-targets):
 	echo "*** Xen x86/32 target no longer supported!"
 endif
 
@@ -406,67 +409,6 @@ $(TARGET): FORCE
 	$(MAKE) -f $(BASEDIR)/Rules.mk arch/$(TARGET_ARCH)/include/asm/asm-offsets.h
 	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) MKRELOC=$(MKRELOC) $@
 
-quiet_cmd_banner = BANNER  $@
-define cmd_banner
-    if which figlet >/dev/null 2>&1 ; then \
-	echo " Xen $(XEN_FULLVERSION)" | figlet -f $< > $@.tmp; \
-    else \
-	echo " Xen $(XEN_FULLVERSION)" > $@.tmp; \
-    fi; \
-    mv -f $@.tmp $@
-endef
-
-.banner: tools/xen.flf FORCE
-	$(call if_changed,banner)
-targets += .banner
-
-# Don't refresh this files during e.g., 'sudo make install'
-quiet_cmd_compile.h = UPD     $@
-define cmd_compile.h
-    if [ ! -r $@ -o -O $@ ]; then \
-	cat .banner; \
-	sed -e 's/@@date@@/$(XEN_BUILD_DATE)/g' \
-	    -e 's/@@time@@/$(XEN_BUILD_TIME)/g' \
-	    -e 's/@@whoami@@/$(XEN_WHOAMI)/g' \
-	    -e 's/@@domain@@/$(XEN_DOMAIN)/g' \
-	    -e 's/@@hostname@@/$(XEN_BUILD_HOST)/g' \
-	    -e 's!@@compiler@@!$(shell $(CC) $(CFLAGS) --version 2>&1 | head -1)!g' \
-	    -e 's/@@version@@/$(XEN_VERSION)/g' \
-	    -e 's/@@subversion@@/$(XEN_SUBVERSION)/g' \
-	    -e 's/@@extraversion@@/$(XEN_EXTRAVERSION)/g' \
-	    -e 's!@@changeset@@!$(shell tools/scmversion $(XEN_ROOT) || echo "unavailable")!g' \
-	    < $< > $(dot-target).tmp; \
-	sed -rf tools/process-banner.sed < .banner >> $(dot-target).tmp; \
-	mv -f $(dot-target).tmp $@; \
-    fi
-endef
-
-include/xen/compile.h: include/xen/compile.h.in .banner FORCE
-	$(if $(filter-out FORCE,$?),$(Q)rm -fv $@)
-	$(call if_changed,compile.h)
-
-targets += include/xen/compile.h
-
-asm-offsets.s: arch/$(TARGET_ARCH)/$(TARGET_SUBARCH)/asm-offsets.c
-	$(CC) $(call cpp_flags,$(c_flags)) -S -g0 -o $@.new -MQ $@ $<
-	$(call move-if-changed,$@.new,$@)
-
-arch/$(TARGET_ARCH)/include/asm/asm-offsets.h: asm-offsets.s
-	@(set -e; \
-	  echo "/*"; \
-	  echo " * DO NOT MODIFY."; \
-	  echo " *"; \
-	  echo " * This file was auto-generated from $<"; \
-	  echo " *"; \
-	  echo " */"; \
-	  echo ""; \
-	  echo "#ifndef __ASM_OFFSETS_H__"; \
-	  echo "#define __ASM_OFFSETS_H__"; \
-	  echo ""; \
-	  sed -rne "/^[^#].*==>/{s:.*==>(.*)<==.*:\1:; s: [\$$#]: :; p;}"; \
-	  echo ""; \
-	  echo "#endif") <$< >$@
-
 SUBDIRS = xsm arch/$(TARGET_ARCH) common drivers lib test
 define all_sources
     ( find include -type f -name '*.h' -print; \
diff --git a/xen/Rules.mk b/xen/Rules.mk
index d21930a7bf71..7b8b9047cfd5 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -8,9 +8,6 @@
 include $(XEN_ROOT)/Config.mk
 include $(BASEDIR)/scripts/Kbuild.include
 
-
-TARGET := $(BASEDIR)/xen
-
 # Note that link order matters!
 ALL_OBJS-y               += $(BASEDIR)/common/built_in.o
 ALL_OBJS-y               += $(BASEDIR)/drivers/built_in.o
@@ -36,7 +33,8 @@ SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
                                             rodata.cst$(a)) \
                          $(foreach r,rel rel.ro,data.$(r).local)
 
-include Makefile
+# The filename build.mk has precedence over Makefile
+include $(firstword $(wildcard build.mk) Makefile)
 
 # Linking
 # ---------------------------------------------------------------------------
diff --git a/xen/build.mk b/xen/build.mk
new file mode 100644
index 000000000000..3d7a91df22d1
--- /dev/null
+++ b/xen/build.mk
@@ -0,0 +1,61 @@
+quiet_cmd_banner = BANNER  $@
+define cmd_banner
+    if which figlet >/dev/null 2>&1 ; then \
+	echo " Xen $(XEN_FULLVERSION)" | figlet -f $< > $@.tmp; \
+    else \
+	echo " Xen $(XEN_FULLVERSION)" > $@.tmp; \
+    fi; \
+    mv -f $@.tmp $@
+endef
+
+.banner: tools/xen.flf FORCE
+	$(call if_changed,banner)
+
+targets += .banner
+
+# Don't refresh this files during e.g., 'sudo make install'
+quiet_cmd_compile.h = UPD     $@
+define cmd_compile.h
+    if [ ! -r $@ -o -O $@ ]; then \
+	cat .banner; \
+	sed -e 's/@@date@@/$(XEN_BUILD_DATE)/g' \
+	    -e 's/@@time@@/$(XEN_BUILD_TIME)/g' \
+	    -e 's/@@whoami@@/$(XEN_WHOAMI)/g' \
+	    -e 's/@@domain@@/$(XEN_DOMAIN)/g' \
+	    -e 's/@@hostname@@/$(XEN_BUILD_HOST)/g' \
+	    -e 's!@@compiler@@!$(shell $(CC) $(CFLAGS) --version 2>&1 | head -1)!g' \
+	    -e 's/@@version@@/$(XEN_VERSION)/g' \
+	    -e 's/@@subversion@@/$(XEN_SUBVERSION)/g' \
+	    -e 's/@@extraversion@@/$(XEN_EXTRAVERSION)/g' \
+	    -e 's!@@changeset@@!$(shell tools/scmversion $(XEN_ROOT) || echo "unavailable")!g' \
+	    < $< > $(dot-target).tmp; \
+	sed -rf tools/process-banner.sed < .banner >> $(dot-target).tmp; \
+	mv -f $(dot-target).tmp $@; \
+    fi
+endef
+
+include/xen/compile.h: include/xen/compile.h.in .banner FORCE
+	$(if $(filter-out FORCE,$?),$(Q)rm -fv $@)
+	$(call if_changed,compile.h)
+
+targets += include/xen/compile.h
+
+asm-offsets.s: arch/$(TARGET_ARCH)/$(TARGET_SUBARCH)/asm-offsets.c
+	$(CC) $(call cpp_flags,$(c_flags)) -S -g0 -o $@.new -MQ $@ $<
+	$(call move-if-changed,$@.new,$@)
+
+arch/$(TARGET_ARCH)/include/asm/asm-offsets.h: asm-offsets.s
+	@(set -e; \
+	  echo "/*"; \
+	  echo " * DO NOT MODIFY."; \
+	  echo " *"; \
+	  echo " * This file was auto-generated from $<"; \
+	  echo " *"; \
+	  echo " */"; \
+	  echo ""; \
+	  echo "#ifndef __ASM_OFFSETS_H__"; \
+	  echo "#define __ASM_OFFSETS_H__"; \
+	  echo ""; \
+	  sed -rne "/^[^#].*==>/{s:.*==>(.*)<==.*:\1:; s: [\$$#]: :; p;}"; \
+	  echo ""; \
+	  echo "#endif") <$< >$@
-- 
Anthony PERARD

[XEN PATCH v9 03/30] build: fix exported variable name CFLAGS_stack_boundary

[Anthony PERARD]

Exporting a variable with a dash doesn't work reliably, they may be
striped from the environment when calling a sub-make or sub-shell.

CFLAGS-stack-boundary start to be removed from env in patch "build:
set ALL_OBJS in main Makefile; move prelink.o to main Makefile" when
running `make "ALL_OBJS=.."` due to the addition of the quote. At
least in my empirical tests.

Fixes: 2740d96efd ("xen/build: have the root Makefile generates the CFLAGS")
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v9:
    - new patch

 xen/arch/x86/Rules.mk     | 4 ++--
 xen/arch/x86/arch.mk      | 4 ++--
 xen/arch/x86/efi/Makefile | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/Rules.mk b/xen/arch/x86/Rules.mk
index 56fe22c979ea..7aef93f5f3a0 100644
--- a/xen/arch/x86/Rules.mk
+++ b/xen/arch/x86/Rules.mk
@@ -6,5 +6,5 @@ object_label_flags = '-D__OBJECT_LABEL__=$(subst $(BASEDIR)/,,$(CURDIR))/$@'
 else
 object_label_flags = '-D__OBJECT_LABEL__=$(subst /,$$,$(subst -,_,$(subst $(BASEDIR)/,,$(CURDIR))/$@))'
 endif
-c_flags += $(object_label_flags) $(CFLAGS-stack-boundary)
-a_flags += $(object_label_flags) $(CFLAGS-stack-boundary)
+c_flags += $(object_label_flags) $(CFLAGS_stack_boundary)
+a_flags += $(object_label_flags) $(CFLAGS_stack_boundary)
diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
index a93fa6d2e4c9..fa7cf3844362 100644
--- a/xen/arch/x86/arch.mk
+++ b/xen/arch/x86/arch.mk
@@ -49,8 +49,8 @@ CFLAGS-$(CONFIG_INDIRECT_THUNK) += -fno-jump-tables
 
 # If supported by the compiler, reduce stack alignment to 8 bytes. But allow
 # this to be overridden elsewhere.
-$(call cc-option-add,CFLAGS-stack-boundary,CC,-mpreferred-stack-boundary=3)
-export CFLAGS-stack-boundary
+$(call cc-option-add,CFLAGS_stack_boundary,CC,-mpreferred-stack-boundary=3)
+export CFLAGS_stack_boundary
 
 ifeq ($(CONFIG_UBSAN),y)
 # Don't enable alignment sanitisation.  x86 has efficient unaligned accesses,
diff --git a/xen/arch/x86/efi/Makefile b/xen/arch/x86/efi/Makefile
index 87b927ed865b..abae493bf344 100644
--- a/xen/arch/x86/efi/Makefile
+++ b/xen/arch/x86/efi/Makefile
@@ -12,7 +12,7 @@ EFIOBJ-y := boot.init.o pe.init.o ebmalloc.o runtime.o
 EFIOBJ-$(CONFIG_COMPAT) += compat.o
 
 $(call cc-option-add,cflags-stack-boundary,CC,-mpreferred-stack-boundary=4)
-$(EFIOBJ-y): CFLAGS-stack-boundary := $(cflags-stack-boundary)
+$(EFIOBJ-y): CFLAGS_stack_boundary := $(cflags-stack-boundary)
 
 obj-y := stub.o
 obj-$(XEN_BUILD_EFI) := $(filter-out %.init.o,$(EFIOBJ-y))
-- 
Anthony PERARD

[XEN PATCH v9 04/30] build: set ALL_OBJS in main Makefile; move prelink.o to main Makefile

[Anthony PERARD]

This is to avoid arch/$arch/Makefile having to recurse into parents
directories.

This avoid duplication of the logic to build prelink.o between arches.

In order to do that, we cut the $(TARGET) target in the main Makefile in
two, there is a "prepare" phase/target runned before starting to build
"prelink.o" which will prepare "include/" among other things, then all
the $(ALL_OBJS) will be generated in order to build "prelink.o" and
finally $(TARGET) will be generated by calling into "arch/*/" to make
$(TARGET).

Now we don't need to prefix $(ALL_OBJS) with $(BASEDIR) as it is now
only used from the main Makefile. Other changes is to use "$<" instead
of spelling "prelink.o" in the target "$(TARGET)" in both
arch/*/Makefile.

Beside "prelink.o" been at a different location, no other functional
change intended.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v9:
    - set ALL_OBJS and ALL_LIBS on make command line instead of exporting
      them.
    - arm/Rules.mk, have one rule instead of two for head.o
    - fix typo in title
    
    v8:
    - rebased
    - move Arm specific dependencies between $(ALL_OBJS) objects (head.o)
      into Arm specific "Rules.mk" instead of the common "build.mk".
    
    v7:
    - change, now things are in build.mk: no more prepare phase needed

 xen/Makefile          | 12 +++++++++++-
 xen/Rules.mk          | 13 -------------
 xen/arch/arm/Makefile | 31 ++++---------------------------
 xen/arch/arm/Rules.mk |  4 ++++
 xen/arch/arm/arch.mk  |  2 ++
 xen/arch/x86/Makefile | 29 ++++++-----------------------
 xen/arch/x86/arch.mk  |  2 ++
 xen/build.mk          | 18 ++++++++++++++++++
 8 files changed, 47 insertions(+), 64 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index fb37043d08e0..d953dc50ac6a 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -285,6 +285,16 @@ CFLAGS += -flto
 LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
 endif
 
+# Note that link order matters!
+ALL_OBJS-y                := common/built_in.o
+ALL_OBJS-y                += drivers/built_in.o
+ALL_OBJS-y                += lib/built_in.o
+ALL_OBJS-y                += xsm/built_in.o
+ALL_OBJS-y                += arch/$(TARGET_ARCH)/built_in.o
+ALL_OBJS-$(CONFIG_CRYPTO) += crypto/built_in.o
+
+ALL_LIBS-y                := lib/lib.a
+
 include $(BASEDIR)/arch/$(TARGET_ARCH)/arch.mk
 
 # define new variables to avoid the ones defined in Config.mk
@@ -407,7 +417,7 @@ $(TARGET): FORCE
 	$(MAKE) -f $(BASEDIR)/Rules.mk -C include
 	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) include
 	$(MAKE) -f $(BASEDIR)/Rules.mk arch/$(TARGET_ARCH)/include/asm/asm-offsets.h
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) MKRELOC=$(MKRELOC) $@
+	$(MAKE) -f $(BASEDIR)/Rules.mk MKRELOC=$(MKRELOC) "ALL_OBJS=$(ALL_OBJS-y)" "ALL_LIBS=$(ALL_LIBS-y)" $@
 
 SUBDIRS = xsm arch/$(TARGET_ARCH) common drivers lib test
 define all_sources
diff --git a/xen/Rules.mk b/xen/Rules.mk
index 7b8b9047cfd5..77d359bedaf8 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -8,25 +8,12 @@
 include $(XEN_ROOT)/Config.mk
 include $(BASEDIR)/scripts/Kbuild.include
 
-# Note that link order matters!
-ALL_OBJS-y               += $(BASEDIR)/common/built_in.o
-ALL_OBJS-y               += $(BASEDIR)/drivers/built_in.o
-ALL_OBJS-y               += $(BASEDIR)/lib/built_in.o
-ALL_OBJS-y               += $(BASEDIR)/xsm/built_in.o
-ALL_OBJS-y               += $(BASEDIR)/arch/$(TARGET_ARCH)/built_in.o
-ALL_OBJS-$(CONFIG_CRYPTO)   += $(BASEDIR)/crypto/built_in.o
-
-ALL_LIBS-y               := $(BASEDIR)/lib/lib.a
-
 # Initialise some variables
 lib-y :=
 targets :=
 CFLAGS-y :=
 AFLAGS-y :=
 
-ALL_OBJS := $(ALL_OBJS-y)
-ALL_LIBS := $(ALL_LIBS-y)
-
 SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
                                             $(foreach w,1 2 4, \
                                                         rodata.str$(w).$(a)) \
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index d0dee10102b6..14952275772b 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -75,14 +75,6 @@ ifneq ($(CONFIG_DTB_FILE),"")
 obj-y += dtb.o
 endif
 
-ALL_OBJS := $(TARGET_SUBARCH)/head.o $(ALL_OBJS)
-
-# head.o is built by descending into the sub-directory, depends on the part of
-# $(ALL_OBJS) that will eventually recurse into $(TARGET_SUBARCH)/ and build
-# head.o
-$(TARGET_SUBARCH)/head.o: $(BASEDIR)/arch/arm/built_in.o
-$(TARGET_SUBARCH)/head.o: ;
-
 ifdef CONFIG_LIVEPATCH
 all_symbols = --all-symbols
 ifdef CONFIG_FAST_SYMBOL_LOOKUP
@@ -98,33 +90,18 @@ ifeq ($(CONFIG_ARM_64),y)
 	ln -sf $(@F) $@.efi
 endif
 
-ifeq ($(CONFIG_LTO),y)
-# Gather all LTO objects together
-prelink_lto.o: $(ALL_OBJS) $(ALL_LIBS)
-	$(LD_LTO) -r -o $@ $(filter-out %.a,$^) --start-group $(filter %.a,$^) --end-group
-
-# Link it with all the binary objects
-prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o
-	$(call if_changed,ld)
-else
-prelink.o: $(ALL_OBJS) $(ALL_LIBS) FORCE
-	$(call if_changed,ld)
-endif
-
-targets += prelink.o
-
-$(TARGET)-syms: prelink.o xen.lds
-	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o \
+$(TARGET)-syms: $(BASEDIR)/prelink.o xen.lds
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< \
 	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
-	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
-	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
 		| $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort \
diff --git a/xen/arch/arm/Rules.mk b/xen/arch/arm/Rules.mk
index e69de29bb2d1..c6463a433efd 100644
--- a/xen/arch/arm/Rules.mk
+++ b/xen/arch/arm/Rules.mk
@@ -0,0 +1,4 @@
+# head.o is built by descending into arch/arm/$(TARGET_SUBARCH), depends on the
+# part of $(ALL_OBJS) that will eventually recurse into $(TARGET_SUBARCH)/ and
+# build head.o
+arch/arm/$(TARGET_SUBARCH)/head.o: arch/arm/built_in.o ;
diff --git a/xen/arch/arm/arch.mk b/xen/arch/arm/arch.mk
index c3ac443b3788..ba3f140e2ea7 100644
--- a/xen/arch/arm/arch.mk
+++ b/xen/arch/arm/arch.mk
@@ -26,3 +26,5 @@ ifeq ($(CONFIG_ARM64_ERRATUM_843419),y)
         LDFLAGS += --fix-cortex-a53-843419
     endif
 endif
+
+ALL_OBJS-y := arch/arm/$(TARGET_SUBARCH)/head.o $(ALL_OBJS-y)
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 9fc884813cb5..a830b5791e8b 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -123,37 +123,20 @@ $(TARGET): $(TARGET)-syms $(efi-y) boot/mkelf32
 
 CFLAGS-$(XEN_BUILD_EFI) += -DXEN_BUILD_EFI
 
-ALL_OBJS := $(BASEDIR)/arch/x86/boot/built_in.o $(BASEDIR)/arch/x86/efi/built_in.o $(ALL_OBJS)
-
-ifeq ($(CONFIG_LTO),y)
-# Gather all LTO objects together
-prelink_lto.o: $(ALL_OBJS) $(ALL_LIBS)
-	$(LD_LTO) -r -o $@ $(filter-out %.a,$^) --start-group $(filter %.a,$^) --end-group
-
-# Link it with all the binary objects
-prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o FORCE
-	$(call if_changed,ld)
-else
-prelink.o: $(ALL_OBJS) $(ALL_LIBS) FORCE
-	$(call if_changed,ld)
-endif
-
-targets += prelink.o
-
-$(TARGET)-syms: prelink.o xen.lds
-	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+$(TARGET)-syms: $(BASEDIR)/prelink.o xen.lds
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< $(build_id_linker) \
 	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort \
 		>$(@D)/.$(@F).0.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
-	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< $(build_id_linker) \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort $(syms-warn-dup-y) \
 		>$(@D)/.$(@F).1.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
-	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
 		| $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort \
@@ -206,7 +189,7 @@ note_file_option ?= $(note_file)
 
 ifeq ($(XEN_BUILD_PE),y)
 extra-y += efi.lds
-$(TARGET).efi: prelink.o $(note_file) efi.lds efi/relocs-dummy.o efi/mkreloc
+$(TARGET).efi: $(BASEDIR)/prelink.o $(note_file) efi.lds efi/relocs-dummy.o efi/mkreloc
 ifeq ($(CONFIG_DEBUG_INFO),y)
 	$(if $(filter --strip-debug,$(EFI_LDFLAGS)),echo,:) "Will strip debug info from $(@F)"
 endif
@@ -235,7 +218,7 @@ $(TARGET).efi: FORCE
 	echo '$(if $(filter y,$(XEN_BUILD_EFI)),xen.efi generation,EFI support) disabled'
 endif
 
-efi/buildid.o efi/relocs-dummy.o: $(BASEDIR)/arch/x86/efi/built_in.o
+# These should already have been rebuilt when building the prerequisite of "prelink.o"
 efi/buildid.o efi/relocs-dummy.o: ;
 
 .PHONY: include
diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
index fa7cf3844362..bfd5eaa35f25 100644
--- a/xen/arch/x86/arch.mk
+++ b/xen/arch/x86/arch.mk
@@ -104,3 +104,5 @@ endif
 
 # Set up the assembler include path properly for older toolchains.
 CFLAGS += -Wa,-I$(BASEDIR)/include
+
+ALL_OBJS-y := arch/x86/boot/built_in.o arch/x86/efi/built_in.o $(ALL_OBJS-y)
diff --git a/xen/build.mk b/xen/build.mk
index 3d7a91df22d1..af1b28311393 100644
--- a/xen/build.mk
+++ b/xen/build.mk
@@ -59,3 +59,21 @@ arch/$(TARGET_ARCH)/include/asm/asm-offsets.h: asm-offsets.s
 	  sed -rne "/^[^#].*==>/{s:.*==>(.*)<==.*:\1:; s: [\$$#]: :; p;}"; \
 	  echo ""; \
 	  echo "#endif") <$< >$@
+
+ifeq ($(CONFIG_LTO),y)
+# Gather all LTO objects together
+prelink_lto.o: $(ALL_OBJS) $(ALL_LIBS)
+	$(LD_LTO) -r -o $@ $(filter-out %.a,$^) --start-group $(filter %.a,$^) --end-group
+
+# Link it with all the binary objects
+prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o FORCE
+	$(call if_changed,ld)
+else
+prelink.o: $(ALL_OBJS) $(ALL_LIBS) FORCE
+	$(call if_changed,ld)
+endif
+
+targets += prelink.o
+
+$(TARGET): prelink.o FORCE
+	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) $@
-- 
Anthony PERARD

[XEN PATCH v9 05/30] build: prepare to always invoke $(MAKE) from xen/, use $(obj)

[Anthony PERARD]

In a future patch, when building a subdirectory, we will set
"obj=$subdir" rather than change directory.

Before that, we add "$(obj)" and "$(src)" in as many places as
possible where we will need to know which subdirectory is been built.
"$(obj)" is for files been generated during the build, and "$(src)" is
for files present in the source tree.

For now, we set both to "." in Rules.mk and Makefile.clean.

A few places don't tolerate the addition of "./", this is because make
remove the leading "./" in targets and dependencies in rules, so these
will be change later.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
---

Notes:
    v8:
    - acked
    - rebased

 xen/Rules.mk                            |  5 ++-
 xen/arch/arm/Makefile                   | 14 ++++----
 xen/arch/x86/Makefile                   | 48 ++++++++++++-------------
 xen/arch/x86/boot/Makefile              | 14 ++++----
 xen/arch/x86/efi/Makefile               |  6 ++--
 xen/common/Makefile                     |  8 ++---
 xen/common/libelf/Makefile              |  4 +--
 xen/common/libfdt/Makefile              |  6 ++--
 xen/include/Makefile                    | 44 +++++++++++------------
 xen/scripts/Makefile.clean              |  5 ++-
 xen/xsm/flask/Makefile                  | 36 +++++++++----------
 xen/xsm/flask/policy/mkaccess_vector.sh |  7 ++--
 12 files changed, 103 insertions(+), 94 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index 77d359bedaf8..60d1d6c4f583 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -3,6 +3,9 @@
 # Makefile and are consumed by Rules.mk
 #
 
+obj := .
+src := $(obj)
+
 -include $(BASEDIR)/include/config/auto.conf
 
 include $(XEN_ROOT)/Config.mk
@@ -21,7 +24,7 @@ SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
                          $(foreach r,rel rel.ro,data.$(r).local)
 
 # The filename build.mk has precedence over Makefile
-include $(firstword $(wildcard build.mk) Makefile)
+include $(firstword $(wildcard $(src)/build.mk) $(src)/Makefile)
 
 # Linking
 # ---------------------------------------------------------------------------
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 14952275772b..c993ce72a341 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -90,18 +90,18 @@ ifeq ($(CONFIG_ARM_64),y)
 	ln -sf $(@F) $@.efi
 endif
 
-$(TARGET)-syms: $(BASEDIR)/prelink.o xen.lds
-	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< \
+$(TARGET)-syms: $(BASEDIR)/prelink.o $(obj)/xen.lds
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
 	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
-	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< \
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
-	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
 		| $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort \
@@ -111,13 +111,13 @@ $(TARGET)-syms: $(BASEDIR)/prelink.o xen.lds
 .PHONY: include
 include:
 
-xen.lds: xen.lds.S FORCE
+$(obj)/xen.lds: $(src)/xen.lds.S FORCE
 	$(call if_changed,cpp_lds_S)
 
-dtb.o: $(patsubst "%",%,$(CONFIG_DTB_FILE))
+$(obj)/dtb.o: $(patsubst "%",%,$(CONFIG_DTB_FILE))
 
 .PHONY: clean
 clean::
-	rm -f xen.lds
+	rm -f $(obj)/xen.lds
 	rm -f $(BASEDIR)/.xen-syms.[0-9]*
 	rm -f $(TARGET).efi
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index a830b5791e8b..db97ae8c07f0 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -81,7 +81,7 @@ extra-y += asm-macros.i
 extra-y += xen.lds
 
 ifneq ($(CONFIG_HVM),y)
-x86_emulate.o: CFLAGS-y += -Wno-unused-label
+$(obj)/x86_emulate.o: CFLAGS-y += -Wno-unused-label
 endif
 
 efi-y := $(shell if [ ! -r $(BASEDIR)/include/xen/compile.h -o \
@@ -112,8 +112,8 @@ syms-warn-dup-$(CONFIG_SUPPRESS_DUPLICATE_SYMBOL_WARNINGS) :=
 syms-warn-dup-$(CONFIG_ENFORCE_UNIQUE_SYMBOLS) := --error-dup
 
 $(TARGET): TMP = $(@D)/.$(@F).elf32
-$(TARGET): $(TARGET)-syms $(efi-y) boot/mkelf32
-	./boot/mkelf32 $(notes_phdrs) $(TARGET)-syms $(TMP) $(XEN_IMG_OFFSET) \
+$(TARGET): $(TARGET)-syms $(efi-y) $(obj)/boot/mkelf32
+	$(obj)/boot/mkelf32 $(notes_phdrs) $(TARGET)-syms $(TMP) $(XEN_IMG_OFFSET) \
 	               `$(NM) $(TARGET)-syms | sed -ne 's/^\([^ ]*\) . __2M_rwdata_end$$/0x\1/p'`
 	od -t x4 -N 8192 $(TMP)  | grep 1badb002 > /dev/null || \
 		{ echo "No Multiboot1 header found" >&2; false; }
@@ -123,27 +123,27 @@ $(TARGET): $(TARGET)-syms $(efi-y) boot/mkelf32
 
 CFLAGS-$(XEN_BUILD_EFI) += -DXEN_BUILD_EFI
 
-$(TARGET)-syms: $(BASEDIR)/prelink.o xen.lds
-	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< $(build_id_linker) \
+$(TARGET)-syms: $(BASEDIR)/prelink.o $(obj)/xen.lds
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
 	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort \
 		>$(@D)/.$(@F).0.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
-	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort $(syms-warn-dup-y) \
 		>$(@D)/.$(@F).1.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
-	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
 		| $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort \
 		>$(@D)/$(@F).map
 	rm -f $(@D)/.$(@F).[0-9]* $(@D)/..$(@F).[0-9]*
 
-note.o: $(TARGET)-syms
+$(obj)/note.o: $(TARGET)-syms
 	$(OBJCOPY) -O binary --only-section=.note.gnu.build-id $< $@.bin
 	$(OBJCOPY) -I binary -O elf64-x86-64 -B i386:x86-64 \
 		--rename-section=.data=.note.gnu.build-id -S $@.bin $@
@@ -162,25 +162,25 @@ ifdef SOURCE_DATE_EPOCH
 EFI_LDFLAGS += --no-insert-timestamp
 endif
 
-$(TARGET).efi: VIRT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A VIRT_START$$,,p')
+$(TARGET).efi: VIRT_BASE = 0x$(shell $(NM) $(obj)/efi/relocs-dummy.o | sed -n 's, A VIRT_START$$,,p')
 ifeq ($(MKRELOC),:)
 relocs-dummy :=
 $(TARGET).efi: ALT_BASE :=
 else
-relocs-dummy := efi/relocs-dummy.o
-$(TARGET).efi: ALT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A ALT_START$$,,p')
+relocs-dummy := $(obj)/efi/relocs-dummy.o
+$(TARGET).efi: ALT_BASE = 0x$(shell $(NM) $(obj)/efi/relocs-dummy.o | sed -n 's, A ALT_START$$,,p')
 endif
 
 ifneq ($(build_id_linker),)
 ifeq ($(call ld-ver-build-id,$(LD) $(filter -m%,$(EFI_LDFLAGS))),y)
 CFLAGS-y += -DBUILD_ID_EFI
 EFI_LDFLAGS += $(build_id_linker)
-note_file := efi/buildid.o
+note_file := $(obj)/efi/buildid.o
 # NB: this must be the last input in the linker call, because inputs following
 # the -b option will all be treated as being in the specified format.
 note_file_option := -b pe-x86-64 $(note_file)
 else
-note_file := note.o
+note_file := $(obj)/note.o
 endif
 else
 note_file :=
@@ -189,25 +189,25 @@ note_file_option ?= $(note_file)
 
 ifeq ($(XEN_BUILD_PE),y)
 extra-y += efi.lds
-$(TARGET).efi: $(BASEDIR)/prelink.o $(note_file) efi.lds efi/relocs-dummy.o efi/mkreloc
+$(TARGET).efi: $(BASEDIR)/prelink.o $(note_file) $(obj)/efi.lds $(obj)/efi/relocs-dummy.o $(obj)/efi/mkreloc
 ifeq ($(CONFIG_DEBUG_INFO),y)
 	$(if $(filter --strip-debug,$(EFI_LDFLAGS)),echo,:) "Will strip debug info from $(@F)"
 endif
 	$(foreach base, $(VIRT_BASE) $(ALT_BASE), \
-	          $(LD) $(call EFI_LDFLAGS,$(base)) -T efi.lds -N $< $(relocs-dummy) \
+	          $(LD) $(call EFI_LDFLAGS,$(base)) -T $(obj)/efi.lds -N $< $(relocs-dummy) \
 	                $(BASEDIR)/common/symbols-dummy.o $(note_file_option) -o $(@D)/.$(@F).$(base).0 &&) :
 	$(MKRELOC) $(foreach base,$(VIRT_BASE) $(ALT_BASE),$(@D)/.$(@F).$(base).0) >$(@D)/.$(@F).0r.S
 	$(NM) -pa --format=sysv $(@D)/.$(@F).$(VIRT_BASE).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0s.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0r.o $(@D)/.$(@F).0s.o
 	$(foreach base, $(VIRT_BASE) $(ALT_BASE), \
-	          $(LD) $(call EFI_LDFLAGS,$(base)) -T efi.lds -N $< \
+	          $(LD) $(call EFI_LDFLAGS,$(base)) -T $(obj)/efi.lds -N $< \
 	                $(@D)/.$(@F).0r.o $(@D)/.$(@F).0s.o $(note_file_option) -o $(@D)/.$(@F).$(base).1 &&) :
 	$(MKRELOC) $(foreach base,$(VIRT_BASE) $(ALT_BASE),$(@D)/.$(@F).$(base).1) >$(@D)/.$(@F).1r.S
 	$(NM) -pa --format=sysv $(@D)/.$(@F).$(VIRT_BASE).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1s.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o
-	$(LD) $(call EFI_LDFLAGS,$(VIRT_BASE)) -T efi.lds -N $< \
+	$(LD) $(call EFI_LDFLAGS,$(VIRT_BASE)) -T $(obj)/efi.lds -N $< \
 	                $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o $(note_file_option) -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
 		| $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort >$(@D)/$(@F).map
@@ -219,14 +219,14 @@ $(TARGET).efi: FORCE
 endif
 
 # These should already have been rebuilt when building the prerequisite of "prelink.o"
-efi/buildid.o efi/relocs-dummy.o: ;
+$(obj)/efi/buildid.o $(obj)/efi/relocs-dummy.o: ;
 
 .PHONY: include
 include: $(BASEDIR)/arch/x86/include/asm/asm-macros.h
 
-asm-macros.i: CFLAGS-y += -D__ASSEMBLY__ -P
+$(obj)/asm-macros.i: CFLAGS-y += -D__ASSEMBLY__ -P
 
-$(BASEDIR)/arch/x86/include/asm/asm-macros.h: asm-macros.i Makefile
+$(BASEDIR)/arch/x86/include/asm/asm-macros.h: $(obj)/asm-macros.i $(src)/Makefile
 	echo '#if 0' >$@.new
 	echo '.if 0' >>$@.new
 	echo '#endif' >>$@.new
@@ -240,14 +240,14 @@ $(BASEDIR)/arch/x86/include/asm/asm-macros.h: asm-macros.i Makefile
 	echo '#endif' >>$@.new
 	$(call move-if-changed,$@.new,$@)
 
-efi.lds: AFLAGS-y += -DEFI
-xen.lds efi.lds: xen.lds.S FORCE
+$(obj)/efi.lds: AFLAGS-y += -DEFI
+$(obj)/xen.lds $(obj)/efi.lds: $(src)/xen.lds.S FORCE
 	$(call if_changed,cpp_lds_S)
 
-boot/mkelf32: boot/mkelf32.c
+$(obj)/boot/mkelf32: $(src)/boot/mkelf32.c
 	$(HOSTCC) $(HOSTCFLAGS) -o $@ $<
 
-efi/mkreloc: efi/mkreloc.c
+$(obj)/efi/mkreloc: $(src)/efi/mkreloc.c
 	$(HOSTCC) $(HOSTCFLAGS) -g -o $@ $<
 
 .PHONY: clean
diff --git a/xen/arch/x86/boot/Makefile b/xen/arch/x86/boot/Makefile
index d2eb277d428f..0aec8a464364 100644
--- a/xen/arch/x86/boot/Makefile
+++ b/xen/arch/x86/boot/Makefile
@@ -1,8 +1,8 @@
 obj-bin-y += head.o
 
-DEFS_H_DEPS = defs.h $(BASEDIR)/include/xen/stdbool.h
+DEFS_H_DEPS = $(src)/defs.h $(BASEDIR)/include/xen/stdbool.h
 
-CMDLINE_DEPS = $(DEFS_H_DEPS) video.h \
+CMDLINE_DEPS = $(DEFS_H_DEPS) $(src)/video.h \
 	       $(BASEDIR)/include/xen/kconfig.h \
 	       $(BASEDIR)/include/generated/autoconf.h
 
@@ -14,10 +14,10 @@ RELOC_DEPS = $(DEFS_H_DEPS) \
 	     $(BASEDIR)/include/xen/const.h \
 	     $(BASEDIR)/include/public/arch-x86/hvm/start_info.h
 
-head.o: cmdline.S reloc.S
+$(obj)/head.o: $(obj)/cmdline.S $(obj)/reloc.S
 
-cmdline.S: cmdline.c $(CMDLINE_DEPS) build32.lds
-	$(MAKE) -f build32.mk $@ CMDLINE_DEPS="$(CMDLINE_DEPS)"
+$(obj)/cmdline.S: $(src)/cmdline.c $(CMDLINE_DEPS) $(src)/build32.lds
+	$(MAKE) -f build32.mk -C $(obj) $(@F) CMDLINE_DEPS="$(CMDLINE_DEPS)"
 
-reloc.S: reloc.c $(RELOC_DEPS) build32.lds
-	$(MAKE) -f build32.mk $@ RELOC_DEPS="$(RELOC_DEPS)"
+$(obj)/reloc.S: $(src)/reloc.c $(RELOC_DEPS) $(src)/build32.lds
+	$(MAKE) -f build32.mk -C $(obj) $(@F) RELOC_DEPS="$(RELOC_DEPS)"
diff --git a/xen/arch/x86/efi/Makefile b/xen/arch/x86/efi/Makefile
index abae493bf344..e08b4d8e4808 100644
--- a/xen/arch/x86/efi/Makefile
+++ b/xen/arch/x86/efi/Makefile
@@ -3,16 +3,16 @@ CFLAGS-y += -fshort-wchar
 quiet_cmd_objcopy_o_ihex = OBJCOPY $@
 cmd_objcopy_o_ihex = $(OBJCOPY) -I ihex -O binary $< $@
 
-%.o: %.ihex FORCE
+$(obj)/%.o: $(src)/%.ihex FORCE
 	$(call if_changed,objcopy_o_ihex)
 
-boot.init.o: buildid.o
+$(obj)/boot.init.o: $(obj)/buildid.o
 
 EFIOBJ-y := boot.init.o pe.init.o ebmalloc.o runtime.o
 EFIOBJ-$(CONFIG_COMPAT) += compat.o
 
 $(call cc-option-add,cflags-stack-boundary,CC,-mpreferred-stack-boundary=4)
-$(EFIOBJ-y): CFLAGS_stack_boundary := $(cflags-stack-boundary)
+$(addprefix $(obj)/,$(EFIOBJ-y)): CFLAGS_stack_boundary := $(cflags-stack-boundary)
 
 obj-y := stub.o
 obj-$(XEN_BUILD_EFI) := $(filter-out %.init.o,$(EFIOBJ-y))
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 141d7d40d3dc..ca839118e4d1 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -75,13 +75,13 @@ obj-$(CONFIG_NEEDS_LIBELF) += libelf/
 obj-$(CONFIG_HAS_DEVICE_TREE) += libfdt/
 
 CONF_FILE := $(if $(patsubst /%,,$(KCONFIG_CONFIG)),$(BASEDIR)/)$(KCONFIG_CONFIG)
-config.gz: $(CONF_FILE)
+$(obj)/config.gz: $(CONF_FILE)
 	gzip -n -c $< >$@
 
-config_data.o: config.gz
+$(obj)/config_data.o: $(obj)/config.gz
 
-config_data.S: $(BASEDIR)/tools/binfile FORCE
-	$(call if_changed,binfile,config.gz xen_config_data)
+$(obj)/config_data.S: $(BASEDIR)/tools/binfile FORCE
+	$(call if_changed,binfile,$(obj)/config.gz xen_config_data)
 targets += config_data.S
 
 clean::
diff --git a/xen/common/libelf/Makefile b/xen/common/libelf/Makefile
index a92326c982e9..8a4522e4e141 100644
--- a/xen/common/libelf/Makefile
+++ b/xen/common/libelf/Makefile
@@ -7,10 +7,10 @@ OBJCOPYFLAGS := $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s))
 
 CFLAGS-y += -Wno-pointer-sign
 
-libelf.o: libelf-temp.o FORCE
+$(obj)/libelf.o: $(obj)/libelf-temp.o FORCE
 	$(call if_changed,objcopy)
 
-libelf-temp.o: $(libelf-objs) FORCE
+$(obj)/libelf-temp.o: $(addprefix $(obj)/,$(libelf-objs)) FORCE
 	$(call if_changed,ld)
 
 extra-y += libelf-temp.o $(libelf-objs)
diff --git a/xen/common/libfdt/Makefile b/xen/common/libfdt/Makefile
index 6bd207cf8ffa..6708af12e583 100644
--- a/xen/common/libfdt/Makefile
+++ b/xen/common/libfdt/Makefile
@@ -1,4 +1,4 @@
-include Makefile.libfdt
+include $(src)/Makefile.libfdt
 
 SECTIONS := text data $(SPECIAL_DATA_SECTIONS)
 OBJCOPYFLAGS := $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s))
@@ -8,10 +8,10 @@ nocov-y += libfdt.o
 
 CFLAGS-y += -I$(BASEDIR)/include/xen/libfdt/
 
-libfdt.o: libfdt-temp.o FORCE
+$(obj)/libfdt.o: $(obj)/libfdt-temp.o FORCE
 	$(call if_changed,objcopy)
 
-libfdt-temp.o: $(LIBFDT_OBJS) FORCE
+$(obj)/libfdt-temp.o: $(addprefix $(obj)/,$(LIBFDT_OBJS)) FORCE
 	$(call if_changed,ld)
 
 extra-y += libfdt-temp.o $(LIBFDT_OBJS)
diff --git a/xen/include/Makefile b/xen/include/Makefile
index 95daa8a28975..d2f5a956a11a 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -39,57 +39,57 @@ cppflags-$(CONFIG_X86)    += -m32
 
 endif
 
-public-$(CONFIG_X86) := $(wildcard public/arch-x86/*.h public/arch-x86/*/*.h)
-public-$(CONFIG_ARM) := $(wildcard public/arch-arm/*.h public/arch-arm/*/*.h)
+public-$(CONFIG_X86) := $(wildcard $(src)/public/arch-x86/*.h $(src)/public/arch-x86/*/*.h)
+public-$(CONFIG_ARM) := $(wildcard $(src)/public/arch-arm/*.h $(src)/public/arch-arm/*/*.h)
 
 .PHONY: all
-all: $(headers-y)
+all: $(addprefix $(obj)/,$(headers-y))
 
-compat/%.h: compat/%.i Makefile $(BASEDIR)/tools/compat-build-header.py
-	$(PYTHON) $(BASEDIR)/tools/compat-build-header.py <$< $@ >>$@.new; \
+$(obj)/compat/%.h: $(obj)/compat/%.i $(src)/Makefile $(BASEDIR)/tools/compat-build-header.py
+	$(PYTHON) $(BASEDIR)/tools/compat-build-header.py <$< $(patsubst $(obj)/%,%,$@) >>$@.new; \
 	mv -f $@.new $@
 
-compat/%.i: compat/%.c Makefile
+$(obj)/compat/%.i: $(obj)/compat/%.c $(src)/Makefile
 	$(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(XEN_CFLAGS)) $(cppflags-y) -o $@ $<
 
-compat/%.c: public/%.h xlat.lst Makefile $(BASEDIR)/tools/compat-build-source.py
+$(obj)/compat/%.c: $(src)/public/%.h $(src)/xlat.lst $(src)/Makefile $(BASEDIR)/tools/compat-build-source.py
 	mkdir -p $(@D)
-	$(PYTHON) $(BASEDIR)/tools/compat-build-source.py xlat.lst <$< >$@.new
+	$(PYTHON) $(BASEDIR)/tools/compat-build-source.py $(src)/xlat.lst <$< >$@.new
 	mv -f $@.new $@
 
-compat/.xlat/%.h: compat/%.h compat/.xlat/%.lst $(BASEDIR)/tools/get-fields.sh Makefile
+$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(BASEDIR)/tools/get-fields.sh $(src)/Makefile
 	export PYTHON=$(PYTHON); \
 	while read what name; do \
 		$(SHELL) $(BASEDIR)/tools/get-fields.sh "$$what" compat_$$name $< || exit $$?; \
 	done <$(patsubst compat/%,compat/.xlat/%,$(basename $<)).lst >$@.new
 	mv -f $@.new $@
 
-.PRECIOUS: compat/.xlat/%.lst
-compat/.xlat/%.lst: xlat.lst Makefile
+.PRECIOUS: $(obj)/compat/.xlat/%.lst
+$(obj)/compat/.xlat/%.lst: $(src)/xlat.lst $(src)/Makefile
 	mkdir -p $(@D)
 	grep -v '^[[:blank:]]*#' $< | sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,[[:blank:]]+$*\.h[[:blank:]]*$$,,p' >$@.new
 	$(call move-if-changed,$@.new,$@)
 
-xlat-y := $(shell sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,^[?!][[:blank:]]+[^[:blank:]]+[[:blank:]]+,,p' xlat.lst | uniq)
+xlat-y := $(shell sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,^[?!][[:blank:]]+[^[:blank:]]+[[:blank:]]+,,p' $(src)/xlat.lst | uniq)
 xlat-y := $(filter $(patsubst compat/%,%,$(headers-y)),$(xlat-y))
 
-compat/xlat.h: $(addprefix compat/.xlat/,$(xlat-y)) config/auto.conf Makefile
+$(obj)/compat/xlat.h: $(addprefix $(obj)/compat/.xlat/,$(xlat-y)) $(obj)/config/auto.conf $(src)/Makefile
 	cat $(filter %.h,$^) >$@.new
 	mv -f $@.new $@
 
 ifeq ($(XEN_TARGET_ARCH),$(XEN_COMPILE_ARCH))
 
-all: headers.chk headers99.chk headers++.chk
+all: $(obj)/headers.chk $(obj)/headers99.chk $(obj)/headers++.chk
 
-PUBLIC_HEADERS := $(filter-out public/arch-% public/dom0_ops.h, $(wildcard public/*.h public/*/*.h) $(public-y))
+PUBLIC_HEADERS := $(filter-out $(src)/public/arch-% $(src)/public/dom0_ops.h, $(wildcard $(src)/public/*.h $(src)/public/*/*.h) $(public-y))
 
-PUBLIC_C99_HEADERS := public/io/9pfs.h public/io/pvcalls.h
-PUBLIC_ANSI_HEADERS := $(filter-out public/%ctl.h public/xsm/% public/%hvm/save.h $(PUBLIC_C99_HEADERS), $(PUBLIC_HEADERS))
+PUBLIC_C99_HEADERS := $(src)/public/io/9pfs.h $(src)/public/io/pvcalls.h
+PUBLIC_ANSI_HEADERS := $(filter-out $(src)/public/%ctl.h $(src)/public/xsm/% $(src)/public/%hvm/save.h $(PUBLIC_C99_HEADERS), $(PUBLIC_HEADERS))
 
 public/io/9pfs.h-prereq := string
 public/io/pvcalls.h-prereq := string
 
-headers.chk: $(PUBLIC_ANSI_HEADERS) Makefile
+$(obj)/headers.chk: $(PUBLIC_ANSI_HEADERS) $(src)/Makefile
 	for i in $(filter %.h,$^); do \
 	    $(CC) -x c -ansi -Wall -Werror -include stdint.h \
 	          -S -o /dev/null $$i || exit 1; \
@@ -97,7 +97,7 @@ headers.chk: $(PUBLIC_ANSI_HEADERS) Makefile
 	done >$@.new
 	mv $@.new $@
 
-headers99.chk: $(PUBLIC_C99_HEADERS) Makefile
+$(obj)/headers99.chk: $(PUBLIC_C99_HEADERS) $(src)/Makefile
 	rm -f $@.new
 	$(foreach i, $(filter %.h,$^),                                        \
 	    echo "#include "\"$(i)\"                                          \
@@ -107,7 +107,7 @@ headers99.chk: $(PUBLIC_C99_HEADERS) Makefile
 	    || exit $$?; echo $(i) >> $@.new;)
 	mv $@.new $@
 
-headers++.chk: $(PUBLIC_HEADERS) Makefile
+$(obj)/headers++.chk: $(PUBLIC_HEADERS) $(src)/Makefile
 	rm -f $@.new
 	if ! $(CXX) -v >/dev/null 2>&1; then                                  \
 	    touch $@.new;                                                     \
@@ -116,7 +116,7 @@ headers++.chk: $(PUBLIC_HEADERS) Makefile
 	$(foreach i, $(filter %.h,$^),                                        \
 	    echo "#include "\"$(i)\"                                          \
 	    | $(CXX) -x c++ -std=gnu++98 -Wall -Werror -D__XEN_TOOLS__        \
-	      -include stdint.h -include public/xen.h                         \
+	      -include stdint.h -include $(src)/public/xen.h                  \
 	      $(foreach j, $($(i)-prereq), -include c$(j)) -S -o /dev/null -  \
 	    || exit $$?; echo $(i) >> $@.new;)
 	mv $@.new $@
@@ -126,7 +126,7 @@ endif
 ifeq ($(XEN_TARGET_ARCH),x86_64)
 .PHONY: lib-x86-all
 lib-x86-all:
-	$(MAKE) -C xen/lib/x86 all
+	$(MAKE) -C $(obj)/xen/lib/x86 all
 
 all: lib-x86-all
 endif
diff --git a/xen/scripts/Makefile.clean b/xen/scripts/Makefile.clean
index 8582ec35e4da..c3b0681611da 100644
--- a/xen/scripts/Makefile.clean
+++ b/xen/scripts/Makefile.clean
@@ -3,11 +3,14 @@
 # Cleaning up
 # ==========================================================================
 
+obj := .
+src := $(obj)
+
 clean::
 
 include $(BASEDIR)/scripts/Kbuild.include
 
-include Makefile
+include $(src)/Makefile
 
 # Figure out what we need to clean from the various variables
 # ==========================================================================
diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index 11c530dcf458..51fd37f6c4d5 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -4,46 +4,46 @@ obj-y += flask_op.o
 
 obj-y += ss/
 
-CFLAGS-y += -I./include
+CFLAGS-y += -I$(obj)/include
 
 AWK = awk
 
-FLASK_H_DEPEND = policy/security_classes policy/initial_sids
-AV_H_DEPEND = policy/access_vectors
+FLASK_H_DEPEND := $(addprefix $(src)/policy/,security_classes initial_sids)
+AV_H_DEPEND = $(src)/policy/access_vectors
 
-FLASK_H_FILES = include/flask.h include/class_to_string.h include/initial_sid_to_string.h
-AV_H_FILES = include/av_perm_to_string.h include/av_permissions.h
-ALL_H_FILES = $(FLASK_H_FILES) $(AV_H_FILES)
+FLASK_H_FILES := flask.h class_to_string.h initial_sid_to_string.h
+AV_H_FILES := av_perm_to_string.h av_permissions.h
+ALL_H_FILES := $(addprefix include/,$(FLASK_H_FILES) $(AV_H_FILES))
 
-$(obj-y) ss/built_in.o: $(ALL_H_FILES)
+$(addprefix $(obj)/,$(obj-y)) $(obj)/ss/built_in.o: $(addprefix $(obj)/,$(ALL_H_FILES))
 extra-y += $(ALL_H_FILES)
 
-mkflask := policy/mkflask.sh
+mkflask := $(src)/policy/mkflask.sh
 quiet_cmd_mkflask = MKFLASK $@
-cmd_mkflask = $(SHELL) $(mkflask) $(AWK) include $(FLASK_H_DEPEND)
+cmd_mkflask = $(SHELL) $(mkflask) $(AWK) $(obj)/include $(FLASK_H_DEPEND)
 
-$(subst include/,%/,$(FLASK_H_FILES)): $(FLASK_H_DEPEND) $(mkflask) FORCE
+$(addprefix $(obj)/%/,$(FLASK_H_FILES)): $(FLASK_H_DEPEND) $(mkflask) FORCE
 	$(call if_changed,mkflask)
 
-mkaccess := policy/mkaccess_vector.sh
+mkaccess := $(src)/policy/mkaccess_vector.sh
 quiet_cmd_mkaccess = MKACCESS VECTOR $@
-cmd_mkaccess = $(SHELL) $(mkaccess) $(AWK) $(AV_H_DEPEND)
+cmd_mkaccess = $(SHELL) $(mkaccess) $(AWK) $(obj)/include $(AV_H_DEPEND)
 
-$(subst include/,%/,$(AV_H_FILES)): $(AV_H_DEPEND) $(mkaccess) FORCE
+$(addprefix $(obj)/%/,$(AV_H_FILES)): $(AV_H_DEPEND) $(mkaccess) FORCE
 	$(call if_changed,mkaccess)
 
 obj-bin-$(CONFIG_XSM_FLASK_POLICY) += flask-policy.o
-flask-policy.o: policy.bin
+$(obj)/flask-policy.o: $(obj)/policy.bin
 
-flask-policy.S: BINFILE_FLAGS := -i
-flask-policy.S: $(BASEDIR)/tools/binfile FORCE
-	$(call if_changed,binfile,policy.bin xsm_flask_init_policy)
+$(obj)/flask-policy.S: BINFILE_FLAGS := -i
+$(obj)/flask-policy.S: $(BASEDIR)/tools/binfile FORCE
+	$(call if_changed,binfile,$(obj)/policy.bin xsm_flask_init_policy)
 targets += flask-policy.S
 
 FLASK_BUILD_DIR := $(CURDIR)
 POLICY_SRC := $(FLASK_BUILD_DIR)/xenpolicy-$(XEN_FULLVERSION)
 
-policy.bin: FORCE
+$(obj)/policy.bin: FORCE
 	$(MAKE) -f $(XEN_ROOT)/tools/flask/policy/Makefile.common \
 	        -C $(XEN_ROOT)/tools/flask/policy \
 	        FLASK_BUILD_DIR=$(FLASK_BUILD_DIR) POLICY_FILENAME=$(POLICY_SRC)
diff --git a/xen/xsm/flask/policy/mkaccess_vector.sh b/xen/xsm/flask/policy/mkaccess_vector.sh
index 942ede4713f1..ad9772193bff 100755
--- a/xen/xsm/flask/policy/mkaccess_vector.sh
+++ b/xen/xsm/flask/policy/mkaccess_vector.sh
@@ -8,9 +8,12 @@ set -e
 awk=$1
 shift
 
+output_dir=$1
+shift
+
 # output files
-av_permissions="include/av_permissions.h"
-av_perm_to_string="include/av_perm_to_string.h"
+av_permissions="$output_dir/av_permissions.h"
+av_perm_to_string="$output_dir/av_perm_to_string.h"
 
 cat $* | $awk "
 BEGIN	{
-- 
Anthony PERARD

[XEN PATCH v9 06/30] build: rework test/livepatch/Makefile

[Anthony PERARD]

This rework the livepatch/Makefile to make it less repetitive and make
use of the facilities. All the targets to be built are now listed in
$(extra-y) which will allow Rules.mk to build them without the need of
a local target in a future patch.

There are some changes/fixes in this patch:
- when "xen-syms" is used for a target, it is added to the dependency
  list of the target, which allow to rebuild the target when xen-syms
  changes. But if "xen-syms" is missing, make simply fails.
- modinfo.o wasn't removing it's $@.bin file like the other targets,
  this is now done.
- The command to build *.livepatch targets as been fixed to use
  $(XEN_LDFLAGS) rather than just $(LDFLAGS) which is a fallout from
  2740d96efdd3 ("xen/build: have the root Makefile generates the
  CFLAGS")

make will findout the dependencies of the *.livepatch files and thus
what to built by "looking" at the objects listed in the *-objs
variables. The actual dependencies is generated by the new
"multi-depend" macro.

"$(targets)" needs to be updated with the objects listed in the
different *-objs variables to allow make to load the .*.cmd dependency
files.

This patch copies the macro "multi_depend" from Linux 5.12, and rename
it to "multi-depend".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v9:
    - half acked, still need "livepatch" maintainer ack.
    
    v8:
    - rename multi_depend to multi-depend
    - use $() for single-letter make variable
    - re-indent one line

 xen/scripts/Kbuild.include  |   9 ++
 xen/test/livepatch/Makefile | 213 ++++++++++++++----------------------
 2 files changed, 91 insertions(+), 131 deletions(-)

diff --git a/xen/scripts/Kbuild.include b/xen/scripts/Kbuild.include
index f75d724ed7fd..73caf238d42c 100644
--- a/xen/scripts/Kbuild.include
+++ b/xen/scripts/Kbuild.include
@@ -151,3 +151,12 @@ why =                                                                        \
 
 echo-why = $(call escsq, $(strip $(why)))
 endif
+
+# Useful for describing the dependency of composite objects
+# Usage:
+#   $(call multi-depend, multi-used-targets, suffix-to-remove, suffix-to-add)
+define multi-depend
+$(foreach m, $(notdir $1), \
+	$(eval $(obj)/$(m): \
+	$(addprefix $(obj)/, $(foreach s, $3, $($(m:%$(strip $2)=%$(s)))))))
+endef
diff --git a/xen/test/livepatch/Makefile b/xen/test/livepatch/Makefile
index 148dddb90473..69fadccd01ea 100644
--- a/xen/test/livepatch/Makefile
+++ b/xen/test/livepatch/Makefile
@@ -12,81 +12,29 @@ CODE_ADDR=$(shell nm --defined $(1) | grep $(2) | awk '{print "0x"$$1}')
 CODE_SZ=$(shell nm --defined -S $(1) | grep $(2) | awk '{ print "0x"$$2}')
 
 .PHONY: default
-
-LIVEPATCH := xen_hello_world.livepatch
-LIVEPATCH_BYE := xen_bye_world.livepatch
-LIVEPATCH_REPLACE := xen_replace_world.livepatch
-LIVEPATCH_NOP := xen_nop.livepatch
-LIVEPATCH_NO_XEN_BUILDID := xen_no_xen_buildid.livepatch
-LIVEPATCH_PREPOST_HOOKS := xen_prepost_hooks.livepatch
-LIVEPATCH_PREPOST_HOOKS_FAIL := xen_prepost_hooks_fail.livepatch
-LIVEPATCH_ACTION_HOOKS := xen_action_hooks.livepatch
-LIVEPATCH_ACTION_HOOKS_NOFUNC := xen_action_hooks_nofunc.livepatch
-LIVEPATCH_ACTION_HOOKS_MARKER:= xen_action_hooks_marker.livepatch
-LIVEPATCH_ACTION_HOOKS_NOAPPLY:= xen_action_hooks_noapply.livepatch
-LIVEPATCH_ACTION_HOOKS_NOREVERT:= xen_action_hooks_norevert.livepatch
-LIVEPATCH_EXPECTATIONS:= xen_expectations.livepatch
-LIVEPATCH_EXPECTATIONS_FAIL:= xen_expectations_fail.livepatch
-
-LIVEPATCHES += $(LIVEPATCH)
-LIVEPATCHES += $(LIVEPATCH_BYE)
-LIVEPATCHES += $(LIVEPATCH_REPLACE)
-LIVEPATCHES += $(LIVEPATCH_NOP)
-LIVEPATCHES += $(LIVEPATCH_NO_XEN_BUILDID)
-LIVEPATCHES += $(LIVEPATCH_PREPOST_HOOKS)
-LIVEPATCHES += $(LIVEPATCH_PREPOST_HOOKS_FAIL)
-LIVEPATCHES += $(LIVEPATCH_ACTION_HOOKS)
-LIVEPATCHES += $(LIVEPATCH_ACTION_HOOKS_NOFUNC)
-LIVEPATCHES += $(LIVEPATCH_ACTION_HOOKS_MARKER)
-LIVEPATCHES += $(LIVEPATCH_ACTION_HOOKS_NOAPPLY)
-LIVEPATCHES += $(LIVEPATCH_ACTION_HOOKS_NOREVERT)
-LIVEPATCHES += $(LIVEPATCH_EXPECTATIONS)
-LIVEPATCHES += $(LIVEPATCH_EXPECTATIONS_FAIL)
-
-LIVEPATCH_DEBUG_DIR ?= $(DEBUG_DIR)/xen-livepatch
-
 build default: livepatch
 
-install: livepatch
-	$(INSTALL_DIR) $(DESTDIR)$(LIVEPATCH_DEBUG_DIR)
-	$(INSTALL_DATA) $(LIVEPATCHES) $(DESTDIR)$(LIVEPATCH_DEBUG_DIR)
-
-uninstall:
-	cd $(DESTDIR)$(LIVEPATCH_DEBUG_DIR) && rm -f $(LIVEPATCHES)
-
-.PHONY: clean
-clean::
-	rm -f *.o .*.o.d *.livepatch config.h expect_config.h
+extra-y += xen_hello_world.livepatch
+xen_hello_world-objs := xen_hello_world_func.o xen_hello_world.o note.o xen_note.o modinfo.o
+$(obj)/xen_hello_world.o: $(obj)/config.h
 
 #
 # To compute these values we need the binary files: xen-syms
 # and xen_hello_world_func.o to be already compiled.
 #
-.PHONY: config.h
-config.h: OLD_CODE_SZ=$(call CODE_SZ,$(BASEDIR)/xen-syms,xen_extra_version)
-config.h: NEW_CODE_SZ=$(call CODE_SZ,$<,xen_hello_world)
-config.h: MINOR_VERSION_SZ=$(call CODE_SZ,$(BASEDIR)/xen-syms,xen_minor_version)
-config.h: MINOR_VERSION_ADDR=$(call CODE_ADDR,$(BASEDIR)/xen-syms,xen_minor_version)
-config.h: xen_hello_world_func.o
+$(obj)/config.h: $(obj)/xen_hello_world_func.o
 	(set -e; \
-	 echo "#define NEW_CODE_SZ $(NEW_CODE_SZ)"; \
-	 echo "#define MINOR_VERSION_SZ $(MINOR_VERSION_SZ)"; \
-	 echo "#define MINOR_VERSION_ADDR $(MINOR_VERSION_ADDR)"; \
-	 echo "#define OLD_CODE_SZ $(OLD_CODE_SZ)") > $@
+	 echo "#define NEW_CODE_SZ $(call CODE_SZ,$<,xen_hello_world)"; \
+	 echo "#define MINOR_VERSION_SZ $(call CODE_SZ,$(BASEDIR)/xen-syms,xen_minor_version)"; \
+	 echo "#define MINOR_VERSION_ADDR $(call CODE_ADDR,$(BASEDIR)/xen-syms,xen_minor_version)"; \
+	 echo "#define OLD_CODE_SZ $(call CODE_SZ,$(BASEDIR)/xen-syms,xen_extra_version)") > $@
 
-xen_hello_world.o: config.h
-
-.PHONY: $(LIVEPATCH)
-$(LIVEPATCH): xen_hello_world_func.o xen_hello_world.o note.o xen_note.o modinfo.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH) $^
-
-.PHONY: modinfo.o
-modinfo.o:
+$(obj)/modinfo.o:
 	(set -e; \
 	 printf "LIVEPATCH_RULEZ\0") > $@.bin
 	$(OBJCOPY) $(OBJCOPY_MAGIC) \
 		   --rename-section=.data=.modinfo,alloc,load,readonly,data,contents -S $@.bin $@
-	#rm -f $@.bin
+	rm -f $@.bin
 
 #
 # This target is only accessible if CONFIG_LIVEPATCH is defined, which
@@ -97,9 +45,8 @@ modinfo.o:
 # not be built (it is for EFI builds), and that we do not have
 # the note.o.bin to muck with (as it gets deleted)
 #
-.PHONY: note.o
-note.o:
-	$(OBJCOPY) -O binary --only-section=.note.gnu.build-id $(BASEDIR)/xen-syms $@.bin
+$(obj)/note.o: $(BASEDIR)/xen-syms
+	$(OBJCOPY) -O binary --only-section=.note.gnu.build-id $< $@.bin
 	$(OBJCOPY) $(OBJCOPY_MAGIC) \
 		   --rename-section=.data=.livepatch.depends,alloc,load,readonly,data,contents -S $@.bin $@
 	rm -f $@.bin
@@ -108,9 +55,8 @@ note.o:
 # Append .livepatch.xen_depends section
 # with Xen build-id derived from xen-syms.
 #
-.PHONY: xen_note.o
-xen_note.o:
-	$(OBJCOPY) -O binary --only-section=.note.gnu.build-id $(BASEDIR)/xen-syms $@.bin
+$(obj)/xen_note.o: $(BASEDIR)/xen-syms
+	$(OBJCOPY) -O binary --only-section=.note.gnu.build-id $< $@.bin
 	$(OBJCOPY) $(OBJCOPY_MAGIC) \
 		   --rename-section=.data=.livepatch.xen_depends,alloc,load,readonly,data,contents -S $@.bin $@
 	rm -f $@.bin
@@ -119,102 +65,107 @@ xen_note.o:
 # Extract the build-id of the xen_hello_world.livepatch
 # (which xen_bye_world will depend on).
 #
-.PHONY: hello_world_note.o
-hello_world_note.o: $(LIVEPATCH)
-	$(OBJCOPY) -O binary --only-section=.note.gnu.build-id $(LIVEPATCH) $@.bin
+$(obj)/hello_world_note.o: $(obj)/xen_hello_world.livepatch
+	$(OBJCOPY) -O binary --only-section=.note.gnu.build-id $< $@.bin
 	$(OBJCOPY) $(OBJCOPY_MAGIC) \
 		   --rename-section=.data=.livepatch.depends,alloc,load,readonly,data,contents -S $@.bin $@
 	rm -f $@.bin
 
-xen_bye_world.o: config.h
 
-.PHONY: $(LIVEPATCH_BYE)
-$(LIVEPATCH_BYE): xen_bye_world_func.o xen_bye_world.o hello_world_note.o xen_note.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_BYE) $^
+extra-y += xen_bye_world.livepatch
+xen_bye_world-objs := xen_bye_world_func.o xen_bye_world.o hello_world_note.o xen_note.o
+$(obj)/xen_bye_world.o: $(obj)/config.h
 
-xen_replace_world.o: config.h
 
-.PHONY: $(LIVEPATCH_REPLACE)
-$(LIVEPATCH_REPLACE): xen_replace_world_func.o xen_replace_world.o note.o xen_note.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_REPLACE) $^
+extra-y += xen_replace_world.livepatch
+xen_replace_world-objs := xen_replace_world_func.o xen_replace_world.o note.o xen_note.o
+$(obj)/xen_replace_world.o: $(obj)/config.h
 
-xen_nop.o: config.h
 
-.PHONY: $(LIVEPATCH_NOP)
-$(LIVEPATCH_NOP): xen_nop.o note.o xen_note.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_NOP) $^
+extra-y += xen_nop.livepatch
+xen_nop-objs := xen_nop.o note.o xen_note.o
+$(obj)/xen_nop.o: $(obj)/config.h
 
 # This one always fails upon upload, because it deliberately
 # does not have a .livepatch.xen_depends (xen_note.o) section.
-xen_no_xen_buildid.o: config.h
-
-.PHONY: $(LIVEPATCH_NO_XEN_BUILDID)
-$(LIVEPATCH_NO_XEN_BUILDID): xen_nop.o note.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_NO_XEN_BUILDID) $^
+extra-y += xen_no_xen_buildid.livepatch
+xen_no_xen_buildid-objs := xen_nop.o note.o
 
-xen_prepost_hooks.o: config.h
+$(obj)/xen_prepost_hooks.o: $(obj)/config.h
 
-.PHONY: $(LIVEPATCH_PREPOST_HOOKS)
-$(LIVEPATCH_PREPOST_HOOKS): xen_prepost_hooks.o xen_hello_world_func.o note.o xen_note.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_PREPOST_HOOKS) $^
+extra-y += xen_prepost_hooks.livepatch
+xen_prepost_hooks-objs := xen_prepost_hooks.o xen_hello_world_func.o note.o xen_note.o
 
-xen_prepost_hooks_fail.o: config.h
+$(obj)/xen_prepost_hooks_fail.o: $(obj)/config.h
 
-.PHONY: $(LIVEPATCH_PREPOST_HOOKS_FAIL)
-$(LIVEPATCH_PREPOST_HOOKS_FAIL): xen_prepost_hooks_fail.o xen_hello_world_func.o note.o xen_note.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_PREPOST_HOOKS_FAIL) $^
+extra-y += xen_prepost_hooks_fail.livepatch
+xen_prepost_hooks_fail-objs := xen_prepost_hooks_fail.o xen_hello_world_func.o note.o xen_note.o
 
-xen_action_hooks.o: config.h
+$(obj)/xen_action_hooks.o: $(obj)/config.h
 
-.PHONY: $(LIVEPATCH_ACTION_HOOKS)
-$(LIVEPATCH_ACTION_HOOKS): xen_action_hooks.o xen_hello_world_func.o note.o xen_note.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_ACTION_HOOKS) $^
+extra-y += xen_action_hooks.livepatch
+xen_action_hooks-objs := xen_action_hooks.o xen_hello_world_func.o note.o xen_note.o
 
-xen_action_hooks_nofunc.o: config.h
+$(obj)/xen_action_hooks_nofunc.o: $(obj)/config.h
 
-.PHONY: $(LIVEPATCH_ACTION_HOOKS_NOFUNC)
-$(LIVEPATCH_ACTION_HOOKS_NOFUNC): xen_action_hooks_nofunc.o note.o xen_note.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_ACTION_HOOKS_NOFUNC) $^
+extra-y += xen_action_hooks_nofunc.livepatch
+xen_action_hooks_nofunc-objs := xen_action_hooks_nofunc.o note.o xen_note.o
 
-xen_action_hooks_marker.o: config.h
+$(obj)/xen_action_hooks_marker.o: $(obj)/config.h
 
-.PHONY: $(LIVEPATCH_ACTION_HOOKS_MARKER)
-$(LIVEPATCH_ACTION_HOOKS_MARKER): xen_action_hooks_marker.o xen_hello_world_func.o note.o xen_note.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_ACTION_HOOKS_MARKER) $^
+extra-y += xen_action_hooks_marker.livepatch
+xen_action_hooks_marker-objs := xen_action_hooks_marker.o xen_hello_world_func.o note.o xen_note.o
 
-xen_action_hooks_noapply.o: config.h
+$(obj)/xen_action_hooks_noapply.o: $(obj)/config.h
 
-.PHONY: $(LIVEPATCH_ACTION_HOOKS_NOAPPLY)
-$(LIVEPATCH_ACTION_HOOKS_NOAPPLY): xen_action_hooks_marker.o xen_hello_world_func.o note.o xen_note.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_ACTION_HOOKS_NOAPPLY) $^
+extra-y += xen_action_hooks_noapply.livepatch
+xen_action_hooks_noapply-objs := xen_action_hooks_marker.o xen_hello_world_func.o note.o xen_note.o
 
-xen_action_hooks_norevert.o: config.h
+$(obj)/xen_action_hooks_norevert.o: $(obj)/config.h
 
-.PHONY: $(LIVEPATCH_ACTION_HOOKS_NOREVERT)
-$(LIVEPATCH_ACTION_HOOKS_NOREVERT): xen_action_hooks_marker.o xen_hello_world_func.o note.o xen_note.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_ACTION_HOOKS_NOREVERT) $^
+extra-y += xen_action_hooks_norevert.livepatch
+xen_action_hooks_norevert-objs := xen_action_hooks_marker.o xen_hello_world_func.o note.o xen_note.o
 
 EXPECT_BYTES_COUNT := 8
 CODE_GET_EXPECT=$(shell $(OBJDUMP) -d --insn-width=1 $(1) | sed -n -e '/<'$(2)'>:$$/,/^$$/ p' | tail -n +2 | head -n $(EXPECT_BYTES_COUNT) | awk '{$$0=$$2; printf "%s", substr($$0,length-1)}' | sed 's/.\{2\}/0x&,/g' | sed 's/^/{/;s/,$$/}/g')
-.PHONY: expect_config.h
-expect_config.h: EXPECT_BYTES=$(call CODE_GET_EXPECT,$(BASEDIR)/xen-syms,xen_extra_version)
-expect_config.h:
+$(obj)/expect_config.h: $(BASEDIR)/xen-syms
 	(set -e; \
-	 echo "#define EXPECT_BYTES $(EXPECT_BYTES)"; \
+	 echo "#define EXPECT_BYTES $(call CODE_GET_EXPECT,$<,xen_extra_version)"; \
          echo "#define EXPECT_BYTES_COUNT $(EXPECT_BYTES_COUNT)") > $@
 
-xen_expectations.o: expect_config.h
+$(obj)/xen_expectations.o: $(obj)/expect_config.h
 
-.PHONY: $(LIVEPATCH_EXPECTATIONS)
-$(LIVEPATCH_EXPECTATIONS): xen_expectations.o xen_hello_world_func.o note.o xen_note.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_EXPECTATIONS) $^
+extra-y += xen_expectations.livepatch
+xen_expectations-objs := xen_expectations.o xen_hello_world_func.o note.o xen_note.o
 
-.PHONY: $(LIVEPATCH_EXPECTATIONS_FAIL)
-$(LIVEPATCH_EXPECTATIONS_FAIL): xen_expectations_fail.o xen_hello_world_func.o note.o xen_note.o
-	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_EXPECTATIONS_FAIL) $^
+extra-y += xen_expectations_fail.livepatch
+xen_expectations_fail-objs := xen_expectations_fail.o xen_hello_world_func.o note.o xen_note.o
+
+
+quiet_cmd_livepatch = LD      $@
+cmd_livepatch = $(LD) $(XEN_LDFLAGS) $(build_id_linker) -r -o $@ $(real-prereqs)
+
+$(obj)/%.livepatch: FORCE
+	$(call if_changed,livepatch)
+
+$(call multi-depend, $(filter %.livepatch,$(extra-y)), .livepatch, -objs)
+targets += $(sort $(foreach m,$(basename $(notdir $(filter %.livepatch,$(extra-y)))), \
+                    $($(m)-objs)))
+
+LIVEPATCHES := $(filter %.livepatch,$(extra-y))
+
+LIVEPATCH_DEBUG_DIR ?= $(DEBUG_DIR)/xen-livepatch
 
 .PHONY: livepatch
-livepatch: $(LIVEPATCH) $(LIVEPATCH_BYE) $(LIVEPATCH_REPLACE) $(LIVEPATCH_NOP) $(LIVEPATCH_NO_XEN_BUILDID) \
-           $(LIVEPATCH_PREPOST_HOOKS) $(LIVEPATCH_PREPOST_HOOKS_FAIL) $(LIVEPATCH_ACTION_HOOKS) \
-           $(LIVEPATCH_ACTION_HOOKS_NOFUNC) $(LIVEPATCH_ACTION_HOOKS_MARKER) $(LIVEPATCH_ACTION_HOOKS_NOAPPLY) \
-           $(LIVEPATCH_ACTION_HOOKS_NOREVERT) $(LIVEPATCH_EXPECTATIONS) $(LIVEPATCH_EXPECTATIONS_FAIL)
+livepatch: $(LIVEPATCHES)
+
+install: $(addprefix $(obj)/,$(LIVEPATCHES))
+	$(INSTALL_DIR) $(DESTDIR)$(LIVEPATCH_DEBUG_DIR)
+	$(INSTALL_DATA) $(addprefix $(obj)/,$(LIVEPATCHES)) $(DESTDIR)$(LIVEPATCH_DEBUG_DIR)
+
+uninstall:
+	cd $(DESTDIR)$(LIVEPATCH_DEBUG_DIR) && rm -f $(LIVEPATCHES)
+
+.PHONY: clean
+clean::
+	rm -f *.o .*.o.d *.livepatch config.h expect_config.h
-- 
Anthony PERARD

[XEN PATCH v9 07/30] build: rework cloc recipe

[Anthony PERARD]

We are going to make other modifications to the cloc recipe, so this
patch prepare make those modification easier.

We replace the Makefile meta programming by just a shell script which
should be easier to read and is actually faster to execute.

Instead of looking for files in "$(BASEDIR)", we use "." which is give
the same result overall. We also avoid the need for a temporary file
as cloc can read the list of files from stdin.

No change intended to the output of `cloc`.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v9:
    - acked
    
    v8:
    - rework cloc patch, move it ahead of problematic changes
      this is only a convertion to shell script with a single other change.

 xen/Makefile | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index d953dc50ac6a..ec24856a5d46 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -473,14 +473,12 @@ _MAP:
 
 .PHONY: cloc
 cloc:
-	$(eval tmpfile := $(shell mktemp))
-	$(foreach f, $(shell find $(BASEDIR) -name *.o.d), \
-		$(eval path := $(dir $(f))) \
-		$(eval names := $(shell grep -o "[a-zA-Z0-9_/-]*\.[cS]" $(f))) \
-		$(foreach sf, $(names), \
-			$(shell if test -f $(path)/$(sf) ; then echo $(path)/$(sf) >> $(tmpfile); fi;)))
-	cloc --list-file=$(tmpfile)
-	rm $(tmpfile)
+	find . -name '*.o.d' | while read f; do \
+	    for sf in $$(grep -o "[a-zA-Z0-9_/-]*\.[cS]" $$f); do \
+		sf="$$(dirname $$f)/$$sf"; \
+		test -f "$$sf" && echo "$$sf"; \
+	    done; \
+	done | cloc --list-file=-
 
 endif #config-build
 
-- 
Anthony PERARD

[XEN PATCH v9 08/30] build: fix enforce unique symbols for recent clang version

[Anthony PERARD]

clang 6.0 and newer behave like gcc in regards for the FILE symbol, so
only the filename rather than the full path to the source file.

clang 3.8.1-24 (in our debian:stretch container) and 3.5.0-10
(in our debian:jessie container) do store the full path to the source
file in the FILE symbol.

Also, based on commit 81ecb38b83 ("build: provide option to
disambiguate symbol names"), which were using clang 5, the change of
behavior likely happened in clang 6.0.

This means that we also need to check clang version to figure out
which command we need to use to redefine symbol.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

"enforce unique symbols" works by chance with recent clang version.
The few object built from source in subdir don't pose an issue.
---

Notes:
    v9:
    - checking for clang 6 instead of clang 4, based on 81ecb38b83, and
      update commit message.
    
    v8:
    - new patch, extracted from "build: build everything from the root dir, use obj=$subdir"

 xen/Rules.mk               | 2 +-
 xen/scripts/Kbuild.include | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index 60d1d6c4f583..1e7f47a3d8a8 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -166,7 +166,7 @@ SRCPATH := $(patsubst $(BASEDIR)/%,%,$(CURDIR))
 quiet_cmd_cc_o_c = CC      $@
 ifeq ($(CONFIG_ENFORCE_UNIQUE_SYMBOLS),y)
     cmd_cc_o_c = $(CC) $(c_flags) -c $< -o $(dot-target).tmp -MQ $@
-    ifeq ($(CONFIG_CC_IS_CLANG),y)
+    ifeq ($(CONFIG_CC_IS_CLANG)$(call clang-ifversion,-lt,600,y),yy)
         cmd_objcopy_fix_sym = $(OBJCOPY) --redefine-sym $<=$(SRCPATH)/$< $(dot-target).tmp $@
     else
         cmd_objcopy_fix_sym = $(OBJCOPY) --redefine-sym $(<F)=$(SRCPATH)/$< $(dot-target).tmp $@
diff --git a/xen/scripts/Kbuild.include b/xen/scripts/Kbuild.include
index 73caf238d42c..4875bb28c282 100644
--- a/xen/scripts/Kbuild.include
+++ b/xen/scripts/Kbuild.include
@@ -59,6 +59,8 @@ ld-option = $(call success,$(LD) -v $(1))
 # Usage:  EXTRA_CFLAGS += $(call cc-ifversion, -lt, 0402, -O1)
 cc-ifversion = $(shell [ $(CONFIG_GCC_VERSION)0 $(1) $(2)000 ] && echo $(3) || echo $(4))
 
+clang-ifversion = $(shell [ $(CONFIG_CLANG_VERSION)0 $(1) $(2)000 ] && echo $(3) || echo $(4))
+
 # Shorthand for $(MAKE) clean
 # Usage:
 # $(MAKE) $(clean) dir
-- 
Anthony PERARD

[XEN PATCH v9 09/30] build: build everything from the root dir, use obj=$subdir

[Anthony PERARD]

A subdirectory is now built by setting "$(obj)" instead of changing
directory. "$(obj)" should always be set when using "Rules.mk" and
thus a shortcut "$(build)" is introduced and should be used.

A new variable "$(need-builtin)" is introduce. It is to be used
whenever a "built_in.o" is wanted from a subdirectory. "built_in.o"
isn't the main target anymore, and thus only needs to depends on the
objects that should be part of "built_in.o".

Introduce $(srctree) and $(objtree) to replace $(BASEDIR) in cases a
relative path is better, and $(abs_srctree) and $(abs_objtree) which
have an absolute path.

DEPS is updated as the existing macro to deal with it doesn't know
about $(obj).

There's some changes in "Rules.mk" which in addition to deal with
"$(obj)" also make it's looks more like "Makefile.build" from Linux
v5.12.

test/Makefile doesn't need special handling in order to build
everything under test/, Rules.mk will visit test/livepatch via
$(subdir-y), thus "tests" "all" and "build" target are removed.
"subtree-force-update" target isn't useful so it is removed as well.

test/livepatch/Makefile doesn't need default target anymore, Rules.mk
will build everything in $(extra-y) and thus all *.livepatch.

Adjust cloc recipe: dependency files generated by CC will now have the
full path to the source file, so we don't need to prepend the
subdirectory. This fix some issue with source not been parsed by cloc
before. Also source from tools/kconfig would be listed with changes in
this patch so adjust the find command to stop listing the "tools"
directory and thus kconfig. With a default build of Xen on X86, they
are a few new files parsed by cloc:
    arch/x86/x86_64/compat/mm.c
    arch/x86/x86_64/mm.c
    common/compat/domain.c
    common/compat/memory.c
    common/compat/xlat.c

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Bob Eshleman <bobbyeshleman@gmail.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v9:
    - reviewed
    
    v8:
    - rebased
    - move check for $(obj) in Rules.mk earlier
    - use $(sort ) when adding directories in $(subdir-y)
    - fix missing space between "DEPS" and ":="
    - use -iquote when adding '.' for xen/arch/x86/usercopy.o
    - introduce also abs_srctree and abs_objtree
      those were in patch "build: replace $(BASEDIR) by $(objtree)"
    - reworked cmd_objcopy_fix_sym, we don't need to do anything for old
      version of clang, and for recent version of clang the change is the
      same a for gcc.
    - adjust "cloc" recipe
    - add some more $(Q) to silent $(MAKE) $(build)=...
    
    v7:
    - Spell "Makefile.build" as "Rules.mk" in the warning in Rules.mk about
      undefined $(obj).
    - use $(srctree) for include/xen/config.h
    - handle $(nocov-y) and $(noubsan-y), needed to add $(obj)/

 xen/Makefile                |  36 ++++++----
 xen/Rules.mk                | 138 ++++++++++++++++++++++++------------
 xen/arch/arm/Makefile       |   4 +-
 xen/arch/arm/Rules.mk       |   4 --
 xen/arch/arm/arch.mk        |   4 +-
 xen/arch/riscv/arch.mk      |   4 +-
 xen/arch/x86/Makefile       |  11 +--
 xen/arch/x86/Rules.mk       |   4 +-
 xen/arch/x86/arch.mk        |  12 ++--
 xen/arch/x86/boot/Makefile  |   8 +--
 xen/build.mk                |  12 +++-
 xen/include/Makefile        |   6 +-
 xen/scripts/Kbuild.include  |   6 ++
 xen/test/Makefile           |   7 +-
 xen/test/livepatch/Makefile |   6 --
 xen/xsm/flask/Makefile      |   2 +-
 xen/xsm/flask/ss/Makefile   |   2 +-
 17 files changed, 162 insertions(+), 104 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index ec24856a5d46..780d555e8620 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -22,6 +22,15 @@ export CHECKPOLICY	?= checkpolicy
 export BASEDIR := $(CURDIR)
 export XEN_ROOT := $(BASEDIR)/..
 
+abs_objtree := $(CURDIR)
+abs_srctree := $(CURDIR)
+
+export abs_srctree abs_objtree
+
+srctree := .
+objtree := .
+export srctree objtree
+
 # Do not use make's built-in rules and variables
 MAKEFLAGS += -rR
 
@@ -47,7 +56,7 @@ export KCONFIG_CONFIG ?= .config
 
 export CC CXX LD
 
-export TARGET := $(BASEDIR)/xen
+export TARGET := xen
 
 .PHONY: default
 default: build
@@ -250,7 +259,7 @@ endif
 CFLAGS += -nostdinc -fno-builtin -fno-common
 CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
 $(call cc-option-add,CFLAGS,CC,-Wvla)
-CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
+CFLAGS += -pipe -D__XEN__ -include $(srctree)/include/xen/config.h
 CFLAGS-$(CONFIG_DEBUG_INFO) += -g
 
 ifneq ($(CONFIG_CC_IS_CLANG),y)
@@ -349,10 +358,10 @@ _install: $(TARGET)$(CONFIG_XEN_INSTALL_SUFFIX)
 
 .PHONY: tests
 tests:
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C test tests
+	$(Q)$(MAKE) $(build)=test
 .PHONY: install-tests
 install-tests:
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C test install
+	$(Q)$(MAKE) $(build)=test install
 
 .PHONY: _uninstall
 _uninstall: D=$(DESTDIR)
@@ -408,16 +417,16 @@ $(TARGET).gz: $(TARGET)
 
 $(TARGET): FORCE
 	$(MAKE) -C tools
-	$(MAKE) -f $(BASEDIR)/Rules.mk include/xen/compile.h
+	$(Q)$(MAKE) $(build)=. include/xen/compile.h
 	[ -e arch/$(TARGET_ARCH)/efi ] && for f in $$(cd common/efi; echo *.[ch]); \
 		do test -r arch/$(TARGET_ARCH)/efi/$$f || \
 		   ln -nsf ../../../common/efi/$$f arch/$(TARGET_ARCH)/efi/; \
 		done; \
 		true
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C include
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) include
-	$(MAKE) -f $(BASEDIR)/Rules.mk arch/$(TARGET_ARCH)/include/asm/asm-offsets.h
-	$(MAKE) -f $(BASEDIR)/Rules.mk MKRELOC=$(MKRELOC) "ALL_OBJS=$(ALL_OBJS-y)" "ALL_LIBS=$(ALL_LIBS-y)" $@
+	$(Q)$(MAKE) $(build)=include all
+	$(Q)$(MAKE) $(build)=arch/$(TARGET_ARCH) include
+	$(Q)$(MAKE) $(build)=. arch/$(TARGET_ARCH)/include/asm/asm-offsets.h
+	$(Q)$(MAKE) $(build)=. MKRELOC=$(MKRELOC) "ALL_OBJS=$(ALL_OBJS-y)" "ALL_LIBS=$(ALL_LIBS-y)" $@
 
 SUBDIRS = xsm arch/$(TARGET_ARCH) common drivers lib test
 define all_sources
@@ -463,19 +472,18 @@ _MAP:
 	$(NM) -n $(TARGET)-syms | grep -v '\(compiled\)\|\(\.o$$\)\|\( [aUw] \)\|\(\.\.ng$$\)\|\(LASH[RL]DI\)' > System.map
 
 %.o %.i %.s: %.c FORCE
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C $(*D) $(@F)
+	$(Q)$(MAKE) $(build)=$(*D) $(*D)/$(@F)
 
 %.o %.s: %.S FORCE
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C $(*D) $(@F)
+	$(Q)$(MAKE) $(build)=$(*D) $(*D)/$(@F)
 
 %/: FORCE
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C $* built_in.o built_in_bin.o
+	$(Q)$(MAKE) $(build)=$* need-builtin=1
 
 .PHONY: cloc
 cloc:
-	find . -name '*.o.d' | while read f; do \
+	find . -name tools -prune -o -name '*.o.d' -print | while read f; do \
 	    for sf in $$(grep -o "[a-zA-Z0-9_/-]*\.[cS]" $$f); do \
-		sf="$$(dirname $$f)/$$sf"; \
 		test -f "$$sf" && echo "$$sf"; \
 	    done; \
 	done | cloc --list-file=-
diff --git a/xen/Rules.mk b/xen/Rules.mk
index 1e7f47a3d8a8..67112e00773f 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -3,19 +3,29 @@
 # Makefile and are consumed by Rules.mk
 #
 
-obj := .
+ifndef obj
+$(warning kbuild: Rules.mk is included improperly)
+endif
+
 src := $(obj)
 
+PHONY := __build
+__build:
+
 -include $(BASEDIR)/include/config/auto.conf
 
 include $(XEN_ROOT)/Config.mk
 include $(BASEDIR)/scripts/Kbuild.include
 
 # Initialise some variables
+obj-y :=
 lib-y :=
 targets :=
+subdir-y :=
 CFLAGS-y :=
 AFLAGS-y :=
+nocov-y :=
+noubsan-y :=
 
 SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
                                             $(foreach w,1 2 4, \
@@ -50,27 +60,54 @@ cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS) $< $@
 quiet_cmd_binfile = BINFILE $@
 cmd_binfile = $(SHELL) $(BASEDIR)/tools/binfile $(BINFILE_FLAGS) $@ $(2)
 
-define gendep
-    ifneq ($(1),$(subst /,:,$(1)))
-        DEPS += $(dir $(1)).$(notdir $(1)).d
-    endif
-endef
-$(foreach o,$(filter-out %/,$(obj-y) $(obj-bin-y) $(extra-y)),$(eval $(call gendep,$(o))))
+# Figure out what we need to build from the various variables
+# ===========================================================================
+
+# Libraries are always collected in one lib file.
+# Filter out objects already built-in
+lib-y := $(filter-out $(obj-y), $(sort $(lib-y)))
+
+# Subdirectories we need to descend into
+subdir-y := $(sort $(subdir-y) $(patsubst %/,%,$(filter %/, $(obj-y))))
 
 # Handle objects in subdirs
-# ---------------------------------------------------------------------------
-# o if we encounter foo/ in $(obj-y), replace it by foo/built_in.o
-#   and add the directory to the list of dirs to descend into: $(subdir-y)
-subdir-y := $(subdir-y) $(filter %/, $(obj-y))
+# - if we encounter foo/ in $(obj-y), replace it by foo/built_in.o
+ifdef need-builtin
 obj-y    := $(patsubst %/, %/built_in.o, $(obj-y))
+else
+obj-y    := $(filter-out %/, $(obj-y))
+endif
 
-# $(subdir-obj-y) is the list of objects in $(obj-y) which uses dir/ to
-# tell kbuild to descend
-subdir-obj-y := $(filter %/built_in.o, $(obj-y))
+# Add subdir path
 
-# Libraries are always collected in one lib file.
-# Filter out objects already built-in
-lib-y := $(filter-out $(obj-y), $(sort $(lib-y)))
+extra-y         := $(addprefix $(obj)/,$(extra-y))
+targets         := $(addprefix $(obj)/,$(targets))
+lib-y           := $(addprefix $(obj)/,$(lib-y))
+obj-y           := $(addprefix $(obj)/,$(obj-y))
+obj-bin-y       := $(addprefix $(obj)/,$(obj-bin-y))
+subdir-y        := $(addprefix $(obj)/,$(subdir-y))
+nocov-y         := $(addprefix $(obj)/,$(nocov-y))
+noubsan-y       := $(addprefix $(obj)/,$(noubsan-y))
+
+# subdir-builtin may contain duplications. Use $(sort ...)
+subdir-builtin := $(sort $(filter %/built_in.o, $(obj-y)))
+
+targets-for-builtin := $(extra-y)
+
+ifneq ($(strip $(lib-y)),)
+    targets-for-builtin += $(obj)/lib.a
+endif
+
+ifdef need-builtin
+    targets-for-builtin += $(obj)/built_in.o
+    ifneq ($(strip $(obj-bin-y)),)
+        ifeq ($(CONFIG_LTO),y)
+            targets-for-builtin += $(obj)/built_in_bin.o
+        endif
+    endif
+endif
+
+targets += $(targets-for-builtin)
 
 $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS-y += -DINIT_SECTIONS_ONLY
 
@@ -122,29 +159,28 @@ quiet_cmd_cc_builtin = CC      $@
 cmd_cc_builtin = \
     $(CC) $(XEN_CFLAGS) -c -x c /dev/null -o $@
 
+# To build objects in subdirs, we need to descend into the directories
+$(subdir-builtin): $(obj)/%/built_in.o: $(obj)/% ;
+
 quiet_cmd_ld_builtin = LD      $@
 ifeq ($(CONFIG_LTO),y)
 cmd_ld_builtin = \
-    $(LD_LTO) -r -o $@ $(filter $(obj-y),$(real-prereqs))
+    $(LD_LTO) -r -o $@ $(real-prereqs)
 else
 cmd_ld_builtin = \
-    $(LD) $(XEN_LDFLAGS) -r -o $@ $(filter $(obj-y),$(real-prereqs))
+    $(LD) $(XEN_LDFLAGS) -r -o $@ $(real-prereqs)
 endif
 
-built_in.o: $(obj-y) $(if $(strip $(lib-y)),lib.a) $(extra-y) FORCE
+$(obj)/built_in.o: $(obj-y) FORCE
 	$(call if_changed,$(if $(strip $(obj-y)),ld_builtin,cc_builtin))
 
-lib.a: $(lib-y) FORCE
+$(obj)/lib.a: $(lib-y) FORCE
 	$(call if_changed,ar)
 
-targets += built_in.o
-ifneq ($(strip $(lib-y)),)
-targets += lib.a
-endif
-targets += $(filter-out $(subdir-obj-y), $(obj-y) $(lib-y)) $(extra-y)
-targets += $(MAKECMDGOALS)
+targets += $(filter-out $(subdir-builtin), $(obj-y))
+targets += $(lib-y) $(MAKECMDGOALS)
 
-built_in_bin.o: $(obj-bin-y) $(extra-y)
+$(obj)/built_in_bin.o: $(obj-bin-y)
 ifeq ($(strip $(obj-bin-y)),)
 	$(CC) $(a_flags) -c -x assembler /dev/null -o $@
 else
@@ -155,23 +191,15 @@ endif
 PHONY += FORCE
 FORCE:
 
-%/built_in.o %/lib.a: FORCE
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C $* built_in.o
-
-%/built_in_bin.o: FORCE
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C $* built_in_bin.o
-
-SRCPATH := $(patsubst $(BASEDIR)/%,%,$(CURDIR))
-
 quiet_cmd_cc_o_c = CC      $@
 ifeq ($(CONFIG_ENFORCE_UNIQUE_SYMBOLS),y)
     cmd_cc_o_c = $(CC) $(c_flags) -c $< -o $(dot-target).tmp -MQ $@
-    ifeq ($(CONFIG_CC_IS_CLANG)$(call clang-ifversion,-lt,600,y),yy)
-        cmd_objcopy_fix_sym = $(OBJCOPY) --redefine-sym $<=$(SRCPATH)/$< $(dot-target).tmp $@
+    ifneq ($(CONFIG_CC_IS_CLANG)$(call clang-ifversion,-lt,600,y),yy)
+        cmd_objcopy_fix_sym = \
+	    $(OBJCOPY) --redefine-sym $(<F)=$< $(dot-target).tmp $@ && rm -f $(dot-target).tmp
     else
-        cmd_objcopy_fix_sym = $(OBJCOPY) --redefine-sym $(<F)=$(SRCPATH)/$< $(dot-target).tmp $@
+        cmd_objcopy_fix_sym = mv -f $(dot-target).tmp $@
     endif
-    cmd_objcopy_fix_sym += && rm -f $(dot-target).tmp
 else
     cmd_cc_o_c = $(CC) $(c_flags) -c $< -o $@
 endif
@@ -181,13 +209,13 @@ define rule_cc_o_c
     $(call cmd,objcopy_fix_sym)
 endef
 
-%.o: %.c FORCE
+$(obj)/%.o: $(src)/%.c FORCE
 	$(call if_changed_rule,cc_o_c)
 
 quiet_cmd_cc_o_S = CC      $@
 cmd_cc_o_S = $(CC) $(a_flags) -c $< -o $@
 
-%.o: %.S FORCE
+$(obj)/%.o: $(src)/%.S FORCE
 	$(call if_changed,cc_o_S)
 
 
@@ -205,7 +233,7 @@ define cmd_obj_init_o
     $(OBJCOPY) $(foreach s,$(SPECIAL_DATA_SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
 endef
 
-$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o FORCE
+$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): $(obj)/%.init.o: $(obj)/%.o FORCE
 	$(call if_changed,obj_init_o)
 
 quiet_cmd_cpp_i_c = CPP     $@
@@ -217,19 +245,21 @@ cmd_cc_s_c = $(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -S $< -o $@
 quiet_cmd_cpp_s_S = CPP     $@
 cmd_cpp_s_S = $(CPP) $(call cpp_flags,$(a_flags)) -MQ $@ -o $@ $<
 
-%.i: %.c FORCE
+$(obj)/%.i: $(src)/%.c FORCE
 	$(call if_changed,cpp_i_c)
 
-%.s: %.c FORCE
+$(obj)/%.s: $(src)/%.c FORCE
 	$(call if_changed,cc_s_c)
 
-%.s: %.S FORCE
+$(obj)/%.s: $(src)/%.S FORCE
 	$(call if_changed,cpp_s_S)
 
 # Linker scripts, .lds.S -> .lds
 quiet_cmd_cpp_lds_S = LDS     $@
 cmd_cpp_lds_S = $(CPP) -P $(call cpp_flags,$(a_flags)) -D__LINKER__ -MQ $@ -o $@ $<
 
+targets := $(filter-out $(PHONY), $(targets))
+
 # Add intermediate targets:
 # When building objects with specific suffix patterns, add intermediate
 # targets that the final targets are derived from.
@@ -239,7 +269,18 @@ intermediate_targets = $(foreach sfx, $(2), \
 # %.init.o <- %.o
 targets += $(call intermediate_targets, .init.o, .o)
 
--include $(DEPS_INCLUDE)
+# Build
+# ---------------------------------------------------------------------------
+
+__build: $(targets-for-builtin) $(subdir-y)
+	@:
+
+# Descending
+# ---------------------------------------------------------------------------
+
+PHONY += $(subdir-y)
+$(subdir-y):
+	$(Q)$(MAKE) $(build)=$@ need-builtin=$(if $(filter $@/built_in.o, $(subdir-builtin)),1)
 
 # Read all saved command lines and dependencies for the $(targets) we
 # may be building above, using $(if_changed{,_dep}). As an
@@ -250,6 +291,9 @@ existing-targets := $(wildcard $(sort $(targets)))
 
 -include $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).cmd)
 
+DEPS := $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).d)
+-include $(DEPS_INCLUDE)
+
 # Declare the contents of the PHONY variable as phony.  We keep that
 # information in a variable so we can use it in if_changed and friends.
 .PHONY: $(PHONY)
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index c993ce72a341..fd24f0212ffa 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -95,12 +95,12 @@ $(TARGET)-syms: $(BASEDIR)/prelink.o $(obj)/xen.lds
 	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
+	$(MAKE) $(build)=$(@D) $(@D)/.$(@F).0.o
 	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
+	$(MAKE) $(build)=$(@D) $(@D)/.$(@F).1.o
 	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
diff --git a/xen/arch/arm/Rules.mk b/xen/arch/arm/Rules.mk
index c6463a433efd..e69de29bb2d1 100644
--- a/xen/arch/arm/Rules.mk
+++ b/xen/arch/arm/Rules.mk
@@ -1,4 +0,0 @@
-# head.o is built by descending into arch/arm/$(TARGET_SUBARCH), depends on the
-# part of $(ALL_OBJS) that will eventually recurse into $(TARGET_SUBARCH)/ and
-# build head.o
-arch/arm/$(TARGET_SUBARCH)/head.o: arch/arm/built_in.o ;
diff --git a/xen/arch/arm/arch.mk b/xen/arch/arm/arch.mk
index ba3f140e2ea7..4e3f7014305e 100644
--- a/xen/arch/arm/arch.mk
+++ b/xen/arch/arm/arch.mk
@@ -1,8 +1,8 @@
 ########################################
 # arm-specific definitions
 
-CFLAGS += -I$(BASEDIR)/include
-CFLAGS += -I$(BASEDIR)/arch/$(TARGET_ARCH)/include
+CFLAGS += -I$(srctree)/include
+CFLAGS += -I$(srctree)/arch/$(TARGET_ARCH)/include
 
 $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
 $(call cc-option-add,CFLAGS,CC,-Wnested-externs)
diff --git a/xen/arch/riscv/arch.mk b/xen/arch/riscv/arch.mk
index 39ae6ffea94d..694ba053ceab 100644
--- a/xen/arch/riscv/arch.mk
+++ b/xen/arch/riscv/arch.mk
@@ -11,5 +11,5 @@ riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
 # -mcmodel=medlow would force Xen into the lower half.
 
 CFLAGS += -march=$(riscv-march-y) -mstrict-align -mcmodel=medany
-CFLAGS += -I$(BASEDIR)/include
-CFLAGS += -I$(BASEDIR)/arch/$(TARGET_ARCH)/include
+CFLAGS += -I$(srctree)/include
+CFLAGS += -I$(srctree)/arch/$(TARGET_ARCH)/include
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index db97ae8c07f0..8fe54ba137ca 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -80,6 +80,9 @@ endif
 extra-y += asm-macros.i
 extra-y += xen.lds
 
+# Allows usercopy.c to include itself
+$(obj)/usercopy.o: CFLAGS-y += -iquote .
+
 ifneq ($(CONFIG_HVM),y)
 $(obj)/x86_emulate.o: CFLAGS-y += -Wno-unused-label
 endif
@@ -129,13 +132,13 @@ $(TARGET)-syms: $(BASEDIR)/prelink.o $(obj)/xen.lds
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort \
 		>$(@D)/.$(@F).0.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
+	$(MAKE) $(build)=$(@D) $(@D)/.$(@F).0.o
 	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort $(syms-warn-dup-y) \
 		>$(@D)/.$(@F).1.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
+	$(MAKE) $(build)=$(@D) $(@D)/.$(@F).1.o
 	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
@@ -199,14 +202,14 @@ endif
 	$(MKRELOC) $(foreach base,$(VIRT_BASE) $(ALT_BASE),$(@D)/.$(@F).$(base).0) >$(@D)/.$(@F).0r.S
 	$(NM) -pa --format=sysv $(@D)/.$(@F).$(VIRT_BASE).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0s.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0r.o $(@D)/.$(@F).0s.o
+	$(MAKE) $(build)=$(@D) .$(@F).0r.o .$(@F).0s.o
 	$(foreach base, $(VIRT_BASE) $(ALT_BASE), \
 	          $(LD) $(call EFI_LDFLAGS,$(base)) -T $(obj)/efi.lds -N $< \
 	                $(@D)/.$(@F).0r.o $(@D)/.$(@F).0s.o $(note_file_option) -o $(@D)/.$(@F).$(base).1 &&) :
 	$(MKRELOC) $(foreach base,$(VIRT_BASE) $(ALT_BASE),$(@D)/.$(@F).$(base).1) >$(@D)/.$(@F).1r.S
 	$(NM) -pa --format=sysv $(@D)/.$(@F).$(VIRT_BASE).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1s.S
-	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o
+	$(MAKE) $(build)=$(@D) .$(@F).1r.o .$(@F).1s.o
 	$(LD) $(call EFI_LDFLAGS,$(VIRT_BASE)) -T $(obj)/efi.lds -N $< \
 	                $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o $(note_file_option) -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
diff --git a/xen/arch/x86/Rules.mk b/xen/arch/x86/Rules.mk
index 7aef93f5f3a0..ff7c7dd68ad7 100644
--- a/xen/arch/x86/Rules.mk
+++ b/xen/arch/x86/Rules.mk
@@ -2,9 +2,9 @@
 # x86-specific definitions
 
 ifneq ($(filter -DHAVE_AS_QUOTED_SYM,$(XEN_CFLAGS)),)
-object_label_flags = '-D__OBJECT_LABEL__=$(subst $(BASEDIR)/,,$(CURDIR))/$@'
+object_label_flags = '-D__OBJECT_LABEL__=$@'
 else
-object_label_flags = '-D__OBJECT_LABEL__=$(subst /,$$,$(subst -,_,$(subst $(BASEDIR)/,,$(CURDIR))/$@))'
+object_label_flags = '-D__OBJECT_LABEL__=$(subst /,$$,$(subst -,_,$@))'
 endif
 c_flags += $(object_label_flags) $(CFLAGS_stack_boundary)
 a_flags += $(object_label_flags) $(CFLAGS_stack_boundary)
diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
index bfd5eaa35f25..e39fa15be7e3 100644
--- a/xen/arch/x86/arch.mk
+++ b/xen/arch/x86/arch.mk
@@ -3,10 +3,10 @@
 
 export XEN_IMG_OFFSET := 0x200000
 
-CFLAGS += -I$(BASEDIR)/include
-CFLAGS += -I$(BASEDIR)/arch/$(TARGET_ARCH)/include
-CFLAGS += -I$(BASEDIR)/arch/x86/include/asm/mach-generic
-CFLAGS += -I$(BASEDIR)/arch/x86/include/asm/mach-default
+CFLAGS += -I$(srctree)/include
+CFLAGS += -I$(srctree)/arch/$(TARGET_ARCH)/include
+CFLAGS += -I$(srctree)/arch/x86/include/asm/mach-generic
+CFLAGS += -I$(srctree)/arch/x86/include/asm/mach-default
 CFLAGS += -DXEN_IMG_OFFSET=$(XEN_IMG_OFFSET)
 
 # Prevent floating-point variables from creeping into Xen.
@@ -87,7 +87,7 @@ efi-nr-fixups := $(shell $(OBJDUMP) -p $(efi-check).efi | grep '^[[:blank:]]*rel
 ifeq ($(efi-nr-fixups),2)
 MKRELOC := :
 else
-MKRELOC := efi/mkreloc
+MKRELOC := arch/x86/efi/mkreloc
 # If the linker produced fixups but not precisely two of them, we need to
 # disable it doing so.  But if it didn't produce any fixups, it also wouldn't
 # recognize the option.
@@ -103,6 +103,6 @@ export EFI_LDFLAGS
 endif
 
 # Set up the assembler include path properly for older toolchains.
-CFLAGS += -Wa,-I$(BASEDIR)/include
+CFLAGS += -Wa,-I$(srctree)/include
 
 ALL_OBJS-y := arch/x86/boot/built_in.o arch/x86/efi/built_in.o $(ALL_OBJS-y)
diff --git a/xen/arch/x86/boot/Makefile b/xen/arch/x86/boot/Makefile
index 0aec8a464364..ba732e4a88c3 100644
--- a/xen/arch/x86/boot/Makefile
+++ b/xen/arch/x86/boot/Makefile
@@ -1,8 +1,8 @@
 obj-bin-y += head.o
 
-DEFS_H_DEPS = $(src)/defs.h $(BASEDIR)/include/xen/stdbool.h
+DEFS_H_DEPS = $(abs_srctree)/$(src)/defs.h $(abs_srctree)/include/xen/stdbool.h
 
-CMDLINE_DEPS = $(DEFS_H_DEPS) $(src)/video.h \
+CMDLINE_DEPS = $(DEFS_H_DEPS) $(abs_srctree)/$(src)/video.h \
 	       $(BASEDIR)/include/xen/kconfig.h \
 	       $(BASEDIR)/include/generated/autoconf.h
 
@@ -17,7 +17,7 @@ RELOC_DEPS = $(DEFS_H_DEPS) \
 $(obj)/head.o: $(obj)/cmdline.S $(obj)/reloc.S
 
 $(obj)/cmdline.S: $(src)/cmdline.c $(CMDLINE_DEPS) $(src)/build32.lds
-	$(MAKE) -f build32.mk -C $(obj) $(@F) CMDLINE_DEPS="$(CMDLINE_DEPS)"
+	$(MAKE) -f $(abs_srctree)/$(src)/build32.mk -C $(obj) $(@F) CMDLINE_DEPS="$(CMDLINE_DEPS)"
 
 $(obj)/reloc.S: $(src)/reloc.c $(RELOC_DEPS) $(src)/build32.lds
-	$(MAKE) -f build32.mk -C $(obj) $(@F) RELOC_DEPS="$(RELOC_DEPS)"
+	$(MAKE) -f $(abs_srctree)/$(src)/build32.mk -C $(obj) $(@F) RELOC_DEPS="$(RELOC_DEPS)"
diff --git a/xen/build.mk b/xen/build.mk
index af1b28311393..487db14c5884 100644
--- a/xen/build.mk
+++ b/xen/build.mk
@@ -60,6 +60,16 @@ arch/$(TARGET_ARCH)/include/asm/asm-offsets.h: asm-offsets.s
 	  echo ""; \
 	  echo "#endif") <$< >$@
 
+build-dirs := $(patsubst %/built_in.o,%,$(filter %/built_in.o,$(ALL_OBJS) $(ALL_LIBS)))
+
+# The actual objects are generated when descending,
+# make sure no implicit rule kicks in
+$(sort $(ALL_OBJS) $(ALL_LIBS)): $(build-dirs) ;
+
+PHONY += $(build-dirs)
+$(build-dirs): FORCE
+	$(Q)$(MAKE) $(build)=$@ need-builtin=1
+
 ifeq ($(CONFIG_LTO),y)
 # Gather all LTO objects together
 prelink_lto.o: $(ALL_OBJS) $(ALL_LIBS)
@@ -76,4 +86,4 @@ endif
 targets += prelink.o
 
 $(TARGET): prelink.o FORCE
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) $@
+	$(Q)$(MAKE) $(build)=arch/$(TARGET_ARCH) $@
diff --git a/xen/include/Makefile b/xen/include/Makefile
index d2f5a956a11a..cd40d5b4c923 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -61,7 +61,7 @@ $(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(BASEDIR)/
 	export PYTHON=$(PYTHON); \
 	while read what name; do \
 		$(SHELL) $(BASEDIR)/tools/get-fields.sh "$$what" compat_$$name $< || exit $$?; \
-	done <$(patsubst compat/%,compat/.xlat/%,$(basename $<)).lst >$@.new
+	done <$(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst >$@.new
 	mv -f $@.new $@
 
 .PRECIOUS: $(obj)/compat/.xlat/%.lst
@@ -86,8 +86,8 @@ PUBLIC_HEADERS := $(filter-out $(src)/public/arch-% $(src)/public/dom0_ops.h, $(
 PUBLIC_C99_HEADERS := $(src)/public/io/9pfs.h $(src)/public/io/pvcalls.h
 PUBLIC_ANSI_HEADERS := $(filter-out $(src)/public/%ctl.h $(src)/public/xsm/% $(src)/public/%hvm/save.h $(PUBLIC_C99_HEADERS), $(PUBLIC_HEADERS))
 
-public/io/9pfs.h-prereq := string
-public/io/pvcalls.h-prereq := string
+$(src)/public/io/9pfs.h-prereq := string
+$(src)/public/io/pvcalls.h-prereq := string
 
 $(obj)/headers.chk: $(PUBLIC_ANSI_HEADERS) $(src)/Makefile
 	for i in $(filter %.h,$^); do \
diff --git a/xen/scripts/Kbuild.include b/xen/scripts/Kbuild.include
index 4875bb28c282..c159136adb20 100644
--- a/xen/scripts/Kbuild.include
+++ b/xen/scripts/Kbuild.include
@@ -61,6 +61,12 @@ cc-ifversion = $(shell [ $(CONFIG_GCC_VERSION)0 $(1) $(2)000 ] && echo $(3) || e
 
 clang-ifversion = $(shell [ $(CONFIG_CLANG_VERSION)0 $(1) $(2)000 ] && echo $(3) || echo $(4))
 
+###
+# Shorthand for $(Q)$(MAKE) -f scripts/Makefile.build obj=
+# Usage:
+# $(Q)$(MAKE) $(build)=dir
+build := -f $(srctree)/Rules.mk obj
+
 # Shorthand for $(MAKE) clean
 # Usage:
 # $(MAKE) $(clean) dir
diff --git a/xen/test/Makefile b/xen/test/Makefile
index 41e4d7bdb78b..080763c80766 100644
--- a/xen/test/Makefile
+++ b/xen/test/Makefile
@@ -1,13 +1,10 @@
 
-tests all: build
-
-
 ifneq ($(XEN_TARGET_ARCH),x86_32)
 # Xen 32-bit x86 hypervisor no longer supported, so has no test livepatches
 subdir-y += livepatch
 endif
 
-install build subtree-force-update uninstall: %:
+install uninstall: %:
 	set -e; for s in $(subdir-y); do \
-		$(MAKE) -f $(BASEDIR)/Rules.mk -C $$s $*; \
+		$(MAKE) $(build)=$$s $*; \
 	done
diff --git a/xen/test/livepatch/Makefile b/xen/test/livepatch/Makefile
index 69fadccd01ea..afb8d589ecae 100644
--- a/xen/test/livepatch/Makefile
+++ b/xen/test/livepatch/Makefile
@@ -11,9 +11,6 @@ endif
 CODE_ADDR=$(shell nm --defined $(1) | grep $(2) | awk '{print "0x"$$1}')
 CODE_SZ=$(shell nm --defined -S $(1) | grep $(2) | awk '{ print "0x"$$2}')
 
-.PHONY: default
-build default: livepatch
-
 extra-y += xen_hello_world.livepatch
 xen_hello_world-objs := xen_hello_world_func.o xen_hello_world.o note.o xen_note.o modinfo.o
 $(obj)/xen_hello_world.o: $(obj)/config.h
@@ -156,9 +153,6 @@ LIVEPATCHES := $(filter %.livepatch,$(extra-y))
 
 LIVEPATCH_DEBUG_DIR ?= $(DEBUG_DIR)/xen-livepatch
 
-.PHONY: livepatch
-livepatch: $(LIVEPATCHES)
-
 install: $(addprefix $(obj)/,$(LIVEPATCHES))
 	$(INSTALL_DIR) $(DESTDIR)$(LIVEPATCH_DEBUG_DIR)
 	$(INSTALL_DATA) $(addprefix $(obj)/,$(LIVEPATCHES)) $(DESTDIR)$(LIVEPATCH_DEBUG_DIR)
diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index 51fd37f6c4d5..49cf730cf098 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -40,7 +40,7 @@ $(obj)/flask-policy.S: $(BASEDIR)/tools/binfile FORCE
 	$(call if_changed,binfile,$(obj)/policy.bin xsm_flask_init_policy)
 targets += flask-policy.S
 
-FLASK_BUILD_DIR := $(CURDIR)
+FLASK_BUILD_DIR := $(abs_objtree)/$(obj)
 POLICY_SRC := $(FLASK_BUILD_DIR)/xenpolicy-$(XEN_FULLVERSION)
 
 $(obj)/policy.bin: FORCE
diff --git a/xen/xsm/flask/ss/Makefile b/xen/xsm/flask/ss/Makefile
index d32b9e07138e..aba1339f3808 100644
--- a/xen/xsm/flask/ss/Makefile
+++ b/xen/xsm/flask/ss/Makefile
@@ -8,4 +8,4 @@ obj-y += services.o
 obj-y += conditional.o
 obj-y += mls.o
 
-CFLAGS-y += -I../include
+CFLAGS-y += -I$(srctree)/xsm/flask/include
-- 
Anthony PERARD

[XEN PATCH v9 10/30] build: introduce if_changed_deps

[Anthony PERARD]

This macro does compare command line like if_changed, but it also
rewrite the dependencies generated by $(CC) in order to depend on a
CONFIG_* as generated by kconfig instead of depending on autoconf.h.
This allow to make a change in kconfig options and only rebuild the
object that uses that CONFIG_* option.

cmd_and_record isn't needed anymore as it is replace by
cmd_and_fixdep.

There's only one .*.d dependency file left which is explicitly
included as a workound, all the other are been absorb into the .*.cmd
dependency files via `fixdep`. So including .*.d can be removed from
the makefile.

Also adjust "cloc" recipe due to .*.d been replace by .*.cmd files.

This imports fixdep.c and if_changed_deps macro from Linux v5.12.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v8:
    - reviewd
    - remove "tools_fixdep" prerequisite in xen/Makefile from "$(TARGET)"
      target as the first command of the recipe will build everything in
      tools/ and thus tools/fixdep.
    - adjust "cloc" recipe.

 .gitignore                 |   1 +
 xen/Makefile               |  15 +-
 xen/Rules.mk               |  17 +-
 xen/arch/x86/Makefile      |   8 +-
 xen/build.mk               |   1 +
 xen/scripts/Kbuild.include |  17 +-
 xen/tools/Makefile         |   7 +-
 xen/tools/fixdep.c         | 404 +++++++++++++++++++++++++++++++++++++
 8 files changed, 441 insertions(+), 29 deletions(-)
 create mode 100644 xen/tools/fixdep.c

diff --git a/.gitignore b/.gitignore
index 3f9d55ba87f0..cc87a7dfae7b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -332,6 +332,7 @@ xen/include/xen/lib/x86/cpuid-autogen.h
 xen/test/livepatch/config.h
 xen/test/livepatch/expect_config.h
 xen/test/livepatch/*.livepatch
+xen/tools/fixdep
 xen/tools/kconfig/.tmp_gtkcheck
 xen/tools/kconfig/.tmp_qtcheck
 xen/tools/symbols
diff --git a/xen/Makefile b/xen/Makefile
index 780d555e8620..3dc7b4da10fb 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -196,6 +196,13 @@ export XEN_HAS_CHECKPOLICY := $(call success,$(CHECKPOLICY) -h 2>&1 | grep -q xe
 export root-make-done := y
 endif # root-make-done
 
+# ===========================================================================
+# Rules shared between *config targets and build targets
+
+PHONY += tools_fixdep
+tools_fixdep:
+	$(MAKE) -C tools fixdep
+
 # Shorthand for kconfig
 kconfig = -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)"
 
@@ -471,18 +478,18 @@ cscope:
 _MAP:
 	$(NM) -n $(TARGET)-syms | grep -v '\(compiled\)\|\(\.o$$\)\|\( [aUw] \)\|\(\.\.ng$$\)\|\(LASH[RL]DI\)' > System.map
 
-%.o %.i %.s: %.c FORCE
+%.o %.i %.s: %.c tools_fixdep FORCE
 	$(Q)$(MAKE) $(build)=$(*D) $(*D)/$(@F)
 
-%.o %.s: %.S FORCE
+%.o %.s: %.S tools_fixdep FORCE
 	$(Q)$(MAKE) $(build)=$(*D) $(*D)/$(@F)
 
-%/: FORCE
+%/: tools_fixdep FORCE
 	$(Q)$(MAKE) $(build)=$* need-builtin=1
 
 .PHONY: cloc
 cloc:
-	find . -name tools -prune -o -name '*.o.d' -print | while read f; do \
+	find . -name tools -prune -o -name '*.o.cmd' -print | while read f; do \
 	    for sf in $$(grep -o "[a-zA-Z0-9_/-]*\.[cS]" $$f); do \
 		test -f "$$sf" && echo "$$sf"; \
 	    done; \
diff --git a/xen/Rules.mk b/xen/Rules.mk
index 67112e00773f..567a23a54cd9 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -147,8 +147,8 @@ cpp_flags = $(filter-out -Wa$(comma)% -flto,$(1))
 # Calculation of flags, first the generic flags, then the arch specific flags,
 # and last the flags modified for a target or a directory.
 
-c_flags = -MMD -MP -MF $(@D)/.$(@F).d $(XEN_CFLAGS)
-a_flags = -MMD -MP -MF $(@D)/.$(@F).d $(XEN_AFLAGS)
+c_flags = -MMD -MP -MF $(depfile) $(XEN_CFLAGS)
+a_flags = -MMD -MP -MF $(depfile) $(XEN_AFLAGS)
 
 include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
 
@@ -205,7 +205,7 @@ else
 endif
 
 define rule_cc_o_c
-    $(call cmd_and_record,cc_o_c)
+    $(call cmd_and_fixdep,cc_o_c)
     $(call cmd,objcopy_fix_sym)
 endef
 
@@ -216,7 +216,7 @@ quiet_cmd_cc_o_S = CC      $@
 cmd_cc_o_S = $(CC) $(a_flags) -c $< -o $@
 
 $(obj)/%.o: $(src)/%.S FORCE
-	$(call if_changed,cc_o_S)
+	$(call if_changed_dep,cc_o_S)
 
 
 quiet_cmd_obj_init_o = INIT_O  $@
@@ -246,13 +246,13 @@ quiet_cmd_cpp_s_S = CPP     $@
 cmd_cpp_s_S = $(CPP) $(call cpp_flags,$(a_flags)) -MQ $@ -o $@ $<
 
 $(obj)/%.i: $(src)/%.c FORCE
-	$(call if_changed,cpp_i_c)
+	$(call if_changed_dep,cpp_i_c)
 
 $(obj)/%.s: $(src)/%.c FORCE
-	$(call if_changed,cc_s_c)
+	$(call if_changed_dep,cc_s_c)
 
 $(obj)/%.s: $(src)/%.S FORCE
-	$(call if_changed,cpp_s_S)
+	$(call if_changed_dep,cpp_s_S)
 
 # Linker scripts, .lds.S -> .lds
 quiet_cmd_cpp_lds_S = LDS     $@
@@ -291,9 +291,6 @@ existing-targets := $(wildcard $(sort $(targets)))
 
 -include $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).cmd)
 
-DEPS := $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).d)
--include $(DEPS_INCLUDE)
-
 # Declare the contents of the PHONY variable as phony.  We keep that
 # information in a variable so we can use it in if_changed and friends.
 .PHONY: $(PHONY)
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 8fe54ba137ca..9494e4a3df79 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -245,7 +245,7 @@ $(BASEDIR)/arch/x86/include/asm/asm-macros.h: $(obj)/asm-macros.i $(src)/Makefil
 
 $(obj)/efi.lds: AFLAGS-y += -DEFI
 $(obj)/xen.lds $(obj)/efi.lds: $(src)/xen.lds.S FORCE
-	$(call if_changed,cpp_lds_S)
+	$(call if_changed_dep,cpp_lds_S)
 
 $(obj)/boot/mkelf32: $(src)/boot/mkelf32.c
 	$(HOSTCC) $(HOSTCFLAGS) -o $@ $<
@@ -261,9 +261,3 @@ clean::
 	rm -f $(BASEDIR)/.xen.efi.[0-9]* efi/*.efi efi/mkreloc
 	rm -f boot/cmdline.S boot/reloc.S boot/*.lnk boot/*.bin
 	rm -f note.o
-
-# Suppress loading of DEPS files for internal, temporary target files.  This
-# then also suppresses re-generation of the respective .*.d2 files.
-ifeq ($(filter-out .xen%.o,$(notdir $(MAKECMDGOALS))),)
-DEPS_INCLUDE:=
-endif
diff --git a/xen/build.mk b/xen/build.mk
index 487db14c5884..e718743ef772 100644
--- a/xen/build.mk
+++ b/xen/build.mk
@@ -40,6 +40,7 @@ include/xen/compile.h: include/xen/compile.h.in .banner FORCE
 
 targets += include/xen/compile.h
 
+-include $(wildcard .asm-offsets.s.d)
 asm-offsets.s: arch/$(TARGET_ARCH)/$(TARGET_SUBARCH)/asm-offsets.c
 	$(CC) $(call cpp_flags,$(c_flags)) -S -g0 -o $@.new -MQ $@ $<
 	$(call move-if-changed,$@.new,$@)
diff --git a/xen/scripts/Kbuild.include b/xen/scripts/Kbuild.include
index c159136adb20..872adfb456c3 100644
--- a/xen/scripts/Kbuild.include
+++ b/xen/scripts/Kbuild.include
@@ -15,8 +15,7 @@ dot-target = $(@D)/.$(@F)
 
 ###
 # dependencies
-DEPS = .*.d
-DEPS_INCLUDE = $(addsuffix .d2, $(basename $(wildcard $(DEPS))))
+depfile = $(dot-target).d
 
 ###
 # real prerequisites without phony targets
@@ -83,6 +82,8 @@ cmd = @set -e; $(echo-cmd) $(cmd_$(1))
 ###
 # if_changed      - execute command if any prerequisite is newer than
 #                   target, or command line has changed
+# if_changed_dep  - as if_changed, but uses fixdep to reveal dependencies
+#                   including used config symbols
 # if_changed_rule - as if_changed but execute rule instead
 
 ifneq ($(KBUILD_NOCMDDEP),1)
@@ -111,15 +112,19 @@ if_changed = $(if $(any-prereq)$(cmd-check),                                 \
         $(cmd);                                                              \
         printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:)
 
+# Execute the command and also postprocess generated .d dependencies file.
+if_changed_dep = $(if $(any-prereq)$(cmd-check),$(cmd_and_fixdep),@:)
+
+cmd_and_fixdep =                                                             \
+        $(cmd);                                                              \
+        tools/fixdep $(depfile) $@ '$(make-cmd)' > $(dot-target).cmd;        \
+        rm -f $(depfile)
+
 # Usage: $(call if_changed_rule,foo)
 # Will check if $(cmd_foo) or any of the prerequisites changed,
 # and if so will execute $(rule_foo).
 if_changed_rule = $(if $(any-prereq)$(cmd-check),$(rule_$(1)),@:)
 
-cmd_and_record =                                                             \
-        $(cmd);                                                              \
-        printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd
-
 ###
 # why - tell why a target got built
 #       enabled by make V=2
diff --git a/xen/tools/Makefile b/xen/tools/Makefile
index 4e42163f981c..722f3664549d 100644
--- a/xen/tools/Makefile
+++ b/xen/tools/Makefile
@@ -2,11 +2,14 @@
 include $(XEN_ROOT)/Config.mk
 
 .PHONY: default
-default: symbols
+default: symbols fixdep
 
 .PHONY: clean
 clean:
-	rm -f *.o symbols
+	rm -f *.o symbols fixdep
 
 symbols: symbols.c
 	$(HOSTCC) $(HOSTCFLAGS) -o $@ $<
+
+fixdep: fixdep.c
+	$(HOSTCC) $(HOSTCFLAGS) -o $@ $<
diff --git a/xen/tools/fixdep.c b/xen/tools/fixdep.c
new file mode 100644
index 000000000000..d98540552941
--- /dev/null
+++ b/xen/tools/fixdep.c
@@ -0,0 +1,404 @@
+/*
+ * "Optimize" a list of dependencies as spit out by gcc -MD
+ * for the kernel build
+ * ===========================================================================
+ *
+ * Author       Kai Germaschewski
+ * Copyright    2002 by Kai Germaschewski  <kai.germaschewski@gmx.de>
+ *
+ * This software may be used and distributed according to the terms
+ * of the GNU General Public License, incorporated herein by reference.
+ *
+ *
+ * Introduction:
+ *
+ * gcc produces a very nice and correct list of dependencies which
+ * tells make when to remake a file.
+ *
+ * To use this list as-is however has the drawback that virtually
+ * every file in the kernel includes autoconf.h.
+ *
+ * If the user re-runs make *config, autoconf.h will be
+ * regenerated.  make notices that and will rebuild every file which
+ * includes autoconf.h, i.e. basically all files. This is extremely
+ * annoying if the user just changed CONFIG_HIS_DRIVER from n to m.
+ *
+ * So we play the same trick that "mkdep" played before. We replace
+ * the dependency on autoconf.h by a dependency on every config
+ * option which is mentioned in any of the listed prerequisites.
+ *
+ * kconfig populates a tree in include/config/ with an empty file
+ * for each config symbol and when the configuration is updated
+ * the files representing changed config options are touched
+ * which then let make pick up the changes and the files that use
+ * the config symbols are rebuilt.
+ *
+ * So if the user changes his CONFIG_HIS_DRIVER option, only the objects
+ * which depend on "include/config/his/driver.h" will be rebuilt,
+ * so most likely only his driver ;-)
+ *
+ * The idea above dates, by the way, back to Michael E Chastain, AFAIK.
+ *
+ * So to get dependencies right, there are two issues:
+ * o if any of the files the compiler read changed, we need to rebuild
+ * o if the command line given to the compile the file changed, we
+ *   better rebuild as well.
+ *
+ * The former is handled by using the -MD output, the later by saving
+ * the command line used to compile the old object and comparing it
+ * to the one we would now use.
+ *
+ * Again, also this idea is pretty old and has been discussed on
+ * kbuild-devel a long time ago. I don't have a sensibly working
+ * internet connection right now, so I rather don't mention names
+ * without double checking.
+ *
+ * This code here has been based partially based on mkdep.c, which
+ * says the following about its history:
+ *
+ *   Copyright abandoned, Michael Chastain, <mailto:mec@shout.net>.
+ *   This is a C version of syncdep.pl by Werner Almesberger.
+ *
+ *
+ * It is invoked as
+ *
+ *   fixdep <depfile> <target> <cmdline>
+ *
+ * and will read the dependency file <depfile>
+ *
+ * The transformed dependency snipped is written to stdout.
+ *
+ * It first generates a line
+ *
+ *   cmd_<target> = <cmdline>
+ *
+ * and then basically copies the .<target>.d file to stdout, in the
+ * process filtering out the dependency on autoconf.h and adding
+ * dependencies on include/config/my/option.h for every
+ * CONFIG_MY_OPTION encountered in any of the prerequisites.
+ *
+ * We don't even try to really parse the header files, but
+ * merely grep, i.e. if CONFIG_FOO is mentioned in a comment, it will
+ * be picked up as well. It's not a problem with respect to
+ * correctness, since that can only give too many dependencies, thus
+ * we cannot miss a rebuild. Since people tend to not mention totally
+ * unrelated CONFIG_ options all over the place, it's not an
+ * efficiency problem either.
+ *
+ * (Note: it'd be easy to port over the complete mkdep state machine,
+ *  but I don't think the added complexity is worth it)
+ */
+
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <string.h>
+#include <stdarg.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <ctype.h>
+
+static void usage(void)
+{
+	fprintf(stderr, "Usage: fixdep <depfile> <target> <cmdline>\n");
+	exit(1);
+}
+
+/*
+ * In the intended usage of this program, the stdout is redirected to .*.cmd
+ * files. The return value of printf() and putchar() must be checked to catch
+ * any error, e.g. "No space left on device".
+ */
+static void xprintf(const char *format, ...)
+{
+	va_list ap;
+	int ret;
+
+	va_start(ap, format);
+	ret = vprintf(format, ap);
+	if (ret < 0) {
+		perror("fixdep");
+		exit(1);
+	}
+	va_end(ap);
+}
+
+static void xputchar(int c)
+{
+	int ret;
+
+	ret = putchar(c);
+	if (ret == EOF) {
+		perror("fixdep");
+		exit(1);
+	}
+}
+
+/*
+ * Print out a dependency path from a symbol name
+ */
+static void print_dep(const char *m, int slen, const char *dir)
+{
+	int c, prev_c = '/', i;
+
+	xprintf("    $(wildcard %s/", dir);
+	for (i = 0; i < slen; i++) {
+		c = m[i];
+		if (c == '_')
+			c = '/';
+		else
+			c = tolower(c);
+		if (c != '/' || prev_c != '/')
+			xputchar(c);
+		prev_c = c;
+	}
+	xprintf(".h) \\\n");
+}
+
+struct item {
+	struct item	*next;
+	unsigned int	len;
+	unsigned int	hash;
+	char		name[];
+};
+
+#define HASHSZ 256
+static struct item *hashtab[HASHSZ];
+
+static unsigned int strhash(const char *str, unsigned int sz)
+{
+	/* fnv32 hash */
+	unsigned int i, hash = 2166136261U;
+
+	for (i = 0; i < sz; i++)
+		hash = (hash ^ str[i]) * 0x01000193;
+	return hash;
+}
+
+/*
+ * Lookup a value in the configuration string.
+ */
+static int is_defined_config(const char *name, int len, unsigned int hash)
+{
+	struct item *aux;
+
+	for (aux = hashtab[hash % HASHSZ]; aux; aux = aux->next) {
+		if (aux->hash == hash && aux->len == len &&
+		    memcmp(aux->name, name, len) == 0)
+			return 1;
+	}
+	return 0;
+}
+
+/*
+ * Add a new value to the configuration string.
+ */
+static void define_config(const char *name, int len, unsigned int hash)
+{
+	struct item *aux = malloc(sizeof(*aux) + len);
+
+	if (!aux) {
+		perror("fixdep:malloc");
+		exit(1);
+	}
+	memcpy(aux->name, name, len);
+	aux->len = len;
+	aux->hash = hash;
+	aux->next = hashtab[hash % HASHSZ];
+	hashtab[hash % HASHSZ] = aux;
+}
+
+/*
+ * Record the use of a CONFIG_* word.
+ */
+static void use_config(const char *m, int slen)
+{
+	unsigned int hash = strhash(m, slen);
+
+	if (is_defined_config(m, slen, hash))
+	    return;
+
+	define_config(m, slen, hash);
+	print_dep(m, slen, "include/config");
+}
+
+/* test if s ends in sub */
+static int str_ends_with(const char *s, int slen, const char *sub)
+{
+	int sublen = strlen(sub);
+
+	if (sublen > slen)
+		return 0;
+
+	return !memcmp(s + slen - sublen, sub, sublen);
+}
+
+static void parse_config_file(const char *p)
+{
+	const char *q, *r;
+	const char *start = p;
+
+	while ((p = strstr(p, "CONFIG_"))) {
+		if (p > start && (isalnum(p[-1]) || p[-1] == '_')) {
+			p += 7;
+			continue;
+		}
+		p += 7;
+		q = p;
+		while (isalnum(*q) || *q == '_')
+			q++;
+		if (str_ends_with(p, q - p, "_MODULE"))
+			r = q - 7;
+		else
+			r = q;
+		if (r > p)
+			use_config(p, r - p);
+		p = q;
+	}
+}
+
+static void *read_file(const char *filename)
+{
+	struct stat st;
+	int fd;
+	char *buf;
+
+	fd = open(filename, O_RDONLY);
+	if (fd < 0) {
+		fprintf(stderr, "fixdep: error opening file: ");
+		perror(filename);
+		exit(2);
+	}
+	if (fstat(fd, &st) < 0) {
+		fprintf(stderr, "fixdep: error fstat'ing file: ");
+		perror(filename);
+		exit(2);
+	}
+	buf = malloc(st.st_size + 1);
+	if (!buf) {
+		perror("fixdep: malloc");
+		exit(2);
+	}
+	if (read(fd, buf, st.st_size) != st.st_size) {
+		perror("fixdep: read");
+		exit(2);
+	}
+	buf[st.st_size] = '\0';
+	close(fd);
+
+	return buf;
+}
+
+/* Ignore certain dependencies */
+static int is_ignored_file(const char *s, int len)
+{
+	return str_ends_with(s, len, "include/generated/autoconf.h") ||
+	       str_ends_with(s, len, "include/generated/autoksyms.h");
+}
+
+/*
+ * Important: The below generated source_foo.o and deps_foo.o variable
+ * assignments are parsed not only by make, but also by the rather simple
+ * parser in scripts/mod/sumversion.c.
+ */
+static void parse_dep_file(char *m, const char *target)
+{
+	char *p;
+	int is_last, is_target;
+	int saw_any_target = 0;
+	int is_first_dep = 0;
+	void *buf;
+
+	while (1) {
+		/* Skip any "white space" */
+		while (*m == ' ' || *m == '\\' || *m == '\n')
+			m++;
+
+		if (!*m)
+			break;
+
+		/* Find next "white space" */
+		p = m;
+		while (*p && *p != ' ' && *p != '\\' && *p != '\n')
+			p++;
+		is_last = (*p == '\0');
+		/* Is the token we found a target name? */
+		is_target = (*(p-1) == ':');
+		/* Don't write any target names into the dependency file */
+		if (is_target) {
+			/* The /next/ file is the first dependency */
+			is_first_dep = 1;
+		} else if (!is_ignored_file(m, p - m)) {
+			*p = '\0';
+
+			/*
+			 * Do not list the source file as dependency, so that
+			 * kbuild is not confused if a .c file is rewritten
+			 * into .S or vice versa. Storing it in source_* is
+			 * needed for modpost to compute srcversions.
+			 */
+			if (is_first_dep) {
+				/*
+				 * If processing the concatenation of multiple
+				 * dependency files, only process the first
+				 * target name, which will be the original
+				 * source name, and ignore any other target
+				 * names, which will be intermediate temporary
+				 * files.
+				 */
+				if (!saw_any_target) {
+					saw_any_target = 1;
+					xprintf("source_%s := %s\n\n",
+						target, m);
+					xprintf("deps_%s := \\\n", target);
+				}
+				is_first_dep = 0;
+			} else {
+				xprintf("  %s \\\n", m);
+			}
+
+			buf = read_file(m);
+			parse_config_file(buf);
+			free(buf);
+		}
+
+		if (is_last)
+			break;
+
+		/*
+		 * Start searching for next token immediately after the first
+		 * "whitespace" character that follows this token.
+		 */
+		m = p + 1;
+	}
+
+	if (!saw_any_target) {
+		fprintf(stderr, "fixdep: parse error; no targets found\n");
+		exit(1);
+	}
+
+	xprintf("\n%s: $(deps_%s)\n\n", target, target);
+	xprintf("$(deps_%s):\n", target);
+}
+
+int main(int argc, char *argv[])
+{
+	const char *depfile, *target, *cmdline;
+	void *buf;
+
+	if (argc != 4)
+		usage();
+
+	depfile = argv[1];
+	target = argv[2];
+	cmdline = argv[3];
+
+	xprintf("cmd_%s := %s\n\n", target, cmdline);
+
+	buf = read_file(depfile);
+	parse_dep_file(buf, target);
+	free(buf);
+
+	return 0;
+}
-- 
Anthony PERARD

[XEN PATCH v9 11/30] build: rename __LINKER__ to LINKER_SCRIPT

[Anthony PERARD]

For two reasons: this macro is used to generate a "linker script" and
is not by the linker, and name starting with an underscore '_' are
supposed to be reserved, so better avoid them when not needed.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
---
 xen/Rules.mk                      | 2 +-
 xen/arch/arm/include/asm/config.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index 567a23a54cd9..fea3f70cdb72 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -256,7 +256,7 @@ $(obj)/%.s: $(src)/%.S FORCE
 
 # Linker scripts, .lds.S -> .lds
 quiet_cmd_cpp_lds_S = LDS     $@
-cmd_cpp_lds_S = $(CPP) -P $(call cpp_flags,$(a_flags)) -D__LINKER__ -MQ $@ -o $@ $<
+cmd_cpp_lds_S = $(CPP) -P $(call cpp_flags,$(a_flags)) -DLINKER_SCRIPT -MQ $@ -o $@ $<
 
 targets := $(filter-out $(PHONY), $(targets))
 
diff --git a/xen/arch/arm/include/asm/config.h b/xen/arch/arm/include/asm/config.h
index c7b77912013e..2aced0bc3b8b 100644
--- a/xen/arch/arm/include/asm/config.h
+++ b/xen/arch/arm/include/asm/config.h
@@ -191,7 +191,7 @@ extern unsigned long frametable_virt_end;
 #define watchdog_disable() ((void)0)
 #define watchdog_enable()  ((void)0)
 
-#if defined(__ASSEMBLY__) && !defined(__LINKER__)
+#if defined(__ASSEMBLY__) && !defined(LINKER_SCRIPT)
 #include <asm/asm_defns.h>
 #include <asm/macros.h>
 #endif
-- 
Anthony PERARD

[XEN PATCH v9 12/30] build: hook kconfig into xen build system

[Anthony PERARD]

Now that xen's build system is very close to Linux's ones, we can hook
"Makefile.host" into Xen's build system, and we can build Kconfig with
that.

"tools/kconfig/Makefile" now needs a workaround to not rebuild
"$(XEN_ROOT)/.config", as `make` tries the rules "%.config" which
fails with:
    tools/kconfig/Makefile:95: *** No configuration exists for this target on this architecture.  Stop.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v8:
    - reviewed
    - adjust Makefile.host to use the renamed macro "multi-depend".
    - move .config empty rules workaround into kconfig/Makefile (from Rules.mk)
      and merge the change of patch
            build: add an other explicite rules to not build $(XEN_ROOT)/.config
      into
            build: hook kconfig into xen build system

 xen/Makefile                                 |  34 ++++--
 xen/Rules.mk                                 |  13 ++-
 xen/scripts/Kbuild.include                   |  31 ++++++
 xen/scripts/Makefile.clean                   |  11 +-
 xen/{tools/kconfig => scripts}/Makefile.host |  11 +-
 xen/tools/kconfig/Makefile                   |   3 +
 xen/tools/kconfig/Makefile.kconfig           | 106 -------------------
 7 files changed, 86 insertions(+), 123 deletions(-)
 rename xen/{tools/kconfig => scripts}/Makefile.host (95%)
 delete mode 100644 xen/tools/kconfig/Makefile.kconfig

diff --git a/xen/Makefile b/xen/Makefile
index 3dc7b4da10fb..0705e67a9e7d 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -40,6 +40,7 @@ ARCH=$(XEN_TARGET_ARCH)
 SRCARCH=$(shell echo $(ARCH) | \
           sed -e 's/x86.*/x86/' -e s'/arm\(32\|64\)/arm/g' \
               -e s'/riscv.*/riscv/g')
+export ARCH SRCARCH
 
 # Don't break if the build process wasn't called from the top level
 # we need XEN_TARGET_ARCH to generate the proper config
@@ -163,6 +164,13 @@ ifneq ($(filter %config,$(MAKECMDGOALS)),)
     config-build := y
 endif
 
+export CONFIG_SHELL := $(SHELL)
+export YACC = $(if $(BISON),$(BISON),bison)
+export LEX = $(if $(FLEX),$(FLEX),flex)
+
+# Default file for 'make defconfig'.
+export KBUILD_DEFCONFIG := $(ARCH)_defconfig
+
 # CLANG_FLAGS needs to be calculated before calling Kconfig
 ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
 CLANG_FLAGS :=
@@ -203,9 +211,6 @@ PHONY += tools_fixdep
 tools_fixdep:
 	$(MAKE) -C tools fixdep
 
-# Shorthand for kconfig
-kconfig = -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)"
-
 ifeq ($(config-build),y)
 # ===========================================================================
 # *config targets only - make sure prerequisites are updated, and descend
@@ -221,14 +226,14 @@ filechk_kconfig_allconfig = \
 .allconfig.tmp: FORCE
 	set -e; { $(call filechk_kconfig_allconfig); } > $@
 
-config: FORCE
-	$(MAKE) $(kconfig) $@
+config: tools_fixdep FORCE
+	$(Q)$(MAKE) $(build)=tools/kconfig $@
 
 # Config.mk tries to include .config file, don't try to remake it
 %/.config: ;
 
-%config: .allconfig.tmp FORCE
-	$(MAKE) $(kconfig) KCONFIG_ALLCONFIG=$< $@
+%config: .allconfig.tmp tools_fixdep FORCE
+	$(Q)$(MAKE) $(build)=tools/kconfig KCONFIG_ALLCONFIG=$< $@
 
 else # !config-build
 
@@ -238,9 +243,15 @@ ifeq ($(need-config),y)
 # changes are detected.
 -include include/config/auto.conf.cmd
 
+# This allows make to build fixdep before invoking defconfig. We can't use
+# "tools_fixdep" which is a .PHONY target and would force make to call
+# "defconfig" again to update $(KCONFIG_CONFIG).
+tools/fixdep:
+	$(MAKE) -C tools fixdep
+
 # Allow people to just run `make` as before and not force them to configure
-$(KCONFIG_CONFIG):
-	$(MAKE) $(kconfig) defconfig
+$(KCONFIG_CONFIG): tools/fixdep
+	$(Q)$(MAKE) $(build)=tools/kconfig defconfig
 
 # The actual configuration files used during the build are stored in
 # include/generated/ and include/config/. Update them if .config is newer than
@@ -249,7 +260,7 @@ $(KCONFIG_CONFIG):
 # This exploits the 'multi-target pattern rule' trick.
 # The syncconfig should be executed only once to make all the targets.
 include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
-	$(MAKE) $(kconfig) syncconfig
+	$(Q)$(MAKE) $(build)=tools/kconfig syncconfig
 
 ifeq ($(CONFIG_DEBUG),y)
 CFLAGS += -O1
@@ -406,9 +417,10 @@ _clean:
 	$(MAKE) $(clean) arch/riscv
 	$(MAKE) $(clean) arch/x86
 	$(MAKE) $(clean) test
-	$(MAKE) $(kconfig) clean
+	$(MAKE) $(clean) tools/kconfig
 	find . \( -name "*.o" -o -name ".*.d" -o -name ".*.d2" \
 		-o -name ".*.o.tmp" -o -name "*~" -o -name "core" \
+		-o -name '*.lex.c' -o -name '*.tab.[ch]' \
 		-o -name "*.gcno" -o -name ".*.cmd" -o -name "lib.a" \) -exec rm -f {} \;
 	rm -f include/asm $(TARGET) $(TARGET).gz $(TARGET).efi $(TARGET).efi.map $(TARGET)-syms $(TARGET)-syms.map *~ core
 	rm -f asm-offsets.s arch/*/include/asm/asm-offsets.h
diff --git a/xen/Rules.mk b/xen/Rules.mk
index fea3f70cdb72..13c1943da97e 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -89,6 +89,13 @@ subdir-y        := $(addprefix $(obj)/,$(subdir-y))
 nocov-y         := $(addprefix $(obj)/,$(nocov-y))
 noubsan-y       := $(addprefix $(obj)/,$(noubsan-y))
 
+# Do not include hostprogs rules unless needed.
+# $(sort ...) is used here to remove duplicated words and excessive spaces.
+hostprogs-y := $(sort $(hostprogs-y))
+ifneq ($(hostprogs-y),)
+include scripts/Makefile.host
+endif
+
 # subdir-builtin may contain duplications. Use $(sort ...)
 subdir-builtin := $(sort $(filter %/built_in.o, $(obj-y)))
 
@@ -267,7 +274,11 @@ intermediate_targets = $(foreach sfx, $(2), \
 				$(patsubst %$(strip $(1)),%$(sfx), \
 					$(filter %$(strip $(1)), $(targets))))
 # %.init.o <- %.o
-targets += $(call intermediate_targets, .init.o, .o)
+# %.lex.o <- %.lex.c <- %.l
+# %.tab.o <- %.tab.[ch] <- %.y
+targets += $(call intermediate_targets, .init.o, .o) \
+	   $(call intermediate_targets, .lex.o, .lex.c) \
+	   $(call intermediate_targets, .tab.o, .tab.c .tab.h)
 
 # Build
 # ---------------------------------------------------------------------------
diff --git a/xen/scripts/Kbuild.include b/xen/scripts/Kbuild.include
index 872adfb456c3..ed48fb39f1f1 100644
--- a/xen/scripts/Kbuild.include
+++ b/xen/scripts/Kbuild.include
@@ -25,6 +25,37 @@ real-prereqs = $(filter-out $(PHONY), $^)
 # Escape single quote for use in echo statements
 escsq = $(subst $(squote),'\$(squote)',$1)
 
+###
+# Easy method for doing a status message
+       kecho := :
+ quiet_kecho := echo
+silent_kecho := :
+kecho := $($(quiet)kecho)
+
+###
+# filechk is used to check if the content of a generated file is updated.
+# Sample usage:
+#
+# filechk_sample = echo $(KERNELRELEASE)
+# version.h: FORCE
+#	$(call filechk,sample)
+#
+# The rule defined shall write to stdout the content of the new file.
+# The existing file will be compared with the new one.
+# - If no file exist it is created
+# - If the content differ the new file is used
+# - If they are equal no change, and no timestamp update
+define filechk
+	$(Q)set -e;						\
+	mkdir -p $(dir $@);					\
+	trap "rm -f $(dot-target).tmp" EXIT;			\
+	{ $(filechk_$(1)); } > $(dot-target).tmp;		\
+	if [ ! -r $@ ] || ! cmp -s $@ $(dot-target).tmp; then	\
+		$(kecho) '  UPD     $@';			\
+		mv -f $(dot-target).tmp $@;			\
+	fi
+endef
+
 # as-insn: Check whether assembler supports an instruction.
 # Usage: cflags-y += $(call as-insn,CC FLAGS,"insn",option-yes,option-no)
 as-insn = $(if $(shell echo 'void _(void) { asm volatile ( $(2) ); }' \
diff --git a/xen/scripts/Makefile.clean b/xen/scripts/Makefile.clean
index c3b0681611da..156d6307cf83 100644
--- a/xen/scripts/Makefile.clean
+++ b/xen/scripts/Makefile.clean
@@ -17,8 +17,17 @@ include $(src)/Makefile
 subdir-all := $(subdir-y) $(subdir-n) $(subdir-) \
               $(patsubst %/,%, $(filter %/, $(obj-y) $(obj-n) $(obj-)))
 
+__clean-files := \
+    $(clean-files) $(hostprogs-y) $(hostprogs-)
+
+__clean-files := $(wildcard $(__clean-files))
+
 .PHONY: clean
-clean:: $(subdir-all) ;
+clean:: $(subdir-all)
+ifneq ($(strip $(__clean-files)),)
+	rm -rf $(__clean-files)
+endif
+	@:
 
 # Descending
 # ---------------------------------------------------------------------------
diff --git a/xen/tools/kconfig/Makefile.host b/xen/scripts/Makefile.host
similarity index 95%
rename from xen/tools/kconfig/Makefile.host
rename to xen/scripts/Makefile.host
index 4c51c95d40f4..8a85f94316bc 100644
--- a/xen/tools/kconfig/Makefile.host
+++ b/xen/scripts/Makefile.host
@@ -1,5 +1,8 @@
 # SPDX-License-Identifier: GPL-2.0
 
+# target with $(obj)/ and its suffix stripped
+target-stem = $(basename $(patsubst $(obj)/%,%,$@))
+
 # LEX
 # ---------------------------------------------------------------------------
 quiet_cmd_flex = LEX     $@
@@ -114,7 +117,7 @@ quiet_cmd_host-cmulti	= HOSTLD  $@
 			  $(KBUILD_HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem))
 $(host-cmulti): FORCE
 	$(call if_changed,host-cmulti)
-$(call multi_depend, $(host-cmulti), , -objs)
+$(call multi-depend, $(host-cmulti), , -objs)
 
 # Create .o file from a single .c file
 # host-cobjs -> .o
@@ -132,7 +135,7 @@ quiet_cmd_host-cxxmulti	= HOSTLD  $@
 			  $(KBUILD_HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem))
 $(host-cxxmulti): FORCE
 	$(call if_changed,host-cxxmulti)
-$(call multi_depend, $(host-cxxmulti), , -objs -cxxobjs)
+$(call multi-depend, $(host-cxxmulti), , -objs -cxxobjs)
 
 # Create .o file from a single .cc (C++) file
 quiet_cmd_host-cxxobjs	= HOSTCXX $@
@@ -165,7 +168,7 @@ quiet_cmd_host-cshlib	= HOSTLLD -shared $@
 			  $(KBUILD_HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem).so)
 $(host-cshlib): FORCE
 	$(call if_changed,host-cshlib)
-$(call multi_depend, $(host-cshlib), .so, -objs)
+$(call multi-depend, $(host-cshlib), .so, -objs)
 
 # Link a shared library, based on position independent .o files
 # *.o -> .so shared library (host-cxxshlib)
@@ -175,7 +178,7 @@ quiet_cmd_host-cxxshlib	= HOSTLLD -shared $@
 			  $(KBUILD_HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem).so)
 $(host-cxxshlib): FORCE
 	$(call if_changed,host-cxxshlib)
-$(call multi_depend, $(host-cxxshlib), .so, -objs)
+$(call multi-depend, $(host-cxxshlib), .so, -objs)
 
 targets += $(host-csingle)  $(host-cmulti) $(host-cobjs)\
 	   $(host-cxxmulti) $(host-cxxobjs) $(host-cshlib) $(host-cshobjs) $(host-cxxshlib) $(host-cxxshobjs)
diff --git a/xen/tools/kconfig/Makefile b/xen/tools/kconfig/Makefile
index f39521a0ed5b..b7b9a419ad59 100644
--- a/xen/tools/kconfig/Makefile
+++ b/xen/tools/kconfig/Makefile
@@ -91,6 +91,9 @@ endif
 
 configfiles=$(wildcard $(srctree)/kernel/configs/$@ $(srctree)/arch/$(SRCARCH)/configs/$@)
 
+# Don't try to remake this file included by Config.mk.
+$(XEN_ROOT)/.config: ;
+
 %.config: $(obj)/conf
 	$(if $(call configfiles),, $(error No configuration exists for this target on this architecture))
 	$(Q)$(CONFIG_SHELL) $(srctree)/tools/kconfig/merge_config.sh -m .config $(configfiles)
diff --git a/xen/tools/kconfig/Makefile.kconfig b/xen/tools/kconfig/Makefile.kconfig
deleted file mode 100644
index 799321ec4d07..000000000000
--- a/xen/tools/kconfig/Makefile.kconfig
+++ /dev/null
@@ -1,106 +0,0 @@
-# xen/tools/kconfig
-
-# default rule to do nothing
-all:
-
-# Xen doesn't have a silent build flag
-quiet :=
-Q :=
-kecho := :
-
-# eventually you'll want to do out of tree builds
-srctree := $(BASEDIR)
-objtree := $(srctree)
-src := tools/kconfig
-obj := $(src)
-
-# handle functions (most of these lifted from different Linux makefiles
-dot-target = $(dir $@).$(notdir $@)
-depfile = $(subst $(comma),,$(dot-target).d)
-basetarget = $(basename $(notdir $@))
-# target with $(obj)/ and its suffix stripped
-target-stem = $(basename $(patsubst $(obj)/%,%,$@))
-cmd = $(cmd_$(1))
-if_changed = $(cmd_$(1))
-if_changed_dep = $(cmd_$(1))
-
-###
-# filechk is used to check if the content of a generated file is updated.
-# Sample usage:
-#
-# filechk_sample = echo $(KERNELRELEASE)
-# version.h: FORCE
-#	$(call filechk,sample)
-#
-# The rule defined shall write to stdout the content of the new file.
-# The existing file will be compared with the new one.
-# - If no file exist it is created
-# - If the content differ the new file is used
-# - If they are equal no change, and no timestamp update
-# - stdin is piped in from the first prerequisite ($<) so one has
-#   to specify a valid file as first prerequisite (often the kbuild file)
-define filechk
-	$(Q)set -e;				\
-	mkdir -p $(dir $@);			\
-	{ $(filechk_$(1)); } > $@.tmp;		\
-	if [ -r $@ ] && cmp -s $@ $@.tmp; then	\
-		rm -f $@.tmp;			\
-	else					\
-		$(kecho) '  UPD     $@';	\
-		mv -f $@.tmp $@;		\
-	fi
-endef
-
-define multi_depend
-$(foreach m, $(notdir $1), \
-	$(eval $(obj)/$m: \
-	$(addprefix $(obj)/, $(foreach s, $3, $($(m:%$(strip $2)=%$(s)))))))
-endef
-
-# Set our default defconfig file
-KBUILD_DEFCONFIG := $(ARCH)_defconfig
-
-# provide our shell
-CONFIG_SHELL := $(SHELL)
-
-# provide the host compiler
-HOSTCC ?= gcc
-HOSTCXX ?= g++
-YACC = $(if $(BISON),$(BISON),bison)
-LEX = $(if $(FLEX),$(FLEX),flex)
-
-# force target
-PHONY += FORCE
-
-FORCE:
-
-# include the original Makefile and Makefile.host from Linux
-include $(src)/Makefile
-include $(src)/Makefile.host
-
-# Add intermediate targets:
-# When building objects with specific suffix patterns, add intermediate
-# targets that the final targets are derived from.
-intermediate_targets = $(foreach sfx, $(2), \
-				$(patsubst %$(strip $(1)),%$(sfx), \
-					$(filter %$(strip $(1)), $(targets))))
-
-# %.lex.o <- %.lex.c <- %.l
-# %.tab.o <- %.tab.[ch] <- %.y
-targets += $(call intermediate_targets, .lex.o, .lex.c) \
-	   $(call intermediate_targets, .tab.o, .tab.c .tab.h)
-
-# clean up rule
-clean-deps = $(foreach f,$(host-cobjs) $(host-cxxobjs),$(dir $f).$(notdir $f).d)
-clean-shipped = $(patsubst %_shipped,%,$(wildcard $(obj)/*_shipped))
-
-clean:
-	rm -rf $(clean-files)
-	rm -rf $(clean-deps)
-	rm -rf $(host-csingle) $(host-cmulti) $(host-cxxmulti) $(host-cobjs) $(host-cxxobjs)
-	rm -rf $(clean-shipped)
-
-$(obj)/zconf%: $(src)/zconf%_shipped
-	cp -f $< $@
-
-.PHONY: $(PHONY)
-- 
Anthony PERARD

[XEN PATCH v9 13/30] xen/tools/kconfig: fix build with -Wdeclaration-after-statement

[Anthony PERARD]

We are going to start building kconfig with HOSTCFLAGS from Config.mk,
it has the flag "-Wdeclaration-after-statement".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v9:
    - reviewed

 xen/tools/kconfig/confdata.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/tools/kconfig/confdata.c b/xen/tools/kconfig/confdata.c
index 3569d2dec37c..a69250c91355 100644
--- a/xen/tools/kconfig/confdata.c
+++ b/xen/tools/kconfig/confdata.c
@@ -1237,6 +1237,7 @@ void set_all_choice_values(struct symbol *csym)
 
 bool conf_set_all_new_symbols(enum conf_def_mode mode)
 {
+	bool has_changed = false;
 	struct symbol *sym, *csym;
 	int i, cnt, pby, pty, ptm;	/* pby: probability of bool     = y
 					 * pty: probability of tristate = y
@@ -1283,7 +1284,6 @@ bool conf_set_all_new_symbols(enum conf_def_mode mode)
 			exit( 1 );
 		}
 	}
-	bool has_changed = false;
 
 	for_all_symbols(i, sym) {
 		if (sym_has_value(sym) || (sym->flags & SYMBOL_VALID))
-- 
Anthony PERARD

[XEN PATCH v9 14/30] build: remove KBUILD_ specific from Makefile.host

[Anthony PERARD]

This will allow $(HOSTCFLAGS) to actually be used when building
programmes for the build-host.

The other variable don't exist in our build system.

Also remove $(KBUILD_EXTMOD) since it should always be empty.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
 xen/scripts/Makefile.host | 26 ++++++++++++--------------
 1 file changed, 12 insertions(+), 14 deletions(-)

diff --git a/xen/scripts/Makefile.host b/xen/scripts/Makefile.host
index 8a85f94316bc..d6c358095ee8 100644
--- a/xen/scripts/Makefile.host
+++ b/xen/scripts/Makefile.host
@@ -82,18 +82,16 @@ host-cxxshobjs	:= $(addprefix $(obj)/,$(host-cxxshobjs))
 #####
 # Handle options to gcc. Support building with separate output directory
 
-_hostc_flags   = $(KBUILD_HOSTCFLAGS)   $(HOST_EXTRACFLAGS)   \
+_hostc_flags   = $(HOSTCFLAGS)   $(HOST_EXTRACFLAGS)   \
                  $(HOSTCFLAGS_$(target-stem).o)
-_hostcxx_flags = $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
+_hostcxx_flags = $(HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
 # $(objtree)/$(obj) for including generated headers from checkin source files
-ifeq ($(KBUILD_EXTMOD),)
 ifdef building_out_of_srctree
 _hostc_flags   += -I $(objtree)/$(obj)
 _hostcxx_flags += -I $(objtree)/$(obj)
 endif
-endif
 
 hostc_flags    = -Wp,-MD,$(depfile) $(_hostc_flags)
 hostcxx_flags  = -Wp,-MD,$(depfile) $(_hostcxx_flags)
@@ -104,17 +102,17 @@ hostcxx_flags  = -Wp,-MD,$(depfile) $(_hostcxx_flags)
 # Create executable from a single .c file
 # host-csingle -> Executable
 quiet_cmd_host-csingle 	= HOSTCC  $@
-      cmd_host-csingle	= $(HOSTCC) $(hostc_flags) $(KBUILD_HOSTLDFLAGS) -o $@ $< \
-		$(KBUILD_HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem))
+      cmd_host-csingle	= $(HOSTCC) $(hostc_flags) $(HOSTLDFLAGS) -o $@ $< \
+		$(HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem))
 $(host-csingle): $(obj)/%: $(src)/%.c FORCE
 	$(call if_changed_dep,host-csingle)
 
 # Link an executable based on list of .o files, all plain c
 # host-cmulti -> executable
 quiet_cmd_host-cmulti	= HOSTLD  $@
-      cmd_host-cmulti	= $(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ \
+      cmd_host-cmulti	= $(HOSTCC) $(HOSTLDFLAGS) -o $@ \
 			  $(addprefix $(obj)/, $($(target-stem)-objs)) \
-			  $(KBUILD_HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem))
+			  $(HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem))
 $(host-cmulti): FORCE
 	$(call if_changed,host-cmulti)
 $(call multi-depend, $(host-cmulti), , -objs)
@@ -129,10 +127,10 @@ $(host-cobjs): $(obj)/%.o: $(src)/%.c FORCE
 # Link an executable based on list of .o files, a mixture of .c and .cc
 # host-cxxmulti -> executable
 quiet_cmd_host-cxxmulti	= HOSTLD  $@
-      cmd_host-cxxmulti	= $(HOSTCXX) $(KBUILD_HOSTLDFLAGS) -o $@ \
+      cmd_host-cxxmulti	= $(HOSTCXX) $(HOSTLDFLAGS) -o $@ \
 			  $(foreach o,objs cxxobjs,\
 			  $(addprefix $(obj)/, $($(target-stem)-$(o)))) \
-			  $(KBUILD_HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem))
+			  $(HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem))
 $(host-cxxmulti): FORCE
 	$(call if_changed,host-cxxmulti)
 $(call multi-depend, $(host-cxxmulti), , -objs -cxxobjs)
@@ -163,9 +161,9 @@ $(host-cxxshobjs): $(obj)/%.o: $(src)/%.c FORCE
 # Link a shared library, based on position independent .o files
 # *.o -> .so shared library (host-cshlib)
 quiet_cmd_host-cshlib	= HOSTLLD -shared $@
-      cmd_host-cshlib	= $(HOSTCC) $(KBUILD_HOSTLDFLAGS) -shared -o $@ \
+      cmd_host-cshlib	= $(HOSTCC) $(HOSTLDFLAGS) -shared -o $@ \
 			  $(addprefix $(obj)/, $($(target-stem)-objs)) \
-			  $(KBUILD_HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem).so)
+			  $(HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem).so)
 $(host-cshlib): FORCE
 	$(call if_changed,host-cshlib)
 $(call multi-depend, $(host-cshlib), .so, -objs)
@@ -173,9 +171,9 @@ $(call multi-depend, $(host-cshlib), .so, -objs)
 # Link a shared library, based on position independent .o files
 # *.o -> .so shared library (host-cxxshlib)
 quiet_cmd_host-cxxshlib	= HOSTLLD -shared $@
-      cmd_host-cxxshlib	= $(HOSTCXX) $(KBUILD_HOSTLDFLAGS) -shared -o $@ \
+      cmd_host-cxxshlib	= $(HOSTCXX) $(HOSTLDFLAGS) -shared -o $@ \
 			  $(addprefix $(obj)/, $($(target-stem)-objs)) \
-			  $(KBUILD_HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem).so)
+			  $(HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem).so)
 $(host-cxxshlib): FORCE
 	$(call if_changed,host-cxxshlib)
 $(call multi-depend, $(host-cxxshlib), .so, -objs)
-- 
Anthony PERARD

[XEN PATCH v9 15/30] build: handle always-y and hostprogs-always-y

[Anthony PERARD]

This will be used for xen/tools/.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
 xen/Rules.mk               | 10 +++++++++-
 xen/scripts/Makefile.clean |  3 ++-
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index 13c1943da97e..5f2368805b36 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -78,9 +78,17 @@ else
 obj-y    := $(filter-out %/, $(obj-y))
 endif
 
+# hostprogs-always-y += foo
+# ... is a shorthand for
+# hostprogs-y += foo
+# always-y  += foo
+hostprogs-y += $(hostprogs-always-y)
+always-y += $(hostprogs-always-y)
+
 # Add subdir path
 
 extra-y         := $(addprefix $(obj)/,$(extra-y))
+always-y        := $(addprefix $(obj)/,$(always-y))
 targets         := $(addprefix $(obj)/,$(targets))
 lib-y           := $(addprefix $(obj)/,$(lib-y))
 obj-y           := $(addprefix $(obj)/,$(obj-y))
@@ -283,7 +291,7 @@ targets += $(call intermediate_targets, .init.o, .o) \
 # Build
 # ---------------------------------------------------------------------------
 
-__build: $(targets-for-builtin) $(subdir-y)
+__build: $(targets-for-builtin) $(subdir-y) $(always-y)
 	@:
 
 # Descending
diff --git a/xen/scripts/Makefile.clean b/xen/scripts/Makefile.clean
index 156d6307cf83..c2689d4af5fa 100644
--- a/xen/scripts/Makefile.clean
+++ b/xen/scripts/Makefile.clean
@@ -18,7 +18,8 @@ subdir-all := $(subdir-y) $(subdir-n) $(subdir-) \
               $(patsubst %/,%, $(filter %/, $(obj-y) $(obj-n) $(obj-)))
 
 __clean-files := \
-    $(clean-files) $(hostprogs-y) $(hostprogs-)
+    $(clean-files) $(hostprogs-y) $(hostprogs-) \
+    $(hostprogs-always-y) $(hostprogs-always-)
 
 __clean-files := $(wildcard $(__clean-files))
 
-- 
Anthony PERARD

[XEN PATCH v9 16/30] build: start building the tools with the main makefiles

[Anthony PERARD]

This will make out-of-tree build easier.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v8:
    - add some $(Q)

 xen/Makefile       |  8 ++++----
 xen/tools/Makefile | 17 ++---------------
 2 files changed, 6 insertions(+), 19 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index 0705e67a9e7d..6c5c69d2e19b 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -209,7 +209,7 @@ endif # root-make-done
 
 PHONY += tools_fixdep
 tools_fixdep:
-	$(MAKE) -C tools fixdep
+	$(Q)$(MAKE) $(build)=tools tools/fixdep
 
 ifeq ($(config-build),y)
 # ===========================================================================
@@ -247,7 +247,7 @@ ifeq ($(need-config),y)
 # "tools_fixdep" which is a .PHONY target and would force make to call
 # "defconfig" again to update $(KCONFIG_CONFIG).
 tools/fixdep:
-	$(MAKE) -C tools fixdep
+	$(Q)$(MAKE) $(build)=tools tools/fixdep
 
 # Allow people to just run `make` as before and not force them to configure
 $(KCONFIG_CONFIG): tools/fixdep
@@ -406,7 +406,7 @@ _debug:
 
 .PHONY: _clean
 _clean:
-	$(MAKE) -C tools clean
+	$(MAKE) $(clean) tools
 	$(MAKE) $(clean) include
 	$(MAKE) $(clean) common
 	$(MAKE) $(clean) drivers
@@ -435,7 +435,7 @@ $(TARGET).gz: $(TARGET)
 	mv $@.new $@
 
 $(TARGET): FORCE
-	$(MAKE) -C tools
+	$(Q)$(MAKE) $(build)=tools
 	$(Q)$(MAKE) $(build)=. include/xen/compile.h
 	[ -e arch/$(TARGET_ARCH)/efi ] && for f in $$(cd common/efi; echo *.[ch]); \
 		do test -r arch/$(TARGET_ARCH)/efi/$$f || \
diff --git a/xen/tools/Makefile b/xen/tools/Makefile
index 722f3664549d..a5078b7cb8de 100644
--- a/xen/tools/Makefile
+++ b/xen/tools/Makefile
@@ -1,15 +1,2 @@
-
-include $(XEN_ROOT)/Config.mk
-
-.PHONY: default
-default: symbols fixdep
-
-.PHONY: clean
-clean:
-	rm -f *.o symbols fixdep
-
-symbols: symbols.c
-	$(HOSTCC) $(HOSTCFLAGS) -o $@ $<
-
-fixdep: fixdep.c
-	$(HOSTCC) $(HOSTCFLAGS) -o $@ $<
+hostprogs-always-y += symbols
+hostprogs-always-y += fixdep
-- 
Anthony PERARD

[XEN PATCH v9 17/30] build: add headers path to CFLAGS once for all archs

[Anthony PERARD]

This just remove duplication.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
---
 xen/Makefile           | 3 +++
 xen/arch/arm/arch.mk   | 3 ---
 xen/arch/riscv/arch.mk | 2 --
 xen/arch/x86/arch.mk   | 2 --
 4 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index 6c5c69d2e19b..65b09c20a599 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -312,6 +312,9 @@ CFLAGS += -flto
 LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
 endif
 
+CFLAGS += -I$(srctree)/include
+CFLAGS += -I$(srctree)/arch/$(TARGET_ARCH)/include
+
 # Note that link order matters!
 ALL_OBJS-y                := common/built_in.o
 ALL_OBJS-y                += drivers/built_in.o
diff --git a/xen/arch/arm/arch.mk b/xen/arch/arm/arch.mk
index 4e3f7014305e..094b67072304 100644
--- a/xen/arch/arm/arch.mk
+++ b/xen/arch/arm/arch.mk
@@ -1,9 +1,6 @@
 ########################################
 # arm-specific definitions
 
-CFLAGS += -I$(srctree)/include
-CFLAGS += -I$(srctree)/arch/$(TARGET_ARCH)/include
-
 $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
 $(call cc-option-add,CFLAGS,CC,-Wnested-externs)
 
diff --git a/xen/arch/riscv/arch.mk b/xen/arch/riscv/arch.mk
index 694ba053ceab..ae8fe9dec730 100644
--- a/xen/arch/riscv/arch.mk
+++ b/xen/arch/riscv/arch.mk
@@ -11,5 +11,3 @@ riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
 # -mcmodel=medlow would force Xen into the lower half.
 
 CFLAGS += -march=$(riscv-march-y) -mstrict-align -mcmodel=medany
-CFLAGS += -I$(srctree)/include
-CFLAGS += -I$(srctree)/arch/$(TARGET_ARCH)/include
diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
index e39fa15be7e3..8bc0e01ceb2b 100644
--- a/xen/arch/x86/arch.mk
+++ b/xen/arch/x86/arch.mk
@@ -3,8 +3,6 @@
 
 export XEN_IMG_OFFSET := 0x200000
 
-CFLAGS += -I$(srctree)/include
-CFLAGS += -I$(srctree)/arch/$(TARGET_ARCH)/include
 CFLAGS += -I$(srctree)/arch/x86/include/asm/mach-generic
 CFLAGS += -I$(srctree)/arch/x86/include/asm/mach-default
 CFLAGS += -DXEN_IMG_OFFSET=$(XEN_IMG_OFFSET)
-- 
Anthony PERARD

[XEN PATCH v9 18/30] build: generate x86's asm-macros.h with filechk

[Anthony PERARD]

When we will build out-of-tree, make is going to try to generate
"asm-macros.h" before the directories "arch/x86/include/asm" exist,
thus we would need to call `mkdir` explicitly. We will use "filechk"
for that as it does everything that the current recipe does and does
call `mkdir`.

Also, they are no more "*.new" files generated in this directory.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
 xen/arch/x86/Makefile | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)

diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 9494e4a3df79..de7f7ce07b98 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -230,18 +230,21 @@ include: $(BASEDIR)/arch/x86/include/asm/asm-macros.h
 $(obj)/asm-macros.i: CFLAGS-y += -D__ASSEMBLY__ -P
 
 $(BASEDIR)/arch/x86/include/asm/asm-macros.h: $(obj)/asm-macros.i $(src)/Makefile
-	echo '#if 0' >$@.new
-	echo '.if 0' >>$@.new
-	echo '#endif' >>$@.new
-	echo '#ifndef __ASM_MACROS_H__' >>$@.new
-	echo '#define __ASM_MACROS_H__' >>$@.new
-	echo 'asm ( ".include \"$@\"" );' >>$@.new
-	echo '#endif /* __ASM_MACROS_H__ */' >>$@.new
-	echo '#if 0' >>$@.new
-	echo '.endif' >>$@.new
-	cat $< >>$@.new
-	echo '#endif' >>$@.new
-	$(call move-if-changed,$@.new,$@)
+	$(call filechk,asm-macros.h)
+
+define filechk_asm-macros.h
+    echo '#if 0'; \
+    echo '.if 0'; \
+    echo '#endif'; \
+    echo '#ifndef __ASM_MACROS_H__'; \
+    echo '#define __ASM_MACROS_H__'; \
+    echo 'asm ( ".include \"$@\"" );'; \
+    echo '#endif /* __ASM_MACROS_H__ */'; \
+    echo '#if 0'; \
+    echo '.endif'; \
+    cat $<; \
+    echo '#endif'
+endef
 
 $(obj)/efi.lds: AFLAGS-y += -DEFI
 $(obj)/xen.lds $(obj)/efi.lds: $(src)/xen.lds.S FORCE
@@ -255,7 +258,7 @@ $(obj)/efi/mkreloc: $(src)/efi/mkreloc.c
 
 .PHONY: clean
 clean::
-	rm -f *.lds *.new boot/*.o boot/*~ boot/core boot/mkelf32
+	rm -f *.lds boot/*.o boot/*~ boot/core boot/mkelf32
 	rm -f asm-macros.i $(BASEDIR)/arch/x86/include/asm/asm-macros.*
 	rm -f $(BASEDIR)/.xen-syms.[0-9]* boot/.*.d $(BASEDIR)/.xen.elf32
 	rm -f $(BASEDIR)/.xen.efi.[0-9]* efi/*.efi efi/mkreloc
-- 
Anthony PERARD

[XEN PATCH v9 19/30] build: clean-up "clean" rules of duplication

[Anthony PERARD]

All those files to be removed are already done in the main Makefile,
either by the "find" command or directly (for $(TARGET).efi).

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
---
 xen/Makefile                | 2 +-
 xen/arch/arm/Makefile       | 1 -
 xen/arch/x86/Makefile       | 5 ++---
 xen/test/livepatch/Makefile | 2 +-
 xen/xsm/flask/Makefile      | 2 +-
 5 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index 65b09c20a599..b2df072d2a62 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -425,7 +425,7 @@ _clean:
 		-o -name ".*.o.tmp" -o -name "*~" -o -name "core" \
 		-o -name '*.lex.c' -o -name '*.tab.[ch]' \
 		-o -name "*.gcno" -o -name ".*.cmd" -o -name "lib.a" \) -exec rm -f {} \;
-	rm -f include/asm $(TARGET) $(TARGET).gz $(TARGET).efi $(TARGET).efi.map $(TARGET)-syms $(TARGET)-syms.map *~ core
+	rm -f include/asm $(TARGET) $(TARGET).gz $(TARGET).efi $(TARGET).efi.map $(TARGET)-syms $(TARGET)-syms.map
 	rm -f asm-offsets.s arch/*/include/asm/asm-offsets.h
 	rm -f .banner .allconfig.tmp include/xen/compile.h
 
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index fd24f0212ffa..3ce5f1674f6f 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -120,4 +120,3 @@ $(obj)/dtb.o: $(patsubst "%",%,$(CONFIG_DTB_FILE))
 clean::
 	rm -f $(obj)/xen.lds
 	rm -f $(BASEDIR)/.xen-syms.[0-9]*
-	rm -f $(TARGET).efi
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index de7f7ce07b98..60e802ba9503 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -258,9 +258,8 @@ $(obj)/efi/mkreloc: $(src)/efi/mkreloc.c
 
 .PHONY: clean
 clean::
-	rm -f *.lds boot/*.o boot/*~ boot/core boot/mkelf32
+	rm -f *.lds boot/mkelf32
 	rm -f asm-macros.i $(BASEDIR)/arch/x86/include/asm/asm-macros.*
-	rm -f $(BASEDIR)/.xen-syms.[0-9]* boot/.*.d $(BASEDIR)/.xen.elf32
+	rm -f $(BASEDIR)/.xen-syms.[0-9]* $(BASEDIR)/.xen.elf32
 	rm -f $(BASEDIR)/.xen.efi.[0-9]* efi/*.efi efi/mkreloc
 	rm -f boot/cmdline.S boot/reloc.S boot/*.lnk boot/*.bin
-	rm -f note.o
diff --git a/xen/test/livepatch/Makefile b/xen/test/livepatch/Makefile
index afb8d589ecae..adb484dc5d2c 100644
--- a/xen/test/livepatch/Makefile
+++ b/xen/test/livepatch/Makefile
@@ -162,4 +162,4 @@ uninstall:
 
 .PHONY: clean
 clean::
-	rm -f *.o .*.o.d *.livepatch config.h expect_config.h
+	rm -f *.livepatch config.h expect_config.h
diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index 49cf730cf098..832f65274cc0 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -51,4 +51,4 @@ $(obj)/policy.bin: FORCE
 
 .PHONY: clean
 clean::
-	rm -f $(ALL_H_FILES) *.o $(DEPS_RM) policy.* $(POLICY_SRC) flask-policy.S
+	rm -f $(ALL_H_FILES) policy.* $(POLICY_SRC) flask-policy.S
-- 
Anthony PERARD

[XEN PATCH v9 20/30] build: rework "clean" to clean from the root dir

[Anthony PERARD]

This will allow "clean" to work from an out-of-tree build when
it will be available.

Some of the file been removed in current "clean" target aren't added
to $(clean-files) because they are already listed in $(extra-) or
$(extra-y).

Also start to clean files listed in $(targets). This allows to clean
"common/config_data.S" and "xsm/flask/flask-policy.S" without
having to list them a second time.

Also clean files in "arch/x86/boot" from that directory by allowing
"clean" to descend into the subdir by adding "boot" into $(subdir-).

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v9:
    - reviewed
    
    v8:
    - fix comment for the shorthand
    - move this patch earlier in the series
    - replace some BASEDIR by objtree
    - start to clean $(targets) files.

 xen/Makefile                | 24 ++++++++++++------------
 xen/arch/arm/Makefile       |  5 +----
 xen/arch/x86/Makefile       | 20 ++++++++++++--------
 xen/arch/x86/boot/Makefile  |  2 ++
 xen/common/Makefile         |  3 +--
 xen/include/Makefile        |  4 +---
 xen/scripts/Kbuild.include  |  4 ++--
 xen/scripts/Makefile.clean  | 14 +++++++++++---
 xen/test/livepatch/Makefile |  4 +---
 xen/xsm/flask/Makefile      |  4 +---
 10 files changed, 44 insertions(+), 40 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index b2df072d2a62..dc25fa443b82 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -409,18 +409,18 @@ _debug:
 
 .PHONY: _clean
 _clean:
-	$(MAKE) $(clean) tools
-	$(MAKE) $(clean) include
-	$(MAKE) $(clean) common
-	$(MAKE) $(clean) drivers
-	$(MAKE) $(clean) lib
-	$(MAKE) $(clean) xsm
-	$(MAKE) $(clean) crypto
-	$(MAKE) $(clean) arch/arm
-	$(MAKE) $(clean) arch/riscv
-	$(MAKE) $(clean) arch/x86
-	$(MAKE) $(clean) test
-	$(MAKE) $(clean) tools/kconfig
+	$(Q)$(MAKE) $(clean)=tools
+	$(Q)$(MAKE) $(clean)=include
+	$(Q)$(MAKE) $(clean)=common
+	$(Q)$(MAKE) $(clean)=drivers
+	$(Q)$(MAKE) $(clean)=lib
+	$(Q)$(MAKE) $(clean)=xsm
+	$(Q)$(MAKE) $(clean)=crypto
+	$(Q)$(MAKE) $(clean)=arch/arm
+	$(Q)$(MAKE) $(clean)=arch/riscv
+	$(Q)$(MAKE) $(clean)=arch/x86
+	$(Q)$(MAKE) $(clean)=test
+	$(Q)$(MAKE) $(clean)=tools/kconfig
 	find . \( -name "*.o" -o -name ".*.d" -o -name ".*.d2" \
 		-o -name ".*.o.tmp" -o -name "*~" -o -name "core" \
 		-o -name '*.lex.c' -o -name '*.tab.[ch]' \
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 3ce5f1674f6f..cecfaf4f3c0f 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -116,7 +116,4 @@ $(obj)/xen.lds: $(src)/xen.lds.S FORCE
 
 $(obj)/dtb.o: $(patsubst "%",%,$(CONFIG_DTB_FILE))
 
-.PHONY: clean
-clean::
-	rm -f $(obj)/xen.lds
-	rm -f $(BASEDIR)/.xen-syms.[0-9]*
+clean-files := $(objtree)/.xen-syms.[0-9]*
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 60e802ba9503..bb57ed5096d5 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -77,6 +77,9 @@ obj-$(CONFIG_COMPAT) += x86_64/platform_hypercall.o
 obj-y += sysctl.o
 endif
 
+# Allows "clean" to descend into boot/
+subdir- += boot
+
 extra-y += asm-macros.i
 extra-y += xen.lds
 
@@ -190,8 +193,8 @@ note_file :=
 endif
 note_file_option ?= $(note_file)
 
+extra-$(XEN_BUILD_PE) += efi.lds
 ifeq ($(XEN_BUILD_PE),y)
-extra-y += efi.lds
 $(TARGET).efi: $(BASEDIR)/prelink.o $(note_file) $(obj)/efi.lds $(obj)/efi/relocs-dummy.o $(obj)/efi/mkreloc
 ifeq ($(CONFIG_DEBUG_INFO),y)
 	$(if $(filter --strip-debug,$(EFI_LDFLAGS)),echo,:) "Will strip debug info from $(@F)"
@@ -256,10 +259,11 @@ $(obj)/boot/mkelf32: $(src)/boot/mkelf32.c
 $(obj)/efi/mkreloc: $(src)/efi/mkreloc.c
 	$(HOSTCC) $(HOSTCFLAGS) -g -o $@ $<
 
-.PHONY: clean
-clean::
-	rm -f *.lds boot/mkelf32
-	rm -f asm-macros.i $(BASEDIR)/arch/x86/include/asm/asm-macros.*
-	rm -f $(BASEDIR)/.xen-syms.[0-9]* $(BASEDIR)/.xen.elf32
-	rm -f $(BASEDIR)/.xen.efi.[0-9]* efi/*.efi efi/mkreloc
-	rm -f boot/cmdline.S boot/reloc.S boot/*.lnk boot/*.bin
+clean-files := \
+    boot/mkelf32 \
+    include/asm/asm-macros.* \
+    $(objtree)/.xen-syms.[0-9]* \
+    $(objtree)/.xen.elf32 \
+    $(objtree)/.xen.efi.[0-9]* \
+    efi/*.efi \
+    efi/mkreloc
diff --git a/xen/arch/x86/boot/Makefile b/xen/arch/x86/boot/Makefile
index ba732e4a88c3..1ac8cb435e0e 100644
--- a/xen/arch/x86/boot/Makefile
+++ b/xen/arch/x86/boot/Makefile
@@ -21,3 +21,5 @@ $(obj)/cmdline.S: $(src)/cmdline.c $(CMDLINE_DEPS) $(src)/build32.lds
 
 $(obj)/reloc.S: $(src)/reloc.c $(RELOC_DEPS) $(src)/build32.lds
 	$(MAKE) -f $(abs_srctree)/$(src)/build32.mk -C $(obj) $(@F) RELOC_DEPS="$(RELOC_DEPS)"
+
+clean-files := cmdline.S reloc.S *.lnk *.bin
diff --git a/xen/common/Makefile b/xen/common/Makefile
index ca839118e4d1..dc8d3a13f5b8 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -84,5 +84,4 @@ $(obj)/config_data.S: $(BASEDIR)/tools/binfile FORCE
 	$(call if_changed,binfile,$(obj)/config.gz xen_config_data)
 targets += config_data.S
 
-clean::
-	rm -f config_data.S config.gz 2>/dev/null
+clean-files := config.gz
diff --git a/xen/include/Makefile b/xen/include/Makefile
index cd40d5b4c923..a3c2511f5f60 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -131,6 +131,4 @@ lib-x86-all:
 all: lib-x86-all
 endif
 
-clean::
-	rm -rf compat config generated headers*.chk
-	rm -f $(BASEDIR)/include/xen/lib/x86/cpuid-autogen.h
+clean-files := compat config generated headers*.chk xen/lib/x86/cpuid-autogen.h
diff --git a/xen/scripts/Kbuild.include b/xen/scripts/Kbuild.include
index ed48fb39f1f1..6e7a403b353c 100644
--- a/xen/scripts/Kbuild.include
+++ b/xen/scripts/Kbuild.include
@@ -99,8 +99,8 @@ build := -f $(srctree)/Rules.mk obj
 
 # Shorthand for $(MAKE) clean
 # Usage:
-# $(MAKE) $(clean) dir
-clean := -f $(BASEDIR)/scripts/Makefile.clean clean -C
+# $(Q)$(MAKE) $(clean)=dir
+clean := -f $(srctree)/scripts/Makefile.clean obj
 
 # echo command.
 # Short version is used, if $(quiet) equals `quiet_', otherwise full one.
diff --git a/xen/scripts/Makefile.clean b/xen/scripts/Makefile.clean
index c2689d4af5fa..4eed31974509 100644
--- a/xen/scripts/Makefile.clean
+++ b/xen/scripts/Makefile.clean
@@ -3,7 +3,6 @@
 # Cleaning up
 # ==========================================================================
 
-obj := .
 src := $(obj)
 
 clean::
@@ -17,11 +16,20 @@ include $(src)/Makefile
 subdir-all := $(subdir-y) $(subdir-n) $(subdir-) \
               $(patsubst %/,%, $(filter %/, $(obj-y) $(obj-n) $(obj-)))
 
+subdir-all := $(addprefix $(obj)/,$(subdir-all))
+
 __clean-files := \
     $(clean-files) $(hostprogs-y) $(hostprogs-) \
+    $(extra-y) $(extra-) $(targets) \
     $(hostprogs-always-y) $(hostprogs-always-)
 
-__clean-files := $(wildcard $(__clean-files))
+# clean-files is given relative to the current directory, unless it
+# starts with $(objtree)/ (which means "./", so do not add "./" unless
+# you want to delete a file from the toplevel object directory).
+
+__clean-files := $(wildcard \
+		   $(addprefix $(obj)/, $(filter-out /% $(objtree)/%, $(__clean-files))) \
+		   $(filter /% $(objtree)/%, $(__clean-files)))
 
 .PHONY: clean
 clean:: $(subdir-all)
@@ -35,6 +43,6 @@ endif
 
 PHONY += $(subdir-all)
 $(subdir-all):
-	$(MAKE) $(clean) $@
+	$(Q)$(MAKE) $(clean)=$@
 
 .PHONY: $(PHONY)
diff --git a/xen/test/livepatch/Makefile b/xen/test/livepatch/Makefile
index adb484dc5d2c..e6fee84b69da 100644
--- a/xen/test/livepatch/Makefile
+++ b/xen/test/livepatch/Makefile
@@ -160,6 +160,4 @@ install: $(addprefix $(obj)/,$(LIVEPATCHES))
 uninstall:
 	cd $(DESTDIR)$(LIVEPATCH_DEBUG_DIR) && rm -f $(LIVEPATCHES)
 
-.PHONY: clean
-clean::
-	rm -f *.livepatch config.h expect_config.h
+clean-files := config.h expect_config.h
diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index 832f65274cc0..4ac6fb8778ae 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -49,6 +49,4 @@ $(obj)/policy.bin: FORCE
 	        FLASK_BUILD_DIR=$(FLASK_BUILD_DIR) POLICY_FILENAME=$(POLICY_SRC)
 	cmp -s $(POLICY_SRC) $@ || cp $(POLICY_SRC) $@
 
-.PHONY: clean
-clean::
-	rm -f $(ALL_H_FILES) policy.* $(POLICY_SRC) flask-policy.S
+clean-files := policy.* $(POLICY_SRC)
-- 
Anthony PERARD

[XEN PATCH v9 21/30] build: use main rune to build host binary x86's mkelf32 and mkreloc

[Anthony PERARD]

Also, remove the HOSTCFLAGS "-g" from "mkreloc" command line.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v8:
    - acked
    - remove the cflags "-g" from "mkreloc"

 xen/arch/x86/Makefile | 13 ++++---------
 1 file changed, 4 insertions(+), 9 deletions(-)

diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index bb57ed5096d5..c94b4092d4c1 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -83,6 +83,9 @@ subdir- += boot
 extra-y += asm-macros.i
 extra-y += xen.lds
 
+hostprogs-y += boot/mkelf32
+hostprogs-y += efi/mkreloc
+
 # Allows usercopy.c to include itself
 $(obj)/usercopy.o: CFLAGS-y += -iquote .
 
@@ -253,17 +256,9 @@ $(obj)/efi.lds: AFLAGS-y += -DEFI
 $(obj)/xen.lds $(obj)/efi.lds: $(src)/xen.lds.S FORCE
 	$(call if_changed_dep,cpp_lds_S)
 
-$(obj)/boot/mkelf32: $(src)/boot/mkelf32.c
-	$(HOSTCC) $(HOSTCFLAGS) -o $@ $<
-
-$(obj)/efi/mkreloc: $(src)/efi/mkreloc.c
-	$(HOSTCC) $(HOSTCFLAGS) -g -o $@ $<
-
 clean-files := \
-    boot/mkelf32 \
     include/asm/asm-macros.* \
     $(objtree)/.xen-syms.[0-9]* \
     $(objtree)/.xen.elf32 \
     $(objtree)/.xen.efi.[0-9]* \
-    efi/*.efi \
-    efi/mkreloc
+    efi/*.efi
-- 
Anthony PERARD

[XEN PATCH v9 22/30] build: rework coverage and ubsan CFLAGS handling

[Anthony PERARD]

When assigning a value a target-specific variable, that also affect
prerequisite of the target. This is mostly fine, but there is one case
where we will not want the COV_FLAGS added to the CFLAGS.

In arch/x86/boot, we have "head.o" with "cmdline.S" as prerequisite
and ultimately "cmdline.o", we don't want COV_FLAGS to that last one.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v8:
    - reorder target-specific assignments
    - fix typo in comment
    - reviewed

 xen/Rules.mk | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index 5f2368805b36..abeba1ab7494 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -126,19 +126,31 @@ targets += $(targets-for-builtin)
 
 $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS-y += -DINIT_SECTIONS_ONLY
 
+non-init-objects = $(filter-out %.init.o, $(obj-y) $(obj-bin-y) $(extra-y))
+
 ifeq ($(CONFIG_COVERAGE),y)
 ifeq ($(CONFIG_CC_IS_CLANG),y)
     COV_FLAGS := -fprofile-instr-generate -fcoverage-mapping
 else
     COV_FLAGS := -fprofile-arcs -ftest-coverage
 endif
-$(filter-out %.init.o $(nocov-y),$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS-y += $(COV_FLAGS)
+
+# Reset COV_FLAGS in cases where an objects has another one as prerequisite
+$(nocov-y) $(filter %.init.o, $(obj-y) $(obj-bin-y) $(extra-y)): \
+    COV_FLAGS :=
+
+$(non-init-objects): _c_flags += $(COV_FLAGS)
 endif
 
 ifeq ($(CONFIG_UBSAN),y)
 # Any -fno-sanitize= options need to come after any -fsanitize= options
-$(filter-out %.init.o $(noubsan-y),$(obj-y) $(obj-bin-y) $(extra-y)): \
-CFLAGS-y += $(filter-out -fno-%,$(CFLAGS_UBSAN)) $(filter -fno-%,$(CFLAGS_UBSAN))
+UBSAN_FLAGS := $(filter-out -fno-%,$(CFLAGS_UBSAN)) $(filter -fno-%,$(CFLAGS_UBSAN))
+
+# Reset UBSAN_FLAGS in cases where an objects has another one as prerequisite
+$(noubsan-y) $(filter %.init.o, $(obj-y) $(obj-bin-y) $(extra-y)): \
+    UBSAN_FLAGS :=
+
+$(non-init-objects): _c_flags += $(UBSAN_FLAGS)
 endif
 
 ifeq ($(CONFIG_LTO),y)
@@ -167,6 +179,9 @@ a_flags = -MMD -MP -MF $(depfile) $(XEN_AFLAGS)
 
 include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
 
+c_flags += $(_c_flags)
+a_flags += $(_c_flags)
+
 c_flags += $(CFLAGS-y)
 a_flags += $(CFLAGS-y) $(AFLAGS-y)
 
-- 
Anthony PERARD

[XEN PATCH v9 23/30] build,x86: remove the need for build32.mk

[Anthony PERARD]

Rework "arch/x86/boot/Makefile" to allow it to build both file
"cmdline.S" and "reloc.S" without "build32.mk".

These will now use the main rules for "%.o: %.c", and thus generate a
dependency file. (We will not need to track the dependency manually
anymore.)

But for that, we need to override the main CFLAGS to do a 32bit build.
We introduce XEN_TREEWIDE_CFLAGS which can be reused in boot/Makefile,
and avoid the need to reparse Config.mk with a different value for
XEN_TARGET_ARCH. From this new $(XEN_TREEWIDE_CFLAGS), we only need to
change -m64 to have the 32bit flags. Then those are applied only to
"cmdline.o" and "reloc.o".

Specifically apply the rule "%.S: %.bin" to both cmdline.S and reloc.S
to avoid make trying to regenerate other %.S files with it.

There is no change expected to the resulting "cmdline.S" and
"reloc.S", only the *.o file changes as their symbol for FILE goes
from "cmdline.c" to "arch/x86//cmdline.c". (No idea why "boot" is
missing from the string.) (I've only check with GCC, not clang.)

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v9:
    - rename XEN_COMMON_CFLAGS to XEN_TREEWIDE_CFLAGS
    - rename $(head-objs) to $(head-srcs)
    - substitute LDFLAGS_DIRECT in %.lnk rule, rather than changing the value
      of it.
    
    v8:
    - avoid the need to list CFLAGS from Config.mk a second time by
      introducing XEN_COMMON_CFLAGS, and using it in boot/
    - improve LDFLAGS_DIRECT, by just substitute x64 to i368 from x86
      LDFLAGS_DIRECT. And thus avoid copying the logic from Config.mk.

 xen/Makefile                 |  4 +++
 xen/arch/x86/boot/Makefile   | 49 ++++++++++++++++++++++++------------
 xen/arch/x86/boot/build32.mk | 40 -----------------------------
 3 files changed, 37 insertions(+), 56 deletions(-)
 delete mode 100644 xen/arch/x86/boot/build32.mk

diff --git a/xen/Makefile b/xen/Makefile
index dc25fa443b82..8baa260b93a7 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -171,6 +171,10 @@ export LEX = $(if $(FLEX),$(FLEX),flex)
 # Default file for 'make defconfig'.
 export KBUILD_DEFCONFIG := $(ARCH)_defconfig
 
+# Copy CFLAGS generated by "Config.mk" so they can be reused later without
+# reparsing Config.mk by e.g. arch/x86/boot/.
+export XEN_TREEWIDE_CFLAGS := $(CFLAGS)
+
 # CLANG_FLAGS needs to be calculated before calling Kconfig
 ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
 CLANG_FLAGS :=
diff --git a/xen/arch/x86/boot/Makefile b/xen/arch/x86/boot/Makefile
index 1ac8cb435e0e..ca8001c72b23 100644
--- a/xen/arch/x86/boot/Makefile
+++ b/xen/arch/x86/boot/Makefile
@@ -1,25 +1,42 @@
 obj-bin-y += head.o
+head-srcs := cmdline.S reloc.S
 
-DEFS_H_DEPS = $(abs_srctree)/$(src)/defs.h $(abs_srctree)/include/xen/stdbool.h
+nocov-y += $(head-srcs:.S=.o)
+noubsan-y += $(head-srcs:.S=.o)
+targets += $(head-srcs:.S=.o)
 
-CMDLINE_DEPS = $(DEFS_H_DEPS) $(abs_srctree)/$(src)/video.h \
-	       $(BASEDIR)/include/xen/kconfig.h \
-	       $(BASEDIR)/include/generated/autoconf.h
+head-srcs := $(addprefix $(obj)/, $(head-srcs))
 
-RELOC_DEPS = $(DEFS_H_DEPS) \
-	     $(BASEDIR)/include/generated/autoconf.h \
-	     $(BASEDIR)/include/xen/kconfig.h \
-	     $(BASEDIR)/include/xen/multiboot.h \
-	     $(BASEDIR)/include/xen/multiboot2.h \
-	     $(BASEDIR)/include/xen/const.h \
-	     $(BASEDIR)/include/public/arch-x86/hvm/start_info.h
+$(obj)/head.o: $(head-srcs)
 
-$(obj)/head.o: $(obj)/cmdline.S $(obj)/reloc.S
+CFLAGS_x86_32 := $(subst -m64,-m32 -march=i686,$(XEN_TREEWIDE_CFLAGS))
+$(call cc-options-add,CFLAGS_x86_32,CC,$(EMBEDDED_EXTRA_CFLAGS))
+CFLAGS_x86_32 += -Werror -fno-builtin -g0 -msoft-float
+CFLAGS_x86_32 += -I$(srctree)/include
 
-$(obj)/cmdline.S: $(src)/cmdline.c $(CMDLINE_DEPS) $(src)/build32.lds
-	$(MAKE) -f $(abs_srctree)/$(src)/build32.mk -C $(obj) $(@F) CMDLINE_DEPS="$(CMDLINE_DEPS)"
+# override for 32bit binaries
+$(head-srcs:.S=.o): CFLAGS_stack_boundary :=
+$(head-srcs:.S=.o): XEN_CFLAGS := $(CFLAGS_x86_32) -fpic
 
-$(obj)/reloc.S: $(src)/reloc.c $(RELOC_DEPS) $(src)/build32.lds
-	$(MAKE) -f $(abs_srctree)/$(src)/build32.mk -C $(obj) $(@F) RELOC_DEPS="$(RELOC_DEPS)"
+$(head-srcs): %.S: %.bin
+	(od -v -t x $< | tr -s ' ' | awk 'NR > 1 {print s} {s=$$0}' | \
+	sed 's/ /,0x/g' | sed 's/,0x$$//' | sed 's/^[0-9]*,/ .long /') >$@
+
+# Drop .got.plt during conversion to plain binary format.
+# Please check build32.lds for more details.
+%.bin: %.lnk
+	$(OBJDUMP) -h $< | sed -n '/[0-9]/{s,00*,0,g;p;}' | \
+		while read idx name sz rest; do \
+			case "$$name" in \
+			.got.plt) \
+				test $$sz != 0c || continue; \
+				echo "Error: non-empty $$name: 0x$$sz" >&2; \
+				exit $$(expr $$idx + 1);; \
+			esac; \
+		done
+	$(OBJCOPY) -O binary -R .got.plt $< $@
+
+%.lnk: %.o $(src)/build32.lds
+	$(LD) $(subst x86_64,i386,$(LDFLAGS_DIRECT)) -N -T $(filter %.lds,$^) -o $@ $<
 
 clean-files := cmdline.S reloc.S *.lnk *.bin
diff --git a/xen/arch/x86/boot/build32.mk b/xen/arch/x86/boot/build32.mk
deleted file mode 100644
index e90680cd9f52..000000000000
--- a/xen/arch/x86/boot/build32.mk
+++ /dev/null
@@ -1,40 +0,0 @@
-override XEN_TARGET_ARCH=x86_32
-CFLAGS =
-include $(XEN_ROOT)/Config.mk
-
-$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
-
-CFLAGS += -Werror -fno-builtin -g0 -msoft-float
-CFLAGS += -I$(BASEDIR)/include
-CFLAGS := $(filter-out -flto,$(CFLAGS)) 
-
-# NB. awk invocation is a portable alternative to 'head -n -1'
-%.S: %.bin
-	(od -v -t x $< | tr -s ' ' | awk 'NR > 1 {print s} {s=$$0}' | \
-	sed 's/ /,0x/g' | sed 's/,0x$$//' | sed 's/^[0-9]*,/ .long /') >$@
-
-# Drop .got.plt during conversion to plain binary format.
-# Please check build32.lds for more details.
-%.bin: %.lnk
-	$(OBJDUMP) -h $< | sed -n '/[0-9]/{s,00*,0,g;p;}' | \
-		while read idx name sz rest; do \
-			case "$$name" in \
-			.got.plt) \
-				test $$sz != 0c || continue; \
-				echo "Error: non-empty $$name: 0x$$sz" >&2; \
-				exit $$(expr $$idx + 1);; \
-			esac; \
-		done
-	$(OBJCOPY) -O binary -R .got.plt $< $@
-
-%.lnk: %.o build32.lds
-	$(LD) $(LDFLAGS_DIRECT) -N -T build32.lds -o $@ $<
-
-%.o: %.c
-	$(CC) $(CFLAGS) -c -fpic $< -o $@
-
-cmdline.o: cmdline.c $(CMDLINE_DEPS)
-
-reloc.o: reloc.c $(RELOC_DEPS)
-
-.PRECIOUS: %.bin %.lnk
-- 
Anthony PERARD

[XEN PATCH v9 24/30] build: grab common EFI source files in arch specific dir

[Anthony PERARD]

Rather than preparing the efi source file, we will make the symbolic
link as needed from the build location.

The `ln` command is run every time to allow to update the link in case
the source tree change location.

This patch also introduce "efi-common.mk" which allow to reuse the
common make instructions without having to duplicate them into each
arch.

And now that we have a list of common source file, we can start to
remove the links to the source files on clean.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v9:
    - rename efi_common.mk to efi-common.mk
    - generalize comment about cleaning "efi" and "boot" subdir in x86.
    - add a space after the other comma of $(patsubst )
    - create a relative symlink instead of an absolute one
    - with the above, we don't need to use $(abs_srctree) anymore in the
      prerequisite of the link to the efi source file, use $(srctree).
    
    v8:
    - use symbolic link instead of making a copy of the source
    - introduce efi_common.mk
    - remove links to source file on clean
    - use -iquote for "efi.h" headers in common/efi

 xen/Makefile                 |  5 -----
 xen/arch/arm/efi/Makefile    |  4 ++--
 xen/arch/x86/Makefile        |  3 ++-
 xen/arch/x86/efi/Makefile    |  5 +----
 xen/common/efi/efi-common.mk | 15 +++++++++++++++
 5 files changed, 20 insertions(+), 12 deletions(-)
 create mode 100644 xen/common/efi/efi-common.mk

diff --git a/xen/Makefile b/xen/Makefile
index 8baa260b93a7..443784dfce80 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -444,11 +444,6 @@ $(TARGET).gz: $(TARGET)
 $(TARGET): FORCE
 	$(Q)$(MAKE) $(build)=tools
 	$(Q)$(MAKE) $(build)=. include/xen/compile.h
-	[ -e arch/$(TARGET_ARCH)/efi ] && for f in $$(cd common/efi; echo *.[ch]); \
-		do test -r arch/$(TARGET_ARCH)/efi/$$f || \
-		   ln -nsf ../../../common/efi/$$f arch/$(TARGET_ARCH)/efi/; \
-		done; \
-		true
 	$(Q)$(MAKE) $(build)=include all
 	$(Q)$(MAKE) $(build)=arch/$(TARGET_ARCH) include
 	$(Q)$(MAKE) $(build)=. arch/$(TARGET_ARCH)/include/asm/asm-offsets.h
diff --git a/xen/arch/arm/efi/Makefile b/xen/arch/arm/efi/Makefile
index 1b1ed06feddc..4313c390665f 100644
--- a/xen/arch/arm/efi/Makefile
+++ b/xen/arch/arm/efi/Makefile
@@ -1,4 +1,4 @@
-CFLAGS-y += -fshort-wchar
+include $(srctree)/common/efi/efi-common.mk
 
-obj-y += boot.init.o pe.init.o ebmalloc.o runtime.o
+obj-y += $(EFIOBJ-y)
 obj-$(CONFIG_ACPI) +=  efi-dom0.init.o
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index c94b4092d4c1..a847c989ff92 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -77,8 +77,9 @@ obj-$(CONFIG_COMPAT) += x86_64/platform_hypercall.o
 obj-y += sysctl.o
 endif
 
-# Allows "clean" to descend into boot/
+# Allows "clean" to descend
 subdir- += boot
+subdir- += efi
 
 extra-y += asm-macros.i
 extra-y += xen.lds
diff --git a/xen/arch/x86/efi/Makefile b/xen/arch/x86/efi/Makefile
index e08b4d8e4808..034ec87895df 100644
--- a/xen/arch/x86/efi/Makefile
+++ b/xen/arch/x86/efi/Makefile
@@ -1,4 +1,4 @@
-CFLAGS-y += -fshort-wchar
+include $(srctree)/common/efi/efi-common.mk
 
 quiet_cmd_objcopy_o_ihex = OBJCOPY $@
 cmd_objcopy_o_ihex = $(OBJCOPY) -I ihex -O binary $< $@
@@ -8,9 +8,6 @@ $(obj)/%.o: $(src)/%.ihex FORCE
 
 $(obj)/boot.init.o: $(obj)/buildid.o
 
-EFIOBJ-y := boot.init.o pe.init.o ebmalloc.o runtime.o
-EFIOBJ-$(CONFIG_COMPAT) += compat.o
-
 $(call cc-option-add,cflags-stack-boundary,CC,-mpreferred-stack-boundary=4)
 $(addprefix $(obj)/,$(EFIOBJ-y)): CFLAGS_stack_boundary := $(cflags-stack-boundary)
 
diff --git a/xen/common/efi/efi-common.mk b/xen/common/efi/efi-common.mk
new file mode 100644
index 000000000000..ad3c6f2569c3
--- /dev/null
+++ b/xen/common/efi/efi-common.mk
@@ -0,0 +1,15 @@
+EFIOBJ-y := boot.init.o pe.init.o ebmalloc.o runtime.o
+EFIOBJ-$(CONFIG_COMPAT) += compat.o
+
+CFLAGS-y += -fshort-wchar
+CFLAGS-y += -iquote $(srctree)/common/efi
+
+# Part of the command line transforms $(obj) in to a relative reverted path.
+# e.g.: It transforms "dir/foo/bar" into successively
+#       "dir foo bar", ".. .. ..", "../../.."
+$(obj)/%.c: $(srctree)/common/efi/%.c FORCE
+	$(Q)ln -nfs $(subst $(space),/,$(patsubst %,..,$(subst /, ,$(obj))))/common/efi/$(<F) $@
+
+clean-files += $(patsubst %.o, %.c, $(EFIOBJ-y:.init.o=.o) $(EFIOBJ-))
+
+.PRECIOUS: $(obj)/%.c
-- 
Anthony PERARD

[XEN PATCH v9 25/30] build: replace $(BASEDIR) by $(objtree)

[Anthony PERARD]

We need to differentiate between source files and generated/built
files. We will be replacing $(BASEDIR) by $(objtree) for files that
are generated.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v9:
    - acked
    
    v8:
    - rebased

 xen/Rules.mk                |  2 +-
 xen/arch/arm/Makefile       | 10 +++++-----
 xen/arch/x86/Makefile       | 28 ++++++++++++++--------------
 xen/common/Makefile         |  2 +-
 xen/test/livepatch/Makefile | 12 ++++++------
 5 files changed, 27 insertions(+), 27 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index abeba1ab7494..7712bfa063e0 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -12,7 +12,7 @@ src := $(obj)
 PHONY := __build
 __build:
 
--include $(BASEDIR)/include/config/auto.conf
+-include $(objtree)/include/config/auto.conf
 
 include $(XEN_ROOT)/Config.mk
 include $(BASEDIR)/scripts/Kbuild.include
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index cecfaf4f3c0f..ae7a2f907540 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -90,21 +90,21 @@ ifeq ($(CONFIG_ARM_64),y)
 	ln -sf $(@F) $@.efi
 endif
 
-$(TARGET)-syms: $(BASEDIR)/prelink.o $(obj)/xen.lds
+$(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
 	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
-	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
+	    $(objtree)/common/symbols-dummy.o -o $(@D)/.$(@F).0
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
-		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0.S
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0.S
 	$(MAKE) $(build)=$(@D) $(@D)/.$(@F).0.o
 	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
-		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1.S
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1.S
 	$(MAKE) $(build)=$(@D) $(@D)/.$(@F).1.o
 	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
-		| $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort \
+		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
 		>$(@D)/$(@F).map
 	rm -f $(@D)/.$(@F).[0-9]*
 
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index a847c989ff92..3fb59987776b 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -94,8 +94,8 @@ ifneq ($(CONFIG_HVM),y)
 $(obj)/x86_emulate.o: CFLAGS-y += -Wno-unused-label
 endif
 
-efi-y := $(shell if [ ! -r $(BASEDIR)/include/xen/compile.h -o \
-                      -O $(BASEDIR)/include/xen/compile.h ]; then \
+efi-y := $(shell if [ ! -r $(objtree)/include/xen/compile.h -o \
+                      -O $(objtree)/include/xen/compile.h ]; then \
                          echo '$(TARGET).efi'; fi) \
          $(space)
 efi-$(CONFIG_PV_SHIM_EXCLUSIVE) :=
@@ -133,23 +133,23 @@ $(TARGET): $(TARGET)-syms $(efi-y) $(obj)/boot/mkelf32
 
 CFLAGS-$(XEN_BUILD_EFI) += -DXEN_BUILD_EFI
 
-$(TARGET)-syms: $(BASEDIR)/prelink.o $(obj)/xen.lds
+$(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
 	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
-	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
+	    $(objtree)/common/symbols-dummy.o -o $(@D)/.$(@F).0
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
-		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
 		>$(@D)/.$(@F).0.S
 	$(MAKE) $(build)=$(@D) $(@D)/.$(@F).0.o
 	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
-		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort $(syms-warn-dup-y) \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort $(syms-warn-dup-y) \
 		>$(@D)/.$(@F).1.S
 	$(MAKE) $(build)=$(@D) $(@D)/.$(@F).1.o
 	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
-		| $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort \
+		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
 		>$(@D)/$(@F).map
 	rm -f $(@D)/.$(@F).[0-9]* $(@D)/..$(@F).[0-9]*
 
@@ -199,28 +199,28 @@ note_file_option ?= $(note_file)
 
 extra-$(XEN_BUILD_PE) += efi.lds
 ifeq ($(XEN_BUILD_PE),y)
-$(TARGET).efi: $(BASEDIR)/prelink.o $(note_file) $(obj)/efi.lds $(obj)/efi/relocs-dummy.o $(obj)/efi/mkreloc
+$(TARGET).efi: $(objtree)/prelink.o $(note_file) $(obj)/efi.lds $(obj)/efi/relocs-dummy.o $(obj)/efi/mkreloc
 ifeq ($(CONFIG_DEBUG_INFO),y)
 	$(if $(filter --strip-debug,$(EFI_LDFLAGS)),echo,:) "Will strip debug info from $(@F)"
 endif
 	$(foreach base, $(VIRT_BASE) $(ALT_BASE), \
 	          $(LD) $(call EFI_LDFLAGS,$(base)) -T $(obj)/efi.lds -N $< $(relocs-dummy) \
-	                $(BASEDIR)/common/symbols-dummy.o $(note_file_option) -o $(@D)/.$(@F).$(base).0 &&) :
+	                $(objtree)/common/symbols-dummy.o $(note_file_option) -o $(@D)/.$(@F).$(base).0 &&) :
 	$(MKRELOC) $(foreach base,$(VIRT_BASE) $(ALT_BASE),$(@D)/.$(@F).$(base).0) >$(@D)/.$(@F).0r.S
 	$(NM) -pa --format=sysv $(@D)/.$(@F).$(VIRT_BASE).0 \
-		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0s.S
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0s.S
 	$(MAKE) $(build)=$(@D) .$(@F).0r.o .$(@F).0s.o
 	$(foreach base, $(VIRT_BASE) $(ALT_BASE), \
 	          $(LD) $(call EFI_LDFLAGS,$(base)) -T $(obj)/efi.lds -N $< \
 	                $(@D)/.$(@F).0r.o $(@D)/.$(@F).0s.o $(note_file_option) -o $(@D)/.$(@F).$(base).1 &&) :
 	$(MKRELOC) $(foreach base,$(VIRT_BASE) $(ALT_BASE),$(@D)/.$(@F).$(base).1) >$(@D)/.$(@F).1r.S
 	$(NM) -pa --format=sysv $(@D)/.$(@F).$(VIRT_BASE).1 \
-		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1s.S
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1s.S
 	$(MAKE) $(build)=$(@D) .$(@F).1r.o .$(@F).1s.o
 	$(LD) $(call EFI_LDFLAGS,$(VIRT_BASE)) -T $(obj)/efi.lds -N $< \
 	                $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o $(note_file_option) -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
-		| $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort >$(@D)/$(@F).map
+		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort >$(@D)/$(@F).map
 	rm -f $(@D)/.$(@F).[0-9]* $(@D)/..$(@F).[0-9]*
 else
 $(TARGET).efi: FORCE
@@ -232,11 +232,11 @@ endif
 $(obj)/efi/buildid.o $(obj)/efi/relocs-dummy.o: ;
 
 .PHONY: include
-include: $(BASEDIR)/arch/x86/include/asm/asm-macros.h
+include: $(objtree)/arch/x86/include/asm/asm-macros.h
 
 $(obj)/asm-macros.i: CFLAGS-y += -D__ASSEMBLY__ -P
 
-$(BASEDIR)/arch/x86/include/asm/asm-macros.h: $(obj)/asm-macros.i $(src)/Makefile
+$(objtree)/arch/x86/include/asm/asm-macros.h: $(obj)/asm-macros.i $(src)/Makefile
 	$(call filechk,asm-macros.h)
 
 define filechk_asm-macros.h
diff --git a/xen/common/Makefile b/xen/common/Makefile
index dc8d3a13f5b8..30641a737231 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -74,7 +74,7 @@ obj-$(CONFIG_UBSAN) += ubsan/
 obj-$(CONFIG_NEEDS_LIBELF) += libelf/
 obj-$(CONFIG_HAS_DEVICE_TREE) += libfdt/
 
-CONF_FILE := $(if $(patsubst /%,,$(KCONFIG_CONFIG)),$(BASEDIR)/)$(KCONFIG_CONFIG)
+CONF_FILE := $(if $(patsubst /%,,$(KCONFIG_CONFIG)),$(objtree)/)$(KCONFIG_CONFIG)
 $(obj)/config.gz: $(CONF_FILE)
 	gzip -n -c $< >$@
 
diff --git a/xen/test/livepatch/Makefile b/xen/test/livepatch/Makefile
index e6fee84b69da..ddb07371315e 100644
--- a/xen/test/livepatch/Makefile
+++ b/xen/test/livepatch/Makefile
@@ -22,9 +22,9 @@ $(obj)/xen_hello_world.o: $(obj)/config.h
 $(obj)/config.h: $(obj)/xen_hello_world_func.o
 	(set -e; \
 	 echo "#define NEW_CODE_SZ $(call CODE_SZ,$<,xen_hello_world)"; \
-	 echo "#define MINOR_VERSION_SZ $(call CODE_SZ,$(BASEDIR)/xen-syms,xen_minor_version)"; \
-	 echo "#define MINOR_VERSION_ADDR $(call CODE_ADDR,$(BASEDIR)/xen-syms,xen_minor_version)"; \
-	 echo "#define OLD_CODE_SZ $(call CODE_SZ,$(BASEDIR)/xen-syms,xen_extra_version)") > $@
+	 echo "#define MINOR_VERSION_SZ $(call CODE_SZ,$(objtree)/xen-syms,xen_minor_version)"; \
+	 echo "#define MINOR_VERSION_ADDR $(call CODE_ADDR,$(objtree)/xen-syms,xen_minor_version)"; \
+	 echo "#define OLD_CODE_SZ $(call CODE_SZ,$(objtree)/xen-syms,xen_extra_version)") > $@
 
 $(obj)/modinfo.o:
 	(set -e; \
@@ -42,7 +42,7 @@ $(obj)/modinfo.o:
 # not be built (it is for EFI builds), and that we do not have
 # the note.o.bin to muck with (as it gets deleted)
 #
-$(obj)/note.o: $(BASEDIR)/xen-syms
+$(obj)/note.o: $(objtree)/xen-syms
 	$(OBJCOPY) -O binary --only-section=.note.gnu.build-id $< $@.bin
 	$(OBJCOPY) $(OBJCOPY_MAGIC) \
 		   --rename-section=.data=.livepatch.depends,alloc,load,readonly,data,contents -S $@.bin $@
@@ -52,7 +52,7 @@ $(obj)/note.o: $(BASEDIR)/xen-syms
 # Append .livepatch.xen_depends section
 # with Xen build-id derived from xen-syms.
 #
-$(obj)/xen_note.o: $(BASEDIR)/xen-syms
+$(obj)/xen_note.o: $(objtree)/xen-syms
 	$(OBJCOPY) -O binary --only-section=.note.gnu.build-id $< $@.bin
 	$(OBJCOPY) $(OBJCOPY_MAGIC) \
 		   --rename-section=.data=.livepatch.xen_depends,alloc,load,readonly,data,contents -S $@.bin $@
@@ -125,7 +125,7 @@ xen_action_hooks_norevert-objs := xen_action_hooks_marker.o xen_hello_world_func
 
 EXPECT_BYTES_COUNT := 8
 CODE_GET_EXPECT=$(shell $(OBJDUMP) -d --insn-width=1 $(1) | sed -n -e '/<'$(2)'>:$$/,/^$$/ p' | tail -n +2 | head -n $(EXPECT_BYTES_COUNT) | awk '{$$0=$$2; printf "%s", substr($$0,length-1)}' | sed 's/.\{2\}/0x&,/g' | sed 's/^/{/;s/,$$/}/g')
-$(obj)/expect_config.h: $(BASEDIR)/xen-syms
+$(obj)/expect_config.h: $(objtree)/xen-syms
 	(set -e; \
 	 echo "#define EXPECT_BYTES $(call CODE_GET_EXPECT,$<,xen_extra_version)"; \
          echo "#define EXPECT_BYTES_COUNT $(EXPECT_BYTES_COUNT)") > $@
-- 
Anthony PERARD

[XEN PATCH v9 26/30] build: replace $(BASEDIR) and use $(srctree)

[Anthony PERARD]

$(srctree) is a better description for the source directory than
$(BASEDIR) that has been used for both source and build directory
(which where the same).

This adds $(srctree) to a few path where make's VPATH=$(srctree) won't
apply. And replace $(BASEDIR) by $(srctree).

Introduce "$(srcdir)" as a shortcut for "$(srctree)/$(src)" as the
later is used often enough.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v9:
    - acked
    - reword "# shortcut for $(srctree)/$(src)" into "# shortcuts" in both
      places.
    
    v8:
    - merge of two patchs from v7:
        build: add $(srctree) in few key places
        build: replace $(BASEDIR) by $(srctree)
      both patch were acked
    - introduce $(srcdir) as a shortcut for $(srctree)/$(src)

 xen/Kconfig                 |  4 ++--
 xen/Makefile                |  7 +++----
 xen/Rules.mk                |  9 ++++++---
 xen/arch/x86/arch.mk        |  2 +-
 xen/build.mk                |  4 ++--
 xen/common/Makefile         |  2 +-
 xen/common/libfdt/Makefile  |  2 +-
 xen/include/Makefile        | 14 +++++++-------
 xen/scripts/Kconfig.include |  2 +-
 xen/scripts/Makefile.clean  |  5 ++++-
 xen/xsm/flask/Makefile      | 10 +++++-----
 11 files changed, 33 insertions(+), 28 deletions(-)

diff --git a/xen/Kconfig b/xen/Kconfig
index bcbd2758e5d3..ac9a638d372e 100644
--- a/xen/Kconfig
+++ b/xen/Kconfig
@@ -14,14 +14,14 @@ config CC_IS_GCC
 
 config GCC_VERSION
 	int
-	default $(shell,$(BASEDIR)/scripts/gcc-version.sh $(CC))
+	default $(shell,$(srctree)/scripts/gcc-version.sh $(CC))
 
 config CC_IS_CLANG
 	def_bool $(success,$(CC) --version | head -n 1 | grep -q clang)
 
 config CLANG_VERSION
 	int
-	default $(shell,$(BASEDIR)/scripts/clang-version.sh $(CC))
+	default $(shell,$(srctree)/scripts/clang-version.sh $(CC))
 
 # -fvisibility=hidden reduces -fpic cost, if it's available
 config CC_HAS_VISIBILITY_ATTRIBUTE
diff --git a/xen/Makefile b/xen/Makefile
index 443784dfce80..c39c1699966f 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -19,8 +19,7 @@ export PYTHON		?= $(PYTHON_INTERPRETER)
 
 export CHECKPOLICY	?= checkpolicy
 
-export BASEDIR := $(CURDIR)
-export XEN_ROOT := $(BASEDIR)/..
+export XEN_ROOT := $(CURDIR)/..
 
 abs_objtree := $(CURDIR)
 abs_srctree := $(CURDIR)
@@ -189,7 +188,7 @@ ifeq ($(TARGET_ARCH),x86)
 t1 = $(call as-insn,$(CC),".L0: .L1: .skip (.L1 - .L0)",,-no-integrated-as)
 
 # Check whether clang asm()-s support .include.
-t2 = $(call as-insn,$(CC) -I$(BASEDIR)/arch/x86/include,".include \"asm/asm-defns.h\"",,-no-integrated-as)
+t2 = $(call as-insn,$(CC) -I$(srctree)/arch/x86/include,".include \"asm/asm-defns.h\"",,-no-integrated-as)
 
 # Check whether clang keeps .macro-s between asm()-s:
 # https://bugs.llvm.org/show_bug.cgi?id=36110
@@ -329,7 +328,7 @@ ALL_OBJS-$(CONFIG_CRYPTO) += crypto/built_in.o
 
 ALL_LIBS-y                := lib/lib.a
 
-include $(BASEDIR)/arch/$(TARGET_ARCH)/arch.mk
+include $(srctree)/arch/$(TARGET_ARCH)/arch.mk
 
 # define new variables to avoid the ones defined in Config.mk
 export XEN_CFLAGS := $(CFLAGS)
diff --git a/xen/Rules.mk b/xen/Rules.mk
index 7712bfa063e0..57a029455586 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -9,13 +9,16 @@ endif
 
 src := $(obj)
 
+# shortcuts
+srcdir := $(srctree)/$(src)
+
 PHONY := __build
 __build:
 
 -include $(objtree)/include/config/auto.conf
 
 include $(XEN_ROOT)/Config.mk
-include $(BASEDIR)/scripts/Kbuild.include
+include $(srctree)/scripts/Kbuild.include
 
 # Initialise some variables
 obj-y :=
@@ -58,7 +61,7 @@ cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS) $< $@
 # binfile
 # use e.g. $(call if_changed,binfile,binary-file varname)
 quiet_cmd_binfile = BINFILE $@
-cmd_binfile = $(SHELL) $(BASEDIR)/tools/binfile $(BINFILE_FLAGS) $@ $(2)
+cmd_binfile = $(SHELL) $(srctree)/tools/binfile $(BINFILE_FLAGS) $@ $(2)
 
 # Figure out what we need to build from the various variables
 # ===========================================================================
@@ -177,7 +180,7 @@ cpp_flags = $(filter-out -Wa$(comma)% -flto,$(1))
 c_flags = -MMD -MP -MF $(depfile) $(XEN_CFLAGS)
 a_flags = -MMD -MP -MF $(depfile) $(XEN_AFLAGS)
 
-include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
+include $(srctree)/arch/$(TARGET_ARCH)/Rules.mk
 
 c_flags += $(_c_flags)
 a_flags += $(_c_flags)
diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
index 8bc0e01ceb2b..2fcb6271faf7 100644
--- a/xen/arch/x86/arch.mk
+++ b/xen/arch/x86/arch.mk
@@ -63,7 +63,7 @@ ifneq ($(CONFIG_PV_SHIM_EXCLUSIVE),y)
 efi-check := arch/x86/efi/check
 
 # Check if the compiler supports the MS ABI.
-XEN_BUILD_EFI := $(call if-success,$(CC) $(CFLAGS) -c $(efi-check).c -o $(efi-check).o,y)
+XEN_BUILD_EFI := $(call if-success,$(CC) $(CFLAGS) -c $(srctree)/$(efi-check).c -o $(efi-check).o,y)
 
 # Check if the linker supports PE.
 EFI_LDFLAGS := $(patsubst -m%,-mi386pep,$(LDFLAGS)) --subsystem=10
diff --git a/xen/build.mk b/xen/build.mk
index e718743ef772..758590c68eab 100644
--- a/xen/build.mk
+++ b/xen/build.mk
@@ -27,9 +27,9 @@ define cmd_compile.h
 	    -e 's/@@version@@/$(XEN_VERSION)/g' \
 	    -e 's/@@subversion@@/$(XEN_SUBVERSION)/g' \
 	    -e 's/@@extraversion@@/$(XEN_EXTRAVERSION)/g' \
-	    -e 's!@@changeset@@!$(shell tools/scmversion $(XEN_ROOT) || echo "unavailable")!g' \
+	    -e 's!@@changeset@@!$(shell $(srctree)/tools/scmversion $(XEN_ROOT) || echo "unavailable")!g' \
 	    < $< > $(dot-target).tmp; \
-	sed -rf tools/process-banner.sed < .banner >> $(dot-target).tmp; \
+	sed -rf $(srctree)/tools/process-banner.sed < .banner >> $(dot-target).tmp; \
 	mv -f $(dot-target).tmp $@; \
     fi
 endef
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 30641a737231..b1e076c30b81 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -80,7 +80,7 @@ $(obj)/config.gz: $(CONF_FILE)
 
 $(obj)/config_data.o: $(obj)/config.gz
 
-$(obj)/config_data.S: $(BASEDIR)/tools/binfile FORCE
+$(obj)/config_data.S: $(srctree)/tools/binfile FORCE
 	$(call if_changed,binfile,$(obj)/config.gz xen_config_data)
 targets += config_data.S
 
diff --git a/xen/common/libfdt/Makefile b/xen/common/libfdt/Makefile
index 6708af12e583..75aaefa2e37f 100644
--- a/xen/common/libfdt/Makefile
+++ b/xen/common/libfdt/Makefile
@@ -6,7 +6,7 @@ OBJCOPYFLAGS := $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s))
 obj-y += libfdt.o
 nocov-y += libfdt.o
 
-CFLAGS-y += -I$(BASEDIR)/include/xen/libfdt/
+CFLAGS-y += -I$(srctree)/include/xen/libfdt/
 
 $(obj)/libfdt.o: $(obj)/libfdt-temp.o FORCE
 	$(call if_changed,objcopy)
diff --git a/xen/include/Makefile b/xen/include/Makefile
index a3c2511f5f60..5a2b4c9f65fa 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -45,22 +45,22 @@ public-$(CONFIG_ARM) := $(wildcard $(src)/public/arch-arm/*.h $(src)/public/arch
 .PHONY: all
 all: $(addprefix $(obj)/,$(headers-y))
 
-$(obj)/compat/%.h: $(obj)/compat/%.i $(src)/Makefile $(BASEDIR)/tools/compat-build-header.py
-	$(PYTHON) $(BASEDIR)/tools/compat-build-header.py <$< $(patsubst $(obj)/%,%,$@) >>$@.new; \
+$(obj)/compat/%.h: $(obj)/compat/%.i $(src)/Makefile $(srctree)/tools/compat-build-header.py
+	$(PYTHON) $(srctree)/tools/compat-build-header.py <$< $(patsubst $(obj)/%,%,$@) >>$@.new; \
 	mv -f $@.new $@
 
 $(obj)/compat/%.i: $(obj)/compat/%.c $(src)/Makefile
 	$(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(XEN_CFLAGS)) $(cppflags-y) -o $@ $<
 
-$(obj)/compat/%.c: $(src)/public/%.h $(src)/xlat.lst $(src)/Makefile $(BASEDIR)/tools/compat-build-source.py
+$(obj)/compat/%.c: $(src)/public/%.h $(src)/xlat.lst $(src)/Makefile $(srctree)/tools/compat-build-source.py
 	mkdir -p $(@D)
-	$(PYTHON) $(BASEDIR)/tools/compat-build-source.py $(src)/xlat.lst <$< >$@.new
+	$(PYTHON) $(srctree)/tools/compat-build-source.py $(srcdir)/xlat.lst <$< >$@.new
 	mv -f $@.new $@
 
-$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(BASEDIR)/tools/get-fields.sh $(src)/Makefile
+$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/tools/get-fields.sh $(src)/Makefile
 	export PYTHON=$(PYTHON); \
 	while read what name; do \
-		$(SHELL) $(BASEDIR)/tools/get-fields.sh "$$what" compat_$$name $< || exit $$?; \
+		$(SHELL) $(srctree)/tools/get-fields.sh "$$what" compat_$$name $< || exit $$?; \
 	done <$(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst >$@.new
 	mv -f $@.new $@
 
@@ -70,7 +70,7 @@ $(obj)/compat/.xlat/%.lst: $(src)/xlat.lst $(src)/Makefile
 	grep -v '^[[:blank:]]*#' $< | sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,[[:blank:]]+$*\.h[[:blank:]]*$$,,p' >$@.new
 	$(call move-if-changed,$@.new,$@)
 
-xlat-y := $(shell sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,^[?!][[:blank:]]+[^[:blank:]]+[[:blank:]]+,,p' $(src)/xlat.lst | uniq)
+xlat-y := $(shell sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,^[?!][[:blank:]]+[^[:blank:]]+[[:blank:]]+,,p' $(srcdir)/xlat.lst | uniq)
 xlat-y := $(filter $(patsubst compat/%,%,$(headers-y)),$(xlat-y))
 
 $(obj)/compat/xlat.h: $(addprefix $(obj)/compat/.xlat/,$(xlat-y)) $(obj)/config/auto.conf $(src)/Makefile
diff --git a/xen/scripts/Kconfig.include b/xen/scripts/Kconfig.include
index e1f13e17207e..389a690a127a 100644
--- a/xen/scripts/Kconfig.include
+++ b/xen/scripts/Kconfig.include
@@ -40,4 +40,4 @@ $(error-if,$(failure,command -v $(CC)),compiler '$(CC)' not found)
 $(error-if,$(failure,command -v $(LD)),linker '$(LD)' not found)
 
 # gcc version including patch level
-gcc-version := $(shell,$(BASEDIR)/scripts/gcc-version.sh $(CC))
+gcc-version := $(shell,$(srctree)/scripts/gcc-version.sh $(CC))
diff --git a/xen/scripts/Makefile.clean b/xen/scripts/Makefile.clean
index 4eed31974509..f97ecd3b8eee 100644
--- a/xen/scripts/Makefile.clean
+++ b/xen/scripts/Makefile.clean
@@ -5,9 +5,12 @@
 
 src := $(obj)
 
+# shortcuts
+srcdir := $(srctree)/$(src)
+
 clean::
 
-include $(BASEDIR)/scripts/Kbuild.include
+include $(srctree)/scripts/Kbuild.include
 
 include $(src)/Makefile
 
diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index 4ac6fb8778ae..a99038cb5722 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -8,8 +8,8 @@ CFLAGS-y += -I$(obj)/include
 
 AWK = awk
 
-FLASK_H_DEPEND := $(addprefix $(src)/policy/,security_classes initial_sids)
-AV_H_DEPEND = $(src)/policy/access_vectors
+FLASK_H_DEPEND := $(addprefix $(srcdir)/policy/,security_classes initial_sids)
+AV_H_DEPEND := $(srcdir)/policy/access_vectors
 
 FLASK_H_FILES := flask.h class_to_string.h initial_sid_to_string.h
 AV_H_FILES := av_perm_to_string.h av_permissions.h
@@ -18,14 +18,14 @@ ALL_H_FILES := $(addprefix include/,$(FLASK_H_FILES) $(AV_H_FILES))
 $(addprefix $(obj)/,$(obj-y)) $(obj)/ss/built_in.o: $(addprefix $(obj)/,$(ALL_H_FILES))
 extra-y += $(ALL_H_FILES)
 
-mkflask := $(src)/policy/mkflask.sh
+mkflask := $(srcdir)/policy/mkflask.sh
 quiet_cmd_mkflask = MKFLASK $@
 cmd_mkflask = $(SHELL) $(mkflask) $(AWK) $(obj)/include $(FLASK_H_DEPEND)
 
 $(addprefix $(obj)/%/,$(FLASK_H_FILES)): $(FLASK_H_DEPEND) $(mkflask) FORCE
 	$(call if_changed,mkflask)
 
-mkaccess := $(src)/policy/mkaccess_vector.sh
+mkaccess := $(srcdir)/policy/mkaccess_vector.sh
 quiet_cmd_mkaccess = MKACCESS VECTOR $@
 cmd_mkaccess = $(SHELL) $(mkaccess) $(AWK) $(obj)/include $(AV_H_DEPEND)
 
@@ -36,7 +36,7 @@ obj-bin-$(CONFIG_XSM_FLASK_POLICY) += flask-policy.o
 $(obj)/flask-policy.o: $(obj)/policy.bin
 
 $(obj)/flask-policy.S: BINFILE_FLAGS := -i
-$(obj)/flask-policy.S: $(BASEDIR)/tools/binfile FORCE
+$(obj)/flask-policy.S: $(srctree)/tools/binfile FORCE
 	$(call if_changed,binfile,$(obj)/policy.bin xsm_flask_init_policy)
 targets += flask-policy.S
 
-- 
Anthony PERARD

[XEN PATCH v9 27/30] build: rework "headers*.chk" prerequisite in include/

[Anthony PERARD]

Listing public headers when out-of-tree build are involved becomes
more annoying where every path to every headers needs to start with
"$(srctree)/$(src)", or $(wildcard ) will not work. This means more
repetition. ( "$(srcdir)" is a shortcut for "$(srctree)/$(src)" )

This patch attempt to reduce the amount of duplication and make better
use of make's meta programming capability. The filters are now listed
in a variable and don't have to repeat the path to the headers files
as this is added later as needed.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v9:
    - reorder *-headers variables
    - call public-filter-headers on $(public-c99-headers) as well, for this I
      needed to rework $(public-c99-headers) and $(public-ansi-headers-filter)
    
    v8:
    - add prefix "public-" to newly introduced macros.
    - make use of the new "$(srcdir)" shortcut.

 xen/include/Makefile | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/xen/include/Makefile b/xen/include/Makefile
index 5a2b4c9f65fa..fddf5a575bf6 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -39,8 +39,8 @@ cppflags-$(CONFIG_X86)    += -m32
 
 endif
 
-public-$(CONFIG_X86) := $(wildcard $(src)/public/arch-x86/*.h $(src)/public/arch-x86/*/*.h)
-public-$(CONFIG_ARM) := $(wildcard $(src)/public/arch-arm/*.h $(src)/public/arch-arm/*/*.h)
+public-$(CONFIG_X86) := $(wildcard $(srcdir)/public/arch-x86/*.h $(srcdir)/public/arch-x86/*/*.h)
+public-$(CONFIG_ARM) := $(wildcard $(srcdir)/public/arch-arm/*.h $(srcdir)/public/arch-arm/*/*.h)
 
 .PHONY: all
 all: $(addprefix $(obj)/,$(headers-y))
@@ -81,10 +81,23 @@ ifeq ($(XEN_TARGET_ARCH),$(XEN_COMPILE_ARCH))
 
 all: $(obj)/headers.chk $(obj)/headers99.chk $(obj)/headers++.chk
 
-PUBLIC_HEADERS := $(filter-out $(src)/public/arch-% $(src)/public/dom0_ops.h, $(wildcard $(src)/public/*.h $(src)/public/*/*.h) $(public-y))
+public-hdrs-path := $(srcdir)/public
 
-PUBLIC_C99_HEADERS := $(src)/public/io/9pfs.h $(src)/public/io/pvcalls.h
-PUBLIC_ANSI_HEADERS := $(filter-out $(src)/public/%ctl.h $(src)/public/xsm/% $(src)/public/%hvm/save.h $(PUBLIC_C99_HEADERS), $(PUBLIC_HEADERS))
+public-list-headers = $(wildcard $1/*.h $1/*/*.h)
+public-filter-headers = $(filter-out $(addprefix $(public-hdrs-path)/, $($1-filter)), $($1))
+
+public-headers := $(call public-list-headers, $(public-hdrs-path)) $(public-y)
+public-ansi-headers := $(public-headers)
+public-c99-headers := $(addprefix $(public-hdrs-path)/, io/9pfs.h io/pvcalls.h)
+
+public-headers-filter := dom0_ops.h arch-%
+public-ansi-headers-filter := %ctl.h xsm/% %hvm/save.h $(public-headers-filter) \
+    $(patsubst $(public-hdrs-path)/%,%,$(public-c99-headers))
+public-c99-headers-filter :=
+
+PUBLIC_HEADERS := $(call public-filter-headers,public-headers)
+PUBLIC_ANSI_HEADERS := $(call public-filter-headers,public-ansi-headers)
+PUBLIC_C99_HEADERS := $(call public-filter-headers,public-c99-headers)
 
 $(src)/public/io/9pfs.h-prereq := string
 $(src)/public/io/pvcalls.h-prereq := string
-- 
Anthony PERARD

[XEN PATCH v9 28/30] build: specify source tree in include/ for prerequisite

[Anthony PERARD]

When doing an out-of-tree build, and thus setting VPATH,
GNU Make 3.81 on Ubuntu Trusty complains about Circular dependency of
include/Makefile and include/xlat.lst and drop them. The build fails
later due to headers malformed.

This might be due to bug #13529
    "Incorrect circular dependancy"
    https://savannah.gnu.org/bugs/?13529
which was fixed in 3.82.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v9:
    - add potential bug report link in commit message
    
    v8:
    - make use of the new "$(srcdir)" shortcut
    - move the patch ahead of the problematic patch:
        build: adding out-of-tree support to the xen build

 xen/include/Makefile | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/xen/include/Makefile b/xen/include/Makefile
index fddf5a575bf6..c8c4bcd93bd3 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -45,19 +45,19 @@ public-$(CONFIG_ARM) := $(wildcard $(srcdir)/public/arch-arm/*.h $(srcdir)/publi
 .PHONY: all
 all: $(addprefix $(obj)/,$(headers-y))
 
-$(obj)/compat/%.h: $(obj)/compat/%.i $(src)/Makefile $(srctree)/tools/compat-build-header.py
+$(obj)/compat/%.h: $(obj)/compat/%.i $(srcdir)/Makefile $(srctree)/tools/compat-build-header.py
 	$(PYTHON) $(srctree)/tools/compat-build-header.py <$< $(patsubst $(obj)/%,%,$@) >>$@.new; \
 	mv -f $@.new $@
 
-$(obj)/compat/%.i: $(obj)/compat/%.c $(src)/Makefile
+$(obj)/compat/%.i: $(obj)/compat/%.c $(srcdir)/Makefile
 	$(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(XEN_CFLAGS)) $(cppflags-y) -o $@ $<
 
-$(obj)/compat/%.c: $(src)/public/%.h $(src)/xlat.lst $(src)/Makefile $(srctree)/tools/compat-build-source.py
+$(obj)/compat/%.c: $(src)/public/%.h $(srcdir)/xlat.lst $(srcdir)/Makefile $(srctree)/tools/compat-build-source.py
 	mkdir -p $(@D)
 	$(PYTHON) $(srctree)/tools/compat-build-source.py $(srcdir)/xlat.lst <$< >$@.new
 	mv -f $@.new $@
 
-$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/tools/get-fields.sh $(src)/Makefile
+$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/tools/get-fields.sh $(srcdir)/Makefile
 	export PYTHON=$(PYTHON); \
 	while read what name; do \
 		$(SHELL) $(srctree)/tools/get-fields.sh "$$what" compat_$$name $< || exit $$?; \
@@ -65,7 +65,7 @@ $(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst $(srctree)/
 	mv -f $@.new $@
 
 .PRECIOUS: $(obj)/compat/.xlat/%.lst
-$(obj)/compat/.xlat/%.lst: $(src)/xlat.lst $(src)/Makefile
+$(obj)/compat/.xlat/%.lst: $(srcdir)/xlat.lst $(srcdir)/Makefile
 	mkdir -p $(@D)
 	grep -v '^[[:blank:]]*#' $< | sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,[[:blank:]]+$*\.h[[:blank:]]*$$,,p' >$@.new
 	$(call move-if-changed,$@.new,$@)
@@ -73,7 +73,7 @@ $(obj)/compat/.xlat/%.lst: $(src)/xlat.lst $(src)/Makefile
 xlat-y := $(shell sed -ne 's,@arch@,$(compat-arch-y),g' -re 's,^[?!][[:blank:]]+[^[:blank:]]+[[:blank:]]+,,p' $(srcdir)/xlat.lst | uniq)
 xlat-y := $(filter $(patsubst compat/%,%,$(headers-y)),$(xlat-y))
 
-$(obj)/compat/xlat.h: $(addprefix $(obj)/compat/.xlat/,$(xlat-y)) $(obj)/config/auto.conf $(src)/Makefile
+$(obj)/compat/xlat.h: $(addprefix $(obj)/compat/.xlat/,$(xlat-y)) $(obj)/config/auto.conf $(srcdir)/Makefile
 	cat $(filter %.h,$^) >$@.new
 	mv -f $@.new $@
 
@@ -102,7 +102,7 @@ PUBLIC_C99_HEADERS := $(call public-filter-headers,public-c99-headers)
 $(src)/public/io/9pfs.h-prereq := string
 $(src)/public/io/pvcalls.h-prereq := string
 
-$(obj)/headers.chk: $(PUBLIC_ANSI_HEADERS) $(src)/Makefile
+$(obj)/headers.chk: $(PUBLIC_ANSI_HEADERS) $(srcdir)/Makefile
 	for i in $(filter %.h,$^); do \
 	    $(CC) -x c -ansi -Wall -Werror -include stdint.h \
 	          -S -o /dev/null $$i || exit 1; \
@@ -110,7 +110,7 @@ $(obj)/headers.chk: $(PUBLIC_ANSI_HEADERS) $(src)/Makefile
 	done >$@.new
 	mv $@.new $@
 
-$(obj)/headers99.chk: $(PUBLIC_C99_HEADERS) $(src)/Makefile
+$(obj)/headers99.chk: $(PUBLIC_C99_HEADERS) $(srcdir)/Makefile
 	rm -f $@.new
 	$(foreach i, $(filter %.h,$^),                                        \
 	    echo "#include "\"$(i)\"                                          \
@@ -120,7 +120,7 @@ $(obj)/headers99.chk: $(PUBLIC_C99_HEADERS) $(src)/Makefile
 	    || exit $$?; echo $(i) >> $@.new;)
 	mv $@.new $@
 
-$(obj)/headers++.chk: $(PUBLIC_HEADERS) $(src)/Makefile
+$(obj)/headers++.chk: $(PUBLIC_HEADERS) $(srcdir)/Makefile
 	rm -f $@.new
 	if ! $(CXX) -v >/dev/null 2>&1; then                                  \
 	    touch $@.new;                                                     \
-- 
Anthony PERARD

[XEN PATCH v9 29/30] build: shuffle main Makefile

[Anthony PERARD]

Reorganize a bit the Makefile ahead of patch
"build: adding out-of-tree support to the xen build"

We are going to want to calculate all the $(*srctree) and $(*objtree)
once, when we can calculate them. This can happen within the
"$(root-make-done)" guard, in an out-of-tree build scenario, so move
those variable there.

$(XEN_ROOT) is going to depends on the value of $(abs_srctree) so
needs to move as well. "Kbuild.include" also depends on $(srctree).

Next, "Config.mk" depends on $(XEN_ROOT) and $(TARGET_*ARCH) depends
on "Config.mk" so those needs to move as well.

This should only be code movement without functional changes.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v9:
    - add some explanation in the commit message about why the code movement
      is needed.
    
    v8:
    - new patch

 xen/Makefile | 46 +++++++++++++++++++++++-----------------------
 1 file changed, 23 insertions(+), 23 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index c39c1699966f..08114b3fef1e 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -19,17 +19,6 @@ export PYTHON		?= $(PYTHON_INTERPRETER)
 
 export CHECKPOLICY	?= checkpolicy
 
-export XEN_ROOT := $(CURDIR)/..
-
-abs_objtree := $(CURDIR)
-abs_srctree := $(CURDIR)
-
-export abs_srctree abs_objtree
-
-srctree := .
-objtree := .
-export srctree objtree
-
 # Do not use make's built-in rules and variables
 MAKEFLAGS += -rR
 
@@ -41,16 +30,6 @@ SRCARCH=$(shell echo $(ARCH) | \
               -e s'/riscv.*/riscv/g')
 export ARCH SRCARCH
 
-# Don't break if the build process wasn't called from the top level
-# we need XEN_TARGET_ARCH to generate the proper config
-include $(XEN_ROOT)/Config.mk
-
-# Set ARCH/SUBARCH appropriately.
-export TARGET_SUBARCH  := $(XEN_TARGET_ARCH)
-export TARGET_ARCH     := $(shell echo $(XEN_TARGET_ARCH) | \
-                            sed -e 's/x86.*/x86/' -e s'/arm\(32\|64\)/arm/g' \
-                                -e s'/riscv.*/riscv/g')
-
 # Allow someone to change their config file
 export KCONFIG_CONFIG ?= .config
 
@@ -64,8 +43,6 @@ default: build
 .PHONY: dist
 dist: install
 
-include scripts/Kbuild.include
-
 ifneq ($(root-make-done),y)
 # section to run before calling Rules.mk, but only once.
 
@@ -141,6 +118,17 @@ endif
 
 export quiet Q KBUILD_VERBOSE
 
+abs_objtree := $(CURDIR)
+abs_srctree := $(CURDIR)
+
+export abs_srctree abs_objtree
+
+srctree := .
+objtree := .
+export srctree objtree
+
+export XEN_ROOT := $(CURDIR)/..
+
 # To make sure we do not include .config for any of the *config targets
 # catch them early, and hand them over to tools/kconfig/Makefile
 
@@ -163,6 +151,18 @@ ifneq ($(filter %config,$(MAKECMDGOALS)),)
     config-build := y
 endif
 
+include scripts/Kbuild.include
+
+# Don't break if the build process wasn't called from the top level
+# we need XEN_TARGET_ARCH to generate the proper config
+include $(XEN_ROOT)/Config.mk
+
+# Set ARCH/SUBARCH appropriately.
+export TARGET_SUBARCH  := $(XEN_TARGET_ARCH)
+export TARGET_ARCH     := $(shell echo $(XEN_TARGET_ARCH) | \
+                            sed -e 's/x86.*/x86/' -e s'/arm\(32\|64\)/arm/g' \
+                                -e s'/riscv.*/riscv/g')
+
 export CONFIG_SHELL := $(SHELL)
 export YACC = $(if $(BISON),$(BISON),bison)
 export LEX = $(if $(FLEX),$(FLEX),flex)
-- 
Anthony PERARD

[XEN PATCH v9 30/30] build: adding out-of-tree support to the xen build

[Anthony PERARD]

This implement out-of-tree support, there's two ways to create an
out-of-tree build tree (after that, `make` in that new directory
works):
    make O=build
    mkdir build; cd build; make -f ../Makefile
also works with an absolute path for both.

This implementation only works if the source tree is clean, as we use
VPATH.

This patch copies most new code with handling out-of-tree build from
Linux v5.12.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v9:
    - acked
    - add "source -> ." symlink in the in-tree build as well. This allow to
      make relative symlink to the "common/efi/*.c" sources.
    - also now, the "source" symlink is removed on `distclean`, add added to
      .gitignore.
    
    v8:
    - replace script/mkmakefile by cmd_makefile like it's been done in Linux
      5.13.
    - fix GNU Make version, replace few 0.81 by 3.81.
    - reorder include header path in CFLAGS ( -I ), to have the $(objtree)
      paths listed before the $(srctree) paths. This will be usefull later
      if we allow to build out-of-tree from a dirty source tree.
    - make use of -iquote
    - sometime, add -iquote path even when not necessary when building in-tree.
    - extract code movement into a separate patch
    - make use of the new "$(srcdir)" shortcut
    - split command line in headers*.chk target into more lines

 .gitignore                   |   1 +
 xen/Makefile                 | 143 +++++++++++++++++++++++++++++++----
 xen/Rules.mk                 |  11 ++-
 xen/arch/x86/arch.mk         |   5 +-
 xen/arch/x86/boot/Makefile   |   6 ++
 xen/common/efi/efi-common.mk |   3 +-
 xen/include/Makefile         |  11 ++-
 xen/test/livepatch/Makefile  |   2 +
 xen/xsm/flask/Makefile       |   3 +-
 xen/xsm/flask/ss/Makefile    |   1 +
 10 files changed, 164 insertions(+), 22 deletions(-)

diff --git a/.gitignore b/.gitignore
index cc87a7dfae7b..0464098ed2b5 100644
--- a/.gitignore
+++ b/.gitignore
@@ -329,6 +329,7 @@ xen/include/xen/*.new
 xen/include/xen/acm_policy.h
 xen/include/xen/compile.h
 xen/include/xen/lib/x86/cpuid-autogen.h
+xen/source
 xen/test/livepatch/config.h
 xen/test/livepatch/expect_config.h
 xen/test/livepatch/*.livepatch
diff --git a/xen/Makefile b/xen/Makefile
index 08114b3fef1e..499f9580dab8 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -1,3 +1,7 @@
+# $(lastword,) for GNU Make older than 3.81
+lastword = $(word $(words $(1)),$(1))
+this-makefile := $(call lastword,$(MAKEFILE_LIST))
+
 # This is the correct place to edit the build version.
 # All other places this is stored (eg. compile.h) should be autogenerated.
 export XEN_VERSION       = 4
@@ -19,6 +23,13 @@ export PYTHON		?= $(PYTHON_INTERPRETER)
 
 export CHECKPOLICY	?= checkpolicy
 
+$(if $(filter __%, $(MAKECMDGOALS)), \
+    $(error targets prefixed with '__' are only for internal use))
+
+# That's our default target when none is given on the command line
+PHONY := __all
+__all:
+
 # Do not use make's built-in rules and variables
 MAKEFLAGS += -rR
 
@@ -37,9 +48,6 @@ export CC CXX LD
 
 export TARGET := xen
 
-.PHONY: default
-default: build
-
 .PHONY: dist
 dist: install
 
@@ -118,16 +126,88 @@ endif
 
 export quiet Q KBUILD_VERBOSE
 
+# $(realpath,) for GNU Make older than 3.81
+realpath = $(wildcard $(foreach file,$(1),$(shell cd -P $(dir $(file)) && echo "$$PWD/$(notdir $(file))")))
+
+ifeq ("$(origin O)", "command line")
+    KBUILD_OUTPUT := $(O)
+endif
+
+ifneq ($(KBUILD_OUTPUT),)
+# Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
+# expand a shell special character '~'. We use a somewhat tedious way here.
+abs_objtree := $(shell mkdir -p $(KBUILD_OUTPUT) && cd $(KBUILD_OUTPUT) && pwd)
+$(if $(abs_objtree),, \
+     $(error failed to create output directory "$(KBUILD_OUTPUT)"))
+
+# $(realpath ...) resolves symlinks
+abs_objtree := $(call realpath,$(abs_objtree))
+else
 abs_objtree := $(CURDIR)
-abs_srctree := $(CURDIR)
+endif
+
+ifeq ($(abs_objtree),$(CURDIR))
+# Suppress "Entering directory ..." unless we are changing the work directory.
+MAKEFLAGS += --no-print-directory
+else
+need-sub-make := 1
+endif
+
+abs_srctree := $(call realpath,$(dir $(this-makefile)))
+
+ifneq ($(words $(subst :, ,$(abs_srctree))), 1)
+$(error source directory cannot contain spaces or colons)
+endif
+
+ifneq ($(abs_srctree),$(abs_objtree))
+# Look for make include files relative to root of kernel src
+#
+# This does not become effective immediately because MAKEFLAGS is re-parsed
+# once after the Makefile is read. We need to invoke sub-make.
+MAKEFLAGS += --include-dir=$(abs_srctree)
+need-sub-make := 1
+endif
 
 export abs_srctree abs_objtree
+export root-make-done := y
+
+ifeq ($(need-sub-make),1)
+
+PHONY += $(MAKECMDGOALS) __sub-make
+
+$(filter-out $(this-makefile), $(MAKECMDGOALS)) __all: __sub-make
+	@:
+
+# Invoke a second make in the output directory, passing relevant variables
+__sub-make:
+	$(Q)$(MAKE) -C $(abs_objtree) -f $(abs_srctree)/Makefile $(MAKECMDGOALS)
+
+endif # need-sub-make
+endif # root-make-done
+
+# We process the rest of the Makefile if this is the final invocation of make
+ifeq ($(need-sub-make),)
+
+ifeq ($(abs_srctree),$(abs_objtree))
+    # building in the source tree
+    srctree := .
+    building_out_of_srctree :=
+else
+    ifeq ($(abs_srctree)/,$(dir $(abs_objtree)))
+        # building in a subdirectory of the source tree
+        srctree := ..
+    else
+        srctree := $(abs_srctree)
+    endif
+    building_out_of_srctree := 1
+endif
 
-srctree := .
 objtree := .
-export srctree objtree
+VPATH := $(srctree)
+
+export building_out_of_srctree srctree objtree VPATH
 
-export XEN_ROOT := $(CURDIR)/..
+export XEN_ROOT := $(abs_srctree)/..
 
 # To make sure we do not include .config for any of the *config targets
 # catch them early, and hand them over to tools/kconfig/Makefile
@@ -204,9 +284,6 @@ endif
 
 export XEN_HAS_CHECKPOLICY := $(call success,$(CHECKPOLICY) -h 2>&1 | grep -q xen)
 
-export root-make-done := y
-endif # root-make-done
-
 # ===========================================================================
 # Rules shared between *config targets and build targets
 
@@ -214,6 +291,37 @@ PHONY += tools_fixdep
 tools_fixdep:
 	$(Q)$(MAKE) $(build)=tools tools/fixdep
 
+PHONY += outputmakefile
+# Before starting out-of-tree build, make sure the source tree is clean.
+# outputmakefile generates a Makefile in the output directory, if using a
+# separate output directory. This allows convenient use of make in the
+# output directory.
+# At the same time when output Makefile generated, generate .gitignore to
+# ignore whole output directory
+
+quiet_cmd_makefile = GEN     Makefile
+cmd_makefile = { \
+    echo "\# Automatically generated by $(srctree)/Makefile: don't edit"; \
+    echo "include $(srctree)/Makefile"; \
+    } > Makefile
+
+outputmakefile:
+	$(Q)ln -fsn $(srctree) source
+ifdef building_out_of_srctree
+	$(Q)if [ -f $(srctree)/.config -o \
+		 -d $(srctree)/include/config -o \
+		 -d $(srctree)/include/generated ]; then \
+		echo >&2 "***"; \
+		echo >&2 "*** The source tree is not clean, please run 'make$(if $(findstring command line, $(origin XEN_TARGET_ARCH)), XEN_TARGET_ARCH=$(XEN_TARGET_ARCH)) distclean'"; \
+		echo >&2 "*** in $(abs_srctree)";\
+		echo >&2 "***"; \
+		false; \
+	fi
+	$(call cmd,makefile)
+	$(Q)test -e .gitignore || \
+	{ echo "# this is build directory, ignore it"; echo "*"; } > .gitignore
+endif
+
 ifeq ($(config-build),y)
 # ===========================================================================
 # *config targets only - make sure prerequisites are updated, and descend
@@ -229,13 +337,13 @@ filechk_kconfig_allconfig = \
 .allconfig.tmp: FORCE
 	set -e; { $(call filechk_kconfig_allconfig); } > $@
 
-config: tools_fixdep FORCE
+config: tools_fixdep outputmakefile FORCE
 	$(Q)$(MAKE) $(build)=tools/kconfig $@
 
 # Config.mk tries to include .config file, don't try to remake it
 %/.config: ;
 
-%config: .allconfig.tmp tools_fixdep FORCE
+%config: .allconfig.tmp tools_fixdep outputmakefile FORCE
 	$(Q)$(MAKE) $(build)=tools/kconfig KCONFIG_ALLCONFIG=$< $@
 
 else # !config-build
@@ -315,6 +423,10 @@ CFLAGS += -flto
 LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
 endif
 
+ifdef building_out_of_srctree
+    CFLAGS += -I$(objtree)/include
+    CFLAGS += -I$(objtree)/arch/$(TARGET_ARCH)/include
+endif
 CFLAGS += -I$(srctree)/include
 CFLAGS += -I$(srctree)/arch/$(TARGET_ARCH)/include
 
@@ -338,6 +450,8 @@ export CFLAGS_UBSAN
 
 endif # need-config
 
+__all: build
+
 main-targets := build install uninstall clean distclean MAP
 .PHONY: $(main-targets)
 ifneq ($(XEN_TARGET_ARCH),x86_32)
@@ -434,13 +548,13 @@ _clean:
 
 .PHONY: _distclean
 _distclean: clean
-	rm -f tags TAGS cscope.files cscope.in.out cscope.out cscope.po.out GTAGS GPATH GRTAGS GSYMS .config
+	rm -f tags TAGS cscope.files cscope.in.out cscope.out cscope.po.out GTAGS GPATH GRTAGS GSYMS .config source
 
 $(TARGET).gz: $(TARGET)
 	gzip -n -f -9 < $< > $@.new
 	mv $@.new $@
 
-$(TARGET): FORCE
+$(TARGET): outputmakefile FORCE
 	$(Q)$(MAKE) $(build)=tools
 	$(Q)$(MAKE) $(build)=. include/xen/compile.h
 	$(Q)$(MAKE) $(build)=include all
@@ -509,6 +623,7 @@ cloc:
 	done | cloc --list-file=-
 
 endif #config-build
+endif # need-sub-make
 
 PHONY += FORCE
 FORCE:
diff --git a/xen/Rules.mk b/xen/Rules.mk
index 57a029455586..70b7489ea89b 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -37,7 +37,7 @@ SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
                          $(foreach r,rel rel.ro,data.$(r).local)
 
 # The filename build.mk has precedence over Makefile
-include $(firstword $(wildcard $(src)/build.mk) $(src)/Makefile)
+include $(firstword $(wildcard $(srcdir)/build.mk) $(srcdir)/Makefile)
 
 # Linking
 # ---------------------------------------------------------------------------
@@ -328,6 +328,15 @@ existing-targets := $(wildcard $(sort $(targets)))
 
 -include $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).cmd)
 
+# Create directories for object files if they do not exist
+obj-dirs := $(sort $(patsubst %/,%, $(dir $(targets))))
+# If targets exist, their directories apparently exist. Skip mkdir.
+existing-dirs := $(sort $(patsubst %/,%, $(dir $(existing-targets))))
+obj-dirs := $(strip $(filter-out $(existing-dirs), $(obj-dirs)))
+ifneq ($(obj-dirs),)
+$(shell mkdir -p $(obj-dirs))
+endif
+
 # Declare the contents of the PHONY variable as phony.  We keep that
 # information in a variable so we can use it in if_changed and friends.
 .PHONY: $(PHONY)
diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
index 2fcb6271faf7..5c5795d17bd0 100644
--- a/xen/arch/x86/arch.mk
+++ b/xen/arch/x86/arch.mk
@@ -62,6 +62,9 @@ ifneq ($(CONFIG_PV_SHIM_EXCLUSIVE),y)
 
 efi-check := arch/x86/efi/check
 
+# Create the directory for out-of-tree build
+$(shell mkdir -p $(dir $(efi-check)))
+
 # Check if the compiler supports the MS ABI.
 XEN_BUILD_EFI := $(call if-success,$(CC) $(CFLAGS) -c $(srctree)/$(efi-check).c -o $(efi-check).o,y)
 
@@ -101,6 +104,6 @@ export EFI_LDFLAGS
 endif
 
 # Set up the assembler include path properly for older toolchains.
-CFLAGS += -Wa,-I$(srctree)/include
+CFLAGS += -Wa,-I$(objtree)/include -Wa,-I$(srctree)/include
 
 ALL_OBJS-y := arch/x86/boot/built_in.o arch/x86/efi/built_in.o $(ALL_OBJS-y)
diff --git a/xen/arch/x86/boot/Makefile b/xen/arch/x86/boot/Makefile
index ca8001c72b23..784655f5e2bd 100644
--- a/xen/arch/x86/boot/Makefile
+++ b/xen/arch/x86/boot/Makefile
@@ -7,11 +7,17 @@ targets += $(head-srcs:.S=.o)
 
 head-srcs := $(addprefix $(obj)/, $(head-srcs))
 
+ifdef building_out_of_srctree
+$(obj)/head.o: CFLAGS-y += -iquote $(obj)
+endif
 $(obj)/head.o: $(head-srcs)
 
 CFLAGS_x86_32 := $(subst -m64,-m32 -march=i686,$(XEN_TREEWIDE_CFLAGS))
 $(call cc-options-add,CFLAGS_x86_32,CC,$(EMBEDDED_EXTRA_CFLAGS))
 CFLAGS_x86_32 += -Werror -fno-builtin -g0 -msoft-float
+ifdef building_out_of_srctree
+CFLAGS_x86_32 += -I$(objtree)/include
+endif
 CFLAGS_x86_32 += -I$(srctree)/include
 
 # override for 32bit binaries
diff --git a/xen/common/efi/efi-common.mk b/xen/common/efi/efi-common.mk
index ad3c6f2569c3..0331cb57e789 100644
--- a/xen/common/efi/efi-common.mk
+++ b/xen/common/efi/efi-common.mk
@@ -3,12 +3,13 @@ EFIOBJ-$(CONFIG_COMPAT) += compat.o
 
 CFLAGS-y += -fshort-wchar
 CFLAGS-y += -iquote $(srctree)/common/efi
+CFLAGS-y += -iquote $(srcdir)
 
 # Part of the command line transforms $(obj) in to a relative reverted path.
 # e.g.: It transforms "dir/foo/bar" into successively
 #       "dir foo bar", ".. .. ..", "../../.."
 $(obj)/%.c: $(srctree)/common/efi/%.c FORCE
-	$(Q)ln -nfs $(subst $(space),/,$(patsubst %,..,$(subst /, ,$(obj))))/common/efi/$(<F) $@
+	$(Q)ln -nfs $(subst $(space),/,$(patsubst %,..,$(subst /, ,$(obj))))/source/common/efi/$(<F) $@
 
 clean-files += $(patsubst %.o, %.c, $(EFIOBJ-y:.init.o=.o) $(EFIOBJ-))
 
diff --git a/xen/include/Makefile b/xen/include/Makefile
index c8c4bcd93bd3..03baf10efb77 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -115,7 +115,8 @@ $(obj)/headers99.chk: $(PUBLIC_C99_HEADERS) $(srcdir)/Makefile
 	$(foreach i, $(filter %.h,$^),                                        \
 	    echo "#include "\"$(i)\"                                          \
 	    | $(CC) -x c -std=c99 -Wall -Werror                               \
-	      -include stdint.h $(foreach j, $($(i)-prereq), -include $(j).h) \
+	      -include stdint.h                                               \
+	      $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include $(j).h) \
 	      -S -o /dev/null -                                               \
 	    || exit $$?; echo $(i) >> $@.new;)
 	mv $@.new $@
@@ -129,8 +130,9 @@ $(obj)/headers++.chk: $(PUBLIC_HEADERS) $(srcdir)/Makefile
 	$(foreach i, $(filter %.h,$^),                                        \
 	    echo "#include "\"$(i)\"                                          \
 	    | $(CXX) -x c++ -std=gnu++98 -Wall -Werror -D__XEN_TOOLS__        \
-	      -include stdint.h -include $(src)/public/xen.h                  \
-	      $(foreach j, $($(i)-prereq), -include c$(j)) -S -o /dev/null -  \
+	      -include stdint.h -include $(srcdir)/public/xen.h               \
+	      $(foreach j, $($(patsubst $(srctree)/%,%,$i)-prereq), -include c$(j)) \
+	      -S -o /dev/null -                                               \
 	    || exit $$?; echo $(i) >> $@.new;)
 	mv $@.new $@
 
@@ -139,7 +141,8 @@ endif
 ifeq ($(XEN_TARGET_ARCH),x86_64)
 .PHONY: lib-x86-all
 lib-x86-all:
-	$(MAKE) -C $(obj)/xen/lib/x86 all
+	@mkdir -p $(obj)/xen/lib/x86
+	$(MAKE) -C $(obj)/xen/lib/x86 -f $(abs_srctree)/$(src)/xen/lib/x86/Makefile all
 
 all: lib-x86-all
 endif
diff --git a/xen/test/livepatch/Makefile b/xen/test/livepatch/Makefile
index ddb07371315e..c258ab0b5940 100644
--- a/xen/test/livepatch/Makefile
+++ b/xen/test/livepatch/Makefile
@@ -11,6 +11,8 @@ endif
 CODE_ADDR=$(shell nm --defined $(1) | grep $(2) | awk '{print "0x"$$1}')
 CODE_SZ=$(shell nm --defined -S $(1) | grep $(2) | awk '{ print "0x"$$2}')
 
+CFLAGS-y += -iquote $(obj)
+
 extra-y += xen_hello_world.livepatch
 xen_hello_world-objs := xen_hello_world_func.o xen_hello_world.o note.o xen_note.o modinfo.o
 $(obj)/xen_hello_world.o: $(obj)/config.h
diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index a99038cb5722..d25312f4fa1c 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -4,7 +4,8 @@ obj-y += flask_op.o
 
 obj-y += ss/
 
-CFLAGS-y += -I$(obj)/include
+CFLAGS-y += -iquote $(obj)/include
+CFLAGS-y += -I$(srcdir)/include
 
 AWK = awk
 
diff --git a/xen/xsm/flask/ss/Makefile b/xen/xsm/flask/ss/Makefile
index aba1339f3808..ffe92ec19ed6 100644
--- a/xen/xsm/flask/ss/Makefile
+++ b/xen/xsm/flask/ss/Makefile
@@ -8,4 +8,5 @@ obj-y += services.o
 obj-y += conditional.o
 obj-y += mls.o
 
+CFLAGS-y += -iquote $(objtree)/xsm/flask/include
 CFLAGS-y += -I$(srctree)/xsm/flask/include
-- 
Anthony PERARD

Re: [XEN PATCH v9 30/30] build: adding out-of-tree support to the xen build

[Anthony PERARD]

On Tue, Jan 25, 2022 at 11:01:03AM +0000, Anthony PERARD wrote:
> This implement out-of-tree support, there's two ways to create an
> out-of-tree build tree (after that, `make` in that new directory
> works):
>     make O=build
>     mkdir build; cd build; make -f ../Makefile
> also works with an absolute path for both.

Maybe I should have made it more explicit in the description that those
only works within the xen dir, i.e. not the whole project.

-- 
Anthony PERARD

Re: [XEN PATCH v9 05/30] build: prepare to always invoke $(MAKE) from xen/, use $(obj)

[Daniel P. Smith]

On 1/25/22 06:00, Anthony PERARD wrote:
> In a future patch, when building a subdirectory, we will set
> "obj=$subdir" rather than change directory.
> 
> Before that, we add "$(obj)" and "$(src)" in as many places as
> possible where we will need to know which subdirectory is been built.
> "$(obj)" is for files been generated during the build, and "$(src)" is
> for files present in the source tree.
> 
> For now, we set both to "." in Rules.mk and Makefile.clean.
> 
> A few places don't tolerate the addition of "./", this is because make
> remove the leading "./" in targets and dependencies in rules, so these
> will be change later.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Julien Grall <jgrall@amazon.com>
> ---
> 
> Notes:
>     v8:
>     - acked
>     - rebased
> 
>  xen/Rules.mk                            |  5 ++-
>  xen/arch/arm/Makefile                   | 14 ++++----
>  xen/arch/x86/Makefile                   | 48 ++++++++++++-------------
>  xen/arch/x86/boot/Makefile              | 14 ++++----
>  xen/arch/x86/efi/Makefile               |  6 ++--
>  xen/common/Makefile                     |  8 ++---
>  xen/common/libelf/Makefile              |  4 +--
>  xen/common/libfdt/Makefile              |  6 ++--
>  xen/include/Makefile                    | 44 +++++++++++------------
>  xen/scripts/Makefile.clean              |  5 ++-
>  xen/xsm/flask/Makefile                  | 36 +++++++++----------
>  xen/xsm/flask/policy/mkaccess_vector.sh |  7 ++--
>  12 files changed, 103 insertions(+), 94 deletions(-)
> 

<snip/>

> diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
> index 11c530dcf458..51fd37f6c4d5 100644
> --- a/xen/xsm/flask/Makefile
> +++ b/xen/xsm/flask/Makefile
> @@ -4,46 +4,46 @@ obj-y += flask_op.o
>  
>  obj-y += ss/
>  
> -CFLAGS-y += -I./include
> +CFLAGS-y += -I$(obj)/include
>  
>  AWK = awk
>  
> -FLASK_H_DEPEND = policy/security_classes policy/initial_sids
> -AV_H_DEPEND = policy/access_vectors
> +FLASK_H_DEPEND := $(addprefix $(src)/policy/,security_classes initial_sids)
> +AV_H_DEPEND = $(src)/policy/access_vectors
>  
> -FLASK_H_FILES = include/flask.h include/class_to_string.h include/initial_sid_to_string.h
> -AV_H_FILES = include/av_perm_to_string.h include/av_permissions.h
> -ALL_H_FILES = $(FLASK_H_FILES) $(AV_H_FILES)
> +FLASK_H_FILES := flask.h class_to_string.h initial_sid_to_string.h
> +AV_H_FILES := av_perm_to_string.h av_permissions.h
> +ALL_H_FILES := $(addprefix include/,$(FLASK_H_FILES) $(AV_H_FILES))
>  
> -$(obj-y) ss/built_in.o: $(ALL_H_FILES)
> +$(addprefix $(obj)/,$(obj-y)) $(obj)/ss/built_in.o: $(addprefix $(obj)/,$(ALL_H_FILES))
>  extra-y += $(ALL_H_FILES)
>  
> -mkflask := policy/mkflask.sh
> +mkflask := $(src)/policy/mkflask.sh
>  quiet_cmd_mkflask = MKFLASK $@
> -cmd_mkflask = $(SHELL) $(mkflask) $(AWK) include $(FLASK_H_DEPEND)
> +cmd_mkflask = $(SHELL) $(mkflask) $(AWK) $(obj)/include $(FLASK_H_DEPEND)
>  
> -$(subst include/,%/,$(FLASK_H_FILES)): $(FLASK_H_DEPEND) $(mkflask) FORCE
> +$(addprefix $(obj)/%/,$(FLASK_H_FILES)): $(FLASK_H_DEPEND) $(mkflask) FORCE
>  	$(call if_changed,mkflask)
>  
> -mkaccess := policy/mkaccess_vector.sh
> +mkaccess := $(src)/policy/mkaccess_vector.sh
>  quiet_cmd_mkaccess = MKACCESS VECTOR $@
> -cmd_mkaccess = $(SHELL) $(mkaccess) $(AWK) $(AV_H_DEPEND)
> +cmd_mkaccess = $(SHELL) $(mkaccess) $(AWK) $(obj)/include $(AV_H_DEPEND)
>  
> -$(subst include/,%/,$(AV_H_FILES)): $(AV_H_DEPEND) $(mkaccess) FORCE
> +$(addprefix $(obj)/%/,$(AV_H_FILES)): $(AV_H_DEPEND) $(mkaccess) FORCE
>  	$(call if_changed,mkaccess)
>  
>  obj-bin-$(CONFIG_XSM_FLASK_POLICY) += flask-policy.o
> -flask-policy.o: policy.bin
> +$(obj)/flask-policy.o: $(obj)/policy.bin
>  
> -flask-policy.S: BINFILE_FLAGS := -i
> -flask-policy.S: $(BASEDIR)/tools/binfile FORCE
> -	$(call if_changed,binfile,policy.bin xsm_flask_init_policy)
> +$(obj)/flask-policy.S: BINFILE_FLAGS := -i
> +$(obj)/flask-policy.S: $(BASEDIR)/tools/binfile FORCE
> +	$(call if_changed,binfile,$(obj)/policy.bin xsm_flask_init_policy)
>  targets += flask-policy.S
>  
>  FLASK_BUILD_DIR := $(CURDIR)
>  POLICY_SRC := $(FLASK_BUILD_DIR)/xenpolicy-$(XEN_FULLVERSION)
>  
> -policy.bin: FORCE
> +$(obj)/policy.bin: FORCE
>  	$(MAKE) -f $(XEN_ROOT)/tools/flask/policy/Makefile.common \
>  	        -C $(XEN_ROOT)/tools/flask/policy \
>  	        FLASK_BUILD_DIR=$(FLASK_BUILD_DIR) POLICY_FILENAME=$(POLICY_SRC)
> diff --git a/xen/xsm/flask/policy/mkaccess_vector.sh b/xen/xsm/flask/policy/mkaccess_vector.sh
> index 942ede4713f1..ad9772193bff 100755
> --- a/xen/xsm/flask/policy/mkaccess_vector.sh
> +++ b/xen/xsm/flask/policy/mkaccess_vector.sh
> @@ -8,9 +8,12 @@ set -e
>  awk=$1
>  shift
>  
> +output_dir=$1
> +shift
> +
>  # output files
> -av_permissions="include/av_permissions.h"
> -av_perm_to_string="include/av_perm_to_string.h"
> +av_permissions="$output_dir/av_permissions.h"
> +av_perm_to_string="$output_dir/av_perm_to_string.h"
>  
>  cat $* | $awk "
>  BEGIN	{

Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>

V/r,
Daniel P. Smith
Apertus Solutions, LLC

Re: [XEN PATCH v9 09/30] build: build everything from the root dir, use obj=$subdir

[Daniel P. Smith]

On 1/25/22 06:00, Anthony PERARD wrote:
> A subdirectory is now built by setting "$(obj)" instead of changing
> directory. "$(obj)" should always be set when using "Rules.mk" and
> thus a shortcut "$(build)" is introduced and should be used.
> 
> A new variable "$(need-builtin)" is introduce. It is to be used
> whenever a "built_in.o" is wanted from a subdirectory. "built_in.o"
> isn't the main target anymore, and thus only needs to depends on the
> objects that should be part of "built_in.o".
> 
> Introduce $(srctree) and $(objtree) to replace $(BASEDIR) in cases a
> relative path is better, and $(abs_srctree) and $(abs_objtree) which
> have an absolute path.
> 
> DEPS is updated as the existing macro to deal with it doesn't know
> about $(obj).
> 
> There's some changes in "Rules.mk" which in addition to deal with
> "$(obj)" also make it's looks more like "Makefile.build" from Linux
> v5.12.
> 
> test/Makefile doesn't need special handling in order to build
> everything under test/, Rules.mk will visit test/livepatch via
> $(subdir-y), thus "tests" "all" and "build" target are removed.
> "subtree-force-update" target isn't useful so it is removed as well.
> 
> test/livepatch/Makefile doesn't need default target anymore, Rules.mk
> will build everything in $(extra-y) and thus all *.livepatch.
> 
> Adjust cloc recipe: dependency files generated by CC will now have the
> full path to the source file, so we don't need to prepend the
> subdirectory. This fix some issue with source not been parsed by cloc
> before. Also source from tools/kconfig would be listed with changes in
> this patch so adjust the find command to stop listing the "tools"
> directory and thus kconfig. With a default build of Xen on X86, they
> are a few new files parsed by cloc:
>     arch/x86/x86_64/compat/mm.c
>     arch/x86/x86_64/mm.c
>     common/compat/domain.c
>     common/compat/memory.c
>     common/compat/xlat.c
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Acked-by: Bob Eshleman <bobbyeshleman@gmail.com>
> Acked-by: Julien Grall <jgrall@amazon.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> ---
> 
> Notes:
>     v9:
>     - reviewed
>     
>     v8:
>     - rebased
>     - move check for $(obj) in Rules.mk earlier
>     - use $(sort ) when adding directories in $(subdir-y)
>     - fix missing space between "DEPS" and ":="
>     - use -iquote when adding '.' for xen/arch/x86/usercopy.o
>     - introduce also abs_srctree and abs_objtree
>       those were in patch "build: replace $(BASEDIR) by $(objtree)"
>     - reworked cmd_objcopy_fix_sym, we don't need to do anything for old
>       version of clang, and for recent version of clang the change is the
>       same a for gcc.
>     - adjust "cloc" recipe
>     - add some more $(Q) to silent $(MAKE) $(build)=...
>     
>     v7:
>     - Spell "Makefile.build" as "Rules.mk" in the warning in Rules.mk about
>       undefined $(obj).
>     - use $(srctree) for include/xen/config.h
>     - handle $(nocov-y) and $(noubsan-y), needed to add $(obj)/
> 
>  xen/Makefile                |  36 ++++++----
>  xen/Rules.mk                | 138 ++++++++++++++++++++++++------------
>  xen/arch/arm/Makefile       |   4 +-
>  xen/arch/arm/Rules.mk       |   4 --
>  xen/arch/arm/arch.mk        |   4 +-
>  xen/arch/riscv/arch.mk      |   4 +-
>  xen/arch/x86/Makefile       |  11 +--
>  xen/arch/x86/Rules.mk       |   4 +-
>  xen/arch/x86/arch.mk        |  12 ++--
>  xen/arch/x86/boot/Makefile  |   8 +--
>  xen/build.mk                |  12 +++-
>  xen/include/Makefile        |   6 +-
>  xen/scripts/Kbuild.include  |   6 ++
>  xen/test/Makefile           |   7 +-
>  xen/test/livepatch/Makefile |   6 --
>  xen/xsm/flask/Makefile      |   2 +-
>  xen/xsm/flask/ss/Makefile   |   2 +-
>  17 files changed, 162 insertions(+), 104 deletions(-)
> 

<snip/>

> diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
> index 51fd37f6c4d5..49cf730cf098 100644
> --- a/xen/xsm/flask/Makefile
> +++ b/xen/xsm/flask/Makefile
> @@ -40,7 +40,7 @@ $(obj)/flask-policy.S: $(BASEDIR)/tools/binfile FORCE
>  	$(call if_changed,binfile,$(obj)/policy.bin xsm_flask_init_policy)
>  targets += flask-policy.S
>  
> -FLASK_BUILD_DIR := $(CURDIR)
> +FLASK_BUILD_DIR := $(abs_objtree)/$(obj)
>  POLICY_SRC := $(FLASK_BUILD_DIR)/xenpolicy-$(XEN_FULLVERSION)
>  
>  $(obj)/policy.bin: FORCE
> diff --git a/xen/xsm/flask/ss/Makefile b/xen/xsm/flask/ss/Makefile
> index d32b9e07138e..aba1339f3808 100644
> --- a/xen/xsm/flask/ss/Makefile
> +++ b/xen/xsm/flask/ss/Makefile
> @@ -8,4 +8,4 @@ obj-y += services.o
>  obj-y += conditional.o
>  obj-y += mls.o
>  
> -CFLAGS-y += -I../include
> +CFLAGS-y += -I$(srctree)/xsm/flask/include


Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>

V/r,
Daniel P. Smith
Apertus Solutions, LLC

Re: [XEN PATCH v9 19/30] build: clean-up "clean" rules of duplication

[Daniel P. Smith]

On 1/25/22 06:00, Anthony PERARD wrote:
> All those files to be removed are already done in the main Makefile,
> either by the "find" command or directly (for $(TARGET).efi).
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Julien Grall <jgrall@amazon.com>
> ---
>  xen/Makefile                | 2 +-
>  xen/arch/arm/Makefile       | 1 -
>  xen/arch/x86/Makefile       | 5 ++---
>  xen/test/livepatch/Makefile | 2 +-
>  xen/xsm/flask/Makefile      | 2 +-
>  5 files changed, 5 insertions(+), 7 deletions(-)
> 

<snip/>

> diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
> index 49cf730cf098..832f65274cc0 100644
> --- a/xen/xsm/flask/Makefile
> +++ b/xen/xsm/flask/Makefile
> @@ -51,4 +51,4 @@ $(obj)/policy.bin: FORCE
>  
>  .PHONY: clean
>  clean::
> -	rm -f $(ALL_H_FILES) *.o $(DEPS_RM) policy.* $(POLICY_SRC) flask-policy.S
> +	rm -f $(ALL_H_FILES) policy.* $(POLICY_SRC) flask-policy.S


Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>

V/r,
Daniel P. Smith
Apertus Solutions, LLC

Re: [XEN PATCH v9 20/30] build: rework "clean" to clean from the root dir

[Daniel P. Smith]

On 1/25/22 06:00, Anthony PERARD wrote:
> This will allow "clean" to work from an out-of-tree build when
> it will be available.
> 
> Some of the file been removed in current "clean" target aren't added
> to $(clean-files) because they are already listed in $(extra-) or
> $(extra-y).
> 
> Also start to clean files listed in $(targets). This allows to clean
> "common/config_data.S" and "xsm/flask/flask-policy.S" without
> having to list them a second time.
> 
> Also clean files in "arch/x86/boot" from that directory by allowing
> "clean" to descend into the subdir by adding "boot" into $(subdir-).
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> ---
> 
> Notes:
>     v9:
>     - reviewed
>     
>     v8:
>     - fix comment for the shorthand
>     - move this patch earlier in the series
>     - replace some BASEDIR by objtree
>     - start to clean $(targets) files.
> 
>  xen/Makefile                | 24 ++++++++++++------------
>  xen/arch/arm/Makefile       |  5 +----
>  xen/arch/x86/Makefile       | 20 ++++++++++++--------
>  xen/arch/x86/boot/Makefile  |  2 ++
>  xen/common/Makefile         |  3 +--
>  xen/include/Makefile        |  4 +---
>  xen/scripts/Kbuild.include  |  4 ++--
>  xen/scripts/Makefile.clean  | 14 +++++++++++---
>  xen/test/livepatch/Makefile |  4 +---
>  xen/xsm/flask/Makefile      |  4 +---
>  10 files changed, 44 insertions(+), 40 deletions(-)
> 

<snip/>

> diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
> index 832f65274cc0..4ac6fb8778ae 100644
> --- a/xen/xsm/flask/Makefile
> +++ b/xen/xsm/flask/Makefile
> @@ -49,6 +49,4 @@ $(obj)/policy.bin: FORCE
>  	        FLASK_BUILD_DIR=$(FLASK_BUILD_DIR) POLICY_FILENAME=$(POLICY_SRC)
>  	cmp -s $(POLICY_SRC) $@ || cp $(POLICY_SRC) $@
>  
> -.PHONY: clean
> -clean::
> -	rm -f $(ALL_H_FILES) policy.* $(POLICY_SRC) flask-policy.S
> +clean-files := policy.* $(POLICY_SRC)

Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>

V/r,
Daniel P. Smith
Apertus Solutions, LLC

Re: [XEN PATCH v9 26/30] build: replace $(BASEDIR) and use $(srctree)

[Daniel P. Smith]

On 1/25/22 06:00, Anthony PERARD wrote:
> $(srctree) is a better description for the source directory than
> $(BASEDIR) that has been used for both source and build directory
> (which where the same).
> 
> This adds $(srctree) to a few path where make's VPATH=$(srctree) won't
> apply. And replace $(BASEDIR) by $(srctree).
> 
> Introduce "$(srcdir)" as a shortcut for "$(srctree)/$(src)" as the
> later is used often enough.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> ---
> 
> Notes:
>     v9:
>     - acked
>     - reword "# shortcut for $(srctree)/$(src)" into "# shortcuts" in both
>       places.
>     
>     v8:
>     - merge of two patchs from v7:
>         build: add $(srctree) in few key places
>         build: replace $(BASEDIR) by $(srctree)
>       both patch were acked
>     - introduce $(srcdir) as a shortcut for $(srctree)/$(src)
> 
>  xen/Kconfig                 |  4 ++--
>  xen/Makefile                |  7 +++----
>  xen/Rules.mk                |  9 ++++++---
>  xen/arch/x86/arch.mk        |  2 +-
>  xen/build.mk                |  4 ++--
>  xen/common/Makefile         |  2 +-
>  xen/common/libfdt/Makefile  |  2 +-
>  xen/include/Makefile        | 14 +++++++-------
>  xen/scripts/Kconfig.include |  2 +-
>  xen/scripts/Makefile.clean  |  5 ++++-
>  xen/xsm/flask/Makefile      | 10 +++++-----
>  11 files changed, 33 insertions(+), 28 deletions(-)

<snip/>

> diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
> index 4ac6fb8778ae..a99038cb5722 100644
> --- a/xen/xsm/flask/Makefile
> +++ b/xen/xsm/flask/Makefile
> @@ -8,8 +8,8 @@ CFLAGS-y += -I$(obj)/include
>  
>  AWK = awk
>  
> -FLASK_H_DEPEND := $(addprefix $(src)/policy/,security_classes initial_sids)
> -AV_H_DEPEND = $(src)/policy/access_vectors
> +FLASK_H_DEPEND := $(addprefix $(srcdir)/policy/,security_classes initial_sids)
> +AV_H_DEPEND := $(srcdir)/policy/access_vectors
>  
>  FLASK_H_FILES := flask.h class_to_string.h initial_sid_to_string.h
>  AV_H_FILES := av_perm_to_string.h av_permissions.h
> @@ -18,14 +18,14 @@ ALL_H_FILES := $(addprefix include/,$(FLASK_H_FILES) $(AV_H_FILES))
>  $(addprefix $(obj)/,$(obj-y)) $(obj)/ss/built_in.o: $(addprefix $(obj)/,$(ALL_H_FILES))
>  extra-y += $(ALL_H_FILES)
>  
> -mkflask := $(src)/policy/mkflask.sh
> +mkflask := $(srcdir)/policy/mkflask.sh
>  quiet_cmd_mkflask = MKFLASK $@
>  cmd_mkflask = $(SHELL) $(mkflask) $(AWK) $(obj)/include $(FLASK_H_DEPEND)
>  
>  $(addprefix $(obj)/%/,$(FLASK_H_FILES)): $(FLASK_H_DEPEND) $(mkflask) FORCE
>  	$(call if_changed,mkflask)
>  
> -mkaccess := $(src)/policy/mkaccess_vector.sh
> +mkaccess := $(srcdir)/policy/mkaccess_vector.sh
>  quiet_cmd_mkaccess = MKACCESS VECTOR $@
>  cmd_mkaccess = $(SHELL) $(mkaccess) $(AWK) $(obj)/include $(AV_H_DEPEND)
>  
> @@ -36,7 +36,7 @@ obj-bin-$(CONFIG_XSM_FLASK_POLICY) += flask-policy.o
>  $(obj)/flask-policy.o: $(obj)/policy.bin
>  
>  $(obj)/flask-policy.S: BINFILE_FLAGS := -i
> -$(obj)/flask-policy.S: $(BASEDIR)/tools/binfile FORCE
> +$(obj)/flask-policy.S: $(srctree)/tools/binfile FORCE
>  	$(call if_changed,binfile,$(obj)/policy.bin xsm_flask_init_policy)
>  targets += flask-policy.S
>  

Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>

V/r,
Daniel P. Smith
Apertus Solutions, LLC

Re: [XEN PATCH v9 30/30] build: adding out-of-tree support to the xen build

[Daniel P. Smith]

On 1/25/22 06:01, Anthony PERARD wrote:
> This implement out-of-tree support, there's two ways to create an
> out-of-tree build tree (after that, `make` in that new directory
> works):
>     make O=build
>     mkdir build; cd build; make -f ../Makefile
> also works with an absolute path for both.
> 
> This implementation only works if the source tree is clean, as we use
> VPATH.
> 
> This patch copies most new code with handling out-of-tree build from
> Linux v5.12.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> ---
> 
> Notes:
>     v9:
>     - acked
>     - add "source -> ." symlink in the in-tree build as well. This allow to
>       make relative symlink to the "common/efi/*.c" sources.
>     - also now, the "source" symlink is removed on `distclean`, add added to
>       .gitignore.
>     
>     v8:
>     - replace script/mkmakefile by cmd_makefile like it's been done in Linux
>       5.13.
>     - fix GNU Make version, replace few 0.81 by 3.81.
>     - reorder include header path in CFLAGS ( -I ), to have the $(objtree)
>       paths listed before the $(srctree) paths. This will be usefull later
>       if we allow to build out-of-tree from a dirty source tree.
>     - make use of -iquote
>     - sometime, add -iquote path even when not necessary when building in-tree.
>     - extract code movement into a separate patch
>     - make use of the new "$(srcdir)" shortcut
>     - split command line in headers*.chk target into more lines
> 
>  .gitignore                   |   1 +
>  xen/Makefile                 | 143 +++++++++++++++++++++++++++++++----
>  xen/Rules.mk                 |  11 ++-
>  xen/arch/x86/arch.mk         |   5 +-
>  xen/arch/x86/boot/Makefile   |   6 ++
>  xen/common/efi/efi-common.mk |   3 +-
>  xen/include/Makefile         |  11 ++-
>  xen/test/livepatch/Makefile  |   2 +
>  xen/xsm/flask/Makefile       |   3 +-
>  xen/xsm/flask/ss/Makefile    |   1 +
>  10 files changed, 164 insertions(+), 22 deletions(-)
> 

<snip/>

> diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
> index a99038cb5722..d25312f4fa1c 100644
> --- a/xen/xsm/flask/Makefile
> +++ b/xen/xsm/flask/Makefile
> @@ -4,7 +4,8 @@ obj-y += flask_op.o
>  
>  obj-y += ss/
>  
> -CFLAGS-y += -I$(obj)/include
> +CFLAGS-y += -iquote $(obj)/include
> +CFLAGS-y += -I$(srcdir)/include
>  
>  AWK = awk
>  
> diff --git a/xen/xsm/flask/ss/Makefile b/xen/xsm/flask/ss/Makefile
> index aba1339f3808..ffe92ec19ed6 100644
> --- a/xen/xsm/flask/ss/Makefile
> +++ b/xen/xsm/flask/ss/Makefile
> @@ -8,4 +8,5 @@ obj-y += services.o
>  obj-y += conditional.o
>  obj-y += mls.o
>  
> +CFLAGS-y += -iquote $(objtree)/xsm/flask/include
>  CFLAGS-y += -I$(srctree)/xsm/flask/include

Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>

V/r,
Daniel P. Smith
Apertus Solutions, LLC

Re: [XEN PATCH v9 01/30] build: set XEN_BUILD_EFI earlier

[Jan Beulich]

On 25.01.2022 12:00, Anthony PERARD wrote:
> We are going to need the variable XEN_BUILD_EFI earlier.
> 
> But a side effect of calculating the value of $(XEN_BUILD_EFI) is to
> also to generate "efi/check.o" which is used for further checks.
> Thus the whole chain that check for EFI support is moved to
> "arch.mk".
> 
> Some other changes are made to avoid too much duplication:
>     - $(efi-check): Used to avoid repeating "efi/check.*". We don't
>       set it to the path to the source as it would be wrong as soon
>       as we support out-of-tree build.
>     - $(LD_PE_check_cmd): As it is called twice, with an updated
>       $(EFI_LDFLAGS).
> 
> $(nr-fixups) is renamed to $(efi-nr-fixups) as the former might be
> a bit too generic.
> 
> In order to avoid exporting MKRELOC, the variable is added to $(MAKE)
> command line. The only modification needed is in target "build", the
> modification target "$(TARGET)" will be needed with a following patch
> "build: avoid re-executing the main Makefile by introducing build.mk".
> 
> We can now revert 24b0ce9a5da2, we don't need to override efi-y on
> recursion anymore.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

Re: [XEN PATCH v9 03/30] build: fix exported variable name CFLAGS_stack_boundary

[Jan Beulich]

On 25.01.2022 12:00, Anthony PERARD wrote:
> Exporting a variable with a dash doesn't work reliably, they may be
> striped from the environment when calling a sub-make or sub-shell.
> 
> CFLAGS-stack-boundary start to be removed from env in patch "build:
> set ALL_OBJS in main Makefile; move prelink.o to main Makefile" when
> running `make "ALL_OBJS=.."` due to the addition of the quote. At
> least in my empirical tests.
> 
> Fixes: 2740d96efd ("xen/build: have the root Makefile generates the CFLAGS")
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

Re: [XEN PATCH v9 04/30] build: set ALL_OBJS in main Makefile; move prelink.o to main Makefile

[Jan Beulich]

On 25.01.2022 12:00, Anthony PERARD wrote:
> This is to avoid arch/$arch/Makefile having to recurse into parents
> directories.
> 
> This avoid duplication of the logic to build prelink.o between arches.
> 
> In order to do that, we cut the $(TARGET) target in the main Makefile in
> two, there is a "prepare" phase/target runned before starting to build
> "prelink.o" which will prepare "include/" among other things, then all
> the $(ALL_OBJS) will be generated in order to build "prelink.o" and
> finally $(TARGET) will be generated by calling into "arch/*/" to make
> $(TARGET).
> 
> Now we don't need to prefix $(ALL_OBJS) with $(BASEDIR) as it is now
> only used from the main Makefile. Other changes is to use "$<" instead
> of spelling "prelink.o" in the target "$(TARGET)" in both
> arch/*/Makefile.
> 
> Beside "prelink.o" been at a different location, no other functional
> change intended.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with a remark and maybe a small adjustment suggestion:

> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -285,6 +285,16 @@ CFLAGS += -flto
>  LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
>  endif
>  
> +# Note that link order matters!

Merely as a remark: I wonder how applicable that comment is anymore.
If anything I'd expect it to be relevant to $(TARGET_SUBARCH)/head.o
(Arm) and boot/built_in.o (x86), neither of which get named here.

> @@ -407,7 +417,7 @@ $(TARGET): FORCE
>  	$(MAKE) -f $(BASEDIR)/Rules.mk -C include
>  	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) include
>  	$(MAKE) -f $(BASEDIR)/Rules.mk arch/$(TARGET_ARCH)/include/asm/asm-offsets.h
> -	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) MKRELOC=$(MKRELOC) $@
> +	$(MAKE) -f $(BASEDIR)/Rules.mk MKRELOC=$(MKRELOC) "ALL_OBJS=$(ALL_OBJS-y)" "ALL_LIBS=$(ALL_LIBS-y)" $@

I'm always a little wary of using double quotes when it's not clear
what exactly a macro may expand to. Single quotes at least have less
restrictions ...

Jan

Re: [XEN PATCH v9 08/30] build: fix enforce unique symbols for recent clang version

[Jan Beulich]

On 25.01.2022 12:00, Anthony PERARD wrote:
> clang 6.0 and newer behave like gcc in regards for the FILE symbol, so
> only the filename rather than the full path to the source file.
> 
> clang 3.8.1-24 (in our debian:stretch container) and 3.5.0-10
> (in our debian:jessie container) do store the full path to the source
> file in the FILE symbol.
> 
> Also, based on commit 81ecb38b83 ("build: provide option to
> disambiguate symbol names"), which were using clang 5, the change of
> behavior likely happened in clang 6.0.
> 
> This means that we also need to check clang version to figure out
> which command we need to use to redefine symbol.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

The "likely" in the description still worries me some. Roger, would
you happen to know, or know of a way to find out for sure ("sure"
not meaning to exclude the usual risk associated with version
number checks)?

Jan

Re: [XEN PATCH v9 03/30] build: fix exported variable name CFLAGS_stack_boundary

[Jan Beulich]

On 25.01.2022 12:00, Anthony PERARD wrote:
> Exporting a variable with a dash doesn't work reliably, they may be
> striped from the environment when calling a sub-make or sub-shell.
> 
> CFLAGS-stack-boundary start to be removed from env in patch "build:
> set ALL_OBJS in main Makefile; move prelink.o to main Makefile" when
> running `make "ALL_OBJS=.."` due to the addition of the quote. At
> least in my empirical tests.
> 
> Fixes: 2740d96efd ("xen/build: have the root Makefile generates the CFLAGS")
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

While I did commit this, I'm still somewhat confused. How would quoting
of elements on a make command line make a difference to which variables
get exported?

In any event I understand the description that prior to the subsequent
change there's not actually any issue. Hence I'm not going to queue
this for backporting despite the Fixes: tag. Unless of course I'm told
otherwise (with justification).

Jan

Re: [XEN PATCH v9 04/30] build: set ALL_OBJS in main Makefile; move prelink.o to main Makefile

[Anthony PERARD]

On Thu, Jan 27, 2022 at 04:50:32PM +0100, Jan Beulich wrote:
> On 25.01.2022 12:00, Anthony PERARD wrote:
> > --- a/xen/Makefile
> > +++ b/xen/Makefile
> > @@ -285,6 +285,16 @@ CFLAGS += -flto
> >  LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
> >  endif
> >  
> > +# Note that link order matters!
> 
> Merely as a remark: I wonder how applicable that comment is anymore.
> If anything I'd expect it to be relevant to $(TARGET_SUBARCH)/head.o
> (Arm) and boot/built_in.o (x86), neither of which get named here.

Indeed, the order here probably doesn't matter. I tried to build on x86
with the list reversed (so still leaving boot/ first) and the build
works. I didn't try to boot it.

Maybe it's time to retire the comment?

> > @@ -407,7 +417,7 @@ $(TARGET): FORCE
> >  	$(MAKE) -f $(BASEDIR)/Rules.mk -C include
> >  	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) include
> >  	$(MAKE) -f $(BASEDIR)/Rules.mk arch/$(TARGET_ARCH)/include/asm/asm-offsets.h
> > -	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) MKRELOC=$(MKRELOC) $@
> > +	$(MAKE) -f $(BASEDIR)/Rules.mk MKRELOC=$(MKRELOC) "ALL_OBJS=$(ALL_OBJS-y)" "ALL_LIBS=$(ALL_LIBS-y)" $@
> 
> I'm always a little wary of using double quotes when it's not clear
> what exactly a macro may expand to. Single quotes at least have less
> restrictions ...

Using single quotes sounds good.

Thanks,

-- 
Anthony PERARD

Re: [XEN PATCH v9 04/30] build: set ALL_OBJS in main Makefile; move prelink.o to main Makefile

[Jan Beulich]

On 28.01.2022 12:32, Anthony PERARD wrote:
> On Thu, Jan 27, 2022 at 04:50:32PM +0100, Jan Beulich wrote:
>> On 25.01.2022 12:00, Anthony PERARD wrote:
>>> --- a/xen/Makefile
>>> +++ b/xen/Makefile
>>> @@ -285,6 +285,16 @@ CFLAGS += -flto
>>>  LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
>>>  endif
>>>  
>>> +# Note that link order matters!
>>
>> Merely as a remark: I wonder how applicable that comment is anymore.
>> If anything I'd expect it to be relevant to $(TARGET_SUBARCH)/head.o
>> (Arm) and boot/built_in.o (x86), neither of which get named here.
> 
> Indeed, the order here probably doesn't matter. I tried to build on x86
> with the list reversed (so still leaving boot/ first) and the build
> works. I didn't try to boot it.

It's quite unlikely for the order to matter at build time. Being able
to boot the result is the minimum. Even then you can't be sure you
merely avoided the problematic piece of code on the particular
hardware you did the test on. Perhaps the most fragile parts are
sections holding pointers which get processed in the order the linker
put them. E.g. unexpected interdependencies between initcalls.

> Maybe it's time to retire the comment?

Probably, but Arm folks would want to confirm that's fine on their side
as well.

Jan

Re: [XEN PATCH v9 08/30] build: fix enforce unique symbols for recent clang version

[Anthony PERARD]

On Thu, Jan 27, 2022 at 04:57:20PM +0100, Jan Beulich wrote:
> On 25.01.2022 12:00, Anthony PERARD wrote:
> > clang 6.0 and newer behave like gcc in regards for the FILE symbol, so
> > only the filename rather than the full path to the source file.
> > 
> > clang 3.8.1-24 (in our debian:stretch container) and 3.5.0-10
> > (in our debian:jessie container) do store the full path to the source
> > file in the FILE symbol.
> > 
> > Also, based on commit 81ecb38b83 ("build: provide option to
> > disambiguate symbol names"), which were using clang 5, the change of
> > behavior likely happened in clang 6.0.
> > 
> > This means that we also need to check clang version to figure out
> > which command we need to use to redefine symbol.
> > 
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> The "likely" in the description still worries me some. Roger, would
> you happen to know, or know of a way to find out for sure ("sure"
> not meaning to exclude the usual risk associated with version
> number checks)?

I found f5040b9685a7 ("Make .file directive to have basename only") as
part of LLVM's "release/6.x" branch (and "llvmorg-6.0.0" tag), but not
in "release/5.x".

https://github.com/llvm/llvm-project/commit/f5040b9685a760e584c576e9185295e54635d51e

This patch would seems to be the one changing the behavior. This still
suggest clang 6.0.

Cheers,

-- 
Anthony PERARD

Re: [XEN PATCH v9 08/30] build: fix enforce unique symbols for recent clang version

[Jan Beulich]

On 28.01.2022 13:03, Anthony PERARD wrote:
> On Thu, Jan 27, 2022 at 04:57:20PM +0100, Jan Beulich wrote:
>> On 25.01.2022 12:00, Anthony PERARD wrote:
>>> clang 6.0 and newer behave like gcc in regards for the FILE symbol, so
>>> only the filename rather than the full path to the source file.
>>>
>>> clang 3.8.1-24 (in our debian:stretch container) and 3.5.0-10
>>> (in our debian:jessie container) do store the full path to the source
>>> file in the FILE symbol.
>>>
>>> Also, based on commit 81ecb38b83 ("build: provide option to
>>> disambiguate symbol names"), which were using clang 5, the change of
>>> behavior likely happened in clang 6.0.
>>>
>>> This means that we also need to check clang version to figure out
>>> which command we need to use to redefine symbol.
>>>
>>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
>>
>> The "likely" in the description still worries me some. Roger, would
>> you happen to know, or know of a way to find out for sure ("sure"
>> not meaning to exclude the usual risk associated with version
>> number checks)?
> 
> I found f5040b9685a7 ("Make .file directive to have basename only") as
> part of LLVM's "release/6.x" branch (and "llvmorg-6.0.0" tag), but not
> in "release/5.x".
> 
> https://github.com/llvm/llvm-project/commit/f5040b9685a760e584c576e9185295e54635d51e
> 
> This patch would seems to be the one changing the behavior. This still
> suggest clang 6.0.

Oh, thanks for digging this out. May I suggest to replace (or delete)
"likely" then in the description?

Acked-by: Jan Beulich <jbeulich@suse.com>

Jan

Re: [XEN PATCH v9 03/30] build: fix exported variable name CFLAGS_stack_boundary

[Anthony PERARD]

On Fri, Jan 28, 2022 at 12:14:58PM +0100, Jan Beulich wrote:
> On 25.01.2022 12:00, Anthony PERARD wrote:
> > Exporting a variable with a dash doesn't work reliably, they may be
> > striped from the environment when calling a sub-make or sub-shell.
> > 
> > CFLAGS-stack-boundary start to be removed from env in patch "build:
> > set ALL_OBJS in main Makefile; move prelink.o to main Makefile" when
> > running `make "ALL_OBJS=.."` due to the addition of the quote. At
> > least in my empirical tests.
> > 
> > Fixes: 2740d96efd ("xen/build: have the root Makefile generates the CFLAGS")
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> While I did commit this, I'm still somewhat confused. How would quoting
> of elements on a make command line make a difference to which variables
> get exported?

I don't know, maybe without quote, make run sub-make directly, but with
double-quote, a shell is used.

But after reading the manual, I didn't expect the variable to be passed
down sub-make at all:

    5.7.2 Communicating Variables to a Sub-make

    Except by explicit request, make exports a variable only if it is
    either defined in the environment initially or set on the command
    line, and if its name consists only of letters, numbers, and
    underscores.

But somehow, sub-makes also export the non-shell-exportable variables,
unless the command line in a recipe invoking make has double-quotes.


I've tried again, and checked the process list:
  - when running `$(MAKE) -C foo`; make just execute make
  - when running `$(MAKE) -C 'foo'`; make just execute make
  - when running `$(MAKE) -C "foo"`; make execute /bin/sh -c ...
  - when running `$(MAKE) -C foo | tee`; make execute /bin/sh -c ...

So, CFLAGS-stack-boundary disappear when /bin/sh is involved.

> In any event I understand the description that prior to the subsequent
> change there's not actually any issue. Hence I'm not going to queue
> this for backporting despite the Fixes: tag. Unless of course I'm told
> otherwise (with justification).

Justification would be that it's not supposed to work, based on
information from make's manual.

Thanks,

-- 
Anthony PERARD

Re: [XEN PATCH v9 08/30] build: fix enforce unique symbols for recent clang version

[Anthony PERARD]

On Fri, Jan 28, 2022 at 01:43:38PM +0100, Jan Beulich wrote:
> On 28.01.2022 13:03, Anthony PERARD wrote:
> > On Thu, Jan 27, 2022 at 04:57:20PM +0100, Jan Beulich wrote:
> >> On 25.01.2022 12:00, Anthony PERARD wrote:
> >>> clang 6.0 and newer behave like gcc in regards for the FILE symbol, so
> >>> only the filename rather than the full path to the source file.
> >>>
> >>> clang 3.8.1-24 (in our debian:stretch container) and 3.5.0-10
> >>> (in our debian:jessie container) do store the full path to the source
> >>> file in the FILE symbol.
> >>>
> >>> Also, based on commit 81ecb38b83 ("build: provide option to
> >>> disambiguate symbol names"), which were using clang 5, the change of
> >>> behavior likely happened in clang 6.0.
> >>>
> >>> This means that we also need to check clang version to figure out
> >>> which command we need to use to redefine symbol.
> >>>
> >>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> >>
> >> The "likely" in the description still worries me some. Roger, would
> >> you happen to know, or know of a way to find out for sure ("sure"
> >> not meaning to exclude the usual risk associated with version
> >> number checks)?
> > 
> > I found f5040b9685a7 ("Make .file directive to have basename only") as
> > part of LLVM's "release/6.x" branch (and "llvmorg-6.0.0" tag), but not
> > in "release/5.x".
> > 
> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fllvm%2Fllvm-project%2Fcommit%2Ff5040b9685a760e584c576e9185295e54635d51e&amp;data=04%7C01%7Canthony.perard%40citrix.com%7C1ce7898a15bb4024260008d9e25be6f9%7C335836de42ef43a2b145348c2ee9ca5b%7C0%7C0%7C637789706644173026%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=V73NmkJWAHpqlzY9sAysf6%2Fw7q8ik6twT6lMLgglR3s%3D&amp;reserved=0
> > 
> > This patch would seems to be the one changing the behavior. This still
> > suggest clang 6.0.
> 
> Oh, thanks for digging this out. May I suggest to replace (or delete)
> "likely" then in the description?

Maybe something like that? Or just delete the word might be enough.

    Also we have commit 81ecb38b83 ("build: provide option to
    disambiguate symbol names") which were using clang 5, and LLVM's
    commit f5040b9685a7 [1] ("Make .file directive to have basename
    only") which is part of "llvmorg-6.0.0" tag but not "release/5.x"
    branch. Both suggest that clang change of behavior happened with
    clang 6.0.

    [1] https://github.com/llvm/llvm-project/commit/f5040b9685a760e584c576e9185295e54635d51e

> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks,

-- 
Anthony PERARD

Re: [XEN PATCH v9 03/30] build: fix exported variable name CFLAGS_stack_boundary

[Jan Beulich]

On 28.01.2022 16:04, Anthony PERARD wrote:
> On Fri, Jan 28, 2022 at 12:14:58PM +0100, Jan Beulich wrote:
>> On 25.01.2022 12:00, Anthony PERARD wrote:
>>> Exporting a variable with a dash doesn't work reliably, they may be
>>> striped from the environment when calling a sub-make or sub-shell.
>>>
>>> CFLAGS-stack-boundary start to be removed from env in patch "build:
>>> set ALL_OBJS in main Makefile; move prelink.o to main Makefile" when
>>> running `make "ALL_OBJS=.."` due to the addition of the quote. At
>>> least in my empirical tests.
>>>
>>> Fixes: 2740d96efd ("xen/build: have the root Makefile generates the CFLAGS")
>>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
>>
>> While I did commit this, I'm still somewhat confused. How would quoting
>> of elements on a make command line make a difference to which variables
>> get exported?
> 
> I don't know, maybe without quote, make run sub-make directly, but with
> double-quote, a shell is used.
> 
> But after reading the manual, I didn't expect the variable to be passed
> down sub-make at all:
> 
>     5.7.2 Communicating Variables to a Sub-make
> 
>     Except by explicit request, make exports a variable only if it is
>     either defined in the environment initially or set on the command
>     line, and if its name consists only of letters, numbers, and
>     underscores.
> 
> But somehow, sub-makes also export the non-shell-exportable variables,
> unless the command line in a recipe invoking make has double-quotes.
> 
> 
> I've tried again, and checked the process list:
>   - when running `$(MAKE) -C foo`; make just execute make
>   - when running `$(MAKE) -C 'foo'`; make just execute make
>   - when running `$(MAKE) -C "foo"`; make execute /bin/sh -c ...
>   - when running `$(MAKE) -C foo | tee`; make execute /bin/sh -c ...
> 
> So, CFLAGS-stack-boundary disappear when /bin/sh is involved.

Very "interesting" behavior.

>> In any event I understand the description that prior to the subsequent
>> change there's not actually any issue. Hence I'm not going to queue
>> this for backporting despite the Fixes: tag. Unless of course I'm told
>> otherwise (with justification).
> 
> Justification would be that it's not supposed to work, based on
> information from make's manual.

Okay, I'll queue it then, but in case it doesn't backport
straightforwardly I'll still consider leaving it out.

Jan

Re: [XEN PATCH v9 08/30] build: fix enforce unique symbols for recent clang version

[Jan Beulich]

On 28.01.2022 16:52, Anthony PERARD wrote:
> On Fri, Jan 28, 2022 at 01:43:38PM +0100, Jan Beulich wrote:
>> On 28.01.2022 13:03, Anthony PERARD wrote:
>>> On Thu, Jan 27, 2022 at 04:57:20PM +0100, Jan Beulich wrote:
>>>> On 25.01.2022 12:00, Anthony PERARD wrote:
>>>>> clang 6.0 and newer behave like gcc in regards for the FILE symbol, so
>>>>> only the filename rather than the full path to the source file.
>>>>>
>>>>> clang 3.8.1-24 (in our debian:stretch container) and 3.5.0-10
>>>>> (in our debian:jessie container) do store the full path to the source
>>>>> file in the FILE symbol.
>>>>>
>>>>> Also, based on commit 81ecb38b83 ("build: provide option to
>>>>> disambiguate symbol names"), which were using clang 5, the change of
>>>>> behavior likely happened in clang 6.0.
>>>>>
>>>>> This means that we also need to check clang version to figure out
>>>>> which command we need to use to redefine symbol.
>>>>>
>>>>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
>>>>
>>>> The "likely" in the description still worries me some. Roger, would
>>>> you happen to know, or know of a way to find out for sure ("sure"
>>>> not meaning to exclude the usual risk associated with version
>>>> number checks)?
>>>
>>> I found f5040b9685a7 ("Make .file directive to have basename only") as
>>> part of LLVM's "release/6.x" branch (and "llvmorg-6.0.0" tag), but not
>>> in "release/5.x".
>>>
>>> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fllvm%2Fllvm-project%2Fcommit%2Ff5040b9685a760e584c576e9185295e54635d51e&amp;data=04%7C01%7Canthony.perard%40citrix.com%7C1ce7898a15bb4024260008d9e25be6f9%7C335836de42ef43a2b145348c2ee9ca5b%7C0%7C0%7C637789706644173026%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=V73NmkJWAHpqlzY9sAysf6%2Fw7q8ik6twT6lMLgglR3s%3D&amp;reserved=0
>>>
>>> This patch would seems to be the one changing the behavior. This still
>>> suggest clang 6.0.
>>
>> Oh, thanks for digging this out. May I suggest to replace (or delete)
>> "likely" then in the description?
> 
> Maybe something like that? Or just delete the word might be enough.
> 
>     Also we have commit 81ecb38b83 ("build: provide option to
>     disambiguate symbol names") which were using clang 5, and LLVM's
>     commit f5040b9685a7 [1] ("Make .file directive to have basename
>     only") which is part of "llvmorg-6.0.0" tag but not "release/5.x"
>     branch. Both suggest that clang change of behavior happened with
>     clang 6.0.
> 
>     [1] https://github.com/llvm/llvm-project/commit/f5040b9685a760e584c576e9185295e54635d51e

This sounds better to me than just dropping the one word.

Jan

Re: [XEN PATCH v9 24/30] build: grab common EFI source files in arch specific dir

[Julien Grall]

Hi,

On 25/01/2022 11:00, Anthony PERARD wrote:
> Rather than preparing the efi source file, we will make the symbolic
> link as needed from the build location.

It is not a request for this series. I have started to look a bit more 
how the EFI code work on Arm and I was wondering why we are using 
symbolic here?

IOW, why wouldn't it be fine to build the "common" and "arch" specific 
bit (if any) separately and then assemble them?

Cheers,


-- 
Julien Grall

Ping: [XEN PATCH v9 06/30] build: rework test/livepatch/Makefile

[Anthony PERARD]

Hi Ross, Konrad,

Could you ack or review this patch?

There is also a few other patches in this series that make some changes
to "xen/test/livepatch/Makefile", an ack for those would be nice as
well:

- 09/30 build: build everything from the root dir, use obj=$subdir
- 19/30 build: clean-up "clean" rules of duplication
- 20/30 build: rework "clean" to clean from the root dir
- 25/30 build: replace $(BASEDIR) by $(objtree)
- 30/30 build: adding out-of-tree support to the xen build

Cheers.

On Tue, Jan 25, 2022 at 11:00:39AM +0000, Anthony PERARD wrote:
> This rework the livepatch/Makefile to make it less repetitive and make
> use of the facilities. All the targets to be built are now listed in
> $(extra-y) which will allow Rules.mk to build them without the need of
> a local target in a future patch.
> 
> There are some changes/fixes in this patch:
> - when "xen-syms" is used for a target, it is added to the dependency
>   list of the target, which allow to rebuild the target when xen-syms
>   changes. But if "xen-syms" is missing, make simply fails.
> - modinfo.o wasn't removing it's $@.bin file like the other targets,
>   this is now done.
> - The command to build *.livepatch targets as been fixed to use
>   $(XEN_LDFLAGS) rather than just $(LDFLAGS) which is a fallout from
>   2740d96efdd3 ("xen/build: have the root Makefile generates the
>   CFLAGS")
> 
> make will findout the dependencies of the *.livepatch files and thus
> what to built by "looking" at the objects listed in the *-objs
> variables. The actual dependencies is generated by the new
> "multi-depend" macro.
> 
> "$(targets)" needs to be updated with the objects listed in the
> different *-objs variables to allow make to load the .*.cmd dependency
> files.
> 
> This patch copies the macro "multi_depend" from Linux 5.12, and rename
> it to "multi-depend".
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> ---
> 
> Notes:
>     v9:
>     - half acked, still need "livepatch" maintainer ack.
>     
>     v8:
>     - rename multi_depend to multi-depend
>     - use $() for single-letter make variable
>     - re-indent one line
> 
>  xen/scripts/Kbuild.include  |   9 ++
>  xen/test/livepatch/Makefile | 213 ++++++++++++++----------------------
>  2 files changed, 91 insertions(+), 131 deletions(-)

-- 
Anthony PERARD

Ping: [XEN PATCH v9 04/30] build: set ALL_OBJS in main Makefile; move prelink.o to main Makefile

[Anthony PERARD]

Hi Julien, Stefano,

This patch has some changes to the Arm makefiles. Could you have a quick
look or ack the changes?

They are some other patches in the series which might want a Arm ack:
- 20/30 build: rework "clean" to clean from the root dir
- 24/30 build: grab common EFI source files in arch specific dir
- 25/30 build: replace $(BASEDIR) by $(objtree)

Cheers.

On Tue, Jan 25, 2022 at 11:00:37AM +0000, Anthony PERARD wrote:
> This is to avoid arch/$arch/Makefile having to recurse into parents
> directories.
> 
> This avoid duplication of the logic to build prelink.o between arches.
> 
> In order to do that, we cut the $(TARGET) target in the main Makefile in
> two, there is a "prepare" phase/target runned before starting to build
> "prelink.o" which will prepare "include/" among other things, then all
> the $(ALL_OBJS) will be generated in order to build "prelink.o" and
> finally $(TARGET) will be generated by calling into "arch/*/" to make
> $(TARGET).
> 
> Now we don't need to prefix $(ALL_OBJS) with $(BASEDIR) as it is now
> only used from the main Makefile. Other changes is to use "$<" instead
> of spelling "prelink.o" in the target "$(TARGET)" in both
> arch/*/Makefile.
> 
> Beside "prelink.o" been at a different location, no other functional
> change intended.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
> 
>  xen/Makefile          | 12 +++++++++++-
>  xen/Rules.mk          | 13 -------------
>  xen/arch/arm/Makefile | 31 ++++---------------------------
>  xen/arch/arm/Rules.mk |  4 ++++
>  xen/arch/arm/arch.mk  |  2 ++
>  xen/arch/x86/Makefile | 29 ++++++-----------------------
>  xen/arch/x86/arch.mk  |  2 ++
>  xen/build.mk          | 18 ++++++++++++++++++
>  8 files changed, 47 insertions(+), 64 deletions(-)

-- 
Anthony PERARD

Re: [XEN PATCH v9 04/30] build: set ALL_OBJS in main Makefile; move prelink.o to main Makefile

[Julien Grall]

Hi,

On 28/01/2022 11:41, Jan Beulich wrote:
> On 28.01.2022 12:32, Anthony PERARD wrote:
>> On Thu, Jan 27, 2022 at 04:50:32PM +0100, Jan Beulich wrote:
>>> On 25.01.2022 12:00, Anthony PERARD wrote:
>>>> --- a/xen/Makefile
>>>> +++ b/xen/Makefile
>>>> @@ -285,6 +285,16 @@ CFLAGS += -flto
>>>>   LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
>>>>   endif
>>>>   
>>>> +# Note that link order matters!
>>>
>>> Merely as a remark: I wonder how applicable that comment is anymore.
>>> If anything I'd expect it to be relevant to $(TARGET_SUBARCH)/head.o
>>> (Arm) and boot/built_in.o (x86), neither of which get named here.
>>
>> Indeed, the order here probably doesn't matter. I tried to build on x86
>> with the list reversed (so still leaving boot/ first) and the build
>> works. I didn't try to boot it.
> 
> It's quite unlikely for the order to matter at build time. Being able
> to boot the result is the minimum. Even then you can't be sure you
> merely avoided the problematic piece of code on the particular
> hardware you did the test on. Perhaps the most fragile parts are
> sections holding pointers which get processed in the order the linker
> put them. E.g. unexpected interdependencies between initcalls.
> 
>> Maybe it's time to retire the comment?
> 
> Probably, but Arm folks would want to confirm that's fine on their side
> as well.

Sorry I missed that comment. On Arm, the ordering is enforced in 
xen/arm/Makefile:

ALL_OBJS := $(TARGET_SUBARCH)/head.o $(ALL_OBJS)

AFAIK, the ordering in xen/Makefile doesn't matter to us. So I think the 
comment in xen/Makefile can be droppped.

That said, I always found this approach too magic. I have a TODO (low 
priority though) to enforce the ordering in the link script instead.

Anyway, I will build Xen with your branch and test it to confirm there 
are no changes.

Cheers,

-- 
Julien Grall

Re: [XEN PATCH v9 04/30] build: set ALL_OBJS in main Makefile; move prelink.o to main Makefile

[Julien Grall]

Hi,

On 17/02/2022 16:29, Julien Grall wrote:
> On 28/01/2022 11:41, Jan Beulich wrote:
>> On 28.01.2022 12:32, Anthony PERARD wrote:
>>> On Thu, Jan 27, 2022 at 04:50:32PM +0100, Jan Beulich wrote:
>>>> On 25.01.2022 12:00, Anthony PERARD wrote:
>>>>> --- a/xen/Makefile
>>>>> +++ b/xen/Makefile
>>>>> @@ -285,6 +285,16 @@ CFLAGS += -flto
>>>>>   LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
>>>>>   endif
>>>>> +# Note that link order matters!
>>>>
>>>> Merely as a remark: I wonder how applicable that comment is anymore.
>>>> If anything I'd expect it to be relevant to $(TARGET_SUBARCH)/head.o
>>>> (Arm) and boot/built_in.o (x86), neither of which get named here.
>>>
>>> Indeed, the order here probably doesn't matter. I tried to build on x86
>>> with the list reversed (so still leaving boot/ first) and the build
>>> works. I didn't try to boot it.
>>
>> It's quite unlikely for the order to matter at build time. Being able
>> to boot the result is the minimum. Even then you can't be sure you
>> merely avoided the problematic piece of code on the particular
>> hardware you did the test on. Perhaps the most fragile parts are
>> sections holding pointers which get processed in the order the linker
>> put them. E.g. unexpected interdependencies between initcalls.
>>
>>> Maybe it's time to retire the comment?
>>
>> Probably, but Arm folks would want to confirm that's fine on their side
>> as well.
> 
> Sorry I missed that comment. On Arm, the ordering is enforced in 
> xen/arm/Makefile:
> 
> ALL_OBJS := $(TARGET_SUBARCH)/head.o $(ALL_OBJS)
> 
> AFAIK, the ordering in xen/Makefile doesn't matter to us. So I think the 
> comment in xen/Makefile can be droppped.
> 
> That said, I always found this approach too magic. I have a TODO (low 
> priority though) to enforce the ordering in the link script instead.
> 
> Anyway, I will build Xen with your branch and test it to confirm there 
> are no changes.

Tested on Arm64. I confirm this can still boot :).

Cheers,

-- 
Julien Grall

Re: [XEN PATCH v9 04/30] build: set ALL_OBJS in main Makefile; move prelink.o to main Makefile

[Julien Grall]

Hi Anthony,

On 25/01/2022 11:00, Anthony PERARD wrote:
> This is to avoid arch/$arch/Makefile having to recurse into parents
> directories.
> 
> This avoid duplication of the logic to build prelink.o between arches.
> 
> In order to do that, we cut the $(TARGET) target in the main Makefile in
> two, there is a "prepare" phase/target runned before starting to build
> "prelink.o" which will prepare "include/" among other things, then all
> the $(ALL_OBJS) will be generated in order to build "prelink.o" and
> finally $(TARGET) will be generated by calling into "arch/*/" to make
> $(TARGET).
> 
> Now we don't need to prefix $(ALL_OBJS) with $(BASEDIR) as it is now
> only used from the main Makefile. Other changes is to use "$<" instead
> of spelling "prelink.o" in the target "$(TARGET)" in both
> arch/*/Makefile.
> 
> Beside "prelink.o" been at a different location, no other functional
> change intended.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

> ---
> 
> Notes:
>      v9:
>      - set ALL_OBJS and ALL_LIBS on make command line instead of exporting
>        them.
>      - arm/Rules.mk, have one rule instead of two for head.o
>      - fix typo in title
>      
>      v8:
>      - rebased
>      - move Arm specific dependencies between $(ALL_OBJS) objects (head.o)
>        into Arm specific "Rules.mk" instead of the common "build.mk".
>      
>      v7:
>      - change, now things are in build.mk: no more prepare phase needed
> 
>   xen/Makefile          | 12 +++++++++++-
>   xen/Rules.mk          | 13 -------------
>   xen/arch/arm/Makefile | 31 ++++---------------------------
>   xen/arch/arm/Rules.mk |  4 ++++
>   xen/arch/arm/arch.mk  |  2 ++
>   xen/arch/x86/Makefile | 29 ++++++-----------------------
>   xen/arch/x86/arch.mk  |  2 ++
>   xen/build.mk          | 18 ++++++++++++++++++
>   8 files changed, 47 insertions(+), 64 deletions(-)
> 
> diff --git a/xen/Makefile b/xen/Makefile
> index fb37043d08e0..d953dc50ac6a 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -285,6 +285,16 @@ CFLAGS += -flto
>   LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
>   endif
>   
> +# Note that link order matters!
> +ALL_OBJS-y                := common/built_in.o
> +ALL_OBJS-y                += drivers/built_in.o
> +ALL_OBJS-y                += lib/built_in.o
> +ALL_OBJS-y                += xsm/built_in.o
> +ALL_OBJS-y                += arch/$(TARGET_ARCH)/built_in.o
> +ALL_OBJS-$(CONFIG_CRYPTO) += crypto/built_in.o
> +
> +ALL_LIBS-y                := lib/lib.a
> +
>   include $(BASEDIR)/arch/$(TARGET_ARCH)/arch.mk
>   
>   # define new variables to avoid the ones defined in Config.mk
> @@ -407,7 +417,7 @@ $(TARGET): FORCE
>   	$(MAKE) -f $(BASEDIR)/Rules.mk -C include
>   	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) include
>   	$(MAKE) -f $(BASEDIR)/Rules.mk arch/$(TARGET_ARCH)/include/asm/asm-offsets.h
> -	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) MKRELOC=$(MKRELOC) $@
> +	$(MAKE) -f $(BASEDIR)/Rules.mk MKRELOC=$(MKRELOC) "ALL_OBJS=$(ALL_OBJS-y)" "ALL_LIBS=$(ALL_LIBS-y)" $@
>   
>   SUBDIRS = xsm arch/$(TARGET_ARCH) common drivers lib test
>   define all_sources
> diff --git a/xen/Rules.mk b/xen/Rules.mk
> index 7b8b9047cfd5..77d359bedaf8 100644
> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -8,25 +8,12 @@
>   include $(XEN_ROOT)/Config.mk
>   include $(BASEDIR)/scripts/Kbuild.include
>   
> -# Note that link order matters!
> -ALL_OBJS-y               += $(BASEDIR)/common/built_in.o
> -ALL_OBJS-y               += $(BASEDIR)/drivers/built_in.o
> -ALL_OBJS-y               += $(BASEDIR)/lib/built_in.o
> -ALL_OBJS-y               += $(BASEDIR)/xsm/built_in.o
> -ALL_OBJS-y               += $(BASEDIR)/arch/$(TARGET_ARCH)/built_in.o
> -ALL_OBJS-$(CONFIG_CRYPTO)   += $(BASEDIR)/crypto/built_in.o
> -
> -ALL_LIBS-y               := $(BASEDIR)/lib/lib.a
> -
>   # Initialise some variables
>   lib-y :=
>   targets :=
>   CFLAGS-y :=
>   AFLAGS-y :=
>   
> -ALL_OBJS := $(ALL_OBJS-y)
> -ALL_LIBS := $(ALL_LIBS-y)
> -
>   SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
>                                               $(foreach w,1 2 4, \
>                                                           rodata.str$(w).$(a)) \
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index d0dee10102b6..14952275772b 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -75,14 +75,6 @@ ifneq ($(CONFIG_DTB_FILE),"")
>   obj-y += dtb.o
>   endif
>   
> -ALL_OBJS := $(TARGET_SUBARCH)/head.o $(ALL_OBJS)
> -
> -# head.o is built by descending into the sub-directory, depends on the part of
> -# $(ALL_OBJS) that will eventually recurse into $(TARGET_SUBARCH)/ and build
> -# head.o
> -$(TARGET_SUBARCH)/head.o: $(BASEDIR)/arch/arm/built_in.o
> -$(TARGET_SUBARCH)/head.o: ;
> -
>   ifdef CONFIG_LIVEPATCH
>   all_symbols = --all-symbols
>   ifdef CONFIG_FAST_SYMBOL_LOOKUP
> @@ -98,33 +90,18 @@ ifeq ($(CONFIG_ARM_64),y)
>   	ln -sf $(@F) $@.efi
>   endif
>   
> -ifeq ($(CONFIG_LTO),y)
> -# Gather all LTO objects together
> -prelink_lto.o: $(ALL_OBJS) $(ALL_LIBS)
> -	$(LD_LTO) -r -o $@ $(filter-out %.a,$^) --start-group $(filter %.a,$^) --end-group
> -
> -# Link it with all the binary objects
> -prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o
> -	$(call if_changed,ld)
> -else
> -prelink.o: $(ALL_OBJS) $(ALL_LIBS) FORCE
> -	$(call if_changed,ld)
> -endif
> -
> -targets += prelink.o
> -
> -$(TARGET)-syms: prelink.o xen.lds
> -	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o \
> +$(TARGET)-syms: $(BASEDIR)/prelink.o xen.lds
> +	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< \
>   	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
>   	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
>   		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0.S
>   	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
> -	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o \
> +	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< \
>   	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
>   	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
>   		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1.S
>   	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
> -	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
> +	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< $(build_id_linker) \
>   	    $(@D)/.$(@F).1.o -o $@
>   	$(NM) -pa --format=sysv $(@D)/$(@F) \
>   		| $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort \
> diff --git a/xen/arch/arm/Rules.mk b/xen/arch/arm/Rules.mk
> index e69de29bb2d1..c6463a433efd 100644
> --- a/xen/arch/arm/Rules.mk
> +++ b/xen/arch/arm/Rules.mk
> @@ -0,0 +1,4 @@
> +# head.o is built by descending into arch/arm/$(TARGET_SUBARCH), depends on the
> +# part of $(ALL_OBJS) that will eventually recurse into $(TARGET_SUBARCH)/ and
> +# build head.o
> +arch/arm/$(TARGET_SUBARCH)/head.o: arch/arm/built_in.o ;
> diff --git a/xen/arch/arm/arch.mk b/xen/arch/arm/arch.mk
> index c3ac443b3788..ba3f140e2ea7 100644
> --- a/xen/arch/arm/arch.mk
> +++ b/xen/arch/arm/arch.mk
> @@ -26,3 +26,5 @@ ifeq ($(CONFIG_ARM64_ERRATUM_843419),y)
>           LDFLAGS += --fix-cortex-a53-843419
>       endif
>   endif
> +
> +ALL_OBJS-y := arch/arm/$(TARGET_SUBARCH)/head.o $(ALL_OBJS-y)
> diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
> index 9fc884813cb5..a830b5791e8b 100644
> --- a/xen/arch/x86/Makefile
> +++ b/xen/arch/x86/Makefile
> @@ -123,37 +123,20 @@ $(TARGET): $(TARGET)-syms $(efi-y) boot/mkelf32
>   
>   CFLAGS-$(XEN_BUILD_EFI) += -DXEN_BUILD_EFI
>   
> -ALL_OBJS := $(BASEDIR)/arch/x86/boot/built_in.o $(BASEDIR)/arch/x86/efi/built_in.o $(ALL_OBJS)
> -
> -ifeq ($(CONFIG_LTO),y)
> -# Gather all LTO objects together
> -prelink_lto.o: $(ALL_OBJS) $(ALL_LIBS)
> -	$(LD_LTO) -r -o $@ $(filter-out %.a,$^) --start-group $(filter %.a,$^) --end-group
> -
> -# Link it with all the binary objects
> -prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o FORCE
> -	$(call if_changed,ld)
> -else
> -prelink.o: $(ALL_OBJS) $(ALL_LIBS) FORCE
> -	$(call if_changed,ld)
> -endif
> -
> -targets += prelink.o
> -
> -$(TARGET)-syms: prelink.o xen.lds
> -	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
> +$(TARGET)-syms: $(BASEDIR)/prelink.o xen.lds
> +	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< $(build_id_linker) \
>   	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
>   	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
>   		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort \
>   		>$(@D)/.$(@F).0.S
>   	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
> -	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
> +	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< $(build_id_linker) \
>   	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
>   	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
>   		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort $(syms-warn-dup-y) \
>   		>$(@D)/.$(@F).1.S
>   	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
> -	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
> +	$(LD) $(XEN_LDFLAGS) -T xen.lds -N $< $(build_id_linker) \
>   	    $(@D)/.$(@F).1.o -o $@
>   	$(NM) -pa --format=sysv $(@D)/$(@F) \
>   		| $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort \
> @@ -206,7 +189,7 @@ note_file_option ?= $(note_file)
>   
>   ifeq ($(XEN_BUILD_PE),y)
>   extra-y += efi.lds
> -$(TARGET).efi: prelink.o $(note_file) efi.lds efi/relocs-dummy.o efi/mkreloc
> +$(TARGET).efi: $(BASEDIR)/prelink.o $(note_file) efi.lds efi/relocs-dummy.o efi/mkreloc
>   ifeq ($(CONFIG_DEBUG_INFO),y)
>   	$(if $(filter --strip-debug,$(EFI_LDFLAGS)),echo,:) "Will strip debug info from $(@F)"
>   endif
> @@ -235,7 +218,7 @@ $(TARGET).efi: FORCE
>   	echo '$(if $(filter y,$(XEN_BUILD_EFI)),xen.efi generation,EFI support) disabled'
>   endif
>   
> -efi/buildid.o efi/relocs-dummy.o: $(BASEDIR)/arch/x86/efi/built_in.o
> +# These should already have been rebuilt when building the prerequisite of "prelink.o"
>   efi/buildid.o efi/relocs-dummy.o: ;
>   
>   .PHONY: include
> diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
> index fa7cf3844362..bfd5eaa35f25 100644
> --- a/xen/arch/x86/arch.mk
> +++ b/xen/arch/x86/arch.mk
> @@ -104,3 +104,5 @@ endif
>   
>   # Set up the assembler include path properly for older toolchains.
>   CFLAGS += -Wa,-I$(BASEDIR)/include
> +
> +ALL_OBJS-y := arch/x86/boot/built_in.o arch/x86/efi/built_in.o $(ALL_OBJS-y)
> diff --git a/xen/build.mk b/xen/build.mk
> index 3d7a91df22d1..af1b28311393 100644
> --- a/xen/build.mk
> +++ b/xen/build.mk
> @@ -59,3 +59,21 @@ arch/$(TARGET_ARCH)/include/asm/asm-offsets.h: asm-offsets.s
>   	  sed -rne "/^[^#].*==>/{s:.*==>(.*)<==.*:\1:; s: [\$$#]: :; p;}"; \
>   	  echo ""; \
>   	  echo "#endif") <$< >$@
> +
> +ifeq ($(CONFIG_LTO),y)
> +# Gather all LTO objects together
> +prelink_lto.o: $(ALL_OBJS) $(ALL_LIBS)
> +	$(LD_LTO) -r -o $@ $(filter-out %.a,$^) --start-group $(filter %.a,$^) --end-group
> +
> +# Link it with all the binary objects
> +prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o FORCE
> +	$(call if_changed,ld)
> +else
> +prelink.o: $(ALL_OBJS) $(ALL_LIBS) FORCE
> +	$(call if_changed,ld)
> +endif
> +
> +targets += prelink.o
> +
> +$(TARGET): prelink.o FORCE
> +	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) $@

-- 
Julien Grall

Re: [XEN PATCH v9 20/30] build: rework "clean" to clean from the root dir

[Julien Grall]

Hi Anthony,

On 25/01/2022 11:00, Anthony PERARD wrote:
> This will allow "clean" to work from an out-of-tree build when
> it will be available.
> 
> Some of the file been removed in current "clean" target aren't added
> to $(clean-files) because they are already listed in $(extra-) or
> $(extra-y).
> 
> Also start to clean files listed in $(targets). This allows to clean
> "common/config_data.S" and "xsm/flask/flask-policy.S" without
> having to list them a second time.
> 
> Also clean files in "arch/x86/boot" from that directory by allowing
> "clean" to descend into the subdir by adding "boot" into $(subdir-).
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall

Re: [XEN PATCH v9 30/30] build: adding out-of-tree support to the xen build

[Julien Grall]

Hi Anthony,

On 25/01/2022 11:37, Anthony PERARD wrote:
> On Tue, Jan 25, 2022 at 11:01:03AM +0000, Anthony PERARD wrote:
>> This implement out-of-tree support, there's two ways to create an
>> out-of-tree build tree (after that, `make` in that new directory
>> works):
>>      make O=build
>>      mkdir build; cd build; make -f ../Makefile
>> also works with an absolute path for both.
> 
> Maybe I should have made it more explicit in the description that those
> only works within the xen dir, i.e. not the whole project.

That would be useful. In any case:

Acked-by: Julien Grall <jgrall@amazon.com>
Tested-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall

Re: [XEN PATCH v9 23/30] build,x86: remove the need for build32.mk

[Jan Beulich]

On 25.01.2022 12:00, Anthony PERARD wrote:
> Rework "arch/x86/boot/Makefile" to allow it to build both file
> "cmdline.S" and "reloc.S" without "build32.mk".
> 
> These will now use the main rules for "%.o: %.c", and thus generate a
> dependency file. (We will not need to track the dependency manually
> anymore.)
> 
> But for that, we need to override the main CFLAGS to do a 32bit build.
> We introduce XEN_TREEWIDE_CFLAGS which can be reused in boot/Makefile,
> and avoid the need to reparse Config.mk with a different value for
> XEN_TARGET_ARCH. From this new $(XEN_TREEWIDE_CFLAGS), we only need to
> change -m64 to have the 32bit flags. Then those are applied only to
> "cmdline.o" and "reloc.o".
> 
> Specifically apply the rule "%.S: %.bin" to both cmdline.S and reloc.S
> to avoid make trying to regenerate other %.S files with it.
> 
> There is no change expected to the resulting "cmdline.S" and
> "reloc.S", only the *.o file changes as their symbol for FILE goes
> from "cmdline.c" to "arch/x86//cmdline.c". (No idea why "boot" is
> missing from the string.) (I've only check with GCC, not clang.)
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with one question, just to be sure I understand things right:

> --- a/xen/arch/x86/boot/Makefile
> +++ b/xen/arch/x86/boot/Makefile
> @@ -1,25 +1,42 @@
>  obj-bin-y += head.o
> +head-srcs := cmdline.S reloc.S
>  
> -DEFS_H_DEPS = $(abs_srctree)/$(src)/defs.h $(abs_srctree)/include/xen/stdbool.h
> +nocov-y += $(head-srcs:.S=.o)
> +noubsan-y += $(head-srcs:.S=.o)
> +targets += $(head-srcs:.S=.o)
>  
> -CMDLINE_DEPS = $(DEFS_H_DEPS) $(abs_srctree)/$(src)/video.h \
> -	       $(BASEDIR)/include/xen/kconfig.h \
> -	       $(BASEDIR)/include/generated/autoconf.h
> +head-srcs := $(addprefix $(obj)/, $(head-srcs))
>  
> -RELOC_DEPS = $(DEFS_H_DEPS) \
> -	     $(BASEDIR)/include/generated/autoconf.h \
> -	     $(BASEDIR)/include/xen/kconfig.h \
> -	     $(BASEDIR)/include/xen/multiboot.h \
> -	     $(BASEDIR)/include/xen/multiboot2.h \
> -	     $(BASEDIR)/include/xen/const.h \
> -	     $(BASEDIR)/include/public/arch-x86/hvm/start_info.h
> +$(obj)/head.o: $(head-srcs)
>  
> -$(obj)/head.o: $(obj)/cmdline.S $(obj)/reloc.S
> +CFLAGS_x86_32 := $(subst -m64,-m32 -march=i686,$(XEN_TREEWIDE_CFLAGS))
> +$(call cc-options-add,CFLAGS_x86_32,CC,$(EMBEDDED_EXTRA_CFLAGS))
> +CFLAGS_x86_32 += -Werror -fno-builtin -g0 -msoft-float
> +CFLAGS_x86_32 += -I$(srctree)/include
>  
> -$(obj)/cmdline.S: $(src)/cmdline.c $(CMDLINE_DEPS) $(src)/build32.lds
> -	$(MAKE) -f $(abs_srctree)/$(src)/build32.mk -C $(obj) $(@F) CMDLINE_DEPS="$(CMDLINE_DEPS)"
> +# override for 32bit binaries
> +$(head-srcs:.S=.o): CFLAGS_stack_boundary :=

You overriding CFLAGS_stack_boundary but not object_label_flags is
merely because the latter has no (unwanted) effect on the compilation?

Jan

Re: [XEN PATCH v9 24/30] build: grab common EFI source files in arch specific dir

[Jan Beulich]

On 25.01.2022 12:00, Anthony PERARD wrote:
> --- a/xen/arch/x86/Makefile
> +++ b/xen/arch/x86/Makefile
> @@ -77,8 +77,9 @@ obj-$(CONFIG_COMPAT) += x86_64/platform_hypercall.o
>  obj-y += sysctl.o
>  endif
>  
> -# Allows "clean" to descend into boot/
> +# Allows "clean" to descend
>  subdir- += boot
> +subdir- += efi

No similar addition is needed for Arm?

> --- /dev/null
> +++ b/xen/common/efi/efi-common.mk
> @@ -0,0 +1,15 @@
> +EFIOBJ-y := boot.init.o pe.init.o ebmalloc.o runtime.o
> +EFIOBJ-$(CONFIG_COMPAT) += compat.o
> +
> +CFLAGS-y += -fshort-wchar
> +CFLAGS-y += -iquote $(srctree)/common/efi
> +
> +# Part of the command line transforms $(obj) in to a relative reverted path.
> +# e.g.: It transforms "dir/foo/bar" into successively
> +#       "dir foo bar", ".. .. ..", "../../.."
> +$(obj)/%.c: $(srctree)/common/efi/%.c FORCE
> +	$(Q)ln -nfs $(subst $(space),/,$(patsubst %,..,$(subst /, ,$(obj))))/common/efi/$(<F) $@

What is the "reverted" about in the comment? Also (nit) I think you want
s/in to/into/.

Jan

Re: [XEN PATCH v9 27/30] build: rework "headers*.chk" prerequisite in include/

[Jan Beulich]

On 25.01.2022 12:01, Anthony PERARD wrote:
> Listing public headers when out-of-tree build are involved becomes
> more annoying where every path to every headers needs to start with
> "$(srctree)/$(src)", or $(wildcard ) will not work. This means more
> repetition. ( "$(srcdir)" is a shortcut for "$(srctree)/$(src)" )
> 
> This patch attempt to reduce the amount of duplication and make better
> use of make's meta programming capability. The filters are now listed
> in a variable and don't have to repeat the path to the headers files
> as this is added later as needed.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

Re: [XEN PATCH v9 28/30] build: specify source tree in include/ for prerequisite

[Jan Beulich]

On 25.01.2022 12:01, Anthony PERARD wrote:
> When doing an out-of-tree build, and thus setting VPATH,
> GNU Make 3.81 on Ubuntu Trusty complains about Circular dependency of
> include/Makefile and include/xlat.lst and drop them. The build fails
> later due to headers malformed.
> 
> This might be due to bug #13529
>     "Incorrect circular dependancy"
>     https://savannah.gnu.org/bugs/?13529
> which was fixed in 3.82.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

Re: [XEN PATCH v9 29/30] build: shuffle main Makefile

[Jan Beulich]

On 25.01.2022 12:01, Anthony PERARD wrote:
> Reorganize a bit the Makefile ahead of patch
> "build: adding out-of-tree support to the xen build"
> 
> We are going to want to calculate all the $(*srctree) and $(*objtree)
> once, when we can calculate them. This can happen within the
> "$(root-make-done)" guard, in an out-of-tree build scenario, so move
> those variable there.
> 
> $(XEN_ROOT) is going to depends on the value of $(abs_srctree) so
> needs to move as well. "Kbuild.include" also depends on $(srctree).
> 
> Next, "Config.mk" depends on $(XEN_ROOT) and $(TARGET_*ARCH) depends
> on "Config.mk" so those needs to move as well.
> 
> This should only be code movement without functional changes.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

Re: [XEN PATCH v9 26/30] build: replace $(BASEDIR) and use $(srctree)

[Anthony PERARD]

On Tue, Jan 25, 2022 at 11:00:59AM +0000, Anthony PERARD wrote:
> $(srctree) is a better description for the source directory than
> $(BASEDIR) that has been used for both source and build directory
> (which where the same).
> 
> This adds $(srctree) to a few path where make's VPATH=$(srctree) won't
> apply. And replace $(BASEDIR) by $(srctree).
> 
> Introduce "$(srcdir)" as a shortcut for "$(srctree)/$(src)" as the
> later is used often enough.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>

This patch is now missing two hunks due to recent changes in the tree:

diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 11ac0d5e28a4..1ab9db3424c2 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -153,7 +153,7 @@ $(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
                >$(@D)/$(@F).map
        rm -f $(@D)/.$(@F).[0-9]* $(@D)/..$(@F).[0-9]*
 ifeq ($(CONFIG_XEN_IBT),y)
-       $(SHELL) $(BASEDIR)/tools/check-endbr.sh $@
+       $(SHELL) $(srctree)/tools/check-endbr.sh $@
 endif

 $(obj)/note.o: $(TARGET)-syms
@@ -226,7 +226,7 @@ endif
                | $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort >$(@D)/$(@F).map
        rm -f $(@D)/.$(@F).[0-9]* $(@D)/..$(@F).[0-9]*
 ifeq ($(CONFIG_XEN_IBT),y)
-       $(SHELL) $(BASEDIR)/tools/check-endbr.sh $@
+       $(SHELL) $(srctree)/tools/check-endbr.sh $@
 endif
 else
 $(TARGET).efi: FORCE

-- 
Anthony PERARD

Re: [XEN PATCH v9 23/30] build,x86: remove the need for build32.mk

[Anthony PERARD]

On Thu, Mar 03, 2022 at 11:29:36AM +0100, Jan Beulich wrote:
> On 25.01.2022 12:00, Anthony PERARD wrote:
> > Rework "arch/x86/boot/Makefile" to allow it to build both file
> > "cmdline.S" and "reloc.S" without "build32.mk".
> > 
> > These will now use the main rules for "%.o: %.c", and thus generate a
> > dependency file. (We will not need to track the dependency manually
> > anymore.)
> > 
> > But for that, we need to override the main CFLAGS to do a 32bit build.
> > We introduce XEN_TREEWIDE_CFLAGS which can be reused in boot/Makefile,
> > and avoid the need to reparse Config.mk with a different value for
> > XEN_TARGET_ARCH. From this new $(XEN_TREEWIDE_CFLAGS), we only need to
> > change -m64 to have the 32bit flags. Then those are applied only to
> > "cmdline.o" and "reloc.o".
> > 
> > Specifically apply the rule "%.S: %.bin" to both cmdline.S and reloc.S
> > to avoid make trying to regenerate other %.S files with it.
> > 
> > There is no change expected to the resulting "cmdline.S" and
> > "reloc.S", only the *.o file changes as their symbol for FILE goes
> > from "cmdline.c" to "arch/x86//cmdline.c". (No idea why "boot" is
> > missing from the string.) (I've only check with GCC, not clang.)
> > 
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> with one question, just to be sure I understand things right:
> 
> > --- a/xen/arch/x86/boot/Makefile
> > +++ b/xen/arch/x86/boot/Makefile
> > @@ -1,25 +1,42 @@
> >  obj-bin-y += head.o
> > +head-srcs := cmdline.S reloc.S
> >  
> > -DEFS_H_DEPS = $(abs_srctree)/$(src)/defs.h $(abs_srctree)/include/xen/stdbool.h
> > +nocov-y += $(head-srcs:.S=.o)
> > +noubsan-y += $(head-srcs:.S=.o)
> > +targets += $(head-srcs:.S=.o)
> >  
> > -CMDLINE_DEPS = $(DEFS_H_DEPS) $(abs_srctree)/$(src)/video.h \
> > -	       $(BASEDIR)/include/xen/kconfig.h \
> > -	       $(BASEDIR)/include/generated/autoconf.h
> > +head-srcs := $(addprefix $(obj)/, $(head-srcs))
> >  
> > -RELOC_DEPS = $(DEFS_H_DEPS) \
> > -	     $(BASEDIR)/include/generated/autoconf.h \
> > -	     $(BASEDIR)/include/xen/kconfig.h \
> > -	     $(BASEDIR)/include/xen/multiboot.h \
> > -	     $(BASEDIR)/include/xen/multiboot2.h \
> > -	     $(BASEDIR)/include/xen/const.h \
> > -	     $(BASEDIR)/include/public/arch-x86/hvm/start_info.h
> > +$(obj)/head.o: $(head-srcs)
> >  
> > -$(obj)/head.o: $(obj)/cmdline.S $(obj)/reloc.S
> > +CFLAGS_x86_32 := $(subst -m64,-m32 -march=i686,$(XEN_TREEWIDE_CFLAGS))
> > +$(call cc-options-add,CFLAGS_x86_32,CC,$(EMBEDDED_EXTRA_CFLAGS))
> > +CFLAGS_x86_32 += -Werror -fno-builtin -g0 -msoft-float
> > +CFLAGS_x86_32 += -I$(srctree)/include
> >  
> > -$(obj)/cmdline.S: $(src)/cmdline.c $(CMDLINE_DEPS) $(src)/build32.lds
> > -	$(MAKE) -f $(abs_srctree)/$(src)/build32.mk -C $(obj) $(@F) CMDLINE_DEPS="$(CMDLINE_DEPS)"
> > +# override for 32bit binaries
> > +$(head-srcs:.S=.o): CFLAGS_stack_boundary :=
> 
> You overriding CFLAGS_stack_boundary but not object_label_flags is
> merely because the latter has no (unwanted) effect on the compilation?

Yes.

Thanks,

-- 
Anthony PERARD

Re: [XEN PATCH v9 24/30] build: grab common EFI source files in arch specific dir

[Anthony PERARD]

On Thu, Mar 03, 2022 at 11:37:08AM +0100, Jan Beulich wrote:
> On 25.01.2022 12:00, Anthony PERARD wrote:
> > --- a/xen/arch/x86/Makefile
> > +++ b/xen/arch/x86/Makefile
> > @@ -77,8 +77,9 @@ obj-$(CONFIG_COMPAT) += x86_64/platform_hypercall.o
> >  obj-y += sysctl.o
> >  endif
> >  
> > -# Allows "clean" to descend into boot/
> > +# Allows "clean" to descend
> >  subdir- += boot
> > +subdir- += efi
> 
> No similar addition is needed for Arm?

No, because Arm already have "obj-$(CONFIG_ARM_64) += efi/", which has
the same effect on clean.

Make clean doesn't use ${ALL_OBJS} to find out which directory to clean, so
adding "subdir-" is needed at the moment.

> > --- /dev/null
> > +++ b/xen/common/efi/efi-common.mk
> > @@ -0,0 +1,15 @@
> > +EFIOBJ-y := boot.init.o pe.init.o ebmalloc.o runtime.o
> > +EFIOBJ-$(CONFIG_COMPAT) += compat.o
> > +
> > +CFLAGS-y += -fshort-wchar
> > +CFLAGS-y += -iquote $(srctree)/common/efi
> > +
> > +# Part of the command line transforms $(obj) in to a relative reverted path.
> > +# e.g.: It transforms "dir/foo/bar" into successively
> > +#       "dir foo bar", ".. .. ..", "../../.."
> > +$(obj)/%.c: $(srctree)/common/efi/%.c FORCE
> > +	$(Q)ln -nfs $(subst $(space),/,$(patsubst %,..,$(subst /, ,$(obj))))/common/efi/$(<F) $@
> 
> What is the "reverted" about in the comment? Also (nit) I think you want
> s/in to/into/.

I've tried to described in the single word that the result is a relative
path that goes in the opposite direction to the original relative path.
Instead of going down, it goes up the hierarchy of directories.
Maybe "reversed" would be better? Do you have other suggestion?

Thanks,

-- 
Anthony PERARD

Re: [XEN PATCH v9 24/30] build: grab common EFI source files in arch specific dir

[Jan Beulich]

On 03.03.2022 16:41, Anthony PERARD wrote:
> On Thu, Mar 03, 2022 at 11:37:08AM +0100, Jan Beulich wrote:
>> On 25.01.2022 12:00, Anthony PERARD wrote:
>>> --- a/xen/arch/x86/Makefile
>>> +++ b/xen/arch/x86/Makefile
>>> @@ -77,8 +77,9 @@ obj-$(CONFIG_COMPAT) += x86_64/platform_hypercall.o
>>>  obj-y += sysctl.o
>>>  endif
>>>  
>>> -# Allows "clean" to descend into boot/
>>> +# Allows "clean" to descend
>>>  subdir- += boot
>>> +subdir- += efi
>>
>> No similar addition is needed for Arm?
> 
> No, because Arm already have "obj-$(CONFIG_ARM_64) += efi/", which has
> the same effect on clean.
> 
> Make clean doesn't use ${ALL_OBJS} to find out which directory to clean, so
> adding "subdir-" is needed at the moment.

Oh, I see.

>>> --- /dev/null
>>> +++ b/xen/common/efi/efi-common.mk
>>> @@ -0,0 +1,15 @@
>>> +EFIOBJ-y := boot.init.o pe.init.o ebmalloc.o runtime.o
>>> +EFIOBJ-$(CONFIG_COMPAT) += compat.o
>>> +
>>> +CFLAGS-y += -fshort-wchar
>>> +CFLAGS-y += -iquote $(srctree)/common/efi
>>> +
>>> +# Part of the command line transforms $(obj) in to a relative reverted path.
>>> +# e.g.: It transforms "dir/foo/bar" into successively
>>> +#       "dir foo bar", ".. .. ..", "../../.."
>>> +$(obj)/%.c: $(srctree)/common/efi/%.c FORCE
>>> +	$(Q)ln -nfs $(subst $(space),/,$(patsubst %,..,$(subst /, ,$(obj))))/common/efi/$(<F) $@
>>
>> What is the "reverted" about in the comment? Also (nit) I think you want
>> s/in to/into/.
> 
> I've tried to described in the single word that the result is a relative
> path that goes in the opposite direction to the original relative path.
> Instead of going down, it goes up the hierarchy of directories.
> Maybe "reversed" would be better? Do you have other suggestion?

I'd simply omit the word. In case you're fine with that, I'd be happy
to adjust while committing.

Jan

Re: [XEN PATCH v9 24/30] build: grab common EFI source files in arch specific dir

[Anthony PERARD]

On Thu, Mar 03, 2022 at 05:01:07PM +0100, Jan Beulich wrote:
> On 03.03.2022 16:41, Anthony PERARD wrote:
> > On Thu, Mar 03, 2022 at 11:37:08AM +0100, Jan Beulich wrote:
> >> On 25.01.2022 12:00, Anthony PERARD wrote:
> >>> +# Part of the command line transforms $(obj) in to a relative reverted path.
> >>> +# e.g.: It transforms "dir/foo/bar" into successively
> >>> +#       "dir foo bar", ".. .. ..", "../../.."
> >>> +$(obj)/%.c: $(srctree)/common/efi/%.c FORCE
> >>> +	$(Q)ln -nfs $(subst $(space),/,$(patsubst %,..,$(subst /, ,$(obj))))/common/efi/$(<F) $@
> >>
> >> What is the "reverted" about in the comment? Also (nit) I think you want
> >> s/in to/into/.
> > 
> > I've tried to described in the single word that the result is a relative
> > path that goes in the opposite direction to the original relative path.
> > Instead of going down, it goes up the hierarchy of directories.
> > Maybe "reversed" would be better? Do you have other suggestion?
> 
> I'd simply omit the word. In case you're fine with that, I'd be happy
> to adjust while committing.

I think that would sound kind of strange. $(obj) is already a relative
path. It would probably be better to just drop the end of the sentence
in that case. With the example showing what is happening, that would
probably be enough. The sentence would then be:

    # Part of the command line transforms $(obj).
    # e.g.: It transforms "dir/foo/bar" into successively
    #       "dir foo bar", ".. .. ..", "../../.."

-- 
Anthony PERARD

Re: [XEN PATCH v9 24/30] build: grab common EFI source files in arch specific dir

[Jan Beulich]

On 03.03.2022 17:50, Anthony PERARD wrote:
> On Thu, Mar 03, 2022 at 05:01:07PM +0100, Jan Beulich wrote:
>> On 03.03.2022 16:41, Anthony PERARD wrote:
>>> On Thu, Mar 03, 2022 at 11:37:08AM +0100, Jan Beulich wrote:
>>>> On 25.01.2022 12:00, Anthony PERARD wrote:
>>>>> +# Part of the command line transforms $(obj) in to a relative reverted path.
>>>>> +# e.g.: It transforms "dir/foo/bar" into successively
>>>>> +#       "dir foo bar", ".. .. ..", "../../.."
>>>>> +$(obj)/%.c: $(srctree)/common/efi/%.c FORCE
>>>>> +	$(Q)ln -nfs $(subst $(space),/,$(patsubst %,..,$(subst /, ,$(obj))))/common/efi/$(<F) $@
>>>>
>>>> What is the "reverted" about in the comment? Also (nit) I think you want
>>>> s/in to/into/.
>>>
>>> I've tried to described in the single word that the result is a relative
>>> path that goes in the opposite direction to the original relative path.
>>> Instead of going down, it goes up the hierarchy of directories.
>>> Maybe "reversed" would be better? Do you have other suggestion?
>>
>> I'd simply omit the word. In case you're fine with that, I'd be happy
>> to adjust while committing.
> 
> I think that would sound kind of strange. $(obj) is already a relative
> path. It would probably be better to just drop the end of the sentence
> in that case. With the example showing what is happening, that would
> probably be enough. The sentence would then be:
> 
>     # Part of the command line transforms $(obj).
>     # e.g.: It transforms "dir/foo/bar" into successively
>     #       "dir foo bar", ".. .. ..", "../../.."

Fine with me. Then:
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan

Re: [XEN PATCH v9 06/30] build: rework test/livepatch/Makefile

[Ross Lagerwall]

> From: Anthony PERARD <anthony.perard@citrix.com>
> Sent: Tuesday, January 25, 2022 11:00 AM
> To: xen-devel@lists.xenproject.org <xen-devel@lists.xenproject.org>
> Cc: Anthony Perard <anthony.perard@citrix.com>; Jan Beulich <jbeulich@suse.com>; Andrew Cooper <Andrew.Cooper3@citrix.com>; George Dunlap <George.Dunlap@citrix.com>; Julien Grall <julien@xen.org>; Stefano Stabellini <sstabellini@kernel.org>; Wei Liu <wl@xen.org>; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>; Ross Lagerwall <ross.lagerwall@citrix.com>
> Subject: [XEN PATCH v9 06/30] build: rework test/livepatch/Makefile 
>  
> This rework the livepatch/Makefile to make it less repetitive and make
> use of the facilities. All the targets to be built are now listed in
> $(extra-y) which will allow Rules.mk to build them without the need of
> a local target in a future patch.
> 
> There are some changes/fixes in this patch:
> - when "xen-syms" is used for a target, it is added to the dependency
>   list of the target, which allow to rebuild the target when xen-syms
>   changes. But if "xen-syms" is missing, make simply fails.
> - modinfo.o wasn't removing it's $@.bin file like the other targets,
>   this is now done.
> - The command to build *.livepatch targets as been fixed to use
>   $(XEN_LDFLAGS) rather than just $(LDFLAGS) which is a fallout from
>   2740d96efdd3 ("xen/build: have the root Makefile generates the
>   CFLAGS")
> 
> make will findout the dependencies of the *.livepatch files and thus
> what to built by "looking" at the objects listed in the *-objs
> variables. The actual dependencies is generated by the new
> "multi-depend" macro.
> 
> "$(targets)" needs to be updated with the objects listed in the
> different *-objs variables to allow make to load the .*.cmd dependency
> files.
> 
> This patch copies the macro "multi_depend" from Linux 5.12, and rename
> it to "multi-depend".
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> ---
> 

Acked-by: Ross Lagerwall <ross.lagerwall@citrix.com>