Xen-Devel Archive on lore.kernel.org
 help / color / Atom feed
* [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements
@ 2020-02-26 11:33 Anthony PERARD
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 01/23] xen/include: remove include of Config.mk Anthony PERARD
                   ` (23 more replies)
  0 siblings, 24 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Kevin Tian, Stefano Stabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, George Dunlap, Tim Deegan, Ian Jackson,
	Jan Beulich, Andrew Cooper, Anthony PERARD, Daniel De Graaf,
	Volodymyr Babchuk, Roger Pau Monné

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-v3

v3:
- new patches that do some cleanup or fix issues
- have rework most patches, to have better commit message or change the coding
  style, or fix issues that I've seen. There were some cases where CFLAGS were
  missing.
  See patch notes for details
- introduce if_changed*. That plenty of new patches on top of what we had in v2.
  (those changes ignore CONFIG_LTO=y, I'll see about fixing that later)

(There is more to come in order to use fixdep from Linux, but that's not ready)

v2.1:
- some fixes

v2:
Rather than taking Kbuild and making it work with Xen, the v2 takes the opposite
approach of slowly transforming our current build system into Kbuild. That have
the advantage of keeping all the feature we have and making the patches much
easier to review. Kconfig update is done in an other patch series.

Hi,

I have work toward building Xen (the hypervisor) with Linux's build system,
Kbuild.

The main reason for that is to be able to have out-of-tree build. It's annoying
when a build fail because of the pvshim. Other benefit is a much faster
rebuild, and `make clean` doesn't take ages, and better dependencies to figure
out what needs to be rebuild.

So, we are not there yet, but the series already contain quite a few
improvement and cleanup. More patches are going to be added to the series.

Cheers,

Anthony PERARD (23):
  xen/include: remove include of Config.mk
  Makefile: Fix install-tests
  xen/build: Remove confusing comment on the %.s:%.S rule
  xen/build: remove use of AFLAGS-y
  xen/build: Allow to test clang .include without asm symlink
  xen/build: Fix section-renaming of libfdt and libelf
  xen/build: Use obj-y += subdir/ instead of subdir-y
  xen/build: use $(clean) shorthand for clean targets
  xen/build: extract clean target from Rules.mk
  xen/build: run targets csopes,tags,.. without Rules.mk
  xen/build: make tests in test/ directly
  xen/build: Move as-option-add to xen/
  xen/build: include include/config/auto.conf in main Makefile
  xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS)
  xen/build: have the root Makefile generates the CFLAGS
  xen/build: introduce if_changed and if_changed_rule
  xen/build: Start using if_changed
  xen/build: use if_changed on built_in.o
  xen/build: Use if_changed_rules with %.o:%.c targets
  xen/build: factorise generation of the linker scripts
  xen/build: Use if_changed for prelink*.o
  xen,symbols: rework file symbols selection
  xen/build: use if_changed to build guest_%.o

 .gitignore                           |   1 +
 Config.mk                            |  17 --
 Makefile                             |   6 +-
 xen/Makefile                         | 257 ++++++++++++++++++++-----
 xen/Rules.mk                         | 269 +++++++++++++++------------
 xen/arch/arm/Makefile                |  33 ++--
 xen/arch/arm/Rules.mk                |  93 ---------
 xen/arch/arm/arch.mk                 |  88 +++++++++
 xen/arch/arm/arm32/Makefile          |   2 +-
 xen/arch/arm/arm64/Makefile          |   2 +-
 xen/arch/arm/efi/Makefile            |   2 +-
 xen/arch/x86/Makefile                |  55 +++---
 xen/arch/x86/Rules.mk                |  91 +--------
 xen/arch/x86/acpi/Makefile           |   2 +-
 xen/arch/x86/arch.mk                 |  84 +++++++++
 xen/arch/x86/cpu/Makefile            |   4 +-
 xen/arch/x86/efi/Makefile            |   9 +-
 xen/arch/x86/guest/Makefile          |   4 +-
 xen/arch/x86/hvm/Makefile            |   6 +-
 xen/arch/x86/mm/Makefile             |  19 +-
 xen/arch/x86/mm/hap/Makefile         |  15 +-
 xen/arch/x86/mm/shadow/Makefile      |  15 +-
 xen/arch/x86/x86_64/Makefile         |   2 +-
 xen/common/Makefile                  |  10 +-
 xen/common/libelf/Makefile           |  14 +-
 xen/common/libfdt/Makefile           |  11 +-
 xen/drivers/Makefile                 |  14 +-
 xen/drivers/acpi/Makefile            |   6 +-
 xen/drivers/passthrough/Makefile     |   8 +-
 xen/drivers/passthrough/vtd/Makefile |   2 +-
 xen/include/Makefile                 |   4 +-
 xen/lib/Makefile                     |   2 +-
 xen/scripts/Kbuild.include           | 134 +++++++++++++
 xen/scripts/Makefile.clean           |  30 +++
 xen/tools/symbols.c                  |  22 ++-
 xen/xsm/Makefile                     |   2 +-
 xen/xsm/flask/Makefile               |  21 ++-
 xen/xsm/flask/ss/Makefile            |   2 +-
 38 files changed, 870 insertions(+), 488 deletions(-)
 create mode 100644 xen/arch/arm/arch.mk
 create mode 100644 xen/arch/x86/arch.mk
 create mode 100644 xen/scripts/Makefile.clean

-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 01/23] xen/include: remove include of Config.mk
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 02/23] Makefile: Fix install-tests Anthony PERARD
                   ` (22 subsequent siblings)
  23 siblings, 0 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD

It isn't necessary to include Config.mk here because this Makefile is
only used by xen/Rules.mk which already includes Config.mk.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
 xen/include/Makefile | 2 --
 1 file changed, 2 deletions(-)

diff --git a/xen/include/Makefile b/xen/include/Makefile
index fde0ca013121..433bad9055b2 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -1,5 +1,3 @@
-include $(XEN_ROOT)/Config.mk
-
 ifneq ($(CONFIG_COMPAT),)
 
 compat-arch-$(CONFIG_X86) := x86_32
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 02/23] Makefile: Fix install-tests
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 01/23] xen/include: remove include of Config.mk Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 03/23] xen/build: Remove confusing comment on the %.s:%.S rule Anthony PERARD
                   ` (21 subsequent siblings)
  23 siblings, 0 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD

The top-level makefile make uses of internal implementation detail of
the xen build system. Avoid that by creating a new target
"install-tests" in xen/Makefile, and by fixing the top-level Makefile
to not call xen/Rules.mk anymore.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---

Notes:
    v2.1:
    - new patch, fix `make dist-tests` in osstest.

 Makefile     | 6 ++----
 xen/Makefile | 3 +++
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index 512d6b73c898..9ad2602f63f0 100644
--- a/Makefile
+++ b/Makefile
@@ -155,13 +155,11 @@ install-docs:
 # We only have build-tests install-tests, not uninstall-tests etc.
 .PHONY: build-tests
 build-tests: build-xen
-	export BASEDIR=$(XEN_ROOT)/xen; \
-	$(MAKE) -f $$BASEDIR/Rules.mk -C xen/test build
+	$(MAKE) -C xen tests
 
 .PHONY: install-tests
 install-tests: install-xen
-	export BASEDIR=$(XEN_ROOT)/xen; \
-	$(MAKE) -f $$BASEDIR/Rules.mk -C xen/test install
+	$(MAKE) -C xen $@
 
 # build xen and the tools and place them in the install
 # directory. 'make install' should then copy them to the normal system
diff --git a/xen/Makefile b/xen/Makefile
index c326fee5880e..72bc89924622 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -90,6 +90,9 @@ _install: $(TARGET)$(CONFIG_XEN_INSTALL_SUFFIX)
 .PHONY: _tests
 _tests:
 	$(MAKE) -f $(BASEDIR)/Rules.mk -C test tests
+.PHONY: install-tests
+install-tests:
+	$(MAKE) -f $(BASEDIR)/Rules.mk -C test install
 
 .PHONY: _uninstall
 _uninstall: D=$(DESTDIR)
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 03/23] xen/build: Remove confusing comment on the %.s:%.S rule
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 01/23] xen/include: remove include of Config.mk Anthony PERARD
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 02/23] Makefile: Fix install-tests Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-26 11:53   ` Wei Liu
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 04/23] xen/build: remove use of AFLAGS-y Anthony PERARD
                   ` (20 subsequent siblings)
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD

That comment was introduce by 3943db776371 ("[XEN] Can be built
-std=gnu99 (except for .S files).") to explain why CFLAGS was removed
from the command line. The comment is already written where the
-std=gnu flags gets remove from AFLAGS, no need to repeat it.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v3:
    - new patch

 xen/Rules.mk | 1 -
 1 file changed, 1 deletion(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index d22a16d28282..c21203351a9f 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -230,7 +230,6 @@ $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o Makefile
 %.s: %.c Makefile
 	$(CC) $(filter-out -Wa$(comma)%,$(CFLAGS)) -S $< -o $@
 
-# -std=gnu{89,99} gets confused by # as an end-of-line comment marker
 %.s: %.S Makefile
 	$(CPP) $(filter-out -Wa$(comma)%,$(AFLAGS)) $< -o $@
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 04/23] xen/build: remove use of AFLAGS-y
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (2 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 03/23] xen/build: Remove confusing comment on the %.s:%.S rule Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-26 12:58   ` Jan Beulich
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 05/23] xen/build: Allow to test clang .include without asm symlink Anthony PERARD
                   ` (19 subsequent siblings)
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD

And simply add directly to AFLAGS.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v3:
    - new patch

 xen/Rules.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index c21203351a9f..154269bfd96c 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -71,7 +71,7 @@ ifneq ($(CONFIG_CC_IS_CLANG),y)
 CFLAGS += -Wa,--strip-local-absolute
 endif
 
-AFLAGS-y                += -D__ASSEMBLY__
+AFLAGS += -D__ASSEMBLY__
 
 ALL_OBJS := $(ALL_OBJS-y)
 
@@ -85,7 +85,7 @@ CFLAGS += $(EXTRA_CFLAGS_XEN_CORE)
 # Most CFLAGS are safe for assembly files:
 #  -std=gnu{89,99} gets confused by #-prefixed end-of-line comments
 #  -flto makes no sense and annoys clang
-AFLAGS += $(AFLAGS-y) $(filter-out -std=gnu% -flto,$(CFLAGS))
+AFLAGS += $(filter-out -std=gnu% -flto,$(CFLAGS))
 
 # LDFLAGS are only passed directly to $(LD)
 LDFLAGS += $(LDFLAGS_DIRECT)
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 05/23] xen/build: Allow to test clang .include without asm symlink
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (3 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 04/23] xen/build: remove use of AFLAGS-y Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-27  9:05   ` Roger Pau Monné
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 06/23] xen/build: Fix section-renaming of libfdt and libelf Anthony PERARD
                   ` (18 subsequent siblings)
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Anthony PERARD, Andrew Cooper, Wei Liu, Jan Beulich,
	Roger Pau Monné

The clang test for "asm()-s support .include." needs to be modified
because the symbolic link asm -> asm-x86 may not exist when the test
is runned. Since it's an x86 test, we don't need the link.

This will be an issue with the following patch "xen/build: have the
root Makefile generates the CFLAGS".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v3:
    - new patch
      (needed for "xen/build: have the root Makefile generates the CFLAGS")

 xen/arch/x86/Rules.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/Rules.mk b/xen/arch/x86/Rules.mk
index e69b8e697cc0..4b7ab784670c 100644
--- a/xen/arch/x86/Rules.mk
+++ b/xen/arch/x86/Rules.mk
@@ -26,7 +26,7 @@ $(call as-option-add,CFLAGS,CC,".L0: .L1: .skip (.L1 - .L0)",,\
                      -no-integrated-as)
 
 # Check whether clang asm()-s support .include.
-$(call as-option-add,CFLAGS,CC,".include \"asm/indirect_thunk_asm.h\"",,\
+$(call as-option-add,CFLAGS,CC,".include \"asm-x86/indirect_thunk_asm.h\"",,\
                      -no-integrated-as)
 
 # Check whether clang keeps .macro-s between asm()-s:
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 06/23] xen/build: Fix section-renaming of libfdt and libelf
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (4 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 05/23] xen/build: Allow to test clang .include without asm symlink Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-27  9:38   ` Jan Beulich
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y Anthony PERARD
                   ` (17 subsequent siblings)
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD

In common/libelf/Makefile, when SECTIONS gets defined
SPECIAL_DATA_SECTIONS doesn't exist, so only "text data" sections are
been renamed. This was different before 48115d14743e ("Move more
kernel decompression bits to .init.* sections").

Move SPECIAL_DATA_SECTIONS in Rules.mk before including "Makefile" to
fix this.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen/Rules.mk | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index 154269bfd96c..c7a067d25409 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -92,6 +92,12 @@ LDFLAGS += $(LDFLAGS_DIRECT)
 
 LDFLAGS += $(LDFLAGS-y)
 
+SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
+                                            $(foreach w,1 2 4, \
+                                                        rodata.str$(w).$(a)) \
+                                            rodata.cst$(a)) \
+                         $(foreach r,rel rel.ro,data.$(r).local)
+
 include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
 
 DEPS = .*.d
@@ -206,12 +212,6 @@ endif
 %.o: %.S Makefile
 	$(CC) $(AFLAGS) -c $< -o $@
 
-SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
-					    $(foreach w,1 2 4, \
-							rodata.str$(w).$(a)) \
-					    rodata.cst$(a)) \
-			 $(foreach r,rel rel.ro,data.$(r).local)
-
 $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o Makefile
 	$(OBJDUMP) -h $< | sed -n '/[0-9]/{s,00*,0,g;p;}' | while read idx name sz rest; do \
 		case "$$name" in \
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (5 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 06/23] xen/build: Fix section-renaming of libfdt and libelf Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-27  9:22   ` Roger Pau Monné
                     ` (2 more replies)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 08/23] xen/build: use $(clean) shorthand for clean targets Anthony PERARD
                   ` (16 subsequent siblings)
  23 siblings, 3 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Kevin Tian, Stefano Stabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper, Ian Jackson,
	Jan Beulich, Anthony PERARD, Daniel De Graaf, Volodymyr Babchuk,
	Roger Pau Monné

This is part of upgrading our build system and import more of Linux's
one.

In Linux, subdir-y in Makefiles is only used to descend into
subdirectory when there are no object to build, Xen doesn't have that
and all subdir have object to be included in the final binary.

To allow the new syntax, the "obj-y" and "subdir-*" calculation in
Rules.mk is changed and partially imported from Linux's Kbuild.

The command used to modify the Makefile was:
    sed -i -r 's#^subdir-(.*)#obj-\1/#;' **/Makefile

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v3:
    - no more tabs
    - reshuffle variable, and remove __subdir-y

 xen/Rules.mk                         | 19 ++++++++-----------
 xen/arch/arm/Makefile                | 14 +++++++-------
 xen/arch/arm/arm32/Makefile          |  2 +-
 xen/arch/arm/arm64/Makefile          |  2 +-
 xen/arch/x86/Makefile                | 18 +++++++++---------
 xen/arch/x86/acpi/Makefile           |  2 +-
 xen/arch/x86/cpu/Makefile            |  4 ++--
 xen/arch/x86/guest/Makefile          |  4 ++--
 xen/arch/x86/hvm/Makefile            |  6 +++---
 xen/arch/x86/mm/Makefile             |  4 ++--
 xen/arch/x86/x86_64/Makefile         |  2 +-
 xen/common/Makefile                  | 10 +++++-----
 xen/drivers/Makefile                 | 14 +++++++-------
 xen/drivers/acpi/Makefile            |  6 +++---
 xen/drivers/passthrough/Makefile     |  8 ++++----
 xen/drivers/passthrough/vtd/Makefile |  2 +-
 xen/lib/Makefile                     |  2 +-
 xen/xsm/Makefile                     |  2 +-
 xen/xsm/flask/Makefile               |  2 +-
 19 files changed, 60 insertions(+), 63 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index c7a067d25409..cc9c71bb1327 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -111,17 +111,14 @@ define gendep
 endef
 $(foreach o,$(filter-out %/,$(obj-y) $(obj-bin-y) $(extra-y)),$(eval $(call gendep,$(o))))
 
-# Ensure each subdirectory has exactly one trailing slash.
-subdir-n := $(patsubst %,%/,$(patsubst %/,%,$(subdir-n) $(subdir-)))
-subdir-y := $(patsubst %,%/,$(patsubst %/,%,$(subdir-y)))
-
-# Add explicitly declared subdirectories to the object lists.
-obj-y += $(patsubst %/,%/built_in.o,$(subdir-y))
-
-# Add implicitly declared subdirectories (in the object lists) to the
-# subdirectory list, and rewrite the object-list entry.
-subdir-y += $(filter %/,$(obj-y))
-obj-y    := $(patsubst %/,%/built-in.o,$(obj-y))
+# Handle objects in subdirs
+# ---------------------------------------------------------------------------
+# o if we encounter foo/ in $(obj-y), replace it by foo/built_in.o
+#   and add the directory to the list of dirs to descend into: $(subdir-y)
+subdir-y := $(subdir-y) $(filter %/, $(obj-y))
+obj-y    := $(patsubst %/, %/built_in.o, $(obj-y))
+
+subdir-n   := $(subdir-n) $(subdir-) $(filter %/, $(obj-n) $(obj-))
 
 subdir-all := $(subdir-y) $(subdir-n)
 
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 70f532e42a06..1044c2298a05 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -1,11 +1,11 @@
-subdir-$(CONFIG_ARM_32) += arm32
-subdir-$(CONFIG_ARM_64) += arm64
-subdir-$(CONFIG_ARM_64) += efi
-subdir-$(CONFIG_ACPI) += acpi
+obj-$(CONFIG_ARM_32) += arm32/
+obj-$(CONFIG_ARM_64) += arm64/
+obj-$(CONFIG_ARM_64) += efi/
+obj-$(CONFIG_ACPI) += acpi/
 ifneq ($(CONFIG_NO_PLAT),y)
-subdir-y += platforms
+obj-y += platforms/
 endif
-subdir-$(CONFIG_TEE) += tee
+obj-$(CONFIG_TEE) += tee/
 
 obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o
 obj-y += bootfdt.init.o
@@ -48,7 +48,7 @@ obj-y += sysctl.o
 obj-y += time.o
 obj-y += traps.o
 obj-y += vcpreg.o
-subdir-$(CONFIG_NEW_VGIC) += vgic
+obj-$(CONFIG_NEW_VGIC) += vgic/
 ifneq ($(CONFIG_NEW_VGIC),y)
 obj-y += gic-vgic.o
 obj-y += vgic.o
diff --git a/xen/arch/arm/arm32/Makefile b/xen/arch/arm/arm32/Makefile
index 0ac254f34714..539bbef298a7 100644
--- a/xen/arch/arm/arm32/Makefile
+++ b/xen/arch/arm/arm32/Makefile
@@ -1,4 +1,4 @@
-subdir-y += lib
+obj-y += lib/
 
 obj-$(EARLY_PRINTK) += debug.o
 obj-y += domctl.o
diff --git a/xen/arch/arm/arm64/Makefile b/xen/arch/arm/arm64/Makefile
index c4f3a28a0d0b..db8565b71a33 100644
--- a/xen/arch/arm/arm64/Makefile
+++ b/xen/arch/arm/arm64/Makefile
@@ -1,4 +1,4 @@
-subdir-y += lib
+obj-y += lib/
 
 obj-y += cache.o
 obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index bce5fdb3170f..ed709e2373ac 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -1,12 +1,12 @@
-subdir-y += acpi
-subdir-y += cpu
-subdir-y += genapic
-subdir-$(CONFIG_GUEST) += guest
-subdir-$(CONFIG_HVM) += hvm
-subdir-y += mm
-subdir-$(CONFIG_XENOPROF) += oprofile
-subdir-$(CONFIG_PV) += pv
-subdir-y += x86_64
+obj-y += acpi/
+obj-y += cpu/
+obj-y += genapic/
+obj-$(CONFIG_GUEST) += guest/
+obj-$(CONFIG_HVM) += hvm/
+obj-y += mm/
+obj-$(CONFIG_XENOPROF) += oprofile/
+obj-$(CONFIG_PV) += pv/
+obj-y += x86_64/
 
 alternative-y := alternative.init.o
 alternative-$(CONFIG_LIVEPATCH) :=
diff --git a/xen/arch/x86/acpi/Makefile b/xen/arch/x86/acpi/Makefile
index 27b4aa30b0ca..1b9e62571301 100644
--- a/xen/arch/x86/acpi/Makefile
+++ b/xen/arch/x86/acpi/Makefile
@@ -1,4 +1,4 @@
-subdir-y += cpufreq
+obj-y += cpufreq/
 
 obj-y += lib.o power.o suspend.o cpu_idle.o cpuidle_menu.o
 obj-bin-y += boot.init.o wakeup_prot.o
diff --git a/xen/arch/x86/cpu/Makefile b/xen/arch/x86/cpu/Makefile
index 466acc8b10e5..de983006a1b1 100644
--- a/xen/arch/x86/cpu/Makefile
+++ b/xen/arch/x86/cpu/Makefile
@@ -1,5 +1,5 @@
-subdir-y += mcheck
-subdir-y += mtrr
+obj-y += mcheck/
+obj-y += mtrr/
 
 obj-y += amd.o
 obj-y += centaur.o
diff --git a/xen/arch/x86/guest/Makefile b/xen/arch/x86/guest/Makefile
index f164196772e8..a1e370d69df8 100644
--- a/xen/arch/x86/guest/Makefile
+++ b/xen/arch/x86/guest/Makefile
@@ -1,4 +1,4 @@
 obj-y += hypervisor.o
 
-subdir-$(CONFIG_HYPERV_GUEST) += hyperv
-subdir-$(CONFIG_XEN_GUEST) += xen
+obj-$(CONFIG_HYPERV_GUEST) += hyperv/
+obj-$(CONFIG_XEN_GUEST) += xen/
diff --git a/xen/arch/x86/hvm/Makefile b/xen/arch/x86/hvm/Makefile
index 43e5f3a21f8b..346419154460 100644
--- a/xen/arch/x86/hvm/Makefile
+++ b/xen/arch/x86/hvm/Makefile
@@ -1,6 +1,6 @@
-subdir-y += svm
-subdir-y += vmx
-subdir-y += viridian
+obj-y += svm/
+obj-y += vmx/
+obj-y += viridian/
 
 obj-y += asid.o
 obj-y += dm.o
diff --git a/xen/arch/x86/mm/Makefile b/xen/arch/x86/mm/Makefile
index 5010a29d6cb0..d87dc0aa6eeb 100644
--- a/xen/arch/x86/mm/Makefile
+++ b/xen/arch/x86/mm/Makefile
@@ -1,5 +1,5 @@
-subdir-y += shadow
-subdir-$(CONFIG_HVM) += hap
+obj-y += shadow/
+obj-$(CONFIG_HVM) += hap/
 
 obj-$(CONFIG_HVM) += altp2m.o
 obj-$(CONFIG_HVM) += guest_walk_2.o guest_walk_3.o guest_walk_4.o
diff --git a/xen/arch/x86/x86_64/Makefile b/xen/arch/x86/x86_64/Makefile
index 4bfa1480eb7e..2bb1eb0a8131 100644
--- a/xen/arch/x86/x86_64/Makefile
+++ b/xen/arch/x86/x86_64/Makefile
@@ -1,4 +1,4 @@
-subdir-$(CONFIG_PV) += compat
+obj-$(CONFIG_PV) += compat/
 
 obj-bin-y += entry.o
 obj-y += traps.o
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 2abb8250b0f2..e8cde653708f 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -66,9 +66,9 @@ obj-$(CONFIG_COMPAT) += $(addprefix compat/,domain.o kernel.o memory.o multicall
 
 extra-y := symbols-dummy.o
 
-subdir-$(CONFIG_COVERAGE) += coverage
-subdir-y += sched
-subdir-$(CONFIG_UBSAN) += ubsan
+obj-$(CONFIG_COVERAGE) += coverage/
+obj-y += sched/
+obj-$(CONFIG_UBSAN) += ubsan/
 
-subdir-$(CONFIG_NEEDS_LIBELF) += libelf
-subdir-$(CONFIG_HAS_DEVICE_TREE) += libfdt
+obj-$(CONFIG_NEEDS_LIBELF) += libelf/
+obj-$(CONFIG_HAS_DEVICE_TREE) += libfdt/
diff --git a/xen/drivers/Makefile b/xen/drivers/Makefile
index 30bab3cfdb36..2a1ae8ad130a 100644
--- a/xen/drivers/Makefile
+++ b/xen/drivers/Makefile
@@ -1,7 +1,7 @@
-subdir-y += char
-subdir-$(CONFIG_HAS_CPUFREQ) += cpufreq
-subdir-$(CONFIG_HAS_PCI) += pci
-subdir-$(CONFIG_HAS_VPCI) += vpci
-subdir-$(CONFIG_HAS_PASSTHROUGH) += passthrough
-subdir-$(CONFIG_ACPI) += acpi
-subdir-$(CONFIG_VIDEO) += video
+obj-y += char/
+obj-$(CONFIG_HAS_CPUFREQ) += cpufreq/
+obj-$(CONFIG_HAS_PCI) += pci/
+obj-$(CONFIG_HAS_VPCI) += vpci/
+obj-$(CONFIG_HAS_PASSTHROUGH) += passthrough/
+obj-$(CONFIG_ACPI) += acpi/
+obj-$(CONFIG_VIDEO) += video/
diff --git a/xen/drivers/acpi/Makefile b/xen/drivers/acpi/Makefile
index 444b11d5839d..4f8e97228ee2 100644
--- a/xen/drivers/acpi/Makefile
+++ b/xen/drivers/acpi/Makefile
@@ -1,6 +1,6 @@
-subdir-y += tables
-subdir-y += utilities
-subdir-$(CONFIG_X86) += apei
+obj-y += tables/
+obj-y += utilities/
+obj-$(CONFIG_X86) += apei/
 
 obj-bin-y += tables.init.o
 obj-$(CONFIG_NUMA) += numa.o
diff --git a/xen/drivers/passthrough/Makefile b/xen/drivers/passthrough/Makefile
index d50ab188c83c..e973e16c7484 100644
--- a/xen/drivers/passthrough/Makefile
+++ b/xen/drivers/passthrough/Makefile
@@ -1,7 +1,7 @@
-subdir-$(CONFIG_X86) += vtd
-subdir-$(CONFIG_X86) += amd
-subdir-$(CONFIG_X86) += x86
-subdir-$(CONFIG_ARM) += arm
+obj-$(CONFIG_X86) += vtd/
+obj-$(CONFIG_X86) += amd/
+obj-$(CONFIG_X86) += x86/
+obj-$(CONFIG_ARM) += arm/
 
 obj-y += iommu.o
 obj-$(CONFIG_HAS_PCI) += pci.o
diff --git a/xen/drivers/passthrough/vtd/Makefile b/xen/drivers/passthrough/vtd/Makefile
index f302653858a0..fde7555fac07 100644
--- a/xen/drivers/passthrough/vtd/Makefile
+++ b/xen/drivers/passthrough/vtd/Makefile
@@ -1,4 +1,4 @@
-subdir-$(CONFIG_X86) += x86
+obj-$(CONFIG_X86) += x86/
 
 obj-y += iommu.o
 obj-y += dmar.o
diff --git a/xen/lib/Makefile b/xen/lib/Makefile
index dcdb75931378..7019ca00e8fd 100644
--- a/xen/lib/Makefile
+++ b/xen/lib/Makefile
@@ -1 +1 @@
-subdir-$(CONFIG_X86) += x86
+obj-$(CONFIG_X86) += x86/
diff --git a/xen/xsm/Makefile b/xen/xsm/Makefile
index e4d581e065f8..cf0a728f1c96 100644
--- a/xen/xsm/Makefile
+++ b/xen/xsm/Makefile
@@ -3,4 +3,4 @@ obj-$(CONFIG_XSM) += xsm_policy.o
 obj-$(CONFIG_XSM) += dummy.o
 obj-$(CONFIG_XSM_SILO) += silo.o
 
-subdir-$(CONFIG_XSM_FLASK) += flask
+obj-$(CONFIG_XSM_FLASK) += flask/
diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index 7c3f381287be..b1fd45421993 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -2,7 +2,7 @@ obj-y += avc.o
 obj-y += hooks.o
 obj-y += flask_op.o
 
-subdir-y += ss
+obj-y += ss/
 
 CFLAGS += -I./include
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 08/23] xen/build: use $(clean) shorthand for clean targets
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (6 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 09/23] xen/build: extract clean target from Rules.mk Anthony PERARD
                   ` (15 subsequent siblings)
  23 siblings, 0 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Anthony PERARD,
	Jan Beulich, Anthony PERARD

From: Anthony PERARD <anthony.perard@gmail.com>

Collect all the clean targets as we are going to modify it shortly.
Also, this is inspired by Linux's Kbuild.

"Kbuild.include" isn't included by "Makefile", but the "_clean" target
is only used by Rules.mk which include Kbuild.include.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
 xen/Makefile               | 16 ++++++++--------
 xen/Rules.mk               |  2 +-
 xen/scripts/Kbuild.include |  5 +++++
 3 files changed, 14 insertions(+), 9 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index 72bc89924622..65bd913cd133 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -120,14 +120,14 @@ _debug:
 .PHONY: _clean
 _clean: delete-unfresh-files
 	$(MAKE) -C tools clean
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C include clean
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C common clean
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C drivers clean
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C xsm clean
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C crypto clean
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/arm clean
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/x86 clean
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C test clean
+	$(MAKE) $(clean) include
+	$(MAKE) $(clean) common
+	$(MAKE) $(clean) drivers
+	$(MAKE) $(clean) xsm
+	$(MAKE) $(clean) crypto
+	$(MAKE) $(clean) arch/arm
+	$(MAKE) $(clean) arch/x86
+	$(MAKE) $(clean) test
 	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) clean
 	find . \( -name "*.o" -o -name ".*.d" -o -name ".*.d2" -o -name "*.gcno" \) -exec rm -f {} \;
 	rm -f include/asm $(TARGET) $(TARGET).gz $(TARGET).efi $(TARGET).efi.map $(TARGET)-syms $(TARGET)-syms.map *~ core
diff --git a/xen/Rules.mk b/xen/Rules.mk
index cc9c71bb1327..e3b19319b1f5 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -189,7 +189,7 @@ FORCE:
 clean:: $(addprefix _clean_, $(subdir-all))
 	rm -f *.o .*.o.tmp *~ core $(DEPS_RM)
 _clean_%/: FORCE
-	$(MAKE) -f $(BASEDIR)/Rules.mk -C $* clean
+	$(MAKE) $(clean) $*
 
 SRCPATH := $(patsubst $(BASEDIR)/%,%,$(CURDIR))
 
diff --git a/xen/scripts/Kbuild.include b/xen/scripts/Kbuild.include
index a5c462fd9777..2465cc4060c3 100644
--- a/xen/scripts/Kbuild.include
+++ b/xen/scripts/Kbuild.include
@@ -5,3 +5,8 @@
 # cc-ifversion
 # Usage:  EXTRA_CFLAGS += $(call cc-ifversion, -lt, 0402, -O1)
 cc-ifversion = $(shell [ $(CONFIG_GCC_VERSION)0 $(1) $(2)000 ] && echo $(3) || echo $(4))
+
+# Shorthand for $(MAKE) clean
+# Usage:
+# $(MAKE) $(clean) dir
+clean := -f $(BASEDIR)/Rules.mk clean -C
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 09/23] xen/build: extract clean target from Rules.mk
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (7 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 08/23] xen/build: use $(clean) shorthand for clean targets Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-03-04 14:13   ` Jan Beulich
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 10/23] xen/build: run targets csopes, tags, .. without Rules.mk Anthony PERARD
                   ` (14 subsequent siblings)
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Anthony PERARD,
	Jan Beulich, Anthony PERARD

From: Anthony PERARD <anthony.perard@gmail.com>

Most of the code executed by Rules.mk isn't necessary for the clean
target, especially not the CFLAGS. This patch makes running make clean
much faster.

The patch extract the clean target into a different Makefile,
Makefile.clean.

Since Makefile.clean, doesn't want to include Config.mk, we need to
define the variables DEPS_INCLUDE and DEPS in a place common to
Rules.mk and Makefile.clean, this is Kbuild.include. DEPS_RM is only
needed in Makefile.clean so can be defined there.

Even so Rules.mk includes Config.mk, it includes Kbuild.include after,
so the effective definition of DEPS_INCLUDE is "xen/" one and the
same one as used by Makefile.clean.

This is inspired by Kbuild, with Makefile.clean partially copied from
Linux v5.4.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v3:
    - rewrite of commit message
    - have only subdir-all, without intermediare variable subdir-y and
      subdir-n in Makefile.clean

 xen/Rules.mk               | 12 ------------
 xen/scripts/Kbuild.include |  7 ++++++-
 xen/scripts/Makefile.clean | 30 ++++++++++++++++++++++++++++++
 3 files changed, 36 insertions(+), 13 deletions(-)
 create mode 100644 xen/scripts/Makefile.clean

diff --git a/xen/Rules.mk b/xen/Rules.mk
index e3b19319b1f5..0c1a3ee5905d 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -100,8 +100,6 @@ SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
 
 include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
 
-DEPS = .*.d
-
 include Makefile
 
 define gendep
@@ -118,10 +116,6 @@ $(foreach o,$(filter-out %/,$(obj-y) $(obj-bin-y) $(extra-y)),$(eval $(call gend
 subdir-y := $(subdir-y) $(filter %/, $(obj-y))
 obj-y    := $(patsubst %/, %/built_in.o, $(obj-y))
 
-subdir-n   := $(subdir-n) $(subdir-) $(filter %/, $(obj-n) $(obj-))
-
-subdir-all := $(subdir-y) $(subdir-n)
-
 $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS += -DINIT_SECTIONS_ONLY
 
 ifeq ($(CONFIG_COVERAGE),y)
@@ -185,12 +179,6 @@ FORCE:
 %/built_in_bin.o: FORCE
 	$(MAKE) -f $(BASEDIR)/Rules.mk -C $* built_in_bin.o
 
-.PHONY: clean
-clean:: $(addprefix _clean_, $(subdir-all))
-	rm -f *.o .*.o.tmp *~ core $(DEPS_RM)
-_clean_%/: FORCE
-	$(MAKE) $(clean) $*
-
 SRCPATH := $(patsubst $(BASEDIR)/%,%,$(CURDIR))
 
 %.o: %.c Makefile
diff --git a/xen/scripts/Kbuild.include b/xen/scripts/Kbuild.include
index 2465cc4060c3..6a9b0c39da53 100644
--- a/xen/scripts/Kbuild.include
+++ b/xen/scripts/Kbuild.include
@@ -2,6 +2,11 @@
 ####
 # kbuild: Generic definitions
 
+###
+# dependencies
+DEPS = .*.d
+DEPS_INCLUDE = $(addsuffix .d2, $(basename $(wildcard $(DEPS))))
+
 # cc-ifversion
 # Usage:  EXTRA_CFLAGS += $(call cc-ifversion, -lt, 0402, -O1)
 cc-ifversion = $(shell [ $(CONFIG_GCC_VERSION)0 $(1) $(2)000 ] && echo $(3) || echo $(4))
@@ -9,4 +14,4 @@ cc-ifversion = $(shell [ $(CONFIG_GCC_VERSION)0 $(1) $(2)000 ] && echo $(3) || e
 # Shorthand for $(MAKE) clean
 # Usage:
 # $(MAKE) $(clean) dir
-clean := -f $(BASEDIR)/Rules.mk clean -C
+clean := -f $(BASEDIR)/scripts/Makefile.clean clean -C
diff --git a/xen/scripts/Makefile.clean b/xen/scripts/Makefile.clean
new file mode 100644
index 000000000000..53379e6102cc
--- /dev/null
+++ b/xen/scripts/Makefile.clean
@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: GPL-2.0
+# ==========================================================================
+# Cleaning up
+# ==========================================================================
+
+clean::
+
+include $(BASEDIR)/scripts/Kbuild.include
+
+include Makefile
+
+# Figure out what we need to clean from the various variables
+# ==========================================================================
+subdir-all := $(subdir-y) $(subdir-n) $(subdir-) \
+              $(filter %/, $(obj-y) $(obj-n) $(obj-))
+
+DEPS_RM = $(DEPS) $(DEPS_INCLUDE)
+.PHONY: clean
+clean:: $(addprefix _clean_, $(subdir-all))
+	rm -f *.o .*.o.tmp *~ core $(DEPS_RM)
+
+# Descending
+# ---------------------------------------------------------------------------
+
+_clean_%/: FORCE
+	$(MAKE) $(clean) $*
+
+# Force execution of pattern rules (for which PHONY cannot be directly used).
+.PHONY: FORCE
+FORCE:
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 10/23] xen/build: run targets csopes, tags, .. without Rules.mk
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (8 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 09/23] xen/build: extract clean target from Rules.mk Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-03-04 14:17   ` Jan Beulich
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 11/23] xen/build: make tests in test/ directly Anthony PERARD
                   ` (13 subsequent siblings)
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD

Those targets make use of $(all_sources) which depends on TARGET_ARCH,
so we just need to set TARGET_ARCH earlier and once.

XEN_TARGET_ARCH isn't expected to change during the build, so
TARGET_SUBARCH and TARGET_ARCH aren't going to change either. Set them
once and for all in the Xen root Makefile. This allows to run more
targets without Rules.mk.

XEN_TARGET_ARCH is actually changed in arch/x86/boot/build32.mk, but
it doesn't use the TARGET_{,SUB}ARCH variables either, and doesn't use
Rules.mk (it replaces it).

TARGET_{,SUB}ARCH are no longer overridden because that would have
no effect on the values that Rules.mk will use.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v3:
    - Improve commit message, try to explain why override disappeared

 xen/Makefile | 25 +++++++++++++++----------
 xen/Rules.mk |  5 -----
 2 files changed, 15 insertions(+), 15 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index 65bd913cd133..10bc4bf3646c 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -35,6 +35,11 @@ SRCARCH=$(shell echo $(ARCH) | sed -e 's/x86.*/x86/' -e s'/arm\(32\|64\)/arm/g')
 # we need XEN_TARGET_ARCH to generate the proper config
 include $(XEN_ROOT)/Config.mk
 
+# Set ARCH/SUBARCH appropriately.
+export TARGET_SUBARCH  := $(XEN_TARGET_ARCH)
+export TARGET_ARCH     := $(shell echo $(XEN_TARGET_ARCH) | \
+                            sed -e 's/x86.*/x86/' -e s'/arm\(32\|64\)/arm/g')
+
 # Allow someone to change their config file
 export KCONFIG_CONFIG ?= .config
 
@@ -46,8 +51,8 @@ dist: install
 
 build install:: include/config/auto.conf
 
-.PHONY: build install uninstall clean distclean cscope TAGS tags MAP gtags tests
-build install uninstall debug clean distclean cscope TAGS tags MAP gtags tests::
+.PHONY: build install uninstall clean distclean MAP tests
+build install uninstall debug clean distclean MAP tests::
 ifneq ($(XEN_TARGET_ARCH),x86_32)
 	$(MAKE) -f Rules.mk _$@
 else
@@ -223,25 +228,25 @@ endef
 xenversion:
 	@echo $(XEN_FULLVERSION)
 
-.PHONY: _TAGS
-_TAGS: 
+.PHONY: TAGS
+TAGS:
 	set -e; rm -f TAGS; \
 	$(call set_exuberant_flags,etags); \
 	$(all_sources) | xargs etags $$exuberant_flags -a
 
-.PHONY: _tags
-_tags: 
+.PHONY: tags
+tags:
 	set -e; rm -f tags; \
 	$(call set_exuberant_flags,ctags); \
 	$(all_sources) | xargs ctags $$exuberant_flags -a
 
-.PHONY: _gtags
-_gtags:
+.PHONY: gtags
+gtags:
 	set -e; rm -f GTAGS GSYMS GPATH GRTAGS
 	$(all_sources) | gtags -f -
 
-.PHONY: _cscope
-_cscope:
+.PHONY: cscope
+cscope:
 	$(all_sources) > cscope.files
 	cscope -k -b -q
 
diff --git a/xen/Rules.mk b/xen/Rules.mk
index 0c1a3ee5905d..92a13ca60163 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -27,11 +27,6 @@ ifneq ($(origin verbose),undefined)
 $(error "You must use 'make menuconfig' to enable/disable verbose now.")
 endif
 
-# Set ARCH/SUBARCH appropriately.
-override TARGET_SUBARCH  := $(XEN_TARGET_ARCH)
-override TARGET_ARCH     := $(shell echo $(XEN_TARGET_ARCH) | \
-                              sed -e 's/x86.*/x86/' -e s'/arm\(32\|64\)/arm/g')
-
 TARGET := $(BASEDIR)/xen
 
 # Note that link order matters!
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 11/23] xen/build: make tests in test/ directly
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (9 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 10/23] xen/build: run targets csopes, tags, .. without Rules.mk Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 12/23] xen/build: Move as-option-add to xen/ Anthony PERARD
                   ` (12 subsequent siblings)
  23 siblings, 0 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD

It is unnecessary to make _tests via Rules.mk because the target
use Rules.mk as well.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
 xen/Makefile | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index 10bc4bf3646c..8267ace51b0d 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -51,8 +51,8 @@ dist: install
 
 build install:: include/config/auto.conf
 
-.PHONY: build install uninstall clean distclean MAP tests
-build install uninstall debug clean distclean MAP tests::
+.PHONY: build install uninstall clean distclean MAP
+build install uninstall debug clean distclean MAP::
 ifneq ($(XEN_TARGET_ARCH),x86_32)
 	$(MAKE) -f Rules.mk _$@
 else
@@ -92,8 +92,8 @@ _install: $(TARGET)$(CONFIG_XEN_INSTALL_SUFFIX)
 		fi; \
 	fi
 
-.PHONY: _tests
-_tests:
+.PHONY: tests
+tests:
 	$(MAKE) -f $(BASEDIR)/Rules.mk -C test tests
 .PHONY: install-tests
 install-tests:
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 12/23] xen/build: Move as-option-add to xen/
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (10 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 11/23] xen/build: make tests in test/ directly Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-03-04 14:17   ` Jan Beulich
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 13/23] xen/build: include include/config/auto.conf in main Makefile Anthony PERARD
                   ` (11 subsequent siblings)
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD

Only xen/ uses as-option-add and as-insn, so there aren't needed in
Config.mk.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v3:
    - new patch

 Config.mk                  | 17 -----------------
 xen/scripts/Kbuild.include | 17 +++++++++++++++++
 2 files changed, 17 insertions(+), 17 deletions(-)

diff --git a/Config.mk b/Config.mk
index 65649d6122d1..dc6e7d03dffe 100644
--- a/Config.mk
+++ b/Config.mk
@@ -143,23 +143,6 @@ ifndef XEN_HAS_CHECKPOLICY
     export XEN_HAS_CHECKPOLICY
 endif
 
-# as-insn: Check whether assembler supports an instruction.
-# Usage: cflags-y += $(call as-insn,CC FLAGS,"insn",option-yes,option-no)
-as-insn = $(if $(shell echo 'void _(void) { asm volatile ( $(2) ); }' \
-                       | $(filter-out -M% %.d -include %/include/xen/config.h,$(1)) \
-                              -c -x c -o /dev/null - 2>&1),$(4),$(3))
-
-# as-option-add: Conditionally add options to flags
-# Usage: $(call as-option-add,CFLAGS,CC,"insn",option-yes,option-no)
-as-option-add = $(eval $(call as-option-add-closure,$(1),$(2),$(3),$(4),$(5)))
-define as-option-add-closure
-    ifeq ($$(call as-insn,$$($(2)) $$($(1)),$(3),y,n),y)
-        $(1) += $(4)
-    else
-        $(1) += $(5)
-    endif
-endef
-
 define buildmakevars2shellvars
     export PREFIX="$(prefix)";                                            \
     export XEN_SCRIPT_DIR="$(XEN_SCRIPT_DIR)";                            \
diff --git a/xen/scripts/Kbuild.include b/xen/scripts/Kbuild.include
index 6a9b0c39da53..806c68824ed5 100644
--- a/xen/scripts/Kbuild.include
+++ b/xen/scripts/Kbuild.include
@@ -7,6 +7,23 @@
 DEPS = .*.d
 DEPS_INCLUDE = $(addsuffix .d2, $(basename $(wildcard $(DEPS))))
 
+# as-insn: Check whether assembler supports an instruction.
+# Usage: cflags-y += $(call as-insn,CC FLAGS,"insn",option-yes,option-no)
+as-insn = $(if $(shell echo 'void _(void) { asm volatile ( $(2) ); }' \
+                       | $(filter-out -M% %.d -include %/include/xen/config.h,$(1)) \
+                              -c -x c -o /dev/null - 2>&1),$(4),$(3))
+
+# as-option-add: Conditionally add options to flags
+# Usage: $(call as-option-add,CFLAGS,CC,"insn",option-yes,option-no)
+as-option-add = $(eval $(call as-option-add-closure,$(1),$(2),$(3),$(4),$(5)))
+define as-option-add-closure
+    ifeq ($$(call as-insn,$$($(2)) $$($(1)),$(3),y,n),y)
+        $(1) += $(4)
+    else
+        $(1) += $(5)
+    endif
+endef
+
 # cc-ifversion
 # Usage:  EXTRA_CFLAGS += $(call cc-ifversion, -lt, 0402, -O1)
 cc-ifversion = $(shell [ $(CONFIG_GCC_VERSION)0 $(1) $(2)000 ] && echo $(3) || echo $(4))
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 13/23] xen/build: include include/config/auto.conf in main Makefile
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (11 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 12/23] xen/build: Move as-option-add to xen/ Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-03-04 14:29   ` Jan Beulich
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 14/23] xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS) Anthony PERARD
                   ` (10 subsequent siblings)
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD

We are going to generate the CFLAGS early from "xen/Makefile" instead
of in "Rules.mk", but we need to include "config/auto.conf", so
include it in "Makefile".

Before including "config/auto.conf" we check which make target a user
is calling, as some targets don't need "auto.conf". For targets that
needs auto.conf, make will generate it (and a default .config if
missing).

root-make-done is to avoid doing the calculation again once Rules.mk
takes over and is been executed with the root Makefile. When Rules.mk
is including xen/Makefile, `config-build' and `need-config' are
undefined so auto.conf will not be included again (it is already
included by Rules.mk) and kconfig target are out of reach of Rules.mk.

We are introducing a target %config to catch all targets for kconfig.
So we need an extra target %/.config to prevent make from trying to
regenerate $(XEN_ROOT)/.config that is included in Config.mk.

The way targets are filtered is inspired by Kbuild, with some code
imported from Linux. That's why there is PHONY variable that isn't
used yet, for example.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v3:
    - filter only for %config instead of both config %config
    - keep the multi-target pattern rule trick for include/config/auto.conf
      instead of using Linux's newer pattern (we dont have tristate.conf so
      don't need to change it)
    - use y/n for root-make-done, config-build, need-config instead of
      relying on ifdef and ifndef and on assigning an empty value meaning
      undef
    - use space for indentation
    - explain why %/.config is suddenly needed.

 xen/Makefile | 96 +++++++++++++++++++++++++++++++++++++++-------------
 1 file changed, 73 insertions(+), 23 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index 8267ace51b0d..a6120e577e9b 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -49,7 +49,71 @@ default: build
 .PHONY: dist
 dist: install
 
-build install:: include/config/auto.conf
+
+ifeq ($(root-make-done),)
+# section to run before calling Rules.mk, but only once.
+#
+# To make sure we do not include .config for any of the *config targets
+# catch them early, and hand them over to tools/kconfig/Makefile
+
+clean-targets := %clean
+no-dot-config-targets := $(clean-targets) \
+                         uninstall debug cloc \
+                         cscope TAGS tags MAP gtags \
+                         xenversion
+
+config-build    := n
+need-config     := y
+
+ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),)
+    ifeq ($(filter-out $(no-dot-config-targets), $(MAKECMDGOALS)),)
+        need-config := n
+    endif
+endif
+
+ifneq ($(filter %config,$(MAKECMDGOALS)),)
+    config-build := y
+endif
+
+export root-make-done := y
+endif # root-make-done
+
+ifeq ($(config-build),y)
+# ===========================================================================
+# *config targets only - make sure prerequisites are updated, and descend
+# in tools/kconfig to make the *config target
+
+config: FORCE
+	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" $@
+
+# Config.mk tries to include .config file, don't try to remake it
+%/.config: ;
+
+%config: FORCE
+	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" $@
+
+else # !config-build
+
+ifeq ($(need-config),y)
+include include/config/auto.conf
+# Read in dependencies to all Kconfig* files, make sure to run syncconfig if
+# changes are detected.
+include include/config/auto.conf.cmd
+
+# Allow people to just run `make` as before and not force them to configure
+$(KCONFIG_CONFIG):
+	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" defconfig
+
+# The actual configuration files used during the build are stored in
+# include/generated/ and include/config/. Update them if .config is newer than
+# include/config/auto.conf (which mirrors .config).
+#
+# This exploits the 'multi-target pattern rule' trick.
+# The syncconfig should be executed only once to make all the targets.
+include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
+	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" syncconfig
+
+endif # need-config
 
 .PHONY: build install uninstall clean distclean MAP
 build install uninstall debug clean distclean MAP::
@@ -254,9 +318,6 @@ cscope:
 _MAP:
 	$(NM) -n $(TARGET)-syms | grep -v '\(compiled\)\|\(\.o$$\)\|\( [aUw] \)\|\(\.\.ng$$\)\|\(LASH[RL]DI\)' > System.map
 
-.PHONY: FORCE
-FORCE:
-
 %.o %.i %.s: %.c FORCE
 	$(MAKE) -f $(BASEDIR)/Rules.mk -C $(*D) $(@F)
 
@@ -277,25 +338,6 @@ $(foreach base,arch/x86/mm/guest_walk_% \
                arch/x86/mm/shadow/guest_%, \
     $(foreach ext,o i s,$(call build-intermediate,$(base).$(ext))))
 
-kconfig := oldconfig config menuconfig defconfig \
-	nconfig xconfig gconfig savedefconfig listnewconfig olddefconfig \
-	randconfig $(notdir $(wildcard arch/$(SRCARCH)/configs/*_defconfig))
-.PHONY: $(kconfig)
-$(kconfig):
-	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" $@
-
-include/config/%.conf: include/config/auto.conf.cmd $(KCONFIG_CONFIG)
-	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" syncconfig
-
-# Allow people to just run `make` as before and not force them to configure
-$(KCONFIG_CONFIG):
-	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" defconfig
-
-# Break the dependency chain for the first run
-include/config/auto.conf.cmd: ;
-
--include $(BASEDIR)/include/config/auto.conf.cmd
-
 .PHONY: cloc
 cloc:
 	$(eval tmpfile := $(shell mktemp))
@@ -307,3 +349,11 @@ cloc:
 	cloc --list-file=$(tmpfile)
 	rm $(tmpfile)
 
+endif #config-build
+
+PHONY += FORCE
+FORCE:
+
+# Declare the contents of the PHONY variable as phony.  We keep that
+# information in a variable so we can use it in if_changed and friends.
+.PHONY: $(PHONY)
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 14/23] xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS)
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (12 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 13/23] xen/build: include include/config/auto.conf in main Makefile Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-27 10:22   ` Roger Pau Monné
  2020-03-04 14:42   ` Jan Beulich
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS Anthony PERARD
                   ` (9 subsequent siblings)
  23 siblings, 2 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Tim Deegan,
	Anthony PERARD, Jan Beulich, Anthony PERARD, Volodymyr Babchuk,
	Roger Pau Monné

From: Anthony PERARD <anthony.perard@gmail.com>

In a later patch ("xen/build: have the root Makefile generates the
CFLAGS), we want to generate the CFLAGS in xen/Makefile, then export
it and have Rules.mk use a CFLAGS from the environment variables. That
changes the flavor of the CFLAGS and flags intended for one target
(like -D__OBJECT_FILE__ and -M%) gets propagated and duplicated. So we
start by moving such flags out of $(CFLAGS) and into $(c_flags) which
is to be modified by only Rules.mk.

__OBJECT_FILE__ is only used by arch/x86/mm/*.c files, so having it in
$(c_flags) is enough, we don't need it in $(a_flags).

For include/Makefile and as-insn we can keep using CFLAGS, but since
it doesn't have -M* flags anymore there is no need to filter them out.

The XEN_BUILD_EFI tests in arch/x86/Makefile was filtering out
CFLAGS-y, but according to dd40177c1bc8 ("x86-64/EFI: add CFLAGS to
check compile"), it was done to filter out -MF. CFLAGS doesn't
have those flags anymore, so no filtering is needed.

This is inspired by the way Kbuild generates CFLAGS for each targets.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v3:
    - include/Makefile: Keep using CFLAGS, but since it doesn't have -M*
      flags anymore, no need to filter it.
    - Write c_flags and a_flags on a single line.
    - arch/x86/Makefile: remove the filter-out of dependency flags
      they are remove from CFLAGS anyway.
      (was intended to be done in xen/build: have the root Makefile
      generates the CFLAGS originally, move the change to this patch).
    - also modify as-insn as it is now xen/ only.

 xen/Rules.mk                    | 23 +++++++++++------------
 xen/arch/arm/Makefile           |  4 ++--
 xen/arch/x86/Makefile           |  6 +++---
 xen/arch/x86/mm/Makefile        |  6 +++---
 xen/arch/x86/mm/hap/Makefile    |  6 +++---
 xen/arch/x86/mm/shadow/Makefile |  6 +++---
 xen/include/Makefile            |  2 +-
 xen/scripts/Kbuild.include      |  2 +-
 8 files changed, 27 insertions(+), 28 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index 92a13ca60163..4aa119a90c27 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -57,7 +57,6 @@ CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
 $(call cc-option-add,CFLAGS,CC,-Wvla)
 CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
 CFLAGS-$(CONFIG_DEBUG_INFO) += -g
-CFLAGS += '-D__OBJECT_FILE__="$@"'
 
 ifneq ($(CONFIG_CC_IS_CLANG),y)
 # Clang doesn't understand this command line argument, and doesn't appear to
@@ -70,9 +69,6 @@ AFLAGS += -D__ASSEMBLY__
 
 ALL_OBJS := $(ALL_OBJS-y)
 
-# Get gcc to generate the dependencies for us.
-CFLAGS-y += -MMD -MF $(@D)/.$(@F).d
-
 CFLAGS += $(CFLAGS-y)
 # allow extra CFLAGS externally via EXTRA_CFLAGS_XEN_CORE
 CFLAGS += $(EXTRA_CFLAGS_XEN_CORE)
@@ -146,9 +142,12 @@ endif
 # Always build obj-bin files as binary even if they come from C source. 
 $(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))
 
+c_flags = -MMD -MF $(@D)/.$(@F).d $(CFLAGS) '-D__OBJECT_FILE__="$@"'
+a_flags = -MMD -MF $(@D)/.$(@F).d $(AFLAGS)
+
 built_in.o: $(obj-y) $(extra-y)
 ifeq ($(obj-y),)
-	$(CC) $(CFLAGS) -c -x c /dev/null -o $@
+	$(CC) $(c_flags) -c -x c /dev/null -o $@
 else
 ifeq ($(CONFIG_LTO),y)
 	$(LD_LTO) -r -o $@ $(filter-out $(extra-y),$^)
@@ -159,7 +158,7 @@ endif
 
 built_in_bin.o: $(obj-bin-y) $(extra-y)
 ifeq ($(obj-bin-y),)
-	$(CC) $(AFLAGS) -c -x assembler /dev/null -o $@
+	$(CC) $(a_flags) -c -x assembler /dev/null -o $@
 else
 	$(LD) $(LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
 endif
@@ -178,7 +177,7 @@ SRCPATH := $(patsubst $(BASEDIR)/%,%,$(CURDIR))
 
 %.o: %.c Makefile
 ifeq ($(CONFIG_ENFORCE_UNIQUE_SYMBOLS),y)
-	$(CC) $(CFLAGS) -c $< -o $(@D)/.$(@F).tmp -MQ $@
+	$(CC) $(c_flags) -c $< -o $(@D)/.$(@F).tmp -MQ $@
 ifeq ($(CONFIG_CC_IS_CLANG),y)
 	$(OBJCOPY) --redefine-sym $<=$(SRCPATH)/$< $(@D)/.$(@F).tmp $@
 else
@@ -186,11 +185,11 @@ else
 endif
 	rm -f $(@D)/.$(@F).tmp
 else
-	$(CC) $(CFLAGS) -c $< -o $@
+	$(CC) $(c_flags) -c $< -o $@
 endif
 
 %.o: %.S Makefile
-	$(CC) $(AFLAGS) -c $< -o $@
+	$(CC) $(a_flags) -c $< -o $@
 
 $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o Makefile
 	$(OBJDUMP) -h $< | sed -n '/[0-9]/{s,00*,0,g;p;}' | while read idx name sz rest; do \
@@ -205,12 +204,12 @@ $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o Makefile
 	$(OBJCOPY) $(foreach s,$(SPECIAL_DATA_SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
 
 %.i: %.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(CFLAGS)) $< -o $@
+	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) $< -o $@
 
 %.s: %.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(CFLAGS)) -S $< -o $@
+	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -S $< -o $@
 
 %.s: %.S Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(AFLAGS)) $< -o $@
+	$(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
 
 -include $(DEPS_INCLUDE)
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 1044c2298a05..7f1427630b96 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -121,10 +121,10 @@ $(TARGET)-syms: prelink.o xen.lds
 	rm -f $(@D)/.$(@F).[0-9]*
 
 asm-offsets.s: $(TARGET_SUBARCH)/asm-offsets.c
-	$(CC) $(filter-out -flto,$(CFLAGS)) -S -o $@ $<
+	$(CC) $(filter-out -flto,$(c_flags)) -S -o $@ $<
 
 xen.lds: xen.lds.S
-	$(CC) -P -E -Ui386 $(AFLAGS) -o $@ $<
+	$(CC) -P -E -Ui386 $(a_flags) -o $@ $<
 	sed -e 's/xen\.lds\.o:/xen\.lds:/g' <.xen.lds.d >.xen.lds.d.new
 	mv -f .xen.lds.d.new .xen.lds.d
 
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index ed709e2373ac..7fbac8ac525d 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -171,7 +171,7 @@ EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0
 EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0
 
 # Check if the compiler supports the MS ABI.
-export XEN_BUILD_EFI := $(shell $(CC) $(filter-out $(CFLAGS-y) .%.d,$(CFLAGS)) -c efi/check.c -o efi/check.o 2>/dev/null && echo y)
+export XEN_BUILD_EFI := $(shell $(CC) $(CFLAGS) -c efi/check.c -o efi/check.o 2>/dev/null && echo y)
 # Check if the linker supports PE.
 XEN_BUILD_PE := $(if $(XEN_BUILD_EFI),$(shell $(LD) -mi386pep --subsystem=10 -o efi/check.efi efi/check.o 2>/dev/null && echo y))
 CFLAGS-$(XEN_BUILD_EFI) += -DXEN_BUILD_EFI
@@ -226,7 +226,7 @@ efi/boot.init.o efi/runtime.o efi/compat.o efi/buildid.o efi/relocs-dummy.o: $(B
 efi/boot.init.o efi/runtime.o efi/compat.o efi/buildid.o efi/relocs-dummy.o: ;
 
 asm-offsets.s: $(TARGET_SUBARCH)/asm-offsets.c $(BASEDIR)/include/asm-x86/asm-macros.h
-	$(CC) $(filter-out -Wa$(comma)% -flto,$(CFLAGS)) -S -o $@ $<
+	$(CC) $(filter-out -Wa$(comma)% -flto,$(c_flags)) -S -o $@ $<
 
 asm-macros.i: CFLAGS += -D__ASSEMBLY__ -P
 
@@ -243,7 +243,7 @@ $(BASEDIR)/include/asm-x86/asm-macros.h: asm-macros.i Makefile
 
 efi.lds: AFLAGS += -DEFI
 xen.lds efi.lds: xen.lds.S
-	$(CC) -P -E -Ui386 $(filter-out -Wa$(comma)%,$(AFLAGS)) -o $@ $<
+	$(CC) -P -E -Ui386 $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<
 	sed -e 's/.*\.lds\.o:/$(@F):/g' <.$(@F).d >.$(@F).d.new
 	mv -f .$(@F).d.new .$(@F).d
 
diff --git a/xen/arch/x86/mm/Makefile b/xen/arch/x86/mm/Makefile
index d87dc0aa6eeb..a2431fde6bb4 100644
--- a/xen/arch/x86/mm/Makefile
+++ b/xen/arch/x86/mm/Makefile
@@ -12,10 +12,10 @@ obj-$(CONFIG_HVM) += p2m-ept.o p2m-pod.o
 obj-y += paging.o
 
 guest_walk_%.o: guest_walk.c Makefile
-	$(CC) $(CFLAGS) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
 
 guest_walk_%.i: guest_walk.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(CFLAGS)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
 
 guest_walk_%.s: guest_walk.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(CFLAGS)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
+	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
diff --git a/xen/arch/x86/mm/hap/Makefile b/xen/arch/x86/mm/hap/Makefile
index b14a9aff93d2..22e7ad54bd33 100644
--- a/xen/arch/x86/mm/hap/Makefile
+++ b/xen/arch/x86/mm/hap/Makefile
@@ -6,10 +6,10 @@ obj-y += nested_hap.o
 obj-y += nested_ept.o
 
 guest_walk_%level.o: guest_walk.c Makefile
-	$(CC) $(CFLAGS) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
 
 guest_walk_%level.i: guest_walk.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(CFLAGS)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
 
 guest_walk_%level.s: guest_walk.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(CFLAGS)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
+	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
diff --git a/xen/arch/x86/mm/shadow/Makefile b/xen/arch/x86/mm/shadow/Makefile
index ff03a9937f9b..23d3ff10802c 100644
--- a/xen/arch/x86/mm/shadow/Makefile
+++ b/xen/arch/x86/mm/shadow/Makefile
@@ -7,10 +7,10 @@ obj-y += none.o
 endif
 
 guest_%.o: multi.c Makefile
-	$(CC) $(CFLAGS) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
 
 guest_%.i: multi.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(CFLAGS)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
 
 guest_%.s: multi.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(CFLAGS)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
+	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
diff --git a/xen/include/Makefile b/xen/include/Makefile
index 433bad9055b2..a488a98d8bb7 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -64,7 +64,7 @@ compat/%.h: compat/%.i Makefile $(BASEDIR)/tools/compat-build-header.py
 	mv -f $@.new $@
 
 compat/%.i: compat/%.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)% -M% %.d -include %/include/xen/config.h,$(CFLAGS)) $(cppflags-y) -o $@ $<
+	$(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(CFLAGS)) $(cppflags-y) -o $@ $<
 
 compat/%.c: public/%.h xlat.lst Makefile $(BASEDIR)/tools/compat-build-source.py
 	mkdir -p $(@D)
diff --git a/xen/scripts/Kbuild.include b/xen/scripts/Kbuild.include
index 806c68824ed5..14bd4e110b45 100644
--- a/xen/scripts/Kbuild.include
+++ b/xen/scripts/Kbuild.include
@@ -10,7 +10,7 @@ DEPS_INCLUDE = $(addsuffix .d2, $(basename $(wildcard $(DEPS))))
 # as-insn: Check whether assembler supports an instruction.
 # Usage: cflags-y += $(call as-insn,CC FLAGS,"insn",option-yes,option-no)
 as-insn = $(if $(shell echo 'void _(void) { asm volatile ( $(2) ); }' \
-                       | $(filter-out -M% %.d -include %/include/xen/config.h,$(1)) \
+                       | $(filter-out -include %/include/xen/config.h,$(1)) \
                               -c -x c -o /dev/null - 2>&1),$(4),$(3))
 
 # as-option-add: Conditionally add options to flags
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (13 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 14/23] xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS) Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-27 11:05   ` Roger Pau Monné
  2020-03-04 15:00   ` Jan Beulich
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 16/23] xen/build: introduce if_changed and if_changed_rule Anthony PERARD
                   ` (8 subsequent siblings)
  23 siblings, 2 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD, Daniel De Graaf, Volodymyr Babchuk,
	Roger Pau Monné

Instead of generating the CFLAGS in Rules.mk everytime we enter a new
subdirectory, we are going to generate most of them a single time, and
export the result in the environment so that Rules.mk can use it.  The
only flags left to generates are the one that depends on the targets,
but the variable $(c_flags) takes care of that.

Arch specific CFLAGS are generated by a new file "arch/*/arch.mk"
which is included by the root Makefile.

We export the *FLAGS via the environment variables XEN_*FLAGS because
Rules.mk still includes Config.mk and would add duplicated flags to
CFLAGS.

When running Rules.mk in the root directory (xen/), the variable
`root-make-done' is set, so `need-config' will remain undef and so the
root Makefile will not generate the cflags again.

We can't use CFLAGS in subdirectories to add flags to particular
targets, instead start to use CFLAGS-y. Idem for AFLAGS.
So there are two different CFLAGS-y, the one in xen/Makefile (and
arch.mk), and the one in subdirs that Rules.mk is going to use.
We can't add to XEN_CFLAGS because it is exported, so making change to
it might be propagated to subdirectory which isn't intended.

Some style change are introduced in this patch:
    when LDFLAGS_DIRECT is included in LDFLAGS
    use of CFLAGS-$(CONFIG_INDIRECT_THUNK) instead of ifeq().

There is on FIXME added about LTO build, but since LTO is marked as
BROKEN, this commit doesn't attempt to filter -flto flags out of the
CFLAGS.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    v3:
    - squash "xen/build: introduce ccflags-y and CFLAGS_$@" here, with
      those changes:
        - rename ccflags-y to simply CFLAGS-y and start using AFLAGS-y in
          subdirs.
        - remove CFLAGS_$@, we don't need it yet.
        - fix build of xen.lds and efi.lds which needed -D to be a_flags
    - remove arch_ccflags, and modify c_flags directly
      with that change, reorder c_flags, so that target specific flags are last.
    - remove HAVE_AS_QUOTED_SYM from envvar and check XEN_CFLAGS to find if
      it's there when adding -D__OBJECT_LABEL__.
    - fix missing some flags in AFLAGS
      (like -fshort-wchar in xen/arch/x86/efi/Makefile,
       and -D__OBJECT_LABEL__ and CFLAGS-stack-boundary)
    - keep COV_FLAGS generation in Rules.mk since it doesn't invovle to
      call CC
    - fix clang test for "asm()-s support .include." (in a new patch done
      ahead)
    - include Kconfig.include in xen/Makefile because as-option-add is
      defined there now.

 xen/Makefile               | 60 ++++++++++++++++++++++++
 xen/Rules.mk               | 74 ++++++++----------------------
 xen/arch/arm/Makefile      | 10 ++--
 xen/arch/arm/Rules.mk      | 93 --------------------------------------
 xen/arch/arm/arch.mk       | 88 ++++++++++++++++++++++++++++++++++++
 xen/arch/arm/efi/Makefile  |  2 +-
 xen/arch/x86/Makefile      | 24 +++++-----
 xen/arch/x86/Rules.mk      | 91 +++----------------------------------
 xen/arch/x86/arch.mk       | 84 ++++++++++++++++++++++++++++++++++
 xen/arch/x86/efi/Makefile  |  2 +-
 xen/common/libelf/Makefile |  4 +-
 xen/common/libfdt/Makefile |  4 +-
 xen/include/Makefile       |  2 +-
 xen/xsm/flask/Makefile     |  2 +-
 xen/xsm/flask/ss/Makefile  |  2 +-
 15 files changed, 283 insertions(+), 259 deletions(-)
 create mode 100644 xen/arch/arm/arch.mk
 create mode 100644 xen/arch/x86/arch.mk

diff --git a/xen/Makefile b/xen/Makefile
index a6120e577e9b..da017dc29d36 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -94,6 +94,8 @@ config: FORCE
 
 else # !config-build
 
+include scripts/Kbuild.include
+
 ifeq ($(need-config),y)
 include include/config/auto.conf
 # Read in dependencies to all Kconfig* files, make sure to run syncconfig if
@@ -113,6 +115,64 @@ $(KCONFIG_CONFIG):
 include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
 	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" syncconfig
 
+ifeq ($(CONFIG_DEBUG),y)
+CFLAGS += -O1
+else
+CFLAGS += -O2
+endif
+
+ifeq ($(CONFIG_FRAME_POINTER),y)
+CFLAGS += -fno-omit-frame-pointer
+else
+CFLAGS += -fomit-frame-pointer
+endif
+
+CFLAGS += -nostdinc -fno-builtin -fno-common
+CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
+$(call cc-option-add,CFLAGS,CC,-Wvla)
+CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
+CFLAGS-$(CONFIG_DEBUG_INFO) += -g
+
+ifneq ($(CONFIG_CC_IS_CLANG),y)
+# Clang doesn't understand this command line argument, and doesn't appear to
+# have an suitable alternative.  The resulting compiled binary does function,
+# but has an excessively large symbol table.
+CFLAGS += -Wa,--strip-local-absolute
+endif
+
+AFLAGS += -D__ASSEMBLY__
+
+CFLAGS += $(CFLAGS-y)
+# allow extra CFLAGS externally via EXTRA_CFLAGS_XEN_CORE
+CFLAGS += $(EXTRA_CFLAGS_XEN_CORE)
+
+# Most CFLAGS are safe for assembly files:
+#  -std=gnu{89,99} gets confused by #-prefixed end-of-line comments
+#  -flto makes no sense and annoys clang
+AFLAGS += $(filter-out -std=gnu% -flto,$(CFLAGS))
+
+# LDFLAGS are only passed directly to $(LD)
+LDFLAGS += $(LDFLAGS_DIRECT) $(LDFLAGS-y)
+
+ifeq ($(CONFIG_UBSAN),y)
+CFLAGS_UBSAN := -fsanitize=undefined
+else
+CFLAGS_UBSAN :=
+endif
+
+ifeq ($(CONFIG_LTO),y)
+CFLAGS += -flto
+LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
+endif
+
+include $(BASEDIR)/arch/$(TARGET_ARCH)/arch.mk
+
+# define new variables to avoid the ones defines in Config.mk
+export XEN_CFLAGS := $(CFLAGS)
+export XEN_AFLAGS := $(AFLAGS)
+export XEN_LDFLAGS := $(LDFLAGS)
+export CFLAGS_UBSAN
+
 endif # need-config
 
 .PHONY: build install uninstall clean distclean MAP
diff --git a/xen/Rules.mk b/xen/Rules.mk
index 4aa119a90c27..f1311c45a372 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -38,59 +38,17 @@ ALL_OBJS-y               += $(BASEDIR)/arch/$(TARGET_ARCH)/built_in.o
 ALL_OBJS-$(CONFIG_CRYPTO)   += $(BASEDIR)/crypto/built_in.o
 
 # Initialise some variables
-CFLAGS_UBSAN :=
-
-ifeq ($(CONFIG_DEBUG),y)
-CFLAGS += -O1
-else
-CFLAGS += -O2
-endif
-
-ifeq ($(CONFIG_FRAME_POINTER),y)
-CFLAGS += -fno-omit-frame-pointer
-else
-CFLAGS += -fomit-frame-pointer
-endif
-
-CFLAGS += -nostdinc -fno-builtin -fno-common
-CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
-$(call cc-option-add,CFLAGS,CC,-Wvla)
-CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
-CFLAGS-$(CONFIG_DEBUG_INFO) += -g
-
-ifneq ($(CONFIG_CC_IS_CLANG),y)
-# Clang doesn't understand this command line argument, and doesn't appear to
-# have an suitable alternative.  The resulting compiled binary does function,
-# but has an excessively large symbol table.
-CFLAGS += -Wa,--strip-local-absolute
-endif
-
-AFLAGS += -D__ASSEMBLY__
+CFLAGS-y :=
+AFLAGS-y :=
 
 ALL_OBJS := $(ALL_OBJS-y)
 
-CFLAGS += $(CFLAGS-y)
-# allow extra CFLAGS externally via EXTRA_CFLAGS_XEN_CORE
-CFLAGS += $(EXTRA_CFLAGS_XEN_CORE)
-
-# Most CFLAGS are safe for assembly files:
-#  -std=gnu{89,99} gets confused by #-prefixed end-of-line comments
-#  -flto makes no sense and annoys clang
-AFLAGS += $(filter-out -std=gnu% -flto,$(CFLAGS))
-
-# LDFLAGS are only passed directly to $(LD)
-LDFLAGS += $(LDFLAGS_DIRECT)
-
-LDFLAGS += $(LDFLAGS-y)
-
 SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
                                             $(foreach w,1 2 4, \
                                                         rodata.str$(w).$(a)) \
                                             rodata.cst$(a)) \
                          $(foreach r,rel rel.ro,data.$(r).local)
 
-include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
-
 include Makefile
 
 define gendep
@@ -107,7 +65,7 @@ $(foreach o,$(filter-out %/,$(obj-y) $(obj-bin-y) $(extra-y)),$(eval $(call gend
 subdir-y := $(subdir-y) $(filter %/, $(obj-y))
 obj-y    := $(patsubst %/, %/built_in.o, $(obj-y))
 
-$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS += -DINIT_SECTIONS_ONLY
+$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS-y += -DINIT_SECTIONS_ONLY
 
 ifeq ($(CONFIG_COVERAGE),y)
 ifeq ($(CONFIG_CC_IS_CLANG),y)
@@ -115,19 +73,16 @@ ifeq ($(CONFIG_CC_IS_CLANG),y)
 else
     COV_FLAGS := -fprofile-arcs -ftest-coverage
 endif
-$(filter-out %.init.o $(nocov-y),$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS += $(COV_FLAGS)
+$(filter-out %.init.o $(nocov-y),$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS-y += $(COV_FLAGS)
 endif
 
 ifeq ($(CONFIG_UBSAN),y)
-CFLAGS_UBSAN += -fsanitize=undefined
 # Any -fno-sanitize= options need to come after any -fsanitize= options
 $(filter-out %.init.o $(noubsan-y),$(obj-y) $(obj-bin-y) $(extra-y)): \
-CFLAGS += $(filter-out -fno-%,$(CFLAGS_UBSAN)) $(filter -fno-%,$(CFLAGS_UBSAN))
+CFLAGS-y += $(filter-out -fno-%,$(CFLAGS_UBSAN)) $(filter -fno-%,$(CFLAGS_UBSAN))
 endif
 
 ifeq ($(CONFIG_LTO),y)
-CFLAGS += -flto
-LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
 # Would like to handle all object files as bitcode, but objects made from
 # pure asm are in a different format and have to be collected separately.
 # Mirror the directory tree, collecting them as built_in_bin.o.
@@ -140,10 +95,19 @@ obj-bin-y :=
 endif
 
 # Always build obj-bin files as binary even if they come from C source. 
-$(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))
+# FIXME LTO broken, but we would need a different way to filter -flto out
+# $(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))
+
+# Calculation of flags, first the generic flags, then the arch specific flags,
+# and last the flags modified for a target or a directory.
+
+c_flags = -MMD -MF $(@D)/.$(@F).d $(XEN_CFLAGS) '-D__OBJECT_FILE__="$@"'
+a_flags = -MMD -MF $(@D)/.$(@F).d $(XEN_AFLAGS)
+
+include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
 
-c_flags = -MMD -MF $(@D)/.$(@F).d $(CFLAGS) '-D__OBJECT_FILE__="$@"'
-a_flags = -MMD -MF $(@D)/.$(@F).d $(AFLAGS)
+c_flags += $(CFLAGS-y)
+a_flags += $(CFLAGS-y) $(AFLAGS-y)
 
 built_in.o: $(obj-y) $(extra-y)
 ifeq ($(obj-y),)
@@ -152,7 +116,7 @@ else
 ifeq ($(CONFIG_LTO),y)
 	$(LD_LTO) -r -o $@ $(filter-out $(extra-y),$^)
 else
-	$(LD) $(LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
 endif
 endif
 
@@ -160,7 +124,7 @@ built_in_bin.o: $(obj-bin-y) $(extra-y)
 ifeq ($(obj-bin-y),)
 	$(CC) $(a_flags) -c -x assembler /dev/null -o $@
 else
-	$(LD) $(LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
 endif
 
 # Force execution of pattern rules (for which PHONY cannot be directly used).
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 7f1427630b96..1599e2ba4058 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -96,24 +96,24 @@ prelink_lto.o: $(ALL_OBJS)
 
 # Link it with all the binary objects
 prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 else
 prelink.o: $(ALL_OBJS)
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 endif
 
 $(TARGET)-syms: prelink.o xen.lds
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o \
 	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).0.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort >$(@D)/.$(@F).1.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
 		| $(BASEDIR)/tools/symbols --xensyms --sysv --sort \
diff --git a/xen/arch/arm/Rules.mk b/xen/arch/arm/Rules.mk
index 022a3a6f82ba..e69de29bb2d1 100644
--- a/xen/arch/arm/Rules.mk
+++ b/xen/arch/arm/Rules.mk
@@ -1,93 +0,0 @@
-########################################
-# arm-specific definitions
-
-#
-# If you change any of these configuration options then you must
-# 'make clean' before rebuilding.
-#
-
-CFLAGS += -I$(BASEDIR)/include
-
-$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
-$(call cc-option-add,CFLAGS,CC,-Wnested-externs)
-
-# Prevent floating-point variables from creeping into Xen.
-CFLAGS-$(CONFIG_ARM_32) += -msoft-float
-CFLAGS-$(CONFIG_ARM_32) += -mcpu=cortex-a15
-
-CFLAGS-$(CONFIG_ARM_64) += -mcpu=generic
-CFLAGS-$(CONFIG_ARM_64) += -mgeneral-regs-only # No fp registers etc
-
-EARLY_PRINTK := n
-
-ifeq ($(CONFIG_DEBUG),y)
-
-# See docs/misc/arm/early-printk.txt for syntax
-
-EARLY_PRINTK_brcm           := 8250,0xF040AB00,2
-EARLY_PRINTK_dra7           := 8250,0x4806A000,2
-EARLY_PRINTK_fastmodel      := pl011,0x1c090000,115200
-EARLY_PRINTK_exynos5250     := exynos4210,0x12c20000
-EARLY_PRINTK_hikey960       := pl011,0xfff32000
-EARLY_PRINTK_juno           := pl011,0x7ff80000
-EARLY_PRINTK_lager          := scif,0xe6e60000
-EARLY_PRINTK_midway         := pl011,0xfff36000
-EARLY_PRINTK_mvebu          := mvebu,0xd0012000
-EARLY_PRINTK_omap5432       := 8250,0x48020000,2
-EARLY_PRINTK_rcar3          := scif,0xe6e88000
-EARLY_PRINTK_seattle        := pl011,0xe1010000
-EARLY_PRINTK_sun6i          := 8250,0x01c28000,2
-EARLY_PRINTK_sun7i          := 8250,0x01c28000,2
-EARLY_PRINTK_thunderx       := pl011,0x87e024000000
-EARLY_PRINTK_vexpress       := pl011,0x1c090000
-EARLY_PRINTK_xgene-mcdivitt := 8250,0x1c021000,2
-EARLY_PRINTK_xgene-storm    := 8250,0x1c020000,2
-EARLY_PRINTK_zynqmp         := cadence,0xff000000
-
-ifneq ($(EARLY_PRINTK_$(CONFIG_EARLY_PRINTK)),)
-EARLY_PRINTK_CFG := $(subst $(comma), ,$(EARLY_PRINTK_$(CONFIG_EARLY_PRINTK)))
-else
-EARLY_PRINTK_CFG := $(subst $(comma), ,$(CONFIG_EARLY_PRINTK))
-endif
-
-# Extract configuration from string
-EARLY_PRINTK_INC := $(word 1,$(EARLY_PRINTK_CFG))
-EARLY_UART_BASE_ADDRESS := $(word 2,$(EARLY_PRINTK_CFG))
-
-# UART specific options
-ifeq ($(EARLY_PRINTK_INC),8250)
-EARLY_UART_REG_SHIFT := $(word 3,$(EARLY_PRINTK_CFG))
-endif
-ifeq ($(EARLY_PRINTK_INC),pl011)
-ifneq ($(word 3,$(EARLY_PRINTK_CFG)),)
-EARLY_PRINTK_INIT_UART := y
-EARLY_PRINTK_BAUD := $(word 3,$(EARLY_PRINTK_CFG))
-endif
-endif
-ifeq ($(EARLY_PRINTK_INC),scif)
-ifneq ($(word 3,$(EARLY_PRINTK_CFG)),)
-CFLAGS-y += -DEARLY_PRINTK_VERSION_$(word 3,$(EARLY_PRINTK_CFG))
-else
-CFLAGS-y += -DEARLY_PRINTK_VERSION_NONE
-endif
-endif
-
-ifneq ($(EARLY_PRINTK_INC),)
-EARLY_PRINTK := y
-endif
-
-CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_PRINTK
-CFLAGS-$(EARLY_PRINTK_INIT_UART) += -DEARLY_PRINTK_INIT_UART
-CFLAGS-$(EARLY_PRINTK) += -DEARLY_PRINTK_INC=\"debug-$(EARLY_PRINTK_INC).inc\"
-CFLAGS-$(EARLY_PRINTK) += -DEARLY_PRINTK_BAUD=$(EARLY_PRINTK_BAUD)
-CFLAGS-$(EARLY_PRINTK) += -DEARLY_UART_BASE_ADDRESS=$(EARLY_UART_BASE_ADDRESS)
-CFLAGS-$(EARLY_PRINTK) += -DEARLY_UART_REG_SHIFT=$(EARLY_UART_REG_SHIFT)
-
-else # !CONFIG_DEBUG
-
-ifneq ($(CONFIG_EARLY_PRINTK),)
-# Early printk is dependant on a debug build.
-$(error CONFIG_EARLY_PRINTK enabled for non-debug build)
-endif
-
-endif
diff --git a/xen/arch/arm/arch.mk b/xen/arch/arm/arch.mk
new file mode 100644
index 000000000000..296d6c6cf526
--- /dev/null
+++ b/xen/arch/arm/arch.mk
@@ -0,0 +1,88 @@
+########################################
+# arm-specific definitions
+
+CFLAGS += -I$(BASEDIR)/include
+
+$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
+$(call cc-option-add,CFLAGS,CC,-Wnested-externs)
+
+# Prevent floating-point variables from creeping into Xen.
+CFLAGS-$(CONFIG_ARM_32) += -msoft-float
+CFLAGS-$(CONFIG_ARM_32) += -mcpu=cortex-a15
+
+CFLAGS-$(CONFIG_ARM_64) += -mcpu=generic
+CFLAGS-$(CONFIG_ARM_64) += -mgeneral-regs-only # No fp registers etc
+
+EARLY_PRINTK := n
+
+ifeq ($(CONFIG_DEBUG),y)
+
+# See docs/misc/arm/early-printk.txt for syntax
+
+EARLY_PRINTK_brcm           := 8250,0xF040AB00,2
+EARLY_PRINTK_dra7           := 8250,0x4806A000,2
+EARLY_PRINTK_fastmodel      := pl011,0x1c090000,115200
+EARLY_PRINTK_exynos5250     := exynos4210,0x12c20000
+EARLY_PRINTK_hikey960       := pl011,0xfff32000
+EARLY_PRINTK_juno           := pl011,0x7ff80000
+EARLY_PRINTK_lager          := scif,0xe6e60000
+EARLY_PRINTK_midway         := pl011,0xfff36000
+EARLY_PRINTK_mvebu          := mvebu,0xd0012000
+EARLY_PRINTK_omap5432       := 8250,0x48020000,2
+EARLY_PRINTK_rcar3          := scif,0xe6e88000
+EARLY_PRINTK_seattle        := pl011,0xe1010000
+EARLY_PRINTK_sun6i          := 8250,0x01c28000,2
+EARLY_PRINTK_sun7i          := 8250,0x01c28000,2
+EARLY_PRINTK_thunderx       := pl011,0x87e024000000
+EARLY_PRINTK_vexpress       := pl011,0x1c090000
+EARLY_PRINTK_xgene-mcdivitt := 8250,0x1c021000,2
+EARLY_PRINTK_xgene-storm    := 8250,0x1c020000,2
+EARLY_PRINTK_zynqmp         := cadence,0xff000000
+
+ifneq ($(EARLY_PRINTK_$(CONFIG_EARLY_PRINTK)),)
+EARLY_PRINTK_CFG := $(subst $(comma), ,$(EARLY_PRINTK_$(CONFIG_EARLY_PRINTK)))
+else
+EARLY_PRINTK_CFG := $(subst $(comma), ,$(CONFIG_EARLY_PRINTK))
+endif
+
+# Extract configuration from string
+EARLY_PRINTK_INC := $(word 1,$(EARLY_PRINTK_CFG))
+EARLY_UART_BASE_ADDRESS := $(word 2,$(EARLY_PRINTK_CFG))
+
+# UART specific options
+ifeq ($(EARLY_PRINTK_INC),8250)
+EARLY_UART_REG_SHIFT := $(word 3,$(EARLY_PRINTK_CFG))
+endif
+ifeq ($(EARLY_PRINTK_INC),pl011)
+ifneq ($(word 3,$(EARLY_PRINTK_CFG)),)
+EARLY_PRINTK_INIT_UART := y
+EARLY_PRINTK_BAUD := $(word 3,$(EARLY_PRINTK_CFG))
+endif
+endif
+ifeq ($(EARLY_PRINTK_INC),scif)
+ifneq ($(word 3,$(EARLY_PRINTK_CFG)),)
+CFLAGS-y += -DEARLY_PRINTK_VERSION_$(word 3,$(EARLY_PRINTK_CFG))
+else
+CFLAGS-y += -DEARLY_PRINTK_VERSION_NONE
+endif
+endif
+
+ifneq ($(EARLY_PRINTK_INC),)
+EARLY_PRINTK := y
+endif
+
+CFLAGS-$(EARLY_PRINTK) += -DCONFIG_EARLY_PRINTK
+CFLAGS-$(EARLY_PRINTK_INIT_UART) += -DEARLY_PRINTK_INIT_UART
+CFLAGS-$(EARLY_PRINTK) += -DEARLY_PRINTK_INC=\"debug-$(EARLY_PRINTK_INC).inc\"
+CFLAGS-$(EARLY_PRINTK) += -DEARLY_PRINTK_BAUD=$(EARLY_PRINTK_BAUD)
+CFLAGS-$(EARLY_PRINTK) += -DEARLY_UART_BASE_ADDRESS=$(EARLY_UART_BASE_ADDRESS)
+CFLAGS-$(EARLY_PRINTK) += -DEARLY_UART_REG_SHIFT=$(EARLY_UART_REG_SHIFT)
+
+else # !CONFIG_DEBUG
+
+ifneq ($(CONFIG_EARLY_PRINTK),)
+# Early printk is dependant on a debug build.
+$(error CONFIG_EARLY_PRINTK enabled for non-debug build)
+endif
+
+endif
diff --git a/xen/arch/arm/efi/Makefile b/xen/arch/arm/efi/Makefile
index d34c9168914a..e3ff2c3f283c 100644
--- a/xen/arch/arm/efi/Makefile
+++ b/xen/arch/arm/efi/Makefile
@@ -1,4 +1,4 @@
-CFLAGS += -fshort-wchar
+CFLAGS-y += -fshort-wchar
 
 obj-y +=  boot.init.o runtime.o
 obj-$(CONFIG_ACPI) +=  efi-dom0.init.o
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 7fbac8ac525d..5de873cf693e 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -124,32 +124,32 @@ prelink-efi_lto.o: $(ALL_OBJS) efi/runtime.o efi/compat.o
 
 # Link it with all the binary objects
 prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 
 prelink-efi.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink-efi_lto.o efi/boot.init.o
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 else
 prelink.o: $(ALL_OBJS)
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 
 prelink-efi.o: $(ALL_OBJS) efi/boot.init.o efi/runtime.o efi/compat.o
-	$(LD) $(LDFLAGS) -r -o $@ $(filter-out %/efi/built_in.o,$^)
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out %/efi/built_in.o,$^)
 endif
 
 $(TARGET)-syms: prelink.o xen.lds
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(BASEDIR)/common/symbols-dummy.o -o $(@D)/.$(@F).0
 	$(NM) -pa --format=sysv $(@D)/.$(@F).0 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort \
 		>$(@D)/.$(@F).0.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).0.o
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
 	$(NM) -pa --format=sysv $(@D)/.$(@F).1 \
 		| $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort $(syms-warn-dup-y) \
 		>$(@D)/.$(@F).1.S
 	$(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
-	$(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
+	$(LD) $(XEN_LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
 	    $(@D)/.$(@F).1.o -o $@
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
 		| $(BASEDIR)/tools/symbols --xensyms --sysv --sort \
@@ -162,7 +162,7 @@ note.o: $(TARGET)-syms
 		--rename-section=.data=.note.gnu.build-id -S $@.bin $@
 	rm -f $@.bin
 
-EFI_LDFLAGS = $(patsubst -m%,-mi386pep,$(LDFLAGS)) --subsystem=10
+EFI_LDFLAGS = $(patsubst -m%,-mi386pep,$(XEN_LDFLAGS)) --subsystem=10
 EFI_LDFLAGS += --image-base=$(1) --stack=0,0 --heap=0,0 --strip-debug
 EFI_LDFLAGS += --section-alignment=0x200000 --file-alignment=0x20
 EFI_LDFLAGS += --major-image-version=$(XEN_VERSION)
@@ -171,7 +171,7 @@ EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0
 EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0
 
 # Check if the compiler supports the MS ABI.
-export XEN_BUILD_EFI := $(shell $(CC) $(CFLAGS) -c efi/check.c -o efi/check.o 2>/dev/null && echo y)
+export XEN_BUILD_EFI := $(shell $(CC) $(XEN_CFLAGS) -c efi/check.c -o efi/check.o 2>/dev/null && echo y)
 # Check if the linker supports PE.
 XEN_BUILD_PE := $(if $(XEN_BUILD_EFI),$(shell $(LD) -mi386pep --subsystem=10 -o efi/check.efi efi/check.o 2>/dev/null && echo y))
 CFLAGS-$(XEN_BUILD_EFI) += -DXEN_BUILD_EFI
@@ -181,7 +181,7 @@ $(TARGET).efi: ALT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A ALT_
 
 ifneq ($(build_id_linker),)
 ifeq ($(call ld-ver-build-id,$(LD) $(filter -m%,$(EFI_LDFLAGS))),y)
-CFLAGS += -DBUILD_ID_EFI
+CFLAGS-y += -DBUILD_ID_EFI
 EFI_LDFLAGS += $(build_id_linker)
 note_file := efi/buildid.o
 # NB: this must be the last input in the linker call, because inputs following
@@ -228,7 +228,7 @@ efi/boot.init.o efi/runtime.o efi/compat.o efi/buildid.o efi/relocs-dummy.o: ;
 asm-offsets.s: $(TARGET_SUBARCH)/asm-offsets.c $(BASEDIR)/include/asm-x86/asm-macros.h
 	$(CC) $(filter-out -Wa$(comma)% -flto,$(c_flags)) -S -o $@ $<
 
-asm-macros.i: CFLAGS += -D__ASSEMBLY__ -P
+asm-macros.i: CFLAGS-y += -D__ASSEMBLY__ -P
 
 $(BASEDIR)/include/asm-x86/asm-macros.h: asm-macros.i Makefile
 	echo '#if 0' >$@.new
@@ -241,7 +241,7 @@ $(BASEDIR)/include/asm-x86/asm-macros.h: asm-macros.i Makefile
 	echo '#endif' >>$@.new
 	$(call move-if-changed,$@.new,$@)
 
-efi.lds: AFLAGS += -DEFI
+efi.lds: AFLAGS-y += -DEFI
 xen.lds efi.lds: xen.lds.S
 	$(CC) -P -E -Ui386 $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<
 	sed -e 's/.*\.lds\.o:/$(@F):/g' <.$(@F).d >.$(@F).d.new
diff --git a/xen/arch/x86/Rules.mk b/xen/arch/x86/Rules.mk
index 4b7ab784670c..56fe22c979ea 100644
--- a/xen/arch/x86/Rules.mk
+++ b/xen/arch/x86/Rules.mk
@@ -1,89 +1,10 @@
 ########################################
 # x86-specific definitions
 
-XEN_IMG_OFFSET := 0x200000
-
-CFLAGS += -I$(BASEDIR)/include
-CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-generic
-CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-default
-CFLAGS += -DXEN_IMG_OFFSET=$(XEN_IMG_OFFSET)
-CFLAGS += '-D__OBJECT_LABEL__=$(subst /,$$,$(subst -,_,$(subst $(BASEDIR)/,,$(CURDIR))/$@))'
-
-# Prevent floating-point variables from creeping into Xen.
-CFLAGS += -msoft-float
-
-ifeq ($(CONFIG_CC_IS_CLANG),y)
-# Note: Any test which adds -no-integrated-as will cause subsequent tests to
-# succeed, and not trigger further additions.
-#
-# The tests to select whether the integrated assembler is usable need to happen
-# before testing any assembler features, or else the result of the tests would
-# be stale if the integrated assembler is not used.
-
-# Older clang's built-in assembler doesn't understand .skip with labels:
-# https://bugs.llvm.org/show_bug.cgi?id=27369
-$(call as-option-add,CFLAGS,CC,".L0: .L1: .skip (.L1 - .L0)",,\
-                     -no-integrated-as)
-
-# Check whether clang asm()-s support .include.
-$(call as-option-add,CFLAGS,CC,".include \"asm-x86/indirect_thunk_asm.h\"",,\
-                     -no-integrated-as)
-
-# Check whether clang keeps .macro-s between asm()-s:
-# https://bugs.llvm.org/show_bug.cgi?id=36110
-$(call as-option-add,CFLAGS,CC,\
-                     ".macro FOO;.endm"$$(close); asm volatile $$(open)".macro FOO;.endm",\
-                     -no-integrated-as)
-endif
-
-$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
-$(call cc-option-add,CFLAGS,CC,-Wnested-externs)
-$(call as-option-add,CFLAGS,CC,"vmcall",-DHAVE_AS_VMX)
-$(call as-option-add,CFLAGS,CC,"crc32 %eax$$(comma)%eax",-DHAVE_AS_SSE4_2)
-$(call as-option-add,CFLAGS,CC,"invept (%rax)$$(comma)%rax",-DHAVE_AS_EPT)
-$(call as-option-add,CFLAGS,CC,"rdrand %eax",-DHAVE_AS_RDRAND)
-$(call as-option-add,CFLAGS,CC,"rdfsbase %rax",-DHAVE_AS_FSGSBASE)
-$(call as-option-add,CFLAGS,CC,"xsaveopt (%rax)",-DHAVE_AS_XSAVEOPT)
-$(call as-option-add,CFLAGS,CC,"rdseed %eax",-DHAVE_AS_RDSEED)
-$(call as-option-add,CFLAGS,CC,"clwb (%rax)",-DHAVE_AS_CLWB)
-$(call as-option-add,CFLAGS,CC,".equ \"x\"$$(comma)1", \
-                     -U__OBJECT_LABEL__ -DHAVE_AS_QUOTED_SYM \
-                     '-D__OBJECT_LABEL__=$(subst $(BASEDIR)/,,$(CURDIR))/$$@')
-$(call as-option-add,CFLAGS,CC,"invpcid (%rax)$$(comma)%rax",-DHAVE_AS_INVPCID)
-
-# GAS's idea of true is -1.  Clang's idea is 1
-$(call as-option-add,CFLAGS,CC,\
-    ".if ((1 > 0) < 0); .error \"\";.endif",,-DHAVE_AS_NEGATIVE_TRUE)
-
-# Check to see whether the assmbler supports the .nop directive.
-$(call as-option-add,CFLAGS,CC,\
-    ".L1: .L2: .nops (.L2 - .L1)$$(comma)9",-DHAVE_AS_NOPS_DIRECTIVE)
-
-CFLAGS += -mno-red-zone -fpic -fno-asynchronous-unwind-tables
-
-# Xen doesn't use SSE interally.  If the compiler supports it, also skip the
-# SSE setup for variadic function calls.
-CFLAGS += -mno-sse $(call cc-option,$(CC),-mskip-rax-setup)
-
-# Compile with thunk-extern, indirect-branch-register if avaiable.
-ifeq ($(CONFIG_INDIRECT_THUNK),y)
-CFLAGS += -mindirect-branch=thunk-extern -mindirect-branch-register
-CFLAGS += -fno-jump-tables
+ifneq ($(filter -DHAVE_AS_QUOTED_SYM,$(XEN_CFLAGS)),)
+object_label_flags = '-D__OBJECT_LABEL__=$(subst $(BASEDIR)/,,$(CURDIR))/$@'
+else
+object_label_flags = '-D__OBJECT_LABEL__=$(subst /,$$,$(subst -,_,$(subst $(BASEDIR)/,,$(CURDIR))/$@))'
 endif
-
-# If supported by the compiler, reduce stack alignment to 8 bytes. But allow
-# this to be overridden elsewhere.
-$(call cc-option-add,CFLAGS-stack-boundary,CC,-mpreferred-stack-boundary=3)
-CFLAGS += $(CFLAGS-stack-boundary)
-
-ifeq ($(CONFIG_UBSAN),y)
-# Don't enable alignment sanitisation.  x86 has efficient unaligned accesses,
-# and various things (ACPI tables, hypercall pages, stubs, etc) are wont-fix.
-# It also causes an as-yet-unidentified crash on native boot before the
-# console starts.
-$(call cc-option-add,CFLAGS_UBSAN,CC,-fno-sanitize=alignment)
-endif
-
-# Set up the assembler include path properly for older toolchains.
-CFLAGS += -Wa,-I$(BASEDIR)/include
-
+c_flags += $(object_label_flags) $(CFLAGS-stack-boundary)
+a_flags += $(object_label_flags) $(CFLAGS-stack-boundary)
diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
new file mode 100644
index 000000000000..2a51553edb3c
--- /dev/null
+++ b/xen/arch/x86/arch.mk
@@ -0,0 +1,84 @@
+########################################
+# x86-specific definitions
+
+export XEN_IMG_OFFSET := 0x200000
+
+CFLAGS += -I$(BASEDIR)/include
+CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-generic
+CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-default
+CFLAGS += -DXEN_IMG_OFFSET=$(XEN_IMG_OFFSET)
+
+# Prevent floating-point variables from creeping into Xen.
+CFLAGS += -msoft-float
+
+ifeq ($(CONFIG_CC_IS_CLANG),y)
+# Note: Any test which adds -no-integrated-as will cause subsequent tests to
+# succeed, and not trigger further additions.
+#
+# The tests to select whether the integrated assembler is usable need to happen
+# before testing any assembler features, or else the result of the tests would
+# be stale if the integrated assembler is not used.
+
+# Older clang's built-in assembler doesn't understand .skip with labels:
+# https://bugs.llvm.org/show_bug.cgi?id=27369
+$(call as-option-add,CFLAGS,CC,".L0: .L1: .skip (.L1 - .L0)",,\
+                     -no-integrated-as)
+
+# Check whether clang asm()-s support .include.
+$(call as-option-add,CFLAGS,CC,".include \"asm-x86/indirect_thunk_asm.h\"",,\
+                     -no-integrated-as)
+
+# Check whether clang keeps .macro-s between asm()-s:
+# https://bugs.llvm.org/show_bug.cgi?id=36110
+$(call as-option-add,CFLAGS,CC,\
+                     ".macro FOO;.endm"$$(close); asm volatile $$(open)".macro FOO;.endm",\
+                     -no-integrated-as)
+endif
+
+$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
+$(call cc-option-add,CFLAGS,CC,-Wnested-externs)
+$(call as-option-add,CFLAGS,CC,"vmcall",-DHAVE_AS_VMX)
+$(call as-option-add,CFLAGS,CC,"crc32 %eax$$(comma)%eax",-DHAVE_AS_SSE4_2)
+$(call as-option-add,CFLAGS,CC,"invept (%rax)$$(comma)%rax",-DHAVE_AS_EPT)
+$(call as-option-add,CFLAGS,CC,"rdrand %eax",-DHAVE_AS_RDRAND)
+$(call as-option-add,CFLAGS,CC,"rdfsbase %rax",-DHAVE_AS_FSGSBASE)
+$(call as-option-add,CFLAGS,CC,"xsaveopt (%rax)",-DHAVE_AS_XSAVEOPT)
+$(call as-option-add,CFLAGS,CC,"rdseed %eax",-DHAVE_AS_RDSEED)
+$(call as-option-add,CFLAGS,CC,"clwb (%rax)",-DHAVE_AS_CLWB)
+$(call as-option-add,CFLAGS,CC,".equ \"x\"$$(comma)1",-DHAVE_AS_QUOTED_SYM)
+$(call as-option-add,CFLAGS,CC,"invpcid (%rax)$$(comma)%rax",-DHAVE_AS_INVPCID)
+
+# GAS's idea of true is -1.  Clang's idea is 1
+$(call as-option-add,CFLAGS,CC,\
+    ".if ((1 > 0) < 0); .error \"\";.endif",,-DHAVE_AS_NEGATIVE_TRUE)
+
+# Check to see whether the assmbler supports the .nop directive.
+$(call as-option-add,CFLAGS,CC,\
+    ".L1: .L2: .nops (.L2 - .L1)$$(comma)9",-DHAVE_AS_NOPS_DIRECTIVE)
+
+CFLAGS += -mno-red-zone -fpic -fno-asynchronous-unwind-tables
+
+# Xen doesn't use SSE interally.  If the compiler supports it, also skip the
+# SSE setup for variadic function calls.
+CFLAGS += -mno-sse $(call cc-option,$(CC),-mskip-rax-setup)
+
+# Compile with thunk-extern, indirect-branch-register if avaiable.
+CFLAGS-$(CONFIG_INDIRECT_THUNK) += -mindirect-branch=thunk-extern
+CFLAGS-$(CONFIG_INDIRECT_THUNK) += -mindirect-branch-register
+CFLAGS-$(CONFIG_INDIRECT_THUNK) += -fno-jump-tables
+
+# If supported by the compiler, reduce stack alignment to 8 bytes. But allow
+# this to be overridden elsewhere.
+$(call cc-option-add,CFLAGS-stack-boundary,CC,-mpreferred-stack-boundary=3)
+export CFLAGS-stack-boundary
+
+ifeq ($(CONFIG_UBSAN),y)
+# Don't enable alignment sanitisation.  x86 has efficient unaligned accesses,
+# and various things (ACPI tables, hypercall pages, stubs, etc) are wont-fix.
+# It also causes an as-yet-unidentified crash on native boot before the
+# console starts.
+$(call cc-option-add,CFLAGS_UBSAN,CC,-fno-sanitize=alignment)
+endif
+
+# Set up the assembler include path properly for older toolchains.
+CFLAGS += -Wa,-I$(BASEDIR)/include
diff --git a/xen/arch/x86/efi/Makefile b/xen/arch/x86/efi/Makefile
index 4bc0a196e9ca..490d791aae2d 100644
--- a/xen/arch/x86/efi/Makefile
+++ b/xen/arch/x86/efi/Makefile
@@ -1,4 +1,4 @@
-CFLAGS += -fshort-wchar
+CFLAGS-y += -fshort-wchar
 
 %.o: %.ihex
 	$(OBJCOPY) -I ihex -O binary $< $@
diff --git a/xen/common/libelf/Makefile b/xen/common/libelf/Makefile
index 3d9e38f27e65..464c448d9d37 100644
--- a/xen/common/libelf/Makefile
+++ b/xen/common/libelf/Makefile
@@ -3,10 +3,10 @@ nocov-y += libelf.o
 
 SECTIONS := text data $(SPECIAL_DATA_SECTIONS)
 
-CFLAGS += -Wno-pointer-sign
+CFLAGS-y += -Wno-pointer-sign
 
 libelf.o: libelf-temp.o Makefile
 	$(OBJCOPY) $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
 
 libelf-temp.o: libelf-tools.o libelf-loader.o libelf-dominfo.o #libelf-relocate.o
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
diff --git a/xen/common/libfdt/Makefile b/xen/common/libfdt/Makefile
index c075bbf5462a..e2a5e59380a0 100644
--- a/xen/common/libfdt/Makefile
+++ b/xen/common/libfdt/Makefile
@@ -5,10 +5,10 @@ SECTIONS := text data $(SPECIAL_DATA_SECTIONS)
 obj-y += libfdt.o
 nocov-y += libfdt.o
 
-CFLAGS += -I$(BASEDIR)/include/xen/libfdt/
+CFLAGS-y += -I$(BASEDIR)/include/xen/libfdt/
 
 libfdt.o: libfdt-temp.o Makefile
 	$(OBJCOPY) $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
 
 libfdt-temp.o: $(LIBFDT_OBJS)
-	$(LD) $(LDFLAGS) -r -o $@ $^
+	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
diff --git a/xen/include/Makefile b/xen/include/Makefile
index a488a98d8bb7..2a10725d689b 100644
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -64,7 +64,7 @@ compat/%.h: compat/%.i Makefile $(BASEDIR)/tools/compat-build-header.py
 	mv -f $@.new $@
 
 compat/%.i: compat/%.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(CFLAGS)) $(cppflags-y) -o $@ $<
+	$(CPP) $(filter-out -Wa$(comma)% -include %/include/xen/config.h,$(XEN_CFLAGS)) $(cppflags-y) -o $@ $<
 
 compat/%.c: public/%.h xlat.lst Makefile $(BASEDIR)/tools/compat-build-source.py
 	mkdir -p $(@D)
diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index b1fd45421993..011ef5ca91f8 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -4,7 +4,7 @@ obj-y += flask_op.o
 
 obj-y += ss/
 
-CFLAGS += -I./include
+CFLAGS-y += -I./include
 
 AWK = awk
 
diff --git a/xen/xsm/flask/ss/Makefile b/xen/xsm/flask/ss/Makefile
index 046ce8f53326..d32b9e07138e 100644
--- a/xen/xsm/flask/ss/Makefile
+++ b/xen/xsm/flask/ss/Makefile
@@ -8,4 +8,4 @@ obj-y += services.o
 obj-y += conditional.o
 obj-y += mls.o
 
-CFLAGS += -I../include
+CFLAGS-y += -I../include
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 16/23] xen/build: introduce if_changed and if_changed_rule
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (14 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-03-04 15:45   ` Jan Beulich
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 17/23] xen/build: Start using if_changed Anthony PERARD
                   ` (7 subsequent siblings)
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD

The if_changed macro from Linux can record the command used to build a
target then compare it on rebuild. Thus if a command has changed, for
example due to introducing new flags in CFLAGS or due to using a
different compiler, the target will be rebuilt.

if_changed_rule checks dependencies like if_changed, but execute
rule_$(1) instead of cmd_$(1) when the command is different. A rule_
macro can call more than one cmd_ macro. One of the cmd_ macro in a
rule need to be call using a macro that record the command line, so
cmd_and_record is introduced. It is similar to cmd_and_fixup from
Linux but without a call to fixdep which we don't have yet. (We will
later replace cmd_and_record by cmd_and_fixup.)

Example of a rule_ macro:
define rule_cc_o_c
    $(call cmd_and_record,cc_o_o)
    $(call cmd,objcopy)
endef

This needs one of the call to use cmd_and_record, otherwise no .*.cmd
file will be created, and the target will keep been rebuilt.

In order for if_changed to works correctly, we need to load the .%.cmd
files that the macro generates, this is done by adding targets in to
the $(targets) variable. We use intermediate_targets to add %.init.o
dependency %.o to target since there aren't in obj-y.

We also add $(MAKECMDGOALS) to targets so that when running for
example `make common/memory.i`, make will load the associated .%.cmd
dependency file.

Beside the if_changed*, we import the machinery used for a "beautify
output". The important one is when running make with V=2 which help to
debug the makefiles by printing why a target is been rebuilt, via the
$(echo-why) macro.

if_changed and if_changed_rule aren't used yet.

Most of this code is copied from Linux v5.4.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 .gitignore                 |   1 +
 xen/Makefile               |  53 +++++++++++++++++-
 xen/Rules.mk               |  33 +++++++++++-
 xen/scripts/Kbuild.include | 107 +++++++++++++++++++++++++++++++++++++
 4 files changed, 192 insertions(+), 2 deletions(-)

diff --git a/.gitignore b/.gitignore
index 4ca679ddbc9a..c73f9f480780 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,6 +6,7 @@
 *.o
 *.d
 *.d2
+.*.cmd
 *.opic
 *.a
 *.so
diff --git a/xen/Makefile b/xen/Makefile
index da017dc29d36..fbd087e6f360 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -52,7 +52,57 @@ dist: install
 
 ifeq ($(root-make-done),)
 # section to run before calling Rules.mk, but only once.
+
+# Beautify output
+# ---------------------------------------------------------------------------
+#
+# Normally, we echo the whole command before executing it. By making
+# that echo $($(quiet)$(cmd)), we now have the possibility to set
+# $(quiet) to choose other forms of output instead, e.g.
+#
+#         quiet_cmd_cc_o_c = Compiling $(RELDIR)/$@
+#         cmd_cc_o_c       = $(CC) $(c_flags) -c -o $@ $<
+#
+# If $(quiet) is empty, the whole command will be printed.
+# If it is set to "quiet_", only the short version will be printed.
+# If it is set to "silent_", nothing will be printed at all, since
+# the variable $(silent_cmd_cc_o_c) doesn't exist.
+#
+# A simple variant is to prefix commands with $(Q) - that's useful
+# for commands that shall be hidden in non-verbose mode.
 #
+#	$(Q)ln $@ :<
+#
+# If KBUILD_VERBOSE equals 0 then the above command will be hidden.
+# If KBUILD_VERBOSE equals 1 then the above command is displayed.
+#
+# To put more focus on warnings, be less verbose as default
+# Use 'make V=1' to see the full commands
+
+ifeq ("$(origin V)", "command line")
+  KBUILD_VERBOSE = $(V)
+endif
+ifndef KBUILD_VERBOSE
+  KBUILD_VERBOSE = 0
+endif
+
+ifeq ($(KBUILD_VERBOSE),1)
+  quiet =
+  Q =
+else
+  quiet=quiet_
+  Q = @
+endif
+
+# If the user is running make -s (silent mode), suppress echoing of
+# commands
+
+ifneq ($(findstring s,$(filter-out --%,$(MAKEFLAGS))),)
+  quiet=silent_
+endif
+
+export quiet Q KBUILD_VERBOSE
+
 # To make sure we do not include .config for any of the *config targets
 # catch them early, and hand them over to tools/kconfig/Makefile
 
@@ -258,7 +308,8 @@ _clean: delete-unfresh-files
 	$(MAKE) $(clean) arch/x86
 	$(MAKE) $(clean) test
 	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) clean
-	find . \( -name "*.o" -o -name ".*.d" -o -name ".*.d2" -o -name "*.gcno" \) -exec rm -f {} \;
+	find . \( -name "*.o" -o -name ".*.d" -o -name ".*.d2" \
+		-o -name "*.gcno" -o -name ".*.cmd" \) -exec rm -f {} \;
 	rm -f include/asm $(TARGET) $(TARGET).gz $(TARGET).efi $(TARGET).efi.map $(TARGET)-syms $(TARGET)-syms.map *~ core
 	rm -f include/asm-*/asm-offsets.h
 	rm -f .banner
diff --git a/xen/Rules.mk b/xen/Rules.mk
index f1311c45a372..8807a2e21c94 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -38,6 +38,7 @@ ALL_OBJS-y               += $(BASEDIR)/arch/$(TARGET_ARCH)/built_in.o
 ALL_OBJS-$(CONFIG_CRYPTO)   += $(BASEDIR)/crypto/built_in.o
 
 # Initialise some variables
+targets :=
 CFLAGS-y :=
 AFLAGS-y :=
 
@@ -65,6 +66,10 @@ $(foreach o,$(filter-out %/,$(obj-y) $(obj-bin-y) $(extra-y)),$(eval $(call gend
 subdir-y := $(subdir-y) $(filter %/, $(obj-y))
 obj-y    := $(patsubst %/, %/built_in.o, $(obj-y))
 
+# $(subdir-obj-y) is the list of objects in $(obj-y) which uses dir/ to
+# tell kbuild to descend
+subdir-obj-y := $(filter %/built_in.o, $(obj-y))
+
 $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS-y += -DINIT_SECTIONS_ONLY
 
 ifeq ($(CONFIG_COVERAGE),y)
@@ -120,6 +125,10 @@ else
 endif
 endif
 
+targets += built_in.o
+targets += $(filter-out $(subdir-obj-y), $(obj-y)) $(extra-y)
+targets += $(MAKECMDGOALS)
+
 built_in_bin.o: $(obj-bin-y) $(extra-y)
 ifeq ($(obj-bin-y),)
 	$(CC) $(a_flags) -c -x assembler /dev/null -o $@
@@ -128,7 +137,7 @@ else
 endif
 
 # Force execution of pattern rules (for which PHONY cannot be directly used).
-.PHONY: FORCE
+PHONY += FORCE
 FORCE:
 
 %/built_in.o: FORCE
@@ -176,4 +185,26 @@ $(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o Makefile
 %.s: %.S Makefile
 	$(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
 
+# Add intermediate targets:
+# When building objects with specific suffix patterns, add intermediate
+# targets that the final targets are derived from.
+intermediate_targets = $(foreach sfx, $(2), \
+				$(patsubst %$(strip $(1)),%$(sfx), \
+					$(filter %$(strip $(1)), $(targets))))
+# %.init.o <- %.o
+targets += $(call intermediate_targets, .init.o, .o)
+
 -include $(DEPS_INCLUDE)
+
+# Read all saved command lines and dependencies for the $(targets) we
+# may be building above, using $(if_changed{,_dep}). As an
+# optimization, we don't need to read them if the target does not
+# exist, we will rebuild anyway in that case.
+
+existing-targets := $(wildcard $(sort $(targets)))
+
+-include $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).cmd)
+
+# Declare the contents of the PHONY variable as phony.  We keep that
+# information in a variable so we can use it in if_changed and friends.
+.PHONY: $(PHONY)
diff --git a/xen/scripts/Kbuild.include b/xen/scripts/Kbuild.include
index 14bd4e110b45..f24d664db5ff 100644
--- a/xen/scripts/Kbuild.include
+++ b/xen/scripts/Kbuild.include
@@ -2,11 +2,30 @@
 ####
 # kbuild: Generic definitions
 
+# Convenient variables
+squote  := '
+empty   :=
+space   := $(empty) $(empty)
+space_escape := _-_SPACE_-_
+pound := \#
+
+###
+# Name of target with a '.' as filename prefix. foo/bar.o => foo/.bar.o
+dot-target = $(@D)/.$(@F)
+
 ###
 # dependencies
 DEPS = .*.d
 DEPS_INCLUDE = $(addsuffix .d2, $(basename $(wildcard $(DEPS))))
 
+###
+# real prerequisites without phony targets
+real-prereqs = $(filter-out $(PHONY), $^)
+
+###
+# Escape single quote for use in echo statements
+escsq = $(subst $(squote),'\$(squote)',$1)
+
 # as-insn: Check whether assembler supports an instruction.
 # Usage: cflags-y += $(call as-insn,CC FLAGS,"insn",option-yes,option-no)
 as-insn = $(if $(shell echo 'void _(void) { asm volatile ( $(2) ); }' \
@@ -32,3 +51,91 @@ cc-ifversion = $(shell [ $(CONFIG_GCC_VERSION)0 $(1) $(2)000 ] && echo $(3) || e
 # Usage:
 # $(MAKE) $(clean) dir
 clean := -f $(BASEDIR)/scripts/Makefile.clean clean -C
+
+# echo command.
+# Short version is used, if $(quiet) equals `quiet_', otherwise full one.
+echo-cmd = $(if $($(quiet)cmd_$(1)),\
+        echo '  $(call escsq,$($(quiet)cmd_$(1)))$(echo-why)';)
+
+# printing commands
+cmd = @set -e; $(echo-cmd) $(cmd_$(1))
+
+###
+# if_changed      - execute command if any prerequisite is newer than
+#                   target, or command line has changed
+# if_changed_rule - as if_changed but execute rule instead
+
+ifneq ($(KBUILD_NOCMDDEP),1)
+# Check if both commands are the same including their order. Result is empty
+# string if equal. User may override this check using make KBUILD_NOCMDDEP=1
+cmd-check = $(filter-out $(subst $(space),$(space_escape),$(strip $(cmd_$@))), \
+                         $(subst $(space),$(space_escape),$(strip $(cmd_$1))))
+else
+cmd-check = $(if $(strip $(cmd_$@)),,1)
+endif
+
+# Replace >$< with >$$< to preserve $ when reloading the .cmd file
+# (needed for make)
+# Replace >#< with >$(pound)< to avoid starting a comment in the .cmd file
+# (needed for make)
+# Replace >'< with >'\''< to be able to enclose the whole string in '...'
+# (needed for the shell)
+make-cmd = $(call escsq,$(subst $(pound),$$(pound),$(subst $$,$$$$,$(cmd_$(1)))))
+
+# Find any prerequisites that is newer than target or that does not exist.
+# PHONY targets skipped in both cases.
+any-prereq = $(filter-out $(PHONY),$?)$(filter-out $(PHONY) $(wildcard $^),$^)
+
+# Execute command if command has changed or prerequisite(s) are updated.
+if_changed = $(if $(any-prereq)$(cmd-check),                                 \
+        $(cmd);                                                              \
+        printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:)
+
+# Usage: $(call if_changed_rule,foo)
+# Will check if $(cmd_foo) or any of the prerequisites changed,
+# and if so will execute $(rule_foo).
+if_changed_rule = $(if $(any-prereq)$(cmd-check),$(rule_$(1)),@:)
+
+cmd_and_record =                                                             \
+        $(cmd);                                                              \
+        printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd
+
+###
+# why - tell why a target got built
+#       enabled by make V=2
+#       Output (listed in the order they are checked):
+#          (1) - due to target is PHONY
+#          (2) - due to target missing
+#          (3) - due to: file1.h file2.h
+#          (4) - due to command line change
+#          (5) - due to missing .cmd file
+#          (6) - due to target not in $(targets)
+# (1) PHONY targets are always build
+# (2) No target, so we better build it
+# (3) Prerequisite is newer than target
+# (4) The command line stored in the file named dir/.target.cmd
+#     differed from actual command line. This happens when compiler
+#     options changes
+# (5) No dir/.target.cmd file (used to store command line)
+# (6) No dir/.target.cmd file and target not listed in $(targets)
+#     This is a good hint that there is a bug in the kbuild file
+ifeq ($(KBUILD_VERBOSE),2)
+why =                                                                        \
+    $(if $(filter $@, $(PHONY)),- due to target is PHONY,                    \
+        $(if $(wildcard $@),                                                 \
+            $(if $(any-prereq),- due to: $(any-prereq),                      \
+                $(if $(cmd-check),                                           \
+                    $(if $(cmd_$@),- due to command line change,             \
+                        $(if $(filter $@, $(targets)),                       \
+                            - due to missing .cmd file,                      \
+                            - due to $(notdir $@) not in $$(targets)         \
+                         )                                                   \
+                     )                                                       \
+                 )                                                           \
+             ),                                                              \
+             - due to target missing                                         \
+         )                                                                   \
+     )
+
+echo-why = $(call escsq, $(strip $(why)))
+endif
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 17/23] xen/build: Start using if_changed
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (15 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 16/23] xen/build: introduce if_changed and if_changed_rule Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-27 13:09   ` Roger Pau Monné
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 18/23] xen/build: use if_changed on built_in.o Anthony PERARD
                   ` (6 subsequent siblings)
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD, Daniel De Graaf, Volodymyr Babchuk,
	Roger Pau Monné

This patch start to use if_changed introduced in a previous commit.

Whenever if_changed is called, the target must have FORCE as
dependency so that if_changed can check if the command line to be
run as changed, so the macro $(real-prereqs) must be use to
discover the dependencies without "FORCE".

Whenever a target isn't in obj-y, it should be added to extra-y so the
.*.cmd dependency file associated with the target can be loaded. This
is done for xsm/flask/ and both common/lib{elf,fdt}/ and
arch/x86/Makefile.

For the targets that generates .*.d dependency files, there's going to
be two dependency files (.*.d and .*.cmd) until we can merge them
together in a later patch via fixdep from Linux.

One cleanup, libelf-relocate.o doesn't exist anymore.

We import cmd_ld and cmd_objcopy from Linux v5.4.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen/Rules.mk               | 68 +++++++++++++++++++++++++++-----------
 xen/arch/arm/Makefile      |  4 +--
 xen/arch/x86/Makefile      |  1 +
 xen/arch/x86/efi/Makefile  |  7 ++--
 xen/common/libelf/Makefile | 12 ++++---
 xen/common/libfdt/Makefile |  9 +++--
 xen/xsm/flask/Makefile     | 17 +++++++---
 7 files changed, 84 insertions(+), 34 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index 8807a2e21c94..bb4ced5f0dd4 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -52,6 +52,18 @@ SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
 
 include Makefile
 
+# Linking
+# ---------------------------------------------------------------------------
+
+quiet_cmd_ld = LD      $@
+cmd_ld = $(LD) $(XEN_LDFLAGS) -r -o $@ $(real-prereqs)
+
+# Objcopy
+# ---------------------------------------------------------------------------
+
+quiet_cmd_objcopy = OBJCOPY $@
+cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS) $< $@
+
 define gendep
     ifneq ($(1),$(subst /,:,$(1)))
         DEPS += $(dir $(1)).$(notdir $(1)).d
@@ -161,29 +173,47 @@ else
 	$(CC) $(c_flags) -c $< -o $@
 endif
 
-%.o: %.S Makefile
-	$(CC) $(a_flags) -c $< -o $@
+quiet_cmd_cc_o_S = CC      $@
+cmd_cc_o_S = $(CC) $(a_flags) -c $< -o $@
+
+%.o: %.S FORCE
+	$(call if_changed,cc_o_S)
+
+
+quiet_cmd_obj_init_o = INIT_O  $@
+define cmd_obj_init_o
+    $(OBJDUMP) -h $< | sed -n '/[0-9]/{s,00*,0,g;p;}' | while read idx name sz rest; do \
+        case "$$name" in \
+        .*.local) ;; \
+        .text|.text.*|.data|.data.*|.bss) \
+            test $$sz != 0 || continue; \
+            echo "Error: size of $<:$$name is 0x$$sz" >&2; \
+            exit $$(expr $$idx + 1);; \
+        esac; \
+    done; \
+    $(OBJCOPY) $(foreach s,$(SPECIAL_DATA_SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
+endef
+
+$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o FORCE
+	$(call if_changed,obj_init_o)
+
+quiet_cmd_cpp_i_c = CPP     $@
+cmd_cpp_i_c = $(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) $< -o $@
+
+quiet_cmd_cc_s_c = CC      $@
+cmd_cc_s_c = $(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -S $< -o $@
 
-$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): %.init.o: %.o Makefile
-	$(OBJDUMP) -h $< | sed -n '/[0-9]/{s,00*,0,g;p;}' | while read idx name sz rest; do \
-		case "$$name" in \
-		.*.local) ;; \
-		.text|.text.*|.data|.data.*|.bss) \
-			test $$sz != 0 || continue; \
-			echo "Error: size of $<:$$name is 0x$$sz" >&2; \
-			exit $$(expr $$idx + 1);; \
-		esac; \
-	done
-	$(OBJCOPY) $(foreach s,$(SPECIAL_DATA_SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
+quiet_cmd_s_S = CPP     $@
+cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
 
-%.i: %.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) $< -o $@
+%.i: %.c FORCE
+	$(call if_changed,cpp_i_c)
 
-%.s: %.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -S $< -o $@
+%.s: %.c FORCE
+	$(call if_changed,cc_s_c)
 
-%.s: %.S Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
+%.s: %.S FORCE
+	$(call if_changed,cpp_s_S)
 
 # Add intermediate targets:
 # When building objects with specific suffix patterns, add intermediate
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 1599e2ba4058..37ca6d25c08e 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -98,8 +98,8 @@ prelink_lto.o: $(ALL_OBJS)
 prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o
 	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 else
-prelink.o: $(ALL_OBJS)
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
+prelink.o: $(ALL_OBJS) FORCE
+	$(call if_changed,ld)
 endif
 
 $(TARGET)-syms: prelink.o xen.lds
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 5de873cf693e..6fb6cafdf47b 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -74,6 +74,7 @@ obj-$(CONFIG_TBOOT) += tboot.o
 obj-y += hpet.o
 obj-y += vm_event.o
 obj-y += xstate.o
+extra-y += asm-macros.i
 
 x86_emulate.o: x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h
 
diff --git a/xen/arch/x86/efi/Makefile b/xen/arch/x86/efi/Makefile
index 490d791aae2d..3e4c395b7535 100644
--- a/xen/arch/x86/efi/Makefile
+++ b/xen/arch/x86/efi/Makefile
@@ -1,7 +1,10 @@
 CFLAGS-y += -fshort-wchar
 
-%.o: %.ihex
-	$(OBJCOPY) -I ihex -O binary $< $@
+quiet_cmd_objcopy_o_ihex = OBJCOPY $@
+cmd_objcopy_o_ihex = $(OBJCOPY) -I ihex -O binary $< $@
+
+%.o: %.ihex FORCE
+	$(call if_changed,objcopy_o_ihex)
 
 boot.init.o: buildid.o
 
diff --git a/xen/common/libelf/Makefile b/xen/common/libelf/Makefile
index 464c448d9d37..a92326c982e9 100644
--- a/xen/common/libelf/Makefile
+++ b/xen/common/libelf/Makefile
@@ -1,12 +1,16 @@
 obj-bin-y := libelf.o
 nocov-y += libelf.o
+libelf-objs := libelf-tools.o libelf-loader.o libelf-dominfo.o
 
 SECTIONS := text data $(SPECIAL_DATA_SECTIONS)
+OBJCOPYFLAGS := $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s))
 
 CFLAGS-y += -Wno-pointer-sign
 
-libelf.o: libelf-temp.o Makefile
-	$(OBJCOPY) $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
+libelf.o: libelf-temp.o FORCE
+	$(call if_changed,objcopy)
 
-libelf-temp.o: libelf-tools.o libelf-loader.o libelf-dominfo.o #libelf-relocate.o
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
+libelf-temp.o: $(libelf-objs) FORCE
+	$(call if_changed,ld)
+
+extra-y += libelf-temp.o $(libelf-objs)
diff --git a/xen/common/libfdt/Makefile b/xen/common/libfdt/Makefile
index e2a5e59380a0..3efa5c5f8ed3 100644
--- a/xen/common/libfdt/Makefile
+++ b/xen/common/libfdt/Makefile
@@ -1,14 +1,17 @@
 include Makefile.libfdt
 
 SECTIONS := text data $(SPECIAL_DATA_SECTIONS)
+OBJCOPYFLAGS := $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s))
 
 obj-y += libfdt.o
 nocov-y += libfdt.o
 
 CFLAGS-y += -I$(BASEDIR)/include/xen/libfdt/
 
-libfdt.o: libfdt-temp.o Makefile
-	$(OBJCOPY) $(foreach s,$(SECTIONS),--rename-section .$(s)=.init.$(s)) $< $@
+libfdt.o: libfdt-temp.o FORCE
+	$(call if_changed,objcopy)
 
 libfdt-temp.o: $(LIBFDT_OBJS)
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
+	$(call if_changed,ld)
+
+extra-y += libfdt-temp.o $(LIBFDT_OBJS)
diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index 011ef5ca91f8..789e8fa1e3f9 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -20,12 +20,21 @@ AV_H_FILES = include/av_perm_to_string.h include/av_permissions.h
 ALL_H_FILES = $(FLASK_H_FILES) $(AV_H_FILES)
 
 $(obj-y) ss/built_in.o: $(ALL_H_FILES)
+extra-y += $(ALL_H_FILES)
 
-$(FLASK_H_FILES): $(FLASK_H_DEPEND)
-	$(CONFIG_SHELL) policy/mkflask.sh $(AWK) include $(FLASK_H_DEPEND)
+mkflask := policy/mkflask.sh
+quiet_cmd_mkflash = MKFLASH $@
+cmd_mkflash = $(CONFIG_SHELL) $(mkflask) $(AWK) include $(real-prereqs)
 
-$(AV_H_FILES): $(AV_H_DEPEND)
-	$(CONFIG_SHELL) policy/mkaccess_vector.sh $(AWK) $(AV_H_DEPEND)
+$(FLASK_H_FILES): $(FLASK_H_DEPEND) $(mkflash) FORCE
+	$(call if_changed,mkflash)
+
+mkaccess_vector := policy/mkaccess_vector.sh
+quiet_cmd_mkaccess_vector = MKACCESS VECTOR $@
+cmd_mkaccess_vector = $(CONFIG_SHELL) $(mkaccess_vector) $(AWK) $(real-prereqs)
+
+$(AV_H_FILES): $(AV_H_DEPEND) $(mkaccess_vector) FORCE
+	$(call if_changed,mkaccess_vector)
 
 obj-bin-$(CONFIG_XSM_FLASK_POLICY) += flask-policy.o
 flask-policy.o: policy.bin
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 18/23] xen/build: use if_changed on built_in.o
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (16 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 17/23] xen/build: Start using if_changed Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-03-04 16:03   ` Jan Beulich
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 19/23] xen/build: Use if_changed_rules with %.o:%.c targets Anthony PERARD
                   ` (5 subsequent siblings)
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD

In the case where $(obj-y) is empty, we also replace $(c_flags) by
$(XEN_CFLAGS) to avoid generating an .%.d dependency file. This avoid
make trying to include %.h file in the ld command if $(obj-y) isn't
empty anymore on a second run.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen/Rules.mk | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index bb4ced5f0dd4..cbf4feba0e0f 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -126,14 +126,21 @@ include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
 c_flags += $(CFLAGS-y)
 a_flags += $(CFLAGS-y) $(AFLAGS-y)
 
-built_in.o: $(obj-y) $(extra-y)
+quiet_cmd_ld_builtin = LD      $@
+cmd_ld_builtin = \
+    $(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
+quiet_cmd_cc_builtin = LD      $@
+cmd_cc_builtin = \
+    $(CC) $(XEN_CFLAGS) -c -x c /dev/null -o $@
+
+built_in.o: $(obj-y) $(extra-y) FORCE
 ifeq ($(obj-y),)
-	$(CC) $(c_flags) -c -x c /dev/null -o $@
+	$(call if_changed,cc_builtin)
 else
 ifeq ($(CONFIG_LTO),y)
 	$(LD_LTO) -r -o $@ $(filter-out $(extra-y),$^)
 else
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
+	$(call if_changed,ld_builtin)
 endif
 endif
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 19/23] xen/build: Use if_changed_rules with %.o:%.c targets
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (17 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 18/23] xen/build: use if_changed on built_in.o Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-03-04 16:09   ` Jan Beulich
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 20/23] xen/build: factorise generation of the linker scripts Anthony PERARD
                   ` (4 subsequent siblings)
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD

Use $(dot-target) to have the target name prefix with a dot.

Now, when the CC command has run, it is recorded in .*.cmd
file, then if_changed_rules will compare it on subsequent runs.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen/Rules.mk | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index cbf4feba0e0f..8c7dba9211d1 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -167,19 +167,27 @@ FORCE:
 
 SRCPATH := $(patsubst $(BASEDIR)/%,%,$(CURDIR))
 
-%.o: %.c Makefile
+quiet_cmd_cc_o_c = CC      $@
 ifeq ($(CONFIG_ENFORCE_UNIQUE_SYMBOLS),y)
-	$(CC) $(c_flags) -c $< -o $(@D)/.$(@F).tmp -MQ $@
-ifeq ($(CONFIG_CC_IS_CLANG),y)
-	$(OBJCOPY) --redefine-sym $<=$(SRCPATH)/$< $(@D)/.$(@F).tmp $@
-else
-	$(OBJCOPY) --redefine-sym $(<F)=$(SRCPATH)/$< $(@D)/.$(@F).tmp $@
-endif
-	rm -f $(@D)/.$(@F).tmp
+    cmd_cc_o_c = $(CC) $(c_flags) -c $< -o $(dot-target).tmp -MQ $@
+    ifeq ($(CONFIG_CC_IS_CLANG),y)
+        cmd_objcopy_fix_sym = $(OBJCOPY) --redefine-sym $<=$(SRCPATH)/$< $(dot-target).tmp $@
+    else
+        cmd_objcopy_fix_sym = $(OBJCOPY) --redefine-sym $(<F)=$(SRCPATH)/$< $(dot-target).tmp $@
+    endif
+    cmd_objcopy_fix_sym += && rm -f $(dot-target).tmp
 else
-	$(CC) $(c_flags) -c $< -o $@
+    cmd_cc_o_c = $(CC) $(c_flags) -c $< -o $@
 endif
 
+define rule_cc_o_c
+    $(call cmd_and_record,cc_o_c)
+    $(call cmd,objcopy_fix_sym)
+endef
+
+%.o: %.c FORCE
+	$(call if_changed_rule,cc_o_c)
+
 quiet_cmd_cc_o_S = CC      $@
 cmd_cc_o_S = $(CC) $(a_flags) -c $< -o $@
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 20/23] xen/build: factorise generation of the linker scripts
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (18 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 19/23] xen/build: Use if_changed_rules with %.o:%.c targets Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-27 13:14   ` Roger Pau Monné
  2020-03-05 11:05   ` Jan Beulich
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 21/23] xen/build: Use if_changed for prelink*.o Anthony PERARD
                   ` (3 subsequent siblings)
  23 siblings, 2 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD, Volodymyr Babchuk, Roger Pau Monné

In Arm and X86 makefile, generating the linker script is the same, so
we can simply have both call the same macro.

We need to add *.lds files into extra-y so that Rules.mk can find the
.*.cmd dependency file and load it.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen/Rules.mk          | 8 ++++++++
 xen/arch/arm/Makefile | 5 ++---
 xen/arch/x86/Makefile | 6 +++---
 3 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index 8c7dba9211d1..02cd37d04054 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -230,6 +230,14 @@ cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
 %.s: %.S FORCE
 	$(call if_changed,cpp_s_S)
 
+# Linker scripts, .lds.S -> .lds
+quiet_cmd_cc_lds_S = LDS     $@
+define cmd_cc_lds_S
+    $(CC) -P -E -Ui386 $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<; \
+    sed -e 's/.*\.lds\.o:/$(@F):/g' <$(dot-target).d >$(dot-target).d.new; \
+    mv -f $(dot-target).d.new $(dot-target).d
+endef
+
 # Add intermediate targets:
 # When building objects with specific suffix patterns, add intermediate
 # targets that the final targets are derived from.
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 37ca6d25c08e..b3ee4adb9ac4 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -124,9 +124,8 @@ asm-offsets.s: $(TARGET_SUBARCH)/asm-offsets.c
 	$(CC) $(filter-out -flto,$(c_flags)) -S -o $@ $<
 
 xen.lds: xen.lds.S
-	$(CC) -P -E -Ui386 $(a_flags) -o $@ $<
-	sed -e 's/xen\.lds\.o:/xen\.lds:/g' <.xen.lds.d >.xen.lds.d.new
-	mv -f .xen.lds.d.new .xen.lds.d
+	$(call if_changed,cc_lds_S)
+extra-y += xen.lds
 
 dtb.o: $(CONFIG_DTB_FILE)
 
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 6fb6cafdf47b..1be94846e11f 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -75,6 +75,7 @@ obj-y += hpet.o
 obj-y += vm_event.o
 obj-y += xstate.o
 extra-y += asm-macros.i
+extra-y += xen.lds
 
 x86_emulate.o: x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h
 
@@ -197,6 +198,7 @@ endif
 note_file_option ?= $(note_file)
 
 ifeq ($(XEN_BUILD_PE),y)
+extra-y += efi.lds
 $(TARGET).efi: prelink-efi.o $(note_file) efi.lds efi/relocs-dummy.o efi/mkreloc
 	$(foreach base, $(VIRT_BASE) $(ALT_BASE), \
 	          $(LD) $(call EFI_LDFLAGS,$(base)) -T efi.lds -N $< efi/relocs-dummy.o \
@@ -244,9 +246,7 @@ $(BASEDIR)/include/asm-x86/asm-macros.h: asm-macros.i Makefile
 
 efi.lds: AFLAGS-y += -DEFI
 xen.lds efi.lds: xen.lds.S
-	$(CC) -P -E -Ui386 $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<
-	sed -e 's/.*\.lds\.o:/$(@F):/g' <.$(@F).d >.$(@F).d.new
-	mv -f .$(@F).d.new .$(@F).d
+	$(call if_changed,cc_lds_S)
 
 boot/mkelf32: boot/mkelf32.c
 	$(HOSTCC) $(HOSTCFLAGS) -o $@ $<
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 21/23] xen/build: Use if_changed for prelink*.o
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (19 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 20/23] xen/build: factorise generation of the linker scripts Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-02-27 13:16   ` Roger Pau Monné
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 22/23] xen, symbols: rework file symbols selection Anthony PERARD
                   ` (2 subsequent siblings)
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Anthony PERARD, Andrew Cooper, Wei Liu, Jan Beulich,
	Roger Pau Monné

We change the dependencies of prelink-efi.o so that we can use the
same command line. The dependency on efi/built_in.o isn't needed
because, we already have:
    efi/*.o: efi/built_in.o
to build efi/*.o files that prelink-efi.o needs.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen/arch/x86/Makefile | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 1be94846e11f..55c6ae8ce0d2 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -131,11 +131,11 @@ prelink.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink_lto.o
 prelink-efi.o: $(patsubst %/built_in.o,%/built_in_bin.o,$(ALL_OBJS)) prelink-efi_lto.o efi/boot.init.o
 	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
 else
-prelink.o: $(ALL_OBJS)
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $^
+prelink.o: $(ALL_OBJS) FORCE
+	$(call if_changed,ld)
 
-prelink-efi.o: $(ALL_OBJS) efi/boot.init.o efi/runtime.o efi/compat.o
-	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out %/efi/built_in.o,$^)
+prelink-efi.o: $(filter-out %/efi/built_in.o,$(ALL_OBJS)) efi/boot.init.o efi/runtime.o efi/compat.o FORCE
+	$(call if_changed,ld)
 endif
 
 $(TARGET)-syms: prelink.o xen.lds
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 22/23] xen, symbols: rework file symbols selection
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (20 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 21/23] xen/build: Use if_changed for prelink*.o Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-03-05 14:44   ` Jan Beulich
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 23/23] xen/build: use if_changed to build guest_%.o Anthony PERARD
  2020-02-27 21:17 ` [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Stewart Hildebrand
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	Anthony PERARD

Rework symbols so it prefer file symbols that names an object file to
file symbols that have a directory component.

But have symbols still prefer the first file symbol if one of the above
is true, or prefer the second file symbols if it names a source file
without directory component.

In a future patch, we are going want to run $(CC) from the root directory
(xen.git/xen that is). So the guest_walk_%.o files are going to have
two file symbols, one with a directory component and another one
which name an object file. We still want to prefer the file symbols
that names an object file, no mater if it is first or second.

And before running everything from the root directory, we will be able
to use the same runes to build the guest_%.o as to build any other %.o
files from %.c files (the rule with the objcopy --redefine-sym).

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen/tools/symbols.c | 22 +++++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)

diff --git a/xen/tools/symbols.c b/xen/tools/symbols.c
index 9f9e2c990061..b7a00b4be487 100644
--- a/xen/tools/symbols.c
+++ b/xen/tools/symbols.c
@@ -80,11 +80,17 @@ static inline int is_arm_mapping_symbol(const char *str)
 	       && (str[2] == '\0' || str[2] == '.');
 }
 
+enum symbol_type {
+     symbol = 0,
+     single_source = 1,
+     dir_source = 2,
+     obj_source = 3,
+};
 static int read_symbol(FILE *in, struct sym_entry *s)
 {
 	char str[500], type[20] = "";
 	char *sym, stype;
-	static enum { symbol, single_source, multi_source } last;
+	static enum symbol_type last;
 	static char *filename;
 	int rc = -1;
 
@@ -125,13 +131,19 @@ static int read_symbol(FILE *in, struct sym_entry *s)
 		 * prefer the first one if that names an object file or has a
 		 * directory component (to cover multiply compiled files).
 		 */
-		bool multi = strchr(str, '/') || (sym && sym[1] == 'o');
-
-		if (multi || last != multi_source) {
+		enum symbol_type current;
+		if (sym && sym[1] == 'o')
+		    current = obj_source;
+		else if (strchr(str, '/'))
+		    current = dir_source;
+		else
+		    current = single_source;
+
+		if (current > last || last == single_source) {
 			free(filename);
 			filename = *str ? strdup(str) : NULL;
+			last = current;
 		}
-		last = multi ? multi_source : single_source;
 		goto skip_tail;
 	}
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Xen-devel] [XEN PATCH v3 23/23] xen/build: use if_changed to build guest_%.o
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (21 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 22/23] xen, symbols: rework file symbols selection Anthony PERARD
@ 2020-02-26 11:33 ` Anthony PERARD
  2020-03-05 15:12   ` Jan Beulich
  2020-02-27 21:17 ` [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Stewart Hildebrand
  23 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-02-26 11:33 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Tim Deegan,
	Jan Beulich, Anthony PERARD, Roger Pau Monné

Use if_changed for building all guest_%.o objects, and make use of
command that already exist.

This patch also introduces CFLAGS_$@, it is used so that flags are
applied to all .o .i and .s targets.

This patch make a change to the way guest_%.o files are built, and now
run the same commands that enforce unique symbols. But with patch
"xen,symbols: rework file symbols selection", symbols should still
select the file symbols directive intended to be used for guest_%.o
objects.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen/Rules.mk                    |  5 ++++-
 xen/arch/x86/mm/Makefile        | 15 +++++++++------
 xen/arch/x86/mm/hap/Makefile    | 15 +++++++++------
 xen/arch/x86/mm/shadow/Makefile | 15 +++++++++------
 4 files changed, 31 insertions(+), 19 deletions(-)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index 02cd37d04054..1ffb02f42914 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -115,6 +115,9 @@ endif
 # FIXME LTO broken, but we would need a different way to filter -flto out
 # $(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))
 
+# target with its suffix stripped
+target-stem = $(basename $@)
+
 # Calculation of flags, first the generic flags, then the arch specific flags,
 # and last the flags modified for a target or a directory.
 
@@ -123,7 +126,7 @@ a_flags = -MMD -MF $(@D)/.$(@F).d $(XEN_AFLAGS)
 
 include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
 
-c_flags += $(CFLAGS-y)
+c_flags += $(CFLAGS-y) $(CFLAGS_$(target-stem).o)
 a_flags += $(CFLAGS-y) $(AFLAGS-y)
 
 quiet_cmd_ld_builtin = LD      $@
diff --git a/xen/arch/x86/mm/Makefile b/xen/arch/x86/mm/Makefile
index a2431fde6bb4..4750bfa0ff91 100644
--- a/xen/arch/x86/mm/Makefile
+++ b/xen/arch/x86/mm/Makefile
@@ -11,11 +11,14 @@ obj-y += p2m.o p2m-pt.o
 obj-$(CONFIG_HVM) += p2m-ept.o p2m-pod.o
 obj-y += paging.o
 
-guest_walk_%.o: guest_walk.c Makefile
-	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+$(foreach gw,$(filter guest_walk_%.o,$(obj-y)),\
+    $(eval CFLAGS_$(gw) = -DGUEST_PAGING_LEVELS=$$*))
 
-guest_walk_%.i: guest_walk.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+guest_walk_%.o: guest_walk.c FORCE
+	$(call if_changed_rule,cc_o_c)
 
-guest_walk_%.s: guest_walk.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
+guest_walk_%.i: guest_walk.c FORCE
+	$(call if_changed,cpp_i_c)
+
+guest_walk_%.s: guest_walk.c FORCE
+	$(call if_changed,cc_s_c)
diff --git a/xen/arch/x86/mm/hap/Makefile b/xen/arch/x86/mm/hap/Makefile
index 22e7ad54bd33..8cd31e9cdc5e 100644
--- a/xen/arch/x86/mm/hap/Makefile
+++ b/xen/arch/x86/mm/hap/Makefile
@@ -5,11 +5,14 @@ obj-y += guest_walk_4level.o
 obj-y += nested_hap.o
 obj-y += nested_ept.o
 
-guest_walk_%level.o: guest_walk.c Makefile
-	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+$(foreach gw,$(filter guest_walk_%level.o,$(obj-y)),\
+    $(eval CFLAGS_$(gw) = -DGUEST_PAGING_LEVELS=$$*))
 
-guest_walk_%level.i: guest_walk.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+guest_walk_%level.o: guest_walk.c FORCE
+	$(call if_changed_rule,cc_o_c)
 
-guest_walk_%level.s: guest_walk.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
+guest_walk_%level.i: guest_walk.c FORCE
+	$(call if_changed,cpp_i_c)
+
+guest_walk_%level.s: guest_walk.c FORCE
+	$(call if_changed,cc_s_c)
diff --git a/xen/arch/x86/mm/shadow/Makefile b/xen/arch/x86/mm/shadow/Makefile
index 23d3ff10802c..d11e9e2fac08 100644
--- a/xen/arch/x86/mm/shadow/Makefile
+++ b/xen/arch/x86/mm/shadow/Makefile
@@ -6,11 +6,14 @@ else
 obj-y += none.o
 endif
 
-guest_%.o: multi.c Makefile
-	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+$(foreach gw,$(filter guest_%.o,$(obj-y)),\
+    $(eval CFLAGS_$(gw) = -DGUEST_PAGING_LEVELS=$$*))
 
-guest_%.i: multi.c Makefile
-	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
+guest_%.o: multi.c FORCE
+	$(call if_changed_rule,cc_o_c)
 
-guest_%.s: multi.c Makefile
-	$(CC) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -S $< -o $@
+guest_%.i: multi.c FORCE
+	$(call if_changed,cpp_i_c)
+
+guest_%.s: multi.c FORCE
+	$(call if_changed,cc_s_c)
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 03/23] xen/build: Remove confusing comment on the %.s:%.S rule
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 03/23] xen/build: Remove confusing comment on the %.s:%.S rule Anthony PERARD
@ 2020-02-26 11:53   ` Wei Liu
  0 siblings, 0 replies; 76+ messages in thread
From: Wei Liu @ 2020-02-26 11:53 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	xen-devel

On Wed, Feb 26, 2020 at 11:33:35AM +0000, Anthony PERARD wrote:
> That comment was introduce by 3943db776371 ("[XEN] Can be built
> -std=gnu99 (except for .S files).") to explain why CFLAGS was removed
> from the command line. The comment is already written where the
> -std=gnu flags gets remove from AFLAGS, no need to repeat it.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Wei Liu <wl@xen.org>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 04/23] xen/build: remove use of AFLAGS-y
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 04/23] xen/build: remove use of AFLAGS-y Anthony PERARD
@ 2020-02-26 12:58   ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-02-26 12:58 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On 26.02.2020 12:33, Anthony PERARD wrote:
> And simply add directly to AFLAGS.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 05/23] xen/build: Allow to test clang .include without asm symlink
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 05/23] xen/build: Allow to test clang .include without asm symlink Anthony PERARD
@ 2020-02-27  9:05   ` Roger Pau Monné
  2020-02-27  9:22     ` Jan Beulich
  0 siblings, 1 reply; 76+ messages in thread
From: Roger Pau Monné @ 2020-02-27  9:05 UTC (permalink / raw)
  To: Anthony PERARD; +Cc: xen-devel, Wei Liu, Jan Beulich, Andrew Cooper

On Wed, Feb 26, 2020 at 11:33:37AM +0000, Anthony PERARD wrote:
> The clang test for "asm()-s support .include." needs to be modified
> because the symbolic link asm -> asm-x86 may not exist when the test
> is runned. Since it's an x86 test, we don't need the link.
> 
> This will be an issue with the following patch "xen/build: have the
> root Makefile generates the CFLAGS".
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

So this is just a preparatory change in order to be able to run the
checks before the headers are linked, but the current usage is
perfectly fine AFAICT? (might be worth to try to clarify the commit
message a bit in this regard).

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 05/23] xen/build: Allow to test clang .include without asm symlink
  2020-02-27  9:05   ` Roger Pau Monné
@ 2020-02-27  9:22     ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-02-27  9:22 UTC (permalink / raw)
  To: Roger Pau Monné, Anthony PERARD; +Cc: xen-devel, Wei Liu, Andrew Cooper

On 27.02.2020 10:05, Roger Pau Monné wrote:
> On Wed, Feb 26, 2020 at 11:33:37AM +0000, Anthony PERARD wrote:
>> The clang test for "asm()-s support .include." needs to be modified
>> because the symbolic link asm -> asm-x86 may not exist when the test
>> is runned. Since it's an x86 test, we don't need the link.
>>
>> This will be an issue with the following patch "xen/build: have the
>> root Makefile generates the CFLAGS".
>>
>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

> So this is just a preparatory change in order to be able to run the
> checks before the headers are linked, but the current usage is
> perfectly fine AFAICT? (might be worth to try to clarify the commit
> message a bit in this regard).

To be honest to me the description looks clear enough in this
regard. But of course if improvements get suggested before
this actually gets committed ...

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y Anthony PERARD
@ 2020-02-27  9:22   ` Roger Pau Monné
  2020-02-27  9:43   ` Jan Beulich
  2020-03-05  9:24   ` Jan Beulich
  2 siblings, 0 replies; 76+ messages in thread
From: Roger Pau Monné @ 2020-02-27  9:22 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Kevin Tian, Stefano Stabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper, Ian Jackson,
	Jan Beulich, xen-devel, Daniel De Graaf, Volodymyr Babchuk

On Wed, Feb 26, 2020 at 11:33:39AM +0000, Anthony PERARD wrote:
> This is part of upgrading our build system and import more of Linux's
> one.
> 
> In Linux, subdir-y in Makefiles is only used to descend into
> subdirectory when there are no object to build, Xen doesn't have that
> and all subdir have object to be included in the final binary.
> 
> To allow the new syntax, the "obj-y" and "subdir-*" calculation in
> Rules.mk is changed and partially imported from Linux's Kbuild.
> 
> The command used to modify the Makefile was:
>     sed -i -r 's#^subdir-(.*)#obj-\1/#;' **/Makefile
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

LGTM:

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 06/23] xen/build: Fix section-renaming of libfdt and libelf
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 06/23] xen/build: Fix section-renaming of libfdt and libelf Anthony PERARD
@ 2020-02-27  9:38   ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-02-27  9:38 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On 26.02.2020 12:33, Anthony PERARD wrote:
> In common/libelf/Makefile, when SECTIONS gets defined
> SPECIAL_DATA_SECTIONS doesn't exist, so only "text data" sections are
> been renamed. This was different before 48115d14743e ("Move more
> kernel decompression bits to .init.* sections").

Obviously I can't have checked the generated binary back then.

Why is libfdt mentioned in just the title though?

> Move SPECIAL_DATA_SECTIONS in Rules.mk before including "Makefile" to
> fix this.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
(preferably with a sentence added to the description making clear
how/why that one's also affected; could be as simple as "The same
mistake has then been propagate into libfdt, by way of commit ...",
which ought to be easy enough to add while committing.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y Anthony PERARD
  2020-02-27  9:22   ` Roger Pau Monné
@ 2020-02-27  9:43   ` Jan Beulich
  2020-03-04 14:14     ` Jan Beulich
  2020-03-05  9:24   ` Jan Beulich
  2 siblings, 1 reply; 76+ messages in thread
From: Jan Beulich @ 2020-02-27  9:43 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Kevin Tian, Stefano Stabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper, Ian Jackson,
	xen-devel, Daniel De Graaf, Volodymyr Babchuk,
	Roger Pau Monné

On 26.02.2020 12:33, Anthony PERARD wrote:
> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -111,17 +111,14 @@ define gendep
>  endef
>  $(foreach o,$(filter-out %/,$(obj-y) $(obj-bin-y) $(extra-y)),$(eval $(call gendep,$(o))))
>  
> -# Ensure each subdirectory has exactly one trailing slash.
> -subdir-n := $(patsubst %,%/,$(patsubst %/,%,$(subdir-n) $(subdir-)))
> -subdir-y := $(patsubst %,%/,$(patsubst %/,%,$(subdir-y)))
> -
> -# Add explicitly declared subdirectories to the object lists.
> -obj-y += $(patsubst %/,%/built_in.o,$(subdir-y))
> -
> -# Add implicitly declared subdirectories (in the object lists) to the
> -# subdirectory list, and rewrite the object-list entry.
> -subdir-y += $(filter %/,$(obj-y))
> -obj-y    := $(patsubst %/,%/built-in.o,$(obj-y))
> +# Handle objects in subdirs
> +# ---------------------------------------------------------------------------
> +# o if we encounter foo/ in $(obj-y), replace it by foo/built_in.o
> +#   and add the directory to the list of dirs to descend into: $(subdir-y)
> +subdir-y := $(subdir-y) $(filter %/, $(obj-y))
> +obj-y    := $(patsubst %/, %/built_in.o, $(obj-y))
> +
> +subdir-n   := $(subdir-n) $(subdir-) $(filter %/, $(obj-n) $(obj-))

I'm slightly puzzled by the mismatch in blank padding on the three
lines above. I assume the last one is to match ...

>  subdir-all := $(subdir-y) $(subdir-n)

... this, but I think it would be better for all of them to match,
or as the 2nd best option, for subdir-n to match subdir-y. Easy
enough to do while committing I guess, but this would want your
consent.

Applicable parts

Acked-by: Jan Beulich <jbeulich@suse.com>

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 14/23] xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 14/23] xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS) Anthony PERARD
@ 2020-02-27 10:22   ` Roger Pau Monné
  2020-03-10 17:55     ` Anthony PERARD
  2020-03-04 14:42   ` Jan Beulich
  1 sibling, 1 reply; 76+ messages in thread
From: Roger Pau Monné @ 2020-02-27 10:22 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Tim Deegan,
	Jan Beulich, Anthony PERARD, xen-devel, Volodymyr Babchuk

On Wed, Feb 26, 2020 at 11:33:46AM +0000, Anthony PERARD wrote:
> From: Anthony PERARD <anthony.perard@gmail.com>
> 
> In a later patch ("xen/build: have the root Makefile generates the
> CFLAGS), we want to generate the CFLAGS in xen/Makefile, then export
> it and have Rules.mk use a CFLAGS from the environment variables. That
> changes the flavor of the CFLAGS and flags intended for one target
> (like -D__OBJECT_FILE__ and -M%) gets propagated and duplicated. So we
> start by moving such flags out of $(CFLAGS) and into $(c_flags) which
> is to be modified by only Rules.mk.
> 
> __OBJECT_FILE__ is only used by arch/x86/mm/*.c files, so having it in
> $(c_flags) is enough, we don't need it in $(a_flags).

This seem to be used only by source files that are build multiple
times with different parameters in order to generate different object
files.

Is there any harm in having it also in the assembler flags? (in case
we require such usage in the future)

Or maybe we could even limit __OBJECT_FILE__ to mm/ files that require
it only?

> 
> For include/Makefile and as-insn we can keep using CFLAGS, but since
> it doesn't have -M* flags anymore there is no need to filter them out.
> 
> The XEN_BUILD_EFI tests in arch/x86/Makefile was filtering out
> CFLAGS-y, but according to dd40177c1bc8 ("x86-64/EFI: add CFLAGS to
> check compile"), it was done to filter out -MF. CFLAGS doesn't
> have those flags anymore, so no filtering is needed.
> 
> This is inspired by the way Kbuild generates CFLAGS for each targets.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS Anthony PERARD
@ 2020-02-27 11:05   ` Roger Pau Monné
  2020-03-17 18:05     ` Anthony PERARD
  2020-03-04 15:00   ` Jan Beulich
  1 sibling, 1 reply; 76+ messages in thread
From: Roger Pau Monné @ 2020-02-27 11:05 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	xen-devel, Daniel De Graaf, Volodymyr Babchuk

On Wed, Feb 26, 2020 at 11:33:47AM +0000, Anthony PERARD wrote:
> Instead of generating the CFLAGS in Rules.mk everytime we enter a new
> subdirectory, we are going to generate most of them a single time, and
> export the result in the environment so that Rules.mk can use it.  The
> only flags left to generates are the one that depends on the targets,
                     ^ be generated    ^ ones   ^ depend
I think (albeit I'm not a native speaker).

> but the variable $(c_flags) takes care of that.
> 
> Arch specific CFLAGS are generated by a new file "arch/*/arch.mk"
> which is included by the root Makefile.
> 
> We export the *FLAGS via the environment variables XEN_*FLAGS because
> Rules.mk still includes Config.mk and would add duplicated flags to
> CFLAGS.
> 
> When running Rules.mk in the root directory (xen/), the variable
> `root-make-done' is set, so `need-config' will remain undef and so the
> root Makefile will not generate the cflags again.
> 
> We can't use CFLAGS in subdirectories to add flags to particular
> targets, instead start to use CFLAGS-y. Idem for AFLAGS.
> So there are two different CFLAGS-y, the one in xen/Makefile (and
> arch.mk), and the one in subdirs that Rules.mk is going to use.
> We can't add to XEN_CFLAGS because it is exported, so making change to
> it might be propagated to subdirectory which isn't intended.
> 
> Some style change are introduced in this patch:
>     when LDFLAGS_DIRECT is included in LDFLAGS
>     use of CFLAGS-$(CONFIG_INDIRECT_THUNK) instead of ifeq().
> 
> There is on FIXME added about LTO build, but since LTO is marked as
> BROKEN, this commit doesn't attempt to filter -flto flags out of the
> CFLAGS.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
> 
> Notes:
>     v3:
>     - squash "xen/build: introduce ccflags-y and CFLAGS_$@" here, with
>       those changes:
>         - rename ccflags-y to simply CFLAGS-y and start using AFLAGS-y in
>           subdirs.
>         - remove CFLAGS_$@, we don't need it yet.
>         - fix build of xen.lds and efi.lds which needed -D to be a_flags
>     - remove arch_ccflags, and modify c_flags directly
>       with that change, reorder c_flags, so that target specific flags are last.
>     - remove HAVE_AS_QUOTED_SYM from envvar and check XEN_CFLAGS to find if
>       it's there when adding -D__OBJECT_LABEL__.
>     - fix missing some flags in AFLAGS
>       (like -fshort-wchar in xen/arch/x86/efi/Makefile,
>        and -D__OBJECT_LABEL__ and CFLAGS-stack-boundary)
>     - keep COV_FLAGS generation in Rules.mk since it doesn't invovle to
>       call CC
>     - fix clang test for "asm()-s support .include." (in a new patch done
>       ahead)
>     - include Kconfig.include in xen/Makefile because as-option-add is
>       defined there now.
> 
>  xen/Makefile               | 60 ++++++++++++++++++++++++
>  xen/Rules.mk               | 74 ++++++++----------------------
>  xen/arch/arm/Makefile      | 10 ++--
>  xen/arch/arm/Rules.mk      | 93 --------------------------------------
>  xen/arch/arm/arch.mk       | 88 ++++++++++++++++++++++++++++++++++++
>  xen/arch/arm/efi/Makefile  |  2 +-
>  xen/arch/x86/Makefile      | 24 +++++-----
>  xen/arch/x86/Rules.mk      | 91 +++----------------------------------
>  xen/arch/x86/arch.mk       | 84 ++++++++++++++++++++++++++++++++++
>  xen/arch/x86/efi/Makefile  |  2 +-
>  xen/common/libelf/Makefile |  4 +-
>  xen/common/libfdt/Makefile |  4 +-
>  xen/include/Makefile       |  2 +-
>  xen/xsm/flask/Makefile     |  2 +-
>  xen/xsm/flask/ss/Makefile  |  2 +-
>  15 files changed, 283 insertions(+), 259 deletions(-)
>  create mode 100644 xen/arch/arm/arch.mk
>  create mode 100644 xen/arch/x86/arch.mk
> 
> diff --git a/xen/Makefile b/xen/Makefile
> index a6120e577e9b..da017dc29d36 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -94,6 +94,8 @@ config: FORCE
>  
>  else # !config-build
>  
> +include scripts/Kbuild.include
> +
>  ifeq ($(need-config),y)
>  include include/config/auto.conf
>  # Read in dependencies to all Kconfig* files, make sure to run syncconfig if
> @@ -113,6 +115,64 @@ $(KCONFIG_CONFIG):
>  include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
>  	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" syncconfig
>  
> +ifeq ($(CONFIG_DEBUG),y)
> +CFLAGS += -O1
> +else
> +CFLAGS += -O2
> +endif

Long term we might want to make the optimization level selectable in
Kconfig IMO.

> +
> +ifeq ($(CONFIG_FRAME_POINTER),y)
> +CFLAGS += -fno-omit-frame-pointer
> +else
> +CFLAGS += -fomit-frame-pointer
> +endif
> +
> +CFLAGS += -nostdinc -fno-builtin -fno-common
> +CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
> +$(call cc-option-add,CFLAGS,CC,-Wvla)
> +CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
> +CFLAGS-$(CONFIG_DEBUG_INFO) += -g
> +
> +ifneq ($(CONFIG_CC_IS_CLANG),y)
> +# Clang doesn't understand this command line argument, and doesn't appear to
> +# have an suitable alternative.  The resulting compiled binary does function,
          ^ a
> +# but has an excessively large symbol table.
> +CFLAGS += -Wa,--strip-local-absolute

This is not really related to clang, but to the assembler. If clang is
used with -no-integrated-as it's quite likely that the GNU assembler
will be used, and hence this option would be available.

Can we use cc-option-add here in order to detect whether the build
toolchain support the option?

Ideally this should be done after the integrated assembler tests
performed in x86/Rules.mk.

> +endif
> +
> +AFLAGS += -D__ASSEMBLY__
> +
> +CFLAGS += $(CFLAGS-y)
> +# allow extra CFLAGS externally via EXTRA_CFLAGS_XEN_CORE
> +CFLAGS += $(EXTRA_CFLAGS_XEN_CORE)
> +
> +# Most CFLAGS are safe for assembly files:
> +#  -std=gnu{89,99} gets confused by #-prefixed end-of-line comments
> +#  -flto makes no sense and annoys clang
> +AFLAGS += $(filter-out -std=gnu% -flto,$(CFLAGS))
> +
> +# LDFLAGS are only passed directly to $(LD)
> +LDFLAGS += $(LDFLAGS_DIRECT) $(LDFLAGS-y)
> +
> +ifeq ($(CONFIG_UBSAN),y)
> +CFLAGS_UBSAN := -fsanitize=undefined
> +else
> +CFLAGS_UBSAN :=

Do you need to define this to empty so it can be exported below? Isn't
it enough to just not set it at all?

> +endif
> +
> +ifeq ($(CONFIG_LTO),y)
> +CFLAGS += -flto
> +LDFLAGS-$(CONFIG_CC_IS_CLANG) += -plugin LLVMgold.so
> +endif
> +
> +include $(BASEDIR)/arch/$(TARGET_ARCH)/arch.mk

The strip-local-absolute check should be done after this AFAICT.

> +
> +# define new variables to avoid the ones defines in Config.mk
> +export XEN_CFLAGS := $(CFLAGS)
> +export XEN_AFLAGS := $(AFLAGS)
> +export XEN_LDFLAGS := $(LDFLAGS)
> +export CFLAGS_UBSAN

You might want to rename this to XEN_CFLAGS_UBSAN for coherency with
the rest of the exported variables?

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 17/23] xen/build: Start using if_changed
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 17/23] xen/build: Start using if_changed Anthony PERARD
@ 2020-02-27 13:09   ` Roger Pau Monné
  2020-03-04 16:00     ` Jan Beulich
  0 siblings, 1 reply; 76+ messages in thread
From: Roger Pau Monné @ 2020-02-27 13:09 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	xen-devel, Daniel De Graaf, Volodymyr Babchuk

On Wed, Feb 26, 2020 at 11:33:49AM +0000, Anthony PERARD wrote:
> This patch start to use if_changed introduced in a previous commit.
> 
> Whenever if_changed is called, the target must have FORCE as
> dependency so that if_changed can check if the command line to be
> run as changed, so the macro $(real-prereqs) must be use to
      ^ has                                            ^ used
> discover the dependencies without "FORCE".
> 
> Whenever a target isn't in obj-y, it should be added to extra-y so the
> .*.cmd dependency file associated with the target can be loaded. This
> is done for xsm/flask/ and both common/lib{elf,fdt}/ and
> arch/x86/Makefile.
> 
> For the targets that generates .*.d dependency files, there's going to
                       ^ generate
> be two dependency files (.*.d and .*.cmd) until we can merge them
> together in a later patch via fixdep from Linux.
> 
> One cleanup, libelf-relocate.o doesn't exist anymore.
> 
> We import cmd_ld and cmd_objcopy from Linux v5.4.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
>  xen/Rules.mk               | 68 +++++++++++++++++++++++++++-----------
>  xen/arch/arm/Makefile      |  4 +--
>  xen/arch/x86/Makefile      |  1 +
>  xen/arch/x86/efi/Makefile  |  7 ++--
>  xen/common/libelf/Makefile | 12 ++++---
>  xen/common/libfdt/Makefile |  9 +++--
>  xen/xsm/flask/Makefile     | 17 +++++++---
>  7 files changed, 84 insertions(+), 34 deletions(-)
> 
> diff --git a/xen/Rules.mk b/xen/Rules.mk
> index 8807a2e21c94..bb4ced5f0dd4 100644
> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -52,6 +52,18 @@ SPECIAL_DATA_SECTIONS := rodata $(foreach a,1 2 4 8 16, \
>  
>  include Makefile
>  
> +# Linking
> +# ---------------------------------------------------------------------------

I'm not sure adding such delimiters is helpful, people tend to forget
to add them in newer changes and it ends up being all asymmetric.

> +
> +quiet_cmd_ld = LD      $@
> +cmd_ld = $(LD) $(XEN_LDFLAGS) -r -o $@ $(real-prereqs)
> +
> +# Objcopy
> +# ---------------------------------------------------------------------------
> +
> +quiet_cmd_objcopy = OBJCOPY $@
> +cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS) $< $@
> +
>  define gendep
>      ifneq ($(1),$(subst /,:,$(1)))
>          DEPS += $(dir $(1)).$(notdir $(1)).d
> @@ -161,29 +173,47 @@ else
>  	$(CC) $(c_flags) -c $< -o $@
>  endif
>  
> -%.o: %.S Makefile
> -	$(CC) $(a_flags) -c $< -o $@
> +quiet_cmd_cc_o_S = CC      $@
> +cmd_cc_o_S = $(CC) $(a_flags) -c $< -o $@
> +
> +%.o: %.S FORCE
> +	$(call if_changed,cc_o_S)
> +
> +
> +quiet_cmd_obj_init_o = INIT_O  $@

INIT_O seems kind of weird, maybe just using CHECK would be OK?

The rest LGTM:

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 20/23] xen/build: factorise generation of the linker scripts
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 20/23] xen/build: factorise generation of the linker scripts Anthony PERARD
@ 2020-02-27 13:14   ` Roger Pau Monné
  2020-03-05 11:07     ` Jan Beulich
  2020-03-05 11:05   ` Jan Beulich
  1 sibling, 1 reply; 76+ messages in thread
From: Roger Pau Monné @ 2020-02-27 13:14 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	xen-devel, Volodymyr Babchuk

On Wed, Feb 26, 2020 at 11:33:52AM +0000, Anthony PERARD wrote:
> In Arm and X86 makefile, generating the linker script is the same, so
> we can simply have both call the same macro.
> 
> We need to add *.lds files into extra-y so that Rules.mk can find the
> .*.cmd dependency file and load it.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
>  xen/Rules.mk          | 8 ++++++++
>  xen/arch/arm/Makefile | 5 ++---
>  xen/arch/x86/Makefile | 6 +++---
>  3 files changed, 13 insertions(+), 6 deletions(-)
> 
> diff --git a/xen/Rules.mk b/xen/Rules.mk
> index 8c7dba9211d1..02cd37d04054 100644
> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -230,6 +230,14 @@ cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
>  %.s: %.S FORCE
>  	$(call if_changed,cpp_s_S)
>  
> +# Linker scripts, .lds.S -> .lds
> +quiet_cmd_cc_lds_S = LDS     $@
> +define cmd_cc_lds_S
> +    $(CC) -P -E -Ui386 $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<; \

Do you know why the -Ui386 is needed?

Also, can this use the CPP rune? I would at least consider naming this
CPP, as it's pre-processing the link script, LDS seems quite obscure.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 21/23] xen/build: Use if_changed for prelink*.o
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 21/23] xen/build: Use if_changed for prelink*.o Anthony PERARD
@ 2020-02-27 13:16   ` Roger Pau Monné
  2020-03-04 16:12     ` Jan Beulich
  0 siblings, 1 reply; 76+ messages in thread
From: Roger Pau Monné @ 2020-02-27 13:16 UTC (permalink / raw)
  To: Anthony PERARD; +Cc: xen-devel, Wei Liu, Jan Beulich, Andrew Cooper

On Wed, Feb 26, 2020 at 11:33:53AM +0000, Anthony PERARD wrote:
> We change the dependencies of prelink-efi.o so that we can use the
> same command line. The dependency on efi/built_in.o isn't needed
> because, we already have:
>     efi/*.o: efi/built_in.o
> to build efi/*.o files that prelink-efi.o needs.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements
  2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
                   ` (22 preceding siblings ...)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 23/23] xen/build: use if_changed to build guest_%.o Anthony PERARD
@ 2020-02-27 21:17 ` Stewart Hildebrand
  2020-03-06 10:12   ` Anthony PERARD
  23 siblings, 1 reply; 76+ messages in thread
From: Stewart Hildebrand @ 2020-02-27 21:17 UTC (permalink / raw)
  To: Anthony PERARD, xen-devel
  Cc: Kevin Tian, Stefano Stabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, George Dunlap, Ian Jackson, Tim Deegan,
	Jan Beulich, Andrew Cooper, Daniel De Graaf, Volodymyr Babchuk,
	Roger Pau Monné

On Wednesday, February 26, 2020 6:34 AM, Anthony PERARD wrote:
>Patch series available in this git branch:
>https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-v3
>
>v3:
>- new patches that do some cleanup or fix issues
>- have rework most patches, to have better commit message or change the coding
>  style, or fix issues that I've seen. There were some cases where CFLAGS were
>  missing.
>  See patch notes for details
>- introduce if_changed*. That plenty of new patches on top of what we had in v2.
>  (those changes ignore CONFIG_LTO=y, I'll see about fixing that later)
>
>(There is more to come in order to use fixdep from Linux, but that's not ready)
>
>v2.1:
>- some fixes
>
>v2:
>Rather than taking Kbuild and making it work with Xen, the v2 takes the opposite
>approach of slowly transforming our current build system into Kbuild. That have
>the advantage of keeping all the feature we have and making the patches much
>easier to review. Kconfig update is done in an other patch series.
>
>Hi,
>
>I have work toward building Xen (the hypervisor) with Linux's build system,
>Kbuild.
>
>The main reason for that is to be able to have out-of-tree build. It's annoying
>when a build fail because of the pvshim. Other benefit is a much faster
>rebuild, and `make clean` doesn't take ages, and better dependencies to figure
>out what needs to be rebuild.
>
>So, we are not there yet, but the series already contain quite a few
>improvement and cleanup. More patches are going to be added to the series.

Thanks for your efforts with this. With your br.build-system-xen-v3
branch, I'm having trouble doing an aarch64 build with early printk
enabled. I suspect the following unmerged patch that Julien authored
last September may have some helpful information:
https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg01207.html

I tried the following...

echo "CONFIG_DEBUG=y" > xen/arch/arm/configs/arm64_defconfig
make -C xen XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CONFIG_EARLY_PRINTK=zynqmp defconfig
make XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CONFIG_EARLY_PRINTK=zynqmp dist-xen

... but I was met with:
prelink.o:(.data+0x578): undefined reference to `early_puts'
aarch64-linux-gnu-ld: /home/stew/rpi/xen/xen/.xen-syms.0: hidden symbol `early_puts' isn't defined

Stew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 09/23] xen/build: extract clean target from Rules.mk
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 09/23] xen/build: extract clean target from Rules.mk Anthony PERARD
@ 2020-03-04 14:13   ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-03-04 14:13 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Anthony PERARD,
	xen-devel

On 26.02.2020 12:33, Anthony PERARD wrote:
> From: Anthony PERARD <anthony.perard@gmail.com>
> 
> Most of the code executed by Rules.mk isn't necessary for the clean
> target, especially not the CFLAGS. This patch makes running make clean
> much faster.
> 
> The patch extract the clean target into a different Makefile,
> Makefile.clean.
> 
> Since Makefile.clean, doesn't want to include Config.mk, we need to
> define the variables DEPS_INCLUDE and DEPS in a place common to
> Rules.mk and Makefile.clean, this is Kbuild.include. DEPS_RM is only
> needed in Makefile.clean so can be defined there.
> 
> Even so Rules.mk includes Config.mk, it includes Kbuild.include after,
> so the effective definition of DEPS_INCLUDE is "xen/" one and the
> same one as used by Makefile.clean.
> 
> This is inspired by Kbuild, with Makefile.clean partially copied from
> Linux v5.4.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y
  2020-02-27  9:43   ` Jan Beulich
@ 2020-03-04 14:14     ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-03-04 14:14 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Kevin Tian, Stefano Stabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper, Ian Jackson,
	xen-devel, Daniel De Graaf, Volodymyr Babchuk,
	Roger Pau Monné

On 27.02.2020 10:43, Jan Beulich wrote:
> On 26.02.2020 12:33, Anthony PERARD wrote:
>> --- a/xen/Rules.mk
>> +++ b/xen/Rules.mk
>> @@ -111,17 +111,14 @@ define gendep
>>  endef
>>  $(foreach o,$(filter-out %/,$(obj-y) $(obj-bin-y) $(extra-y)),$(eval $(call gendep,$(o))))
>>  
>> -# Ensure each subdirectory has exactly one trailing slash.
>> -subdir-n := $(patsubst %,%/,$(patsubst %/,%,$(subdir-n) $(subdir-)))
>> -subdir-y := $(patsubst %,%/,$(patsubst %/,%,$(subdir-y)))
>> -
>> -# Add explicitly declared subdirectories to the object lists.
>> -obj-y += $(patsubst %/,%/built_in.o,$(subdir-y))
>> -
>> -# Add implicitly declared subdirectories (in the object lists) to the
>> -# subdirectory list, and rewrite the object-list entry.
>> -subdir-y += $(filter %/,$(obj-y))
>> -obj-y    := $(patsubst %/,%/built-in.o,$(obj-y))
>> +# Handle objects in subdirs
>> +# ---------------------------------------------------------------------------
>> +# o if we encounter foo/ in $(obj-y), replace it by foo/built_in.o
>> +#   and add the directory to the list of dirs to descend into: $(subdir-y)
>> +subdir-y := $(subdir-y) $(filter %/, $(obj-y))
>> +obj-y    := $(patsubst %/, %/built_in.o, $(obj-y))
>> +
>> +subdir-n   := $(subdir-n) $(subdir-) $(filter %/, $(obj-n) $(obj-))
> 
> I'm slightly puzzled by the mismatch in blank padding on the three
> lines above. I assume the last one is to match ...
> 
>>  subdir-all := $(subdir-y) $(subdir-n)
> 
> ... this, but I think it would be better for all of them to match,
> or as the 2nd best option, for subdir-n to match subdir-y. Easy
> enough to do while committing I guess, but this would want your
> consent.

Oh, these two lines go away again in patch 9. No need for any
adjustment then.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 10/23] xen/build: run targets csopes, tags, .. without Rules.mk
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 10/23] xen/build: run targets csopes, tags, .. without Rules.mk Anthony PERARD
@ 2020-03-04 14:17   ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-03-04 14:17 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On 26.02.2020 12:33, Anthony PERARD wrote:
> Those targets make use of $(all_sources) which depends on TARGET_ARCH,
> so we just need to set TARGET_ARCH earlier and once.
> 
> XEN_TARGET_ARCH isn't expected to change during the build, so
> TARGET_SUBARCH and TARGET_ARCH aren't going to change either. Set them
> once and for all in the Xen root Makefile. This allows to run more
> targets without Rules.mk.
> 
> XEN_TARGET_ARCH is actually changed in arch/x86/boot/build32.mk, but
> it doesn't use the TARGET_{,SUB}ARCH variables either, and doesn't use
> Rules.mk (it replaces it).
> 
> TARGET_{,SUB}ARCH are no longer overridden because that would have
> no effect on the values that Rules.mk will use.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 12/23] xen/build: Move as-option-add to xen/
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 12/23] xen/build: Move as-option-add to xen/ Anthony PERARD
@ 2020-03-04 14:17   ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-03-04 14:17 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On 26.02.2020 12:33, Anthony PERARD wrote:
> Only xen/ uses as-option-add and as-insn, so there aren't needed in
> Config.mk.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 13/23] xen/build: include include/config/auto.conf in main Makefile
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 13/23] xen/build: include include/config/auto.conf in main Makefile Anthony PERARD
@ 2020-03-04 14:29   ` Jan Beulich
  2020-03-10 17:10     ` Anthony PERARD
  0 siblings, 1 reply; 76+ messages in thread
From: Jan Beulich @ 2020-03-04 14:29 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On 26.02.2020 12:33, Anthony PERARD wrote:
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -49,7 +49,71 @@ default: build
>  .PHONY: dist
>  dist: install
>  
> -build install:: include/config/auto.conf
> +
> +ifeq ($(root-make-done),)

This getting communicated between make recursion instances via ...

> +# section to run before calling Rules.mk, but only once.
> +#
> +# To make sure we do not include .config for any of the *config targets
> +# catch them early, and hand them over to tools/kconfig/Makefile
> +
> +clean-targets := %clean
> +no-dot-config-targets := $(clean-targets) \
> +                         uninstall debug cloc \
> +                         cscope TAGS tags MAP gtags \
> +                         xenversion
> +
> +config-build    := n
> +need-config     := y
> +
> +ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),)
> +    ifeq ($(filter-out $(no-dot-config-targets), $(MAKECMDGOALS)),)
> +        need-config := n
> +    endif
> +endif
> +
> +ifneq ($(filter %config,$(MAKECMDGOALS)),)
> +    config-build := y
> +endif
> +
> +export root-make-done := y

... the environment, can we be as resilient as possible against a
variable of this name already existing in the environment before
the top level make invocation, by making the construct above

ifneq ($(root-make-done),y)

?

> +endif # root-make-done
> +
> +ifeq ($(config-build),y)
> +# ===========================================================================
> +# *config targets only - make sure prerequisites are updated, and descend
> +# in tools/kconfig to make the *config target
> +
> +config: FORCE
> +	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" $@

This, ...

> +
> +# Config.mk tries to include .config file, don't try to remake it
> +%/.config: ;
> +
> +%config: FORCE
> +	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" $@

... this, ...

> +else # !config-build
> +
> +ifeq ($(need-config),y)
> +include include/config/auto.conf
> +# Read in dependencies to all Kconfig* files, make sure to run syncconfig if
> +# changes are detected.
> +include include/config/auto.conf.cmd
> +
> +# Allow people to just run `make` as before and not force them to configure
> +$(KCONFIG_CONFIG):
> +	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" defconfig

... this, and ...

> +# The actual configuration files used during the build are stored in
> +# include/generated/ and include/config/. Update them if .config is newer than
> +# include/config/auto.conf (which mirrors .config).
> +#
> +# This exploits the 'multi-target pattern rule' trick.
> +# The syncconfig should be executed only once to make all the targets.
> +include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
> +	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" syncconfig

... this are almost identical, pretty long lines. Can this be macroized,
please, with the actual make goal as parameter?

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 14/23] xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS)
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 14/23] xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS) Anthony PERARD
  2020-02-27 10:22   ` Roger Pau Monné
@ 2020-03-04 14:42   ` Jan Beulich
  2020-03-10 17:43     ` Anthony PERARD
  1 sibling, 1 reply; 76+ messages in thread
From: Jan Beulich @ 2020-03-04 14:42 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Tim Deegan,
	Anthony PERARD, xen-devel, Volodymyr Babchuk,
	Roger Pau Monné

On 26.02.2020 12:33, Anthony PERARD wrote:
> --- a/xen/scripts/Kbuild.include
> +++ b/xen/scripts/Kbuild.include
> @@ -10,7 +10,7 @@ DEPS_INCLUDE = $(addsuffix .d2, $(basename $(wildcard $(DEPS))))
>  # as-insn: Check whether assembler supports an instruction.
>  # Usage: cflags-y += $(call as-insn,CC FLAGS,"insn",option-yes,option-no)
>  as-insn = $(if $(shell echo 'void _(void) { asm volatile ( $(2) ); }' \
> -                       | $(filter-out -M% %.d -include %/include/xen/config.h,$(1)) \
> +                       | $(filter-out -include %/include/xen/config.h,$(1)) \
>                                -c -x c -o /dev/null - 2>&1),$(4),$(3))

I'm sorry, while it was me to suggest this change - is this
correct? The variable to modify is a parameter of this macro,
i.e. things aren't limited to CFLAGS here. If we want to
disallow use with e.g. c_flags or anything derived from it,
then we should find some way to actually enforce this (like
dropping the respective parameter; I'm uncertain though whether
we wouldn't regret this if we ever got to the point where we
wanted to use a newer insn in a .S file).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS Anthony PERARD
  2020-02-27 11:05   ` Roger Pau Monné
@ 2020-03-04 15:00   ` Jan Beulich
  2020-03-17 18:35     ` Anthony PERARD
  1 sibling, 1 reply; 76+ messages in thread
From: Jan Beulich @ 2020-03-04 15:00 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel,
	Daniel De Graaf, Volodymyr Babchuk, Roger Pau Monné

On 26.02.2020 12:33, Anthony PERARD wrote:
> @@ -113,6 +115,64 @@ $(KCONFIG_CONFIG):
>  include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
>  	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" syncconfig
>  
> +ifeq ($(CONFIG_DEBUG),y)
> +CFLAGS += -O1
> +else
> +CFLAGS += -O2
> +endif
> +
> +ifeq ($(CONFIG_FRAME_POINTER),y)
> +CFLAGS += -fno-omit-frame-pointer
> +else
> +CFLAGS += -fomit-frame-pointer
> +endif
> +
> +CFLAGS += -nostdinc -fno-builtin -fno-common
> +CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
> +$(call cc-option-add,CFLAGS,CC,-Wvla)
> +CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
> +CFLAGS-$(CONFIG_DEBUG_INFO) += -g
> +
> +ifneq ($(CONFIG_CC_IS_CLANG),y)
> +# Clang doesn't understand this command line argument, and doesn't appear to
> +# have an suitable alternative.  The resulting compiled binary does function,
> +# but has an excessively large symbol table.
> +CFLAGS += -Wa,--strip-local-absolute
> +endif
> +
> +AFLAGS += -D__ASSEMBLY__
> +
> +CFLAGS += $(CFLAGS-y)

I can't seem to be able to spot a similar line for AFLAGS.

> @@ -107,7 +65,7 @@ $(foreach o,$(filter-out %/,$(obj-y) $(obj-bin-y) $(extra-y)),$(eval $(call gend
>  subdir-y := $(subdir-y) $(filter %/, $(obj-y))
>  obj-y    := $(patsubst %/, %/built_in.o, $(obj-y))
>  
> -$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS += -DINIT_SECTIONS_ONLY
> +$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS-y += -DINIT_SECTIONS_ONLY

While in the description you say "We can't use CFLAGS in
subdirectories to add flags to particular targets, ...", it
remains unclear there why that is, and hence why changes like
this one are necessary. If this is a restriction that's going to
remain, this also needs writing down in a prominent place. After
all if (for example) special compiler options are needed, CFLAGS
would be the natural thing one would want to alter. (Even better
if wrong playing with CFLAGS could be detected and at least
warned about, but I'm completely unclear on how feasible this
would be.)

> diff --git a/xen/arch/arm/Rules.mk b/xen/arch/arm/Rules.mk
> index 022a3a6f82ba..e69de29bb2d1 100644
> --- a/xen/arch/arm/Rules.mk
> +++ b/xen/arch/arm/Rules.mk
> @@ -1,93 +0,0 @@

As per the header here you're using git. Can you please arrange for
this file movement (to arch.mk, and also for x86) to actually be
expressed here as a rename, i.e. such that one can see what - if
anything - changes?

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 16/23] xen/build: introduce if_changed and if_changed_rule
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 16/23] xen/build: introduce if_changed and if_changed_rule Anthony PERARD
@ 2020-03-04 15:45   ` Jan Beulich
  2020-03-18 10:44     ` Anthony PERARD
  0 siblings, 1 reply; 76+ messages in thread
From: Jan Beulich @ 2020-03-04 15:45 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On 26.02.2020 12:33, Anthony PERARD wrote:
> The if_changed macro from Linux can record the command used to build a
> target then compare it on rebuild. Thus if a command has changed, for
> example due to introducing new flags in CFLAGS or due to using a
> different compiler, the target will be rebuilt.

As to using a different compiler - I suppose this means "a compiler
with a different executable name" here? What about me having, say
gcc-5 in use, and then updating my system such that a 5.2 based
compiler of this name would be upgraded to a 5.4 based one of this
same name. If this newer compiler has better capabilities (that we
would want to use if available), would this or anything else trigger
a rebuild then too?

> --- a/.gitignore
> +++ b/.gitignore
> @@ -6,6 +6,7 @@
>  *.o
>  *.d
>  *.d2
> +.*.cmd
>  *.opic
>  *.a
>  *.so

I admit these entries aren't sorted very well, but anyway - how
did you end up with this insertion point? There are entries
starting with . at the very top of the file. (As an aside, I
wonder why it's *.d and *.d2 rather than .*.d and .*.d2 .)

> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -52,7 +52,57 @@ dist: install
>  
>  ifeq ($(root-make-done),)
>  # section to run before calling Rules.mk, but only once.
> +
> +# Beautify output
> +# ---------------------------------------------------------------------------
> +#
> +# Normally, we echo the whole command before executing it. By making
> +# that echo $($(quiet)$(cmd)), we now have the possibility to set
> +# $(quiet) to choose other forms of output instead, e.g.
> +#
> +#         quiet_cmd_cc_o_c = Compiling $(RELDIR)/$@
> +#         cmd_cc_o_c       = $(CC) $(c_flags) -c -o $@ $<
> +#
> +# If $(quiet) is empty, the whole command will be printed.
> +# If it is set to "quiet_", only the short version will be printed.
> +# If it is set to "silent_", nothing will be printed at all, since
> +# the variable $(silent_cmd_cc_o_c) doesn't exist.
> +#
> +# A simple variant is to prefix commands with $(Q) - that's useful
> +# for commands that shall be hidden in non-verbose mode.
>  #
> +#	$(Q)ln $@ :<
> +#
> +# If KBUILD_VERBOSE equals 0 then the above command will be hidden.
> +# If KBUILD_VERBOSE equals 1 then the above command is displayed.
> +#
> +# To put more focus on warnings, be less verbose as default
> +# Use 'make V=1' to see the full commands
> +
> +ifeq ("$(origin V)", "command line")
> +  KBUILD_VERBOSE = $(V)
> +endif
> +ifndef KBUILD_VERBOSE
> +  KBUILD_VERBOSE = 0
> +endif
> +
> +ifeq ($(KBUILD_VERBOSE),1)
> +  quiet =
> +  Q =
> +else
> +  quiet=quiet_
> +  Q = @
> +endif
> +
> +# If the user is running make -s (silent mode), suppress echoing of
> +# commands
> +
> +ifneq ($(findstring s,$(filter-out --%,$(MAKEFLAGS))),)
> +  quiet=silent_
> +endif

Throughout the above, can the uses of = please become consistent?
Preferable all with a blank on the left and - unless there's no
value getting assigned - one on the right, plus := preferred over
= where not prohibited by other constraints (none here afaics).

> --- a/xen/scripts/Kbuild.include
> +++ b/xen/scripts/Kbuild.include
> @@ -2,11 +2,30 @@
>  ####
>  # kbuild: Generic definitions
>  
> +# Convenient variables
> +squote  := '
> +empty   :=
> +space   := $(empty) $(empty)
> +space_escape := _-_SPACE_-_
> +pound := \#

Nit: To fit with the three ones above space_escape you want to
add two blanks here.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 17/23] xen/build: Start using if_changed
  2020-02-27 13:09   ` Roger Pau Monné
@ 2020-03-04 16:00     ` Jan Beulich
  2020-03-18 10:52       ` Anthony PERARD
  0 siblings, 1 reply; 76+ messages in thread
From: Jan Beulich @ 2020-03-04 16:00 UTC (permalink / raw)
  To: Roger Pau Monné, Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel,
	Daniel De Graaf, Volodymyr Babchuk

On 27.02.2020 14:09, Roger Pau Monné wrote:
> On Wed, Feb 26, 2020 at 11:33:49AM +0000, Anthony PERARD wrote:
>> @@ -161,29 +173,47 @@ else
>>  	$(CC) $(c_flags) -c $< -o $@
>>  endif
>>  
>> -%.o: %.S Makefile
>> -	$(CC) $(a_flags) -c $< -o $@
>> +quiet_cmd_cc_o_S = CC      $@
>> +cmd_cc_o_S = $(CC) $(a_flags) -c $< -o $@
>> +
>> +%.o: %.S FORCE
>> +	$(call if_changed,cc_o_S)
>> +
>> +
>> +quiet_cmd_obj_init_o = INIT_O  $@
> 
> INIT_O seems kind of weird, maybe just using CHECK would be OK?

CHECK is not expressing what's going on - one could/would imply
that the object file doesn't get changed at all, but its sections
get renamed. I think INIT_O is sufficiently expressive at least
to people knowing the build system.

> The rest LGTM:
> 
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 18/23] xen/build: use if_changed on built_in.o
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 18/23] xen/build: use if_changed on built_in.o Anthony PERARD
@ 2020-03-04 16:03   ` Jan Beulich
  2020-03-18 10:55     ` Anthony PERARD
  0 siblings, 1 reply; 76+ messages in thread
From: Jan Beulich @ 2020-03-04 16:03 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On 26.02.2020 12:33, Anthony PERARD wrote:
> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -126,14 +126,21 @@ include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
>  c_flags += $(CFLAGS-y)
>  a_flags += $(CFLAGS-y) $(AFLAGS-y)
>  
> -built_in.o: $(obj-y) $(extra-y)
> +quiet_cmd_ld_builtin = LD      $@
> +cmd_ld_builtin = \
> +    $(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
> +quiet_cmd_cc_builtin = LD      $@
> +cmd_cc_builtin = \
> +    $(CC) $(XEN_CFLAGS) -c -x c /dev/null -o $@
> +
> +built_in.o: $(obj-y) $(extra-y) FORCE
>  ifeq ($(obj-y),)
> -	$(CC) $(c_flags) -c -x c /dev/null -o $@
> +	$(call if_changed,cc_builtin)
>  else
>  ifeq ($(CONFIG_LTO),y)
>  	$(LD_LTO) -r -o $@ $(filter-out $(extra-y),$^)

What about this? Couldn't you simply vary what cmd_ld_builtin
expands to, and drop this inner ifeq()?

Jan

>  else
> -	$(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$^)
> +	$(call if_changed,ld_builtin)
>  endif
>  endif
>  
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 19/23] xen/build: Use if_changed_rules with %.o:%.c targets
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 19/23] xen/build: Use if_changed_rules with %.o:%.c targets Anthony PERARD
@ 2020-03-04 16:09   ` Jan Beulich
  2020-03-18 11:14     ` Anthony PERARD
  0 siblings, 1 reply; 76+ messages in thread
From: Jan Beulich @ 2020-03-04 16:09 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On 26.02.2020 12:33, Anthony PERARD wrote:
> Use $(dot-target) to have the target name prefix with a dot.
> 
> Now, when the CC command has run, it is recorded in .*.cmd
> file, then if_changed_rules will compare it on subsequent runs.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with one question:

> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -167,19 +167,27 @@ FORCE:
>  
>  SRCPATH := $(patsubst $(BASEDIR)/%,%,$(CURDIR))
>  
> -%.o: %.c Makefile
> +quiet_cmd_cc_o_c = CC      $@
>  ifeq ($(CONFIG_ENFORCE_UNIQUE_SYMBOLS),y)
> -	$(CC) $(c_flags) -c $< -o $(@D)/.$(@F).tmp -MQ $@
> -ifeq ($(CONFIG_CC_IS_CLANG),y)
> -	$(OBJCOPY) --redefine-sym $<=$(SRCPATH)/$< $(@D)/.$(@F).tmp $@
> -else
> -	$(OBJCOPY) --redefine-sym $(<F)=$(SRCPATH)/$< $(@D)/.$(@F).tmp $@
> -endif
> -	rm -f $(@D)/.$(@F).tmp
> +    cmd_cc_o_c = $(CC) $(c_flags) -c $< -o $(dot-target).tmp -MQ $@
> +    ifeq ($(CONFIG_CC_IS_CLANG),y)
> +        cmd_objcopy_fix_sym = $(OBJCOPY) --redefine-sym $<=$(SRCPATH)/$< $(dot-target).tmp $@
> +    else
> +        cmd_objcopy_fix_sym = $(OBJCOPY) --redefine-sym $(<F)=$(SRCPATH)/$< $(dot-target).tmp $@
> +    endif
> +    cmd_objcopy_fix_sym += && rm -f $(dot-target).tmp
>  else
> -	$(CC) $(c_flags) -c $< -o $@
> +    cmd_cc_o_c = $(CC) $(c_flags) -c $< -o $@
>  endif
>  
> +define rule_cc_o_c
> +    $(call cmd_and_record,cc_o_c)
> +    $(call cmd,objcopy_fix_sym)

The machinery is resilient to a command (here: cmd_objcopy_fix_sym)
not being defined, and will neither produce any undue output nor
else incur any unnecessary overhead?

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 21/23] xen/build: Use if_changed for prelink*.o
  2020-02-27 13:16   ` Roger Pau Monné
@ 2020-03-04 16:12     ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-03-04 16:12 UTC (permalink / raw)
  To: Anthony PERARD; +Cc: xen-devel, Andrew Cooper, Wei Liu, Roger Pau Monné

On 27.02.2020 14:16, Roger Pau Monné wrote:
> On Wed, Feb 26, 2020 at 11:33:53AM +0000, Anthony PERARD wrote:
>> We change the dependencies of prelink-efi.o so that we can use the
>> same command line. The dependency on efi/built_in.o isn't needed
>> because, we already have:
>>     efi/*.o: efi/built_in.o
>> to build efi/*.o files that prelink-efi.o needs.
>>
>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y Anthony PERARD
  2020-02-27  9:22   ` Roger Pau Monné
  2020-02-27  9:43   ` Jan Beulich
@ 2020-03-05  9:24   ` Jan Beulich
  2020-03-05 13:42     ` Andrew Cooper
                       ` (2 more replies)
  2 siblings, 3 replies; 76+ messages in thread
From: Jan Beulich @ 2020-03-05  9:24 UTC (permalink / raw)
  To: Julien Grall, Stefano Stabellini, Kevin Tian, Daniel De Graaf
  Cc: Wei Liu, Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper,
	Ian Jackson, Anthony PERARD, xen-devel, Volodymyr Babchuk,
	Roger Pau Monné

On 26.02.2020 12:33, Anthony PERARD wrote:
> This is part of upgrading our build system and import more of Linux's
> one.
> 
> In Linux, subdir-y in Makefiles is only used to descend into
> subdirectory when there are no object to build, Xen doesn't have that
> and all subdir have object to be included in the final binary.
> 
> To allow the new syntax, the "obj-y" and "subdir-*" calculation in
> Rules.mk is changed and partially imported from Linux's Kbuild.
> 
> The command used to modify the Makefile was:
>     sed -i -r 's#^subdir-(.*)#obj-\1/#;' **/Makefile
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
> 
> Notes:
>     v3:
>     - no more tabs
>     - reshuffle variable, and remove __subdir-y
> 
>  xen/Rules.mk                         | 19 ++++++++-----------
>  xen/arch/arm/Makefile                | 14 +++++++-------
>  xen/arch/arm/arm32/Makefile          |  2 +-
>  xen/arch/arm/arm64/Makefile          |  2 +-

Julien, Stefano - any chance of getting an ack here?

>  xen/arch/x86/Makefile                | 18 +++++++++---------
>  xen/arch/x86/acpi/Makefile           |  2 +-
>  xen/arch/x86/cpu/Makefile            |  4 ++--
>  xen/arch/x86/guest/Makefile          |  4 ++--
>  xen/arch/x86/hvm/Makefile            |  6 +++---
>  xen/arch/x86/mm/Makefile             |  4 ++--
>  xen/arch/x86/x86_64/Makefile         |  2 +-
>  xen/common/Makefile                  | 10 +++++-----
>  xen/drivers/Makefile                 | 14 +++++++-------
>  xen/drivers/acpi/Makefile            |  6 +++---
>  xen/drivers/passthrough/Makefile     |  8 ++++----
>  xen/drivers/passthrough/vtd/Makefile |  2 +-

Kevin, how about this one?

>  xen/lib/Makefile                     |  2 +-
>  xen/xsm/Makefile                     |  2 +-
>  xen/xsm/flask/Makefile               |  2 +-

Daniel, how about these?

I guess the latter two are small enough to skip further waiting for
acks once the Arm one would be in place. Getting this patch in
would unblock a fair part of the remainder of this series.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 20/23] xen/build: factorise generation of the linker scripts
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 20/23] xen/build: factorise generation of the linker scripts Anthony PERARD
  2020-02-27 13:14   ` Roger Pau Monné
@ 2020-03-05 11:05   ` Jan Beulich
  2020-03-18 11:59     ` Anthony PERARD
  1 sibling, 1 reply; 76+ messages in thread
From: Jan Beulich @ 2020-03-05 11:05 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel,
	Volodymyr Babchuk, Roger Pau Monné

On 26.02.2020 12:33, Anthony PERARD wrote:
> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -230,6 +230,14 @@ cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
>  %.s: %.S FORCE
>  	$(call if_changed,cpp_s_S)
>  
> +# Linker scripts, .lds.S -> .lds
> +quiet_cmd_cc_lds_S = LDS     $@
> +define cmd_cc_lds_S
> +    $(CC) -P -E -Ui386 $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<; \

$(CPP)? And then also name the thing cmd_cpp_lds_S?

> +    sed -e 's/.*\.lds\.o:/$(@F):/g' <$(dot-target).d >$(dot-target).d.new; \
> +    mv -f $(dot-target).d.new $(dot-target).d

This would benefit from also switching to move-if-changed at
this occasion.

With you using "define" - is there really a need for adding the
trailing "; \" sequence to the first two lines of the macro?

> --- a/xen/arch/x86/Makefile
> +++ b/xen/arch/x86/Makefile
> @@ -75,6 +75,7 @@ obj-y += hpet.o
>  obj-y += vm_event.o
>  obj-y += xstate.o
>  extra-y += asm-macros.i
> +extra-y += xen.lds
>  
>  x86_emulate.o: x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h
>  
> @@ -197,6 +198,7 @@ endif
>  note_file_option ?= $(note_file)
>  
>  ifeq ($(XEN_BUILD_PE),y)
> +extra-y += efi.lds

Would be nice if this was moved up using

extra-$(XEN_BUILD_PE) += efi.lds

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 20/23] xen/build: factorise generation of the linker scripts
  2020-02-27 13:14   ` Roger Pau Monné
@ 2020-03-05 11:07     ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-03-05 11:07 UTC (permalink / raw)
  To: Roger Pau Monné, Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel,
	Volodymyr Babchuk

On 27.02.2020 14:14, Roger Pau Monné wrote:
> On Wed, Feb 26, 2020 at 11:33:52AM +0000, Anthony PERARD wrote:
>> In Arm and X86 makefile, generating the linker script is the same, so
>> we can simply have both call the same macro.
>>
>> We need to add *.lds files into extra-y so that Rules.mk can find the
>> .*.cmd dependency file and load it.
>>
>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
>> ---
>>  xen/Rules.mk          | 8 ++++++++
>>  xen/arch/arm/Makefile | 5 ++---
>>  xen/arch/x86/Makefile | 6 +++---
>>  3 files changed, 13 insertions(+), 6 deletions(-)
>>
>> diff --git a/xen/Rules.mk b/xen/Rules.mk
>> index 8c7dba9211d1..02cd37d04054 100644
>> --- a/xen/Rules.mk
>> +++ b/xen/Rules.mk
>> @@ -230,6 +230,14 @@ cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
>>  %.s: %.S FORCE
>>  	$(call if_changed,cpp_s_S)
>>  
>> +# Linker scripts, .lds.S -> .lds
>> +quiet_cmd_cc_lds_S = LDS     $@
>> +define cmd_cc_lds_S
>> +    $(CC) -P -E -Ui386 $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<; \
> 
> Do you know why the -Ui386 is needed?

It was needed for the 32-bit hypervisor build, to avoid corrupting

OUTPUT_ARCH(i386)

but it's not needed anymore. Arm shouldn't have had it in the first
place.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y
  2020-03-05  9:24   ` Jan Beulich
@ 2020-03-05 13:42     ` Andrew Cooper
  2020-03-05 15:02     ` Julien Grall
  2020-03-09  6:46     ` Tian, Kevin
  2 siblings, 0 replies; 76+ messages in thread
From: Andrew Cooper @ 2020-03-05 13:42 UTC (permalink / raw)
  To: Jan Beulich, Julien Grall, Stefano Stabellini, Kevin Tian,
	Daniel De Graaf
  Cc: Wei Liu, Konrad Rzeszutek Wilk, George Dunlap, Ian Jackson,
	Anthony PERARD, xen-devel, Volodymyr Babchuk,
	Roger Pau Monné

On 05/03/2020 09:24, Jan Beulich wrote:
> On 26.02.2020 12:33, Anthony PERARD wrote:
>> This is part of upgrading our build system and import more of Linux's
>> one.
>>
>> In Linux, subdir-y in Makefiles is only used to descend into
>> subdirectory when there are no object to build, Xen doesn't have that
>> and all subdir have object to be included in the final binary.
>>
>> To allow the new syntax, the "obj-y" and "subdir-*" calculation in
>> Rules.mk is changed and partially imported from Linux's Kbuild.
>>
>> The command used to modify the Makefile was:
>>     sed -i -r 's#^subdir-(.*)#obj-\1/#;' **/Makefile
>>
>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
>> ---
>>
>> Notes:
>>     v3:
>>     - no more tabs
>>     - reshuffle variable, and remove __subdir-y
>>
>>  xen/Rules.mk                         | 19 ++++++++-----------
>>  xen/arch/arm/Makefile                | 14 +++++++-------
>>  xen/arch/arm/arm32/Makefile          |  2 +-
>>  xen/arch/arm/arm64/Makefile          |  2 +-
> Julien, Stefano - any chance of getting an ack here?
>
>>  xen/arch/x86/Makefile                | 18 +++++++++---------
>>  xen/arch/x86/acpi/Makefile           |  2 +-
>>  xen/arch/x86/cpu/Makefile            |  4 ++--
>>  xen/arch/x86/guest/Makefile          |  4 ++--
>>  xen/arch/x86/hvm/Makefile            |  6 +++---
>>  xen/arch/x86/mm/Makefile             |  4 ++--
>>  xen/arch/x86/x86_64/Makefile         |  2 +-
>>  xen/common/Makefile                  | 10 +++++-----
>>  xen/drivers/Makefile                 | 14 +++++++-------
>>  xen/drivers/acpi/Makefile            |  6 +++---
>>  xen/drivers/passthrough/Makefile     |  8 ++++----
>>  xen/drivers/passthrough/vtd/Makefile |  2 +-
> Kevin, how about this one?
>
>>  xen/lib/Makefile                     |  2 +-
>>  xen/xsm/Makefile                     |  2 +-
>>  xen/xsm/flask/Makefile               |  2 +-
> Daniel, how about these?
>
> I guess the latter two are small enough to skip further waiting for
> acks once the Arm one would be in place. Getting this patch in
> would unblock a fair part of the remainder of this series.

This is a mechanical change to the entire tree, unrelated to the logic
in the maintainers areas.

It is a good example where "The Rest" qualifies better than the sum of
every sub-maintainer.  Given that this has already been pending for a
week, I'd say it is fine to go in now.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 22/23] xen, symbols: rework file symbols selection
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 22/23] xen, symbols: rework file symbols selection Anthony PERARD
@ 2020-03-05 14:44   ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-03-05 14:44 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On 26.02.2020 12:33, Anthony PERARD wrote:
> Rework symbols so it prefer file symbols that names an object file to
> file symbols that have a directory component.

I'm afraid I don't understand the distinction you apparently mean to
make: Something having a directory component may still name an
object file. I guess you want to refer to source file names.

> But have symbols still prefer the first file symbol if one of the above
> is true, or prefer the second file symbols if it names a source file
> without directory component.

"one of the above is true" meaning "'object file' or 'has directory
component'"? The first paragraph being a preference statement imo
doesn't lend itself to continuing like this.

Further I guess you mean "last" instead of "second"?

In total I understand the intended order of preference is
- object file name
- source file name with path component(s)
- source file name without any path component

> In a future patch, we are going want to run $(CC) from the root directory
> (xen.git/xen that is). So the guest_walk_%.o files are going to have
> two file symbols, one with a directory component and another one
> which name an object file.

Depending on the Kconfig settings, even today there may be two
file symbols there. Please could you (a) consider both build
modes in you description and (b) make clear - perhaps by way of
giving an example - what would result without your change, and
what will result with in in place (and then also before and
after that future change)? And, knowing they behave differently,
perhaps (c) also cover gcc vs clang (which will then likely also
cover the "why is this" part of the description).

> We still want to prefer the file symbols
> that names an object file, no mater if it is first or second.
> 
> And before running everything from the root directory, we will be able
> to use the same runes to build the guest_%.o as to build any other %.o
> files from %.c files (the rule with the objcopy --redefine-sym).

And when running everything from the root directory, we again
won't be able to? If so, what's the point of mentioning this,
when the almost immediate goal is to run everything from the
root directory?

> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
> --- a/xen/tools/symbols.c
> +++ b/xen/tools/symbols.c
> @@ -80,11 +80,17 @@ static inline int is_arm_mapping_symbol(const char *str)
>  	       && (str[2] == '\0' || str[2] == '.');
>  }
>  
> +enum symbol_type {
> +     symbol = 0,
> +     single_source = 1,
> +     dir_source = 2,
> +     obj_source = 3,

If numeric values matter, please say so in a comment. There's
no need at all to assign numeric values like you do here -
the same numbering will result with the "= <N>" dropped. I
guess you also mean obj_file rather than the pretty ambiguous
obj_source. Similarly with you renaming multi_source to
dir_source, I don't think single_source makes sense anymore.
Maybe simple_source or file_source, and maybe also path_source
instead of dir_source?

> +};
>  static int read_symbol(FILE *in, struct sym_entry *s)

Please have a blank line between these. I don't, however, see
why the scope of this enum gets widened to the entire file.

>  {
>  	char str[500], type[20] = "";
>  	char *sym, stype;
> -	static enum { symbol, single_source, multi_source } last;
> +	static enum symbol_type last;
>  	static char *filename;
>  	int rc = -1;
>  
> @@ -125,13 +131,19 @@ static int read_symbol(FILE *in, struct sym_entry *s)
>  		 * prefer the first one if that names an object file or has a
>  		 * directory component (to cover multiply compiled files).
>  		 */
> -		bool multi = strchr(str, '/') || (sym && sym[1] == 'o');
> -
> -		if (multi || last != multi_source) {
> +		enum symbol_type current;
> +		if (sym && sym[1] == 'o')

Blank line between declaration(s) and statement(s) please.

Jan

> +		    current = obj_source;
> +		else if (strchr(str, '/'))
> +		    current = dir_source;
> +		else
> +		    current = single_source;
> +
> +		if (current > last || last == single_source) {
>  			free(filename);
>  			filename = *str ? strdup(str) : NULL;
> +			last = current;
>  		}
> -		last = multi ? multi_source : single_source;
>  		goto skip_tail;
>  	}
>  
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y
  2020-03-05  9:24   ` Jan Beulich
  2020-03-05 13:42     ` Andrew Cooper
@ 2020-03-05 15:02     ` Julien Grall
  2020-03-05 15:59       ` Anthony PERARD
  2020-03-09  6:46     ` Tian, Kevin
  2 siblings, 1 reply; 76+ messages in thread
From: Julien Grall @ 2020-03-05 15:02 UTC (permalink / raw)
  To: Jan Beulich, Stefano Stabellini, Kevin Tian, Daniel De Graaf
  Cc: Wei Liu, Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper,
	Ian Jackson, Anthony PERARD, xen-devel, Volodymyr Babchuk,
	Roger Pau Monné

Hi Jan,

On 05/03/2020 09:24, Jan Beulich wrote:
> On 26.02.2020 12:33, Anthony PERARD wrote:
>> This is part of upgrading our build system and import more of Linux's
>> one.
>>
>> In Linux, subdir-y in Makefiles is only used to descend into
>> subdirectory when there are no object to build, Xen doesn't have that
>> and all subdir have object to be included in the final binary.
>>
>> To allow the new syntax, the "obj-y" and "subdir-*" calculation in
>> Rules.mk is changed and partially imported from Linux's Kbuild.
>>
>> The command used to modify the Makefile was:
>>      sed -i -r 's#^subdir-(.*)#obj-\1/#;' **/Makefile
>>
>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
>> ---
>>
>> Notes:
>>      v3:
>>      - no more tabs
>>      - reshuffle variable, and remove __subdir-y
>>
>>   xen/Rules.mk                         | 19 ++++++++-----------
>>   xen/arch/arm/Makefile                | 14 +++++++-------
>>   xen/arch/arm/arm32/Makefile          |  2 +-
>>   xen/arch/arm/arm64/Makefile          |  2 +-
> 
> Julien, Stefano - any chance of getting an ack here?

Stewart pointed one build issue on Arm in the cover letter. I am not 
sure where the bug lies, so I would like to hold off my ack until 
someone figure out what's going on.

> 
>>   xen/arch/x86/Makefile                | 18 +++++++++---------
>>   xen/arch/x86/acpi/Makefile           |  2 +-
>>   xen/arch/x86/cpu/Makefile            |  4 ++--
>>   xen/arch/x86/guest/Makefile          |  4 ++--
>>   xen/arch/x86/hvm/Makefile            |  6 +++---
>>   xen/arch/x86/mm/Makefile             |  4 ++--
>>   xen/arch/x86/x86_64/Makefile         |  2 +-
>>   xen/common/Makefile                  | 10 +++++-----
>>   xen/drivers/Makefile                 | 14 +++++++-------
>>   xen/drivers/acpi/Makefile            |  6 +++---
>>   xen/drivers/passthrough/Makefile     |  8 ++++----
>>   xen/drivers/passthrough/vtd/Makefile |  2 +-
> 
> Kevin, how about this one?
> 
>>   xen/lib/Makefile                     |  2 +-
>>   xen/xsm/Makefile                     |  2 +-
>>   xen/xsm/flask/Makefile               |  2 +-
> 
> Daniel, how about these?
> 
> I guess the latter two are small enough to skip further waiting for
> acks once the Arm one would be in place. Getting this patch in
> would unblock a fair part of the remainder of this series.
> 
> Jan
> 

Cheers,


-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 23/23] xen/build: use if_changed to build guest_%.o
  2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 23/23] xen/build: use if_changed to build guest_%.o Anthony PERARD
@ 2020-03-05 15:12   ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-03-05 15:12 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Tim Deegan, xen-devel,
	Roger Pau Monné

On 26.02.2020 12:33, Anthony PERARD wrote:
> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -115,6 +115,9 @@ endif
>  # FIXME LTO broken, but we would need a different way to filter -flto out
>  # $(obj-bin-y): CFLAGS := $(filter-out -flto,$(CFLAGS))
>  
> +# target with its suffix stripped
> +target-stem = $(basename $@)

I'd appreciate if the word "stem" was used in a makefile only for
what make doc uses it for - the part of the target of a pattern
rule that % matches. I.e. here perhaps name the variable
target-basename? (But see below, maybe this isn't needed.)

> --- a/xen/arch/x86/mm/Makefile
> +++ b/xen/arch/x86/mm/Makefile
> @@ -11,11 +11,14 @@ obj-y += p2m.o p2m-pt.o
>  obj-$(CONFIG_HVM) += p2m-ept.o p2m-pod.o
>  obj-y += paging.o
>  
> -guest_walk_%.o: guest_walk.c Makefile
> -	$(CC) $(c_flags) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
> +$(foreach gw,$(filter guest_walk_%.o,$(obj-y)),\
> +    $(eval CFLAGS_$(gw) = -DGUEST_PAGING_LEVELS=$$*))

So the $$* here matches ...

> -guest_walk_%.i: guest_walk.c Makefile
> -	$(CPP) $(filter-out -Wa$(comma)%,$(c_flags)) -DGUEST_PAGING_LEVELS=$* -c $< -o $@
> +guest_walk_%.o: guest_walk.c FORCE
> +	$(call if_changed_rule,cc_o_c)

... the stem of the target of this rule. This is not good. Can't
you have something like

guest_walk_%.o guest_walk_%.i guest_walk_%.s: CFLAGS_$(target-stem).o = -DGUEST_PAGING_LEVELS=$*

on a line immediately ahead of the rule, so that them having to
match up will be very obvious, and breakage of the connection
very noticable?

(Of course this also demonstrates that tying the CFLAGS modifier
to the object file name may be slightly confusing. But I don't
have a better suggestion. Question is whether here use of an
object [or whatever else] file specific variable is helpful at
all, when make already offers per-target variable customization.
Is there a specific reason the above couldn't e.g. be

guest_walk_%.o guest_walk_%.i guest_walk_%.s: CFLAGS-y += -DGUEST_PAGING_LEVELS=$*

?)

If this alternative worked, there'd be the positive side effect
of us avoiding the use of $(eval ) here - ISTR it not working
very reliably in make 3.80, which we still document as acceptable
for building.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y
  2020-03-05 15:02     ` Julien Grall
@ 2020-03-05 15:59       ` Anthony PERARD
  2020-03-05 16:31         ` Julien Grall
  0 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-03-05 15:59 UTC (permalink / raw)
  To: Julien Grall
  Cc: Kevin Tian, Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	xen-devel, Daniel De Graaf, Volodymyr Babchuk,
	Roger Pau Monné

On Thu, Mar 05, 2020 at 03:02:22PM +0000, Julien Grall wrote:
> Hi Jan,
> 
> > >   xen/Rules.mk                         | 19 ++++++++-----------
> > >   xen/arch/arm/Makefile                | 14 +++++++-------
> > >   xen/arch/arm/arm32/Makefile          |  2 +-
> > >   xen/arch/arm/arm64/Makefile          |  2 +-
> > 
> > Julien, Stefano - any chance of getting an ack here?
> 
> Stewart pointed one build issue on Arm in the cover letter. I am not sure
> where the bug lies, so I would like to hold off my ack until someone figure
> out what's going on.

The bug is in patch "[XEN PATCH v3 15/23] xen/build: have the root
Makefile generates the CFLAGS". So this patch is fine :-).

Cheers,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y
  2020-03-05 15:59       ` Anthony PERARD
@ 2020-03-05 16:31         ` Julien Grall
  0 siblings, 0 replies; 76+ messages in thread
From: Julien Grall @ 2020-03-05 16:31 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Kevin Tian, Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	xen-devel, Daniel De Graaf, Volodymyr Babchuk,
	Roger Pau Monné

Hi Anthony,

On 05/03/2020 15:59, Anthony PERARD wrote:
> On Thu, Mar 05, 2020 at 03:02:22PM +0000, Julien Grall wrote:
>> Hi Jan,
>>
>>>>    xen/Rules.mk                         | 19 ++++++++-----------
>>>>    xen/arch/arm/Makefile                | 14 +++++++-------
>>>>    xen/arch/arm/arm32/Makefile          |  2 +-
>>>>    xen/arch/arm/arm64/Makefile          |  2 +-
>>>
>>> Julien, Stefano - any chance of getting an ack here?
>>
>> Stewart pointed one build issue on Arm in the cover letter. I am not sure
>> where the bug lies, so I would like to hold off my ack until someone figure
>> out what's going on.
> 
> The bug is in patch "[XEN PATCH v3 15/23] xen/build: have the root
> Makefile generates the CFLAGS". So this patch is fine :-).

Fine :). Thank you for looking at it. You can add my Acked-by on this patch:

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements
  2020-02-27 21:17 ` [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Stewart Hildebrand
@ 2020-03-06 10:12   ` Anthony PERARD
  0 siblings, 0 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-03-06 10:12 UTC (permalink / raw)
  To: Stewart Hildebrand
  Cc: Kevin Tian, Stefano Stabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, George Dunlap, Ian Jackson, Tim Deegan,
	Jan Beulich, Andrew Cooper, xen-devel, Daniel De Graaf,
	Volodymyr Babchuk, Roger Pau Monné

On Thu, Feb 27, 2020 at 09:17:51PM +0000, Stewart Hildebrand wrote:
> Thanks for your efforts with this. With your br.build-system-xen-v3
> branch, I'm having trouble doing an aarch64 build with early printk
> enabled. I suspect the following unmerged patch that Julien authored
> last September may have some helpful information:
> https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg01207.html

Thank you Stewart for testing! And thanks for pointing out Julien's
patch, that looks like the best way to fix the problem. I'll fix the
issue.

The issue is in patch
    [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS
where EARLY_PRINTK isn't set when needed anymore.

> I tried the following...
> 
> echo "CONFIG_DEBUG=y" > xen/arch/arm/configs/arm64_defconfig
> make -C xen XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CONFIG_EARLY_PRINTK=zynqmp defconfig
> make XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CONFIG_EARLY_PRINTK=zynqmp dist-xen
> 
> ... but I was met with:
> prelink.o:(.data+0x578): undefined reference to `early_puts'
> aarch64-linux-gnu-ld: /home/stew/rpi/xen/xen/.xen-syms.0: hidden symbol `early_puts' isn't defined

Cheers,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y
  2020-03-05  9:24   ` Jan Beulich
  2020-03-05 13:42     ` Andrew Cooper
  2020-03-05 15:02     ` Julien Grall
@ 2020-03-09  6:46     ` Tian, Kevin
  2 siblings, 0 replies; 76+ messages in thread
From: Tian, Kevin @ 2020-03-09  6:46 UTC (permalink / raw)
  To: Jan Beulich, Julien Grall, Stefano Stabellini, Daniel De Graaf
  Cc: Wei Liu, Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper,
	Ian Jackson, Anthony PERARD, xen-devel, Volodymyr Babchuk,
	Roger Pau Monné

> From: Jan Beulich <jbeulich@suse.com>
> Sent: Thursday, March 5, 2020 5:24 PM
> 
> On 26.02.2020 12:33, Anthony PERARD wrote:
> > This is part of upgrading our build system and import more of Linux's
> > one.
> >
> > In Linux, subdir-y in Makefiles is only used to descend into
> > subdirectory when there are no object to build, Xen doesn't have that
> > and all subdir have object to be included in the final binary.
> >
> > To allow the new syntax, the "obj-y" and "subdir-*" calculation in
> > Rules.mk is changed and partially imported from Linux's Kbuild.
> >
> > The command used to modify the Makefile was:
> >     sed -i -r 's#^subdir-(.*)#obj-\1/#;' **/Makefile
> >
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> > ---
> >
> > Notes:
> >     v3:
> >     - no more tabs
> >     - reshuffle variable, and remove __subdir-y
> >
> >  xen/Rules.mk                         | 19 ++++++++-----------
> >  xen/arch/arm/Makefile                | 14 +++++++-------
> >  xen/arch/arm/arm32/Makefile          |  2 +-
> >  xen/arch/arm/arm64/Makefile          |  2 +-
> 
> Julien, Stefano - any chance of getting an ack here?
> 
> >  xen/arch/x86/Makefile                | 18 +++++++++---------
> >  xen/arch/x86/acpi/Makefile           |  2 +-
> >  xen/arch/x86/cpu/Makefile            |  4 ++--
> >  xen/arch/x86/guest/Makefile          |  4 ++--
> >  xen/arch/x86/hvm/Makefile            |  6 +++---
> >  xen/arch/x86/mm/Makefile             |  4 ++--
> >  xen/arch/x86/x86_64/Makefile         |  2 +-
> >  xen/common/Makefile                  | 10 +++++-----
> >  xen/drivers/Makefile                 | 14 +++++++-------
> >  xen/drivers/acpi/Makefile            |  6 +++---
> >  xen/drivers/passthrough/Makefile     |  8 ++++----
> >  xen/drivers/passthrough/vtd/Makefile |  2 +-
> 
> Kevin, how about this one?

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

> 
> >  xen/lib/Makefile                     |  2 +-
> >  xen/xsm/Makefile                     |  2 +-
> >  xen/xsm/flask/Makefile               |  2 +-
> 
> Daniel, how about these?
> 
> I guess the latter two are small enough to skip further waiting for
> acks once the Arm one would be in place. Getting this patch in
> would unblock a fair part of the remainder of this series.
> 
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 13/23] xen/build: include include/config/auto.conf in main Makefile
  2020-03-04 14:29   ` Jan Beulich
@ 2020-03-10 17:10     ` Anthony PERARD
  2020-03-11  9:26       ` Jan Beulich
  0 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-03-10 17:10 UTC (permalink / raw)
  To: Jan Beulich
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On Wed, Mar 04, 2020 at 03:29:55PM +0100, Jan Beulich wrote:
> On 26.02.2020 12:33, Anthony PERARD wrote:
> > --- a/xen/Makefile
> > +++ b/xen/Makefile
> > @@ -49,7 +49,71 @@ default: build
> >  .PHONY: dist
> >  dist: install
> >  
> > -build install:: include/config/auto.conf
> > +
> > +ifeq ($(root-make-done),)
> 
> This getting communicated between make recursion instances via ...
> 
> > +# section to run before calling Rules.mk, but only once.
> > +#
> > +# To make sure we do not include .config for any of the *config targets
> > +# catch them early, and hand them over to tools/kconfig/Makefile
> > +
> > +clean-targets := %clean
> > +no-dot-config-targets := $(clean-targets) \
> > +                         uninstall debug cloc \
> > +                         cscope TAGS tags MAP gtags \
> > +                         xenversion
> > +
> > +config-build    := n
> > +need-config     := y
> > +
> > +ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),)
> > +    ifeq ($(filter-out $(no-dot-config-targets), $(MAKECMDGOALS)),)
> > +        need-config := n
> > +    endif
> > +endif
> > +
> > +ifneq ($(filter %config,$(MAKECMDGOALS)),)
> > +    config-build := y
> > +endif
> > +
> > +export root-make-done := y
> 
> ... the environment, can we be as resilient as possible against a
> variable of this name already existing in the environment before
> the top level make invocation, by making the construct above
> 
> ifneq ($(root-make-done),y)
> 
> ?

Sound good, I'll do that.

> > +endif # root-make-done
> > +
> > +ifeq ($(config-build),y)
> > +# ===========================================================================
> > +# *config targets only - make sure prerequisites are updated, and descend
> > +# in tools/kconfig to make the *config target
> > +
> > +config: FORCE
> > +	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" $@
> 
> This, ...
> 
> > +
> > +# Config.mk tries to include .config file, don't try to remake it
> > +%/.config: ;
> > +
> > +%config: FORCE
> > +	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" $@
> 
> ... this, ...
> 
> > +else # !config-build
> > +
> > +ifeq ($(need-config),y)
> > +include include/config/auto.conf
> > +# Read in dependencies to all Kconfig* files, make sure to run syncconfig if
> > +# changes are detected.
> > +include include/config/auto.conf.cmd
> > +
> > +# Allow people to just run `make` as before and not force them to configure
> > +$(KCONFIG_CONFIG):
> > +	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" defconfig
> 
> ... this, and ...
> 
> > +# The actual configuration files used during the build are stored in
> > +# include/generated/ and include/config/. Update them if .config is newer than
> > +# include/config/auto.conf (which mirrors .config).
> > +#
> > +# This exploits the 'multi-target pattern rule' trick.
> > +# The syncconfig should be executed only once to make all the targets.
> > +include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
> > +	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" syncconfig
> 
> ... this are almost identical, pretty long lines. Can this be macroized,
> please, with the actual make goal as parameter?

Sound good, would the following be fine?

kconfig = -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)"
config:
    $(MAKE) $(kconfig) $@

I will put that new `kconfig' macro in Kbuild.include, along the
shorthand for clean.

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 14/23] xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS)
  2020-03-04 14:42   ` Jan Beulich
@ 2020-03-10 17:43     ` Anthony PERARD
  0 siblings, 0 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-03-10 17:43 UTC (permalink / raw)
  To: Jan Beulich
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Tim Deegan,
	Anthony PERARD, xen-devel, Volodymyr Babchuk,
	Roger Pau Monné

On Wed, Mar 04, 2020 at 03:42:36PM +0100, Jan Beulich wrote:
> On 26.02.2020 12:33, Anthony PERARD wrote:
> > --- a/xen/scripts/Kbuild.include
> > +++ b/xen/scripts/Kbuild.include
> > @@ -10,7 +10,7 @@ DEPS_INCLUDE = $(addsuffix .d2, $(basename $(wildcard $(DEPS))))
> >  # as-insn: Check whether assembler supports an instruction.
> >  # Usage: cflags-y += $(call as-insn,CC FLAGS,"insn",option-yes,option-no)
> >  as-insn = $(if $(shell echo 'void _(void) { asm volatile ( $(2) ); }' \
> > -                       | $(filter-out -M% %.d -include %/include/xen/config.h,$(1)) \
> > +                       | $(filter-out -include %/include/xen/config.h,$(1)) \
> >                                -c -x c -o /dev/null - 2>&1),$(4),$(3))
> 
> I'm sorry, while it was me to suggest this change - is this
> correct? The variable to modify is a parameter of this macro,
> i.e. things aren't limited to CFLAGS here. If we want to
> disallow use with e.g. c_flags or anything derived from it,
> then we should find some way to actually enforce this (like
> dropping the respective parameter; I'm uncertain though whether
> we wouldn't regret this if we ever got to the point where we
> wanted to use a newer insn in a .S file).

It is probably better to leave the macro as it is for now. I'll drop
this hunk.

I think it would be nice to have the macro use directly CFLAGS (or
XEN_CFLAGS), but since the macro is used within as-option-add, which
needs the flags variable name as parameter, we can't really change the
way as-insn works.

We could come back to that later, remove the use of as-option-add, and
have as-insn use CFLAGS (or AFLAGS) directly. That's how the macro
as-instr of Linux works, the macro always uses KBUILD_AFLAGS.

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 14/23] xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS)
  2020-02-27 10:22   ` Roger Pau Monné
@ 2020-03-10 17:55     ` Anthony PERARD
  0 siblings, 0 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-03-10 17:55 UTC (permalink / raw)
  To: Roger Pau Monné
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Tim Deegan,
	Jan Beulich, Anthony PERARD, xen-devel, Volodymyr Babchuk

On Thu, Feb 27, 2020 at 11:22:38AM +0100, Roger Pau Monné wrote:
> On Wed, Feb 26, 2020 at 11:33:46AM +0000, Anthony PERARD wrote:
> > From: Anthony PERARD <anthony.perard@gmail.com>
> > 
> > In a later patch ("xen/build: have the root Makefile generates the
> > CFLAGS), we want to generate the CFLAGS in xen/Makefile, then export
> > it and have Rules.mk use a CFLAGS from the environment variables. That
> > changes the flavor of the CFLAGS and flags intended for one target
> > (like -D__OBJECT_FILE__ and -M%) gets propagated and duplicated. So we
> > start by moving such flags out of $(CFLAGS) and into $(c_flags) which
> > is to be modified by only Rules.mk.
> > 
> > __OBJECT_FILE__ is only used by arch/x86/mm/*.c files, so having it in
> > $(c_flags) is enough, we don't need it in $(a_flags).
> 
> This seem to be used only by source files that are build multiple
> times with different parameters in order to generate different object
> files.
> 
> Is there any harm in having it also in the assembler flags? (in case
> we require such usage in the future)

Not really any harm, no, but that can be done later when needed I think.

> Or maybe we could even limit __OBJECT_FILE__ to mm/ files that require
> it only?

That's a possibility, yes. I'll be adding flags to those specific files
anyway (GUEST_PAGING_LEVELS, done in a later patch), I could add
__OBJECT_FILE__ to the list.

> > 
> > For include/Makefile and as-insn we can keep using CFLAGS, but since
> > it doesn't have -M* flags anymore there is no need to filter them out.
> > 
> > The XEN_BUILD_EFI tests in arch/x86/Makefile was filtering out
> > CFLAGS-y, but according to dd40177c1bc8 ("x86-64/EFI: add CFLAGS to
> > check compile"), it was done to filter out -MF. CFLAGS doesn't
> > have those flags anymore, so no filtering is needed.
> > 
> > This is inspired by the way Kbuild generates CFLAGS for each targets.
> > 
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 13/23] xen/build: include include/config/auto.conf in main Makefile
  2020-03-10 17:10     ` Anthony PERARD
@ 2020-03-11  9:26       ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-03-11  9:26 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On 10.03.2020 18:10, Anthony PERARD wrote:
> On Wed, Mar 04, 2020 at 03:29:55PM +0100, Jan Beulich wrote:
>> On 26.02.2020 12:33, Anthony PERARD wrote:
>>> +# The actual configuration files used during the build are stored in
>>> +# include/generated/ and include/config/. Update them if .config is newer than
>>> +# include/config/auto.conf (which mirrors .config).
>>> +#
>>> +# This exploits the 'multi-target pattern rule' trick.
>>> +# The syncconfig should be executed only once to make all the targets.
>>> +include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
>>> +	$(MAKE) -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)" syncconfig
>>
>> ... this are almost identical, pretty long lines. Can this be macroized,
>> please, with the actual make goal as parameter?
> 
> Sound good, would the following be fine?
> 
> kconfig = -f $(BASEDIR)/tools/kconfig/Makefile.kconfig ARCH=$(ARCH) SRCARCH=$(SRCARCH) HOSTCC="$(HOSTCC)" HOSTCXX="$(HOSTCXX)"
> config:
>     $(MAKE) $(kconfig) $@
> 
> I will put that new `kconfig' macro in Kbuild.include, along the
> shorthand for clean.

Looks okay at the first glance.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS
  2020-02-27 11:05   ` Roger Pau Monné
@ 2020-03-17 18:05     ` Anthony PERARD
  2020-03-19 16:24       ` Anthony PERARD
  0 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-03-17 18:05 UTC (permalink / raw)
  To: Roger Pau Monné
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	xen-devel, Daniel De Graaf, Volodymyr Babchuk

On Thu, Feb 27, 2020 at 12:05:04PM +0100, Roger Pau Monné wrote:
> On Wed, Feb 26, 2020 at 11:33:47AM +0000, Anthony PERARD wrote:
> > +ifeq ($(CONFIG_DEBUG),y)
> > +CFLAGS += -O1
> > +else
> > +CFLAGS += -O2
> > +endif
> 
> Long term we might want to make the optimization level selectable in
> Kconfig IMO.

Yep.

> > +ifneq ($(CONFIG_CC_IS_CLANG),y)
> > +# Clang doesn't understand this command line argument, and doesn't appear to
> > +# have an suitable alternative.  The resulting compiled binary does function,
> > +# but has an excessively large symbol table.
> > +CFLAGS += -Wa,--strip-local-absolute
> 
> This is not really related to clang, but to the assembler. If clang is
> used with -no-integrated-as it's quite likely that the GNU assembler
> will be used, and hence this option would be available.
> 
> Can we use cc-option-add here in order to detect whether the build
> toolchain support the option?

That can be done, but I think I'll do that as a follow up of this patch,
to avoid too many changes when moving the cflags around.

> Ideally this should be done after the integrated assembler tests
> performed in x86/Rules.mk.

> > +ifeq ($(CONFIG_UBSAN),y)
> > +CFLAGS_UBSAN := -fsanitize=undefined
> > +else
> > +CFLAGS_UBSAN :=
> 
> Do you need to define this to empty so it can be exported below? Isn't
> it enough to just not set it at all?

That has two functions, setting the correct flavor for the variable and
reset the value if the value happen to already be in the environment.
The second one is important I think.

> > +# define new variables to avoid the ones defines in Config.mk
> > +export XEN_CFLAGS := $(CFLAGS)
> > +export XEN_AFLAGS := $(AFLAGS)
> > +export XEN_LDFLAGS := $(LDFLAGS)
> > +export CFLAGS_UBSAN
> 
> You might want to rename this to XEN_CFLAGS_UBSAN for coherency with
> the rest of the exported variables?

I don't know, I think it's fine like that. They are other exported
variables, like CFLAGS-stack-boundary for flags. Not adding XEN_ prefix
to all of them mean less modifications, and less errors when doing so.

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS
  2020-03-04 15:00   ` Jan Beulich
@ 2020-03-17 18:35     ` Anthony PERARD
  2020-03-18 10:20       ` Jan Beulich
  0 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-03-17 18:35 UTC (permalink / raw)
  To: Jan Beulich
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel,
	Daniel De Graaf, Volodymyr Babchuk, Roger Pau Monné

On Wed, Mar 04, 2020 at 04:00:52PM +0100, Jan Beulich wrote:
> On 26.02.2020 12:33, Anthony PERARD wrote:
> > @@ -113,6 +115,64 @@ $(KCONFIG_CONFIG):
> > +AFLAGS += -D__ASSEMBLY__
> > +
> > +CFLAGS += $(CFLAGS-y)
> 
> I can't seem to be able to spot a similar line for AFLAGS.

I didn't add any because it wasn't necessary.

> > @@ -107,7 +65,7 @@ $(foreach o,$(filter-out %/,$(obj-y) $(obj-bin-y) $(extra-y)),$(eval $(call gend
> >  subdir-y := $(subdir-y) $(filter %/, $(obj-y))
> >  obj-y    := $(patsubst %/, %/built_in.o, $(obj-y))
> >  
> > -$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS += -DINIT_SECTIONS_ONLY
> > +$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS-y += -DINIT_SECTIONS_ONLY
> 
> While in the description you say "We can't use CFLAGS in
> subdirectories to add flags to particular targets, ...", it
> remains unclear there why that is, and hence why changes like
> this one are necessary. If this is a restriction that's going to
> remain, this also needs writing down in a prominent place. After

Yes, I should probably start writing some documentation, probably
following the kind of documentation that Linux has for kbuild, and I
could put that in docs/misc, along side the kconfig doc. That probably
not a prominent place, but I don't know if there's a better place.

> all if (for example) special compiler options are needed, CFLAGS
> would be the natural thing one would want to alter. (Even better
> if wrong playing with CFLAGS could be detected and at least
> warned about, but I'm completely unclear on how feasible this
> would be.)

The issue with the CFLAGS variable is that it is still set by Config.mk
which Rules.mk includes.
I guess I could reset the CFLAGS variable in Rules.mk after loading
Config.mk, and allow it to be used. I want to stop including Config.mk
in Rules.mk at some point anyway.

I don't think it's possible to check if a variable has been altered,
because it could be changed for a specific target where it's not easy to
check.

> > diff --git a/xen/arch/arm/Rules.mk b/xen/arch/arm/Rules.mk
> > index 022a3a6f82ba..e69de29bb2d1 100644
> > --- a/xen/arch/arm/Rules.mk
> > +++ b/xen/arch/arm/Rules.mk
> > @@ -1,93 +0,0 @@
> 
> As per the header here you're using git. Can you please arrange for
> this file movement (to arch.mk, and also for x86) to actually be
> expressed here as a rename, i.e. such that one can see what - if
> anything - changes?

That should be possible, I'll try to remember to add the right options.

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS
  2020-03-17 18:35     ` Anthony PERARD
@ 2020-03-18 10:20       ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-03-18 10:20 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel,
	Daniel De Graaf, Volodymyr Babchuk, Roger Pau Monné

On 17.03.2020 19:35, Anthony PERARD wrote:
> On Wed, Mar 04, 2020 at 04:00:52PM +0100, Jan Beulich wrote:
>> On 26.02.2020 12:33, Anthony PERARD wrote:
>>> @@ -113,6 +115,64 @@ $(KCONFIG_CONFIG):
>>> +AFLAGS += -D__ASSEMBLY__
>>> +
>>> +CFLAGS += $(CFLAGS-y)
>>
>> I can't seem to be able to spot a similar line for AFLAGS.
> 
> I didn't add any because it wasn't necessary.

Symmetry / consistency would still be desirable imo.

>>> @@ -107,7 +65,7 @@ $(foreach o,$(filter-out %/,$(obj-y) $(obj-bin-y) $(extra-y)),$(eval $(call gend
>>>   subdir-y := $(subdir-y) $(filter %/, $(obj-y))
>>>   obj-y    := $(patsubst %/, %/built_in.o, $(obj-y))
>>>   
>>> -$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS += -DINIT_SECTIONS_ONLY
>>> +$(filter %.init.o,$(obj-y) $(obj-bin-y) $(extra-y)): CFLAGS-y += -DINIT_SECTIONS_ONLY
>>
>> While in the description you say "We can't use CFLAGS in
>> subdirectories to add flags to particular targets, ...", it
>> remains unclear there why that is, and hence why changes like
>> this one are necessary. If this is a restriction that's going to
>> remain, this also needs writing down in a prominent place. After
> 
> Yes, I should probably start writing some documentation, probably
> following the kind of documentation that Linux has for kbuild, and I
> could put that in docs/misc, along side the kconfig doc. That probably
> not a prominent place, but I don't know if there's a better place.

If it's to live under docs/, then I guess xen/Rules.mk should at
least point to that documentation (perhaps in a comment near the
first setting or consumption of CFLAGS-y).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 16/23] xen/build: introduce if_changed and if_changed_rule
  2020-03-04 15:45   ` Jan Beulich
@ 2020-03-18 10:44     ` Anthony PERARD
  2020-03-18 11:14       ` Jan Beulich
  0 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-03-18 10:44 UTC (permalink / raw)
  To: Jan Beulich
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On Wed, Mar 04, 2020 at 04:45:36PM +0100, Jan Beulich wrote:
> On 26.02.2020 12:33, Anthony PERARD wrote:
> > The if_changed macro from Linux can record the command used to build a
> > target then compare it on rebuild. Thus if a command has changed, for
> > example due to introducing new flags in CFLAGS or due to using a
> > different compiler, the target will be rebuilt.
> 
> As to using a different compiler - I suppose this means "a compiler
> with a different executable name" here? What about me having, say
> gcc-5 in use, and then updating my system such that a 5.2 based
> compiler of this name would be upgraded to a 5.4 based one of this
> same name. If this newer compiler has better capabilities (that we
> would want to use if available), would this or anything else trigger
> a rebuild then too?

I think I should have written "command line" instead of just "command".
When writing about "different compiler" I was mostly thinking about GCC
vs clang, not really about versions. I think Linux has something that
detects when the compiler version changes, but that maybe to only
trigger kconfig, to regenerate the .config file.

But as you say, if the newer compiler has better capabilities, and the
*FLAGS are changed, then that would trigger a rebuild if other
dependency hasn't changed.

I'll try to reword the commit message, and copy some documentation from
Linux, since it has some for this.

> 
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -6,6 +6,7 @@
> >  *.o
> >  *.d
> >  *.d2
> > +.*.cmd
> >  *.opic
> >  *.a
> >  *.so
> 
> I admit these entries aren't sorted very well, but anyway - how
> did you end up with this insertion point? There are entries

I basically put it with the other dependency files, *.d and *.d2.

> starting with . at the very top of the file. (As an aside, I
> wonder why it's *.d and *.d2 rather than .*.d and .*.d2 .)

I'll move .*.cmd to the top and ignore that .gitignore ignore more
than necessary.

> > --- a/xen/Makefile
> > +++ b/xen/Makefile
> > @@ -52,7 +52,57 @@ dist: install
> > +ifneq ($(findstring s,$(filter-out --%,$(MAKEFLAGS))),)
> > +  quiet=silent_
> > +endif
> 
> Throughout the above, can the uses of = please become consistent?
> Preferable all with a blank on the left and - unless there's no
> value getting assigned - one on the right, plus := preferred over
> = where not prohibited by other constraints (none here afaics).

I'll try.

> > --- a/xen/scripts/Kbuild.include
> > +++ b/xen/scripts/Kbuild.include
> > @@ -2,11 +2,30 @@
> >  ####
> >  # kbuild: Generic definitions
> >  
> > +# Convenient variables
> > +squote  := '
> > +empty   :=
> > +space   := $(empty) $(empty)
> > +space_escape := _-_SPACE_-_
> > +pound := \#
> 
> Nit: To fit with the three ones above space_escape you want to
> add two blanks here.

Will do.

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 17/23] xen/build: Start using if_changed
  2020-03-04 16:00     ` Jan Beulich
@ 2020-03-18 10:52       ` Anthony PERARD
  0 siblings, 0 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-03-18 10:52 UTC (permalink / raw)
  To: Jan Beulich
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel,
	Daniel De Graaf, Volodymyr Babchuk, Roger Pau Monné

On Wed, Mar 04, 2020 at 05:00:28PM +0100, Jan Beulich wrote:
> On 27.02.2020 14:09, Roger Pau Monné wrote:
> > On Wed, Feb 26, 2020 at 11:33:49AM +0000, Anthony PERARD wrote:
> >> @@ -161,29 +173,47 @@ else
> >>  	$(CC) $(c_flags) -c $< -o $@
> >>  endif
> >>  
> >> -%.o: %.S Makefile
> >> -	$(CC) $(a_flags) -c $< -o $@
> >> +quiet_cmd_cc_o_S = CC      $@
> >> +cmd_cc_o_S = $(CC) $(a_flags) -c $< -o $@
> >> +
> >> +%.o: %.S FORCE
> >> +	$(call if_changed,cc_o_S)
> >> +
> >> +
> >> +quiet_cmd_obj_init_o = INIT_O  $@
> > 
> > INIT_O seems kind of weird, maybe just using CHECK would be OK?
> 
> CHECK is not expressing what's going on - one could/would imply
> that the object file doesn't get changed at all, but its sections
> get renamed. I think INIT_O is sufficiently expressive at least
> to people knowing the build system.

OBJCOPY instead of INIT_O could work, since it's going to read
"OBJCOPY boot.init.o" which should be obvious enough that the object
file is generated. But I'll leave INIT_O for now.

> > The rest LGTM:
> > 
> > Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 18/23] xen/build: use if_changed on built_in.o
  2020-03-04 16:03   ` Jan Beulich
@ 2020-03-18 10:55     ` Anthony PERARD
  0 siblings, 0 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-03-18 10:55 UTC (permalink / raw)
  To: Jan Beulich
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On Wed, Mar 04, 2020 at 05:03:40PM +0100, Jan Beulich wrote:
> On 26.02.2020 12:33, Anthony PERARD wrote:
> > --- a/xen/Rules.mk
> > +++ b/xen/Rules.mk
> > @@ -126,14 +126,21 @@ include $(BASEDIR)/arch/$(TARGET_ARCH)/Rules.mk
> >  c_flags += $(CFLAGS-y)
> >  a_flags += $(CFLAGS-y) $(AFLAGS-y)
> >  
> > -built_in.o: $(obj-y) $(extra-y)
> > +quiet_cmd_ld_builtin = LD      $@
> > +cmd_ld_builtin = \
> > +    $(LD) $(XEN_LDFLAGS) -r -o $@ $(filter-out $(extra-y),$(real-prereqs))
> > +quiet_cmd_cc_builtin = LD      $@
> > +cmd_cc_builtin = \
> > +    $(CC) $(XEN_CFLAGS) -c -x c /dev/null -o $@
> > +
> > +built_in.o: $(obj-y) $(extra-y) FORCE
> >  ifeq ($(obj-y),)
> > -	$(CC) $(c_flags) -c -x c /dev/null -o $@
> > +	$(call if_changed,cc_builtin)
> >  else
> >  ifeq ($(CONFIG_LTO),y)
> >  	$(LD_LTO) -r -o $@ $(filter-out $(extra-y),$^)
> 
> What about this? Couldn't you simply vary what cmd_ld_builtin
> expands to, and drop this inner ifeq()?

Yes, that should be possible. I was trying to leave CONFIG_LTO out, but
it's probably just make readability worse.

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 16/23] xen/build: introduce if_changed and if_changed_rule
  2020-03-18 10:44     ` Anthony PERARD
@ 2020-03-18 11:14       ` Jan Beulich
  0 siblings, 0 replies; 76+ messages in thread
From: Jan Beulich @ 2020-03-18 11:14 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On 18.03.2020 11:44, Anthony PERARD wrote:
> On Wed, Mar 04, 2020 at 04:45:36PM +0100, Jan Beulich wrote:
>> On 26.02.2020 12:33, Anthony PERARD wrote:
>>> The if_changed macro from Linux can record the command used to build a
>>> target then compare it on rebuild. Thus if a command has changed, for
>>> example due to introducing new flags in CFLAGS or due to using a
>>> different compiler, the target will be rebuilt.
>>
>> As to using a different compiler - I suppose this means "a compiler
>> with a different executable name" here? What about me having, say
>> gcc-5 in use, and then updating my system such that a 5.2 based
>> compiler of this name would be upgraded to a 5.4 based one of this
>> same name. If this newer compiler has better capabilities (that we
>> would want to use if available), would this or anything else trigger
>> a rebuild then too?
> 
> I think I should have written "command line" instead of just "command".
> When writing about "different compiler" I was mostly thinking about GCC
> vs clang, not really about versions. I think Linux has something that
> detects when the compiler version changes, but that maybe to only
> trigger kconfig, to regenerate the .config file.
> 
> But as you say, if the newer compiler has better capabilities, and the
> *FLAGS are changed, then that would trigger a rebuild if other
> dependency hasn't changed.

"Would" as in "would", or merely "it would be nice if it did"? I'm
simply not seeing where such a detection would be happening.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 19/23] xen/build: Use if_changed_rules with %.o:%.c targets
  2020-03-04 16:09   ` Jan Beulich
@ 2020-03-18 11:14     ` Anthony PERARD
  0 siblings, 0 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-03-18 11:14 UTC (permalink / raw)
  To: Jan Beulich
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel

On Wed, Mar 04, 2020 at 05:09:19PM +0100, Jan Beulich wrote:
> On 26.02.2020 12:33, Anthony PERARD wrote:
> > +define rule_cc_o_c
> > +    $(call cmd_and_record,cc_o_c)
> > +    $(call cmd,objcopy_fix_sym)
> 
> The machinery is resilient to a command (here: cmd_objcopy_fix_sym)
> not being defined, and will neither produce any undue output nor
> else incur any unnecessary overhead?

Yes, it's fine when cmd_objcopy_fix_sym isn't defined, nothing gets
printed on the console, and there is no error.

As for unnecessary overhead, I don't know. The macro still expand to
"@set -e;". But Linux uses that a lot (having undefined cmd_*), so I
guess it's not too bad.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 20/23] xen/build: factorise generation of the linker scripts
  2020-03-05 11:05   ` Jan Beulich
@ 2020-03-18 11:59     ` Anthony PERARD
  0 siblings, 0 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-03-18 11:59 UTC (permalink / raw)
  To: Jan Beulich
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, xen-devel,
	Volodymyr Babchuk, Roger Pau Monné

On Thu, Mar 05, 2020 at 12:05:02PM +0100, Jan Beulich wrote:
> On 26.02.2020 12:33, Anthony PERARD wrote:
> > --- a/xen/Rules.mk
> > +++ b/xen/Rules.mk
> > @@ -230,6 +230,14 @@ cmd_s_S = $(CPP) $(filter-out -Wa$(comma)%,$(a_flags)) $< -o $@
> >  %.s: %.S FORCE
> >  	$(call if_changed,cpp_s_S)
> >  
> > +# Linker scripts, .lds.S -> .lds
> > +quiet_cmd_cc_lds_S = LDS     $@
> > +define cmd_cc_lds_S
> > +    $(CC) -P -E -Ui386 $(filter-out -Wa$(comma)%,$(a_flags)) -o $@ $<; \
> 
> $(CPP)? And then also name the thing cmd_cpp_lds_S?

Will do.

> > +    sed -e 's/.*\.lds\.o:/$(@F):/g' <$(dot-target).d >$(dot-target).d.new; \
> > +    mv -f $(dot-target).d.new $(dot-target).d
> 
> This would benefit from also switching to move-if-changed at
> this occasion.

I don't think using move-if-changed here is a good idea. The *.lds file
should be generated if it's dependency *.lds.S changed.

move-if-changed might prevent some rebuild, but the *.lds file will be
rebuild over and over again. I think move-if-changed is only useful if a
file needs to be generated at every make invocation.

> With you using "define" - is there really a need for adding the
> trailing "; \" sequence to the first two lines of the macro?

I think it is, but I'll check again if it's necessary. Maybe just ';' is
enough.

> > --- a/xen/arch/x86/Makefile
> > +++ b/xen/arch/x86/Makefile
> > @@ -75,6 +75,7 @@ obj-y += hpet.o
> >  obj-y += vm_event.o
> >  obj-y += xstate.o
> >  extra-y += asm-macros.i
> > +extra-y += xen.lds
> >  
> >  x86_emulate.o: x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h
> >  
> > @@ -197,6 +198,7 @@ endif
> >  note_file_option ?= $(note_file)
> >  
> >  ifeq ($(XEN_BUILD_PE),y)
> > +extra-y += efi.lds
> 
> Would be nice if this was moved up using
> 
> extra-$(XEN_BUILD_PE) += efi.lds

I can try, but XEN_BUILD_PE is defined in the middle of the Makefile, so
I wouldn't be able to move that with the other obj-y and extra-y
definitions.

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS
  2020-03-17 18:05     ` Anthony PERARD
@ 2020-03-19 16:24       ` Anthony PERARD
  2020-03-23 15:11         ` Roger Pau Monné
  0 siblings, 1 reply; 76+ messages in thread
From: Anthony PERARD @ 2020-03-19 16:24 UTC (permalink / raw)
  To: Roger Pau Monné
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	xen-devel, Daniel De Graaf, Volodymyr Babchuk

On Tue, Mar 17, 2020 at 06:05:24PM +0000, Anthony PERARD wrote:
> On Thu, Feb 27, 2020 at 12:05:04PM +0100, Roger Pau Monné wrote:
> > On Wed, Feb 26, 2020 at 11:33:47AM +0000, Anthony PERARD wrote:
> > > +ifneq ($(CONFIG_CC_IS_CLANG),y)
> > > +# Clang doesn't understand this command line argument, and doesn't appear to
> > > +# have an suitable alternative.  The resulting compiled binary does function,
> > > +# but has an excessively large symbol table.
> > > +CFLAGS += -Wa,--strip-local-absolute
> > 
> > This is not really related to clang, but to the assembler. If clang is
> > used with -no-integrated-as it's quite likely that the GNU assembler
> > will be used, and hence this option would be available.
> > 
> > Can we use cc-option-add here in order to detect whether the build
> > toolchain support the option?
> 
> That can be done, but I think I'll do that as a follow up of this patch,
> to avoid too many changes when moving the cflags around.
> 
> > Ideally this should be done after the integrated assembler tests
> > performed in x86/Rules.mk.

So, testing for the -Wa,--strip-local-absolute flags turns out to be
more complicated than I though it would be.
 - cc-option-add doesn't work because it doesn't test with the current list
   of CFLAGS. And if I add the CFLAGS, clang says the option is unused,
   it doesn't matter if -no-integrated-as is present or not.
 - I tried to use as-option macro from Linux but that comes with issues
   as well. I don't think that enough work as been done to make it work
   well with clang. (I probably need to filter -Wall out of the CFLAGS
   when testing, and they are probably other issues.)

So I think I leave the enhancement for later. Having the flag depends
on GCC is good enough for now.

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS
  2020-03-19 16:24       ` Anthony PERARD
@ 2020-03-23 15:11         ` Roger Pau Monné
  2020-03-24 17:11           ` [Xen-devel] " Anthony PERARD
  0 siblings, 1 reply; 76+ messages in thread
From: Roger Pau Monné @ 2020-03-23 15:11 UTC (permalink / raw)
  To: Anthony PERARD
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	xen-devel, Daniel De Graaf, Volodymyr Babchuk

On Thu, Mar 19, 2020 at 04:24:12PM +0000, Anthony PERARD wrote:
> On Tue, Mar 17, 2020 at 06:05:24PM +0000, Anthony PERARD wrote:
> > On Thu, Feb 27, 2020 at 12:05:04PM +0100, Roger Pau Monné wrote:
> > > On Wed, Feb 26, 2020 at 11:33:47AM +0000, Anthony PERARD wrote:
> > > > +ifneq ($(CONFIG_CC_IS_CLANG),y)
> > > > +# Clang doesn't understand this command line argument, and doesn't appear to
> > > > +# have an suitable alternative.  The resulting compiled binary does function,
> > > > +# but has an excessively large symbol table.
> > > > +CFLAGS += -Wa,--strip-local-absolute
> > > 
> > > This is not really related to clang, but to the assembler. If clang is
> > > used with -no-integrated-as it's quite likely that the GNU assembler
> > > will be used, and hence this option would be available.
> > > 
> > > Can we use cc-option-add here in order to detect whether the build
> > > toolchain support the option?
> > 
> > That can be done, but I think I'll do that as a follow up of this patch,
> > to avoid too many changes when moving the cflags around.
> > 
> > > Ideally this should be done after the integrated assembler tests
> > > performed in x86/Rules.mk.
> 
> So, testing for the -Wa,--strip-local-absolute flags turns out to be
> more complicated than I though it would be.
>  - cc-option-add doesn't work because it doesn't test with the current list
>    of CFLAGS. And if I add the CFLAGS, clang says the option is unused,
>    it doesn't matter if -no-integrated-as is present or not.

Oh, that seems like completely bogus clang behavior. It's my
understanding (from reading the manual) that whatever gets appended to
-Wa,<arg> is just passed to the assembler, in which case the compiler
has no idea whether it's used by it or not.

Which version of clang did you use to test it?

Isn't it fine to just attempt to test if it works, and if the compiler
complains just don't append it? (regardless of whether
-no-integrated-as is used or not)

>  - I tried to use as-option macro from Linux but that comes with issues
>    as well. I don't think that enough work as been done to make it work
>    well with clang. (I probably need to filter -Wall out of the CFLAGS
>    when testing, and they are probably other issues.)
> 
> So I think I leave the enhancement for later. Having the flag depends
> on GCC is good enough for now.

Right, until clang adds support for it there's no pressure.

Thanks, Roger.


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Xen-devel] [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS
  2020-03-23 15:11         ` Roger Pau Monné
@ 2020-03-24 17:11           ` " Anthony PERARD
  0 siblings, 0 replies; 76+ messages in thread
From: Anthony PERARD @ 2020-03-24 17:11 UTC (permalink / raw)
  To: Roger Pau Monné
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Ian Jackson, Jan Beulich,
	xen-devel, Daniel De Graaf, Volodymyr Babchuk

On Mon, Mar 23, 2020 at 04:11:53PM +0100, Roger Pau Monné wrote:
> On Thu, Mar 19, 2020 at 04:24:12PM +0000, Anthony PERARD wrote:
> > So, testing for the -Wa,--strip-local-absolute flags turns out to be
> > more complicated than I though it would be.
> >  - cc-option-add doesn't work because it doesn't test with the current list
> >    of CFLAGS. And if I add the CFLAGS, clang says the option is unused,
> >    it doesn't matter if -no-integrated-as is present or not.
> 
> Oh, that seems like completely bogus clang behavior. It's my
> understanding (from reading the manual) that whatever gets appended to
> -Wa,<arg> is just passed to the assembler, in which case the compiler
> has no idea whether it's used by it or not.
> 
> Which version of clang did you use to test it?

Probably 9.0.1, I don't think I've upgraded since my tests.

-- 
Anthony PERARD


^ permalink raw reply	[flat|nested] 76+ messages in thread

end of thread, back to index

Thread overview: 76+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-26 11:33 [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Anthony PERARD
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 01/23] xen/include: remove include of Config.mk Anthony PERARD
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 02/23] Makefile: Fix install-tests Anthony PERARD
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 03/23] xen/build: Remove confusing comment on the %.s:%.S rule Anthony PERARD
2020-02-26 11:53   ` Wei Liu
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 04/23] xen/build: remove use of AFLAGS-y Anthony PERARD
2020-02-26 12:58   ` Jan Beulich
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 05/23] xen/build: Allow to test clang .include without asm symlink Anthony PERARD
2020-02-27  9:05   ` Roger Pau Monné
2020-02-27  9:22     ` Jan Beulich
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 06/23] xen/build: Fix section-renaming of libfdt and libelf Anthony PERARD
2020-02-27  9:38   ` Jan Beulich
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 07/23] xen/build: Use obj-y += subdir/ instead of subdir-y Anthony PERARD
2020-02-27  9:22   ` Roger Pau Monné
2020-02-27  9:43   ` Jan Beulich
2020-03-04 14:14     ` Jan Beulich
2020-03-05  9:24   ` Jan Beulich
2020-03-05 13:42     ` Andrew Cooper
2020-03-05 15:02     ` Julien Grall
2020-03-05 15:59       ` Anthony PERARD
2020-03-05 16:31         ` Julien Grall
2020-03-09  6:46     ` Tian, Kevin
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 08/23] xen/build: use $(clean) shorthand for clean targets Anthony PERARD
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 09/23] xen/build: extract clean target from Rules.mk Anthony PERARD
2020-03-04 14:13   ` Jan Beulich
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 10/23] xen/build: run targets csopes, tags, .. without Rules.mk Anthony PERARD
2020-03-04 14:17   ` Jan Beulich
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 11/23] xen/build: make tests in test/ directly Anthony PERARD
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 12/23] xen/build: Move as-option-add to xen/ Anthony PERARD
2020-03-04 14:17   ` Jan Beulich
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 13/23] xen/build: include include/config/auto.conf in main Makefile Anthony PERARD
2020-03-04 14:29   ` Jan Beulich
2020-03-10 17:10     ` Anthony PERARD
2020-03-11  9:26       ` Jan Beulich
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 14/23] xen/build: use new $(c_flags) and $(a_flags) instead of $(CFLAGS) Anthony PERARD
2020-02-27 10:22   ` Roger Pau Monné
2020-03-10 17:55     ` Anthony PERARD
2020-03-04 14:42   ` Jan Beulich
2020-03-10 17:43     ` Anthony PERARD
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 15/23] xen/build: have the root Makefile generates the CFLAGS Anthony PERARD
2020-02-27 11:05   ` Roger Pau Monné
2020-03-17 18:05     ` Anthony PERARD
2020-03-19 16:24       ` Anthony PERARD
2020-03-23 15:11         ` Roger Pau Monné
2020-03-24 17:11           ` [Xen-devel] " Anthony PERARD
2020-03-04 15:00   ` Jan Beulich
2020-03-17 18:35     ` Anthony PERARD
2020-03-18 10:20       ` Jan Beulich
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 16/23] xen/build: introduce if_changed and if_changed_rule Anthony PERARD
2020-03-04 15:45   ` Jan Beulich
2020-03-18 10:44     ` Anthony PERARD
2020-03-18 11:14       ` Jan Beulich
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 17/23] xen/build: Start using if_changed Anthony PERARD
2020-02-27 13:09   ` Roger Pau Monné
2020-03-04 16:00     ` Jan Beulich
2020-03-18 10:52       ` Anthony PERARD
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 18/23] xen/build: use if_changed on built_in.o Anthony PERARD
2020-03-04 16:03   ` Jan Beulich
2020-03-18 10:55     ` Anthony PERARD
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 19/23] xen/build: Use if_changed_rules with %.o:%.c targets Anthony PERARD
2020-03-04 16:09   ` Jan Beulich
2020-03-18 11:14     ` Anthony PERARD
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 20/23] xen/build: factorise generation of the linker scripts Anthony PERARD
2020-02-27 13:14   ` Roger Pau Monné
2020-03-05 11:07     ` Jan Beulich
2020-03-05 11:05   ` Jan Beulich
2020-03-18 11:59     ` Anthony PERARD
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 21/23] xen/build: Use if_changed for prelink*.o Anthony PERARD
2020-02-27 13:16   ` Roger Pau Monné
2020-03-04 16:12     ` Jan Beulich
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 22/23] xen, symbols: rework file symbols selection Anthony PERARD
2020-03-05 14:44   ` Jan Beulich
2020-02-26 11:33 ` [Xen-devel] [XEN PATCH v3 23/23] xen/build: use if_changed to build guest_%.o Anthony PERARD
2020-03-05 15:12   ` Jan Beulich
2020-02-27 21:17 ` [Xen-devel] [XEN PATCH v3 00/23] xen: Build system improvements Stewart Hildebrand
2020-03-06 10:12   ` Anthony PERARD

Xen-Devel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/xen-devel/0 xen-devel/git/0.git
	git clone --mirror https://lore.kernel.org/xen-devel/1 xen-devel/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 xen-devel xen-devel/ https://lore.kernel.org/xen-devel \
		xen-devel@lists.xenproject.org xen-devel@lists.xen.org
	public-inbox-index xen-devel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.xenproject.lists.xen-devel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git