Re: [PATCH v9 02/10] Makefile: Prepare for using macros for inline asm

[Nadav Amit <namit@vmware.com>]

From: Nadav Amit
Sent: November 8, 2018 at 8:18:23 PM GMT
> To: Logan Gunthorpe <logang@deltatee.com>, hpa@zytor.com <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>
> Cc: LKML <linux-kernel@vger.kernel.org>, X86 ML <x86@kernel.org>, Sam Ravnborg <sam@ravnborg.org>, Michal Marek <michal.lkml@markovi.net>, Thomas Gleixner <tglx@linutronix.de>, Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>, Stephen Bates <sbates@raithlin.com>
> Subject: Re: [PATCH v9 02/10] Makefile: Prepare for using macros for inline asm
> 
> 
> From: Logan Gunthorpe
> Sent: November 8, 2018 at 8:00:33 PM GMT
>> To: Nadav Amit <namit@vmware.com>, hpa@zytor.com <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>
>> Cc: LKML <linux-kernel@vger.kernel.org>, X86 ML <x86@kernel.org>, Sam Ravnborg <sam@ravnborg.org>, Michal Marek <michal.lkml@markovi.net>, Thomas Gleixner <tglx@linutronix.de>, Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>, Stephen Bates <sbates@raithlin.com>
>> Subject: Re: [PATCH v9 02/10] Makefile: Prepare for using macros for inline asm
>> 
>> 
>> 
>> 
>> On 2018-11-08 12:54 p.m., Nadav Amit wrote:
>>> I don’t think the assembly stage needs to be done locally. gcc can still be
>>> used to deploy the assembler. I am not too familiar with distcc, so I don’t
>>> know whether the preprocessing supports multiple source-files, and whether
>>> it has some strange-behavior when it comes to .S/.s files.
>> 
>> The problem is that it's the assembly stage that needs the extra
>> macros.s and without that file being transferred somehow, that stage
>> must be done locally.
> 
> I understand, but the idea is to have two stages, for instance:
> 
>  gcc ... -S -o .tmp_[unit].S [unit].c
> 
> and then
> 
>  gcc ... -D__ASSEMBLY__ arch/x86/kernel/macros.S .tmp_[unit].S
> 
> (Yes, I realize the .tmp_[unit].S was already preprocessed, but this way you
> can combine both inputs)
> 
> Unfortunately, as I write this email (and run tests) I realize distcc is too
> dumb to handle two input files:
> 
>  "(dcc_scan_args) do we have two inputs?  i give up "
> 
> Just great. So I guess macros.s would need to be concatenated with
> .tmp_[unit].s as a separate (local) interim stage.

Err.. I hate makefiles and distcc doesn’t make life easier. For instance, if
it sees two source files on the command-line, it freaks out and compiles it
locally. It also has an option to distribute assembly, but it is not enabled
by default.

Anyhow, can you try the following patch?

-- >8 --

Subject: [PATCH] Makefile: Fix distcc compilation with x86 macros

Introducing the use of asm macros in c-code broke distcc, since it only
sends the preprocessed source file. The solution is to break the
compilation into two separate phases of compilation and assembly, and
between the two concatanate the assembly macros and the compiled (yet
not assembled) source file. Since this is less efficient, this
compilation mode is only used when make is called with the "DISTCC=y"
parameter.

Note that the assembly stage should also be distributable, if distcc is
configured using "CFLAGS=-DENABLE_REMOTE_ASSEMBLE".

Reported-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Nadav Amit <namit@vmware.com>
---
 Makefile               |  4 +++-
 arch/x86/Makefile      |  4 +++-
 scripts/Makefile.build | 29 +++++++++++++++++++++++++++--
 3 files changed, 33 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index 9fce8b91c15f..c07349fc38c7 100644
--- a/Makefile
+++ b/Makefile
@@ -743,7 +743,9 @@ KBUILD_CFLAGS   += $(call cc-option, -gsplit-dwarf, -g)
 else
 KBUILD_CFLAGS	+= -g
 endif
-KBUILD_AFLAGS	+= -Wa,-gdwarf-2
+AFLAGS_DEBUG_INFO = -Wa,-gdwarf-2
+export AFLAGS_DEBUG_INFO
+KBUILD_AFLAGS	+= $(AFLAGS_DEBUG_INFO)
 endif
 ifdef CONFIG_DEBUG_INFO_DWARF4
 KBUILD_CFLAGS	+= $(call cc-option, -gdwarf-4,)
diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index f5d7f4134524..080bd9cbc4e1 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -238,7 +238,9 @@ archheaders:
 archmacros:
 	$(Q)$(MAKE) $(build)=arch/x86/kernel arch/x86/kernel/macros.s
 
-ASM_MACRO_FLAGS = -Wa,arch/x86/kernel/macros.s
+ASM_MACRO_FILE = arch/x86/kernel/macros.s
+export ASM_MACRO_FILE
+ASM_MACRO_FLAGS = -Wa,$(ASM_MACRO_FILE)
 export ASM_MACRO_FLAGS
 KBUILD_CFLAGS += $(ASM_MACRO_FLAGS)
 
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 6a6be9f440cf..2b79789a3e10 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -155,8 +155,33 @@ $(obj)/%.ll: $(src)/%.c FORCE
 
 quiet_cmd_cc_o_c = CC $(quiet_modtag)  $@
 
+# If distcc is used, then when an assembly macro files is needed, the
+# compilation stage and the assembly stage need to be separated. Providing
+# "DISTCC=y" option enables the separate compilation and assembly.
+cmd_cc_o_c_simple = $(CC) $(c_flags) -c -o $(1) $<
+
+ifeq ($(DISTCC),y)
+a_flags_no_debug = $(filter-out $(AFLAGS_DEBUG_INFO), $(a_flags))
+c_flags_no_macros = $(filter-out $(ASM_MACRO_FLAGS), $(c_flags))
+
+cmd_cc_o_c_two_steps =							\
+	$(CC) $(c_flags_no_macros) $(DISABLE_LTO) -fverbose-asm -S 	\
+		-o $(@D)/.$(@F:.o=.s) $< ;				\
+	cat $(ASM_MACRO_FILE) $(@D)/.$(@F:.o=.s) > 			\
+		$(@D)/.tmp_$(@F:.o=.s);					\
+	$(CC) $(a_flags_no_debug) -c -o $(1) $(@D)/.tmp_$(@F:.o=.s) ; 	\
+	rm -f $(@D)/.$(@F:.o=.s) $(@D)/.tmp_$(@F:.o=.s)			\
+
+cmd_cc_o_c_helper =    							\
+	$(if $(findstring $(ASM_MACRO_FLAGS),$(c_flags)),		\
+		$(call cmd_cc_o_c_two_steps, $(1)),			\
+		$(call cmd_cc_o_c_simple, $(1)))
+else
+cmd_cc_o_c_helper = $(call cmd_cc_o_c_simple, $(1))
+endif
+
 ifndef CONFIG_MODVERSIONS
-cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $<
+cmd_cc_o_c = $(call cmd_cc_o_c_helper,$@)
 
 else
 # When module versioning is enabled the following steps are executed:
@@ -171,7 +196,7 @@ else
 #   replace the unresolved symbols __crc_exported_symbol with
 #   the actual value of the checksum generated by genksyms
 
-cmd_cc_o_c = $(CC) $(c_flags) -c -o $(@D)/.tmp_$(@F) $<
+cmd_cc_o_c = $(call cmd_cc_o_c_helper,$(@D)/.tmp_$(@F))
 
 cmd_modversions_c =								\
 	if $(OBJDUMP) -h $(@D)/.tmp_$(@F) | grep -q __ksymtab; then		\
-- 
2.17.1