[PATCH v2] Makefile.llvm: simplify LLVM build

[PATCH v2] Makefile.llvm: simplify LLVM build

[Nick Desaulniers]

Prior to this patch, building the Linux kernel with Clang
looked like:

$ make CC=clang

or when cross compiling:

$ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang

which got very verbose and unwieldy when using all of LLVM's substitutes
for GNU binutils:

$ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang AS=clang \
  LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \
  OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump OBJSIZE=llvm-objsize \
  READELF=llvm-readelf HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar \
  HOSTLD=ld.lld

This change adds a new Makefile under scripts/ which will be included in
the top level Makefile AFTER CC and friends are set, in order to make
the use of LLVM utilities when building a Linux kernel more ergonomic.

With this patch, the above now looks like:

$ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y

Then you can "opt out" of certain LLVM utilities explicitly:

$ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y AS=as

will instead invoke arm-linux-gnueabihf-as in place of clang for AS.

Or when not cross compiling:

$ make LLVM=y AS=as

This would make it more verbose to opt into just one tool from LLVM, but
this patch doesn't actually break the old style; just leave off LLVM=y.
Also, LLVM=y CC=clang would wind up prefixing clang with $CROSS_COMPILE.
In that case, it's recommended to just drop LLVM=y and use the old
style. So LLVM=y can be thought of as default to LLVM with explicit opt
ins for GNU, vs the current case of default to GNU and opt in for LLVM.

A key part of the design of this patch is to be minimally invasive to
the top level Makefile and not break existing workflows. We could get
more aggressive, but I'd prefer to save larger refactorings for another
day.

Finally, some linux distributions package specific versions of LLVM
utilities with naming conventions that use the version as a suffix, ie.
clang-11.  In that case, you can use LLVM=<number> and that number will
be used as a suffix. Example:

$ make LLVM=11

will invoke clang-11, ld.lld-11, llvm-objcopy-11, etc.

About the script:
The pattern used in the script is in the form:

ifeq "$(origin $(CC))" "file"
$(CC) := $(clang)
else
override $(CC) := $(CROSS_COMPILE)$(CC)
endif

"Metaprogramming" (eval) is used to template the above to make it more
concise for specifying all of the substitutions.

The "origin" of a variable tracks whether a variable was explicitly set
via "command line", "environment", was defined earlier via Makefile
"file", was provided by "default", or was "undefined".

Variable assignment in GNU Make has some special and complicated rules.

If the variable was set earlier explicitly in the Makefile, we can
simply reassign a new value to it. If a variable was unspecified, then
earlier assignments were executed and change the origin to file.
Otherwise, the variable was explicitly specified.

If a variable's "origin" was "command line" or "environment",
non-"override" assignments are not executed. The "override" directive
forces the assignment regardless of "origin".

Some tips I found useful for debugging for future travelers:

$(info $$origin of $$CC is $(origin CC))

at the start of the new script for all of the variables can help you
understand "default" vs "undefined" variable origins.

$(info $$CC is [${CC}])

in the top level Makefile after including the new script, for all of the
variables can help you check that they're being set as expected.

Link: https://www.gnu.org/software/make/manual/html_node/Eval-Function.html
Link: https://www.gnu.org/software/make/manual/html_node/Origin-Function.html
Link: https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
Link: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html
Suggested-by: Nathan Chancellor <natechancellor@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
Changes V1 -> V2:
* Rather than LLVM=1, use LLVM=y to enable all.
* LLVM=<anything other than y> becomes a suffix, LLVM_SUFFIX.
* strip has to be used on the LLVM_SUFFIX to avoid an extra whitespace.


 Makefile              |  4 ++++
 scripts/Makefile.llvm | 30 ++++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100644 scripts/Makefile.llvm

diff --git a/Makefile b/Makefile
index 402f276da062..72ec9dfea15e 100644
--- a/Makefile
+++ b/Makefile
@@ -475,6 +475,10 @@ KBUILD_LDFLAGS :=
 GCC_PLUGINS_CFLAGS :=
 CLANG_FLAGS :=
 
+ifneq ($(LLVM),)
+include scripts/Makefile.llvm
+endif
+
 export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE AS LD CC
 export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL
 export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
diff --git a/scripts/Makefile.llvm b/scripts/Makefile.llvm
new file mode 100644
index 000000000000..0bab45a100a3
--- /dev/null
+++ b/scripts/Makefile.llvm
@@ -0,0 +1,30 @@
+LLVM_SUFFIX=
+
+ifneq ($(LLVM),y)
+LLVM_SUFFIX += -$(LLVM)
+endif
+
+define META_set =
+ifeq "$(origin $(1))" "file"
+$(1) := $(2)$(strip $(LLVM_SUFFIX))
+else
+override $(1) := $(CROSS_COMPILE)$($(1))
+endif
+endef
+
+$(eval $(call META_set,CC,clang))
+$(eval $(call META_set,AS,clang))
+$(eval $(call META_set,LD,ld.lld))
+$(eval $(call META_set,AR,llvm-ar))
+$(eval $(call META_set,NM,llvm-nm))
+$(eval $(call META_set,STRIP,llvm-strip))
+$(eval $(call META_set,OBJCOPY,llvm-objcopy))
+$(eval $(call META_set,OBJDUMP,llvm-objdump))
+$(eval $(call META_set,OBJSIZE,llvm-objsize))
+$(eval $(call META_set,READELF,llvm-readelf))
+$(eval $(call META_set,HOSTCC,clang))
+$(eval $(call META_set,HOSTCXX,clang++))
+$(eval $(call META_set,HOSTAR,llvm-ar))
+$(eval $(call META_set,HOSTLD,ld.lld))
+
+## TODO: HOSTAR, HOSTLD in tools/objtool/Makefile
-- 
2.25.1.481.gfbce0eb801-goog

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Nick Desaulniers]

Masahiro and Nathan,
I get daily reminders that `make CC=clang LD=ld.lld NM=llvm-nm ...`
sucks. I always prioritize code reviews for you, can you please carve
out time to review+test+provide feedback for me?

On Tue, Mar 17, 2020 at 2:55 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> Prior to this patch, building the Linux kernel with Clang
> looked like:
>
> $ make CC=clang
>
> or when cross compiling:
>
> $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang
>
> which got very verbose and unwieldy when using all of LLVM's substitutes
> for GNU binutils:
>
> $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang AS=clang \
>   LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \
>   OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump OBJSIZE=llvm-objsize \
>   READELF=llvm-readelf HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar \
>   HOSTLD=ld.lld
>
> This change adds a new Makefile under scripts/ which will be included in
> the top level Makefile AFTER CC and friends are set, in order to make
> the use of LLVM utilities when building a Linux kernel more ergonomic.
>
> With this patch, the above now looks like:
>
> $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y
>
> Then you can "opt out" of certain LLVM utilities explicitly:
>
> $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y AS=as
>
> will instead invoke arm-linux-gnueabihf-as in place of clang for AS.
>
> Or when not cross compiling:
>
> $ make LLVM=y AS=as
>
> This would make it more verbose to opt into just one tool from LLVM, but
> this patch doesn't actually break the old style; just leave off LLVM=y.
> Also, LLVM=y CC=clang would wind up prefixing clang with $CROSS_COMPILE.
> In that case, it's recommended to just drop LLVM=y and use the old
> style. So LLVM=y can be thought of as default to LLVM with explicit opt
> ins for GNU, vs the current case of default to GNU and opt in for LLVM.
>
> A key part of the design of this patch is to be minimally invasive to
> the top level Makefile and not break existing workflows. We could get
> more aggressive, but I'd prefer to save larger refactorings for another
> day.
>
> Finally, some linux distributions package specific versions of LLVM
> utilities with naming conventions that use the version as a suffix, ie.
> clang-11.  In that case, you can use LLVM=<number> and that number will
> be used as a suffix. Example:
>
> $ make LLVM=11
>
> will invoke clang-11, ld.lld-11, llvm-objcopy-11, etc.
>
> About the script:
> The pattern used in the script is in the form:
>
> ifeq "$(origin $(CC))" "file"
> $(CC) := $(clang)
> else
> override $(CC) := $(CROSS_COMPILE)$(CC)
> endif
>
> "Metaprogramming" (eval) is used to template the above to make it more
> concise for specifying all of the substitutions.
>
> The "origin" of a variable tracks whether a variable was explicitly set
> via "command line", "environment", was defined earlier via Makefile
> "file", was provided by "default", or was "undefined".
>
> Variable assignment in GNU Make has some special and complicated rules.
>
> If the variable was set earlier explicitly in the Makefile, we can
> simply reassign a new value to it. If a variable was unspecified, then
> earlier assignments were executed and change the origin to file.
> Otherwise, the variable was explicitly specified.
>
> If a variable's "origin" was "command line" or "environment",
> non-"override" assignments are not executed. The "override" directive
> forces the assignment regardless of "origin".
>
> Some tips I found useful for debugging for future travelers:
>
> $(info $$origin of $$CC is $(origin CC))
>
> at the start of the new script for all of the variables can help you
> understand "default" vs "undefined" variable origins.
>
> $(info $$CC is [${CC}])
>
> in the top level Makefile after including the new script, for all of the
> variables can help you check that they're being set as expected.
>
> Link: https://www.gnu.org/software/make/manual/html_node/Eval-Function.html
> Link: https://www.gnu.org/software/make/manual/html_node/Origin-Function.html
> Link: https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
> Link: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html
> Suggested-by: Nathan Chancellor <natechancellor@gmail.com>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
> Changes V1 -> V2:
> * Rather than LLVM=1, use LLVM=y to enable all.
> * LLVM=<anything other than y> becomes a suffix, LLVM_SUFFIX.
> * strip has to be used on the LLVM_SUFFIX to avoid an extra whitespace.
>
>
>  Makefile              |  4 ++++
>  scripts/Makefile.llvm | 30 ++++++++++++++++++++++++++++++
>  2 files changed, 34 insertions(+)
>  create mode 100644 scripts/Makefile.llvm
>
> diff --git a/Makefile b/Makefile
> index 402f276da062..72ec9dfea15e 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -475,6 +475,10 @@ KBUILD_LDFLAGS :=
>  GCC_PLUGINS_CFLAGS :=
>  CLANG_FLAGS :=
>
> +ifneq ($(LLVM),)
> +include scripts/Makefile.llvm
> +endif
> +
>  export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE AS LD CC
>  export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL
>  export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
> diff --git a/scripts/Makefile.llvm b/scripts/Makefile.llvm
> new file mode 100644
> index 000000000000..0bab45a100a3
> --- /dev/null
> +++ b/scripts/Makefile.llvm
> @@ -0,0 +1,30 @@
> +LLVM_SUFFIX=
> +
> +ifneq ($(LLVM),y)
> +LLVM_SUFFIX += -$(LLVM)
> +endif
> +
> +define META_set =
> +ifeq "$(origin $(1))" "file"
> +$(1) := $(2)$(strip $(LLVM_SUFFIX))
> +else
> +override $(1) := $(CROSS_COMPILE)$($(1))
> +endif
> +endef
> +
> +$(eval $(call META_set,CC,clang))
> +$(eval $(call META_set,AS,clang))
> +$(eval $(call META_set,LD,ld.lld))
> +$(eval $(call META_set,AR,llvm-ar))
> +$(eval $(call META_set,NM,llvm-nm))
> +$(eval $(call META_set,STRIP,llvm-strip))
> +$(eval $(call META_set,OBJCOPY,llvm-objcopy))
> +$(eval $(call META_set,OBJDUMP,llvm-objdump))
> +$(eval $(call META_set,OBJSIZE,llvm-objsize))
> +$(eval $(call META_set,READELF,llvm-readelf))
> +$(eval $(call META_set,HOSTCC,clang))
> +$(eval $(call META_set,HOSTCXX,clang++))
> +$(eval $(call META_set,HOSTAR,llvm-ar))
> +$(eval $(call META_set,HOSTLD,ld.lld))
> +
> +## TODO: HOSTAR, HOSTLD in tools/objtool/Makefile
> --
> 2.25.1.481.gfbce0eb801-goog
>


-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Nathan Chancellor]

Sorry for the delay in review :(

On Tue, Mar 17, 2020 at 02:55:15PM -0700, Nick Desaulniers wrote:
> Prior to this patch, building the Linux kernel with Clang
> looked like:
> 
> $ make CC=clang
> 
> or when cross compiling:
> 
> $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang
> 
> which got very verbose and unwieldy when using all of LLVM's substitutes
> for GNU binutils:
> 
> $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang AS=clang \
>   LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \
>   OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump OBJSIZE=llvm-objsize \
>   READELF=llvm-readelf HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar \
>   HOSTLD=ld.lld
> 
> This change adds a new Makefile under scripts/ which will be included in
> the top level Makefile AFTER CC and friends are set, in order to make
> the use of LLVM utilities when building a Linux kernel more ergonomic.
> 
> With this patch, the above now looks like:
> 
> $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y
> 
> Then you can "opt out" of certain LLVM utilities explicitly:
> 
> $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y AS=as
> 
> will instead invoke arm-linux-gnueabihf-as in place of clang for AS.
> 
> Or when not cross compiling:
> 
> $ make LLVM=y AS=as
> 
> This would make it more verbose to opt into just one tool from LLVM, but
> this patch doesn't actually break the old style; just leave off LLVM=y.
> Also, LLVM=y CC=clang would wind up prefixing clang with $CROSS_COMPILE.
> In that case, it's recommended to just drop LLVM=y and use the old
> style. So LLVM=y can be thought of as default to LLVM with explicit opt
> ins for GNU, vs the current case of default to GNU and opt in for LLVM.
> 
> A key part of the design of this patch is to be minimally invasive to
> the top level Makefile and not break existing workflows. We could get
> more aggressive, but I'd prefer to save larger refactorings for another
> day.
> 
> Finally, some linux distributions package specific versions of LLVM
> utilities with naming conventions that use the version as a suffix, ie.
> clang-11.  In that case, you can use LLVM=<number> and that number will
> be used as a suffix. Example:
> 
> $ make LLVM=11
> 
> will invoke clang-11, ld.lld-11, llvm-objcopy-11, etc.
> 
> About the script:
> The pattern used in the script is in the form:
> 
> ifeq "$(origin $(CC))" "file"
> $(CC) := $(clang)
> else
> override $(CC) := $(CROSS_COMPILE)$(CC)
> endif
> 
> "Metaprogramming" (eval) is used to template the above to make it more
> concise for specifying all of the substitutions.
> 
> The "origin" of a variable tracks whether a variable was explicitly set
> via "command line", "environment", was defined earlier via Makefile
> "file", was provided by "default", or was "undefined".
> 
> Variable assignment in GNU Make has some special and complicated rules.
> 
> If the variable was set earlier explicitly in the Makefile, we can
> simply reassign a new value to it. If a variable was unspecified, then
> earlier assignments were executed and change the origin to file.
> Otherwise, the variable was explicitly specified.
> 
> If a variable's "origin" was "command line" or "environment",
> non-"override" assignments are not executed. The "override" directive
> forces the assignment regardless of "origin".
> 
> Some tips I found useful for debugging for future travelers:
> 
> $(info $$origin of $$CC is $(origin CC))
> 
> at the start of the new script for all of the variables can help you
> understand "default" vs "undefined" variable origins.
> 
> $(info $$CC is [${CC}])
> 
> in the top level Makefile after including the new script, for all of the
> variables can help you check that they're being set as expected.
> 
> Link: https://www.gnu.org/software/make/manual/html_node/Eval-Function.html
> Link: https://www.gnu.org/software/make/manual/html_node/Origin-Function.html
> Link: https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
> Link: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html
> Suggested-by: Nathan Chancellor <natechancellor@gmail.com>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
> Changes V1 -> V2:
> * Rather than LLVM=1, use LLVM=y to enable all.
> * LLVM=<anything other than y> becomes a suffix, LLVM_SUFFIX.
> * strip has to be used on the LLVM_SUFFIX to avoid an extra whitespace.
> 
> 
>  Makefile              |  4 ++++
>  scripts/Makefile.llvm | 30 ++++++++++++++++++++++++++++++
>  2 files changed, 34 insertions(+)
>  create mode 100644 scripts/Makefile.llvm
> 
> diff --git a/Makefile b/Makefile
> index 402f276da062..72ec9dfea15e 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -475,6 +475,10 @@ KBUILD_LDFLAGS :=
>  GCC_PLUGINS_CFLAGS :=
>  CLANG_FLAGS :=
>  
> +ifneq ($(LLVM),)
> +include scripts/Makefile.llvm
> +endif
> +
>  export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE AS LD CC
>  export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL
>  export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
> diff --git a/scripts/Makefile.llvm b/scripts/Makefile.llvm
> new file mode 100644
> index 000000000000..0bab45a100a3
> --- /dev/null
> +++ b/scripts/Makefile.llvm
> @@ -0,0 +1,30 @@
> +LLVM_SUFFIX=
> +
> +ifneq ($(LLVM),y)
> +LLVM_SUFFIX += -$(LLVM)
> +endif
> +
> +define META_set =
> +ifeq "$(origin $(1))" "file"
> +$(1) := $(2)$(strip $(LLVM_SUFFIX))
> +else
> +override $(1) := $(CROSS_COMPILE)$($(1))
> +endif
> +endef
> +
> +$(eval $(call META_set,CC,clang))
> +$(eval $(call META_set,AS,clang))
> +$(eval $(call META_set,LD,ld.lld))
> +$(eval $(call META_set,AR,llvm-ar))
> +$(eval $(call META_set,NM,llvm-nm))
> +$(eval $(call META_set,STRIP,llvm-strip))
> +$(eval $(call META_set,OBJCOPY,llvm-objcopy))
> +$(eval $(call META_set,OBJDUMP,llvm-objdump))
> +$(eval $(call META_set,OBJSIZE,llvm-objsize))
> +$(eval $(call META_set,READELF,llvm-readelf))
> +$(eval $(call META_set,HOSTCC,clang))
> +$(eval $(call META_set,HOSTCXX,clang++))
> +$(eval $(call META_set,HOSTAR,llvm-ar))
> +$(eval $(call META_set,HOSTLD,ld.lld))
> +
> +## TODO: HOSTAR, HOSTLD in tools/objtool/Makefile
> -- 
> 2.25.1.481.gfbce0eb801-goog
> 

The use of override appears to break exporting the value to Kconfig,
which causes Kconfig to endlessly loop at the --syncconfig step:

$ make -j$(nproc) ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- LD=ld LLVM=y defconfig prepare
...
Makefile: LD is riscv64-linux-gnu-ld
*** Default configuration is based on 'defconfig'
Kconfig: LD is ld
#
# No change to .config
#
Makefile: LD is riscv64-linux-gnu-ld
Makefile: LD is riscv64-linux-gnu-ld
scripts/kconfig/conf  --syncconfig Kconfig
Kconfig: LD is ld
Makefile: LD is riscv64-linux-gnu-ld
auto.conf.cmd: LD is riscv64-linux-gnu-ld
Makefile: LD is riscv64-linux-gnu-ld
scripts/kconfig/conf  --syncconfig Kconfig
Kconfig: LD is ld
Makefile: LD is riscv64-linux-gnu-ld
auto.conf.cmd: LD is riscv64-linux-gnu-ld
Makefile: LD is riscv64-linux-gnu-ld
scripts/kconfig/conf  --syncconfig Kconfig
Kconfig: LD is ld
...

This endless loops happens because auto.conf.cmd is constantly being
regenerated because Kconfig is not picking up the override value, as you
can see from the debugging output above and auto.conf.cmd below:

$ rg -A 2 "LD\)" include/config/auto.conf.cmd | cat
ifneq "$(LD)" "ld"
include/config/auto.conf: FORCE
endif

$(LD) evaluates to riscv64-linux-gnu-ld, which is not equal to ld, so
include/config/auto.conf is regenerated, but env_write_dep in
scripts/kconfig/preprocess.c just picks up the environment value, rather
than the value that was overridden. This appears to be related to
recursive make calls from what I can tell.

$ cat Makefile
override FOO := foo
export FOO

all:
	$(info Makefile: $$(FOO) is $(FOO))
	$(MAKE) -f Makefile.sub all

$ cat Makefile.sub
all:
	$(info Makefile.sub: $$(FOO) is $(FOO))

$ make -s
Makefile: $(FOO) is foo
Makefile.sub: $(FOO) is foo

$ make -s FOO=bar
Makefile: $(FOO) is foo
Makefile.sub: $(FOO) is bar

No idea if this is a bug in make or not, this seems subtle. Not really
sure where to go from here, hopefully this is useful to you or Masahiro.

You will only notice this with variables that impact Kconfig, which
is why you never noticed with AS=as. See include/config/auto.conf.cmd
for the full list (CC, LD, NM, OBJCOPY, HOSTCC, and HOSTCXX are the ones
I see).

Debug patch if you want to play around with this.

Cheers,
Nathan

================================

diff --git a/Makefile b/Makefile
index acd8022ddb52..81cbb940e035 100644
--- a/Makefile
+++ b/Makefile
@@ -479,6 +479,8 @@ ifneq ($(LLVM),)
 include scripts/Makefile.llvm
 endif
 
+$(info Makefile: LD is $(LD))
+
 export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE AS LD CC
 export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL
 export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
diff --git a/scripts/Kconfig.include b/scripts/Kconfig.include
index 8074f14d9d0d..43a852e0ee93 100644
--- a/scripts/Kconfig.include
+++ b/scripts/Kconfig.include
@@ -57,3 +57,5 @@ gcc-version := $(shell,$(srctree)/scripts/gcc-version.sh $(CC))
 cc-option-bit = $(if-success,$(CC) -Werror $(1) -E -x c /dev/null -o /dev/null,$(1))
 m32-flag := $(cc-option-bit,-m32)
 m64-flag := $(cc-option-bit,-m64)
+
+$(info,Kconfig: LD is $(LD))
diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index 0243086fb168..fd828efc85fc 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -93,6 +93,7 @@ void env_write_dep(FILE *f, const char *autoconfig_name)
 
 	list_for_each_entry_safe(e, tmp, &env_list, node) {
 		fprintf(f, "ifneq \"$(%s)\" \"%s\"\n", e->name, e->value);
+		fprintf(f, "$(info auto.conf.cmd: %s is $(%s))\n", e->name, e->name);
 		fprintf(f, "%s: FORCE\n", autoconfig_name);
 		fprintf(f, "endif\n");
 		env_del(e);

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Masahiro Yamada]

On Sat, Mar 28, 2020 at 7:42 AM Nathan Chancellor
<natechancellor@gmail.com> wrote:
>
> Sorry for the delay in review :(
>
> On Tue, Mar 17, 2020 at 02:55:15PM -0700, Nick Desaulniers wrote:
> > Prior to this patch, building the Linux kernel with Clang
> > looked like:
> >
> > $ make CC=clang
> >
> > or when cross compiling:
> >
> > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang
> >
> > which got very verbose and unwieldy when using all of LLVM's substitutes
> > for GNU binutils:
> >
> > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang AS=clang \
> >   LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \
> >   OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump OBJSIZE=llvm-objsize \
> >   READELF=llvm-readelf HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar \
> >   HOSTLD=ld.lld
> >
> > This change adds a new Makefile under scripts/ which will be included in
> > the top level Makefile AFTER CC and friends are set, in order to make
> > the use of LLVM utilities when building a Linux kernel more ergonomic.
> >
> > With this patch, the above now looks like:
> >
> > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y
> >
> > Then you can "opt out" of certain LLVM utilities explicitly:
> >
> > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y AS=as
> >
> > will instead invoke arm-linux-gnueabihf-as in place of clang for AS.
> >
> > Or when not cross compiling:
> >
> > $ make LLVM=y AS=as
> >
> > This would make it more verbose to opt into just one tool from LLVM, but
> > this patch doesn't actually break the old style; just leave off LLVM=y.
> > Also, LLVM=y CC=clang would wind up prefixing clang with $CROSS_COMPILE.
> > In that case, it's recommended to just drop LLVM=y and use the old
> > style. So LLVM=y can be thought of as default to LLVM with explicit opt
> > ins for GNU, vs the current case of default to GNU and opt in for LLVM.
> >
> > A key part of the design of this patch is to be minimally invasive to
> > the top level Makefile and not break existing workflows. We could get
> > more aggressive, but I'd prefer to save larger refactorings for another
> > day.
> >
> > Finally, some linux distributions package specific versions of LLVM
> > utilities with naming conventions that use the version as a suffix, ie.
> > clang-11.  In that case, you can use LLVM=<number> and that number will
> > be used as a suffix. Example:
> >
> > $ make LLVM=11
> >
> > will invoke clang-11, ld.lld-11, llvm-objcopy-11, etc.
> >
> > About the script:
> > The pattern used in the script is in the form:
> >
> > ifeq "$(origin $(CC))" "file"
> > $(CC) := $(clang)
> > else
> > override $(CC) := $(CROSS_COMPILE)$(CC)
> > endif
> >
> > "Metaprogramming" (eval) is used to template the above to make it more
> > concise for specifying all of the substitutions.
> >
> > The "origin" of a variable tracks whether a variable was explicitly set
> > via "command line", "environment", was defined earlier via Makefile
> > "file", was provided by "default", or was "undefined".
> >
> > Variable assignment in GNU Make has some special and complicated rules.
> >
> > If the variable was set earlier explicitly in the Makefile, we can
> > simply reassign a new value to it. If a variable was unspecified, then
> > earlier assignments were executed and change the origin to file.
> > Otherwise, the variable was explicitly specified.
> >
> > If a variable's "origin" was "command line" or "environment",
> > non-"override" assignments are not executed. The "override" directive
> > forces the assignment regardless of "origin".
> >
> > Some tips I found useful for debugging for future travelers:
> >
> > $(info $$origin of $$CC is $(origin CC))
> >
> > at the start of the new script for all of the variables can help you
> > understand "default" vs "undefined" variable origins.
> >
> > $(info $$CC is [${CC}])
> >
> > in the top level Makefile after including the new script, for all of the
> > variables can help you check that they're being set as expected.
> >
> > Link: https://www.gnu.org/software/make/manual/html_node/Eval-Function.html
> > Link: https://www.gnu.org/software/make/manual/html_node/Origin-Function.html
> > Link: https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
> > Link: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html
> > Suggested-by: Nathan Chancellor <natechancellor@gmail.com>
> > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> > ---
> > Changes V1 -> V2:
> > * Rather than LLVM=1, use LLVM=y to enable all.
> > * LLVM=<anything other than y> becomes a suffix, LLVM_SUFFIX.
> > * strip has to be used on the LLVM_SUFFIX to avoid an extra whitespace.
> >
> >
> >  Makefile              |  4 ++++
> >  scripts/Makefile.llvm | 30 ++++++++++++++++++++++++++++++
> >  2 files changed, 34 insertions(+)
> >  create mode 100644 scripts/Makefile.llvm
> >
> > diff --git a/Makefile b/Makefile
> > index 402f276da062..72ec9dfea15e 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -475,6 +475,10 @@ KBUILD_LDFLAGS :=
> >  GCC_PLUGINS_CFLAGS :=
> >  CLANG_FLAGS :=
> >
> > +ifneq ($(LLVM),)
> > +include scripts/Makefile.llvm
> > +endif
> > +
> >  export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE AS LD CC
> >  export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL
> >  export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
> > diff --git a/scripts/Makefile.llvm b/scripts/Makefile.llvm
> > new file mode 100644
> > index 000000000000..0bab45a100a3
> > --- /dev/null
> > +++ b/scripts/Makefile.llvm
> > @@ -0,0 +1,30 @@
> > +LLVM_SUFFIX=
> > +
> > +ifneq ($(LLVM),y)
> > +LLVM_SUFFIX += -$(LLVM)
> > +endif
> > +
> > +define META_set =
> > +ifeq "$(origin $(1))" "file"
> > +$(1) := $(2)$(strip $(LLVM_SUFFIX))
> > +else
> > +override $(1) := $(CROSS_COMPILE)$($(1))
> > +endif
> > +endef
> > +
> > +$(eval $(call META_set,CC,clang))
> > +$(eval $(call META_set,AS,clang))
> > +$(eval $(call META_set,LD,ld.lld))
> > +$(eval $(call META_set,AR,llvm-ar))
> > +$(eval $(call META_set,NM,llvm-nm))
> > +$(eval $(call META_set,STRIP,llvm-strip))
> > +$(eval $(call META_set,OBJCOPY,llvm-objcopy))
> > +$(eval $(call META_set,OBJDUMP,llvm-objdump))
> > +$(eval $(call META_set,OBJSIZE,llvm-objsize))
> > +$(eval $(call META_set,READELF,llvm-readelf))
> > +$(eval $(call META_set,HOSTCC,clang))
> > +$(eval $(call META_set,HOSTCXX,clang++))
> > +$(eval $(call META_set,HOSTAR,llvm-ar))
> > +$(eval $(call META_set,HOSTLD,ld.lld))
> > +
> > +## TODO: HOSTAR, HOSTLD in tools/objtool/Makefile
> > --
> > 2.25.1.481.gfbce0eb801-goog
> >
>
> The use of override appears to break exporting the value to Kconfig,
> which causes Kconfig to endlessly loop at the --syncconfig step:
>
> $ make -j$(nproc) ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- LD=ld LLVM=y defconfig prepare
> ...
> Makefile: LD is riscv64-linux-gnu-ld
> *** Default configuration is based on 'defconfig'
> Kconfig: LD is ld
> #
> # No change to .config
> #
> Makefile: LD is riscv64-linux-gnu-ld
> Makefile: LD is riscv64-linux-gnu-ld
> scripts/kconfig/conf  --syncconfig Kconfig
> Kconfig: LD is ld
> Makefile: LD is riscv64-linux-gnu-ld
> auto.conf.cmd: LD is riscv64-linux-gnu-ld
> Makefile: LD is riscv64-linux-gnu-ld
> scripts/kconfig/conf  --syncconfig Kconfig
> Kconfig: LD is ld
> Makefile: LD is riscv64-linux-gnu-ld
> auto.conf.cmd: LD is riscv64-linux-gnu-ld
> Makefile: LD is riscv64-linux-gnu-ld
> scripts/kconfig/conf  --syncconfig Kconfig
> Kconfig: LD is ld
> ...





Kconfig is invoked from scripts/kconfig/Makefile
instead of the top Makefile.


Just add one more debug code

   $(info scripts/kconfig/Makefile: LD is $(LD))

into scripts/kconfig/Makefile.


It is "ld", not "riscv64-linux-gnu-ld".



The variable passed from the command line
is strong, and overrides the variable in sub-make as well.




> This endless loops happens because auto.conf.cmd is constantly being
> regenerated because Kconfig is not picking up the override value, as you
> can see from the debugging output above and auto.conf.cmd below:
>
> $ rg -A 2 "LD\)" include/config/auto.conf.cmd | cat
> ifneq "$(LD)" "ld"
> include/config/auto.conf: FORCE
> endif
>
> $(LD) evaluates to riscv64-linux-gnu-ld, which is not equal to ld, so
> include/config/auto.conf is regenerated, but env_write_dep in
> scripts/kconfig/preprocess.c just picks up the environment value, rather
> than the value that was overridden. This appears to be related to
> recursive make calls from what I can tell.
>
> $ cat Makefile
> override FOO := foo
> export FOO
>
> all:
>         $(info Makefile: $$(FOO) is $(FOO))
>         $(MAKE) -f Makefile.sub all
>
> $ cat Makefile.sub
> all:
>         $(info Makefile.sub: $$(FOO) is $(FOO))
>
> $ make -s
> Makefile: $(FOO) is foo
> Makefile.sub: $(FOO) is foo
>
> $ make -s FOO=bar
> Makefile: $(FOO) is foo
> Makefile.sub: $(FOO) is bar
>
> No idea if this is a bug in make or not, this seems subtle. Not really
> sure where to go from here, hopefully this is useful to you or Masahiro.


This is not a bug.

The variable passed from the command line
(which is inherited to sub-make as well)
is stronger than the environment variable.

"make -s FOO=BAR" is not equivalent to "FOO=BAR make -s".




-- 
Best Regards
Masahiro Yamada

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Masahiro Yamada]

[-- Attachment #1: Type: text/plain, Size: 7951 bytes --]

On Sat, Mar 28, 2020 at 1:54 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Sat, Mar 28, 2020 at 7:42 AM Nathan Chancellor
> <natechancellor@gmail.com> wrote:
> >
> > Sorry for the delay in review :(
> >
> > On Tue, Mar 17, 2020 at 02:55:15PM -0700, Nick Desaulniers wrote:
> > > Prior to this patch, building the Linux kernel with Clang
> > > looked like:
> > >
> > > $ make CC=clang
> > >
> > > or when cross compiling:
> > >
> > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang
> > >
> > > which got very verbose and unwieldy when using all of LLVM's substitutes
> > > for GNU binutils:
> > >
> > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang AS=clang \
> > >   LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \
> > >   OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump OBJSIZE=llvm-objsize \
> > >   READELF=llvm-readelf HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar \
> > >   HOSTLD=ld.lld
> > >
> > > This change adds a new Makefile under scripts/ which will be included in
> > > the top level Makefile AFTER CC and friends are set, in order to make
> > > the use of LLVM utilities when building a Linux kernel more ergonomic.
> > >
> > > With this patch, the above now looks like:
> > >
> > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y
> > >
> > > Then you can "opt out" of certain LLVM utilities explicitly:
> > >
> > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y AS=as
> > >
> > > will instead invoke arm-linux-gnueabihf-as in place of clang for AS.
> > >
> > > Or when not cross compiling:
> > >
> > > $ make LLVM=y AS=as
> > >
> > > This would make it more verbose to opt into just one tool from LLVM, but
> > > this patch doesn't actually break the old style; just leave off LLVM=y.
> > > Also, LLVM=y CC=clang would wind up prefixing clang with $CROSS_COMPILE.
> > > In that case, it's recommended to just drop LLVM=y and use the old
> > > style. So LLVM=y can be thought of as default to LLVM with explicit opt
> > > ins for GNU, vs the current case of default to GNU and opt in for LLVM.
> > >
> > > A key part of the design of this patch is to be minimally invasive to
> > > the top level Makefile and not break existing workflows. We could get
> > > more aggressive, but I'd prefer to save larger refactorings for another
> > > day.
> > >
> > > Finally, some linux distributions package specific versions of LLVM
> > > utilities with naming conventions that use the version as a suffix, ie.
> > > clang-11.  In that case, you can use LLVM=<number> and that number will
> > > be used as a suffix. Example:
> > >
> > > $ make LLVM=11
> > >
> > > will invoke clang-11, ld.lld-11, llvm-objcopy-11, etc.
> > >
> > > About the script:
> > > The pattern used in the script is in the form:
> > >
> > > ifeq "$(origin $(CC))" "file"
> > > $(CC) := $(clang)
> > > else
> > > override $(CC) := $(CROSS_COMPILE)$(CC)
> > > endif
> > >
> > > "Metaprogramming" (eval) is used to template the above to make it more
> > > concise for specifying all of the substitutions.
> > >
> > > The "origin" of a variable tracks whether a variable was explicitly set
> > > via "command line", "environment", was defined earlier via Makefile
> > > "file", was provided by "default", or was "undefined".
> > >
> > > Variable assignment in GNU Make has some special and complicated rules.
> > >
> > > If the variable was set earlier explicitly in the Makefile, we can
> > > simply reassign a new value to it. If a variable was unspecified, then
> > > earlier assignments were executed and change the origin to file.
> > > Otherwise, the variable was explicitly specified.
> > >
> > > If a variable's "origin" was "command line" or "environment",
> > > non-"override" assignments are not executed. The "override" directive
> > > forces the assignment regardless of "origin".
> > >
> > > Some tips I found useful for debugging for future travelers:
> > >
> > > $(info $$origin of $$CC is $(origin CC))
> > >
> > > at the start of the new script for all of the variables can help you
> > > understand "default" vs "undefined" variable origins.
> > >
> > > $(info $$CC is [${CC}])
> > >
> > > in the top level Makefile after including the new script, for all of the
> > > variables can help you check that they're being set as expected.
> > >
> > > Link: https://www.gnu.org/software/make/manual/html_node/Eval-Function.html
> > > Link: https://www.gnu.org/software/make/manual/html_node/Origin-Function.html
> > > Link: https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
> > > Link: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html
> > > Suggested-by: Nathan Chancellor <natechancellor@gmail.com>
> > > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> > > ---
> > > Changes V1 -> V2:
> > > * Rather than LLVM=1, use LLVM=y to enable all.
> > > * LLVM=<anything other than y> becomes a suffix, LLVM_SUFFIX.
> > > * strip has to be used on the LLVM_SUFFIX to avoid an extra whitespace.
> > >
> > >
> > >  Makefile              |  4 ++++
> > >  scripts/Makefile.llvm | 30 ++++++++++++++++++++++++++++++
> > >  2 files changed, 34 insertions(+)
> > >  create mode 100644 scripts/Makefile.llvm
> > >
> > > diff --git a/Makefile b/Makefile
> > > index 402f276da062..72ec9dfea15e 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -475,6 +475,10 @@ KBUILD_LDFLAGS :=
> > >  GCC_PLUGINS_CFLAGS :=
> > >  CLANG_FLAGS :=
> > >
> > > +ifneq ($(LLVM),)
> > > +include scripts/Makefile.llvm
> > > +endif
> > > +
> > >  export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE AS LD CC
> > >  export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL
> > >  export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
> > > diff --git a/scripts/Makefile.llvm b/scripts/Makefile.llvm
> > > new file mode 100644
> > > index 000000000000..0bab45a100a3
> > > --- /dev/null
> > > +++ b/scripts/Makefile.llvm
> > > @@ -0,0 +1,30 @@
> > > +LLVM_SUFFIX=
> > > +
> > > +ifneq ($(LLVM),y)
> > > +LLVM_SUFFIX += -$(LLVM)
> > > +endif
> > > +
> > > +define META_set =
> > > +ifeq "$(origin $(1))" "file"
> > > +$(1) := $(2)$(strip $(LLVM_SUFFIX))
> > > +else
> > > +override $(1) := $(CROSS_COMPILE)$($(1))
> > > +endif
> > > +endef
> > > +
> > > +$(eval $(call META_set,CC,clang))
> > > +$(eval $(call META_set,AS,clang))
> > > +$(eval $(call META_set,LD,ld.lld))
> > > +$(eval $(call META_set,AR,llvm-ar))
> > > +$(eval $(call META_set,NM,llvm-nm))
> > > +$(eval $(call META_set,STRIP,llvm-strip))
> > > +$(eval $(call META_set,OBJCOPY,llvm-objcopy))
> > > +$(eval $(call META_set,OBJDUMP,llvm-objdump))
> > > +$(eval $(call META_set,OBJSIZE,llvm-objsize))
> > > +$(eval $(call META_set,READELF,llvm-readelf))
> > > +$(eval $(call META_set,HOSTCC,clang))
> > > +$(eval $(call META_set,HOSTCXX,clang++))
> > > +$(eval $(call META_set,HOSTAR,llvm-ar))
> > > +$(eval $(call META_set,HOSTLD,ld.lld))
> > > +
> > > +## TODO: HOSTAR, HOSTLD in tools/objtool/Makefile
> > > --


I also had planned to provide a single switch to change
all the tool defaults to LLVM.

So, supporting 'LLVM' is fine, but I'd rather want this
look symmetrical, and easy to understand.

CPP        = $(CC) -E
ifneq ($(LLVM),)
CC         = $(LLVM_DIR)clang
LD         = $(LLVM_DIR)ld.lld
AR         = $(LLVM_DIR)llvm-ar
NM         = $(LLVM_DIR)llvm-nm
OBJCOPY    = $(LLVM_DIR)llvm-objcopy
OBJDUMP    = $(LLVM_DIR)llvm-objdump
READELF    = $(LLVM_DIR)llvm-readelf
OBJSIZE    = $(LLVM_DIR)llvm-size
STRIP      = $(LLVM_DIR)llvm-strip
else
CC         = $(CROSS_COMPILE)gcc
LD         = $(CROSS_COMPILE)ld
AR         = $(CROSS_COMPILE)ar
NM         = $(CROSS_COMPILE)nm
OBJCOPY    = $(CROSS_COMPILE)objcopy
OBJDUMP    = $(CROSS_COMPILE)objdump
READELF    = $(CROSS_COMPILE)readelf
OBJSIZE    = $(CROSS_COMPILE)size
STRIP      = $(CROSS_COMPILE)strip
endif



I attached two patches.
Comments appreciated.

-- 
Best Regards
Masahiro Yamada

[-- Attachment #2: 0002-kbuild-change-the-default-of-HOST-CC-CXX-to-cc-and-c.patch --]
[-- Type: text/x-patch, Size: 2128 bytes --]

From f3a620abee5e26dda04f7a747a54ab075f45b70d Mon Sep 17 00:00:00 2001
From: Masahiro Yamada <masahiroy@kernel.org>
Date: Sat, 28 Mar 2020 18:57:25 +0900
Subject: [PATCH 2/2] kbuild: change the default of HOST{CC,CXX} to cc and c++

I have been thinking how to cater to those who want to build host
programs with Clang.

We could use the variable 'LLVM' to switch the default of HOST{CC,CXX}:

  ifneq ($(LLVM),)
  HOSTCC       = clang
  HOSTCXX      = clang++
  else
  HOSTCC       = gcc
  HOSTCXX      = g++
  endif

But, I do not know how many people care about this. There is no tricky
code in userspace programs, and we know we can compile them with GCC,
Clang, or whatever. No matter what you use for host programs, it makes
no difference to the kernel itself.

If you are so addicted to LLVM or if you are using a system with no GCC
installation, probably you had already had 'cc' and 'c++' point to Clang.

So, another approach is to just leave this up to the system. You can
manually set up symlinks, or maybe your distro provides 'alternatives'.

  $ update-alternatives --list cc
  /usr/bin/clang
  /usr/bin/gcc
  $ update-alternatives --list c++
  /usr/bin/clang++
  /usr/bin/g++

I have no idea what to do for tools/objtool/Makefile, which uses HOSTAR
and HOSTLD, but this is because objtool intentionally opts out Kbuild.
If objtool is willing to join the Kbuild infrastructure, a patch exists:

  https://patchwork.kernel.org/patch/10839051/

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---
 Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index f192f9bd8343..ecdd34ad0750 100644
--- a/Makefile
+++ b/Makefile
@@ -399,8 +399,8 @@ HOST_LFS_CFLAGS := $(shell getconf LFS_CFLAGS 2>/dev/null)
 HOST_LFS_LDFLAGS := $(shell getconf LFS_LDFLAGS 2>/dev/null)
 HOST_LFS_LIBS := $(shell getconf LFS_LIBS 2>/dev/null)
 
-HOSTCC       = gcc
-HOSTCXX      = g++
+HOSTCC       = cc
+HOSTCXX      = c++
 KBUILD_HOSTCFLAGS   := -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 \
 		-fomit-frame-pointer -std=gnu89 $(HOST_LFS_CFLAGS) \
 		$(HOSTCFLAGS)
-- 
2.17.1


[-- Attachment #3: 0001-kbuild-support-LLVM-to-switch-the-default-tools-to-C.patch --]
[-- Type: text/x-patch, Size: 4367 bytes --]

From 6620f13807b466c5e4af08b2d5d33f5a433b1e3f Mon Sep 17 00:00:00 2001
From: Masahiro Yamada <masahiroy@kernel.org>
Date: Sat, 28 Mar 2020 15:54:47 +0900
Subject: [PATCH 1/2] kbuild: support 'LLVM' to switch the default tools to
 Clang/LLVM

As Documentation/kbuild/llvm.rst implies, building the kernel with a
full set of LLVM tools gets very verbose and unwieldy.

Provide a single switch 'LLVM' to use Clang and LLVM tools instead of
GCC and Binutils. You can pass LLVM=1 from the command line or as an
environment variable. Then, Kbuild will use LLVM toolchains in your
PATH environment. This may not be convenient if you have multiple
versions of LLVM.

CROSS_COMPILE is used to specify not only the tool prefix, but also
the directory path to the tools. For example,

  $ make ARCH=arm64 CROSS_COMPILE=/path/to/my/gcc/bin/aarch64-linux-gnu-

To support a similar flow, this commit adds another variable, LLVM_DIR,
to point to the specific installation of LLVM:

  $ make LLVM=1 LLVM_DIR=/path/to/my/llvm/bin/

It might be tedious to set two variables. So, the following is the
shorthand:

  $ make LLVM=/path/to/my/llvm/bin/

Please note LLVM=1 does not turn on the LLVM integrated assembler.
You need to pass AS=clang to use it. When the upstream kernel is
ready for the integrated assembler, we can make it default. We will
get rid of --no-integrated-as, then CROSS_COMPILE will be no longer
needed. The --target option will be specified by other means.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---
 Documentation/kbuild/kbuild.rst |  6 ++++++
 Documentation/kbuild/llvm.rst   |  5 +++++
 Makefile                        | 29 +++++++++++++++++++++++++----
 3 files changed, 36 insertions(+), 4 deletions(-)

diff --git a/Documentation/kbuild/kbuild.rst b/Documentation/kbuild/kbuild.rst
index f1e5dce86af7..39bb3636e4f4 100644
--- a/Documentation/kbuild/kbuild.rst
+++ b/Documentation/kbuild/kbuild.rst
@@ -262,3 +262,9 @@ KBUILD_BUILD_USER, KBUILD_BUILD_HOST
 These two variables allow to override the user@host string displayed during
 boot and in /proc/version. The default value is the output of the commands
 whoami and host, respectively.
+
+LLVM
+----
+If this variable is set to 1, Kbuild will use Clang and LLVM utilities instead
+of GCC and GNU binutils to build the kernel.
+If set to a value other than 1, it points to directory path to LLVM to be used.
diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst
index d6c79eb4e23e..4602369f6a4f 100644
--- a/Documentation/kbuild/llvm.rst
+++ b/Documentation/kbuild/llvm.rst
@@ -55,6 +55,11 @@ additional parameters to `make`.
 	  READELF=llvm-readelf HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar \\
 	  HOSTLD=ld.lld
 
+You can use a single switch `LLVM=1` to use LLVM utilities by default (except
+for building host programs).
+
+	make LLVM=1 HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar HOSTLD=ld.lld
+
 Getting Help
 ------------
 
diff --git a/Makefile b/Makefile
index a3bc8bc562ee..f192f9bd8343 100644
--- a/Makefile
+++ b/Makefile
@@ -408,17 +408,38 @@ KBUILD_HOSTCXXFLAGS := -O2 $(HOST_LFS_CFLAGS) $(HOSTCXXFLAGS)
 KBUILD_HOSTLDFLAGS  := $(HOST_LFS_LDFLAGS) $(HOSTLDFLAGS)
 KBUILD_HOSTLDLIBS   := $(HOST_LFS_LIBS) $(HOSTLDLIBS)
 
+# LLVM=1 tells Kbuild to use Clang and LLVM utilities by default.
+# You can still override CC, LD, etc. individually if desired.
+#
+# If LLVM is set to a value other than 1, it is set to LLVM_DIR,
+# which is useful to select a specific LLVM installation.
+ifneq ($(filter-out 1,$(LLVM)),)
+LLVM_DIR := $(LLVM)
+endif
+
 # Make variables (CC, etc...)
-LD		= $(CROSS_COMPILE)ld
-CC		= $(CROSS_COMPILE)gcc
 CPP		= $(CC) -E
+ifneq ($(LLVM),)
+CC		= $(LLVM_DIR)clang
+LD		= $(LLVM_DIR)ld.lld
+AR		= $(LLVM_DIR)llvm-ar
+NM		= $(LLVM_DIR)llvm-nm
+OBJCOPY		= $(LLVM_DIR)llvm-objcopy
+OBJDUMP		= $(LLVM_DIR)llvm-objdump
+READELF		= $(LLVM_DIR)llvm-readelf
+OBJSIZE		= $(LLVM_DIR)llvm-size
+STRIP		= $(LLVM_DIR)llvm-strip
+else
+CC		= $(CROSS_COMPILE)gcc
+LD		= $(CROSS_COMPILE)ld
 AR		= $(CROSS_COMPILE)ar
 NM		= $(CROSS_COMPILE)nm
-STRIP		= $(CROSS_COMPILE)strip
 OBJCOPY		= $(CROSS_COMPILE)objcopy
 OBJDUMP		= $(CROSS_COMPILE)objdump
-OBJSIZE		= $(CROSS_COMPILE)size
 READELF		= $(CROSS_COMPILE)readelf
+OBJSIZE		= $(CROSS_COMPILE)size
+STRIP		= $(CROSS_COMPILE)strip
+endif
 PAHOLE		= pahole
 LEX		= flex
 YACC		= bison
-- 
2.17.1

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Nick Desaulniers]

On Sat, Mar 28, 2020 at 6:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> I also had planned to provide a single switch to change
> all the tool defaults to LLVM.
>
> So, supporting 'LLVM' is fine, but I'd rather want this
> look symmetrical, and easy to understand.
>
> CPP        = $(CC) -E
> ifneq ($(LLVM),)

Yes, a simple if statement is much simpler than the overly complex patch I had.

> CC         = $(LLVM_DIR)clang

Do we need $LLVM_DIR? Shouldn't users just have that in their $PATH?

Also, I think we need to support suffixed binaries, as debian
distributes these with version suffixes, as Nathan points out.  Or do
the debian packages install suffixed binaries AND path versioned
non-suffixed binaries?

> LD         = $(LLVM_DIR)ld.lld
> AR         = $(LLVM_DIR)llvm-ar
> NM         = $(LLVM_DIR)llvm-nm
> OBJCOPY    = $(LLVM_DIR)llvm-objcopy
> OBJDUMP    = $(LLVM_DIR)llvm-objdump
> READELF    = $(LLVM_DIR)llvm-readelf
> OBJSIZE    = $(LLVM_DIR)llvm-size
> STRIP      = $(LLVM_DIR)llvm-strip
> else
> CC         = $(CROSS_COMPILE)gcc
> LD         = $(CROSS_COMPILE)ld
> AR         = $(CROSS_COMPILE)ar
> NM         = $(CROSS_COMPILE)nm
> OBJCOPY    = $(CROSS_COMPILE)objcopy
> OBJDUMP    = $(CROSS_COMPILE)objdump
> READELF    = $(CROSS_COMPILE)readelf
> OBJSIZE    = $(CROSS_COMPILE)size
> STRIP      = $(CROSS_COMPILE)strip
> endif
>
>
>
> I attached two patches.
> Comments appreciated.

I'm not sure the second one that recommends changing cc/c++ is the way
to go; I think it might harm hermeticity.

> If you are so addicted to LLVM

Lol, maybe?
-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Nathan Chancellor]

On Mon, Mar 30, 2020 at 11:58:19AM -0700, Nick Desaulniers wrote:
> On Sat, Mar 28, 2020 at 6:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > I also had planned to provide a single switch to change
> > all the tool defaults to LLVM.
> >
> > So, supporting 'LLVM' is fine, but I'd rather want this
> > look symmetrical, and easy to understand.
> >
> > CPP        = $(CC) -E
> > ifneq ($(LLVM),)
> 
> Yes, a simple if statement is much simpler than the overly complex patch I had.
> 
> > CC         = $(LLVM_DIR)clang
> 
> Do we need $LLVM_DIR? Shouldn't users just have that in their $PATH?
> 
> Also, I think we need to support suffixed binaries, as debian
> distributes these with version suffixes, as Nathan points out.  Or do
> the debian packages install suffixed binaries AND path versioned
> non-suffixed binaries?

I think the idea here is that ultimately, the suffixed versions of clang
that Debian has in /usr/bin are symlinks to binaries in
/usr/lib/llvm-#/bin; as a result, a user could say
LLVM_DIR=/usr/lib/llvm-#/bin/ and all of those tools would be picked up
automatically. I am not really sure what is better.

I'll try to have some other comments by later today/tonight.

> > LD         = $(LLVM_DIR)ld.lld
> > AR         = $(LLVM_DIR)llvm-ar
> > NM         = $(LLVM_DIR)llvm-nm
> > OBJCOPY    = $(LLVM_DIR)llvm-objcopy
> > OBJDUMP    = $(LLVM_DIR)llvm-objdump
> > READELF    = $(LLVM_DIR)llvm-readelf
> > OBJSIZE    = $(LLVM_DIR)llvm-size
> > STRIP      = $(LLVM_DIR)llvm-strip
> > else
> > CC         = $(CROSS_COMPILE)gcc
> > LD         = $(CROSS_COMPILE)ld
> > AR         = $(CROSS_COMPILE)ar
> > NM         = $(CROSS_COMPILE)nm
> > OBJCOPY    = $(CROSS_COMPILE)objcopy
> > OBJDUMP    = $(CROSS_COMPILE)objdump
> > READELF    = $(CROSS_COMPILE)readelf
> > OBJSIZE    = $(CROSS_COMPILE)size
> > STRIP      = $(CROSS_COMPILE)strip
> > endif
> >
> >
> >
> > I attached two patches.
> > Comments appreciated.
> 
> I'm not sure the second one that recommends changing cc/c++ is the way
> to go; I think it might harm hermeticity.

Agreed. I do not modify my host system at all for this project, just
relying on PATH modification. In theory, we can still override HOSTCC
and HOSTCXX but that would defeat the purpose of that patch.

Cheers,
Nathan

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Masahiro Yamada]

On Tue, Mar 31, 2020 at 4:03 AM Nathan Chancellor
<natechancellor@gmail.com> wrote:
>
> On Mon, Mar 30, 2020 at 11:58:19AM -0700, Nick Desaulniers wrote:
> > On Sat, Mar 28, 2020 at 6:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > I also had planned to provide a single switch to change
> > > all the tool defaults to LLVM.
> > >
> > > So, supporting 'LLVM' is fine, but I'd rather want this
> > > look symmetrical, and easy to understand.
> > >
> > > CPP        = $(CC) -E
> > > ifneq ($(LLVM),)
> >
> > Yes, a simple if statement is much simpler than the overly complex patch I had.
> >
> > > CC         = $(LLVM_DIR)clang
> >
> > Do we need $LLVM_DIR? Shouldn't users just have that in their $PATH?
> >
> > Also, I think we need to support suffixed binaries, as debian
> > distributes these with version suffixes, as Nathan points out.  Or do
> > the debian packages install suffixed binaries AND path versioned
> > non-suffixed binaries?
>
> I think the idea here is that ultimately, the suffixed versions of clang
> that Debian has in /usr/bin are symlinks to binaries in
> /usr/lib/llvm-#/bin; as a result, a user could say
> LLVM_DIR=/usr/lib/llvm-#/bin/ and all of those tools would be picked up
> automatically. I am not really sure what is better.


I periodically build the latest llvm from the trunk,
and install it under my home directory.

So, I just thought it would be useful to
allow a user to specify the llvm directory.

Of course, I can do the equivalent by tweaking PATH, but
I hesitate to make the non-released version my default.



Having both LLVM_DIR and LLVM_SUFFIX seems verbose.

In fact, the debian provides multiple versions of GCC.
For example, my machine has

masahiro@pug:~$ ls -1 /usr/bin/gcc-*
/usr/bin/gcc-4.8
/usr/bin/gcc-5
/usr/bin/gcc-7
/usr/bin/gcc-ar
/usr/bin/gcc-ar-4.8
/usr/bin/gcc-ar-5
/usr/bin/gcc-ar-7
/usr/bin/gcc-nm
/usr/bin/gcc-nm-4.8
/usr/bin/gcc-nm-5
/usr/bin/gcc-nm-7
/usr/bin/gcc-ranlib
/usr/bin/gcc-ranlib-4.8
/usr/bin/gcc-ranlib-5
/usr/bin/gcc-ranlib-7

But, nobody has suggested GCC_SUFFIX.

So, I guess CROSS_COMPILE was enough to
choose a specific tool version.




> I'll try to have some other comments by later today/tonight.
>
> > > LD         = $(LLVM_DIR)ld.lld
> > > AR         = $(LLVM_DIR)llvm-ar
> > > NM         = $(LLVM_DIR)llvm-nm
> > > OBJCOPY    = $(LLVM_DIR)llvm-objcopy
> > > OBJDUMP    = $(LLVM_DIR)llvm-objdump
> > > READELF    = $(LLVM_DIR)llvm-readelf
> > > OBJSIZE    = $(LLVM_DIR)llvm-size
> > > STRIP      = $(LLVM_DIR)llvm-strip
> > > else
> > > CC         = $(CROSS_COMPILE)gcc
> > > LD         = $(CROSS_COMPILE)ld
> > > AR         = $(CROSS_COMPILE)ar
> > > NM         = $(CROSS_COMPILE)nm
> > > OBJCOPY    = $(CROSS_COMPILE)objcopy
> > > OBJDUMP    = $(CROSS_COMPILE)objdump
> > > READELF    = $(CROSS_COMPILE)readelf
> > > OBJSIZE    = $(CROSS_COMPILE)size
> > > STRIP      = $(CROSS_COMPILE)strip
> > > endif
> > >
> > >
> > >
> > > I attached two patches.
> > > Comments appreciated.
> >
> > I'm not sure the second one that recommends changing cc/c++ is the way
> > to go; I think it might harm hermeticity.
>
> Agreed. I do not modify my host system at all for this project, just
> relying on PATH modification. In theory, we can still override HOSTCC
> and HOSTCXX but that would defeat the purpose of that patch.
>
> Cheers,
> Nathan
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20200330190312.GA32257%40ubuntu-m2-xlarge-x86.



-- 
Best Regards
Masahiro Yamada

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Nick Desaulniers]

On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Tue, Mar 31, 2020 at 4:03 AM Nathan Chancellor
> <natechancellor@gmail.com> wrote:
> >
> > On Mon, Mar 30, 2020 at 11:58:19AM -0700, Nick Desaulniers wrote:
> > > On Sat, Mar 28, 2020 at 6:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > >
> > > > I also had planned to provide a single switch to change
> > > > all the tool defaults to LLVM.
> > > >
> > > > So, supporting 'LLVM' is fine, but I'd rather want this
> > > > look symmetrical, and easy to understand.
> > > >
> > > > CPP        = $(CC) -E
> > > > ifneq ($(LLVM),)
> > >
> > > Yes, a simple if statement is much simpler than the overly complex patch I had.
> > >
> > > > CC         = $(LLVM_DIR)clang
> > >
> > > Do we need $LLVM_DIR? Shouldn't users just have that in their $PATH?
> > >
> > > Also, I think we need to support suffixed binaries, as debian
> > > distributes these with version suffixes, as Nathan points out.  Or do
> > > the debian packages install suffixed binaries AND path versioned
> > > non-suffixed binaries?
> >
> > I think the idea here is that ultimately, the suffixed versions of clang
> > that Debian has in /usr/bin are symlinks to binaries in
> > /usr/lib/llvm-#/bin; as a result, a user could say
> > LLVM_DIR=/usr/lib/llvm-#/bin/ and all of those tools would be picked up
> > automatically. I am not really sure what is better.

$ sudo apt install clang-8
$ which clang-8
/usr/bin/clang-8
$ ls -l `!!`
/usr/bin/clang-8 -> ../lib/llvm-8/bin/clang
$ ls /usr/lib/llvm-8/bin
<non suffixed versions>

Ok, so Nathan, it looks like we don't need the version suffixes.
Instead, we can be more explicit with our $PATH, and only add the
above (and bintutils).  I was thinking supporting the suffix was
required for our CI, but it seems like maybe not.

> I periodically build the latest llvm from the trunk,
> and install it under my home directory.
> So, I just thought it would be useful to
> allow a user to specify the llvm directory.
> Of course, I can do the equivalent by tweaking PATH, but
> I hesitate to make the non-released version my default.

Respectfully, I strongly disagree.  This should be handled by
modifications to $PATH, either by your shell's .rc file when you
always want it, or exported for a session when you want it, or
prefixed to an invocation for the duration of that command.  We should
not have a new variable just for the path of a few tools.

Rather than `make LLVM_DIR=~/llvm-project LLVM=1`, you can do
`PATH=$PATH:~/llvm-project make LLVM=1`. (or export it manually or via
your shell .rc, depending on how comfortable you are with that
version).

> Having both LLVM_DIR and LLVM_SUFFIX seems verbose.

I agree, so maybe just LLVM=y, and we can support both non-standard
locations and debian suffixes via modifications to PATH.

>
> In fact, the debian provides multiple versions of GCC.
> For example, my machine has
>
> masahiro@pug:~$ ls -1 /usr/bin/gcc-*
> /usr/bin/gcc-4.8
> /usr/bin/gcc-5
> /usr/bin/gcc-7
> /usr/bin/gcc-ar
> /usr/bin/gcc-ar-4.8
> /usr/bin/gcc-ar-5
> /usr/bin/gcc-ar-7
> /usr/bin/gcc-nm
> /usr/bin/gcc-nm-4.8
> /usr/bin/gcc-nm-5
> /usr/bin/gcc-nm-7
> /usr/bin/gcc-ranlib
> /usr/bin/gcc-ranlib-4.8
> /usr/bin/gcc-ranlib-5
> /usr/bin/gcc-ranlib-7
>
> But, nobody has suggested GCC_SUFFIX.
>
> So, I guess CROSS_COMPILE was enough to
> choose a specific tool version.

Or no one was testing specific versions of gcc with more than one
installed.  I can ask the KernelCI folks next week if this is an issue
they face or have faced.
-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Nathan Chancellor]

On Tue, Mar 31, 2020 at 11:39:27AM -0700, Nick Desaulniers wrote:
> On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > On Tue, Mar 31, 2020 at 4:03 AM Nathan Chancellor
> > <natechancellor@gmail.com> wrote:
> > >
> > > On Mon, Mar 30, 2020 at 11:58:19AM -0700, Nick Desaulniers wrote:
> > > > On Sat, Mar 28, 2020 at 6:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > > >
> > > > > I also had planned to provide a single switch to change
> > > > > all the tool defaults to LLVM.
> > > > >
> > > > > So, supporting 'LLVM' is fine, but I'd rather want this
> > > > > look symmetrical, and easy to understand.
> > > > >
> > > > > CPP        = $(CC) -E
> > > > > ifneq ($(LLVM),)
> > > >
> > > > Yes, a simple if statement is much simpler than the overly complex patch I had.
> > > >
> > > > > CC         = $(LLVM_DIR)clang
> > > >
> > > > Do we need $LLVM_DIR? Shouldn't users just have that in their $PATH?
> > > >
> > > > Also, I think we need to support suffixed binaries, as debian
> > > > distributes these with version suffixes, as Nathan points out.  Or do
> > > > the debian packages install suffixed binaries AND path versioned
> > > > non-suffixed binaries?
> > >
> > > I think the idea here is that ultimately, the suffixed versions of clang
> > > that Debian has in /usr/bin are symlinks to binaries in
> > > /usr/lib/llvm-#/bin; as a result, a user could say
> > > LLVM_DIR=/usr/lib/llvm-#/bin/ and all of those tools would be picked up
> > > automatically. I am not really sure what is better.
> 
> $ sudo apt install clang-8
> $ which clang-8
> /usr/bin/clang-8
> $ ls -l `!!`
> /usr/bin/clang-8 -> ../lib/llvm-8/bin/clang
> $ ls /usr/lib/llvm-8/bin
> <non suffixed versions>
> 
> Ok, so Nathan, it looks like we don't need the version suffixes.
> Instead, we can be more explicit with our $PATH, and only add the
> above (and bintutils).  I was thinking supporting the suffix was
> required for our CI, but it seems like maybe not.

Correct. This should probably be documented though, otherwise people
might use LLVM=y and be surprised when the latest version of their tools
are not being picked up.

> > I periodically build the latest llvm from the trunk,
> > and install it under my home directory.
> > So, I just thought it would be useful to
> > allow a user to specify the llvm directory.
> > Of course, I can do the equivalent by tweaking PATH, but
> > I hesitate to make the non-released version my default.
> 
> Respectfully, I strongly disagree.  This should be handled by
> modifications to $PATH, either by your shell's .rc file when you
> always want it, or exported for a session when you want it, or
> prefixed to an invocation for the duration of that command.  We should
> not have a new variable just for the path of a few tools.
> 
> Rather than `make LLVM_DIR=~/llvm-project LLVM=1`, you can do
> `PATH=$PATH:~/llvm-project make LLVM=1`. (or export it manually or via
> your shell .rc, depending on how comfortable you are with that
> version).

I always do PATH=...:${PATH} make CC=clang...

> > Having both LLVM_DIR and LLVM_SUFFIX seems verbose.
> 
> I agree, so maybe just LLVM=y, and we can support both non-standard
> locations and debian suffixes via modifications to PATH.
> 
> >
> > In fact, the debian provides multiple versions of GCC.
> > For example, my machine has
> >
> > masahiro@pug:~$ ls -1 /usr/bin/gcc-*
> > /usr/bin/gcc-4.8
> > /usr/bin/gcc-5
> > /usr/bin/gcc-7
> > /usr/bin/gcc-ar
> > /usr/bin/gcc-ar-4.8
> > /usr/bin/gcc-ar-5
> > /usr/bin/gcc-ar-7
> > /usr/bin/gcc-nm
> > /usr/bin/gcc-nm-4.8
> > /usr/bin/gcc-nm-5
> > /usr/bin/gcc-nm-7
> > /usr/bin/gcc-ranlib
> > /usr/bin/gcc-ranlib-4.8
> > /usr/bin/gcc-ranlib-5
> > /usr/bin/gcc-ranlib-7
> >
> > But, nobody has suggested GCC_SUFFIX.
> >
> > So, I guess CROSS_COMPILE was enough to
> > choose a specific tool version.
> 
> Or no one was testing specific versions of gcc with more than one
> installed.  I can ask the KernelCI folks next week if this is an issue
> they face or have faced.

Well gcc is just one tool, so specified CC=gcc-5 is not that
complicated; it would get a lot more gnarly if one had different
versions of binutils as well.

Cheers,
Nathan

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Nick Desaulniers]

On Tue, Mar 31, 2020 at 12:35 PM Nathan Chancellor
<natechancellor@gmail.com> wrote:
>
> On Tue, Mar 31, 2020 at 11:39:27AM -0700, Nick Desaulniers wrote:
> > On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > In fact, the debian provides multiple versions of GCC.
> > > For example, my machine has
> > >
> > > masahiro@pug:~$ ls -1 /usr/bin/gcc-*
> > > /usr/bin/gcc-4.8
> > > /usr/bin/gcc-5
> > > /usr/bin/gcc-7
> > > /usr/bin/gcc-ar
> > > /usr/bin/gcc-ar-4.8
> > > /usr/bin/gcc-ar-5
> > > /usr/bin/gcc-ar-7
> > > /usr/bin/gcc-nm
> > > /usr/bin/gcc-nm-4.8
> > > /usr/bin/gcc-nm-5
> > > /usr/bin/gcc-nm-7
> > > /usr/bin/gcc-ranlib
> > > /usr/bin/gcc-ranlib-4.8
> > > /usr/bin/gcc-ranlib-5
> > > /usr/bin/gcc-ranlib-7
> > >
> > > But, nobody has suggested GCC_SUFFIX.
> > >
> > > So, I guess CROSS_COMPILE was enough to
> > > choose a specific tool version.
> >
> > Or no one was testing specific versions of gcc with more than one
> > installed.  I can ask the KernelCI folks next week if this is an issue
> > they face or have faced.
>
> Well gcc is just one tool, so specified CC=gcc-5 is not that
> complicated; it would get a lot more gnarly if one had different
> versions of binutils as well.

Have you had to test different releases of binutils yet? I have, and
it was not fun.  I don't even remember what I did but I recall it
being painful trying to get it to work.  (I think I finally solved it
via temporary symlink).  Speaking of, I should get back to those
dwarf-5 patches I started, now that binutils devs implemented every
missing feature I could find.
-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Masahiro Yamada]

On Wed, Apr 1, 2020 at 3:39 AM Nick Desaulniers <ndesaulniers@google.com> wrote:
>
> On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > On Tue, Mar 31, 2020 at 4:03 AM Nathan Chancellor
> > <natechancellor@gmail.com> wrote:
> > >
> > > On Mon, Mar 30, 2020 at 11:58:19AM -0700, Nick Desaulniers wrote:
> > > > On Sat, Mar 28, 2020 at 6:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > > >
> > > > > I also had planned to provide a single switch to change
> > > > > all the tool defaults to LLVM.
> > > > >
> > > > > So, supporting 'LLVM' is fine, but I'd rather want this
> > > > > look symmetrical, and easy to understand.
> > > > >
> > > > > CPP        = $(CC) -E
> > > > > ifneq ($(LLVM),)
> > > >
> > > > Yes, a simple if statement is much simpler than the overly complex patch I had.
> > > >
> > > > > CC         = $(LLVM_DIR)clang
> > > >
> > > > Do we need $LLVM_DIR? Shouldn't users just have that in their $PATH?
> > > >
> > > > Also, I think we need to support suffixed binaries, as debian
> > > > distributes these with version suffixes, as Nathan points out.  Or do
> > > > the debian packages install suffixed binaries AND path versioned
> > > > non-suffixed binaries?
> > >
> > > I think the idea here is that ultimately, the suffixed versions of clang
> > > that Debian has in /usr/bin are symlinks to binaries in
> > > /usr/lib/llvm-#/bin; as a result, a user could say
> > > LLVM_DIR=/usr/lib/llvm-#/bin/ and all of those tools would be picked up
> > > automatically. I am not really sure what is better.
>
> $ sudo apt install clang-8
> $ which clang-8
> /usr/bin/clang-8
> $ ls -l `!!`
> /usr/bin/clang-8 -> ../lib/llvm-8/bin/clang
> $ ls /usr/lib/llvm-8/bin
> <non suffixed versions>
>
> Ok, so Nathan, it looks like we don't need the version suffixes.
> Instead, we can be more explicit with our $PATH, and only add the
> above (and bintutils).  I was thinking supporting the suffix was
> required for our CI, but it seems like maybe not.
>
> > I periodically build the latest llvm from the trunk,
> > and install it under my home directory.
> > So, I just thought it would be useful to
> > allow a user to specify the llvm directory.
> > Of course, I can do the equivalent by tweaking PATH, but
> > I hesitate to make the non-released version my default.
>
> Respectfully, I strongly disagree.  This should be handled by
> modifications to $PATH, either by your shell's .rc file when you
> always want it, or exported for a session when you want it, or
> prefixed to an invocation for the duration of that command.  We should
> not have a new variable just for the path of a few tools.
>
> Rather than `make LLVM_DIR=~/llvm-project LLVM=1`, you can do
> `PATH=$PATH:~/llvm-project make LLVM=1`. (or export it manually or via
> your shell .rc, depending on how comfortable you are with that
> version).
>
> > Having both LLVM_DIR and LLVM_SUFFIX seems verbose.
>
> I agree, so maybe just LLVM=y, and we can support both non-standard
> locations and debian suffixes via modifications to PATH.



OK, so we will start with the boolean switch 'LLVM'.

People can use PATH to cope with directory path and suffixes.


-- 
Best Regards
Masahiro Yamada

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Nick Desaulniers]

On Tue, Mar 31, 2020 at 11:11 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Wed, Apr 1, 2020 at 3:39 AM Nick Desaulniers <ndesaulniers@google.com> wrote:
> >
> > On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > Having both LLVM_DIR and LLVM_SUFFIX seems verbose.
> >
> > I agree, so maybe just LLVM=y, and we can support both non-standard
> > locations and debian suffixes via modifications to PATH.
>
>
>
> OK, so we will start with the boolean switch 'LLVM'.
>
> People can use PATH to cope with directory path and suffixes.

Sounds good, we will modify our CI to use PATH modifications rather
than suffixes. We can even do that before such a patch to Makefile
exists.
-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Fangrui Song]

On 2020-04-01, 'Nick Desaulniers' via Clang Built Linux wrote:
>On Tue, Mar 31, 2020 at 11:11 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>>
>> On Wed, Apr 1, 2020 at 3:39 AM Nick Desaulniers <ndesaulniers@google.com> wrote:
>> >
>> > On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>> > >
>> > > Having both LLVM_DIR and LLVM_SUFFIX seems verbose.
>> >
>> > I agree, so maybe just LLVM=y, and we can support both non-standard
>> > locations and debian suffixes via modifications to PATH.
>>
>>
>>
>> OK, so we will start with the boolean switch 'LLVM'.
>>
>> People can use PATH to cope with directory path and suffixes.
>
>Sounds good, we will modify our CI to use PATH modifications rather
>than suffixes. We can even do that before such a patch to Makefile
>exists.

The proposed LLVM=1 + PATH scheme looks good to me.


There seems to be one issue.
OBJSIZE=llvm-objsize added in
commit fcf1b6a35c16ac500fa908a4022238e5d666eabf "Documentation/llvm: add documentation on building w/ Clang/LLVM"
is wrong.

The tool is named llvm-size. OBJSIZE is only used once:

   arch/s390/scripts/Makefile.chkbss
   14:     if ! $(OBJSIZE) --common $< | $(AWK) 'END { if ($$3) exit 1 }'; then \

Re: [PATCH v2] Makefile.llvm: simplify LLVM build

[Masahiro Yamada]

On Fri, Apr 3, 2020 at 1:40 AM Fangrui Song <maskray@google.com> wrote:
>
>
> On 2020-04-01, 'Nick Desaulniers' via Clang Built Linux wrote:
> >On Tue, Mar 31, 2020 at 11:11 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >>
> >> On Wed, Apr 1, 2020 at 3:39 AM Nick Desaulniers <ndesaulniers@google.com> wrote:
> >> >
> >> > On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >> > >
> >> > > Having both LLVM_DIR and LLVM_SUFFIX seems verbose.
> >> >
> >> > I agree, so maybe just LLVM=y, and we can support both non-standard
> >> > locations and debian suffixes via modifications to PATH.
> >>
> >>
> >>
> >> OK, so we will start with the boolean switch 'LLVM'.
> >>
> >> People can use PATH to cope with directory path and suffixes.
> >
> >Sounds good, we will modify our CI to use PATH modifications rather
> >than suffixes. We can even do that before such a patch to Makefile
> >exists.
>
> The proposed LLVM=1 + PATH scheme looks good to me.
>
>
> There seems to be one issue.
> OBJSIZE=llvm-objsize added in
> commit fcf1b6a35c16ac500fa908a4022238e5d666eabf "Documentation/llvm: add documentation on building w/ Clang/LLVM"
> is wrong.
>
> The tool is named llvm-size. OBJSIZE is only used once:
>
>    arch/s390/scripts/Makefile.chkbss
>    14:     if ! $(OBJSIZE) --common $< | $(AWK) 'END { if ($$3) exit 1 }'; then \


Good catch.

Could you send a patch, please?




-- 
Best Regards
Masahiro Yamada