* [PATCH v2] Makefile.llvm: simplify LLVM build [not found] <20200317202404.GA20746@ubuntu-m2-xlarge-x86> @ 2020-03-17 21:55 ` Nick Desaulniers 2020-03-27 19:51 ` Nick Desaulniers 2020-03-27 22:42 ` Nathan Chancellor 0 siblings, 2 replies; 15+ messages in thread From: Nick Desaulniers @ 2020-03-17 21:55 UTC (permalink / raw) To: masahiroy, natechancellor Cc: clang-built-linux, linux-kbuild, linux-kernel, Nick Desaulniers Prior to this patch, building the Linux kernel with Clang looked like: $ make CC=clang or when cross compiling: $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang which got very verbose and unwieldy when using all of LLVM's substitutes for GNU binutils: $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang AS=clang \ LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \ OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump OBJSIZE=llvm-objsize \ READELF=llvm-readelf HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar \ HOSTLD=ld.lld This change adds a new Makefile under scripts/ which will be included in the top level Makefile AFTER CC and friends are set, in order to make the use of LLVM utilities when building a Linux kernel more ergonomic. With this patch, the above now looks like: $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y Then you can "opt out" of certain LLVM utilities explicitly: $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y AS=as will instead invoke arm-linux-gnueabihf-as in place of clang for AS. Or when not cross compiling: $ make LLVM=y AS=as This would make it more verbose to opt into just one tool from LLVM, but this patch doesn't actually break the old style; just leave off LLVM=y. Also, LLVM=y CC=clang would wind up prefixing clang with $CROSS_COMPILE. In that case, it's recommended to just drop LLVM=y and use the old style. So LLVM=y can be thought of as default to LLVM with explicit opt ins for GNU, vs the current case of default to GNU and opt in for LLVM. A key part of the design of this patch is to be minimally invasive to the top level Makefile and not break existing workflows. We could get more aggressive, but I'd prefer to save larger refactorings for another day. Finally, some linux distributions package specific versions of LLVM utilities with naming conventions that use the version as a suffix, ie. clang-11. In that case, you can use LLVM=<number> and that number will be used as a suffix. Example: $ make LLVM=11 will invoke clang-11, ld.lld-11, llvm-objcopy-11, etc. About the script: The pattern used in the script is in the form: ifeq "$(origin $(CC))" "file" $(CC) := $(clang) else override $(CC) := $(CROSS_COMPILE)$(CC) endif "Metaprogramming" (eval) is used to template the above to make it more concise for specifying all of the substitutions. The "origin" of a variable tracks whether a variable was explicitly set via "command line", "environment", was defined earlier via Makefile "file", was provided by "default", or was "undefined". Variable assignment in GNU Make has some special and complicated rules. If the variable was set earlier explicitly in the Makefile, we can simply reassign a new value to it. If a variable was unspecified, then earlier assignments were executed and change the origin to file. Otherwise, the variable was explicitly specified. If a variable's "origin" was "command line" or "environment", non-"override" assignments are not executed. The "override" directive forces the assignment regardless of "origin". Some tips I found useful for debugging for future travelers: $(info $$origin of $$CC is $(origin CC)) at the start of the new script for all of the variables can help you understand "default" vs "undefined" variable origins. $(info $$CC is [${CC}]) in the top level Makefile after including the new script, for all of the variables can help you check that they're being set as expected. Link: https://www.gnu.org/software/make/manual/html_node/Eval-Function.html Link: https://www.gnu.org/software/make/manual/html_node/Origin-Function.html Link: https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html Link: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html Suggested-by: Nathan Chancellor <natechancellor@gmail.com> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com> --- Changes V1 -> V2: * Rather than LLVM=1, use LLVM=y to enable all. * LLVM=<anything other than y> becomes a suffix, LLVM_SUFFIX. * strip has to be used on the LLVM_SUFFIX to avoid an extra whitespace. Makefile | 4 ++++ scripts/Makefile.llvm | 30 ++++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+) create mode 100644 scripts/Makefile.llvm diff --git a/Makefile b/Makefile index 402f276da062..72ec9dfea15e 100644 --- a/Makefile +++ b/Makefile @@ -475,6 +475,10 @@ KBUILD_LDFLAGS := GCC_PLUGINS_CFLAGS := CLANG_FLAGS := +ifneq ($(LLVM),) +include scripts/Makefile.llvm +endif + export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE AS LD CC export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX diff --git a/scripts/Makefile.llvm b/scripts/Makefile.llvm new file mode 100644 index 000000000000..0bab45a100a3 --- /dev/null +++ b/scripts/Makefile.llvm @@ -0,0 +1,30 @@ +LLVM_SUFFIX= + +ifneq ($(LLVM),y) +LLVM_SUFFIX += -$(LLVM) +endif + +define META_set = +ifeq "$(origin $(1))" "file" +$(1) := $(2)$(strip $(LLVM_SUFFIX)) +else +override $(1) := $(CROSS_COMPILE)$($(1)) +endif +endef + +$(eval $(call META_set,CC,clang)) +$(eval $(call META_set,AS,clang)) +$(eval $(call META_set,LD,ld.lld)) +$(eval $(call META_set,AR,llvm-ar)) +$(eval $(call META_set,NM,llvm-nm)) +$(eval $(call META_set,STRIP,llvm-strip)) +$(eval $(call META_set,OBJCOPY,llvm-objcopy)) +$(eval $(call META_set,OBJDUMP,llvm-objdump)) +$(eval $(call META_set,OBJSIZE,llvm-objsize)) +$(eval $(call META_set,READELF,llvm-readelf)) +$(eval $(call META_set,HOSTCC,clang)) +$(eval $(call META_set,HOSTCXX,clang++)) +$(eval $(call META_set,HOSTAR,llvm-ar)) +$(eval $(call META_set,HOSTLD,ld.lld)) + +## TODO: HOSTAR, HOSTLD in tools/objtool/Makefile -- 2.25.1.481.gfbce0eb801-goog ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-03-17 21:55 ` [PATCH v2] Makefile.llvm: simplify LLVM build Nick Desaulniers @ 2020-03-27 19:51 ` Nick Desaulniers 2020-03-27 22:42 ` Nathan Chancellor 1 sibling, 0 replies; 15+ messages in thread From: Nick Desaulniers @ 2020-03-27 19:51 UTC (permalink / raw) To: Masahiro Yamada, Nathan Chancellor Cc: clang-built-linux, Linux Kbuild mailing list, LKML, Sandeep Patil Masahiro and Nathan, I get daily reminders that `make CC=clang LD=ld.lld NM=llvm-nm ...` sucks. I always prioritize code reviews for you, can you please carve out time to review+test+provide feedback for me? On Tue, Mar 17, 2020 at 2:55 PM Nick Desaulniers <ndesaulniers@google.com> wrote: > > Prior to this patch, building the Linux kernel with Clang > looked like: > > $ make CC=clang > > or when cross compiling: > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang > > which got very verbose and unwieldy when using all of LLVM's substitutes > for GNU binutils: > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang AS=clang \ > LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \ > OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump OBJSIZE=llvm-objsize \ > READELF=llvm-readelf HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar \ > HOSTLD=ld.lld > > This change adds a new Makefile under scripts/ which will be included in > the top level Makefile AFTER CC and friends are set, in order to make > the use of LLVM utilities when building a Linux kernel more ergonomic. > > With this patch, the above now looks like: > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y > > Then you can "opt out" of certain LLVM utilities explicitly: > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y AS=as > > will instead invoke arm-linux-gnueabihf-as in place of clang for AS. > > Or when not cross compiling: > > $ make LLVM=y AS=as > > This would make it more verbose to opt into just one tool from LLVM, but > this patch doesn't actually break the old style; just leave off LLVM=y. > Also, LLVM=y CC=clang would wind up prefixing clang with $CROSS_COMPILE. > In that case, it's recommended to just drop LLVM=y and use the old > style. So LLVM=y can be thought of as default to LLVM with explicit opt > ins for GNU, vs the current case of default to GNU and opt in for LLVM. > > A key part of the design of this patch is to be minimally invasive to > the top level Makefile and not break existing workflows. We could get > more aggressive, but I'd prefer to save larger refactorings for another > day. > > Finally, some linux distributions package specific versions of LLVM > utilities with naming conventions that use the version as a suffix, ie. > clang-11. In that case, you can use LLVM=<number> and that number will > be used as a suffix. Example: > > $ make LLVM=11 > > will invoke clang-11, ld.lld-11, llvm-objcopy-11, etc. > > About the script: > The pattern used in the script is in the form: > > ifeq "$(origin $(CC))" "file" > $(CC) := $(clang) > else > override $(CC) := $(CROSS_COMPILE)$(CC) > endif > > "Metaprogramming" (eval) is used to template the above to make it more > concise for specifying all of the substitutions. > > The "origin" of a variable tracks whether a variable was explicitly set > via "command line", "environment", was defined earlier via Makefile > "file", was provided by "default", or was "undefined". > > Variable assignment in GNU Make has some special and complicated rules. > > If the variable was set earlier explicitly in the Makefile, we can > simply reassign a new value to it. If a variable was unspecified, then > earlier assignments were executed and change the origin to file. > Otherwise, the variable was explicitly specified. > > If a variable's "origin" was "command line" or "environment", > non-"override" assignments are not executed. The "override" directive > forces the assignment regardless of "origin". > > Some tips I found useful for debugging for future travelers: > > $(info $$origin of $$CC is $(origin CC)) > > at the start of the new script for all of the variables can help you > understand "default" vs "undefined" variable origins. > > $(info $$CC is [${CC}]) > > in the top level Makefile after including the new script, for all of the > variables can help you check that they're being set as expected. > > Link: https://www.gnu.org/software/make/manual/html_node/Eval-Function.html > Link: https://www.gnu.org/software/make/manual/html_node/Origin-Function.html > Link: https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html > Link: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html > Suggested-by: Nathan Chancellor <natechancellor@gmail.com> > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com> > --- > Changes V1 -> V2: > * Rather than LLVM=1, use LLVM=y to enable all. > * LLVM=<anything other than y> becomes a suffix, LLVM_SUFFIX. > * strip has to be used on the LLVM_SUFFIX to avoid an extra whitespace. > > > Makefile | 4 ++++ > scripts/Makefile.llvm | 30 ++++++++++++++++++++++++++++++ > 2 files changed, 34 insertions(+) > create mode 100644 scripts/Makefile.llvm > > diff --git a/Makefile b/Makefile > index 402f276da062..72ec9dfea15e 100644 > --- a/Makefile > +++ b/Makefile > @@ -475,6 +475,10 @@ KBUILD_LDFLAGS := > GCC_PLUGINS_CFLAGS := > CLANG_FLAGS := > > +ifneq ($(LLVM),) > +include scripts/Makefile.llvm > +endif > + > export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE AS LD CC > export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL > export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX > diff --git a/scripts/Makefile.llvm b/scripts/Makefile.llvm > new file mode 100644 > index 000000000000..0bab45a100a3 > --- /dev/null > +++ b/scripts/Makefile.llvm > @@ -0,0 +1,30 @@ > +LLVM_SUFFIX= > + > +ifneq ($(LLVM),y) > +LLVM_SUFFIX += -$(LLVM) > +endif > + > +define META_set = > +ifeq "$(origin $(1))" "file" > +$(1) := $(2)$(strip $(LLVM_SUFFIX)) > +else > +override $(1) := $(CROSS_COMPILE)$($(1)) > +endif > +endef > + > +$(eval $(call META_set,CC,clang)) > +$(eval $(call META_set,AS,clang)) > +$(eval $(call META_set,LD,ld.lld)) > +$(eval $(call META_set,AR,llvm-ar)) > +$(eval $(call META_set,NM,llvm-nm)) > +$(eval $(call META_set,STRIP,llvm-strip)) > +$(eval $(call META_set,OBJCOPY,llvm-objcopy)) > +$(eval $(call META_set,OBJDUMP,llvm-objdump)) > +$(eval $(call META_set,OBJSIZE,llvm-objsize)) > +$(eval $(call META_set,READELF,llvm-readelf)) > +$(eval $(call META_set,HOSTCC,clang)) > +$(eval $(call META_set,HOSTCXX,clang++)) > +$(eval $(call META_set,HOSTAR,llvm-ar)) > +$(eval $(call META_set,HOSTLD,ld.lld)) > + > +## TODO: HOSTAR, HOSTLD in tools/objtool/Makefile > -- > 2.25.1.481.gfbce0eb801-goog > -- Thanks, ~Nick Desaulniers ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-03-17 21:55 ` [PATCH v2] Makefile.llvm: simplify LLVM build Nick Desaulniers 2020-03-27 19:51 ` Nick Desaulniers @ 2020-03-27 22:42 ` Nathan Chancellor 2020-03-28 4:54 ` Masahiro Yamada 1 sibling, 1 reply; 15+ messages in thread From: Nathan Chancellor @ 2020-03-27 22:42 UTC (permalink / raw) To: Nick Desaulniers Cc: masahiroy, clang-built-linux, linux-kbuild, linux-kernel, Sandeep Patil Sorry for the delay in review :( On Tue, Mar 17, 2020 at 02:55:15PM -0700, Nick Desaulniers wrote: > Prior to this patch, building the Linux kernel with Clang > looked like: > > $ make CC=clang > > or when cross compiling: > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang > > which got very verbose and unwieldy when using all of LLVM's substitutes > for GNU binutils: > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang AS=clang \ > LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \ > OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump OBJSIZE=llvm-objsize \ > READELF=llvm-readelf HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar \ > HOSTLD=ld.lld > > This change adds a new Makefile under scripts/ which will be included in > the top level Makefile AFTER CC and friends are set, in order to make > the use of LLVM utilities when building a Linux kernel more ergonomic. > > With this patch, the above now looks like: > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y > > Then you can "opt out" of certain LLVM utilities explicitly: > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y AS=as > > will instead invoke arm-linux-gnueabihf-as in place of clang for AS. > > Or when not cross compiling: > > $ make LLVM=y AS=as > > This would make it more verbose to opt into just one tool from LLVM, but > this patch doesn't actually break the old style; just leave off LLVM=y. > Also, LLVM=y CC=clang would wind up prefixing clang with $CROSS_COMPILE. > In that case, it's recommended to just drop LLVM=y and use the old > style. So LLVM=y can be thought of as default to LLVM with explicit opt > ins for GNU, vs the current case of default to GNU and opt in for LLVM. > > A key part of the design of this patch is to be minimally invasive to > the top level Makefile and not break existing workflows. We could get > more aggressive, but I'd prefer to save larger refactorings for another > day. > > Finally, some linux distributions package specific versions of LLVM > utilities with naming conventions that use the version as a suffix, ie. > clang-11. In that case, you can use LLVM=<number> and that number will > be used as a suffix. Example: > > $ make LLVM=11 > > will invoke clang-11, ld.lld-11, llvm-objcopy-11, etc. > > About the script: > The pattern used in the script is in the form: > > ifeq "$(origin $(CC))" "file" > $(CC) := $(clang) > else > override $(CC) := $(CROSS_COMPILE)$(CC) > endif > > "Metaprogramming" (eval) is used to template the above to make it more > concise for specifying all of the substitutions. > > The "origin" of a variable tracks whether a variable was explicitly set > via "command line", "environment", was defined earlier via Makefile > "file", was provided by "default", or was "undefined". > > Variable assignment in GNU Make has some special and complicated rules. > > If the variable was set earlier explicitly in the Makefile, we can > simply reassign a new value to it. If a variable was unspecified, then > earlier assignments were executed and change the origin to file. > Otherwise, the variable was explicitly specified. > > If a variable's "origin" was "command line" or "environment", > non-"override" assignments are not executed. The "override" directive > forces the assignment regardless of "origin". > > Some tips I found useful for debugging for future travelers: > > $(info $$origin of $$CC is $(origin CC)) > > at the start of the new script for all of the variables can help you > understand "default" vs "undefined" variable origins. > > $(info $$CC is [${CC}]) > > in the top level Makefile after including the new script, for all of the > variables can help you check that they're being set as expected. > > Link: https://www.gnu.org/software/make/manual/html_node/Eval-Function.html > Link: https://www.gnu.org/software/make/manual/html_node/Origin-Function.html > Link: https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html > Link: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html > Suggested-by: Nathan Chancellor <natechancellor@gmail.com> > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com> > --- > Changes V1 -> V2: > * Rather than LLVM=1, use LLVM=y to enable all. > * LLVM=<anything other than y> becomes a suffix, LLVM_SUFFIX. > * strip has to be used on the LLVM_SUFFIX to avoid an extra whitespace. > > > Makefile | 4 ++++ > scripts/Makefile.llvm | 30 ++++++++++++++++++++++++++++++ > 2 files changed, 34 insertions(+) > create mode 100644 scripts/Makefile.llvm > > diff --git a/Makefile b/Makefile > index 402f276da062..72ec9dfea15e 100644 > --- a/Makefile > +++ b/Makefile > @@ -475,6 +475,10 @@ KBUILD_LDFLAGS := > GCC_PLUGINS_CFLAGS := > CLANG_FLAGS := > > +ifneq ($(LLVM),) > +include scripts/Makefile.llvm > +endif > + > export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE AS LD CC > export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL > export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX > diff --git a/scripts/Makefile.llvm b/scripts/Makefile.llvm > new file mode 100644 > index 000000000000..0bab45a100a3 > --- /dev/null > +++ b/scripts/Makefile.llvm > @@ -0,0 +1,30 @@ > +LLVM_SUFFIX= > + > +ifneq ($(LLVM),y) > +LLVM_SUFFIX += -$(LLVM) > +endif > + > +define META_set = > +ifeq "$(origin $(1))" "file" > +$(1) := $(2)$(strip $(LLVM_SUFFIX)) > +else > +override $(1) := $(CROSS_COMPILE)$($(1)) > +endif > +endef > + > +$(eval $(call META_set,CC,clang)) > +$(eval $(call META_set,AS,clang)) > +$(eval $(call META_set,LD,ld.lld)) > +$(eval $(call META_set,AR,llvm-ar)) > +$(eval $(call META_set,NM,llvm-nm)) > +$(eval $(call META_set,STRIP,llvm-strip)) > +$(eval $(call META_set,OBJCOPY,llvm-objcopy)) > +$(eval $(call META_set,OBJDUMP,llvm-objdump)) > +$(eval $(call META_set,OBJSIZE,llvm-objsize)) > +$(eval $(call META_set,READELF,llvm-readelf)) > +$(eval $(call META_set,HOSTCC,clang)) > +$(eval $(call META_set,HOSTCXX,clang++)) > +$(eval $(call META_set,HOSTAR,llvm-ar)) > +$(eval $(call META_set,HOSTLD,ld.lld)) > + > +## TODO: HOSTAR, HOSTLD in tools/objtool/Makefile > -- > 2.25.1.481.gfbce0eb801-goog > The use of override appears to break exporting the value to Kconfig, which causes Kconfig to endlessly loop at the --syncconfig step: $ make -j$(nproc) ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- LD=ld LLVM=y defconfig prepare ... Makefile: LD is riscv64-linux-gnu-ld *** Default configuration is based on 'defconfig' Kconfig: LD is ld # # No change to .config # Makefile: LD is riscv64-linux-gnu-ld Makefile: LD is riscv64-linux-gnu-ld scripts/kconfig/conf --syncconfig Kconfig Kconfig: LD is ld Makefile: LD is riscv64-linux-gnu-ld auto.conf.cmd: LD is riscv64-linux-gnu-ld Makefile: LD is riscv64-linux-gnu-ld scripts/kconfig/conf --syncconfig Kconfig Kconfig: LD is ld Makefile: LD is riscv64-linux-gnu-ld auto.conf.cmd: LD is riscv64-linux-gnu-ld Makefile: LD is riscv64-linux-gnu-ld scripts/kconfig/conf --syncconfig Kconfig Kconfig: LD is ld ... This endless loops happens because auto.conf.cmd is constantly being regenerated because Kconfig is not picking up the override value, as you can see from the debugging output above and auto.conf.cmd below: $ rg -A 2 "LD\)" include/config/auto.conf.cmd | cat ifneq "$(LD)" "ld" include/config/auto.conf: FORCE endif $(LD) evaluates to riscv64-linux-gnu-ld, which is not equal to ld, so include/config/auto.conf is regenerated, but env_write_dep in scripts/kconfig/preprocess.c just picks up the environment value, rather than the value that was overridden. This appears to be related to recursive make calls from what I can tell. $ cat Makefile override FOO := foo export FOO all: $(info Makefile: $$(FOO) is $(FOO)) $(MAKE) -f Makefile.sub all $ cat Makefile.sub all: $(info Makefile.sub: $$(FOO) is $(FOO)) $ make -s Makefile: $(FOO) is foo Makefile.sub: $(FOO) is foo $ make -s FOO=bar Makefile: $(FOO) is foo Makefile.sub: $(FOO) is bar No idea if this is a bug in make or not, this seems subtle. Not really sure where to go from here, hopefully this is useful to you or Masahiro. You will only notice this with variables that impact Kconfig, which is why you never noticed with AS=as. See include/config/auto.conf.cmd for the full list (CC, LD, NM, OBJCOPY, HOSTCC, and HOSTCXX are the ones I see). Debug patch if you want to play around with this. Cheers, Nathan ================================ diff --git a/Makefile b/Makefile index acd8022ddb52..81cbb940e035 100644 --- a/Makefile +++ b/Makefile @@ -479,6 +479,8 @@ ifneq ($(LLVM),) include scripts/Makefile.llvm endif +$(info Makefile: LD is $(LD)) + export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE AS LD CC export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX diff --git a/scripts/Kconfig.include b/scripts/Kconfig.include index 8074f14d9d0d..43a852e0ee93 100644 --- a/scripts/Kconfig.include +++ b/scripts/Kconfig.include @@ -57,3 +57,5 @@ gcc-version := $(shell,$(srctree)/scripts/gcc-version.sh $(CC)) cc-option-bit = $(if-success,$(CC) -Werror $(1) -E -x c /dev/null -o /dev/null,$(1)) m32-flag := $(cc-option-bit,-m32) m64-flag := $(cc-option-bit,-m64) + +$(info,Kconfig: LD is $(LD)) diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c index 0243086fb168..fd828efc85fc 100644 --- a/scripts/kconfig/preprocess.c +++ b/scripts/kconfig/preprocess.c @@ -93,6 +93,7 @@ void env_write_dep(FILE *f, const char *autoconfig_name) list_for_each_entry_safe(e, tmp, &env_list, node) { fprintf(f, "ifneq \"$(%s)\" \"%s\"\n", e->name, e->value); + fprintf(f, "$(info auto.conf.cmd: %s is $(%s))\n", e->name, e->name); fprintf(f, "%s: FORCE\n", autoconfig_name); fprintf(f, "endif\n"); env_del(e); ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-03-27 22:42 ` Nathan Chancellor @ 2020-03-28 4:54 ` Masahiro Yamada 2020-03-29 1:56 ` Masahiro Yamada 0 siblings, 1 reply; 15+ messages in thread From: Masahiro Yamada @ 2020-03-28 4:54 UTC (permalink / raw) To: Nathan Chancellor Cc: Nick Desaulniers, clang-built-linux, Linux Kbuild mailing list, Linux Kernel Mailing List, Sandeep Patil On Sat, Mar 28, 2020 at 7:42 AM Nathan Chancellor <natechancellor@gmail.com> wrote: > > Sorry for the delay in review :( > > On Tue, Mar 17, 2020 at 02:55:15PM -0700, Nick Desaulniers wrote: > > Prior to this patch, building the Linux kernel with Clang > > looked like: > > > > $ make CC=clang > > > > or when cross compiling: > > > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang > > > > which got very verbose and unwieldy when using all of LLVM's substitutes > > for GNU binutils: > > > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang AS=clang \ > > LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \ > > OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump OBJSIZE=llvm-objsize \ > > READELF=llvm-readelf HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar \ > > HOSTLD=ld.lld > > > > This change adds a new Makefile under scripts/ which will be included in > > the top level Makefile AFTER CC and friends are set, in order to make > > the use of LLVM utilities when building a Linux kernel more ergonomic. > > > > With this patch, the above now looks like: > > > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y > > > > Then you can "opt out" of certain LLVM utilities explicitly: > > > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y AS=as > > > > will instead invoke arm-linux-gnueabihf-as in place of clang for AS. > > > > Or when not cross compiling: > > > > $ make LLVM=y AS=as > > > > This would make it more verbose to opt into just one tool from LLVM, but > > this patch doesn't actually break the old style; just leave off LLVM=y. > > Also, LLVM=y CC=clang would wind up prefixing clang with $CROSS_COMPILE. > > In that case, it's recommended to just drop LLVM=y and use the old > > style. So LLVM=y can be thought of as default to LLVM with explicit opt > > ins for GNU, vs the current case of default to GNU and opt in for LLVM. > > > > A key part of the design of this patch is to be minimally invasive to > > the top level Makefile and not break existing workflows. We could get > > more aggressive, but I'd prefer to save larger refactorings for another > > day. > > > > Finally, some linux distributions package specific versions of LLVM > > utilities with naming conventions that use the version as a suffix, ie. > > clang-11. In that case, you can use LLVM=<number> and that number will > > be used as a suffix. Example: > > > > $ make LLVM=11 > > > > will invoke clang-11, ld.lld-11, llvm-objcopy-11, etc. > > > > About the script: > > The pattern used in the script is in the form: > > > > ifeq "$(origin $(CC))" "file" > > $(CC) := $(clang) > > else > > override $(CC) := $(CROSS_COMPILE)$(CC) > > endif > > > > "Metaprogramming" (eval) is used to template the above to make it more > > concise for specifying all of the substitutions. > > > > The "origin" of a variable tracks whether a variable was explicitly set > > via "command line", "environment", was defined earlier via Makefile > > "file", was provided by "default", or was "undefined". > > > > Variable assignment in GNU Make has some special and complicated rules. > > > > If the variable was set earlier explicitly in the Makefile, we can > > simply reassign a new value to it. If a variable was unspecified, then > > earlier assignments were executed and change the origin to file. > > Otherwise, the variable was explicitly specified. > > > > If a variable's "origin" was "command line" or "environment", > > non-"override" assignments are not executed. The "override" directive > > forces the assignment regardless of "origin". > > > > Some tips I found useful for debugging for future travelers: > > > > $(info $$origin of $$CC is $(origin CC)) > > > > at the start of the new script for all of the variables can help you > > understand "default" vs "undefined" variable origins. > > > > $(info $$CC is [${CC}]) > > > > in the top level Makefile after including the new script, for all of the > > variables can help you check that they're being set as expected. > > > > Link: https://www.gnu.org/software/make/manual/html_node/Eval-Function.html > > Link: https://www.gnu.org/software/make/manual/html_node/Origin-Function.html > > Link: https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html > > Link: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html > > Suggested-by: Nathan Chancellor <natechancellor@gmail.com> > > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com> > > --- > > Changes V1 -> V2: > > * Rather than LLVM=1, use LLVM=y to enable all. > > * LLVM=<anything other than y> becomes a suffix, LLVM_SUFFIX. > > * strip has to be used on the LLVM_SUFFIX to avoid an extra whitespace. > > > > > > Makefile | 4 ++++ > > scripts/Makefile.llvm | 30 ++++++++++++++++++++++++++++++ > > 2 files changed, 34 insertions(+) > > create mode 100644 scripts/Makefile.llvm > > > > diff --git a/Makefile b/Makefile > > index 402f276da062..72ec9dfea15e 100644 > > --- a/Makefile > > +++ b/Makefile > > @@ -475,6 +475,10 @@ KBUILD_LDFLAGS := > > GCC_PLUGINS_CFLAGS := > > CLANG_FLAGS := > > > > +ifneq ($(LLVM),) > > +include scripts/Makefile.llvm > > +endif > > + > > export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE AS LD CC > > export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL > > export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX > > diff --git a/scripts/Makefile.llvm b/scripts/Makefile.llvm > > new file mode 100644 > > index 000000000000..0bab45a100a3 > > --- /dev/null > > +++ b/scripts/Makefile.llvm > > @@ -0,0 +1,30 @@ > > +LLVM_SUFFIX= > > + > > +ifneq ($(LLVM),y) > > +LLVM_SUFFIX += -$(LLVM) > > +endif > > + > > +define META_set = > > +ifeq "$(origin $(1))" "file" > > +$(1) := $(2)$(strip $(LLVM_SUFFIX)) > > +else > > +override $(1) := $(CROSS_COMPILE)$($(1)) > > +endif > > +endef > > + > > +$(eval $(call META_set,CC,clang)) > > +$(eval $(call META_set,AS,clang)) > > +$(eval $(call META_set,LD,ld.lld)) > > +$(eval $(call META_set,AR,llvm-ar)) > > +$(eval $(call META_set,NM,llvm-nm)) > > +$(eval $(call META_set,STRIP,llvm-strip)) > > +$(eval $(call META_set,OBJCOPY,llvm-objcopy)) > > +$(eval $(call META_set,OBJDUMP,llvm-objdump)) > > +$(eval $(call META_set,OBJSIZE,llvm-objsize)) > > +$(eval $(call META_set,READELF,llvm-readelf)) > > +$(eval $(call META_set,HOSTCC,clang)) > > +$(eval $(call META_set,HOSTCXX,clang++)) > > +$(eval $(call META_set,HOSTAR,llvm-ar)) > > +$(eval $(call META_set,HOSTLD,ld.lld)) > > + > > +## TODO: HOSTAR, HOSTLD in tools/objtool/Makefile > > -- > > 2.25.1.481.gfbce0eb801-goog > > > > The use of override appears to break exporting the value to Kconfig, > which causes Kconfig to endlessly loop at the --syncconfig step: > > $ make -j$(nproc) ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- LD=ld LLVM=y defconfig prepare > ... > Makefile: LD is riscv64-linux-gnu-ld > *** Default configuration is based on 'defconfig' > Kconfig: LD is ld > # > # No change to .config > # > Makefile: LD is riscv64-linux-gnu-ld > Makefile: LD is riscv64-linux-gnu-ld > scripts/kconfig/conf --syncconfig Kconfig > Kconfig: LD is ld > Makefile: LD is riscv64-linux-gnu-ld > auto.conf.cmd: LD is riscv64-linux-gnu-ld > Makefile: LD is riscv64-linux-gnu-ld > scripts/kconfig/conf --syncconfig Kconfig > Kconfig: LD is ld > Makefile: LD is riscv64-linux-gnu-ld > auto.conf.cmd: LD is riscv64-linux-gnu-ld > Makefile: LD is riscv64-linux-gnu-ld > scripts/kconfig/conf --syncconfig Kconfig > Kconfig: LD is ld > ... Kconfig is invoked from scripts/kconfig/Makefile instead of the top Makefile. Just add one more debug code $(info scripts/kconfig/Makefile: LD is $(LD)) into scripts/kconfig/Makefile. It is "ld", not "riscv64-linux-gnu-ld". The variable passed from the command line is strong, and overrides the variable in sub-make as well. > This endless loops happens because auto.conf.cmd is constantly being > regenerated because Kconfig is not picking up the override value, as you > can see from the debugging output above and auto.conf.cmd below: > > $ rg -A 2 "LD\)" include/config/auto.conf.cmd | cat > ifneq "$(LD)" "ld" > include/config/auto.conf: FORCE > endif > > $(LD) evaluates to riscv64-linux-gnu-ld, which is not equal to ld, so > include/config/auto.conf is regenerated, but env_write_dep in > scripts/kconfig/preprocess.c just picks up the environment value, rather > than the value that was overridden. This appears to be related to > recursive make calls from what I can tell. > > $ cat Makefile > override FOO := foo > export FOO > > all: > $(info Makefile: $$(FOO) is $(FOO)) > $(MAKE) -f Makefile.sub all > > $ cat Makefile.sub > all: > $(info Makefile.sub: $$(FOO) is $(FOO)) > > $ make -s > Makefile: $(FOO) is foo > Makefile.sub: $(FOO) is foo > > $ make -s FOO=bar > Makefile: $(FOO) is foo > Makefile.sub: $(FOO) is bar > > No idea if this is a bug in make or not, this seems subtle. Not really > sure where to go from here, hopefully this is useful to you or Masahiro. This is not a bug. The variable passed from the command line (which is inherited to sub-make as well) is stronger than the environment variable. "make -s FOO=BAR" is not equivalent to "FOO=BAR make -s". -- Best Regards Masahiro Yamada ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-03-28 4:54 ` Masahiro Yamada @ 2020-03-29 1:56 ` Masahiro Yamada 2020-03-30 18:58 ` Nick Desaulniers 0 siblings, 1 reply; 15+ messages in thread From: Masahiro Yamada @ 2020-03-29 1:56 UTC (permalink / raw) To: Nathan Chancellor Cc: Nick Desaulniers, clang-built-linux, Linux Kbuild mailing list, Linux Kernel Mailing List, Sandeep Patil [-- Attachment #1: Type: text/plain, Size: 7951 bytes --] On Sat, Mar 28, 2020 at 1:54 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > On Sat, Mar 28, 2020 at 7:42 AM Nathan Chancellor > <natechancellor@gmail.com> wrote: > > > > Sorry for the delay in review :( > > > > On Tue, Mar 17, 2020 at 02:55:15PM -0700, Nick Desaulniers wrote: > > > Prior to this patch, building the Linux kernel with Clang > > > looked like: > > > > > > $ make CC=clang > > > > > > or when cross compiling: > > > > > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang > > > > > > which got very verbose and unwieldy when using all of LLVM's substitutes > > > for GNU binutils: > > > > > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make CC=clang AS=clang \ > > > LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \ > > > OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump OBJSIZE=llvm-objsize \ > > > READELF=llvm-readelf HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar \ > > > HOSTLD=ld.lld > > > > > > This change adds a new Makefile under scripts/ which will be included in > > > the top level Makefile AFTER CC and friends are set, in order to make > > > the use of LLVM utilities when building a Linux kernel more ergonomic. > > > > > > With this patch, the above now looks like: > > > > > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y > > > > > > Then you can "opt out" of certain LLVM utilities explicitly: > > > > > > $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make LLVM=y AS=as > > > > > > will instead invoke arm-linux-gnueabihf-as in place of clang for AS. > > > > > > Or when not cross compiling: > > > > > > $ make LLVM=y AS=as > > > > > > This would make it more verbose to opt into just one tool from LLVM, but > > > this patch doesn't actually break the old style; just leave off LLVM=y. > > > Also, LLVM=y CC=clang would wind up prefixing clang with $CROSS_COMPILE. > > > In that case, it's recommended to just drop LLVM=y and use the old > > > style. So LLVM=y can be thought of as default to LLVM with explicit opt > > > ins for GNU, vs the current case of default to GNU and opt in for LLVM. > > > > > > A key part of the design of this patch is to be minimally invasive to > > > the top level Makefile and not break existing workflows. We could get > > > more aggressive, but I'd prefer to save larger refactorings for another > > > day. > > > > > > Finally, some linux distributions package specific versions of LLVM > > > utilities with naming conventions that use the version as a suffix, ie. > > > clang-11. In that case, you can use LLVM=<number> and that number will > > > be used as a suffix. Example: > > > > > > $ make LLVM=11 > > > > > > will invoke clang-11, ld.lld-11, llvm-objcopy-11, etc. > > > > > > About the script: > > > The pattern used in the script is in the form: > > > > > > ifeq "$(origin $(CC))" "file" > > > $(CC) := $(clang) > > > else > > > override $(CC) := $(CROSS_COMPILE)$(CC) > > > endif > > > > > > "Metaprogramming" (eval) is used to template the above to make it more > > > concise for specifying all of the substitutions. > > > > > > The "origin" of a variable tracks whether a variable was explicitly set > > > via "command line", "environment", was defined earlier via Makefile > > > "file", was provided by "default", or was "undefined". > > > > > > Variable assignment in GNU Make has some special and complicated rules. > > > > > > If the variable was set earlier explicitly in the Makefile, we can > > > simply reassign a new value to it. If a variable was unspecified, then > > > earlier assignments were executed and change the origin to file. > > > Otherwise, the variable was explicitly specified. > > > > > > If a variable's "origin" was "command line" or "environment", > > > non-"override" assignments are not executed. The "override" directive > > > forces the assignment regardless of "origin". > > > > > > Some tips I found useful for debugging for future travelers: > > > > > > $(info $$origin of $$CC is $(origin CC)) > > > > > > at the start of the new script for all of the variables can help you > > > understand "default" vs "undefined" variable origins. > > > > > > $(info $$CC is [${CC}]) > > > > > > in the top level Makefile after including the new script, for all of the > > > variables can help you check that they're being set as expected. > > > > > > Link: https://www.gnu.org/software/make/manual/html_node/Eval-Function.html > > > Link: https://www.gnu.org/software/make/manual/html_node/Origin-Function.html > > > Link: https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html > > > Link: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html > > > Suggested-by: Nathan Chancellor <natechancellor@gmail.com> > > > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com> > > > --- > > > Changes V1 -> V2: > > > * Rather than LLVM=1, use LLVM=y to enable all. > > > * LLVM=<anything other than y> becomes a suffix, LLVM_SUFFIX. > > > * strip has to be used on the LLVM_SUFFIX to avoid an extra whitespace. > > > > > > > > > Makefile | 4 ++++ > > > scripts/Makefile.llvm | 30 ++++++++++++++++++++++++++++++ > > > 2 files changed, 34 insertions(+) > > > create mode 100644 scripts/Makefile.llvm > > > > > > diff --git a/Makefile b/Makefile > > > index 402f276da062..72ec9dfea15e 100644 > > > --- a/Makefile > > > +++ b/Makefile > > > @@ -475,6 +475,10 @@ KBUILD_LDFLAGS := > > > GCC_PLUGINS_CFLAGS := > > > CLANG_FLAGS := > > > > > > +ifneq ($(LLVM),) > > > +include scripts/Makefile.llvm > > > +endif > > > + > > > export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE AS LD CC > > > export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK INSTALLKERNEL > > > export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX > > > diff --git a/scripts/Makefile.llvm b/scripts/Makefile.llvm > > > new file mode 100644 > > > index 000000000000..0bab45a100a3 > > > --- /dev/null > > > +++ b/scripts/Makefile.llvm > > > @@ -0,0 +1,30 @@ > > > +LLVM_SUFFIX= > > > + > > > +ifneq ($(LLVM),y) > > > +LLVM_SUFFIX += -$(LLVM) > > > +endif > > > + > > > +define META_set = > > > +ifeq "$(origin $(1))" "file" > > > +$(1) := $(2)$(strip $(LLVM_SUFFIX)) > > > +else > > > +override $(1) := $(CROSS_COMPILE)$($(1)) > > > +endif > > > +endef > > > + > > > +$(eval $(call META_set,CC,clang)) > > > +$(eval $(call META_set,AS,clang)) > > > +$(eval $(call META_set,LD,ld.lld)) > > > +$(eval $(call META_set,AR,llvm-ar)) > > > +$(eval $(call META_set,NM,llvm-nm)) > > > +$(eval $(call META_set,STRIP,llvm-strip)) > > > +$(eval $(call META_set,OBJCOPY,llvm-objcopy)) > > > +$(eval $(call META_set,OBJDUMP,llvm-objdump)) > > > +$(eval $(call META_set,OBJSIZE,llvm-objsize)) > > > +$(eval $(call META_set,READELF,llvm-readelf)) > > > +$(eval $(call META_set,HOSTCC,clang)) > > > +$(eval $(call META_set,HOSTCXX,clang++)) > > > +$(eval $(call META_set,HOSTAR,llvm-ar)) > > > +$(eval $(call META_set,HOSTLD,ld.lld)) > > > + > > > +## TODO: HOSTAR, HOSTLD in tools/objtool/Makefile > > > -- I also had planned to provide a single switch to change all the tool defaults to LLVM. So, supporting 'LLVM' is fine, but I'd rather want this look symmetrical, and easy to understand. CPP = $(CC) -E ifneq ($(LLVM),) CC = $(LLVM_DIR)clang LD = $(LLVM_DIR)ld.lld AR = $(LLVM_DIR)llvm-ar NM = $(LLVM_DIR)llvm-nm OBJCOPY = $(LLVM_DIR)llvm-objcopy OBJDUMP = $(LLVM_DIR)llvm-objdump READELF = $(LLVM_DIR)llvm-readelf OBJSIZE = $(LLVM_DIR)llvm-size STRIP = $(LLVM_DIR)llvm-strip else CC = $(CROSS_COMPILE)gcc LD = $(CROSS_COMPILE)ld AR = $(CROSS_COMPILE)ar NM = $(CROSS_COMPILE)nm OBJCOPY = $(CROSS_COMPILE)objcopy OBJDUMP = $(CROSS_COMPILE)objdump READELF = $(CROSS_COMPILE)readelf OBJSIZE = $(CROSS_COMPILE)size STRIP = $(CROSS_COMPILE)strip endif I attached two patches. Comments appreciated. -- Best Regards Masahiro Yamada [-- Attachment #2: 0002-kbuild-change-the-default-of-HOST-CC-CXX-to-cc-and-c.patch --] [-- Type: text/x-patch, Size: 2128 bytes --] From f3a620abee5e26dda04f7a747a54ab075f45b70d Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Sat, 28 Mar 2020 18:57:25 +0900 Subject: [PATCH 2/2] kbuild: change the default of HOST{CC,CXX} to cc and c++ I have been thinking how to cater to those who want to build host programs with Clang. We could use the variable 'LLVM' to switch the default of HOST{CC,CXX}: ifneq ($(LLVM),) HOSTCC = clang HOSTCXX = clang++ else HOSTCC = gcc HOSTCXX = g++ endif But, I do not know how many people care about this. There is no tricky code in userspace programs, and we know we can compile them with GCC, Clang, or whatever. No matter what you use for host programs, it makes no difference to the kernel itself. If you are so addicted to LLVM or if you are using a system with no GCC installation, probably you had already had 'cc' and 'c++' point to Clang. So, another approach is to just leave this up to the system. You can manually set up symlinks, or maybe your distro provides 'alternatives'. $ update-alternatives --list cc /usr/bin/clang /usr/bin/gcc $ update-alternatives --list c++ /usr/bin/clang++ /usr/bin/g++ I have no idea what to do for tools/objtool/Makefile, which uses HOSTAR and HOSTLD, but this is because objtool intentionally opts out Kbuild. If objtool is willing to join the Kbuild infrastructure, a patch exists: https://patchwork.kernel.org/patch/10839051/ Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index f192f9bd8343..ecdd34ad0750 100644 --- a/Makefile +++ b/Makefile @@ -399,8 +399,8 @@ HOST_LFS_CFLAGS := $(shell getconf LFS_CFLAGS 2>/dev/null) HOST_LFS_LDFLAGS := $(shell getconf LFS_LDFLAGS 2>/dev/null) HOST_LFS_LIBS := $(shell getconf LFS_LIBS 2>/dev/null) -HOSTCC = gcc -HOSTCXX = g++ +HOSTCC = cc +HOSTCXX = c++ KBUILD_HOSTCFLAGS := -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 \ -fomit-frame-pointer -std=gnu89 $(HOST_LFS_CFLAGS) \ $(HOSTCFLAGS) -- 2.17.1 [-- Attachment #3: 0001-kbuild-support-LLVM-to-switch-the-default-tools-to-C.patch --] [-- Type: text/x-patch, Size: 4367 bytes --] From 6620f13807b466c5e4af08b2d5d33f5a433b1e3f Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Sat, 28 Mar 2020 15:54:47 +0900 Subject: [PATCH 1/2] kbuild: support 'LLVM' to switch the default tools to Clang/LLVM As Documentation/kbuild/llvm.rst implies, building the kernel with a full set of LLVM tools gets very verbose and unwieldy. Provide a single switch 'LLVM' to use Clang and LLVM tools instead of GCC and Binutils. You can pass LLVM=1 from the command line or as an environment variable. Then, Kbuild will use LLVM toolchains in your PATH environment. This may not be convenient if you have multiple versions of LLVM. CROSS_COMPILE is used to specify not only the tool prefix, but also the directory path to the tools. For example, $ make ARCH=arm64 CROSS_COMPILE=/path/to/my/gcc/bin/aarch64-linux-gnu- To support a similar flow, this commit adds another variable, LLVM_DIR, to point to the specific installation of LLVM: $ make LLVM=1 LLVM_DIR=/path/to/my/llvm/bin/ It might be tedious to set two variables. So, the following is the shorthand: $ make LLVM=/path/to/my/llvm/bin/ Please note LLVM=1 does not turn on the LLVM integrated assembler. You need to pass AS=clang to use it. When the upstream kernel is ready for the integrated assembler, we can make it default. We will get rid of --no-integrated-as, then CROSS_COMPILE will be no longer needed. The --target option will be specified by other means. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- Documentation/kbuild/kbuild.rst | 6 ++++++ Documentation/kbuild/llvm.rst | 5 +++++ Makefile | 29 +++++++++++++++++++++++++---- 3 files changed, 36 insertions(+), 4 deletions(-) diff --git a/Documentation/kbuild/kbuild.rst b/Documentation/kbuild/kbuild.rst index f1e5dce86af7..39bb3636e4f4 100644 --- a/Documentation/kbuild/kbuild.rst +++ b/Documentation/kbuild/kbuild.rst @@ -262,3 +262,9 @@ KBUILD_BUILD_USER, KBUILD_BUILD_HOST These two variables allow to override the user@host string displayed during boot and in /proc/version. The default value is the output of the commands whoami and host, respectively. + +LLVM +---- +If this variable is set to 1, Kbuild will use Clang and LLVM utilities instead +of GCC and GNU binutils to build the kernel. +If set to a value other than 1, it points to directory path to LLVM to be used. diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst index d6c79eb4e23e..4602369f6a4f 100644 --- a/Documentation/kbuild/llvm.rst +++ b/Documentation/kbuild/llvm.rst @@ -55,6 +55,11 @@ additional parameters to `make`. READELF=llvm-readelf HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar \\ HOSTLD=ld.lld +You can use a single switch `LLVM=1` to use LLVM utilities by default (except +for building host programs). + + make LLVM=1 HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar HOSTLD=ld.lld + Getting Help ------------ diff --git a/Makefile b/Makefile index a3bc8bc562ee..f192f9bd8343 100644 --- a/Makefile +++ b/Makefile @@ -408,17 +408,38 @@ KBUILD_HOSTCXXFLAGS := -O2 $(HOST_LFS_CFLAGS) $(HOSTCXXFLAGS) KBUILD_HOSTLDFLAGS := $(HOST_LFS_LDFLAGS) $(HOSTLDFLAGS) KBUILD_HOSTLDLIBS := $(HOST_LFS_LIBS) $(HOSTLDLIBS) +# LLVM=1 tells Kbuild to use Clang and LLVM utilities by default. +# You can still override CC, LD, etc. individually if desired. +# +# If LLVM is set to a value other than 1, it is set to LLVM_DIR, +# which is useful to select a specific LLVM installation. +ifneq ($(filter-out 1,$(LLVM)),) +LLVM_DIR := $(LLVM) +endif + # Make variables (CC, etc...) -LD = $(CROSS_COMPILE)ld -CC = $(CROSS_COMPILE)gcc CPP = $(CC) -E +ifneq ($(LLVM),) +CC = $(LLVM_DIR)clang +LD = $(LLVM_DIR)ld.lld +AR = $(LLVM_DIR)llvm-ar +NM = $(LLVM_DIR)llvm-nm +OBJCOPY = $(LLVM_DIR)llvm-objcopy +OBJDUMP = $(LLVM_DIR)llvm-objdump +READELF = $(LLVM_DIR)llvm-readelf +OBJSIZE = $(LLVM_DIR)llvm-size +STRIP = $(LLVM_DIR)llvm-strip +else +CC = $(CROSS_COMPILE)gcc +LD = $(CROSS_COMPILE)ld AR = $(CROSS_COMPILE)ar NM = $(CROSS_COMPILE)nm -STRIP = $(CROSS_COMPILE)strip OBJCOPY = $(CROSS_COMPILE)objcopy OBJDUMP = $(CROSS_COMPILE)objdump -OBJSIZE = $(CROSS_COMPILE)size READELF = $(CROSS_COMPILE)readelf +OBJSIZE = $(CROSS_COMPILE)size +STRIP = $(CROSS_COMPILE)strip +endif PAHOLE = pahole LEX = flex YACC = bison -- 2.17.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-03-29 1:56 ` Masahiro Yamada @ 2020-03-30 18:58 ` Nick Desaulniers 2020-03-30 19:03 ` Nathan Chancellor 0 siblings, 1 reply; 15+ messages in thread From: Nick Desaulniers @ 2020-03-30 18:58 UTC (permalink / raw) To: Masahiro Yamada Cc: Nathan Chancellor, clang-built-linux, Linux Kbuild mailing list, Linux Kernel Mailing List, Sandeep Patil On Sat, Mar 28, 2020 at 6:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > I also had planned to provide a single switch to change > all the tool defaults to LLVM. > > So, supporting 'LLVM' is fine, but I'd rather want this > look symmetrical, and easy to understand. > > CPP = $(CC) -E > ifneq ($(LLVM),) Yes, a simple if statement is much simpler than the overly complex patch I had. > CC = $(LLVM_DIR)clang Do we need $LLVM_DIR? Shouldn't users just have that in their $PATH? Also, I think we need to support suffixed binaries, as debian distributes these with version suffixes, as Nathan points out. Or do the debian packages install suffixed binaries AND path versioned non-suffixed binaries? > LD = $(LLVM_DIR)ld.lld > AR = $(LLVM_DIR)llvm-ar > NM = $(LLVM_DIR)llvm-nm > OBJCOPY = $(LLVM_DIR)llvm-objcopy > OBJDUMP = $(LLVM_DIR)llvm-objdump > READELF = $(LLVM_DIR)llvm-readelf > OBJSIZE = $(LLVM_DIR)llvm-size > STRIP = $(LLVM_DIR)llvm-strip > else > CC = $(CROSS_COMPILE)gcc > LD = $(CROSS_COMPILE)ld > AR = $(CROSS_COMPILE)ar > NM = $(CROSS_COMPILE)nm > OBJCOPY = $(CROSS_COMPILE)objcopy > OBJDUMP = $(CROSS_COMPILE)objdump > READELF = $(CROSS_COMPILE)readelf > OBJSIZE = $(CROSS_COMPILE)size > STRIP = $(CROSS_COMPILE)strip > endif > > > > I attached two patches. > Comments appreciated. I'm not sure the second one that recommends changing cc/c++ is the way to go; I think it might harm hermeticity. > If you are so addicted to LLVM Lol, maybe? -- Thanks, ~Nick Desaulniers ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-03-30 18:58 ` Nick Desaulniers @ 2020-03-30 19:03 ` Nathan Chancellor 2020-03-31 6:24 ` Masahiro Yamada 0 siblings, 1 reply; 15+ messages in thread From: Nathan Chancellor @ 2020-03-30 19:03 UTC (permalink / raw) To: Nick Desaulniers Cc: Masahiro Yamada, clang-built-linux, Linux Kbuild mailing list, Linux Kernel Mailing List, Sandeep Patil On Mon, Mar 30, 2020 at 11:58:19AM -0700, Nick Desaulniers wrote: > On Sat, Mar 28, 2020 at 6:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > I also had planned to provide a single switch to change > > all the tool defaults to LLVM. > > > > So, supporting 'LLVM' is fine, but I'd rather want this > > look symmetrical, and easy to understand. > > > > CPP = $(CC) -E > > ifneq ($(LLVM),) > > Yes, a simple if statement is much simpler than the overly complex patch I had. > > > CC = $(LLVM_DIR)clang > > Do we need $LLVM_DIR? Shouldn't users just have that in their $PATH? > > Also, I think we need to support suffixed binaries, as debian > distributes these with version suffixes, as Nathan points out. Or do > the debian packages install suffixed binaries AND path versioned > non-suffixed binaries? I think the idea here is that ultimately, the suffixed versions of clang that Debian has in /usr/bin are symlinks to binaries in /usr/lib/llvm-#/bin; as a result, a user could say LLVM_DIR=/usr/lib/llvm-#/bin/ and all of those tools would be picked up automatically. I am not really sure what is better. I'll try to have some other comments by later today/tonight. > > LD = $(LLVM_DIR)ld.lld > > AR = $(LLVM_DIR)llvm-ar > > NM = $(LLVM_DIR)llvm-nm > > OBJCOPY = $(LLVM_DIR)llvm-objcopy > > OBJDUMP = $(LLVM_DIR)llvm-objdump > > READELF = $(LLVM_DIR)llvm-readelf > > OBJSIZE = $(LLVM_DIR)llvm-size > > STRIP = $(LLVM_DIR)llvm-strip > > else > > CC = $(CROSS_COMPILE)gcc > > LD = $(CROSS_COMPILE)ld > > AR = $(CROSS_COMPILE)ar > > NM = $(CROSS_COMPILE)nm > > OBJCOPY = $(CROSS_COMPILE)objcopy > > OBJDUMP = $(CROSS_COMPILE)objdump > > READELF = $(CROSS_COMPILE)readelf > > OBJSIZE = $(CROSS_COMPILE)size > > STRIP = $(CROSS_COMPILE)strip > > endif > > > > > > > > I attached two patches. > > Comments appreciated. > > I'm not sure the second one that recommends changing cc/c++ is the way > to go; I think it might harm hermeticity. Agreed. I do not modify my host system at all for this project, just relying on PATH modification. In theory, we can still override HOSTCC and HOSTCXX but that would defeat the purpose of that patch. Cheers, Nathan ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-03-30 19:03 ` Nathan Chancellor @ 2020-03-31 6:24 ` Masahiro Yamada 2020-03-31 18:39 ` Nick Desaulniers 0 siblings, 1 reply; 15+ messages in thread From: Masahiro Yamada @ 2020-03-31 6:24 UTC (permalink / raw) To: Nathan Chancellor Cc: Nick Desaulniers, clang-built-linux, Linux Kbuild mailing list, Linux Kernel Mailing List, Sandeep Patil On Tue, Mar 31, 2020 at 4:03 AM Nathan Chancellor <natechancellor@gmail.com> wrote: > > On Mon, Mar 30, 2020 at 11:58:19AM -0700, Nick Desaulniers wrote: > > On Sat, Mar 28, 2020 at 6:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > > > I also had planned to provide a single switch to change > > > all the tool defaults to LLVM. > > > > > > So, supporting 'LLVM' is fine, but I'd rather want this > > > look symmetrical, and easy to understand. > > > > > > CPP = $(CC) -E > > > ifneq ($(LLVM),) > > > > Yes, a simple if statement is much simpler than the overly complex patch I had. > > > > > CC = $(LLVM_DIR)clang > > > > Do we need $LLVM_DIR? Shouldn't users just have that in their $PATH? > > > > Also, I think we need to support suffixed binaries, as debian > > distributes these with version suffixes, as Nathan points out. Or do > > the debian packages install suffixed binaries AND path versioned > > non-suffixed binaries? > > I think the idea here is that ultimately, the suffixed versions of clang > that Debian has in /usr/bin are symlinks to binaries in > /usr/lib/llvm-#/bin; as a result, a user could say > LLVM_DIR=/usr/lib/llvm-#/bin/ and all of those tools would be picked up > automatically. I am not really sure what is better. I periodically build the latest llvm from the trunk, and install it under my home directory. So, I just thought it would be useful to allow a user to specify the llvm directory. Of course, I can do the equivalent by tweaking PATH, but I hesitate to make the non-released version my default. Having both LLVM_DIR and LLVM_SUFFIX seems verbose. In fact, the debian provides multiple versions of GCC. For example, my machine has masahiro@pug:~$ ls -1 /usr/bin/gcc-* /usr/bin/gcc-4.8 /usr/bin/gcc-5 /usr/bin/gcc-7 /usr/bin/gcc-ar /usr/bin/gcc-ar-4.8 /usr/bin/gcc-ar-5 /usr/bin/gcc-ar-7 /usr/bin/gcc-nm /usr/bin/gcc-nm-4.8 /usr/bin/gcc-nm-5 /usr/bin/gcc-nm-7 /usr/bin/gcc-ranlib /usr/bin/gcc-ranlib-4.8 /usr/bin/gcc-ranlib-5 /usr/bin/gcc-ranlib-7 But, nobody has suggested GCC_SUFFIX. So, I guess CROSS_COMPILE was enough to choose a specific tool version. > I'll try to have some other comments by later today/tonight. > > > > LD = $(LLVM_DIR)ld.lld > > > AR = $(LLVM_DIR)llvm-ar > > > NM = $(LLVM_DIR)llvm-nm > > > OBJCOPY = $(LLVM_DIR)llvm-objcopy > > > OBJDUMP = $(LLVM_DIR)llvm-objdump > > > READELF = $(LLVM_DIR)llvm-readelf > > > OBJSIZE = $(LLVM_DIR)llvm-size > > > STRIP = $(LLVM_DIR)llvm-strip > > > else > > > CC = $(CROSS_COMPILE)gcc > > > LD = $(CROSS_COMPILE)ld > > > AR = $(CROSS_COMPILE)ar > > > NM = $(CROSS_COMPILE)nm > > > OBJCOPY = $(CROSS_COMPILE)objcopy > > > OBJDUMP = $(CROSS_COMPILE)objdump > > > READELF = $(CROSS_COMPILE)readelf > > > OBJSIZE = $(CROSS_COMPILE)size > > > STRIP = $(CROSS_COMPILE)strip > > > endif > > > > > > > > > > > > I attached two patches. > > > Comments appreciated. > > > > I'm not sure the second one that recommends changing cc/c++ is the way > > to go; I think it might harm hermeticity. > > Agreed. I do not modify my host system at all for this project, just > relying on PATH modification. In theory, we can still override HOSTCC > and HOSTCXX but that would defeat the purpose of that patch. > > Cheers, > Nathan > > -- > You received this message because you are subscribed to the Google Groups "Clang Built Linux" group. > To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20200330190312.GA32257%40ubuntu-m2-xlarge-x86. -- Best Regards Masahiro Yamada ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-03-31 6:24 ` Masahiro Yamada @ 2020-03-31 18:39 ` Nick Desaulniers 2020-03-31 19:35 ` Nathan Chancellor 2020-04-01 6:10 ` Masahiro Yamada 0 siblings, 2 replies; 15+ messages in thread From: Nick Desaulniers @ 2020-03-31 18:39 UTC (permalink / raw) To: Masahiro Yamada, Nathan Chancellor Cc: clang-built-linux, Linux Kbuild mailing list, Linux Kernel Mailing List, Sandeep Patil On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > On Tue, Mar 31, 2020 at 4:03 AM Nathan Chancellor > <natechancellor@gmail.com> wrote: > > > > On Mon, Mar 30, 2020 at 11:58:19AM -0700, Nick Desaulniers wrote: > > > On Sat, Mar 28, 2020 at 6:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > > > > > I also had planned to provide a single switch to change > > > > all the tool defaults to LLVM. > > > > > > > > So, supporting 'LLVM' is fine, but I'd rather want this > > > > look symmetrical, and easy to understand. > > > > > > > > CPP = $(CC) -E > > > > ifneq ($(LLVM),) > > > > > > Yes, a simple if statement is much simpler than the overly complex patch I had. > > > > > > > CC = $(LLVM_DIR)clang > > > > > > Do we need $LLVM_DIR? Shouldn't users just have that in their $PATH? > > > > > > Also, I think we need to support suffixed binaries, as debian > > > distributes these with version suffixes, as Nathan points out. Or do > > > the debian packages install suffixed binaries AND path versioned > > > non-suffixed binaries? > > > > I think the idea here is that ultimately, the suffixed versions of clang > > that Debian has in /usr/bin are symlinks to binaries in > > /usr/lib/llvm-#/bin; as a result, a user could say > > LLVM_DIR=/usr/lib/llvm-#/bin/ and all of those tools would be picked up > > automatically. I am not really sure what is better. $ sudo apt install clang-8 $ which clang-8 /usr/bin/clang-8 $ ls -l `!!` /usr/bin/clang-8 -> ../lib/llvm-8/bin/clang $ ls /usr/lib/llvm-8/bin <non suffixed versions> Ok, so Nathan, it looks like we don't need the version suffixes. Instead, we can be more explicit with our $PATH, and only add the above (and bintutils). I was thinking supporting the suffix was required for our CI, but it seems like maybe not. > I periodically build the latest llvm from the trunk, > and install it under my home directory. > So, I just thought it would be useful to > allow a user to specify the llvm directory. > Of course, I can do the equivalent by tweaking PATH, but > I hesitate to make the non-released version my default. Respectfully, I strongly disagree. This should be handled by modifications to $PATH, either by your shell's .rc file when you always want it, or exported for a session when you want it, or prefixed to an invocation for the duration of that command. We should not have a new variable just for the path of a few tools. Rather than `make LLVM_DIR=~/llvm-project LLVM=1`, you can do `PATH=$PATH:~/llvm-project make LLVM=1`. (or export it manually or via your shell .rc, depending on how comfortable you are with that version). > Having both LLVM_DIR and LLVM_SUFFIX seems verbose. I agree, so maybe just LLVM=y, and we can support both non-standard locations and debian suffixes via modifications to PATH. > > In fact, the debian provides multiple versions of GCC. > For example, my machine has > > masahiro@pug:~$ ls -1 /usr/bin/gcc-* > /usr/bin/gcc-4.8 > /usr/bin/gcc-5 > /usr/bin/gcc-7 > /usr/bin/gcc-ar > /usr/bin/gcc-ar-4.8 > /usr/bin/gcc-ar-5 > /usr/bin/gcc-ar-7 > /usr/bin/gcc-nm > /usr/bin/gcc-nm-4.8 > /usr/bin/gcc-nm-5 > /usr/bin/gcc-nm-7 > /usr/bin/gcc-ranlib > /usr/bin/gcc-ranlib-4.8 > /usr/bin/gcc-ranlib-5 > /usr/bin/gcc-ranlib-7 > > But, nobody has suggested GCC_SUFFIX. > > So, I guess CROSS_COMPILE was enough to > choose a specific tool version. Or no one was testing specific versions of gcc with more than one installed. I can ask the KernelCI folks next week if this is an issue they face or have faced. -- Thanks, ~Nick Desaulniers ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-03-31 18:39 ` Nick Desaulniers @ 2020-03-31 19:35 ` Nathan Chancellor 2020-03-31 19:40 ` Nick Desaulniers 2020-04-01 6:10 ` Masahiro Yamada 1 sibling, 1 reply; 15+ messages in thread From: Nathan Chancellor @ 2020-03-31 19:35 UTC (permalink / raw) To: Nick Desaulniers Cc: Masahiro Yamada, clang-built-linux, Linux Kbuild mailing list, Linux Kernel Mailing List, Sandeep Patil On Tue, Mar 31, 2020 at 11:39:27AM -0700, Nick Desaulniers wrote: > On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > On Tue, Mar 31, 2020 at 4:03 AM Nathan Chancellor > > <natechancellor@gmail.com> wrote: > > > > > > On Mon, Mar 30, 2020 at 11:58:19AM -0700, Nick Desaulniers wrote: > > > > On Sat, Mar 28, 2020 at 6:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > > > > > > > I also had planned to provide a single switch to change > > > > > all the tool defaults to LLVM. > > > > > > > > > > So, supporting 'LLVM' is fine, but I'd rather want this > > > > > look symmetrical, and easy to understand. > > > > > > > > > > CPP = $(CC) -E > > > > > ifneq ($(LLVM),) > > > > > > > > Yes, a simple if statement is much simpler than the overly complex patch I had. > > > > > > > > > CC = $(LLVM_DIR)clang > > > > > > > > Do we need $LLVM_DIR? Shouldn't users just have that in their $PATH? > > > > > > > > Also, I think we need to support suffixed binaries, as debian > > > > distributes these with version suffixes, as Nathan points out. Or do > > > > the debian packages install suffixed binaries AND path versioned > > > > non-suffixed binaries? > > > > > > I think the idea here is that ultimately, the suffixed versions of clang > > > that Debian has in /usr/bin are symlinks to binaries in > > > /usr/lib/llvm-#/bin; as a result, a user could say > > > LLVM_DIR=/usr/lib/llvm-#/bin/ and all of those tools would be picked up > > > automatically. I am not really sure what is better. > > $ sudo apt install clang-8 > $ which clang-8 > /usr/bin/clang-8 > $ ls -l `!!` > /usr/bin/clang-8 -> ../lib/llvm-8/bin/clang > $ ls /usr/lib/llvm-8/bin > <non suffixed versions> > > Ok, so Nathan, it looks like we don't need the version suffixes. > Instead, we can be more explicit with our $PATH, and only add the > above (and bintutils). I was thinking supporting the suffix was > required for our CI, but it seems like maybe not. Correct. This should probably be documented though, otherwise people might use LLVM=y and be surprised when the latest version of their tools are not being picked up. > > I periodically build the latest llvm from the trunk, > > and install it under my home directory. > > So, I just thought it would be useful to > > allow a user to specify the llvm directory. > > Of course, I can do the equivalent by tweaking PATH, but > > I hesitate to make the non-released version my default. > > Respectfully, I strongly disagree. This should be handled by > modifications to $PATH, either by your shell's .rc file when you > always want it, or exported for a session when you want it, or > prefixed to an invocation for the duration of that command. We should > not have a new variable just for the path of a few tools. > > Rather than `make LLVM_DIR=~/llvm-project LLVM=1`, you can do > `PATH=$PATH:~/llvm-project make LLVM=1`. (or export it manually or via > your shell .rc, depending on how comfortable you are with that > version). I always do PATH=...:${PATH} make CC=clang... > > Having both LLVM_DIR and LLVM_SUFFIX seems verbose. > > I agree, so maybe just LLVM=y, and we can support both non-standard > locations and debian suffixes via modifications to PATH. > > > > > In fact, the debian provides multiple versions of GCC. > > For example, my machine has > > > > masahiro@pug:~$ ls -1 /usr/bin/gcc-* > > /usr/bin/gcc-4.8 > > /usr/bin/gcc-5 > > /usr/bin/gcc-7 > > /usr/bin/gcc-ar > > /usr/bin/gcc-ar-4.8 > > /usr/bin/gcc-ar-5 > > /usr/bin/gcc-ar-7 > > /usr/bin/gcc-nm > > /usr/bin/gcc-nm-4.8 > > /usr/bin/gcc-nm-5 > > /usr/bin/gcc-nm-7 > > /usr/bin/gcc-ranlib > > /usr/bin/gcc-ranlib-4.8 > > /usr/bin/gcc-ranlib-5 > > /usr/bin/gcc-ranlib-7 > > > > But, nobody has suggested GCC_SUFFIX. > > > > So, I guess CROSS_COMPILE was enough to > > choose a specific tool version. > > Or no one was testing specific versions of gcc with more than one > installed. I can ask the KernelCI folks next week if this is an issue > they face or have faced. Well gcc is just one tool, so specified CC=gcc-5 is not that complicated; it would get a lot more gnarly if one had different versions of binutils as well. Cheers, Nathan ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-03-31 19:35 ` Nathan Chancellor @ 2020-03-31 19:40 ` Nick Desaulniers 0 siblings, 0 replies; 15+ messages in thread From: Nick Desaulniers @ 2020-03-31 19:40 UTC (permalink / raw) To: Nathan Chancellor Cc: Masahiro Yamada, clang-built-linux, Linux Kbuild mailing list, Linux Kernel Mailing List, Sandeep Patil On Tue, Mar 31, 2020 at 12:35 PM Nathan Chancellor <natechancellor@gmail.com> wrote: > > On Tue, Mar 31, 2020 at 11:39:27AM -0700, Nick Desaulniers wrote: > > On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > > > In fact, the debian provides multiple versions of GCC. > > > For example, my machine has > > > > > > masahiro@pug:~$ ls -1 /usr/bin/gcc-* > > > /usr/bin/gcc-4.8 > > > /usr/bin/gcc-5 > > > /usr/bin/gcc-7 > > > /usr/bin/gcc-ar > > > /usr/bin/gcc-ar-4.8 > > > /usr/bin/gcc-ar-5 > > > /usr/bin/gcc-ar-7 > > > /usr/bin/gcc-nm > > > /usr/bin/gcc-nm-4.8 > > > /usr/bin/gcc-nm-5 > > > /usr/bin/gcc-nm-7 > > > /usr/bin/gcc-ranlib > > > /usr/bin/gcc-ranlib-4.8 > > > /usr/bin/gcc-ranlib-5 > > > /usr/bin/gcc-ranlib-7 > > > > > > But, nobody has suggested GCC_SUFFIX. > > > > > > So, I guess CROSS_COMPILE was enough to > > > choose a specific tool version. > > > > Or no one was testing specific versions of gcc with more than one > > installed. I can ask the KernelCI folks next week if this is an issue > > they face or have faced. > > Well gcc is just one tool, so specified CC=gcc-5 is not that > complicated; it would get a lot more gnarly if one had different > versions of binutils as well. Have you had to test different releases of binutils yet? I have, and it was not fun. I don't even remember what I did but I recall it being painful trying to get it to work. (I think I finally solved it via temporary symlink). Speaking of, I should get back to those dwarf-5 patches I started, now that binutils devs implemented every missing feature I could find. -- Thanks, ~Nick Desaulniers ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-03-31 18:39 ` Nick Desaulniers 2020-03-31 19:35 ` Nathan Chancellor @ 2020-04-01 6:10 ` Masahiro Yamada 2020-04-01 17:48 ` Nick Desaulniers 1 sibling, 1 reply; 15+ messages in thread From: Masahiro Yamada @ 2020-04-01 6:10 UTC (permalink / raw) To: Nick Desaulniers Cc: Nathan Chancellor, clang-built-linux, Linux Kbuild mailing list, Linux Kernel Mailing List, Sandeep Patil On Wed, Apr 1, 2020 at 3:39 AM Nick Desaulniers <ndesaulniers@google.com> wrote: > > On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > On Tue, Mar 31, 2020 at 4:03 AM Nathan Chancellor > > <natechancellor@gmail.com> wrote: > > > > > > On Mon, Mar 30, 2020 at 11:58:19AM -0700, Nick Desaulniers wrote: > > > > On Sat, Mar 28, 2020 at 6:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > > > > > > > I also had planned to provide a single switch to change > > > > > all the tool defaults to LLVM. > > > > > > > > > > So, supporting 'LLVM' is fine, but I'd rather want this > > > > > look symmetrical, and easy to understand. > > > > > > > > > > CPP = $(CC) -E > > > > > ifneq ($(LLVM),) > > > > > > > > Yes, a simple if statement is much simpler than the overly complex patch I had. > > > > > > > > > CC = $(LLVM_DIR)clang > > > > > > > > Do we need $LLVM_DIR? Shouldn't users just have that in their $PATH? > > > > > > > > Also, I think we need to support suffixed binaries, as debian > > > > distributes these with version suffixes, as Nathan points out. Or do > > > > the debian packages install suffixed binaries AND path versioned > > > > non-suffixed binaries? > > > > > > I think the idea here is that ultimately, the suffixed versions of clang > > > that Debian has in /usr/bin are symlinks to binaries in > > > /usr/lib/llvm-#/bin; as a result, a user could say > > > LLVM_DIR=/usr/lib/llvm-#/bin/ and all of those tools would be picked up > > > automatically. I am not really sure what is better. > > $ sudo apt install clang-8 > $ which clang-8 > /usr/bin/clang-8 > $ ls -l `!!` > /usr/bin/clang-8 -> ../lib/llvm-8/bin/clang > $ ls /usr/lib/llvm-8/bin > <non suffixed versions> > > Ok, so Nathan, it looks like we don't need the version suffixes. > Instead, we can be more explicit with our $PATH, and only add the > above (and bintutils). I was thinking supporting the suffix was > required for our CI, but it seems like maybe not. > > > I periodically build the latest llvm from the trunk, > > and install it under my home directory. > > So, I just thought it would be useful to > > allow a user to specify the llvm directory. > > Of course, I can do the equivalent by tweaking PATH, but > > I hesitate to make the non-released version my default. > > Respectfully, I strongly disagree. This should be handled by > modifications to $PATH, either by your shell's .rc file when you > always want it, or exported for a session when you want it, or > prefixed to an invocation for the duration of that command. We should > not have a new variable just for the path of a few tools. > > Rather than `make LLVM_DIR=~/llvm-project LLVM=1`, you can do > `PATH=$PATH:~/llvm-project make LLVM=1`. (or export it manually or via > your shell .rc, depending on how comfortable you are with that > version). > > > Having both LLVM_DIR and LLVM_SUFFIX seems verbose. > > I agree, so maybe just LLVM=y, and we can support both non-standard > locations and debian suffixes via modifications to PATH. OK, so we will start with the boolean switch 'LLVM'. People can use PATH to cope with directory path and suffixes. -- Best Regards Masahiro Yamada ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-04-01 6:10 ` Masahiro Yamada @ 2020-04-01 17:48 ` Nick Desaulniers 2020-04-02 16:39 ` Fangrui Song 0 siblings, 1 reply; 15+ messages in thread From: Nick Desaulniers @ 2020-04-01 17:48 UTC (permalink / raw) To: Masahiro Yamada Cc: Nathan Chancellor, clang-built-linux, Linux Kbuild mailing list, Linux Kernel Mailing List, Sandeep Patil On Tue, Mar 31, 2020 at 11:11 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > On Wed, Apr 1, 2020 at 3:39 AM Nick Desaulniers <ndesaulniers@google.com> wrote: > > > > On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > > > > > > Having both LLVM_DIR and LLVM_SUFFIX seems verbose. > > > > I agree, so maybe just LLVM=y, and we can support both non-standard > > locations and debian suffixes via modifications to PATH. > > > > OK, so we will start with the boolean switch 'LLVM'. > > People can use PATH to cope with directory path and suffixes. Sounds good, we will modify our CI to use PATH modifications rather than suffixes. We can even do that before such a patch to Makefile exists. -- Thanks, ~Nick Desaulniers ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-04-01 17:48 ` Nick Desaulniers @ 2020-04-02 16:39 ` Fangrui Song 2020-04-02 16:54 ` Masahiro Yamada 0 siblings, 1 reply; 15+ messages in thread From: Fangrui Song @ 2020-04-02 16:39 UTC (permalink / raw) To: 'Nick Desaulniers' via Clang Built Linux Cc: Masahiro Yamada, Nathan Chancellor, Linux Kbuild mailing list, Linux Kernel Mailing List, Sandeep Patil On 2020-04-01, 'Nick Desaulniers' via Clang Built Linux wrote: >On Tue, Mar 31, 2020 at 11:11 PM Masahiro Yamada <masahiroy@kernel.org> wrote: >> >> On Wed, Apr 1, 2020 at 3:39 AM Nick Desaulniers <ndesaulniers@google.com> wrote: >> > >> > On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote: >> > > >> > > Having both LLVM_DIR and LLVM_SUFFIX seems verbose. >> > >> > I agree, so maybe just LLVM=y, and we can support both non-standard >> > locations and debian suffixes via modifications to PATH. >> >> >> >> OK, so we will start with the boolean switch 'LLVM'. >> >> People can use PATH to cope with directory path and suffixes. > >Sounds good, we will modify our CI to use PATH modifications rather >than suffixes. We can even do that before such a patch to Makefile >exists. The proposed LLVM=1 + PATH scheme looks good to me. There seems to be one issue. OBJSIZE=llvm-objsize added in commit fcf1b6a35c16ac500fa908a4022238e5d666eabf "Documentation/llvm: add documentation on building w/ Clang/LLVM" is wrong. The tool is named llvm-size. OBJSIZE is only used once: arch/s390/scripts/Makefile.chkbss 14: if ! $(OBJSIZE) --common $< | $(AWK) 'END { if ($$3) exit 1 }'; then \ ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] Makefile.llvm: simplify LLVM build 2020-04-02 16:39 ` Fangrui Song @ 2020-04-02 16:54 ` Masahiro Yamada 0 siblings, 0 replies; 15+ messages in thread From: Masahiro Yamada @ 2020-04-02 16:54 UTC (permalink / raw) To: Fangrui Song Cc: 'Nick Desaulniers' via Clang Built Linux, Nathan Chancellor, Linux Kbuild mailing list, Linux Kernel Mailing List, Sandeep Patil On Fri, Apr 3, 2020 at 1:40 AM Fangrui Song <maskray@google.com> wrote: > > > On 2020-04-01, 'Nick Desaulniers' via Clang Built Linux wrote: > >On Tue, Mar 31, 2020 at 11:11 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > >> > >> On Wed, Apr 1, 2020 at 3:39 AM Nick Desaulniers <ndesaulniers@google.com> wrote: > >> > > >> > On Mon, Mar 30, 2020 at 11:25 PM Masahiro Yamada <masahiroy@kernel.org> wrote: > >> > > > >> > > Having both LLVM_DIR and LLVM_SUFFIX seems verbose. > >> > > >> > I agree, so maybe just LLVM=y, and we can support both non-standard > >> > locations and debian suffixes via modifications to PATH. > >> > >> > >> > >> OK, so we will start with the boolean switch 'LLVM'. > >> > >> People can use PATH to cope with directory path and suffixes. > > > >Sounds good, we will modify our CI to use PATH modifications rather > >than suffixes. We can even do that before such a patch to Makefile > >exists. > > The proposed LLVM=1 + PATH scheme looks good to me. > > > There seems to be one issue. > OBJSIZE=llvm-objsize added in > commit fcf1b6a35c16ac500fa908a4022238e5d666eabf "Documentation/llvm: add documentation on building w/ Clang/LLVM" > is wrong. > > The tool is named llvm-size. OBJSIZE is only used once: > > arch/s390/scripts/Makefile.chkbss > 14: if ! $(OBJSIZE) --common $< | $(AWK) 'END { if ($$3) exit 1 }'; then \ Good catch. Could you send a patch, please? -- Best Regards Masahiro Yamada ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2020-04-02 16:54 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <20200317202404.GA20746@ubuntu-m2-xlarge-x86> 2020-03-17 21:55 ` [PATCH v2] Makefile.llvm: simplify LLVM build Nick Desaulniers 2020-03-27 19:51 ` Nick Desaulniers 2020-03-27 22:42 ` Nathan Chancellor 2020-03-28 4:54 ` Masahiro Yamada 2020-03-29 1:56 ` Masahiro Yamada 2020-03-30 18:58 ` Nick Desaulniers 2020-03-30 19:03 ` Nathan Chancellor 2020-03-31 6:24 ` Masahiro Yamada 2020-03-31 18:39 ` Nick Desaulniers 2020-03-31 19:35 ` Nathan Chancellor 2020-03-31 19:40 ` Nick Desaulniers 2020-04-01 6:10 ` Masahiro Yamada 2020-04-01 17:48 ` Nick Desaulniers 2020-04-02 16:39 ` Fangrui Song 2020-04-02 16:54 ` Masahiro Yamada
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).