* [PATCH] kbuild: Support clang-$ver builds @ 2021-10-28 13:43 Peter Zijlstra 2021-10-28 14:07 ` Masahiro Yamada 0 siblings, 1 reply; 13+ messages in thread From: Peter Zijlstra @ 2021-10-28 13:43 UTC (permalink / raw) To: nathan, ndesaulniers, masahiroy, michal.lkml Cc: x86, linux-kernel, linux-kbuild, llvm Hi, Debian (and derived) distros ship their compilers as -$ver suffixed binaries. For gcc it is sufficent to use: $ make CC=gcc-12 However, clang builds (esp. clang-lto) need a whole array of tools to be exactly right, leading to unweildy stuff like: $ make CC=clang-13 LD=ld.lld=14 AR=llvm-ar-13 NM=llvm-nm-13 OBJCOPY=llvm-objcopy-13 OBJDUMP=llvm-objdump-13 READELF=llvm-readelf-13 STRIP=llvm-strip-13 LLVM=1 which is, quite franktly, totally insane and unusable. Instead use the already mandatory LLVM variable to convey this, enabling one such as myself to use: $ make LLVM=-13 This also lets one quickly test different clang versions. Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> --- Makefile | 27 +++++++++++++++++---------- tools/scripts/Makefile.include | 23 +++++++++++++++-------- 2 files changed, 32 insertions(+), 18 deletions(-) diff --git a/Makefile b/Makefile index 30c7c81d0437..a38f38f7f190 100644 --- a/Makefile +++ b/Makefile @@ -423,9 +423,16 @@ HOST_LFS_CFLAGS := $(shell getconf LFS_CFLAGS 2>/dev/null) HOST_LFS_LDFLAGS := $(shell getconf LFS_LDFLAGS 2>/dev/null) HOST_LFS_LIBS := $(shell getconf LFS_LIBS 2>/dev/null) +# When LLVM=-ver use clang-ver binaries, useful for Debian and other +# multi-version setups +ifeq ($(shell test $(LLVM) -lt 0; echo $$?),0) +LLVM_SFX=$(LLVM) +export LLVM_SFX +endif + ifneq ($(LLVM),) -HOSTCC = clang -HOSTCXX = clang++ +HOSTCC = clang$(LLVM_SFX) +HOSTCXX = clang++$(LLVM_SFX) else HOSTCC = gcc HOSTCXX = g++ @@ -443,14 +450,14 @@ KBUILD_HOSTLDLIBS := $(HOST_LFS_LIBS) $(HOSTLDLIBS) # Make variables (CC, etc...) CPP = $(CC) -E ifneq ($(LLVM),) -CC = clang -LD = ld.lld -AR = llvm-ar -NM = llvm-nm -OBJCOPY = llvm-objcopy -OBJDUMP = llvm-objdump -READELF = llvm-readelf -STRIP = llvm-strip +CC = clang$(LLVM_SFX) +LD = ld.lld$(LLVM_SFX) +AR = llvm-ar$(LLVM_SFX) +NM = llvm-nm$(LLVM_SFX) +OBJCOPY = llvm-objcopy$(LLVM_SFX) +OBJDUMP = llvm-objdump$(LLVM_SFX) +READELF = llvm-readelf$(LLVM_SFX) +STRIP = llvm-strip$(LLVM_SFX) else CC = $(CROSS_COMPILE)gcc LD = $(CROSS_COMPILE)ld diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include index 071312f5eb92..a2b53cc91378 100644 --- a/tools/scripts/Makefile.include +++ b/tools/scripts/Makefile.include @@ -51,12 +51,19 @@ define allow-override $(eval $(1) = $(2))) endef +# When LLVM=-ver use clang-ver binaries, useful for Debian and other +# multi-version setups +ifeq ($(shell test $(LLVM) -lt 0; echo $$?),0) +LLVM_SFX=$(LLVM) +export LLVM_SFX +endif + ifneq ($(LLVM),) -$(call allow-override,CC,clang) -$(call allow-override,AR,llvm-ar) -$(call allow-override,LD,ld.lld) -$(call allow-override,CXX,clang++) -$(call allow-override,STRIP,llvm-strip) +$(call allow-override,CC,clang$(LLVM_SFX)) +$(call allow-override,AR,llvm-ar$(LLVM_SFX)) +$(call allow-override,LD,ld.lld$(LLVM_SFX)) +$(call allow-override,CXX,clang++$(LLVM_SFX)) +$(call allow-override,STRIP,llvm-strip$(LLVM_SFX)) else # Allow setting various cross-compile vars or setting CROSS_COMPILE as a prefix. $(call allow-override,CC,$(CROSS_COMPILE)gcc) @@ -69,9 +76,9 @@ endif CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?) ifneq ($(LLVM),) -HOSTAR ?= llvm-ar -HOSTCC ?= clang -HOSTLD ?= ld.lld +HOSTAR ?= llvm-ar$(LLVM_SFX) +HOSTCC ?= clang$(LLVM_SFX) +HOSTLD ?= ld.lld$(LLVM_SFX) else HOSTAR ?= ar HOSTCC ?= gcc ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH] kbuild: Support clang-$ver builds 2021-10-28 13:43 [PATCH] kbuild: Support clang-$ver builds Peter Zijlstra @ 2021-10-28 14:07 ` Masahiro Yamada 2021-10-28 14:15 ` Peter Zijlstra 0 siblings, 1 reply; 13+ messages in thread From: Masahiro Yamada @ 2021-10-28 14:07 UTC (permalink / raw) To: Peter Zijlstra Cc: Nathan Chancellor, Nick Desaulniers, Michal Marek, X86 ML, Linux Kernel Mailing List, Linux Kbuild mailing list, llvm On Thu, Oct 28, 2021 at 10:44 PM Peter Zijlstra <peterz@infradead.org> wrote: > > Hi, > > Debian (and derived) distros ship their compilers as -$ver suffixed > binaries. For gcc it is sufficent to use: > > $ make CC=gcc-12 > > However, clang builds (esp. clang-lto) need a whole array of tools to be > exactly right, leading to unweildy stuff like: > > $ make CC=clang-13 LD=ld.lld=14 AR=llvm-ar-13 NM=llvm-nm-13 OBJCOPY=llvm-objcopy-13 OBJDUMP=llvm-objdump-13 READELF=llvm-readelf-13 STRIP=llvm-strip-13 LLVM=1 > > which is, quite franktly, totally insane and unusable. Instead use the > already mandatory LLVM variable to convey this, enabling one such as > myself to use: > > $ make LLVM=-13 > > This also lets one quickly test different clang versions. Please read the commit log of a0d1c951ef08ed24f35129267e3595d86f57f5d3 > Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> > --- > Makefile | 27 +++++++++++++++++---------- > tools/scripts/Makefile.include | 23 +++++++++++++++-------- > 2 files changed, 32 insertions(+), 18 deletions(-) > > diff --git a/Makefile b/Makefile > index 30c7c81d0437..a38f38f7f190 100644 > --- a/Makefile > +++ b/Makefile > @@ -423,9 +423,16 @@ HOST_LFS_CFLAGS := $(shell getconf LFS_CFLAGS 2>/dev/null) > HOST_LFS_LDFLAGS := $(shell getconf LFS_LDFLAGS 2>/dev/null) > HOST_LFS_LIBS := $(shell getconf LFS_LIBS 2>/dev/null) > > +# When LLVM=-ver use clang-ver binaries, useful for Debian and other > +# multi-version setups > +ifeq ($(shell test $(LLVM) -lt 0; echo $$?),0) > +LLVM_SFX=$(LLVM) > +export LLVM_SFX > +endif > + > ifneq ($(LLVM),) > -HOSTCC = clang > -HOSTCXX = clang++ > +HOSTCC = clang$(LLVM_SFX) > +HOSTCXX = clang++$(LLVM_SFX) > else > HOSTCC = gcc > HOSTCXX = g++ > @@ -443,14 +450,14 @@ KBUILD_HOSTLDLIBS := $(HOST_LFS_LIBS) $(HOSTLDLIBS) > # Make variables (CC, etc...) > CPP = $(CC) -E > ifneq ($(LLVM),) > -CC = clang > -LD = ld.lld > -AR = llvm-ar > -NM = llvm-nm > -OBJCOPY = llvm-objcopy > -OBJDUMP = llvm-objdump > -READELF = llvm-readelf > -STRIP = llvm-strip > +CC = clang$(LLVM_SFX) > +LD = ld.lld$(LLVM_SFX) > +AR = llvm-ar$(LLVM_SFX) > +NM = llvm-nm$(LLVM_SFX) > +OBJCOPY = llvm-objcopy$(LLVM_SFX) > +OBJDUMP = llvm-objdump$(LLVM_SFX) > +READELF = llvm-readelf$(LLVM_SFX) > +STRIP = llvm-strip$(LLVM_SFX) > else > CC = $(CROSS_COMPILE)gcc > LD = $(CROSS_COMPILE)ld > diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include > index 071312f5eb92..a2b53cc91378 100644 > --- a/tools/scripts/Makefile.include > +++ b/tools/scripts/Makefile.include > @@ -51,12 +51,19 @@ define allow-override > $(eval $(1) = $(2))) > endef > > +# When LLVM=-ver use clang-ver binaries, useful for Debian and other > +# multi-version setups > +ifeq ($(shell test $(LLVM) -lt 0; echo $$?),0) > +LLVM_SFX=$(LLVM) > +export LLVM_SFX > +endif > + > ifneq ($(LLVM),) > -$(call allow-override,CC,clang) > -$(call allow-override,AR,llvm-ar) > -$(call allow-override,LD,ld.lld) > -$(call allow-override,CXX,clang++) > -$(call allow-override,STRIP,llvm-strip) > +$(call allow-override,CC,clang$(LLVM_SFX)) > +$(call allow-override,AR,llvm-ar$(LLVM_SFX)) > +$(call allow-override,LD,ld.lld$(LLVM_SFX)) > +$(call allow-override,CXX,clang++$(LLVM_SFX)) > +$(call allow-override,STRIP,llvm-strip$(LLVM_SFX)) > else > # Allow setting various cross-compile vars or setting CROSS_COMPILE as a prefix. > $(call allow-override,CC,$(CROSS_COMPILE)gcc) > @@ -69,9 +76,9 @@ endif > CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?) > > ifneq ($(LLVM),) > -HOSTAR ?= llvm-ar > -HOSTCC ?= clang > -HOSTLD ?= ld.lld > +HOSTAR ?= llvm-ar$(LLVM_SFX) > +HOSTCC ?= clang$(LLVM_SFX) > +HOSTLD ?= ld.lld$(LLVM_SFX) > else > HOSTAR ?= ar > HOSTCC ?= gcc -- Best Regards Masahiro Yamada ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] kbuild: Support clang-$ver builds 2021-10-28 14:07 ` Masahiro Yamada @ 2021-10-28 14:15 ` Peter Zijlstra 2021-10-28 14:22 ` Peter Zijlstra 0 siblings, 1 reply; 13+ messages in thread From: Peter Zijlstra @ 2021-10-28 14:15 UTC (permalink / raw) To: Masahiro Yamada Cc: Nathan Chancellor, Nick Desaulniers, Michal Marek, X86 ML, Linux Kernel Mailing List, Linux Kbuild mailing list, llvm On Thu, Oct 28, 2021 at 11:07:40PM +0900, Masahiro Yamada wrote: > On Thu, Oct 28, 2021 at 10:44 PM Peter Zijlstra <peterz@infradead.org> wrote: > > > > Hi, > > > > Debian (and derived) distros ship their compilers as -$ver suffixed > > binaries. For gcc it is sufficent to use: > > > > $ make CC=gcc-12 > > > > However, clang builds (esp. clang-lto) need a whole array of tools to be > > exactly right, leading to unweildy stuff like: > > > > $ make CC=clang-13 LD=ld.lld=14 AR=llvm-ar-13 NM=llvm-nm-13 OBJCOPY=llvm-objcopy-13 OBJDUMP=llvm-objdump-13 READELF=llvm-readelf-13 STRIP=llvm-strip-13 LLVM=1 > > > > which is, quite franktly, totally insane and unusable. Instead use the > > already mandatory LLVM variable to convey this, enabling one such as > > myself to use: > > > > $ make LLVM=-13 > > > > This also lets one quickly test different clang versions. > > > Please read the commit log of > a0d1c951ef08ed24f35129267e3595d86f57f5d3 That's yuck, I like LLVM=-13 or LLVM=-12 much better to select between compilers. Means I don't have to remember wth they live and or wreck PATH. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] kbuild: Support clang-$ver builds 2021-10-28 14:15 ` Peter Zijlstra @ 2021-10-28 14:22 ` Peter Zijlstra 2021-10-28 15:03 ` Peter Zijlstra 0 siblings, 1 reply; 13+ messages in thread From: Peter Zijlstra @ 2021-10-28 14:22 UTC (permalink / raw) To: Masahiro Yamada Cc: Nathan Chancellor, Nick Desaulniers, Michal Marek, X86 ML, Linux Kernel Mailing List, Linux Kbuild mailing list, llvm On Thu, Oct 28, 2021 at 04:15:02PM +0200, Peter Zijlstra wrote: > On Thu, Oct 28, 2021 at 11:07:40PM +0900, Masahiro Yamada wrote: > > On Thu, Oct 28, 2021 at 10:44 PM Peter Zijlstra <peterz@infradead.org> wrote: > > > > > > Hi, > > > > > > Debian (and derived) distros ship their compilers as -$ver suffixed > > > binaries. For gcc it is sufficent to use: > > > > > > $ make CC=gcc-12 > > > > > > However, clang builds (esp. clang-lto) need a whole array of tools to be > > > exactly right, leading to unweildy stuff like: > > > > > > $ make CC=clang-13 LD=ld.lld=14 AR=llvm-ar-13 NM=llvm-nm-13 OBJCOPY=llvm-objcopy-13 OBJDUMP=llvm-objdump-13 READELF=llvm-readelf-13 STRIP=llvm-strip-13 LLVM=1 > > > > > > which is, quite franktly, totally insane and unusable. Instead use the > > > already mandatory LLVM variable to convey this, enabling one such as > > > myself to use: > > > > > > $ make LLVM=-13 > > > > > > This also lets one quickly test different clang versions. > > > > > > Please read the commit log of > > a0d1c951ef08ed24f35129267e3595d86f57f5d3 > > That's yuck, I like LLVM=-13 or LLVM=-12 much better to select between > compilers. Means I don't have to remember wth they live and or wreck > PATH. Even better, why not do something like: if test "${CC:0:5}" == "clang" then LLVM=1 LLVM_SFX=${CC:5} fi Then we can simply use: make CC=clang-12 and have it all just work. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] kbuild: Support clang-$ver builds 2021-10-28 14:22 ` Peter Zijlstra @ 2021-10-28 15:03 ` Peter Zijlstra 2021-10-28 17:44 ` Nathan Chancellor 0 siblings, 1 reply; 13+ messages in thread From: Peter Zijlstra @ 2021-10-28 15:03 UTC (permalink / raw) To: Masahiro Yamada Cc: Nathan Chancellor, Nick Desaulniers, Michal Marek, X86 ML, Linux Kernel Mailing List, Linux Kbuild mailing list, llvm On Thu, Oct 28, 2021 at 04:22:22PM +0200, Peter Zijlstra wrote: > On Thu, Oct 28, 2021 at 04:15:02PM +0200, Peter Zijlstra wrote: > > On Thu, Oct 28, 2021 at 11:07:40PM +0900, Masahiro Yamada wrote: > > > On Thu, Oct 28, 2021 at 10:44 PM Peter Zijlstra <peterz@infradead.org> wrote: > > > > > > > > Hi, > > > > > > > > Debian (and derived) distros ship their compilers as -$ver suffixed > > > > binaries. For gcc it is sufficent to use: > > > > > > > > $ make CC=gcc-12 > > > > > > > > However, clang builds (esp. clang-lto) need a whole array of tools to be > > > > exactly right, leading to unweildy stuff like: > > > > > > > > $ make CC=clang-13 LD=ld.lld=14 AR=llvm-ar-13 NM=llvm-nm-13 OBJCOPY=llvm-objcopy-13 OBJDUMP=llvm-objdump-13 READELF=llvm-readelf-13 STRIP=llvm-strip-13 LLVM=1 > > > > > > > > which is, quite franktly, totally insane and unusable. Instead use the > > > > already mandatory LLVM variable to convey this, enabling one such as > > > > myself to use: > > > > > > > > $ make LLVM=-13 > > > > > > > > This also lets one quickly test different clang versions. > > > > > > > > > Please read the commit log of > > > a0d1c951ef08ed24f35129267e3595d86f57f5d3 > > > > That's yuck, I like LLVM=-13 or LLVM=-12 much better to select between > > compilers. Means I don't have to remember wth they live and or wreck > > PATH. > > Even better, why not do something like: > > if test "${CC:0:5}" == "clang" > then > LLVM=1 > LLVM_SFX=${CC:5} > fi > > Then we can simply use: make CC=clang-12 and have it all just work. > Like so.. --- Makefile | 26 ++++++++++++++++---------- tools/scripts/Makefile.include | 22 ++++++++++++++-------- 2 files changed, 30 insertions(+), 18 deletions(-) diff --git a/Makefile b/Makefile index 30c7c81d0437..1d0d4494e01d 100644 --- a/Makefile +++ b/Makefile @@ -423,9 +423,15 @@ HOST_LFS_CFLAGS := $(shell getconf LFS_CFLAGS 2>/dev/null) HOST_LFS_LDFLAGS := $(shell getconf LFS_LDFLAGS 2>/dev/null) HOST_LFS_LIBS := $(shell getconf LFS_LIBS 2>/dev/null) +CC_BASE = $(shell echo $(CC) | cut -b "1-5") +ifeq ($(shell test $(CC_BASE) = clang; echo $$?),0) +LLVM := 1 +LLVM_SFX := $(shell echo $(CC) | cut -b "6-") +endif + ifneq ($(LLVM),) -HOSTCC = clang -HOSTCXX = clang++ +HOSTCC = clang$(LLVM_SFX) +HOSTCXX = clang++$(LLVM_SFX) else HOSTCC = gcc HOSTCXX = g++ @@ -443,14 +449,14 @@ KBUILD_HOSTLDLIBS := $(HOST_LFS_LIBS) $(HOSTLDLIBS) # Make variables (CC, etc...) CPP = $(CC) -E ifneq ($(LLVM),) -CC = clang -LD = ld.lld -AR = llvm-ar -NM = llvm-nm -OBJCOPY = llvm-objcopy -OBJDUMP = llvm-objdump -READELF = llvm-readelf -STRIP = llvm-strip +CC = clang$(LLVM_SFX) +LD = ld.lld$(LLVM_SFX) +AR = llvm-ar$(LLVM_SFX) +NM = llvm-nm$(LLVM_SFX) +OBJCOPY = llvm-objcopy$(LLVM_SFX) +OBJDUMP = llvm-objdump$(LLVM_SFX) +READELF = llvm-readelf$(LLVM_SFX) +STRIP = llvm-strip$(LLVM_SFX) else CC = $(CROSS_COMPILE)gcc LD = $(CROSS_COMPILE)ld diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include index 071312f5eb92..94c41083a7c7 100644 --- a/tools/scripts/Makefile.include +++ b/tools/scripts/Makefile.include @@ -51,12 +51,18 @@ define allow-override $(eval $(1) = $(2))) endef +CC_BASE = $(shell echo $(CC) | cut -b "1-5") +ifeq ($(shell test $(CC_BASE) = clang; echo $$?),0) +LLVM := 1 +LLVM_SFX := $(shell echo $(CC) | cut -b "6-") +endif + ifneq ($(LLVM),) -$(call allow-override,CC,clang) -$(call allow-override,AR,llvm-ar) -$(call allow-override,LD,ld.lld) -$(call allow-override,CXX,clang++) -$(call allow-override,STRIP,llvm-strip) +$(call allow-override,CC,clang$(LLVM_SFX)) +$(call allow-override,AR,llvm-ar$(LLVM_SFX)) +$(call allow-override,LD,ld.lld$(LLVM_SFX)) +$(call allow-override,CXX,clang++$(LLVM_SFX)) +$(call allow-override,STRIP,llvm-strip$(LLVM_SFX)) else # Allow setting various cross-compile vars or setting CROSS_COMPILE as a prefix. $(call allow-override,CC,$(CROSS_COMPILE)gcc) @@ -69,9 +75,9 @@ endif CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?) ifneq ($(LLVM),) -HOSTAR ?= llvm-ar -HOSTCC ?= clang -HOSTLD ?= ld.lld +HOSTAR ?= llvm-ar$(LLVM_SFX) +HOSTCC ?= clang$(LLVM_SFX) +HOSTLD ?= ld.lld$(LLVM_SFX) else HOSTAR ?= ar HOSTCC ?= gcc ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH] kbuild: Support clang-$ver builds 2021-10-28 15:03 ` Peter Zijlstra @ 2021-10-28 17:44 ` Nathan Chancellor 2021-10-28 20:48 ` Peter Zijlstra 0 siblings, 1 reply; 13+ messages in thread From: Nathan Chancellor @ 2021-10-28 17:44 UTC (permalink / raw) To: Peter Zijlstra Cc: Masahiro Yamada, Nick Desaulniers, Michal Marek, X86 ML, Linux Kernel Mailing List, Linux Kbuild mailing list, llvm On Thu, Oct 28, 2021 at 05:03:55PM +0200, Peter Zijlstra wrote: > On Thu, Oct 28, 2021 at 04:22:22PM +0200, Peter Zijlstra wrote: > > On Thu, Oct 28, 2021 at 04:15:02PM +0200, Peter Zijlstra wrote: > > > On Thu, Oct 28, 2021 at 11:07:40PM +0900, Masahiro Yamada wrote: > > > > On Thu, Oct 28, 2021 at 10:44 PM Peter Zijlstra <peterz@infradead.org> wrote: > > > > > > > > > > Hi, > > > > > > > > > > Debian (and derived) distros ship their compilers as -$ver suffixed > > > > > binaries. For gcc it is sufficent to use: > > > > > > > > > > $ make CC=gcc-12 > > > > > > > > > > However, clang builds (esp. clang-lto) need a whole array of tools to be > > > > > exactly right, leading to unweildy stuff like: > > > > > > > > > > $ make CC=clang-13 LD=ld.lld=14 AR=llvm-ar-13 NM=llvm-nm-13 OBJCOPY=llvm-objcopy-13 OBJDUMP=llvm-objdump-13 READELF=llvm-readelf-13 STRIP=llvm-strip-13 LLVM=1 > > > > > > > > > > which is, quite franktly, totally insane and unusable. Instead use the > > > > > already mandatory LLVM variable to convey this, enabling one such as > > > > > myself to use: > > > > > > > > > > $ make LLVM=-13 > > > > > > > > > > This also lets one quickly test different clang versions. > > > > > > > > > > > > Please read the commit log of > > > > a0d1c951ef08ed24f35129267e3595d86f57f5d3 > > > > > > That's yuck, I like LLVM=-13 or LLVM=-12 much better to select between > > > compilers. Means I don't have to remember wth they live and or wreck > > > PATH. > > > > Even better, why not do something like: > > > > if test "${CC:0:5}" == "clang" > > then > > LLVM=1 > > LLVM_SFX=${CC:5} > > fi > > > > Then we can simply use: make CC=clang-12 and have it all just work. > > > > Like so.. I like the first iteration of this idea most, as CC=clang does not currently imply using all LLVM tools. There are some architectures that work with CC=clang but not LLVM=1 so I am not sure we want that meaning to change. I do think this issue of Debian's different clang versions needs to be resolved as more and more LLVM features are being developed and non-LLVM or ClangBuiltLinux developers want to test these features and make sure that they work properly. There are a few different options: 1. Take Peter's first patch so that LLVM=1 and LLVM=-# can be used. The documentation should be updated to convey this change. The argument against doing this might be that Debian/Ubuntu/derivatives of either are special for using versioned suffixes and we do not want to add this to the build system but I think that might be short sighted. Peter's patch is not super complex nor does it change the status quo of LLVM=1. 2. Update the documentation to describe using the $ PATH=/usr/lib/llvm-#/bin:$PATH make LLVM=1 ... trick. This has been the preferred method for using different versions of LLVM but it has never been documented anywhere. This would allow us to keep the current build infrastructure while giving people clear instructions for how to handle different versions of clang. As Peter has noted, this would require people who are not familiar with building with LLVM to be constantly looking at the documentation to remember the command to invoke, whereas with LLVM=-#, it is easy to remember. 3. Explore recommending update-alternatives in the documentation or working with Debian to provide this automatically. As this is a Debian and derivatives only problem, a solution for it could come from the distribution itself. I found one post from a few years ago that talks about this but a command to use all LLVM tools from a particular version would be rather unweildy: https://askubuntu.com/questions/791616/set-clang-3-8-as-default Overall, I think I like #1 the best. It is not that much more complicated than what we have now and it leaves it entirely up to the user how they want to build and it is hard for them to shoot themselves in the foot. I am open to other suggestions or arguments though :) Cheers, Nathan > --- > Makefile | 26 ++++++++++++++++---------- > tools/scripts/Makefile.include | 22 ++++++++++++++-------- > 2 files changed, 30 insertions(+), 18 deletions(-) > > diff --git a/Makefile b/Makefile > index 30c7c81d0437..1d0d4494e01d 100644 > --- a/Makefile > +++ b/Makefile > @@ -423,9 +423,15 @@ HOST_LFS_CFLAGS := $(shell getconf LFS_CFLAGS 2>/dev/null) > HOST_LFS_LDFLAGS := $(shell getconf LFS_LDFLAGS 2>/dev/null) > HOST_LFS_LIBS := $(shell getconf LFS_LIBS 2>/dev/null) > > +CC_BASE = $(shell echo $(CC) | cut -b "1-5") > +ifeq ($(shell test $(CC_BASE) = clang; echo $$?),0) > +LLVM := 1 > +LLVM_SFX := $(shell echo $(CC) | cut -b "6-") > +endif > + > ifneq ($(LLVM),) > -HOSTCC = clang > -HOSTCXX = clang++ > +HOSTCC = clang$(LLVM_SFX) > +HOSTCXX = clang++$(LLVM_SFX) > else > HOSTCC = gcc > HOSTCXX = g++ > @@ -443,14 +449,14 @@ KBUILD_HOSTLDLIBS := $(HOST_LFS_LIBS) $(HOSTLDLIBS) > # Make variables (CC, etc...) > CPP = $(CC) -E > ifneq ($(LLVM),) > -CC = clang > -LD = ld.lld > -AR = llvm-ar > -NM = llvm-nm > -OBJCOPY = llvm-objcopy > -OBJDUMP = llvm-objdump > -READELF = llvm-readelf > -STRIP = llvm-strip > +CC = clang$(LLVM_SFX) > +LD = ld.lld$(LLVM_SFX) > +AR = llvm-ar$(LLVM_SFX) > +NM = llvm-nm$(LLVM_SFX) > +OBJCOPY = llvm-objcopy$(LLVM_SFX) > +OBJDUMP = llvm-objdump$(LLVM_SFX) > +READELF = llvm-readelf$(LLVM_SFX) > +STRIP = llvm-strip$(LLVM_SFX) > else > CC = $(CROSS_COMPILE)gcc > LD = $(CROSS_COMPILE)ld > diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include > index 071312f5eb92..94c41083a7c7 100644 > --- a/tools/scripts/Makefile.include > +++ b/tools/scripts/Makefile.include > @@ -51,12 +51,18 @@ define allow-override > $(eval $(1) = $(2))) > endef > > +CC_BASE = $(shell echo $(CC) | cut -b "1-5") > +ifeq ($(shell test $(CC_BASE) = clang; echo $$?),0) > +LLVM := 1 > +LLVM_SFX := $(shell echo $(CC) | cut -b "6-") > +endif > + > ifneq ($(LLVM),) > -$(call allow-override,CC,clang) > -$(call allow-override,AR,llvm-ar) > -$(call allow-override,LD,ld.lld) > -$(call allow-override,CXX,clang++) > -$(call allow-override,STRIP,llvm-strip) > +$(call allow-override,CC,clang$(LLVM_SFX)) > +$(call allow-override,AR,llvm-ar$(LLVM_SFX)) > +$(call allow-override,LD,ld.lld$(LLVM_SFX)) > +$(call allow-override,CXX,clang++$(LLVM_SFX)) > +$(call allow-override,STRIP,llvm-strip$(LLVM_SFX)) > else > # Allow setting various cross-compile vars or setting CROSS_COMPILE as a prefix. > $(call allow-override,CC,$(CROSS_COMPILE)gcc) > @@ -69,9 +75,9 @@ endif > CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?) > > ifneq ($(LLVM),) > -HOSTAR ?= llvm-ar > -HOSTCC ?= clang > -HOSTLD ?= ld.lld > +HOSTAR ?= llvm-ar$(LLVM_SFX) > +HOSTCC ?= clang$(LLVM_SFX) > +HOSTLD ?= ld.lld$(LLVM_SFX) > else > HOSTAR ?= ar > HOSTCC ?= gcc ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] kbuild: Support clang-$ver builds 2021-10-28 17:44 ` Nathan Chancellor @ 2021-10-28 20:48 ` Peter Zijlstra 2021-10-28 23:27 ` Nick Desaulniers 0 siblings, 1 reply; 13+ messages in thread From: Peter Zijlstra @ 2021-10-28 20:48 UTC (permalink / raw) To: Nathan Chancellor Cc: Masahiro Yamada, Nick Desaulniers, Michal Marek, X86 ML, Linux Kernel Mailing List, Linux Kbuild mailing list, llvm On Thu, Oct 28, 2021 at 10:44:06AM -0700, Nathan Chancellor wrote: > I like the first iteration of this idea most, as CC=clang does not > currently imply using all LLVM tools. There are some architectures that > work with CC=clang but not LLVM=1 so I am not sure we want that meaning > to change. Could such architecture set a CONFIG symbol to indicate they're handicapped and have that make the Makefile not use the full LLVM suite? > I do think this issue of Debian's different clang versions needs to be > resolved as more and more LLVM features are being developed and non-LLVM > or ClangBuiltLinux developers want to test these features and make sure > that they work properly. There are a few different options: > > 1. Take Peter's first patch so that LLVM=1 and LLVM=-# can be used. The > documentation should be updated to convey this change. The argument > against doing this might be that Debian/Ubuntu/derivatives of either > are special for using versioned suffixes and we do not want to add > this to the build system but I think that might be short sighted. > Peter's patch is not super complex nor does it change the status quo > of LLVM=1. Also, I *greatly* appreciate having multiple versions of the compiler at hand. I really think debian does the right thing here and other distros are at a disadvantage. Note: also Segher's buildall script generates version suffixed GCC builds. > 2. Update the documentation to describe using the > > $ PATH=/usr/lib/llvm-#/bin:$PATH make LLVM=1 ... > > trick. This has been the preferred method for using different > versions of LLVM but it has never been documented anywhere. This > would allow us to keep the current build infrastructure while giving > people clear instructions for how to handle different versions of > clang. As Peter has noted, this would require people who are not > familiar with building with LLVM to be constantly looking at the > documentation to remember the command to invoke, whereas with > LLVM=-#, it is easy to remember. Right, this is no more than a crude hack and is super unfriendly. It should never have been accepted. > 3. Explore recommending update-alternatives in the documentation or > working with Debian to provide this automatically. As this is a > Debian and derivatives only problem, a solution for it could come > from the distribution itself. I found one post from a few years ago > that talks about this but a command to use all LLVM tools from a > particular version would be rather unweildy: > > https://askubuntu.com/questions/791616/set-clang-3-8-as-default I also don't really appreciate this, I don't want to change the system default 'clang' just to build a kernel. I might build the kernel with various different compilers consecutively to look at differences in code gen for example. Why should I not be able to build them concurrently? > Overall, I think I like #1 the best. It is not that much more > complicated than what we have now and it leaves it entirely up to the > user how they want to build and it is hard for them to shoot themselves > in the foot. I am open to other suggestions or arguments though :) So I've been doing kernel builds using: $ make O=foo CC=gcc-$ver for *years*, and I really don't understand or appreciate why building a clang kernel needs to be so damn difficult. The whole: LLVM=1 thing is just wrong. $ make O=foo CC=clang-$ver *should* just work. This not working is a pain bug; people peddling crazy things like 2) above is just mind-boggling insane. Having a document explaining how to do an LLVM build should be a clue it's wrong. So how about we do the below instead? Then the normal way of things will be: $ make CC=clang-12 and it will all *just*work*. The incomplete LLVM archs can revert back to the old hybrid (clang+binutils) way of things by doing: $ make CC=clang LLVM=0 Which explicitly states, use clang but don't use the full llvm suite. Or they can do that CONFIG_LLVM_BROKEN (or whatever) to have them default to LLVM=0 ifdef CONFIG_LLVM_BROKEN LLVM ?= 0 endif in front of the CC magic in the below patch should take care of that. (also, having all this gunk twice isn't useful) --- Makefile | 26 ++++++++++++++++---------- tools/scripts/Makefile.include | 22 ++++++++++++++-------- 2 files changed, 30 insertions(+), 18 deletions(-) --- a/Makefile +++ b/Makefile @@ -423,9 +423,15 @@ HOST_LFS_CFLAGS := $(shell getconf LFS_C HOST_LFS_LDFLAGS := $(shell getconf LFS_LDFLAGS 2>/dev/null) HOST_LFS_LIBS := $(shell getconf LFS_LIBS 2>/dev/null) +CC_BASE = $(shell echo $(CC) | cut -b "1-5") +ifeq ($(shell test "$(CC_BASE)" = "clang"; echo $$?),0) +LLVM ?= 1 +LLVM_SFX := $(shell echo $(CC) | cut -b "6-") +endif + ifneq ($(LLVM),) -HOSTCC = clang -HOSTCXX = clang++ +HOSTCC = clang$(LLVM_SFX) +HOSTCXX = clang++$(LLVM_SFX) else HOSTCC = gcc HOSTCXX = g++ @@ -443,14 +449,14 @@ KBUILD_HOSTLDLIBS := $(HOST_LFS_LIBS) # Make variables (CC, etc...) CPP = $(CC) -E ifneq ($(LLVM),) -CC = clang -LD = ld.lld -AR = llvm-ar -NM = llvm-nm -OBJCOPY = llvm-objcopy -OBJDUMP = llvm-objdump -READELF = llvm-readelf -STRIP = llvm-strip +CC = clang$(LLVM_SFX) +LD = ld.lld$(LLVM_SFX) +AR = llvm-ar$(LLVM_SFX) +NM = llvm-nm$(LLVM_SFX) +OBJCOPY = llvm-objcopy$(LLVM_SFX) +OBJDUMP = llvm-objdump$(LLVM_SFX) +READELF = llvm-readelf$(LLVM_SFX) +STRIP = llvm-strip$(LLVM_SFX) else CC = $(CROSS_COMPILE)gcc LD = $(CROSS_COMPILE)ld --- a/tools/scripts/Makefile.include +++ b/tools/scripts/Makefile.include @@ -51,12 +51,18 @@ define allow-override $(eval $(1) = $(2))) endef +CC_BASE = $(shell echo $(CC) | cut -b "1-5") +ifeq ($(shell test "$(CC_BASE)" = "clang"; echo $$?),0) +LLVM ?= 1 +LLVM_SFX := $(shell echo $(CC) | cut -b "6-") +endif + ifneq ($(LLVM),) -$(call allow-override,CC,clang) -$(call allow-override,AR,llvm-ar) -$(call allow-override,LD,ld.lld) -$(call allow-override,CXX,clang++) -$(call allow-override,STRIP,llvm-strip) +$(call allow-override,CC,clang$(LLVM_SFX)) +$(call allow-override,AR,llvm-ar$(LLVM_SFX)) +$(call allow-override,LD,ld.lld$(LLVM_SFX)) +$(call allow-override,CXX,clang++$(LLVM_SFX)) +$(call allow-override,STRIP,llvm-strip$(LLVM_SFX)) else # Allow setting various cross-compile vars or setting CROSS_COMPILE as a prefix. $(call allow-override,CC,$(CROSS_COMPILE)gcc) @@ -69,9 +75,9 @@ endif CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?) ifneq ($(LLVM),) -HOSTAR ?= llvm-ar -HOSTCC ?= clang -HOSTLD ?= ld.lld +HOSTAR ?= llvm-ar$(LLVM_SFX) +HOSTCC ?= clang$(LLVM_SFX) +HOSTLD ?= ld.lld$(LLVM_SFX) else HOSTAR ?= ar HOSTCC ?= gcc ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] kbuild: Support clang-$ver builds 2021-10-28 20:48 ` Peter Zijlstra @ 2021-10-28 23:27 ` Nick Desaulniers 2021-10-29 8:54 ` Peter Zijlstra 2021-10-29 8:56 ` David Laight 0 siblings, 2 replies; 13+ messages in thread From: Nick Desaulniers @ 2021-10-28 23:27 UTC (permalink / raw) To: Peter Zijlstra Cc: Nathan Chancellor, Masahiro Yamada, Michal Marek, X86 ML, Linux Kernel Mailing List, Linux Kbuild mailing list, llvm On Thu, Oct 28, 2021 at 1:49 PM Peter Zijlstra <peterz@infradead.org> wrote: > > On Thu, Oct 28, 2021 at 10:44:06AM -0700, Nathan Chancellor wrote: > > > I like the first iteration of this idea most, as CC=clang does not > > currently imply using all LLVM tools. There are some architectures that > > work with CC=clang but not LLVM=1 so I am not sure we want that meaning > > to change. > > Could such architecture set a CONFIG symbol to indicate they're > handicapped and have that make the Makefile not use the full LLVM suite? At this point, it's just powerpc and s390 where the assembler in llvm needs more work. Every other arch we can compile we generally can generally link. (modulo some long tail configs). > > 2. Update the documentation to describe using the > > > > $ PATH=/usr/lib/llvm-#/bin:$PATH make LLVM=1 ... > > > > trick. This has been the preferred method for using different > > versions of LLVM but it has never been documented anywhere. This > > would allow us to keep the current build infrastructure while giving > > people clear instructions for how to handle different versions of > > clang. As Peter has noted, this would require people who are not > > familiar with building with LLVM to be constantly looking at the > > documentation to remember the command to invoke, whereas with > > LLVM=-#, it is easy to remember. > > Right, this is no more than a crude hack and is super unfriendly. It > should never have been accepted. Lots of tools modify PATH in your `.bashrc` or `.zshrc` etc. I don't see how that's any different, other than one off commands in which you actually intend to use a specific version. > So how about we do the below instead? Then the normal way of things will > be: > > $ make CC=clang-12 > > and it will all *just*work*. The incomplete LLVM archs can revert back > to the old hybrid (clang+binutils) way of things by doing: > > $ make CC=clang LLVM=0 > > Which explicitly states, use clang but don't use the full llvm suite. Or > they can do that CONFIG_LLVM_BROKEN (or whatever) to have them default > to LLVM=0 > > ifdef CONFIG_LLVM_BROKEN > LLVM ?= 0 > endif > > in front of the CC magic in the below patch should take care of that. I don't think setting CC should affect LD and friends (if you were suggesting that CC=clang-12 replace LLVM=1). Like Nathan, I'm sympathetic and think that your first patch is probably closer to what I'd accept, but it needs to handle non-suffixed versions (looks like it should) and add this info to Documentations/kbuild/llvm.rst. -- Thanks, ~Nick Desaulniers ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] kbuild: Support clang-$ver builds 2021-10-28 23:27 ` Nick Desaulniers @ 2021-10-29 8:54 ` Peter Zijlstra 2021-10-30 14:33 ` Peter Zijlstra 2021-10-29 8:56 ` David Laight 1 sibling, 1 reply; 13+ messages in thread From: Peter Zijlstra @ 2021-10-29 8:54 UTC (permalink / raw) To: Nick Desaulniers Cc: Nathan Chancellor, Masahiro Yamada, Michal Marek, X86 ML, Linux Kernel Mailing List, Linux Kbuild mailing list, llvm On Thu, Oct 28, 2021 at 04:27:55PM -0700, Nick Desaulniers wrote: > On Thu, Oct 28, 2021 at 1:49 PM Peter Zijlstra <peterz@infradead.org> wrote: > > > 2. Update the documentation to describe using the > > > > > > $ PATH=/usr/lib/llvm-#/bin:$PATH make LLVM=1 ... > > > > > > trick. This has been the preferred method for using different > > > versions of LLVM but it has never been documented anywhere. This > > > would allow us to keep the current build infrastructure while giving > > > people clear instructions for how to handle different versions of > > > clang. As Peter has noted, this would require people who are not > > > familiar with building with LLVM to be constantly looking at the > > > documentation to remember the command to invoke, whereas with > > > LLVM=-#, it is easy to remember. > > > > Right, this is no more than a crude hack and is super unfriendly. It > > should never have been accepted. > > Lots of tools modify PATH in your `.bashrc` or `.zshrc` etc. I don't > see how that's any different, other than one off commands in which you > actually intend to use a specific version. Typically /etc/profile.d/ stuff adds to PATH for easy access to installed programs but it is very poor at handling conflicting versions of the same program (IIRC you get the first one). Why should I want to remember where clang-12 vs gcc-9 lives? I don't want to remember or even type those paths, Debian has already kindly given me these unambiguous names to use. > > So how about we do the below instead? Then the normal way of things will > > be: > > > > $ make CC=clang-12 > > > > and it will all *just*work*. The incomplete LLVM archs can revert back > > to the old hybrid (clang+binutils) way of things by doing: > > > > $ make CC=clang LLVM=0 > > > > Which explicitly states, use clang but don't use the full llvm suite. Or > > they can do that CONFIG_LLVM_BROKEN (or whatever) to have them default > > to LLVM=0 > > > > ifdef CONFIG_LLVM_BROKEN > > LLVM ?= 0 > > endif > > > > in front of the CC magic in the below patch should take care of that. > > I don't think setting CC should affect LD and friends (if you were > suggesting that CC=clang-12 replace LLVM=1). Why not? Why should clang not default to ld.lld? If you want weird mixes you can always override them. Again, why should CC=clang not default to the whole LLVM suite? Why should using LLVM as a whole, be made more difficult? It's almost as if you don't want people to use it. You're explicitly making clang-lto/cfi difficult to use; is that on purpose? > Like Nathan, I'm > sympathetic and think that your first patch is probably closer to what > I'd accept, but it needs to handle non-suffixed versions (looks like > it should) and add this info to Documentations/kbuild/llvm.rst. That first patch has some issues there, but should be fixable. The latest CC= using one works there. As does the below. --- a/Makefile +++ b/Makefile @@ -423,9 +423,27 @@ HOST_LFS_CFLAGS := $(shell getconf LFS_C HOST_LFS_LDFLAGS := $(shell getconf LFS_LDFLAGS 2>/dev/null) HOST_LFS_LIBS := $(shell getconf LFS_LIBS 2>/dev/null) -ifneq ($(LLVM),) -HOSTCC = clang -HOSTCXX = clang++ +# powerpc and s390 don't yet work with LLVM as a whole +ifeq ($(ARCH),powerpc) +LLVM = 0 +endif +ifeq ($(ARCH),s390) +LLVM = 0 +endif + +# otherwise, if CC=clang, default to using LLVM to enable LTO +CC_BASE := $(shell echo $(CC) | cut -b "1-5") +ifeq ($(shell test "$(CC_BASE)" = "clang"; echo $$?),0) +LLVM ?= 1 +LLVM_SFX := $(shell echo $(CC) | cut -b "6-") +endif + +# if not set by now, do not use LLVM +LLVM ?= 0 + +ifneq ($(LLVM),0) +HOSTCC = clang$(LLVM_SFX) +HOSTCXX = clang++$(LLVM_SFX) else HOSTCC = gcc HOSTCXX = g++ @@ -442,15 +460,15 @@ KBUILD_HOSTLDLIBS := $(HOST_LFS_LIBS) # Make variables (CC, etc...) CPP = $(CC) -E -ifneq ($(LLVM),) -CC = clang -LD = ld.lld -AR = llvm-ar -NM = llvm-nm -OBJCOPY = llvm-objcopy -OBJDUMP = llvm-objdump -READELF = llvm-readelf -STRIP = llvm-strip +ifneq ($(LLVM),0) +CC = clang$(LLVM_SFX) +LD = ld.lld$(LLVM_SFX) +AR = llvm-ar$(LLVM_SFX) +NM = llvm-nm$(LLVM_SFX) +OBJCOPY = llvm-objcopy$(LLVM_SFX) +OBJDUMP = llvm-objdump$(LLVM_SFX) +READELF = llvm-readelf$(LLVM_SFX) +STRIP = llvm-strip$(LLVM_SFX) else CC = $(CROSS_COMPILE)gcc LD = $(CROSS_COMPILE)ld ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] kbuild: Support clang-$ver builds 2021-10-29 8:54 ` Peter Zijlstra @ 2021-10-30 14:33 ` Peter Zijlstra 2021-10-31 1:18 ` Sedat Dilek 0 siblings, 1 reply; 13+ messages in thread From: Peter Zijlstra @ 2021-10-30 14:33 UTC (permalink / raw) To: Nick Desaulniers Cc: Nathan Chancellor, Masahiro Yamada, Michal Marek, X86 ML, Linux Kernel Mailing List, Linux Kbuild mailing list, llvm Latestest greatness... :-) --- Subject: kbuild: Fix clang/llvm build infra Debian (and derived) distros ship their compilers as -$ver suffixed binaries. For gcc it is sufficent to use: $ make CC=gcc-12 However, clang builds (esp. clang-lto) need a whole array of tools to be exactly right, leading to unweildy stuff like: $ make CC=clang-13 LD=ld.lld=14 AR=llvm-ar-13 NM=llvm-nm-13 OBJCOPY=llvm-objcopy-13 OBJDUMP=llvm-objdump-13 READELF=llvm-readelf-13 STRIP=llvm-strip-13 LLVM=1 which is, quite franktly, totally insane and unusable. Instead make the CC variable DTRT, enabling one such as myself to use: $ make CC=clang-13 This also lets one quickly test different clang versions. Additionally, also support path based LLVM suites like: $ make CC=/opt/llvm/bin/clang Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> --- Makefile | 45 +++++++++++++++++++++++++++--------- tools/scripts/Makefile.include | 50 ++++++++++++++++++++++++++++------------- 2 files changed, 68 insertions(+), 27 deletions(-) --- a/Makefile +++ b/Makefile @@ -423,9 +423,29 @@ HOST_LFS_CFLAGS := $(shell getconf LFS_C HOST_LFS_LDFLAGS := $(shell getconf LFS_LDFLAGS 2>/dev/null) HOST_LFS_LIBS := $(shell getconf LFS_LIBS 2>/dev/null) -ifneq ($(LLVM),) -HOSTCC = clang -HOSTCXX = clang++ +# powerpc and s390 don't yet work with LLVM as a whole +ifeq ($(ARCH),powerpc) +LLVM = 0 +endif +ifeq ($(ARCH),s390) +LLVM = 0 +endif + +# otherwise, if CC=clang, default to using LLVM to enable LTO +CC_BASE := $(shell echo $(CC) | sed 's/.*\///') +CC_NAME := $(shell echo $(CC_BASE) | cut -b "1-5") +ifeq ($(shell test "$(CC_NAME)" = "clang"; echo $$?),0) +LLVM ?= 1 +LLVM_PFX := $(shell echo $(CC) | sed 's/\(.*\/\)\?.*/\1/') +LLVM_SFX := $(shell echo $(CC_BASE) | cut -b "6-") +endif + +# if not set by now, do not use LLVM +LLVM ?= 0 + +ifneq ($(LLVM),0) +HOSTCC = $(LLVM_PFX)clang$(LLVM_SFX) +HOSTCXX = $(LLVM_PFX)clang++$(LLVM_SFX) else HOSTCC = gcc HOSTCXX = g++ @@ -442,15 +462,15 @@ KBUILD_HOSTLDLIBS := $(HOST_LFS_LIBS) # Make variables (CC, etc...) CPP = $(CC) -E -ifneq ($(LLVM),) -CC = clang -LD = ld.lld -AR = llvm-ar -NM = llvm-nm -OBJCOPY = llvm-objcopy -OBJDUMP = llvm-objdump -READELF = llvm-readelf -STRIP = llvm-strip +ifneq ($(LLVM),0) +CC = $(LLVM_PFX)clang$(LLVM_SFX) +LD = $(LLVM_PFX)ld.lld$(LLVM_SFX) +AR = $(LLVM_PFX)llvm-ar$(LLVM_SFX) +NM = $(LLVM_PFX)llvm-nm$(LLVM_SFX) +OBJCOPY = $(LLVM_PFX)llvm-objcopy$(LLVM_SFX) +OBJDUMP = $(LLVM_PFX)llvm-objdump$(LLVM_SFX) +READELF = $(LLVM_PFX)llvm-readelf$(LLVM_SFX) +STRIP = $(LLVM_PFX)llvm-strip$(LLVM_SFX) else CC = $(CROSS_COMPILE)gcc LD = $(CROSS_COMPILE)ld @@ -461,6 +481,7 @@ OBJDUMP = $(CROSS_COMPILE)objdump READELF = $(CROSS_COMPILE)readelf STRIP = $(CROSS_COMPILE)strip endif + PAHOLE = pahole RESOLVE_BTFIDS = $(objtree)/tools/bpf/resolve_btfids/resolve_btfids LEX = flex --- a/tools/scripts/Makefile.include +++ b/tools/scripts/Makefile.include @@ -51,12 +51,32 @@ define allow-override $(eval $(1) = $(2))) endef -ifneq ($(LLVM),) -$(call allow-override,CC,clang) -$(call allow-override,AR,llvm-ar) -$(call allow-override,LD,ld.lld) -$(call allow-override,CXX,clang++) -$(call allow-override,STRIP,llvm-strip) +# powerpc and s390 don't yet work with LLVM as a whole +ifeq ($(ARCH),powerpc) +LLVM = 0 +endif +ifeq ($(ARCH),s390) +LLVM = 0 +endif + +# otherwise, if CC=clang, default to using LLVM to enable LTO +CC_BASE := $(shell echo $(CC) | sed 's/.*\///') +CC_NAME := $(shell echo $(CC_BASE) | cut -b "1-5") +ifeq ($(shell test "$(CC_NAME)" = "clang"; echo $$?),0) +LLVM ?= 1 +LLVM_PFX := $(shell echo $(CC) | sed 's/\(.*\/\)\?.*/\1/') +LLVM_SFX := $(shell echo $(CC_BASE) | cut -b "6-") +endif + +# if not set by now, do not use LLVM +LLVM ?= 0 + +ifneq ($(LLVM),0) +$(call allow-override,CC,$(LLVM_PFX)clang$(LLVM_SFX)) +$(call allow-override,AR,$(LLVM_PFX)llvm-ar$(LLVM_SFX)) +$(call allow-override,LD,$(LLVM_PFX)ld.lld$(LLVM_SFX)) +$(call allow-override,CXX,$(LLVM_PFX)clang++$(LLVM_SFX)) +$(call allow-override,STRIP,$(LLVM_PFX)llvm-strip$(LLVM_SFX)) else # Allow setting various cross-compile vars or setting CROSS_COMPILE as a prefix. $(call allow-override,CC,$(CROSS_COMPILE)gcc) @@ -68,10 +88,10 @@ endif CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?) -ifneq ($(LLVM),) -HOSTAR ?= llvm-ar -HOSTCC ?= clang -HOSTLD ?= ld.lld +ifneq ($(LLVM),0) +HOSTAR ?= $(LLVM_PFX)llvm-ar$(LLVM_SFX) +HOSTCC ?= $(LLVM_PFX)clang$(LLVM_SFX) +HOSTLD ?= $(LLVM_PFX)ld.lld$(LLVM_SFX) else HOSTAR ?= ar HOSTCC ?= gcc @@ -79,11 +99,11 @@ HOSTLD ?= ld endif # Some tools require Clang, LLC and/or LLVM utils -CLANG ?= clang -LLC ?= llc -LLVM_CONFIG ?= llvm-config -LLVM_OBJCOPY ?= llvm-objcopy -LLVM_STRIP ?= llvm-strip +CLANG ?= $(LLVM_PFX)clang$(LLVM_SFX) +LLC ?= $(LLVM_PFX)llc$(LLVM_SFX) +LLVM_CONFIG ?= $(LLVM_PFX)llvm-config$(LLVM_SFX) +LLVM_OBJCOPY ?= $(LLVM_PFX)llvm-objcopy$(LLVM_SFX) +LLVM_STRIP ?= $(LLVM_PFX)llvm-strip$(LLVM_SFX) ifeq ($(CC_NO_CLANG), 1) EXTRA_WARNINGS += -Wstrict-aliasing=3 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] kbuild: Support clang-$ver builds 2021-10-30 14:33 ` Peter Zijlstra @ 2021-10-31 1:18 ` Sedat Dilek 2021-10-31 12:38 ` Peter Zijlstra 0 siblings, 1 reply; 13+ messages in thread From: Sedat Dilek @ 2021-10-31 1:18 UTC (permalink / raw) To: Peter Zijlstra Cc: Nick Desaulniers, Nathan Chancellor, Masahiro Yamada, Michal Marek, X86 ML, Linux Kernel Mailing List, Linux Kbuild mailing list, llvm [-- Attachment #1: Type: text/plain, Size: 9259 bytes --] On Sat, Oct 30, 2021 at 4:34 PM Peter Zijlstra <peterz@infradead.org> wrote: > > > Latestest greatness... :-) > > --- > Subject: kbuild: Fix clang/llvm build infra > > Debian (and derived) distros ship their compilers as -$ver suffixed > binaries. For gcc it is sufficent to use: > > $ make CC=gcc-12 > > However, clang builds (esp. clang-lto) need a whole array of tools to be > exactly right, leading to unweildy stuff like: > > $ make CC=clang-13 LD=ld.lld=14 AR=llvm-ar-13 NM=llvm-nm-13 OBJCOPY=llvm-objcopy-13 OBJDUMP=llvm-objdump-13 READELF=llvm-readelf-13 STRIP=llvm-strip-13 LLVM=1 > > which is, quite franktly, totally insane and unusable. Instead make > the CC variable DTRT, enabling one such as myself to use: > > $ make CC=clang-13 > > This also lets one quickly test different clang versions. > Additionally, also support path based LLVM suites like: > > $ make CC=/opt/llvm/bin/clang > Hi Peter, Thanks for bringing this up again. Your issue is not new for Debianist and Linux-LLVM/Clang folks. But let me comment. I agree, it is preferable or should be treated the same way in using clang-$ver like gcc-$ver. Background: Debian uses as meta-package llvm-toolchain-$ver and splits stuff into different Debian packages like the Clang compiler. LLD linker and LLVM (bin)utils. These packages place their files in separate directories to distinguish llvm-toolchain-$ver. Binaries like clang-$ver, ld.lld-$ver, llvm-strip-$ver, etc. you find in the /usr/bin/ directory which are symlinks. Clang compiler is shipped with clang-$ver Debian package. LLD linker is shipped as a separate Debian package called lld-$ver. LLVM (bin)utils like llvm-strip-$ver are shipped with the llvm-$ver Debian package. Link: https://www.kernel.org/doc/html/latest/kbuild/llvm.html#llvm-utilities EXAMPLES (here: llvm-toolchain-13): $ dpkg -S $(which clang-13) clang-13: /usr/bin/clang-13 $ dpkg -S /usr/bin/ld.lld-13 lld-13: /usr/bin/ld.lld-13 $ dpkg -S /usr/bin/llvm-strip-13 llvm-13: /usr/bin/llvm-strip-13 $ LC_ALL=C ll /usr/bin/clang-13 lrwxrwxrwx 1 root root 24 Oct 23 10:33 /usr/bin/clang-13 -> ../lib/llvm-13/bin/clang $ LC_ALL=C ll /usr/bin/ld.lld-13 lrwxrwxrwx 1 root root 25 Oct 23 10:33 /usr/bin/ld.lld-13 -> ../lib/llvm-13/bin/ld.lld $ LC_ALL=C ll /usr/bin/llvm-strip-13 lrwxrwxrwx 1 root root 29 Oct 23 10:33 /usr/bin/llvm-strip-13 -> ../lib/llvm-13/bin/llvm-strip As a compromise I use in my build_linux-llvm-toolchain.sh script $LLVM_TOOLCHAIN_PATH: ... ### GCC and LLVM version settings ##GCC_MAJOR_VER="10" LLVM_MAJOR_VER="13" ### LLVM toolchain path options # Explicitly set PATH to simplify `make LLVM=1` handling (see "LLVM tools options") # Option-1: Use system's LLVM toolchain # Option-2: Use selfmade LLVM toolchain (here: ThinLTO+PGO optimized via tc-build) # Option-3: Set custom PATH in ~/.bashrc ##LLVM_TOOLCHAIN_PATH="/usr/lib/llvm-${LLVM_MAJOR_VER}/bin" LLVM_TOOLCHAIN_PATH="/opt/llvm-toolchain/bin" if [ -d ${LLVM_TOOLCHAIN_PATH} ]; then export PATH="${LLVM_TOOLCHAIN_PATH}:${PATH}" fi ... But... ...for several new features like Clang-LTO we need the LLVM full "eco system". LLVM eco system means compiler, linker and binutils. In several talks at LPC 2021 talkers used the term "LLVM/binutils" like for GNU/binutils. The LLVM/Clang kernel-docs talk about "LLVM util(itie)s" which is set by LLVM=1 or not (LLVM=0). Link: https://www.kernel.org/doc/html/latest/kbuild/llvm.html#llvm-utilities AFAICS your patch has a primary focus to fix the LLVM/Clang infrastructure when CONFIG_LTO_CLANG=y, right? Can you massage the commit message to reflect this? IMHO your patch can be improved to check for CONFIG_LTO_CLANG=y where LLVM=1 is mandatory and IIRC LLVM_IAS=1, too. ( The default to use LLVM/Clang-IAS (Integrated ASsembler) was changed recently for some combinations arches + Clang-LTO? ) Before I forgot: Can you add a comment for the PPC and S390 situation in the commit message. Please remember people also want to use combinations like gcc-$ver and LLVM/binutils like ld.lld-$ver. GCC-10 + LLD-$VER was the fastest combination to build a Linux Debian package here on my Intel SandyBridge CPU system. ( There exists overrides like LD=... STRIP=... and I use XXX_FOR_BUILD=... (see my attached build-script). ) Thanks. Regards, - Sedat - > Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> > --- > Makefile | 45 +++++++++++++++++++++++++++--------- > tools/scripts/Makefile.include | 50 ++++++++++++++++++++++++++++------------- > 2 files changed, 68 insertions(+), 27 deletions(-) > > --- a/Makefile > +++ b/Makefile > @@ -423,9 +423,29 @@ HOST_LFS_CFLAGS := $(shell getconf LFS_C > HOST_LFS_LDFLAGS := $(shell getconf LFS_LDFLAGS 2>/dev/null) > HOST_LFS_LIBS := $(shell getconf LFS_LIBS 2>/dev/null) > > -ifneq ($(LLVM),) > -HOSTCC = clang > -HOSTCXX = clang++ > +# powerpc and s390 don't yet work with LLVM as a whole > +ifeq ($(ARCH),powerpc) > +LLVM = 0 > +endif > +ifeq ($(ARCH),s390) > +LLVM = 0 > +endif > + > +# otherwise, if CC=clang, default to using LLVM to enable LTO > +CC_BASE := $(shell echo $(CC) | sed 's/.*\///') > +CC_NAME := $(shell echo $(CC_BASE) | cut -b "1-5") > +ifeq ($(shell test "$(CC_NAME)" = "clang"; echo $$?),0) > +LLVM ?= 1 > +LLVM_PFX := $(shell echo $(CC) | sed 's/\(.*\/\)\?.*/\1/') > +LLVM_SFX := $(shell echo $(CC_BASE) | cut -b "6-") > +endif > + > +# if not set by now, do not use LLVM > +LLVM ?= 0 > + > +ifneq ($(LLVM),0) > +HOSTCC = $(LLVM_PFX)clang$(LLVM_SFX) > +HOSTCXX = $(LLVM_PFX)clang++$(LLVM_SFX) > else > HOSTCC = gcc > HOSTCXX = g++ > @@ -442,15 +462,15 @@ KBUILD_HOSTLDLIBS := $(HOST_LFS_LIBS) > > # Make variables (CC, etc...) > CPP = $(CC) -E > -ifneq ($(LLVM),) > -CC = clang > -LD = ld.lld > -AR = llvm-ar > -NM = llvm-nm > -OBJCOPY = llvm-objcopy > -OBJDUMP = llvm-objdump > -READELF = llvm-readelf > -STRIP = llvm-strip > +ifneq ($(LLVM),0) > +CC = $(LLVM_PFX)clang$(LLVM_SFX) > +LD = $(LLVM_PFX)ld.lld$(LLVM_SFX) > +AR = $(LLVM_PFX)llvm-ar$(LLVM_SFX) > +NM = $(LLVM_PFX)llvm-nm$(LLVM_SFX) > +OBJCOPY = $(LLVM_PFX)llvm-objcopy$(LLVM_SFX) > +OBJDUMP = $(LLVM_PFX)llvm-objdump$(LLVM_SFX) > +READELF = $(LLVM_PFX)llvm-readelf$(LLVM_SFX) > +STRIP = $(LLVM_PFX)llvm-strip$(LLVM_SFX) > else > CC = $(CROSS_COMPILE)gcc > LD = $(CROSS_COMPILE)ld > @@ -461,6 +481,7 @@ OBJDUMP = $(CROSS_COMPILE)objdump > READELF = $(CROSS_COMPILE)readelf > STRIP = $(CROSS_COMPILE)strip > endif > + > PAHOLE = pahole > RESOLVE_BTFIDS = $(objtree)/tools/bpf/resolve_btfids/resolve_btfids > LEX = flex > --- a/tools/scripts/Makefile.include > +++ b/tools/scripts/Makefile.include > @@ -51,12 +51,32 @@ define allow-override > $(eval $(1) = $(2))) > endef > > -ifneq ($(LLVM),) > -$(call allow-override,CC,clang) > -$(call allow-override,AR,llvm-ar) > -$(call allow-override,LD,ld.lld) > -$(call allow-override,CXX,clang++) > -$(call allow-override,STRIP,llvm-strip) > +# powerpc and s390 don't yet work with LLVM as a whole > +ifeq ($(ARCH),powerpc) > +LLVM = 0 > +endif > +ifeq ($(ARCH),s390) > +LLVM = 0 > +endif > + > +# otherwise, if CC=clang, default to using LLVM to enable LTO > +CC_BASE := $(shell echo $(CC) | sed 's/.*\///') > +CC_NAME := $(shell echo $(CC_BASE) | cut -b "1-5") > +ifeq ($(shell test "$(CC_NAME)" = "clang"; echo $$?),0) > +LLVM ?= 1 > +LLVM_PFX := $(shell echo $(CC) | sed 's/\(.*\/\)\?.*/\1/') > +LLVM_SFX := $(shell echo $(CC_BASE) | cut -b "6-") > +endif > + > +# if not set by now, do not use LLVM > +LLVM ?= 0 > + > +ifneq ($(LLVM),0) > +$(call allow-override,CC,$(LLVM_PFX)clang$(LLVM_SFX)) > +$(call allow-override,AR,$(LLVM_PFX)llvm-ar$(LLVM_SFX)) > +$(call allow-override,LD,$(LLVM_PFX)ld.lld$(LLVM_SFX)) > +$(call allow-override,CXX,$(LLVM_PFX)clang++$(LLVM_SFX)) > +$(call allow-override,STRIP,$(LLVM_PFX)llvm-strip$(LLVM_SFX)) > else > # Allow setting various cross-compile vars or setting CROSS_COMPILE as a prefix. > $(call allow-override,CC,$(CROSS_COMPILE)gcc) > @@ -68,10 +88,10 @@ endif > > CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?) > > -ifneq ($(LLVM),) > -HOSTAR ?= llvm-ar > -HOSTCC ?= clang > -HOSTLD ?= ld.lld > +ifneq ($(LLVM),0) > +HOSTAR ?= $(LLVM_PFX)llvm-ar$(LLVM_SFX) > +HOSTCC ?= $(LLVM_PFX)clang$(LLVM_SFX) > +HOSTLD ?= $(LLVM_PFX)ld.lld$(LLVM_SFX) > else > HOSTAR ?= ar > HOSTCC ?= gcc > @@ -79,11 +99,11 @@ HOSTLD ?= ld > endif > > # Some tools require Clang, LLC and/or LLVM utils > -CLANG ?= clang > -LLC ?= llc > -LLVM_CONFIG ?= llvm-config > -LLVM_OBJCOPY ?= llvm-objcopy > -LLVM_STRIP ?= llvm-strip > +CLANG ?= $(LLVM_PFX)clang$(LLVM_SFX) > +LLC ?= $(LLVM_PFX)llc$(LLVM_SFX) > +LLVM_CONFIG ?= $(LLVM_PFX)llvm-config$(LLVM_SFX) > +LLVM_OBJCOPY ?= $(LLVM_PFX)llvm-objcopy$(LLVM_SFX) > +LLVM_STRIP ?= $(LLVM_PFX)llvm-strip$(LLVM_SFX) > > ifeq ($(CC_NO_CLANG), 1) > EXTRA_WARNINGS += -Wstrict-aliasing=3 [-- Attachment #2: build_linux-llvm-toolchain.sh --] [-- Type: application/x-shellscript, Size: 14509 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] kbuild: Support clang-$ver builds 2021-10-31 1:18 ` Sedat Dilek @ 2021-10-31 12:38 ` Peter Zijlstra 0 siblings, 0 replies; 13+ messages in thread From: Peter Zijlstra @ 2021-10-31 12:38 UTC (permalink / raw) To: Sedat Dilek Cc: Nick Desaulniers, Nathan Chancellor, Masahiro Yamada, Michal Marek, X86 ML, Linux Kernel Mailing List, Linux Kbuild mailing list, llvm On Sun, Oct 31, 2021 at 02:18:01AM +0100, Sedat Dilek wrote: > On Sat, Oct 30, 2021 at 4:34 PM Peter Zijlstra <peterz@infradead.org> wrote: > > > > > > Latestest greatness... :-) > > > > --- > > Subject: kbuild: Fix clang/llvm build infra > > > > Debian (and derived) distros ship their compilers as -$ver suffixed > > binaries. For gcc it is sufficent to use: > > > > $ make CC=gcc-12 > > > > However, clang builds (esp. clang-lto) need a whole array of tools to be > > exactly right, leading to unweildy stuff like: > > > > $ make CC=clang-13 LD=ld.lld=14 AR=llvm-ar-13 NM=llvm-nm-13 OBJCOPY=llvm-objcopy-13 OBJDUMP=llvm-objdump-13 READELF=llvm-readelf-13 STRIP=llvm-strip-13 LLVM=1 > > > > which is, quite franktly, totally insane and unusable. Instead make > > the CC variable DTRT, enabling one such as myself to use: > > > > $ make CC=clang-13 > > > > This also lets one quickly test different clang versions. > > Additionally, also support path based LLVM suites like: > > > > $ make CC=/opt/llvm/bin/clang > > > > Hi Peter, > > Thanks for bringing this up again. > > Your issue is not new for Debianist and Linux-LLVM/Clang folks. > > But let me comment. > > I agree, it is preferable or should be treated the same way in using > clang-$ver like gcc-$ver. > > Background: Right, I know that. > As a compromise I use in my build_linux-llvm-toolchain.sh script > But... > > ...for several new features like Clang-LTO we need the LLVM full "eco system". > LLVM eco system means compiler, linker and binutils. > In several talks at LPC 2021 talkers used the term "LLVM/binutils" > like for GNU/binutils. > The LLVM/Clang kernel-docs talk about "LLVM util(itie)s" which is set > by LLVM=1 or not (LLVM=0). > > Link: https://www.kernel.org/doc/html/latest/kbuild/llvm.html#llvm-utilities > > AFAICS your patch has a primary focus to fix the LLVM/Clang > infrastructure when CONFIG_LTO_CLANG=y, right? > Can you massage the commit message to reflect this? It already does :-) Also, it's a little bit more complicated, you can't actually select CONFIG_LTO *unless* you've configured your toolchain just right. > IMHO your patch can be improved to check for CONFIG_LTO_CLANG=y where > LLVM=1 is mandatory and IIRC LLVM_IAS=1, too. > ( The default to use LLVM/Clang-IAS (Integrated ASsembler) was changed > recently for some combinations arches + Clang-LTO? ) Egg, meet chicken :-) > Before I forgot: > Can you add a comment for the PPC and S390 situation in the commit message. Sure. > Please remember people also want to use combinations like gcc-$ver and > LLVM/binutils like ld.lld-$ver. make CC=gcc LLVM=1 make CC=clang LLVM=0 are still valid combinations that work, I just don't think they ought to be the default. > GCC-10 + LLD-$VER was the fastest combination to build a Linux Debian > package here on my Intel SandyBridge CPU system. > ( There exists overrides like LD=... STRIP=... and I use > XXX_FOR_BUILD=... (see my attached build-script). ) Right, that's what overrides are for, non-default setups. I just, quite strongly, feel that our defaults are completely insane. ~ Peter ^ permalink raw reply [flat|nested] 13+ messages in thread
* RE: [PATCH] kbuild: Support clang-$ver builds 2021-10-28 23:27 ` Nick Desaulniers 2021-10-29 8:54 ` Peter Zijlstra @ 2021-10-29 8:56 ` David Laight 1 sibling, 0 replies; 13+ messages in thread From: David Laight @ 2021-10-29 8:56 UTC (permalink / raw) To: 'Nick Desaulniers', Peter Zijlstra Cc: Nathan Chancellor, Masahiro Yamada, Michal Marek, X86 ML, Linux Kernel Mailing List, Linux Kbuild mailing list, llvm From: Nick Desaulniers > Sent: 29 October 2021 00:28 ... > > > 2. Update the documentation to describe using the > > > > > > $ PATH=/usr/lib/llvm-#/bin:$PATH make LLVM=1 ... > > > > > > trick. This has been the preferred method for using different > > > versions of LLVM but it has never been documented anywhere. This > > > would allow us to keep the current build infrastructure while giving > > > people clear instructions for how to handle different versions of > > > clang. As Peter has noted, this would require people who are not > > > familiar with building with LLVM to be constantly looking at the > > > documentation to remember the command to invoke, whereas with > > > LLVM=-#, it is easy to remember. > > > > Right, this is no more than a crude hack and is super unfriendly. It > > should never have been accepted. > > Lots of tools modify PATH in your `.bashrc` or `.zshrc` etc. I don't > see how that's any different, other than one off commands in which you > actually intend to use a specific version. Nothing should be modifying the default $PATH, either by hacking my .bashrc or any system default files that get run first. It is as horrid and broken as setting LD_LIBRARY_PATH. Even ldconfig is pretty much a broken idea. David - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales) ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2021-10-31 12:38 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-10-28 13:43 [PATCH] kbuild: Support clang-$ver builds Peter Zijlstra 2021-10-28 14:07 ` Masahiro Yamada 2021-10-28 14:15 ` Peter Zijlstra 2021-10-28 14:22 ` Peter Zijlstra 2021-10-28 15:03 ` Peter Zijlstra 2021-10-28 17:44 ` Nathan Chancellor 2021-10-28 20:48 ` Peter Zijlstra 2021-10-28 23:27 ` Nick Desaulniers 2021-10-29 8:54 ` Peter Zijlstra 2021-10-30 14:33 ` Peter Zijlstra 2021-10-31 1:18 ` Sedat Dilek 2021-10-31 12:38 ` Peter Zijlstra 2021-10-29 8:56 ` David Laight
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).