linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 0/2] infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
@ 2021-07-08 23:25 Nick Desaulniers
  2021-07-08 23:25 ` [PATCH v2 1/2] Makefile: move initial clang flag handling into scripts/Makefile.clang Nick Desaulniers
  2021-07-08 23:25 ` [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1 Nick Desaulniers
  0 siblings, 2 replies; 39+ messages in thread
From: Nick Desaulniers @ 2021-07-08 23:25 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Miguel Ojeda, Fangrui Song, Michal Marek, Arnd Bergmann,
	linux-kernel, linux-kbuild, clang-built-linux,
	Geert Uytterhoeven, Christoph Hellwig, Linus Torvalds,
	Nick Desaulniers

We get constant feedback that the command line invocation of make is too
long. CROSS_COMPILE is helpful when a toolchain has a prefix of the
target triple, or is an absolute path outside of $PATH, but it's mostly
redundant for a given ARCH.

Instead, let's infer it from SRCARCH, and move some flag handling into a
new file included from the top level Makefile.

Changes v1 -> v2:
* patch 1/2 untouched.
* Fix typos in commit message as per Geert and Masahiro.
* Use SRCARCH instead of ARCH, simplifying x86 handling, as per
  Masahiro. Add his sugguested by tag.
* change commit oneline from 'drop' to 'infer.'
* Add detail about explicit host --target and relationship of ARCH to
  SRCARCH, as per Masahiro.

Nick Desaulniers (2):
  Makefile: move initial clang flag handling into scripts/Makefile.clang
  Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1

 Documentation/kbuild/llvm.rst |  5 ++++
 MAINTAINERS                   |  1 +
 Makefile                      | 15 +-----------
 scripts/Makefile.clang        | 44 +++++++++++++++++++++++++++++++++++
 4 files changed, 51 insertions(+), 14 deletions(-)
 create mode 100644 scripts/Makefile.clang


base-commit: a0e781a2a35a8dd4e6a38571998d59c6b0e32cd8
-- 
2.32.0.93.g670b81a890-goog


^ permalink raw reply	[flat|nested] 39+ messages in thread

* [PATCH v2 1/2] Makefile: move initial clang flag handling into scripts/Makefile.clang
  2021-07-08 23:25 [PATCH v2 0/2] infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1 Nick Desaulniers
@ 2021-07-08 23:25 ` Nick Desaulniers
  2021-07-09 20:12   ` Nathan Chancellor
  2021-07-08 23:25 ` [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1 Nick Desaulniers
  1 sibling, 1 reply; 39+ messages in thread
From: Nick Desaulniers @ 2021-07-08 23:25 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Miguel Ojeda, Fangrui Song, Michal Marek, Arnd Bergmann,
	linux-kernel, linux-kbuild, clang-built-linux,
	Geert Uytterhoeven, Christoph Hellwig, Linus Torvalds,
	Nick Desaulniers

With some of the changes we'd like to make to CROSS_COMPILE, the initial
block of clang flag handling which controls things like the target triple,
whether or not to use the integrated assembler and how to find GAS,
and erroring on unknown warnings is becoming unwieldy. Move it into its
own file under scripts/.

Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
 MAINTAINERS            |  1 +
 Makefile               | 15 +--------------
 scripts/Makefile.clang | 14 ++++++++++++++
 3 files changed, 16 insertions(+), 14 deletions(-)
 create mode 100644 scripts/Makefile.clang

diff --git a/MAINTAINERS b/MAINTAINERS
index 81e1edeceae4..9c1205c258c7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4433,6 +4433,7 @@ B:	https://github.com/ClangBuiltLinux/linux/issues
 C:	irc://chat.freenode.net/clangbuiltlinux
 F:	Documentation/kbuild/llvm.rst
 F:	include/linux/compiler-clang.h
+F:	scripts/Makefile.clang
 F:	scripts/clang-tools/
 K:	\b(?i:clang|llvm)\b
 
diff --git a/Makefile b/Makefile
index cbab0dc53065..010e3a4e770b 100644
--- a/Makefile
+++ b/Makefile
@@ -586,20 +586,7 @@ endif
 CC_VERSION_TEXT = $(subst $(pound),,$(shell $(CC) --version 2>/dev/null | head -n 1))
 
 ifneq ($(findstring clang,$(CC_VERSION_TEXT)),)
-ifneq ($(CROSS_COMPILE),)
-CLANG_FLAGS	+= --target=$(notdir $(CROSS_COMPILE:%-=%))
-endif
-ifeq ($(LLVM_IAS),1)
-CLANG_FLAGS	+= -integrated-as
-else
-CLANG_FLAGS	+= -no-integrated-as
-GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit))
-CLANG_FLAGS	+= --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE))
-endif
-CLANG_FLAGS	+= -Werror=unknown-warning-option
-KBUILD_CFLAGS	+= $(CLANG_FLAGS)
-KBUILD_AFLAGS	+= $(CLANG_FLAGS)
-export CLANG_FLAGS
+include $(srctree)/scripts/Makefile.clang
 endif
 
 # Include this also for config targets because some architectures need
diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
new file mode 100644
index 000000000000..297932e973d4
--- /dev/null
+++ b/scripts/Makefile.clang
@@ -0,0 +1,14 @@
+ifneq ($(CROSS_COMPILE),)
+CLANG_FLAGS	+= --target=$(notdir $(CROSS_COMPILE:%-=%))
+endif
+ifeq ($(LLVM_IAS),1)
+CLANG_FLAGS	+= -integrated-as
+else
+CLANG_FLAGS	+= -no-integrated-as
+GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit))
+CLANG_FLAGS	+= --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE))
+endif
+CLANG_FLAGS	+= -Werror=unknown-warning-option
+KBUILD_CFLAGS	+= $(CLANG_FLAGS)
+KBUILD_AFLAGS	+= $(CLANG_FLAGS)
+export CLANG_FLAGS
-- 
2.32.0.93.g670b81a890-goog


^ permalink raw reply	[flat|nested] 39+ messages in thread

* [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-08 23:25 [PATCH v2 0/2] infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1 Nick Desaulniers
  2021-07-08 23:25 ` [PATCH v2 1/2] Makefile: move initial clang flag handling into scripts/Makefile.clang Nick Desaulniers
@ 2021-07-08 23:25 ` Nick Desaulniers
  2021-07-09 20:44   ` Nathan Chancellor
  2021-07-20  8:04   ` Masahiro Yamada
  1 sibling, 2 replies; 39+ messages in thread
From: Nick Desaulniers @ 2021-07-08 23:25 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Miguel Ojeda, Fangrui Song, Michal Marek, Arnd Bergmann,
	linux-kernel, linux-kbuild, clang-built-linux,
	Geert Uytterhoeven, Christoph Hellwig, Linus Torvalds,
	Nick Desaulniers, Nathan Chancellor

We get constant feedback that the command line invocation of make is too
long. CROSS_COMPILE is helpful when a toolchain has a prefix of the
target triple, or is an absolute path outside of $PATH, but it's mostly
redundant for a given SRCARCH. SRCARCH itself is derived from ARCH
(normalized for a few different targets).

If CROSS_COMPILE is not set, simply set --target= for CLANG_FLAGS,
KBUILD_CFLAGS, and KBUILD_AFLAGS based on $SRCARCH.

Previously, we'd cross compile via:
$ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make LLVM=1 LLVM_IAS=1
Now:
$ ARCH=arm64 make LLVM=1 LLVM_IAS=1

For native builds (not involving cross compilation) we now explicitly
specify a target triple rather than rely on the implicit host triple.

Link: https://github.com/ClangBuiltLinux/linux/issues/1399
Suggested-by: Arnd Bergmann <arnd@kernel.org>
Suggested-by: Nathan Chancellor <nathan@kernel.org>
Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
Changes v1 -> v2:
* Fix typos in commit message as per Geert and Masahiro.
* Use SRCARCH instead of ARCH, simplifying x86 handling, as per
  Masahiro. Add his sugguested by tag.
* change commit oneline from 'drop' to 'infer.'
* Add detail about explicit host --target and relationship of ARCH to
  SRCARCH, as per Masahiro.

Changes RFC -> v1:
* Rebase onto linux-kbuild/for-next
* Keep full target triples since missing the gnueabi suffix messes up
  32b ARM. Drop Fangrui's sugguested by tag. Update commit message to
  drop references to arm64.
* Flush out TODOS.
* Add note about -EL/-EB, -m32/-m64.
* Add note to Documentation/.

 Documentation/kbuild/llvm.rst |  5 +++++
 scripts/Makefile.clang        | 34 ++++++++++++++++++++++++++++++++--
 2 files changed, 37 insertions(+), 2 deletions(-)

diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst
index b18401d2ba82..80c63dd9a6d1 100644
--- a/Documentation/kbuild/llvm.rst
+++ b/Documentation/kbuild/llvm.rst
@@ -46,6 +46,11 @@ example: ::
 
 	clang --target=aarch64-linux-gnu foo.c
 
+When both ``LLVM=1`` and ``LLVM_IAS=1`` are used, ``CROSS_COMPILE`` becomes
+unnecessary and can be inferred from ``ARCH``. Example: ::
+
+	ARCH=arm64 make LLVM=1 LLVM_IAS=1
+
 LLVM Utilities
 --------------
 
diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
index 297932e973d4..956603f56724 100644
--- a/scripts/Makefile.clang
+++ b/scripts/Makefile.clang
@@ -1,6 +1,36 @@
-ifneq ($(CROSS_COMPILE),)
+# Individual arch/{arch}/Makfiles should use -EL/-EB to set intended endianness
+# and -m32/-m64 to set word size based on Kconfigs instead of relying on the
+# target triple.
+ifeq ($(CROSS_COMPILE),)
+ifneq ($(LLVM),)
+ifeq ($(LLVM_IAS),1)
+ifeq ($(SRCARCH),arm)
+CLANG_FLAGS	+= --target=arm-linux-gnueabi
+else ifeq ($(SRCARCH),arm64)
+CLANG_FLAGS	+= --target=aarch64-linux-gnu
+else ifeq ($(SRCARCH),hexagon)
+CLANG_FLAGS	+= --target=hexagon-linux-gnu
+else ifeq ($(SRCARCH),m68k)
+CLANG_FLAGS	+= --target=m68k-linux-gnu
+else ifeq ($(SRCARCH),mips)
+CLANG_FLAGS	+= --target=mipsel-linux-gnu
+else ifeq ($(SRCARCH),powerpc)
+CLANG_FLAGS	+= --target=powerpc64le-linux-gnu
+else ifeq ($(SRCARCH),riscv)
+CLANG_FLAGS	+= --target=riscv64-linux-gnu
+else ifeq ($(SRCARCH),s390)
+CLANG_FLAGS	+= --target=s390x-linux-gnu
+else ifeq ($(SRCARCH),x86)
+CLANG_FLAGS	+= --target=x86_64-linux-gnu
+else
+$(error Specify CROSS_COMPILE or add '--target=' option to scripts/Makefile.clang)
+endif # SRCARCH
+endif # LLVM_IAS
+endif # LLVM
+else
 CLANG_FLAGS	+= --target=$(notdir $(CROSS_COMPILE:%-=%))
-endif
+endif # CROSS_COMPILE
+
 ifeq ($(LLVM_IAS),1)
 CLANG_FLAGS	+= -integrated-as
 else
-- 
2.32.0.93.g670b81a890-goog


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 1/2] Makefile: move initial clang flag handling into scripts/Makefile.clang
  2021-07-08 23:25 ` [PATCH v2 1/2] Makefile: move initial clang flag handling into scripts/Makefile.clang Nick Desaulniers
@ 2021-07-09 20:12   ` Nathan Chancellor
  0 siblings, 0 replies; 39+ messages in thread
From: Nathan Chancellor @ 2021-07-09 20:12 UTC (permalink / raw)
  To: Nick Desaulniers, Masahiro Yamada
  Cc: Miguel Ojeda, Fangrui Song, Michal Marek, Arnd Bergmann,
	linux-kernel, linux-kbuild, clang-built-linux,
	Geert Uytterhoeven, Christoph Hellwig, Linus Torvalds

On 7/8/2021 4:25 PM, 'Nick Desaulniers' via Clang Built Linux wrote:
> With some of the changes we'd like to make to CROSS_COMPILE, the initial
> block of clang flag handling which controls things like the target triple,
> whether or not to use the integrated assembler and how to find GAS,
> and erroring on unknown warnings is becoming unwieldy. Move it into its
> own file under scripts/.
> 
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>

Reviewed-by: Nathan Chancellor <nathan@kernel.org>

> ---
>   MAINTAINERS            |  1 +
>   Makefile               | 15 +--------------
>   scripts/Makefile.clang | 14 ++++++++++++++
>   3 files changed, 16 insertions(+), 14 deletions(-)
>   create mode 100644 scripts/Makefile.clang
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 81e1edeceae4..9c1205c258c7 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -4433,6 +4433,7 @@ B:	https://github.com/ClangBuiltLinux/linux/issues
>   C:	irc://chat.freenode.net/clangbuiltlinux
>   F:	Documentation/kbuild/llvm.rst
>   F:	include/linux/compiler-clang.h
> +F:	scripts/Makefile.clang
>   F:	scripts/clang-tools/
>   K:	\b(?i:clang|llvm)\b
>   
> diff --git a/Makefile b/Makefile
> index cbab0dc53065..010e3a4e770b 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -586,20 +586,7 @@ endif
>   CC_VERSION_TEXT = $(subst $(pound),,$(shell $(CC) --version 2>/dev/null | head -n 1))
>   
>   ifneq ($(findstring clang,$(CC_VERSION_TEXT)),)
> -ifneq ($(CROSS_COMPILE),)
> -CLANG_FLAGS	+= --target=$(notdir $(CROSS_COMPILE:%-=%))
> -endif
> -ifeq ($(LLVM_IAS),1)
> -CLANG_FLAGS	+= -integrated-as
> -else
> -CLANG_FLAGS	+= -no-integrated-as
> -GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit))
> -CLANG_FLAGS	+= --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE))
> -endif
> -CLANG_FLAGS	+= -Werror=unknown-warning-option
> -KBUILD_CFLAGS	+= $(CLANG_FLAGS)
> -KBUILD_AFLAGS	+= $(CLANG_FLAGS)
> -export CLANG_FLAGS
> +include $(srctree)/scripts/Makefile.clang
>   endif
>   
>   # Include this also for config targets because some architectures need
> diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
> new file mode 100644
> index 000000000000..297932e973d4
> --- /dev/null
> +++ b/scripts/Makefile.clang
> @@ -0,0 +1,14 @@
> +ifneq ($(CROSS_COMPILE),)
> +CLANG_FLAGS	+= --target=$(notdir $(CROSS_COMPILE:%-=%))
> +endif
> +ifeq ($(LLVM_IAS),1)
> +CLANG_FLAGS	+= -integrated-as
> +else
> +CLANG_FLAGS	+= -no-integrated-as
> +GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit))
> +CLANG_FLAGS	+= --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE))
> +endif
> +CLANG_FLAGS	+= -Werror=unknown-warning-option
> +KBUILD_CFLAGS	+= $(CLANG_FLAGS)
> +KBUILD_AFLAGS	+= $(CLANG_FLAGS)
> +export CLANG_FLAGS
> 

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-08 23:25 ` [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1 Nick Desaulniers
@ 2021-07-09 20:44   ` Nathan Chancellor
  2021-07-20  8:04   ` Masahiro Yamada
  1 sibling, 0 replies; 39+ messages in thread
From: Nathan Chancellor @ 2021-07-09 20:44 UTC (permalink / raw)
  To: Nick Desaulniers, Masahiro Yamada
  Cc: Miguel Ojeda, Fangrui Song, Michal Marek, Arnd Bergmann,
	linux-kernel, linux-kbuild, clang-built-linux,
	Geert Uytterhoeven, Christoph Hellwig, Linus Torvalds

On 7/8/2021 4:25 PM, 'Nick Desaulniers' via Clang Built Linux wrote:
> We get constant feedback that the command line invocation of make is too
> long. CROSS_COMPILE is helpful when a toolchain has a prefix of the
> target triple, or is an absolute path outside of $PATH, but it's mostly
> redundant for a given SRCARCH. SRCARCH itself is derived from ARCH
> (normalized for a few different targets).
> 
> If CROSS_COMPILE is not set, simply set --target= for CLANG_FLAGS,
> KBUILD_CFLAGS, and KBUILD_AFLAGS based on $SRCARCH.
> 
> Previously, we'd cross compile via:
> $ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make LLVM=1 LLVM_IAS=1
> Now:
> $ ARCH=arm64 make LLVM=1 LLVM_IAS=1
> 
> For native builds (not involving cross compilation) we now explicitly
> specify a target triple rather than rely on the implicit host triple.
> 
> Link: https://github.com/ClangBuiltLinux/linux/issues/1399
> Suggested-by: Arnd Bergmann <arnd@kernel.org>
> Suggested-by: Nathan Chancellor <nathan@kernel.org>
> Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
I tested arm, arm64, i386, riscv, and x86_64 and verified that the error 
fired for an unsupported arch. Consider this:

Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>

> ---
> Changes v1 -> v2:
> * Fix typos in commit message as per Geert and Masahiro.
> * Use SRCARCH instead of ARCH, simplifying x86 handling, as per
>    Masahiro. Add his sugguested by tag.
> * change commit oneline from 'drop' to 'infer.'
> * Add detail about explicit host --target and relationship of ARCH to
>    SRCARCH, as per Masahiro.
> 
> Changes RFC -> v1:
> * Rebase onto linux-kbuild/for-next
> * Keep full target triples since missing the gnueabi suffix messes up
>    32b ARM. Drop Fangrui's sugguested by tag. Update commit message to
>    drop references to arm64.
> * Flush out TODOS.
> * Add note about -EL/-EB, -m32/-m64.
> * Add note to Documentation/.
> 
>   Documentation/kbuild/llvm.rst |  5 +++++
>   scripts/Makefile.clang        | 34 ++++++++++++++++++++++++++++++++--
>   2 files changed, 37 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst
> index b18401d2ba82..80c63dd9a6d1 100644
> --- a/Documentation/kbuild/llvm.rst
> +++ b/Documentation/kbuild/llvm.rst
> @@ -46,6 +46,11 @@ example: ::
>   
>   	clang --target=aarch64-linux-gnu foo.c
>   
> +When both ``LLVM=1`` and ``LLVM_IAS=1`` are used, ``CROSS_COMPILE`` becomes
> +unnecessary and can be inferred from ``ARCH``. Example: ::
> +
> +	ARCH=arm64 make LLVM=1 LLVM_IAS=1
> +
>   LLVM Utilities
>   --------------
>   
> diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
> index 297932e973d4..956603f56724 100644
> --- a/scripts/Makefile.clang
> +++ b/scripts/Makefile.clang
> @@ -1,6 +1,36 @@
> -ifneq ($(CROSS_COMPILE),)
> +# Individual arch/{arch}/Makfiles should use -EL/-EB to set intended endianness
> +# and -m32/-m64 to set word size based on Kconfigs instead of relying on the
> +# target triple.
> +ifeq ($(CROSS_COMPILE),)
> +ifneq ($(LLVM),)
> +ifeq ($(LLVM_IAS),1)
> +ifeq ($(SRCARCH),arm)
> +CLANG_FLAGS	+= --target=arm-linux-gnueabi
> +else ifeq ($(SRCARCH),arm64)
> +CLANG_FLAGS	+= --target=aarch64-linux-gnu
> +else ifeq ($(SRCARCH),hexagon)
> +CLANG_FLAGS	+= --target=hexagon-linux-gnu
> +else ifeq ($(SRCARCH),m68k)
> +CLANG_FLAGS	+= --target=m68k-linux-gnu
> +else ifeq ($(SRCARCH),mips)
> +CLANG_FLAGS	+= --target=mipsel-linux-gnu
> +else ifeq ($(SRCARCH),powerpc)
> +CLANG_FLAGS	+= --target=powerpc64le-linux-gnu
> +else ifeq ($(SRCARCH),riscv)
> +CLANG_FLAGS	+= --target=riscv64-linux-gnu
> +else ifeq ($(SRCARCH),s390)
> +CLANG_FLAGS	+= --target=s390x-linux-gnu
> +else ifeq ($(SRCARCH),x86)
> +CLANG_FLAGS	+= --target=x86_64-linux-gnu
> +else
> +$(error Specify CROSS_COMPILE or add '--target=' option to scripts/Makefile.clang)
> +endif # SRCARCH
> +endif # LLVM_IAS
> +endif # LLVM
> +else
>   CLANG_FLAGS	+= --target=$(notdir $(CROSS_COMPILE:%-=%))
> -endif
> +endif # CROSS_COMPILE
> +
>   ifeq ($(LLVM_IAS),1)
>   CLANG_FLAGS	+= -integrated-as
>   else
> 

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-08 23:25 ` [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1 Nick Desaulniers
  2021-07-09 20:44   ` Nathan Chancellor
@ 2021-07-20  8:04   ` Masahiro Yamada
  2021-07-20 17:30     ` Nathan Chancellor
  2021-07-20 17:42     ` Linus Torvalds
  1 sibling, 2 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-20  8:04 UTC (permalink / raw)
  To: Nick Desaulniers
  Cc: Miguel Ojeda, Fangrui Song, Michal Marek, Arnd Bergmann,
	Linux Kernel Mailing List, Linux Kbuild mailing list,
	clang-built-linux, Geert Uytterhoeven, Christoph Hellwig,
	Linus Torvalds, Nathan Chancellor

On Fri, Jul 9, 2021 at 8:25 AM 'Nick Desaulniers' via Clang Built
Linux <clang-built-linux@googlegroups.com> wrote:
>
> We get constant feedback that the command line invocation of make is too
> long. CROSS_COMPILE is helpful when a toolchain has a prefix of the
> target triple, or is an absolute path outside of $PATH, but it's mostly
> redundant for a given SRCARCH. SRCARCH itself is derived from ARCH
> (normalized for a few different targets).
>
> If CROSS_COMPILE is not set, simply set --target= for CLANG_FLAGS,
> KBUILD_CFLAGS, and KBUILD_AFLAGS based on $SRCARCH.
>
> Previously, we'd cross compile via:
> $ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make LLVM=1 LLVM_IAS=1
> Now:
> $ ARCH=arm64 make LLVM=1 LLVM_IAS=1
>
> For native builds (not involving cross compilation) we now explicitly
> specify a target triple rather than rely on the implicit host triple.
>
> Link: https://github.com/ClangBuiltLinux/linux/issues/1399
> Suggested-by: Arnd Bergmann <arnd@kernel.org>
> Suggested-by: Nathan Chancellor <nathan@kernel.org>
> Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
> Changes v1 -> v2:
> * Fix typos in commit message as per Geert and Masahiro.
> * Use SRCARCH instead of ARCH, simplifying x86 handling, as per
>   Masahiro. Add his sugguested by tag.
> * change commit oneline from 'drop' to 'infer.'
> * Add detail about explicit host --target and relationship of ARCH to
>   SRCARCH, as per Masahiro.
>
> Changes RFC -> v1:
> * Rebase onto linux-kbuild/for-next
> * Keep full target triples since missing the gnueabi suffix messes up
>   32b ARM. Drop Fangrui's sugguested by tag. Update commit message to
>   drop references to arm64.
> * Flush out TODOS.
> * Add note about -EL/-EB, -m32/-m64.
> * Add note to Documentation/.
>
>  Documentation/kbuild/llvm.rst |  5 +++++
>  scripts/Makefile.clang        | 34 ++++++++++++++++++++++++++++++++--
>  2 files changed, 37 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst
> index b18401d2ba82..80c63dd9a6d1 100644
> --- a/Documentation/kbuild/llvm.rst
> +++ b/Documentation/kbuild/llvm.rst
> @@ -46,6 +46,11 @@ example: ::
>
>         clang --target=aarch64-linux-gnu foo.c
>
> +When both ``LLVM=1`` and ``LLVM_IAS=1`` are used, ``CROSS_COMPILE`` becomes
> +unnecessary and can be inferred from ``ARCH``. Example: ::
> +
> +       ARCH=arm64 make LLVM=1 LLVM_IAS=1
> +
>  LLVM Utilities
>  --------------
>
> diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
> index 297932e973d4..956603f56724 100644
> --- a/scripts/Makefile.clang
> +++ b/scripts/Makefile.clang
> @@ -1,6 +1,36 @@
> -ifneq ($(CROSS_COMPILE),)
> +# Individual arch/{arch}/Makfiles should use -EL/-EB to set intended endianness
> +# and -m32/-m64 to set word size based on Kconfigs instead of relying on the
> +# target triple.
> +ifeq ($(CROSS_COMPILE),)
> +ifneq ($(LLVM),)


Do you need to check $(LLVM) ?


LLVM=1 is a convenient switch to change all the
defaults, but yet you can flip each tool individually.

Instead of LLVM=1, you still should be able to
get the equivalent setups by:


  make CC=clang LD=ld.lld AR=llvm-ar OBJCOPY=llvm-objcopy ...


The --target option is passed to only
KBUILD_CFLAGS and KBUILD_AFLAGS.

So, when we talk about --target=,
we only care about whether $(CC) is Clang.
Not caring about $(AR), $(LD), or $(OBJCOPY).


scripts/Makefile.clang is already guarded by:

ifneq ($(findstring clang,$(CC_VERSION_TEXT)),









> +ifeq ($(LLVM_IAS),1)
> +ifeq ($(SRCARCH),arm)
> +CLANG_FLAGS    += --target=arm-linux-gnueabi
> +else ifeq ($(SRCARCH),arm64)
> +CLANG_FLAGS    += --target=aarch64-linux-gnu
> +else ifeq ($(SRCARCH),hexagon)
> +CLANG_FLAGS    += --target=hexagon-linux-gnu
> +else ifeq ($(SRCARCH),m68k)
> +CLANG_FLAGS    += --target=m68k-linux-gnu
> +else ifeq ($(SRCARCH),mips)
> +CLANG_FLAGS    += --target=mipsel-linux-gnu
> +else ifeq ($(SRCARCH),powerpc)
> +CLANG_FLAGS    += --target=powerpc64le-linux-gnu
> +else ifeq ($(SRCARCH),riscv)
> +CLANG_FLAGS    += --target=riscv64-linux-gnu
> +else ifeq ($(SRCARCH),s390)
> +CLANG_FLAGS    += --target=s390x-linux-gnu
> +else ifeq ($(SRCARCH),x86)
> +CLANG_FLAGS    += --target=x86_64-linux-gnu
> +else
> +$(error Specify CROSS_COMPILE or add '--target=' option to scripts/Makefile.clang)
> +endif # SRCARCH
> +endif # LLVM_IAS
> +endif # LLVM
> +else
>  CLANG_FLAGS    += --target=$(notdir $(CROSS_COMPILE:%-=%))
> -endif
> +endif # CROSS_COMPILE
> +
>  ifeq ($(LLVM_IAS),1)
>  CLANG_FLAGS    += -integrated-as
>  else
> --
> 2.32.0.93.g670b81a890-goog
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20210708232522.3118208-3-ndesaulniers%40google.com.



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20  8:04   ` Masahiro Yamada
@ 2021-07-20 17:30     ` Nathan Chancellor
  2021-07-21  3:49       ` Masahiro Yamada
  2021-07-20 17:42     ` Linus Torvalds
  1 sibling, 1 reply; 39+ messages in thread
From: Nathan Chancellor @ 2021-07-20 17:30 UTC (permalink / raw)
  To: Masahiro Yamada, Nick Desaulniers
  Cc: Miguel Ojeda, Fangrui Song, Michal Marek, Arnd Bergmann,
	Linux Kernel Mailing List, Linux Kbuild mailing list,
	clang-built-linux, Geert Uytterhoeven, Christoph Hellwig,
	Linus Torvalds

On 7/20/2021 1:04 AM, Masahiro Yamada wrote:
> On Fri, Jul 9, 2021 at 8:25 AM 'Nick Desaulniers' via Clang Built
> Linux <clang-built-linux@googlegroups.com> wrote:
>>
>> We get constant feedback that the command line invocation of make is too
>> long. CROSS_COMPILE is helpful when a toolchain has a prefix of the
>> target triple, or is an absolute path outside of $PATH, but it's mostly
>> redundant for a given SRCARCH. SRCARCH itself is derived from ARCH
>> (normalized for a few different targets).
>>
>> If CROSS_COMPILE is not set, simply set --target= for CLANG_FLAGS,
>> KBUILD_CFLAGS, and KBUILD_AFLAGS based on $SRCARCH.
>>
>> Previously, we'd cross compile via:
>> $ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make LLVM=1 LLVM_IAS=1
>> Now:
>> $ ARCH=arm64 make LLVM=1 LLVM_IAS=1
>>
>> For native builds (not involving cross compilation) we now explicitly
>> specify a target triple rather than rely on the implicit host triple.
>>
>> Link: https://github.com/ClangBuiltLinux/linux/issues/1399
>> Suggested-by: Arnd Bergmann <arnd@kernel.org>
>> Suggested-by: Nathan Chancellor <nathan@kernel.org>
>> Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
>> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
>> ---
>> Changes v1 -> v2:
>> * Fix typos in commit message as per Geert and Masahiro.
>> * Use SRCARCH instead of ARCH, simplifying x86 handling, as per
>>    Masahiro. Add his sugguested by tag.
>> * change commit oneline from 'drop' to 'infer.'
>> * Add detail about explicit host --target and relationship of ARCH to
>>    SRCARCH, as per Masahiro.
>>
>> Changes RFC -> v1:
>> * Rebase onto linux-kbuild/for-next
>> * Keep full target triples since missing the gnueabi suffix messes up
>>    32b ARM. Drop Fangrui's sugguested by tag. Update commit message to
>>    drop references to arm64.
>> * Flush out TODOS.
>> * Add note about -EL/-EB, -m32/-m64.
>> * Add note to Documentation/.
>>
>>   Documentation/kbuild/llvm.rst |  5 +++++
>>   scripts/Makefile.clang        | 34 ++++++++++++++++++++++++++++++++--
>>   2 files changed, 37 insertions(+), 2 deletions(-)
>>
>> diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst
>> index b18401d2ba82..80c63dd9a6d1 100644
>> --- a/Documentation/kbuild/llvm.rst
>> +++ b/Documentation/kbuild/llvm.rst
>> @@ -46,6 +46,11 @@ example: ::
>>
>>          clang --target=aarch64-linux-gnu foo.c
>>
>> +When both ``LLVM=1`` and ``LLVM_IAS=1`` are used, ``CROSS_COMPILE`` becomes
>> +unnecessary and can be inferred from ``ARCH``. Example: ::
>> +
>> +       ARCH=arm64 make LLVM=1 LLVM_IAS=1
>> +
>>   LLVM Utilities
>>   --------------
>>
>> diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
>> index 297932e973d4..956603f56724 100644
>> --- a/scripts/Makefile.clang
>> +++ b/scripts/Makefile.clang
>> @@ -1,6 +1,36 @@
>> -ifneq ($(CROSS_COMPILE),)
>> +# Individual arch/{arch}/Makfiles should use -EL/-EB to set intended endianness
>> +# and -m32/-m64 to set word size based on Kconfigs instead of relying on the
>> +# target triple.
>> +ifeq ($(CROSS_COMPILE),)
>> +ifneq ($(LLVM),)
> 
> 
> Do you need to check $(LLVM) ?
> 
> 
> LLVM=1 is a convenient switch to change all the
> defaults, but yet you can flip each tool individually.
> 
> Instead of LLVM=1, you still should be able to
> get the equivalent setups by:
> 
> 
>    make CC=clang LD=ld.lld AR=llvm-ar OBJCOPY=llvm-objcopy ...
> 
> 
> The --target option is passed to only
> KBUILD_CFLAGS and KBUILD_AFLAGS.
> 
> So, when we talk about --target=,
> we only care about whether $(CC) is Clang.
> Not caring about $(AR), $(LD), or $(OBJCOPY).
> 
> 
> scripts/Makefile.clang is already guarded by:
> 
> ifneq ($(findstring clang,$(CC_VERSION_TEXT)),

$ make ARCH=arm64 CC=clang LLVM_IAS=1

will use the right compiler and assembler but none of the other binary 
tools because '--prefix=' will not be set so CROSS_COMPILE needs to be 
specified still, which defeats the purpose of this whole change. This 
patch is designed to work for the "normal" case of saying "I want to use 
all of the LLVM tools", not "I want to use clang by itself".

>> +ifeq ($(LLVM_IAS),1)
>> +ifeq ($(SRCARCH),arm)
>> +CLANG_FLAGS    += --target=arm-linux-gnueabi
>> +else ifeq ($(SRCARCH),arm64)
>> +CLANG_FLAGS    += --target=aarch64-linux-gnu
>> +else ifeq ($(SRCARCH),hexagon)
>> +CLANG_FLAGS    += --target=hexagon-linux-gnu
>> +else ifeq ($(SRCARCH),m68k)
>> +CLANG_FLAGS    += --target=m68k-linux-gnu
>> +else ifeq ($(SRCARCH),mips)
>> +CLANG_FLAGS    += --target=mipsel-linux-gnu
>> +else ifeq ($(SRCARCH),powerpc)
>> +CLANG_FLAGS    += --target=powerpc64le-linux-gnu
>> +else ifeq ($(SRCARCH),riscv)
>> +CLANG_FLAGS    += --target=riscv64-linux-gnu
>> +else ifeq ($(SRCARCH),s390)
>> +CLANG_FLAGS    += --target=s390x-linux-gnu
>> +else ifeq ($(SRCARCH),x86)
>> +CLANG_FLAGS    += --target=x86_64-linux-gnu
>> +else
>> +$(error Specify CROSS_COMPILE or add '--target=' option to scripts/Makefile.clang)
>> +endif # SRCARCH
>> +endif # LLVM_IAS
>> +endif # LLVM
>> +else
>>   CLANG_FLAGS    += --target=$(notdir $(CROSS_COMPILE:%-=%))
>> -endif
>> +endif # CROSS_COMPILE
>> +
>>   ifeq ($(LLVM_IAS),1)
>>   CLANG_FLAGS    += -integrated-as
>>   else
>> --
>> 2.32.0.93.g670b81a890-goog
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20210708232522.3118208-3-ndesaulniers%40google.com.
> 
> 
> 

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20  8:04   ` Masahiro Yamada
  2021-07-20 17:30     ` Nathan Chancellor
@ 2021-07-20 17:42     ` Linus Torvalds
  2021-07-20 19:58       ` Arnd Bergmann
                         ` (4 more replies)
  1 sibling, 5 replies; 39+ messages in thread
From: Linus Torvalds @ 2021-07-20 17:42 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Nick Desaulniers, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> LLVM=1 is a convenient switch to change all the
> defaults, but yet you can flip each tool individually.

Actually, I'd argue that "LLVM=1" is *not* a convenient switch.
Neither are the individual other command line settings.

When clang was the odd man out, and special, it all made sense.
Changing the path to CC was similar to changing the path to AWK. And
that's obviously why we have what we have.

But clang has become a primary compiler for some kernel communities,
and I think it might be time to just re-visit that entirely.

In particular, I think we should just make it a Kconfig option. I hate
the command flag stuff so much, that my clang tree literally has this
patch in it:

    -CC = $(CROSS_COMPILE)gcc
    +CC = $(CROSS_COMPILE)clang

so that I can just do the same "make -j128" in both my gcc tree and my
clang tree.

But each build tree already has its own .config file, so it would be a
lot more convenient if that was how the compiler was chosen, and then
"make oldconfig" would just DTRT.

We do most of the other heavy lifting in this area in Kconfig anyway,
why not add that compiler choice?

Obviously it would be gated by the tests to see which compilers are
_installed_ (and that they are valid versions), so that it doesn't ask
stupid things ("do you want gcc or clang" when only one of them is
installed and/or viable).

Hmm? So then any "LLVM=1" thing would be about the "make config"
stage, not the actual build stage.

(It has annoyed me for years that if you want to cross-compile, you
first have to do "make ARCH=xyz config" and then remember to do "make
ARCH=xyz" for the build too, but I cross-compile so seldom that I've
never really cared).

Let the flame wars^H^Hpolite discussions ensue..

                        Linus

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 17:42     ` Linus Torvalds
@ 2021-07-20 19:58       ` Arnd Bergmann
  2021-07-20 20:18         ` Nick Desaulniers
  2021-07-21  4:04         ` Masahiro Yamada
  2021-07-20 20:52       ` Nick Desaulniers
                         ` (3 subsequent siblings)
  4 siblings, 2 replies; 39+ messages in thread
From: Arnd Bergmann @ 2021-07-20 19:58 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Masahiro Yamada, Nick Desaulniers, Miguel Ojeda, Fangrui Song,
	Michal Marek, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> We do most of the other heavy lifting in this area in Kconfig anyway,
> why not add that compiler choice?
>
> Obviously it would be gated by the tests to see which compilers are
> _installed_ (and that they are valid versions), so that it doesn't ask
> stupid things ("do you want gcc or clang" when only one of them is
> installed and/or viable).

I don't see a good way of making Kconfig options both select the
compiler and defining variables based on the compiler, since that
would mean teaching Kconfig about re-evaluating all compiler
dependent settings whenever the first option changes.

I do have another idea that I think would work though.

> Hmm? So then any "LLVM=1" thing would be about the "make config"
> stage, not the actual build stage.
>
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).

The best thing that I have come up with is a pre-configure step, where
an object tree gets seeded with a makefile fragment that gets included
for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
'CC=' and possibly any other option that gets passed to 'make' as
a variable and has to exist before calling 'make *config'.

This is probably the easiest part here, and it lets you set up multiple
object directories in which you can then do

make O=obj-x86 defconfig vmlinux modules -skj32
make O=obj-x86-clang
make O=obj-arm64 randconfig
...

without ever having to type the additional CC/CROSS_COMPILE/LLVM
variables again.

One step further is the script that automatically finds a working toolchain
for a given architecture and sets up that object directory accordingly,
or even downloads a working cross-toolchain from kernel.org when
asked to do that.

        Arnd

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 19:58       ` Arnd Bergmann
@ 2021-07-20 20:18         ` Nick Desaulniers
  2021-07-21  4:04         ` Masahiro Yamada
  1 sibling, 0 replies; 39+ messages in thread
From: Nick Desaulniers @ 2021-07-20 20:18 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Linus Torvalds, Masahiro Yamada, Miguel Ojeda, Fangrui Song,
	Michal Marek, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Tue, Jul 20, 2021 at 12:58 PM Arnd Bergmann <arnd@kernel.org> wrote:
>
> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > We do most of the other heavy lifting in this area in Kconfig anyway,
> > why not add that compiler choice?
> >
> > Obviously it would be gated by the tests to see which compilers are
> > _installed_ (and that they are valid versions), so that it doesn't ask
> > stupid things ("do you want gcc or clang" when only one of them is
> > installed and/or viable).
>
> I don't see a good way of making Kconfig options both select the
> compiler and defining variables based on the compiler, since that
> would mean teaching Kconfig about re-evaluating all compiler
> dependent settings whenever the first option changes.

Right, there are many Kconfigs that are predicated on knowing whether
the user is building with gcc vs clang.  We probably could note the
users preference in Kconfig, but we would need to guarantee that we
evaluate that ASAP and before most other Kconfigs were checked.
-- 
Thanks,
~Nick Desaulniers

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 17:42     ` Linus Torvalds
  2021-07-20 19:58       ` Arnd Bergmann
@ 2021-07-20 20:52       ` Nick Desaulniers
  2021-07-20 21:11         ` Linus Torvalds
  2021-07-21  4:53         ` Masahiro Yamada
  2021-07-20 21:29       ` Nick Desaulniers
                         ` (2 subsequent siblings)
  4 siblings, 2 replies; 39+ messages in thread
From: Nick Desaulniers @ 2021-07-20 20:52 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Tue, Jul 20, 2021 at 10:43 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > LLVM=1 is a convenient switch to change all the
> > defaults, but yet you can flip each tool individually.
>
> Actually, I'd argue that "LLVM=1" is *not* a convenient switch.

Compared to the old way of CC=clang LD=ld.lld OBJCOPY=.... it certainly is.

> Neither are the individual other command line settings.

Agreed, but we needed flexibility until we could get all of the
command line tools working for each architecture.  They're still
useful when there's a regression and we need to fall back.  So I
wouldn't be in favor of removing them (not that that's been proposed).

> When clang was the odd man out, and special, it all made sense.
> Changing the path to CC was similar to changing the path to AWK. And
> that's obviously why we have what we have.
>
> But clang has become a primary compiler for some kernel communities,
> and I think it might be time to just re-visit that entirely.

:^)

> In particular, I think we should just make it a Kconfig option. I hate
> the command flag stuff so much, that my clang tree literally has this
> patch in it:
>
>     -CC = $(CROSS_COMPILE)gcc
>     +CC = $(CROSS_COMPILE)clang
>
> so that I can just do the same "make -j128" in both my gcc tree and my
> clang tree.

So you haven't been using LLD... :( (imagine using more than one
thread to link, and being faster than ld.gold)  If anything you should
be hard coding LLVM=1 in that tree.  Also, please be careful you don't
accidentally commit that! 0:-)

> But each build tree already has its own .config file, so it would be a
> lot more convenient if that was how the compiler was chosen, and then
> "make oldconfig" would just DTRT.
>
> We do most of the other heavy lifting in this area in Kconfig anyway,
> why not add that compiler choice?
>
> Obviously it would be gated by the tests to see which compilers are
> _installed_ (and that they are valid versions), so that it doesn't ask
> stupid things ("do you want gcc or clang" when only one of them is
> installed and/or viable).
>
> Hmm? So then any "LLVM=1" thing would be about the "make config"
> stage, not the actual build stage.
>
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).
>
> Let the flame wars^H^Hpolite discussions ensue..

I agree with you.  Overall the command line invocation of make when
cross compiling, or when using LLVM is too long.  You even call out
LLVM=1 and ARCH separately.  Each one of these had good reasons to
exist for years.

But I disagree that all needs to be sorted out together, or right now.
And I'd much rather tackle them separately, one by one, than try to
completely rewrite how we cross compile the kernel today.

Right now, we have:
$ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make LLVM=1 LLVM_IAS=1 -j72

This series is concerned with just CROSS_COMPILE (and just for LLVM=1).
Next I plan to default on LLVM_IAS=1 for all architectures we support,
minus ppc and s390 where we still have some assembler bugs.
Your/Arnd's ideas about LLVM=1 or not via Kconfig, or pre-Kconfig is a
good idea for eliminating LLVM=1.
Then that just leaves ARCH.
Arnd's idea about helping you install a toolchain from kernel.org is
one I support, but orthogonal to the above somewhat.  Do you allow
someone to have a config that denotes intent to build with clang then
prompt if they don't have clang installed to download it? Or do you
prevent someone from selecting building with clang because it's not in
the $PATH?
Your/Arnd's idea about detecting which toolchains are installed is one
I support, but orthogonal to the above somewhat.  (For that, I'm
curious for our build servers if that means having to put tools in
certain locations; I prefer we reference $PATH when possible. Or if
.configs can no longer be shared if tools are in different locations.
But perhaps that's a non-issue).  I'm also curious how many stat calls
we'll need to test/probe/find these, and how we prioritize which tools
are selected when there's more than one version installed.

I encourage us to make steps in the right direction; but I think this
series is ready to go for at least one of the command line variables.
I don't think we need to wait for some probing machinery to eliminate
CROSS_COMPILE when LLVM=1; and if we ever get such machinery we can
revisit whether that helps this case at all.
-- 
Thanks,
~Nick Desaulniers

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 20:52       ` Nick Desaulniers
@ 2021-07-20 21:11         ` Linus Torvalds
  2021-07-20 21:27           ` Nick Desaulniers
  2021-07-21  4:53         ` Masahiro Yamada
  1 sibling, 1 reply; 39+ messages in thread
From: Linus Torvalds @ 2021-07-20 21:11 UTC (permalink / raw)
  To: Nick Desaulniers
  Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Tue, Jul 20, 2021 at 1:52 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> >     -CC = $(CROSS_COMPILE)gcc
> >     +CC = $(CROSS_COMPILE)clang
>
> So you haven't been using LLD... :( (imagine using more than one
> thread to link, and being faster than ld.gold)

I started out just building my own clang - nothing else.

And now that I'm using the distro clang, the linker isn't even _close_
to a performance issue. Since clang is a big, slow, bloated pig.

I posted profiles and performance numbers of clang being three times
slower than gcc, and it was all just clang itself, not the linker. All
due to insanely expensive startup costs.

I suspect a lot of clang users build bloated C++ code where the time
to compile a single object file is so big that the startup costs don't
even show up. But for the (fairly) lean C kernel header files that
don't bring in millions of lines of headers for some template library,
startup costs are a big deal.

So honestly, I don't care at all about "imagine being faster than ld.gold".

I can only _dream_ of clang itself not being 3x slower than gcc.

              Linus

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 21:11         ` Linus Torvalds
@ 2021-07-20 21:27           ` Nick Desaulniers
  0 siblings, 0 replies; 39+ messages in thread
From: Nick Desaulniers @ 2021-07-20 21:27 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor, Tom Stellard

On Tue, Jul 20, 2021 at 2:11 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Tue, Jul 20, 2021 at 1:52 PM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> >
> > >     -CC = $(CROSS_COMPILE)gcc
> > >     +CC = $(CROSS_COMPILE)clang
> >
> > So you haven't been using LLD... :( (imagine using more than one
> > thread to link, and being faster than ld.gold)
>
> I started out just building my own clang - nothing else.
>
> And now that I'm using the distro clang, the linker isn't even _close_
> to a performance issue. Since clang is a big, slow, bloated pig.
>
> I posted profiles and performance numbers of clang being three times
> slower than gcc, and it was all just clang itself, not the linker. All
> due to insanely expensive startup costs.

Yeah, it's unfortunate that package maintainers for your distro chose
to distribute clang built in such a manner.  But in response to that
thread:
1. I planned a whole conference around how folks distribute LLVM
(particularly how the configurations are done, and important ones to
use):
https://lists.llvm.org/pipermail/llvm-dev/2021-June/150861.html
We have a pretty good set of CFPs lined up (and I need to go chase
folks from Apple and Qualcomm that are working on proposals...)
https://github.com/ClangBuiltLinux/llvm-distributors-conf-2021/issues

2. I brainstormed a whole set of ways to make Clang even faster.
https://github.com/ClangBuiltLinux/tc-build/issues
Most promising I think would be building musl with LTO and statically
linking that into the toolchain.  I don't have a ton of time to pursue
that, but I will continue to bang on that drum in front of management.
It's just very hard to fund improving build times for users on
thinkpads when the corporate world has moved on to distributed and
cached building in the cloud.

3. Pretty sure we've resolved that config with your distro's packager,
and made other changes to upstream LLVM to default on some flags that
help a lot when those unfortunate configs are used.
https://reviews.llvm.org/D102090

> I suspect a lot of clang users build bloated C++ code where the time
> to compile a single object file is so big that the startup costs don't
> even show up. But for the (fairly) lean C kernel header files that
> don't bring in millions of lines of headers for some template library,
> startup costs are a big deal.
>
> So honestly, I don't care at all about "imagine being faster than ld.gold".
>
> I can only _dream_ of clang itself not being 3x slower than gcc.

Let's be precise here, because quotes (even when imprecise) lead to PR
cycles.  Clang startup time for NULL input when testing command line
flags as part of a build configure step, when clang is misconfigured
is slower than GCC. Clang is not generally 3x slower than GCC.
-- 
Thanks,
~Nick Desaulniers

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 17:42     ` Linus Torvalds
  2021-07-20 19:58       ` Arnd Bergmann
  2021-07-20 20:52       ` Nick Desaulniers
@ 2021-07-20 21:29       ` Nick Desaulniers
  2021-07-20 21:54         ` Linus Torvalds
  2021-07-21  4:31       ` Masahiro Yamada
  2021-07-21  4:44       ` Christoph Hellwig
  4 siblings, 1 reply; 39+ messages in thread
From: Nick Desaulniers @ 2021-07-20 21:29 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Tue, Jul 20, 2021 at 10:43 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> But clang has become a primary compiler for some kernel communities,
> and I think it might be time to just re-visit that entirely.
>
> In particular, I think we should just make it a Kconfig option. I hate
> the command flag stuff so much, that my clang tree literally has this
> patch in it:
>
>     -CC = $(CROSS_COMPILE)gcc
>     +CC = $(CROSS_COMPILE)clang
>
> so that I can just do the same "make -j128" in both my gcc tree and my
> clang tree.
>
> But each build tree already has its own .config file, so it would be a
> lot more convenient if that was how the compiler was chosen, and then
> "make oldconfig" would just DTRT.
>
> We do most of the other heavy lifting in this area in Kconfig anyway,
> why not add that compiler choice?
>
> Obviously it would be gated by the tests to see which compilers are
> _installed_ (and that they are valid versions), so that it doesn't ask
> stupid things ("do you want gcc or clang" when only one of them is
> installed and/or viable).
>
> Hmm? So then any "LLVM=1" thing would be about the "make config"
> stage, not the actual build stage.
>
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).
>
> Let the flame wars^H^Hpolite discussions ensue..

I will concede that "why do I need to respecify all of my command line
args/env vars to make when I already did so for a configure step?" is
a reasonable question to ask, and may be worth pursuing.

There's still some that we can eliminate for BOTH configure and actual
build, like CROSS_COMPILE. :^)
-- 
Thanks,
~Nick Desaulniers

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 21:29       ` Nick Desaulniers
@ 2021-07-20 21:54         ` Linus Torvalds
  2021-07-20 23:19           ` Linus Torvalds
  2021-07-21  4:52           ` Christoph Hellwig
  0 siblings, 2 replies; 39+ messages in thread
From: Linus Torvalds @ 2021-07-20 21:54 UTC (permalink / raw)
  To: Nick Desaulniers
  Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Tue, Jul 20, 2021 at 2:29 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> There's still some that we can eliminate for BOTH configure and actual
> build, like CROSS_COMPILE. :^)

I have this (perhaps odd) model where I use separate trees for
different things, which is something git obviously is partly designed
for.

So when I create a source tree, I usually create it for one particular
use - whether it be for one particular configuration (ie "build with
clang" or "set up a tree for a 32-bit cross compile" or whatever), or
whether it be for some particular development issue ("long-term branch
for testing asm goto with inputs").

Yeah, sometimes I use multiple git branches in the same tree for
things - but I tend to do that mainly for short-term things where I
don't expect to keep working on it over time.

End result: I find 'make' command line flags to be really really
annoying, because then I have to match them to whatever tree I'm in.

I'd much rather have a local per-tree configuration that just sticks
around. It's why ".config" works fairly well for me.

But there are other things that stick around. In particular, I have
considered simply using git config variables for that.

Something like this in the main Makefile:

    ARCH=$(shell git config build.arch)
    CC=$(shell git config --default=gcc build.cc)

would actually be optimal for what _I_ do. Then for my clang build tree, I'd do

    git config build.cc clang

and it would stick in that tree. If you don't set the option, it would use gcc.

Maybe that would be an acceptable model these days? We've used git
long enough that non-git worries aren't a big deal any more, and I
guess people could use the old-fashioned

    make CC=clang

if they don't use git.

We could get rid of all the CROSS_COMPILE etc garbage that way,
because you'd just set it in the .git/config file. Those are really
convenient and easy to edit.

Would people hate that kind of model?

                   Linus

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 21:54         ` Linus Torvalds
@ 2021-07-20 23:19           ` Linus Torvalds
  2021-07-20 23:22             ` Linus Torvalds
  2021-07-21  5:12             ` Masahiro Yamada
  2021-07-21  4:52           ` Christoph Hellwig
  1 sibling, 2 replies; 39+ messages in thread
From: Linus Torvalds @ 2021-07-20 23:19 UTC (permalink / raw)
  To: Nick Desaulniers
  Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

[-- Attachment #1: Type: text/plain, Size: 649 bytes --]

On Tue, Jul 20, 2021 at 2:54 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> But there are other things that stick around. In particular, I have
> considered simply using git config variables for that.
>
> Something like this in the main Makefile:

A slightly more fleshed-out patch to do this just to see if somebody
wants to work on something like this.

It really would seem to make it very easy to set up any random build
environment, and not have to remember it once it's done.

There are probably better ways to do this, I normally try to avoid
doing Makefile magic, since others have taken it over. Masahiro?

             Linus

[-- Attachment #2: patch.diff --]
[-- Type: application/x-patch, Size: 2083 bytes --]

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 23:19           ` Linus Torvalds
@ 2021-07-20 23:22             ` Linus Torvalds
  2021-07-21  5:12             ` Masahiro Yamada
  1 sibling, 0 replies; 39+ messages in thread
From: Linus Torvalds @ 2021-07-20 23:22 UTC (permalink / raw)
  To: Nick Desaulniers
  Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Tue, Jul 20, 2021 at 4:19 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> There are probably better ways to do this

Oh. In particular, it probably should use a '?=' in order to allow
overriding, and not expanding that $(shell git config..) thing every
time $(CC) is used.

Or something. I didn't really think about those things very much.
Think of that patch - as so many of my patches - as a starting point
and a "how about something like this".

              Linus

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 17:30     ` Nathan Chancellor
@ 2021-07-21  3:49       ` Masahiro Yamada
  2021-07-28 18:59         ` Nick Desaulniers
  0 siblings, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-21  3:49 UTC (permalink / raw)
  To: Nathan Chancellor
  Cc: Nick Desaulniers, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Linus Torvalds

On Wed, Jul 21, 2021 at 2:30 AM Nathan Chancellor <nathan@kernel.org> wrote:
>
> On 7/20/2021 1:04 AM, Masahiro Yamada wrote:
> > On Fri, Jul 9, 2021 at 8:25 AM 'Nick Desaulniers' via Clang Built
> > Linux <clang-built-linux@googlegroups.com> wrote:
> >>
> >> We get constant feedback that the command line invocation of make is too
> >> long. CROSS_COMPILE is helpful when a toolchain has a prefix of the
> >> target triple, or is an absolute path outside of $PATH, but it's mostly
> >> redundant for a given SRCARCH. SRCARCH itself is derived from ARCH
> >> (normalized for a few different targets).
> >>
> >> If CROSS_COMPILE is not set, simply set --target= for CLANG_FLAGS,
> >> KBUILD_CFLAGS, and KBUILD_AFLAGS based on $SRCARCH.
> >>
> >> Previously, we'd cross compile via:
> >> $ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make LLVM=1 LLVM_IAS=1
> >> Now:
> >> $ ARCH=arm64 make LLVM=1 LLVM_IAS=1
> >>
> >> For native builds (not involving cross compilation) we now explicitly
> >> specify a target triple rather than rely on the implicit host triple.
> >>
> >> Link: https://github.com/ClangBuiltLinux/linux/issues/1399
> >> Suggested-by: Arnd Bergmann <arnd@kernel.org>
> >> Suggested-by: Nathan Chancellor <nathan@kernel.org>
> >> Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> >> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> >> ---
> >> Changes v1 -> v2:
> >> * Fix typos in commit message as per Geert and Masahiro.
> >> * Use SRCARCH instead of ARCH, simplifying x86 handling, as per
> >>    Masahiro. Add his sugguested by tag.
> >> * change commit oneline from 'drop' to 'infer.'
> >> * Add detail about explicit host --target and relationship of ARCH to
> >>    SRCARCH, as per Masahiro.
> >>
> >> Changes RFC -> v1:
> >> * Rebase onto linux-kbuild/for-next
> >> * Keep full target triples since missing the gnueabi suffix messes up
> >>    32b ARM. Drop Fangrui's sugguested by tag. Update commit message to
> >>    drop references to arm64.
> >> * Flush out TODOS.
> >> * Add note about -EL/-EB, -m32/-m64.
> >> * Add note to Documentation/.
> >>
> >>   Documentation/kbuild/llvm.rst |  5 +++++
> >>   scripts/Makefile.clang        | 34 ++++++++++++++++++++++++++++++++--
> >>   2 files changed, 37 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst
> >> index b18401d2ba82..80c63dd9a6d1 100644
> >> --- a/Documentation/kbuild/llvm.rst
> >> +++ b/Documentation/kbuild/llvm.rst
> >> @@ -46,6 +46,11 @@ example: ::
> >>
> >>          clang --target=aarch64-linux-gnu foo.c
> >>
> >> +When both ``LLVM=1`` and ``LLVM_IAS=1`` are used, ``CROSS_COMPILE`` becomes
> >> +unnecessary and can be inferred from ``ARCH``. Example: ::
> >> +
> >> +       ARCH=arm64 make LLVM=1 LLVM_IAS=1
> >> +
> >>   LLVM Utilities
> >>   --------------
> >>
> >> diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
> >> index 297932e973d4..956603f56724 100644
> >> --- a/scripts/Makefile.clang
> >> +++ b/scripts/Makefile.clang
> >> @@ -1,6 +1,36 @@
> >> -ifneq ($(CROSS_COMPILE),)
> >> +# Individual arch/{arch}/Makfiles should use -EL/-EB to set intended endianness
> >> +# and -m32/-m64 to set word size based on Kconfigs instead of relying on the
> >> +# target triple.
> >> +ifeq ($(CROSS_COMPILE),)
> >> +ifneq ($(LLVM),)
> >
> >
> > Do you need to check $(LLVM) ?
> >
> >
> > LLVM=1 is a convenient switch to change all the
> > defaults, but yet you can flip each tool individually.
> >
> > Instead of LLVM=1, you still should be able to
> > get the equivalent setups by:
> >
> >
> >    make CC=clang LD=ld.lld AR=llvm-ar OBJCOPY=llvm-objcopy ...
> >
> >
> > The --target option is passed to only
> > KBUILD_CFLAGS and KBUILD_AFLAGS.
> >
> > So, when we talk about --target=,
> > we only care about whether $(CC) is Clang.
> > Not caring about $(AR), $(LD), or $(OBJCOPY).
> >
> >
> > scripts/Makefile.clang is already guarded by:
> >
> > ifneq ($(findstring clang,$(CC_VERSION_TEXT)),
>
> $ make ARCH=arm64 CC=clang LLVM_IAS=1
>
> will use the right compiler and assembler but none of the other binary
> tools because '--prefix=' will not be set so CROSS_COMPILE needs to be
> specified still, which defeats the purpose of this whole change. This
> patch is designed to work for the "normal" case of saying "I want to use
> all of the LLVM tools", not "I want to use clang by itself".


I disagree.

LLVM=1 is a shorthand.



make LLVM=1 LLVM_IAS=1

  should be equivalent to:

make CC=clang LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \
  OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump READELF=llvm-readelf \
  HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar HOSTLD=ld.lld \
  LLVM_IAS=1



We do not care about the origin of CC=clang,
whether it came from LLVM=1 or every tool was explicitly,
individually specified.



ifneq ($(LLVM),) is a garbage code
that checks a pointless thing.







> >> +ifeq ($(LLVM_IAS),1)
> >> +ifeq ($(SRCARCH),arm)
> >> +CLANG_FLAGS    += --target=arm-linux-gnueabi
> >> +else ifeq ($(SRCARCH),arm64)
> >> +CLANG_FLAGS    += --target=aarch64-linux-gnu
> >> +else ifeq ($(SRCARCH),hexagon)
> >> +CLANG_FLAGS    += --target=hexagon-linux-gnu
> >> +else ifeq ($(SRCARCH),m68k)
> >> +CLANG_FLAGS    += --target=m68k-linux-gnu
> >> +else ifeq ($(SRCARCH),mips)
> >> +CLANG_FLAGS    += --target=mipsel-linux-gnu
> >> +else ifeq ($(SRCARCH),powerpc)
> >> +CLANG_FLAGS    += --target=powerpc64le-linux-gnu
> >> +else ifeq ($(SRCARCH),riscv)
> >> +CLANG_FLAGS    += --target=riscv64-linux-gnu
> >> +else ifeq ($(SRCARCH),s390)
> >> +CLANG_FLAGS    += --target=s390x-linux-gnu
> >> +else ifeq ($(SRCARCH),x86)
> >> +CLANG_FLAGS    += --target=x86_64-linux-gnu
> >> +else
> >> +$(error Specify CROSS_COMPILE or add '--target=' option to scripts/Makefile.clang)
> >> +endif # SRCARCH
> >> +endif # LLVM_IAS
> >> +endif # LLVM
> >> +else
> >>   CLANG_FLAGS    += --target=$(notdir $(CROSS_COMPILE:%-=%))
> >> -endif
> >> +endif # CROSS_COMPILE
> >> +
> >>   ifeq ($(LLVM_IAS),1)
> >>   CLANG_FLAGS    += -integrated-as
> >>   else
> >> --
> >> 2.32.0.93.g670b81a890-goog
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> >> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20210708232522.3118208-3-ndesaulniers%40google.com.
> >
> >
> >



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 19:58       ` Arnd Bergmann
  2021-07-20 20:18         ` Nick Desaulniers
@ 2021-07-21  4:04         ` Masahiro Yamada
  2021-07-23 19:54           ` Geert Uytterhoeven
  2021-07-26 20:27           ` Eric W. Biederman
  1 sibling, 2 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-21  4:04 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Linus Torvalds, Nick Desaulniers, Miguel Ojeda, Fangrui Song,
	Michal Marek, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
>
> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > We do most of the other heavy lifting in this area in Kconfig anyway,
> > why not add that compiler choice?
> >
> > Obviously it would be gated by the tests to see which compilers are
> > _installed_ (and that they are valid versions), so that it doesn't ask
> > stupid things ("do you want gcc or clang" when only one of them is
> > installed and/or viable).
>
> I don't see a good way of making Kconfig options both select the
> compiler and defining variables based on the compiler, since that
> would mean teaching Kconfig about re-evaluating all compiler
> dependent settings whenever the first option changes.
>
> I do have another idea that I think would work though.
>
> > Hmm? So then any "LLVM=1" thing would be about the "make config"
> > stage, not the actual build stage.
> >
> > (It has annoyed me for years that if you want to cross-compile, you
> > first have to do "make ARCH=xyz config" and then remember to do "make
> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> > never really cared).
>
> The best thing that I have come up with is a pre-configure step, where
> an object tree gets seeded with a makefile fragment that gets included
> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> 'CC=' and possibly any other option that gets passed to 'make' as
> a variable and has to exist before calling 'make *config'.


There is no need to add a hook to include such makefile fragment(s).

Quite opposite, you can put your Makefile (in a different filename)
that includes the top Makefile.


I think this is what people are already doing:


GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
in this order.


So, you can put 'GNUmakefile' with your favorite setups.


$ cat GNUmakefile
ARCH=arm64
CROSS_COMPILE=aarch64-linux-gnu-
CC=clang
include Makefile



Then, you can simply do "make defconfig; make"
for building arm64 with clang.



If you want to have multiple setups, you can put
custom Makefiles, and select by the -f option.


For example,



$ cat Makefile-arm64
ARCH=arm64
CROSS_COMPILE=aarch64-linux-gnu-
include Makefile


$ cat  Makefile-arm64-llvm
ARCH=arm64
CROSS_COMPILE=aarch64-linux-gnu-
LLVM=1
include Makefile


$ cat Makefile-arm
ARCH=arm
CROSS_COMPILE=arm-linux-gnueabihf-
include Makefile


Then,

$ make -f Makefile-arm64  defconfig all   -j24

$ make -f Makefile-arm64-llvm   defconfig all -j24

$ make -f Makefile-arm  defconfig all -j24


You can use TAB-completion for the 'Makefile-*' part,
so you can save many typings than you do
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- ...



It is just a tip to ease your life without any change to upstream.
Maybe, some people might be doing that.




>
> This is probably the easiest part here, and it lets you set up multiple
> object directories in which you can then do
>
> make O=obj-x86 defconfig vmlinux modules -skj32
> make O=obj-x86-clang
> make O=obj-arm64 randconfig
> ...
>
> without ever having to type the additional CC/CROSS_COMPILE/LLVM
> variables again.
>
> One step further is the script that automatically finds a working toolchain
> for a given architecture and sets up that object directory accordingly,
> or even downloads a working cross-toolchain from kernel.org when
> asked to do that.


I saw Intel's 0-day bot attached a shell script to download and
set up cross compilers.
(make.cross)




--
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 17:42     ` Linus Torvalds
                         ` (2 preceding siblings ...)
  2021-07-20 21:29       ` Nick Desaulniers
@ 2021-07-21  4:31       ` Masahiro Yamada
  2021-07-21  4:44       ` Christoph Hellwig
  4 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-21  4:31 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Nick Desaulniers, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Wed, Jul 21, 2021 at 2:43 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > LLVM=1 is a convenient switch to change all the
> > defaults, but yet you can flip each tool individually.
>
> Actually, I'd argue that "LLVM=1" is *not* a convenient switch.
> Neither are the individual other command line settings.
>
> When clang was the odd man out, and special, it all made sense.
> Changing the path to CC was similar to changing the path to AWK. And
> that's obviously why we have what we have.
>
> But clang has become a primary compiler for some kernel communities,
> and I think it might be time to just re-visit that entirely.
>
> In particular, I think we should just make it a Kconfig option. I hate
> the command flag stuff so much, that my clang tree literally has this
> patch in it:
>
>     -CC = $(CROSS_COMPILE)gcc
>     +CC = $(CROSS_COMPILE)clang
>
> so that I can just do the same "make -j128" in both my gcc tree and my
> clang tree.
>
> But each build tree already has its own .config file, so it would be a
> lot more convenient if that was how the compiler was chosen, and then
> "make oldconfig" would just DTRT.
>
> We do most of the other heavy lifting in this area in Kconfig anyway,
> why not add that compiler choice?


This does not work in the current design.

When we moved compiler tests to Kconfig from Makefile at v4.18,
I discussed a lot about the implementation.
Especially 'static' vs 'dynamic'.
(mostly Ulf Magnusson actively gave me feed backs)


static model
    Kconfig is given with a fixed set of toolchains.
    All macros such as cc-option are expanded in the early parse stage
    before proceeding to the configuration stage.


dynamic  model
   Every time a user toggles a CONFIG option, it feeds back to
   cc-option because CONFIG options (such as CONFIG_64BIT)
   might affect cc-option. In this model, CONFIG_CROSS_COMPILE
   works well.




Of course, the dynamic model is much more complex,
and I could not find a sane syntax to achieve it.

So, I implemented the static model; in this model,
$(cc-option ) are all constant values.

So, you cannot do:


  choice
         prompt "Compiler suite"

  config GCC
          "GCC and GNU binutils"

   config LLVM
           "LLVM (clang and llvm tools)"

  endchoice



To achieve this, we need the dynamic model.

Every time a user switches between CONFIG_GCC and CONFIG_LLVM,
all the cc-options must be re-evaluated.
This does not work in the current implementation.






BTW, keeping the command line interface
such as CROSS_COMPILE=...  CC=... LD=...
is useful when the kernel is built from umbrella
projects such as Buildroot, OpenEmbedded, etc.

Such projects build compilers by themselves,
and feed the exact compiler paths to the kernel.

Too long command line is an annoyance when
we build the kernel manually.












>
> Obviously it would be gated by the tests to see which compilers are
> _installed_ (and that they are valid versions), so that it doesn't ask
> stupid things ("do you want gcc or clang" when only one of them is
> installed and/or viable).
>
> Hmm? So then any "LLVM=1" thing would be about the "make config"
> stage, not the actual build stage.
>
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).
>
> Let the flame wars^H^Hpolite discussions ensue..
>
>                         Linus

--
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 17:42     ` Linus Torvalds
                         ` (3 preceding siblings ...)
  2021-07-21  4:31       ` Masahiro Yamada
@ 2021-07-21  4:44       ` Christoph Hellwig
  4 siblings, 0 replies; 39+ messages in thread
From: Christoph Hellwig @ 2021-07-21  4:44 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Masahiro Yamada, Nick Desaulniers, Miguel Ojeda, Fangrui Song,
	Michal Marek, Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Tue, Jul 20, 2021 at 10:42:57AM -0700, Linus Torvalds wrote:
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).

Same here.  I'd so much love to just have ARCH an CROSS_COMPILE in my
.config for cross-builds.  Oterwise I keep forgetting them, or keep
forgetting to unset CROSS_COMPILE and will just waist a lot of time
either way.

The fact the the .config files even mention what architecture they are
for in a comment but don't make use of that information is just hostile
to users.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 21:54         ` Linus Torvalds
  2021-07-20 23:19           ` Linus Torvalds
@ 2021-07-21  4:52           ` Christoph Hellwig
  2021-07-21  5:33             ` Masahiro Yamada
  1 sibling, 1 reply; 39+ messages in thread
From: Christoph Hellwig @ 2021-07-21  4:52 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Nick Desaulniers, Masahiro Yamada, Miguel Ojeda, Fangrui Song,
	Michal Marek, Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Tue, Jul 20, 2021 at 02:54:35PM -0700, Linus Torvalds wrote:
> But there are other things that stick around. In particular, I have
> considered simply using git config variables for that.
> 
> Something like this in the main Makefile:
> 
>     ARCH=$(shell git config build.arch)
>     CC=$(shell git config --default=gcc build.cc)
> 
> would actually be optimal for what _I_ do. Then for my clang build tree, I'd do
> 
>     git config build.cc clang
> 
> and it would stick in that tree. If you don't set the option, it would use gcc.
> 
> Maybe that would be an acceptable model these days? We've used git
> long enough that non-git worries aren't a big deal any more, and I
> guess people could use the old-fashioned

I still this only papers around the fact that the architecture is
fundamentally part of the kernel config and should come from .config,
and the toolchain probably as well.

I for one do have a few different tree for different projects, but
need to try cross compiles in each of them.  And unfortunately sometimes
with clang (nothing against clang itself [1], just that having to deal
with multiple toolchains is a pain)

[1] well, except for the sometimes idiotic diagnostics..

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 20:52       ` Nick Desaulniers
  2021-07-20 21:11         ` Linus Torvalds
@ 2021-07-21  4:53         ` Masahiro Yamada
  1 sibling, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-21  4:53 UTC (permalink / raw)
  To: Nick Desaulniers
  Cc: Linus Torvalds, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Wed, Jul 21, 2021 at 5:52 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Tue, Jul 20, 2021 at 10:43 AM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > LLVM=1 is a convenient switch to change all the
> > > defaults, but yet you can flip each tool individually.
> >
> > Actually, I'd argue that "LLVM=1" is *not* a convenient switch.
>
> Compared to the old way of CC=clang LD=ld.lld OBJCOPY=.... it certainly is.
>
> > Neither are the individual other command line settings.
>
> Agreed, but we needed flexibility until we could get all of the
> command line tools working for each architecture.  They're still
> useful when there's a regression and we need to fall back.  So I
> wouldn't be in favor of removing them (not that that's been proposed).
>
> > When clang was the odd man out, and special, it all made sense.
> > Changing the path to CC was similar to changing the path to AWK. And
> > that's obviously why we have what we have.
> >
> > But clang has become a primary compiler for some kernel communities,
> > and I think it might be time to just re-visit that entirely.
>
> :^)
>
> > In particular, I think we should just make it a Kconfig option. I hate
> > the command flag stuff so much, that my clang tree literally has this
> > patch in it:
> >
> >     -CC = $(CROSS_COMPILE)gcc
> >     +CC = $(CROSS_COMPILE)clang
> >
> > so that I can just do the same "make -j128" in both my gcc tree and my
> > clang tree.
>
> So you haven't been using LLD... :( (imagine using more than one
> thread to link, and being faster than ld.gold)  If anything you should
> be hard coding LLVM=1 in that tree.  Also, please be careful you don't
> accidentally commit that! 0:-)
>
> > But each build tree already has its own .config file, so it would be a
> > lot more convenient if that was how the compiler was chosen, and then
> > "make oldconfig" would just DTRT.
> >
> > We do most of the other heavy lifting in this area in Kconfig anyway,
> > why not add that compiler choice?
> >
> > Obviously it would be gated by the tests to see which compilers are
> > _installed_ (and that they are valid versions), so that it doesn't ask
> > stupid things ("do you want gcc or clang" when only one of them is
> > installed and/or viable).
> >
> > Hmm? So then any "LLVM=1" thing would be about the "make config"
> > stage, not the actual build stage.
> >
> > (It has annoyed me for years that if you want to cross-compile, you
> > first have to do "make ARCH=xyz config" and then remember to do "make
> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> > never really cared).
> >
> > Let the flame wars^H^Hpolite discussions ensue..
>
> I agree with you.  Overall the command line invocation of make when
> cross compiling, or when using LLVM is too long.  You even call out
> LLVM=1 and ARCH separately.  Each one of these had good reasons to
> exist for years.
>
> But I disagree that all needs to be sorted out together, or right now.
> And I'd much rather tackle them separately, one by one, than try to
> completely rewrite how we cross compile the kernel today.
>
> Right now, we have:
> $ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make LLVM=1 LLVM_IAS=1 -j72
>
> This series is concerned with just CROSS_COMPILE (and just for LLVM=1).
> Next I plan to default on LLVM_IAS=1 for all architectures we support,
> minus ppc and s390 where we still have some assembler bugs.
> Your/Arnd's ideas about LLVM=1 or not via Kconfig, or pre-Kconfig is a
> good idea for eliminating LLVM=1.


I also like to make the integrated assembler our default.

We can add LLVM_DISABLE_IAS=1 to
replace LLVM_IAS=1.







> Then that just leaves ARCH.
> Arnd's idea about helping you install a toolchain from kernel.org is
> one I support, but orthogonal to the above somewhat.  Do you allow
> someone to have a config that denotes intent to build with clang then
> prompt if they don't have clang installed to download it? Or do you
> prevent someone from selecting building with clang because it's not in
> the $PATH?
> Your/Arnd's idea about detecting which toolchains are installed is one
> I support, but orthogonal to the above somewhat.  (For that, I'm
> curious for our build servers if that means having to put tools in
> certain locations; I prefer we reference $PATH when possible. Or if
> .configs can no longer be shared if tools are in different locations.
> But perhaps that's a non-issue).  I'm also curious how many stat calls
> we'll need to test/probe/find these, and how we prioritize which tools
> are selected when there's more than one version installed.
>
> I encourage us to make steps in the right direction; but I think this
> series is ready to go for at least one of the command line variables.
> I don't think we need to wait for some probing machinery to eliminate
> CROSS_COMPILE when LLVM=1; and if we ever get such machinery we can
> revisit whether that helps this case at all.
> --
> Thanks,
> ~Nick Desaulniers



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-20 23:19           ` Linus Torvalds
  2021-07-20 23:22             ` Linus Torvalds
@ 2021-07-21  5:12             ` Masahiro Yamada
  1 sibling, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-21  5:12 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Nick Desaulniers, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Wed, Jul 21, 2021 at 8:20 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Tue, Jul 20, 2021 at 2:54 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > But there are other things that stick around. In particular, I have
> > considered simply using git config variables for that.
> >
> > Something like this in the main Makefile:
>
> A slightly more fleshed-out patch to do this just to see if somebody
> wants to work on something like this.
>
> It really would seem to make it very easy to set up any random build
> environment, and not have to remember it once it's done.
>
> There are probably better ways to do this, I normally try to avoid
> doing Makefile magic, since others have taken it over. Masahiro?


The patch would uglify the Makefile.

If git is not installed, $(shell git config  ...)
will spit an error message, so '2>/dev/null' is needed.

Actually, I believe we should be able to build the kernel
from a source tarball without relying on git in any way.

In fact, we rely on git in some places, for example,
scripts/setlocalversion can add commit hash to the release
string, but I want to minimize the git-dependent code.








>              Linus
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAHk-%3DwhS1zBU%3DuhDTk2M1LwkqG7AKLsR0_%2BXG%2BsaY_s2FHZr-A%40mail.gmail.com.



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-21  4:52           ` Christoph Hellwig
@ 2021-07-21  5:33             ` Masahiro Yamada
  0 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-21  5:33 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: Linus Torvalds, Nick Desaulniers, Miguel Ojeda, Fangrui Song,
	Michal Marek, Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Nathan Chancellor

On Wed, Jul 21, 2021 at 1:53 PM Christoph Hellwig <hch@infradead.org> wrote:
>
> On Tue, Jul 20, 2021 at 02:54:35PM -0700, Linus Torvalds wrote:
> > But there are other things that stick around. In particular, I have
> > considered simply using git config variables for that.
> >
> > Something like this in the main Makefile:
> >
> >     ARCH=$(shell git config build.arch)
> >     CC=$(shell git config --default=gcc build.cc)
> >
> > would actually be optimal for what _I_ do. Then for my clang build tree, I'd do
> >
> >     git config build.cc clang
> >
> > and it would stick in that tree. If you don't set the option, it would use gcc.
> >
> > Maybe that would be an acceptable model these days? We've used git
> > long enough that non-git worries aren't a big deal any more, and I
> > guess people could use the old-fashioned
>
> I still this only papers around the fact that the architecture is
> fundamentally part of the kernel config


In fact, I am the person who did it in U-boot several years ago.

In U-Boot Kconfig, the architecture is selected by a Kconfig choice:

https://github.com/u-boot/u-boot/blob/v2021.07/arch/Kconfig#L22

(but, if you look at Makefiles and Kconfigs of U-Boot,
you will notice they are uglier than the kernel's ones.)



Theoretically, yes, it is possible to make ARCH as part of the .config
in the kernel as well, but we need careful consideration to make
it robust, clean, and efficient.

In the current model, the Kconfig stage would be even slower
because Kconfig would end up with a bunch of unrelated cc-option
from other architectures.

For example, arch/s390/Kconfig has so many cc-option.
If we allow Kconfig to choose "ARCH', all Kconfig files
of all architectures must be parsed.
If you intend to build the x86 kernel, you might think that
parsing the expensive arch/s390/Kconfig is pointless.





> and should come from .config,
> and the toolchain probably as well.
>
> I for one do have a few different tree for different projects, but
> need to try cross compiles in each of them.  And unfortunately sometimes
> with clang (nothing against clang itself [1], just that having to deal
> with multiple toolchains is a pain)
>
> [1] well, except for the sometimes idiotic diagnostics..


--
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-21  4:04         ` Masahiro Yamada
@ 2021-07-23 19:54           ` Geert Uytterhoeven
  2021-07-24 13:46             ` Masahiro Yamada
  2021-07-26 20:27           ` Eric W. Biederman
  1 sibling, 1 reply; 39+ messages in thread
From: Geert Uytterhoeven @ 2021-07-23 19:54 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
	Fangrui Song, Michal Marek, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Christoph Hellwig,
	Nathan Chancellor

Hi Yamada-san,

On Wed, Jul 21, 2021 at 6:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> > On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> > <torvalds@linux-foundation.org> wrote:
> > > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > We do most of the other heavy lifting in this area in Kconfig anyway,
> > > why not add that compiler choice?
> > >
> > > Obviously it would be gated by the tests to see which compilers are
> > > _installed_ (and that they are valid versions), so that it doesn't ask
> > > stupid things ("do you want gcc or clang" when only one of them is
> > > installed and/or viable).
> >
> > I don't see a good way of making Kconfig options both select the
> > compiler and defining variables based on the compiler, since that
> > would mean teaching Kconfig about re-evaluating all compiler
> > dependent settings whenever the first option changes.
> >
> > I do have another idea that I think would work though.
> >
> > > Hmm? So then any "LLVM=1" thing would be about the "make config"
> > > stage, not the actual build stage.
> > >
> > > (It has annoyed me for years that if you want to cross-compile, you
> > > first have to do "make ARCH=xyz config" and then remember to do "make
> > > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> > > never really cared).
> >
> > The best thing that I have come up with is a pre-configure step, where
> > an object tree gets seeded with a makefile fragment that gets included
> > for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> > 'CC=' and possibly any other option that gets passed to 'make' as
> > a variable and has to exist before calling 'make *config'.
>
>
> There is no need to add a hook to include such makefile fragment(s).
>
> Quite opposite, you can put your Makefile (in a different filename)
> that includes the top Makefile.
>
>
> I think this is what people are already doing:
>
>
> GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> in this order.

Exactly. I only have a few source repositories, but lots of build directories
(I never build in a repo directory). Each build directory has a GNUmakefile:

    build/linux-riscv-starlight$ cat GNUmakefile
    MAKEARGS = ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu-

    include ~/linux/default.mk
    build/linux-riscv-starlight$ cat ~/linux/default.mk
    MAKEFLAGS += --no-print-directory

    .PHONY: all $(MAKECMDGOALS)

    all := $(filter-out all Makefile,$(MAKECMDGOALS))

    all:
            @$(MAKE) $(MAKEARGS) $(all) -f Makefile

    Makefile:;

    $(all): all
            @:

    %/: all
            @:
    build/linux-riscv-starlight$

Perhaps we could have a "make setup ARCH=foo CROSS_COMPILE=bar" target,
which creates such a GNUmakefile?

P.S. I put the extra logic in ~/linux/default.mk, so I don't have to update all
     GNUmakefiles when I want to make a change to the main logic.

     For build dirs where I want to track a specific config, I have a slightly
     different version:

         build/linux-m68k-allmodconfig-sun3$ cat GNUmakefile
         MAKEARGS = ARCH=m68k KCONFIG_ALLCONFIG=1
         DEFCONFIG = allmodconfig

         include ~/linux/override-oldconfig.mk
         build/linux-m68k-allmodconfig-sun3$ cat ~/linux/override-oldconfig.mk
         MAKEFLAGS += --no-print-directory

         .PHONY: all oldconfig realoldconfig $(MAKECMDGOALS)

         all := $(filter-out all oldconfig realoldconfig
Makefile,$(MAKECMDGOALS))

         all:
                 @$(MAKE) $(MAKEARGS) $(all) -f Makefile

         # Replace oldconfig by $(DEFCONFIG)
         oldconfig:
                 @$(MAKE) $(MAKEARGS) $(DEFCONFIG) -f Makefile

         realoldconfig:
                 @$(MAKE) $(MAKEARGS) oldconfig -f Makefile

         Makefile:;

         $(all): all
                 @:

         %/: all
                 @:
         build/linux-m68k-allmodconfig-sun3$

     That way I can always just type "make oldconfig", and it will do what
     I want.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-23 19:54           ` Geert Uytterhoeven
@ 2021-07-24 13:46             ` Masahiro Yamada
  0 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-24 13:46 UTC (permalink / raw)
  To: Geert Uytterhoeven
  Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
	Fangrui Song, Michal Marek, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Christoph Hellwig,
	Nathan Chancellor

On Sat, Jul 24, 2021 at 4:54 AM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> Hi Yamada-san,
>
> On Wed, Jul 21, 2021 at 6:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> > > On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> > > <torvalds@linux-foundation.org> wrote:
> > > > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > >
> > > > We do most of the other heavy lifting in this area in Kconfig anyway,
> > > > why not add that compiler choice?
> > > >
> > > > Obviously it would be gated by the tests to see which compilers are
> > > > _installed_ (and that they are valid versions), so that it doesn't ask
> > > > stupid things ("do you want gcc or clang" when only one of them is
> > > > installed and/or viable).
> > >
> > > I don't see a good way of making Kconfig options both select the
> > > compiler and defining variables based on the compiler, since that
> > > would mean teaching Kconfig about re-evaluating all compiler
> > > dependent settings whenever the first option changes.
> > >
> > > I do have another idea that I think would work though.
> > >
> > > > Hmm? So then any "LLVM=1" thing would be about the "make config"
> > > > stage, not the actual build stage.
> > > >
> > > > (It has annoyed me for years that if you want to cross-compile, you
> > > > first have to do "make ARCH=xyz config" and then remember to do "make
> > > > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> > > > never really cared).
> > >
> > > The best thing that I have come up with is a pre-configure step, where
> > > an object tree gets seeded with a makefile fragment that gets included
> > > for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> > > 'CC=' and possibly any other option that gets passed to 'make' as
> > > a variable and has to exist before calling 'make *config'.
> >
> >
> > There is no need to add a hook to include such makefile fragment(s).
> >
> > Quite opposite, you can put your Makefile (in a different filename)
> > that includes the top Makefile.
> >
> >
> > I think this is what people are already doing:
> >
> >
> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> > in this order.
>
> Exactly. I only have a few source repositories, but lots of build directories
> (I never build in a repo directory). Each build directory has a GNUmakefile:
>
>     build/linux-riscv-starlight$ cat GNUmakefile
>     MAKEARGS = ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu-
>
>     include ~/linux/default.mk
>     build/linux-riscv-starlight$ cat ~/linux/default.mk
>     MAKEFLAGS += --no-print-directory
>
>     .PHONY: all $(MAKECMDGOALS)
>
>     all := $(filter-out all Makefile,$(MAKECMDGOALS))
>
>     all:
>             @$(MAKE) $(MAKEARGS) $(all) -f Makefile
>
>     Makefile:;
>
>     $(all): all
>             @:
>
>     %/: all
>             @:
>     build/linux-riscv-starlight$
>
> Perhaps we could have a "make setup ARCH=foo CROSS_COMPILE=bar" target,
> which creates such a GNUmakefile?

No, I do not think so.

Your GNUmakefile is customized in your way.
Other people implement GNUmakefile in their own way.

Upstream is not a place to merge everybody's local tools.








> P.S. I put the extra logic in ~/linux/default.mk, so I don't have to update all
>      GNUmakefiles when I want to make a change to the main logic.
>
>      For build dirs where I want to track a specific config, I have a slightly
>      different version:
>
>          build/linux-m68k-allmodconfig-sun3$ cat GNUmakefile
>          MAKEARGS = ARCH=m68k KCONFIG_ALLCONFIG=1
>          DEFCONFIG = allmodconfig
>
>          include ~/linux/override-oldconfig.mk
>          build/linux-m68k-allmodconfig-sun3$ cat ~/linux/override-oldconfig.mk
>          MAKEFLAGS += --no-print-directory
>
>          .PHONY: all oldconfig realoldconfig $(MAKECMDGOALS)
>
>          all := $(filter-out all oldconfig realoldconfig
> Makefile,$(MAKECMDGOALS))
>
>          all:
>                  @$(MAKE) $(MAKEARGS) $(all) -f Makefile
>
>          # Replace oldconfig by $(DEFCONFIG)
>          oldconfig:
>                  @$(MAKE) $(MAKEARGS) $(DEFCONFIG) -f Makefile
>
>          realoldconfig:
>                  @$(MAKE) $(MAKEARGS) oldconfig -f Makefile
>
>          Makefile:;
>
>          $(all): all
>                  @:
>
>          %/: all
>                  @:
>          build/linux-m68k-allmodconfig-sun3$
>
>      That way I can always just type "make oldconfig", and it will do what
>      I want.
>
> Gr{oetje,eeting}s,
>
>                         Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
>                                 -- Linus Torvalds
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAMuHMdXJBqrpzaSNDJgic14ESiHV6cCcb_5E-st6iniXdmm9_g%40mail.gmail.com.



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-21  4:04         ` Masahiro Yamada
  2021-07-23 19:54           ` Geert Uytterhoeven
@ 2021-07-26 20:27           ` Eric W. Biederman
  2021-07-27  7:07             ` Geert Uytterhoeven
  2021-07-27 10:10             ` Masahiro Yamada
  1 sibling, 2 replies; 39+ messages in thread
From: Eric W. Biederman @ 2021-07-26 20:27 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
	Fangrui Song, Michal Marek, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

Masahiro Yamada <masahiroy@kernel.org> writes:

> On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
>>
>> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
>> <torvalds@linux-foundation.org> wrote:
>> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>> >
>> > We do most of the other heavy lifting in this area in Kconfig anyway,
>> > why not add that compiler choice?
>> >
>> > Obviously it would be gated by the tests to see which compilers are
>> > _installed_ (and that they are valid versions), so that it doesn't ask
>> > stupid things ("do you want gcc or clang" when only one of them is
>> > installed and/or viable).
>>
>> I don't see a good way of making Kconfig options both select the
>> compiler and defining variables based on the compiler, since that
>> would mean teaching Kconfig about re-evaluating all compiler
>> dependent settings whenever the first option changes.
>>
>> I do have another idea that I think would work though.
>>
>> > Hmm? So then any "LLVM=1" thing would be about the "make config"
>> > stage, not the actual build stage.
>> >
>> > (It has annoyed me for years that if you want to cross-compile, you
>> > first have to do "make ARCH=xyz config" and then remember to do "make
>> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
>> > never really cared).
>>
>> The best thing that I have come up with is a pre-configure step, where
>> an object tree gets seeded with a makefile fragment that gets included
>> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
>> 'CC=' and possibly any other option that gets passed to 'make' as
>> a variable and has to exist before calling 'make *config'.
>
>
> There is no need to add a hook to include such makefile fragment(s).
>
> Quite opposite, you can put your Makefile (in a different filename)
> that includes the top Makefile.
>
>
> I think this is what people are already doing:
>
>
> GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> in this order.
>
>
> So, you can put 'GNUmakefile' with your favorite setups.
>
>
> $ cat GNUmakefile
> ARCH=arm64
> CROSS_COMPILE=aarch64-linux-gnu-
> CC=clang
> include Makefile

Very weird.

I just tested this and it does not work.
I did this:

$ cat GNUmakefile
ARCH = alpha
CROSS_COMPILE = $(arch-prefix alpha)
include Makefile

In one of my build directories and the main makefile simply does not see
the value of ARCH or CROSS_COMPILE I set.  I have confirmed that my
GNUmakefile is being read, because everything breaks if I remove the
include line.

Does anyone have any ideas?

Something so we don't have to specify all of these variables on the make
command line would be nice.

Eric

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-26 20:27           ` Eric W. Biederman
@ 2021-07-27  7:07             ` Geert Uytterhoeven
  2021-07-27  7:49               ` Arnd Bergmann
  2021-07-27 10:10             ` Masahiro Yamada
  1 sibling, 1 reply; 39+ messages in thread
From: Geert Uytterhoeven @ 2021-07-27  7:07 UTC (permalink / raw)
  To: Eric W. Biederman
  Cc: Masahiro Yamada, Arnd Bergmann, Linus Torvalds, Nick Desaulniers,
	Miguel Ojeda, Fangrui Song, Michal Marek,
	Linux Kernel Mailing List, Linux Kbuild mailing list,
	clang-built-linux, Christoph Hellwig, Nathan Chancellor

[-- Attachment #1: Type: text/plain, Size: 3724 bytes --]

Hi Eric,

On Mon, Jul 26, 2021 at 10:27 PM Eric W. Biederman
<ebiederm@xmission.com> wrote:
> Masahiro Yamada <masahiroy@kernel.org> writes:
> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> >> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> >> <torvalds@linux-foundation.org> wrote:
> >> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >> >
> >> > We do most of the other heavy lifting in this area in Kconfig anyway,
> >> > why not add that compiler choice?
> >> >
> >> > Obviously it would be gated by the tests to see which compilers are
> >> > _installed_ (and that they are valid versions), so that it doesn't ask
> >> > stupid things ("do you want gcc or clang" when only one of them is
> >> > installed and/or viable).
> >>
> >> I don't see a good way of making Kconfig options both select the
> >> compiler and defining variables based on the compiler, since that
> >> would mean teaching Kconfig about re-evaluating all compiler
> >> dependent settings whenever the first option changes.
> >>
> >> I do have another idea that I think would work though.
> >>
> >> > Hmm? So then any "LLVM=1" thing would be about the "make config"
> >> > stage, not the actual build stage.
> >> >
> >> > (It has annoyed me for years that if you want to cross-compile, you
> >> > first have to do "make ARCH=xyz config" and then remember to do "make
> >> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> >> > never really cared).
> >>
> >> The best thing that I have come up with is a pre-configure step, where
> >> an object tree gets seeded with a makefile fragment that gets included
> >> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> >> 'CC=' and possibly any other option that gets passed to 'make' as
> >> a variable and has to exist before calling 'make *config'.
> >
> >
> > There is no need to add a hook to include such makefile fragment(s).
> >
> > Quite opposite, you can put your Makefile (in a different filename)
> > that includes the top Makefile.
> >
> >
> > I think this is what people are already doing:
> >
> >
> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> > in this order.
> >
> >
> > So, you can put 'GNUmakefile' with your favorite setups.
> >
> >
> > $ cat GNUmakefile
> > ARCH=arm64
> > CROSS_COMPILE=aarch64-linux-gnu-
> > CC=clang
> > include Makefile
>
> Very weird.
>
> I just tested this and it does not work.
> I did this:
>
> $ cat GNUmakefile
> ARCH = alpha
> CROSS_COMPILE = $(arch-prefix alpha)
> include Makefile
>
> In one of my build directories and the main makefile simply does not see
> the value of ARCH or CROSS_COMPILE I set.  I have confirmed that my
> GNUmakefile is being read, because everything breaks if I remove the
> include line.
>
> Does anyone have any ideas?
>
> Something so we don't have to specify all of these variables on the make
> command line would be nice.

Just including the main Makefile does not work.
That's why I went with the more convoluted solution in
https://lore.kernel.org/linux-kbuild/CAMuHMdXJBqrpzaSNDJgic14ESiHV6cCcb_5E-st6iniXdmm9_g@mail.gmail.com/

Please try the attached, which combines everything above in a single
file, and which works for me. Note that "$(arch-prefix alpha)" didn't
work for me (it resolved to "gcc"?), so I used "alpha-linux-gnu-"
instead.

Good luck!

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

[-- Attachment #2: GNUmakefile --]
[-- Type: application/octet-stream, Size: 257 bytes --]

MAKEARGS = ARCH=alpha CROSS_COMPILE=alpha-linux-gnu-

MAKEFLAGS += --no-print-directory

.PHONY: all $(MAKECMDGOALS)

all	:= $(filter-out all Makefile,$(MAKECMDGOALS))

all:
	@$(MAKE) $(MAKEARGS) $(all) -f Makefile

Makefile:;

$(all): all
	@:

%/: all
	@:

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-27  7:07             ` Geert Uytterhoeven
@ 2021-07-27  7:49               ` Arnd Bergmann
  2021-07-27  7:55                 ` Geert Uytterhoeven
  0 siblings, 1 reply; 39+ messages in thread
From: Arnd Bergmann @ 2021-07-27  7:49 UTC (permalink / raw)
  To: Geert Uytterhoeven
  Cc: Eric W. Biederman, Masahiro Yamada, Linus Torvalds,
	Nick Desaulniers, Miguel Ojeda, Fangrui Song, Michal Marek,
	Linux Kernel Mailing List, Linux Kbuild mailing list,
	clang-built-linux, Christoph Hellwig, Nathan Chancellor

On Tue, Jul 27, 2021 at 9:07 AM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> On Mon, Jul 26, 2021 at 10:27 PM Eric W. Biederman <ebiederm@xmission.com> wrote:
> > Masahiro Yamada <masahiroy@kernel.org> writes:
> > > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> >
> > Does anyone have any ideas?
> >
> > Something so we don't have to specify all of these variables on the make
> > command line would be nice.
>
> Just including the main Makefile does not work.
> That's why I went with the more convoluted solution in
> https://lore.kernel.org/linux-kbuild/CAMuHMdXJBqrpzaSNDJgic14ESiHV6cCcb_5E-st6iniXdmm9_g@mail.gmail.com/

This is roughly what I use as well, but it does have the downside that
it confuses
the inner 'make' when you build multiple targets in parallel, e.g.

make -skj30 kernel/ mm/ init/

works with the normal Makefile, but fails spectacularly with my nested
GNUmakefile because it starts multiple sub-processes that each try to
build the same preparation files (I did not try your version).

What I had in mind was to use something like (but not exactly)

-include $(O)/Makefile.cross

to conditionally include another file from the top-level Makefile before
doing anything else.

          Arnd

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-27  7:49               ` Arnd Bergmann
@ 2021-07-27  7:55                 ` Geert Uytterhoeven
  2021-07-27  8:21                   ` Arnd Bergmann
  0 siblings, 1 reply; 39+ messages in thread
From: Geert Uytterhoeven @ 2021-07-27  7:55 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Eric W. Biederman, Masahiro Yamada, Linus Torvalds,
	Nick Desaulniers, Miguel Ojeda, Fangrui Song, Michal Marek,
	Linux Kernel Mailing List, Linux Kbuild mailing list,
	clang-built-linux, Christoph Hellwig, Nathan Chancellor

Hi Arnd,

On Tue, Jul 27, 2021 at 9:49 AM Arnd Bergmann <arnd@kernel.org> wrote:
> On Tue, Jul 27, 2021 at 9:07 AM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > On Mon, Jul 26, 2021 at 10:27 PM Eric W. Biederman <ebiederm@xmission.com> wrote:
> > > Masahiro Yamada <masahiroy@kernel.org> writes:
> > > > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> > >
> > > Does anyone have any ideas?
> > >
> > > Something so we don't have to specify all of these variables on the make
> > > command line would be nice.
> >
> > Just including the main Makefile does not work.
> > That's why I went with the more convoluted solution in
> > https://lore.kernel.org/linux-kbuild/CAMuHMdXJBqrpzaSNDJgic14ESiHV6cCcb_5E-st6iniXdmm9_g@mail.gmail.com/
>
> This is roughly what I use as well, but it does have the downside that
> it confuses
> the inner 'make' when you build multiple targets in parallel, e.g.
>
> make -skj30 kernel/ mm/ init/
>
> works with the normal Makefile, but fails spectacularly with my nested
> GNUmakefile because it starts multiple sub-processes that each try to
> build the same preparation files (I did not try your version).

Seems to work fine with mine...

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-27  7:55                 ` Geert Uytterhoeven
@ 2021-07-27  8:21                   ` Arnd Bergmann
  0 siblings, 0 replies; 39+ messages in thread
From: Arnd Bergmann @ 2021-07-27  8:21 UTC (permalink / raw)
  To: Geert Uytterhoeven
  Cc: Eric W. Biederman, Masahiro Yamada, Linus Torvalds,
	Nick Desaulniers, Miguel Ojeda, Fangrui Song, Michal Marek,
	Linux Kernel Mailing List, Linux Kbuild mailing list,
	clang-built-linux, Christoph Hellwig, Nathan Chancellor

On Tue, Jul 27, 2021 at 9:55 AM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> On Tue, Jul 27, 2021 at 9:49 AM Arnd Bergmann <arnd@kernel.org> wrote:
> > make -skj30 kernel/ mm/ init/
> >
> > works with the normal Makefile, but fails spectacularly with my nested
> > GNUmakefile because it starts multiple sub-processes that each try to
> > build the same preparation files (I did not try your version).
>
> Seems to work fine with mine...
>

Ok, nevermind. I tried your version and it works for me too. The next
time I get annoyed by this, I'll figure out what I did wrong then;-)

       Arnd

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-26 20:27           ` Eric W. Biederman
  2021-07-27  7:07             ` Geert Uytterhoeven
@ 2021-07-27 10:10             ` Masahiro Yamada
  2021-07-27 14:16               ` Eric W. Biederman
  1 sibling, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-27 10:10 UTC (permalink / raw)
  To: Eric W. Biederman
  Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
	Fangrui Song, Michal Marek, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Tue, Jul 27, 2021 at 5:27 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
>
> Masahiro Yamada <masahiroy@kernel.org> writes:
>
> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> >>
> >> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> >> <torvalds@linux-foundation.org> wrote:
> >> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >> >
> >> > We do most of the other heavy lifting in this area in Kconfig anyway,
> >> > why not add that compiler choice?
> >> >
> >> > Obviously it would be gated by the tests to see which compilers are
> >> > _installed_ (and that they are valid versions), so that it doesn't ask
> >> > stupid things ("do you want gcc or clang" when only one of them is
> >> > installed and/or viable).
> >>
> >> I don't see a good way of making Kconfig options both select the
> >> compiler and defining variables based on the compiler, since that
> >> would mean teaching Kconfig about re-evaluating all compiler
> >> dependent settings whenever the first option changes.
> >>
> >> I do have another idea that I think would work though.
> >>
> >> > Hmm? So then any "LLVM=1" thing would be about the "make config"
> >> > stage, not the actual build stage.
> >> >
> >> > (It has annoyed me for years that if you want to cross-compile, you
> >> > first have to do "make ARCH=xyz config" and then remember to do "make
> >> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> >> > never really cared).
> >>
> >> The best thing that I have come up with is a pre-configure step, where
> >> an object tree gets seeded with a makefile fragment that gets included
> >> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> >> 'CC=' and possibly any other option that gets passed to 'make' as
> >> a variable and has to exist before calling 'make *config'.
> >
> >
> > There is no need to add a hook to include such makefile fragment(s).
> >
> > Quite opposite, you can put your Makefile (in a different filename)
> > that includes the top Makefile.
> >
> >
> > I think this is what people are already doing:
> >
> >
> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> > in this order.
> >
> >
> > So, you can put 'GNUmakefile' with your favorite setups.
> >
> >
> > $ cat GNUmakefile
> > ARCH=arm64
> > CROSS_COMPILE=aarch64-linux-gnu-
> > CC=clang
> > include Makefile
>
> Very weird.
>
> I just tested this and it does not work.
> I did this:
>
> $ cat GNUmakefile
> ARCH = alpha
> CROSS_COMPILE = $(arch-prefix alpha)
> include Makefile
>
> In one of my build directories and the main makefile simply does not see
> the value of ARCH or CROSS_COMPILE I set.  I have confirmed that my
> GNUmakefile is being read, because everything breaks if I remove the
> include line.
>
> Does anyone have any ideas?
>
> Something so we don't have to specify all of these variables on the make
> command line would be nice.
>
> Eric


Worked for me.

Could you tell me the exact steps you did?


This is my case:

My kernel source tree is located at $HOME/ref/linux
alpha tool chains are located at $HOME/tools/alpha-10.1.0/bin



I tried a simple GNUmakefile with 3 lines.

You can see 'make' is building the alpha kernel


Please see below:




masahiro@grover:~/ref/linux$ cat GNUmakefile
ARCH = alpha
CROSS_COMPILE = $(HOME)/tools/alpha-10.1.0/bin/alpha-linux-
include Makefile
masahiro@grover:~/ref/linux$ make mrproper
  CLEAN   arch/alpha/kernel
  CLEAN   certs
  CLEAN   drivers/tty/vt
  CLEAN   drivers/video/logo
  CLEAN   kernel
  CLEAN   lib
  CLEAN   security/selinux
  CLEAN   usr/include
  CLEAN   usr
  CLEAN   vmlinux.symvers modules-only.symvers modules.builtin
modules.builtin.modinfo
  CLEAN   scripts/basic
  CLEAN   scripts/dtc
  CLEAN   scripts/genksyms
  CLEAN   scripts/kconfig
  CLEAN   scripts/mod
  CLEAN   scripts/selinux/genheaders
  CLEAN   scripts/selinux/mdp
  CLEAN   scripts
  CLEAN   include/config include/generated
arch/alpha/include/generated .config .config.old .version
Module.symvers
masahiro@grover:~/ref/linux$ make defconfig
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  YACC    scripts/kconfig/parser.tab.[ch]
  HOSTCC  scripts/kconfig/lexer.lex.o
  HOSTCC  scripts/kconfig/menu.o
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTCC  scripts/kconfig/util.o
  HOSTLD  scripts/kconfig/conf
*** Default configuration is based on 'defconfig'
#
# configuration written to .config
#
masahiro@grover:~/ref/linux$ make
  SYSHDR  arch/alpha/include/generated/uapi/asm/unistd_32.h
  SYSTBL  arch/alpha/include/generated/asm/syscall_table.h
  HOSTCC  scripts/kallsyms
  WRAP    arch/alpha/include/generated/uapi/asm/bpf_perf_event.h
  WRAP    arch/alpha/include/generated/uapi/asm/ipcbuf.h
  WRAP    arch/alpha/include/generated/uapi/asm/msgbuf.h
  WRAP    arch/alpha/include/generated/uapi/asm/poll.h
  WRAP    arch/alpha/include/generated/uapi/asm/sembuf.h
  WRAP    arch/alpha/include/generated/uapi/asm/shmbuf.h
  WRAP    arch/alpha/include/generated/asm/export.h
  WRAP    arch/alpha/include/generated/asm/kvm_para.h
  WRAP    arch/alpha/include/generated/asm/mcs_spinlock.h
  WRAP    arch/alpha/include/generated/asm/compat.h
  WRAP    arch/alpha/include/generated/asm/current.h
  WRAP    arch/alpha/include/generated/asm/exec.h
  WRAP    arch/alpha/include/generated/asm/fb.h
  WRAP    arch/alpha/include/generated/asm/irq_work.h
  WRAP    arch/alpha/include/generated/asm/kmap_size.h
  WRAP    arch/alpha/include/generated/asm/kprobes.h
  WRAP    arch/alpha/include/generated/asm/local64.h
  WRAP    arch/alpha/include/generated/asm/mmiowb.h
  WRAP    arch/alpha/include/generated/asm/module.lds.h
  WRAP    arch/alpha/include/generated/asm/msi.h
  WRAP    arch/alpha/include/generated/asm/preempt.h
  WRAP    arch/alpha/include/generated/asm/sections.h
  WRAP    arch/alpha/include/generated/asm/simd.h
  WRAP    arch/alpha/include/generated/asm/softirq_stack.h
  WRAP    arch/alpha/include/generated/asm/trace_clock.h
  WRAP    arch/alpha/include/generated/asm/unaligned.h
  WRAP    arch/alpha/include/generated/asm/vermagic.h
  UPD     include/config/kernel.release
  UPD     include/generated/uapi/linux/version.h
  UPD     include/generated/utsrelease.h
  CC      scripts/mod/empty.o
  HOSTCC  scripts/mod/mk_elfconfig
  MKELF   scripts/mod/elfconfig.h
  HOSTCC  scripts/mod/modpost.o
  CC      scripts/mod/devicetable-offsets.s
  UPD     scripts/mod/devicetable-offsets.h
  HOSTCC  scripts/mod/file2alias.o
  HOSTCC  scripts/mod/sumversion.o
  HOSTLD  scripts/mod/modpost
  CC      kernel/bounds.s
  UPD     include/generated/bounds.h
  UPD     include/generated/timeconst.h
  CC      arch/alpha/kernel/asm-offsets.s
  UPD     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
<stdin>:1515:2: warning: #warning syscall clone3 not implemented [-Wcpp]
  CALL    scripts/atomic/check-atomics.sh
  CC      init/main.o
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  CC      init/do_mounts.o
  CC      init/noinitramfs.o
  CC      init/calibrate.o
  CC      init/init_task.o
  AR      init/built-in.a
  AR      usr/built-in.a
  AS      arch/alpha/kernel/head.o
  LDS     arch/alpha/kernel/vmlinux.lds
  AS      arch/alpha/kernel/entry.o
  CC      arch/alpha/kernel/traps.o
  CC      arch/alpha/kernel/process.o
  CC      arch/alpha/kernel/osf_sys.o
  CC      arch/alpha/kernel/irq.o
  CC      arch/alpha/kernel/irq_alpha.o
  CC      arch/alpha/kernel/signal.o
  CC      arch/alpha/kernel/setup.o
  CC      arch/alpha/kernel/ptrace.o
  CC      arch/alpha/kernel/time.o
  AS      arch/alpha/kernel/systbls.o
  CC      arch/alpha/kernel/err_common.o
  CC      arch/alpha/kernel/io.o
  CC      arch/alpha/kernel/bugs.o
  CC      arch/alpha/kernel/console.o
  CC      arch/alpha/kernel/pci.o
  CC      arch/alpha/kernel/pci_iommu.o
  CC      arch/alpha/kernel/pci-sysfs.o
  CC      arch/alpha/kernel/module.o
  CC      arch/alpha/kernel/rtc.o
  CC      arch/alpha/kernel/core_apecs.o
  CC      arch/alpha/kernel/core_cia.o
  CC      arch/alpha/kernel/core_irongate.o
  CC      arch/alpha/kernel/core_lca.o
  CC      arch/alpha/kernel/core_mcpcia.o
  CC      arch/alpha/kernel/core_polaris.o
  CC      arch/alpha/kernel/core_t2.o
  CC      arch/alpha/kernel/core_tsunami.o
  CC      arch/alpha/kernel/sys_alcor.o
  CC      arch/alpha/kernel/sys_cabriolet.o
  CC      arch/alpha/kernel/sys_dp264.o
  CC      arch/alpha/kernel/sys_eb64p.o
  CC      arch/alpha/kernel/sys_eiger.o
  CC      arch/alpha/kernel/sys_jensen.o
  CC      arch/alpha/kernel/sys_miata.o
  CC      arch/alpha/kernel/sys_mikasa.o
  CC      arch/alpha/kernel/sys_nautilus.o
  CC      arch/alpha/kernel/sys_noritake.o
  CC      arch/alpha/kernel/sys_rawhide.o
  CC      arch/alpha/kernel/sys_ruffian.o
  CC      arch/alpha/kernel/sys_rx164.o
  CC      arch/alpha/kernel/sys_sable.o
  CC      arch/alpha/kernel/sys_sio.o
  CC      arch/alpha/kernel/sys_sx164.o
  CC      arch/alpha/kernel/sys_takara.o
  CC      arch/alpha/kernel/irq_pyxis.o
  CC      arch/alpha/kernel/irq_i8259.o
  CC      arch/alpha/kernel/irq_srm.o
  CC      arch/alpha/kernel/err_ev6.o
  CC      arch/alpha/kernel/es1888.o
  CC      arch/alpha/kernel/smc37c669.o
  CC      arch/alpha/kernel/smc37c93x.o
  CC      arch/alpha/kernel/pc873xx.o
  CC      arch/alpha/kernel/gct.o
  CC      arch/alpha/kernel/srmcons.o
  AR      arch/alpha/kernel/built-in.a
  CC [M]  arch/alpha/kernel/srm_env.o
  CC      arch/alpha/mm/init.o
  CC      arch/alpha/mm/fault.o
  AR      arch/alpha/mm/built-in.a
  CC      arch/alpha/math-emu/math.o
^Cmake[2]: *** Deleting file 'arch/alpha/math-emu/math.o'
make[2]: *** [scripts/Makefile.build:271: arch/alpha/math-emu/math.o] Interrupt
make[1]: *** [scripts/Makefile.build:514: arch/alpha/math-emu] Interrupt
make: *** [Makefile:1842: arch/alpha] Interrupt





-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-27 10:10             ` Masahiro Yamada
@ 2021-07-27 14:16               ` Eric W. Biederman
  2021-07-27 15:45                 ` Masahiro Yamada
  0 siblings, 1 reply; 39+ messages in thread
From: Eric W. Biederman @ 2021-07-27 14:16 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
	Fangrui Song, Michal Marek, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

Masahiro Yamada <masahiroy@kernel.org> writes:

> On Tue, Jul 27, 2021 at 5:27 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
>>
>> Masahiro Yamada <masahiroy@kernel.org> writes:
>>
>> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
>> >>
>> >> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
>> >> <torvalds@linux-foundation.org> wrote:
>> >> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>> >> >
>> >> > We do most of the other heavy lifting in this area in Kconfig anyway,
>> >> > why not add that compiler choice?
>> >> >
>> >> > Obviously it would be gated by the tests to see which compilers are
>> >> > _installed_ (and that they are valid versions), so that it doesn't ask
>> >> > stupid things ("do you want gcc or clang" when only one of them is
>> >> > installed and/or viable).
>> >>
>> >> I don't see a good way of making Kconfig options both select the
>> >> compiler and defining variables based on the compiler, since that
>> >> would mean teaching Kconfig about re-evaluating all compiler
>> >> dependent settings whenever the first option changes.
>> >>
>> >> I do have another idea that I think would work though.
>> >>
>> >> > Hmm? So then any "LLVM=1" thing would be about the "make config"
>> >> > stage, not the actual build stage.
>> >> >
>> >> > (It has annoyed me for years that if you want to cross-compile, you
>> >> > first have to do "make ARCH=xyz config" and then remember to do "make
>> >> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
>> >> > never really cared).
>> >>
>> >> The best thing that I have come up with is a pre-configure step, where
>> >> an object tree gets seeded with a makefile fragment that gets included
>> >> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
>> >> 'CC=' and possibly any other option that gets passed to 'make' as
>> >> a variable and has to exist before calling 'make *config'.
>> >
>> >
>> > There is no need to add a hook to include such makefile fragment(s).
>> >
>> > Quite opposite, you can put your Makefile (in a different filename)
>> > that includes the top Makefile.
>> >
>> >
>> > I think this is what people are already doing:
>> >
>> >
>> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
>> > in this order.
>> >
>> >
>> > So, you can put 'GNUmakefile' with your favorite setups.
>> >
>> >
>> > $ cat GNUmakefile
>> > ARCH=arm64
>> > CROSS_COMPILE=aarch64-linux-gnu-
>> > CC=clang
>> > include Makefile
>>
>> Very weird.
>>
>> I just tested this and it does not work.
>> I did this:
>>
>> $ cat GNUmakefile
>> ARCH = alpha
>> CROSS_COMPILE = $(arch-prefix alpha)
>> include Makefile
>>
>> In one of my build directories and the main makefile simply does not see
>> the value of ARCH or CROSS_COMPILE I set.  I have confirmed that my
>> GNUmakefile is being read, because everything breaks if I remove the
>> include line.
>>
>> Does anyone have any ideas?
>>
>> Something so we don't have to specify all of these variables on the make
>> command line would be nice.
>>
>> Eric
>
>
> Worked for me.
>
> Could you tell me the exact steps you did?
>
>
> This is my case:
>
> My kernel source tree is located at $HOME/ref/linux
> alpha tool chains are located at $HOME/tools/alpha-10.1.0/bin
>
>
>
> I tried a simple GNUmakefile with 3 lines.
>
> You can see 'make' is building the alpha kernel
>
>
> Please see below:

Interesting.  That appears to work if I don't specify a build directory.
Once I specify a build directory with O= it does not work.

When I am working on a change that affects multiple architectures
I really want a build directory that is not my source tree so I can
test small changes on multiple architectures without needing to rebuild
everything.

Eric

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-27 14:16               ` Eric W. Biederman
@ 2021-07-27 15:45                 ` Masahiro Yamada
  2021-07-27 18:46                   ` Eric W. Biederman
  0 siblings, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-27 15:45 UTC (permalink / raw)
  To: Eric W. Biederman
  Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
	Fangrui Song, Michal Marek, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Tue, Jul 27, 2021 at 11:16 PM Eric W. Biederman
<ebiederm@xmission.com> wrote:
>
> Masahiro Yamada <masahiroy@kernel.org> writes:
>
> > On Tue, Jul 27, 2021 at 5:27 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
> >>
> >> Masahiro Yamada <masahiroy@kernel.org> writes:
> >>
> >> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> >> >>
> >> >> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> >> >> <torvalds@linux-foundation.org> wrote:
> >> >> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >> >> >
> >> >> > We do most of the other heavy lifting in this area in Kconfig anyway,
> >> >> > why not add that compiler choice?
> >> >> >
> >> >> > Obviously it would be gated by the tests to see which compilers are
> >> >> > _installed_ (and that they are valid versions), so that it doesn't ask
> >> >> > stupid things ("do you want gcc or clang" when only one of them is
> >> >> > installed and/or viable).
> >> >>
> >> >> I don't see a good way of making Kconfig options both select the
> >> >> compiler and defining variables based on the compiler, since that
> >> >> would mean teaching Kconfig about re-evaluating all compiler
> >> >> dependent settings whenever the first option changes.
> >> >>
> >> >> I do have another idea that I think would work though.
> >> >>
> >> >> > Hmm? So then any "LLVM=1" thing would be about the "make config"
> >> >> > stage, not the actual build stage.
> >> >> >
> >> >> > (It has annoyed me for years that if you want to cross-compile, you
> >> >> > first have to do "make ARCH=xyz config" and then remember to do "make
> >> >> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> >> >> > never really cared).
> >> >>
> >> >> The best thing that I have come up with is a pre-configure step, where
> >> >> an object tree gets seeded with a makefile fragment that gets included
> >> >> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> >> >> 'CC=' and possibly any other option that gets passed to 'make' as
> >> >> a variable and has to exist before calling 'make *config'.
> >> >
> >> >
> >> > There is no need to add a hook to include such makefile fragment(s).
> >> >
> >> > Quite opposite, you can put your Makefile (in a different filename)
> >> > that includes the top Makefile.
> >> >
> >> >
> >> > I think this is what people are already doing:
> >> >
> >> >
> >> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> >> > in this order.
> >> >
> >> >
> >> > So, you can put 'GNUmakefile' with your favorite setups.
> >> >
> >> >
> >> > $ cat GNUmakefile
> >> > ARCH=arm64
> >> > CROSS_COMPILE=aarch64-linux-gnu-
> >> > CC=clang
> >> > include Makefile
> >>
> >> Very weird.
> >>
> >> I just tested this and it does not work.
> >> I did this:
> >>
> >> $ cat GNUmakefile
> >> ARCH = alpha
> >> CROSS_COMPILE = $(arch-prefix alpha)
> >> include Makefile
> >>
> >> In one of my build directories and the main makefile simply does not see
> >> the value of ARCH or CROSS_COMPILE I set.  I have confirmed that my
> >> GNUmakefile is being read, because everything breaks if I remove the
> >> include line.
> >>
> >> Does anyone have any ideas?
> >>
> >> Something so we don't have to specify all of these variables on the make
> >> command line would be nice.
> >>
> >> Eric
> >
> >
> > Worked for me.
> >
> > Could you tell me the exact steps you did?
> >
> >
> > This is my case:
> >
> > My kernel source tree is located at $HOME/ref/linux
> > alpha tool chains are located at $HOME/tools/alpha-10.1.0/bin
> >
> >
> >
> > I tried a simple GNUmakefile with 3 lines.
> >
> > You can see 'make' is building the alpha kernel
> >
> >
> > Please see below:
>
> Interesting.  That appears to work if I don't specify a build directory.
> Once I specify a build directory with O= it does not work.
>
> When I am working on a change that affects multiple architectures
> I really want a build directory that is not my source tree so I can
> test small changes on multiple architectures without needing to rebuild
> everything.
>
> Eric


Ah, OK.
When you specify O= option, Kbuild will
invoke sub-make.
So you need to add 'export' to the
variables.

Please see the following modified version.





masahiro@grover:~/ref/linux$ rm -rf build-dir/
masahiro@grover:~/ref/linux$ cat GNUmakefile
export ARCH = alpha
export CROSS_COMPILE = $(HOME)/tools/alpha-10.1.0/bin/alpha-linux-
include Makefile
masahiro@grover:~/ref/linux$ make O=build-dir defconfig
make[1]: Entering directory '/home/masahiro/ref/linux/build-dir'
  GEN     Makefile
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  YACC    scripts/kconfig/parser.tab.[ch]
  HOSTCC  scripts/kconfig/lexer.lex.o
  HOSTCC  scripts/kconfig/menu.o
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTCC  scripts/kconfig/util.o
  HOSTLD  scripts/kconfig/conf
*** Default configuration is based on 'defconfig'
#
# configuration written to .config
#
make[1]: Leaving directory '/home/masahiro/ref/linux/build-dir'
masahiro@grover:~/ref/linux$ make O=build-dir
make[1]: Entering directory '/home/masahiro/ref/linux/build-dir'
  GEN     Makefile
  SYSHDR  arch/alpha/include/generated/uapi/asm/unistd_32.h
  SYSTBL  arch/alpha/include/generated/asm/syscall_table.h
  HOSTCC  scripts/kallsyms
  WRAP    arch/alpha/include/generated/uapi/asm/bpf_perf_event.h
  WRAP    arch/alpha/include/generated/uapi/asm/ipcbuf.h
  WRAP    arch/alpha/include/generated/uapi/asm/msgbuf.h
  WRAP    arch/alpha/include/generated/uapi/asm/poll.h
  WRAP    arch/alpha/include/generated/uapi/asm/sembuf.h
  WRAP    arch/alpha/include/generated/uapi/asm/shmbuf.h
  WRAP    arch/alpha/include/generated/asm/export.h
  WRAP    arch/alpha/include/generated/asm/kvm_para.h
  WRAP    arch/alpha/include/generated/asm/mcs_spinlock.h
  WRAP    arch/alpha/include/generated/asm/compat.h
  WRAP    arch/alpha/include/generated/asm/current.h
  WRAP    arch/alpha/include/generated/asm/exec.h
  WRAP    arch/alpha/include/generated/asm/fb.h
  WRAP    arch/alpha/include/generated/asm/irq_work.h
  WRAP    arch/alpha/include/generated/asm/kmap_size.h
  WRAP    arch/alpha/include/generated/asm/kprobes.h
  WRAP    arch/alpha/include/generated/asm/local64.h
  WRAP    arch/alpha/include/generated/asm/mmiowb.h
  WRAP    arch/alpha/include/generated/asm/module.lds.h
  WRAP    arch/alpha/include/generated/asm/msi.h
  WRAP    arch/alpha/include/generated/asm/preempt.h
  WRAP    arch/alpha/include/generated/asm/sections.h
  WRAP    arch/alpha/include/generated/asm/simd.h
  WRAP    arch/alpha/include/generated/asm/softirq_stack.h
  WRAP    arch/alpha/include/generated/asm/trace_clock.h
  WRAP    arch/alpha/include/generated/asm/unaligned.h
  WRAP    arch/alpha/include/generated/asm/vermagic.h
  UPD     include/config/kernel.release
  UPD     include/generated/uapi/linux/version.h
  UPD     include/generated/utsrelease.h
  CC      scripts/mod/empty.o
  HOSTCC  scripts/mod/mk_elfconfig
  MKELF   scripts/mod/elfconfig.h
  HOSTCC  scripts/mod/modpost.o
  CC      scripts/mod/devicetable-offsets.s
  UPD     scripts/mod/devicetable-offsets.h
  HOSTCC  scripts/mod/file2alias.o
  HOSTCC  scripts/mod/sumversion.o
  HOSTLD  scripts/mod/modpost
  CC      kernel/bounds.s
  UPD     include/generated/bounds.h
  UPD     include/generated/timeconst.h
  CC      arch/alpha/kernel/asm-offsets.s
  UPD     include/generated/asm-offsets.h
  CALL    ../scripts/checksyscalls.sh
<stdin>:1515:2: warning: #warning syscall clone3 not implemented [-Wcpp]
  CALL    ../scripts/atomic/check-atomics.sh
  CC      init/main.o
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  CC      init/do_mounts.o
  CC      init/noinitramfs.o
  CC      init/calibrate.o
  CC      init/init_task.o
  AR      init/built-in.a
  AR      usr/built-in.a
  AS      arch/alpha/kernel/head.o
  LDS     arch/alpha/kernel/vmlinux.lds
  AS      arch/alpha/kernel/entry.o
  CC      arch/alpha/kernel/traps.o
  CC      arch/alpha/kernel/process.o
  CC      arch/alpha/kernel/osf_sys.o
  CC      arch/alpha/kernel/irq.o
  CC      arch/alpha/kernel/irq_alpha.o
  CC      arch/alpha/kernel/signal.o
  CC      arch/alpha/kernel/setup.o
  CC      arch/alpha/kernel/ptrace.o
  CC      arch/alpha/kernel/time.o
  AS      arch/alpha/kernel/systbls.o
  CC      arch/alpha/kernel/err_common.o
  CC      arch/alpha/kernel/io.o
  CC      arch/alpha/kernel/bugs.o
  CC      arch/alpha/kernel/console.o
  CC      arch/alpha/kernel/pci.o
  CC      arch/alpha/kernel/pci_iommu.o
  CC      arch/alpha/kernel/pci-sysfs.o
  CC      arch/alpha/kernel/module.o
  CC      arch/alpha/kernel/rtc.o
  CC      arch/alpha/kernel/core_apecs.o
  CC      arch/alpha/kernel/core_cia.o
  CC      arch/alpha/kernel/core_irongate.o
  CC      arch/alpha/kernel/core_lca.o
  CC      arch/alpha/kernel/core_mcpcia.o
  CC      arch/alpha/kernel/core_polaris.o
  CC      arch/alpha/kernel/core_t2.o
  CC      arch/alpha/kernel/core_tsunami.o
  CC      arch/alpha/kernel/sys_alcor.o
  CC      arch/alpha/kernel/sys_cabriolet.o
  CC      arch/alpha/kernel/sys_dp264.o
  CC      arch/alpha/kernel/sys_eb64p.o
  CC      arch/alpha/kernel/sys_eiger.o
  CC      arch/alpha/kernel/sys_jensen.o
  CC      arch/alpha/kernel/sys_miata.o
  CC      arch/alpha/kernel/sys_mikasa.o
  CC      arch/alpha/kernel/sys_nautilus.o
  CC      arch/alpha/kernel/sys_noritake.o
  CC      arch/alpha/kernel/sys_rawhide.o
  CC      arch/alpha/kernel/sys_ruffian.o
  CC      arch/alpha/kernel/sys_rx164.o
  CC      arch/alpha/kernel/sys_sable.o
  CC      arch/alpha/kernel/sys_sio.o
  CC      arch/alpha/kernel/sys_sx164.o
  CC      arch/alpha/kernel/sys_takara.o
  CC      arch/alpha/kernel/irq_pyxis.o
  CC      arch/alpha/kernel/irq_i8259.o
  CC      arch/alpha/kernel/irq_srm.o
  CC      arch/alpha/kernel/err_ev6.o
  CC      arch/alpha/kernel/es1888.o
  CC      arch/alpha/kernel/smc37c669.o
  CC      arch/alpha/kernel/smc37c93x.o
  CC      arch/alpha/kernel/pc873xx.o
  CC      arch/alpha/kernel/gct.o
  CC      arch/alpha/kernel/srmcons.o
  AR      arch/alpha/kernel/built-in.a
  CC [M]  arch/alpha/kernel/srm_env.o
  CC      arch/alpha/mm/init.o
  CC      arch/alpha/mm/fault.o
  AR      arch/alpha/mm/built-in.a
  CC      arch/alpha/math-emu/math.o
  AS      arch/alpha/math-emu/qrnnd.o
  AR      arch/alpha/math-emu/built-in.a
  AR      arch/alpha/built-in.a
  CC      kernel/sched/core.o
  CC      kernel/sched/loadavg.o
  CC      kernel/sched/clock.o
  CC      kernel/sched/cputime.o
  CC      kernel/sched/idle.o
  CC      kernel/sched/fair.o
  CC      kernel/sched/rt.o
  CC      kernel/sched/deadline.o
  CC      kernel/sched/wait.o
  CC      kernel/sched/wait_bit.o
  CC      kernel/sched/swait.o
  CC      kernel/sched/completion.o
  CC      kernel/sched/debug.o
  CC      kernel/sched/membarrier.o
  AR      kernel/sched/built-in.a
  CC      kernel/locking/mutex.o
  CC      kernel/locking/semaphore.o
  CC      kernel/locking/rwsem.o
  CC      kernel/locking/percpu-rwsem.o
  CC      kernel/locking/rtmutex.o
  AR      kernel/locking/built-in.a
  CC      kernel/power/qos.o
  CC      kernel/power/poweroff.o
  AR      kernel/power/built-in.a
  CC      kernel/printk/printk.o
  CC      kernel/printk/printk_safe.o
  CC      kernel/printk/printk_ringbuffer.o
  AR      kernel/printk/built-in.a
  CC      kernel/irq/irqdesc.o
  CC      kernel/irq/handle.o
  CC      kernel/irq/manage.o
  CC      kernel/irq/spurious.o
  CC      kernel/irq/resend.o
  CC      kernel/irq/chip.o
  CC      kernel/irq/dummychip.o
  CC      kernel/irq/devres.o
  CC      kernel/irq/autoprobe.o
  CC      kernel/irq/proc.o
  AR      kernel/irq/built-in.a
  CC      kernel/rcu/update.o
  CC      kernel/rcu/sync.o
  CC      kernel/rcu/srcutiny.o
  CC      kernel/rcu/tiny.o
  AR      kernel/rcu/built-in.a
  AR      kernel/livepatch/built-in.a
  CC      kernel/dma/mapping.o
  CC      kernel/dma/direct.o
  CC      kernel/dma/ops_helpers.o
  CC      kernel/dma/dummy.o
  AR      kernel/dma/built-in.a
  AR      kernel/entry/built-in.a
  CC      kernel/time/time.o
  CC      kernel/time/timer.o
  CC      kernel/time/hrtimer.o
  CC      kernel/time/timekeeping.o
  CC      kernel/time/ntp.o
  CC      kernel/time/clocksource.o
  CC      kernel/time/jiffies.o
  CC      kernel/time/timer_list.o
  CC      kernel/time/timeconv.o
  CC      kernel/time/timecounter.o
  CC      kernel/time/alarmtimer.o
  CC      kernel/time/posix-timers.o
  CC      kernel/time/posix-cpu-timers.o
  CC      kernel/time/posix-clock.o
  CC      kernel/time/itimer.o
  CC      kernel/time/clockevents.o
  CC      kernel/time/tick-common.o
  AR      kernel/time/built-in.a
  CC      kernel/bpf/core.o
  AR      kernel/bpf/built-in.a




-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-27 15:45                 ` Masahiro Yamada
@ 2021-07-27 18:46                   ` Eric W. Biederman
  2021-07-28 22:31                     ` Masahiro Yamada
  0 siblings, 1 reply; 39+ messages in thread
From: Eric W. Biederman @ 2021-07-27 18:46 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
	Fangrui Song, Michal Marek, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

Masahiro Yamada <masahiroy@kernel.org> writes:

> On Tue, Jul 27, 2021 at 11:16 PM Eric W. Biederman
> <ebiederm@xmission.com> wrote:
>>
>> Masahiro Yamada <masahiroy@kernel.org> writes:
>>
>> > On Tue, Jul 27, 2021 at 5:27 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
>> >>
>> >> Masahiro Yamada <masahiroy@kernel.org> writes:
>> >>
>> >> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
>> >> >>
>> >> >> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
>> >> >> <torvalds@linux-foundation.org> wrote:
>> >> >> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>> >> >> >
>> >> >> > We do most of the other heavy lifting in this area in Kconfig anyway,
>> >> >> > why not add that compiler choice?
>> >> >> >
>> >> >> > Obviously it would be gated by the tests to see which compilers are
>> >> >> > _installed_ (and that they are valid versions), so that it doesn't ask
>> >> >> > stupid things ("do you want gcc or clang" when only one of them is
>> >> >> > installed and/or viable).
>> >> >>
>> >> >> I don't see a good way of making Kconfig options both select the
>> >> >> compiler and defining variables based on the compiler, since that
>> >> >> would mean teaching Kconfig about re-evaluating all compiler
>> >> >> dependent settings whenever the first option changes.
>> >> >>
>> >> >> I do have another idea that I think would work though.
>> >> >>
>> >> >> > Hmm? So then any "LLVM=1" thing would be about the "make config"
>> >> >> > stage, not the actual build stage.
>> >> >> >
>> >> >> > (It has annoyed me for years that if you want to cross-compile, you
>> >> >> > first have to do "make ARCH=xyz config" and then remember to do "make
>> >> >> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
>> >> >> > never really cared).
>> >> >>
>> >> >> The best thing that I have come up with is a pre-configure step, where
>> >> >> an object tree gets seeded with a makefile fragment that gets included
>> >> >> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
>> >> >> 'CC=' and possibly any other option that gets passed to 'make' as
>> >> >> a variable and has to exist before calling 'make *config'.
>> >> >
>> >> >
>> >> > There is no need to add a hook to include such makefile fragment(s).
>> >> >
>> >> > Quite opposite, you can put your Makefile (in a different filename)
>> >> > that includes the top Makefile.
>> >> >
>> >> >
>> >> > I think this is what people are already doing:
>> >> >
>> >> >
>> >> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
>> >> > in this order.
>> >> >
>> >> >
>> >> > So, you can put 'GNUmakefile' with your favorite setups.
>> >> >
>> >> >
>> >> > $ cat GNUmakefile
>> >> > ARCH=arm64
>> >> > CROSS_COMPILE=aarch64-linux-gnu-
>> >> > CC=clang
>> >> > include Makefile
>> >>
>> >> Very weird.
>> >>
>> >> I just tested this and it does not work.
>> >> I did this:
>> >>
>> >> $ cat GNUmakefile
>> >> ARCH = alpha
>> >> CROSS_COMPILE = $(arch-prefix alpha)
>> >> include Makefile
>> >>
>> >> In one of my build directories and the main makefile simply does not see
>> >> the value of ARCH or CROSS_COMPILE I set.  I have confirmed that my
>> >> GNUmakefile is being read, because everything breaks if I remove the
>> >> include line.
>> >>
>> >> Does anyone have any ideas?
>> >>
>> >> Something so we don't have to specify all of these variables on the make
>> >> command line would be nice.
>> >>
>> >> Eric
>> >
>> >
>> > Worked for me.
>> >
>> > Could you tell me the exact steps you did?
>> >
>> >
>> > This is my case:
>> >
>> > My kernel source tree is located at $HOME/ref/linux
>> > alpha tool chains are located at $HOME/tools/alpha-10.1.0/bin
>> >
>> >
>> >
>> > I tried a simple GNUmakefile with 3 lines.
>> >
>> > You can see 'make' is building the alpha kernel
>> >
>> >
>> > Please see below:
>>
>> Interesting.  That appears to work if I don't specify a build directory.
>> Once I specify a build directory with O= it does not work.
>>
>> When I am working on a change that affects multiple architectures
>> I really want a build directory that is not my source tree so I can
>> test small changes on multiple architectures without needing to rebuild
>> everything.
>>
>> Eric
>
>
> Ah, OK.
> When you specify O= option, Kbuild will
> invoke sub-make.
> So you need to add 'export' to the
> variables.
>
> Please see the following modified version.

Yes. That does seem to work and be simpler.

Is there any chance we can get the command that generates the Makefile
in the build directory to include variables like ARCH and CROSS_COMPILE
in the Makefile as well?  Just so they don't need to be set by hand to
the same value every time?

Eric

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-21  3:49       ` Masahiro Yamada
@ 2021-07-28 18:59         ` Nick Desaulniers
  2021-07-28 22:35           ` Masahiro Yamada
  0 siblings, 1 reply; 39+ messages in thread
From: Nick Desaulniers @ 2021-07-28 18:59 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Nathan Chancellor, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Linus Torvalds

On Tue, Jul 20, 2021 at 8:50 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Wed, Jul 21, 2021 at 2:30 AM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > On 7/20/2021 1:04 AM, Masahiro Yamada wrote:
> > > On Fri, Jul 9, 2021 at 8:25 AM 'Nick Desaulniers' via Clang Built
> > > Linux <clang-built-linux@googlegroups.com> wrote:
> > >>
> > >> diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
> > >> index 297932e973d4..956603f56724 100644
> > >> --- a/scripts/Makefile.clang
> > >> +++ b/scripts/Makefile.clang
> > >> @@ -1,6 +1,36 @@
> > >> -ifneq ($(CROSS_COMPILE),)
> > >> +# Individual arch/{arch}/Makfiles should use -EL/-EB to set intended endianness
> > >> +# and -m32/-m64 to set word size based on Kconfigs instead of relying on the
> > >> +# target triple.
> > >> +ifeq ($(CROSS_COMPILE),)
> > >> +ifneq ($(LLVM),)
> > >
> > >
> > > Do you need to check $(LLVM) ?
> > >
> > >
> > > LLVM=1 is a convenient switch to change all the
> > > defaults, but yet you can flip each tool individually.
> > >
> > > Instead of LLVM=1, you still should be able to
> > > get the equivalent setups by:
> > >
> > >
> > >    make CC=clang LD=ld.lld AR=llvm-ar OBJCOPY=llvm-objcopy ...
> > >
> > >
> > > The --target option is passed to only
> > > KBUILD_CFLAGS and KBUILD_AFLAGS.
> > >
> > > So, when we talk about --target=,
> > > we only care about whether $(CC) is Clang.
> > > Not caring about $(AR), $(LD), or $(OBJCOPY).
> > >
> > >
> > > scripts/Makefile.clang is already guarded by:
> > >
> > > ifneq ($(findstring clang,$(CC_VERSION_TEXT)),
> >
> > $ make ARCH=arm64 CC=clang LLVM_IAS=1
> >
> > will use the right compiler and assembler but none of the other binary
> > tools because '--prefix=' will not be set so CROSS_COMPILE needs to be
> > specified still, which defeats the purpose of this whole change. This
> > patch is designed to work for the "normal" case of saying "I want to use
> > all of the LLVM tools", not "I want to use clang by itself".
>
>
> I disagree.
>
> LLVM=1 is a shorthand.
>
>
>
> make LLVM=1 LLVM_IAS=1
>
>   should be equivalent to:
>
> make CC=clang LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \
>   OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump READELF=llvm-readelf \
>   HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar HOSTLD=ld.lld \
>   LLVM_IAS=1
>
>
>
> We do not care about the origin of CC=clang,
> whether it came from LLVM=1 or every tool was explicitly,
> individually specified.
>
>
>
> ifneq ($(LLVM),) is a garbage code
> that checks a pointless thing.

Masahiro,
Nathan is correct.  Test for yourself; if you apply these two patches,
then apply:

diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
index 956603f56724..a1b46811bdc6 100644
--- a/scripts/Makefile.clang
+++ b/scripts/Makefile.clang
@@ -2,7 +2,6 @@
 # and -m32/-m64 to set word size based on Kconfigs instead of relying on the
 # target triple.
 ifeq ($(CROSS_COMPILE),)
-ifneq ($(LLVM),)
 ifeq ($(LLVM_IAS),1)
 ifeq ($(SRCARCH),arm)
 CLANG_FLAGS    += --target=arm-linux-gnueabi
@@ -26,7 +25,6 @@ else
 $(error Specify CROSS_COMPILE or add '--target=' option to
scripts/Makefile.clang)
 endif # SRCARCH
 endif # LLVM_IAS
-endif # LLVM
 else
 CLANG_FLAGS    += --target=$(notdir $(CROSS_COMPILE:%-=%))
 endif # CROSS_COMPILE

Then build as Nathan specified:
$ ARCH=arm64 make CC=clang LLVM_IAS=1 -j72 defconfig all
...
arch/arm64/Makefile:25: ld does not support --fix-cortex-a53-843419;
kernel may be susceptible to erratum
...
  LD      arch/arm64/kernel/vdso/vdso.so.dbg
ld: unrecognised emulation mode: aarch64linux
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu
elf_l1om elf_k1om i386pep i386pe
make[1]: *** [arch/arm64/kernel/vdso/Makefile:56:
arch/arm64/kernel/vdso/vdso.so.dbg] Error 1
make: *** [arch/arm64/Makefile:193: vdso_prepare] Error 2

Nathan referred to --prefix, but in this failure, because
CROSS_COMPILE was never set, the top level Makefile set LD to:
 452 LD    = $(CROSS_COMPILE)ld
in this case `ld` in my path was my host x86 linker, which is not
correct for a cross compilation of arm64 target.

Perhaps we can somehow support "implicit CROSS_COMPILE" with just
CC=clang, and not LLVM=1, but I think it would be inflexible to
hardcode such target triple prefixes.  What if someone has
arm-linux-gnueabi-as but not arm-linux-gnueabihf-as installed?  That's
the point of CROSS_COMPILE in my opinion to provide such flexibility
at the cost of additional command line verbosity.

For the common case of LLVM=1 though, this series is a simplification.
If users want to specify CC=clang, then they MUST use CROSS_COMPILE
when cross compiling.

Please review the current approach and see if there's more I can
improve in a v3; otherwise I still think this series is good to go.
-- 
Thanks,
~Nick Desaulniers

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-27 18:46                   ` Eric W. Biederman
@ 2021-07-28 22:31                     ` Masahiro Yamada
  0 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-28 22:31 UTC (permalink / raw)
  To: Eric W. Biederman
  Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
	Fangrui Song, Michal Marek, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Nathan Chancellor

On Wed, Jul 28, 2021 at 3:46 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
>
> Masahiro Yamada <masahiroy@kernel.org> writes:
>
> > On Tue, Jul 27, 2021 at 11:16 PM Eric W. Biederman
> > <ebiederm@xmission.com> wrote:
> >>
> >> Masahiro Yamada <masahiroy@kernel.org> writes:
> >>
> >> > On Tue, Jul 27, 2021 at 5:27 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
> >> >>
> >> >> Masahiro Yamada <masahiroy@kernel.org> writes:
> >> >>
> >> >> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> >> >> >>
> >> >> >> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> >> >> >> <torvalds@linux-foundation.org> wrote:
> >> >> >> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >> >> >> >
> >> >> >> > We do most of the other heavy lifting in this area in Kconfig anyway,
> >> >> >> > why not add that compiler choice?
> >> >> >> >
> >> >> >> > Obviously it would be gated by the tests to see which compilers are
> >> >> >> > _installed_ (and that they are valid versions), so that it doesn't ask
> >> >> >> > stupid things ("do you want gcc or clang" when only one of them is
> >> >> >> > installed and/or viable).
> >> >> >>
> >> >> >> I don't see a good way of making Kconfig options both select the
> >> >> >> compiler and defining variables based on the compiler, since that
> >> >> >> would mean teaching Kconfig about re-evaluating all compiler
> >> >> >> dependent settings whenever the first option changes.
> >> >> >>
> >> >> >> I do have another idea that I think would work though.
> >> >> >>
> >> >> >> > Hmm? So then any "LLVM=1" thing would be about the "make config"
> >> >> >> > stage, not the actual build stage.
> >> >> >> >
> >> >> >> > (It has annoyed me for years that if you want to cross-compile, you
> >> >> >> > first have to do "make ARCH=xyz config" and then remember to do "make
> >> >> >> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> >> >> >> > never really cared).
> >> >> >>
> >> >> >> The best thing that I have come up with is a pre-configure step, where
> >> >> >> an object tree gets seeded with a makefile fragment that gets included
> >> >> >> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> >> >> >> 'CC=' and possibly any other option that gets passed to 'make' as
> >> >> >> a variable and has to exist before calling 'make *config'.
> >> >> >
> >> >> >
> >> >> > There is no need to add a hook to include such makefile fragment(s).
> >> >> >
> >> >> > Quite opposite, you can put your Makefile (in a different filename)
> >> >> > that includes the top Makefile.
> >> >> >
> >> >> >
> >> >> > I think this is what people are already doing:
> >> >> >
> >> >> >
> >> >> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> >> >> > in this order.
> >> >> >
> >> >> >
> >> >> > So, you can put 'GNUmakefile' with your favorite setups.
> >> >> >
> >> >> >
> >> >> > $ cat GNUmakefile
> >> >> > ARCH=arm64
> >> >> > CROSS_COMPILE=aarch64-linux-gnu-
> >> >> > CC=clang
> >> >> > include Makefile
> >> >>
> >> >> Very weird.
> >> >>
> >> >> I just tested this and it does not work.
> >> >> I did this:
> >> >>
> >> >> $ cat GNUmakefile
> >> >> ARCH = alpha
> >> >> CROSS_COMPILE = $(arch-prefix alpha)
> >> >> include Makefile
> >> >>
> >> >> In one of my build directories and the main makefile simply does not see
> >> >> the value of ARCH or CROSS_COMPILE I set.  I have confirmed that my
> >> >> GNUmakefile is being read, because everything breaks if I remove the
> >> >> include line.
> >> >>
> >> >> Does anyone have any ideas?
> >> >>
> >> >> Something so we don't have to specify all of these variables on the make
> >> >> command line would be nice.
> >> >>
> >> >> Eric
> >> >
> >> >
> >> > Worked for me.
> >> >
> >> > Could you tell me the exact steps you did?
> >> >
> >> >
> >> > This is my case:
> >> >
> >> > My kernel source tree is located at $HOME/ref/linux
> >> > alpha tool chains are located at $HOME/tools/alpha-10.1.0/bin
> >> >
> >> >
> >> >
> >> > I tried a simple GNUmakefile with 3 lines.
> >> >
> >> > You can see 'make' is building the alpha kernel
> >> >
> >> >
> >> > Please see below:
> >>
> >> Interesting.  That appears to work if I don't specify a build directory.
> >> Once I specify a build directory with O= it does not work.
> >>
> >> When I am working on a change that affects multiple architectures
> >> I really want a build directory that is not my source tree so I can
> >> test small changes on multiple architectures without needing to rebuild
> >> everything.
> >>
> >> Eric
> >
> >
> > Ah, OK.
> > When you specify O= option, Kbuild will
> > invoke sub-make.
> > So you need to add 'export' to the
> > variables.
> >
> > Please see the following modified version.
>
> Yes. That does seem to work and be simpler.
>
> Is there any chance we can get the command that generates the Makefile
> in the build directory to include variables like ARCH and CROSS_COMPILE
> in the Makefile as well?  Just so they don't need to be set by hand to
> the same value every time?
>
> Eric


I am OK to document this small tip
in Documentation/kbuild/ if it is useful,
but am reluctant to add any code to
generate custom Makefile or GNUmakefile.

It is highly user-dependent after all.

If we merged the code to set up ARCH=...
and CROSS_COMPILE=...,
somebody would send a patch
"I need LLVM=1 in my GNUmakefile", then
another person, "I want CC=clang", and so on.

You can edit it by yourself.


-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
  2021-07-28 18:59         ` Nick Desaulniers
@ 2021-07-28 22:35           ` Masahiro Yamada
  0 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-28 22:35 UTC (permalink / raw)
  To: Nick Desaulniers
  Cc: Nathan Chancellor, Miguel Ojeda, Fangrui Song, Michal Marek,
	Arnd Bergmann, Linux Kernel Mailing List,
	Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
	Christoph Hellwig, Linus Torvalds

On Thu, Jul 29, 2021 at 4:00 AM 'Nick Desaulniers' via Clang Built
Linux <clang-built-linux@googlegroups.com> wrote:
>
> On Tue, Jul 20, 2021 at 8:50 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > On Wed, Jul 21, 2021 at 2:30 AM Nathan Chancellor <nathan@kernel.org> wrote:
> > >
> > > On 7/20/2021 1:04 AM, Masahiro Yamada wrote:
> > > > On Fri, Jul 9, 2021 at 8:25 AM 'Nick Desaulniers' via Clang Built
> > > > Linux <clang-built-linux@googlegroups.com> wrote:
> > > >>
> > > >> diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
> > > >> index 297932e973d4..956603f56724 100644
> > > >> --- a/scripts/Makefile.clang
> > > >> +++ b/scripts/Makefile.clang
> > > >> @@ -1,6 +1,36 @@
> > > >> -ifneq ($(CROSS_COMPILE),)
> > > >> +# Individual arch/{arch}/Makfiles should use -EL/-EB to set intended endianness
> > > >> +# and -m32/-m64 to set word size based on Kconfigs instead of relying on the
> > > >> +# target triple.
> > > >> +ifeq ($(CROSS_COMPILE),)
> > > >> +ifneq ($(LLVM),)
> > > >
> > > >
> > > > Do you need to check $(LLVM) ?
> > > >
> > > >
> > > > LLVM=1 is a convenient switch to change all the
> > > > defaults, but yet you can flip each tool individually.
> > > >
> > > > Instead of LLVM=1, you still should be able to
> > > > get the equivalent setups by:
> > > >
> > > >
> > > >    make CC=clang LD=ld.lld AR=llvm-ar OBJCOPY=llvm-objcopy ...
> > > >
> > > >
> > > > The --target option is passed to only
> > > > KBUILD_CFLAGS and KBUILD_AFLAGS.
> > > >
> > > > So, when we talk about --target=,
> > > > we only care about whether $(CC) is Clang.
> > > > Not caring about $(AR), $(LD), or $(OBJCOPY).
> > > >
> > > >
> > > > scripts/Makefile.clang is already guarded by:
> > > >
> > > > ifneq ($(findstring clang,$(CC_VERSION_TEXT)),
> > >
> > > $ make ARCH=arm64 CC=clang LLVM_IAS=1
> > >
> > > will use the right compiler and assembler but none of the other binary
> > > tools because '--prefix=' will not be set so CROSS_COMPILE needs to be
> > > specified still, which defeats the purpose of this whole change. This
> > > patch is designed to work for the "normal" case of saying "I want to use
> > > all of the LLVM tools", not "I want to use clang by itself".
> >
> >
> > I disagree.
> >
> > LLVM=1 is a shorthand.
> >
> >
> >
> > make LLVM=1 LLVM_IAS=1
> >
> >   should be equivalent to:
> >
> > make CC=clang LD=ld.lld AR=llvm-ar NM=llvm-nm STRIP=llvm-strip \
> >   OBJCOPY=llvm-objcopy OBJDUMP=llvm-objdump READELF=llvm-readelf \
> >   HOSTCC=clang HOSTCXX=clang++ HOSTAR=llvm-ar HOSTLD=ld.lld \
> >   LLVM_IAS=1
> >
> >
> >
> > We do not care about the origin of CC=clang,
> > whether it came from LLVM=1 or every tool was explicitly,
> > individually specified.
> >
> >
> >
> > ifneq ($(LLVM),) is a garbage code
> > that checks a pointless thing.
>
> Masahiro,
> Nathan is correct.  Test for yourself; if you apply these two patches,
> then apply:
>
> diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang
> index 956603f56724..a1b46811bdc6 100644
> --- a/scripts/Makefile.clang
> +++ b/scripts/Makefile.clang
> @@ -2,7 +2,6 @@
>  # and -m32/-m64 to set word size based on Kconfigs instead of relying on the
>  # target triple.
>  ifeq ($(CROSS_COMPILE),)
> -ifneq ($(LLVM),)
>  ifeq ($(LLVM_IAS),1)
>  ifeq ($(SRCARCH),arm)
>  CLANG_FLAGS    += --target=arm-linux-gnueabi
> @@ -26,7 +25,6 @@ else
>  $(error Specify CROSS_COMPILE or add '--target=' option to
> scripts/Makefile.clang)
>  endif # SRCARCH
>  endif # LLVM_IAS
> -endif # LLVM
>  else
>  CLANG_FLAGS    += --target=$(notdir $(CROSS_COMPILE:%-=%))
>  endif # CROSS_COMPILE
>
> Then build as Nathan specified:
> $ ARCH=arm64 make CC=clang LLVM_IAS=1 -j72 defconfig all


Yes, LD is set to GNU ld, which is only for x86.

$ ARCH=arm64  make  CC=clang LLVM_IAS=1  LD=ld.lld \
   OBJCOPY=llvm-objcopy STRIP=llvm-strip \
    -j72 defconfig all

worked for me.



It is true that the most common use-case would be
LLVM=1 LLVM_IAS=1, but as I said, there is no good
reason to prevent this feature from working when
CC, LD, etc. are specified individually.







> ...
> arch/arm64/Makefile:25: ld does not support --fix-cortex-a53-843419;
> kernel may be susceptible to erratum
> ...
>   LD      arch/arm64/kernel/vdso/vdso.so.dbg
> ld: unrecognised emulation mode: aarch64linux
> Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu
> elf_l1om elf_k1om i386pep i386pe
> make[1]: *** [arch/arm64/kernel/vdso/Makefile:56:
> arch/arm64/kernel/vdso/vdso.so.dbg] Error 1
> make: *** [arch/arm64/Makefile:193: vdso_prepare] Error 2
>
> Nathan referred to --prefix, but in this failure, because
> CROSS_COMPILE was never set, the top level Makefile set LD to:
>  452 LD    = $(CROSS_COMPILE)ld
> in this case `ld` in my path was my host x86 linker, which is not
> correct for a cross compilation of arm64 target.
>
> Perhaps we can somehow support "implicit CROSS_COMPILE" with just
> CC=clang, and not LLVM=1, but I think it would be inflexible to
> hardcode such target triple prefixes.  What if someone has
> arm-linux-gnueabi-as but not arm-linux-gnueabihf-as installed?  That's
> the point of CROSS_COMPILE in my opinion to provide such flexibility
> at the cost of additional command line verbosity.
>
> For the common case of LLVM=1 though, this series is a simplification.
> If users want to specify CC=clang, then they MUST use CROSS_COMPILE
> when cross compiling.
>
> Please review the current approach and see if there's more I can
> improve in a v3; otherwise I still think this series is good to go.
> --
> Thanks,
> ~Nick Desaulniers
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAKwvOdkENUWd7OgJO%3DdNiYjH6D1aJ0puBgs4W7uuYO9xQiAiNg%40mail.gmail.com.



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 39+ messages in thread

end of thread, other threads:[~2021-07-28 22:36 UTC | newest]

Thread overview: 39+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-08 23:25 [PATCH v2 0/2] infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1 Nick Desaulniers
2021-07-08 23:25 ` [PATCH v2 1/2] Makefile: move initial clang flag handling into scripts/Makefile.clang Nick Desaulniers
2021-07-09 20:12   ` Nathan Chancellor
2021-07-08 23:25 ` [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1 Nick Desaulniers
2021-07-09 20:44   ` Nathan Chancellor
2021-07-20  8:04   ` Masahiro Yamada
2021-07-20 17:30     ` Nathan Chancellor
2021-07-21  3:49       ` Masahiro Yamada
2021-07-28 18:59         ` Nick Desaulniers
2021-07-28 22:35           ` Masahiro Yamada
2021-07-20 17:42     ` Linus Torvalds
2021-07-20 19:58       ` Arnd Bergmann
2021-07-20 20:18         ` Nick Desaulniers
2021-07-21  4:04         ` Masahiro Yamada
2021-07-23 19:54           ` Geert Uytterhoeven
2021-07-24 13:46             ` Masahiro Yamada
2021-07-26 20:27           ` Eric W. Biederman
2021-07-27  7:07             ` Geert Uytterhoeven
2021-07-27  7:49               ` Arnd Bergmann
2021-07-27  7:55                 ` Geert Uytterhoeven
2021-07-27  8:21                   ` Arnd Bergmann
2021-07-27 10:10             ` Masahiro Yamada
2021-07-27 14:16               ` Eric W. Biederman
2021-07-27 15:45                 ` Masahiro Yamada
2021-07-27 18:46                   ` Eric W. Biederman
2021-07-28 22:31                     ` Masahiro Yamada
2021-07-20 20:52       ` Nick Desaulniers
2021-07-20 21:11         ` Linus Torvalds
2021-07-20 21:27           ` Nick Desaulniers
2021-07-21  4:53         ` Masahiro Yamada
2021-07-20 21:29       ` Nick Desaulniers
2021-07-20 21:54         ` Linus Torvalds
2021-07-20 23:19           ` Linus Torvalds
2021-07-20 23:22             ` Linus Torvalds
2021-07-21  5:12             ` Masahiro Yamada
2021-07-21  4:52           ` Christoph Hellwig
2021-07-21  5:33             ` Masahiro Yamada
2021-07-21  4:31       ` Masahiro Yamada
2021-07-21  4:44       ` Christoph Hellwig

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).