* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 17:42 ` Linus Torvalds
@ 2021-07-20 19:58 ` Arnd Bergmann
2021-07-20 20:18 ` Nick Desaulniers
2021-07-21 4:04 ` Masahiro Yamada
2021-07-20 20:52 ` Nick Desaulniers
` (3 subsequent siblings)
4 siblings, 2 replies; 39+ messages in thread
From: Arnd Bergmann @ 2021-07-20 19:58 UTC (permalink / raw)
To: Linus Torvalds
Cc: Masahiro Yamada, Nick Desaulniers, Miguel Ojeda, Fangrui Song,
Michal Marek, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> We do most of the other heavy lifting in this area in Kconfig anyway,
> why not add that compiler choice?
>
> Obviously it would be gated by the tests to see which compilers are
> _installed_ (and that they are valid versions), so that it doesn't ask
> stupid things ("do you want gcc or clang" when only one of them is
> installed and/or viable).
I don't see a good way of making Kconfig options both select the
compiler and defining variables based on the compiler, since that
would mean teaching Kconfig about re-evaluating all compiler
dependent settings whenever the first option changes.
I do have another idea that I think would work though.
> Hmm? So then any "LLVM=1" thing would be about the "make config"
> stage, not the actual build stage.
>
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).
The best thing that I have come up with is a pre-configure step, where
an object tree gets seeded with a makefile fragment that gets included
for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
'CC=' and possibly any other option that gets passed to 'make' as
a variable and has to exist before calling 'make *config'.
This is probably the easiest part here, and it lets you set up multiple
object directories in which you can then do
make O=obj-x86 defconfig vmlinux modules -skj32
make O=obj-x86-clang
make O=obj-arm64 randconfig
...
without ever having to type the additional CC/CROSS_COMPILE/LLVM
variables again.
One step further is the script that automatically finds a working toolchain
for a given architecture and sets up that object directory accordingly,
or even downloads a working cross-toolchain from kernel.org when
asked to do that.
Arnd
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 19:58 ` Arnd Bergmann
@ 2021-07-20 20:18 ` Nick Desaulniers
2021-07-21 4:04 ` Masahiro Yamada
1 sibling, 0 replies; 39+ messages in thread
From: Nick Desaulniers @ 2021-07-20 20:18 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Linus Torvalds, Masahiro Yamada, Miguel Ojeda, Fangrui Song,
Michal Marek, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Tue, Jul 20, 2021 at 12:58 PM Arnd Bergmann <arnd@kernel.org> wrote:
>
> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > We do most of the other heavy lifting in this area in Kconfig anyway,
> > why not add that compiler choice?
> >
> > Obviously it would be gated by the tests to see which compilers are
> > _installed_ (and that they are valid versions), so that it doesn't ask
> > stupid things ("do you want gcc or clang" when only one of them is
> > installed and/or viable).
>
> I don't see a good way of making Kconfig options both select the
> compiler and defining variables based on the compiler, since that
> would mean teaching Kconfig about re-evaluating all compiler
> dependent settings whenever the first option changes.
Right, there are many Kconfigs that are predicated on knowing whether
the user is building with gcc vs clang. We probably could note the
users preference in Kconfig, but we would need to guarantee that we
evaluate that ASAP and before most other Kconfigs were checked.
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 19:58 ` Arnd Bergmann
2021-07-20 20:18 ` Nick Desaulniers
@ 2021-07-21 4:04 ` Masahiro Yamada
2021-07-23 19:54 ` Geert Uytterhoeven
2021-07-26 20:27 ` Eric W. Biederman
1 sibling, 2 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-21 4:04 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Linus Torvalds, Nick Desaulniers, Miguel Ojeda, Fangrui Song,
Michal Marek, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
>
> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > We do most of the other heavy lifting in this area in Kconfig anyway,
> > why not add that compiler choice?
> >
> > Obviously it would be gated by the tests to see which compilers are
> > _installed_ (and that they are valid versions), so that it doesn't ask
> > stupid things ("do you want gcc or clang" when only one of them is
> > installed and/or viable).
>
> I don't see a good way of making Kconfig options both select the
> compiler and defining variables based on the compiler, since that
> would mean teaching Kconfig about re-evaluating all compiler
> dependent settings whenever the first option changes.
>
> I do have another idea that I think would work though.
>
> > Hmm? So then any "LLVM=1" thing would be about the "make config"
> > stage, not the actual build stage.
> >
> > (It has annoyed me for years that if you want to cross-compile, you
> > first have to do "make ARCH=xyz config" and then remember to do "make
> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> > never really cared).
>
> The best thing that I have come up with is a pre-configure step, where
> an object tree gets seeded with a makefile fragment that gets included
> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> 'CC=' and possibly any other option that gets passed to 'make' as
> a variable and has to exist before calling 'make *config'.
There is no need to add a hook to include such makefile fragment(s).
Quite opposite, you can put your Makefile (in a different filename)
that includes the top Makefile.
I think this is what people are already doing:
GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
in this order.
So, you can put 'GNUmakefile' with your favorite setups.
$ cat GNUmakefile
ARCH=arm64
CROSS_COMPILE=aarch64-linux-gnu-
CC=clang
include Makefile
Then, you can simply do "make defconfig; make"
for building arm64 with clang.
If you want to have multiple setups, you can put
custom Makefiles, and select by the -f option.
For example,
$ cat Makefile-arm64
ARCH=arm64
CROSS_COMPILE=aarch64-linux-gnu-
include Makefile
$ cat Makefile-arm64-llvm
ARCH=arm64
CROSS_COMPILE=aarch64-linux-gnu-
LLVM=1
include Makefile
$ cat Makefile-arm
ARCH=arm
CROSS_COMPILE=arm-linux-gnueabihf-
include Makefile
Then,
$ make -f Makefile-arm64 defconfig all -j24
$ make -f Makefile-arm64-llvm defconfig all -j24
$ make -f Makefile-arm defconfig all -j24
You can use TAB-completion for the 'Makefile-*' part,
so you can save many typings than you do
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- ...
It is just a tip to ease your life without any change to upstream.
Maybe, some people might be doing that.
>
> This is probably the easiest part here, and it lets you set up multiple
> object directories in which you can then do
>
> make O=obj-x86 defconfig vmlinux modules -skj32
> make O=obj-x86-clang
> make O=obj-arm64 randconfig
> ...
>
> without ever having to type the additional CC/CROSS_COMPILE/LLVM
> variables again.
>
> One step further is the script that automatically finds a working toolchain
> for a given architecture and sets up that object directory accordingly,
> or even downloads a working cross-toolchain from kernel.org when
> asked to do that.
I saw Intel's 0-day bot attached a shell script to download and
set up cross compilers.
(make.cross)
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-21 4:04 ` Masahiro Yamada
@ 2021-07-23 19:54 ` Geert Uytterhoeven
2021-07-24 13:46 ` Masahiro Yamada
2021-07-26 20:27 ` Eric W. Biederman
1 sibling, 1 reply; 39+ messages in thread
From: Geert Uytterhoeven @ 2021-07-23 19:54 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
Fangrui Song, Michal Marek, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Christoph Hellwig,
Nathan Chancellor
Hi Yamada-san,
On Wed, Jul 21, 2021 at 6:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> > On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> > <torvalds@linux-foundation.org> wrote:
> > > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > We do most of the other heavy lifting in this area in Kconfig anyway,
> > > why not add that compiler choice?
> > >
> > > Obviously it would be gated by the tests to see which compilers are
> > > _installed_ (and that they are valid versions), so that it doesn't ask
> > > stupid things ("do you want gcc or clang" when only one of them is
> > > installed and/or viable).
> >
> > I don't see a good way of making Kconfig options both select the
> > compiler and defining variables based on the compiler, since that
> > would mean teaching Kconfig about re-evaluating all compiler
> > dependent settings whenever the first option changes.
> >
> > I do have another idea that I think would work though.
> >
> > > Hmm? So then any "LLVM=1" thing would be about the "make config"
> > > stage, not the actual build stage.
> > >
> > > (It has annoyed me for years that if you want to cross-compile, you
> > > first have to do "make ARCH=xyz config" and then remember to do "make
> > > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> > > never really cared).
> >
> > The best thing that I have come up with is a pre-configure step, where
> > an object tree gets seeded with a makefile fragment that gets included
> > for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> > 'CC=' and possibly any other option that gets passed to 'make' as
> > a variable and has to exist before calling 'make *config'.
>
>
> There is no need to add a hook to include such makefile fragment(s).
>
> Quite opposite, you can put your Makefile (in a different filename)
> that includes the top Makefile.
>
>
> I think this is what people are already doing:
>
>
> GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> in this order.
Exactly. I only have a few source repositories, but lots of build directories
(I never build in a repo directory). Each build directory has a GNUmakefile:
build/linux-riscv-starlight$ cat GNUmakefile
MAKEARGS = ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu-
include ~/linux/default.mk
build/linux-riscv-starlight$ cat ~/linux/default.mk
MAKEFLAGS += --no-print-directory
.PHONY: all $(MAKECMDGOALS)
all := $(filter-out all Makefile,$(MAKECMDGOALS))
all:
@$(MAKE) $(MAKEARGS) $(all) -f Makefile
Makefile:;
$(all): all
@:
%/: all
@:
build/linux-riscv-starlight$
Perhaps we could have a "make setup ARCH=foo CROSS_COMPILE=bar" target,
which creates such a GNUmakefile?
P.S. I put the extra logic in ~/linux/default.mk, so I don't have to update all
GNUmakefiles when I want to make a change to the main logic.
For build dirs where I want to track a specific config, I have a slightly
different version:
build/linux-m68k-allmodconfig-sun3$ cat GNUmakefile
MAKEARGS = ARCH=m68k KCONFIG_ALLCONFIG=1
DEFCONFIG = allmodconfig
include ~/linux/override-oldconfig.mk
build/linux-m68k-allmodconfig-sun3$ cat ~/linux/override-oldconfig.mk
MAKEFLAGS += --no-print-directory
.PHONY: all oldconfig realoldconfig $(MAKECMDGOALS)
all := $(filter-out all oldconfig realoldconfig
Makefile,$(MAKECMDGOALS))
all:
@$(MAKE) $(MAKEARGS) $(all) -f Makefile
# Replace oldconfig by $(DEFCONFIG)
oldconfig:
@$(MAKE) $(MAKEARGS) $(DEFCONFIG) -f Makefile
realoldconfig:
@$(MAKE) $(MAKEARGS) oldconfig -f Makefile
Makefile:;
$(all): all
@:
%/: all
@:
build/linux-m68k-allmodconfig-sun3$
That way I can always just type "make oldconfig", and it will do what
I want.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-23 19:54 ` Geert Uytterhoeven
@ 2021-07-24 13:46 ` Masahiro Yamada
0 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-24 13:46 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
Fangrui Song, Michal Marek, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Christoph Hellwig,
Nathan Chancellor
On Sat, Jul 24, 2021 at 4:54 AM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> Hi Yamada-san,
>
> On Wed, Jul 21, 2021 at 6:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> > > On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> > > <torvalds@linux-foundation.org> wrote:
> > > > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > >
> > > > We do most of the other heavy lifting in this area in Kconfig anyway,
> > > > why not add that compiler choice?
> > > >
> > > > Obviously it would be gated by the tests to see which compilers are
> > > > _installed_ (and that they are valid versions), so that it doesn't ask
> > > > stupid things ("do you want gcc or clang" when only one of them is
> > > > installed and/or viable).
> > >
> > > I don't see a good way of making Kconfig options both select the
> > > compiler and defining variables based on the compiler, since that
> > > would mean teaching Kconfig about re-evaluating all compiler
> > > dependent settings whenever the first option changes.
> > >
> > > I do have another idea that I think would work though.
> > >
> > > > Hmm? So then any "LLVM=1" thing would be about the "make config"
> > > > stage, not the actual build stage.
> > > >
> > > > (It has annoyed me for years that if you want to cross-compile, you
> > > > first have to do "make ARCH=xyz config" and then remember to do "make
> > > > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> > > > never really cared).
> > >
> > > The best thing that I have come up with is a pre-configure step, where
> > > an object tree gets seeded with a makefile fragment that gets included
> > > for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> > > 'CC=' and possibly any other option that gets passed to 'make' as
> > > a variable and has to exist before calling 'make *config'.
> >
> >
> > There is no need to add a hook to include such makefile fragment(s).
> >
> > Quite opposite, you can put your Makefile (in a different filename)
> > that includes the top Makefile.
> >
> >
> > I think this is what people are already doing:
> >
> >
> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> > in this order.
>
> Exactly. I only have a few source repositories, but lots of build directories
> (I never build in a repo directory). Each build directory has a GNUmakefile:
>
> build/linux-riscv-starlight$ cat GNUmakefile
> MAKEARGS = ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu-
>
> include ~/linux/default.mk
> build/linux-riscv-starlight$ cat ~/linux/default.mk
> MAKEFLAGS += --no-print-directory
>
> .PHONY: all $(MAKECMDGOALS)
>
> all := $(filter-out all Makefile,$(MAKECMDGOALS))
>
> all:
> @$(MAKE) $(MAKEARGS) $(all) -f Makefile
>
> Makefile:;
>
> $(all): all
> @:
>
> %/: all
> @:
> build/linux-riscv-starlight$
>
> Perhaps we could have a "make setup ARCH=foo CROSS_COMPILE=bar" target,
> which creates such a GNUmakefile?
No, I do not think so.
Your GNUmakefile is customized in your way.
Other people implement GNUmakefile in their own way.
Upstream is not a place to merge everybody's local tools.
> P.S. I put the extra logic in ~/linux/default.mk, so I don't have to update all
> GNUmakefiles when I want to make a change to the main logic.
>
> For build dirs where I want to track a specific config, I have a slightly
> different version:
>
> build/linux-m68k-allmodconfig-sun3$ cat GNUmakefile
> MAKEARGS = ARCH=m68k KCONFIG_ALLCONFIG=1
> DEFCONFIG = allmodconfig
>
> include ~/linux/override-oldconfig.mk
> build/linux-m68k-allmodconfig-sun3$ cat ~/linux/override-oldconfig.mk
> MAKEFLAGS += --no-print-directory
>
> .PHONY: all oldconfig realoldconfig $(MAKECMDGOALS)
>
> all := $(filter-out all oldconfig realoldconfig
> Makefile,$(MAKECMDGOALS))
>
> all:
> @$(MAKE) $(MAKEARGS) $(all) -f Makefile
>
> # Replace oldconfig by $(DEFCONFIG)
> oldconfig:
> @$(MAKE) $(MAKEARGS) $(DEFCONFIG) -f Makefile
>
> realoldconfig:
> @$(MAKE) $(MAKEARGS) oldconfig -f Makefile
>
> Makefile:;
>
> $(all): all
> @:
>
> %/: all
> @:
> build/linux-m68k-allmodconfig-sun3$
>
> That way I can always just type "make oldconfig", and it will do what
> I want.
>
> Gr{oetje,eeting}s,
>
> Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
> -- Linus Torvalds
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAMuHMdXJBqrpzaSNDJgic14ESiHV6cCcb_5E-st6iniXdmm9_g%40mail.gmail.com.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-21 4:04 ` Masahiro Yamada
2021-07-23 19:54 ` Geert Uytterhoeven
@ 2021-07-26 20:27 ` Eric W. Biederman
2021-07-27 7:07 ` Geert Uytterhoeven
2021-07-27 10:10 ` Masahiro Yamada
1 sibling, 2 replies; 39+ messages in thread
From: Eric W. Biederman @ 2021-07-26 20:27 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
Fangrui Song, Michal Marek, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
Masahiro Yamada <masahiroy@kernel.org> writes:
> On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
>>
>> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
>> <torvalds@linux-foundation.org> wrote:
>> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>> >
>> > We do most of the other heavy lifting in this area in Kconfig anyway,
>> > why not add that compiler choice?
>> >
>> > Obviously it would be gated by the tests to see which compilers are
>> > _installed_ (and that they are valid versions), so that it doesn't ask
>> > stupid things ("do you want gcc or clang" when only one of them is
>> > installed and/or viable).
>>
>> I don't see a good way of making Kconfig options both select the
>> compiler and defining variables based on the compiler, since that
>> would mean teaching Kconfig about re-evaluating all compiler
>> dependent settings whenever the first option changes.
>>
>> I do have another idea that I think would work though.
>>
>> > Hmm? So then any "LLVM=1" thing would be about the "make config"
>> > stage, not the actual build stage.
>> >
>> > (It has annoyed me for years that if you want to cross-compile, you
>> > first have to do "make ARCH=xyz config" and then remember to do "make
>> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
>> > never really cared).
>>
>> The best thing that I have come up with is a pre-configure step, where
>> an object tree gets seeded with a makefile fragment that gets included
>> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
>> 'CC=' and possibly any other option that gets passed to 'make' as
>> a variable and has to exist before calling 'make *config'.
>
>
> There is no need to add a hook to include such makefile fragment(s).
>
> Quite opposite, you can put your Makefile (in a different filename)
> that includes the top Makefile.
>
>
> I think this is what people are already doing:
>
>
> GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> in this order.
>
>
> So, you can put 'GNUmakefile' with your favorite setups.
>
>
> $ cat GNUmakefile
> ARCH=arm64
> CROSS_COMPILE=aarch64-linux-gnu-
> CC=clang
> include Makefile
Very weird.
I just tested this and it does not work.
I did this:
$ cat GNUmakefile
ARCH = alpha
CROSS_COMPILE = $(arch-prefix alpha)
include Makefile
In one of my build directories and the main makefile simply does not see
the value of ARCH or CROSS_COMPILE I set. I have confirmed that my
GNUmakefile is being read, because everything breaks if I remove the
include line.
Does anyone have any ideas?
Something so we don't have to specify all of these variables on the make
command line would be nice.
Eric
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-26 20:27 ` Eric W. Biederman
@ 2021-07-27 7:07 ` Geert Uytterhoeven
2021-07-27 7:49 ` Arnd Bergmann
2021-07-27 10:10 ` Masahiro Yamada
1 sibling, 1 reply; 39+ messages in thread
From: Geert Uytterhoeven @ 2021-07-27 7:07 UTC (permalink / raw)
To: Eric W. Biederman
Cc: Masahiro Yamada, Arnd Bergmann, Linus Torvalds, Nick Desaulniers,
Miguel Ojeda, Fangrui Song, Michal Marek,
Linux Kernel Mailing List, Linux Kbuild mailing list,
clang-built-linux, Christoph Hellwig, Nathan Chancellor
[-- Attachment #1: Type: text/plain, Size: 3724 bytes --]
Hi Eric,
On Mon, Jul 26, 2021 at 10:27 PM Eric W. Biederman
<ebiederm@xmission.com> wrote:
> Masahiro Yamada <masahiroy@kernel.org> writes:
> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> >> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> >> <torvalds@linux-foundation.org> wrote:
> >> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >> >
> >> > We do most of the other heavy lifting in this area in Kconfig anyway,
> >> > why not add that compiler choice?
> >> >
> >> > Obviously it would be gated by the tests to see which compilers are
> >> > _installed_ (and that they are valid versions), so that it doesn't ask
> >> > stupid things ("do you want gcc or clang" when only one of them is
> >> > installed and/or viable).
> >>
> >> I don't see a good way of making Kconfig options both select the
> >> compiler and defining variables based on the compiler, since that
> >> would mean teaching Kconfig about re-evaluating all compiler
> >> dependent settings whenever the first option changes.
> >>
> >> I do have another idea that I think would work though.
> >>
> >> > Hmm? So then any "LLVM=1" thing would be about the "make config"
> >> > stage, not the actual build stage.
> >> >
> >> > (It has annoyed me for years that if you want to cross-compile, you
> >> > first have to do "make ARCH=xyz config" and then remember to do "make
> >> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> >> > never really cared).
> >>
> >> The best thing that I have come up with is a pre-configure step, where
> >> an object tree gets seeded with a makefile fragment that gets included
> >> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> >> 'CC=' and possibly any other option that gets passed to 'make' as
> >> a variable and has to exist before calling 'make *config'.
> >
> >
> > There is no need to add a hook to include such makefile fragment(s).
> >
> > Quite opposite, you can put your Makefile (in a different filename)
> > that includes the top Makefile.
> >
> >
> > I think this is what people are already doing:
> >
> >
> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> > in this order.
> >
> >
> > So, you can put 'GNUmakefile' with your favorite setups.
> >
> >
> > $ cat GNUmakefile
> > ARCH=arm64
> > CROSS_COMPILE=aarch64-linux-gnu-
> > CC=clang
> > include Makefile
>
> Very weird.
>
> I just tested this and it does not work.
> I did this:
>
> $ cat GNUmakefile
> ARCH = alpha
> CROSS_COMPILE = $(arch-prefix alpha)
> include Makefile
>
> In one of my build directories and the main makefile simply does not see
> the value of ARCH or CROSS_COMPILE I set. I have confirmed that my
> GNUmakefile is being read, because everything breaks if I remove the
> include line.
>
> Does anyone have any ideas?
>
> Something so we don't have to specify all of these variables on the make
> command line would be nice.
Just including the main Makefile does not work.
That's why I went with the more convoluted solution in
https://lore.kernel.org/linux-kbuild/CAMuHMdXJBqrpzaSNDJgic14ESiHV6cCcb_5E-st6iniXdmm9_g@mail.gmail.com/
Please try the attached, which combines everything above in a single
file, and which works for me. Note that "$(arch-prefix alpha)" didn't
work for me (it resolved to "gcc"?), so I used "alpha-linux-gnu-"
instead.
Good luck!
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
[-- Attachment #2: GNUmakefile --]
[-- Type: application/octet-stream, Size: 257 bytes --]
MAKEARGS = ARCH=alpha CROSS_COMPILE=alpha-linux-gnu-
MAKEFLAGS += --no-print-directory
.PHONY: all $(MAKECMDGOALS)
all := $(filter-out all Makefile,$(MAKECMDGOALS))
all:
@$(MAKE) $(MAKEARGS) $(all) -f Makefile
Makefile:;
$(all): all
@:
%/: all
@:
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-27 7:07 ` Geert Uytterhoeven
@ 2021-07-27 7:49 ` Arnd Bergmann
2021-07-27 7:55 ` Geert Uytterhoeven
0 siblings, 1 reply; 39+ messages in thread
From: Arnd Bergmann @ 2021-07-27 7:49 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Eric W. Biederman, Masahiro Yamada, Linus Torvalds,
Nick Desaulniers, Miguel Ojeda, Fangrui Song, Michal Marek,
Linux Kernel Mailing List, Linux Kbuild mailing list,
clang-built-linux, Christoph Hellwig, Nathan Chancellor
On Tue, Jul 27, 2021 at 9:07 AM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> On Mon, Jul 26, 2021 at 10:27 PM Eric W. Biederman <ebiederm@xmission.com> wrote:
> > Masahiro Yamada <masahiroy@kernel.org> writes:
> > > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> >
> > Does anyone have any ideas?
> >
> > Something so we don't have to specify all of these variables on the make
> > command line would be nice.
>
> Just including the main Makefile does not work.
> That's why I went with the more convoluted solution in
> https://lore.kernel.org/linux-kbuild/CAMuHMdXJBqrpzaSNDJgic14ESiHV6cCcb_5E-st6iniXdmm9_g@mail.gmail.com/
This is roughly what I use as well, but it does have the downside that
it confuses
the inner 'make' when you build multiple targets in parallel, e.g.
make -skj30 kernel/ mm/ init/
works with the normal Makefile, but fails spectacularly with my nested
GNUmakefile because it starts multiple sub-processes that each try to
build the same preparation files (I did not try your version).
What I had in mind was to use something like (but not exactly)
-include $(O)/Makefile.cross
to conditionally include another file from the top-level Makefile before
doing anything else.
Arnd
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-27 7:49 ` Arnd Bergmann
@ 2021-07-27 7:55 ` Geert Uytterhoeven
2021-07-27 8:21 ` Arnd Bergmann
0 siblings, 1 reply; 39+ messages in thread
From: Geert Uytterhoeven @ 2021-07-27 7:55 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Eric W. Biederman, Masahiro Yamada, Linus Torvalds,
Nick Desaulniers, Miguel Ojeda, Fangrui Song, Michal Marek,
Linux Kernel Mailing List, Linux Kbuild mailing list,
clang-built-linux, Christoph Hellwig, Nathan Chancellor
Hi Arnd,
On Tue, Jul 27, 2021 at 9:49 AM Arnd Bergmann <arnd@kernel.org> wrote:
> On Tue, Jul 27, 2021 at 9:07 AM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > On Mon, Jul 26, 2021 at 10:27 PM Eric W. Biederman <ebiederm@xmission.com> wrote:
> > > Masahiro Yamada <masahiroy@kernel.org> writes:
> > > > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> > >
> > > Does anyone have any ideas?
> > >
> > > Something so we don't have to specify all of these variables on the make
> > > command line would be nice.
> >
> > Just including the main Makefile does not work.
> > That's why I went with the more convoluted solution in
> > https://lore.kernel.org/linux-kbuild/CAMuHMdXJBqrpzaSNDJgic14ESiHV6cCcb_5E-st6iniXdmm9_g@mail.gmail.com/
>
> This is roughly what I use as well, but it does have the downside that
> it confuses
> the inner 'make' when you build multiple targets in parallel, e.g.
>
> make -skj30 kernel/ mm/ init/
>
> works with the normal Makefile, but fails spectacularly with my nested
> GNUmakefile because it starts multiple sub-processes that each try to
> build the same preparation files (I did not try your version).
Seems to work fine with mine...
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-27 7:55 ` Geert Uytterhoeven
@ 2021-07-27 8:21 ` Arnd Bergmann
0 siblings, 0 replies; 39+ messages in thread
From: Arnd Bergmann @ 2021-07-27 8:21 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Eric W. Biederman, Masahiro Yamada, Linus Torvalds,
Nick Desaulniers, Miguel Ojeda, Fangrui Song, Michal Marek,
Linux Kernel Mailing List, Linux Kbuild mailing list,
clang-built-linux, Christoph Hellwig, Nathan Chancellor
On Tue, Jul 27, 2021 at 9:55 AM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> On Tue, Jul 27, 2021 at 9:49 AM Arnd Bergmann <arnd@kernel.org> wrote:
> > make -skj30 kernel/ mm/ init/
> >
> > works with the normal Makefile, but fails spectacularly with my nested
> > GNUmakefile because it starts multiple sub-processes that each try to
> > build the same preparation files (I did not try your version).
>
> Seems to work fine with mine...
>
Ok, nevermind. I tried your version and it works for me too. The next
time I get annoyed by this, I'll figure out what I did wrong then;-)
Arnd
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-26 20:27 ` Eric W. Biederman
2021-07-27 7:07 ` Geert Uytterhoeven
@ 2021-07-27 10:10 ` Masahiro Yamada
2021-07-27 14:16 ` Eric W. Biederman
1 sibling, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-27 10:10 UTC (permalink / raw)
To: Eric W. Biederman
Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
Fangrui Song, Michal Marek, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Tue, Jul 27, 2021 at 5:27 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
>
> Masahiro Yamada <masahiroy@kernel.org> writes:
>
> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> >>
> >> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> >> <torvalds@linux-foundation.org> wrote:
> >> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >> >
> >> > We do most of the other heavy lifting in this area in Kconfig anyway,
> >> > why not add that compiler choice?
> >> >
> >> > Obviously it would be gated by the tests to see which compilers are
> >> > _installed_ (and that they are valid versions), so that it doesn't ask
> >> > stupid things ("do you want gcc or clang" when only one of them is
> >> > installed and/or viable).
> >>
> >> I don't see a good way of making Kconfig options both select the
> >> compiler and defining variables based on the compiler, since that
> >> would mean teaching Kconfig about re-evaluating all compiler
> >> dependent settings whenever the first option changes.
> >>
> >> I do have another idea that I think would work though.
> >>
> >> > Hmm? So then any "LLVM=1" thing would be about the "make config"
> >> > stage, not the actual build stage.
> >> >
> >> > (It has annoyed me for years that if you want to cross-compile, you
> >> > first have to do "make ARCH=xyz config" and then remember to do "make
> >> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> >> > never really cared).
> >>
> >> The best thing that I have come up with is a pre-configure step, where
> >> an object tree gets seeded with a makefile fragment that gets included
> >> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> >> 'CC=' and possibly any other option that gets passed to 'make' as
> >> a variable and has to exist before calling 'make *config'.
> >
> >
> > There is no need to add a hook to include such makefile fragment(s).
> >
> > Quite opposite, you can put your Makefile (in a different filename)
> > that includes the top Makefile.
> >
> >
> > I think this is what people are already doing:
> >
> >
> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> > in this order.
> >
> >
> > So, you can put 'GNUmakefile' with your favorite setups.
> >
> >
> > $ cat GNUmakefile
> > ARCH=arm64
> > CROSS_COMPILE=aarch64-linux-gnu-
> > CC=clang
> > include Makefile
>
> Very weird.
>
> I just tested this and it does not work.
> I did this:
>
> $ cat GNUmakefile
> ARCH = alpha
> CROSS_COMPILE = $(arch-prefix alpha)
> include Makefile
>
> In one of my build directories and the main makefile simply does not see
> the value of ARCH or CROSS_COMPILE I set. I have confirmed that my
> GNUmakefile is being read, because everything breaks if I remove the
> include line.
>
> Does anyone have any ideas?
>
> Something so we don't have to specify all of these variables on the make
> command line would be nice.
>
> Eric
Worked for me.
Could you tell me the exact steps you did?
This is my case:
My kernel source tree is located at $HOME/ref/linux
alpha tool chains are located at $HOME/tools/alpha-10.1.0/bin
I tried a simple GNUmakefile with 3 lines.
You can see 'make' is building the alpha kernel
Please see below:
masahiro@grover:~/ref/linux$ cat GNUmakefile
ARCH = alpha
CROSS_COMPILE = $(HOME)/tools/alpha-10.1.0/bin/alpha-linux-
include Makefile
masahiro@grover:~/ref/linux$ make mrproper
CLEAN arch/alpha/kernel
CLEAN certs
CLEAN drivers/tty/vt
CLEAN drivers/video/logo
CLEAN kernel
CLEAN lib
CLEAN security/selinux
CLEAN usr/include
CLEAN usr
CLEAN vmlinux.symvers modules-only.symvers modules.builtin
modules.builtin.modinfo
CLEAN scripts/basic
CLEAN scripts/dtc
CLEAN scripts/genksyms
CLEAN scripts/kconfig
CLEAN scripts/mod
CLEAN scripts/selinux/genheaders
CLEAN scripts/selinux/mdp
CLEAN scripts
CLEAN include/config include/generated
arch/alpha/include/generated .config .config.old .version
Module.symvers
masahiro@grover:~/ref/linux$ make defconfig
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/expr.o
LEX scripts/kconfig/lexer.lex.c
YACC scripts/kconfig/parser.tab.[ch]
HOSTCC scripts/kconfig/lexer.lex.o
HOSTCC scripts/kconfig/menu.o
HOSTCC scripts/kconfig/parser.tab.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/symbol.o
HOSTCC scripts/kconfig/util.o
HOSTLD scripts/kconfig/conf
*** Default configuration is based on 'defconfig'
#
# configuration written to .config
#
masahiro@grover:~/ref/linux$ make
SYSHDR arch/alpha/include/generated/uapi/asm/unistd_32.h
SYSTBL arch/alpha/include/generated/asm/syscall_table.h
HOSTCC scripts/kallsyms
WRAP arch/alpha/include/generated/uapi/asm/bpf_perf_event.h
WRAP arch/alpha/include/generated/uapi/asm/ipcbuf.h
WRAP arch/alpha/include/generated/uapi/asm/msgbuf.h
WRAP arch/alpha/include/generated/uapi/asm/poll.h
WRAP arch/alpha/include/generated/uapi/asm/sembuf.h
WRAP arch/alpha/include/generated/uapi/asm/shmbuf.h
WRAP arch/alpha/include/generated/asm/export.h
WRAP arch/alpha/include/generated/asm/kvm_para.h
WRAP arch/alpha/include/generated/asm/mcs_spinlock.h
WRAP arch/alpha/include/generated/asm/compat.h
WRAP arch/alpha/include/generated/asm/current.h
WRAP arch/alpha/include/generated/asm/exec.h
WRAP arch/alpha/include/generated/asm/fb.h
WRAP arch/alpha/include/generated/asm/irq_work.h
WRAP arch/alpha/include/generated/asm/kmap_size.h
WRAP arch/alpha/include/generated/asm/kprobes.h
WRAP arch/alpha/include/generated/asm/local64.h
WRAP arch/alpha/include/generated/asm/mmiowb.h
WRAP arch/alpha/include/generated/asm/module.lds.h
WRAP arch/alpha/include/generated/asm/msi.h
WRAP arch/alpha/include/generated/asm/preempt.h
WRAP arch/alpha/include/generated/asm/sections.h
WRAP arch/alpha/include/generated/asm/simd.h
WRAP arch/alpha/include/generated/asm/softirq_stack.h
WRAP arch/alpha/include/generated/asm/trace_clock.h
WRAP arch/alpha/include/generated/asm/unaligned.h
WRAP arch/alpha/include/generated/asm/vermagic.h
UPD include/config/kernel.release
UPD include/generated/uapi/linux/version.h
UPD include/generated/utsrelease.h
CC scripts/mod/empty.o
HOSTCC scripts/mod/mk_elfconfig
MKELF scripts/mod/elfconfig.h
HOSTCC scripts/mod/modpost.o
CC scripts/mod/devicetable-offsets.s
UPD scripts/mod/devicetable-offsets.h
HOSTCC scripts/mod/file2alias.o
HOSTCC scripts/mod/sumversion.o
HOSTLD scripts/mod/modpost
CC kernel/bounds.s
UPD include/generated/bounds.h
UPD include/generated/timeconst.h
CC arch/alpha/kernel/asm-offsets.s
UPD include/generated/asm-offsets.h
CALL scripts/checksyscalls.sh
<stdin>:1515:2: warning: #warning syscall clone3 not implemented [-Wcpp]
CALL scripts/atomic/check-atomics.sh
CC init/main.o
CHK include/generated/compile.h
UPD include/generated/compile.h
CC init/version.o
CC init/do_mounts.o
CC init/noinitramfs.o
CC init/calibrate.o
CC init/init_task.o
AR init/built-in.a
AR usr/built-in.a
AS arch/alpha/kernel/head.o
LDS arch/alpha/kernel/vmlinux.lds
AS arch/alpha/kernel/entry.o
CC arch/alpha/kernel/traps.o
CC arch/alpha/kernel/process.o
CC arch/alpha/kernel/osf_sys.o
CC arch/alpha/kernel/irq.o
CC arch/alpha/kernel/irq_alpha.o
CC arch/alpha/kernel/signal.o
CC arch/alpha/kernel/setup.o
CC arch/alpha/kernel/ptrace.o
CC arch/alpha/kernel/time.o
AS arch/alpha/kernel/systbls.o
CC arch/alpha/kernel/err_common.o
CC arch/alpha/kernel/io.o
CC arch/alpha/kernel/bugs.o
CC arch/alpha/kernel/console.o
CC arch/alpha/kernel/pci.o
CC arch/alpha/kernel/pci_iommu.o
CC arch/alpha/kernel/pci-sysfs.o
CC arch/alpha/kernel/module.o
CC arch/alpha/kernel/rtc.o
CC arch/alpha/kernel/core_apecs.o
CC arch/alpha/kernel/core_cia.o
CC arch/alpha/kernel/core_irongate.o
CC arch/alpha/kernel/core_lca.o
CC arch/alpha/kernel/core_mcpcia.o
CC arch/alpha/kernel/core_polaris.o
CC arch/alpha/kernel/core_t2.o
CC arch/alpha/kernel/core_tsunami.o
CC arch/alpha/kernel/sys_alcor.o
CC arch/alpha/kernel/sys_cabriolet.o
CC arch/alpha/kernel/sys_dp264.o
CC arch/alpha/kernel/sys_eb64p.o
CC arch/alpha/kernel/sys_eiger.o
CC arch/alpha/kernel/sys_jensen.o
CC arch/alpha/kernel/sys_miata.o
CC arch/alpha/kernel/sys_mikasa.o
CC arch/alpha/kernel/sys_nautilus.o
CC arch/alpha/kernel/sys_noritake.o
CC arch/alpha/kernel/sys_rawhide.o
CC arch/alpha/kernel/sys_ruffian.o
CC arch/alpha/kernel/sys_rx164.o
CC arch/alpha/kernel/sys_sable.o
CC arch/alpha/kernel/sys_sio.o
CC arch/alpha/kernel/sys_sx164.o
CC arch/alpha/kernel/sys_takara.o
CC arch/alpha/kernel/irq_pyxis.o
CC arch/alpha/kernel/irq_i8259.o
CC arch/alpha/kernel/irq_srm.o
CC arch/alpha/kernel/err_ev6.o
CC arch/alpha/kernel/es1888.o
CC arch/alpha/kernel/smc37c669.o
CC arch/alpha/kernel/smc37c93x.o
CC arch/alpha/kernel/pc873xx.o
CC arch/alpha/kernel/gct.o
CC arch/alpha/kernel/srmcons.o
AR arch/alpha/kernel/built-in.a
CC [M] arch/alpha/kernel/srm_env.o
CC arch/alpha/mm/init.o
CC arch/alpha/mm/fault.o
AR arch/alpha/mm/built-in.a
CC arch/alpha/math-emu/math.o
^Cmake[2]: *** Deleting file 'arch/alpha/math-emu/math.o'
make[2]: *** [scripts/Makefile.build:271: arch/alpha/math-emu/math.o] Interrupt
make[1]: *** [scripts/Makefile.build:514: arch/alpha/math-emu] Interrupt
make: *** [Makefile:1842: arch/alpha] Interrupt
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-27 10:10 ` Masahiro Yamada
@ 2021-07-27 14:16 ` Eric W. Biederman
2021-07-27 15:45 ` Masahiro Yamada
0 siblings, 1 reply; 39+ messages in thread
From: Eric W. Biederman @ 2021-07-27 14:16 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
Fangrui Song, Michal Marek, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
Masahiro Yamada <masahiroy@kernel.org> writes:
> On Tue, Jul 27, 2021 at 5:27 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
>>
>> Masahiro Yamada <masahiroy@kernel.org> writes:
>>
>> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
>> >>
>> >> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
>> >> <torvalds@linux-foundation.org> wrote:
>> >> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>> >> >
>> >> > We do most of the other heavy lifting in this area in Kconfig anyway,
>> >> > why not add that compiler choice?
>> >> >
>> >> > Obviously it would be gated by the tests to see which compilers are
>> >> > _installed_ (and that they are valid versions), so that it doesn't ask
>> >> > stupid things ("do you want gcc or clang" when only one of them is
>> >> > installed and/or viable).
>> >>
>> >> I don't see a good way of making Kconfig options both select the
>> >> compiler and defining variables based on the compiler, since that
>> >> would mean teaching Kconfig about re-evaluating all compiler
>> >> dependent settings whenever the first option changes.
>> >>
>> >> I do have another idea that I think would work though.
>> >>
>> >> > Hmm? So then any "LLVM=1" thing would be about the "make config"
>> >> > stage, not the actual build stage.
>> >> >
>> >> > (It has annoyed me for years that if you want to cross-compile, you
>> >> > first have to do "make ARCH=xyz config" and then remember to do "make
>> >> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
>> >> > never really cared).
>> >>
>> >> The best thing that I have come up with is a pre-configure step, where
>> >> an object tree gets seeded with a makefile fragment that gets included
>> >> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
>> >> 'CC=' and possibly any other option that gets passed to 'make' as
>> >> a variable and has to exist before calling 'make *config'.
>> >
>> >
>> > There is no need to add a hook to include such makefile fragment(s).
>> >
>> > Quite opposite, you can put your Makefile (in a different filename)
>> > that includes the top Makefile.
>> >
>> >
>> > I think this is what people are already doing:
>> >
>> >
>> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
>> > in this order.
>> >
>> >
>> > So, you can put 'GNUmakefile' with your favorite setups.
>> >
>> >
>> > $ cat GNUmakefile
>> > ARCH=arm64
>> > CROSS_COMPILE=aarch64-linux-gnu-
>> > CC=clang
>> > include Makefile
>>
>> Very weird.
>>
>> I just tested this and it does not work.
>> I did this:
>>
>> $ cat GNUmakefile
>> ARCH = alpha
>> CROSS_COMPILE = $(arch-prefix alpha)
>> include Makefile
>>
>> In one of my build directories and the main makefile simply does not see
>> the value of ARCH or CROSS_COMPILE I set. I have confirmed that my
>> GNUmakefile is being read, because everything breaks if I remove the
>> include line.
>>
>> Does anyone have any ideas?
>>
>> Something so we don't have to specify all of these variables on the make
>> command line would be nice.
>>
>> Eric
>
>
> Worked for me.
>
> Could you tell me the exact steps you did?
>
>
> This is my case:
>
> My kernel source tree is located at $HOME/ref/linux
> alpha tool chains are located at $HOME/tools/alpha-10.1.0/bin
>
>
>
> I tried a simple GNUmakefile with 3 lines.
>
> You can see 'make' is building the alpha kernel
>
>
> Please see below:
Interesting. That appears to work if I don't specify a build directory.
Once I specify a build directory with O= it does not work.
When I am working on a change that affects multiple architectures
I really want a build directory that is not my source tree so I can
test small changes on multiple architectures without needing to rebuild
everything.
Eric
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-27 14:16 ` Eric W. Biederman
@ 2021-07-27 15:45 ` Masahiro Yamada
2021-07-27 18:46 ` Eric W. Biederman
0 siblings, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-27 15:45 UTC (permalink / raw)
To: Eric W. Biederman
Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
Fangrui Song, Michal Marek, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Tue, Jul 27, 2021 at 11:16 PM Eric W. Biederman
<ebiederm@xmission.com> wrote:
>
> Masahiro Yamada <masahiroy@kernel.org> writes:
>
> > On Tue, Jul 27, 2021 at 5:27 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
> >>
> >> Masahiro Yamada <masahiroy@kernel.org> writes:
> >>
> >> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> >> >>
> >> >> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> >> >> <torvalds@linux-foundation.org> wrote:
> >> >> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >> >> >
> >> >> > We do most of the other heavy lifting in this area in Kconfig anyway,
> >> >> > why not add that compiler choice?
> >> >> >
> >> >> > Obviously it would be gated by the tests to see which compilers are
> >> >> > _installed_ (and that they are valid versions), so that it doesn't ask
> >> >> > stupid things ("do you want gcc or clang" when only one of them is
> >> >> > installed and/or viable).
> >> >>
> >> >> I don't see a good way of making Kconfig options both select the
> >> >> compiler and defining variables based on the compiler, since that
> >> >> would mean teaching Kconfig about re-evaluating all compiler
> >> >> dependent settings whenever the first option changes.
> >> >>
> >> >> I do have another idea that I think would work though.
> >> >>
> >> >> > Hmm? So then any "LLVM=1" thing would be about the "make config"
> >> >> > stage, not the actual build stage.
> >> >> >
> >> >> > (It has annoyed me for years that if you want to cross-compile, you
> >> >> > first have to do "make ARCH=xyz config" and then remember to do "make
> >> >> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> >> >> > never really cared).
> >> >>
> >> >> The best thing that I have come up with is a pre-configure step, where
> >> >> an object tree gets seeded with a makefile fragment that gets included
> >> >> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> >> >> 'CC=' and possibly any other option that gets passed to 'make' as
> >> >> a variable and has to exist before calling 'make *config'.
> >> >
> >> >
> >> > There is no need to add a hook to include such makefile fragment(s).
> >> >
> >> > Quite opposite, you can put your Makefile (in a different filename)
> >> > that includes the top Makefile.
> >> >
> >> >
> >> > I think this is what people are already doing:
> >> >
> >> >
> >> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> >> > in this order.
> >> >
> >> >
> >> > So, you can put 'GNUmakefile' with your favorite setups.
> >> >
> >> >
> >> > $ cat GNUmakefile
> >> > ARCH=arm64
> >> > CROSS_COMPILE=aarch64-linux-gnu-
> >> > CC=clang
> >> > include Makefile
> >>
> >> Very weird.
> >>
> >> I just tested this and it does not work.
> >> I did this:
> >>
> >> $ cat GNUmakefile
> >> ARCH = alpha
> >> CROSS_COMPILE = $(arch-prefix alpha)
> >> include Makefile
> >>
> >> In one of my build directories and the main makefile simply does not see
> >> the value of ARCH or CROSS_COMPILE I set. I have confirmed that my
> >> GNUmakefile is being read, because everything breaks if I remove the
> >> include line.
> >>
> >> Does anyone have any ideas?
> >>
> >> Something so we don't have to specify all of these variables on the make
> >> command line would be nice.
> >>
> >> Eric
> >
> >
> > Worked for me.
> >
> > Could you tell me the exact steps you did?
> >
> >
> > This is my case:
> >
> > My kernel source tree is located at $HOME/ref/linux
> > alpha tool chains are located at $HOME/tools/alpha-10.1.0/bin
> >
> >
> >
> > I tried a simple GNUmakefile with 3 lines.
> >
> > You can see 'make' is building the alpha kernel
> >
> >
> > Please see below:
>
> Interesting. That appears to work if I don't specify a build directory.
> Once I specify a build directory with O= it does not work.
>
> When I am working on a change that affects multiple architectures
> I really want a build directory that is not my source tree so I can
> test small changes on multiple architectures without needing to rebuild
> everything.
>
> Eric
Ah, OK.
When you specify O= option, Kbuild will
invoke sub-make.
So you need to add 'export' to the
variables.
Please see the following modified version.
masahiro@grover:~/ref/linux$ rm -rf build-dir/
masahiro@grover:~/ref/linux$ cat GNUmakefile
export ARCH = alpha
export CROSS_COMPILE = $(HOME)/tools/alpha-10.1.0/bin/alpha-linux-
include Makefile
masahiro@grover:~/ref/linux$ make O=build-dir defconfig
make[1]: Entering directory '/home/masahiro/ref/linux/build-dir'
GEN Makefile
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/expr.o
LEX scripts/kconfig/lexer.lex.c
YACC scripts/kconfig/parser.tab.[ch]
HOSTCC scripts/kconfig/lexer.lex.o
HOSTCC scripts/kconfig/menu.o
HOSTCC scripts/kconfig/parser.tab.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/symbol.o
HOSTCC scripts/kconfig/util.o
HOSTLD scripts/kconfig/conf
*** Default configuration is based on 'defconfig'
#
# configuration written to .config
#
make[1]: Leaving directory '/home/masahiro/ref/linux/build-dir'
masahiro@grover:~/ref/linux$ make O=build-dir
make[1]: Entering directory '/home/masahiro/ref/linux/build-dir'
GEN Makefile
SYSHDR arch/alpha/include/generated/uapi/asm/unistd_32.h
SYSTBL arch/alpha/include/generated/asm/syscall_table.h
HOSTCC scripts/kallsyms
WRAP arch/alpha/include/generated/uapi/asm/bpf_perf_event.h
WRAP arch/alpha/include/generated/uapi/asm/ipcbuf.h
WRAP arch/alpha/include/generated/uapi/asm/msgbuf.h
WRAP arch/alpha/include/generated/uapi/asm/poll.h
WRAP arch/alpha/include/generated/uapi/asm/sembuf.h
WRAP arch/alpha/include/generated/uapi/asm/shmbuf.h
WRAP arch/alpha/include/generated/asm/export.h
WRAP arch/alpha/include/generated/asm/kvm_para.h
WRAP arch/alpha/include/generated/asm/mcs_spinlock.h
WRAP arch/alpha/include/generated/asm/compat.h
WRAP arch/alpha/include/generated/asm/current.h
WRAP arch/alpha/include/generated/asm/exec.h
WRAP arch/alpha/include/generated/asm/fb.h
WRAP arch/alpha/include/generated/asm/irq_work.h
WRAP arch/alpha/include/generated/asm/kmap_size.h
WRAP arch/alpha/include/generated/asm/kprobes.h
WRAP arch/alpha/include/generated/asm/local64.h
WRAP arch/alpha/include/generated/asm/mmiowb.h
WRAP arch/alpha/include/generated/asm/module.lds.h
WRAP arch/alpha/include/generated/asm/msi.h
WRAP arch/alpha/include/generated/asm/preempt.h
WRAP arch/alpha/include/generated/asm/sections.h
WRAP arch/alpha/include/generated/asm/simd.h
WRAP arch/alpha/include/generated/asm/softirq_stack.h
WRAP arch/alpha/include/generated/asm/trace_clock.h
WRAP arch/alpha/include/generated/asm/unaligned.h
WRAP arch/alpha/include/generated/asm/vermagic.h
UPD include/config/kernel.release
UPD include/generated/uapi/linux/version.h
UPD include/generated/utsrelease.h
CC scripts/mod/empty.o
HOSTCC scripts/mod/mk_elfconfig
MKELF scripts/mod/elfconfig.h
HOSTCC scripts/mod/modpost.o
CC scripts/mod/devicetable-offsets.s
UPD scripts/mod/devicetable-offsets.h
HOSTCC scripts/mod/file2alias.o
HOSTCC scripts/mod/sumversion.o
HOSTLD scripts/mod/modpost
CC kernel/bounds.s
UPD include/generated/bounds.h
UPD include/generated/timeconst.h
CC arch/alpha/kernel/asm-offsets.s
UPD include/generated/asm-offsets.h
CALL ../scripts/checksyscalls.sh
<stdin>:1515:2: warning: #warning syscall clone3 not implemented [-Wcpp]
CALL ../scripts/atomic/check-atomics.sh
CC init/main.o
CHK include/generated/compile.h
UPD include/generated/compile.h
CC init/version.o
CC init/do_mounts.o
CC init/noinitramfs.o
CC init/calibrate.o
CC init/init_task.o
AR init/built-in.a
AR usr/built-in.a
AS arch/alpha/kernel/head.o
LDS arch/alpha/kernel/vmlinux.lds
AS arch/alpha/kernel/entry.o
CC arch/alpha/kernel/traps.o
CC arch/alpha/kernel/process.o
CC arch/alpha/kernel/osf_sys.o
CC arch/alpha/kernel/irq.o
CC arch/alpha/kernel/irq_alpha.o
CC arch/alpha/kernel/signal.o
CC arch/alpha/kernel/setup.o
CC arch/alpha/kernel/ptrace.o
CC arch/alpha/kernel/time.o
AS arch/alpha/kernel/systbls.o
CC arch/alpha/kernel/err_common.o
CC arch/alpha/kernel/io.o
CC arch/alpha/kernel/bugs.o
CC arch/alpha/kernel/console.o
CC arch/alpha/kernel/pci.o
CC arch/alpha/kernel/pci_iommu.o
CC arch/alpha/kernel/pci-sysfs.o
CC arch/alpha/kernel/module.o
CC arch/alpha/kernel/rtc.o
CC arch/alpha/kernel/core_apecs.o
CC arch/alpha/kernel/core_cia.o
CC arch/alpha/kernel/core_irongate.o
CC arch/alpha/kernel/core_lca.o
CC arch/alpha/kernel/core_mcpcia.o
CC arch/alpha/kernel/core_polaris.o
CC arch/alpha/kernel/core_t2.o
CC arch/alpha/kernel/core_tsunami.o
CC arch/alpha/kernel/sys_alcor.o
CC arch/alpha/kernel/sys_cabriolet.o
CC arch/alpha/kernel/sys_dp264.o
CC arch/alpha/kernel/sys_eb64p.o
CC arch/alpha/kernel/sys_eiger.o
CC arch/alpha/kernel/sys_jensen.o
CC arch/alpha/kernel/sys_miata.o
CC arch/alpha/kernel/sys_mikasa.o
CC arch/alpha/kernel/sys_nautilus.o
CC arch/alpha/kernel/sys_noritake.o
CC arch/alpha/kernel/sys_rawhide.o
CC arch/alpha/kernel/sys_ruffian.o
CC arch/alpha/kernel/sys_rx164.o
CC arch/alpha/kernel/sys_sable.o
CC arch/alpha/kernel/sys_sio.o
CC arch/alpha/kernel/sys_sx164.o
CC arch/alpha/kernel/sys_takara.o
CC arch/alpha/kernel/irq_pyxis.o
CC arch/alpha/kernel/irq_i8259.o
CC arch/alpha/kernel/irq_srm.o
CC arch/alpha/kernel/err_ev6.o
CC arch/alpha/kernel/es1888.o
CC arch/alpha/kernel/smc37c669.o
CC arch/alpha/kernel/smc37c93x.o
CC arch/alpha/kernel/pc873xx.o
CC arch/alpha/kernel/gct.o
CC arch/alpha/kernel/srmcons.o
AR arch/alpha/kernel/built-in.a
CC [M] arch/alpha/kernel/srm_env.o
CC arch/alpha/mm/init.o
CC arch/alpha/mm/fault.o
AR arch/alpha/mm/built-in.a
CC arch/alpha/math-emu/math.o
AS arch/alpha/math-emu/qrnnd.o
AR arch/alpha/math-emu/built-in.a
AR arch/alpha/built-in.a
CC kernel/sched/core.o
CC kernel/sched/loadavg.o
CC kernel/sched/clock.o
CC kernel/sched/cputime.o
CC kernel/sched/idle.o
CC kernel/sched/fair.o
CC kernel/sched/rt.o
CC kernel/sched/deadline.o
CC kernel/sched/wait.o
CC kernel/sched/wait_bit.o
CC kernel/sched/swait.o
CC kernel/sched/completion.o
CC kernel/sched/debug.o
CC kernel/sched/membarrier.o
AR kernel/sched/built-in.a
CC kernel/locking/mutex.o
CC kernel/locking/semaphore.o
CC kernel/locking/rwsem.o
CC kernel/locking/percpu-rwsem.o
CC kernel/locking/rtmutex.o
AR kernel/locking/built-in.a
CC kernel/power/qos.o
CC kernel/power/poweroff.o
AR kernel/power/built-in.a
CC kernel/printk/printk.o
CC kernel/printk/printk_safe.o
CC kernel/printk/printk_ringbuffer.o
AR kernel/printk/built-in.a
CC kernel/irq/irqdesc.o
CC kernel/irq/handle.o
CC kernel/irq/manage.o
CC kernel/irq/spurious.o
CC kernel/irq/resend.o
CC kernel/irq/chip.o
CC kernel/irq/dummychip.o
CC kernel/irq/devres.o
CC kernel/irq/autoprobe.o
CC kernel/irq/proc.o
AR kernel/irq/built-in.a
CC kernel/rcu/update.o
CC kernel/rcu/sync.o
CC kernel/rcu/srcutiny.o
CC kernel/rcu/tiny.o
AR kernel/rcu/built-in.a
AR kernel/livepatch/built-in.a
CC kernel/dma/mapping.o
CC kernel/dma/direct.o
CC kernel/dma/ops_helpers.o
CC kernel/dma/dummy.o
AR kernel/dma/built-in.a
AR kernel/entry/built-in.a
CC kernel/time/time.o
CC kernel/time/timer.o
CC kernel/time/hrtimer.o
CC kernel/time/timekeeping.o
CC kernel/time/ntp.o
CC kernel/time/clocksource.o
CC kernel/time/jiffies.o
CC kernel/time/timer_list.o
CC kernel/time/timeconv.o
CC kernel/time/timecounter.o
CC kernel/time/alarmtimer.o
CC kernel/time/posix-timers.o
CC kernel/time/posix-cpu-timers.o
CC kernel/time/posix-clock.o
CC kernel/time/itimer.o
CC kernel/time/clockevents.o
CC kernel/time/tick-common.o
AR kernel/time/built-in.a
CC kernel/bpf/core.o
AR kernel/bpf/built-in.a
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-27 15:45 ` Masahiro Yamada
@ 2021-07-27 18:46 ` Eric W. Biederman
2021-07-28 22:31 ` Masahiro Yamada
0 siblings, 1 reply; 39+ messages in thread
From: Eric W. Biederman @ 2021-07-27 18:46 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
Fangrui Song, Michal Marek, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
Masahiro Yamada <masahiroy@kernel.org> writes:
> On Tue, Jul 27, 2021 at 11:16 PM Eric W. Biederman
> <ebiederm@xmission.com> wrote:
>>
>> Masahiro Yamada <masahiroy@kernel.org> writes:
>>
>> > On Tue, Jul 27, 2021 at 5:27 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
>> >>
>> >> Masahiro Yamada <masahiroy@kernel.org> writes:
>> >>
>> >> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
>> >> >>
>> >> >> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
>> >> >> <torvalds@linux-foundation.org> wrote:
>> >> >> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>> >> >> >
>> >> >> > We do most of the other heavy lifting in this area in Kconfig anyway,
>> >> >> > why not add that compiler choice?
>> >> >> >
>> >> >> > Obviously it would be gated by the tests to see which compilers are
>> >> >> > _installed_ (and that they are valid versions), so that it doesn't ask
>> >> >> > stupid things ("do you want gcc or clang" when only one of them is
>> >> >> > installed and/or viable).
>> >> >>
>> >> >> I don't see a good way of making Kconfig options both select the
>> >> >> compiler and defining variables based on the compiler, since that
>> >> >> would mean teaching Kconfig about re-evaluating all compiler
>> >> >> dependent settings whenever the first option changes.
>> >> >>
>> >> >> I do have another idea that I think would work though.
>> >> >>
>> >> >> > Hmm? So then any "LLVM=1" thing would be about the "make config"
>> >> >> > stage, not the actual build stage.
>> >> >> >
>> >> >> > (It has annoyed me for years that if you want to cross-compile, you
>> >> >> > first have to do "make ARCH=xyz config" and then remember to do "make
>> >> >> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
>> >> >> > never really cared).
>> >> >>
>> >> >> The best thing that I have come up with is a pre-configure step, where
>> >> >> an object tree gets seeded with a makefile fragment that gets included
>> >> >> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
>> >> >> 'CC=' and possibly any other option that gets passed to 'make' as
>> >> >> a variable and has to exist before calling 'make *config'.
>> >> >
>> >> >
>> >> > There is no need to add a hook to include such makefile fragment(s).
>> >> >
>> >> > Quite opposite, you can put your Makefile (in a different filename)
>> >> > that includes the top Makefile.
>> >> >
>> >> >
>> >> > I think this is what people are already doing:
>> >> >
>> >> >
>> >> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
>> >> > in this order.
>> >> >
>> >> >
>> >> > So, you can put 'GNUmakefile' with your favorite setups.
>> >> >
>> >> >
>> >> > $ cat GNUmakefile
>> >> > ARCH=arm64
>> >> > CROSS_COMPILE=aarch64-linux-gnu-
>> >> > CC=clang
>> >> > include Makefile
>> >>
>> >> Very weird.
>> >>
>> >> I just tested this and it does not work.
>> >> I did this:
>> >>
>> >> $ cat GNUmakefile
>> >> ARCH = alpha
>> >> CROSS_COMPILE = $(arch-prefix alpha)
>> >> include Makefile
>> >>
>> >> In one of my build directories and the main makefile simply does not see
>> >> the value of ARCH or CROSS_COMPILE I set. I have confirmed that my
>> >> GNUmakefile is being read, because everything breaks if I remove the
>> >> include line.
>> >>
>> >> Does anyone have any ideas?
>> >>
>> >> Something so we don't have to specify all of these variables on the make
>> >> command line would be nice.
>> >>
>> >> Eric
>> >
>> >
>> > Worked for me.
>> >
>> > Could you tell me the exact steps you did?
>> >
>> >
>> > This is my case:
>> >
>> > My kernel source tree is located at $HOME/ref/linux
>> > alpha tool chains are located at $HOME/tools/alpha-10.1.0/bin
>> >
>> >
>> >
>> > I tried a simple GNUmakefile with 3 lines.
>> >
>> > You can see 'make' is building the alpha kernel
>> >
>> >
>> > Please see below:
>>
>> Interesting. That appears to work if I don't specify a build directory.
>> Once I specify a build directory with O= it does not work.
>>
>> When I am working on a change that affects multiple architectures
>> I really want a build directory that is not my source tree so I can
>> test small changes on multiple architectures without needing to rebuild
>> everything.
>>
>> Eric
>
>
> Ah, OK.
> When you specify O= option, Kbuild will
> invoke sub-make.
> So you need to add 'export' to the
> variables.
>
> Please see the following modified version.
Yes. That does seem to work and be simpler.
Is there any chance we can get the command that generates the Makefile
in the build directory to include variables like ARCH and CROSS_COMPILE
in the Makefile as well? Just so they don't need to be set by hand to
the same value every time?
Eric
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-27 18:46 ` Eric W. Biederman
@ 2021-07-28 22:31 ` Masahiro Yamada
0 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-28 22:31 UTC (permalink / raw)
To: Eric W. Biederman
Cc: Arnd Bergmann, Linus Torvalds, Nick Desaulniers, Miguel Ojeda,
Fangrui Song, Michal Marek, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Wed, Jul 28, 2021 at 3:46 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
>
> Masahiro Yamada <masahiroy@kernel.org> writes:
>
> > On Tue, Jul 27, 2021 at 11:16 PM Eric W. Biederman
> > <ebiederm@xmission.com> wrote:
> >>
> >> Masahiro Yamada <masahiroy@kernel.org> writes:
> >>
> >> > On Tue, Jul 27, 2021 at 5:27 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
> >> >>
> >> >> Masahiro Yamada <masahiroy@kernel.org> writes:
> >> >>
> >> >> > On Wed, Jul 21, 2021 at 4:58 AM Arnd Bergmann <arnd@kernel.org> wrote:
> >> >> >>
> >> >> >> On Tue, Jul 20, 2021 at 7:43 PM Linus Torvalds
> >> >> >> <torvalds@linux-foundation.org> wrote:
> >> >> >> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >> >> >> >
> >> >> >> > We do most of the other heavy lifting in this area in Kconfig anyway,
> >> >> >> > why not add that compiler choice?
> >> >> >> >
> >> >> >> > Obviously it would be gated by the tests to see which compilers are
> >> >> >> > _installed_ (and that they are valid versions), so that it doesn't ask
> >> >> >> > stupid things ("do you want gcc or clang" when only one of them is
> >> >> >> > installed and/or viable).
> >> >> >>
> >> >> >> I don't see a good way of making Kconfig options both select the
> >> >> >> compiler and defining variables based on the compiler, since that
> >> >> >> would mean teaching Kconfig about re-evaluating all compiler
> >> >> >> dependent settings whenever the first option changes.
> >> >> >>
> >> >> >> I do have another idea that I think would work though.
> >> >> >>
> >> >> >> > Hmm? So then any "LLVM=1" thing would be about the "make config"
> >> >> >> > stage, not the actual build stage.
> >> >> >> >
> >> >> >> > (It has annoyed me for years that if you want to cross-compile, you
> >> >> >> > first have to do "make ARCH=xyz config" and then remember to do "make
> >> >> >> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> >> >> >> > never really cared).
> >> >> >>
> >> >> >> The best thing that I have come up with is a pre-configure step, where
> >> >> >> an object tree gets seeded with a makefile fragment that gets included
> >> >> >> for any 'make' invocation. This would set 'ARCH=', 'CROSS_COMPILE',
> >> >> >> 'CC=' and possibly any other option that gets passed to 'make' as
> >> >> >> a variable and has to exist before calling 'make *config'.
> >> >> >
> >> >> >
> >> >> > There is no need to add a hook to include such makefile fragment(s).
> >> >> >
> >> >> > Quite opposite, you can put your Makefile (in a different filename)
> >> >> > that includes the top Makefile.
> >> >> >
> >> >> >
> >> >> > I think this is what people are already doing:
> >> >> >
> >> >> >
> >> >> > GNU Make looks for 'GNUmakefile', 'makefile', and 'Makefile'
> >> >> > in this order.
> >> >> >
> >> >> >
> >> >> > So, you can put 'GNUmakefile' with your favorite setups.
> >> >> >
> >> >> >
> >> >> > $ cat GNUmakefile
> >> >> > ARCH=arm64
> >> >> > CROSS_COMPILE=aarch64-linux-gnu-
> >> >> > CC=clang
> >> >> > include Makefile
> >> >>
> >> >> Very weird.
> >> >>
> >> >> I just tested this and it does not work.
> >> >> I did this:
> >> >>
> >> >> $ cat GNUmakefile
> >> >> ARCH = alpha
> >> >> CROSS_COMPILE = $(arch-prefix alpha)
> >> >> include Makefile
> >> >>
> >> >> In one of my build directories and the main makefile simply does not see
> >> >> the value of ARCH or CROSS_COMPILE I set. I have confirmed that my
> >> >> GNUmakefile is being read, because everything breaks if I remove the
> >> >> include line.
> >> >>
> >> >> Does anyone have any ideas?
> >> >>
> >> >> Something so we don't have to specify all of these variables on the make
> >> >> command line would be nice.
> >> >>
> >> >> Eric
> >> >
> >> >
> >> > Worked for me.
> >> >
> >> > Could you tell me the exact steps you did?
> >> >
> >> >
> >> > This is my case:
> >> >
> >> > My kernel source tree is located at $HOME/ref/linux
> >> > alpha tool chains are located at $HOME/tools/alpha-10.1.0/bin
> >> >
> >> >
> >> >
> >> > I tried a simple GNUmakefile with 3 lines.
> >> >
> >> > You can see 'make' is building the alpha kernel
> >> >
> >> >
> >> > Please see below:
> >>
> >> Interesting. That appears to work if I don't specify a build directory.
> >> Once I specify a build directory with O= it does not work.
> >>
> >> When I am working on a change that affects multiple architectures
> >> I really want a build directory that is not my source tree so I can
> >> test small changes on multiple architectures without needing to rebuild
> >> everything.
> >>
> >> Eric
> >
> >
> > Ah, OK.
> > When you specify O= option, Kbuild will
> > invoke sub-make.
> > So you need to add 'export' to the
> > variables.
> >
> > Please see the following modified version.
>
> Yes. That does seem to work and be simpler.
>
> Is there any chance we can get the command that generates the Makefile
> in the build directory to include variables like ARCH and CROSS_COMPILE
> in the Makefile as well? Just so they don't need to be set by hand to
> the same value every time?
>
> Eric
I am OK to document this small tip
in Documentation/kbuild/ if it is useful,
but am reluctant to add any code to
generate custom Makefile or GNUmakefile.
It is highly user-dependent after all.
If we merged the code to set up ARCH=...
and CROSS_COMPILE=...,
somebody would send a patch
"I need LLVM=1 in my GNUmakefile", then
another person, "I want CC=clang", and so on.
You can edit it by yourself.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 17:42 ` Linus Torvalds
2021-07-20 19:58 ` Arnd Bergmann
@ 2021-07-20 20:52 ` Nick Desaulniers
2021-07-20 21:11 ` Linus Torvalds
2021-07-21 4:53 ` Masahiro Yamada
2021-07-20 21:29 ` Nick Desaulniers
` (2 subsequent siblings)
4 siblings, 2 replies; 39+ messages in thread
From: Nick Desaulniers @ 2021-07-20 20:52 UTC (permalink / raw)
To: Linus Torvalds
Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
Arnd Bergmann, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Tue, Jul 20, 2021 at 10:43 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > LLVM=1 is a convenient switch to change all the
> > defaults, but yet you can flip each tool individually.
>
> Actually, I'd argue that "LLVM=1" is *not* a convenient switch.
Compared to the old way of CC=clang LD=ld.lld OBJCOPY=.... it certainly is.
> Neither are the individual other command line settings.
Agreed, but we needed flexibility until we could get all of the
command line tools working for each architecture. They're still
useful when there's a regression and we need to fall back. So I
wouldn't be in favor of removing them (not that that's been proposed).
> When clang was the odd man out, and special, it all made sense.
> Changing the path to CC was similar to changing the path to AWK. And
> that's obviously why we have what we have.
>
> But clang has become a primary compiler for some kernel communities,
> and I think it might be time to just re-visit that entirely.
:^)
> In particular, I think we should just make it a Kconfig option. I hate
> the command flag stuff so much, that my clang tree literally has this
> patch in it:
>
> -CC = $(CROSS_COMPILE)gcc
> +CC = $(CROSS_COMPILE)clang
>
> so that I can just do the same "make -j128" in both my gcc tree and my
> clang tree.
So you haven't been using LLD... :( (imagine using more than one
thread to link, and being faster than ld.gold) If anything you should
be hard coding LLVM=1 in that tree. Also, please be careful you don't
accidentally commit that! 0:-)
> But each build tree already has its own .config file, so it would be a
> lot more convenient if that was how the compiler was chosen, and then
> "make oldconfig" would just DTRT.
>
> We do most of the other heavy lifting in this area in Kconfig anyway,
> why not add that compiler choice?
>
> Obviously it would be gated by the tests to see which compilers are
> _installed_ (and that they are valid versions), so that it doesn't ask
> stupid things ("do you want gcc or clang" when only one of them is
> installed and/or viable).
>
> Hmm? So then any "LLVM=1" thing would be about the "make config"
> stage, not the actual build stage.
>
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).
>
> Let the flame wars^H^Hpolite discussions ensue..
I agree with you. Overall the command line invocation of make when
cross compiling, or when using LLVM is too long. You even call out
LLVM=1 and ARCH separately. Each one of these had good reasons to
exist for years.
But I disagree that all needs to be sorted out together, or right now.
And I'd much rather tackle them separately, one by one, than try to
completely rewrite how we cross compile the kernel today.
Right now, we have:
$ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make LLVM=1 LLVM_IAS=1 -j72
This series is concerned with just CROSS_COMPILE (and just for LLVM=1).
Next I plan to default on LLVM_IAS=1 for all architectures we support,
minus ppc and s390 where we still have some assembler bugs.
Your/Arnd's ideas about LLVM=1 or not via Kconfig, or pre-Kconfig is a
good idea for eliminating LLVM=1.
Then that just leaves ARCH.
Arnd's idea about helping you install a toolchain from kernel.org is
one I support, but orthogonal to the above somewhat. Do you allow
someone to have a config that denotes intent to build with clang then
prompt if they don't have clang installed to download it? Or do you
prevent someone from selecting building with clang because it's not in
the $PATH?
Your/Arnd's idea about detecting which toolchains are installed is one
I support, but orthogonal to the above somewhat. (For that, I'm
curious for our build servers if that means having to put tools in
certain locations; I prefer we reference $PATH when possible. Or if
.configs can no longer be shared if tools are in different locations.
But perhaps that's a non-issue). I'm also curious how many stat calls
we'll need to test/probe/find these, and how we prioritize which tools
are selected when there's more than one version installed.
I encourage us to make steps in the right direction; but I think this
series is ready to go for at least one of the command line variables.
I don't think we need to wait for some probing machinery to eliminate
CROSS_COMPILE when LLVM=1; and if we ever get such machinery we can
revisit whether that helps this case at all.
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 20:52 ` Nick Desaulniers
@ 2021-07-20 21:11 ` Linus Torvalds
2021-07-20 21:27 ` Nick Desaulniers
2021-07-21 4:53 ` Masahiro Yamada
1 sibling, 1 reply; 39+ messages in thread
From: Linus Torvalds @ 2021-07-20 21:11 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
Arnd Bergmann, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Tue, Jul 20, 2021 at 1:52 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> > -CC = $(CROSS_COMPILE)gcc
> > +CC = $(CROSS_COMPILE)clang
>
> So you haven't been using LLD... :( (imagine using more than one
> thread to link, and being faster than ld.gold)
I started out just building my own clang - nothing else.
And now that I'm using the distro clang, the linker isn't even _close_
to a performance issue. Since clang is a big, slow, bloated pig.
I posted profiles and performance numbers of clang being three times
slower than gcc, and it was all just clang itself, not the linker. All
due to insanely expensive startup costs.
I suspect a lot of clang users build bloated C++ code where the time
to compile a single object file is so big that the startup costs don't
even show up. But for the (fairly) lean C kernel header files that
don't bring in millions of lines of headers for some template library,
startup costs are a big deal.
So honestly, I don't care at all about "imagine being faster than ld.gold".
I can only _dream_ of clang itself not being 3x slower than gcc.
Linus
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 21:11 ` Linus Torvalds
@ 2021-07-20 21:27 ` Nick Desaulniers
0 siblings, 0 replies; 39+ messages in thread
From: Nick Desaulniers @ 2021-07-20 21:27 UTC (permalink / raw)
To: Linus Torvalds
Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
Arnd Bergmann, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor, Tom Stellard
On Tue, Jul 20, 2021 at 2:11 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Tue, Jul 20, 2021 at 1:52 PM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> >
> > > -CC = $(CROSS_COMPILE)gcc
> > > +CC = $(CROSS_COMPILE)clang
> >
> > So you haven't been using LLD... :( (imagine using more than one
> > thread to link, and being faster than ld.gold)
>
> I started out just building my own clang - nothing else.
>
> And now that I'm using the distro clang, the linker isn't even _close_
> to a performance issue. Since clang is a big, slow, bloated pig.
>
> I posted profiles and performance numbers of clang being three times
> slower than gcc, and it was all just clang itself, not the linker. All
> due to insanely expensive startup costs.
Yeah, it's unfortunate that package maintainers for your distro chose
to distribute clang built in such a manner. But in response to that
thread:
1. I planned a whole conference around how folks distribute LLVM
(particularly how the configurations are done, and important ones to
use):
https://lists.llvm.org/pipermail/llvm-dev/2021-June/150861.html
We have a pretty good set of CFPs lined up (and I need to go chase
folks from Apple and Qualcomm that are working on proposals...)
https://github.com/ClangBuiltLinux/llvm-distributors-conf-2021/issues
2. I brainstormed a whole set of ways to make Clang even faster.
https://github.com/ClangBuiltLinux/tc-build/issues
Most promising I think would be building musl with LTO and statically
linking that into the toolchain. I don't have a ton of time to pursue
that, but I will continue to bang on that drum in front of management.
It's just very hard to fund improving build times for users on
thinkpads when the corporate world has moved on to distributed and
cached building in the cloud.
3. Pretty sure we've resolved that config with your distro's packager,
and made other changes to upstream LLVM to default on some flags that
help a lot when those unfortunate configs are used.
https://reviews.llvm.org/D102090
> I suspect a lot of clang users build bloated C++ code where the time
> to compile a single object file is so big that the startup costs don't
> even show up. But for the (fairly) lean C kernel header files that
> don't bring in millions of lines of headers for some template library,
> startup costs are a big deal.
>
> So honestly, I don't care at all about "imagine being faster than ld.gold".
>
> I can only _dream_ of clang itself not being 3x slower than gcc.
Let's be precise here, because quotes (even when imprecise) lead to PR
cycles. Clang startup time for NULL input when testing command line
flags as part of a build configure step, when clang is misconfigured
is slower than GCC. Clang is not generally 3x slower than GCC.
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 20:52 ` Nick Desaulniers
2021-07-20 21:11 ` Linus Torvalds
@ 2021-07-21 4:53 ` Masahiro Yamada
1 sibling, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-21 4:53 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Linus Torvalds, Miguel Ojeda, Fangrui Song, Michal Marek,
Arnd Bergmann, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Wed, Jul 21, 2021 at 5:52 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Tue, Jul 20, 2021 at 10:43 AM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > LLVM=1 is a convenient switch to change all the
> > > defaults, but yet you can flip each tool individually.
> >
> > Actually, I'd argue that "LLVM=1" is *not* a convenient switch.
>
> Compared to the old way of CC=clang LD=ld.lld OBJCOPY=.... it certainly is.
>
> > Neither are the individual other command line settings.
>
> Agreed, but we needed flexibility until we could get all of the
> command line tools working for each architecture. They're still
> useful when there's a regression and we need to fall back. So I
> wouldn't be in favor of removing them (not that that's been proposed).
>
> > When clang was the odd man out, and special, it all made sense.
> > Changing the path to CC was similar to changing the path to AWK. And
> > that's obviously why we have what we have.
> >
> > But clang has become a primary compiler for some kernel communities,
> > and I think it might be time to just re-visit that entirely.
>
> :^)
>
> > In particular, I think we should just make it a Kconfig option. I hate
> > the command flag stuff so much, that my clang tree literally has this
> > patch in it:
> >
> > -CC = $(CROSS_COMPILE)gcc
> > +CC = $(CROSS_COMPILE)clang
> >
> > so that I can just do the same "make -j128" in both my gcc tree and my
> > clang tree.
>
> So you haven't been using LLD... :( (imagine using more than one
> thread to link, and being faster than ld.gold) If anything you should
> be hard coding LLVM=1 in that tree. Also, please be careful you don't
> accidentally commit that! 0:-)
>
> > But each build tree already has its own .config file, so it would be a
> > lot more convenient if that was how the compiler was chosen, and then
> > "make oldconfig" would just DTRT.
> >
> > We do most of the other heavy lifting in this area in Kconfig anyway,
> > why not add that compiler choice?
> >
> > Obviously it would be gated by the tests to see which compilers are
> > _installed_ (and that they are valid versions), so that it doesn't ask
> > stupid things ("do you want gcc or clang" when only one of them is
> > installed and/or viable).
> >
> > Hmm? So then any "LLVM=1" thing would be about the "make config"
> > stage, not the actual build stage.
> >
> > (It has annoyed me for years that if you want to cross-compile, you
> > first have to do "make ARCH=xyz config" and then remember to do "make
> > ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> > never really cared).
> >
> > Let the flame wars^H^Hpolite discussions ensue..
>
> I agree with you. Overall the command line invocation of make when
> cross compiling, or when using LLVM is too long. You even call out
> LLVM=1 and ARCH separately. Each one of these had good reasons to
> exist for years.
>
> But I disagree that all needs to be sorted out together, or right now.
> And I'd much rather tackle them separately, one by one, than try to
> completely rewrite how we cross compile the kernel today.
>
> Right now, we have:
> $ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make LLVM=1 LLVM_IAS=1 -j72
>
> This series is concerned with just CROSS_COMPILE (and just for LLVM=1).
> Next I plan to default on LLVM_IAS=1 for all architectures we support,
> minus ppc and s390 where we still have some assembler bugs.
> Your/Arnd's ideas about LLVM=1 or not via Kconfig, or pre-Kconfig is a
> good idea for eliminating LLVM=1.
I also like to make the integrated assembler our default.
We can add LLVM_DISABLE_IAS=1 to
replace LLVM_IAS=1.
> Then that just leaves ARCH.
> Arnd's idea about helping you install a toolchain from kernel.org is
> one I support, but orthogonal to the above somewhat. Do you allow
> someone to have a config that denotes intent to build with clang then
> prompt if they don't have clang installed to download it? Or do you
> prevent someone from selecting building with clang because it's not in
> the $PATH?
> Your/Arnd's idea about detecting which toolchains are installed is one
> I support, but orthogonal to the above somewhat. (For that, I'm
> curious for our build servers if that means having to put tools in
> certain locations; I prefer we reference $PATH when possible. Or if
> .configs can no longer be shared if tools are in different locations.
> But perhaps that's a non-issue). I'm also curious how many stat calls
> we'll need to test/probe/find these, and how we prioritize which tools
> are selected when there's more than one version installed.
>
> I encourage us to make steps in the right direction; but I think this
> series is ready to go for at least one of the command line variables.
> I don't think we need to wait for some probing machinery to eliminate
> CROSS_COMPILE when LLVM=1; and if we ever get such machinery we can
> revisit whether that helps this case at all.
> --
> Thanks,
> ~Nick Desaulniers
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 17:42 ` Linus Torvalds
2021-07-20 19:58 ` Arnd Bergmann
2021-07-20 20:52 ` Nick Desaulniers
@ 2021-07-20 21:29 ` Nick Desaulniers
2021-07-20 21:54 ` Linus Torvalds
2021-07-21 4:31 ` Masahiro Yamada
2021-07-21 4:44 ` Christoph Hellwig
4 siblings, 1 reply; 39+ messages in thread
From: Nick Desaulniers @ 2021-07-20 21:29 UTC (permalink / raw)
To: Linus Torvalds
Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
Arnd Bergmann, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Tue, Jul 20, 2021 at 10:43 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> But clang has become a primary compiler for some kernel communities,
> and I think it might be time to just re-visit that entirely.
>
> In particular, I think we should just make it a Kconfig option. I hate
> the command flag stuff so much, that my clang tree literally has this
> patch in it:
>
> -CC = $(CROSS_COMPILE)gcc
> +CC = $(CROSS_COMPILE)clang
>
> so that I can just do the same "make -j128" in both my gcc tree and my
> clang tree.
>
> But each build tree already has its own .config file, so it would be a
> lot more convenient if that was how the compiler was chosen, and then
> "make oldconfig" would just DTRT.
>
> We do most of the other heavy lifting in this area in Kconfig anyway,
> why not add that compiler choice?
>
> Obviously it would be gated by the tests to see which compilers are
> _installed_ (and that they are valid versions), so that it doesn't ask
> stupid things ("do you want gcc or clang" when only one of them is
> installed and/or viable).
>
> Hmm? So then any "LLVM=1" thing would be about the "make config"
> stage, not the actual build stage.
>
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).
>
> Let the flame wars^H^Hpolite discussions ensue..
I will concede that "why do I need to respecify all of my command line
args/env vars to make when I already did so for a configure step?" is
a reasonable question to ask, and may be worth pursuing.
There's still some that we can eliminate for BOTH configure and actual
build, like CROSS_COMPILE. :^)
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 21:29 ` Nick Desaulniers
@ 2021-07-20 21:54 ` Linus Torvalds
2021-07-20 23:19 ` Linus Torvalds
2021-07-21 4:52 ` Christoph Hellwig
0 siblings, 2 replies; 39+ messages in thread
From: Linus Torvalds @ 2021-07-20 21:54 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
Arnd Bergmann, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Tue, Jul 20, 2021 at 2:29 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> There's still some that we can eliminate for BOTH configure and actual
> build, like CROSS_COMPILE. :^)
I have this (perhaps odd) model where I use separate trees for
different things, which is something git obviously is partly designed
for.
So when I create a source tree, I usually create it for one particular
use - whether it be for one particular configuration (ie "build with
clang" or "set up a tree for a 32-bit cross compile" or whatever), or
whether it be for some particular development issue ("long-term branch
for testing asm goto with inputs").
Yeah, sometimes I use multiple git branches in the same tree for
things - but I tend to do that mainly for short-term things where I
don't expect to keep working on it over time.
End result: I find 'make' command line flags to be really really
annoying, because then I have to match them to whatever tree I'm in.
I'd much rather have a local per-tree configuration that just sticks
around. It's why ".config" works fairly well for me.
But there are other things that stick around. In particular, I have
considered simply using git config variables for that.
Something like this in the main Makefile:
ARCH=$(shell git config build.arch)
CC=$(shell git config --default=gcc build.cc)
would actually be optimal for what _I_ do. Then for my clang build tree, I'd do
git config build.cc clang
and it would stick in that tree. If you don't set the option, it would use gcc.
Maybe that would be an acceptable model these days? We've used git
long enough that non-git worries aren't a big deal any more, and I
guess people could use the old-fashioned
make CC=clang
if they don't use git.
We could get rid of all the CROSS_COMPILE etc garbage that way,
because you'd just set it in the .git/config file. Those are really
convenient and easy to edit.
Would people hate that kind of model?
Linus
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 21:54 ` Linus Torvalds
@ 2021-07-20 23:19 ` Linus Torvalds
2021-07-20 23:22 ` Linus Torvalds
2021-07-21 5:12 ` Masahiro Yamada
2021-07-21 4:52 ` Christoph Hellwig
1 sibling, 2 replies; 39+ messages in thread
From: Linus Torvalds @ 2021-07-20 23:19 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
Arnd Bergmann, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
[-- Attachment #1: Type: text/plain, Size: 649 bytes --]
On Tue, Jul 20, 2021 at 2:54 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> But there are other things that stick around. In particular, I have
> considered simply using git config variables for that.
>
> Something like this in the main Makefile:
A slightly more fleshed-out patch to do this just to see if somebody
wants to work on something like this.
It really would seem to make it very easy to set up any random build
environment, and not have to remember it once it's done.
There are probably better ways to do this, I normally try to avoid
doing Makefile magic, since others have taken it over. Masahiro?
Linus
[-- Attachment #2: patch.diff --]
[-- Type: application/x-patch, Size: 2083 bytes --]
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 23:19 ` Linus Torvalds
@ 2021-07-20 23:22 ` Linus Torvalds
2021-07-21 5:12 ` Masahiro Yamada
1 sibling, 0 replies; 39+ messages in thread
From: Linus Torvalds @ 2021-07-20 23:22 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Masahiro Yamada, Miguel Ojeda, Fangrui Song, Michal Marek,
Arnd Bergmann, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Tue, Jul 20, 2021 at 4:19 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> There are probably better ways to do this
Oh. In particular, it probably should use a '?=' in order to allow
overriding, and not expanding that $(shell git config..) thing every
time $(CC) is used.
Or something. I didn't really think about those things very much.
Think of that patch - as so many of my patches - as a starting point
and a "how about something like this".
Linus
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 23:19 ` Linus Torvalds
2021-07-20 23:22 ` Linus Torvalds
@ 2021-07-21 5:12 ` Masahiro Yamada
1 sibling, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-21 5:12 UTC (permalink / raw)
To: Linus Torvalds
Cc: Nick Desaulniers, Miguel Ojeda, Fangrui Song, Michal Marek,
Arnd Bergmann, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Wed, Jul 21, 2021 at 8:20 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Tue, Jul 20, 2021 at 2:54 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > But there are other things that stick around. In particular, I have
> > considered simply using git config variables for that.
> >
> > Something like this in the main Makefile:
>
> A slightly more fleshed-out patch to do this just to see if somebody
> wants to work on something like this.
>
> It really would seem to make it very easy to set up any random build
> environment, and not have to remember it once it's done.
>
> There are probably better ways to do this, I normally try to avoid
> doing Makefile magic, since others have taken it over. Masahiro?
The patch would uglify the Makefile.
If git is not installed, $(shell git config ...)
will spit an error message, so '2>/dev/null' is needed.
Actually, I believe we should be able to build the kernel
from a source tarball without relying on git in any way.
In fact, we rely on git in some places, for example,
scripts/setlocalversion can add commit hash to the release
string, but I want to minimize the git-dependent code.
> Linus
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAHk-%3DwhS1zBU%3DuhDTk2M1LwkqG7AKLsR0_%2BXG%2BsaY_s2FHZr-A%40mail.gmail.com.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 21:54 ` Linus Torvalds
2021-07-20 23:19 ` Linus Torvalds
@ 2021-07-21 4:52 ` Christoph Hellwig
2021-07-21 5:33 ` Masahiro Yamada
1 sibling, 1 reply; 39+ messages in thread
From: Christoph Hellwig @ 2021-07-21 4:52 UTC (permalink / raw)
To: Linus Torvalds
Cc: Nick Desaulniers, Masahiro Yamada, Miguel Ojeda, Fangrui Song,
Michal Marek, Arnd Bergmann, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Tue, Jul 20, 2021 at 02:54:35PM -0700, Linus Torvalds wrote:
> But there are other things that stick around. In particular, I have
> considered simply using git config variables for that.
>
> Something like this in the main Makefile:
>
> ARCH=$(shell git config build.arch)
> CC=$(shell git config --default=gcc build.cc)
>
> would actually be optimal for what _I_ do. Then for my clang build tree, I'd do
>
> git config build.cc clang
>
> and it would stick in that tree. If you don't set the option, it would use gcc.
>
> Maybe that would be an acceptable model these days? We've used git
> long enough that non-git worries aren't a big deal any more, and I
> guess people could use the old-fashioned
I still this only papers around the fact that the architecture is
fundamentally part of the kernel config and should come from .config,
and the toolchain probably as well.
I for one do have a few different tree for different projects, but
need to try cross compiles in each of them. And unfortunately sometimes
with clang (nothing against clang itself [1], just that having to deal
with multiple toolchains is a pain)
[1] well, except for the sometimes idiotic diagnostics..
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-21 4:52 ` Christoph Hellwig
@ 2021-07-21 5:33 ` Masahiro Yamada
0 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-21 5:33 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Linus Torvalds, Nick Desaulniers, Miguel Ojeda, Fangrui Song,
Michal Marek, Arnd Bergmann, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Nathan Chancellor
On Wed, Jul 21, 2021 at 1:53 PM Christoph Hellwig <hch@infradead.org> wrote:
>
> On Tue, Jul 20, 2021 at 02:54:35PM -0700, Linus Torvalds wrote:
> > But there are other things that stick around. In particular, I have
> > considered simply using git config variables for that.
> >
> > Something like this in the main Makefile:
> >
> > ARCH=$(shell git config build.arch)
> > CC=$(shell git config --default=gcc build.cc)
> >
> > would actually be optimal for what _I_ do. Then for my clang build tree, I'd do
> >
> > git config build.cc clang
> >
> > and it would stick in that tree. If you don't set the option, it would use gcc.
> >
> > Maybe that would be an acceptable model these days? We've used git
> > long enough that non-git worries aren't a big deal any more, and I
> > guess people could use the old-fashioned
>
> I still this only papers around the fact that the architecture is
> fundamentally part of the kernel config
In fact, I am the person who did it in U-boot several years ago.
In U-Boot Kconfig, the architecture is selected by a Kconfig choice:
https://github.com/u-boot/u-boot/blob/v2021.07/arch/Kconfig#L22
(but, if you look at Makefiles and Kconfigs of U-Boot,
you will notice they are uglier than the kernel's ones.)
Theoretically, yes, it is possible to make ARCH as part of the .config
in the kernel as well, but we need careful consideration to make
it robust, clean, and efficient.
In the current model, the Kconfig stage would be even slower
because Kconfig would end up with a bunch of unrelated cc-option
from other architectures.
For example, arch/s390/Kconfig has so many cc-option.
If we allow Kconfig to choose "ARCH', all Kconfig files
of all architectures must be parsed.
If you intend to build the x86 kernel, you might think that
parsing the expensive arch/s390/Kconfig is pointless.
> and should come from .config,
> and the toolchain probably as well.
>
> I for one do have a few different tree for different projects, but
> need to try cross compiles in each of them. And unfortunately sometimes
> with clang (nothing against clang itself [1], just that having to deal
> with multiple toolchains is a pain)
>
> [1] well, except for the sometimes idiotic diagnostics..
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 17:42 ` Linus Torvalds
` (2 preceding siblings ...)
2021-07-20 21:29 ` Nick Desaulniers
@ 2021-07-21 4:31 ` Masahiro Yamada
2021-07-21 4:44 ` Christoph Hellwig
4 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2021-07-21 4:31 UTC (permalink / raw)
To: Linus Torvalds
Cc: Nick Desaulniers, Miguel Ojeda, Fangrui Song, Michal Marek,
Arnd Bergmann, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Wed, Jul 21, 2021 at 2:43 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Tue, Jul 20, 2021 at 1:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > LLVM=1 is a convenient switch to change all the
> > defaults, but yet you can flip each tool individually.
>
> Actually, I'd argue that "LLVM=1" is *not* a convenient switch.
> Neither are the individual other command line settings.
>
> When clang was the odd man out, and special, it all made sense.
> Changing the path to CC was similar to changing the path to AWK. And
> that's obviously why we have what we have.
>
> But clang has become a primary compiler for some kernel communities,
> and I think it might be time to just re-visit that entirely.
>
> In particular, I think we should just make it a Kconfig option. I hate
> the command flag stuff so much, that my clang tree literally has this
> patch in it:
>
> -CC = $(CROSS_COMPILE)gcc
> +CC = $(CROSS_COMPILE)clang
>
> so that I can just do the same "make -j128" in both my gcc tree and my
> clang tree.
>
> But each build tree already has its own .config file, so it would be a
> lot more convenient if that was how the compiler was chosen, and then
> "make oldconfig" would just DTRT.
>
> We do most of the other heavy lifting in this area in Kconfig anyway,
> why not add that compiler choice?
This does not work in the current design.
When we moved compiler tests to Kconfig from Makefile at v4.18,
I discussed a lot about the implementation.
Especially 'static' vs 'dynamic'.
(mostly Ulf Magnusson actively gave me feed backs)
static model
Kconfig is given with a fixed set of toolchains.
All macros such as cc-option are expanded in the early parse stage
before proceeding to the configuration stage.
dynamic model
Every time a user toggles a CONFIG option, it feeds back to
cc-option because CONFIG options (such as CONFIG_64BIT)
might affect cc-option. In this model, CONFIG_CROSS_COMPILE
works well.
Of course, the dynamic model is much more complex,
and I could not find a sane syntax to achieve it.
So, I implemented the static model; in this model,
$(cc-option ) are all constant values.
So, you cannot do:
choice
prompt "Compiler suite"
config GCC
"GCC and GNU binutils"
config LLVM
"LLVM (clang and llvm tools)"
endchoice
To achieve this, we need the dynamic model.
Every time a user switches between CONFIG_GCC and CONFIG_LLVM,
all the cc-options must be re-evaluated.
This does not work in the current implementation.
BTW, keeping the command line interface
such as CROSS_COMPILE=... CC=... LD=...
is useful when the kernel is built from umbrella
projects such as Buildroot, OpenEmbedded, etc.
Such projects build compilers by themselves,
and feed the exact compiler paths to the kernel.
Too long command line is an annoyance when
we build the kernel manually.
>
> Obviously it would be gated by the tests to see which compilers are
> _installed_ (and that they are valid versions), so that it doesn't ask
> stupid things ("do you want gcc or clang" when only one of them is
> installed and/or viable).
>
> Hmm? So then any "LLVM=1" thing would be about the "make config"
> stage, not the actual build stage.
>
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).
>
> Let the flame wars^H^Hpolite discussions ensue..
>
> Linus
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: [PATCH v2 2/2] Makefile: infer CROSS_COMPILE from SRCARCH for LLVM=1 LLVM_IAS=1
2021-07-20 17:42 ` Linus Torvalds
` (3 preceding siblings ...)
2021-07-21 4:31 ` Masahiro Yamada
@ 2021-07-21 4:44 ` Christoph Hellwig
4 siblings, 0 replies; 39+ messages in thread
From: Christoph Hellwig @ 2021-07-21 4:44 UTC (permalink / raw)
To: Linus Torvalds
Cc: Masahiro Yamada, Nick Desaulniers, Miguel Ojeda, Fangrui Song,
Michal Marek, Arnd Bergmann, Linux Kernel Mailing List,
Linux Kbuild mailing list, clang-built-linux, Geert Uytterhoeven,
Christoph Hellwig, Nathan Chancellor
On Tue, Jul 20, 2021 at 10:42:57AM -0700, Linus Torvalds wrote:
> (It has annoyed me for years that if you want to cross-compile, you
> first have to do "make ARCH=xyz config" and then remember to do "make
> ARCH=xyz" for the build too, but I cross-compile so seldom that I've
> never really cared).
Same here. I'd so much love to just have ARCH an CROSS_COMPILE in my
.config for cross-builds. Oterwise I keep forgetting them, or keep
forgetting to unset CROSS_COMPILE and will just waist a lot of time
either way.
The fact the the .config files even mention what architecture they are
for in a comment but don't make use of that information is just hostile
to users.
^ permalink raw reply [flat|nested] 39+ messages in thread