[RFC PATCH] kbuild: Add an option to enable -O1 and speed-up compilation time

[RFC PATCH] kbuild: Add an option to enable -O1 and speed-up compilation time

[Christophe JAILLET]

Add a new compilation option which speeds-up compilation time.
This can be useful when using static checker such as smatch or build-bots.
In such cases, the speed and quality of the generated code is not
important.

Using -O0 would be even better, but unfortunately, building fails with
this option.

Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
---
 Makefile     | 5 ++++-
 init/Kconfig | 8 ++++++++
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 1f8bef92868f..14467386f947 100644
--- a/Makefile
+++ b/Makefile
@@ -817,7 +817,10 @@ KBUILD_CFLAGS	+= $(call cc-disable-warning, format-truncation)
 KBUILD_CFLAGS	+= $(call cc-disable-warning, format-overflow)
 KBUILD_CFLAGS	+= $(call cc-disable-warning, address-of-packed-member)
 
-ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE
+ifdef CONFIG_CC_OPTIMIZE_FOR_COMPILATION_SPEED
+KBUILD_CFLAGS += -O1
+KBUILD_RUSTFLAGS_OPT_LEVEL_MAP := 1
+else ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE
 KBUILD_CFLAGS += -O2
 KBUILD_RUSTFLAGS_OPT_LEVEL_MAP := 2
 else ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE_O3
diff --git a/init/Kconfig b/init/Kconfig
index a96776a9b080..3177a1830c9a 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1384,6 +1384,14 @@ choice
 	prompt "Compiler optimization level"
 	default CC_OPTIMIZE_FOR_PERFORMANCE
 
+config CC_OPTIMIZE_FOR_COMPILATION_SPEED
+	bool "Optimize for compilation speed (-O1)"
+	help
+	  This option can be useful when running a static checker such as smatch
+	  or a build-bot.
+	  Compilation time is slighly faster than -O2 and it requires less
+	  memory.
+
 config CC_OPTIMIZE_FOR_PERFORMANCE
 	bool "Optimize for performance (-O2)"
 	help
-- 
2.34.1

Re: [RFC PATCH] kbuild: Add an option to enable -O1 and speed-up compilation time

[Masahiro Yamada]

(+CC: Arnd, Changbin Du)


If -O1 support does not require additional effort,
I have no objection to this patch.

(but I do not have enough insight about
the compiler's inlining heuristic)



BTW, when we attempted to add the -Og support,
we fixed various parts, and Linus rejected it.

https://lore.kernel.org/linux-kbuild/CAK7LNARQggM3aKEPRKJqa4tunFAfmfErMZuS-rrnRv6UB1VpPQ@mail.gmail.com/







On Sat, May 21, 2022 at 6:04 PM Christophe JAILLET
<christophe.jaillet@wanadoo.fr> wrote:
>
> Add a new compilation option which speeds-up compilation time.
> This can be useful when using static checker such as smatch or build-bots.
> In such cases, the speed and quality of the generated code is not
> important.
>
> Using -O0 would be even better, but unfortunately, building fails with
> this option.
>
> Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
> ---
>  Makefile     | 5 ++++-
>  init/Kconfig | 8 ++++++++
>  2 files changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/Makefile b/Makefile
> index 1f8bef92868f..14467386f947 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -817,7 +817,10 @@ KBUILD_CFLAGS      += $(call cc-disable-warning, format-truncation)
>  KBUILD_CFLAGS  += $(call cc-disable-warning, format-overflow)
>  KBUILD_CFLAGS  += $(call cc-disable-warning, address-of-packed-member)
>
> -ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE
> +ifdef CONFIG_CC_OPTIMIZE_FOR_COMPILATION_SPEED
> +KBUILD_CFLAGS += -O1
> +KBUILD_RUSTFLAGS_OPT_LEVEL_MAP := 1
> +else ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE
>  KBUILD_CFLAGS += -O2
>  KBUILD_RUSTFLAGS_OPT_LEVEL_MAP := 2
>  else ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE_O3
> diff --git a/init/Kconfig b/init/Kconfig
> index a96776a9b080..3177a1830c9a 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -1384,6 +1384,14 @@ choice
>         prompt "Compiler optimization level"
>         default CC_OPTIMIZE_FOR_PERFORMANCE
>
> +config CC_OPTIMIZE_FOR_COMPILATION_SPEED
> +       bool "Optimize for compilation speed (-O1)"
> +       help
> +         This option can be useful when running a static checker such as smatch
> +         or a build-bot.
> +         Compilation time is slighly faster than -O2 and it requires less
> +         memory.
> +
>  config CC_OPTIMIZE_FOR_PERFORMANCE
>         bool "Optimize for performance (-O2)"
>         help
> --
> 2.34.1
>


-- 
Best Regards
Masahiro Yamada

Re: [RFC PATCH] kbuild: Add an option to enable -O1 and speed-up compilation time

[Changbin Du]

On Sun, May 22, 2022 at 12:38:17AM +0900, Masahiro Yamada wrote:
> (+CC: Arnd, Changbin Du)
> 
> 
> If -O1 support does not require additional effort,
> I have no objection to this patch.
> 
> (but I do not have enough insight about
> the compiler's inlining heuristic)
> 
> 
> 
> BTW, when we attempted to add the -Og support,
> we fixed various parts, and Linus rejected it.
> 
> https://lore.kernel.org/linux-kbuild/CAK7LNARQggM3aKEPRKJqa4tunFAfmfErMZuS-rrnRv6UB1VpPQ@mail.gmail.com/
> 
> 
> 
> 
> 
> 
> 
I am afraid that '-O1' has the same situation with '-Og'. As described in GCC
mannual:

Like -O0, -Og completely disables a number of optimization passes so that 
individual options controlling them have no effect. Otherwise -Og enables all
-O1 optimization flags except for those that may interfere with debugging:
 -fbranch-count-reg  -fdelayed-branch 
 -fdse  -fif-conversion  -fif-conversion2  
 -finline-functions-called-once 
 -fmove-loop-invariants  -fmove-loop-stores  -fssa-phiopt 
 -ftree-bit-ccp  -ftree-dse  -ftree-pta  -ftree-sra


> On Sat, May 21, 2022 at 6:04 PM Christophe JAILLET
> <christophe.jaillet@wanadoo.fr> wrote:
> >
> > Add a new compilation option which speeds-up compilation time.
> > This can be useful when using static checker such as smatch or build-bots.
> > In such cases, the speed and quality of the generated code is not
> > important.
> >
> > Using -O0 would be even better, but unfortunately, building fails with
> > this option.
> >
> > Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
> > ---
> >  Makefile     | 5 ++++-
> >  init/Kconfig | 8 ++++++++
> >  2 files changed, 12 insertions(+), 1 deletion(-)
> >
> > diff --git a/Makefile b/Makefile
> > index 1f8bef92868f..14467386f947 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -817,7 +817,10 @@ KBUILD_CFLAGS      += $(call cc-disable-warning, format-truncation)
> >  KBUILD_CFLAGS  += $(call cc-disable-warning, format-overflow)
> >  KBUILD_CFLAGS  += $(call cc-disable-warning, address-of-packed-member)
> >
> > -ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE
> > +ifdef CONFIG_CC_OPTIMIZE_FOR_COMPILATION_SPEED
> > +KBUILD_CFLAGS += -O1
> > +KBUILD_RUSTFLAGS_OPT_LEVEL_MAP := 1
> > +else ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE
> >  KBUILD_CFLAGS += -O2
> >  KBUILD_RUSTFLAGS_OPT_LEVEL_MAP := 2
> >  else ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE_O3
> > diff --git a/init/Kconfig b/init/Kconfig
> > index a96776a9b080..3177a1830c9a 100644
> > --- a/init/Kconfig
> > +++ b/init/Kconfig
> > @@ -1384,6 +1384,14 @@ choice
> >         prompt "Compiler optimization level"
> >         default CC_OPTIMIZE_FOR_PERFORMANCE
> >
> > +config CC_OPTIMIZE_FOR_COMPILATION_SPEED
> > +       bool "Optimize for compilation speed (-O1)"
> > +       help
> > +         This option can be useful when running a static checker such as smatch
> > +         or a build-bot.
> > +         Compilation time is slighly faster than -O2 and it requires less
> > +         memory.
> > +
> >  config CC_OPTIMIZE_FOR_PERFORMANCE
> >         bool "Optimize for performance (-O2)"
> >         help
> > --
> > 2.34.1
> >
> 
> 
> -- 
> Best Regards
> Masahiro Yamada

-- 
Cheers,
Changbin Du

Re: [RFC PATCH] kbuild: Add an option to enable -O1 and speed-up compilation time

[Arnd Bergmann]

On Mon, May 23, 2022 at 2:34 PM Changbin Du <changbin.du@gmail.com> wrote:
> On Sun, May 22, 2022 at 12:38:17AM +0900, Masahiro Yamada wrote:
> > (+CC: Arnd, Changbin Du)
> >
> >
> > If -O1 support does not require additional effort,
> > I have no objection to this patch.
> >
> > (but I do not have enough insight about
> > the compiler's inlining heuristic)
> >
> >
> >
> > BTW, when we attempted to add the -Og support,
> > we fixed various parts, and Linus rejected it.
> >
> > https://lore.kernel.org/linux-kbuild/CAK7LNARQggM3aKEPRKJqa4tunFAfmfErMZuS-rrnRv6UB1VpPQ@mail.gmail.com/
> >
> >
> >
> >
> >
> >
> >
> I am afraid that '-O1' has the same situation with '-Og'. As described in GCC
> mannual:
>
> Like -O0, -Og completely disables a number of optimization passes so that
> individual options controlling them have no effect. Otherwise -Og enables all
> -O1 optimization flags except for those that may interfere with debugging:
>  -fbranch-count-reg  -fdelayed-branch
>  -fdse  -fif-conversion  -fif-conversion2
>  -finline-functions-called-once
>  -fmove-loop-invariants  -fmove-loop-stores  -fssa-phiopt
>  -ftree-bit-ccp  -ftree-dse  -ftree-pta  -ftree-sra

Correct, I'm fairly sure this never worked. It may be possible to narrow down
the individual optimization flags to have something that works inbetween
-O1 and -O2, but that requires a lot of testing work, and it is questionable
whether there are any practical upsides.

Note that we already disable some optimizations for certain configurations,
e.g. CONFIG_READABLE_ASM, CONFIG_FRAME_POINTER or
CONFIG_DEBUG_SECTION_MISMATCH. The gcc manual suggests
listing all optimization flags with

$ gcc -c -Q -O1 --help=optimizers > O1-opts
$ gcc -c -Q -O2 --help=optimizers > O2-opts
$ diff -u O1-opts O2-opts

If the goal is to speed up compilation, there may be value in trying
which of the various options make the most difference here and
still produce a working kernel when disabled. I know we need the
inlining to happen or things go badly wrong, but I don't know if e.g.
-fno-expensive-optimizations would work.

       Arnd

Re: [RFC PATCH] kbuild: Add an option to enable -O1 and speed-up compilation time

[Nick Desaulniers]

On Sat, May 21, 2022 at 2:04 AM Christophe JAILLET
<christophe.jaillet@wanadoo.fr> wrote:
>
> Add a new compilation option which speeds-up compilation time.
> This can be useful when using static checker such as smatch or build-bots.
> In such cases, the speed and quality of the generated code is not
> important.
>
> Using -O0 would be even better, but unfortunately, building fails with
> this option.

Which is a tragedy.

As with the Rust series, I'm not a fan of this (or
CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE_O3):
https://lore.kernel.org/lkml/CAKwvOd=7QTUH69+ZbT7e8einvgcosTbDkyohmPaUBv6_y8RfrQ@mail.gmail.com/

These feel more like attempts to wrap every conceivable command line
flag in a kconfig option, which makes me think of that meme from
Jurassic Park: "your scientists were so preoccupied with whether or
not they could, they didn't stop to think if they should."
Not a fan.  I'd ask for measurements, but that would be a request for
a "rock fetching quest" for something I still wouldn't be a fan of.

>
> Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
> ---
>  Makefile     | 5 ++++-
>  init/Kconfig | 8 ++++++++
>  2 files changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/Makefile b/Makefile
> index 1f8bef92868f..14467386f947 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -817,7 +817,10 @@ KBUILD_CFLAGS      += $(call cc-disable-warning, format-truncation)
>  KBUILD_CFLAGS  += $(call cc-disable-warning, format-overflow)
>  KBUILD_CFLAGS  += $(call cc-disable-warning, address-of-packed-member)
>
> -ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE
> +ifdef CONFIG_CC_OPTIMIZE_FOR_COMPILATION_SPEED
> +KBUILD_CFLAGS += -O1
> +KBUILD_RUSTFLAGS_OPT_LEVEL_MAP := 1
> +else ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE
>  KBUILD_CFLAGS += -O2
>  KBUILD_RUSTFLAGS_OPT_LEVEL_MAP := 2
>  else ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE_O3
> diff --git a/init/Kconfig b/init/Kconfig
> index a96776a9b080..3177a1830c9a 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -1384,6 +1384,14 @@ choice
>         prompt "Compiler optimization level"
>         default CC_OPTIMIZE_FOR_PERFORMANCE
>
> +config CC_OPTIMIZE_FOR_COMPILATION_SPEED
> +       bool "Optimize for compilation speed (-O1)"
> +       help
> +         This option can be useful when running a static checker such as smatch
> +         or a build-bot.
> +         Compilation time is slighly faster than -O2 and it requires less

s/slighly/slightly/

> +         memory.
> +
>  config CC_OPTIMIZE_FOR_PERFORMANCE
>         bool "Optimize for performance (-O2)"
>         help
> --
> 2.34.1
>


-- 
Thanks,
~Nick Desaulniers

Re: [RFC PATCH] kbuild: Add an option to enable -O1 and speed-up compilation time

[Christophe JAILLET]

Le 23/05/2022 à 20:26, Nick Desaulniers a écrit :
> On Sat, May 21, 2022 at 2:04 AM Christophe JAILLET
> <christophe.jaillet@wanadoo.fr> wrote:
>>
>> Add a new compilation option which speeds-up compilation time.
>> This can be useful when using static checker such as smatch or build-bots.
>> In such cases, the speed and quality of the generated code is not
>> important.
>>
>> Using -O0 would be even better, but unfortunately, building fails with
>> this option.
> 
> Which is a tragedy.
> 
> As with the Rust series, I'm not a fan of this (or
> CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE_O3):
> https://lore.kernel.org/lkml/CAKwvOd=7QTUH69+ZbT7e8einvgcosTbDkyohmPaUBv6_y8RfrQ@mail.gmail.com/
> 
> These feel more like attempts to wrap every conceivable command line
> flag in a kconfig option, which makes me think of that meme from
> Jurassic Park: "your scientists were so preoccupied with whether or
> not they could, they didn't stop to think if they should."
> Not a fan.  I'd ask for measurements, but that would be a request for
> a "rock fetching quest" for something I still wouldn't be a fan of.

Hi,

some preliminary tests gave roughly a 5% build time speed-up with -O1.
That is to say that the impact seems quite limited, after all.
This was done on building /drivers/net/ or /drivers/net/ethernet/ only.

I don't have numbers for the impact on memory consumption.

Given the feedbacks on -O1 which is unlikely to be able to build a full 
kernel and your position on yet another option for a very specific use 
case, I won't push the idea further.

I'll keep it in my own tree for (small) build time improvement when 
running smatch.

The idea of turning off a subset of -O2 options would just be some too 
fine tuning. I'll certainly try it for my own use, but it would end to 
an un-understandable list of flags in Makefile.
-O1 (or -O0) was simple. A more complex solution is way too much.


Thanks for your feedbacks.

CJ

> 
>>
>> Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
>> ---
>>   Makefile     | 5 ++++-
>>   init/Kconfig | 8 ++++++++
>>   2 files changed, 12 insertions(+), 1 deletion(-)
>>
>> diff --git a/Makefile b/Makefile
>> index 1f8bef92868f..14467386f947 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -817,7 +817,10 @@ KBUILD_CFLAGS      += $(call cc-disable-warning, format-truncation)
>>   KBUILD_CFLAGS  += $(call cc-disable-warning, format-overflow)
>>   KBUILD_CFLAGS  += $(call cc-disable-warning, address-of-packed-member)
>>
>> -ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE
>> +ifdef CONFIG_CC_OPTIMIZE_FOR_COMPILATION_SPEED
>> +KBUILD_CFLAGS += -O1
>> +KBUILD_RUSTFLAGS_OPT_LEVEL_MAP := 1
>> +else ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE
>>   KBUILD_CFLAGS += -O2
>>   KBUILD_RUSTFLAGS_OPT_LEVEL_MAP := 2
>>   else ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE_O3
>> diff --git a/init/Kconfig b/init/Kconfig
>> index a96776a9b080..3177a1830c9a 100644
>> --- a/init/Kconfig
>> +++ b/init/Kconfig
>> @@ -1384,6 +1384,14 @@ choice
>>          prompt "Compiler optimization level"
>>          default CC_OPTIMIZE_FOR_PERFORMANCE
>>
>> +config CC_OPTIMIZE_FOR_COMPILATION_SPEED
>> +       bool "Optimize for compilation speed (-O1)"
>> +       help
>> +         This option can be useful when running a static checker such as smatch
>> +         or a build-bot.
>> +         Compilation time is slighly faster than -O2 and it requires less
> 
> s/slighly/slightly/

Thanks for taking time for a full review, even on something you are not 
fan on. This is really appreciated.

CJ

> 
>> +         memory.
>> +
>>   config CC_OPTIMIZE_FOR_PERFORMANCE
>>          bool "Optimize for performance (-O2)"
>>          help
>> --
>> 2.34.1
>>
> 
> 

Re: [RFC PATCH] kbuild: Add an option to enable -O1 and speed-up compilation time

[Nick Desaulniers]

On Mon, May 23, 2022 at 1:16 PM Christophe JAILLET
<christophe.jaillet@wanadoo.fr> wrote:
>
> Thanks for taking time for a full review, even on something you are not
> fan on. This is really appreciated.

Of course! It's still a question worth asking, and having a lore link
to refer back to should the question arise again in the future. Thanks
for asking.
-- 
Thanks,
~Nick Desaulniers

Re: [RFC PATCH] kbuild: Add an option to enable -O1 and speed-up compilation time

[Miguel Ojeda]

On Mon, May 23, 2022 at 8:27 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> As with the Rust series, I'm not a fan of this (or
> CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE_O3):
> https://lore.kernel.org/lkml/CAKwvOd=7QTUH69+ZbT7e8einvgcosTbDkyohmPaUBv6_y8RfrQ@mail.gmail.com/

I think it should be decided whether we want extra levels or not, and
apply that policy to both the C and Rust sides.

If we don't, then yeah, let's remove them in both sides and just let
users modify `CFLAGS` as needed.

If we do, then having them as config options makes sense so that they
can be kept working.

Cheers,
Miguel

Re: [RFC PATCH] kbuild: Add an option to enable -O1 and speed-up compilation time

[Miguel Ojeda]

Hi Christophe,

On Mon, May 23, 2022 at 9:11 AM Christophe JAILLET
<christophe.jaillet@wanadoo.fr> wrote:
>
> +KBUILD_RUSTFLAGS_OPT_LEVEL_MAP := 1

The patch seems based on linux-next -- is this intended? Thanks!

Cheers,
Miguel

Re: [RFC PATCH] kbuild: Add an option to enable -O1 and speed-up compilation time

[Christophe JAILLET]

Le 23/05/2022 à 22:25, Miguel Ojeda a écrit :
> Hi Christophe,
> 
> On Mon, May 23, 2022 at 9:11 AM Christophe JAILLET
> <christophe.jaillet@wanadoo.fr> wrote:
>>
>> +KBUILD_RUSTFLAGS_OPT_LEVEL_MAP := 1
> 
> The patch seems based on linux-next -- is this intended? Thanks!

Yes, I work on -next.

Honestly, this "RUST flag" was just a mimic of the KBUILD_CFLAGS.
No real opinion on it, I just put something that looked consistent for a 
patch that was only a RFC.

CJ

> 
> Cheers,
> Miguel
> 

Re: [RFC PATCH] kbuild: Add an option to enable -O1 and speed-up compilation time

[Miguel Ojeda]

On Mon, May 23, 2022 at 10:44 PM Christophe JAILLET
<christophe.jaillet@wanadoo.fr> wrote:
>
> Yes, I work on -next.
>
> Honestly, this "RUST flag" was just a mimic of the KBUILD_CFLAGS.
> No real opinion on it, I just put something that looked consistent for a
> patch that was only a RFC.

Understood, thanks!

Cheers,
Miguel