linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Masahiro Yamada <yamada.masahiro@socionext.com>
To: linux-kbuild <linux-kbuild@vger.kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	Sam Ravnborg <sam@ravnborg.org>,
	Ulf Magnusson <ulfalizer@gmail.com>,
	Nicholas Piggin <npiggin@gmail.com>,
	Emese Revfy <re.emese@gmail.com>, X86 ML <x86@kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Kees Cook <keescook@chromium.org>
Subject: Re: [PATCH 00/30] kconfig: move compiler capability tests to Kconfig
Date: Fri, 13 Apr 2018 22:55:59 +0900	[thread overview]
Message-ID: <CAK7LNAQxCvBnntT-X5o_NNiv2wYga=pBQnYeni7D+5V4CGaHNw@mail.gmail.com> (raw)
In-Reply-To: <CAK7LNASrWiJZ0qOvjut5BxRXahgQZ7gjpF1WRvCGxvF2Qq74PA@mail.gmail.com>

2018-04-13 21:21 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
> 2018-04-13 14:52 GMT+09:00 Kees Cook <keescook@chromium.org>:
>> On Thu, Apr 12, 2018 at 10:06 PM, Masahiro Yamada
>> <yamada.masahiro@socionext.com> wrote:
>>> [Major Changes in V3]
>>
>> Awesome work! I don't see this pushed to your git tree? I'd like to
>> test it, but I'd rather "git fetch" instead of "git am" :)
>>
>> -Kees
>>
>
> I pushed this series to the following branch.
>
> git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> kconfig-shell-v3
>


If this approach is successful, we will move more and more
compiler option tests to the Kconfig stage in the future.

People (including me) might be worried about
how slow Kconfig will become.



First, I compared the before/after on my PC.


Without this series,

masahiro@grover:~/workspace/linux-kbuild$ time make -s defconfig

real 0m0.175s
user 0m0.128s
sys 0m0.008s



With this series,
masahiro@grover:~/workspace/linux-kbuild$ time make -s defconfig

real 0m0.729s
user 0m0.400s
sys 0m0.056s


This is noticeable difference.


Then, I looked into per-commit analysis.

Here is the result of the real time of 'time make -s defconfig'

[30/30] kbuild: test dead code/data elimination...     0m0.719s
[29/30] arm64: move GCC version check for...           0m0.711s
[28/30] gcc-plugins: allow to enable GCC_PLUGINS...    0m0.722s
[27/30] gcc-plugins: test plugin support in...         0m0.719s   [+0.31]
[26/30] gcc-plugins: move GCC version check...         0m0.410s
[25/30] kcov: test compiler capability in...           0m0.392s
[24/30] gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT     0m0.400s
[23/30] kconfig: add CC_IS_CLANG and CLANG_VERSION     0m0.396s
[22/30] kconfig: add CC_IS_GCC and GCC_VERSION         0m0.392s
[21/30] stack-protector: test compiler capability...   0m0.381s   [+0.04]
[20/30] kconfig: add basic helper macros to...         0m0.343s
[19/30] kconfig: show compiler version text...         0m0.345s
[18/30] kconfig: test: test text expansion...          0m0.342s
[17/30] Documentation: kconfig: document...            0m0.344s
[16/30] kconfig: add 'info' and 'warning'...           0m0.347s
[15/30] kconfig: expand lefthand side of...            0m0.340s
[14/30] kconfig: support append assignment...          0m0.342s
[13/30] kconfig: support simply expanded...            0m0.341s
[12/30] kconfig: support variable and...               0m0.344s
[11/30] kconfig: begin PARAM state only...             0m0.342s
[10/30] kconfig: replace $(UNAME_RELEASE)...           0m0.347s
[09/30] kconfig: add 'shell' built-in function         0m0.344s
[08/30] kconfig: add built-in function support         0m0.350s
[07/30] kconfig: remove sym_expand_string_value()      0m0.344s
[06/30] kconfig: remove string expansion...            0m0.349s
[05/30] kconfig: remove string expansion...            0m0.342s
[04/30] kconfig: reference environment...              0m0.342s
[03/30] kbuild: remove CONFIG_CROSS_COMPILE...         0m0.347s
[02/30] kbuild: remove kbuild cache                    0m0.347s  [+0.17]
[01/30] gcc-plugins: fix build condition...            0m0.171s
[00/30] Merge tag 'drm-fixes-for-v4.17-rc1'...         0m0.176s


There are three big jump points.

The first one is [02/30]  (+0.17)
We are removing the build cache, so this is what we expect.


The second one is  [21/30]  (+0.04)
For x86, Kconfig runs scripts/gcc-x86_{32,64}-has-stack-protector.sh


The biggest one is [27/30]  (+0.31)
scripts/gcc-plugins.sh is probably very costly script.
If we bump the minimum gcc version to GCC 4.8
the script will be much cleaner in the future.


I was also interested in the cost of
a single $(cc-option ...) invocation.

It is pretty easy to measure this.


For example, copy $(cc-option -fstack-protector)
1000 lines like follows.

config FOO
       bool
       default $(cc-option -fstack-protector)
       default $(cc-option -fstack-protector)
       default $(cc-option -fstack-protector)
       default $(cc-option -fstack-protector)
       default $(cc-option -fstack-protector)
       default $(cc-option -fstack-protector)
       default $(cc-option -fstack-protector)
       default $(cc-option -fstack-protector)
       default $(cc-option -fstack-protector)
       default $(cc-option -fstack-protector)
         ...  [ repeat 1000 line ]



On my core i7 PC, it took 7.2 msec
to run $(cc-option -fstack-protector) 1000 times.


We can make it much faster.

Currently we use
   $(CC) -Werror $(1) -c -x c /dev/null
to test the compiler flag.


Ulf Magnusson suggested to use -S instead of -c
(https://patchwork.kernel.org/patch/10309297/)
With -S, the compiler stops after the compilation stage.
It took only 4.0 msec
to run $(cc-option -fstack-protector) 1000 times


If I use -E  (only pre-process stage), it becomes even faster.
It took only 2.6 msec.


As for $(cc-option ...), probably this will not be a problem.

For some feature, we need special shell-scripts,
some of which can be more costly.




-- 
Best Regards
Masahiro Yamada

      reply	other threads:[~2018-04-13 13:56 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-04-13  5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
2018-04-13  5:06 ` [PATCH 01/30] gcc-plugins: fix build condition of SANCOV plugin Masahiro Yamada
2018-05-04 14:21   ` Masahiro Yamada
2018-05-04 16:21     ` Kees Cook
2018-05-05  1:35       ` Masahiro Yamada
2018-04-13  5:06 ` [PATCH 02/30] kbuild: remove kbuild cache Masahiro Yamada
2018-04-13  5:06 ` [PATCH 03/30] kbuild: remove CONFIG_CROSS_COMPILE support Masahiro Yamada
2018-04-13  5:06 ` [PATCH 04/30] kconfig: reference environment variables directly and remove 'option env=' Masahiro Yamada
2018-04-13  5:06 ` [PATCH 05/30] kconfig: remove string expansion in file_lookup() Masahiro Yamada
2018-04-13  5:06 ` [PATCH 06/30] kconfig: remove string expansion for mainmenu after yyparse() Masahiro Yamada
2018-04-13  5:06 ` [PATCH 07/30] kconfig: remove sym_expand_string_value() Masahiro Yamada
2018-04-13  5:06 ` [PATCH 08/30] kconfig: add built-in function support Masahiro Yamada
2018-04-15  7:57   ` Ulf Magnusson
2018-04-15 15:12     ` Masahiro Yamada
2018-04-13  5:06 ` [PATCH 09/30] kconfig: add 'shell' built-in function Masahiro Yamada
2018-04-13  5:06 ` [PATCH 10/30] kconfig: replace $(UNAME_RELEASE) with function call Masahiro Yamada
2018-04-13  5:06 ` [PATCH 11/30] kconfig: begin PARAM state only when seeing a command keyword Masahiro Yamada
2018-04-13  5:06 ` [PATCH 12/30] kconfig: support variable and user-defined function Masahiro Yamada
2018-04-13  5:06 ` [PATCH 13/30] kconfig: support simply expanded variable Masahiro Yamada
2018-04-13  5:06 ` [PATCH 14/30] kconfig: support append assignment operator Masahiro Yamada
2018-04-13  5:06 ` [PATCH 15/30] kconfig: expand lefthand side of assignment statement Masahiro Yamada
2018-04-13  5:06 ` [PATCH 16/30] kconfig: add 'info' and 'warning' built-in functions Masahiro Yamada
2018-04-13  5:06 ` [PATCH 17/30] Documentation: kconfig: document a new Kconfig macro language Masahiro Yamada
2018-04-14 23:33   ` Randy Dunlap
2018-04-17 15:05     ` Masahiro Yamada
2018-04-15  8:08   ` Ulf Magnusson
2018-04-17 15:07     ` Masahiro Yamada
2018-04-18  8:33       ` Ulf Magnusson
2018-04-13  5:06 ` [PATCH 18/30] kconfig: test: test text expansion Masahiro Yamada
2018-04-13  5:06 ` [PATCH 19/30] kconfig: show compiler version text in the top comment Masahiro Yamada
2018-04-13  5:06 ` [PATCH 20/30] kconfig: add basic helper macros to scripts/Kconfig.include Masahiro Yamada
2018-04-15  7:41   ` Ulf Magnusson
2018-04-15 15:02     ` Masahiro Yamada
2018-04-13  5:06 ` [PATCH 21/30] stack-protector: test compiler capability in Kconfig and drop AUTO mode Masahiro Yamada
2018-04-13 16:41   ` Kees Cook
2018-04-13 18:11     ` Linus Torvalds
2018-04-13 20:41       ` Kees Cook
2018-04-15 13:28       ` Masahiro Yamada
2018-04-15 16:04         ` Kees Cook
2018-04-15  9:40     ` Masahiro Yamada
2018-04-13  5:06 ` [PATCH 22/30] kconfig: add CC_IS_GCC and GCC_VERSION Masahiro Yamada
2018-04-13  5:06 ` [PATCH 23/30] kconfig: add CC_IS_CLANG and CLANG_VERSION Masahiro Yamada
2018-04-13  5:06 ` [PATCH 24/30] gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT Masahiro Yamada
2018-04-13  5:06 ` [PATCH 25/30] kcov: test compiler capability in Kconfig and correct dependency Masahiro Yamada
2018-04-13  5:06 ` [PATCH 26/30] gcc-plugins: move GCC version check for PowerPC to Kconfig Masahiro Yamada
2018-04-13  5:06 ` [PATCH 27/30] gcc-plugins: test plugin support in Kconfig and clean up Makefile Masahiro Yamada
2018-04-13  5:06 ` [PATCH 28/30] gcc-plugins: allow to enable GCC_PLUGINS for COMPILE_TEST Masahiro Yamada
2018-04-13  5:06 ` [PATCH 29/30] arm64: move GCC version check for ARCH_SUPPORTS_INT128 to Kconfig Masahiro Yamada
2018-04-13  5:06 ` [PATCH 30/30] kbuild: test dead code/data elimination support in Kconfig Masahiro Yamada
2018-04-13  5:17 ` [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
2018-04-13  5:52 ` Kees Cook
2018-04-13 12:21   ` Masahiro Yamada
2018-04-13 13:55     ` Masahiro Yamada [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAK7LNAQxCvBnntT-X5o_NNiv2wYga=pBQnYeni7D+5V4CGaHNw@mail.gmail.com' \
    --to=yamada.masahiro@socionext.com \
    --cc=keescook@chromium.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=npiggin@gmail.com \
    --cc=re.emese@gmail.com \
    --cc=sam@ravnborg.org \
    --cc=torvalds@linux-foundation.org \
    --cc=ulfalizer@gmail.com \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).