* [PATCH 01/30] gcc-plugins: fix build condition of SANCOV plugin
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-05-04 14:21 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 02/30] kbuild: remove kbuild cache Masahiro Yamada
` (30 subsequent siblings)
31 siblings, 1 reply; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Since commit d677a4d60193 ("Makefile: support flag
-fsanitizer-coverage=trace-cmp"), you miss to build the SANCOV
plugin under some circumstances.
CONFIG_KCOV=y
CONFIG_KCOV_ENABLE_COMPARISONS=y
Your compiler does not support -fsanitize-coverage=trace-pc
Your compiler does not support -fsanitize-coverage=trace-cmp
Under this condition, $(CFLAGS_KCOV) is not empty but contains a
space, so the following ifeq-conditional is false.
ifeq ($(CFLAGS_KCOV),)
Then, scripts/Makefile.gcc-plugins misses to add sancov_plugin.so to
gcc-plugin-y while the SANCOV plugin is necessary as an alternative
means.
Fixes: d677a4d60193 ("Makefile: support flag -fsanitizer-coverage=trace-cmp")
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3:
- newly added
Changes in v2: None
scripts/Makefile.gcc-plugins | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins
index b2a95af..7f5c862 100644
--- a/scripts/Makefile.gcc-plugins
+++ b/scripts/Makefile.gcc-plugins
@@ -14,7 +14,7 @@ ifdef CONFIG_GCC_PLUGINS
endif
ifdef CONFIG_GCC_PLUGIN_SANCOV
- ifeq ($(CFLAGS_KCOV),)
+ ifeq ($(strip $(CFLAGS_KCOV)),)
# It is needed because of the gcc-plugin.sh and gcc version checks.
gcc-plugin-$(CONFIG_GCC_PLUGIN_SANCOV) += sancov_plugin.so
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [PATCH 01/30] gcc-plugins: fix build condition of SANCOV plugin
2018-04-13 5:06 ` [PATCH 01/30] gcc-plugins: fix build condition of SANCOV plugin Masahiro Yamada
@ 2018-05-04 14:21 ` Masahiro Yamada
2018-05-04 16:21 ` Kees Cook
0 siblings, 1 reply; 53+ messages in thread
From: Masahiro Yamada @ 2018-05-04 14:21 UTC (permalink / raw)
To: Kees Cook
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Emese Revfy, X86 ML, Masahiro Yamada, Linux Kernel Mailing List,
Linux Kbuild mailing list
Hi Kees,
2018-04-13 14:06 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
> Since commit d677a4d60193 ("Makefile: support flag
> -fsanitizer-coverage=trace-cmp"), you miss to build the SANCOV
> plugin under some circumstances.
>
> CONFIG_KCOV=y
> CONFIG_KCOV_ENABLE_COMPARISONS=y
> Your compiler does not support -fsanitize-coverage=trace-pc
> Your compiler does not support -fsanitize-coverage=trace-cmp
>
> Under this condition, $(CFLAGS_KCOV) is not empty but contains a
> space, so the following ifeq-conditional is false.
>
> ifeq ($(CFLAGS_KCOV),)
>
> Then, scripts/Makefile.gcc-plugins misses to add sancov_plugin.so to
> gcc-plugin-y while the SANCOV plugin is necessary as an alternative
> means.
>
> Fixes: d677a4d60193 ("Makefile: support flag -fsanitizer-coverage=trace-cmp")
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> ---
I am planning to queue this up to the fixes branch
since this is a bug fix.
Do you have any comment on this?
> Changes in v3:
> - newly added
>
> Changes in v2: None
>
> scripts/Makefile.gcc-plugins | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins
> index b2a95af..7f5c862 100644
> --- a/scripts/Makefile.gcc-plugins
> +++ b/scripts/Makefile.gcc-plugins
> @@ -14,7 +14,7 @@ ifdef CONFIG_GCC_PLUGINS
> endif
>
> ifdef CONFIG_GCC_PLUGIN_SANCOV
> - ifeq ($(CFLAGS_KCOV),)
> + ifeq ($(strip $(CFLAGS_KCOV)),)
> # It is needed because of the gcc-plugin.sh and gcc version checks.
> gcc-plugin-$(CONFIG_GCC_PLUGIN_SANCOV) += sancov_plugin.so
>
> --
> 2.7.4
>
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH 01/30] gcc-plugins: fix build condition of SANCOV plugin
2018-05-04 14:21 ` Masahiro Yamada
@ 2018-05-04 16:21 ` Kees Cook
2018-05-05 1:35 ` Masahiro Yamada
0 siblings, 1 reply; 53+ messages in thread
From: Kees Cook @ 2018-05-04 16:21 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Emese Revfy, X86 ML, Linux Kernel Mailing List,
Linux Kbuild mailing list
On Fri, May 4, 2018 at 7:21 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> Hi Kees,
>
>
> 2018-04-13 14:06 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
>> Since commit d677a4d60193 ("Makefile: support flag
>> -fsanitizer-coverage=trace-cmp"), you miss to build the SANCOV
>> plugin under some circumstances.
>>
>> CONFIG_KCOV=y
>> CONFIG_KCOV_ENABLE_COMPARISONS=y
>> Your compiler does not support -fsanitize-coverage=trace-pc
>> Your compiler does not support -fsanitize-coverage=trace-cmp
>>
>> Under this condition, $(CFLAGS_KCOV) is not empty but contains a
>> space, so the following ifeq-conditional is false.
>>
>> ifeq ($(CFLAGS_KCOV),)
>>
>> Then, scripts/Makefile.gcc-plugins misses to add sancov_plugin.so to
>> gcc-plugin-y while the SANCOV plugin is necessary as an alternative
>> means.
>>
>> Fixes: d677a4d60193 ("Makefile: support flag -fsanitizer-coverage=trace-cmp")
>> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
>> ---
>
>
> I am planning to queue this up to the fixes branch
> since this is a bug fix.
>
> Do you have any comment on this?
Looks fine to me; thanks!
Acked-by: Kees Cook <keescook@chromium.org>
-Kees
>
>
>
>
>
>> Changes in v3:
>> - newly added
>>
>> Changes in v2: None
>>
>> scripts/Makefile.gcc-plugins | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins
>> index b2a95af..7f5c862 100644
>> --- a/scripts/Makefile.gcc-plugins
>> +++ b/scripts/Makefile.gcc-plugins
>> @@ -14,7 +14,7 @@ ifdef CONFIG_GCC_PLUGINS
>> endif
>>
>> ifdef CONFIG_GCC_PLUGIN_SANCOV
>> - ifeq ($(CFLAGS_KCOV),)
>> + ifeq ($(strip $(CFLAGS_KCOV)),)
>> # It is needed because of the gcc-plugin.sh and gcc version checks.
>> gcc-plugin-$(CONFIG_GCC_PLUGIN_SANCOV) += sancov_plugin.so
>>
>> --
>> 2.7.4
>>
>
>
>
> --
> Best Regards
> Masahiro Yamada
--
Kees Cook
Pixel Security
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH 01/30] gcc-plugins: fix build condition of SANCOV plugin
2018-05-04 16:21 ` Kees Cook
@ 2018-05-05 1:35 ` Masahiro Yamada
0 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-05-05 1:35 UTC (permalink / raw)
To: Kees Cook
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Emese Revfy, X86 ML, Linux Kernel Mailing List,
Linux Kbuild mailing list
2018-05-05 1:21 GMT+09:00 Kees Cook <keescook@chromium.org>:
> On Fri, May 4, 2018 at 7:21 AM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> Hi Kees,
>>
>>
>> 2018-04-13 14:06 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
>>> Since commit d677a4d60193 ("Makefile: support flag
>>> -fsanitizer-coverage=trace-cmp"), you miss to build the SANCOV
>>> plugin under some circumstances.
>>>
>>> CONFIG_KCOV=y
>>> CONFIG_KCOV_ENABLE_COMPARISONS=y
>>> Your compiler does not support -fsanitize-coverage=trace-pc
>>> Your compiler does not support -fsanitize-coverage=trace-cmp
>>>
>>> Under this condition, $(CFLAGS_KCOV) is not empty but contains a
>>> space, so the following ifeq-conditional is false.
>>>
>>> ifeq ($(CFLAGS_KCOV),)
>>>
>>> Then, scripts/Makefile.gcc-plugins misses to add sancov_plugin.so to
>>> gcc-plugin-y while the SANCOV plugin is necessary as an alternative
>>> means.
>>>
>>> Fixes: d677a4d60193 ("Makefile: support flag -fsanitizer-coverage=trace-cmp")
>>> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
>>> ---
>>
>>
>> I am planning to queue this up to the fixes branch
>> since this is a bug fix.
>>
>> Do you have any comment on this?
>
> Looks fine to me; thanks!
>
> Acked-by: Kees Cook <keescook@chromium.org>
>
> -Kees
>
>>
Applied to linux-kbuild/fixes.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 53+ messages in thread
* [PATCH 02/30] kbuild: remove kbuild cache
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
2018-04-13 5:06 ` [PATCH 01/30] gcc-plugins: fix build condition of SANCOV plugin Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 03/30] kbuild: remove CONFIG_CROSS_COMPILE support Masahiro Yamada
` (29 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
The kbuild cache was introduced to remember the result of shell
commands, some of which are expensive to compute, such as
$(call cc-option,...).
However, this turned out not so clever as I had first expected.
Actually, it is problematic. For example, "$(CC) -print-file-name"
is cached. If the compiler is updated, the stale search path causes
build error, which is difficult to figure out. Another problem
scenario is cache files could be touched while install targets are
running under the root permission. We can patch them if desired,
but the build infrastructure is getting uglier and uglier.
Now, we are going to move compiler flag tests to the configuration
phase. If this is completed, the result of compiler tests will be
naturally cached in the .config file. We will not have performance
issues of incremental building since this testing only happens at
Kconfig time.
To start this work with a cleaner code base, remove the kbuild
cache first.
Revert the following commits:
Commit 9a234a2e3843 ("kbuild: create directory for make cache only when necessary")
Commit e17c400ae194 ("kbuild: shrink .cache.mk when it exceeds 1000 lines")
Commit 4e56207130ed ("kbuild: Cache a few more calls to the compiler")
Commit 3298b690b21c ("kbuild: Add a cache for generated variables")
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---
Changes in v3: None
Changes in v2: None
Makefile | 5 +--
scripts/Kbuild.include | 101 +++++++------------------------------------------
2 files changed, 16 insertions(+), 90 deletions(-)
diff --git a/Makefile b/Makefile
index c1a608a..d85cee0 100644
--- a/Makefile
+++ b/Makefile
@@ -501,7 +501,7 @@ RETPOLINE_CFLAGS := $(call cc-option,$(RETPOLINE_CFLAGS_GCC),$(call cc-option,$(
export RETPOLINE_CFLAGS
# check for 'asm goto'
-ifeq ($(call shell-cached,$(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC) $(KBUILD_CFLAGS)), y)
+ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC) $(KBUILD_CFLAGS)), y)
CC_HAVE_ASM_GOTO := 1
KBUILD_CFLAGS += -DCC_HAVE_ASM_GOTO
KBUILD_AFLAGS += -DCC_HAVE_ASM_GOTO
@@ -804,7 +804,7 @@ KBUILD_CFLAGS += $(call cc-option,-fdata-sections,)
endif
# arch Makefile may override CC so keep this after arch Makefile is included
-NOSTDINC_FLAGS += -nostdinc -isystem $(call shell-cached,$(CC) -print-file-name=include)
+NOSTDINC_FLAGS += -nostdinc -isystem $(shell $(CC) -print-file-name=include)
CHECKFLAGS += $(NOSTDINC_FLAGS)
# warn about C99 declaration after statement
@@ -1617,7 +1617,6 @@ clean: $(clean-dirs)
-o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \
-o -name '*.symtypes' -o -name 'modules.order' \
-o -name modules.builtin -o -name '.tmp_*.o.*' \
- -o -name .cache.mk \
-o -name '*.c.[012]*.*' \
-o -name '*.ll' \
-o -name '*.gcno' \) -type f -print | xargs rm -f
diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
index cce31ee..9f7eb10 100644
--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -8,8 +8,6 @@ squote := '
empty :=
space := $(empty) $(empty)
space_escape := _-_SPACE_-_
-right_paren := )
-left_paren := (
###
# Name of target with a '.' as filename prefix. foo/bar.o => foo/.bar.o
@@ -82,71 +80,6 @@ cc-cross-prefix = \
echo $(c); \
fi)))
-# Tools for caching Makefile variables that are "expensive" to compute.
-#
-# Here we want to help deal with variables that take a long time to compute
-# by making it easy to store these variables in a cache.
-#
-# The canonical example here is testing for compiler flags. On a simple system
-# each call to the compiler takes 10 ms, but on a system with a compiler that's
-# called through various wrappers it can take upwards of 100 ms. If we have
-# 100 calls to the compiler this can take 1 second (on a simple system) or 10
-# seconds (on a complicated system).
-#
-# The "cache" will be in Makefile syntax and can be directly included.
-# Any time we try to reference a variable that's not in the cache we'll
-# calculate it and store it in the cache for next time.
-
-# Include values from last time
-make-cache := $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/,$(if $(obj),$(obj)/)).cache.mk
-$(make-cache): ;
--include $(make-cache)
-
-cached-data := $(filter __cached_%, $(.VARIABLES))
-
-# If cache exceeds 1000 lines, shrink it down to 500.
-ifneq ($(word 1000,$(cached-data)),)
-$(shell tail -n 500 $(make-cache) > $(make-cache).tmp; \
- mv $(make-cache).tmp $(make-cache))
-endif
-
-create-cache-dir := $(if $(KBUILD_SRC),$(if $(cache-data),,1))
-
-# Usage: $(call __sanitize-opt,Hello=Hola$(comma)Goodbye Adios)
-#
-# Convert all '$', ')', '(', '\', '=', ' ', ',', ':' to '_'
-__sanitize-opt = $(subst $$,_,$(subst $(right_paren),_,$(subst $(left_paren),_,$(subst \,_,$(subst =,_,$(subst $(space),_,$(subst $(comma),_,$(subst :,_,$(1)))))))))
-
-# Usage: $(call shell-cached,shell_command)
-# Example: $(call shell-cached,md5sum /usr/bin/gcc)
-#
-# If we've already seen a call to this exact shell command (even in a
-# previous invocation of make!) we'll return the value. If not, we'll
-# compute it and store the result for future runs.
-#
-# This is a bit of voodoo, but basic explanation is that if the variable
-# was undefined then we'll evaluate the shell command and store the result
-# into the variable. We'll then store that value in the cache and finally
-# output the value.
-#
-# NOTE: The $$(2) here isn't actually a parameter to __run-and-store. We
-# happen to know that the caller will have their shell command in $(2) so the
-# result of "call"ing this will produce a reference to that $(2). The reason
-# for this strangeness is to avoid an extra level of eval (and escaping) of
-# $(2).
-define __run-and-store
-ifeq ($(origin $(1)),undefined)
- $$(eval $(1) := $$(shell $$(2)))
-ifeq ($(create-cache-dir),1)
- $$(shell mkdir -p $(dir $(make-cache)))
- $$(eval create-cache-dir :=)
-endif
- $$(shell echo '$(1) := $$($(1))' >> $(make-cache))
-endif
-endef
-__shell-cached = $(eval $(call __run-and-store,$(1)))$($(1))
-shell-cached = $(call __shell-cached,__cached_$(call __sanitize-opt,$(1)),$(1))
-
# output directory for tests below
TMPOUT := $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/)
@@ -154,36 +87,30 @@ TMPOUT := $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/)
# Usage: option = $(call try-run, $(CC)...-o "$$TMP",option-ok,otherwise)
# Exit code chooses option. "$$TMP" serves as a temporary file and is
# automatically cleaned up.
-__try-run = set -e; \
+try-run = $(shell set -e; \
TMP="$(TMPOUT).$$$$.tmp"; \
TMPO="$(TMPOUT).$$$$.o"; \
if ($(1)) >/dev/null 2>&1; \
then echo "$(2)"; \
else echo "$(3)"; \
fi; \
- rm -f "$$TMP" "$$TMPO"
-
-try-run = $(shell $(__try-run))
-
-# try-run-cached
-# This works like try-run, but the result is cached.
-try-run-cached = $(call shell-cached,$(__try-run))
+ rm -f "$$TMP" "$$TMPO")
# as-option
# Usage: cflags-y += $(call as-option,-Wa$(comma)-isa=foo,)
-as-option = $(call try-run-cached,\
+as-option = $(call try-run,\
$(CC) $(KBUILD_CFLAGS) $(1) -c -x assembler /dev/null -o "$$TMP",$(1),$(2))
# as-instr
# Usage: cflags-y += $(call as-instr,instr,option1,option2)
-as-instr = $(call try-run-cached,\
+as-instr = $(call try-run,\
printf "%b\n" "$(1)" | $(CC) $(KBUILD_AFLAGS) -c -x assembler -o "$$TMP" -,$(2),$(3))
# __cc-option
# Usage: MY_CFLAGS += $(call __cc-option,$(CC),$(MY_CFLAGS),-march=winchip-c6,-march=i586)
-__cc-option = $(call try-run-cached,\
+__cc-option = $(call try-run,\
$(1) -Werror $(2) $(3) -c -x c /dev/null -o "$$TMP",$(3),$(4))
# Do not attempt to build with gcc plugins during cc-option tests.
@@ -203,23 +130,23 @@ hostcc-option = $(call __cc-option, $(HOSTCC),\
# cc-option-yn
# Usage: flag := $(call cc-option-yn,-march=winchip-c6)
-cc-option-yn = $(call try-run-cached,\
+cc-option-yn = $(call try-run,\
$(CC) -Werror $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) $(1) -c -x c /dev/null -o "$$TMP",y,n)
# cc-disable-warning
# Usage: cflags-y += $(call cc-disable-warning,unused-but-set-variable)
-cc-disable-warning = $(call try-run-cached,\
+cc-disable-warning = $(call try-run,\
$(CC) -Werror $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1)))
# cc-name
# Expands to either gcc or clang
-cc-name = $(call shell-cached,$(CC) -v 2>&1 | grep -q "clang version" && echo clang || echo gcc)
+cc-name = $(shell $(CC) -v 2>&1 | grep -q "clang version" && echo clang || echo gcc)
# cc-version
-cc-version = $(call shell-cached,$(CONFIG_SHELL) $(srctree)/scripts/gcc-version.sh $(CC))
+cc-version = $(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-version.sh $(CC))
# cc-fullversion
-cc-fullversion = $(call shell-cached,$(CONFIG_SHELL) \
+cc-fullversion = $(shell $(CONFIG_SHELL) \
$(srctree)/scripts/gcc-version.sh -p $(CC))
# cc-ifversion
@@ -232,21 +159,21 @@ cc-if-fullversion = $(shell [ $(cc-fullversion) $(1) $(2) ] && echo $(3) || echo
# cc-ldoption
# Usage: ldflags += $(call cc-ldoption, -Wl$(comma)--hash-style=both)
-cc-ldoption = $(call try-run-cached,\
+cc-ldoption = $(call try-run,\
$(CC) $(1) $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2))
# ld-option
# Usage: LDFLAGS += $(call ld-option, -X)
-ld-option = $(call try-run-cached, $(LD) $(LDFLAGS) $(1) -v,$(1),$(2))
+ld-option = $(call try-run, $(LD) $(LDFLAGS) $(1) -v,$(1),$(2))
# ar-option
# Usage: KBUILD_ARFLAGS := $(call ar-option,D)
# Important: no spaces around options
-ar-option = $(call try-run-cached, $(AR) rc$(1) "$$TMP",$(1),$(2))
+ar-option = $(call try-run, $(AR) rc$(1) "$$TMP",$(1),$(2))
# ld-version
# Note this is mainly for HJ Lu's 3 number binutil versions
-ld-version = $(call shell-cached,$(LD) --version | $(srctree)/scripts/ld-version.sh)
+ld-version = $(shell $(LD) --version | $(srctree)/scripts/ld-version.sh)
# ld-ifversion
# Usage: $(call ld-ifversion, -ge, 22252, y)
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 03/30] kbuild: remove CONFIG_CROSS_COMPILE support
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
2018-04-13 5:06 ` [PATCH 01/30] gcc-plugins: fix build condition of SANCOV plugin Masahiro Yamada
2018-04-13 5:06 ` [PATCH 02/30] kbuild: remove kbuild cache Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 04/30] kconfig: reference environment variables directly and remove 'option env=' Masahiro Yamada
` (28 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Kbuild provides a couple of ways to specify CROSS_COMPILE:
[1] Command line
[2] Environment
[3] arch/*/Makefile (only some architectures)
[4] CONFIG_CROSS_COMPILE
[4] is problematic for the compiler capability tests in Kconfig.
CONFIG_CROSS_COMPILE allows users to change the compiler prefix from
'make menuconfig', etc. It means, the compiler options would have
to be all re-calculated everytime CONFIG_CROSS_COMPILE is changed.
To avoid complexity and performance issues, I'd like to evaluate
the shell commands statically, i.e. only parsing Kconfig files.
I guess the majority is [1] or [2]. Currently, there are only
5 defconfig files that specify CONFIG_CROSS_COMPILE.
arch/arm/configs/lpc18xx_defconfig
arch/hexagon/configs/comet_defconfig
arch/nds32/configs/defconfig
arch/openrisc/configs/or1ksim_defconfig
arch/openrisc/configs/simple_smp_defconfig
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---
Changes in v3: None
Changes in v2: None
Makefile | 3 ---
init/Kconfig | 9 ---------
2 files changed, 12 deletions(-)
diff --git a/Makefile b/Makefile
index d85cee0..ca3e3e8 100644
--- a/Makefile
+++ b/Makefile
@@ -316,12 +316,9 @@ SUBARCH := $(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ \
# CROSS_COMPILE can be set on the command line
# make CROSS_COMPILE=ia64-linux-
# Alternatively CROSS_COMPILE can be set in the environment.
-# A third alternative is to store a setting in .config so that plain
-# "make" in the configured kernel build directory always uses that.
# Default value for CROSS_COMPILE is not to prefix executables
# Note: Some architectures assign CROSS_COMPILE in their arch/*/Makefile
ARCH ?= $(SUBARCH)
-CROSS_COMPILE ?= $(CONFIG_CROSS_COMPILE:"%"=%)
# Architecture as present in compile.h
UTS_MACHINE := $(ARCH)
diff --git a/init/Kconfig b/init/Kconfig
index 9d167a5..c1ca920 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -54,15 +54,6 @@ config INIT_ENV_ARG_LIMIT
Maximum of each of the number of arguments and environment
variables passed to init from the kernel command line.
-
-config CROSS_COMPILE
- string "Cross-compiler tool prefix"
- help
- Same as running 'make CROSS_COMPILE=prefix-' but stored for
- default make runs in this kernel build directory. You don't
- need to set this unless you want the configured kernel build
- directory to select the cross-compiler automatically.
-
config COMPILE_TEST
bool "Compile also drivers which will not load"
depends on !UML
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 04/30] kconfig: reference environment variables directly and remove 'option env='
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (2 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 03/30] kbuild: remove CONFIG_CROSS_COMPILE support Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 05/30] kconfig: remove string expansion in file_lookup() Masahiro Yamada
` (27 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
To get access to environment variables, Kconfig needs to define a
symbol using "option env=" syntax. It is tedious to add a symbol entry
for each environment variable given that we need to define much more
such as 'CC', 'AS', 'srctree' etc. to evaluate the compiler capability
in Kconfig.
Adding '$' for symbol references is grammatically weird. Looking at
the code, the symbols prefixed with 'S' are expanded by:
- conf_expand_value()
This is used to expand 'arch/$ARCH/defconfig' and 'defconfig_list'
- sym_expand_string_value()
This is used to expand strings in 'source' and 'mainmenu'
All of them are fixed values independent of user configuration. So,
they can be changed into the direct expansion instead of symbols.
This change makes the code much cleaner. The bounce symbols 'SRCARCH',
'ARCH', 'SUBARCH', 'KERNELVERSION' are gone.
sym_init() hard-coding 'UNAME_RELEASE' is also gone. 'UNAME_RELEASE'
should be replaced with an environment variable.
ARCH_DEFCONFIG is a normal symbol, so it should be simply referenced
without '$' prefix.
The new syntax is addicted by Make. The variable reference needs
parentheses, like $(FOO), but you can omit them for single-letter
variables, like $F. Yet, in Makefiles, people tend to use the
parenthetical form for consistency / clarification.
At this moment, only the environment variable is supported, but I will
extend the concept of 'variable' later on.
The variables are expanded in the lexer so we can simplify the token
handling on the parser side.
For example, the following code works.
[Example code]
config MY_TOOLCHAIN_LIST
string
default "My tools: CC=$(CC), AS=$(AS), CPP=$(CPP)"
[Result]
$ make -s alldefconfig && tail -n 1 .config
CONFIG_MY_TOOLCHAIN_LIST="My tools: CC=gcc, AS=as, CPP=gcc -E"
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---
Changes in v3:
- Reimplement
- Variable reference need parentheses except single-letter variable
Changes in v2:
- Move the string expansion to the lexer phase.
- Split environment helpers to env.c
Documentation/kbuild/kconfig-language.txt | 8 -
Kconfig | 8 +-
Makefile | 3 +-
arch/sh/Kconfig | 4 +-
arch/sparc/Kconfig | 4 +-
arch/um/Kconfig.common | 4 -
arch/x86/Kconfig | 4 +-
arch/x86/um/Kconfig | 6 +-
init/Kconfig | 16 +-
scripts/kconfig/confdata.c | 31 +---
scripts/kconfig/kconf_id.c | 1 -
scripts/kconfig/lkc.h | 4 -
scripts/kconfig/lkc_proto.h | 6 +
| 3 -
scripts/kconfig/preprocess.c | 261 ++++++++++++++++++++++++++++++
scripts/kconfig/symbol.c | 56 -------
scripts/kconfig/util.c | 18 +--
scripts/kconfig/zconf.l | 67 +++++++-
scripts/kconfig/zconf.y | 2 +-
19 files changed, 353 insertions(+), 153 deletions(-)
create mode 100644 scripts/kconfig/preprocess.c
diff --git a/Documentation/kbuild/kconfig-language.txt b/Documentation/kbuild/kconfig-language.txt
index f5b9493..0e966e8 100644
--- a/Documentation/kbuild/kconfig-language.txt
+++ b/Documentation/kbuild/kconfig-language.txt
@@ -198,14 +198,6 @@ applicable everywhere (see syntax).
enables the third modular state for all config symbols.
At most one symbol may have the "modules" option set.
- - "env"=<value>
- This imports the environment variable into Kconfig. It behaves like
- a default, except that the value comes from the environment, this
- also means that the behaviour when mixing it with normal defaults is
- undefined at this point. The symbol is currently not exported back
- to the build environment (if this is desired, it can be done via
- another symbol).
-
- "allnoconfig_y"
This declares the symbol as one that should have the value y when
using "allnoconfig". Used for symbols that hide other symbols.
diff --git a/Kconfig b/Kconfig
index 8c4c1cb..4af1b42 100644
--- a/Kconfig
+++ b/Kconfig
@@ -3,10 +3,6 @@
# For a description of the syntax of this configuration file,
# see Documentation/kbuild/kconfig-language.txt.
#
-mainmenu "Linux/$ARCH $KERNELVERSION Kernel Configuration"
+mainmenu "Linux/$(ARCH) $(KERNELVERSION) Kernel Configuration"
-config SRCARCH
- string
- option env="SRCARCH"
-
-source "arch/$SRCARCH/Kconfig"
+source "arch/$(SRCARCH)/Kconfig"
diff --git a/Makefile b/Makefile
index ca3e3e8..5298ad7 100644
--- a/Makefile
+++ b/Makefile
@@ -284,7 +284,8 @@ include scripts/Kbuild.include
# Read KERNELRELEASE from include/config/kernel.release (if it exists)
KERNELRELEASE = $(shell cat include/config/kernel.release 2> /dev/null)
KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(SUBLEVEL)))$(EXTRAVERSION)
-export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION
+UNAME_RELEASE := $(shell uname --release)
+export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION UNAME_RELEASE
# SUBARCH tells the usermode build what the underlying arch is. That is set
# first, and if a usermode build is happening, the "ARCH=um" on the command
diff --git a/arch/sh/Kconfig b/arch/sh/Kconfig
index 97fe293..19297b9 100644
--- a/arch/sh/Kconfig
+++ b/arch/sh/Kconfig
@@ -57,7 +57,7 @@ config SUPERH
<http://www.linux-sh.org/>.
config SUPERH32
- def_bool ARCH = "sh"
+ def_bool "$(ARCH)" = "sh"
select HAVE_KPROBES
select HAVE_KRETPROBES
select HAVE_IOREMAP_PROT if MMU && !X2TLB
@@ -76,7 +76,7 @@ config SUPERH32
select HAVE_CC_STACKPROTECTOR
config SUPERH64
- def_bool ARCH = "sh64"
+ def_bool "$(ARCH)" = "sh64"
select HAVE_EXIT_THREAD
select KALLSYMS
diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index 8767e45..df7410c 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -1,6 +1,6 @@
config 64BIT
- bool "64-bit kernel" if ARCH = "sparc"
- default ARCH = "sparc64"
+ bool "64-bit kernel" if "$(ARCH)" = "sparc"
+ default "$(ARCH)" = "sparc64"
help
SPARC is a family of RISC microprocessors designed and marketed by
Sun Microsystems, incorporated. They are very widely found in Sun
diff --git a/arch/um/Kconfig.common b/arch/um/Kconfig.common
index c68add8..07f84c8 100644
--- a/arch/um/Kconfig.common
+++ b/arch/um/Kconfig.common
@@ -54,10 +54,6 @@ config HZ
int
default 100
-config SUBARCH
- string
- option env="SUBARCH"
-
config NR_CPUS
int
range 1 1
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index d234cca..72350d5 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1,8 +1,8 @@
# SPDX-License-Identifier: GPL-2.0
# Select 32 or 64 bit
config 64BIT
- bool "64-bit kernel" if ARCH = "x86"
- default ARCH != "i386"
+ bool "64-bit kernel" if "$(ARCH)" = "x86"
+ default "$(ARCH)" != "i386"
---help---
Say yes to build a 64-bit kernel - formerly known as x86_64
Say no to build a 32-bit kernel - formerly known as i386
diff --git a/arch/x86/um/Kconfig b/arch/x86/um/Kconfig
index 13ed827..6a15c4d 100644
--- a/arch/x86/um/Kconfig
+++ b/arch/x86/um/Kconfig
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
-mainmenu "User Mode Linux/$SUBARCH $KERNELVERSION Kernel Configuration"
+mainmenu "User Mode Linux/$(SUBARCH) $(KERNELVERSION) Kernel Configuration"
source "arch/um/Kconfig.common"
@@ -16,8 +16,8 @@ config UML_X86
select GENERIC_FIND_FIRST_BIT
config 64BIT
- bool "64-bit kernel" if SUBARCH = "x86"
- default SUBARCH != "i386"
+ bool "64-bit kernel" if "$(SUBARCH)" = "x86"
+ default "$(SUBARCH)" != "i386"
config X86_32
def_bool !64BIT
diff --git a/init/Kconfig b/init/Kconfig
index c1ca920..e6dafed5 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1,20 +1,12 @@
-config ARCH
- string
- option env="ARCH"
-
-config KERNELVERSION
- string
- option env="KERNELVERSION"
-
config DEFCONFIG_LIST
string
depends on !UML
option defconfig_list
- default "/lib/modules/$UNAME_RELEASE/.config"
+ default "/lib/modules/$(UNAME_RELEASE)/.config"
default "/etc/kernel-config"
- default "/boot/config-$UNAME_RELEASE"
- default "$ARCH_DEFCONFIG"
- default "arch/$ARCH/defconfig"
+ default "/boot/config-$(UNAME_RELEASE)"
+ default ARCH_DEFCONFIG
+ default "arch/$(ARCH)/defconfig"
config CONSTRUCTORS
bool
diff --git a/scripts/kconfig/confdata.c b/scripts/kconfig/confdata.c
index df26c7b..0698cc3 100644
--- a/scripts/kconfig/confdata.c
+++ b/scripts/kconfig/confdata.c
@@ -81,39 +81,13 @@ const char *conf_get_autoconfig_name(void)
return name ? name : "include/config/auto.conf";
}
-static char *conf_expand_value(const char *in)
-{
- struct symbol *sym;
- const char *src;
- static char res_value[SYMBOL_MAXLENGTH];
- char *dst, name[SYMBOL_MAXLENGTH];
-
- res_value[0] = 0;
- dst = name;
- while ((src = strchr(in, '$'))) {
- strncat(res_value, in, src - in);
- src++;
- dst = name;
- while (isalnum(*src) || *src == '_')
- *dst++ = *src++;
- *dst = 0;
- sym = sym_lookup(name, 0);
- sym_calc_value(sym);
- strcat(res_value, sym_get_string_value(sym));
- in = src;
- }
- strcat(res_value, in);
-
- return res_value;
-}
-
char *conf_get_default_confname(void)
{
struct stat buf;
static char fullname[PATH_MAX+1];
char *env, *name;
- name = conf_expand_value(conf_defname);
+ name = expand_string(conf_defname);
env = getenv(SRCTREE);
if (env) {
sprintf(fullname, "%s/%s", env, name);
@@ -274,7 +248,8 @@ int conf_read_simple(const char *name, int def)
if (expr_calc_value(prop->visible.expr) == no ||
prop->expr->type != E_SYMBOL)
continue;
- name = conf_expand_value(prop->expr->left.sym->name);
+ sym_calc_value(prop->expr->left.sym);
+ name = sym_get_string_value(prop->expr->left.sym);
in = zconf_fopen(name);
if (in) {
conf_message(_("using defaults found in %s"),
diff --git a/scripts/kconfig/kconf_id.c b/scripts/kconfig/kconf_id.c
index 3ea9c5f..b3e0ea0 100644
--- a/scripts/kconfig/kconf_id.c
+++ b/scripts/kconfig/kconf_id.c
@@ -32,7 +32,6 @@ static struct kconf_id kconf_id_array[] = {
{ "on", T_ON, TF_PARAM },
{ "modules", T_OPT_MODULES, TF_OPTION },
{ "defconfig_list", T_OPT_DEFCONFIG_LIST, TF_OPTION },
- { "env", T_OPT_ENV, TF_OPTION },
{ "allnoconfig_y", T_OPT_ALLNOCONFIG_Y, TF_OPTION },
};
diff --git a/scripts/kconfig/lkc.h b/scripts/kconfig/lkc.h
index f4394af..553098a 100644
--- a/scripts/kconfig/lkc.h
+++ b/scripts/kconfig/lkc.h
@@ -58,7 +58,6 @@ enum conf_def_mode {
#define T_OPT_MODULES 1
#define T_OPT_DEFCONFIG_LIST 2
-#define T_OPT_ENV 3
#define T_OPT_ALLNOCONFIG_Y 4
struct kconf_id {
@@ -134,9 +133,6 @@ void str_printf(struct gstr *gs, const char *fmt, ...);
const char *str_get(struct gstr *gs);
/* symbol.c */
-extern struct expr *sym_env_list;
-
-void sym_init(void);
void sym_clear_all_valid(void);
struct symbol *sym_choice_default(struct symbol *sym);
const char *sym_get_string_default(struct symbol *sym);
diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h
index 9dc8abf..9f465fe 100644
--- a/scripts/kconfig/lkc_proto.h
+++ b/scripts/kconfig/lkc_proto.h
@@ -49,5 +49,11 @@ const char * sym_get_string_value(struct symbol *sym);
const char * prop_get_type_name(enum prop_type type);
+/* preprocess.c */
+void env_write_dep(FILE *f, const char *auto_conf_name);
+char *expand_string(const char *in);
+char *expand_dollar(const char **str);
+char *expand_one_token(const char **str);
+
/* expr.c */
void expr_print(struct expr *e, void (*fn)(void *, struct symbol *, const char *), void *data, int prevtoken);
--git a/scripts/kconfig/menu.c b/scripts/kconfig/menu.c
index 5c5c137..8148305 100644
--- a/scripts/kconfig/menu.c
+++ b/scripts/kconfig/menu.c
@@ -214,9 +214,6 @@ void menu_add_option(int token, char *arg)
zconf_error("trying to redefine defconfig symbol");
sym_defconfig_list->flags |= SYMBOL_AUTO;
break;
- case T_OPT_ENV:
- prop_add_env(arg);
- break;
case T_OPT_ALLNOCONFIG_Y:
current_entry->sym->flags |= SYMBOL_ALLNOCONFIG_Y;
break;
diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
new file mode 100644
index 0000000..fa4abc8
--- /dev/null
+++ b/scripts/kconfig/preprocess.c
@@ -0,0 +1,261 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (C) 2018 Masahiro Yamada <yamada.masahiro@socionext.com>
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "list.h"
+
+/*
+ * Environment variables
+ */
+static LIST_HEAD(env_list);
+
+struct env {
+ char *name;
+ char *value;
+ struct list_head node;
+};
+
+static void env_add(const char *name, const char *value)
+{
+ struct env *e;
+
+ e = xmalloc(sizeof(*e));
+ e->name = xstrdup(name);
+ e->value = xstrdup(value);
+
+ list_add_tail(&e->node, &env_list);
+}
+
+static void env_del(struct env *e)
+{
+ list_del(&e->node);
+ free(e->name);
+ free(e->value);
+ free(e);
+}
+
+/* The returned pointer must be freed when done */
+static char *env_expand(const char *name)
+{
+ struct env *e;
+ const char *value;
+
+ list_for_each_entry(e, &env_list, node) {
+ if (!strcmp(name, e->name))
+ return xstrdup(e->value);
+ }
+
+ value = getenv(name);
+ if (!value)
+ value = "";
+
+ /*
+ * We need to remember all referenced environments.
+ * They will be written out to include/config/auto.conf.cmd
+ */
+ env_add(name, value);
+
+ return xstrdup(value);
+}
+
+void env_write_dep(FILE *f, const char *autoconfig_name)
+{
+ struct env *e, *tmp;
+
+ list_for_each_entry_safe(e, tmp, &env_list, node) {
+ fprintf(f, "ifneq \"$(%s)\" \"%s\"\n", e->name, e->value);
+ fprintf(f, "%s: FORCE\n", autoconfig_name);
+ fprintf(f, "endif\n");
+ env_del(e);
+ }
+}
+
+static char *eval_clause(const char *in)
+{
+ char *res, *name;
+
+ /*
+ * Returns an empty string because '$()' should be evaluated
+ * to a null string.
+ */
+ if (!*in)
+ return xstrdup("");
+
+ name = expand_string(in);
+
+ res = env_expand(name);
+ free(name);
+
+ return res;
+}
+
+/*
+ * Expand a string that follows '$'
+ *
+ * For example, if the input string is
+ * ($(FOO)$($(BAR)))$(BAZ)
+ * this helper evaluates
+ * $($(FOO)$($(BAR)))
+ * and returns the resulted string, then updates 'str' to point to the next
+ * character after the corresponding closing parenthesis, in this case, *str
+ * will be
+ * $(BAR)
+ *
+ * If the input does not start with '(', the first character is assumed as
+ * a variable name. For example, if the input string is
+ * ABC
+ * this helper evaluates
+ * $A
+ * and *str will be updated to
+ * BC
+ * This is because $ABC is equivalent to $(A)BC. (Like Make, you can omit
+ * parentheses if the variable name consists of a single character.
+ */
+char *expand_dollar(const char **str)
+{
+ const char *p = *str;
+ const char *q;
+ char *tmp, *out;
+ int nest = 0;
+
+ /*
+ * A standalone '$' at the end of a token is treated as-is.
+ */
+ if (!*p)
+ return xstrdup("$");
+
+ /* '$$' represents an escaped '$' */
+ if (*p == '$') {
+ *str = p + 1;
+ return xstrdup("$");
+ }
+
+ /* Single-letter variable like '$A' */
+ if (*p != '(') {
+ tmp = xmalloc(2);
+ tmp[0] = *p;
+ tmp[1] = 0;
+ *str = p + 1;
+ out = eval_clause(tmp);
+ free(tmp);
+ return out;
+ }
+
+ /*
+ * Variables that consist of multiple letters
+ * must be surrounded with parentheses.
+ * For example, $(FOO)
+ */
+ p++;
+ q = p;
+ while (*q) {
+ switch (*q) {
+ case '(':
+ nest++;
+ break;
+ case ')':
+ if (nest-- == 0) {
+ tmp = xmalloc(q - p + 1);
+ memcpy(tmp, p, q - p);
+ tmp[q - p] = 0;
+ *str = q + 1;
+ out = eval_clause(tmp);
+ free(tmp);
+ return out;
+ }
+ break;
+ default:
+ break;
+ }
+ q++;
+ }
+
+ fprintf(stderr, "%s:%d:missing ')'\n", zconf_curname(), zconf_lineno());
+ exit(1);
+}
+
+/*
+ * Expand variables in the given string. Undefined variables
+ * expand to an empty string.
+ * The returned string must be freed when done.
+ */
+char *expand_string(const char *in)
+{
+ const char *prev_in, *p;
+ char *new, *out;
+ size_t outlen;
+
+ out = xmalloc(1);
+ *out = 0;
+
+ while ((p = strchr(in, '$'))) {
+ prev_in = in;
+ in = p + 1;
+ new = expand_dollar(&in);
+ outlen = strlen(out) + (p - prev_in) + strlen(new) + 1;
+ out = xrealloc(out, outlen);
+ strncat(out, prev_in, p - prev_in);
+ strcat(out, new);
+ free(new);
+ }
+
+ outlen = strlen(out) + strlen(in) + 1;
+ out = xrealloc(out, outlen);
+ strcat(out, in);
+
+ return out;
+}
+
+/*
+ * Expand variables in a token. The parsing stops when a token separater
+ * (in most cases, it is a whitespace) is encountered. 'str' is updated to
+ * point to the next character.
+ *
+ * The returned string must be freed when done.
+ */
+char *expand_one_token(const char **str)
+{
+ const char *in, *prev_in, *p;
+ char *new, *out;
+ size_t outlen;
+
+ out = xmalloc(1);
+ *out = 0;
+
+ p = in = *str;
+
+ while (1) {
+ if (*p == '$') {
+ prev_in = in;
+ in = p + 1;
+ new = expand_dollar(&in);
+ outlen = strlen(out) + (p - prev_in) + strlen(new) + 1;
+ out = xrealloc(out, outlen);
+ strncat(out, prev_in, p - prev_in);
+ strcat(out, new);
+ free(new);
+ p = in;
+ continue;
+ }
+
+ /* Valid characters in a symbol (why '.' and '/' ?) */
+ if (isalnum(*p) || *p == '_' || *p == '-' || *p == '.' || *p == '/') {
+ p++;
+ continue;
+ }
+
+ break;
+ }
+
+ outlen = strlen(out) + (p - in) + 1;
+ out = xrealloc(out, outlen);
+ strncat(out, in, p - in);
+
+ *str = p;
+
+ return out;
+}
diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c
index f0b2e3b..2460648 100644
--- a/scripts/kconfig/symbol.c
+++ b/scripts/kconfig/symbol.c
@@ -33,33 +33,6 @@ struct symbol *sym_defconfig_list;
struct symbol *modules_sym;
tristate modules_val;
-struct expr *sym_env_list;
-
-static void sym_add_default(struct symbol *sym, const char *def)
-{
- struct property *prop = prop_alloc(P_DEFAULT, sym);
-
- prop->expr = expr_alloc_symbol(sym_lookup(def, SYMBOL_CONST));
-}
-
-void sym_init(void)
-{
- struct symbol *sym;
- struct utsname uts;
- static bool inited = false;
-
- if (inited)
- return;
- inited = true;
-
- uname(&uts);
-
- sym = sym_lookup("UNAME_RELEASE", 0);
- sym->type = S_STRING;
- sym->flags |= SYMBOL_AUTO;
- sym_add_default(sym, uts.release);
-}
-
enum symbol_type sym_get_type(struct symbol *sym)
{
enum symbol_type type = sym->type;
@@ -1401,32 +1374,3 @@ const char *prop_get_type_name(enum prop_type type)
}
return "unknown";
}
-
-static void prop_add_env(const char *env)
-{
- struct symbol *sym, *sym2;
- struct property *prop;
- char *p;
-
- sym = current_entry->sym;
- sym->flags |= SYMBOL_AUTO;
- for_all_properties(sym, prop, P_ENV) {
- sym2 = prop_get_symbol(prop);
- if (strcmp(sym2->name, env))
- menu_warn(current_entry, "redefining environment symbol from %s",
- sym2->name);
- return;
- }
-
- prop = prop_alloc(P_ENV, sym);
- prop->expr = expr_alloc_symbol(sym_lookup(env, SYMBOL_CONST));
-
- sym_env_list = expr_alloc_one(E_LIST, sym_env_list);
- sym_env_list->right.sym = sym;
-
- p = getenv(env);
- if (p)
- sym_add_default(sym, p);
- else
- menu_warn(current_entry, "environment variable %s undefined", env);
-}
diff --git a/scripts/kconfig/util.c b/scripts/kconfig/util.c
index c6f6e21..807147e 100644
--- a/scripts/kconfig/util.c
+++ b/scripts/kconfig/util.c
@@ -34,8 +34,6 @@ struct file *file_lookup(const char *name)
/* write a dependency file as used by kbuild to track dependencies */
int file_write_dep(const char *name)
{
- struct symbol *sym, *env_sym;
- struct expr *e;
struct file *file;
FILE *out;
@@ -54,21 +52,7 @@ int file_write_dep(const char *name)
fprintf(out, "\n%s: \\\n"
"\t$(deps_config)\n\n", conf_get_autoconfig_name());
- expr_list_for_each_sym(sym_env_list, e, sym) {
- struct property *prop;
- const char *value;
-
- prop = sym_get_env_prop(sym);
- env_sym = prop_get_symbol(prop);
- if (!env_sym)
- continue;
- value = getenv(env_sym->name);
- if (!value)
- value = "";
- fprintf(out, "ifneq \"$(%s)\" \"%s\"\n", env_sym->name, value);
- fprintf(out, "%s: FORCE\n", conf_get_autoconfig_name());
- fprintf(out, "endif\n");
- }
+ env_write_dep(out, conf_get_autoconfig_name());
fprintf(out, "\n$(deps_config): ;\n");
fclose(out);
diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
index 045093d..9dc5fe3 100644
--- a/scripts/kconfig/zconf.l
+++ b/scripts/kconfig/zconf.l
@@ -1,6 +1,5 @@
%option nostdinit noyywrap never-interactive full ecs
%option 8bit nodefault yylineno
-%option noinput
%x COMMAND HELP STRING PARAM
%{
/*
@@ -35,6 +34,8 @@ struct buffer *current_buf;
static int last_ts, first_ts;
+static char *expand_token(const char *in, size_t n);
+static void append_expanded_string(const char *in);
static void zconf_endhelp(void);
static void zconf_endfile(void);
@@ -147,6 +148,13 @@ n [A-Za-z0-9_-]
yylval.string = text;
return T_WORD;
}
+ ({n}|[/.$])+ {
+ /* this token includes at least one '$' */
+ yylval.string = expand_token(yytext, yyleng);
+ if (strlen(yylval.string))
+ return T_WORD;
+ free(yylval.string);
+ }
#.* /* comment */
\\\n ;
[[:blank:]]+
@@ -157,12 +165,13 @@ n [A-Za-z0-9_-]
}
<STRING>{
- [^'"\\\n]+/\n {
+ "$".* append_expanded_string(yytext);
+ [^$'"\\\n]+/\n {
append_string(yytext, yyleng);
yylval.string = text;
return T_WORD_QUOTE;
}
- [^'"\\\n]+ {
+ [^$'"\\\n]+ {
append_string(yytext, yyleng);
}
\\.?/\n {
@@ -249,6 +258,58 @@ n [A-Za-z0-9_-]
}
%%
+static char *expand_token(const char *in, size_t n)
+{
+ char *out;
+ int c;
+ char c2;
+ const char *rest, *end;
+
+ new_string();
+ append_string(in, n);
+
+ /* get the whole the line because we do not know the end of token. */
+ while ((c = input()) != EOF) {
+ if (c == '\n') {
+ unput(c);
+ break;
+ }
+ c2 = c;
+ append_string(&c2, 1);
+ }
+
+ rest = text;
+ out = expand_one_token(&rest);
+
+ /* push back unused characters to the input stream */
+ end = rest + strlen(rest);
+ while (end > rest)
+ unput(*--end);
+
+ free(text);
+
+ return out;
+}
+
+static void append_expanded_string(const char *str)
+{
+ const char *end;
+ char *res;
+
+ str++;
+
+ res = expand_dollar(&str);
+
+ /* push back unused characters to the input stream */
+ end = str + strlen(str);
+ while (end > str)
+ unput(*--end);
+
+ append_string(res, strlen(res));
+
+ free(res);
+}
+
void zconf_starthelp(void)
{
new_string();
diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y
index ad6305b..3a4a0fa 100644
--- a/scripts/kconfig/zconf.y
+++ b/scripts/kconfig/zconf.y
@@ -534,7 +534,6 @@ void conf_parse(const char *name)
zconf_initscan(name);
- sym_init();
_menu_init();
if (getenv("ZCONF_DEBUG"))
@@ -780,3 +779,4 @@ void zconfdump(FILE *out)
#include "expr.c"
#include "symbol.c"
#include "menu.c"
+#include "preprocess.c"
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 05/30] kconfig: remove string expansion in file_lookup()
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (3 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 04/30] kconfig: reference environment variables directly and remove 'option env=' Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 06/30] kconfig: remove string expansion for mainmenu after yyparse() Masahiro Yamada
` (26 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
There are two callers of file_lookup(), but there is no more reason
to expand the given path.
[1] zconf_initscan()
This is used to open the first Kconfig. sym_expand_string_value()
has never been used in a useful way here; before opening the first
Kconfig file, obviously there is no symbol to expand. If you use
expand_string_value() instead, environments in KBUILD_KCONFIG would
be expanded, but I do not see practical benefits for that.
[2] zconf_nextfile()
This is used to open the next file from 'source' statement.
Symbols in the path like "arch/$SRCARCH/Kconfig" needed expanding,
but it was replaced with the direct environment expansion. The
environment has already been expanded before the token is passed
to the parser.
By the way, file_lookup() was already buggy; it expanded a given path,
but it used the path before expansion for look-up:
if (!strcmp(name, file->name)) {
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
---
Changes in v3: None
Changes in v2:
- Simplify the patch. Just remove text expansion.
scripts/kconfig/util.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/scripts/kconfig/util.c b/scripts/kconfig/util.c
index 807147e..790967df 100644
--- a/scripts/kconfig/util.c
+++ b/scripts/kconfig/util.c
@@ -14,18 +14,16 @@
struct file *file_lookup(const char *name)
{
struct file *file;
- char *file_name = sym_expand_string_value(name);
for (file = file_list; file; file = file->next) {
if (!strcmp(name, file->name)) {
- free(file_name);
return file;
}
}
file = xmalloc(sizeof(*file));
memset(file, 0, sizeof(*file));
- file->name = file_name;
+ file->name = xstrdup(name);
file->next = file_list;
file_list = file;
return file;
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 06/30] kconfig: remove string expansion for mainmenu after yyparse()
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (4 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 05/30] kconfig: remove string expansion in file_lookup() Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 07/30] kconfig: remove sym_expand_string_value() Masahiro Yamada
` (25 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Now that environments are expanded in the lexer, conf_parse() does
not need to expand them explicitly.
The hack introduced by commit 0724a7c32a54 ("kconfig: Don't leak
main menus during parsing") can go away.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
---
Changes in v3: None
Changes in v2:
- Simplify the patch. Just remove the text expansion.
scripts/kconfig/zconf.y | 24 +++++-------------------
1 file changed, 5 insertions(+), 19 deletions(-)
diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y
index 3a4a0fa..22e318c 100644
--- a/scripts/kconfig/zconf.y
+++ b/scripts/kconfig/zconf.y
@@ -109,7 +109,7 @@ static struct menu *current_menu, *current_entry;
%%
input: nl start | start;
-start: mainmenu_stmt stmt_list | no_mainmenu_stmt stmt_list;
+start: mainmenu_stmt stmt_list | stmt_list;
/* mainmenu entry */
@@ -118,19 +118,6 @@ mainmenu_stmt: T_MAINMENU prompt nl
menu_add_prompt(P_MENU, $2, NULL);
};
-/* Default main menu, if there's no mainmenu entry */
-
-no_mainmenu_stmt: /* empty */
-{
- /*
- * Hack: Keep the main menu title on the heap so we can safely free it
- * later regardless of whether it comes from the 'prompt' in
- * mainmenu_stmt or here
- */
- menu_add_prompt(P_MENU, xstrdup("Linux Kernel Configuration"), NULL);
-};
-
-
stmt_list:
/* empty */
| stmt_list common_stmt
@@ -528,7 +515,6 @@ word_opt: /* empty */ { $$ = NULL; }
void conf_parse(const char *name)
{
- const char *tmp;
struct symbol *sym;
int i;
@@ -544,10 +530,10 @@ void conf_parse(const char *name)
if (!modules_sym)
modules_sym = sym_find( "n" );
- tmp = rootmenu.prompt->text;
- rootmenu.prompt->text = _(rootmenu.prompt->text);
- rootmenu.prompt->text = sym_expand_string_value(rootmenu.prompt->text);
- free((char*)tmp);
+ if (!menu_has_prompt(&rootmenu)) {
+ current_entry = &rootmenu;
+ menu_add_prompt(P_MENU, "Linux Kernel Configuration", NULL);
+ }
menu_finalize(&rootmenu);
for_all_symbols(i, sym) {
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 07/30] kconfig: remove sym_expand_string_value()
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (5 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 06/30] kconfig: remove string expansion for mainmenu after yyparse() Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 08/30] kconfig: add built-in function support Masahiro Yamada
` (24 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
There is no more caller of sym_expand_string_value().
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3:
- newly added
Changes in v2: None
scripts/kconfig/lkc_proto.h | 1 -
scripts/kconfig/symbol.c | 53 ---------------------------------------------
2 files changed, 54 deletions(-)
diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h
index 9f465fe..c46929f 100644
--- a/scripts/kconfig/lkc_proto.h
+++ b/scripts/kconfig/lkc_proto.h
@@ -31,7 +31,6 @@ extern struct symbol * symbol_hash[SYMBOL_HASHSIZE];
struct symbol * sym_lookup(const char *name, int flags);
struct symbol * sym_find(const char *name);
-char *sym_expand_string_value(const char *in);
const char * sym_escape_string_value(const char *in);
struct symbol ** sym_re_search(const char *pattern);
const char * sym_type_name(enum symbol_type type);
diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c
index 2460648..7c9a88e 100644
--- a/scripts/kconfig/symbol.c
+++ b/scripts/kconfig/symbol.c
@@ -879,59 +879,6 @@ struct symbol *sym_find(const char *name)
return symbol;
}
-/*
- * Expand symbol's names embedded in the string given in argument. Symbols'
- * name to be expanded shall be prefixed by a '$'. Unknown symbol expands to
- * the empty string.
- */
-char *sym_expand_string_value(const char *in)
-{
- const char *src;
- char *res;
- size_t reslen;
-
- /*
- * Note: 'in' might come from a token that's about to be
- * freed, so make sure to always allocate a new string
- */
- reslen = strlen(in) + 1;
- res = xmalloc(reslen);
- res[0] = '\0';
-
- while ((src = strchr(in, '$'))) {
- char *p, name[SYMBOL_MAXLENGTH];
- const char *symval = "";
- struct symbol *sym;
- size_t newlen;
-
- strncat(res, in, src - in);
- src++;
-
- p = name;
- while (isalnum(*src) || *src == '_')
- *p++ = *src++;
- *p = '\0';
-
- sym = sym_find(name);
- if (sym != NULL) {
- sym_calc_value(sym);
- symval = sym_get_string_value(sym);
- }
-
- newlen = strlen(res) + strlen(symval) + strlen(src) + 1;
- if (newlen > reslen) {
- reslen = newlen;
- res = xrealloc(res, reslen);
- }
-
- strcat(res, symval);
- in = src;
- }
- strcat(res, in);
-
- return res;
-}
-
const char *sym_escape_string_value(const char *in)
{
const char *p;
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 08/30] kconfig: add built-in function support
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (6 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 07/30] kconfig: remove sym_expand_string_value() Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-15 7:57 ` Ulf Magnusson
2018-04-13 5:06 ` [PATCH 09/30] kconfig: add 'shell' built-in function Masahiro Yamada
` (23 subsequent siblings)
31 siblings, 1 reply; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
This commit adds a new concept 'function' to do more text processing
in Kconfig.
A function call looks like this:
$(function arg1, arg2, arg3, ...)
This commit adds the basic infrastructure to expand functions.
Change the text expansion helpers to take arguments.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3:
- Split base infrastructure and 'shell' function
into separate patches.
Changes in v2:
- Use 'shell' for getting stdout from the comment.
It was 'shell-stdout' in the previous version.
- Symplify the implementation since the expansion has been moved to
lexer.
scripts/kconfig/preprocess.c | 142 +++++++++++++++++++++++++++++++++++++++----
1 file changed, 130 insertions(+), 12 deletions(-)
diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index fa4abc8..e77cf7c 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -8,6 +8,10 @@
#include "list.h"
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+
+static char *expand_string_with_args(const char *in, int argc, char *argv[]);
+
/*
* Environment variables
*/
@@ -74,9 +78,47 @@ void env_write_dep(FILE *f, const char *autoconfig_name)
}
}
-static char *eval_clause(const char *in)
+/*
+ * Built-in Functions
+ */
+struct function {
+ char *name;
+ char *(*func)(int argc, char *argv[]);
+};
+
+static const struct function function_table[] = {
+};
+
+static char *function_call(const char *name, int argc, char *argv[])
{
- char *res, *name;
+ const struct function *f;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(function_table); i++) {
+ f = &function_table[i];
+ if (!strcmp(f->name, name))
+ return f->func(argc, argv);
+ }
+
+ return NULL;
+}
+
+#define FUNCTION_MAX_ARGS 16
+
+/*
+ * Evaluate a clause with arguments. argc/argv are arguments from the upper
+ * function call.
+ *
+ * Returned string must be freed when done
+ */
+static char *eval_clause(const char *in, int argc, char *argv[])
+{
+ char *tmp, *prev, *p, *res, *name;
+ char delim = ' ';
+ int new_argc = 0;
+ char *new_argv[FUNCTION_MAX_ARGS];
+ int nest = 0;
+ int i;
/*
* Returns an empty string because '$()' should be evaluated
@@ -85,10 +127,73 @@ static char *eval_clause(const char *in)
if (!*in)
return xstrdup("");
- name = expand_string(in);
+ tmp = xstrdup(in);
+
+ prev = p = tmp;
- res = env_expand(name);
+ /*
+ * Split into tokens
+ * The function name and the first argument are separated by a space.
+ * Arguments are separated by a comma.
+ * For example, if the function call is like this:
+ * $(foo abc,$(x),$(y))
+ *
+ * The input string for this helper should be:
+ * foo abc,$(x),$(y)
+ *
+ * and split into:
+ * new_argv[0]: foo
+ * new_argv[1]: abc
+ * new_argv[2]: $(x)
+ * new_argv[3]: $(y)
+ */
+ while (*p) {
+ if (nest == 0 && *p == delim) {
+ *p = 0;
+ new_argv[new_argc++] = prev;
+ prev = p + 1;
+ delim = ',';
+ } else if (*p == '(') {
+ nest++;
+ } else if (*p == ')') {
+ nest--;
+ }
+
+ p++;
+ }
+ new_argv[new_argc++] = prev;
+
+ /*
+ * Shift arguments
+ * new_argv[0] represents a function name or a variable name. Put it
+ * into 'name', then shift the rest of the arguments. This simplifies
+ * 'const' handling.
+ */
+ name = expand_string_with_args(new_argv[0], argc, argv);
+ new_argc--;
+ for (i = 0; i < new_argc; i++)
+ new_argv[i] = expand_string_with_args(new_argv[i + 1],
+ argc, argv);
+
+ free(tmp);
+
+ /* Look for built-in functions */
+ res = function_call(name, new_argc, new_argv);
+ if (res)
+ goto out;
+
+ /* Last, try environment variable */
+ if (new_argc == 0) {
+ res = env_expand(name);
+ if (res)
+ goto out;
+ }
+
+ res = xstrdup("");
+out:
free(name);
+ for (i = 0; i < new_argc; i++)
+ free(new_argv[i]);
return res;
}
@@ -115,7 +220,7 @@ static char *eval_clause(const char *in)
* This is because $ABC is equivalent to $(A)BC. (Like Make, you can omit
* parentheses if the variable name consists of a single character.
*/
-char *expand_dollar(const char **str)
+static char *expand_dollar_with_args(const char **str, int argc, char *argv[])
{
const char *p = *str;
const char *q;
@@ -124,6 +229,9 @@ char *expand_dollar(const char **str)
/*
* A standalone '$' at the end of a token is treated as-is.
+ * For example, the '$' before the comma in $(foo $,A) and the '$'
+ * before the closing parenthesis in $(shell echo $) lose the special
+ * meaning. This is the behavior of Make 4.2 or newer.
*/
if (!*p)
return xstrdup("$");
@@ -140,15 +248,15 @@ char *expand_dollar(const char **str)
tmp[0] = *p;
tmp[1] = 0;
*str = p + 1;
- out = eval_clause(tmp);
+ out = eval_clause(tmp, argc, argv);
free(tmp);
return out;
}
/*
- * Variables that consist of multiple letters
+ * Variables that consist of multiple letters, and function calls
* must be surrounded with parentheses.
- * For example, $(FOO)
+ * For example, $(FOO), $(shell echo helloworld)
*/
p++;
q = p;
@@ -163,7 +271,7 @@ char *expand_dollar(const char **str)
memcpy(tmp, p, q - p);
tmp[q - p] = 0;
*str = q + 1;
- out = eval_clause(tmp);
+ out = eval_clause(tmp, argc, argv);
free(tmp);
return out;
}
@@ -179,11 +287,11 @@ char *expand_dollar(const char **str)
}
/*
- * Expand variables in the given string. Undefined variables
+ * Expand variables, functions, etc. in the given string. Undefined variables
* expand to an empty string.
* The returned string must be freed when done.
*/
-char *expand_string(const char *in)
+static char *expand_string_with_args(const char *in, int argc, char *argv[])
{
const char *prev_in, *p;
char *new, *out;
@@ -195,7 +303,7 @@ char *expand_string(const char *in)
while ((p = strchr(in, '$'))) {
prev_in = in;
in = p + 1;
- new = expand_dollar(&in);
+ new = expand_dollar_with_args(&in, argc, argv);
outlen = strlen(out) + (p - prev_in) + strlen(new) + 1;
out = xrealloc(out, outlen);
strncat(out, prev_in, p - prev_in);
@@ -210,6 +318,16 @@ char *expand_string(const char *in)
return out;
}
+char *expand_string(const char *in)
+{
+ return expand_string_with_args(in, 0, NULL);
+}
+
+char *expand_dollar(const char **str)
+{
+ return expand_dollar_with_args(str, 0, NULL);
+}
+
/*
* Expand variables in a token. The parsing stops when a token separater
* (in most cases, it is a whitespace) is encountered. 'str' is updated to
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [PATCH 08/30] kconfig: add built-in function support
2018-04-13 5:06 ` [PATCH 08/30] kconfig: add built-in function support Masahiro Yamada
@ 2018-04-15 7:57 ` Ulf Magnusson
2018-04-15 15:12 ` Masahiro Yamada
0 siblings, 1 reply; 53+ messages in thread
From: Ulf Magnusson @ 2018-04-15 7:57 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Linus Torvalds, Sam Ravnborg,
Nicholas Piggin, Kees Cook, Emese Revfy, X86 ML,
Linux Kernel Mailing List
On Fri, Apr 13, 2018 at 7:06 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> This commit adds a new concept 'function' to do more text processing
> in Kconfig.
>
> A function call looks like this:
>
> $(function arg1, arg2, arg3, ...)
>
> This commit adds the basic infrastructure to expand functions.
> Change the text expansion helpers to take arguments.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> ---
>
> Changes in v3:
> - Split base infrastructure and 'shell' function
> into separate patches.
>
> Changes in v2:
> - Use 'shell' for getting stdout from the comment.
> It was 'shell-stdout' in the previous version.
> - Symplify the implementation since the expansion has been moved to
> lexer.
>
> scripts/kconfig/preprocess.c | 142 +++++++++++++++++++++++++++++++++++++++----
> 1 file changed, 130 insertions(+), 12 deletions(-)
>
> diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
> index fa4abc8..e77cf7c 100644
> --- a/scripts/kconfig/preprocess.c
> +++ b/scripts/kconfig/preprocess.c
> @@ -8,6 +8,10 @@
>
> #include "list.h"
>
> +#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
> +
> +static char *expand_string_with_args(const char *in, int argc, char *argv[]);
> +
> /*
> * Environment variables
> */
> @@ -74,9 +78,47 @@ void env_write_dep(FILE *f, const char *autoconfig_name)
> }
> }
>
> -static char *eval_clause(const char *in)
> +/*
> + * Built-in Functions
> + */
> +struct function {
> + char *name;
> + char *(*func)(int argc, char *argv[]);
> +};
> +
> +static const struct function function_table[] = {
> +};
> +
> +static char *function_call(const char *name, int argc, char *argv[])
> {
> - char *res, *name;
> + const struct function *f;
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(function_table); i++) {
> + f = &function_table[i];
> + if (!strcmp(f->name, name))
> + return f->func(argc, argv);
> + }
> +
> + return NULL;
> +}
> +
> +#define FUNCTION_MAX_ARGS 16
> +
> +/*
> + * Evaluate a clause with arguments. argc/argv are arguments from the upper
> + * function call.
> + *
> + * Returned string must be freed when done
> + */
> +static char *eval_clause(const char *in, int argc, char *argv[])
> +{
> + char *tmp, *prev, *p, *res, *name;
> + char delim = ' ';
> + int new_argc = 0;
> + char *new_argv[FUNCTION_MAX_ARGS];
> + int nest = 0;
> + int i;
>
> /*
> * Returns an empty string because '$()' should be evaluated
> @@ -85,10 +127,73 @@ static char *eval_clause(const char *in)
> if (!*in)
> return xstrdup("");
>
> - name = expand_string(in);
> + tmp = xstrdup(in);
> +
> + prev = p = tmp;
>
> - res = env_expand(name);
> + /*
> + * Split into tokens
> + * The function name and the first argument are separated by a space.
> + * Arguments are separated by a comma.
> + * For example, if the function call is like this:
> + * $(foo abc,$(x),$(y))
> + *
> + * The input string for this helper should be:
> + * foo abc,$(x),$(y)
> + *
> + * and split into:
> + * new_argv[0]: foo
> + * new_argv[1]: abc
> + * new_argv[2]: $(x)
> + * new_argv[3]: $(y)
> + */
> + while (*p) {
> + if (nest == 0 && *p == delim) {
> + *p = 0;
> + new_argv[new_argc++] = prev;
Erroring out would be nicer than overflowing the array.
Cheers,
Ulf
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH 08/30] kconfig: add built-in function support
2018-04-15 7:57 ` Ulf Magnusson
@ 2018-04-15 15:12 ` Masahiro Yamada
0 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-15 15:12 UTC (permalink / raw)
To: Ulf Magnusson
Cc: Linux Kbuild mailing list, Linus Torvalds, Sam Ravnborg,
Nicholas Piggin, Kees Cook, Emese Revfy, X86 ML,
Linux Kernel Mailing List
2018-04-15 16:57 GMT+09:00 Ulf Magnusson <ulfalizer@gmail.com>:
> On Fri, Apr 13, 2018 at 7:06 AM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> This commit adds a new concept 'function' to do more text processing
>> in Kconfig.
>>
>> A function call looks like this:
>>
>> $(function arg1, arg2, arg3, ...)
>>
>> This commit adds the basic infrastructure to expand functions.
>> Change the text expansion helpers to take arguments.
>>
>> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
>> ---
>>
>> Changes in v3:
>> - Split base infrastructure and 'shell' function
>> into separate patches.
>>
>> Changes in v2:
>> - Use 'shell' for getting stdout from the comment.
>> It was 'shell-stdout' in the previous version.
>> - Symplify the implementation since the expansion has been moved to
>> lexer.
>>
>> scripts/kconfig/preprocess.c | 142 +++++++++++++++++++++++++++++++++++++++----
>> 1 file changed, 130 insertions(+), 12 deletions(-)
>>
>> diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
>> index fa4abc8..e77cf7c 100644
>> --- a/scripts/kconfig/preprocess.c
>> +++ b/scripts/kconfig/preprocess.c
>> @@ -8,6 +8,10 @@
>>
>> #include "list.h"
>>
>> +#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
>> +
>> +static char *expand_string_with_args(const char *in, int argc, char *argv[]);
>> +
>> /*
>> * Environment variables
>> */
>> @@ -74,9 +78,47 @@ void env_write_dep(FILE *f, const char *autoconfig_name)
>> }
>> }
>>
>> -static char *eval_clause(const char *in)
>> +/*
>> + * Built-in Functions
>> + */
>> +struct function {
>> + char *name;
>> + char *(*func)(int argc, char *argv[]);
>> +};
>> +
>> +static const struct function function_table[] = {
>> +};
>> +
>> +static char *function_call(const char *name, int argc, char *argv[])
>> {
>> - char *res, *name;
>> + const struct function *f;
>> + int i;
>> +
>> + for (i = 0; i < ARRAY_SIZE(function_table); i++) {
>> + f = &function_table[i];
>> + if (!strcmp(f->name, name))
>> + return f->func(argc, argv);
>> + }
>> +
>> + return NULL;
>> +}
>> +
>> +#define FUNCTION_MAX_ARGS 16
>> +
>> +/*
>> + * Evaluate a clause with arguments. argc/argv are arguments from the upper
>> + * function call.
>> + *
>> + * Returned string must be freed when done
>> + */
>> +static char *eval_clause(const char *in, int argc, char *argv[])
>> +{
>> + char *tmp, *prev, *p, *res, *name;
>> + char delim = ' ';
>> + int new_argc = 0;
>> + char *new_argv[FUNCTION_MAX_ARGS];
>> + int nest = 0;
>> + int i;
>>
>> /*
>> * Returns an empty string because '$()' should be evaluated
>> @@ -85,10 +127,73 @@ static char *eval_clause(const char *in)
>> if (!*in)
>> return xstrdup("");
>>
>> - name = expand_string(in);
>> + tmp = xstrdup(in);
>> +
>> + prev = p = tmp;
>>
>> - res = env_expand(name);
>> + /*
>> + * Split into tokens
>> + * The function name and the first argument are separated by a space.
>> + * Arguments are separated by a comma.
>> + * For example, if the function call is like this:
>> + * $(foo abc,$(x),$(y))
>> + *
>> + * The input string for this helper should be:
>> + * foo abc,$(x),$(y)
>> + *
>> + * and split into:
>> + * new_argv[0]: foo
>> + * new_argv[1]: abc
>> + * new_argv[2]: $(x)
>> + * new_argv[3]: $(y)
>> + */
>> + while (*p) {
>> + if (nest == 0 && *p == delim) {
>> + *p = 0;
>> + new_argv[new_argc++] = prev;
>
> Erroring out would be nicer than overflowing the array.
OK, will do.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 53+ messages in thread
* [PATCH 09/30] kconfig: add 'shell' built-in function
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (7 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 08/30] kconfig: add built-in function support Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 10/30] kconfig: replace $(UNAME_RELEASE) with function call Masahiro Yamada
` (22 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
This accepts a single command to execute. It returns the standard
output from it.
[Example code]
config HELLO
string
default "$(shell echo hello world)"
config Y
def_bool $(shell echo y)
[Result]
$ make -s alldefconfig && tail -n 2 .config
CONFIG_HELLO="hello world"
CONFIG_Y=y
Caveat:
Like environments, functions are expanded in the lexer. You cannot
pass symbols to function arguments. This is a limitation to simplify
the implementation. I want to avoid the dynamic function evaluation,
which would introduce much more complexity.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
scripts/kconfig/preprocess.c | 66 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 66 insertions(+)
diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index e77cf7c..f4c606f 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -86,7 +86,73 @@ struct function {
char *(*func)(int argc, char *argv[]);
};
+/*
+ * Some commands treats commas verbatim. Concatenate arguments to get
+ * back the original input. The returned string must be freed when done.
+ */
+static char *join_args(int argc, char *argv[])
+{
+ size_t len = 0;
+ char *out;
+ int i;
+
+ for (i = 0; i < argc; i++)
+ len += strlen(argv[i]) + 1;
+
+ out = xmalloc(len);
+ out[0] = 0;
+ for (i = 0; i < argc; i++) {
+ strcat(out, argv[i]);
+ if (i != argc - 1)
+ strcat(out, ",");
+ }
+
+ return out;
+}
+
+static char *do_shell(int argc, char *argv[])
+{
+ FILE *p;
+ char buf[256];
+ char *cmd;
+ size_t nread;
+ int i;
+
+ cmd = join_args(argc, argv);
+
+ p = popen(cmd, "r");
+ if (!p) {
+ perror(cmd);
+ goto free;
+ }
+
+ nread = fread(buf, 1, sizeof(buf), p);
+ if (nread == sizeof(buf))
+ nread--;
+
+ /* remove trailing new lines */
+ while (buf[nread - 1] == '\n')
+ nread--;
+
+ buf[nread] = 0;
+
+ /* replace a new line with a space */
+ for (i = 0; i < nread; i++) {
+ if (buf[i] == '\n')
+ buf[i] = ' ';
+ }
+
+ if (pclose(p) == -1)
+ perror(cmd);
+
+free:
+ free(cmd);
+
+ return xstrdup(buf);
+}
+
static const struct function function_table[] = {
+ { .name = "shell", .func = do_shell },
};
static char *function_call(const char *name, int argc, char *argv[])
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 10/30] kconfig: replace $(UNAME_RELEASE) with function call
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (8 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 09/30] kconfig: add 'shell' built-in function Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 11/30] kconfig: begin PARAM state only when seeing a command keyword Masahiro Yamada
` (21 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Now that 'shell' function is supported, this can be self-contained in
Kconfig.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
---
Changes in v3: None
Changes in v2: None
Makefile | 3 +--
init/Kconfig | 4 ++--
2 files changed, 3 insertions(+), 4 deletions(-)
diff --git a/Makefile b/Makefile
index 5298ad7..ca3e3e8 100644
--- a/Makefile
+++ b/Makefile
@@ -284,8 +284,7 @@ include scripts/Kbuild.include
# Read KERNELRELEASE from include/config/kernel.release (if it exists)
KERNELRELEASE = $(shell cat include/config/kernel.release 2> /dev/null)
KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(SUBLEVEL)))$(EXTRAVERSION)
-UNAME_RELEASE := $(shell uname --release)
-export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION UNAME_RELEASE
+export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION
# SUBARCH tells the usermode build what the underlying arch is. That is set
# first, and if a usermode build is happening, the "ARCH=um" on the command
diff --git a/init/Kconfig b/init/Kconfig
index e6dafed5..4b0b636 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2,9 +2,9 @@ config DEFCONFIG_LIST
string
depends on !UML
option defconfig_list
- default "/lib/modules/$(UNAME_RELEASE)/.config"
+ default "/lib/modules/$(shell uname --release)/.config"
default "/etc/kernel-config"
- default "/boot/config-$(UNAME_RELEASE)"
+ default "/boot/config-$(shell uname --release)"
default ARCH_DEFCONFIG
default "arch/$(ARCH)/defconfig"
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 11/30] kconfig: begin PARAM state only when seeing a command keyword
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (9 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 10/30] kconfig: replace $(UNAME_RELEASE) with function call Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 12/30] kconfig: support variable and user-defined function Masahiro Yamada
` (20 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Currently, any statement line starts with a keyword with TF_COMMAND
flag. So, the following three lines are dead code.
alloc_string(yytext, yyleng);
zconflval.string = text;
return T_WORD;
If a T_WORD token is returned in this context, it will cause syntax
error in the parser anyway.
The next commit will support the assignment statement where a line
starts with an arbitrary identifier. So, I want the lexer to switch
to the PARAM state only when it sees a command keyword.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
scripts/kconfig/zconf.l | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
index 9dc5fe3..5e53348 100644
--- a/scripts/kconfig/zconf.l
+++ b/scripts/kconfig/zconf.l
@@ -102,10 +102,10 @@ n [A-Za-z0-9_-]
<COMMAND>{
{n}+ {
const struct kconf_id *id = kconf_id_lookup(yytext, yyleng);
- BEGIN(PARAM);
current_pos.file = current_file;
current_pos.lineno = yylineno;
if (id && id->flags & TF_COMMAND) {
+ BEGIN(PARAM);
yylval.id = id;
return id->token;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 12/30] kconfig: support variable and user-defined function
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (10 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 11/30] kconfig: begin PARAM state only when seeing a command keyword Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 13/30] kconfig: support simply expanded variable Masahiro Yamada
` (19 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Now, we got a basic ability to test compiler capability in Kconfig.
config CC_HAS_STACKPROTECTOR
def_bool $(shell (($(CC) -Werror -fstack-protector -c -x c /dev/null -o /dev/null 2>/dev/null) && echo y) || echo n)
This works, but it is ugly to repeat this long boilerplate.
We want to describe like this:
config CC_HAS_STACKPROTECTOR
bool
default $(cc-option -fstack-protector)
It is straight-forward to add a new function, but I do not like to
hard-code specialized functions like this. Hence, here is another
feature, user-defined function. This works as a textual shorthand
with parameterization.
A user-defined function is defined by using the = operator, and can
be referenced in the same way as built-in functions. A user-defined
function in Make is referenced like $(call func-name, arg1, arg2),
but I omitted the 'call' to make the syntax shorter.
The definition of a user-defined function contains $(1), $(2), etc.
in its body to reference the parameters. It is grammatically valid
to pass more or fewer arguments when calling it. We already exploit
this feature in our makefiles; scripts/Kbuild.include defines cc-option
which takes two arguments at most, but most of the callers pass only
one argument.
By the way, a variable is supported at a subset of this feature since
a variable is "a user-defined function with zero argument". In this
context, I mean "variable" as recursively expanded variable. I will
add a different flavored variable later on.
The code above can be written as follows:
[Example Code]
success = $(shell ($(1)) >/dev/null 2>&1 && echo y || echo n)
cc-option = $(success $(CC) -Werror $(1) -c -x c /dev/null -o /dev/null)
config CC_HAS_STACKPROTECTOR
def_bool $(cc-option -fstack-protector)
[Result]
$ make -s alldefconfig && tail -n 1 .config
CONFIG_CC_HAS_STACKPROTECTOR=y
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3:
- Re-implement the parse logic
- Use = operator to define a user-defined function
Changes in v2:
- Use 'macro' directly instead of inside the string type symbol.
scripts/kconfig/lkc_proto.h | 2 +
scripts/kconfig/preprocess.c | 96 +++++++++++++++++++++++++++++++++++++++++++-
scripts/kconfig/zconf.l | 17 +++++++-
scripts/kconfig/zconf.y | 21 +++++++++-
4 files changed, 132 insertions(+), 4 deletions(-)
diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h
index c46929f..2b16d6e 100644
--- a/scripts/kconfig/lkc_proto.h
+++ b/scripts/kconfig/lkc_proto.h
@@ -50,6 +50,8 @@ const char * prop_get_type_name(enum prop_type type);
/* preprocess.c */
void env_write_dep(FILE *f, const char *auto_conf_name);
+void variable_add(const char *name, const char *value);
+void variable_all_del(void);
char *expand_string(const char *in);
char *expand_dollar(const char **str);
char *expand_one_token(const char **str);
diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index f4c606f..1b746e0 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -79,6 +79,72 @@ void env_write_dep(FILE *f, const char *autoconfig_name)
}
/*
+ * Variables and User-defined Functions
+ */
+static LIST_HEAD(variable_list);
+
+struct variable {
+ char *name;
+ char *value;
+ struct list_head node;
+};
+
+static struct variable *variable_lookup(const char *name)
+{
+ struct variable *v;
+
+ list_for_each_entry(v, &variable_list, node) {
+ if (!strcmp(name, v->name))
+ return v;
+ }
+
+ return NULL;
+}
+
+static char *variable_expand(const char *name, int argc, char *argv[])
+{
+ struct variable *v;
+
+ v = variable_lookup(name);
+ if (!v)
+ return NULL;
+
+ return expand_string_with_args(v->value, argc, argv);
+}
+
+void variable_add(const char *name, const char *value)
+{
+ struct variable *v;
+
+ v = variable_lookup(name);
+ if (v) {
+ free(v->value);
+ } else {
+ v = xmalloc(sizeof(*v));
+ v->name = xstrdup(name);
+ list_add_tail(&v->node, &variable_list);
+ }
+
+ v->value = xstrdup(value);
+}
+
+static void variable_del(struct variable *v)
+{
+ list_del(&v->node);
+ free(v->name);
+ free(v->value);
+ free(v);
+}
+
+void variable_all_del(void)
+{
+ struct variable *v, *tmp;
+
+ list_for_each_entry_safe(v, tmp, &variable_list, node)
+ variable_del(v);
+}
+
+/*
* Built-in Functions
*/
struct function {
@@ -175,16 +241,30 @@ static char *function_call(const char *name, int argc, char *argv[])
* Evaluate a clause with arguments. argc/argv are arguments from the upper
* function call.
*
+ * Let's say 'foo' is defined as:
+ * foo = ABC$(1)PQR(2)XYZ
+ * and you want to evaluate $(foo x,y)
+ *
+ * First, this helper is called with:
+ * in : foo x,y
+ * argc: 0
+ * and then, recursively called with:
+ * in: ABC$(1)PQR(2)XYZ
+ * argc: 2
+ * argv[0]: x
+ * argv[1]: y
+ *
* Returned string must be freed when done
*/
static char *eval_clause(const char *in, int argc, char *argv[])
{
- char *tmp, *prev, *p, *res, *name;
+ char *tmp, *prev, *p, *res, *endptr, *name;
char delim = ' ';
int new_argc = 0;
char *new_argv[FUNCTION_MAX_ARGS];
int nest = 0;
int i;
+ unsigned long n;
/*
* Returns an empty string because '$()' should be evaluated
@@ -193,6 +273,15 @@ static char *eval_clause(const char *in, int argc, char *argv[])
if (!*in)
return xstrdup("");
+ /*
+ * If variable name is '1', '2', etc. It is generally an argument
+ * from a user-function call (i.e. local-scope variable). If not
+ * available, then look-up global-scope variables.
+ */
+ n = strtoul(in, &endptr, 10);
+ if (!*endptr && n > 0 && n <= argc)
+ return xstrdup(argv[n - 1]);
+
tmp = xstrdup(in);
prev = p = tmp;
@@ -248,6 +337,11 @@ static char *eval_clause(const char *in, int argc, char *argv[])
if (res)
goto out;
+ /* Search for variable or user-defined function */
+ res = variable_expand(name, new_argc, new_argv);
+ if (res)
+ goto out;
+
/* Last, try environment variable */
if (new_argc == 0) {
res = env_expand(name);
diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
index 5e53348..19e5ebf 100644
--- a/scripts/kconfig/zconf.l
+++ b/scripts/kconfig/zconf.l
@@ -1,12 +1,13 @@
%option nostdinit noyywrap never-interactive full ecs
%option 8bit nodefault yylineno
-%x COMMAND HELP STRING PARAM
+%x COMMAND HELP STRING PARAM ASSIGN_VAL
%{
/*
* Copyright (C) 2002 Roman Zippel <zippel@linux-m68k.org>
* Released under the terms of the GNU GPL v2.0.
*/
+#include <assert.h>
#include <limits.h>
#include <stdio.h>
#include <stdlib.h>
@@ -111,8 +112,10 @@ n [A-Za-z0-9_-]
}
alloc_string(yytext, yyleng);
yylval.string = text;
- return T_WORD;
+ return T_VARIABLE;
}
+ "=" { BEGIN(ASSIGN_VAL); return T_ASSIGN; }
+ [[:blank:]]+
. warn_ignored_character(*yytext);
\n {
BEGIN(INITIAL);
@@ -120,6 +123,16 @@ n [A-Za-z0-9_-]
}
}
+<ASSIGN_VAL>{
+ [^[:blank:]\n]+.* {
+ alloc_string(yytext, yyleng);
+ yylval.string = text;
+ return T_ASSIGN_VAL;
+ }
+ \n { BEGIN(INITIAL); return T_EOL; }
+ .
+}
+
<PARAM>{
"&&" return T_AND;
"||" return T_OR;
diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y
index 22e318c..493388c 100644
--- a/scripts/kconfig/zconf.y
+++ b/scripts/kconfig/zconf.y
@@ -77,6 +77,9 @@ static struct menu *current_menu, *current_entry;
%token T_CLOSE_PAREN
%token T_OPEN_PAREN
%token T_EOL
+%token <string> T_VARIABLE
+%token T_ASSIGN
+%token <string> T_ASSIGN_VAL
%left T_OR
%left T_AND
@@ -92,7 +95,7 @@ static struct menu *current_menu, *current_entry;
%type <id> end
%type <id> option_name
%type <menu> if_entry menu_entry choice_entry
-%type <string> symbol_option_arg word_opt
+%type <string> symbol_option_arg word_opt assign_val
%destructor {
fprintf(stderr, "%s:%d: missing end statement for this entry\n",
@@ -143,6 +146,7 @@ common_stmt:
| config_stmt
| menuconfig_stmt
| source_stmt
+ | assignment_stmt
;
option_error:
@@ -511,6 +515,15 @@ symbol: nonconst_symbol
word_opt: /* empty */ { $$ = NULL; }
| T_WORD
+/* assignment statement */
+
+assignment_stmt: T_VARIABLE T_ASSIGN assign_val T_EOL { variable_add($1, $3); free($1); free($3); }
+
+assign_val:
+ /* empty */ { $$ = xstrdup(""); };
+ | T_ASSIGN_VAL
+;
+
%%
void conf_parse(const char *name)
@@ -525,6 +538,12 @@ void conf_parse(const char *name)
if (getenv("ZCONF_DEBUG"))
yydebug = 1;
yyparse();
+
+ /*
+ * Variables are expanded in the parse phase. We can free them here.
+ */
+ variable_all_del();
+
if (yynerrs)
exit(1);
if (!modules_sym)
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 13/30] kconfig: support simply expanded variable
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (11 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 12/30] kconfig: support variable and user-defined function Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 14/30] kconfig: support append assignment operator Masahiro Yamada
` (18 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
The previous commit added variable and user-defined function. They
work similarly in the sense that the evaluation is deferred until
they are used.
This commit adds another type of variable, simply expanded variable,
as we see in Make.
The := operator defines a simply expanded variable, expanding the
righthand side immediately. This works like traditional programming
language variables.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3:
- newly added
Changes in v2: None
scripts/kconfig/lkc_proto.h | 7 ++++++-
scripts/kconfig/preprocess.c | 6 ++++--
scripts/kconfig/zconf.l | 3 ++-
scripts/kconfig/zconf.y | 5 +++--
4 files changed, 15 insertions(+), 6 deletions(-)
diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h
index 2b16d6e..6303193 100644
--- a/scripts/kconfig/lkc_proto.h
+++ b/scripts/kconfig/lkc_proto.h
@@ -49,8 +49,13 @@ const char * sym_get_string_value(struct symbol *sym);
const char * prop_get_type_name(enum prop_type type);
/* preprocess.c */
+enum variable_flavor {
+ VAR_SIMPLE,
+ VAR_RECURSIVE,
+};
void env_write_dep(FILE *f, const char *auto_conf_name);
-void variable_add(const char *name, const char *value);
+void variable_add(const char *name, const char *value,
+ enum variable_flavor flavor);
void variable_all_del(void);
char *expand_string(const char *in);
char *expand_dollar(const char **str);
diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index 1b746e0..a835c94 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -112,7 +112,8 @@ static char *variable_expand(const char *name, int argc, char *argv[])
return expand_string_with_args(v->value, argc, argv);
}
-void variable_add(const char *name, const char *value)
+void variable_add(const char *name, const char *value,
+ enum variable_flavor flavor)
{
struct variable *v;
@@ -125,7 +126,8 @@ void variable_add(const char *name, const char *value)
list_add_tail(&v->node, &variable_list);
}
- v->value = xstrdup(value);
+ v->value = (flavor == VAR_SIMPLE) ? expand_string(value) :
+ xstrdup(value);
}
static void variable_del(struct variable *v)
diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
index 19e5ebf..aa76942 100644
--- a/scripts/kconfig/zconf.l
+++ b/scripts/kconfig/zconf.l
@@ -114,7 +114,8 @@ n [A-Za-z0-9_-]
yylval.string = text;
return T_VARIABLE;
}
- "=" { BEGIN(ASSIGN_VAL); return T_ASSIGN; }
+ "=" { BEGIN(ASSIGN_VAL); yylval.flavor = VAR_RECURSIVE; return T_ASSIGN; }
+ ":=" { BEGIN(ASSIGN_VAL); yylval.flavor = VAR_SIMPLE; return T_ASSIGN; }
[[:blank:]]+
. warn_ignored_character(*yytext);
\n {
diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y
index 493388c..1ebbf9e 100644
--- a/scripts/kconfig/zconf.y
+++ b/scripts/kconfig/zconf.y
@@ -41,6 +41,7 @@ static struct menu *current_menu, *current_entry;
struct expr *expr;
struct menu *menu;
const struct kconf_id *id;
+ enum variable_flavor flavor;
}
%token <id>T_MAINMENU
@@ -78,7 +79,7 @@ static struct menu *current_menu, *current_entry;
%token T_OPEN_PAREN
%token T_EOL
%token <string> T_VARIABLE
-%token T_ASSIGN
+%token <flavor> T_ASSIGN
%token <string> T_ASSIGN_VAL
%left T_OR
@@ -517,7 +518,7 @@ word_opt: /* empty */ { $$ = NULL; }
/* assignment statement */
-assignment_stmt: T_VARIABLE T_ASSIGN assign_val T_EOL { variable_add($1, $3); free($1); free($3); }
+assignment_stmt: T_VARIABLE T_ASSIGN assign_val T_EOL { variable_add($1, $3, $2); free($1); free($3); }
assign_val:
/* empty */ { $$ = xstrdup(""); };
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 14/30] kconfig: support append assignment operator
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (12 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 13/30] kconfig: support simply expanded variable Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 15/30] kconfig: expand lefthand side of assignment statement Masahiro Yamada
` (17 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Support += operator. This appends a space and the text on the
righthand side to a variable.
The timing of the evaluation of the righthand side depends on the
flavor of the variable. If the lefthand side was originally defined
as a simple variable, the righthand side is expanded immediately.
Otherwise, the expansion is deferred. Appending something to an
undefined variable results in a recursive variable.
To implement this, we need to remember the flavor of variables.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3:
- newly added
Changes in v2: None
scripts/kconfig/lkc_proto.h | 1 +
scripts/kconfig/preprocess.c | 29 +++++++++++++++++++++++++++--
scripts/kconfig/zconf.l | 1 +
3 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h
index 6303193..a8b7a33 100644
--- a/scripts/kconfig/lkc_proto.h
+++ b/scripts/kconfig/lkc_proto.h
@@ -52,6 +52,7 @@ const char * prop_get_type_name(enum prop_type type);
enum variable_flavor {
VAR_SIMPLE,
VAR_RECURSIVE,
+ VAR_APPEND,
};
void env_write_dep(FILE *f, const char *auto_conf_name);
void variable_add(const char *name, const char *value,
diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index a835c94..85972fb 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -86,6 +86,7 @@ static LIST_HEAD(variable_list);
struct variable {
char *name;
char *value;
+ enum variable_flavor flavor;
struct list_head node;
};
@@ -116,18 +117,42 @@ void variable_add(const char *name, const char *value,
enum variable_flavor flavor)
{
struct variable *v;
+ char *new_value;
+ bool append = false;
v = variable_lookup(name);
if (v) {
- free(v->value);
+ /* For defined variables, += inherits the existing flavor */
+ if (flavor == VAR_APPEND) {
+ flavor = v->flavor;
+ append = true;
+ } else {
+ free(v->value);
+ }
} else {
+ /* For undefined variables, += assumes the recursive flavor */
+ if (flavor == VAR_APPEND)
+ flavor = VAR_RECURSIVE;
+
v = xmalloc(sizeof(*v));
v->name = xstrdup(name);
list_add_tail(&v->node, &variable_list);
}
- v->value = (flavor == VAR_SIMPLE) ? expand_string(value) :
+ v->flavor = flavor;
+
+ new_value = (flavor == VAR_SIMPLE) ? expand_string(value) :
xstrdup(value);
+
+ if (append) {
+ v->value = xrealloc(v->value,
+ strlen(v->value) + strlen(new_value) + 2);
+ strcat(v->value, " ");
+ strcat(v->value, new_value);
+ free(new_value);
+ } else {
+ v->value = new_value;
+ }
}
static void variable_del(struct variable *v)
diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
index aa76942..c68ca56b 100644
--- a/scripts/kconfig/zconf.l
+++ b/scripts/kconfig/zconf.l
@@ -116,6 +116,7 @@ n [A-Za-z0-9_-]
}
"=" { BEGIN(ASSIGN_VAL); yylval.flavor = VAR_RECURSIVE; return T_ASSIGN; }
":=" { BEGIN(ASSIGN_VAL); yylval.flavor = VAR_SIMPLE; return T_ASSIGN; }
+ "+=" { BEGIN(ASSIGN_VAL); yylval.flavor = VAR_APPEND; return T_ASSIGN; }
[[:blank:]]+
. warn_ignored_character(*yytext);
\n {
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 15/30] kconfig: expand lefthand side of assignment statement
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (13 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 14/30] kconfig: support append assignment operator Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 16/30] kconfig: add 'info' and 'warning' built-in functions Masahiro Yamada
` (16 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Make allows variable references in the lefthand side of assignment.
X = A
Y = B
$(X)$(Y) = 1
This does 'AB = 1' Do likewise in Kconfig as well.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
scripts/kconfig/zconf.l | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
index c68ca56b..ae33e9b 100644
--- a/scripts/kconfig/zconf.l
+++ b/scripts/kconfig/zconf.l
@@ -114,6 +114,13 @@ n [A-Za-z0-9_-]
yylval.string = text;
return T_VARIABLE;
}
+ ({n}|$)+ {
+ /* this token includes at least one '$' */
+ yylval.string = expand_token(yytext, yyleng);
+ if (strlen(yylval.string))
+ return T_VARIABLE;
+ free(yylval.string);
+ }
"=" { BEGIN(ASSIGN_VAL); yylval.flavor = VAR_RECURSIVE; return T_ASSIGN; }
":=" { BEGIN(ASSIGN_VAL); yylval.flavor = VAR_SIMPLE; return T_ASSIGN; }
"+=" { BEGIN(ASSIGN_VAL); yylval.flavor = VAR_APPEND; return T_ASSIGN; }
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 16/30] kconfig: add 'info' and 'warning' built-in functions
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (14 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 15/30] kconfig: expand lefthand side of assignment statement Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 17/30] Documentation: kconfig: document a new Kconfig macro language Masahiro Yamada
` (15 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Add 'info' and 'warning' functions as in Make.
They print messages during parsing Kconfig files, which is useful
when debugging Kconfig at least.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
scripts/kconfig/preprocess.c | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index 85972fb..f8d06d9 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -203,6 +203,19 @@ static char *join_args(int argc, char *argv[])
return out;
}
+static char *do_info(int argc, char *argv[])
+{
+ char *msg;
+
+ msg = join_args(argc, argv);
+
+ printf("%s\n", msg);
+
+ free(msg);
+
+ return xstrdup("");
+}
+
static char *do_shell(int argc, char *argv[])
{
FILE *p;
@@ -244,8 +257,23 @@ static char *do_shell(int argc, char *argv[])
return xstrdup(buf);
}
+static char *do_warning(int argc, char *argv[])
+{
+ char *msg;
+
+ msg = join_args(argc, argv);
+
+ fprintf(stderr, "%s:%d: %s\n", current_file->name, yylineno, msg);
+
+ free(msg);
+
+ return xstrdup("");
+}
+
static const struct function function_table[] = {
+ { .name = "info", .func = do_info },
{ .name = "shell", .func = do_shell },
+ { .name = "warning", .func = do_warning },
};
static char *function_call(const char *name, int argc, char *argv[])
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 17/30] Documentation: kconfig: document a new Kconfig macro language
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (15 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 16/30] kconfig: add 'info' and 'warning' built-in functions Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-14 23:33 ` Randy Dunlap
2018-04-15 8:08 ` Ulf Magnusson
2018-04-13 5:06 ` [PATCH 18/30] kconfig: test: test text expansion Masahiro Yamada
` (14 subsequent siblings)
31 siblings, 2 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Add a document for the macro language introduced to Kconfig.
The motivation of this work is to move the compiler option tests to
Kconfig from Makefile. A number of kernel features require the
compiler support. Enabling such features blindly in Kconfig ends up
with a lot of nasty build-time testing in Makefiles. If a chosen
feature turns out unsupported by the compiler, what the build system
can do is either to disable it (silently!) or to forcibly break the
build, despite Kconfig has let the user to enable it.
This change was strongly prompted by Linus Torvalds. You can find
his suggestions [1] [2] in ML. The original idea was to add a new
'option', but I found generalized text expansion would make Kconfig
more powerful and lovely. While polishing up the implementation, I
noticed sort of similarity between Make and Kconfig. This might be
too immature to be called 'language', but anyway here it is. All
ideas are from Make (you can even say it is addicted), so people
will easily understand how it works.
[1]: https://lkml.org/lkml/2016/12/9/577
[2]: https://lkml.org/lkml/2018/2/7/527
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
Documentation/kbuild/kconfig-macro-language.txt | 179 ++++++++++++++++++++++++
MAINTAINERS | 2 +-
2 files changed, 180 insertions(+), 1 deletion(-)
create mode 100644 Documentation/kbuild/kconfig-macro-language.txt
diff --git a/Documentation/kbuild/kconfig-macro-language.txt b/Documentation/kbuild/kconfig-macro-language.txt
new file mode 100644
index 0000000..1f6281b
--- /dev/null
+++ b/Documentation/kbuild/kconfig-macro-language.txt
@@ -0,0 +1,179 @@
+Concept
+-------
+
+The basic idea was inspired by Make. When we look at Make, we notice sort of
+two languages in one. One language describes dependency graphs consisting of
+targets and prerequisites. The other is a macro language for performing textual
+substitution.
+
+There is clear distinction between the two language stages. For example, you
+can write a makefile like follows:
+
+ APP := foo
+ SRC := foo.c
+ CC := gcc
+
+ $(APP): $(SRC)
+ $(CC) -o $(APP) $(SRC)
+
+The macro language replaces the variable references with their expanded form,
+and handles as if the source file were input like follows:
+
+ foo: foo.c
+ gcc -o foo foo.c
+
+Then, Make analyzes the dependency graph and determines the targets to be
+updated.
+
+The idea is quite similar in Kconfig - it is possible to describe a Kconfig
+file like this:
+
+ CC := gcc
+
+ config CC_HAS_FOO
+ def_bool $(shell $(srctree)/scripts/gcc-check-foo.sh $(CC))
+
+The macro language in Kconfig processes the source file into the following
+intermediate:
+
+ config CC_HAS_FOO
+ def_bool y
+
+Then, Kconfig moves onto the evaluation stage to resolve inter-symbol
+dependency, which is explained in kconfig-language.txt.
+
+
+Variables
+---------
+
+Like in Make, a variable in Kconfig works as a macro variable. A macro
+variable is expanded "in place" to yield a text string that may then expanded
+further. To get the value of a variable, enclose the variable name in $( ).
+As a special case, single-letter variable names can omit the parentheses and is
+simply referenced like $X. Unlike Make, Kconfig does not support curly braces
+as in ${CC}.
+
+There are two types of variables: simply expanded variables and recursively
+expanded variables.
+
+A simply expanded variable is defined using the := assignment operator. Its
+righthand side is expanded immediately upon reading the line from the Kconfig
+file.
+
+A recursively expanded variable is defined using the = assignment operator.
+Its righthand side is simply stored as the value of the variable without
+expanding it in any way. Instead, the expansion is performed when the variable
+is used.
+
+There is another type of assignment operator; += is used to append text to a
+variable. The righthand side of += is expanded immediately if the lefthand
+side was originally defined as a simple variable. Otherwise, its evaluation is
+deferred.
+
+
+Functions
+---------
+
+Like Make, Kconfig supports both built-in and user-defined functions. A
+function invocation looks much like a variable reference, but includes one or
+more parameters separated by commas:
+
+ $(function-name arg1, arg2, arg3)
+
+Some functions are implemented as a built-in function. Currently, Kconfig
+supports the following:
+
+ - $(shell command)
+
+ The 'shell' function accepts a single argument that is expanded and passed
+ to a subshell for execution. The standard output of the command is then read
+ and returned as the value of the function. Every newline in the output is
+ replaced with a space. Any trailing newlines are deleted. The standard error
+ is not returned, nor is any program exit status.
+
+ - $(warning text)
+
+ The 'warning' function prints its arguments to stderr. The output is prefixed
+ with the name of the current Kconfig file, the current line number. It
+ evaluates to an empty string.
+
+ - $(info text)
+
+ The 'info' function is similar to 'warning' except that it sends its argument
+ to stdout without any Kconfig name or line number.
+
+A user-defined function is defined by using the = operator. The parameters are
+referenced within the body definition with $1, $2, etc. (or $(1), $(2), etc.)
+In fact, a user-defined function is internally treated as a recursive variable.
+
+A user-defined function is referenced in the same way as a built-in function:
+
+ $(my_func, arg0, arg1, arg2)
+
+Note 1:
+There is a slight difference in the whitespace handling of the function call
+between Make and Kconfig. In Make, leading whitespaces are trimmed from the
+first argument. So, $(info FOO) is equivalent to $(info FOO). Kconfig keeps
+any leading whitespaces except the one right after the function name, which
+works as a separator. So, $(info FOO) prints " FOO" to the stdout.
+
+Note 2:
+In Make, a user-defined function is referenced by using a built-in function,
+'call', like this:
+
+ $(call my_func, arg0, arg1, arg2)
+
+However, Kconfig did not adopt this form just for the purpose of shortening the
+syntax.
+
+
+Caveats
+-------
+
+A variable (or function) can not be expanded across tokens. So, you can not use
+a variable as a shorthand for an expression that consists of multiple tokens.
+The following works:
+
+ RANGE_MIN := 1
+ RANGE_MAX := 3
+
+ config FOO
+ int "foo"
+ range $(RANGE_MIN) $(RANGE_MAX)
+
+But, the following does not work:
+
+ RANGES := 1 3
+
+ config FOO
+ int "foo"
+ range $(RANGES)
+
+A variable can not be expanded to any keyword in Kconfig. The following does
+not work:
+
+ MY_TYPE := tristate
+
+ config FOO
+ $(MY_TYPE) "foo"
+ default y
+
+Obviously from the design, $(shell command) is expanded in the textual
+substitution phase. You can not pass symbols to the 'shell' function.
+The following does not work as expected.
+
+ config ENDIAN_OPTION
+ string
+ default "-mbig-endian" if CPU_BIG_ENDIAN
+ default "-mlittle-endian" if CPU_LITTLE_ENDIAN
+
+ config CC_HAS_ENDIAN_OPTION
+ def_bool $(shell $(srctree)/scripts/gcc-check-option ENDIAN_OPTION)
+
+Instead, you can do like follows so that any function call is statically
+expanded.
+
+ config CC_HAS_ENDIAN_OPTION
+ bool
+ default $(shell $(srctree)/scripts/gcc-check-option -mbig-endian) if CPU_BIG_ENDIAN
+ default $(shell $(srctree)/scripts/gcc-check-option -mlittle-endian) if CPU_LITTLE_ENDIAN
diff --git a/MAINTAINERS b/MAINTAINERS
index b60179d..b9dab38 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7622,7 +7622,7 @@ M: Masahiro Yamada <yamada.masahiro@socionext.com>
T: git git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git kconfig
L: linux-kbuild@vger.kernel.org
S: Maintained
-F: Documentation/kbuild/kconfig-language.txt
+F: Documentation/kbuild/kconfig*
F: scripts/kconfig/
KDUMP
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [PATCH 17/30] Documentation: kconfig: document a new Kconfig macro language
2018-04-13 5:06 ` [PATCH 17/30] Documentation: kconfig: document a new Kconfig macro language Masahiro Yamada
@ 2018-04-14 23:33 ` Randy Dunlap
2018-04-17 15:05 ` Masahiro Yamada
2018-04-15 8:08 ` Ulf Magnusson
1 sibling, 1 reply; 53+ messages in thread
From: Randy Dunlap @ 2018-04-14 23:33 UTC (permalink / raw)
To: Masahiro Yamada, linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, linux-kernel
On 04/12/18 22:06, Masahiro Yamada wrote:
> Add a document for the macro language introduced to Kconfig.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> ---
>
> Changes in v3: None
> Changes in v2: None
>
> Documentation/kbuild/kconfig-macro-language.txt | 179 ++++++++++++++++++++++++
> MAINTAINERS | 2 +-
> 2 files changed, 180 insertions(+), 1 deletion(-)
> create mode 100644 Documentation/kbuild/kconfig-macro-language.txt
>
> diff --git a/Documentation/kbuild/kconfig-macro-language.txt b/Documentation/kbuild/kconfig-macro-language.txt
> new file mode 100644
> index 0000000..1f6281b
> --- /dev/null
> +++ b/Documentation/kbuild/kconfig-macro-language.txt
> @@ -0,0 +1,179 @@
> +Concept
> +-------
> +
> +The basic idea was inspired by Make. When we look at Make, we notice sort of
> +two languages in one. One language describes dependency graphs consisting of
> +targets and prerequisites. The other is a macro language for performing textual
> +substitution.
> +
> +There is clear distinction between the two language stages. For example, you
> +can write a makefile like follows:
> +
> + APP := foo
> + SRC := foo.c
> + CC := gcc
> +
> + $(APP): $(SRC)
> + $(CC) -o $(APP) $(SRC)
> +
> +The macro language replaces the variable references with their expanded form,
> +and handles as if the source file were input like follows:
> +
> + foo: foo.c
> + gcc -o foo foo.c
> +
> +Then, Make analyzes the dependency graph and determines the targets to be
> +updated.
> +
> +The idea is quite similar in Kconfig - it is possible to describe a Kconfig
> +file like this:
> +
> + CC := gcc
> +
> + config CC_HAS_FOO
> + def_bool $(shell $(srctree)/scripts/gcc-check-foo.sh $(CC))
> +
> +The macro language in Kconfig processes the source file into the following
> +intermediate:
> +
> + config CC_HAS_FOO
> + def_bool y
> +
> +Then, Kconfig moves onto the evaluation stage to resolve inter-symbol
> +dependency, which is explained in kconfig-language.txt.
> +
> +
> +Variables
> +---------
> +
> +Like in Make, a variable in Kconfig works as a macro variable. A macro
> +variable is expanded "in place" to yield a text string that may then expanded
may then be expanded
> +further. To get the value of a variable, enclose the variable name in $( ).
> +As a special case, single-letter variable names can omit the parentheses and is
and are
> +simply referenced like $X. Unlike Make, Kconfig does not support curly braces
> +as in ${CC}.
> +
> +There are two types of variables: simply expanded variables and recursively
> +expanded variables.
> +
> +A simply expanded variable is defined using the := assignment operator. Its
> +righthand side is expanded immediately upon reading the line from the Kconfig
> +file.
> +
> +A recursively expanded variable is defined using the = assignment operator.
> +Its righthand side is simply stored as the value of the variable without
> +expanding it in any way. Instead, the expansion is performed when the variable
> +is used.
> +
> +There is another type of assignment operator; += is used to append text to a
> +variable. The righthand side of += is expanded immediately if the lefthand
> +side was originally defined as a simple variable. Otherwise, its evaluation is
> +deferred.
> +
> +
> +Functions
> +---------
> +
> +Like Make, Kconfig supports both built-in and user-defined functions. A
> +function invocation looks much like a variable reference, but includes one or
> +more parameters separated by commas:
> +
> + $(function-name arg1, arg2, arg3)
> +
> +Some functions are implemented as a built-in function. Currently, Kconfig
> +supports the following:
> +
> + - $(shell command)
> +
> + The 'shell' function accepts a single argument that is expanded and passed
> + to a subshell for execution. The standard output of the command is then read
> + and returned as the value of the function. Every newline in the output is
> + replaced with a space. Any trailing newlines are deleted. The standard error
> + is not returned, nor is any program exit status.
> +
> + - $(warning text)
> +
> + The 'warning' function prints its arguments to stderr. The output is prefixed
> + with the name of the current Kconfig file, the current line number. It
file and the current line number. It
> + evaluates to an empty string.
> +
> + - $(info text)
> +
> + The 'info' function is similar to 'warning' except that it sends its argument
> + to stdout without any Kconfig name or line number.
Are current Kconfig file name and line number available so that someone can
construct their own $(info message) messages?
> +
> +A user-defined function is defined by using the = operator. The parameters are
> +referenced within the body definition with $1, $2, etc. (or $(1), $(2), etc.)
> +In fact, a user-defined function is internally treated as a recursive variable.
so the difference is just whether there are arguments?
A recursive variable does not have arguments and a function always has at least
one argument?
> +
> +A user-defined function is referenced in the same way as a built-in function:
> +
> + $(my_func, arg0, arg1, arg2)
Syntax given above is:
+ $(function-name arg1, arg2, arg3)
with no comma after the function name. Which is it?
> +
> +Note 1:
> +There is a slight difference in the whitespace handling of the function call
> +between Make and Kconfig. In Make, leading whitespaces are trimmed from the
> +first argument. So, $(info FOO) is equivalent to $(info FOO). Kconfig keeps
> +any leading whitespaces except the one right after the function name, which
> +works as a separator. So, $(info FOO) prints " FOO" to the stdout.
> +
> +Note 2:
> +In Make, a user-defined function is referenced by using a built-in function,
> +'call', like this:
> +
> + $(call my_func, arg0, arg1, arg2)
> +
> +However, Kconfig did not adopt this form just for the purpose of shortening the
> +syntax.
> +
> +
> +Caveats
> +-------
> +
> +A variable (or function) can not be expanded across tokens. So, you can not use
cannot cannot
> +a variable as a shorthand for an expression that consists of multiple tokens.
> +The following works:
> +
> + RANGE_MIN := 1
> + RANGE_MAX := 3
> +
> + config FOO
> + int "foo"
> + range $(RANGE_MIN) $(RANGE_MAX)
> +
> +But, the following does not work:
> +
> + RANGES := 1 3
> +
> + config FOO
> + int "foo"
> + range $(RANGES)
> +
> +A variable can not be expanded to any keyword in Kconfig. The following does
cannot
> +not work:
> +
> + MY_TYPE := tristate
> +
> + config FOO
> + $(MY_TYPE) "foo"
> + default y
> +
> +Obviously from the design, $(shell command) is expanded in the textual
> +substitution phase. You can not pass symbols to the 'shell' function.
cannot
> +The following does not work as expected.
> +
> + config ENDIAN_OPTION
> + string
> + default "-mbig-endian" if CPU_BIG_ENDIAN
> + default "-mlittle-endian" if CPU_LITTLE_ENDIAN
> +
> + config CC_HAS_ENDIAN_OPTION
> + def_bool $(shell $(srctree)/scripts/gcc-check-option ENDIAN_OPTION)
> +
> +Instead, you can do like follows so that any function call is statically
> +expanded.
> +
> + config CC_HAS_ENDIAN_OPTION
> + bool
fix indentation?
> + default $(shell $(srctree)/scripts/gcc-check-option -mbig-endian) if CPU_BIG_ENDIAN
> + default $(shell $(srctree)/scripts/gcc-check-option -mlittle-endian) if CPU_LITTLE_ENDIAN
--
~Randy
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH 17/30] Documentation: kconfig: document a new Kconfig macro language
2018-04-14 23:33 ` Randy Dunlap
@ 2018-04-17 15:05 ` Masahiro Yamada
0 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-17 15:05 UTC (permalink / raw)
To: Randy Dunlap
Cc: Linux Kbuild mailing list, Linus Torvalds, Sam Ravnborg,
Ulf Magnusson, Nicholas Piggin, Kees Cook, Emese Revfy, X86 ML,
Linux Kernel Mailing List
2018-04-15 8:33 GMT+09:00 Randy Dunlap <rdunlap@infradead.org>:
> On 04/12/18 22:06, Masahiro Yamada wrote:
>> Add a document for the macro language introduced to Kconfig.
>>
>> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
>> ---
>>
>> Changes in v3: None
>> Changes in v2: None
>>
>> Documentation/kbuild/kconfig-macro-language.txt | 179 ++++++++++++++++++++++++
>> MAINTAINERS | 2 +-
>> 2 files changed, 180 insertions(+), 1 deletion(-)
>> create mode 100644 Documentation/kbuild/kconfig-macro-language.txt
>>
>> diff --git a/Documentation/kbuild/kconfig-macro-language.txt b/Documentation/kbuild/kconfig-macro-language.txt
>> new file mode 100644
>> index 0000000..1f6281b
>> --- /dev/null
>> +++ b/Documentation/kbuild/kconfig-macro-language.txt
>> @@ -0,0 +1,179 @@
>> +Concept
>> +-------
>> +
>> +The basic idea was inspired by Make. When we look at Make, we notice sort of
>> +two languages in one. One language describes dependency graphs consisting of
>> +targets and prerequisites. The other is a macro language for performing textual
>> +substitution.
>> +
>> +There is clear distinction between the two language stages. For example, you
>> +can write a makefile like follows:
>> +
>> + APP := foo
>> + SRC := foo.c
>> + CC := gcc
>> +
>> + $(APP): $(SRC)
>> + $(CC) -o $(APP) $(SRC)
>> +
>> +The macro language replaces the variable references with their expanded form,
>> +and handles as if the source file were input like follows:
>> +
>> + foo: foo.c
>> + gcc -o foo foo.c
>> +
>> +Then, Make analyzes the dependency graph and determines the targets to be
>> +updated.
>> +
>> +The idea is quite similar in Kconfig - it is possible to describe a Kconfig
>> +file like this:
>> +
>> + CC := gcc
>> +
>> + config CC_HAS_FOO
>> + def_bool $(shell $(srctree)/scripts/gcc-check-foo.sh $(CC))
>> +
>> +The macro language in Kconfig processes the source file into the following
>> +intermediate:
>> +
>> + config CC_HAS_FOO
>> + def_bool y
>> +
>> +Then, Kconfig moves onto the evaluation stage to resolve inter-symbol
>> +dependency, which is explained in kconfig-language.txt.
>> +
>> +
>> +Variables
>> +---------
>> +
>> +Like in Make, a variable in Kconfig works as a macro variable. A macro
>> +variable is expanded "in place" to yield a text string that may then expanded
>
> may then be expanded
>
>> +further. To get the value of a variable, enclose the variable name in $( ).
>> +As a special case, single-letter variable names can omit the parentheses and is
>
> and are
>
>> +simply referenced like $X. Unlike Make, Kconfig does not support curly braces
>> +as in ${CC}.
>> +
>> +There are two types of variables: simply expanded variables and recursively
>> +expanded variables.
>> +
>> +A simply expanded variable is defined using the := assignment operator. Its
>> +righthand side is expanded immediately upon reading the line from the Kconfig
>> +file.
>> +
>> +A recursively expanded variable is defined using the = assignment operator.
>> +Its righthand side is simply stored as the value of the variable without
>> +expanding it in any way. Instead, the expansion is performed when the variable
>> +is used.
>> +
>> +There is another type of assignment operator; += is used to append text to a
>> +variable. The righthand side of += is expanded immediately if the lefthand
>> +side was originally defined as a simple variable. Otherwise, its evaluation is
>> +deferred.
>> +
>> +
>> +Functions
>> +---------
>> +
>> +Like Make, Kconfig supports both built-in and user-defined functions. A
>> +function invocation looks much like a variable reference, but includes one or
>> +more parameters separated by commas:
>> +
>> + $(function-name arg1, arg2, arg3)
>> +
>> +Some functions are implemented as a built-in function. Currently, Kconfig
>> +supports the following:
>> +
>> + - $(shell command)
>> +
>> + The 'shell' function accepts a single argument that is expanded and passed
>> + to a subshell for execution. The standard output of the command is then read
>> + and returned as the value of the function. Every newline in the output is
>> + replaced with a space. Any trailing newlines are deleted. The standard error
>> + is not returned, nor is any program exit status.
>> +
>> + - $(warning text)
>> +
>> + The 'warning' function prints its arguments to stderr. The output is prefixed
>> + with the name of the current Kconfig file, the current line number. It
>
> file and the current line number. It
>
>> + evaluates to an empty string.
>> +
>> + - $(info text)
>> +
>> + The 'info' function is similar to 'warning' except that it sends its argument
>> + to stdout without any Kconfig name or line number.
>
> Are current Kconfig file name and line number available so that someone can
> construct their own $(info message) messages?
Not available for now, but it would be easy to support such special variables
that expand to the current file name, and line number.
For example, '$(file)' and '$(lineno)' ?
>> +
>> +A user-defined function is defined by using the = operator. The parameters are
>> +referenced within the body definition with $1, $2, etc. (or $(1), $(2), etc.)
>> +In fact, a user-defined function is internally treated as a recursive variable.
>
> so the difference is just whether there are arguments?
> A recursive variable does not have arguments and a function always has at least
> one argument?
At least in my implementation, yes, they are implemented in the same way.
The difference is "what we call it".
I just called it in an intuitive name.
If there is no argument, people generally assume it is a variable.
If it takes arguments, a function.
But, in fact, they are the same.
Probably, similar situation in GNU Make.
For example, you can use $(call ...) to expand a variable
Test Makefile:
A = 1
all:
@echo $(A)
@echo $(call A)
Both print the value of A.
>
>> +
>> +A user-defined function is referenced in the same way as a built-in function:
>> +
>> + $(my_func, arg0, arg1, arg2)
>
> Syntax given above is:
> + $(function-name arg1, arg2, arg3)
>
> with no comma after the function name. Which is it?
Sorry, no comma after the function name.
But, I personally think a comma after the function name is more consistent.
>> +
>> +Note 1:
>> +There is a slight difference in the whitespace handling of the function call
>> +between Make and Kconfig. In Make, leading whitespaces are trimmed from the
>> +first argument. So, $(info FOO) is equivalent to $(info FOO). Kconfig keeps
>> +any leading whitespaces except the one right after the function name, which
>> +works as a separator. So, $(info FOO) prints " FOO" to the stdout.
>> +
>> +Note 2:
>> +In Make, a user-defined function is referenced by using a built-in function,
>> +'call', like this:
>> +
>> + $(call my_func, arg0, arg1, arg2)
>> +
>> +However, Kconfig did not adopt this form just for the purpose of shortening the
>> +syntax.
>> +
>> +
>> +Caveats
>> +-------
>> +
>> +A variable (or function) can not be expanded across tokens. So, you can not use
>
> cannot cannot
>
>> +a variable as a shorthand for an expression that consists of multiple tokens.
>> +The following works:
>> +
>> + RANGE_MIN := 1
>> + RANGE_MAX := 3
>> +
>> + config FOO
>> + int "foo"
>> + range $(RANGE_MIN) $(RANGE_MAX)
>> +
>> +But, the following does not work:
>> +
>> + RANGES := 1 3
>> +
>> + config FOO
>> + int "foo"
>> + range $(RANGES)
>> +
>> +A variable can not be expanded to any keyword in Kconfig. The following does
>
> cannot
>
>> +not work:
>> +
>> + MY_TYPE := tristate
>> +
>> + config FOO
>> + $(MY_TYPE) "foo"
>> + default y
>> +
>> +Obviously from the design, $(shell command) is expanded in the textual
>> +substitution phase. You can not pass symbols to the 'shell' function.
>
> cannot
>
>> +The following does not work as expected.
>> +
>> + config ENDIAN_OPTION
>> + string
>> + default "-mbig-endian" if CPU_BIG_ENDIAN
>> + default "-mlittle-endian" if CPU_LITTLE_ENDIAN
>> +
>> + config CC_HAS_ENDIAN_OPTION
>> + def_bool $(shell $(srctree)/scripts/gcc-check-option ENDIAN_OPTION)
>> +
>> +Instead, you can do like follows so that any function call is statically
>> +expanded.
>> +
>> + config CC_HAS_ENDIAN_OPTION
>> + bool
>
> fix indentation?
Good catch. I will fix them all.
>> + default $(shell $(srctree)/scripts/gcc-check-option -mbig-endian) if CPU_BIG_ENDIAN
>> + default $(shell $(srctree)/scripts/gcc-check-option -mlittle-endian) if CPU_LITTLE_ENDIAN
>
>
> --
> ~Randy
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH 17/30] Documentation: kconfig: document a new Kconfig macro language
2018-04-13 5:06 ` [PATCH 17/30] Documentation: kconfig: document a new Kconfig macro language Masahiro Yamada
2018-04-14 23:33 ` Randy Dunlap
@ 2018-04-15 8:08 ` Ulf Magnusson
2018-04-17 15:07 ` Masahiro Yamada
1 sibling, 1 reply; 53+ messages in thread
From: Ulf Magnusson @ 2018-04-15 8:08 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Linus Torvalds, Sam Ravnborg,
Nicholas Piggin, Kees Cook, Emese Revfy, X86 ML,
Linux Kernel Mailing List
On Fri, Apr 13, 2018 at 7:06 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> Add a document for the macro language introduced to Kconfig.
>
> The motivation of this work is to move the compiler option tests to
> Kconfig from Makefile. A number of kernel features require the
> compiler support. Enabling such features blindly in Kconfig ends up
> with a lot of nasty build-time testing in Makefiles. If a chosen
> feature turns out unsupported by the compiler, what the build system
> can do is either to disable it (silently!) or to forcibly break the
> build, despite Kconfig has let the user to enable it.
>
> This change was strongly prompted by Linus Torvalds. You can find
> his suggestions [1] [2] in ML. The original idea was to add a new
> 'option', but I found generalized text expansion would make Kconfig
> more powerful and lovely. While polishing up the implementation, I
> noticed sort of similarity between Make and Kconfig. This might be
> too immature to be called 'language', but anyway here it is. All
> ideas are from Make (you can even say it is addicted), so people
> will easily understand how it works.
>
> [1]: https://lkml.org/lkml/2016/12/9/577
> [2]: https://lkml.org/lkml/2018/2/7/527
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> ---
>
> Changes in v3: None
> Changes in v2: None
>
> Documentation/kbuild/kconfig-macro-language.txt | 179 ++++++++++++++++++++++++
> MAINTAINERS | 2 +-
> 2 files changed, 180 insertions(+), 1 deletion(-)
> create mode 100644 Documentation/kbuild/kconfig-macro-language.txt
>
> diff --git a/Documentation/kbuild/kconfig-macro-language.txt b/Documentation/kbuild/kconfig-macro-language.txt
> new file mode 100644
> index 0000000..1f6281b
> --- /dev/null
> +++ b/Documentation/kbuild/kconfig-macro-language.txt
> @@ -0,0 +1,179 @@
> +Concept
> +-------
> +
> +The basic idea was inspired by Make. When we look at Make, we notice sort of
> +two languages in one. One language describes dependency graphs consisting of
> +targets and prerequisites. The other is a macro language for performing textual
> +substitution.
> +
> +There is clear distinction between the two language stages. For example, you
> +can write a makefile like follows:
> +
> + APP := foo
> + SRC := foo.c
> + CC := gcc
> +
> + $(APP): $(SRC)
> + $(CC) -o $(APP) $(SRC)
> +
> +The macro language replaces the variable references with their expanded form,
> +and handles as if the source file were input like follows:
> +
> + foo: foo.c
> + gcc -o foo foo.c
> +
> +Then, Make analyzes the dependency graph and determines the targets to be
> +updated.
> +
> +The idea is quite similar in Kconfig - it is possible to describe a Kconfig
> +file like this:
> +
> + CC := gcc
> +
> + config CC_HAS_FOO
> + def_bool $(shell $(srctree)/scripts/gcc-check-foo.sh $(CC))
> +
> +The macro language in Kconfig processes the source file into the following
> +intermediate:
> +
> + config CC_HAS_FOO
> + def_bool y
> +
> +Then, Kconfig moves onto the evaluation stage to resolve inter-symbol
> +dependency, which is explained in kconfig-language.txt.
> +
> +
> +Variables
> +---------
> +
> +Like in Make, a variable in Kconfig works as a macro variable. A macro
> +variable is expanded "in place" to yield a text string that may then expanded
> +further. To get the value of a variable, enclose the variable name in $( ).
> +As a special case, single-letter variable names can omit the parentheses and is
> +simply referenced like $X. Unlike Make, Kconfig does not support curly braces
> +as in ${CC}.
Do we need single-letter variable names for anything? It looks like
we're deviating
a bit from Make behavior already.
I suspect they're just a side effect of Make having automatic variables like $@.
The Make manual discourages them otherwise:
"A dollar sign followed by a character other than a dollar sign,
open-parenthesis or
open-brace treats that single character as the variable name. Thus, you could
reference the variable x with `$x'. However, this practice is strongly
discouraged,
except in the case of the automatic variables (see section Automatic
Variables)."
Cheers,
Ulf
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH 17/30] Documentation: kconfig: document a new Kconfig macro language
2018-04-15 8:08 ` Ulf Magnusson
@ 2018-04-17 15:07 ` Masahiro Yamada
2018-04-18 8:33 ` Ulf Magnusson
0 siblings, 1 reply; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-17 15:07 UTC (permalink / raw)
To: Ulf Magnusson
Cc: Linux Kbuild mailing list, Linus Torvalds, Sam Ravnborg,
Nicholas Piggin, Kees Cook, Emese Revfy, X86 ML,
Linux Kernel Mailing List
2018-04-15 17:08 GMT+09:00 Ulf Magnusson <ulfalizer@gmail.com>:
> On Fri, Apr 13, 2018 at 7:06 AM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> Add a document for the macro language introduced to Kconfig.
>>
>> The motivation of this work is to move the compiler option tests to
>> Kconfig from Makefile. A number of kernel features require the
>> compiler support. Enabling such features blindly in Kconfig ends up
>> with a lot of nasty build-time testing in Makefiles. If a chosen
>> feature turns out unsupported by the compiler, what the build system
>> can do is either to disable it (silently!) or to forcibly break the
>> build, despite Kconfig has let the user to enable it.
>>
>> This change was strongly prompted by Linus Torvalds. You can find
>> his suggestions [1] [2] in ML. The original idea was to add a new
>> 'option', but I found generalized text expansion would make Kconfig
>> more powerful and lovely. While polishing up the implementation, I
>> noticed sort of similarity between Make and Kconfig. This might be
>> too immature to be called 'language', but anyway here it is. All
>> ideas are from Make (you can even say it is addicted), so people
>> will easily understand how it works.
>>
>> [1]: https://lkml.org/lkml/2016/12/9/577
>> [2]: https://lkml.org/lkml/2018/2/7/527
>>
>> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
>> ---
>>
>> Changes in v3: None
>> Changes in v2: None
>>
>> Documentation/kbuild/kconfig-macro-language.txt | 179 ++++++++++++++++++++++++
>> MAINTAINERS | 2 +-
>> 2 files changed, 180 insertions(+), 1 deletion(-)
>> create mode 100644 Documentation/kbuild/kconfig-macro-language.txt
>>
>> diff --git a/Documentation/kbuild/kconfig-macro-language.txt b/Documentation/kbuild/kconfig-macro-language.txt
>> new file mode 100644
>> index 0000000..1f6281b
>> --- /dev/null
>> +++ b/Documentation/kbuild/kconfig-macro-language.txt
>> @@ -0,0 +1,179 @@
>> +Concept
>> +-------
>> +
>> +The basic idea was inspired by Make. When we look at Make, we notice sort of
>> +two languages in one. One language describes dependency graphs consisting of
>> +targets and prerequisites. The other is a macro language for performing textual
>> +substitution.
>> +
>> +There is clear distinction between the two language stages. For example, you
>> +can write a makefile like follows:
>> +
>> + APP := foo
>> + SRC := foo.c
>> + CC := gcc
>> +
>> + $(APP): $(SRC)
>> + $(CC) -o $(APP) $(SRC)
>> +
>> +The macro language replaces the variable references with their expanded form,
>> +and handles as if the source file were input like follows:
>> +
>> + foo: foo.c
>> + gcc -o foo foo.c
>> +
>> +Then, Make analyzes the dependency graph and determines the targets to be
>> +updated.
>> +
>> +The idea is quite similar in Kconfig - it is possible to describe a Kconfig
>> +file like this:
>> +
>> + CC := gcc
>> +
>> + config CC_HAS_FOO
>> + def_bool $(shell $(srctree)/scripts/gcc-check-foo.sh $(CC))
>> +
>> +The macro language in Kconfig processes the source file into the following
>> +intermediate:
>> +
>> + config CC_HAS_FOO
>> + def_bool y
>> +
>> +Then, Kconfig moves onto the evaluation stage to resolve inter-symbol
>> +dependency, which is explained in kconfig-language.txt.
>> +
>> +
>> +Variables
>> +---------
>> +
>> +Like in Make, a variable in Kconfig works as a macro variable. A macro
>> +variable is expanded "in place" to yield a text string that may then expanded
>> +further. To get the value of a variable, enclose the variable name in $( ).
>> +As a special case, single-letter variable names can omit the parentheses and is
>> +simply referenced like $X. Unlike Make, Kconfig does not support curly braces
>> +as in ${CC}.
>
> Do we need single-letter variable names for anything? It looks like
> we're deviating
> a bit from Make behavior already.
>
> I suspect they're just a side effect of Make having automatic variables like $@.
> The Make manual discourages them otherwise:
>
> "A dollar sign followed by a character other than a dollar sign,
> open-parenthesis or
> open-brace treats that single character as the variable name. Thus, you could
> reference the variable x with `$x'. However, this practice is strongly
> discouraged,
> except in the case of the automatic variables (see section Automatic
> Variables)."
>
OK. We do not need two ways to do the same thing.
I will consider it
although supporting single-letter variable is not costly.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH 17/30] Documentation: kconfig: document a new Kconfig macro language
2018-04-17 15:07 ` Masahiro Yamada
@ 2018-04-18 8:33 ` Ulf Magnusson
0 siblings, 0 replies; 53+ messages in thread
From: Ulf Magnusson @ 2018-04-18 8:33 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Linus Torvalds, Sam Ravnborg,
Nicholas Piggin, Kees Cook, Emese Revfy, X86 ML,
Linux Kernel Mailing List
On Tue, Apr 17, 2018 at 5:07 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2018-04-15 17:08 GMT+09:00 Ulf Magnusson <ulfalizer@gmail.com>:
>> On Fri, Apr 13, 2018 at 7:06 AM, Masahiro Yamada
>> <yamada.masahiro@socionext.com> wrote:
>>> Add a document for the macro language introduced to Kconfig.
>>>
>>> The motivation of this work is to move the compiler option tests to
>>> Kconfig from Makefile. A number of kernel features require the
>>> compiler support. Enabling such features blindly in Kconfig ends up
>>> with a lot of nasty build-time testing in Makefiles. If a chosen
>>> feature turns out unsupported by the compiler, what the build system
>>> can do is either to disable it (silently!) or to forcibly break the
>>> build, despite Kconfig has let the user to enable it.
>>>
>>> This change was strongly prompted by Linus Torvalds. You can find
>>> his suggestions [1] [2] in ML. The original idea was to add a new
>>> 'option', but I found generalized text expansion would make Kconfig
>>> more powerful and lovely. While polishing up the implementation, I
>>> noticed sort of similarity between Make and Kconfig. This might be
>>> too immature to be called 'language', but anyway here it is. All
>>> ideas are from Make (you can even say it is addicted), so people
>>> will easily understand how it works.
>>>
>>> [1]: https://lkml.org/lkml/2016/12/9/577
>>> [2]: https://lkml.org/lkml/2018/2/7/527
>>>
>>> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
>>> ---
>>>
>>> Changes in v3: None
>>> Changes in v2: None
>>>
>>> Documentation/kbuild/kconfig-macro-language.txt | 179 ++++++++++++++++++++++++
>>> MAINTAINERS | 2 +-
>>> 2 files changed, 180 insertions(+), 1 deletion(-)
>>> create mode 100644 Documentation/kbuild/kconfig-macro-language.txt
>>>
>>> diff --git a/Documentation/kbuild/kconfig-macro-language.txt b/Documentation/kbuild/kconfig-macro-language.txt
>>> new file mode 100644
>>> index 0000000..1f6281b
>>> --- /dev/null
>>> +++ b/Documentation/kbuild/kconfig-macro-language.txt
>>> @@ -0,0 +1,179 @@
>>> +Concept
>>> +-------
>>> +
>>> +The basic idea was inspired by Make. When we look at Make, we notice sort of
>>> +two languages in one. One language describes dependency graphs consisting of
>>> +targets and prerequisites. The other is a macro language for performing textual
>>> +substitution.
>>> +
>>> +There is clear distinction between the two language stages. For example, you
>>> +can write a makefile like follows:
>>> +
>>> + APP := foo
>>> + SRC := foo.c
>>> + CC := gcc
>>> +
>>> + $(APP): $(SRC)
>>> + $(CC) -o $(APP) $(SRC)
>>> +
>>> +The macro language replaces the variable references with their expanded form,
>>> +and handles as if the source file were input like follows:
>>> +
>>> + foo: foo.c
>>> + gcc -o foo foo.c
>>> +
>>> +Then, Make analyzes the dependency graph and determines the targets to be
>>> +updated.
>>> +
>>> +The idea is quite similar in Kconfig - it is possible to describe a Kconfig
>>> +file like this:
>>> +
>>> + CC := gcc
>>> +
>>> + config CC_HAS_FOO
>>> + def_bool $(shell $(srctree)/scripts/gcc-check-foo.sh $(CC))
>>> +
>>> +The macro language in Kconfig processes the source file into the following
>>> +intermediate:
>>> +
>>> + config CC_HAS_FOO
>>> + def_bool y
>>> +
>>> +Then, Kconfig moves onto the evaluation stage to resolve inter-symbol
>>> +dependency, which is explained in kconfig-language.txt.
>>> +
>>> +
>>> +Variables
>>> +---------
>>> +
>>> +Like in Make, a variable in Kconfig works as a macro variable. A macro
>>> +variable is expanded "in place" to yield a text string that may then expanded
>>> +further. To get the value of a variable, enclose the variable name in $( ).
>>> +As a special case, single-letter variable names can omit the parentheses and is
>>> +simply referenced like $X. Unlike Make, Kconfig does not support curly braces
>>> +as in ${CC}.
>>
>> Do we need single-letter variable names for anything? It looks like
>> we're deviating
>> a bit from Make behavior already.
>>
>> I suspect they're just a side effect of Make having automatic variables like $@.
>> The Make manual discourages them otherwise:
>>
>> "A dollar sign followed by a character other than a dollar sign,
>> open-parenthesis or
>> open-brace treats that single character as the variable name. Thus, you could
>> reference the variable x with `$x'. However, this practice is strongly
>> discouraged,
>> except in the case of the automatic variables (see section Automatic
>> Variables)."
>>
>
> OK. We do not need two ways to do the same thing.
>
> I will consider it
> although supporting single-letter variable is not costly.
>
>
>
> --
> Best Regards
> Masahiro Yamada
Can you think of any cases where dynamic generation of Kconfig symbol
names would be a good solution by the way?
Personally, I'd make the semantics obvious by requiring quotes, so you
could do e.g.
bar := "bar value"
baz := "baz value"
foo := " $(bar) $(baz) "
and get " bar value baz value " in 'foo'.
That'd get rid of those $(empty) hacks and the like, and would make
things super easy to understand.
If expansion was then also limited to within "" (so that you always
produce values), things would get absolutely ridiculously simple, both
behavior- and implementation-wise. All the unput() shenanigans would
disappear (shaving off code), and everyone would be able to tell at a
glance how stuff works.
I know that you're trying to create something that will be immediately
familiar to people used to Make, and that allows things to be easily
moved over, but I wonder if it's worthwhile to carry over the parts that
people find the most confusing and subtle about Make. Simply and
recursively expanded variables and functions are great and make sense,
but the rest I'm not so sure about, especially in Kconfig context.
When I see $(foo) outside of quotes, I assume a textual substitution,
like in Make, because it sticks out like a sore thumb compared to the
rest of the Kconfig syntax (I also think "Perl scalar"). It's actually a
token substitution here. "$(foo)" makes it obvious.
Cheers,
Ulf
^ permalink raw reply [flat|nested] 53+ messages in thread
* [PATCH 18/30] kconfig: test: test text expansion
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (16 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 17/30] Documentation: kconfig: document a new Kconfig macro language Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 19/30] kconfig: show compiler version text in the top comment Masahiro Yamada
` (13 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Here are the test cases I used for developing the text expansion
feature.
I implemented a similar language as you see in Make. The implementation
is different (the source code in GNU Make is much longer, so I did not
want to pull it in), but the behavior is hopefully almost the same.
I intentionally changed some behavior and syntax, but I tried to stick
to the make-like behavior where possible.
It might be interesting to compare the behavior between Make and
Kconfig.
[1] Variable test
You can directly run scripts/kconfig/tests/preprocess/variable/Kconfig
by make.
Make and Kconfig produce the exactly the same output for the variable
test. The output from Make:
$ cd scripts/kconfig/tests/preprocess/variable && make -f Kconfig
Kconfig:5: SIMPLE = 1
Kconfig:11: RECURSIVE = 2
Kconfig:17: SIMPLE = 1 3
Kconfig:23: RECURSIVE = 2 4
Kconfig:30: UNDEFINED_VARIABLE = 4
Kconfig:36: AB = 5
Kconfig:39: X = A
make: *** No targets. Stop.
[2] Built-in function test
The output from Make:
$ cd scripts/kconfig/tests/preprocess/builtin_func && make -f Kconfig
hello,world 0
Kconfig:7: hello,world 1
Kconfig:11: hello, world 2
Kconfig:15: hello,world 3
Kconfig:19: hello, world 4
make: *** No targets. Stop.
The output from "$(warning hello, world 2)" is different.
"Kconfig:11: hello, world 2" vs "Kconfig:11: hello, world 2"
Make strips all leading spaces from the first argument, but does not
touch the other arguments. I thought this was inconsistent. So, I
changed the behavior to not touch any arguments at all.
[3] User-defined function test
I changed the syntax for calling a user-defined function. In
Make, it is invoked by using the 'call' built-in function as in
$(call greeting,Hello,John) but in Kconfig it is invoked without
'call' as in $(greeting Hello,John). Except the syntax difference,
the test case works exactly in the same way for Make and Kconfig.
[4] Escape sequence test
Except the syntax of user-defined function, Make and Kconfig work
in the same way. The behavior of a standalone '$' is different
among Make versions.
By fixing the user-defined function syntax, Make 4.1 or older
works like this:
Kconfig:5: arg0= arg1=
Kconfig:9: arg0=, arg1=,
Kconfig:13: ' " '" ' ''' "'"
Kconfig:16: $
Kconfig:20: $X
Kconfig:26: nasty
Kconfig:32: super_nasty
Kconfig:41:
Kconfig:46: X
Kconfig:49:
make: *** No targets. Stop.
Make 4.2 or newer is like this:
Kconfig:5: arg0= arg1=
Kconfig:9: arg0=, arg1=,
Kconfig:13: ' " '" ' ''' "'"
Kconfig:16: $
Kconfig:20: $X
Kconfig:26: nasty
Kconfig:32: super_nasty
Kconfig:41: $
Kconfig:46: $X
Kconfig:49: nasty
make: *** No targets. Stop.
The last three lines are different. I adopted the behavior of the
newer Make versions. Of course, you should not write such code.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
.../kconfig/tests/preprocess/builtin_func/Kconfig | 19 +++++++++
.../tests/preprocess/builtin_func/__init__.py | 8 ++++
.../tests/preprocess/builtin_func/expected_stderr | 4 ++
.../tests/preprocess/builtin_func/expected_stdout | 1 +
scripts/kconfig/tests/preprocess/escape/Kconfig | 49 ++++++++++++++++++++++
.../kconfig/tests/preprocess/escape/__init__.py | 9 ++++
.../tests/preprocess/escape/expected_stderr | 10 +++++
scripts/kconfig/tests/preprocess/user_func/Kconfig | 19 +++++++++
.../kconfig/tests/preprocess/user_func/__init__.py | 7 ++++
.../tests/preprocess/user_func/expected_stderr | 5 +++
scripts/kconfig/tests/preprocess/variable/Kconfig | 39 +++++++++++++++++
.../kconfig/tests/preprocess/variable/__init__.py | 7 ++++
.../tests/preprocess/variable/expected_stderr | 7 ++++
13 files changed, 184 insertions(+)
create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/Kconfig
create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/__init__.py
create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/expected_stderr
create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/expected_stdout
create mode 100644 scripts/kconfig/tests/preprocess/escape/Kconfig
create mode 100644 scripts/kconfig/tests/preprocess/escape/__init__.py
create mode 100644 scripts/kconfig/tests/preprocess/escape/expected_stderr
create mode 100644 scripts/kconfig/tests/preprocess/user_func/Kconfig
create mode 100644 scripts/kconfig/tests/preprocess/user_func/__init__.py
create mode 100644 scripts/kconfig/tests/preprocess/user_func/expected_stderr
create mode 100644 scripts/kconfig/tests/preprocess/variable/Kconfig
create mode 100644 scripts/kconfig/tests/preprocess/variable/__init__.py
create mode 100644 scripts/kconfig/tests/preprocess/variable/expected_stderr
diff --git a/scripts/kconfig/tests/preprocess/builtin_func/Kconfig b/scripts/kconfig/tests/preprocess/builtin_func/Kconfig
new file mode 100644
index 0000000..5dc454e
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/builtin_func/Kconfig
@@ -0,0 +1,19 @@
+# 'info' prints the argument to stdout.
+# commas are treated verbatim instead of as argument separaters.
+$(info hello,world 0)
+
+# 'warning' is similar, but it sends its argument to stderr,
+# and the message is prefixed with the current file name and line number.
+$(warning hello,world 1)
+
+# leading spaces of the first argument are preserved except
+# the one right after the function name. (This is different from Make)
+$(warning hello, world 2)
+
+# 'shell' executes a command, and returns its stdout.
+# commas are treated verbatim instead of as argument separaters.
+$(warning $(shell echo hello,world 3))
+
+# Every newline in the output is replaced with a space,
+# but any trailing newlines are deleted.
+$(warning $(shell printf 'hello,\nworld\n\n4\n\n\n'))
diff --git a/scripts/kconfig/tests/preprocess/builtin_func/__init__.py b/scripts/kconfig/tests/preprocess/builtin_func/__init__.py
new file mode 100644
index 0000000..ec7c3e2
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/builtin_func/__init__.py
@@ -0,0 +1,8 @@
+"""
+Built-in function tests.
+"""
+
+def test(conf):
+ assert conf.oldaskconfig() == 0
+ assert conf.stdout_contains('expected_stdout')
+ assert conf.stderr_matches('expected_stderr')
diff --git a/scripts/kconfig/tests/preprocess/builtin_func/expected_stderr b/scripts/kconfig/tests/preprocess/builtin_func/expected_stderr
new file mode 100644
index 0000000..b6c046e
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/builtin_func/expected_stderr
@@ -0,0 +1,4 @@
+Kconfig:7: hello,world 1
+Kconfig:11: hello, world 2
+Kconfig:15: hello,world 3
+Kconfig:19: hello, world 4
diff --git a/scripts/kconfig/tests/preprocess/builtin_func/expected_stdout b/scripts/kconfig/tests/preprocess/builtin_func/expected_stdout
new file mode 100644
index 0000000..036c34a
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/builtin_func/expected_stdout
@@ -0,0 +1 @@
+hello,world 0
diff --git a/scripts/kconfig/tests/preprocess/escape/Kconfig b/scripts/kconfig/tests/preprocess/escape/Kconfig
new file mode 100644
index 0000000..b99dfa2
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/escape/Kconfig
@@ -0,0 +1,49 @@
+foo = arg0=$1 arg1=$2
+
+# You can not pass a comma directly as a argument since it is treated as an
+# argument separator. In the following, $1 and $2 will be given a null string.
+$(warning $(foo ,))
+
+# Assign ',' to a variable, then use it if you want to pass in commas
+comma := ,
+$(warning $(foo $(comma),$(comma)))
+
+# Like Make, single quotes, double quotes, spaces are treated verbatim.
+# The following prints the text as-is.
+$(warning ' " '" ' ''' "'")
+
+# You can use '$$' to escape '$' itself
+$(warning $$)
+
+# The escaped '$' loses its special meaning. The following should print '$X'.
+# Do not expand '$X' even further.
+$(warning $$X)
+
+# In Make, a variable name can contain almost any characters. The only
+# disallowed characters are : # and =
+# '$' can be used as a variable name in Kconfig, although it is nasty
+$$ = nasty
+$(warning $($$))
+
+# Even a space can be a variable name by using the following trick.
+empty :=
+space := $(empty) $(empty)
+$(space) = super_nasty
+$(warning $($(space)))
+
+# The behavior of a standalone '$' at the end of a token is undefined.
+# It is evaluated to an empty string in Make 4.1 or older, while
+# to '$' as-is in Make 4.2 or newer. Kconfig follows the behavior of
+# the newer Make version, but it is not important.
+
+# In the implementation of Kconfig, a standalone '$' at the end of a token
+# is treated as-is. So, the following also prints '$'
+$(warning $)
+
+# Likewise, a standalone '$' before a comma loses its special meaning.
+# The following prints '$X'. Do not expand '$X' even further.
+bar = $1$2
+$(warning $(bar $,X))
+
+# The following prints the value of the variable '$'.
+$(warning $($))
diff --git a/scripts/kconfig/tests/preprocess/escape/__init__.py b/scripts/kconfig/tests/preprocess/escape/__init__.py
new file mode 100644
index 0000000..841a754
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/escape/__init__.py
@@ -0,0 +1,9 @@
+"""
+Escape sequence tests.
+
+Test tricky cases related to escaping.
+"""
+
+def test(conf):
+ assert conf.oldaskconfig() == 0
+ assert conf.stderr_matches('expected_stderr')
diff --git a/scripts/kconfig/tests/preprocess/escape/expected_stderr b/scripts/kconfig/tests/preprocess/escape/expected_stderr
new file mode 100644
index 0000000..81df7e5
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/escape/expected_stderr
@@ -0,0 +1,10 @@
+Kconfig:5: arg0= arg1=
+Kconfig:9: arg0=, arg1=,
+Kconfig:13: ' " '" ' ''' "'"
+Kconfig:16: $
+Kconfig:20: $X
+Kconfig:26: nasty
+Kconfig:32: super_nasty
+Kconfig:41: $
+Kconfig:46: $X
+Kconfig:49: nasty
diff --git a/scripts/kconfig/tests/preprocess/user_func/Kconfig b/scripts/kconfig/tests/preprocess/user_func/Kconfig
new file mode 100644
index 0000000..9d248ca
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/user_func/Kconfig
@@ -0,0 +1,19 @@
+greeting = $(1), my name is $(2).
+$(warning $(greeting Hello,John))
+
+# It is allowed to pass more arguments than referenced.
+# Unreferenced parameters are just ignored.
+$(warning $(greeting Hello,John,ignored,ignored))
+
+# It is also allowed to give fewer arguments. $(2) will be blank in this case.
+$(warning $(greeting Hello))
+
+# Passing zero argument is OK. In fact, a user-defined function
+# is handle in the same way as a recursively expanded variable
+$(warning $(greeting))
+
+# However, if 1, 2 are defined as a global scope variable,
+# user-defined function will use them where arguments are missing
+1 = Hi
+2 = Tom
+$(warning $(greeting))
diff --git a/scripts/kconfig/tests/preprocess/user_func/__init__.py b/scripts/kconfig/tests/preprocess/user_func/__init__.py
new file mode 100644
index 0000000..8da60f6
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/user_func/__init__.py
@@ -0,0 +1,7 @@
+"""
+User-defined function tests.
+"""
+
+def test(conf):
+ assert conf.oldaskconfig() == 0
+ assert conf.stderr_matches('expected_stderr')
diff --git a/scripts/kconfig/tests/preprocess/user_func/expected_stderr b/scripts/kconfig/tests/preprocess/user_func/expected_stderr
new file mode 100644
index 0000000..1dd6684
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/user_func/expected_stderr
@@ -0,0 +1,5 @@
+Kconfig:2: Hello, my name is John.
+Kconfig:6: Hello, my name is John.
+Kconfig:9: Hello, my name is .
+Kconfig:13: , my name is .
+Kconfig:19: Hi, my name is Tom.
diff --git a/scripts/kconfig/tests/preprocess/variable/Kconfig b/scripts/kconfig/tests/preprocess/variable/Kconfig
new file mode 100644
index 0000000..df4446b
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/variable/Kconfig
@@ -0,0 +1,39 @@
+# Simply expanded variable.
+X := 1
+SIMPLE := $(X)
+X := 2
+$(warning SIMPLE = $(SIMPLE))
+
+# Recursively expanded variable.
+X := 1
+RECURSIVE = $(X)
+X := 2
+$(warning RECURSIVE = $(RECURSIVE))
+
+# Append something to a simply expanded variable.
+Y := 3
+SIMPLE += $(Y)
+Y := 4
+$(warning SIMPLE = $(SIMPLE))
+
+# Append something to a recursively expanded variable.
+Y := 3
+RECURSIVE += $(Y)
+Y := 4
+$(warning RECURSIVE = $(RECURSIVE))
+
+# Use += operator to an undefined variable.
+# This works as a recursively expanded variable.
+Y := 3
+UNDEFINED_VARIABLE += $(Y)
+Y := 4
+$(warning UNDEFINED_VARIABLE = $(UNDEFINED_VARIABLE))
+
+# You can use variable references for the lefthand side of assignment statement.
+X := A
+Y := B
+$(X)$(Y) := 5
+$(warning AB = $(AB))
+
+# By the way, you can omit the parentheses for a single-letter variable
+$(warning X = $X)
diff --git a/scripts/kconfig/tests/preprocess/variable/__init__.py b/scripts/kconfig/tests/preprocess/variable/__init__.py
new file mode 100644
index 0000000..e087698
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/variable/__init__.py
@@ -0,0 +1,7 @@
+"""
+Variable tests.
+"""
+
+def test(conf):
+ assert conf.oldaskconfig() == 0
+ assert conf.stderr_matches('expected_stderr')
diff --git a/scripts/kconfig/tests/preprocess/variable/expected_stderr b/scripts/kconfig/tests/preprocess/variable/expected_stderr
new file mode 100644
index 0000000..61b06e1
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/variable/expected_stderr
@@ -0,0 +1,7 @@
+Kconfig:5: SIMPLE = 1
+Kconfig:11: RECURSIVE = 2
+Kconfig:17: SIMPLE = 1 3
+Kconfig:23: RECURSIVE = 2 4
+Kconfig:30: UNDEFINED_VARIABLE = 4
+Kconfig:36: AB = 5
+Kconfig:39: X = A
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 19/30] kconfig: show compiler version text in the top comment
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (17 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 18/30] kconfig: test: test text expansion Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 20/30] kconfig: add basic helper macros to scripts/Kconfig.include Masahiro Yamada
` (12 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
The kernel configuration phase is now tightly coupled with the compiler
in use. It will be nice to show the compiler information in Kconfig.
The compiler information will be displayed like this:
$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- config
scripts/kconfig/conf --oldaskconfig Kconfig
*
* Linux/arm64 4.16.0-rc1 Kernel Configuration
*
*
* Compiler: aarch64-linux-gnu-gcc (Linaro GCC 7.2-2017.11) 7.2.1 20171011
*
*
* General setup
*
Compile also drivers which will not load (COMPILE_TEST) [N/y/?]
If you use GUI methods such as menuconfig, it will be displayed in the
top menu.
This is simply implemented by using the 'comment' statement. So, it
will be saved into the .config file as well.
This commit has a very important meaning. If the compiler is upgraded,
Kconfig must be re-run since different compilers have different sets
of supported options.
All referenced environments are written to include/config/auto.conf.cmd
so that any environment change triggers syncconfig, and prompt the user
to input new values if needed.
With this commit, something like follows will be added to
include/config/auto.conf.cmd
ifneq "$(CC_VERSION_TEXT)" "aarch64-linux-gnu-gcc (Linaro GCC 7.2-2017.11) 7.2.1 20171011"
include/config/auto.conf: FORCE
endif
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---
Changes in v3: None
Changes in v2: None
Kconfig | 2 ++
Makefile | 2 ++
2 files changed, 4 insertions(+)
diff --git a/Kconfig b/Kconfig
index 4af1b42..5b55d87 100644
--- a/Kconfig
+++ b/Kconfig
@@ -5,4 +5,6 @@
#
mainmenu "Linux/$(ARCH) $(KERNELVERSION) Kernel Configuration"
+comment "Compiler: $(CC_VERSION_TEXT)"
+
source "arch/$(SRCARCH)/Kconfig"
diff --git a/Makefile b/Makefile
index ca3e3e8..4b36329 100644
--- a/Makefile
+++ b/Makefile
@@ -442,6 +442,8 @@ export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_LDFLAGS_MODULE
export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL
export KBUILD_ARFLAGS
+export CC_VERSION_TEXT := $(shell $(CC) --version | head -n 1)
+
# When compiling out-of-tree modules, put MODVERDIR in the module
# tree rather than in the kernel tree. The kernel tree might
# even be read-only.
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 20/30] kconfig: add basic helper macros to scripts/Kconfig.include
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (18 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 19/30] kconfig: show compiler version text in the top comment Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-15 7:41 ` Ulf Magnusson
2018-04-13 5:06 ` [PATCH 21/30] stack-protector: test compiler capability in Kconfig and drop AUTO mode Masahiro Yamada
` (11 subsequent siblings)
31 siblings, 1 reply; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Kconfig got text processing tools like we see in Make. Add Kconfig
helper macros to scripts/Kconfig.include like we collect Makefile
macros in scripts/Kbuild.include.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
---
Changes in v3:
- Move helpers to scripts/Kconfig.include
Changes in v2: None
Kconfig | 2 ++
MAINTAINERS | 1 +
scripts/Kconfig.include | 17 +++++++++++++++++
3 files changed, 20 insertions(+)
create mode 100644 scripts/Kconfig.include
diff --git a/Kconfig b/Kconfig
index 5b55d87..a90d9f9 100644
--- a/Kconfig
+++ b/Kconfig
@@ -7,4 +7,6 @@ mainmenu "Linux/$(ARCH) $(KERNELVERSION) Kernel Configuration"
comment "Compiler: $(CC_VERSION_TEXT)"
+source "scripts/Kconfig.include"
+
source "arch/$(SRCARCH)/Kconfig"
diff --git a/MAINTAINERS b/MAINTAINERS
index b9dab38..d962f4a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7624,6 +7624,7 @@ L: linux-kbuild@vger.kernel.org
S: Maintained
F: Documentation/kbuild/kconfig*
F: scripts/kconfig/
+F: scripts/Kconfig.include
KDUMP
M: Dave Young <dyoung@redhat.com>
diff --git a/scripts/Kconfig.include b/scripts/Kconfig.include
new file mode 100644
index 0000000..cac7a81
--- /dev/null
+++ b/scripts/Kconfig.include
@@ -0,0 +1,17 @@
+# Kconfig helper macros
+
+# Convenient variables
+comma := ,
+quote := "
+squote := '
+empty :=
+space := $(empty) $(empty)
+
+# y if the command exits with 0, n otherwise
+success = $(shell ($(1)) >/dev/null 2>&1 && echo y || echo n)
+
+# y if the given compiler flag is supported, n otherwise
+cc-option = $(success $(CC) -Werror $(1) -c -x c /dev/null -o /dev/null)
+
+# y if the given linker flag is supported, n otherwise
+ld-option = $(success $(LD) -v $(1))
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [PATCH 20/30] kconfig: add basic helper macros to scripts/Kconfig.include
2018-04-13 5:06 ` [PATCH 20/30] kconfig: add basic helper macros to scripts/Kconfig.include Masahiro Yamada
@ 2018-04-15 7:41 ` Ulf Magnusson
2018-04-15 15:02 ` Masahiro Yamada
0 siblings, 1 reply; 53+ messages in thread
From: Ulf Magnusson @ 2018-04-15 7:41 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Linus Torvalds, Sam Ravnborg,
Nicholas Piggin, Kees Cook, Emese Revfy, X86 ML,
Linux Kernel Mailing List
On Fri, Apr 13, 2018 at 7:06 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> Kconfig got text processing tools like we see in Make. Add Kconfig
> helper macros to scripts/Kconfig.include like we collect Makefile
> macros in scripts/Kbuild.include.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> Reviewed-by: Kees Cook <keescook@chromium.org>
> Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
> ---
>
> Changes in v3:
> - Move helpers to scripts/Kconfig.include
>
> Changes in v2: None
>
> Kconfig | 2 ++
> MAINTAINERS | 1 +
> scripts/Kconfig.include | 17 +++++++++++++++++
> 3 files changed, 20 insertions(+)
> create mode 100644 scripts/Kconfig.include
>
> diff --git a/Kconfig b/Kconfig
> index 5b55d87..a90d9f9 100644
> --- a/Kconfig
> +++ b/Kconfig
> @@ -7,4 +7,6 @@ mainmenu "Linux/$(ARCH) $(KERNELVERSION) Kernel Configuration"
>
> comment "Compiler: $(CC_VERSION_TEXT)"
>
> +source "scripts/Kconfig.include"
> +
> source "arch/$(SRCARCH)/Kconfig"
> diff --git a/MAINTAINERS b/MAINTAINERS
> index b9dab38..d962f4a 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -7624,6 +7624,7 @@ L: linux-kbuild@vger.kernel.org
> S: Maintained
> F: Documentation/kbuild/kconfig*
> F: scripts/kconfig/
> +F: scripts/Kconfig.include
>
> KDUMP
> M: Dave Young <dyoung@redhat.com>
> diff --git a/scripts/Kconfig.include b/scripts/Kconfig.include
> new file mode 100644
> index 0000000..cac7a81
> --- /dev/null
> +++ b/scripts/Kconfig.include
> @@ -0,0 +1,17 @@
> +# Kconfig helper macros
> +
> +# Convenient variables
> +comma := ,
> +quote := "
> +squote := '
> +empty :=
> +space := $(empty) $(empty)
> +
> +# y if the command exits with 0, n otherwise
> +success = $(shell ($(1)) >/dev/null 2>&1 && echo y || echo n)
'{ $(1); }' might work here, to avoid the extra subshell.
Shaves 20-30% off the runtime here in a silly
system("(true) >/dev/null 2>&1 && echo y || echo n") vs.
system("{ true; } >/dev/null 2>&1 && echo y || echo n")
comparison.
> +
> +# y if the given compiler flag is supported, n otherwise
> +cc-option = $(success $(CC) -Werror $(1) -c -x c /dev/null -o /dev/null)
> +
> +# y if the given linker flag is supported, n otherwise
> +ld-option = $(success $(LD) -v $(1))
> --
> 2.7.4
>
Cheers,
Ulf
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH 20/30] kconfig: add basic helper macros to scripts/Kconfig.include
2018-04-15 7:41 ` Ulf Magnusson
@ 2018-04-15 15:02 ` Masahiro Yamada
0 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-15 15:02 UTC (permalink / raw)
To: Ulf Magnusson
Cc: Linux Kbuild mailing list, Linus Torvalds, Sam Ravnborg,
Nicholas Piggin, Kees Cook, Emese Revfy, X86 ML,
Linux Kernel Mailing List
2018-04-15 16:41 GMT+09:00 Ulf Magnusson <ulfalizer@gmail.com>:
> On Fri, Apr 13, 2018 at 7:06 AM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> Kconfig got text processing tools like we see in Make. Add Kconfig
>> helper macros to scripts/Kconfig.include like we collect Makefile
>> macros in scripts/Kbuild.include.
>>
>> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
>> Reviewed-by: Kees Cook <keescook@chromium.org>
>> Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
>> ---
>>
>> Changes in v3:
>> - Move helpers to scripts/Kconfig.include
>>
>> Changes in v2: None
>>
>> Kconfig | 2 ++
>> MAINTAINERS | 1 +
>> scripts/Kconfig.include | 17 +++++++++++++++++
>> 3 files changed, 20 insertions(+)
>> create mode 100644 scripts/Kconfig.include
>>
>> diff --git a/Kconfig b/Kconfig
>> index 5b55d87..a90d9f9 100644
>> --- a/Kconfig
>> +++ b/Kconfig
>> @@ -7,4 +7,6 @@ mainmenu "Linux/$(ARCH) $(KERNELVERSION) Kernel Configuration"
>>
>> comment "Compiler: $(CC_VERSION_TEXT)"
>>
>> +source "scripts/Kconfig.include"
>> +
>> source "arch/$(SRCARCH)/Kconfig"
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index b9dab38..d962f4a 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -7624,6 +7624,7 @@ L: linux-kbuild@vger.kernel.org
>> S: Maintained
>> F: Documentation/kbuild/kconfig*
>> F: scripts/kconfig/
>> +F: scripts/Kconfig.include
>>
>> KDUMP
>> M: Dave Young <dyoung@redhat.com>
>> diff --git a/scripts/Kconfig.include b/scripts/Kconfig.include
>> new file mode 100644
>> index 0000000..cac7a81
>> --- /dev/null
>> +++ b/scripts/Kconfig.include
>> @@ -0,0 +1,17 @@
>> +# Kconfig helper macros
>> +
>> +# Convenient variables
>> +comma := ,
>> +quote := "
>> +squote := '
>> +empty :=
>> +space := $(empty) $(empty)
>> +
>> +# y if the command exits with 0, n otherwise
>> +success = $(shell ($(1)) >/dev/null 2>&1 && echo y || echo n)
>
> '{ $(1); }' might work here, to avoid the extra subshell.
>
> Shaves 20-30% off the runtime here in a silly
> system("(true) >/dev/null 2>&1 && echo y || echo n") vs.
> system("{ true; } >/dev/null 2>&1 && echo y || echo n")
> comparison.
You are right. Will do so in the next version.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 53+ messages in thread
* [PATCH 21/30] stack-protector: test compiler capability in Kconfig and drop AUTO mode
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (19 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 20/30] kconfig: add basic helper macros to scripts/Kconfig.include Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 16:41 ` Kees Cook
2018-04-13 5:06 ` [PATCH 22/30] kconfig: add CC_IS_GCC and GCC_VERSION Masahiro Yamada
` (10 subsequent siblings)
31 siblings, 1 reply; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Move the test for -fstack-protector(-strong) option to Kconfig.
If the compiler does not support the option, the corresponding menu
is automatically hidden. If STRONG is not supported, it will fall
back to REGULAR. If REGULAR is not supported, it will be disabled.
This means, AUTO is implicitly handled by the dependency solver of
Kconfig, hence removed.
I also turned the 'choice' into only two boolean symbols. The use of
'choice' is not a good idea here, because all of all{yes,mod,no}config
would choose the first visible value, while we want allnoconfig to
disable as many features as possible.
X86 has additional shell scripts in case the compiler supports those
options, but generates broken code. I added CC_HAS_SANE_STACKPROTECTOR
to test this. I had to add -m32 to gcc-x86_32-has-stack-protector.sh
to make it work correctly.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
Makefile | 93 ++-----------------------------
arch/Kconfig | 32 ++++-------
arch/x86/Kconfig | 11 +++-
scripts/gcc-x86_32-has-stack-protector.sh | 7 +--
scripts/gcc-x86_64-has-stack-protector.sh | 5 --
5 files changed, 28 insertions(+), 120 deletions(-)
diff --git a/Makefile b/Makefile
index 4b36329..889d002 100644
--- a/Makefile
+++ b/Makefile
@@ -672,55 +672,11 @@ ifneq ($(CONFIG_FRAME_WARN),0)
KBUILD_CFLAGS += $(call cc-option,-Wframe-larger-than=${CONFIG_FRAME_WARN})
endif
-# This selects the stack protector compiler flag. Testing it is delayed
-# until after .config has been reprocessed, in the prepare-compiler-check
-# target.
-ifdef CONFIG_CC_STACKPROTECTOR_AUTO
- stackp-flag := $(call cc-option,-fstack-protector-strong,$(call cc-option,-fstack-protector))
- stackp-name := AUTO
-else
-ifdef CONFIG_CC_STACKPROTECTOR_REGULAR
- stackp-flag := -fstack-protector
- stackp-name := REGULAR
-else
-ifdef CONFIG_CC_STACKPROTECTOR_STRONG
- stackp-flag := -fstack-protector-strong
- stackp-name := STRONG
-else
- # If either there is no stack protector for this architecture or
- # CONFIG_CC_STACKPROTECTOR_NONE is selected, we're done, and $(stackp-name)
- # is empty, skipping all remaining stack protector tests.
- #
- # Force off for distro compilers that enable stack protector by default.
- KBUILD_CFLAGS += $(call cc-option, -fno-stack-protector)
-endif
-endif
-endif
-# Find arch-specific stack protector compiler sanity-checking script.
-ifdef stackp-name
-ifneq ($(stackp-flag),)
- stackp-path := $(srctree)/scripts/gcc-$(SRCARCH)_$(BITS)-has-stack-protector.sh
- stackp-check := $(wildcard $(stackp-path))
- # If the wildcard test matches a test script, run it to check functionality.
- ifdef stackp-check
- ifneq ($(shell $(CONFIG_SHELL) $(stackp-check) $(CC) $(KBUILD_CPPFLAGS) $(biarch)),y)
- stackp-broken := y
- endif
- endif
- ifndef stackp-broken
- # If the stack protector is functional, enable code that depends on it.
- KBUILD_CPPFLAGS += -DCONFIG_CC_STACKPROTECTOR
- # Either we've already detected the flag (for AUTO) or we'll fail the
- # build in the prepare-compiler-check rule (for specific flag).
- KBUILD_CFLAGS += $(stackp-flag)
- else
- # We have to make sure stack protector is unconditionally disabled if
- # the compiler is broken (in case we're going to continue the build in
- # AUTO mode).
- KBUILD_CFLAGS += $(call cc-option, -fno-stack-protector)
- endif
-endif
-endif
+stackp-flags-$(CONFIG_CC_HAS_STACKPROTECTOR_NONE) := -fno-stack-protector
+stackp-flags-$(CONFIG_CC_STACKPROTECTOR) := -fstack-protector
+stackp-flags-$(CONFIG_CC_STACKPROTECTOR_STRONG) := -fstack-protector-strong
+
+KBUILD_CFLAGS += $(stackp-flags-y)
ifeq ($(cc-name),clang)
KBUILD_CPPFLAGS += $(call cc-option,-Qunused-arguments,)
@@ -1096,7 +1052,7 @@ endif
# prepare2 creates a makefile if using a separate output directory.
# From this point forward, .config has been reprocessed, so any rules
# that need to depend on updated CONFIG_* values can be checked here.
-prepare2: prepare3 prepare-compiler-check outputmakefile asm-generic
+prepare2: prepare3 outputmakefile asm-generic
prepare1: prepare2 $(version_h) $(autoksyms_h) include/generated/utsrelease.h \
include/config/auto.conf
@@ -1122,43 +1078,6 @@ uapi-asm-generic:
PHONY += prepare-objtool
prepare-objtool: $(objtool_target)
-# Check for CONFIG flags that require compiler support. Abort the build
-# after .config has been processed, but before the kernel build starts.
-#
-# For security-sensitive CONFIG options, we don't want to fallback and/or
-# silently change which compiler flags will be used, since that leads to
-# producing kernels with different security feature characteristics
-# depending on the compiler used. (For example, "But I selected
-# CC_STACKPROTECTOR_STRONG! Why did it build with _REGULAR?!")
-PHONY += prepare-compiler-check
-prepare-compiler-check: FORCE
-# Make sure compiler supports requested stack protector flag.
-ifdef stackp-name
- # Warn about CONFIG_CC_STACKPROTECTOR_AUTO having found no option.
- ifeq ($(stackp-flag),)
- @echo CONFIG_CC_STACKPROTECTOR_$(stackp-name): \
- Compiler does not support any known stack-protector >&2
- else
- # Fail if specifically requested stack protector is missing.
- ifeq ($(call cc-option, $(stackp-flag)),)
- @echo Cannot use CONFIG_CC_STACKPROTECTOR_$(stackp-name): \
- $(stackp-flag) not supported by compiler >&2 && exit 1
- endif
- endif
-endif
-# Make sure compiler does not have buggy stack-protector support. If a
-# specific stack-protector was requested, fail the build, otherwise warn.
-ifdef stackp-broken
- ifeq ($(stackp-name),AUTO)
- @echo CONFIG_CC_STACKPROTECTOR_$(stackp-name): \
- $(stackp-flag) available but compiler is broken: disabling >&2
- else
- @echo Cannot use CONFIG_CC_STACKPROTECTOR_$(stackp-name): \
- $(stackp-flag) available but compiler is broken >&2 && exit 1
- endif
-endif
- @:
-
# Generate some files
# ---------------------------------------------------------------------------
diff --git a/arch/Kconfig b/arch/Kconfig
index 8e0d665..b0582f2 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -535,13 +535,16 @@ config HAVE_CC_STACKPROTECTOR
bool
help
An arch should select this symbol if:
- - its compiler supports the -fstack-protector option
- it has implemented a stack canary (e.g. __stack_chk_guard)
-choice
- prompt "Stack Protector buffer overflow detection"
+config CC_HAS_STACKPROTECTOR_NONE
+ def_bool $(cc-option -fno-stack-protector)
+
+config CC_STACKPROTECTOR
+ bool "Stack Protector buffer overflow detection"
depends on HAVE_CC_STACKPROTECTOR
- default CC_STACKPROTECTOR_AUTO
+ depends on $(cc-option -fstack-protector)
+ default y
help
This option turns on the "stack-protector" GCC feature. This
feature puts, at the beginning of functions, a canary value on
@@ -551,14 +554,6 @@ choice
overwrite the canary, which gets detected and the attack is then
neutralized via a kernel panic.
-config CC_STACKPROTECTOR_NONE
- bool "None"
- help
- Disable "stack-protector" GCC feature.
-
-config CC_STACKPROTECTOR_REGULAR
- bool "Regular"
- help
Functions will have the stack-protector canary logic added if they
have an 8-byte or larger character array on the stack.
@@ -570,7 +565,10 @@ config CC_STACKPROTECTOR_REGULAR
by about 0.3%.
config CC_STACKPROTECTOR_STRONG
- bool "Strong"
+ bool "Strong Stack Protector"
+ depends on CC_STACKPROTECTOR
+ depends on $(cc-option -fstack-protector-strong)
+ default y
help
Functions will have the stack-protector canary logic added in any
of the following conditions:
@@ -588,14 +586,6 @@ config CC_STACKPROTECTOR_STRONG
about 20% of all kernel functions, which increases the kernel code
size by about 2%.
-config CC_STACKPROTECTOR_AUTO
- bool "Automatic"
- help
- If the compiler supports it, the best available stack-protector
- option will be chosen.
-
-endchoice
-
config LD_DEAD_CODE_DATA_ELIMINATION
bool
help
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 72350d5..6f65146 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -124,7 +124,7 @@ config X86
select HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD if X86_64
select HAVE_ARCH_VMAP_STACK if X86_64
select HAVE_ARCH_WITHIN_STACK_FRAMES
- select HAVE_CC_STACKPROTECTOR
+ select HAVE_CC_STACKPROTECTOR if CC_HAS_SANE_STACKPROTECTOR
select HAVE_CMPXCHG_DOUBLE
select HAVE_CMPXCHG_LOCAL
select HAVE_CONTEXT_TRACKING if X86_64
@@ -340,6 +340,15 @@ config PGTABLE_LEVELS
default 2
source "init/Kconfig"
+
+config CC_HAS_SANE_STACKPROTECTOR
+ bool
+ default $(success $(srctree)/scripts/gcc-x86_64-has-stack-protector.sh $(CC)) if 64BIT
+ default $(success $(srctree)/scripts/gcc-x86_32-has-stack-protector.sh $(CC))
+ help
+ We have to make sure stack protector is unconditionally disabled if
+ the compiler produces broken code.
+
source "kernel/Kconfig.freezer"
menu "Processor type and features"
diff --git a/scripts/gcc-x86_32-has-stack-protector.sh b/scripts/gcc-x86_32-has-stack-protector.sh
index 6b2aeef..f5c1194 100755
--- a/scripts/gcc-x86_32-has-stack-protector.sh
+++ b/scripts/gcc-x86_32-has-stack-protector.sh
@@ -1,9 +1,4 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
-echo "int foo(void) { char X[200]; return 3; }" | $* -S -x c -c -O0 -fstack-protector - -o - 2> /dev/null | grep -q "%gs"
-if [ "$?" -eq "0" ] ; then
- echo y
-else
- echo n
-fi
+echo "int foo(void) { char X[200]; return 3; }" | $* -S -x c -c -m32 -O0 -fstack-protector - -o - 2> /dev/null | grep -q "%gs"
diff --git a/scripts/gcc-x86_64-has-stack-protector.sh b/scripts/gcc-x86_64-has-stack-protector.sh
index 4a48bdc..3755af0 100755
--- a/scripts/gcc-x86_64-has-stack-protector.sh
+++ b/scripts/gcc-x86_64-has-stack-protector.sh
@@ -2,8 +2,3 @@
# SPDX-License-Identifier: GPL-2.0
echo "int foo(void) { char X[200]; return 3; }" | $* -S -x c -c -O0 -mcmodel=kernel -fno-PIE -fstack-protector - -o - 2> /dev/null | grep -q "%gs"
-if [ "$?" -eq "0" ] ; then
- echo y
-else
- echo n
-fi
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [PATCH 21/30] stack-protector: test compiler capability in Kconfig and drop AUTO mode
2018-04-13 5:06 ` [PATCH 21/30] stack-protector: test compiler capability in Kconfig and drop AUTO mode Masahiro Yamada
@ 2018-04-13 16:41 ` Kees Cook
2018-04-13 18:11 ` Linus Torvalds
2018-04-15 9:40 ` Masahiro Yamada
0 siblings, 2 replies; 53+ messages in thread
From: Kees Cook @ 2018-04-13 16:41 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, Linus Torvalds, Sam Ravnborg, Ulf Magnusson,
Nicholas Piggin, Emese Revfy, X86 ML, LKML
On Thu, Apr 12, 2018 at 10:06 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> +stackp-flags-$(CONFIG_CC_HAS_STACKPROTECTOR_NONE) := -fno-stack-protector
> +stackp-flags-$(CONFIG_CC_STACKPROTECTOR) := -fstack-protector
> +stackp-flags-$(CONFIG_CC_STACKPROTECTOR_STRONG) := -fstack-protector-strong
> +
> +KBUILD_CFLAGS += $(stackp-flags-y)
So, technically, this works just fine. I wonder if it has an overly
confusing result, in that the compiler under normal situations will
see:
gcc ... -fno-stack-protector -fstack-protector -fstack-protector-strong ...
How about something like this instead:
ifdef CONFIG_CC_STACKPROTECTOR_STRONG
KBUILD_CFLAGS += -fstack-protector-strong
else
ifdef CONFIG_CC_STACKPROTECTOR
KBUILD_CFLAGS += -fstack-protector
else
KBUILD_CFLAGS += -fno-stack-protector
endif
endif
-Kees
--
Kees Cook
Pixel Security
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH 21/30] stack-protector: test compiler capability in Kconfig and drop AUTO mode
2018-04-13 16:41 ` Kees Cook
@ 2018-04-13 18:11 ` Linus Torvalds
2018-04-13 20:41 ` Kees Cook
2018-04-15 13:28 ` Masahiro Yamada
2018-04-15 9:40 ` Masahiro Yamada
1 sibling, 2 replies; 53+ messages in thread
From: Linus Torvalds @ 2018-04-13 18:11 UTC (permalink / raw)
To: Kees Cook
Cc: Masahiro Yamada, linux-kbuild, Sam Ravnborg, Ulf Magnusson,
Nicholas Piggin, Emese Revfy, X86 ML, LKML
On Fri, Apr 13, 2018 at 9:41 AM, Kees Cook <keescook@chromium.org> wrote:
>
> How about something like this instead:
I'd rather avoid the ifdef's in the Makefile if at all possible.
I'd rather expose this as a Kconfig rule, and in the Kconfig just have
an entry something like this
config STACKPROTECTOR_FLAGS
string
default "-fstack-protector-strong" if CC_STACKPROTECTOR_STRONG
default "-fstack-protector" if CC_STACKPROTECTOR
default "-fno-stack-protector" if CC_HAS_STACKPROTECTOR_NONE
default ""
which is really simple and straightforward. In the presense of
multiple defaults, the first is picked, so this _automatically_ does
that whole priority ordering.
And then the Makefile can just have
KBUILD_CFLAGS += $(CONFIG_STACKPROTECTOR_FLAGS)
which seems much simpler.
It also makes more complex conditionals easier (ie different compilers
with different flags, since clang sometimes does the same thing with
another flag name), so I'd rather see this pattern in general.
I'd also *much* rather do as much as possible at Kconfig time compared
to build time. Maybe it's just shifting the costs around, but the less
"clever" things we ask "make" to do, the better.
I find our Makefiles an odd combination of really clean and simply
(the ones that just have "obj-$(CONFIG_X) += xyz.o" are just lovely)
and completely incomprehensible (all of our infrastructure support for
the simple stuff).
I'd rather have more of the simple stuff in Makefiles, less of the
complex conditionals.
Linus
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH 21/30] stack-protector: test compiler capability in Kconfig and drop AUTO mode
2018-04-13 18:11 ` Linus Torvalds
@ 2018-04-13 20:41 ` Kees Cook
2018-04-15 13:28 ` Masahiro Yamada
1 sibling, 0 replies; 53+ messages in thread
From: Kees Cook @ 2018-04-13 20:41 UTC (permalink / raw)
To: Linus Torvalds
Cc: Masahiro Yamada, linux-kbuild, Sam Ravnborg, Ulf Magnusson,
Nicholas Piggin, Emese Revfy, X86 ML, LKML
On Fri, Apr 13, 2018 at 11:11 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> config STACKPROTECTOR_FLAGS
> string
> default "-fstack-protector-strong" if CC_STACKPROTECTOR_STRONG
> default "-fstack-protector" if CC_STACKPROTECTOR
> default "-fno-stack-protector" if CC_HAS_STACKPROTECTOR_NONE
> default ""
>
> which is really simple and straightforward. In the presense of
> multiple defaults, the first is picked, so this _automatically_ does
> that whole priority ordering.
Ah, perfect! Yes, this is a much better solution.
-Kees
--
Kees Cook
Pixel Security
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH 21/30] stack-protector: test compiler capability in Kconfig and drop AUTO mode
2018-04-13 18:11 ` Linus Torvalds
2018-04-13 20:41 ` Kees Cook
@ 2018-04-15 13:28 ` Masahiro Yamada
2018-04-15 16:04 ` Kees Cook
1 sibling, 1 reply; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-15 13:28 UTC (permalink / raw)
To: Linus Torvalds
Cc: Kees Cook, linux-kbuild, Sam Ravnborg, Ulf Magnusson,
Nicholas Piggin, Emese Revfy, X86 ML, LKML
2018-04-14 3:11 GMT+09:00 Linus Torvalds <torvalds@linux-foundation.org>:
> On Fri, Apr 13, 2018 at 9:41 AM, Kees Cook <keescook@chromium.org> wrote:
>>
>> How about something like this instead:
>
> I'd rather avoid the ifdef's in the Makefile if at all possible.
>
> I'd rather expose this as a Kconfig rule, and in the Kconfig just have
> an entry something like this
>
> config STACKPROTECTOR_FLAGS
> string
> default "-fstack-protector-strong" if CC_STACKPROTECTOR_STRONG
> default "-fstack-protector" if CC_STACKPROTECTOR
> default "-fno-stack-protector" if CC_HAS_STACKPROTECTOR_NONE
> default ""
>
> which is really simple and straightforward. In the presense of
> multiple defaults, the first is picked, so this _automatically_ does
> that whole priority ordering.
>
> And then the Makefile can just have
>
> KBUILD_CFLAGS += $(CONFIG_STACKPROTECTOR_FLAGS)
>
> which seems much simpler.
This has a minor problem.
A string type symbol encloses a value with "..." like
CONFIG_STACKPROTECTOR_FLAGS="-fstack-protector-strong"
I think the least ugliest notation to rip off "..."
is to do like follows:
KBUILD_CFLAGS += $(CONFIG_STACKPROTECTOR_FLAGS:"%"=%)
> It also makes more complex conditionals easier (ie different compilers
> with different flags, since clang sometimes does the same thing with
> another flag name), so I'd rather see this pattern in general.
>
> I'd also *much* rather do as much as possible at Kconfig time compared
> to build time. Maybe it's just shifting the costs around, but the less
> "clever" things we ask "make" to do, the better.
>
> I find our Makefiles an odd combination of really clean and simply
> (the ones that just have "obj-$(CONFIG_X) += xyz.o" are just lovely)
> and completely incomprehensible (all of our infrastructure support for
> the simple stuff).
>
> I'd rather have more of the simple stuff in Makefiles, less of the
> complex conditionals.
>
> Linus
You showed a preference of the following,
CONFIG_XYZ
bool "Enable xyz"
depends on $(cc-option -fxyz)
CONFIG_XYZ_FLAGS
string "-fxyz" if XYZ
Then, in Makefile
KBUILD_CFLAGS += $(CONFIG_XYZ_FLAGS:"%"=%)
Users usually say y/n to a question "do you want to use xyz?"
So, if you make this a general pattern,
it would require one additional symbol
to convert a bool symbol to a string.
I think the following was your original idea.
CONFIG_XYZ
bool "xyz"
depends on $(cc-option -fxyz)
Then, in Makefile
KBUILD_CFLAGS-$(CONFIG_XYZ) += -fxyz
Both notations have pros/cons, but
I slightly prefer the latter at the moment of time.
There are some complicated cases such as stack protector flags.
Your idea,
config STACKPROTECTOR_FLAGS
string
default "-fstack-protector-strong" if CC_STACKPROTECTOR_STRONG
default "-fstack-protector" if CC_STACKPROTECTOR
default "-fno-stack-protector" if CC_HAS_STACKPROTECTOR_NONE
default ""
works fine since the first visible default is picked.
In my Makefile ...
stackp-flags-$(CONFIG_CC_HAS_STACKPROTECTOR_NONE) := -fno-stack-protector
stackp-flags-$(CONFIG_CC_STACKPROTECTOR) := -fstack-protector
stackp-flags-$(CONFIG_CC_STACKPROTECTOR_STRONG) := -fstack-protector-strong
KBUILD_CFLAGS += $(stackp-flags-y)
the last one is picked.
IMHO, they are tied in terms of "cleanness".
Perhaps I may miss something, but we will have more time
to consider which approach is better.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH 21/30] stack-protector: test compiler capability in Kconfig and drop AUTO mode
2018-04-15 13:28 ` Masahiro Yamada
@ 2018-04-15 16:04 ` Kees Cook
0 siblings, 0 replies; 53+ messages in thread
From: Kees Cook @ 2018-04-15 16:04 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linus Torvalds, linux-kbuild, Sam Ravnborg, Ulf Magnusson,
Nicholas Piggin, Emese Revfy, X86 ML, LKML
On Sun, Apr 15, 2018 at 6:28 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> In my Makefile ...
>
> stackp-flags-$(CONFIG_CC_HAS_STACKPROTECTOR_NONE) := -fno-stack-protector
> stackp-flags-$(CONFIG_CC_STACKPROTECTOR) := -fstack-protector
> stackp-flags-$(CONFIG_CC_STACKPROTECTOR_STRONG) := -fstack-protector-strong
>
> KBUILD_CFLAGS += $(stackp-flags-y)
>
> the last one is picked.
Ah! Yes, sorry for the noise. I see the := instead of += now. Thanks!
-Kees
--
Kees Cook
Pixel Security
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH 21/30] stack-protector: test compiler capability in Kconfig and drop AUTO mode
2018-04-13 16:41 ` Kees Cook
2018-04-13 18:11 ` Linus Torvalds
@ 2018-04-15 9:40 ` Masahiro Yamada
1 sibling, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-15 9:40 UTC (permalink / raw)
To: Kees Cook
Cc: linux-kbuild, Linus Torvalds, Sam Ravnborg, Ulf Magnusson,
Nicholas Piggin, Emese Revfy, X86 ML, LKML
2018-04-14 1:41 GMT+09:00 Kees Cook <keescook@chromium.org>:
> On Thu, Apr 12, 2018 at 10:06 PM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> +stackp-flags-$(CONFIG_CC_HAS_STACKPROTECTOR_NONE) := -fno-stack-protector
>> +stackp-flags-$(CONFIG_CC_STACKPROTECTOR) := -fstack-protector
>> +stackp-flags-$(CONFIG_CC_STACKPROTECTOR_STRONG) := -fstack-protector-strong
>> +
>> +KBUILD_CFLAGS += $(stackp-flags-y)
>
> So, technically, this works just fine. I wonder if it has an overly
> confusing result, in that the compiler under normal situations will
> see:
>
> gcc ... -fno-stack-protector -fstack-protector -fstack-protector-strong ...
Kees, you are wrong.
Look at my code closely.
I used := operator instead of +=.
$(stackp-flags-y) contains only one flag at most.
> How about something like this instead:
>
> ifdef CONFIG_CC_STACKPROTECTOR_STRONG
> KBUILD_CFLAGS += -fstack-protector-strong
> else
> ifdef CONFIG_CC_STACKPROTECTOR
> KBUILD_CFLAGS += -fstack-protector
> else
> KBUILD_CFLAGS += -fno-stack-protector
> endif
> endif
>
My code is much cleaner, and working fine.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 53+ messages in thread
* [PATCH 22/30] kconfig: add CC_IS_GCC and GCC_VERSION
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (20 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 21/30] stack-protector: test compiler capability in Kconfig and drop AUTO mode Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 23/30] kconfig: add CC_IS_CLANG and CLANG_VERSION Masahiro Yamada
` (9 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
This will be useful to specify the required compiler version,
like this:
config FOO
bool "Use Foo"
depends on GCC_VERSION >= 408000
help
This feature requires GCC 4.8 or newer.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---
Changes in v3: None
Changes in v2: None
init/Kconfig | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/init/Kconfig b/init/Kconfig
index 4b0b636..145a534 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -8,6 +8,14 @@ config DEFCONFIG_LIST
default ARCH_DEFCONFIG
default "arch/$(ARCH)/defconfig"
+config CC_IS_GCC
+ def_bool $(success $(CC) --version | grep -q gcc)
+
+config GCC_VERSION
+ int
+ default $(shell $(srctree)/scripts/gcc-version.sh -p $(CC) | sed 's/^0*//') if CC_IS_GCC
+ default 0
+
config CONSTRUCTORS
bool
depends on !UML
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 23/30] kconfig: add CC_IS_CLANG and CLANG_VERSION
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (21 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 22/30] kconfig: add CC_IS_GCC and GCC_VERSION Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 24/30] gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT Masahiro Yamada
` (8 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
This will be useful to describe the clang version dependency.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---
Changes in v3: None
Changes in v2: None
init/Kconfig | 7 +++++++
scripts/clang-version.sh | 18 ++++--------------
2 files changed, 11 insertions(+), 14 deletions(-)
diff --git a/init/Kconfig b/init/Kconfig
index 145a534..c8f4ba3 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -16,6 +16,13 @@ config GCC_VERSION
default $(shell $(srctree)/scripts/gcc-version.sh -p $(CC) | sed 's/^0*//') if CC_IS_GCC
default 0
+config CC_IS_CLANG
+ def_bool $(success $(CC) --version | grep -q clang)
+
+config CLANG_VERSION
+ int
+ default $(shell $(srctree)/scripts/clang-version.sh $(CC))
+
config CONSTRUCTORS
bool
depends on !UML
diff --git a/scripts/clang-version.sh b/scripts/clang-version.sh
index 9780efa..dbf0a31 100755
--- a/scripts/clang-version.sh
+++ b/scripts/clang-version.sh
@@ -10,24 +10,14 @@
# clang-5.0.1 etc.
#
-if [ "$1" = "-p" ] ; then
- with_patchlevel=1;
- shift;
-fi
-
compiler="$*"
-if [ ${#compiler} -eq 0 ]; then
- echo "Error: No compiler specified."
- printf "Usage:\n\t$0 <clang-command>\n"
+if !( $compiler --version | grep -q clang) ; then
+ echo 0
exit 1
fi
MAJOR=$(echo __clang_major__ | $compiler -E -x c - | tail -n 1)
MINOR=$(echo __clang_minor__ | $compiler -E -x c - | tail -n 1)
-if [ "x$with_patchlevel" != "x" ] ; then
- PATCHLEVEL=$(echo __clang_patchlevel__ | $compiler -E -x c - | tail -n 1)
- printf "%02d%02d%02d\\n" $MAJOR $MINOR $PATCHLEVEL
-else
- printf "%02d%02d\\n" $MAJOR $MINOR
-fi
+PATCHLEVEL=$(echo __clang_patchlevel__ | $compiler -E -x c - | tail -n 1)
+printf "%d%02d%02d\\n" $MAJOR $MINOR $PATCHLEVEL
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 24/30] gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (22 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 23/30] kconfig: add CC_IS_CLANG and CLANG_VERSION Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 25/30] kcov: test compiler capability in Kconfig and correct dependency Masahiro Yamada
` (7 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
CONFIG_GCOV_FORMAT_AUTODETECT compiles either gcc_3_4.c or gcc_4_7.c
according to your GCC version.
We can achieve the equivalent behavior by setting reasonable dependency
with the knowledge of the compiler version.
If GCC older than 4.7 is used, GCOV_FORMAT_3_4 is the default, but users
are still allowed to select GCOV_FORMAT_4_7 in case the newer format is
back-ported.
On the other hand, If GCC 4.7 or newer is used, there is no reason to
use GCOV_FORMAT_3_4, so it should be hidden.
If you downgrade the compiler to GCC 4.7 or older, oldconfig/syncconfig
will display a prompt for the choice because GCOV_FORMAT_3_4 becomes
visible as a new symbol.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Peter Oberparleiter <oberpar@linux.vnet.ibm.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---
Changes in v3: None
Changes in v2: None
kernel/gcov/Kconfig | 17 +++++------------
kernel/gcov/Makefile | 2 --
2 files changed, 5 insertions(+), 14 deletions(-)
diff --git a/kernel/gcov/Kconfig b/kernel/gcov/Kconfig
index 1276aab..1e3823f 100644
--- a/kernel/gcov/Kconfig
+++ b/kernel/gcov/Kconfig
@@ -53,23 +53,16 @@ config GCOV_PROFILE_ALL
choice
prompt "Specify GCOV format"
depends on GCOV_KERNEL
- default GCOV_FORMAT_AUTODETECT
---help---
- The gcov format is usually determined by the GCC version, but there are
+ The gcov format is usually determined by the GCC version, and the
+ default is chosen according to your GCC version. However, there are
exceptions where format changes are integrated in lower-version GCCs.
- In such a case use this option to adjust the format used in the kernel
- accordingly.
-
- If unsure, choose "Autodetect".
-
-config GCOV_FORMAT_AUTODETECT
- bool "Autodetect"
- ---help---
- Select this option to use the format that corresponds to your GCC
- version.
+ In such a case, change this option to adjust the format used in the
+ kernel accordingly.
config GCOV_FORMAT_3_4
bool "GCC 3.4 format"
+ depends on CC_IS_GCC && GCC_VERSION < 40700
---help---
Select this option to use the format defined by GCC 3.4.
diff --git a/kernel/gcov/Makefile b/kernel/gcov/Makefile
index c6c50e5..ff06d64 100644
--- a/kernel/gcov/Makefile
+++ b/kernel/gcov/Makefile
@@ -4,5 +4,3 @@ ccflags-y := -DSRCTREE='"$(srctree)"' -DOBJTREE='"$(objtree)"'
obj-y := base.o fs.o
obj-$(CONFIG_GCOV_FORMAT_3_4) += gcc_3_4.o
obj-$(CONFIG_GCOV_FORMAT_4_7) += gcc_4_7.o
-obj-$(CONFIG_GCOV_FORMAT_AUTODETECT) += $(call cc-ifversion, -lt, 0407, \
- gcc_3_4.o, gcc_4_7.o)
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 25/30] kcov: test compiler capability in Kconfig and correct dependency
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (23 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 24/30] gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 26/30] gcc-plugins: move GCC version check for PowerPC to Kconfig Masahiro Yamada
` (6 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
As Documentation/kbuild/kconfig-language.txt notes, 'select' should be
be used with care - it forces a lower limit of another symbol, ignoring
the dependency. Currently, KCOV can select GCC_PLUGINS even if arch
does not select HAVE_GCC_PLUGINS. This could cause the unmet direct
dependency.
Now that Kconfig can test compiler capability, let's handle this in a
more sophisticated way.
There are two ways to enable KCOV; use the compiler that natively
supports -fsanitize-coverage=trace-pc, or build the SANCOV plugin if
the compiler has ability to build GCC plugins. Hence, the correct
dependency for KCOV is:
depends on CC_HAS_SANCOV_TRACE_PC || GCC_PLUGINS
You do not need to build the SANCOV plugin if the compiler already
supports -fsanitize-coverage=trace-pc. Hence, the select should be:
select GCC_PLUGIN_SANCOV if !CC_HAS_SANCOV_TRACE_PC
With this, GCC_PLUGIN_SANCOV is selected only when necessary, so
scripts/Makefile.gcc-plugins can be cleaner.
I also cleaned up Kconfig and scripts/Makefile.kcov as well.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3:
- Replace the previous 'select -> imply' patch with
a new approach
Changes in v2:
- Drop depends on GCC_VERSION
Makefile | 2 +-
lib/Kconfig.debug | 11 +++++++----
scripts/Makefile.gcc-plugins | 6 +-----
scripts/Makefile.kcov | 10 ++++++----
4 files changed, 15 insertions(+), 14 deletions(-)
diff --git a/Makefile b/Makefile
index 889d002..88733b7 100644
--- a/Makefile
+++ b/Makefile
@@ -623,7 +623,7 @@ all: vmlinux
KBUILD_CFLAGS += $(call cc-option,-fno-PIE)
KBUILD_AFLAGS += $(call cc-option,-fno-PIE)
CFLAGS_GCOV := -fprofile-arcs -ftest-coverage -fno-tree-loop-im $(call cc-disable-warning,maybe-uninitialized,)
-export CFLAGS_GCOV CFLAGS_KCOV
+export CFLAGS_GCOV
# The arch Makefile can set ARCH_{CPP,A,C}FLAGS to override the default
# values of the respective KBUILD_* variables
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index c40c7b7..1335717 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -736,12 +736,15 @@ config ARCH_HAS_KCOV
only for x86_64. KCOV requires testing on other archs, and most likely
disabling of instrumentation for some early boot code.
+config CC_HAS_SANCOV_TRACE_PC
+ def_bool $(cc-option -fsanitize-coverage=trace-pc)
+
config KCOV
bool "Code coverage for fuzzing"
depends on ARCH_HAS_KCOV
+ depends on CC_HAS_SANCOV_TRACE_PC || GCC_PLUGINS
select DEBUG_FS
- select GCC_PLUGINS if !COMPILE_TEST
- select GCC_PLUGIN_SANCOV if !COMPILE_TEST
+ select GCC_PLUGIN_SANCOV if !CC_HAS_SANCOV_TRACE_PC
help
KCOV exposes kernel code coverage information in a form suitable
for coverage-guided fuzzing (randomized testing).
@@ -755,7 +758,7 @@ config KCOV
config KCOV_ENABLE_COMPARISONS
bool "Enable comparison operands collection by KCOV"
depends on KCOV
- default n
+ depends on $(cc-option -fsanitize-coverage=trace-cmp)
help
KCOV also exposes operands of every comparison in the instrumented
code along with operand sizes and PCs of the comparison instructions.
@@ -765,7 +768,7 @@ config KCOV_ENABLE_COMPARISONS
config KCOV_INSTRUMENT_ALL
bool "Instrument all code by default"
depends on KCOV
- default y if KCOV
+ default y
help
If you are doing generic system call fuzzing (like e.g. syzkaller),
then you will want to instrument the whole kernel and you should
diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins
index 7f5c862..0ce3802 100644
--- a/scripts/Makefile.gcc-plugins
+++ b/scripts/Makefile.gcc-plugins
@@ -14,16 +14,12 @@ ifdef CONFIG_GCC_PLUGINS
endif
ifdef CONFIG_GCC_PLUGIN_SANCOV
- ifeq ($(strip $(CFLAGS_KCOV)),)
# It is needed because of the gcc-plugin.sh and gcc version checks.
gcc-plugin-$(CONFIG_GCC_PLUGIN_SANCOV) += sancov_plugin.so
- ifneq ($(PLUGINCC),)
- CFLAGS_KCOV := $(SANCOV_PLUGIN)
- else
+ ifeq ($(PLUGINCC),)
$(warning warning: cannot use CONFIG_KCOV: -fsanitize-coverage=trace-pc is not supported by compiler)
endif
- endif
endif
gcc-plugin-$(CONFIG_GCC_PLUGIN_STRUCTLEAK) += structleak_plugin.so
diff --git a/scripts/Makefile.kcov b/scripts/Makefile.kcov
index 5cc7203..d71ba73 100644
--- a/scripts/Makefile.kcov
+++ b/scripts/Makefile.kcov
@@ -1,7 +1,9 @@
ifdef CONFIG_KCOV
-CFLAGS_KCOV := $(call cc-option,-fsanitize-coverage=trace-pc,)
-ifeq ($(CONFIG_KCOV_ENABLE_COMPARISONS),y)
-CFLAGS_KCOV += $(call cc-option,-fsanitize-coverage=trace-cmp,)
-endif
+
+kcov-flags-$(CONFIG_CC_HAS_SANCOV_TRACE_PC) += -fsanitize-coverage=trace-pc
+kcov-flags-$(CONFIG_KCOV_ENABLE_COMPARISONS) += -fsanitize-coverage=trace-cmp
+kcov-flags-$(CONFIG_GCC_PLUGIN_SANKOV) += -fplugin=$(objtree)/scripts/gcc-plugins/sancov_plugin.so
+
+export CFLAGS_KCOV := $(kcov-flags-y)
endif
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 26/30] gcc-plugins: move GCC version check for PowerPC to Kconfig
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (24 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 25/30] kcov: test compiler capability in Kconfig and correct dependency Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 27/30] gcc-plugins: test plugin support in Kconfig and clean up Makefile Masahiro Yamada
` (5 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
For PowerPC, GCC 5.2 is the requirement for GCC plugins. Move the
version check to Kconfig so that the GCC plugin menus will be hidden
if an older compiler is in use.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
---
Changes in v3:
- Move comment to Kconfig as well
Changes in v2: None
arch/powerpc/Kconfig | 2 +-
scripts/Makefile.gcc-plugins | 8 --------
2 files changed, 1 insertion(+), 9 deletions(-)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 73ce5dd..512fcc1 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -195,7 +195,7 @@ config PPC
select HAVE_FTRACE_MCOUNT_RECORD
select HAVE_FUNCTION_GRAPH_TRACER
select HAVE_FUNCTION_TRACER
- select HAVE_GCC_PLUGINS
+ select HAVE_GCC_PLUGINS if GCC_VERSION >= 50200 # plugin support on gcc <= 5.1 is buggy on PPC
select HAVE_GENERIC_GUP
select HAVE_HW_BREAKPOINT if PERF_EVENTS && (PPC_BOOK3S || PPC_8xx)
select HAVE_IDE
diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins
index 0ce3802..1e92353 100644
--- a/scripts/Makefile.gcc-plugins
+++ b/scripts/Makefile.gcc-plugins
@@ -53,14 +53,6 @@ gcc-plugins-check: FORCE
ifdef CONFIG_GCC_PLUGINS
ifeq ($(PLUGINCC),)
ifneq ($(GCC_PLUGINS_CFLAGS),)
- # Various gccs between 4.5 and 5.1 have bugs on powerpc due to missing
- # header files. gcc <= 4.6 doesn't work at all, gccs from 4.8 to 5.1 have
- # issues with 64-bit targets.
- ifeq ($(ARCH),powerpc)
- ifeq ($(call cc-ifversion, -le, 0501, y), y)
- @echo "Cannot use CONFIG_GCC_PLUGINS: plugin support on gcc <= 5.1 is buggy on powerpc, please upgrade to gcc 5.2 or newer" >&2 && exit 1
- endif
- endif
ifeq ($(call cc-ifversion, -ge, 0405, y), y)
$(Q)$(srctree)/scripts/gcc-plugin.sh --show-error "$(__PLUGINCC)" "$(HOSTCXX)" "$(CC)" || true
@echo "Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?" >&2 && exit 1
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 27/30] gcc-plugins: test plugin support in Kconfig and clean up Makefile
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (25 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 26/30] gcc-plugins: move GCC version check for PowerPC to Kconfig Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 28/30] gcc-plugins: allow to enable GCC_PLUGINS for COMPILE_TEST Masahiro Yamada
` (4 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Run scripts/gcc-plugin.sh from Kconfig so that users can enable
GCC_PLUGINS only when the compiler supports building plugins.
Kconfig defines a new symbol, PLUGIN_HOSTCC. This will contain
the compiler (g++ or gcc) used for building plugins, or empty
if the plugin can not be supported at all.
This allows us to remove all ugly testing in Makefile.gcc-plugins.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3:
- Respin to keep scripts/gcc-plugin.sh as-is.
Changes in v2: None
arch/Kconfig | 10 ++++++
scripts/Kconfig.include | 3 ++
scripts/Makefile.gcc-plugins | 79 ++++++++++++--------------------------------
scripts/gcc-plugins/Makefile | 1 +
4 files changed, 36 insertions(+), 57 deletions(-)
diff --git a/arch/Kconfig b/arch/Kconfig
index b0582f2..9166157 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -401,6 +401,15 @@ config SECCOMP_FILTER
See Documentation/prctl/seccomp_filter.txt for details.
+preferred-plugin-hostcc := $(shell [ $(gcc-version) -ge 40800 ] && echo $(HOSTCXX) || echo $(HOSTCC))
+
+config PLUGIN_HOSTCC
+ string
+ default "$(shell $(srctree)/scripts/gcc-plugin.sh $(preferred-plugin-hostcc) $(HOSTCXX) $(CC))"
+ help
+ Host compiler used to build GCC plugins. This can be $(HOSTCXX),
+ $(HOSTCC), or a null string if GCC plugin is unsupported.
+
config HAVE_GCC_PLUGINS
bool
help
@@ -410,6 +419,7 @@ config HAVE_GCC_PLUGINS
menuconfig GCC_PLUGINS
bool "GCC plugins"
depends on HAVE_GCC_PLUGINS
+ depends on PLUGIN_HOSTCC != ""
depends on !COMPILE_TEST
help
GCC plugins are loadable modules that provide extra features to the
diff --git a/scripts/Kconfig.include b/scripts/Kconfig.include
index cac7a81..0864a19 100644
--- a/scripts/Kconfig.include
+++ b/scripts/Kconfig.include
@@ -15,3 +15,6 @@ cc-option = $(success $(CC) -Werror $(1) -c -x c /dev/null -o /dev/null)
# y if the given linker flag is supported, n otherwise
ld-option = $(success $(LD) -v $(1))
+
+# gcc version including patch level
+gcc-version := $(shell $(srctree)/scripts/gcc-version.sh -p $(CC) | sed 's/^0*//')
diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins
index 1e92353..da5d38d 100644
--- a/scripts/Makefile.gcc-plugins
+++ b/scripts/Makefile.gcc-plugins
@@ -1,72 +1,37 @@
# SPDX-License-Identifier: GPL-2.0
-ifdef CONFIG_GCC_PLUGINS
- __PLUGINCC := $(call cc-ifversion, -ge, 0408, $(HOSTCXX), $(HOSTCC))
- PLUGINCC := $(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-plugin.sh "$(__PLUGINCC)" "$(HOSTCXX)" "$(CC)")
-
- SANCOV_PLUGIN := -fplugin=$(objtree)/scripts/gcc-plugins/sancov_plugin.so
-
- gcc-plugin-$(CONFIG_GCC_PLUGIN_CYC_COMPLEXITY) += cyc_complexity_plugin.so
+gcc-plugin-$(CONFIG_GCC_PLUGIN_CYC_COMPLEXITY) += cyc_complexity_plugin.so
- gcc-plugin-$(CONFIG_GCC_PLUGIN_LATENT_ENTROPY) += latent_entropy_plugin.so
- gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_LATENT_ENTROPY) += -DLATENT_ENTROPY_PLUGIN
- ifdef CONFIG_GCC_PLUGIN_LATENT_ENTROPY
+gcc-plugin-$(CONFIG_GCC_PLUGIN_LATENT_ENTROPY) += latent_entropy_plugin.so
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_LATENT_ENTROPY) += -DLATENT_ENTROPY_PLUGIN
+ifdef CONFIG_GCC_PLUGIN_LATENT_ENTROPY
DISABLE_LATENT_ENTROPY_PLUGIN += -fplugin-arg-latent_entropy_plugin-disable
- endif
-
- ifdef CONFIG_GCC_PLUGIN_SANCOV
- # It is needed because of the gcc-plugin.sh and gcc version checks.
- gcc-plugin-$(CONFIG_GCC_PLUGIN_SANCOV) += sancov_plugin.so
-
- ifeq ($(PLUGINCC),)
- $(warning warning: cannot use CONFIG_KCOV: -fsanitize-coverage=trace-pc is not supported by compiler)
- endif
- endif
-
- gcc-plugin-$(CONFIG_GCC_PLUGIN_STRUCTLEAK) += structleak_plugin.so
- gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK_VERBOSE) += -fplugin-arg-structleak_plugin-verbose
- gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK_BYREF_ALL) += -fplugin-arg-structleak_plugin-byref-all
- gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK) += -DSTRUCTLEAK_PLUGIN
+endif
- gcc-plugin-$(CONFIG_GCC_PLUGIN_RANDSTRUCT) += randomize_layout_plugin.so
- gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_RANDSTRUCT) += -DRANDSTRUCT_PLUGIN
- gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_RANDSTRUCT_PERFORMANCE) += -fplugin-arg-randomize_layout_plugin-performance-mode
+gcc-plugin-$(CONFIG_GCC_PLUGIN_SANCOV) += sancov_plugin.so
+gcc-plugin-$(CONFIG_GCC_PLUGIN_STRUCTLEAK) += structleak_plugin.so
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK_VERBOSE) += -fplugin-arg-structleak_plugin-verbose
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK_BYREF_ALL) += -fplugin-arg-structleak_plugin-byref-all
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK) += -DSTRUCTLEAK_PLUGIN
- GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y))
+gcc-plugin-$(CONFIG_GCC_PLUGIN_RANDSTRUCT) += randomize_layout_plugin.so
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_RANDSTRUCT) += -DRANDSTRUCT_PLUGIN
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_RANDSTRUCT_PERFORMANCE) += -fplugin-arg-randomize_layout_plugin-performance-mode
- export PLUGINCC GCC_PLUGINS_CFLAGS GCC_PLUGIN GCC_PLUGIN_SUBDIR
- export SANCOV_PLUGIN DISABLE_LATENT_ENTROPY_PLUGIN
+GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y))
- ifneq ($(PLUGINCC),)
- # SANCOV_PLUGIN can be only in CFLAGS_KCOV because avoid duplication.
- GCC_PLUGINS_CFLAGS := $(filter-out $(SANCOV_PLUGIN), $(GCC_PLUGINS_CFLAGS))
- endif
+export GCC_PLUGINS_CFLAGS GCC_PLUGIN GCC_PLUGIN_SUBDIR
+export DISABLE_LATENT_ENTROPY_PLUGIN
- KBUILD_CFLAGS += $(GCC_PLUGINS_CFLAGS)
- GCC_PLUGIN := $(gcc-plugin-y)
- GCC_PLUGIN_SUBDIR := $(gcc-plugin-subdir-y)
-endif
+# SANCOV_PLUGIN can be only in CFLAGS_KCOV because avoid duplication.
+GCC_PLUGINS_CFLAGS := $(filter-out %/sancov_plugin.so, $(GCC_PLUGINS_CFLAGS))
-# If plugins aren't supported, abort the build before hard-to-read compiler
-# errors start getting spewed by the main build.
-PHONY += gcc-plugins-check
-gcc-plugins-check: FORCE
-ifdef CONFIG_GCC_PLUGINS
- ifeq ($(PLUGINCC),)
- ifneq ($(GCC_PLUGINS_CFLAGS),)
- ifeq ($(call cc-ifversion, -ge, 0405, y), y)
- $(Q)$(srctree)/scripts/gcc-plugin.sh --show-error "$(__PLUGINCC)" "$(HOSTCXX)" "$(CC)" || true
- @echo "Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?" >&2 && exit 1
- else
- @echo "Cannot use CONFIG_GCC_PLUGINS: your gcc version does not support plugins, you should upgrade it to at least gcc 4.5" >&2 && exit 1
- endif
- endif
- endif
-endif
- @:
+KBUILD_CFLAGS += $(GCC_PLUGINS_CFLAGS)
+GCC_PLUGIN := $(gcc-plugin-y)
+GCC_PLUGIN_SUBDIR := $(gcc-plugin-subdir-y)
# Actually do the build, if requested.
PHONY += gcc-plugins
-gcc-plugins: scripts_basic gcc-plugins-check
+gcc-plugins: scripts_basic
ifdef CONFIG_GCC_PLUGINS
$(Q)$(MAKE) $(build)=scripts/gcc-plugins
endif
diff --git a/scripts/gcc-plugins/Makefile b/scripts/gcc-plugins/Makefile
index e2ff425..c47198c 100644
--- a/scripts/gcc-plugins/Makefile
+++ b/scripts/gcc-plugins/Makefile
@@ -1,4 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
+PLUGINCC := $(CONFIG_PLUGIN_HOSTCC:"%"=%)
GCC_PLUGINS_DIR := $(shell $(CC) -print-file-name=plugin)
ifeq ($(PLUGINCC),$(HOSTCC))
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 28/30] gcc-plugins: allow to enable GCC_PLUGINS for COMPILE_TEST
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (26 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 27/30] gcc-plugins: test plugin support in Kconfig and clean up Makefile Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 29/30] arm64: move GCC version check for ARCH_SUPPORTS_INT128 to Kconfig Masahiro Yamada
` (3 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
Now that the compiler's plugin support is checked in Kconfig,
all{yes,mod}config will not be bothered.
Remove 'depends on !COMPILE_TEST' for GCC_PLUGINS.
'depends on !COMPILE_TEST' for the following three are still kept:
GCC_PLUGIN_CYC_COMPLEXITY
GCC_PLUGIN_STRUCTLEAK_VERBOSE
GCC_PLUGIN_RANDSTRUCT_PERFORMANCE
Kees said to do so because the first two are too noisy, and the last
one would reduce the compile test coverage. I commented the reasons
in arch/Kconfig.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
arch/Kconfig | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/arch/Kconfig b/arch/Kconfig
index 9166157..95b9b2e 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -420,7 +420,6 @@ menuconfig GCC_PLUGINS
bool "GCC plugins"
depends on HAVE_GCC_PLUGINS
depends on PLUGIN_HOSTCC != ""
- depends on !COMPILE_TEST
help
GCC plugins are loadable modules that provide extra features to the
compiler. They are useful for runtime instrumentation and static analysis.
@@ -430,7 +429,7 @@ menuconfig GCC_PLUGINS
config GCC_PLUGIN_CYC_COMPLEXITY
bool "Compute the cyclomatic complexity of a function" if EXPERT
depends on GCC_PLUGINS
- depends on !COMPILE_TEST
+ depends on !COMPILE_TEST # too noisy
help
The complexity M of a function's control flow graph is defined as:
M = E - N + 2P
@@ -493,7 +492,7 @@ config GCC_PLUGIN_STRUCTLEAK_BYREF_ALL
config GCC_PLUGIN_STRUCTLEAK_VERBOSE
bool "Report forcefully initialized variables"
depends on GCC_PLUGIN_STRUCTLEAK
- depends on !COMPILE_TEST
+ depends on !COMPILE_TEST # too noisy
help
This option will cause a warning to be printed each time the
structleak plugin finds a variable it thinks needs to be
@@ -533,7 +532,7 @@ config GCC_PLUGIN_RANDSTRUCT
config GCC_PLUGIN_RANDSTRUCT_PERFORMANCE
bool "Use cacheline-aware structure randomization"
depends on GCC_PLUGIN_RANDSTRUCT
- depends on !COMPILE_TEST
+ depends on !COMPILE_TEST # do not reduce test coverage
help
If you say Y here, the RANDSTRUCT randomization will make a
best effort at restricting randomization to cacheline-sized
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 29/30] arm64: move GCC version check for ARCH_SUPPORTS_INT128 to Kconfig
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (27 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 28/30] gcc-plugins: allow to enable GCC_PLUGINS for COMPILE_TEST Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:06 ` [PATCH 30/30] kbuild: test dead code/data elimination support in Kconfig Masahiro Yamada
` (2 subsequent siblings)
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
This becomes much neater in Kconfig.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---
Changes in v3: None
Changes in v2: None
arch/arm64/Kconfig | 1 +
arch/arm64/Makefile | 2 --
2 files changed, 1 insertion(+), 2 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index eb2cf49..09d1aee 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -43,6 +43,7 @@ config ARM64
select ARCH_USE_QUEUED_RWLOCKS
select ARCH_SUPPORTS_MEMORY_FAILURE
select ARCH_SUPPORTS_ATOMIC_RMW
+ select ARCH_SUPPORTS_INT128 if GCC_VERSION >= 50000
select ARCH_SUPPORTS_NUMA_BALANCING
select ARCH_WANT_COMPAT_IPC_PARSE_VERSION
select ARCH_WANT_FRAME_POINTERS
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 1540286..cefd1e9 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -56,8 +56,6 @@ KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
-KBUILD_CFLAGS += $(call cc-ifversion, -ge, 0500, -DCONFIG_ARCH_SUPPORTS_INT128)
-
ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
KBUILD_CPPFLAGS += -mbig-endian
CHECKFLAGS += -D__AARCH64EB__
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [PATCH 30/30] kbuild: test dead code/data elimination support in Kconfig
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (28 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 29/30] arm64: move GCC version check for ARCH_SUPPORTS_INT128 to Kconfig Masahiro Yamada
@ 2018-04-13 5:06 ` Masahiro Yamada
2018-04-13 5:17 ` [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
2018-04-13 5:52 ` Kees Cook
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:06 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, x86, Masahiro Yamada, linux-kernel
This config option should be enabled only when both the compiler and
the linker support necessary flags. Add proper dependencies to Kconfig.
Unlike 'select', 'imply' is modest enough to observe those dependencies.
I suggested this in the help message.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
Makefile | 8 ++------
arch/Kconfig | 4 +++-
2 files changed, 5 insertions(+), 7 deletions(-)
diff --git a/Makefile b/Makefile
index 88733b7..92db767 100644
--- a/Makefile
+++ b/Makefile
@@ -754,8 +754,8 @@ KBUILD_CFLAGS += $(call cc-option, -fno-inline-functions-called-once)
endif
ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
-KBUILD_CFLAGS += $(call cc-option,-ffunction-sections,)
-KBUILD_CFLAGS += $(call cc-option,-fdata-sections,)
+KBUILD_CFLAGS += -ffunction-sections -fdata-sections
+LDFLAGS_vmlinux += --gc-sections
endif
# arch Makefile may override CC so keep this after arch Makefile is included
@@ -819,10 +819,6 @@ LDFLAGS_BUILD_ID := $(call ld-option, --build-id)
KBUILD_LDFLAGS_MODULE += $(LDFLAGS_BUILD_ID)
LDFLAGS_vmlinux += $(LDFLAGS_BUILD_ID)
-ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
-LDFLAGS_vmlinux += $(call ld-option, --gc-sections,)
-endif
-
ifeq ($(CONFIG_STRIP_ASM_SYMS),y)
LDFLAGS_vmlinux += $(call ld-option, -X,)
endif
diff --git a/arch/Kconfig b/arch/Kconfig
index 95b9b2e..e88f1ba 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -597,8 +597,10 @@ config CC_STACKPROTECTOR_STRONG
config LD_DEAD_CODE_DATA_ELIMINATION
bool
+ depends on $(cc-option -ffunction-sections -fdata-sections)
+ depends on $(ld-option --gc-sections)
help
- Select this if the architecture wants to do dead code and
+ Imply this if the architecture wants to do dead code and
data elimination with the linker by compiling with
-ffunction-sections -fdata-sections and linking with
--gc-sections.
--
2.7.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [PATCH 00/30] kconfig: move compiler capability tests to Kconfig
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (29 preceding siblings ...)
2018-04-13 5:06 ` [PATCH 30/30] kbuild: test dead code/data elimination support in Kconfig Masahiro Yamada
@ 2018-04-13 5:17 ` Masahiro Yamada
2018-04-13 5:52 ` Kees Cook
31 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 5:17 UTC (permalink / raw)
To: Linux Kbuild mailing list
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Kees Cook, Emese Revfy, X86 ML, Masahiro Yamada,
Linux Kernel Mailing List
2018-04-13 14:06 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
I forgot to prefix the patch subjects with the version.
This series is V3.
> [Introduction]
>
> The motivation of this work is to move the compiler option tests to
> Kconfig from Makefile. A number of kernel features require the
> compiler support. Enabling such features blindly in Kconfig ends up
> with a lot of nasty build-time testing in Makefiles. If a chosen
> feature turns out unsupported by the compiler, what the build system
> can do is either to disable it (silently!) or to forcibly break the
> build, despite Kconfig has let the user to enable it.
>
> [Major Changes in V3]
>
> This version looks more like Make.
>
> - Use = operator instead of 'macro' keyword
> to define a user-defined function.
>
> - 'Recursively expanded variable' is implemented as a side-effect.
> A variable is a function with zero argument.
>
> - Support simply expanded variable which is defined by := operator
>
> - Support += operator.
> Probably, this feature will be useful to accumulate compiler flags.
> At least, Clang needs some prerequisite flags such as triplet
> to test other compiler flags.
>
> - Support $(info ...) and $(warning ...) built-in functions,
> which were useful while I was debugging this.
>
> - Add documentation
>
> - Add unit tests
>
> - Collect helpers to scripts/Kconfig.include
>
> [TODO items]
>
> I have not been able to finish this. Maybe aiming for v4.18-rc1.
>
> - The text expansion should be able to defer argument expansion.
> This is necessary if we want to implement $(if ...) built-in function.
>
> - Error out recursive expansion of itself.
> For example,
> X = $(X)
> refers itself.
> If referenced, the expansion continues eternally. This must be terminated.
>
> - More compatible behavior?
> Kconfig works mostly like Make, but there are naive cases
> where the behavior is different.
>
> [Old Versions]
>
> V2: https://lkml.org/lkml/2018/2/16/610
> V1: https://lkml.org/lkml/2018/2/16/610
> RFC: https://lkml.org/lkml/2018/2/8/429
>
>
>
> Masahiro Yamada (30):
> gcc-plugins: fix build condition of SANCOV plugin
> kbuild: remove kbuild cache
> kbuild: remove CONFIG_CROSS_COMPILE support
> kconfig: reference environment variables directly and remove 'option
> env='
> kconfig: remove string expansion in file_lookup()
> kconfig: remove string expansion for mainmenu after yyparse()
> kconfig: remove sym_expand_string_value()
> kconfig: add built-in function support
> kconfig: add 'shell' built-in function
> kconfig: replace $(UNAME_RELEASE) with function call
> kconfig: begin PARAM state only when seeing a command keyword
> kconfig: support variable and user-defined function
> kconfig: support simply expanded variable
> kconfig: support append assignment operator
> kconfig: expand lefthand side of assignment statement
> kconfig: add 'info' and 'warning' built-in functions
> Documentation: kconfig: document a new Kconfig macro language
> kconfig: test: test text expansion
> kconfig: show compiler version text in the top comment
> kconfig: add basic helper macros to scripts/Kconfig.include
> stack-protector: test compiler capability in Kconfig and drop AUTO
> mode
> kconfig: add CC_IS_GCC and GCC_VERSION
> kconfig: add CC_IS_CLANG and CLANG_VERSION
> gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT
> kcov: test compiler capability in Kconfig and correct dependency
> gcc-plugins: move GCC version check for PowerPC to Kconfig
> gcc-plugins: test plugin support in Kconfig and clean up Makefile
> gcc-plugins: allow to enable GCC_PLUGINS for COMPILE_TEST
> arm64: move GCC version check for ARCH_SUPPORTS_INT128 to Kconfig
> kbuild: test dead code/data elimination support in Kconfig
>
> Documentation/kbuild/kconfig-language.txt | 8 -
> Documentation/kbuild/kconfig-macro-language.txt | 179 +++++++
> Kconfig | 10 +-
> MAINTAINERS | 3 +-
> Makefile | 113 +---
> arch/Kconfig | 53 +-
> arch/arm64/Kconfig | 1 +
> arch/arm64/Makefile | 2 -
> arch/powerpc/Kconfig | 2 +-
> arch/sh/Kconfig | 4 +-
> arch/sparc/Kconfig | 4 +-
> arch/um/Kconfig.common | 4 -
> arch/x86/Kconfig | 15 +-
> arch/x86/um/Kconfig | 6 +-
> init/Kconfig | 40 +-
> kernel/gcov/Kconfig | 17 +-
> kernel/gcov/Makefile | 2 -
> lib/Kconfig.debug | 11 +-
> scripts/Kbuild.include | 101 +---
> scripts/Kconfig.include | 20 +
> scripts/Makefile.gcc-plugins | 91 +---
> scripts/Makefile.kcov | 10 +-
> scripts/clang-version.sh | 18 +-
> scripts/gcc-plugins/Makefile | 1 +
> scripts/gcc-x86_32-has-stack-protector.sh | 7 +-
> scripts/gcc-x86_64-has-stack-protector.sh | 5 -
> scripts/kconfig/confdata.c | 31 +-
> scripts/kconfig/kconf_id.c | 1 -
> scripts/kconfig/lkc.h | 4 -
> scripts/kconfig/lkc_proto.h | 15 +-
> scripts/kconfig/menu.c | 3 -
> scripts/kconfig/preprocess.c | 594 +++++++++++++++++++++
> scripts/kconfig/symbol.c | 109 ----
> .../kconfig/tests/preprocess/builtin_func/Kconfig | 19 +
> .../tests/preprocess/builtin_func/__init__.py | 8 +
> .../tests/preprocess/builtin_func/expected_stderr | 4 +
> .../tests/preprocess/builtin_func/expected_stdout | 1 +
> scripts/kconfig/tests/preprocess/escape/Kconfig | 49 ++
> .../kconfig/tests/preprocess/escape/__init__.py | 9 +
> .../tests/preprocess/escape/expected_stderr | 10 +
> scripts/kconfig/tests/preprocess/user_func/Kconfig | 19 +
> .../kconfig/tests/preprocess/user_func/__init__.py | 7 +
> .../tests/preprocess/user_func/expected_stderr | 5 +
> scripts/kconfig/tests/preprocess/variable/Kconfig | 39 ++
> .../kconfig/tests/preprocess/variable/__init__.py | 7 +
> .../tests/preprocess/variable/expected_stderr | 7 +
> scripts/kconfig/util.c | 22 +-
> scripts/kconfig/zconf.l | 95 +++-
> scripts/kconfig/zconf.y | 48 +-
> 49 files changed, 1259 insertions(+), 574 deletions(-)
> create mode 100644 Documentation/kbuild/kconfig-macro-language.txt
> create mode 100644 scripts/Kconfig.include
> create mode 100644 scripts/kconfig/preprocess.c
> create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/Kconfig
> create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/__init__.py
> create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/expected_stderr
> create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/expected_stdout
> create mode 100644 scripts/kconfig/tests/preprocess/escape/Kconfig
> create mode 100644 scripts/kconfig/tests/preprocess/escape/__init__.py
> create mode 100644 scripts/kconfig/tests/preprocess/escape/expected_stderr
> create mode 100644 scripts/kconfig/tests/preprocess/user_func/Kconfig
> create mode 100644 scripts/kconfig/tests/preprocess/user_func/__init__.py
> create mode 100644 scripts/kconfig/tests/preprocess/user_func/expected_stderr
> create mode 100644 scripts/kconfig/tests/preprocess/variable/Kconfig
> create mode 100644 scripts/kconfig/tests/preprocess/variable/__init__.py
> create mode 100644 scripts/kconfig/tests/preprocess/variable/expected_stderr
>
> --
> 2.7.4
>
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH 00/30] kconfig: move compiler capability tests to Kconfig
2018-04-13 5:06 [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
` (30 preceding siblings ...)
2018-04-13 5:17 ` [PATCH 00/30] kconfig: move compiler capability tests to Kconfig Masahiro Yamada
@ 2018-04-13 5:52 ` Kees Cook
2018-04-13 12:21 ` Masahiro Yamada
31 siblings, 1 reply; 53+ messages in thread
From: Kees Cook @ 2018-04-13 5:52 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, Linus Torvalds, Sam Ravnborg, Ulf Magnusson,
Nicholas Piggin, Emese Revfy, X86 ML, LKML
On Thu, Apr 12, 2018 at 10:06 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> [Major Changes in V3]
Awesome work! I don't see this pushed to your git tree? I'd like to
test it, but I'd rather "git fetch" instead of "git am" :)
-Kees
--
Kees Cook
Pixel Security
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH 00/30] kconfig: move compiler capability tests to Kconfig
2018-04-13 5:52 ` Kees Cook
@ 2018-04-13 12:21 ` Masahiro Yamada
2018-04-13 13:55 ` Masahiro Yamada
0 siblings, 1 reply; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 12:21 UTC (permalink / raw)
To: Kees Cook
Cc: linux-kbuild, Linus Torvalds, Sam Ravnborg, Ulf Magnusson,
Nicholas Piggin, Emese Revfy, X86 ML, LKML
2018-04-13 14:52 GMT+09:00 Kees Cook <keescook@chromium.org>:
> On Thu, Apr 12, 2018 at 10:06 PM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> [Major Changes in V3]
>
> Awesome work! I don't see this pushed to your git tree? I'd like to
> test it, but I'd rather "git fetch" instead of "git am" :)
>
> -Kees
>
I pushed this series to the following branch.
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
kconfig-shell-v3
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH 00/30] kconfig: move compiler capability tests to Kconfig
2018-04-13 12:21 ` Masahiro Yamada
@ 2018-04-13 13:55 ` Masahiro Yamada
0 siblings, 0 replies; 53+ messages in thread
From: Masahiro Yamada @ 2018-04-13 13:55 UTC (permalink / raw)
To: linux-kbuild
Cc: Linus Torvalds, Sam Ravnborg, Ulf Magnusson, Nicholas Piggin,
Emese Revfy, X86 ML, LKML, Kees Cook
2018-04-13 21:21 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
> 2018-04-13 14:52 GMT+09:00 Kees Cook <keescook@chromium.org>:
>> On Thu, Apr 12, 2018 at 10:06 PM, Masahiro Yamada
>> <yamada.masahiro@socionext.com> wrote:
>>> [Major Changes in V3]
>>
>> Awesome work! I don't see this pushed to your git tree? I'd like to
>> test it, but I'd rather "git fetch" instead of "git am" :)
>>
>> -Kees
>>
>
> I pushed this series to the following branch.
>
> git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> kconfig-shell-v3
>
If this approach is successful, we will move more and more
compiler option tests to the Kconfig stage in the future.
People (including me) might be worried about
how slow Kconfig will become.
First, I compared the before/after on my PC.
Without this series,
masahiro@grover:~/workspace/linux-kbuild$ time make -s defconfig
real 0m0.175s
user 0m0.128s
sys 0m0.008s
With this series,
masahiro@grover:~/workspace/linux-kbuild$ time make -s defconfig
real 0m0.729s
user 0m0.400s
sys 0m0.056s
This is noticeable difference.
Then, I looked into per-commit analysis.
Here is the result of the real time of 'time make -s defconfig'
[30/30] kbuild: test dead code/data elimination... 0m0.719s
[29/30] arm64: move GCC version check for... 0m0.711s
[28/30] gcc-plugins: allow to enable GCC_PLUGINS... 0m0.722s
[27/30] gcc-plugins: test plugin support in... 0m0.719s [+0.31]
[26/30] gcc-plugins: move GCC version check... 0m0.410s
[25/30] kcov: test compiler capability in... 0m0.392s
[24/30] gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT 0m0.400s
[23/30] kconfig: add CC_IS_CLANG and CLANG_VERSION 0m0.396s
[22/30] kconfig: add CC_IS_GCC and GCC_VERSION 0m0.392s
[21/30] stack-protector: test compiler capability... 0m0.381s [+0.04]
[20/30] kconfig: add basic helper macros to... 0m0.343s
[19/30] kconfig: show compiler version text... 0m0.345s
[18/30] kconfig: test: test text expansion... 0m0.342s
[17/30] Documentation: kconfig: document... 0m0.344s
[16/30] kconfig: add 'info' and 'warning'... 0m0.347s
[15/30] kconfig: expand lefthand side of... 0m0.340s
[14/30] kconfig: support append assignment... 0m0.342s
[13/30] kconfig: support simply expanded... 0m0.341s
[12/30] kconfig: support variable and... 0m0.344s
[11/30] kconfig: begin PARAM state only... 0m0.342s
[10/30] kconfig: replace $(UNAME_RELEASE)... 0m0.347s
[09/30] kconfig: add 'shell' built-in function 0m0.344s
[08/30] kconfig: add built-in function support 0m0.350s
[07/30] kconfig: remove sym_expand_string_value() 0m0.344s
[06/30] kconfig: remove string expansion... 0m0.349s
[05/30] kconfig: remove string expansion... 0m0.342s
[04/30] kconfig: reference environment... 0m0.342s
[03/30] kbuild: remove CONFIG_CROSS_COMPILE... 0m0.347s
[02/30] kbuild: remove kbuild cache 0m0.347s [+0.17]
[01/30] gcc-plugins: fix build condition... 0m0.171s
[00/30] Merge tag 'drm-fixes-for-v4.17-rc1'... 0m0.176s
There are three big jump points.
The first one is [02/30] (+0.17)
We are removing the build cache, so this is what we expect.
The second one is [21/30] (+0.04)
For x86, Kconfig runs scripts/gcc-x86_{32,64}-has-stack-protector.sh
The biggest one is [27/30] (+0.31)
scripts/gcc-plugins.sh is probably very costly script.
If we bump the minimum gcc version to GCC 4.8
the script will be much cleaner in the future.
I was also interested in the cost of
a single $(cc-option ...) invocation.
It is pretty easy to measure this.
For example, copy $(cc-option -fstack-protector)
1000 lines like follows.
config FOO
bool
default $(cc-option -fstack-protector)
default $(cc-option -fstack-protector)
default $(cc-option -fstack-protector)
default $(cc-option -fstack-protector)
default $(cc-option -fstack-protector)
default $(cc-option -fstack-protector)
default $(cc-option -fstack-protector)
default $(cc-option -fstack-protector)
default $(cc-option -fstack-protector)
default $(cc-option -fstack-protector)
... [ repeat 1000 line ]
On my core i7 PC, it took 7.2 msec
to run $(cc-option -fstack-protector) 1000 times.
We can make it much faster.
Currently we use
$(CC) -Werror $(1) -c -x c /dev/null
to test the compiler flag.
Ulf Magnusson suggested to use -S instead of -c
(https://patchwork.kernel.org/patch/10309297/)
With -S, the compiler stops after the compilation stage.
It took only 4.0 msec
to run $(cc-option -fstack-protector) 1000 times
If I use -E (only pre-process stage), it becomes even faster.
It took only 2.6 msec.
As for $(cc-option ...), probably this will not be a problem.
For some feature, we need special shell-scripts,
some of which can be more costly.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 53+ messages in thread