[PATCH v5 00/31] kconfig: move compiler capability tests to Kconfig

[PATCH v5 00/31] kconfig: move compiler capability tests to Kconfig

[Masahiro Yamada]

[Introduction]

The motivation of this work is to move the compiler option tests to
Kconfig from Makefile.  A number of kernel features require the
compiler support.  Enabling such features blindly in Kconfig ends up
with a lot of nasty build-time testing in Makefiles.  If a chosen
feature turns out unsupported by the compiler, what the build system
can do is either to disable it (silently!) or to forcibly break the
build, despite Kconfig has let the user to enable it.  By moving the
compiler capability tests to Kconfig, Kconfig entries will be visible
only when they are available.

[Major Changes in V5]

V5 removed the complexity unnecessary for Kconfig.

 - In order to make the implementation simpler, I drop the lazy
   argument expansion.
   In V5, parameters are expanded before passed to a function.
   (the same behavior as v3)

 - Replace 'warning', 'error' functions with 'warning-if',
   'error-if' functions.  The new functions are effective
   only when the condition part is 'y'.

 - In V4, '$' must be followed by '$' or '('.
   '$$' meant an escaped '$'.

   V4 caused a build error for ARC architecture due to the
   following line:
   https://github.com/torvalds/linux/blob/v4.17-rc4/arch/arc/Kconfig#L252

   Of course, we could rephrase "D$" to "D cache", but
   I treaked the grammar a little bit.

   Kconfig decided to not support single-letter variable as in "$X".
   Nor curly braces as in "${CC}" are not supported.
   This means variable/function references in Kconfig always start
   with "$(".
   We can use '$' not followed by '(' as-is.
   We do not need the "$$" escape sequence.
   If necessary, we can use a trick to print '$(' verbatim.  See below.

 - Fix a wrong behavior of simply expanded variables.
   V4 expanded simply expanded variables when used, but it should not.

   For example,

   dollar := $
   X := 1
   Y := $(dollar)(X)
   $(info,$(Y))

   V5 prints '$(X)', not '1'.

 - Remove a space after a comma is $( ) context.
   Actually, spaces do not matter in most cases.

   You can write either

      $(cc-option,-fstack-protector)

   or

      $(cc-option, -fstack-protector)

   In fact, both style co-exist in our Makefiles too.
   However, there are some cases where they do matter.

   I remove spaces after commas for consistency.
   If you are unsure, no-space-after-comma is always safe.

[Major Changes in V4]

 - In V4, I slightly change the syntax of a function call.
   I chose grammatical consistency and simplicity.
   Anyway, Kconfig is deviating from Make behavior already.

   In v3, a function call looked like this:

      $(func-name arg1,arg2,arg3)

   In v4, a function is invoked like follows:

      $(func-name,arg1,arg2,arg3)

   The difference is that the function name and the first argument
   are separated by a comma.

 - V3 supported single-letter variable like $X.
   V4 dropped it because we do not need multiple ways to do the
   same thing.
   You must always enclose a variable name like $(X).

 - Support lazy argument expansion.  This is necessary for implementing
   'if', 'and', 'or' functions as in Make.

 - Add more built-in functions:
   'if', 'error', 'filename', 'lineno'

 - Error out if a recursive variable references itself eventually.
   For example,  X = $(X)
   ends up with a circular expansion.  It must be terminated
   since the expansion would continue eternally.

 - Update Documentation and unit-tests, accordingly.

[Major Changes in V3]

This version looks more like Make.

  - Use = operator instead of 'macro' keyword
    to define a user-defined function.

  - 'Recursively expanded variable' is implemented as a side-effect.
     A variable is a function with zero argument.

  - Support simply expanded variable which is defined by := operator

  - Support += operator.
    Probably, this feature will be useful to accumulate compiler flags.
    At least, Clang needs some prerequisite flags such as triplet
    to test other compiler flags.

  - Support $(info ...) and $(warning ...) built-in functions,
    which were useful while I was debugging this.

  - Add documentation

  - Add unit tests

  - Collect helpers to scripts/Kconfig.include

[Old Versions]

V4:  https://lkml.org/lkml/2018/5/17/97
V3:  https://lkml.org/lkml/2018/4/13/37
V2:  https://lkml.org/lkml/2018/2/16/610
V1:  https://lkml.org/lkml/2018/2/16/610
RFC: https://lkml.org/lkml/2018/2/8/429


Masahiro Yamada (31):
  kbuild: remove kbuild cache
  kbuild: remove CONFIG_CROSS_COMPILE support
  kconfig: reference environment variables directly and remove 'option
    env='
  kconfig: remove string expansion in file_lookup()
  kconfig: remove string expansion for mainmenu after yyparse()
  kconfig: remove sym_expand_string_value()
  kconfig: make default prompt of mainmenu less specific
  kconfig: add built-in function support
  kconfig: add 'shell' built-in function
  kconfig: replace $(UNAME_RELEASE) with function call
  kconfig: begin PARAM state only when seeing a command keyword
  kconfig: support user-defined function and recursively expanded
    variable
  kconfig: support simply expanded variable
  kconfig: support append assignment operator
  kconfig: expand lefthand side of assignment statement
  kconfig: add 'info', 'warning-if', and 'error-if' built-in functions
  kconfig: add 'filename' and 'lineno' built-in variables
  kconfig: error out if a recursive variable references itself
  Documentation: kconfig: document a new Kconfig macro language
  kconfig: test: add Kconfig macro language tests
  kconfig: show compiler version text in the top comment
  kconfig: add basic helper macros to scripts/Kconfig.include
  stack-protector: test compiler capability in Kconfig and drop AUTO
    mode
  kconfig: add CC_IS_GCC and GCC_VERSION
  kconfig: add CC_IS_CLANG and CLANG_VERSION
  gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT
  kcov: test compiler capability in Kconfig and correct dependency
  gcc-plugins: move GCC version check for PowerPC to Kconfig
  gcc-plugins: test plugin support in Kconfig and clean up Makefile
  gcc-plugins: allow to enable GCC_PLUGINS for COMPILE_TEST
  Documentation: kconfig: add recommended way to describe compiler
    support

 Documentation/kbuild/kconfig-language.txt          |  26 +-
 Documentation/kbuild/kconfig-macro-language.txt    | 242 +++++++++
 Kconfig                                            |  10 +-
 MAINTAINERS                                        |   3 +-
 Makefile                                           | 105 +---
 arch/Kconfig                                       |  49 +-
 arch/powerpc/Kconfig                               |   2 +-
 arch/sh/Kconfig                                    |   4 +-
 arch/sparc/Kconfig                                 |   4 +-
 arch/um/Kconfig.common                             |   4 -
 arch/x86/Kconfig                                   |  15 +-
 arch/x86/um/Kconfig                                |  10 +-
 init/Kconfig                                       |  40 +-
 kernel/gcov/Kconfig                                |  17 +-
 kernel/gcov/Makefile                               |   2 -
 lib/Kconfig.debug                                  |  11 +-
 scripts/Kbuild.include                             | 101 +---
 scripts/Kconfig.include                            |  30 ++
 scripts/Makefile.gcc-plugins                       |  91 +---
 scripts/Makefile.kcov                              |  10 +-
 scripts/clang-version.sh                           |  18 +-
 scripts/gcc-plugins/Makefile                       |   1 +
 scripts/gcc-x86_32-has-stack-protector.sh          |   7 +-
 scripts/gcc-x86_64-has-stack-protector.sh          |   5 -
 scripts/kconfig/confdata.c                         |  33 +-
 scripts/kconfig/kconf_id.c                         |   1 -
 scripts/kconfig/lkc.h                              |   5 +-
 scripts/kconfig/lkc_proto.h                        |  15 +-
 scripts/kconfig/menu.c                             |   3 -
 scripts/kconfig/preprocess.c                       | 572 +++++++++++++++++++++
 scripts/kconfig/symbol.c                           | 109 ----
 .../tests/no_write_if_dep_unmet/expected_config    |   2 +-
 .../kconfig/tests/preprocess/builtin_func/Kconfig  |  27 +
 .../tests/preprocess/builtin_func/__init__.py      |   9 +
 .../tests/preprocess/builtin_func/expected_stderr  |   5 +
 .../tests/preprocess/builtin_func/expected_stdout  |   1 +
 .../tests/preprocess/circular_expansion/Kconfig    |   5 +
 .../preprocess/circular_expansion/__init__.py      |  11 +
 .../preprocess/circular_expansion/expected_stderr  |   1 +
 scripts/kconfig/tests/preprocess/escape/Kconfig    |  44 ++
 .../kconfig/tests/preprocess/escape/__init__.py    |   8 +
 .../tests/preprocess/escape/expected_stderr        |  10 +
 scripts/kconfig/tests/preprocess/variable/Kconfig  |  53 ++
 .../kconfig/tests/preprocess/variable/__init__.py  |   8 +
 .../tests/preprocess/variable/expected_stderr      |   9 +
 scripts/kconfig/util.c                             |  33 +-
 scripts/kconfig/zconf.l                            |  95 +++-
 scripts/kconfig/zconf.y                            |  46 +-
 48 files changed, 1345 insertions(+), 567 deletions(-)
 create mode 100644 Documentation/kbuild/kconfig-macro-language.txt
 create mode 100644 scripts/Kconfig.include
 create mode 100644 scripts/kconfig/preprocess.c
 create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/Kconfig
 create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/__init__.py
 create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/expected_stderr
 create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/expected_stdout
 create mode 100644 scripts/kconfig/tests/preprocess/circular_expansion/Kconfig
 create mode 100644 scripts/kconfig/tests/preprocess/circular_expansion/__init__.py
 create mode 100644 scripts/kconfig/tests/preprocess/circular_expansion/expected_stderr
 create mode 100644 scripts/kconfig/tests/preprocess/escape/Kconfig
 create mode 100644 scripts/kconfig/tests/preprocess/escape/__init__.py
 create mode 100644 scripts/kconfig/tests/preprocess/escape/expected_stderr
 create mode 100644 scripts/kconfig/tests/preprocess/variable/Kconfig
 create mode 100644 scripts/kconfig/tests/preprocess/variable/__init__.py
 create mode 100644 scripts/kconfig/tests/preprocess/variable/expected_stderr

-- 
2.7.4

[PATCH v5 01/31] kbuild: remove kbuild cache

[Masahiro Yamada]

The kbuild cache was introduced to remember the result of shell
commands, some of which are expensive to compute, such as
$(call cc-option,...).

However, this turned out not so clever as I had first expected.
Actually, it is problematic.  For example, "$(CC) -print-file-name"
is cached.  If the compiler is updated, the stale search path causes
build error, which is difficult to figure out.  Another problem
scenario is cache files could be touched while install targets are
running under the root permission.  We can patch them if desired,
but the build infrastructure is getting uglier and uglier.

Now, we are going to move compiler flag tests to the configuration
phase.  If this is completed, the result of compiler tests will be
naturally cached in the .config file.  We will not have performance
issues of incremental building since this testing only happens at
Kconfig time.

To start this work with a cleaner code base, remove the kbuild
cache first.

Revert the following commits:
Commit 9a234a2e3843 ("kbuild: create directory for make cache only when necessary")
Commit e17c400ae194 ("kbuild: shrink .cache.mk when it exceeds 1000 lines")
Commit 4e56207130ed ("kbuild: Cache a few more calls to the compiler")
Commit 3298b690b21c ("kbuild: Add a cache for generated variables")

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---

Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None

 Makefile               |   5 +--
 scripts/Kbuild.include | 101 +++++++------------------------------------------
 2 files changed, 16 insertions(+), 90 deletions(-)

diff --git a/Makefile b/Makefile
index 56ba070..02202d3 100644
--- a/Makefile
+++ b/Makefile
@@ -504,7 +504,7 @@ KBUILD_CFLAGS	+= $(call cc-option,-fno-PIE)
 KBUILD_AFLAGS	+= $(call cc-option,-fno-PIE)
 
 # check for 'asm goto'
-ifeq ($(call shell-cached,$(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC) $(KBUILD_CFLAGS)), y)
+ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC) $(KBUILD_CFLAGS)), y)
   CC_HAVE_ASM_GOTO := 1
   KBUILD_CFLAGS += -DCC_HAVE_ASM_GOTO
   KBUILD_AFLAGS += -DCC_HAVE_ASM_GOTO
@@ -807,7 +807,7 @@ KBUILD_CFLAGS	+= $(call cc-option,-fdata-sections,)
 endif
 
 # arch Makefile may override CC so keep this after arch Makefile is included
-NOSTDINC_FLAGS += -nostdinc -isystem $(call shell-cached,$(CC) -print-file-name=include)
+NOSTDINC_FLAGS += -nostdinc -isystem $(shell $(CC) -print-file-name=include)
 CHECKFLAGS     += $(NOSTDINC_FLAGS)
 
 # warn about C99 declaration after statement
@@ -1625,7 +1625,6 @@ clean: $(clean-dirs)
 		-o -name '*.asn1.[ch]' \
 		-o -name '*.symtypes' -o -name 'modules.order' \
 		-o -name modules.builtin -o -name '.tmp_*.o.*' \
-		-o -name .cache.mk \
 		-o -name '*.c.[012]*.*' \
 		-o -name '*.ll' \
 		-o -name '*.gcno' \) -type f -print | xargs rm -f
diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
index 50cee53..925a851 100644
--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -8,8 +8,6 @@ squote  := '
 empty   :=
 space   := $(empty) $(empty)
 space_escape := _-_SPACE_-_
-right_paren := )
-left_paren := (
 pound := \#
 
 ###
@@ -83,71 +81,6 @@ cc-cross-prefix =  \
 			echo $(c);                                    \
 		fi)))
 
-# Tools for caching Makefile variables that are "expensive" to compute.
-#
-# Here we want to help deal with variables that take a long time to compute
-# by making it easy to store these variables in a cache.
-#
-# The canonical example here is testing for compiler flags.  On a simple system
-# each call to the compiler takes 10 ms, but on a system with a compiler that's
-# called through various wrappers it can take upwards of 100 ms.  If we have
-# 100 calls to the compiler this can take 1 second (on a simple system) or 10
-# seconds (on a complicated system).
-#
-# The "cache" will be in Makefile syntax and can be directly included.
-# Any time we try to reference a variable that's not in the cache we'll
-# calculate it and store it in the cache for next time.
-
-# Include values from last time
-make-cache := $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/,$(if $(obj),$(obj)/)).cache.mk
-$(make-cache): ;
--include $(make-cache)
-
-cached-data := $(filter __cached_%, $(.VARIABLES))
-
-# If cache exceeds 1000 lines, shrink it down to 500.
-ifneq ($(word 1000,$(cached-data)),)
-$(shell tail -n 500 $(make-cache) > $(make-cache).tmp; \
-	mv $(make-cache).tmp $(make-cache))
-endif
-
-create-cache-dir := $(if $(KBUILD_SRC),$(if $(cache-data),,1))
-
-# Usage: $(call __sanitize-opt,Hello=Hola$(comma)Goodbye Adios)
-#
-# Convert all '$', ')', '(', '\', '=', ' ', ',', ':' to '_'
-__sanitize-opt = $(subst $$,_,$(subst $(right_paren),_,$(subst $(left_paren),_,$(subst \,_,$(subst =,_,$(subst $(space),_,$(subst $(comma),_,$(subst :,_,$(1)))))))))
-
-# Usage:   $(call shell-cached,shell_command)
-# Example: $(call shell-cached,md5sum /usr/bin/gcc)
-#
-# If we've already seen a call to this exact shell command (even in a
-# previous invocation of make!) we'll return the value.  If not, we'll
-# compute it and store the result for future runs.
-#
-# This is a bit of voodoo, but basic explanation is that if the variable
-# was undefined then we'll evaluate the shell command and store the result
-# into the variable.  We'll then store that value in the cache and finally
-# output the value.
-#
-# NOTE: The $$(2) here isn't actually a parameter to __run-and-store.  We
-# happen to know that the caller will have their shell command in $(2) so the
-# result of "call"ing this will produce a reference to that $(2).  The reason
-# for this strangeness is to avoid an extra level of eval (and escaping) of
-# $(2).
-define __run-and-store
-ifeq ($(origin $(1)),undefined)
-  $$(eval $(1) := $$(shell $$(2)))
-ifeq ($(create-cache-dir),1)
-  $$(shell mkdir -p $(dir $(make-cache)))
-  $$(eval create-cache-dir :=)
-endif
-  $$(shell echo '$(1) := $$($(1))' >> $(make-cache))
-endif
-endef
-__shell-cached = $(eval $(call __run-and-store,$(1)))$($(1))
-shell-cached = $(call __shell-cached,__cached_$(call __sanitize-opt,$(1)),$(1))
-
 # output directory for tests below
 TMPOUT := $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/)
 
@@ -155,36 +88,30 @@ TMPOUT := $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/)
 # Usage: option = $(call try-run, $(CC)...-o "$$TMP",option-ok,otherwise)
 # Exit code chooses option. "$$TMP" serves as a temporary file and is
 # automatically cleaned up.
-__try-run = set -e;			\
+try-run = $(shell set -e;		\
 	TMP="$(TMPOUT).$$$$.tmp";	\
 	TMPO="$(TMPOUT).$$$$.o";	\
 	if ($(1)) >/dev/null 2>&1;	\
 	then echo "$(2)";		\
 	else echo "$(3)";		\
 	fi;				\
-	rm -f "$$TMP" "$$TMPO"
-
-try-run = $(shell $(__try-run))
-
-# try-run-cached
-# This works like try-run, but the result is cached.
-try-run-cached = $(call shell-cached,$(__try-run))
+	rm -f "$$TMP" "$$TMPO")
 
 # as-option
 # Usage: cflags-y += $(call as-option,-Wa$(comma)-isa=foo,)
 
-as-option = $(call try-run-cached,\
+as-option = $(call try-run,\
 	$(CC) $(KBUILD_CFLAGS) $(1) -c -x assembler /dev/null -o "$$TMP",$(1),$(2))
 
 # as-instr
 # Usage: cflags-y += $(call as-instr,instr,option1,option2)
 
-as-instr = $(call try-run-cached,\
+as-instr = $(call try-run,\
 	printf "%b\n" "$(1)" | $(CC) $(KBUILD_AFLAGS) -c -x assembler -o "$$TMP" -,$(2),$(3))
 
 # __cc-option
 # Usage: MY_CFLAGS += $(call __cc-option,$(CC),$(MY_CFLAGS),-march=winchip-c6,-march=i586)
-__cc-option = $(call try-run-cached,\
+__cc-option = $(call try-run,\
 	$(1) -Werror $(2) $(3) -c -x c /dev/null -o "$$TMP",$(3),$(4))
 
 # Do not attempt to build with gcc plugins during cc-option tests.
@@ -204,23 +131,23 @@ hostcc-option = $(call __cc-option, $(HOSTCC),\
 
 # cc-option-yn
 # Usage: flag := $(call cc-option-yn,-march=winchip-c6)
-cc-option-yn = $(call try-run-cached,\
+cc-option-yn = $(call try-run,\
 	$(CC) -Werror $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) $(1) -c -x c /dev/null -o "$$TMP",y,n)
 
 # cc-disable-warning
 # Usage: cflags-y += $(call cc-disable-warning,unused-but-set-variable)
-cc-disable-warning = $(call try-run-cached,\
+cc-disable-warning = $(call try-run,\
 	$(CC) -Werror $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1)))
 
 # cc-name
 # Expands to either gcc or clang
-cc-name = $(call shell-cached,$(CC) -v 2>&1 | grep -q "clang version" && echo clang || echo gcc)
+cc-name = $(shell $(CC) -v 2>&1 | grep -q "clang version" && echo clang || echo gcc)
 
 # cc-version
-cc-version = $(call shell-cached,$(CONFIG_SHELL) $(srctree)/scripts/gcc-version.sh $(CC))
+cc-version = $(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-version.sh $(CC))
 
 # cc-fullversion
-cc-fullversion = $(call shell-cached,$(CONFIG_SHELL) \
+cc-fullversion = $(shell $(CONFIG_SHELL) \
 	$(srctree)/scripts/gcc-version.sh -p $(CC))
 
 # cc-ifversion
@@ -233,21 +160,21 @@ cc-if-fullversion = $(shell [ $(cc-fullversion) $(1) $(2) ] && echo $(3) || echo
 
 # cc-ldoption
 # Usage: ldflags += $(call cc-ldoption, -Wl$(comma)--hash-style=both)
-cc-ldoption = $(call try-run-cached,\
+cc-ldoption = $(call try-run,\
 	$(CC) $(1) $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2))
 
 # ld-option
 # Usage: LDFLAGS += $(call ld-option, -X)
-ld-option = $(call try-run-cached, $(LD) $(LDFLAGS) $(1) -v,$(1),$(2))
+ld-option = $(call try-run, $(LD) $(LDFLAGS) $(1) -v,$(1),$(2))
 
 # ar-option
 # Usage: KBUILD_ARFLAGS := $(call ar-option,D)
 # Important: no spaces around options
-ar-option = $(call try-run-cached, $(AR) rc$(1) "$$TMP",$(1),$(2))
+ar-option = $(call try-run, $(AR) rc$(1) "$$TMP",$(1),$(2))
 
 # ld-version
 # Note this is mainly for HJ Lu's 3 number binutil versions
-ld-version = $(call shell-cached,$(LD) --version | $(srctree)/scripts/ld-version.sh)
+ld-version = $(shell $(LD) --version | $(srctree)/scripts/ld-version.sh)
 
 # ld-ifversion
 # Usage:  $(call ld-ifversion, -ge, 22252, y)
-- 
2.7.4

[PATCH v5 02/31] kbuild: remove CONFIG_CROSS_COMPILE support

[Masahiro Yamada]

Kbuild provides a couple of ways to specify CROSS_COMPILE:

[1] Command line
[2] Environment
[3] arch/*/Makefile (only some architectures)
[4] CONFIG_CROSS_COMPILE

[4] is problematic for the compiler capability tests in Kconfig.
CONFIG_CROSS_COMPILE allows users to change the compiler prefix from
'make menuconfig', etc.  It means, the compiler options would have
to be all re-calculated everytime CONFIG_CROSS_COMPILE is changed.

To avoid complexity and performance issues, I'd like to evaluate
the shell commands statically, i.e. only parsing Kconfig files.

I guess the majority is [1] or [2].  Currently, there are only
5 defconfig files that specify CONFIG_CROSS_COMPILE.
  arch/arm/configs/lpc18xx_defconfig
  arch/hexagon/configs/comet_defconfig
  arch/nds32/configs/defconfig
  arch/openrisc/configs/or1ksim_defconfig
  arch/openrisc/configs/simple_smp_defconfig

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---

Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None

 Makefile     | 3 ---
 init/Kconfig | 9 ---------
 2 files changed, 12 deletions(-)

diff --git a/Makefile b/Makefile
index 02202d3..58afa07 100644
--- a/Makefile
+++ b/Makefile
@@ -316,12 +316,9 @@ SUBARCH := $(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ \
 # CROSS_COMPILE can be set on the command line
 # make CROSS_COMPILE=ia64-linux-
 # Alternatively CROSS_COMPILE can be set in the environment.
-# A third alternative is to store a setting in .config so that plain
-# "make" in the configured kernel build directory always uses that.
 # Default value for CROSS_COMPILE is not to prefix executables
 # Note: Some architectures assign CROSS_COMPILE in their arch/*/Makefile
 ARCH		?= $(SUBARCH)
-CROSS_COMPILE	?= $(CONFIG_CROSS_COMPILE:"%"=%)
 
 # Architecture as present in compile.h
 UTS_MACHINE 	:= $(ARCH)
diff --git a/init/Kconfig b/init/Kconfig
index 18b151f..15aae32 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -54,15 +54,6 @@ config INIT_ENV_ARG_LIMIT
 	  Maximum of each of the number of arguments and environment
 	  variables passed to init from the kernel command line.
 
-
-config CROSS_COMPILE
-	string "Cross-compiler tool prefix"
-	help
-	  Same as running 'make CROSS_COMPILE=prefix-' but stored for
-	  default make runs in this kernel build directory.  You don't
-	  need to set this unless you want the configured kernel build
-	  directory to select the cross-compiler automatically.
-
 config COMPILE_TEST
 	bool "Compile also drivers which will not load"
 	depends on !UML
-- 
2.7.4

[PATCH v5 03/31] kconfig: reference environment variables directly and remove 'option env='

[Masahiro Yamada]

To get access to environment variables, Kconfig needs to define a
symbol using "option env=" syntax.  It is tedious to add a symbol entry
for each environment variable given that we need to define much more
such as 'CC', 'AS', 'srctree' etc. to evaluate the compiler capability
in Kconfig.

Adding '$' for symbol references is grammatically inconsistent.
Looking at the code, the symbols prefixed with 'S' are expanded by:
 - conf_expand_value()
   This is used to expand 'arch/$ARCH/defconfig' and 'defconfig_list'
 - sym_expand_string_value()
   This is used to expand strings in 'source' and 'mainmenu'

All of them are fixed values independent of user configuration.  So,
they can be changed into the direct expansion instead of symbols.

This change makes the code much cleaner.  The bounce symbols 'SRCARCH',
'ARCH', 'SUBARCH', 'KERNELVERSION' are gone.

sym_init() hard-coding 'UNAME_RELEASE' is also gone.  'UNAME_RELEASE'
should be replaced with an environment variable.

ARCH_DEFCONFIG is a normal symbol, so it should be simply referenced
without '$' prefix.

The new syntax is addicted by Make.  The variable reference needs
parentheses, like $(FOO), but you can omit them for single-letter
variables, like $F.  Yet, in Makefiles, people tend to use the
parenthetical form for consistency / clarification.

At this moment, only the environment variable is supported, but I will
extend the concept of 'variable' later on.

The variables are expanded in the lexer so we can simplify the token
handling on the parser side.

For example, the following code works.

[Example code]

  config MY_TOOLCHAIN_LIST
          string
          default "My tools: CC=$(CC), AS=$(AS), CPP=$(CPP)"

[Result]

  $ make -s alldefconfig && tail -n 1 .config
  CONFIG_MY_TOOLCHAIN_LIST="My tools: CC=gcc, AS=as, CPP=gcc -E"

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---

Changes in v5:
  - More comments like "advance pointer to ..."
  - Factor out the duplicated code into __expand_string()
  - Move the empty name check to env_expand().
  - Remove escape sequence of '$'

Changes in v4:
  - Enclose ARCH in conf_defname
  - Drop single-letter support

Changes in v3:
  - Reimplement
  - Variable reference need parentheses except single-letter variable

Changes in v2:
  - Move the string expansion to the lexer phase.
  - Split environment helpers to env.c

 Documentation/kbuild/kconfig-language.txt |   8 -
 Kconfig                                   |   8 +-
 Makefile                                  |   3 +-
 arch/sh/Kconfig                           |   4 +-
 arch/sparc/Kconfig                        |   4 +-
 arch/um/Kconfig.common                    |   4 -
 arch/x86/Kconfig                          |   4 +-
 arch/x86/um/Kconfig                       |   6 +-
 init/Kconfig                              |  16 +-
 scripts/kconfig/confdata.c                |  33 +----
 scripts/kconfig/kconf_id.c                |   1 -
 scripts/kconfig/lkc.h                     |   5 +-
 scripts/kconfig/lkc_proto.h               |   6 +
 scripts/kconfig/menu.c                    |   3 -
 scripts/kconfig/preprocess.c              | 238 ++++++++++++++++++++++++++++++
 scripts/kconfig/symbol.c                  |  56 -------
 scripts/kconfig/util.c                    |  29 ++--
 scripts/kconfig/zconf.l                   |  67 ++++++++-
 scripts/kconfig/zconf.y                   |   2 +-
 19 files changed, 343 insertions(+), 154 deletions(-)
 create mode 100644 scripts/kconfig/preprocess.c

diff --git a/Documentation/kbuild/kconfig-language.txt b/Documentation/kbuild/kconfig-language.txt
index f5b9493..0e966e8 100644
--- a/Documentation/kbuild/kconfig-language.txt
+++ b/Documentation/kbuild/kconfig-language.txt
@@ -198,14 +198,6 @@ applicable everywhere (see syntax).
     enables the third modular state for all config symbols.
     At most one symbol may have the "modules" option set.
 
-  - "env"=<value>
-    This imports the environment variable into Kconfig. It behaves like
-    a default, except that the value comes from the environment, this
-    also means that the behaviour when mixing it with normal defaults is
-    undefined at this point. The symbol is currently not exported back
-    to the build environment (if this is desired, it can be done via
-    another symbol).
-
   - "allnoconfig_y"
     This declares the symbol as one that should have the value y when
     using "allnoconfig". Used for symbols that hide other symbols.
diff --git a/Kconfig b/Kconfig
index 8c4c1cb..4af1b42 100644
--- a/Kconfig
+++ b/Kconfig
@@ -3,10 +3,6 @@
 # For a description of the syntax of this configuration file,
 # see Documentation/kbuild/kconfig-language.txt.
 #
-mainmenu "Linux/$ARCH $KERNELVERSION Kernel Configuration"
+mainmenu "Linux/$(ARCH) $(KERNELVERSION) Kernel Configuration"
 
-config SRCARCH
-	string
-	option env="SRCARCH"
-
-source "arch/$SRCARCH/Kconfig"
+source "arch/$(SRCARCH)/Kconfig"
diff --git a/Makefile b/Makefile
index 58afa07..59b8654 100644
--- a/Makefile
+++ b/Makefile
@@ -284,7 +284,8 @@ include scripts/Kbuild.include
 # Read KERNELRELEASE from include/config/kernel.release (if it exists)
 KERNELRELEASE = $(shell cat include/config/kernel.release 2> /dev/null)
 KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(SUBLEVEL)))$(EXTRAVERSION)
-export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION
+UNAME_RELEASE := $(shell uname --release)
+export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION UNAME_RELEASE
 
 # SUBARCH tells the usermode build what the underlying arch is.  That is set
 # first, and if a usermode build is happening, the "ARCH=um" on the command
diff --git a/arch/sh/Kconfig b/arch/sh/Kconfig
index 1851eae..c8400e3 100644
--- a/arch/sh/Kconfig
+++ b/arch/sh/Kconfig
@@ -58,7 +58,7 @@ config SUPERH
 	  <http://www.linux-sh.org/>.
 
 config SUPERH32
-	def_bool ARCH = "sh"
+	def_bool "$(ARCH)" = "sh"
 	select HAVE_KPROBES
 	select HAVE_KRETPROBES
 	select HAVE_IOREMAP_PROT if MMU && !X2TLB
@@ -77,7 +77,7 @@ config SUPERH32
 	select HAVE_CC_STACKPROTECTOR
 
 config SUPERH64
-	def_bool ARCH = "sh64"
+	def_bool "$(ARCH)" = "sh64"
 	select HAVE_EXIT_THREAD
 	select KALLSYMS
 
diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index 8767e45..df7410c 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -1,6 +1,6 @@
 config 64BIT
-	bool "64-bit kernel" if ARCH = "sparc"
-	default ARCH = "sparc64"
+	bool "64-bit kernel" if "$(ARCH)" = "sparc"
+	default "$(ARCH)" = "sparc64"
 	help
 	  SPARC is a family of RISC microprocessors designed and marketed by
 	  Sun Microsystems, incorporated.  They are very widely found in Sun
diff --git a/arch/um/Kconfig.common b/arch/um/Kconfig.common
index c68add8..07f84c8 100644
--- a/arch/um/Kconfig.common
+++ b/arch/um/Kconfig.common
@@ -54,10 +54,6 @@ config HZ
 	int
 	default 100
 
-config SUBARCH
-	string
-	option env="SUBARCH"
-
 config NR_CPUS
 	int
 	range 1 1
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index c07f492..2236505 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1,8 +1,8 @@
 # SPDX-License-Identifier: GPL-2.0
 # Select 32 or 64 bit
 config 64BIT
-	bool "64-bit kernel" if ARCH = "x86"
-	default ARCH != "i386"
+	bool "64-bit kernel" if "$(ARCH)" = "x86"
+	default "$(ARCH)" != "i386"
 	---help---
 	  Say yes to build a 64-bit kernel - formerly known as x86_64
 	  Say no to build a 32-bit kernel - formerly known as i386
diff --git a/arch/x86/um/Kconfig b/arch/x86/um/Kconfig
index 13ed827..6a15c4d 100644
--- a/arch/x86/um/Kconfig
+++ b/arch/x86/um/Kconfig
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0
-mainmenu "User Mode Linux/$SUBARCH $KERNELVERSION Kernel Configuration"
+mainmenu "User Mode Linux/$(SUBARCH) $(KERNELVERSION) Kernel Configuration"
 
 source "arch/um/Kconfig.common"
 
@@ -16,8 +16,8 @@ config UML_X86
 	select GENERIC_FIND_FIRST_BIT
 
 config 64BIT
-	bool "64-bit kernel" if SUBARCH = "x86"
-	default SUBARCH != "i386"
+	bool "64-bit kernel" if "$(SUBARCH)" = "x86"
+	default "$(SUBARCH)" != "i386"
 
 config X86_32
 	def_bool !64BIT
diff --git a/init/Kconfig b/init/Kconfig
index 15aae32..1217fc6 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1,20 +1,12 @@
-config ARCH
-	string
-	option env="ARCH"
-
-config KERNELVERSION
-	string
-	option env="KERNELVERSION"
-
 config DEFCONFIG_LIST
 	string
 	depends on !UML
 	option defconfig_list
-	default "/lib/modules/$UNAME_RELEASE/.config"
+	default "/lib/modules/$(UNAME_RELEASE)/.config"
 	default "/etc/kernel-config"
-	default "/boot/config-$UNAME_RELEASE"
-	default "$ARCH_DEFCONFIG"
-	default "arch/$ARCH/defconfig"
+	default "/boot/config-$(UNAME_RELEASE)"
+	default ARCH_DEFCONFIG
+	default "arch/$(ARCH)/defconfig"
 
 config CONSTRUCTORS
 	bool
diff --git a/scripts/kconfig/confdata.c b/scripts/kconfig/confdata.c
index df26c7b..f72587c 100644
--- a/scripts/kconfig/confdata.c
+++ b/scripts/kconfig/confdata.c
@@ -30,7 +30,7 @@ static void conf_message(const char *fmt, ...)
 static const char *conf_filename;
 static int conf_lineno, conf_warnings;
 
-const char conf_defname[] = "arch/$ARCH/defconfig";
+const char conf_defname[] = "arch/$(ARCH)/defconfig";
 
 static void conf_warning(const char *fmt, ...)
 {
@@ -81,39 +81,13 @@ const char *conf_get_autoconfig_name(void)
 	return name ? name : "include/config/auto.conf";
 }
 
-static char *conf_expand_value(const char *in)
-{
-	struct symbol *sym;
-	const char *src;
-	static char res_value[SYMBOL_MAXLENGTH];
-	char *dst, name[SYMBOL_MAXLENGTH];
-
-	res_value[0] = 0;
-	dst = name;
-	while ((src = strchr(in, '$'))) {
-		strncat(res_value, in, src - in);
-		src++;
-		dst = name;
-		while (isalnum(*src) || *src == '_')
-			*dst++ = *src++;
-		*dst = 0;
-		sym = sym_lookup(name, 0);
-		sym_calc_value(sym);
-		strcat(res_value, sym_get_string_value(sym));
-		in = src;
-	}
-	strcat(res_value, in);
-
-	return res_value;
-}
-
 char *conf_get_default_confname(void)
 {
 	struct stat buf;
 	static char fullname[PATH_MAX+1];
 	char *env, *name;
 
-	name = conf_expand_value(conf_defname);
+	name = expand_string(conf_defname);
 	env = getenv(SRCTREE);
 	if (env) {
 		sprintf(fullname, "%s/%s", env, name);
@@ -274,7 +248,8 @@ int conf_read_simple(const char *name, int def)
 			if (expr_calc_value(prop->visible.expr) == no ||
 			    prop->expr->type != E_SYMBOL)
 				continue;
-			name = conf_expand_value(prop->expr->left.sym->name);
+			sym_calc_value(prop->expr->left.sym);
+			name = sym_get_string_value(prop->expr->left.sym);
 			in = zconf_fopen(name);
 			if (in) {
 				conf_message(_("using defaults found in %s"),
diff --git a/scripts/kconfig/kconf_id.c b/scripts/kconfig/kconf_id.c
index 3ea9c5f..b3e0ea0 100644
--- a/scripts/kconfig/kconf_id.c
+++ b/scripts/kconfig/kconf_id.c
@@ -32,7 +32,6 @@ static struct kconf_id kconf_id_array[] = {
 	{ "on",			T_ON,			TF_PARAM },
 	{ "modules",		T_OPT_MODULES,		TF_OPTION },
 	{ "defconfig_list",	T_OPT_DEFCONFIG_LIST,	TF_OPTION },
-	{ "env",		T_OPT_ENV,		TF_OPTION },
 	{ "allnoconfig_y",	T_OPT_ALLNOCONFIG_Y,	TF_OPTION },
 };
 
diff --git a/scripts/kconfig/lkc.h b/scripts/kconfig/lkc.h
index f4394af..658efb0 100644
--- a/scripts/kconfig/lkc.h
+++ b/scripts/kconfig/lkc.h
@@ -58,7 +58,6 @@ enum conf_def_mode {
 
 #define T_OPT_MODULES		1
 #define T_OPT_DEFCONFIG_LIST	2
-#define T_OPT_ENV		3
 #define T_OPT_ALLNOCONFIG_Y	4
 
 struct kconf_id {
@@ -117,6 +116,7 @@ void *xmalloc(size_t size);
 void *xcalloc(size_t nmemb, size_t size);
 void *xrealloc(void *p, size_t size);
 char *xstrdup(const char *s);
+char *xstrndup(const char *s, size_t n);
 
 struct gstr {
 	size_t len;
@@ -134,9 +134,6 @@ void str_printf(struct gstr *gs, const char *fmt, ...);
 const char *str_get(struct gstr *gs);
 
 /* symbol.c */
-extern struct expr *sym_env_list;
-
-void sym_init(void);
 void sym_clear_all_valid(void);
 struct symbol *sym_choice_default(struct symbol *sym);
 const char *sym_get_string_default(struct symbol *sym);
diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h
index 9dc8abf..9f465fe 100644
--- a/scripts/kconfig/lkc_proto.h
+++ b/scripts/kconfig/lkc_proto.h
@@ -49,5 +49,11 @@ const char * sym_get_string_value(struct symbol *sym);
 
 const char * prop_get_type_name(enum prop_type type);
 
+/* preprocess.c */
+void env_write_dep(FILE *f, const char *auto_conf_name);
+char *expand_string(const char *in);
+char *expand_dollar(const char **str);
+char *expand_one_token(const char **str);
+
 /* expr.c */
 void expr_print(struct expr *e, void (*fn)(void *, struct symbol *, const char *), void *data, int prevtoken);
diff --git a/scripts/kconfig/menu.c b/scripts/kconfig/menu.c
index 5c5c137..8148305 100644
--- a/scripts/kconfig/menu.c
+++ b/scripts/kconfig/menu.c
@@ -214,9 +214,6 @@ void menu_add_option(int token, char *arg)
 			zconf_error("trying to redefine defconfig symbol");
 		sym_defconfig_list->flags |= SYMBOL_AUTO;
 		break;
-	case T_OPT_ENV:
-		prop_add_env(arg);
-		break;
 	case T_OPT_ALLNOCONFIG_Y:
 		current_entry->sym->flags |= SYMBOL_ALLNOCONFIG_Y;
 		break;
diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
new file mode 100644
index 0000000..a2eb2eb
--- /dev/null
+++ b/scripts/kconfig/preprocess.c
@@ -0,0 +1,238 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (C) 2018 Masahiro Yamada <yamada.masahiro@socionext.com>
+
+#include <stdarg.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "list.h"
+
+static void __attribute__((noreturn)) pperror(const char *format, ...)
+{
+	va_list ap;
+
+	fprintf(stderr, "%s:%d: ", current_file->name, yylineno);
+	va_start(ap, format);
+	vfprintf(stderr, format, ap);
+	va_end(ap);
+	fprintf(stderr, "\n");
+
+	exit(1);
+}
+
+/*
+ * Environment variables
+ */
+static LIST_HEAD(env_list);
+
+struct env {
+	char *name;
+	char *value;
+	struct list_head node;
+};
+
+static void env_add(const char *name, const char *value)
+{
+	struct env *e;
+
+	e = xmalloc(sizeof(*e));
+	e->name = xstrdup(name);
+	e->value = xstrdup(value);
+
+	list_add_tail(&e->node, &env_list);
+}
+
+static void env_del(struct env *e)
+{
+	list_del(&e->node);
+	free(e->name);
+	free(e->value);
+	free(e);
+}
+
+/* The returned pointer must be freed when done */
+static char *env_expand(const char *name)
+{
+	struct env *e;
+	const char *value;
+
+	if (!*name)
+		return NULL;
+
+	list_for_each_entry(e, &env_list, node) {
+		if (!strcmp(name, e->name))
+			return xstrdup(e->value);
+	}
+
+	value = getenv(name);
+	if (!value)
+		return NULL;
+
+	/*
+	 * We need to remember all referenced environment variables.
+	 * They will be written out to include/config/auto.conf.cmd
+	 */
+	env_add(name, value);
+
+	return xstrdup(value);
+}
+
+void env_write_dep(FILE *f, const char *autoconfig_name)
+{
+	struct env *e, *tmp;
+
+	list_for_each_entry_safe(e, tmp, &env_list, node) {
+		fprintf(f, "ifneq \"$(%s)\" \"%s\"\n", e->name, e->value);
+		fprintf(f, "%s: FORCE\n", autoconfig_name);
+		fprintf(f, "endif\n");
+		env_del(e);
+	}
+}
+
+static char *eval_clause(const char *str, size_t len)
+{
+	char *tmp, *name, *res;
+
+	tmp = xstrndup(str, len);
+
+	name = expand_string(tmp);
+
+	res = env_expand(name);
+	if (res)
+		goto free;
+
+	res = xstrdup("");
+free:
+	free(name);
+	free(tmp);
+
+	return res;
+}
+
+/*
+ * Expand a string that follows '$'
+ *
+ * For example, if the input string is
+ *     ($(FOO)$($(BAR)))$(BAZ)
+ * this helper evaluates
+ *     $($(FOO)$($(BAR)))
+ * and returns a new string containing the expansion (note that the string is
+ * recursively expanded), also advancing 'str' to point to the next character
+ * after the corresponding closing parenthesis, in this case, *str will be
+ *     $(BAR)
+ */
+char *expand_dollar(const char **str)
+{
+	const char *p = *str;
+	const char *q;
+	int nest = 0;
+
+	/*
+	 * In Kconfig, variable references always start with "$(".
+	 * Neither single-letter variables as in $A nor curly braces as in ${CC}
+	 * are supported.  '$' not followed by '(' loses its special meaning.
+	 */
+	if (*p != '(') {
+		*str = p;
+		return xstrdup("$");
+	}
+
+	p++;
+	q = p;
+	while (*q) {
+		if (*q == '(') {
+			nest++;
+		} else if (*q == ')') {
+			if (nest-- == 0)
+				break;
+		}
+		q++;
+	}
+
+	if (!*q)
+		pperror("unterminated reference to '%s': missing ')'", p);
+
+	/* Advance 'str' to after the expanded initial portion of the string */
+	*str = q + 1;
+
+	return eval_clause(p, q - p);
+}
+
+static char *__expand_string(const char **str, bool (*is_end)(char c))
+{
+	const char *in, *p;
+	char *expansion, *out;
+	size_t in_len, out_len;
+
+	out = xmalloc(1);
+	*out = 0;
+	out_len = 1;
+
+	p = in = *str;
+
+	while (1) {
+		if (*p == '$') {
+			in_len = p - in;
+			p++;
+			expansion = expand_dollar(&p);
+			out_len += in_len + strlen(expansion);
+			out = xrealloc(out, out_len);
+			strncat(out, in, in_len);
+			strcat(out, expansion);
+			free(expansion);
+			in = p;
+			continue;
+		}
+
+		if (is_end(*p))
+			break;
+
+		p++;
+	}
+
+	in_len = p - in;
+	out_len += in_len;
+	out = xrealloc(out, out_len);
+	strncat(out, in, in_len);
+
+	/* Advance 'str' to the end character */
+	*str = p;
+
+	return out;
+}
+
+static bool is_end_of_str(char c)
+{
+	return !c;
+}
+
+/*
+ * Expand variables in the given string.  Undefined variables
+ * expand to an empty string.
+ * The returned string must be freed when done.
+ */
+char *expand_string(const char *in)
+{
+	return __expand_string(&in, is_end_of_str);
+}
+
+static bool is_end_of_token(char c)
+{
+	/* Why are '.' and '/' valid characters for symbols? */
+	return !(isalnum(c) || c == '_' || c == '-' || c == '.' || c == '/');
+}
+
+/*
+ * Expand variables in a token.  The parsing stops when a token separater
+ * (in most cases, it is a whitespace) is encountered.  'str' is updated to
+ * point to the next character.
+ *
+ * The returned string must be freed when done.
+ */
+char *expand_one_token(const char **str)
+{
+	return __expand_string(str, is_end_of_token);
+}
diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c
index f0b2e3b..2460648 100644
--- a/scripts/kconfig/symbol.c
+++ b/scripts/kconfig/symbol.c
@@ -33,33 +33,6 @@ struct symbol *sym_defconfig_list;
 struct symbol *modules_sym;
 tristate modules_val;
 
-struct expr *sym_env_list;
-
-static void sym_add_default(struct symbol *sym, const char *def)
-{
-	struct property *prop = prop_alloc(P_DEFAULT, sym);
-
-	prop->expr = expr_alloc_symbol(sym_lookup(def, SYMBOL_CONST));
-}
-
-void sym_init(void)
-{
-	struct symbol *sym;
-	struct utsname uts;
-	static bool inited = false;
-
-	if (inited)
-		return;
-	inited = true;
-
-	uname(&uts);
-
-	sym = sym_lookup("UNAME_RELEASE", 0);
-	sym->type = S_STRING;
-	sym->flags |= SYMBOL_AUTO;
-	sym_add_default(sym, uts.release);
-}
-
 enum symbol_type sym_get_type(struct symbol *sym)
 {
 	enum symbol_type type = sym->type;
@@ -1401,32 +1374,3 @@ const char *prop_get_type_name(enum prop_type type)
 	}
 	return "unknown";
 }
-
-static void prop_add_env(const char *env)
-{
-	struct symbol *sym, *sym2;
-	struct property *prop;
-	char *p;
-
-	sym = current_entry->sym;
-	sym->flags |= SYMBOL_AUTO;
-	for_all_properties(sym, prop, P_ENV) {
-		sym2 = prop_get_symbol(prop);
-		if (strcmp(sym2->name, env))
-			menu_warn(current_entry, "redefining environment symbol from %s",
-				  sym2->name);
-		return;
-	}
-
-	prop = prop_alloc(P_ENV, sym);
-	prop->expr = expr_alloc_symbol(sym_lookup(env, SYMBOL_CONST));
-
-	sym_env_list = expr_alloc_one(E_LIST, sym_env_list);
-	sym_env_list->right.sym = sym;
-
-	p = getenv(env);
-	if (p)
-		sym_add_default(sym, p);
-	else
-		menu_warn(current_entry, "environment variable %s undefined", env);
-}
diff --git a/scripts/kconfig/util.c b/scripts/kconfig/util.c
index c6f6e21..703ee49 100644
--- a/scripts/kconfig/util.c
+++ b/scripts/kconfig/util.c
@@ -34,8 +34,6 @@ struct file *file_lookup(const char *name)
 /* write a dependency file as used by kbuild to track dependencies */
 int file_write_dep(const char *name)
 {
-	struct symbol *sym, *env_sym;
-	struct expr *e;
 	struct file *file;
 	FILE *out;
 
@@ -54,21 +52,7 @@ int file_write_dep(const char *name)
 	fprintf(out, "\n%s: \\\n"
 		     "\t$(deps_config)\n\n", conf_get_autoconfig_name());
 
-	expr_list_for_each_sym(sym_env_list, e, sym) {
-		struct property *prop;
-		const char *value;
-
-		prop = sym_get_env_prop(sym);
-		env_sym = prop_get_symbol(prop);
-		if (!env_sym)
-			continue;
-		value = getenv(env_sym->name);
-		if (!value)
-			value = "";
-		fprintf(out, "ifneq \"$(%s)\" \"%s\"\n", env_sym->name, value);
-		fprintf(out, "%s: FORCE\n", conf_get_autoconfig_name());
-		fprintf(out, "endif\n");
-	}
+	env_write_dep(out, conf_get_autoconfig_name());
 
 	fprintf(out, "\n$(deps_config): ;\n");
 	fclose(out);
@@ -165,3 +149,14 @@ char *xstrdup(const char *s)
 	fprintf(stderr, "Out of memory.\n");
 	exit(1);
 }
+
+char *xstrndup(const char *s, size_t n)
+{
+	char *p;
+
+	p = strndup(s, n);
+	if (p)
+		return p;
+	fprintf(stderr, "Out of memory.\n");
+	exit(1);
+}
diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
index 045093d..b385590 100644
--- a/scripts/kconfig/zconf.l
+++ b/scripts/kconfig/zconf.l
@@ -1,6 +1,5 @@
 %option nostdinit noyywrap never-interactive full ecs
 %option 8bit nodefault yylineno
-%option noinput
 %x COMMAND HELP STRING PARAM
 %{
 /*
@@ -35,6 +34,8 @@ struct buffer *current_buf;
 
 static int last_ts, first_ts;
 
+static char *expand_token(const char *in, size_t n);
+static void append_expanded_string(const char *in);
 static void zconf_endhelp(void);
 static void zconf_endfile(void);
 
@@ -147,6 +148,13 @@ n	[A-Za-z0-9_-]
 		yylval.string = text;
 		return T_WORD;
 	}
+	({n}|[/.$])+	{
+		/* this token includes at least one '$' */
+		yylval.string = expand_token(yytext, yyleng);
+		if (strlen(yylval.string))
+			return T_WORD;
+		free(yylval.string);
+	}
 	#.*	/* comment */
 	\\\n	;
 	[[:blank:]]+
@@ -157,12 +165,13 @@ n	[A-Za-z0-9_-]
 }
 
 <STRING>{
-	[^'"\\\n]+/\n	{
+	"$".*	append_expanded_string(yytext);
+	[^$'"\\\n]+/\n	{
 		append_string(yytext, yyleng);
 		yylval.string = text;
 		return T_WORD_QUOTE;
 	}
-	[^'"\\\n]+	{
+	[^$'"\\\n]+	{
 		append_string(yytext, yyleng);
 	}
 	\\.?/\n	{
@@ -249,6 +258,58 @@ n	[A-Za-z0-9_-]
 }
 
 %%
+static char *expand_token(const char *in, size_t n)
+{
+	char *out;
+	int c;
+	char c2;
+	const char *rest, *end;
+
+	new_string();
+	append_string(in, n);
+
+	/* get the whole line because we do not know the end of token. */
+	while ((c = input()) != EOF) {
+		if (c == '\n') {
+			unput(c);
+			break;
+		}
+		c2 = c;
+		append_string(&c2, 1);
+	}
+
+	rest = text;
+	out = expand_one_token(&rest);
+
+	/* push back unused characters to the input stream */
+	end = rest + strlen(rest);
+	while (end > rest)
+		unput(*--end);
+
+	free(text);
+
+	return out;
+}
+
+static void append_expanded_string(const char *str)
+{
+	const char *end;
+	char *res;
+
+	str++;
+
+	res = expand_dollar(&str);
+
+	/* push back unused characters to the input stream */
+	end = str + strlen(str);
+	while (end > str)
+		unput(*--end);
+
+	append_string(res, strlen(res));
+
+	free(res);
+}
+
 void zconf_starthelp(void)
 {
 	new_string();
diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y
index ad6305b..3a4a0fa 100644
--- a/scripts/kconfig/zconf.y
+++ b/scripts/kconfig/zconf.y
@@ -534,7 +534,6 @@ void conf_parse(const char *name)
 
 	zconf_initscan(name);
 
-	sym_init();
 	_menu_init();
 
 	if (getenv("ZCONF_DEBUG"))
@@ -780,3 +779,4 @@ void zconfdump(FILE *out)
 #include "expr.c"
 #include "symbol.c"
 #include "menu.c"
+#include "preprocess.c"
-- 
2.7.4

[PATCH v5 04/31] kconfig: remove string expansion in file_lookup()

[Masahiro Yamada]

There are two callers of file_lookup(), but there is no more reason
to expand the given path.

[1] zconf_initscan()
    This is used to open the first Kconfig.  sym_expand_string_value()
    has never been used in a useful way here; before opening the first
    Kconfig file, obviously there is no symbol to expand.  If you use
    expand_string_value() instead, environments in KBUILD_KCONFIG would
    be expanded, but I do not see practical benefits for that.

[2] zconf_nextfile()
    This is used to open the next file from 'source' statement.
    Symbols in the path like "arch/$SRCARCH/Kconfig" needed expanding,
    but it was replaced with the direct environment expansion.  The
    environment has already been expanded before the token is passed
    to the parser.

By the way, file_lookup() was already buggy; it expanded a given path,
but it used the path before expansion for look-up:
        if (!strcmp(name, file->name)) {

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
---

Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2:
  - Simplify the patch.  Just remove text expansion.

 scripts/kconfig/util.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/scripts/kconfig/util.c b/scripts/kconfig/util.c
index 703ee49..a365594 100644
--- a/scripts/kconfig/util.c
+++ b/scripts/kconfig/util.c
@@ -14,18 +14,16 @@
 struct file *file_lookup(const char *name)
 {
 	struct file *file;
-	char *file_name = sym_expand_string_value(name);
 
 	for (file = file_list; file; file = file->next) {
 		if (!strcmp(name, file->name)) {
-			free(file_name);
 			return file;
 		}
 	}
 
 	file = xmalloc(sizeof(*file));
 	memset(file, 0, sizeof(*file));
-	file->name = file_name;
+	file->name = xstrdup(name);
 	file->next = file_list;
 	file_list = file;
 	return file;
-- 
2.7.4

[PATCH v5 05/31] kconfig: remove string expansion for mainmenu after yyparse()

[Masahiro Yamada]

Now that environments are expanded in the lexer, conf_parse() does
not need to expand them explicitly.

The hack introduced by commit 0724a7c32a54 ("kconfig: Don't leak
main menus during parsing") can go away.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
---

Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2:
  - Simplify the patch.  Just remove the text expansion.

 scripts/kconfig/zconf.y | 24 +++++-------------------
 1 file changed, 5 insertions(+), 19 deletions(-)

diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y
index 3a4a0fa..22e318c 100644
--- a/scripts/kconfig/zconf.y
+++ b/scripts/kconfig/zconf.y
@@ -109,7 +109,7 @@ static struct menu *current_menu, *current_entry;
 %%
 input: nl start | start;
 
-start: mainmenu_stmt stmt_list | no_mainmenu_stmt stmt_list;
+start: mainmenu_stmt stmt_list | stmt_list;
 
 /* mainmenu entry */
 
@@ -118,19 +118,6 @@ mainmenu_stmt: T_MAINMENU prompt nl
 	menu_add_prompt(P_MENU, $2, NULL);
 };
 
-/* Default main menu, if there's no mainmenu entry */
-
-no_mainmenu_stmt: /* empty */
-{
-	/*
-	 * Hack: Keep the main menu title on the heap so we can safely free it
-	 * later regardless of whether it comes from the 'prompt' in
-	 * mainmenu_stmt or here
-	 */
-	menu_add_prompt(P_MENU, xstrdup("Linux Kernel Configuration"), NULL);
-};
-
-
 stmt_list:
 	  /* empty */
 	| stmt_list common_stmt
@@ -528,7 +515,6 @@ word_opt: /* empty */			{ $$ = NULL; }
 
 void conf_parse(const char *name)
 {
-	const char *tmp;
 	struct symbol *sym;
 	int i;
 
@@ -544,10 +530,10 @@ void conf_parse(const char *name)
 	if (!modules_sym)
 		modules_sym = sym_find( "n" );
 
-	tmp = rootmenu.prompt->text;
-	rootmenu.prompt->text = _(rootmenu.prompt->text);
-	rootmenu.prompt->text = sym_expand_string_value(rootmenu.prompt->text);
-	free((char*)tmp);
+	if (!menu_has_prompt(&rootmenu)) {
+		current_entry = &rootmenu;
+		menu_add_prompt(P_MENU, "Linux Kernel Configuration", NULL);
+	}
 
 	menu_finalize(&rootmenu);
 	for_all_symbols(i, sym) {
-- 
2.7.4

[PATCH v5 06/31] kconfig: remove sym_expand_string_value()

[Masahiro Yamada]

There is no more caller of sym_expand_string_value().

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---

Changes in v5: None
Changes in v4: None
Changes in v3:
  - newly added

Changes in v2: None

 scripts/kconfig/lkc_proto.h |  1 -
 scripts/kconfig/symbol.c    | 53 ---------------------------------------------
 2 files changed, 54 deletions(-)

diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h
index 9f465fe..c46929f 100644
--- a/scripts/kconfig/lkc_proto.h
+++ b/scripts/kconfig/lkc_proto.h
@@ -31,7 +31,6 @@ extern struct symbol * symbol_hash[SYMBOL_HASHSIZE];
 
 struct symbol * sym_lookup(const char *name, int flags);
 struct symbol * sym_find(const char *name);
-char *sym_expand_string_value(const char *in);
 const char * sym_escape_string_value(const char *in);
 struct symbol ** sym_re_search(const char *pattern);
 const char * sym_type_name(enum symbol_type type);
diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c
index 2460648..7c9a88e 100644
--- a/scripts/kconfig/symbol.c
+++ b/scripts/kconfig/symbol.c
@@ -879,59 +879,6 @@ struct symbol *sym_find(const char *name)
 	return symbol;
 }
 
-/*
- * Expand symbol's names embedded in the string given in argument. Symbols'
- * name to be expanded shall be prefixed by a '$'. Unknown symbol expands to
- * the empty string.
- */
-char *sym_expand_string_value(const char *in)
-{
-	const char *src;
-	char *res;
-	size_t reslen;
-
-	/*
-	 * Note: 'in' might come from a token that's about to be
-	 * freed, so make sure to always allocate a new string
-	 */
-	reslen = strlen(in) + 1;
-	res = xmalloc(reslen);
-	res[0] = '\0';
-
-	while ((src = strchr(in, '$'))) {
-		char *p, name[SYMBOL_MAXLENGTH];
-		const char *symval = "";
-		struct symbol *sym;
-		size_t newlen;
-
-		strncat(res, in, src - in);
-		src++;
-
-		p = name;
-		while (isalnum(*src) || *src == '_')
-			*p++ = *src++;
-		*p = '\0';
-
-		sym = sym_find(name);
-		if (sym != NULL) {
-			sym_calc_value(sym);
-			symval = sym_get_string_value(sym);
-		}
-
-		newlen = strlen(res) + strlen(symval) + strlen(src) + 1;
-		if (newlen > reslen) {
-			reslen = newlen;
-			res = xrealloc(res, reslen);
-		}
-
-		strcat(res, symval);
-		in = src;
-	}
-	strcat(res, in);
-
-	return res;
-}
-
 const char *sym_escape_string_value(const char *in)
 {
 	const char *p;
-- 
2.7.4

[PATCH v5 07/31] kconfig: make default prompt of mainmenu less specific

[Masahiro Yamada]

If "mainmenu" is not specified, "Linux Kernel Configuration" is used
as a default prompt.

Given that Kconfig is used in other projects than Linux, let's use
a more generic prompt, "Main menu".

Suggested-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v5:
  - Newly added

Changes in v4: None
Changes in v3: None
Changes in v2: None

 scripts/kconfig/tests/no_write_if_dep_unmet/expected_config | 2 +-
 scripts/kconfig/zconf.y                                     | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/kconfig/tests/no_write_if_dep_unmet/expected_config b/scripts/kconfig/tests/no_write_if_dep_unmet/expected_config
index 0d15e41..4732288 100644
--- a/scripts/kconfig/tests/no_write_if_dep_unmet/expected_config
+++ b/scripts/kconfig/tests/no_write_if_dep_unmet/expected_config
@@ -1,5 +1,5 @@
 #
 # Automatically generated file; DO NOT EDIT.
-# Linux Kernel Configuration
+# Main menu
 #
 # CONFIG_A is not set
diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y
index 22e318c..8a82aaf 100644
--- a/scripts/kconfig/zconf.y
+++ b/scripts/kconfig/zconf.y
@@ -532,7 +532,7 @@ void conf_parse(const char *name)
 
 	if (!menu_has_prompt(&rootmenu)) {
 		current_entry = &rootmenu;
-		menu_add_prompt(P_MENU, "Linux Kernel Configuration", NULL);
+		menu_add_prompt(P_MENU, "Main menu", NULL);
 	}
 
 	menu_finalize(&rootmenu);
-- 
2.7.4

[PATCH v5 08/31] kconfig: add built-in function support

[Masahiro Yamada]

This commit adds a new concept 'function' to do more text processing
in Kconfig.

A function call looks like this:

  $(function,arg1,arg2,arg3,...)

This commit adds the basic infrastructure to expand functions.
Change the text expansion helpers to take arguments.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v5:
  - Drop delayed expansion of arguments.

Changes in v4:
  - Error out if arguments more than FUNCTION_MAX_ARGS are passed
  - Use a comma as a delimiter between the function name and the
    first argument
  - Check the number of arguments accepted by each function
  - Support delayed expansion of arguments.
    This will be needed to implement 'if' function

Changes in v3:
  - Split base infrastructure and 'shell' function
    into separate patches.

Changes in v2:
  - Use 'shell' for getting stdout from the comment.
    It was 'shell-stdout' in the previous version.
  - Simplify the implementation since the expansion has been moved to
    lexer.

 scripts/kconfig/preprocess.c | 142 +++++++++++++++++++++++++++++++++++++++----
 1 file changed, 130 insertions(+), 12 deletions(-)

diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index a2eb2eb..f32a496 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -10,6 +10,10 @@
 
 #include "list.h"
 
+#define ARRAY_SIZE(arr)		(sizeof(arr) / sizeof((arr)[0]))
+
+static char *expand_string_with_args(const char *in, int argc, char *argv[]);
+
 static void __attribute__((noreturn)) pperror(const char *format, ...)
 {
 	va_list ap;
@@ -92,20 +96,123 @@ void env_write_dep(FILE *f, const char *autoconfig_name)
 	}
 }
 
-static char *eval_clause(const char *str, size_t len)
+/*
+ * Built-in functions
+ */
+struct function {
+	const char *name;
+	unsigned int min_args;
+	unsigned int max_args;
+	char *(*func)(int argc, char *argv[]);
+};
+
+static const struct function function_table[] = {
+	/* Name		MIN	MAX	Function */
+};
+
+#define FUNCTION_MAX_ARGS		16
+
+static char *function_expand(const char *name, int argc, char *argv[])
 {
-	char *tmp, *name, *res;
+	const struct function *f;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(function_table); i++) {
+		f = &function_table[i];
+		if (strcmp(f->name, name))
+			continue;
+
+		if (argc < f->min_args)
+			pperror("too few function arguments passed to '%s'",
+				name);
+
+		if (argc > f->max_args)
+			pperror("too many function arguments passed to '%s'",
+				name);
+
+		return f->func(argc, argv);
+	}
+
+	return NULL;
+}
+
+/*
+ * Evaluate a clause with arguments.  argc/argv are arguments from the upper
+ * function call.
+ *
+ * Returned string must be freed when done
+ */
+static char *eval_clause(const char *str, size_t len, int argc, char *argv[])
+{
+	char *tmp, *name, *res, *prev, *p;
+	int new_argc = 0;
+	char *new_argv[FUNCTION_MAX_ARGS];
+	int nest = 0;
+	int i;
 
 	tmp = xstrndup(str, len);
 
-	name = expand_string(tmp);
+	prev = p = tmp;
+
+	/*
+	 * Split into tokens
+	 * The function name and arguments are separated by a comma.
+	 * For example, if the function call is like this:
+	 *   $(foo,$(x),$(y))
+	 *
+	 * The input string for this helper should be:
+	 *   foo,$(x),$(y)
+	 *
+	 * and split into:
+	 *   new_argv[0] = 'foo'
+	 *   new_argv[1] = '$(x)'
+	 *   new_argv[2] = '$(y)'
+	 */
+	while (*p) {
+		if (nest == 0 && *p == ',') {
+			*p = 0;
+			if (new_argc >= FUNCTION_MAX_ARGS)
+				pperror("too many function arguments");
+			new_argv[new_argc++] = prev;
+			prev = p + 1;
+		} else if (*p == '(') {
+			nest++;
+		} else if (*p == ')') {
+			nest--;
+		}
 
-	res = env_expand(name);
+		p++;
+	}
+	new_argv[new_argc++] = prev;
+
+	/*
+	 * Shift arguments
+	 * new_argv[0] represents a function name or a variable name.  Put it
+	 * into 'name', then shift the rest of the arguments.  This simplifies
+	 * 'const' handling.
+	 */
+	name = expand_string_with_args(new_argv[0], argc, argv);
+	new_argc--;
+	for (i = 0; i < new_argc; i++)
+		new_argv[i] = expand_string_with_args(new_argv[i + 1],
+						      argc, argv);
+
+	/* Look for built-in functions */
+	res = function_expand(name, new_argc, new_argv);
 	if (res)
 		goto free;
 
+	/* Last, try environment variable */
+	if (new_argc == 0) {
+		res = env_expand(name);
+		if (res)
+			goto free;
+	}
+
 	res = xstrdup("");
 free:
+	for (i = 0; i < new_argc; i++)
+		free(new_argv[i]);
 	free(name);
 	free(tmp);
 
@@ -124,14 +231,14 @@ static char *eval_clause(const char *str, size_t len)
  * after the corresponding closing parenthesis, in this case, *str will be
  *     $(BAR)
  */
-char *expand_dollar(const char **str)
+static char *expand_dollar_with_args(const char **str, int argc, char *argv[])
 {
 	const char *p = *str;
 	const char *q;
 	int nest = 0;
 
 	/*
-	 * In Kconfig, variable references always start with "$(".
+	 * In Kconfig, variable/function references always start with "$(".
 	 * Neither single-letter variables as in $A nor curly braces as in ${CC}
 	 * are supported.  '$' not followed by '(' loses its special meaning.
 	 */
@@ -158,10 +265,16 @@ char *expand_dollar(const char **str)
 	/* Advance 'str' to after the expanded initial portion of the string */
 	*str = q + 1;
 
-	return eval_clause(p, q - p);
+	return eval_clause(p, q - p, argc, argv);
+}
+
+char *expand_dollar(const char **str)
+{
+	return expand_dollar_with_args(str, 0, NULL);
 }
 
-static char *__expand_string(const char **str, bool (*is_end)(char c))
+static char *__expand_string(const char **str, bool (*is_end)(char c),
+			     int argc, char *argv[])
 {
 	const char *in, *p;
 	char *expansion, *out;
@@ -177,7 +290,7 @@ static char *__expand_string(const char **str, bool (*is_end)(char c))
 		if (*p == '$') {
 			in_len = p - in;
 			p++;
-			expansion = expand_dollar(&p);
+			expansion = expand_dollar_with_args(&p, argc, argv);
 			out_len += in_len + strlen(expansion);
 			out = xrealloc(out, out_len);
 			strncat(out, in, in_len);
@@ -210,13 +323,18 @@ static bool is_end_of_str(char c)
 }
 
 /*
- * Expand variables in the given string.  Undefined variables
+ * Expand variables and functions in the given string.  Undefined variables
  * expand to an empty string.
  * The returned string must be freed when done.
  */
+static char *expand_string_with_args(const char *in, int argc, char *argv[])
+{
+	return __expand_string(&in, is_end_of_str, argc, argv);
+}
+
 char *expand_string(const char *in)
 {
-	return __expand_string(&in, is_end_of_str);
+	return expand_string_with_args(in, 0, NULL);
 }
 
 static bool is_end_of_token(char c)
@@ -234,5 +352,5 @@ static bool is_end_of_token(char c)
  */
 char *expand_one_token(const char **str)
 {
-	return __expand_string(str, is_end_of_token);
+	return __expand_string(str, is_end_of_token, 0, NULL);
 }
-- 
2.7.4

[PATCH v5 09/31] kconfig: add 'shell' built-in function

[Masahiro Yamada]

This accepts a single command to execute.  It returns the standard
output from it.

[Example code]

  config HELLO
          string
          default "$(shell,echo hello world)"

  config Y
          def_bool $(shell,echo y)

[Result]

  $ make -s alldefconfig && tail -n 2 .config
  CONFIG_HELLO="hello world"
  CONFIG_Y=y

Caveat:
Like environments, functions are expanded in the lexer.  You cannot
pass symbols to function arguments.  This is a limitation to simplify
the implementation.  I want to avoid the dynamic function evaluation,
which would introduce much more complexity.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v5: None
Changes in v4:
  - Accept only one argument to simplify the implementation

Changes in v3: None
Changes in v2: None

 scripts/kconfig/preprocess.c | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index f32a496..528be59 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -106,8 +106,49 @@ struct function {
 	char *(*func)(int argc, char *argv[]);
 };
 
+static char *do_shell(int argc, char *argv[])
+{
+	FILE *p;
+	char buf[256];
+	char *cmd;
+	size_t nread;
+	int i;
+
+	cmd = argv[0];
+
+	p = popen(cmd, "r");
+	if (!p) {
+		perror(cmd);
+		exit(1);
+	}
+
+	nread = fread(buf, 1, sizeof(buf), p);
+	if (nread == sizeof(buf))
+		nread--;
+
+	/* remove trailing new lines */
+	while (buf[nread - 1] == '\n')
+		nread--;
+
+	buf[nread] = 0;
+
+	/* replace a new line with a space */
+	for (i = 0; i < nread; i++) {
+		if (buf[i] == '\n')
+			buf[i] = ' ';
+	}
+
+	if (pclose(p) == -1) {
+		perror(cmd);
+		exit(1);
+	}
+
+	return xstrdup(buf);
+}
+
 static const struct function function_table[] = {
 	/* Name		MIN	MAX	Function */
+	{ "shell",	1,	1,	do_shell },
 };
 
 #define FUNCTION_MAX_ARGS		16
-- 
2.7.4

[PATCH v5 10/31] kconfig: replace $(UNAME_RELEASE) with function call

[Masahiro Yamada]

Now that 'shell' function is supported, this can be self-contained in
Kconfig.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
---

Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None

 Makefile     | 3 +--
 init/Kconfig | 4 ++--
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index 59b8654..58afa07 100644
--- a/Makefile
+++ b/Makefile
@@ -284,8 +284,7 @@ include scripts/Kbuild.include
 # Read KERNELRELEASE from include/config/kernel.release (if it exists)
 KERNELRELEASE = $(shell cat include/config/kernel.release 2> /dev/null)
 KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(SUBLEVEL)))$(EXTRAVERSION)
-UNAME_RELEASE := $(shell uname --release)
-export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION UNAME_RELEASE
+export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION
 
 # SUBARCH tells the usermode build what the underlying arch is.  That is set
 # first, and if a usermode build is happening, the "ARCH=um" on the command
diff --git a/init/Kconfig b/init/Kconfig
index 1217fc6..f1b0cfb 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2,9 +2,9 @@ config DEFCONFIG_LIST
 	string
 	depends on !UML
 	option defconfig_list
-	default "/lib/modules/$(UNAME_RELEASE)/.config"
+	default "/lib/modules/$(shell,uname --release)/.config"
 	default "/etc/kernel-config"
-	default "/boot/config-$(UNAME_RELEASE)"
+	default "/boot/config-$(shell,uname --release)"
 	default ARCH_DEFCONFIG
 	default "arch/$(ARCH)/defconfig"
 
-- 
2.7.4

[PATCH v5 11/31] kconfig: begin PARAM state only when seeing a command keyword

[Masahiro Yamada]

Currently, any statement line starts with a keyword with TF_COMMAND
flag.  So, the following three lines are dead code.

        alloc_string(yytext, yyleng);
        zconflval.string = text;
        return T_WORD;

If a T_WORD token is returned in this context, it will cause syntax
error in the parser anyway.

The next commit will support the assignment statement where a line
starts with an arbitrary identifier.  So, I want the lexer to switch
to the PARAM state only when it sees a command keyword.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v5: None
Changes in v4: None
Changes in v3:
 - Newly added

Changes in v2: None

 scripts/kconfig/zconf.l | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
index b385590..9a14797 100644
--- a/scripts/kconfig/zconf.l
+++ b/scripts/kconfig/zconf.l
@@ -102,10 +102,10 @@ n	[A-Za-z0-9_-]
 <COMMAND>{
 	{n}+	{
 		const struct kconf_id *id = kconf_id_lookup(yytext, yyleng);
-		BEGIN(PARAM);
 		current_pos.file = current_file;
 		current_pos.lineno = yylineno;
 		if (id && id->flags & TF_COMMAND) {
+			BEGIN(PARAM);
 			yylval.id = id;
 			return id->token;
 		}
-- 
2.7.4

[PATCH v5 12/31] kconfig: support user-defined function and recursively expanded variable

[Masahiro Yamada]

Now, we got a basic ability to test compiler capability in Kconfig.

config CC_HAS_STACKPROTECTOR
        def_bool $(shell,($(CC) -Werror -fstack-protector -E -x c /dev/null -o /dev/null 2>/dev/null) && echo y || echo n)

This works, but it is ugly to repeat this long boilerplate.

We want to describe like this:

config CC_HAS_STACKPROTECTOR
        bool
        default $(cc-option,-fstack-protector)

It is straight-forward to add a new function, but I do not like to
hard-code specialized functions like that.  Hence, here is another
feature, user-defined function.  This works as a textual shorthand
with parameterization.

A user-defined function is defined by using the = operator, and can
be referenced in the same way as built-in functions.  A user-defined
function in Make is referenced like $(call my-func,arg1,arg2), but I
omitted the 'call' to make the syntax shorter.

The definition of a user-defined function contains $(1), $(2), etc.
in its body to reference the parameters.  It is grammatically valid
to pass more or fewer arguments when calling it.  We already exploit
this feature in our makefiles; scripts/Kbuild.include defines cc-option
which takes two arguments at most, but most of the callers pass only
one argument.

By the way, a variable is supported as a subset of this feature since
a variable is "a user-defined function with zero argument".  In this
context, I mean "variable" as recursively expanded variable.  I will
add a different flavored variable in the next commit.

The code above can be written as follows:

[Example Code]

  success = $(shell,($(1)) >/dev/null 2>&1 && echo y || echo n)
  cc-option = $(success,$(CC) -Werror $(1) -E -x c /dev/null -o /dev/null)

  config CC_HAS_STACKPROTECTOR
          def_bool $(cc-option,-fstack-protector)

[Result]
  $ make -s alldefconfig && tail -n 1 .config
  CONFIG_CC_HAS_STACKPROTECTOR=y

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v5:
  - Remove lazy expansion support and simplify the code again

Changes in v4: None
Changes in v3:
  - Re-implement the parse logic
  - Use = operator to define a user-defined function

Changes in v2:
  - Use 'macro' directly instead of inside the string type symbol.

 scripts/kconfig/lkc_proto.h  |  2 ++
 scripts/kconfig/preprocess.c | 86 +++++++++++++++++++++++++++++++++++++++++++-
 scripts/kconfig/zconf.l      | 17 +++++++--
 scripts/kconfig/zconf.y      | 19 +++++++++-
 4 files changed, 120 insertions(+), 4 deletions(-)

diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h
index c46929f..2b16d6e 100644
--- a/scripts/kconfig/lkc_proto.h
+++ b/scripts/kconfig/lkc_proto.h
@@ -50,6 +50,8 @@ const char * prop_get_type_name(enum prop_type type);
 
 /* preprocess.c */
 void env_write_dep(FILE *f, const char *auto_conf_name);
+void variable_add(const char *name, const char *value);
+void variable_all_del(void);
 char *expand_string(const char *in);
 char *expand_dollar(const char **str);
 char *expand_one_token(const char **str);
diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index 528be59..46487fe 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -178,6 +178,72 @@ static char *function_expand(const char *name, int argc, char *argv[])
 }
 
 /*
+ * Variables (and user-defined functions)
+ */
+static LIST_HEAD(variable_list);
+
+struct variable {
+	char *name;
+	char *value;
+	struct list_head node;
+};
+
+static struct variable *variable_lookup(const char *name)
+{
+	struct variable *v;
+
+	list_for_each_entry(v, &variable_list, node) {
+		if (!strcmp(name, v->name))
+			return v;
+	}
+
+	return NULL;
+}
+
+static char *variable_expand(const char *name, int argc, char *argv[])
+{
+	struct variable *v;
+
+	v = variable_lookup(name);
+	if (!v)
+		return NULL;
+
+	return expand_string_with_args(v->value, argc, argv);
+}
+
+void variable_add(const char *name, const char *value)
+{
+	struct variable *v;
+
+	v = variable_lookup(name);
+	if (v) {
+		free(v->value);
+	} else {
+		v = xmalloc(sizeof(*v));
+		v->name = xstrdup(name);
+		list_add_tail(&v->node, &variable_list);
+	}
+
+	v->value = xstrdup(value);
+}
+
+static void variable_del(struct variable *v)
+{
+	list_del(&v->node);
+	free(v->name);
+	free(v->value);
+	free(v);
+}
+
+void variable_all_del(void)
+{
+	struct variable *v, *tmp;
+
+	list_for_each_entry_safe(v, tmp, &variable_list, node)
+		variable_del(v);
+}
+
+/*
  * Evaluate a clause with arguments.  argc/argv are arguments from the upper
  * function call.
  *
@@ -185,14 +251,26 @@ static char *function_expand(const char *name, int argc, char *argv[])
  */
 static char *eval_clause(const char *str, size_t len, int argc, char *argv[])
 {
-	char *tmp, *name, *res, *prev, *p;
+	char *tmp, *name, *res, *endptr, *prev, *p;
 	int new_argc = 0;
 	char *new_argv[FUNCTION_MAX_ARGS];
 	int nest = 0;
 	int i;
+	unsigned long n;
 
 	tmp = xstrndup(str, len);
 
+	/*
+	 * If variable name is '1', '2', etc.  It is generally an argument
+	 * from a user-function call (i.e. local-scope variable).  If not
+	 * available, then look-up global-scope variables.
+	 */
+	n = strtoul(tmp, &endptr, 10);
+	if (!*endptr && n > 0 && n <= argc) {
+		res = xstrdup(argv[n - 1]);
+		goto free_tmp;
+	}
+
 	prev = p = tmp;
 
 	/*
@@ -238,6 +316,11 @@ static char *eval_clause(const char *str, size_t len, int argc, char *argv[])
 		new_argv[i] = expand_string_with_args(new_argv[i + 1],
 						      argc, argv);
 
+	/* Search for variables */
+	res = variable_expand(name, new_argc, new_argv);
+	if (res)
+		goto free;
+
 	/* Look for built-in functions */
 	res = function_expand(name, new_argc, new_argv);
 	if (res)
@@ -255,6 +338,7 @@ static char *eval_clause(const char *str, size_t len, int argc, char *argv[])
 	for (i = 0; i < new_argc; i++)
 		free(new_argv[i]);
 	free(name);
+free_tmp:
 	free(tmp);
 
 	return res;
diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
index 9a14797..dd08f7a 100644
--- a/scripts/kconfig/zconf.l
+++ b/scripts/kconfig/zconf.l
@@ -1,12 +1,13 @@
 %option nostdinit noyywrap never-interactive full ecs
 %option 8bit nodefault yylineno
-%x COMMAND HELP STRING PARAM
+%x COMMAND HELP STRING PARAM ASSIGN_VAL
 %{
 /*
  * Copyright (C) 2002 Roman Zippel <zippel@linux-m68k.org>
  * Released under the terms of the GNU GPL v2.0.
  */
 
+#include <assert.h>
 #include <limits.h>
 #include <stdio.h>
 #include <stdlib.h>
@@ -111,8 +112,10 @@ n	[A-Za-z0-9_-]
 		}
 		alloc_string(yytext, yyleng);
 		yylval.string = text;
-		return T_WORD;
+		return T_VARIABLE;
 	}
+	"="	{ BEGIN(ASSIGN_VAL); return T_ASSIGN; }
+	[[:blank:]]+
 	.	warn_ignored_character(*yytext);
 	\n	{
 		BEGIN(INITIAL);
@@ -120,6 +123,16 @@ n	[A-Za-z0-9_-]
 	}
 }
 
+<ASSIGN_VAL>{
+	[^[:blank:]\n]+.*	{
+		alloc_string(yytext, yyleng);
+		yylval.string = text;
+		return T_ASSIGN_VAL;
+	}
+	\n	{ BEGIN(INITIAL); return T_EOL; }
+	.
+}
+
 <PARAM>{
 	"&&"	return T_AND;
 	"||"	return T_OR;
diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y
index 8a82aaf..e15e8c7 100644
--- a/scripts/kconfig/zconf.y
+++ b/scripts/kconfig/zconf.y
@@ -77,6 +77,9 @@ static struct menu *current_menu, *current_entry;
 %token T_CLOSE_PAREN
 %token T_OPEN_PAREN
 %token T_EOL
+%token <string> T_VARIABLE
+%token T_ASSIGN
+%token <string> T_ASSIGN_VAL
 
 %left T_OR
 %left T_AND
@@ -92,7 +95,7 @@ static struct menu *current_menu, *current_entry;
 %type <id> end
 %type <id> option_name
 %type <menu> if_entry menu_entry choice_entry
-%type <string> symbol_option_arg word_opt
+%type <string> symbol_option_arg word_opt assign_val
 
 %destructor {
 	fprintf(stderr, "%s:%d: missing end statement for this entry\n",
@@ -143,6 +146,7 @@ common_stmt:
 	| config_stmt
 	| menuconfig_stmt
 	| source_stmt
+	| assignment_stmt
 ;
 
 option_error:
@@ -511,6 +515,15 @@ symbol:	  nonconst_symbol
 word_opt: /* empty */			{ $$ = NULL; }
 	| T_WORD
 
+/* assignment statement */
+
+assignment_stmt:  T_VARIABLE T_ASSIGN assign_val T_EOL	{ variable_add($1, $3); free($1); free($3); }
+
+assign_val:
+	/* empty */		{ $$ = xstrdup(""); };
+	| T_ASSIGN_VAL
+;
+
 %%
 
 void conf_parse(const char *name)
@@ -525,6 +538,10 @@ void conf_parse(const char *name)
 	if (getenv("ZCONF_DEBUG"))
 		yydebug = 1;
 	yyparse();
+
+	/* Variables are expanded in the parse phase. We can free them here. */
+	variable_all_del();
+
 	if (yynerrs)
 		exit(1);
 	if (!modules_sym)
-- 
2.7.4

[PATCH v5 13/31] kconfig: support simply expanded variable

[Masahiro Yamada]

The previous commit added variable and user-defined function.  They
work similarly in the sense that the evaluation is deferred until
they are used.

This commit adds another type of variable, simply expanded variable,
as we see in Make.

The := operator defines a simply expanded variable, expanding the
righthand side immediately.  This works like traditional programming
language variables.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v5:
  - Fix multiple expansion bug of simple variables

Changes in v4: None
Changes in v3:
  - newly added

Changes in v2: None

 scripts/kconfig/lkc_proto.h  |  7 ++++++-
 scripts/kconfig/preprocess.c | 19 ++++++++++++++++---
 scripts/kconfig/zconf.l      |  3 ++-
 scripts/kconfig/zconf.y      |  5 +++--
 4 files changed, 27 insertions(+), 7 deletions(-)

diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h
index 2b16d6e..6303193 100644
--- a/scripts/kconfig/lkc_proto.h
+++ b/scripts/kconfig/lkc_proto.h
@@ -49,8 +49,13 @@ const char * sym_get_string_value(struct symbol *sym);
 const char * prop_get_type_name(enum prop_type type);
 
 /* preprocess.c */
+enum variable_flavor {
+	VAR_SIMPLE,
+	VAR_RECURSIVE,
+};
 void env_write_dep(FILE *f, const char *auto_conf_name);
-void variable_add(const char *name, const char *value);
+void variable_add(const char *name, const char *value,
+		  enum variable_flavor flavor);
 void variable_all_del(void);
 char *expand_string(const char *in);
 char *expand_dollar(const char **str);
diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index 46487fe..d103683 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -185,6 +185,7 @@ static LIST_HEAD(variable_list);
 struct variable {
 	char *name;
 	char *value;
+	enum variable_flavor flavor;
 	struct list_head node;
 };
 
@@ -203,15 +204,22 @@ static struct variable *variable_lookup(const char *name)
 static char *variable_expand(const char *name, int argc, char *argv[])
 {
 	struct variable *v;
+	char *res;
 
 	v = variable_lookup(name);
 	if (!v)
 		return NULL;
 
-	return expand_string_with_args(v->value, argc, argv);
+	if (v->flavor == VAR_RECURSIVE)
+		res = expand_string_with_args(v->value, argc, argv);
+	else
+		res = xstrdup(v->value);
+
+	return res;
 }
 
-void variable_add(const char *name, const char *value)
+void variable_add(const char *name, const char *value,
+		  enum variable_flavor flavor)
 {
 	struct variable *v;
 
@@ -224,7 +232,12 @@ void variable_add(const char *name, const char *value)
 		list_add_tail(&v->node, &variable_list);
 	}
 
-	v->value = xstrdup(value);
+	v->flavor = flavor;
+
+	if (flavor == VAR_SIMPLE)
+		v->value = expand_string(value);
+	else
+		v->value = xstrdup(value);
 }
 
 static void variable_del(struct variable *v)
diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
index dd08f7a..376af6c 100644
--- a/scripts/kconfig/zconf.l
+++ b/scripts/kconfig/zconf.l
@@ -114,7 +114,8 @@ n	[A-Za-z0-9_-]
 		yylval.string = text;
 		return T_VARIABLE;
 	}
-	"="	{ BEGIN(ASSIGN_VAL); return T_ASSIGN; }
+	"="	{ BEGIN(ASSIGN_VAL); yylval.flavor = VAR_RECURSIVE; return T_ASSIGN; }
+	":="	{ BEGIN(ASSIGN_VAL); yylval.flavor = VAR_SIMPLE; return T_ASSIGN; }
 	[[:blank:]]+
 	.	warn_ignored_character(*yytext);
 	\n	{
diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y
index e15e8c7..6f9b0aa 100644
--- a/scripts/kconfig/zconf.y
+++ b/scripts/kconfig/zconf.y
@@ -41,6 +41,7 @@ static struct menu *current_menu, *current_entry;
 	struct expr *expr;
 	struct menu *menu;
 	const struct kconf_id *id;
+	enum variable_flavor flavor;
 }
 
 %token <id>T_MAINMENU
@@ -78,7 +79,7 @@ static struct menu *current_menu, *current_entry;
 %token T_OPEN_PAREN
 %token T_EOL
 %token <string> T_VARIABLE
-%token T_ASSIGN
+%token <flavor> T_ASSIGN
 %token <string> T_ASSIGN_VAL
 
 %left T_OR
@@ -517,7 +518,7 @@ word_opt: /* empty */			{ $$ = NULL; }
 
 /* assignment statement */
 
-assignment_stmt:  T_VARIABLE T_ASSIGN assign_val T_EOL	{ variable_add($1, $3); free($1); free($3); }
+assignment_stmt:  T_VARIABLE T_ASSIGN assign_val T_EOL	{ variable_add($1, $3, $2); free($1); free($3); }
 
 assign_val:
 	/* empty */		{ $$ = xstrdup(""); };
-- 
2.7.4

[PATCH v5 14/31] kconfig: support append assignment operator

[Masahiro Yamada]

Support += operator.  This appends a space and the text on the
righthand side to a variable.

The timing of the evaluation of the righthand side depends on the
flavor of the variable.  If the lefthand side was originally defined
as a simple variable, the righthand side is expanded immediately.
Otherwise, the expansion is deferred.  Appending something to an
undefined variable results in a recursive variable.

To implement this, we need to remember the flavor of variables.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v5: None
Changes in v4: None
Changes in v3:
  - newly added

Changes in v2: None

 scripts/kconfig/lkc_proto.h  |  1 +
 scripts/kconfig/preprocess.c | 28 +++++++++++++++++++++++++---
 scripts/kconfig/zconf.l      |  1 +
 3 files changed, 27 insertions(+), 3 deletions(-)

diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h
index 6303193..a8b7a33 100644
--- a/scripts/kconfig/lkc_proto.h
+++ b/scripts/kconfig/lkc_proto.h
@@ -52,6 +52,7 @@ const char * prop_get_type_name(enum prop_type type);
 enum variable_flavor {
 	VAR_SIMPLE,
 	VAR_RECURSIVE,
+	VAR_APPEND,
 };
 void env_write_dep(FILE *f, const char *auto_conf_name);
 void variable_add(const char *name, const char *value,
diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index d103683..56aa1f0 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -222,11 +222,23 @@ void variable_add(const char *name, const char *value,
 		  enum variable_flavor flavor)
 {
 	struct variable *v;
+	char *new_value;
+	bool append = false;
 
 	v = variable_lookup(name);
 	if (v) {
-		free(v->value);
+		/* For defined variables, += inherits the existing flavor */
+		if (flavor == VAR_APPEND) {
+			flavor = v->flavor;
+			append = true;
+		} else {
+			free(v->value);
+		}
 	} else {
+		/* For undefined variables, += assumes the recursive flavor */
+		if (flavor == VAR_APPEND)
+			flavor = VAR_RECURSIVE;
+
 		v = xmalloc(sizeof(*v));
 		v->name = xstrdup(name);
 		list_add_tail(&v->node, &variable_list);
@@ -235,9 +247,19 @@ void variable_add(const char *name, const char *value,
 	v->flavor = flavor;
 
 	if (flavor == VAR_SIMPLE)
-		v->value = expand_string(value);
+		new_value = expand_string(value);
 	else
-		v->value = xstrdup(value);
+		new_value = xstrdup(value);
+
+	if (append) {
+		v->value = xrealloc(v->value,
+				    strlen(v->value) + strlen(new_value) + 2);
+		strcat(v->value, " ");
+		strcat(v->value, new_value);
+		free(new_value);
+	} else {
+		v->value = new_value;
+	}
 }
 
 static void variable_del(struct variable *v)
diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
index 376af6c..a6cbe2d 100644
--- a/scripts/kconfig/zconf.l
+++ b/scripts/kconfig/zconf.l
@@ -116,6 +116,7 @@ n	[A-Za-z0-9_-]
 	}
 	"="	{ BEGIN(ASSIGN_VAL); yylval.flavor = VAR_RECURSIVE; return T_ASSIGN; }
 	":="	{ BEGIN(ASSIGN_VAL); yylval.flavor = VAR_SIMPLE; return T_ASSIGN; }
+	"+="	{ BEGIN(ASSIGN_VAL); yylval.flavor = VAR_APPEND; return T_ASSIGN; }
 	[[:blank:]]+
 	.	warn_ignored_character(*yytext);
 	\n	{
-- 
2.7.4

[PATCH v5 15/31] kconfig: expand lefthand side of assignment statement

[Masahiro Yamada]

Make expands the lefthand side of assignment statements.  In fact,
Kbuild relies on it since kernel makefiles mostly look like this:

  obj-$(CONFIG_FOO) += foo.o

Do likewise in Kconfig.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None

 scripts/kconfig/zconf.l | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
index a6cbe2d..25bd2b8 100644
--- a/scripts/kconfig/zconf.l
+++ b/scripts/kconfig/zconf.l
@@ -114,6 +114,13 @@ n	[A-Za-z0-9_-]
 		yylval.string = text;
 		return T_VARIABLE;
 	}
+	({n}|$)+	{
+		/* this token includes at least one '$' */
+		yylval.string = expand_token(yytext, yyleng);
+		if (strlen(yylval.string))
+			return T_VARIABLE;
+		free(yylval.string);
+	}
 	"="	{ BEGIN(ASSIGN_VAL); yylval.flavor = VAR_RECURSIVE; return T_ASSIGN; }
 	":="	{ BEGIN(ASSIGN_VAL); yylval.flavor = VAR_SIMPLE; return T_ASSIGN; }
 	"+="	{ BEGIN(ASSIGN_VAL); yylval.flavor = VAR_APPEND; return T_ASSIGN; }
-- 
2.7.4

[PATCH v5 16/31] kconfig: add 'info', 'warning-if', and 'error-if' built-in functions

[Masahiro Yamada]

Syntax:
  $(info,<text>)
  $(warning-if,<condition>,<text>)
  $(error-if,<condition>,<text)

The 'info' function prints a message to stdout as in Make.

The 'warning-if' and 'error-if' are similar to 'warning' and 'error'
in Make, but take the condition parameter.  They are effective only
when the <condition> part is y.

Kconfig does not implement the lazy expansion as used in the 'if'
'and, 'or' functions in Make.  In other words, Kconfig does not
support conditional expansion.  The unconditional 'error' function
would always terminate the parsing, hence would be useless in Kconfig.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---

Changes in v5:
  - Replace 'warning', 'error' with 'warning-if', 'error-if'

Changes in v4:
  - Add 'error' function

Changes in v3: None
Changes in v2: None

 scripts/kconfig/preprocess.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index 56aa1f0..5ee58ee 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -106,6 +106,21 @@ struct function {
 	char *(*func)(int argc, char *argv[]);
 };
 
+static char *do_error_if(int argc, char *argv[])
+{
+	if (!strcmp(argv[0], "y"))
+		pperror("%s", argv[1]);
+
+	return NULL;
+}
+
+static char *do_info(int argc, char *argv[])
+{
+	printf("%s\n", argv[0]);
+
+	return xstrdup("");
+}
+
 static char *do_shell(int argc, char *argv[])
 {
 	FILE *p;
@@ -146,9 +161,21 @@ static char *do_shell(int argc, char *argv[])
 	return xstrdup(buf);
 }
 
+static char *do_warning_if(int argc, char *argv[])
+{
+	if (!strcmp(argv[0], "y"))
+		fprintf(stderr, "%s:%d: %s\n",
+			current_file->name, yylineno, argv[1]);
+
+	return xstrdup("");
+}
+
 static const struct function function_table[] = {
 	/* Name		MIN	MAX	Function */
+	{ "error-if",	2,	2,	do_error_if },
+	{ "info",	1,	1,	do_info },
 	{ "shell",	1,	1,	do_shell },
+	{ "warning-if",	2,	2,	do_warning_if },
 };
 
 #define FUNCTION_MAX_ARGS		16
-- 
2.7.4

[PATCH v5 17/31] kconfig: add 'filename' and 'lineno' built-in variables

[Masahiro Yamada]

The special variables, $(filename) and $(lineno), are expanded to a
file name and its line number being parsed, respectively.

Suggested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---

Changes in v5: None
Changes in v4:
 - Newly added

Changes in v3: None
Changes in v2: None

 scripts/kconfig/preprocess.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index 5ee58ee..0574039 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -114,6 +114,11 @@ static char *do_error_if(int argc, char *argv[])
 	return NULL;
 }
 
+static char *do_filename(int argc, char *argv[])
+{
+	return xstrdup(current_file->name);
+}
+
 static char *do_info(int argc, char *argv[])
 {
 	printf("%s\n", argv[0]);
@@ -121,6 +126,15 @@ static char *do_info(int argc, char *argv[])
 	return xstrdup("");
 }
 
+static char *do_lineno(int argc, char *argv[])
+{
+	char buf[16];
+
+	sprintf(buf, "%d", yylineno);
+
+	return xstrdup(buf);
+}
+
 static char *do_shell(int argc, char *argv[])
 {
 	FILE *p;
@@ -173,7 +187,9 @@ static char *do_warning_if(int argc, char *argv[])
 static const struct function function_table[] = {
 	/* Name		MIN	MAX	Function */
 	{ "error-if",	2,	2,	do_error_if },
+	{ "filename",	0,	0,	do_filename },
 	{ "info",	1,	1,	do_info },
+	{ "lineno",	0,	0,	do_lineno },
 	{ "shell",	1,	1,	do_shell },
 	{ "warning-if",	2,	2,	do_warning_if },
 };
-- 
2.7.4

[PATCH v5 18/31] kconfig: error out if a recursive variable references itself

[Masahiro Yamada]

When using a recursively expanded variable, it is a common mistake
to make circular reference.

For example, Make terminates the following code:

  X = $(X)
  Y := $(X)

Let's detect the circular expansion in Kconfig, too.

On the other hand, a function that recurses itself is a commonly-used
programming technique.  So, Make does not check recursion in the
reference with 'call'.  For example, the following code continues
running eternally:

  X = $(call X)
  Y := $(X)

Kconfig allows circular expansion if one or more arguments are given,
but terminates when the same function is recursively invoked 1000 times,
assuming it is a programming mistake.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v5: None
Changes in v4:
 - Newly added

Changes in v3: None
Changes in v2: None

 scripts/kconfig/preprocess.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
index 0574039..65da87f 100644
--- a/scripts/kconfig/preprocess.c
+++ b/scripts/kconfig/preprocess.c
@@ -229,6 +229,7 @@ struct variable {
 	char *name;
 	char *value;
 	enum variable_flavor flavor;
+	int exp_count;
 	struct list_head node;
 };
 
@@ -253,11 +254,22 @@ static char *variable_expand(const char *name, int argc, char *argv[])
 	if (!v)
 		return NULL;
 
+	if (argc == 0 && v->exp_count)
+		pperror("Recursive variable '%s' references itself (eventually)",
+			name);
+
+	if (v->exp_count > 1000)
+		pperror("Too deep recursive expansion");
+
+	v->exp_count++;
+
 	if (v->flavor == VAR_RECURSIVE)
 		res = expand_string_with_args(v->value, argc, argv);
 	else
 		res = xstrdup(v->value);
 
+	v->exp_count--;
+
 	return res;
 }
 
@@ -284,6 +296,7 @@ void variable_add(const char *name, const char *value,
 
 		v = xmalloc(sizeof(*v));
 		v->name = xstrdup(name);
+		v->exp_count = 0;
 		list_add_tail(&v->node, &variable_list);
 	}
 
-- 
2.7.4

[PATCH v5 19/31] Documentation: kconfig: document a new Kconfig macro language

[Masahiro Yamada]

Add a document for the macro language introduced to Kconfig.

The motivation of this work is to move the compiler option tests to
Kconfig from Makefile.  A number of kernel features require the
compiler support.  Enabling such features blindly in Kconfig ends up
with a lot of nasty build-time testing in Makefiles.  If a chosen
feature turns out unsupported by the compiler, what the build system
can do is either to disable it (silently!) or to forcibly break the
build, despite Kconfig has let the user to enable it.  By moving the
compiler capability tests to Kconfig, features unsupported by the
compiler will be hidden automatically.

This change was strongly prompted by Linus Torvalds.  You can find
his suggestions [1] [2] in ML.  The original idea was to add a new
attribute with 'option shell=...', but I found more generalized text
expansion would make Kconfig more powerful and lovely.  The basic
ideas are from Make, but there are some differences.

[1]: https://lkml.org/lkml/2016/12/9/577
[2]: https://lkml.org/lkml/2018/2/7/527

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
---

Changes in v5: None
Changes in v4:
 - Update according to the syntax change

Changes in v3:
 - Newly added

Changes in v2: None

 Documentation/kbuild/kconfig-macro-language.txt | 242 ++++++++++++++++++++++++
 MAINTAINERS                                     |   2 +-
 2 files changed, 243 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/kbuild/kconfig-macro-language.txt

diff --git a/Documentation/kbuild/kconfig-macro-language.txt b/Documentation/kbuild/kconfig-macro-language.txt
new file mode 100644
index 0000000..07da2ea
--- /dev/null
+++ b/Documentation/kbuild/kconfig-macro-language.txt
@@ -0,0 +1,242 @@
+Concept
+-------
+
+The basic idea was inspired by Make. When we look at Make, we notice sort of
+two languages in one. One language describes dependency graphs consisting of
+targets and prerequisites. The other is a macro language for performing textual
+substitution.
+
+There is clear distinction between the two language stages. For example, you
+can write a makefile like follows:
+
+    APP := foo
+    SRC := foo.c
+    CC := gcc
+
+    $(APP): $(SRC)
+            $(CC) -o $(APP) $(SRC)
+
+The macro language replaces the variable references with their expanded form,
+and handles as if the source file were input like follows:
+
+    foo: foo.c
+            gcc -o foo foo.c
+
+Then, Make analyzes the dependency graph and determines the targets to be
+updated.
+
+The idea is quite similar in Kconfig - it is possible to describe a Kconfig
+file like this:
+
+    CC := gcc
+
+    config CC_HAS_FOO
+            def_bool $(shell, $(srctree)/scripts/gcc-check-foo.sh $(CC))
+
+The macro language in Kconfig processes the source file into the following
+intermediate:
+
+    config CC_HAS_FOO
+            def_bool y
+
+Then, Kconfig moves onto the evaluation stage to resolve inter-symbol
+dependency as explained in kconfig-language.txt.
+
+
+Variables
+---------
+
+Like in Make, a variable in Kconfig works as a macro variable.  A macro
+variable is expanded "in place" to yield a text string that may then be
+expanded further. To get the value of a variable, enclose the variable name in
+$( ). The parentheses are required even for single-letter variable names; $X is
+a syntax error. The curly brace form as in ${CC} is not supported either.
+
+There are two types of variables: simply expanded variables and recursively
+expanded variables.
+
+A simply expanded variable is defined using the := assignment operator. Its
+righthand side is expanded immediately upon reading the line from the Kconfig
+file.
+
+A recursively expanded variable is defined using the = assignment operator.
+Its righthand side is simply stored as the value of the variable without
+expanding it in any way. Instead, the expansion is performed when the variable
+is used.
+
+There is another type of assignment operator; += is used to append text to a
+variable. The righthand side of += is expanded immediately if the lefthand
+side was originally defined as a simple variable. Otherwise, its evaluation is
+deferred.
+
+The variable reference can take parameters, in the following form:
+
+  $(name,arg1,arg2,arg3)
+
+You can consider the parameterized reference as a function. (more precisely,
+"user-defined function" in contrast to "built-in function" listed below).
+
+Useful functions must be expanded when they are used since the same function is
+expanded differently if different parameters are passed. Hence, a user-defined
+function is defined using the = assignment operator. The parameters are
+referenced within the body definition with $(1), $(2), etc.
+
+In fact, recursively expanded variables and user-defined functions are the same
+internally. (In other words, "variable" is "function with zero argument".)
+When we say "variable" in a broad sense, it includes "user-defined function".
+
+
+Built-in functions
+------------------
+
+Like Make, Kconfig provides several built-in functions. Every function takes a
+particular number of arguments.
+
+In Make, every built-in function takes at least one argument. Kconfig allows
+zero argument for built-in functions, such as $(fileno), $(lineno). You could
+consider those as "built-in variable", but it is just a matter of how we call
+it after all. Let's say "built-in function" here to refer to natively supported
+functionality.
+
+Kconfig currently supports the following built-in functions.
+
+ - $(shell,command)
+
+  The "shell" function accepts a single argument that is expanded and passed
+  to a subshell for execution. The standard output of the command is then read
+  and returned as the value of the function. Every newline in the output is
+  replaced with a space. Any trailing newlines are deleted. The standard error
+  is not returned, nor is any program exit status.
+
+ - $(info,text)
+
+  The "info" function takes a single argument and prints it to stdout.
+  It evaluates to an empty string.
+
+ - $(warning-if,condition,text)
+
+  The "warning-if" function takes two arguments. If the condition part is "y",
+  the text part is sent to stderr. The text is prefixed with the name of the
+  current Kconfig file and the current line number.
+
+ - $(error-if,condition,text)
+
+  The "error-if" function is similar to "warning-if", but it terminates the
+  parsing immediately if the condition part is "y".
+
+ - $(filename)
+
+  The 'filename' takes no argument, and $(filename) is expanded to the file
+  name being parsed.
+
+ - $(lineno)
+
+  The 'lineno' takes no argument, and $(lineno) is expanded to the line number
+  being parsed.
+
+
+Make vs Kconfig
+---------------
+
+Kconfig adopts Make-like macro language, but the function call syntax is
+slightly different.
+
+A function call in Make looks like this:
+
+  $(func-name arg1,arg2,arg3)
+
+The function name and the first argument are separated by at least one
+whitespace. Then, leading whitespaces are trimmed from the first argument,
+while whitespaces in the other arguments are kept. You need to use a kind of
+trick to start the first parameter with spaces. For example, if you want
+to make "info" function print "  hello", you can write like follows:
+
+  empty :=
+  space := $(empty) $(empty)
+  $(info $(space)$(space)hello)
+
+Kconfig uses only commas for delimiters, and keeps all whitespaces in the
+function call. Some people prefer putting a space after each comma delimiter:
+
+  $(func-name, arg1, arg2, arg3)
+
+In this case, "func-name" will receive " arg1", " arg2", " arg3". The presence
+of leading spaces may matter depending on the function. The same applies to
+Make - for example, $(subst .c, .o, $(sources)) is a typical mistake; it
+replaces ".c" with " .o".
+
+In Make, a user-defined function is referenced by using a built-in function,
+'call', like this:
+
+    $(call my-func,arg1,arg2,arg3)
+
+Kconfig invokes user-defined functions and built-in functions in the same way.
+The omission of 'call' makes the syntax shorter.
+
+In Make, some functions treat commas verbatim instead of argument separators.
+For example, $(shell echo hello, world) runs the command "echo hello, world".
+Likewise, $(info hello, world) prints "hello, world" to stdout. You could say
+this is _useful_ inconsistency.
+
+In Kconfig, for simpler implementation and grammatical consistency, commas that
+appear in the $( ) context are always delimiters. It means
+
+  $(shell, echo hello, world)
+
+is an error because it is passing two parameters where the 'shell' function
+accepts only one. To pass commas in arguments, you can use the following trick:
+
+  comma := ,
+  $(shell, echo hello$(comma) world)
+
+
+Caveats
+-------
+
+A variable (or function) cannot be expanded across tokens. So, you cannot use
+a variable as a shorthand for an expression that consists of multiple tokens.
+The following works:
+
+    RANGE_MIN := 1
+    RANGE_MAX := 3
+
+    config FOO
+            int "foo"
+            range $(RANGE_MIN) $(RANGE_MAX)
+
+But, the following does not work:
+
+    RANGES := 1 3
+
+    config FOO
+            int "foo"
+            range $(RANGES)
+
+A variable cannot be expanded to any keyword in Kconfig.  The following does
+not work:
+
+    MY_TYPE := tristate
+
+    config FOO
+            $(MY_TYPE) "foo"
+            default y
+
+Obviously from the design, $(shell command) is expanded in the textual
+substitution phase. You cannot pass symbols to the 'shell' function.
+The following does not work as expected.
+
+    config ENDIAN_FLAG
+            string
+            default "-mbig-endian" if CPU_BIG_ENDIAN
+            default "-mlittle-endian" if CPU_LITTLE_ENDIAN
+
+    config CC_HAS_ENDIAN_FLAG
+            def_bool $(shell $(srctree)/scripts/gcc-check-flag ENDIAN_FLAG)
+
+Instead, you can do like follows so that any function call is statically
+expanded.
+
+    config CC_HAS_ENDIAN_FLAG
+            bool
+            default $(shell $(srctree)/scripts/gcc-check-flag -mbig-endian) if CPU_BIG_ENDIAN
+            default $(shell $(srctree)/scripts/gcc-check-flag -mlittle-endian) if CPU_LITTLE_ENDIAN
diff --git a/MAINTAINERS b/MAINTAINERS
index ca4afd6..b87723a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7637,7 +7637,7 @@ M:	Masahiro Yamada <yamada.masahiro@socionext.com>
 T:	git git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git kconfig
 L:	linux-kbuild@vger.kernel.org
 S:	Maintained
-F:	Documentation/kbuild/kconfig-language.txt
+F:	Documentation/kbuild/kconfig*
 F:	scripts/kconfig/
 
 KDUMP
-- 
2.7.4

[PATCH v5 20/31] kconfig: test: add Kconfig macro language tests

[Masahiro Yamada]

Here are the test cases I used for developing the text expansion
feature.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v5:
  - Add SPDX License Identifier to test files
  - Update tests according to the simplification of grammar
  - Add more escape sequence tricks

Changes in v4:
  - Adjust for the change in function call syntax.
  - Remove unnecessarily tricky tests.

Changes in v3:
  - newly added

Changes in v2: None

 .../kconfig/tests/preprocess/builtin_func/Kconfig  | 27 +++++++++++
 .../tests/preprocess/builtin_func/__init__.py      |  9 ++++
 .../tests/preprocess/builtin_func/expected_stderr  |  5 ++
 .../tests/preprocess/builtin_func/expected_stdout  |  1 +
 .../tests/preprocess/circular_expansion/Kconfig    |  5 ++
 .../preprocess/circular_expansion/__init__.py      | 11 +++++
 .../preprocess/circular_expansion/expected_stderr  |  1 +
 scripts/kconfig/tests/preprocess/escape/Kconfig    | 44 ++++++++++++++++++
 .../kconfig/tests/preprocess/escape/__init__.py    |  8 ++++
 .../tests/preprocess/escape/expected_stderr        | 10 ++++
 scripts/kconfig/tests/preprocess/variable/Kconfig  | 53 ++++++++++++++++++++++
 .../kconfig/tests/preprocess/variable/__init__.py  |  8 ++++
 .../tests/preprocess/variable/expected_stderr      |  9 ++++
 13 files changed, 191 insertions(+)
 create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/Kconfig
 create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/__init__.py
 create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/expected_stderr
 create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/expected_stdout
 create mode 100644 scripts/kconfig/tests/preprocess/circular_expansion/Kconfig
 create mode 100644 scripts/kconfig/tests/preprocess/circular_expansion/__init__.py
 create mode 100644 scripts/kconfig/tests/preprocess/circular_expansion/expected_stderr
 create mode 100644 scripts/kconfig/tests/preprocess/escape/Kconfig
 create mode 100644 scripts/kconfig/tests/preprocess/escape/__init__.py
 create mode 100644 scripts/kconfig/tests/preprocess/escape/expected_stderr
 create mode 100644 scripts/kconfig/tests/preprocess/variable/Kconfig
 create mode 100644 scripts/kconfig/tests/preprocess/variable/__init__.py
 create mode 100644 scripts/kconfig/tests/preprocess/variable/expected_stderr

diff --git a/scripts/kconfig/tests/preprocess/builtin_func/Kconfig b/scripts/kconfig/tests/preprocess/builtin_func/Kconfig
new file mode 100644
index 0000000..baa3288
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/builtin_func/Kconfig
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: GPL-2.0
+
+# 'info' prints the argument to stdout.
+$(info,hello world 0)
+
+# 'warning-if', if the first argument is y, sends the second argument to stderr,
+# and the message is prefixed with the current file name and line number.
+$(warning-if,y,hello world 1)
+
+# 'error-if' is similar, but it terminates the parsing immediately.
+# The following is just no-op since the first argument is not y.
+$(error-if,n,this should not be printed)
+
+# Shorthand
+warning = $(warning-if,y,$(1))
+
+# 'shell' executes a command, and returns its stdout.
+$(warning,$(shell,echo hello world 3))
+
+# Every newline in the output is replaced with a space,
+# but any trailing newlines are deleted.
+$(warning,$(shell,printf 'hello\nworld\n\n4\n\n\n'))
+
+# 'filename' is expanded to the currently parsed file name,
+# 'lineno' to the line number.
+$(warning,filename=$(filename))
+$(warning,lineno=$(lineno))
diff --git a/scripts/kconfig/tests/preprocess/builtin_func/__init__.py b/scripts/kconfig/tests/preprocess/builtin_func/__init__.py
new file mode 100644
index 0000000..2e53ba0
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/builtin_func/__init__.py
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0
+"""
+Built-in function tests.
+"""
+
+def test(conf):
+    assert conf.oldaskconfig() == 0
+    assert conf.stdout_contains('expected_stdout')
+    assert conf.stderr_matches('expected_stderr')
diff --git a/scripts/kconfig/tests/preprocess/builtin_func/expected_stderr b/scripts/kconfig/tests/preprocess/builtin_func/expected_stderr
new file mode 100644
index 0000000..33ea9ca
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/builtin_func/expected_stderr
@@ -0,0 +1,5 @@
+Kconfig:8: hello world 1
+Kconfig:18: hello world 3
+Kconfig:22: hello world  4
+Kconfig:26: filename=Kconfig
+Kconfig:27: lineno=27
diff --git a/scripts/kconfig/tests/preprocess/builtin_func/expected_stdout b/scripts/kconfig/tests/preprocess/builtin_func/expected_stdout
new file mode 100644
index 0000000..82de3a7
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/builtin_func/expected_stdout
@@ -0,0 +1 @@
+hello world 0
diff --git a/scripts/kconfig/tests/preprocess/circular_expansion/Kconfig b/scripts/kconfig/tests/preprocess/circular_expansion/Kconfig
new file mode 100644
index 0000000..6838997
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/circular_expansion/Kconfig
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0
+
+X = $(Y)
+Y = $(X)
+$(info $(X))
diff --git a/scripts/kconfig/tests/preprocess/circular_expansion/__init__.py b/scripts/kconfig/tests/preprocess/circular_expansion/__init__.py
new file mode 100644
index 0000000..419bda3
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/circular_expansion/__init__.py
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0
+"""
+Detect circular variable expansion.
+
+If a recursively expanded variable references itself (eventually),
+it should fail with an error message.
+"""
+
+def test(conf):
+    assert conf.oldaskconfig() != 0
+    assert conf.stderr_matches('expected_stderr')
diff --git a/scripts/kconfig/tests/preprocess/circular_expansion/expected_stderr b/scripts/kconfig/tests/preprocess/circular_expansion/expected_stderr
new file mode 100644
index 0000000..cde68fa
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/circular_expansion/expected_stderr
@@ -0,0 +1 @@
+Kconfig:5: Recursive variable 'X' references itself (eventually)
diff --git a/scripts/kconfig/tests/preprocess/escape/Kconfig b/scripts/kconfig/tests/preprocess/escape/Kconfig
new file mode 100644
index 0000000..4e3f444
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/escape/Kconfig
@@ -0,0 +1,44 @@
+# SPDX-License-Identifier: GPL-2.0
+
+# Shorthand
+warning = $(warning-if,y,$(1))
+
+# You can not pass commas directly to a function since they are treated as
+# delimiters. You can use the following trick to do so.
+comma   := ,
+$(warning,hello$(comma) world)
+
+# Like Make, single quotes, double quotes, spaces are treated verbatim.
+# The following prints the text as-is.
+$(warning,  ' " '"   ' ''' "'")
+
+# Unlike Make, '$' has special meaning only when it is followed by '('.
+# No need to escape '$' itself.
+$(warning,$)
+$(warning,$$)
+$ := 1
+$(warning,$($))
+
+# You need a trick to escape '$' followed by '('
+# The following should print "$(X)". It should not be expanded further.
+dollar := $
+$(warning,$(dollar)(X))
+
+# You need a trick to treat unbalanced parentheses.
+# The following should print "(".
+left_paren := (
+$(warning,$(left_paren))
+
+# A simple expanded should not be expanded multiple times.
+# The following should print "$(X)". It should not be expanded further.
+Y := $(dollar)(X)
+$(warning,$(Y))
+
+# The following should print "$(X)" as well.
+Y = $(dollar)(X)
+$(warning,$(Y))
+
+# The following should print "$(".
+# It should not be emit "unterminated reference" error.
+unterminated := $(dollar)(
+$(warning,$(unterminated))
diff --git a/scripts/kconfig/tests/preprocess/escape/__init__.py b/scripts/kconfig/tests/preprocess/escape/__init__.py
new file mode 100644
index 0000000..7ee8e74
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/escape/__init__.py
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+"""
+Escape sequence tests.
+"""
+
+def test(conf):
+    assert conf.oldaskconfig() == 0
+    assert conf.stderr_matches('expected_stderr')
diff --git a/scripts/kconfig/tests/preprocess/escape/expected_stderr b/scripts/kconfig/tests/preprocess/escape/expected_stderr
new file mode 100644
index 0000000..1c00957
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/escape/expected_stderr
@@ -0,0 +1,10 @@
+Kconfig:9: hello, world
+Kconfig:13:   ' " '"   ' ''' "'"
+Kconfig:17: $
+Kconfig:18: $$
+Kconfig:20: 1
+Kconfig:25: $(X)
+Kconfig:30: (
+Kconfig:35: $(X)
+Kconfig:39: $(X)
+Kconfig:44: $(
diff --git a/scripts/kconfig/tests/preprocess/variable/Kconfig b/scripts/kconfig/tests/preprocess/variable/Kconfig
new file mode 100644
index 0000000..9ce2f95
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/variable/Kconfig
@@ -0,0 +1,53 @@
+# SPDX-License-Identifier: GPL-2.0
+
+# Shorthand
+warning = $(warning-if,y,$(1))
+
+# Simply expanded variable.
+X := 1
+SIMPLE := $(X)
+X := 2
+$(warning,SIMPLE = $(SIMPLE))
+
+# Recursively expanded variable.
+X := 1
+RECURSIVE = $(X)
+X := 2
+$(warning,RECURSIVE = $(RECURSIVE))
+
+# Append something to a simply expanded variable.
+Y := 3
+SIMPLE += $(Y)
+Y := 4
+$(warning,SIMPLE = $(SIMPLE))
+
+# Append something to a recursively expanded variable.
+Y := 3
+RECURSIVE += $(Y)
+Y := 4
+$(warning,RECURSIVE = $(RECURSIVE))
+
+# Use += operator to an undefined variable.
+# This works as a recursively expanded variable.
+Y := 3
+UNDEFINED_VARIABLE += $(Y)
+Y := 4
+$(warning,UNDEFINED_VARIABLE = $(UNDEFINED_VARIABLE))
+
+# You can use variable references for the lefthand side of assignment statement.
+X := A
+Y := B
+$(X)$(Y) := 5
+$(warning,AB = $(AB))
+
+# User-defined function.
+greeting = $(1), my name is $(2).
+$(warning,$(greeting,Hello,John))
+
+# The number of arguments is not checked for user-defined functions.
+# If some arguments are optional, it is useful to pass fewer parameters.
+# $(2) will be blank in this case.
+$(warning,$(greeting,Hello))
+
+# Unreferenced parameters are just ignored.
+$(warning,$(greeting,Hello,John,ignored,ignored))
diff --git a/scripts/kconfig/tests/preprocess/variable/__init__.py b/scripts/kconfig/tests/preprocess/variable/__init__.py
new file mode 100644
index 0000000..e88b170
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/variable/__init__.py
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+"""
+Variable and user-defined function tests.
+"""
+
+def test(conf):
+    assert conf.oldaskconfig() == 0
+    assert conf.stderr_matches('expected_stderr')
diff --git a/scripts/kconfig/tests/preprocess/variable/expected_stderr b/scripts/kconfig/tests/preprocess/variable/expected_stderr
new file mode 100644
index 0000000..a4841c3
--- /dev/null
+++ b/scripts/kconfig/tests/preprocess/variable/expected_stderr
@@ -0,0 +1,9 @@
+Kconfig:10: SIMPLE = 1
+Kconfig:16: RECURSIVE = 2
+Kconfig:22: SIMPLE = 1 3
+Kconfig:28: RECURSIVE = 2 4
+Kconfig:35: UNDEFINED_VARIABLE = 4
+Kconfig:41: AB = 5
+Kconfig:45: Hello, my name is John.
+Kconfig:50: Hello, my name is .
+Kconfig:53: Hello, my name is John.
-- 
2.7.4

[PATCH v5 21/31] kconfig: show compiler version text in the top comment

[Masahiro Yamada]

The kernel configuration phase is now tightly coupled with the compiler
in use.  It will be nice to show the compiler information in Kconfig.

The compiler information will be displayed like this:

  $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- config
  scripts/kconfig/conf  --oldaskconfig Kconfig
  *
  * Linux/arm64 4.16.0-rc1 Kernel Configuration
  *
  *
  * Compiler: aarch64-linux-gnu-gcc (Linaro GCC 7.2-2017.11) 7.2.1 20171011
  *
  *
  * General setup
  *
  Compile also drivers which will not load (COMPILE_TEST) [N/y/?]

If you use GUI methods such as menuconfig, it will be displayed in the
top menu.

This is simply implemented by using the 'comment' statement.  So, it
will be saved into the .config file as well.

This commit has a very important meaning.  If the compiler is upgraded,
Kconfig must be re-run since different compilers have different sets
of supported options.

All referenced environments are written to include/config/auto.conf.cmd
so that any environment change triggers syncconfig, and prompt the user
to input new values if needed.

With this commit, something like follows will be added to
include/config/auto.conf.cmd

  ifneq "$(CC_VERSION_TEXT)" "aarch64-linux-gnu-gcc (Linaro GCC 7.2-2017.11) 7.2.1 20171011"
  include/config/auto.conf: FORCE
  endif

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---

Changes in v5: None
Changes in v4:
  - Add comment to arch/x86/um/Kconfig as well

Changes in v3: None
Changes in v2: None

 Kconfig             | 2 ++
 Makefile            | 2 ++
 arch/x86/um/Kconfig | 2 ++
 3 files changed, 6 insertions(+)

diff --git a/Kconfig b/Kconfig
index 4af1b42..5b55d87 100644
--- a/Kconfig
+++ b/Kconfig
@@ -5,4 +5,6 @@
 #
 mainmenu "Linux/$(ARCH) $(KERNELVERSION) Kernel Configuration"
 
+comment "Compiler: $(CC_VERSION_TEXT)"
+
 source "arch/$(SRCARCH)/Kconfig"
diff --git a/Makefile b/Makefile
index 58afa07..defb383 100644
--- a/Makefile
+++ b/Makefile
@@ -442,6 +442,8 @@ export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_LDFLAGS_MODULE
 export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL
 export KBUILD_ARFLAGS
 
+export CC_VERSION_TEXT := $(shell $(CC) --version | head -n 1)
+
 # When compiling out-of-tree modules, put MODVERDIR in the module
 # tree rather than in the kernel tree. The kernel tree might
 # even be read-only.
diff --git a/arch/x86/um/Kconfig b/arch/x86/um/Kconfig
index 6a15c4d..a992f8e 100644
--- a/arch/x86/um/Kconfig
+++ b/arch/x86/um/Kconfig
@@ -1,6 +1,8 @@
 # SPDX-License-Identifier: GPL-2.0
 mainmenu "User Mode Linux/$(SUBARCH) $(KERNELVERSION) Kernel Configuration"
 
+comment "Compiler: $(CC_VERSION_TEXT)"
+
 source "arch/um/Kconfig.common"
 
 menu "UML-specific options"
-- 
2.7.4

[PATCH v5 22/31] kconfig: add basic helper macros to scripts/Kconfig.include

[Masahiro Yamada]

Kconfig got text processing tools like we see in Make.  Add Kconfig
helper macros to scripts/Kconfig.include like we collect Makefile
macros in scripts/Kbuild.include.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
---

Changes in v5:
  - Add dollar, right_paren, left_paren variables

Changes in v4:
  - source scripts/Kconfig.include
    from arch/x86/um/Kconfig since UML is special
  - Use { $(1); } for 'success' macro for better performance
  - Use -E instead of -c for 'cc-option' macro for better performance
  - Add 'if-success' macro

Changes in v3:
  - Move helpers to scripts/Kconfig.include

Changes in v2: None

 Kconfig                 |  2 ++
 MAINTAINERS             |  1 +
 arch/x86/um/Kconfig     |  2 ++
 scripts/Kconfig.include | 27 +++++++++++++++++++++++++++
 4 files changed, 32 insertions(+)
 create mode 100644 scripts/Kconfig.include

diff --git a/Kconfig b/Kconfig
index 5b55d87..a90d9f9 100644
--- a/Kconfig
+++ b/Kconfig
@@ -7,4 +7,6 @@ mainmenu "Linux/$(ARCH) $(KERNELVERSION) Kernel Configuration"
 
 comment "Compiler: $(CC_VERSION_TEXT)"
 
+source "scripts/Kconfig.include"
+
 source "arch/$(SRCARCH)/Kconfig"
diff --git a/MAINTAINERS b/MAINTAINERS
index b87723a..79decb1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7639,6 +7639,7 @@ L:	linux-kbuild@vger.kernel.org
 S:	Maintained
 F:	Documentation/kbuild/kconfig*
 F:	scripts/kconfig/
+F:	scripts/Kconfig.include
 
 KDUMP
 M:	Dave Young <dyoung@redhat.com>
diff --git a/arch/x86/um/Kconfig b/arch/x86/um/Kconfig
index a992f8e..9d529f2 100644
--- a/arch/x86/um/Kconfig
+++ b/arch/x86/um/Kconfig
@@ -3,6 +3,8 @@ mainmenu "User Mode Linux/$(SUBARCH) $(KERNELVERSION) Kernel Configuration"
 
 comment "Compiler: $(CC_VERSION_TEXT)"
 
+source "scripts/Kconfig.include"
+
 source "arch/um/Kconfig.common"
 
 menu "UML-specific options"
diff --git a/scripts/Kconfig.include b/scripts/Kconfig.include
new file mode 100644
index 0000000..bf7c0c9
--- /dev/null
+++ b/scripts/Kconfig.include
@@ -0,0 +1,27 @@
+# Kconfig helper macros
+
+# Convenient variables
+comma       := ,
+quote       := "
+squote      := '
+empty       :=
+space       := $(empty) $(empty)
+dollar      := $
+right_paren := )
+left_paren  := (
+
+# $(if-success,<command>,<then>,<else>)
+# Return <then> if <command> exits with 0, <else> otherwise.
+if-success = $(shell,{ $(1); } >/dev/null 2>&1 && echo "$(2)" || echo "$(3)")
+
+# $(success,<command>)
+# Return y if <command> exits with 0, n otherwise
+success = $(if-success,$(1),y,n)
+
+# $(cc-option,<flag>)
+# Return y if the compiler supports <flag>, n otherwise
+cc-option = $(success,$(CC) -Werror $(1) -E -x c /dev/null -o /dev/null)
+
+# $(ld-option,<flag>)
+# Return y if the linker supports <flag>, n otherwise
+ld-option = $(success,$(LD) -v $(1))
-- 
2.7.4

[PATCH v5 23/31] stack-protector: test compiler capability in Kconfig and drop AUTO mode

[Masahiro Yamada]

Move the test for -fstack-protector(-strong) option to Kconfig.

If the compiler does not support the option, the corresponding menu
is automatically hidden.  If STRONG is not supported, it will fall
back to REGULAR.  If REGULAR is not supported, it will be disabled.
This means, AUTO is implicitly handled by the dependency solver of
Kconfig, hence removed.

I also turned the 'choice' into only two boolean symbols.  The use of
'choice' is not a good idea here, because all of all{yes,mod,no}config
would choose the first visible value, while we want allnoconfig to
disable as many features as possible.

X86 has additional shell scripts in case the compiler supports those
options, but generates broken code.  I added CC_HAS_SANE_STACKPROTECTOR
to test this.  I had to add -m32 to gcc-x86_32-has-stack-protector.sh
to make it work correctly.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Kees Cook <keescook@chromium.org>
---

Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None

 Makefile                                  | 93 ++-----------------------------
 arch/Kconfig                              | 32 ++++-------
 arch/x86/Kconfig                          | 11 +++-
 scripts/gcc-x86_32-has-stack-protector.sh |  7 +--
 scripts/gcc-x86_64-has-stack-protector.sh |  5 --
 5 files changed, 28 insertions(+), 120 deletions(-)

diff --git a/Makefile b/Makefile
index defb383..80e0800 100644
--- a/Makefile
+++ b/Makefile
@@ -675,55 +675,11 @@ ifneq ($(CONFIG_FRAME_WARN),0)
 KBUILD_CFLAGS += $(call cc-option,-Wframe-larger-than=${CONFIG_FRAME_WARN})
 endif
 
-# This selects the stack protector compiler flag. Testing it is delayed
-# until after .config has been reprocessed, in the prepare-compiler-check
-# target.
-ifdef CONFIG_CC_STACKPROTECTOR_AUTO
-  stackp-flag := $(call cc-option,-fstack-protector-strong,$(call cc-option,-fstack-protector))
-  stackp-name := AUTO
-else
-ifdef CONFIG_CC_STACKPROTECTOR_REGULAR
-  stackp-flag := -fstack-protector
-  stackp-name := REGULAR
-else
-ifdef CONFIG_CC_STACKPROTECTOR_STRONG
-  stackp-flag := -fstack-protector-strong
-  stackp-name := STRONG
-else
-  # If either there is no stack protector for this architecture or
-  # CONFIG_CC_STACKPROTECTOR_NONE is selected, we're done, and $(stackp-name)
-  # is empty, skipping all remaining stack protector tests.
-  #
-  # Force off for distro compilers that enable stack protector by default.
-  KBUILD_CFLAGS += $(call cc-option, -fno-stack-protector)
-endif
-endif
-endif
-# Find arch-specific stack protector compiler sanity-checking script.
-ifdef stackp-name
-ifneq ($(stackp-flag),)
-  stackp-path := $(srctree)/scripts/gcc-$(SRCARCH)_$(BITS)-has-stack-protector.sh
-  stackp-check := $(wildcard $(stackp-path))
-  # If the wildcard test matches a test script, run it to check functionality.
-  ifdef stackp-check
-    ifneq ($(shell $(CONFIG_SHELL) $(stackp-check) $(CC) $(KBUILD_CPPFLAGS) $(biarch)),y)
-      stackp-broken := y
-    endif
-  endif
-  ifndef stackp-broken
-    # If the stack protector is functional, enable code that depends on it.
-    KBUILD_CPPFLAGS += -DCONFIG_CC_STACKPROTECTOR
-    # Either we've already detected the flag (for AUTO) or we'll fail the
-    # build in the prepare-compiler-check rule (for specific flag).
-    KBUILD_CFLAGS += $(stackp-flag)
-  else
-    # We have to make sure stack protector is unconditionally disabled if
-    # the compiler is broken (in case we're going to continue the build in
-    # AUTO mode).
-    KBUILD_CFLAGS += $(call cc-option, -fno-stack-protector)
-  endif
-endif
-endif
+stackp-flags-$(CONFIG_CC_HAS_STACKPROTECTOR_NONE) := -fno-stack-protector
+stackp-flags-$(CONFIG_CC_STACKPROTECTOR)          := -fstack-protector
+stackp-flags-$(CONFIG_CC_STACKPROTECTOR_STRONG)   := -fstack-protector-strong
+
+KBUILD_CFLAGS += $(stackp-flags-y)
 
 ifeq ($(cc-name),clang)
 KBUILD_CPPFLAGS += $(call cc-option,-Qunused-arguments,)
@@ -1102,7 +1058,7 @@ endif
 # prepare2 creates a makefile if using a separate output directory.
 # From this point forward, .config has been reprocessed, so any rules
 # that need to depend on updated CONFIG_* values can be checked here.
-prepare2: prepare3 prepare-compiler-check outputmakefile asm-generic
+prepare2: prepare3 outputmakefile asm-generic
 
 prepare1: prepare2 $(version_h) $(autoksyms_h) include/generated/utsrelease.h \
                    include/config/auto.conf
@@ -1128,43 +1084,6 @@ uapi-asm-generic:
 PHONY += prepare-objtool
 prepare-objtool: $(objtool_target)
 
-# Check for CONFIG flags that require compiler support. Abort the build
-# after .config has been processed, but before the kernel build starts.
-#
-# For security-sensitive CONFIG options, we don't want to fallback and/or
-# silently change which compiler flags will be used, since that leads to
-# producing kernels with different security feature characteristics
-# depending on the compiler used. (For example, "But I selected
-# CC_STACKPROTECTOR_STRONG! Why did it build with _REGULAR?!")
-PHONY += prepare-compiler-check
-prepare-compiler-check: FORCE
-# Make sure compiler supports requested stack protector flag.
-ifdef stackp-name
-  # Warn about CONFIG_CC_STACKPROTECTOR_AUTO having found no option.
-  ifeq ($(stackp-flag),)
-	@echo CONFIG_CC_STACKPROTECTOR_$(stackp-name): \
-		  Compiler does not support any known stack-protector >&2
-  else
-  # Fail if specifically requested stack protector is missing.
-  ifeq ($(call cc-option, $(stackp-flag)),)
-	@echo Cannot use CONFIG_CC_STACKPROTECTOR_$(stackp-name): \
-		  $(stackp-flag) not supported by compiler >&2 && exit 1
-  endif
-  endif
-endif
-# Make sure compiler does not have buggy stack-protector support. If a
-# specific stack-protector was requested, fail the build, otherwise warn.
-ifdef stackp-broken
-  ifeq ($(stackp-name),AUTO)
-	@echo CONFIG_CC_STACKPROTECTOR_$(stackp-name): \
-                  $(stackp-flag) available but compiler is broken: disabling >&2
-  else
-	@echo Cannot use CONFIG_CC_STACKPROTECTOR_$(stackp-name): \
-                  $(stackp-flag) available but compiler is broken >&2 && exit 1
-  endif
-endif
-	@:
-
 # Generate some files
 # ---------------------------------------------------------------------------
 
diff --git a/arch/Kconfig b/arch/Kconfig
index 75dd23a..1556ef4 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -539,13 +539,16 @@ config HAVE_CC_STACKPROTECTOR
 	bool
 	help
 	  An arch should select this symbol if:
-	  - its compiler supports the -fstack-protector option
 	  - it has implemented a stack canary (e.g. __stack_chk_guard)
 
-choice
-	prompt "Stack Protector buffer overflow detection"
+config CC_HAS_STACKPROTECTOR_NONE
+	def_bool $(cc-option,-fno-stack-protector)
+
+config CC_STACKPROTECTOR
+	bool "Stack Protector buffer overflow detection"
 	depends on HAVE_CC_STACKPROTECTOR
-	default CC_STACKPROTECTOR_AUTO
+	depends on $(cc-option,-fstack-protector)
+	default y
 	help
 	  This option turns on the "stack-protector" GCC feature. This
 	  feature puts, at the beginning of functions, a canary value on
@@ -555,14 +558,6 @@ choice
 	  overwrite the canary, which gets detected and the attack is then
 	  neutralized via a kernel panic.
 
-config CC_STACKPROTECTOR_NONE
-	bool "None"
-	help
-	  Disable "stack-protector" GCC feature.
-
-config CC_STACKPROTECTOR_REGULAR
-	bool "Regular"
-	help
 	  Functions will have the stack-protector canary logic added if they
 	  have an 8-byte or larger character array on the stack.
 
@@ -574,7 +569,10 @@ config CC_STACKPROTECTOR_REGULAR
 	  by about 0.3%.
 
 config CC_STACKPROTECTOR_STRONG
-	bool "Strong"
+	bool "Strong Stack Protector"
+	depends on CC_STACKPROTECTOR
+	depends on $(cc-option,-fstack-protector-strong)
+	default y
 	help
 	  Functions will have the stack-protector canary logic added in any
 	  of the following conditions:
@@ -592,14 +590,6 @@ config CC_STACKPROTECTOR_STRONG
 	  about 20% of all kernel functions, which increases the kernel code
 	  size by about 2%.
 
-config CC_STACKPROTECTOR_AUTO
-	bool "Automatic"
-	help
-	  If the compiler supports it, the best available stack-protector
-	  option will be chosen.
-
-endchoice
-
 config LD_DEAD_CODE_DATA_ELIMINATION
 	bool
 	help
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 2236505..9ab335f 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -126,7 +126,7 @@ config X86
 	select HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD if X86_64
 	select HAVE_ARCH_VMAP_STACK		if X86_64
 	select HAVE_ARCH_WITHIN_STACK_FRAMES
-	select HAVE_CC_STACKPROTECTOR
+	select HAVE_CC_STACKPROTECTOR		if CC_HAS_SANE_STACKPROTECTOR
 	select HAVE_CMPXCHG_DOUBLE
 	select HAVE_CMPXCHG_LOCAL
 	select HAVE_CONTEXT_TRACKING		if X86_64
@@ -345,6 +345,15 @@ config PGTABLE_LEVELS
 	default 2
 
 source "init/Kconfig"
+
+config CC_HAS_SANE_STACKPROTECTOR
+	bool
+	default $(success,$(srctree)/scripts/gcc-x86_64-has-stack-protector.sh $(CC)) if 64BIT
+	default $(success,$(srctree)/scripts/gcc-x86_32-has-stack-protector.sh $(CC))
+	help
+	   We have to make sure stack protector is unconditionally disabled if
+	   the compiler produces broken code.
+
 source "kernel/Kconfig.freezer"
 
 menu "Processor type and features"
diff --git a/scripts/gcc-x86_32-has-stack-protector.sh b/scripts/gcc-x86_32-has-stack-protector.sh
index 6b2aeef..f5c1194 100755
--- a/scripts/gcc-x86_32-has-stack-protector.sh
+++ b/scripts/gcc-x86_32-has-stack-protector.sh
@@ -1,9 +1,4 @@
 #!/bin/sh
 # SPDX-License-Identifier: GPL-2.0
 
-echo "int foo(void) { char X[200]; return 3; }" | $* -S -x c -c -O0 -fstack-protector - -o - 2> /dev/null | grep -q "%gs"
-if [ "$?" -eq "0" ] ; then
-	echo y
-else
-	echo n
-fi
+echo "int foo(void) { char X[200]; return 3; }" | $* -S -x c -c -m32 -O0 -fstack-protector - -o - 2> /dev/null | grep -q "%gs"
diff --git a/scripts/gcc-x86_64-has-stack-protector.sh b/scripts/gcc-x86_64-has-stack-protector.sh
index 4a48bdc..3755af0 100755
--- a/scripts/gcc-x86_64-has-stack-protector.sh
+++ b/scripts/gcc-x86_64-has-stack-protector.sh
@@ -2,8 +2,3 @@
 # SPDX-License-Identifier: GPL-2.0
 
 echo "int foo(void) { char X[200]; return 3; }" | $* -S -x c -c -O0 -mcmodel=kernel -fno-PIE -fstack-protector - -o - 2> /dev/null | grep -q "%gs"
-if [ "$?" -eq "0" ] ; then
-	echo y
-else
-	echo n
-fi
-- 
2.7.4

[PATCH v5 24/31] kconfig: add CC_IS_GCC and GCC_VERSION

[Masahiro Yamada]

This will be useful to specify the required compiler version,
like this:

config FOO
        bool "Use Foo"
        depends on GCC_VERSION >= 40800
        help
          This feature requires GCC 4.8 or newer.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---

Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None

 init/Kconfig | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/init/Kconfig b/init/Kconfig
index f1b0cfb..2e33d93 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -8,6 +8,14 @@ config DEFCONFIG_LIST
 	default ARCH_DEFCONFIG
 	default "arch/$(ARCH)/defconfig"
 
+config CC_IS_GCC
+	def_bool $(success,$(CC) --version | grep -q gcc)
+
+config GCC_VERSION
+	int
+	default $(shell,$(srctree)/scripts/gcc-version.sh -p $(CC) | sed 's/^0*//') if CC_IS_GCC
+	default 0
+
 config CONSTRUCTORS
 	bool
 	depends on !UML
-- 
2.7.4

[PATCH v5 25/31] kconfig: add CC_IS_CLANG and CLANG_VERSION

[Masahiro Yamada]

This will be useful to describe the clang version dependency.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---

Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None

 init/Kconfig             |  7 +++++++
 scripts/clang-version.sh | 18 ++++--------------
 2 files changed, 11 insertions(+), 14 deletions(-)

diff --git a/init/Kconfig b/init/Kconfig
index 2e33d93..22e095d 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -16,6 +16,13 @@ config GCC_VERSION
 	default $(shell,$(srctree)/scripts/gcc-version.sh -p $(CC) | sed 's/^0*//') if CC_IS_GCC
 	default 0
 
+config CC_IS_CLANG
+	def_bool $(success,$(CC) --version | grep -q clang)
+
+config CLANG_VERSION
+	int
+	default $(shell,$(srctree)/scripts/clang-version.sh $(CC))
+
 config CONSTRUCTORS
 	bool
 	depends on !UML
diff --git a/scripts/clang-version.sh b/scripts/clang-version.sh
index 9780efa..dbf0a31 100755
--- a/scripts/clang-version.sh
+++ b/scripts/clang-version.sh
@@ -10,24 +10,14 @@
 # clang-5.0.1 etc.
 #
 
-if [ "$1" = "-p" ] ; then
-	with_patchlevel=1;
-	shift;
-fi
-
 compiler="$*"
 
-if [ ${#compiler} -eq 0 ]; then
-	echo "Error: No compiler specified."
-	printf "Usage:\n\t$0 <clang-command>\n"
+if !( $compiler --version | grep -q clang) ; then
+	echo 0
 	exit 1
 fi
 
 MAJOR=$(echo __clang_major__ | $compiler -E -x c - | tail -n 1)
 MINOR=$(echo __clang_minor__ | $compiler -E -x c - | tail -n 1)
-if [ "x$with_patchlevel" != "x" ] ; then
-	PATCHLEVEL=$(echo __clang_patchlevel__ | $compiler -E -x c - | tail -n 1)
-	printf "%02d%02d%02d\\n" $MAJOR $MINOR $PATCHLEVEL
-else
-	printf "%02d%02d\\n" $MAJOR $MINOR
-fi
+PATCHLEVEL=$(echo __clang_patchlevel__ | $compiler -E -x c - | tail -n 1)
+printf "%d%02d%02d\\n" $MAJOR $MINOR $PATCHLEVEL
-- 
2.7.4

[PATCH v5 26/31] gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT

[Masahiro Yamada]

CONFIG_GCOV_FORMAT_AUTODETECT compiles either gcc_3_4.c or gcc_4_7.c
according to your GCC version.

We can achieve the equivalent behavior by setting reasonable dependency
with the knowledge of the compiler version.

If GCC older than 4.7 is used, GCOV_FORMAT_3_4 is the default, but users
are still allowed to select GCOV_FORMAT_4_7 in case the newer format is
back-ported.

On the other hand, If GCC 4.7 or newer is used, there is no reason to
use GCOV_FORMAT_3_4, so it should be hidden.

If you downgrade the compiler to GCC 4.7 or older, oldconfig/syncconfig
will display a prompt for the choice because GCOV_FORMAT_3_4 becomes
visible as a new symbol.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Peter Oberparleiter <oberpar@linux.vnet.ibm.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---

Changes in v5: None
Changes in v4: None
Changes in v3: None
Changes in v2: None

 kernel/gcov/Kconfig  | 17 +++++------------
 kernel/gcov/Makefile |  2 --
 2 files changed, 5 insertions(+), 14 deletions(-)

diff --git a/kernel/gcov/Kconfig b/kernel/gcov/Kconfig
index 1276aab..1e3823f 100644
--- a/kernel/gcov/Kconfig
+++ b/kernel/gcov/Kconfig
@@ -53,23 +53,16 @@ config GCOV_PROFILE_ALL
 choice
 	prompt "Specify GCOV format"
 	depends on GCOV_KERNEL
-	default GCOV_FORMAT_AUTODETECT
 	---help---
-	The gcov format is usually determined by the GCC version, but there are
+	The gcov format is usually determined by the GCC version, and the
+	default is chosen according to your GCC version. However, there are
 	exceptions where format changes are integrated in lower-version GCCs.
-	In such a case use this option to adjust the format used in the kernel
-	accordingly.
-
-	If unsure, choose "Autodetect".
-
-config GCOV_FORMAT_AUTODETECT
-	bool "Autodetect"
-	---help---
-	Select this option to use the format that corresponds to your GCC
-	version.
+	In such a case, change this option to adjust the format used in the
+	kernel accordingly.
 
 config GCOV_FORMAT_3_4
 	bool "GCC 3.4 format"
+	depends on CC_IS_GCC && GCC_VERSION < 40700
 	---help---
 	Select this option to use the format defined by GCC 3.4.
 
diff --git a/kernel/gcov/Makefile b/kernel/gcov/Makefile
index c6c50e5..ff06d64 100644
--- a/kernel/gcov/Makefile
+++ b/kernel/gcov/Makefile
@@ -4,5 +4,3 @@ ccflags-y := -DSRCTREE='"$(srctree)"' -DOBJTREE='"$(objtree)"'
 obj-y := base.o fs.o
 obj-$(CONFIG_GCOV_FORMAT_3_4) += gcc_3_4.o
 obj-$(CONFIG_GCOV_FORMAT_4_7) += gcc_4_7.o
-obj-$(CONFIG_GCOV_FORMAT_AUTODETECT) += $(call cc-ifversion, -lt, 0407, \
-							gcc_3_4.o, gcc_4_7.o)
-- 
2.7.4

[PATCH v5 27/31] kcov: test compiler capability in Kconfig and correct dependency

[Masahiro Yamada]

As Documentation/kbuild/kconfig-language.txt notes, 'select' should be
be used with care - it forces a lower limit of another symbol, ignoring
the dependency.  Currently, KCOV can select GCC_PLUGINS even if arch
does not select HAVE_GCC_PLUGINS.  This could cause the unmet direct
dependency.

Now that Kconfig can test compiler capability, let's handle this in a
more sophisticated way.

There are two ways to enable KCOV; use the compiler that natively
supports -fsanitize-coverage=trace-pc, or build the SANCOV plugin if
the compiler has ability to build GCC plugins.  Hence, the correct
dependency for KCOV is:

  depends on CC_HAS_SANCOV_TRACE_PC || GCC_PLUGINS

You do not need to build the SANCOV plugin if the compiler already
supports -fsanitize-coverage=trace-pc.  Hence, the select should be:

  select GCC_PLUGIN_SANCOV if !CC_HAS_SANCOV_TRACE_PC

With this, GCC_PLUGIN_SANCOV is selected only when necessary, so
scripts/Makefile.gcc-plugins can be cleaner.

I also cleaned up Kconfig and scripts/Makefile.kcov as well.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---

Changes in v5: None
Changes in v4: None
Changes in v3:
  - Replace the previous 'select -> imply' patch with
    a new approach

Changes in v2:
  - Drop depends on GCC_VERSION

 Makefile                     |  2 +-
 lib/Kconfig.debug            | 11 +++++++----
 scripts/Makefile.gcc-plugins |  6 +-----
 scripts/Makefile.kcov        | 10 ++++++----
 4 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/Makefile b/Makefile
index 80e0800..4b5a17a 100644
--- a/Makefile
+++ b/Makefile
@@ -626,7 +626,7 @@ all: vmlinux
 CFLAGS_GCOV	:= -fprofile-arcs -ftest-coverage \
 	$(call cc-option,-fno-tree-loop-im) \
 	$(call cc-disable-warning,maybe-uninitialized,)
-export CFLAGS_GCOV CFLAGS_KCOV
+export CFLAGS_GCOV
 
 # The arch Makefile can set ARCH_{CPP,A,C}FLAGS to override the default
 # values of the respective KBUILD_* variables
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index c40c7b7..b76cab2 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -736,12 +736,15 @@ config ARCH_HAS_KCOV
 	  only for x86_64. KCOV requires testing on other archs, and most likely
 	  disabling of instrumentation for some early boot code.
 
+config CC_HAS_SANCOV_TRACE_PC
+	def_bool $(cc-option,-fsanitize-coverage=trace-pc)
+
 config KCOV
 	bool "Code coverage for fuzzing"
 	depends on ARCH_HAS_KCOV
+	depends on CC_HAS_SANCOV_TRACE_PC || GCC_PLUGINS
 	select DEBUG_FS
-	select GCC_PLUGINS if !COMPILE_TEST
-	select GCC_PLUGIN_SANCOV if !COMPILE_TEST
+	select GCC_PLUGIN_SANCOV if !CC_HAS_SANCOV_TRACE_PC
 	help
 	  KCOV exposes kernel code coverage information in a form suitable
 	  for coverage-guided fuzzing (randomized testing).
@@ -755,7 +758,7 @@ config KCOV
 config KCOV_ENABLE_COMPARISONS
 	bool "Enable comparison operands collection by KCOV"
 	depends on KCOV
-	default n
+	depends on $(cc-option,-fsanitize-coverage=trace-cmp)
 	help
 	  KCOV also exposes operands of every comparison in the instrumented
 	  code along with operand sizes and PCs of the comparison instructions.
@@ -765,7 +768,7 @@ config KCOV_ENABLE_COMPARISONS
 config KCOV_INSTRUMENT_ALL
 	bool "Instrument all code by default"
 	depends on KCOV
-	default y if KCOV
+	default y
 	help
 	  If you are doing generic system call fuzzing (like e.g. syzkaller),
 	  then you will want to instrument the whole kernel and you should
diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins
index 7f5c862..0ce3802 100644
--- a/scripts/Makefile.gcc-plugins
+++ b/scripts/Makefile.gcc-plugins
@@ -14,16 +14,12 @@ ifdef CONFIG_GCC_PLUGINS
   endif
 
   ifdef CONFIG_GCC_PLUGIN_SANCOV
-    ifeq ($(strip $(CFLAGS_KCOV)),)
       # It is needed because of the gcc-plugin.sh and gcc version checks.
       gcc-plugin-$(CONFIG_GCC_PLUGIN_SANCOV)           += sancov_plugin.so
 
-      ifneq ($(PLUGINCC),)
-        CFLAGS_KCOV := $(SANCOV_PLUGIN)
-      else
+      ifeq ($(PLUGINCC),)
         $(warning warning: cannot use CONFIG_KCOV: -fsanitize-coverage=trace-pc is not supported by compiler)
       endif
-    endif
   endif
 
   gcc-plugin-$(CONFIG_GCC_PLUGIN_STRUCTLEAK)	+= structleak_plugin.so
diff --git a/scripts/Makefile.kcov b/scripts/Makefile.kcov
index 5cc7203..d71ba73 100644
--- a/scripts/Makefile.kcov
+++ b/scripts/Makefile.kcov
@@ -1,7 +1,9 @@
 ifdef CONFIG_KCOV
-CFLAGS_KCOV	:= $(call cc-option,-fsanitize-coverage=trace-pc,)
-ifeq ($(CONFIG_KCOV_ENABLE_COMPARISONS),y)
-CFLAGS_KCOV += $(call cc-option,-fsanitize-coverage=trace-cmp,)
-endif
+
+kcov-flags-$(CONFIG_CC_HAS_SANCOV_TRACE_PC)	+= -fsanitize-coverage=trace-pc
+kcov-flags-$(CONFIG_KCOV_ENABLE_COMPARISONS)	+= -fsanitize-coverage=trace-cmp
+kcov-flags-$(CONFIG_GCC_PLUGIN_SANKOV)		+= -fplugin=$(objtree)/scripts/gcc-plugins/sancov_plugin.so
+
+export CFLAGS_KCOV := $(kcov-flags-y)
 
 endif
-- 
2.7.4

[PATCH v5 28/31] gcc-plugins: move GCC version check for PowerPC to Kconfig

[Masahiro Yamada]

For PowerPC, GCC 5.2 is the requirement for GCC plugins.  Move the
version check to Kconfig so that the GCC plugin menus will be hidden
if an older compiler is in use.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
---

Changes in v5: None
Changes in v4: None
Changes in v3:
  - Move comment to Kconfig as well

Changes in v2: None

 arch/powerpc/Kconfig         | 2 +-
 scripts/Makefile.gcc-plugins | 8 --------
 2 files changed, 1 insertion(+), 9 deletions(-)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index c32a181..17e53e6 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -195,7 +195,7 @@ config PPC
 	select HAVE_FTRACE_MCOUNT_RECORD
 	select HAVE_FUNCTION_GRAPH_TRACER
 	select HAVE_FUNCTION_TRACER
-	select HAVE_GCC_PLUGINS
+	select HAVE_GCC_PLUGINS			if GCC_VERSION >= 50200   # plugin support on gcc <= 5.1 is buggy on PPC
 	select HAVE_GENERIC_GUP
 	select HAVE_HW_BREAKPOINT		if PERF_EVENTS && (PPC_BOOK3S || PPC_8xx)
 	select HAVE_IDE
diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins
index 0ce3802..1e92353 100644
--- a/scripts/Makefile.gcc-plugins
+++ b/scripts/Makefile.gcc-plugins
@@ -53,14 +53,6 @@ gcc-plugins-check: FORCE
 ifdef CONFIG_GCC_PLUGINS
   ifeq ($(PLUGINCC),)
     ifneq ($(GCC_PLUGINS_CFLAGS),)
-      # Various gccs between 4.5 and 5.1 have bugs on powerpc due to missing
-      # header files. gcc <= 4.6 doesn't work at all, gccs from 4.8 to 5.1 have
-      # issues with 64-bit targets.
-      ifeq ($(ARCH),powerpc)
-        ifeq ($(call cc-ifversion, -le, 0501, y), y)
-	  @echo "Cannot use CONFIG_GCC_PLUGINS: plugin support on gcc <= 5.1 is buggy on powerpc, please upgrade to gcc 5.2 or newer" >&2 && exit 1
-        endif
-      endif
       ifeq ($(call cc-ifversion, -ge, 0405, y), y)
 	$(Q)$(srctree)/scripts/gcc-plugin.sh --show-error "$(__PLUGINCC)" "$(HOSTCXX)" "$(CC)" || true
 	@echo "Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?" >&2 && exit 1
-- 
2.7.4

[PATCH v5 29/31] gcc-plugins: test plugin support in Kconfig and clean up Makefile

[Masahiro Yamada]

Run scripts/gcc-plugin.sh from Kconfig so that users can enable
GCC_PLUGINS only when the compiler supports building plugins.

Kconfig defines a new symbol, PLUGIN_HOSTCC.  This will contain
the compiler (g++ or gcc) used for building plugins, or empty
if the plugin can not be supported at all.

This allows us to remove all ugly testing in Makefile.gcc-plugins.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Kees Cook <keescook@chromium.org>
---

Changes in v5: None
Changes in v4:
  - Use if-success macro for clean-up

Changes in v3:
  - Respin to keep scripts/gcc-plugin.sh as-is.

Changes in v2: None

 arch/Kconfig                 | 10 ++++++
 scripts/Kconfig.include      |  3 ++
 scripts/Makefile.gcc-plugins | 79 ++++++++++++--------------------------------
 scripts/gcc-plugins/Makefile |  1 +
 4 files changed, 36 insertions(+), 57 deletions(-)

diff --git a/arch/Kconfig b/arch/Kconfig
index 1556ef4..6051386 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -401,6 +401,15 @@ config SECCOMP_FILTER
 
 	  See Documentation/prctl/seccomp_filter.txt for details.
 
+preferred-plugin-hostcc := $(if-success,[ $(gcc-version) -ge 40800 ],$(HOSTCXX),$(HOSTCC))
+
+config PLUGIN_HOSTCC
+	string
+	default "$(shell,$(srctree)/scripts/gcc-plugin.sh $(preferred-plugin-hostcc) $(HOSTCXX) $(CC))"
+	help
+	  Host compiler used to build GCC plugins.  This can be $(HOSTCXX),
+	  $(HOSTCC), or a null string if GCC plugin is unsupported.
+
 config HAVE_GCC_PLUGINS
 	bool
 	help
@@ -410,6 +419,7 @@ config HAVE_GCC_PLUGINS
 menuconfig GCC_PLUGINS
 	bool "GCC plugins"
 	depends on HAVE_GCC_PLUGINS
+	depends on PLUGIN_HOSTCC != ""
 	depends on !COMPILE_TEST
 	help
 	  GCC plugins are loadable modules that provide extra features to the
diff --git a/scripts/Kconfig.include b/scripts/Kconfig.include
index bf7c0c9..dad5583 100644
--- a/scripts/Kconfig.include
+++ b/scripts/Kconfig.include
@@ -25,3 +25,6 @@ cc-option = $(success,$(CC) -Werror $(1) -E -x c /dev/null -o /dev/null)
 # $(ld-option,<flag>)
 # Return y if the linker supports <flag>, n otherwise
 ld-option = $(success,$(LD) -v $(1))
+
+# gcc version including patch level
+gcc-version := $(shell,$(srctree)/scripts/gcc-version.sh -p $(CC) | sed 's/^0*//')
diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins
index 1e92353..da5d38d 100644
--- a/scripts/Makefile.gcc-plugins
+++ b/scripts/Makefile.gcc-plugins
@@ -1,72 +1,37 @@
 # SPDX-License-Identifier: GPL-2.0
-ifdef CONFIG_GCC_PLUGINS
-  __PLUGINCC := $(call cc-ifversion, -ge, 0408, $(HOSTCXX), $(HOSTCC))
-  PLUGINCC := $(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-plugin.sh "$(__PLUGINCC)" "$(HOSTCXX)" "$(CC)")
-
-  SANCOV_PLUGIN := -fplugin=$(objtree)/scripts/gcc-plugins/sancov_plugin.so
-
-  gcc-plugin-$(CONFIG_GCC_PLUGIN_CYC_COMPLEXITY)	+= cyc_complexity_plugin.so
+gcc-plugin-$(CONFIG_GCC_PLUGIN_CYC_COMPLEXITY)	+= cyc_complexity_plugin.so
 
-  gcc-plugin-$(CONFIG_GCC_PLUGIN_LATENT_ENTROPY)	+= latent_entropy_plugin.so
-  gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_LATENT_ENTROPY)	+= -DLATENT_ENTROPY_PLUGIN
-  ifdef CONFIG_GCC_PLUGIN_LATENT_ENTROPY
+gcc-plugin-$(CONFIG_GCC_PLUGIN_LATENT_ENTROPY)	+= latent_entropy_plugin.so
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_LATENT_ENTROPY)	+= -DLATENT_ENTROPY_PLUGIN
+ifdef CONFIG_GCC_PLUGIN_LATENT_ENTROPY
     DISABLE_LATENT_ENTROPY_PLUGIN			+= -fplugin-arg-latent_entropy_plugin-disable
-  endif
-
-  ifdef CONFIG_GCC_PLUGIN_SANCOV
-      # It is needed because of the gcc-plugin.sh and gcc version checks.
-      gcc-plugin-$(CONFIG_GCC_PLUGIN_SANCOV)           += sancov_plugin.so
-
-      ifeq ($(PLUGINCC),)
-        $(warning warning: cannot use CONFIG_KCOV: -fsanitize-coverage=trace-pc is not supported by compiler)
-      endif
-  endif
-
-  gcc-plugin-$(CONFIG_GCC_PLUGIN_STRUCTLEAK)	+= structleak_plugin.so
-  gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK_VERBOSE)	+= -fplugin-arg-structleak_plugin-verbose
-  gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK_BYREF_ALL)	+= -fplugin-arg-structleak_plugin-byref-all
-  gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK)	+= -DSTRUCTLEAK_PLUGIN
+endif
 
-  gcc-plugin-$(CONFIG_GCC_PLUGIN_RANDSTRUCT)	+= randomize_layout_plugin.so
-  gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_RANDSTRUCT)	+= -DRANDSTRUCT_PLUGIN
-  gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_RANDSTRUCT_PERFORMANCE)	+= -fplugin-arg-randomize_layout_plugin-performance-mode
+gcc-plugin-$(CONFIG_GCC_PLUGIN_SANCOV)		+= sancov_plugin.so
+gcc-plugin-$(CONFIG_GCC_PLUGIN_STRUCTLEAK)	+= structleak_plugin.so
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK_VERBOSE)	+= -fplugin-arg-structleak_plugin-verbose
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK_BYREF_ALL)	+= -fplugin-arg-structleak_plugin-byref-all
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK)	+= -DSTRUCTLEAK_PLUGIN
 
-  GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y))
+gcc-plugin-$(CONFIG_GCC_PLUGIN_RANDSTRUCT)	+= randomize_layout_plugin.so
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_RANDSTRUCT)	+= -DRANDSTRUCT_PLUGIN
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_RANDSTRUCT_PERFORMANCE)	+= -fplugin-arg-randomize_layout_plugin-performance-mode
 
-  export PLUGINCC GCC_PLUGINS_CFLAGS GCC_PLUGIN GCC_PLUGIN_SUBDIR
-  export SANCOV_PLUGIN DISABLE_LATENT_ENTROPY_PLUGIN
+GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y))
 
-  ifneq ($(PLUGINCC),)
-    # SANCOV_PLUGIN can be only in CFLAGS_KCOV because avoid duplication.
-    GCC_PLUGINS_CFLAGS := $(filter-out $(SANCOV_PLUGIN), $(GCC_PLUGINS_CFLAGS))
-  endif
+export GCC_PLUGINS_CFLAGS GCC_PLUGIN GCC_PLUGIN_SUBDIR
+export DISABLE_LATENT_ENTROPY_PLUGIN
 
-  KBUILD_CFLAGS += $(GCC_PLUGINS_CFLAGS)
-  GCC_PLUGIN := $(gcc-plugin-y)
-  GCC_PLUGIN_SUBDIR := $(gcc-plugin-subdir-y)
-endif
+# SANCOV_PLUGIN can be only in CFLAGS_KCOV because avoid duplication.
+GCC_PLUGINS_CFLAGS := $(filter-out %/sancov_plugin.so, $(GCC_PLUGINS_CFLAGS))
 
-# If plugins aren't supported, abort the build before hard-to-read compiler
-# errors start getting spewed by the main build.
-PHONY += gcc-plugins-check
-gcc-plugins-check: FORCE
-ifdef CONFIG_GCC_PLUGINS
-  ifeq ($(PLUGINCC),)
-    ifneq ($(GCC_PLUGINS_CFLAGS),)
-      ifeq ($(call cc-ifversion, -ge, 0405, y), y)
-	$(Q)$(srctree)/scripts/gcc-plugin.sh --show-error "$(__PLUGINCC)" "$(HOSTCXX)" "$(CC)" || true
-	@echo "Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?" >&2 && exit 1
-      else
-	@echo "Cannot use CONFIG_GCC_PLUGINS: your gcc version does not support plugins, you should upgrade it to at least gcc 4.5" >&2 && exit 1
-      endif
-    endif
-  endif
-endif
-	@:
+KBUILD_CFLAGS += $(GCC_PLUGINS_CFLAGS)
+GCC_PLUGIN := $(gcc-plugin-y)
+GCC_PLUGIN_SUBDIR := $(gcc-plugin-subdir-y)
 
 # Actually do the build, if requested.
 PHONY += gcc-plugins
-gcc-plugins: scripts_basic gcc-plugins-check
+gcc-plugins: scripts_basic
 ifdef CONFIG_GCC_PLUGINS
 	$(Q)$(MAKE) $(build)=scripts/gcc-plugins
 endif
diff --git a/scripts/gcc-plugins/Makefile b/scripts/gcc-plugins/Makefile
index e2ff425..c47198c 100644
--- a/scripts/gcc-plugins/Makefile
+++ b/scripts/gcc-plugins/Makefile
@@ -1,4 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0
+PLUGINCC := $(CONFIG_PLUGIN_HOSTCC:"%"=%)
 GCC_PLUGINS_DIR := $(shell $(CC) -print-file-name=plugin)
 
 ifeq ($(PLUGINCC),$(HOSTCC))
-- 
2.7.4

[PATCH v5 30/31] gcc-plugins: allow to enable GCC_PLUGINS for COMPILE_TEST

[Masahiro Yamada]

Now that the compiler's plugin support is checked in Kconfig,
all{yes,mod}config will not be bothered.

Remove 'depends on !COMPILE_TEST' for GCC_PLUGINS.

'depends on !COMPILE_TEST' for the following three are still kept:
  GCC_PLUGIN_CYC_COMPLEXITY
  GCC_PLUGIN_STRUCTLEAK_VERBOSE
  GCC_PLUGIN_RANDSTRUCT_PERFORMANCE

Kees suggested to do so because the first two are too noisy, and the
last one would reduce the compile test coverage.  I commented the
reasons in arch/Kconfig.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Kees Cook <keescook@chromium.org>
---

Changes in v5: None
Changes in v4: None
Changes in v3:
  - Add comment about reason

Changes in v2: None

 arch/Kconfig | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/arch/Kconfig b/arch/Kconfig
index 6051386..e9475d0 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -420,7 +420,6 @@ menuconfig GCC_PLUGINS
 	bool "GCC plugins"
 	depends on HAVE_GCC_PLUGINS
 	depends on PLUGIN_HOSTCC != ""
-	depends on !COMPILE_TEST
 	help
 	  GCC plugins are loadable modules that provide extra features to the
 	  compiler. They are useful for runtime instrumentation and static analysis.
@@ -430,7 +429,7 @@ menuconfig GCC_PLUGINS
 config GCC_PLUGIN_CYC_COMPLEXITY
 	bool "Compute the cyclomatic complexity of a function" if EXPERT
 	depends on GCC_PLUGINS
-	depends on !COMPILE_TEST
+	depends on !COMPILE_TEST	# too noisy
 	help
 	  The complexity M of a function's control flow graph is defined as:
 	   M = E - N + 2P
@@ -497,7 +496,7 @@ config GCC_PLUGIN_STRUCTLEAK_BYREF_ALL
 config GCC_PLUGIN_STRUCTLEAK_VERBOSE
 	bool "Report forcefully initialized variables"
 	depends on GCC_PLUGIN_STRUCTLEAK
-	depends on !COMPILE_TEST
+	depends on !COMPILE_TEST	# too noisy
 	help
 	  This option will cause a warning to be printed each time the
 	  structleak plugin finds a variable it thinks needs to be
@@ -537,7 +536,7 @@ config GCC_PLUGIN_RANDSTRUCT
 config GCC_PLUGIN_RANDSTRUCT_PERFORMANCE
 	bool "Use cacheline-aware structure randomization"
 	depends on GCC_PLUGIN_RANDSTRUCT
-	depends on !COMPILE_TEST
+	depends on !COMPILE_TEST	# do not reduce test coverage
 	help
 	  If you say Y here, the RANDSTRUCT randomization will make a
 	  best effort at restricting randomization to cacheline-sized
-- 
2.7.4

[PATCH v5 31/31] Documentation: kconfig: add recommended way to describe compiler support

[Masahiro Yamada]

It would be nice if the source code is written in the same style.
This proposes the convention for describing the compiler capability
in Kconfig.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

This is not a technical issue, but it would be nice if everybody
follows the same coding style.

Without any documemation, people may write the code in their own way.
For example, there could be some possibilities for the option naming.

  CC_HAS_
  CC_SUPPORTS_
  CC_HAVE_
  ...

IMHO, CC_HAS_ is short and enough to understand the meaning.

CC_SUPPORTS_ is also good, but I personally prefer the shorter one.

Comments are appreciated.


Changes in v5:
  - Newly added

Changes in v4: None
Changes in v3: None
Changes in v2: None

 Documentation/kbuild/kconfig-language.txt | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/Documentation/kbuild/kconfig-language.txt b/Documentation/kbuild/kconfig-language.txt
index 0e966e8..a4eb018 100644
--- a/Documentation/kbuild/kconfig-language.txt
+++ b/Documentation/kbuild/kconfig-language.txt
@@ -473,6 +473,24 @@ config option to 'y' no matter the dependencies.
 The dependencies are moved to the symbol GENERIC_IOMAP and we avoid the
 situation where select forces a symbol equals to 'y'.
 
+Adding features that need compiler support
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+There are several features that need compiler support. The recommended way
+to describe the dependency on the compiler feature is to use "depends on"
+followed by a test macro.
+
+config CC_STACKPROTECTOR
+	bool "Stack Protector buffer overflow detection"
+	depends on $(cc-option,-fstack-protector)
+	...
+
+If you need to expose a compiler capability to makefiles and/or C source files,
+CC_HAS_ is the recommended prefix for the config option.
+
+config CC_HAS_STACKPROTECTOR_NONE
+	def_bool $(cc-option,-fno-stack-protector)
+
 Build as module only
 ~~~~~~~~~~~~~~~~~~~~
 To restrict a component build to module-only, qualify its config symbol
-- 
2.7.4

Re: [PATCH v5 00/31] kconfig: move compiler capability tests to Kconfig

[Masahiro Yamada]

2018-05-28 18:21 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
>
> [Introduction]
>
> The motivation of this work is to move the compiler option tests to
> Kconfig from Makefile.  A number of kernel features require the
> compiler support.  Enabling such features blindly in Kconfig ends up
> with a lot of nasty build-time testing in Makefiles.  If a chosen
> feature turns out unsupported by the compiler, what the build system
> can do is either to disable it (silently!) or to forcibly break the
> build, despite Kconfig has let the user to enable it.  By moving the
> compiler capability tests to Kconfig, Kconfig entries will be visible
> only when they are available.
>
> [Major Changes in V5]
>
> V5 removed the complexity unnecessary for Kconfig.
>
>  - In order to make the implementation simpler, I drop the lazy
>    argument expansion.
>    In V5, parameters are expanded before passed to a function.
>    (the same behavior as v3)
>
>  - Replace 'warning', 'error' functions with 'warning-if',
>    'error-if' functions.  The new functions are effective
>    only when the condition part is 'y'.
>
>  - In V4, '$' must be followed by '$' or '('.
>    '$$' meant an escaped '$'.
>
>    V4 caused a build error for ARC architecture due to the
>    following line:
>    https://github.com/torvalds/linux/blob/v4.17-rc4/arch/arc/Kconfig#L252
>
>    Of course, we could rephrase "D$" to "D cache", but
>    I treaked the grammar a little bit.
>
>    Kconfig decided to not support single-letter variable as in "$X".
>    Nor curly braces as in "${CC}" are not supported.
>    This means variable/function references in Kconfig always start
>    with "$(".
>    We can use '$' not followed by '(' as-is.
>    We do not need the "$$" escape sequence.
>    If necessary, we can use a trick to print '$(' verbatim.  See below.
>
>  - Fix a wrong behavior of simply expanded variables.
>    V4 expanded simply expanded variables when used, but it should not.
>
>    For example,
>
>    dollar := $
>    X := 1
>    Y := $(dollar)(X)
>    $(info,$(Y))
>
>    V5 prints '$(X)', not '1'.
>
>  - Remove a space after a comma is $( ) context.
>    Actually, spaces do not matter in most cases.
>
>    You can write either
>
>       $(cc-option,-fstack-protector)
>
>    or
>
>       $(cc-option, -fstack-protector)
>
>    In fact, both style co-exist in our Makefiles too.
>    However, there are some cases where they do matter.
>
>    I remove spaces after commas for consistency.
>    If you are unsure, no-space-after-comma is always safe.
>
> [Major Changes in V4]
>
>  - In V4, I slightly change the syntax of a function call.
>    I chose grammatical consistency and simplicity.
>    Anyway, Kconfig is deviating from Make behavior already.
>
>    In v3, a function call looked like this:
>
>       $(func-name arg1,arg2,arg3)
>
>    In v4, a function is invoked like follows:
>
>       $(func-name,arg1,arg2,arg3)
>
>    The difference is that the function name and the first argument
>    are separated by a comma.
>
>  - V3 supported single-letter variable like $X.
>    V4 dropped it because we do not need multiple ways to do the
>    same thing.
>    You must always enclose a variable name like $(X).
>
>  - Support lazy argument expansion.  This is necessary for implementing
>    'if', 'and', 'or' functions as in Make.
>
>  - Add more built-in functions:
>    'if', 'error', 'filename', 'lineno'
>
>  - Error out if a recursive variable references itself eventually.
>    For example,  X = $(X)
>    ends up with a circular expansion.  It must be terminated
>    since the expansion would continue eternally.
>
>  - Update Documentation and unit-tests, accordingly.
>
> [Major Changes in V3]
>
> This version looks more like Make.
>
>   - Use = operator instead of 'macro' keyword
>     to define a user-defined function.
>
>   - 'Recursively expanded variable' is implemented as a side-effect.
>      A variable is a function with zero argument.
>
>   - Support simply expanded variable which is defined by := operator
>
>   - Support += operator.
>     Probably, this feature will be useful to accumulate compiler flags.
>     At least, Clang needs some prerequisite flags such as triplet
>     to test other compiler flags.
>
>   - Support $(info ...) and $(warning ...) built-in functions,
>     which were useful while I was debugging this.
>
>   - Add documentation
>
>   - Add unit tests
>
>   - Collect helpers to scripts/Kconfig.include
>
> [Old Versions]
>
> V4:  https://lkml.org/lkml/2018/5/17/97
> V3:  https://lkml.org/lkml/2018/4/13/37
> V2:  https://lkml.org/lkml/2018/2/16/610
> V1:  https://lkml.org/lkml/2018/2/16/610
> RFC: https://lkml.org/lkml/2018/2/8/429
>
>
> Masahiro Yamada (31):
>   kbuild: remove kbuild cache
>   kbuild: remove CONFIG_CROSS_COMPILE support
>   kconfig: reference environment variables directly and remove 'option
>     env='
>   kconfig: remove string expansion in file_lookup()
>   kconfig: remove string expansion for mainmenu after yyparse()
>   kconfig: remove sym_expand_string_value()
>   kconfig: make default prompt of mainmenu less specific
>   kconfig: add built-in function support
>   kconfig: add 'shell' built-in function
>   kconfig: replace $(UNAME_RELEASE) with function call
>   kconfig: begin PARAM state only when seeing a command keyword
>   kconfig: support user-defined function and recursively expanded
>     variable
>   kconfig: support simply expanded variable
>   kconfig: support append assignment operator
>   kconfig: expand lefthand side of assignment statement
>   kconfig: add 'info', 'warning-if', and 'error-if' built-in functions
>   kconfig: add 'filename' and 'lineno' built-in variables
>   kconfig: error out if a recursive variable references itself
>   Documentation: kconfig: document a new Kconfig macro language
>   kconfig: test: add Kconfig macro language tests
>   kconfig: show compiler version text in the top comment
>   kconfig: add basic helper macros to scripts/Kconfig.include
>   stack-protector: test compiler capability in Kconfig and drop AUTO
>     mode
>   kconfig: add CC_IS_GCC and GCC_VERSION
>   kconfig: add CC_IS_CLANG and CLANG_VERSION
>   gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT
>   kcov: test compiler capability in Kconfig and correct dependency
>   gcc-plugins: move GCC version check for PowerPC to Kconfig
>   gcc-plugins: test plugin support in Kconfig and clean up Makefile
>   gcc-plugins: allow to enable GCC_PLUGINS for COMPILE_TEST
>   Documentation: kconfig: add recommended way to describe compiler
>     support
>
>  Documentation/kbuild/kconfig-language.txt          |  26 +-
>  Documentation/kbuild/kconfig-macro-language.txt    | 242 +++++++++
>  Kconfig                                            |  10 +-
>  MAINTAINERS                                        |   3 +-
>  Makefile                                           | 105 +---
>  arch/Kconfig                                       |  49 +-
>  arch/powerpc/Kconfig                               |   2 +-
>  arch/sh/Kconfig                                    |   4 +-
>  arch/sparc/Kconfig                                 |   4 +-
>  arch/um/Kconfig.common                             |   4 -
>  arch/x86/Kconfig                                   |  15 +-
>  arch/x86/um/Kconfig                                |  10 +-
>  init/Kconfig                                       |  40 +-
>  kernel/gcov/Kconfig                                |  17 +-
>  kernel/gcov/Makefile                               |   2 -
>  lib/Kconfig.debug                                  |  11 +-
>  scripts/Kbuild.include                             | 101 +---
>  scripts/Kconfig.include                            |  30 ++
>  scripts/Makefile.gcc-plugins                       |  91 +---
>  scripts/Makefile.kcov                              |  10 +-
>  scripts/clang-version.sh                           |  18 +-
>  scripts/gcc-plugins/Makefile                       |   1 +
>  scripts/gcc-x86_32-has-stack-protector.sh          |   7 +-
>  scripts/gcc-x86_64-has-stack-protector.sh          |   5 -
>  scripts/kconfig/confdata.c                         |  33 +-
>  scripts/kconfig/kconf_id.c                         |   1 -
>  scripts/kconfig/lkc.h                              |   5 +-
>  scripts/kconfig/lkc_proto.h                        |  15 +-
>  scripts/kconfig/menu.c                             |   3 -
>  scripts/kconfig/preprocess.c                       | 572 +++++++++++++++++++++
>  scripts/kconfig/symbol.c                           | 109 ----
>  .../tests/no_write_if_dep_unmet/expected_config    |   2 +-
>  .../kconfig/tests/preprocess/builtin_func/Kconfig  |  27 +
>  .../tests/preprocess/builtin_func/__init__.py      |   9 +
>  .../tests/preprocess/builtin_func/expected_stderr  |   5 +
>  .../tests/preprocess/builtin_func/expected_stdout  |   1 +
>  .../tests/preprocess/circular_expansion/Kconfig    |   5 +
>  .../preprocess/circular_expansion/__init__.py      |  11 +
>  .../preprocess/circular_expansion/expected_stderr  |   1 +
>  scripts/kconfig/tests/preprocess/escape/Kconfig    |  44 ++
>  .../kconfig/tests/preprocess/escape/__init__.py    |   8 +
>  .../tests/preprocess/escape/expected_stderr        |  10 +
>  scripts/kconfig/tests/preprocess/variable/Kconfig  |  53 ++
>  .../kconfig/tests/preprocess/variable/__init__.py  |   8 +
>  .../tests/preprocess/variable/expected_stderr      |   9 +
>  scripts/kconfig/util.c                             |  33 +-
>  scripts/kconfig/zconf.l                            |  95 +++-
>  scripts/kconfig/zconf.y                            |  46 +-
>  48 files changed, 1345 insertions(+), 567 deletions(-)
>  create mode 100644 Documentation/kbuild/kconfig-macro-language.txt
>  create mode 100644 scripts/Kconfig.include
>  create mode 100644 scripts/kconfig/preprocess.c
>  create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/Kconfig
>  create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/__init__.py
>  create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/expected_stderr
>  create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/expected_stdout
>  create mode 100644 scripts/kconfig/tests/preprocess/circular_expansion/Kconfig
>  create mode 100644 scripts/kconfig/tests/preprocess/circular_expansion/__init__.py
>  create mode 100644 scripts/kconfig/tests/preprocess/circular_expansion/expected_stderr
>  create mode 100644 scripts/kconfig/tests/preprocess/escape/Kconfig
>  create mode 100644 scripts/kconfig/tests/preprocess/escape/__init__.py
>  create mode 100644 scripts/kconfig/tests/preprocess/escape/expected_stderr
>  create mode 100644 scripts/kconfig/tests/preprocess/variable/Kconfig
>  create mode 100644 scripts/kconfig/tests/preprocess/variable/__init__.py
>  create mode 100644 scripts/kconfig/tests/preprocess/variable/expected_stderr
>
> --
> 2.7.4
>



I pushed this series to

git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
kconfig-shell-v5


-- 
Best Regards
Masahiro Yamada

Re: [PATCH v5 00/31] kconfig: move compiler capability tests to Kconfig

[Masahiro Yamada]

2018-05-28 21:23 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
> 2018-05-28 18:21 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
>>
>> [Introduction]
>>
>> The motivation of this work is to move the compiler option tests to
>> Kconfig from Makefile.  A number of kernel features require the
>> compiler support.  Enabling such features blindly in Kconfig ends up
>> with a lot of nasty build-time testing in Makefiles.  If a chosen
>> feature turns out unsupported by the compiler, what the build system
>> can do is either to disable it (silently!) or to forcibly break the
>> build, despite Kconfig has let the user to enable it.  By moving the
>> compiler capability tests to Kconfig, Kconfig entries will be visible
>> only when they are available.
>>
>> [Major Changes in V5]
>>
>> V5 removed the complexity unnecessary for Kconfig.
>>
>>  - In order to make the implementation simpler, I drop the lazy
>>    argument expansion.
>>    In V5, parameters are expanded before passed to a function.
>>    (the same behavior as v3)
>>
>>  - Replace 'warning', 'error' functions with 'warning-if',
>>    'error-if' functions.  The new functions are effective
>>    only when the condition part is 'y'.
>>
>>  - In V4, '$' must be followed by '$' or '('.
>>    '$$' meant an escaped '$'.
>>
>>    V4 caused a build error for ARC architecture due to the
>>    following line:
>>    https://github.com/torvalds/linux/blob/v4.17-rc4/arch/arc/Kconfig#L252
>>
>>    Of course, we could rephrase "D$" to "D cache", but
>>    I treaked the grammar a little bit.
>>
>>    Kconfig decided to not support single-letter variable as in "$X".
>>    Nor curly braces as in "${CC}" are not supported.
>>    This means variable/function references in Kconfig always start
>>    with "$(".
>>    We can use '$' not followed by '(' as-is.
>>    We do not need the "$$" escape sequence.
>>    If necessary, we can use a trick to print '$(' verbatim.  See below.
>>
>>  - Fix a wrong behavior of simply expanded variables.
>>    V4 expanded simply expanded variables when used, but it should not.
>>
>>    For example,
>>
>>    dollar := $
>>    X := 1
>>    Y := $(dollar)(X)
>>    $(info,$(Y))
>>
>>    V5 prints '$(X)', not '1'.
>>
>>  - Remove a space after a comma is $( ) context.
>>    Actually, spaces do not matter in most cases.
>>
>>    You can write either
>>
>>       $(cc-option,-fstack-protector)
>>
>>    or
>>
>>       $(cc-option, -fstack-protector)
>>
>>    In fact, both style co-exist in our Makefiles too.
>>    However, there are some cases where they do matter.
>>
>>    I remove spaces after commas for consistency.
>>    If you are unsure, no-space-after-comma is always safe.
>>
>> [Major Changes in V4]
>>
>>  - In V4, I slightly change the syntax of a function call.
>>    I chose grammatical consistency and simplicity.
>>    Anyway, Kconfig is deviating from Make behavior already.
>>
>>    In v3, a function call looked like this:
>>
>>       $(func-name arg1,arg2,arg3)
>>
>>    In v4, a function is invoked like follows:
>>
>>       $(func-name,arg1,arg2,arg3)
>>
>>    The difference is that the function name and the first argument
>>    are separated by a comma.
>>
>>  - V3 supported single-letter variable like $X.
>>    V4 dropped it because we do not need multiple ways to do the
>>    same thing.
>>    You must always enclose a variable name like $(X).
>>
>>  - Support lazy argument expansion.  This is necessary for implementing
>>    'if', 'and', 'or' functions as in Make.
>>
>>  - Add more built-in functions:
>>    'if', 'error', 'filename', 'lineno'
>>
>>  - Error out if a recursive variable references itself eventually.
>>    For example,  X = $(X)
>>    ends up with a circular expansion.  It must be terminated
>>    since the expansion would continue eternally.
>>
>>  - Update Documentation and unit-tests, accordingly.
>>
>> [Major Changes in V3]
>>
>> This version looks more like Make.
>>
>>   - Use = operator instead of 'macro' keyword
>>     to define a user-defined function.
>>
>>   - 'Recursively expanded variable' is implemented as a side-effect.
>>      A variable is a function with zero argument.
>>
>>   - Support simply expanded variable which is defined by := operator
>>
>>   - Support += operator.
>>     Probably, this feature will be useful to accumulate compiler flags.
>>     At least, Clang needs some prerequisite flags such as triplet
>>     to test other compiler flags.
>>
>>   - Support $(info ...) and $(warning ...) built-in functions,
>>     which were useful while I was debugging this.
>>
>>   - Add documentation
>>
>>   - Add unit tests
>>
>>   - Collect helpers to scripts/Kconfig.include
>>
>> [Old Versions]
>>
>> V4:  https://lkml.org/lkml/2018/5/17/97
>> V3:  https://lkml.org/lkml/2018/4/13/37
>> V2:  https://lkml.org/lkml/2018/2/16/610
>> V1:  https://lkml.org/lkml/2018/2/16/610
>> RFC: https://lkml.org/lkml/2018/2/8/429
>>
>>
>> Masahiro Yamada (31):
>>   kbuild: remove kbuild cache
>>   kbuild: remove CONFIG_CROSS_COMPILE support
>>   kconfig: reference environment variables directly and remove 'option
>>     env='
>>   kconfig: remove string expansion in file_lookup()
>>   kconfig: remove string expansion for mainmenu after yyparse()
>>   kconfig: remove sym_expand_string_value()
>>   kconfig: make default prompt of mainmenu less specific
>>   kconfig: add built-in function support
>>   kconfig: add 'shell' built-in function
>>   kconfig: replace $(UNAME_RELEASE) with function call
>>   kconfig: begin PARAM state only when seeing a command keyword
>>   kconfig: support user-defined function and recursively expanded
>>     variable
>>   kconfig: support simply expanded variable
>>   kconfig: support append assignment operator
>>   kconfig: expand lefthand side of assignment statement
>>   kconfig: add 'info', 'warning-if', and 'error-if' built-in functions
>>   kconfig: add 'filename' and 'lineno' built-in variables
>>   kconfig: error out if a recursive variable references itself
>>   Documentation: kconfig: document a new Kconfig macro language
>>   kconfig: test: add Kconfig macro language tests
>>   kconfig: show compiler version text in the top comment
>>   kconfig: add basic helper macros to scripts/Kconfig.include
>>   stack-protector: test compiler capability in Kconfig and drop AUTO
>>     mode
>>   kconfig: add CC_IS_GCC and GCC_VERSION
>>   kconfig: add CC_IS_CLANG and CLANG_VERSION
>>   gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT
>>   kcov: test compiler capability in Kconfig and correct dependency
>>   gcc-plugins: move GCC version check for PowerPC to Kconfig
>>   gcc-plugins: test plugin support in Kconfig and clean up Makefile
>>   gcc-plugins: allow to enable GCC_PLUGINS for COMPILE_TEST
>>   Documentation: kconfig: add recommended way to describe compiler
>>     support
>>
>>  Documentation/kbuild/kconfig-language.txt          |  26 +-
>>  Documentation/kbuild/kconfig-macro-language.txt    | 242 +++++++++
>>  Kconfig                                            |  10 +-
>>  MAINTAINERS                                        |   3 +-
>>  Makefile                                           | 105 +---
>>  arch/Kconfig                                       |  49 +-
>>  arch/powerpc/Kconfig                               |   2 +-
>>  arch/sh/Kconfig                                    |   4 +-
>>  arch/sparc/Kconfig                                 |   4 +-
>>  arch/um/Kconfig.common                             |   4 -
>>  arch/x86/Kconfig                                   |  15 +-
>>  arch/x86/um/Kconfig                                |  10 +-
>>  init/Kconfig                                       |  40 +-
>>  kernel/gcov/Kconfig                                |  17 +-
>>  kernel/gcov/Makefile                               |   2 -
>>  lib/Kconfig.debug                                  |  11 +-
>>  scripts/Kbuild.include                             | 101 +---
>>  scripts/Kconfig.include                            |  30 ++
>>  scripts/Makefile.gcc-plugins                       |  91 +---
>>  scripts/Makefile.kcov                              |  10 +-
>>  scripts/clang-version.sh                           |  18 +-
>>  scripts/gcc-plugins/Makefile                       |   1 +
>>  scripts/gcc-x86_32-has-stack-protector.sh          |   7 +-
>>  scripts/gcc-x86_64-has-stack-protector.sh          |   5 -
>>  scripts/kconfig/confdata.c                         |  33 +-
>>  scripts/kconfig/kconf_id.c                         |   1 -
>>  scripts/kconfig/lkc.h                              |   5 +-
>>  scripts/kconfig/lkc_proto.h                        |  15 +-
>>  scripts/kconfig/menu.c                             |   3 -
>>  scripts/kconfig/preprocess.c                       | 572 +++++++++++++++++++++
>>  scripts/kconfig/symbol.c                           | 109 ----
>>  .../tests/no_write_if_dep_unmet/expected_config    |   2 +-
>>  .../kconfig/tests/preprocess/builtin_func/Kconfig  |  27 +
>>  .../tests/preprocess/builtin_func/__init__.py      |   9 +
>>  .../tests/preprocess/builtin_func/expected_stderr  |   5 +
>>  .../tests/preprocess/builtin_func/expected_stdout  |   1 +
>>  .../tests/preprocess/circular_expansion/Kconfig    |   5 +
>>  .../preprocess/circular_expansion/__init__.py      |  11 +
>>  .../preprocess/circular_expansion/expected_stderr  |   1 +
>>  scripts/kconfig/tests/preprocess/escape/Kconfig    |  44 ++
>>  .../kconfig/tests/preprocess/escape/__init__.py    |   8 +
>>  .../tests/preprocess/escape/expected_stderr        |  10 +
>>  scripts/kconfig/tests/preprocess/variable/Kconfig  |  53 ++
>>  .../kconfig/tests/preprocess/variable/__init__.py  |   8 +
>>  .../tests/preprocess/variable/expected_stderr      |   9 +
>>  scripts/kconfig/util.c                             |  33 +-
>>  scripts/kconfig/zconf.l                            |  95 +++-
>>  scripts/kconfig/zconf.y                            |  46 +-
>>  48 files changed, 1345 insertions(+), 567 deletions(-)
>>  create mode 100644 Documentation/kbuild/kconfig-macro-language.txt
>>  create mode 100644 scripts/Kconfig.include
>>  create mode 100644 scripts/kconfig/preprocess.c
>>  create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/Kconfig
>>  create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/__init__.py
>>  create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/expected_stderr
>>  create mode 100644 scripts/kconfig/tests/preprocess/builtin_func/expected_stdout
>>  create mode 100644 scripts/kconfig/tests/preprocess/circular_expansion/Kconfig
>>  create mode 100644 scripts/kconfig/tests/preprocess/circular_expansion/__init__.py
>>  create mode 100644 scripts/kconfig/tests/preprocess/circular_expansion/expected_stderr
>>  create mode 100644 scripts/kconfig/tests/preprocess/escape/Kconfig
>>  create mode 100644 scripts/kconfig/tests/preprocess/escape/__init__.py
>>  create mode 100644 scripts/kconfig/tests/preprocess/escape/expected_stderr
>>  create mode 100644 scripts/kconfig/tests/preprocess/variable/Kconfig
>>  create mode 100644 scripts/kconfig/tests/preprocess/variable/__init__.py
>>  create mode 100644 scripts/kconfig/tests/preprocess/variable/expected_stderr
>>
>> --
>> 2.7.4
>>
>
>
>
> I pushed this series to
>
> git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> kconfig-shell-v5
>


I pushed this series to the for-next branch.
Unless a significant problem is reported,
I'd like to merge this series in the upcoming MW.
At least, I need to keep it in linux-next for a while.

Your comments and tags are still appreciated.
I will add tags (or replace patches if necessary) later,
but I want this series tested in linux-next
to expose to more developers' eyes.

It passed my local test.  O-day bot did not report anything so far.


-- 
Best Regards
Masahiro Yamada

Re: [PATCH v5 00/31] kconfig: move compiler capability tests to Kconfig

[Arnd Bergmann]

On Wed, May 30, 2018 at 2:38 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2018-05-28 21:23 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
>> 2018-05-28 18:21 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:

>
> I pushed this series to the for-next branch.
> Unless a significant problem is reported,
> I'd like to merge this series in the upcoming MW.
> At least, I need to keep it in linux-next for a while.
>
> Your comments and tags are still appreciated.
> I will add tags (or replace patches if necessary) later,
> but I want this series tested in linux-next
> to expose to more developers' eyes.
>
> It passed my local test.  O-day bot did not report anything so far.

I found a problem when using ccache, resulting in this output while building
scripts/kconfig:

$  make V=1 O=obj-tmp/ CC="ccache gcc" defconfig
make[1]: Entering directory '/git/arm-soc'
make -C /home/arnd/arm-soc/obj-tmp KBUILD_SRC=/git/arm-soc \
-f /git/arm-soc/Makefile defconfig
make[2]: Entering directory '/git/arm-soc/obj-tmp'
make -f ../scripts/Makefile.build obj=scripts/basic
  gcc -Wp,-MD,scripts/basic/.fixdep.d -Iscripts/basic -Wall
-Wmissing-prototypes -Wstrict-prototypes -O2 -fomit-frame-pointer
-std=gnu89  -o scripts/basic/fixdep ../scripts/basic/fixdep.c
rm -f .tmp_quiet_recordmcount
ln -fsn .. source
/bin/bash ../scripts/mkmakefile \
    .. . 4 17
make -f ../scripts/Makefile.build obj=scripts/kconfig defconfig
  gcc -Wp,-MD,scripts/kconfig/.conf.o.d -Iscripts/kconfig -Wall
-Wmissing-prototypes -Wstrict-prototypes -O2 -fomit-frame-pointer
-std=gnu89 -c -o scripts/kconfig/conf.o ../scripts/kconfig/conf.c
  bison -oscripts/kconfig/zconf.tab.c -t -l ../scripts/kconfig/zconf.y
  flex -oscripts/kconfig/zconf.lex.c -L ../scripts/kconfig/zconf.l
  gcc -Wp,-MD,scripts/kconfig/.zconf.tab.o.d -Iscripts/kconfig -Wall
-Wmissing-prototypes -Wstrict-prototypes -O2 -fomit-frame-pointer
-std=gnu89  -I../scripts/kconfig -c -o scripts/kconfig/zconf.tab.o
scripts/kconfig/zconf.tab.c
  gcc  -o scripts/kconfig/conf scripts/kconfig/conf.o
scripts/kconfig/zconf.tab.o
scripts/kconfig/conf  --defconfig=arch/arm/configs/multi_v7_defconfig Kconfig
ccache: invalid option -- 'r'
Usage:
    ccache [options]
    ccache compiler [compiler options]
    compiler [compiler options]          (via symbolic link)

Options:
    -c, --cleanup         delete old files and recalculate size counters
                          (normally not needed as this is done automatically)
    -C, --clear           clear the cache completely (except configuration)
    -F, --max-files=N     set maximum number of files in cache to N (use 0 for
                          no limit)
    -M, --max-size=SIZE   set maximum size of cache to SIZE (use 0 for no
                          limit); available suffixes: k, M, G, T (decimal) and
                          Ki, Mi, Gi, Ti (binary); default suffix: G
    -o, --set-config=K=V  set configuration key K to value V
    -p, --print-config    print current configuration options
    -s, --show-stats      show statistics summary
    -z, --zero-stats      zero statistics counters

    -h, --help            print this help text
    -V, --version         print version and copyright information

See also <https://ccache.samba.org>.
#
# configuration written to .config
#
make[2]: Leaving directory '/git/arm-soc/obj-tmp'
make[1]: Leaving directory '/git/arm-soc'

It appears to just keep going after this output, but clearly something is wrong.
I bisected it to bde197706614 ("gcc-plugins: test plugin support in Kconfig
and clean up Makefile").

     Arnd

Re: [PATCH v5 00/31] kconfig: move compiler capability tests to Kconfig

[Masahiro Yamada]

2018-06-01 17:31 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
> On Wed, May 30, 2018 at 2:38 AM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> 2018-05-28 21:23 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
>>> 2018-05-28 18:21 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
>
>>
>> I pushed this series to the for-next branch.
>> Unless a significant problem is reported,
>> I'd like to merge this series in the upcoming MW.
>> At least, I need to keep it in linux-next for a while.
>>
>> Your comments and tags are still appreciated.
>> I will add tags (or replace patches if necessary) later,
>> but I want this series tested in linux-next
>> to expose to more developers' eyes.
>>
>> It passed my local test.  O-day bot did not report anything so far.
>
> I found a problem when using ccache, resulting in this output while building
> scripts/kconfig:
>
> $  make V=1 O=obj-tmp/ CC="ccache gcc" defconfig
> make[1]: Entering directory '/git/arm-soc'
> make -C /home/arnd/arm-soc/obj-tmp KBUILD_SRC=/git/arm-soc \
> -f /git/arm-soc/Makefile defconfig
> make[2]: Entering directory '/git/arm-soc/obj-tmp'
> make -f ../scripts/Makefile.build obj=scripts/basic
>   gcc -Wp,-MD,scripts/basic/.fixdep.d -Iscripts/basic -Wall
> -Wmissing-prototypes -Wstrict-prototypes -O2 -fomit-frame-pointer
> -std=gnu89  -o scripts/basic/fixdep ../scripts/basic/fixdep.c
> rm -f .tmp_quiet_recordmcount
> ln -fsn .. source
> /bin/bash ../scripts/mkmakefile \
>     .. . 4 17
> make -f ../scripts/Makefile.build obj=scripts/kconfig defconfig
>   gcc -Wp,-MD,scripts/kconfig/.conf.o.d -Iscripts/kconfig -Wall
> -Wmissing-prototypes -Wstrict-prototypes -O2 -fomit-frame-pointer
> -std=gnu89 -c -o scripts/kconfig/conf.o ../scripts/kconfig/conf.c
>   bison -oscripts/kconfig/zconf.tab.c -t -l ../scripts/kconfig/zconf.y
>   flex -oscripts/kconfig/zconf.lex.c -L ../scripts/kconfig/zconf.l
>   gcc -Wp,-MD,scripts/kconfig/.zconf.tab.o.d -Iscripts/kconfig -Wall
> -Wmissing-prototypes -Wstrict-prototypes -O2 -fomit-frame-pointer
> -std=gnu89  -I../scripts/kconfig -c -o scripts/kconfig/zconf.tab.o
> scripts/kconfig/zconf.tab.c
>   gcc  -o scripts/kconfig/conf scripts/kconfig/conf.o
> scripts/kconfig/zconf.tab.o
> scripts/kconfig/conf  --defconfig=arch/arm/configs/multi_v7_defconfig Kconfig
> ccache: invalid option -- 'r'
> Usage:
>     ccache [options]
>     ccache compiler [compiler options]
>     compiler [compiler options]          (via symbolic link)
>
> Options:
>     -c, --cleanup         delete old files and recalculate size counters
>                           (normally not needed as this is done automatically)
>     -C, --clear           clear the cache completely (except configuration)
>     -F, --max-files=N     set maximum number of files in cache to N (use 0 for
>                           no limit)
>     -M, --max-size=SIZE   set maximum size of cache to SIZE (use 0 for no
>                           limit); available suffixes: k, M, G, T (decimal) and
>                           Ki, Mi, Gi, Ti (binary); default suffix: G
>     -o, --set-config=K=V  set configuration key K to value V
>     -p, --print-config    print current configuration options
>     -s, --show-stats      show statistics summary
>     -z, --zero-stats      zero statistics counters
>
>     -h, --help            print this help text
>     -V, --version         print version and copyright information
>
> See also <https://ccache.samba.org>.
> #
> # configuration written to .config
> #
> make[2]: Leaving directory '/git/arm-soc/obj-tmp'
> make[1]: Leaving directory '/git/arm-soc'
>
> It appears to just keep going after this output, but clearly something is wrong.
> I bisected it to bde197706614 ("gcc-plugins: test plugin support in Kconfig
> and clean up Makefile").


Thanks for the report!


In the original scripts/Makefile.gcc-plugins,
the parameters to scripts/gcc-plugin.sh were quoted by "..."

I need to keep quoting.
https://patchwork.kernel.org/patch/10442941/

Hmm, too much quoting is messy.
It works though...





-- 
Best Regards
Masahiro Yamada

Re: [PATCH v5 00/31] kconfig: move compiler capability tests to Kconfig

[Arnd Bergmann]

On Fri, Jun 1, 2018 at 12:29 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2018-06-01 17:31 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
>> On Wed, May 30, 2018 at 2:38 AM, Masahiro Yamada
>> <yamada.masahiro@socionext.com> wrote:
>>> 2018-05-28 21:23 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
>>>> 2018-05-28 18:21 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
>
> In the original scripts/Makefile.gcc-plugins,
> the parameters to scripts/gcc-plugin.sh were quoted by "..."
>
> I need to keep quoting.
> https://patchwork.kernel.org/patch/10442941/
>
> Hmm, too much quoting is messy.
> It works though...

Ok, confirmed it fixes the problem for me.

      Arnd

Re: [PATCH v5 24/31] kconfig: add CC_IS_GCC and GCC_VERSION

[Stefan Agner]

Hi Masahiro,

On 28.05.2018 11:22, Masahiro Yamada wrote:
> This will be useful to specify the required compiler version,
> like this:
> 
> config FOO
>         bool "Use Foo"
>         depends on GCC_VERSION >= 40800
>         help
>           This feature requires GCC 4.8 or newer.
> 

I tried using CC_IS_GCC today while using clang. It seems that it is set
to y despite I am using CC=clang.

.config looks like this after config:

...
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=40201
CONFIG_CC_IS_CLANG=y
CONFIG_CLANG_VERSION=60000
...


I am using clang 6.0.0 on Arch Linux, which seems to return a version
when using gcc-version.sh:
./scripts/gcc-version.sh clang | sed 's/^0*//'
402

I guess that should not be the case?

--
Stefan

> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> Reviewed-by: Kees Cook <keescook@chromium.org>
> ---
> 
> Changes in v5: None
> Changes in v4: None
> Changes in v3: None
> Changes in v2: None
> 
>  init/Kconfig | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/init/Kconfig b/init/Kconfig
> index f1b0cfb..2e33d93 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -8,6 +8,14 @@ config DEFCONFIG_LIST
>  	default ARCH_DEFCONFIG
>  	default "arch/$(ARCH)/defconfig"
>  
> +config CC_IS_GCC
> +	def_bool $(success,$(CC) --version | grep -q gcc)
> +
> +config GCC_VERSION
> +	int
> +	default $(shell,$(srctree)/scripts/gcc-version.sh -p $(CC) | sed
> 's/^0*//') if CC_IS_GCC
> +	default 0
> +
>  config CONSTRUCTORS
>  	bool
>  	depends on !UML

Re: [PATCH v5 24/31] kconfig: add CC_IS_GCC and GCC_VERSION

[Masahiro Yamada]

Hi Stefan

2018-06-05 6:49 GMT+09:00 Stefan Agner <stefan@agner.ch>:
> Hi Masahiro,
>
> On 28.05.2018 11:22, Masahiro Yamada wrote:
>> This will be useful to specify the required compiler version,
>> like this:
>>
>> config FOO
>>         bool "Use Foo"
>>         depends on GCC_VERSION >= 40800
>>         help
>>           This feature requires GCC 4.8 or newer.
>>
>
> I tried using CC_IS_GCC today while using clang. It seems that it is set
> to y despite I am using CC=clang.
>
> .config looks like this after config:
>
> ...
> CONFIG_CC_IS_GCC=y
> CONFIG_GCC_VERSION=40201
> CONFIG_CC_IS_CLANG=y
> CONFIG_CLANG_VERSION=60000
> ...
>
>
> I am using clang 6.0.0 on Arch Linux, which seems to return a version
> when using gcc-version.sh:
> ./scripts/gcc-version.sh clang | sed 's/^0*//'
> 402
>
> I guess that should not be the case?
>


What will 'clang --version' print on your machine?

Re: [PATCH v5 24/31] kconfig: add CC_IS_GCC and GCC_VERSION

[Stefan Agner]

On 05.06.2018 02:07, Masahiro Yamada wrote:
> Hi Stefan
> 
> 2018-06-05 6:49 GMT+09:00 Stefan Agner <stefan@agner.ch>:
>> Hi Masahiro,
>>
>> On 28.05.2018 11:22, Masahiro Yamada wrote:
>>> This will be useful to specify the required compiler version,
>>> like this:
>>>
>>> config FOO
>>>         bool "Use Foo"
>>>         depends on GCC_VERSION >= 40800
>>>         help
>>>           This feature requires GCC 4.8 or newer.
>>>
>>
>> I tried using CC_IS_GCC today while using clang. It seems that it is set
>> to y despite I am using CC=clang.
>>
>> .config looks like this after config:
>>
>> ...
>> CONFIG_CC_IS_GCC=y
>> CONFIG_GCC_VERSION=40201
>> CONFIG_CC_IS_CLANG=y
>> CONFIG_CLANG_VERSION=60000
>> ...
>>
>>
>> I am using clang 6.0.0 on Arch Linux, which seems to return a version
>> when using gcc-version.sh:
>> ./scripts/gcc-version.sh clang | sed 's/^0*//'
>> 402
>>
>> I guess that should not be the case?
>>
> 
> 
> What will 'clang --version' print on your machine?

$ clang --version
clang version 6.0.0 (tags/RELEASE_600/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir:
/home/ags/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin

I use a symlink to clang in my cross compiler toolchain, that is why
InstalledDir points to a GCC toolchain.

--
Stefan

Re: [PATCH v5 24/31] kconfig: add CC_IS_GCC and GCC_VERSION

[Masahiro Yamada]

2018-06-05 14:50 GMT+09:00 Stefan Agner <stefan@agner.ch>:
> On 05.06.2018 02:07, Masahiro Yamada wrote:
>> Hi Stefan
>>
>> 2018-06-05 6:49 GMT+09:00 Stefan Agner <stefan@agner.ch>:
>>> Hi Masahiro,
>>>
>>> On 28.05.2018 11:22, Masahiro Yamada wrote:
>>>> This will be useful to specify the required compiler version,
>>>> like this:
>>>>
>>>> config FOO
>>>>         bool "Use Foo"
>>>>         depends on GCC_VERSION >= 40800
>>>>         help
>>>>           This feature requires GCC 4.8 or newer.
>>>>
>>>
>>> I tried using CC_IS_GCC today while using clang. It seems that it is set
>>> to y despite I am using CC=clang.
>>>
>>> .config looks like this after config:
>>>
>>> ...
>>> CONFIG_CC_IS_GCC=y
>>> CONFIG_GCC_VERSION=40201
>>> CONFIG_CC_IS_CLANG=y
>>> CONFIG_CLANG_VERSION=60000
>>> ...
>>>
>>>
>>> I am using clang 6.0.0 on Arch Linux, which seems to return a version
>>> when using gcc-version.sh:
>>> ./scripts/gcc-version.sh clang | sed 's/^0*//'
>>> 402
>>>
>>> I guess that should not be the case?
>>>
>>
>>
>> What will 'clang --version' print on your machine?
>
> $ clang --version
> clang version 6.0.0 (tags/RELEASE_600/final)
> Target: x86_64-pc-linux-gnu
> Thread model: posix
> InstalledDir:
> /home/ags/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin
>
> I use a symlink to clang in my cross compiler toolchain, that is why
> InstalledDir points to a GCC toolchain.
>

Ah, I see.


I will fix it up like follows:


diff --git a/init/Kconfig b/init/Kconfig
index e5a0d89..efc43c6 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -9,7 +9,7 @@ config DEFCONFIG_LIST
        default "arch/$(ARCH)/defconfig"

 config CC_IS_GCC
-       def_bool $(success,$(CC) --version | grep -q gcc)
+       def_bool $(success,$(CC) --version | head -n 1 | grep -q gcc)

 config GCC_VERSION
        int
@@ -17,7 +17,7 @@ config GCC_VERSION
        default 0

 config CC_IS_CLANG
-       def_bool $(success,$(CC) --version | grep -q clang)
+       def_bool $(success,$(CC) --version | head -n 1 | grep -q clang)

 config CLANG_VERSION
        int




Best Regards
Masahiro Yamada

Re: [PATCH v5 24/31] kconfig: add CC_IS_GCC and GCC_VERSION

[Stefan Agner]

On 05.06.2018 08:27, Masahiro Yamada wrote:
> 2018-06-05 14:50 GMT+09:00 Stefan Agner <stefan@agner.ch>:
>> On 05.06.2018 02:07, Masahiro Yamada wrote:
>>> Hi Stefan
>>>
>>> 2018-06-05 6:49 GMT+09:00 Stefan Agner <stefan@agner.ch>:
>>>> Hi Masahiro,
>>>>
>>>> On 28.05.2018 11:22, Masahiro Yamada wrote:
>>>>> This will be useful to specify the required compiler version,
>>>>> like this:
>>>>>
>>>>> config FOO
>>>>>         bool "Use Foo"
>>>>>         depends on GCC_VERSION >= 40800
>>>>>         help
>>>>>           This feature requires GCC 4.8 or newer.
>>>>>
>>>>
>>>> I tried using CC_IS_GCC today while using clang. It seems that it is set
>>>> to y despite I am using CC=clang.
>>>>
>>>> .config looks like this after config:
>>>>
>>>> ...
>>>> CONFIG_CC_IS_GCC=y
>>>> CONFIG_GCC_VERSION=40201
>>>> CONFIG_CC_IS_CLANG=y
>>>> CONFIG_CLANG_VERSION=60000
>>>> ...
>>>>
>>>>
>>>> I am using clang 6.0.0 on Arch Linux, which seems to return a version
>>>> when using gcc-version.sh:
>>>> ./scripts/gcc-version.sh clang | sed 's/^0*//'
>>>> 402
>>>>
>>>> I guess that should not be the case?
>>>>
>>>
>>>
>>> What will 'clang --version' print on your machine?
>>
>> $ clang --version
>> clang version 6.0.0 (tags/RELEASE_600/final)
>> Target: x86_64-pc-linux-gnu
>> Thread model: posix
>> InstalledDir:
>> /home/ags/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin
>>
>> I use a symlink to clang in my cross compiler toolchain, that is why
>> InstalledDir points to a GCC toolchain.
>>
> 
> Ah, I see.
> 
> 
> I will fix it up like follows:
> 
> 
> diff --git a/init/Kconfig b/init/Kconfig
> index e5a0d89..efc43c6 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -9,7 +9,7 @@ config DEFCONFIG_LIST
>         default "arch/$(ARCH)/defconfig"
> 
>  config CC_IS_GCC
> -       def_bool $(success,$(CC) --version | grep -q gcc)
> +       def_bool $(success,$(CC) --version | head -n 1 | grep -q gcc)

Yes that works for me:

...
CONFIG_GCC_VERSION=0                                                    
                                                                   
CONFIG_CC_IS_CLANG=y                                                    
                                                                   
CONFIG_CLANG_VERSION=60000
...

--
Stefan

> 
>  config GCC_VERSION
>         int
> @@ -17,7 +17,7 @@ config GCC_VERSION
>         default 0
> 
>  config CC_IS_CLANG
> -       def_bool $(success,$(CC) --version | grep -q clang)
> +       def_bool $(success,$(CC) --version | head -n 1 | grep -q clang)
> 
>  config CLANG_VERSION
>         int
> 
> 
> 
> 
> Best Regards
> Masahiro Yamada

Re: [PATCH v5 21/31] kconfig: show compiler version text in the top comment

[Geert Uytterhoeven]

Hi Yamada-san,

On Mon, May 28, 2018 at 11:21 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> The kernel configuration phase is now tightly coupled with the compiler
> in use.  It will be nice to show the compiler information in Kconfig.
>
> The compiler information will be displayed like this:
>
>   $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- config
>   scripts/kconfig/conf  --oldaskconfig Kconfig
>   *
>   * Linux/arm64 4.16.0-rc1 Kernel Configuration
>   *
>   *
>   * Compiler: aarch64-linux-gnu-gcc (Linaro GCC 7.2-2017.11) 7.2.1 20171011
>   *
>   *
>   * General setup
>   *
>   Compile also drivers which will not load (COMPILE_TEST) [N/y/?]
>
> If you use GUI methods such as menuconfig, it will be displayed in the
> top menu.
>
> This is simply implemented by using the 'comment' statement.  So, it
> will be saved into the .config file as well.
>
> This commit has a very important meaning.  If the compiler is upgraded,
> Kconfig must be re-run since different compilers have different sets
> of supported options.
>
> All referenced environments are written to include/config/auto.conf.cmd
> so that any environment change triggers syncconfig, and prompt the user
> to input new values if needed.
>
> With this commit, something like follows will be added to
> include/config/auto.conf.cmd
>
>   ifneq "$(CC_VERSION_TEXT)" "aarch64-linux-gnu-gcc (Linaro GCC 7.2-2017.11) 7.2.1 20171011"
>   include/config/auto.conf: FORCE
>   endif
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> Reviewed-by: Kees Cook <keescook@chromium.org>

This is now commit 21c54b774744719c ("kconfig: show compiler version text
in the top comment") upstream.

This commit broke cross-compilation for me:

    $ make ARCH=m68k
    scripts/kconfig/conf  --syncconfig Kconfig
    scripts/kconfig/conf  --syncconfig Kconfig
    scripts/kconfig/conf  --syncconfig Kconfig
    scripts/kconfig/conf  --syncconfig Kconfig
    ...

with O=:

      GEN     ./Makefile
     scripts/kconfig/conf  --syncconfig Kconfig
      GEN     ./Makefile
    scripts/kconfig/conf  --syncconfig Kconfig
      GEN     ./Makefile
    scripts/kconfig/conf  --syncconfig Kconfig
      GEN     ./Makefile
    ...

.config gained (both with/without O=):

    #
    # Compiler: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
    #

which is definitely wrong when cross-compiling for m68k.

Reverting the commit, and fixing up the conflicts, fixes the issue for me

Do you have a clue?
Thanks!

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH v5 21/31] kconfig: show compiler version text in the top comment

[Masahiro Yamada]

Hi Geert,


2018-06-07 17:42 GMT+09:00 Geert Uytterhoeven <geert@linux-m68k.org>:
> Hi Yamada-san,
>
> On Mon, May 28, 2018 at 11:21 AM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> The kernel configuration phase is now tightly coupled with the compiler
>> in use.  It will be nice to show the compiler information in Kconfig.
>>
>> The compiler information will be displayed like this:
>>
>>   $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- config
>>   scripts/kconfig/conf  --oldaskconfig Kconfig
>>   *
>>   * Linux/arm64 4.16.0-rc1 Kernel Configuration
>>   *
>>   *
>>   * Compiler: aarch64-linux-gnu-gcc (Linaro GCC 7.2-2017.11) 7.2.1 20171011
>>   *
>>   *
>>   * General setup
>>   *
>>   Compile also drivers which will not load (COMPILE_TEST) [N/y/?]
>>
>> If you use GUI methods such as menuconfig, it will be displayed in the
>> top menu.
>>
>> This is simply implemented by using the 'comment' statement.  So, it
>> will be saved into the .config file as well.
>>
>> This commit has a very important meaning.  If the compiler is upgraded,
>> Kconfig must be re-run since different compilers have different sets
>> of supported options.
>>
>> All referenced environments are written to include/config/auto.conf.cmd
>> so that any environment change triggers syncconfig, and prompt the user
>> to input new values if needed.
>>
>> With this commit, something like follows will be added to
>> include/config/auto.conf.cmd
>>
>>   ifneq "$(CC_VERSION_TEXT)" "aarch64-linux-gnu-gcc (Linaro GCC 7.2-2017.11) 7.2.1 20171011"
>>   include/config/auto.conf: FORCE
>>   endif
>>
>> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
>> Reviewed-by: Kees Cook <keescook@chromium.org>
>
> This is now commit 21c54b774744719c ("kconfig: show compiler version text
> in the top comment") upstream.
>
> This commit broke cross-compilation for me:
>
>     $ make ARCH=m68k
>     scripts/kconfig/conf  --syncconfig Kconfig
>     scripts/kconfig/conf  --syncconfig Kconfig
>     scripts/kconfig/conf  --syncconfig Kconfig
>     scripts/kconfig/conf  --syncconfig Kconfig
>     ...
>
> with O=:
>
>       GEN     ./Makefile
>      scripts/kconfig/conf  --syncconfig Kconfig
>       GEN     ./Makefile
>     scripts/kconfig/conf  --syncconfig Kconfig
>       GEN     ./Makefile
>     scripts/kconfig/conf  --syncconfig Kconfig
>       GEN     ./Makefile
>     ...
>
> .config gained (both with/without O=):
>
>     #
>     # Compiler: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
>     #
>
> which is definitely wrong when cross-compiling for m68k.
>
> Reverting the commit, and fixing up the conflicts, fixes the issue for me
>
> Do you have a clue?
> Thanks!


Agh, I missed the case where CROSS_COMPILE is
set by arch/$(SRCARCH)/Makefile.


I will fix it.  Thanks for the report!






-- 
Best Regards
Masahiro Yamada

Re: [PATCH v5 21/31] kconfig: show compiler version text in the top comment

[Masahiro Yamada]

Hi Geert,


2018-06-07 17:58 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
> Hi Geert,
>
>
> 2018-06-07 17:42 GMT+09:00 Geert Uytterhoeven <geert@linux-m68k.org>:
>> Hi Yamada-san,
>>
>> On Mon, May 28, 2018 at 11:21 AM, Masahiro Yamada
>> <yamada.masahiro@socionext.com> wrote:
>>> The kernel configuration phase is now tightly coupled with the compiler
>>> in use.  It will be nice to show the compiler information in Kconfig.
>>>
>>> The compiler information will be displayed like this:
>>>
>>>   $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- config
>>>   scripts/kconfig/conf  --oldaskconfig Kconfig
>>>   *
>>>   * Linux/arm64 4.16.0-rc1 Kernel Configuration
>>>   *
>>>   *
>>>   * Compiler: aarch64-linux-gnu-gcc (Linaro GCC 7.2-2017.11) 7.2.1 20171011
>>>   *
>>>   *
>>>   * General setup
>>>   *
>>>   Compile also drivers which will not load (COMPILE_TEST) [N/y/?]
>>>
>>> If you use GUI methods such as menuconfig, it will be displayed in the
>>> top menu.
>>>
>>> This is simply implemented by using the 'comment' statement.  So, it
>>> will be saved into the .config file as well.
>>>
>>> This commit has a very important meaning.  If the compiler is upgraded,
>>> Kconfig must be re-run since different compilers have different sets
>>> of supported options.
>>>
>>> All referenced environments are written to include/config/auto.conf.cmd
>>> so that any environment change triggers syncconfig, and prompt the user
>>> to input new values if needed.
>>>
>>> With this commit, something like follows will be added to
>>> include/config/auto.conf.cmd
>>>
>>>   ifneq "$(CC_VERSION_TEXT)" "aarch64-linux-gnu-gcc (Linaro GCC 7.2-2017.11) 7.2.1 20171011"
>>>   include/config/auto.conf: FORCE
>>>   endif
>>>
>>> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
>>> Reviewed-by: Kees Cook <keescook@chromium.org>
>>
>> This is now commit 21c54b774744719c ("kconfig: show compiler version text
>> in the top comment") upstream.
>>
>> This commit broke cross-compilation for me:
>>
>>     $ make ARCH=m68k
>>     scripts/kconfig/conf  --syncconfig Kconfig
>>     scripts/kconfig/conf  --syncconfig Kconfig
>>     scripts/kconfig/conf  --syncconfig Kconfig
>>     scripts/kconfig/conf  --syncconfig Kconfig
>>     ...
>>
>> with O=:
>>
>>       GEN     ./Makefile
>>      scripts/kconfig/conf  --syncconfig Kconfig
>>       GEN     ./Makefile
>>     scripts/kconfig/conf  --syncconfig Kconfig
>>       GEN     ./Makefile
>>     scripts/kconfig/conf  --syncconfig Kconfig
>>       GEN     ./Makefile
>>     ...
>>
>> .config gained (both with/without O=):
>>
>>     #
>>     # Compiler: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
>>     #
>>
>> which is definitely wrong when cross-compiling for m68k.
>>
>> Reverting the commit, and fixing up the conflicts, fixes the issue for me
>>
>> Do you have a clue?
>> Thanks!
>
>
> Agh, I missed the case where CROSS_COMPILE is
> set by arch/$(SRCARCH)/Makefile.
>
>
> I will fix it.  Thanks for the report!


I posted a patch.
https://patchwork.kernel.org/patch/10453685/

I hope it will fix your problem.


Thanks,

-- 
Best Regards
Masahiro Yamada

Re: [PATCH v5 21/31] kconfig: show compiler version text in the top comment

[Geert Uytterhoeven]

Hi Yamada-san,

On Fri, Jun 8, 2018 at 9:04 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2018-06-07 17:58 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
>> 2018-06-07 17:42 GMT+09:00 Geert Uytterhoeven <geert@linux-m68k.org>:
>>> On Mon, May 28, 2018 at 11:21 AM, Masahiro Yamada
>>> <yamada.masahiro@socionext.com> wrote:
>>>> The kernel configuration phase is now tightly coupled with the compiler
>>>> in use.  It will be nice to show the compiler information in Kconfig.

[...]

>>>> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
>>>> Reviewed-by: Kees Cook <keescook@chromium.org>
>>>
>>> This is now commit 21c54b774744719c ("kconfig: show compiler version text
>>> in the top comment") upstream.
>>>
>>> This commit broke cross-compilation for me:
>>>
>>>     $ make ARCH=m68k
>>>     scripts/kconfig/conf  --syncconfig Kconfig
>>>     scripts/kconfig/conf  --syncconfig Kconfig
>>>     scripts/kconfig/conf  --syncconfig Kconfig
>>>     scripts/kconfig/conf  --syncconfig Kconfig
>>>     ...
>>>
>>> with O=:
>>>
>>>       GEN     ./Makefile
>>>      scripts/kconfig/conf  --syncconfig Kconfig
>>>       GEN     ./Makefile
>>>     scripts/kconfig/conf  --syncconfig Kconfig
>>>       GEN     ./Makefile
>>>     scripts/kconfig/conf  --syncconfig Kconfig
>>>       GEN     ./Makefile
>>>     ...
>>>
>>> .config gained (both with/without O=):
>>>
>>>     #
>>>     # Compiler: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
>>>     #
>>>
>>> which is definitely wrong when cross-compiling for m68k.
>>>
>>> Reverting the commit, and fixing up the conflicts, fixes the issue for me
>>>
>>> Do you have a clue?
>>> Thanks!
>>
>>
>> Agh, I missed the case where CROSS_COMPILE is
>> set by arch/$(SRCARCH)/Makefile.
>>
>>
>> I will fix it.  Thanks for the report!
>
> I posted a patch.
> https://patchwork.kernel.org/patch/10453685/
>
> I hope it will fix your problem.

Yes it has. Thanks!

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [v5, 03/31] kconfig: reference environment variables directly and remove 'option env='

[Andrei Vagin]

Hi Masahiro,

localyesconfig doesn't work and git bisect points on this patch.

[avagin@laptop linux]$ make  localyesconfig
using config: '.config'
Can't open arch/$(SRCARCH)/Kconfig at ./scripts/kconfig/streamline_config.pl line 174, <$kinfile> line 8.
make[1]: *** [scripts/kconfig/Makefile:45: localyesconfig] Error 2
make: *** [Makefile:526: localyesconfig] Error 2

On Mon, May 28, 2018 at 06:21:40PM +0900, Masahiro Yamada wrote:
> To get access to environment variables, Kconfig needs to define a
> symbol using "option env=" syntax.  It is tedious to add a symbol entry
> for each environment variable given that we need to define much more
> such as 'CC', 'AS', 'srctree' etc. to evaluate the compiler capability
> in Kconfig.
> 
> Adding '$' for symbol references is grammatically inconsistent.
> Looking at the code, the symbols prefixed with 'S' are expanded by:
>  - conf_expand_value()
>    This is used to expand 'arch/$ARCH/defconfig' and 'defconfig_list'
>  - sym_expand_string_value()
>    This is used to expand strings in 'source' and 'mainmenu'
> 
> All of them are fixed values independent of user configuration.  So,
> they can be changed into the direct expansion instead of symbols.
> 
> This change makes the code much cleaner.  The bounce symbols 'SRCARCH',
> 'ARCH', 'SUBARCH', 'KERNELVERSION' are gone.
> 
> sym_init() hard-coding 'UNAME_RELEASE' is also gone.  'UNAME_RELEASE'
> should be replaced with an environment variable.
> 
> ARCH_DEFCONFIG is a normal symbol, so it should be simply referenced
> without '$' prefix.
> 
> The new syntax is addicted by Make.  The variable reference needs
> parentheses, like $(FOO), but you can omit them for single-letter
> variables, like $F.  Yet, in Makefiles, people tend to use the
> parenthetical form for consistency / clarification.
> 
> At this moment, only the environment variable is supported, but I will
> extend the concept of 'variable' later on.
> 
> The variables are expanded in the lexer so we can simplify the token
> handling on the parser side.
> 
> For example, the following code works.
> 
> [Example code]
> 
>   config MY_TOOLCHAIN_LIST
>           string
>           default "My tools: CC=$(CC), AS=$(AS), CPP=$(CPP)"
> 
> [Result]
> 
>   $ make -s alldefconfig && tail -n 1 .config
>   CONFIG_MY_TOOLCHAIN_LIST="My tools: CC=gcc, AS=as, CPP=gcc -E"
> 
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> Reviewed-by: Kees Cook <keescook@chromium.org>
> ---
> 
> Changes in v5:
>   - More comments like "advance pointer to ..."
>   - Factor out the duplicated code into __expand_string()
>   - Move the empty name check to env_expand().
>   - Remove escape sequence of '$'
> 
> Changes in v4:
>   - Enclose ARCH in conf_defname
>   - Drop single-letter support
> 
> Changes in v3:
>   - Reimplement
>   - Variable reference need parentheses except single-letter variable
> 
> Changes in v2:
>   - Move the string expansion to the lexer phase.
>   - Split environment helpers to env.c
> 
>  Documentation/kbuild/kconfig-language.txt |   8 -
>  Kconfig                                   |   8 +-
>  Makefile                                  |   3 +-
>  arch/sh/Kconfig                           |   4 +-
>  arch/sparc/Kconfig                        |   4 +-
>  arch/um/Kconfig.common                    |   4 -
>  arch/x86/Kconfig                          |   4 +-
>  arch/x86/um/Kconfig                       |   6 +-
>  init/Kconfig                              |  16 +-
>  scripts/kconfig/confdata.c                |  33 +----
>  scripts/kconfig/kconf_id.c                |   1 -
>  scripts/kconfig/lkc.h                     |   5 +-
>  scripts/kconfig/lkc_proto.h               |   6 +
>  scripts/kconfig/menu.c                    |   3 -
>  scripts/kconfig/preprocess.c              | 238 ++++++++++++++++++++++++++++++
>  scripts/kconfig/symbol.c                  |  56 -------
>  scripts/kconfig/util.c                    |  29 ++--
>  scripts/kconfig/zconf.l                   |  67 ++++++++-
>  scripts/kconfig/zconf.y                   |   2 +-
>  19 files changed, 343 insertions(+), 154 deletions(-)
>  create mode 100644 scripts/kconfig/preprocess.c
> 
> diff --git a/Documentation/kbuild/kconfig-language.txt b/Documentation/kbuild/kconfig-language.txt
> index f5b9493..0e966e8 100644
> --- a/Documentation/kbuild/kconfig-language.txt
> +++ b/Documentation/kbuild/kconfig-language.txt
> @@ -198,14 +198,6 @@ applicable everywhere (see syntax).
>      enables the third modular state for all config symbols.
>      At most one symbol may have the "modules" option set.
>  
> -  - "env"=<value>
> -    This imports the environment variable into Kconfig. It behaves like
> -    a default, except that the value comes from the environment, this
> -    also means that the behaviour when mixing it with normal defaults is
> -    undefined at this point. The symbol is currently not exported back
> -    to the build environment (if this is desired, it can be done via
> -    another symbol).
> -
>    - "allnoconfig_y"
>      This declares the symbol as one that should have the value y when
>      using "allnoconfig". Used for symbols that hide other symbols.
> diff --git a/Kconfig b/Kconfig
> index 8c4c1cb..4af1b42 100644
> --- a/Kconfig
> +++ b/Kconfig
> @@ -3,10 +3,6 @@
>  # For a description of the syntax of this configuration file,
>  # see Documentation/kbuild/kconfig-language.txt.
>  #
> -mainmenu "Linux/$ARCH $KERNELVERSION Kernel Configuration"
> +mainmenu "Linux/$(ARCH) $(KERNELVERSION) Kernel Configuration"
>  
> -config SRCARCH
> -	string
> -	option env="SRCARCH"
> -
> -source "arch/$SRCARCH/Kconfig"
> +source "arch/$(SRCARCH)/Kconfig"
> diff --git a/Makefile b/Makefile
> index 58afa07..59b8654 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -284,7 +284,8 @@ include scripts/Kbuild.include
>  # Read KERNELRELEASE from include/config/kernel.release (if it exists)
>  KERNELRELEASE = $(shell cat include/config/kernel.release 2> /dev/null)
>  KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(SUBLEVEL)))$(EXTRAVERSION)
> -export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION
> +UNAME_RELEASE := $(shell uname --release)
> +export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION UNAME_RELEASE
>  
>  # SUBARCH tells the usermode build what the underlying arch is.  That is set
>  # first, and if a usermode build is happening, the "ARCH=um" on the command
> diff --git a/arch/sh/Kconfig b/arch/sh/Kconfig
> index 1851eae..c8400e3 100644
> --- a/arch/sh/Kconfig
> +++ b/arch/sh/Kconfig
> @@ -58,7 +58,7 @@ config SUPERH
>  	  <http://www.linux-sh.org/>.
>  
>  config SUPERH32
> -	def_bool ARCH = "sh"
> +	def_bool "$(ARCH)" = "sh"
>  	select HAVE_KPROBES
>  	select HAVE_KRETPROBES
>  	select HAVE_IOREMAP_PROT if MMU && !X2TLB
> @@ -77,7 +77,7 @@ config SUPERH32
>  	select HAVE_CC_STACKPROTECTOR
>  
>  config SUPERH64
> -	def_bool ARCH = "sh64"
> +	def_bool "$(ARCH)" = "sh64"
>  	select HAVE_EXIT_THREAD
>  	select KALLSYMS
>  
> diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
> index 8767e45..df7410c 100644
> --- a/arch/sparc/Kconfig
> +++ b/arch/sparc/Kconfig
> @@ -1,6 +1,6 @@
>  config 64BIT
> -	bool "64-bit kernel" if ARCH = "sparc"
> -	default ARCH = "sparc64"
> +	bool "64-bit kernel" if "$(ARCH)" = "sparc"
> +	default "$(ARCH)" = "sparc64"
>  	help
>  	  SPARC is a family of RISC microprocessors designed and marketed by
>  	  Sun Microsystems, incorporated.  They are very widely found in Sun
> diff --git a/arch/um/Kconfig.common b/arch/um/Kconfig.common
> index c68add8..07f84c8 100644
> --- a/arch/um/Kconfig.common
> +++ b/arch/um/Kconfig.common
> @@ -54,10 +54,6 @@ config HZ
>  	int
>  	default 100
>  
> -config SUBARCH
> -	string
> -	option env="SUBARCH"
> -
>  config NR_CPUS
>  	int
>  	range 1 1
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index c07f492..2236505 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -1,8 +1,8 @@
>  # SPDX-License-Identifier: GPL-2.0
>  # Select 32 or 64 bit
>  config 64BIT
> -	bool "64-bit kernel" if ARCH = "x86"
> -	default ARCH != "i386"
> +	bool "64-bit kernel" if "$(ARCH)" = "x86"
> +	default "$(ARCH)" != "i386"
>  	---help---
>  	  Say yes to build a 64-bit kernel - formerly known as x86_64
>  	  Say no to build a 32-bit kernel - formerly known as i386
> diff --git a/arch/x86/um/Kconfig b/arch/x86/um/Kconfig
> index 13ed827..6a15c4d 100644
> --- a/arch/x86/um/Kconfig
> +++ b/arch/x86/um/Kconfig
> @@ -1,5 +1,5 @@
>  # SPDX-License-Identifier: GPL-2.0
> -mainmenu "User Mode Linux/$SUBARCH $KERNELVERSION Kernel Configuration"
> +mainmenu "User Mode Linux/$(SUBARCH) $(KERNELVERSION) Kernel Configuration"
>  
>  source "arch/um/Kconfig.common"
>  
> @@ -16,8 +16,8 @@ config UML_X86
>  	select GENERIC_FIND_FIRST_BIT
>  
>  config 64BIT
> -	bool "64-bit kernel" if SUBARCH = "x86"
> -	default SUBARCH != "i386"
> +	bool "64-bit kernel" if "$(SUBARCH)" = "x86"
> +	default "$(SUBARCH)" != "i386"
>  
>  config X86_32
>  	def_bool !64BIT
> diff --git a/init/Kconfig b/init/Kconfig
> index 15aae32..1217fc6 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -1,20 +1,12 @@
> -config ARCH
> -	string
> -	option env="ARCH"
> -
> -config KERNELVERSION
> -	string
> -	option env="KERNELVERSION"
> -
>  config DEFCONFIG_LIST
>  	string
>  	depends on !UML
>  	option defconfig_list
> -	default "/lib/modules/$UNAME_RELEASE/.config"
> +	default "/lib/modules/$(UNAME_RELEASE)/.config"
>  	default "/etc/kernel-config"
> -	default "/boot/config-$UNAME_RELEASE"
> -	default "$ARCH_DEFCONFIG"
> -	default "arch/$ARCH/defconfig"
> +	default "/boot/config-$(UNAME_RELEASE)"
> +	default ARCH_DEFCONFIG
> +	default "arch/$(ARCH)/defconfig"
>  
>  config CONSTRUCTORS
>  	bool
> diff --git a/scripts/kconfig/confdata.c b/scripts/kconfig/confdata.c
> index df26c7b..f72587c 100644
> --- a/scripts/kconfig/confdata.c
> +++ b/scripts/kconfig/confdata.c
> @@ -30,7 +30,7 @@ static void conf_message(const char *fmt, ...)
>  static const char *conf_filename;
>  static int conf_lineno, conf_warnings;
>  
> -const char conf_defname[] = "arch/$ARCH/defconfig";
> +const char conf_defname[] = "arch/$(ARCH)/defconfig";
>  
>  static void conf_warning(const char *fmt, ...)
>  {
> @@ -81,39 +81,13 @@ const char *conf_get_autoconfig_name(void)
>  	return name ? name : "include/config/auto.conf";
>  }
>  
> -static char *conf_expand_value(const char *in)
> -{
> -	struct symbol *sym;
> -	const char *src;
> -	static char res_value[SYMBOL_MAXLENGTH];
> -	char *dst, name[SYMBOL_MAXLENGTH];
> -
> -	res_value[0] = 0;
> -	dst = name;
> -	while ((src = strchr(in, '$'))) {
> -		strncat(res_value, in, src - in);
> -		src++;
> -		dst = name;
> -		while (isalnum(*src) || *src == '_')
> -			*dst++ = *src++;
> -		*dst = 0;
> -		sym = sym_lookup(name, 0);
> -		sym_calc_value(sym);
> -		strcat(res_value, sym_get_string_value(sym));
> -		in = src;
> -	}
> -	strcat(res_value, in);
> -
> -	return res_value;
> -}
> -
>  char *conf_get_default_confname(void)
>  {
>  	struct stat buf;
>  	static char fullname[PATH_MAX+1];
>  	char *env, *name;
>  
> -	name = conf_expand_value(conf_defname);
> +	name = expand_string(conf_defname);
>  	env = getenv(SRCTREE);
>  	if (env) {
>  		sprintf(fullname, "%s/%s", env, name);
> @@ -274,7 +248,8 @@ int conf_read_simple(const char *name, int def)
>  			if (expr_calc_value(prop->visible.expr) == no ||
>  			    prop->expr->type != E_SYMBOL)
>  				continue;
> -			name = conf_expand_value(prop->expr->left.sym->name);
> +			sym_calc_value(prop->expr->left.sym);
> +			name = sym_get_string_value(prop->expr->left.sym);
>  			in = zconf_fopen(name);
>  			if (in) {
>  				conf_message(_("using defaults found in %s"),
> diff --git a/scripts/kconfig/kconf_id.c b/scripts/kconfig/kconf_id.c
> index 3ea9c5f..b3e0ea0 100644
> --- a/scripts/kconfig/kconf_id.c
> +++ b/scripts/kconfig/kconf_id.c
> @@ -32,7 +32,6 @@ static struct kconf_id kconf_id_array[] = {
>  	{ "on",			T_ON,			TF_PARAM },
>  	{ "modules",		T_OPT_MODULES,		TF_OPTION },
>  	{ "defconfig_list",	T_OPT_DEFCONFIG_LIST,	TF_OPTION },
> -	{ "env",		T_OPT_ENV,		TF_OPTION },
>  	{ "allnoconfig_y",	T_OPT_ALLNOCONFIG_Y,	TF_OPTION },
>  };
>  
> diff --git a/scripts/kconfig/lkc.h b/scripts/kconfig/lkc.h
> index f4394af..658efb0 100644
> --- a/scripts/kconfig/lkc.h
> +++ b/scripts/kconfig/lkc.h
> @@ -58,7 +58,6 @@ enum conf_def_mode {
>  
>  #define T_OPT_MODULES		1
>  #define T_OPT_DEFCONFIG_LIST	2
> -#define T_OPT_ENV		3
>  #define T_OPT_ALLNOCONFIG_Y	4
>  
>  struct kconf_id {
> @@ -117,6 +116,7 @@ void *xmalloc(size_t size);
>  void *xcalloc(size_t nmemb, size_t size);
>  void *xrealloc(void *p, size_t size);
>  char *xstrdup(const char *s);
> +char *xstrndup(const char *s, size_t n);
>  
>  struct gstr {
>  	size_t len;
> @@ -134,9 +134,6 @@ void str_printf(struct gstr *gs, const char *fmt, ...);
>  const char *str_get(struct gstr *gs);
>  
>  /* symbol.c */
> -extern struct expr *sym_env_list;
> -
> -void sym_init(void);
>  void sym_clear_all_valid(void);
>  struct symbol *sym_choice_default(struct symbol *sym);
>  const char *sym_get_string_default(struct symbol *sym);
> diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h
> index 9dc8abf..9f465fe 100644
> --- a/scripts/kconfig/lkc_proto.h
> +++ b/scripts/kconfig/lkc_proto.h
> @@ -49,5 +49,11 @@ const char * sym_get_string_value(struct symbol *sym);
>  
>  const char * prop_get_type_name(enum prop_type type);
>  
> +/* preprocess.c */
> +void env_write_dep(FILE *f, const char *auto_conf_name);
> +char *expand_string(const char *in);
> +char *expand_dollar(const char **str);
> +char *expand_one_token(const char **str);
> +
>  /* expr.c */
>  void expr_print(struct expr *e, void (*fn)(void *, struct symbol *, const char *), void *data, int prevtoken);
> diff --git a/scripts/kconfig/menu.c b/scripts/kconfig/menu.c
> index 5c5c137..8148305 100644
> --- a/scripts/kconfig/menu.c
> +++ b/scripts/kconfig/menu.c
> @@ -214,9 +214,6 @@ void menu_add_option(int token, char *arg)
>  			zconf_error("trying to redefine defconfig symbol");
>  		sym_defconfig_list->flags |= SYMBOL_AUTO;
>  		break;
> -	case T_OPT_ENV:
> -		prop_add_env(arg);
> -		break;
>  	case T_OPT_ALLNOCONFIG_Y:
>  		current_entry->sym->flags |= SYMBOL_ALLNOCONFIG_Y;
>  		break;
> diff --git a/scripts/kconfig/preprocess.c b/scripts/kconfig/preprocess.c
> new file mode 100644
> index 0000000..a2eb2eb
> --- /dev/null
> +++ b/scripts/kconfig/preprocess.c
> @@ -0,0 +1,238 @@
> +// SPDX-License-Identifier: GPL-2.0
> +//
> +// Copyright (C) 2018 Masahiro Yamada <yamada.masahiro@socionext.com>
> +
> +#include <stdarg.h>
> +#include <stdbool.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +
> +#include "list.h"
> +
> +static void __attribute__((noreturn)) pperror(const char *format, ...)
> +{
> +	va_list ap;
> +
> +	fprintf(stderr, "%s:%d: ", current_file->name, yylineno);
> +	va_start(ap, format);
> +	vfprintf(stderr, format, ap);
> +	va_end(ap);
> +	fprintf(stderr, "\n");
> +
> +	exit(1);
> +}
> +
> +/*
> + * Environment variables
> + */
> +static LIST_HEAD(env_list);
> +
> +struct env {
> +	char *name;
> +	char *value;
> +	struct list_head node;
> +};
> +
> +static void env_add(const char *name, const char *value)
> +{
> +	struct env *e;
> +
> +	e = xmalloc(sizeof(*e));
> +	e->name = xstrdup(name);
> +	e->value = xstrdup(value);
> +
> +	list_add_tail(&e->node, &env_list);
> +}
> +
> +static void env_del(struct env *e)
> +{
> +	list_del(&e->node);
> +	free(e->name);
> +	free(e->value);
> +	free(e);
> +}
> +
> +/* The returned pointer must be freed when done */
> +static char *env_expand(const char *name)
> +{
> +	struct env *e;
> +	const char *value;
> +
> +	if (!*name)
> +		return NULL;
> +
> +	list_for_each_entry(e, &env_list, node) {
> +		if (!strcmp(name, e->name))
> +			return xstrdup(e->value);
> +	}
> +
> +	value = getenv(name);
> +	if (!value)
> +		return NULL;
> +
> +	/*
> +	 * We need to remember all referenced environment variables.
> +	 * They will be written out to include/config/auto.conf.cmd
> +	 */
> +	env_add(name, value);
> +
> +	return xstrdup(value);
> +}
> +
> +void env_write_dep(FILE *f, const char *autoconfig_name)
> +{
> +	struct env *e, *tmp;
> +
> +	list_for_each_entry_safe(e, tmp, &env_list, node) {
> +		fprintf(f, "ifneq \"$(%s)\" \"%s\"\n", e->name, e->value);
> +		fprintf(f, "%s: FORCE\n", autoconfig_name);
> +		fprintf(f, "endif\n");
> +		env_del(e);
> +	}
> +}
> +
> +static char *eval_clause(const char *str, size_t len)
> +{
> +	char *tmp, *name, *res;
> +
> +	tmp = xstrndup(str, len);
> +
> +	name = expand_string(tmp);
> +
> +	res = env_expand(name);
> +	if (res)
> +		goto free;
> +
> +	res = xstrdup("");
> +free:
> +	free(name);
> +	free(tmp);
> +
> +	return res;
> +}
> +
> +/*
> + * Expand a string that follows '$'
> + *
> + * For example, if the input string is
> + *     ($(FOO)$($(BAR)))$(BAZ)
> + * this helper evaluates
> + *     $($(FOO)$($(BAR)))
> + * and returns a new string containing the expansion (note that the string is
> + * recursively expanded), also advancing 'str' to point to the next character
> + * after the corresponding closing parenthesis, in this case, *str will be
> + *     $(BAR)
> + */
> +char *expand_dollar(const char **str)
> +{
> +	const char *p = *str;
> +	const char *q;
> +	int nest = 0;
> +
> +	/*
> +	 * In Kconfig, variable references always start with "$(".
> +	 * Neither single-letter variables as in $A nor curly braces as in ${CC}
> +	 * are supported.  '$' not followed by '(' loses its special meaning.
> +	 */
> +	if (*p != '(') {
> +		*str = p;
> +		return xstrdup("$");
> +	}
> +
> +	p++;
> +	q = p;
> +	while (*q) {
> +		if (*q == '(') {
> +			nest++;
> +		} else if (*q == ')') {
> +			if (nest-- == 0)
> +				break;
> +		}
> +		q++;
> +	}
> +
> +	if (!*q)
> +		pperror("unterminated reference to '%s': missing ')'", p);
> +
> +	/* Advance 'str' to after the expanded initial portion of the string */
> +	*str = q + 1;
> +
> +	return eval_clause(p, q - p);
> +}
> +
> +static char *__expand_string(const char **str, bool (*is_end)(char c))
> +{
> +	const char *in, *p;
> +	char *expansion, *out;
> +	size_t in_len, out_len;
> +
> +	out = xmalloc(1);
> +	*out = 0;
> +	out_len = 1;
> +
> +	p = in = *str;
> +
> +	while (1) {
> +		if (*p == '$') {
> +			in_len = p - in;
> +			p++;
> +			expansion = expand_dollar(&p);
> +			out_len += in_len + strlen(expansion);
> +			out = xrealloc(out, out_len);
> +			strncat(out, in, in_len);
> +			strcat(out, expansion);
> +			free(expansion);
> +			in = p;
> +			continue;
> +		}
> +
> +		if (is_end(*p))
> +			break;
> +
> +		p++;
> +	}
> +
> +	in_len = p - in;
> +	out_len += in_len;
> +	out = xrealloc(out, out_len);
> +	strncat(out, in, in_len);
> +
> +	/* Advance 'str' to the end character */
> +	*str = p;
> +
> +	return out;
> +}
> +
> +static bool is_end_of_str(char c)
> +{
> +	return !c;
> +}
> +
> +/*
> + * Expand variables in the given string.  Undefined variables
> + * expand to an empty string.
> + * The returned string must be freed when done.
> + */
> +char *expand_string(const char *in)
> +{
> +	return __expand_string(&in, is_end_of_str);
> +}
> +
> +static bool is_end_of_token(char c)
> +{
> +	/* Why are '.' and '/' valid characters for symbols? */
> +	return !(isalnum(c) || c == '_' || c == '-' || c == '.' || c == '/');
> +}
> +
> +/*
> + * Expand variables in a token.  The parsing stops when a token separater
> + * (in most cases, it is a whitespace) is encountered.  'str' is updated to
> + * point to the next character.
> + *
> + * The returned string must be freed when done.
> + */
> +char *expand_one_token(const char **str)
> +{
> +	return __expand_string(str, is_end_of_token);
> +}
> diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c
> index f0b2e3b..2460648 100644
> --- a/scripts/kconfig/symbol.c
> +++ b/scripts/kconfig/symbol.c
> @@ -33,33 +33,6 @@ struct symbol *sym_defconfig_list;
>  struct symbol *modules_sym;
>  tristate modules_val;
>  
> -struct expr *sym_env_list;
> -
> -static void sym_add_default(struct symbol *sym, const char *def)
> -{
> -	struct property *prop = prop_alloc(P_DEFAULT, sym);
> -
> -	prop->expr = expr_alloc_symbol(sym_lookup(def, SYMBOL_CONST));
> -}
> -
> -void sym_init(void)
> -{
> -	struct symbol *sym;
> -	struct utsname uts;
> -	static bool inited = false;
> -
> -	if (inited)
> -		return;
> -	inited = true;
> -
> -	uname(&uts);
> -
> -	sym = sym_lookup("UNAME_RELEASE", 0);
> -	sym->type = S_STRING;
> -	sym->flags |= SYMBOL_AUTO;
> -	sym_add_default(sym, uts.release);
> -}
> -
>  enum symbol_type sym_get_type(struct symbol *sym)
>  {
>  	enum symbol_type type = sym->type;
> @@ -1401,32 +1374,3 @@ const char *prop_get_type_name(enum prop_type type)
>  	}
>  	return "unknown";
>  }
> -
> -static void prop_add_env(const char *env)
> -{
> -	struct symbol *sym, *sym2;
> -	struct property *prop;
> -	char *p;
> -
> -	sym = current_entry->sym;
> -	sym->flags |= SYMBOL_AUTO;
> -	for_all_properties(sym, prop, P_ENV) {
> -		sym2 = prop_get_symbol(prop);
> -		if (strcmp(sym2->name, env))
> -			menu_warn(current_entry, "redefining environment symbol from %s",
> -				  sym2->name);
> -		return;
> -	}
> -
> -	prop = prop_alloc(P_ENV, sym);
> -	prop->expr = expr_alloc_symbol(sym_lookup(env, SYMBOL_CONST));
> -
> -	sym_env_list = expr_alloc_one(E_LIST, sym_env_list);
> -	sym_env_list->right.sym = sym;
> -
> -	p = getenv(env);
> -	if (p)
> -		sym_add_default(sym, p);
> -	else
> -		menu_warn(current_entry, "environment variable %s undefined", env);
> -}
> diff --git a/scripts/kconfig/util.c b/scripts/kconfig/util.c
> index c6f6e21..703ee49 100644
> --- a/scripts/kconfig/util.c
> +++ b/scripts/kconfig/util.c
> @@ -34,8 +34,6 @@ struct file *file_lookup(const char *name)
>  /* write a dependency file as used by kbuild to track dependencies */
>  int file_write_dep(const char *name)
>  {
> -	struct symbol *sym, *env_sym;
> -	struct expr *e;
>  	struct file *file;
>  	FILE *out;
>  
> @@ -54,21 +52,7 @@ int file_write_dep(const char *name)
>  	fprintf(out, "\n%s: \\\n"
>  		     "\t$(deps_config)\n\n", conf_get_autoconfig_name());
>  
> -	expr_list_for_each_sym(sym_env_list, e, sym) {
> -		struct property *prop;
> -		const char *value;
> -
> -		prop = sym_get_env_prop(sym);
> -		env_sym = prop_get_symbol(prop);
> -		if (!env_sym)
> -			continue;
> -		value = getenv(env_sym->name);
> -		if (!value)
> -			value = "";
> -		fprintf(out, "ifneq \"$(%s)\" \"%s\"\n", env_sym->name, value);
> -		fprintf(out, "%s: FORCE\n", conf_get_autoconfig_name());
> -		fprintf(out, "endif\n");
> -	}
> +	env_write_dep(out, conf_get_autoconfig_name());
>  
>  	fprintf(out, "\n$(deps_config): ;\n");
>  	fclose(out);
> @@ -165,3 +149,14 @@ char *xstrdup(const char *s)
>  	fprintf(stderr, "Out of memory.\n");
>  	exit(1);
>  }
> +
> +char *xstrndup(const char *s, size_t n)
> +{
> +	char *p;
> +
> +	p = strndup(s, n);
> +	if (p)
> +		return p;
> +	fprintf(stderr, "Out of memory.\n");
> +	exit(1);
> +}
> diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
> index 045093d..b385590 100644
> --- a/scripts/kconfig/zconf.l
> +++ b/scripts/kconfig/zconf.l
> @@ -1,6 +1,5 @@
>  %option nostdinit noyywrap never-interactive full ecs
>  %option 8bit nodefault yylineno
> -%option noinput
>  %x COMMAND HELP STRING PARAM
>  %{
>  /*
> @@ -35,6 +34,8 @@ struct buffer *current_buf;
>  
>  static int last_ts, first_ts;
>  
> +static char *expand_token(const char *in, size_t n);
> +static void append_expanded_string(const char *in);
>  static void zconf_endhelp(void);
>  static void zconf_endfile(void);
>  
> @@ -147,6 +148,13 @@ n	[A-Za-z0-9_-]
>  		yylval.string = text;
>  		return T_WORD;
>  	}
> +	({n}|[/.$])+	{
> +		/* this token includes at least one '$' */
> +		yylval.string = expand_token(yytext, yyleng);
> +		if (strlen(yylval.string))
> +			return T_WORD;
> +		free(yylval.string);
> +	}
>  	#.*	/* comment */
>  	\\\n	;
>  	[[:blank:]]+
> @@ -157,12 +165,13 @@ n	[A-Za-z0-9_-]
>  }
>  
>  <STRING>{
> -	[^'"\\\n]+/\n	{
> +	"$".*	append_expanded_string(yytext);
> +	[^$'"\\\n]+/\n	{
>  		append_string(yytext, yyleng);
>  		yylval.string = text;
>  		return T_WORD_QUOTE;
>  	}
> -	[^'"\\\n]+	{
> +	[^$'"\\\n]+	{
>  		append_string(yytext, yyleng);
>  	}
>  	\\.?/\n	{
> @@ -249,6 +258,58 @@ n	[A-Za-z0-9_-]
>  }
>  
>  %%
> +static char *expand_token(const char *in, size_t n)
> +{
> +	char *out;
> +	int c;
> +	char c2;
> +	const char *rest, *end;
> +
> +	new_string();
> +	append_string(in, n);
> +
> +	/* get the whole line because we do not know the end of token. */
> +	while ((c = input()) != EOF) {
> +		if (c == '\n') {
> +			unput(c);
> +			break;
> +		}
> +		c2 = c;
> +		append_string(&c2, 1);
> +	}
> +
> +	rest = text;
> +	out = expand_one_token(&rest);
> +
> +	/* push back unused characters to the input stream */
> +	end = rest + strlen(rest);
> +	while (end > rest)
> +		unput(*--end);
> +
> +	free(text);
> +
> +	return out;
> +}
> +
> +static void append_expanded_string(const char *str)
> +{
> +	const char *end;
> +	char *res;
> +
> +	str++;
> +
> +	res = expand_dollar(&str);
> +
> +	/* push back unused characters to the input stream */
> +	end = str + strlen(str);
> +	while (end > str)
> +		unput(*--end);
> +
> +	append_string(res, strlen(res));
> +
> +	free(res);
> +}
> +
>  void zconf_starthelp(void)
>  {
>  	new_string();
> diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y
> index ad6305b..3a4a0fa 100644
> --- a/scripts/kconfig/zconf.y
> +++ b/scripts/kconfig/zconf.y
> @@ -534,7 +534,6 @@ void conf_parse(const char *name)
>  
>  	zconf_initscan(name);
>  
> -	sym_init();
>  	_menu_init();
>  
>  	if (getenv("ZCONF_DEBUG"))
> @@ -780,3 +779,4 @@ void zconfdump(FILE *out)
>  #include "expr.c"
>  #include "symbol.c"
>  #include "menu.c"
> +#include "preprocess.c"

Re: [PATCH v5 27/31] kcov: test compiler capability in Kconfig and correct dependency

[Masahiro Yamada]

2018-05-28 18:22 GMT+09:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
> As Documentation/kbuild/kconfig-language.txt notes, 'select' should be
> be used with care - it forces a lower limit of another symbol, ignoring
> the dependency.  Currently, KCOV can select GCC_PLUGINS even if arch
> does not select HAVE_GCC_PLUGINS.  This could cause the unmet direct
> dependency.
>
> Now that Kconfig can test compiler capability, let's handle this in a
> more sophisticated way.
>
> There are two ways to enable KCOV; use the compiler that natively
> supports -fsanitize-coverage=trace-pc, or build the SANCOV plugin if
> the compiler has ability to build GCC plugins.  Hence, the correct
> dependency for KCOV is:
>
>   depends on CC_HAS_SANCOV_TRACE_PC || GCC_PLUGINS
>
> You do not need to build the SANCOV plugin if the compiler already
> supports -fsanitize-coverage=trace-pc.  Hence, the select should be:
>
>   select GCC_PLUGIN_SANCOV if !CC_HAS_SANCOV_TRACE_PC
>
> With this, GCC_PLUGIN_SANCOV is selected only when necessary, so
> scripts/Makefile.gcc-plugins can be cleaner.
>
> I also cleaned up Kconfig and scripts/Makefile.kcov as well.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> Reviewed-by: Kees Cook <keescook@chromium.org>
> ---
>
> Changes in v5: None
> Changes in v4: None
> Changes in v3:
>   - Replace the previous 'select -> imply' patch with
>     a new approach
>
> Changes in v2:
>   - Drop depends on GCC_VERSION
>
>  Makefile                     |  2 +-
>  lib/Kconfig.debug            | 11 +++++++----
>  scripts/Makefile.gcc-plugins |  6 +-----
>  scripts/Makefile.kcov        | 10 ++++++----
>  4 files changed, 15 insertions(+), 14 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 80e0800..4b5a17a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -626,7 +626,7 @@ all: vmlinux
>  CFLAGS_GCOV    := -fprofile-arcs -ftest-coverage \
>         $(call cc-option,-fno-tree-loop-im) \
>         $(call cc-disable-warning,maybe-uninitialized,)
> -export CFLAGS_GCOV CFLAGS_KCOV
> +export CFLAGS_GCOV
>
>  # The arch Makefile can set ARCH_{CPP,A,C}FLAGS to override the default
>  # values of the respective KBUILD_* variables
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index c40c7b7..b76cab2 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -736,12 +736,15 @@ config ARCH_HAS_KCOV
>           only for x86_64. KCOV requires testing on other archs, and most likely
>           disabling of instrumentation for some early boot code.
>
> +config CC_HAS_SANCOV_TRACE_PC
> +       def_bool $(cc-option,-fsanitize-coverage=trace-pc)
> +
>  config KCOV
>         bool "Code coverage for fuzzing"
>         depends on ARCH_HAS_KCOV
> +       depends on CC_HAS_SANCOV_TRACE_PC || GCC_PLUGINS
>         select DEBUG_FS
> -       select GCC_PLUGINS if !COMPILE_TEST
> -       select GCC_PLUGIN_SANCOV if !COMPILE_TEST
> +       select GCC_PLUGIN_SANCOV if !CC_HAS_SANCOV_TRACE_PC
>         help
>           KCOV exposes kernel code coverage information in a form suitable
>           for coverage-guided fuzzing (randomized testing).
> @@ -755,7 +758,7 @@ config KCOV
>  config KCOV_ENABLE_COMPARISONS
>         bool "Enable comparison operands collection by KCOV"
>         depends on KCOV
> -       default n
> +       depends on $(cc-option,-fsanitize-coverage=trace-cmp)
>         help
>           KCOV also exposes operands of every comparison in the instrumented
>           code along with operand sizes and PCs of the comparison instructions.
> @@ -765,7 +768,7 @@ config KCOV_ENABLE_COMPARISONS
>  config KCOV_INSTRUMENT_ALL
>         bool "Instrument all code by default"
>         depends on KCOV
> -       default y if KCOV
> +       default y
>         help
>           If you are doing generic system call fuzzing (like e.g. syzkaller),
>           then you will want to instrument the whole kernel and you should
> diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins
> index 7f5c862..0ce3802 100644
> --- a/scripts/Makefile.gcc-plugins
> +++ b/scripts/Makefile.gcc-plugins
> @@ -14,16 +14,12 @@ ifdef CONFIG_GCC_PLUGINS
>    endif
>
>    ifdef CONFIG_GCC_PLUGIN_SANCOV
> -    ifeq ($(strip $(CFLAGS_KCOV)),)
>        # It is needed because of the gcc-plugin.sh and gcc version checks.
>        gcc-plugin-$(CONFIG_GCC_PLUGIN_SANCOV)           += sancov_plugin.so
>
> -      ifneq ($(PLUGINCC),)
> -        CFLAGS_KCOV := $(SANCOV_PLUGIN)
> -      else
> +      ifeq ($(PLUGINCC),)
>          $(warning warning: cannot use CONFIG_KCOV: -fsanitize-coverage=trace-pc is not supported by compiler)
>        endif
> -    endif
>    endif
>
>    gcc-plugin-$(CONFIG_GCC_PLUGIN_STRUCTLEAK)   += structleak_plugin.so
> diff --git a/scripts/Makefile.kcov b/scripts/Makefile.kcov
> index 5cc7203..d71ba73 100644
> --- a/scripts/Makefile.kcov
> +++ b/scripts/Makefile.kcov
> @@ -1,7 +1,9 @@
>  ifdef CONFIG_KCOV
> -CFLAGS_KCOV    := $(call cc-option,-fsanitize-coverage=trace-pc,)
> -ifeq ($(CONFIG_KCOV_ENABLE_COMPARISONS),y)
> -CFLAGS_KCOV += $(call cc-option,-fsanitize-coverage=trace-cmp,)
> -endif
> +
> +kcov-flags-$(CONFIG_CC_HAS_SANCOV_TRACE_PC)    += -fsanitize-coverage=trace-pc
> +kcov-flags-$(CONFIG_KCOV_ENABLE_COMPARISONS)   += -fsanitize-coverage=trace-cmp
> +kcov-flags-$(CONFIG_GCC_PLUGIN_SANKOV)         += -fplugin=$(objtree)/scripts/gcc-plugins/sancov_plugin.so
> +


This is my mistake.

SANKOV  ->  SANCOV


I fixed it locally.







> +export CFLAGS_KCOV := $(kcov-flags-y)
>
>  endif
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Best Regards
Masahiro Yamada