[PATCH v2 1/3] kbuild: use 'include' directive to load auto.conf from top Makefile

[PATCH v2 1/3] kbuild: use 'include' directive to load auto.conf from top Makefile

[Masahiro Yamada]

When you build targets that require the kernel configuration, dot-config
is set to 1, then the top-level Makefile includes auto.conf.  However,
Make considers its inclusion is optional because the '-include' directive
is used.

If a necessary configuration file is missing for the external module
building, the following error message is displayed:

  ERROR: Kernel configuration is invalid.
         include/generated/autoconf.h or include/config/auto.conf are missing.
         Run 'make oldconfig && make prepare' on kernel src to fix it.

However, Make still continues building; /bin/false let the creation of
'include/config/auto.config' fail, but Make can ignore the error since
it is included by the '-include' directive.

Use the 'include' directive instead.  This allows the /bin/false
to correctly terminate the whole build process.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v2:
  - New patch

 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index ca2af1a..413c2ed 100644
--- a/Makefile
+++ b/Makefile
@@ -589,7 +589,7 @@ virt-y		:= virt/
 endif # KBUILD_EXTMOD
 
 ifeq ($(dot-config),1)
--include include/config/auto.conf
+include include/config/auto.conf
 endif
 
 # The all: target is the default when no target is given on the
-- 
2.7.4

[PATCH v2 2/3] kbuild: do not update config when running install targets

[Masahiro Yamada]

"make syncconfig" is automatically invoked when any of the following
happens:

 - .config is updated
 - any of Kconfig files is updated
 - any of environment variables referenced in Kconfig is changed

Then, it updates configuration files such as include/config/auto.conf
include/generated/autoconf.h, etc.

Even install targets (install, modules_install, etc.) are no exception.
However, they should never ever modify the source tree.  Install
targets are often run with root privileges.  Once those configuration
files are owned by root, "make mrproper" would end up with permission
error.

Install targets should just copy things blindly.  They should not care
whether the configuration is up-to-date or not.  This makes more sense
because we are interested in the configuration that was used in the
previous kernel building.

This issue has existed since before, but rarely happened.  I expect
more chance where people are hit by this; with the new Kconfig syntax
extension, the .config now contains the compiler information.  If you
cross-compile the kernel with CROSS_COMPILE, but forget to pass it
for "make install", you meet "any of environment variables referenced
in Kconfig is changed" because $(CC) is referenced in Kconfig.
Another scenario is the compiler upgrade before the installation.

Install targets need the configuration.  "make modules_install" refer
to CONFIG_MODULES etc.  "make dtbs_install" also needs CONFIG_ARCH_*
to decide which dtb files to install.  However, the auto-update of
the configuration files should be avoided.  We already do this for
external modules.

Now, Make targets are categorized into 3 groups:

[1] Do not need the kernel configuration at all

    help, coccicheck, headers_install etc.

[2] Need the latest kernel configuration

    If new config options are added, Kconfig will show prompt to
    ask user's selection.

    Build targets such as vmlinux, in-kernel modules are the cases.

[3] Need the kernel configuration, but do not want to update it

    Install targets except headers_install, and external modules
    are the cases.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v2: None

 Makefile | 22 +++++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
index 413c2ed..f1e06fba 100644
--- a/Makefile
+++ b/Makefile
@@ -225,10 +225,12 @@ no-dot-config-targets := $(clean-targets) \
 			 cscope gtags TAGS tags help% %docs check% coccicheck \
 			 $(version_h) headers_% archheaders archscripts \
 			 kernelversion %src-pkg
+no-sync-config-targets := $(no-dot-config-targets) install %install
 
-config-targets := 0
-mixed-targets  := 0
-dot-config     := 1
+config-targets  := 0
+mixed-targets   := 0
+dot-config      := 1
+may-sync-config := 1
 
 ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),)
 	ifeq ($(filter-out $(no-dot-config-targets), $(MAKECMDGOALS)),)
@@ -236,6 +238,16 @@ ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),)
 	endif
 endif
 
+ifneq ($(filter $(no-sync-config-targets), $(MAKECMDGOALS)),)
+	ifeq ($(filter-out $(no-sync-config-targets), $(MAKECMDGOALS)),)
+		may-sync-config := 0
+	endif
+endif
+
+ifneq ($(KBUILD_EXTMOD),)
+	may-sync-config := 0
+endif
+
 ifeq ($(KBUILD_EXTMOD),)
         ifneq ($(filter config %config,$(MAKECMDGOALS)),)
                 config-targets := 1
@@ -611,7 +623,7 @@ ARCH_CFLAGS :=
 include arch/$(SRCARCH)/Makefile
 
 ifeq ($(dot-config),1)
-ifeq ($(KBUILD_EXTMOD),)
+ifeq ($(may-sync-config),1)
 # Read in dependencies to all Kconfig* files, make sure to run syncconfig if
 # changes are detected. This should be included after arch/$(SRCARCH)/Makefile
 # because some architectures define CROSS_COMPILE there.
@@ -639,7 +651,7 @@ include/config/auto.conf:
 	echo >&2 ;							\
 	/bin/false)
 
-endif # KBUILD_EXTMOD
+endif # may-sync-config
 
 else
 # Dummy target needed, because used as prerequisite
-- 
2.7.4

[PATCH v2 3/3] kbuild: do not update config for 'make kernelrelease'

[Masahiro Yamada]

'make kernelrelease' depends on CONFIG_LOCALVERSION(_AUTO), but
for the same reason as install targets, we do not want to update
the configuration just for printing the kernelrelease string.

This is likely to happen when you compiled the kernel with
CROSS_COMPILE, but forget to pass it to 'make kernelrelease'.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---

Changes in v2: None

 Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index f1e06fba..bb7394a 100644
--- a/Makefile
+++ b/Makefile
@@ -225,7 +225,8 @@ no-dot-config-targets := $(clean-targets) \
 			 cscope gtags TAGS tags help% %docs check% coccicheck \
 			 $(version_h) headers_% archheaders archscripts \
 			 kernelversion %src-pkg
-no-sync-config-targets := $(no-dot-config-targets) install %install
+no-sync-config-targets := $(no-dot-config-targets) install %install \
+			   kernelrelease
 
 config-targets  := 0
 mixed-targets   := 0
-- 
2.7.4

Re: [PATCH v2 1/3] kbuild: use 'include' directive to load auto.conf from top Makefile

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 1626 bytes --]

Hi Masahiro,

I love your patch! Yet something to improve:

[auto build test ERROR on kbuild/for-next]
[also build test ERROR on v4.18-rc1 next-20180622]
[cannot apply to mmarek/for-next mmarek/rc-fixes]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Masahiro-Yamada/kbuild-use-include-directive-to-load-auto-conf-from-top-Makefile/20180623-220114
base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
config: xtensa-nommu_kc705_defconfig (attached as .config)
compiler: xtensa-de212-elf-gcc (crosstool-NG crosstool-ng-1.23.0-307-g452ee331) 7.3.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        GCC_VERSION=7.3.0 make.cross ARCH=xtensa 

All errors (new ones prefixed by >>):

>> Makefile:592: include/config/auto.conf: No such file or directory
   drivers/staging/mt7621-dts/Kconfig:4:warning: 'BUILTIN_DTB' has wrong type. 'select' only accept arguments of bool and tristate type
   <stdin>:1332:2: warning: #warning syscall io_pgetevents not implemented [-Wcpp]
   <stdin>:1335:2: warning: #warning syscall rseq not implemented [-Wcpp]

vim +592 Makefile

   590	
   591	ifeq ($(dot-config),1)
 > 592	include include/config/auto.conf
   593	endif
   594	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 10466 bytes --]

Re: [PATCH v2 1/3] kbuild: use 'include' directive to load auto.conf from top Makefile

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 1626 bytes --]

Hi Masahiro,

I love your patch! Yet something to improve:

[auto build test ERROR on kbuild/for-next]
[also build test ERROR on v4.18-rc1 next-20180622]
[cannot apply to mmarek/for-next mmarek/rc-fixes]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Masahiro-Yamada/kbuild-use-include-directive-to-load-auto-conf-from-top-Makefile/20180623-220114
base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
config: xtensa-nommu_kc705_defconfig (attached as .config)
compiler: xtensa-de212-elf-gcc (crosstool-NG crosstool-ng-1.23.0-307-g452ee331) 7.3.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        GCC_VERSION=7.3.0 make.cross ARCH=xtensa 

All errors (new ones prefixed by >>):

>> Makefile:592: include/config/auto.conf: No such file or directory
   drivers/staging/mt7621-dts/Kconfig:4:warning: 'BUILTIN_DTB' has wrong type. 'select' only accept arguments of bool and tristate type
   <stdin>:1332:2: warning: #warning syscall io_pgetevents not implemented [-Wcpp]
   <stdin>:1335:2: warning: #warning syscall rseq not implemented [-Wcpp]

vim +592 Makefile

   590	
   591	ifeq ($(dot-config),1)
 > 592	include include/config/auto.conf
   593	endif
   594	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 10466 bytes --]

Re: [PATCH v2 1/3] kbuild: use 'include' directive to load auto.conf from top Makefile

[Masahiro Yamada]

Hi.

2018-06-24 3:59 GMT+09:00 kbuild test robot <lkp@intel.com>:
> Hi Masahiro,
>
> I love your patch! Yet something to improve:
>
> [auto build test ERROR on kbuild/for-next]
> [also build test ERROR on v4.18-rc1 next-20180622]
> [cannot apply to mmarek/for-next mmarek/rc-fixes]
> [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
>
> url:    https://github.com/0day-ci/linux/commits/Masahiro-Yamada/kbuild-use-include-directive-to-load-auto-conf-from-top-Makefile/20180623-220114
> base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
> config: xtensa-nommu_kc705_defconfig (attached as .config)
> compiler: xtensa-de212-elf-gcc (crosstool-NG crosstool-ng-1.23.0-307-g452ee331) 7.3.0
> reproduce:
>         wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>         chmod +x ~/bin/make.cross
>         # save the attached .config to linux build tree
>         GCC_VERSION=7.3.0 make.cross ARCH=xtensa

This did not work for me
because the make.cross script tries to download GCC 7.3.0 xtensa-linux
but it is missing in the web site.

$ GCC_VERSION=7.3.0 make.cross ARCH=xtensa
Cannot find xtensa-linux under
https://download.01.org/0day-ci/cross-package check
/tmp/crosstool-files


I installed xtensa-de212-elf-gcc by hand,
but I could not reproduce the error.


GCC_VERSION=7.2.0 make.cross ARCH=xtensa
worked, but I could not reproduce the issue either.
(the build terminated with a different error.)


Where can I see the full build log?


> All errors (new ones prefixed by >>):
>
>>> Makefile:592: include/config/auto.conf: No such file or directory
>    drivers/staging/mt7621-dts/Kconfig:4:warning: 'BUILTIN_DTB' has wrong type. 'select' only accept arguments of bool and tristate type
>    <stdin>:1332:2: warning: #warning syscall io_pgetevents not implemented [-Wcpp]
>    <stdin>:1335:2: warning: #warning syscall rseq not implemented [-Wcpp]
>
> vim +592 Makefile
>
>    590
>    591  ifeq ($(dot-config),1)
>  > 592  include include/config/auto.conf
>    593  endif
>    594
>
> ---
> 0-DAY kernel test infrastructure                Open Source Technology Center
> https://lists.01.org/pipermail/kbuild-all                   Intel Corporation



-- 
Best Regards
Masahiro Yamada

Re: [kbuild-all] [PATCH v2 1/3] kbuild: use 'include' directive to load auto.conf from top Makefile

[Philip Li]

On Sun, Jun 24, 2018 at 05:56:24PM +0900, Masahiro Yamada wrote:
> Hi.
> 
> 2018-06-24 3:59 GMT+09:00 kbuild test robot <lkp@intel.com>:
> > Hi Masahiro,
> >
> > I love your patch! Yet something to improve:
> >
> > [auto build test ERROR on kbuild/for-next]
> > [also build test ERROR on v4.18-rc1 next-20180622]
> > [cannot apply to mmarek/for-next mmarek/rc-fixes]
> > [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
> >
> > url:    https://github.com/0day-ci/linux/commits/Masahiro-Yamada/kbuild-use-include-directive-to-load-auto-conf-from-top-Makefile/20180623-220114
> > base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
> > config: xtensa-nommu_kc705_defconfig (attached as .config)
> > compiler: xtensa-de212-elf-gcc (crosstool-NG crosstool-ng-1.23.0-307-g452ee331) 7.3.0
> > reproduce:
> >         wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
> >         chmod +x ~/bin/make.cross
> >         # save the attached .config to linux build tree
> >         GCC_VERSION=7.3.0 make.cross ARCH=xtensa
> 
> This did not work for me
> because the make.cross script tries to download GCC 7.3.0 xtensa-linux
> but it is missing in the web site.
> 
> $ GCC_VERSION=7.3.0 make.cross ARCH=xtensa
> Cannot find xtensa-linux under
> https://download.01.org/0day-ci/cross-package check
> /tmp/crosstool-files
thanks for info, we will fix it as early as possible, and
reproduce the issue to provide feedback later.

> 
> 
> I installed xtensa-de212-elf-gcc by hand,
> but I could not reproduce the error.
> 
> 
> GCC_VERSION=7.2.0 make.cross ARCH=xtensa
> worked, but I could not reproduce the issue either.
> (the build terminated with a different error.)
> 
> 
> Where can I see the full build log?
> 
> 
> > All errors (new ones prefixed by >>):
> >
> >>> Makefile:592: include/config/auto.conf: No such file or directory
> >    drivers/staging/mt7621-dts/Kconfig:4:warning: 'BUILTIN_DTB' has wrong type. 'select' only accept arguments of bool and tristate type
> >    <stdin>:1332:2: warning: #warning syscall io_pgetevents not implemented [-Wcpp]
> >    <stdin>:1335:2: warning: #warning syscall rseq not implemented [-Wcpp]
> >
> > vim +592 Makefile
> >
> >    590
> >    591  ifeq ($(dot-config),1)
> >  > 592  include include/config/auto.conf
> >    593  endif
> >    594
> >
> > ---
> > 0-DAY kernel test infrastructure                Open Source Technology Center
> > https://lists.01.org/pipermail/kbuild-all                   Intel Corporation
> 
> 
> 
> -- 
> Best Regards
> Masahiro Yamada
> _______________________________________________
> kbuild-all mailing list
> kbuild-all@lists.01.org
> https://lists.01.org/mailman/listinfo/kbuild-all

Re: [PATCH v2 1/3] kbuild: use 'include' directive to load auto.conf from top Makefile

[Masahiro Yamada]

2018-06-24 3:59 GMT+09:00 kbuild test robot <lkp@intel.com>:
> Hi Masahiro,
>
> I love your patch! Yet something to improve:
>
> [auto build test ERROR on kbuild/for-next]
> [also build test ERROR on v4.18-rc1 next-20180622]
> [cannot apply to mmarek/for-next mmarek/rc-fixes]
> [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
>
> url:    https://github.com/0day-ci/linux/commits/Masahiro-Yamada/kbuild-use-include-directive-to-load-auto-conf-from-top-Makefile/20180623-220114
> base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
> config: xtensa-nommu_kc705_defconfig (attached as .config)
> compiler: xtensa-de212-elf-gcc (crosstool-NG crosstool-ng-1.23.0-307-g452ee331) 7.3.0
> reproduce:
>         wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>         chmod +x ~/bin/make.cross
>         # save the attached .config to linux build tree
>         GCC_VERSION=7.3.0 make.cross ARCH=xtensa
>
> All errors (new ones prefixed by >>):
>
>>> Makefile:592: include/config/auto.conf: No such file or directory


I was able to reproduce this.


This depends on the version of GNU Make.

The warning is displayed if you use GNU Make 4.1 or older.
No more annoying warning is displayed for GNU Make 4.2 or later.

Probably, this commit in GNU Make:

commit 87a5f98d248fe63fe6e3e53ee3e1b1b1fa5e49dc
Author: Paul Smith <psmith@gnu.org>
Date:   Sat Apr 9 19:49:27 2016 -0400

    [SV 102] Don't show unnecessary include file errors.


However, 4.2 is quite new.  I will do something with it.


Thanks.






>    drivers/staging/mt7621-dts/Kconfig:4:warning: 'BUILTIN_DTB' has wrong type. 'select' only accept arguments of bool and tristate type
>    <stdin>:1332:2: warning: #warning syscall io_pgetevents not implemented [-Wcpp]
>    <stdin>:1335:2: warning: #warning syscall rseq not implemented [-Wcpp]
>
> vim +592 Makefile
>
>    590
>    591  ifeq ($(dot-config),1)
>  > 592  include include/config/auto.conf
>    593  endif
>    594
>
> ---
> 0-DAY kernel test infrastructure                Open Source Technology Center
> https://lists.01.org/pipermail/kbuild-all                   Intel Corporation



-- 
Best Regards
Masahiro Yamada