From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D01DFC4321D for ; Mon, 20 Aug 2018 18:16:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 580452173D for ; Mon, 20 Aug 2018 18:16:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=nifty.com header.i=@nifty.com header.b="uIYFTlm5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 580452173D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=socionext.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726681AbeHTVco (ORCPT ); Mon, 20 Aug 2018 17:32:44 -0400 Received: from conssluserg-05.nifty.com ([210.131.2.90]:39930 "EHLO conssluserg-05.nifty.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726156AbeHTVcn (ORCPT ); Mon, 20 Aug 2018 17:32:43 -0400 X-Greylist: delayed 3999 seconds by postgrey-1.27 at vger.kernel.org; Mon, 20 Aug 2018 17:32:40 EDT Received: from mail-vk0-f52.google.com (mail-vk0-f52.google.com [209.85.213.52]) (authenticated) by conssluserg-05.nifty.com with ESMTP id w7KIFrVQ027645; Tue, 21 Aug 2018 03:15:54 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conssluserg-05.nifty.com w7KIFrVQ027645 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1534788955; bh=kgrNRe0i5ib8EmiwjZE9gOlxm957SnPGHHl+bS6ZRLU=; h=In-Reply-To:References:From:Date:Subject:To:Cc:From; b=uIYFTlm52jrY7EMR/g2EKO6cS2GLC4uVk/1k9P0lri5FzdlwLsy+8ASDIbb4f5dJq 5u5U/ZTZgU4HFyHEdwkneTOOWr67p0beypx5Vz1uUIotYfTBxSTCgQ83k7asAGfMSx SEoPL/f/0UMlOzT65l3+qIsE8XzSgpGcnUAut6oz8c9fGVjlGgzmAqMEZk76PA3RzA Zf6l7inPkP3OpABdqpZT0WK3ASrqEBzfzPRPJaSor1l2cD/YEQTGnsv83Xsxj2ckkI ZFIFD6zAD95euu5ZUkyXbgJvWdnV4/ttFcpIl6IuskGS4OiZ186FcUyE7PGL376a6Q uth2dB2Tflcvw== X-Nifty-SrcIP: [209.85.213.52] Received: by mail-vk0-f52.google.com with SMTP id b14-v6so6965117vke.13; Mon, 20 Aug 2018 11:15:54 -0700 (PDT) X-Gm-Message-State: APzg51De0v2nxkVZriH/ZLdmEiO7kya2EunsBvCd8fBN/+dpcrNltL5s QmxnMIhsWsV9MnlOWnYUK1/+LOYHJzE9BeOs/x8= X-Google-Smtp-Source: ANB0Vda67cafu4QAdwpgLuDoebFYd2C90xSdAQOx17LiNQ2z0QTVpqslHt8W4d28PnoUpVgv9DaRM69/yBN+WVysoL0= X-Received: by 2002:a1f:f8c2:: with SMTP id w185-v6mr88675vkh.135.1534788953000; Mon, 20 Aug 2018 11:15:53 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:ab0:2642:0:0:0:0:0 with HTTP; Mon, 20 Aug 2018 11:15:12 -0700 (PDT) In-Reply-To: <20180806200725.4efa5d35@kitsune.suse.cz> References: <20180627143705.5a1fed1c@kitsune.suse.cz> <20180628111623.3807fe9b@naga.suse.cz> <20180806200725.4efa5d35@kitsune.suse.cz> From: Masahiro Yamada Date: Tue, 21 Aug 2018 03:15:12 +0900 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: due to kconfig changes kernel config file is no longer sufficient for configuring the kernel To: =?UTF-8?Q?Michal_Such=C3=A1nek?= Cc: Linux Kernel Mailing List , Takashi Iwai , Andreas Schwab , Michal Kubecek , Michal Marek , Jonathan Corbet , Yoshinori Sato , Rich Felker , "David S. Miller" , Jeff Dike , Richard Weinberger , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , X86 ML , Kees Cook , Philippe Ombredanne , Greg Kroah-Hartman , Ulf Magnusson , Jeff Mahoney , "Peter Zijlstra," , Mathieu Desnoyers , Frederic Weisbecker , Randy Dunlap , Dominik Brodowski , Nicholas Piggin , Linux Kbuild mailing list , "open list:DOCUMENTATION" , Linux-sh list , sparclinux , linux-um@lists.infradead.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 2018-08-07 3:07 GMT+09:00 Michal Such=C3=A1nek : > On Mon, 30 Jul 2018 17:02:42 +0900 > Masahiro Yamada wrote: > >> 2018-06-28 18:16 GMT+09:00 Michal Such=C3=A1nek : >> > On Wed, 27 Jun 2018 23:07:21 +0900 >> > Masahiro Yamada wrote: >> > >> >> Hi. >> >> >> >> >> >> 2018-06-27 21:37 GMT+09:00 Michal Such=C3=A1nek : >> >> > Hello, >> >> > >> >> > in the x86 Kconfig we have this: >> >> > >> >> > # Select 32 or 64 bit >> >> > config 64BIT >> >> > bool "64-bit kernel" if "$(ARCH)" =3D "x86" >> >> > default "$(ARCH)" !=3D "i386" >> >> > ---help--- >> >> > Say yes to build a 64-bit kernel - formerly known as >> >> > x86_64 Say no to build a 32-bit kernel - formerly known as i386 >> >> > >> >> > Since commit 104daea149c4 ("kconfig: reference environment >> >> > variables directly and remove 'option env=3D'") the value of ARCH >> >> > is not saved in the kernel config. >> >> >> >> I think this commit is unrelated. It was just a syntax change. >> > >> > This does not look like syntax only change to me: >> > >> > diff --git a/init/Kconfig b/init/Kconfig >> > index 15aae32e0719..1217fc62ca61 100644 >> > --- a/init/Kconfig >> > +++ b/init/Kconfig >> > @@ -1,20 +1,12 @@ >> > -config ARCH >> > - string >> > - option env=3D"ARCH" >> > - >> > -config KERNELVERSION >> > - string >> > - option env=3D"KERNELVERSION" >> > - >> >> This is just syntax change. >> >> 'option env=3D' was used to reference an environment variable. >> >> Now, $(ARCH), $(KERNELVERSION) are simpler forms. >> >> >> >> >> >> Unless I am missing something, >> >> we have never saved ARCH in the .config in the past. >> > >> > There was a config symbol defined for it before the commit removed >> > it. >> >> No. >> >> CONFIG symbols with'option env=3D' >> are not written out to the .config file. >> >> We have never had CONFIG_ARCH or CONFIG_KERNELVERSION. >> >> >> >> >> >> >> >> >> >> > Since commit f467c5640c29 ("kconfig: only write '# >> >> > CONFIG_FOO is not set' for visible symbols") the value of 64BIT >> >> > is not saved if the ARCH is set i386 or x86_64 because the >> >> > symbol is not visible. >> >> >> >> This is correct. >> >> >> >> It was discussed a few weeks ago. >> >> >> >> https://lkml.org/lkml/2018/6/5/847 >> >> >> >> >> >> > There is a number of ways to hack this particular case to work. >> >> > >> >> > However, there is a more general problem with this. Some config >> >> > options may depend on the environment, may not be saved, and the >> >> > environment is not saved either. >> >> >> >> Which environment variables in particular are in your mind? >> > >> > Any that is used in Kconfig. >> >> They are provided from outside of Kconfig. >> This is the behavior we keep since a long time ago. >> >> ARCH is given by the environment variable or the command line. >> KERNELVERSION is supplied by the top Makefile. >> >> >> >> >> As for ARCH, you need to pass the same ARCH as you used for >> >> building the kernel. (For native building, you do not have to pass >> >> ARCH explicitly, though.) >> > >> > Except if you do pass it to make config you may need to pass it to >> > to make later as well. >> >> Right. >> >> For exmaple 'make ARCH=3Darm config' will create the config suitable >> only for ARM architecture. >> Then, you need to do 'make ARCH=3Darm' to build the kernel. >> >> If it is tedious to give 'ARCH=3Darm' to every make command, >> you can do 'export ARCH=3Darm' in your shell. >> >> Again, this is the behavior we have for a long time. > > No, that's not what we had. The kernel build would fail instead of > reconfiguring the kernel for the current arch. At least it used to work > that way at some point. This is not true for most architectures. It worked for i386 prior to commit f467c5640c29 ("kconfig: only write '# CONFIG_FOO is not set' for visible symbols"). This was already discussed, and this can be solved by setting ARCH=3Di386. https://lkml.org/lkml/2018/6/7/1119 https://lkml.org/lkml/2018/6/8/149 You complaint about commit 104daea149c4 ("kconfig: reference environment variables directly and remove 'option env=3D'"), but it is unrelated. >> >> >> >> >> >> >> As for CC, HOSTCC, etc. >> >> yes, these are new 'unsaved' environments. >> >> >> >> CONFIG options now depend on the compiler. >> >> This is the concept suggested by Linus Torvalds. >> >> >> >> >> >> > So in the end all the infrastructure with symlinks >> >> > from module directory pointing to the kernel source and object >> >> > directory is useless. To interpret the config stored there you >> >> > need the environment and that is not saved anywhere. So if you >> >> > try to build out-of-tree module it might end up reconfiguring >> >> > your kernel and producing useless modules. >> >> >> >> No. out-of-tree module building never ever re-configures the >> >> kernel. >> > >> > It does implicitly because the config values depend on environment >> > that is not saved and the values themselves are not saved either. >> > If that happens to expose a new variable it is even explicitly >> > reconfigured. >> >> >> You should have a built kernel tree >> before building external modules. >> >> The .config is already there. >> >> The .config works for external modules, given that >> >> - ARCH is the same >> - the compiler is the same > > - the compiler additional plugins and/or external libraries used to > implement advanced features are the same > >> >> >> >> >> >> >> out-of-tree modules are built with exactly the same configuration >> >> as used for the kernel. >> > >> > It is not true. And that is the problem. You need the config file >> > and dump of the environment passed to the make command at >> > configuration time to get the exact same configuration. The >> > environment is not saved anywhere, though. >> >> >> Why dump of the environment? >> >> >> If you are building external modules natively >> your distribution provides /lib/modules/$(uname -r)/build, >> which contains files enough for building external modules. >> >> You can pass the directory path to M=3D... parameter. That's it. > > No, that's not it. Since passing ARCH=3Di386 is the de-facto standard to > configure a 32bit kernel and the result of passing that was not saved > you need to pass it to make as well. If you pass ARCH=3D for the configuration phase, you need to pass the same ARCH=3D in the build phase. What I can suggest for you is: $ make ARCH=3Di386 defconfig $ make ARCH=3Di386 OR $ make i386_defconfig $ make > And you need to patch a number of > 3rd party build scripts that build a kernel module as part of a bigger > project. You do not need to patch a number of scripts. Just one liner fix. Add 'export ARCH=3Di386' in the top level script. >> >> >> >> If you are cross-building external modules, >> you also need to >> >> - pass ARCH=3D >> - use the same compiler with CROSS_COMPILE=3D >> >> You should know both >> because you have built the kernel by your self. > > No, I am not cross-compiling. I am building 32bit modules and because > the kconfig system did not save the information that the kernel is > 32bit I get 64bit modules instead. You are semi-cross-compiling if you build 32-bit modules on a 64-bit host machine. >> >> You do not need any other information, do you? > > I need the information what ARCH was passed to the x86 kernel at > configuration time when building on x86. This is new. > > To avoid this and similar surprises in the future I suggest to flag any > option that depends on something dynamic (compiler option, environment > variable) so that kconfig saves it even if it would not be saved > otherwise. It is saved in include/config/auto.conf.cmd > Then kconfig can do whatever seems sensible for oldconfig but > syncconfig should not change any options based on dynamic input. It > should verify that the option matches (ie compiler can accept -foobar > if the option is on and defaults to cc-option -foobar) and fail if it > does not. Kconfig already supports this. Try KCONFIG_NOSILENTUPDATE=3D1 > >> >> > And it went nowhere. >> > >> > Anyway, the observed issue with CONFIG_64BIT on x86 is the tip of a >> > larger problem which was unnoticed for ages. The .config simply does >> > not contain the whole kernel configuration. ie. make oldconfig (and >> > make syncconfig) is *not* expected to just work. It used to work >> > just by luck until f467c5640c29 ("kconfig: only write '# CONFIG_FOO >> > is not set' for visible symbols") finally exposed the problem. >> >> If you want to build the kernel for an architecture >> other than the host machine architecture, you need to pass ARCH=3D. >> >> Building the i386 kernel on a x86_64 machine, it is a _kind_ of >> cross-compiling. So, passing ARCH=3Di386 is not so weird. >> >> >> > So is .config supposed to contain the kernel configuration or is it >> > just some byproduct of the kernel build which is meaningless >> > outside of your build environment (the object tree, shell >> > environment, etc). >> >> The .config is supposed to contain the kernel configuration, >> 'ARCH' and the compiler are exceptions. >> >> 'ARCH' must be passed separately. >> >> The .config now depends on the compiler. So, if you pass your .config >> to somebody else, some symbols that depend on the compiler support >> might be configured differently. >> >> 'make syncconfig' will notice the compiler difference, >> and show prompts for user input as needed. > > It should not prompt. I want to build the kernel non-interactively. I > do not want kernel rpm build to wait for user input or silently > change the kernel ABI just because I installed a new gcc plugin. 'make syncconfig' shows prompt. This is the behavior since a long time ago. I cannot change it to meet your requirement. If you really want to avoid prompt, run "make olddefconfig" beforehand in your script. > Thanks > > Michal > -- > To unsubscribe from this list: send the line "unsubscribe linux-kbuild" i= n > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html --=20 Best Regards Masahiro Yamada