All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/3] fixing the last failures in randconfig builds
@ 2018-02-16 21:41 Arnd Bergmann
  2018-02-16 21:41 ` [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing Arnd Bergmann
                   ` (2 more replies)
  0 siblings, 3 replies; 20+ messages in thread
From: Arnd Bergmann @ 2018-02-16 21:41 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Yann E . MORIN, Masahiro Yamada, Michal Marek,
	Greg Kroah-Hartman, linux-kernel, Arnd Bergmann

I've recently done some build testing on 4.14-stable and to my surprise
found only a handful of bugfixes were required to build all x86 and arm64
randconfig output without any warnings using gcc-7.2.1. I have submitted
the remaining trivial patches now, and this is the non-trivial remaining
part, where we turn off some options that are known to cause problems in
randconfig builds.

While I've carried the same patches in my randconfig test tree for a long
time, I wasn't entirely sure about my approach and never submitted them
for inclusion. Now that this is literally all that is remains, I'd like
to propose all three for inclusion.

If someone has another idea for how to address these, that's fine with
me as well, as long as we can make sure at least COMPILE_TEST is enabled
for randconfig builds by default (everything else can be derived from that
if necessary).

Note that the 4.15 release has a couple of build-time regressions that I
have sent bugfixes for but that so far have not made it in, and 4.16-rc
still has a number of other issues without patches.

       Arnd

Arnd Bergmann (3):
  Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
  Kconfig: improve handling for all{rand,yes,no,}.config fragments
  Kconfig: add a default allrandom.config

 kernel/configs/allrandom.config |  6 ++++++
 kernel/trace/Kconfig            |  1 +
 scripts/kconfig/conf.c          | 27 +++++++++++++++++++--------
 3 files changed, 26 insertions(+), 8 deletions(-)
 create mode 100644 kernel/configs/allrandom.config

-- 
2.9.0

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
  2018-02-16 21:41 [PATCH 0/3] fixing the last failures in randconfig builds Arnd Bergmann
@ 2018-02-16 21:41 ` Arnd Bergmann
  2018-02-16 22:03   ` Steven Rostedt
  2018-02-20  9:32   ` Masahiro Yamada
  2018-02-16 21:41 ` [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments Arnd Bergmann
  2018-02-16 21:41 ` [PATCH 3/3] Kconfig: add a default allrandom.config Arnd Bergmann
  2 siblings, 2 replies; 20+ messages in thread
From: Arnd Bergmann @ 2018-02-16 21:41 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Yann E . MORIN, Masahiro Yamada, Michal Marek,
	Greg Kroah-Hartman, linux-kernel, Arnd Bergmann, Steven Rostedt,
	Ingo Molnar

This can easily double the time for compiling a driver but does not
provide any benefit for the compile tester, so it's better left disabled.

In addition, any 'inline' function that is not also 'static' and that
contains an 'if' causes a warning like

include/linux/string.h:212:2: note: in expansion of macro 'if'
  if (strscpy(p, q, p_size < q_size ? p_size : q_size) < 0)
  ^~
include/linux/compiler.h:162:4: warning: '______f' is static but declared in inline function 'strcpy' which is not static

without this patch, and I could not come up with a nice fix for that.
In combination with my patch to always enable 'CONFIG_COMPILE_TEST'
during 'randconfig' builds, we can at least hide these warnings for
most users.

Cc: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
 kernel/trace/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 434c840e2d82..faaf687b13b1 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -345,6 +345,7 @@ config PROFILE_ANNOTATED_BRANCHES
 config PROFILE_ALL_BRANCHES
 	bool "Profile all if conditionals"
 	select TRACE_BRANCH_PROFILING
+	depends on !COMPILE_TEST
 	help
 	  This tracer profiles all branch conditions. Every if ()
 	  taken in the kernel is recorded whether it hit or miss.
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments
  2018-02-16 21:41 [PATCH 0/3] fixing the last failures in randconfig builds Arnd Bergmann
  2018-02-16 21:41 ` [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing Arnd Bergmann
@ 2018-02-16 21:41 ` Arnd Bergmann
  2018-02-20  9:26   ` Masahiro Yamada
  2018-02-16 21:41 ` [PATCH 3/3] Kconfig: add a default allrandom.config Arnd Bergmann
  2 siblings, 1 reply; 20+ messages in thread
From: Arnd Bergmann @ 2018-02-16 21:41 UTC (permalink / raw)
  To: linux-kbuild, Yann E. MORIN
  Cc: Masahiro Yamada, Michal Marek, Greg Kroah-Hartman, linux-kernel,
	Arnd Bergmann

The kernel currently supports two methods of dealing with config
fragments in the tree:

a) Running "make foo.config" looks for arch/$(ARCH)/configs/foo.config
   and kernel/configs/foo.config, and applies the defaults from those
   files on top of the current configuration.

b) Running "KCONFIG_ALLCONFIG=1 make randconfig" (or the equivalent
   allmodconfig/allnoconfig/allyesconfig/alldefconfig) will look
   for a "allrandconfig.config" file in the current directory or the
   top of the $(srctree). These are used as defaults before we generate
   the remaining options.

This is rather inconsistent, and prevents us from easily shipping
good defaults for "randconfig". I'm extending the logic here so that
the second case also looks for the hardcoded file names in the standard
directories (first arch/$(ARCH)/configs/, then kernel/configs) in the
source tree.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
 scripts/kconfig/conf.c | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)

diff --git a/scripts/kconfig/conf.c b/scripts/kconfig/conf.c
index 866369f10ff8..848bf4d15e9a 100644
--- a/scripts/kconfig/conf.c
+++ b/scripts/kconfig/conf.c
@@ -493,8 +493,9 @@ int main(int ac, char **av)
 {
 	const char *progname = av[0];
 	int opt;
-	const char *name, *defconfig_file = NULL /* gcc uninit */;
+	const char *arch, *name, *defconfig_file = NULL /* gcc uninit */;
 	struct stat tmpstat;
+	char fullname[PATH_MAX+1];
 
 	setlocale(LC_ALL, "");
 	bindtextdomain(PACKAGE, LOCALEDIR);
@@ -621,14 +622,24 @@ int main(int ac, char **av)
 		case randconfig:	name = "allrandom.config"; break;
 		default: break;
 		}
-		if (conf_read_simple(name, S_DEF_USER) &&
-		    conf_read_simple("all.config", S_DEF_USER)) {
-			fprintf(stderr,
-				_("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n"),
-				name);
-			exit(1);
+		/* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */
+		if (!conf_read_simple(name, S_DEF_USER))
+			break;
+		arch = getenv("ARCH");
+		if (arch) {
+			snprintf(fullname, sizeof(fullname), "arch/%s/configs/%s",
+				 arch, name);
+			if (!conf_read_simple(fullname, S_DEF_USER))
+				break;
 		}
-		break;
+		snprintf(fullname, sizeof(fullname), "kernel/configs/%s", name);
+		if (!conf_read_simple(fullname, S_DEF_USER))
+			break;
+
+		fprintf(stderr,
+			_("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n"),
+				name);
+		exit(1);
 	default:
 		break;
 	}
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 3/3] Kconfig: add a default allrandom.config
  2018-02-16 21:41 [PATCH 0/3] fixing the last failures in randconfig builds Arnd Bergmann
  2018-02-16 21:41 ` [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing Arnd Bergmann
  2018-02-16 21:41 ` [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments Arnd Bergmann
@ 2018-02-16 21:41 ` Arnd Bergmann
  2018-02-20  9:16     ` Masahiro Yamada
  2 siblings, 1 reply; 20+ messages in thread
From: Arnd Bergmann @ 2018-02-16 21:41 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Yann E . MORIN, Masahiro Yamada, Michal Marek,
	Greg Kroah-Hartman, linux-kernel, Arnd Bergmann

Building randconfig kernels frequently leads to build errors from
drivers that have additional build dependencies, or that we don't
want to build for compile-testing for some other reason.

We already have a couple of compile-time options that can deal with this
problem, but the complete set is not documented well.  We also have a
method to override some options during 'make randconfig' and a couple
of other make targets. However, we don't yet combine those two.

This adds a new allrandom.config file to turn on CONFIG_COMPILE_TEST,
CONFIG_STANDALONE and CONFIG_PREVENT_FIRMWARE_BUILD for all randconfig
builds. This in turn disables some other options that we already try
to avoid in 'allmodconfig' builds and that make no sense in a general
randconfig build.

Building with 'make randconfig KCONFIG_ALLCONFIG=1' should now always
succeed without warnings on x86 and arm64, aside from recent regressions.
Other architectures probably need additional bugfixes.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
 kernel/configs/allrandom.config | 6 ++++++
 1 file changed, 6 insertions(+)
 create mode 100644 kernel/configs/allrandom.config

diff --git a/kernel/configs/allrandom.config b/kernel/configs/allrandom.config
new file mode 100644
index 000000000000..67294ef2e3a2
--- /dev/null
+++ b/kernel/configs/allrandom.config
@@ -0,0 +1,6 @@
+# maximize search space, disable options not worth testing
+CONFIG_COMPILE_TEST=y
+
+# reduce compile-time dependencies
+CONFIG_STANDALONE=y
+CONFIG_PREVENT_FIRMWARE_BUILD=y
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 20+ messages in thread

* Re: [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
  2018-02-16 21:41 ` [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing Arnd Bergmann
@ 2018-02-16 22:03   ` Steven Rostedt
  2018-02-16 22:14     ` Arnd Bergmann
  2018-02-20  9:32   ` Masahiro Yamada
  1 sibling, 1 reply; 20+ messages in thread
From: Steven Rostedt @ 2018-02-16 22:03 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: linux-kbuild, Yann E . MORIN, Masahiro Yamada, Michal Marek,
	Greg Kroah-Hartman, linux-kernel, Ingo Molnar, Randy Dunlap

On Fri, 16 Feb 2018 22:41:11 +0100
Arnd Bergmann <arnd@arndb.de> wrote:

> This can easily double the time for compiling a driver but does not
> provide any benefit for the compile tester, so it's better left disabled.
> 
> In addition, any 'inline' function that is not also 'static' and that
> contains an 'if' causes a warning like
> 
> include/linux/string.h:212:2: note: in expansion of macro 'if'
>   if (strscpy(p, q, p_size < q_size ? p_size : q_size) < 0)
>   ^~
> include/linux/compiler.h:162:4: warning: '______f' is static but declared in inline function 'strcpy' which is not static
> 
> without this patch, and I could not come up with a nice fix for that.
> In combination with my patch to always enable 'CONFIG_COMPILE_TEST'
> during 'randconfig' builds, we can at least hide these warnings for
> most users.

This looks like it fixes the same issue that was already fixed and is
in Linus's tree.

 http://lkml.kernel.org/r/9199446b-a141-c0c3-9678-a3f9107f2750@infradead.org

See commit 68e76e034b6b1 ("tracing: Prevent PROFILE_ALL_BRANCHES when
FORTIFY_SOURCE=y")

-- Steve

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
  2018-02-16 22:03   ` Steven Rostedt
@ 2018-02-16 22:14     ` Arnd Bergmann
  2018-02-16 22:40       ` Arnd Bergmann
  2018-02-17 13:32       ` Greg Kroah-Hartman
  0 siblings, 2 replies; 20+ messages in thread
From: Arnd Bergmann @ 2018-02-16 22:14 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: Linux Kbuild mailing list, Yann E . MORIN, Masahiro Yamada,
	Michal Marek, Greg Kroah-Hartman, Linux Kernel Mailing List,
	Ingo Molnar, Randy Dunlap, stable

On Fri, Feb 16, 2018 at 11:03 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
> On Fri, 16 Feb 2018 22:41:11 +0100
> Arnd Bergmann <arnd@arndb.de> wrote:
>
>> This can easily double the time for compiling a driver but does not
>> provide any benefit for the compile tester, so it's better left disabled.
>>
>> In addition, any 'inline' function that is not also 'static' and that
>> contains an 'if' causes a warning like
>>
>> include/linux/string.h:212:2: note: in expansion of macro 'if'
>>   if (strscpy(p, q, p_size < q_size ? p_size : q_size) < 0)
>>   ^~
>> include/linux/compiler.h:162:4: warning: '______f' is static but declared in inline function 'strcpy' which is not static
>>
>> without this patch, and I could not come up with a nice fix for that.
>> In combination with my patch to always enable 'CONFIG_COMPILE_TEST'
>> during 'randconfig' builds, we can at least hide these warnings for
>> most users.
>
> This looks like it fixes the same issue that was already fixed and is
> in Linus's tree.
>
>  http://lkml.kernel.org/r/9199446b-a141-c0c3-9678-a3f9107f2750@infradead.org
>
> See commit 68e76e034b6b1 ("tracing: Prevent PROFILE_ALL_BRANCHES when
> FORTIFY_SOURCE=y")

Ah, right. I missed that when I wrote the new changelog text for this old
patch of mine. It also means I should rebase the patch so it applies
on mainline, as I still want PROFILE_ALL_BRANCHES to be disabled
in COMPILE_TEST kernels for the build speed aspect.

Greg, could you add the 68e76e034b6b1 commit to 4.14-stable and
4.15-stable in the meantime?

        Arnd

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
  2018-02-16 22:14     ` Arnd Bergmann
@ 2018-02-16 22:40       ` Arnd Bergmann
  2018-02-16 22:50         ` Steven Rostedt
  2018-02-17 13:32       ` Greg Kroah-Hartman
  1 sibling, 1 reply; 20+ messages in thread
From: Arnd Bergmann @ 2018-02-16 22:40 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: Linux Kbuild mailing list, Yann E . MORIN, Masahiro Yamada,
	Michal Marek, Greg Kroah-Hartman, Linux Kernel Mailing List,
	Ingo Molnar, Randy Dunlap, stable

On Fri, Feb 16, 2018 at 11:14 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Fri, Feb 16, 2018 at 11:03 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
>> On Fri, 16 Feb 2018 22:41:11 +0100
>> Arnd Bergmann <arnd@arndb.de> wrote:
>>
>>> This can easily double the time for compiling a driver but does not
>>> provide any benefit for the compile tester, so it's better left disabled.
>>>
>>> In addition, any 'inline' function that is not also 'static' and that
>>> contains an 'if' causes a warning like
>>>
>>> include/linux/string.h:212:2: note: in expansion of macro 'if'
>>>   if (strscpy(p, q, p_size < q_size ? p_size : q_size) < 0)
>>>   ^~
>>> include/linux/compiler.h:162:4: warning: '______f' is static but declared in inline function 'strcpy' which is not static
>>>
>>> without this patch, and I could not come up with a nice fix for that.
>>> In combination with my patch to always enable 'CONFIG_COMPILE_TEST'
>>> during 'randconfig' builds, we can at least hide these warnings for
>>> most users.
>>
>> This looks like it fixes the same issue that was already fixed and is
>> in Linus's tree.
>>
>>  http://lkml.kernel.org/r/9199446b-a141-c0c3-9678-a3f9107f2750@infradead.org
>>
>> See commit 68e76e034b6b1 ("tracing: Prevent PROFILE_ALL_BRANCHES when
>> FORTIFY_SOURCE=y")
>
> Ah, right. I missed that when I wrote the new changelog text for this old
> patch of mine. It also means I should rebase the patch so it applies
> on mainline, as I still want PROFILE_ALL_BRANCHES to be disabled
> in COMPILE_TEST kernels for the build speed aspect.

I retested on top of that patch and found a couple of other warnings show up
in an allmodconfig build with PROFILE_ALL_BRANCHES:

lib/zstd/decompress.c: In function 'ZSTD_decompressStream':
lib/zstd/decompress.c:416:2: error: argument 1 null where non-null
expected [-Werror=nonnull]
drivers/crypto/qat/qat_common/qat_algs.c: In function 'qat_alg_do_precomputes':
drivers/crypto/qat/qat_common/qat_algs.c:156:7: error: argument 1
range [18446744071562067968, 18446744073709551615] exceeds maximum
object size 9223372036854775807 [-Werror=alloc-size-larger-than=]
drivers/isdn/hardware/eicon/message.c: In function 'mixer_notify_update':
drivers/isdn/hardware/eicon/message.c:11162:54: error: array subscript
is above array bounds [-Werror=array-bounds]
     ((CAPI_MSG *) msg)->info.facility_req.structs[1] =
LI_REQ_SILENT_UPDATE & 0xff;
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/isdn/hardware/eicon/message.c:11163:54: error: array subscript
is above array bounds [-Werror=array-bounds]
     ((CAPI_MSG *) msg)->info.facility_req.structs[2] =
LI_REQ_SILENT_UPDATE >> 8;
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/isdn/hardware/eicon/message.c:11164:54: error: array subscript
is above array bounds [-Werror=array-bounds]
     ((CAPI_MSG *) msg)->info.facility_req.structs[3] = 0;

All those are nonsense AFAICT, and we see them only because the "if()" override
ends up confusing gcc's value-range tracking in the same way it used to cause
lots of -Wmaybe-uninitialized warnings (which we just disable these days
with PROFILE_ALL_BRANCHES).

        Arnd

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
  2018-02-16 22:40       ` Arnd Bergmann
@ 2018-02-16 22:50         ` Steven Rostedt
  0 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2018-02-16 22:50 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Linux Kbuild mailing list, Yann E . MORIN, Masahiro Yamada,
	Michal Marek, Greg Kroah-Hartman, Linux Kernel Mailing List,
	Ingo Molnar, Randy Dunlap, stable

On Fri, 16 Feb 2018 23:40:22 +0100
Arnd Bergmann <arnd@arndb.de> wrote:

>      ((CAPI_MSG *) msg)->info.facility_req.structs[1] =
> LI_REQ_SILENT_UPDATE & 0xff;
>      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> drivers/isdn/hardware/eicon/message.c:11163:54: error: array subscript
> is above array bounds [-Werror=array-bounds]
>      ((CAPI_MSG *) msg)->info.facility_req.structs[2] =
> LI_REQ_SILENT_UPDATE >> 8;
>      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
> drivers/isdn/hardware/eicon/message.c:11164:54: error: array subscript
> is above array bounds [-Werror=array-bounds]
>      ((CAPI_MSG *) msg)->info.facility_req.structs[3] = 0;
> 
> All those are nonsense AFAICT, and we see them only because the "if()" override
> ends up confusing gcc's value-range tracking in the same way it used to cause
> lots of -Wmaybe-uninitialized warnings (which we just disable these days
> with PROFILE_ALL_BRANCHES).

I'm fine with your patch then.

-- Steve

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
  2018-02-16 22:14     ` Arnd Bergmann
  2018-02-16 22:40       ` Arnd Bergmann
@ 2018-02-17 13:32       ` Greg Kroah-Hartman
  1 sibling, 0 replies; 20+ messages in thread
From: Greg Kroah-Hartman @ 2018-02-17 13:32 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Steven Rostedt, Linux Kbuild mailing list, Yann E . MORIN,
	Masahiro Yamada, Michal Marek, Linux Kernel Mailing List,
	Ingo Molnar, Randy Dunlap, stable

On Fri, Feb 16, 2018 at 11:14:42PM +0100, Arnd Bergmann wrote:
> On Fri, Feb 16, 2018 at 11:03 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
> > On Fri, 16 Feb 2018 22:41:11 +0100
> > Arnd Bergmann <arnd@arndb.de> wrote:
> >
> >> This can easily double the time for compiling a driver but does not
> >> provide any benefit for the compile tester, so it's better left disabled.
> >>
> >> In addition, any 'inline' function that is not also 'static' and that
> >> contains an 'if' causes a warning like
> >>
> >> include/linux/string.h:212:2: note: in expansion of macro 'if'
> >>   if (strscpy(p, q, p_size < q_size ? p_size : q_size) < 0)
> >>   ^~
> >> include/linux/compiler.h:162:4: warning: '______f' is static but declared in inline function 'strcpy' which is not static
> >>
> >> without this patch, and I could not come up with a nice fix for that.
> >> In combination with my patch to always enable 'CONFIG_COMPILE_TEST'
> >> during 'randconfig' builds, we can at least hide these warnings for
> >> most users.
> >
> > This looks like it fixes the same issue that was already fixed and is
> > in Linus's tree.
> >
> >  http://lkml.kernel.org/r/9199446b-a141-c0c3-9678-a3f9107f2750@infradead.org
> >
> > See commit 68e76e034b6b1 ("tracing: Prevent PROFILE_ALL_BRANCHES when
> > FORTIFY_SOURCE=y")
> 
> Ah, right. I missed that when I wrote the new changelog text for this old
> patch of mine. It also means I should rebase the patch so it applies
> on mainline, as I still want PROFILE_ALL_BRANCHES to be disabled
> in COMPILE_TEST kernels for the build speed aspect.
> 
> Greg, could you add the 68e76e034b6b1 commit to 4.14-stable and
> 4.15-stable in the meantime?

It's already in 4.15, so I'll just queue it up to 4.14.y.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 3/3] Kconfig: add a default allrandom.config
  2018-02-16 21:41 ` [PATCH 3/3] Kconfig: add a default allrandom.config Arnd Bergmann
@ 2018-02-20  9:16     ` Masahiro Yamada
  0 siblings, 0 replies; 20+ messages in thread
From: Masahiro Yamada @ 2018-02-20  9:16 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Linux Kbuild mailing list, Yann E . MORIN, Michal Marek,
	Greg Kroah-Hartman, Linux Kernel Mailing List

2018-02-17 6:41 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
> Building randconfig kernels frequently leads to build errors from
> drivers that have additional build dependencies, or that we don't
> want to build for compile-testing for some other reason.

Can you fix the former case?

The latter seems OK.


> We already have a couple of compile-time options that can deal with this
> problem, but the complete set is not documented well.  We also have a
> method to override some options during 'make randconfig' and a couple
> of other make targets. However, we don't yet combine those two.


>From this statement,

kernel/configs/compile_test.config

would make more sense?




> This adds a new allrandom.config file to turn on CONFIG_COMPILE_TEST,
> CONFIG_STANDALONE and CONFIG_PREVENT_FIRMWARE_BUILD for all randconfig
> builds. This in turn disables some other options that we already try
> to avoid in 'allmodconfig' builds and that make no sense in a general
> randconfig build.
>
> Building with 'make randconfig KCONFIG_ALLCONFIG=1' should now always


Can you type 'make randconfig compile_test.config', instead?

(Or 'make allmodconfig compile_test.config' for full-build test)


The merge_config has a little bit different logic
from KCONFIG_ALLCONFIG=1, but I guess it leads to the almost same result.



> succeed without warnings on x86 and arm64, aside from recent regressions.
> Other architectures probably need additional bugfixes.
>
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ---
>  kernel/configs/allrandom.config | 6 ++++++
>  1 file changed, 6 insertions(+)
>  create mode 100644 kernel/configs/allrandom.config
>
> diff --git a/kernel/configs/allrandom.config b/kernel/configs/allrandom.config
> new file mode 100644
> index 000000000000..67294ef2e3a2
> --- /dev/null
> +++ b/kernel/configs/allrandom.config
> @@ -0,0 +1,6 @@
> +# maximize search space, disable options not worth testing
> +CONFIG_COMPILE_TEST=y
> +
> +# reduce compile-time dependencies
> +CONFIG_STANDALONE=y
> +CONFIG_PREVENT_FIRMWARE_BUILD=y
> --
> 2.9.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 3/3] Kconfig: add a default allrandom.config
@ 2018-02-20  9:16     ` Masahiro Yamada
  0 siblings, 0 replies; 20+ messages in thread
From: Masahiro Yamada @ 2018-02-20  9:16 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Linux Kbuild mailing list, Yann E . MORIN, Michal Marek,
	Greg Kroah-Hartman, Linux Kernel Mailing List

2018-02-17 6:41 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
> Building randconfig kernels frequently leads to build errors from
> drivers that have additional build dependencies, or that we don't
> want to build for compile-testing for some other reason.

Can you fix the former case?

The latter seems OK.


> We already have a couple of compile-time options that can deal with this
> problem, but the complete set is not documented well.  We also have a
> method to override some options during 'make randconfig' and a couple
> of other make targets. However, we don't yet combine those two.


From this statement,

kernel/configs/compile_test.config

would make more sense?




> This adds a new allrandom.config file to turn on CONFIG_COMPILE_TEST,
> CONFIG_STANDALONE and CONFIG_PREVENT_FIRMWARE_BUILD for all randconfig
> builds. This in turn disables some other options that we already try
> to avoid in 'allmodconfig' builds and that make no sense in a general
> randconfig build.
>
> Building with 'make randconfig KCONFIG_ALLCONFIG=1' should now always


Can you type 'make randconfig compile_test.config', instead?

(Or 'make allmodconfig compile_test.config' for full-build test)


The merge_config has a little bit different logic
from KCONFIG_ALLCONFIG=1, but I guess it leads to the almost same result.



> succeed without warnings on x86 and arm64, aside from recent regressions.
> Other architectures probably need additional bugfixes.
>
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ---
>  kernel/configs/allrandom.config | 6 ++++++
>  1 file changed, 6 insertions(+)
>  create mode 100644 kernel/configs/allrandom.config
>
> diff --git a/kernel/configs/allrandom.config b/kernel/configs/allrandom.config
> new file mode 100644
> index 000000000000..67294ef2e3a2
> --- /dev/null
> +++ b/kernel/configs/allrandom.config
> @@ -0,0 +1,6 @@
> +# maximize search space, disable options not worth testing
> +CONFIG_COMPILE_TEST=y
> +
> +# reduce compile-time dependencies
> +CONFIG_STANDALONE=y
> +CONFIG_PREVENT_FIRMWARE_BUILD=y
> --
> 2.9.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments
  2018-02-16 21:41 ` [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments Arnd Bergmann
@ 2018-02-20  9:26   ` Masahiro Yamada
  2018-02-20  9:59     ` Arnd Bergmann
  0 siblings, 1 reply; 20+ messages in thread
From: Masahiro Yamada @ 2018-02-20  9:26 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Linux Kbuild mailing list, Yann E. MORIN, Michal Marek,
	Greg Kroah-Hartman, Linux Kernel Mailing List

2018-02-17 6:41 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
> The kernel currently supports two methods of dealing with config
> fragments in the tree:
>
> a) Running "make foo.config" looks for arch/$(ARCH)/configs/foo.config
>    and kernel/configs/foo.config, and applies the defaults from those
>    files on top of the current configuration.
>
> b) Running "KCONFIG_ALLCONFIG=1 make randconfig" (or the equivalent
>    allmodconfig/allnoconfig/allyesconfig/alldefconfig) will look
>    for a "allrandconfig.config" file in the current directory or the
>    top of the $(srctree). These are used as defaults before we generate
>    the remaining options.
>
> This is rather inconsistent, and prevents us from easily shipping
> good defaults for "randconfig". I'm extending the logic here so that
> the second case also looks for the hardcoded file names in the standard
> directories (first arch/$(ARCH)/configs/, then kernel/configs) in the
> source tree.
>
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ---
>  scripts/kconfig/conf.c | 27 +++++++++++++++++++--------
>  1 file changed, 19 insertions(+), 8 deletions(-)
>
> diff --git a/scripts/kconfig/conf.c b/scripts/kconfig/conf.c
> index 866369f10ff8..848bf4d15e9a 100644
> --- a/scripts/kconfig/conf.c
> +++ b/scripts/kconfig/conf.c
> @@ -493,8 +493,9 @@ int main(int ac, char **av)
>  {
>         const char *progname = av[0];
>         int opt;
> -       const char *name, *defconfig_file = NULL /* gcc uninit */;
> +       const char *arch, *name, *defconfig_file = NULL /* gcc uninit */;
>         struct stat tmpstat;
> +       char fullname[PATH_MAX+1];
>
>         setlocale(LC_ALL, "");
>         bindtextdomain(PACKAGE, LOCALEDIR);
> @@ -621,14 +622,24 @@ int main(int ac, char **av)
>                 case randconfig:        name = "allrandom.config"; break;
>                 default: break;
>                 }
> -               if (conf_read_simple(name, S_DEF_USER) &&
> -                   conf_read_simple("all.config", S_DEF_USER)) {
> -                       fprintf(stderr,
> -                               _("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n"),
> -                               name);
> -                       exit(1);
> +               /* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */
> +               if (!conf_read_simple(name, S_DEF_USER))
> +                       break;
> +               arch = getenv("ARCH");
> +               if (arch) {
> +                       snprintf(fullname, sizeof(fullname), "arch/%s/configs/%s",
> +                                arch, name);
> +                       if (!conf_read_simple(fullname, S_DEF_USER))
> +                               break;
>                 }


I am not a big fan of hard-coding the kernel directory structure.

We already do this [1], but I am thinking of kicking this out.
[1] https://github.com/torvalds/linux/blob/v4.16-rc1/scripts/kconfig/confdata.c#L33



BTW, I am trying to compiler capability check to Kconfig.
If this work is done, I hope some "depends on !COMPILE_TEST" will go
away (but not all?)
https://patchwork.kernel.org/patch/10225375/



> -               break;
> +               snprintf(fullname, sizeof(fullname), "kernel/configs/%s", name);
> +               if (!conf_read_simple(fullname, S_DEF_USER))
> +                       break;
> +
> +               fprintf(stderr,
> +                       _("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n"),
> +                               name);
> +               exit(1);
>         default:
>                 break;
>         }
> --
> 2.9.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
  2018-02-16 21:41 ` [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing Arnd Bergmann
  2018-02-16 22:03   ` Steven Rostedt
@ 2018-02-20  9:32   ` Masahiro Yamada
  1 sibling, 0 replies; 20+ messages in thread
From: Masahiro Yamada @ 2018-02-20  9:32 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Linux Kbuild mailing list, Yann E . MORIN, Michal Marek,
	Greg Kroah-Hartman, Linux Kernel Mailing List, Steven Rostedt,
	Ingo Molnar

2018-02-17 6:41 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
> This can easily double the time for compiling a driver but does not
> provide any benefit for the compile tester, so it's better left disabled.
>
> In addition, any 'inline' function that is not also 'static' and that
> contains an 'if' causes a warning like
>
> include/linux/string.h:212:2: note: in expansion of macro 'if'
>   if (strscpy(p, q, p_size < q_size ? p_size : q_size) < 0)
>   ^~
> include/linux/compiler.h:162:4: warning: '______f' is static but declared in inline function 'strcpy' which is not static
>
> without this patch, and I could not come up with a nice fix for that.
> In combination with my patch to always enable 'CONFIG_COMPILE_TEST'
> during 'randconfig' builds, we can at least hide these warnings for
> most users.
>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>


I took a look at this just in case
because the subject is prefixed with "Kconfig:",
but it is actually "trace:".

I expect "Kconfig:" for core changes of Kconfig.

Using precise patch prefix would get more attention from
right people.




> ---
>  kernel/trace/Kconfig | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
> index 434c840e2d82..faaf687b13b1 100644
> --- a/kernel/trace/Kconfig
> +++ b/kernel/trace/Kconfig
> @@ -345,6 +345,7 @@ config PROFILE_ANNOTATED_BRANCHES
>  config PROFILE_ALL_BRANCHES
>         bool "Profile all if conditionals"
>         select TRACE_BRANCH_PROFILING
> +       depends on !COMPILE_TEST
>         help
>           This tracer profiles all branch conditions. Every if ()
>           taken in the kernel is recorded whether it hit or miss.
> --
> 2.9.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments
  2018-02-20  9:26   ` Masahiro Yamada
@ 2018-02-20  9:59     ` Arnd Bergmann
  2018-02-20 17:04       ` Masahiro Yamada
  0 siblings, 1 reply; 20+ messages in thread
From: Arnd Bergmann @ 2018-02-20  9:59 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Linux Kbuild mailing list, Yann E. MORIN, Michal Marek,
	Greg Kroah-Hartman, Linux Kernel Mailing List

On Tue, Feb 20, 2018 at 10:26 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2018-02-17 6:41 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:

>> @@ -621,14 +622,24 @@ int main(int ac, char **av)
>>                 case randconfig:        name = "allrandom.config"; break;
>>                 default: break;
>>                 }
>> -               if (conf_read_simple(name, S_DEF_USER) &&
>> -                   conf_read_simple("all.config", S_DEF_USER)) {
>> -                       fprintf(stderr,
>> -                               _("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n"),
>> -                               name);
>> -                       exit(1);
>> +               /* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */
>> +               if (!conf_read_simple(name, S_DEF_USER))
>> +                       break;
>> +               arch = getenv("ARCH");
>> +               if (arch) {
>> +                       snprintf(fullname, sizeof(fullname), "arch/%s/configs/%s",
>> +                                arch, name);
>> +                       if (!conf_read_simple(fullname, S_DEF_USER))
>> +                               break;
>>                 }
>
>
> I am not a big fan of hard-coding the kernel directory structure.
>
> We already do this [1], but I am thinking of kicking this out.
> [1] https://github.com/torvalds/linux/blob/v4.16-rc1/scripts/kconfig/confdata.c#L33

Ok, I see. How about adding a way to detect that we are build-testing
with randconfig instead?

> BTW, I am trying to compiler capability check to Kconfig.
> If this work is done, I hope some "depends on !COMPILE_TEST" will go
> away (but not all?)
> https://patchwork.kernel.org/patch/10225375/

I think most of the 'depends on !COMPILE_TEST' are for cases that
impact compile-testing in some way (more false-positive warnings,
fewer real warnings, much slower compile speed), rather than a lack
of check for a specific compiler version.

We might be able to replace the CONFIG_STANDALONE and
CONFIG_PREVENT_FIRMWARE_BUILD with Kconfig $(shell)
checks if you think that leads to an improvement.

        Arnd

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 3/3] Kconfig: add a default allrandom.config
  2018-02-20  9:16     ` Masahiro Yamada
  (?)
@ 2018-02-20 11:11     ` Arnd Bergmann
  2018-02-20 16:51       ` Masahiro Yamada
  -1 siblings, 1 reply; 20+ messages in thread
From: Arnd Bergmann @ 2018-02-20 11:11 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Linux Kbuild mailing list, Yann E . MORIN, Michal Marek,
	Greg Kroah-Hartman, Linux Kernel Mailing List

On Tue, Feb 20, 2018 at 10:16 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2018-02-17 6:41 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
>> Building randconfig kernels frequently leads to build errors from
>> drivers that have additional build dependencies, or that we don't
>> want to build for compile-testing for some other reason.
>
> Can you fix the former case?

I'm sure it can be done with runtime detection of the tools, not sure if that's
a good idea. What we would need here is

CONFIG_WANXL_BUILD_FIRMWARE: needs to check for as68k and ld68k
CONFIG_AIC79XX_BUILD_FIRMWARE: needs flex, bison and berkeley db.
        I have all those but it still failed to build for me.
STANDALONE now (after the removal of sound/oss) only controls
      ACPI_CUSTOM_DSDT_FILE, we could remove STANDALONE now
      I think, it doesn't actually get in the way of anything. For the stable
      kernels, I'd still need to deal with CONFIG_STANDALONE for the oss
      drivers.

My feeling right now is that we're better off making both STANDALONE
and PREVENT_FIRMWARE_BUILD depend on !COMPILE_TEST,
and then find a way to force that.

>> We already have a couple of compile-time options that can deal with this
>> problem, but the complete set is not documented well.  We also have a
>> method to override some options during 'make randconfig' and a couple
>> of other make targets. However, we don't yet combine those two.
>
>
> From this statement,
>
> kernel/configs/compile_test.config
>
> would make more sense?

My original thought was that the allrandom.config file name is already
documented and I just wanted a more consistent behavior, not come
up with something new.

>> This adds a new allrandom.config file to turn on CONFIG_COMPILE_TEST,
>> CONFIG_STANDALONE and CONFIG_PREVENT_FIRMWARE_BUILD for all randconfig
>> builds. This in turn disables some other options that we already try
>> to avoid in 'allmodconfig' builds and that make no sense in a general
>> randconfig build.
>>
>> Building with 'make randconfig KCONFIG_ALLCONFIG=1' should now always
>
>
> Can you type 'make randconfig compile_test.config', instead?
>
> (Or 'make allmodconfig compile_test.config' for full-build test)
>
>
> The merge_config has a little bit different logic
> from KCONFIG_ALLCONFIG=1, but I guess it leads to the almost same result.

I suspect that one difference in a randconfig kernel is that all options that
depend on CONFIG_COMPILE_TEST have a 50% lower chance of getting
enabled that way, but eventually they will get enabled.

      Arnd

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 3/3] Kconfig: add a default allrandom.config
  2018-02-20 11:11     ` Arnd Bergmann
@ 2018-02-20 16:51       ` Masahiro Yamada
  0 siblings, 0 replies; 20+ messages in thread
From: Masahiro Yamada @ 2018-02-20 16:51 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Linux Kbuild mailing list, Yann E . MORIN, Michal Marek,
	Greg Kroah-Hartman, Linux Kernel Mailing List

2018-02-20 20:11 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
> On Tue, Feb 20, 2018 at 10:16 AM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> 2018-02-17 6:41 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
>>> Building randconfig kernels frequently leads to build errors from
>>> drivers that have additional build dependencies, or that we don't
>>> want to build for compile-testing for some other reason.
>>
>> Can you fix the former case?
>
> I'm sure it can be done with runtime detection of the tools, not sure if that's
> a good idea. What we would need here is
>
> CONFIG_WANXL_BUILD_FIRMWARE: needs to check for as68k and ld68k
> CONFIG_AIC79XX_BUILD_FIRMWARE: needs flex, bison and berkeley db.
>         I have all those but it still failed to build for me.
> STANDALONE now (after the removal of sound/oss) only controls
>       ACPI_CUSTOM_DSDT_FILE, we could remove STANDALONE now
>       I think, it doesn't actually get in the way of anything. For the stable
>       kernels, I'd still need to deal with CONFIG_STANDALONE for the oss
>       drivers.
>
> My feeling right now is that we're better off making both STANDALONE
> and PREVENT_FIRMWARE_BUILD depend on !COMPILE_TEST,
> and then find a way to force that.
>
>>> We already have a couple of compile-time options that can deal with this
>>> problem, but the complete set is not documented well.  We also have a
>>> method to override some options during 'make randconfig' and a couple
>>> of other make targets. However, we don't yet combine those two.
>>
>>
>> From this statement,
>>
>> kernel/configs/compile_test.config
>>
>> would make more sense?
>
> My original thought was that the allrandom.config file name is already
> documented and I just wanted a more consistent behavior, not come
> up with something new.


I thought this would be used by merge_config.sh,
but I understood allrandom.config would be better.



>>> This adds a new allrandom.config file to turn on CONFIG_COMPILE_TEST,
>>> CONFIG_STANDALONE and CONFIG_PREVENT_FIRMWARE_BUILD for all randconfig
>>> builds. This in turn disables some other options that we already try
>>> to avoid in 'allmodconfig' builds and that make no sense in a general
>>> randconfig build.
>>>
>>> Building with 'make randconfig KCONFIG_ALLCONFIG=1' should now always
>>
>>
>> Can you type 'make randconfig compile_test.config', instead?
>>
>> (Or 'make allmodconfig compile_test.config' for full-build test)
>>
>>
>> The merge_config has a little bit different logic
>> from KCONFIG_ALLCONFIG=1, but I guess it leads to the almost same result.
>
> I suspect that one difference in a randconfig kernel is that all options that
> depend on CONFIG_COMPILE_TEST have a 50% lower chance of getting
> enabled that way, but eventually they will get enabled.
>

You are right.

The behavior is slightly different.


-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments
  2018-02-20  9:59     ` Arnd Bergmann
@ 2018-02-20 17:04       ` Masahiro Yamada
  2018-02-21 16:57         ` Arnd Bergmann
  0 siblings, 1 reply; 20+ messages in thread
From: Masahiro Yamada @ 2018-02-20 17:04 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Linux Kbuild mailing list, Yann E. MORIN, Michal Marek,
	Greg Kroah-Hartman, Linux Kernel Mailing List

2018-02-20 18:59 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
> On Tue, Feb 20, 2018 at 10:26 AM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> 2018-02-17 6:41 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
>
>>> @@ -621,14 +622,24 @@ int main(int ac, char **av)
>>>                 case randconfig:        name = "allrandom.config"; break;
>>>                 default: break;
>>>                 }
>>> -               if (conf_read_simple(name, S_DEF_USER) &&
>>> -                   conf_read_simple("all.config", S_DEF_USER)) {
>>> -                       fprintf(stderr,
>>> -                               _("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n"),
>>> -                               name);
>>> -                       exit(1);
>>> +               /* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */
>>> +               if (!conf_read_simple(name, S_DEF_USER))
>>> +                       break;
>>> +               arch = getenv("ARCH");
>>> +               if (arch) {
>>> +                       snprintf(fullname, sizeof(fullname), "arch/%s/configs/%s",
>>> +                                arch, name);
>>> +                       if (!conf_read_simple(fullname, S_DEF_USER))
>>> +                               break;
>>>                 }
>>
>>
>> I am not a big fan of hard-coding the kernel directory structure.
>>
>> We already do this [1], but I am thinking of kicking this out.
>> [1] https://github.com/torvalds/linux/blob/v4.16-rc1/scripts/kconfig/confdata.c#L33
>
> Ok, I see. How about adding a way to detect that we are build-testing
> with randconfig instead?


How about implementing something in scripts/kconfig/Makefile?

merge_config collects config fragments into 'configfiles'

I was thinking of a similar thing.


If KCONFIG_ALLCONFIG is 1,
scripts/kconfig/Makefile searches
'./', '$srctree/', 'arch/$(SRCARCH)/configs/', 'kernel/configs/'
and sets the found file in it,

If KCONFIG_ALLCONFIG already contains a file name, it is used as-is.

scripts/kconfig/conf.c will be simplified.



>> BTW, I am trying to compiler capability check to Kconfig.
>> If this work is done, I hope some "depends on !COMPILE_TEST" will go
>> away (but not all?)
>> https://patchwork.kernel.org/patch/10225375/
>
> I think most of the 'depends on !COMPILE_TEST' are for cases that
> impact compile-testing in some way (more false-positive warnings,
> fewer real warnings, much slower compile speed), rather than a lack
> of check for a specific compiler version.
>
> We might be able to replace the CONFIG_STANDALONE and
> CONFIG_PREVENT_FIRMWARE_BUILD with Kconfig $(shell)
> checks if you think that leads to an improvement.
>

OK, then your kernel/configs/allrandom.config approach will be useful.



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments
  2018-02-20 17:04       ` Masahiro Yamada
@ 2018-02-21 16:57         ` Arnd Bergmann
  2018-02-22 17:27           ` Masahiro Yamada
  2018-02-23  4:13           ` Masahiro Yamada
  0 siblings, 2 replies; 20+ messages in thread
From: Arnd Bergmann @ 2018-02-21 16:57 UTC (permalink / raw)
  To: Masahiro Yamada
  Cc: Linux Kbuild mailing list, Yann E. MORIN, Michal Marek,
	Greg Kroah-Hartman, Linux Kernel Mailing List

On Tue, Feb 20, 2018 at 6:04 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2018-02-20 18:59 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
>> On Tue, Feb 20, 2018 at 10:26 AM, Masahiro Yamada
>> <yamada.masahiro@socionext.com> wrote:
>>> 2018-02-17 6:41 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
>>
>>>> @@ -621,14 +622,24 @@ int main(int ac, char **av)
>>>>                 case randconfig:        name = "allrandom.config"; break;
>>>>                 default: break;
>>>>                 }
>>>> -               if (conf_read_simple(name, S_DEF_USER) &&
>>>> -                   conf_read_simple("all.config", S_DEF_USER)) {
>>>> -                       fprintf(stderr,
>>>> -                               _("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n"),
>>>> -                               name);
>>>> -                       exit(1);
>>>> +               /* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */
>>>> +               if (!conf_read_simple(name, S_DEF_USER))
>>>> +                       break;
>>>> +               arch = getenv("ARCH");
>>>> +               if (arch) {
>>>> +                       snprintf(fullname, sizeof(fullname), "arch/%s/configs/%s",
>>>> +                                arch, name);
>>>> +                       if (!conf_read_simple(fullname, S_DEF_USER))
>>>> +                               break;
>>>>                 }
>>>
>>>
>>> I am not a big fan of hard-coding the kernel directory structure.
>>>
>>> We already do this [1], but I am thinking of kicking this out.
>>> [1] https://github.com/torvalds/linux/blob/v4.16-rc1/scripts/kconfig/confdata.c#L33
>>
>> Ok, I see. How about adding a way to detect that we are build-testing
>> with randconfig instead?
>
>
> How about implementing something in scripts/kconfig/Makefile?
>
> merge_config collects config fragments into 'configfiles'
>
> I was thinking of a similar thing.
>
>
> If KCONFIG_ALLCONFIG is 1,
> scripts/kconfig/Makefile searches
> './', '$srctree/', 'arch/$(SRCARCH)/configs/', 'kernel/configs/'
> and sets the found file in it,
>
> If KCONFIG_ALLCONFIG already contains a file name, it is used as-is.
>
> scripts/kconfig/conf.c will be simplified.

Yes, good idea. I'm struggling a bit with the implementation (fitting
it into the
simple-tagets rule), can you come up with a prototype?

We also need to decide what happens when e.g. both a ./all.config and
arch/${SRCARCH}/configs/allrand.config file exist, should the first path
take precedence or the more specific file?

     Arnd

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments
  2018-02-21 16:57         ` Arnd Bergmann
@ 2018-02-22 17:27           ` Masahiro Yamada
  2018-02-23  4:13           ` Masahiro Yamada
  1 sibling, 0 replies; 20+ messages in thread
From: Masahiro Yamada @ 2018-02-22 17:27 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Linux Kbuild mailing list, Yann E. MORIN, Michal Marek,
	Greg Kroah-Hartman, Linux Kernel Mailing List

2018-02-22 1:57 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
> On Tue, Feb 20, 2018 at 6:04 PM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> 2018-02-20 18:59 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
>>> On Tue, Feb 20, 2018 at 10:26 AM, Masahiro Yamada
>>> <yamada.masahiro@socionext.com> wrote:
>>>> 2018-02-17 6:41 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
>>>
>>>>> @@ -621,14 +622,24 @@ int main(int ac, char **av)
>>>>>                 case randconfig:        name = "allrandom.config"; break;
>>>>>                 default: break;
>>>>>                 }
>>>>> -               if (conf_read_simple(name, S_DEF_USER) &&
>>>>> -                   conf_read_simple("all.config", S_DEF_USER)) {
>>>>> -                       fprintf(stderr,
>>>>> -                               _("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n"),
>>>>> -                               name);
>>>>> -                       exit(1);
>>>>> +               /* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */
>>>>> +               if (!conf_read_simple(name, S_DEF_USER))
>>>>> +                       break;
>>>>> +               arch = getenv("ARCH");
>>>>> +               if (arch) {
>>>>> +                       snprintf(fullname, sizeof(fullname), "arch/%s/configs/%s",
>>>>> +                                arch, name);
>>>>> +                       if (!conf_read_simple(fullname, S_DEF_USER))
>>>>> +                               break;
>>>>>                 }
>>>>
>>>>
>>>> I am not a big fan of hard-coding the kernel directory structure.
>>>>
>>>> We already do this [1], but I am thinking of kicking this out.
>>>> [1] https://github.com/torvalds/linux/blob/v4.16-rc1/scripts/kconfig/confdata.c#L33
>>>
>>> Ok, I see. How about adding a way to detect that we are build-testing
>>> with randconfig instead?
>>
>>
>> How about implementing something in scripts/kconfig/Makefile?
>>
>> merge_config collects config fragments into 'configfiles'
>>
>> I was thinking of a similar thing.
>>
>>
>> If KCONFIG_ALLCONFIG is 1,
>> scripts/kconfig/Makefile searches
>> './', '$srctree/', 'arch/$(SRCARCH)/configs/', 'kernel/configs/'
>> and sets the found file in it,
>>
>> If KCONFIG_ALLCONFIG already contains a file name, it is used as-is.
>>
>> scripts/kconfig/conf.c will be simplified.
>
> Yes, good idea. I'm struggling a bit with the implementation (fitting
> it into the
> simple-tagets rule), can you come up with a prototype?
>
> We also need to decide what happens when e.g. both a ./all.config and
> arch/${SRCARCH}/configs/allrand.config file exist, should the first path
> take precedence or the more specific file?
>

Hmm, maybe the following order?

[1] ./allrandom.config
[2] $(srctree)/allrandom.config
[3] ./all.config
[4] $(srctree)/all.config
[5] kernel/configs/allrandom.config + arch/$(SRCARCH)/configs/allrandom.config


I think you want to include
both kernel/configs/allrandom.config and
arch/$(SRCARCH)/configs/allrandom.config
if exist.

Some options are common, some are arch-specific.

This is how tinyconfig works.


-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments
  2018-02-21 16:57         ` Arnd Bergmann
  2018-02-22 17:27           ` Masahiro Yamada
@ 2018-02-23  4:13           ` Masahiro Yamada
  1 sibling, 0 replies; 20+ messages in thread
From: Masahiro Yamada @ 2018-02-23  4:13 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Linux Kbuild mailing list, Yann E. MORIN, Michal Marek,
	Greg Kroah-Hartman, Linux Kernel Mailing List

2018-02-22 1:57 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
> On Tue, Feb 20, 2018 at 6:04 PM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> 2018-02-20 18:59 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
>>> On Tue, Feb 20, 2018 at 10:26 AM, Masahiro Yamada
>>> <yamada.masahiro@socionext.com> wrote:
>>>> 2018-02-17 6:41 GMT+09:00 Arnd Bergmann <arnd@arndb.de>:
>>>
>>>>> @@ -621,14 +622,24 @@ int main(int ac, char **av)
>>>>>                 case randconfig:        name = "allrandom.config"; break;
>>>>>                 default: break;
>>>>>                 }
>>>>> -               if (conf_read_simple(name, S_DEF_USER) &&
>>>>> -                   conf_read_simple("all.config", S_DEF_USER)) {
>>>>> -                       fprintf(stderr,
>>>>> -                               _("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n"),
>>>>> -                               name);
>>>>> -                       exit(1);
>>>>> +               /* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */
>>>>> +               if (!conf_read_simple(name, S_DEF_USER))
>>>>> +                       break;
>>>>> +               arch = getenv("ARCH");
>>>>> +               if (arch) {
>>>>> +                       snprintf(fullname, sizeof(fullname), "arch/%s/configs/%s",
>>>>> +                                arch, name);
>>>>> +                       if (!conf_read_simple(fullname, S_DEF_USER))
>>>>> +                               break;
>>>>>                 }
>>>>
>>>>
>>>> I am not a big fan of hard-coding the kernel directory structure.
>>>>
>>>> We already do this [1], but I am thinking of kicking this out.
>>>> [1] https://github.com/torvalds/linux/blob/v4.16-rc1/scripts/kconfig/confdata.c#L33
>>>
>>> Ok, I see. How about adding a way to detect that we are build-testing
>>> with randconfig instead?
>>
>>
>> How about implementing something in scripts/kconfig/Makefile?
>>
>> merge_config collects config fragments into 'configfiles'
>>
>> I was thinking of a similar thing.
>>
>>
>> If KCONFIG_ALLCONFIG is 1,
>> scripts/kconfig/Makefile searches
>> './', '$srctree/', 'arch/$(SRCARCH)/configs/', 'kernel/configs/'
>> and sets the found file in it,
>>
>> If KCONFIG_ALLCONFIG already contains a file name, it is used as-is.
>>
>> scripts/kconfig/conf.c will be simplified.
>
> Yes, good idea. I'm struggling a bit with the implementation (fitting
> it into the
> simple-tagets rule), can you come up with a prototype?


Just a prototype for the Makefile part.
(We need to agree with the search order specification, though)


------------>8-------------
configs_dirs := $(srctree)/kernel/configs/ $(srctree)/arch/$(SRCARCH)/configs/

allconfig_files = $(if $(filter 1, $(KCONFIG_ALLCONFIG)),$(firstword \
        $(foreach f, $(1), $(wildcard $(addsuffix $(f), ./ $(srctree)/))) \
        $(foreach f, $(1), $(subst $(space),$(comma),$(wildcard
$(addsuffix $(f), $(configs_dirs)))))), \
        $(KCONFIG_ALLCONFIG))

allyesconfig: override KCONFIG_ALLCONFIG := $(call allconfig_files,
allyes.config all.config)
allmodconfig: override KCONFIG_ALLCONFIG := $(call allconfig_files,
allmoyes.config all.config)
 (likewise for allnoconfig, alldefconfig)
randconfig: override KCONFIG_ALLCONFIG := $(call allconfig_files,
allrandom.config all.config)
------------->8-----------


If a user specifies a file path in KCONFIG_ALLCONFIG, it is used as-is.

IF KCONFIG_ALLCONFIG=1, the input file is searched in the following order.
( [1] - [4] is the same as the current specification)

[1] ./allrandom.config
[2] $(srctree)/allrandom.config
[3] ./all.config
[4] $(srctree)/all.config
[5] $(srctree)/kernel/configs/allrandom.config +
$(srctree)/arch/$(SRCARCH)/configs/allrandom.config
[6] $(srctree)/kernel/configs/all.config +
$(srctree)/arch/$(SRCARCH)/configs/all.config

For [5] and [6], if multiple files are found, KCONFIG_ALLCONFIG
is set to comma-separated file paths.

scripts/kconfig/conf.c must be tweaked a little bit
if we need to read multiple config fragments.

Currently, tinyconfig for x86 comes from common part and arch-specific part.
./kernel/configs/tiny.config
./arch/x86/configs/tiny.config


What do you think?







> We also need to decide what happens when e.g. both a ./all.config and
> arch/${SRCARCH}/configs/allrand.config file exist, should the first path
> take precedence or the more specific file?
>



-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2018-02-23  4:13 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-02-16 21:41 [PATCH 0/3] fixing the last failures in randconfig builds Arnd Bergmann
2018-02-16 21:41 ` [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing Arnd Bergmann
2018-02-16 22:03   ` Steven Rostedt
2018-02-16 22:14     ` Arnd Bergmann
2018-02-16 22:40       ` Arnd Bergmann
2018-02-16 22:50         ` Steven Rostedt
2018-02-17 13:32       ` Greg Kroah-Hartman
2018-02-20  9:32   ` Masahiro Yamada
2018-02-16 21:41 ` [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments Arnd Bergmann
2018-02-20  9:26   ` Masahiro Yamada
2018-02-20  9:59     ` Arnd Bergmann
2018-02-20 17:04       ` Masahiro Yamada
2018-02-21 16:57         ` Arnd Bergmann
2018-02-22 17:27           ` Masahiro Yamada
2018-02-23  4:13           ` Masahiro Yamada
2018-02-16 21:41 ` [PATCH 3/3] Kconfig: add a default allrandom.config Arnd Bergmann
2018-02-20  9:16   ` Masahiro Yamada
2018-02-20  9:16     ` Masahiro Yamada
2018-02-20 11:11     ` Arnd Bergmann
2018-02-20 16:51       ` Masahiro Yamada

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.