[PATCH 00/14] kconfig: streamline distro configs for testers

[PATCH 00/14] kconfig: streamline distro configs for testers

[Steven Rostedt]

Here's my rebase of the prior push of streamline-config.pl.

What this patch series does, is add my streamline-config.pl as a make
option for users. What streamline-config.pl does is to look at the
modules loaded in the system (lsmod) and then scans all the Makefiles
and Kconfigs in the current directory tree to find what configs need
to be set for the modules that are loaded.

It then looks at the .config (if .config is not found, it looks
in /boot, /proc/config.gz, vmlinux, configs.ko for .configs that it
may be able to use). This script will remove all options set as =m that
does not need to be set in order to build the modules found by lsmod.

NOTE: This does not touch any =y option. If you do make allyesconfig
and then run this, it will have not effect anything. Nor does this enable
any thing that is not set. Although we may change this in the future.

This patch series provides:

  make localmodconfig

which will look for a .config to streamline. Then it will run silentoldconfig
to clean up any missing options.

  make localyesconfig

will remove all =m that is not needed to build the modules provided
by lsmod, and then convert the rest to =y. Note, once you do this, no more
changes will be made by these scripts.

This is idea to help testers. Since a lot of people download a kernel
from kernel.org and then uses the distro config to configure it. This can
take hours to build. Now all a tester needs to do, is to download the
kernel from kernel.org and run "make localmodconfig". This will disable
most of the modules that are set in distro kernels and speed up the
build time by magnitudes!

Lets try to get this in soon,

-- Steve


The following patches are in:

  git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git

    branch: kconfig


Steven Rostedt (14):
      kconfig: add streamline_config.pl to scripts
      kconfig: make localmodconfig to run streamline_config.pl
      kconfig: add make localyesconfig option
      kconfig: streamline_config.pl do not stop with no depends
      kconfig: do not warn about modules built in
      kconfig: enable CONFIG_IKCONFIG from streamline_config.pl
      kconfig: add check if end exists in extract-ikconfig
      kconfig: have extract-ikconfig read ELF files
      kconfig: keep config.gz around even if CONFIG_IKCONFIG_PROC is not set
      kconfig: search for a config to base the local(mod|yes)config on
      kconfig: unset IKCONFIG_PROC and clean up nesting
      kconfig: test for /boot/config-uname after /proc/config.gz in localconfig
      kconfig: make local .config default for streamline_config
      kconfig: test if a .config already exists

----
 kernel/Makefile                      |    2 +-
 scripts/extract-ikconfig             |   14 ++
 scripts/kconfig/Makefile             |   34 +++-
 scripts/kconfig/streamline_config.pl |  366 ++++++++++++++++++++++++++++++++++
 4 files changed, 414 insertions(+), 2 deletions(-)
-- 

[PATCH 01/14] kconfig: add streamline_config.pl to scripts

[Steven Rostedt]

[-- Attachment #1: 0001-kconfig-add-streamline_config.pl-to-scripts.patch --]
[-- Type: text/plain, Size: 8930 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

streamline_config.pl is a very powerful tool. For those that install
a kernel to a new box using the config file from the distribution know that
it can take forever to compile the kernel.

Making a custom config file that will still boot your box, but bring
down the compile time of the kernel can be quit painful, and to ask
someone that reported a bug to do this can be a large burdon since that
person may not even know how to build a kernel.

This script will perform "lsmod" to find all the modules loaded on the
current running system. It will read all the Makefiles to map which
CONFIG enables a module. It will read the Kconfig files to find the
dependencies and selects that may be needed to support a CONFIG.
Finally, it reads the .config file and removes any module "=m" that is
not needed to enable the currently loaded modules. The output goes to
standard out.

Here's a way to run the script. From the Linux directory that holds
a distribution .config.

 $ scripts/kconfig/streamline_config.pl arch/x86/Kconfig > config-sl
 $ mv .config config-save
 $ mv config-sl .config
 $ make oldconfig

Now you have a .config that will still build all your modules, but also
take much less time to build the kernel.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/streamline_config.pl |  291 ++++++++++++++++++++++++++++++++++
 1 files changed, 291 insertions(+), 0 deletions(-)
 create mode 100644 scripts/kconfig/streamline_config.pl

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
new file mode 100644
index 0000000..79d8557
--- /dev/null
+++ b/scripts/kconfig/streamline_config.pl
@@ -0,0 +1,291 @@
+#!/usr/bin/perl -w
+#
+# Copywrite 2005-2009 - Steven Rostedt
+# Licensed under the terms of the GNU GPL License version 2
+#
+#  It's simple enough to figure out how this works.
+#  If not, then you can ask me at stripconfig@goodmis.org
+#
+# What it does?
+#
+#   If you have installed a Linux kernel from a distribution
+#   that turns on way too many modules than you need, and
+#   you only want the modules you use, then this program
+#   is perfect for you.
+#
+#   It gives you the ability to turn off all the modules that are
+#   not loaded on your system.
+#
+# Howto:
+#
+#  1. Boot up the kernel that you want to stream line the config on.
+#  2. Change directory to the directory holding the source of the
+#       kernel that you just booted.
+#  3. Copy the configuraton file to this directory as .config
+#  4. Have all your devices that you need modules for connected and
+#      operational (make sure that their corresponding modules are loaded)
+#  5. Run this script redirecting the output to some other file
+#       like config_strip.
+#  6. Back up your old config (if you want too).
+#  7. copy the config_strip file to .config
+#  8. Run "make oldconfig"
+#
+#  Now your kernel is ready to be built with only the modules that
+#  are loaded.
+#
+# Here's what I did with my Debian distribution.
+#
+#    cd /usr/src/linux-2.6.10
+#    cp /boot/config-2.6.10-1-686-smp .config
+#    ~/bin/streamline_config > config_strip
+#    mv .config config_sav
+#    mv config_strip .config
+#    make oldconfig
+#
+my $config = ".config";
+my $linuxpath = ".";
+
+open(CIN,$config) || die "Can't open current config file: $config";
+my @makefiles = `find $linuxpath -name Makefile`;
+my %depends;
+my %selects;
+my %prompts;
+my %objects;
+my $var;
+my $cont = 0;
+
+# Get the top level Kconfig file (passed in)
+my $kconfig = $ARGV[0];
+
+# prevent recursion
+my %read_kconfigs;
+
+sub read_kconfig {
+    my ($kconfig) = @_;
+
+    my $state = "NONE";
+    my $config;
+    my @kconfigs;
+
+    open(KIN, $kconfig) || die "Can't open $kconfig";
+    while (<KIN>) {
+	chomp;
+
+	# collect any Kconfig sources
+	if (/^source\s*"(.*)"/) {
+	    $kconfigs[$#kconfigs+1] = $1;
+	}
+
+	# configs found
+	if (/^\s*config\s+(\S+)\s*$/) {
+	    $state = "NEW";
+	    $config = $1;
+
+	# collect the depends for the config
+	} elsif ($state eq "NEW" && /^\s*depends\s+on\s+(.*)$/) {
+	    $state = "DEP";
+	    $depends{$config} = $1;
+	} elsif ($state eq "DEP" && /^\s*depends\s+on\s+(.*)$/) {
+	    $depends{$config} .= " " . $1;
+
+	# Get the configs that select this config
+	} elsif ($state ne "NONE" && /^\s*select\s+(\S+)/) {
+	    if (defined($selects{$1})) {
+		$selects{$1} .= " " . $config;
+	    } else {
+		$selects{$1} = $config;
+	    }
+
+	# configs without prompts must be selected
+	} elsif ($state ne "NONE" && /^\s*tristate\s\S/) {
+	    # note if the config has a prompt
+	    $prompt{$config} = 1;
+
+	# stop on "help"
+	} elsif (/^\s*help\s*$/) {
+	    $state = "NONE";
+	}
+    }
+    close(KIN);
+
+    # read in any configs that were found.
+    foreach $kconfig (@kconfigs) {
+	if (!defined($read_kconfigs{$kconfig})) {
+	    $read_kconfigs{$kconfig} = 1;
+	    read_kconfig($kconfig);
+	}
+    }
+}
+
+if ($kconfig) {
+    read_kconfig($kconfig);
+}
+
+# Read all Makefiles to map the configs to the objects
+foreach my $makefile (@makefiles) {
+    chomp $makefile;
+
+    open(MIN,$makefile) || die "Can't open $makefile";
+    while (<MIN>) {
+	my $objs;
+
+	# is this a line after a line with a backslash?
+	if ($cont && /(\S.*)$/) {
+	    $objs = $1;
+	}
+	$cont = 0;
+
+	# collect objects after obj-$(CONFIG_FOO_BAR)
+	if (/obj-\$\((CONFIG_[^\)]*)\)\s*[+:]?=\s*(.*)/) {
+	    $var = $1;
+	    $objs = $2;
+	}
+	if (defined($objs)) {
+	    # test if the line ends with a backslash
+	    if ($objs =~ m,(.*)\\$,) {
+		$objs = $1;
+		$cont = 1;
+	    }
+
+	    foreach my $obj (split /\s+/,$objs) {
+		$obj =~ s/-/_/g;
+		if ($obj =~ /(.*)\.o$/) {
+		    # Objects may bes enabled by more than one config.
+		    # Store configs in an array.
+		    my @arr;
+
+		    if (defined($objects{$1})) {
+			@arr = @{$objects{$1}};
+		    }
+
+		    $arr[$#arr+1] = $var;
+
+		    # The objects have a hash mapping to a reference
+		    # of an array of configs.
+		    $objects{$1} = \@arr;
+		}
+	    }
+	}
+    }
+    close(MIN);
+}
+
+my %modules;
+
+# see what modules are loaded on this system
+open(LIN,"/sbin/lsmod|") || die "Cant lsmod";
+while (<LIN>) {
+	next if (/^Module/);  # Skip the first line.
+	if (/^(\S+)/) {
+		$modules{$1} = 1;
+	}
+}
+close (LIN);
+
+# add to the configs hash all configs that are needed to enable
+# a loaded module.
+my %configs;
+foreach my $module (keys(%modules)) {
+    if (defined($objects{$module})) {
+	@arr = @{$objects{$module}};
+	foreach my $conf (@arr) {
+	    $configs{$conf} = $module;
+	}
+    } else {
+	# Most likely, someone has a custom (binary?) module loaded.
+	print STDERR "$module config not found!!\n";
+    }
+}
+
+my $valid = "A-Za-z_0-9";
+my $repeat = 1;
+
+#
+# Note, we do not care about operands (like: &&, ||, !) we want to add any
+# config that is in the depend list of another config. This script does
+# not enable configs that are not already enabled. If we come across a
+# config A that depends on !B, we can still add B to the list of depends
+# to keep on. If A was on in the original config, B would not have been
+# and B would not be turned on by this script.
+#
+sub parse_config_dep_select
+{
+    my ($p) = @_;
+
+    while ($p =~ /[$valid]/) {
+
+	if ($p =~ /^[^$valid]*([$valid]+)/) {
+	    my $conf = "CONFIG_" . $1;
+
+	    $p =~ s/^[^$valid]*[$valid]+//;
+
+	    if (!defined($configs{$conf})) {
+		# We must make sure that this config has its
+		# dependencies met.
+		$repeat = 1; # do again
+		$configs{$conf} = 1;
+	    }
+	} else {
+	    die "this should never happen";
+	}
+    }
+}
+
+while ($repeat) {
+    $repeat = 0;
+
+    foreach my $config (keys %configs) {
+	$config =~ s/^CONFIG_//;
+
+	if (!defined($depends{$config})) {
+	    next;
+	}
+
+	# This config has dependencies. Make sure they are also included
+	parse_config_dep_select $depends{$config};
+
+	if (defined($prompt{$config}) || !defined($selects{$config})) {
+	    next;
+	}
+
+	# config has no prompt and must be selected.
+	parse_config_dep_select $selects{$config};
+    }
+}
+
+my %setconfigs;
+
+# Finally, read the .config file and turn off any module enabled that
+# we could not find a reason to keep enabled.
+while(<CIN>) {
+	if (/^(CONFIG.*)=m/) {
+		if (defined($configs{$1})) {
+		    $setconfigs{$1} = 1;
+		    print;
+		} else {
+		    print "# $1 is not set\n";
+		}
+	} else {
+		print;
+	}
+}
+close(CIN);
+
+# Integrity check, make sure all modules that we want enabled do
+# indeed have their configs set.
+loop:
+foreach my $module (keys(%modules)) {
+    if (defined($objects{$module})) {
+	my @arr = @{$objects{$module}};
+	foreach my $conf (@arr) {
+	    if (defined($setconfigs{$conf})) {
+		next loop;
+	    }
+	}
+	print STDERR "module $module did not have configs";
+	foreach my $conf (@arr) {
+	    print STDERR " " , $conf;
+	}
+	print STDERR "\n";
+    }
+}
-- 
1.6.3.3

-- 

[PATCH 02/14] kconfig: make localmodconfig to run streamline_config.pl

[Steven Rostedt]

[-- Attachment #1: 0002-kconfig-make-localmodconfig-to-run-streamline_config.patch --]
[-- Type: text/plain, Size: 2283 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Running the streamline_config.pl script manually can still be confusing
for some users. This patch adds the localmodconfig option. This will
automatically run streamline_config.pl on the current .config and
then run "make silentoldconfig" to fix any wholes that might have been
created.

 $ make localmodconfig

This will remove any module configurations in .config that are not needed
to compile the modules that are loaded.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/Makefile |   13 ++++++++++++-
 1 files changed, 12 insertions(+), 1 deletions(-)

diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index 5ddf8be..e4d8394 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -2,7 +2,8 @@
 # Kernel configuration targets
 # These targets are used from top-level makefile
 
-PHONY += oldconfig xconfig gconfig menuconfig config silentoldconfig update-po-config
+PHONY += oldconfig xconfig gconfig menuconfig config silentoldconfig update-po-config \
+	localmodconfig
 
 ifdef KBUILD_KCONFIG
 Kconfig := $(KBUILD_KCONFIG)
@@ -28,6 +29,15 @@ oldconfig: $(obj)/conf
 silentoldconfig: $(obj)/conf
 	$< -s $(Kconfig)
 
+localmodconfig: $(obj)/streamline_config.pl $(obj)/conf
+	$(Q)perl $< $(Kconfig) > .tmp.config
+	$(Q)cmp -s .tmp.config .config ||		\
+		(mv -f .config .config.old.1;		\
+		 mv -f .tmp.config .config;		\
+		 $(obj)/conf -s $(Kconfig);		\
+		 mv -f .config.old.1 .config.old)
+	$(Q)rm -f .tmp.config
+
 # Create new linux.pot file
 # Adjust charset to UTF-8 in .po file to accept UTF-8 in Kconfig files
 # The symlink is used to repair a deficiency in arch/um
@@ -83,6 +93,7 @@ help:
 	@echo  '  xconfig	  - Update current config utilising a QT based front-end'
 	@echo  '  gconfig	  - Update current config utilising a GTK based front-end'
 	@echo  '  oldconfig	  - Update current config utilising a provided .config as base'
+	@echo  '  localmodconfig  - Update current config disabling modules not loaded'
 	@echo  '  silentoldconfig - Same as oldconfig, but quietly, additionally update deps'
 	@echo  '  randconfig	  - New config with random answer to all options'
 	@echo  '  defconfig	  - New config with default answer to all options'
-- 
1.6.3.3

-- 

[PATCH 03/14] kconfig: add make localyesconfig option

[Steven Rostedt]

[-- Attachment #1: 0003-kconfig-add-make-localyesconfig-option.patch --]
[-- Type: text/plain, Size: 2123 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

This adds the option localyesconfig to make. This is similar to
localmodconfig, but after it removes unnecessary modules it runs

  sed -i s/=m/=y/

on the .config file. It then runs "make silentoldconfig" to fix any
wholes that were created by the conversion of modules to core.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/Makefile |   13 ++++++++++++-
 1 files changed, 12 insertions(+), 1 deletions(-)

diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index e4d8394..12a4d9e 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -3,7 +3,7 @@
 # These targets are used from top-level makefile
 
 PHONY += oldconfig xconfig gconfig menuconfig config silentoldconfig update-po-config \
-	localmodconfig
+	localmodconfig localyesconfig
 
 ifdef KBUILD_KCONFIG
 Kconfig := $(KBUILD_KCONFIG)
@@ -38,6 +38,16 @@ localmodconfig: $(obj)/streamline_config.pl $(obj)/conf
 		 mv -f .config.old.1 .config.old)
 	$(Q)rm -f .tmp.config
 
+localyesconfig: $(obj)/streamline_config.pl
+	$(Q)perl $< $(Kconfig) > .tmp.config
+	$(Q)sed -i s/=m/=y/ .tmp.config
+	$(Q)cmp -s .tmp.config .config ||		\
+		(mv -f .config .config.old.1;		\
+		 mv -f .tmp.config .config;		\
+		 $(obj)/conf -s $(Kconfig);		\
+		 mv -f .config.old.1 .config.old)
+	$(Q)rm -f .tmp.config
+
 # Create new linux.pot file
 # Adjust charset to UTF-8 in .po file to accept UTF-8 in Kconfig files
 # The symlink is used to repair a deficiency in arch/um
@@ -94,6 +104,7 @@ help:
 	@echo  '  gconfig	  - Update current config utilising a GTK based front-end'
 	@echo  '  oldconfig	  - Update current config utilising a provided .config as base'
 	@echo  '  localmodconfig  - Update current config disabling modules not loaded'
+	@echo  '  localyesconfig  - Update current config converting local mods to core'
 	@echo  '  silentoldconfig - Same as oldconfig, but quietly, additionally update deps'
 	@echo  '  randconfig	  - New config with random answer to all options'
 	@echo  '  defconfig	  - New config with default answer to all options'
-- 
1.6.3.3

-- 

[PATCH 04/14] kconfig: streamline_config.pl do not stop with no depends

[Steven Rostedt]

[-- Attachment #1: 0004-kconfig-streamline_config.pl-do-not-stop-with-no-dep.patch --]
[-- Type: text/plain, Size: 1603 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

If a config does not have a prompt, it must be selected.
streamline_config.pl keeps track of all configs that select other configs.
If a config that does not have a prompt needs to be set to enable a
current module, it will include all configs that select it.
Note, streamline_config.pl does not enable modules that are not already
enabled. It only keeps enabled those that were enabled and might be
needed to compile the current modules.

The code to find the selects of a config is after the code that
adds the depends. But if a config needed selects but had no dependencies,
it would not be set. Because the code would stop before getting to
the select.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/streamline_config.pl |    8 +++-----
 1 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 79d8557..1774905 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -237,13 +237,11 @@ while ($repeat) {
     foreach my $config (keys %configs) {
 	$config =~ s/^CONFIG_//;
 
-	if (!defined($depends{$config})) {
-	    next;
+	if (defined($depends{$config})) {
+	    # This config has dependencies. Make sure they are also included
+	    parse_config_dep_select $depends{$config};
 	}
 
-	# This config has dependencies. Make sure they are also included
-	parse_config_dep_select $depends{$config};
-
 	if (defined($prompt{$config}) || !defined($selects{$config})) {
 	    next;
 	}
-- 
1.6.3.3

-- 

[PATCH 05/14] kconfig: do not warn about modules built in

[Steven Rostedt]

[-- Attachment #1: 0005-kconfig-do-not-warn-about-modules-built-in.patch --]
[-- Type: text/plain, Size: 1279 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

The streamline_config.pl finds all the configs that are needed to
compile the currently loaded modules. After it creates the .config
file, it tests to make sure all the configs that are needed were
set.

It only looks at the configs that are modules, it does not look
at the builtin configs. This causes unnecessary warnings about modules
not being covered.

Reported-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/streamline_config.pl |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 1774905..caac952 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -256,12 +256,14 @@ my %setconfigs;
 # Finally, read the .config file and turn off any module enabled that
 # we could not find a reason to keep enabled.
 while(<CIN>) {
-	if (/^(CONFIG.*)=m/) {
+	if (/^(CONFIG.*)=(m|y)/) {
 		if (defined($configs{$1})) {
-		    $setconfigs{$1} = 1;
+		    $setconfigs{$1} = $2;
 		    print;
-		} else {
+		} elsif ($2 eq "m") {
 		    print "# $1 is not set\n";
+		} else {
+		    print;
 		}
 	} else {
 		print;
-- 
1.6.3.3

-- 

[PATCH 06/14] kconfig: enable CONFIG_IKCONFIG from streamline_config.pl

[Steven Rostedt]

[-- Attachment #1: 0006-kconfig-enable-CONFIG_IKCONFIG-from-streamline_confi.patch --]
[-- Type: text/plain, Size: 1760 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Ingo Molnar suggested that the streamline_config.pl should enable
CONFIG_IKCONFIG to keep the current config in the kernel.
Then we can use scripts/extract-ikconfig to find the current
modules.

This patch changes streamline_config.pl to check if CONFIG_IKCONFIG
is not set, and if it is not, it enables it to be a module.

[ Impact: make current config options easier to find ]

Reported-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/streamline_config.pl |   33 +++++++++++++++++++++++----------
 1 files changed, 23 insertions(+), 10 deletions(-)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index caac952..2334641 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -256,18 +256,31 @@ my %setconfigs;
 # Finally, read the .config file and turn off any module enabled that
 # we could not find a reason to keep enabled.
 while(<CIN>) {
-	if (/^(CONFIG.*)=(m|y)/) {
-		if (defined($configs{$1})) {
-		    $setconfigs{$1} = $2;
-		    print;
-		} elsif ($2 eq "m") {
-		    print "# $1 is not set\n";
-		} else {
-		    print;
-		}
+
+    if (/CONFIG_IKCONFIG/) {
+	if (/# CONFIG_IKCONFIG is not set/) {
+	    # enable IKCONFIG at least as a module
+	    print "CONFIG_IKCONFIG=m\n";
+	    # don't ask about PROC
+	    print "# CONFIG_IKCONFIG is not set\n";
+	} else {
+	    print;
+	}
+	next;
+    }
+
+    if (/^(CONFIG.*)=(m|y)/) {
+	if (defined($configs{$1})) {
+	    $setconfigs{$1} = $2;
+	    print;
+	} elsif ($2 eq "m") {
+	    print "# $1 is not set\n";
 	} else {
-		print;
+	    print;
 	}
+    } else {
+	print;
+    }
 }
 close(CIN);
 
-- 
1.6.3.3

-- 

[PATCH 07/14] kconfig: add check if end exists in extract-ikconfig

[Steven Rostedt]

[-- Attachment #1: 0007-kconfig-add-check-if-end-exists-in-extract-ikconfig.patch --]
[-- Type: text/plain, Size: 709 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Both start and end should be tested for existence before continuing
to parse the config.gz file.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/extract-ikconfig |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/scripts/extract-ikconfig b/scripts/extract-ikconfig
index 72997c3..42d6bce 100755
--- a/scripts/extract-ikconfig
+++ b/scripts/extract-ikconfig
@@ -17,6 +17,10 @@ dump_config() {
 	return
     fi
     end=`$binoffset $file $IKCFG_ED 2>/dev/null`
+    [ "$?" != "0" ] && end="-1"
+    if [ "$end" -eq "-1" ]; then
+	return
+    fi
 
     start=`expr $start + 8`
     size=`expr $end - $start`
-- 
1.6.3.3

-- 

[PATCH 08/14] kconfig: have extract-ikconfig read ELF files

[Steven Rostedt]

[-- Attachment #1: 0008-kconfig-have-extract-ikconfig-read-ELF-files.patch --]
[-- Type: text/plain, Size: 1290 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

It would be nice to use extract-ikconfig to find the congfig.gz
in either vmlinux (not vmlinuz) or configs.ko.

This patch changes the script to also be able to read ELF files directly.

[ Impact: find config.gz in vmlinux and configs.ko ]

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/extract-ikconfig |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/scripts/extract-ikconfig b/scripts/extract-ikconfig
index 42d6bce..de233ff 100755
--- a/scripts/extract-ikconfig
+++ b/scripts/extract-ikconfig
@@ -59,6 +59,8 @@ dump_config "$image"
 GZHDR1="0x1f 0x8b 0x08 0x00"
 GZHDR2="0x1f 0x8b 0x08 0x08"
 
+ELFHDR="0x7f 0x45 0x4c 0x46"
+
 # vmlinux.gz: Check for a compressed images
 off=`$binoffset "$image" $GZHDR1 2>/dev/null`
 [ "$?" != "0" ] && off="-1"
@@ -73,6 +75,14 @@ elif [ "$off" -ne "-1" ]; then
 	(dd ibs="$off" skip=1 count=0 && dd bs=512k) <"$image" 2>/dev/null | \
 		zcat >"$TMPFILE"
 	dump_config "$TMPFILE"
+
+# check if this is simply an ELF file
+else
+	off=`$binoffset "$image" $ELFHDR 2>/dev/null`
+	[ "$?" != "0" ] && off="-1"
+	if [ "$off" -eq "0" ]; then
+		dump_config "$image"
+	fi
 fi
 
 echo "ERROR: Unable to extract kernel configuration information."
-- 
1.6.3.3

-- 

[PATCH 09/14] kconfig: keep config.gz around even if CONFIG_IKCONFIG_PROC is not set

[Steven Rostedt]

[-- Attachment #1: 0009-kconfig-keep-config.gz-around-even-if-CONFIG_IKCONFI.patch --]
[-- Type: text/plain, Size: 1120 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

If CONFIG_IKCONFIG is set but CONFIG_IKCONFIG_PROC is not, then
gcc will optimize the config.gz out, because nobody uses it.

This patch adds "__used" to the config.gz data to keep it around so that
code like extract-ikconfig can still find it.

[ Impact: allow extract-ikconfig to find config.gz ]

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 kernel/Makefile |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/kernel/Makefile b/kernel/Makefile
index 2093a69..d0c84e6 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -119,7 +119,7 @@ $(obj)/config_data.gz: .config FORCE
 	$(call if_changed,gzip)
 
 quiet_cmd_ikconfiggz = IKCFG   $@
-      cmd_ikconfiggz = (echo "static const char kernel_config_data[] = MAGIC_START"; cat $< | scripts/bin2c; echo "MAGIC_END;") > $@
+      cmd_ikconfiggz = (echo "static const char kernel_config_data[] __used = MAGIC_START"; cat $< | scripts/bin2c; echo "MAGIC_END;") > $@
 targets += config_data.h
 $(obj)/config_data.h: $(obj)/config_data.gz FORCE
 	$(call if_changed,ikconfiggz)
-- 
1.6.3.3

-- 

[PATCH 10/14] kconfig: search for a config to base the local(mod|yes)config on

[Steven Rostedt]

[-- Attachment #1: 0010-kconfig-search-for-a-config-to-base-the-local-mod-ye.patch --]
[-- Type: text/plain, Size: 2524 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Instead of using the .config in the local directory. This patch
changes streamline_config.pl to search various locations for a config.

Here's the list and order of search:

  /proc/config.gz
  /boot/vmlinuz-`uname -r`
  vmlinux  # local to the directory
  /lib/modules/`uname -r`/kernel/kernel/configs.ko
  kernel/configs.ko
  kernel/configs.o
  .config

Once it finds a file that contains a config (it checks if the binary
objects have configs first) it then uses it to create the .config
with minimum modules needed.

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/streamline_config.pl |   63 +++++++++++++++++++++++++++++++++-
 1 files changed, 62 insertions(+), 1 deletions(-)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 2334641..9fa3f81 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -45,7 +45,68 @@
 my $config = ".config";
 my $linuxpath = ".";
 
-open(CIN,$config) || die "Can't open current config file: $config";
+my $uname = `uname -r`;
+chomp $uname;
+
+my @searchconfigs = (
+	{
+	    "file" => "/proc/config.gz",
+	    "exec" => "zcat",
+	},
+	{
+	    "file" => "/boot/vmlinuz-$uname",
+	    "exec" => "scripts/extract-ikconfig",
+	    "test" => "scripts/extract-ikconfig",
+	},
+	{
+	    "file" => "vmlinux",
+	    "exec" => "scripts/extract-ikconfig",
+	    "test" => "scripts/extract-ikconfig",
+	},
+	{
+	    "file" => "/lib/modules/$uname/kernel/kernel/configs.ko",
+	    "exec" => "scripts/extract-ikconfig",
+	    "test" => "scripts/extract-ikconfig",
+	},
+	{
+	    "file" => "kernel/configs.ko",
+	    "exec" => "scripts/extract-ikconfig",
+	    "test" => "scripts/extract-ikconfig",
+	},
+	{
+	    "file" => "kernel/configs.o",
+	    "exec" => "scripts/extract-ikconfig",
+	    "test" => "scripts/extract-ikconfig",
+	},
+	{
+	    "file" => ".config",
+	    "exec" => "cat",
+	},
+);
+
+sub find_config {
+    foreach my $conf (@searchconfigs) {
+	my $file = $conf->{"file"};
+
+	next if ( ! -f "$file");
+
+	if (defined($conf->{"test"})) {
+	    `$conf->{"test"} $conf->{"file"} 2>/dev/null`;
+	    next if ($?);
+	}
+
+	my $exec = $conf->{"exec"};
+
+	print STDERR "using config: '$file'\n";
+
+	open(CIN, "$exec $file |") || die "Failed to run $exec $file";
+	return;
+    }
+    die "No config file found";
+}
+
+find_config;
+
 my @makefiles = `find $linuxpath -name Makefile`;
 my %depends;
 my %selects;
-- 
1.6.3.3

-- 

[PATCH 11/14] kconfig: unset IKCONFIG_PROC and clean up nesting

[Steven Rostedt]

[-- Attachment #1: 0011-kconfig-unset-IKCONFIG_PROC-and-clean-up-nesting.patch --]
[-- Type: text/plain, Size: 1196 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Due to cut and paste error IKCONFIG was both set and cleared.
It was suppose to be IKCONFIG_PROC to be cleared.

Also cleaned up if nesting.

Reported-by: Alan Jenkins <sourcejedi.lkml@googlemail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/streamline_config.pl |    9 +++------
 1 files changed, 3 insertions(+), 6 deletions(-)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 9fa3f81..69b7c3f 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -323,7 +323,7 @@ while(<CIN>) {
 	    # enable IKCONFIG at least as a module
 	    print "CONFIG_IKCONFIG=m\n";
 	    # don't ask about PROC
-	    print "# CONFIG_IKCONFIG is not set\n";
+	    print "# CONFIG_IKCONFIG_PROC is not set\n";
 	} else {
 	    print;
 	}
@@ -333,15 +333,12 @@ while(<CIN>) {
     if (/^(CONFIG.*)=(m|y)/) {
 	if (defined($configs{$1})) {
 	    $setconfigs{$1} = $2;
-	    print;
 	} elsif ($2 eq "m") {
 	    print "# $1 is not set\n";
-	} else {
-	    print;
+	    next;
 	}
-    } else {
-	print;
     }
+    print;
 }
 close(CIN);
 
-- 
1.6.3.3

-- 

[PATCH 12/14] kconfig: test for /boot/config-uname after /proc/config.gz in localconfig

[Steven Rostedt]

[-- Attachment #1: 0012-kconfig-test-for-boot-config-uname-after-proc-config.patch --]
[-- Type: text/plain, Size: 863 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Many distros put their config in /boot/config-`uname -r`, add a check
for that right after /proc/config.gz

Reported-by: Alan Jenkins <sourcejedi.lkml@googlemail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/streamline_config.pl |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 69b7c3f..46ca62d 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -54,6 +54,10 @@ my @searchconfigs = (
 	    "exec" => "zcat",
 	},
 	{
+	    "file" => "/boot/config-$uname",
+	    "exec" => "cat",
+	},
+	{
 	    "file" => "/boot/vmlinuz-$uname",
 	    "exec" => "scripts/extract-ikconfig",
 	    "test" => "scripts/extract-ikconfig",
-- 
1.6.3.3

-- 

[PATCH 13/14] kconfig: make local .config default for streamline_config

[Steven Rostedt]

[-- Attachment #1: 0013-kconfig-make-local-.config-default-for-streamline_co.patch --]
[-- Type: text/plain, Size: 1108 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

As Andi Kleen pointed out, most people would expect that the local .config
file to be based for a streamline config. This patch changes the order
of searching for a config file to consider the .config in the local
directory first.

Reported-by: Andi Kleen <andi@firstfloor.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/streamline_config.pl |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 46ca62d..95984db 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -50,6 +50,10 @@ chomp $uname;
 
 my @searchconfigs = (
 	{
+	    "file" => ".config",
+	    "exec" => "cat",
+	},
+	{
 	    "file" => "/proc/config.gz",
 	    "exec" => "zcat",
 	},
@@ -82,10 +86,6 @@ my @searchconfigs = (
 	    "exec" => "scripts/extract-ikconfig",
 	    "test" => "scripts/extract-ikconfig",
 	},
-	{
-	    "file" => ".config",
-	    "exec" => "cat",
-	},
 );
 
 sub find_config {
-- 
1.6.3.3

-- 

[PATCH 14/14] kconfig: test if a .config already exists

[Steven Rostedt]

[-- Attachment #1: 0014-kconfig-test-if-a-.config-already-exists.patch --]
[-- Type: text/plain, Size: 2056 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

If one were to run localmodconfig or localyesconfig without having
a .config already in the file, then the end of the process would give
a warning when it tries to move the old .config to .config.old.

This patch adds a test to check if .config exists and avoid the moves
if it does not.

[ Impact: remove warning after make localmodconfig ]

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/Makefile |   30 ++++++++++++++++++++----------
 1 files changed, 20 insertions(+), 10 deletions(-)

diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index 12a4d9e..915a39a 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -31,21 +31,31 @@ silentoldconfig: $(obj)/conf
 
 localmodconfig: $(obj)/streamline_config.pl $(obj)/conf
 	$(Q)perl $< $(Kconfig) > .tmp.config
-	$(Q)cmp -s .tmp.config .config ||		\
-		(mv -f .config .config.old.1;		\
-		 mv -f .tmp.config .config;		\
-		 $(obj)/conf -s $(Kconfig);		\
-		 mv -f .config.old.1 .config.old)
+	$(Q)if [ -f .config ]; then 				\
+			cmp -s .tmp.config .config ||		\
+			(mv -f .config .config.old.1;		\
+			 mv -f .tmp.config .config;		\
+			 $(obj)/conf -s $(Kconfig);		\
+			 mv -f .config.old.1 .config.old)	\
+	else							\
+			mv -f .tmp.config .config;		\
+			$(obj)/conf -s $(Kconfig);		\
+	fi
 	$(Q)rm -f .tmp.config
 
 localyesconfig: $(obj)/streamline_config.pl
 	$(Q)perl $< $(Kconfig) > .tmp.config
 	$(Q)sed -i s/=m/=y/ .tmp.config
-	$(Q)cmp -s .tmp.config .config ||		\
-		(mv -f .config .config.old.1;		\
-		 mv -f .tmp.config .config;		\
-		 $(obj)/conf -s $(Kconfig);		\
-		 mv -f .config.old.1 .config.old)
+	$(Q)if [ -f .config ]; then 				\
+			cmp -s .tmp.config .config ||		\
+			(mv -f .config .config.old.1;		\
+			 mv -f .tmp.config .config;		\
+			 $(obj)/conf -s $(Kconfig);		\
+			 mv -f .config.old.1 .config.old)	\
+	else							\
+			mv -f .tmp.config .config;		\
+			$(obj)/conf -s $(Kconfig);		\
+	fi
 	$(Q)rm -f .tmp.config
 
 # Create new linux.pot file
-- 
1.6.3.3

-- 

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Greg KH]

On Tue, Aug 18, 2009 at 10:56:34PM -0400, Steven Rostedt wrote:
> Here's my rebase of the prior push of streamline-config.pl.

Very nice, thanks for pushing this forward, it's much needed.

greg k-h

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Andrew Morton]

On Tue, 18 Aug 2009 22:56:34 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> Here's my rebase of the prior push of streamline-config.pl.

ow, you made me look at perl code.

> The following patches are in:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
> 
>     branch: kconfig

Sam's been quiet lately.  I suggest that you add this to linux-next
now so people get a chance to poke at it.

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Steven Rostedt]

On Thu, 20 Aug 2009, Andrew Morton wrote:

> On Tue, 18 Aug 2009 22:56:34 -0400
> Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > Here's my rebase of the prior push of streamline-config.pl.
> 
> ow, you made me look at perl code.

What? Pearls are gentle on the eyes. Much better than looking at snakes.

> 
> > The following patches are in:
> > 
> >   git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
> > 
> >     branch: kconfig
> 
> Sam's been quiet lately.  I suggest that you add this to linux-next
> now so people get a chance to poke at it.

I have no means of pushing into next. Would someone like to do a pull for 
me?

-- Steve

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Andrew Morton]

On Thu, 20 Aug 2009 17:43:38 -0400 (EDT)
Steven Rostedt <rostedt@goodmis.org> wrote:

> > 
> > > The following patches are in:
> > > 
> > >   git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
> > > 
> > >     branch: kconfig
> > 
> > Sam's been quiet lately.  I suggest that you add this to linux-next
> > now so people get a chance to poke at it.
> 
> I have no means of pushing into next. Would someone like to do a pull for 
> me?

If suitably bribed, Stephen will add you tree directly to the linux-nxt
lineup.

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Amerigo Wang]

On Tue, Aug 18, 2009 at 10:56:34PM -0400, Steven Rostedt wrote:
>Here's my rebase of the prior push of streamline-config.pl.
>
>What this patch series does, is add my streamline-config.pl as a make
>option for users. What streamline-config.pl does is to look at the
>modules loaded in the system (lsmod) and then scans all the Makefiles
>and Kconfigs in the current directory tree to find what configs need
>to be set for the modules that are loaded.
>
>It then looks at the .config (if .config is not found, it looks
>in /boot, /proc/config.gz, vmlinux, configs.ko for .configs that it
>may be able to use). This script will remove all options set as =m that
>does not need to be set in order to build the modules found by lsmod.
>
>NOTE: This does not touch any =y option. If you do make allyesconfig
>and then run this, it will have not effect anything. Nor does this enable
>any thing that is not set. Although we may change this in the future.
>
>This patch series provides:
>
>  make localmodconfig
>
>which will look for a .config to streamline. Then it will run silentoldconfig
>to clean up any missing options.
>
>  make localyesconfig
>
>will remove all =m that is not needed to build the modules provided
>by lsmod, and then convert the rest to =y. Note, once you do this, no more
>changes will be made by these scripts.
>
>This is idea to help testers. Since a lot of people download a kernel
>from kernel.org and then uses the distro config to configure it. This can
>take hours to build. Now all a tester needs to do, is to download the
>kernel from kernel.org and run "make localmodconfig". This will disable
>most of the modules that are set in distro kernels and speed up the
>build time by magnitudes!

Awesome!!

If I understand this correctly, this is what many people desired to
have to configure kernel automatically.

I am just a little bit confused by the names "streamline" and
"localXXconfig", how about "autoconfig" ?

Thanks!

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Ingo Molnar]

* Andrew Morton <akpm@linux-foundation.org> wrote:

> On Thu, 20 Aug 2009 17:43:38 -0400 (EDT)
> Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > > 
> > > > The following patches are in:
> > > > 
> > > >   git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
> > > > 
> > > >     branch: kconfig
> > > 
> > > Sam's been quiet lately.  I suggest that you add this to linux-next
> > > now so people get a chance to poke at it.
> > 
> > I have no means of pushing into next. Would someone like to do a pull for 
> > me?
> 
> If suitably bribed, Stephen will add you tree directly to the 
> linux-nxt lineup.

Yep, we always needed another kconfig maintainer (Sam's being mostly 
busy with kbuild), and it's nice to see that Steve volunteered for 
that job now! ;-)

	Ingo

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Ingo Molnar]

* Steven Rostedt <rostedt@goodmis.org> wrote:

> Here's my rebase of the prior push of streamline-config.pl.
> 
> What this patch series does, is add my streamline-config.pl as a 
> make option for users. What streamline-config.pl does is to look 
> at the modules loaded in the system (lsmod) and then scans all the 
> Makefiles and Kconfigs in the current directory tree to find what 
> configs need to be set for the modules that are loaded.
> 
> It then looks at the .config (if .config is not found, it looks in 
> /boot, /proc/config.gz, vmlinux, configs.ko for .configs that it 
> may be able to use). This script will remove all options set as =m 
> that does not need to be set in order to build the modules found 
> by lsmod.
> 
> NOTE: This does not touch any =y option. If you do make 
> allyesconfig and then run this, it will have not effect anything. 
> Nor does this enable any thing that is not set. Although we may 
> change this in the future.
> 
> This patch series provides:
> 
>   make localmodconfig
> 
> which will look for a .config to streamline. Then it will run 
> silentoldconfig to clean up any missing options.
> 
>   make localyesconfig
> 
> will remove all =m that is not needed to build the modules 
> provided by lsmod, and then convert the rest to =y. Note, once you 
> do this, no more changes will be made by these scripts.
> 
> This is idea to help testers. Since a lot of people download a 
> kernel from kernel.org and then uses the distro config to 
> configure it. This can take hours to build. Now all a tester needs 
> to do, is to download the kernel from kernel.org and run "make 
> localmodconfig". This will disable most of the modules that are 
> set in distro kernels and speed up the build time by magnitudes!
> 
> Lets try to get this in soon,

btw., the way i tested this in the past (and which i dont have time 
to test right now) is that i took a plain Fedora box with a regular 
rpm kernel booted up and typed:

	make localyesconfig

and built it and then 1) checked the size of the .config (it should 
be much smaller than allyesconfig) and 2) tried to boot the 
resulting bzImage.

It either didnt boot in the past, or had a very large .config. It 
would be nice if you could check whether it does on a box of yours. 
(maybe you already did that!)

	Ingo

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Steven Rostedt]

On Fri, 21 Aug 2009, Ingo Molnar wrote:
> 
> btw., the way i tested this in the past (and which i dont have time 
> to test right now) is that i took a plain Fedora box with a regular 
> rpm kernel booted up and typed:
> 
> 	make localyesconfig
> 
> and built it and then 1) checked the size of the .config (it should 
> be much smaller than allyesconfig) and 2) tried to boot the 
> resulting bzImage.

Did you check the size without the # comments?

> 
> It either didnt boot in the past, or had a very large .config. It 
> would be nice if you could check whether it does on a box of yours. 
> (maybe you already did that!)

I went back to your old config and I could not reproduce it. But I'll try 
again with a new kernel rpm, and do the following:

# rpm -Uvh <some new kernel>
# reboot    # to that new kernel
$ git clone <latest-linus.git> linus.git
$ cd linus.git
$ make localyesconfig

and see what happens. Is this a good test for you?

Thanks,

-- Steve

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Ingo Molnar]

* Steven Rostedt <rostedt@goodmis.org> wrote:

> 
> On Fri, 21 Aug 2009, Ingo Molnar wrote:
> > 
> > btw., the way i tested this in the past (and which i dont have time 
> > to test right now) is that i took a plain Fedora box with a regular 
> > rpm kernel booted up and typed:
> > 
> > 	make localyesconfig
> > 
> > and built it and then 1) checked the size of the .config (it should 
> > be much smaller than allyesconfig) and 2) tried to boot the 
> > resulting bzImage.
> 
> Did you check the size without the # comments?

hm, i dont remember precisely - but maybe i only considered the size 
of the file. Your patch would expand everything and then turn off 
the leafs, leaving a large (but still small ;-) .config in place, 
right?

> > It either didnt boot in the past, or had a very large .config. 
> > It would be nice if you could check whether it does on a box of 
> > yours. (maybe you already did that!)
> 
> I went back to your old config and I could not reproduce it. But I'll try 
> again with a new kernel rpm, and do the following:
> 
> # rpm -Uvh <some new kernel>
> # reboot    # to that new kernel
> $ git clone <latest-linus.git> linus.git
> $ cd linus.git
> $ make localyesconfig
> 
> and see what happens. Is this a good test for you?

Yes, of course. The whole patch-set is lovely and the sooner we put 
this upstream the better.

	Ingo

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Steven Rostedt]

On Fri, 21 Aug 2009, Ingo Molnar wrote:

> 
> * Andrew Morton <akpm@linux-foundation.org> wrote:
> 
> > On Thu, 20 Aug 2009 17:43:38 -0400 (EDT)
> > Steven Rostedt <rostedt@goodmis.org> wrote:
> > 
> > > > 
> > > > > The following patches are in:
> > > > > 
> > > > >   git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
> > > > > 
> > > > >     branch: kconfig
> > > > 
> > > > Sam's been quiet lately.  I suggest that you add this to linux-next
> > > > now so people get a chance to poke at it.
> > > 
> > > I have no means of pushing into next. Would someone like to do a pull for 
> > > me?
> > 
> > If suitably bribed, Stephen will add you tree directly to the 
> > linux-nxt lineup.
> 
> Yep, we always needed another kconfig maintainer (Sam's being mostly 
> busy with kbuild), and it's nice to see that Steve volunteered for 
> that job now! ;-)

Sure why not?

I've created a new repo:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-kconfig.git

and the same changes are in the branch: for-next

I still need to do the testing.

-- Steve

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Steven Rostedt]

On Fri, 21 Aug 2009, Ingo Molnar wrote:
> * Steven Rostedt <rostedt@goodmis.org> wrote:
> > I went back to your old config and I could not reproduce it. But I'll try 
> > again with a new kernel rpm, and do the following:
> > 
> > # rpm -Uvh <some new kernel>
> > # reboot    # to that new kernel
> > $ git clone <latest-linus.git> linus.git
> > $ cd linus.git
> > $ make localyesconfig
> > 
> > and see what happens. Is this a good test for you?
> 
> Yes, of course. The whole patch-set is lovely and the sooner we put 
> this upstream the better.

OK, I found the bug. There's a dependency missing.

$ make localyesconfig
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/basic/docproc
  HOSTCC  scripts/basic/hash
using config: '/boot/config-2.6.27.25-78.2.56.fc9.x86_64'
/bin/sh: line 8: scripts/kconfig/conf: No such file or directory
make[1]: *** [localyesconfig] Error 127
make: *** [localyesconfig] Error 2

Thus the script failed to run. But the sed command that converts the '=m' 
to '=y' still ran. This gives us a distro config with all modules 
converted to built in!

I'll fix this.

Thanks,

-- Steve

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Steven Rostedt]

On Fri, 21 Aug 2009, Steven Rostedt wrote:
> 
> OK, I found the bug. There's a dependency missing.
> 
> $ make localyesconfig
>   HOSTCC  scripts/basic/fixdep
>   HOSTCC  scripts/basic/docproc
>   HOSTCC  scripts/basic/hash
> using config: '/boot/config-2.6.27.25-78.2.56.fc9.x86_64'
> /bin/sh: line 8: scripts/kconfig/conf: No such file or directory
> make[1]: *** [localyesconfig] Error 127
> make: *** [localyesconfig] Error 2
> 
> Thus the script failed to run. But the sed command that converts the '=m' 
> to '=y' still ran. This gives us a distro config with all modules 
> converted to built in!

Funny that localmodconfig does not have this bug.

-- Steve

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Ingo Molnar]

* Steven Rostedt <rostedt@goodmis.org> wrote:

> 
> On Fri, 21 Aug 2009, Steven Rostedt wrote:
> > 
> > OK, I found the bug. There's a dependency missing.
> > 
> > $ make localyesconfig
> >   HOSTCC  scripts/basic/fixdep
> >   HOSTCC  scripts/basic/docproc
> >   HOSTCC  scripts/basic/hash
> > using config: '/boot/config-2.6.27.25-78.2.56.fc9.x86_64'
> > /bin/sh: line 8: scripts/kconfig/conf: No such file or directory
> > make[1]: *** [localyesconfig] Error 127
> > make: *** [localyesconfig] Error 2
> > 
> > Thus the script failed to run. But the sed command that converts the '=m' 
> > to '=y' still ran. This gives us a distro config with all modules 
> > converted to built in!
> 
> Funny that localmodconfig does not have this bug.

... add to that i dont use modules much so i probably hit that 
localyesconfig bug ;-)

btw: mind adding localrandconfig too? It would be a perfect tool to 
randomize all configs except a 'must have' core set of options.

( Or is this already possible via some existing kconfig feature? I
  remember having seen something related, did that ever get
  upstream? A quick grep in scripts/ shows nothing but i might have
  missed it.)

	Ingo

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Ingo Molnar]

* Steven Rostedt <rostedt@goodmis.org> wrote:

> 
> On Fri, 21 Aug 2009, Ingo Molnar wrote:
> 
> > 
> > * Andrew Morton <akpm@linux-foundation.org> wrote:
> > 
> > > On Thu, 20 Aug 2009 17:43:38 -0400 (EDT)
> > > Steven Rostedt <rostedt@goodmis.org> wrote:
> > > 
> > > > > 
> > > > > > The following patches are in:
> > > > > > 
> > > > > >   git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
> > > > > > 
> > > > > >     branch: kconfig
> > > > > 
> > > > > Sam's been quiet lately.  I suggest that you add this to linux-next
> > > > > now so people get a chance to poke at it.
> > > > 
> > > > I have no means of pushing into next. Would someone like to do a pull for 
> > > > me?
> > > 
> > > If suitably bribed, Stephen will add you tree directly to the 
> > > linux-nxt lineup.
> > 
> > Yep, we always needed another kconfig maintainer (Sam's being mostly 
> > busy with kbuild), and it's nice to see that Steve volunteered for 
> > that job now! ;-)
> 
> Sure why not?
> 
> I've created a new repo:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-kconfig.git
> 
> and the same changes are in the branch: for-next

Cool!

I'm wondering, what do you think about Vegard's 'warn about complex 
selects' feature for kconfig which he submitted some time ago:

  http://lkml.org/lkml/2008/5/4/13

That's a really sore kconfig spot we have. It's not a bug, but it 
sure bites people again and again, and it would be nice to have some 
built-in mechanism that gently nudges us away from complex selects. 
(many of which are inadvertent or become complex after the fact)

	Ingo

[PATCH] kconfig: add missing dependency of conf to localyesconfig

[Steven Rostedt]

The following patch is in:

  git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-kconfig.git

    branch: for-next


Steven Rostedt (1):
      kconfig: add missing dependency of conf to localyesconfig

----
 scripts/kconfig/Makefile |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
---------------------------
commit 48586218b6515b9bd70694e3cd8c901a6a6ee69c
Author: Steven Rostedt <srostedt@redhat.com>
Date:   Fri Aug 21 12:42:20 2009 -0400

    kconfig: add missing dependency of conf to localyesconfig
    
    There's a dependency missing.
    
    $ make localyesconfig
      HOSTCC  scripts/basic/fixdep
      HOSTCC  scripts/basic/docproc
      HOSTCC  scripts/basic/hash
    using config: '/boot/config-2.6.27.25-78.2.56.fc9.x86_64'
    /bin/sh: line 8: scripts/kconfig/conf: No such file or directory
    make[1]: *** [localyesconfig] Error 127
    make: *** [localyesconfig] Error 2
    
    Thus the script failed to run. But the sed command that converts the '=m'
    to '=y' still ran. This gives us a distro config with all modules
    converted to built in!
    
    The missing dependency was for conf for localyesconfig. This
    dependency was already set for localmodconfig.
    
    Reported-by: Ingo Molnar <mingo@elte.hu>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>

diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index 915a39a..6d69c7c 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -43,7 +43,7 @@ localmodconfig: $(obj)/streamline_config.pl $(obj)/conf
 	fi
 	$(Q)rm -f .tmp.config
 
-localyesconfig: $(obj)/streamline_config.pl
+localyesconfig: $(obj)/streamline_config.pl $(obj)/conf
 	$(Q)perl $< $(Kconfig) > .tmp.config
 	$(Q)sed -i s/=m/=y/ .tmp.config
 	$(Q)if [ -f .config ]; then 				\

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Steven Rostedt]

On Fri, 21 Aug 2009, Ingo Molnar wrote:

> 
> * Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > 
> > On Fri, 21 Aug 2009, Steven Rostedt wrote:
> > > 
> > > OK, I found the bug. There's a dependency missing.
> > > 
> > > $ make localyesconfig
> > >   HOSTCC  scripts/basic/fixdep
> > >   HOSTCC  scripts/basic/docproc
> > >   HOSTCC  scripts/basic/hash
> > > using config: '/boot/config-2.6.27.25-78.2.56.fc9.x86_64'
> > > /bin/sh: line 8: scripts/kconfig/conf: No such file or directory
> > > make[1]: *** [localyesconfig] Error 127
> > > make: *** [localyesconfig] Error 2
> > > 
> > > Thus the script failed to run. But the sed command that converts the '=m' 
> > > to '=y' still ran. This gives us a distro config with all modules 
> > > converted to built in!
> > 
> > Funny that localmodconfig does not have this bug.
> 
> ... add to that i dont use modules much so i probably hit that 
> localyesconfig bug ;-)
> 
> btw: mind adding localrandconfig too? It would be a perfect tool to 
> randomize all configs except a 'must have' core set of options.
> 
> ( Or is this already possible via some existing kconfig feature? I
>   remember having seen something related, did that ever get
>   upstream? A quick grep in scripts/ shows nothing but i might have
>   missed it.)

OK, I'll add that to my todo list. Note, it the local config is only as 
good as the starting .config. That is, it never touches =y settings, 
because it only uses loaded modules as a base. Thus if something is set to 
=y it will always be set to =y.

What would you have in mind. Just randomly enabling modules? If we enable 
a "y" config then the next local*config will still keep that set.

-- Steve

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Steven Rostedt]

On Fri, 21 Aug 2009, Ingo Molnar wrote:
> * Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> I'm wondering, what do you think about Vegard's 'warn about complex 
> selects' feature for kconfig which he submitted some time ago:
> 
>   http://lkml.org/lkml/2008/5/4/13
> 
> That's a really sore kconfig spot we have. It's not a bug, but it 
> sure bites people again and again, and it would be nice to have some 
> built-in mechanism that gently nudges us away from complex selects. 
> (many of which are inadvertent or become complex after the fact)

I'll take a look at that too. We need to be careful about kicking out too 
many warnings. Perhaps a "make configcheck" is in order that will do 
various checks. This is very similar to my check for select dependency 
error patch:

  http://lkml.org/lkml/2009/2/21/2

I may take a look at getting both of these in (or a merge of the two).

-- Steve

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Ingo Molnar]

* Steven Rostedt <rostedt@goodmis.org> wrote:

> 
> On Fri, 21 Aug 2009, Ingo Molnar wrote:
> 
> > 
> > * Steven Rostedt <rostedt@goodmis.org> wrote:
> > 
> > > 
> > > On Fri, 21 Aug 2009, Steven Rostedt wrote:
> > > > 
> > > > OK, I found the bug. There's a dependency missing.
> > > > 
> > > > $ make localyesconfig
> > > >   HOSTCC  scripts/basic/fixdep
> > > >   HOSTCC  scripts/basic/docproc
> > > >   HOSTCC  scripts/basic/hash
> > > > using config: '/boot/config-2.6.27.25-78.2.56.fc9.x86_64'
> > > > /bin/sh: line 8: scripts/kconfig/conf: No such file or directory
> > > > make[1]: *** [localyesconfig] Error 127
> > > > make: *** [localyesconfig] Error 2
> > > > 
> > > > Thus the script failed to run. But the sed command that converts the '=m' 
> > > > to '=y' still ran. This gives us a distro config with all modules 
> > > > converted to built in!
> > > 
> > > Funny that localmodconfig does not have this bug.
> > 
> > ... add to that i dont use modules much so i probably hit that 
> > localyesconfig bug ;-)
> > 
> > btw: mind adding localrandconfig too? It would be a perfect tool to 
> > randomize all configs except a 'must have' core set of options.
> > 
> > ( Or is this already possible via some existing kconfig feature? I
> >   remember having seen something related, did that ever get
> >   upstream? A quick grep in scripts/ shows nothing but i might have
> >   missed it.)
> 
> OK, I'll add that to my todo list. Note, it the local config is 
> only as good as the starting .config. That is, it never touches =y 
> settings, because it only uses loaded modules as a base. Thus if 
> something is set to =y it will always be set to =y.
> 
> What would you have in mind. Just randomly enabling modules? If we 
> enable a "y" config then the next local*config will still keep 
> that set.

Why - isnt local*config picking up environmental (currently running 
kernel) properties? I.e. it would always set those things that are 
needed - and randomize the rest. Have i misunderstood how it works?

	Ingo

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Steven Rostedt]

On Fri, 21 Aug 2009, Ingo Molnar wrote:
> > OK, I'll add that to my todo list. Note, it the local config is 
> > only as good as the starting .config. That is, it never touches =y 
> > settings, because it only uses loaded modules as a base. Thus if 
> > something is set to =y it will always be set to =y.
> > 
> > What would you have in mind. Just randomly enabling modules? If we 
> > enable a "y" config then the next local*config will still keep 
> > that set.
> 
> Why - isnt local*config picking up environmental (currently running 
> kernel) properties? I.e. it would always set those things that are 
> needed - and randomize the rest. Have i misunderstood how it works?

Yes, this is a simple script. The only thing it does to find out what is 
configured is run "lsmod". Others have tried (in vain) to be able to look 
into things like lspci and lsusb to figure out what options to enable, 
just to end up with something so complex that it doesn't work.

It truly would be great to be able to have something examine the box and 
be able to enable everything that it finds. But that is not what I set out 
to do. I only wanted to disable the hundreds of modules that a distro 
gives us so I can have a streamed line config that will boot my system and 
build in a decent time.

I've been using this script since 2005 and it has worked out great. I even 
posted it after I first wrote it:

  http://lkml.org/lkml/2005/3/11/39

This seems to be a very useful tool. I don't want to prevent it from going 
further because we try to make it do too much. Lets get what we have now 
in, and then we can look at alternatives.

-- Steve

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Greg KH]

On Fri, Aug 21, 2009 at 04:12:18PM -0400, Steven Rostedt wrote:
> 
> This seems to be a very useful tool. I don't want to prevent it from going 
> further because we try to make it do too much. Lets get what we have now 
> in, and then we can look at alternatives.

I totally agree.  It is useful as-is and should go in for the
functionality it has today.

thanks,

greg k-h

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Steven Rostedt]

On Fri, 21 Aug 2009, Steven Rostedt wrote:

> 
> On Fri, 21 Aug 2009, Ingo Molnar wrote:
> > > OK, I'll add that to my todo list. Note, it the local config is 
> > > only as good as the starting .config. That is, it never touches =y 
> > > settings, because it only uses loaded modules as a base. Thus if 
> > > something is set to =y it will always be set to =y.
> > > 
> > > What would you have in mind. Just randomly enabling modules? If we 
> > > enable a "y" config then the next local*config will still keep 
> > > that set.
> > 
> > Why - isnt local*config picking up environmental (currently running 
> > kernel) properties? I.e. it would always set those things that are 
> > needed - and randomize the rest. Have i misunderstood how it works?
> 
> Yes, this is a simple script. The only thing it does to find out what is 
> configured is run "lsmod". Others have tried (in vain) to be able to look 
> into things like lspci and lsusb to figure out what options to enable, 
> just to end up with something so complex that it doesn't work.

Just to make this clearer. Since the script only uses lsmod to figure out 
what options are configured, only touching "=m" options is safe. There's 
no good way to know if a "=y" option can be disabled, so I just leave them 
as is. Unfortunatly, once you run localyesconfig, the script will stop 
modifying it, since there are no more "=m" options, nor will you have 
modules listed in "lsmod".

-- Steve

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Ingo Molnar]

* Steven Rostedt <rostedt@goodmis.org> wrote:

> 
> On Fri, 21 Aug 2009, Steven Rostedt wrote:
> 
> > 
> > On Fri, 21 Aug 2009, Ingo Molnar wrote:
> > > > OK, I'll add that to my todo list. Note, it the local config is 
> > > > only as good as the starting .config. That is, it never touches =y 
> > > > settings, because it only uses loaded modules as a base. Thus if 
> > > > something is set to =y it will always be set to =y.
> > > > 
> > > > What would you have in mind. Just randomly enabling modules? If we 
> > > > enable a "y" config then the next local*config will still keep 
> > > > that set.
> > > 
> > > Why - isnt local*config picking up environmental (currently running 
> > > kernel) properties? I.e. it would always set those things that are 
> > > needed - and randomize the rest. Have i misunderstood how it works?
> > 
> > Yes, this is a simple script. The only thing it does to find out what is 
> > configured is run "lsmod". Others have tried (in vain) to be able to look 
> > into things like lspci and lsusb to figure out what options to enable, 
> > just to end up with something so complex that it doesn't work.
> 
> Just to make this clearer. Since the script only uses lsmod to 
> figure out what options are configured, only touching "=m" options 
> is safe. There's no good way to know if a "=y" option can be 
> disabled, so I just leave them as is. Unfortunatly, once you run 
> localyesconfig, the script will stop modifying it, since there are 
> no more "=m" options, nor will you have modules listed in "lsmod".

ah - indeed, i forgot that it bases its decision off /proc/config, 
if available, right?

Fair enough - forget my suggestion. It's good as-is.

	Ingo

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Steven Rostedt]

On Fri, 21 Aug 2009, Ingo Molnar wrote:
> 
> ah - indeed, i forgot that it bases its decision off /proc/config, 
> if available, right?

Here's the list in order of seach:

1. local .config file  (in tree root)
2. /proc/config.gz
3. /boot/config-`uname -r`
4. /boot/vmlinuz-`uname -r` (extracts .config if configured in)
5. vmlinux (local built vmlinux if .config is configured in)
6. /lib/modules/`uname -r`/kernel/kernel/configs.ko
7. kernel/configs.ko
8. kernel/configs.o

I originally had /proc/config.gz but Andi Kleen correctly pointed out that 
it would be more intuitive if we use the local .config first. This would 
be something that I myself would expect. If you don't want the local 
.config used, then simply delete it.

> 
> Fair enough - forget my suggestion. It's good as-is.

Great!

Thanks,

-- Steve

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Ingo Molnar]

* Steven Rostedt <rostedt@goodmis.org> wrote:

> On Fri, 21 Aug 2009, Ingo Molnar wrote:
> > 
> > ah - indeed, i forgot that it bases its decision off /proc/config, 
> > if available, right?
> 
> Here's the list in order of seach:
> 
> 1. local .config file  (in tree root)
> 2. /proc/config.gz
> 3. /boot/config-`uname -r`
> 4. /boot/vmlinuz-`uname -r` (extracts .config if configured in)
> 5. vmlinux (local built vmlinux if .config is configured in)
> 6. /lib/modules/`uname -r`/kernel/kernel/configs.ko
> 7. kernel/configs.ko
> 8. kernel/configs.o
> 
> I originally had /proc/config.gz but Andi Kleen correctly pointed 
> out that it would be more intuitive if we use the local .config 
> first. This would be something that I myself would expect. If you 
> don't want the local .config used, then simply delete it.

Well, but there's already a mechanism to include configs as a 
must-have list: KCONFIG_ALLCONFIG=.config.

	Ingo

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Steven Rostedt]

On Fri, 21 Aug 2009, Ingo Molnar wrote:

> 
> * Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > On Fri, 21 Aug 2009, Ingo Molnar wrote:
> > > 
> > > ah - indeed, i forgot that it bases its decision off /proc/config, 
> > > if available, right?
> > 
> > Here's the list in order of seach:
> > 
> > 1. local .config file  (in tree root)
> > 2. /proc/config.gz
> > 3. /boot/config-`uname -r`
> > 4. /boot/vmlinuz-`uname -r` (extracts .config if configured in)
> > 5. vmlinux (local built vmlinux if .config is configured in)
> > 6. /lib/modules/`uname -r`/kernel/kernel/configs.ko
> > 7. kernel/configs.ko
> > 8. kernel/configs.o
> > 
> > I originally had /proc/config.gz but Andi Kleen correctly pointed 
> > out that it would be more intuitive if we use the local .config 
> > first. This would be something that I myself would expect. If you 
> > don't want the local .config used, then simply delete it.
> 
> Well, but there's already a mechanism to include configs as a 
> must-have list: KCONFIG_ALLCONFIG=.config.

That's something I can add to try first.

-- Steve

Re: [PATCH 00/14] kconfig: streamline distro configs for testers

[Sam Ravnborg]

On Thu, Aug 20, 2009 at 02:30:25PM -0700, Andrew Morton wrote:
> On Tue, 18 Aug 2009 22:56:34 -0400
> Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > Here's my rebase of the prior push of streamline-config.pl.
> 
> ow, you made me look at perl code.
> 
> > The following patches are in:
> > 
> >   git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
> > 
> >     branch: kconfig
> 
> Sam's been quiet lately.  I suggest that you add this to linux-next
> now so people get a chance to poke at it.

Busy at day time job so no spare time left for Linux stuff recently.
Will get back in a few weeks if we get the release out in time.

	Sam