[PATCH 0/3] removing unwanted module configs

[PATCH 0/3] removing unwanted module configs

[Steven Rostedt]

As it has been brought up last Kernel Summit, we want to make it easier
for those that report bugs to build their own kernels, and maybe even
bisect with git.  Some of these people are not programmers and do not
understand the complexity of the configuration options. But to compile
a distribution configured kernel on their boxes can take hours.

This patch series comes to the rescue. I wrote the first instance of
streamline config when I bought a new box in 2005 and got frustrated
with finding all the necessary configurations to boot it. It is a
small (yet powerful) perl script.

Here's what it does:

 * Reads the modules that are load by using lsmod.
 * Reads all Makefiles to map modules to CONFIG_* options
 * Reads the Kconfig files to find dependencies and selects
 * Figures out what CONFIGS are needed to compile the loaded modules
 * Reads the .config and prints out a version with all module configurations
    that not needed, disabled.

The next two patches add options to make.

 localmodconfig - this will run streamline_config.pl on the .config file
	and replace it at the end.

 localyesconfig - this will do the same as localmodconfig but will also
	sed -i s/=m/=y/  to turn all modules to core. It will also run
	the 'make oldcondfig' to fix it up and let the user handle
	andything that was changed by converting a module to core.

Anyway, this is now in git and as a series of patches here. My git tree
is based off of the latest Linus git tree.

Have fun!

-- Steve



The following patches are in:

  git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git

    branch: kconfig


Steven Rostedt (3):
      kconfig: add streamline_config.pl to scripts
      kconfig: make localmodconfig to run streamline_config.pl
      kconfig: add make localyesconfig option

----
 scripts/kconfig/Makefile             |   24 +++-
 scripts/kconfig/streamline_config.pl |  291 ++++++++++++++++++++++++++++++++++
 2 files changed, 314 insertions(+), 1 deletions(-)
-- 

[PATCH 1/3] kconfig: add streamline_config.pl to scripts

[Steven Rostedt]

[-- Attachment #1: 0001-kconfig-add-streamline_config.pl-to-scripts.patch --]
[-- Type: text/plain, Size: 8993 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

streamline_config.pl is a very powerful tool. For those that install
a kernel to a new box using the config file from the distribution know that
it can take forever to compile the kernel.

Making a custom config file that will still boot your box, but bring
down the compile time of the kernel can be quit painful, and to ask
someone that reported a bug to do this can be a large burdon since that
person may not even know how to build a kernel.

This script will perform "lsmod" to find all the modules loaded on the
current running system. It will read all the Makefiles to map which
CONFIG enables a module. It will read the Kconfig files to find the
dependencies and selects that may be needed to support a CONFIG.
Finally, it reads the .config file and removes any module "=m" that is
not needed to enable the currently loaded modules. The output goes to
standard out.

Here's a way to run the script. From the Linux directory that holds
a distribution .config.

 $ scripts/kconfig/streamline_config.pl arch/x86/Kconfig > config-sl
 $ mv .config config-save
 $ mv config-sl .config
 $ make oldconfig

Now you have a .config that will still build all your modules, but also
take much less time to build the kernel.

[ Impact: shorten the time for users to build their kernels ]

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/streamline_config.pl |  291 ++++++++++++++++++++++++++++++++++
 1 files changed, 291 insertions(+), 0 deletions(-)
 create mode 100644 scripts/kconfig/streamline_config.pl

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
new file mode 100644
index 0000000..79d8557
--- /dev/null
+++ b/scripts/kconfig/streamline_config.pl
@@ -0,0 +1,291 @@
+#!/usr/bin/perl -w
+#
+# Copywrite 2005-2009 - Steven Rostedt
+# Licensed under the terms of the GNU GPL License version 2
+#
+#  It's simple enough to figure out how this works.
+#  If not, then you can ask me at stripconfig@goodmis.org
+#
+# What it does?
+#
+#   If you have installed a Linux kernel from a distribution
+#   that turns on way too many modules than you need, and
+#   you only want the modules you use, then this program
+#   is perfect for you.
+#
+#   It gives you the ability to turn off all the modules that are
+#   not loaded on your system.
+#
+# Howto:
+#
+#  1. Boot up the kernel that you want to stream line the config on.
+#  2. Change directory to the directory holding the source of the
+#       kernel that you just booted.
+#  3. Copy the configuraton file to this directory as .config
+#  4. Have all your devices that you need modules for connected and
+#      operational (make sure that their corresponding modules are loaded)
+#  5. Run this script redirecting the output to some other file
+#       like config_strip.
+#  6. Back up your old config (if you want too).
+#  7. copy the config_strip file to .config
+#  8. Run "make oldconfig"
+#
+#  Now your kernel is ready to be built with only the modules that
+#  are loaded.
+#
+# Here's what I did with my Debian distribution.
+#
+#    cd /usr/src/linux-2.6.10
+#    cp /boot/config-2.6.10-1-686-smp .config
+#    ~/bin/streamline_config > config_strip
+#    mv .config config_sav
+#    mv config_strip .config
+#    make oldconfig
+#
+my $config = ".config";
+my $linuxpath = ".";
+
+open(CIN,$config) || die "Can't open current config file: $config";
+my @makefiles = `find $linuxpath -name Makefile`;
+my %depends;
+my %selects;
+my %prompts;
+my %objects;
+my $var;
+my $cont = 0;
+
+# Get the top level Kconfig file (passed in)
+my $kconfig = $ARGV[0];
+
+# prevent recursion
+my %read_kconfigs;
+
+sub read_kconfig {
+    my ($kconfig) = @_;
+
+    my $state = "NONE";
+    my $config;
+    my @kconfigs;
+
+    open(KIN, $kconfig) || die "Can't open $kconfig";
+    while (<KIN>) {
+	chomp;
+
+	# collect any Kconfig sources
+	if (/^source\s*"(.*)"/) {
+	    $kconfigs[$#kconfigs+1] = $1;
+	}
+
+	# configs found
+	if (/^\s*config\s+(\S+)\s*$/) {
+	    $state = "NEW";
+	    $config = $1;
+
+	# collect the depends for the config
+	} elsif ($state eq "NEW" && /^\s*depends\s+on\s+(.*)$/) {
+	    $state = "DEP";
+	    $depends{$config} = $1;
+	} elsif ($state eq "DEP" && /^\s*depends\s+on\s+(.*)$/) {
+	    $depends{$config} .= " " . $1;
+
+	# Get the configs that select this config
+	} elsif ($state ne "NONE" && /^\s*select\s+(\S+)/) {
+	    if (defined($selects{$1})) {
+		$selects{$1} .= " " . $config;
+	    } else {
+		$selects{$1} = $config;
+	    }
+
+	# configs without prompts must be selected
+	} elsif ($state ne "NONE" && /^\s*tristate\s\S/) {
+	    # note if the config has a prompt
+	    $prompt{$config} = 1;
+
+	# stop on "help"
+	} elsif (/^\s*help\s*$/) {
+	    $state = "NONE";
+	}
+    }
+    close(KIN);
+
+    # read in any configs that were found.
+    foreach $kconfig (@kconfigs) {
+	if (!defined($read_kconfigs{$kconfig})) {
+	    $read_kconfigs{$kconfig} = 1;
+	    read_kconfig($kconfig);
+	}
+    }
+}
+
+if ($kconfig) {
+    read_kconfig($kconfig);
+}
+
+# Read all Makefiles to map the configs to the objects
+foreach my $makefile (@makefiles) {
+    chomp $makefile;
+
+    open(MIN,$makefile) || die "Can't open $makefile";
+    while (<MIN>) {
+	my $objs;
+
+	# is this a line after a line with a backslash?
+	if ($cont && /(\S.*)$/) {
+	    $objs = $1;
+	}
+	$cont = 0;
+
+	# collect objects after obj-$(CONFIG_FOO_BAR)
+	if (/obj-\$\((CONFIG_[^\)]*)\)\s*[+:]?=\s*(.*)/) {
+	    $var = $1;
+	    $objs = $2;
+	}
+	if (defined($objs)) {
+	    # test if the line ends with a backslash
+	    if ($objs =~ m,(.*)\\$,) {
+		$objs = $1;
+		$cont = 1;
+	    }
+
+	    foreach my $obj (split /\s+/,$objs) {
+		$obj =~ s/-/_/g;
+		if ($obj =~ /(.*)\.o$/) {
+		    # Objects may bes enabled by more than one config.
+		    # Store configs in an array.
+		    my @arr;
+
+		    if (defined($objects{$1})) {
+			@arr = @{$objects{$1}};
+		    }
+
+		    $arr[$#arr+1] = $var;
+
+		    # The objects have a hash mapping to a reference
+		    # of an array of configs.
+		    $objects{$1} = \@arr;
+		}
+	    }
+	}
+    }
+    close(MIN);
+}
+
+my %modules;
+
+# see what modules are loaded on this system
+open(LIN,"/sbin/lsmod|") || die "Cant lsmod";
+while (<LIN>) {
+	next if (/^Module/);  # Skip the first line.
+	if (/^(\S+)/) {
+		$modules{$1} = 1;
+	}
+}
+close (LIN);
+
+# add to the configs hash all configs that are needed to enable
+# a loaded module.
+my %configs;
+foreach my $module (keys(%modules)) {
+    if (defined($objects{$module})) {
+	@arr = @{$objects{$module}};
+	foreach my $conf (@arr) {
+	    $configs{$conf} = $module;
+	}
+    } else {
+	# Most likely, someone has a custom (binary?) module loaded.
+	print STDERR "$module config not found!!\n";
+    }
+}
+
+my $valid = "A-Za-z_0-9";
+my $repeat = 1;
+
+#
+# Note, we do not care about operands (like: &&, ||, !) we want to add any
+# config that is in the depend list of another config. This script does
+# not enable configs that are not already enabled. If we come across a
+# config A that depends on !B, we can still add B to the list of depends
+# to keep on. If A was on in the original config, B would not have been
+# and B would not be turned on by this script.
+#
+sub parse_config_dep_select
+{
+    my ($p) = @_;
+
+    while ($p =~ /[$valid]/) {
+
+	if ($p =~ /^[^$valid]*([$valid]+)/) {
+	    my $conf = "CONFIG_" . $1;
+
+	    $p =~ s/^[^$valid]*[$valid]+//;
+
+	    if (!defined($configs{$conf})) {
+		# We must make sure that this config has its
+		# dependencies met.
+		$repeat = 1; # do again
+		$configs{$conf} = 1;
+	    }
+	} else {
+	    die "this should never happen";
+	}
+    }
+}
+
+while ($repeat) {
+    $repeat = 0;
+
+    foreach my $config (keys %configs) {
+	$config =~ s/^CONFIG_//;
+
+	if (!defined($depends{$config})) {
+	    next;
+	}
+
+	# This config has dependencies. Make sure they are also included
+	parse_config_dep_select $depends{$config};
+
+	if (defined($prompt{$config}) || !defined($selects{$config})) {
+	    next;
+	}
+
+	# config has no prompt and must be selected.
+	parse_config_dep_select $selects{$config};
+    }
+}
+
+my %setconfigs;
+
+# Finally, read the .config file and turn off any module enabled that
+# we could not find a reason to keep enabled.
+while(<CIN>) {
+	if (/^(CONFIG.*)=m/) {
+		if (defined($configs{$1})) {
+		    $setconfigs{$1} = 1;
+		    print;
+		} else {
+		    print "# $1 is not set\n";
+		}
+	} else {
+		print;
+	}
+}
+close(CIN);
+
+# Integrity check, make sure all modules that we want enabled do
+# indeed have their configs set.
+loop:
+foreach my $module (keys(%modules)) {
+    if (defined($objects{$module})) {
+	my @arr = @{$objects{$module}};
+	foreach my $conf (@arr) {
+	    if (defined($setconfigs{$conf})) {
+		next loop;
+	    }
+	}
+	print STDERR "module $module did not have configs";
+	foreach my $conf (@arr) {
+	    print STDERR " " , $conf;
+	}
+	print STDERR "\n";
+    }
+}
-- 
1.6.2.1

-- 

[PATCH 2/3] kconfig: make localmodconfig to run streamline_config.pl

[Steven Rostedt]

[-- Attachment #1: 0002-kconfig-make-localmodconfig-to-run-streamline_confi.patch --]
[-- Type: text/plain, Size: 2308 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

Running the streamline_config.pl script manually can still be confusing
for some users. This patch adds the localmodconfig option. This will
automatically run streamline_config.pl on the current .config and
then run "make silentoldconfig" to fix any wholes that might have been
created.

 $ make localmodconfig

This will remove any module configurations in .config that are not needed
to compile the modules that are loaded.

[ Impact: simplify running streamline_config.pl ]

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/Makefile |   13 ++++++++++++-
 1 files changed, 12 insertions(+), 1 deletions(-)

diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index fa8c2dd..4b289ad 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -2,7 +2,8 @@
 # Kernel configuration targets
 # These targets are used from top-level makefile
 
-PHONY += oldconfig xconfig gconfig menuconfig config silentoldconfig update-po-config
+PHONY += oldconfig xconfig gconfig menuconfig config silentoldconfig update-po-config \
+	localmodconfig
 
 ifdef KBUILD_KCONFIG
 Kconfig := $(KBUILD_KCONFIG)
@@ -28,6 +29,15 @@ oldconfig: $(obj)/conf
 silentoldconfig: $(obj)/conf
 	$< -s $(Kconfig)
 
+localmodconfig: $(obj)/streamline_config.pl $(obj)/conf
+	$(Q)perl $< $(Kconfig) > .tmp.config
+	$(Q)cmp -s .tmp.config .config ||		\
+		(mv -f .config .config.old.1;		\
+		 mv -f .tmp.config .config;		\
+		 $(obj)/conf -s $(Kconfig);		\
+		 mv -f .config.old.1 .config.old)
+	$(Q)rm -f .tmp.config
+
 # Create new linux.pot file
 # Adjust charset to UTF-8 in .po file to accept UTF-8 in Kconfig files
 # The symlink is used to repair a deficiency in arch/um
@@ -83,6 +93,7 @@ help:
 	@echo  '  xconfig	  - Update current config utilising a QT based front-end'
 	@echo  '  gconfig	  - Update current config utilising a GTK based front-end'
 	@echo  '  oldconfig	  - Update current config utilising a provided .config as base'
+	@echo  '  localmodconfig  - Update current config disabling modules not loaded'
 	@echo  '  silentoldconfig - Same as oldconfig, but quietly'
 	@echo  '  randconfig	  - New config with random answer to all options'
 	@echo  '  defconfig	  - New config with default answer to all options'
-- 
1.6.2.1

-- 

[PATCH 3/3] kconfig: add make localyesconfig option

[Steven Rostedt]

[-- Attachment #1: 0003-kconfig-add-make-localyesconfig-option.patch --]
[-- Type: text/plain, Size: 2148 bytes --]

From: Steven Rostedt <srostedt@redhat.com>

This adds the option localyesconfig to make. This is similar to
localmodconfig, but after it removes unnecessary modules it runs

  sed -i s/=m/=y/

on the .config file. It then runs "make silentoldconfig" to fix any
wholes that were created by the conversion of modules to core.

[ Impact: facilitate making a moduleless config ]

Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
 scripts/kconfig/Makefile |   13 ++++++++++++-
 1 files changed, 12 insertions(+), 1 deletions(-)

diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index 4b289ad..c080424 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -3,7 +3,7 @@
 # These targets are used from top-level makefile
 
 PHONY += oldconfig xconfig gconfig menuconfig config silentoldconfig update-po-config \
-	localmodconfig
+	localmodconfig localyesconfig
 
 ifdef KBUILD_KCONFIG
 Kconfig := $(KBUILD_KCONFIG)
@@ -38,6 +38,16 @@ localmodconfig: $(obj)/streamline_config.pl $(obj)/conf
 		 mv -f .config.old.1 .config.old)
 	$(Q)rm -f .tmp.config
 
+localyesconfig: $(obj)/streamline_config.pl
+	$(Q)perl $< $(Kconfig) > .tmp.config
+	$(Q)sed -i s/=m/=y/ .tmp.config
+	$(Q)cmp -s .tmp.config .config ||		\
+		(mv -f .config .config.old.1;		\
+		 mv -f .tmp.config .config;		\
+		 $(obj)/conf -s $(Kconfig);		\
+		 mv -f .config.old.1 .config.old)
+	$(Q)rm -f .tmp.config
+
 # Create new linux.pot file
 # Adjust charset to UTF-8 in .po file to accept UTF-8 in Kconfig files
 # The symlink is used to repair a deficiency in arch/um
@@ -94,6 +104,7 @@ help:
 	@echo  '  gconfig	  - Update current config utilising a GTK based front-end'
 	@echo  '  oldconfig	  - Update current config utilising a provided .config as base'
 	@echo  '  localmodconfig  - Update current config disabling modules not loaded'
+	@echo  '  localyesconfig  - Update current config converting local mods to core'
 	@echo  '  silentoldconfig - Same as oldconfig, but quietly'
 	@echo  '  randconfig	  - New config with random answer to all options'
 	@echo  '  defconfig	  - New config with default answer to all options'
-- 
1.6.2.1

-- 

Re: [PATCH 0/3] removing unwanted module configs

[Ingo Molnar]

* Steven Rostedt <rostedt@goodmis.org> wrote:

> As it has been brought up last Kernel Summit, we want to make it easier
> for those that report bugs to build their own kernels, and maybe even
> bisect with git.  Some of these people are not programmers and do not
> understand the complexity of the configuration options. But to compile
> a distribution configured kernel on their boxes can take hours.
> 
> This patch series comes to the rescue. I wrote the first instance of
> streamline config when I bought a new box in 2005 and got frustrated
> with finding all the necessary configurations to boot it. It is a
> small (yet powerful) perl script.
> 
> Here's what it does:
> 
>  * Reads the modules that are load by using lsmod.
>  * Reads all Makefiles to map modules to CONFIG_* options
>  * Reads the Kconfig files to find dependencies and selects
>  * Figures out what CONFIGS are needed to compile the loaded modules
>  * Reads the .config and prints out a version with all module configurations
>     that not needed, disabled.
> 
> The next two patches add options to make.
> 
>  localmodconfig - this will run streamline_config.pl on the .config file
> 	and replace it at the end.
> 
>  localyesconfig - this will do the same as localmodconfig but will also
> 	sed -i s/=m/=y/  to turn all modules to core. It will also run
> 	the 'make oldcondfig' to fix it up and let the user handle
> 	andything that was changed by converting a module to core.
> 
> Anyway, this is now in git and as a series of patches here. My git 
> tree is based off of the latest Linus git tree.
> 
> Have fun!

Very nice and useful!

I have given it a try - it works to a certain degree, but does not 
seem to work fully. If i boot a Fedora distro kernel with this 
module setup:

aldebaran:~/linux/linux> lsmod
Module                  Size  Used by
sunrpc                253904  1 
ipv6                  349568  50 
cpufreq_ondemand       73248  0 
acpi_cpufreq           75408  0 
freq_table             70400  2 cpufreq_ondemand,acpi_cpufreq
dm_multipath           82256  0 
i2c_i801               75932  0 
i2c_core               87192  1 i2c_i801
serio_raw              71556  0 
pcspkr                 68352  0 
pata_jmicron           69504  0 
iTCO_wdt               78208  0 
shpchp                 99160  0 
iTCO_vendor_support    68868  1 iTCO_wdt
igb                   144412  0 
dca                    71976  1 igb
pata_acpi              70528  0 
ata_generic            71428  0 

note the 'igb' driver that is essential to be picked up. 'make 
localyesconfig' complains:

  module igb did not have configs CONFIG_IGB

But:

  # CONFIG_IGB is not set

It did pick up other essential drivers - such as ext3. Why did it 
miss IGB? Here are all the missed drivers:

aldebaran:~/linux/linux> make localyesconfig
module ata_generic did not have configs CONFIG_ATA_GENERIC
module iTCO_vendor_support did not have configs CONFIG_ITCO_WDT
module shpchp did not have configs CONFIG_HOTPLUG_PCI_SHPC
module acpi_cpufreq did not have configs CONFIG_IA64_ACPI_CPUFREQ CONFIG_X86_ACPI_CPUFREQ
module pata_jmicron did not have configs CONFIG_PATA_JMICRON
module i2c_i801 did not have configs CONFIG_I2C_I801
module serio_raw did not have configs CONFIG_SERIO_RAW
module iTCO_wdt did not have configs CONFIG_ITCO_WDT
module freq_table did not have configs CONFIG_CPU_FREQ_TABLE
module igb did not have configs CONFIG_IGB
module ipv6 did not have configs CONFIG_IPV6
module dca did not have configs CONFIG_DCA
module sunrpc did not have configs CONFIG_SUNRPC
module pata_acpi did not have configs CONFIG_PATA_ACPI
module dm_multipath did not have configs CONFIG_DM_MULTIPATH
module cpufreq_ondemand did not have configs 
CONFIG_CPU_FREQ_GOV_ONDEMAND
module i2c_core did not have configs CONFIG_I2C
module pcspkr did not have configs CONFIG_INPUT_PCSPKR

Of those drivers, igb is the only truly boot-critical one - the box 
wont be very useful if it has no network support.

Another comment: 'make localyesconfig' does not seem to be 
self-invariant. I.e. it does not handle the case well when we 
already have booted a localyesconfig kernel and do 'make 
localyesconfig' again. It will find no modules and will merrily 
create an almost empty .config.

This could be addressed the following way: i think the script should 
implicitly turn on CONFIG_IKCONFIG=y, and should also check for the 
presence of /proc/config.gz and use it as a starting point. This 
makes the whole concept nicely self-invariant.

	Ingo

Re: [PATCH 0/3] removing unwanted module configs

[Steven Rostedt]

On Thu, 30 Apr 2009, Ingo Molnar wrote:

> 
> * Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > As it has been brought up last Kernel Summit, we want to make it easier
> > for those that report bugs to build their own kernels, and maybe even
> > bisect with git.  Some of these people are not programmers and do not
> > understand the complexity of the configuration options. But to compile
> > a distribution configured kernel on their boxes can take hours.
> > 
> > This patch series comes to the rescue. I wrote the first instance of
> > streamline config when I bought a new box in 2005 and got frustrated
> > with finding all the necessary configurations to boot it. It is a
> > small (yet powerful) perl script.
> > 
> > Here's what it does:
> > 
> >  * Reads the modules that are load by using lsmod.
> >  * Reads all Makefiles to map modules to CONFIG_* options
> >  * Reads the Kconfig files to find dependencies and selects
> >  * Figures out what CONFIGS are needed to compile the loaded modules
> >  * Reads the .config and prints out a version with all module configurations
> >     that not needed, disabled.
> > 
> > The next two patches add options to make.
> > 
> >  localmodconfig - this will run streamline_config.pl on the .config file
> > 	and replace it at the end.
> > 
> >  localyesconfig - this will do the same as localmodconfig but will also
> > 	sed -i s/=m/=y/  to turn all modules to core. It will also run
> > 	the 'make oldcondfig' to fix it up and let the user handle
> > 	andything that was changed by converting a module to core.
> > 
> > Anyway, this is now in git and as a series of patches here. My git 
> > tree is based off of the latest Linus git tree.
> > 
> > Have fun!
> 
> Very nice and useful!
> 
> I have given it a try - it works to a certain degree, but does not 
> seem to work fully. If i boot a Fedora distro kernel with this 
> module setup:
> 
> aldebaran:~/linux/linux> lsmod
> Module                  Size  Used by
> sunrpc                253904  1 
> ipv6                  349568  50 
> cpufreq_ondemand       73248  0 
> acpi_cpufreq           75408  0 
> freq_table             70400  2 cpufreq_ondemand,acpi_cpufreq
> dm_multipath           82256  0 
> i2c_i801               75932  0 
> i2c_core               87192  1 i2c_i801
> serio_raw              71556  0 
> pcspkr                 68352  0 
> pata_jmicron           69504  0 
> iTCO_wdt               78208  0 
> shpchp                 99160  0 
> iTCO_vendor_support    68868  1 iTCO_wdt
> igb                   144412  0 
> dca                    71976  1 igb
> pata_acpi              70528  0 
> ata_generic            71428  0 
> 
> note the 'igb' driver that is essential to be picked up. 'make 
> localyesconfig' complains:
> 
>   module igb did not have configs CONFIG_IGB

Did the original .config have it set. It will not enable any modules that 
are not already set.

> 
> But:
> 
>   # CONFIG_IGB is not set
> 
> It did pick up other essential drivers - such as ext3. Why did it 
> miss IGB? Here are all the missed drivers:
> 
> aldebaran:~/linux/linux> make localyesconfig
> module ata_generic did not have configs CONFIG_ATA_GENERIC
> module iTCO_vendor_support did not have configs CONFIG_ITCO_WDT
> module shpchp did not have configs CONFIG_HOTPLUG_PCI_SHPC
> module acpi_cpufreq did not have configs CONFIG_IA64_ACPI_CPUFREQ CONFIG_X86_ACPI_CPUFREQ
> module pata_jmicron did not have configs CONFIG_PATA_JMICRON
> module i2c_i801 did not have configs CONFIG_I2C_I801
> module serio_raw did not have configs CONFIG_SERIO_RAW
> module iTCO_wdt did not have configs CONFIG_ITCO_WDT
> module freq_table did not have configs CONFIG_CPU_FREQ_TABLE
> module igb did not have configs CONFIG_IGB
> module ipv6 did not have configs CONFIG_IPV6
> module dca did not have configs CONFIG_DCA
> module sunrpc did not have configs CONFIG_SUNRPC
> module pata_acpi did not have configs CONFIG_PATA_ACPI
> module dm_multipath did not have configs CONFIG_DM_MULTIPATH
> module cpufreq_ondemand did not have configs 
> CONFIG_CPU_FREQ_GOV_ONDEMAND
> module i2c_core did not have configs CONFIG_I2C
> module pcspkr did not have configs CONFIG_INPUT_PCSPKR

Hmm, I just added the "localyesconfig" and have not played with it much. I 
need to change the integrity check of the script to test for modules that 
are already compiled in (CONFIG_FOO=y). Right now it expects the modules 
to be modules :-/


> 
> Of those drivers, igb is the only truly boot-critical one - the box 
> wont be very useful if it has no network support.
> 
> Another comment: 'make localyesconfig' does not seem to be 
> self-invariant. I.e. it does not handle the case well when we 
> already have booted a localyesconfig kernel and do 'make 
> localyesconfig' again. It will find no modules and will merrily 
> create an almost empty .config.

It should not touch CONFIG_FOO=y. It should only disable the modules that 
are already set but not needed.

I will update the script to not warn about modules that are compiled in.

> 
> This could be addressed the following way: i think the script should 
> implicitly turn on CONFIG_IKCONFIG=y, and should also check for the 
> presence of /proc/config.gz and use it as a starting point. This 
> makes the whole concept nicely self-invariant.

I really want to stay away from enabling anything in the .config scripts. 
I can probably do it for the trivial cases. But there are cases where more 
than one config will enable a module, or it takes several types of 
dependencies to enable it. The script can not know which is the best set 
of dependencies to enable, and I would not want to enable all of them.

But if you have a simple linier script, or no "or" cases. If we have

CONFIG_A depends on CONFIG_X && CONFIG_Y and CONFIG_X and CONFIG_Y do not 
have any depends or needs to be selected, I could enable all of them if 
that is the only way to enable a module. But this would come later.

Right now the goal is to remove modules that are in .config but are not 
used. If CONFIG_IGB above was set (CONFIG_IGB=m) and the script disabled 
it, then that would be a bug.

I will fix the warnings when the module is compiled in.

Thanks!

-- Steve

Re: [PATCH 0/3] removing unwanted module configs

[Ingo Molnar]

* Steven Rostedt <rostedt@goodmis.org> wrote:

> 
> On Thu, 30 Apr 2009, Ingo Molnar wrote:
> 
> > 
> > * Steven Rostedt <rostedt@goodmis.org> wrote:
> > 
> > > As it has been brought up last Kernel Summit, we want to make it easier
> > > for those that report bugs to build their own kernels, and maybe even
> > > bisect with git.  Some of these people are not programmers and do not
> > > understand the complexity of the configuration options. But to compile
> > > a distribution configured kernel on their boxes can take hours.
> > > 
> > > This patch series comes to the rescue. I wrote the first instance of
> > > streamline config when I bought a new box in 2005 and got frustrated
> > > with finding all the necessary configurations to boot it. It is a
> > > small (yet powerful) perl script.
> > > 
> > > Here's what it does:
> > > 
> > >  * Reads the modules that are load by using lsmod.
> > >  * Reads all Makefiles to map modules to CONFIG_* options
> > >  * Reads the Kconfig files to find dependencies and selects
> > >  * Figures out what CONFIGS are needed to compile the loaded modules
> > >  * Reads the .config and prints out a version with all module configurations
> > >     that not needed, disabled.
> > > 
> > > The next two patches add options to make.
> > > 
> > >  localmodconfig - this will run streamline_config.pl on the .config file
> > > 	and replace it at the end.
> > > 
> > >  localyesconfig - this will do the same as localmodconfig but will also
> > > 	sed -i s/=m/=y/  to turn all modules to core. It will also run
> > > 	the 'make oldcondfig' to fix it up and let the user handle
> > > 	andything that was changed by converting a module to core.
> > > 
> > > Anyway, this is now in git and as a series of patches here. My git 
> > > tree is based off of the latest Linus git tree.
> > > 
> > > Have fun!
> > 
> > Very nice and useful!
> > 
> > I have given it a try - it works to a certain degree, but does not 
> > seem to work fully. If i boot a Fedora distro kernel with this 
> > module setup:
> > 
> > aldebaran:~/linux/linux> lsmod
> > Module                  Size  Used by
> > sunrpc                253904  1 
> > ipv6                  349568  50 
> > cpufreq_ondemand       73248  0 
> > acpi_cpufreq           75408  0 
> > freq_table             70400  2 cpufreq_ondemand,acpi_cpufreq
> > dm_multipath           82256  0 
> > i2c_i801               75932  0 
> > i2c_core               87192  1 i2c_i801
> > serio_raw              71556  0 
> > pcspkr                 68352  0 
> > pata_jmicron           69504  0 
> > iTCO_wdt               78208  0 
> > shpchp                 99160  0 
> > iTCO_vendor_support    68868  1 iTCO_wdt
> > igb                   144412  0 
> > dca                    71976  1 igb
> > pata_acpi              70528  0 
> > ata_generic            71428  0 
> > 
> > note the 'igb' driver that is essential to be picked up. 'make 
> > localyesconfig' complains:
> > 
> >   module igb did not have configs CONFIG_IGB
> 
> Did the original .config have it set. It will not enable any 
> modules that are not already set.

no, it wasnt enabled. 

> > But:
> > 
> >   # CONFIG_IGB is not set
> > 
> > It did pick up other essential drivers - such as ext3. Why did it 
> > miss IGB? Here are all the missed drivers:
> > 
> > aldebaran:~/linux/linux> make localyesconfig
> > module ata_generic did not have configs CONFIG_ATA_GENERIC
> > module iTCO_vendor_support did not have configs CONFIG_ITCO_WDT
> > module shpchp did not have configs CONFIG_HOTPLUG_PCI_SHPC
> > module acpi_cpufreq did not have configs CONFIG_IA64_ACPI_CPUFREQ CONFIG_X86_ACPI_CPUFREQ
> > module pata_jmicron did not have configs CONFIG_PATA_JMICRON
> > module i2c_i801 did not have configs CONFIG_I2C_I801
> > module serio_raw did not have configs CONFIG_SERIO_RAW
> > module iTCO_wdt did not have configs CONFIG_ITCO_WDT
> > module freq_table did not have configs CONFIG_CPU_FREQ_TABLE
> > module igb did not have configs CONFIG_IGB
> > module ipv6 did not have configs CONFIG_IPV6
> > module dca did not have configs CONFIG_DCA
> > module sunrpc did not have configs CONFIG_SUNRPC
> > module pata_acpi did not have configs CONFIG_PATA_ACPI
> > module dm_multipath did not have configs CONFIG_DM_MULTIPATH
> > module cpufreq_ondemand did not have configs 
> > CONFIG_CPU_FREQ_GOV_ONDEMAND
> > module i2c_core did not have configs CONFIG_I2C
> > module pcspkr did not have configs CONFIG_INPUT_PCSPKR
> 
> Hmm, I just added the "localyesconfig" and have not played with it 
> much. I need to change the integrity check of the script to test 
> for modules that are already compiled in (CONFIG_FOO=y). Right now 
> it expects the modules to be modules :-/
> 
> > Of those drivers, igb is the only truly boot-critical one - the 
> > box wont be very useful if it has no network support.
> > 
> > Another comment: 'make localyesconfig' does not seem to be 
> > self-invariant. I.e. it does not handle the case well when we 
> > already have booted a localyesconfig kernel and do 'make 
> > localyesconfig' again. It will find no modules and will merrily 
> > create an almost empty .config.
> 
> It should not touch CONFIG_FOO=y. It should only disable the modules that 
> are already set but not needed.
> 
> I will update the script to not warn about modules that are compiled in.
> 
> > This could be addressed the following way: i think the script 
> > should implicitly turn on CONFIG_IKCONFIG=y, and should also 
> > check for the presence of /proc/config.gz and use it as a 
> > starting point. This makes the whole concept nicely 
> > self-invariant.
> 
> I really want to stay away from enabling anything in the .config 
> scripts. I can probably do it for the trivial cases. But there are 
> cases where more than one config will enable a module, or it takes 
> several types of dependencies to enable it. The script can not 
> know which is the best set of dependencies to enable, and I would 
> not want to enable all of them.

CONFIG_IKCONFIG=y would be a one-off thing, only to make the whole 
concept self-hosting. If i boot a localyesconfig kernel, how does 
the script figure out what is built in? It cannot, unless i provide 
it the precise .config - but often that wont be provided. The script 
should clone the environment it is running in really.

	Ingo

Re: [PATCH 0/3] removing unwanted module configs

[Steven Rostedt]

On Thu, 30 Apr 2009, Ingo Molnar wrote:
> 
> CONFIG_IKCONFIG=y would be a one-off thing, only to make the whole 
> concept self-hosting. If i boot a localyesconfig kernel, how does 
> the script figure out what is built in? It cannot, unless i provide 
> it the precise .config - but often that wont be provided. The script 
> should clone the environment it is running in really.

Does IKCONFIG get loaded in the running kernel if IKCONFIG_PROC is not 
set, or does it just sit in the vmlinux file.

Since this option can also be used for helping embedded developers (I used 
it for that) I would not want to bloat the kernel with running a script 
that is suppose to minimize it. But if IKCONFIG && !IKCONFIG_PROC does not 
add more data to the kernel, then I would be happy to turn it on by 
default.

I see I can use scripts/extract-ikconfig to get the config from the 
current image. Now which image should it try?

/boot/vmlinuz-`uname -r`
./vmlinux

I could have it first try /proc/config.gz and if it does not find it then
try the /boot kernel, if it does not find it or does not find a config 
file in it, it would then try ./vmlinux file. If it does not find the file 
or the config then it would just use the local .config.

-- Steve

Re: [PATCH 0/3] removing unwanted module configs

[Steven Rostedt]

On Thu, 30 Apr 2009, Steven Rostedt wrote:
> 
> Since this option can also be used for helping embedded developers (I used 
> it for that) I would not want to bloat the kernel with running a script 
> that is suppose to minimize it. But if IKCONFIG && !IKCONFIG_PROC does not 
> add more data to the kernel, then I would be happy to turn it on by 
> default.

Actually, I can let the user decide. If it sees that IKCONFIG is not set, 
it can remove it from the .config output. When the silentoldconfig runs 
afterwards, it will ask the user if they want to enable it.

-- Steve

Re: [PATCH 0/3] removing unwanted module configs

[Ingo Molnar]

* Steven Rostedt <rostedt@goodmis.org> wrote:

> On Thu, 30 Apr 2009, Steven Rostedt wrote:
> > 
> > Since this option can also be used for helping embedded 
> > developers (I used it for that) I would not want to bloat the 
> > kernel with running a script that is suppose to minimize it. But 
> > if IKCONFIG && !IKCONFIG_PROC does not add more data to the 
> > kernel, then I would be happy to turn it on by default.
> 
> Actually, I can let the user decide. If it sees that IKCONFIG is 
> not set, it can remove it from the .config output. When the 
> silentoldconfig runs afterwards, it will ask the user if they want 
> to enable it.

No, please offer reasonable non-interactive default behavior. It's 
pretty well-defined, and it can be engineered to be self-sufficient 
as well. Why not do that?

	Ingo

Re: [PATCH 0/3] removing unwanted module configs

[Ingo Molnar]

* Steven Rostedt <rostedt@goodmis.org> wrote:

> On Thu, 30 Apr 2009, Ingo Molnar wrote:
> > 
> > CONFIG_IKCONFIG=y would be a one-off thing, only to make the 
> > whole concept self-hosting. If i boot a localyesconfig kernel, 
> > how does the script figure out what is built in? It cannot, 
> > unless i provide it the precise .config - but often that wont be 
> > provided. The script should clone the environment it is running 
> > in really.
> 
> Does IKCONFIG get loaded in the running kernel if IKCONFIG_PROC is 
> not set, or does it just sit in the vmlinux file.
> 
> Since this option can also be used for helping embedded developers 
> (I used it for that) I would not want to bloat the kernel with 
> running a script that is suppose to minimize it. But if IKCONFIG 
> && !IKCONFIG_PROC does not add more data to the kernel, then I 
> would be happy to turn it on by default.

this is a basic usability issue. The main goal is to help regular 
Linux users.

> I see I can use scripts/extract-ikconfig to get the config from 
> the current image. Now which image should it try?

no, enable IKCONFIG_PROC and it will be under /proc/config.gz.

> /boot/vmlinuz-`uname -r`
> ./vmlinux
> 
> I could have it first try /proc/config.gz and if it does not find 
> it then try the /boot kernel, if it does not find it or does not 
> find a config file in it, it would then try ./vmlinux file. If it 
> does not find the file or the config then it would just use the 
> local .config.

yeah, that sounds like a good plan.

	Ingo