[PATCH 0/4] kconfig: streamline_config.pl: optionally accept lsmod output

[PATCH 0/4] kconfig: streamline_config.pl: optionally accept lsmod output

[Alan Jenkins]

On 11/13/09, Luis R. Rodriguez <lrodriguez@atheros.com> wrote:
> On Fri, Nov 13, 2009 at 04:02:04AM -0800, Alan Jenkins wrote:
>> On 11/13/09, Luis R. Rodriguez <lrodriguez@atheros.com> wrote:
>> > On Thu, Nov 12, 2009 at 3:31 PM, Alan Jenkins
>> > <sourcejedi.lkml@googlemail.com> wrote:
>> >> On 11/12/09, Luis R. Rodriguez <lrodriguez@atheros.com> wrote:
>> >>>
>> >>> The new 'make localmodconfig' proves very useful but on low end
>> >>> systems we do not want to 'git clone' an entire kernel tree or
>> >>> download a whole kernel but just cross compile the kernel on a
>> >>> bigger machine. The .config generated with 'make localmodconfig'
>> >>> is still very helpfup though
>> >>
>> >> When I tried this for my netbook, my approach was to copy
>> >> /proc/modules and /boot/config from the netbook.  As a shameless hack,
>> >> I temporarily bind mounted /proc/modules on the build system.  I guess
>> >> the cleaner way would be to copy the result of "lsmod", and have
>> >> localmodconfig accept "LSMOD_FILE=file".
>> >>
>> >> Naturally I think my approach is simpler, despite not having actually
>> >> implemented it properly :).  What do you think about it?
>
> Patches welcomed :)

Here goes :).  The first three patches are unrelated fixes.  Patch 4 implements
"make localmodconfig MODULES_LIST=lsmod.out".

1. kconfig: streamline_config.pl: "use strict" (and fix the errors)
2. kconfig: streamline_config.pl: fix out-of-tree builds
3. kconfig: streamline_config.pl: add handling for "if" statements in Kconfig
4. kconfig: streamline_config.pl: optionally accept lsmod output

Regards
Alan

[PATCH 1/4] kconfig: streamline_config.pl: "use strict" (and fix the errors)

[Alan Jenkins]

Global symbol "%prompt" requires explicit package name at scripts/kconfig/streamline_config.pl line 178.
 -> Use "prompts" instead, which is what we actually declared.

Global symbol "@arr" requires explicit package name at scripts/kconfig/streamline_config.pl line 266.
 -> Declare "arr" as a local variable.

Signed-off-by: Alan Jenkins <alan-jenkins@tuffmail.co.uk>
---
 scripts/kconfig/streamline_config.pl |   11 +++++++----
 1 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 95984db..7d898e3 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -1,4 +1,7 @@
-#!/usr/bin/perl -w
+#!/usr/bin/perl
+use strict;
+use warnings;
+
 #
 # Copywrite 2005-2009 - Steven Rostedt
 # Licensed under the terms of the GNU GPL License version 2
@@ -164,7 +167,7 @@ sub read_kconfig {
 	# configs without prompts must be selected
 	} elsif ($state ne "NONE" && /^\s*tristate\s\S/) {
 	    # note if the config has a prompt
-	    $prompt{$config} = 1;
+	    $prompts{$config} = 1;
 
 	# stop on "help"
 	} elsif (/^\s*help\s*$/) {
@@ -252,7 +255,7 @@ close (LIN);
 my %configs;
 foreach my $module (keys(%modules)) {
     if (defined($objects{$module})) {
-	@arr = @{$objects{$module}};
+	my @arr = @{$objects{$module}};
 	foreach my $conf (@arr) {
 	    $configs{$conf} = $module;
 	}
@@ -307,7 +310,7 @@ while ($repeat) {
 	    parse_config_dep_select $depends{$config};
 	}
 
-	if (defined($prompt{$config}) || !defined($selects{$config})) {
+	if (defined($prompts{$config}) || !defined($selects{$config})) {
 	    next;
 	}
 
-- 
1.6.3.3

[PATCH 2/4] kconfig: streamline_config.pl: fix out-of-tree builds

[Alan Jenkins]

We should look for Kconfig under $srctree, just like zconf.l does.
This allows "make localmodconfig" to work when using a separate build
directory (e.g. make O=../build).

Signed-off-by: Alan Jenkins <alan-jenkins@tuffmail.co.uk>
---
 scripts/kconfig/streamline_config.pl |   10 +++++++---
 1 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 7d898e3..7a7bcf7 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -46,7 +46,7 @@ use warnings;
 #    make oldconfig
 #
 my $config = ".config";
-my $linuxpath = ".";
+my $srctree = $ENV{"srctree"} || ".";
 
 my $uname = `uname -r`;
 chomp $uname;
@@ -114,7 +114,7 @@ sub find_config {
 
 find_config;
 
-my @makefiles = `find $linuxpath -name Makefile`;
+my @makefiles = `find $srctree -name Makefile`;
 my %depends;
 my %selects;
 my %prompts;
@@ -135,7 +135,11 @@ sub read_kconfig {
     my $config;
     my @kconfigs;
 
-    open(KIN, $kconfig) || die "Can't open $kconfig";
+    open(KIN, $kconfig) || (
+	$kconfig =~ "^[^/]" &&
+	open(KIN, $srctree . "/" . $kconfig)
+    ) || die "Can't open $kconfig";
+
     while (<KIN>) {
 	chomp;
 
-- 
1.6.3.3

[PATCH 3/4] kconfig: streamline_config.pl: add handling for "if" statements in Kconfig

[Alan Jenkins]

I found that certain modules e.g. snd_hda_intel were not being enabled
by "make localmodconfig".  stream_config.pl was unaware of the
dependencies implied by the "if" statements which surround many configs.

1) Maintain a global stack of the dependencies implied by "if" statements
   These dependencies can then added to each config symbol as it is
   created.

2) Refactor read_kconfig() to use more immediate recursion
   This is necessary to ensure correct handling of "source" statements
   which are nested within "if" statements.

 * Declare $_ as a local variable, so that it is not clobbered by
   recursive calls to read_kconfig().

 * Similarly, we change the global file handle KIN to an indirect file
   handle $kin, which we can then declare as a local variable.

Signed-off-by: Alan Jenkins <alan-jenkins@tuffmail.co.uk>
---
 scripts/kconfig/streamline_config.pl |   37 ++++++++++++++++++++-------------
 1 files changed, 22 insertions(+), 15 deletions(-)

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 7a7bcf7..f5b44c1 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -128,30 +128,45 @@ my $kconfig = $ARGV[0];
 # prevent recursion
 my %read_kconfigs;
 
+# stack of dependencies implied by "if" statements
+my @if_deps;
+
 sub read_kconfig {
     my ($kconfig) = @_;
 
     my $state = "NONE";
     my $config;
-    my @kconfigs;
+    my $kin;
+    my $_;
 
-    open(KIN, $kconfig) || (
+    open($kin, $kconfig) || (
 	$kconfig =~ "^[^/]" &&
-	open(KIN, $srctree . "/" . $kconfig)
+	open($kin, $srctree . "/" . $kconfig)
     ) || die "Can't open $kconfig";
 
-    while (<KIN>) {
+    while (<$kin>) {
 	chomp;
 
-	# collect any Kconfig sources
+	# collect the conditions of "if" statements
+	if (/^if\s+(\S+)\s*$/) {
+	    push(@if_deps, $1);
+	} elsif (/^endif/) {
+	    pop(@if_deps);
+	}
+
+	# read in any Kconfig sources
 	if (/^source\s*"(.*)"/) {
-	    $kconfigs[$#kconfigs+1] = $1;
+	    if (!defined($read_kconfigs{$1})) {
+		$read_kconfigs{$1} = 1;
+		read_kconfig($1);
+	    }
 	}
 
 	# configs found
 	if (/^\s*config\s+(\S+)\s*$/) {
 	    $state = "NEW";
 	    $config = $1;
+	    $depends{$config} = join(" ", @if_deps);
 
 	# collect the depends for the config
 	} elsif ($state eq "NEW" && /^\s*depends\s+on\s+(.*)$/) {
@@ -178,15 +193,7 @@ sub read_kconfig {
 	    $state = "NONE";
 	}
     }
-    close(KIN);
-
-    # read in any configs that were found.
-    foreach $kconfig (@kconfigs) {
-	if (!defined($read_kconfigs{$kconfig})) {
-	    $read_kconfigs{$kconfig} = 1;
-	    read_kconfig($kconfig);
-	}
-    }
+    close($kin);
 }
 
 if ($kconfig) {
-- 
1.6.3.3

[PATCH 4/4] kconfig: streamline_config.pl: let users specify the list of desired modules

[Alan Jenkins]

This allows "make localmodconfig MODULES_LIST=lsmod.out",
where "lsmod.out" may have been created on a different system.

You can also use this with a manually editted file which
simply lists the desired modules.

Signed-off-by: Alan Jenkins <alan-jenkins@tuffmail.co.uk>
---
 README                               |    8 ++++++++
 scripts/kconfig/Makefile             |    3 ++-
 scripts/kconfig/streamline_config.pl |   17 ++++++++++++++---
 3 files changed, 24 insertions(+), 4 deletions(-)

diff --git a/README b/README
index 737838f..9481dd5 100644
--- a/README
+++ b/README
@@ -175,6 +175,14 @@ CONFIGURING the kernel:
 			   Like above, but avoids cluttering the screen
 			   with questions already answered.
 			   Additionally updates the dependencies.
+
+	"make localmodconfig"
+			   Minimize your existing ./.config file, based
+			   on the kernel modules currently loaded on your
+			   system.  You can also specify the list of
+			   modules manually by appending an option
+			   "MODULES_LIST=lsmod.out".
+
 	"make defconfig"   Create a ./.config file by using the default
 			   symbol values from either arch/$ARCH/defconfig
 			   or arch/$ARCH/configs/${PLATFORM}_defconfig,
diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index 6d69c7c..6a1e41d 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -30,7 +30,7 @@ silentoldconfig: $(obj)/conf
 	$< -s $(Kconfig)
 
 localmodconfig: $(obj)/streamline_config.pl $(obj)/conf
-	$(Q)perl $< $(Kconfig) > .tmp.config
+	$(Q)perl $< $(Kconfig) $(MODULES_LIST) > .tmp.config
 	$(Q)if [ -f .config ]; then 				\
 			cmp -s .tmp.config .config ||		\
 			(mv -f .config .config.old.1;		\
@@ -114,6 +114,7 @@ help:
 	@echo  '  gconfig	  - Update current config utilising a GTK based front-end'
 	@echo  '  oldconfig	  - Update current config utilising a provided .config as base'
 	@echo  '  localmodconfig  - Update current config disabling modules not loaded'
+	@echo  '                    (or modules not listed in the file MODULES_LIST)'
 	@echo  '  localyesconfig  - Update current config converting local mods to core'
 	@echo  '  silentoldconfig - Same as oldconfig, but quietly, additionally update deps'
 	@echo  '  randconfig	  - New config with random answer to all options'
diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index f5b44c1..9fde7fd 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -40,7 +40,7 @@ use warnings;
 #
 #    cd /usr/src/linux-2.6.10
 #    cp /boot/config-2.6.10-1-686-smp .config
-#    ~/bin/streamline_config > config_strip
+#    ~/bin/streamline_config arch/x86/Kconfig > config_strip
 #    mv .config config_sav
 #    mv config_strip .config
 #    make oldconfig
@@ -122,9 +122,15 @@ my %objects;
 my $var;
 my $cont = 0;
 
+if ($#ARGV < 0 || $#ARGV > 1) {
+    die "usage: streamline_config.pl <Kconfig> [<modules.list>]"
+}
+
 # Get the top level Kconfig file (passed in)
 my $kconfig = $ARGV[0];
 
+my $modules_list = $ARGV[1];
+
 # prevent recursion
 my %read_kconfigs;
 
@@ -251,8 +257,13 @@ foreach my $makefile (@makefiles) {
 
 my %modules;
 
-# see what modules are loaded on this system
-open(LIN,"/sbin/lsmod|") || die "Cant lsmod";
+# see what modules are loaded
+if ($modules_list) {
+    open(LIN,$modules_list) || die "Can't open $modules_list";
+} else {
+    open(LIN,"/sbin/lsmod|") || die "Can't lsmod";
+}
+
 while (<LIN>) {
 	next if (/^Module/);  # Skip the first line.
 	if (/^(\S+)/) {
-- 
1.6.3.3

Re: [PATCH 2/4] kconfig: streamline_config.pl: fix out-of-tree builds

[Luis R. Rodriguez]

On Fri, Nov 20, 2009 at 07:50:52AM -0800, Alan Jenkins wrote:
> We should look for Kconfig under $srctree, just like zconf.l does.
> This allows "make localmodconfig" to work when using a separate build
> directory (e.g. make O=../build).
> 
> Signed-off-by: Alan Jenkins <alan-jenkins@tuffmail.co.uk>
> ---
>  scripts/kconfig/streamline_config.pl |   10 +++++++---
>  1 files changed, 7 insertions(+), 3 deletions(-)
> 
> diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
> index 7d898e3..7a7bcf7 100644
> --- a/scripts/kconfig/streamline_config.pl
> +++ b/scripts/kconfig/streamline_config.pl
> @@ -46,7 +46,7 @@ use warnings;
>  #    make oldconfig
>  #
>  my $config = ".config";
> -my $linuxpath = ".";
> +my $srctree = $ENV{"srctree"} || ".";
> 
>  my $uname = `uname -r`;
>  chomp $uname;
> @@ -114,7 +114,7 @@ sub find_config {
> 
>  find_config;
> 
> -my @makefiles = `find $linuxpath -name Makefile`;
> +my @makefiles = `find $srctree -name Makefile`;
>  my %depends;
>  my %selects;
>  my %prompts;
> @@ -135,7 +135,11 @@ sub read_kconfig {
>      my $config;
>      my @kconfigs;
> 
> -    open(KIN, $kconfig) || die "Can't open $kconfig";
> +    open(KIN, $kconfig) || (
> +       $kconfig =~ "^[^/]" &&
> +       open(KIN, $srctree . "/" . $kconfig)
> +    ) || die "Can't open $kconfig";
> +

God I hate perl, can this be a little perl-un-obfuscated a bit?

  Luis

Re: [PATCH 2/4] kconfig: streamline_config.pl: fix out-of-tree builds

[Alan Jenkins]

On 11/20/09, Luis R. Rodriguez <lrodriguez@atheros.com> wrote:
> On Fri, Nov 20, 2009 at 07:50:52AM -0800, Alan Jenkins wrote:
>> We should look for Kconfig under $srctree, just like zconf.l does.
>> This allows "make localmodconfig" to work when using a separate build
>> directory (e.g. make O=../build).
>>
>> Signed-off-by: Alan Jenkins <alan-jenkins@tuffmail.co.uk>
>> ---
>>  scripts/kconfig/streamline_config.pl |   10 +++++++---
>>  1 files changed, 7 insertions(+), 3 deletions(-)
>>
>> diff --git a/scripts/kconfig/streamline_config.pl
>> b/scripts/kconfig/streamline_config.pl
>> index 7d898e3..7a7bcf7 100644
>> --- a/scripts/kconfig/streamline_config.pl
>> +++ b/scripts/kconfig/streamline_config.pl
>> @@ -46,7 +46,7 @@ use warnings;
>>  #    make oldconfig
>>  #
>>  my $config = ".config";
>> -my $linuxpath = ".";
>> +my $srctree = $ENV{"srctree"} || ".";
>>
>>  my $uname = `uname -r`;
>>  chomp $uname;
>> @@ -114,7 +114,7 @@ sub find_config {
>>
>>  find_config;
>>
>> -my @makefiles = `find $linuxpath -name Makefile`;
>> +my @makefiles = `find $srctree -name Makefile`;
>>  my %depends;
>>  my %selects;
>>  my %prompts;
>> @@ -135,7 +135,11 @@ sub read_kconfig {
>>      my $config;
>>      my @kconfigs;
>>
>> -    open(KIN, $kconfig) || die "Can't open $kconfig";
>> +    open(KIN, $kconfig) || (
>> +       $kconfig =~ "^[^/]" &&
>> +       open(KIN, $srctree . "/" . $kconfig)
>> +    ) || die "Can't open $kconfig";
>> +
>
> God I hate perl, can this be a little perl-un-obfuscated a bit?

Agreed.  I'll simplify it to

$kconfig = $srctree . "/" . $kconfig;
open(KIN, $kconfig) || die "Can't open $kconfig";

I'll re-send when I've figured out how to stop CONFIG_ATA=m being
deselected.  (It looks like I need to handle "menuconfig" as well as
"config").

Thanks
Alan

Re: [PATCH 4/4] kconfig: streamline_config.pl: let users specify the list of desired modules

[Luis R. Rodriguez]

On Fri, Nov 20, 2009 at 07:50:54AM -0800, Alan Jenkins wrote:
> This allows "make localmodconfig MODULES_LIST=lsmod.out",
> where "lsmod.out" may have been created on a different system.
> 
> You can also use this with a manually editted file which
> simply lists the desired modules.

The current localmodconfig is designed to be run on your own system,
not an external system. The changes outlined here give no guidelines
to users what they should do if they want to use this on another system
and a novice user may just assume an old distribution config file
can be used here along with an lsmod output. If this is the case
it should be documented as such but I don't think this is the case.
Consider the case of someone trying to build a kernel for a single
processor ARM or x86 netbook/nettop on a beefy quad core x86_64.

Think this is definitely better than my approach though, that's for sure :)

Thanks for working on it :)

> Signed-off-by: Alan Jenkins <alan-jenkins@tuffmail.co.uk>
> ---
>  README                               |    8 ++++++++
>  scripts/kconfig/Makefile             |    3 ++-
>  scripts/kconfig/streamline_config.pl |   17 ++++++++++++++---
>  3 files changed, 24 insertions(+), 4 deletions(-)
> 
> diff --git a/README b/README
> index 737838f..9481dd5 100644
> --- a/README
> +++ b/README
> @@ -175,6 +175,14 @@ CONFIGURING the kernel:
>                            Like above, but avoids cluttering the screen
>                            with questions already answered.
>                            Additionally updates the dependencies.
> +
> +       "make localmodconfig"
> +                          Minimize your existing ./.config file, based
> +                          on the kernel modules currently loaded on your
> +                          system.  You can also specify the list of
> +                          modules manually by appending an option
> +                          "MODULES_LIST=lsmod.out".

Best to add a plain entry for localmodconfig separately and then just
add the new stuff for MODULES_LIST on another patch.

> +
>         "make defconfig"   Create a ./.config file by using the default
>                            symbol values from either arch/$ARCH/defconfig
>                            or arch/$ARCH/configs/${PLATFORM}_defconfig,
> diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
> index 6d69c7c..6a1e41d 100644
> --- a/scripts/kconfig/Makefile
> +++ b/scripts/kconfig/Makefile
> @@ -30,7 +30,7 @@ silentoldconfig: $(obj)/conf
>         $< -s $(Kconfig)
> 
>  localmodconfig: $(obj)/streamline_config.pl $(obj)/conf
> -       $(Q)perl $< $(Kconfig) > .tmp.config
> +       $(Q)perl $< $(Kconfig) $(MODULES_LIST) > .tmp.config
>         $(Q)if [ -f .config ]; then                             \
>                         cmp -s .tmp.config .config ||           \
>                         (mv -f .config .config.old.1;           \
> @@ -114,6 +114,7 @@ help:
>         @echo  '  gconfig         - Update current config utilising a GTK based front-end'
>         @echo  '  oldconfig       - Update current config utilising a provided .config as base'
>         @echo  '  localmodconfig  - Update current config disabling modules not loaded'
> +       @echo  '                    (or modules not listed in the file MODULES_LIST)'
>         @echo  '  localyesconfig  - Update current config converting local mods to core'
>         @echo  '  silentoldconfig - Same as oldconfig, but quietly, additionally update deps'
>         @echo  '  randconfig      - New config with random answer to all options'
> diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
> index f5b44c1..9fde7fd 100644
> --- a/scripts/kconfig/streamline_config.pl
> +++ b/scripts/kconfig/streamline_config.pl
> @@ -40,7 +40,7 @@ use warnings;
>  #
>  #    cd /usr/src/linux-2.6.10
>  #    cp /boot/config-2.6.10-1-686-smp .config
> -#    ~/bin/streamline_config > config_strip
> +#    ~/bin/streamline_config arch/x86/Kconfig > config_strip

Why was x86 added here?

>  #    mv .config config_sav
>  #    mv config_strip .config
>  #    make oldconfig
> @@ -122,9 +122,15 @@ my %objects;
>  my $var;
>  my $cont = 0;
> 
> +if ($#ARGV < 0 || $#ARGV > 1) {
> +    die "usage: streamline_config.pl <Kconfig> [<modules.list>]"
> +}
> +
>  # Get the top level Kconfig file (passed in)
>  my $kconfig = $ARGV[0];
> 
> +my $modules_list = $ARGV[1];
> +
>  # prevent recursion
>  my %read_kconfigs;
> 
> @@ -251,8 +257,13 @@ foreach my $makefile (@makefiles) {
> 
>  my %modules;
> 
> -# see what modules are loaded on this system
> -open(LIN,"/sbin/lsmod|") || die "Cant lsmod";
> +# see what modules are loaded
> +if ($modules_list) {
> +    open(LIN,$modules_list) || die "Can't open $modules_list";
> +} else {
> +    open(LIN,"/sbin/lsmod|") || die "Can't lsmod";
> +}
> +
>  while (<LIN>) {
>         next if (/^Module/);  # Skip the first line.
>         if (/^(\S+)/) {
> --
> 1.6.3.3
>