* optimized script [Was: ARM defconfig files]
[not found] ` <20100713070741.GB26442@pengutronix.de>
@ 2010-07-13 8:07 ` Uwe Kleine-König
2010-07-13 18:04 ` Olof Johansson
0 siblings, 1 reply; 3+ messages in thread
From: Uwe Kleine-König @ 2010-07-13 8:07 UTC (permalink / raw)
To: Linus Torvalds
Cc: Stephen Rothwell, Daniel Walker, Russell King - ARM Linux,
Nicolas Pitre, Kevin Hilman, linux-arm-msm,
Linux Kernel Mailing List, Eric Miao, Olof Johansson, linux-omap,
linuxppc-dev, linux-arm-kernel
[-- Attachment #1: Type: text/plain, Size: 1548 bytes --]
Hello,
On Tue, Jul 13, 2010 at 09:07:41AM +0200, Uwe Kleine-König wrote:
> Hi
>
> On Mon, Jul 12, 2010 at 01:50:47PM -0600, Grant Likely wrote:
> > On Mon, Jul 12, 2010 at 1:34 PM, Linus Torvalds
> > <torvalds@linux-foundation.org> wrote:
> > > On Mon, Jul 12, 2010 at 12:17 PM, Nicolas Pitre <nico@fluxnic.net> wrote:
> > >> I think Uwe could provide his script and add it to the kernel tree.
> > >> Then all architectures could benefit from it. Having the defconfig
> > >> files contain only those options which are different from the defaults
> > >> is certainly more readable, even on x86.
> > >
> > > Quite possible. But maintainers would need to be on the lookout of
> > > people actually using the script, and refusing to apply patches that
> > > re-introduce the whole big thing.
> >
> > I can (partially) speak for powerpc. If ARM uses this approach, then
> > I think we can do the same. After the defconfigs are trimmed, I
> > certainly won't pick up any more full defconfigs.
> I just restarted my script on the powerpc defconfigs basing on rc5, I
> assume they complete in a few days time.
So Stephen was faster than me. I don't know yet how he optimised my
script, meanwhile I put some efforts into it, too by just checking lines
that match "^(# )?CONFIG_".
Find it attached.
I will start to reduce the remaining configs (i.e. all but arm and
powerpc).
Best regards
Uwe
--
Pengutronix e.K. | Uwe Kleine-König |
Industrial Linux Solutions | http://www.pengutronix.de/ |
[-- Attachment #2: reduce_defconfig --]
[-- Type: text/plain, Size: 1899 bytes --]
#! /usr/bin/env python
# vim: set fileencoding=utf-8 :
# Copyright (C) 2010 by Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
import getopt
import re
import os
import subprocess
import sys
# This prevents including a timestamp in the .config which makes comparing a
# bit easier.
os.environ['KCONFIG_NOTIMESTAMP'] = 'Yes, please'
re_interesting = re.compile(r'^(# )?CONFIG_')
opts, args = getopt.getopt(sys.argv[1:], '', ['arch=', 'src='])
src = ''
arch = 'arm'
for o, a in opts:
if o == '--arch':
arch = a
elif o == '--src':
src = a
configdir = os.path.join(src, 'arch', arch, 'configs')
def all_defconfigs():
lc = len(configdir)
for root, dirs, files in os.walk(configdir):
root = root[lc + 1:]
for f in filter(lambda s: s.endswith('_defconfig'), files):
yield os.path.join(root, f)
if not args:
args = all_defconfigs()
for target in args:
defconfig_src = os.path.join(configdir, target)
subprocess.check_call(['make', '-s', 'ARCH=%s' % arch, target])
origconfig = list(open('.config'))
config = list(origconfig)
config_size = os.stat('.config').st_size
i = 0
while i < len(config):
mo = re_interesting.match(config[i])
if mo:
defconfig = open(defconfig_src, 'w')
defconfig.writelines(config[:i])
defconfig.writelines(config[i + 1:])
defconfig.close()
subprocess.check_call(['make', '-s', 'ARCH=%s' % arch, target])
if os.stat('.config').st_size == config_size and list(open('.config')) == origconfig:
print '-%s' % config[i][:-1]
del config[i]
else:
print ' %s' % config[i][:-1]
i += 1
else:
del config[i]
defconfig = open(defconfig_src, 'w')
defconfig.writelines(config)
defconfig.close()
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: optimized script [Was: ARM defconfig files]
2010-07-13 8:07 ` optimized script [Was: ARM defconfig files] Uwe Kleine-König
@ 2010-07-13 18:04 ` Olof Johansson
2010-07-13 23:39 ` Nicolas Pitre
0 siblings, 1 reply; 3+ messages in thread
From: Olof Johansson @ 2010-07-13 18:04 UTC (permalink / raw)
To: Uwe Kleine-König
Cc: Stephen Rothwell, Daniel Walker, Russell King - ARM Linux,
Nicolas Pitre, Kevin Hilman, linux-arm-msm, linuxppc-dev,
Linux Kernel Mailing List, Eric Miao, linux-omap, Linus Torvalds,
linux-arm-kernel
On Tue, Jul 13, 2010 at 10:07:05AM +0200, Uwe Kleine-König wrote:
> Hello,
>
> On Tue, Jul 13, 2010 at 09:07:41AM +0200, Uwe Kleine-König wrote:
> > Hi
> >
> > On Mon, Jul 12, 2010 at 01:50:47PM -0600, Grant Likely wrote:
> > > On Mon, Jul 12, 2010 at 1:34 PM, Linus Torvalds
> > > <torvalds@linux-foundation.org> wrote:
> > > > On Mon, Jul 12, 2010 at 12:17 PM, Nicolas Pitre <nico@fluxnic.net> wrote:
> > > >> I think Uwe could provide his script and add it to the kernel tree.
> > > >> Then all architectures could benefit from it. Having the defconfig
> > > >> files contain only those options which are different from the defaults
> > > >> is certainly more readable, even on x86.
> > > >
> > > > Quite possible. But maintainers would need to be on the lookout of
> > > > people actually using the script, and refusing to apply patches that
> > > > re-introduce the whole big thing.
> > >
> > > I can (partially) speak for powerpc. If ARM uses this approach, then
> > > I think we can do the same. After the defconfigs are trimmed, I
> > > certainly won't pick up any more full defconfigs.
> > I just restarted my script on the powerpc defconfigs basing on rc5, I
> > assume they complete in a few days time.
> So Stephen was faster than me. I don't know yet how he optimised my
> script, meanwhile I put some efforts into it, too by just checking lines
> that match "^(# )?CONFIG_".
>
> Find it attached.
>
> I will start to reduce the remaining configs (i.e. all but arm and
> powerpc).
I added just a simple heuristic: If I could remove a line, I attempted
to remove twice the amount next time around (and fall back to 1 if it failed).
I.e. main loop:
i = 0
lines = 1
while i < len(config):
print 'test for %r + %d' % (config[i], lines)
defconfig = open(defconfig_src, 'w')
defconfig.writelines(config[:i])
defconfig.writelines(config[i + lines:])
defconfig.close()
subprocess.check_call(['make', '-s', 'ARCH=%s' % arch, target])
if os.stat('.config').st_size == config_size and list(open('.config')) == origconfig:
del config[i:i+lines]
lines *= 2
else:
if lines > 1:
lines = 1
else:
i += 1
I didn't measure what the actual improvement was, but I saw a fair amount
of 2/4/8-attempts passing, so I let it run. Stephen beat me to posting
the resulting patch though. :P
While this script is great, it is somewhat painful to run given that it
attempts one config per line. Even on a fast machine that tends to take
a while.
-Olof
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: optimized script [Was: ARM defconfig files]
2010-07-13 18:04 ` Olof Johansson
@ 2010-07-13 23:39 ` Nicolas Pitre
0 siblings, 0 replies; 3+ messages in thread
From: Nicolas Pitre @ 2010-07-13 23:39 UTC (permalink / raw)
To: Olof Johansson
Cc: Stephen Rothwell, Daniel Walker, Russell King - ARM Linux,
Kevin Hilman, linux-arm-msm, linuxppc-dev,
Linux Kernel Mailing List, Eric Miao, Uwe Kleine-König,
linux-omap, Linus Torvalds, linux-arm-kernel
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2203 bytes --]
On Tue, 13 Jul 2010, Olof Johansson wrote:
> On Tue, Jul 13, 2010 at 10:07:05AM +0200, Uwe Kleine-König wrote:
> > Hello,
> >
> > On Tue, Jul 13, 2010 at 09:07:41AM +0200, Uwe Kleine-König wrote:
> > > Hi
> > >
> > > On Mon, Jul 12, 2010 at 01:50:47PM -0600, Grant Likely wrote:
> > > > On Mon, Jul 12, 2010 at 1:34 PM, Linus Torvalds
> > > > <torvalds@linux-foundation.org> wrote:
> > > > > On Mon, Jul 12, 2010 at 12:17 PM, Nicolas Pitre <nico@fluxnic.net> wrote:
> > > > >> I think Uwe could provide his script and add it to the kernel tree.
> > > > >> Then all architectures could benefit from it. Having the defconfig
> > > > >> files contain only those options which are different from the defaults
> > > > >> is certainly more readable, even on x86.
> > > > >
> > > > > Quite possible. But maintainers would need to be on the lookout of
> > > > > people actually using the script, and refusing to apply patches that
> > > > > re-introduce the whole big thing.
> > > >
> > > > I can (partially) speak for powerpc. If ARM uses this approach, then
> > > > I think we can do the same. After the defconfigs are trimmed, I
> > > > certainly won't pick up any more full defconfigs.
> > > I just restarted my script on the powerpc defconfigs basing on rc5, I
> > > assume they complete in a few days time.
> > So Stephen was faster than me. I don't know yet how he optimised my
> > script, meanwhile I put some efforts into it, too by just checking lines
> > that match "^(# )?CONFIG_".
> >
> > Find it attached.
> >
> > I will start to reduce the remaining configs (i.e. all but arm and
> > powerpc).
>
> I added just a simple heuristic: If I could remove a line, I attempted
> to remove twice the amount next time around (and fall back to 1 if it failed).
>
[...]
>
> While this script is great, it is somewhat painful to run given that it
> attempts one config per line. Even on a fast machine that tends to take
> a while.
The optimal solution is to add that .config reduction ability straight
into the Kconfig parser (scripts/kconfig/*). There you can find out
right away what are the non default state for each config option without
actually trying them out one by one.
Nicolas
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2010-07-14 0:39 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20100712155518.GA24144@pengutronix.de>
[not found] ` <AANLkTilhI9O6YwnuAylo_jNgwR8VXmpSVJ1kylIYPCGT@mail.gmail.com>
[not found] ` <20100712173228.GC9897@n2100.arm.linux.org.uk>
[not found] ` <AANLkTinxUqzxClVDTyruJ6kzFTganXUGJJHJ5wRi9Iak@mail.gmail.com>
[not found] ` <20100712185029.GB14425@pengutronix.de>
[not found] ` <AANLkTinOuU-VVXaMwMp69CBp2Y4wxpoWybEQ34H5j0lJ@mail.gmail.com>
[not found] ` <alpine.LFD.2.00.1007121510260.10598@xanadu.home>
[not found] ` <AANLkTim-O00n9YPv7w4VCKjN-ETLijyTHomWUn4mZtTV@mail.gmail.com>
[not found] ` <AANLkTilDoGbCx_0oKKPOeGG7TKL2VSNVII5ydmnQ77B7@mail.gmail.com>
[not found] ` <20100713070741.GB26442@pengutronix.de>
2010-07-13 8:07 ` optimized script [Was: ARM defconfig files] Uwe Kleine-König
2010-07-13 18:04 ` Olof Johansson
2010-07-13 23:39 ` Nicolas Pitre
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).