* [U-Boot] [PATCH v3 02/10] moveconfig: Continue moving even if one board fails
2015-05-13 22:28 ` [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk Joe Hershberger
@ 2015-05-13 22:28 ` Joe Hershberger
2015-05-14 13:09 ` Masahiro Yamada
2015-05-13 22:28 ` [U-Boot] [PATCH v3 03/10] moveconfig: Error if missing the include/autoconf.mk Joe Hershberger
` (9 subsequent siblings)
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-13 22:28 UTC (permalink / raw)
To: u-boot
Some compilers are hard to come by or have so few boards they are not
worth messing with for this tool. Provide a list that need manual
intervention and continue moving the bulk of boards.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v3: None
Changes in v2:
-Print which compiler is missing
tools/moveconfig.py | 35 ++++++++++++++++++++++++++++++-----
1 file changed, 30 insertions(+), 5 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 30dc4f6..d9ae859 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -17,6 +17,7 @@ import os
import re
import shutil
import subprocess
+from subprocess import PIPE
import sys
import tempfile
import time
@@ -276,6 +277,13 @@ class Slot:
self.state = STATE_DEFCONFIG
return True
+ def defconfig_error(self, errmsg):
+ output = self.defconfig[:-len('_defconfig')].ljust(37) + ': '
+ print output + errmsg
+
+ """Save a list of targets that have to be checked by hand"""
+ open('moveconfig.failed', 'a+').write("%s\n" % self.defconfig)
+
def poll(self):
"""Check if the subprocess is running and invoke the .config
parser if the subprocess is terminated.
@@ -290,19 +298,25 @@ class Slot:
return False
if self.ps.poll() != 0:
- sys.exit("failed to process '%s'" % self.defconfig)
+ errmsg = 'ERROR - build error'
+ errout = self.ps.stderr.read()
+ if errout.find('gcc: command not found') != -1:
+ errmsg = 'ERROR - compiler not found (%s)' % self.cross_compile
+ self.defconfig_error(errmsg)
+ self.state = STATE_IDLE
+ return True
if self.state == STATE_SILENTOLDCONFIG:
self.parser.update_defconfig(self.defconfig)
self.state = STATE_IDLE
return True
- cross_compile = self.parser.get_cross_compile()
+ self.cross_compile = self.parser.get_cross_compile()
cmd = list(self.make_cmd)
- if cross_compile:
- cmd.append('CROSS_COMPILE=%s' % cross_compile)
+ if self.cross_compile:
+ cmd.append('CROSS_COMPILE=%s' % self.cross_compile)
cmd.append('include/autoconf.mk')
- self.ps = subprocess.Popen(cmd, stdout=self.devnull)
+ self.ps = subprocess.Popen(cmd, stdout=self.devnull, stderr=PIPE)
self.state = STATE_SILENTOLDCONFIG
return False
@@ -362,6 +376,7 @@ class Slots:
def move_config(config_attr, jobs=1):
check_top_directory()
+
print 'Moving %s (type: %s, default: %s, no_spl: %s) ... (jobs: %d)' % (
config_attr['config'],
config_attr['type'],
@@ -378,6 +393,10 @@ def move_config(config_attr, jobs=1):
continue
defconfigs.append(os.path.join(dirpath, filename))
+ """Clean up any previous log of failed moves"""
+ if os.path.exists('moveconfig.failed'):
+ os.remove('moveconfig.failed')
+
slots = Slots(config_attr, jobs)
# Main loop to process defconfig files:
@@ -395,6 +414,12 @@ def move_config(config_attr, jobs=1):
cleanup_headers(config_attr['config'])
+ if os.path.exists('moveconfig.failed'):
+ print '!!! Some boards were not processed; move the config manually.'
+ print '!!! The list of failed boards are saved in moveconfig.failed'
+ print
+ print open('moveconfig.failed', 'r').read()
+
def main():
try:
cpu_count = multiprocessing.cpu_count()
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 02/10] moveconfig: Continue moving even if one board fails
2015-05-13 22:28 ` [U-Boot] [PATCH v3 02/10] moveconfig: Continue moving even if one board fails Joe Hershberger
@ 2015-05-14 13:09 ` Masahiro Yamada
0 siblings, 0 replies; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-14 13:09 UTC (permalink / raw)
To: u-boot
2015-05-14 7:28 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> Some compilers are hard to come by or have so few boards they are not
> worth messing with for this tool. Provide a list that need manual
> intervention and continue moving the bulk of boards.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>
> ---
>
> Changes in v3: None
> Changes in v2:
> -Print which compiler is missing
>
> tools/moveconfig.py | 35 ++++++++++++++++++++++++++++++-----
> 1 file changed, 30 insertions(+), 5 deletions(-)
>
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index 30dc4f6..d9ae859 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -17,6 +17,7 @@ import os
> import re
> import shutil
> import subprocess
> +from subprocess import PIPE
> import sys
> import tempfile
> import time
> @@ -276,6 +277,13 @@ class Slot:
> self.state = STATE_DEFCONFIG
> return True
>
> + def defconfig_error(self, errmsg):
> + output = self.defconfig[:-len('_defconfig')].ljust(37) + ': '
> + print output + errmsg
> +
> + """Save a list of targets that have to be checked by hand"""
> + open('moveconfig.failed', 'a+').write("%s\n" % self.defconfig)
> +
> def poll(self):
> """Check if the subprocess is running and invoke the .config
> parser if the subprocess is terminated.
> @@ -290,19 +298,25 @@ class Slot:
> return False
>
> if self.ps.poll() != 0:
> - sys.exit("failed to process '%s'" % self.defconfig)
> + errmsg = 'ERROR - build error'
> + errout = self.ps.stderr.read()
> + if errout.find('gcc: command not found') != -1:
> + errmsg = 'ERROR - compiler not found (%s)' % self.cross_compile
> + self.defconfig_error(errmsg)
> + self.state = STATE_IDLE
> + return True
>
> if self.state == STATE_SILENTOLDCONFIG:
> self.parser.update_defconfig(self.defconfig)
> self.state = STATE_IDLE
> return True
>
> - cross_compile = self.parser.get_cross_compile()
> + self.cross_compile = self.parser.get_cross_compile()
> cmd = list(self.make_cmd)
> - if cross_compile:
> - cmd.append('CROSS_COMPILE=%s' % cross_compile)
> + if self.cross_compile:
> + cmd.append('CROSS_COMPILE=%s' % self.cross_compile)
> cmd.append('include/autoconf.mk')
> - self.ps = subprocess.Popen(cmd, stdout=self.devnull)
> + self.ps = subprocess.Popen(cmd, stdout=self.devnull, stderr=PIPE)
> self.state = STATE_SILENTOLDCONFIG
> return False
>
> @@ -362,6 +376,7 @@ class Slots:
>
> def move_config(config_attr, jobs=1):
> check_top_directory()
> +
> print 'Moving %s (type: %s, default: %s, no_spl: %s) ... (jobs: %d)' % (
> config_attr['config'],
> config_attr['type'],
> @@ -378,6 +393,10 @@ def move_config(config_attr, jobs=1):
> continue
> defconfigs.append(os.path.join(dirpath, filename))
>
> + """Clean up any previous log of failed moves"""
> + if os.path.exists('moveconfig.failed'):
> + os.remove('moveconfig.failed')
> +
> slots = Slots(config_attr, jobs)
>
> # Main loop to process defconfig files:
> @@ -395,6 +414,12 @@ def move_config(config_attr, jobs=1):
>
> cleanup_headers(config_attr['config'])
>
> + if os.path.exists('moveconfig.failed'):
> + print '!!! Some boards were not processed; move the config manually.'
> + print '!!! The list of failed boards are saved in moveconfig.failed'
> + print
> + print open('moveconfig.failed', 'r').read()
> +
> def main():
> try:
> cpu_count = multiprocessing.cpu_count()
I could successfully convert all the defconfigs with my toolchains,
but I thought this feature would be helpful.
I merged this feature in my new version with your signed-off-by
although the implementation is a bit different.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 03/10] moveconfig: Error if missing the include/autoconf.mk
2015-05-13 22:28 ` [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk Joe Hershberger
2015-05-13 22:28 ` [U-Boot] [PATCH v3 02/10] moveconfig: Continue moving even if one board fails Joe Hershberger
@ 2015-05-13 22:28 ` Joe Hershberger
2015-05-13 22:28 ` [U-Boot] [PATCH v3 04/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
` (8 subsequent siblings)
10 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-13 22:28 UTC (permalink / raw)
To: u-boot
The main image autoconf.mk certainly had better exist or this tool won't
function at all. Check for this and error if missing.
The SPL and TPL don't even exist as separate .config files any more.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v3: None
Changes in v2: None
tools/moveconfig.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index d9ae859..b4ee0e3 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -183,6 +183,8 @@ class KconfigParser:
'autoconf.mk')
if not os.path.exists(autoconf):
+ if img == '.':
+ return True
values.append('')
continue
@@ -227,6 +229,7 @@ class KconfigParser:
if prefixes[line] != '+':
line = prefixes[line] + ':' + line
f.write(line + '\n')
+ return False
class Slot:
@@ -307,7 +310,8 @@ class Slot:
return True
if self.state == STATE_SILENTOLDCONFIG:
- self.parser.update_defconfig(self.defconfig)
+ if self.parser.update_defconfig(self.defconfig):
+ self.defconfig_error('ERROR - autoconf.mk not found')
self.state = STATE_IDLE
return True
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 04/10] moveconfig: Always run savedefconfig on the moved config
2015-05-13 22:28 ` [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk Joe Hershberger
2015-05-13 22:28 ` [U-Boot] [PATCH v3 02/10] moveconfig: Continue moving even if one board fails Joe Hershberger
2015-05-13 22:28 ` [U-Boot] [PATCH v3 03/10] moveconfig: Error if missing the include/autoconf.mk Joe Hershberger
@ 2015-05-13 22:28 ` Joe Hershberger
2015-05-14 15:15 ` Masahiro Yamada
2015-05-13 22:28 ` [U-Boot] [PATCH v3 05/10] moveconfig: Add a parameter to accept a list to build Joe Hershberger
` (7 subsequent siblings)
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-13 22:28 UTC (permalink / raw)
To: u-boot
This will ensure that the order of the defconfig entries will always
match that of the Kconfig files. After one slightly painful (but
still early in the process) pass over all boards, this should keep
the defconfigs clean from here on.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v3: None
Changes in v2: None
tools/moveconfig.py | 24 +++++++++++++++++++++---
1 file changed, 21 insertions(+), 3 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index b4ee0e3..f2651a9 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -45,6 +45,7 @@ CROSS_COMPILE = {
STATE_IDLE = 0
STATE_DEFCONFIG = 1
STATE_SILENTOLDCONFIG = 2
+STATE_SAVEDEFCONFIG = 3
### helper functions ###
def get_devnull():
@@ -149,8 +150,8 @@ class KconfigParser:
"""
arch = ''
cpu = ''
- dotconfig = os.path.join(self.build_dir, '.config')
- for line in open(dotconfig):
+ self.dotconfig = os.path.join(self.build_dir, '.config')
+ for line in open(self.dotconfig):
m = self.re_arch.match(line)
if m:
arch = m.group(1)
@@ -224,7 +225,7 @@ class KconfigParser:
print output.strip()
- with open(os.path.join('configs', defconfig), 'a') as f:
+ with open(os.path.join(self.dotconfig), 'a') as f:
for line in output_lines:
if prefixes[line] != '+':
line = prefixes[line] + ':' + line
@@ -312,6 +313,23 @@ class Slot:
if self.state == STATE_SILENTOLDCONFIG:
if self.parser.update_defconfig(self.defconfig):
self.defconfig_error('ERROR - autoconf.mk not found')
+ self.state = STATE_IDLE
+ return True
+
+ """Save off the defconfig in a consistent way"""
+ cmd = list(self.make_cmd)
+ cmd.append('savedefconfig')
+ self.ps = subprocess.Popen(cmd, stdout=self.devnull,
+ stderr=self.devnull)
+ self.state = STATE_SAVEDEFCONFIG
+ return False
+
+ if self.state == STATE_SAVEDEFCONFIG:
+ defconfig_path = os.path.join(self.build_dir, 'defconfig')
+ if not os.path.exists(defconfig_path):
+ self.defconfig_error('ERROR - defconfig not updated')
+ shutil.move(defconfig_path,
+ os.path.join('configs', self.defconfig))
self.state = STATE_IDLE
return True
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 04/10] moveconfig: Always run savedefconfig on the moved config
2015-05-13 22:28 ` [U-Boot] [PATCH v3 04/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
@ 2015-05-14 15:15 ` Masahiro Yamada
2015-05-14 17:57 ` Joe Hershberger
0 siblings, 1 reply; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-14 15:15 UTC (permalink / raw)
To: u-boot
2015-05-14 7:28 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> This will ensure that the order of the defconfig entries will always
> match that of the Kconfig files. After one slightly painful (but
> still early in the process) pass over all boards, this should keep
> the defconfigs clean from here on.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
I a bit hesitate to always enable this feature
because savedefconfig might make git-diff noisier.
Is it possible to make it optional?
-s, --savedefconfig or --no-savedefconfig?
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 04/10] moveconfig: Always run savedefconfig on the moved config
2015-05-14 15:15 ` Masahiro Yamada
@ 2015-05-14 17:57 ` Joe Hershberger
0 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-14 17:57 UTC (permalink / raw)
To: u-boot
Hi Masahiro-san.
On Thu, May 14, 2015 at 10:15 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2015-05-14 7:28 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>> This will ensure that the order of the defconfig entries will always
>> match that of the Kconfig files. After one slightly painful (but
>> still early in the process) pass over all boards, this should keep
>> the defconfigs clean from here on.
>>
>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>
>
> I a bit hesitate to always enable this feature
> because savedefconfig might make git-diff noisier.
That should never be the case. Its consistent use should always lead
to the least noisy diff.
> Is it possible to make it optional?
> -s, --savedefconfig or --no-savedefconfig?
It is possible, but I recommend against it. Its inconsistent use would
lead to noisy diffs.
Cheers,
-Joe
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 05/10] moveconfig: Add a parameter to accept a list to build
2015-05-13 22:28 ` [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk Joe Hershberger
` (2 preceding siblings ...)
2015-05-13 22:28 ` [U-Boot] [PATCH v3 04/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
@ 2015-05-13 22:28 ` Joe Hershberger
2015-05-14 13:06 ` Masahiro Yamada
2015-05-13 22:28 ` [U-Boot] [PATCH v3 06/10] moveconfig: Add a switch to enable printing errors Joe Hershberger
` (6 subsequent siblings)
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-13 22:28 UTC (permalink / raw)
To: u-boot
This is helpful to re-attempt to move failed boards from a previous run
without starting over.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v3:
-Fixed command line options order (alphabetize)
Changes in v2:
-New for version 2
tools/moveconfig.py | 31 ++++++++++++++++++-------------
1 file changed, 18 insertions(+), 13 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index f2651a9..2688ad1 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -396,7 +396,7 @@ class Slots:
ret = False
return ret
-def move_config(config_attr, jobs=1):
+def move_config(config_attr, options):
check_top_directory()
print 'Moving %s (type: %s, default: %s, no_spl: %s) ... (jobs: %d)' % (
@@ -404,22 +404,25 @@ def move_config(config_attr, jobs=1):
config_attr['type'],
config_attr['default'],
config_attr['no_spl_support'],
- jobs)
-
- # All the defconfig files to be processed
- defconfigs = []
- for (dirpath, dirnames, filenames) in os.walk('configs'):
- dirpath = dirpath[len('configs') + 1:]
- for filename in fnmatch.filter(filenames, '*_defconfig'):
- if fnmatch.fnmatch(filename, '.*'):
- continue
- defconfigs.append(os.path.join(dirpath, filename))
+ options.jobs)
+
+ if options.defconfigs:
+ defconfigs = [line.strip() for line in open(options.defconfigs, 'r')]
+ else:
+ # All the defconfig files to be processed
+ defconfigs = []
+ for (dirpath, dirnames, filenames) in os.walk('configs'):
+ dirpath = dirpath[len('configs') + 1:]
+ for filename in fnmatch.filter(filenames, '*_defconfig'):
+ if fnmatch.fnmatch(filename, '.*'):
+ continue
+ defconfigs.append(os.path.join(dirpath, filename))
"""Clean up any previous log of failed moves"""
if os.path.exists('moveconfig.failed'):
os.remove('moveconfig.failed')
- slots = Slots(config_attr, jobs)
+ slots = Slots(config_attr, options.jobs)
# Main loop to process defconfig files:
# Add a new subprocess into a vacant slot.
@@ -450,6 +453,8 @@ def main():
parser = optparse.OptionParser()
# Add options here
+ parser.add_option('-d', '--defconfigs', type='string',
+ help='a file containing a list of defconfigs to move')
parser.add_option('-j', '--jobs', type='int', default=cpu_count,
help='the number of jobs to run simultaneously')
parser.usage += ' config type default no_spl_support'
@@ -480,7 +485,7 @@ def main():
if not config_attr['config'].startswith('CONFIG_'):
config_attr['config'] = 'CONFIG_' + config_attr['config']
- move_config(config_attr, options.jobs)
+ move_config(config_attr, options)
if __name__ == '__main__':
main()
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 06/10] moveconfig: Add a switch to enable printing errors
2015-05-13 22:28 ` [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk Joe Hershberger
` (3 preceding siblings ...)
2015-05-13 22:28 ` [U-Boot] [PATCH v3 05/10] moveconfig: Add a parameter to accept a list to build Joe Hershberger
@ 2015-05-13 22:28 ` Joe Hershberger
2015-05-13 22:28 ` [U-Boot] [PATCH v3 07/10] moveconfig: Ignore duplicate configs when moving Joe Hershberger
` (5 subsequent siblings)
10 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-13 22:28 UTC (permalink / raw)
To: u-boot
In some cases the build for the autoconf breaks. This outputs the errors
following the status so that action can be taken without building again
manually.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v3: None
Changes in v2:
-New for version 2
tools/moveconfig.py | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 2688ad1..9135e1d 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -241,7 +241,7 @@ class Slot:
for faster processing.
"""
- def __init__(self, config_attr, devnull, make_cmd):
+ def __init__(self, config_attr, devnull, make_cmd, options):
"""Create a new slot.
Arguments:
@@ -250,6 +250,7 @@ class Slot:
self.build_dir = tempfile.mkdtemp()
self.devnull = devnull
self.make_cmd = (make_cmd, 'O=' + self.build_dir)
+ self.options = options
self.parser = KconfigParser(self.build_dir, config_attr)
self.state = STATE_IDLE
@@ -307,6 +308,8 @@ class Slot:
if errout.find('gcc: command not found') != -1:
errmsg = 'ERROR - compiler not found (%s)' % self.cross_compile
self.defconfig_error(errmsg)
+ if self.options.verbose:
+ print errout
self.state = STATE_IDLE
return True
@@ -346,7 +349,7 @@ class Slots:
"""Controller of the array of subprocess slots."""
- def __init__(self, config_attr, jobs):
+ def __init__(self, config_attr, options):
"""Create a new slots controller.
Arguments:
@@ -355,8 +358,8 @@ class Slots:
self.slots = []
devnull = get_devnull()
make_cmd = get_make_cmd()
- for i in range(jobs):
- self.slots.append(Slot(config_attr, devnull, make_cmd))
+ for i in range(options.jobs):
+ self.slots.append(Slot(config_attr, devnull, make_cmd, options))
def add(self, defconfig):
"""Add a new subprocess if a vacant slot is available.
@@ -422,7 +425,7 @@ def move_config(config_attr, options):
if os.path.exists('moveconfig.failed'):
os.remove('moveconfig.failed')
- slots = Slots(config_attr, options.jobs)
+ slots = Slots(config_attr, options)
# Main loop to process defconfig files:
# Add a new subprocess into a vacant slot.
@@ -457,6 +460,9 @@ def main():
help='a file containing a list of defconfigs to move')
parser.add_option('-j', '--jobs', type='int', default=cpu_count,
help='the number of jobs to run simultaneously')
+ parser.add_option('-v', '--verbose', dest='verbose',
+ action='store_true', default=False,
+ help='show any build errors as boards are build')
parser.usage += ' config type default no_spl_support'
(options, args) = parser.parse_args()
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 07/10] moveconfig: Ignore duplicate configs when moving
2015-05-13 22:28 ` [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk Joe Hershberger
` (4 preceding siblings ...)
2015-05-13 22:28 ` [U-Boot] [PATCH v3 06/10] moveconfig: Add a switch to enable printing errors Joe Hershberger
@ 2015-05-13 22:28 ` Joe Hershberger
2015-05-14 15:36 ` Masahiro Yamada
2015-05-13 22:28 ` [U-Boot] [PATCH v3 08/10] moveconfig: Handle moving multiple configs at once Joe Hershberger
` (4 subsequent siblings)
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-13 22:28 UTC (permalink / raw)
To: u-boot
When moving configs, it is important to know what was defined in the
config header even if it duplicates the configs coming from Kconfig.
tools/moveconfig.py now defines KCONFIG_IGNORE_DUPLICATES to prevent the
filtering from happening and selecting wrong values for the defconfig.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v3:
-New for version 3
Changes in v2: None
scripts/Makefile.autoconf | 3 ++-
tools/moveconfig.py | 1 +
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/scripts/Makefile.autoconf b/scripts/Makefile.autoconf
index f054081..36bfa17 100644
--- a/scripts/Makefile.autoconf
+++ b/scripts/Makefile.autoconf
@@ -58,7 +58,8 @@ quiet_cmd_autoconf = GEN $@
$(CPP) $(c_flags) $2 -DDO_DEPS_ONLY -dM $(srctree)/include/common.h > $@.tmp && { \
sed -n -f $(srctree)/tools/scripts/define2mk.sed $@.tmp | \
while read line; do \
- if ! grep -q "$${line%=*}=" include/config/auto.conf; then \
+ if [ -n "${KCONFIG_IGNORE_DUPLICATES}" ] || \
+ ! grep -q "$${line%=*}=" include/config/auto.conf; then \
echo "$$line"; \
fi \
done > $@; \
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 9135e1d..97ff597 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -340,6 +340,7 @@ class Slot:
cmd = list(self.make_cmd)
if self.cross_compile:
cmd.append('CROSS_COMPILE=%s' % self.cross_compile)
+ cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
cmd.append('include/autoconf.mk')
self.ps = subprocess.Popen(cmd, stdout=self.devnull, stderr=PIPE)
self.state = STATE_SILENTOLDCONFIG
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 07/10] moveconfig: Ignore duplicate configs when moving
2015-05-13 22:28 ` [U-Boot] [PATCH v3 07/10] moveconfig: Ignore duplicate configs when moving Joe Hershberger
@ 2015-05-14 15:36 ` Masahiro Yamada
2015-05-14 18:02 ` Joe Hershberger
0 siblings, 1 reply; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-14 15:36 UTC (permalink / raw)
To: u-boot
2015-05-14 7:28 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> When moving configs, it is important to know what was defined in the
> config header even if it duplicates the configs coming from Kconfig.
>
> tools/moveconfig.py now defines KCONFIG_IGNORE_DUPLICATES to prevent the
> filtering from happening and selecting wrong values for the defconfig.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>
> ---
>
> Changes in v3:
> -New for version 3
>
> Changes in v2: None
>
> scripts/Makefile.autoconf | 3 ++-
> tools/moveconfig.py | 1 +
> 2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/Makefile.autoconf b/scripts/Makefile.autoconf
> index f054081..36bfa17 100644
> --- a/scripts/Makefile.autoconf
> +++ b/scripts/Makefile.autoconf
> @@ -58,7 +58,8 @@ quiet_cmd_autoconf = GEN $@
> $(CPP) $(c_flags) $2 -DDO_DEPS_ONLY -dM $(srctree)/include/common.h > $@.tmp && { \
> sed -n -f $(srctree)/tools/scripts/define2mk.sed $@.tmp | \
> while read line; do \
> - if ! grep -q "$${line%=*}=" include/config/auto.conf; then \
> + if [ -n "${KCONFIG_IGNORE_DUPLICATES}" ] || \
> + ! grep -q "$${line%=*}=" include/config/auto.conf; then \
> echo "$$line"; \
> fi \
> done > $@; \
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index 9135e1d..97ff597 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -340,6 +340,7 @@ class Slot:
> cmd = list(self.make_cmd)
> if self.cross_compile:
> cmd.append('CROSS_COMPILE=%s' % self.cross_compile)
> + cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
> cmd.append('include/autoconf.mk')
> self.ps = subprocess.Popen(cmd, stdout=self.devnull, stderr=PIPE)
> self.state = STATE_SILENTOLDCONFIG
This patch is not clear to me.
Isn't it the same concept as u-boot.cfg
instroduced by commit 741e58e0fc8.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 07/10] moveconfig: Ignore duplicate configs when moving
2015-05-14 15:36 ` Masahiro Yamada
@ 2015-05-14 18:02 ` Joe Hershberger
2015-05-15 5:10 ` Masahiro Yamada
0 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-14 18:02 UTC (permalink / raw)
To: u-boot
Hi Masahiro-san,
On Thu, May 14, 2015 at 10:36 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2015-05-14 7:28 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>> When moving configs, it is important to know what was defined in the
>> config header even if it duplicates the configs coming from Kconfig.
>>
>> tools/moveconfig.py now defines KCONFIG_IGNORE_DUPLICATES to prevent the
>> filtering from happening and selecting wrong values for the defconfig.
>>
>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>>
>> ---
>>
>> Changes in v3:
>> -New for version 3
>>
>> Changes in v2: None
>>
>> scripts/Makefile.autoconf | 3 ++-
>> tools/moveconfig.py | 1 +
>> 2 files changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/scripts/Makefile.autoconf b/scripts/Makefile.autoconf
>> index f054081..36bfa17 100644
>> --- a/scripts/Makefile.autoconf
>> +++ b/scripts/Makefile.autoconf
>> @@ -58,7 +58,8 @@ quiet_cmd_autoconf = GEN $@
>> $(CPP) $(c_flags) $2 -DDO_DEPS_ONLY -dM $(srctree)/include/common.h > $@.tmp && { \
>> sed -n -f $(srctree)/tools/scripts/define2mk.sed $@.tmp | \
>> while read line; do \
>> - if ! grep -q "$${line%=*}=" include/config/auto.conf; then \
>> + if [ -n "${KCONFIG_IGNORE_DUPLICATES}" ] || \
>> + ! grep -q "$${line%=*}=" include/config/auto.conf; then \
>> echo "$$line"; \
>> fi \
>> done > $@; \
>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>> index 9135e1d..97ff597 100755
>> --- a/tools/moveconfig.py
>> +++ b/tools/moveconfig.py
>> @@ -340,6 +340,7 @@ class Slot:
>> cmd = list(self.make_cmd)
>> if self.cross_compile:
>> cmd.append('CROSS_COMPILE=%s' % self.cross_compile)
>> + cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
>> cmd.append('include/autoconf.mk')
>> self.ps = subprocess.Popen(cmd, stdout=self.devnull, stderr=PIPE)
>> self.state = STATE_SILENTOLDCONFIG
>
>
> This patch is not clear to me.
This was specifically a problem when moving a config that defaulted to
true in the Kconfig. That meant that it would be "duplicate removed"
from the include since it was already true (by default) in the
Kconfig. So it had the effect of disabling that config in every board.
This stops the filtering in the case where we are trying to move
configs instead of trying to build the board in this mixed config
system environment.
> Isn't it the same concept as u-boot.cfg
> instroduced by commit 741e58e0fc8.
I don't believe it is the same thing because this will still include
the configs enabled through the config header, but (I think) not those
from Kconfig. And anyway, we are evaluating autoconf.mk, not
u-boot.cfg.
Cheers,
-Joe
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 07/10] moveconfig: Ignore duplicate configs when moving
2015-05-14 18:02 ` Joe Hershberger
@ 2015-05-15 5:10 ` Masahiro Yamada
2015-05-16 2:57 ` Joe Hershberger
0 siblings, 1 reply; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-15 5:10 UTC (permalink / raw)
To: u-boot
Hi Joe,
2015-05-15 3:02 GMT+09:00 Joe Hershberger <joe.hershberger@gmail.com>:
> Hi Masahiro-san,
>
> On Thu, May 14, 2015 at 10:36 AM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> 2015-05-14 7:28 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>>> When moving configs, it is important to know what was defined in the
>>> config header even if it duplicates the configs coming from Kconfig.
>>>
>>> tools/moveconfig.py now defines KCONFIG_IGNORE_DUPLICATES to prevent the
>>> filtering from happening and selecting wrong values for the defconfig.
>>>
>>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>>>
>>> ---
>>>
>>> Changes in v3:
>>> -New for version 3
>>>
>>> Changes in v2: None
>>>
>>> scripts/Makefile.autoconf | 3 ++-
>>> tools/moveconfig.py | 1 +
>>> 2 files changed, 3 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/scripts/Makefile.autoconf b/scripts/Makefile.autoconf
>>> index f054081..36bfa17 100644
>>> --- a/scripts/Makefile.autoconf
>>> +++ b/scripts/Makefile.autoconf
>>> @@ -58,7 +58,8 @@ quiet_cmd_autoconf = GEN $@
>>> $(CPP) $(c_flags) $2 -DDO_DEPS_ONLY -dM $(srctree)/include/common.h > $@.tmp && { \
>>> sed -n -f $(srctree)/tools/scripts/define2mk.sed $@.tmp | \
>>> while read line; do \
>>> - if ! grep -q "$${line%=*}=" include/config/auto.conf; then \
>>> + if [ -n "${KCONFIG_IGNORE_DUPLICATES}" ] || \
>>> + ! grep -q "$${line%=*}=" include/config/auto.conf; then \
>>> echo "$$line"; \
>>> fi \
>>> done > $@; \
>>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>>> index 9135e1d..97ff597 100755
>>> --- a/tools/moveconfig.py
>>> +++ b/tools/moveconfig.py
>>> @@ -340,6 +340,7 @@ class Slot:
>>> cmd = list(self.make_cmd)
>>> if self.cross_compile:
>>> cmd.append('CROSS_COMPILE=%s' % self.cross_compile)
>>> + cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
>>> cmd.append('include/autoconf.mk')
>>> self.ps = subprocess.Popen(cmd, stdout=self.devnull, stderr=PIPE)
>>> self.state = STATE_SILENTOLDCONFIG
>>
>>
>> This patch is not clear to me.
>
> This was specifically a problem when moving a config that defaulted to
> true in the Kconfig. That meant that it would be "duplicate removed"
> from the include since it was already true (by default) in the
> Kconfig. So it had the effect of disabling that config in every board.
> This stops the filtering in the case where we are trying to move
> configs instead of trying to build the board in this mixed config
> system environment.
Uh, make sense.
I was almost forgetting how the "don't apply" version was implemented.
My new one parses .config, defconfig as well as autoconf.mk,
so please check if we still need to this hack.
>> Isn't it the same concept as u-boot.cfg
>> instroduced by commit 741e58e0fc8.
>
> I don't believe it is the same thing because this will still include
> the configs enabled through the config header, but (I think) not those
> from Kconfig. And anyway, we are evaluating autoconf.mk, not
> u-boot.cfg.
Now, I understood your intention.
u-boot.cfg is not useful for this tool,
although u-boot.cfg also includes configs from Kconfig.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 07/10] moveconfig: Ignore duplicate configs when moving
2015-05-15 5:10 ` Masahiro Yamada
@ 2015-05-16 2:57 ` Joe Hershberger
0 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-16 2:57 UTC (permalink / raw)
To: u-boot
Hi Masahiro-san,
On Fri, May 15, 2015 at 12:10 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> Hi Joe,
>
> 2015-05-15 3:02 GMT+09:00 Joe Hershberger <joe.hershberger@gmail.com>:
>> Hi Masahiro-san,
>>
>> On Thu, May 14, 2015 at 10:36 AM, Masahiro Yamada
>> <yamada.masahiro@socionext.com> wrote:
>>> 2015-05-14 7:28 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>>>> When moving configs, it is important to know what was defined in the
>>>> config header even if it duplicates the configs coming from Kconfig.
>>>>
>>>> tools/moveconfig.py now defines KCONFIG_IGNORE_DUPLICATES to prevent the
>>>> filtering from happening and selecting wrong values for the defconfig.
>>>>
>>>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>>>>
>>>> ---
>>>>
>>>> Changes in v3:
>>>> -New for version 3
>>>>
>>>> Changes in v2: None
>>>>
>>>> scripts/Makefile.autoconf | 3 ++-
>>>> tools/moveconfig.py | 1 +
>>>> 2 files changed, 3 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/scripts/Makefile.autoconf b/scripts/Makefile.autoconf
>>>> index f054081..36bfa17 100644
>>>> --- a/scripts/Makefile.autoconf
>>>> +++ b/scripts/Makefile.autoconf
>>>> @@ -58,7 +58,8 @@ quiet_cmd_autoconf = GEN $@
>>>> $(CPP) $(c_flags) $2 -DDO_DEPS_ONLY -dM $(srctree)/include/common.h > $@.tmp && { \
>>>> sed -n -f $(srctree)/tools/scripts/define2mk.sed $@.tmp | \
>>>> while read line; do \
>>>> - if ! grep -q "$${line%=*}=" include/config/auto.conf; then \
>>>> + if [ -n "${KCONFIG_IGNORE_DUPLICATES}" ] || \
>>>> + ! grep -q "$${line%=*}=" include/config/auto.conf; then \
>>>> echo "$$line"; \
>>>> fi \
>>>> done > $@; \
>>>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>>>> index 9135e1d..97ff597 100755
>>>> --- a/tools/moveconfig.py
>>>> +++ b/tools/moveconfig.py
>>>> @@ -340,6 +340,7 @@ class Slot:
>>>> cmd = list(self.make_cmd)
>>>> if self.cross_compile:
>>>> cmd.append('CROSS_COMPILE=%s' % self.cross_compile)
>>>> + cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
>>>> cmd.append('include/autoconf.mk')
>>>> self.ps = subprocess.Popen(cmd, stdout=self.devnull, stderr=PIPE)
>>>> self.state = STATE_SILENTOLDCONFIG
>>>
>>>
>>> This patch is not clear to me.
>>
>> This was specifically a problem when moving a config that defaulted to
>> true in the Kconfig. That meant that it would be "duplicate removed"
>> from the include since it was already true (by default) in the
>> Kconfig. So it had the effect of disabling that config in every board.
>> This stops the filtering in the case where we are trying to move
>> configs instead of trying to build the board in this mixed config
>> system environment.
>
>
> Uh, make sense.
>
> I was almost forgetting how the "don't apply" version was implemented.
>
> My new one parses .config, defconfig as well as autoconf.mk,
> so please check if we still need to this hack.
I checked it out. The same problem exists and the patch is still needed.
Also, I don't believe it is valid to check the .config for things to
skip. Just because there is a mention of something does not mean it is
the correct value. This is specifically the case for using
savedefconfig. For savedefconfig to work properly, the user must add
the entry to the Kconfig menu first, then run moveconfig.py. You seem
to be expecting that users run moveconfig.py before editing the
Kconfigs.
8<snip>8
Cheers,
-Joe
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 08/10] moveconfig: Handle moving multiple configs at once
2015-05-13 22:28 ` [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk Joe Hershberger
` (5 preceding siblings ...)
2015-05-13 22:28 ` [U-Boot] [PATCH v3 07/10] moveconfig: Ignore duplicate configs when moving Joe Hershberger
@ 2015-05-13 22:28 ` Joe Hershberger
2015-05-14 14:37 ` Masahiro Yamada
2015-05-13 22:28 ` [U-Boot] [PATCH v3 09/10] moveconfig: Print status about the processed defconfigs Joe Hershberger
` (3 subsequent siblings)
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-13 22:28 UTC (permalink / raw)
To: u-boot
Moving configs is a fairly slow process since each board config must
pass through a compiler to evaluate the configs. Also, many configs
are related in a way that makes sense to atomically move.
Add support to tools/moveconfig.py to read multiple lines from the
.moveconfig file and create a parser for each. After compiling the
configs, simply run all the parsers on the autoconf.mk to find the
state of each of those configs.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v3:
-New for version 3
Changes in v2: None
tools/moveconfig.py | 75 ++++++++++++++++++++++++++++++-----------------------
1 file changed, 42 insertions(+), 33 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 97ff597..798f717 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -91,9 +91,9 @@ def cleanup_header(header_path, patterns):
if not i in matched:
f.write(line)
-def cleanup_headers(config):
+def cleanup_headers(config_attrs):
while True:
- choice = raw_input('Clean up %s in headers? [y/n]: ' % config).lower()
+ choice = raw_input('Clean up headers? [y/n]: ').lower()
print choice
if choice == 'y' or choice == 'n':
break
@@ -102,8 +102,9 @@ def cleanup_headers(config):
return
patterns = []
- patterns.append(re.compile(r'#\s*define\s+%s\W' % config))
- patterns.append(re.compile(r'#\s*undef\s+%s\W' % config))
+ for config_attr in config_attrs:
+ patterns.append(re.compile(r'#\s*define\s+%s\W' % config_attr['config']))
+ patterns.append(re.compile(r'#\s*undef\s+%s\W' % config_attr['config']))
for (dirpath, dirnames, filenames) in os.walk('include'):
for filename in filenames:
@@ -126,7 +127,7 @@ class KconfigParser:
PREFIX = { '.': '+', 'spl': 'S', 'tpl': 'T' }
def __init__(self, build_dir, config_attr):
- """Create a new .config perser.
+ """Create a new .config parser.
Arguments:
build_dir: Build directory where .config is located
@@ -141,6 +142,7 @@ class KconfigParser:
else:
self.no_spl_support = False
self.re = re.compile(r'%s=(.*)' % self.config)
+ self.dotconfig = os.path.join(self.build_dir, '.config')
def get_cross_compile(self):
"""Parse .config file and return CROSS_COMPILE
@@ -150,7 +152,6 @@ class KconfigParser:
"""
arch = ''
cpu = ''
- self.dotconfig = os.path.join(self.build_dir, '.config')
for line in open(self.dotconfig):
m = self.re_arch.match(line)
if m:
@@ -204,7 +205,6 @@ class KconfigParser:
value = '(default)'
values.append(value)
- os.remove(autoconf)
if value == '(undef)' or value == '(default)':
continue
@@ -220,6 +220,7 @@ class KconfigParser:
prefixes[output_line] = self.PREFIX[img]
output = defconfig[:-len('_defconfig')].ljust(37) + ': '
+ output += self.config.replace('CONFIG_','').ljust(18) + ': '
for value in values:
output += value.ljust(12)
@@ -241,7 +242,7 @@ class Slot:
for faster processing.
"""
- def __init__(self, config_attr, devnull, make_cmd, options):
+ def __init__(self, config_attrs, devnull, make_cmd, options):
"""Create a new slot.
Arguments:
@@ -251,7 +252,9 @@ class Slot:
self.devnull = devnull
self.make_cmd = (make_cmd, 'O=' + self.build_dir)
self.options = options
- self.parser = KconfigParser(self.build_dir, config_attr)
+ self.parsers = []
+ for config_attr in config_attrs:
+ self.parsers.append(KconfigParser(self.build_dir, config_attr))
self.state = STATE_IDLE
def __del__(self):
@@ -314,10 +317,11 @@ class Slot:
return True
if self.state == STATE_SILENTOLDCONFIG:
- if self.parser.update_defconfig(self.defconfig):
- self.defconfig_error('ERROR - autoconf.mk not found')
- self.state = STATE_IDLE
- return True
+ for parser in self.parsers:
+ if parser.update_defconfig(self.defconfig):
+ self.defconfig_error('ERROR - autoconf.mk not found')
+ self.state = STATE_IDLE
+ return True
"""Save off the defconfig in a consistent way"""
cmd = list(self.make_cmd)
@@ -336,7 +340,7 @@ class Slot:
self.state = STATE_IDLE
return True
- self.cross_compile = self.parser.get_cross_compile()
+ self.cross_compile = self.parsers[0].get_cross_compile()
cmd = list(self.make_cmd)
if self.cross_compile:
cmd.append('CROSS_COMPILE=%s' % self.cross_compile)
@@ -350,7 +354,7 @@ class Slots:
"""Controller of the array of subprocess slots."""
- def __init__(self, config_attr, options):
+ def __init__(self, config_attrs, options):
"""Create a new slots controller.
Arguments:
@@ -360,7 +364,7 @@ class Slots:
devnull = get_devnull()
make_cmd = get_make_cmd()
for i in range(options.jobs):
- self.slots.append(Slot(config_attr, devnull, make_cmd, options))
+ self.slots.append(Slot(config_attrs, devnull, make_cmd, options))
def add(self, defconfig):
"""Add a new subprocess if a vacant slot is available.
@@ -400,15 +404,16 @@ class Slots:
ret = False
return ret
-def move_config(config_attr, options):
+def move_config(config_attrs, options):
check_top_directory()
- print 'Moving %s (type: %s, default: %s, no_spl: %s) ... (jobs: %d)' % (
- config_attr['config'],
- config_attr['type'],
- config_attr['default'],
- config_attr['no_spl_support'],
- options.jobs)
+ for config_attr in config_attrs:
+ print 'Moving %s (type: %s, default: %s, no_spl: %s)' % (
+ config_attr['config'],
+ config_attr['type'],
+ config_attr['default'],
+ config_attr['no_spl_support'])
+ print '%d jobs...' % options.jobs
if options.defconfigs:
defconfigs = [line.strip() for line in open(options.defconfigs, 'r')]
@@ -426,7 +431,7 @@ def move_config(config_attr, options):
if os.path.exists('moveconfig.failed'):
os.remove('moveconfig.failed')
- slots = Slots(config_attr, options)
+ slots = Slots(config_attrs, options)
# Main loop to process defconfig files:
# Add a new subprocess into a vacant slot.
@@ -441,7 +446,7 @@ def move_config(config_attr, options):
while not slots.empty():
time.sleep(SLEEP_TIME)
- cleanup_headers(config_attr['config'])
+ cleanup_headers(config_attrs)
if os.path.exists('moveconfig.failed'):
print '!!! Some boards were not processed; move the config manually.'
@@ -468,31 +473,35 @@ def main():
(options, args) = parser.parse_args()
args_key = ('config', 'type', 'default', 'no_spl_support')
- config_attr = {}
+ config_attrs = []
if len(args) >= len(args_key):
saved_attr = ''
for i, key in enumerate(args_key):
- config_attr[key] = args[i]
+ config_attrs.append({})
+ config_attrs[0][key] = args[i]
saved_attr = saved_attr + ' %s' % args[i]
with open('.moveconfig', 'w') as f:
f.write(saved_attr)
elif os.path.exists('.moveconfig'):
f = open('.moveconfig')
try:
- saved_attr = f.readline().split()
- for i, key in enumerate(args_key):
- config_attr[key] = saved_attr[i]
+ for j, line in enumerate(f):
+ config_attrs.append({})
+ saved_attr = line.split()
+ for i, key in enumerate(args_key):
+ config_attrs[j][key] = saved_attr[i]
except:
sys.exit('%s: broken format' % '.moveconfig')
else:
parser.print_usage()
sys.exit(1)
- if not config_attr['config'].startswith('CONFIG_'):
- config_attr['config'] = 'CONFIG_' + config_attr['config']
+ for config_attr in config_attrs:
+ if not config_attr['config'].startswith('CONFIG_'):
+ config_attr['config'] = 'CONFIG_' + config_attr['config']
- move_config(config_attr, options)
+ move_config(config_attrs, options)
if __name__ == '__main__':
main()
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 08/10] moveconfig: Handle moving multiple configs at once
2015-05-13 22:28 ` [U-Boot] [PATCH v3 08/10] moveconfig: Handle moving multiple configs at once Joe Hershberger
@ 2015-05-14 14:37 ` Masahiro Yamada
2015-05-14 18:05 ` Joe Hershberger
0 siblings, 1 reply; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-14 14:37 UTC (permalink / raw)
To: u-boot
Hi Joe,
2015-05-14 7:28 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> Moving configs is a fairly slow process since each board config must
> pass through a compiler to evaluate the configs. Also, many configs
> are related in a way that makes sense to atomically move.
>
> Add support to tools/moveconfig.py to read multiple lines from the
> .moveconfig file and create a parser for each. After compiling the
> configs, simply run all the parsers on the autoconf.mk to find the
> state of each of those configs.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>
Sorry for leaving this scripts for such a long time.
(I have been busy in the Linux side lately.)
causing two people to develope a similar feature.
I thought the same thing.
My idea was to stop giving config attributes from the command arguments.
Instead, the name of the input file is given as the argument.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 08/10] moveconfig: Handle moving multiple configs at once
2015-05-14 14:37 ` Masahiro Yamada
@ 2015-05-14 18:05 ` Joe Hershberger
0 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-14 18:05 UTC (permalink / raw)
To: u-boot
Hi Masahiro-san,
On Thu, May 14, 2015 at 9:37 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> Hi Joe,
>
> 2015-05-14 7:28 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>> Moving configs is a fairly slow process since each board config must
>> pass through a compiler to evaluate the configs. Also, many configs
>> are related in a way that makes sense to atomically move.
>>
>> Add support to tools/moveconfig.py to read multiple lines from the
>> .moveconfig file and create a parser for each. After compiling the
>> configs, simply run all the parsers on the autoconf.mk to find the
>> state of each of those configs.
>>
>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>>
>
> Sorry for leaving this scripts for such a long time.
> (I have been busy in the Linux side lately.)
> causing two people to develope a similar feature.
Not too big a deal. Hopefully this will be in the tree soon so we can
stop managing this in every branch.
> I thought the same thing.
> My idea was to stop giving config attributes from the command arguments.
> Instead, the name of the input file is given as the argument.
That's fine. I was just trying to maintain your original features.
Cheers,
-Joe
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 09/10] moveconfig: Print status about the processed defconfigs
2015-05-13 22:28 ` [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk Joe Hershberger
` (6 preceding siblings ...)
2015-05-13 22:28 ` [U-Boot] [PATCH v3 08/10] moveconfig: Handle moving multiple configs at once Joe Hershberger
@ 2015-05-13 22:28 ` Joe Hershberger
2015-05-13 22:28 ` [U-Boot] [PATCH v3 10/10] moveconfig: Add a switch to only cleanup headers Joe Hershberger
` (2 subsequent siblings)
10 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-13 22:28 UTC (permalink / raw)
To: u-boot
This gives a basic idea about progress.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v3:
-New for version 3
Changes in v2: None
tools/moveconfig.py | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 798f717..bb087d4 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -93,7 +93,7 @@ def cleanup_header(header_path, patterns):
def cleanup_headers(config_attrs):
while True:
- choice = raw_input('Clean up headers? [y/n]: ').lower()
+ choice = raw_input('\nClean up headers? [y/n]: ').lower()
print choice
if choice == 'y' or choice == 'n':
break
@@ -264,7 +264,7 @@ class Slot:
pass
shutil.rmtree(self.build_dir)
- def add(self, defconfig):
+ def add(self, defconfig, num, total):
"""Add a new subprocess to the slot.
Fails if the slot is occupied, that is, the current subprocess
@@ -283,6 +283,8 @@ class Slot:
self.ps = subprocess.Popen(cmd, stdout=self.devnull)
self.defconfig = defconfig
self.state = STATE_DEFCONFIG
+ self.num = num
+ self.total = total
return True
def defconfig_error(self, errmsg):
@@ -323,6 +325,9 @@ class Slot:
self.state = STATE_IDLE
return True
+ print ' %d defconfigs out of %d\r' % (self.num + 1, self.total),
+ sys.stdout.flush()
+
"""Save off the defconfig in a consistent way"""
cmd = list(self.make_cmd)
cmd.append('savedefconfig')
@@ -366,7 +371,7 @@ class Slots:
for i in range(options.jobs):
self.slots.append(Slot(config_attrs, devnull, make_cmd, options))
- def add(self, defconfig):
+ def add(self, defconfig, num, total):
"""Add a new subprocess if a vacant slot is available.
Arguments:
@@ -376,7 +381,7 @@ class Slots:
Return True on success or False on fail
"""
for slot in self.slots:
- if slot.add(defconfig):
+ if slot.add(defconfig, num, total):
return True
return False
@@ -436,8 +441,8 @@ def move_config(config_attrs, options):
# Main loop to process defconfig files:
# Add a new subprocess into a vacant slot.
# Sleep if there is no available slot.
- for defconfig in defconfigs:
- while not slots.add(defconfig):
+ for i, defconfig in enumerate(defconfigs):
+ while not slots.add(defconfig, i, len(defconfigs)):
while not slots.available():
# No available slot: sleep for a while
time.sleep(SLEEP_TIME)
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 10/10] moveconfig: Add a switch to only cleanup headers
2015-05-13 22:28 ` [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk Joe Hershberger
` (7 preceding siblings ...)
2015-05-13 22:28 ` [U-Boot] [PATCH v3 09/10] moveconfig: Print status about the processed defconfigs Joe Hershberger
@ 2015-05-13 22:28 ` Joe Hershberger
2015-05-14 14:51 ` Masahiro Yamada
2015-05-14 13:04 ` [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk Masahiro Yamada
2015-05-15 21:40 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-13 22:28 UTC (permalink / raw)
To: u-boot
In some case you may want to only cleanup the headers. Make it possible
without waiting for all boards to compile.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v3:
-New for version 3
Changes in v2: None
tools/moveconfig.py | 83 ++++++++++++++++++++++++++++-------------------------
1 file changed, 44 insertions(+), 39 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index bb087d4..d63f47f 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -412,48 +412,50 @@ class Slots:
def move_config(config_attrs, options):
check_top_directory()
- for config_attr in config_attrs:
- print 'Moving %s (type: %s, default: %s, no_spl: %s)' % (
- config_attr['config'],
- config_attr['type'],
- config_attr['default'],
- config_attr['no_spl_support'])
- print '%d jobs...' % options.jobs
-
- if options.defconfigs:
- defconfigs = [line.strip() for line in open(options.defconfigs, 'r')]
- else:
- # All the defconfig files to be processed
- defconfigs = []
- for (dirpath, dirnames, filenames) in os.walk('configs'):
- dirpath = dirpath[len('configs') + 1:]
- for filename in fnmatch.filter(filenames, '*_defconfig'):
- if fnmatch.fnmatch(filename, '.*'):
- continue
- defconfigs.append(os.path.join(dirpath, filename))
-
- """Clean up any previous log of failed moves"""
- if os.path.exists('moveconfig.failed'):
- os.remove('moveconfig.failed')
-
- slots = Slots(config_attrs, options)
-
- # Main loop to process defconfig files:
- # Add a new subprocess into a vacant slot.
- # Sleep if there is no available slot.
- for i, defconfig in enumerate(defconfigs):
- while not slots.add(defconfig, i, len(defconfigs)):
- while not slots.available():
- # No available slot: sleep for a while
- time.sleep(SLEEP_TIME)
-
- # wait until all the subprocesses finish
- while not slots.empty():
- time.sleep(SLEEP_TIME)
+ if not options.clean_only:
+ for config_attr in config_attrs:
+ print 'Moving %s (type: %s, default: %s, no_spl: %s)' % (
+ config_attr['config'],
+ config_attr['type'],
+ config_attr['default'],
+ config_attr['no_spl_support'])
+ print '%d jobs...' % options.jobs
+
+ if options.defconfigs:
+ defconfigs = [line.strip() for line in
+ open(options.defconfigs, 'r')]
+ else:
+ # All the defconfig files to be processed
+ defconfigs = []
+ for (dirpath, dirnames, filenames) in os.walk('configs'):
+ dirpath = dirpath[len('configs') + 1:]
+ for filename in fnmatch.filter(filenames, '*_defconfig'):
+ if fnmatch.fnmatch(filename, '.*'):
+ continue
+ defconfigs.append(os.path.join(dirpath, filename))
+
+ """Clean up any previous log of failed moves"""
+ if os.path.exists('moveconfig.failed'):
+ os.remove('moveconfig.failed')
+
+ slots = Slots(config_attrs, options)
+
+ # Main loop to process defconfig files:
+ # Add a new subprocess into a vacant slot.
+ # Sleep if there is no available slot.
+ for i, defconfig in enumerate(defconfigs):
+ while not slots.add(defconfig, i, len(defconfigs)):
+ while not slots.available():
+ # No available slot: sleep for a while
+ time.sleep(SLEEP_TIME)
+
+ # wait until all the subprocesses finish
+ while not slots.empty():
+ time.sleep(SLEEP_TIME)
cleanup_headers(config_attrs)
- if os.path.exists('moveconfig.failed'):
+ if (not options.clean_only) & os.path.exists('moveconfig.failed'):
print '!!! Some boards were not processed; move the config manually.'
print '!!! The list of failed boards are saved in moveconfig.failed'
print
@@ -467,6 +469,9 @@ def main():
parser = optparse.OptionParser()
# Add options here
+ parser.add_option('-c', '--clean-only', dest='clean_only',
+ action='store_true', default=False,
+ help='only clean the headers')
parser.add_option('-d', '--defconfigs', type='string',
help='a file containing a list of defconfigs to move')
parser.add_option('-j', '--jobs', type='int', default=cpu_count,
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 10/10] moveconfig: Add a switch to only cleanup headers
2015-05-13 22:28 ` [U-Boot] [PATCH v3 10/10] moveconfig: Add a switch to only cleanup headers Joe Hershberger
@ 2015-05-14 14:51 ` Masahiro Yamada
2015-05-14 18:03 ` Joe Hershberger
0 siblings, 1 reply; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-14 14:51 UTC (permalink / raw)
To: u-boot
2015-05-14 7:28 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> In some case you may want to only cleanup the headers. Make it possible
> without waiting for all boards to compile.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>
> ---
>
> Changes in v3:
> -New for version 3
>
> Changes in v2: None
>
> tools/moveconfig.py | 83 ++++++++++++++++++++++++++++-------------------------
> 1 file changed, 44 insertions(+), 39 deletions(-)
>
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index bb087d4..d63f47f 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -412,48 +412,50 @@ class Slots:
> def move_config(config_attrs, options):
> check_top_directory()
>
> - for config_attr in config_attrs:
> - print 'Moving %s (type: %s, default: %s, no_spl: %s)' % (
> - config_attr['config'],
> - config_attr['type'],
> - config_attr['default'],
> - config_attr['no_spl_support'])
> - print '%d jobs...' % options.jobs
> -
> - if options.defconfigs:
> - defconfigs = [line.strip() for line in open(options.defconfigs, 'r')]
> - else:
> - # All the defconfig files to be processed
> - defconfigs = []
> - for (dirpath, dirnames, filenames) in os.walk('configs'):
> - dirpath = dirpath[len('configs') + 1:]
> - for filename in fnmatch.filter(filenames, '*_defconfig'):
> - if fnmatch.fnmatch(filename, '.*'):
> - continue
> - defconfigs.append(os.path.join(dirpath, filename))
> -
> - """Clean up any previous log of failed moves"""
> - if os.path.exists('moveconfig.failed'):
> - os.remove('moveconfig.failed')
> -
> - slots = Slots(config_attrs, options)
> -
> - # Main loop to process defconfig files:
> - # Add a new subprocess into a vacant slot.
> - # Sleep if there is no available slot.
> - for i, defconfig in enumerate(defconfigs):
> - while not slots.add(defconfig, i, len(defconfigs)):
> - while not slots.available():
> - # No available slot: sleep for a while
> - time.sleep(SLEEP_TIME)
> -
> - # wait until all the subprocesses finish
> - while not slots.empty():
> - time.sleep(SLEEP_TIME)
> + if not options.clean_only:
> + for config_attr in config_attrs:
> + print 'Moving %s (type: %s, default: %s, no_spl: %s)' % (
> + config_attr['config'],
> + config_attr['type'],
> + config_attr['default'],
> + config_attr['no_spl_support'])
> + print '%d jobs...' % options.jobs
> +
> + if options.defconfigs:
> + defconfigs = [line.strip() for line in
> + open(options.defconfigs, 'r')]
> + else:
> + # All the defconfig files to be processed
> + defconfigs = []
> + for (dirpath, dirnames, filenames) in os.walk('configs'):
> + dirpath = dirpath[len('configs') + 1:]
> + for filename in fnmatch.filter(filenames, '*_defconfig'):
> + if fnmatch.fnmatch(filename, '.*'):
> + continue
> + defconfigs.append(os.path.join(dirpath, filename))
> +
> + """Clean up any previous log of failed moves"""
> + if os.path.exists('moveconfig.failed'):
> + os.remove('moveconfig.failed')
> +
> + slots = Slots(config_attrs, options)
> +
> + # Main loop to process defconfig files:
> + # Add a new subprocess into a vacant slot.
> + # Sleep if there is no available slot.
> + for i, defconfig in enumerate(defconfigs):
> + while not slots.add(defconfig, i, len(defconfigs)):
> + while not slots.available():
> + # No available slot: sleep for a while
> + time.sleep(SLEEP_TIME)
> +
> + # wait until all the subprocesses finish
> + while not slots.empty():
> + time.sleep(SLEEP_TIME)
>
> cleanup_headers(config_attrs)
>
> - if os.path.exists('moveconfig.failed'):
> + if (not options.clean_only) & os.path.exists('moveconfig.failed'):
> print '!!! Some boards were not processed; move the config manually.'
> print '!!! The list of failed boards are saved in moveconfig.failed'
> print
I am OK with this feature, but this 'if' statement ranges over many code lines.
Perhaps, moving the cleanup_headers to the main function is simpler?
def main():
[snip]
if not options.clean_only:
move_config(config_attrs, options)
cleanup_headers(config_attrs)
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 10/10] moveconfig: Add a switch to only cleanup headers
2015-05-14 14:51 ` Masahiro Yamada
@ 2015-05-14 18:03 ` Joe Hershberger
0 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-14 18:03 UTC (permalink / raw)
To: u-boot
Hi Masahiro-san,
On Thu, May 14, 2015 at 9:51 AM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2015-05-14 7:28 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>> In some case you may want to only cleanup the headers. Make it possible
>> without waiting for all boards to compile.
>>
>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>>
>> ---
>>
>> Changes in v3:
>> -New for version 3
>>
>> Changes in v2: None
>>
>> tools/moveconfig.py | 83 ++++++++++++++++++++++++++++-------------------------
>> 1 file changed, 44 insertions(+), 39 deletions(-)
>>
>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>> index bb087d4..d63f47f 100755
>> --- a/tools/moveconfig.py
>> +++ b/tools/moveconfig.py
>> @@ -412,48 +412,50 @@ class Slots:
>> def move_config(config_attrs, options):
>> check_top_directory()
>>
>> - for config_attr in config_attrs:
>> - print 'Moving %s (type: %s, default: %s, no_spl: %s)' % (
>> - config_attr['config'],
>> - config_attr['type'],
>> - config_attr['default'],
>> - config_attr['no_spl_support'])
>> - print '%d jobs...' % options.jobs
>> -
>> - if options.defconfigs:
>> - defconfigs = [line.strip() for line in open(options.defconfigs, 'r')]
>> - else:
>> - # All the defconfig files to be processed
>> - defconfigs = []
>> - for (dirpath, dirnames, filenames) in os.walk('configs'):
>> - dirpath = dirpath[len('configs') + 1:]
>> - for filename in fnmatch.filter(filenames, '*_defconfig'):
>> - if fnmatch.fnmatch(filename, '.*'):
>> - continue
>> - defconfigs.append(os.path.join(dirpath, filename))
>> -
>> - """Clean up any previous log of failed moves"""
>> - if os.path.exists('moveconfig.failed'):
>> - os.remove('moveconfig.failed')
>> -
>> - slots = Slots(config_attrs, options)
>> -
>> - # Main loop to process defconfig files:
>> - # Add a new subprocess into a vacant slot.
>> - # Sleep if there is no available slot.
>> - for i, defconfig in enumerate(defconfigs):
>> - while not slots.add(defconfig, i, len(defconfigs)):
>> - while not slots.available():
>> - # No available slot: sleep for a while
>> - time.sleep(SLEEP_TIME)
>> -
>> - # wait until all the subprocesses finish
>> - while not slots.empty():
>> - time.sleep(SLEEP_TIME)
>> + if not options.clean_only:
>> + for config_attr in config_attrs:
>> + print 'Moving %s (type: %s, default: %s, no_spl: %s)' % (
>> + config_attr['config'],
>> + config_attr['type'],
>> + config_attr['default'],
>> + config_attr['no_spl_support'])
>> + print '%d jobs...' % options.jobs
>> +
>> + if options.defconfigs:
>> + defconfigs = [line.strip() for line in
>> + open(options.defconfigs, 'r')]
>> + else:
>> + # All the defconfig files to be processed
>> + defconfigs = []
>> + for (dirpath, dirnames, filenames) in os.walk('configs'):
>> + dirpath = dirpath[len('configs') + 1:]
>> + for filename in fnmatch.filter(filenames, '*_defconfig'):
>> + if fnmatch.fnmatch(filename, '.*'):
>> + continue
>> + defconfigs.append(os.path.join(dirpath, filename))
>> +
>> + """Clean up any previous log of failed moves"""
>> + if os.path.exists('moveconfig.failed'):
>> + os.remove('moveconfig.failed')
>> +
>> + slots = Slots(config_attrs, options)
>> +
>> + # Main loop to process defconfig files:
>> + # Add a new subprocess into a vacant slot.
>> + # Sleep if there is no available slot.
>> + for i, defconfig in enumerate(defconfigs):
>> + while not slots.add(defconfig, i, len(defconfigs)):
>> + while not slots.available():
>> + # No available slot: sleep for a while
>> + time.sleep(SLEEP_TIME)
>> +
>> + # wait until all the subprocesses finish
>> + while not slots.empty():
>> + time.sleep(SLEEP_TIME)
>>
>> cleanup_headers(config_attrs)
>>
>> - if os.path.exists('moveconfig.failed'):
>> + if (not options.clean_only) & os.path.exists('moveconfig.failed'):
>> print '!!! Some boards were not processed; move the config manually.'
>> print '!!! The list of failed boards are saved in moveconfig.failed'
>> print
>
>
> I am OK with this feature, but this 'if' statement ranges over many code lines.
>
> Perhaps, moving the cleanup_headers to the main function is simpler?
>
>
> def main():
> [snip]
>
> if not options.clean_only:
> move_config(config_attrs, options)
>
> cleanup_headers(config_attrs)
OK.
-Joe
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk
2015-05-13 22:28 ` [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk Joe Hershberger
` (8 preceding siblings ...)
2015-05-13 22:28 ` [U-Boot] [PATCH v3 10/10] moveconfig: Add a switch to only cleanup headers Joe Hershberger
@ 2015-05-14 13:04 ` Masahiro Yamada
2015-05-15 21:40 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
10 siblings, 0 replies; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-14 13:04 UTC (permalink / raw)
To: u-boot
2015-05-14 7:28 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> The existing target won't actually build the needed .mk file that is
> expected by the rest of the script. It seems that silentoldconfig does
> not actually cause this to be built any longer.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
> ---
> This patch is based on:
> "Don't apply: tools: add a tool to move automatically CONFIGs from headers to defconfigs"
> from the list.
>
> Also dropped the patch that adds an arc cross-tool mapping.
>
> Changes in v3: None
> Changes in v2: None
>
> tools/moveconfig.py | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index cb7f7ea..30dc4f6 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -301,7 +301,7 @@ class Slot:
> cmd = list(self.make_cmd)
> if cross_compile:
> cmd.append('CROSS_COMPILE=%s' % cross_compile)
> - cmd.append('silentoldconfig')
> + cmd.append('include/autoconf.mk')
> self.ps = subprocess.Popen(cmd, stdout=self.devnull)
> self.state = STATE_SILENTOLDCONFIG
> return False
Yup, the 'Don't apply' version is not working since I removed
scripts/multiconfig.sh.
I had locally modified this tool though I tweaked a bit differently.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config
2015-05-13 22:28 ` [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk Joe Hershberger
` (9 preceding siblings ...)
2015-05-14 13:04 ` [U-Boot] [PATCH v3 01/10] moveconfig: Actually build autoconf.mk Masahiro Yamada
@ 2015-05-15 21:40 ` Joe Hershberger
2015-05-15 21:40 ` [U-Boot] [PATCH v4 02/10] moveconfig: Ignore duplicate configs when moving Joe Hershberger
` (10 more replies)
10 siblings, 11 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-15 21:40 UTC (permalink / raw)
To: u-boot
This will ensure that the order of the defconfig entries will always
match that of the Kconfig files. After one slightly painful (but
still early in the process) pass over all boards, this should keep
the defconfigs clean from here on.
Users must edit the Kconfig first to add the menu entries and then run
moveconfig.py to update the defconfig files and the include configs.
As such, moveconfig.py cannot compare against the '.config' contents.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
This is based on https://patchwork.ozlabs.org/patch/472591/
Changes in v4:
-Rebased series on Masahiro's v2
Changes in v3: None
Changes in v2: None
tools/moveconfig.py | 35 ++++++++++++++++++++++++++---------
1 file changed, 26 insertions(+), 9 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index c39ea95..544f6af 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -46,6 +46,9 @@ should look like this:
CONFIG_CMD_USB bool n
CONFIG_SYS_TEXT_BASE hex 0x00000000
+Next you must edit the Kconfig to add the menu entries for the configs
+you are moving.
+
And then run this tool giving the file name of the recipe
$ tools/moveconfig.py recipe
@@ -192,6 +195,7 @@ CROSS_COMPILE = {
STATE_IDLE = 0
STATE_DEFCONFIG = 1
STATE_AUTOCONF = 2
+STATE_SAVEDEFCONFIG = 3
ACTION_MOVE = 0
ACTION_DEFAULT_VALUE = 1
@@ -390,8 +394,7 @@ class KconfigParser:
return CROSS_COMPILE.get(arch, '')
- def parse_one_config(self, config_attr, defconfig_lines,
- dotconfig_lines, autoconf_lines):
+ def parse_one_config(self, config_attr, defconfig_lines, autoconf_lines):
"""Parse .config, defconfig, include/autoconf.mk for one config.
This function looks for the config options in the lines from
@@ -402,7 +405,6 @@ class KconfigParser:
config_attr: A dictionary including the name, the type,
and the default value of the target config.
defconfig_lines: lines from the original defconfig file.
- dotconfig_lines: lines from the .config file.
autoconf_lines: lines from the include/autoconf.mk file.
Returns:
@@ -418,7 +420,7 @@ class KconfigParser:
else:
default = config + '=' + config_attr['default']
- for line in defconfig_lines + dotconfig_lines:
+ for line in defconfig_lines:
line = line.rstrip()
if line.startswith(config + '=') or line == not_set:
return (ACTION_ALREADY_EXIST, line)
@@ -463,15 +465,12 @@ class KconfigParser:
with open(defconfig_path) as f:
defconfig_lines = f.readlines()
- with open(dotconfig_path) as f:
- dotconfig_lines = f.readlines()
-
with open(autoconf_path) as f:
autoconf_lines = f.readlines()
for config_attr in self.config_attrs:
result = self.parse_one_config(config_attr, defconfig_lines,
- dotconfig_lines, autoconf_lines)
+ autoconf_lines)
results.append(result)
log = ''
@@ -499,7 +498,7 @@ class KconfigParser:
print log,
if not self.options.dry_run:
- with open(defconfig_path, 'a') as f:
+ with open(dotconfig_path, 'a') as f:
for (action, value) in results:
if action == ACTION_MOVE:
f.write(value + '\n')
@@ -608,6 +607,24 @@ class Slot:
if self.state == STATE_AUTOCONF:
self.parser.update_defconfig(self.defconfig)
+
+ """Save off the defconfig in a consistent way"""
+ cmd = list(self.make_cmd)
+ cmd.append('savedefconfig')
+ self.ps = subprocess.Popen(cmd, stdout=self.devnull,
+ stderr=self.devnull)
+ self.state = STATE_SAVEDEFCONFIG
+ return False
+
+ if self.state == STATE_SAVEDEFCONFIG:
+ defconfig_path = os.path.join(self.build_dir, 'defconfig')
+ if not os.path.exists(defconfig_path):
+ print >> sys.stderr, log_msg(self.options.color,
+ COLOR_LIGHT_RED,
+ self.defconfig,
+ 'The defconfig was not updated')
+ shutil.move(defconfig_path,
+ os.path.join('configs', self.defconfig))
self.state = STATE_IDLE
return True
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 02/10] moveconfig: Ignore duplicate configs when moving
2015-05-15 21:40 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
@ 2015-05-15 21:40 ` Joe Hershberger
2015-05-19 4:37 ` Masahiro Yamada
2015-05-15 21:40 ` [U-Boot] [PATCH v4 03/10] moveconfig: Add a parameter to accept a list to build Joe Hershberger
` (9 subsequent siblings)
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-15 21:40 UTC (permalink / raw)
To: u-boot
When moving configs, it is important to know what was defined in the
config header even if it duplicates the configs coming from Kconfig.
This is specifically needed for the case where a config is set to
default 'y' in the Kconfig. This would previously cause the actual value
from the include config to be filtered out, and moveconfig.py would
think that it was 'n'... This means that the value that should be 'y'
is now (in every defconfig) set to 'not set'.
tools/moveconfig.py now defines KCONFIG_IGNORE_DUPLICATES to prevent the
filtering from happening and selecting wrong values for the defconfig.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v4: None
Changes in v3:
-New for version 3
Changes in v2: None
scripts/Makefile.autoconf | 3 ++-
tools/moveconfig.py | 1 +
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/scripts/Makefile.autoconf b/scripts/Makefile.autoconf
index f054081..36bfa17 100644
--- a/scripts/Makefile.autoconf
+++ b/scripts/Makefile.autoconf
@@ -58,7 +58,8 @@ quiet_cmd_autoconf = GEN $@
$(CPP) $(c_flags) $2 -DDO_DEPS_ONLY -dM $(srctree)/include/common.h > $@.tmp && { \
sed -n -f $(srctree)/tools/scripts/define2mk.sed $@.tmp | \
while read line; do \
- if ! grep -q "$${line%=*}=" include/config/auto.conf; then \
+ if [ -n "${KCONFIG_IGNORE_DUPLICATES}" ] || \
+ ! grep -q "$${line%=*}=" include/config/auto.conf; then \
echo "$$line"; \
fi \
done > $@; \
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 544f6af..d3009de 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -632,6 +632,7 @@ class Slot:
cmd = list(self.make_cmd)
if cross_compile:
cmd.append('CROSS_COMPILE=%s' % cross_compile)
+ cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
cmd.append('include/config/auto.conf')
self.ps = subprocess.Popen(cmd, stdout=self.devnull)
self.state = STATE_AUTOCONF
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 02/10] moveconfig: Ignore duplicate configs when moving
2015-05-15 21:40 ` [U-Boot] [PATCH v4 02/10] moveconfig: Ignore duplicate configs when moving Joe Hershberger
@ 2015-05-19 4:37 ` Masahiro Yamada
0 siblings, 0 replies; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-19 4:37 UTC (permalink / raw)
To: u-boot
2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> When moving configs, it is important to know what was defined in the
> config header even if it duplicates the configs coming from Kconfig.
>
> This is specifically needed for the case where a config is set to
> default 'y' in the Kconfig. This would previously cause the actual value
> from the include config to be filtered out, and moveconfig.py would
> think that it was 'n'... This means that the value that should be 'y'
> is now (in every defconfig) set to 'not set'.
>
> tools/moveconfig.py now defines KCONFIG_IGNORE_DUPLICATES to prevent the
> filtering from happening and selecting wrong values for the defconfig.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>
> ---
>
> Changes in v4: None
> Changes in v3:
> -New for version 3
>
> Changes in v2: None
>
> scripts/Makefile.autoconf | 3 ++-
> tools/moveconfig.py | 1 +
> 2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/Makefile.autoconf b/scripts/Makefile.autoconf
> index f054081..36bfa17 100644
> --- a/scripts/Makefile.autoconf
> +++ b/scripts/Makefile.autoconf
> @@ -58,7 +58,8 @@ quiet_cmd_autoconf = GEN $@
> $(CPP) $(c_flags) $2 -DDO_DEPS_ONLY -dM $(srctree)/include/common.h > $@.tmp && { \
> sed -n -f $(srctree)/tools/scripts/define2mk.sed $@.tmp | \
> while read line; do \
> - if ! grep -q "$${line%=*}=" include/config/auto.conf; then \
> + if [ -n "${KCONFIG_IGNORE_DUPLICATES}" ] || \
> + ! grep -q "$${line%=*}=" include/config/auto.conf; then \
> echo "$$line"; \
> fi \
> done > $@; \
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index 544f6af..d3009de 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -632,6 +632,7 @@ class Slot:
> cmd = list(self.make_cmd)
> if cross_compile:
> cmd.append('CROSS_COMPILE=%s' % cross_compile)
> + cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
> cmd.append('include/config/auto.conf')
> self.ps = subprocess.Popen(cmd, stdout=self.devnull)
> self.state = STATE_AUTOCONF
> --
Now, I fully understood your intention.
This comes from our work-flow differences.
Because I edited the Kconfig after moving configs that are default to y,
so I did not notice this necessity.
Anyway, we should do it beforehand for savedefconfig.
So, this looks good to me.
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 03/10] moveconfig: Add a parameter to accept a list to build
2015-05-15 21:40 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
2015-05-15 21:40 ` [U-Boot] [PATCH v4 02/10] moveconfig: Ignore duplicate configs when moving Joe Hershberger
@ 2015-05-15 21:40 ` Joe Hershberger
2015-05-19 4:33 ` Masahiro Yamada
2015-05-15 21:40 ` [U-Boot] [PATCH v4 04/10] moveconfig: Add a switch to only cleanup headers Joe Hershberger
` (8 subsequent siblings)
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-15 21:40 UTC (permalink / raw)
To: u-boot
This is helpful to re-attempt to move failed boards from a previous run
without starting over.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v4: None
Changes in v3:
-Fixed command line options order (alphabetize)
Changes in v2:
-New for version 2
tools/moveconfig.py | 26 ++++++++++++++++++++------
1 file changed, 20 insertions(+), 6 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index d3009de..3f31e81 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -135,6 +135,9 @@ Available options
Surround each portion of the log with escape sequences to display it
in color on the terminal.
+ -d, --defconfigs
+ Specify a file containing a list of defconfigs to move
+
-n, --dry-run
Peform a trial run that does not make any changes. It is useful to
see what is going to happen before one actually runs it.
@@ -734,12 +737,21 @@ def move_config(config_attrs, options):
config_attr['type'],
config_attr['default'])
- # All the defconfig files to be processed
- defconfigs = []
- for (dirpath, dirnames, filenames) in os.walk('configs'):
- dirpath = dirpath[len('configs') + 1:]
- for filename in fnmatch.filter(filenames, '*_defconfig'):
- defconfigs.append(os.path.join(dirpath, filename))
+ if options.defconfigs:
+ defconfigs = [line.strip() for line in open(options.defconfigs, 'r')]
+ for i, defconfig in enumerate(defconfigs):
+ if not defconfig.endswith('_defconfig'):
+ defconfigs[i] = defconfig + '_defconfig'
+ if not os.path.exists(os.path.join('configs', defconfigs[i])):
+ sys.exit('%s - defconfig does not exist. Stopping.' %
+ defconfigs[i])
+ else:
+ # All the defconfig files to be processed
+ defconfigs = []
+ for (dirpath, dirnames, filenames) in os.walk('configs'):
+ dirpath = dirpath[len('configs') + 1:]
+ for filename in fnmatch.filter(filenames, '*_defconfig'):
+ defconfigs.append(os.path.join(dirpath, filename))
slots = Slots(config_attrs, options)
@@ -840,6 +852,8 @@ def main():
# Add options here
parser.add_option('-c', '--color', action='store_true', default=False,
help='display the log in color')
+ parser.add_option('-d', '--defconfigs', type='string',
+ help='a file containing a list of defconfigs to move')
parser.add_option('-n', '--dry-run', action='store_true', default=False,
help='perform a trial run (show log with no changes)')
parser.add_option('-e', '--exit-on-error', action='store_true',
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 03/10] moveconfig: Add a parameter to accept a list to build
2015-05-15 21:40 ` [U-Boot] [PATCH v4 03/10] moveconfig: Add a parameter to accept a list to build Joe Hershberger
@ 2015-05-19 4:33 ` Masahiro Yamada
2015-05-19 17:58 ` Joe Hershberger
0 siblings, 1 reply; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-19 4:33 UTC (permalink / raw)
To: u-boot
2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> This is helpful to re-attempt to move failed boards from a previous run
> without starting over.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>
> ---
>
> Changes in v4: None
> Changes in v3:
> -Fixed command line options order (alphabetize)
>
> Changes in v2:
> -New for version 2
>
> tools/moveconfig.py | 26 ++++++++++++++++++++------
> 1 file changed, 20 insertions(+), 6 deletions(-)
>
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index d3009de..3f31e81 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -135,6 +135,9 @@ Available options
> Surround each portion of the log with escape sequences to display it
> in color on the terminal.
>
> + -d, --defconfigs
> + Specify a file containing a list of defconfigs to move
> +
> -n, --dry-run
> Peform a trial run that does not make any changes. It is useful to
> see what is going to happen before one actually runs it.
> @@ -734,12 +737,21 @@ def move_config(config_attrs, options):
> config_attr['type'],
> config_attr['default'])
>
> - # All the defconfig files to be processed
> - defconfigs = []
> - for (dirpath, dirnames, filenames) in os.walk('configs'):
> - dirpath = dirpath[len('configs') + 1:]
> - for filename in fnmatch.filter(filenames, '*_defconfig'):
> - defconfigs.append(os.path.join(dirpath, filename))
> + if options.defconfigs:
> + defconfigs = [line.strip() for line in open(options.defconfigs, 'r')]
> + for i, defconfig in enumerate(defconfigs):
> + if not defconfig.endswith('_defconfig'):
> + defconfigs[i] = defconfig + '_defconfig'
> + if not os.path.exists(os.path.join('configs', defconfigs[i])):
> + sys.exit('%s - defconfig does not exist. Stopping.' %
> + defconfigs[i])
'r' is redundant for open() because read-mode is default.
moveconfig.failed always prefixes _defconfig,
so we need not to do so again, I think. (or will we make this file by hand?)
Then, we can drop enumarate and write the error message within 80 columns.
if options.defconfigs:
defconfigs = [line.strip() for line in open(options.defconfigs)]
for defconfig in defconfigs:
if not os.path.exists(os.path.join('configs', defconfig)):
sys.exit('%s: defconfig does not exist. Stopping.' % defconfig)
> slots = Slots(config_attrs, options)
>
> @@ -840,6 +852,8 @@ def main():
> # Add options here
> parser.add_option('-c', '--color', action='store_true', default=False,
> help='display the log in color')
> + parser.add_option('-d', '--defconfigs', type='string',
> + help='a file containing a list of defconfigs to move')
> parser.add_option('-n', '--dry-run', action='store_true', default=False,
> help='perform a trial run (show log with no changes)')
> parser.add_option('-e', '--exit-on-error', action='store_true',
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 03/10] moveconfig: Add a parameter to accept a list to build
2015-05-19 4:33 ` Masahiro Yamada
@ 2015-05-19 17:58 ` Joe Hershberger
2015-05-20 3:01 ` Masahiro Yamada
0 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 17:58 UTC (permalink / raw)
To: u-boot
Hi Masahiro-san,
On Mon, May 18, 2015 at 11:33 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>> This is helpful to re-attempt to move failed boards from a previous run
>> without starting over.
>>
>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>>
>> ---
>>
>> Changes in v4: None
>> Changes in v3:
>> -Fixed command line options order (alphabetize)
>>
>> Changes in v2:
>> -New for version 2
>>
>> tools/moveconfig.py | 26 ++++++++++++++++++++------
>> 1 file changed, 20 insertions(+), 6 deletions(-)
>>
>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>> index d3009de..3f31e81 100755
>> --- a/tools/moveconfig.py
>> +++ b/tools/moveconfig.py
>> @@ -135,6 +135,9 @@ Available options
>> Surround each portion of the log with escape sequences to display it
>> in color on the terminal.
>>
>> + -d, --defconfigs
>> + Specify a file containing a list of defconfigs to move
>> +
>> -n, --dry-run
>> Peform a trial run that does not make any changes. It is useful to
>> see what is going to happen before one actually runs it.
>> @@ -734,12 +737,21 @@ def move_config(config_attrs, options):
>> config_attr['type'],
>> config_attr['default'])
>>
>> - # All the defconfig files to be processed
>> - defconfigs = []
>> - for (dirpath, dirnames, filenames) in os.walk('configs'):
>> - dirpath = dirpath[len('configs') + 1:]
>> - for filename in fnmatch.filter(filenames, '*_defconfig'):
>> - defconfigs.append(os.path.join(dirpath, filename))
>> + if options.defconfigs:
>> + defconfigs = [line.strip() for line in open(options.defconfigs, 'r')]
>> + for i, defconfig in enumerate(defconfigs):
>> + if not defconfig.endswith('_defconfig'):
>> + defconfigs[i] = defconfig + '_defconfig'
>> + if not os.path.exists(os.path.join('configs', defconfigs[i])):
>> + sys.exit('%s - defconfig does not exist. Stopping.' %
>> + defconfigs[i])
>
>
> 'r' is redundant for open() because read-mode is default.
OK.
> moveconfig.failed always prefixes _defconfig,
> so we need not to do so again, I think. (or will we make this file by hand?)
I have done both moveconfig.failed as well as a hand-written file for
testing certain boards. That's why I added both the append
'_defconfig' as well as the check for it actually existing.
> Then, we can drop enumarate and write the error message within 80 columns.
>
> if options.defconfigs:
> defconfigs = [line.strip() for line in open(options.defconfigs)]
> for defconfig in defconfigs:
> if not os.path.exists(os.path.join('configs', defconfig)):
> sys.exit('%s: defconfig does not exist. Stopping.' % defconfig)
I think we should keep the check for endswith('_defconfig').
>> slots = Slots(config_attrs, options)
>>
>> @@ -840,6 +852,8 @@ def main():
>> # Add options here
>> parser.add_option('-c', '--color', action='store_true', default=False,
>> help='display the log in color')
>> + parser.add_option('-d', '--defconfigs', type='string',
>> + help='a file containing a list of defconfigs to move')
>> parser.add_option('-n', '--dry-run', action='store_true', default=False,
>> help='perform a trial run (show log with no changes)')
>> parser.add_option('-e', '--exit-on-error', action='store_true',
Cheers,
-Joe
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 03/10] moveconfig: Add a parameter to accept a list to build
2015-05-19 17:58 ` Joe Hershberger
@ 2015-05-20 3:01 ` Masahiro Yamada
0 siblings, 0 replies; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-20 3:01 UTC (permalink / raw)
To: u-boot
2015-05-20 2:58 GMT+09:00 Joe Hershberger <joe.hershberger@gmail.com>:
> Hi Masahiro-san,
>
> On Mon, May 18, 2015 at 11:33 PM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> 2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>>> This is helpful to re-attempt to move failed boards from a previous run
>>> without starting over.
>>>
>>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>>>
>>> ---
>>>
>>> Changes in v4: None
>>> Changes in v3:
>>> -Fixed command line options order (alphabetize)
>>>
>>> Changes in v2:
>>> -New for version 2
>>>
>>> tools/moveconfig.py | 26 ++++++++++++++++++++------
>>> 1 file changed, 20 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>>> index d3009de..3f31e81 100755
>>> --- a/tools/moveconfig.py
>>> +++ b/tools/moveconfig.py
>>> @@ -135,6 +135,9 @@ Available options
>>> Surround each portion of the log with escape sequences to display it
>>> in color on the terminal.
>>>
>>> + -d, --defconfigs
>>> + Specify a file containing a list of defconfigs to move
>>> +
>>> -n, --dry-run
>>> Peform a trial run that does not make any changes. It is useful to
>>> see what is going to happen before one actually runs it.
>>> @@ -734,12 +737,21 @@ def move_config(config_attrs, options):
>>> config_attr['type'],
>>> config_attr['default'])
>>>
>>> - # All the defconfig files to be processed
>>> - defconfigs = []
>>> - for (dirpath, dirnames, filenames) in os.walk('configs'):
>>> - dirpath = dirpath[len('configs') + 1:]
>>> - for filename in fnmatch.filter(filenames, '*_defconfig'):
>>> - defconfigs.append(os.path.join(dirpath, filename))
>>> + if options.defconfigs:
>>> + defconfigs = [line.strip() for line in open(options.defconfigs, 'r')]
>>> + for i, defconfig in enumerate(defconfigs):
>>> + if not defconfig.endswith('_defconfig'):
>>> + defconfigs[i] = defconfig + '_defconfig'
>>> + if not os.path.exists(os.path.join('configs', defconfigs[i])):
>>> + sys.exit('%s - defconfig does not exist. Stopping.' %
>>> + defconfigs[i])
>>
>>
>> 'r' is redundant for open() because read-mode is default.
>
> OK.
>
>> moveconfig.failed always prefixes _defconfig,
>> so we need not to do so again, I think. (or will we make this file by hand?)
>
> I have done both moveconfig.failed as well as a hand-written file for
> testing certain boards. That's why I added both the append
> '_defconfig' as well as the check for it actually existing.
>
>> Then, we can drop enumarate and write the error message within 80 columns.
>>
>> if options.defconfigs:
>> defconfigs = [line.strip() for line in open(options.defconfigs)]
>> for defconfig in defconfigs:
>> if not os.path.exists(os.path.join('configs', defconfig)):
>> sys.exit('%s: defconfig does not exist. Stopping.' % defconfig)
>
> I think we should keep the check for endswith('_defconfig').
OK, then. Let's keep it.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 04/10] moveconfig: Add a switch to only cleanup headers
2015-05-15 21:40 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
2015-05-15 21:40 ` [U-Boot] [PATCH v4 02/10] moveconfig: Ignore duplicate configs when moving Joe Hershberger
2015-05-15 21:40 ` [U-Boot] [PATCH v4 03/10] moveconfig: Add a parameter to accept a list to build Joe Hershberger
@ 2015-05-15 21:40 ` Joe Hershberger
2015-05-19 2:03 ` Masahiro Yamada
2015-05-15 21:40 ` [U-Boot] [PATCH v4 05/10] moveconfig: Cleanup headers in arch and board Joe Hershberger
` (7 subsequent siblings)
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-15 21:40 UTC (permalink / raw)
To: u-boot
In some case you may want to only cleanup the headers. Make it possible
without waiting for all boards to compile.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v4: None
Changes in v3:
-New for version 3
Changes in v2: None
tools/moveconfig.py | 13 ++++++++++---
1 file changed, 10 insertions(+), 3 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 3f31e81..b6db058 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -146,6 +146,9 @@ Available options
Exit immediately if Make exits with a non-zero status while processing
a defconfig file.
+ -H, --headers-only
+ Only cleanup the headers; skip the defconfig processing
+
-j, --jobs
Specify the number of threads to run simultaneously. If not specified,
the number of threads is the same as the number of CPU cores.
@@ -770,8 +773,6 @@ def move_config(config_attrs, options):
slots.show_failed_boards()
- cleanup_headers(config_attrs, options.dry_run)
-
def bad_recipe(filename, linenum, msg):
"""Print error message with the file name and the line number and exit."""
sys.exit("%s: line %d: error : " % (filename, linenum) + msg)
@@ -859,6 +860,9 @@ def main():
parser.add_option('-e', '--exit-on-error', action='store_true',
default=False,
help='exit immediately on any error')
+ parser.add_option('-H', '--headers-only', dest='cleanup_headers_only',
+ action='store_true', default=False,
+ help='only cleanup the headers')
parser.add_option('-j', '--jobs', type='int', default=cpu_count,
help='the number of jobs to run simultaneously')
parser.usage += ' recipe_file\n\n' + \
@@ -879,7 +883,10 @@ def main():
update_cross_compile()
- move_config(config_attrs, options)
+ if not options.cleanup_headers_only:
+ move_config(config_attrs, options)
+
+ cleanup_headers(config_attrs, options.dry_run)
if __name__ == '__main__':
main()
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 04/10] moveconfig: Add a switch to only cleanup headers
2015-05-15 21:40 ` [U-Boot] [PATCH v4 04/10] moveconfig: Add a switch to only cleanup headers Joe Hershberger
@ 2015-05-19 2:03 ` Masahiro Yamada
2015-05-19 15:36 ` Joe Hershberger
0 siblings, 1 reply; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-19 2:03 UTC (permalink / raw)
To: u-boot
Hi Joe,
2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> In some case you may want to only cleanup the headers. Make it possible
> without waiting for all boards to compile.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>
> ---
>
> Changes in v4: None
> Changes in v3:
> -New for version 3
>
> Changes in v2: None
>
> tools/moveconfig.py | 13 ++++++++++---
> 1 file changed, 10 insertions(+), 3 deletions(-)
>
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index 3f31e81..b6db058 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -146,6 +146,9 @@ Available options
> Exit immediately if Make exits with a non-zero status while processing
> a defconfig file.
>
> + -H, --headers-only
> + Only cleanup the headers; skip the defconfig processing
> +
> -j, --jobs
> Specify the number of threads to run simultaneously. If not specified,
> the number of threads is the same as the number of CPU cores.
> @@ -770,8 +773,6 @@ def move_config(config_attrs, options):
>
> slots.show_failed_boards()
>
> - cleanup_headers(config_attrs, options.dry_run)
> -
> def bad_recipe(filename, linenum, msg):
> """Print error message with the file name and the line number and exit."""
> sys.exit("%s: line %d: error : " % (filename, linenum) + msg)
> @@ -859,6 +860,9 @@ def main():
> parser.add_option('-e', '--exit-on-error', action='store_true',
> default=False,
> help='exit immediately on any error')
> + parser.add_option('-H', '--headers-only', dest='cleanup_headers_only',
> + action='store_true', default=False,
> + help='only cleanup the headers')
> parser.add_option('-j', '--jobs', type='int', default=cpu_count,
> help='the number of jobs to run simultaneously')
> parser.usage += ' recipe_file\n\n' + \
> @@ -879,7 +883,10 @@ def main():
>
> update_cross_compile()
>
> - move_config(config_attrs, options)
> + if not options.cleanup_headers_only:
> + move_config(config_attrs, options)
> +
> + cleanup_headers(config_attrs, options.dry_run)
>
> if __name__ == '__main__':
> main()
Could you also move the check_top_directory() call
to main() function, above the move_config call.
We should make sure we are at the top directory
also for cleaning headers.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 04/10] moveconfig: Add a switch to only cleanup headers
2015-05-19 2:03 ` Masahiro Yamada
@ 2015-05-19 15:36 ` Joe Hershberger
0 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 15:36 UTC (permalink / raw)
To: u-boot
Hi Masahiro-san,
On Mon, May 18, 2015 at 9:03 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> Hi Joe,
>
> 2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>> In some case you may want to only cleanup the headers. Make it possible
>> without waiting for all boards to compile.
>>
>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>>
>> ---
>>
>> Changes in v4: None
>> Changes in v3:
>> -New for version 3
>>
>> Changes in v2: None
>>
>> tools/moveconfig.py | 13 ++++++++++---
>> 1 file changed, 10 insertions(+), 3 deletions(-)
>>
>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>> index 3f31e81..b6db058 100755
>> --- a/tools/moveconfig.py
>> +++ b/tools/moveconfig.py
>> @@ -146,6 +146,9 @@ Available options
>> Exit immediately if Make exits with a non-zero status while processing
>> a defconfig file.
>>
>> + -H, --headers-only
>> + Only cleanup the headers; skip the defconfig processing
>> +
>> -j, --jobs
>> Specify the number of threads to run simultaneously. If not specified,
>> the number of threads is the same as the number of CPU cores.
>> @@ -770,8 +773,6 @@ def move_config(config_attrs, options):
>>
>> slots.show_failed_boards()
>>
>> - cleanup_headers(config_attrs, options.dry_run)
>> -
>> def bad_recipe(filename, linenum, msg):
>> """Print error message with the file name and the line number and exit."""
>> sys.exit("%s: line %d: error : " % (filename, linenum) + msg)
>> @@ -859,6 +860,9 @@ def main():
>> parser.add_option('-e', '--exit-on-error', action='store_true',
>> default=False,
>> help='exit immediately on any error')
>> + parser.add_option('-H', '--headers-only', dest='cleanup_headers_only',
>> + action='store_true', default=False,
>> + help='only cleanup the headers')
>> parser.add_option('-j', '--jobs', type='int', default=cpu_count,
>> help='the number of jobs to run simultaneously')
>> parser.usage += ' recipe_file\n\n' + \
>> @@ -879,7 +883,10 @@ def main():
>>
>> update_cross_compile()
>>
>> - move_config(config_attrs, options)
>> + if not options.cleanup_headers_only:
>> + move_config(config_attrs, options)
>> +
>> + cleanup_headers(config_attrs, options.dry_run)
>>
>> if __name__ == '__main__':
>> main()
>
>
> Could you also move the check_top_directory() call
> to main() function, above the move_config call.
>
> We should make sure we are at the top directory
> also for cleaning headers.
Yes... moving it.
-Joe
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 05/10] moveconfig: Cleanup headers in arch and board
2015-05-15 21:40 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
` (2 preceding siblings ...)
2015-05-15 21:40 ` [U-Boot] [PATCH v4 04/10] moveconfig: Add a switch to only cleanup headers Joe Hershberger
@ 2015-05-15 21:40 ` Joe Hershberger
2015-05-19 1:41 ` Masahiro Yamada
2015-05-15 21:40 ` [U-Boot] [PATCH v4 06/10] moveconfig: Remove probable debug print Joe Hershberger
` (6 subsequent siblings)
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-15 21:40 UTC (permalink / raw)
To: u-boot
Some config.h files live in arch and board directories. They will need
to be cleaned up as well, so run the same filters there.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v4:
-New for version 4
Changes in v3: None
Changes in v2: None
tools/moveconfig.py | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index b6db058..25cee21 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -352,6 +352,16 @@ def cleanup_headers(config_attrs, dry_run):
if not fnmatch.fnmatch(filename, '*~'):
cleanup_one_header(os.path.join(dirpath, filename), patterns,
dry_run)
+ for (dirpath, dirnames, filenames) in os.walk('arch'):
+ for filename in filenames:
+ if not fnmatch.fnmatch(filename, '*~'):
+ cleanup_one_header(os.path.join(dirpath, filename), patterns,
+ dry_run)
+ for (dirpath, dirnames, filenames) in os.walk('board'):
+ for filename in filenames:
+ if not fnmatch.fnmatch(filename, '*~'):
+ cleanup_one_header(os.path.join(dirpath, filename), patterns,
+ dry_run)
### classes ###
class KconfigParser:
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 05/10] moveconfig: Cleanup headers in arch and board
2015-05-15 21:40 ` [U-Boot] [PATCH v4 05/10] moveconfig: Cleanup headers in arch and board Joe Hershberger
@ 2015-05-19 1:41 ` Masahiro Yamada
2015-05-19 14:33 ` Joe Hershberger
0 siblings, 1 reply; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-19 1:41 UTC (permalink / raw)
To: u-boot
Hi Joe,
2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> Some config.h files live in arch and board directories. They will need
> to be cleaned up as well, so run the same filters there.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>
> ---
>
> Changes in v4:
> -New for version 4
>
> Changes in v3: None
> Changes in v2: None
>
> tools/moveconfig.py | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index b6db058..25cee21 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -352,6 +352,16 @@ def cleanup_headers(config_attrs, dry_run):
> if not fnmatch.fnmatch(filename, '*~'):
> cleanup_one_header(os.path.join(dirpath, filename), patterns,
> dry_run)
> + for (dirpath, dirnames, filenames) in os.walk('arch'):
> + for filename in filenames:
> + if not fnmatch.fnmatch(filename, '*~'):
> + cleanup_one_header(os.path.join(dirpath, filename), patterns,
> + dry_run)
> + for (dirpath, dirnames, filenames) in os.walk('board'):
> + for filename in filenames:
> + if not fnmatch.fnmatch(filename, '*~'):
> + cleanup_one_header(os.path.join(dirpath, filename), patterns,
> + dry_run)
To reduce code duplication, can we write like this or something?
for dir in 'include', 'arch', 'board':
for (dirpath, dirnames, filenames) in os.walk(dir):
for filename in filenames:
if not fnmatch.fnmatch(filename, '*~'):
cleanup_one_header(os.path.join(dirpath, filename),
patterns, dry_run)
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 05/10] moveconfig: Cleanup headers in arch and board
2015-05-19 1:41 ` Masahiro Yamada
@ 2015-05-19 14:33 ` Joe Hershberger
0 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 14:33 UTC (permalink / raw)
To: u-boot
On Mon, May 18, 2015 at 8:41 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> Hi Joe,
>
>
> 2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>> Some config.h files live in arch and board directories. They will need
>> to be cleaned up as well, so run the same filters there.
>>
>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>>
>> ---
>>
>> Changes in v4:
>> -New for version 4
>>
>> Changes in v3: None
>> Changes in v2: None
>>
>> tools/moveconfig.py | 10 ++++++++++
>> 1 file changed, 10 insertions(+)
>>
>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>> index b6db058..25cee21 100755
>> --- a/tools/moveconfig.py
>> +++ b/tools/moveconfig.py
>> @@ -352,6 +352,16 @@ def cleanup_headers(config_attrs, dry_run):
>> if not fnmatch.fnmatch(filename, '*~'):
>> cleanup_one_header(os.path.join(dirpath, filename), patterns,
>> dry_run)
>> + for (dirpath, dirnames, filenames) in os.walk('arch'):
>> + for filename in filenames:
>> + if not fnmatch.fnmatch(filename, '*~'):
>> + cleanup_one_header(os.path.join(dirpath, filename), patterns,
>> + dry_run)
>> + for (dirpath, dirnames, filenames) in os.walk('board'):
>> + for filename in filenames:
>> + if not fnmatch.fnmatch(filename, '*~'):
>> + cleanup_one_header(os.path.join(dirpath, filename), patterns,
>> + dry_run)
>
>
> To reduce code duplication, can we write like this or something?
>
> for dir in 'include', 'arch', 'board':
> for (dirpath, dirnames, filenames) in os.walk(dir):
> for filename in filenames:
> if not fnmatch.fnmatch(filename, '*~'):
> cleanup_one_header(os.path.join(dirpath, filename),
> patterns, dry_run)
OK.
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 06/10] moveconfig: Remove probable debug print
2015-05-15 21:40 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
` (3 preceding siblings ...)
2015-05-15 21:40 ` [U-Boot] [PATCH v4 05/10] moveconfig: Cleanup headers in arch and board Joe Hershberger
@ 2015-05-15 21:40 ` Joe Hershberger
2015-05-19 2:10 ` Masahiro Yamada
2015-05-15 21:40 ` [U-Boot] [PATCH v4 07/10] moveconfig: Output a list of failed boards Joe Hershberger
` (5 subsequent siblings)
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-15 21:40 UTC (permalink / raw)
To: u-boot
This print seems to be redundant and unformatted compared to the next
few lines, so remove it.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/moveconfig.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 25cee21..15b0f2b 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -725,7 +725,6 @@ class Slots:
if len(failed_boards) > 0:
msg = [ "The following boards were not processed due to error:" ]
msg += failed_boards
- print msg
for line in msg:
print >> sys.stderr, color_text(self.options.color,
COLOR_LIGHT_RED, line)
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 06/10] moveconfig: Remove probable debug print
2015-05-15 21:40 ` [U-Boot] [PATCH v4 06/10] moveconfig: Remove probable debug print Joe Hershberger
@ 2015-05-19 2:10 ` Masahiro Yamada
2015-05-19 15:36 ` Joe Hershberger
0 siblings, 1 reply; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-19 2:10 UTC (permalink / raw)
To: u-boot
Hi Joe
2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> This print seems to be redundant and unformatted compared to the next
> few lines, so remove it.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
> ---
>
> Changes in v4: None
> Changes in v3: None
> Changes in v2: None
>
> tools/moveconfig.py | 1 -
> 1 file changed, 1 deletion(-)
>
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index 25cee21..15b0f2b 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -725,7 +725,6 @@ class Slots:
> if len(failed_boards) > 0:
> msg = [ "The following boards were not processed due to error:" ]
> msg += failed_boards
> - print msg
> for line in msg:
> print >> sys.stderr, color_text(self.options.color,
> COLOR_LIGHT_RED, line)
> --
Oops, my mistake.
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
(Or, because it is a simply bug fix, you can comment on my base patch.)
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 06/10] moveconfig: Remove probable debug print
2015-05-19 2:10 ` Masahiro Yamada
@ 2015-05-19 15:36 ` Joe Hershberger
0 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 15:36 UTC (permalink / raw)
To: u-boot
Hi Masahiro-san,
On Mon, May 18, 2015 at 9:10 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> Hi Joe
>
> 2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>> This print seems to be redundant and unformatted compared to the next
>> few lines, so remove it.
>>
>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>> ---
>>
>> Changes in v4: None
>> Changes in v3: None
>> Changes in v2: None
>>
>> tools/moveconfig.py | 1 -
>> 1 file changed, 1 deletion(-)
>>
>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>> index 25cee21..15b0f2b 100755
>> --- a/tools/moveconfig.py
>> +++ b/tools/moveconfig.py
>> @@ -725,7 +725,6 @@ class Slots:
>> if len(failed_boards) > 0:
>> msg = [ "The following boards were not processed due to error:" ]
>> msg += failed_boards
>> - print msg
>> for line in msg:
>> print >> sys.stderr, color_text(self.options.color,
>> COLOR_LIGHT_RED, line)
>> --
>
> Oops, my mistake.
>
> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
>
>
> (Or, because it is a simply bug fix, you can comment on my base patch.)
I'm dropping this patch and sent a reply to your original patch (not
applied to master yet).
-Joe
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 07/10] moveconfig: Output a list of failed boards
2015-05-15 21:40 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
` (4 preceding siblings ...)
2015-05-15 21:40 ` [U-Boot] [PATCH v4 06/10] moveconfig: Remove probable debug print Joe Hershberger
@ 2015-05-15 21:40 ` Joe Hershberger
2015-05-19 3:12 ` Masahiro Yamada
2015-05-15 21:40 ` [U-Boot] [PATCH v4 08/10] moveconfig: Print a message for missing compiler Joe Hershberger
` (4 subsequent siblings)
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-15 21:40 UTC (permalink / raw)
To: u-boot
If boards fail, output that list to a file so that it can easily be
passed back into moveconfig.py using the -d option.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/moveconfig.py | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 15b0f2b..9e923da 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -728,6 +728,10 @@ class Slots:
for line in msg:
print >> sys.stderr, color_text(self.options.color,
COLOR_LIGHT_RED, line)
+ ffail = open('moveconfig.failed', 'w')
+ for failed_board in failed_boards:
+ ffail.write("%s\n" % failed_board)
+ ffail.close()
def move_config(config_attrs, options):
"""Move config options to defconfig files.
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 07/10] moveconfig: Output a list of failed boards
2015-05-15 21:40 ` [U-Boot] [PATCH v4 07/10] moveconfig: Output a list of failed boards Joe Hershberger
@ 2015-05-19 3:12 ` Masahiro Yamada
2015-05-19 15:40 ` Joe Hershberger
0 siblings, 1 reply; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-19 3:12 UTC (permalink / raw)
To: u-boot
2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> If boards fail, output that list to a file so that it can easily be
> passed back into moveconfig.py using the -d option.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
> ---
>
> Changes in v4: None
> Changes in v3: None
> Changes in v2: None
>
> tools/moveconfig.py | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index 15b0f2b..9e923da 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -728,6 +728,10 @@ class Slots:
> for line in msg:
> print >> sys.stderr, color_text(self.options.color,
> COLOR_LIGHT_RED, line)
> + ffail = open('moveconfig.failed', 'w')
> + for failed_board in failed_boards:
> + ffail.write("%s\n" % failed_board)
> + ffail.close()
>
> def move_config(config_attrs, options):
> """Move config options to defconfig files.
If you use with ... as ..., it will automatically close
the file on exit.
with open('moveconfig.failed', 'w') as f:
for board in failed_boards:
f.write(board + '\n')
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 07/10] moveconfig: Output a list of failed boards
2015-05-19 3:12 ` Masahiro Yamada
@ 2015-05-19 15:40 ` Joe Hershberger
0 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 15:40 UTC (permalink / raw)
To: u-boot
Hi Masahiro-san,
On Mon, May 18, 2015 at 10:12 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>> If boards fail, output that list to a file so that it can easily be
>> passed back into moveconfig.py using the -d option.
>>
>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>> ---
>>
>> Changes in v4: None
>> Changes in v3: None
>> Changes in v2: None
>>
>> tools/moveconfig.py | 4 ++++
>> 1 file changed, 4 insertions(+)
>>
>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>> index 15b0f2b..9e923da 100755
>> --- a/tools/moveconfig.py
>> +++ b/tools/moveconfig.py
>> @@ -728,6 +728,10 @@ class Slots:
>> for line in msg:
>> print >> sys.stderr, color_text(self.options.color,
>> COLOR_LIGHT_RED, line)
>> + ffail = open('moveconfig.failed', 'w')
>> + for failed_board in failed_boards:
>> + ffail.write("%s\n" % failed_board)
>> + ffail.close()
>>
>> def move_config(config_attrs, options):
>> """Move config options to defconfig files.
>
>
> If you use with ... as ..., it will automatically close
> the file on exit.
>
> with open('moveconfig.failed', 'w') as f:
> for board in failed_boards:
> f.write(board + '\n')
OK.
-Joe
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 08/10] moveconfig: Print a message for missing compiler
2015-05-15 21:40 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
` (5 preceding siblings ...)
2015-05-15 21:40 ` [U-Boot] [PATCH v4 07/10] moveconfig: Output a list of failed boards Joe Hershberger
@ 2015-05-15 21:40 ` Joe Hershberger
2015-05-19 3:23 ` Masahiro Yamada
2015-05-15 21:40 ` [U-Boot] [PATCH v4 09/10] moveconfig: Add a switch to enable printing errors Joe Hershberger
` (3 subsequent siblings)
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-15 21:40 UTC (permalink / raw)
To: u-boot
A common case for failed builds is a missing compiler. Print a message
for that case to tell the user concisely which compiler was expected
that was not found.
This patch also has the effect of not printing build errors any longer.
The next patch will add a switch to optionally bring that back.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/moveconfig.py | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 9e923da..f986f55 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -166,6 +166,7 @@ import os
import re
import shutil
import subprocess
+from subprocess import PIPE
import sys
import tempfile
import time
@@ -606,11 +607,14 @@ class Slot:
return False
if self.ps.poll() != 0:
-
+ errmsg = 'Failed to process.'
+ errout = self.ps.stderr.read()
+ if errout.find('gcc: command not found') != -1:
+ errmsg = 'Compiler not found (%s)' % self.cross_compile
print >> sys.stderr, log_msg(self.options.color,
COLOR_LIGHT_RED,
self.defconfig,
- "failed to process.")
+ errmsg),
if self.options.exit_on_error:
sys.exit("Exit on error.")
else:
@@ -644,13 +648,13 @@ class Slot:
self.state = STATE_IDLE
return True
- cross_compile = self.parser.get_cross_compile()
+ self.cross_compile = self.parser.get_cross_compile()
cmd = list(self.make_cmd)
- if cross_compile:
- cmd.append('CROSS_COMPILE=%s' % cross_compile)
+ if self.cross_compile:
+ cmd.append('CROSS_COMPILE=%s' % self.cross_compile)
cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
cmd.append('include/config/auto.conf')
- self.ps = subprocess.Popen(cmd, stdout=self.devnull)
+ self.ps = subprocess.Popen(cmd, stdout=self.devnull, stderr=PIPE)
self.state = STATE_AUTOCONF
return False
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 08/10] moveconfig: Print a message for missing compiler
2015-05-15 21:40 ` [U-Boot] [PATCH v4 08/10] moveconfig: Print a message for missing compiler Joe Hershberger
@ 2015-05-19 3:23 ` Masahiro Yamada
2015-05-19 17:51 ` Joe Hershberger
0 siblings, 1 reply; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-19 3:23 UTC (permalink / raw)
To: u-boot
2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> A common case for failed builds is a missing compiler. Print a message
> for that case to tell the user concisely which compiler was expected
> that was not found.
>
> This patch also has the effect of not printing build errors any longer.
> The next patch will add a switch to optionally bring that back.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
> ---
>
> Changes in v4: None
> Changes in v3: None
> Changes in v2: None
>
> tools/moveconfig.py | 16 ++++++++++------
> 1 file changed, 10 insertions(+), 6 deletions(-)
>
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index 9e923da..f986f55 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -166,6 +166,7 @@ import os
> import re
> import shutil
> import subprocess
> +from subprocess import PIPE
Personally I do not prefer from ... import
because it disturbs the naming space.
Could you use subprocess.PIPE instead?
> import sys
> import tempfile
> import time
> @@ -606,11 +607,14 @@ class Slot:
> return False
>
> if self.ps.poll() != 0:
> -
> + errmsg = 'Failed to process.'
> + errout = self.ps.stderr.read()
This throws exception if "make *_defconfig" or "make savedefconfig" fail.
Traceback (most recent call last):
File "tools/moveconfig.py", line 924, in <module>
main()
File "tools/moveconfig.py", line 919, in main
move_config(config_attrs, options)
File "tools/moveconfig.py", line 794, in move_config
while not slots.available():
File "tools/moveconfig.py", line 717, in available
if slot.poll():
File "tools/moveconfig.py", line 616, in poll
errout = self.ps.stderr.read()
AttributeError: 'NoneType' object has no attribute 'read'
Seems better to add PIPE for all the call of subprocess.Popen()
> + if errout.find('gcc: command not found') != -1:
> + errmsg = 'Compiler not found (%s)' % self.cross_compile
If you do this, should the locale be changed?
Without LANG=C, "command not found" is displayed in Japanese on my computer.
If --verbose is given, we will be able to know the cause of erorr.
"missing compiler" is a special case error?
> print >> sys.stderr, log_msg(self.options.color,
> COLOR_LIGHT_RED,
> self.defconfig,
> - "failed to process.")
> + errmsg),
> if self.options.exit_on_error:
> sys.exit("Exit on error.")
> else:
> @@ -644,13 +648,13 @@ class Slot:
> self.state = STATE_IDLE
> return True
>
> - cross_compile = self.parser.get_cross_compile()
> + self.cross_compile = self.parser.get_cross_compile()
> cmd = list(self.make_cmd)
> - if cross_compile:
> - cmd.append('CROSS_COMPILE=%s' % cross_compile)
> + if self.cross_compile:
> + cmd.append('CROSS_COMPILE=%s' % self.cross_compile)
> cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
> cmd.append('include/config/auto.conf')
> - self.ps = subprocess.Popen(cmd, stdout=self.devnull)
> + self.ps = subprocess.Popen(cmd, stdout=self.devnull, stderr=PIPE)
> self.state = STATE_AUTOCONF
> return False
>
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 08/10] moveconfig: Print a message for missing compiler
2015-05-19 3:23 ` Masahiro Yamada
@ 2015-05-19 17:51 ` Joe Hershberger
2015-05-20 3:04 ` Masahiro Yamada
0 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 17:51 UTC (permalink / raw)
To: u-boot
Hi Masahiro-san,
On Mon, May 18, 2015 at 10:23 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>> A common case for failed builds is a missing compiler. Print a message
>> for that case to tell the user concisely which compiler was expected
>> that was not found.
>>
>> This patch also has the effect of not printing build errors any longer.
>> The next patch will add a switch to optionally bring that back.
>>
>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>> ---
>>
>> Changes in v4: None
>> Changes in v3: None
>> Changes in v2: None
>>
>> tools/moveconfig.py | 16 ++++++++++------
>> 1 file changed, 10 insertions(+), 6 deletions(-)
>>
>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>> index 9e923da..f986f55 100755
>> --- a/tools/moveconfig.py
>> +++ b/tools/moveconfig.py
>> @@ -166,6 +166,7 @@ import os
>> import re
>> import shutil
>> import subprocess
>> +from subprocess import PIPE
>
> Personally I do not prefer from ... import
> because it disturbs the naming space.
>
> Could you use subprocess.PIPE instead?
OK.
>> import sys
>> import tempfile
>> import time
>> @@ -606,11 +607,14 @@ class Slot:
>> return False
>>
>> if self.ps.poll() != 0:
>> -
>> + errmsg = 'Failed to process.'
>> + errout = self.ps.stderr.read()
>
> This throws exception if "make *_defconfig" or "make savedefconfig" fail.
>
>
>
> Traceback (most recent call last):
> File "tools/moveconfig.py", line 924, in <module>
> main()
> File "tools/moveconfig.py", line 919, in main
> move_config(config_attrs, options)
> File "tools/moveconfig.py", line 794, in move_config
> while not slots.available():
> File "tools/moveconfig.py", line 717, in available
> if slot.poll():
> File "tools/moveconfig.py", line 616, in poll
> errout = self.ps.stderr.read()
> AttributeError: 'NoneType' object has no attribute 'read'
>
>
>
> Seems better to add PIPE for all the call of subprocess.Popen()
OK
>> + if errout.find('gcc: command not found') != -1:
>> + errmsg = 'Compiler not found (%s)' % self.cross_compile
>
> If you do this, should the locale be changed?
>
> Without LANG=C, "command not found" is displayed in Japanese on my computer.
>
>
> If --verbose is given, we will be able to know the cause of erorr.
> "missing compiler" is a special case error?
That's true, but at least for my use-case before I spent several days
getting all tool-chains working, it was nice to know what the build
was trying to use for a cross tool-chain in a concise one-liner
instead of parsing a bunch of error prints. That's part of why I added
the -v flag. It's also helpful (naturally) in getting the compilers
all working.
>> print >> sys.stderr, log_msg(self.options.color,
>> COLOR_LIGHT_RED,
>> self.defconfig,
>> - "failed to process.")
>> + errmsg),
>> if self.options.exit_on_error:
>> sys.exit("Exit on error.")
>> else:
>> @@ -644,13 +648,13 @@ class Slot:
>> self.state = STATE_IDLE
>> return True
>>
>> - cross_compile = self.parser.get_cross_compile()
>> + self.cross_compile = self.parser.get_cross_compile()
>> cmd = list(self.make_cmd)
>> - if cross_compile:
>> - cmd.append('CROSS_COMPILE=%s' % cross_compile)
>> + if self.cross_compile:
>> + cmd.append('CROSS_COMPILE=%s' % self.cross_compile)
>> cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
>> cmd.append('include/config/auto.conf')
>> - self.ps = subprocess.Popen(cmd, stdout=self.devnull)
>> + self.ps = subprocess.Popen(cmd, stdout=self.devnull, stderr=PIPE)
>> self.state = STATE_AUTOCONF
>> return False
>>
>
> --
> Best Regards
> Masahiro Yamada
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 08/10] moveconfig: Print a message for missing compiler
2015-05-19 17:51 ` Joe Hershberger
@ 2015-05-20 3:04 ` Masahiro Yamada
0 siblings, 0 replies; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-20 3:04 UTC (permalink / raw)
To: u-boot
2015-05-20 2:51 GMT+09:00 Joe Hershberger <joe.hershberger@gmail.com>:
> Hi Masahiro-san,
>
> On Mon, May 18, 2015 at 10:23 PM, Masahiro Yamada
> <yamada.masahiro@socionext.com> wrote:
>> 2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>>> A common case for failed builds is a missing compiler. Print a message
>>> for that case to tell the user concisely which compiler was expected
>>> that was not found.
>>>
>>> This patch also has the effect of not printing build errors any longer.
>>> The next patch will add a switch to optionally bring that back.
>>>
>>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>>> ---
>>>
>>> Changes in v4: None
>>> Changes in v3: None
>>> Changes in v2: None
>>>
>>> tools/moveconfig.py | 16 ++++++++++------
>>> 1 file changed, 10 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>>> index 9e923da..f986f55 100755
>>> --- a/tools/moveconfig.py
>>> +++ b/tools/moveconfig.py
>>> @@ -166,6 +166,7 @@ import os
>>> import re
>>> import shutil
>>> import subprocess
>>> +from subprocess import PIPE
>>
>> Personally I do not prefer from ... import
>> because it disturbs the naming space.
>>
>> Could you use subprocess.PIPE instead?
>
> OK.
>
>>> import sys
>>> import tempfile
>>> import time
>>> @@ -606,11 +607,14 @@ class Slot:
>>> return False
>>>
>>> if self.ps.poll() != 0:
>>> -
>>> + errmsg = 'Failed to process.'
>>> + errout = self.ps.stderr.read()
>>
>> This throws exception if "make *_defconfig" or "make savedefconfig" fail.
>>
>>
>>
>> Traceback (most recent call last):
>> File "tools/moveconfig.py", line 924, in <module>
>> main()
>> File "tools/moveconfig.py", line 919, in main
>> move_config(config_attrs, options)
>> File "tools/moveconfig.py", line 794, in move_config
>> while not slots.available():
>> File "tools/moveconfig.py", line 717, in available
>> if slot.poll():
>> File "tools/moveconfig.py", line 616, in poll
>> errout = self.ps.stderr.read()
>> AttributeError: 'NoneType' object has no attribute 'read'
>>
>>
>>
>> Seems better to add PIPE for all the call of subprocess.Popen()
>
> OK
>
>>> + if errout.find('gcc: command not found') != -1:
>>> + errmsg = 'Compiler not found (%s)' % self.cross_compile
>>
>> If you do this, should the locale be changed?
>>
>> Without LANG=C, "command not found" is displayed in Japanese on my computer.
>>
>>
>> If --verbose is given, we will be able to know the cause of erorr.
>> "missing compiler" is a special case error?
>
> That's true, but at least for my use-case before I spent several days
> getting all tool-chains working, it was nice to know what the build
> was trying to use for a cross tool-chain in a concise one-liner
> instead of parsing a bunch of error prints. That's part of why I added
> the -v flag. It's also helpful (naturally) in getting the compilers
> all working.
OK.
You pass LANG=C in v5, so it works fine for me.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 09/10] moveconfig: Add a switch to enable printing errors
2015-05-15 21:40 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
` (6 preceding siblings ...)
2015-05-15 21:40 ` [U-Boot] [PATCH v4 08/10] moveconfig: Print a message for missing compiler Joe Hershberger
@ 2015-05-15 21:40 ` Joe Hershberger
2015-05-19 3:25 ` Masahiro Yamada
2015-05-15 21:40 ` [U-Boot] [PATCH v4 10/10] moveconfig: Print status about the processed defconfigs Joe Hershberger
` (2 subsequent siblings)
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-15 21:40 UTC (permalink / raw)
To: u-boot
In some cases the build for the autoconf breaks. This outputs the errors
following the status so that action can be taken without building again
manually.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v4: None
Changes in v3: None
Changes in v2:
-New for version 2
tools/moveconfig.py | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index f986f55..685b47b 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -153,6 +153,9 @@ Available options
Specify the number of threads to run simultaneously. If not specified,
the number of threads is the same as the number of CPU cores.
+ -v, --verbose
+ Show any build errors as boards are built
+
To see the complete list of supported options, run
$ tools/moveconfig.py -h
@@ -615,6 +618,9 @@ class Slot:
COLOR_LIGHT_RED,
self.defconfig,
errmsg),
+ if self.options.verbose:
+ print >> sys.stderr, color_text(self.options.color,
+ COLOR_LIGHT_CYAN, errout)
if self.options.exit_on_error:
sys.exit("Exit on error.")
else:
@@ -882,6 +888,9 @@ def main():
help='only cleanup the headers')
parser.add_option('-j', '--jobs', type='int', default=cpu_count,
help='the number of jobs to run simultaneously')
+ parser.add_option('-v', '--verbose', dest='verbose',
+ action='store_true', default=False,
+ help='show any build errors as boards are built')
parser.usage += ' recipe_file\n\n' + \
'The recipe_file should describe config options you want to move.\n' + \
'Each line should contain config_name, type, default_value\n\n' + \
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 09/10] moveconfig: Add a switch to enable printing errors
2015-05-15 21:40 ` [U-Boot] [PATCH v4 09/10] moveconfig: Add a switch to enable printing errors Joe Hershberger
@ 2015-05-19 3:25 ` Masahiro Yamada
2015-05-19 17:52 ` Joe Hershberger
0 siblings, 1 reply; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-19 3:25 UTC (permalink / raw)
To: u-boot
2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> In some cases the build for the autoconf breaks. This outputs the errors
> following the status so that action can be taken without building again
> manually.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>
> ---
>
> Changes in v4: None
> Changes in v3: None
> Changes in v2:
> -New for version 2
>
> tools/moveconfig.py | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index f986f55..685b47b 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -153,6 +153,9 @@ Available options
> Specify the number of threads to run simultaneously. If not specified,
> the number of threads is the same as the number of CPU cores.
>
> + -v, --verbose
> + Show any build errors as boards are built
> +
> To see the complete list of supported options, run
>
> $ tools/moveconfig.py -h
> @@ -615,6 +618,9 @@ class Slot:
> COLOR_LIGHT_RED,
> self.defconfig,
> errmsg),
> + if self.options.verbose:
> + print >> sys.stderr, color_text(self.options.color,
> + COLOR_LIGHT_CYAN, errout)
> if self.options.exit_on_error:
> sys.exit("Exit on error.")
> else:
> @@ -882,6 +888,9 @@ def main():
> help='only cleanup the headers')
> parser.add_option('-j', '--jobs', type='int', default=cpu_count,
> help='the number of jobs to run simultaneously')
> + parser.add_option('-v', '--verbose', dest='verbose',
> + action='store_true', default=False,
> + help='show any build errors as boards are built')
dest='verbose' is redundant.
The destination is chosen based on the long option name.
Apart from that, I think this is a nice improvement.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 09/10] moveconfig: Add a switch to enable printing errors
2015-05-19 3:25 ` Masahiro Yamada
@ 2015-05-19 17:52 ` Joe Hershberger
0 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 17:52 UTC (permalink / raw)
To: u-boot
Hi Masahiro-san,
On Mon, May 18, 2015 at 10:25 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> 2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>> In some cases the build for the autoconf breaks. This outputs the errors
>> following the status so that action can be taken without building again
>> manually.
>>
>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>>
>> ---
>>
>> Changes in v4: None
>> Changes in v3: None
>> Changes in v2:
>> -New for version 2
>>
>> tools/moveconfig.py | 9 +++++++++
>> 1 file changed, 9 insertions(+)
>>
>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>> index f986f55..685b47b 100755
>> --- a/tools/moveconfig.py
>> +++ b/tools/moveconfig.py
>> @@ -153,6 +153,9 @@ Available options
>> Specify the number of threads to run simultaneously. If not specified,
>> the number of threads is the same as the number of CPU cores.
>>
>> + -v, --verbose
>> + Show any build errors as boards are built
>> +
>> To see the complete list of supported options, run
>>
>> $ tools/moveconfig.py -h
>> @@ -615,6 +618,9 @@ class Slot:
>> COLOR_LIGHT_RED,
>> self.defconfig,
>> errmsg),
>> + if self.options.verbose:
>> + print >> sys.stderr, color_text(self.options.color,
>> + COLOR_LIGHT_CYAN, errout)
>> if self.options.exit_on_error:
>> sys.exit("Exit on error.")
>> else:
>> @@ -882,6 +888,9 @@ def main():
>> help='only cleanup the headers')
>> parser.add_option('-j', '--jobs', type='int', default=cpu_count,
>> help='the number of jobs to run simultaneously')
>> + parser.add_option('-v', '--verbose', dest='verbose',
>> + action='store_true', default=False,
>> + help='show any build errors as boards are built')
>
> dest='verbose' is redundant.
>
> The destination is chosen based on the long option name.
OK. Dropped.
> Apart from that, I think this is a nice improvement.
>
>
> --
> Best Regards
> Masahiro Yamada
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 10/10] moveconfig: Print status about the processed defconfigs
2015-05-15 21:40 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
` (7 preceding siblings ...)
2015-05-15 21:40 ` [U-Boot] [PATCH v4 09/10] moveconfig: Add a switch to enable printing errors Joe Hershberger
@ 2015-05-15 21:40 ` Joe Hershberger
2015-05-19 4:05 ` Masahiro Yamada
2015-05-19 1:58 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Masahiro Yamada
2015-05-19 18:21 ` [U-Boot] [PATCH v5 1/9] " Joe Hershberger
10 siblings, 1 reply; 82+ messages in thread
From: Joe Hershberger @ 2015-05-15 21:40 UTC (permalink / raw)
To: u-boot
This gives a basic idea about progress.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v4: None
Changes in v3:
-New for version 3
Changes in v2: None
tools/moveconfig.py | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 685b47b..fca0197 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -568,7 +568,7 @@ class Slot:
pass
shutil.rmtree(self.build_dir)
- def add(self, defconfig):
+ def add(self, defconfig, num, total):
"""Assign a new subprocess for defconfig and add it to the slot.
If the slot is vacant, create a new subprocess for processing the
@@ -588,6 +588,8 @@ class Slot:
self.ps = subprocess.Popen(cmd, stdout=self.devnull)
self.defconfig = defconfig
self.state = STATE_DEFCONFIG
+ self.num = num
+ self.total = total
return True
def poll(self):
@@ -634,6 +636,9 @@ class Slot:
if self.state == STATE_AUTOCONF:
self.parser.update_defconfig(self.defconfig)
+ print ' %d defconfigs out of %d\r' % (self.num + 1, self.total),
+ sys.stdout.flush()
+
"""Save off the defconfig in a consistent way"""
cmd = list(self.make_cmd)
cmd.append('savedefconfig')
@@ -688,7 +693,7 @@ class Slots:
for i in range(options.jobs):
self.slots.append(Slot(config_attrs, options, devnull, make_cmd))
- def add(self, defconfig):
+ def add(self, defconfig, num, total):
"""Add a new subprocess if a vacant slot is found.
Arguments:
@@ -698,7 +703,7 @@ class Slots:
Return True on success or False on failure
"""
for slot in self.slots:
- if slot.add(defconfig):
+ if slot.add(defconfig, num, total):
return True
return False
@@ -784,8 +789,8 @@ def move_config(config_attrs, options):
# Main loop to process defconfig files:
# Add a new subprocess into a vacant slot.
# Sleep if there is no available slot.
- for defconfig in defconfigs:
- while not slots.add(defconfig):
+ for i, defconfig in enumerate(defconfigs):
+ while not slots.add(defconfig, i, len(defconfigs)):
while not slots.available():
# No available slot: sleep for a while
time.sleep(SLEEP_TIME)
@@ -794,6 +799,7 @@ def move_config(config_attrs, options):
while not slots.empty():
time.sleep(SLEEP_TIME)
+ print ''
slots.show_failed_boards()
def bad_recipe(filename, linenum, msg):
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config
2015-05-15 21:40 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
` (8 preceding siblings ...)
2015-05-15 21:40 ` [U-Boot] [PATCH v4 10/10] moveconfig: Print status about the processed defconfigs Joe Hershberger
@ 2015-05-19 1:58 ` Masahiro Yamada
2015-05-19 15:35 ` Joe Hershberger
2015-05-19 18:07 ` Joe Hershberger
2015-05-19 18:21 ` [U-Boot] [PATCH v5 1/9] " Joe Hershberger
10 siblings, 2 replies; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-19 1:58 UTC (permalink / raw)
To: u-boot
Hi Joe,
2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> This will ensure that the order of the defconfig entries will always
> match that of the Kconfig files. After one slightly painful (but
> still early in the process) pass over all boards, this should keep
> the defconfigs clean from here on.
>
> Users must edit the Kconfig first to add the menu entries and then run
> moveconfig.py to update the defconfig files and the include configs.
>
> As such, moveconfig.py cannot compare against the '.config' contents.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
This feature expects the defconfigs are all clean.
Otherwise, savedefconfig would make "git diff" noisier.
It is safer to use "make menuconfig && make savedefconfig"
for adding new options, but some people still try to
edit the defconfig directly...
Perhaps, should do the global cleanup periodically?
> ---
> This is based on https://patchwork.ozlabs.org/patch/472591/
>
> Changes in v4:
> -Rebased series on Masahiro's v2
>
> Changes in v3: None
> Changes in v2: None
>
> tools/moveconfig.py | 35 ++++++++++++++++++++++++++---------
> 1 file changed, 26 insertions(+), 9 deletions(-)
>
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index c39ea95..544f6af 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -46,6 +46,9 @@ should look like this:
> CONFIG_CMD_USB bool n
> CONFIG_SYS_TEXT_BASE hex 0x00000000
>
> +Next you must edit the Kconfig to add the menu entries for the configs
> +you are moving.
> +
> And then run this tool giving the file name of the recipe
Uh, I was doing in a different work-flow.
(I edit the Kconfig after I move CONFIGs to defconfigs).
But, the Kconfig must be edited beforehand
to do savedefconfig.
So, I am OK with this change.
> $ tools/moveconfig.py recipe
> @@ -192,6 +195,7 @@ CROSS_COMPILE = {
> STATE_IDLE = 0
> STATE_DEFCONFIG = 1
> STATE_AUTOCONF = 2
> +STATE_SAVEDEFCONFIG = 3
>
> ACTION_MOVE = 0
> ACTION_DEFAULT_VALUE = 1
> @@ -390,8 +394,7 @@ class KconfigParser:
>
> return CROSS_COMPILE.get(arch, '')
>
> - def parse_one_config(self, config_attr, defconfig_lines,
> - dotconfig_lines, autoconf_lines):
> + def parse_one_config(self, config_attr, defconfig_lines, autoconf_lines):
> """Parse .config, defconfig, include/autoconf.mk for one config.
>
> This function looks for the config options in the lines from
> @@ -402,7 +405,6 @@ class KconfigParser:
> config_attr: A dictionary including the name, the type,
> and the default value of the target config.
> defconfig_lines: lines from the original defconfig file.
> - dotconfig_lines: lines from the .config file.
> autoconf_lines: lines from the include/autoconf.mk file.
>
> Returns:
> @@ -418,7 +420,7 @@ class KconfigParser:
> else:
> default = config + '=' + config_attr['default']
>
> - for line in defconfig_lines + dotconfig_lines:
> + for line in defconfig_lines:
> line = line.rstrip()
> if line.startswith(config + '=') or line == not_set:
> return (ACTION_ALREADY_EXIST, line)
> @@ -463,15 +465,12 @@ class KconfigParser:
> with open(defconfig_path) as f:
> defconfig_lines = f.readlines()
>
> - with open(dotconfig_path) as f:
> - dotconfig_lines = f.readlines()
> -
> with open(autoconf_path) as f:
> autoconf_lines = f.readlines()
>
> for config_attr in self.config_attrs:
> result = self.parse_one_config(config_attr, defconfig_lines,
> - dotconfig_lines, autoconf_lines)
> + autoconf_lines)
> results.append(result)
>
> log = ''
With the change of the work-flow above,
we need not parse the .config, so this seems OK.
> @@ -499,7 +498,7 @@ class KconfigParser:
> print log,
>
> if not self.options.dry_run:
> - with open(defconfig_path, 'a') as f:
> + with open(dotconfig_path, 'a') as f:
> for (action, value) in results:
> if action == ACTION_MOVE:
> f.write(value + '\n')
> @@ -608,6 +607,24 @@ class Slot:
>
> if self.state == STATE_AUTOCONF:
> self.parser.update_defconfig(self.defconfig)
> +
> + """Save off the defconfig in a consistent way"""
> + cmd = list(self.make_cmd)
> + cmd.append('savedefconfig')
> + self.ps = subprocess.Popen(cmd, stdout=self.devnull,
> + stderr=self.devnull)
> + self.state = STATE_SAVEDEFCONFIG
> + return False
> +
> + if self.state == STATE_SAVEDEFCONFIG:
> + defconfig_path = os.path.join(self.build_dir, 'defconfig')
> + if not os.path.exists(defconfig_path):
> + print >> sys.stderr, log_msg(self.options.color,
> + COLOR_LIGHT_RED,
> + self.defconfig,
> + 'The defconfig was not updated')
Is this warning message reachable?
The missing defconfig means "make savedefconfig" failed.
That case has been already caught by 'Failed to process.' above, I think.
> + shutil.move(defconfig_path,
> + os.path.join('configs', self.defconfig))
> self.state = STATE_IDLE
> return True
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config
2015-05-19 1:58 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Masahiro Yamada
@ 2015-05-19 15:35 ` Joe Hershberger
2015-05-19 18:07 ` Joe Hershberger
1 sibling, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 15:35 UTC (permalink / raw)
To: u-boot
Hi Masahiro-san,
On Mon, May 18, 2015 at 8:58 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> Hi Joe,
>
> 2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>> This will ensure that the order of the defconfig entries will always
>> match that of the Kconfig files. After one slightly painful (but
>> still early in the process) pass over all boards, this should keep
>> the defconfigs clean from here on.
>>
>> Users must edit the Kconfig first to add the menu entries and then run
>> moveconfig.py to update the defconfig files and the include configs.
>>
>> As such, moveconfig.py cannot compare against the '.config' contents.
>>
>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>
>
> This feature expects the defconfigs are all clean.
> Otherwise, savedefconfig would make "git diff" noisier.
>
> It is safer to use "make menuconfig && make savedefconfig"
> for adding new options, but some people still try to
> edit the defconfig directly...
>
> Perhaps, should do the global cleanup periodically?
>
>
>> ---
>> This is based on https://patchwork.ozlabs.org/patch/472591/
>>
>> Changes in v4:
>> -Rebased series on Masahiro's v2
>>
>> Changes in v3: None
>> Changes in v2: None
>>
>> tools/moveconfig.py | 35 ++++++++++++++++++++++++++---------
>> 1 file changed, 26 insertions(+), 9 deletions(-)
>>
>> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
>> index c39ea95..544f6af 100755
>> --- a/tools/moveconfig.py
>> +++ b/tools/moveconfig.py
>> @@ -46,6 +46,9 @@ should look like this:
>> CONFIG_CMD_USB bool n
>> CONFIG_SYS_TEXT_BASE hex 0x00000000
>>
>> +Next you must edit the Kconfig to add the menu entries for the configs
>> +you are moving.
>> +
>> And then run this tool giving the file name of the recipe
>
> Uh, I was doing in a different work-flow.
> (I edit the Kconfig after I move CONFIGs to defconfigs).
>
> But, the Kconfig must be edited beforehand
> to do savedefconfig.
>
> So, I am OK with this change.
>
>
>
>> $ tools/moveconfig.py recipe
>> @@ -192,6 +195,7 @@ CROSS_COMPILE = {
>> STATE_IDLE = 0
>> STATE_DEFCONFIG = 1
>> STATE_AUTOCONF = 2
>> +STATE_SAVEDEFCONFIG = 3
>>
>> ACTION_MOVE = 0
>> ACTION_DEFAULT_VALUE = 1
>> @@ -390,8 +394,7 @@ class KconfigParser:
>>
>> return CROSS_COMPILE.get(arch, '')
>>
>> - def parse_one_config(self, config_attr, defconfig_lines,
>> - dotconfig_lines, autoconf_lines):
>> + def parse_one_config(self, config_attr, defconfig_lines, autoconf_lines):
>> """Parse .config, defconfig, include/autoconf.mk for one config.
>>
>> This function looks for the config options in the lines from
>> @@ -402,7 +405,6 @@ class KconfigParser:
>> config_attr: A dictionary including the name, the type,
>> and the default value of the target config.
>> defconfig_lines: lines from the original defconfig file.
>> - dotconfig_lines: lines from the .config file.
>> autoconf_lines: lines from the include/autoconf.mk file.
>>
>> Returns:
>> @@ -418,7 +420,7 @@ class KconfigParser:
>> else:
>> default = config + '=' + config_attr['default']
>>
>> - for line in defconfig_lines + dotconfig_lines:
>> + for line in defconfig_lines:
>> line = line.rstrip()
>> if line.startswith(config + '=') or line == not_set:
>> return (ACTION_ALREADY_EXIST, line)
>> @@ -463,15 +465,12 @@ class KconfigParser:
>> with open(defconfig_path) as f:
>> defconfig_lines = f.readlines()
>>
>> - with open(dotconfig_path) as f:
>> - dotconfig_lines = f.readlines()
>> -
>> with open(autoconf_path) as f:
>> autoconf_lines = f.readlines()
>>
>> for config_attr in self.config_attrs:
>> result = self.parse_one_config(config_attr, defconfig_lines,
>> - dotconfig_lines, autoconf_lines)
>> + autoconf_lines)
>> results.append(result)
>>
>> log = ''
>
>
> With the change of the work-flow above,
> we need not parse the .config, so this seems OK.
>
>
>
>> @@ -499,7 +498,7 @@ class KconfigParser:
>> print log,
>>
>> if not self.options.dry_run:
>> - with open(defconfig_path, 'a') as f:
>> + with open(dotconfig_path, 'a') as f:
>> for (action, value) in results:
>> if action == ACTION_MOVE:
>> f.write(value + '\n')
>> @@ -608,6 +607,24 @@ class Slot:
>>
>> if self.state == STATE_AUTOCONF:
>> self.parser.update_defconfig(self.defconfig)
>> +
>> + """Save off the defconfig in a consistent way"""
>> + cmd = list(self.make_cmd)
>> + cmd.append('savedefconfig')
>> + self.ps = subprocess.Popen(cmd, stdout=self.devnull,
>> + stderr=self.devnull)
>> + self.state = STATE_SAVEDEFCONFIG
>> + return False
>> +
>> + if self.state == STATE_SAVEDEFCONFIG:
>> + defconfig_path = os.path.join(self.build_dir, 'defconfig')
>> + if not os.path.exists(defconfig_path):
>> + print >> sys.stderr, log_msg(self.options.color,
>> + COLOR_LIGHT_RED,
>> + self.defconfig,
>> + 'The defconfig was not updated')
>
> Is this warning message reachable?
>
> The missing defconfig means "make savedefconfig" failed.
>
> That case has been already caught by 'Failed to process.' above, I think.
OK... dropping that warning.
>> + shutil.move(defconfig_path,
>> + os.path.join('configs', self.defconfig))
>> self.state = STATE_IDLE
>> return True
>
>
>
>
>
> --
> Best Regards
> Masahiro Yamada
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config
2015-05-19 1:58 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Masahiro Yamada
2015-05-19 15:35 ` Joe Hershberger
@ 2015-05-19 18:07 ` Joe Hershberger
1 sibling, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 18:07 UTC (permalink / raw)
To: u-boot
Hi Masahiro-san,
On Mon, May 18, 2015 at 8:58 PM, Masahiro Yamada
<yamada.masahiro@socionext.com> wrote:
> Hi Joe,
>
> 2015-05-16 6:40 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
>> This will ensure that the order of the defconfig entries will always
>> match that of the Kconfig files. After one slightly painful (but
>> still early in the process) pass over all boards, this should keep
>> the defconfigs clean from here on.
>>
>> Users must edit the Kconfig first to add the menu entries and then run
>> moveconfig.py to update the defconfig files and the include configs.
>>
>> As such, moveconfig.py cannot compare against the '.config' contents.
>>
>> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
>
>
> This feature expects the defconfigs are all clean.
> Otherwise, savedefconfig would make "git diff" noisier.
>
> It is safer to use "make menuconfig && make savedefconfig"
> for adding new options, but some people still try to
> edit the defconfig directly...
>
> Perhaps, should do the global cleanup periodically?
I agree that until people (both contributors and custodians) get used
to this as the best way, we will likely need a few global clean-ups,
but I think with the addition of this tool to main-line the instance
of this will likely drop sharply. Moving configs is a hassle, so with
a tool available I doubt many people will do it manually. That's why
it is important that the tool do it the cleanest way. Hopefully for
the case where a config is changed on a board, the custodian will
notice if it is just added to the bottom of the file. It seems to work
out for Linux.
Cheers,
-Joe
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v5 1/9] moveconfig: Always run savedefconfig on the moved config
2015-05-15 21:40 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Joe Hershberger
` (9 preceding siblings ...)
2015-05-19 1:58 ` [U-Boot] [PATCH v4 01/10] moveconfig: Always run savedefconfig on the moved config Masahiro Yamada
@ 2015-05-19 18:21 ` Joe Hershberger
2015-05-19 18:21 ` [U-Boot] [PATCH v5 2/9] moveconfig: Ignore duplicate configs when moving Joe Hershberger
` (9 more replies)
10 siblings, 10 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 18:21 UTC (permalink / raw)
To: u-boot
This will ensure that the order of the defconfig entries will always
match that of the Kconfig files. After one slightly painful (but
still early in the process) pass over all boards, this should keep
the defconfigs clean from here on.
Users must edit the Kconfig first to add the menu entries and then run
moveconfig.py to update the defconfig files and the include configs.
As such, moveconfig.py cannot compare against the '.config' contents.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
This is based on https://patchwork.ozlabs.org/patch/472591/
Changes in v5:
-Removed warning that may never be reached
Changes in v4:
-Rebased series on Masahiro's v2
Changes in v3: None
Changes in v2: None
tools/moveconfig.py | 30 +++++++++++++++++++++---------
1 file changed, 21 insertions(+), 9 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index c39ea95..c9984b4 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -46,6 +46,9 @@ should look like this:
CONFIG_CMD_USB bool n
CONFIG_SYS_TEXT_BASE hex 0x00000000
+Next you must edit the Kconfig to add the menu entries for the configs
+you are moving.
+
And then run this tool giving the file name of the recipe
$ tools/moveconfig.py recipe
@@ -192,6 +195,7 @@ CROSS_COMPILE = {
STATE_IDLE = 0
STATE_DEFCONFIG = 1
STATE_AUTOCONF = 2
+STATE_SAVEDEFCONFIG = 3
ACTION_MOVE = 0
ACTION_DEFAULT_VALUE = 1
@@ -390,8 +394,7 @@ class KconfigParser:
return CROSS_COMPILE.get(arch, '')
- def parse_one_config(self, config_attr, defconfig_lines,
- dotconfig_lines, autoconf_lines):
+ def parse_one_config(self, config_attr, defconfig_lines, autoconf_lines):
"""Parse .config, defconfig, include/autoconf.mk for one config.
This function looks for the config options in the lines from
@@ -402,7 +405,6 @@ class KconfigParser:
config_attr: A dictionary including the name, the type,
and the default value of the target config.
defconfig_lines: lines from the original defconfig file.
- dotconfig_lines: lines from the .config file.
autoconf_lines: lines from the include/autoconf.mk file.
Returns:
@@ -418,7 +420,7 @@ class KconfigParser:
else:
default = config + '=' + config_attr['default']
- for line in defconfig_lines + dotconfig_lines:
+ for line in defconfig_lines:
line = line.rstrip()
if line.startswith(config + '=') or line == not_set:
return (ACTION_ALREADY_EXIST, line)
@@ -463,15 +465,12 @@ class KconfigParser:
with open(defconfig_path) as f:
defconfig_lines = f.readlines()
- with open(dotconfig_path) as f:
- dotconfig_lines = f.readlines()
-
with open(autoconf_path) as f:
autoconf_lines = f.readlines()
for config_attr in self.config_attrs:
result = self.parse_one_config(config_attr, defconfig_lines,
- dotconfig_lines, autoconf_lines)
+ autoconf_lines)
results.append(result)
log = ''
@@ -499,7 +498,7 @@ class KconfigParser:
print log,
if not self.options.dry_run:
- with open(defconfig_path, 'a') as f:
+ with open(dotconfig_path, 'a') as f:
for (action, value) in results:
if action == ACTION_MOVE:
f.write(value + '\n')
@@ -608,6 +607,19 @@ class Slot:
if self.state == STATE_AUTOCONF:
self.parser.update_defconfig(self.defconfig)
+
+ """Save off the defconfig in a consistent way"""
+ cmd = list(self.make_cmd)
+ cmd.append('savedefconfig')
+ self.ps = subprocess.Popen(cmd, stdout=self.devnull,
+ stderr=self.devnull)
+ self.state = STATE_SAVEDEFCONFIG
+ return False
+
+ if self.state == STATE_SAVEDEFCONFIG:
+ defconfig_path = os.path.join(self.build_dir, 'defconfig')
+ shutil.move(defconfig_path,
+ os.path.join('configs', self.defconfig))
self.state = STATE_IDLE
return True
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v5 2/9] moveconfig: Ignore duplicate configs when moving
2015-05-19 18:21 ` [U-Boot] [PATCH v5 1/9] " Joe Hershberger
@ 2015-05-19 18:21 ` Joe Hershberger
2015-05-19 18:21 ` [U-Boot] [PATCH v5 3/9] moveconfig: Add a parameter to accept a list to build Joe Hershberger
` (8 subsequent siblings)
9 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 18:21 UTC (permalink / raw)
To: u-boot
When moving configs, it is important to know what was defined in the
config header even if it duplicates the configs coming from Kconfig.
This is specifically needed for the case where a config is set to
default 'y' in the Kconfig. This would previously cause the actual value
from the include config to be filtered out, and moveconfig.py would
think that it was 'n'... This means that the value that should be 'y'
is now (in every defconfig) set to 'not set'.
tools/moveconfig.py now defines KCONFIG_IGNORE_DUPLICATES to prevent the
filtering from happening and selecting wrong values for the defconfig.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v5: None
Changes in v4: None
Changes in v3:
-New for version 3
Changes in v2: None
scripts/Makefile.autoconf | 3 ++-
tools/moveconfig.py | 1 +
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/scripts/Makefile.autoconf b/scripts/Makefile.autoconf
index f054081..36bfa17 100644
--- a/scripts/Makefile.autoconf
+++ b/scripts/Makefile.autoconf
@@ -58,7 +58,8 @@ quiet_cmd_autoconf = GEN $@
$(CPP) $(c_flags) $2 -DDO_DEPS_ONLY -dM $(srctree)/include/common.h > $@.tmp && { \
sed -n -f $(srctree)/tools/scripts/define2mk.sed $@.tmp | \
while read line; do \
- if ! grep -q "$${line%=*}=" include/config/auto.conf; then \
+ if [ -n "${KCONFIG_IGNORE_DUPLICATES}" ] || \
+ ! grep -q "$${line%=*}=" include/config/auto.conf; then \
echo "$$line"; \
fi \
done > $@; \
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index c9984b4..dd9434d 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -627,6 +627,7 @@ class Slot:
cmd = list(self.make_cmd)
if cross_compile:
cmd.append('CROSS_COMPILE=%s' % cross_compile)
+ cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
cmd.append('include/config/auto.conf')
self.ps = subprocess.Popen(cmd, stdout=self.devnull)
self.state = STATE_AUTOCONF
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v5 3/9] moveconfig: Add a parameter to accept a list to build
2015-05-19 18:21 ` [U-Boot] [PATCH v5 1/9] " Joe Hershberger
2015-05-19 18:21 ` [U-Boot] [PATCH v5 2/9] moveconfig: Ignore duplicate configs when moving Joe Hershberger
@ 2015-05-19 18:21 ` Joe Hershberger
2015-05-19 18:21 ` [U-Boot] [PATCH v5 4/9] moveconfig: Add a switch to only cleanup headers Joe Hershberger
` (7 subsequent siblings)
9 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 18:21 UTC (permalink / raw)
To: u-boot
This is helpful to re-attempt to move failed boards from a previous run
without starting over.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v5:
-Remove default 'r' mode on open
Changes in v4: None
Changes in v3:
-Fixed command line options order (alphabetize)
Changes in v2:
-New for version 2
tools/moveconfig.py | 26 ++++++++++++++++++++------
1 file changed, 20 insertions(+), 6 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index dd9434d..e93548c 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -135,6 +135,9 @@ Available options
Surround each portion of the log with escape sequences to display it
in color on the terminal.
+ -d, --defconfigs
+ Specify a file containing a list of defconfigs to move
+
-n, --dry-run
Peform a trial run that does not make any changes. It is useful to
see what is going to happen before one actually runs it.
@@ -729,12 +732,21 @@ def move_config(config_attrs, options):
config_attr['type'],
config_attr['default'])
- # All the defconfig files to be processed
- defconfigs = []
- for (dirpath, dirnames, filenames) in os.walk('configs'):
- dirpath = dirpath[len('configs') + 1:]
- for filename in fnmatch.filter(filenames, '*_defconfig'):
- defconfigs.append(os.path.join(dirpath, filename))
+ if options.defconfigs:
+ defconfigs = [line.strip() for line in open(options.defconfigs)]
+ for i, defconfig in enumerate(defconfigs):
+ if not defconfig.endswith('_defconfig'):
+ defconfigs[i] = defconfig + '_defconfig'
+ if not os.path.exists(os.path.join('configs', defconfigs[i])):
+ sys.exit('%s - defconfig does not exist. Stopping.' %
+ defconfigs[i])
+ else:
+ # All the defconfig files to be processed
+ defconfigs = []
+ for (dirpath, dirnames, filenames) in os.walk('configs'):
+ dirpath = dirpath[len('configs') + 1:]
+ for filename in fnmatch.filter(filenames, '*_defconfig'):
+ defconfigs.append(os.path.join(dirpath, filename))
slots = Slots(config_attrs, options)
@@ -835,6 +847,8 @@ def main():
# Add options here
parser.add_option('-c', '--color', action='store_true', default=False,
help='display the log in color')
+ parser.add_option('-d', '--defconfigs', type='string',
+ help='a file containing a list of defconfigs to move')
parser.add_option('-n', '--dry-run', action='store_true', default=False,
help='perform a trial run (show log with no changes)')
parser.add_option('-e', '--exit-on-error', action='store_true',
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v5 4/9] moveconfig: Add a switch to only cleanup headers
2015-05-19 18:21 ` [U-Boot] [PATCH v5 1/9] " Joe Hershberger
2015-05-19 18:21 ` [U-Boot] [PATCH v5 2/9] moveconfig: Ignore duplicate configs when moving Joe Hershberger
2015-05-19 18:21 ` [U-Boot] [PATCH v5 3/9] moveconfig: Add a parameter to accept a list to build Joe Hershberger
@ 2015-05-19 18:21 ` Joe Hershberger
2015-05-19 18:21 ` [U-Boot] [PATCH v5 5/9] moveconfig: Cleanup headers in arch and board Joe Hershberger
` (6 subsequent siblings)
9 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 18:21 UTC (permalink / raw)
To: u-boot
In some case you may want to only cleanup the headers. Make it possible
without waiting for all boards to compile.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v5:
-Move check_top_directory to main
Changes in v4: None
Changes in v3:
-New for version 3
Changes in v2: None
tools/moveconfig.py | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index e93548c..561cd9a 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -146,6 +146,9 @@ Available options
Exit immediately if Make exits with a non-zero status while processing
a defconfig file.
+ -H, --headers-only
+ Only cleanup the headers; skip the defconfig processing
+
-j, --jobs
Specify the number of threads to run simultaneously. If not specified,
the number of threads is the same as the number of CPU cores.
@@ -720,8 +723,6 @@ def move_config(config_attrs, options):
the type, and the default value of the target config.
options: option flags
"""
- check_top_directory()
-
if len(config_attrs) == 0:
print 'Nothing to do. exit.'
sys.exit(0)
@@ -765,8 +766,6 @@ def move_config(config_attrs, options):
slots.show_failed_boards()
- cleanup_headers(config_attrs, options.dry_run)
-
def bad_recipe(filename, linenum, msg):
"""Print error message with the file name and the line number and exit."""
sys.exit("%s: line %d: error : " % (filename, linenum) + msg)
@@ -854,6 +853,9 @@ def main():
parser.add_option('-e', '--exit-on-error', action='store_true',
default=False,
help='exit immediately on any error')
+ parser.add_option('-H', '--headers-only', dest='cleanup_headers_only',
+ action='store_true', default=False,
+ help='only cleanup the headers')
parser.add_option('-j', '--jobs', type='int', default=cpu_count,
help='the number of jobs to run simultaneously')
parser.usage += ' recipe_file\n\n' + \
@@ -874,7 +876,12 @@ def main():
update_cross_compile()
- move_config(config_attrs, options)
+ check_top_directory()
+
+ if not options.cleanup_headers_only:
+ move_config(config_attrs, options)
+
+ cleanup_headers(config_attrs, options.dry_run)
if __name__ == '__main__':
main()
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v5 5/9] moveconfig: Cleanup headers in arch and board
2015-05-19 18:21 ` [U-Boot] [PATCH v5 1/9] " Joe Hershberger
` (2 preceding siblings ...)
2015-05-19 18:21 ` [U-Boot] [PATCH v5 4/9] moveconfig: Add a switch to only cleanup headers Joe Hershberger
@ 2015-05-19 18:21 ` Joe Hershberger
2015-05-19 18:21 ` [U-Boot] [PATCH v5 6/9] moveconfig: Output a list of failed boards Joe Hershberger
` (5 subsequent siblings)
9 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 18:21 UTC (permalink / raw)
To: u-boot
Some config.h files live in arch and board directories. They will need
to be cleaned up as well, so run the same filters there.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v5:
-Consolidate code to clean up dirs
Changes in v4:
-New for version 4
Changes in v3: None
Changes in v2: None
tools/moveconfig.py | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 561cd9a..f38dac1 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -347,11 +347,12 @@ def cleanup_headers(config_attrs, dry_run):
patterns.append(re.compile(r'#\s*define\s+%s\W' % config))
patterns.append(re.compile(r'#\s*undef\s+%s\W' % config))
- for (dirpath, dirnames, filenames) in os.walk('include'):
- for filename in filenames:
- if not fnmatch.fnmatch(filename, '*~'):
- cleanup_one_header(os.path.join(dirpath, filename), patterns,
- dry_run)
+ for dir in 'include', 'arch', 'board':
+ for (dirpath, dirnames, filenames) in os.walk(dir):
+ for filename in filenames:
+ if not fnmatch.fnmatch(filename, '*~'):
+ cleanup_one_header(os.path.join(dirpath, filename),
+ patterns, dry_run)
### classes ###
class KconfigParser:
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v5 6/9] moveconfig: Output a list of failed boards
2015-05-19 18:21 ` [U-Boot] [PATCH v5 1/9] " Joe Hershberger
` (3 preceding siblings ...)
2015-05-19 18:21 ` [U-Boot] [PATCH v5 5/9] moveconfig: Cleanup headers in arch and board Joe Hershberger
@ 2015-05-19 18:21 ` Joe Hershberger
2015-05-19 18:21 ` [U-Boot] [PATCH v5 7/9] moveconfig: Print a message for missing compiler Joe Hershberger
` (4 subsequent siblings)
9 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 18:21 UTC (permalink / raw)
To: u-boot
If boards fail, output that list to a file so that it can easily be
passed back into moveconfig.py using the -d option.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v5:
-Changed file handling to use with/as
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/moveconfig.py | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index f38dac1..8b8eed6 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -716,6 +716,10 @@ class Slots:
print >> sys.stderr, color_text(self.options.color,
COLOR_LIGHT_RED, line)
+ with open('moveconfig.failed', 'w') as f:
+ for board in failed_boards:
+ f.write(board + '\n')
+
def move_config(config_attrs, options):
"""Move config options to defconfig files.
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v5 7/9] moveconfig: Print a message for missing compiler
2015-05-19 18:21 ` [U-Boot] [PATCH v5 1/9] " Joe Hershberger
` (4 preceding siblings ...)
2015-05-19 18:21 ` [U-Boot] [PATCH v5 6/9] moveconfig: Output a list of failed boards Joe Hershberger
@ 2015-05-19 18:21 ` Joe Hershberger
2015-05-19 18:21 ` [U-Boot] [PATCH v5 8/9] moveconfig: Add a switch to enable printing errors Joe Hershberger
` (3 subsequent siblings)
9 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 18:21 UTC (permalink / raw)
To: u-boot
A common case for failed builds is a missing compiler. Print a message
for that case to tell the user concisely which compiler was expected
that was not found.
This patch also has the effect of not printing build errors any longer.
The next patch will add a switch to optionally bring that back.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v5:
-Use scoped PIPE instead of importing it
-Set LANG=C for calling the compiler when planning to screen-scrape
-Use PIPE for all Popen
-Make missing compilers yellow
Changes in v4: None
Changes in v3: None
Changes in v2: None
tools/moveconfig.py | 28 ++++++++++++++++++++--------
1 file changed, 20 insertions(+), 8 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 8b8eed6..2fa98e3 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -572,7 +572,8 @@ class Slot:
return False
cmd = list(self.make_cmd)
cmd.append(defconfig)
- self.ps = subprocess.Popen(cmd, stdout=self.devnull)
+ self.ps = subprocess.Popen(cmd, stdout=self.devnull,
+ stderr=subprocess.PIPE)
self.defconfig = defconfig
self.state = STATE_DEFCONFIG
return True
@@ -597,11 +598,18 @@ class Slot:
return False
if self.ps.poll() != 0:
-
+ errmsg = 'Failed to process.'
+ errout = self.ps.stderr.read()
+ if errout.find('gcc: command not found') != -1:
+ errmsg = 'Compiler not found ('
+ errmsg += color_text(self.options.color, COLOR_YELLOW,
+ self.cross_compile)
+ errmsg += color_text(self.options.color, COLOR_LIGHT_RED,
+ ')')
print >> sys.stderr, log_msg(self.options.color,
COLOR_LIGHT_RED,
self.defconfig,
- "failed to process.")
+ errmsg),
if self.options.exit_on_error:
sys.exit("Exit on error.")
else:
@@ -619,7 +627,7 @@ class Slot:
cmd = list(self.make_cmd)
cmd.append('savedefconfig')
self.ps = subprocess.Popen(cmd, stdout=self.devnull,
- stderr=self.devnull)
+ stderr=subprocess.PIPE)
self.state = STATE_SAVEDEFCONFIG
return False
@@ -630,13 +638,17 @@ class Slot:
self.state = STATE_IDLE
return True
- cross_compile = self.parser.get_cross_compile()
+ self.cross_compile = self.parser.get_cross_compile()
cmd = list(self.make_cmd)
- if cross_compile:
- cmd.append('CROSS_COMPILE=%s' % cross_compile)
+ if self.cross_compile:
+ cmd.append('CROSS_COMPILE=%s' % self.cross_compile)
cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
cmd.append('include/config/auto.conf')
- self.ps = subprocess.Popen(cmd, stdout=self.devnull)
+ """This will be screen-scraped, so be sure the expected text will be
+ returned consistently on every machine by setting LANG=C"""
+ self.ps = subprocess.Popen(cmd, stdout=self.devnull,
+ env=dict(os.environ, LANG='C'),
+ stderr=subprocess.PIPE)
self.state = STATE_AUTOCONF
return False
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v5 8/9] moveconfig: Add a switch to enable printing errors
2015-05-19 18:21 ` [U-Boot] [PATCH v5 1/9] " Joe Hershberger
` (5 preceding siblings ...)
2015-05-19 18:21 ` [U-Boot] [PATCH v5 7/9] moveconfig: Print a message for missing compiler Joe Hershberger
@ 2015-05-19 18:21 ` Joe Hershberger
2015-05-19 18:21 ` [U-Boot] [PATCH v5 9/9] moveconfig: Print status about the processed defconfigs Joe Hershberger
` (2 subsequent siblings)
9 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 18:21 UTC (permalink / raw)
To: u-boot
In some cases the build for the autoconf breaks. This outputs the errors
following the status so that action can be taken without building again
manually.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v5:
-Remove redundant destination
Changes in v4: None
Changes in v3: None
Changes in v2:
-New for version 2
tools/moveconfig.py | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 2fa98e3..a6f1853 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -153,6 +153,9 @@ Available options
Specify the number of threads to run simultaneously. If not specified,
the number of threads is the same as the number of CPU cores.
+ -v, --verbose
+ Show any build errors as boards are built
+
To see the complete list of supported options, run
$ tools/moveconfig.py -h
@@ -610,6 +613,9 @@ class Slot:
COLOR_LIGHT_RED,
self.defconfig,
errmsg),
+ if self.options.verbose:
+ print >> sys.stderr, color_text(self.options.color,
+ COLOR_LIGHT_CYAN, errout)
if self.options.exit_on_error:
sys.exit("Exit on error.")
else:
@@ -875,6 +881,8 @@ def main():
help='only cleanup the headers')
parser.add_option('-j', '--jobs', type='int', default=cpu_count,
help='the number of jobs to run simultaneously')
+ parser.add_option('-v', '--verbose', action='store_true', default=False,
+ help='show any build errors as boards are built')
parser.usage += ' recipe_file\n\n' + \
'The recipe_file should describe config options you want to move.\n' + \
'Each line should contain config_name, type, default_value\n\n' + \
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v5 9/9] moveconfig: Print status about the processed defconfigs
2015-05-19 18:21 ` [U-Boot] [PATCH v5 1/9] " Joe Hershberger
` (6 preceding siblings ...)
2015-05-19 18:21 ` [U-Boot] [PATCH v5 8/9] moveconfig: Add a switch to enable printing errors Joe Hershberger
@ 2015-05-19 18:21 ` Joe Hershberger
2015-05-20 4:48 ` [U-Boot] [PATCH v5 1/9] moveconfig: Always run savedefconfig on the moved config Masahiro Yamada
2015-05-26 23:43 ` Masahiro Yamada
9 siblings, 0 replies; 82+ messages in thread
From: Joe Hershberger @ 2015-05-19 18:21 UTC (permalink / raw)
To: u-boot
This gives a basic idea about progress.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
---
Changes in v5: None
Changes in v4: None
Changes in v3:
-New for version 3
Changes in v2: None
tools/moveconfig.py | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index a6f1853..a6e7c22 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -558,7 +558,7 @@ class Slot:
pass
shutil.rmtree(self.build_dir)
- def add(self, defconfig):
+ def add(self, defconfig, num, total):
"""Assign a new subprocess for defconfig and add it to the slot.
If the slot is vacant, create a new subprocess for processing the
@@ -579,6 +579,8 @@ class Slot:
stderr=subprocess.PIPE)
self.defconfig = defconfig
self.state = STATE_DEFCONFIG
+ self.num = num
+ self.total = total
return True
def poll(self):
@@ -629,6 +631,9 @@ class Slot:
if self.state == STATE_AUTOCONF:
self.parser.update_defconfig(self.defconfig)
+ print ' %d defconfigs out of %d\r' % (self.num + 1, self.total),
+ sys.stdout.flush()
+
"""Save off the defconfig in a consistent way"""
cmd = list(self.make_cmd)
cmd.append('savedefconfig')
@@ -682,7 +687,7 @@ class Slots:
for i in range(options.jobs):
self.slots.append(Slot(config_attrs, options, devnull, make_cmd))
- def add(self, defconfig):
+ def add(self, defconfig, num, total):
"""Add a new subprocess if a vacant slot is found.
Arguments:
@@ -692,7 +697,7 @@ class Slots:
Return True on success or False on failure
"""
for slot in self.slots:
- if slot.add(defconfig):
+ if slot.add(defconfig, num, total):
return True
return False
@@ -777,8 +782,8 @@ def move_config(config_attrs, options):
# Main loop to process defconfig files:
# Add a new subprocess into a vacant slot.
# Sleep if there is no available slot.
- for defconfig in defconfigs:
- while not slots.add(defconfig):
+ for i, defconfig in enumerate(defconfigs):
+ while not slots.add(defconfig, i, len(defconfigs)):
while not slots.available():
# No available slot: sleep for a while
time.sleep(SLEEP_TIME)
@@ -787,6 +792,7 @@ def move_config(config_attrs, options):
while not slots.empty():
time.sleep(SLEEP_TIME)
+ print ''
slots.show_failed_boards()
def bad_recipe(filename, linenum, msg):
--
1.7.11.5
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v5 1/9] moveconfig: Always run savedefconfig on the moved config
2015-05-19 18:21 ` [U-Boot] [PATCH v5 1/9] " Joe Hershberger
` (7 preceding siblings ...)
2015-05-19 18:21 ` [U-Boot] [PATCH v5 9/9] moveconfig: Print status about the processed defconfigs Joe Hershberger
@ 2015-05-20 4:48 ` Masahiro Yamada
2015-05-26 23:43 ` Masahiro Yamada
9 siblings, 0 replies; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-20 4:48 UTC (permalink / raw)
To: u-boot
Hi Joe,
2015-05-20 3:21 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> This will ensure that the order of the defconfig entries will always
> match that of the Kconfig files. After one slightly painful (but
> still early in the process) pass over all boards, this should keep
> the defconfigs clean from here on.
>
> Users must edit the Kconfig first to add the menu entries and then run
> moveconfig.py to update the defconfig files and the include configs.
>
> As such, moveconfig.py cannot compare against the '.config' contents.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
> ---
> This is based on https://patchwork.ozlabs.org/patch/472591/
>
> Changes in v5:
> -Removed warning that may never be reached
The whole series of v5 looks good to me.
Tom,
looks like this series is delegated to me.
Shall I apply my base patch and Joe's great improvements and then send
a pull-req?
Or would you do it?
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread
* [U-Boot] [PATCH v5 1/9] moveconfig: Always run savedefconfig on the moved config
2015-05-19 18:21 ` [U-Boot] [PATCH v5 1/9] " Joe Hershberger
` (8 preceding siblings ...)
2015-05-20 4:48 ` [U-Boot] [PATCH v5 1/9] moveconfig: Always run savedefconfig on the moved config Masahiro Yamada
@ 2015-05-26 23:43 ` Masahiro Yamada
9 siblings, 0 replies; 82+ messages in thread
From: Masahiro Yamada @ 2015-05-26 23:43 UTC (permalink / raw)
To: u-boot
2015-05-20 3:21 GMT+09:00 Joe Hershberger <joe.hershberger@ni.com>:
> This will ensure that the order of the defconfig entries will always
> match that of the Kconfig files. After one slightly painful (but
> still early in the process) pass over all boards, this should keep
> the defconfigs clean from here on.
>
> Users must edit the Kconfig first to add the menu entries and then run
> moveconfig.py to update the defconfig files and the include configs.
>
> As such, moveconfig.py cannot compare against the '.config' contents.
>
> Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
> ---
> This is based on https://patchwork.ozlabs.org/patch/472591/
>
> Changes in v5:
> -Removed warning that may never be reached
The series, applied to u-boot-uniphier/misc with my Acked-by.
Thanks!
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 82+ messages in thread