* [U-Boot] [PATCH v2 0/8] tools: moveconfig: a collection of improvement of garbage cleaning
@ 2016-07-25 10:15 Masahiro Yamada
2016-07-25 10:15 ` [U-Boot] [PATCH v2 1/8] tools: moveconfig: do not cleanup headers in include/generated Masahiro Yamada
` (7 more replies)
0 siblings, 8 replies; 23+ messages in thread
From: Masahiro Yamada @ 2016-07-25 10:15 UTC (permalink / raw)
To: u-boot
Masahiro Yamada (8):
tools: moveconfig: do not cleanup headers in include/generated
tools: moveconfig: do not check clean tree and compilers for -H option
tools: moveconfig: trim garbage lines after header cleanups
tools: moveconfig: show result of header cleaning in unified diff
tools: moveconfig: show diffs of cleaned headers in color
tools: moveconfig: fix cleanup of defines across multiple lines
tools: moveconfig: make getting all defconfigs into helper function
tools: moveconfig: support CONFIG_SYS_EXTRA_OPTIONS cleaning
tools/moveconfig.py | 231 ++++++++++++++++++++++++++++++++++++++++++++++------
1 file changed, 208 insertions(+), 23 deletions(-)
--
1.9.1
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 1/8] tools: moveconfig: do not cleanup headers in include/generated
2016-07-25 10:15 [U-Boot] [PATCH v2 0/8] tools: moveconfig: a collection of improvement of garbage cleaning Masahiro Yamada
@ 2016-07-25 10:15 ` Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 2/8] tools: moveconfig: do not check clean tree and compilers for -H option Masahiro Yamada
` (6 subsequent siblings)
7 siblings, 2 replies; 23+ messages in thread
From: Masahiro Yamada @ 2016-07-25 10:15 UTC (permalink / raw)
To: u-boot
The files in include/generated are generated during build and removed
by "make mrproper", so it has no point to touch them by this tool.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v2: None
tools/moveconfig.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index bf60dbc..b1190e2 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -371,6 +371,8 @@ def cleanup_headers(configs, dry_run):
for dir in 'include', 'arch', 'board':
for (dirpath, dirnames, filenames) in os.walk(dir):
+ if dirpath == os.path.join('include', 'generated'):
+ continue
for filename in filenames:
if not fnmatch.fnmatch(filename, '*~'):
cleanup_one_header(os.path.join(dirpath, filename),
--
1.9.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 2/8] tools: moveconfig: do not check clean tree and compilers for -H option
2016-07-25 10:15 [U-Boot] [PATCH v2 0/8] tools: moveconfig: a collection of improvement of garbage cleaning Masahiro Yamada
2016-07-25 10:15 ` [U-Boot] [PATCH v2 1/8] tools: moveconfig: do not cleanup headers in include/generated Masahiro Yamada
@ 2016-07-25 10:15 ` Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 3/8] tools: moveconfig: trim garbage lines after header cleanups Masahiro Yamada
` (5 subsequent siblings)
7 siblings, 2 replies; 23+ messages in thread
From: Masahiro Yamada @ 2016-07-25 10:15 UTC (permalink / raw)
To: u-boot
The clean tree (make mrproper) and compilers are required when moving
config options, but not needed when we only cleanup headers.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v2: None
tools/moveconfig.py | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index b1190e2..05a581f 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -1042,11 +1042,9 @@ def main():
check_top_directory()
- check_clean_directory()
-
- update_cross_compile(options.color)
-
if not options.cleanup_headers_only:
+ check_clean_directory()
+ update_cross_compile(options.color)
move_config(configs, options)
if configs:
--
1.9.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 3/8] tools: moveconfig: trim garbage lines after header cleanups
2016-07-25 10:15 [U-Boot] [PATCH v2 0/8] tools: moveconfig: a collection of improvement of garbage cleaning Masahiro Yamada
2016-07-25 10:15 ` [U-Boot] [PATCH v2 1/8] tools: moveconfig: do not cleanup headers in include/generated Masahiro Yamada
2016-07-25 10:15 ` [U-Boot] [PATCH v2 2/8] tools: moveconfig: do not check clean tree and compilers for -H option Masahiro Yamada
@ 2016-07-25 10:15 ` Masahiro Yamada
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 4/8] tools: moveconfig: show result of header cleaning in unified diff Masahiro Yamada
` (4 subsequent siblings)
7 siblings, 1 reply; 23+ messages in thread
From: Masahiro Yamada @ 2016-07-25 10:15 UTC (permalink / raw)
To: u-boot
The tools/moveconfig.py has a feature to cleanup #define/#undef's
of moved config options, but I want this tool to do a better job.
For example, when we are moving CONFIG_FOO and its define is
surrounded by #ifdef ... #endif, like follows:
#ifdef CONFIG_BAR
# define CONFIG_FOO
#endif
The header cleanup will leave empty #ifdef ... #endif:
#ifdef CONFIG_BAR
#endif
Likewise, if a define line between two blank lines
<blank line>
#define CONFIG_FOO
<blank lines.
... is deleted, the result of the clean-up will be successive empty
lines, which is a coding-style violation.
It is tedious to remove left-over garbage lines manually, so I want
the tool to take care of this. The tool's job is still not perfect,
so we should check the output of the tool, but I hope our life will
be much easier with this patch.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v2:
- cleanup between #elif ... #endif,
#elif ... #elif
tools/moveconfig.py | 85 ++++++++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 81 insertions(+), 4 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 05a581f..03e4953 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -160,6 +160,7 @@ To see the complete list of supported options, run
"""
+import copy
import filecmp
import fnmatch
import multiprocessing
@@ -319,6 +320,57 @@ def update_cross_compile(color_enabled):
CROSS_COMPILE[arch] = cross_compile
+def extend_matched_lines(lines, matched, pre_patterns, post_patterns, extend_pre,
+ extend_post):
+ """Extend matched lines if desired patterns are found before/after already
+ matched lines.
+
+ Arguments:
+ lines: A list of lines handled.
+ matched: A list of line numbers that have been already matched.
+ (will be updated by this function)
+ pre_patterns: A list of regular expression that should be matched as
+ preamble.
+ post_patterns: A list of regular expression that should be matched as
+ postamble.
+ extend_pre: Add the line number of matched preamble to the matched list.
+ extend_post: Add the line number of matched postamble to the matched list.
+ """
+ extended_matched = []
+
+ j = matched[0]
+
+ for i in matched:
+ if i == 0 or i < j:
+ continue
+ j = i
+ while j in matched:
+ j += 1
+ if j >= len(lines):
+ break
+
+ for p in pre_patterns:
+ if p.search(lines[i - 1]):
+ break
+ else:
+ # not matched
+ continue
+
+ for p in post_patterns:
+ if p.search(lines[j]):
+ break
+ else:
+ # not matched
+ continue
+
+ if extend_pre:
+ extended_matched.append(i - 1)
+ if extend_post:
+ extended_matched.append(j)
+
+ matched += extended_matched
+ matched.sort()
+
def cleanup_one_header(header_path, patterns, dry_run):
"""Clean regex-matched lines away from a file.
@@ -334,13 +386,38 @@ def cleanup_one_header(header_path, patterns, dry_run):
matched = []
for i, line in enumerate(lines):
for pattern in patterns:
- m = pattern.search(line)
- if m:
- print '%s: %s: %s' % (header_path, i + 1, line),
+ if pattern.search(line):
matched.append(i)
break
- if dry_run or not matched:
+ if not matched:
+ return
+
+ # remove empty #ifdef ... #endif, successive blank lines
+ pattern_if = re.compile(r'#\s*if(def|ndef)?\W') # #if, #ifdef, #ifndef
+ pattern_elif = re.compile(r'#\s*el(if|se)\W') # #elif, #else
+ pattern_endif = re.compile(r'#\s*endif\W') # #endif
+ pattern_blank = re.compile(r'^\s*$') # empty line
+
+ while True:
+ old_matched = copy.copy(matched)
+ extend_matched_lines(lines, matched, [pattern_if],
+ [pattern_endif], True, True)
+ extend_matched_lines(lines, matched, [pattern_elif],
+ [pattern_elif, pattern_endif], True, False)
+ extend_matched_lines(lines, matched, [pattern_if, pattern_elif],
+ [pattern_blank], False, True)
+ extend_matched_lines(lines, matched, [pattern_blank],
+ [pattern_elif, pattern_endif], True, False)
+ extend_matched_lines(lines, matched, [pattern_blank],
+ [pattern_blank], True, False)
+ if matched == old_matched:
+ break
+
+ for i in matched:
+ print '%s: %s: %s' % (header_path, i + 1, lines[i]),
+
+ if dry_run:
return
with open(header_path, 'w') as f:
--
1.9.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 4/8] tools: moveconfig: show result of header cleaning in unified diff
2016-07-25 10:15 [U-Boot] [PATCH v2 0/8] tools: moveconfig: a collection of improvement of garbage cleaning Masahiro Yamada
` (2 preceding siblings ...)
2016-07-25 10:15 ` [U-Boot] [PATCH v2 3/8] tools: moveconfig: trim garbage lines after header cleanups Masahiro Yamada
@ 2016-07-25 10:15 ` Masahiro Yamada
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 5/8] tools: moveconfig: show diffs of cleaned headers in color Masahiro Yamada
` (3 subsequent siblings)
7 siblings, 1 reply; 23+ messages in thread
From: Masahiro Yamada @ 2016-07-25 10:15 UTC (permalink / raw)
To: u-boot
The header cleanup feature of this tool now removes empty ifdef's,
successive blank lines as well as moved option defines. So, we
want to see a little more context to check which lines were deleted.
It is true that we can see it by "git diff", but it would not work
in the --dry-run mode. So, here, this commit.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v2:
- Make diffing into a helper function
tools/moveconfig.py | 30 +++++++++++++++++++++++++-----
1 file changed, 25 insertions(+), 5 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 03e4953..4f07782 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -161,6 +161,7 @@ To see the complete list of supported options, run
"""
import copy
+import difflib
import filecmp
import fnmatch
import multiprocessing
@@ -275,6 +276,22 @@ def color_text(color_enabled, color, string):
else:
return string
+def show_diff(a, b, file_path):
+ """Show unidified diff.
+
+ Arguments:
+ a: A list of lines (before)
+ b: A list of lines (after)
+ file_path: Path to the file
+ """
+
+ diff = difflib.unified_diff(a, b,
+ fromfile=os.path.join('a', file_path),
+ tofile=os.path.join('b', file_path))
+
+ for line in diff:
+ print line,
+
def update_cross_compile(color_enabled):
"""Update per-arch CROSS_COMPILE via environment variables
@@ -414,16 +431,19 @@ def cleanup_one_header(header_path, patterns, dry_run):
if matched == old_matched:
break
- for i in matched:
- print '%s: %s: %s' % (header_path, i + 1, lines[i]),
+ tolines = copy.copy(lines)
+
+ for i in reversed(matched):
+ tolines.pop(i)
+
+ show_diff(lines, tolines, header_path)
if dry_run:
return
with open(header_path, 'w') as f:
- for i, line in enumerate(lines):
- if not i in matched:
- f.write(line)
+ for line in tolines:
+ f.write(line)
def cleanup_headers(configs, dry_run):
"""Delete config defines from board headers.
--
1.9.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 5/8] tools: moveconfig: show diffs of cleaned headers in color
2016-07-25 10:15 [U-Boot] [PATCH v2 0/8] tools: moveconfig: a collection of improvement of garbage cleaning Masahiro Yamada
` (3 preceding siblings ...)
2016-07-25 10:15 ` [U-Boot] [PATCH v2 4/8] tools: moveconfig: show result of header cleaning in unified diff Masahiro Yamada
@ 2016-07-25 10:15 ` Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 6/8] tools: moveconfig: fix cleanup of defines across multiple lines Masahiro Yamada
` (2 subsequent siblings)
7 siblings, 2 replies; 23+ messages in thread
From: Masahiro Yamada @ 2016-07-25 10:15 UTC (permalink / raw)
To: u-boot
Show code diff in color if --color option is given.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v2:
- Make diffing into a helper function
tools/moveconfig.py | 26 ++++++++++++++++----------
1 file changed, 16 insertions(+), 10 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 4f07782..4edcb6c 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -276,13 +276,14 @@ def color_text(color_enabled, color, string):
else:
return string
-def show_diff(a, b, file_path):
+def show_diff(a, b, file_path, color_enabled):
"""Show unidified diff.
Arguments:
a: A list of lines (before)
b: A list of lines (after)
file_path: Path to the file
+ color_enabled: Display the diff in color
"""
diff = difflib.unified_diff(a, b,
@@ -290,7 +291,12 @@ def show_diff(a, b, file_path):
tofile=os.path.join('b', file_path))
for line in diff:
- print line,
+ if line[0] == '-' and line[1] != '-':
+ print color_text(color_enabled, COLOR_RED, line),
+ elif line[0] == '+' and line[1] != '+':
+ print color_text(color_enabled, COLOR_GREEN, line),
+ else:
+ print line,
def update_cross_compile(color_enabled):
"""Update per-arch CROSS_COMPILE via environment variables
@@ -388,14 +394,14 @@ def extend_matched_lines(lines, matched, pre_patterns, post_patterns, extend_pre
matched += extended_matched
matched.sort()
-def cleanup_one_header(header_path, patterns, dry_run):
+def cleanup_one_header(header_path, patterns, options):
"""Clean regex-matched lines away from a file.
Arguments:
header_path: path to the cleaned file.
patterns: list of regex patterns. Any lines matching to these
patterns are deleted.
- dry_run: make no changes, but still display log.
+ options: option flags.
"""
with open(header_path) as f:
lines = f.readlines()
@@ -436,21 +442,21 @@ def cleanup_one_header(header_path, patterns, dry_run):
for i in reversed(matched):
tolines.pop(i)
- show_diff(lines, tolines, header_path)
+ show_diff(lines, tolines, header_path, options.color)
- if dry_run:
+ if options.dry_run:
return
with open(header_path, 'w') as f:
for line in tolines:
f.write(line)
-def cleanup_headers(configs, dry_run):
+def cleanup_headers(configs, options):
"""Delete config defines from board headers.
Arguments:
configs: A list of CONFIGs to remove.
- dry_run: make no changes, but still display log.
+ options: option flags.
"""
while True:
choice = raw_input('Clean up headers? [y/n]: ').lower()
@@ -473,7 +479,7 @@ def cleanup_headers(configs, dry_run):
for filename in filenames:
if not fnmatch.fnmatch(filename, '*~'):
cleanup_one_header(os.path.join(dirpath, filename),
- patterns, dry_run)
+ patterns, options)
### classes ###
class Progress:
@@ -1145,7 +1151,7 @@ def main():
move_config(configs, options)
if configs:
- cleanup_headers(configs, options.dry_run)
+ cleanup_headers(configs, options)
if __name__ == '__main__':
main()
--
1.9.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 6/8] tools: moveconfig: fix cleanup of defines across multiple lines
2016-07-25 10:15 [U-Boot] [PATCH v2 0/8] tools: moveconfig: a collection of improvement of garbage cleaning Masahiro Yamada
` (4 preceding siblings ...)
2016-07-25 10:15 ` [U-Boot] [PATCH v2 5/8] tools: moveconfig: show diffs of cleaned headers in color Masahiro Yamada
@ 2016-07-25 10:15 ` Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 7/8] tools: moveconfig: make getting all defconfigs into helper function Masahiro Yamada
2016-07-25 10:15 ` [U-Boot] [PATCH v2 8/8] tools: moveconfig: support CONFIG_SYS_EXTRA_OPTIONS cleaning Masahiro Yamada
7 siblings, 2 replies; 23+ messages in thread
From: Masahiro Yamada @ 2016-07-25 10:15 UTC (permalink / raw)
To: u-boot
Correct the clean-up of such defines that continue across multiple
lines, like follows:
#define CONFIG_FOO "this continues to the next line " \
"this line should be removed too" \
"this line should be removed as well"
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v2: None
tools/moveconfig.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 4edcb6c..5b1fa92 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -408,6 +408,9 @@ def cleanup_one_header(header_path, patterns, options):
matched = []
for i, line in enumerate(lines):
+ if i - 1 in matched and lines[i - 1][-2:] == '\\\n':
+ matched.append(i)
+ continue
for pattern in patterns:
if pattern.search(line):
matched.append(i)
--
1.9.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 7/8] tools: moveconfig: make getting all defconfigs into helper function
2016-07-25 10:15 [U-Boot] [PATCH v2 0/8] tools: moveconfig: a collection of improvement of garbage cleaning Masahiro Yamada
` (5 preceding siblings ...)
2016-07-25 10:15 ` [U-Boot] [PATCH v2 6/8] tools: moveconfig: fix cleanup of defines across multiple lines Masahiro Yamada
@ 2016-07-25 10:15 ` Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 8/8] tools: moveconfig: support CONFIG_SYS_EXTRA_OPTIONS cleaning Masahiro Yamada
7 siblings, 2 replies; 23+ messages in thread
From: Masahiro Yamada @ 2016-07-25 10:15 UTC (permalink / raw)
To: u-boot
I want to reuse this routine in the next commit.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v2:
- Newly added
tools/moveconfig.py | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 5b1fa92..3b7499b 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -266,6 +266,16 @@ def get_make_cmd():
sys.exit('GNU Make not found')
return ret[0].rstrip()
+def get_all_defconfigs():
+ """Get all the defconfig files under the configs/ directory."""
+ defconfigs = []
+ for (dirpath, dirnames, filenames) in os.walk('configs'):
+ dirpath = dirpath[len('configs') + 1:]
+ for filename in fnmatch.filter(filenames, '*_defconfig'):
+ defconfigs.append(os.path.join(dirpath, filename))
+
+ return defconfigs
+
def color_text(color_enabled, color, string):
"""Return colored string."""
if color_enabled:
@@ -1079,12 +1089,7 @@ def move_config(configs, options):
sys.exit('%s - defconfig does not exist. Stopping.' %
defconfigs[i])
else:
- # All the defconfig files to be processed
- defconfigs = []
- for (dirpath, dirnames, filenames) in os.walk('configs'):
- dirpath = dirpath[len('configs') + 1:]
- for filename in fnmatch.filter(filenames, '*_defconfig'):
- defconfigs.append(os.path.join(dirpath, filename))
+ defconfigs = get_all_defconfigs()
progress = Progress(len(defconfigs))
slots = Slots(configs, options, progress, reference_src_dir)
--
1.9.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 8/8] tools: moveconfig: support CONFIG_SYS_EXTRA_OPTIONS cleaning
2016-07-25 10:15 [U-Boot] [PATCH v2 0/8] tools: moveconfig: a collection of improvement of garbage cleaning Masahiro Yamada
` (6 preceding siblings ...)
2016-07-25 10:15 ` [U-Boot] [PATCH v2 7/8] tools: moveconfig: make getting all defconfigs into helper function Masahiro Yamada
@ 2016-07-25 10:15 ` Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
7 siblings, 2 replies; 23+ messages in thread
From: Masahiro Yamada @ 2016-07-25 10:15 UTC (permalink / raw)
To: u-boot
We mostly move config options from board header files to Kconfig,
but sometimes config defines come from CONFIG_SYS_EXTRA_OPTIONS.
Historically, CONFIG_SYS_EXTRA_OPTIONS originates in boards.cfg,
which was used as a central database of configuration prior to the
Kconfig conversion.
Now, we want to migrate to primary entries in Kconfig rather than
option list in CONFIG_SYS_EXTRA_OPTIONS, so it should be helpful to
have the tool to cleanup CONFIG_SYS_EXTRA_OPTIONS automatically.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v2:
- Make diffing into a helper function
tools/moveconfig.py | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 74 insertions(+)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 3b7499b..e7b245c 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -494,6 +494,79 @@ def cleanup_headers(configs, options):
cleanup_one_header(os.path.join(dirpath, filename),
patterns, options)
+def cleanup_one_extra_option(defconfig_path, configs, options):
+ """Delete config defines in CONFIG_SYS_EXTRA_OPTIONS in one defconfig file.
+
+ Arguments:
+ defconfig_path: path to the cleaned defconfig file.
+ configs: A list of CONFIGs to remove.
+ options: option flags.
+ """
+
+ start = 'CONFIG_SYS_EXTRA_OPTIONS="'
+ end = '"\n'
+
+ with open(defconfig_path) as f:
+ lines = f.readlines()
+
+ for i, line in enumerate(lines):
+ if line.startswith(start) and line.endswith(end):
+ break
+ else:
+ # CONFIG_SYS_EXTRA_OPTIONS was not found in this defconfig
+ return
+
+ old_tokens = line[len(start):-len(end)].split(',')
+ new_tokens = []
+
+ for token in old_tokens:
+ pos = token.find('=')
+ if not (token[:pos] if pos >= 0 else token) in configs:
+ new_tokens.append(token)
+
+ if new_tokens == old_tokens:
+ return
+
+ tolines = copy.copy(lines)
+
+ if new_tokens:
+ tolines[i] = start + ','.join(new_tokens) + end
+ else:
+ tolines.pop(i)
+
+ show_diff(lines, tolines, defconfig_path, options.color)
+
+ if options.dry_run:
+ return
+
+ with open(defconfig_path, 'w') as f:
+ for line in tolines:
+ f.write(line)
+
+def cleanup_extra_options(configs, options):
+ """Delete config defines in CONFIG_SYS_EXTRA_OPTIONS in defconfig files.
+
+ Arguments:
+ configs: A list of CONFIGs to remove.
+ options: option flags.
+ """
+ while True:
+ choice = raw_input('Clean up CONFIG_SYS_EXTRA_OPTIONS? [y/n]: ').lower()
+ print choice
+ if choice == 'y' or choice == 'n':
+ break
+
+ if choice == 'n':
+ return
+
+ configs = [ config[len('CONFIG_'):] for config in configs ]
+
+ defconfigs = get_all_defconfigs()
+
+ for defconfig in defconfigs:
+ cleanup_one_extra_option(os.path.join('configs', defconfig), configs,
+ options)
+
### classes ###
class Progress:
@@ -1160,6 +1233,7 @@ def main():
if configs:
cleanup_headers(configs, options)
+ cleanup_extra_options(configs, options)
if __name__ == '__main__':
main()
--
1.9.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 1/8] tools: moveconfig: do not cleanup headers in include/generated
2016-07-25 10:15 ` [U-Boot] [PATCH v2 1/8] tools: moveconfig: do not cleanup headers in include/generated Masahiro Yamada
@ 2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
1 sibling, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-07-25 13:37 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:22PM +0900, Masahiro Yamada wrote:
> The files in include/generated are generated during build and removed
> by "make mrproper", so it has no point to touch them by this tool.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160725/2c17c4aa/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 2/8] tools: moveconfig: do not check clean tree and compilers for -H option
2016-07-25 10:15 ` [U-Boot] [PATCH v2 2/8] tools: moveconfig: do not check clean tree and compilers for -H option Masahiro Yamada
@ 2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
1 sibling, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-07-25 13:37 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:23PM +0900, Masahiro Yamada wrote:
> The clean tree (make mrproper) and compilers are required when moving
> config options, but not needed when we only cleanup headers.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160725/a6260156/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 6/8] tools: moveconfig: fix cleanup of defines across multiple lines
2016-07-25 10:15 ` [U-Boot] [PATCH v2 6/8] tools: moveconfig: fix cleanup of defines across multiple lines Masahiro Yamada
@ 2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
1 sibling, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-07-25 13:37 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:27PM +0900, Masahiro Yamada wrote:
> Correct the clean-up of such defines that continue across multiple
> lines, like follows:
>
> #define CONFIG_FOO "this continues to the next line " \
> "this line should be removed too" \
> "this line should be removed as well"
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160725/442886a7/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 8/8] tools: moveconfig: support CONFIG_SYS_EXTRA_OPTIONS cleaning
2016-07-25 10:15 ` [U-Boot] [PATCH v2 8/8] tools: moveconfig: support CONFIG_SYS_EXTRA_OPTIONS cleaning Masahiro Yamada
@ 2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
1 sibling, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-07-25 13:37 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:29PM +0900, Masahiro Yamada wrote:
> We mostly move config options from board header files to Kconfig,
> but sometimes config defines come from CONFIG_SYS_EXTRA_OPTIONS.
>
> Historically, CONFIG_SYS_EXTRA_OPTIONS originates in boards.cfg,
> which was used as a central database of configuration prior to the
> Kconfig conversion.
>
> Now, we want to migrate to primary entries in Kconfig rather than
> option list in CONFIG_SYS_EXTRA_OPTIONS, so it should be helpful to
> have the tool to cleanup CONFIG_SYS_EXTRA_OPTIONS automatically.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160725/411ed7bf/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 5/8] tools: moveconfig: show diffs of cleaned headers in color
2016-07-25 10:15 ` [U-Boot] [PATCH v2 5/8] tools: moveconfig: show diffs of cleaned headers in color Masahiro Yamada
@ 2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
1 sibling, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-07-25 13:37 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:26PM +0900, Masahiro Yamada wrote:
> Show code diff in color if --color option is given.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160725/f8438d42/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [PATCH v2 7/8] tools: moveconfig: make getting all defconfigs into helper function
2016-07-25 10:15 ` [U-Boot] [PATCH v2 7/8] tools: moveconfig: make getting all defconfigs into helper function Masahiro Yamada
@ 2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
1 sibling, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-07-25 13:37 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:28PM +0900, Masahiro Yamada wrote:
> I want to reuse this routine in the next commit.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160725/464352e2/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [U-Boot, v2, 1/8] tools: moveconfig: do not cleanup headers in include/generated
2016-07-25 10:15 ` [U-Boot] [PATCH v2 1/8] tools: moveconfig: do not cleanup headers in include/generated Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
@ 2016-08-06 0:58 ` Tom Rini
1 sibling, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-08-06 0:58 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:22PM +0900, Masahiro Yamada wrote:
> The files in include/generated are generated during build and removed
> by "make mrproper", so it has no point to touch them by this tool.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> Reviewed-by: Tom Rini <trini@konsulko.com>
Applied to u-boot/master, thanks!
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160805/5c703587/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [U-Boot, v2, 2/8] tools: moveconfig: do not check clean tree and compilers for -H option
2016-07-25 10:15 ` [U-Boot] [PATCH v2 2/8] tools: moveconfig: do not check clean tree and compilers for -H option Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
@ 2016-08-06 0:58 ` Tom Rini
1 sibling, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-08-06 0:58 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:23PM +0900, Masahiro Yamada wrote:
> The clean tree (make mrproper) and compilers are required when moving
> config options, but not needed when we only cleanup headers.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> Reviewed-by: Tom Rini <trini@konsulko.com>
Applied to u-boot/master, thanks!
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160805/f38eea7a/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [U-Boot, v2, 3/8] tools: moveconfig: trim garbage lines after header cleanups
2016-07-25 10:15 ` [U-Boot] [PATCH v2 3/8] tools: moveconfig: trim garbage lines after header cleanups Masahiro Yamada
@ 2016-08-06 0:58 ` Tom Rini
0 siblings, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-08-06 0:58 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:24PM +0900, Masahiro Yamada wrote:
> The tools/moveconfig.py has a feature to cleanup #define/#undef's
> of moved config options, but I want this tool to do a better job.
>
> For example, when we are moving CONFIG_FOO and its define is
> surrounded by #ifdef ... #endif, like follows:
>
> #ifdef CONFIG_BAR
> # define CONFIG_FOO
> #endif
>
> The header cleanup will leave empty #ifdef ... #endif:
>
> #ifdef CONFIG_BAR
> #endif
>
> Likewise, if a define line between two blank lines
>
> <blank line>
> #define CONFIG_FOO
> <blank lines.
>
> ... is deleted, the result of the clean-up will be successive empty
> lines, which is a coding-style violation.
>
> It is tedious to remove left-over garbage lines manually, so I want
> the tool to take care of this. The tool's job is still not perfect,
> so we should check the output of the tool, but I hope our life will
> be much easier with this patch.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Applied to u-boot/master, thanks!
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160805/85b886cd/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [U-Boot, v2, 4/8] tools: moveconfig: show result of header cleaning in unified diff
2016-07-25 10:15 ` [U-Boot] [PATCH v2 4/8] tools: moveconfig: show result of header cleaning in unified diff Masahiro Yamada
@ 2016-08-06 0:58 ` Tom Rini
0 siblings, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-08-06 0:58 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:25PM +0900, Masahiro Yamada wrote:
> The header cleanup feature of this tool now removes empty ifdef's,
> successive blank lines as well as moved option defines. So, we
> want to see a little more context to check which lines were deleted.
>
> It is true that we can see it by "git diff", but it would not work
> in the --dry-run mode. So, here, this commit.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Applied to u-boot/master, thanks!
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160805/395e702a/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [U-Boot, v2, 5/8] tools: moveconfig: show diffs of cleaned headers in color
2016-07-25 10:15 ` [U-Boot] [PATCH v2 5/8] tools: moveconfig: show diffs of cleaned headers in color Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
@ 2016-08-06 0:58 ` Tom Rini
1 sibling, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-08-06 0:58 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:26PM +0900, Masahiro Yamada wrote:
> Show code diff in color if --color option is given.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> Reviewed-by: Tom Rini <trini@konsulko.com>
Applied to u-boot/master, thanks!
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160805/8a30803f/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [U-Boot, v2, 6/8] tools: moveconfig: fix cleanup of defines across multiple lines
2016-07-25 10:15 ` [U-Boot] [PATCH v2 6/8] tools: moveconfig: fix cleanup of defines across multiple lines Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
@ 2016-08-06 0:58 ` Tom Rini
1 sibling, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-08-06 0:58 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:27PM +0900, Masahiro Yamada wrote:
> Correct the clean-up of such defines that continue across multiple
> lines, like follows:
>
> #define CONFIG_FOO "this continues to the next line " \
> "this line should be removed too" \
> "this line should be removed as well"
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> Reviewed-by: Tom Rini <trini@konsulko.com>
Applied to u-boot/master, thanks!
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160805/e04b5c04/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [U-Boot, v2, 7/8] tools: moveconfig: make getting all defconfigs into helper function
2016-07-25 10:15 ` [U-Boot] [PATCH v2 7/8] tools: moveconfig: make getting all defconfigs into helper function Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
@ 2016-08-06 0:58 ` Tom Rini
1 sibling, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-08-06 0:58 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:28PM +0900, Masahiro Yamada wrote:
> I want to reuse this routine in the next commit.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> Reviewed-by: Tom Rini <trini@konsulko.com>
Applied to u-boot/master, thanks!
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160805/5081c5ec/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
* [U-Boot] [U-Boot, v2, 8/8] tools: moveconfig: support CONFIG_SYS_EXTRA_OPTIONS cleaning
2016-07-25 10:15 ` [U-Boot] [PATCH v2 8/8] tools: moveconfig: support CONFIG_SYS_EXTRA_OPTIONS cleaning Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
@ 2016-08-06 0:58 ` Tom Rini
1 sibling, 0 replies; 23+ messages in thread
From: Tom Rini @ 2016-08-06 0:58 UTC (permalink / raw)
To: u-boot
On Mon, Jul 25, 2016 at 07:15:29PM +0900, Masahiro Yamada wrote:
> We mostly move config options from board header files to Kconfig,
> but sometimes config defines come from CONFIG_SYS_EXTRA_OPTIONS.
>
> Historically, CONFIG_SYS_EXTRA_OPTIONS originates in boards.cfg,
> which was used as a central database of configuration prior to the
> Kconfig conversion.
>
> Now, we want to migrate to primary entries in Kconfig rather than
> option list in CONFIG_SYS_EXTRA_OPTIONS, so it should be helpful to
> have the tool to cleanup CONFIG_SYS_EXTRA_OPTIONS automatically.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> Reviewed-by: Tom Rini <trini@konsulko.com>
Applied to u-boot/master, thanks!
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20160805/114dab18/attachment.sig>
^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2016-08-06 0:58 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-07-25 10:15 [U-Boot] [PATCH v2 0/8] tools: moveconfig: a collection of improvement of garbage cleaning Masahiro Yamada
2016-07-25 10:15 ` [U-Boot] [PATCH v2 1/8] tools: moveconfig: do not cleanup headers in include/generated Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 2/8] tools: moveconfig: do not check clean tree and compilers for -H option Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 3/8] tools: moveconfig: trim garbage lines after header cleanups Masahiro Yamada
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 4/8] tools: moveconfig: show result of header cleaning in unified diff Masahiro Yamada
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 5/8] tools: moveconfig: show diffs of cleaned headers in color Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 6/8] tools: moveconfig: fix cleanup of defines across multiple lines Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 7/8] tools: moveconfig: make getting all defconfigs into helper function Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
2016-07-25 10:15 ` [U-Boot] [PATCH v2 8/8] tools: moveconfig: support CONFIG_SYS_EXTRA_OPTIONS cleaning Masahiro Yamada
2016-07-25 13:37 ` Tom Rini
2016-08-06 0:58 ` [U-Boot] [U-Boot, v2, " Tom Rini
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.