* [U-Boot] [PATCH 00/21] patman: Update to support Python 3
@ 2019-05-14 21:53 Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 01/21] patman: Update cros_subprocess to use bytearray Simon Glass
` (21 more replies)
0 siblings, 22 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
This series updates patman to support Python 3:
- Avoid using 'unicode' type directly
- Use items() instead of iteritems()
- Make sure file I/O uses binary mode where necessary
- Change print statements to functions
- Use the built-in set() class
- Fix up generation of repeated bytes
A few patches for binman are included, but this still requires Python 2.
Simon Glass (21):
patman: Update cros_subprocess to use bytearray
patman: Convert print statements to Python 3
binman: Convert print statements to Python 3
binman: Don't show errors for failed tests
binman: Remove use of Set()
patman: Use items() instead of iteritems()
binman: Use items() instead of iteritems()
tools: binman: Open all binary files in binary mode
tools: dtoc: Open all binary files in binary mode
patman: Provide a way to get program output in binary mode
binman: Use binary mode when compressing data
binman: Drop an unused input file
binman: Handle repeated bytes for Python 3
Add a simple script to remove boards
patman: Support use of stringIO in Python 3
patman: Move unicode helpers to tools
patman: Sort series output for repeatabily
patman: Avoid unicode type in settings unit tests
patman: Adjust functional tests for Python 3
patman: Tidy up a few more unicode conversions
patman: Don't require Python 2
tools/binman/binman.py | 26 +++-
tools/binman/bsection.py | 7 +-
tools/binman/control.py | 8 +-
tools/binman/elf.py | 4 +-
tools/binman/elf_test.py | 2 +-
tools/binman/entry.py | 5 +-
tools/binman/etype/blob.py | 2 +-
tools/binman/etype/fill.py | 4 +-
tools/binman/etype/gbb.py | 2 +-
tools/binman/etype/u_boot_spl_bss_pad.py | 2 +-
tools/binman/ftest.py | 81 ++++++------
tools/binman/state.py | 7 +-
tools/dtoc/fdt.py | 2 +-
tools/patman/cros_subprocess.py | 53 +++++---
tools/patman/func_test.py | 41 ++++---
tools/patman/gitutil.py | 16 +--
tools/patman/patman.py | 2 +-
tools/patman/series.py | 20 +--
tools/patman/settings.py | 34 ++---
tools/patman/test_util.py | 16 +--
tools/patman/tools.py | 55 ++++++++-
tools/rmboard.py | 150 +++++++++++++++++++++++
22 files changed, 385 insertions(+), 154 deletions(-)
create mode 100755 tools/rmboard.py
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 01/21] patman: Update cros_subprocess to use bytearray
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 02/21] patman: Convert print statements to Python 3 Simon Glass
` (20 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
At present this function uses lists and strings. This does not work so
well with Python 3, and testing against '' does not work for a bytearray.
Update the code to fix these issues.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/patman/cros_subprocess.py | 48 ++++++++++++++++++++-------------
tools/patman/gitutil.py | 2 ++
2 files changed, 32 insertions(+), 18 deletions(-)
diff --git a/tools/patman/cros_subprocess.py b/tools/patman/cros_subprocess.py
index ebd4300dfd..4de230ac63 100644
--- a/tools/patman/cros_subprocess.py
+++ b/tools/patman/cros_subprocess.py
@@ -100,6 +100,24 @@ class Popen(subprocess.Popen):
if kwargs:
raise ValueError("Unit tests do not test extra args - please add tests")
+ def ConvertData(self, data):
+ """Convert stdout/stderr data to the correct format for output
+
+ Args:
+ data: Data to convert, or None for ''
+
+ Returns:
+ Converted data to a unicode string
+ """
+ if data is None:
+ return ''
+ else:
+ try:
+ data = data.decode('utf-8')
+ except UnicodeDecodeError:
+ data = data.decode('utf-8', 'ignore')
+ return data
+
def CommunicateFilter(self, output):
"""Interact with process: Read data from stdout and stderr.
@@ -156,11 +174,11 @@ class Popen(subprocess.Popen):
self.stdin.close()
if self.stdout:
read_set.append(self.stdout)
- stdout = []
+ stdout = bytearray()
if self.stderr and self.stderr != self.stdout:
read_set.append(self.stderr)
- stderr = []
- combined = []
+ stderr = bytearray()
+ combined = bytearray()
input_offset = 0
while read_set or write_set:
@@ -192,12 +210,12 @@ class Popen(subprocess.Popen):
data = os.read(self.stdout.fileno(), 1024)
except OSError:
pass
- if data == "":
+ if not len(data):
self.stdout.close()
read_set.remove(self.stdout)
else:
- stdout.append(data)
- combined.append(data)
+ stdout += data
+ combined += data
if output:
output(sys.stdout, data)
if self.stderr in rlist:
@@ -207,25 +225,19 @@ class Popen(subprocess.Popen):
data = os.read(self.stderr.fileno(), 1024)
except OSError:
pass
- if data == "":
+ if not len(data):
self.stderr.close()
read_set.remove(self.stderr)
else:
- stderr.append(data)
- combined.append(data)
+ stderr += data
+ combined += data
if output:
output(sys.stderr, data)
# All data exchanged. Translate lists into strings.
- if stdout is not None:
- stdout = ''.join(stdout)
- else:
- stdout = ''
- if stderr is not None:
- stderr = ''.join(stderr)
- else:
- stderr = ''
- combined = ''.join(combined)
+ stdout = self.ConvertData(stdout)
+ stderr = self.ConvertData(stderr)
+ combined = self.ConvertData(combined)
# Translate newlines, if requested. We cannot let the file
# object do the translation: It is based on stdio, which is
diff --git a/tools/patman/gitutil.py b/tools/patman/gitutil.py
index 9905bb0bbd..7650b51bd5 100644
--- a/tools/patman/gitutil.py
+++ b/tools/patman/gitutil.py
@@ -326,6 +326,8 @@ def BuildEmailList(in_list, tag=None, alias=None, raise_on_error=True):
result = []
for item in raw:
if not item in result:
+ if type(item) == unicode:
+ item = item.encode('utf-8')
result.append(item)
if tag:
return ['%s %s%s%s' % (tag, quote, email, quote) for email in result]
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 02/21] patman: Convert print statements to Python 3
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 01/21] patman: Update cros_subprocess to use bytearray Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 03/21] binman: " Simon Glass
` (19 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
Update all print statements to be functions, as required by Python 3.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/patman/func_test.py | 3 ---
tools/patman/gitutil.py | 10 +++++-----
tools/patman/test_util.py | 16 +++++++++-------
3 files changed, 14 insertions(+), 15 deletions(-)
diff --git a/tools/patman/func_test.py b/tools/patman/func_test.py
index d79e716074..31481157fc 100644
--- a/tools/patman/func_test.py
+++ b/tools/patman/func_test.py
@@ -159,7 +159,6 @@ class TestFunctional(unittest.TestCase):
os.remove(cc_file)
lines = out[0].splitlines()
- #print '\n'.join(lines)
self.assertEqual('Cleaned %s patches' % len(series.commits), lines[0])
self.assertEqual('Change log missing for v2', lines[1])
self.assertEqual('Change log missing for v3', lines[2])
@@ -223,7 +222,6 @@ Simon Glass (2):
'''
lines = open(cover_fname).read().splitlines()
- #print '\n'.join(lines)
self.assertEqual(
'Subject: [RFC PATCH v3 0/2] test: A test patch series',
lines[3])
@@ -231,7 +229,6 @@ Simon Glass (2):
for i, fname in enumerate(args):
lines = open(fname).read().splitlines()
- #print '\n'.join(lines)
subject = [line for line in lines if line.startswith('Subject')]
self.assertEqual('Subject: [RFC %d/%d]' % (i + 1, count),
subject[0][:18])
diff --git a/tools/patman/gitutil.py b/tools/patman/gitutil.py
index 7650b51bd5..11aeb73b74 100644
--- a/tools/patman/gitutil.py
+++ b/tools/patman/gitutil.py
@@ -397,11 +397,11 @@ def EmailPatches(series, cover_fname, args, dry_run, raise_on_error, cc_fname,
git_config_to = command.Output('git', 'config', 'sendemail.to',
raise_on_error=False)
if not git_config_to:
- print ("No recipient.\n"
- "Please add something like this to a commit\n"
- "Series-to: Fred Bloggs <f.blogs@napier.co.nz>\n"
- "Or do something like this\n"
- "git config sendemail.to u-boot at lists.denx.de")
+ print("No recipient.\n"
+ "Please add something like this to a commit\n"
+ "Series-to: Fred Bloggs <f.blogs@napier.co.nz>\n"
+ "Or do something like this\n"
+ "git config sendemail.to u-boot at lists.denx.de")
return
cc = BuildEmailList(list(set(series.get('cc')) - set(series.get('to'))),
'--cc', alias, raise_on_error)
diff --git a/tools/patman/test_util.py b/tools/patman/test_util.py
index 687d40704a..e462ec8f72 100644
--- a/tools/patman/test_util.py
+++ b/tools/patman/test_util.py
@@ -3,6 +3,8 @@
# Copyright (c) 2016 Google, Inc
#
+from __future__ import print_function
+
from contextlib import contextmanager
import glob
import os
@@ -54,18 +56,18 @@ def RunTestCoverage(prog, filter_fname, exclude_list, build_dir, required=None):
missing_list = required
missing_list.difference_update(test_set)
if missing_list:
- print 'Missing tests for %s' % (', '.join(missing_list))
- print stdout
+ print('Missing tests for %s' % (', '.join(missing_list)))
+ print(stdout)
ok = False
coverage = lines[-1].split(' ')[-1]
ok = True
- print coverage
+ print(coverage)
if coverage != '100%':
- print stdout
- print ("Type 'python-coverage html' to get a report in "
- 'htmlcov/index.html')
- print 'Coverage error: %s, but should be 100%%' % coverage
+ print(stdout)
+ print("Type 'python-coverage html' to get a report in "
+ 'htmlcov/index.html')
+ print('Coverage error: %s, but should be 100%%' % coverage)
ok = False
if not ok:
raise ValueError('Test coverage failure')
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 03/21] binman: Convert print statements to Python 3
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 01/21] patman: Update cros_subprocess to use bytearray Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 02/21] patman: Convert print statements to Python 3 Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 04/21] binman: Don't show errors for failed tests Simon Glass
` (18 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
Update all print statements to be functions, as required by Python 3.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/binman/binman.py | 14 ++++++++------
tools/binman/control.py | 6 ++++--
2 files changed, 12 insertions(+), 8 deletions(-)
diff --git a/tools/binman/binman.py b/tools/binman/binman.py
index 439908e665..4206d2b0e3 100755
--- a/tools/binman/binman.py
+++ b/tools/binman/binman.py
@@ -9,6 +9,8 @@
"""See README for more information"""
+from __future__ import print_function
+
import glob
import multiprocessing
import os
@@ -85,13 +87,13 @@ def RunTests(debug, processes, args):
else:
suite.run(result)
- print result
+ print(result)
for test, err in result.errors:
- print test.id(), err
+ print(test.id(), err)
for test, err in result.failures:
- print err, result.failures
+ print(err, result.failures)
if result.errors or result.failures:
- print 'binman tests FAILED'
+ print('binman tests FAILED')
return 1
return 0
@@ -143,9 +145,9 @@ def RunBinman(options, args):
try:
ret_code = control.Binman(options, args)
except Exception as e:
- print 'binman: %s' % e
+ print('binman: %s' % e)
if options.debug:
- print
+ print()
traceback.print_exc()
ret_code = 1
return ret_code
diff --git a/tools/binman/control.py b/tools/binman/control.py
index b32e4e1996..8f7f906860 100644
--- a/tools/binman/control.py
+++ b/tools/binman/control.py
@@ -5,6 +5,8 @@
# Creates binary images from input files controlled by a description
#
+from __future__ import print_function
+
from collections import OrderedDict
import os
import sys
@@ -134,7 +136,7 @@ def Binman(options, args):
del images[name]
skip.append(name)
if skip and options.verbosity >= 2:
- print 'Skipping images: %s' % ', '.join(skip)
+ print('Skipping images: %s' % ', '.join(skip))
state.Prepare(images, dtb)
@@ -170,7 +172,7 @@ def Binman(options, args):
except Exception as e:
if options.map:
fname = image.WriteMap()
- print "Wrote map file '%s' to show errors" % fname
+ print("Wrote map file '%s' to show errors" % fname)
raise
image.SetImagePos()
if options.update_fdt:
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 04/21] binman: Don't show errors for failed tests
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (2 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 03/21] binman: " Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 05/21] binman: Remove use of Set() Simon Glass
` (17 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
An unfortunate new feature in Python 3.5 causes binman to print errors for
non-existent tests, when the test filter is used. Work around this by
detecting the unwanted tests and removing them from the result.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/binman/binman.py | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/tools/binman/binman.py b/tools/binman/binman.py
index 4206d2b0e3..aad2e9c8bc 100755
--- a/tools/binman/binman.py
+++ b/tools/binman/binman.py
@@ -87,6 +87,18 @@ def RunTests(debug, processes, args):
else:
suite.run(result)
+ # Remove errors which just indicate a missing test. Since Python v3.5 If an
+ # ImportError or AttributeError occurs while traversing name then a
+ # synthetic test that raises that error when run will be returned. These
+ # errors are included in the errors accumulated by result.errors.
+ if test_name:
+ errors = []
+ for test, err in result.errors:
+ if ("has no attribute '%s'" % test_name) not in err:
+ errors.append((test, err))
+ result.testsRun -= 1
+ result.errors = errors
+
print(result)
for test, err in result.errors:
print(test.id(), err)
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 05/21] binman: Remove use of Set()
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (3 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 04/21] binman: Don't show errors for failed tests Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 06/21] patman: Use items() instead of iteritems() Simon Glass
` (16 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
A new built-in set() is used in both Python 2 and 3 now. Move it to use
that instead of Set().
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/binman/bsection.py | 3 +--
tools/binman/entry.py | 5 ++---
tools/binman/state.py | 7 +++----
3 files changed, 6 insertions(+), 9 deletions(-)
diff --git a/tools/binman/bsection.py b/tools/binman/bsection.py
index 0ba542ee98..be67ff957e 100644
--- a/tools/binman/bsection.py
+++ b/tools/binman/bsection.py
@@ -8,7 +8,6 @@
from __future__ import print_function
from collections import OrderedDict
-from sets import Set
import sys
import fdt_util
@@ -109,7 +108,7 @@ class Section(object):
def GetFdtSet(self):
"""Get the set of device tree files used by this image"""
- fdt_set = Set()
+ fdt_set = set()
for entry in self._entries.values():
fdt_set.update(entry.GetFdtSet())
return fdt_set
diff --git a/tools/binman/entry.py b/tools/binman/entry.py
index 648cfd241f..d842d89dd6 100644
--- a/tools/binman/entry.py
+++ b/tools/binman/entry.py
@@ -18,7 +18,6 @@ except:
have_importlib = False
import os
-from sets import Set
import sys
import fdt_util
@@ -178,8 +177,8 @@ class Entry(object):
# It would be better to use isinstance(self, Entry_blob_dtb) here but
# we cannot access Entry_blob_dtb
if fname and fname.endswith('.dtb'):
- return Set([fname])
- return Set()
+ return set([fname])
+ return set()
def ExpandEntries(self):
pass
diff --git a/tools/binman/state.py b/tools/binman/state.py
index d945e4bf65..af9678649c 100644
--- a/tools/binman/state.py
+++ b/tools/binman/state.py
@@ -7,7 +7,6 @@
import hashlib
import re
-from sets import Set
import os
import tools
@@ -24,10 +23,10 @@ entry_args = {}
use_fake_dtb = False
# Set of all device tree files references by images
-fdt_set = Set()
+fdt_set = set()
# Same as above, but excluding the main one
-fdt_subset = Set()
+fdt_subset = set()
# The DTB which contains the full image information
main_dtb = None
@@ -136,7 +135,7 @@ def Prepare(images, dtb):
main_dtb = dtb
fdt_files.clear()
fdt_files['u-boot.dtb'] = dtb
- fdt_subset = Set()
+ fdt_subset = set()
if not use_fake_dtb:
for image in images.values():
fdt_subset.update(image.GetFdtSet())
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 06/21] patman: Use items() instead of iteritems()
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (4 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 05/21] binman: Remove use of Set() Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 07/21] binman: " Simon Glass
` (15 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
Python 3 requires this, and Python 2 allows it. Convert the code over to
ensure compatibility with Python 3.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/patman/settings.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/patman/settings.py b/tools/patman/settings.py
index ea2bc74f75..07bf6a6ea4 100644
--- a/tools/patman/settings.py
+++ b/tools/patman/settings.py
@@ -163,7 +163,7 @@ class _ProjectConfigParser(ConfigParser.SafeConfigParser):
item_dict = dict(top_items)
item_dict.update(project_items)
return {(self._to_unicode(item), self._to_unicode(val))
- for item, val in item_dict.iteritems()}
+ for item, val in item_dict.items()}
def ReadGitAliases(fname):
"""Read a git alias file. This is in the form used by git:
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 07/21] binman: Use items() instead of iteritems()
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (5 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 06/21] patman: Use items() instead of iteritems() Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 08/21] tools: binman: Open all binary files in binary mode Simon Glass
` (14 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
Python 3 requires this, and Python 2 allows it. Convert the code over to
ensure compatibility with Python 3.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/binman/bsection.py | 2 +-
tools/binman/control.py | 2 +-
tools/binman/elf.py | 4 ++--
tools/binman/etype/gbb.py | 2 +-
tools/binman/ftest.py | 2 +-
5 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/tools/binman/bsection.py b/tools/binman/bsection.py
index be67ff957e..3ca0592fe1 100644
--- a/tools/binman/bsection.py
+++ b/tools/binman/bsection.py
@@ -253,7 +253,7 @@ class Section(object):
"""
for entry in self._entries.values():
offset_dict = entry.GetOffsets()
- for name, info in offset_dict.iteritems():
+ for name, info in offset_dict.items():
self._SetEntryOffsetSize(name, *info)
def PackEntries(self):
diff --git a/tools/binman/control.py b/tools/binman/control.py
index 8f7f906860..ce25eb5485 100644
--- a/tools/binman/control.py
+++ b/tools/binman/control.py
@@ -131,7 +131,7 @@ def Binman(options, args):
if options.image:
skip = []
- for name, image in images.iteritems():
+ for name, image in images.items():
if name not in options.image:
del images[name]
skip.append(name)
diff --git a/tools/binman/elf.py b/tools/binman/elf.py
index 97df8e32c5..828681d76d 100644
--- a/tools/binman/elf.py
+++ b/tools/binman/elf.py
@@ -59,7 +59,7 @@ def GetSymbols(fname, patterns):
flags[1] == 'w')
# Sort dict by address
- return OrderedDict(sorted(syms.iteritems(), key=lambda x: x[1].address))
+ return OrderedDict(sorted(syms.items(), key=lambda x: x[1].address))
def GetSymbolAddress(fname, sym_name):
"""Get a value of a symbol from an ELF file
@@ -98,7 +98,7 @@ def LookupAndWriteSymbols(elf_fname, entry, section):
base = syms.get('__image_copy_start')
if not base:
return
- for name, sym in syms.iteritems():
+ for name, sym in syms.items():
if name.startswith('_binman'):
msg = ("Section '%s': Symbol '%s'\n in entry '%s'" %
(section.GetPath(), name, entry.GetPath()))
diff --git a/tools/binman/etype/gbb.py b/tools/binman/etype/gbb.py
index 8fe10f4713..a94c0fca9d 100644
--- a/tools/binman/etype/gbb.py
+++ b/tools/binman/etype/gbb.py
@@ -64,7 +64,7 @@ class Entry_gbb(Entry):
self.gbb_flags = 0
flags_node = node.FindNode('flags')
if flags_node:
- for flag, value in gbb_flag_properties.iteritems():
+ for flag, value in gbb_flag_properties.items():
if fdt_util.GetBool(flags_node, flag):
self.gbb_flags |= value
diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py
index daea1ea138..7cf17526a7 100644
--- a/tools/binman/ftest.py
+++ b/tools/binman/ftest.py
@@ -214,7 +214,7 @@ class TestFunctional(unittest.TestCase):
if verbosity is not None:
args.append('-v%d' % verbosity)
if entry_args:
- for arg, value in entry_args.iteritems():
+ for arg, value in entry_args.items():
args.append('-a%s=%s' % (arg, value))
if images:
for image in images:
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 08/21] tools: binman: Open all binary files in binary mode
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (6 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 07/21] binman: " Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 09/21] tools: dtoc: " Simon Glass
` (13 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
At present some files are opened in text mode despite containing binary
data. This works on Python 2 but not always on Python 3, due to unicode
problems. BC&D are not my favourite people. Adjust the affected open()
statements to use binary mode.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/binman/ftest.py | 30 +++++++++++++++---------------
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py
index 7cf17526a7..1111852a40 100644
--- a/tools/binman/ftest.py
+++ b/tools/binman/ftest.py
@@ -119,11 +119,11 @@ class TestFunctional(unittest.TestCase):
TestFunctional._MakeInputFile('refcode.bin', REFCODE_DATA)
# ELF file with a '_dt_ucode_base_size' symbol
- with open(self.TestFile('u_boot_ucode_ptr')) as fd:
+ with open(self.TestFile('u_boot_ucode_ptr'), 'rb') as fd:
TestFunctional._MakeInputFile('u-boot', fd.read())
# Intel flash descriptor file
- with open(self.TestFile('descriptor.bin')) as fd:
+ with open(self.TestFile('descriptor.bin'), 'rb') as fd:
TestFunctional._MakeInputFile('descriptor.bin', fd.read())
shutil.copytree(self.TestFile('files'),
@@ -236,7 +236,7 @@ class TestFunctional(unittest.TestCase):
"""
tools.PrepareOutputDir(None)
dtb = fdt_util.EnsureCompiled(self.TestFile(fname))
- with open(dtb) as fd:
+ with open(dtb, 'rb') as fd:
data = fd.read()
TestFunctional._MakeInputFile(outfile, data)
tools.FinaliseOutputDir()
@@ -317,7 +317,7 @@ class TestFunctional(unittest.TestCase):
map_data = fd.read()
else:
map_data = None
- with open(image_fname) as fd:
+ with open(image_fname, 'rb') as fd:
return fd.read(), dtb_data, map_data, out_dtb_fname
finally:
# Put the test file back
@@ -379,7 +379,7 @@ class TestFunctional(unittest.TestCase):
Args:
Filename of ELF file to use as SPL
"""
- with open(self.TestFile(src_fname)) as fd:
+ with open(self.TestFile(src_fname), 'rb') as fd:
TestFunctional._MakeInputFile('spl/u-boot-spl', fd.read())
@classmethod
@@ -541,7 +541,7 @@ class TestFunctional(unittest.TestCase):
self.assertEqual(len(U_BOOT_DATA), image._size)
fname = tools.GetOutputFilename('image1.bin')
self.assertTrue(os.path.exists(fname))
- with open(fname) as fd:
+ with open(fname, 'rb') as fd:
data = fd.read()
self.assertEqual(U_BOOT_DATA, data)
@@ -549,7 +549,7 @@ class TestFunctional(unittest.TestCase):
self.assertEqual(3 + len(U_BOOT_DATA) + 5, image._size)
fname = tools.GetOutputFilename('image2.bin')
self.assertTrue(os.path.exists(fname))
- with open(fname) as fd:
+ with open(fname, 'rb') as fd:
data = fd.read()
self.assertEqual(U_BOOT_DATA, data[3:7])
self.assertEqual(chr(0) * 3, data[:3])
@@ -970,7 +970,7 @@ class TestFunctional(unittest.TestCase):
"""Test that a U-Boot binary without the microcode symbol is detected"""
# ELF file without a '_dt_ucode_base_size' symbol
try:
- with open(self.TestFile('u_boot_no_ucode_ptr')) as fd:
+ with open(self.TestFile('u_boot_no_ucode_ptr'), 'rb') as fd:
TestFunctional._MakeInputFile('u-boot', fd.read())
with self.assertRaises(ValueError) as e:
@@ -980,7 +980,7 @@ class TestFunctional(unittest.TestCase):
finally:
# Put the original file back
- with open(self.TestFile('u_boot_ucode_ptr')) as fd:
+ with open(self.TestFile('u_boot_ucode_ptr'), 'rb') as fd:
TestFunctional._MakeInputFile('u-boot', fd.read())
def testMicrocodeNotInImage(self):
@@ -993,7 +993,7 @@ class TestFunctional(unittest.TestCase):
def testWithoutMicrocode(self):
"""Test that we can cope with an image without microcode (e.g. qemu)"""
- with open(self.TestFile('u_boot_no_ucode_ptr')) as fd:
+ with open(self.TestFile('u_boot_no_ucode_ptr'), 'rb') as fd:
TestFunctional._MakeInputFile('u-boot', fd.read())
data, dtb, _, _ = self._DoReadFileDtb('044_x86_optional_ucode.dts', True)
@@ -1357,7 +1357,7 @@ class TestFunctional(unittest.TestCase):
fname = pipe_list[0][-1]
# Append our GBB data to the file, which will happen every time the
# futility command is called.
- with open(fname, 'a') as fd:
+ with open(fname, 'ab') as fd:
fd.write(GBB_DATA)
return command.CommandResult()
@@ -1431,7 +1431,7 @@ class TestFunctional(unittest.TestCase):
def testTpl(self):
"""Test that an image with TPL and ots device tree can be created"""
# ELF file with a '__bss_size' symbol
- with open(self.TestFile('bss_data')) as fd:
+ with open(self.TestFile('bss_data'), 'rb') as fd:
TestFunctional._MakeInputFile('tpl/u-boot-tpl', fd.read())
data = self._DoReadFile('078_u_boot_tpl.dts')
self.assertEqual(U_BOOT_TPL_DATA + U_BOOT_TPL_DTB_DATA, data)
@@ -1693,7 +1693,7 @@ class TestFunctional(unittest.TestCase):
u-boot-tpl.dtb with the microcode removed
the microcode
"""
- with open(self.TestFile('u_boot_ucode_ptr')) as fd:
+ with open(self.TestFile('u_boot_ucode_ptr'), 'rb') as fd:
TestFunctional._MakeInputFile('tpl/u-boot-tpl', fd.read())
first, pos_and_size = self._RunMicrocodeTest('093_x86_tpl_ucode.dts',
U_BOOT_TPL_NODTB_DATA)
@@ -1748,14 +1748,14 @@ class TestFunctional(unittest.TestCase):
def testElf(self):
"""Basic test of ELF entries"""
self._SetupSplElf()
- with open(self.TestFile('bss_data')) as fd:
+ with open(self.TestFile('bss_data'), 'rb') as fd:
TestFunctional._MakeInputFile('-boot', fd.read())
data = self._DoReadFile('096_elf.dts')
def testElfStripg(self):
"""Basic test of ELF entries"""
self._SetupSplElf()
- with open(self.TestFile('bss_data')) as fd:
+ with open(self.TestFile('bss_data'), 'rb') as fd:
TestFunctional._MakeInputFile('-boot', fd.read())
data = self._DoReadFile('097_elf_strip.dts')
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 09/21] tools: dtoc: Open all binary files in binary mode
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (7 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 08/21] tools: binman: Open all binary files in binary mode Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 10/21] patman: Provide a way to get program output " Simon Glass
` (12 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
Fix an open() statement to use binary mode, so that it works as expected
with Python 3.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/dtoc/fdt.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/dtoc/fdt.py b/tools/dtoc/fdt.py
index 9ad72f89ec..031b3a0084 100644
--- a/tools/dtoc/fdt.py
+++ b/tools/dtoc/fdt.py
@@ -469,7 +469,7 @@ class Fdt:
if self._fname:
self._fname = fdt_util.EnsureCompiled(self._fname)
- with open(self._fname) as fd:
+ with open(self._fname, 'rb') as fd:
self._fdt_obj = libfdt.Fdt(fd.read())
@staticmethod
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 10/21] patman: Provide a way to get program output in binary mode
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (8 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 09/21] tools: dtoc: " Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 11/21] binman: Use binary mode when compressing data Simon Glass
` (11 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
At present cros_subprocess and the tools library use a string to obtain
stdout from a program. This works fine on Python 2. With Python 3 we end
up with unicode errors in some cases. Fix this by providing a binary mode,
which returns the data as bytes() instead of a string.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/patman/cros_subprocess.py | 15 +++++++++------
tools/patman/tools.py | 4 ++--
2 files changed, 11 insertions(+), 8 deletions(-)
diff --git a/tools/patman/cros_subprocess.py b/tools/patman/cros_subprocess.py
index 4de230ac63..39d4f21980 100644
--- a/tools/patman/cros_subprocess.py
+++ b/tools/patman/cros_subprocess.py
@@ -54,7 +54,7 @@ class Popen(subprocess.Popen):
"""
def __init__(self, args, stdin=None, stdout=PIPE_PTY, stderr=PIPE_PTY,
- shell=False, cwd=None, env=None, **kwargs):
+ shell=False, cwd=None, env=None, binary=False, **kwargs):
"""Cut-down constructor
Args:
@@ -72,6 +72,7 @@ class Popen(subprocess.Popen):
"""
stdout_pty = None
stderr_pty = None
+ self.binary = binary
if stdout == PIPE_PTY:
stdout_pty = pty.openpty()
@@ -107,15 +108,17 @@ class Popen(subprocess.Popen):
data: Data to convert, or None for ''
Returns:
- Converted data to a unicode string
+ Converted data, normally as a unicode string, but this uses bytes
+ in binary mode
"""
if data is None:
return ''
else:
- try:
- data = data.decode('utf-8')
- except UnicodeDecodeError:
- data = data.decode('utf-8', 'ignore')
+ if not self.binary:
+ try:
+ data = data.decode('utf-8')
+ except UnicodeDecodeError:
+ data = data.decode('utf-8', 'ignore')
return data
def CommunicateFilter(self, output):
diff --git a/tools/patman/tools.py b/tools/patman/tools.py
index bf099798e6..1df8f2ecd2 100644
--- a/tools/patman/tools.py
+++ b/tools/patman/tools.py
@@ -167,9 +167,9 @@ def PathHasFile(fname):
return True
return False
-def Run(name, *args):
+def Run(name, *args, **kwargs):
try:
- return command.Run(name, *args, cwd=outdir, capture=True)
+ return command.Run(name, *args, cwd=outdir, capture=True, **kwargs)
except:
if not PathHasFile(name):
msg = "Plesae install tool '%s'" % name
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 11/21] binman: Use binary mode when compressing data
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (9 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 10/21] patman: Provide a way to get program output " Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 12/21] binman: Drop an unused input file Simon Glass
` (10 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
The lz4 utility inserts binary data in its output which cannot always be
converted to unicode (nor should it be). Fix this by using the new binary
mode for program output.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/binman/etype/blob.py | 2 +-
tools/binman/ftest.py | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/binman/etype/blob.py b/tools/binman/etype/blob.py
index ae80bbee53..f56a1f8768 100644
--- a/tools/binman/etype/blob.py
+++ b/tools/binman/etype/blob.py
@@ -60,7 +60,7 @@ class Entry_blob(Entry):
except AttributeError:
data = lz4.compress(data)
'''
- data = tools.Run('lz4', '-c', self._pathname)
+ data = tools.Run('lz4', '-c', self._pathname, binary=True)
self.SetContents(data)
return True
diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py
index 1111852a40..4d96933cb4 100644
--- a/tools/binman/ftest.py
+++ b/tools/binman/ftest.py
@@ -1557,7 +1557,7 @@ class TestFunctional(unittest.TestCase):
out = os.path.join(self._indir, 'lz4.tmp')
with open(out, 'wb') as fd:
fd.write(data)
- return tools.Run('lz4', '-dc', out)
+ return tools.Run('lz4', '-dc', out, binary=True)
'''
try:
orig = lz4.frame.decompress(data)
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 12/21] binman: Drop an unused input file
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (10 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 11/21] binman: Use binary mode when compressing data Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 13/21] binman: Handle repeated bytes for Python 3 Simon Glass
` (9 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
Drop this line which is not used or needed.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/binman/ftest.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py
index 4d96933cb4..e1fc9e8e9e 100644
--- a/tools/binman/ftest.py
+++ b/tools/binman/ftest.py
@@ -291,7 +291,6 @@ class TestFunctional(unittest.TestCase):
# Use the compiled test file as the u-boot-dtb input
if use_real_dtb:
dtb_data = self._SetupDtb(fname)
- infile = os.path.join(self._indir, 'u-boot.dtb')
# For testing purposes, make a copy of the DT for SPL and TPL. Add
# a node indicating which it is, so aid verification.
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 13/21] binman: Handle repeated bytes for Python 3
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (11 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 12/21] binman: Drop an unused input file Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 14/21] Add a simple script to remove boards Simon Glass
` (8 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
The method of multiplying a character by a number works well for creating
a repeated string in Python 2. But in Python 3 we need to use bytes()
instead, to avoid unicode problems, since 'bytes' is no-longer just an
alias of 'str'.
Create a function to handle this detail and call it from the relevant
places in binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/binman/bsection.py | 2 +-
tools/binman/elf_test.py | 2 +-
tools/binman/etype/fill.py | 4 +--
tools/binman/etype/u_boot_spl_bss_pad.py | 2 +-
tools/binman/ftest.py | 46 +++++++++++++-----------
tools/patman/tools.py | 19 ++++++++++
6 files changed, 50 insertions(+), 25 deletions(-)
diff --git a/tools/binman/bsection.py b/tools/binman/bsection.py
index 3ca0592fe1..03dfa2f805 100644
--- a/tools/binman/bsection.py
+++ b/tools/binman/bsection.py
@@ -332,7 +332,7 @@ class Section(object):
def GetData(self):
"""Get the contents of the section"""
- section_data = chr(self._pad_byte) * self._size
+ section_data = tools.GetBytes(self._pad_byte, self._size)
for entry in self._entries.values():
data = entry.GetData()
diff --git a/tools/binman/elf_test.py b/tools/binman/elf_test.py
index b68530c19b..3d55cb1946 100644
--- a/tools/binman/elf_test.py
+++ b/tools/binman/elf_test.py
@@ -122,7 +122,7 @@ class TestElf(unittest.TestCase):
section = FakeSection(sym_value=None)
elf_fname = os.path.join(binman_dir, 'test', 'u_boot_binman_syms')
syms = elf.LookupAndWriteSymbols(elf_fname, entry, section)
- self.assertEqual(chr(255) * 16 + 'a' * 4, entry.data)
+ self.assertEqual(tools.GetBytes(255, 16) + 'a' * 4, entry.data)
def testDebug(self):
"""Check that enabling debug in the elf module produced debug output"""
diff --git a/tools/binman/etype/fill.py b/tools/binman/etype/fill.py
index dcfe978a5b..68efe42ec0 100644
--- a/tools/binman/etype/fill.py
+++ b/tools/binman/etype/fill.py
@@ -5,7 +5,7 @@
from entry import Entry
import fdt_util
-
+import tools
class Entry_fill(Entry):
"""An entry which is filled to a particular byte value
@@ -28,5 +28,5 @@ class Entry_fill(Entry):
self.fill_value = fdt_util.GetByte(self._node, 'fill-byte', 0)
def ObtainContents(self):
- self.SetContents(chr(self.fill_value) * self.size)
+ self.SetContents(tools.GetBytes(self.fill_value, self.size))
return True
diff --git a/tools/binman/etype/u_boot_spl_bss_pad.py b/tools/binman/etype/u_boot_spl_bss_pad.py
index 00b7ac5004..66a296a6f8 100644
--- a/tools/binman/etype/u_boot_spl_bss_pad.py
+++ b/tools/binman/etype/u_boot_spl_bss_pad.py
@@ -38,5 +38,5 @@ class Entry_u_boot_spl_bss_pad(Entry_blob):
bss_size = elf.GetSymbolAddress(fname, '__bss_size')
if not bss_size:
self.Raise('Expected __bss_size symbol in spl/u-boot-spl')
- self.SetContents(chr(0) * bss_size)
+ self.SetContents(tools.GetBytes(0, bss_size))
return True
diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py
index e1fc9e8e9e..971fade343 100644
--- a/tools/binman/ftest.py
+++ b/tools/binman/ftest.py
@@ -551,8 +551,8 @@ class TestFunctional(unittest.TestCase):
with open(fname, 'rb') as fd:
data = fd.read()
self.assertEqual(U_BOOT_DATA, data[3:7])
- self.assertEqual(chr(0) * 3, data[:3])
- self.assertEqual(chr(0) * 5, data[7:])
+ self.assertEqual(tools.GetBytes(0, 3), data[:3])
+ self.assertEqual(tools.GetBytes(0, 5), data[7:])
def testBadAlign(self):
"""Test that an invalid alignment value is detected"""
@@ -731,7 +731,8 @@ class TestFunctional(unittest.TestCase):
"""Test that the image pad byte can be specified"""
self._SetupSplElf()
data = self._DoReadFile('021_image_pad.dts')
- self.assertEqual(U_BOOT_SPL_DATA + (chr(0xff) * 1) + U_BOOT_DATA, data)
+ self.assertEqual(U_BOOT_SPL_DATA + tools.GetBytes(0xff, 1) +
+ U_BOOT_DATA, data)
def testImageName(self):
"""Test that image files can be named"""
@@ -754,8 +755,8 @@ class TestFunctional(unittest.TestCase):
"""Test that entries can be sorted"""
self._SetupSplElf()
data = self._DoReadFile('024_sorted.dts')
- self.assertEqual(chr(0) * 1 + U_BOOT_SPL_DATA + chr(0) * 2 +
- U_BOOT_DATA, data)
+ self.assertEqual(tools.GetBytes(0, 1) + U_BOOT_SPL_DATA +
+ tools.GetBytes(0, 2) + U_BOOT_DATA, data)
def testPackZeroOffset(self):
"""Test that an entry at offset 0 is not given a new offset"""
@@ -797,8 +798,8 @@ class TestFunctional(unittest.TestCase):
"""Test that a basic x86 ROM can be created"""
self._SetupSplElf()
data = self._DoReadFile('029_x86-rom.dts')
- self.assertEqual(U_BOOT_DATA + chr(0) * 7 + U_BOOT_SPL_DATA +
- chr(0) * 2, data)
+ self.assertEqual(U_BOOT_DATA + tools.GetBytes(0, 7) + U_BOOT_SPL_DATA +
+ tools.GetBytes(0, 2), data)
def testPackX86RomMeNoDesc(self):
"""Test that an invalid Intel descriptor entry is detected"""
@@ -1005,7 +1006,7 @@ class TestFunctional(unittest.TestCase):
used_len = len(U_BOOT_NODTB_DATA) + fdt_len
third = data[used_len:]
- self.assertEqual(chr(0) * (0x200 - used_len), third)
+ self.assertEqual(tools.GetBytes(0, 0x200 - used_len), third)
def testUnknownPosSize(self):
"""Test that microcode must be placed within the image"""
@@ -1034,7 +1035,8 @@ class TestFunctional(unittest.TestCase):
# ELF file with a '__bss_size' symbol
self._SetupSplElf()
data = self._DoReadFile('047_spl_bss_pad.dts')
- self.assertEqual(U_BOOT_SPL_DATA + (chr(0) * 10) + U_BOOT_DATA, data)
+ self.assertEqual(U_BOOT_SPL_DATA + tools.GetBytes(0, 10) + U_BOOT_DATA,
+ data)
def testSplBssPadMissing(self):
"""Test that a missing symbol is detected"""
@@ -1108,9 +1110,9 @@ class TestFunctional(unittest.TestCase):
self._SetupSplElf('u_boot_binman_syms')
data = self._DoReadFile('053_symbols.dts')
sym_values = struct.pack('<LQL', 0x24 + 0, 0x24 + 24, 0x24 + 20)
- expected = (sym_values + U_BOOT_SPL_DATA[16:] + chr(0xff) +
- U_BOOT_DATA +
- sym_values + U_BOOT_SPL_DATA[16:])
+ expected = (sym_values + U_BOOT_SPL_DATA[16:] +
+ tools.GetBytes(0xff, 1) + U_BOOT_DATA + sym_values +
+ U_BOOT_SPL_DATA[16:])
self.assertEqual(expected, data)
def testPackUnitAddress(self):
@@ -1280,8 +1282,8 @@ class TestFunctional(unittest.TestCase):
}
data, _, _, _ = self._DoReadFileDtb('066_text.dts',
entry_args=entry_args)
- expected = (TEXT_DATA + chr(0) * (8 - len(TEXT_DATA)) + TEXT_DATA2 +
- TEXT_DATA3 + 'some text')
+ expected = (TEXT_DATA + tools.GetBytes(0, 8 - len(TEXT_DATA)) +
+ TEXT_DATA2 + TEXT_DATA3 + 'some text')
self.assertEqual(expected, data)
def testEntryDocs(self):
@@ -1340,7 +1342,7 @@ class TestFunctional(unittest.TestCase):
def testFill(self):
"""Test for an fill entry type"""
data = self._DoReadFile('069_fill.dts')
- expected = 8 * chr(0xff) + 8 * chr(0)
+ expected = tools.GetBytes(0xff, 8) + tools.GetBytes(0, 8)
self.assertEqual(expected, data)
def testFillNoSize(self):
@@ -1370,7 +1372,8 @@ class TestFunctional(unittest.TestCase):
data, _, _, _ = self._DoReadFileDtb('071_gbb.dts', entry_args=entry_args)
# Since futility
- expected = GBB_DATA + GBB_DATA + 8 * chr(0) + (0x2180 - 16) * chr(0)
+ expected = (GBB_DATA + GBB_DATA + tools.GetBytes(0, 8) +
+ tools.GetBytes(0, 0x2180 - 16))
self.assertEqual(expected, data)
def testGbbTooSmall(self):
@@ -1445,7 +1448,7 @@ class TestFunctional(unittest.TestCase):
def testFillZero(self):
"""Test for an fill entry type with a size of 0"""
data = self._DoReadFile('080_fill_empty.dts')
- self.assertEqual(chr(0) * 16, data)
+ self.assertEqual(tools.GetBytes(0, 16), data)
def testTextMissing(self):
"""Test for a text entry type where there is no text"""
@@ -1796,9 +1799,12 @@ class TestFunctional(unittest.TestCase):
0000002c 00000000 00000004 u-boot
''', map_data)
self.assertEqual(data,
- 4 * chr(0x26) + U_BOOT_DATA + 12 * chr(0x21) +
- 4 * chr(0x26) + U_BOOT_DATA + 12 * chr(0x61) +
- 4 * chr(0x26) + U_BOOT_DATA + 8 * chr(0x26))
+ tools.GetBytes(0x26, 4) + U_BOOT_DATA +
+ tools.GetBytes(0x21, 12) +
+ tools.GetBytes(0x26, 4) + U_BOOT_DATA +
+ tools.GetBytes(0x61, 12) +
+ tools.GetBytes(0x26, 4) + U_BOOT_DATA +
+ tools.GetBytes(0x26, 8))
if __name__ == "__main__":
diff --git a/tools/patman/tools.py b/tools/patman/tools.py
index 1df8f2ecd2..0ad0fb9705 100644
--- a/tools/patman/tools.py
+++ b/tools/patman/tools.py
@@ -7,6 +7,7 @@ import command
import glob
import os
import shutil
+import sys
import tempfile
import tout
@@ -239,3 +240,21 @@ def WriteFile(fname, data):
#(fname, len(data), len(data)))
with open(Filename(fname), 'wb') as fd:
fd.write(data)
+
+def GetBytes(byte, size):
+ """Get a string of bytes of a given size
+
+ This handles the unfortunate different between Python 2 and Python 2.
+
+ Args:
+ byte: Numeric byte value to use
+ size: Size of bytes/string to return
+
+ Returns:
+ A bytes type with 'byte' repeated 'size' times
+ """
+ if sys.version_info[0] >= 3:
+ data = bytes([byte]) * size
+ else:
+ data = chr(byte) * size
+ return data
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 14/21] Add a simple script to remove boards
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (12 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 13/21] binman: Handle repeated bytes for Python 3 Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-15 7:53 ` Chris Packham
2019-05-14 21:53 ` [U-Boot] [PATCH 15/21] patman: Support use of stringIO in Python 3 Simon Glass
` (7 subsequent siblings)
21 siblings, 1 reply; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
This script attempts to create a git commit which removes a single board.
It is quite fallible and everything it does needs checking. But it can
help speed up the process.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/rmboard.py | 150 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 150 insertions(+)
create mode 100755 tools/rmboard.py
diff --git a/tools/rmboard.py b/tools/rmboard.py
new file mode 100755
index 0000000000..17952f795d
--- /dev/null
+++ b/tools/rmboard.py
@@ -0,0 +1,150 @@
+#! /usr/bin/python
+# SPDX-License-Identifier: GPL-2.0+
+# Copyright 2019 Google LLC
+#
+
+"""
+Script to remove boards
+
+Usage:
+ rmboard.py <board_name>...
+
+A single commit is created for each board removed.
+
+Some boards may depend on files provided by another and this will cause
+problems, generally the removal of files which should not be removed.
+
+This script works by:
+ - Looking through the MAINTAINERS files which mention a board to find out
+ what files the board uses
+ - Looking through the Kconfig files which mention a board to find one that
+ needs to have material removed
+
+Search for ## to update the commit message manually.
+"""
+
+from __future__ import print_function
+
+import glob
+import os
+import re
+import sys
+
+# Bring in the patman libraries
+our_path = os.path.dirname(os.path.realpath(__file__))
+sys.path.append(os.path.join(our_path, '../tools/patman'))
+
+import command
+
+def rm_kconfig_include(path):
+ """Remove a path from Kconfig files
+
+ This function finds the given path in a 'source' statement in a Kconfig
+ file and removes that line from the file. This is needed because the path
+ is going to be removed, so any reference to it will cause a problem with
+ Kconfig parsing.
+
+ The changes are made locally and then added to the git staging area.
+
+ Args:
+ path: Path to search for and remove
+ """
+ cmd = ['git', 'grep', path]
+ stdout = command.RunPipe([cmd], capture=True, raise_on_error=False).stdout
+ if not stdout:
+ return
+ fname = stdout.split(':')[0]
+
+ print("Fixing up '%s' to remove reference to '%s'" % (fname, path))
+ cmd = ['sed', '-i', '\|%s|d' % path, fname]
+ stdout = command.RunPipe([cmd], capture=True).stdout
+
+ cmd = ['git', 'add', fname]
+ stdout = command.RunPipe([cmd], capture=True).stdout
+
+def rm_board(board):
+ """Create a commit which removes a single board
+
+ This looks up the MAINTAINERS file to file files that need to be removed,
+ then removes pieces from the Kconfig files that mention the board.
+
+
+ Args:
+ board: Board name to remove
+ """
+
+ # Find all MAINTAINERS and Kconfig files which mention the board
+ cmd = ['git', 'grep', '-l', board]
+ stdout = command.RunPipe([cmd], capture=True).stdout
+ maintain = []
+ kconfig = []
+ for line in stdout.splitlines():
+ line = line.strip()
+ if 'MAINTAINERS' in line:
+ if line not in maintain:
+ maintain.append(line)
+ elif 'Kconfig' in line:
+ kconfig.append(line)
+ paths = []
+ cc = []
+
+ # Look through the MAINTAINERS file to find things to remove
+ for fname in maintain:
+ with open(fname) as fd:
+ for line in fd:
+ line = line.strip()
+ fields = re.split('[ \t]', line, 1)
+ if len(fields) == 2:
+ if fields[0] == 'M:':
+ cc.append(fields[1])
+ elif fields[0] == 'F:':
+ paths.append(fields[1].strip())
+
+ # Expannd any wildcards in the MAINTAINRERS file
+ real = []
+ for path in paths:
+ if path[-1] == '/':
+ path = path[:-1]
+ if '*' in path:
+ globbed = glob.glob(path)
+ print("Expanded '%s' to '%s'" % (path, globbed))
+ real += globbed
+ else:
+ real.append(path)
+
+ # Search for Kconfig files in the resulting list. Remove any 'source' lines
+ # which reference Kconfig files we want to remove
+ for path in real:
+ cmd = ['find', path]
+ stdout = (command.RunPipe([cmd], capture=True, raise_on_error=False).
+ stdout)
+ for fname in stdout.splitlines():
+ if fname.endswith('Kconfig'):
+ rm_kconfig_include(fname)
+
+ # Remove unwanted files
+ cmd = ['git', 'rm', '-r'] + real
+ stdout = command.RunPipe([cmd], capture=True).stdout
+
+ ## Change the messages as needed
+ msg = '''arm: Remove %s board
+
+This board has not been converted to CONFIG_DM_MMC by the deadline.
+Remove it.
+
+''' % board
+ for name in cc:
+ msg += 'Patch-cc: %s\n' % name
+
+ # Create the commit
+ cmd = ['git', 'commit', '-s', '-m', msg]
+ stdout = command.RunPipe([cmd], capture=True).stdout
+
+ # Check if the board is mentioned anywhere else. The user will need to deal
+ # with this
+ cmd = ['git', 'grep', '-il', board]
+ print(command.RunPipe([cmd], capture=True, raise_on_error=False).stdout)
+ print(' '.join(cmd))
+
+for board in sys.argv[1:]:
+ rm_board(board)
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 15/21] patman: Support use of stringIO in Python 3
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (13 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 14/21] Add a simple script to remove boards Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 16/21] patman: Move unicode helpers to tools Simon Glass
` (6 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
With Python 3 this class has moved. Update the code to handle both cases.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/patman/func_test.py | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/tools/patman/func_test.py b/tools/patman/func_test.py
index 31481157fc..2c4392dbc1 100644
--- a/tools/patman/func_test.py
+++ b/tools/patman/func_test.py
@@ -12,15 +12,20 @@ import sys
import tempfile
import unittest
+try:
+ from StringIO import StringIO
+except ImportError:
+ from io import StringIO
+
import gitutil
import patchstream
import settings
+import tools
@contextlib.contextmanager
def capture():
import sys
- from cStringIO import StringIO
oldout,olderr = sys.stdout, sys.stderr
try:
out=[StringIO(), StringIO()]
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 16/21] patman: Move unicode helpers to tools
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (14 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 15/21] patman: Support use of stringIO in Python 3 Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 17/21] patman: Sort series output for repeatabily Simon Glass
` (5 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
Create helper functions in the tools module to deal with the differences
between unicode in Python 2 (where we use the 'unicode' type) and Python 3
(where we use the 'str' type).
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/patman/gitutil.py | 4 ++--
tools/patman/series.py | 3 ++-
tools/patman/settings.py | 16 +++-------------
tools/patman/tools.py | 32 ++++++++++++++++++++++++++++++++
4 files changed, 39 insertions(+), 16 deletions(-)
diff --git a/tools/patman/gitutil.py b/tools/patman/gitutil.py
index 11aeb73b74..fb9e67c0f0 100644
--- a/tools/patman/gitutil.py
+++ b/tools/patman/gitutil.py
@@ -12,6 +12,7 @@ import terminal
import checkpatch
import settings
+import tools
# True to use --no-decorate - we check this in Setup()
use_no_decorate = True
@@ -325,9 +326,8 @@ def BuildEmailList(in_list, tag=None, alias=None, raise_on_error=True):
raw += LookupEmail(item, alias, raise_on_error=raise_on_error)
result = []
for item in raw:
+ item = tools.FromUnicode(item)
if not item in result:
- if type(item) == unicode:
- item = item.encode('utf-8')
result.append(item)
if tag:
return ['%s %s%s%s' % (tag, quote, email, quote) for email in result]
diff --git a/tools/patman/series.py b/tools/patman/series.py
index 2735afaf88..0b71a896cb 100644
--- a/tools/patman/series.py
+++ b/tools/patman/series.py
@@ -11,6 +11,7 @@ import get_maintainer
import gitutil
import settings
import terminal
+import tools
# Series-xxx tags that we understand
valid_series = ['to', 'cc', 'version', 'changes', 'prefix', 'notes', 'name',
@@ -249,7 +250,7 @@ class Series(dict):
cover_cc = gitutil.BuildEmailList(self.get('cover_cc', ''))
cover_cc = [m.encode('utf-8') if type(m) != str else m
for m in cover_cc]
- cc_list = ', '.join([x.decode('utf-8')
+ cc_list = ', '.join([tools.ToUnicode(x)
for x in set(cover_cc + all_ccs)])
print(cover_fname, cc_list.encode('utf-8'), file=fd)
diff --git a/tools/patman/settings.py b/tools/patman/settings.py
index 07bf6a6ea4..b080136d88 100644
--- a/tools/patman/settings.py
+++ b/tools/patman/settings.py
@@ -14,6 +14,7 @@ import re
import command
import gitutil
+import tools
"""Default settings per-project.
@@ -99,17 +100,6 @@ class _ProjectConfigParser(ConfigParser.SafeConfigParser):
for setting_name, setting_value in project_defaults.items():
self.set(project_settings, setting_name, setting_value)
- def _to_unicode(self, val):
- """Make sure a value is of type 'unicode'
-
- Args:
- val: string or unicode object
-
- Returns:
- unicode version of val
- """
- return val if isinstance(val, unicode) else val.decode('utf-8')
-
def get(self, section, option, *args, **kwargs):
"""Extend SafeConfigParser to try project_section before section.
@@ -127,7 +117,7 @@ class _ProjectConfigParser(ConfigParser.SafeConfigParser):
val = ConfigParser.SafeConfigParser.get(
self, section, option, *args, **kwargs
)
- return self._to_unicode(val)
+ return tools.ToUnicode(val)
def items(self, section, *args, **kwargs):
"""Extend SafeConfigParser to add project_section to section.
@@ -162,7 +152,7 @@ class _ProjectConfigParser(ConfigParser.SafeConfigParser):
item_dict = dict(top_items)
item_dict.update(project_items)
- return {(self._to_unicode(item), self._to_unicode(val))
+ return {(tools.ToUnicode(item), tools.ToUnicode(val))
for item, val in item_dict.items()}
def ReadGitAliases(fname):
diff --git a/tools/patman/tools.py b/tools/patman/tools.py
index 0ad0fb9705..7e6a45a3b0 100644
--- a/tools/patman/tools.py
+++ b/tools/patman/tools.py
@@ -258,3 +258,35 @@ def GetBytes(byte, size):
else:
data = chr(byte) * size
return data
+
+def ToUnicode(val):
+ """Make sure a value is a unicode string
+
+ This allows some amount of compatibility between Python 2 and Python3. For
+ the former, it returns a unicode object.
+
+ Args:
+ val: string or unicode object
+
+ Returns:
+ unicode version of val
+ """
+ if sys.version_info[0] >= 3:
+ return val
+ return val if isinstance(val, unicode) else val.decode('utf-8')
+
+def FromUnicode(val):
+ """Make sure a value is a non-unicode string
+
+ This allows some amount of compatibility between Python 2 and Python3. For
+ the former, it converts a unicode object to a string.
+
+ Args:
+ val: string or unicode object
+
+ Returns:
+ non-unicode version of val
+ """
+ if sys.version_info[0] >= 3:
+ return val
+ return val if isinstance(val, str) else val.encode('utf-8')
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 17/21] patman: Sort series output for repeatabily
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (15 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 16/21] patman: Move unicode helpers to tools Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 18/21] patman: Avoid unicode type in settings unit tests Simon Glass
` (4 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
We use sets to produce the list of To and Cc lines for a series. This does
not result in stable ordering of the recipients. Sort each list to ensure
that the output is repeatable. This is necessary for tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/patman/func_test.py | 12 ++++++------
tools/patman/series.py | 12 ++++++------
2 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/tools/patman/func_test.py b/tools/patman/func_test.py
index 2c4392dbc1..f7d5ad68f6 100644
--- a/tools/patman/func_test.py
+++ b/tools/patman/func_test.py
@@ -183,10 +183,10 @@ class TestFunctional(unittest.TestCase):
self.assertEqual('Prefix:\t RFC', lines[line + 3])
self.assertEqual('Cover: 4 lines', lines[line + 4])
line += 5
- self.assertEqual(' Cc: %s' % mel.encode('utf-8'), lines[line + 0])
- self.assertEqual(' Cc: %s' % rick, lines[line + 1])
- self.assertEqual(' Cc: %s' % fred, lines[line + 2])
- self.assertEqual(' Cc: %s' % ed.encode('utf-8'), lines[line + 3])
+ self.assertEqual(' Cc: %s' % fred, lines[line + 0])
+ self.assertEqual(' Cc: %s' % ed.encode('utf-8'), lines[line + 1])
+ self.assertEqual(' Cc: %s' % mel.encode('utf-8'), lines[line + 2])
+ self.assertEqual(' Cc: %s' % rick, lines[line + 3])
expected = ('Git command: git send-email --annotate '
'--in-reply-to="%s" --to "u-boot at lists.denx.de" '
'--cc "%s" --cc-cmd "%s --cc-cmd %s" %s %s'
@@ -197,8 +197,8 @@ class TestFunctional(unittest.TestCase):
self.assertEqual(('%s %s, %s' % (args[0], rick, stefan))
.encode('utf-8'), cc_lines[0])
- self.assertEqual(('%s %s, %s, %s, %s' % (args[1], fred, rick, stefan,
- ed)).encode('utf-8'), cc_lines[1])
+ self.assertEqual(('%s %s, %s, %s, %s' % (args[1], fred, ed, rick,
+ stefan)).encode('utf-8'), cc_lines[1])
expected = '''
This is a test of how the cover
diff --git a/tools/patman/series.py b/tools/patman/series.py
index 0b71a896cb..bbb30d849e 100644
--- a/tools/patman/series.py
+++ b/tools/patman/series.py
@@ -115,16 +115,16 @@ class Series(dict):
commit = self.commits[upto]
print(col.Color(col.GREEN, ' %s' % args[upto]))
cc_list = list(self._generated_cc[commit.patch])
- for email in set(cc_list) - to_set - cc_set:
+ for email in sorted(set(cc_list) - to_set - cc_set):
if email == None:
email = col.Color(col.YELLOW, "<alias '%s' not found>"
% tag)
if email:
print(' Cc: ', email)
print
- for item in to_set:
+ for item in sorted(to_set):
print('To:\t ', item)
- for item in cc_set - to_set:
+ for item in sorted(cc_set - to_set):
print('Cc:\t ', item)
print('Version: ', self.get('version'))
print('Prefix:\t ', self.get('prefix'))
@@ -132,7 +132,7 @@ class Series(dict):
print('Cover: %d lines' % len(self.cover))
cover_cc = gitutil.BuildEmailList(self.get('cover_cc', ''))
all_ccs = itertools.chain(cover_cc, *self._generated_cc.values())
- for email in set(all_ccs) - to_set - cc_set:
+ for email in sorted(set(all_ccs) - to_set - cc_set):
print(' Cc: ', email)
if cmd:
print('Git command: %s' % cmd)
@@ -243,7 +243,7 @@ class Series(dict):
if limit is not None:
cc = cc[:limit]
all_ccs += cc
- print(commit.patch, ', '.join(set(cc)), file=fd)
+ print(commit.patch, ', '.join(sorted(set(cc))), file=fd)
self._generated_cc[commit.patch] = cc
if cover_fname:
@@ -251,7 +251,7 @@ class Series(dict):
cover_cc = [m.encode('utf-8') if type(m) != str else m
for m in cover_cc]
cc_list = ', '.join([tools.ToUnicode(x)
- for x in set(cover_cc + all_ccs)])
+ for x in sorted(set(cover_cc + all_ccs))])
print(cover_fname, cc_list.encode('utf-8'), file=fd)
fd.close()
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 18/21] patman: Avoid unicode type in settings unit tests
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (16 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 17/21] patman: Sort series output for repeatabily Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 19/21] patman: Adjust functional tests for Python 3 Simon Glass
` (3 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
The unicode type does not exist in Python 3 and when displaying strings
they do not have the 'u' prefix. Adjusts the settings unit tests to deal
with this difference, by converting the comparison value to a string, thus
dropping the 'u'.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/patman/settings.py | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/tools/patman/settings.py b/tools/patman/settings.py
index b080136d88..c98911d522 100644
--- a/tools/patman/settings.py
+++ b/tools/patman/settings.py
@@ -58,26 +58,26 @@ class _ProjectConfigParser(ConfigParser.SafeConfigParser):
# Check to make sure that bogus project gets general alias.
>>> config = _ProjectConfigParser("zzz")
>>> config.readfp(StringIO(sample_config))
- >>> config.get("alias", "enemies")
- u'Evil <evil@example.com>'
+ >>> str(config.get("alias", "enemies"))
+ 'Evil <evil@example.com>'
# Check to make sure that alias gets overridden by project.
>>> config = _ProjectConfigParser("sm")
>>> config.readfp(StringIO(sample_config))
- >>> config.get("alias", "enemies")
- u'Green G. <ugly@example.com>'
+ >>> str(config.get("alias", "enemies"))
+ 'Green G. <ugly@example.com>'
# Check to make sure that settings get merged with project.
>>> config = _ProjectConfigParser("linux")
>>> config.readfp(StringIO(sample_config))
- >>> sorted(config.items("settings"))
- [(u'am_hero', u'True'), (u'process_tags', u'False')]
+ >>> sorted((str(a), str(b)) for (a, b) in config.items("settings"))
+ [('am_hero', 'True'), ('process_tags', 'False')]
# Check to make sure that settings works with unknown project.
>>> config = _ProjectConfigParser("unknown")
>>> config.readfp(StringIO(sample_config))
- >>> sorted(config.items("settings"))
- [(u'am_hero', u'True')]
+ >>> sorted((str(a), str(b)) for (a, b) in config.items("settings"))
+ [('am_hero', 'True')]
"""
def __init__(self, project_name):
"""Construct _ProjectConfigParser.
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 19/21] patman: Adjust functional tests for Python 3
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (17 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 18/21] patman: Avoid unicode type in settings unit tests Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 20/21] patman: Tidy up a few more unicode conversions Simon Glass
` (2 subsequent siblings)
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
Change the code so that it works on both Python 2 and Python 3. This works
by using unicode instead of latin1 for the test input, and ensuring that
the output is converted to a string rather than a unicode object on
Python 2.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/patman/func_test.py | 25 ++++++++++++++-----------
1 file changed, 14 insertions(+), 11 deletions(-)
diff --git a/tools/patman/func_test.py b/tools/patman/func_test.py
index f7d5ad68f6..50a2741439 100644
--- a/tools/patman/func_test.py
+++ b/tools/patman/func_test.py
@@ -129,10 +129,10 @@ class TestFunctional(unittest.TestCase):
"""
process_tags = True
ignore_bad_tags = True
- stefan = u'Stefan Brüns <stefan.bruens@rwth-aachen.de>'
+ stefan = b'Stefan Br\xc3\xbcns <stefan.bruens@rwth-aachen.de>'.decode('utf-8')
rick = 'Richard III <richard@palace.gov>'
- mel = u'Lord Mëlchett <clergy@palace.gov>'
- ed = u'Lond Edmund Blackaddër <weasel at blackadder.org'
+ mel = b'Lord M\xc3\xablchett <clergy@palace.gov>'.decode('utf-8')
+ ed = b'Lond Edmund Blackadd\xc3\xabr <weasel at blackadder.org'.decode('utf-8')
fred = 'Fred Bloggs <f.bloggs@napier.net>'
add_maintainers = [stefan, rick]
dry_run = True
@@ -178,27 +178,30 @@ class TestFunctional(unittest.TestCase):
while 'Cc:' in lines[line]:
line += 1
self.assertEqual('To: u-boot at lists.denx.de', lines[line])
- self.assertEqual('Cc: %s' % stefan.encode('utf-8'), lines[line + 1])
+ self.assertEqual('Cc: %s' % tools.FromUnicode(stefan),
+ lines[line + 1])
self.assertEqual('Version: 3', lines[line + 2])
self.assertEqual('Prefix:\t RFC', lines[line + 3])
self.assertEqual('Cover: 4 lines', lines[line + 4])
line += 5
self.assertEqual(' Cc: %s' % fred, lines[line + 0])
- self.assertEqual(' Cc: %s' % ed.encode('utf-8'), lines[line + 1])
- self.assertEqual(' Cc: %s' % mel.encode('utf-8'), lines[line + 2])
+ self.assertEqual(' Cc: %s' % tools.FromUnicode(ed),
+ lines[line + 1])
+ self.assertEqual(' Cc: %s' % tools.FromUnicode(mel),
+ lines[line + 2])
self.assertEqual(' Cc: %s' % rick, lines[line + 3])
expected = ('Git command: git send-email --annotate '
'--in-reply-to="%s" --to "u-boot at lists.denx.de" '
'--cc "%s" --cc-cmd "%s --cc-cmd %s" %s %s'
% (in_reply_to, stefan, sys.argv[0], cc_file, cover_fname,
- ' '.join(args))).encode('utf-8')
+ ' '.join(args)))
line += 4
- self.assertEqual(expected, lines[line])
+ self.assertEqual(expected, tools.ToUnicode(lines[line]))
- self.assertEqual(('%s %s, %s' % (args[0], rick, stefan))
- .encode('utf-8'), cc_lines[0])
+ self.assertEqual(('%s %s, %s' % (args[0], rick, stefan)),
+ tools.ToUnicode(cc_lines[0]))
self.assertEqual(('%s %s, %s, %s, %s' % (args[1], fred, ed, rick,
- stefan)).encode('utf-8'), cc_lines[1])
+ stefan)), tools.ToUnicode(cc_lines[1]))
expected = '''
This is a test of how the cover
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 20/21] patman: Tidy up a few more unicode conversions
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (18 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 19/21] patman: Adjust functional tests for Python 3 Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 21/21] patman: Don't require Python 2 Simon Glass
2019-05-15 8:01 ` [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Chris Packham
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
Use the new functions in the tools module to handle conversion.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/patman/gitutil.py | 4 +---
tools/patman/series.py | 5 ++---
2 files changed, 3 insertions(+), 6 deletions(-)
diff --git a/tools/patman/gitutil.py b/tools/patman/gitutil.py
index fb9e67c0f0..dce7fa25b6 100644
--- a/tools/patman/gitutil.py
+++ b/tools/patman/gitutil.py
@@ -412,9 +412,7 @@ def EmailPatches(series, cover_fname, args, dry_run, raise_on_error, cc_fname,
if smtp_server:
cmd.append('--smtp-server=%s' % smtp_server)
if in_reply_to:
- if type(in_reply_to) != str:
- in_reply_to = in_reply_to.encode('utf-8')
- cmd.append('--in-reply-to="%s"' % in_reply_to)
+ cmd.append('--in-reply-to="%s"' % tools.FromUnicode(in_reply_to))
if thread:
cmd.append('--thread')
diff --git a/tools/patman/series.py b/tools/patman/series.py
index bbb30d849e..67103f03e6 100644
--- a/tools/patman/series.py
+++ b/tools/patman/series.py
@@ -239,7 +239,7 @@ class Series(dict):
for x in set(cc) & set(settings.bounces):
print(col.Color(col.YELLOW, 'Skipping "%s"' % x))
cc = set(cc) - set(settings.bounces)
- cc = [m.encode('utf-8') if type(m) != str else m for m in cc]
+ cc = [tools.FromUnicode(m) for m in cc]
if limit is not None:
cc = cc[:limit]
all_ccs += cc
@@ -248,8 +248,7 @@ class Series(dict):
if cover_fname:
cover_cc = gitutil.BuildEmailList(self.get('cover_cc', ''))
- cover_cc = [m.encode('utf-8') if type(m) != str else m
- for m in cover_cc]
+ cover_cc = [tools.FromUnicode(m) for m in cover_cc]
cc_list = ', '.join([tools.ToUnicode(x)
for x in sorted(set(cover_cc + all_ccs))])
print(cover_fname, cc_list.encode('utf-8'), file=fd)
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 21/21] patman: Don't require Python 2
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (19 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 20/21] patman: Tidy up a few more unicode conversions Simon Glass
@ 2019-05-14 21:53 ` Simon Glass
2019-05-15 8:01 ` [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Chris Packham
21 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-14 21:53 UTC (permalink / raw)
To: u-boot
Update the shebang to allow either Python 2 or Python 3.
Signed-off-by: Simon Glass <sjg@chromium.org>
---
tools/patman/patman.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/patman/patman.py b/tools/patman/patman.py
index 27a2febf70..9605a36eff 100755
--- a/tools/patman/patman.py
+++ b/tools/patman/patman.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python
# SPDX-License-Identifier: GPL-2.0+
#
# Copyright (c) 2011 The Chromium OS Authors.
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 14/21] Add a simple script to remove boards
2019-05-14 21:53 ` [U-Boot] [PATCH 14/21] Add a simple script to remove boards Simon Glass
@ 2019-05-15 7:53 ` Chris Packham
2019-05-18 16:06 ` Simon Glass
0 siblings, 1 reply; 26+ messages in thread
From: Chris Packham @ 2019-05-15 7:53 UTC (permalink / raw)
To: u-boot
On Wed, May 15, 2019 at 9:54 AM Simon Glass <sjg@chromium.org> wrote:
>
> This script attempts to create a git commit which removes a single board.
> It is quite fallible and everything it does needs checking. But it can
> help speed up the process.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
Did you mean to include this file in the series. It seems unrelated to
the other patman/binman changes.
>
> tools/rmboard.py | 150 +++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 150 insertions(+)
> create mode 100755 tools/rmboard.py
>
> diff --git a/tools/rmboard.py b/tools/rmboard.py
> new file mode 100755
> index 0000000000..17952f795d
> --- /dev/null
> +++ b/tools/rmboard.py
> @@ -0,0 +1,150 @@
> +#! /usr/bin/python
> +# SPDX-License-Identifier: GPL-2.0+
> +# Copyright 2019 Google LLC
> +#
> +
> +"""
> +Script to remove boards
> +
> +Usage:
> + rmboard.py <board_name>...
> +
> +A single commit is created for each board removed.
> +
> +Some boards may depend on files provided by another and this will cause
> +problems, generally the removal of files which should not be removed.
> +
> +This script works by:
> + - Looking through the MAINTAINERS files which mention a board to find out
> + what files the board uses
> + - Looking through the Kconfig files which mention a board to find one that
> + needs to have material removed
> +
> +Search for ## to update the commit message manually.
> +"""
> +
> +from __future__ import print_function
> +
> +import glob
> +import os
> +import re
> +import sys
> +
> +# Bring in the patman libraries
> +our_path = os.path.dirname(os.path.realpath(__file__))
> +sys.path.append(os.path.join(our_path, '../tools/patman'))
> +
> +import command
> +
> +def rm_kconfig_include(path):
> + """Remove a path from Kconfig files
> +
> + This function finds the given path in a 'source' statement in a Kconfig
> + file and removes that line from the file. This is needed because the path
> + is going to be removed, so any reference to it will cause a problem with
> + Kconfig parsing.
> +
> + The changes are made locally and then added to the git staging area.
> +
> + Args:
> + path: Path to search for and remove
> + """
> + cmd = ['git', 'grep', path]
> + stdout = command.RunPipe([cmd], capture=True, raise_on_error=False).stdout
Is there any specific advantage of using patman/command.py instead of
python's subprocess module directly?
> + if not stdout:
> + return
> + fname = stdout.split(':')[0]
> +
> + print("Fixing up '%s' to remove reference to '%s'" % (fname, path))
> + cmd = ['sed', '-i', '\|%s|d' % path, fname]
> + stdout = command.RunPipe([cmd], capture=True).stdout
> +
> + cmd = ['git', 'add', fname]
> + stdout = command.RunPipe([cmd], capture=True).stdout
> +
> +def rm_board(board):
> + """Create a commit which removes a single board
> +
> + This looks up the MAINTAINERS file to file files that need to be removed,
> + then removes pieces from the Kconfig files that mention the board.
> +
> +
> + Args:
> + board: Board name to remove
> + """
> +
> + # Find all MAINTAINERS and Kconfig files which mention the board
> + cmd = ['git', 'grep', '-l', board]
> + stdout = command.RunPipe([cmd], capture=True).stdout
> + maintain = []
> + kconfig = []
> + for line in stdout.splitlines():
> + line = line.strip()
> + if 'MAINTAINERS' in line:
> + if line not in maintain:
> + maintain.append(line)
> + elif 'Kconfig' in line:
> + kconfig.append(line)
> + paths = []
> + cc = []
> +
> + # Look through the MAINTAINERS file to find things to remove
> + for fname in maintain:
> + with open(fname) as fd:
> + for line in fd:
> + line = line.strip()
> + fields = re.split('[ \t]', line, 1)
> + if len(fields) == 2:
> + if fields[0] == 'M:':
> + cc.append(fields[1])
> + elif fields[0] == 'F:':
> + paths.append(fields[1].strip())
> +
> + # Expannd any wildcards in the MAINTAINRERS file
Couple of typos s/Expannd/Expand/ and s/MAINTAINRERS/MAINTAINERS/
> + real = []
> + for path in paths:
> + if path[-1] == '/':
> + path = path[:-1]
> + if '*' in path:
> + globbed = glob.glob(path)
> + print("Expanded '%s' to '%s'" % (path, globbed))
> + real += globbed
> + else:
> + real.append(path)
> +
> + # Search for Kconfig files in the resulting list. Remove any 'source' lines
> + # which reference Kconfig files we want to remove
> + for path in real:
> + cmd = ['find', path]
> + stdout = (command.RunPipe([cmd], capture=True, raise_on_error=False).
> + stdout)
> + for fname in stdout.splitlines():
> + if fname.endswith('Kconfig'):
> + rm_kconfig_include(fname)
> +
> + # Remove unwanted files
> + cmd = ['git', 'rm', '-r'] + real
> + stdout = command.RunPipe([cmd], capture=True).stdout
> +
> + ## Change the messages as needed
> + msg = '''arm: Remove %s board
> +
> +This board has not been converted to CONFIG_DM_MMC by the deadline.
> +Remove it.
> +
> +''' % board
> + for name in cc:
> + msg += 'Patch-cc: %s\n' % name
> +
> + # Create the commit
> + cmd = ['git', 'commit', '-s', '-m', msg]
> + stdout = command.RunPipe([cmd], capture=True).stdout
> +
> + # Check if the board is mentioned anywhere else. The user will need to deal
> + # with this
> + cmd = ['git', 'grep', '-il', board]
> + print(command.RunPipe([cmd], capture=True, raise_on_error=False).stdout)
> + print(' '.join(cmd))
> +
> +for board in sys.argv[1:]:
> + rm_board(board)
> --
> 2.21.0.1020.gf2820cf01a-goog
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 00/21] patman: Update to support Python 3
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
` (20 preceding siblings ...)
2019-05-14 21:53 ` [U-Boot] [PATCH 21/21] patman: Don't require Python 2 Simon Glass
@ 2019-05-15 8:01 ` Chris Packham
2019-06-24 17:17 ` Simon Glass
21 siblings, 1 reply; 26+ messages in thread
From: Chris Packham @ 2019-05-15 8:01 UTC (permalink / raw)
To: u-boot
On Wed, May 15, 2019 at 9:54 AM Simon Glass <sjg@chromium.org> wrote:
>
> This series updates patman to support Python 3:
>
> - Avoid using 'unicode' type directly
> - Use items() instead of iteritems()
> - Make sure file I/O uses binary mode where necessary
> - Change print statements to functions
> - Use the built-in set() class
> - Fix up generation of repeated bytes
>
> A few patches for binman are included, but this still requires Python 2.
>
Couple of comments on 14/21 but the rest of the series
Reviewed-by: Chris Packham <judge.packham@gmail.com>
>
> Simon Glass (21):
> patman: Update cros_subprocess to use bytearray
> patman: Convert print statements to Python 3
> binman: Convert print statements to Python 3
> binman: Don't show errors for failed tests
> binman: Remove use of Set()
> patman: Use items() instead of iteritems()
> binman: Use items() instead of iteritems()
> tools: binman: Open all binary files in binary mode
> tools: dtoc: Open all binary files in binary mode
> patman: Provide a way to get program output in binary mode
> binman: Use binary mode when compressing data
> binman: Drop an unused input file
> binman: Handle repeated bytes for Python 3
> Add a simple script to remove boards
> patman: Support use of stringIO in Python 3
> patman: Move unicode helpers to tools
> patman: Sort series output for repeatabily
> patman: Avoid unicode type in settings unit tests
> patman: Adjust functional tests for Python 3
> patman: Tidy up a few more unicode conversions
> patman: Don't require Python 2
>
> tools/binman/binman.py | 26 +++-
> tools/binman/bsection.py | 7 +-
> tools/binman/control.py | 8 +-
> tools/binman/elf.py | 4 +-
> tools/binman/elf_test.py | 2 +-
> tools/binman/entry.py | 5 +-
> tools/binman/etype/blob.py | 2 +-
> tools/binman/etype/fill.py | 4 +-
> tools/binman/etype/gbb.py | 2 +-
> tools/binman/etype/u_boot_spl_bss_pad.py | 2 +-
> tools/binman/ftest.py | 81 ++++++------
> tools/binman/state.py | 7 +-
> tools/dtoc/fdt.py | 2 +-
> tools/patman/cros_subprocess.py | 53 +++++---
> tools/patman/func_test.py | 41 ++++---
> tools/patman/gitutil.py | 16 +--
> tools/patman/patman.py | 2 +-
> tools/patman/series.py | 20 +--
> tools/patman/settings.py | 34 ++---
> tools/patman/test_util.py | 16 +--
> tools/patman/tools.py | 55 ++++++++-
> tools/rmboard.py | 150 +++++++++++++++++++++++
> 22 files changed, 385 insertions(+), 154 deletions(-)
> create mode 100755 tools/rmboard.py
>
> --
> 2.21.0.1020.gf2820cf01a-goog
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 14/21] Add a simple script to remove boards
2019-05-15 7:53 ` Chris Packham
@ 2019-05-18 16:06 ` Simon Glass
0 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-05-18 16:06 UTC (permalink / raw)
To: u-boot
Hi Chris,
On Wed, 15 May 2019 at 01:54, Chris Packham <judge.packham@gmail.com> wrote:
>
> On Wed, May 15, 2019 at 9:54 AM Simon Glass <sjg@chromium.org> wrote:
> >
> > This script attempts to create a git commit which removes a single board.
> > It is quite fallible and everything it does needs checking. But it can
> > help speed up the process.
> >
> > Signed-off-by: Simon Glass <sjg@chromium.org>
> > ---
>
> Did you mean to include this file in the series. It seems unrelated to
> the other patman/binman changes.
Well I originally had it as part of the series to drop/mangle boards
that have not been migrated. But I decided it might fit better here
since it was Python.
I'll send it again as a separate tool.
>
> >
> > tools/rmboard.py | 150 +++++++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 150 insertions(+)
> > create mode 100755 tools/rmboard.py
> >
> > diff --git a/tools/rmboard.py b/tools/rmboard.py
> > new file mode 100755
> > index 0000000000..17952f795d
> > --- /dev/null
> > +++ b/tools/rmboard.py
> > @@ -0,0 +1,150 @@
> > +#! /usr/bin/python
> > +# SPDX-License-Identifier: GPL-2.0+
> > +# Copyright 2019 Google LLC
> > +#
> > +
> > +"""
> > +Script to remove boards
> > +
> > +Usage:
> > + rmboard.py <board_name>...
> > +
> > +A single commit is created for each board removed.
> > +
> > +Some boards may depend on files provided by another and this will cause
> > +problems, generally the removal of files which should not be removed.
> > +
> > +This script works by:
> > + - Looking through the MAINTAINERS files which mention a board to find out
> > + what files the board uses
> > + - Looking through the Kconfig files which mention a board to find one that
> > + needs to have material removed
> > +
> > +Search for ## to update the commit message manually.
> > +"""
> > +
> > +from __future__ import print_function
> > +
> > +import glob
> > +import os
> > +import re
> > +import sys
> > +
> > +# Bring in the patman libraries
> > +our_path = os.path.dirname(os.path.realpath(__file__))
> > +sys.path.append(os.path.join(our_path, '../tools/patman'))
> > +
> > +import command
> > +
> > +def rm_kconfig_include(path):
> > + """Remove a path from Kconfig files
> > +
> > + This function finds the given path in a 'source' statement in a Kconfig
> > + file and removes that line from the file. This is needed because the path
> > + is going to be removed, so any reference to it will cause a problem with
> > + Kconfig parsing.
> > +
> > + The changes are made locally and then added to the git staging area.
> > +
> > + Args:
> > + path: Path to search for and remove
> > + """
> > + cmd = ['git', 'grep', path]
> > + stdout = command.RunPipe([cmd], capture=True, raise_on_error=False).stdout
>
> Is there any specific advantage of using patman/command.py instead of
> python's subprocess module directly?
Just for consistency with the rest of the code.
The only difference really is that 'command' allows output to be
collected while the command is running rather than hanging the process
until the command exits. In this case the difference doesn't matter
though. since the command is pretty quick and we just wait for all the
output to come.
So, mostly just for consistency.
>
> > + if not stdout:
> > + return
> > + fname = stdout.split(':')[0]
> > +
> > + print("Fixing up '%s' to remove reference to '%s'" % (fname, path))
> > + cmd = ['sed', '-i', '\|%s|d' % path, fname]
> > + stdout = command.RunPipe([cmd], capture=True).stdout
> > +
> > + cmd = ['git', 'add', fname]
> > + stdout = command.RunPipe([cmd], capture=True).stdout
> > +
> > +def rm_board(board):
> > + """Create a commit which removes a single board
> > +
> > + This looks up the MAINTAINERS file to file files that need to be removed,
> > + then removes pieces from the Kconfig files that mention the board.
> > +
> > +
> > + Args:
> > + board: Board name to remove
> > + """
> > +
> > + # Find all MAINTAINERS and Kconfig files which mention the board
> > + cmd = ['git', 'grep', '-l', board]
> > + stdout = command.RunPipe([cmd], capture=True).stdout
> > + maintain = []
> > + kconfig = []
> > + for line in stdout.splitlines():
> > + line = line.strip()
> > + if 'MAINTAINERS' in line:
> > + if line not in maintain:
> > + maintain.append(line)
> > + elif 'Kconfig' in line:
> > + kconfig.append(line)
> > + paths = []
> > + cc = []
> > +
> > + # Look through the MAINTAINERS file to find things to remove
> > + for fname in maintain:
> > + with open(fname) as fd:
> > + for line in fd:
> > + line = line.strip()
> > + fields = re.split('[ \t]', line, 1)
> > + if len(fields) == 2:
> > + if fields[0] == 'M:':
> > + cc.append(fields[1])
> > + elif fields[0] == 'F:':
> > + paths.append(fields[1].strip())
> > +
> > + # Expannd any wildcards in the MAINTAINRERS file
>
> Couple of typos s/Expannd/Expand/ and s/MAINTAINRERS/MAINTAINERS/
>
OK will fix, thanks.
Regards,
Simon
^ permalink raw reply [flat|nested] 26+ messages in thread
* [U-Boot] [PATCH 00/21] patman: Update to support Python 3
2019-05-15 8:01 ` [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Chris Packham
@ 2019-06-24 17:17 ` Simon Glass
0 siblings, 0 replies; 26+ messages in thread
From: Simon Glass @ 2019-06-24 17:17 UTC (permalink / raw)
To: u-boot
On Wed, 15 May 2019 at 02:02, Chris Packham <judge.packham@gmail.com> wrote:
>
> On Wed, May 15, 2019 at 9:54 AM Simon Glass <sjg@chromium.org> wrote:
> >
> > This series updates patman to support Python 3:
> >
> > - Avoid using 'unicode' type directly
> > - Use items() instead of iteritems()
> > - Make sure file I/O uses binary mode where necessary
> > - Change print statements to functions
> > - Use the built-in set() class
> > - Fix up generation of repeated bytes
> >
> > A few patches for binman are included, but this still requires Python 2.
> >
>
> Couple of comments on 14/21 but the rest of the series
>
> Reviewed-by: Chris Packham <judge.packham@gmail.com>
>
> >
> > Simon Glass (21):
> > patman: Update cros_subprocess to use bytearray
> > patman: Convert print statements to Python 3
> > binman: Convert print statements to Python 3
> > binman: Don't show errors for failed tests
> > binman: Remove use of Set()
> > patman: Use items() instead of iteritems()
> > binman: Use items() instead of iteritems()
> > tools: binman: Open all binary files in binary mode
> > tools: dtoc: Open all binary files in binary mode
> > patman: Provide a way to get program output in binary mode
> > binman: Use binary mode when compressing data
> > binman: Drop an unused input file
> > binman: Handle repeated bytes for Python 3
> > Add a simple script to remove boards
> > patman: Support use of stringIO in Python 3
> > patman: Move unicode helpers to tools
> > patman: Sort series output for repeatabily
> > patman: Avoid unicode type in settings unit tests
> > patman: Adjust functional tests for Python 3
> > patman: Tidy up a few more unicode conversions
> > patman: Don't require Python 2
> >
> > tools/binman/binman.py | 26 +++-
> > tools/binman/bsection.py | 7 +-
> > tools/binman/control.py | 8 +-
> > tools/binman/elf.py | 4 +-
> > tools/binman/elf_test.py | 2 +-
> > tools/binman/entry.py | 5 +-
> > tools/binman/etype/blob.py | 2 +-
> > tools/binman/etype/fill.py | 4 +-
> > tools/binman/etype/gbb.py | 2 +-
> > tools/binman/etype/u_boot_spl_bss_pad.py | 2 +-
> > tools/binman/ftest.py | 81 ++++++------
> > tools/binman/state.py | 7 +-
> > tools/dtoc/fdt.py | 2 +-
> > tools/patman/cros_subprocess.py | 53 +++++---
> > tools/patman/func_test.py | 41 ++++---
> > tools/patman/gitutil.py | 16 +--
> > tools/patman/patman.py | 2 +-
> > tools/patman/series.py | 20 +--
> > tools/patman/settings.py | 34 ++---
> > tools/patman/test_util.py | 16 +--
> > tools/patman/tools.py | 55 ++++++++-
> > tools/rmboard.py | 150 +++++++++++++++++++++++
> > 22 files changed, 385 insertions(+), 154 deletions(-)
> > create mode 100755 tools/rmboard.py
> >
> > --
> > 2.21.0.1020.gf2820cf01a-goog
> >
Applied series to u-boot-dm/next
Due to a patchwork bug, some patches show up with missing spaces in
the UI, e.g. here http://patchwork.ozlabs.org/bundle/sjg/dm/
So my script which emails on each patch doesn't work, sorry.
- Simon
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2019-06-24 17:17 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-14 21:53 [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 01/21] patman: Update cros_subprocess to use bytearray Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 02/21] patman: Convert print statements to Python 3 Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 03/21] binman: " Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 04/21] binman: Don't show errors for failed tests Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 05/21] binman: Remove use of Set() Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 06/21] patman: Use items() instead of iteritems() Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 07/21] binman: " Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 08/21] tools: binman: Open all binary files in binary mode Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 09/21] tools: dtoc: " Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 10/21] patman: Provide a way to get program output " Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 11/21] binman: Use binary mode when compressing data Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 12/21] binman: Drop an unused input file Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 13/21] binman: Handle repeated bytes for Python 3 Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 14/21] Add a simple script to remove boards Simon Glass
2019-05-15 7:53 ` Chris Packham
2019-05-18 16:06 ` Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 15/21] patman: Support use of stringIO in Python 3 Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 16/21] patman: Move unicode helpers to tools Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 17/21] patman: Sort series output for repeatabily Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 18/21] patman: Avoid unicode type in settings unit tests Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 19/21] patman: Adjust functional tests for Python 3 Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 20/21] patman: Tidy up a few more unicode conversions Simon Glass
2019-05-14 21:53 ` [U-Boot] [PATCH 21/21] patman: Don't require Python 2 Simon Glass
2019-05-15 8:01 ` [U-Boot] [PATCH 00/21] patman: Update to support Python 3 Chris Packham
2019-06-24 17:17 ` Simon Glass
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.