[U-Boot] [PATCH v2 0/5] tools: patman: Fixes to improve handling of various wrong usage

[U-Boot] [PATCH v2 0/5] tools: patman: Fixes to improve handling of various wrong usage

[Bin Meng]

This fixes several issues when abnormal tags are given to patman.
For example, when we forget to put an 'END' after 'Cover-letter',
patman does not handle such cases very well. This adds several
additional checking to let patman be a little more tolerant on
handling such exceptional paths.

This series is available at u-boot-x86/patman-working for testing.

Changes in v2:
- New patch to use cover_match for 'Cover-letter'
- Update commit message to explain when such scenario is hit
- Add cover_match to the test logic
- Generate warning for missing 'END' case
- Update commit message to explain when such case is hit
- Reset in_section, skip_blank and section variables too
- Geneate warning when blank line is not found in 'Series-changes'
- New patch to handle missing 'END' in non-last commit of a series

Bin Meng (5):
  tools: patman: Use cover_match for 'Cover-letter'
  tools: patman: Handle tag sections without an 'END'
  tools: patman: Generate cover letter correctly when 'END' is missing
  tools: patman: Handle missing blank line for 'Series-changes'
  tools: patman: Handle missing 'END' in non-last commit of a series

 tools/patman/patchstream.py | 38 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 37 insertions(+), 1 deletion(-)

-- 
2.7.4

[U-Boot] [PATCH v2 1/5] tools: patman: Use cover_match for 'Cover-letter'

[Bin Meng]

Like other patman tags, use a new variable cover_match to indicate
a match for 'Cover-letter'.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

---

Changes in v2:
- New patch to use cover_match for 'Cover-letter'

 tools/patman/patchstream.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/patman/patchstream.py b/tools/patman/patchstream.py
index 27d031e..2c4efc5 100644
--- a/tools/patman/patchstream.py
+++ b/tools/patman/patchstream.py
@@ -150,6 +150,7 @@ class PatchStream:
         # Handle state transition and skipping blank lines
         series_tag_match = re_series_tag.match(line)
         commit_tag_match = re_commit_tag.match(line)
+        cover_match = re_cover.match(line)
         cover_cc_match = re_cover_cc.match(line)
         signoff_match = re_signoff.match(line)
         tag_match = None
@@ -203,7 +204,7 @@ class PatchStream:
             self.skip_blank = False
 
         # Detect the start of a cover letter section
-        elif re_cover.match(line):
+        elif cover_match:
             self.in_section = 'cover'
             self.skip_blank = False
 
-- 
2.7.4

[U-Boot] [PATCH v2 2/5] tools: patman: Handle tag sections without an 'END'

[Bin Meng]

'Cover-letter', 'Series-notes' and 'Commit-notes' tags require an
'END' to be put at the end of its section. If we forget to put an
'END' in those sections, and these sections are followed by another
patman tag, patman generates incorrect patches. This adds codes to
handle such scenario.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>

---

Changes in v2:
- Update commit message to explain when such scenario is hit
- Add cover_match to the test logic
- Generate warning for missing 'END' case

 tools/patman/patchstream.py | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/tools/patman/patchstream.py b/tools/patman/patchstream.py
index 2c4efc5..ce8ffb8 100644
--- a/tools/patman/patchstream.py
+++ b/tools/patman/patchstream.py
@@ -169,6 +169,26 @@ class PatchStream:
         elif commit_match:
             self.state = STATE_MSG_HEADER
 
+        # If a tag is detected, but we are already in a section,
+        # this means 'END' is missing for that section, fix it up.
+        if series_tag_match or commit_tag_match or \
+           cover_match or cover_cc_match or signoff_match:
+            if self.in_section:
+                self.warn.append("Missing 'END' in section '%s'" % self.in_section)
+                if self.in_section == 'cover':
+                    self.series.cover = self.section
+                elif self.in_section == 'notes':
+                    if self.is_log:
+                        self.series.notes += self.section
+                elif self.in_section == 'commit-notes':
+                    if self.is_log:
+                        self.commit.notes += self.section
+                else:
+                    self.warn.append("Unknown section '%s'" % self.in_section)
+                self.in_section = None
+                self.skip_blank = True
+                self.section = []
+
         # If we are in a section, keep collecting lines until we see END
         if self.in_section:
             if line == 'END':
-- 
2.7.4

[U-Boot] [PATCH v2 3/5] tools: patman: Generate cover letter correctly when 'END' is missing

[Bin Meng]

If 'END' is missing in a 'Cover-letter' section, and that section
happens to show up at the very end of the commit message, and the
commit is the last commit of the series, patman fails to generate
cover letter for us. Handle this in CloseCommit of patchstream.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

---

Changes in v2:
- Update commit message to explain when such case is hit
- Reset in_section, skip_blank and section variables too

 tools/patman/patchstream.py | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tools/patman/patchstream.py b/tools/patman/patchstream.py
index ce8ffb8..9ae977f 100644
--- a/tools/patman/patchstream.py
+++ b/tools/patman/patchstream.py
@@ -112,6 +112,14 @@ class PatchStream:
         if self.commit and self.is_log:
             self.series.AddCommit(self.commit)
             self.commit = None
+        # If 'END' is missing in a 'Cover-letter' section, and that section
+        # happens to show up at the very end of the commit message, this is
+        # the chance for us to fix it up.
+        if self.in_section == 'cover' and self.is_log:
+            self.series.cover = self.section
+            self.in_section = None
+            self.skip_blank = True
+            self.section = []
 
     def ProcessLine(self, line):
         """Process a single line of a patch file or commit log
-- 
2.7.4

[U-Boot] [PATCH v2 4/5] tools: patman: Handle missing blank line for 'Series-changes'

[Bin Meng]

'Series-changes' uses blank line to indicate its end. If that is
missing, series internal state variable 'in_change' may be wrong.
Correct its state.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>

---

Changes in v2:
- Geneate warning when blank line is not found in 'Series-changes'

 tools/patman/patchstream.py | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/tools/patman/patchstream.py b/tools/patman/patchstream.py
index 9ae977f..0612612 100644
--- a/tools/patman/patchstream.py
+++ b/tools/patman/patchstream.py
@@ -177,10 +177,11 @@ class PatchStream:
         elif commit_match:
             self.state = STATE_MSG_HEADER
 
-        # If a tag is detected, but we are already in a section,
-        # this means 'END' is missing for that section, fix it up.
+        # If a tag is detected
         if series_tag_match or commit_tag_match or \
            cover_match or cover_cc_match or signoff_match:
+            # but we are already in a section, this means 'END' is missing
+            # for that section, fix it up.
             if self.in_section:
                 self.warn.append("Missing 'END' in section '%s'" % self.in_section)
                 if self.in_section == 'cover':
@@ -196,6 +197,11 @@ class PatchStream:
                 self.in_section = None
                 self.skip_blank = True
                 self.section = []
+            # but we are already in a change list, that means a blank line
+            # is missing, fix it up.
+            if self.in_change:
+                self.warn.append("Missing 'blank line' in section 'Series-changes'")
+                self.in_change = 0
 
         # If we are in a section, keep collecting lines until we see END
         if self.in_section:
-- 
2.7.4

[U-Boot] [PATCH v2 5/5] tools: patman: Handle missing 'END' in non-last commit of a series

[Bin Meng]

The following python error:

Traceback (most recent call last):
  File "./tools/patman/patman", line 144, in <module>
    series = patchstream.FixPatches(series, args)
  File "./tools/patman/patchstream.py", line 477, in FixPatches
    commit = series.commits[count]
IndexError: list index out of range

is seen when:

- 'END' is missing in those tags
- those tags are put in the last part in a commit message
- the commit is not the last commit of the series

Add testing logic to see if a new commit starts.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

---

Changes in v2:
- New patch to handle missing 'END' in non-last commit of a series

 tools/patman/patchstream.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/tools/patman/patchstream.py b/tools/patman/patchstream.py
index 0612612..69d5cfb 100644
--- a/tools/patman/patchstream.py
+++ b/tools/patman/patchstream.py
@@ -177,9 +177,10 @@ class PatchStream:
         elif commit_match:
             self.state = STATE_MSG_HEADER
 
-        # If a tag is detected
+        # If a tag is detected, or a new commit starts
         if series_tag_match or commit_tag_match or \
-           cover_match or cover_cc_match or signoff_match:
+           cover_match or cover_cc_match or signoff_match or \
+           self.state == STATE_MSG_HEADER:
             # but we are already in a section, this means 'END' is missing
             # for that section, fix it up.
             if self.in_section:
-- 
2.7.4

[U-Boot] [PATCH v2 1/5] tools: patman: Use cover_match for 'Cover-letter'

[Simon Glass]

On 26 June 2016 at 23:24, Bin Meng <bmeng.cn@gmail.com> wrote:
> Like other patman tags, use a new variable cover_match to indicate
> a match for 'Cover-letter'.
>
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
>
> ---
>
> Changes in v2:
> - New patch to use cover_match for 'Cover-letter'
>
>  tools/patman/patchstream.py | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)

Acked-by: Simon Glass <sjg@chromium.org>

[U-Boot] [PATCH v2 3/5] tools: patman: Generate cover letter correctly when 'END' is missing

[Simon Glass]

On 26 June 2016 at 23:24, Bin Meng <bmeng.cn@gmail.com> wrote:
> If 'END' is missing in a 'Cover-letter' section, and that section
> happens to show up at the very end of the commit message, and the
> commit is the last commit of the series, patman fails to generate
> cover letter for us. Handle this in CloseCommit of patchstream.
>
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
>
> ---
>
> Changes in v2:
> - Update commit message to explain when such case is hit
> - Reset in_section, skip_blank and section variables too

Acked-by: Simon Glass <sjg@chromium.org>

[U-Boot] [PATCH v2 5/5] tools: patman: Handle missing 'END' in non-last commit of a series

[Simon Glass]

On 26 June 2016 at 23:24, Bin Meng <bmeng.cn@gmail.com> wrote:
> The following python error:
>
> Traceback (most recent call last):
>   File "./tools/patman/patman", line 144, in <module>
>     series = patchstream.FixPatches(series, args)
>   File "./tools/patman/patchstream.py", line 477, in FixPatches
>     commit = series.commits[count]
> IndexError: list index out of range
>
> is seen when:
>
> - 'END' is missing in those tags
> - those tags are put in the last part in a commit message
> - the commit is not the last commit of the series
>
> Add testing logic to see if a new commit starts.
>
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
>
> ---
>
> Changes in v2:
> - New patch to handle missing 'END' in non-last commit of a series
>
>  tools/patman/patchstream.py | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)

Acked-by: Simon Glass <sjg@chromium.org>

[U-Boot] [PATCH v2 1/5] tools: patman: Use cover_match for 'Cover-letter'

[Simon Glass]

On 28 June 2016 at 21:27, Simon Glass <sjg@chromium.org> wrote:
> On 26 June 2016 at 23:24, Bin Meng <bmeng.cn@gmail.com> wrote:
>> Like other patman tags, use a new variable cover_match to indicate
>> a match for 'Cover-letter'.
>>
>> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
>>
>> ---
>>
>> Changes in v2:
>> - New patch to use cover_match for 'Cover-letter'
>>
>>  tools/patman/patchstream.py | 3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> Acked-by: Simon Glass <sjg@chromium.org>

Applied to u-boot-dm/next, thanks!

[U-Boot] [PATCH v2 2/5] tools: patman: Handle tag sections without an 'END'

[Simon Glass]

On 27 June 2016 at 00:24, Bin Meng <bmeng.cn@gmail.com> wrote:
> 'Cover-letter', 'Series-notes' and 'Commit-notes' tags require an
> 'END' to be put at the end of its section. If we forget to put an
> 'END' in those sections, and these sections are followed by another
> patman tag, patman generates incorrect patches. This adds codes to
> handle such scenario.
>
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
> Acked-by: Simon Glass <sjg@chromium.org>
>
> ---
>
> Changes in v2:
> - Update commit message to explain when such scenario is hit
> - Add cover_match to the test logic
> - Generate warning for missing 'END' case
>
>  tools/patman/patchstream.py | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
>

Applied to u-boot-dm/next, thanks!

[U-Boot] [PATCH v2 3/5] tools: patman: Generate cover letter correctly when 'END' is missing

[Simon Glass]

On 28 June 2016 at 21:27, Simon Glass <sjg@chromium.org> wrote:
> On 26 June 2016 at 23:24, Bin Meng <bmeng.cn@gmail.com> wrote:
>> If 'END' is missing in a 'Cover-letter' section, and that section
>> happens to show up at the very end of the commit message, and the
>> commit is the last commit of the series, patman fails to generate
>> cover letter for us. Handle this in CloseCommit of patchstream.
>>
>> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
>>
>> ---
>>
>> Changes in v2:
>> - Update commit message to explain when such case is hit
>> - Reset in_section, skip_blank and section variables too
>
> Acked-by: Simon Glass <sjg@chromium.org>

Applied to u-boot-dm/next, thanks!

[U-Boot] [PATCH v2 4/5] tools: patman: Handle missing blank line for 'Series-changes'

[Simon Glass]

On 27 June 2016 at 00:24, Bin Meng <bmeng.cn@gmail.com> wrote:
> 'Series-changes' uses blank line to indicate its end. If that is
> missing, series internal state variable 'in_change' may be wrong.
> Correct its state.
>
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
> Acked-by: Simon Glass <sjg@chromium.org>
>
> ---
>
> Changes in v2:
> - Geneate warning when blank line is not found in 'Series-changes'
>
>  tools/patman/patchstream.py | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)

Applied to u-boot-dm/next, thanks!

[U-Boot] [PATCH v2 5/5] tools: patman: Handle missing 'END' in non-last commit of a series

[Simon Glass]

On 28 June 2016 at 21:27, Simon Glass <sjg@chromium.org> wrote:
> On 26 June 2016 at 23:24, Bin Meng <bmeng.cn@gmail.com> wrote:
>> The following python error:
>>
>> Traceback (most recent call last):
>>   File "./tools/patman/patman", line 144, in <module>
>>     series = patchstream.FixPatches(series, args)
>>   File "./tools/patman/patchstream.py", line 477, in FixPatches
>>     commit = series.commits[count]
>> IndexError: list index out of range
>>
>> is seen when:
>>
>> - 'END' is missing in those tags
>> - those tags are put in the last part in a commit message
>> - the commit is not the last commit of the series
>>
>> Add testing logic to see if a new commit starts.
>>
>> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
>>
>> ---
>>
>> Changes in v2:
>> - New patch to handle missing 'END' in non-last commit of a series
>>
>>  tools/patman/patchstream.py | 5 +++--
>>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> Acked-by: Simon Glass <sjg@chromium.org>

Applied to u-boot-dm/next, thanks!