[PATCH] sideband: color lines with keyword only

[PATCH] sideband: color lines with keyword only

[Stefan Beller]

When bf1a11f0a1 (sideband: highlight keywords in remote sideband output,
2018-08-07) was introduced, it was carefully considered which strings
would be highlighted. However 59a255aef0 (sideband: do not read beyond
the end of input, 2018-08-18) brought in a regression that the original
did not test for. A line containing only the keyword and nothing else
("SUCCESS") should still be colored.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 sideband.c                          | 5 +++--
 t/t5409-colorize-remote-messages.sh | 2 ++
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/sideband.c b/sideband.c
index 368647acf8..7c3d33d3f8 100644
--- a/sideband.c
+++ b/sideband.c
@@ -87,7 +87,7 @@ static void maybe_colorize_sideband(struct strbuf *dest, const char *src, int n)
 		struct keyword_entry *p = keywords + i;
 		int len = strlen(p->keyword);
 
-		if (n <= len)
+		if (n < len)
 			continue;
 		/*
 		 * Match case insensitively, so we colorize output from existing
@@ -95,7 +95,8 @@ static void maybe_colorize_sideband(struct strbuf *dest, const char *src, int n)
 		 * messages. We only highlight the word precisely, so
 		 * "successful" stays uncolored.
 		 */
-		if (!strncasecmp(p->keyword, src, len) && !isalnum(src[len])) {
+		if (!strncasecmp(p->keyword, src, len) &&
+		    (len == n || !isalnum(src[len]))) {
 			strbuf_addstr(dest, p->color);
 			strbuf_add(dest, src, len);
 			strbuf_addstr(dest, GIT_COLOR_RESET);
diff --git a/t/t5409-colorize-remote-messages.sh b/t/t5409-colorize-remote-messages.sh
index f81b6813c0..2a8c449661 100755
--- a/t/t5409-colorize-remote-messages.sh
+++ b/t/t5409-colorize-remote-messages.sh
@@ -17,6 +17,7 @@ test_expect_success 'setup' '
 	echo " " "error: leading space"
 	echo "    "
 	echo Err
+	echo SUCCESS
 	exit 0
 	EOF
 	echo 1 >file &&
@@ -35,6 +36,7 @@ test_expect_success 'keywords' '
 	grep "<BOLD;RED>error<RESET>: error" decoded &&
 	grep "<YELLOW>hint<RESET>:" decoded &&
 	grep "<BOLD;GREEN>success<RESET>:" decoded &&
+	grep "<BOLD;GREEN>SUCCESS<RESET>" decoded &&
 	grep "<BOLD;YELLOW>warning<RESET>:" decoded
 '
 
-- 
2.20.0.rc2.403.gdbc3b29805-goog

Re: [PATCH] sideband: color lines with keyword only

[Jonathan Nieder]

Hi,

Stefan Beller wrote:

> When bf1a11f0a1 (sideband: highlight keywords in remote sideband output,
> 2018-08-07) was introduced, it was carefully considered which strings
> would be highlighted. However 59a255aef0 (sideband: do not read beyond
> the end of input, 2018-08-18) brought in a regression that the original
> did not test for. A line containing only the keyword and nothing else
> ("SUCCESS") should still be colored.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  sideband.c                          | 5 +++--
>  t/t5409-colorize-remote-messages.sh | 2 ++
>  2 files changed, 5 insertions(+), 2 deletions(-)

Thanks for writing this.

I was curious about what versions of Gerrit this is designed to
support (or in other words whether it's a bug fix or a feature).
Looking at examples like [1], it seems that Gerrit historically always
used "ERROR:" so the 59a255aef0 logic would work for it.  More
recently, [2] (ReceiveCommits: add a "SUCCESS" marker for successful
change updates, 2018-08-21) put SUCCESS on a line of its own.  That
puts this squarely in the new-feature category.

"success" on its own line is even less likely to be a false positive
than "success" followed by punctuation (for example a period marking
the end of a sentence).  So I like this change.

[1] https://gerrit-review.googlesource.com/c/gerrit/+/22361
[2] https://gerrit-review.googlesource.com/c/gerrit/+/193570

> diff --git a/sideband.c b/sideband.c
> index 368647acf8..7c3d33d3f8 100644
> --- a/sideband.c
> +++ b/sideband.c
> @@ -87,7 +87,7 @@ static void maybe_colorize_sideband(struct strbuf *dest, const char *src, int n)
>  		struct keyword_entry *p = keywords + i;
>  		int len = strlen(p->keyword);
>  
> -		if (n <= len)
> +		if (n < len)
>  			continue;

In the old code, we would escape early if 'n == len', but we didn't
need to.  If 'n == len', then

	src[len] == '\0'
	src .. &src[len-1] is a valid buffer to read from

so the strncasecmp and strbuf_add operations used in this function are
valid.  Good.

>  		/*
>  		 * Match case insensitively, so we colorize output from existing
> @@ -95,7 +95,8 @@ static void maybe_colorize_sideband(struct strbuf *dest, const char *src, int n)
>  		 * messages. We only highlight the word precisely, so
>  		 * "successful" stays uncolored.
>  		 */
> -		if (!strncasecmp(p->keyword, src, len) && !isalnum(src[len])) {
> +		if (!strncasecmp(p->keyword, src, len) &&
> +		    (len == n || !isalnum(src[len]))) {

Our custom isalnum treats '\0' as not alphanumeric (sane_ctype[0] ==
GIT_CNTRL) so this part of the patch is unnecessary.  That said, it's
good for clarity and defensive programming.

>  			strbuf_addstr(dest, p->color);
>  			strbuf_add(dest, src, len);
>  			strbuf_addstr(dest, GIT_COLOR_RESET);
> diff --git a/t/t5409-colorize-remote-messages.sh b/t/t5409-colorize-remote-messages.sh
> index f81b6813c0..2a8c449661 100755
> --- a/t/t5409-colorize-remote-messages.sh
> +++ b/t/t5409-colorize-remote-messages.sh
> @@ -17,6 +17,7 @@ test_expect_success 'setup' '
>  	echo " " "error: leading space"
>  	echo "    "
>  	echo Err
> +	echo SUCCESS
>  	exit 0
>  	EOF
>  	echo 1 >file &&
> @@ -35,6 +36,7 @@ test_expect_success 'keywords' '
>  	grep "<BOLD;RED>error<RESET>: error" decoded &&
>  	grep "<YELLOW>hint<RESET>:" decoded &&
>  	grep "<BOLD;GREEN>success<RESET>:" decoded &&
> +	grep "<BOLD;GREEN>SUCCESS<RESET>" decoded &&
>  	grep "<BOLD;YELLOW>warning<RESET>:" decoded
>  '

Nice tests.

The "hinting: not highlighted" example shows that we aren't
introducing false positives here, so the coverage seems sufficient.
It might be nice to include a line

	echo ERROR:

as well to match another idiom that Gerrit sometimes uses.

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

Thanks again for a pleasant read.

Re: [PATCH] sideband: color lines with keyword only

[Jonathan Nieder]

Jonathan Nieder wrote:
> Stefan Beller wrote:

>>  		/*
>>  		 * Match case insensitively, so we colorize output from existing
>> @@ -95,7 +95,8 @@ static void maybe_colorize_sideband(struct strbuf *dest, const char *src, int n)
>>  		 * messages. We only highlight the word precisely, so
>>  		 * "successful" stays uncolored.
>>  		 */
>> -		if (!strncasecmp(p->keyword, src, len) && !isalnum(src[len])) {
>> +		if (!strncasecmp(p->keyword, src, len) &&
>> +		    (len == n || !isalnum(src[len]))) {
>
> Our custom isalnum treats '\0' as not alphanumeric (sane_ctype[0] ==
> GIT_CNTRL) so this part of the patch is unnecessary.  That said, it's
> good for clarity and defensive programming.

Correction: I am being silly here.  src[len] can be '\0', '\n', or
'\r' --- it's not always '\0'.  And the contract of this function is
that src[len] could be anything.  Thanks for having handled it
correctly. :)

Jonathan

Re: [PATCH] sideband: color lines with keyword only

[Stefan Beller]

On Mon, Dec 3, 2018 at 3:23 PM Jonathan Nieder <jrnieder@gmail.com> wrote:

> I was curious about what versions of Gerrit this is designed to
> support (or in other words whether it's a bug fix or a feature).
> Looking at examples like [1], it seems that Gerrit historically always
> used "ERROR:" so the 59a255aef0 logic would work for it.  More
> recently, [2] (ReceiveCommits: add a "SUCCESS" marker for successful
> change updates, 2018-08-21) put SUCCESS on a line of its own.  That
> puts this squarely in the new-feature category.

Ooops. From the internal bug, I assumed this to be long standing Gerrit
behavior, which is why I sent it out in -rc to begin with.

> > --- a/sideband.c
> > +++ b/sideband.c
> > @@ -87,7 +87,7 @@ static void maybe_colorize_sideband(struct strbuf *dest, const char *src, int n)
> >               struct keyword_entry *p = keywords + i;
> >               int len = strlen(p->keyword);
> >
> > -             if (n <= len)
> > +             if (n < len)
> >                       continue;
>
> In the old code, we would escape early if 'n == len', but we didn't
> need to.  If 'n == len', then
>
>         src[len] == '\0'

src[len] could also be one of "\n\r", see the caller
recv_sideband for sidebase case 2.

>         src .. &src[len-1] is a valid buffer to read from
>
> so the strncasecmp and strbuf_add operations used in this function are
> valid.  Good.

Yes, they are all valid...

> > -             if (!strncasecmp(p->keyword, src, len) && !isalnum(src[len])) {
> > +             if (!strncasecmp(p->keyword, src, len) &&
> > +                 (len == n || !isalnum(src[len]))) {
>
> Our custom isalnum treats '\0' as not alphanumeric (sane_ctype[0] ==
> GIT_CNTRL) so this part of the patch is unnecessary.  That said, it's
> good for clarity and defensive programming.

... but here we need to check for src[len] for validity.

I made no assumptions about isalnum, but rather needed to shortcut
the condition, as accessing src[len] would be out of bounds, no?

>
> >                       strbuf_addstr(dest, p->color);
> >                       strbuf_add(dest, src, len);

unlike here (or the rest of the block), where len is used correctly.

Re: [PATCH] sideband: color lines with keyword only

[Jonathan Nieder]

Stefan Beller wrote:
> On Mon, Dec 3, 2018 at 3:23 PM Jonathan Nieder <jrnieder@gmail.com> wrote:

>> I was curious about what versions of Gerrit this is designed to
>> support (or in other words whether it's a bug fix or a feature).
>> Looking at examples like [1], it seems that Gerrit historically always
>> used "ERROR:" so the 59a255aef0 logic would work for it.  More
>> recently, [2] (ReceiveCommits: add a "SUCCESS" marker for successful
>> change updates, 2018-08-21) put SUCCESS on a line of its own.  That
>> puts this squarely in the new-feature category.
>
> Ooops. From the internal bug, I assumed this to be long standing Gerrit
> behavior, which is why I sent it out in -rc to begin with.

No worries.  Can't hurt for Junio to have a few patches to apply to
"pu" or "next" to practice using the release candidates. :)

[...]
>> In the old code, we would escape early if 'n == len', but we didn't
>> need to.  If 'n == len', then
>>
>>         src[len] == '\0'
>
> src[len] could also be one of "\n\r", see the caller
> recv_sideband for sidebase case 2.

Yes, I noticed too late[*].  Sorry for the noise.

The patch still looks good.

Jonathan

[*] https://public-inbox.org/git/20181203233439.GB157301@google.com/

Re: [PATCH] sideband: color lines with keyword only

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Stefan Beller wrote:
>> On Mon, Dec 3, 2018 at 3:23 PM Jonathan Nieder <jrnieder@gmail.com> wrote:
>
>>> I was curious about what versions of Gerrit this is designed to
>>> support (or in other words whether it's a bug fix or a feature).

Well, bf1a11f0 ("sideband: highlight keywords in remote sideband
output", 2018-08-07) clearly wanted to allow a keyword followed by
anything !isalnum() to be painted, and we accepted that change
because we thought it was a good idea, so anything that made a
keyword alone not to be painted is a bug, isn't it?  Whether output
lines from Gerrit benefits from this fix is a different matter, of
course.

> No worries.  Can't hurt for Junio to have a few patches to apply to
> "pu" or "next" to practice using the release candidates. :)

This change falls into "an obvious and small fix to a bug that went
unnoticed and is in an older release (2.19)" category, which is not
eligible for the upcoming release this late in the cycle.  I think
enough eyeballs looked at the change already, so let's not waste the
already-spent review braincycle and mark it as "Will merge to 'next'".

Re: [PATCH] sideband: color lines with keyword only

[Han-Wen Nienhuys]

On Tue, Dec 4, 2018 at 12:23 AM Jonathan Nieder <jrnieder@gmail.com> wrote:
> > When bf1a11f0a1 (sideband: highlight keywords in remote sideband output,
> > 2018-08-07) was introduced, it was carefully considered which strings
> > would be highlighted. However 59a255aef0 (sideband: do not read beyond
> > the end of input, 2018-08-18) brought in a regression that the original
> > did not test for. A line containing only the keyword and nothing else
> > ("SUCCESS") should still be colored.

I had intended SUCCESS on a line of its to be highlighted too, and
some earlier versions of my patch did that, but it regressed as the
patch was reworked.  The SUCCESS on a line of its own is a recent
behavior of Gerrit, and is live in Gerrit 2.16.

--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado