All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCHv5 00/17] Diff machine: highlight moved lines.
       [not found] <20170523024048.16879-1-sbeller@google.com/>
@ 2017-05-24 21:40 ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 01/17] diff: readability fix Stefan Beller
                     ` (17 more replies)
  0 siblings, 18 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

v5:
* removed the color passing to the submodule to make the tests pass again.
* fixed an indentation issue that was introduced from v3 -> v4.
* I merged it with origin/next and tests pass here.

Thanks,
Stefan

diff to v4:
diff --git a/diff.c b/diff.c
index 23e70d348e..1292d3c4ad 100644
--- a/diff.c
+++ b/diff.c
@@ -751,7 +751,7 @@ static void mark_color_as_moved(struct diff_options *o,
 }
 
 static void emit_diff_line(struct diff_options *o,
-				     struct diff_line *e)
+			   struct diff_line *e)
 {
 	const char *ws;
 	int has_trailing_newline, has_trailing_carriage_return;
@@ -804,7 +804,7 @@ static void emit_diff_line(struct diff_options *o,
 }
 
 static void append_diff_line(struct diff_options *o,
-				       struct diff_line *e)
+			     struct diff_line *e)
 {
 	struct diff_line *f;
 	ALLOC_GROW(o->line_buffer,
diff --git a/submodule.c b/submodule.c
index 428c996c97..19c63197fb 100644
--- a/submodule.c
+++ b/submodule.c
@@ -550,8 +550,6 @@ void show_submodule_inline_diff(struct diff_options *o, const char *path,
 
 	/* TODO: other options may need to be passed here. */
 	argv_array_push(&cp.args, "diff");
-	if (o->use_color)
-		argv_array_push(&cp.args, "--color=always");
 	argv_array_pushf(&cp.args, "--line-prefix=%s", diff_line_prefix(o));
 	if (DIFF_OPT_TST(o, REVERSE_DIFF)) {
 		argv_array_pushf(&cp.args, "--src-prefix=%s%s/",

v4:
* interdiff to v3 (what is currently origin/sb/diff-color-move) below.
* renamed the "buffered_patch_line" to "diff_line". Originally I planned
  to not carry the "line" part as it can be a piece of a line as well.
  But for the intended functionality it is best to keep the name.
  If we'd want to add more functionality to say have a move detection
  for words as well, we'd rename the struct to have a better name then.
  For now diff_line is the best. (Thanks Jonathan Nieder!)
* tests to demonstrate it doesn't mess with --color-words as well as
  submodules. (Thanks Jonathan Tan!)
* added in the statics (Thanks Ramsay!)
* smaller scope for the hashmaps (Thanks Jonathan Tan!)
* some commit messages were updated, prior patch 4-7 is squashed into one
  (Thanks Jonathan Tan!)
* the tests added revealed an actual fault: now that the submodule process
  is not attached to a dupe of our stdout, it would stop coloring the
  output. We need to pass on use-color explicitly.
* updated the NEEDSWORK comment in the second last patch.

Thanks for bearing,
Stefan

v3:
* see interdiff below.
* fixing one invalid computation (Thanks Junio!)
* I reasoned more about submodule and word diffing, see the commit message
  of the last patch:
  
    A note on the options '--submodule=diff' and '--color-words/--word-diff':
    In the conversion to use emit_line in the prior patches both submodules
    as well as word diff output carefully chose to call emit_line with sign=0.
    All output with sign=0 is ignored for move detection purposes in this
    patch, such that no weird looking output will be generated for these
    cases. This leads to another thought: We could pass on '--color-moved' to
    submodules such that they color up moved lines for themselves. If we'd do
    so only line moves within a repository boundary are marked up.

* better name for emit_line outside of diff.[ch]

v2:
* emit_line now takes an argument that indicates if we want it
  to emit the line prefix as well. This should allow for a more faithful
  refactoring in the beginning. (Thanks Jonathan!)
* fixed memleaks (Thanks Brandon!)
* "git -c color.moved=true log -p" works now! (Thanks Jeff)
* interdiff below, though it is large.
* less intrusive than v1 (Thanks Jonathan!)

v1:

For details on *why* see the commit message of the last commit.

The first five patches are slight refactorings to get into good
shape, the next patches are funneling all output through emit_line_*.

The second last patch introduces an option to buffer up all output
before printing, and then the last patch can color up moved lines
of code.

Any feedback welcome.

Thanks,
Stefan

Stefan Beller (17):
  diff: readability fix
  diff: move line ending check into emit_hunk_header
  diff.c: factor out diff_flush_patch_all_file_pairs
  diff: introduce more flexible emit function
  diff.c: convert fn_out_consume to use emit_line
  diff.c: convert builtin_diff to use emit_line_*
  diff.c: convert emit_rewrite_diff to use emit_line_*
  diff.c: convert emit_rewrite_lines to use emit_line_*
  submodule.c: convert show_submodule_summary to use emit_line_fmt
  diff.c: convert emit_binary_diff_body to use emit_line_*
  diff.c: convert show_stats to use emit_line_*
  diff.c: convert word diffing to use emit_line_*
  diff.c: convert diff_flush to use emit_line_*
  diff.c: convert diff_summary to use emit_line_*
  diff.c: emit_line includes whitespace highlighting
  diff: buffer all output if asked to
  diff.c: color moved lines differently

 Documentation/config.txt   |  14 +-
 diff.c                     | 858 +++++++++++++++++++++++++++++++++------------
 diff.h                     |  59 +++-
 submodule.c                |  87 ++---
 submodule.h                |   9 +-
 t/t4015-diff-whitespace.sh | 267 ++++++++++++++
 6 files changed, 1016 insertions(+), 278 deletions(-)

-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 01/17] diff: readability fix
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 02/17] diff: move line ending check into emit_hunk_header Stefan Beller
                     ` (16 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

We already have dereferenced 'p->two' into a local variable 'two'. Use
that.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/diff.c b/diff.c
index 74283d9001..3f5bf8b5a4 100644
--- a/diff.c
+++ b/diff.c
@@ -3283,8 +3283,8 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)
 	const char *other;
 	const char *attr_path;
 
-	name  = p->one->path;
-	other = (strcmp(name, p->two->path) ? p->two->path : NULL);
+	name  = one->path;
+	other = (strcmp(name, two->path) ? two->path : NULL);
 	attr_path = name;
 	if (o->prefix_length)
 		strip_prefix(o->prefix_length, &name, &other);
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 02/17] diff: move line ending check into emit_hunk_header
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 01/17] diff: readability fix Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 03/17] diff.c: factor out diff_flush_patch_all_file_pairs Stefan Beller
                     ` (15 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

The emit_hunk_header() function is responsible for assembling a
hunk header and calling emit_line() to send the hunk header
to the output file.  Its only caller fn_out_consume() needs
to prepare for a case where the function emits an incomplete
line and add the terminating LF.

Instead make sure emit_hunk_header() to always send a
completed line to emit_line().

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/diff.c b/diff.c
index 3f5bf8b5a4..c2ed605cd0 100644
--- a/diff.c
+++ b/diff.c
@@ -677,6 +677,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 	}
 
 	strbuf_add(&msgbuf, line + len, org_len - len);
+	strbuf_complete_line(&msgbuf);
+
 	emit_line(ecbdata->opt, "", "", msgbuf.buf, msgbuf.len);
 	strbuf_release(&msgbuf);
 }
@@ -1315,8 +1317,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 		len = sane_truncate_line(ecbdata, line, len);
 		find_lno(line, ecbdata);
 		emit_hunk_header(ecbdata, line, len);
-		if (line[len-1] != '\n')
-			putc('\n', o->file);
 		return;
 	}
 
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 03/17] diff.c: factor out diff_flush_patch_all_file_pairs
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 01/17] diff: readability fix Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 02/17] diff: move line ending check into emit_hunk_header Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 04/17] diff: introduce more flexible emit function Stefan Beller
                     ` (14 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

In a later patch we want to do more things before and after all filepairs
are flushed. So factor flushing out all file pairs into its own function
that the new code can be plugged in easily.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/diff.c b/diff.c
index c2ed605cd0..2f9722b382 100644
--- a/diff.c
+++ b/diff.c
@@ -4737,6 +4737,17 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)
 		warning(_(rename_limit_advice), varname, needed);
 }
 
+static void diff_flush_patch_all_file_pairs(struct diff_options *o)
+{
+	int i;
+	struct diff_queue_struct *q = &diff_queued_diff;
+	for (i = 0; i < q->nr; i++) {
+		struct diff_filepair *p = q->queue[i];
+		if (check_pair_status(p))
+			diff_flush_patch(p, o);
+	}
+}
+
 void diff_flush(struct diff_options *options)
 {
 	struct diff_queue_struct *q = &diff_queued_diff;
@@ -4831,11 +4842,7 @@ void diff_flush(struct diff_options *options)
 			}
 		}
 
-		for (i = 0; i < q->nr; i++) {
-			struct diff_filepair *p = q->queue[i];
-			if (check_pair_status(p))
-				diff_flush_patch(p, options);
-		}
+		diff_flush_patch_all_file_pairs(options);
 	}
 
 	if (output_format & DIFF_FORMAT_CALLBACK)
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 04/17] diff: introduce more flexible emit function
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (2 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 03/17] diff.c: factor out diff_flush_patch_all_file_pairs Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-06-13 21:54     ` Jonathan Tan
  2017-05-24 21:40   ` [PATCHv5 05/17] diff.c: convert fn_out_consume to use emit_line Stefan Beller
                     ` (13 subsequent siblings)
  17 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

Currently, diff output is written either through the emit_line_0
function or through the FILE * in struct diff_options directly. To
make it easier to teach diff to buffer its output (which will be done
in a subsequent commit), introduce a more flexible emit_line() function.
In this commit, direct usages of emit_line_0() are replaced with
emit_line(); subsequent commits will also replace usages of the
FILE * with emit().

Instead of having a 'first' parameter containing the first character
of a line, have a dedicated 'sign' parameter that is just set when
the first character of the line is part of the actual content, i.e.
' ', '+', '-'.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 76 +++++++++++++++++++++++++++++-------------------------------------
 1 file changed, 33 insertions(+), 43 deletions(-)

diff --git a/diff.c b/diff.c
index 2f9722b382..3569857818 100644
--- a/diff.c
+++ b/diff.c
@@ -516,36 +516,30 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,
 	ecbdata->blank_at_eof_in_postimage = (at - l2) + 1;
 }
 
-static void emit_line_0(struct diff_options *o, const char *set, const char *reset,
-			int first, const char *line, int len)
+static void emit_line(struct diff_options *o, const char *set, const char *reset,
+		      int add_line_prefix, int sign, const char *line, int len)
 {
 	int has_trailing_newline, has_trailing_carriage_return;
-	int nofirst;
 	FILE *file = o->file;
 
-	fputs(diff_line_prefix(o), file);
+	if (add_line_prefix)
+		fputs(diff_line_prefix(o), file);
 
-	if (len == 0) {
-		has_trailing_newline = (first == '\n');
-		has_trailing_carriage_return = (!has_trailing_newline &&
-						(first == '\r'));
-		nofirst = has_trailing_newline || has_trailing_carriage_return;
-	} else {
-		has_trailing_newline = (len > 0 && line[len-1] == '\n');
-		if (has_trailing_newline)
-			len--;
-		has_trailing_carriage_return = (len > 0 && line[len-1] == '\r');
-		if (has_trailing_carriage_return)
-			len--;
-		nofirst = 0;
-	}
+	has_trailing_newline = (len > 0 && line[len-1] == '\n');
+	if (has_trailing_newline)
+		len--;
+	has_trailing_carriage_return = (len > 0 && line[len-1] == '\r');
+	if (has_trailing_carriage_return)
+		len--;
 
-	if (len || !nofirst) {
-		fputs(set, file);
-		if (!nofirst)
-			fputc(first, file);
+	if (len || sign) {
+		if (set)
+			fputs(set, file);
+		if (sign)
+			fputc(sign, file);
 		fwrite(line, len, 1, file);
-		fputs(reset, file);
+		if (reset)
+			fputs(reset, file);
 	}
 	if (has_trailing_carriage_return)
 		fputc('\r', file);
@@ -553,12 +547,6 @@ static void emit_line_0(struct diff_options *o, const char *set, const char *res
 		fputc('\n', file);
 }
 
-static void emit_line(struct diff_options *o, const char *set, const char *reset,
-		      const char *line, int len)
-{
-	emit_line_0(o, set, reset, line[0], line+1, len-1);
-}
-
 static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)
 {
 	if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
@@ -587,13 +575,13 @@ static void emit_line_checked(const char *reset,
 	}
 
 	if (!ws)
-		emit_line_0(ecbdata->opt, set, reset, sign, line, len);
+		emit_line(ecbdata->opt, set, reset, 1, sign, line, len);
 	else if (sign == '+' && new_blank_line_at_eof(ecbdata, line, len))
 		/* Blank line at EOF - paint '+' as well */
-		emit_line_0(ecbdata->opt, ws, reset, sign, line, len);
+		emit_line(ecbdata->opt, ws, reset, 1, sign, line, len);
 	else {
 		/* Emit just the prefix, then the rest. */
-		emit_line_0(ecbdata->opt, set, reset, sign, "", 0);
+		emit_line(ecbdata->opt, set, reset, 1, sign, "", 0);
 		ws_check_emit(line, len, ecbdata->ws_rule,
 			      ecbdata->opt->file, set, reset, ws);
 	}
@@ -643,7 +631,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 	if (len < 10 ||
 	    memcmp(line, atat, 2) ||
 	    !(ep = memmem(line + 2, len - 2, atat, 2))) {
-		emit_line(ecbdata->opt, context, reset, line, len);
+		emit_line(ecbdata->opt, context, reset, 1, 0, line, len);
 		return;
 	}
 	ep += 2; /* skip over @@ */
@@ -679,7 +667,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 	strbuf_add(&msgbuf, line + len, org_len - len);
 	strbuf_complete_line(&msgbuf);
 
-	emit_line(ecbdata->opt, "", "", msgbuf.buf, msgbuf.len);
+	emit_line(ecbdata->opt, "", "", 1, 0, msgbuf.buf, msgbuf.len);
 	strbuf_release(&msgbuf);
 }
 
@@ -742,8 +730,8 @@ static void emit_rewrite_lines(struct emit_callback *ecb,
 		const char *context = diff_get_color(ecb->color_diff,
 						     DIFF_CONTEXT);
 		putc('\n', ecb->opt->file);
-		emit_line_0(ecb->opt, context, reset, '\\',
-			    nneof, strlen(nneof));
+		emit_line(ecb->opt, context, reset, 1, '\\',
+			  nneof, strlen(nneof));
 	}
 }
 
@@ -1341,7 +1329,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 		}
 		diff_words_flush(ecbdata);
 		if (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {
-			emit_line(o, context, reset, line, len);
+			emit_line(o, context, reset, 1, 0, line, len);
 			fputs("~\n", o->file);
 		} else {
 			/*
@@ -1353,7 +1341,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 			      line++;
 			      len--;
 			}
-			emit_line(o, context, reset, line, len);
+			emit_line(o, context, reset, 1, 0, line, len);
 		}
 		return;
 	}
@@ -1376,7 +1364,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 		/* incomplete line at the end */
 		ecbdata->lno_in_preimage++;
 		emit_line(o, diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),
-			  reset, line, len);
+			  reset, 1, 0, line, len);
 		break;
 	}
 }
@@ -2188,7 +2176,7 @@ static void checkdiff_consume(void *priv, char *line, unsigned long len)
 		fprintf(data->o->file, "%s%s:%d: %s.\n",
 			line_prefix, data->filename, data->lineno, err);
 		free(err);
-		emit_line(data->o, set, reset, line, 1);
+		emit_line(data->o, set, reset, 1, 0, line, 1);
 		ws_check_emit(line + 1, len - 1, data->ws_rule,
 			      data->o->file, set, reset, ws);
 	} else if (line[0] == ' ') {
@@ -4833,9 +4821,11 @@ void diff_flush(struct diff_options *options)
 
 	if (output_format & DIFF_FORMAT_PATCH) {
 		if (separator) {
-			fprintf(options->file, "%s%c",
-				diff_line_prefix(options),
-				options->line_termination);
+			char term[2];
+			term[0] = options->line_termination;
+			term[1] = '\0';
+
+			emit_line(options, NULL, NULL, 1, 0, term, !!term[0]);
 			if (options->stat_sep) {
 				/* attach patch instead of inline */
 				fputs(options->stat_sep, options->file);
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 05/17] diff.c: convert fn_out_consume to use emit_line
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (3 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 04/17] diff: introduce more flexible emit function Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 06/17] diff.c: convert builtin_diff to use emit_line_* Stefan Beller
                     ` (12 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

In a later patch, I want to propose an option to detect&color
moved lines in a diff, which cannot be done in a one-pass over
the diff. Instead we need to go over the whole diff twice,
because we cannot detect the first line of the two corresponding
lines (+ and -) that got moved.

So to prepare the diff machinery for two pass algorithms
(i.e. buffer it all up and then operate on the result),
move all emissions to places, such that the only emitting
function is emit_line.

This covers the parts of fn_out_consume.  In the next
patches we'll convert more functions that want to emit
formatted output, so we'd want to have a formatted emit
function. Add it here.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)

diff --git a/diff.c b/diff.c
index 3569857818..8186289734 100644
--- a/diff.c
+++ b/diff.c
@@ -547,6 +547,21 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 		fputc('\n', file);
 }
 
+static void emit_line_fmt(struct diff_options *o,
+			  const char *set, const char *reset,
+			  int add_line_prefix,
+			  const char *fmt, ...)
+{
+	struct strbuf sb = STRBUF_INIT;
+	va_list ap;
+	va_start(ap, fmt);
+	strbuf_vaddf(&sb, fmt, ap);
+	va_end(ap);
+
+	emit_line(o, set, reset, add_line_prefix, 0, sb.buf, sb.len);
+	strbuf_release(&sb);
+}
+
 static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)
 {
 	if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
@@ -1270,7 +1285,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 	const char *context = diff_get_color(ecbdata->color_diff, DIFF_CONTEXT);
 	const char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);
 	struct diff_options *o = ecbdata->opt;
-	const char *line_prefix = diff_line_prefix(o);
 
 	o->found_changes = 1;
 
@@ -1282,14 +1296,12 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 
 	if (ecbdata->label_path[0]) {
 		const char *name_a_tab, *name_b_tab;
-
 		name_a_tab = strchr(ecbdata->label_path[0], ' ') ? "\t" : "";
 		name_b_tab = strchr(ecbdata->label_path[1], ' ') ? "\t" : "";
-
-		fprintf(o->file, "%s%s--- %s%s%s\n",
-			line_prefix, meta, ecbdata->label_path[0], reset, name_a_tab);
-		fprintf(o->file, "%s%s+++ %s%s%s\n",
-			line_prefix, meta, ecbdata->label_path[1], reset, name_b_tab);
+		emit_line_fmt(o, meta, reset, 1, "--- %s%s\n",
+			      ecbdata->label_path[0], name_a_tab);
+		emit_line_fmt(o, meta, reset, 1, "+++ %s%s\n",
+			      ecbdata->label_path[1], name_b_tab);
 		ecbdata->label_path[0] = ecbdata->label_path[1] = NULL;
 	}
 
@@ -1330,7 +1342,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 		diff_words_flush(ecbdata);
 		if (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {
 			emit_line(o, context, reset, 1, 0, line, len);
-			fputs("~\n", o->file);
+			emit_line(o, NULL, NULL, 0, 0, "~\n", 2);
 		} else {
 			/*
 			 * Skip the prefix character, if any.  With
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 06/17] diff.c: convert builtin_diff to use emit_line_*
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (4 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 05/17] diff.c: convert fn_out_consume to use emit_line Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 07/17] diff.c: convert emit_rewrite_diff " Stefan Beller
                     ` (11 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

In a later patch, I want to propose an option to detect&color
moved lines in a diff, which cannot be done in a one-pass over
the diff. Instead we need to go over the whole diff twice,
because we cannot detect the first line of the two corresponding
lines (+ and -) that got moved.

So to prepare the diff machinery for two pass algorithms
(i.e. buffer it all up and then operate on the result),
move all emissions to places, such that the only emitting
function is emit_line_0.

This covers builtin_diff.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 33 ++++++++++++++++++++-------------
 1 file changed, 20 insertions(+), 13 deletions(-)

diff --git a/diff.c b/diff.c
index 8186289734..4fa976d43c 100644
--- a/diff.c
+++ b/diff.c
@@ -1289,8 +1289,9 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 	o->found_changes = 1;
 
 	if (ecbdata->header) {
-		fprintf(o->file, "%s", ecbdata->header->buf);
-		strbuf_reset(ecbdata->header);
+		emit_line(o, NULL, NULL, 0, 0,
+			  ecbdata->header->buf, ecbdata->header->len);
+		strbuf_release(ecbdata->header);
 		ecbdata->header = NULL;
 	}
 
@@ -2435,7 +2436,7 @@ static void builtin_diff(const char *name_a,
 		if (complete_rewrite &&
 		    (textconv_one || !diff_filespec_is_binary(one)) &&
 		    (textconv_two || !diff_filespec_is_binary(two))) {
-			fprintf(o->file, "%s", header.buf);
+			emit_line(o, NULL, NULL, 0, 0, header.buf, header.len);
 			strbuf_reset(&header);
 			emit_rewrite_diff(name_a, name_b, one, two,
 						textconv_one, textconv_two, o);
@@ -2445,7 +2446,7 @@ static void builtin_diff(const char *name_a,
 	}
 
 	if (o->irreversible_delete && lbl[1][0] == '/') {
-		fprintf(o->file, "%s", header.buf);
+		emit_line(o, NULL, NULL, 0, 0, header.buf, header.len);
 		strbuf_reset(&header);
 		goto free_ab_and_return;
 	} else if (!DIFF_OPT_TST(o, TEXT) &&
@@ -2456,12 +2457,15 @@ static void builtin_diff(const char *name_a,
 		    !DIFF_OPT_TST(o, BINARY)) {
 			if (!oidcmp(&one->oid, &two->oid)) {
 				if (must_show_header)
-					fprintf(o->file, "%s", header.buf);
+					emit_line(o, NULL, NULL, 0, 0,
+						  header.buf, header.len);
 				goto free_ab_and_return;
 			}
-			fprintf(o->file, "%s", header.buf);
-			fprintf(o->file, "%sBinary files %s and %s differ\n",
-				line_prefix, lbl[0], lbl[1]);
+			emit_line(o, NULL, NULL, 0, 0,
+				  header.buf, header.len);
+			emit_line_fmt(o, NULL, NULL, 1,
+				      "Binary files %s and %s differ\n",
+				      lbl[0], lbl[1]);
 			goto free_ab_and_return;
 		}
 		if (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)
@@ -2470,16 +2474,19 @@ static void builtin_diff(const char *name_a,
 		if (mf1.size == mf2.size &&
 		    !memcmp(mf1.ptr, mf2.ptr, mf1.size)) {
 			if (must_show_header)
-				fprintf(o->file, "%s", header.buf);
+				emit_line(o, NULL, NULL, 0, 0,
+					  header.buf, header.len);
 			goto free_ab_and_return;
 		}
-		fprintf(o->file, "%s", header.buf);
+		emit_line(o, NULL, NULL, 0, 0,
+			  header.buf, header.len);
 		strbuf_reset(&header);
 		if (DIFF_OPT_TST(o, BINARY))
 			emit_binary_diff(o->file, &mf1, &mf2, line_prefix);
 		else
-			fprintf(o->file, "%sBinary files %s and %s differ\n",
-				line_prefix, lbl[0], lbl[1]);
+			emit_line_fmt(o, NULL, NULL, 1,
+				      "Binary files %s and %s differ\n",
+				      lbl[0], lbl[1]);
 		o->found_changes = 1;
 	} else {
 		/* Crazy xdl interfaces.. */
@@ -2491,7 +2498,7 @@ static void builtin_diff(const char *name_a,
 		const struct userdiff_funcname *pe;
 
 		if (must_show_header) {
-			fprintf(o->file, "%s", header.buf);
+			emit_line(o, NULL, NULL, 0, 0, header.buf, header.len);
 			strbuf_reset(&header);
 		}
 
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 07/17] diff.c: convert emit_rewrite_diff to use emit_line_*
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (5 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 06/17] diff.c: convert builtin_diff to use emit_line_* Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 08/17] diff.c: convert emit_rewrite_lines " Stefan Beller
                     ` (10 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

In a later patch, I want to propose an option to detect&color
moved lines in a diff, which cannot be done in a one-pass over
the diff. Instead we need to go over the whole diff twice,
because we cannot detect the first line of the two corresponding
lines (+ and -) that got moved.

So to prepare the diff machinery for two pass algorithms
(i.e. buffer it all up and then operate on the result),
move all emissions to places, such that the only emitting
function is emit_line_0.

This covers emit_rewrite_diff.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 33 ++++++++++++++++++---------------
 1 file changed, 18 insertions(+), 15 deletions(-)

diff --git a/diff.c b/diff.c
index 4fa976d43c..3dda9f3c8e 100644
--- a/diff.c
+++ b/diff.c
@@ -704,17 +704,17 @@ static void remove_tempfile(void)
 	}
 }
 
-static void print_line_count(FILE *file, int count)
+static void add_line_count(struct strbuf *out, int count)
 {
 	switch (count) {
 	case 0:
-		fprintf(file, "0,0");
+		strbuf_addstr(out, "0,0");
 		break;
 	case 1:
-		fprintf(file, "1");
+		strbuf_addstr(out, "1");
 		break;
 	default:
-		fprintf(file, "1,%d", count);
+		strbuf_addf(out, "1,%d", count);
 		break;
 	}
 }
@@ -768,7 +768,7 @@ static void emit_rewrite_diff(const char *name_a,
 	char *data_one, *data_two;
 	size_t size_one, size_two;
 	struct emit_callback ecbdata;
-	const char *line_prefix = diff_line_prefix(o);
+	struct strbuf out = STRBUF_INIT;
 
 	if (diff_mnemonic_prefix && DIFF_OPT_TST(o, REVERSE_DIFF)) {
 		a_prefix = o->b_prefix;
@@ -806,20 +806,23 @@ static void emit_rewrite_diff(const char *name_a,
 	ecbdata.lno_in_preimage = 1;
 	ecbdata.lno_in_postimage = 1;
 
+	emit_line_fmt(o, metainfo, reset, 1, "--- %s%s\n", a_name.buf, name_a_tab);
+	emit_line_fmt(o, metainfo, reset, 1, "+++ %s%s\n", b_name.buf, name_b_tab);
+
 	lc_a = count_lines(data_one, size_one);
 	lc_b = count_lines(data_two, size_two);
-	fprintf(o->file,
-		"%s%s--- %s%s%s\n%s%s+++ %s%s%s\n%s%s@@ -",
-		line_prefix, metainfo, a_name.buf, name_a_tab, reset,
-		line_prefix, metainfo, b_name.buf, name_b_tab, reset,
-		line_prefix, fraginfo);
+
+	strbuf_addstr(&out, "@@ -");
 	if (!o->irreversible_delete)
-		print_line_count(o->file, lc_a);
+		add_line_count(&out, lc_a);
 	else
-		fprintf(o->file, "?,?");
-	fprintf(o->file, " +");
-	print_line_count(o->file, lc_b);
-	fprintf(o->file, " @@%s\n", reset);
+		strbuf_addstr(&out, "?,?");
+	strbuf_addstr(&out, " +");
+	add_line_count(&out, lc_b);
+	strbuf_addstr(&out, " @@\n");
+	emit_line(o, fraginfo, reset, 1, 0, out.buf, out.len);
+	strbuf_release(&out);
+
 	if (lc_a && !o->irreversible_delete)
 		emit_rewrite_lines(&ecbdata, '-', data_one, size_one);
 	if (lc_b)
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 08/17] diff.c: convert emit_rewrite_lines to use emit_line_*
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (6 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 07/17] diff.c: convert emit_rewrite_diff " Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 09/17] submodule.c: convert show_submodule_summary to use emit_line_fmt Stefan Beller
                     ` (9 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

In a later patch, I want to propose an option to detect&color
moved lines in a diff, which cannot be done in a one-pass over
the diff. Instead we need to go over the whole diff twice,
because we cannot detect the first line of the two corresponding
lines (+ and -) that got moved.

So to prepare the diff machinery for two pass algorithms
(i.e. buffer it all up and then operate on the result),
move all emissions to places, such that the only emitting
function is emit_line_0.

This covers emit_rewrite_lines.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)

diff --git a/diff.c b/diff.c
index 3dda9f3c8e..ca6b48cf49 100644
--- a/diff.c
+++ b/diff.c
@@ -722,15 +722,23 @@ static void add_line_count(struct strbuf *out, int count)
 static void emit_rewrite_lines(struct emit_callback *ecb,
 			       int prefix, const char *data, int size)
 {
-	const char *endp = NULL;
-	static const char *nneof = " No newline at end of file\n";
 	const char *reset = diff_get_color(ecb->color_diff, DIFF_RESET);
+	struct strbuf sb = STRBUF_INIT;
 
 	while (0 < size) {
 		int len;
 
-		endp = memchr(data, '\n', size);
-		len = endp ? (endp - data + 1) : size;
+		const char *endp = memchr(data, '\n', size);
+		if (endp)
+			len = endp - data + 1;
+		else {
+			strbuf_add(&sb, data, size);
+			strbuf_addch(&sb, '\n');
+			size = 0; /* to exit the loop. */
+
+			data = sb.buf;
+			len = sb.len;
+		}
 		if (prefix != '+') {
 			ecb->lno_in_preimage++;
 			emit_del_line(reset, ecb, data, len);
@@ -741,12 +749,13 @@ static void emit_rewrite_lines(struct emit_callback *ecb,
 		size -= len;
 		data += len;
 	}
-	if (!endp) {
+	if (sb.len) {
+		static const char *nneof = "\\ No newline at end of file\n";
 		const char *context = diff_get_color(ecb->color_diff,
 						     DIFF_CONTEXT);
-		putc('\n', ecb->opt->file);
-		emit_line(ecb->opt, context, reset, 1, '\\',
-			  nneof, strlen(nneof));
+		emit_line(ecb->opt, context, reset, 1, 0,
+			    nneof, strlen(nneof));
+		strbuf_release(&sb);
 	}
 }
 
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 09/17] submodule.c: convert show_submodule_summary to use emit_line_fmt
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (7 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 08/17] diff.c: convert emit_rewrite_lines " Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 10/17] diff.c: convert emit_binary_diff_body to use emit_line_* Stefan Beller
                     ` (8 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

In a later patch, I want to propose an option to detect&color
moved lines in a diff, which cannot be done in a one-pass over
the diff. Instead we need to go over the whole diff twice,
because we cannot detect the first line of the two corresponding
lines (+ and -) that got moved.

So to prepare the diff machinery for two pass algorithms
(i.e. buffer it all up and then operate on the result),
move all emissions to places, such that the only emitting
function is emit_line_0.

This prepares the code for submodules to go through the emit_line
function.

As the submodule process is no longer attached to the same stdout as
the superprojects process, one might imagine that we would need to
pass on the usage of colors explicitly as the subprocess can no longer
determine where the output is going to land eventually. But this is not
the case. Apparently coloring submodule diffs never worked, so defer
the submodule diff coloring to a future patch series.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c      | 14 ++++++----
 diff.h      |  3 +++
 submodule.c | 87 ++++++++++++++++++++++++++++++++-----------------------------
 submodule.h |  9 +++----
 4 files changed, 61 insertions(+), 52 deletions(-)

diff --git a/diff.c b/diff.c
index ca6b48cf49..3357c0fca3 100644
--- a/diff.c
+++ b/diff.c
@@ -562,6 +562,12 @@ static void emit_line_fmt(struct diff_options *o,
 	strbuf_release(&sb);
 }
 
+void diff_emit_line(struct diff_options *o, const char *set, const char *reset,
+		    const char *line, int len)
+{
+	emit_line(o, set, reset, 1, 0, line, len);
+}
+
 static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)
 {
 	if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
@@ -2384,8 +2390,7 @@ static void builtin_diff(const char *name_a,
 	    (!two->mode || S_ISGITLINK(two->mode))) {
 		const char *del = diff_get_color_opt(o, DIFF_FILE_OLD);
 		const char *add = diff_get_color_opt(o, DIFF_FILE_NEW);
-		show_submodule_summary(o->file, one->path ? one->path : two->path,
-				line_prefix,
+		show_submodule_summary(o, one->path ? one->path : two->path,
 				&one->oid, &two->oid,
 				two->dirty_submodule,
 				meta, del, add, reset);
@@ -2395,11 +2400,10 @@ static void builtin_diff(const char *name_a,
 		   (!two->mode || S_ISGITLINK(two->mode))) {
 		const char *del = diff_get_color_opt(o, DIFF_FILE_OLD);
 		const char *add = diff_get_color_opt(o, DIFF_FILE_NEW);
-		show_submodule_inline_diff(o->file, one->path ? one->path : two->path,
-				line_prefix,
+		show_submodule_inline_diff(o, one->path ? one->path : two->path,
 				&one->oid, &two->oid,
 				two->dirty_submodule,
-				meta, del, add, reset, o);
+				meta, del, add, reset);
 		return;
 	}
 
diff --git a/diff.h b/diff.h
index 5be1ee77a7..9ad546361a 100644
--- a/diff.h
+++ b/diff.h
@@ -188,6 +188,9 @@ struct diff_options {
 	int diff_path_counter;
 };
 
+void diff_emit_line(struct diff_options *o, const char *set, const char *reset,
+		    const char *line, int len);
+
 enum color_diff {
 	DIFF_RESET = 0,
 	DIFF_CONTEXT = 1,
diff --git a/submodule.c b/submodule.c
index d3299e29c0..19c63197fb 100644
--- a/submodule.c
+++ b/submodule.c
@@ -362,8 +362,8 @@ static int prepare_submodule_summary(struct rev_info *rev, const char *path,
 	return prepare_revision_walk(rev);
 }
 
-static void print_submodule_summary(struct rev_info *rev, FILE *f,
-		const char *line_prefix,
+static void print_submodule_summary(struct rev_info *rev,
+		struct diff_options *o,
 		const char *del, const char *add, const char *reset)
 {
 	static const char format[] = "  %m %s";
@@ -375,18 +375,12 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,
 		ctx.date_mode = rev->date_mode;
 		ctx.output_encoding = get_log_output_encoding();
 		strbuf_setlen(&sb, 0);
-		strbuf_addstr(&sb, line_prefix);
-		if (commit->object.flags & SYMMETRIC_LEFT) {
-			if (del)
-				strbuf_addstr(&sb, del);
-		}
-		else if (add)
-			strbuf_addstr(&sb, add);
 		format_commit_message(commit, format, &sb, &ctx);
-		if (reset)
-			strbuf_addstr(&sb, reset);
 		strbuf_addch(&sb, '\n');
-		fprintf(f, "%s", sb.buf);
+		if (commit->object.flags & SYMMETRIC_LEFT)
+			diff_emit_line(o, del, reset, sb.buf, sb.len);
+		else if (add)
+			diff_emit_line(o, add, reset, sb.buf, sb.len);
 	}
 	strbuf_release(&sb);
 }
@@ -413,8 +407,7 @@ void prepare_submodule_repo_env(struct argv_array *out)
  * attempt to lookup both the left and right commits and put them into the
  * left and right pointers.
  */
-static void show_submodule_header(FILE *f, const char *path,
-		const char *line_prefix,
+static void show_submodule_header(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
 		unsigned dirty_submodule, const char *meta,
 		const char *reset,
@@ -425,12 +418,17 @@ static void show_submodule_header(FILE *f, const char *path,
 	struct strbuf sb = STRBUF_INIT;
 	int fast_forward = 0, fast_backward = 0;
 
-	if (dirty_submodule & DIRTY_SUBMODULE_UNTRACKED)
-		fprintf(f, "%sSubmodule %s contains untracked content\n",
-			line_prefix, path);
-	if (dirty_submodule & DIRTY_SUBMODULE_MODIFIED)
-		fprintf(f, "%sSubmodule %s contains modified content\n",
-			line_prefix, path);
+	if (dirty_submodule & DIRTY_SUBMODULE_UNTRACKED) {
+		strbuf_addf(&sb, "Submodule %s contains untracked content\n", path);
+		diff_emit_line(o, NULL, NULL, sb.buf, sb.len);
+		strbuf_reset(&sb);
+	}
+
+	if (dirty_submodule & DIRTY_SUBMODULE_MODIFIED) {
+		strbuf_addf(&sb, "Submodule %s contains modified content\n", path);
+		diff_emit_line(o, NULL, NULL, sb.buf, sb.len);
+		strbuf_reset(&sb);
+	}
 
 	if (is_null_oid(one))
 		message = "(new submodule)";
@@ -472,21 +470,20 @@ static void show_submodule_header(FILE *f, const char *path,
 	}
 
 output_header:
-	strbuf_addf(&sb, "%s%sSubmodule %s ", line_prefix, meta, path);
+	strbuf_addf(&sb, "Submodule %s ", path);
 	strbuf_add_unique_abbrev(&sb, one->hash, DEFAULT_ABBREV);
 	strbuf_addstr(&sb, (fast_backward || fast_forward) ? ".." : "...");
 	strbuf_add_unique_abbrev(&sb, two->hash, DEFAULT_ABBREV);
 	if (message)
-		strbuf_addf(&sb, " %s%s\n", message, reset);
+		strbuf_addf(&sb, " %s\n", message);
 	else
-		strbuf_addf(&sb, "%s:%s\n", fast_backward ? " (rewind)" : "", reset);
-	fwrite(sb.buf, sb.len, 1, f);
+		strbuf_addf(&sb, "%s:\n", fast_backward ? " (rewind)" : "");
+	diff_emit_line(o, meta, reset, sb.buf, sb.len);
 
 	strbuf_release(&sb);
 }
 
-void show_submodule_summary(FILE *f, const char *path,
-		const char *line_prefix,
+void show_submodule_summary(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
 		unsigned dirty_submodule, const char *meta,
 		const char *del, const char *add, const char *reset)
@@ -495,7 +492,7 @@ void show_submodule_summary(FILE *f, const char *path,
 	struct commit *left = NULL, *right = NULL;
 	struct commit_list *merge_bases = NULL;
 
-	show_submodule_header(f, path, line_prefix, one, two, dirty_submodule,
+	show_submodule_header(o, path, one, two, dirty_submodule,
 			      meta, reset, &left, &right, &merge_bases);
 
 	/*
@@ -508,11 +505,12 @@ void show_submodule_summary(FILE *f, const char *path,
 
 	/* Treat revision walker failure the same as missing commits */
 	if (prepare_submodule_summary(&rev, path, left, right, merge_bases)) {
-		fprintf(f, "%s(revision walker failed)\n", line_prefix);
+		const char *error = "(revision walker failed)\n";
+		diff_emit_line(o, NULL, NULL, error, strlen(error));
 		goto out;
 	}
 
-	print_submodule_summary(&rev, f, line_prefix, del, add, reset);
+	print_submodule_summary(&rev, o, del, add, reset);
 
 out:
 	if (merge_bases)
@@ -521,20 +519,18 @@ void show_submodule_summary(FILE *f, const char *path,
 	clear_commit_marks(right, ~0);
 }
 
-void show_submodule_inline_diff(FILE *f, const char *path,
-		const char *line_prefix,
+void show_submodule_inline_diff(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
 		unsigned dirty_submodule, const char *meta,
-		const char *del, const char *add, const char *reset,
-		const struct diff_options *o)
+		const char *del, const char *add, const char *reset)
 {
 	const struct object_id *old = &empty_tree_oid, *new = &empty_tree_oid;
 	struct commit *left = NULL, *right = NULL;
 	struct commit_list *merge_bases = NULL;
-	struct strbuf submodule_dir = STRBUF_INIT;
 	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf sb = STRBUF_INIT;
 
-	show_submodule_header(f, path, line_prefix, one, two, dirty_submodule,
+	show_submodule_header(o, path, one, two, dirty_submodule,
 			      meta, reset, &left, &right, &merge_bases);
 
 	/* We need a valid left and right commit to display a difference */
@@ -547,15 +543,14 @@ void show_submodule_inline_diff(FILE *f, const char *path,
 	if (right)
 		new = two;
 
-	fflush(f);
 	cp.git_cmd = 1;
 	cp.dir = path;
-	cp.out = dup(fileno(f));
+	cp.out = -1;
 	cp.no_stdin = 1;
 
 	/* TODO: other options may need to be passed here. */
 	argv_array_push(&cp.args, "diff");
-	argv_array_pushf(&cp.args, "--line-prefix=%s", line_prefix);
+	argv_array_pushf(&cp.args, "--line-prefix=%s", diff_line_prefix(o));
 	if (DIFF_OPT_TST(o, REVERSE_DIFF)) {
 		argv_array_pushf(&cp.args, "--src-prefix=%s%s/",
 				 o->b_prefix, path);
@@ -578,11 +573,21 @@ void show_submodule_inline_diff(FILE *f, const char *path,
 		argv_array_push(&cp.args, oid_to_hex(new));
 
 	prepare_submodule_repo_env(&cp.env_array);
-	if (run_command(&cp))
-		fprintf(f, "(diff failed)\n");
+	if (start_command(&cp)) {
+		const char *error = "(diff failed)\n";
+		diff_emit_line(o, NULL, NULL, error, strlen(error));
+	}
+
+	while (strbuf_getwholeline_fd(&sb, cp.out, '\n') != EOF)
+		diff_emit_line(o, NULL, NULL, sb.buf, sb.len);
+
+	if (finish_command(&cp)) {
+		const char *error = "(diff failed)\n";
+		diff_emit_line(o, NULL, NULL, error, strlen(error));
+	}
 
 done:
-	strbuf_release(&submodule_dir);
+	strbuf_release(&sb);
 	if (merge_bases)
 		free_commit_list(merge_bases);
 	if (left)
diff --git a/submodule.h b/submodule.h
index 1277480add..9df0a3aea2 100644
--- a/submodule.h
+++ b/submodule.h
@@ -53,17 +53,14 @@ extern int parse_submodule_update_strategy(const char *value,
 		struct submodule_update_strategy *dst);
 extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);
 extern void handle_ignore_submodules_arg(struct diff_options *, const char *);
-extern void show_submodule_summary(FILE *f, const char *path,
-		const char *line_prefix,
+extern void show_submodule_summary(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
 		unsigned dirty_submodule, const char *meta,
 		const char *del, const char *add, const char *reset);
-extern void show_submodule_inline_diff(FILE *f, const char *path,
-		const char *line_prefix,
+extern void show_submodule_inline_diff(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
 		unsigned dirty_submodule, const char *meta,
-		const char *del, const char *add, const char *reset,
-		const struct diff_options *opt);
+		const char *del, const char *add, const char *reset);
 extern void set_config_fetch_recurse_submodules(int value);
 extern void set_config_update_recurse_submodules(int value);
 /* Check if we want to update any submodule.*/
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 10/17] diff.c: convert emit_binary_diff_body to use emit_line_*
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (8 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 09/17] submodule.c: convert show_submodule_summary to use emit_line_fmt Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 11/17] diff.c: convert show_stats " Stefan Beller
                     ` (7 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

In a later patch, I want to propose an option to detect&color
moved lines in a diff, which cannot be done in a one-pass over
the diff. Instead we need to go over the whole diff twice,
because we cannot detect the first line of the two corresponding
lines (+ and -) that got moved.

So to prepare the diff machinery for two pass algorithms
(i.e. buffer it all up and then operate on the result),
move all emissions to places, such that the only emitting
function is emit_line_0.

This covers emit_binary_diff_body.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 39 ++++++++++++++++++++++-----------------
 1 file changed, 22 insertions(+), 17 deletions(-)

diff --git a/diff.c b/diff.c
index 3357c0fca3..b5a5261a4e 100644
--- a/diff.c
+++ b/diff.c
@@ -2244,8 +2244,8 @@ static unsigned char *deflate_it(char *data,
 	return deflated;
 }
 
-static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
-				  const char *prefix)
+static void emit_binary_diff_body(struct diff_options *o,
+				  mmfile_t *one, mmfile_t *two)
 {
 	void *cp;
 	void *delta;
@@ -2274,13 +2274,12 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
 	}
 
 	if (delta && delta_size < deflate_size) {
-		fprintf(file, "%sdelta %lu\n", prefix, orig_size);
+		emit_line_fmt(o, NULL, NULL, 1, "delta %lu\n", orig_size);
 		free(deflated);
 		data = delta;
 		data_size = delta_size;
-	}
-	else {
-		fprintf(file, "%sliteral %lu\n", prefix, two->size);
+	} else {
+		emit_line_fmt(o, NULL, NULL, 1, "literal %lu\n", two->size);
 		free(delta);
 		data = deflated;
 		data_size = deflate_size;
@@ -2289,8 +2288,9 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
 	/* emit data encoded in base85 */
 	cp = data;
 	while (data_size) {
+		int len;
 		int bytes = (52 < data_size) ? 52 : data_size;
-		char line[70];
+		char line[71];
 		data_size -= bytes;
 		if (bytes <= 26)
 			line[0] = bytes + 'A' - 1;
@@ -2298,20 +2298,25 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
 			line[0] = bytes - 26 + 'a' - 1;
 		encode_85(line + 1, cp, bytes);
 		cp = (char *) cp + bytes;
-		fprintf(file, "%s", prefix);
-		fputs(line, file);
-		fputc('\n', file);
+
+		len = strlen(line);
+		line[len++] = '\n';
+		line[len] = '\0';
+
+		emit_line(o, NULL, NULL, 1, 0, line, len);
 	}
-	fprintf(file, "%s\n", prefix);
+	emit_line(o, NULL, NULL, 1, 0, "\n", 1);
 	free(data);
 }
 
-static void emit_binary_diff(FILE *file, mmfile_t *one, mmfile_t *two,
-			     const char *prefix)
+static void emit_binary_diff(struct diff_options *o,
+			     mmfile_t *one, mmfile_t *two)
 {
-	fprintf(file, "%sGIT binary patch\n", prefix);
-	emit_binary_diff_body(file, one, two, prefix);
-	emit_binary_diff_body(file, two, one, prefix);
+	const char *s = "GIT binary patch\n";
+	const int len = strlen(s);
+	emit_line(o, NULL, NULL, 1, 0, s, len);
+	emit_binary_diff_body(o, one, two);
+	emit_binary_diff_body(o, two, one);
 }
 
 int diff_filespec_is_binary(struct diff_filespec *one)
@@ -2498,7 +2503,7 @@ static void builtin_diff(const char *name_a,
 			  header.buf, header.len);
 		strbuf_reset(&header);
 		if (DIFF_OPT_TST(o, BINARY))
-			emit_binary_diff(o->file, &mf1, &mf2, line_prefix);
+			emit_binary_diff(o, &mf1, &mf2);
 		else
 			emit_line_fmt(o, NULL, NULL, 1,
 				      "Binary files %s and %s differ\n",
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 11/17] diff.c: convert show_stats to use emit_line_*
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (9 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 10/17] diff.c: convert emit_binary_diff_body to use emit_line_* Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 12/17] diff.c: convert word diffing " Stefan Beller
                     ` (6 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

In a later patch, I want to propose an option to detect&color
moved lines in a diff, which cannot be done in a one-pass over
the diff. Instead we need to go over the whole diff twice,
because we cannot detect the first line of the two corresponding
lines (+ and -) that got moved.

So to prepare the diff machinery for two pass algorithms
(i.e. buffer it all up and then operate on the result),
move all emissions to places, such that the only emitting
function is emit_line_0.

We call print_stat_summary from builtin/apply, so we still
need the version with a file pointer, so introduce
print_stat_summary_0 that uses emit_line_* machinery and
keep print_stat_summary with the same arguments around.

The responsibility to print the line prefix moves from the callers
of print_stat_summary_0 into the function itself.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 89 ++++++++++++++++++++++++++++++++++++++----------------------------
 diff.h |  4 +--
 2 files changed, 53 insertions(+), 40 deletions(-)

diff --git a/diff.c b/diff.c
index b5a5261a4e..a76abf5f69 100644
--- a/diff.c
+++ b/diff.c
@@ -1540,20 +1540,19 @@ static int scale_linear(int it, int width, int max_change)
 	return 1 + (it * (width - 1) / max_change);
 }
 
-static void show_name(FILE *file,
+static void show_name(struct strbuf *out,
 		      const char *prefix, const char *name, int len)
 {
-	fprintf(file, " %s%-*s |", prefix, len, name);
+	strbuf_addf(out, " %s%-*s |", prefix, len, name);
 }
 
-static void show_graph(FILE *file, char ch, int cnt, const char *set, const char *reset)
+static void show_graph(struct strbuf *out, char ch, int cnt, const char *set, const char *reset)
 {
 	if (cnt <= 0)
 		return;
-	fprintf(file, "%s", set);
-	while (cnt--)
-		putc(ch, file);
-	fprintf(file, "%s", reset);
+	strbuf_addstr(out, set);
+	strbuf_addchars(out, ch, cnt);
+	strbuf_addstr(out, reset);
 }
 
 static void fill_print_name(struct diffstat_file *file)
@@ -1577,14 +1576,16 @@ static void fill_print_name(struct diffstat_file *file)
 	file->print_name = pname;
 }
 
-int print_stat_summary(FILE *fp, int files, int insertions, int deletions)
+static void print_stat_summary_0(struct diff_options *options, int files,
+				 int insertions, int deletions)
 {
 	struct strbuf sb = STRBUF_INIT;
-	int ret;
 
 	if (!files) {
 		assert(insertions == 0 && deletions == 0);
-		return fprintf(fp, "%s\n", " 0 files changed");
+		strbuf_addstr(&sb, " 0 files changed");
+		emit_line(options, NULL, NULL, 1, 0, sb.buf, sb.len);
+		return;
 	}
 
 	strbuf_addf(&sb,
@@ -1611,9 +1612,17 @@ int print_stat_summary(FILE *fp, int files, int insertions, int deletions)
 			    deletions);
 	}
 	strbuf_addch(&sb, '\n');
-	ret = fputs(sb.buf, fp);
+	emit_line(options, NULL, NULL, 1, 0, sb.buf, sb.len);
 	strbuf_release(&sb);
-	return ret;
+}
+
+void print_stat_summary(FILE *fp, int files,
+			int insertions, int deletions)
+{
+	struct diff_options o;
+	memset(&o, 0, sizeof(o));
+	o.file = fp;
+	print_stat_summary_0(&o, files, insertions, deletions);
 }
 
 static void show_stats(struct diffstat_t *data, struct diff_options *options)
@@ -1623,13 +1632,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 	int total_files = data->nr, count;
 	int width, name_width, graph_width, number_width = 0, bin_width = 0;
 	const char *reset, *add_c, *del_c;
-	const char *line_prefix = "";
 	int extra_shown = 0;
+	const char *line_prefix = diff_line_prefix(options);
+	struct strbuf out = STRBUF_INIT;
 
 	if (data->nr == 0)
 		return;
 
-	line_prefix = diff_line_prefix(options);
 	count = options->stat_count ? options->stat_count : data->nr;
 
 	reset = diff_get_color_opt(options, DIFF_RESET);
@@ -1783,26 +1792,29 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 		}
 
 		if (file->is_binary) {
-			fprintf(options->file, "%s", line_prefix);
-			show_name(options->file, prefix, name, len);
-			fprintf(options->file, " %*s", number_width, "Bin");
+			show_name(&out, prefix, name, len);
+			strbuf_addf(&out, " %*s", number_width, "Bin");
 			if (!added && !deleted) {
-				putc('\n', options->file);
+				strbuf_addch(&out, '\n');
+				emit_line(options, NULL, NULL, 1, 0, out.buf, out.len);
+				strbuf_reset(&out);
 				continue;
 			}
-			fprintf(options->file, " %s%"PRIuMAX"%s",
+			strbuf_addf(&out, " %s%"PRIuMAX"%s",
 				del_c, deleted, reset);
-			fprintf(options->file, " -> ");
-			fprintf(options->file, "%s%"PRIuMAX"%s",
+			strbuf_addstr(&out, " -> ");
+			strbuf_addf(&out, "%s%"PRIuMAX"%s",
 				add_c, added, reset);
-			fprintf(options->file, " bytes");
-			fprintf(options->file, "\n");
+			strbuf_addstr(&out, " bytes\n");
+			emit_line(options, NULL, NULL, 1, 0, out.buf, out.len);
+			strbuf_reset(&out);
 			continue;
 		}
 		else if (file->is_unmerged) {
-			fprintf(options->file, "%s", line_prefix);
-			show_name(options->file, prefix, name, len);
-			fprintf(options->file, " Unmerged\n");
+			show_name(&out, prefix, name, len);
+			strbuf_addstr(&out, " Unmerged\n");
+			emit_line(options, NULL, NULL, 1, 0, out.buf, out.len);
+			strbuf_reset(&out);
 			continue;
 		}
 
@@ -1825,14 +1837,15 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 				add = total - del;
 			}
 		}
-		fprintf(options->file, "%s", line_prefix);
-		show_name(options->file, prefix, name, len);
-		fprintf(options->file, " %*"PRIuMAX"%s",
+		show_name(&out, prefix, name, len);
+		strbuf_addf(&out, " %*"PRIuMAX"%s",
 			number_width, added + deleted,
 			added + deleted ? " " : "");
-		show_graph(options->file, '+', add, add_c, reset);
-		show_graph(options->file, '-', del, del_c, reset);
-		fprintf(options->file, "\n");
+		show_graph(&out, '+', add, add_c, reset);
+		show_graph(&out, '-', del, del_c, reset);
+		strbuf_addch(&out, '\n');
+		emit_line(options, NULL, NULL, 1, 0, out.buf, out.len);
+		strbuf_reset(&out);
 	}
 
 	for (i = 0; i < data->nr; i++) {
@@ -1853,11 +1866,12 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 		if (i < count)
 			continue;
 		if (!extra_shown)
-			fprintf(options->file, "%s ...\n", line_prefix);
+			emit_line(options, NULL, NULL, 1, 0,
+				  " ...\n", strlen(" ...\n"));
 		extra_shown = 1;
 	}
-	fprintf(options->file, "%s", line_prefix);
-	print_stat_summary(options->file, total_files, adds, dels);
+
+	print_stat_summary_0(options, total_files, adds, dels);
 }
 
 static void show_shortstats(struct diffstat_t *data, struct diff_options *options)
@@ -1869,7 +1883,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option
 
 	for (i = 0; i < data->nr; i++) {
 		int added = data->files[i]->added;
-		int deleted= data->files[i]->deleted;
+		int deleted = data->files[i]->deleted;
 
 		if (data->files[i]->is_unmerged ||
 		    (!data->files[i]->is_interesting && (added + deleted == 0))) {
@@ -1879,8 +1893,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option
 			dels += deleted;
 		}
 	}
-	fprintf(options->file, "%s", diff_line_prefix(options));
-	print_stat_summary(options->file, total_files, adds, dels);
+	print_stat_summary_0(options, total_files, adds, dels);
 }
 
 static void show_numstat(struct diffstat_t *data, struct diff_options *options)
diff --git a/diff.h b/diff.h
index 9ad546361a..56d8dd036e 100644
--- a/diff.h
+++ b/diff.h
@@ -392,8 +392,8 @@ extern int parse_rename_score(const char **cp_p);
 
 extern long parse_algorithm_value(const char *value);
 
-extern int print_stat_summary(FILE *fp, int files,
-			      int insertions, int deletions);
+extern void print_stat_summary(FILE *fp, int files,
+			       int insertions, int deletions);
 extern void setup_diff_pager(struct diff_options *);
 
 #endif /* DIFF_H */
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 12/17] diff.c: convert word diffing to use emit_line_*
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (10 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 11/17] diff.c: convert show_stats " Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 13/17] diff.c: convert diff_flush " Stefan Beller
                     ` (5 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

In a later patch, I want to propose an option to detect&color
moved lines in a diff, which cannot be done in a one-pass over
the diff. Instead we need to go over the whole diff twice,
because we cannot detect the first line of the two corresponding
lines (+ and -) that got moved.

So to prepare the diff machinery for two pass algorithms
(i.e. buffer it all up and then operate on the result),
move all emissions to places, such that the only emitting
function is emit_line_0.

This covers all code related to diffing words.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 73 +++++++++++++++++++++++++++++-------------------------------------
 1 file changed, 32 insertions(+), 41 deletions(-)

diff --git a/diff.c b/diff.c
index a76abf5f69..8317824963 100644
--- a/diff.c
+++ b/diff.c
@@ -897,37 +897,42 @@ struct diff_words_data {
 	struct diff_words_style *style;
 };
 
-static int fn_out_diff_words_write_helper(FILE *fp,
+static int fn_out_diff_words_write_helper(struct diff_options *o,
 					  struct diff_words_style_elem *st_el,
 					  const char *newline,
-					  size_t count, const char *buf,
-					  const char *line_prefix)
+					  size_t count, const char *buf)
 {
 	int print = 0;
+	struct strbuf sb = STRBUF_INIT;
 
 	while (count) {
 		char *p = memchr(buf, '\n', count);
 		if (print)
-			fputs(line_prefix, fp);
+			emit_line(o, NULL, NULL, 1, 0, "", 0);
+
 		if (p != buf) {
-			if (st_el->color && fputs(st_el->color, fp) < 0)
-				return -1;
-			if (fputs(st_el->prefix, fp) < 0 ||
-			    fwrite(buf, p ? p - buf : count, 1, fp) != 1 ||
-			    fputs(st_el->suffix, fp) < 0)
-				return -1;
-			if (st_el->color && *st_el->color
-			    && fputs(GIT_COLOR_RESET, fp) < 0)
-				return -1;
+			const char *reset = st_el->color && *st_el->color ?
+					    GIT_COLOR_RESET : NULL;
+			strbuf_addstr(&sb, st_el->prefix);
+			strbuf_add(&sb, buf, p ? p - buf : count);
+			strbuf_addstr(&sb, st_el->suffix);
+			emit_line(o, st_el->color, reset,
+				  0, 0, sb.buf, sb.len);
+			strbuf_reset(&sb);
 		}
 		if (!p)
-			return 0;
-		if (fputs(newline, fp) < 0)
-			return -1;
+			goto out;
+
+		strbuf_addstr(&sb, newline);
+		emit_line(o, NULL, NULL, 0, 0, sb.buf, sb.len);
+		strbuf_reset(&sb);
 		count -= p + 1 - buf;
 		buf = p + 1;
 		print = 1;
 	}
+
+out:
+	strbuf_release(&sb);
 	return 0;
 }
 
@@ -981,14 +986,12 @@ static void fn_out_diff_words_aux(void *priv, char *line, unsigned long len)
 	int minus_first, minus_len, plus_first, plus_len;
 	const char *minus_begin, *minus_end, *plus_begin, *plus_end;
 	struct diff_options *opt = diff_words->opt;
-	const char *line_prefix;
 
 	if (line[0] != '@' || parse_hunk_header(line, len,
 			&minus_first, &minus_len, &plus_first, &plus_len))
 		return;
 
 	assert(opt);
-	line_prefix = diff_line_prefix(opt);
 
 	/* POSIX requires that first be decremented by one if len == 0... */
 	if (minus_len) {
@@ -1005,28 +1008,21 @@ static void fn_out_diff_words_aux(void *priv, char *line, unsigned long len)
 	} else
 		plus_begin = plus_end = diff_words->plus.orig[plus_first].end;
 
-	if (color_words_output_graph_prefix(diff_words)) {
-		fputs(line_prefix, diff_words->opt->file);
-	}
 	if (diff_words->current_plus != plus_begin) {
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+		fn_out_diff_words_write_helper(diff_words->opt,
 				&style->ctx, style->newline,
 				plus_begin - diff_words->current_plus,
-				diff_words->current_plus, line_prefix);
-		if (*(plus_begin - 1) == '\n')
-			fputs(line_prefix, diff_words->opt->file);
+				diff_words->current_plus);
 	}
 	if (minus_begin != minus_end) {
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+		fn_out_diff_words_write_helper(diff_words->opt,
 				&style->old, style->newline,
-				minus_end - minus_begin, minus_begin,
-				line_prefix);
+				minus_end - minus_begin, minus_begin);
 	}
 	if (plus_begin != plus_end) {
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+		fn_out_diff_words_write_helper(diff_words->opt,
 				&style->new, style->newline,
-				plus_end - plus_begin, plus_begin,
-				line_prefix);
+				plus_end - plus_begin, plus_begin);
 	}
 
 	diff_words->current_plus = plus_end;
@@ -1113,18 +1109,14 @@ static void diff_words_show(struct diff_words_data *diff_words)
 	struct diff_words_style *style = diff_words->style;
 
 	struct diff_options *opt = diff_words->opt;
-	const char *line_prefix;
-
 	assert(opt);
-	line_prefix = diff_line_prefix(opt);
 
 	/* special case: only removal */
 	if (!diff_words->plus.text.size) {
-		fputs(line_prefix, diff_words->opt->file);
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+		fn_out_diff_words_write_helper(diff_words->opt,
 			&style->old, style->newline,
 			diff_words->minus.text.size,
-			diff_words->minus.text.ptr, line_prefix);
+			diff_words->minus.text.ptr);
 		diff_words->minus.text.size = 0;
 		return;
 	}
@@ -1147,12 +1139,11 @@ static void diff_words_show(struct diff_words_data *diff_words)
 	if (diff_words->current_plus != diff_words->plus.text.ptr +
 			diff_words->plus.text.size) {
 		if (color_words_output_graph_prefix(diff_words))
-			fputs(line_prefix, diff_words->opt->file);
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+			emit_line(diff_words->opt, NULL, NULL, 1, 0, "", 0);
+		fn_out_diff_words_write_helper(diff_words->opt,
 			&style->ctx, style->newline,
 			diff_words->plus.text.ptr + diff_words->plus.text.size
-			- diff_words->current_plus, diff_words->current_plus,
-			line_prefix);
+			- diff_words->current_plus, diff_words->current_plus);
 	}
 	diff_words->minus.text.size = diff_words->plus.text.size = 0;
 }
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 13/17] diff.c: convert diff_flush to use emit_line_*
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (11 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 12/17] diff.c: convert word diffing " Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 14/17] diff.c: convert diff_summary " Stefan Beller
                     ` (4 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

In a later patch, I want to propose an option to detect&color
moved lines in a diff, which cannot be done in a one-pass over
the diff. Instead we need to go over the whole diff twice,
because we cannot detect the first line of the two corresponding
lines (+ and -) that got moved.

So to prepare the diff machinery for two pass algorithms
(i.e. buffer it all up and then operate on the result),
move all emissions to places, such that the only emitting
function is emit_line_0.

This covers diff_flush.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/diff.c b/diff.c
index 8317824963..8ebe673331 100644
--- a/diff.c
+++ b/diff.c
@@ -4872,7 +4872,9 @@ void diff_flush(struct diff_options *options)
 			emit_line(options, NULL, NULL, 1, 0, term, !!term[0]);
 			if (options->stat_sep) {
 				/* attach patch instead of inline */
-				fputs(options->stat_sep, options->file);
+				emit_line(options, NULL, NULL, 0, 0,
+					  options->stat_sep,
+					  strlen(options->stat_sep));
 			}
 		}
 
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 14/17] diff.c: convert diff_summary to use emit_line_*
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (12 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 13/17] diff.c: convert diff_flush " Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 15/17] diff.c: emit_line includes whitespace highlighting Stefan Beller
                     ` (3 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

In a later patch, I want to propose an option to detect&color
moved lines in a diff, which cannot be done in a one-pass over
the diff. Instead we need to go over the whole diff twice,
because we cannot detect the first line of the two corresponding
lines (+ and -) that got moved.

So to prepare the diff machinery for two pass algorithms
(i.e. buffer it all up and then operate on the result),
move all emissions to places, such that the only emitting
function is emit_line_0.

This covers diff_summary.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 64 ++++++++++++++++++++++++++++++++++------------------------------
 1 file changed, 34 insertions(+), 30 deletions(-)

diff --git a/diff.c b/diff.c
index 8ebe673331..76cafde4be 100644
--- a/diff.c
+++ b/diff.c
@@ -4504,67 +4504,71 @@ static void flush_one_pair(struct diff_filepair *p, struct diff_options *opt)
 	}
 }
 
-static void show_file_mode_name(FILE *file, const char *newdelete, struct diff_filespec *fs)
+static void show_file_mode_name(struct diff_options *opt, const char *newdelete, struct diff_filespec *fs)
 {
+	struct strbuf sb = STRBUF_INIT;
 	if (fs->mode)
-		fprintf(file, " %s mode %06o ", newdelete, fs->mode);
+		strbuf_addf(&sb, " %s mode %06o ", newdelete, fs->mode);
 	else
-		fprintf(file, " %s ", newdelete);
-	write_name_quoted(fs->path, file, '\n');
-}
+		strbuf_addf(&sb, " %s ", newdelete);
 
+	quote_c_style(fs->path, &sb, NULL, 0);
+	strbuf_addch(&sb, '\n');
+	emit_line(opt, NULL, NULL, 1, 0, sb.buf, sb.len);
+	strbuf_release(&sb);
+}
 
-static void show_mode_change(FILE *file, struct diff_filepair *p, int show_name,
-		const char *line_prefix)
+static void show_mode_change(struct diff_options *opt, struct diff_filepair *p,
+		int show_name)
 {
 	if (p->one->mode && p->two->mode && p->one->mode != p->two->mode) {
-		fprintf(file, "%s mode change %06o => %06o%c", line_prefix, p->one->mode,
-			p->two->mode, show_name ? ' ' : '\n');
+		struct strbuf sb = STRBUF_INIT;
 		if (show_name) {
-			write_name_quoted(p->two->path, file, '\n');
+			strbuf_addch(&sb, ' ');
+			quote_c_style(p->two->path, &sb, NULL, 0);
 		}
+		emit_line_fmt(opt, NULL, NULL, 1,
+			      " mode change %06o => %06o%s\n",
+			      p->one->mode, p->two->mode,
+			      show_name ? sb.buf : "");
+		strbuf_release(&sb);
 	}
 }
 
-static void show_rename_copy(FILE *file, const char *renamecopy, struct diff_filepair *p,
-			const char *line_prefix)
+static void show_rename_copy(struct diff_options *opt, const char *renamecopy,
+		struct diff_filepair *p)
 {
 	char *names = pprint_rename(p->one->path, p->two->path);
-
-	fprintf(file, " %s %s (%d%%)\n", renamecopy, names, similarity_index(p));
+	emit_line_fmt(opt, NULL, NULL, 1, " %s %s (%d%%)\n",
+		      renamecopy, names, similarity_index(p));
 	free(names);
-	show_mode_change(file, p, 0, line_prefix);
+	show_mode_change(opt, p, 0);
 }
 
 static void diff_summary(struct diff_options *opt, struct diff_filepair *p)
 {
-	FILE *file = opt->file;
-	const char *line_prefix = diff_line_prefix(opt);
-
 	switch(p->status) {
 	case DIFF_STATUS_DELETED:
-		fputs(line_prefix, file);
-		show_file_mode_name(file, "delete", p->one);
+		show_file_mode_name(opt, "delete", p->one);
 		break;
 	case DIFF_STATUS_ADDED:
-		fputs(line_prefix, file);
-		show_file_mode_name(file, "create", p->two);
+		show_file_mode_name(opt, "create", p->two);
 		break;
 	case DIFF_STATUS_COPIED:
-		fputs(line_prefix, file);
-		show_rename_copy(file, "copy", p, line_prefix);
+		show_rename_copy(opt, "copy", p);
 		break;
 	case DIFF_STATUS_RENAMED:
-		fputs(line_prefix, file);
-		show_rename_copy(file, "rename", p, line_prefix);
+		show_rename_copy(opt, "rename", p);
 		break;
 	default:
 		if (p->score) {
-			fprintf(file, "%s rewrite ", line_prefix);
-			write_name_quoted(p->two->path, file, ' ');
-			fprintf(file, "(%d%%)\n", similarity_index(p));
+			struct strbuf sb = STRBUF_INIT;
+			strbuf_addstr(&sb, " rewrite ");
+			quote_c_style(p->two->path, &sb, NULL, 0);
+			strbuf_addf(&sb, " (%d%%)\n", similarity_index(p));
+			emit_line(opt, NULL, NULL, 1, 0, sb.buf, sb.len);
 		}
-		show_mode_change(file, p, !p->score, line_prefix);
+		show_mode_change(opt, p, !p->score);
 		break;
 	}
 }
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 15/17] diff.c: emit_line includes whitespace highlighting
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (13 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 14/17] diff.c: convert diff_summary " Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-24 21:40   ` [PATCHv5 16/17] diff: buffer all output if asked to Stefan Beller
                     ` (2 subsequent siblings)
  17 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

Currently any whitespace highlighting happens outside the emit_line
function. Teach the highlighting to emit_line, triggered by a new
parameter.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 107 ++++++++++++++++++++++++++++++++++++++---------------------------
 diff.h |   2 ++
 2 files changed, 65 insertions(+), 44 deletions(-)

diff --git a/diff.c b/diff.c
index 76cafde4be..514c5facd7 100644
--- a/diff.c
+++ b/diff.c
@@ -516,15 +516,34 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,
 	ecbdata->blank_at_eof_in_postimage = (at - l2) + 1;
 }
 
-static void emit_line(struct diff_options *o, const char *set, const char *reset,
-		      int add_line_prefix, int sign, const char *line, int len)
+
+static void emit_line(struct diff_options *o,
+		      const char *set, const char *reset,
+		      int add_line_prefix, int markup_ws,
+		      int sign, const char *line, int len)
 {
+	const char *ws;
 	int has_trailing_newline, has_trailing_carriage_return;
 	FILE *file = o->file;
 
 	if (add_line_prefix)
 		fputs(diff_line_prefix(o), file);
 
+	if (markup_ws) {
+		ws = diff_get_color(o->use_color, DIFF_WHITESPACE);
+
+		if (set)
+			fputs(set, file);
+		if (sign)
+			fputc(sign, file);
+		if (reset)
+			fputs(reset, file);
+		ws = diff_get_color(o->use_color, DIFF_WHITESPACE);
+		ws_check_emit(line, len, o->ws_rule,
+			      file, set, reset, ws);
+		return;
+	}
+
 	has_trailing_newline = (len > 0 && line[len-1] == '\n');
 	if (has_trailing_newline)
 		len--;
@@ -558,14 +577,14 @@ static void emit_line_fmt(struct diff_options *o,
 	strbuf_vaddf(&sb, fmt, ap);
 	va_end(ap);
 
-	emit_line(o, set, reset, add_line_prefix, 0, sb.buf, sb.len);
+	emit_line(o, set, reset, add_line_prefix, 0, 0, sb.buf, sb.len);
 	strbuf_release(&sb);
 }
 
 void diff_emit_line(struct diff_options *o, const char *set, const char *reset,
 		    const char *line, int len)
 {
-	emit_line(o, set, reset, 1, 0, line, len);
+	emit_line(o, set, reset, 1, 0, 0, line, len);
 }
 
 static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)
@@ -596,16 +615,15 @@ static void emit_line_checked(const char *reset,
 	}
 
 	if (!ws)
-		emit_line(ecbdata->opt, set, reset, 1, sign, line, len);
+		emit_line(ecbdata->opt, set, reset, 1, 0, sign, line, len);
 	else if (sign == '+' && new_blank_line_at_eof(ecbdata, line, len))
 		/* Blank line at EOF - paint '+' as well */
-		emit_line(ecbdata->opt, ws, reset, 1, sign, line, len);
+		emit_line(ecbdata->opt, ws, reset, 1, 1, sign, line, len);
 	else {
 		/* Emit just the prefix, then the rest. */
-		emit_line(ecbdata->opt, set, reset, 1, sign, "", 0);
-		ws_check_emit(line, len, ecbdata->ws_rule,
-			      ecbdata->opt->file, set, reset, ws);
+		emit_line(ecbdata->opt, set, reset, 1, 1, sign, line, len);
 	}
+
 }
 
 static void emit_add_line(const char *reset,
@@ -652,7 +670,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 	if (len < 10 ||
 	    memcmp(line, atat, 2) ||
 	    !(ep = memmem(line + 2, len - 2, atat, 2))) {
-		emit_line(ecbdata->opt, context, reset, 1, 0, line, len);
+		emit_line(ecbdata->opt, context, reset, 1, 0, 0, line, len);
 		return;
 	}
 	ep += 2; /* skip over @@ */
@@ -688,7 +706,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 	strbuf_add(&msgbuf, line + len, org_len - len);
 	strbuf_complete_line(&msgbuf);
 
-	emit_line(ecbdata->opt, "", "", 1, 0, msgbuf.buf, msgbuf.len);
+	emit_line(ecbdata->opt, "", "", 1, 0, 0, msgbuf.buf, msgbuf.len);
 	strbuf_release(&msgbuf);
 }
 
@@ -759,7 +777,7 @@ static void emit_rewrite_lines(struct emit_callback *ecb,
 		static const char *nneof = "\\ No newline at end of file\n";
 		const char *context = diff_get_color(ecb->color_diff,
 						     DIFF_CONTEXT);
-		emit_line(ecb->opt, context, reset, 1, 0,
+		emit_line(ecb->opt, context, reset, 1, 0, 0,
 			    nneof, strlen(nneof));
 		strbuf_release(&sb);
 	}
@@ -835,7 +853,7 @@ static void emit_rewrite_diff(const char *name_a,
 	strbuf_addstr(&out, " +");
 	add_line_count(&out, lc_b);
 	strbuf_addstr(&out, " @@\n");
-	emit_line(o, fraginfo, reset, 1, 0, out.buf, out.len);
+	emit_line(o, fraginfo, reset, 1, 0, 0, out.buf, out.len);
 	strbuf_release(&out);
 
 	if (lc_a && !o->irreversible_delete)
@@ -908,7 +926,7 @@ static int fn_out_diff_words_write_helper(struct diff_options *o,
 	while (count) {
 		char *p = memchr(buf, '\n', count);
 		if (print)
-			emit_line(o, NULL, NULL, 1, 0, "", 0);
+			emit_line(o, NULL, NULL, 1, 0, 0, "", 0);
 
 		if (p != buf) {
 			const char *reset = st_el->color && *st_el->color ?
@@ -917,14 +935,14 @@ static int fn_out_diff_words_write_helper(struct diff_options *o,
 			strbuf_add(&sb, buf, p ? p - buf : count);
 			strbuf_addstr(&sb, st_el->suffix);
 			emit_line(o, st_el->color, reset,
-				  0, 0, sb.buf, sb.len);
+				  0, 0, 0, sb.buf, sb.len);
 			strbuf_reset(&sb);
 		}
 		if (!p)
 			goto out;
 
 		strbuf_addstr(&sb, newline);
-		emit_line(o, NULL, NULL, 0, 0, sb.buf, sb.len);
+		emit_line(o, NULL, NULL, 0, 0, 0, sb.buf, sb.len);
 		strbuf_reset(&sb);
 		count -= p + 1 - buf;
 		buf = p + 1;
@@ -1139,7 +1157,7 @@ static void diff_words_show(struct diff_words_data *diff_words)
 	if (diff_words->current_plus != diff_words->plus.text.ptr +
 			diff_words->plus.text.size) {
 		if (color_words_output_graph_prefix(diff_words))
-			emit_line(diff_words->opt, NULL, NULL, 1, 0, "", 0);
+			emit_line(diff_words->opt, NULL, NULL, 1, 0, 0, "", 0);
 		fn_out_diff_words_write_helper(diff_words->opt,
 			&style->ctx, style->newline,
 			diff_words->plus.text.ptr + diff_words->plus.text.size
@@ -1298,7 +1316,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 	o->found_changes = 1;
 
 	if (ecbdata->header) {
-		emit_line(o, NULL, NULL, 0, 0,
+		emit_line(o, NULL, NULL, 0, 0, 0,
 			  ecbdata->header->buf, ecbdata->header->len);
 		strbuf_release(ecbdata->header);
 		ecbdata->header = NULL;
@@ -1351,8 +1369,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 		}
 		diff_words_flush(ecbdata);
 		if (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {
-			emit_line(o, context, reset, 1, 0, line, len);
-			emit_line(o, NULL, NULL, 0, 0, "~\n", 2);
+			emit_line(o, context, reset, 1, 0, 0, line, len);
+			emit_line(o, NULL, NULL, 0, 0, 0, "~\n", 2);
 		} else {
 			/*
 			 * Skip the prefix character, if any.  With
@@ -1363,7 +1381,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 			      line++;
 			      len--;
 			}
-			emit_line(o, context, reset, 1, 0, line, len);
+			emit_line(o, context, reset, 1, 0, 0, line, len);
 		}
 		return;
 	}
@@ -1386,7 +1404,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 		/* incomplete line at the end */
 		ecbdata->lno_in_preimage++;
 		emit_line(o, diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),
-			  reset, 1, 0, line, len);
+			  reset, 1, 0, 0, line, len);
 		break;
 	}
 }
@@ -1575,7 +1593,7 @@ static void print_stat_summary_0(struct diff_options *options, int files,
 	if (!files) {
 		assert(insertions == 0 && deletions == 0);
 		strbuf_addstr(&sb, " 0 files changed");
-		emit_line(options, NULL, NULL, 1, 0, sb.buf, sb.len);
+		emit_line(options, NULL, NULL, 1, 0, 0, sb.buf, sb.len);
 		return;
 	}
 
@@ -1603,7 +1621,7 @@ static void print_stat_summary_0(struct diff_options *options, int files,
 			    deletions);
 	}
 	strbuf_addch(&sb, '\n');
-	emit_line(options, NULL, NULL, 1, 0, sb.buf, sb.len);
+	emit_line(options, NULL, NULL, 1, 0, 0, sb.buf, sb.len);
 	strbuf_release(&sb);
 }
 
@@ -1787,7 +1805,7 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 			strbuf_addf(&out, " %*s", number_width, "Bin");
 			if (!added && !deleted) {
 				strbuf_addch(&out, '\n');
-				emit_line(options, NULL, NULL, 1, 0, out.buf, out.len);
+				emit_line(options, NULL, NULL, 1, 0, 0, out.buf, out.len);
 				strbuf_reset(&out);
 				continue;
 			}
@@ -1797,14 +1815,14 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 			strbuf_addf(&out, "%s%"PRIuMAX"%s",
 				add_c, added, reset);
 			strbuf_addstr(&out, " bytes\n");
-			emit_line(options, NULL, NULL, 1, 0, out.buf, out.len);
+			emit_line(options, NULL, NULL, 1, 0, 0, out.buf, out.len);
 			strbuf_reset(&out);
 			continue;
 		}
 		else if (file->is_unmerged) {
 			show_name(&out, prefix, name, len);
 			strbuf_addstr(&out, " Unmerged\n");
-			emit_line(options, NULL, NULL, 1, 0, out.buf, out.len);
+			emit_line(options, NULL, NULL, 1, 0, 0, out.buf, out.len);
 			strbuf_reset(&out);
 			continue;
 		}
@@ -1835,7 +1853,7 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 		show_graph(&out, '+', add, add_c, reset);
 		show_graph(&out, '-', del, del_c, reset);
 		strbuf_addch(&out, '\n');
-		emit_line(options, NULL, NULL, 1, 0, out.buf, out.len);
+		emit_line(options, NULL, NULL, 1, 0, 0, out.buf, out.len);
 		strbuf_reset(&out);
 	}
 
@@ -1857,7 +1875,7 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 		if (i < count)
 			continue;
 		if (!extra_shown)
-			emit_line(options, NULL, NULL, 1, 0,
+			emit_line(options, NULL, NULL, 1, 0, 0,
 				  " ...\n", strlen(" ...\n"));
 		extra_shown = 1;
 	}
@@ -2211,7 +2229,7 @@ static void checkdiff_consume(void *priv, char *line, unsigned long len)
 		fprintf(data->o->file, "%s%s:%d: %s.\n",
 			line_prefix, data->filename, data->lineno, err);
 		free(err);
-		emit_line(data->o, set, reset, 1, 0, line, 1);
+		emit_line(data->o, set, reset, 1, 0, 0, line, 1);
 		ws_check_emit(line + 1, len - 1, data->ws_rule,
 			      data->o->file, set, reset, ws);
 	} else if (line[0] == ' ') {
@@ -2307,9 +2325,9 @@ static void emit_binary_diff_body(struct diff_options *o,
 		line[len++] = '\n';
 		line[len] = '\0';
 
-		emit_line(o, NULL, NULL, 1, 0, line, len);
+		emit_line(o, NULL, NULL, 1, 0, 0, line, len);
 	}
-	emit_line(o, NULL, NULL, 1, 0, "\n", 1);
+	emit_line(o, NULL, NULL, 1, 0, 0, "\n", 1);
 	free(data);
 }
 
@@ -2318,7 +2336,7 @@ static void emit_binary_diff(struct diff_options *o,
 {
 	const char *s = "GIT binary patch\n";
 	const int len = strlen(s);
-	emit_line(o, NULL, NULL, 1, 0, s, len);
+	emit_line(o, NULL, NULL, 1, 0, 0, s, len);
 	emit_binary_diff_body(o, one, two);
 	emit_binary_diff_body(o, two, one);
 }
@@ -2461,7 +2479,7 @@ static void builtin_diff(const char *name_a,
 		if (complete_rewrite &&
 		    (textconv_one || !diff_filespec_is_binary(one)) &&
 		    (textconv_two || !diff_filespec_is_binary(two))) {
-			emit_line(o, NULL, NULL, 0, 0, header.buf, header.len);
+			emit_line(o, NULL, NULL, 0, 0, 0, header.buf, header.len);
 			strbuf_reset(&header);
 			emit_rewrite_diff(name_a, name_b, one, two,
 						textconv_one, textconv_two, o);
@@ -2471,7 +2489,7 @@ static void builtin_diff(const char *name_a,
 	}
 
 	if (o->irreversible_delete && lbl[1][0] == '/') {
-		emit_line(o, NULL, NULL, 0, 0, header.buf, header.len);
+		emit_line(o, NULL, NULL, 0, 0, 0, header.buf, header.len);
 		strbuf_reset(&header);
 		goto free_ab_and_return;
 	} else if (!DIFF_OPT_TST(o, TEXT) &&
@@ -2482,11 +2500,11 @@ static void builtin_diff(const char *name_a,
 		    !DIFF_OPT_TST(o, BINARY)) {
 			if (!oidcmp(&one->oid, &two->oid)) {
 				if (must_show_header)
-					emit_line(o, NULL, NULL, 0, 0,
+					emit_line(o, NULL, NULL, 0, 0, 0,
 						  header.buf, header.len);
 				goto free_ab_and_return;
 			}
-			emit_line(o, NULL, NULL, 0, 0,
+			emit_line(o, NULL, NULL, 0, 0, 0,
 				  header.buf, header.len);
 			emit_line_fmt(o, NULL, NULL, 1,
 				      "Binary files %s and %s differ\n",
@@ -2499,11 +2517,11 @@ static void builtin_diff(const char *name_a,
 		if (mf1.size == mf2.size &&
 		    !memcmp(mf1.ptr, mf2.ptr, mf1.size)) {
 			if (must_show_header)
-				emit_line(o, NULL, NULL, 0, 0,
+				emit_line(o, NULL, NULL, 0, 0, 0,
 					  header.buf, header.len);
 			goto free_ab_and_return;
 		}
-		emit_line(o, NULL, NULL, 0, 0,
+		emit_line(o, NULL, NULL, 0, 0, 0,
 			  header.buf, header.len);
 		strbuf_reset(&header);
 		if (DIFF_OPT_TST(o, BINARY))
@@ -2523,7 +2541,7 @@ static void builtin_diff(const char *name_a,
 		const struct userdiff_funcname *pe;
 
 		if (must_show_header) {
-			emit_line(o, NULL, NULL, 0, 0, header.buf, header.len);
+			emit_line(o, NULL, NULL, 0, 0, 0, header.buf, header.len);
 			strbuf_reset(&header);
 		}
 
@@ -2540,6 +2558,7 @@ static void builtin_diff(const char *name_a,
 		ecbdata.label_path = lbl;
 		ecbdata.color_diff = want_color(o->use_color);
 		ecbdata.ws_rule = whitespace_rule(name_b);
+		o->ws_rule = ecbdata.ws_rule;
 		if (ecbdata.ws_rule & WS_BLANK_AT_EOF)
 			check_blank_at_eof(&mf1, &mf2, &ecbdata);
 		ecbdata.opt = o;
@@ -4514,7 +4533,7 @@ static void show_file_mode_name(struct diff_options *opt, const char *newdelete,
 
 	quote_c_style(fs->path, &sb, NULL, 0);
 	strbuf_addch(&sb, '\n');
-	emit_line(opt, NULL, NULL, 1, 0, sb.buf, sb.len);
+	emit_line(opt, NULL, NULL, 1, 0, 0, sb.buf, sb.len);
 	strbuf_release(&sb);
 }
 
@@ -4566,7 +4585,7 @@ static void diff_summary(struct diff_options *opt, struct diff_filepair *p)
 			strbuf_addstr(&sb, " rewrite ");
 			quote_c_style(p->two->path, &sb, NULL, 0);
 			strbuf_addf(&sb, " (%d%%)\n", similarity_index(p));
-			emit_line(opt, NULL, NULL, 1, 0, sb.buf, sb.len);
+			emit_line(opt, NULL, NULL, 1, 0, 0, sb.buf, sb.len);
 		}
 		show_mode_change(opt, p, !p->score);
 		break;
@@ -4873,10 +4892,10 @@ void diff_flush(struct diff_options *options)
 			term[0] = options->line_termination;
 			term[1] = '\0';
 
-			emit_line(options, NULL, NULL, 1, 0, term, !!term[0]);
+			emit_line(options, NULL, NULL, 1, 0, 0, term, !!term[0]);
 			if (options->stat_sep) {
 				/* attach patch instead of inline */
-				emit_line(options, NULL, NULL, 0, 0,
+				emit_line(options, NULL, NULL, 0, 0, 0,
 					  options->stat_sep,
 					  strlen(options->stat_sep));
 			}
diff --git a/diff.h b/diff.h
index 56d8dd036e..85948ed65a 100644
--- a/diff.h
+++ b/diff.h
@@ -186,6 +186,8 @@ struct diff_options {
 	void *output_prefix_data;
 
 	int diff_path_counter;
+
+	unsigned ws_rule;
 };
 
 void diff_emit_line(struct diff_options *o, const char *set, const char *reset,
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 16/17] diff: buffer all output if asked to
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (14 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 15/17] diff.c: emit_line includes whitespace highlighting Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-25  2:26     ` Junio C Hamano
  2017-06-13 22:07     ` Jonathan Tan
  2017-05-24 21:40   ` [PATCHv5 17/17] diff.c: color moved lines differently Stefan Beller
  2017-05-25  6:44   ` [PATCHv5 00/17] Diff machine: highlight moved lines Junio C Hamano
  17 siblings, 2 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

Introduce a new option 'use_buffer' in the struct diff_options which
controls whether all output is buffered up until all output is available.

We'll have a new struct 'diff_line' in diff.h which will be used to buffer
each line.  The diff_line will duplicate the memory of the line to buffer
as that is easiest to reason about for now. In a future patch we may want
to decrease the memory usage by not duplicating all output for buffering
but rather we may want to store offsets into the file or in case of hunk
descriptions such as the similarity score, we could just store the
relevant number and reproduce the text later on.

This approach was chosen as a first step because it is quite simple
compared to the alternative with less memory footprint.

emit_line factors out the emission part into emit_line_emission,
and depending on the diff_options->use_buffer the emission
will be performed directly when calling emit_line or after the
whole process is done, i.e. by buffering we have add the possibility
for a second pass over the whole output before doing the actual
output.

In 6440d34 (2012-03-14, diff: tweak a _copy_ of diff_options with
word-diff) we introduced a duplicate diff options struct for word
emissions as we may have different regex settings in there.
When buffering the output, we need to operate on just one buffer,
so we have to copy back the emissions of the word buffer into the
main buffer.

Unconditionally enable output via buffer in this patch as it yields
a great opportunity for testing, i.e. all the diff tests from the
test suite pass without having reordering issues (i.e. only parts
of the output got buffered, and we forgot to buffer other parts).
The test suite passes, which gives confidence that we converted all
functions to use emit_line for output.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++---------------
 diff.h |  41 +++++++++++++++++
 2 files changed, 161 insertions(+), 35 deletions(-)

diff --git a/diff.c b/diff.c
index 514c5facd7..8e06206881 100644
--- a/diff.c
+++ b/diff.c
@@ -516,54 +516,85 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,
 	ecbdata->blank_at_eof_in_postimage = (at - l2) + 1;
 }
 
-
-static void emit_line(struct diff_options *o,
-		      const char *set, const char *reset,
-		      int add_line_prefix, int markup_ws,
-		      int sign, const char *line, int len)
+static void emit_diff_line(struct diff_options *o,
+			   struct diff_line *e)
 {
 	const char *ws;
 	int has_trailing_newline, has_trailing_carriage_return;
+	int len = e->len;
 	FILE *file = o->file;
 
-	if (add_line_prefix)
+	if (e->add_line_prefix)
 		fputs(diff_line_prefix(o), file);
 
-	if (markup_ws) {
+	switch (e->state) {
+	case DIFF_LINE_WS:
 		ws = diff_get_color(o->use_color, DIFF_WHITESPACE);
+		if (e->set)
+			fputs(e->set, file);
+		if (e->sign)
+			fputc(e->sign, file);
+		if (e->reset)
+			fputs(e->reset, file);
+		ws_check_emit(e->line, e->len, o->ws_rule,
+			      file, e->set, e->reset, ws);
+		return;
+	case DIFF_LINE_ASIS:
+		has_trailing_newline = (len > 0 && e->line[len-1] == '\n');
+		if (has_trailing_newline)
+			len--;
+		has_trailing_carriage_return = (len > 0 && e->line[len-1] == '\r');
+		if (has_trailing_carriage_return)
+			len--;
 
-		if (set)
-			fputs(set, file);
-		if (sign)
-			fputc(sign, file);
-		if (reset)
-			fputs(reset, file);
-		ws = diff_get_color(o->use_color, DIFF_WHITESPACE);
-		ws_check_emit(line, len, o->ws_rule,
-			      file, set, reset, ws);
+		if (len || e->sign) {
+			if (e->set)
+				fputs(e->set, file);
+			if (e->sign)
+				fputc(e->sign, file);
+			fwrite(e->line, len, 1, file);
+			if (e->reset)
+				fputs(e->reset, file);
+		}
+		if (has_trailing_carriage_return)
+			fputc('\r', file);
+		if (has_trailing_newline)
+			fputc('\n', file);
+		return;
+	case DIFF_LINE_RELOAD_WS_RULE:
+		o->ws_rule = whitespace_rule(e->line);
 		return;
+	default:
+		die("BUG: malformatted buffered patch line: '%d'", e->state);
 	}
+}
 
-	has_trailing_newline = (len > 0 && line[len-1] == '\n');
-	if (has_trailing_newline)
-		len--;
-	has_trailing_carriage_return = (len > 0 && line[len-1] == '\r');
-	if (has_trailing_carriage_return)
-		len--;
+static void append_diff_line(struct diff_options *o,
+			     struct diff_line *e)
+{
+	struct diff_line *f;
+	ALLOC_GROW(o->line_buffer,
+		   o->line_buffer_nr + 1,
+		   o->line_buffer_alloc);
+	f = &o->line_buffer[o->line_buffer_nr++];
 
-	if (len || sign) {
-		if (set)
-			fputs(set, file);
-		if (sign)
-			fputc(sign, file);
-		fwrite(line, len, 1, file);
-		if (reset)
-			fputs(reset, file);
-	}
-	if (has_trailing_carriage_return)
-		fputc('\r', file);
-	if (has_trailing_newline)
-		fputc('\n', file);
+	memcpy(f, e, sizeof(struct diff_line));
+	f->line = e->line ? xmemdupz(e->line, e->len) : NULL;
+}
+
+static void emit_line(struct diff_options *o,
+		      const char *set, const char *reset,
+		      int add_line_prefix, int markup_ws,
+		      int sign, const char *line, int len)
+{
+	struct diff_line e = {set, reset, line,
+		len, sign, add_line_prefix,
+		markup_ws ? DIFF_LINE_WS : DIFF_LINE_ASIS};
+
+	if (o->use_buffer)
+		append_diff_line(o, &e);
+	else
+		emit_diff_line(o, &e);
 }
 
 static void emit_line_fmt(struct diff_options *o,
@@ -1172,6 +1203,18 @@ static void diff_words_flush(struct emit_callback *ecbdata)
 	if (ecbdata->diff_words->minus.text.size ||
 	    ecbdata->diff_words->plus.text.size)
 		diff_words_show(ecbdata->diff_words);
+
+	if (ecbdata->diff_words->opt->line_buffer_nr) {
+		int i;
+		for (i = 0; i < ecbdata->diff_words->opt->line_buffer_nr; i++)
+			append_diff_line(ecbdata->opt,
+				&ecbdata->diff_words->opt->line_buffer[i]);
+
+		for (i = 0; i < ecbdata->diff_words->opt->line_buffer_nr; i++)
+			free((void *)ecbdata->diff_words->opt->line_buffer[i].line);
+
+		ecbdata->diff_words->opt->line_buffer_nr = 0;
+	}
 }
 
 static void diff_filespec_load_driver(struct diff_filespec *one)
@@ -1207,6 +1250,11 @@ static void init_diff_words_data(struct emit_callback *ecbdata,
 		xcalloc(1, sizeof(struct diff_words_data));
 	ecbdata->diff_words->type = o->word_diff;
 	ecbdata->diff_words->opt = o;
+
+	o->line_buffer = NULL;
+	o->line_buffer_nr = 0;
+	o->line_buffer_alloc = 0;
+
 	if (!o->word_regex)
 		o->word_regex = userdiff_word_regex(one);
 	if (!o->word_regex)
@@ -1241,6 +1289,7 @@ static void free_diff_words_data(struct emit_callback *ecbdata)
 {
 	if (ecbdata->diff_words) {
 		diff_words_flush(ecbdata);
+		free (ecbdata->diff_words->opt->line_buffer);
 		free (ecbdata->diff_words->opt);
 		free (ecbdata->diff_words->minus.text.ptr);
 		free (ecbdata->diff_words->minus.orig);
@@ -2579,6 +2628,13 @@ static void builtin_diff(const char *name_a,
 			xecfg.ctxlen = strtoul(v, NULL, 10);
 		if (o->word_diff)
 			init_diff_words_data(&ecbdata, o, one, two);
+		if (o->use_buffer) {
+			struct diff_line e = diff_line_INIT;
+			e.state = DIFF_LINE_RELOAD_WS_RULE;
+			e.line = name_b;
+			e.len = strlen(name_b);
+			append_diff_line(o, &e);
+		}
 		if (xdi_diff_outf(&mf1, &mf2, fn_out_consume, &ecbdata,
 				  &xpp, &xecfg))
 			die("unable to generate diff for %s", one->path);
@@ -3458,6 +3514,10 @@ void diff_setup(struct diff_options *options)
 		options->a_prefix = "a/";
 		options->b_prefix = "b/";
 	}
+
+	options->line_buffer = NULL;
+	options->line_buffer_nr = 0;
+	options->line_buffer_alloc = 0;
 }
 
 void diff_setup_done(struct diff_options *options)
@@ -4796,11 +4856,36 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 {
 	int i;
 	struct diff_queue_struct *q = &diff_queued_diff;
+	/*
+	 * For testing purposes we want to make sure the diff machinery
+	 * works completely with the buffer. If there is anything emitted
+	 * outside the emit_diff_line, then the order is screwed
+	 * up and the tests will fail.
+	 *
+	 * TODO (later in this series):
+	 * We'll unset this flag in a later patch.
+	 */
+	o->use_buffer = 1;
+
 	for (i = 0; i < q->nr; i++) {
 		struct diff_filepair *p = q->queue[i];
 		if (check_pair_status(p))
 			diff_flush_patch(p, o);
 	}
+
+	if (o->use_buffer) {
+		for (i = 0; i < o->line_buffer_nr; i++)
+			emit_diff_line(o, &o->line_buffer[i]);
+
+		for (i = 0; i < o->line_buffer_nr; i++)
+			free((void *)o->line_buffer[i].line);
+
+		free(o->line_buffer);
+
+		o->line_buffer = NULL;
+		o->line_buffer_nr = 0;
+		o->line_buffer_alloc = 0;
+	}
 }
 
 void diff_flush(struct diff_options *options)
diff --git a/diff.h b/diff.h
index 85948ed65a..fad1258556 100644
--- a/diff.h
+++ b/diff.h
@@ -115,6 +115,42 @@ enum diff_submodule_format {
 	DIFF_SUBMODULE_INLINE_DIFF
 };
 
+/*
+ * This struct is used when we need to buffer the output of the diff output.
+ *
+ * NEEDSWORK: Instead of storing a copy of the line, add an offset pointer
+ * into the pre/post image file. This pointer could be a union with the
+ * line pointer. By storing an offset into the file instead of the literal line,
+ * we can decrease the memory footprint for the buffered output. At first we
+ * may want to only have indirection for the content lines, but we could also
+ * enhance the state for emitting prefabricated lines, e.g. the similarity
+ * score line or hunk/file headers would only need to store a number or path
+ * and then the output can be constructed later on depending on state.
+ */
+struct diff_line {
+	const char *set;
+	const char *reset;
+	const char *line;
+	int len;
+	int sign;
+	int add_line_prefix;
+	enum {
+		/*
+		 * Emits [lineprefix][set][sign][reset] and then calls
+		 * ws_check_emit which will output "line", marked up
+		 * according to ws_rule.
+		 */
+		DIFF_LINE_WS,
+
+		/* Emits [lineprefix][set][sign] line [reset] */
+		DIFF_LINE_ASIS,
+
+		/* Reloads the ws_rule; line contains the file name */
+		DIFF_LINE_RELOAD_WS_RULE
+	} state;
+};
+#define diff_line_INIT {NULL, NULL, NULL, 0, 0, 0}
+
 struct diff_options {
 	const char *orderfile;
 	const char *pickaxe;
@@ -188,8 +224,13 @@ struct diff_options {
 	int diff_path_counter;
 
 	unsigned ws_rule;
+	int use_buffer;
+
+	struct diff_line *line_buffer;
+	int line_buffer_nr, line_buffer_alloc;
 };
 
+/* Emit [line_prefix] [set] line [reset] */
 void diff_emit_line(struct diff_options *o, const char *set, const char *reset,
 		    const char *line, int len);
 
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv5 17/17] diff.c: color moved lines differently
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (15 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 16/17] diff: buffer all output if asked to Stefan Beller
@ 2017-05-24 21:40   ` Stefan Beller
  2017-05-25  2:27     ` Junio C Hamano
  2017-05-25  6:44   ` [PATCHv5 00/17] Diff machine: highlight moved lines Junio C Hamano
  17 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-05-24 21:40 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger, Stefan Beller

When a patch consists mostly of moving blocks of code around, it can
be quite tedious to ensure that the blocks are moved verbatim, and not
undesirably modified in the move. To that end, color blocks that are
moved within the same patch differently. For example (OM, del, add,
and NM are different colors):

    [OM]  -void sensitive_stuff(void)
    [OM]  -{
    [OM]  -        if (!is_authorized_user())
    [OM]  -                die("unauthorized");
    [OM]  -        sensitive_stuff(spanning,
    [OM]  -                        multiple,
    [OM]  -                        lines);
    [OM]  -}

           void another_function()
           {
    [del] -        printf("foo");
    [add] +        printf("bar");
           }

    [NM]  +void sensitive_stuff(void)
    [NM]  +{
    [NM]  +        if (!is_authorized_user())
    [NM]  +                die("unauthorized");
    [NM]  +        sensitive_stuff(spanning,
    [NM]  +                        multiple,
    [NM]  +                        lines);
    [NM]  +}

Adjacent blocks are colored differently. For example, in this
potentially malicious patch, the swapping of blocks can be spotted:

    [OM]  -void sensitive_stuff(void)
    [OM]  -{
    [OMA] -        if (!is_authorized_user())
    [OMA] -                die("unauthorized");
    [OM]  -        sensitive_stuff(spanning,
    [OM]  -                        multiple,
    [OM]  -                        lines);
    [OMA] -}

           void another_function()
           {
    [del] -        printf("foo");
    [add] +        printf("bar");
           }

    [NM]  +void sensitive_stuff(void)
    [NM]  +{
    [NMA] +        sensitive_stuff(spanning,
    [NMA] +                        multiple,
    [NMA] +                        lines);
    [NM]  +        if (!is_authorized_user())
    [NM]  +                die("unauthorized");
    [NMA] +}

If the moved code is larger, it is easier to hide some permutation in the
code, which is why the alternative coloring is really needed.

As the reviewers attention should be brought to the places, where the
difference is introduced to the moved code, we cannot just have one new
color for all of moved code.

First I implemented an alternative design, which would show a moved hunk
in one color, and its boundaries in another color. This idea was error
prone as it inspected each line and its neighboring lines to determine
if the line was (a) moved and (b) if was deep inside a hunk by having
matching neighboring lines. This is unreliable as the we can construct
hunks which have equal neighbors that just exceed the number of lines
inspected. (Think of 'AXYZBXYZCXYZD..' with each letter as a line, that
is permutated to AXYZCXYZBXYZD..').

Instead this provides a dynamic programming greedy algorithm that finds
the largest moved hunk and then switches color to the alternative color
for the next hunk. By doing this any permutation is recognized and
displayed. That implies that there is no dedicated boundary or
inside-hunk color, but instead we'll have just two colors alternating
for hunks.

It would be a bit more UX friendly if the two corresponding hunks
(of added and deleted lines) for one move would get the same color id.
(Both get "regular moved" or "alternative moved"). This problem is
deferred to a later patch for now.

A note on the options '--submodule=diff' and '--color-words/--word-diff':
In the conversion to use emit_line in the prior patches both submodules
as well as word diff output carefully chose to call emit_line with sign=0.
All output with sign=0 is ignored for move detection purposes in this
patch, such that no weird looking output will be generated for these
cases. This leads to another thought: We could pass on '--color-moved' to
submodules such that they color up moved lines for themselves. If we'd do
so only line moves within a repository boundary are marked up.

Helped-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

# Conflicts:
#	diff.c
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 Documentation/config.txt   |  14 ++-
 diff.c                     | 275 +++++++++++++++++++++++++++++++++++++++++++--
 diff.h                     |   9 +-
 t/t4015-diff-whitespace.sh | 267 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 552 insertions(+), 13 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 475e874d51..902d017c3b 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1051,14 +1051,24 @@ This does not affect linkgit:git-format-patch[1] or the
 'git-diff-{asterisk}' plumbing commands.  Can be overridden on the
 command line with the `--color[=<when>]` option.
 
+color.moved::
+	A boolean value, whether a diff should color moved lines
+	differently. The moved lines are searched for in the diff only.
+	Duplicated lines from somewhere in the project that are not
+	part of the diff are not colored as moved.
+	Defaults to false.
+
 color.diff.<slot>::
 	Use customized color for diff colorization.  `<slot>` specifies
 	which part of the patch to use the specified color, and is one
 	of `context` (context text - `plain` is a historical synonym),
 	`meta` (metainformation), `frag`
 	(hunk header), 'func' (function in hunk header), `old` (removed lines),
-	`new` (added lines), `commit` (commit headers), or `whitespace`
-	(highlighting whitespace errors).
+	`new` (added lines), `commit` (commit headers), `whitespace`
+	(highlighting whitespace errors), `oldMoved` (removed lines that
+	reappear), `newMoved` (added lines that were removed elsewhere),
+	`oldMovedAlternative` and `newMovedAlternative` (as a fallback to
+	cover adjacent blocks of moved code)
 
 color.decorate.<slot>::
 	Use customized color for 'git log --decorate' output.  `<slot>` is one
diff --git a/diff.c b/diff.c
index 8e06206881..1292d3c4ad 100644
--- a/diff.c
+++ b/diff.c
@@ -31,6 +31,7 @@ static int diff_indent_heuristic; /* experimental */
 static int diff_rename_limit_default = 400;
 static int diff_suppress_blank_empty;
 static int diff_use_color_default = -1;
+static int diff_color_moved_default;
 static int diff_context_default = 3;
 static int diff_interhunk_context_default;
 static const char *diff_word_regex_cfg;
@@ -55,6 +56,10 @@ static char diff_colors[][COLOR_MAXLEN] = {
 	GIT_COLOR_YELLOW,	/* COMMIT */
 	GIT_COLOR_BG_RED,	/* WHITESPACE */
 	GIT_COLOR_NORMAL,	/* FUNCINFO */
+	GIT_COLOR_BOLD_RED,	/* OLD_MOVED_A */
+	GIT_COLOR_BG_RED,	/* OLD_MOVED_B */
+	GIT_COLOR_BOLD_GREEN,	/* NEW_MOVED_A */
+	GIT_COLOR_BG_GREEN,	/* NEW_MOVED_B */
 };
 
 static NORETURN void die_want_option(const char *option_name)
@@ -80,6 +85,14 @@ static int parse_diff_color_slot(const char *var)
 		return DIFF_WHITESPACE;
 	if (!strcasecmp(var, "func"))
 		return DIFF_FUNCINFO;
+	if (!strcasecmp(var, "oldmoved"))
+		return DIFF_FILE_OLD_MOVED;
+	if (!strcasecmp(var, "oldmovedalternative"))
+		return DIFF_FILE_OLD_MOVED_ALT;
+	if (!strcasecmp(var, "newmoved"))
+		return DIFF_FILE_NEW_MOVED;
+	if (!strcasecmp(var, "newmovedalternative"))
+		return DIFF_FILE_NEW_MOVED_ALT;
 	return -1;
 }
 
@@ -234,6 +247,10 @@ int git_diff_ui_config(const char *var, const char *value, void *cb)
 		diff_use_color_default = git_config_colorbool(var, value);
 		return 0;
 	}
+	if (!strcmp(var, "color.moved")) {
+		diff_color_moved_default = git_config_bool(var, value);
+		return 0;
+	}
 	if (!strcmp(var, "diff.context")) {
 		diff_context_default = git_config_int(var, value);
 		if (diff_context_default < 0)
@@ -354,6 +371,88 @@ int git_diff_basic_config(const char *var, const char *value, void *cb)
 	return git_default_config(var, value, cb);
 }
 
+struct moved_entry {
+	struct hashmap_entry ent;
+	const struct diff_line *line;
+	struct moved_entry *next_line;
+};
+
+static void get_ws_cleaned_string(const struct diff_line *l,
+				  struct strbuf *out)
+{
+	int i;
+	for (i = 0; i < l->len; i++) {
+		if (isspace(l->line[i]))
+			continue;
+		strbuf_addch(out, l->line[i]);
+	}
+}
+
+static int diff_line_cmp_no_ws(const struct diff_line *a,
+					 const struct diff_line *b,
+					 const void *keydata)
+{
+	int ret;
+	struct strbuf sba = STRBUF_INIT;
+	struct strbuf sbb = STRBUF_INIT;
+
+	get_ws_cleaned_string(a, &sba);
+	get_ws_cleaned_string(b, &sbb);
+	ret = sba.len != sbb.len || strncmp(sba.buf, sbb.buf, sba.len);
+
+	strbuf_release(&sba);
+	strbuf_release(&sbb);
+	return ret;
+}
+
+static int diff_line_cmp(const struct diff_line *a,
+				   const struct diff_line *b,
+				   const void *keydata)
+{
+	return a->len != b->len || strncmp(a->line, b->line, a->len);
+}
+
+static int moved_entry_cmp(const struct moved_entry *a,
+			   const struct moved_entry *b,
+			   const void *keydata)
+{
+	return diff_line_cmp(a->line, b->line, keydata);
+}
+
+static int moved_entry_cmp_no_ws(const struct moved_entry *a,
+				 const struct moved_entry *b,
+				 const void *keydata)
+{
+	return diff_line_cmp_no_ws(a->line, b->line, keydata);
+}
+
+static unsigned get_line_hash(struct diff_line *line, unsigned ignore_ws)
+{
+	static struct strbuf sb = STRBUF_INIT;
+
+	if (ignore_ws) {
+		strbuf_reset(&sb);
+		get_ws_cleaned_string(line, &sb);
+		return memhash(sb.buf, sb.len);
+	} else {
+		return memhash(line->line, line->len);
+	}
+}
+
+static struct moved_entry *prepare_entry(struct diff_options *o,
+					 int line_no)
+{
+	struct moved_entry *ret = xmalloc(sizeof(*ret));
+	unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
+	struct diff_line *l = &o->line_buffer[line_no];
+
+	ret->ent.hash = get_line_hash(l, ignore_ws);
+	ret->line = l;
+	ret->next_line = NULL;
+
+	return ret;
+}
+
 static char *quote_two(const char *one, const char *two)
 {
 	int need_one = quote_c_style(one, NULL, NULL, 1);
@@ -516,6 +615,141 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,
 	ecbdata->blank_at_eof_in_postimage = (at - l2) + 1;
 }
 
+static void add_lines_to_move_detection(struct diff_options *o,
+					struct hashmap *add_lines,
+					struct hashmap *del_lines)
+{
+	struct moved_entry *prev_line = NULL;
+
+	int n;
+	for (n = 0; n < o->line_buffer_nr; n++) {
+		int sign = 0;
+		struct hashmap *hm;
+		struct moved_entry *key;
+
+		switch (o->line_buffer[n].sign) {
+		case '+':
+			sign = '+';
+			hm = add_lines;
+			break;
+		case '-':
+			sign = '-';
+			hm = del_lines;
+			break;
+		case ' ':
+		default:
+			prev_line = NULL;
+			continue;
+		}
+
+		key = prepare_entry(o, n);
+		if (prev_line &&
+		    prev_line->line->sign == sign)
+			prev_line->next_line = key;
+
+		hashmap_add(hm, key);
+		prev_line = key;
+	}
+}
+
+static void mark_color_as_moved(struct diff_options *o,
+				struct hashmap *add_lines,
+				struct hashmap *del_lines)
+{
+	struct moved_entry **pmb = NULL; /* potentially moved blocks */
+	int pmb_nr = 0, pmb_alloc = 0;
+	int use_alt_color = 0;
+	int n;
+
+	for (n = 0; n < o->line_buffer_nr; n++) {
+		struct hashmap *hm = NULL;
+		struct moved_entry *key;
+		struct moved_entry *match = NULL;
+		struct diff_line *l = &o->line_buffer[n];
+		int i, lp, rp;
+
+		switch (l->sign) {
+		case '+':
+			hm = del_lines;
+			break;
+		case '-':
+			hm = add_lines;
+			break;
+		default:
+			use_alt_color = 0;
+			pmb_nr = 0; /* no running sets */
+			continue;
+		}
+
+		/* Check for any match to color it as a move. */
+		key = prepare_entry(o, n);
+		match = hashmap_get(hm, key, o);
+		free(key);
+		if (!match)
+			continue;
+
+		/* Check any potential block runs, advance each or nullify */
+		for (i = 0; i < pmb_nr; i++) {
+			struct moved_entry *p = pmb[i];
+			struct moved_entry *pnext = (p && p->next_line) ?
+					p->next_line : NULL;
+			if (pnext &&
+			    !diff_line_cmp(pnext->line, l, o)) {
+				pmb[i] = p->next_line;
+			} else {
+				pmb[i] = NULL;
+			}
+		}
+
+		/* Shrink the set to the remaining runs */
+		for (lp = 0, rp = pmb_nr - 1; lp <= rp;) {
+			while (lp < pmb_nr && pmb[lp])
+				lp++;
+			/* lp points at the first NULL now */
+
+			while (rp > -1 && !pmb[rp])
+				rp--;
+			/* rp points at the last non-NULL */
+
+			if (lp < pmb_nr && rp > -1 && lp < rp) {
+				pmb[lp] = pmb[rp];
+				pmb[rp] = NULL;
+				rp--;
+				lp++;
+			}
+		}
+
+		if (rp > -1) {
+			/* Remember the number of running sets */
+			pmb_nr = rp + 1;
+		} else {
+			/* Toggle color */
+			use_alt_color = (use_alt_color + 1) % 2;
+
+			/* Build up a new set */
+			pmb_nr = 0;
+			for (; match; match = hashmap_get_next(hm, match)) {
+				ALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);
+				pmb[pmb_nr++] = match;
+			}
+		}
+
+		switch (l->sign) {
+		case '+':
+			l->set = diff_get_color_opt(o,
+				DIFF_FILE_NEW_MOVED + use_alt_color);
+			break;
+		case '-':
+			l->set = diff_get_color_opt(o,
+				DIFF_FILE_OLD_MOVED + use_alt_color);
+			break;
+		default:
+			die("BUG: we should have continued earlier?");
+		}
+	}
+	free(pmb);
+}
+
 static void emit_diff_line(struct diff_options *o,
 			   struct diff_line *e)
 {
@@ -3518,6 +3752,8 @@ void diff_setup(struct diff_options *options)
 	options->line_buffer = NULL;
 	options->line_buffer_nr = 0;
 	options->line_buffer_alloc = 0;
+
+	options->color_moved = diff_color_moved_default;
 }
 
 void diff_setup_done(struct diff_options *options)
@@ -3627,6 +3863,9 @@ void diff_setup_done(struct diff_options *options)
 
 	if (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)
 		die(_("--follow requires exactly one pathspec"));
+
+	if (!options->use_color || external_diff())
+		options->color_moved = 0;
 }
 
 static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)
@@ -4051,6 +4290,10 @@ int diff_opt_parse(struct diff_options *options,
 	}
 	else if (!strcmp(arg, "--no-color"))
 		options->use_color = 0;
+	else if (!strcmp(arg, "--color-moved"))
+		options->color_moved = 1;
+	else if (!strcmp(arg, "--no-color-moved"))
+		options->color_moved = 0;
 	else if (!strcmp(arg, "--color-words")) {
 		options->use_color = 1;
 		options->word_diff = DIFF_WORDS_COLOR;
@@ -4856,16 +5099,9 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 {
 	int i;
 	struct diff_queue_struct *q = &diff_queued_diff;
-	/*
-	 * For testing purposes we want to make sure the diff machinery
-	 * works completely with the buffer. If there is anything emitted
-	 * outside the emit_diff_line, then the order is screwed
-	 * up and the tests will fail.
-	 *
-	 * TODO (later in this series):
-	 * We'll unset this flag in a later patch.
-	 */
-	o->use_buffer = 1;
+
+	if (o->color_moved)
+		o->use_buffer = 1;
 
 	for (i = 0; i < q->nr; i++) {
 		struct diff_filepair *p = q->queue[i];
@@ -4874,6 +5110,24 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 	}
 
 	if (o->use_buffer) {
+		if (o->color_moved) {
+			struct hashmap add_lines, del_lines;
+			unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
+
+			hashmap_init(&del_lines, ignore_ws ?
+				(hashmap_cmp_fn)moved_entry_cmp_no_ws :
+				(hashmap_cmp_fn)moved_entry_cmp, 0);
+			hashmap_init(&add_lines, ignore_ws ?
+				(hashmap_cmp_fn)moved_entry_cmp_no_ws :
+				(hashmap_cmp_fn)moved_entry_cmp, 0);
+
+			add_lines_to_move_detection(o, &add_lines, &del_lines);
+			mark_color_as_moved(o, &add_lines, &del_lines);
+
+			hashmap_free(&add_lines, 0);
+			hashmap_free(&del_lines, 0);
+		}
+
 		for (i = 0; i < o->line_buffer_nr; i++)
 			emit_diff_line(o, &o->line_buffer[i]);
 
@@ -4962,6 +5216,7 @@ void diff_flush(struct diff_options *options)
 		if (!options->file)
 			die_errno("Could not open /dev/null");
 		options->close_file = 1;
+		options->color_moved = 0;
 		for (i = 0; i < q->nr; i++) {
 			struct diff_filepair *p = q->queue[i];
 			if (check_pair_status(p))
diff --git a/diff.h b/diff.h
index fad1258556..445259ebf7 100644
--- a/diff.h
+++ b/diff.h
@@ -7,6 +7,7 @@
 #include "tree-walk.h"
 #include "pathspec.h"
 #include "object.h"
+#include "hashmap.h"
 
 struct rev_info;
 struct diff_options;
@@ -228,6 +229,8 @@ struct diff_options {
 
 	struct diff_line *line_buffer;
 	int line_buffer_nr, line_buffer_alloc;
+
+	int color_moved;
 };
 
 /* Emit [line_prefix] [set] line [reset] */
@@ -243,7 +246,11 @@ enum color_diff {
 	DIFF_FILE_NEW = 5,
 	DIFF_COMMIT = 6,
 	DIFF_WHITESPACE = 7,
-	DIFF_FUNCINFO = 8
+	DIFF_FUNCINFO = 8,
+	DIFF_FILE_OLD_MOVED = 9,
+	DIFF_FILE_OLD_MOVED_ALT = 10,
+	DIFF_FILE_NEW_MOVED = 11,
+	DIFF_FILE_NEW_MOVED_ALT = 12
 };
 const char *diff_get_color(int diff_use_color, enum color_diff ix);
 #define diff_get_color_opt(o, ix) \
diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
index 289806d0c7..0e92bf94bf 100755
--- a/t/t4015-diff-whitespace.sh
+++ b/t/t4015-diff-whitespace.sh
@@ -972,4 +972,271 @@ test_expect_success 'option overrides diff.wsErrorHighlight' '
 
 '
 
+test_expect_success 'detect moved code, complete file' '
+	git reset --hard &&
+	cat <<-\EOF >test.c &&
+	#include<stdio.h>
+	main()
+	{
+	printf("Hello World");
+	}
+	EOF
+	git add test.c &&
+	git commit -m "add main function" &&
+	git mv test.c main.c &&
+	git diff HEAD --color-moved --no-renames | test_decode_color >actual &&
+	cat >expected <<-\EOF &&
+	<BOLD>diff --git a/main.c b/main.c<RESET>
+	<BOLD>new file mode 100644<RESET>
+	<BOLD>index 0000000..a986c57<RESET>
+	<BOLD>--- /dev/null<RESET>
+	<BOLD>+++ b/main.c<RESET>
+	<CYAN>@@ -0,0 +1,5 @@<RESET>
+	<BGREEN>+<RESET><BGREEN>#include<stdio.h><RESET>
+	<BGREEN>+<RESET><BGREEN>main()<RESET>
+	<BGREEN>+<RESET><BGREEN>{<RESET>
+	<BGREEN>+<RESET><BGREEN>printf("Hello World");<RESET>
+	<BGREEN>+<RESET><BGREEN>}<RESET>
+	<BOLD>diff --git a/test.c b/test.c<RESET>
+	<BOLD>deleted file mode 100644<RESET>
+	<BOLD>index a986c57..0000000<RESET>
+	<BOLD>--- a/test.c<RESET>
+	<BOLD>+++ /dev/null<RESET>
+	<CYAN>@@ -1,5 +0,0 @@<RESET>
+	<BRED>-#include<stdio.h><RESET>
+	<BRED>-main()<RESET>
+	<BRED>-{<RESET>
+	<BRED>-printf("Hello World");<RESET>
+	<BRED>-}<RESET>
+	EOF
+
+	test_cmp expected actual
+'
+
+test_expect_success 'detect moved code, inside file' '
+	git reset --hard &&
+	cat <<-\EOF >main.c &&
+		#include<stdio.h>
+		int stuff()
+		{
+			printf("Hello ");
+			printf("World\n");
+		}
+
+		int secure_foo(struct user *u)
+		{
+			if (!u->is_allowed_foo)
+				return;
+			foo(u);
+		}
+
+		int main()
+		{
+			foo();
+		}
+	EOF
+	cat <<-\EOF >test.c &&
+		#include<stdio.h>
+		int bar()
+		{
+			printf("Hello World, but different\n");
+		}
+
+		int another_function()
+		{
+			bar();
+		}
+	EOF
+	git add main.c test.c &&
+	git commit -m "add main and test file" &&
+	cat <<-\EOF >main.c &&
+		#include<stdio.h>
+		int stuff()
+		{
+			printf("Hello ");
+			printf("World\n");
+		}
+
+		int main()
+		{
+			foo();
+		}
+	EOF
+	cat <<-\EOF >test.c &&
+		#include<stdio.h>
+		int bar()
+		{
+			printf("Hello World, but different\n");
+		}
+
+		int secure_foo(struct user *u)
+		{
+			if (!u->is_allowed_foo)
+				return;
+			foo(u);
+		}
+
+		int another_function()
+		{
+			bar();
+		}
+	EOF
+	git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/main.c b/main.c<RESET>
+	<BOLD>index 27a619c..7cf9336 100644<RESET>
+	<BOLD>--- a/main.c<RESET>
+	<BOLD>+++ b/main.c<RESET>
+	<CYAN>@@ -5,13 +5,6 @@<RESET> <RESET>printf("Hello ");<RESET>
+	 printf("World\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BRED>-int secure_foo(struct user *u)<RESET>
+	<BRED>-{<RESET>
+	<BRED>-if (!u->is_allowed_foo)<RESET>
+	<BRED>-return;<RESET>
+	<BRED>-foo(u);<RESET>
+	<BRED>-}<RESET>
+	<BRED>-<RESET>
+	 int main()<RESET>
+	 {<RESET>
+	 foo();<RESET>
+	<BOLD>diff --git a/test.c b/test.c<RESET>
+	<BOLD>index 1dc1d85..e34eb69 100644<RESET>
+	<BOLD>--- a/test.c<RESET>
+	<BOLD>+++ b/test.c<RESET>
+	<CYAN>@@ -4,6 +4,13 @@<RESET> <RESET>int bar()<RESET>
+	 printf("Hello World, but different\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BGREEN>+<RESET><BGREEN>int secure_foo(struct user *u)<RESET>
+	<BGREEN>+<RESET><BGREEN>{<RESET>
+	<BGREEN>+<RESET><BGREEN>if (!u->is_allowed_foo)<RESET>
+	<BGREEN>+<RESET><BGREEN>return;<RESET>
+	<BGREEN>+<RESET><BGREEN>foo(u);<RESET>
+	<BGREEN>+<RESET><BGREEN>}<RESET>
+	<BGREEN>+<RESET>
+	 int another_function()<RESET>
+	 {<RESET>
+	 bar();<RESET>
+	EOF
+
+	test_cmp expected actual
+'
+
+test_expect_success 'detect permutations inside moved code' '
+	# reusing the move example from last test:
+	cat <<-\EOF >main.c &&
+		#include<stdio.h>
+		int stuff()
+		{
+			printf("Hello ");
+			printf("World\n");
+		}
+
+		int main()
+		{
+			foo();
+		}
+	EOF
+	cat <<-\EOF >test.c &&
+		#include<stdio.h>
+		int bar()
+		{
+			printf("Hello World, but different\n");
+		}
+
+		int secure_foo(struct user *u)
+		{
+			foo(u);
+			if (!u->is_allowed_foo)
+				return;
+		}
+
+		int another_function()
+		{
+			bar();
+		}
+	EOF
+	git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/main.c b/main.c<RESET>
+	<BOLD>index 27a619c..7cf9336 100644<RESET>
+	<BOLD>--- a/main.c<RESET>
+	<BOLD>+++ b/main.c<RESET>
+	<CYAN>@@ -5,13 +5,6 @@<RESET> <RESET>printf("Hello ");<RESET>
+	 printf("World\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BRED>-int secure_foo(struct user *u)<RESET>
+	<BRED>-{<RESET>
+	<BOLD;RED>-if (!u->is_allowed_foo)<RESET>
+	<BOLD;RED>-return;<RESET>
+	<BRED>-foo(u);<RESET>
+	<BOLD;RED>-}<RESET>
+	<BOLD;RED>-<RESET>
+	 int main()<RESET>
+	 {<RESET>
+	 foo();<RESET>
+	<BOLD>diff --git a/test.c b/test.c<RESET>
+	<BOLD>index 1dc1d85..2bedec9 100644<RESET>
+	<BOLD>--- a/test.c<RESET>
+	<BOLD>+++ b/test.c<RESET>
+	<CYAN>@@ -4,6 +4,13 @@<RESET> <RESET>int bar()<RESET>
+	 printf("Hello World, but different\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BGREEN>+<RESET><BGREEN>int secure_foo(struct user *u)<RESET>
+	<BGREEN>+<RESET><BGREEN>{<RESET>
+	<BOLD;GREEN>+<RESET><BOLD;GREEN>foo(u);<RESET>
+	<BGREEN>+<RESET><BGREEN>if (!u->is_allowed_foo)<RESET>
+	<BGREEN>+<RESET><BGREEN>return;<RESET>
+	<BOLD;GREEN>+<RESET><BOLD;GREEN>}<RESET>
+	<BOLD;GREEN>+<RESET>
+	 int another_function()<RESET>
+	 {<RESET>
+	 bar();<RESET>
+	EOF
+
+	test_cmp expected actual
+'
+
+test_expect_success 'move detection does not mess up colored words' '
+	cat <<-\EOF >text.txt &&
+	Lorem Ipsum is simply dummy text of the printing and typesetting industry.
+	EOF
+	git add text.txt &&
+	git commit -a -m "clean state" &&
+	cat <<-\EOF >text.txt &&
+	simply Lorem Ipsum dummy is text of the typesetting and printing industry.
+	EOF
+	git diff --color-moved --word-diff >actual &&
+	git diff --word-diff >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'move detection with submodules' '
+	test_create_repo bananas &&
+	echo ripe >bananas/recipe &&
+	git -C bananas add recipe &&
+	test_commit fruit &&
+	test_commit -C bananas recipe &&
+	git submodule add ./bananas &&
+	git add bananas &&
+	git commit -a -m "bananas are like a heavy library?" &&
+	echo foul >bananas/recipe &&
+	echo ripe >fruit.t &&
+
+	git diff --submodule=diff --color-moved >actual &&
+
+	# no move detection as the moved line is across repository boundaries.
+	test_decode_color <actual >decoded_actual &&
+	! grep BGREEN decoded_actual &&
+	! grep BRED decoded_actual &&
+
+	# nor did we mess with it another way
+	git diff --submodule=diff | test_decode_color >expect &&
+	test_cmp expect decoded_actual
+'
+
 test_done
-- 
2.13.0.18.g7d86cc8ba0


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 16/17] diff: buffer all output if asked to
  2017-05-24 21:40   ` [PATCHv5 16/17] diff: buffer all output if asked to Stefan Beller
@ 2017-05-25  2:26     ` Junio C Hamano
  2017-05-25  5:34       ` Stefan Beller
  2017-06-13 22:07     ` Jonathan Tan
  1 sibling, 1 reply; 125+ messages in thread
From: Junio C Hamano @ 2017-05-25  2:26 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger

Stefan Beller <sbeller@google.com> writes:

> Introduce a new option 'use_buffer' in the struct diff_options which
> controls whether all output is buffered up until all output is available.
> ...
> Unconditionally enable output via buffer in this patch as it yields
> a great opportunity for testing, i.e. all the diff tests from the
> test suite pass without having reordering issues (i.e. only parts
> of the output got buffered, and we forgot to buffer other parts).
> The test suite passes, which gives confidence that we converted all
> functions to use emit_line for output.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Oh, did I?

> ---
>  diff.c | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++---------------
>  diff.h |  41 +++++++++++++++++
>  2 files changed, 161 insertions(+), 35 deletions(-)
>
> diff --git a/diff.c b/diff.c
> index 514c5facd7..8e06206881 100644
> --- a/diff.c
> +++ b/diff.c
> ...
> @@ -2579,6 +2628,13 @@ static void builtin_diff(const char *name_a,
>  			xecfg.ctxlen = strtoul(v, NULL, 10);
>  		if (o->word_diff)
>  			init_diff_words_data(&ecbdata, o, one, two);
> +		if (o->use_buffer) {
> +			struct diff_line e = diff_line_INIT;

This ...

> +			e.state = DIFF_LINE_RELOAD_WS_RULE;
> ...
> +#define diff_line_INIT {NULL, NULL, NULL, 0, 0, 0}

... and this should be in all caps.   We do not say

	struct strbuf buf = strbuf_INIT;

and we should do the same for this new thing.


^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 17/17] diff.c: color moved lines differently
  2017-05-24 21:40   ` [PATCHv5 17/17] diff.c: color moved lines differently Stefan Beller
@ 2017-05-25  2:27     ` Junio C Hamano
  2017-05-25  5:39       ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Junio C Hamano @ 2017-05-25  2:27 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger

Stefan Beller <sbeller@google.com> writes:

> When a patch consists mostly of moving blocks of code around, it can
> be quite tedious to ensure that the blocks are moved verbatim, and not
> ...
> cases. This leads to another thought: We could pass on '--color-moved' to
> submodules such that they color up moved lines for themselves. If we'd do
> so only line moves within a repository boundary are marked up.
>
> Helped-by: Jonathan Tan <jonathantanmy@google.com>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
>
> # Conflicts:
> #	diff.c
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---

Hmph, what are these final lines about?

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 16/17] diff: buffer all output if asked to
  2017-05-25  2:26     ` Junio C Hamano
@ 2017-05-25  5:34       ` Stefan Beller
  2017-05-26  1:09         ` Junio C Hamano
  0 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-05-25  5:34 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Brandon Williams, Jonathan Nieder, Jonathan Tan, Jeff King,
	Michael Haggerty

On Wed, May 24, 2017 at 7:26 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> Introduce a new option 'use_buffer' in the struct diff_options which
>> controls whether all output is buffered up until all output is available.
>> ...
>> Unconditionally enable output via buffer in this patch as it yields
>> a great opportunity for testing, i.e. all the diff tests from the
>> test suite pass without having reordering issues (i.e. only parts
>> of the output got buffered, and we forgot to buffer other parts).
>> The test suite passes, which gives confidence that we converted all
>> functions to use emit_line for output.
>>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> Signed-off-by: Junio C Hamano <gitster@pobox.com>
>
> Oh, did I?

Yes, this is essentially the v4 with small indentation fixes, so I
assumed your signoff is still valid.

Which leads to explaining my workflow (which I think we
discussed that half a year ago with Dscho in a longer thread).

As soon as you apply a series I take that series and work off
that series, because you explained you would do smaller
fixups occasionally.

Patches that change drastically, I strip off your signoff and
pretend like I created them from scratch, but for those which
I barely touch (if at all), I do not remove your signoff, as
I'd be just passing them along again.

Maybe I have to rethink that strategy.

>
>> ---
>>  diff.c | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++---------------
>>  diff.h |  41 +++++++++++++++++
>>  2 files changed, 161 insertions(+), 35 deletions(-)
>>
>> diff --git a/diff.c b/diff.c
>> index 514c5facd7..8e06206881 100644
>> --- a/diff.c
>> +++ b/diff.c
>> ...
>> @@ -2579,6 +2628,13 @@ static void builtin_diff(const char *name_a,
>>                       xecfg.ctxlen = strtoul(v, NULL, 10);
>>               if (o->word_diff)
>>                       init_diff_words_data(&ecbdata, o, one, two);
>> +             if (o->use_buffer) {
>> +                     struct diff_line e = diff_line_INIT;
>
> This ...
>
>> +                     e.state = DIFF_LINE_RELOAD_WS_RULE;
>> ...
>> +#define diff_line_INIT {NULL, NULL, NULL, 0, 0, 0}
>
> ... and this should be in all caps.   We do not say
>
>         struct strbuf buf = strbuf_INIT;
>
> and we should do the same for this new thing.

Yes. Totally agree. That is fallout from a mechanical
replace all s/buffered_patch_line/diff_line/ and the case
sensitivity got lost.

Will fix again.

Stefan

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 17/17] diff.c: color moved lines differently
  2017-05-25  2:27     ` Junio C Hamano
@ 2017-05-25  5:39       ` Stefan Beller
  0 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-05-25  5:39 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Brandon Williams, Jonathan Nieder, Jonathan Tan, Jeff King,
	Michael Haggerty

On Wed, May 24, 2017 at 7:27 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> When a patch consists mostly of moving blocks of code around, it can
>> be quite tedious to ensure that the blocks are moved verbatim, and not
>> ...
>> cases. This leads to another thought: We could pass on '--color-moved' to
>> submodules such that they color up moved lines for themselves. If we'd do
>> so only line moves within a repository boundary are marked up.
>>
>> Helped-by: Jonathan Tan <jonathantanmy@google.com>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> Signed-off-by: Junio C Hamano <gitster@pobox.com>
>>
>> # Conflicts:
>> #     diff.c
>> Signed-off-by: Junio C Hamano <gitster@pobox.com>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>
> Hmph, what are these final lines about?

See the explanation in the patch 16/17.
My guess is that one of us (me) was careless again.

The commented lines are easily produced by git-gui, that
I use. I think it just takes the commit message from the underlying
git-core. But unlike git-core, it doesn't strip off commented lines
as there is no extra information in comments presented.
I wonder how you are the first signoff after the conflict markers
though, as that would hint that you signed off a commit message
with the commented conflict lines first, before I had them.

Puzzeled, will fix in a reroll.

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 00/17] Diff machine: highlight moved lines.
  2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
                     ` (16 preceding siblings ...)
  2017-05-24 21:40   ` [PATCHv5 17/17] diff.c: color moved lines differently Stefan Beller
@ 2017-05-25  6:44   ` Junio C Hamano
  2017-05-25 16:31     ` Stefan Beller
  17 siblings, 1 reply; 125+ messages in thread
From: Junio C Hamano @ 2017-05-25  6:44 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, bmwill, jrnieder, jonathantanmy, peff, mhagger

I was trying to see how this is useful in code moving change, with
this command line:

$ git -c color.moved diff js/blame-lib~6 js/blame-lib blame.c blame.h builtin/blame.c

Some random observations:

 * You do not seem to have any command line option yet.  I guess
   that is OK while the series is still in RFC state, but when we
   are ready to seriously start considering this for 'next', we'd
   need something like --color-moved that can be used across "diff",
   "log -p" and "show".

 * As configuration variable names go, "color.moved" is probably in
   a wrong section.  Perhaps "diff.colorMoved" or something?

 * The fact that I am using

     [diff "color"] 
	old = red reverse
	whitespace = blue reverse

   on a "black ink on white paper" terminal might have an effect on
   this, but lines printed in either bold green and on green
   background (i.e. not new ones but are the ones moved from
   elsewhere) stood out a lot more prominently than lines printed in
   green (i.e. truly new additions), and I felt that this is totally
   backwards from what I needed for this exercise.  That is, while
   reviewing a code moving change, I want to be able to concentrate
   primarily of three things:

   - What are the new lines that are *not* moved from elsewhere?
     Did the submitter try to sneak in unrelated changes?

   - What are the changes that are truly lost, not moved to
     elsewhere?

   - Do the lines moved from elsewhere form a coherent whole?  Where
     is the boundary between blocks of text that are copied?  Do
     adjacent two blocks of moved text come from the same old place
     next to each other?

   Using colors that stand out more prominently than for the regular
   new/old lines is counter-productive for all of these, especially
   for the first two purposes.  I may suggest using cyan (or any
   color that is very close to the background) so that the presense
   of moved lines are merely felt without being distracting.  IOW,
   while reviewing code moving patch, moved parts want to be dimmed,
   not highlighted.


^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 00/17] Diff machine: highlight moved lines.
  2017-05-25  6:44   ` [PATCHv5 00/17] Diff machine: highlight moved lines Junio C Hamano
@ 2017-05-25 16:31     ` Stefan Beller
  2017-05-26  1:20       ` Junio C Hamano
  0 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-05-25 16:31 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Brandon Williams, Jonathan Nieder, Jonathan Tan, Jeff King,
	Michael Haggerty

On Wed, May 24, 2017 at 11:44 PM, Junio C Hamano <gitster@pobox.com> wrote:
> I was trying to see how this is useful in code moving change, with
> this command line:
>
> $ git -c color.moved diff js/blame-lib~6 js/blame-lib blame.c blame.h builtin/blame.c
>
> Some random observations:
>
>  * You do not seem to have any command line option yet.  I guess
>    that is OK while the series is still in RFC state, but when we
>    are ready to seriously start considering this for 'next', we'd
>    need something like --color-moved that can be used across "diff",
>    "log -p" and "show".

There is "--color-moved" as a command line option. (See diff.c:4290)
Oh, it is not documented! That will be fixed.

>  * As configuration variable names go, "color.moved" is probably in
>    a wrong section.  Perhaps "diff.colorMoved" or something?

As you turn on/off normal coloring via "color.diff" and this only extends
the coloring scheme, I have the impression "color" is the right section.
Maybe color.diffStyle=[normal/enhanced] and "enhanced" means this
feature for now?

The only option in the "diff" section related to color is diff.wsErrorHighlight
which has a very similar purpose, so "diff.colorMoved" would fit in that
scheme.

>
>  * The fact that I am using
>
>      [diff "color"]
>         old = red reverse
>         whitespace = blue reverse
>
>    on a "black ink on white paper" terminal might have an effect on
>    this,

Yes very much!

>    but lines printed in either bold green and on green
>    background (i.e. not new ones but are the ones moved from
>    elsewhere) stood out a lot more prominently than lines printed in
>    green (i.e. truly new additions), and I felt that this is totally
>    backwards from what I needed for this exercise.  That is, while
>    reviewing a code moving change, I want to be able to concentrate
>    primarily of three things:
>
>    - What are the new lines that are *not* moved from elsewhere?
>      Did the submitter try to sneak in unrelated changes?
>
>    - What are the changes that are truly lost, not moved to
>      elsewhere?
>
>    - Do the lines moved from elsewhere form a coherent whole?  Where
>      is the boundary between blocks of text that are copied?  Do
>      adjacent two blocks of moved text come from the same old place
>      next to each other?

So with these questions, I wonder if we want to color moved lines
as "color.diff.context" (i.e. plain white text in the normal coloring scheme)
This would serve the intended purpose of
dimming the attention to moved lines.

Regarding the last point of marking up adjacent blocks (which would
indicate that there is a coherency issue or just moving from different
places), we could highlight the last line of the previous block
and first line of the next block in their "normal" colors (i.e.
color.diff.old and color.diff.new).

The very first version had some boundary coloring, but then
I switched to alternating block coloring based on an idea
by Jonathan Tan.

Maybe it is time to go back to boundary coloring, but optional
and apply the boundary color only if two blocks are adjacent?

Example of how it could look:
---8<---
    diff --git a/poetry.txt b/poetry.txt
    index 9d32b3b..cc50ca1 100644
    --- a/poetry.txt
    +++ b/poetry.txt
    @@ -1,12 +1,4 @@
[W] -A simple text is
[W] -written in paragraphs and
[W] -many more lines.
[R] -
[W]  A diff is smaller
[W]  than the pre or post image text form
[W]  used for review
[W]
[W] -In between focus
[W] -is hard to keep consistently
[W] -errors may occur
[W] -
    diff --git a/engineer.txt b/engineer.txt
    new file mode 100644
    index 0000000..8fbc0ce
    --- /dev/null
    +++ b/engineer.txt
    @@ -0,0 +1,7 @@
[W] +A simple text is
[W] +written in paragraphs and
[B] +many more lines.
[B] +In between focus
[W] +is hard to keep consistently
[W] +errors may occur
[W] +
---8<---

W -> simple White text (diff.color.context)
R -> Red text (diff.color.old)
B -> Boundary marker (Maybe just diff.color.{old/new}
      or a new color to configure)

>
>    Using colors that stand out more prominently than for the regular
>    new/old lines is counter-productive for all of these, especially
>    for the first two purposes.  I may suggest using cyan (or any
>    color that is very close to the background) so that the presense
>    of moved lines are merely felt without being distracting.  IOW,
>    while reviewing code moving patch, moved parts want to be dimmed,
>    not highlighted.

I agree. So I could resend the algorithm used with other defaults
or try out the "boundary only iff adjacent blocks, else context color".

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 16/17] diff: buffer all output if asked to
  2017-05-25  5:34       ` Stefan Beller
@ 2017-05-26  1:09         ` Junio C Hamano
  0 siblings, 0 replies; 125+ messages in thread
From: Junio C Hamano @ 2017-05-26  1:09 UTC (permalink / raw)
  To: Stefan Beller
  Cc: git, Brandon Williams, Jonathan Nieder, Jonathan Tan, Jeff King,
	Michael Haggerty

Stefan Beller <sbeller@google.com> writes:

> Yes, this is essentially the v4 with small indentation fixes, so I
> assumed your signoff is still valid.

OK.  I just didn't expect to see one without "no changes since v4"
under the three-dash line.

Thanks.

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 00/17] Diff machine: highlight moved lines.
  2017-05-25 16:31     ` Stefan Beller
@ 2017-05-26  1:20       ` Junio C Hamano
  2017-05-26 19:30         ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Junio C Hamano @ 2017-05-26  1:20 UTC (permalink / raw)
  To: Stefan Beller
  Cc: git, Brandon Williams, Jonathan Nieder, Jonathan Tan, Jeff King,
	Michael Haggerty

Stefan Beller <sbeller@google.com> writes:

> As you turn on/off normal coloring via "color.diff" and this only extends
> the coloring scheme, I have the impression "color" is the right section.
> Maybe color.diffStyle=[normal/enhanced] and "enhanced" means this
> feature for now?

Hmph, I thought the intent of color.diff is "is the diff command
itself is colored?"  In other words, color.diff=false should give
you monochrome if you say "diff --word-diff", etc.

> The only option in the "diff" section related to color is diff.wsErrorHighlight
> which has a very similar purpose, so "diff.colorMoved" would fit in that
> scheme.

I didn't have "should diff output highlight whitespace errors?" in
mind when I wrote the message you are responding to, but yes, that
is quite similar to "should diff output show lines moved and lines
deleted/added differently?".

> So with these questions, I wonder if we want to color moved lines
> as "color.diff.context" (i.e. plain white text in the normal coloring scheme)
> This would serve the intended purpose of
> dimming the attention to moved lines.

Yes, but two points.

 (1) We want to do so while making it obvious where the boundary
     between two moved blocks of text whose destination (for
     moved-deleted lines) or source (for moved-added lines) is.

 (2) My message was an impression from using the code to review a
     patch that is meant to be "move without changing other things".
     For other purposes, there may be cases where moved ones may
     want to be highlighted, not dimmed.

> Regarding the last point of marking up adjacent blocks (which would
> indicate that there is a coherency issue or just moving from different
> places), we could highlight the last line of the previous block
> and first line of the next block in their "normal" colors (i.e.
> color.diff.old and color.diff.new).

Hmm.  That is an interesting thought.

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 00/17] Diff machine: highlight moved lines.
  2017-05-26  1:20       ` Junio C Hamano
@ 2017-05-26 19:30         ` Stefan Beller
  2017-05-27  0:18           ` [PATCH 0/1] " Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-05-26 19:30 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Brandon Williams, Jonathan Nieder, Jonathan Tan, Jeff King,
	Michael Haggerty

On Thu, May 25, 2017 at 6:20 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> As you turn on/off normal coloring via "color.diff" and this only extends
>> the coloring scheme, I have the impression "color" is the right section.
>> Maybe color.diffStyle=[normal/enhanced] and "enhanced" means this
>> feature for now?
>
> Hmph, I thought the intent of color.diff is "is the diff command
> itself is colored?"  In other words, color.diff=false should give
> you monochrome if you say "diff --word-diff", etc.

Yes, but in my understanding the "diff" doesn't apply to
the command, but the part of the output. I arrived at that
understanding as other commands (show/log/..) also respect
that setting, so the "diff" in color.diff is not the command, but
referring to something else, the output being the closest match. :)

And by that understanding color.diffStyle is just natural?

But I think that setting would be a bad idea as we'd rather have
multiple uncorrelated settings for coloring, which a style argument
would not.

>> The only option in the "diff" section related to color is diff.wsErrorHighlight
>> which has a very similar purpose, so "diff.colorMoved" would fit in that
>> scheme.

With the above reasoning, this may be missnamed and should rather be
color.wsErrorHighlight as it applies to more than just the diff command.

Note: The average user may not aware that diff/show/log are tiny wrappers
around the same backend for the heavy lifting.

> I didn't have "should diff output highlight whitespace errors?" in
> mind when I wrote the message you are responding to, but yes, that
> is quite similar to "should diff output show lines moved and lines
> deleted/added differently?".
>
>> So with these questions, I wonder if we want to color moved lines
>> as "color.diff.context" (i.e. plain white text in the normal coloring scheme)
>> This would serve the intended purpose of
>> dimming the attention to moved lines.
>
> Yes, but two points.
>
>  (1) We want to do so while making it obvious where the boundary
>      between two moved blocks of text whose destination (for
>      moved-deleted lines) or source (for moved-added lines) is.

Yes, that is what the boundary color would accomplish.
Any two adjacent blocks with the same sign would have
their boundary line colored this way.

>  (2) My message was an impression from using the code to review a
>      patch that is meant to be "move without changing other things".

Ok, glad you found it somewhat useful already.

>      For other purposes, there may be cases where moved ones may
>      want to be highlighted, not dimmed.

I wonder what these use cases would be?
(barring a --find-copies harder extension that would not just search the
current diff, but the whole tree)

That hints at just having an extra slot for the moved block, but the default
color could be the same as color.diff.context for dimming.

By now I also think we may not need different colors for additions
or removals, but keeping two colors is easy enough.

>> Regarding the last point of marking up adjacent blocks (which would
>> indicate that there is a coherency issue or just moving from different
>> places), we could highlight the last line of the previous block
>> and first line of the next block in their "normal" colors (i.e.
>> color.diff.old and color.diff.new).
>
> Hmm.  That is an interesting thought.

I'll try the implementation for that and see if it looks good.

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 125+ messages in thread

* [PATCH 0/1] Diff machine: highlight moved lines.
  2017-05-26 19:30         ` Stefan Beller
@ 2017-05-27  0:18           ` Stefan Beller
  2017-05-27  0:18             ` [PATCH 1/1] diff.c: color moved lines differently Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-05-27  0:18 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, peff, mhagger, jonathantanmy, Stefan Beller

This is a new version of the latest patch of sb/diff-color-move.

It seems as if different people prefer different settings for the
boundary/alternate coloring, so let's have all of them. (As the
block detection is rather simple we do not need a lot of overhead
to support multiple modes).

With more tests:
 * for each mode;
 * check that diff.colorMoved is respected
 
And it is documented as well.

Thanks,
Stefan

Stefan Beller (1):
  diff.c: color moved lines differently

 Documentation/config.txt       |  10 +-
 Documentation/diff-options.txt |  32 ++++
 color.h                        |   2 +
 diff.c                         | 342 +++++++++++++++++++++++++++++++++++--
 diff.h                         |  15 +-
 t/t4015-diff-whitespace.sh     | 373 +++++++++++++++++++++++++++++++++++++++++
 6 files changed, 760 insertions(+), 14 deletions(-)

Interdiff to what Junio has queued:
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 902d017c3b..73511a4603 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1051,12 +1051,9 @@ This does not affect linkgit:git-format-patch[1] or the
 'git-diff-{asterisk}' plumbing commands.  Can be overridden on the
 command line with the `--color[=<when>]` option.
 
-color.moved::
-	A boolean value, whether a diff should color moved lines
-	differently. The moved lines are searched for in the diff only.
-	Duplicated lines from somewhere in the project that are not
-	part of the diff are not colored as moved.
-	Defaults to false.
+diff.colorMoved::
+	If set moved lines in a diff are colored differently,
+	for details see '--color-moved' in linkgit:git-diff[1].
 
 color.diff.<slot>::
 	Use customized color for diff colorization.  `<slot>` specifies
@@ -1065,10 +1062,9 @@ color.diff.<slot>::
 	`meta` (metainformation), `frag`
 	(hunk header), 'func' (function in hunk header), `old` (removed lines),
 	`new` (added lines), `commit` (commit headers), `whitespace`
-	(highlighting whitespace errors), `oldMoved` (removed lines that
-	reappear), `newMoved` (added lines that were removed elsewhere),
-	`oldMovedAlternative` and `newMovedAlternative` (as a fallback to
-	cover adjacent blocks of moved code)
+	(highlighting whitespace errors), `oldMoved`, `newMoved`,
+	`oldMovedAlternative` and `newMovedAlternative` (See the '<mode>'
+	setting of '--color-moved' in linkgit:git-diff[1] for details).
 
 color.decorate.<slot>::
 	Use customized color for 'git log --decorate' output.  `<slot>` is one
diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt
index 89cc0f48de..25259dbbc3 100644
--- a/Documentation/diff-options.txt
+++ b/Documentation/diff-options.txt
@@ -231,6 +231,38 @@ ifdef::git-diff[]
 endif::git-diff[]
 	It is the same as `--color=never`.
 
+--color-moved[=<mode>]::
+	Moved lines of code are colored differently.
+ifdef::git-diff[]
+	It can be changed by the `diff.colorMoved` configuration setting.
+endif::git-diff[]
+	The <mode> defaults to 'no' if the option is not given
+	and to 'adjacentbounds' if the option with no mode is given.
+	The mode must be one of:
++
+--
+no::
+	Moved lines are not highlighted.
+nobounds::
+	Any line that is added in on location and was removed
+	in another location will be colored with 'color.diff.newmoved'.
+	Any line that is removed in on location and was added
+	in another location will be colored with 'color.diff.oldmoved'.
+allbounds::
+	Based on 'nobounds'. Additionally blocks of moved code are
+	detected and the first and last line of a block will be highlighted
+	using 'color.diff.newMovedAlternate' or
+	'color.diff.oldMovedAlternate'.
+adjacentbounds::
+	The same as 'allbounds' except that highlighting is only performed
+	at adjacent block boundaries of blocks that have the same sign.
+alternate::
+	Based on 'nobounds'. Additionally blocks of moved code are
+	detected. If moved blocks are adjacent mark one of them with the
+	alternative move color using 'color.diff.newMovedAlternate' or
+	'color.diff.oldMovedAlternate'.
+--
+
 --word-diff[=<mode>]::
 	Show a word diff, using the <mode> to delimit changed words.
 	By default, words are delimited by whitespace; see
diff --git a/color.h b/color.h
index 90627650fc..04b3b87929 100644
--- a/color.h
+++ b/color.h
@@ -42,6 +42,8 @@ struct strbuf;
 #define GIT_COLOR_BG_BLUE	"\033[44m"
 #define GIT_COLOR_BG_MAGENTA	"\033[45m"
 #define GIT_COLOR_BG_CYAN	"\033[46m"
+#define GIT_COLOR_DI_IT_CYAN	"\033[2;3;36m"
+#define GIT_COLOR_DI_IT_MAGENTA	"\033[2;3;35m"
 
 /* A special value meaning "no color selected" */
 #define GIT_COLOR_NIL "NIL"
diff --git a/diff.c b/diff.c
index 40cccafb67..efd2530a89 100644
--- a/diff.c
+++ b/diff.c
@@ -56,10 +56,10 @@ static char diff_colors[][COLOR_MAXLEN] = {
 	GIT_COLOR_YELLOW,	/* COMMIT */
 	GIT_COLOR_BG_RED,	/* WHITESPACE */
 	GIT_COLOR_NORMAL,	/* FUNCINFO */
-	GIT_COLOR_BOLD_RED,	/* OLD_MOVED_A */
-	GIT_COLOR_BG_RED,	/* OLD_MOVED_B */
-	GIT_COLOR_BOLD_GREEN,	/* NEW_MOVED_A */
-	GIT_COLOR_BG_GREEN,	/* NEW_MOVED_B */
+	GIT_COLOR_DI_IT_MAGENTA,/* OLD_MOVED */
+	GIT_COLOR_BG_RED,	/* OLD_MOVED ALTERNATIVE */
+	GIT_COLOR_DI_IT_CYAN,	/* NEW_MOVED */
+	GIT_COLOR_BG_GREEN,	/* NEW_MOVED ALTERNATIVE */
 };
 
 static NORETURN void die_want_option(const char *option_name)
@@ -241,14 +241,33 @@ int git_diff_heuristic_config(const char *var, const char *value, void *cb)
 	return 0;
 }
 
+static int parse_color_moved(const char *arg)
+{
+	if (!strcmp(arg, "no"))
+		return MOVED_LINES_NO;
+	else if (!strcmp(arg, "nobounds"))
+		return MOVED_LINES_BOUNDARY_NO;
+	else if (!strcmp(arg, "allbounds"))
+		return MOVED_LINES_BOUNDARY_ALL;
+	else if (!strcmp(arg, "adjacentbounds"))
+		return MOVED_LINES_BOUNDARY_ADJACENT;
+	else if (!strcmp(arg, "alternate"))
+		return MOVED_LINES_ALTERNATE;
+	else
+		return -1;
+}
+
 int git_diff_ui_config(const char *var, const char *value, void *cb)
 {
 	if (!strcmp(var, "diff.color") || !strcmp(var, "color.diff")) {
 		diff_use_color_default = git_config_colorbool(var, value);
 		return 0;
 	}
-	if (!strcmp(var, "color.moved")) {
-		diff_color_moved_default = git_config_bool(var, value);
+	if (!strcmp(var, "diff.colormoved")) {
+		int cm = parse_color_moved(value);
+		if (cm < 0)
+			return -1;
+		diff_color_moved_default = cm;
 		return 0;
 	}
 	if (!strcmp(var, "diff.context")) {
@@ -652,41 +671,69 @@ static void add_lines_to_move_detection(struct diff_options *o,
 	}
 }
 
+static void mark_color_as_moved_single_line(struct diff_options *o,
+					    struct diff_line *l, int alt_color)
+{
+	switch (l->sign) {
+	case '+':
+		l->set = diff_get_color_opt(o,
+			DIFF_FILE_NEW_MOVED + alt_color);
+		break;
+	case '-':
+		l->set = diff_get_color_opt(o,
+			DIFF_FILE_OLD_MOVED + alt_color);
+		break;
+	default:
+		die("BUG: we should have continued earlier?");
+	}
+}
+
 static void mark_color_as_moved(struct diff_options *o,
 				struct hashmap *add_lines,
 				struct hashmap *del_lines)
 {
 	struct moved_entry **pmb = NULL; /* potentially moved blocks */
+	struct diff_line *prev_line = NULL;
 	int pmb_nr = 0, pmb_alloc = 0;
-	int use_alt_color = 0;
-	int n;
+	int n, flipped_block = 0;
 
 	for (n = 0; n < o->line_buffer_nr; n++) {
 		struct hashmap *hm = NULL;
 		struct moved_entry *key;
 		struct moved_entry *match = NULL;
 		struct diff_line *l = &o->line_buffer[n];
-		int i, lp, rp;
+		int i, lp, rp, adjacent_blocks = 0;
 
+		/* Check for any match to color it as a move. */
 		switch (l->sign) {
 		case '+':
 			hm = del_lines;
+			key = prepare_entry(o, n);
+			match = hashmap_get(hm, key, o);
+			free(key);
 			break;
 		case '-':
 			hm = add_lines;
+			key = prepare_entry(o, n);
+			match = hashmap_get(hm, key, o);
+			free(key);
 			break;
-		default:
-			use_alt_color = 0;
-			pmb_nr = 0; /* no running sets */
+		default: ;
+		}
+
+		if (!match) {
+			pmb_nr = 0;
+			if (prev_line &&
+			    o->color_moved == MOVED_LINES_BOUNDARY_ALL)
+				mark_color_as_moved_single_line(o, prev_line, 1);
+			prev_line = NULL;
 			continue;
 		}
 
-		/* Check for any match to color it as a move. */
-		key = prepare_entry(o, n);
-		match = hashmap_get(hm, key, o);
-		free(key);
-		if (!match)
+		if (o->color_moved == MOVED_LINES_BOUNDARY_NO) {
+			mark_color_as_moved_single_line(o, l, 0);
 			continue;
+		}
 
 		/* Check any potential block runs, advance each or nullify */
 		for (i = 0; i < pmb_nr; i++) {
@@ -701,7 +748,7 @@ static void mark_color_as_moved(struct diff_options *o,
 			}
 		}
 
-		/* Shrink the set to the remaining runs */
+		/* Shrink the set of potential block to the remaining running */
 		for (lp = 0, rp = pmb_nr - 1; lp <= rp;) {
 			while (lp < pmb_nr && pmb[lp])
 				lp++;
@@ -719,34 +766,44 @@ static void mark_color_as_moved(struct diff_options *o,
 			}
 		}
 
-		if (rp > -1) {
-			/* Remember the number of running sets */
-			pmb_nr = rp + 1;
-		} else {
-			/* Toggle color */
-			use_alt_color = (use_alt_color + 1) % 2;
+		/* Remember the number of running sets */
+		pmb_nr = rp + 1;
 
-			/* Build up a new set */
-			pmb_nr = 0;
+		if (pmb_nr == 0) {
+			/*
+			 * This line is the start of a new block.
+			 * Setup the set of potential blocks.
+			 */
 			for (; match; match = hashmap_get_next(hm, match)) {
 				ALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);
 				pmb[pmb_nr++] = match;
 			}
+
+			if (o->color_moved == MOVED_LINES_BOUNDARY_ALL) {
+				adjacent_blocks = 1;
+			} else {
+				/* Check if two blocks are adjacent */
+				adjacent_blocks = prev_line &&
+						  prev_line->sign == l->sign;
+			}
 		}
 
-		switch (l->sign) {
-		case '+':
-			l->set = diff_get_color_opt(o,
-				DIFF_FILE_NEW_MOVED + use_alt_color);
-			break;
-		case '-':
-			l->set = diff_get_color_opt(o,
-				DIFF_FILE_OLD_MOVED + use_alt_color);
-			break;
-		default:
-			die("BUG: we should have continued earlier?");
+		if (o->color_moved == MOVED_LINES_ALTERNATE) {
+			if (adjacent_blocks)
+				flipped_block = (flipped_block + 1) % 2;
+			mark_color_as_moved_single_line(o, l, flipped_block);
+		} else {
+			/* MOVED_LINES_BOUNDARY_{ADJACENT, ALL} */
+			mark_color_as_moved_single_line(o, l, adjacent_blocks);
+			if (adjacent_blocks && prev_line)
+				prev_line->set = l->set;
 		}
+
+		prev_line = l;
 	}
+	if (prev_line && o->color_moved == MOVED_LINES_BOUNDARY_ALL)
+		mark_color_as_moved_single_line(o, prev_line, 1);
+
 	free(pmb);
 }
 
@@ -4291,10 +4348,18 @@ int diff_opt_parse(struct diff_options *options,
 	else if (!strcmp(arg, "--no-color"))
 		options->use_color = 0;
 	else if (!strcmp(arg, "--color-moved"))
-		options->color_moved = 1;
+		if (diff_color_moved_default)
+			options->color_moved = diff_color_moved_default;
+		else
+			options->color_moved = MOVED_LINES_BOUNDARY_ADJACENT;
 	else if (!strcmp(arg, "--no-color-moved"))
-		options->color_moved = 0;
-	else if (!strcmp(arg, "--color-words")) {
+		options->color_moved = MOVED_LINES_NO;
+	else if (skip_prefix(arg, "--color-moved=", &arg)) {
+		int cm = parse_color_moved(arg);
+		if (cm < 0)
+			die("bad --color-moved argument: %s", arg);
+		options->color_moved = cm;
+	} else if (!strcmp(arg, "--color-words")) {
 		options->use_color = 1;
 		options->word_diff = DIFF_WORDS_COLOR;
 	}
diff --git a/diff.h b/diff.h
index c6d806c6b5..d9fbafd383 100644
--- a/diff.h
+++ b/diff.h
@@ -230,7 +230,13 @@ struct diff_options {
 	struct diff_line *line_buffer;
 	int line_buffer_nr, line_buffer_alloc;
 
-	int color_moved;
+	enum {
+		MOVED_LINES_NO = 0,
+		MOVED_LINES_BOUNDARY_NO = 1,
+		MOVED_LINES_BOUNDARY_ALL = 2,
+		MOVED_LINES_BOUNDARY_ADJACENT = 3,
+		MOVED_LINES_ALTERNATE = 4,
+	} color_moved;
 };
 
 /* Emit [line_prefix] [set] line [reset] */
diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
index 0e92bf94bf..d4bd082af7 100755
--- a/t/t4015-diff-whitespace.sh
+++ b/t/t4015-diff-whitespace.sh
@@ -984,6 +984,8 @@ test_expect_success 'detect moved code, complete file' '
 	git add test.c &&
 	git commit -m "add main function" &&
 	git mv test.c main.c &&
+	test_config color.diff.oldMoved "normal red" &&
+	test_config color.diff.newMoved "normal green" &&
 	git diff HEAD --color-moved --no-renames | test_decode_color >actual &&
 	cat >expected <<-\EOF &&
 	<BOLD>diff --git a/main.c b/main.c<RESET>
@@ -1081,6 +1083,10 @@ test_expect_success 'detect moved code, inside file' '
 			bar();
 		}
 	EOF
+	test_config color.diff.oldMoved "normal red" &&
+	test_config color.diff.newMoved "normal green" &&
+	test_config color.diff.oldMovedAlternative "bold red" &&
+	test_config color.diff.newMovedAlternative "bold green" &&
 	git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
 	cat <<-\EOF >expected &&
 	<BOLD>diff --git a/main.c b/main.c<RESET>
@@ -1125,79 +1131,179 @@ test_expect_success 'detect moved code, inside file' '
 '
 
 test_expect_success 'detect permutations inside moved code' '
-	# reusing the move example from last test:
-	cat <<-\EOF >main.c &&
-		#include<stdio.h>
-		int stuff()
-		{
-			printf("Hello ");
-			printf("World\n");
-		}
-
-		int main()
-		{
-			foo();
-		}
+	git reset --hard &&
+	cat <<-\EOF >lines.txt &&
+		line 1
+		line 2
+		line 3
+		line 4
+		line 5
+		line 6
+		line 7
+		line 8
+		line 9
+		line 10
+		line 11
+		line 12
+		line 13
+		line 14
+		line 15
+		line 16
 	EOF
-	cat <<-\EOF >test.c &&
-		#include<stdio.h>
-		int bar()
-		{
-			printf("Hello World, but different\n");
-		}
+	git add lines.txt &&
+	git commit -m "add poetry" &&
+	cat <<-\EOF >lines.txt &&
+		line 4
+		line 5
+		line 6
+		line 7
+		line 8
+		line 9
+		line 1
+		line 2
+		line 3
+		line 14
+		line 15
+		line 16
+		line 10
+		line 11
+		line 12
+		line 13
+	EOF
+	test_config color.diff.oldMoved "magenta" &&
+	test_config color.diff.newMoved "cyan" &&
+	test_config color.diff.oldMovedAlternative "blue" &&
+	test_config color.diff.newMovedAlternative "yellow" &&
 
-		int secure_foo(struct user *u)
-		{
-			foo(u);
-			if (!u->is_allowed_foo)
-				return;
-		}
 
-		int another_function()
-		{
-			bar();
-		}
+	git diff HEAD --no-renames --color-moved=nobounds| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+		<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+		<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+		<BOLD>--- a/lines.txt<RESET>
+		<BOLD>+++ b/lines.txt<RESET>
+		<CYAN>@@ -1,16 +1,16 @@<RESET>
+		<MAGENTA>-line 1<RESET>
+		<MAGENTA>-line 2<RESET>
+		<MAGENTA>-line 3<RESET>
+		 line 4<RESET>
+		 line 5<RESET>
+		 line 6<RESET>
+		 line 7<RESET>
+		 line 8<RESET>
+		 line 9<RESET>
+		<CYAN>+<RESET><CYAN>line 1<RESET>
+		<CYAN>+<RESET><CYAN>line 2<RESET>
+		<CYAN>+<RESET><CYAN>line 3<RESET>
+		<CYAN>+<RESET><CYAN>line 14<RESET>
+		<CYAN>+<RESET><CYAN>line 15<RESET>
+		<CYAN>+<RESET><CYAN>line 16<RESET>
+		 line 10<RESET>
+		 line 11<RESET>
+		 line 12<RESET>
+		 line 13<RESET>
+		<MAGENTA>-line 14<RESET>
+		<MAGENTA>-line 15<RESET>
+		<MAGENTA>-line 16<RESET>
+	EOF
+	test_cmp expected actual &&
+
+	git diff HEAD --no-renames --color-moved=adjacentbounds| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+	<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+	<BOLD>--- a/lines.txt<RESET>
+	<BOLD>+++ b/lines.txt<RESET>
+	<CYAN>@@ -1,16 +1,16 @@<RESET>
+	<MAGENTA>-line 1<RESET>
+	<MAGENTA>-line 2<RESET>
+	<MAGENTA>-line 3<RESET>
+	 line 4<RESET>
+	 line 5<RESET>
+	 line 6<RESET>
+	 line 7<RESET>
+	 line 8<RESET>
+	 line 9<RESET>
+	<CYAN>+<RESET><CYAN>line 1<RESET>
+	<CYAN>+<RESET><CYAN>line 2<RESET>
+	<YELLOW>+<RESET><YELLOW>line 3<RESET>
+	<YELLOW>+<RESET><YELLOW>line 14<RESET>
+	<CYAN>+<RESET><CYAN>line 15<RESET>
+	<CYAN>+<RESET><CYAN>line 16<RESET>
+	 line 10<RESET>
+	 line 11<RESET>
+	 line 12<RESET>
+	 line 13<RESET>
+	<MAGENTA>-line 14<RESET>
+	<MAGENTA>-line 15<RESET>
+	<MAGENTA>-line 16<RESET>
 	EOF
+	test_cmp expected actual &&
+
+	test_config diff.colorMoved alternate &&
 	git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
 	cat <<-\EOF >expected &&
-	<BOLD>diff --git a/main.c b/main.c<RESET>
-	<BOLD>index 27a619c..7cf9336 100644<RESET>
-	<BOLD>--- a/main.c<RESET>
-	<BOLD>+++ b/main.c<RESET>
-	<CYAN>@@ -5,13 +5,6 @@<RESET> <RESET>printf("Hello ");<RESET>
-	 printf("World\n");<RESET>
-	 }<RESET>
-	 <RESET>
-	<BRED>-int secure_foo(struct user *u)<RESET>
-	<BRED>-{<RESET>
-	<BOLD;RED>-if (!u->is_allowed_foo)<RESET>
-	<BOLD;RED>-return;<RESET>
-	<BRED>-foo(u);<RESET>
-	<BOLD;RED>-}<RESET>
-	<BOLD;RED>-<RESET>
-	 int main()<RESET>
-	 {<RESET>
-	 foo();<RESET>
-	<BOLD>diff --git a/test.c b/test.c<RESET>
-	<BOLD>index 1dc1d85..2bedec9 100644<RESET>
-	<BOLD>--- a/test.c<RESET>
-	<BOLD>+++ b/test.c<RESET>
-	<CYAN>@@ -4,6 +4,13 @@<RESET> <RESET>int bar()<RESET>
-	 printf("Hello World, but different\n");<RESET>
-	 }<RESET>
-	 <RESET>
-	<BGREEN>+<RESET><BGREEN>int secure_foo(struct user *u)<RESET>
-	<BGREEN>+<RESET><BGREEN>{<RESET>
-	<BOLD;GREEN>+<RESET><BOLD;GREEN>foo(u);<RESET>
-	<BGREEN>+<RESET><BGREEN>if (!u->is_allowed_foo)<RESET>
-	<BGREEN>+<RESET><BGREEN>return;<RESET>
-	<BOLD;GREEN>+<RESET><BOLD;GREEN>}<RESET>
-	<BOLD;GREEN>+<RESET>
-	 int another_function()<RESET>
-	 {<RESET>
-	 bar();<RESET>
+	<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+	<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+	<BOLD>--- a/lines.txt<RESET>
+	<BOLD>+++ b/lines.txt<RESET>
+	<CYAN>@@ -1,16 +1,16 @@<RESET>
+	<MAGENTA>-line 1<RESET>
+	<MAGENTA>-line 2<RESET>
+	<MAGENTA>-line 3<RESET>
+	 line 4<RESET>
+	 line 5<RESET>
+	 line 6<RESET>
+	 line 7<RESET>
+	 line 8<RESET>
+	 line 9<RESET>
+	<CYAN>+<RESET><CYAN>line 1<RESET>
+	<CYAN>+<RESET><CYAN>line 2<RESET>
+	<CYAN>+<RESET><CYAN>line 3<RESET>
+	<YELLOW>+<RESET><YELLOW>line 14<RESET>
+	<YELLOW>+<RESET><YELLOW>line 15<RESET>
+	<YELLOW>+<RESET><YELLOW>line 16<RESET>
+	 line 10<RESET>
+	 line 11<RESET>
+	 line 12<RESET>
+	 line 13<RESET>
+	<BLUE>-line 14<RESET>
+	<BLUE>-line 15<RESET>
+	<BLUE>-line 16<RESET>
 	EOF
+	test_cmp expected actual &&
 
+	test_config diff.colorMoved allbounds &&
+	git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+	<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+	<BOLD>--- a/lines.txt<RESET>
+	<BOLD>+++ b/lines.txt<RESET>
+	<CYAN>@@ -1,16 +1,16 @@<RESET>
+	<BLUE>-line 1<RESET>
+	<MAGENTA>-line 2<RESET>
+	<BLUE>-line 3<RESET>
+	 line 4<RESET>
+	 line 5<RESET>
+	 line 6<RESET>
+	 line 7<RESET>
+	 line 8<RESET>
+	 line 9<RESET>
+	<YELLOW>+<RESET><YELLOW>line 1<RESET>
+	<CYAN>+<RESET><CYAN>line 2<RESET>
+	<YELLOW>+<RESET><YELLOW>line 3<RESET>
+	<YELLOW>+<RESET><YELLOW>line 14<RESET>
+	<CYAN>+<RESET><CYAN>line 15<RESET>
+	<YELLOW>+<RESET><YELLOW>line 16<RESET>
+	 line 10<RESET>
+	 line 11<RESET>
+	 line 12<RESET>
+	 line 13<RESET>
+	<BLUE>-line 14<RESET>
+	<MAGENTA>-line 15<RESET>
+	<BLUE>-line 16<RESET>
+	EOF
 	test_cmp expected actual
 '
 



-- 
2.13.0.17.gab62347cd9


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 1/1] diff.c: color moved lines differently
  2017-05-27  0:18           ` [PATCH 0/1] " Stefan Beller
@ 2017-05-27  0:18             ` Stefan Beller
  2017-05-27  7:05               ` Philip Oakley
  0 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-05-27  0:18 UTC (permalink / raw)
  To: gitster
  Cc: git, bmwill, jrnieder, peff, mhagger, jonathantanmy, Stefan Beller

When a patch consists mostly of moving blocks of code around, it can
be quite tedious to ensure that the blocks are moved verbatim, and not
undesirably modified in the move. To that end, color blocks that are
moved within the same patch differently. For example (OM, del, add,
and NM are different colors):

    [OM]  -void sensitive_stuff(void)
    [OM]  -{
    [OM]  -        if (!is_authorized_user())
    [OM]  -                die("unauthorized");
    [OM]  -        sensitive_stuff(spanning,
    [OM]  -                        multiple,
    [OM]  -                        lines);
    [OM]  -}

           void another_function()
           {
    [del] -        printf("foo");
    [add] +        printf("bar");
           }

    [NM]  +void sensitive_stuff(void)
    [NM]  +{
    [NM]  +        if (!is_authorized_user())
    [NM]  +                die("unauthorized");
    [NM]  +        sensitive_stuff(spanning,
    [NM]  +                        multiple,
    [NM]  +                        lines);
    [NM]  +}

However adjacent blocks may be problematic. For example, in this
potentially malicious patch, the swapping of blocks can be spotted:

    [OM]  -void sensitive_stuff(void)
    [OM]  -{
    [OMA] -        if (!is_authorized_user())
    [OMA] -                die("unauthorized");
    [OM]  -        sensitive_stuff(spanning,
    [OM]  -                        multiple,
    [OM]  -                        lines);
    [OMA] -}

           void another_function()
           {
    [del] -        printf("foo");
    [add] +        printf("bar");
           }

    [NM]  +void sensitive_stuff(void)
    [NM]  +{
    [NMA] +        sensitive_stuff(spanning,
    [NMA] +                        multiple,
    [NMA] +                        lines);
    [NM]  +        if (!is_authorized_user())
    [NM]  +                die("unauthorized");
    [NMA] +}

If the moved code is larger, it is easier to hide some permutation in the
code, which is why some alternative coloring is needed.

As the reviewers attention should be brought to the places, where the
difference is introduced to the moved code, we cannot just have one new
color for all of moved code.

First I implemented an alternative design, which would try to fingerprint
a line by its neighbors to detect if we are in a block or at the boundary.
This idea iss error prone as it inspected each line and its neighboring
lines to determine if the line was (a) moved and (b) if was deep inside
a hunk by having matching neighboring lines. This is unreliable as the
we can construct hunks which have equal neighbors that just exceed the
number of lines inspected. (Think of 'AXYZBXYZCXYZD..' with each letter
as a line, that is permutated to AXYZCXYZBXYZD..').

Instead this provides a dynamic programming greedy algorithm that finds
the largest moved hunk and then has several modes on highlighting bounds.

A note on the options '--submodule=diff' and '--color-words/--word-diff':
In the conversion to use emit_line in the prior patches both submodules
as well as word diff output carefully chose to call emit_line with sign=0.
All output with sign=0 is ignored for move detection purposes in this
patch, such that no weird looking output will be generated for these
cases. This leads to another thought: We could pass on '--color-moved' to
submodules such that they color up moved lines for themselves. If we'd do
so only line moves within a repository boundary are marked up.

Helped-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/config.txt       |  10 +-
 Documentation/diff-options.txt |  32 ++++
 color.h                        |   2 +
 diff.c                         | 342 +++++++++++++++++++++++++++++++++++--
 diff.h                         |  15 +-
 t/t4015-diff-whitespace.sh     | 373 +++++++++++++++++++++++++++++++++++++++++
 6 files changed, 760 insertions(+), 14 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 475e874d51..73511a4603 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1051,14 +1051,20 @@ This does not affect linkgit:git-format-patch[1] or the
 'git-diff-{asterisk}' plumbing commands.  Can be overridden on the
 command line with the `--color[=<when>]` option.
 
+diff.colorMoved::
+	If set moved lines in a diff are colored differently,
+	for details see '--color-moved' in linkgit:git-diff[1].
+
 color.diff.<slot>::
 	Use customized color for diff colorization.  `<slot>` specifies
 	which part of the patch to use the specified color, and is one
 	of `context` (context text - `plain` is a historical synonym),
 	`meta` (metainformation), `frag`
 	(hunk header), 'func' (function in hunk header), `old` (removed lines),
-	`new` (added lines), `commit` (commit headers), or `whitespace`
-	(highlighting whitespace errors).
+	`new` (added lines), `commit` (commit headers), `whitespace`
+	(highlighting whitespace errors), `oldMoved`, `newMoved`,
+	`oldMovedAlternative` and `newMovedAlternative` (See the '<mode>'
+	setting of '--color-moved' in linkgit:git-diff[1] for details).
 
 color.decorate.<slot>::
 	Use customized color for 'git log --decorate' output.  `<slot>` is one
diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt
index 89cc0f48de..25259dbbc3 100644
--- a/Documentation/diff-options.txt
+++ b/Documentation/diff-options.txt
@@ -231,6 +231,38 @@ ifdef::git-diff[]
 endif::git-diff[]
 	It is the same as `--color=never`.
 
+--color-moved[=<mode>]::
+	Moved lines of code are colored differently.
+ifdef::git-diff[]
+	It can be changed by the `diff.colorMoved` configuration setting.
+endif::git-diff[]
+	The <mode> defaults to 'no' if the option is not given
+	and to 'adjacentbounds' if the option with no mode is given.
+	The mode must be one of:
++
+--
+no::
+	Moved lines are not highlighted.
+nobounds::
+	Any line that is added in on location and was removed
+	in another location will be colored with 'color.diff.newmoved'.
+	Any line that is removed in on location and was added
+	in another location will be colored with 'color.diff.oldmoved'.
+allbounds::
+	Based on 'nobounds'. Additionally blocks of moved code are
+	detected and the first and last line of a block will be highlighted
+	using 'color.diff.newMovedAlternate' or
+	'color.diff.oldMovedAlternate'.
+adjacentbounds::
+	The same as 'allbounds' except that highlighting is only performed
+	at adjacent block boundaries of blocks that have the same sign.
+alternate::
+	Based on 'nobounds'. Additionally blocks of moved code are
+	detected. If moved blocks are adjacent mark one of them with the
+	alternative move color using 'color.diff.newMovedAlternate' or
+	'color.diff.oldMovedAlternate'.
+--
+
 --word-diff[=<mode>]::
 	Show a word diff, using the <mode> to delimit changed words.
 	By default, words are delimited by whitespace; see
diff --git a/color.h b/color.h
index 90627650fc..04b3b87929 100644
--- a/color.h
+++ b/color.h
@@ -42,6 +42,8 @@ struct strbuf;
 #define GIT_COLOR_BG_BLUE	"\033[44m"
 #define GIT_COLOR_BG_MAGENTA	"\033[45m"
 #define GIT_COLOR_BG_CYAN	"\033[46m"
+#define GIT_COLOR_DI_IT_CYAN	"\033[2;3;36m"
+#define GIT_COLOR_DI_IT_MAGENTA	"\033[2;3;35m"
 
 /* A special value meaning "no color selected" */
 #define GIT_COLOR_NIL "NIL"
diff --git a/diff.c b/diff.c
index a3c16ef827..efd2530a89 100644
--- a/diff.c
+++ b/diff.c
@@ -31,6 +31,7 @@ static int diff_indent_heuristic; /* experimental */
 static int diff_rename_limit_default = 400;
 static int diff_suppress_blank_empty;
 static int diff_use_color_default = -1;
+static int diff_color_moved_default;
 static int diff_context_default = 3;
 static int diff_interhunk_context_default;
 static const char *diff_word_regex_cfg;
@@ -55,6 +56,10 @@ static char diff_colors[][COLOR_MAXLEN] = {
 	GIT_COLOR_YELLOW,	/* COMMIT */
 	GIT_COLOR_BG_RED,	/* WHITESPACE */
 	GIT_COLOR_NORMAL,	/* FUNCINFO */
+	GIT_COLOR_DI_IT_MAGENTA,/* OLD_MOVED */
+	GIT_COLOR_BG_RED,	/* OLD_MOVED ALTERNATIVE */
+	GIT_COLOR_DI_IT_CYAN,	/* NEW_MOVED */
+	GIT_COLOR_BG_GREEN,	/* NEW_MOVED ALTERNATIVE */
 };
 
 static NORETURN void die_want_option(const char *option_name)
@@ -80,6 +85,14 @@ static int parse_diff_color_slot(const char *var)
 		return DIFF_WHITESPACE;
 	if (!strcasecmp(var, "func"))
 		return DIFF_FUNCINFO;
+	if (!strcasecmp(var, "oldmoved"))
+		return DIFF_FILE_OLD_MOVED;
+	if (!strcasecmp(var, "oldmovedalternative"))
+		return DIFF_FILE_OLD_MOVED_ALT;
+	if (!strcasecmp(var, "newmoved"))
+		return DIFF_FILE_NEW_MOVED;
+	if (!strcasecmp(var, "newmovedalternative"))
+		return DIFF_FILE_NEW_MOVED_ALT;
 	return -1;
 }
 
@@ -228,12 +241,35 @@ int git_diff_heuristic_config(const char *var, const char *value, void *cb)
 	return 0;
 }
 
+static int parse_color_moved(const char *arg)
+{
+	if (!strcmp(arg, "no"))
+		return MOVED_LINES_NO;
+	else if (!strcmp(arg, "nobounds"))
+		return MOVED_LINES_BOUNDARY_NO;
+	else if (!strcmp(arg, "allbounds"))
+		return MOVED_LINES_BOUNDARY_ALL;
+	else if (!strcmp(arg, "adjacentbounds"))
+		return MOVED_LINES_BOUNDARY_ADJACENT;
+	else if (!strcmp(arg, "alternate"))
+		return MOVED_LINES_ALTERNATE;
+	else
+		return -1;
+}
+
 int git_diff_ui_config(const char *var, const char *value, void *cb)
 {
 	if (!strcmp(var, "diff.color") || !strcmp(var, "color.diff")) {
 		diff_use_color_default = git_config_colorbool(var, value);
 		return 0;
 	}
+	if (!strcmp(var, "diff.colormoved")) {
+		int cm = parse_color_moved(value);
+		if (cm < 0)
+			return -1;
+		diff_color_moved_default = cm;
+		return 0;
+	}
 	if (!strcmp(var, "diff.context")) {
 		diff_context_default = git_config_int(var, value);
 		if (diff_context_default < 0)
@@ -354,6 +390,88 @@ int git_diff_basic_config(const char *var, const char *value, void *cb)
 	return git_default_config(var, value, cb);
 }
 
+struct moved_entry {
+	struct hashmap_entry ent;
+	const struct diff_line *line;
+	struct moved_entry *next_line;
+};
+
+static void get_ws_cleaned_string(const struct diff_line *l,
+				  struct strbuf *out)
+{
+	int i;
+	for (i = 0; i < l->len; i++) {
+		if (isspace(l->line[i]))
+			continue;
+		strbuf_addch(out, l->line[i]);
+	}
+}
+
+static int diff_line_cmp_no_ws(const struct diff_line *a,
+					 const struct diff_line *b,
+					 const void *keydata)
+{
+	int ret;
+	struct strbuf sba = STRBUF_INIT;
+	struct strbuf sbb = STRBUF_INIT;
+
+	get_ws_cleaned_string(a, &sba);
+	get_ws_cleaned_string(b, &sbb);
+	ret = sba.len != sbb.len || strncmp(sba.buf, sbb.buf, sba.len);
+
+	strbuf_release(&sba);
+	strbuf_release(&sbb);
+	return ret;
+}
+
+static int diff_line_cmp(const struct diff_line *a,
+				   const struct diff_line *b,
+				   const void *keydata)
+{
+	return a->len != b->len || strncmp(a->line, b->line, a->len);
+}
+
+static int moved_entry_cmp(const struct moved_entry *a,
+			   const struct moved_entry *b,
+			   const void *keydata)
+{
+	return diff_line_cmp(a->line, b->line, keydata);
+}
+
+static int moved_entry_cmp_no_ws(const struct moved_entry *a,
+				 const struct moved_entry *b,
+				 const void *keydata)
+{
+	return diff_line_cmp_no_ws(a->line, b->line, keydata);
+}
+
+static unsigned get_line_hash(struct diff_line *line, unsigned ignore_ws)
+{
+	static struct strbuf sb = STRBUF_INIT;
+
+	if (ignore_ws) {
+		strbuf_reset(&sb);
+		get_ws_cleaned_string(line, &sb);
+		return memhash(sb.buf, sb.len);
+	} else {
+		return memhash(line->line, line->len);
+	}
+}
+
+static struct moved_entry *prepare_entry(struct diff_options *o,
+					 int line_no)
+{
+	struct moved_entry *ret = xmalloc(sizeof(*ret));
+	unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
+	struct diff_line *l = &o->line_buffer[line_no];
+
+	ret->ent.hash = get_line_hash(l, ignore_ws);
+	ret->line = l;
+	ret->next_line = NULL;
+
+	return ret;
+}
+
 static char *quote_two(const char *one, const char *two)
 {
 	int need_one = quote_c_style(one, NULL, NULL, 1);
@@ -516,6 +634,179 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,
 	ecbdata->blank_at_eof_in_postimage = (at - l2) + 1;
 }
 
+static void add_lines_to_move_detection(struct diff_options *o,
+					struct hashmap *add_lines,
+					struct hashmap *del_lines)
+{
+	struct moved_entry *prev_line = NULL;
+
+	int n;
+	for (n = 0; n < o->line_buffer_nr; n++) {
+		int sign = 0;
+		struct hashmap *hm;
+		struct moved_entry *key;
+
+		switch (o->line_buffer[n].sign) {
+		case '+':
+			sign = '+';
+			hm = add_lines;
+			break;
+		case '-':
+			sign = '-';
+			hm = del_lines;
+			break;
+		case ' ':
+		default:
+			prev_line = NULL;
+			continue;
+		}
+
+		key = prepare_entry(o, n);
+		if (prev_line &&
+		    prev_line->line->sign == sign)
+			prev_line->next_line = key;
+
+		hashmap_add(hm, key);
+		prev_line = key;
+	}
+}
+
+static void mark_color_as_moved_single_line(struct diff_options *o,
+					    struct diff_line *l, int alt_color)
+{
+	switch (l->sign) {
+	case '+':
+		l->set = diff_get_color_opt(o,
+			DIFF_FILE_NEW_MOVED + alt_color);
+		break;
+	case '-':
+		l->set = diff_get_color_opt(o,
+			DIFF_FILE_OLD_MOVED + alt_color);
+		break;
+	default:
+		die("BUG: we should have continued earlier?");
+	}
+}
+
+static void mark_color_as_moved(struct diff_options *o,
+				struct hashmap *add_lines,
+				struct hashmap *del_lines)
+{
+	struct moved_entry **pmb = NULL; /* potentially moved blocks */
+	struct diff_line *prev_line = NULL;
+	int pmb_nr = 0, pmb_alloc = 0;
+	int n, flipped_block = 0;
+
+	for (n = 0; n < o->line_buffer_nr; n++) {
+		struct hashmap *hm = NULL;
+		struct moved_entry *key;
+		struct moved_entry *match = NULL;
+		struct diff_line *l = &o->line_buffer[n];
+		int i, lp, rp, adjacent_blocks = 0;
+
+		/* Check for any match to color it as a move. */
+		switch (l->sign) {
+		case '+':
+			hm = del_lines;
+			key = prepare_entry(o, n);
+			match = hashmap_get(hm, key, o);
+			free(key);
+			break;
+		case '-':
+			hm = add_lines;
+			key = prepare_entry(o, n);
+			match = hashmap_get(hm, key, o);
+			free(key);
+			break;
+		default: ;
+		}
+
+		if (!match) {
+			pmb_nr = 0;
+			if (prev_line &&
+			    o->color_moved == MOVED_LINES_BOUNDARY_ALL)
+				mark_color_as_moved_single_line(o, prev_line, 1);
+			prev_line = NULL;
+			continue;
+		}
+
+		if (o->color_moved == MOVED_LINES_BOUNDARY_NO) {
+			mark_color_as_moved_single_line(o, l, 0);
+			continue;
+		}
+
+		/* Check any potential block runs, advance each or nullify */
+		for (i = 0; i < pmb_nr; i++) {
+			struct moved_entry *p = pmb[i];
+			struct moved_entry *pnext = (p && p->next_line) ?
+					p->next_line : NULL;
+			if (pnext &&
+			    !diff_line_cmp(pnext->line, l, o)) {
+				pmb[i] = p->next_line;
+			} else {
+				pmb[i] = NULL;
+			}
+		}
+
+		/* Shrink the set of potential block to the remaining running */
+		for (lp = 0, rp = pmb_nr - 1; lp <= rp;) {
+			while (lp < pmb_nr && pmb[lp])
+				lp++;
+			/* lp points at the first NULL now */
+
+			while (rp > -1 && !pmb[rp])
+				rp--;
+			/* rp points at the last non-NULL */
+
+			if (lp < pmb_nr && rp > -1 && lp < rp) {
+				pmb[lp] = pmb[rp];
+				pmb[rp] = NULL;
+				rp--;
+				lp++;
+			}
+		}
+
+		/* Remember the number of running sets */
+		pmb_nr = rp + 1;
+
+		if (pmb_nr == 0) {
+			/*
+			 * This line is the start of a new block.
+			 * Setup the set of potential blocks.
+			 */
+			for (; match; match = hashmap_get_next(hm, match)) {
+				ALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);
+				pmb[pmb_nr++] = match;
+			}
+
+			if (o->color_moved == MOVED_LINES_BOUNDARY_ALL) {
+				adjacent_blocks = 1;
+			} else {
+				/* Check if two blocks are adjacent */
+				adjacent_blocks = prev_line &&
+						  prev_line->sign == l->sign;
+			}
+		}
+
+		if (o->color_moved == MOVED_LINES_ALTERNATE) {
+			if (adjacent_blocks)
+				flipped_block = (flipped_block + 1) % 2;
+			mark_color_as_moved_single_line(o, l, flipped_block);
+		} else {
+			/* MOVED_LINES_BOUNDARY_{ADJACENT, ALL} */
+			mark_color_as_moved_single_line(o, l, adjacent_blocks);
+			if (adjacent_blocks && prev_line)
+				prev_line->set = l->set;
+		}
+
+		prev_line = l;
+	}
+	if (prev_line && o->color_moved == MOVED_LINES_BOUNDARY_ALL)
+		mark_color_as_moved_single_line(o, prev_line, 1);
+
+	free(pmb);
+}
+
 static void emit_diff_line(struct diff_options *o,
 			   struct diff_line *e)
 {
@@ -3518,6 +3809,8 @@ void diff_setup(struct diff_options *options)
 	options->line_buffer = NULL;
 	options->line_buffer_nr = 0;
 	options->line_buffer_alloc = 0;
+
+	options->color_moved = diff_color_moved_default;
 }
 
 void diff_setup_done(struct diff_options *options)
@@ -3627,6 +3920,9 @@ void diff_setup_done(struct diff_options *options)
 
 	if (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)
 		die(_("--follow requires exactly one pathspec"));
+
+	if (!options->use_color || external_diff())
+		options->color_moved = 0;
 }
 
 static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)
@@ -4051,7 +4347,19 @@ int diff_opt_parse(struct diff_options *options,
 	}
 	else if (!strcmp(arg, "--no-color"))
 		options->use_color = 0;
-	else if (!strcmp(arg, "--color-words")) {
+	else if (!strcmp(arg, "--color-moved"))
+		if (diff_color_moved_default)
+			options->color_moved = diff_color_moved_default;
+		else
+			options->color_moved = MOVED_LINES_BOUNDARY_ADJACENT;
+	else if (!strcmp(arg, "--no-color-moved"))
+		options->color_moved = MOVED_LINES_NO;
+	else if (skip_prefix(arg, "--color-moved=", &arg)) {
+		int cm = parse_color_moved(arg);
+		if (cm < 0)
+			die("bad --color-moved argument: %s", arg);
+		options->color_moved = cm;
+	} else if (!strcmp(arg, "--color-words")) {
 		options->use_color = 1;
 		options->word_diff = DIFF_WORDS_COLOR;
 	}
@@ -4856,16 +5164,9 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 {
 	int i;
 	struct diff_queue_struct *q = &diff_queued_diff;
-	/*
-	 * For testing purposes we want to make sure the diff machinery
-	 * works completely with the buffer. If there is anything emitted
-	 * outside the emit_diff_line, then the order is screwed
-	 * up and the tests will fail.
-	 *
-	 * TODO (later in this series):
-	 * We'll unset this flag in a later patch.
-	 */
-	o->use_buffer = 1;
+
+	if (o->color_moved)
+		o->use_buffer = 1;
 
 	for (i = 0; i < q->nr; i++) {
 		struct diff_filepair *p = q->queue[i];
@@ -4874,6 +5175,24 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 	}
 
 	if (o->use_buffer) {
+		if (o->color_moved) {
+			struct hashmap add_lines, del_lines;
+			unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
+
+			hashmap_init(&del_lines, ignore_ws ?
+				(hashmap_cmp_fn)moved_entry_cmp_no_ws :
+				(hashmap_cmp_fn)moved_entry_cmp, 0);
+			hashmap_init(&add_lines, ignore_ws ?
+				(hashmap_cmp_fn)moved_entry_cmp_no_ws :
+				(hashmap_cmp_fn)moved_entry_cmp, 0);
+
+			add_lines_to_move_detection(o, &add_lines, &del_lines);
+			mark_color_as_moved(o, &add_lines, &del_lines);
+
+			hashmap_free(&add_lines, 0);
+			hashmap_free(&del_lines, 0);
+		}
+
 		for (i = 0; i < o->line_buffer_nr; i++)
 			emit_diff_line(o, &o->line_buffer[i]);
 
@@ -4962,6 +5281,7 @@ void diff_flush(struct diff_options *options)
 		if (!options->file)
 			die_errno("Could not open /dev/null");
 		options->close_file = 1;
+		options->color_moved = 0;
 		for (i = 0; i < q->nr; i++) {
 			struct diff_filepair *p = q->queue[i];
 			if (check_pair_status(p))
diff --git a/diff.h b/diff.h
index be51e8f867..d9fbafd383 100644
--- a/diff.h
+++ b/diff.h
@@ -7,6 +7,7 @@
 #include "tree-walk.h"
 #include "pathspec.h"
 #include "object.h"
+#include "hashmap.h"
 
 struct rev_info;
 struct diff_options;
@@ -228,6 +229,14 @@ struct diff_options {
 
 	struct diff_line *line_buffer;
 	int line_buffer_nr, line_buffer_alloc;
+
+	enum {
+		MOVED_LINES_NO = 0,
+		MOVED_LINES_BOUNDARY_NO = 1,
+		MOVED_LINES_BOUNDARY_ALL = 2,
+		MOVED_LINES_BOUNDARY_ADJACENT = 3,
+		MOVED_LINES_ALTERNATE = 4,
+	} color_moved;
 };
 
 /* Emit [line_prefix] [set] line [reset] */
@@ -243,7 +252,11 @@ enum color_diff {
 	DIFF_FILE_NEW = 5,
 	DIFF_COMMIT = 6,
 	DIFF_WHITESPACE = 7,
-	DIFF_FUNCINFO = 8
+	DIFF_FUNCINFO = 8,
+	DIFF_FILE_OLD_MOVED = 9,
+	DIFF_FILE_OLD_MOVED_ALT = 10,
+	DIFF_FILE_NEW_MOVED = 11,
+	DIFF_FILE_NEW_MOVED_ALT = 12
 };
 const char *diff_get_color(int diff_use_color, enum color_diff ix);
 #define diff_get_color_opt(o, ix) \
diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
index 289806d0c7..d4bd082af7 100755
--- a/t/t4015-diff-whitespace.sh
+++ b/t/t4015-diff-whitespace.sh
@@ -972,4 +972,377 @@ test_expect_success 'option overrides diff.wsErrorHighlight' '
 
 '
 
+test_expect_success 'detect moved code, complete file' '
+	git reset --hard &&
+	cat <<-\EOF >test.c &&
+	#include<stdio.h>
+	main()
+	{
+	printf("Hello World");
+	}
+	EOF
+	git add test.c &&
+	git commit -m "add main function" &&
+	git mv test.c main.c &&
+	test_config color.diff.oldMoved "normal red" &&
+	test_config color.diff.newMoved "normal green" &&
+	git diff HEAD --color-moved --no-renames | test_decode_color >actual &&
+	cat >expected <<-\EOF &&
+	<BOLD>diff --git a/main.c b/main.c<RESET>
+	<BOLD>new file mode 100644<RESET>
+	<BOLD>index 0000000..a986c57<RESET>
+	<BOLD>--- /dev/null<RESET>
+	<BOLD>+++ b/main.c<RESET>
+	<CYAN>@@ -0,0 +1,5 @@<RESET>
+	<BGREEN>+<RESET><BGREEN>#include<stdio.h><RESET>
+	<BGREEN>+<RESET><BGREEN>main()<RESET>
+	<BGREEN>+<RESET><BGREEN>{<RESET>
+	<BGREEN>+<RESET><BGREEN>printf("Hello World");<RESET>
+	<BGREEN>+<RESET><BGREEN>}<RESET>
+	<BOLD>diff --git a/test.c b/test.c<RESET>
+	<BOLD>deleted file mode 100644<RESET>
+	<BOLD>index a986c57..0000000<RESET>
+	<BOLD>--- a/test.c<RESET>
+	<BOLD>+++ /dev/null<RESET>
+	<CYAN>@@ -1,5 +0,0 @@<RESET>
+	<BRED>-#include<stdio.h><RESET>
+	<BRED>-main()<RESET>
+	<BRED>-{<RESET>
+	<BRED>-printf("Hello World");<RESET>
+	<BRED>-}<RESET>
+	EOF
+
+	test_cmp expected actual
+'
+
+test_expect_success 'detect moved code, inside file' '
+	git reset --hard &&
+	cat <<-\EOF >main.c &&
+		#include<stdio.h>
+		int stuff()
+		{
+			printf("Hello ");
+			printf("World\n");
+		}
+
+		int secure_foo(struct user *u)
+		{
+			if (!u->is_allowed_foo)
+				return;
+			foo(u);
+		}
+
+		int main()
+		{
+			foo();
+		}
+	EOF
+	cat <<-\EOF >test.c &&
+		#include<stdio.h>
+		int bar()
+		{
+			printf("Hello World, but different\n");
+		}
+
+		int another_function()
+		{
+			bar();
+		}
+	EOF
+	git add main.c test.c &&
+	git commit -m "add main and test file" &&
+	cat <<-\EOF >main.c &&
+		#include<stdio.h>
+		int stuff()
+		{
+			printf("Hello ");
+			printf("World\n");
+		}
+
+		int main()
+		{
+			foo();
+		}
+	EOF
+	cat <<-\EOF >test.c &&
+		#include<stdio.h>
+		int bar()
+		{
+			printf("Hello World, but different\n");
+		}
+
+		int secure_foo(struct user *u)
+		{
+			if (!u->is_allowed_foo)
+				return;
+			foo(u);
+		}
+
+		int another_function()
+		{
+			bar();
+		}
+	EOF
+	test_config color.diff.oldMoved "normal red" &&
+	test_config color.diff.newMoved "normal green" &&
+	test_config color.diff.oldMovedAlternative "bold red" &&
+	test_config color.diff.newMovedAlternative "bold green" &&
+	git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/main.c b/main.c<RESET>
+	<BOLD>index 27a619c..7cf9336 100644<RESET>
+	<BOLD>--- a/main.c<RESET>
+	<BOLD>+++ b/main.c<RESET>
+	<CYAN>@@ -5,13 +5,6 @@<RESET> <RESET>printf("Hello ");<RESET>
+	 printf("World\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BRED>-int secure_foo(struct user *u)<RESET>
+	<BRED>-{<RESET>
+	<BRED>-if (!u->is_allowed_foo)<RESET>
+	<BRED>-return;<RESET>
+	<BRED>-foo(u);<RESET>
+	<BRED>-}<RESET>
+	<BRED>-<RESET>
+	 int main()<RESET>
+	 {<RESET>
+	 foo();<RESET>
+	<BOLD>diff --git a/test.c b/test.c<RESET>
+	<BOLD>index 1dc1d85..e34eb69 100644<RESET>
+	<BOLD>--- a/test.c<RESET>
+	<BOLD>+++ b/test.c<RESET>
+	<CYAN>@@ -4,6 +4,13 @@<RESET> <RESET>int bar()<RESET>
+	 printf("Hello World, but different\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BGREEN>+<RESET><BGREEN>int secure_foo(struct user *u)<RESET>
+	<BGREEN>+<RESET><BGREEN>{<RESET>
+	<BGREEN>+<RESET><BGREEN>if (!u->is_allowed_foo)<RESET>
+	<BGREEN>+<RESET><BGREEN>return;<RESET>
+	<BGREEN>+<RESET><BGREEN>foo(u);<RESET>
+	<BGREEN>+<RESET><BGREEN>}<RESET>
+	<BGREEN>+<RESET>
+	 int another_function()<RESET>
+	 {<RESET>
+	 bar();<RESET>
+	EOF
+
+	test_cmp expected actual
+'
+
+test_expect_success 'detect permutations inside moved code' '
+	git reset --hard &&
+	cat <<-\EOF >lines.txt &&
+		line 1
+		line 2
+		line 3
+		line 4
+		line 5
+		line 6
+		line 7
+		line 8
+		line 9
+		line 10
+		line 11
+		line 12
+		line 13
+		line 14
+		line 15
+		line 16
+	EOF
+	git add lines.txt &&
+	git commit -m "add poetry" &&
+	cat <<-\EOF >lines.txt &&
+		line 4
+		line 5
+		line 6
+		line 7
+		line 8
+		line 9
+		line 1
+		line 2
+		line 3
+		line 14
+		line 15
+		line 16
+		line 10
+		line 11
+		line 12
+		line 13
+	EOF
+	test_config color.diff.oldMoved "magenta" &&
+	test_config color.diff.newMoved "cyan" &&
+	test_config color.diff.oldMovedAlternative "blue" &&
+	test_config color.diff.newMovedAlternative "yellow" &&
+
+
+	git diff HEAD --no-renames --color-moved=nobounds| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+		<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+		<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+		<BOLD>--- a/lines.txt<RESET>
+		<BOLD>+++ b/lines.txt<RESET>
+		<CYAN>@@ -1,16 +1,16 @@<RESET>
+		<MAGENTA>-line 1<RESET>
+		<MAGENTA>-line 2<RESET>
+		<MAGENTA>-line 3<RESET>
+		 line 4<RESET>
+		 line 5<RESET>
+		 line 6<RESET>
+		 line 7<RESET>
+		 line 8<RESET>
+		 line 9<RESET>
+		<CYAN>+<RESET><CYAN>line 1<RESET>
+		<CYAN>+<RESET><CYAN>line 2<RESET>
+		<CYAN>+<RESET><CYAN>line 3<RESET>
+		<CYAN>+<RESET><CYAN>line 14<RESET>
+		<CYAN>+<RESET><CYAN>line 15<RESET>
+		<CYAN>+<RESET><CYAN>line 16<RESET>
+		 line 10<RESET>
+		 line 11<RESET>
+		 line 12<RESET>
+		 line 13<RESET>
+		<MAGENTA>-line 14<RESET>
+		<MAGENTA>-line 15<RESET>
+		<MAGENTA>-line 16<RESET>
+	EOF
+	test_cmp expected actual &&
+
+	git diff HEAD --no-renames --color-moved=adjacentbounds| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+	<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+	<BOLD>--- a/lines.txt<RESET>
+	<BOLD>+++ b/lines.txt<RESET>
+	<CYAN>@@ -1,16 +1,16 @@<RESET>
+	<MAGENTA>-line 1<RESET>
+	<MAGENTA>-line 2<RESET>
+	<MAGENTA>-line 3<RESET>
+	 line 4<RESET>
+	 line 5<RESET>
+	 line 6<RESET>
+	 line 7<RESET>
+	 line 8<RESET>
+	 line 9<RESET>
+	<CYAN>+<RESET><CYAN>line 1<RESET>
+	<CYAN>+<RESET><CYAN>line 2<RESET>
+	<YELLOW>+<RESET><YELLOW>line 3<RESET>
+	<YELLOW>+<RESET><YELLOW>line 14<RESET>
+	<CYAN>+<RESET><CYAN>line 15<RESET>
+	<CYAN>+<RESET><CYAN>line 16<RESET>
+	 line 10<RESET>
+	 line 11<RESET>
+	 line 12<RESET>
+	 line 13<RESET>
+	<MAGENTA>-line 14<RESET>
+	<MAGENTA>-line 15<RESET>
+	<MAGENTA>-line 16<RESET>
+	EOF
+	test_cmp expected actual &&
+
+	test_config diff.colorMoved alternate &&
+	git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+	<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+	<BOLD>--- a/lines.txt<RESET>
+	<BOLD>+++ b/lines.txt<RESET>
+	<CYAN>@@ -1,16 +1,16 @@<RESET>
+	<MAGENTA>-line 1<RESET>
+	<MAGENTA>-line 2<RESET>
+	<MAGENTA>-line 3<RESET>
+	 line 4<RESET>
+	 line 5<RESET>
+	 line 6<RESET>
+	 line 7<RESET>
+	 line 8<RESET>
+	 line 9<RESET>
+	<CYAN>+<RESET><CYAN>line 1<RESET>
+	<CYAN>+<RESET><CYAN>line 2<RESET>
+	<CYAN>+<RESET><CYAN>line 3<RESET>
+	<YELLOW>+<RESET><YELLOW>line 14<RESET>
+	<YELLOW>+<RESET><YELLOW>line 15<RESET>
+	<YELLOW>+<RESET><YELLOW>line 16<RESET>
+	 line 10<RESET>
+	 line 11<RESET>
+	 line 12<RESET>
+	 line 13<RESET>
+	<BLUE>-line 14<RESET>
+	<BLUE>-line 15<RESET>
+	<BLUE>-line 16<RESET>
+	EOF
+	test_cmp expected actual &&
+
+	test_config diff.colorMoved allbounds &&
+	git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+	<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+	<BOLD>--- a/lines.txt<RESET>
+	<BOLD>+++ b/lines.txt<RESET>
+	<CYAN>@@ -1,16 +1,16 @@<RESET>
+	<BLUE>-line 1<RESET>
+	<MAGENTA>-line 2<RESET>
+	<BLUE>-line 3<RESET>
+	 line 4<RESET>
+	 line 5<RESET>
+	 line 6<RESET>
+	 line 7<RESET>
+	 line 8<RESET>
+	 line 9<RESET>
+	<YELLOW>+<RESET><YELLOW>line 1<RESET>
+	<CYAN>+<RESET><CYAN>line 2<RESET>
+	<YELLOW>+<RESET><YELLOW>line 3<RESET>
+	<YELLOW>+<RESET><YELLOW>line 14<RESET>
+	<CYAN>+<RESET><CYAN>line 15<RESET>
+	<YELLOW>+<RESET><YELLOW>line 16<RESET>
+	 line 10<RESET>
+	 line 11<RESET>
+	 line 12<RESET>
+	 line 13<RESET>
+	<BLUE>-line 14<RESET>
+	<MAGENTA>-line 15<RESET>
+	<BLUE>-line 16<RESET>
+	EOF
+	test_cmp expected actual
+'
+
+test_expect_success 'move detection does not mess up colored words' '
+	cat <<-\EOF >text.txt &&
+	Lorem Ipsum is simply dummy text of the printing and typesetting industry.
+	EOF
+	git add text.txt &&
+	git commit -a -m "clean state" &&
+	cat <<-\EOF >text.txt &&
+	simply Lorem Ipsum dummy is text of the typesetting and printing industry.
+	EOF
+	git diff --color-moved --word-diff >actual &&
+	git diff --word-diff >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'move detection with submodules' '
+	test_create_repo bananas &&
+	echo ripe >bananas/recipe &&
+	git -C bananas add recipe &&
+	test_commit fruit &&
+	test_commit -C bananas recipe &&
+	git submodule add ./bananas &&
+	git add bananas &&
+	git commit -a -m "bananas are like a heavy library?" &&
+	echo foul >bananas/recipe &&
+	echo ripe >fruit.t &&
+
+	git diff --submodule=diff --color-moved >actual &&
+
+	# no move detection as the moved line is across repository boundaries.
+	test_decode_color <actual >decoded_actual &&
+	! grep BGREEN decoded_actual &&
+	! grep BRED decoded_actual &&
+
+	# nor did we mess with it another way
+	git diff --submodule=diff | test_decode_color >expect &&
+	test_cmp expect decoded_actual
+'
+
 test_done
-- 
2.13.0.17.gab62347cd9


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* Re: [PATCH 1/1] diff.c: color moved lines differently
  2017-05-27  0:18             ` [PATCH 1/1] diff.c: color moved lines differently Stefan Beller
@ 2017-05-27  7:05               ` Philip Oakley
  2017-05-30 21:33                 ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Philip Oakley @ 2017-05-27  7:05 UTC (permalink / raw)
  To: Stefan Beller, gitster
  Cc: git, bmwill, jrnieder, peff, mhagger, jonathantanmy, Stefan Beller

a couple of mispellings in the doc parts:
  s/on location/one location/
[code not checked]
----- Original Message ----- 
From: "Stefan Beller" <sbeller@google.com>
Subject: [PATCH 1/1] diff.c: color moved lines differently


> When a patch consists mostly of moving blocks of code around, it can
> be quite tedious to ensure that the blocks are moved verbatim, and not
> undesirably modified in the move. To that end, color blocks that are
> moved within the same patch differently. For example (OM, del, add,
> and NM are different colors):
>
>    [OM]  -void sensitive_stuff(void)
>    [OM]  -{
>    [OM]  -        if (!is_authorized_user())
>    [OM]  -                die("unauthorized");
>    [OM]  -        sensitive_stuff(spanning,
>    [OM]  -                        multiple,
>    [OM]  -                        lines);
>    [OM]  -}
>
>           void another_function()
>           {
>    [del] -        printf("foo");
>    [add] +        printf("bar");
>           }
>
>    [NM]  +void sensitive_stuff(void)
>    [NM]  +{
>    [NM]  +        if (!is_authorized_user())
>    [NM]  +                die("unauthorized");
>    [NM]  +        sensitive_stuff(spanning,
>    [NM]  +                        multiple,
>    [NM]  +                        lines);
>    [NM]  +}
>
> However adjacent blocks may be problematic. For example, in this
> potentially malicious patch, the swapping of blocks can be spotted:
>
>    [OM]  -void sensitive_stuff(void)
>    [OM]  -{
>    [OMA] -        if (!is_authorized_user())
>    [OMA] -                die("unauthorized");
>    [OM]  -        sensitive_stuff(spanning,
>    [OM]  -                        multiple,
>    [OM]  -                        lines);
>    [OMA] -}
>
>           void another_function()
>           {
>    [del] -        printf("foo");
>    [add] +        printf("bar");
>           }
>
>    [NM]  +void sensitive_stuff(void)
>    [NM]  +{
>    [NMA] +        sensitive_stuff(spanning,
>    [NMA] +                        multiple,
>    [NMA] +                        lines);
>    [NM]  +        if (!is_authorized_user())
>    [NM]  +                die("unauthorized");
>    [NMA] +}
>
> If the moved code is larger, it is easier to hide some permutation in the
> code, which is why some alternative coloring is needed.
>
> As the reviewers attention should be brought to the places, where the
> difference is introduced to the moved code, we cannot just have one new
> color for all of moved code.
>
> First I implemented an alternative design, which would try to fingerprint
> a line by its neighbors to detect if we are in a block or at the boundary.
> This idea iss error prone as it inspected each line and its neighboring
> lines to determine if the line was (a) moved and (b) if was deep inside
> a hunk by having matching neighboring lines. This is unreliable as the
> we can construct hunks which have equal neighbors that just exceed the
> number of lines inspected. (Think of 'AXYZBXYZCXYZD..' with each letter
> as a line, that is permutated to AXYZCXYZBXYZD..').
>
> Instead this provides a dynamic programming greedy algorithm that finds
> the largest moved hunk and then has several modes on highlighting bounds.
>
> A note on the options '--submodule=diff' and '--color-words/--word-diff':
> In the conversion to use emit_line in the prior patches both submodules
> as well as word diff output carefully chose to call emit_line with sign=0.
> All output with sign=0 is ignored for move detection purposes in this
> patch, such that no weird looking output will be generated for these
> cases. This leads to another thought: We could pass on '--color-moved' to
> submodules such that they color up moved lines for themselves. If we'd do
> so only line moves within a repository boundary are marked up.
>
> Helped-by: Jonathan Tan <jonathantanmy@google.com>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
> Documentation/config.txt       |  10 +-
> Documentation/diff-options.txt |  32 ++++
> color.h                        |   2 +
> diff.c                         | 342 +++++++++++++++++++++++++++++++++++--
> diff.h                         |  15 +-
> t/t4015-diff-whitespace.sh     | 373 
> +++++++++++++++++++++++++++++++++++++++++
> 6 files changed, 760 insertions(+), 14 deletions(-)
>
> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index 475e874d51..73511a4603 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -1051,14 +1051,20 @@ This does not affect linkgit:git-format-patch[1] 
> or the
> 'git-diff-{asterisk}' plumbing commands.  Can be overridden on the
> command line with the `--color[=<when>]` option.
>
> +diff.colorMoved::
> + If set moved lines in a diff are colored differently,
> + for details see '--color-moved' in linkgit:git-diff[1].
> +
> color.diff.<slot>::
>  Use customized color for diff colorization.  `<slot>` specifies
>  which part of the patch to use the specified color, and is one
>  of `context` (context text - `plain` is a historical synonym),
>  `meta` (metainformation), `frag`
>  (hunk header), 'func' (function in hunk header), `old` (removed lines),
> - `new` (added lines), `commit` (commit headers), or `whitespace`
> - (highlighting whitespace errors).
> + `new` (added lines), `commit` (commit headers), `whitespace`
> + (highlighting whitespace errors), `oldMoved`, `newMoved`,
> + `oldMovedAlternative` and `newMovedAlternative` (See the '<mode>'
> + setting of '--color-moved' in linkgit:git-diff[1] for details).
>
> color.decorate.<slot>::
>  Use customized color for 'git log --decorate' output.  `<slot>` is one
> diff --git a/Documentation/diff-options.txt 
> b/Documentation/diff-options.txt
> index 89cc0f48de..25259dbbc3 100644
> --- a/Documentation/diff-options.txt
> +++ b/Documentation/diff-options.txt
> @@ -231,6 +231,38 @@ ifdef::git-diff[]
> endif::git-diff[]
>  It is the same as `--color=never`.
>
> +--color-moved[=<mode>]::
> + Moved lines of code are colored differently.
> +ifdef::git-diff[]
> + It can be changed by the `diff.colorMoved` configuration setting.
> +endif::git-diff[]
> + The <mode> defaults to 'no' if the option is not given
> + and to 'adjacentbounds' if the option with no mode is given.
> + The mode must be one of:
> ++
> +--
> +no::
> + Moved lines are not highlighted.
> +nobounds::
> + Any line that is added in on location and was removed
s/on location/one location/

> + in another location will be colored with 'color.diff.newmoved'.
> + Any line that is removed in on location and was added
s/on location/one location/

> + in another location will be colored with 'color.diff.oldmoved'.
> +allbounds::
> + Based on 'nobounds'. Additionally blocks of moved code are
> + detected and the first and last line of a block will be highlighted
> + using 'color.diff.newMovedAlternate' or
> + 'color.diff.oldMovedAlternate'.
> +adjacentbounds::
> + The same as 'allbounds' except that highlighting is only performed
> + at adjacent block boundaries of blocks that have the same sign.
> +alternate::
> + Based on 'nobounds'. Additionally blocks of moved code are
> + detected. If moved blocks are adjacent mark one of them with the
> + alternative move color using 'color.diff.newMovedAlternate' or
> + 'color.diff.oldMovedAlternate'.
> +--
> +
> --word-diff[=<mode>]::
>  Show a word diff, using the <mode> to delimit changed words.
>  By default, words are delimited by whitespace; see

--
Philip

> diff --git a/color.h b/color.h
> index 90627650fc..04b3b87929 100644
> --- a/color.h
> +++ b/color.h
> @@ -42,6 +42,8 @@ struct strbuf;
> #define GIT_COLOR_BG_BLUE "\033[44m"
> #define GIT_COLOR_BG_MAGENTA "\033[45m"
> #define GIT_COLOR_BG_CYAN "\033[46m"
> +#define GIT_COLOR_DI_IT_CYAN "\033[2;3;36m"
> +#define GIT_COLOR_DI_IT_MAGENTA "\033[2;3;35m"
>
> /* A special value meaning "no color selected" */
> #define GIT_COLOR_NIL "NIL"
> diff --git a/diff.c b/diff.c
> index a3c16ef827..efd2530a89 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -31,6 +31,7 @@ static int diff_indent_heuristic; /* experimental */
> static int diff_rename_limit_default = 400;
> static int diff_suppress_blank_empty;
> static int diff_use_color_default = -1;
> +static int diff_color_moved_default;
> static int diff_context_default = 3;
> static int diff_interhunk_context_default;
> static const char *diff_word_regex_cfg;
> @@ -55,6 +56,10 @@ static char diff_colors[][COLOR_MAXLEN] = {
>  GIT_COLOR_YELLOW, /* COMMIT */
>  GIT_COLOR_BG_RED, /* WHITESPACE */
>  GIT_COLOR_NORMAL, /* FUNCINFO */
> + GIT_COLOR_DI_IT_MAGENTA,/* OLD_MOVED */
> + GIT_COLOR_BG_RED, /* OLD_MOVED ALTERNATIVE */
> + GIT_COLOR_DI_IT_CYAN, /* NEW_MOVED */
> + GIT_COLOR_BG_GREEN, /* NEW_MOVED ALTERNATIVE */
> };
>
> static NORETURN void die_want_option(const char *option_name)
> @@ -80,6 +85,14 @@ static int parse_diff_color_slot(const char *var)
>  return DIFF_WHITESPACE;
>  if (!strcasecmp(var, "func"))
>  return DIFF_FUNCINFO;
> + if (!strcasecmp(var, "oldmoved"))
> + return DIFF_FILE_OLD_MOVED;
> + if (!strcasecmp(var, "oldmovedalternative"))
> + return DIFF_FILE_OLD_MOVED_ALT;
> + if (!strcasecmp(var, "newmoved"))
> + return DIFF_FILE_NEW_MOVED;
> + if (!strcasecmp(var, "newmovedalternative"))
> + return DIFF_FILE_NEW_MOVED_ALT;
>  return -1;
> }
>
> @@ -228,12 +241,35 @@ int git_diff_heuristic_config(const char *var, const 
> char *value, void *cb)
>  return 0;
> }
>
> +static int parse_color_moved(const char *arg)
> +{
> + if (!strcmp(arg, "no"))
> + return MOVED_LINES_NO;
> + else if (!strcmp(arg, "nobounds"))
> + return MOVED_LINES_BOUNDARY_NO;
> + else if (!strcmp(arg, "allbounds"))
> + return MOVED_LINES_BOUNDARY_ALL;
> + else if (!strcmp(arg, "adjacentbounds"))
> + return MOVED_LINES_BOUNDARY_ADJACENT;
> + else if (!strcmp(arg, "alternate"))
> + return MOVED_LINES_ALTERNATE;
> + else
> + return -1;
> +}
> +
> int git_diff_ui_config(const char *var, const char *value, void *cb)
> {
>  if (!strcmp(var, "diff.color") || !strcmp(var, "color.diff")) {
>  diff_use_color_default = git_config_colorbool(var, value);
>  return 0;
>  }
> + if (!strcmp(var, "diff.colormoved")) {
> + int cm = parse_color_moved(value);
> + if (cm < 0)
> + return -1;
> + diff_color_moved_default = cm;
> + return 0;
> + }
>  if (!strcmp(var, "diff.context")) {
>  diff_context_default = git_config_int(var, value);
>  if (diff_context_default < 0)
> @@ -354,6 +390,88 @@ int git_diff_basic_config(const char *var, const char 
> *value, void *cb)
>  return git_default_config(var, value, cb);
> }
>
> +struct moved_entry {
> + struct hashmap_entry ent;
> + const struct diff_line *line;
> + struct moved_entry *next_line;
> +};
> +
> +static void get_ws_cleaned_string(const struct diff_line *l,
> +   struct strbuf *out)
> +{
> + int i;
> + for (i = 0; i < l->len; i++) {
> + if (isspace(l->line[i]))
> + continue;
> + strbuf_addch(out, l->line[i]);
> + }
> +}
> +
> +static int diff_line_cmp_no_ws(const struct diff_line *a,
> + const struct diff_line *b,
> + const void *keydata)
> +{
> + int ret;
> + struct strbuf sba = STRBUF_INIT;
> + struct strbuf sbb = STRBUF_INIT;
> +
> + get_ws_cleaned_string(a, &sba);
> + get_ws_cleaned_string(b, &sbb);
> + ret = sba.len != sbb.len || strncmp(sba.buf, sbb.buf, sba.len);
> +
> + strbuf_release(&sba);
> + strbuf_release(&sbb);
> + return ret;
> +}
> +
> +static int diff_line_cmp(const struct diff_line *a,
> +    const struct diff_line *b,
> +    const void *keydata)
> +{
> + return a->len != b->len || strncmp(a->line, b->line, a->len);
> +}
> +
> +static int moved_entry_cmp(const struct moved_entry *a,
> +    const struct moved_entry *b,
> +    const void *keydata)
> +{
> + return diff_line_cmp(a->line, b->line, keydata);
> +}
> +
> +static int moved_entry_cmp_no_ws(const struct moved_entry *a,
> + const struct moved_entry *b,
> + const void *keydata)
> +{
> + return diff_line_cmp_no_ws(a->line, b->line, keydata);
> +}
> +
> +static unsigned get_line_hash(struct diff_line *line, unsigned ignore_ws)
> +{
> + static struct strbuf sb = STRBUF_INIT;
> +
> + if (ignore_ws) {
> + strbuf_reset(&sb);
> + get_ws_cleaned_string(line, &sb);
> + return memhash(sb.buf, sb.len);
> + } else {
> + return memhash(line->line, line->len);
> + }
> +}
> +
> +static struct moved_entry *prepare_entry(struct diff_options *o,
> + int line_no)
> +{
> + struct moved_entry *ret = xmalloc(sizeof(*ret));
> + unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
> + struct diff_line *l = &o->line_buffer[line_no];
> +
> + ret->ent.hash = get_line_hash(l, ignore_ws);
> + ret->line = l;
> + ret->next_line = NULL;
> +
> + return ret;
> +}
> +
> static char *quote_two(const char *one, const char *two)
> {
>  int need_one = quote_c_style(one, NULL, NULL, 1);
> @@ -516,6 +634,179 @@ static void check_blank_at_eof(mmfile_t *mf1, 
> mmfile_t *mf2,
>  ecbdata->blank_at_eof_in_postimage = (at - l2) + 1;
> }
>
> +static void add_lines_to_move_detection(struct diff_options *o,
> + struct hashmap *add_lines,
> + struct hashmap *del_lines)
> +{
> + struct moved_entry *prev_line = NULL;
> +
> + int n;
> + for (n = 0; n < o->line_buffer_nr; n++) {
> + int sign = 0;
> + struct hashmap *hm;
> + struct moved_entry *key;
> +
> + switch (o->line_buffer[n].sign) {
> + case '+':
> + sign = '+';
> + hm = add_lines;
> + break;
> + case '-':
> + sign = '-';
> + hm = del_lines;
> + break;
> + case ' ':
> + default:
> + prev_line = NULL;
> + continue;
> + }
> +
> + key = prepare_entry(o, n);
> + if (prev_line &&
> +     prev_line->line->sign == sign)
> + prev_line->next_line = key;
> +
> + hashmap_add(hm, key);
> + prev_line = key;
> + }
> +}
> +
> +static void mark_color_as_moved_single_line(struct diff_options *o,
> +     struct diff_line *l, int alt_color)
> +{
> + switch (l->sign) {
> + case '+':
> + l->set = diff_get_color_opt(o,
> + DIFF_FILE_NEW_MOVED + alt_color);
> + break;
> + case '-':
> + l->set = diff_get_color_opt(o,
> + DIFF_FILE_OLD_MOVED + alt_color);
> + break;
> + default:
> + die("BUG: we should have continued earlier?");
> + }
> +}
> +
> +static void mark_color_as_moved(struct diff_options *o,
> + struct hashmap *add_lines,
> + struct hashmap *del_lines)
> +{
> + struct moved_entry **pmb = NULL; /* potentially moved blocks */
> + struct diff_line *prev_line = NULL;
> + int pmb_nr = 0, pmb_alloc = 0;
> + int n, flipped_block = 0;
> +
> + for (n = 0; n < o->line_buffer_nr; n++) {
> + struct hashmap *hm = NULL;
> + struct moved_entry *key;
> + struct moved_entry *match = NULL;
> + struct diff_line *l = &o->line_buffer[n];
> + int i, lp, rp, adjacent_blocks = 0;
> +
> + /* Check for any match to color it as a move. */
> + switch (l->sign) {
> + case '+':
> + hm = del_lines;
> + key = prepare_entry(o, n);
> + match = hashmap_get(hm, key, o);
> + free(key);
> + break;
> + case '-':
> + hm = add_lines;
> + key = prepare_entry(o, n);
> + match = hashmap_get(hm, key, o);
> + free(key);
> + break;
> + default: ;
> + }
> +
> + if (!match) {
> + pmb_nr = 0;
> + if (prev_line &&
> +     o->color_moved == MOVED_LINES_BOUNDARY_ALL)
> + mark_color_as_moved_single_line(o, prev_line, 1);
> + prev_line = NULL;
> + continue;
> + }
> +
> + if (o->color_moved == MOVED_LINES_BOUNDARY_NO) {
> + mark_color_as_moved_single_line(o, l, 0);
> + continue;
> + }
> +
> + /* Check any potential block runs, advance each or nullify */
> + for (i = 0; i < pmb_nr; i++) {
> + struct moved_entry *p = pmb[i];
> + struct moved_entry *pnext = (p && p->next_line) ?
> + p->next_line : NULL;
> + if (pnext &&
> +     !diff_line_cmp(pnext->line, l, o)) {
> + pmb[i] = p->next_line;
> + } else {
> + pmb[i] = NULL;
> + }
> + }
> +
> + /* Shrink the set of potential block to the remaining running */
> + for (lp = 0, rp = pmb_nr - 1; lp <= rp;) {
> + while (lp < pmb_nr && pmb[lp])
> + lp++;
> + /* lp points at the first NULL now */
> +
> + while (rp > -1 && !pmb[rp])
> + rp--;
> + /* rp points at the last non-NULL */
> +
> + if (lp < pmb_nr && rp > -1 && lp < rp) {
> + pmb[lp] = pmb[rp];
> + pmb[rp] = NULL;
> + rp--;
> + lp++;
> + }
> + }
> +
> + /* Remember the number of running sets */
> + pmb_nr = rp + 1;
> +
> + if (pmb_nr == 0) {
> + /*
> + * This line is the start of a new block.
> + * Setup the set of potential blocks.
> + */
> + for (; match; match = hashmap_get_next(hm, match)) {
> + ALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);
> + pmb[pmb_nr++] = match;
> + }
> +
> + if (o->color_moved == MOVED_LINES_BOUNDARY_ALL) {
> + adjacent_blocks = 1;
> + } else {
> + /* Check if two blocks are adjacent */
> + adjacent_blocks = prev_line &&
> +   prev_line->sign == l->sign;
> + }
> + }
> +
> + if (o->color_moved == MOVED_LINES_ALTERNATE) {
> + if (adjacent_blocks)
> + flipped_block = (flipped_block + 1) % 2;
> + mark_color_as_moved_single_line(o, l, flipped_block);
> + } else {
> + /* MOVED_LINES_BOUNDARY_{ADJACENT, ALL} */
> + mark_color_as_moved_single_line(o, l, adjacent_blocks);
> + if (adjacent_blocks && prev_line)
> + prev_line->set = l->set;
> + }
> +
> + prev_line = l;
> + }
> + if (prev_line && o->color_moved == MOVED_LINES_BOUNDARY_ALL)
> + mark_color_as_moved_single_line(o, prev_line, 1);
> +
> + free(pmb);
> +}
> +
> static void emit_diff_line(struct diff_options *o,
>     struct diff_line *e)
> {
> @@ -3518,6 +3809,8 @@ void diff_setup(struct diff_options *options)
>  options->line_buffer = NULL;
>  options->line_buffer_nr = 0;
>  options->line_buffer_alloc = 0;
> +
> + options->color_moved = diff_color_moved_default;
> }
>
> void diff_setup_done(struct diff_options *options)
> @@ -3627,6 +3920,9 @@ void diff_setup_done(struct diff_options *options)
>
>  if (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)
>  die(_("--follow requires exactly one pathspec"));
> +
> + if (!options->use_color || external_diff())
> + options->color_moved = 0;
> }
>
> static int opt_arg(const char *arg, int arg_short, const char *arg_long, 
> int *val)
> @@ -4051,7 +4347,19 @@ int diff_opt_parse(struct diff_options *options,
>  }
>  else if (!strcmp(arg, "--no-color"))
>  options->use_color = 0;
> - else if (!strcmp(arg, "--color-words")) {
> + else if (!strcmp(arg, "--color-moved"))
> + if (diff_color_moved_default)
> + options->color_moved = diff_color_moved_default;
> + else
> + options->color_moved = MOVED_LINES_BOUNDARY_ADJACENT;
> + else if (!strcmp(arg, "--no-color-moved"))
> + options->color_moved = MOVED_LINES_NO;
> + else if (skip_prefix(arg, "--color-moved=", &arg)) {
> + int cm = parse_color_moved(arg);
> + if (cm < 0)
> + die("bad --color-moved argument: %s", arg);
> + options->color_moved = cm;
> + } else if (!strcmp(arg, "--color-words")) {
>  options->use_color = 1;
>  options->word_diff = DIFF_WORDS_COLOR;
>  }
> @@ -4856,16 +5164,9 @@ static void diff_flush_patch_all_file_pairs(struct 
> diff_options *o)
> {
>  int i;
>  struct diff_queue_struct *q = &diff_queued_diff;
> - /*
> - * For testing purposes we want to make sure the diff machinery
> - * works completely with the buffer. If there is anything emitted
> - * outside the emit_diff_line, then the order is screwed
> - * up and the tests will fail.
> - *
> - * TODO (later in this series):
> - * We'll unset this flag in a later patch.
> - */
> - o->use_buffer = 1;
> +
> + if (o->color_moved)
> + o->use_buffer = 1;
>
>  for (i = 0; i < q->nr; i++) {
>  struct diff_filepair *p = q->queue[i];
> @@ -4874,6 +5175,24 @@ static void diff_flush_patch_all_file_pairs(struct 
> diff_options *o)
>  }
>
>  if (o->use_buffer) {
> + if (o->color_moved) {
> + struct hashmap add_lines, del_lines;
> + unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
> +
> + hashmap_init(&del_lines, ignore_ws ?
> + (hashmap_cmp_fn)moved_entry_cmp_no_ws :
> + (hashmap_cmp_fn)moved_entry_cmp, 0);
> + hashmap_init(&add_lines, ignore_ws ?
> + (hashmap_cmp_fn)moved_entry_cmp_no_ws :
> + (hashmap_cmp_fn)moved_entry_cmp, 0);
> +
> + add_lines_to_move_detection(o, &add_lines, &del_lines);
> + mark_color_as_moved(o, &add_lines, &del_lines);
> +
> + hashmap_free(&add_lines, 0);
> + hashmap_free(&del_lines, 0);
> + }
> +
>  for (i = 0; i < o->line_buffer_nr; i++)
>  emit_diff_line(o, &o->line_buffer[i]);
>
> @@ -4962,6 +5281,7 @@ void diff_flush(struct diff_options *options)
>  if (!options->file)
>  die_errno("Could not open /dev/null");
>  options->close_file = 1;
> + options->color_moved = 0;
>  for (i = 0; i < q->nr; i++) {
>  struct diff_filepair *p = q->queue[i];
>  if (check_pair_status(p))
> diff --git a/diff.h b/diff.h
> index be51e8f867..d9fbafd383 100644
> --- a/diff.h
> +++ b/diff.h
> @@ -7,6 +7,7 @@
> #include "tree-walk.h"
> #include "pathspec.h"
> #include "object.h"
> +#include "hashmap.h"
>
> struct rev_info;
> struct diff_options;
> @@ -228,6 +229,14 @@ struct diff_options {
>
>  struct diff_line *line_buffer;
>  int line_buffer_nr, line_buffer_alloc;
> +
> + enum {
> + MOVED_LINES_NO = 0,
> + MOVED_LINES_BOUNDARY_NO = 1,
> + MOVED_LINES_BOUNDARY_ALL = 2,
> + MOVED_LINES_BOUNDARY_ADJACENT = 3,
> + MOVED_LINES_ALTERNATE = 4,
> + } color_moved;
> };
>
> /* Emit [line_prefix] [set] line [reset] */
> @@ -243,7 +252,11 @@ enum color_diff {
>  DIFF_FILE_NEW = 5,
>  DIFF_COMMIT = 6,
>  DIFF_WHITESPACE = 7,
> - DIFF_FUNCINFO = 8
> + DIFF_FUNCINFO = 8,
> + DIFF_FILE_OLD_MOVED = 9,
> + DIFF_FILE_OLD_MOVED_ALT = 10,
> + DIFF_FILE_NEW_MOVED = 11,
> + DIFF_FILE_NEW_MOVED_ALT = 12
> };
> const char *diff_get_color(int diff_use_color, enum color_diff ix);
> #define diff_get_color_opt(o, ix) \
> diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
> index 289806d0c7..d4bd082af7 100755
> --- a/t/t4015-diff-whitespace.sh
> +++ b/t/t4015-diff-whitespace.sh
> @@ -972,4 +972,377 @@ test_expect_success 'option overrides 
> diff.wsErrorHighlight' '
>
> '
>
> +test_expect_success 'detect moved code, complete file' '
> + git reset --hard &&
> + cat <<-\EOF >test.c &&
> + #include<stdio.h>
> + main()
> + {
> + printf("Hello World");
> + }
> + EOF
> + git add test.c &&
> + git commit -m "add main function" &&
> + git mv test.c main.c &&
> + test_config color.diff.oldMoved "normal red" &&
> + test_config color.diff.newMoved "normal green" &&
> + git diff HEAD --color-moved --no-renames | test_decode_color >actual &&
> + cat >expected <<-\EOF &&
> + <BOLD>diff --git a/main.c b/main.c<RESET>
> + <BOLD>new file mode 100644<RESET>
> + <BOLD>index 0000000..a986c57<RESET>
> + <BOLD>--- /dev/null<RESET>
> + <BOLD>+++ b/main.c<RESET>
> + <CYAN>@@ -0,0 +1,5 @@<RESET>
> + <BGREEN>+<RESET><BGREEN>#include<stdio.h><RESET>
> + <BGREEN>+<RESET><BGREEN>main()<RESET>
> + <BGREEN>+<RESET><BGREEN>{<RESET>
> + <BGREEN>+<RESET><BGREEN>printf("Hello World");<RESET>
> + <BGREEN>+<RESET><BGREEN>}<RESET>
> + <BOLD>diff --git a/test.c b/test.c<RESET>
> + <BOLD>deleted file mode 100644<RESET>
> + <BOLD>index a986c57..0000000<RESET>
> + <BOLD>--- a/test.c<RESET>
> + <BOLD>+++ /dev/null<RESET>
> + <CYAN>@@ -1,5 +0,0 @@<RESET>
> + <BRED>-#include<stdio.h><RESET>
> + <BRED>-main()<RESET>
> + <BRED>-{<RESET>
> + <BRED>-printf("Hello World");<RESET>
> + <BRED>-}<RESET>
> + EOF
> +
> + test_cmp expected actual
> +'
> +
> +test_expect_success 'detect moved code, inside file' '
> + git reset --hard &&
> + cat <<-\EOF >main.c &&
> + #include<stdio.h>
> + int stuff()
> + {
> + printf("Hello ");
> + printf("World\n");
> + }
> +
> + int secure_foo(struct user *u)
> + {
> + if (!u->is_allowed_foo)
> + return;
> + foo(u);
> + }
> +
> + int main()
> + {
> + foo();
> + }
> + EOF
> + cat <<-\EOF >test.c &&
> + #include<stdio.h>
> + int bar()
> + {
> + printf("Hello World, but different\n");
> + }
> +
> + int another_function()
> + {
> + bar();
> + }
> + EOF
> + git add main.c test.c &&
> + git commit -m "add main and test file" &&
> + cat <<-\EOF >main.c &&
> + #include<stdio.h>
> + int stuff()
> + {
> + printf("Hello ");
> + printf("World\n");
> + }
> +
> + int main()
> + {
> + foo();
> + }
> + EOF
> + cat <<-\EOF >test.c &&
> + #include<stdio.h>
> + int bar()
> + {
> + printf("Hello World, but different\n");
> + }
> +
> + int secure_foo(struct user *u)
> + {
> + if (!u->is_allowed_foo)
> + return;
> + foo(u);
> + }
> +
> + int another_function()
> + {
> + bar();
> + }
> + EOF
> + test_config color.diff.oldMoved "normal red" &&
> + test_config color.diff.newMoved "normal green" &&
> + test_config color.diff.oldMovedAlternative "bold red" &&
> + test_config color.diff.newMovedAlternative "bold green" &&
> + git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
> + cat <<-\EOF >expected &&
> + <BOLD>diff --git a/main.c b/main.c<RESET>
> + <BOLD>index 27a619c..7cf9336 100644<RESET>
> + <BOLD>--- a/main.c<RESET>
> + <BOLD>+++ b/main.c<RESET>
> + <CYAN>@@ -5,13 +5,6 @@<RESET> <RESET>printf("Hello ");<RESET>
> + printf("World\n");<RESET>
> + }<RESET>
> + <RESET>
> + <BRED>-int secure_foo(struct user *u)<RESET>
> + <BRED>-{<RESET>
> + <BRED>-if (!u->is_allowed_foo)<RESET>
> + <BRED>-return;<RESET>
> + <BRED>-foo(u);<RESET>
> + <BRED>-}<RESET>
> + <BRED>-<RESET>
> + int main()<RESET>
> + {<RESET>
> + foo();<RESET>
> + <BOLD>diff --git a/test.c b/test.c<RESET>
> + <BOLD>index 1dc1d85..e34eb69 100644<RESET>
> + <BOLD>--- a/test.c<RESET>
> + <BOLD>+++ b/test.c<RESET>
> + <CYAN>@@ -4,6 +4,13 @@<RESET> <RESET>int bar()<RESET>
> + printf("Hello World, but different\n");<RESET>
> + }<RESET>
> + <RESET>
> + <BGREEN>+<RESET><BGREEN>int secure_foo(struct user *u)<RESET>
> + <BGREEN>+<RESET><BGREEN>{<RESET>
> + <BGREEN>+<RESET><BGREEN>if (!u->is_allowed_foo)<RESET>
> + <BGREEN>+<RESET><BGREEN>return;<RESET>
> + <BGREEN>+<RESET><BGREEN>foo(u);<RESET>
> + <BGREEN>+<RESET><BGREEN>}<RESET>
> + <BGREEN>+<RESET>
> + int another_function()<RESET>
> + {<RESET>
> + bar();<RESET>
> + EOF
> +
> + test_cmp expected actual
> +'
> +
> +test_expect_success 'detect permutations inside moved code' '
> + git reset --hard &&
> + cat <<-\EOF >lines.txt &&
> + line 1
> + line 2
> + line 3
> + line 4
> + line 5
> + line 6
> + line 7
> + line 8
> + line 9
> + line 10
> + line 11
> + line 12
> + line 13
> + line 14
> + line 15
> + line 16
> + EOF
> + git add lines.txt &&
> + git commit -m "add poetry" &&
> + cat <<-\EOF >lines.txt &&
> + line 4
> + line 5
> + line 6
> + line 7
> + line 8
> + line 9
> + line 1
> + line 2
> + line 3
> + line 14
> + line 15
> + line 16
> + line 10
> + line 11
> + line 12
> + line 13
> + EOF
> + test_config color.diff.oldMoved "magenta" &&
> + test_config color.diff.newMoved "cyan" &&
> + test_config color.diff.oldMovedAlternative "blue" &&
> + test_config color.diff.newMovedAlternative "yellow" &&
> +
> +
> + git diff HEAD --no-renames --color-moved=nobounds| test_decode_color 
>  >actual &&
> + cat <<-\EOF >expected &&
> + <BOLD>diff --git a/lines.txt b/lines.txt<RESET>
> + <BOLD>index 47ea9c3..ba96a38 100644<RESET>
> + <BOLD>--- a/lines.txt<RESET>
> + <BOLD>+++ b/lines.txt<RESET>
> + <CYAN>@@ -1,16 +1,16 @@<RESET>
> + <MAGENTA>-line 1<RESET>
> + <MAGENTA>-line 2<RESET>
> + <MAGENTA>-line 3<RESET>
> + line 4<RESET>
> + line 5<RESET>
> + line 6<RESET>
> + line 7<RESET>
> + line 8<RESET>
> + line 9<RESET>
> + <CYAN>+<RESET><CYAN>line 1<RESET>
> + <CYAN>+<RESET><CYAN>line 2<RESET>
> + <CYAN>+<RESET><CYAN>line 3<RESET>
> + <CYAN>+<RESET><CYAN>line 14<RESET>
> + <CYAN>+<RESET><CYAN>line 15<RESET>
> + <CYAN>+<RESET><CYAN>line 16<RESET>
> + line 10<RESET>
> + line 11<RESET>
> + line 12<RESET>
> + line 13<RESET>
> + <MAGENTA>-line 14<RESET>
> + <MAGENTA>-line 15<RESET>
> + <MAGENTA>-line 16<RESET>
> + EOF
> + test_cmp expected actual &&
> +
> + git diff HEAD --no-renames --color-moved=adjacentbounds| 
> test_decode_color >actual &&
> + cat <<-\EOF >expected &&
> + <BOLD>diff --git a/lines.txt b/lines.txt<RESET>
> + <BOLD>index 47ea9c3..ba96a38 100644<RESET>
> + <BOLD>--- a/lines.txt<RESET>
> + <BOLD>+++ b/lines.txt<RESET>
> + <CYAN>@@ -1,16 +1,16 @@<RESET>
> + <MAGENTA>-line 1<RESET>
> + <MAGENTA>-line 2<RESET>
> + <MAGENTA>-line 3<RESET>
> + line 4<RESET>
> + line 5<RESET>
> + line 6<RESET>
> + line 7<RESET>
> + line 8<RESET>
> + line 9<RESET>
> + <CYAN>+<RESET><CYAN>line 1<RESET>
> + <CYAN>+<RESET><CYAN>line 2<RESET>
> + <YELLOW>+<RESET><YELLOW>line 3<RESET>
> + <YELLOW>+<RESET><YELLOW>line 14<RESET>
> + <CYAN>+<RESET><CYAN>line 15<RESET>
> + <CYAN>+<RESET><CYAN>line 16<RESET>
> + line 10<RESET>
> + line 11<RESET>
> + line 12<RESET>
> + line 13<RESET>
> + <MAGENTA>-line 14<RESET>
> + <MAGENTA>-line 15<RESET>
> + <MAGENTA>-line 16<RESET>
> + EOF
> + test_cmp expected actual &&
> +
> + test_config diff.colorMoved alternate &&
> + git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
> + cat <<-\EOF >expected &&
> + <BOLD>diff --git a/lines.txt b/lines.txt<RESET>
> + <BOLD>index 47ea9c3..ba96a38 100644<RESET>
> + <BOLD>--- a/lines.txt<RESET>
> + <BOLD>+++ b/lines.txt<RESET>
> + <CYAN>@@ -1,16 +1,16 @@<RESET>
> + <MAGENTA>-line 1<RESET>
> + <MAGENTA>-line 2<RESET>
> + <MAGENTA>-line 3<RESET>
> + line 4<RESET>
> + line 5<RESET>
> + line 6<RESET>
> + line 7<RESET>
> + line 8<RESET>
> + line 9<RESET>
> + <CYAN>+<RESET><CYAN>line 1<RESET>
> + <CYAN>+<RESET><CYAN>line 2<RESET>
> + <CYAN>+<RESET><CYAN>line 3<RESET>
> + <YELLOW>+<RESET><YELLOW>line 14<RESET>
> + <YELLOW>+<RESET><YELLOW>line 15<RESET>
> + <YELLOW>+<RESET><YELLOW>line 16<RESET>
> + line 10<RESET>
> + line 11<RESET>
> + line 12<RESET>
> + line 13<RESET>
> + <BLUE>-line 14<RESET>
> + <BLUE>-line 15<RESET>
> + <BLUE>-line 16<RESET>
> + EOF
> + test_cmp expected actual &&
> +
> + test_config diff.colorMoved allbounds &&
> + git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
> + cat <<-\EOF >expected &&
> + <BOLD>diff --git a/lines.txt b/lines.txt<RESET>
> + <BOLD>index 47ea9c3..ba96a38 100644<RESET>
> + <BOLD>--- a/lines.txt<RESET>
> + <BOLD>+++ b/lines.txt<RESET>
> + <CYAN>@@ -1,16 +1,16 @@<RESET>
> + <BLUE>-line 1<RESET>
> + <MAGENTA>-line 2<RESET>
> + <BLUE>-line 3<RESET>
> + line 4<RESET>
> + line 5<RESET>
> + line 6<RESET>
> + line 7<RESET>
> + line 8<RESET>
> + line 9<RESET>
> + <YELLOW>+<RESET><YELLOW>line 1<RESET>
> + <CYAN>+<RESET><CYAN>line 2<RESET>
> + <YELLOW>+<RESET><YELLOW>line 3<RESET>
> + <YELLOW>+<RESET><YELLOW>line 14<RESET>
> + <CYAN>+<RESET><CYAN>line 15<RESET>
> + <YELLOW>+<RESET><YELLOW>line 16<RESET>
> + line 10<RESET>
> + line 11<RESET>
> + line 12<RESET>
> + line 13<RESET>
> + <BLUE>-line 14<RESET>
> + <MAGENTA>-line 15<RESET>
> + <BLUE>-line 16<RESET>
> + EOF
> + test_cmp expected actual
> +'
> +
> +test_expect_success 'move detection does not mess up colored words' '
> + cat <<-\EOF >text.txt &&
> + Lorem Ipsum is simply dummy text of the printing and typesetting 
> industry.
> + EOF
> + git add text.txt &&
> + git commit -a -m "clean state" &&
> + cat <<-\EOF >text.txt &&
> + simply Lorem Ipsum dummy is text of the typesetting and printing 
> industry.
> + EOF
> + git diff --color-moved --word-diff >actual &&
> + git diff --word-diff >expect &&
> + test_cmp expect actual
> +'
> +
> +test_expect_success 'move detection with submodules' '
> + test_create_repo bananas &&
> + echo ripe >bananas/recipe &&
> + git -C bananas add recipe &&
> + test_commit fruit &&
> + test_commit -C bananas recipe &&
> + git submodule add ./bananas &&
> + git add bananas &&
> + git commit -a -m "bananas are like a heavy library?" &&
> + echo foul >bananas/recipe &&
> + echo ripe >fruit.t &&
> +
> + git diff --submodule=diff --color-moved >actual &&
> +
> + # no move detection as the moved line is across repository boundaries.
> + test_decode_color <actual >decoded_actual &&
> + ! grep BGREEN decoded_actual &&
> + ! grep BRED decoded_actual &&
> +
> + # nor did we mess with it another way
> + git diff --submodule=diff | test_decode_color >expect &&
> + test_cmp expect decoded_actual
> +'
> +
> test_done
> -- 
> 2.13.0.17.gab62347cd9
> 


^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 1/1] diff.c: color moved lines differently
  2017-05-27  7:05               ` Philip Oakley
@ 2017-05-30 21:33                 ` Stefan Beller
  2017-06-01  0:24                   ` [PATCH] " Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-05-30 21:33 UTC (permalink / raw)
  To: Philip Oakley
  Cc: Junio C Hamano, git, Brandon Williams, Jonathan Nieder,
	Jeff King, Michael Haggerty, Jonathan Tan

On Sat, May 27, 2017 at 12:05 AM, Philip Oakley <philipoakley@iee.org> wrote:
> a couple of mispellings in the doc parts:
>  s/on location/one location/
> [code not checked]

Thanks for proofreading the documentation!

>
> s/on location/one location/
>
>> + in another location will be colored with 'color.diff.newmoved'.
>> + Any line that is removed in on location and was added
>
> s/on location/one location/

For a moment I was worried that my 'e' key is stuck as it happens
to be the same misspelling here. However then I realized I copied
over part of the sentence and just replaced added/removed.

Stepping back a bit, maybe the second sentence can be partially
elided to be more concise. Maybe:

    Similarly 'color.diff.oldmoved' will be used for removed lines
    that are added somewhere else in the diff.

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 125+ messages in thread

* [PATCH] diff.c: color moved lines differently
  2017-05-30 21:33                 ` Stefan Beller
@ 2017-06-01  0:24                   ` Stefan Beller
  2017-06-13 22:51                     ` Jonathan Tan
  0 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-01  0:24 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

When a patch consists mostly of moving blocks of code around, it can
be quite tedious to ensure that the blocks are moved verbatim, and not
undesirably modified in the move. To that end, color blocks that are
moved within the same patch differently. For example (OM, del, add,
and NM are different colors):

    [OM]  -void sensitive_stuff(void)
    [OM]  -{
    [OM]  -        if (!is_authorized_user())
    [OM]  -                die("unauthorized");
    [OM]  -        sensitive_stuff(spanning,
    [OM]  -                        multiple,
    [OM]  -                        lines);
    [OM]  -}

           void another_function()
           {
    [del] -        printf("foo");
    [add] +        printf("bar");
           }

    [NM]  +void sensitive_stuff(void)
    [NM]  +{
    [NM]  +        if (!is_authorized_user())
    [NM]  +                die("unauthorized");
    [NM]  +        sensitive_stuff(spanning,
    [NM]  +                        multiple,
    [NM]  +                        lines);
    [NM]  +}

However adjacent blocks may be problematic. For example, in this
potentially malicious patch, the swapping of blocks can be spotted:

    [OM]  -void sensitive_stuff(void)
    [OM]  -{
    [OMA] -        if (!is_authorized_user())
    [OMA] -                die("unauthorized");
    [OM]  -        sensitive_stuff(spanning,
    [OM]  -                        multiple,
    [OM]  -                        lines);
    [OMA] -}

           void another_function()
           {
    [del] -        printf("foo");
    [add] +        printf("bar");
           }

    [NM]  +void sensitive_stuff(void)
    [NM]  +{
    [NMA] +        sensitive_stuff(spanning,
    [NMA] +                        multiple,
    [NMA] +                        lines);
    [NM]  +        if (!is_authorized_user())
    [NM]  +                die("unauthorized");
    [NMA] +}

If the moved code is larger, it is easier to hide some permutation in the
code, which is why some alternative coloring is needed.

As the reviewers attention should be brought to the places, where the
difference is introduced to the moved code, we cannot just have one new
color for all of moved code.

First I implemented an alternative design, which would try to fingerprint
a line by its neighbors to detect if we are in a block or at the boundary.
This idea iss error prone as it inspected each line and its neighboring
lines to determine if the line was (a) moved and (b) if was deep inside
a hunk by having matching neighboring lines. This is unreliable as the
we can construct hunks which have equal neighbors that just exceed the
number of lines inspected. (Think of 'AXYZBXYZCXYZD..' with each letter
as a line, that is permutated to AXYZCXYZBXYZD..').

Instead this provides a dynamic programming greedy algorithm that finds
the largest moved hunk and then has several modes on highlighting bounds.

A note on the options '--submodule=diff' and '--color-words/--word-diff':
In the conversion to use emit_line in the prior patches both submodules
as well as word diff output carefully chose to call emit_line with sign=0.
All output with sign=0 is ignored for move detection purposes in this
patch, such that no weird looking output will be generated for these
cases. This leads to another thought: We could pass on '--color-moved' to
submodules such that they color up moved lines for themselves. If we'd do
so only line moves within a repository boundary are marked up.

Helped-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---

 Replacing the top commit in origin/sb/diff-color-move, 
 this has the spelling fixes by Philip.
 
 Also a minor fix for the 'alternate' mode, to go back to the default
 after empty lines. Thanks to Jacob.
 
 Thanks,
 Stefan

 Documentation/config.txt       |  10 +-
 Documentation/diff-options.txt |  32 ++++
 color.h                        |   2 +
 diff.c                         | 343 +++++++++++++++++++++++++++++++++++--
 diff.h                         |  15 +-
 t/t4015-diff-whitespace.sh     | 373 +++++++++++++++++++++++++++++++++++++++++
 6 files changed, 761 insertions(+), 14 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 475e874d51..73511a4603 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1051,14 +1051,20 @@ This does not affect linkgit:git-format-patch[1] or the
 'git-diff-{asterisk}' plumbing commands.  Can be overridden on the
 command line with the `--color[=<when>]` option.
 
+diff.colorMoved::
+	If set moved lines in a diff are colored differently,
+	for details see '--color-moved' in linkgit:git-diff[1].
+
 color.diff.<slot>::
 	Use customized color for diff colorization.  `<slot>` specifies
 	which part of the patch to use the specified color, and is one
 	of `context` (context text - `plain` is a historical synonym),
 	`meta` (metainformation), `frag`
 	(hunk header), 'func' (function in hunk header), `old` (removed lines),
-	`new` (added lines), `commit` (commit headers), or `whitespace`
-	(highlighting whitespace errors).
+	`new` (added lines), `commit` (commit headers), `whitespace`
+	(highlighting whitespace errors), `oldMoved`, `newMoved`,
+	`oldMovedAlternative` and `newMovedAlternative` (See the '<mode>'
+	setting of '--color-moved' in linkgit:git-diff[1] for details).
 
 color.decorate.<slot>::
 	Use customized color for 'git log --decorate' output.  `<slot>` is one
diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt
index 89cc0f48de..69bf061c5c 100644
--- a/Documentation/diff-options.txt
+++ b/Documentation/diff-options.txt
@@ -231,6 +231,38 @@ ifdef::git-diff[]
 endif::git-diff[]
 	It is the same as `--color=never`.
 
+--color-moved[=<mode>]::
+	Moved lines of code are colored differently.
+ifdef::git-diff[]
+	It can be changed by the `diff.colorMoved` configuration setting.
+endif::git-diff[]
+	The <mode> defaults to 'no' if the option is not given
+	and to 'adjacentbounds' if the option with no mode is given.
+	The mode must be one of:
++
+--
+no::
+	Moved lines are not highlighted.
+nobounds::
+	Any line that is added in one location and was removed
+	in another location will be colored with 'color.diff.newmoved'.
+	Similarly 'color.diff.oldmoved' will be used for removed lines
+	that are added somewhere else in the diff.
+allbounds::
+	Based on 'nobounds'. Additionally blocks of moved code are
+	detected and the first and last line of a block will be highlighted
+	using 'color.diff.newMovedAlternate' or
+	'color.diff.oldMovedAlternate'.
+adjacentbounds::
+	The same as 'allbounds' except that highlighting is only performed
+	at adjacent block boundaries of blocks that have the same sign.
+alternate::
+	Based on 'nobounds'. Additionally blocks of moved code are
+	detected. If moved blocks are adjacent mark one of them with the
+	alternative move color using 'color.diff.newMovedAlternate' or
+	'color.diff.oldMovedAlternate'.
+--
+
 --word-diff[=<mode>]::
 	Show a word diff, using the <mode> to delimit changed words.
 	By default, words are delimited by whitespace; see
diff --git a/color.h b/color.h
index 90627650fc..04b3b87929 100644
--- a/color.h
+++ b/color.h
@@ -42,6 +42,8 @@ struct strbuf;
 #define GIT_COLOR_BG_BLUE	"\033[44m"
 #define GIT_COLOR_BG_MAGENTA	"\033[45m"
 #define GIT_COLOR_BG_CYAN	"\033[46m"
+#define GIT_COLOR_DI_IT_CYAN	"\033[2;3;36m"
+#define GIT_COLOR_DI_IT_MAGENTA	"\033[2;3;35m"
 
 /* A special value meaning "no color selected" */
 #define GIT_COLOR_NIL "NIL"
diff --git a/diff.c b/diff.c
index a3c16ef827..a1f919ba57 100644
--- a/diff.c
+++ b/diff.c
@@ -31,6 +31,7 @@ static int diff_indent_heuristic; /* experimental */
 static int diff_rename_limit_default = 400;
 static int diff_suppress_blank_empty;
 static int diff_use_color_default = -1;
+static int diff_color_moved_default;
 static int diff_context_default = 3;
 static int diff_interhunk_context_default;
 static const char *diff_word_regex_cfg;
@@ -55,6 +56,10 @@ static char diff_colors[][COLOR_MAXLEN] = {
 	GIT_COLOR_YELLOW,	/* COMMIT */
 	GIT_COLOR_BG_RED,	/* WHITESPACE */
 	GIT_COLOR_NORMAL,	/* FUNCINFO */
+	GIT_COLOR_DI_IT_MAGENTA,/* OLD_MOVED */
+	GIT_COLOR_BG_RED,	/* OLD_MOVED ALTERNATIVE */
+	GIT_COLOR_DI_IT_CYAN,	/* NEW_MOVED */
+	GIT_COLOR_BG_GREEN,	/* NEW_MOVED ALTERNATIVE */
 };
 
 static NORETURN void die_want_option(const char *option_name)
@@ -80,6 +85,14 @@ static int parse_diff_color_slot(const char *var)
 		return DIFF_WHITESPACE;
 	if (!strcasecmp(var, "func"))
 		return DIFF_FUNCINFO;
+	if (!strcasecmp(var, "oldmoved"))
+		return DIFF_FILE_OLD_MOVED;
+	if (!strcasecmp(var, "oldmovedalternative"))
+		return DIFF_FILE_OLD_MOVED_ALT;
+	if (!strcasecmp(var, "newmoved"))
+		return DIFF_FILE_NEW_MOVED;
+	if (!strcasecmp(var, "newmovedalternative"))
+		return DIFF_FILE_NEW_MOVED_ALT;
 	return -1;
 }
 
@@ -228,12 +241,35 @@ int git_diff_heuristic_config(const char *var, const char *value, void *cb)
 	return 0;
 }
 
+static int parse_color_moved(const char *arg)
+{
+	if (!strcmp(arg, "no"))
+		return MOVED_LINES_NO;
+	else if (!strcmp(arg, "nobounds"))
+		return MOVED_LINES_BOUNDARY_NO;
+	else if (!strcmp(arg, "allbounds"))
+		return MOVED_LINES_BOUNDARY_ALL;
+	else if (!strcmp(arg, "adjacentbounds"))
+		return MOVED_LINES_BOUNDARY_ADJACENT;
+	else if (!strcmp(arg, "alternate"))
+		return MOVED_LINES_ALTERNATE;
+	else
+		return -1;
+}
+
 int git_diff_ui_config(const char *var, const char *value, void *cb)
 {
 	if (!strcmp(var, "diff.color") || !strcmp(var, "color.diff")) {
 		diff_use_color_default = git_config_colorbool(var, value);
 		return 0;
 	}
+	if (!strcmp(var, "diff.colormoved")) {
+		int cm = parse_color_moved(value);
+		if (cm < 0)
+			return -1;
+		diff_color_moved_default = cm;
+		return 0;
+	}
 	if (!strcmp(var, "diff.context")) {
 		diff_context_default = git_config_int(var, value);
 		if (diff_context_default < 0)
@@ -354,6 +390,88 @@ int git_diff_basic_config(const char *var, const char *value, void *cb)
 	return git_default_config(var, value, cb);
 }
 
+struct moved_entry {
+	struct hashmap_entry ent;
+	const struct diff_line *line;
+	struct moved_entry *next_line;
+};
+
+static void get_ws_cleaned_string(const struct diff_line *l,
+				  struct strbuf *out)
+{
+	int i;
+	for (i = 0; i < l->len; i++) {
+		if (isspace(l->line[i]))
+			continue;
+		strbuf_addch(out, l->line[i]);
+	}
+}
+
+static int diff_line_cmp_no_ws(const struct diff_line *a,
+					 const struct diff_line *b,
+					 const void *keydata)
+{
+	int ret;
+	struct strbuf sba = STRBUF_INIT;
+	struct strbuf sbb = STRBUF_INIT;
+
+	get_ws_cleaned_string(a, &sba);
+	get_ws_cleaned_string(b, &sbb);
+	ret = sba.len != sbb.len || strncmp(sba.buf, sbb.buf, sba.len);
+
+	strbuf_release(&sba);
+	strbuf_release(&sbb);
+	return ret;
+}
+
+static int diff_line_cmp(const struct diff_line *a,
+				   const struct diff_line *b,
+				   const void *keydata)
+{
+	return a->len != b->len || strncmp(a->line, b->line, a->len);
+}
+
+static int moved_entry_cmp(const struct moved_entry *a,
+			   const struct moved_entry *b,
+			   const void *keydata)
+{
+	return diff_line_cmp(a->line, b->line, keydata);
+}
+
+static int moved_entry_cmp_no_ws(const struct moved_entry *a,
+				 const struct moved_entry *b,
+				 const void *keydata)
+{
+	return diff_line_cmp_no_ws(a->line, b->line, keydata);
+}
+
+static unsigned get_line_hash(struct diff_line *line, unsigned ignore_ws)
+{
+	static struct strbuf sb = STRBUF_INIT;
+
+	if (ignore_ws) {
+		strbuf_reset(&sb);
+		get_ws_cleaned_string(line, &sb);
+		return memhash(sb.buf, sb.len);
+	} else {
+		return memhash(line->line, line->len);
+	}
+}
+
+static struct moved_entry *prepare_entry(struct diff_options *o,
+					 int line_no)
+{
+	struct moved_entry *ret = xmalloc(sizeof(*ret));
+	unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
+	struct diff_line *l = &o->line_buffer[line_no];
+
+	ret->ent.hash = get_line_hash(l, ignore_ws);
+	ret->line = l;
+	ret->next_line = NULL;
+
+	return ret;
+}
+
 static char *quote_two(const char *one, const char *two)
 {
 	int need_one = quote_c_style(one, NULL, NULL, 1);
@@ -516,6 +634,180 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,
 	ecbdata->blank_at_eof_in_postimage = (at - l2) + 1;
 }
 
+static void add_lines_to_move_detection(struct diff_options *o,
+					struct hashmap *add_lines,
+					struct hashmap *del_lines)
+{
+	struct moved_entry *prev_line = NULL;
+
+	int n;
+	for (n = 0; n < o->line_buffer_nr; n++) {
+		int sign = 0;
+		struct hashmap *hm;
+		struct moved_entry *key;
+
+		switch (o->line_buffer[n].sign) {
+		case '+':
+			sign = '+';
+			hm = add_lines;
+			break;
+		case '-':
+			sign = '-';
+			hm = del_lines;
+			break;
+		case ' ':
+		default:
+			prev_line = NULL;
+			continue;
+		}
+
+		key = prepare_entry(o, n);
+		if (prev_line &&
+		    prev_line->line->sign == sign)
+			prev_line->next_line = key;
+
+		hashmap_add(hm, key);
+		prev_line = key;
+	}
+}
+
+static void mark_color_as_moved_single_line(struct diff_options *o,
+					    struct diff_line *l, int alt_color)
+{
+	switch (l->sign) {
+	case '+':
+		l->set = diff_get_color_opt(o,
+			DIFF_FILE_NEW_MOVED + alt_color);
+		break;
+	case '-':
+		l->set = diff_get_color_opt(o,
+			DIFF_FILE_OLD_MOVED + alt_color);
+		break;
+	default:
+		die("BUG: we should have continued earlier?");
+	}
+}
+
+static void mark_color_as_moved(struct diff_options *o,
+				struct hashmap *add_lines,
+				struct hashmap *del_lines)
+{
+	struct moved_entry **pmb = NULL; /* potentially moved blocks */
+	struct diff_line *prev_line = NULL;
+	int pmb_nr = 0, pmb_alloc = 0;
+	int n, flipped_block = 0;
+
+	for (n = 0; n < o->line_buffer_nr; n++) {
+		struct hashmap *hm = NULL;
+		struct moved_entry *key;
+		struct moved_entry *match = NULL;
+		struct diff_line *l = &o->line_buffer[n];
+		int i, lp, rp, adjacent_blocks = 0;
+
+		/* Check for any match to color it as a move. */
+		switch (l->sign) {
+		case '+':
+			hm = del_lines;
+			key = prepare_entry(o, n);
+			match = hashmap_get(hm, key, o);
+			free(key);
+			break;
+		case '-':
+			hm = add_lines;
+			key = prepare_entry(o, n);
+			match = hashmap_get(hm, key, o);
+			free(key);
+			break;
+		default: ;
+			flipped_block = 0;
+		}
+
+		if (!match) {
+			pmb_nr = 0;
+			if (prev_line &&
+			    o->color_moved == MOVED_LINES_BOUNDARY_ALL)
+				mark_color_as_moved_single_line(o, prev_line, 1);
+			prev_line = NULL;
+			continue;
+		}
+
+		if (o->color_moved == MOVED_LINES_BOUNDARY_NO) {
+			mark_color_as_moved_single_line(o, l, 0);
+			continue;
+		}
+
+		/* Check any potential block runs, advance each or nullify */
+		for (i = 0; i < pmb_nr; i++) {
+			struct moved_entry *p = pmb[i];
+			struct moved_entry *pnext = (p && p->next_line) ?
+					p->next_line : NULL;
+			if (pnext &&
+			    !diff_line_cmp(pnext->line, l, o)) {
+				pmb[i] = p->next_line;
+			} else {
+				pmb[i] = NULL;
+			}
+		}
+
+		/* Shrink the set of potential block to the remaining running */
+		for (lp = 0, rp = pmb_nr - 1; lp <= rp;) {
+			while (lp < pmb_nr && pmb[lp])
+				lp++;
+			/* lp points at the first NULL now */
+
+			while (rp > -1 && !pmb[rp])
+				rp--;
+			/* rp points at the last non-NULL */
+
+			if (lp < pmb_nr && rp > -1 && lp < rp) {
+				pmb[lp] = pmb[rp];
+				pmb[rp] = NULL;
+				rp--;
+				lp++;
+			}
+		}
+
+		/* Remember the number of running sets */
+		pmb_nr = rp + 1;
+
+		if (pmb_nr == 0) {
+			/*
+			 * This line is the start of a new block.
+			 * Setup the set of potential blocks.
+			 */
+			for (; match; match = hashmap_get_next(hm, match)) {
+				ALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);
+				pmb[pmb_nr++] = match;
+			}
+
+			if (o->color_moved == MOVED_LINES_BOUNDARY_ALL) {
+				adjacent_blocks = 1;
+			} else {
+				/* Check if two blocks are adjacent */
+				adjacent_blocks = prev_line &&
+						  prev_line->sign == l->sign;
+			}
+		}
+
+		if (o->color_moved == MOVED_LINES_ALTERNATE) {
+			if (adjacent_blocks)
+				flipped_block = (flipped_block + 1) % 2;
+			mark_color_as_moved_single_line(o, l, flipped_block);
+		} else {
+			/* MOVED_LINES_BOUNDARY_{ADJACENT, ALL} */
+			mark_color_as_moved_single_line(o, l, adjacent_blocks);
+			if (adjacent_blocks && prev_line)
+				prev_line->set = l->set;
+		}
+
+		prev_line = l;
+	}
+	if (prev_line && o->color_moved == MOVED_LINES_BOUNDARY_ALL)
+		mark_color_as_moved_single_line(o, prev_line, 1);
+
+	free(pmb);
+}
+
 static void emit_diff_line(struct diff_options *o,
 			   struct diff_line *e)
 {
@@ -3518,6 +3810,8 @@ void diff_setup(struct diff_options *options)
 	options->line_buffer = NULL;
 	options->line_buffer_nr = 0;
 	options->line_buffer_alloc = 0;
+
+	options->color_moved = diff_color_moved_default;
 }
 
 void diff_setup_done(struct diff_options *options)
@@ -3627,6 +3921,9 @@ void diff_setup_done(struct diff_options *options)
 
 	if (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)
 		die(_("--follow requires exactly one pathspec"));
+
+	if (!options->use_color || external_diff())
+		options->color_moved = 0;
 }
 
 static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)
@@ -4051,7 +4348,19 @@ int diff_opt_parse(struct diff_options *options,
 	}
 	else if (!strcmp(arg, "--no-color"))
 		options->use_color = 0;
-	else if (!strcmp(arg, "--color-words")) {
+	else if (!strcmp(arg, "--color-moved"))
+		if (diff_color_moved_default)
+			options->color_moved = diff_color_moved_default;
+		else
+			options->color_moved = MOVED_LINES_BOUNDARY_ADJACENT;
+	else if (!strcmp(arg, "--no-color-moved"))
+		options->color_moved = MOVED_LINES_NO;
+	else if (skip_prefix(arg, "--color-moved=", &arg)) {
+		int cm = parse_color_moved(arg);
+		if (cm < 0)
+			die("bad --color-moved argument: %s", arg);
+		options->color_moved = cm;
+	} else if (!strcmp(arg, "--color-words")) {
 		options->use_color = 1;
 		options->word_diff = DIFF_WORDS_COLOR;
 	}
@@ -4856,16 +5165,9 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 {
 	int i;
 	struct diff_queue_struct *q = &diff_queued_diff;
-	/*
-	 * For testing purposes we want to make sure the diff machinery
-	 * works completely with the buffer. If there is anything emitted
-	 * outside the emit_diff_line, then the order is screwed
-	 * up and the tests will fail.
-	 *
-	 * TODO (later in this series):
-	 * We'll unset this flag in a later patch.
-	 */
-	o->use_buffer = 1;
+
+	if (o->color_moved)
+		o->use_buffer = 1;
 
 	for (i = 0; i < q->nr; i++) {
 		struct diff_filepair *p = q->queue[i];
@@ -4874,6 +5176,24 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 	}
 
 	if (o->use_buffer) {
+		if (o->color_moved) {
+			struct hashmap add_lines, del_lines;
+			unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
+
+			hashmap_init(&del_lines, ignore_ws ?
+				(hashmap_cmp_fn)moved_entry_cmp_no_ws :
+				(hashmap_cmp_fn)moved_entry_cmp, 0);
+			hashmap_init(&add_lines, ignore_ws ?
+				(hashmap_cmp_fn)moved_entry_cmp_no_ws :
+				(hashmap_cmp_fn)moved_entry_cmp, 0);
+
+			add_lines_to_move_detection(o, &add_lines, &del_lines);
+			mark_color_as_moved(o, &add_lines, &del_lines);
+
+			hashmap_free(&add_lines, 0);
+			hashmap_free(&del_lines, 0);
+		}
+
 		for (i = 0; i < o->line_buffer_nr; i++)
 			emit_diff_line(o, &o->line_buffer[i]);
 
@@ -4962,6 +5282,7 @@ void diff_flush(struct diff_options *options)
 		if (!options->file)
 			die_errno("Could not open /dev/null");
 		options->close_file = 1;
+		options->color_moved = 0;
 		for (i = 0; i < q->nr; i++) {
 			struct diff_filepair *p = q->queue[i];
 			if (check_pair_status(p))
diff --git a/diff.h b/diff.h
index be51e8f867..d9fbafd383 100644
--- a/diff.h
+++ b/diff.h
@@ -7,6 +7,7 @@
 #include "tree-walk.h"
 #include "pathspec.h"
 #include "object.h"
+#include "hashmap.h"
 
 struct rev_info;
 struct diff_options;
@@ -228,6 +229,14 @@ struct diff_options {
 
 	struct diff_line *line_buffer;
 	int line_buffer_nr, line_buffer_alloc;
+
+	enum {
+		MOVED_LINES_NO = 0,
+		MOVED_LINES_BOUNDARY_NO = 1,
+		MOVED_LINES_BOUNDARY_ALL = 2,
+		MOVED_LINES_BOUNDARY_ADJACENT = 3,
+		MOVED_LINES_ALTERNATE = 4,
+	} color_moved;
 };
 
 /* Emit [line_prefix] [set] line [reset] */
@@ -243,7 +252,11 @@ enum color_diff {
 	DIFF_FILE_NEW = 5,
 	DIFF_COMMIT = 6,
 	DIFF_WHITESPACE = 7,
-	DIFF_FUNCINFO = 8
+	DIFF_FUNCINFO = 8,
+	DIFF_FILE_OLD_MOVED = 9,
+	DIFF_FILE_OLD_MOVED_ALT = 10,
+	DIFF_FILE_NEW_MOVED = 11,
+	DIFF_FILE_NEW_MOVED_ALT = 12
 };
 const char *diff_get_color(int diff_use_color, enum color_diff ix);
 #define diff_get_color_opt(o, ix) \
diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
index 289806d0c7..e7b821be0a 100755
--- a/t/t4015-diff-whitespace.sh
+++ b/t/t4015-diff-whitespace.sh
@@ -972,4 +972,377 @@ test_expect_success 'option overrides diff.wsErrorHighlight' '
 
 '
 
+test_expect_success 'detect moved code, complete file' '
+	git reset --hard &&
+	cat <<-\EOF >test.c &&
+	#include<stdio.h>
+	main()
+	{
+	printf("Hello World");
+	}
+	EOF
+	git add test.c &&
+	git commit -m "add main function" &&
+	git mv test.c main.c &&
+	test_config color.diff.oldMoved "normal red" &&
+	test_config color.diff.newMoved "normal green" &&
+	git diff HEAD --color-moved --no-renames | test_decode_color >actual &&
+	cat >expected <<-\EOF &&
+	<BOLD>diff --git a/main.c b/main.c<RESET>
+	<BOLD>new file mode 100644<RESET>
+	<BOLD>index 0000000..a986c57<RESET>
+	<BOLD>--- /dev/null<RESET>
+	<BOLD>+++ b/main.c<RESET>
+	<CYAN>@@ -0,0 +1,5 @@<RESET>
+	<BGREEN>+<RESET><BGREEN>#include<stdio.h><RESET>
+	<BGREEN>+<RESET><BGREEN>main()<RESET>
+	<BGREEN>+<RESET><BGREEN>{<RESET>
+	<BGREEN>+<RESET><BGREEN>printf("Hello World");<RESET>
+	<BGREEN>+<RESET><BGREEN>}<RESET>
+	<BOLD>diff --git a/test.c b/test.c<RESET>
+	<BOLD>deleted file mode 100644<RESET>
+	<BOLD>index a986c57..0000000<RESET>
+	<BOLD>--- a/test.c<RESET>
+	<BOLD>+++ /dev/null<RESET>
+	<CYAN>@@ -1,5 +0,0 @@<RESET>
+	<BRED>-#include<stdio.h><RESET>
+	<BRED>-main()<RESET>
+	<BRED>-{<RESET>
+	<BRED>-printf("Hello World");<RESET>
+	<BRED>-}<RESET>
+	EOF
+
+	test_cmp expected actual
+'
+
+test_expect_success 'detect moved code, inside file' '
+	git reset --hard &&
+	cat <<-\EOF >main.c &&
+		#include<stdio.h>
+		int stuff()
+		{
+			printf("Hello ");
+			printf("World\n");
+		}
+
+		int secure_foo(struct user *u)
+		{
+			if (!u->is_allowed_foo)
+				return;
+			foo(u);
+		}
+
+		int main()
+		{
+			foo();
+		}
+	EOF
+	cat <<-\EOF >test.c &&
+		#include<stdio.h>
+		int bar()
+		{
+			printf("Hello World, but different\n");
+		}
+
+		int another_function()
+		{
+			bar();
+		}
+	EOF
+	git add main.c test.c &&
+	git commit -m "add main and test file" &&
+	cat <<-\EOF >main.c &&
+		#include<stdio.h>
+		int stuff()
+		{
+			printf("Hello ");
+			printf("World\n");
+		}
+
+		int main()
+		{
+			foo();
+		}
+	EOF
+	cat <<-\EOF >test.c &&
+		#include<stdio.h>
+		int bar()
+		{
+			printf("Hello World, but different\n");
+		}
+
+		int secure_foo(struct user *u)
+		{
+			if (!u->is_allowed_foo)
+				return;
+			foo(u);
+		}
+
+		int another_function()
+		{
+			bar();
+		}
+	EOF
+	test_config color.diff.oldMoved "normal red" &&
+	test_config color.diff.newMoved "normal green" &&
+	test_config color.diff.oldMovedAlternative "bold red" &&
+	test_config color.diff.newMovedAlternative "bold green" &&
+	git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/main.c b/main.c<RESET>
+	<BOLD>index 27a619c..7cf9336 100644<RESET>
+	<BOLD>--- a/main.c<RESET>
+	<BOLD>+++ b/main.c<RESET>
+	<CYAN>@@ -5,13 +5,6 @@<RESET> <RESET>printf("Hello ");<RESET>
+	 printf("World\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BRED>-int secure_foo(struct user *u)<RESET>
+	<BRED>-{<RESET>
+	<BRED>-if (!u->is_allowed_foo)<RESET>
+	<BRED>-return;<RESET>
+	<BRED>-foo(u);<RESET>
+	<BRED>-}<RESET>
+	<BRED>-<RESET>
+	 int main()<RESET>
+	 {<RESET>
+	 foo();<RESET>
+	<BOLD>diff --git a/test.c b/test.c<RESET>
+	<BOLD>index 1dc1d85..e34eb69 100644<RESET>
+	<BOLD>--- a/test.c<RESET>
+	<BOLD>+++ b/test.c<RESET>
+	<CYAN>@@ -4,6 +4,13 @@<RESET> <RESET>int bar()<RESET>
+	 printf("Hello World, but different\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BGREEN>+<RESET><BGREEN>int secure_foo(struct user *u)<RESET>
+	<BGREEN>+<RESET><BGREEN>{<RESET>
+	<BGREEN>+<RESET><BGREEN>if (!u->is_allowed_foo)<RESET>
+	<BGREEN>+<RESET><BGREEN>return;<RESET>
+	<BGREEN>+<RESET><BGREEN>foo(u);<RESET>
+	<BGREEN>+<RESET><BGREEN>}<RESET>
+	<BGREEN>+<RESET>
+	 int another_function()<RESET>
+	 {<RESET>
+	 bar();<RESET>
+	EOF
+
+	test_cmp expected actual
+'
+
+test_expect_success 'detect permutations inside moved code' '
+	git reset --hard &&
+	cat <<-\EOF >lines.txt &&
+		line 1
+		line 2
+		line 3
+		line 4
+		line 5
+		line 6
+		line 7
+		line 8
+		line 9
+		line 10
+		line 11
+		line 12
+		line 13
+		line 14
+		line 15
+		line 16
+	EOF
+	git add lines.txt &&
+	git commit -m "add poetry" &&
+	cat <<-\EOF >lines.txt &&
+		line 4
+		line 5
+		line 6
+		line 7
+		line 8
+		line 9
+		line 1
+		line 2
+		line 3
+		line 14
+		line 15
+		line 16
+		line 10
+		line 11
+		line 12
+		line 13
+	EOF
+	test_config color.diff.oldMoved "magenta" &&
+	test_config color.diff.newMoved "cyan" &&
+	test_config color.diff.oldMovedAlternative "blue" &&
+	test_config color.diff.newMovedAlternative "yellow" &&
+
+
+	git diff HEAD --no-renames --color-moved=nobounds| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+		<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+		<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+		<BOLD>--- a/lines.txt<RESET>
+		<BOLD>+++ b/lines.txt<RESET>
+		<CYAN>@@ -1,16 +1,16 @@<RESET>
+		<MAGENTA>-line 1<RESET>
+		<MAGENTA>-line 2<RESET>
+		<MAGENTA>-line 3<RESET>
+		 line 4<RESET>
+		 line 5<RESET>
+		 line 6<RESET>
+		 line 7<RESET>
+		 line 8<RESET>
+		 line 9<RESET>
+		<CYAN>+<RESET><CYAN>line 1<RESET>
+		<CYAN>+<RESET><CYAN>line 2<RESET>
+		<CYAN>+<RESET><CYAN>line 3<RESET>
+		<CYAN>+<RESET><CYAN>line 14<RESET>
+		<CYAN>+<RESET><CYAN>line 15<RESET>
+		<CYAN>+<RESET><CYAN>line 16<RESET>
+		 line 10<RESET>
+		 line 11<RESET>
+		 line 12<RESET>
+		 line 13<RESET>
+		<MAGENTA>-line 14<RESET>
+		<MAGENTA>-line 15<RESET>
+		<MAGENTA>-line 16<RESET>
+	EOF
+	test_cmp expected actual &&
+
+	git diff HEAD --no-renames --color-moved=adjacentbounds| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+	<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+	<BOLD>--- a/lines.txt<RESET>
+	<BOLD>+++ b/lines.txt<RESET>
+	<CYAN>@@ -1,16 +1,16 @@<RESET>
+	<MAGENTA>-line 1<RESET>
+	<MAGENTA>-line 2<RESET>
+	<MAGENTA>-line 3<RESET>
+	 line 4<RESET>
+	 line 5<RESET>
+	 line 6<RESET>
+	 line 7<RESET>
+	 line 8<RESET>
+	 line 9<RESET>
+	<CYAN>+<RESET><CYAN>line 1<RESET>
+	<CYAN>+<RESET><CYAN>line 2<RESET>
+	<YELLOW>+<RESET><YELLOW>line 3<RESET>
+	<YELLOW>+<RESET><YELLOW>line 14<RESET>
+	<CYAN>+<RESET><CYAN>line 15<RESET>
+	<CYAN>+<RESET><CYAN>line 16<RESET>
+	 line 10<RESET>
+	 line 11<RESET>
+	 line 12<RESET>
+	 line 13<RESET>
+	<MAGENTA>-line 14<RESET>
+	<MAGENTA>-line 15<RESET>
+	<MAGENTA>-line 16<RESET>
+	EOF
+	test_cmp expected actual &&
+
+	test_config diff.colorMoved alternate &&
+	git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+	<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+	<BOLD>--- a/lines.txt<RESET>
+	<BOLD>+++ b/lines.txt<RESET>
+	<CYAN>@@ -1,16 +1,16 @@<RESET>
+	<MAGENTA>-line 1<RESET>
+	<MAGENTA>-line 2<RESET>
+	<MAGENTA>-line 3<RESET>
+	 line 4<RESET>
+	 line 5<RESET>
+	 line 6<RESET>
+	 line 7<RESET>
+	 line 8<RESET>
+	 line 9<RESET>
+	<CYAN>+<RESET><CYAN>line 1<RESET>
+	<CYAN>+<RESET><CYAN>line 2<RESET>
+	<CYAN>+<RESET><CYAN>line 3<RESET>
+	<YELLOW>+<RESET><YELLOW>line 14<RESET>
+	<YELLOW>+<RESET><YELLOW>line 15<RESET>
+	<YELLOW>+<RESET><YELLOW>line 16<RESET>
+	 line 10<RESET>
+	 line 11<RESET>
+	 line 12<RESET>
+	 line 13<RESET>
+	<MAGENTA>-line 14<RESET>
+	<MAGENTA>-line 15<RESET>
+	<MAGENTA>-line 16<RESET>
+	EOF
+	test_cmp expected actual &&
+
+	test_config diff.colorMoved allbounds &&
+	git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+	<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+	<BOLD>--- a/lines.txt<RESET>
+	<BOLD>+++ b/lines.txt<RESET>
+	<CYAN>@@ -1,16 +1,16 @@<RESET>
+	<BLUE>-line 1<RESET>
+	<MAGENTA>-line 2<RESET>
+	<BLUE>-line 3<RESET>
+	 line 4<RESET>
+	 line 5<RESET>
+	 line 6<RESET>
+	 line 7<RESET>
+	 line 8<RESET>
+	 line 9<RESET>
+	<YELLOW>+<RESET><YELLOW>line 1<RESET>
+	<CYAN>+<RESET><CYAN>line 2<RESET>
+	<YELLOW>+<RESET><YELLOW>line 3<RESET>
+	<YELLOW>+<RESET><YELLOW>line 14<RESET>
+	<CYAN>+<RESET><CYAN>line 15<RESET>
+	<YELLOW>+<RESET><YELLOW>line 16<RESET>
+	 line 10<RESET>
+	 line 11<RESET>
+	 line 12<RESET>
+	 line 13<RESET>
+	<BLUE>-line 14<RESET>
+	<MAGENTA>-line 15<RESET>
+	<BLUE>-line 16<RESET>
+	EOF
+	test_cmp expected actual
+'
+
+test_expect_success 'move detection does not mess up colored words' '
+	cat <<-\EOF >text.txt &&
+	Lorem Ipsum is simply dummy text of the printing and typesetting industry.
+	EOF
+	git add text.txt &&
+	git commit -a -m "clean state" &&
+	cat <<-\EOF >text.txt &&
+	simply Lorem Ipsum dummy is text of the typesetting and printing industry.
+	EOF
+	git diff --color-moved --word-diff >actual &&
+	git diff --word-diff >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'move detection with submodules' '
+	test_create_repo bananas &&
+	echo ripe >bananas/recipe &&
+	git -C bananas add recipe &&
+	test_commit fruit &&
+	test_commit -C bananas recipe &&
+	git submodule add ./bananas &&
+	git add bananas &&
+	git commit -a -m "bananas are like a heavy library?" &&
+	echo foul >bananas/recipe &&
+	echo ripe >fruit.t &&
+
+	git diff --submodule=diff --color-moved >actual &&
+
+	# no move detection as the moved line is across repository boundaries.
+	test_decode_color <actual >decoded_actual &&
+	! grep BGREEN decoded_actual &&
+	! grep BRED decoded_actual &&
+
+	# nor did we mess with it another way
+	git diff --submodule=diff | test_decode_color >expect &&
+	test_cmp expect decoded_actual
+'
+
 test_done
-- 
2.13.0.17.gab62347cd9


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 04/17] diff: introduce more flexible emit function
  2017-05-24 21:40   ` [PATCHv5 04/17] diff: introduce more flexible emit function Stefan Beller
@ 2017-06-13 21:54     ` Jonathan Tan
  2017-06-13 23:41       ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Jonathan Tan @ 2017-06-13 21:54 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git, bmwill, jrnieder, peff, mhagger

On Wed, 24 May 2017 14:40:23 -0700
Stefan Beller <sbeller@google.com> wrote:

> Currently, diff output is written either through the emit_line_0
> function or through the FILE * in struct diff_options directly. To
> make it easier to teach diff to buffer its output (which will be done
> in a subsequent commit), introduce a more flexible emit_line() function.
> In this commit, direct usages of emit_line_0() are replaced with
> emit_line(); subsequent commits will also replace usages of the
> FILE * with emit().

Check the names of the functions in this paragraph.

> diff --git a/diff.c b/diff.c
> index 2f9722b382..3569857818 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -516,36 +516,30 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,
>  	ecbdata->blank_at_eof_in_postimage = (at - l2) + 1;
>  }
>  
> -static void emit_line_0(struct diff_options *o, const char *set, const char *reset,
> -			int first, const char *line, int len)
> +static void emit_line(struct diff_options *o, const char *set, const char *reset,
> +		      int add_line_prefix, int sign, const char *line, int len)

In the future, this function is going to be used even to emit partial
lines - could this be called emit() instead?

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 16/17] diff: buffer all output if asked to
  2017-05-24 21:40   ` [PATCHv5 16/17] diff: buffer all output if asked to Stefan Beller
  2017-05-25  2:26     ` Junio C Hamano
@ 2017-06-13 22:07     ` Jonathan Tan
  2017-06-14  2:52       ` Stefan Beller
  1 sibling, 1 reply; 125+ messages in thread
From: Jonathan Tan @ 2017-06-13 22:07 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git, bmwill, jrnieder, peff, mhagger

On Wed, 24 May 2017 14:40:35 -0700
Stefan Beller <sbeller@google.com> wrote:

> diff --git a/diff.h b/diff.h
> index 85948ed65a..fad1258556 100644
> --- a/diff.h
> +++ b/diff.h
> @@ -115,6 +115,42 @@ enum diff_submodule_format {
>  	DIFF_SUBMODULE_INLINE_DIFF
>  };
>  
> +/*
> + * This struct is used when we need to buffer the output of the diff output.
> + *
> + * NEEDSWORK: Instead of storing a copy of the line, add an offset pointer
> + * into the pre/post image file. This pointer could be a union with the
> + * line pointer. By storing an offset into the file instead of the literal line,
> + * we can decrease the memory footprint for the buffered output. At first we
> + * may want to only have indirection for the content lines, but we could also
> + * enhance the state for emitting prefabricated lines, e.g. the similarity
> + * score line or hunk/file headers would only need to store a number or path
> + * and then the output can be constructed later on depending on state.
> + */
> +struct diff_line {

Probably should be called diff_emission (or just emission), since these
may not be full lines.

Also, can this definition be in the .c file? Callers should use the
diff_emit_line() below, and not need to know how it is implemented
internally.

> +	const char *set;
> +	const char *reset;
> +	const char *line;
> +	int len;
> +	int sign;
> +	int add_line_prefix;
> +	enum {
> +		/*
> +		 * Emits [lineprefix][set][sign][reset] and then calls
> +		 * ws_check_emit which will output "line", marked up
> +		 * according to ws_rule.
> +		 */
> +		DIFF_LINE_WS,
> +
> +		/* Emits [lineprefix][set][sign] line [reset] */
> +		DIFF_LINE_ASIS,
> +
> +		/* Reloads the ws_rule; line contains the file name */
> +		DIFF_LINE_RELOAD_WS_RULE
> +	} state;
> +};
> +#define diff_line_INIT {NULL, NULL, NULL, 0, 0, 0}

Should be DIFF_LINE_INIT (capitalization), and {NULL} is sufficient, I
think.

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH] diff.c: color moved lines differently
  2017-06-01  0:24                   ` [PATCH] " Stefan Beller
@ 2017-06-13 22:51                     ` Jonathan Tan
  2017-06-14 18:55                       ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Jonathan Tan @ 2017-06-13 22:51 UTC (permalink / raw)
  To: Stefan Beller; +Cc: bmwill, git, gitster, jrnieder, mhagger, peff, philipoakley

On Wed, 31 May 2017 17:24:29 -0700
Stefan Beller <sbeller@google.com> wrote:

> When a patch consists mostly of moving blocks of code around, it can
> be quite tedious to ensure that the blocks are moved verbatim, and not
> undesirably modified in the move. To that end, color blocks that are
> moved within the same patch differently. For example (OM, del, add,
> and NM are different colors):

[snip]

Junio asks "are we happy with these changes" [1] and my answer is, in
general, yes - this seems like a very useful feature to have, and I'm OK
with the current design.

I do feel a bit of unease at how the emitted strings are collected
without many guarantees as to their contents (e.g. whether they are full
lines or even whether they originate from the text of a file), but this
is already true for the existing code. The potential danger is that we
are now relying more on the format of these strings, but we don't plan
to do anything other than to color them, so this seems fine.

I would also prefer if there was only one coloring method, to ease
testing, but I can tolerate the current multiplicity of options.

I think there was some discussion at trying to avoid indicating that
small blocks (consisting of few non-whitespace characters) are moved,
but I think that that can be added later.

[1] https://public-inbox.org/git/CAGZ79kY2Z-fJYxczbzheu1hChLkKkdjEcDMwsP-hkN0TjUBotQ@mail.gmail.com/

> --- a/Documentation/diff-options.txt
> +++ b/Documentation/diff-options.txt
> @@ -231,6 +231,38 @@ ifdef::git-diff[]
>  endif::git-diff[]
>  	It is the same as `--color=never`.
>  
> +--color-moved[=<mode>]::
> +	Moved lines of code are colored differently.
> +ifdef::git-diff[]
> +	It can be changed by the `diff.colorMoved` configuration setting.
> +endif::git-diff[]
> +	The <mode> defaults to 'no' if the option is not given
> +	and to 'adjacentbounds' if the option with no mode is given.
> +	The mode must be one of:
> ++
> +--
> +no::
> +	Moved lines are not highlighted.
> +nobounds::
> +	Any line that is added in one location and was removed
> +	in another location will be colored with 'color.diff.newmoved'.
> +	Similarly 'color.diff.oldmoved' will be used for removed lines

Probably best to consistently capitalize newMoved and oldMoved.

> +static unsigned get_line_hash(struct diff_line *line, unsigned ignore_ws)
> +{
> +	static struct strbuf sb = STRBUF_INIT;
> +
> +	if (ignore_ws) {
> +		strbuf_reset(&sb);
> +		get_ws_cleaned_string(line, &sb);
> +		return memhash(sb.buf, sb.len);
> +	} else {
> +		return memhash(line->line, line->len);
> +	}
> +}

Can be written without the "else".

> +test_expect_success 'move detection does not mess up colored words' '

Probably name this test "--color-moved has no effect when used with
--word-diff".

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 04/17] diff: introduce more flexible emit function
  2017-06-13 21:54     ` Jonathan Tan
@ 2017-06-13 23:41       ` Stefan Beller
  2017-06-13 23:46         ` Jonathan Tan
  0 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-13 23:41 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: Junio C Hamano, git, Brandon Williams, Jonathan Nieder,
	Jeff King, Michael Haggerty

On Tue, Jun 13, 2017 at 2:54 PM, Jonathan Tan <jonathantanmy@google.com> wrote:
> On Wed, 24 May 2017 14:40:23 -0700
> Stefan Beller <sbeller@google.com> wrote:
>
>> Currently, diff output is written either through the emit_line_0
>> function or through the FILE * in struct diff_options directly. To
>> make it easier to teach diff to buffer its output (which will be done
>> in a subsequent commit), introduce a more flexible emit_line() function.
>> In this commit, direct usages of emit_line_0() are replaced with
>> emit_line(); subsequent commits will also replace usages of the
>> FILE * with emit().
>
> Check the names of the functions in this paragraph.

Only the very last word needed replacement of /s/emit/emit_line/.

>
>> diff --git a/diff.c b/diff.c
>> index 2f9722b382..3569857818 100644
>> --- a/diff.c
>> +++ b/diff.c
>> @@ -516,36 +516,30 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,
>>       ecbdata->blank_at_eof_in_postimage = (at - l2) + 1;
>>  }
>>
>> -static void emit_line_0(struct diff_options *o, const char *set, const char *reset,
>> -                     int first, const char *line, int len)
>> +static void emit_line(struct diff_options *o, const char *set, const char *reset,
>> +                   int add_line_prefix, int sign, const char *line, int len)
>
> In the future, this function is going to be used even to emit partial
> lines

Yes.

> - could this be called emit() instead?

Despite having good IDEs available some (including me)
very much like working with raw text, and then having a function
named as a common string doesn't help.

After this patch

  $ git grep emit_line |wc -l
  16
  # not all are this function, there is
  emit_line_checked as well. But 16 is not too much.

But if renamed to emit():

  $ git grep emit -- diff.c |wc -l
  60

You could argue I'd just have to grep
for "emit (" instead, but that then I would have
rely on correct whitespacing or use a regex already.
Complexity which I would not like.

So I am not sure if this is helping a reader. (Not the casual
reader, but the one grepping for this function)

Maybe we can settle on a different name though,
such as emit_string which is not a prefix of a dozen
different other functions?

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 04/17] diff: introduce more flexible emit function
  2017-06-13 23:41       ` Stefan Beller
@ 2017-06-13 23:46         ` Jonathan Tan
  0 siblings, 0 replies; 125+ messages in thread
From: Jonathan Tan @ 2017-06-13 23:46 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Junio C Hamano, git, Brandon Williams, Jonathan Nieder,
	Jeff King, Michael Haggerty

On Tue, 13 Jun 2017 16:41:57 -0700
Stefan Beller <sbeller@google.com> wrote:

> On Tue, Jun 13, 2017 at 2:54 PM, Jonathan Tan <jonathantanmy@google.com> wrote:
> > - could this be called emit() instead?
> 
> Despite having good IDEs available some (including me)
> very much like working with raw text, and then having a function
> named as a common string doesn't help.
> 
> After this patch
> 
>   $ git grep emit_line |wc -l
>   16
>   # not all are this function, there is
>   emit_line_checked as well. But 16 is not too much.
> 
> But if renamed to emit():
> 
>   $ git grep emit -- diff.c |wc -l
>   60
> 
> You could argue I'd just have to grep
> for "emit (" instead, but that then I would have
> rely on correct whitespacing or use a regex already.
> Complexity which I would not like.
> 
> So I am not sure if this is helping a reader. (Not the casual
> reader, but the one grepping for this function)
> 
> Maybe we can settle on a different name though,
> such as emit_string which is not a prefix of a dozen
> different other functions?

emit_string sounds good to me.

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv5 16/17] diff: buffer all output if asked to
  2017-06-13 22:07     ` Jonathan Tan
@ 2017-06-14  2:52       ` Stefan Beller
  0 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-14  2:52 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: Junio C Hamano, git, Brandon Williams, Jonathan Nieder,
	Jeff King, Michael Haggerty

On Tue, Jun 13, 2017 at 3:07 PM, Jonathan Tan <jonathantanmy@google.com> wrote:

>> +struct diff_line {
>
> Probably should be called diff_emission (or just emission), since these
> may not be full lines.

I think emitted_string would do as well?

>
> Also, can this definition be in the .c file? Callers should use the
> diff_emit_line() below, and not need to know how it is implemented
> internally.

done

>> +#define diff_line_INIT {NULL, NULL, NULL, 0, 0, 0}
>
> Should be DIFF_LINE_INIT (capitalization), and {NULL} is sufficient, I
> think.

done.

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH] diff.c: color moved lines differently
  2017-06-13 22:51                     ` Jonathan Tan
@ 2017-06-14 18:55                       ` Stefan Beller
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-14 18:55 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: Brandon Williams, git, Junio C Hamano, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

On Tue, Jun 13, 2017 at 3:51 PM, Jonathan Tan <jonathantanmy@google.com> wrote:
> On Wed, 31 May 2017 17:24:29 -0700
> Stefan Beller <sbeller@google.com> wrote:
>
>> When a patch consists mostly of moving blocks of code around, it can
>> be quite tedious to ensure that the blocks are moved verbatim, and not
>> undesirably modified in the move. To that end, color blocks that are
>> moved within the same patch differently. For example (OM, del, add,
>> and NM are different colors):
>
> [snip]
>
> Junio asks "are we happy with these changes" [1] and my answer is, in
> general, yes - this seems like a very useful feature to have, and I'm OK
> with the current design.
>
> I do feel a bit of unease at how the emitted strings are collected
> without many guarantees as to their contents (e.g. whether they are full
> lines or even whether they originate from the text of a file), but this
> is already true for the existing code. The potential danger is that we
> are now relying more on the format of these strings, but we don't plan
> to do anything other than to color them, so this seems fine.

I will add comments into the code for that.

>
> I would also prefer if there was only one coloring method, to ease
> testing, but I can tolerate the current multiplicity of options.

I *think* by now everyone involved in the discussion agrees that we
want Zebra + optional aggressive dimming (inside blocks as well as
at bounds that are not adjacent to other blocks, i.e. anything
non-adjacent to a different block)

^ permalink raw reply	[flat|nested] 125+ messages in thread

* [PATCH 00/26] reroll of sb/diff-color-moved
  2017-06-14 18:55                       ` Stefan Beller
@ 2017-06-20  2:47                         ` Stefan Beller
  2017-06-20  2:47                           ` [PATCH 01/26] diff.c: readability fix Stefan Beller
                                             ` (26 more replies)
  0 siblings, 27 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:47 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

This is a complete rewrite of the series. Highlights:
* instead of buffering partial lines, we'll pretend all diff output
  follows a well defined grammar, and we emit symbols thereof.
  (The difference is mostly mental, though by this trick we reduce
  the memory footprint for storing one of these symbols from 7 variables
  (3 pointers, 3 ints, one state (also int) down to 4 variables
  (one pointer, 2 ints, one state).
* The algorithm for color painting was detangled:
  -> different functions for block detection and dimming
  -> The last patch (not to be applied) is an RFC that shows
     how we would approach non-colored, but machine parseable highlighting
     of moved lines.

Thanks,
Stefan

Stefan Beller (26):
  diff.c: readability fix
  diff.c: move line ending check into emit_hunk_header
  diff.c: factor out diff_flush_patch_all_file_pairs
  diff.c: introduce emit_diff_symbol
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF
  diff.c: migrate emit_line_checked to use emit_diff_symbol
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS{_PORCELAIN}
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER
  diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF
  submodule.c: migrate diff output to use emit_diff_symbol
  diff.c: convert emit_binary_diff_body to use emit_diff_symbol
  diff.c: convert show_stats to use emit_diff_symbol
  diff.c: convert word diffing to use emit_diff_symbol
  diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP
  diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY
  diff.c: buffer all output if asked to
  diff.c: color moved lines differently
  diff.c: color moved lines differently, plain mode
  diff.c: add dimming to moved line detection
  diff: document the new --color-moved setting
  WIP/RFC: diff.c: have a "machine parseable" move coloring

 Documentation/config.txt       |   12 +-
 Documentation/diff-options.txt |   27 +
 cache.h                        |    1 +
 color.h                        |    2 +
 diff.c                         | 1283 ++++++++++++++++++++++++++++++++--------
 diff.h                         |   39 +-
 submodule.c                    |   85 ++-
 submodule.h                    |   13 +-
 t/t4015-diff-whitespace.sh     |  369 ++++++++++++
 9 files changed, 1515 insertions(+), 316 deletions(-)

-- 
2.12.2.575.gb14f27f917


^ permalink raw reply	[flat|nested] 125+ messages in thread

* [PATCH 01/26] diff.c: readability fix
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
@ 2017-06-20  2:47                           ` Stefan Beller
  2017-06-20  2:47                           ` [PATCH 02/26] diff.c: move line ending check into emit_hunk_header Stefan Beller
                                             ` (25 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:47 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

We already have dereferenced 'p->two' into a local variable 'two'.
Use that.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/diff.c b/diff.c
index 74283d9001..3f5bf8b5a4 100644
--- a/diff.c
+++ b/diff.c
@@ -3283,8 +3283,8 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)
 	const char *other;
 	const char *attr_path;
 
-	name  = p->one->path;
-	other = (strcmp(name, p->two->path) ? p->two->path : NULL);
+	name  = one->path;
+	other = (strcmp(name, two->path) ? two->path : NULL);
 	attr_path = name;
 	if (o->prefix_length)
 		strip_prefix(o->prefix_length, &name, &other);
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 02/26] diff.c: move line ending check into emit_hunk_header
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
  2017-06-20  2:47                           ` [PATCH 01/26] diff.c: readability fix Stefan Beller
@ 2017-06-20  2:47                           ` Stefan Beller
  2017-06-20  2:47                           ` [PATCH 03/26] diff.c: factor out diff_flush_patch_all_file_pairs Stefan Beller
                                             ` (24 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:47 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

The emit_hunk_header() function is responsible for assembling a
hunk header and calling emit_line() to send the hunk header
to the output file.  Its only caller fn_out_consume() needs
to prepare for a case where the function emits an incomplete
line and add the terminating LF.

Instead make sure emit_hunk_header() to always send a
completed line to emit_line().

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/diff.c b/diff.c
index 3f5bf8b5a4..c2ed605cd0 100644
--- a/diff.c
+++ b/diff.c
@@ -677,6 +677,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 	}
 
 	strbuf_add(&msgbuf, line + len, org_len - len);
+	strbuf_complete_line(&msgbuf);
+
 	emit_line(ecbdata->opt, "", "", msgbuf.buf, msgbuf.len);
 	strbuf_release(&msgbuf);
 }
@@ -1315,8 +1317,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 		len = sane_truncate_line(ecbdata, line, len);
 		find_lno(line, ecbdata);
 		emit_hunk_header(ecbdata, line, len);
-		if (line[len-1] != '\n')
-			putc('\n', o->file);
 		return;
 	}
 
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 03/26] diff.c: factor out diff_flush_patch_all_file_pairs
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
  2017-06-20  2:47                           ` [PATCH 01/26] diff.c: readability fix Stefan Beller
  2017-06-20  2:47                           ` [PATCH 02/26] diff.c: move line ending check into emit_hunk_header Stefan Beller
@ 2017-06-20  2:47                           ` Stefan Beller
  2017-06-20  2:47                           ` [PATCH 04/26] diff.c: introduce emit_diff_symbol Stefan Beller
                                             ` (23 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:47 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

In a later patch we want to do more things before and after all filepairs
are flushed. So factor flushing out all file pairs into its own function
that the new code can be plugged in easily.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/diff.c b/diff.c
index c2ed605cd0..2f9722b382 100644
--- a/diff.c
+++ b/diff.c
@@ -4737,6 +4737,17 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)
 		warning(_(rename_limit_advice), varname, needed);
 }
 
+static void diff_flush_patch_all_file_pairs(struct diff_options *o)
+{
+	int i;
+	struct diff_queue_struct *q = &diff_queued_diff;
+	for (i = 0; i < q->nr; i++) {
+		struct diff_filepair *p = q->queue[i];
+		if (check_pair_status(p))
+			diff_flush_patch(p, o);
+	}
+}
+
 void diff_flush(struct diff_options *options)
 {
 	struct diff_queue_struct *q = &diff_queued_diff;
@@ -4831,11 +4842,7 @@ void diff_flush(struct diff_options *options)
 			}
 		}
 
-		for (i = 0; i < q->nr; i++) {
-			struct diff_filepair *p = q->queue[i];
-			if (check_pair_status(p))
-				diff_flush_patch(p, options);
-		}
+		diff_flush_patch_all_file_pairs(options);
 	}
 
 	if (output_format & DIFF_FORMAT_CALLBACK)
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 04/26] diff.c: introduce emit_diff_symbol
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (2 preceding siblings ...)
  2017-06-20  2:47                           ` [PATCH 03/26] diff.c: factor out diff_flush_patch_all_file_pairs Stefan Beller
@ 2017-06-20  2:47                           ` Stefan Beller
  2017-06-21 19:36                             ` Junio C Hamano
  2017-06-20  2:47                           ` [PATCH 05/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER Stefan Beller
                                             ` (22 subsequent siblings)
  26 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:47 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/diff.c b/diff.c
index 2f9722b382..89466018e5 100644
--- a/diff.c
+++ b/diff.c
@@ -559,6 +559,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 	emit_line_0(o, set, reset, line[0], line+1, len-1);
 }
 
+enum diff_symbol {
+	DIFF_SYMBOL_SEPARATOR,
+};
+
+static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
+			     const char *line, int len)
+{
+	switch (s) {
+	case DIFF_SYMBOL_SEPARATOR:
+		fprintf(o->file, "%s%c",
+			diff_line_prefix(o),
+			o->line_termination);
+		break;
+	default:
+		die("BUG: unknown diff symbol");
+	}
+}
+
 static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)
 {
 	if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
@@ -4833,9 +4851,7 @@ void diff_flush(struct diff_options *options)
 
 	if (output_format & DIFF_FORMAT_PATCH) {
 		if (separator) {
-			fprintf(options->file, "%s%c",
-				diff_line_prefix(options),
-				options->line_termination);
+			emit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);
 			if (options->stat_sep) {
 				/* attach patch instead of inline */
 				fputs(options->stat_sep, options->file);
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 05/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (3 preceding siblings ...)
  2017-06-20  2:47                           ` [PATCH 04/26] diff.c: introduce emit_diff_symbol Stefan Beller
@ 2017-06-20  2:47                           ` Stefan Beller
  2017-06-20  2:47                           ` [PATCH 06/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO Stefan Beller
                                             ` (21 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:47 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/diff.c b/diff.c
index 89466018e5..3af07fa659 100644
--- a/diff.c
+++ b/diff.c
@@ -561,17 +561,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 
 enum diff_symbol {
 	DIFF_SYMBOL_SEPARATOR,
+	DIFF_SYMBOL_CONTEXT_MARKER,
 };
 
 static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 			     const char *line, int len)
 {
+	const char *context, *reset;
 	switch (s) {
 	case DIFF_SYMBOL_SEPARATOR:
 		fprintf(o->file, "%s%c",
 			diff_line_prefix(o),
 			o->line_termination);
 		break;
+	case DIFF_SYMBOL_CONTEXT_MARKER:
+		context = diff_get_color_opt(o, DIFF_CONTEXT);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line(o, context, reset, line, len);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -661,7 +668,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 	if (len < 10 ||
 	    memcmp(line, atat, 2) ||
 	    !(ep = memmem(line + 2, len - 2, atat, 2))) {
-		emit_line(ecbdata->opt, context, reset, line, len);
+		emit_diff_symbol(ecbdata->opt,
+				 DIFF_SYMBOL_CONTEXT_MARKER, line, len);
 		return;
 	}
 	ep += 2; /* skip over @@ */
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 06/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (4 preceding siblings ...)
  2017-06-20  2:47                           ` [PATCH 05/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER Stefan Beller
@ 2017-06-20  2:47                           ` Stefan Beller
  2017-06-20  2:47                           ` [PATCH 07/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF Stefan Beller
                                             ` (20 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:47 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/diff.c b/diff.c
index 3af07fa659..a1804a7785 100644
--- a/diff.c
+++ b/diff.c
@@ -562,6 +562,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 enum diff_symbol {
 	DIFF_SYMBOL_SEPARATOR,
 	DIFF_SYMBOL_CONTEXT_MARKER,
+	DIFF_SYMBOL_CONTEXT_FRAGINFO,
 };
 
 static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
@@ -579,6 +580,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 		reset = diff_get_color_opt(o, DIFF_RESET);
 		emit_line(o, context, reset, line, len);
 		break;
+	case DIFF_SYMBOL_CONTEXT_FRAGINFO:
+		emit_line(o, "", "", line, len);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -704,8 +708,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 
 	strbuf_add(&msgbuf, line + len, org_len - len);
 	strbuf_complete_line(&msgbuf);
-
-	emit_line(ecbdata->opt, "", "", msgbuf.buf, msgbuf.len);
+	emit_diff_symbol(ecbdata->opt,
+			 DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len);
 	strbuf_release(&msgbuf);
 }
 
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 07/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (5 preceding siblings ...)
  2017-06-20  2:47                           ` [PATCH 06/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO Stefan Beller
@ 2017-06-20  2:47                           ` Stefan Beller
  2017-06-20  2:47                           ` [PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol Stefan Beller
                                             ` (19 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:47 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/diff.c b/diff.c
index a1804a7785..28be3226c2 100644
--- a/diff.c
+++ b/diff.c
@@ -563,11 +563,13 @@ enum diff_symbol {
 	DIFF_SYMBOL_SEPARATOR,
 	DIFF_SYMBOL_CONTEXT_MARKER,
 	DIFF_SYMBOL_CONTEXT_FRAGINFO,
+	DIFF_SYMBOL_NO_LF_EOF,
 };
 
 static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 			     const char *line, int len)
 {
+	static const char *nneof = " No newline at end of file\n";
 	const char *context, *reset;
 	switch (s) {
 	case DIFF_SYMBOL_SEPARATOR:
@@ -583,6 +585,13 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_CONTEXT_FRAGINFO:
 		emit_line(o, "", "", line, len);
 		break;
+	case DIFF_SYMBOL_NO_LF_EOF:
+		context = diff_get_color_opt(o, DIFF_CONTEXT);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		putc('\n', o->file);
+		emit_line_0(o, context, reset, '\\',
+			    nneof, strlen(nneof));
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -750,7 +759,6 @@ static void emit_rewrite_lines(struct emit_callback *ecb,
 			       int prefix, const char *data, int size)
 {
 	const char *endp = NULL;
-	static const char *nneof = " No newline at end of file\n";
 	const char *reset = diff_get_color(ecb->color_diff, DIFF_RESET);
 
 	while (0 < size) {
@@ -768,13 +776,8 @@ static void emit_rewrite_lines(struct emit_callback *ecb,
 		size -= len;
 		data += len;
 	}
-	if (!endp) {
-		const char *context = diff_get_color(ecb->color_diff,
-						     DIFF_CONTEXT);
-		putc('\n', ecb->opt->file);
-		emit_line_0(ecb->opt, context, reset, '\\',
-			    nneof, strlen(nneof));
-	}
+	if (!endp)
+		emit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0);
 }
 
 static void emit_rewrite_diff(const char *name_a,
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (6 preceding siblings ...)
  2017-06-20  2:47                           ` [PATCH 07/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF Stefan Beller
@ 2017-06-20  2:47                           ` Stefan Beller
  2017-06-21 20:05                             ` Junio C Hamano
  2017-06-20  2:47                           ` [PATCH 09/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS{_PORCELAIN} Stefan Beller
                                             ` (18 subsequent siblings)
  26 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:47 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Add a new flags field to emit_diff_symbol, that will be used by
context lines for:
* white space rules that are applicable (The first 12 bits)
* how the rules are evaluated (actually this double encodes the sign
  of the line, but the code is easier to keep this way, bits 13,14,15)
* if the line a blank line at EOF (bit 16)


Signed-off-by: Stefan Beller <sbeller@google.com>
---
 cache.h |   1 +
 diff.c  | 112 ++++++++++++++++++++++++++++++++++++++++------------------------
 diff.h  |   6 ++--
 3 files changed, 75 insertions(+), 44 deletions(-)

diff --git a/cache.h b/cache.h
index e1f0e182ad..4d63c44f07 100644
--- a/cache.h
+++ b/cache.h
@@ -2168,6 +2168,7 @@ void shift_tree_by(const struct object_id *, const struct object_id *, struct ob
 #define WS_TRAILING_SPACE      (WS_BLANK_AT_EOL|WS_BLANK_AT_EOF)
 #define WS_DEFAULT_RULE (WS_TRAILING_SPACE|WS_SPACE_BEFORE_TAB|8)
 #define WS_TAB_WIDTH_MASK        077
+#define WS_RULE_MASK           07777
 extern unsigned whitespace_rule_cfg;
 extern unsigned whitespace_rule(const char *);
 extern unsigned parse_whitespace_rule(const char *);
diff --git a/diff.c b/diff.c
index 28be3226c2..42c9f48dc2 100644
--- a/diff.c
+++ b/diff.c
@@ -564,13 +564,50 @@ enum diff_symbol {
 	DIFF_SYMBOL_CONTEXT_MARKER,
 	DIFF_SYMBOL_CONTEXT_FRAGINFO,
 	DIFF_SYMBOL_NO_LF_EOF,
+	DIFF_SYMBOL_CONTEXT,
+	DIFF_SYMBOL_PLUS,
+	DIFF_SYMBOL_MINUS,
 };
+/*
+ * Flags for content lines:
+ * 0..12 are whitespace rules
+ * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT
+ * 16 is marking if the line is blank at EOF
+ */
+#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)
+#define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)
+
+static void emit_line_ws_markup(struct diff_options *o,
+				const char *set, const char *reset,
+				const char *line, int len, char sign,
+				unsigned ws_rule, int blank_at_eof)
+{
+	const char *ws = NULL;
+
+	if (o->ws_error_highlight & ws_rule) {
+		ws = diff_get_color_opt(o, DIFF_WHITESPACE);
+		if (!*ws)
+			ws = NULL;
+	}
+
+	if (!ws)
+		emit_line_0(o, set, reset, sign, line, len);
+	else if (blank_at_eof)
+		/* Blank line at EOF - paint '+' as well */
+		emit_line_0(o, ws, reset, sign, line, len);
+	else {
+		/* Emit just the prefix, then the rest. */
+		emit_line_0(o, set, reset, sign, "", 0);
+		ws_check_emit(line, len, ws_rule,
+			      o->file, set, reset, ws);
+	}
+}
 
 static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
-			     const char *line, int len)
+			     const char *line, int len, unsigned flags)
 {
 	static const char *nneof = " No newline at end of file\n";
-	const char *context, *reset;
+	const char *context, *reset, *set;
 	switch (s) {
 	case DIFF_SYMBOL_SEPARATOR:
 		fprintf(o->file, "%s%c",
@@ -592,6 +629,25 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 		emit_line_0(o, context, reset, '\\',
 			    nneof, strlen(nneof));
 		break;
+	case DIFF_SYMBOL_CONTEXT:
+		set = diff_get_color_opt(o, DIFF_CONTEXT);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line_ws_markup(o, set, reset, line, len, ' ',
+				    flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);
+		break;
+	case DIFF_SYMBOL_PLUS:
+		set = diff_get_color_opt(o, DIFF_FILE_NEW);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line_ws_markup(o, set, reset, line, len, '+',
+				    flags & DIFF_SYMBOL_CONTENT_WS_MASK,
+				    flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);
+		break;
+	case DIFF_SYMBOL_MINUS:
+		set = diff_get_color_opt(o, DIFF_FILE_OLD);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line_ws_markup(o, set, reset, line, len, '-',
+				    flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -608,57 +664,31 @@ static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line
 	return ws_blank_line(line, len, ecbdata->ws_rule);
 }
 
-static void emit_line_checked(const char *reset,
-			      struct emit_callback *ecbdata,
-			      const char *line, int len,
-			      enum color_diff color,
-			      unsigned ws_error_highlight,
-			      char sign)
-{
-	const char *set = diff_get_color(ecbdata->color_diff, color);
-	const char *ws = NULL;
-
-	if (ecbdata->opt->ws_error_highlight & ws_error_highlight) {
-		ws = diff_get_color(ecbdata->color_diff, DIFF_WHITESPACE);
-		if (!*ws)
-			ws = NULL;
-	}
-
-	if (!ws)
-		emit_line_0(ecbdata->opt, set, reset, sign, line, len);
-	else if (sign == '+' && new_blank_line_at_eof(ecbdata, line, len))
-		/* Blank line at EOF - paint '+' as well */
-		emit_line_0(ecbdata->opt, ws, reset, sign, line, len);
-	else {
-		/* Emit just the prefix, then the rest. */
-		emit_line_0(ecbdata->opt, set, reset, sign, "", 0);
-		ws_check_emit(line, len, ecbdata->ws_rule,
-			      ecbdata->opt->file, set, reset, ws);
-	}
-}
-
 static void emit_add_line(const char *reset,
 			  struct emit_callback *ecbdata,
 			  const char *line, int len)
 {
-	emit_line_checked(reset, ecbdata, line, len,
-			  DIFF_FILE_NEW, WSEH_NEW, '+');
+	unsigned flags = WSEH_NEW | ecbdata->ws_rule;
+	if (new_blank_line_at_eof(ecbdata, line, len))
+		flags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;
+
+	emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);
 }
 
 static void emit_del_line(const char *reset,
 			  struct emit_callback *ecbdata,
 			  const char *line, int len)
 {
-	emit_line_checked(reset, ecbdata, line, len,
-			  DIFF_FILE_OLD, WSEH_OLD, '-');
+	unsigned flags = WSEH_OLD | ecbdata->ws_rule;
+	emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_MINUS, line, len, flags);
 }
 
 static void emit_context_line(const char *reset,
 			      struct emit_callback *ecbdata,
 			      const char *line, int len)
 {
-	emit_line_checked(reset, ecbdata, line, len,
-			  DIFF_CONTEXT, WSEH_CONTEXT, ' ');
+	unsigned flags = WSEH_CONTEXT | ecbdata->ws_rule;
+	emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_CONTEXT, line, len, flags);
 }
 
 static void emit_hunk_header(struct emit_callback *ecbdata,
@@ -682,7 +712,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 	    memcmp(line, atat, 2) ||
 	    !(ep = memmem(line + 2, len - 2, atat, 2))) {
 		emit_diff_symbol(ecbdata->opt,
-				 DIFF_SYMBOL_CONTEXT_MARKER, line, len);
+				 DIFF_SYMBOL_CONTEXT_MARKER, line, len, 0);
 		return;
 	}
 	ep += 2; /* skip over @@ */
@@ -718,7 +748,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 	strbuf_add(&msgbuf, line + len, org_len - len);
 	strbuf_complete_line(&msgbuf);
 	emit_diff_symbol(ecbdata->opt,
-			 DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len);
+			 DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len, 0);
 	strbuf_release(&msgbuf);
 }
 
@@ -777,7 +807,7 @@ static void emit_rewrite_lines(struct emit_callback *ecb,
 		data += len;
 	}
 	if (!endp)
-		emit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0);
+		emit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0, 0);
 }
 
 static void emit_rewrite_diff(const char *name_a,
@@ -4866,7 +4896,7 @@ void diff_flush(struct diff_options *options)
 
 	if (output_format & DIFF_FORMAT_PATCH) {
 		if (separator) {
-			emit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);
+			emit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);
 			if (options->stat_sep) {
 				/* attach patch instead of inline */
 				fputs(options->stat_sep, options->file);
diff --git a/diff.h b/diff.h
index 5be1ee77a7..8483ca0991 100644
--- a/diff.h
+++ b/diff.h
@@ -148,9 +148,9 @@ struct diff_options {
 	int abbrev;
 	int ita_invisible_in_index;
 /* white-space error highlighting */
-#define WSEH_NEW 1
-#define WSEH_CONTEXT 2
-#define WSEH_OLD 4
+#define WSEH_NEW (1<<12)
+#define WSEH_CONTEXT (1<<13)
+#define WSEH_OLD (1<<14)
 	unsigned ws_error_highlight;
 	const char *prefix;
 	int prefix_length;
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 09/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS{_PORCELAIN}
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (7 preceding siblings ...)
  2017-06-20  2:47                           ` [PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol Stefan Beller
@ 2017-06-20  2:47                           ` Stefan Beller
  2017-06-20  2:48                           ` [PATCH 10/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE Stefan Beller
                                             ` (17 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:47 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 42 ++++++++++++++++++++++++++----------------
 1 file changed, 26 insertions(+), 16 deletions(-)

diff --git a/diff.c b/diff.c
index 42c9f48dc2..7096457c8a 100644
--- a/diff.c
+++ b/diff.c
@@ -567,6 +567,8 @@ enum diff_symbol {
 	DIFF_SYMBOL_CONTEXT,
 	DIFF_SYMBOL_PLUS,
 	DIFF_SYMBOL_MINUS,
+	DIFF_SYMBOL_WORDS_PORCELAIN,
+	DIFF_SYMBOL_WORDS,
 };
 /*
  * Flags for content lines:
@@ -648,6 +650,26 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 		emit_line_ws_markup(o, set, reset, line, len, '-',
 				    flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
 		break;
+	case DIFF_SYMBOL_WORDS_PORCELAIN:
+		context = diff_get_color_opt(o, DIFF_CONTEXT);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line(o, context, reset, line, len);
+		fputs("~\n", o->file);
+		break;
+	case DIFF_SYMBOL_WORDS:
+		context = diff_get_color_opt(o, DIFF_CONTEXT);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		/*
+		 * Skip the prefix character, if any.  With
+		 * diff_suppress_blank_empty, there may be
+		 * none.
+		 */
+		if (line[0] != '\n') {
+			line++;
+			len--;
+		}
+		emit_line(o, context, reset, line, len);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -1342,7 +1364,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 {
 	struct emit_callback *ecbdata = priv;
 	const char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);
-	const char *context = diff_get_color(ecbdata->color_diff, DIFF_CONTEXT);
 	const char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);
 	struct diff_options *o = ecbdata->opt;
 	const char *line_prefix = diff_line_prefix(o);
@@ -1384,6 +1405,9 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 	}
 
 	if (ecbdata->diff_words) {
+		enum diff_symbol s =
+			ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN ?
+			DIFF_SYMBOL_WORDS_PORCELAIN : DIFF_SYMBOL_WORDS;
 		if (line[0] == '-') {
 			diff_words_append(line, len,
 					  &ecbdata->diff_words->minus);
@@ -1403,21 +1427,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 			return;
 		}
 		diff_words_flush(ecbdata);
-		if (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {
-			emit_line(o, context, reset, line, len);
-			fputs("~\n", o->file);
-		} else {
-			/*
-			 * Skip the prefix character, if any.  With
-			 * diff_suppress_blank_empty, there may be
-			 * none.
-			 */
-			if (line[0] != '\n') {
-			      line++;
-			      len--;
-			}
-			emit_line(o, context, reset, line, len);
-		}
+		emit_diff_symbol(o, s, line, len, 0);
 		return;
 	}
 
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 10/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (8 preceding siblings ...)
  2017-06-20  2:47                           ` [PATCH 09/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS{_PORCELAIN} Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-20  2:48                           ` [PATCH 11/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR Stefan Beller
                                             ` (16 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/diff.c b/diff.c
index 7096457c8a..e6029e8213 100644
--- a/diff.c
+++ b/diff.c
@@ -565,6 +565,7 @@ enum diff_symbol {
 	DIFF_SYMBOL_CONTEXT_FRAGINFO,
 	DIFF_SYMBOL_NO_LF_EOF,
 	DIFF_SYMBOL_CONTEXT,
+	DIFF_SYMBOL_CONTEXT_INCOMPLETE,
 	DIFF_SYMBOL_PLUS,
 	DIFF_SYMBOL_MINUS,
 	DIFF_SYMBOL_WORDS_PORCELAIN,
@@ -637,6 +638,11 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 		emit_line_ws_markup(o, set, reset, line, len, ' ',
 				    flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);
 		break;
+	case DIFF_SYMBOL_CONTEXT_INCOMPLETE:
+		context = diff_get_color_opt(o, DIFF_CONTEXT);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line(o, context, reset, line, len);
+		break;
 	case DIFF_SYMBOL_PLUS:
 		set = diff_get_color_opt(o, DIFF_FILE_NEW);
 		reset = diff_get_color_opt(o, DIFF_RESET);
@@ -1448,8 +1454,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 	default:
 		/* incomplete line at the end */
 		ecbdata->lno_in_preimage++;
-		emit_line(o, diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),
-			  reset, line, len);
+		emit_diff_symbol(o, DIFF_SYMBOL_CONTEXT_INCOMPLETE,
+				 line, len, 0);
 		break;
 	}
 }
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 11/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (9 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 10/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-20 20:01                             ` Jonathan Tan
  2017-06-20  2:48                           ` [PATCH 12/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER Stefan Beller
                                             ` (15 subsequent siblings)
  26 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 42 +++++++++++++++++++++---------------------
 1 file changed, 21 insertions(+), 21 deletions(-)

diff --git a/diff.c b/diff.c
index e6029e8213..7c56150b56 100644
--- a/diff.c
+++ b/diff.c
@@ -570,6 +570,7 @@ enum diff_symbol {
 	DIFF_SYMBOL_MINUS,
 	DIFF_SYMBOL_WORDS_PORCELAIN,
 	DIFF_SYMBOL_WORDS,
+	DIFF_SYMBOL_FILEPAIR,
 };
 /*
  * Flags for content lines:
@@ -610,7 +611,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 			     const char *line, int len, unsigned flags)
 {
 	static const char *nneof = " No newline at end of file\n";
-	const char *context, *reset, *set;
+	const char *context, *reset, *set, *meta;
 	switch (s) {
 	case DIFF_SYMBOL_SEPARATOR:
 		fprintf(o->file, "%s%c",
@@ -676,6 +677,14 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 		}
 		emit_line(o, context, reset, line, len);
 		break;
+	case DIFF_SYMBOL_FILEPAIR:
+		meta = diff_get_color_opt(o, DIFF_METAINFO);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		fprintf(o->file, "%s%s%s%s%s%s\n", diff_line_prefix(o), meta,
+			flags ? "+++ " : "--- ",
+			line, reset,
+			strchr(line, ' ') ? "\t" : "");
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -847,8 +856,6 @@ static void emit_rewrite_diff(const char *name_a,
 			      struct diff_options *o)
 {
 	int lc_a, lc_b;
-	const char *name_a_tab, *name_b_tab;
-	const char *metainfo = diff_get_color(o->use_color, DIFF_METAINFO);
 	const char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);
 	const char *reset = diff_get_color(o->use_color, DIFF_RESET);
 	static struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;
@@ -868,8 +875,6 @@ static void emit_rewrite_diff(const char *name_a,
 
 	name_a += (*name_a == '/');
 	name_b += (*name_b == '/');
-	name_a_tab = strchr(name_a, ' ') ? "\t" : "";
-	name_b_tab = strchr(name_b, ' ') ? "\t" : "";
 
 	strbuf_reset(&a_name);
 	strbuf_reset(&b_name);
@@ -896,11 +901,11 @@ static void emit_rewrite_diff(const char *name_a,
 
 	lc_a = count_lines(data_one, size_one);
 	lc_b = count_lines(data_two, size_two);
-	fprintf(o->file,
-		"%s%s--- %s%s%s\n%s%s+++ %s%s%s\n%s%s@@ -",
-		line_prefix, metainfo, a_name.buf, name_a_tab, reset,
-		line_prefix, metainfo, b_name.buf, name_b_tab, reset,
-		line_prefix, fraginfo);
+
+	emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR, a_name.buf, a_name.len, 0);
+	emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR, b_name.buf, b_name.len, 1);
+
+	fprintf(o->file, "%s%s@@ -", line_prefix, fraginfo);
 	if (!o->irreversible_delete)
 		print_line_count(o->file, lc_a);
 	else
@@ -1369,10 +1374,8 @@ static void find_lno(const char *line, struct emit_callback *ecbdata)
 static void fn_out_consume(void *priv, char *line, unsigned long len)
 {
 	struct emit_callback *ecbdata = priv;
-	const char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);
 	const char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);
 	struct diff_options *o = ecbdata->opt;
-	const char *line_prefix = diff_line_prefix(o);
 
 	o->found_changes = 1;
 
@@ -1383,15 +1386,12 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 	}
 
 	if (ecbdata->label_path[0]) {
-		const char *name_a_tab, *name_b_tab;
-
-		name_a_tab = strchr(ecbdata->label_path[0], ' ') ? "\t" : "";
-		name_b_tab = strchr(ecbdata->label_path[1], ' ') ? "\t" : "";
-
-		fprintf(o->file, "%s%s--- %s%s%s\n",
-			line_prefix, meta, ecbdata->label_path[0], reset, name_a_tab);
-		fprintf(o->file, "%s%s+++ %s%s%s\n",
-			line_prefix, meta, ecbdata->label_path[1], reset, name_b_tab);
+		emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR,
+				 ecbdata->label_path[0],
+				 strlen(ecbdata->label_path[0]), 0);
+		emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR,
+				 ecbdata->label_path[1],
+				 strlen(ecbdata->label_path[1]), 1);
 		ecbdata->label_path[0] = ecbdata->label_path[1] = NULL;
 	}
 
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 12/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (10 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 11/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-20  2:48                           ` [PATCH 13/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES Stefan Beller
                                             ` (14 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)

diff --git a/diff.c b/diff.c
index 7c56150b56..95f11ea86f 100644
--- a/diff.c
+++ b/diff.c
@@ -571,6 +571,7 @@ enum diff_symbol {
 	DIFF_SYMBOL_WORDS_PORCELAIN,
 	DIFF_SYMBOL_WORDS,
 	DIFF_SYMBOL_FILEPAIR,
+	DIFF_SYMBOL_HEADER,
 };
 /*
  * Flags for content lines:
@@ -685,6 +686,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 			line, reset,
 			strchr(line, ' ') ? "\t" : "");
 		break;
+	case DIFF_SYMBOL_HEADER:
+		fprintf(o->file, "%s", line);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -1380,7 +1384,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 	o->found_changes = 1;
 
 	if (ecbdata->header) {
-		fprintf(o->file, "%s", ecbdata->header->buf);
+		emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
+				 ecbdata->header->buf, ecbdata->header->len, 0);
 		strbuf_reset(ecbdata->header);
 		ecbdata->header = NULL;
 	}
@@ -2514,7 +2519,8 @@ static void builtin_diff(const char *name_a,
 		if (complete_rewrite &&
 		    (textconv_one || !diff_filespec_is_binary(one)) &&
 		    (textconv_two || !diff_filespec_is_binary(two))) {
-			fprintf(o->file, "%s", header.buf);
+			emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
+					 header.buf, header.len, 0);
 			strbuf_reset(&header);
 			emit_rewrite_diff(name_a, name_b, one, two,
 						textconv_one, textconv_two, o);
@@ -2524,7 +2530,8 @@ static void builtin_diff(const char *name_a,
 	}
 
 	if (o->irreversible_delete && lbl[1][0] == '/') {
-		fprintf(o->file, "%s", header.buf);
+		emit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf,
+				 header.len, 0);
 		strbuf_reset(&header);
 		goto free_ab_and_return;
 	} else if (!DIFF_OPT_TST(o, TEXT) &&
@@ -2535,10 +2542,13 @@ static void builtin_diff(const char *name_a,
 		    !DIFF_OPT_TST(o, BINARY)) {
 			if (!oidcmp(&one->oid, &two->oid)) {
 				if (must_show_header)
-					fprintf(o->file, "%s", header.buf);
+					emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
+							 header.buf, header.len,
+							 0);
 				goto free_ab_and_return;
 			}
-			fprintf(o->file, "%s", header.buf);
+			emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
+					 header.buf, header.len, 0);
 			fprintf(o->file, "%sBinary files %s and %s differ\n",
 				line_prefix, lbl[0], lbl[1]);
 			goto free_ab_and_return;
@@ -2549,10 +2559,11 @@ static void builtin_diff(const char *name_a,
 		if (mf1.size == mf2.size &&
 		    !memcmp(mf1.ptr, mf2.ptr, mf1.size)) {
 			if (must_show_header)
-				fprintf(o->file, "%s", header.buf);
+				emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
+						 header.buf, header.len, 0);
 			goto free_ab_and_return;
 		}
-		fprintf(o->file, "%s", header.buf);
+		emit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);
 		strbuf_reset(&header);
 		if (DIFF_OPT_TST(o, BINARY))
 			emit_binary_diff(o->file, &mf1, &mf2, line_prefix);
@@ -2570,7 +2581,8 @@ static void builtin_diff(const char *name_a,
 		const struct userdiff_funcname *pe;
 
 		if (must_show_header) {
-			fprintf(o->file, "%s", header.buf);
+			emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
+					 header.buf, header.len, 0);
 			strbuf_reset(&header);
 		}
 
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 13/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (11 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 12/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-21 20:13                             ` Junio C Hamano
  2017-06-20  2:48                           ` [PATCH 14/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF Stefan Beller
                                             ` (13 subsequent siblings)
  26 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 21 ++++++++++++++++-----
 1 file changed, 16 insertions(+), 5 deletions(-)

diff --git a/diff.c b/diff.c
index 95f11ea86f..e56962b844 100644
--- a/diff.c
+++ b/diff.c
@@ -572,6 +572,7 @@ enum diff_symbol {
 	DIFF_SYMBOL_WORDS,
 	DIFF_SYMBOL_FILEPAIR,
 	DIFF_SYMBOL_HEADER,
+	DIFF_SYMBOL_BINARY_FILES,
 };
 /*
  * Flags for content lines:
@@ -689,6 +690,10 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_HEADER:
 		fprintf(o->file, "%s", line);
 		break;
+	case DIFF_SYMBOL_BINARY_FILES:
+		fprintf(o->file, "%sBinary files %s differ\n",
+			diff_line_prefix(o), line);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -2537,6 +2542,7 @@ static void builtin_diff(const char *name_a,
 	} else if (!DIFF_OPT_TST(o, TEXT) &&
 	    ( (!textconv_one && diff_filespec_is_binary(one)) ||
 	      (!textconv_two && diff_filespec_is_binary(two)) )) {
+		struct strbuf sb = STRBUF_INIT;
 		if (!one->data && !two->data &&
 		    S_ISREG(one->mode) && S_ISREG(two->mode) &&
 		    !DIFF_OPT_TST(o, BINARY)) {
@@ -2549,8 +2555,10 @@ static void builtin_diff(const char *name_a,
 			}
 			emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 					 header.buf, header.len, 0);
-			fprintf(o->file, "%sBinary files %s and %s differ\n",
-				line_prefix, lbl[0], lbl[1]);
+			strbuf_addf(&sb, "%s and %s", lbl[0], lbl[1]);
+			emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
+					 sb.buf, sb.len, 0);
+			strbuf_release(&sb);
 			goto free_ab_and_return;
 		}
 		if (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)
@@ -2567,9 +2575,12 @@ static void builtin_diff(const char *name_a,
 		strbuf_reset(&header);
 		if (DIFF_OPT_TST(o, BINARY))
 			emit_binary_diff(o->file, &mf1, &mf2, line_prefix);
-		else
-			fprintf(o->file, "%sBinary files %s and %s differ\n",
-				line_prefix, lbl[0], lbl[1]);
+		else {
+			strbuf_addf(&sb, "%s and %s", lbl[0], lbl[1]);
+			emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
+					 sb.buf, sb.len, 0);
+			strbuf_release(&sb);
+		}
 		o->found_changes = 1;
 	} else {
 		/* Crazy xdl interfaces.. */
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 14/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (12 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 13/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-20  2:48                           ` [PATCH 15/26] submodule.c: migrate diff output to use emit_diff_symbol Stefan Beller
                                             ` (12 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 35 +++++++++++++++++++++--------------
 1 file changed, 21 insertions(+), 14 deletions(-)

diff --git a/diff.c b/diff.c
index e56962b844..96ce53c5cf 100644
--- a/diff.c
+++ b/diff.c
@@ -573,6 +573,7 @@ enum diff_symbol {
 	DIFF_SYMBOL_FILEPAIR,
 	DIFF_SYMBOL_HEADER,
 	DIFF_SYMBOL_BINARY_FILES,
+	DIFF_SYMBOL_REWRITE_DIFF,
 };
 /*
  * Flags for content lines:
@@ -613,7 +614,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 			     const char *line, int len, unsigned flags)
 {
 	static const char *nneof = " No newline at end of file\n";
-	const char *context, *reset, *set, *meta;
+	const char *context, *reset, *set, *meta, *fraginfo;
 	switch (s) {
 	case DIFF_SYMBOL_SEPARATOR:
 		fprintf(o->file, "%s%c",
@@ -694,6 +695,11 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 		fprintf(o->file, "%sBinary files %s differ\n",
 			diff_line_prefix(o), line);
 		break;
+	case DIFF_SYMBOL_REWRITE_DIFF:
+		fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line(o, fraginfo, reset, line, len);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -816,17 +822,17 @@ static void remove_tempfile(void)
 	}
 }
 
-static void print_line_count(FILE *file, int count)
+static void add_line_count(struct strbuf *out, int count)
 {
 	switch (count) {
 	case 0:
-		fprintf(file, "0,0");
+		strbuf_addstr(out, "0,0");
 		break;
 	case 1:
-		fprintf(file, "1");
+		strbuf_addstr(out, "1");
 		break;
 	default:
-		fprintf(file, "1,%d", count);
+		strbuf_addf(out, "1,%d", count);
 		break;
 	}
 }
@@ -865,14 +871,12 @@ static void emit_rewrite_diff(const char *name_a,
 			      struct diff_options *o)
 {
 	int lc_a, lc_b;
-	const char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);
-	const char *reset = diff_get_color(o->use_color, DIFF_RESET);
 	static struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;
 	const char *a_prefix, *b_prefix;
 	char *data_one, *data_two;
 	size_t size_one, size_two;
 	struct emit_callback ecbdata;
-	const char *line_prefix = diff_line_prefix(o);
+	struct strbuf out = STRBUF_INIT;
 
 	if (diff_mnemonic_prefix && DIFF_OPT_TST(o, REVERSE_DIFF)) {
 		a_prefix = o->b_prefix;
@@ -914,14 +918,17 @@ static void emit_rewrite_diff(const char *name_a,
 	emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR, a_name.buf, a_name.len, 0);
 	emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR, b_name.buf, b_name.len, 1);
 
-	fprintf(o->file, "%s%s@@ -", line_prefix, fraginfo);
+	strbuf_addstr(&out, "@@ -");
 	if (!o->irreversible_delete)
-		print_line_count(o->file, lc_a);
+		add_line_count(&out, lc_a);
 	else
-		fprintf(o->file, "?,?");
-	fprintf(o->file, " +");
-	print_line_count(o->file, lc_b);
-	fprintf(o->file, " @@%s\n", reset);
+		strbuf_addstr(&out, "?,?");
+	strbuf_addstr(&out, " +");
+	add_line_count(&out, lc_b);
+	strbuf_addstr(&out, " @@\n");
+	emit_diff_symbol(o, DIFF_SYMBOL_REWRITE_DIFF, out.buf, out.len, 0);
+	strbuf_release(&out);
+
 	if (lc_a && !o->irreversible_delete)
 		emit_rewrite_lines(&ecbdata, '-', data_one, size_one);
 	if (lc_b)
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 15/26] submodule.c: migrate diff output to use emit_diff_symbol
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (13 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 14/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-20 20:09                             ` Jonathan Tan
  2017-06-20  2:48                           ` [PATCH 16/26] diff.c: convert emit_binary_diff_body " Stefan Beller
                                             ` (11 subsequent siblings)
  26 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

As the submodule process is no longer attached to the same stdout as
the superprojects process, we need to pass coloring explicitly.

Remove the colors from the function signatures, as all the coloring
decisions will be made either inside the child process or the final
emit_diff_symbol.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c      | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++---------
 diff.h      |  9 +++++++
 submodule.c | 85 ++++++++++++++++++++++++++--------------------------------
 submodule.h | 13 +++------
 4 files changed, 128 insertions(+), 68 deletions(-)

diff --git a/diff.c b/diff.c
index 96ce53c5cf..42a9020d95 100644
--- a/diff.c
+++ b/diff.c
@@ -574,6 +574,13 @@ enum diff_symbol {
 	DIFF_SYMBOL_HEADER,
 	DIFF_SYMBOL_BINARY_FILES,
 	DIFF_SYMBOL_REWRITE_DIFF,
+	DIFF_SYMBOL_SUBMODULE_ADD,
+	DIFF_SYMBOL_SUBMODULE_DEL,
+	DIFF_SYMBOL_SUBMODULE_UNTRACKED,
+	DIFF_SYMBOL_SUBMODULE_MODIFIED,
+	DIFF_SYMBOL_SUBMODULE_HEADER,
+	DIFF_SYMBOL_SUBMODULE_ERROR,
+	DIFF_SYMBOL_SUBMODULE_PIPETHROUGH,
 };
 /*
  * Flags for content lines:
@@ -700,11 +707,77 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 		reset = diff_get_color_opt(o, DIFF_RESET);
 		emit_line(o, fraginfo, reset, line, len);
 		break;
+	case DIFF_SYMBOL_SUBMODULE_ADD:
+		set = diff_get_color_opt(o, DIFF_FILE_NEW);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line(o, set, reset, line, len);
+		break;
+	case DIFF_SYMBOL_SUBMODULE_DEL:
+		set = diff_get_color_opt(o, DIFF_FILE_OLD);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line(o, set, reset, line, len);
+		break;
+	case DIFF_SYMBOL_SUBMODULE_UNTRACKED:
+		fprintf(o->file, "%sSubmodule %s contains untracked content\n",
+			diff_line_prefix(o), line);
+		break;
+	case DIFF_SYMBOL_SUBMODULE_MODIFIED:
+		fprintf(o->file, "%sSubmodule %s contains modified content\n",
+			diff_line_prefix(o), line);
+		break;
+	case DIFF_SYMBOL_SUBMODULE_HEADER:
+		fprintf(o->file, "%s%s", diff_line_prefix(o), line);
+		break;
+	case DIFF_SYMBOL_SUBMODULE_ERROR:
+		emit_line(o, "", "", line, len);
+		break;
+	case DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:
+		emit_line(o, "", "", line, len);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
 }
 
+void diff_emit_submodule_del(struct diff_options *o, const char *line)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);
+}
+
+void diff_emit_submodule_add(struct diff_options *o, const char *line)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ADD, line, strlen(line), 0);
+}
+
+void diff_emit_submodule_untracked(struct diff_options *o, const char *path)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_UNTRACKED,
+			 path, strlen(path), 0);
+}
+
+void diff_emit_submodule_modified(struct diff_options *o, const char *path)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_MODIFIED,
+			 path, strlen(path), 0);
+}
+
+void diff_emit_submodule_header(struct diff_options *o, const char *header)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_HEADER,
+			 header, strlen(header), 0);
+}
+
+void diff_emit_submodule_error(struct diff_options *o, const char *err)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ERROR, err, strlen(err), 0);
+}
+
+void diff_emit_submodule_pipethrough(struct diff_options *o,
+				     const char *line, int len)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_PIPETHROUGH, line, len, 0);
+}
+
 static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)
 {
 	if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
@@ -2465,24 +2538,16 @@ static void builtin_diff(const char *name_a,
 	if (o->submodule_format == DIFF_SUBMODULE_LOG &&
 	    (!one->mode || S_ISGITLINK(one->mode)) &&
 	    (!two->mode || S_ISGITLINK(two->mode))) {
-		const char *del = diff_get_color_opt(o, DIFF_FILE_OLD);
-		const char *add = diff_get_color_opt(o, DIFF_FILE_NEW);
-		show_submodule_summary(o->file, one->path ? one->path : two->path,
-				line_prefix,
+		show_submodule_summary(o, one->path ? one->path : two->path,
 				&one->oid, &two->oid,
-				two->dirty_submodule,
-				meta, del, add, reset);
+				two->dirty_submodule);
 		return;
 	} else if (o->submodule_format == DIFF_SUBMODULE_INLINE_DIFF &&
 		   (!one->mode || S_ISGITLINK(one->mode)) &&
 		   (!two->mode || S_ISGITLINK(two->mode))) {
-		const char *del = diff_get_color_opt(o, DIFF_FILE_OLD);
-		const char *add = diff_get_color_opt(o, DIFF_FILE_NEW);
-		show_submodule_inline_diff(o->file, one->path ? one->path : two->path,
-				line_prefix,
+		show_submodule_inline_diff(o, one->path ? one->path : two->path,
 				&one->oid, &two->oid,
-				two->dirty_submodule,
-				meta, del, add, reset, o);
+				two->dirty_submodule);
 		return;
 	}
 
diff --git a/diff.h b/diff.h
index 8483ca0991..2ee0ef3908 100644
--- a/diff.h
+++ b/diff.h
@@ -188,6 +188,15 @@ struct diff_options {
 	int diff_path_counter;
 };
 
+void diff_emit_submodule_del(struct diff_options *o, const char *line);
+void diff_emit_submodule_add(struct diff_options *o, const char *line);
+void diff_emit_submodule_untracked(struct diff_options *o, const char *path);
+void diff_emit_submodule_modified(struct diff_options *o, const char *path);
+void diff_emit_submodule_header(struct diff_options *o, const char *header);
+void diff_emit_submodule_error(struct diff_options *o, const char *err);
+void diff_emit_submodule_pipethrough(struct diff_options *o,
+				     const char *line, int len);
+
 enum color_diff {
 	DIFF_RESET = 0,
 	DIFF_CONTEXT = 1,
diff --git a/submodule.c b/submodule.c
index d3299e29c0..71471f3241 100644
--- a/submodule.c
+++ b/submodule.c
@@ -362,9 +362,7 @@ static int prepare_submodule_summary(struct rev_info *rev, const char *path,
 	return prepare_revision_walk(rev);
 }
 
-static void print_submodule_summary(struct rev_info *rev, FILE *f,
-		const char *line_prefix,
-		const char *del, const char *add, const char *reset)
+static void print_submodule_summary(struct rev_info *rev, struct diff_options *o)
 {
 	static const char format[] = "  %m %s";
 	struct strbuf sb = STRBUF_INIT;
@@ -375,18 +373,12 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,
 		ctx.date_mode = rev->date_mode;
 		ctx.output_encoding = get_log_output_encoding();
 		strbuf_setlen(&sb, 0);
-		strbuf_addstr(&sb, line_prefix);
-		if (commit->object.flags & SYMMETRIC_LEFT) {
-			if (del)
-				strbuf_addstr(&sb, del);
-		}
-		else if (add)
-			strbuf_addstr(&sb, add);
 		format_commit_message(commit, format, &sb, &ctx);
-		if (reset)
-			strbuf_addstr(&sb, reset);
 		strbuf_addch(&sb, '\n');
-		fprintf(f, "%s", sb.buf);
+		if (commit->object.flags & SYMMETRIC_LEFT)
+			diff_emit_submodule_del(o, sb.buf);
+		else
+			diff_emit_submodule_add(o, sb.buf);
 	}
 	strbuf_release(&sb);
 }
@@ -413,11 +405,9 @@ void prepare_submodule_repo_env(struct argv_array *out)
  * attempt to lookup both the left and right commits and put them into the
  * left and right pointers.
  */
-static void show_submodule_header(FILE *f, const char *path,
-		const char *line_prefix,
+static void show_submodule_header(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
-		unsigned dirty_submodule, const char *meta,
-		const char *reset,
+		unsigned dirty_submodule,
 		struct commit **left, struct commit **right,
 		struct commit_list **merge_bases)
 {
@@ -426,11 +416,10 @@ static void show_submodule_header(FILE *f, const char *path,
 	int fast_forward = 0, fast_backward = 0;
 
 	if (dirty_submodule & DIRTY_SUBMODULE_UNTRACKED)
-		fprintf(f, "%sSubmodule %s contains untracked content\n",
-			line_prefix, path);
+		diff_emit_submodule_untracked(o, path);
+
 	if (dirty_submodule & DIRTY_SUBMODULE_MODIFIED)
-		fprintf(f, "%sSubmodule %s contains modified content\n",
-			line_prefix, path);
+		diff_emit_submodule_modified(o, path);
 
 	if (is_null_oid(one))
 		message = "(new submodule)";
@@ -472,31 +461,29 @@ static void show_submodule_header(FILE *f, const char *path,
 	}
 
 output_header:
-	strbuf_addf(&sb, "%s%sSubmodule %s ", line_prefix, meta, path);
+	strbuf_addf(&sb, "Submodule %s ", path);
 	strbuf_add_unique_abbrev(&sb, one->hash, DEFAULT_ABBREV);
 	strbuf_addstr(&sb, (fast_backward || fast_forward) ? ".." : "...");
 	strbuf_add_unique_abbrev(&sb, two->hash, DEFAULT_ABBREV);
 	if (message)
-		strbuf_addf(&sb, " %s%s\n", message, reset);
+		strbuf_addf(&sb, " %s\n", message);
 	else
-		strbuf_addf(&sb, "%s:%s\n", fast_backward ? " (rewind)" : "", reset);
-	fwrite(sb.buf, sb.len, 1, f);
+		strbuf_addf(&sb, "%s:\n", fast_backward ? " (rewind)" : "");
+	diff_emit_submodule_header(o, sb.buf);
 
 	strbuf_release(&sb);
 }
 
-void show_submodule_summary(FILE *f, const char *path,
-		const char *line_prefix,
+void show_submodule_summary(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
-		unsigned dirty_submodule, const char *meta,
-		const char *del, const char *add, const char *reset)
+		unsigned dirty_submodule)
 {
 	struct rev_info rev;
 	struct commit *left = NULL, *right = NULL;
 	struct commit_list *merge_bases = NULL;
 
-	show_submodule_header(f, path, line_prefix, one, two, dirty_submodule,
-			      meta, reset, &left, &right, &merge_bases);
+	show_submodule_header(o, path, one, two, dirty_submodule,
+			      &left, &right, &merge_bases);
 
 	/*
 	 * If we don't have both a left and a right pointer, there is no
@@ -508,11 +495,11 @@ void show_submodule_summary(FILE *f, const char *path,
 
 	/* Treat revision walker failure the same as missing commits */
 	if (prepare_submodule_summary(&rev, path, left, right, merge_bases)) {
-		fprintf(f, "%s(revision walker failed)\n", line_prefix);
+		diff_emit_submodule_error(o, "(revision walker failed)\n");
 		goto out;
 	}
 
-	print_submodule_summary(&rev, f, line_prefix, del, add, reset);
+	print_submodule_summary(&rev, o);
 
 out:
 	if (merge_bases)
@@ -521,21 +508,18 @@ void show_submodule_summary(FILE *f, const char *path,
 	clear_commit_marks(right, ~0);
 }
 
-void show_submodule_inline_diff(FILE *f, const char *path,
-		const char *line_prefix,
+void show_submodule_inline_diff(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
-		unsigned dirty_submodule, const char *meta,
-		const char *del, const char *add, const char *reset,
-		const struct diff_options *o)
+		unsigned dirty_submodule)
 {
 	const struct object_id *old = &empty_tree_oid, *new = &empty_tree_oid;
 	struct commit *left = NULL, *right = NULL;
 	struct commit_list *merge_bases = NULL;
-	struct strbuf submodule_dir = STRBUF_INIT;
 	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf sb = STRBUF_INIT;
 
-	show_submodule_header(f, path, line_prefix, one, two, dirty_submodule,
-			      meta, reset, &left, &right, &merge_bases);
+	show_submodule_header(o, path, one, two, dirty_submodule,
+			      &left, &right, &merge_bases);
 
 	/* We need a valid left and right commit to display a difference */
 	if (!(left || is_null_oid(one)) ||
@@ -547,15 +531,16 @@ void show_submodule_inline_diff(FILE *f, const char *path,
 	if (right)
 		new = two;
 
-	fflush(f);
 	cp.git_cmd = 1;
 	cp.dir = path;
-	cp.out = dup(fileno(f));
+	cp.out = -1;
 	cp.no_stdin = 1;
 
 	/* TODO: other options may need to be passed here. */
 	argv_array_push(&cp.args, "diff");
-	argv_array_pushf(&cp.args, "--line-prefix=%s", line_prefix);
+	argv_array_pushf(&cp.args, "--color=%s", want_color(o->use_color) ?
+			 "always" : "never");
+
 	if (DIFF_OPT_TST(o, REVERSE_DIFF)) {
 		argv_array_pushf(&cp.args, "--src-prefix=%s%s/",
 				 o->b_prefix, path);
@@ -578,11 +563,17 @@ void show_submodule_inline_diff(FILE *f, const char *path,
 		argv_array_push(&cp.args, oid_to_hex(new));
 
 	prepare_submodule_repo_env(&cp.env_array);
-	if (run_command(&cp))
-		fprintf(f, "(diff failed)\n");
+	if (start_command(&cp))
+		diff_emit_submodule_error(o, "(diff failed)\n");
+
+	while (strbuf_getwholeline_fd(&sb, cp.out, '\n') != EOF)
+		diff_emit_submodule_pipethrough(o, sb.buf, sb.len);
+
+	if (finish_command(&cp))
+		diff_emit_submodule_error(o, "(diff failed)\n");
 
 done:
-	strbuf_release(&submodule_dir);
+	strbuf_release(&sb);
 	if (merge_bases)
 		free_commit_list(merge_bases);
 	if (left)
diff --git a/submodule.h b/submodule.h
index 1277480add..e7358863e9 100644
--- a/submodule.h
+++ b/submodule.h
@@ -53,17 +53,12 @@ extern int parse_submodule_update_strategy(const char *value,
 		struct submodule_update_strategy *dst);
 extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);
 extern void handle_ignore_submodules_arg(struct diff_options *, const char *);
-extern void show_submodule_summary(FILE *f, const char *path,
-		const char *line_prefix,
+extern void show_submodule_summary(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
-		unsigned dirty_submodule, const char *meta,
-		const char *del, const char *add, const char *reset);
-extern void show_submodule_inline_diff(FILE *f, const char *path,
-		const char *line_prefix,
+		unsigned dirty_submodule);
+extern void show_submodule_inline_diff(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
-		unsigned dirty_submodule, const char *meta,
-		const char *del, const char *add, const char *reset,
-		const struct diff_options *opt);
+		unsigned dirty_submodule);
 extern void set_config_fetch_recurse_submodules(int value);
 extern void set_config_update_recurse_submodules(int value);
 /* Check if we want to update any submodule.*/
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 16/26] diff.c: convert emit_binary_diff_body to use emit_diff_symbol
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (14 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 15/26] submodule.c: migrate diff output to use emit_diff_symbol Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-21 20:16                             ` Junio C Hamano
  2017-06-20  2:48                           ` [PATCH 17/26] diff.c: convert show_stats " Stefan Beller
                                             ` (10 subsequent siblings)
  26 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 49 insertions(+), 17 deletions(-)

diff --git a/diff.c b/diff.c
index 42a9020d95..e6ade5fde0 100644
--- a/diff.c
+++ b/diff.c
@@ -573,6 +573,11 @@ enum diff_symbol {
 	DIFF_SYMBOL_FILEPAIR,
 	DIFF_SYMBOL_HEADER,
 	DIFF_SYMBOL_BINARY_FILES,
+	DIFF_SYMBOL_BINARY_DIFF_HEADER,
+	DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,
+	DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,
+	DIFF_SYMBOL_BINARY_DIFF_BODY,
+	DIFF_SYMBOL_BINARY_DIFF_FOOTER,
 	DIFF_SYMBOL_REWRITE_DIFF,
 	DIFF_SYMBOL_SUBMODULE_ADD,
 	DIFF_SYMBOL_SUBMODULE_DEL,
@@ -581,6 +586,7 @@ enum diff_symbol {
 	DIFF_SYMBOL_SUBMODULE_HEADER,
 	DIFF_SYMBOL_SUBMODULE_ERROR,
 	DIFF_SYMBOL_SUBMODULE_PIPETHROUGH,
+
 };
 /*
  * Flags for content lines:
@@ -702,6 +708,22 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 		fprintf(o->file, "%sBinary files %s differ\n",
 			diff_line_prefix(o), line);
 		break;
+	case DIFF_SYMBOL_BINARY_DIFF_HEADER:
+		fprintf(o->file, "%sGIT binary patch\n", diff_line_prefix(o));
+		break;
+	case DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA:
+		fprintf(o->file, "%sdelta %s\n", diff_line_prefix(o), line);
+		break;
+	case DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL:
+		fprintf(o->file, "%sliteral %s\n", diff_line_prefix(o), line);
+		break;
+	case DIFF_SYMBOL_BINARY_DIFF_BODY:
+		emit_line(o, "", "", line, len);
+		break;
+	case DIFF_SYMBOL_BINARY_DIFF_FOOTER:
+		fputs(diff_line_prefix(o), o->file);
+		fputc('\n', o->file);
+		break;
 	case DIFF_SYMBOL_REWRITE_DIFF:
 		fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);
 		reset = diff_get_color_opt(o, DIFF_RESET);
@@ -2394,8 +2416,8 @@ static unsigned char *deflate_it(char *data,
 	return deflated;
 }
 
-static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
-				  const char *prefix)
+static void emit_binary_diff_body(struct diff_options *o,
+				  mmfile_t *one, mmfile_t *two)
 {
 	void *cp;
 	void *delta;
@@ -2424,13 +2446,18 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
 	}
 
 	if (delta && delta_size < deflate_size) {
-		fprintf(file, "%sdelta %lu\n", prefix, orig_size);
+		char *s = xstrfmt("%lu", orig_size);
+		emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,
+				 s, strlen(s), 0);
+		free(s);
 		free(deflated);
 		data = delta;
 		data_size = delta_size;
-	}
-	else {
-		fprintf(file, "%sliteral %lu\n", prefix, two->size);
+	} else {
+		char *s = xstrfmt("%lu", two->size);
+		emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,
+				 s, strlen(s), 0);
+		free(s);
 		free(delta);
 		data = deflated;
 		data_size = deflate_size;
@@ -2439,8 +2466,9 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
 	/* emit data encoded in base85 */
 	cp = data;
 	while (data_size) {
+		int len;
 		int bytes = (52 < data_size) ? 52 : data_size;
-		char line[70];
+		char line[71];
 		data_size -= bytes;
 		if (bytes <= 26)
 			line[0] = bytes + 'A' - 1;
@@ -2448,20 +2476,24 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
 			line[0] = bytes - 26 + 'a' - 1;
 		encode_85(line + 1, cp, bytes);
 		cp = (char *) cp + bytes;
-		fprintf(file, "%s", prefix);
-		fputs(line, file);
-		fputc('\n', file);
+
+		len = strlen(line);
+		line[len++] = '\n';
+		line[len] = '\0';
+
+		emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_BODY,
+				 line, len, 0);
 	}
-	fprintf(file, "%s\n", prefix);
+	emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_FOOTER, NULL, 0, 0);
 	free(data);
 }
 
-static void emit_binary_diff(FILE *file, mmfile_t *one, mmfile_t *two,
-			     const char *prefix)
+static void emit_binary_diff(struct diff_options *o,
+			     mmfile_t *one, mmfile_t *two)
 {
-	fprintf(file, "%sGIT binary patch\n", prefix);
-	emit_binary_diff_body(file, one, two, prefix);
-	emit_binary_diff_body(file, two, one, prefix);
+	emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER, NULL, 0, 0);
+	emit_binary_diff_body(o, one, two);
+	emit_binary_diff_body(o, two, one);
 }
 
 int diff_filespec_is_binary(struct diff_filespec *one)
@@ -2646,7 +2678,7 @@ static void builtin_diff(const char *name_a,
 		emit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);
 		strbuf_reset(&header);
 		if (DIFF_OPT_TST(o, BINARY))
-			emit_binary_diff(o->file, &mf1, &mf2, line_prefix);
+			emit_binary_diff(o, &mf1, &mf2);
 		else {
 			strbuf_addf(&sb, "%s and %s", lbl[0], lbl[1]);
 			emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 17/26] diff.c: convert show_stats to use emit_diff_symbol
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (15 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 16/26] diff.c: convert emit_binary_diff_body " Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-21 21:39                             ` Brandon Williams
  2017-06-20  2:48                           ` [PATCH 18/26] diff.c: convert word diffing " Stefan Beller
                                             ` (9 subsequent siblings)
  26 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

We call print_stat_summary from builtin/apply, so we still
need the version with a file pointer, so introduce
print_stat_summary_0 that uses emit_string machinery and
keep print_stat_summary with the same arguments around.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 118 +++++++++++++++++++++++++++++++++++++++++------------------------
 diff.h |   4 +--
 2 files changed, 77 insertions(+), 45 deletions(-)

diff --git a/diff.c b/diff.c
index e6ade5fde0..0a5ff2500f 100644
--- a/diff.c
+++ b/diff.c
@@ -586,7 +586,10 @@ enum diff_symbol {
 	DIFF_SYMBOL_SUBMODULE_HEADER,
 	DIFF_SYMBOL_SUBMODULE_ERROR,
 	DIFF_SYMBOL_SUBMODULE_PIPETHROUGH,
-
+	DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,
+	DIFF_SYMBOL_STATS_SUMMARY_ABBREV,
+	DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
+	DIFF_SYMBOL_STATS_LINE,
 };
 /*
  * Flags for content lines:
@@ -628,6 +631,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 {
 	static const char *nneof = " No newline at end of file\n";
 	const char *context, *reset, *set, *meta, *fraginfo;
+	struct strbuf sb = STRBUF_INIT;
 	switch (s) {
 	case DIFF_SYMBOL_SEPARATOR:
 		fprintf(o->file, "%s%c",
@@ -756,9 +760,22 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:
 		emit_line(o, "", "", line, len);
 		break;
+	case DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:
+		fprintf(o->file, " 0 files changed\n");
+		break;
+	case DIFF_SYMBOL_STATS_SUMMARY_ABBREV:
+		emit_line(o, "", "", " ...\n", strlen(" ...\n"));
+		break;
+	case DIFF_SYMBOL_STATS_LINE:
+		emit_line(o, "", "", line, len);
+		break;
+	case DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:
+		emit_line(o, "", "", line, len);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
+	strbuf_release(&sb);
 }
 
 void diff_emit_submodule_del(struct diff_options *o, const char *line)
@@ -1712,20 +1729,14 @@ static int scale_linear(int it, int width, int max_change)
 	return 1 + (it * (width - 1) / max_change);
 }
 
-static void show_name(FILE *file,
-		      const char *prefix, const char *name, int len)
-{
-	fprintf(file, " %s%-*s |", prefix, len, name);
-}
-
-static void show_graph(FILE *file, char ch, int cnt, const char *set, const char *reset)
+static void show_graph(struct strbuf *out, char ch, int cnt,
+		       const char *set, const char *reset)
 {
 	if (cnt <= 0)
 		return;
-	fprintf(file, "%s", set);
-	while (cnt--)
-		putc(ch, file);
-	fprintf(file, "%s", reset);
+	strbuf_addstr(out, set);
+	strbuf_addchars(out, ch, cnt);
+	strbuf_addstr(out, reset);
 }
 
 static void fill_print_name(struct diffstat_file *file)
@@ -1749,14 +1760,16 @@ static void fill_print_name(struct diffstat_file *file)
 	file->print_name = pname;
 }
 
-int print_stat_summary(FILE *fp, int files, int insertions, int deletions)
+static void print_stat_summary_inserts_deletes(struct diff_options *options,
+		int files, int insertions, int deletions)
 {
 	struct strbuf sb = STRBUF_INIT;
-	int ret;
 
 	if (!files) {
 		assert(insertions == 0 && deletions == 0);
-		return fprintf(fp, "%s\n", " 0 files changed");
+		emit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,
+				 NULL, 0, 0);
+		return;
 	}
 
 	strbuf_addf(&sb,
@@ -1783,9 +1796,19 @@ int print_stat_summary(FILE *fp, int files, int insertions, int deletions)
 			    deletions);
 	}
 	strbuf_addch(&sb, '\n');
-	ret = fputs(sb.buf, fp);
+	emit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
+			 sb.buf, sb.len, 0);
 	strbuf_release(&sb);
-	return ret;
+}
+
+void print_stat_summary(FILE *fp, int files,
+			int insertions, int deletions)
+{
+	struct diff_options o;
+	memset(&o, 0, sizeof(o));
+	o.file = fp;
+
+	print_stat_summary_inserts_deletes(&o, files, insertions, deletions);
 }
 
 static void show_stats(struct diffstat_t *data, struct diff_options *options)
@@ -1795,13 +1818,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 	int total_files = data->nr, count;
 	int width, name_width, graph_width, number_width = 0, bin_width = 0;
 	const char *reset, *add_c, *del_c;
-	const char *line_prefix = "";
 	int extra_shown = 0;
+	const char *line_prefix = diff_line_prefix(options);
+	struct strbuf out = STRBUF_INIT;
 
 	if (data->nr == 0)
 		return;
 
-	line_prefix = diff_line_prefix(options);
 	count = options->stat_count ? options->stat_count : data->nr;
 
 	reset = diff_get_color_opt(options, DIFF_RESET);
@@ -1955,26 +1978,32 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 		}
 
 		if (file->is_binary) {
-			fprintf(options->file, "%s", line_prefix);
-			show_name(options->file, prefix, name, len);
-			fprintf(options->file, " %*s", number_width, "Bin");
+			strbuf_addf(&out, " %s%-*s |", prefix, len, name);
+			strbuf_addf(&out, " %*s", number_width, "Bin");
 			if (!added && !deleted) {
-				putc('\n', options->file);
+				strbuf_addch(&out, '\n');
+				emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
+						 out.buf, out.len, 0);
+				strbuf_reset(&out);
 				continue;
 			}
-			fprintf(options->file, " %s%"PRIuMAX"%s",
+			strbuf_addf(&out, " %s%"PRIuMAX"%s",
 				del_c, deleted, reset);
-			fprintf(options->file, " -> ");
-			fprintf(options->file, "%s%"PRIuMAX"%s",
+			strbuf_addstr(&out, " -> ");
+			strbuf_addf(&out, "%s%"PRIuMAX"%s",
 				add_c, added, reset);
-			fprintf(options->file, " bytes");
-			fprintf(options->file, "\n");
+			strbuf_addstr(&out, " bytes\n");
+			emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
+					 out.buf, out.len, 0);
+			strbuf_reset(&out);
 			continue;
 		}
 		else if (file->is_unmerged) {
-			fprintf(options->file, "%s", line_prefix);
-			show_name(options->file, prefix, name, len);
-			fprintf(options->file, " Unmerged\n");
+			strbuf_addf(&out, " %s%-*s |", prefix, len, name);
+			strbuf_addstr(&out, " Unmerged\n");
+			emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
+					 out.buf, out.len, 0);
+			strbuf_reset(&out);
 			continue;
 		}
 
@@ -1997,14 +2026,16 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 				add = total - del;
 			}
 		}
-		fprintf(options->file, "%s", line_prefix);
-		show_name(options->file, prefix, name, len);
-		fprintf(options->file, " %*"PRIuMAX"%s",
+		strbuf_addf(&out, " %s%-*s |", prefix, len, name);
+		strbuf_addf(&out, " %*"PRIuMAX"%s",
 			number_width, added + deleted,
 			added + deleted ? " " : "");
-		show_graph(options->file, '+', add, add_c, reset);
-		show_graph(options->file, '-', del, del_c, reset);
-		fprintf(options->file, "\n");
+		show_graph(&out, '+', add, add_c, reset);
+		show_graph(&out, '-', del, del_c, reset);
+		strbuf_addch(&out, '\n');
+		emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
+				 out.buf, out.len, 0);
+		strbuf_reset(&out);
 	}
 
 	for (i = 0; i < data->nr; i++) {
@@ -2025,11 +2056,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 		if (i < count)
 			continue;
 		if (!extra_shown)
-			fprintf(options->file, "%s ...\n", line_prefix);
+			emit_diff_symbol(options,
+					 DIFF_SYMBOL_STATS_SUMMARY_ABBREV,
+					 NULL, 0, 0);
 		extra_shown = 1;
 	}
-	fprintf(options->file, "%s", line_prefix);
-	print_stat_summary(options->file, total_files, adds, dels);
+
+	print_stat_summary_inserts_deletes(options, total_files, adds, dels);
 }
 
 static void show_shortstats(struct diffstat_t *data, struct diff_options *options)
@@ -2041,7 +2074,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option
 
 	for (i = 0; i < data->nr; i++) {
 		int added = data->files[i]->added;
-		int deleted= data->files[i]->deleted;
+		int deleted = data->files[i]->deleted;
 
 		if (data->files[i]->is_unmerged ||
 		    (!data->files[i]->is_interesting && (added + deleted == 0))) {
@@ -2051,8 +2084,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option
 			dels += deleted;
 		}
 	}
-	fprintf(options->file, "%s", diff_line_prefix(options));
-	print_stat_summary(options->file, total_files, adds, dels);
+	print_stat_summary_inserts_deletes(options, total_files, adds, dels);
 }
 
 static void show_numstat(struct diffstat_t *data, struct diff_options *options)
diff --git a/diff.h b/diff.h
index 2ee0ef3908..b73f7a1e75 100644
--- a/diff.h
+++ b/diff.h
@@ -398,8 +398,8 @@ extern int parse_rename_score(const char **cp_p);
 
 extern long parse_algorithm_value(const char *value);
 
-extern int print_stat_summary(FILE *fp, int files,
-			      int insertions, int deletions);
+extern void print_stat_summary(FILE *fp, int files,
+			       int insertions, int deletions);
 extern void setup_diff_pager(struct diff_options *);
 
 #endif /* DIFF_H */
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 18/26] diff.c: convert word diffing to use emit_diff_symbol
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (16 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 17/26] diff.c: convert show_stats " Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-20  2:48                           ` [PATCH 19/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP Stefan Beller
                                             ` (8 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

The word diffing is not line oriented and would need some serious
effort to be transformed into a line oriented approach, so
just go with a symbol DIFF_SYMBOL_WORD_DIFF that is a partial line.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 79 ++++++++++++++++++++++++++++++++++++++----------------------------
 1 file changed, 46 insertions(+), 33 deletions(-)

diff --git a/diff.c b/diff.c
index 0a5ff2500f..5b46baa12a 100644
--- a/diff.c
+++ b/diff.c
@@ -590,6 +590,7 @@ enum diff_symbol {
 	DIFF_SYMBOL_STATS_SUMMARY_ABBREV,
 	DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
 	DIFF_SYMBOL_STATS_LINE,
+	DIFF_SYMBOL_WORD_DIFF,
 };
 /*
  * Flags for content lines:
@@ -772,6 +773,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:
 		emit_line(o, "", "", line, len);
 		break;
+	case DIFF_SYMBOL_WORD_DIFF:
+		fprintf(o->file, "%.*s", len, line);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -1100,37 +1104,49 @@ struct diff_words_data {
 	struct diff_words_style *style;
 };
 
-static int fn_out_diff_words_write_helper(FILE *fp,
+static int fn_out_diff_words_write_helper(struct diff_options *o,
 					  struct diff_words_style_elem *st_el,
 					  const char *newline,
-					  size_t count, const char *buf,
-					  const char *line_prefix)
+					  size_t count, const char *buf)
 {
 	int print = 0;
+	struct strbuf sb = STRBUF_INIT;
 
 	while (count) {
 		char *p = memchr(buf, '\n', count);
 		if (print)
-			fputs(line_prefix, fp);
+			strbuf_addstr(&sb, diff_line_prefix(o));
+
 		if (p != buf) {
-			if (st_el->color && fputs(st_el->color, fp) < 0)
-				return -1;
-			if (fputs(st_el->prefix, fp) < 0 ||
-			    fwrite(buf, p ? p - buf : count, 1, fp) != 1 ||
-			    fputs(st_el->suffix, fp) < 0)
-				return -1;
-			if (st_el->color && *st_el->color
-			    && fputs(GIT_COLOR_RESET, fp) < 0)
-				return -1;
+			const char *reset = st_el->color && *st_el->color ?
+					    GIT_COLOR_RESET : NULL;
+			if (st_el->color && *st_el->color)
+				strbuf_addstr(&sb, st_el->color);
+			strbuf_addstr(&sb, st_el->prefix);
+			strbuf_add(&sb, buf, p ? p - buf : count);
+			strbuf_addstr(&sb, st_el->suffix);
+			if (reset)
+				strbuf_addstr(&sb, reset);
 		}
 		if (!p)
-			return 0;
-		if (fputs(newline, fp) < 0)
-			return -1;
+			goto out;
+
+		strbuf_addstr(&sb, newline);
 		count -= p + 1 - buf;
 		buf = p + 1;
 		print = 1;
+		if (count) {
+			emit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,
+					 sb.buf, sb.len, 0);
+			strbuf_reset(&sb);
+		}
 	}
+
+out:
+	if (sb.len)
+		emit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,
+				 sb.buf, sb.len, 0);
+	strbuf_release(&sb);
 	return 0;
 }
 
@@ -1212,24 +1228,20 @@ static void fn_out_diff_words_aux(void *priv, char *line, unsigned long len)
 		fputs(line_prefix, diff_words->opt->file);
 	}
 	if (diff_words->current_plus != plus_begin) {
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+		fn_out_diff_words_write_helper(diff_words->opt,
 				&style->ctx, style->newline,
 				plus_begin - diff_words->current_plus,
-				diff_words->current_plus, line_prefix);
-		if (*(plus_begin - 1) == '\n')
-			fputs(line_prefix, diff_words->opt->file);
+				diff_words->current_plus);
 	}
 	if (minus_begin != minus_end) {
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+		fn_out_diff_words_write_helper(diff_words->opt,
 				&style->old, style->newline,
-				minus_end - minus_begin, minus_begin,
-				line_prefix);
+				minus_end - minus_begin, minus_begin);
 	}
 	if (plus_begin != plus_end) {
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+		fn_out_diff_words_write_helper(diff_words->opt,
 				&style->new, style->newline,
-				plus_end - plus_begin, plus_begin,
-				line_prefix);
+				plus_end - plus_begin, plus_begin);
 	}
 
 	diff_words->current_plus = plus_end;
@@ -1323,11 +1335,12 @@ static void diff_words_show(struct diff_words_data *diff_words)
 
 	/* special case: only removal */
 	if (!diff_words->plus.text.size) {
-		fputs(line_prefix, diff_words->opt->file);
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+		emit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,
+				 line_prefix, strlen(line_prefix), 0);
+		fn_out_diff_words_write_helper(diff_words->opt,
 			&style->old, style->newline,
 			diff_words->minus.text.size,
-			diff_words->minus.text.ptr, line_prefix);
+			diff_words->minus.text.ptr);
 		diff_words->minus.text.size = 0;
 		return;
 	}
@@ -1350,12 +1363,12 @@ static void diff_words_show(struct diff_words_data *diff_words)
 	if (diff_words->current_plus != diff_words->plus.text.ptr +
 			diff_words->plus.text.size) {
 		if (color_words_output_graph_prefix(diff_words))
-			fputs(line_prefix, diff_words->opt->file);
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+			emit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,
+					 line_prefix, strlen(line_prefix), 0);
+		fn_out_diff_words_write_helper(diff_words->opt,
 			&style->ctx, style->newline,
 			diff_words->plus.text.ptr + diff_words->plus.text.size
-			- diff_words->current_plus, diff_words->current_plus,
-			line_prefix);
+			- diff_words->current_plus, diff_words->current_plus);
 	}
 	diff_words->minus.text.size = diff_words->plus.text.size = 0;
 }
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 19/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (17 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 18/26] diff.c: convert word diffing " Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-20  2:48                           ` [PATCH 20/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY Stefan Beller
                                             ` (7 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/diff.c b/diff.c
index 5b46baa12a..18bfc4720b 100644
--- a/diff.c
+++ b/diff.c
@@ -591,6 +591,7 @@ enum diff_symbol {
 	DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
 	DIFF_SYMBOL_STATS_LINE,
 	DIFF_SYMBOL_WORD_DIFF,
+	DIFF_SYMBOL_STAT_SEP,
 };
 /*
  * Flags for content lines:
@@ -776,6 +777,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_WORD_DIFF:
 		fprintf(o->file, "%.*s", len, line);
 		break;
+	case DIFF_SYMBOL_STAT_SEP:
+		fputs(o->stat_sep, o->file);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -5085,10 +5089,10 @@ void diff_flush(struct diff_options *options)
 	if (output_format & DIFF_FORMAT_PATCH) {
 		if (separator) {
 			emit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);
-			if (options->stat_sep) {
+			if (options->stat_sep)
 				/* attach patch instead of inline */
-				fputs(options->stat_sep, options->file);
-			}
+				emit_diff_symbol(options, DIFF_SYMBOL_STAT_SEP,
+						 NULL, 0, 0);
 		}
 
 		diff_flush_patch_all_file_pairs(options);
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 20/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (18 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 19/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-20  2:48                           ` [PATCH 21/26] diff.c: buffer all output if asked to Stefan Beller
                                             ` (6 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 73 +++++++++++++++++++++++++++++++++++++++---------------------------
 1 file changed, 43 insertions(+), 30 deletions(-)

diff --git a/diff.c b/diff.c
index 18bfc4720b..f2f7a4da79 100644
--- a/diff.c
+++ b/diff.c
@@ -592,6 +592,7 @@ enum diff_symbol {
 	DIFF_SYMBOL_STATS_LINE,
 	DIFF_SYMBOL_WORD_DIFF,
 	DIFF_SYMBOL_STAT_SEP,
+	DIFF_SYMBOL_SUMMARY,
 };
 /*
  * Flags for content lines:
@@ -780,6 +781,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_STAT_SEP:
 		fputs(o->stat_sep, o->file);
 		break;
+	case DIFF_SYMBOL_SUMMARY:
+		emit_line(o, "", "", line, len);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -4727,67 +4731,76 @@ static void flush_one_pair(struct diff_filepair *p, struct diff_options *opt)
 	}
 }
 
-static void show_file_mode_name(FILE *file, const char *newdelete, struct diff_filespec *fs)
+static void show_file_mode_name(struct diff_options *opt, const char *newdelete, struct diff_filespec *fs)
 {
+	struct strbuf sb = STRBUF_INIT;
 	if (fs->mode)
-		fprintf(file, " %s mode %06o ", newdelete, fs->mode);
+		strbuf_addf(&sb, " %s mode %06o ", newdelete, fs->mode);
 	else
-		fprintf(file, " %s ", newdelete);
-	write_name_quoted(fs->path, file, '\n');
-}
+		strbuf_addf(&sb, " %s ", newdelete);
 
+	quote_c_style(fs->path, &sb, NULL, 0);
+	strbuf_addch(&sb, '\n');
+	emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
+			 sb.buf, sb.len, 0);
+	strbuf_release(&sb);
+}
 
-static void show_mode_change(FILE *file, struct diff_filepair *p, int show_name,
-		const char *line_prefix)
+static void show_mode_change(struct diff_options *opt, struct diff_filepair *p,
+		int show_name)
 {
 	if (p->one->mode && p->two->mode && p->one->mode != p->two->mode) {
-		fprintf(file, "%s mode change %06o => %06o%c", line_prefix, p->one->mode,
-			p->two->mode, show_name ? ' ' : '\n');
+		struct strbuf sb = STRBUF_INIT;
+		strbuf_addf(&sb, " mode change %06o => %06o",
+			    p->one->mode, p->two->mode);
 		if (show_name) {
-			write_name_quoted(p->two->path, file, '\n');
+			strbuf_addch(&sb, ' ');
+			quote_c_style(p->two->path, &sb, NULL, 0);
 		}
+		emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
+				 sb.buf, sb.len, 0);
+		strbuf_release(&sb);
 	}
 }
 
-static void show_rename_copy(FILE *file, const char *renamecopy, struct diff_filepair *p,
-			const char *line_prefix)
+static void show_rename_copy(struct diff_options *opt, const char *renamecopy,
+		struct diff_filepair *p)
 {
+	struct strbuf sb = STRBUF_INIT;
 	char *names = pprint_rename(p->one->path, p->two->path);
-
-	fprintf(file, " %s %s (%d%%)\n", renamecopy, names, similarity_index(p));
+	strbuf_addf(&sb, " %s %s (%d%%)\n",
+			renamecopy, names, similarity_index(p));
 	free(names);
-	show_mode_change(file, p, 0, line_prefix);
+	emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
+				 sb.buf, sb.len, 0);
+	show_mode_change(opt, p, 0);
 }
 
 static void diff_summary(struct diff_options *opt, struct diff_filepair *p)
 {
-	FILE *file = opt->file;
-	const char *line_prefix = diff_line_prefix(opt);
-
 	switch(p->status) {
 	case DIFF_STATUS_DELETED:
-		fputs(line_prefix, file);
-		show_file_mode_name(file, "delete", p->one);
+		show_file_mode_name(opt, "delete", p->one);
 		break;
 	case DIFF_STATUS_ADDED:
-		fputs(line_prefix, file);
-		show_file_mode_name(file, "create", p->two);
+		show_file_mode_name(opt, "create", p->two);
 		break;
 	case DIFF_STATUS_COPIED:
-		fputs(line_prefix, file);
-		show_rename_copy(file, "copy", p, line_prefix);
+		show_rename_copy(opt, "copy", p);
 		break;
 	case DIFF_STATUS_RENAMED:
-		fputs(line_prefix, file);
-		show_rename_copy(file, "rename", p, line_prefix);
+		show_rename_copy(opt, "rename", p);
 		break;
 	default:
 		if (p->score) {
-			fprintf(file, "%s rewrite ", line_prefix);
-			write_name_quoted(p->two->path, file, ' ');
-			fprintf(file, "(%d%%)\n", similarity_index(p));
+			struct strbuf sb = STRBUF_INIT;
+			strbuf_addstr(&sb, " rewrite ");
+			quote_c_style(p->two->path, &sb, NULL, 0);
+			strbuf_addf(&sb, " (%d%%)\n", similarity_index(p));
+			emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
+					 sb.buf, sb.len, 0);
 		}
-		show_mode_change(file, p, !p->score, line_prefix);
+		show_mode_change(opt, p, !p->score);
 		break;
 	}
 }
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 21/26] diff.c: buffer all output if asked to
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (19 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 20/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-20  2:48                           ` [PATCH 22/26] diff.c: color moved lines differently Stefan Beller
                                             ` (5 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Introduce a new option 'emitted_symbols' in the struct diff_options which
controls whether all output is buffered up until all output is available.
It is set internally in diff.c when necessary.

We'll have a new struct 'emitted_string' in diff.c which will be used to
buffer each line.  The emitted_string will duplicate the memory of the
line to buffer as that is easiest to reason about for now. In a future
patch we may want to decrease the memory usage by not duplicating all
output for buffering but rather we may want to store offsets into the
file or in case of hunk descriptions such as the similarity score, we
could just store the relevant number and reproduce the text later on.

This approach was chosen as a first step because it is quite simple
compared to the alternative with less memory footprint.

emit_diff_symbol factors out the emission part and depending on the
diff_options->emitted_symbols the emission will be performed directly
when calling emit_diff_symbol or after the whole process is done, i.e.
by buffering we have add the possibility for a second pass over the
whole output before doing the actual output.

In 6440d34 (2012-03-14, diff: tweak a _copy_ of diff_options with
word-diff) we introduced a duplicate diff options struct for word
emissions as we may have different regex settings in there.
When buffering the output, we need to operate on just one buffer,
so we have to copy back the emissions of the word buffer into the
main buffer.

Unconditionally enable output via buffer in this patch as it yields
a great opportunity for testing, i.e. all the diff tests from the
test suite pass without having reordering issues (i.e. only parts
of the output got buffered, and we forgot to buffer other parts).
The test suite passes, which gives confidence that we converted all
functions to use emit_string for output.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 diff.h |   2 ++
 2 files changed, 109 insertions(+), 2 deletions(-)

diff --git a/diff.c b/diff.c
index f2f7a4da79..35b5924ff2 100644
--- a/diff.c
+++ b/diff.c
@@ -603,6 +603,47 @@ enum diff_symbol {
 #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)
 #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)
 
+/*
+ * This struct is used when we need to buffer the output of the diff output.
+ *
+ * NEEDSWORK: Instead of storing a copy of the line, add an offset pointer
+ * into the pre/post image file. This pointer could be a union with the
+ * line pointer. By storing an offset into the file instead of the literal line,
+ * we can decrease the memory footprint for the buffered output. At first we
+ * may want to only have indirection for the content lines, but we could also
+ * enhance the state for emitting prefabricated lines, e.g. the similarity
+ * score line or hunk/file headers would only need to store a number or path
+ * and then the output can be constructed later on depending on state.
+ */
+struct emitted_diff_symbol {
+	const char *line;
+	int len;
+	int flags;
+	enum diff_symbol s;
+};
+#define EMITTED_DIFF_SYMBOL_INIT {NULL}
+
+struct emitted_diff_symbols {
+	struct emitted_diff_symbol *buf;
+	int nr, alloc;
+};
+#define EMITTED_DIFF_SYMBOLS_INIT {NULL, 0, 0}
+
+static void append_emitted_diff_symbol(struct diff_options *o,
+				       struct emitted_diff_symbol *e)
+{
+	struct emitted_diff_symbol *f;
+
+	ALLOC_GROW(o->emitted_symbols->buf,
+		   o->emitted_symbols->nr + 1,
+		   o->emitted_symbols->alloc);
+	f = &o->emitted_symbols->buf[o->emitted_symbols->nr++];
+
+	memcpy(f, e, sizeof(struct emitted_diff_symbol));
+	f->line = e->line ? xmemdupz(e->line, e->len) : NULL;
+}
+
+
 static void emit_line_ws_markup(struct diff_options *o,
 				const char *set, const char *reset,
 				const char *line, int len, char sign,
@@ -629,12 +670,18 @@ static void emit_line_ws_markup(struct diff_options *o,
 	}
 }
 
-static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
-			     const char *line, int len, unsigned flags)
+static void emit_diff_symbol_from_struct(struct diff_options *o,
+					 struct emitted_diff_symbol *eds)
 {
 	static const char *nneof = " No newline at end of file\n";
 	const char *context, *reset, *set, *meta, *fraginfo;
 	struct strbuf sb = STRBUF_INIT;
+
+	enum diff_symbol s = eds->s;
+	const char *line = eds->line;
+	int len = eds->len;
+	unsigned flags = eds->flags;
+
 	switch (s) {
 	case DIFF_SYMBOL_SEPARATOR:
 		fprintf(o->file, "%s%c",
@@ -790,6 +837,17 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	strbuf_release(&sb);
 }
 
+static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
+			     const char *line, int len, unsigned flags)
+{
+	struct emitted_diff_symbol e = {line, len, flags, s};
+
+	if (o->emitted_symbols)
+		append_emitted_diff_symbol(o, &e);
+	else
+		emit_diff_symbol_from_struct(o, &e);
+}
+
 void diff_emit_submodule_del(struct diff_options *o, const char *line)
 {
 	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);
@@ -1384,9 +1442,29 @@ static void diff_words_show(struct diff_words_data *diff_words)
 /* In "color-words" mode, show word-diff of words accumulated in the buffer */
 static void diff_words_flush(struct emit_callback *ecbdata)
 {
+	struct diff_options *wo = ecbdata->diff_words->opt;
+
 	if (ecbdata->diff_words->minus.text.size ||
 	    ecbdata->diff_words->plus.text.size)
 		diff_words_show(ecbdata->diff_words);
+
+	if (wo->emitted_symbols) {
+		struct diff_options *o = ecbdata->opt;
+		struct emitted_diff_symbols *wol = wo->emitted_symbols;
+		int i;
+
+		/*
+		 * NEEDSWORK:
+		 * Instead of appending each, concat all words to a line?
+		 */
+		for (i = 0; i < wol->nr; i++)
+			append_emitted_diff_symbol(o, &wol->buf[i]);
+
+		for (i = 0; i < wol->nr; i++)
+			free((void *)wol->buf[i].line);
+
+		wol->nr = 0;
+	}
 }
 
 static void diff_filespec_load_driver(struct diff_filespec *one)
@@ -1422,6 +1500,11 @@ static void init_diff_words_data(struct emit_callback *ecbdata,
 		xcalloc(1, sizeof(struct diff_words_data));
 	ecbdata->diff_words->type = o->word_diff;
 	ecbdata->diff_words->opt = o;
+
+	if (orig_opts->emitted_symbols)
+		o->emitted_symbols =
+			xcalloc(1, sizeof(struct emitted_diff_symbols));
+
 	if (!o->word_regex)
 		o->word_regex = userdiff_word_regex(one);
 	if (!o->word_regex)
@@ -1456,6 +1539,7 @@ static void free_diff_words_data(struct emit_callback *ecbdata)
 {
 	if (ecbdata->diff_words) {
 		diff_words_flush(ecbdata);
+		free (ecbdata->diff_words->opt->emitted_symbols);
 		free (ecbdata->diff_words->opt);
 		free (ecbdata->diff_words->minus.text.ptr);
 		free (ecbdata->diff_words->minus.orig);
@@ -5008,12 +5092,33 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)
 static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 {
 	int i;
+	static struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;
 	struct diff_queue_struct *q = &diff_queued_diff;
+	/*
+	 * For testing purposes we want to make sure the diff machinery
+	 * works completely with the buffer. If there is anything emitted
+	 * outside the emit_string, then the order is screwed
+	 * up and the tests will fail.
+	 *
+	 * TODO (later in this series):
+	 * We'll unset this pointer in a later patch.
+	 */
+	o->emitted_symbols = &esm;
+
 	for (i = 0; i < q->nr; i++) {
 		struct diff_filepair *p = q->queue[i];
 		if (check_pair_status(p))
 			diff_flush_patch(p, o);
 	}
+
+	if (o->emitted_symbols) {
+		for (i = 0; i < esm.nr; i++)
+			emit_diff_symbol_from_struct(o, &esm.buf[i]);
+
+		for (i = 0; i < esm.nr; i++)
+			free((void *)esm.buf[i].line);
+	}
+	esm.nr = 0;
 }
 
 void diff_flush(struct diff_options *options)
diff --git a/diff.h b/diff.h
index b73f7a1e75..65fc9dbb4b 100644
--- a/diff.h
+++ b/diff.h
@@ -186,6 +186,8 @@ struct diff_options {
 	void *output_prefix_data;
 
 	int diff_path_counter;
+
+	struct emitted_diff_symbols *emitted_symbols;
 };
 
 void diff_emit_submodule_del(struct diff_options *o, const char *line);
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 22/26] diff.c: color moved lines differently
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (20 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 21/26] diff.c: buffer all output if asked to Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-20 20:13                             ` Jonathan Tan
  2017-06-20  2:48                           ` [PATCH 23/26] diff.c: color moved lines differently, plain mode Stefan Beller
                                             ` (4 subsequent siblings)
  26 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

When a patch consists mostly of moving blocks of code around, it can
be quite tedious to ensure that the blocks are moved verbatim, and not
undesirably modified in the move. To that end, color blocks that are
moved within the same patch differently. For example (OM, del, add,
and NM are different colors):

    [OM]  -void sensitive_stuff(void)
    [OM]  -{
    [OM]  -        if (!is_authorized_user())
    [OM]  -                die("unauthorized");
    [OM]  -        sensitive_stuff(spanning,
    [OM]  -                        multiple,
    [OM]  -                        lines);
    [OM]  -}

           void another_function()
           {
    [del] -        printf("foo");
    [add] +        printf("bar");
           }

    [NM]  +void sensitive_stuff(void)
    [NM]  +{
    [NM]  +        if (!is_authorized_user())
    [NM]  +                die("unauthorized");
    [NM]  +        sensitive_stuff(spanning,
    [NM]  +                        multiple,
    [NM]  +                        lines);
    [NM]  +}

However adjacent blocks may be problematic. For example, in this
potentially malicious patch, the swapping of blocks can be spotted:

    [OM]  -void sensitive_stuff(void)
    [OM]  -{
    [OMA] -        if (!is_authorized_user())
    [OMA] -                die("unauthorized");
    [OM]  -        sensitive_stuff(spanning,
    [OM]  -                        multiple,
    [OM]  -                        lines);
    [OMA] -}

           void another_function()
           {
    [del] -        printf("foo");
    [add] +        printf("bar");
           }

    [NM]  +void sensitive_stuff(void)
    [NM]  +{
    [NMA] +        sensitive_stuff(spanning,
    [NMA] +                        multiple,
    [NMA] +                        lines);
    [NM]  +        if (!is_authorized_user())
    [NM]  +                die("unauthorized");
    [NMA] +}

If the moved code is larger, it is easier to hide some permutation in the
code, which is why some alternative coloring is needed.

This patch implements the first mode:
* basic alternating 'Zebra' mode
  This conveys all information needed to the user.  Defer customization to
  later patches.

First I implemented an alternative design, which would try to fingerprint
a line by its neighbors to detect if we are in a block or at the boundary.
This idea iss error prone as it inspected each line and its neighboring
lines to determine if the line was (a) moved and (b) if was deep inside
a hunk by having matching neighboring lines. This is unreliable as the
we can construct hunks which have equal neighbors that just exceed the
number of lines inspected. (Think of 'AXYZBXYZCXYZD..' with each letter
as a line, that is permutated to AXYZCXYZBXYZD..').

Instead this provides a dynamic programming greedy algorithm that finds
the largest moved hunk and then has several modes on highlighting bounds.

A note on the options '--submodule=diff' and '--color-words/--word-diff':
In the conversion to use emit_line in the prior patches both submodules
as well as word diff output carefully chose to call emit_line with sign=0.
All output with sign=0 is ignored for move detection purposes in this
patch, such that no weird looking output will be generated for these
cases. This leads to another thought: We could pass on '--color-moved' to
submodules such that they color up moved lines for themselves. If we'd do
so only line moves within a repository boundary are marked up.

Helped-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c                     | 304 ++++++++++++++++++++++++++++++++++++++++++---
 diff.h                     |  10 +-
 t/t4015-diff-whitespace.sh | 196 +++++++++++++++++++++++++++++
 3 files changed, 495 insertions(+), 15 deletions(-)

diff --git a/diff.c b/diff.c
index 35b5924ff2..20c1f9b99f 100644
--- a/diff.c
+++ b/diff.c
@@ -15,6 +15,7 @@
 #include "userdiff.h"
 #include "submodule-config.h"
 #include "submodule.h"
+#include "hashmap.h"
 #include "ll-merge.h"
 #include "string-list.h"
 #include "argv-array.h"
@@ -31,6 +32,7 @@ static int diff_indent_heuristic; /* experimental */
 static int diff_rename_limit_default = 400;
 static int diff_suppress_blank_empty;
 static int diff_use_color_default = -1;
+static int diff_color_moved_default;
 static int diff_context_default = 3;
 static int diff_interhunk_context_default;
 static const char *diff_word_regex_cfg;
@@ -55,6 +57,10 @@ static char diff_colors[][COLOR_MAXLEN] = {
 	GIT_COLOR_YELLOW,	/* COMMIT */
 	GIT_COLOR_BG_RED,	/* WHITESPACE */
 	GIT_COLOR_NORMAL,	/* FUNCINFO */
+	GIT_COLOR_MAGENTA,	/* OLD_MOVED */
+	GIT_COLOR_BLUE,		/* OLD_MOVED ALTERNATIVE */
+	GIT_COLOR_CYAN,		/* NEW_MOVED */
+	GIT_COLOR_YELLOW,	/* NEW_MOVED ALTERNATIVE */
 };
 
 static NORETURN void die_want_option(const char *option_name)
@@ -80,6 +86,14 @@ static int parse_diff_color_slot(const char *var)
 		return DIFF_WHITESPACE;
 	if (!strcasecmp(var, "func"))
 		return DIFF_FUNCINFO;
+	if (!strcasecmp(var, "oldmoved"))
+		return DIFF_FILE_OLD_MOVED;
+	if (!strcasecmp(var, "oldmovedalternative"))
+		return DIFF_FILE_OLD_MOVED_ALT;
+	if (!strcasecmp(var, "newmoved"))
+		return DIFF_FILE_NEW_MOVED;
+	if (!strcasecmp(var, "newmovedalternative"))
+		return DIFF_FILE_NEW_MOVED_ALT;
 	return -1;
 }
 
@@ -228,12 +242,29 @@ int git_diff_heuristic_config(const char *var, const char *value, void *cb)
 	return 0;
 }
 
+static int parse_color_moved(const char *arg)
+{
+	if (!strcmp(arg, "no"))
+		return COLOR_MOVED_NO;
+	else if (!strcmp(arg, "zebra"))
+		return COLOR_MOVED_ZEBRA;
+	else
+		return -1;
+}
+
 int git_diff_ui_config(const char *var, const char *value, void *cb)
 {
 	if (!strcmp(var, "diff.color") || !strcmp(var, "color.diff")) {
 		diff_use_color_default = git_config_colorbool(var, value);
 		return 0;
 	}
+	if (!strcmp(var, "diff.colormoved")) {
+		int cm = parse_color_moved(value);
+		if (cm < 0)
+			return -1;
+		diff_color_moved_default = cm;
+		return 0;
+	}
 	if (!strcmp(var, "diff.context")) {
 		diff_context_default = git_config_int(var, value);
 		if (diff_context_default < 0)
@@ -600,7 +631,9 @@ enum diff_symbol {
  * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT
  * 16 is marking if the line is blank at EOF
  */
-#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)
+#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF	(1<<16)
+#define DIFF_SYMBOL_MOVED_LINE			(1<<17)
+#define DIFF_SYMBOL_MOVED_LINE_ZEBRA		(1<<18)
 #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)
 
 /*
@@ -643,6 +676,213 @@ static void append_emitted_diff_symbol(struct diff_options *o,
 	f->line = e->line ? xmemdupz(e->line, e->len) : NULL;
 }
 
+struct moved_entry {
+	struct hashmap_entry ent;
+	const struct emitted_diff_symbol *es;
+	struct moved_entry *next_line;
+};
+
+static void get_ws_cleaned_string(const struct emitted_diff_symbol *l,
+				  struct strbuf *out)
+{
+	int i;
+	for (i = 0; i < l->len; i++) {
+		if (isspace(l->line[i]))
+			continue;
+		strbuf_addch(out, l->line[i]);
+	}
+}
+
+static int emitted_symbol_cmp_no_ws(const struct emitted_diff_symbol *a,
+				    const struct emitted_diff_symbol *b,
+				    const void *keydata)
+{
+	int ret;
+	struct strbuf sba = STRBUF_INIT;
+	struct strbuf sbb = STRBUF_INIT;
+
+	get_ws_cleaned_string(a, &sba);
+	get_ws_cleaned_string(b, &sbb);
+	ret = sba.len != sbb.len || strncmp(sba.buf, sbb.buf, sba.len);
+
+	strbuf_release(&sba);
+	strbuf_release(&sbb);
+	return ret;
+}
+
+static int emitted_symbol_cmp(const struct emitted_diff_symbol *a,
+			      const struct emitted_diff_symbol *b,
+			      const void *keydata)
+{
+	return a->len != b->len || strncmp(a->line, b->line, a->len);
+}
+
+static int moved_entry_cmp(const struct moved_entry *a,
+			   const struct moved_entry *b,
+			   const void *keydata)
+{
+	return emitted_symbol_cmp(a->es, b->es, keydata);
+}
+
+static int moved_entry_cmp_no_ws(const struct moved_entry *a,
+				 const struct moved_entry *b,
+				 const void *keydata)
+{
+	return emitted_symbol_cmp_no_ws(a->es, b->es, keydata);
+}
+
+static unsigned get_string_hash(struct emitted_diff_symbol *es, unsigned ignore_ws)
+{
+	static struct strbuf sb = STRBUF_INIT;
+
+	if (ignore_ws) {
+		strbuf_reset(&sb);
+		get_ws_cleaned_string(es, &sb);
+		return memhash(sb.buf, sb.len);
+	} else {
+		return memhash(es->line, es->len);
+	}
+}
+
+static struct moved_entry *prepare_entry(struct diff_options *o,
+					 int line_no)
+{
+	struct moved_entry *ret = xmalloc(sizeof(*ret));
+	unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
+	struct emitted_diff_symbol *l = &o->emitted_symbols->buf[line_no];
+
+	ret->ent.hash = get_string_hash(l, ignore_ws);
+	ret->es = l;
+	ret->next_line = NULL;
+
+	return ret;
+}
+
+static void add_lines_to_move_detection(struct diff_options *o,
+					struct hashmap *add_lines,
+					struct hashmap *del_lines)
+{
+	struct moved_entry *prev_line = NULL;
+
+	int n;
+	for (n = 0; n < o->emitted_symbols->nr; n++) {
+		struct hashmap *hm;
+		struct moved_entry *key;
+
+		switch (o->emitted_symbols->buf[n].s) {
+		case DIFF_SYMBOL_PLUS:
+			hm = add_lines;
+			break;
+		case DIFF_SYMBOL_MINUS:
+			hm = del_lines;
+			break;
+		default:
+			prev_line = NULL;
+			continue;
+		}
+
+		key = prepare_entry(o, n);
+		if (prev_line && prev_line->es->s == o->emitted_symbols->buf[n].s)
+			prev_line->next_line = key;
+
+		hashmap_add(hm, key);
+		prev_line = key;
+	}
+}
+
+/* Find blocks of moved code, delegate actual coloring decision to helper */
+static void mark_color_as_moved(struct diff_options *o,
+				struct hashmap *add_lines,
+				struct hashmap *del_lines)
+{
+	struct moved_entry **pmb = NULL; /* potentially moved blocks */
+	int pmb_nr = 0, pmb_alloc = 0;
+	int n, flipped_block = 1;
+
+	for (n = 0; n < o->emitted_symbols->nr; n++) {
+		struct hashmap *hm = NULL;
+		struct moved_entry *key;
+		struct moved_entry *match = NULL;
+		struct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];
+		int i, lp, rp;
+
+		switch (l->s) {
+		case DIFF_SYMBOL_PLUS:
+			hm = del_lines;
+			key = prepare_entry(o, n);
+			match = hashmap_get(hm, key, o);
+			free(key);
+			break;
+		case DIFF_SYMBOL_MINUS:
+			hm = add_lines;
+			key = prepare_entry(o, n);
+			match = hashmap_get(hm, key, o);
+			free(key);
+			break;
+		default:
+			flipped_block = 1;
+		}
+
+		if (!match) {
+			pmb_nr = 0;
+			continue;
+		}
+
+		l->flags |= DIFF_SYMBOL_MOVED_LINE;
+
+		/* Check any potential block runs, advance each or nullify */
+		for (i = 0; i < pmb_nr; i++) {
+			struct moved_entry *p = pmb[i];
+			struct moved_entry *pnext = (p && p->next_line) ?
+					p->next_line : NULL;
+			if (pnext &&
+			    !emitted_symbol_cmp(pnext->es, l, o)) {
+				pmb[i] = p->next_line;
+			} else {
+				pmb[i] = NULL;
+			}
+		}
+
+		/* Shrink the set of potential block to the remaining running */
+		for (lp = 0, rp = pmb_nr - 1; lp <= rp;) {
+			while (lp < pmb_nr && pmb[lp])
+				lp++;
+			/* lp points at the first NULL now */
+
+			while (rp > -1 && !pmb[rp])
+				rp--;
+			/* rp points at the last non-NULL */
+
+			if (lp < pmb_nr && rp > -1 && lp < rp) {
+				pmb[lp] = pmb[rp];
+				pmb[rp] = NULL;
+				rp--;
+				lp++;
+			}
+		}
+
+		/* Remember the number of running sets */
+		pmb_nr = rp + 1;
+
+		if (pmb_nr == 0) {
+			/*
+			 * The current line is the start of a new block.
+			 * Setup the set of potential blocks.
+			 */
+			for (; match; match = hashmap_get_next(hm, match)) {
+				ALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);
+				pmb[pmb_nr++] = match;
+			}
+
+			flipped_block = (flipped_block + 1) % 2;
+		}
+
+		if (flipped_block)
+			l->flags |= DIFF_SYMBOL_MOVED_LINE_ZEBRA;
+	}
+
+	free(pmb);
+}
 
 static void emit_line_ws_markup(struct diff_options *o,
 				const char *set, const char *reset,
@@ -715,14 +955,24 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
 		emit_line(o, context, reset, line, len);
 		break;
 	case DIFF_SYMBOL_PLUS:
-		set = diff_get_color_opt(o, DIFF_FILE_NEW);
+		if (flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA)
+			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);
+		else if (flags & DIFF_SYMBOL_MOVED_LINE)
+			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);
+		else
+			set = diff_get_color_opt(o, DIFF_FILE_NEW);
 		reset = diff_get_color_opt(o, DIFF_RESET);
 		emit_line_ws_markup(o, set, reset, line, len, '+',
 				    flags & DIFF_SYMBOL_CONTENT_WS_MASK,
 				    flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);
 		break;
 	case DIFF_SYMBOL_MINUS:
-		set = diff_get_color_opt(o, DIFF_FILE_OLD);
+		if (flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA)
+			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);
+		else if (flags & DIFF_SYMBOL_MOVED_LINE)
+			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);
+		else
+			set = diff_get_color_opt(o, DIFF_FILE_OLD);
 		reset = diff_get_color_opt(o, DIFF_RESET);
 		emit_line_ws_markup(o, set, reset, line, len, '-',
 				    flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
@@ -3750,6 +4000,8 @@ void diff_setup(struct diff_options *options)
 		options->a_prefix = "a/";
 		options->b_prefix = "b/";
 	}
+
+	options->color_moved = diff_color_moved_default;
 }
 
 void diff_setup_done(struct diff_options *options)
@@ -3859,6 +4111,9 @@ void diff_setup_done(struct diff_options *options)
 
 	if (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)
 		die(_("--follow requires exactly one pathspec"));
+
+	if (!options->use_color || external_diff())
+		options->color_moved = 0;
 }
 
 static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)
@@ -4283,7 +4538,16 @@ int diff_opt_parse(struct diff_options *options,
 	}
 	else if (!strcmp(arg, "--no-color"))
 		options->use_color = 0;
-	else if (!strcmp(arg, "--color-words")) {
+	else if (!strcmp(arg, "--color-moved"))
+		options->color_moved = COLOR_MOVED_ZEBRA;
+	else if (!strcmp(arg, "--no-color-moved"))
+		options->color_moved = COLOR_MOVED_NO;
+	else if (skip_prefix(arg, "--color-moved=", &arg)) {
+		int cm = parse_color_moved(arg);
+		if (cm < 0)
+			die("bad --color-moved argument: %s", arg);
+		options->color_moved = cm;
+	} else if (!strcmp(arg, "--color-words")) {
 		options->use_color = 1;
 		options->word_diff = DIFF_WORDS_COLOR;
 	}
@@ -5094,16 +5358,9 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 	int i;
 	static struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;
 	struct diff_queue_struct *q = &diff_queued_diff;
-	/*
-	 * For testing purposes we want to make sure the diff machinery
-	 * works completely with the buffer. If there is anything emitted
-	 * outside the emit_string, then the order is screwed
-	 * up and the tests will fail.
-	 *
-	 * TODO (later in this series):
-	 * We'll unset this pointer in a later patch.
-	 */
-	o->emitted_symbols = &esm;
+
+	if (o->color_moved)
+		o->emitted_symbols = &esm;
 
 	for (i = 0; i < q->nr; i++) {
 		struct diff_filepair *p = q->queue[i];
@@ -5112,6 +5369,24 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 	}
 
 	if (o->emitted_symbols) {
+		if (o->color_moved) {
+			struct hashmap add_lines, del_lines;
+			unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
+
+			hashmap_init(&del_lines, ignore_ws ?
+				(hashmap_cmp_fn)moved_entry_cmp_no_ws :
+				(hashmap_cmp_fn)moved_entry_cmp, 0);
+			hashmap_init(&add_lines, ignore_ws ?
+				(hashmap_cmp_fn)moved_entry_cmp_no_ws :
+				(hashmap_cmp_fn)moved_entry_cmp, 0);
+
+			add_lines_to_move_detection(o, &add_lines, &del_lines);
+			mark_color_as_moved(o, &add_lines, &del_lines);
+
+			hashmap_free(&add_lines, 0);
+			hashmap_free(&del_lines, 0);
+		}
+
 		for (i = 0; i < esm.nr; i++)
 			emit_diff_symbol_from_struct(o, &esm.buf[i]);
 
@@ -5195,6 +5470,7 @@ void diff_flush(struct diff_options *options)
 		if (!options->file)
 			die_errno("Could not open /dev/null");
 		options->close_file = 1;
+		options->color_moved = 0;
 		for (i = 0; i < q->nr; i++) {
 			struct diff_filepair *p = q->queue[i];
 			if (check_pair_status(p))
diff --git a/diff.h b/diff.h
index 65fc9dbb4b..7726ad255c 100644
--- a/diff.h
+++ b/diff.h
@@ -188,6 +188,10 @@ struct diff_options {
 	int diff_path_counter;
 
 	struct emitted_diff_symbols *emitted_symbols;
+	enum {
+		COLOR_MOVED_NO = 0,
+		COLOR_MOVED_ZEBRA = 2,
+	} color_moved;
 };
 
 void diff_emit_submodule_del(struct diff_options *o, const char *line);
@@ -208,7 +212,11 @@ enum color_diff {
 	DIFF_FILE_NEW = 5,
 	DIFF_COMMIT = 6,
 	DIFF_WHITESPACE = 7,
-	DIFF_FUNCINFO = 8
+	DIFF_FUNCINFO = 8,
+	DIFF_FILE_OLD_MOVED = 9,
+	DIFF_FILE_OLD_MOVED_ALT = 10,
+	DIFF_FILE_NEW_MOVED = 11,
+	DIFF_FILE_NEW_MOVED_ALT = 12
 };
 const char *diff_get_color(int diff_use_color, enum color_diff ix);
 #define diff_get_color_opt(o, ix) \
diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
index 289806d0c7..4a03766f1f 100755
--- a/t/t4015-diff-whitespace.sh
+++ b/t/t4015-diff-whitespace.sh
@@ -972,4 +972,200 @@ test_expect_success 'option overrides diff.wsErrorHighlight' '
 
 '
 
+test_expect_success 'detect moved code, complete file' '
+	git reset --hard &&
+	cat <<-\EOF >test.c &&
+	#include<stdio.h>
+	main()
+	{
+	printf("Hello World");
+	}
+	EOF
+	git add test.c &&
+	git commit -m "add main function" &&
+	git mv test.c main.c &&
+	test_config color.diff.oldMoved "normal red" &&
+	test_config color.diff.newMoved "normal green" &&
+	git diff HEAD --color-moved --no-renames | test_decode_color >actual &&
+	cat >expected <<-\EOF &&
+	<BOLD>diff --git a/main.c b/main.c<RESET>
+	<BOLD>new file mode 100644<RESET>
+	<BOLD>index 0000000..a986c57<RESET>
+	<BOLD>--- /dev/null<RESET>
+	<BOLD>+++ b/main.c<RESET>
+	<CYAN>@@ -0,0 +1,5 @@<RESET>
+	<BGREEN>+<RESET><BGREEN>#include<stdio.h><RESET>
+	<BGREEN>+<RESET><BGREEN>main()<RESET>
+	<BGREEN>+<RESET><BGREEN>{<RESET>
+	<BGREEN>+<RESET><BGREEN>printf("Hello World");<RESET>
+	<BGREEN>+<RESET><BGREEN>}<RESET>
+	<BOLD>diff --git a/test.c b/test.c<RESET>
+	<BOLD>deleted file mode 100644<RESET>
+	<BOLD>index a986c57..0000000<RESET>
+	<BOLD>--- a/test.c<RESET>
+	<BOLD>+++ /dev/null<RESET>
+	<CYAN>@@ -1,5 +0,0 @@<RESET>
+	<BRED>-#include<stdio.h><RESET>
+	<BRED>-main()<RESET>
+	<BRED>-{<RESET>
+	<BRED>-printf("Hello World");<RESET>
+	<BRED>-}<RESET>
+	EOF
+
+	test_cmp expected actual
+'
+
+test_expect_success 'detect malicious moved code, inside file' '
+	test_config color.diff.oldMoved "normal red" &&
+	test_config color.diff.newMoved "normal green" &&
+	test_config color.diff.oldMovedAlternative "blue" &&
+	test_config color.diff.newMovedAlternative "yellow" &&
+	git reset --hard &&
+	cat <<-\EOF >main.c &&
+		#include<stdio.h>
+		int stuff()
+		{
+			printf("Hello ");
+			printf("World\n");
+		}
+
+		int secure_foo(struct user *u)
+		{
+			if (!u->is_allowed_foo)
+				return;
+			foo(u);
+		}
+
+		int main()
+		{
+			foo();
+		}
+	EOF
+	cat <<-\EOF >test.c &&
+		#include<stdio.h>
+		int bar()
+		{
+			printf("Hello World, but different\n");
+		}
+
+		int another_function()
+		{
+			bar();
+		}
+	EOF
+	git add main.c test.c &&
+	git commit -m "add main and test file" &&
+	cat <<-\EOF >main.c &&
+		#include<stdio.h>
+		int stuff()
+		{
+			printf("Hello ");
+			printf("World\n");
+		}
+
+		int main()
+		{
+			foo();
+		}
+	EOF
+	cat <<-\EOF >test.c &&
+		#include<stdio.h>
+		int bar()
+		{
+			printf("Hello World, but different\n");
+		}
+
+		int secure_foo(struct user *u)
+		{
+			foo(u);
+			if (!u->is_allowed_foo)
+				return;
+		}
+
+		int another_function()
+		{
+			bar();
+		}
+	EOF
+	git diff HEAD --no-renames --color-moved=zebra| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/main.c b/main.c<RESET>
+	<BOLD>index 27a619c..7cf9336 100644<RESET>
+	<BOLD>--- a/main.c<RESET>
+	<BOLD>+++ b/main.c<RESET>
+	<CYAN>@@ -5,13 +5,6 @@<RESET> <RESET>printf("Hello ");<RESET>
+	 printf("World\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BRED>-int secure_foo(struct user *u)<RESET>
+	<BRED>-{<RESET>
+	<BLUE>-if (!u->is_allowed_foo)<RESET>
+	<BLUE>-return;<RESET>
+	<BRED>-foo(u);<RESET>
+	<BLUE>-}<RESET>
+	<BLUE>-<RESET>
+	 int main()<RESET>
+	 {<RESET>
+	 foo();<RESET>
+	<BOLD>diff --git a/test.c b/test.c<RESET>
+	<BOLD>index 1dc1d85..2bedec9 100644<RESET>
+	<BOLD>--- a/test.c<RESET>
+	<BOLD>+++ b/test.c<RESET>
+	<CYAN>@@ -4,6 +4,13 @@<RESET> <RESET>int bar()<RESET>
+	 printf("Hello World, but different\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BGREEN>+<RESET><BGREEN>int secure_foo(struct user *u)<RESET>
+	<BGREEN>+<RESET><BGREEN>{<RESET>
+	<YELLOW>+<RESET><YELLOW>foo(u);<RESET>
+	<BGREEN>+<RESET><BGREEN>if (!u->is_allowed_foo)<RESET>
+	<BGREEN>+<RESET><BGREEN>return;<RESET>
+	<YELLOW>+<RESET><YELLOW>}<RESET>
+	<YELLOW>+<RESET>
+	 int another_function()<RESET>
+	 {<RESET>
+	 bar();<RESET>
+	EOF
+
+	test_cmp expected actual
+'
+
+test_expect_success 'no effect from --color-moved with --word-diff' '
+	cat <<-\EOF >text.txt &&
+	Lorem Ipsum is simply dummy text of the printing and typesetting industry.
+	EOF
+	git add text.txt &&
+	git commit -a -m "clean state" &&
+	cat <<-\EOF >text.txt &&
+	simply Lorem Ipsum dummy is text of the typesetting and printing industry.
+	EOF
+	git diff --color-moved --word-diff >actual &&
+	git diff --word-diff >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'move detection with submodules' '
+	test_create_repo bananas &&
+	echo ripe >bananas/recipe &&
+	git -C bananas add recipe &&
+	test_commit fruit &&
+	test_commit -C bananas recipe &&
+	git submodule add ./bananas &&
+	git add bananas &&
+	git commit -a -m "bananas are like a heavy library?" &&
+	echo foul >bananas/recipe &&
+	echo ripe >fruit.t &&
+
+	git diff --submodule=diff --color-moved >actual &&
+
+	# no move detection as the moved line is across repository boundaries.
+	test_decode_color <actual >decoded_actual &&
+	! grep BGREEN decoded_actual &&
+	! grep BRED decoded_actual &&
+
+	# nor did we mess with it another way
+	git diff --submodule=diff | test_decode_color >expect &&
+	test_cmp expect decoded_actual
+'
+
 test_done
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 23/26] diff.c: color moved lines differently, plain mode
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (21 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 22/26] diff.c: color moved lines differently Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-20  2:48                           ` [PATCH 24/26] diff.c: add dimming to moved line detection Stefan Beller
                                             ` (3 subsequent siblings)
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Add the 'plain' mode for move detection of code.
This omits the checking for adjacent blocks, so it is not as useful.
If you have a lot of the same blocks moved in the same patch, the 'Zebra'
would end up slow as it is O(n^2) (n is number of same blocks).
So this may be useful there and is generally easy to add

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c                     |  5 +++++
 diff.h                     |  1 +
 t/t4015-diff-whitespace.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 56 insertions(+), 1 deletion(-)

diff --git a/diff.c b/diff.c
index 20c1f9b99f..0d41a53b76 100644
--- a/diff.c
+++ b/diff.c
@@ -246,6 +246,8 @@ static int parse_color_moved(const char *arg)
 {
 	if (!strcmp(arg, "no"))
 		return COLOR_MOVED_NO;
+	else if (!strcmp(arg, "plain"))
+		return COLOR_MOVED_PLAIN;
 	else if (!strcmp(arg, "zebra"))
 		return COLOR_MOVED_ZEBRA;
 	else
@@ -830,6 +832,9 @@ static void mark_color_as_moved(struct diff_options *o,
 
 		l->flags |= DIFF_SYMBOL_MOVED_LINE;
 
+		if (o->color_moved == COLOR_MOVED_PLAIN)
+			continue;
+
 		/* Check any potential block runs, advance each or nullify */
 		for (i = 0; i < pmb_nr; i++) {
 			struct moved_entry *p = pmb[i];
diff --git a/diff.h b/diff.h
index 7726ad255c..1aae8738ca 100644
--- a/diff.h
+++ b/diff.h
@@ -190,6 +190,7 @@ struct diff_options {
 	struct emitted_diff_symbols *emitted_symbols;
 	enum {
 		COLOR_MOVED_NO = 0,
+		COLOR_MOVED_PLAIN = 1,
 		COLOR_MOVED_ZEBRA = 2,
 	} color_moved;
 };
diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
index 4a03766f1f..1ca16435d6 100755
--- a/t/t4015-diff-whitespace.sh
+++ b/t/t4015-diff-whitespace.sh
@@ -986,7 +986,7 @@ test_expect_success 'detect moved code, complete file' '
 	git mv test.c main.c &&
 	test_config color.diff.oldMoved "normal red" &&
 	test_config color.diff.newMoved "normal green" &&
-	git diff HEAD --color-moved --no-renames | test_decode_color >actual &&
+	git diff HEAD --color-moved=zebra --no-renames | test_decode_color >actual &&
 	cat >expected <<-\EOF &&
 	<BOLD>diff --git a/main.c b/main.c<RESET>
 	<BOLD>new file mode 100644<RESET>
@@ -1130,6 +1130,55 @@ test_expect_success 'detect malicious moved code, inside file' '
 	test_cmp expected actual
 '
 
+test_expect_success 'plain moved code, inside file' '
+	test_config color.diff.oldMoved "normal red" &&
+	test_config color.diff.newMoved "normal green" &&
+	test_config color.diff.oldMovedAlternative "blue" &&
+	test_config color.diff.newMovedAlternative "yellow" &&
+	# needs previous test as setup
+	git diff HEAD --no-renames --color-moved=plain| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/main.c b/main.c<RESET>
+	<BOLD>index 27a619c..7cf9336 100644<RESET>
+	<BOLD>--- a/main.c<RESET>
+	<BOLD>+++ b/main.c<RESET>
+	<CYAN>@@ -5,13 +5,6 @@<RESET> <RESET>printf("Hello ");<RESET>
+	 printf("World\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BRED>-int secure_foo(struct user *u)<RESET>
+	<BRED>-{<RESET>
+	<BRED>-if (!u->is_allowed_foo)<RESET>
+	<BRED>-return;<RESET>
+	<BRED>-foo(u);<RESET>
+	<BRED>-}<RESET>
+	<BRED>-<RESET>
+	 int main()<RESET>
+	 {<RESET>
+	 foo();<RESET>
+	<BOLD>diff --git a/test.c b/test.c<RESET>
+	<BOLD>index 1dc1d85..2bedec9 100644<RESET>
+	<BOLD>--- a/test.c<RESET>
+	<BOLD>+++ b/test.c<RESET>
+	<CYAN>@@ -4,6 +4,13 @@<RESET> <RESET>int bar()<RESET>
+	 printf("Hello World, but different\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BGREEN>+<RESET><BGREEN>int secure_foo(struct user *u)<RESET>
+	<BGREEN>+<RESET><BGREEN>{<RESET>
+	<BGREEN>+<RESET><BGREEN>foo(u);<RESET>
+	<BGREEN>+<RESET><BGREEN>if (!u->is_allowed_foo)<RESET>
+	<BGREEN>+<RESET><BGREEN>return;<RESET>
+	<BGREEN>+<RESET><BGREEN>}<RESET>
+	<BGREEN>+<RESET>
+	 int another_function()<RESET>
+	 {<RESET>
+	 bar();<RESET>
+	EOF
+
+	test_cmp expected actual
+'
+
 test_expect_success 'no effect from --color-moved with --word-diff' '
 	cat <<-\EOF >text.txt &&
 	Lorem Ipsum is simply dummy text of the printing and typesetting industry.
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 24/26] diff.c: add dimming to moved line detection
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (22 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 23/26] diff.c: color moved lines differently, plain mode Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-21 20:23                             ` Junio C Hamano
  2017-06-20  2:48                           ` [PATCH 25/26] diff: document the new --color-moved setting Stefan Beller
                                             ` (2 subsequent siblings)
  26 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Any lines inside a moved block of code are not interesting. Boundaries
of blocks are only interesting if they are next to another block of moved
code.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 color.h                    |   2 +
 diff.c                     | 139 ++++++++++++++++++++++++++++++++++++++++-----
 diff.h                     |   9 ++-
 t/t4015-diff-whitespace.sh | 124 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 259 insertions(+), 15 deletions(-)

diff --git a/color.h b/color.h
index 90627650fc..0e091b0cf5 100644
--- a/color.h
+++ b/color.h
@@ -42,6 +42,8 @@ struct strbuf;
 #define GIT_COLOR_BG_BLUE	"\033[44m"
 #define GIT_COLOR_BG_MAGENTA	"\033[45m"
 #define GIT_COLOR_BG_CYAN	"\033[46m"
+#define GIT_COLOR_DI		"\033[2m"
+#define GIT_COLOR_DI_IT		"\033[2;3m"
 
 /* A special value meaning "no color selected" */
 #define GIT_COLOR_NIL "NIL"
diff --git a/diff.c b/diff.c
index 0d41a53b76..7756f7610c 100644
--- a/diff.c
+++ b/diff.c
@@ -57,10 +57,14 @@ static char diff_colors[][COLOR_MAXLEN] = {
 	GIT_COLOR_YELLOW,	/* COMMIT */
 	GIT_COLOR_BG_RED,	/* WHITESPACE */
 	GIT_COLOR_NORMAL,	/* FUNCINFO */
-	GIT_COLOR_MAGENTA,	/* OLD_MOVED */
-	GIT_COLOR_BLUE,		/* OLD_MOVED ALTERNATIVE */
-	GIT_COLOR_CYAN,		/* NEW_MOVED */
-	GIT_COLOR_YELLOW,	/* NEW_MOVED ALTERNATIVE */
+	GIT_COLOR_BOLD_MAGENTA,	/* OLD_MOVED */
+	GIT_COLOR_BOLD_BLUE,	/* OLD_MOVED ALTERNATIVE */
+	GIT_COLOR_DI,		/* OLD_MOVED_DIM */
+	GIT_COLOR_DI_IT,	/* OLD_MOVED_ALTERNATIVE_DIM */
+	GIT_COLOR_BOLD_CYAN,	/* NEW_MOVED */
+	GIT_COLOR_BOLD_YELLOW,	/* NEW_MOVED ALTERNATIVE */
+	GIT_COLOR_DI,		/* NEW_MOVED_DIM */
+	GIT_COLOR_DI_IT,	/* NEW_MOVED_ALTERNATIVE_DIM */
 };
 
 static NORETURN void die_want_option(const char *option_name)
@@ -90,10 +94,18 @@ static int parse_diff_color_slot(const char *var)
 		return DIFF_FILE_OLD_MOVED;
 	if (!strcasecmp(var, "oldmovedalternative"))
 		return DIFF_FILE_OLD_MOVED_ALT;
+	if (!strcasecmp(var, "oldmoveddimmed"))
+		return DIFF_FILE_OLD_MOVED_DIM;
+	if (!strcasecmp(var, "oldmovedalternativedimmed"))
+		return DIFF_FILE_OLD_MOVED_ALT_DIM;
 	if (!strcasecmp(var, "newmoved"))
 		return DIFF_FILE_NEW_MOVED;
 	if (!strcasecmp(var, "newmovedalternative"))
 		return DIFF_FILE_NEW_MOVED_ALT;
+	if (!strcasecmp(var, "newmoveddimmed"))
+		return DIFF_FILE_NEW_MOVED_DIM;
+	if (!strcasecmp(var, "newmovedalternativedimmed"))
+		return DIFF_FILE_NEW_MOVED_ALT_DIM;
 	return -1;
 }
 
@@ -250,6 +262,8 @@ static int parse_color_moved(const char *arg)
 		return COLOR_MOVED_PLAIN;
 	else if (!strcmp(arg, "zebra"))
 		return COLOR_MOVED_ZEBRA;
+	else if (!strcmp(arg, "dimmed_zebra"))
+		return COLOR_MOVED_ZEBRA_DIM;
 	else
 		return -1;
 }
@@ -636,6 +650,7 @@ enum diff_symbol {
 #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF	(1<<16)
 #define DIFF_SYMBOL_MOVED_LINE			(1<<17)
 #define DIFF_SYMBOL_MOVED_LINE_ZEBRA		(1<<18)
+#define DIFF_SYMBOL_MOVED_LINE_UNINTERESTING	(1<<19)
 #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)
 
 /*
@@ -889,6 +904,67 @@ static void mark_color_as_moved(struct diff_options *o,
 	free(pmb);
 }
 
+#define DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK \
+  (DIFF_SYMBOL_MOVED_LINE | DIFF_SYMBOL_MOVED_LINE_ZEBRA)
+static void dim_moved_lines(struct diff_options *o)
+{
+	int n;
+	for (n = 0; n < o->emitted_symbols->nr; n++) {
+		struct emitted_diff_symbol *prev = (n != 0) ?
+				&o->emitted_symbols->buf[n - 1] : NULL;
+		struct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];
+		struct emitted_diff_symbol *next =
+				(n < o->emitted_symbols->nr - 1) ?
+				&o->emitted_symbols->buf[n + 1] : NULL;
+
+		/* Not a plus or minus line? */
+		if (l->s != DIFF_SYMBOL_PLUS && l->s != DIFF_SYMBOL_MINUS)
+			continue;
+
+		/* Not a moved line? */
+		if (!(l->flags & DIFF_SYMBOL_MOVED_LINE))
+			continue;
+
+		/*
+		 * If prev or next are not a plus or minus line,
+		 * pretend they don't exist
+		 */
+		if (prev && prev->s != DIFF_SYMBOL_PLUS &&
+			    prev->s != DIFF_SYMBOL_MINUS)
+			prev = NULL;
+		if (next && next->s != DIFF_SYMBOL_PLUS &&
+			    next->s != DIFF_SYMBOL_MINUS)
+			next = NULL;
+
+		/* Inside a block? */
+		if ((prev &&
+		    (prev->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==
+		    (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK)) &&
+		    (next &&
+		    (next->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==
+		    (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK))) {
+			l->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;
+			continue;
+		}
+
+		/* Check if we are at an interesting bound: */
+		if (prev && (prev->flags & DIFF_SYMBOL_MOVED_LINE) &&
+		    (prev->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA) !=
+		       (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA))
+			continue;
+		if (next && (next->flags & DIFF_SYMBOL_MOVED_LINE) &&
+		    (next->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA) !=
+		       (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA))
+			continue;
+
+		/*
+		 * The boundary to prev and next are not interesting,
+		 * so this line is not interesting as a whole
+		 */
+		l->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;
+	}
+}
+
 static void emit_line_ws_markup(struct diff_options *o,
 				const char *set, const char *reset,
 				const char *line, int len, char sign,
@@ -960,24 +1036,56 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
 		emit_line(o, context, reset, line, len);
 		break;
 	case DIFF_SYMBOL_PLUS:
-		if (flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA)
+		switch (flags & (DIFF_SYMBOL_MOVED_LINE |
+				 DIFF_SYMBOL_MOVED_LINE_ZEBRA |
+				 DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {
+		case DIFF_SYMBOL_MOVED_LINE |
+		     DIFF_SYMBOL_MOVED_LINE_ZEBRA |
+		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
+			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT_DIM);
+			break;
+		case DIFF_SYMBOL_MOVED_LINE |
+		     DIFF_SYMBOL_MOVED_LINE_ZEBRA:
 			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);
-		else if (flags & DIFF_SYMBOL_MOVED_LINE)
+			break;
+		case DIFF_SYMBOL_MOVED_LINE |
+		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
+			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_DIM);
+			break;
+		case DIFF_SYMBOL_MOVED_LINE:
 			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);
-		else
+			break;
+		default:
 			set = diff_get_color_opt(o, DIFF_FILE_NEW);
+		}
 		reset = diff_get_color_opt(o, DIFF_RESET);
 		emit_line_ws_markup(o, set, reset, line, len, '+',
 				    flags & DIFF_SYMBOL_CONTENT_WS_MASK,
 				    flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);
 		break;
 	case DIFF_SYMBOL_MINUS:
-		if (flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA)
+		switch (flags & (DIFF_SYMBOL_MOVED_LINE |
+				 DIFF_SYMBOL_MOVED_LINE_ZEBRA |
+				 DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {
+		case DIFF_SYMBOL_MOVED_LINE |
+		     DIFF_SYMBOL_MOVED_LINE_ZEBRA |
+		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
+			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT_DIM);
+			break;
+		case DIFF_SYMBOL_MOVED_LINE |
+		     DIFF_SYMBOL_MOVED_LINE_ZEBRA:
 			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);
-		else if (flags & DIFF_SYMBOL_MOVED_LINE)
+			break;
+		case DIFF_SYMBOL_MOVED_LINE |
+		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
+			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_DIM);
+			break;
+		case DIFF_SYMBOL_MOVED_LINE:
 			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);
-		else
+			break;
+		default:
 			set = diff_get_color_opt(o, DIFF_FILE_OLD);
+		}
 		reset = diff_get_color_opt(o, DIFF_RESET);
 		emit_line_ws_markup(o, set, reset, line, len, '-',
 				    flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
@@ -4543,9 +4651,12 @@ int diff_opt_parse(struct diff_options *options,
 	}
 	else if (!strcmp(arg, "--no-color"))
 		options->use_color = 0;
-	else if (!strcmp(arg, "--color-moved"))
-		options->color_moved = COLOR_MOVED_ZEBRA;
-	else if (!strcmp(arg, "--no-color-moved"))
+	else if (!strcmp(arg, "--color-moved")) {
+		if (diff_color_moved_default)
+			options->color_moved = diff_color_moved_default;
+		if (options->color_moved == COLOR_MOVED_NO)
+			options->color_moved = COLOR_MOVED_ZEBRA_DIM;
+	} else if (!strcmp(arg, "--no-color-moved"))
 		options->color_moved = COLOR_MOVED_NO;
 	else if (skip_prefix(arg, "--color-moved=", &arg)) {
 		int cm = parse_color_moved(arg);
@@ -5387,6 +5498,8 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 
 			add_lines_to_move_detection(o, &add_lines, &del_lines);
 			mark_color_as_moved(o, &add_lines, &del_lines);
+			if (o->color_moved == COLOR_MOVED_ZEBRA_DIM)
+				dim_moved_lines(o);
 
 			hashmap_free(&add_lines, 0);
 			hashmap_free(&del_lines, 0);
diff --git a/diff.h b/diff.h
index 1aae8738ca..98abd75521 100644
--- a/diff.h
+++ b/diff.h
@@ -192,6 +192,7 @@ struct diff_options {
 		COLOR_MOVED_NO = 0,
 		COLOR_MOVED_PLAIN = 1,
 		COLOR_MOVED_ZEBRA = 2,
+		COLOR_MOVED_ZEBRA_DIM = 3,
 	} color_moved;
 };
 
@@ -216,8 +217,12 @@ enum color_diff {
 	DIFF_FUNCINFO = 8,
 	DIFF_FILE_OLD_MOVED = 9,
 	DIFF_FILE_OLD_MOVED_ALT = 10,
-	DIFF_FILE_NEW_MOVED = 11,
-	DIFF_FILE_NEW_MOVED_ALT = 12
+	DIFF_FILE_OLD_MOVED_DIM = 11,
+	DIFF_FILE_OLD_MOVED_ALT_DIM = 12,
+	DIFF_FILE_NEW_MOVED = 13,
+	DIFF_FILE_NEW_MOVED_ALT = 14,
+	DIFF_FILE_NEW_MOVED_DIM = 15,
+	DIFF_FILE_NEW_MOVED_ALT_DIM = 16
 };
 const char *diff_get_color(int diff_use_color, enum color_diff ix);
 #define diff_get_color_opt(o, ix) \
diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
index 1ca16435d6..ae8c686f3c 100755
--- a/t/t4015-diff-whitespace.sh
+++ b/t/t4015-diff-whitespace.sh
@@ -1179,6 +1179,130 @@ test_expect_success 'plain moved code, inside file' '
 	test_cmp expected actual
 '
 
+test_expect_success 'detect permutations inside moved code -- dimmed_zebra' '
+	git reset --hard &&
+	cat <<-\EOF >lines.txt &&
+		line 1
+		line 2
+		line 3
+		line 4
+		line 5
+		line 6
+		line 7
+		line 8
+		line 9
+		line 10
+		line 11
+		line 12
+		line 13
+		line 14
+		line 15
+		line 16
+	EOF
+	git add lines.txt &&
+	git commit -m "add poetry" &&
+	cat <<-\EOF >lines.txt &&
+		line 4
+		line 5
+		line 6
+		line 7
+		line 8
+		line 9
+		line 1
+		line 2
+		line 3
+		line 14
+		line 15
+		line 16
+		line 10
+		line 11
+		line 12
+		line 13
+	EOF
+	test_config color.diff.oldMoved "magenta" &&
+	test_config color.diff.newMoved "cyan" &&
+	test_config color.diff.oldMovedAlternative "blue" &&
+	test_config color.diff.newMovedAlternative "yellow" &&
+	test_config color.diff.oldMovedDimmed "normal magenta" &&
+	test_config color.diff.newMovedDimmed "normal cyan" &&
+	test_config color.diff.oldMovedAlternativeDimmed "normal blue" &&
+	test_config color.diff.newMovedAlternativeDimmed "normal yellow" &&
+	git diff HEAD --no-renames --color-moved=dimmed_zebra| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+	<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+	<BOLD>--- a/lines.txt<RESET>
+	<BOLD>+++ b/lines.txt<RESET>
+	<CYAN>@@ -1,16 +1,16 @@<RESET>
+	<BMAGENTA>-line 1<RESET>
+	<BMAGENTA>-line 2<RESET>
+	<BMAGENTA>-line 3<RESET>
+	 line 4<RESET>
+	 line 5<RESET>
+	 line 6<RESET>
+	 line 7<RESET>
+	 line 8<RESET>
+	 line 9<RESET>
+	<BCYAN>+<RESET><BCYAN>line 1<RESET>
+	<BCYAN>+<RESET><BCYAN>line 2<RESET>
+	<CYAN>+<RESET><CYAN>line 3<RESET>
+	<YELLOW>+<RESET><YELLOW>line 14<RESET>
+	<BYELLOW>+<RESET><BYELLOW>line 15<RESET>
+	<BYELLOW>+<RESET><BYELLOW>line 16<RESET>
+	 line 10<RESET>
+	 line 11<RESET>
+	 line 12<RESET>
+	 line 13<RESET>
+	<BMAGENTA>-line 14<RESET>
+	<BMAGENTA>-line 15<RESET>
+	<BMAGENTA>-line 16<RESET>
+	EOF
+	test_cmp expected actual
+'
+
+test_expect_success 'cmd option assumes configured colored-moved' '
+	test_config color.diff.oldMoved "magenta" &&
+	test_config color.diff.newMoved "cyan" &&
+	test_config color.diff.oldMovedAlternative "blue" &&
+	test_config color.diff.newMovedAlternative "yellow" &&
+	test_config color.diff.oldMovedDimmed "normal magenta" &&
+	test_config color.diff.newMovedDimmed "normal cyan" &&
+	test_config color.diff.oldMovedAlternativeDimmed "normal blue" &&
+	test_config color.diff.newMovedAlternativeDimmed "normal yellow" &&
+	test_config diff.colorMoved zebra &&
+	git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+	<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+	<BOLD>--- a/lines.txt<RESET>
+	<BOLD>+++ b/lines.txt<RESET>
+	<CYAN>@@ -1,16 +1,16 @@<RESET>
+	<MAGENTA>-line 1<RESET>
+	<MAGENTA>-line 2<RESET>
+	<MAGENTA>-line 3<RESET>
+	 line 4<RESET>
+	 line 5<RESET>
+	 line 6<RESET>
+	 line 7<RESET>
+	 line 8<RESET>
+	 line 9<RESET>
+	<CYAN>+<RESET><CYAN>line 1<RESET>
+	<CYAN>+<RESET><CYAN>line 2<RESET>
+	<CYAN>+<RESET><CYAN>line 3<RESET>
+	<YELLOW>+<RESET><YELLOW>line 14<RESET>
+	<YELLOW>+<RESET><YELLOW>line 15<RESET>
+	<YELLOW>+<RESET><YELLOW>line 16<RESET>
+	 line 10<RESET>
+	 line 11<RESET>
+	 line 12<RESET>
+	 line 13<RESET>
+	<MAGENTA>-line 14<RESET>
+	<MAGENTA>-line 15<RESET>
+	<MAGENTA>-line 16<RESET>
+	EOF
+	test_cmp expected actual
+'
+
 test_expect_success 'no effect from --color-moved with --word-diff' '
 	cat <<-\EOF >text.txt &&
 	Lorem Ipsum is simply dummy text of the printing and typesetting industry.
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCH 25/26] diff: document the new --color-moved setting
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (23 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 24/26] diff.c: add dimming to moved line detection Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-20  2:48                           ` [showing-off RFC/PATCH 26/26] diff.c: have a "machine parseable" move coloring Stefan Beller
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
  26 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 Documentation/config.txt       | 12 ++++++++++--
 Documentation/diff-options.txt | 27 +++++++++++++++++++++++++++
 2 files changed, 37 insertions(+), 2 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 475e874d51..29e0b9fa69 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1051,14 +1051,22 @@ This does not affect linkgit:git-format-patch[1] or the
 'git-diff-{asterisk}' plumbing commands.  Can be overridden on the
 command line with the `--color[=<when>]` option.
 
+diff.colorMoved::
+	If set moved lines in a diff are colored differently,
+	for details see '--color-moved' in linkgit:git-diff[1].
+
 color.diff.<slot>::
 	Use customized color for diff colorization.  `<slot>` specifies
 	which part of the patch to use the specified color, and is one
 	of `context` (context text - `plain` is a historical synonym),
 	`meta` (metainformation), `frag`
 	(hunk header), 'func' (function in hunk header), `old` (removed lines),
-	`new` (added lines), `commit` (commit headers), or `whitespace`
-	(highlighting whitespace errors).
+	`new` (added lines), `commit` (commit headers), `whitespace`
+	(highlighting whitespace errors), `oldMoved` (deleted lines),
+	`newMoved` (added lines), `oldMovedDimmed`, `oldMovedAlternative`,
+	`oldMovedAlternativeDimmed`, `newMovedDimmed`, `newMovedAlternative`
+	and `newMovedAlternativeDimmed` (See the '<mode>'
+	setting of '--color-moved' in linkgit:git-diff[1] for details).
 
 color.decorate.<slot>::
 	Use customized color for 'git log --decorate' output.  `<slot>` is one
diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt
index 89cc0f48de..058c8014ed 100644
--- a/Documentation/diff-options.txt
+++ b/Documentation/diff-options.txt
@@ -231,6 +231,33 @@ ifdef::git-diff[]
 endif::git-diff[]
 	It is the same as `--color=never`.
 
+--color-moved[=<mode>]::
+	Moved lines of code are colored differently.
+ifdef::git-diff[]
+	It can be changed by the `diff.colorMoved` configuration setting.
+endif::git-diff[]
+	The <mode> defaults to 'no' if the option is not given
+	and to 'dimmed_zebra' if the option with no mode is given.
+	The mode must be one of:
++
+--
+no::
+	Moved lines are not highlighted.
+plain::
+	Any line that is added in one location and was removed
+	in another location will be colored with 'color.diff.newMoved'.
+	Similarly 'color.diff.oldMoved' will be used for removed lines
+	that are added somewhere else in the diff.
+zebra::
+	Blocks of moved code are detected. The detected blocks are
+	painted using the 'color.diff.{old,new}Moved' alternating with
+	'color.diff.{old,new}MovedAlternative'.
+dimmed_zebra::
+	Similar to 'zebra', but additional dimming of uninteresting parts
+	of moved code is performed. The bordering lines of two adjacent
+	blocks are considered interesting, the rest is uninteresting.
+--
+
 --word-diff[=<mode>]::
 	Show a word diff, using the <mode> to delimit changed words.
 	By default, words are delimited by whitespace; see
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [showing-off RFC/PATCH 26/26] diff.c: have a "machine parseable" move coloring
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (24 preceding siblings ...)
  2017-06-20  2:48                           ` [PATCH 25/26] diff: document the new --color-moved setting Stefan Beller
@ 2017-06-20  2:48                           ` Stefan Beller
  2017-06-20  2:50                             ` Stefan Beller
  2017-06-21 21:51                             ` Brandon Williams
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
  26 siblings, 2 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:48 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Ævar asked for it, this is how you would do it.
(plus documentation, tests, CLI knobs, options)

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 15 +++++++++++----
 diff.h |  2 ++
 2 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/diff.c b/diff.c
index 7756f7610c..61caa057ff 100644
--- a/diff.c
+++ b/diff.c
@@ -997,6 +997,7 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
 	static const char *nneof = " No newline at end of file\n";
 	const char *context, *reset, *set, *meta, *fraginfo;
 	struct strbuf sb = STRBUF_INIT;
+	int sign;
 
 	enum diff_symbol s = eds->s;
 	const char *line = eds->line;
@@ -1058,8 +1059,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
 		default:
 			set = diff_get_color_opt(o, DIFF_FILE_NEW);
 		}
+		sign = '+';
+		if (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)
+			sign = '*';
 		reset = diff_get_color_opt(o, DIFF_RESET);
-		emit_line_ws_markup(o, set, reset, line, len, '+',
+		emit_line_ws_markup(o, set, reset, line, len, sign,
 				    flags & DIFF_SYMBOL_CONTENT_WS_MASK,
 				    flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);
 		break;
@@ -1086,8 +1090,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
 		default:
 			set = diff_get_color_opt(o, DIFF_FILE_OLD);
 		}
+		sign = '-';
+		if (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)
+			sign = '_';
 		reset = diff_get_color_opt(o, DIFF_RESET);
-		emit_line_ws_markup(o, set, reset, line, len, '-',
+		emit_line_ws_markup(o, set, reset, line, len, sign,
 				    flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
 		break;
 	case DIFF_SYMBOL_WORDS_PORCELAIN:
@@ -5475,7 +5482,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 	static struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;
 	struct diff_queue_struct *q = &diff_queued_diff;
 
-	if (o->color_moved)
+	if (o->color_moved || o->machine_readable_moves)
 		o->emitted_symbols = &esm;
 
 	for (i = 0; i < q->nr; i++) {
@@ -5485,7 +5492,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 	}
 
 	if (o->emitted_symbols) {
-		if (o->color_moved) {
+		if (o->color_moved || o->machine_readable_moves) {
 			struct hashmap add_lines, del_lines;
 			unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
 
diff --git a/diff.h b/diff.h
index 98abd75521..b6c4d7ab0f 100644
--- a/diff.h
+++ b/diff.h
@@ -194,6 +194,8 @@ struct diff_options {
 		COLOR_MOVED_ZEBRA = 2,
 		COLOR_MOVED_ZEBRA_DIM = 3,
 	} color_moved;
+
+	int machine_readable_moves;
 };
 
 void diff_emit_submodule_del(struct diff_options *o, const char *line);
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* Re: [showing-off RFC/PATCH 26/26] diff.c: have a "machine parseable" move coloring
  2017-06-20  2:48                           ` [showing-off RFC/PATCH 26/26] diff.c: have a "machine parseable" move coloring Stefan Beller
@ 2017-06-20  2:50                             ` Stefan Beller
  2017-06-23 21:43                               ` Ævar Arnfjörð Bjarmason
  2017-06-21 21:51                             ` Brandon Williams
  1 sibling, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-20  2:50 UTC (permalink / raw)
  To: Stefan Beller, Ævar Arnfjörð Bjarmason
  Cc: Brandon Williams, git, Junio C Hamano, Jonathan Tan,
	Jonathan Nieder, Michael Haggerty, Jeff King, Philip Oakley

+ Ævar, who was not part of the email where I copied all recipients
from for this series.

On Mon, Jun 19, 2017 at 7:48 PM, Stefan Beller <sbeller@google.com> wrote:
> Ævar asked for it, this is how you would do it.
> (plus documentation, tests, CLI knobs, options)
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  diff.c | 15 +++++++++++----
>  diff.h |  2 ++
>  2 files changed, 13 insertions(+), 4 deletions(-)
>
> diff --git a/diff.c b/diff.c
> index 7756f7610c..61caa057ff 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -997,6 +997,7 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
>         static const char *nneof = " No newline at end of file\n";
>         const char *context, *reset, *set, *meta, *fraginfo;
>         struct strbuf sb = STRBUF_INIT;
> +       int sign;
>
>         enum diff_symbol s = eds->s;
>         const char *line = eds->line;
> @@ -1058,8 +1059,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
>                 default:
>                         set = diff_get_color_opt(o, DIFF_FILE_NEW);
>                 }
> +               sign = '+';
> +               if (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)
> +                       sign = '*';
>                 reset = diff_get_color_opt(o, DIFF_RESET);
> -               emit_line_ws_markup(o, set, reset, line, len, '+',
> +               emit_line_ws_markup(o, set, reset, line, len, sign,
>                                     flags & DIFF_SYMBOL_CONTENT_WS_MASK,
>                                     flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);
>                 break;
> @@ -1086,8 +1090,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
>                 default:
>                         set = diff_get_color_opt(o, DIFF_FILE_OLD);
>                 }
> +               sign = '-';
> +               if (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)
> +                       sign = '_';
>                 reset = diff_get_color_opt(o, DIFF_RESET);
> -               emit_line_ws_markup(o, set, reset, line, len, '-',
> +               emit_line_ws_markup(o, set, reset, line, len, sign,
>                                     flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
>                 break;
>         case DIFF_SYMBOL_WORDS_PORCELAIN:
> @@ -5475,7 +5482,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
>         static struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;
>         struct diff_queue_struct *q = &diff_queued_diff;
>
> -       if (o->color_moved)
> +       if (o->color_moved || o->machine_readable_moves)
>                 o->emitted_symbols = &esm;
>
>         for (i = 0; i < q->nr; i++) {
> @@ -5485,7 +5492,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
>         }
>
>         if (o->emitted_symbols) {
> -               if (o->color_moved) {
> +               if (o->color_moved || o->machine_readable_moves) {
>                         struct hashmap add_lines, del_lines;
>                         unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
>
> diff --git a/diff.h b/diff.h
> index 98abd75521..b6c4d7ab0f 100644
> --- a/diff.h
> +++ b/diff.h
> @@ -194,6 +194,8 @@ struct diff_options {
>                 COLOR_MOVED_ZEBRA = 2,
>                 COLOR_MOVED_ZEBRA_DIM = 3,
>         } color_moved;
> +
> +       int machine_readable_moves;
>  };
>
>  void diff_emit_submodule_del(struct diff_options *o, const char *line);
> --
> 2.12.2.575.gb14f27f917
>

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 11/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR
  2017-06-20  2:48                           ` [PATCH 11/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR Stefan Beller
@ 2017-06-20 20:01                             ` Jonathan Tan
  2017-06-21 20:09                               ` Junio C Hamano
  0 siblings, 1 reply; 125+ messages in thread
From: Jonathan Tan @ 2017-06-20 20:01 UTC (permalink / raw)
  To: Stefan Beller; +Cc: bmwill, git, gitster, jrnieder, mhagger, peff, philipoakley

On Mon, 19 Jun 2017 19:48:01 -0700
Stefan Beller <sbeller@google.com> wrote:

> @@ -676,6 +677,14 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
>  		}
>  		emit_line(o, context, reset, line, len);
>  		break;
> +	case DIFF_SYMBOL_FILEPAIR:
> +		meta = diff_get_color_opt(o, DIFF_METAINFO);
> +		reset = diff_get_color_opt(o, DIFF_RESET);
> +		fprintf(o->file, "%s%s%s%s%s%s\n", diff_line_prefix(o), meta,
> +			flags ? "+++ " : "--- ",

I think that a constant should be defined for this flag, just like for
content lines. If not it is not immediately obvious that a flag should
be set for the +++ lines.

> +			line, reset,
> +			strchr(line, ' ') ? "\t" : "");
> +		break;
>  	default:
>  		die("BUG: unknown diff symbol");
>  	}

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 15/26] submodule.c: migrate diff output to use emit_diff_symbol
  2017-06-20  2:48                           ` [PATCH 15/26] submodule.c: migrate diff output to use emit_diff_symbol Stefan Beller
@ 2017-06-20 20:09                             ` Jonathan Tan
  0 siblings, 0 replies; 125+ messages in thread
From: Jonathan Tan @ 2017-06-20 20:09 UTC (permalink / raw)
  To: Stefan Beller; +Cc: bmwill, git, gitster, jrnieder, mhagger, peff, philipoakley

On Mon, 19 Jun 2017 19:48:05 -0700
Stefan Beller <sbeller@google.com> wrote:

> As the submodule process is no longer attached to the same stdout as
> the superprojects process, we need to pass coloring explicitly.

I found this confusing - what difference does the stdout make? If they
were the same stdout, one process could set a color for the other
process to follow, but it wouldn't be able to affect color changes
inside the output, right?

> Remove the colors from the function signatures, as all the coloring
> decisions will be made either inside the child process or the final
> emit_diff_symbol.

This seems to be of the same pattern as the others. I would write this
as:

    Create diff_emit_submodule_.* functions that make its own coloring
    decisions and migrate submodule.c to use them. This centralizes
    diff output, including coloring information, in one file.

or something like that.

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 22/26] diff.c: color moved lines differently
  2017-06-20  2:48                           ` [PATCH 22/26] diff.c: color moved lines differently Stefan Beller
@ 2017-06-20 20:13                             ` Jonathan Tan
  2017-06-20 20:57                               ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Jonathan Tan @ 2017-06-20 20:13 UTC (permalink / raw)
  To: Stefan Beller; +Cc: bmwill, git, gitster, jrnieder, mhagger, peff, philipoakley

I just glanced through this file, because it seems similar to the
versions I have previously reviewed.

I'll skip patches 23 onwards in this round of review because (i) I would
be happy if just patches 1-22 were included in the tree and (ii) those
patches might end up changing anyway because of review comments in the
prior patches.

On Mon, 19 Jun 2017 19:48:12 -0700
Stefan Beller <sbeller@google.com> wrote:

> +/* Find blocks of moved code, delegate actual coloring decision to helper */
> +static void mark_color_as_moved(struct diff_options *o,
> +				struct hashmap *add_lines,
> +				struct hashmap *del_lines)
> +{

[snip]

> +		if (flipped_block)
> +			l->flags |= DIFF_SYMBOL_MOVED_LINE_ZEBRA;

This should probably be DIFF_SYMBOL_MOVED_LINE_ALT. "Zebra" refers to
both the stripes, not just the alternate stripe.

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 22/26] diff.c: color moved lines differently
  2017-06-20 20:13                             ` Jonathan Tan
@ 2017-06-20 20:57                               ` Stefan Beller
  0 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-20 20:57 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: Brandon Williams, git, Junio C Hamano, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

On Tue, Jun 20, 2017 at 1:13 PM, Jonathan Tan <jonathantanmy@google.com> wrote:
> I just glanced through this file, because it seems similar to the
> versions I have previously reviewed.

Yes, this has not changed much.
The thing that took so long were patches 1-20 to do.
In these later patches (22) the UX was changed slightly.
The mode is zebra instead of alternating for example

> I'll skip patches 23 onwards in this round of review because (i) I would
> be happy if just patches 1-22 were included in the tree

patch 22 just introduced the zebra mode, which contains all
information necessary, the later "dimming" is purely
UI excitement, no further information added.

In former series, I had documentation at each
patch that added a new mode, now I rolled all of
documentation in patch 25. I will add the basic docs
to that patch in a reroll.

> and (ii) those
> patches might end up changing anyway because of review comments in the
> prior patches.

plain, dimming, docs, and this RFC for machine parsable output,
ok with me.

So I'll focus on the first 22 patches.

>
> On Mon, 19 Jun 2017 19:48:12 -0700
> Stefan Beller <sbeller@google.com> wrote:
>
>> +/* Find blocks of moved code, delegate actual coloring decision to helper */
>> +static void mark_color_as_moved(struct diff_options *o,
>> +                             struct hashmap *add_lines,
>> +                             struct hashmap *del_lines)
>> +{
>
> [snip]
>
>> +             if (flipped_block)
>> +                     l->flags |= DIFF_SYMBOL_MOVED_LINE_ZEBRA;
>
> This should probably be DIFF_SYMBOL_MOVED_LINE_ALT. "Zebra" refers to
> both the stripes, not just the alternate stripe.


ok

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol
  2017-06-20  2:47                           ` [PATCH 04/26] diff.c: introduce emit_diff_symbol Stefan Beller
@ 2017-06-21 19:36                             ` Junio C Hamano
  2017-06-21 19:46                               ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Junio C Hamano @ 2017-06-21 19:36 UTC (permalink / raw)
  To: Stefan Beller
  Cc: bmwill, git, jonathantanmy, jrnieder, mhagger, peff, philipoakley

Stefan Beller <sbeller@google.com> writes:

> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  diff.c | 22 +++++++++++++++++++---
>  1 file changed, 19 insertions(+), 3 deletions(-)
>
> diff --git a/diff.c b/diff.c
> index 2f9722b382..89466018e5 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -559,6 +559,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
>  	emit_line_0(o, set, reset, line[0], line+1, len-1);
>  }
>  
> +enum diff_symbol {
> +	DIFF_SYMBOL_SEPARATOR,

Drop the last comma from enum?

> +static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
> +			     const char *line, int len)
> +{
> +	switch (s) {
> +	case DIFF_SYMBOL_SEPARATOR:
> +		fprintf(o->file, "%s%c",
> +			diff_line_prefix(o),
> +			o->line_termination);
> +		break;

As the first patch in the "diff-symbol" subseries of this topic,
this change must seriously be justified.  Why is it so important
that a printing of an empty line must be moved to a helper function,
which later will gain ability to show other kind of lines?

> +	default:
> +		die("BUG: unknown diff symbol");
> +	}
> +}
> +
>  static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)
>  {
>  	if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
> @@ -4833,9 +4851,7 @@ void diff_flush(struct diff_options *options)
>  
>  	if (output_format & DIFF_FORMAT_PATCH) {
>  		if (separator) {
> -			fprintf(options->file, "%s%c",
> -				diff_line_prefix(options),
> -				options->line_termination);
> +			emit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);
>  			if (options->stat_sep) {
>  				/* attach patch instead of inline */
>  				fputs(options->stat_sep, options->file);

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol
  2017-06-21 19:36                             ` Junio C Hamano
@ 2017-06-21 19:46                               ` Stefan Beller
  2017-06-21 20:26                                 ` Junio C Hamano
                                                   ` (2 more replies)
  0 siblings, 3 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-21 19:46 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Brandon Williams, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

On Wed, Jun 21, 2017 at 12:36 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>>  diff.c | 22 +++++++++++++++++++---
>>  1 file changed, 19 insertions(+), 3 deletions(-)
>>
>> diff --git a/diff.c b/diff.c
>> index 2f9722b382..89466018e5 100644
>> --- a/diff.c
>> +++ b/diff.c
>> @@ -559,6 +559,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
>>       emit_line_0(o, set, reset, line[0], line+1, len-1);
>>  }
>>
>> +enum diff_symbol {
>> +     DIFF_SYMBOL_SEPARATOR,
>
> Drop the last comma from enum?

I looked through out code base and for enums this is
actually strictly enforced, so I guess I have to play
by the rules here as I do not want to be the first
to deviate from an upheld standard.

This will be painful though as the next ~20 patches
add more symbols mostly at the end, maybe I need
to restructure that such that the last symbol stays the same
throughout the series. Thanks for that thought.

>
>> +static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
>> +                          const char *line, int len)
>> +{
>> +     switch (s) {
>> +     case DIFF_SYMBOL_SEPARATOR:
>> +             fprintf(o->file, "%s%c",
>> +                     diff_line_prefix(o),
>> +                     o->line_termination);
>> +             break;
>
> As the first patch in the "diff-symbol" subseries of this topic,
> this change must seriously be justified.  Why is it so important
> that a printing of an empty line must be moved to a helper function,
> which later will gain ability to show other kind of lines?

Ah yes. This got lost in comparison to the currently queued series with
diff_lines. The justification for the change was in the buffer patch,
but now we need to have the justification here.

In the old series, I had copied the same text in all these
refactoring patches, but thought to delete them in this series. The first
refactoring patch makes sense though.

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol
  2017-06-20  2:47                           ` [PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol Stefan Beller
@ 2017-06-21 20:05                             ` Junio C Hamano
  2017-06-22 23:30                               ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Junio C Hamano @ 2017-06-21 20:05 UTC (permalink / raw)
  To: Stefan Beller
  Cc: bmwill, git, jonathantanmy, jrnieder, mhagger, peff, philipoakley

Stefan Beller <sbeller@google.com> writes:

>  static void emit_add_line(const char *reset,
>  			  struct emit_callback *ecbdata,
>  			  const char *line, int len)
>  {
> -	emit_line_checked(reset, ecbdata, line, len,
> -			  DIFF_FILE_NEW, WSEH_NEW, '+');
> +	unsigned flags = WSEH_NEW | ecbdata->ws_rule;
> +	if (new_blank_line_at_eof(ecbdata, line, len))
> +		flags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;
> +
> +	emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);
>  }

This is a bit unsatisfactory that the original code didn't even have
to make a call to new_blank_line_at_eof() at all when we know we are
not checking for / coloring whitespace errors, but the function is
called unconditionally in the new code.

> diff --git a/diff.h b/diff.h
> index 5be1ee77a7..8483ca0991 100644
> --- a/diff.h
> +++ b/diff.h
> @@ -148,9 +148,9 @@ struct diff_options {
>  	int abbrev;
>  	int ita_invisible_in_index;
>  /* white-space error highlighting */
> -#define WSEH_NEW 1
> -#define WSEH_CONTEXT 2
> -#define WSEH_OLD 4
> +#define WSEH_NEW (1<<12)
> +#define WSEH_CONTEXT (1<<13)
> +#define WSEH_OLD (1<<14)

Hopefully this is a safe conversion, as everything should come from
parse_ws_error_highlight() that uses these constants.

But this is brittle; there must be some hint to future readers of
this code why these bits begin at 12th (it is linked to 07777 we saw
earlier); otherwise when they change something that needs to widen
that 07777, they will forget to shift these up, which would screw
up everything that is carefully laid out here.



^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 11/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR
  2017-06-20 20:01                             ` Jonathan Tan
@ 2017-06-21 20:09                               ` Junio C Hamano
  2017-06-22 23:59                                 ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Junio C Hamano @ 2017-06-21 20:09 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: Stefan Beller, bmwill, git, jrnieder, mhagger, peff, philipoakley

Jonathan Tan <jonathantanmy@google.com> writes:

> On Mon, 19 Jun 2017 19:48:01 -0700
> Stefan Beller <sbeller@google.com> wrote:
>
>> @@ -676,6 +677,14 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
>>  		}
>>  		emit_line(o, context, reset, line, len);
>>  		break;
>> +	case DIFF_SYMBOL_FILEPAIR:
>> +		meta = diff_get_color_opt(o, DIFF_METAINFO);
>> +		reset = diff_get_color_opt(o, DIFF_RESET);
>> +		fprintf(o->file, "%s%s%s%s%s%s\n", diff_line_prefix(o), meta,
>> +			flags ? "+++ " : "--- ",
>
> I think that a constant should be defined for this flag, just like for
> content lines. If not it is not immediately obvious that a flag should
> be set for the +++ lines.

True.

>> +			line, reset,
>> +			strchr(line, ' ') ? "\t" : "");
>> +		break;

Also this is somewhat misnamed in that a call to it only deals with
one half of a filepair, iow, the caller must call it twice to show a
single filepair.

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 13/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES
  2017-06-20  2:48                           ` [PATCH 13/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES Stefan Beller
@ 2017-06-21 20:13                             ` Junio C Hamano
  2017-06-21 20:47                               ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Junio C Hamano @ 2017-06-21 20:13 UTC (permalink / raw)
  To: Stefan Beller
  Cc: bmwill, git, jonathantanmy, jrnieder, mhagger, peff, philipoakley

Stefan Beller <sbeller@google.com> writes:

> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  diff.c | 21 ++++++++++++++++-----
>  1 file changed, 16 insertions(+), 5 deletions(-)
>
> diff --git a/diff.c b/diff.c
> index 95f11ea86f..e56962b844 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -572,6 +572,7 @@ enum diff_symbol {
>  	DIFF_SYMBOL_WORDS,
>  	DIFF_SYMBOL_FILEPAIR,
>  	DIFF_SYMBOL_HEADER,
> +	DIFF_SYMBOL_BINARY_FILES,
>  };
>  /*
>   * Flags for content lines:
> @@ -689,6 +690,10 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
>  	case DIFF_SYMBOL_HEADER:
>  		fprintf(o->file, "%s", line);
>  		break;
> +	case DIFF_SYMBOL_BINARY_FILES:
> +		fprintf(o->file, "%sBinary files %s differ\n",
> +			diff_line_prefix(o), line);
> +		break;

It is curious why the "line" is defined to be "A and B" here.  It
could have been defined to be the whole string "Binary files A and B
differ" or even with the terminating LF.

And with that it may have been able to share concepts with the
"HEADER" we see above.  SYMBOL_HEADER becomes "oneline without
prefix", and this one (updated to make the caller prepare the whole
sentence) becomes "oneline with prefix".

> @@ -2549,8 +2555,10 @@ static void builtin_diff(const char *name_a,
>  			}
>  			emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
>  					 header.buf, header.len, 0);
> -			fprintf(o->file, "%sBinary files %s and %s differ\n",
> -				line_prefix, lbl[0], lbl[1]);
> +			strbuf_addf(&sb, "%s and %s", lbl[0], lbl[1]);
> +			emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
> +					 sb.buf, sb.len, 0);
> +			strbuf_release(&sb);
>  			goto free_ab_and_return;
>  		}
>  		if (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)
> @@ -2567,9 +2575,12 @@ static void builtin_diff(const char *name_a,
>  		strbuf_reset(&header);
>  		if (DIFF_OPT_TST(o, BINARY))
>  			emit_binary_diff(o->file, &mf1, &mf2, line_prefix);
> -		else
> -			fprintf(o->file, "%sBinary files %s and %s differ\n",
> -				line_prefix, lbl[0], lbl[1]);
> +		else {
> +			strbuf_addf(&sb, "%s and %s", lbl[0], lbl[1]);
> +			emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
> +					 sb.buf, sb.len, 0);
> +			strbuf_release(&sb);
> +		}
>  		o->found_changes = 1;
>  	} else {
>  		/* Crazy xdl interfaces.. */

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 16/26] diff.c: convert emit_binary_diff_body to use emit_diff_symbol
  2017-06-20  2:48                           ` [PATCH 16/26] diff.c: convert emit_binary_diff_body " Stefan Beller
@ 2017-06-21 20:16                             ` Junio C Hamano
  0 siblings, 0 replies; 125+ messages in thread
From: Junio C Hamano @ 2017-06-21 20:16 UTC (permalink / raw)
  To: Stefan Beller
  Cc: bmwill, git, jonathantanmy, jrnieder, mhagger, peff, philipoakley

Stefan Beller <sbeller@google.com> writes:

> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  diff.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++-----------------
>  1 file changed, 49 insertions(+), 17 deletions(-)
>
> diff --git a/diff.c b/diff.c
> index 42a9020d95..e6ade5fde0 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -573,6 +573,11 @@ enum diff_symbol {
>  	DIFF_SYMBOL_FILEPAIR,
>  	DIFF_SYMBOL_HEADER,
>  	DIFF_SYMBOL_BINARY_FILES,
> +	DIFF_SYMBOL_BINARY_DIFF_HEADER,
> +	DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,
> +	DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,
> +	DIFF_SYMBOL_BINARY_DIFF_BODY,
> +	DIFF_SYMBOL_BINARY_DIFF_FOOTER,
>  	DIFF_SYMBOL_REWRITE_DIFF,
>  	DIFF_SYMBOL_SUBMODULE_ADD,
>  	DIFF_SYMBOL_SUBMODULE_DEL,
> @@ -581,6 +586,7 @@ enum diff_symbol {
>  	DIFF_SYMBOL_SUBMODULE_HEADER,
>  	DIFF_SYMBOL_SUBMODULE_ERROR,
>  	DIFF_SYMBOL_SUBMODULE_PIPETHROUGH,
> +

Line noise?

>  };

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 24/26] diff.c: add dimming to moved line detection
  2017-06-20  2:48                           ` [PATCH 24/26] diff.c: add dimming to moved line detection Stefan Beller
@ 2017-06-21 20:23                             ` Junio C Hamano
  0 siblings, 0 replies; 125+ messages in thread
From: Junio C Hamano @ 2017-06-21 20:23 UTC (permalink / raw)
  To: Stefan Beller
  Cc: bmwill, git, jonathantanmy, jrnieder, mhagger, peff, philipoakley

Stefan Beller <sbeller@google.com> writes:

> Any lines inside a moved block of code are not interesting. Boundaries
> of blocks are only interesting if they are next to another block of moved
> code.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  color.h                    |   2 +
>  diff.c                     | 139 ++++++++++++++++++++++++++++++++++++++++-----
>  diff.h                     |   9 ++-
>  t/t4015-diff-whitespace.sh | 124 ++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 259 insertions(+), 15 deletions(-)
>
> diff --git a/color.h b/color.h
> index 90627650fc..0e091b0cf5 100644
> --- a/color.h
> +++ b/color.h
> @@ -42,6 +42,8 @@ struct strbuf;
>  #define GIT_COLOR_BG_BLUE	"\033[44m"
>  #define GIT_COLOR_BG_MAGENTA	"\033[45m"
>  #define GIT_COLOR_BG_CYAN	"\033[46m"
> +#define GIT_COLOR_DI		"\033[2m"
> +#define GIT_COLOR_DI_IT		"\033[2;3m"

I can and I think most naive readers understand what BOLD, BG and
BLUE means, but I do not think the same can be said to DI and IT.
If you meant FAINT and ITALIC, it is better to spell them out.  You
do not have to type them to too many places anyway.


^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol
  2017-06-21 19:46                               ` Stefan Beller
@ 2017-06-21 20:26                                 ` Junio C Hamano
  2017-06-21 21:13                                 ` Junio C Hamano
  2017-06-21 21:45                                 ` Brandon Williams
  2 siblings, 0 replies; 125+ messages in thread
From: Junio C Hamano @ 2017-06-21 20:26 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Brandon Williams, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

Stefan Beller <sbeller@google.com> writes:

> This will be painful though as the next ~20 patches
> add more symbols mostly at the end.

> , maybe I need
> to restructure that such that the last symbol stays the same
> throughout the series. Thanks for that thought.

Yes, this mostly-no-op one added in the first patch can be kept
always at the end.


^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 13/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES
  2017-06-21 20:13                             ` Junio C Hamano
@ 2017-06-21 20:47                               ` Stefan Beller
  0 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-21 20:47 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Brandon Williams, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

On Wed, Jun 21, 2017 at 1:13 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>>  diff.c | 21 ++++++++++++++++-----
>>  1 file changed, 16 insertions(+), 5 deletions(-)
>>
>> diff --git a/diff.c b/diff.c
>> index 95f11ea86f..e56962b844 100644
>> --- a/diff.c
>> +++ b/diff.c
>> @@ -572,6 +572,7 @@ enum diff_symbol {
>>       DIFF_SYMBOL_WORDS,
>>       DIFF_SYMBOL_FILEPAIR,
>>       DIFF_SYMBOL_HEADER,
>> +     DIFF_SYMBOL_BINARY_FILES,
>>  };
>>  /*
>>   * Flags for content lines:
>> @@ -689,6 +690,10 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
>>       case DIFF_SYMBOL_HEADER:
>>               fprintf(o->file, "%s", line);
>>               break;
>> +     case DIFF_SYMBOL_BINARY_FILES:
>> +             fprintf(o->file, "%sBinary files %s differ\n",
>> +                     diff_line_prefix(o), line);
>> +             break;
>
> It is curious why the "line" is defined to be "A and B" here.  It
> could have been defined to be the whole string "Binary files A and B
> differ" or even with the terminating LF.
>
> And with that it may have been able to share concepts with the
> "HEADER" we see above.  SYMBOL_HEADER becomes "oneline without
> prefix", and this one (updated to make the caller prepare the whole
> sentence) becomes "oneline with prefix".

I thought about saving memory in the buffered operation,
but for simplicity it may be better to make them the same.
Then they can even share the implementation.
With that thought, I'll check more different symbols to share
the same implementation.

>
>> @@ -2549,8 +2555,10 @@ static void builtin_diff(const char *name_a,
>>                       }
>>                       emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
>>                                        header.buf, header.len, 0);
>> -                     fprintf(o->file, "%sBinary files %s and %s differ\n",
>> -                             line_prefix, lbl[0], lbl[1]);
>> +                     strbuf_addf(&sb, "%s and %s", lbl[0], lbl[1]);
>> +                     emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
>> +                                      sb.buf, sb.len, 0);
>> +                     strbuf_release(&sb);
>>                       goto free_ab_and_return;
>>               }
>>               if (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)
>> @@ -2567,9 +2575,12 @@ static void builtin_diff(const char *name_a,
>>               strbuf_reset(&header);
>>               if (DIFF_OPT_TST(o, BINARY))
>>                       emit_binary_diff(o->file, &mf1, &mf2, line_prefix);
>> -             else
>> -                     fprintf(o->file, "%sBinary files %s and %s differ\n",
>> -                             line_prefix, lbl[0], lbl[1]);
>> +             else {
>> +                     strbuf_addf(&sb, "%s and %s", lbl[0], lbl[1]);
>> +                     emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
>> +                                      sb.buf, sb.len, 0);
>> +                     strbuf_release(&sb);
>> +             }
>>               o->found_changes = 1;
>>       } else {
>>               /* Crazy xdl interfaces.. */

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol
  2017-06-21 19:46                               ` Stefan Beller
  2017-06-21 20:26                                 ` Junio C Hamano
@ 2017-06-21 21:13                                 ` Junio C Hamano
  2017-06-21 21:23                                   ` Stefan Beller
  2017-06-21 21:45                                 ` Brandon Williams
  2 siblings, 1 reply; 125+ messages in thread
From: Junio C Hamano @ 2017-06-21 21:13 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Brandon Williams, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

Stefan Beller <sbeller@google.com> writes:

>>>
>>> +enum diff_symbol {
>>> +     DIFF_SYMBOL_SEPARATOR,
>>
>> Drop the last comma from enum?
>
> I looked through out code base and for enums this is
> actually strictly enforced, so I guess I have to play
> by the rules here as I do not want to be the first
> to deviate from an upheld standard.

You sound like you are trying to find an excuse "not to play by the
rules".  Don't.



^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol
  2017-06-21 21:13                                 ` Junio C Hamano
@ 2017-06-21 21:23                                   ` Stefan Beller
  2017-06-21 21:43                                     ` Junio C Hamano
  0 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-21 21:23 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Brandon Williams, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

On Wed, Jun 21, 2017 at 2:13 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>>>>
>>>> +enum diff_symbol {
>>>> +     DIFF_SYMBOL_SEPARATOR,
>>>
>>> Drop the last comma from enum?
>>
>> I looked through out code base and for enums this is
>> actually strictly enforced, so I guess I have to play
>> by the rules here as I do not want to be the first
>> to deviate from an upheld standard.
>
> You sound like you are trying to find an excuse "not to play by the
> rules".  Don't.
>

When the rules are not yielding best results, adapting the rules
should be considered. And having no comma at the last element
produces review noise when working in a line oriented format, such
as diffs. (Add another element at the end, and then you have to
have the diff that just changes the comma)

  C99 allows a trailing comma to follow the last enumeration
  constant initializer within an enumeration type declaration,
  similar to structure member initialization lists.
  C99: §6.7.2.2

  For struct member lists we already do that, e.g.
  builtin/help.c: struct common_guides[]
  builtin/repack.c: struct exts[]
  ...

Yet, I will construct this series in a resend to not
have this issue.

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 17/26] diff.c: convert show_stats to use emit_diff_symbol
  2017-06-20  2:48                           ` [PATCH 17/26] diff.c: convert show_stats " Stefan Beller
@ 2017-06-21 21:39                             ` Brandon Williams
  2017-06-21 22:16                               ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Brandon Williams @ 2017-06-21 21:39 UTC (permalink / raw)
  To: Stefan Beller
  Cc: git, gitster, jonathantanmy, jrnieder, mhagger, peff, philipoakley

On 06/19, Stefan Beller wrote:
> We call print_stat_summary from builtin/apply, so we still
> need the version with a file pointer, so introduce
> print_stat_summary_0 that uses emit_string machinery and
> keep print_stat_summary with the same arguments around.
> 
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  diff.c | 118 +++++++++++++++++++++++++++++++++++++++++------------------------
>  diff.h |   4 +--
>  2 files changed, 77 insertions(+), 45 deletions(-)
> 
> diff --git a/diff.c b/diff.c
> index e6ade5fde0..0a5ff2500f 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -586,7 +586,10 @@ enum diff_symbol {
>  	DIFF_SYMBOL_SUBMODULE_HEADER,
>  	DIFF_SYMBOL_SUBMODULE_ERROR,
>  	DIFF_SYMBOL_SUBMODULE_PIPETHROUGH,
> -
> +	DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,
> +	DIFF_SYMBOL_STATS_SUMMARY_ABBREV,
> +	DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
> +	DIFF_SYMBOL_STATS_LINE,
>  };
>  /*
>   * Flags for content lines:
> @@ -628,6 +631,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
>  {
>  	static const char *nneof = " No newline at end of file\n";
>  	const char *context, *reset, *set, *meta, *fraginfo;
> +	struct strbuf sb = STRBUF_INIT;
>  	switch (s) {
>  	case DIFF_SYMBOL_SEPARATOR:
>  		fprintf(o->file, "%s%c",
> @@ -756,9 +760,22 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
>  	case DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:
>  		emit_line(o, "", "", line, len);
>  		break;
> +	case DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:
> +		fprintf(o->file, " 0 files changed\n");

Why is this one a fprintf call instead of an emit_line call?  I'm sure
this is intended It just stands out to me.

> +		break;
> +	case DIFF_SYMBOL_STATS_SUMMARY_ABBREV:
> +		emit_line(o, "", "", " ...\n", strlen(" ...\n"));
> +		break;
> +	case DIFF_SYMBOL_STATS_LINE:
> +		emit_line(o, "", "", line, len);
> +		break;
> +	case DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:
> +		emit_line(o, "", "", line, len);
> +		break;
>  	default:
>  		die("BUG: unknown diff symbol");
>  	}
> +	strbuf_release(&sb);
>  }
>  
>  void diff_emit_submodule_del(struct diff_options *o, const char *line)
> @@ -1712,20 +1729,14 @@ static int scale_linear(int it, int width, int max_change)
>  	return 1 + (it * (width - 1) / max_change);
>  }
>  
> -static void show_name(FILE *file,
> -		      const char *prefix, const char *name, int len)
> -{
> -	fprintf(file, " %s%-*s |", prefix, len, name);
> -}
> -
> -static void show_graph(FILE *file, char ch, int cnt, const char *set, const char *reset)
> +static void show_graph(struct strbuf *out, char ch, int cnt,
> +		       const char *set, const char *reset)
>  {
>  	if (cnt <= 0)
>  		return;
> -	fprintf(file, "%s", set);
> -	while (cnt--)
> -		putc(ch, file);
> -	fprintf(file, "%s", reset);
> +	strbuf_addstr(out, set);
> +	strbuf_addchars(out, ch, cnt);
> +	strbuf_addstr(out, reset);
>  }
>  
>  static void fill_print_name(struct diffstat_file *file)
> @@ -1749,14 +1760,16 @@ static void fill_print_name(struct diffstat_file *file)
>  	file->print_name = pname;
>  }
>  
> -int print_stat_summary(FILE *fp, int files, int insertions, int deletions)
> +static void print_stat_summary_inserts_deletes(struct diff_options *options,
> +		int files, int insertions, int deletions)
>  {
>  	struct strbuf sb = STRBUF_INIT;
> -	int ret;
>  
>  	if (!files) {
>  		assert(insertions == 0 && deletions == 0);
> -		return fprintf(fp, "%s\n", " 0 files changed");
> +		emit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,
> +				 NULL, 0, 0);
> +		return;
>  	}
>  
>  	strbuf_addf(&sb,
> @@ -1783,9 +1796,19 @@ int print_stat_summary(FILE *fp, int files, int insertions, int deletions)
>  			    deletions);
>  	}
>  	strbuf_addch(&sb, '\n');
> -	ret = fputs(sb.buf, fp);
> +	emit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
> +			 sb.buf, sb.len, 0);
>  	strbuf_release(&sb);
> -	return ret;
> +}
> +
> +void print_stat_summary(FILE *fp, int files,
> +			int insertions, int deletions)
> +{
> +	struct diff_options o;
> +	memset(&o, 0, sizeof(o));
> +	o.file = fp;
> +
> +	print_stat_summary_inserts_deletes(&o, files, insertions, deletions);
>  }
>  
>  static void show_stats(struct diffstat_t *data, struct diff_options *options)
> @@ -1795,13 +1818,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
>  	int total_files = data->nr, count;
>  	int width, name_width, graph_width, number_width = 0, bin_width = 0;
>  	const char *reset, *add_c, *del_c;
> -	const char *line_prefix = "";
>  	int extra_shown = 0;
> +	const char *line_prefix = diff_line_prefix(options);
> +	struct strbuf out = STRBUF_INIT;
>  
>  	if (data->nr == 0)
>  		return;
>  
> -	line_prefix = diff_line_prefix(options);
>  	count = options->stat_count ? options->stat_count : data->nr;
>  
>  	reset = diff_get_color_opt(options, DIFF_RESET);
> @@ -1955,26 +1978,32 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
>  		}
>  
>  		if (file->is_binary) {
> -			fprintf(options->file, "%s", line_prefix);
> -			show_name(options->file, prefix, name, len);
> -			fprintf(options->file, " %*s", number_width, "Bin");
> +			strbuf_addf(&out, " %s%-*s |", prefix, len, name);
> +			strbuf_addf(&out, " %*s", number_width, "Bin");
>  			if (!added && !deleted) {
> -				putc('\n', options->file);
> +				strbuf_addch(&out, '\n');
> +				emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
> +						 out.buf, out.len, 0);
> +				strbuf_reset(&out);
>  				continue;
>  			}
> -			fprintf(options->file, " %s%"PRIuMAX"%s",
> +			strbuf_addf(&out, " %s%"PRIuMAX"%s",
>  				del_c, deleted, reset);
> -			fprintf(options->file, " -> ");
> -			fprintf(options->file, "%s%"PRIuMAX"%s",
> +			strbuf_addstr(&out, " -> ");
> +			strbuf_addf(&out, "%s%"PRIuMAX"%s",
>  				add_c, added, reset);
> -			fprintf(options->file, " bytes");
> -			fprintf(options->file, "\n");
> +			strbuf_addstr(&out, " bytes\n");
> +			emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
> +					 out.buf, out.len, 0);
> +			strbuf_reset(&out);
>  			continue;
>  		}
>  		else if (file->is_unmerged) {
> -			fprintf(options->file, "%s", line_prefix);
> -			show_name(options->file, prefix, name, len);
> -			fprintf(options->file, " Unmerged\n");
> +			strbuf_addf(&out, " %s%-*s |", prefix, len, name);
> +			strbuf_addstr(&out, " Unmerged\n");
> +			emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
> +					 out.buf, out.len, 0);
> +			strbuf_reset(&out);
>  			continue;
>  		}
>  
> @@ -1997,14 +2026,16 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
>  				add = total - del;
>  			}
>  		}
> -		fprintf(options->file, "%s", line_prefix);
> -		show_name(options->file, prefix, name, len);
> -		fprintf(options->file, " %*"PRIuMAX"%s",
> +		strbuf_addf(&out, " %s%-*s |", prefix, len, name);
> +		strbuf_addf(&out, " %*"PRIuMAX"%s",
>  			number_width, added + deleted,
>  			added + deleted ? " " : "");
> -		show_graph(options->file, '+', add, add_c, reset);
> -		show_graph(options->file, '-', del, del_c, reset);
> -		fprintf(options->file, "\n");
> +		show_graph(&out, '+', add, add_c, reset);
> +		show_graph(&out, '-', del, del_c, reset);
> +		strbuf_addch(&out, '\n');
> +		emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
> +				 out.buf, out.len, 0);
> +		strbuf_reset(&out);
>  	}
>  
>  	for (i = 0; i < data->nr; i++) {
> @@ -2025,11 +2056,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
>  		if (i < count)
>  			continue;
>  		if (!extra_shown)
> -			fprintf(options->file, "%s ...\n", line_prefix);
> +			emit_diff_symbol(options,
> +					 DIFF_SYMBOL_STATS_SUMMARY_ABBREV,
> +					 NULL, 0, 0);
>  		extra_shown = 1;
>  	}
> -	fprintf(options->file, "%s", line_prefix);
> -	print_stat_summary(options->file, total_files, adds, dels);
> +
> +	print_stat_summary_inserts_deletes(options, total_files, adds, dels);
>  }
>  
>  static void show_shortstats(struct diffstat_t *data, struct diff_options *options)
> @@ -2041,7 +2074,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option
>  
>  	for (i = 0; i < data->nr; i++) {
>  		int added = data->files[i]->added;
> -		int deleted= data->files[i]->deleted;
> +		int deleted = data->files[i]->deleted;
>  
>  		if (data->files[i]->is_unmerged ||
>  		    (!data->files[i]->is_interesting && (added + deleted == 0))) {
> @@ -2051,8 +2084,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option
>  			dels += deleted;
>  		}
>  	}
> -	fprintf(options->file, "%s", diff_line_prefix(options));
> -	print_stat_summary(options->file, total_files, adds, dels);
> +	print_stat_summary_inserts_deletes(options, total_files, adds, dels);
>  }
>  
>  static void show_numstat(struct diffstat_t *data, struct diff_options *options)
> diff --git a/diff.h b/diff.h
> index 2ee0ef3908..b73f7a1e75 100644
> --- a/diff.h
> +++ b/diff.h
> @@ -398,8 +398,8 @@ extern int parse_rename_score(const char **cp_p);
>  
>  extern long parse_algorithm_value(const char *value);
>  
> -extern int print_stat_summary(FILE *fp, int files,
> -			      int insertions, int deletions);
> +extern void print_stat_summary(FILE *fp, int files,
> +			       int insertions, int deletions);
>  extern void setup_diff_pager(struct diff_options *);
>  
>  #endif /* DIFF_H */
> -- 
> 2.12.2.575.gb14f27f917
> 

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol
  2017-06-21 21:23                                   ` Stefan Beller
@ 2017-06-21 21:43                                     ` Junio C Hamano
  2017-06-21 21:51                                       ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Junio C Hamano @ 2017-06-21 21:43 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Brandon Williams, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

Stefan Beller <sbeller@google.com> writes:

>>> I looked through out code base and for enums this is
>>> actually strictly enforced, so I guess I have to play
>>> by the rules here as I do not want to be the first
>>> to deviate from an upheld standard.
>>
>> You sound like you are trying to find an excuse "not to play by the
>> rules".  Don't.
>
> When the rules are not yielding best results, adapting the rules
> should be considered.

I do not think anybody is saying that it is unreasonable to have a
wish that it would be wonderful if we could use C99 features.  

Trailing comma at the end of enum is the least interesting addition
(it is a fix for a previous editions' brown-paper-bag mistake).  I'd
be delighted if we can drop support for older compilers and start
using designated initializers, for example.

Finding other's violation of the current coding guideline is *not*
the right way to promote for that future, however.


^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol
  2017-06-21 19:46                               ` Stefan Beller
  2017-06-21 20:26                                 ` Junio C Hamano
  2017-06-21 21:13                                 ` Junio C Hamano
@ 2017-06-21 21:45                                 ` Brandon Williams
  2017-06-21 21:52                                   ` Junio C Hamano
  2 siblings, 1 reply; 125+ messages in thread
From: Brandon Williams @ 2017-06-21 21:45 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Junio C Hamano, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

On 06/21, Stefan Beller wrote:
> On Wed, Jun 21, 2017 at 12:36 PM, Junio C Hamano <gitster@pobox.com> wrote:
> > Stefan Beller <sbeller@google.com> writes:
> >
> >> Signed-off-by: Stefan Beller <sbeller@google.com>
> >> ---
> >>  diff.c | 22 +++++++++++++++++++---
> >>  1 file changed, 19 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/diff.c b/diff.c
> >> index 2f9722b382..89466018e5 100644
> >> --- a/diff.c
> >> +++ b/diff.c
> >> @@ -559,6 +559,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
> >>       emit_line_0(o, set, reset, line[0], line+1, len-1);
> >>  }
> >>
> >> +enum diff_symbol {
> >> +     DIFF_SYMBOL_SEPARATOR,
> >
> > Drop the last comma from enum?
> 
> I looked through out code base and for enums this is
> actually strictly enforced, so I guess I have to play
> by the rules here as I do not want to be the first
> to deviate from an upheld standard.
> 
> This will be painful though as the next ~20 patches
> add more symbols mostly at the end, maybe I need
> to restructure that such that the last symbol stays the same
> throughout the series. Thanks for that thought.

I don't think this is strictly enforced.  If you look at grep.h:197 the
enum 'grep_source_type' has a trailing comma.

> 
> >
> >> +static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
> >> +                          const char *line, int len)
> >> +{
> >> +     switch (s) {
> >> +     case DIFF_SYMBOL_SEPARATOR:
> >> +             fprintf(o->file, "%s%c",
> >> +                     diff_line_prefix(o),
> >> +                     o->line_termination);
> >> +             break;
> >
> > As the first patch in the "diff-symbol" subseries of this topic,
> > this change must seriously be justified.  Why is it so important
> > that a printing of an empty line must be moved to a helper function,
> > which later will gain ability to show other kind of lines?
> 
> Ah yes. This got lost in comparison to the currently queued series with
> diff_lines. The justification for the change was in the buffer patch,
> but now we need to have the justification here.
> 
> In the old series, I had copied the same text in all these
> refactoring patches, but thought to delete them in this series. The first
> refactoring patch makes sense though.
> 
> Thanks,
> Stefan

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol
  2017-06-21 21:43                                     ` Junio C Hamano
@ 2017-06-21 21:51                                       ` Stefan Beller
  0 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-21 21:51 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Brandon Williams, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

On Wed, Jun 21, 2017 at 2:43 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>>>> I looked through out code base and for enums this is
>>>> actually strictly enforced, so I guess I have to play
>>>> by the rules here as I do not want to be the first
>>>> to deviate from an upheld standard.
>>>
>>> You sound like you are trying to find an excuse "not to play by the
>>> rules".  Don't.
>>
>> When the rules are not yielding best results, adapting the rules
>> should be considered.
>
> I do not think anybody is saying that it is unreasonable to have a
> wish that it would be wonderful if we could use C99 features.
>
> Trailing comma at the end of enum is the least interesting addition
> (it is a fix for a previous editions' brown-paper-bag mistake).  I'd
> be delighted if we can drop support for older compilers and start
> using designated initializers, for example.
>
> Finding other's violation of the current coding guideline is *not*
> the right way to promote for that future, however.

In practice it is, though.

Due to the nature of this project we cannot obtain a world view
of our users (and compilers used), such that it is impossible to say
when the last usage with a strict C89 happens.

If however there is code violating the C89 style for a long
enough time and nobody speaks up, we can assume we can
safely upgrade to C99.

To do this in a controlled (ideal) way, we would have a
single location in the code where we'd use a new desired
feature from C99, such that a "rollback" to strict C89 is
easy either by reverting to the old style or even just reverting
a commit as the occurrence was in a dummy file.

By violating the style guide in more than just one place
this is cumbersome and should not be done, which is
why I said I will not do it in this series.

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [showing-off RFC/PATCH 26/26] diff.c: have a "machine parseable" move coloring
  2017-06-20  2:48                           ` [showing-off RFC/PATCH 26/26] diff.c: have a "machine parseable" move coloring Stefan Beller
  2017-06-20  2:50                             ` Stefan Beller
@ 2017-06-21 21:51                             ` Brandon Williams
  2017-06-21 21:55                               ` Junio C Hamano
  1 sibling, 1 reply; 125+ messages in thread
From: Brandon Williams @ 2017-06-21 21:51 UTC (permalink / raw)
  To: Stefan Beller
  Cc: git, gitster, jonathantanmy, jrnieder, mhagger, peff, philipoakley

On 06/19, Stefan Beller wrote:
> Ævar asked for it, this is how you would do it.
> (plus documentation, tests, CLI knobs, options)
> 
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  diff.c | 15 +++++++++++----
>  diff.h |  2 ++
>  2 files changed, 13 insertions(+), 4 deletions(-)
> 
> diff --git a/diff.c b/diff.c
> index 7756f7610c..61caa057ff 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -997,6 +997,7 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
>  	static const char *nneof = " No newline at end of file\n";
>  	const char *context, *reset, *set, *meta, *fraginfo;
>  	struct strbuf sb = STRBUF_INIT;
> +	int sign;

should this be a char instead of an int?

>  
>  	enum diff_symbol s = eds->s;
>  	const char *line = eds->line;
> @@ -1058,8 +1059,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
>  		default:
>  			set = diff_get_color_opt(o, DIFF_FILE_NEW);
>  		}
> +		sign = '+';
> +		if (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)
> +			sign = '*';
>  		reset = diff_get_color_opt(o, DIFF_RESET);
> -		emit_line_ws_markup(o, set, reset, line, len, '+',
> +		emit_line_ws_markup(o, set, reset, line, len, sign,
>  				    flags & DIFF_SYMBOL_CONTENT_WS_MASK,
>  				    flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);
>  		break;
> @@ -1086,8 +1090,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
>  		default:
>  			set = diff_get_color_opt(o, DIFF_FILE_OLD);
>  		}
> +		sign = '-';
> +		if (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)
> +			sign = '_';
>  		reset = diff_get_color_opt(o, DIFF_RESET);
> -		emit_line_ws_markup(o, set, reset, line, len, '-',
> +		emit_line_ws_markup(o, set, reset, line, len, sign,
>  				    flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
>  		break;
>  	case DIFF_SYMBOL_WORDS_PORCELAIN:
> @@ -5475,7 +5482,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
>  	static struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;
>  	struct diff_queue_struct *q = &diff_queued_diff;
>  
> -	if (o->color_moved)
> +	if (o->color_moved || o->machine_readable_moves)
>  		o->emitted_symbols = &esm;
>  
>  	for (i = 0; i < q->nr; i++) {
> @@ -5485,7 +5492,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
>  	}
>  
>  	if (o->emitted_symbols) {
> -		if (o->color_moved) {
> +		if (o->color_moved || o->machine_readable_moves) {
>  			struct hashmap add_lines, del_lines;
>  			unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
>  
> diff --git a/diff.h b/diff.h
> index 98abd75521..b6c4d7ab0f 100644
> --- a/diff.h
> +++ b/diff.h
> @@ -194,6 +194,8 @@ struct diff_options {
>  		COLOR_MOVED_ZEBRA = 2,
>  		COLOR_MOVED_ZEBRA_DIM = 3,
>  	} color_moved;
> +
> +	int machine_readable_moves;
>  };
>  
>  void diff_emit_submodule_del(struct diff_options *o, const char *line);
> -- 
> 2.12.2.575.gb14f27f917
> 

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol
  2017-06-21 21:45                                 ` Brandon Williams
@ 2017-06-21 21:52                                   ` Junio C Hamano
  2017-06-21 21:55                                     ` Brandon Williams
  0 siblings, 1 reply; 125+ messages in thread
From: Junio C Hamano @ 2017-06-21 21:52 UTC (permalink / raw)
  To: Brandon Williams
  Cc: Stefan Beller, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

Brandon Williams <bmwill@google.com> writes:

> I don't think this is strictly enforced.  If you look at grep.h:197 the
> enum 'grep_source_type' has a trailing comma.

OK, so we now know that at least since e1327023 ("grep: refactor the
concept of "grep source" into an object", 2012-02-02) we haven't
heard from anybody whose compiler fails to grok this.

Perhaps we should start pushing for C99 features?

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol
  2017-06-21 21:52                                   ` Junio C Hamano
@ 2017-06-21 21:55                                     ` Brandon Williams
  0 siblings, 0 replies; 125+ messages in thread
From: Brandon Williams @ 2017-06-21 21:55 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Stefan Beller, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

On 06/21, Junio C Hamano wrote:
> Brandon Williams <bmwill@google.com> writes:
> 
> > I don't think this is strictly enforced.  If you look at grep.h:197 the
> > enum 'grep_source_type' has a trailing comma.
> 
> OK, so we now know that at least since e1327023 ("grep: refactor the
> concept of "grep source" into an object", 2012-02-02) we haven't
> heard from anybody whose compiler fails to grok this.
> 
> Perhaps we should start pushing for C99 features?

Yeah for a second I thought I introduced the trailing comma when i added
recursion to grep...but you're right, looking at blame shows that the
trailing comma has been there for much longer.

And I'm all for moving to C99 :)

-- 
Brandon Williams

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [showing-off RFC/PATCH 26/26] diff.c: have a "machine parseable" move coloring
  2017-06-21 21:51                             ` Brandon Williams
@ 2017-06-21 21:55                               ` Junio C Hamano
  2017-06-21 22:40                                 ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Junio C Hamano @ 2017-06-21 21:55 UTC (permalink / raw)
  To: Brandon Williams
  Cc: Stefan Beller, git, jonathantanmy, jrnieder, mhagger, peff, philipoakley

Brandon Williams <bmwill@google.com> writes:

> On 06/19, Stefan Beller wrote:
>> Ævar asked for it, this is how you would do it.
>> (plus documentation, tests, CLI knobs, options)
>> 
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>>  diff.c | 15 +++++++++++----
>>  diff.h |  2 ++
>>  2 files changed, 13 insertions(+), 4 deletions(-)
>> 
>> diff --git a/diff.c b/diff.c
>> index 7756f7610c..61caa057ff 100644
>> --- a/diff.c
>> +++ b/diff.c
>> @@ -997,6 +997,7 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
>>  	static const char *nneof = " No newline at end of file\n";
>>  	const char *context, *reset, *set, *meta, *fraginfo;
>>  	struct strbuf sb = STRBUF_INIT;
>> +	int sign;
>
> should this be a char instead of an int?

Perhaps.  Once we start adding things other than + and -, I think
they should no longer be called "sign", though.

If Stefan chose '*' as a replacement for '+' because both makes
things larger (yes, I am a positive person and my numbers are all
positive), probably the replacement for '-' should be '/' (or '%'),
not '_'.  But that is just bikeshedding.


^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 17/26] diff.c: convert show_stats to use emit_diff_symbol
  2017-06-21 21:39                             ` Brandon Williams
@ 2017-06-21 22:16                               ` Stefan Beller
  0 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-21 22:16 UTC (permalink / raw)
  To: Brandon Williams
  Cc: git, Junio C Hamano, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

>>       case DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:
>>               emit_line(o, "", "", line, len);
>>               break;
>> +     case DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:
>> +             fprintf(o->file, " 0 files changed\n");
>
> Why is this one a fprintf call instead of an emit_line call?  I'm sure
> this is intended It just stands out to me.

I am sure it is unintended to stand out in such a way.
Originally I wanted to make it as close 1:1 conversion,
and as we did not emit the line prefix at the time but relied on
the line prefix being emitted prior to when this was printed.
Will convert to emit_line as that seems to be a bug in this patch.

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [showing-off RFC/PATCH 26/26] diff.c: have a "machine parseable" move coloring
  2017-06-21 21:55                               ` Junio C Hamano
@ 2017-06-21 22:40                                 ` Stefan Beller
  0 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-21 22:40 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Brandon Williams, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

On Wed, Jun 21, 2017 at 2:55 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Brandon Williams <bmwill@google.com> writes:
>
>> On 06/19, Stefan Beller wrote:
>>> Ævar asked for it, this is how you would do it.
>>> (plus documentation, tests, CLI knobs, options)
>>>
>>> Signed-off-by: Stefan Beller <sbeller@google.com>
>>> ---
>>>  diff.c | 15 +++++++++++----
>>>  diff.h |  2 ++
>>>  2 files changed, 13 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/diff.c b/diff.c
>>> index 7756f7610c..61caa057ff 100644
>>> --- a/diff.c
>>> +++ b/diff.c
>>> @@ -997,6 +997,7 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
>>>      static const char *nneof = " No newline at end of file\n";
>>>      const char *context, *reset, *set, *meta, *fraginfo;
>>>      struct strbuf sb = STRBUF_INIT;
>>> +    int sign;
>>
>> should this be a char instead of an int?
>
> Perhaps.  Once we start adding things other than + and -, I think
> they should no longer be called "sign", though.

Well 'sign' is currently meant as a mathematical operation,
but it could also be meant as a street sign?

The empty sign ' ' tells you it is context, and not subject
to bikeshedding whilst reviewing a patch. :)

>
> If Stefan chose '*' as a replacement for '+' because both makes
> things larger (yes, I am a positive person and my numbers are all
> positive), probably the replacement for '-' should be '/' (or '%'),
> not '_'.  But that is just bikeshedding.

I was trying to choose characters that sort of resembled the shape,
but slightly differently. But a machine would not care any ways.

There was also temptation to use bogus signs like 'M' and 'm' for
moved (capital M for + and lower m for -, obviously).

But this is a proof of concept on top of the series showing that
this way with bits set in the flags field we do not need to have
colors. The older series was operating on the colors and by that
figuring out if it was say at a boundary, but here we would just
use the flagbits to come up with a reasonable first character.
(for example: Would the machine parse-able format also care
about dimming? I defer such a discussion in the future.)

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol
  2017-06-21 20:05                             ` Junio C Hamano
@ 2017-06-22 23:30                               ` Stefan Beller
  2017-06-22 23:37                                 ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-22 23:30 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Brandon Williams, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

On Wed, Jun 21, 2017 at 1:05 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>>  static void emit_add_line(const char *reset,
>>                         struct emit_callback *ecbdata,
>>                         const char *line, int len)
>>  {
>> -     emit_line_checked(reset, ecbdata, line, len,
>> -                       DIFF_FILE_NEW, WSEH_NEW, '+');
>> +     unsigned flags = WSEH_NEW | ecbdata->ws_rule;
>> +     if (new_blank_line_at_eof(ecbdata, line, len))
>> +             flags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;
>> +
>> +     emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);
>>  }
>
> This is a bit unsatisfactory that the original code didn't even have
> to make a call to new_blank_line_at_eof() at all when we know we are
> not checking for / coloring whitespace errors, but the function is
> called unconditionally in the new code.

We could check if we do colored output here, I think'll just do that for now,
but on the other hand this becomes a maintenance nightmare when we
rely on these flags in the future e.g. for "machine parse-able coloring"
and would markup according to the flags strictly. I guess we can fix it then.

>
>> diff --git a/diff.h b/diff.h
>> index 5be1ee77a7..8483ca0991 100644
>> --- a/diff.h
>> +++ b/diff.h
>> @@ -148,9 +148,9 @@ struct diff_options {
>>       int abbrev;
>>       int ita_invisible_in_index;
>>  /* white-space error highlighting */
>> -#define WSEH_NEW 1
>> -#define WSEH_CONTEXT 2
>> -#define WSEH_OLD 4
>> +#define WSEH_NEW (1<<12)
>> +#define WSEH_CONTEXT (1<<13)
>> +#define WSEH_OLD (1<<14)
>
> Hopefully this is a safe conversion, as everything should come from
> parse_ws_error_highlight() that uses these constants.

I reviewed the conversion again and I think it is safe, still.

We could do the up and downshifting for the flags to keep these
as-is, but I am not sure if that is a good idea, the up and downshifting
is not just run time complexity but also a readability issue.

>
> But this is brittle; there must be some hint to future readers of
> this code why these bits begin at 12th (it is linked to 07777 we saw
> earlier); otherwise when they change something that needs to widen
> that 07777, they will forget to shift these up, which would screw
> up everything that is carefully laid out here.

I added a note there. and a BUG check at the beginning of the output.
Smart compilers will drop that and for dumb compilers we'll have an
extra check per commit, which is not too bad

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol
  2017-06-22 23:30                               ` Stefan Beller
@ 2017-06-22 23:37                                 ` Stefan Beller
  2017-06-23  4:56                                   ` Junio C Hamano
  0 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-22 23:37 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Brandon Williams, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

On Thu, Jun 22, 2017 at 4:30 PM, Stefan Beller <sbeller@google.com> wrote:
> On Wed, Jun 21, 2017 at 1:05 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Stefan Beller <sbeller@google.com> writes:
>>
>>>  static void emit_add_line(const char *reset,
>>>                         struct emit_callback *ecbdata,
>>>                         const char *line, int len)
>>>  {
>>> -     emit_line_checked(reset, ecbdata, line, len,
>>> -                       DIFF_FILE_NEW, WSEH_NEW, '+');
>>> +     unsigned flags = WSEH_NEW | ecbdata->ws_rule;
>>> +     if (new_blank_line_at_eof(ecbdata, line, len))
>>> +             flags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;
>>> +
>>> +     emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);
>>>  }
>>
>> This is a bit unsatisfactory that the original code didn't even have
>> to make a call to new_blank_line_at_eof() at all when we know we are
>> not checking for / coloring whitespace errors, but the function is
>> called unconditionally in the new code.
>
> We could check if we do colored output here, I think'll just do that for now,
> but on the other hand this becomes a maintenance nightmare when we
> rely on these flags in the future e.g. for "machine parse-able coloring"
> and would markup according to the flags strictly. I guess we can fix it then.

Actually that function already has some quick return:

static int new_blank_line_at_eof(struct emit_callback *ecbdata, const
char *line, int len)
{
    if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
         ecbdata->blank_at_eof_in_preimage &&
         ecbdata->blank_at_eof_in_postimage &&
         ecbdata->blank_at_eof_in_preimage <= ecbdata->lno_in_preimage &&
         ecbdata->blank_at_eof_in_postimage <= ecbdata->lno_in_postimage))
                  return 0;
    return ws_blank_line(line, len, ecbdata->ws_rule);
}

so maybe we could 'inline' it, as there is no reasonable faster check
than what we
have in there.

I'll keep it 'a bit unsatisfactory' here and if it actually is a
performance issue, we
can fix it.

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCH 11/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR
  2017-06-21 20:09                               ` Junio C Hamano
@ 2017-06-22 23:59                                 ` Stefan Beller
  0 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-22 23:59 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Jonathan Tan, Brandon Williams, git, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

On Wed, Jun 21, 2017 at 1:09 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jonathan Tan <jonathantanmy@google.com> writes:
>
>> On Mon, 19 Jun 2017 19:48:01 -0700
>> Stefan Beller <sbeller@google.com> wrote:
>>
>>> @@ -676,6 +677,14 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
>>>              }
>>>              emit_line(o, context, reset, line, len);
>>>              break;
>>> +    case DIFF_SYMBOL_FILEPAIR:
>>> +            meta = diff_get_color_opt(o, DIFF_METAINFO);
>>> +            reset = diff_get_color_opt(o, DIFF_RESET);
>>> +            fprintf(o->file, "%s%s%s%s%s%s\n", diff_line_prefix(o), meta,
>>> +                    flags ? "+++ " : "--- ",
>>
>> I think that a constant should be defined for this flag, just like for
>> content lines. If not it is not immediately obvious that a flag should
>> be set for the +++ lines.
>
> True.
>
>>> +                    line, reset,
>>> +                    strchr(line, ' ') ? "\t" : "");
>>> +            break;
>
> Also this is somewhat misnamed in that a call to it only deals with
> one half of a filepair, iow, the caller must call it twice to show a
> single filepair.

Split into two and renamed to DIFF_SYMBOL_FILEPAIR_PLUS
and DIFF_SYMBOL_FILEPAIR_MINUS.

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 125+ messages in thread

* [PATCHv2 00/25] reroll of sb/diff-color-moved
  2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
                                             ` (25 preceding siblings ...)
  2017-06-20  2:48                           ` [showing-off RFC/PATCH 26/26] diff.c: have a "machine parseable" move coloring Stefan Beller
@ 2017-06-23  1:28                           ` Stefan Beller
  2017-06-23  1:28                             ` [PATCHv2 01/25] diff.c: readability fix Stefan Beller
                                               ` (24 more replies)
  26 siblings, 25 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:28 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

v2:
* addressed all issues raised
* last patch dropped (WIP/RFC: diff.c: have a "machine parseable" move coloring)
* interdiff below

v1:
This is a complete rewrite of the series. Highlights:
* instead of buffering partial lines, we'll pretend all diff output
  follows a well defined grammar, and we emit symbols thereof.
  (The difference is mostly mental, though by this trick we reduce
  the memory footprint for storing one of these symbols from 7 variables
  (3 pointers, 3 ints, one state (also int) down to 4 variables
  (one pointer, 2 ints, one state).
* The algorithm for color painting was detangled:
  -> different functions for block detection and dimming
  -> The last patch (not to be applied) is an RFC that shows
     how we would approach non-colored, but machine parseable highlighting
     of moved lines.

Thanks,
Stefan

Stefan Beller (25):
  diff.c: readability fix
  diff.c: move line ending check into emit_hunk_header
  diff.c: factor out diff_flush_patch_all_file_pairs
  diff.c: introduce emit_diff_symbol
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF
  diff.c: migrate emit_line_checked to use emit_diff_symbol
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS[_PORCELAIN]
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR_{PLUS, MINUS}
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER
  diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF
  submodule.c: migrate diff output to use emit_diff_symbol
  diff.c: convert emit_binary_diff_body to use emit_diff_symbol
  diff.c: convert show_stats to use emit_diff_symbol
  diff.c: convert word diffing to use emit_diff_symbol
  diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP
  diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY
  diff.c: buffer all output if asked to
  diff.c: color moved lines differently
  diff.c: color moved lines differently, plain mode
  diff.c: add dimming to moved line detection
  diff: document the new --color-moved setting

 Documentation/config.txt       |   12 +-
 Documentation/diff-options.txt |   27 +
 cache.h                        |    2 +
 color.h                        |    2 +
 diff.c                         | 1270 ++++++++++++++++++++++++++++++++--------
 diff.h                         |   37 +-
 submodule.c                    |   85 ++-
 submodule.h                    |   13 +-
 t/t4015-diff-whitespace.sh     |  369 ++++++++++++
 9 files changed, 1501 insertions(+), 316 deletions(-)

diff to what is currently queued:
diff --git a/cache.h b/cache.h
index 4d63c44f07..d2204bf6d1 100644
--- a/cache.h
+++ b/cache.h
@@ -2168,6 +2168,7 @@ void shift_tree_by(const struct object_id *, const struct object_id *, struct ob
 #define WS_TRAILING_SPACE      (WS_BLANK_AT_EOL|WS_BLANK_AT_EOF)
 #define WS_DEFAULT_RULE (WS_TRAILING_SPACE|WS_SPACE_BEFORE_TAB|8)
 #define WS_TAB_WIDTH_MASK        077
+/* All WS_* -- when extended, adapt diff.c emit_symbol */
 #define WS_RULE_MASK           07777
 extern unsigned whitespace_rule_cfg;
 extern unsigned whitespace_rule(const char *);
diff --git a/color.h b/color.h
index 0e091b0cf5..fd2b688dfb 100644
--- a/color.h
+++ b/color.h
@@ -42,8 +42,8 @@ struct strbuf;
 #define GIT_COLOR_BG_BLUE	"\033[44m"
 #define GIT_COLOR_BG_MAGENTA	"\033[45m"
 #define GIT_COLOR_BG_CYAN	"\033[46m"
-#define GIT_COLOR_DI		"\033[2m"
-#define GIT_COLOR_DI_IT		"\033[2;3m"
+#define GIT_COLOR_FAINT		"\033[2m"
+#define GIT_COLOR_FAINT_ITALIC	"\033[2;3m"
 
 /* A special value meaning "no color selected" */
 #define GIT_COLOR_NIL "NIL"
diff --git a/diff.c b/diff.c
index 7756f7610c..82ace48c38 100644
--- a/diff.c
+++ b/diff.c
@@ -59,12 +59,12 @@ static char diff_colors[][COLOR_MAXLEN] = {
 	GIT_COLOR_NORMAL,	/* FUNCINFO */
 	GIT_COLOR_BOLD_MAGENTA,	/* OLD_MOVED */
 	GIT_COLOR_BOLD_BLUE,	/* OLD_MOVED ALTERNATIVE */
-	GIT_COLOR_DI,		/* OLD_MOVED_DIM */
-	GIT_COLOR_DI_IT,	/* OLD_MOVED_ALTERNATIVE_DIM */
+	GIT_COLOR_FAINT,	/* OLD_MOVED_DIM */
+	GIT_COLOR_FAINT_ITALIC,	/* OLD_MOVED_ALTERNATIVE_DIM */
 	GIT_COLOR_BOLD_CYAN,	/* NEW_MOVED */
 	GIT_COLOR_BOLD_YELLOW,	/* NEW_MOVED ALTERNATIVE */
-	GIT_COLOR_DI,		/* NEW_MOVED_DIM */
-	GIT_COLOR_DI_IT,	/* NEW_MOVED_ALTERNATIVE_DIM */
+	GIT_COLOR_FAINT,	/* NEW_MOVED_DIM */
+	GIT_COLOR_FAINT_ITALIC,	/* NEW_MOVED_ALTERNATIVE_DIM */
 };
 
 static NORETURN void die_want_option(const char *option_name)
@@ -607,32 +607,11 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 }
 
 enum diff_symbol {
-	DIFF_SYMBOL_SEPARATOR,
-	DIFF_SYMBOL_CONTEXT_MARKER,
-	DIFF_SYMBOL_CONTEXT_FRAGINFO,
-	DIFF_SYMBOL_NO_LF_EOF,
-	DIFF_SYMBOL_CONTEXT,
-	DIFF_SYMBOL_CONTEXT_INCOMPLETE,
-	DIFF_SYMBOL_PLUS,
-	DIFF_SYMBOL_MINUS,
-	DIFF_SYMBOL_WORDS_PORCELAIN,
-	DIFF_SYMBOL_WORDS,
-	DIFF_SYMBOL_FILEPAIR,
-	DIFF_SYMBOL_HEADER,
-	DIFF_SYMBOL_BINARY_FILES,
 	DIFF_SYMBOL_BINARY_DIFF_HEADER,
 	DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,
 	DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,
 	DIFF_SYMBOL_BINARY_DIFF_BODY,
 	DIFF_SYMBOL_BINARY_DIFF_FOOTER,
-	DIFF_SYMBOL_REWRITE_DIFF,
-	DIFF_SYMBOL_SUBMODULE_ADD,
-	DIFF_SYMBOL_SUBMODULE_DEL,
-	DIFF_SYMBOL_SUBMODULE_UNTRACKED,
-	DIFF_SYMBOL_SUBMODULE_MODIFIED,
-	DIFF_SYMBOL_SUBMODULE_HEADER,
-	DIFF_SYMBOL_SUBMODULE_ERROR,
-	DIFF_SYMBOL_SUBMODULE_PIPETHROUGH,
 	DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,
 	DIFF_SYMBOL_STATS_SUMMARY_ABBREV,
 	DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
@@ -640,6 +619,28 @@ enum diff_symbol {
 	DIFF_SYMBOL_WORD_DIFF,
 	DIFF_SYMBOL_STAT_SEP,
 	DIFF_SYMBOL_SUMMARY,
+	DIFF_SYMBOL_SUBMODULE_ADD,
+	DIFF_SYMBOL_SUBMODULE_DEL,
+	DIFF_SYMBOL_SUBMODULE_UNTRACKED,
+	DIFF_SYMBOL_SUBMODULE_MODIFIED,
+	DIFF_SYMBOL_SUBMODULE_HEADER,
+	DIFF_SYMBOL_SUBMODULE_ERROR,
+	DIFF_SYMBOL_SUBMODULE_PIPETHROUGH,
+	DIFF_SYMBOL_REWRITE_DIFF,
+	DIFF_SYMBOL_BINARY_FILES,
+	DIFF_SYMBOL_HEADER,
+	DIFF_SYMBOL_FILEPAIR_PLUS,
+	DIFF_SYMBOL_FILEPAIR_MINUS,
+	DIFF_SYMBOL_WORDS_PORCELAIN,
+	DIFF_SYMBOL_WORDS,
+	DIFF_SYMBOL_CONTEXT,
+	DIFF_SYMBOL_CONTEXT_INCOMPLETE,
+	DIFF_SYMBOL_PLUS,
+	DIFF_SYMBOL_MINUS,
+	DIFF_SYMBOL_NO_LF_EOF,
+	DIFF_SYMBOL_CONTEXT_FRAGINFO,
+	DIFF_SYMBOL_CONTEXT_MARKER,
+	DIFF_SYMBOL_SEPARATOR
 };
 /*
  * Flags for content lines:
@@ -649,7 +650,7 @@ enum diff_symbol {
  */
 #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF	(1<<16)
 #define DIFF_SYMBOL_MOVED_LINE			(1<<17)
-#define DIFF_SYMBOL_MOVED_LINE_ZEBRA		(1<<18)
+#define DIFF_SYMBOL_MOVED_LINE_ALT		(1<<18)
 #define DIFF_SYMBOL_MOVED_LINE_UNINTERESTING	(1<<19)
 #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)
 
@@ -898,14 +899,14 @@ static void mark_color_as_moved(struct diff_options *o,
 		}
 
 		if (flipped_block)
-			l->flags |= DIFF_SYMBOL_MOVED_LINE_ZEBRA;
+			l->flags |= DIFF_SYMBOL_MOVED_LINE_ALT;
 	}
 
 	free(pmb);
 }
 
 #define DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK \
-  (DIFF_SYMBOL_MOVED_LINE | DIFF_SYMBOL_MOVED_LINE_ZEBRA)
+  (DIFF_SYMBOL_MOVED_LINE | DIFF_SYMBOL_MOVED_LINE_ALT)
 static void dim_moved_lines(struct diff_options *o)
 {
 	int n;
@@ -949,12 +950,12 @@ static void dim_moved_lines(struct diff_options *o)
 
 		/* Check if we are at an interesting bound: */
 		if (prev && (prev->flags & DIFF_SYMBOL_MOVED_LINE) &&
-		    (prev->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA) !=
-		       (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA))
+		    (prev->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=
+		       (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))
 			continue;
 		if (next && (next->flags & DIFF_SYMBOL_MOVED_LINE) &&
-		    (next->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA) !=
-		       (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA))
+		    (next->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=
+		       (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))
 			continue;
 
 		/*
@@ -1004,25 +1005,33 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
 	unsigned flags = eds->flags;
 
 	switch (s) {
-	case DIFF_SYMBOL_SEPARATOR:
-		fprintf(o->file, "%s%c",
-			diff_line_prefix(o),
-			o->line_termination);
-		break;
-	case DIFF_SYMBOL_CONTEXT_MARKER:
+	case DIFF_SYMBOL_NO_LF_EOF:
 		context = diff_get_color_opt(o, DIFF_CONTEXT);
 		reset = diff_get_color_opt(o, DIFF_RESET);
-		emit_line(o, context, reset, line, len);
+		putc('\n', o->file);
+		emit_line_0(o, context, reset, '\\',
+			    nneof, strlen(nneof));
 		break;
+	case DIFF_SYMBOL_SUBMODULE_HEADER:
+	case DIFF_SYMBOL_SUBMODULE_ERROR:
+	case DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:
+	case DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:
+	case DIFF_SYMBOL_SUMMARY:
+	case DIFF_SYMBOL_STATS_LINE:
+	case DIFF_SYMBOL_BINARY_DIFF_BODY:
 	case DIFF_SYMBOL_CONTEXT_FRAGINFO:
 		emit_line(o, "", "", line, len);
 		break;
-	case DIFF_SYMBOL_NO_LF_EOF:
+	case DIFF_SYMBOL_CONTEXT_INCOMPLETE:
+	case DIFF_SYMBOL_CONTEXT_MARKER:
 		context = diff_get_color_opt(o, DIFF_CONTEXT);
 		reset = diff_get_color_opt(o, DIFF_RESET);
-		putc('\n', o->file);
-		emit_line_0(o, context, reset, '\\',
-			    nneof, strlen(nneof));
+		emit_line(o, context, reset, line, len);
+		break;
+	case DIFF_SYMBOL_SEPARATOR:
+		fprintf(o->file, "%s%c",
+			diff_line_prefix(o),
+			o->line_termination);
 		break;
 	case DIFF_SYMBOL_CONTEXT:
 		set = diff_get_color_opt(o, DIFF_CONTEXT);
@@ -1030,22 +1039,17 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
 		emit_line_ws_markup(o, set, reset, line, len, ' ',
 				    flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);
 		break;
-	case DIFF_SYMBOL_CONTEXT_INCOMPLETE:
-		context = diff_get_color_opt(o, DIFF_CONTEXT);
-		reset = diff_get_color_opt(o, DIFF_RESET);
-		emit_line(o, context, reset, line, len);
-		break;
 	case DIFF_SYMBOL_PLUS:
 		switch (flags & (DIFF_SYMBOL_MOVED_LINE |
-				 DIFF_SYMBOL_MOVED_LINE_ZEBRA |
+				 DIFF_SYMBOL_MOVED_LINE_ALT |
 				 DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {
 		case DIFF_SYMBOL_MOVED_LINE |
-		     DIFF_SYMBOL_MOVED_LINE_ZEBRA |
+		     DIFF_SYMBOL_MOVED_LINE_ALT |
 		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
 			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT_DIM);
 			break;
 		case DIFF_SYMBOL_MOVED_LINE |
-		     DIFF_SYMBOL_MOVED_LINE_ZEBRA:
+		     DIFF_SYMBOL_MOVED_LINE_ALT:
 			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);
 			break;
 		case DIFF_SYMBOL_MOVED_LINE |
@@ -1065,15 +1069,15 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
 		break;
 	case DIFF_SYMBOL_MINUS:
 		switch (flags & (DIFF_SYMBOL_MOVED_LINE |
-				 DIFF_SYMBOL_MOVED_LINE_ZEBRA |
+				 DIFF_SYMBOL_MOVED_LINE_ALT |
 				 DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {
 		case DIFF_SYMBOL_MOVED_LINE |
-		     DIFF_SYMBOL_MOVED_LINE_ZEBRA |
+		     DIFF_SYMBOL_MOVED_LINE_ALT |
 		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
 			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT_DIM);
 			break;
 		case DIFF_SYMBOL_MOVED_LINE |
-		     DIFF_SYMBOL_MOVED_LINE_ZEBRA:
+		     DIFF_SYMBOL_MOVED_LINE_ALT:
 			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);
 			break;
 		case DIFF_SYMBOL_MOVED_LINE |
@@ -1110,20 +1114,23 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
 		}
 		emit_line(o, context, reset, line, len);
 		break;
-	case DIFF_SYMBOL_FILEPAIR:
+	case DIFF_SYMBOL_FILEPAIR_PLUS:
 		meta = diff_get_color_opt(o, DIFF_METAINFO);
 		reset = diff_get_color_opt(o, DIFF_RESET);
-		fprintf(o->file, "%s%s%s%s%s%s\n", diff_line_prefix(o), meta,
-			flags ? "+++ " : "--- ",
+		fprintf(o->file, "%s%s+++ %s%s%s\n", diff_line_prefix(o), meta,
 			line, reset,
 			strchr(line, ' ') ? "\t" : "");
 		break;
-	case DIFF_SYMBOL_HEADER:
-		fprintf(o->file, "%s", line);
+	case DIFF_SYMBOL_FILEPAIR_MINUS:
+		meta = diff_get_color_opt(o, DIFF_METAINFO);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		fprintf(o->file, "%s%s--- %s%s%s\n", diff_line_prefix(o), meta,
+			line, reset,
+			strchr(line, ' ') ? "\t" : "");
 		break;
 	case DIFF_SYMBOL_BINARY_FILES:
-		fprintf(o->file, "%sBinary files %s differ\n",
-			diff_line_prefix(o), line);
+	case DIFF_SYMBOL_HEADER:
+		fprintf(o->file, "%s", line);
 		break;
 	case DIFF_SYMBOL_BINARY_DIFF_HEADER:
 		fprintf(o->file, "%sGIT binary patch\n", diff_line_prefix(o));
@@ -1134,9 +1141,6 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
 	case DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL:
 		fprintf(o->file, "%sliteral %s\n", diff_line_prefix(o), line);
 		break;
-	case DIFF_SYMBOL_BINARY_DIFF_BODY:
-		emit_line(o, "", "", line, len);
-		break;
 	case DIFF_SYMBOL_BINARY_DIFF_FOOTER:
 		fputs(diff_line_prefix(o), o->file);
 		fputc('\n', o->file);
@@ -1164,36 +1168,19 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
 		fprintf(o->file, "%sSubmodule %s contains modified content\n",
 			diff_line_prefix(o), line);
 		break;
-	case DIFF_SYMBOL_SUBMODULE_HEADER:
-		fprintf(o->file, "%s%s", diff_line_prefix(o), line);
-		break;
-	case DIFF_SYMBOL_SUBMODULE_ERROR:
-		emit_line(o, "", "", line, len);
-		break;
-	case DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:
-		emit_line(o, "", "", line, len);
-		break;
 	case DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:
-		fprintf(o->file, " 0 files changed\n");
+		emit_line(o, "", "", " 0 files changed\n",
+			  strlen(" 0 files changed\n"));
 		break;
 	case DIFF_SYMBOL_STATS_SUMMARY_ABBREV:
 		emit_line(o, "", "", " ...\n", strlen(" ...\n"));
 		break;
-	case DIFF_SYMBOL_STATS_LINE:
-		emit_line(o, "", "", line, len);
-		break;
-	case DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:
-		emit_line(o, "", "", line, len);
-		break;
 	case DIFF_SYMBOL_WORD_DIFF:
 		fprintf(o->file, "%.*s", len, line);
 		break;
 	case DIFF_SYMBOL_STAT_SEP:
 		fputs(o->stat_sep, o->file);
 		break;
-	case DIFF_SYMBOL_SUMMARY:
-		emit_line(o, "", "", line, len);
-		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -1460,8 +1447,10 @@ static void emit_rewrite_diff(const char *name_a,
 	lc_a = count_lines(data_one, size_one);
 	lc_b = count_lines(data_two, size_two);
 
-	emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR, a_name.buf, a_name.len, 0);
-	emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR, b_name.buf, b_name.len, 1);
+	emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,
+			 a_name.buf, a_name.len, 0);
+	emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,
+			 b_name.buf, b_name.len, 0);
 
 	strbuf_addstr(&out, "@@ -");
 	if (!o->irreversible_delete)
@@ -1983,12 +1972,12 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 	}
 
 	if (ecbdata->label_path[0]) {
-		emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR,
+		emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,
 				 ecbdata->label_path[0],
 				 strlen(ecbdata->label_path[0]), 0);
-		emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR,
+		emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,
 				 ecbdata->label_path[1],
-				 strlen(ecbdata->label_path[1]), 1);
+				 strlen(ecbdata->label_path[1]), 0);
 		ecbdata->label_path[0] = ecbdata->label_path[1] = NULL;
 	}
 
@@ -3159,7 +3148,8 @@ static void builtin_diff(const char *name_a,
 			}
 			emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 					 header.buf, header.len, 0);
-			strbuf_addf(&sb, "%s and %s", lbl[0], lbl[1]);
+			strbuf_addf(&sb, "%sBinary files %s and %s differ\n",
+				    diff_line_prefix(o), lbl[0], lbl[1]);
 			emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
 					 sb.buf, sb.len, 0);
 			strbuf_release(&sb);
@@ -3180,7 +3170,8 @@ static void builtin_diff(const char *name_a,
 		if (DIFF_OPT_TST(o, BINARY))
 			emit_binary_diff(o, &mf1, &mf2);
 		else {
-			strbuf_addf(&sb, "%s and %s", lbl[0], lbl[1]);
+			strbuf_addf(&sb, "%sBinary files %s and %s differ\n",
+				    diff_line_prefix(o), lbl[0], lbl[1]);
 			emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
 					 sb.buf, sb.len, 0);
 			strbuf_release(&sb);
@@ -5475,6 +5466,9 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 	static struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;
 	struct diff_queue_struct *q = &diff_queued_diff;
 
+	if (WSEH_NEW & WS_RULE_MASK)
+		die("BUG: WS rules bit mask overlaps with diff symbol flags");
+
 	if (o->color_moved)
 		o->emitted_symbols = &esm;
 


-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 01/25] diff.c: readability fix
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
@ 2017-06-23  1:28                             ` Stefan Beller
  2017-06-23  1:28                             ` [PATCHv2 02/25] diff.c: move line ending check into emit_hunk_header Stefan Beller
                                               ` (23 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:28 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

We already have dereferenced 'p->two' into a local variable 'two'.
Use that.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/diff.c b/diff.c
index 74283d9001..3f5bf8b5a4 100644
--- a/diff.c
+++ b/diff.c
@@ -3283,8 +3283,8 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)
 	const char *other;
 	const char *attr_path;
 
-	name  = p->one->path;
-	other = (strcmp(name, p->two->path) ? p->two->path : NULL);
+	name  = one->path;
+	other = (strcmp(name, two->path) ? two->path : NULL);
 	attr_path = name;
 	if (o->prefix_length)
 		strip_prefix(o->prefix_length, &name, &other);
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 02/25] diff.c: move line ending check into emit_hunk_header
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
  2017-06-23  1:28                             ` [PATCHv2 01/25] diff.c: readability fix Stefan Beller
@ 2017-06-23  1:28                             ` Stefan Beller
  2017-06-23  1:28                             ` [PATCHv2 03/25] diff.c: factor out diff_flush_patch_all_file_pairs Stefan Beller
                                               ` (22 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:28 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

The emit_hunk_header() function is responsible for assembling a
hunk header and calling emit_line() to send the hunk header
to the output file.  Its only caller fn_out_consume() needs
to prepare for a case where the function emits an incomplete
line and add the terminating LF.

Instead make sure emit_hunk_header() to always send a
completed line to emit_line().

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/diff.c b/diff.c
index 3f5bf8b5a4..c2ed605cd0 100644
--- a/diff.c
+++ b/diff.c
@@ -677,6 +677,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 	}
 
 	strbuf_add(&msgbuf, line + len, org_len - len);
+	strbuf_complete_line(&msgbuf);
+
 	emit_line(ecbdata->opt, "", "", msgbuf.buf, msgbuf.len);
 	strbuf_release(&msgbuf);
 }
@@ -1315,8 +1317,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 		len = sane_truncate_line(ecbdata, line, len);
 		find_lno(line, ecbdata);
 		emit_hunk_header(ecbdata, line, len);
-		if (line[len-1] != '\n')
-			putc('\n', o->file);
 		return;
 	}
 
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 03/25] diff.c: factor out diff_flush_patch_all_file_pairs
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
  2017-06-23  1:28                             ` [PATCHv2 01/25] diff.c: readability fix Stefan Beller
  2017-06-23  1:28                             ` [PATCHv2 02/25] diff.c: move line ending check into emit_hunk_header Stefan Beller
@ 2017-06-23  1:28                             ` Stefan Beller
  2017-06-23  1:28                             ` [PATCHv2 04/25] diff.c: introduce emit_diff_symbol Stefan Beller
                                               ` (21 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:28 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

In a later patch we want to do more things before and after all filepairs
are flushed. So factor flushing out all file pairs into its own function
that the new code can be plugged in easily.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/diff.c b/diff.c
index c2ed605cd0..2f9722b382 100644
--- a/diff.c
+++ b/diff.c
@@ -4737,6 +4737,17 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)
 		warning(_(rename_limit_advice), varname, needed);
 }
 
+static void diff_flush_patch_all_file_pairs(struct diff_options *o)
+{
+	int i;
+	struct diff_queue_struct *q = &diff_queued_diff;
+	for (i = 0; i < q->nr; i++) {
+		struct diff_filepair *p = q->queue[i];
+		if (check_pair_status(p))
+			diff_flush_patch(p, o);
+	}
+}
+
 void diff_flush(struct diff_options *options)
 {
 	struct diff_queue_struct *q = &diff_queued_diff;
@@ -4831,11 +4842,7 @@ void diff_flush(struct diff_options *options)
 			}
 		}
 
-		for (i = 0; i < q->nr; i++) {
-			struct diff_filepair *p = q->queue[i];
-			if (check_pair_status(p))
-				diff_flush_patch(p, options);
-		}
+		diff_flush_patch_all_file_pairs(options);
 	}
 
 	if (output_format & DIFF_FORMAT_CALLBACK)
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 04/25] diff.c: introduce emit_diff_symbol
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (2 preceding siblings ...)
  2017-06-23  1:28                             ` [PATCHv2 03/25] diff.c: factor out diff_flush_patch_all_file_pairs Stefan Beller
@ 2017-06-23  1:28                             ` Stefan Beller
  2017-06-23 20:07                               ` Junio C Hamano
  2017-06-23  1:28                             ` [PATCHv2 05/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER Stefan Beller
                                               ` (20 subsequent siblings)
  24 siblings, 1 reply; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:28 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

In a later patch we want to buffer all output before emitting it as a
new feature ("markup moved lines") conceptually cannot be implemented
in a single pass over the output.

There are different approaches to buffer all output such as:
* Buffering on the char level, i.e. we'd have a char[] which would
  grow at approximately 80 characters a line. This would keep the
  output completely unstructured, but might be very easy to implement,
  such as redirecting all output to a temporary file and working off
  that. The later passes over the buffer are quite complicated though,
  because we have to parse back any output and then decide if it should
  be modified.

* Buffer on a line level. As the output is mostly line oriented already,
  this would make sense, but it still is a bit awkward as we'd have to
  make sense of it again by looking at the first characters of a line
  to decide what part of a diff a line is.

* Buffer semantically. Imagine there is a formal grammar for the diff
  output and we'd keep the symbols of this grammar around. This keeps
  the highest level of structure in the buffered data, such that the
  actual memory requirements are less than say the first option. Instead
  of buffering the characters of the line, we'll buffer what we intend
  to do plus additional information for the specifics. An output of

    diff --git a/new.txt b/new.txt
    index fa69b07..412428c 100644
    Binary files a/new.txt and b/new.txt differ

  could be buffered as
     DIFF_SYMBOL_DIFF_START + new.txt
     DIFF_SYMBOL_INDEX_MODE + fa69b07 412428c "non-executable" flag
     DIFF_SYMBOL_BINARY_FILES + new.txt

This and the following patches introduce the third option of buffering
by first moving any output to emit_diff_symbol, and then introducing the
buffering in this function.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/diff.c b/diff.c
index 2f9722b382..2257d44e2c 100644
--- a/diff.c
+++ b/diff.c
@@ -559,6 +559,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 	emit_line_0(o, set, reset, line[0], line+1, len-1);
 }
 
+enum diff_symbol {
+	DIFF_SYMBOL_SEPARATOR
+};
+
+static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
+			     const char *line, int len)
+{
+	switch (s) {
+	case DIFF_SYMBOL_SEPARATOR:
+		fprintf(o->file, "%s%c",
+			diff_line_prefix(o),
+			o->line_termination);
+		break;
+	default:
+		die("BUG: unknown diff symbol");
+	}
+}
+
 static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)
 {
 	if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
@@ -4833,9 +4851,7 @@ void diff_flush(struct diff_options *options)
 
 	if (output_format & DIFF_FORMAT_PATCH) {
 		if (separator) {
-			fprintf(options->file, "%s%c",
-				diff_line_prefix(options),
-				options->line_termination);
+			emit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);
 			if (options->stat_sep) {
 				/* attach patch instead of inline */
 				fputs(options->stat_sep, options->file);
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 05/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (3 preceding siblings ...)
  2017-06-23  1:28                             ` [PATCHv2 04/25] diff.c: introduce emit_diff_symbol Stefan Beller
@ 2017-06-23  1:28                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 06/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO Stefan Beller
                                               ` (19 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:28 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/diff.c b/diff.c
index 2257d44e2c..c550f75195 100644
--- a/diff.c
+++ b/diff.c
@@ -560,13 +560,20 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 }
 
 enum diff_symbol {
+	DIFF_SYMBOL_CONTEXT_MARKER,
 	DIFF_SYMBOL_SEPARATOR
 };
 
 static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 			     const char *line, int len)
 {
+	const char *context, *reset;
 	switch (s) {
+	case DIFF_SYMBOL_CONTEXT_MARKER:
+		context = diff_get_color_opt(o, DIFF_CONTEXT);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line(o, context, reset, line, len);
+		break;
 	case DIFF_SYMBOL_SEPARATOR:
 		fprintf(o->file, "%s%c",
 			diff_line_prefix(o),
@@ -661,7 +668,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 	if (len < 10 ||
 	    memcmp(line, atat, 2) ||
 	    !(ep = memmem(line + 2, len - 2, atat, 2))) {
-		emit_line(ecbdata->opt, context, reset, line, len);
+		emit_diff_symbol(ecbdata->opt,
+				 DIFF_SYMBOL_CONTEXT_MARKER, line, len);
 		return;
 	}
 	ep += 2; /* skip over @@ */
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 06/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (4 preceding siblings ...)
  2017-06-23  1:28                             ` [PATCHv2 05/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 07/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF Stefan Beller
                                               ` (18 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/diff.c b/diff.c
index c550f75195..16818fa571 100644
--- a/diff.c
+++ b/diff.c
@@ -560,6 +560,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 }
 
 enum diff_symbol {
+	DIFF_SYMBOL_CONTEXT_FRAGINFO,
 	DIFF_SYMBOL_CONTEXT_MARKER,
 	DIFF_SYMBOL_SEPARATOR
 };
@@ -569,6 +570,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 {
 	const char *context, *reset;
 	switch (s) {
+	case DIFF_SYMBOL_CONTEXT_FRAGINFO:
+		emit_line(o, "", "", line, len);
+		break;
 	case DIFF_SYMBOL_CONTEXT_MARKER:
 		context = diff_get_color_opt(o, DIFF_CONTEXT);
 		reset = diff_get_color_opt(o, DIFF_RESET);
@@ -704,8 +708,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 
 	strbuf_add(&msgbuf, line + len, org_len - len);
 	strbuf_complete_line(&msgbuf);
-
-	emit_line(ecbdata->opt, "", "", msgbuf.buf, msgbuf.len);
+	emit_diff_symbol(ecbdata->opt,
+			 DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len);
 	strbuf_release(&msgbuf);
 }
 
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 07/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (5 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 06/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 08/25] diff.c: migrate emit_line_checked to use emit_diff_symbol Stefan Beller
                                               ` (17 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/diff.c b/diff.c
index 16818fa571..b78f698cad 100644
--- a/diff.c
+++ b/diff.c
@@ -560,6 +560,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 }
 
 enum diff_symbol {
+	DIFF_SYMBOL_NO_LF_EOF,
 	DIFF_SYMBOL_CONTEXT_FRAGINFO,
 	DIFF_SYMBOL_CONTEXT_MARKER,
 	DIFF_SYMBOL_SEPARATOR
@@ -568,8 +569,16 @@ enum diff_symbol {
 static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 			     const char *line, int len)
 {
+	static const char *nneof = " No newline at end of file\n";
 	const char *context, *reset;
 	switch (s) {
+	case DIFF_SYMBOL_NO_LF_EOF:
+		context = diff_get_color_opt(o, DIFF_CONTEXT);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		putc('\n', o->file);
+		emit_line_0(o, context, reset, '\\',
+			    nneof, strlen(nneof));
+		break;
 	case DIFF_SYMBOL_CONTEXT_FRAGINFO:
 		emit_line(o, "", "", line, len);
 		break;
@@ -750,7 +759,6 @@ static void emit_rewrite_lines(struct emit_callback *ecb,
 			       int prefix, const char *data, int size)
 {
 	const char *endp = NULL;
-	static const char *nneof = " No newline at end of file\n";
 	const char *reset = diff_get_color(ecb->color_diff, DIFF_RESET);
 
 	while (0 < size) {
@@ -768,13 +776,8 @@ static void emit_rewrite_lines(struct emit_callback *ecb,
 		size -= len;
 		data += len;
 	}
-	if (!endp) {
-		const char *context = diff_get_color(ecb->color_diff,
-						     DIFF_CONTEXT);
-		putc('\n', ecb->opt->file);
-		emit_line_0(ecb->opt, context, reset, '\\',
-			    nneof, strlen(nneof));
-	}
+	if (!endp)
+		emit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0);
 }
 
 static void emit_rewrite_diff(const char *name_a,
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 08/25] diff.c: migrate emit_line_checked to use emit_diff_symbol
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (6 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 07/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 09/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS[_PORCELAIN] Stefan Beller
                                               ` (16 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Add a new flags field to emit_diff_symbol, that will be used by
context lines for:
* white space rules that are applicable (The first 12 bits)
  Take a note in cahe.c as well, when this ws rules are extended we have
  to fix the bits in the flags field.
* how the rules are evaluated (actually this double encodes the sign
  of the line, but the code is easier to keep this way, bits 13,14,15)
* if the line a blank line at EOF (bit 16)

The check if new lines need to be marked up as extra lines at the end of
file, is now done unconditionally. That should be ok, as
'new_blank_line_at_eof' has a quick early return.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 cache.h |   2 ++
 diff.c  | 116 +++++++++++++++++++++++++++++++++++++++++-----------------------
 diff.h  |   6 ++--
 3 files changed, 80 insertions(+), 44 deletions(-)

diff --git a/cache.h b/cache.h
index e1f0e182ad..d2204bf6d1 100644
--- a/cache.h
+++ b/cache.h
@@ -2168,6 +2168,8 @@ void shift_tree_by(const struct object_id *, const struct object_id *, struct ob
 #define WS_TRAILING_SPACE      (WS_BLANK_AT_EOL|WS_BLANK_AT_EOF)
 #define WS_DEFAULT_RULE (WS_TRAILING_SPACE|WS_SPACE_BEFORE_TAB|8)
 #define WS_TAB_WIDTH_MASK        077
+/* All WS_* -- when extended, adapt diff.c emit_symbol */
+#define WS_RULE_MASK           07777
 extern unsigned whitespace_rule_cfg;
 extern unsigned whitespace_rule(const char *);
 extern unsigned parse_whitespace_rule(const char *);
diff --git a/diff.c b/diff.c
index b78f698cad..e7583efca3 100644
--- a/diff.c
+++ b/diff.c
@@ -560,17 +560,54 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 }
 
 enum diff_symbol {
+	DIFF_SYMBOL_CONTEXT,
+	DIFF_SYMBOL_PLUS,
+	DIFF_SYMBOL_MINUS,
 	DIFF_SYMBOL_NO_LF_EOF,
 	DIFF_SYMBOL_CONTEXT_FRAGINFO,
 	DIFF_SYMBOL_CONTEXT_MARKER,
 	DIFF_SYMBOL_SEPARATOR
 };
+/*
+ * Flags for content lines:
+ * 0..12 are whitespace rules
+ * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT
+ * 16 is marking if the line is blank at EOF
+ */
+#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)
+#define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)
+
+static void emit_line_ws_markup(struct diff_options *o,
+				const char *set, const char *reset,
+				const char *line, int len, char sign,
+				unsigned ws_rule, int blank_at_eof)
+{
+	const char *ws = NULL;
+
+	if (o->ws_error_highlight & ws_rule) {
+		ws = diff_get_color_opt(o, DIFF_WHITESPACE);
+		if (!*ws)
+			ws = NULL;
+	}
+
+	if (!ws)
+		emit_line_0(o, set, reset, sign, line, len);
+	else if (blank_at_eof)
+		/* Blank line at EOF - paint '+' as well */
+		emit_line_0(o, ws, reset, sign, line, len);
+	else {
+		/* Emit just the prefix, then the rest. */
+		emit_line_0(o, set, reset, sign, "", 0);
+		ws_check_emit(line, len, ws_rule,
+			      o->file, set, reset, ws);
+	}
+}
 
 static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
-			     const char *line, int len)
+			     const char *line, int len, unsigned flags)
 {
 	static const char *nneof = " No newline at end of file\n";
-	const char *context, *reset;
+	const char *context, *reset, *set;
 	switch (s) {
 	case DIFF_SYMBOL_NO_LF_EOF:
 		context = diff_get_color_opt(o, DIFF_CONTEXT);
@@ -592,6 +629,25 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 			diff_line_prefix(o),
 			o->line_termination);
 		break;
+	case DIFF_SYMBOL_CONTEXT:
+		set = diff_get_color_opt(o, DIFF_CONTEXT);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line_ws_markup(o, set, reset, line, len, ' ',
+				    flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);
+		break;
+	case DIFF_SYMBOL_PLUS:
+		set = diff_get_color_opt(o, DIFF_FILE_NEW);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line_ws_markup(o, set, reset, line, len, '+',
+				    flags & DIFF_SYMBOL_CONTENT_WS_MASK,
+				    flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);
+		break;
+	case DIFF_SYMBOL_MINUS:
+		set = diff_get_color_opt(o, DIFF_FILE_OLD);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line_ws_markup(o, set, reset, line, len, '-',
+				    flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -608,57 +664,31 @@ static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line
 	return ws_blank_line(line, len, ecbdata->ws_rule);
 }
 
-static void emit_line_checked(const char *reset,
-			      struct emit_callback *ecbdata,
-			      const char *line, int len,
-			      enum color_diff color,
-			      unsigned ws_error_highlight,
-			      char sign)
-{
-	const char *set = diff_get_color(ecbdata->color_diff, color);
-	const char *ws = NULL;
-
-	if (ecbdata->opt->ws_error_highlight & ws_error_highlight) {
-		ws = diff_get_color(ecbdata->color_diff, DIFF_WHITESPACE);
-		if (!*ws)
-			ws = NULL;
-	}
-
-	if (!ws)
-		emit_line_0(ecbdata->opt, set, reset, sign, line, len);
-	else if (sign == '+' && new_blank_line_at_eof(ecbdata, line, len))
-		/* Blank line at EOF - paint '+' as well */
-		emit_line_0(ecbdata->opt, ws, reset, sign, line, len);
-	else {
-		/* Emit just the prefix, then the rest. */
-		emit_line_0(ecbdata->opt, set, reset, sign, "", 0);
-		ws_check_emit(line, len, ecbdata->ws_rule,
-			      ecbdata->opt->file, set, reset, ws);
-	}
-}
-
 static void emit_add_line(const char *reset,
 			  struct emit_callback *ecbdata,
 			  const char *line, int len)
 {
-	emit_line_checked(reset, ecbdata, line, len,
-			  DIFF_FILE_NEW, WSEH_NEW, '+');
+	unsigned flags = WSEH_NEW | ecbdata->ws_rule;
+	if (new_blank_line_at_eof(ecbdata, line, len))
+		flags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;
+
+	emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);
 }
 
 static void emit_del_line(const char *reset,
 			  struct emit_callback *ecbdata,
 			  const char *line, int len)
 {
-	emit_line_checked(reset, ecbdata, line, len,
-			  DIFF_FILE_OLD, WSEH_OLD, '-');
+	unsigned flags = WSEH_OLD | ecbdata->ws_rule;
+	emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_MINUS, line, len, flags);
 }
 
 static void emit_context_line(const char *reset,
 			      struct emit_callback *ecbdata,
 			      const char *line, int len)
 {
-	emit_line_checked(reset, ecbdata, line, len,
-			  DIFF_CONTEXT, WSEH_CONTEXT, ' ');
+	unsigned flags = WSEH_CONTEXT | ecbdata->ws_rule;
+	emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_CONTEXT, line, len, flags);
 }
 
 static void emit_hunk_header(struct emit_callback *ecbdata,
@@ -682,7 +712,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 	    memcmp(line, atat, 2) ||
 	    !(ep = memmem(line + 2, len - 2, atat, 2))) {
 		emit_diff_symbol(ecbdata->opt,
-				 DIFF_SYMBOL_CONTEXT_MARKER, line, len);
+				 DIFF_SYMBOL_CONTEXT_MARKER, line, len, 0);
 		return;
 	}
 	ep += 2; /* skip over @@ */
@@ -718,7 +748,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,
 	strbuf_add(&msgbuf, line + len, org_len - len);
 	strbuf_complete_line(&msgbuf);
 	emit_diff_symbol(ecbdata->opt,
-			 DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len);
+			 DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len, 0);
 	strbuf_release(&msgbuf);
 }
 
@@ -777,7 +807,7 @@ static void emit_rewrite_lines(struct emit_callback *ecb,
 		data += len;
 	}
 	if (!endp)
-		emit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0);
+		emit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0, 0);
 }
 
 static void emit_rewrite_diff(const char *name_a,
@@ -4774,6 +4804,10 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 {
 	int i;
 	struct diff_queue_struct *q = &diff_queued_diff;
+
+	if (WSEH_NEW & WS_RULE_MASK)
+		die("BUG: WS rules bit mask overlaps with diff symbol flags");
+
 	for (i = 0; i < q->nr; i++) {
 		struct diff_filepair *p = q->queue[i];
 		if (check_pair_status(p))
@@ -4866,7 +4900,7 @@ void diff_flush(struct diff_options *options)
 
 	if (output_format & DIFF_FORMAT_PATCH) {
 		if (separator) {
-			emit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);
+			emit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);
 			if (options->stat_sep) {
 				/* attach patch instead of inline */
 				fputs(options->stat_sep, options->file);
diff --git a/diff.h b/diff.h
index 5be1ee77a7..8483ca0991 100644
--- a/diff.h
+++ b/diff.h
@@ -148,9 +148,9 @@ struct diff_options {
 	int abbrev;
 	int ita_invisible_in_index;
 /* white-space error highlighting */
-#define WSEH_NEW 1
-#define WSEH_CONTEXT 2
-#define WSEH_OLD 4
+#define WSEH_NEW (1<<12)
+#define WSEH_CONTEXT (1<<13)
+#define WSEH_OLD (1<<14)
 	unsigned ws_error_highlight;
 	const char *prefix;
 	int prefix_length;
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 09/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS[_PORCELAIN]
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (7 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 08/25] diff.c: migrate emit_line_checked to use emit_diff_symbol Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 10/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE Stefan Beller
                                               ` (15 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 42 ++++++++++++++++++++++++++----------------
 1 file changed, 26 insertions(+), 16 deletions(-)

diff --git a/diff.c b/diff.c
index e7583efca3..3d081edd12 100644
--- a/diff.c
+++ b/diff.c
@@ -560,6 +560,8 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 }
 
 enum diff_symbol {
+	DIFF_SYMBOL_WORDS_PORCELAIN,
+	DIFF_SYMBOL_WORDS,
 	DIFF_SYMBOL_CONTEXT,
 	DIFF_SYMBOL_PLUS,
 	DIFF_SYMBOL_MINUS,
@@ -648,6 +650,26 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 		emit_line_ws_markup(o, set, reset, line, len, '-',
 				    flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
 		break;
+	case DIFF_SYMBOL_WORDS_PORCELAIN:
+		context = diff_get_color_opt(o, DIFF_CONTEXT);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line(o, context, reset, line, len);
+		fputs("~\n", o->file);
+		break;
+	case DIFF_SYMBOL_WORDS:
+		context = diff_get_color_opt(o, DIFF_CONTEXT);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		/*
+		 * Skip the prefix character, if any.  With
+		 * diff_suppress_blank_empty, there may be
+		 * none.
+		 */
+		if (line[0] != '\n') {
+			line++;
+			len--;
+		}
+		emit_line(o, context, reset, line, len);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -1342,7 +1364,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 {
 	struct emit_callback *ecbdata = priv;
 	const char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);
-	const char *context = diff_get_color(ecbdata->color_diff, DIFF_CONTEXT);
 	const char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);
 	struct diff_options *o = ecbdata->opt;
 	const char *line_prefix = diff_line_prefix(o);
@@ -1384,6 +1405,9 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 	}
 
 	if (ecbdata->diff_words) {
+		enum diff_symbol s =
+			ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN ?
+			DIFF_SYMBOL_WORDS_PORCELAIN : DIFF_SYMBOL_WORDS;
 		if (line[0] == '-') {
 			diff_words_append(line, len,
 					  &ecbdata->diff_words->minus);
@@ -1403,21 +1427,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 			return;
 		}
 		diff_words_flush(ecbdata);
-		if (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {
-			emit_line(o, context, reset, line, len);
-			fputs("~\n", o->file);
-		} else {
-			/*
-			 * Skip the prefix character, if any.  With
-			 * diff_suppress_blank_empty, there may be
-			 * none.
-			 */
-			if (line[0] != '\n') {
-			      line++;
-			      len--;
-			}
-			emit_line(o, context, reset, line, len);
-		}
+		emit_diff_symbol(o, s, line, len, 0);
 		return;
 	}
 
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 10/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (8 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 09/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS[_PORCELAIN] Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 11/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR_{PLUS, MINUS} Stefan Beller
                                               ` (14 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

The context marker use the exact same output pattern, so reuse it.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/diff.c b/diff.c
index 3d081edd12..f3d0918810 100644
--- a/diff.c
+++ b/diff.c
@@ -563,6 +563,7 @@ enum diff_symbol {
 	DIFF_SYMBOL_WORDS_PORCELAIN,
 	DIFF_SYMBOL_WORDS,
 	DIFF_SYMBOL_CONTEXT,
+	DIFF_SYMBOL_CONTEXT_INCOMPLETE,
 	DIFF_SYMBOL_PLUS,
 	DIFF_SYMBOL_MINUS,
 	DIFF_SYMBOL_NO_LF_EOF,
@@ -621,6 +622,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_CONTEXT_FRAGINFO:
 		emit_line(o, "", "", line, len);
 		break;
+	case DIFF_SYMBOL_CONTEXT_INCOMPLETE:
 	case DIFF_SYMBOL_CONTEXT_MARKER:
 		context = diff_get_color_opt(o, DIFF_CONTEXT);
 		reset = diff_get_color_opt(o, DIFF_RESET);
@@ -1448,8 +1450,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 	default:
 		/* incomplete line at the end */
 		ecbdata->lno_in_preimage++;
-		emit_line(o, diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),
-			  reset, line, len);
+		emit_diff_symbol(o, DIFF_SYMBOL_CONTEXT_INCOMPLETE,
+				 line, len, 0);
 		break;
 	}
 }
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 11/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR_{PLUS, MINUS}
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (9 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 10/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 12/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER Stefan Beller
                                               ` (13 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

We have to use fprintf instead of emit_line, because we want to emit the
tab after the color. This is important for ancient versions of gnu patch
AFAICT, although we probably do not want to feed colored output to the
patch utility, such that it would not matter if the trailing tab is
colored. Keep the corner case as-is though.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 51 ++++++++++++++++++++++++++++++---------------------
 1 file changed, 30 insertions(+), 21 deletions(-)

diff --git a/diff.c b/diff.c
index f3d0918810..761ee581ad 100644
--- a/diff.c
+++ b/diff.c
@@ -560,6 +560,8 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 }
 
 enum diff_symbol {
+	DIFF_SYMBOL_FILEPAIR_PLUS,
+	DIFF_SYMBOL_FILEPAIR_MINUS,
 	DIFF_SYMBOL_WORDS_PORCELAIN,
 	DIFF_SYMBOL_WORDS,
 	DIFF_SYMBOL_CONTEXT,
@@ -610,7 +612,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 			     const char *line, int len, unsigned flags)
 {
 	static const char *nneof = " No newline at end of file\n";
-	const char *context, *reset, *set;
+	const char *context, *reset, *set, *meta;
 	switch (s) {
 	case DIFF_SYMBOL_NO_LF_EOF:
 		context = diff_get_color_opt(o, DIFF_CONTEXT);
@@ -672,6 +674,20 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 		}
 		emit_line(o, context, reset, line, len);
 		break;
+	case DIFF_SYMBOL_FILEPAIR_PLUS:
+		meta = diff_get_color_opt(o, DIFF_METAINFO);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		fprintf(o->file, "%s%s+++ %s%s%s\n", diff_line_prefix(o), meta,
+			line, reset,
+			strchr(line, ' ') ? "\t" : "");
+		break;
+	case DIFF_SYMBOL_FILEPAIR_MINUS:
+		meta = diff_get_color_opt(o, DIFF_METAINFO);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		fprintf(o->file, "%s%s--- %s%s%s\n", diff_line_prefix(o), meta,
+			line, reset,
+			strchr(line, ' ') ? "\t" : "");
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -843,8 +859,6 @@ static void emit_rewrite_diff(const char *name_a,
 			      struct diff_options *o)
 {
 	int lc_a, lc_b;
-	const char *name_a_tab, *name_b_tab;
-	const char *metainfo = diff_get_color(o->use_color, DIFF_METAINFO);
 	const char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);
 	const char *reset = diff_get_color(o->use_color, DIFF_RESET);
 	static struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;
@@ -864,8 +878,6 @@ static void emit_rewrite_diff(const char *name_a,
 
 	name_a += (*name_a == '/');
 	name_b += (*name_b == '/');
-	name_a_tab = strchr(name_a, ' ') ? "\t" : "";
-	name_b_tab = strchr(name_b, ' ') ? "\t" : "";
 
 	strbuf_reset(&a_name);
 	strbuf_reset(&b_name);
@@ -892,11 +904,13 @@ static void emit_rewrite_diff(const char *name_a,
 
 	lc_a = count_lines(data_one, size_one);
 	lc_b = count_lines(data_two, size_two);
-	fprintf(o->file,
-		"%s%s--- %s%s%s\n%s%s+++ %s%s%s\n%s%s@@ -",
-		line_prefix, metainfo, a_name.buf, name_a_tab, reset,
-		line_prefix, metainfo, b_name.buf, name_b_tab, reset,
-		line_prefix, fraginfo);
+
+	emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,
+			 a_name.buf, a_name.len, 0);
+	emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,
+			 b_name.buf, b_name.len, 0);
+
+	fprintf(o->file, "%s%s@@ -", line_prefix, fraginfo);
 	if (!o->irreversible_delete)
 		print_line_count(o->file, lc_a);
 	else
@@ -1365,10 +1379,8 @@ static void find_lno(const char *line, struct emit_callback *ecbdata)
 static void fn_out_consume(void *priv, char *line, unsigned long len)
 {
 	struct emit_callback *ecbdata = priv;
-	const char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);
 	const char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);
 	struct diff_options *o = ecbdata->opt;
-	const char *line_prefix = diff_line_prefix(o);
 
 	o->found_changes = 1;
 
@@ -1379,15 +1391,12 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 	}
 
 	if (ecbdata->label_path[0]) {
-		const char *name_a_tab, *name_b_tab;
-
-		name_a_tab = strchr(ecbdata->label_path[0], ' ') ? "\t" : "";
-		name_b_tab = strchr(ecbdata->label_path[1], ' ') ? "\t" : "";
-
-		fprintf(o->file, "%s%s--- %s%s%s\n",
-			line_prefix, meta, ecbdata->label_path[0], reset, name_a_tab);
-		fprintf(o->file, "%s%s+++ %s%s%s\n",
-			line_prefix, meta, ecbdata->label_path[1], reset, name_b_tab);
+		emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,
+				 ecbdata->label_path[0],
+				 strlen(ecbdata->label_path[0]), 0);
+		emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,
+				 ecbdata->label_path[1],
+				 strlen(ecbdata->label_path[1]), 0);
 		ecbdata->label_path[0] = ecbdata->label_path[1] = NULL;
 	}
 
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 12/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (10 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 11/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR_{PLUS, MINUS} Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 13/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES Stefan Beller
                                               ` (12 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

The header is constructed lazily including line breaks, so just emit
the raw string as is.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)

diff --git a/diff.c b/diff.c
index 761ee581ad..34314455b5 100644
--- a/diff.c
+++ b/diff.c
@@ -560,6 +560,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 }
 
 enum diff_symbol {
+	DIFF_SYMBOL_HEADER,
 	DIFF_SYMBOL_FILEPAIR_PLUS,
 	DIFF_SYMBOL_FILEPAIR_MINUS,
 	DIFF_SYMBOL_WORDS_PORCELAIN,
@@ -688,6 +689,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 			line, reset,
 			strchr(line, ' ') ? "\t" : "");
 		break;
+	case DIFF_SYMBOL_HEADER:
+		fprintf(o->file, "%s", line);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -1385,7 +1389,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)
 	o->found_changes = 1;
 
 	if (ecbdata->header) {
-		fprintf(o->file, "%s", ecbdata->header->buf);
+		emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
+				 ecbdata->header->buf, ecbdata->header->len, 0);
 		strbuf_reset(ecbdata->header);
 		ecbdata->header = NULL;
 	}
@@ -2519,7 +2524,8 @@ static void builtin_diff(const char *name_a,
 		if (complete_rewrite &&
 		    (textconv_one || !diff_filespec_is_binary(one)) &&
 		    (textconv_two || !diff_filespec_is_binary(two))) {
-			fprintf(o->file, "%s", header.buf);
+			emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
+					 header.buf, header.len, 0);
 			strbuf_reset(&header);
 			emit_rewrite_diff(name_a, name_b, one, two,
 						textconv_one, textconv_two, o);
@@ -2529,7 +2535,8 @@ static void builtin_diff(const char *name_a,
 	}
 
 	if (o->irreversible_delete && lbl[1][0] == '/') {
-		fprintf(o->file, "%s", header.buf);
+		emit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf,
+				 header.len, 0);
 		strbuf_reset(&header);
 		goto free_ab_and_return;
 	} else if (!DIFF_OPT_TST(o, TEXT) &&
@@ -2540,10 +2547,13 @@ static void builtin_diff(const char *name_a,
 		    !DIFF_OPT_TST(o, BINARY)) {
 			if (!oidcmp(&one->oid, &two->oid)) {
 				if (must_show_header)
-					fprintf(o->file, "%s", header.buf);
+					emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
+							 header.buf, header.len,
+							 0);
 				goto free_ab_and_return;
 			}
-			fprintf(o->file, "%s", header.buf);
+			emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
+					 header.buf, header.len, 0);
 			fprintf(o->file, "%sBinary files %s and %s differ\n",
 				line_prefix, lbl[0], lbl[1]);
 			goto free_ab_and_return;
@@ -2554,10 +2564,11 @@ static void builtin_diff(const char *name_a,
 		if (mf1.size == mf2.size &&
 		    !memcmp(mf1.ptr, mf2.ptr, mf1.size)) {
 			if (must_show_header)
-				fprintf(o->file, "%s", header.buf);
+				emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
+						 header.buf, header.len, 0);
 			goto free_ab_and_return;
 		}
-		fprintf(o->file, "%s", header.buf);
+		emit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);
 		strbuf_reset(&header);
 		if (DIFF_OPT_TST(o, BINARY))
 			emit_binary_diff(o->file, &mf1, &mf2, line_prefix);
@@ -2575,7 +2586,8 @@ static void builtin_diff(const char *name_a,
 		const struct userdiff_funcname *pe;
 
 		if (must_show_header) {
-			fprintf(o->file, "%s", header.buf);
+			emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
+					 header.buf, header.len, 0);
 			strbuf_reset(&header);
 		}
 
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 13/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (11 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 12/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 14/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF Stefan Beller
                                               ` (11 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

we could save a little bit of memory when buffering in a later mode
by just passing the inner part ("%s and %s", file1, file 2), but
those a just a few bytes, so instead let's reuse the implementation from
DIFF_SYMBOL_HEADER and keep the whole line around.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/diff.c b/diff.c
index 34314455b5..78cf5ad691 100644
--- a/diff.c
+++ b/diff.c
@@ -560,6 +560,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 }
 
 enum diff_symbol {
+	DIFF_SYMBOL_BINARY_FILES,
 	DIFF_SYMBOL_HEADER,
 	DIFF_SYMBOL_FILEPAIR_PLUS,
 	DIFF_SYMBOL_FILEPAIR_MINUS,
@@ -689,6 +690,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 			line, reset,
 			strchr(line, ' ') ? "\t" : "");
 		break;
+	case DIFF_SYMBOL_BINARY_FILES:
 	case DIFF_SYMBOL_HEADER:
 		fprintf(o->file, "%s", line);
 		break;
@@ -2542,6 +2544,7 @@ static void builtin_diff(const char *name_a,
 	} else if (!DIFF_OPT_TST(o, TEXT) &&
 	    ( (!textconv_one && diff_filespec_is_binary(one)) ||
 	      (!textconv_two && diff_filespec_is_binary(two)) )) {
+		struct strbuf sb = STRBUF_INIT;
 		if (!one->data && !two->data &&
 		    S_ISREG(one->mode) && S_ISREG(two->mode) &&
 		    !DIFF_OPT_TST(o, BINARY)) {
@@ -2554,8 +2557,11 @@ static void builtin_diff(const char *name_a,
 			}
 			emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 					 header.buf, header.len, 0);
-			fprintf(o->file, "%sBinary files %s and %s differ\n",
-				line_prefix, lbl[0], lbl[1]);
+			strbuf_addf(&sb, "%sBinary files %s and %s differ\n",
+				    diff_line_prefix(o), lbl[0], lbl[1]);
+			emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
+					 sb.buf, sb.len, 0);
+			strbuf_release(&sb);
 			goto free_ab_and_return;
 		}
 		if (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)
@@ -2572,9 +2578,13 @@ static void builtin_diff(const char *name_a,
 		strbuf_reset(&header);
 		if (DIFF_OPT_TST(o, BINARY))
 			emit_binary_diff(o->file, &mf1, &mf2, line_prefix);
-		else
-			fprintf(o->file, "%sBinary files %s and %s differ\n",
-				line_prefix, lbl[0], lbl[1]);
+		else {
+			strbuf_addf(&sb, "%sBinary files %s and %s differ\n",
+				    diff_line_prefix(o), lbl[0], lbl[1]);
+			emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
+					 sb.buf, sb.len, 0);
+			strbuf_release(&sb);
+		}
 		o->found_changes = 1;
 	} else {
 		/* Crazy xdl interfaces.. */
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 14/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (12 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 13/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 15/25] submodule.c: migrate diff output to use emit_diff_symbol Stefan Beller
                                               ` (10 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 35 +++++++++++++++++++++--------------
 1 file changed, 21 insertions(+), 14 deletions(-)

diff --git a/diff.c b/diff.c
index 78cf5ad691..0314d57647 100644
--- a/diff.c
+++ b/diff.c
@@ -560,6 +560,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 }
 
 enum diff_symbol {
+	DIFF_SYMBOL_REWRITE_DIFF,
 	DIFF_SYMBOL_BINARY_FILES,
 	DIFF_SYMBOL_HEADER,
 	DIFF_SYMBOL_FILEPAIR_PLUS,
@@ -614,7 +615,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 			     const char *line, int len, unsigned flags)
 {
 	static const char *nneof = " No newline at end of file\n";
-	const char *context, *reset, *set, *meta;
+	const char *context, *reset, *set, *meta, *fraginfo;
 	switch (s) {
 	case DIFF_SYMBOL_NO_LF_EOF:
 		context = diff_get_color_opt(o, DIFF_CONTEXT);
@@ -694,6 +695,11 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_HEADER:
 		fprintf(o->file, "%s", line);
 		break;
+	case DIFF_SYMBOL_REWRITE_DIFF:
+		fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line(o, fraginfo, reset, line, len);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -816,17 +822,17 @@ static void remove_tempfile(void)
 	}
 }
 
-static void print_line_count(FILE *file, int count)
+static void add_line_count(struct strbuf *out, int count)
 {
 	switch (count) {
 	case 0:
-		fprintf(file, "0,0");
+		strbuf_addstr(out, "0,0");
 		break;
 	case 1:
-		fprintf(file, "1");
+		strbuf_addstr(out, "1");
 		break;
 	default:
-		fprintf(file, "1,%d", count);
+		strbuf_addf(out, "1,%d", count);
 		break;
 	}
 }
@@ -865,14 +871,12 @@ static void emit_rewrite_diff(const char *name_a,
 			      struct diff_options *o)
 {
 	int lc_a, lc_b;
-	const char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);
-	const char *reset = diff_get_color(o->use_color, DIFF_RESET);
 	static struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;
 	const char *a_prefix, *b_prefix;
 	char *data_one, *data_two;
 	size_t size_one, size_two;
 	struct emit_callback ecbdata;
-	const char *line_prefix = diff_line_prefix(o);
+	struct strbuf out = STRBUF_INIT;
 
 	if (diff_mnemonic_prefix && DIFF_OPT_TST(o, REVERSE_DIFF)) {
 		a_prefix = o->b_prefix;
@@ -916,14 +920,17 @@ static void emit_rewrite_diff(const char *name_a,
 	emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,
 			 b_name.buf, b_name.len, 0);
 
-	fprintf(o->file, "%s%s@@ -", line_prefix, fraginfo);
+	strbuf_addstr(&out, "@@ -");
 	if (!o->irreversible_delete)
-		print_line_count(o->file, lc_a);
+		add_line_count(&out, lc_a);
 	else
-		fprintf(o->file, "?,?");
-	fprintf(o->file, " +");
-	print_line_count(o->file, lc_b);
-	fprintf(o->file, " @@%s\n", reset);
+		strbuf_addstr(&out, "?,?");
+	strbuf_addstr(&out, " +");
+	add_line_count(&out, lc_b);
+	strbuf_addstr(&out, " @@\n");
+	emit_diff_symbol(o, DIFF_SYMBOL_REWRITE_DIFF, out.buf, out.len, 0);
+	strbuf_release(&out);
+
 	if (lc_a && !o->irreversible_delete)
 		emit_rewrite_lines(&ecbdata, '-', data_one, size_one);
 	if (lc_b)
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 15/25] submodule.c: migrate diff output to use emit_diff_symbol
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (13 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 14/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 16/25] diff.c: convert emit_binary_diff_body " Stefan Beller
                                               ` (9 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

As the submodule process is no longer attached to the same file pointer
'o->file' as the superprojects process, there is a different result in
color.c::check_auto_color. That is why we need to pass coloring explicitly,
such that the submodule coloring decision will be made by the child process
processing the submodule. Only DIFF_SYMBOL_SUBMODULE_PIPETHROUGH contains
color, the other symbols are for embedding the submodule output into the
superprojects output.

Remove the colors from the function signatures, as all the coloring
decisions will be made either inside the child process or the final
emit_diff_symbol, but not in the functions driving the submodule diff.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c      | 83 ++++++++++++++++++++++++++++++++++++++++++++++++++---------
 diff.h      |  9 +++++++
 submodule.c | 85 +++++++++++++++++++++++++++----------------------------------
 submodule.h | 13 +++-------
 4 files changed, 122 insertions(+), 68 deletions(-)

diff --git a/diff.c b/diff.c
index 0314d57647..bc78a216ab 100644
--- a/diff.c
+++ b/diff.c
@@ -560,6 +560,13 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 }
 
 enum diff_symbol {
+	DIFF_SYMBOL_SUBMODULE_ADD,
+	DIFF_SYMBOL_SUBMODULE_DEL,
+	DIFF_SYMBOL_SUBMODULE_UNTRACKED,
+	DIFF_SYMBOL_SUBMODULE_MODIFIED,
+	DIFF_SYMBOL_SUBMODULE_HEADER,
+	DIFF_SYMBOL_SUBMODULE_ERROR,
+	DIFF_SYMBOL_SUBMODULE_PIPETHROUGH,
 	DIFF_SYMBOL_REWRITE_DIFF,
 	DIFF_SYMBOL_BINARY_FILES,
 	DIFF_SYMBOL_HEADER,
@@ -624,6 +631,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 		emit_line_0(o, context, reset, '\\',
 			    nneof, strlen(nneof));
 		break;
+	case DIFF_SYMBOL_SUBMODULE_HEADER:
+	case DIFF_SYMBOL_SUBMODULE_ERROR:
+	case DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:
 	case DIFF_SYMBOL_CONTEXT_FRAGINFO:
 		emit_line(o, "", "", line, len);
 		break;
@@ -700,11 +710,68 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 		reset = diff_get_color_opt(o, DIFF_RESET);
 		emit_line(o, fraginfo, reset, line, len);
 		break;
+	case DIFF_SYMBOL_SUBMODULE_ADD:
+		set = diff_get_color_opt(o, DIFF_FILE_NEW);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line(o, set, reset, line, len);
+		break;
+	case DIFF_SYMBOL_SUBMODULE_DEL:
+		set = diff_get_color_opt(o, DIFF_FILE_OLD);
+		reset = diff_get_color_opt(o, DIFF_RESET);
+		emit_line(o, set, reset, line, len);
+		break;
+	case DIFF_SYMBOL_SUBMODULE_UNTRACKED:
+		fprintf(o->file, "%sSubmodule %s contains untracked content\n",
+			diff_line_prefix(o), line);
+		break;
+	case DIFF_SYMBOL_SUBMODULE_MODIFIED:
+		fprintf(o->file, "%sSubmodule %s contains modified content\n",
+			diff_line_prefix(o), line);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
 }
 
+void diff_emit_submodule_del(struct diff_options *o, const char *line)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);
+}
+
+void diff_emit_submodule_add(struct diff_options *o, const char *line)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ADD, line, strlen(line), 0);
+}
+
+void diff_emit_submodule_untracked(struct diff_options *o, const char *path)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_UNTRACKED,
+			 path, strlen(path), 0);
+}
+
+void diff_emit_submodule_modified(struct diff_options *o, const char *path)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_MODIFIED,
+			 path, strlen(path), 0);
+}
+
+void diff_emit_submodule_header(struct diff_options *o, const char *header)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_HEADER,
+			 header, strlen(header), 0);
+}
+
+void diff_emit_submodule_error(struct diff_options *o, const char *err)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ERROR, err, strlen(err), 0);
+}
+
+void diff_emit_submodule_pipethrough(struct diff_options *o,
+				     const char *line, int len)
+{
+	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_PIPETHROUGH, line, len, 0);
+}
+
 static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)
 {
 	if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
@@ -2467,24 +2534,16 @@ static void builtin_diff(const char *name_a,
 	if (o->submodule_format == DIFF_SUBMODULE_LOG &&
 	    (!one->mode || S_ISGITLINK(one->mode)) &&
 	    (!two->mode || S_ISGITLINK(two->mode))) {
-		const char *del = diff_get_color_opt(o, DIFF_FILE_OLD);
-		const char *add = diff_get_color_opt(o, DIFF_FILE_NEW);
-		show_submodule_summary(o->file, one->path ? one->path : two->path,
-				line_prefix,
+		show_submodule_summary(o, one->path ? one->path : two->path,
 				&one->oid, &two->oid,
-				two->dirty_submodule,
-				meta, del, add, reset);
+				two->dirty_submodule);
 		return;
 	} else if (o->submodule_format == DIFF_SUBMODULE_INLINE_DIFF &&
 		   (!one->mode || S_ISGITLINK(one->mode)) &&
 		   (!two->mode || S_ISGITLINK(two->mode))) {
-		const char *del = diff_get_color_opt(o, DIFF_FILE_OLD);
-		const char *add = diff_get_color_opt(o, DIFF_FILE_NEW);
-		show_submodule_inline_diff(o->file, one->path ? one->path : two->path,
-				line_prefix,
+		show_submodule_inline_diff(o, one->path ? one->path : two->path,
 				&one->oid, &two->oid,
-				two->dirty_submodule,
-				meta, del, add, reset, o);
+				two->dirty_submodule);
 		return;
 	}
 
diff --git a/diff.h b/diff.h
index 8483ca0991..2ee0ef3908 100644
--- a/diff.h
+++ b/diff.h
@@ -188,6 +188,15 @@ struct diff_options {
 	int diff_path_counter;
 };
 
+void diff_emit_submodule_del(struct diff_options *o, const char *line);
+void diff_emit_submodule_add(struct diff_options *o, const char *line);
+void diff_emit_submodule_untracked(struct diff_options *o, const char *path);
+void diff_emit_submodule_modified(struct diff_options *o, const char *path);
+void diff_emit_submodule_header(struct diff_options *o, const char *header);
+void diff_emit_submodule_error(struct diff_options *o, const char *err);
+void diff_emit_submodule_pipethrough(struct diff_options *o,
+				     const char *line, int len);
+
 enum color_diff {
 	DIFF_RESET = 0,
 	DIFF_CONTEXT = 1,
diff --git a/submodule.c b/submodule.c
index d3299e29c0..71471f3241 100644
--- a/submodule.c
+++ b/submodule.c
@@ -362,9 +362,7 @@ static int prepare_submodule_summary(struct rev_info *rev, const char *path,
 	return prepare_revision_walk(rev);
 }
 
-static void print_submodule_summary(struct rev_info *rev, FILE *f,
-		const char *line_prefix,
-		const char *del, const char *add, const char *reset)
+static void print_submodule_summary(struct rev_info *rev, struct diff_options *o)
 {
 	static const char format[] = "  %m %s";
 	struct strbuf sb = STRBUF_INIT;
@@ -375,18 +373,12 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,
 		ctx.date_mode = rev->date_mode;
 		ctx.output_encoding = get_log_output_encoding();
 		strbuf_setlen(&sb, 0);
-		strbuf_addstr(&sb, line_prefix);
-		if (commit->object.flags & SYMMETRIC_LEFT) {
-			if (del)
-				strbuf_addstr(&sb, del);
-		}
-		else if (add)
-			strbuf_addstr(&sb, add);
 		format_commit_message(commit, format, &sb, &ctx);
-		if (reset)
-			strbuf_addstr(&sb, reset);
 		strbuf_addch(&sb, '\n');
-		fprintf(f, "%s", sb.buf);
+		if (commit->object.flags & SYMMETRIC_LEFT)
+			diff_emit_submodule_del(o, sb.buf);
+		else
+			diff_emit_submodule_add(o, sb.buf);
 	}
 	strbuf_release(&sb);
 }
@@ -413,11 +405,9 @@ void prepare_submodule_repo_env(struct argv_array *out)
  * attempt to lookup both the left and right commits and put them into the
  * left and right pointers.
  */
-static void show_submodule_header(FILE *f, const char *path,
-		const char *line_prefix,
+static void show_submodule_header(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
-		unsigned dirty_submodule, const char *meta,
-		const char *reset,
+		unsigned dirty_submodule,
 		struct commit **left, struct commit **right,
 		struct commit_list **merge_bases)
 {
@@ -426,11 +416,10 @@ static void show_submodule_header(FILE *f, const char *path,
 	int fast_forward = 0, fast_backward = 0;
 
 	if (dirty_submodule & DIRTY_SUBMODULE_UNTRACKED)
-		fprintf(f, "%sSubmodule %s contains untracked content\n",
-			line_prefix, path);
+		diff_emit_submodule_untracked(o, path);
+
 	if (dirty_submodule & DIRTY_SUBMODULE_MODIFIED)
-		fprintf(f, "%sSubmodule %s contains modified content\n",
-			line_prefix, path);
+		diff_emit_submodule_modified(o, path);
 
 	if (is_null_oid(one))
 		message = "(new submodule)";
@@ -472,31 +461,29 @@ static void show_submodule_header(FILE *f, const char *path,
 	}
 
 output_header:
-	strbuf_addf(&sb, "%s%sSubmodule %s ", line_prefix, meta, path);
+	strbuf_addf(&sb, "Submodule %s ", path);
 	strbuf_add_unique_abbrev(&sb, one->hash, DEFAULT_ABBREV);
 	strbuf_addstr(&sb, (fast_backward || fast_forward) ? ".." : "...");
 	strbuf_add_unique_abbrev(&sb, two->hash, DEFAULT_ABBREV);
 	if (message)
-		strbuf_addf(&sb, " %s%s\n", message, reset);
+		strbuf_addf(&sb, " %s\n", message);
 	else
-		strbuf_addf(&sb, "%s:%s\n", fast_backward ? " (rewind)" : "", reset);
-	fwrite(sb.buf, sb.len, 1, f);
+		strbuf_addf(&sb, "%s:\n", fast_backward ? " (rewind)" : "");
+	diff_emit_submodule_header(o, sb.buf);
 
 	strbuf_release(&sb);
 }
 
-void show_submodule_summary(FILE *f, const char *path,
-		const char *line_prefix,
+void show_submodule_summary(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
-		unsigned dirty_submodule, const char *meta,
-		const char *del, const char *add, const char *reset)
+		unsigned dirty_submodule)
 {
 	struct rev_info rev;
 	struct commit *left = NULL, *right = NULL;
 	struct commit_list *merge_bases = NULL;
 
-	show_submodule_header(f, path, line_prefix, one, two, dirty_submodule,
-			      meta, reset, &left, &right, &merge_bases);
+	show_submodule_header(o, path, one, two, dirty_submodule,
+			      &left, &right, &merge_bases);
 
 	/*
 	 * If we don't have both a left and a right pointer, there is no
@@ -508,11 +495,11 @@ void show_submodule_summary(FILE *f, const char *path,
 
 	/* Treat revision walker failure the same as missing commits */
 	if (prepare_submodule_summary(&rev, path, left, right, merge_bases)) {
-		fprintf(f, "%s(revision walker failed)\n", line_prefix);
+		diff_emit_submodule_error(o, "(revision walker failed)\n");
 		goto out;
 	}
 
-	print_submodule_summary(&rev, f, line_prefix, del, add, reset);
+	print_submodule_summary(&rev, o);
 
 out:
 	if (merge_bases)
@@ -521,21 +508,18 @@ void show_submodule_summary(FILE *f, const char *path,
 	clear_commit_marks(right, ~0);
 }
 
-void show_submodule_inline_diff(FILE *f, const char *path,
-		const char *line_prefix,
+void show_submodule_inline_diff(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
-		unsigned dirty_submodule, const char *meta,
-		const char *del, const char *add, const char *reset,
-		const struct diff_options *o)
+		unsigned dirty_submodule)
 {
 	const struct object_id *old = &empty_tree_oid, *new = &empty_tree_oid;
 	struct commit *left = NULL, *right = NULL;
 	struct commit_list *merge_bases = NULL;
-	struct strbuf submodule_dir = STRBUF_INIT;
 	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf sb = STRBUF_INIT;
 
-	show_submodule_header(f, path, line_prefix, one, two, dirty_submodule,
-			      meta, reset, &left, &right, &merge_bases);
+	show_submodule_header(o, path, one, two, dirty_submodule,
+			      &left, &right, &merge_bases);
 
 	/* We need a valid left and right commit to display a difference */
 	if (!(left || is_null_oid(one)) ||
@@ -547,15 +531,16 @@ void show_submodule_inline_diff(FILE *f, const char *path,
 	if (right)
 		new = two;
 
-	fflush(f);
 	cp.git_cmd = 1;
 	cp.dir = path;
-	cp.out = dup(fileno(f));
+	cp.out = -1;
 	cp.no_stdin = 1;
 
 	/* TODO: other options may need to be passed here. */
 	argv_array_push(&cp.args, "diff");
-	argv_array_pushf(&cp.args, "--line-prefix=%s", line_prefix);
+	argv_array_pushf(&cp.args, "--color=%s", want_color(o->use_color) ?
+			 "always" : "never");
+
 	if (DIFF_OPT_TST(o, REVERSE_DIFF)) {
 		argv_array_pushf(&cp.args, "--src-prefix=%s%s/",
 				 o->b_prefix, path);
@@ -578,11 +563,17 @@ void show_submodule_inline_diff(FILE *f, const char *path,
 		argv_array_push(&cp.args, oid_to_hex(new));
 
 	prepare_submodule_repo_env(&cp.env_array);
-	if (run_command(&cp))
-		fprintf(f, "(diff failed)\n");
+	if (start_command(&cp))
+		diff_emit_submodule_error(o, "(diff failed)\n");
+
+	while (strbuf_getwholeline_fd(&sb, cp.out, '\n') != EOF)
+		diff_emit_submodule_pipethrough(o, sb.buf, sb.len);
+
+	if (finish_command(&cp))
+		diff_emit_submodule_error(o, "(diff failed)\n");
 
 done:
-	strbuf_release(&submodule_dir);
+	strbuf_release(&sb);
 	if (merge_bases)
 		free_commit_list(merge_bases);
 	if (left)
diff --git a/submodule.h b/submodule.h
index 1277480add..e7358863e9 100644
--- a/submodule.h
+++ b/submodule.h
@@ -53,17 +53,12 @@ extern int parse_submodule_update_strategy(const char *value,
 		struct submodule_update_strategy *dst);
 extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);
 extern void handle_ignore_submodules_arg(struct diff_options *, const char *);
-extern void show_submodule_summary(FILE *f, const char *path,
-		const char *line_prefix,
+extern void show_submodule_summary(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
-		unsigned dirty_submodule, const char *meta,
-		const char *del, const char *add, const char *reset);
-extern void show_submodule_inline_diff(FILE *f, const char *path,
-		const char *line_prefix,
+		unsigned dirty_submodule);
+extern void show_submodule_inline_diff(struct diff_options *o, const char *path,
 		struct object_id *one, struct object_id *two,
-		unsigned dirty_submodule, const char *meta,
-		const char *del, const char *add, const char *reset,
-		const struct diff_options *opt);
+		unsigned dirty_submodule);
 extern void set_config_fetch_recurse_submodules(int value);
 extern void set_config_update_recurse_submodules(int value);
 /* Check if we want to update any submodule.*/
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 16/25] diff.c: convert emit_binary_diff_body to use emit_diff_symbol
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (14 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 15/25] submodule.c: migrate diff output to use emit_diff_symbol Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 17/25] diff.c: convert show_stats " Stefan Beller
                                               ` (8 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 46 insertions(+), 17 deletions(-)

diff --git a/diff.c b/diff.c
index bc78a216ab..7c92675f6f 100644
--- a/diff.c
+++ b/diff.c
@@ -560,6 +560,11 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset
 }
 
 enum diff_symbol {
+	DIFF_SYMBOL_BINARY_DIFF_HEADER,
+	DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,
+	DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,
+	DIFF_SYMBOL_BINARY_DIFF_BODY,
+	DIFF_SYMBOL_BINARY_DIFF_FOOTER,
 	DIFF_SYMBOL_SUBMODULE_ADD,
 	DIFF_SYMBOL_SUBMODULE_DEL,
 	DIFF_SYMBOL_SUBMODULE_UNTRACKED,
@@ -634,6 +639,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_SUBMODULE_HEADER:
 	case DIFF_SYMBOL_SUBMODULE_ERROR:
 	case DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:
+	case DIFF_SYMBOL_BINARY_DIFF_BODY:
 	case DIFF_SYMBOL_CONTEXT_FRAGINFO:
 		emit_line(o, "", "", line, len);
 		break;
@@ -705,6 +711,19 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_HEADER:
 		fprintf(o->file, "%s", line);
 		break;
+	case DIFF_SYMBOL_BINARY_DIFF_HEADER:
+		fprintf(o->file, "%sGIT binary patch\n", diff_line_prefix(o));
+		break;
+	case DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA:
+		fprintf(o->file, "%sdelta %s\n", diff_line_prefix(o), line);
+		break;
+	case DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL:
+		fprintf(o->file, "%sliteral %s\n", diff_line_prefix(o), line);
+		break;
+	case DIFF_SYMBOL_BINARY_DIFF_FOOTER:
+		fputs(diff_line_prefix(o), o->file);
+		fputc('\n', o->file);
+		break;
 	case DIFF_SYMBOL_REWRITE_DIFF:
 		fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);
 		reset = diff_get_color_opt(o, DIFF_RESET);
@@ -2390,8 +2409,8 @@ static unsigned char *deflate_it(char *data,
 	return deflated;
 }
 
-static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
-				  const char *prefix)
+static void emit_binary_diff_body(struct diff_options *o,
+				  mmfile_t *one, mmfile_t *two)
 {
 	void *cp;
 	void *delta;
@@ -2420,13 +2439,18 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
 	}
 
 	if (delta && delta_size < deflate_size) {
-		fprintf(file, "%sdelta %lu\n", prefix, orig_size);
+		char *s = xstrfmt("%lu", orig_size);
+		emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,
+				 s, strlen(s), 0);
+		free(s);
 		free(deflated);
 		data = delta;
 		data_size = delta_size;
-	}
-	else {
-		fprintf(file, "%sliteral %lu\n", prefix, two->size);
+	} else {
+		char *s = xstrfmt("%lu", two->size);
+		emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,
+				 s, strlen(s), 0);
+		free(s);
 		free(delta);
 		data = deflated;
 		data_size = deflate_size;
@@ -2435,8 +2459,9 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
 	/* emit data encoded in base85 */
 	cp = data;
 	while (data_size) {
+		int len;
 		int bytes = (52 < data_size) ? 52 : data_size;
-		char line[70];
+		char line[71];
 		data_size -= bytes;
 		if (bytes <= 26)
 			line[0] = bytes + 'A' - 1;
@@ -2444,20 +2469,24 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
 			line[0] = bytes - 26 + 'a' - 1;
 		encode_85(line + 1, cp, bytes);
 		cp = (char *) cp + bytes;
-		fprintf(file, "%s", prefix);
-		fputs(line, file);
-		fputc('\n', file);
+
+		len = strlen(line);
+		line[len++] = '\n';
+		line[len] = '\0';
+
+		emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_BODY,
+				 line, len, 0);
 	}
-	fprintf(file, "%s\n", prefix);
+	emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_FOOTER, NULL, 0, 0);
 	free(data);
 }
 
-static void emit_binary_diff(FILE *file, mmfile_t *one, mmfile_t *two,
-			     const char *prefix)
+static void emit_binary_diff(struct diff_options *o,
+			     mmfile_t *one, mmfile_t *two)
 {
-	fprintf(file, "%sGIT binary patch\n", prefix);
-	emit_binary_diff_body(file, one, two, prefix);
-	emit_binary_diff_body(file, two, one, prefix);
+	emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER, NULL, 0, 0);
+	emit_binary_diff_body(o, one, two);
+	emit_binary_diff_body(o, two, one);
 }
 
 int diff_filespec_is_binary(struct diff_filespec *one)
@@ -2643,7 +2672,7 @@ static void builtin_diff(const char *name_a,
 		emit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);
 		strbuf_reset(&header);
 		if (DIFF_OPT_TST(o, BINARY))
-			emit_binary_diff(o->file, &mf1, &mf2, line_prefix);
+			emit_binary_diff(o, &mf1, &mf2);
 		else {
 			strbuf_addf(&sb, "%sBinary files %s and %s differ\n",
 				    diff_line_prefix(o), lbl[0], lbl[1]);
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 17/25] diff.c: convert show_stats to use emit_diff_symbol
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (15 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 16/25] diff.c: convert emit_binary_diff_body " Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 18/25] diff.c: convert word diffing " Stefan Beller
                                               ` (7 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

We call print_stat_summary from builtin/apply, so we still
need the version with a file pointer, so introduce
print_stat_summary_0 that uses emit_string machinery and
keep print_stat_summary with the same arguments around.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 114 +++++++++++++++++++++++++++++++++++++++++------------------------
 diff.h |   4 +--
 2 files changed, 74 insertions(+), 44 deletions(-)

diff --git a/diff.c b/diff.c
index 7c92675f6f..2108c191a3 100644
--- a/diff.c
+++ b/diff.c
@@ -565,6 +565,10 @@ enum diff_symbol {
 	DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,
 	DIFF_SYMBOL_BINARY_DIFF_BODY,
 	DIFF_SYMBOL_BINARY_DIFF_FOOTER,
+	DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,
+	DIFF_SYMBOL_STATS_SUMMARY_ABBREV,
+	DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
+	DIFF_SYMBOL_STATS_LINE,
 	DIFF_SYMBOL_SUBMODULE_ADD,
 	DIFF_SYMBOL_SUBMODULE_DEL,
 	DIFF_SYMBOL_SUBMODULE_UNTRACKED,
@@ -628,6 +632,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 {
 	static const char *nneof = " No newline at end of file\n";
 	const char *context, *reset, *set, *meta, *fraginfo;
+	struct strbuf sb = STRBUF_INIT;
 	switch (s) {
 	case DIFF_SYMBOL_NO_LF_EOF:
 		context = diff_get_color_opt(o, DIFF_CONTEXT);
@@ -639,6 +644,8 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_SUBMODULE_HEADER:
 	case DIFF_SYMBOL_SUBMODULE_ERROR:
 	case DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:
+	case DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:
+	case DIFF_SYMBOL_STATS_LINE:
 	case DIFF_SYMBOL_BINARY_DIFF_BODY:
 	case DIFF_SYMBOL_CONTEXT_FRAGINFO:
 		emit_line(o, "", "", line, len);
@@ -747,9 +754,17 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 		fprintf(o->file, "%sSubmodule %s contains modified content\n",
 			diff_line_prefix(o), line);
 		break;
+	case DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:
+		emit_line(o, "", "", " 0 files changed\n",
+			  strlen(" 0 files changed\n"));
+		break;
+	case DIFF_SYMBOL_STATS_SUMMARY_ABBREV:
+		emit_line(o, "", "", " ...\n", strlen(" ...\n"));
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
+	strbuf_release(&sb);
 }
 
 void diff_emit_submodule_del(struct diff_options *o, const char *line)
@@ -1705,20 +1720,14 @@ static int scale_linear(int it, int width, int max_change)
 	return 1 + (it * (width - 1) / max_change);
 }
 
-static void show_name(FILE *file,
-		      const char *prefix, const char *name, int len)
-{
-	fprintf(file, " %s%-*s |", prefix, len, name);
-}
-
-static void show_graph(FILE *file, char ch, int cnt, const char *set, const char *reset)
+static void show_graph(struct strbuf *out, char ch, int cnt,
+		       const char *set, const char *reset)
 {
 	if (cnt <= 0)
 		return;
-	fprintf(file, "%s", set);
-	while (cnt--)
-		putc(ch, file);
-	fprintf(file, "%s", reset);
+	strbuf_addstr(out, set);
+	strbuf_addchars(out, ch, cnt);
+	strbuf_addstr(out, reset);
 }
 
 static void fill_print_name(struct diffstat_file *file)
@@ -1742,14 +1751,16 @@ static void fill_print_name(struct diffstat_file *file)
 	file->print_name = pname;
 }
 
-int print_stat_summary(FILE *fp, int files, int insertions, int deletions)
+static void print_stat_summary_inserts_deletes(struct diff_options *options,
+		int files, int insertions, int deletions)
 {
 	struct strbuf sb = STRBUF_INIT;
-	int ret;
 
 	if (!files) {
 		assert(insertions == 0 && deletions == 0);
-		return fprintf(fp, "%s\n", " 0 files changed");
+		emit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,
+				 NULL, 0, 0);
+		return;
 	}
 
 	strbuf_addf(&sb,
@@ -1776,9 +1787,19 @@ int print_stat_summary(FILE *fp, int files, int insertions, int deletions)
 			    deletions);
 	}
 	strbuf_addch(&sb, '\n');
-	ret = fputs(sb.buf, fp);
+	emit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
+			 sb.buf, sb.len, 0);
 	strbuf_release(&sb);
-	return ret;
+}
+
+void print_stat_summary(FILE *fp, int files,
+			int insertions, int deletions)
+{
+	struct diff_options o;
+	memset(&o, 0, sizeof(o));
+	o.file = fp;
+
+	print_stat_summary_inserts_deletes(&o, files, insertions, deletions);
 }
 
 static void show_stats(struct diffstat_t *data, struct diff_options *options)
@@ -1788,13 +1809,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 	int total_files = data->nr, count;
 	int width, name_width, graph_width, number_width = 0, bin_width = 0;
 	const char *reset, *add_c, *del_c;
-	const char *line_prefix = "";
 	int extra_shown = 0;
+	const char *line_prefix = diff_line_prefix(options);
+	struct strbuf out = STRBUF_INIT;
 
 	if (data->nr == 0)
 		return;
 
-	line_prefix = diff_line_prefix(options);
 	count = options->stat_count ? options->stat_count : data->nr;
 
 	reset = diff_get_color_opt(options, DIFF_RESET);
@@ -1948,26 +1969,32 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 		}
 
 		if (file->is_binary) {
-			fprintf(options->file, "%s", line_prefix);
-			show_name(options->file, prefix, name, len);
-			fprintf(options->file, " %*s", number_width, "Bin");
+			strbuf_addf(&out, " %s%-*s |", prefix, len, name);
+			strbuf_addf(&out, " %*s", number_width, "Bin");
 			if (!added && !deleted) {
-				putc('\n', options->file);
+				strbuf_addch(&out, '\n');
+				emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
+						 out.buf, out.len, 0);
+				strbuf_reset(&out);
 				continue;
 			}
-			fprintf(options->file, " %s%"PRIuMAX"%s",
+			strbuf_addf(&out, " %s%"PRIuMAX"%s",
 				del_c, deleted, reset);
-			fprintf(options->file, " -> ");
-			fprintf(options->file, "%s%"PRIuMAX"%s",
+			strbuf_addstr(&out, " -> ");
+			strbuf_addf(&out, "%s%"PRIuMAX"%s",
 				add_c, added, reset);
-			fprintf(options->file, " bytes");
-			fprintf(options->file, "\n");
+			strbuf_addstr(&out, " bytes\n");
+			emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
+					 out.buf, out.len, 0);
+			strbuf_reset(&out);
 			continue;
 		}
 		else if (file->is_unmerged) {
-			fprintf(options->file, "%s", line_prefix);
-			show_name(options->file, prefix, name, len);
-			fprintf(options->file, " Unmerged\n");
+			strbuf_addf(&out, " %s%-*s |", prefix, len, name);
+			strbuf_addstr(&out, " Unmerged\n");
+			emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
+					 out.buf, out.len, 0);
+			strbuf_reset(&out);
 			continue;
 		}
 
@@ -1990,14 +2017,16 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 				add = total - del;
 			}
 		}
-		fprintf(options->file, "%s", line_prefix);
-		show_name(options->file, prefix, name, len);
-		fprintf(options->file, " %*"PRIuMAX"%s",
+		strbuf_addf(&out, " %s%-*s |", prefix, len, name);
+		strbuf_addf(&out, " %*"PRIuMAX"%s",
 			number_width, added + deleted,
 			added + deleted ? " " : "");
-		show_graph(options->file, '+', add, add_c, reset);
-		show_graph(options->file, '-', del, del_c, reset);
-		fprintf(options->file, "\n");
+		show_graph(&out, '+', add, add_c, reset);
+		show_graph(&out, '-', del, del_c, reset);
+		strbuf_addch(&out, '\n');
+		emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
+				 out.buf, out.len, 0);
+		strbuf_reset(&out);
 	}
 
 	for (i = 0; i < data->nr; i++) {
@@ -2018,11 +2047,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)
 		if (i < count)
 			continue;
 		if (!extra_shown)
-			fprintf(options->file, "%s ...\n", line_prefix);
+			emit_diff_symbol(options,
+					 DIFF_SYMBOL_STATS_SUMMARY_ABBREV,
+					 NULL, 0, 0);
 		extra_shown = 1;
 	}
-	fprintf(options->file, "%s", line_prefix);
-	print_stat_summary(options->file, total_files, adds, dels);
+
+	print_stat_summary_inserts_deletes(options, total_files, adds, dels);
 }
 
 static void show_shortstats(struct diffstat_t *data, struct diff_options *options)
@@ -2034,7 +2065,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option
 
 	for (i = 0; i < data->nr; i++) {
 		int added = data->files[i]->added;
-		int deleted= data->files[i]->deleted;
+		int deleted = data->files[i]->deleted;
 
 		if (data->files[i]->is_unmerged ||
 		    (!data->files[i]->is_interesting && (added + deleted == 0))) {
@@ -2044,8 +2075,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option
 			dels += deleted;
 		}
 	}
-	fprintf(options->file, "%s", diff_line_prefix(options));
-	print_stat_summary(options->file, total_files, adds, dels);
+	print_stat_summary_inserts_deletes(options, total_files, adds, dels);
 }
 
 static void show_numstat(struct diffstat_t *data, struct diff_options *options)
diff --git a/diff.h b/diff.h
index 2ee0ef3908..b73f7a1e75 100644
--- a/diff.h
+++ b/diff.h
@@ -398,8 +398,8 @@ extern int parse_rename_score(const char **cp_p);
 
 extern long parse_algorithm_value(const char *value);
 
-extern int print_stat_summary(FILE *fp, int files,
-			      int insertions, int deletions);
+extern void print_stat_summary(FILE *fp, int files,
+			       int insertions, int deletions);
 extern void setup_diff_pager(struct diff_options *);
 
 #endif /* DIFF_H */
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 18/25] diff.c: convert word diffing to use emit_diff_symbol
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (16 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 17/25] diff.c: convert show_stats " Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 19/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP Stefan Beller
                                               ` (6 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

The word diffing is not line oriented and would need some serious
effort to be transformed into a line oriented approach, so
just go with a symbol DIFF_SYMBOL_WORD_DIFF that is a partial line.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 79 ++++++++++++++++++++++++++++++++++++++----------------------------
 1 file changed, 46 insertions(+), 33 deletions(-)

diff --git a/diff.c b/diff.c
index 2108c191a3..60630d2c3f 100644
--- a/diff.c
+++ b/diff.c
@@ -569,6 +569,7 @@ enum diff_symbol {
 	DIFF_SYMBOL_STATS_SUMMARY_ABBREV,
 	DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
 	DIFF_SYMBOL_STATS_LINE,
+	DIFF_SYMBOL_WORD_DIFF,
 	DIFF_SYMBOL_SUBMODULE_ADD,
 	DIFF_SYMBOL_SUBMODULE_DEL,
 	DIFF_SYMBOL_SUBMODULE_UNTRACKED,
@@ -761,6 +762,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_STATS_SUMMARY_ABBREV:
 		emit_line(o, "", "", " ...\n", strlen(" ...\n"));
 		break;
+	case DIFF_SYMBOL_WORD_DIFF:
+		fprintf(o->file, "%.*s", len, line);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -1091,37 +1095,49 @@ struct diff_words_data {
 	struct diff_words_style *style;
 };
 
-static int fn_out_diff_words_write_helper(FILE *fp,
+static int fn_out_diff_words_write_helper(struct diff_options *o,
 					  struct diff_words_style_elem *st_el,
 					  const char *newline,
-					  size_t count, const char *buf,
-					  const char *line_prefix)
+					  size_t count, const char *buf)
 {
 	int print = 0;
+	struct strbuf sb = STRBUF_INIT;
 
 	while (count) {
 		char *p = memchr(buf, '\n', count);
 		if (print)
-			fputs(line_prefix, fp);
+			strbuf_addstr(&sb, diff_line_prefix(o));
+
 		if (p != buf) {
-			if (st_el->color && fputs(st_el->color, fp) < 0)
-				return -1;
-			if (fputs(st_el->prefix, fp) < 0 ||
-			    fwrite(buf, p ? p - buf : count, 1, fp) != 1 ||
-			    fputs(st_el->suffix, fp) < 0)
-				return -1;
-			if (st_el->color && *st_el->color
-			    && fputs(GIT_COLOR_RESET, fp) < 0)
-				return -1;
+			const char *reset = st_el->color && *st_el->color ?
+					    GIT_COLOR_RESET : NULL;
+			if (st_el->color && *st_el->color)
+				strbuf_addstr(&sb, st_el->color);
+			strbuf_addstr(&sb, st_el->prefix);
+			strbuf_add(&sb, buf, p ? p - buf : count);
+			strbuf_addstr(&sb, st_el->suffix);
+			if (reset)
+				strbuf_addstr(&sb, reset);
 		}
 		if (!p)
-			return 0;
-		if (fputs(newline, fp) < 0)
-			return -1;
+			goto out;
+
+		strbuf_addstr(&sb, newline);
 		count -= p + 1 - buf;
 		buf = p + 1;
 		print = 1;
+		if (count) {
+			emit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,
+					 sb.buf, sb.len, 0);
+			strbuf_reset(&sb);
+		}
 	}
+
+out:
+	if (sb.len)
+		emit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,
+				 sb.buf, sb.len, 0);
+	strbuf_release(&sb);
 	return 0;
 }
 
@@ -1203,24 +1219,20 @@ static void fn_out_diff_words_aux(void *priv, char *line, unsigned long len)
 		fputs(line_prefix, diff_words->opt->file);
 	}
 	if (diff_words->current_plus != plus_begin) {
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+		fn_out_diff_words_write_helper(diff_words->opt,
 				&style->ctx, style->newline,
 				plus_begin - diff_words->current_plus,
-				diff_words->current_plus, line_prefix);
-		if (*(plus_begin - 1) == '\n')
-			fputs(line_prefix, diff_words->opt->file);
+				diff_words->current_plus);
 	}
 	if (minus_begin != minus_end) {
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+		fn_out_diff_words_write_helper(diff_words->opt,
 				&style->old, style->newline,
-				minus_end - minus_begin, minus_begin,
-				line_prefix);
+				minus_end - minus_begin, minus_begin);
 	}
 	if (plus_begin != plus_end) {
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+		fn_out_diff_words_write_helper(diff_words->opt,
 				&style->new, style->newline,
-				plus_end - plus_begin, plus_begin,
-				line_prefix);
+				plus_end - plus_begin, plus_begin);
 	}
 
 	diff_words->current_plus = plus_end;
@@ -1314,11 +1326,12 @@ static void diff_words_show(struct diff_words_data *diff_words)
 
 	/* special case: only removal */
 	if (!diff_words->plus.text.size) {
-		fputs(line_prefix, diff_words->opt->file);
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+		emit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,
+				 line_prefix, strlen(line_prefix), 0);
+		fn_out_diff_words_write_helper(diff_words->opt,
 			&style->old, style->newline,
 			diff_words->minus.text.size,
-			diff_words->minus.text.ptr, line_prefix);
+			diff_words->minus.text.ptr);
 		diff_words->minus.text.size = 0;
 		return;
 	}
@@ -1341,12 +1354,12 @@ static void diff_words_show(struct diff_words_data *diff_words)
 	if (diff_words->current_plus != diff_words->plus.text.ptr +
 			diff_words->plus.text.size) {
 		if (color_words_output_graph_prefix(diff_words))
-			fputs(line_prefix, diff_words->opt->file);
-		fn_out_diff_words_write_helper(diff_words->opt->file,
+			emit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,
+					 line_prefix, strlen(line_prefix), 0);
+		fn_out_diff_words_write_helper(diff_words->opt,
 			&style->ctx, style->newline,
 			diff_words->plus.text.ptr + diff_words->plus.text.size
-			- diff_words->current_plus, diff_words->current_plus,
-			line_prefix);
+			- diff_words->current_plus, diff_words->current_plus);
 	}
 	diff_words->minus.text.size = diff_words->plus.text.size = 0;
 }
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 19/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (17 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 18/25] diff.c: convert word diffing " Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 20/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY Stefan Beller
                                               ` (5 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/diff.c b/diff.c
index 60630d2c3f..0ed86ba984 100644
--- a/diff.c
+++ b/diff.c
@@ -570,6 +570,7 @@ enum diff_symbol {
 	DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
 	DIFF_SYMBOL_STATS_LINE,
 	DIFF_SYMBOL_WORD_DIFF,
+	DIFF_SYMBOL_STAT_SEP,
 	DIFF_SYMBOL_SUBMODULE_ADD,
 	DIFF_SYMBOL_SUBMODULE_DEL,
 	DIFF_SYMBOL_SUBMODULE_UNTRACKED,
@@ -765,6 +766,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_WORD_DIFF:
 		fprintf(o->file, "%.*s", len, line);
 		break;
+	case DIFF_SYMBOL_STAT_SEP:
+		fputs(o->stat_sep, o->file);
+		break;
 	default:
 		die("BUG: unknown diff symbol");
 	}
@@ -5082,10 +5086,10 @@ void diff_flush(struct diff_options *options)
 	if (output_format & DIFF_FORMAT_PATCH) {
 		if (separator) {
 			emit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);
-			if (options->stat_sep) {
+			if (options->stat_sep)
 				/* attach patch instead of inline */
-				fputs(options->stat_sep, options->file);
-			}
+				emit_diff_symbol(options, DIFF_SYMBOL_STAT_SEP,
+						 NULL, 0, 0);
 		}
 
 		diff_flush_patch_all_file_pairs(options);
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 20/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (18 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 19/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 21/25] diff.c: buffer all output if asked to Stefan Beller
                                               ` (4 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 71 ++++++++++++++++++++++++++++++++++++++----------------------------
 1 file changed, 41 insertions(+), 30 deletions(-)

diff --git a/diff.c b/diff.c
index 0ed86ba984..f9fb94b0d3 100644
--- a/diff.c
+++ b/diff.c
@@ -571,6 +571,7 @@ enum diff_symbol {
 	DIFF_SYMBOL_STATS_LINE,
 	DIFF_SYMBOL_WORD_DIFF,
 	DIFF_SYMBOL_STAT_SEP,
+	DIFF_SYMBOL_SUMMARY,
 	DIFF_SYMBOL_SUBMODULE_ADD,
 	DIFF_SYMBOL_SUBMODULE_DEL,
 	DIFF_SYMBOL_SUBMODULE_UNTRACKED,
@@ -647,6 +648,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	case DIFF_SYMBOL_SUBMODULE_ERROR:
 	case DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:
 	case DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:
+	case DIFF_SYMBOL_SUMMARY:
 	case DIFF_SYMBOL_STATS_LINE:
 	case DIFF_SYMBOL_BINARY_DIFF_BODY:
 	case DIFF_SYMBOL_CONTEXT_FRAGINFO:
@@ -4720,67 +4722,76 @@ static void flush_one_pair(struct diff_filepair *p, struct diff_options *opt)
 	}
 }
 
-static void show_file_mode_name(FILE *file, const char *newdelete, struct diff_filespec *fs)
+static void show_file_mode_name(struct diff_options *opt, const char *newdelete, struct diff_filespec *fs)
 {
+	struct strbuf sb = STRBUF_INIT;
 	if (fs->mode)
-		fprintf(file, " %s mode %06o ", newdelete, fs->mode);
+		strbuf_addf(&sb, " %s mode %06o ", newdelete, fs->mode);
 	else
-		fprintf(file, " %s ", newdelete);
-	write_name_quoted(fs->path, file, '\n');
-}
+		strbuf_addf(&sb, " %s ", newdelete);
 
+	quote_c_style(fs->path, &sb, NULL, 0);
+	strbuf_addch(&sb, '\n');
+	emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
+			 sb.buf, sb.len, 0);
+	strbuf_release(&sb);
+}
 
-static void show_mode_change(FILE *file, struct diff_filepair *p, int show_name,
-		const char *line_prefix)
+static void show_mode_change(struct diff_options *opt, struct diff_filepair *p,
+		int show_name)
 {
 	if (p->one->mode && p->two->mode && p->one->mode != p->two->mode) {
-		fprintf(file, "%s mode change %06o => %06o%c", line_prefix, p->one->mode,
-			p->two->mode, show_name ? ' ' : '\n');
+		struct strbuf sb = STRBUF_INIT;
+		strbuf_addf(&sb, " mode change %06o => %06o",
+			    p->one->mode, p->two->mode);
 		if (show_name) {
-			write_name_quoted(p->two->path, file, '\n');
+			strbuf_addch(&sb, ' ');
+			quote_c_style(p->two->path, &sb, NULL, 0);
 		}
+		emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
+				 sb.buf, sb.len, 0);
+		strbuf_release(&sb);
 	}
 }
 
-static void show_rename_copy(FILE *file, const char *renamecopy, struct diff_filepair *p,
-			const char *line_prefix)
+static void show_rename_copy(struct diff_options *opt, const char *renamecopy,
+		struct diff_filepair *p)
 {
+	struct strbuf sb = STRBUF_INIT;
 	char *names = pprint_rename(p->one->path, p->two->path);
-
-	fprintf(file, " %s %s (%d%%)\n", renamecopy, names, similarity_index(p));
+	strbuf_addf(&sb, " %s %s (%d%%)\n",
+			renamecopy, names, similarity_index(p));
 	free(names);
-	show_mode_change(file, p, 0, line_prefix);
+	emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
+				 sb.buf, sb.len, 0);
+	show_mode_change(opt, p, 0);
 }
 
 static void diff_summary(struct diff_options *opt, struct diff_filepair *p)
 {
-	FILE *file = opt->file;
-	const char *line_prefix = diff_line_prefix(opt);
-
 	switch(p->status) {
 	case DIFF_STATUS_DELETED:
-		fputs(line_prefix, file);
-		show_file_mode_name(file, "delete", p->one);
+		show_file_mode_name(opt, "delete", p->one);
 		break;
 	case DIFF_STATUS_ADDED:
-		fputs(line_prefix, file);
-		show_file_mode_name(file, "create", p->two);
+		show_file_mode_name(opt, "create", p->two);
 		break;
 	case DIFF_STATUS_COPIED:
-		fputs(line_prefix, file);
-		show_rename_copy(file, "copy", p, line_prefix);
+		show_rename_copy(opt, "copy", p);
 		break;
 	case DIFF_STATUS_RENAMED:
-		fputs(line_prefix, file);
-		show_rename_copy(file, "rename", p, line_prefix);
+		show_rename_copy(opt, "rename", p);
 		break;
 	default:
 		if (p->score) {
-			fprintf(file, "%s rewrite ", line_prefix);
-			write_name_quoted(p->two->path, file, ' ');
-			fprintf(file, "(%d%%)\n", similarity_index(p));
+			struct strbuf sb = STRBUF_INIT;
+			strbuf_addstr(&sb, " rewrite ");
+			quote_c_style(p->two->path, &sb, NULL, 0);
+			strbuf_addf(&sb, " (%d%%)\n", similarity_index(p));
+			emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
+					 sb.buf, sb.len, 0);
 		}
-		show_mode_change(file, p, !p->score, line_prefix);
+		show_mode_change(opt, p, !p->score);
 		break;
 	}
 }
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 21/25] diff.c: buffer all output if asked to
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (19 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 20/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 22/25] diff.c: color moved lines differently Stefan Beller
                                               ` (3 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Introduce a new option 'emitted_symbols' in the struct diff_options which
controls whether all output is buffered up until all output is available.
It is set internally in diff.c when necessary.

We'll have a new struct 'emitted_string' in diff.c which will be used to
buffer each line.  The emitted_string will duplicate the memory of the
line to buffer as that is easiest to reason about for now. In a future
patch we may want to decrease the memory usage by not duplicating all
output for buffering but rather we may want to store offsets into the
file or in case of hunk descriptions such as the similarity score, we
could just store the relevant number and reproduce the text later on.

This approach was chosen as a first step because it is quite simple
compared to the alternative with less memory footprint.

emit_diff_symbol factors out the emission part and depending on the
diff_options->emitted_symbols the emission will be performed directly
when calling emit_diff_symbol or after the whole process is done, i.e.
by buffering we have add the possibility for a second pass over the
whole output before doing the actual output.

In 6440d34 (2012-03-14, diff: tweak a _copy_ of diff_options with
word-diff) we introduced a duplicate diff options struct for word
emissions as we may have different regex settings in there.
When buffering the output, we need to operate on just one buffer,
so we have to copy back the emissions of the word buffer into the
main buffer.

Unconditionally enable output via buffer in this patch as it yields
a great opportunity for testing, i.e. all the diff tests from the
test suite pass without having reordering issues (i.e. only parts
of the output got buffered, and we forgot to buffer other parts).
The test suite passes, which gives confidence that we converted all
functions to use emit_string for output.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 diff.h |   2 ++
 2 files changed, 109 insertions(+), 2 deletions(-)

diff --git a/diff.c b/diff.c
index f9fb94b0d3..b344808c20 100644
--- a/diff.c
+++ b/diff.c
@@ -604,6 +604,47 @@ enum diff_symbol {
 #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)
 #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)
 
+/*
+ * This struct is used when we need to buffer the output of the diff output.
+ *
+ * NEEDSWORK: Instead of storing a copy of the line, add an offset pointer
+ * into the pre/post image file. This pointer could be a union with the
+ * line pointer. By storing an offset into the file instead of the literal line,
+ * we can decrease the memory footprint for the buffered output. At first we
+ * may want to only have indirection for the content lines, but we could also
+ * enhance the state for emitting prefabricated lines, e.g. the similarity
+ * score line or hunk/file headers would only need to store a number or path
+ * and then the output can be constructed later on depending on state.
+ */
+struct emitted_diff_symbol {
+	const char *line;
+	int len;
+	int flags;
+	enum diff_symbol s;
+};
+#define EMITTED_DIFF_SYMBOL_INIT {NULL}
+
+struct emitted_diff_symbols {
+	struct emitted_diff_symbol *buf;
+	int nr, alloc;
+};
+#define EMITTED_DIFF_SYMBOLS_INIT {NULL, 0, 0}
+
+static void append_emitted_diff_symbol(struct diff_options *o,
+				       struct emitted_diff_symbol *e)
+{
+	struct emitted_diff_symbol *f;
+
+	ALLOC_GROW(o->emitted_symbols->buf,
+		   o->emitted_symbols->nr + 1,
+		   o->emitted_symbols->alloc);
+	f = &o->emitted_symbols->buf[o->emitted_symbols->nr++];
+
+	memcpy(f, e, sizeof(struct emitted_diff_symbol));
+	f->line = e->line ? xmemdupz(e->line, e->len) : NULL;
+}
+
+
 static void emit_line_ws_markup(struct diff_options *o,
 				const char *set, const char *reset,
 				const char *line, int len, char sign,
@@ -630,12 +671,18 @@ static void emit_line_ws_markup(struct diff_options *o,
 	}
 }
 
-static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
-			     const char *line, int len, unsigned flags)
+static void emit_diff_symbol_from_struct(struct diff_options *o,
+					 struct emitted_diff_symbol *eds)
 {
 	static const char *nneof = " No newline at end of file\n";
 	const char *context, *reset, *set, *meta, *fraginfo;
 	struct strbuf sb = STRBUF_INIT;
+
+	enum diff_symbol s = eds->s;
+	const char *line = eds->line;
+	int len = eds->len;
+	unsigned flags = eds->flags;
+
 	switch (s) {
 	case DIFF_SYMBOL_NO_LF_EOF:
 		context = diff_get_color_opt(o, DIFF_CONTEXT);
@@ -777,6 +824,17 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 	strbuf_release(&sb);
 }
 
+static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
+			     const char *line, int len, unsigned flags)
+{
+	struct emitted_diff_symbol e = {line, len, flags, s};
+
+	if (o->emitted_symbols)
+		append_emitted_diff_symbol(o, &e);
+	else
+		emit_diff_symbol_from_struct(o, &e);
+}
+
 void diff_emit_submodule_del(struct diff_options *o, const char *line)
 {
 	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);
@@ -1373,9 +1431,29 @@ static void diff_words_show(struct diff_words_data *diff_words)
 /* In "color-words" mode, show word-diff of words accumulated in the buffer */
 static void diff_words_flush(struct emit_callback *ecbdata)
 {
+	struct diff_options *wo = ecbdata->diff_words->opt;
+
 	if (ecbdata->diff_words->minus.text.size ||
 	    ecbdata->diff_words->plus.text.size)
 		diff_words_show(ecbdata->diff_words);
+
+	if (wo->emitted_symbols) {
+		struct diff_options *o = ecbdata->opt;
+		struct emitted_diff_symbols *wol = wo->emitted_symbols;
+		int i;
+
+		/*
+		 * NEEDSWORK:
+		 * Instead of appending each, concat all words to a line?
+		 */
+		for (i = 0; i < wol->nr; i++)
+			append_emitted_diff_symbol(o, &wol->buf[i]);
+
+		for (i = 0; i < wol->nr; i++)
+			free((void *)wol->buf[i].line);
+
+		wol->nr = 0;
+	}
 }
 
 static void diff_filespec_load_driver(struct diff_filespec *one)
@@ -1411,6 +1489,11 @@ static void init_diff_words_data(struct emit_callback *ecbdata,
 		xcalloc(1, sizeof(struct diff_words_data));
 	ecbdata->diff_words->type = o->word_diff;
 	ecbdata->diff_words->opt = o;
+
+	if (orig_opts->emitted_symbols)
+		o->emitted_symbols =
+			xcalloc(1, sizeof(struct emitted_diff_symbols));
+
 	if (!o->word_regex)
 		o->word_regex = userdiff_word_regex(one);
 	if (!o->word_regex)
@@ -1445,6 +1528,7 @@ static void free_diff_words_data(struct emit_callback *ecbdata)
 {
 	if (ecbdata->diff_words) {
 		diff_words_flush(ecbdata);
+		free (ecbdata->diff_words->opt->emitted_symbols);
 		free (ecbdata->diff_words->opt);
 		free (ecbdata->diff_words->minus.text.ptr);
 		free (ecbdata->diff_words->minus.orig);
@@ -4999,16 +5083,37 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)
 static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 {
 	int i;
+	static struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;
 	struct diff_queue_struct *q = &diff_queued_diff;
 
 	if (WSEH_NEW & WS_RULE_MASK)
 		die("BUG: WS rules bit mask overlaps with diff symbol flags");
 
+	/*
+	 * For testing purposes we want to make sure the diff machinery
+	 * works completely with the buffer. If there is anything emitted
+	 * outside the emit_string, then the order is screwed
+	 * up and the tests will fail.
+	 *
+	 * TODO (later in this series):
+	 * We'll unset this pointer in a later patch.
+	 */
+	o->emitted_symbols = &esm;
+
 	for (i = 0; i < q->nr; i++) {
 		struct diff_filepair *p = q->queue[i];
 		if (check_pair_status(p))
 			diff_flush_patch(p, o);
 	}
+
+	if (o->emitted_symbols) {
+		for (i = 0; i < esm.nr; i++)
+			emit_diff_symbol_from_struct(o, &esm.buf[i]);
+
+		for (i = 0; i < esm.nr; i++)
+			free((void *)esm.buf[i].line);
+	}
+	esm.nr = 0;
 }
 
 void diff_flush(struct diff_options *options)
diff --git a/diff.h b/diff.h
index b73f7a1e75..65fc9dbb4b 100644
--- a/diff.h
+++ b/diff.h
@@ -186,6 +186,8 @@ struct diff_options {
 	void *output_prefix_data;
 
 	int diff_path_counter;
+
+	struct emitted_diff_symbols *emitted_symbols;
 };
 
 void diff_emit_submodule_del(struct diff_options *o, const char *line);
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 22/25] diff.c: color moved lines differently
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (20 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 21/25] diff.c: buffer all output if asked to Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 23/25] diff.c: color moved lines differently, plain mode Stefan Beller
                                               ` (2 subsequent siblings)
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

When a patch consists mostly of moving blocks of code around, it can
be quite tedious to ensure that the blocks are moved verbatim, and not
undesirably modified in the move. To that end, color blocks that are
moved within the same patch differently. For example (OM, del, add,
and NM are different colors):

    [OM]  -void sensitive_stuff(void)
    [OM]  -{
    [OM]  -        if (!is_authorized_user())
    [OM]  -                die("unauthorized");
    [OM]  -        sensitive_stuff(spanning,
    [OM]  -                        multiple,
    [OM]  -                        lines);
    [OM]  -}

           void another_function()
           {
    [del] -        printf("foo");
    [add] +        printf("bar");
           }

    [NM]  +void sensitive_stuff(void)
    [NM]  +{
    [NM]  +        if (!is_authorized_user())
    [NM]  +                die("unauthorized");
    [NM]  +        sensitive_stuff(spanning,
    [NM]  +                        multiple,
    [NM]  +                        lines);
    [NM]  +}

However adjacent blocks may be problematic. For example, in this
potentially malicious patch, the swapping of blocks can be spotted:

    [OM]  -void sensitive_stuff(void)
    [OM]  -{
    [OMA] -        if (!is_authorized_user())
    [OMA] -                die("unauthorized");
    [OM]  -        sensitive_stuff(spanning,
    [OM]  -                        multiple,
    [OM]  -                        lines);
    [OMA] -}

           void another_function()
           {
    [del] -        printf("foo");
    [add] +        printf("bar");
           }

    [NM]  +void sensitive_stuff(void)
    [NM]  +{
    [NMA] +        sensitive_stuff(spanning,
    [NMA] +                        multiple,
    [NMA] +                        lines);
    [NM]  +        if (!is_authorized_user())
    [NM]  +                die("unauthorized");
    [NMA] +}

If the moved code is larger, it is easier to hide some permutation in the
code, which is why some alternative coloring is needed.

This patch implements the first mode:
* basic alternating 'Zebra' mode
  This conveys all information needed to the user.  Defer customization to
  later patches.

First I implemented an alternative design, which would try to fingerprint
a line by its neighbors to detect if we are in a block or at the boundary.
This idea iss error prone as it inspected each line and its neighboring
lines to determine if the line was (a) moved and (b) if was deep inside
a hunk by having matching neighboring lines. This is unreliable as the
we can construct hunks which have equal neighbors that just exceed the
number of lines inspected. (Think of 'AXYZBXYZCXYZD..' with each letter
as a line, that is permutated to AXYZCXYZBXYZD..').

Instead this provides a dynamic programming greedy algorithm that finds
the largest moved hunk and then has several modes on highlighting bounds.

A note on the options '--submodule=diff' and '--color-words/--word-diff':
In the conversion to use emit_line in the prior patches both submodules
as well as word diff output carefully chose to call emit_line with sign=0.
All output with sign=0 is ignored for move detection purposes in this
patch, such that no weird looking output will be generated for these
cases. This leads to another thought: We could pass on '--color-moved' to
submodules such that they color up moved lines for themselves. If we'd do
so only line moves within a repository boundary are marked up.

Helped-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 diff.c                     | 303 ++++++++++++++++++++++++++++++++++++++++++---
 diff.h                     |  10 +-
 t/t4015-diff-whitespace.sh | 196 +++++++++++++++++++++++++++++
 3 files changed, 494 insertions(+), 15 deletions(-)

diff --git a/diff.c b/diff.c
index b344808c20..e6f547804f 100644
--- a/diff.c
+++ b/diff.c
@@ -15,6 +15,7 @@
 #include "userdiff.h"
 #include "submodule-config.h"
 #include "submodule.h"
+#include "hashmap.h"
 #include "ll-merge.h"
 #include "string-list.h"
 #include "argv-array.h"
@@ -31,6 +32,7 @@ static int diff_indent_heuristic; /* experimental */
 static int diff_rename_limit_default = 400;
 static int diff_suppress_blank_empty;
 static int diff_use_color_default = -1;
+static int diff_color_moved_default;
 static int diff_context_default = 3;
 static int diff_interhunk_context_default;
 static const char *diff_word_regex_cfg;
@@ -55,6 +57,10 @@ static char diff_colors[][COLOR_MAXLEN] = {
 	GIT_COLOR_YELLOW,	/* COMMIT */
 	GIT_COLOR_BG_RED,	/* WHITESPACE */
 	GIT_COLOR_NORMAL,	/* FUNCINFO */
+	GIT_COLOR_MAGENTA,	/* OLD_MOVED */
+	GIT_COLOR_BLUE,		/* OLD_MOVED ALTERNATIVE */
+	GIT_COLOR_CYAN,		/* NEW_MOVED */
+	GIT_COLOR_YELLOW,	/* NEW_MOVED ALTERNATIVE */
 };
 
 static NORETURN void die_want_option(const char *option_name)
@@ -80,6 +86,14 @@ static int parse_diff_color_slot(const char *var)
 		return DIFF_WHITESPACE;
 	if (!strcasecmp(var, "func"))
 		return DIFF_FUNCINFO;
+	if (!strcasecmp(var, "oldmoved"))
+		return DIFF_FILE_OLD_MOVED;
+	if (!strcasecmp(var, "oldmovedalternative"))
+		return DIFF_FILE_OLD_MOVED_ALT;
+	if (!strcasecmp(var, "newmoved"))
+		return DIFF_FILE_NEW_MOVED;
+	if (!strcasecmp(var, "newmovedalternative"))
+		return DIFF_FILE_NEW_MOVED_ALT;
 	return -1;
 }
 
@@ -228,12 +242,29 @@ int git_diff_heuristic_config(const char *var, const char *value, void *cb)
 	return 0;
 }
 
+static int parse_color_moved(const char *arg)
+{
+	if (!strcmp(arg, "no"))
+		return COLOR_MOVED_NO;
+	else if (!strcmp(arg, "zebra"))
+		return COLOR_MOVED_ZEBRA;
+	else
+		return -1;
+}
+
 int git_diff_ui_config(const char *var, const char *value, void *cb)
 {
 	if (!strcmp(var, "diff.color") || !strcmp(var, "color.diff")) {
 		diff_use_color_default = git_config_colorbool(var, value);
 		return 0;
 	}
+	if (!strcmp(var, "diff.colormoved")) {
+		int cm = parse_color_moved(value);
+		if (cm < 0)
+			return -1;
+		diff_color_moved_default = cm;
+		return 0;
+	}
 	if (!strcmp(var, "diff.context")) {
 		diff_context_default = git_config_int(var, value);
 		if (diff_context_default < 0)
@@ -601,7 +632,9 @@ enum diff_symbol {
  * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT
  * 16 is marking if the line is blank at EOF
  */
-#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)
+#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF	(1<<16)
+#define DIFF_SYMBOL_MOVED_LINE			(1<<17)
+#define DIFF_SYMBOL_MOVED_LINE_ALT		(1<<18)
 #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)
 
 /*
@@ -644,6 +677,213 @@ static void append_emitted_diff_symbol(struct diff_options *o,
 	f->line = e->line ? xmemdupz(e->line, e->len) : NULL;
 }
 
+struct moved_entry {
+	struct hashmap_entry ent;
+	const struct emitted_diff_symbol *es;
+	struct moved_entry *next_line;
+};
+
+static void get_ws_cleaned_string(const struct emitted_diff_symbol *l,
+				  struct strbuf *out)
+{
+	int i;
+	for (i = 0; i < l->len; i++) {
+		if (isspace(l->line[i]))
+			continue;
+		strbuf_addch(out, l->line[i]);
+	}
+}
+
+static int emitted_symbol_cmp_no_ws(const struct emitted_diff_symbol *a,
+				    const struct emitted_diff_symbol *b,
+				    const void *keydata)
+{
+	int ret;
+	struct strbuf sba = STRBUF_INIT;
+	struct strbuf sbb = STRBUF_INIT;
+
+	get_ws_cleaned_string(a, &sba);
+	get_ws_cleaned_string(b, &sbb);
+	ret = sba.len != sbb.len || strncmp(sba.buf, sbb.buf, sba.len);
+
+	strbuf_release(&sba);
+	strbuf_release(&sbb);
+	return ret;
+}
+
+static int emitted_symbol_cmp(const struct emitted_diff_symbol *a,
+			      const struct emitted_diff_symbol *b,
+			      const void *keydata)
+{
+	return a->len != b->len || strncmp(a->line, b->line, a->len);
+}
+
+static int moved_entry_cmp(const struct moved_entry *a,
+			   const struct moved_entry *b,
+			   const void *keydata)
+{
+	return emitted_symbol_cmp(a->es, b->es, keydata);
+}
+
+static int moved_entry_cmp_no_ws(const struct moved_entry *a,
+				 const struct moved_entry *b,
+				 const void *keydata)
+{
+	return emitted_symbol_cmp_no_ws(a->es, b->es, keydata);
+}
+
+static unsigned get_string_hash(struct emitted_diff_symbol *es, unsigned ignore_ws)
+{
+	static struct strbuf sb = STRBUF_INIT;
+
+	if (ignore_ws) {
+		strbuf_reset(&sb);
+		get_ws_cleaned_string(es, &sb);
+		return memhash(sb.buf, sb.len);
+	} else {
+		return memhash(es->line, es->len);
+	}
+}
+
+static struct moved_entry *prepare_entry(struct diff_options *o,
+					 int line_no)
+{
+	struct moved_entry *ret = xmalloc(sizeof(*ret));
+	unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
+	struct emitted_diff_symbol *l = &o->emitted_symbols->buf[line_no];
+
+	ret->ent.hash = get_string_hash(l, ignore_ws);
+	ret->es = l;
+	ret->next_line = NULL;
+
+	return ret;
+}
+
+static void add_lines_to_move_detection(struct diff_options *o,
+					struct hashmap *add_lines,
+					struct hashmap *del_lines)
+{
+	struct moved_entry *prev_line = NULL;
+
+	int n;
+	for (n = 0; n < o->emitted_symbols->nr; n++) {
+		struct hashmap *hm;
+		struct moved_entry *key;
+
+		switch (o->emitted_symbols->buf[n].s) {
+		case DIFF_SYMBOL_PLUS:
+			hm = add_lines;
+			break;
+		case DIFF_SYMBOL_MINUS:
+			hm = del_lines;
+			break;
+		default:
+			prev_line = NULL;
+			continue;
+		}
+
+		key = prepare_entry(o, n);
+		if (prev_line && prev_line->es->s == o->emitted_symbols->buf[n].s)
+			prev_line->next_line = key;
+
+		hashmap_add(hm, key);
+		prev_line = key;
+	}
+}
+
+/* Find blocks of moved code, delegate actual coloring decision to helper */
+static void mark_color_as_moved(struct diff_options *o,
+				struct hashmap *add_lines,
+				struct hashmap *del_lines)
+{
+	struct moved_entry **pmb = NULL; /* potentially moved blocks */
+	int pmb_nr = 0, pmb_alloc = 0;
+	int n, flipped_block = 1;
+
+	for (n = 0; n < o->emitted_symbols->nr; n++) {
+		struct hashmap *hm = NULL;
+		struct moved_entry *key;
+		struct moved_entry *match = NULL;
+		struct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];
+		int i, lp, rp;
+
+		switch (l->s) {
+		case DIFF_SYMBOL_PLUS:
+			hm = del_lines;
+			key = prepare_entry(o, n);
+			match = hashmap_get(hm, key, o);
+			free(key);
+			break;
+		case DIFF_SYMBOL_MINUS:
+			hm = add_lines;
+			key = prepare_entry(o, n);
+			match = hashmap_get(hm, key, o);
+			free(key);
+			break;
+		default:
+			flipped_block = 1;
+		}
+
+		if (!match) {
+			pmb_nr = 0;
+			continue;
+		}
+
+		l->flags |= DIFF_SYMBOL_MOVED_LINE;
+
+		/* Check any potential block runs, advance each or nullify */
+		for (i = 0; i < pmb_nr; i++) {
+			struct moved_entry *p = pmb[i];
+			struct moved_entry *pnext = (p && p->next_line) ?
+					p->next_line : NULL;
+			if (pnext &&
+			    !emitted_symbol_cmp(pnext->es, l, o)) {
+				pmb[i] = p->next_line;
+			} else {
+				pmb[i] = NULL;
+			}
+		}
+
+		/* Shrink the set of potential block to the remaining running */
+		for (lp = 0, rp = pmb_nr - 1; lp <= rp;) {
+			while (lp < pmb_nr && pmb[lp])
+				lp++;
+			/* lp points at the first NULL now */
+
+			while (rp > -1 && !pmb[rp])
+				rp--;
+			/* rp points at the last non-NULL */
+
+			if (lp < pmb_nr && rp > -1 && lp < rp) {
+				pmb[lp] = pmb[rp];
+				pmb[rp] = NULL;
+				rp--;
+				lp++;
+			}
+		}
+
+		/* Remember the number of running sets */
+		pmb_nr = rp + 1;
+
+		if (pmb_nr == 0) {
+			/*
+			 * The current line is the start of a new block.
+			 * Setup the set of potential blocks.
+			 */
+			for (; match; match = hashmap_get_next(hm, match)) {
+				ALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);
+				pmb[pmb_nr++] = match;
+			}
+
+			flipped_block = (flipped_block + 1) % 2;
+		}
+
+		if (flipped_block)
+			l->flags |= DIFF_SYMBOL_MOVED_LINE_ALT;
+	}
+
+	free(pmb);
+}
 
 static void emit_line_ws_markup(struct diff_options *o,
 				const char *set, const char *reset,
@@ -719,14 +959,24 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
 				    flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);
 		break;
 	case DIFF_SYMBOL_PLUS:
-		set = diff_get_color_opt(o, DIFF_FILE_NEW);
+		if (flags & DIFF_SYMBOL_MOVED_LINE_ALT)
+			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);
+		else if (flags & DIFF_SYMBOL_MOVED_LINE)
+			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);
+		else
+			set = diff_get_color_opt(o, DIFF_FILE_NEW);
 		reset = diff_get_color_opt(o, DIFF_RESET);
 		emit_line_ws_markup(o, set, reset, line, len, '+',
 				    flags & DIFF_SYMBOL_CONTENT_WS_MASK,
 				    flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);
 		break;
 	case DIFF_SYMBOL_MINUS:
-		set = diff_get_color_opt(o, DIFF_FILE_OLD);
+		if (flags & DIFF_SYMBOL_MOVED_LINE_ALT)
+			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);
+		else if (flags & DIFF_SYMBOL_MOVED_LINE)
+			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);
+		else
+			set = diff_get_color_opt(o, DIFF_FILE_OLD);
 		reset = diff_get_color_opt(o, DIFF_RESET);
 		emit_line_ws_markup(o, set, reset, line, len, '-',
 				    flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
@@ -3741,6 +3991,8 @@ void diff_setup(struct diff_options *options)
 		options->a_prefix = "a/";
 		options->b_prefix = "b/";
 	}
+
+	options->color_moved = diff_color_moved_default;
 }
 
 void diff_setup_done(struct diff_options *options)
@@ -3850,6 +4102,9 @@ void diff_setup_done(struct diff_options *options)
 
 	if (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)
 		die(_("--follow requires exactly one pathspec"));
+
+	if (!options->use_color || external_diff())
+		options->color_moved = 0;
 }
 
 static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)
@@ -4274,7 +4529,16 @@ int diff_opt_parse(struct diff_options *options,
 	}
 	else if (!strcmp(arg, "--no-color"))
 		options->use_color = 0;
-	else if (!strcmp(arg, "--color-words")) {
+	else if (!strcmp(arg, "--color-moved"))
+		options->color_moved = COLOR_MOVED_ZEBRA;
+	else if (!strcmp(arg, "--no-color-moved"))
+		options->color_moved = COLOR_MOVED_NO;
+	else if (skip_prefix(arg, "--color-moved=", &arg)) {
+		int cm = parse_color_moved(arg);
+		if (cm < 0)
+			die("bad --color-moved argument: %s", arg);
+		options->color_moved = cm;
+	} else if (!strcmp(arg, "--color-words")) {
 		options->use_color = 1;
 		options->word_diff = DIFF_WORDS_COLOR;
 	}
@@ -5089,16 +5353,8 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 	if (WSEH_NEW & WS_RULE_MASK)
 		die("BUG: WS rules bit mask overlaps with diff symbol flags");
 
-	/*
-	 * For testing purposes we want to make sure the diff machinery
-	 * works completely with the buffer. If there is anything emitted
-	 * outside the emit_string, then the order is screwed
-	 * up and the tests will fail.
-	 *
-	 * TODO (later in this series):
-	 * We'll unset this pointer in a later patch.
-	 */
-	o->emitted_symbols = &esm;
+	if (o->color_moved)
+		o->emitted_symbols = &esm;
 
 	for (i = 0; i < q->nr; i++) {
 		struct diff_filepair *p = q->queue[i];
@@ -5107,6 +5363,24 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 	}
 
 	if (o->emitted_symbols) {
+		if (o->color_moved) {
+			struct hashmap add_lines, del_lines;
+			unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
+
+			hashmap_init(&del_lines, ignore_ws ?
+				(hashmap_cmp_fn)moved_entry_cmp_no_ws :
+				(hashmap_cmp_fn)moved_entry_cmp, 0);
+			hashmap_init(&add_lines, ignore_ws ?
+				(hashmap_cmp_fn)moved_entry_cmp_no_ws :
+				(hashmap_cmp_fn)moved_entry_cmp, 0);
+
+			add_lines_to_move_detection(o, &add_lines, &del_lines);
+			mark_color_as_moved(o, &add_lines, &del_lines);
+
+			hashmap_free(&add_lines, 0);
+			hashmap_free(&del_lines, 0);
+		}
+
 		for (i = 0; i < esm.nr; i++)
 			emit_diff_symbol_from_struct(o, &esm.buf[i]);
 
@@ -5190,6 +5464,7 @@ void diff_flush(struct diff_options *options)
 		if (!options->file)
 			die_errno("Could not open /dev/null");
 		options->close_file = 1;
+		options->color_moved = 0;
 		for (i = 0; i < q->nr; i++) {
 			struct diff_filepair *p = q->queue[i];
 			if (check_pair_status(p))
diff --git a/diff.h b/diff.h
index 65fc9dbb4b..7726ad255c 100644
--- a/diff.h
+++ b/diff.h
@@ -188,6 +188,10 @@ struct diff_options {
 	int diff_path_counter;
 
 	struct emitted_diff_symbols *emitted_symbols;
+	enum {
+		COLOR_MOVED_NO = 0,
+		COLOR_MOVED_ZEBRA = 2,
+	} color_moved;
 };
 
 void diff_emit_submodule_del(struct diff_options *o, const char *line);
@@ -208,7 +212,11 @@ enum color_diff {
 	DIFF_FILE_NEW = 5,
 	DIFF_COMMIT = 6,
 	DIFF_WHITESPACE = 7,
-	DIFF_FUNCINFO = 8
+	DIFF_FUNCINFO = 8,
+	DIFF_FILE_OLD_MOVED = 9,
+	DIFF_FILE_OLD_MOVED_ALT = 10,
+	DIFF_FILE_NEW_MOVED = 11,
+	DIFF_FILE_NEW_MOVED_ALT = 12
 };
 const char *diff_get_color(int diff_use_color, enum color_diff ix);
 #define diff_get_color_opt(o, ix) \
diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
index 289806d0c7..4a03766f1f 100755
--- a/t/t4015-diff-whitespace.sh
+++ b/t/t4015-diff-whitespace.sh
@@ -972,4 +972,200 @@ test_expect_success 'option overrides diff.wsErrorHighlight' '
 
 '
 
+test_expect_success 'detect moved code, complete file' '
+	git reset --hard &&
+	cat <<-\EOF >test.c &&
+	#include<stdio.h>
+	main()
+	{
+	printf("Hello World");
+	}
+	EOF
+	git add test.c &&
+	git commit -m "add main function" &&
+	git mv test.c main.c &&
+	test_config color.diff.oldMoved "normal red" &&
+	test_config color.diff.newMoved "normal green" &&
+	git diff HEAD --color-moved --no-renames | test_decode_color >actual &&
+	cat >expected <<-\EOF &&
+	<BOLD>diff --git a/main.c b/main.c<RESET>
+	<BOLD>new file mode 100644<RESET>
+	<BOLD>index 0000000..a986c57<RESET>
+	<BOLD>--- /dev/null<RESET>
+	<BOLD>+++ b/main.c<RESET>
+	<CYAN>@@ -0,0 +1,5 @@<RESET>
+	<BGREEN>+<RESET><BGREEN>#include<stdio.h><RESET>
+	<BGREEN>+<RESET><BGREEN>main()<RESET>
+	<BGREEN>+<RESET><BGREEN>{<RESET>
+	<BGREEN>+<RESET><BGREEN>printf("Hello World");<RESET>
+	<BGREEN>+<RESET><BGREEN>}<RESET>
+	<BOLD>diff --git a/test.c b/test.c<RESET>
+	<BOLD>deleted file mode 100644<RESET>
+	<BOLD>index a986c57..0000000<RESET>
+	<BOLD>--- a/test.c<RESET>
+	<BOLD>+++ /dev/null<RESET>
+	<CYAN>@@ -1,5 +0,0 @@<RESET>
+	<BRED>-#include<stdio.h><RESET>
+	<BRED>-main()<RESET>
+	<BRED>-{<RESET>
+	<BRED>-printf("Hello World");<RESET>
+	<BRED>-}<RESET>
+	EOF
+
+	test_cmp expected actual
+'
+
+test_expect_success 'detect malicious moved code, inside file' '
+	test_config color.diff.oldMoved "normal red" &&
+	test_config color.diff.newMoved "normal green" &&
+	test_config color.diff.oldMovedAlternative "blue" &&
+	test_config color.diff.newMovedAlternative "yellow" &&
+	git reset --hard &&
+	cat <<-\EOF >main.c &&
+		#include<stdio.h>
+		int stuff()
+		{
+			printf("Hello ");
+			printf("World\n");
+		}
+
+		int secure_foo(struct user *u)
+		{
+			if (!u->is_allowed_foo)
+				return;
+			foo(u);
+		}
+
+		int main()
+		{
+			foo();
+		}
+	EOF
+	cat <<-\EOF >test.c &&
+		#include<stdio.h>
+		int bar()
+		{
+			printf("Hello World, but different\n");
+		}
+
+		int another_function()
+		{
+			bar();
+		}
+	EOF
+	git add main.c test.c &&
+	git commit -m "add main and test file" &&
+	cat <<-\EOF >main.c &&
+		#include<stdio.h>
+		int stuff()
+		{
+			printf("Hello ");
+			printf("World\n");
+		}
+
+		int main()
+		{
+			foo();
+		}
+	EOF
+	cat <<-\EOF >test.c &&
+		#include<stdio.h>
+		int bar()
+		{
+			printf("Hello World, but different\n");
+		}
+
+		int secure_foo(struct user *u)
+		{
+			foo(u);
+			if (!u->is_allowed_foo)
+				return;
+		}
+
+		int another_function()
+		{
+			bar();
+		}
+	EOF
+	git diff HEAD --no-renames --color-moved=zebra| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/main.c b/main.c<RESET>
+	<BOLD>index 27a619c..7cf9336 100644<RESET>
+	<BOLD>--- a/main.c<RESET>
+	<BOLD>+++ b/main.c<RESET>
+	<CYAN>@@ -5,13 +5,6 @@<RESET> <RESET>printf("Hello ");<RESET>
+	 printf("World\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BRED>-int secure_foo(struct user *u)<RESET>
+	<BRED>-{<RESET>
+	<BLUE>-if (!u->is_allowed_foo)<RESET>
+	<BLUE>-return;<RESET>
+	<BRED>-foo(u);<RESET>
+	<BLUE>-}<RESET>
+	<BLUE>-<RESET>
+	 int main()<RESET>
+	 {<RESET>
+	 foo();<RESET>
+	<BOLD>diff --git a/test.c b/test.c<RESET>
+	<BOLD>index 1dc1d85..2bedec9 100644<RESET>
+	<BOLD>--- a/test.c<RESET>
+	<BOLD>+++ b/test.c<RESET>
+	<CYAN>@@ -4,6 +4,13 @@<RESET> <RESET>int bar()<RESET>
+	 printf("Hello World, but different\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BGREEN>+<RESET><BGREEN>int secure_foo(struct user *u)<RESET>
+	<BGREEN>+<RESET><BGREEN>{<RESET>
+	<YELLOW>+<RESET><YELLOW>foo(u);<RESET>
+	<BGREEN>+<RESET><BGREEN>if (!u->is_allowed_foo)<RESET>
+	<BGREEN>+<RESET><BGREEN>return;<RESET>
+	<YELLOW>+<RESET><YELLOW>}<RESET>
+	<YELLOW>+<RESET>
+	 int another_function()<RESET>
+	 {<RESET>
+	 bar();<RESET>
+	EOF
+
+	test_cmp expected actual
+'
+
+test_expect_success 'no effect from --color-moved with --word-diff' '
+	cat <<-\EOF >text.txt &&
+	Lorem Ipsum is simply dummy text of the printing and typesetting industry.
+	EOF
+	git add text.txt &&
+	git commit -a -m "clean state" &&
+	cat <<-\EOF >text.txt &&
+	simply Lorem Ipsum dummy is text of the typesetting and printing industry.
+	EOF
+	git diff --color-moved --word-diff >actual &&
+	git diff --word-diff >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'move detection with submodules' '
+	test_create_repo bananas &&
+	echo ripe >bananas/recipe &&
+	git -C bananas add recipe &&
+	test_commit fruit &&
+	test_commit -C bananas recipe &&
+	git submodule add ./bananas &&
+	git add bananas &&
+	git commit -a -m "bananas are like a heavy library?" &&
+	echo foul >bananas/recipe &&
+	echo ripe >fruit.t &&
+
+	git diff --submodule=diff --color-moved >actual &&
+
+	# no move detection as the moved line is across repository boundaries.
+	test_decode_color <actual >decoded_actual &&
+	! grep BGREEN decoded_actual &&
+	! grep BRED decoded_actual &&
+
+	# nor did we mess with it another way
+	git diff --submodule=diff | test_decode_color >expect &&
+	test_cmp expect decoded_actual
+'
+
 test_done
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 23/25] diff.c: color moved lines differently, plain mode
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (21 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 22/25] diff.c: color moved lines differently Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 24/25] diff.c: add dimming to moved line detection Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 25/25] diff: document the new --color-moved setting Stefan Beller
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Add the 'plain' mode for move detection of code.
This omits the checking for adjacent blocks, so it is not as useful.
If you have a lot of the same blocks moved in the same patch, the 'Zebra'
would end up slow as it is O(n^2) (n is number of same blocks).
So this may be useful there and is generally easy to add

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 diff.c                     |  5 +++++
 diff.h                     |  1 +
 t/t4015-diff-whitespace.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 56 insertions(+), 1 deletion(-)

diff --git a/diff.c b/diff.c
index e6f547804f..0eb744514f 100644
--- a/diff.c
+++ b/diff.c
@@ -246,6 +246,8 @@ static int parse_color_moved(const char *arg)
 {
 	if (!strcmp(arg, "no"))
 		return COLOR_MOVED_NO;
+	else if (!strcmp(arg, "plain"))
+		return COLOR_MOVED_PLAIN;
 	else if (!strcmp(arg, "zebra"))
 		return COLOR_MOVED_ZEBRA;
 	else
@@ -831,6 +833,9 @@ static void mark_color_as_moved(struct diff_options *o,
 
 		l->flags |= DIFF_SYMBOL_MOVED_LINE;
 
+		if (o->color_moved == COLOR_MOVED_PLAIN)
+			continue;
+
 		/* Check any potential block runs, advance each or nullify */
 		for (i = 0; i < pmb_nr; i++) {
 			struct moved_entry *p = pmb[i];
diff --git a/diff.h b/diff.h
index 7726ad255c..1aae8738ca 100644
--- a/diff.h
+++ b/diff.h
@@ -190,6 +190,7 @@ struct diff_options {
 	struct emitted_diff_symbols *emitted_symbols;
 	enum {
 		COLOR_MOVED_NO = 0,
+		COLOR_MOVED_PLAIN = 1,
 		COLOR_MOVED_ZEBRA = 2,
 	} color_moved;
 };
diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
index 4a03766f1f..1ca16435d6 100755
--- a/t/t4015-diff-whitespace.sh
+++ b/t/t4015-diff-whitespace.sh
@@ -986,7 +986,7 @@ test_expect_success 'detect moved code, complete file' '
 	git mv test.c main.c &&
 	test_config color.diff.oldMoved "normal red" &&
 	test_config color.diff.newMoved "normal green" &&
-	git diff HEAD --color-moved --no-renames | test_decode_color >actual &&
+	git diff HEAD --color-moved=zebra --no-renames | test_decode_color >actual &&
 	cat >expected <<-\EOF &&
 	<BOLD>diff --git a/main.c b/main.c<RESET>
 	<BOLD>new file mode 100644<RESET>
@@ -1130,6 +1130,55 @@ test_expect_success 'detect malicious moved code, inside file' '
 	test_cmp expected actual
 '
 
+test_expect_success 'plain moved code, inside file' '
+	test_config color.diff.oldMoved "normal red" &&
+	test_config color.diff.newMoved "normal green" &&
+	test_config color.diff.oldMovedAlternative "blue" &&
+	test_config color.diff.newMovedAlternative "yellow" &&
+	# needs previous test as setup
+	git diff HEAD --no-renames --color-moved=plain| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/main.c b/main.c<RESET>
+	<BOLD>index 27a619c..7cf9336 100644<RESET>
+	<BOLD>--- a/main.c<RESET>
+	<BOLD>+++ b/main.c<RESET>
+	<CYAN>@@ -5,13 +5,6 @@<RESET> <RESET>printf("Hello ");<RESET>
+	 printf("World\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BRED>-int secure_foo(struct user *u)<RESET>
+	<BRED>-{<RESET>
+	<BRED>-if (!u->is_allowed_foo)<RESET>
+	<BRED>-return;<RESET>
+	<BRED>-foo(u);<RESET>
+	<BRED>-}<RESET>
+	<BRED>-<RESET>
+	 int main()<RESET>
+	 {<RESET>
+	 foo();<RESET>
+	<BOLD>diff --git a/test.c b/test.c<RESET>
+	<BOLD>index 1dc1d85..2bedec9 100644<RESET>
+	<BOLD>--- a/test.c<RESET>
+	<BOLD>+++ b/test.c<RESET>
+	<CYAN>@@ -4,6 +4,13 @@<RESET> <RESET>int bar()<RESET>
+	 printf("Hello World, but different\n");<RESET>
+	 }<RESET>
+	 <RESET>
+	<BGREEN>+<RESET><BGREEN>int secure_foo(struct user *u)<RESET>
+	<BGREEN>+<RESET><BGREEN>{<RESET>
+	<BGREEN>+<RESET><BGREEN>foo(u);<RESET>
+	<BGREEN>+<RESET><BGREEN>if (!u->is_allowed_foo)<RESET>
+	<BGREEN>+<RESET><BGREEN>return;<RESET>
+	<BGREEN>+<RESET><BGREEN>}<RESET>
+	<BGREEN>+<RESET>
+	 int another_function()<RESET>
+	 {<RESET>
+	 bar();<RESET>
+	EOF
+
+	test_cmp expected actual
+'
+
 test_expect_success 'no effect from --color-moved with --word-diff' '
 	cat <<-\EOF >text.txt &&
 	Lorem Ipsum is simply dummy text of the printing and typesetting industry.
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 24/25] diff.c: add dimming to moved line detection
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (22 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 23/25] diff.c: color moved lines differently, plain mode Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  2017-06-23  1:29                             ` [PATCHv2 25/25] diff: document the new --color-moved setting Stefan Beller
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Any lines inside a moved block of code are not interesting. Boundaries
of blocks are only interesting if they are next to another block of moved
code.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 color.h                    |   2 +
 diff.c                     | 139 ++++++++++++++++++++++++++++++++++++++++-----
 diff.h                     |   9 ++-
 t/t4015-diff-whitespace.sh | 124 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 259 insertions(+), 15 deletions(-)

diff --git a/color.h b/color.h
index 90627650fc..fd2b688dfb 100644
--- a/color.h
+++ b/color.h
@@ -42,6 +42,8 @@ struct strbuf;
 #define GIT_COLOR_BG_BLUE	"\033[44m"
 #define GIT_COLOR_BG_MAGENTA	"\033[45m"
 #define GIT_COLOR_BG_CYAN	"\033[46m"
+#define GIT_COLOR_FAINT		"\033[2m"
+#define GIT_COLOR_FAINT_ITALIC	"\033[2;3m"
 
 /* A special value meaning "no color selected" */
 #define GIT_COLOR_NIL "NIL"
diff --git a/diff.c b/diff.c
index 0eb744514f..82ace48c38 100644
--- a/diff.c
+++ b/diff.c
@@ -57,10 +57,14 @@ static char diff_colors[][COLOR_MAXLEN] = {
 	GIT_COLOR_YELLOW,	/* COMMIT */
 	GIT_COLOR_BG_RED,	/* WHITESPACE */
 	GIT_COLOR_NORMAL,	/* FUNCINFO */
-	GIT_COLOR_MAGENTA,	/* OLD_MOVED */
-	GIT_COLOR_BLUE,		/* OLD_MOVED ALTERNATIVE */
-	GIT_COLOR_CYAN,		/* NEW_MOVED */
-	GIT_COLOR_YELLOW,	/* NEW_MOVED ALTERNATIVE */
+	GIT_COLOR_BOLD_MAGENTA,	/* OLD_MOVED */
+	GIT_COLOR_BOLD_BLUE,	/* OLD_MOVED ALTERNATIVE */
+	GIT_COLOR_FAINT,	/* OLD_MOVED_DIM */
+	GIT_COLOR_FAINT_ITALIC,	/* OLD_MOVED_ALTERNATIVE_DIM */
+	GIT_COLOR_BOLD_CYAN,	/* NEW_MOVED */
+	GIT_COLOR_BOLD_YELLOW,	/* NEW_MOVED ALTERNATIVE */
+	GIT_COLOR_FAINT,	/* NEW_MOVED_DIM */
+	GIT_COLOR_FAINT_ITALIC,	/* NEW_MOVED_ALTERNATIVE_DIM */
 };
 
 static NORETURN void die_want_option(const char *option_name)
@@ -90,10 +94,18 @@ static int parse_diff_color_slot(const char *var)
 		return DIFF_FILE_OLD_MOVED;
 	if (!strcasecmp(var, "oldmovedalternative"))
 		return DIFF_FILE_OLD_MOVED_ALT;
+	if (!strcasecmp(var, "oldmoveddimmed"))
+		return DIFF_FILE_OLD_MOVED_DIM;
+	if (!strcasecmp(var, "oldmovedalternativedimmed"))
+		return DIFF_FILE_OLD_MOVED_ALT_DIM;
 	if (!strcasecmp(var, "newmoved"))
 		return DIFF_FILE_NEW_MOVED;
 	if (!strcasecmp(var, "newmovedalternative"))
 		return DIFF_FILE_NEW_MOVED_ALT;
+	if (!strcasecmp(var, "newmoveddimmed"))
+		return DIFF_FILE_NEW_MOVED_DIM;
+	if (!strcasecmp(var, "newmovedalternativedimmed"))
+		return DIFF_FILE_NEW_MOVED_ALT_DIM;
 	return -1;
 }
 
@@ -250,6 +262,8 @@ static int parse_color_moved(const char *arg)
 		return COLOR_MOVED_PLAIN;
 	else if (!strcmp(arg, "zebra"))
 		return COLOR_MOVED_ZEBRA;
+	else if (!strcmp(arg, "dimmed_zebra"))
+		return COLOR_MOVED_ZEBRA_DIM;
 	else
 		return -1;
 }
@@ -637,6 +651,7 @@ enum diff_symbol {
 #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF	(1<<16)
 #define DIFF_SYMBOL_MOVED_LINE			(1<<17)
 #define DIFF_SYMBOL_MOVED_LINE_ALT		(1<<18)
+#define DIFF_SYMBOL_MOVED_LINE_UNINTERESTING	(1<<19)
 #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)
 
 /*
@@ -890,6 +905,67 @@ static void mark_color_as_moved(struct diff_options *o,
 	free(pmb);
 }
 
+#define DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK \
+  (DIFF_SYMBOL_MOVED_LINE | DIFF_SYMBOL_MOVED_LINE_ALT)
+static void dim_moved_lines(struct diff_options *o)
+{
+	int n;
+	for (n = 0; n < o->emitted_symbols->nr; n++) {
+		struct emitted_diff_symbol *prev = (n != 0) ?
+				&o->emitted_symbols->buf[n - 1] : NULL;
+		struct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];
+		struct emitted_diff_symbol *next =
+				(n < o->emitted_symbols->nr - 1) ?
+				&o->emitted_symbols->buf[n + 1] : NULL;
+
+		/* Not a plus or minus line? */
+		if (l->s != DIFF_SYMBOL_PLUS && l->s != DIFF_SYMBOL_MINUS)
+			continue;
+
+		/* Not a moved line? */
+		if (!(l->flags & DIFF_SYMBOL_MOVED_LINE))
+			continue;
+
+		/*
+		 * If prev or next are not a plus or minus line,
+		 * pretend they don't exist
+		 */
+		if (prev && prev->s != DIFF_SYMBOL_PLUS &&
+			    prev->s != DIFF_SYMBOL_MINUS)
+			prev = NULL;
+		if (next && next->s != DIFF_SYMBOL_PLUS &&
+			    next->s != DIFF_SYMBOL_MINUS)
+			next = NULL;
+
+		/* Inside a block? */
+		if ((prev &&
+		    (prev->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==
+		    (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK)) &&
+		    (next &&
+		    (next->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==
+		    (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK))) {
+			l->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;
+			continue;
+		}
+
+		/* Check if we are at an interesting bound: */
+		if (prev && (prev->flags & DIFF_SYMBOL_MOVED_LINE) &&
+		    (prev->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=
+		       (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))
+			continue;
+		if (next && (next->flags & DIFF_SYMBOL_MOVED_LINE) &&
+		    (next->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=
+		       (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))
+			continue;
+
+		/*
+		 * The boundary to prev and next are not interesting,
+		 * so this line is not interesting as a whole
+		 */
+		l->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;
+	}
+}
+
 static void emit_line_ws_markup(struct diff_options *o,
 				const char *set, const char *reset,
 				const char *line, int len, char sign,
@@ -964,24 +1040,56 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
 				    flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);
 		break;
 	case DIFF_SYMBOL_PLUS:
-		if (flags & DIFF_SYMBOL_MOVED_LINE_ALT)
+		switch (flags & (DIFF_SYMBOL_MOVED_LINE |
+				 DIFF_SYMBOL_MOVED_LINE_ALT |
+				 DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {
+		case DIFF_SYMBOL_MOVED_LINE |
+		     DIFF_SYMBOL_MOVED_LINE_ALT |
+		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
+			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT_DIM);
+			break;
+		case DIFF_SYMBOL_MOVED_LINE |
+		     DIFF_SYMBOL_MOVED_LINE_ALT:
 			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);
-		else if (flags & DIFF_SYMBOL_MOVED_LINE)
+			break;
+		case DIFF_SYMBOL_MOVED_LINE |
+		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
+			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_DIM);
+			break;
+		case DIFF_SYMBOL_MOVED_LINE:
 			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);
-		else
+			break;
+		default:
 			set = diff_get_color_opt(o, DIFF_FILE_NEW);
+		}
 		reset = diff_get_color_opt(o, DIFF_RESET);
 		emit_line_ws_markup(o, set, reset, line, len, '+',
 				    flags & DIFF_SYMBOL_CONTENT_WS_MASK,
 				    flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);
 		break;
 	case DIFF_SYMBOL_MINUS:
-		if (flags & DIFF_SYMBOL_MOVED_LINE_ALT)
+		switch (flags & (DIFF_SYMBOL_MOVED_LINE |
+				 DIFF_SYMBOL_MOVED_LINE_ALT |
+				 DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {
+		case DIFF_SYMBOL_MOVED_LINE |
+		     DIFF_SYMBOL_MOVED_LINE_ALT |
+		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
+			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT_DIM);
+			break;
+		case DIFF_SYMBOL_MOVED_LINE |
+		     DIFF_SYMBOL_MOVED_LINE_ALT:
 			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);
-		else if (flags & DIFF_SYMBOL_MOVED_LINE)
+			break;
+		case DIFF_SYMBOL_MOVED_LINE |
+		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
+			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_DIM);
+			break;
+		case DIFF_SYMBOL_MOVED_LINE:
 			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);
-		else
+			break;
+		default:
 			set = diff_get_color_opt(o, DIFF_FILE_OLD);
+		}
 		reset = diff_get_color_opt(o, DIFF_RESET);
 		emit_line_ws_markup(o, set, reset, line, len, '-',
 				    flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
@@ -4534,9 +4642,12 @@ int diff_opt_parse(struct diff_options *options,
 	}
 	else if (!strcmp(arg, "--no-color"))
 		options->use_color = 0;
-	else if (!strcmp(arg, "--color-moved"))
-		options->color_moved = COLOR_MOVED_ZEBRA;
-	else if (!strcmp(arg, "--no-color-moved"))
+	else if (!strcmp(arg, "--color-moved")) {
+		if (diff_color_moved_default)
+			options->color_moved = diff_color_moved_default;
+		if (options->color_moved == COLOR_MOVED_NO)
+			options->color_moved = COLOR_MOVED_ZEBRA_DIM;
+	} else if (!strcmp(arg, "--no-color-moved"))
 		options->color_moved = COLOR_MOVED_NO;
 	else if (skip_prefix(arg, "--color-moved=", &arg)) {
 		int cm = parse_color_moved(arg);
@@ -5381,6 +5492,8 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 
 			add_lines_to_move_detection(o, &add_lines, &del_lines);
 			mark_color_as_moved(o, &add_lines, &del_lines);
+			if (o->color_moved == COLOR_MOVED_ZEBRA_DIM)
+				dim_moved_lines(o);
 
 			hashmap_free(&add_lines, 0);
 			hashmap_free(&del_lines, 0);
diff --git a/diff.h b/diff.h
index 1aae8738ca..98abd75521 100644
--- a/diff.h
+++ b/diff.h
@@ -192,6 +192,7 @@ struct diff_options {
 		COLOR_MOVED_NO = 0,
 		COLOR_MOVED_PLAIN = 1,
 		COLOR_MOVED_ZEBRA = 2,
+		COLOR_MOVED_ZEBRA_DIM = 3,
 	} color_moved;
 };
 
@@ -216,8 +217,12 @@ enum color_diff {
 	DIFF_FUNCINFO = 8,
 	DIFF_FILE_OLD_MOVED = 9,
 	DIFF_FILE_OLD_MOVED_ALT = 10,
-	DIFF_FILE_NEW_MOVED = 11,
-	DIFF_FILE_NEW_MOVED_ALT = 12
+	DIFF_FILE_OLD_MOVED_DIM = 11,
+	DIFF_FILE_OLD_MOVED_ALT_DIM = 12,
+	DIFF_FILE_NEW_MOVED = 13,
+	DIFF_FILE_NEW_MOVED_ALT = 14,
+	DIFF_FILE_NEW_MOVED_DIM = 15,
+	DIFF_FILE_NEW_MOVED_ALT_DIM = 16
 };
 const char *diff_get_color(int diff_use_color, enum color_diff ix);
 #define diff_get_color_opt(o, ix) \
diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
index 1ca16435d6..ae8c686f3c 100755
--- a/t/t4015-diff-whitespace.sh
+++ b/t/t4015-diff-whitespace.sh
@@ -1179,6 +1179,130 @@ test_expect_success 'plain moved code, inside file' '
 	test_cmp expected actual
 '
 
+test_expect_success 'detect permutations inside moved code -- dimmed_zebra' '
+	git reset --hard &&
+	cat <<-\EOF >lines.txt &&
+		line 1
+		line 2
+		line 3
+		line 4
+		line 5
+		line 6
+		line 7
+		line 8
+		line 9
+		line 10
+		line 11
+		line 12
+		line 13
+		line 14
+		line 15
+		line 16
+	EOF
+	git add lines.txt &&
+	git commit -m "add poetry" &&
+	cat <<-\EOF >lines.txt &&
+		line 4
+		line 5
+		line 6
+		line 7
+		line 8
+		line 9
+		line 1
+		line 2
+		line 3
+		line 14
+		line 15
+		line 16
+		line 10
+		line 11
+		line 12
+		line 13
+	EOF
+	test_config color.diff.oldMoved "magenta" &&
+	test_config color.diff.newMoved "cyan" &&
+	test_config color.diff.oldMovedAlternative "blue" &&
+	test_config color.diff.newMovedAlternative "yellow" &&
+	test_config color.diff.oldMovedDimmed "normal magenta" &&
+	test_config color.diff.newMovedDimmed "normal cyan" &&
+	test_config color.diff.oldMovedAlternativeDimmed "normal blue" &&
+	test_config color.diff.newMovedAlternativeDimmed "normal yellow" &&
+	git diff HEAD --no-renames --color-moved=dimmed_zebra| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+	<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+	<BOLD>--- a/lines.txt<RESET>
+	<BOLD>+++ b/lines.txt<RESET>
+	<CYAN>@@ -1,16 +1,16 @@<RESET>
+	<BMAGENTA>-line 1<RESET>
+	<BMAGENTA>-line 2<RESET>
+	<BMAGENTA>-line 3<RESET>
+	 line 4<RESET>
+	 line 5<RESET>
+	 line 6<RESET>
+	 line 7<RESET>
+	 line 8<RESET>
+	 line 9<RESET>
+	<BCYAN>+<RESET><BCYAN>line 1<RESET>
+	<BCYAN>+<RESET><BCYAN>line 2<RESET>
+	<CYAN>+<RESET><CYAN>line 3<RESET>
+	<YELLOW>+<RESET><YELLOW>line 14<RESET>
+	<BYELLOW>+<RESET><BYELLOW>line 15<RESET>
+	<BYELLOW>+<RESET><BYELLOW>line 16<RESET>
+	 line 10<RESET>
+	 line 11<RESET>
+	 line 12<RESET>
+	 line 13<RESET>
+	<BMAGENTA>-line 14<RESET>
+	<BMAGENTA>-line 15<RESET>
+	<BMAGENTA>-line 16<RESET>
+	EOF
+	test_cmp expected actual
+'
+
+test_expect_success 'cmd option assumes configured colored-moved' '
+	test_config color.diff.oldMoved "magenta" &&
+	test_config color.diff.newMoved "cyan" &&
+	test_config color.diff.oldMovedAlternative "blue" &&
+	test_config color.diff.newMovedAlternative "yellow" &&
+	test_config color.diff.oldMovedDimmed "normal magenta" &&
+	test_config color.diff.newMovedDimmed "normal cyan" &&
+	test_config color.diff.oldMovedAlternativeDimmed "normal blue" &&
+	test_config color.diff.newMovedAlternativeDimmed "normal yellow" &&
+	test_config diff.colorMoved zebra &&
+	git diff HEAD --no-renames --color-moved| test_decode_color >actual &&
+	cat <<-\EOF >expected &&
+	<BOLD>diff --git a/lines.txt b/lines.txt<RESET>
+	<BOLD>index 47ea9c3..ba96a38 100644<RESET>
+	<BOLD>--- a/lines.txt<RESET>
+	<BOLD>+++ b/lines.txt<RESET>
+	<CYAN>@@ -1,16 +1,16 @@<RESET>
+	<MAGENTA>-line 1<RESET>
+	<MAGENTA>-line 2<RESET>
+	<MAGENTA>-line 3<RESET>
+	 line 4<RESET>
+	 line 5<RESET>
+	 line 6<RESET>
+	 line 7<RESET>
+	 line 8<RESET>
+	 line 9<RESET>
+	<CYAN>+<RESET><CYAN>line 1<RESET>
+	<CYAN>+<RESET><CYAN>line 2<RESET>
+	<CYAN>+<RESET><CYAN>line 3<RESET>
+	<YELLOW>+<RESET><YELLOW>line 14<RESET>
+	<YELLOW>+<RESET><YELLOW>line 15<RESET>
+	<YELLOW>+<RESET><YELLOW>line 16<RESET>
+	 line 10<RESET>
+	 line 11<RESET>
+	 line 12<RESET>
+	 line 13<RESET>
+	<MAGENTA>-line 14<RESET>
+	<MAGENTA>-line 15<RESET>
+	<MAGENTA>-line 16<RESET>
+	EOF
+	test_cmp expected actual
+'
+
 test_expect_success 'no effect from --color-moved with --word-diff' '
 	cat <<-\EOF >text.txt &&
 	Lorem Ipsum is simply dummy text of the printing and typesetting industry.
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* [PATCHv2 25/25] diff: document the new --color-moved setting
  2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
                                               ` (23 preceding siblings ...)
  2017-06-23  1:29                             ` [PATCHv2 24/25] diff.c: add dimming to moved line detection Stefan Beller
@ 2017-06-23  1:29                             ` Stefan Beller
  24 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23  1:29 UTC (permalink / raw)
  To: sbeller
  Cc: bmwill, git, gitster, jonathantanmy, jrnieder, mhagger, peff,
	philipoakley

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/config.txt       | 12 ++++++++++--
 Documentation/diff-options.txt | 27 +++++++++++++++++++++++++++
 2 files changed, 37 insertions(+), 2 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 475e874d51..29e0b9fa69 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1051,14 +1051,22 @@ This does not affect linkgit:git-format-patch[1] or the
 'git-diff-{asterisk}' plumbing commands.  Can be overridden on the
 command line with the `--color[=<when>]` option.
 
+diff.colorMoved::
+	If set moved lines in a diff are colored differently,
+	for details see '--color-moved' in linkgit:git-diff[1].
+
 color.diff.<slot>::
 	Use customized color for diff colorization.  `<slot>` specifies
 	which part of the patch to use the specified color, and is one
 	of `context` (context text - `plain` is a historical synonym),
 	`meta` (metainformation), `frag`
 	(hunk header), 'func' (function in hunk header), `old` (removed lines),
-	`new` (added lines), `commit` (commit headers), or `whitespace`
-	(highlighting whitespace errors).
+	`new` (added lines), `commit` (commit headers), `whitespace`
+	(highlighting whitespace errors), `oldMoved` (deleted lines),
+	`newMoved` (added lines), `oldMovedDimmed`, `oldMovedAlternative`,
+	`oldMovedAlternativeDimmed`, `newMovedDimmed`, `newMovedAlternative`
+	and `newMovedAlternativeDimmed` (See the '<mode>'
+	setting of '--color-moved' in linkgit:git-diff[1] for details).
 
 color.decorate.<slot>::
 	Use customized color for 'git log --decorate' output.  `<slot>` is one
diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt
index 89cc0f48de..058c8014ed 100644
--- a/Documentation/diff-options.txt
+++ b/Documentation/diff-options.txt
@@ -231,6 +231,33 @@ ifdef::git-diff[]
 endif::git-diff[]
 	It is the same as `--color=never`.
 
+--color-moved[=<mode>]::
+	Moved lines of code are colored differently.
+ifdef::git-diff[]
+	It can be changed by the `diff.colorMoved` configuration setting.
+endif::git-diff[]
+	The <mode> defaults to 'no' if the option is not given
+	and to 'dimmed_zebra' if the option with no mode is given.
+	The mode must be one of:
++
+--
+no::
+	Moved lines are not highlighted.
+plain::
+	Any line that is added in one location and was removed
+	in another location will be colored with 'color.diff.newMoved'.
+	Similarly 'color.diff.oldMoved' will be used for removed lines
+	that are added somewhere else in the diff.
+zebra::
+	Blocks of moved code are detected. The detected blocks are
+	painted using the 'color.diff.{old,new}Moved' alternating with
+	'color.diff.{old,new}MovedAlternative'.
+dimmed_zebra::
+	Similar to 'zebra', but additional dimming of uninteresting parts
+	of moved code is performed. The bordering lines of two adjacent
+	blocks are considered interesting, the rest is uninteresting.
+--
+
 --word-diff[=<mode>]::
 	Show a word diff, using the <mode> to delimit changed words.
 	By default, words are delimited by whitespace; see
-- 
2.12.2.575.gb14f27f917


^ permalink raw reply related	[flat|nested] 125+ messages in thread

* Re: [PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol
  2017-06-22 23:37                                 ` Stefan Beller
@ 2017-06-23  4:56                                   ` Junio C Hamano
  0 siblings, 0 replies; 125+ messages in thread
From: Junio C Hamano @ 2017-06-23  4:56 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Brandon Williams, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

Stefan Beller <sbeller@google.com> writes:

> Actually that function already has some quick return:
>
> static int new_blank_line_at_eof(struct emit_callback *ecbdata, const
> char *line, int len)
> {
>     if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
>          ecbdata->blank_at_eof_in_preimage &&
>          ecbdata->blank_at_eof_in_postimage &&
>          ecbdata->blank_at_eof_in_preimage <= ecbdata->lno_in_preimage &&
>          ecbdata->blank_at_eof_in_postimage <= ecbdata->lno_in_postimage))
>                   return 0;
>     return ws_blank_line(line, len, ecbdata->ws_rule);
> }

I wouldn't call the first if() statement "quick return".  It
probably weighs as much as ws_blank_line() itself, which is not that
expensive.  

Even though new_blank_line_at_eof() may have been coded efficiently
and not very expensive as a whole, the fact remains that it didn't
get called when we know it was unneeded, and it now will get called
before we figure out if it is necessary.

And that is why I said it is a bit unsatisfactory (as opposed to
"horrible" or with other more severe adjective).



^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv2 04/25] diff.c: introduce emit_diff_symbol
  2017-06-23  1:28                             ` [PATCHv2 04/25] diff.c: introduce emit_diff_symbol Stefan Beller
@ 2017-06-23 20:07                               ` Junio C Hamano
  2017-06-23 20:13                                 ` Stefan Beller
  0 siblings, 1 reply; 125+ messages in thread
From: Junio C Hamano @ 2017-06-23 20:07 UTC (permalink / raw)
  To: Stefan Beller
  Cc: bmwill, git, jonathantanmy, jrnieder, mhagger, peff, philipoakley

Stefan Beller <sbeller@google.com> writes:

>     diff --git a/new.txt b/new.txt
>     index fa69b07..412428c 100644
>     Binary files a/new.txt and b/new.txt differ
>
>   could be buffered as
>      DIFF_SYMBOL_DIFF_START + new.txt
>      DIFF_SYMBOL_INDEX_MODE + fa69b07 412428c "non-executable" flag
>      DIFF_SYMBOL_BINARY_FILES + new.txt

Hopefully this is an oversimplified example and I'd assume that you
will do the right thing when showing a renamed filepair by allowing
both old and new paths that could be different?


^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [PATCHv2 04/25] diff.c: introduce emit_diff_symbol
  2017-06-23 20:07                               ` Junio C Hamano
@ 2017-06-23 20:13                                 ` Stefan Beller
  0 siblings, 0 replies; 125+ messages in thread
From: Stefan Beller @ 2017-06-23 20:13 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Brandon Williams, git, Jonathan Tan, Jonathan Nieder,
	Michael Haggerty, Jeff King, Philip Oakley

On Fri, Jun 23, 2017 at 1:07 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>
>>     diff --git a/new.txt b/new.txt
>>     index fa69b07..412428c 100644
>>     Binary files a/new.txt and b/new.txt differ
>>
>>   could be buffered as
>>      DIFF_SYMBOL_DIFF_START + new.txt
>>      DIFF_SYMBOL_INDEX_MODE + fa69b07 412428c "non-executable" flag
>>      DIFF_SYMBOL_BINARY_FILES + new.txt
>
> Hopefully this is an oversimplified example and I'd assume that you
> will do the right thing when showing a renamed filepair by allowing
> both old and new paths that could be different?

Yes, this is simplified to drive the point of having as little data as possible,
so I thought we might have a flag that says "same file name" instead of giving
the file name twice.

We do not do such an optimization yet.

^ permalink raw reply	[flat|nested] 125+ messages in thread

* Re: [showing-off RFC/PATCH 26/26] diff.c: have a "machine parseable" move coloring
  2017-06-20  2:50                             ` Stefan Beller
@ 2017-06-23 21:43                               ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 125+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2017-06-23 21:43 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Brandon Williams, git, Junio C Hamano, Jonathan Tan,
	Jonathan Nieder, Michael Haggerty, Jeff King, Philip Oakley


On Tue, Jun 20 2017, Stefan Beller jotted:

> + Ævar, who was not part of the email where I copied all recipients
> from for this series.

I played around with this a bit, it would be great to have something
like this on top of --color-moved eventually. It's a lot easier to /^_
in a pager than looking for color codes.

Of course it's diff-incompatible output, but it's fair enough to have
that hidden behind some option IMO.

> On Mon, Jun 19, 2017 at 7:48 PM, Stefan Beller <sbeller@google.com> wrote:
>> Ævar asked for it, this is how you would do it.
>> (plus documentation, tests, CLI knobs, options)
>>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>>  diff.c | 15 +++++++++++----
>>  diff.h |  2 ++
>>  2 files changed, 13 insertions(+), 4 deletions(-)
>>
>> diff --git a/diff.c b/diff.c
>> index 7756f7610c..61caa057ff 100644
>> --- a/diff.c
>> +++ b/diff.c
>> @@ -997,6 +997,7 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
>>         static const char *nneof = " No newline at end of file\n";
>>         const char *context, *reset, *set, *meta, *fraginfo;
>>         struct strbuf sb = STRBUF_INIT;
>> +       int sign;
>>
>>         enum diff_symbol s = eds->s;
>>         const char *line = eds->line;
>> @@ -1058,8 +1059,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
>>                 default:
>>                         set = diff_get_color_opt(o, DIFF_FILE_NEW);
>>                 }
>> +               sign = '+';
>> +               if (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)
>> +                       sign = '*';
>>                 reset = diff_get_color_opt(o, DIFF_RESET);
>> -               emit_line_ws_markup(o, set, reset, line, len, '+',
>> +               emit_line_ws_markup(o, set, reset, line, len, sign,
>>                                     flags & DIFF_SYMBOL_CONTENT_WS_MASK,
>>                                     flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);
>>                 break;
>> @@ -1086,8 +1090,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,
>>                 default:
>>                         set = diff_get_color_opt(o, DIFF_FILE_OLD);
>>                 }
>> +               sign = '-';
>> +               if (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)
>> +                       sign = '_';
>>                 reset = diff_get_color_opt(o, DIFF_RESET);
>> -               emit_line_ws_markup(o, set, reset, line, len, '-',
>> +               emit_line_ws_markup(o, set, reset, line, len, sign,
>>                                     flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
>>                 break;
>>         case DIFF_SYMBOL_WORDS_PORCELAIN:
>> @@ -5475,7 +5482,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
>>         static struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;
>>         struct diff_queue_struct *q = &diff_queued_diff;
>>
>> -       if (o->color_moved)
>> +       if (o->color_moved || o->machine_readable_moves)
>>                 o->emitted_symbols = &esm;
>>
>>         for (i = 0; i < q->nr; i++) {
>> @@ -5485,7 +5492,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)
>>         }
>>
>>         if (o->emitted_symbols) {
>> -               if (o->color_moved) {
>> +               if (o->color_moved || o->machine_readable_moves) {
>>                         struct hashmap add_lines, del_lines;
>>                         unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);
>>
>> diff --git a/diff.h b/diff.h
>> index 98abd75521..b6c4d7ab0f 100644
>> --- a/diff.h
>> +++ b/diff.h
>> @@ -194,6 +194,8 @@ struct diff_options {
>>                 COLOR_MOVED_ZEBRA = 2,
>>                 COLOR_MOVED_ZEBRA_DIM = 3,
>>         } color_moved;
>> +
>> +       int machine_readable_moves;
>>  };
>>
>>  void diff_emit_submodule_del(struct diff_options *o, const char *line);
>> --
>> 2.12.2.575.gb14f27f917
>>

^ permalink raw reply	[flat|nested] 125+ messages in thread

end of thread, other threads:[~2017-06-23 21:43 UTC | newest]

Thread overview: 125+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <20170523024048.16879-1-sbeller@google.com/>
2017-05-24 21:40 ` [PATCHv5 00/17] Diff machine: highlight moved lines Stefan Beller
2017-05-24 21:40   ` [PATCHv5 01/17] diff: readability fix Stefan Beller
2017-05-24 21:40   ` [PATCHv5 02/17] diff: move line ending check into emit_hunk_header Stefan Beller
2017-05-24 21:40   ` [PATCHv5 03/17] diff.c: factor out diff_flush_patch_all_file_pairs Stefan Beller
2017-05-24 21:40   ` [PATCHv5 04/17] diff: introduce more flexible emit function Stefan Beller
2017-06-13 21:54     ` Jonathan Tan
2017-06-13 23:41       ` Stefan Beller
2017-06-13 23:46         ` Jonathan Tan
2017-05-24 21:40   ` [PATCHv5 05/17] diff.c: convert fn_out_consume to use emit_line Stefan Beller
2017-05-24 21:40   ` [PATCHv5 06/17] diff.c: convert builtin_diff to use emit_line_* Stefan Beller
2017-05-24 21:40   ` [PATCHv5 07/17] diff.c: convert emit_rewrite_diff " Stefan Beller
2017-05-24 21:40   ` [PATCHv5 08/17] diff.c: convert emit_rewrite_lines " Stefan Beller
2017-05-24 21:40   ` [PATCHv5 09/17] submodule.c: convert show_submodule_summary to use emit_line_fmt Stefan Beller
2017-05-24 21:40   ` [PATCHv5 10/17] diff.c: convert emit_binary_diff_body to use emit_line_* Stefan Beller
2017-05-24 21:40   ` [PATCHv5 11/17] diff.c: convert show_stats " Stefan Beller
2017-05-24 21:40   ` [PATCHv5 12/17] diff.c: convert word diffing " Stefan Beller
2017-05-24 21:40   ` [PATCHv5 13/17] diff.c: convert diff_flush " Stefan Beller
2017-05-24 21:40   ` [PATCHv5 14/17] diff.c: convert diff_summary " Stefan Beller
2017-05-24 21:40   ` [PATCHv5 15/17] diff.c: emit_line includes whitespace highlighting Stefan Beller
2017-05-24 21:40   ` [PATCHv5 16/17] diff: buffer all output if asked to Stefan Beller
2017-05-25  2:26     ` Junio C Hamano
2017-05-25  5:34       ` Stefan Beller
2017-05-26  1:09         ` Junio C Hamano
2017-06-13 22:07     ` Jonathan Tan
2017-06-14  2:52       ` Stefan Beller
2017-05-24 21:40   ` [PATCHv5 17/17] diff.c: color moved lines differently Stefan Beller
2017-05-25  2:27     ` Junio C Hamano
2017-05-25  5:39       ` Stefan Beller
2017-05-25  6:44   ` [PATCHv5 00/17] Diff machine: highlight moved lines Junio C Hamano
2017-05-25 16:31     ` Stefan Beller
2017-05-26  1:20       ` Junio C Hamano
2017-05-26 19:30         ` Stefan Beller
2017-05-27  0:18           ` [PATCH 0/1] " Stefan Beller
2017-05-27  0:18             ` [PATCH 1/1] diff.c: color moved lines differently Stefan Beller
2017-05-27  7:05               ` Philip Oakley
2017-05-30 21:33                 ` Stefan Beller
2017-06-01  0:24                   ` [PATCH] " Stefan Beller
2017-06-13 22:51                     ` Jonathan Tan
2017-06-14 18:55                       ` Stefan Beller
2017-06-20  2:47                         ` [PATCH 00/26] reroll of sb/diff-color-moved Stefan Beller
2017-06-20  2:47                           ` [PATCH 01/26] diff.c: readability fix Stefan Beller
2017-06-20  2:47                           ` [PATCH 02/26] diff.c: move line ending check into emit_hunk_header Stefan Beller
2017-06-20  2:47                           ` [PATCH 03/26] diff.c: factor out diff_flush_patch_all_file_pairs Stefan Beller
2017-06-20  2:47                           ` [PATCH 04/26] diff.c: introduce emit_diff_symbol Stefan Beller
2017-06-21 19:36                             ` Junio C Hamano
2017-06-21 19:46                               ` Stefan Beller
2017-06-21 20:26                                 ` Junio C Hamano
2017-06-21 21:13                                 ` Junio C Hamano
2017-06-21 21:23                                   ` Stefan Beller
2017-06-21 21:43                                     ` Junio C Hamano
2017-06-21 21:51                                       ` Stefan Beller
2017-06-21 21:45                                 ` Brandon Williams
2017-06-21 21:52                                   ` Junio C Hamano
2017-06-21 21:55                                     ` Brandon Williams
2017-06-20  2:47                           ` [PATCH 05/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER Stefan Beller
2017-06-20  2:47                           ` [PATCH 06/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO Stefan Beller
2017-06-20  2:47                           ` [PATCH 07/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF Stefan Beller
2017-06-20  2:47                           ` [PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol Stefan Beller
2017-06-21 20:05                             ` Junio C Hamano
2017-06-22 23:30                               ` Stefan Beller
2017-06-22 23:37                                 ` Stefan Beller
2017-06-23  4:56                                   ` Junio C Hamano
2017-06-20  2:47                           ` [PATCH 09/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS{_PORCELAIN} Stefan Beller
2017-06-20  2:48                           ` [PATCH 10/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE Stefan Beller
2017-06-20  2:48                           ` [PATCH 11/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR Stefan Beller
2017-06-20 20:01                             ` Jonathan Tan
2017-06-21 20:09                               ` Junio C Hamano
2017-06-22 23:59                                 ` Stefan Beller
2017-06-20  2:48                           ` [PATCH 12/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER Stefan Beller
2017-06-20  2:48                           ` [PATCH 13/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES Stefan Beller
2017-06-21 20:13                             ` Junio C Hamano
2017-06-21 20:47                               ` Stefan Beller
2017-06-20  2:48                           ` [PATCH 14/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF Stefan Beller
2017-06-20  2:48                           ` [PATCH 15/26] submodule.c: migrate diff output to use emit_diff_symbol Stefan Beller
2017-06-20 20:09                             ` Jonathan Tan
2017-06-20  2:48                           ` [PATCH 16/26] diff.c: convert emit_binary_diff_body " Stefan Beller
2017-06-21 20:16                             ` Junio C Hamano
2017-06-20  2:48                           ` [PATCH 17/26] diff.c: convert show_stats " Stefan Beller
2017-06-21 21:39                             ` Brandon Williams
2017-06-21 22:16                               ` Stefan Beller
2017-06-20  2:48                           ` [PATCH 18/26] diff.c: convert word diffing " Stefan Beller
2017-06-20  2:48                           ` [PATCH 19/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP Stefan Beller
2017-06-20  2:48                           ` [PATCH 20/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY Stefan Beller
2017-06-20  2:48                           ` [PATCH 21/26] diff.c: buffer all output if asked to Stefan Beller
2017-06-20  2:48                           ` [PATCH 22/26] diff.c: color moved lines differently Stefan Beller
2017-06-20 20:13                             ` Jonathan Tan
2017-06-20 20:57                               ` Stefan Beller
2017-06-20  2:48                           ` [PATCH 23/26] diff.c: color moved lines differently, plain mode Stefan Beller
2017-06-20  2:48                           ` [PATCH 24/26] diff.c: add dimming to moved line detection Stefan Beller
2017-06-21 20:23                             ` Junio C Hamano
2017-06-20  2:48                           ` [PATCH 25/26] diff: document the new --color-moved setting Stefan Beller
2017-06-20  2:48                           ` [showing-off RFC/PATCH 26/26] diff.c: have a "machine parseable" move coloring Stefan Beller
2017-06-20  2:50                             ` Stefan Beller
2017-06-23 21:43                               ` Ævar Arnfjörð Bjarmason
2017-06-21 21:51                             ` Brandon Williams
2017-06-21 21:55                               ` Junio C Hamano
2017-06-21 22:40                                 ` Stefan Beller
2017-06-23  1:28                           ` [PATCHv2 00/25] reroll of sb/diff-color-moved Stefan Beller
2017-06-23  1:28                             ` [PATCHv2 01/25] diff.c: readability fix Stefan Beller
2017-06-23  1:28                             ` [PATCHv2 02/25] diff.c: move line ending check into emit_hunk_header Stefan Beller
2017-06-23  1:28                             ` [PATCHv2 03/25] diff.c: factor out diff_flush_patch_all_file_pairs Stefan Beller
2017-06-23  1:28                             ` [PATCHv2 04/25] diff.c: introduce emit_diff_symbol Stefan Beller
2017-06-23 20:07                               ` Junio C Hamano
2017-06-23 20:13                                 ` Stefan Beller
2017-06-23  1:28                             ` [PATCHv2 05/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 06/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 07/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 08/25] diff.c: migrate emit_line_checked to use emit_diff_symbol Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 09/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS[_PORCELAIN] Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 10/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 11/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR_{PLUS, MINUS} Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 12/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 13/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 14/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 15/25] submodule.c: migrate diff output to use emit_diff_symbol Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 16/25] diff.c: convert emit_binary_diff_body " Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 17/25] diff.c: convert show_stats " Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 18/25] diff.c: convert word diffing " Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 19/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 20/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 21/25] diff.c: buffer all output if asked to Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 22/25] diff.c: color moved lines differently Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 23/25] diff.c: color moved lines differently, plain mode Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 24/25] diff.c: add dimming to moved line detection Stefan Beller
2017-06-23  1:29                             ` [PATCHv2 25/25] diff: document the new --color-moved setting Stefan Beller

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.