[PATCH 00/10] Enrich Trailer API

[PATCH 00/10] Enrich Trailer API

[Linus Arver via GitGitGadget]

This patch series is the first 10 patches of a much larger series I've been
working. The main goal of this series is to enrich the API in trailer.h. The
larger series brings a number of additional code simplifications and
cleanups (exposing and fixing some bugs along the way), and builds on top of
this series. The goal of the larger series is to make the trailer interface
ready for unit testing. By "trailer API" I mean those functions exposed in
trailer.h.

Currently, we have "process_trailers()" in trailer.h which does many
different things (parse command-line arguments, create temporary files, etc)
that are independent of the concept of "trailers", whose interface trailer.h
defines. This makes unit-testing this function difficult. While there is no
technical reason why we couldn't write unit tests for the smaller functions
that are called within process_trailers(), doing so would involve testing
private ("static" in trailer.c) functions instead of the public functions
exposed in trailer.h which authoritatively define the API.

As an alternative to this patch series, we could keep trailer.h intact and
decide to unit-test the existing "trailer_info_get()" function which does
most of the trailer parsing work. However this function wouldn't be easy to
test either, because the resulting data structure merely contains the
unparsed "trailers" lines. So the unit test (if it wants to inspect the
result of parsing these lines) would have to invoke additional parsing
functions itself. And at that point it would not longer be a unit test in
the traditional sense, because it would be invoking multiple functions at
once.

This series breaks up "process_trailers()" into smaller pieces, exposing
many of the parts relevant to trailer-related processing in trailer.h. This
forces us to start writing unit tests for these now public functions, but
that is a good thing because those same unit tests should be easy to write
(due to their small(er) sizes), but also, because those unit tests will now
ensure some degree of stability across new versions of trailer.h (we will
start noticing when the behavior of any of these API functions change).

Another benefit is that more clients (perhaps those outside of Git in the
future) will be able to use the same trailer processing algorithms used by
the interpret-trailers builtin. For example, a web server may want to parse
trailers the same way that interpret-trailers would parse them, without
having to shell out to interpret-trailers. Importing the new (richer)
trailer.h file that this series promotes would help them achieve that goal.
This use case was the original motivation behind my work in this area of
Git.

Thanks to the aggressive refactoring in this series, I've been able to
identify and fix a couple bugs in our existing implementation. Those fixes
build on top of this series but were not included here, in order to keep
this series small.

Linus Arver (10):
  trailer: move process_trailers() to interpret-trailers.c
  trailer: include "trailer" term in API functions
  trailer: unify trailer formatting machinery
  trailer: delete obsolete formatting functions
  sequencer: use the trailer iterator
  trailer: make trailer_info struct private
  trailer: spread usage of "trailer_block" language
  trailer: prepare to move parse_trailers_from_command_line_args() to
    builtin
  trailer: move arg handling to interpret-trailers.c
  trailer: delete obsolete argument handling code from API

 builtin/interpret-trailers.c | 169 ++++++++--
 builtin/shortlog.c           |   7 +-
 pretty.c                     |   2 +-
 ref-filter.c                 |   2 +-
 sequencer.c                  |  35 +--
 trailer.c                    | 579 ++++++++++++++++-------------------
 trailer.h                    | 106 ++++---
 7 files changed, 482 insertions(+), 418 deletions(-)


base-commit: a54a84b333adbecf7bc4483c0e36ed5878cac17b
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1632%2Flistx%2Ftrailer-api-refactor-part-1-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1632/listx/trailer-api-refactor-part-1-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1632
-- 
gitgitgadget

[PATCH 01/10] trailer: move process_trailers() to interpret-trailers.c

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

The interpret-trailers.c builtin is the only place we need to call
process_trailers(). As it stands, process_trailers() is inherently tied
to how the builtin behaves, so move its definition there.

Delete the corresponding declaration from trailer.h, which then forces
us to expose the working innards of that function. This enriches
trailer.h to include a more granular API, which can then be unit-tested
in the future (because process_trailers() by itself does too many things
to be able to be easily unit-tested).

Take this opportunity to demote some file-handling functions out of the
trailer API implementation, as these have nothing to do with trailers.

While we're at it, rename process_trailers() to interpret_trailers() in
the builtin for consistency with the existing cmd_interpret_trailers(),
which wraps around this function.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  98 +++++++++++++++++++++++++++-
 trailer.c                    | 120 ++++-------------------------------
 trailer.h                    |  20 +++++-
 3 files changed, 126 insertions(+), 112 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 033bd1556cf..444f8fb70c9 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -9,6 +9,7 @@
 #include "gettext.h"
 #include "parse-options.h"
 #include "string-list.h"
+#include "tempfile.h"
 #include "trailer.h"
 #include "config.h"
 
@@ -91,6 +92,99 @@ static int parse_opt_parse(const struct option *opt, const char *arg,
 	return 0;
 }
 
+static struct tempfile *trailers_tempfile;
+
+static FILE *create_in_place_tempfile(const char *file)
+{
+	struct stat st;
+	struct strbuf filename_template = STRBUF_INIT;
+	const char *tail;
+	FILE *outfile;
+
+	if (stat(file, &st))
+		die_errno(_("could not stat %s"), file);
+	if (!S_ISREG(st.st_mode))
+		die(_("file %s is not a regular file"), file);
+	if (!(st.st_mode & S_IWUSR))
+		die(_("file %s is not writable by user"), file);
+
+	/* Create temporary file in the same directory as the original */
+	tail = strrchr(file, '/');
+	if (tail)
+		strbuf_add(&filename_template, file, tail - file + 1);
+	strbuf_addstr(&filename_template, "git-interpret-trailers-XXXXXX");
+
+	trailers_tempfile = xmks_tempfile_m(filename_template.buf, st.st_mode);
+	strbuf_release(&filename_template);
+	outfile = fdopen_tempfile(trailers_tempfile, "w");
+	if (!outfile)
+		die_errno(_("could not open temporary file"));
+
+	return outfile;
+}
+
+static void read_input_file(struct strbuf *sb, const char *file)
+{
+	if (file) {
+		if (strbuf_read_file(sb, file, 0) < 0)
+			die_errno(_("could not read input file '%s'"), file);
+	} else {
+		if (strbuf_read(sb, fileno(stdin), 0) < 0)
+			die_errno(_("could not read from stdin"));
+	}
+}
+
+static void interpret_trailers(const char *file,
+			       const struct process_trailer_options *opts,
+			       struct list_head *new_trailer_head)
+{
+	LIST_HEAD(head);
+	struct strbuf sb = STRBUF_INIT;
+	struct trailer_info info;
+	FILE *outfile = stdout;
+
+	ensure_configured();
+
+	read_input_file(&sb, file);
+
+	if (opts->in_place)
+		outfile = create_in_place_tempfile(file);
+
+	parse_trailers(&info, sb.buf, &head, opts);
+
+	/* Print the lines before the trailers */
+	if (!opts->only_trailers)
+		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
+
+	if (!opts->only_trailers && !info.blank_line_before_trailer)
+		fprintf(outfile, "\n");
+
+
+	if (!opts->only_input) {
+		LIST_HEAD(config_head);
+		LIST_HEAD(arg_head);
+		parse_trailers_from_config(&config_head);
+		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
+		list_splice(&config_head, &arg_head);
+		process_trailers_lists(&head, &arg_head);
+	}
+
+	print_all(outfile, &head, opts);
+
+	free_all(&head);
+	trailer_info_release(&info);
+
+	/* Print the lines after the trailers as is */
+	if (!opts->only_trailers)
+		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
+
+	if (opts->in_place)
+		if (rename_tempfile(&trailers_tempfile, file))
+			die_errno(_("could not rename temporary file to %s"), file);
+
+	strbuf_release(&sb);
+}
+
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
@@ -132,11 +226,11 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 	if (argc) {
 		int i;
 		for (i = 0; i < argc; i++)
-			process_trailers(argv[i], &opts, &trailers);
+			interpret_trailers(argv[i], &opts, &trailers);
 	} else {
 		if (opts.in_place)
 			die(_("no input file given for in-place editing"));
-		process_trailers(NULL, &opts, &trailers);
+		interpret_trailers(NULL, &opts, &trailers);
 	}
 
 	new_trailers_clear(&trailers);
diff --git a/trailer.c b/trailer.c
index 3a0710a4583..9d70c9946bd 100644
--- a/trailer.c
+++ b/trailer.c
@@ -5,7 +5,6 @@
 #include "string-list.h"
 #include "run-command.h"
 #include "commit.h"
-#include "tempfile.h"
 #include "trailer.h"
 #include "list.h"
 /*
@@ -163,8 +162,8 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-static void print_all(FILE *outfile, struct list_head *head,
-		      const struct process_trailer_options *opts)
+void print_all(FILE *outfile, struct list_head *head,
+	       const struct process_trailer_options *opts)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
@@ -366,8 +365,8 @@ static int find_same_and_apply_arg(struct list_head *head,
 	return 0;
 }
 
-static void process_trailers_lists(struct list_head *head,
-				   struct list_head *arg_head)
+void process_trailers_lists(struct list_head *head,
+			    struct list_head *arg_head)
 {
 	struct list_head *pos, *p;
 	struct arg_item *arg_tok;
@@ -589,7 +588,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	return 0;
 }
 
-static void ensure_configured(void)
+void ensure_configured(void)
 {
 	if (configured)
 		return;
@@ -719,7 +718,7 @@ static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
 	list_add_tail(&new_item->list, arg_head);
 }
 
-static void parse_trailers_from_config(struct list_head *config_head)
+void parse_trailers_from_config(struct list_head *config_head)
 {
 	struct arg_item *item;
 	struct list_head *pos;
@@ -735,8 +734,8 @@ static void parse_trailers_from_config(struct list_head *config_head)
 	}
 }
 
-static void parse_trailers_from_command_line_args(struct list_head *arg_head,
-						  struct list_head *new_trailer_head)
+void parse_trailers_from_command_line_args(struct list_head *arg_head,
+					   struct list_head *new_trailer_head)
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
@@ -775,17 +774,6 @@ static void parse_trailers_from_command_line_args(struct list_head *arg_head,
 	free(cl_separators);
 }
 
-static void read_input_file(struct strbuf *sb, const char *file)
-{
-	if (file) {
-		if (strbuf_read_file(sb, file, 0) < 0)
-			die_errno(_("could not read input file '%s'"), file);
-	} else {
-		if (strbuf_read(sb, fileno(stdin), 0) < 0)
-			die_errno(_("could not read from stdin"));
-	}
-}
-
 static const char *next_line(const char *str)
 {
 	const char *nl = strchrnul(str, '\n');
@@ -1000,10 +988,10 @@ static void unfold_value(struct strbuf *val)
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
  */
-static void parse_trailers(struct trailer_info *info,
-			     const char *str,
-			     struct list_head *head,
-			     const struct process_trailer_options *opts)
+void parse_trailers(struct trailer_info *info,
+		    const char *str,
+		    struct list_head *head,
+		    const struct process_trailer_options *opts)
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
@@ -1035,7 +1023,7 @@ static void parse_trailers(struct trailer_info *info,
 	}
 }
 
-static void free_all(struct list_head *head)
+void free_all(struct list_head *head)
 {
 	struct list_head *pos, *p;
 	list_for_each_safe(pos, p, head) {
@@ -1044,88 +1032,6 @@ static void free_all(struct list_head *head)
 	}
 }
 
-static struct tempfile *trailers_tempfile;
-
-static FILE *create_in_place_tempfile(const char *file)
-{
-	struct stat st;
-	struct strbuf filename_template = STRBUF_INIT;
-	const char *tail;
-	FILE *outfile;
-
-	if (stat(file, &st))
-		die_errno(_("could not stat %s"), file);
-	if (!S_ISREG(st.st_mode))
-		die(_("file %s is not a regular file"), file);
-	if (!(st.st_mode & S_IWUSR))
-		die(_("file %s is not writable by user"), file);
-
-	/* Create temporary file in the same directory as the original */
-	tail = strrchr(file, '/');
-	if (tail)
-		strbuf_add(&filename_template, file, tail - file + 1);
-	strbuf_addstr(&filename_template, "git-interpret-trailers-XXXXXX");
-
-	trailers_tempfile = xmks_tempfile_m(filename_template.buf, st.st_mode);
-	strbuf_release(&filename_template);
-	outfile = fdopen_tempfile(trailers_tempfile, "w");
-	if (!outfile)
-		die_errno(_("could not open temporary file"));
-
-	return outfile;
-}
-
-void process_trailers(const char *file,
-		      const struct process_trailer_options *opts,
-		      struct list_head *new_trailer_head)
-{
-	LIST_HEAD(head);
-	struct strbuf sb = STRBUF_INIT;
-	struct trailer_info info;
-	FILE *outfile = stdout;
-
-	ensure_configured();
-
-	read_input_file(&sb, file);
-
-	if (opts->in_place)
-		outfile = create_in_place_tempfile(file);
-
-	parse_trailers(&info, sb.buf, &head, opts);
-
-	/* Print the lines before the trailers */
-	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
-
-	if (!opts->only_trailers && !info.blank_line_before_trailer)
-		fprintf(outfile, "\n");
-
-
-	if (!opts->only_input) {
-		LIST_HEAD(config_head);
-		LIST_HEAD(arg_head);
-		parse_trailers_from_config(&config_head);
-		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
-		list_splice(&config_head, &arg_head);
-		process_trailers_lists(&head, &arg_head);
-	}
-
-	print_all(outfile, &head, opts);
-
-	free_all(&head);
-	trailer_info_release(&info);
-
-	/* Print the lines after the trailers as is */
-	if (!opts->only_trailers)
-		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
-
-	if (opts->in_place)
-		if (rename_tempfile(&trailers_tempfile, file))
-			die_errno(_("could not rename temporary file to %s"), file);
-
-	strbuf_release(&sb);
-}
-
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts)
 {
diff --git a/trailer.h b/trailer.h
index 1644cd05f60..b3e4a5e127d 100644
--- a/trailer.h
+++ b/trailer.h
@@ -81,15 +81,29 @@ struct process_trailer_options {
 
 #define PROCESS_TRAILER_OPTIONS_INIT {0}
 
-void process_trailers(const char *file,
-		      const struct process_trailer_options *opts,
-		      struct list_head *new_trailer_head);
+void parse_trailers_from_config(struct list_head *config_head);
+
+void parse_trailers_from_command_line_args(struct list_head *arg_head,
+					   struct list_head *new_trailer_head);
+
+void process_trailers_lists(struct list_head *head,
+			    struct list_head *arg_head);
+
+void parse_trailers(struct trailer_info *info,
+		    const char *str,
+		    struct list_head *head,
+		    const struct process_trailer_options *opts);
 
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts);
 
 void trailer_info_release(struct trailer_info *info);
 
+void ensure_configured(void);
+void print_all(FILE *outfile, struct list_head *head,
+	       const struct process_trailer_options *opts);
+void free_all(struct list_head *head);
+
 /*
  * Format the trailers from the commit msg "msg" into the strbuf "out".
  * Note two caveats about "opts":
-- 
gitgitgadget

[PATCH 02/10] trailer: include "trailer" term in API functions

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

These functions are exposed to clients and so they should include
"trailer" in their names for easier identification, just like all the
other functions already exposed by trailer.h.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  7 ++++---
 trailer.c                    | 10 +++++-----
 trailer.h                    | 10 +++++-----
 3 files changed, 14 insertions(+), 13 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 444f8fb70c9..adb74276281 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -143,7 +143,7 @@ static void interpret_trailers(const char *file,
 	struct trailer_info info;
 	FILE *outfile = stdout;
 
-	ensure_configured();
+	trailer_config_init();
 
 	read_input_file(&sb, file);
 
@@ -169,9 +169,10 @@ static void interpret_trailers(const char *file,
 		process_trailers_lists(&head, &arg_head);
 	}
 
-	print_all(outfile, &head, opts);
+	/* Print trailer block. */
+	format_trailers(outfile, &head, opts);
 
-	free_all(&head);
+	free_trailers(&head);
 	trailer_info_release(&info);
 
 	/* Print the lines after the trailers as is */
diff --git a/trailer.c b/trailer.c
index 9d70c9946bd..0ce7e9079ca 100644
--- a/trailer.c
+++ b/trailer.c
@@ -162,8 +162,8 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-void print_all(FILE *outfile, struct list_head *head,
-	       const struct process_trailer_options *opts)
+void format_trailers(FILE *outfile, struct list_head *head,
+		     const struct process_trailer_options *opts)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
@@ -588,7 +588,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	return 0;
 }
 
-void ensure_configured(void)
+void trailer_config_init(void)
 {
 	if (configured)
 		return;
@@ -1023,7 +1023,7 @@ void parse_trailers(struct trailer_info *info,
 	}
 }
 
-void free_all(struct list_head *head)
+void free_trailers(struct list_head *head)
 {
 	struct list_head *pos, *p;
 	list_for_each_safe(pos, p, head) {
@@ -1041,7 +1041,7 @@ void trailer_info_get(struct trailer_info *info, const char *str,
 	size_t nr = 0, alloc = 0;
 	char **last = NULL;
 
-	ensure_configured();
+	trailer_config_init();
 
 	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
 	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
diff --git a/trailer.h b/trailer.h
index b3e4a5e127d..0e4f0ece9b3 100644
--- a/trailer.h
+++ b/trailer.h
@@ -99,11 +99,11 @@ void trailer_info_get(struct trailer_info *info, const char *str,
 
 void trailer_info_release(struct trailer_info *info);
 
-void ensure_configured(void);
-void print_all(FILE *outfile, struct list_head *head,
-	       const struct process_trailer_options *opts);
-void free_all(struct list_head *head);
+void trailer_config_init(void);
+void free_trailers(struct list_head *trailers);
 
+void format_trailers(FILE *outfile, struct list_head *head,
+		     const struct process_trailer_options *opts);
 /*
  * Format the trailers from the commit msg "msg" into the strbuf "out".
  * Note two caveats about "opts":
@@ -111,7 +111,7 @@ void free_all(struct list_head *head);
  *   - this is primarily a helper for pretty.c, and not
  *     all of the flags are supported.
  *
- *   - this differs from process_trailers slightly in that we always format
+ *   - this differs from format_trailers slightly in that we always format
  *     only the trailer block itself, even if the "only_trailers" option is not
  *     set.
  */
-- 
gitgitgadget

[PATCH 03/10] trailer: unify trailer formatting machinery

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Currently have two functions for formatting trailers exposed in
trailer.h:

    void format_trailers(FILE *outfile, struct list_head *head,
                        const struct process_trailer_options *opts);

    void format_trailers_from_commit(struct strbuf *out, const char *msg,
                                    const struct process_trailer_options *opts);

and previously these functions, although similar enough (even taking the
same process_trailer_options struct pointer), did not build on each
other.

Make format_trailers_from_commit() rely on format_trailers(). Teach
format_trailers() to process trailers with the additional
process_trailer_options fields like opts->key_only which is only used by
format_trailers_from_commit() and not builtin/interpret-trailers.c.

This will allow us to delete the format_trailer_info() and
print_tok_val() functions in the next patch. They are not deleted here
in order to keep the diff small.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |   5 +-
 pretty.c                     |   2 +-
 ref-filter.c                 |   2 +-
 trailer.c                    | 105 +++++++++++++++++++++++++++++------
 trailer.h                    |  21 +++----
 5 files changed, 102 insertions(+), 33 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index adb74276281..934833a4645 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -140,6 +140,7 @@ static void interpret_trailers(const char *file,
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
+	struct strbuf trailer_block = STRBUF_INIT;
 	struct trailer_info info;
 	FILE *outfile = stdout;
 
@@ -170,7 +171,9 @@ static void interpret_trailers(const char *file,
 	}
 
 	/* Print trailer block. */
-	format_trailers(outfile, &head, opts);
+	format_trailers(&head, opts, &trailer_block);
+	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
+	strbuf_release(&trailer_block);
 
 	free_trailers(&head);
 	trailer_info_release(&info);
diff --git a/pretty.c b/pretty.c
index cf964b060cd..f0721a5214f 100644
--- a/pretty.c
+++ b/pretty.c
@@ -1759,7 +1759,7 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */
 				goto trailer_out;
 		}
 		if (*arg == ')') {
-			format_trailers_from_commit(sb, msg + c->subject_off, &opts);
+			format_trailers_from_commit(msg + c->subject_off, &opts, sb);
 			ret = arg - placeholder + 1;
 		}
 	trailer_out:
diff --git a/ref-filter.c b/ref-filter.c
index 35b989e1dfe..7fb13818686 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -1985,7 +1985,7 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp
 			struct strbuf s = STRBUF_INIT;
 
 			/* Format the trailer info according to the trailer_opts given */
-			format_trailers_from_commit(&s, subpos, &atom->u.contents.trailer_opts);
+			format_trailers_from_commit(subpos, &atom->u.contents.trailer_opts, &s);
 
 			v->s = strbuf_detach(&s, NULL);
 		} else if (atom->u.contents.option == C_BARE)
diff --git a/trailer.c b/trailer.c
index 0ce7e9079ca..315d90ee1ab 100644
--- a/trailer.c
+++ b/trailer.c
@@ -162,19 +162,6 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-void format_trailers(FILE *outfile, struct list_head *head,
-		     const struct process_trailer_options *opts)
-{
-	struct list_head *pos;
-	struct trailer_item *item;
-	list_for_each(pos, head) {
-		item = list_entry(pos, struct trailer_item, list);
-		if ((!opts->trim_empty || strlen(item->value) > 0) &&
-		    (!opts->only_trailers || item->token))
-			print_tok_val(outfile, item->token, item->value);
-	}
-}
-
 static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
 {
 	struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
@@ -984,6 +971,78 @@ static void unfold_value(struct strbuf *val)
 	strbuf_release(&out);
 }
 
+void format_trailers(struct list_head *head,
+		     const struct process_trailer_options *opts,
+		     struct strbuf *out)
+{
+	struct list_head *pos;
+	struct trailer_item *item;
+	int need_separator = 0;
+
+	list_for_each(pos, head) {
+		item = list_entry(pos, struct trailer_item, list);
+		if (item->token) {
+			char c;
+
+			struct strbuf tok = STRBUF_INIT;
+			struct strbuf val = STRBUF_INIT;
+			strbuf_addstr(&tok, item->token);
+			strbuf_addstr(&val, item->value);
+
+			/*
+			 * Skip key/value pairs where the value was empty. This
+			 * can happen from trailers specified without a
+			 * separator, like `--trailer "Reviewed-by"` (no
+			 * corresponding value).
+			 */
+			if (opts->trim_empty && !strlen(item->value))
+				continue;
+
+			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
+				if (opts->unfold)
+					unfold_value(&val);
+
+				if (opts->separator && need_separator)
+					strbuf_addbuf(out, opts->separator);
+				if (!opts->value_only)
+					strbuf_addbuf(out, &tok);
+				if (!opts->key_only && !opts->value_only) {
+					if (opts->key_value_separator)
+						strbuf_addbuf(out, opts->key_value_separator);
+					else {
+						c = last_non_space_char(tok.buf);
+						if (c) {
+							if (!strchr(separators, c))
+								strbuf_addf(out, "%c ", separators[0]);
+						}
+					}
+				}
+				if (!opts->key_only)
+					strbuf_addbuf(out, &val);
+				if (!opts->separator)
+					strbuf_addch(out, '\n');
+
+				need_separator = 1;
+			}
+
+			strbuf_release(&tok);
+			strbuf_release(&val);
+		} else if (!opts->only_trailers) {
+			if (opts->separator && need_separator) {
+				strbuf_addbuf(out, opts->separator);
+			}
+			strbuf_addstr(out, item->value);
+			if (opts->separator)
+				strbuf_rtrim(out);
+			else
+				strbuf_addch(out, '\n');
+
+			need_separator = 1;
+		}
+
+	}
+}
+
 /*
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
@@ -1144,13 +1203,25 @@ static void format_trailer_info(struct strbuf *out,
 
 }
 
-void format_trailers_from_commit(struct strbuf *out, const char *msg,
-				 const struct process_trailer_options *opts)
+void format_trailers_from_commit(const char *msg,
+				 const struct process_trailer_options *opts,
+				 struct strbuf *out)
 {
+	LIST_HEAD(head);
 	struct trailer_info info;
 
-	trailer_info_get(&info, msg, opts);
-	format_trailer_info(out, &info, msg, opts);
+	parse_trailers(&info, msg, &head, opts);
+
+	/* If we want the whole block untouched, we can take the fast path. */
+	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
+	    !opts->separator && !opts->key_only && !opts->value_only &&
+	    !opts->key_value_separator) {
+		strbuf_add(out, msg + info.trailer_block_start,
+			   info.trailer_block_end - info.trailer_block_start);
+	} else
+		format_trailers(&head, opts, out);
+
+	free_trailers(&head);
 	trailer_info_release(&info);
 }
 
diff --git a/trailer.h b/trailer.h
index 0e4f0ece9b3..50f70556302 100644
--- a/trailer.h
+++ b/trailer.h
@@ -102,21 +102,16 @@ void trailer_info_release(struct trailer_info *info);
 void trailer_config_init(void);
 void free_trailers(struct list_head *trailers);
 
-void format_trailers(FILE *outfile, struct list_head *head,
-		     const struct process_trailer_options *opts);
+void format_trailers(struct list_head *head,
+		     const struct process_trailer_options *opts,
+		     struct strbuf *out);
 /*
- * Format the trailers from the commit msg "msg" into the strbuf "out".
- * Note two caveats about "opts":
- *
- *   - this is primarily a helper for pretty.c, and not
- *     all of the flags are supported.
- *
- *   - this differs from format_trailers slightly in that we always format
- *     only the trailer block itself, even if the "only_trailers" option is not
- *     set.
+ * Convenience function to format the trailers from the commit msg "msg" into
+ * the strbuf "out". Reuses format_trailers internally.
  */
-void format_trailers_from_commit(struct strbuf *out, const char *msg,
-				 const struct process_trailer_options *opts);
+void format_trailers_from_commit(const char *msg,
+				 const struct process_trailer_options *opts,
+				 struct strbuf *out);
 
 /*
  * An interface for iterating over the trailers found in a particular commit
-- 
gitgitgadget

[PATCH 04/10] trailer: delete obsolete formatting functions

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 79 -------------------------------------------------------
 1 file changed, 79 deletions(-)

diff --git a/trailer.c b/trailer.c
index 315d90ee1ab..132f22b3dd7 100644
--- a/trailer.c
+++ b/trailer.c
@@ -144,24 +144,6 @@ static char last_non_space_char(const char *s)
 	return '\0';
 }
 
-static void print_tok_val(FILE *outfile, const char *tok, const char *val)
-{
-	char c;
-
-	if (!tok) {
-		fprintf(outfile, "%s\n", val);
-		return;
-	}
-
-	c = last_non_space_char(tok);
-	if (!c)
-		return;
-	if (strchr(separators, c))
-		fprintf(outfile, "%s%s\n", tok, val);
-	else
-		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
-}
-
 static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
 {
 	struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
@@ -1142,67 +1124,6 @@ void trailer_info_release(struct trailer_info *info)
 	free(info->trailers);
 }
 
-static void format_trailer_info(struct strbuf *out,
-				const struct trailer_info *info,
-				const char *msg,
-				const struct process_trailer_options *opts)
-{
-	size_t origlen = out->len;
-	size_t i;
-
-	/* If we want the whole block untouched, we can take the fast path. */
-	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
-	    !opts->separator && !opts->key_only && !opts->value_only &&
-	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info->trailer_block_start,
-			   info->trailer_block_end - info->trailer_block_start);
-		return;
-	}
-
-	for (i = 0; i < info->trailer_nr; i++) {
-		char *trailer = info->trailers[i];
-		ssize_t separator_pos = find_separator(trailer, separators);
-
-		if (separator_pos >= 1) {
-			struct strbuf tok = STRBUF_INIT;
-			struct strbuf val = STRBUF_INIT;
-
-			parse_trailer(&tok, &val, NULL, trailer, separator_pos);
-			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
-				if (opts->unfold)
-					unfold_value(&val);
-
-				if (opts->separator && out->len != origlen)
-					strbuf_addbuf(out, opts->separator);
-				if (!opts->value_only)
-					strbuf_addbuf(out, &tok);
-				if (!opts->key_only && !opts->value_only) {
-					if (opts->key_value_separator)
-						strbuf_addbuf(out, opts->key_value_separator);
-					else
-						strbuf_addstr(out, ": ");
-				}
-				if (!opts->key_only)
-					strbuf_addbuf(out, &val);
-				if (!opts->separator)
-					strbuf_addch(out, '\n');
-			}
-			strbuf_release(&tok);
-			strbuf_release(&val);
-
-		} else if (!opts->only_trailers) {
-			if (opts->separator && out->len != origlen) {
-				strbuf_addbuf(out, opts->separator);
-			}
-			strbuf_addstr(out, trailer);
-			if (opts->separator) {
-				strbuf_rtrim(out);
-			}
-		}
-	}
-
-}
-
 void format_trailers_from_commit(const char *msg,
 				 const struct process_trailer_options *opts,
 				 struct strbuf *out)
-- 
gitgitgadget

[PATCH 05/10] sequencer: use the trailer iterator

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This patch allows for the removal of "trailer_info_get()" from the
trailer.h API, which will be in the next patch.

Instead of calling "trailer_info_get()", which is a low-level function
in the trailers implementation (trailer.c), call
trailer_iterator_advance(), which was specifically designed for public
consumption in f0939a0eb1 (trailer: add interface for iterating over
commit trailers, 2020-09-27).

Avoiding "trailer_info_get()" means we don't have to worry about options
like "no_divider" (relevant for parsing trailers). We also don't have to
check for things like "info.trailer_start == info.trailer_end" to see
whether there were any trailers (instead we can just check to see
whether the iterator advanced at all).

Also, teach the iterator about non-trailer lines, by adding a new field
called "raw" to hold both trailer and non-trailer lines. This is
necessary because a "trailer block" is a list of trailer lines of at
least 25% trailers (see 146245063e (trailer: allow non-trailers in
trailer block, 2016-10-21)), such that it may hold non-trailer lines.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/shortlog.c |  7 +++++--
 sequencer.c        | 35 +++++++++++++++--------------------
 trailer.c          | 20 ++++++++++++--------
 trailer.h          | 13 +++++++++++++
 4 files changed, 45 insertions(+), 30 deletions(-)

diff --git a/builtin/shortlog.c b/builtin/shortlog.c
index 1307ed2b88a..dc8fd5a5532 100644
--- a/builtin/shortlog.c
+++ b/builtin/shortlog.c
@@ -172,7 +172,7 @@ static void insert_records_from_trailers(struct shortlog *log,
 					 const char *oneline)
 {
 	struct trailer_iterator iter;
-	const char *commit_buffer, *body;
+	const char *commit_buffer, *body, *value;
 	struct strbuf ident = STRBUF_INIT;
 
 	if (!log->trailers.nr)
@@ -190,7 +190,10 @@ static void insert_records_from_trailers(struct shortlog *log,
 
 	trailer_iterator_init(&iter, body);
 	while (trailer_iterator_advance(&iter)) {
-		const char *value = iter.val.buf;
+		if (!iter.is_trailer)
+			continue;
+
+		value = iter.val.buf;
 
 		if (!string_list_has_string(&log->trailers, iter.key.buf))
 			continue;
diff --git a/sequencer.c b/sequencer.c
index 3cc88d8a800..d199869cda9 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -319,37 +319,32 @@ static const char *get_todo_path(const struct replay_opts *opts)
 static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	size_t ignore_footer)
 {
-	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
-	struct trailer_info info;
-	size_t i;
-	int found_sob = 0, found_sob_last = 0;
-	char saved_char;
-
-	opts.no_divider = 1;
+	struct trailer_iterator iter;
+	size_t i = 0, found_sob = 0;
+	char saved_char = sb->buf[sb->len - ignore_footer];
 
 	if (ignore_footer) {
-		saved_char = sb->buf[sb->len - ignore_footer];
 		sb->buf[sb->len - ignore_footer] = '\0';
 	}
 
-	trailer_info_get(&info, sb->buf, &opts);
+	trailer_iterator_init(&iter, sb->buf);
+	while (trailer_iterator_advance(&iter)) {
+		i++;
+		if (sob &&
+		    iter.is_trailer &&
+		    !strncmp(iter.raw.buf, sob->buf, sob->len)) {
+			found_sob = i;
+		}
+	}
+	trailer_iterator_release(&iter);
 
 	if (ignore_footer)
 		sb->buf[sb->len - ignore_footer] = saved_char;
 
-	if (info.trailer_block_start == info.trailer_block_end)
+	if (!i)
 		return 0;
 
-	for (i = 0; i < info.trailer_nr; i++)
-		if (sob && !strncmp(info.trailers[i], sob->buf, sob->len)) {
-			found_sob = 1;
-			if (i == info.trailer_nr - 1)
-				found_sob_last = 1;
-		}
-
-	trailer_info_release(&info);
-
-	if (found_sob_last)
+	if (found_sob == i)
 		return 3;
 	if (found_sob)
 		return 2;
diff --git a/trailer.c b/trailer.c
index 132f22b3dd7..593717fd56c 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1151,6 +1151,7 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
 	strbuf_init(&iter->key, 0);
 	strbuf_init(&iter->val, 0);
+	strbuf_init(&iter->raw, 0);
 	opts.no_divider = 1;
 	trailer_info_get(&iter->internal.info, msg, &opts);
 	iter->internal.cur = 0;
@@ -1158,17 +1159,19 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 
 int trailer_iterator_advance(struct trailer_iterator *iter)
 {
-	while (iter->internal.cur < iter->internal.info.trailer_nr) {
-		char *trailer = iter->internal.info.trailers[iter->internal.cur++];
-		int separator_pos = find_separator(trailer, separators);
-
-		if (separator_pos < 1)
-			continue; /* not a real trailer */
-
+	char *line;
+	int separator_pos;
+	if (iter->internal.cur < iter->internal.info.trailer_nr) {
+		line = iter->internal.info.trailers[iter->internal.cur++];
+		separator_pos = find_separator(line, separators);
+		iter->is_trailer = (separator_pos > 0);
+
+		strbuf_reset(&iter->raw);
+		strbuf_addstr(&iter->raw, line);
 		strbuf_reset(&iter->key);
 		strbuf_reset(&iter->val);
 		parse_trailer(&iter->key, &iter->val, NULL,
-			      trailer, separator_pos);
+			      line, separator_pos);
 		unfold_value(&iter->val);
 		return 1;
 	}
@@ -1180,4 +1183,5 @@ void trailer_iterator_release(struct trailer_iterator *iter)
 	trailer_info_release(&iter->internal.info);
 	strbuf_release(&iter->val);
 	strbuf_release(&iter->key);
+	strbuf_release(&iter->raw);
 }
diff --git a/trailer.h b/trailer.h
index 50f70556302..d50c9fd79b2 100644
--- a/trailer.h
+++ b/trailer.h
@@ -127,6 +127,19 @@ struct trailer_iterator {
 	struct strbuf key;
 	struct strbuf val;
 
+	/*
+	 * Raw line (e.g., "foo: bar baz") before being parsed as a trailer
+	 * key/val pair. This field can contain non-trailer lines because it's
+	 * valid for a trailer block to contain such lines (i.e., we only
+	 * require 25% of the lines in a trailer block to be trailer lines).
+	 */
+	struct strbuf raw;
+
+	/*
+	 * 1 if the raw line was parsed as a separate key/val pair.
+	 */
+	int is_trailer;
+
 	/* private */
 	struct {
 		struct trailer_info info;
-- 
gitgitgadget

[PATCH 06/10] trailer: make trailer_info struct private

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

In 13211ae23f (trailer: separate public from internal portion of
trailer_iterator, 2023-09-09) we moved trailer_info behind an anonymous
struct to discourage use by trailer.h API users. However it still left
open the possibility of external use of trailer_info itself. Now that
there are no external users of trailer_info, we can make this struct
private.

Make this struct private by putting its definition inside trailer.c.
This has two benefits:

(1) it makes the surface area of the public facing interface (trailer.h)
    smaller, and

(2) external API users are unable to peer inside this struct (because it
    is only ever exposed as an opaque pointer).

This change exposes some deficiencies in the API, mainly with regard to
information about the location of the trailer block that was parsed.
Expose new API functions to access this information (needed by
builtin/interpret-trailers.c).

The idea in this patch to hide implementation details behind an "opaque
pointer" is also known as the "pimpl" (pointer to implementation) idiom
in C++ and is a common pattern in that language (where, for example,
abstract classes only have pointers to concrete classes).

However, the original inspiration to use this idiom does not come from
C++, but instead the book "C Interfaces and Implementations: Techniques
for Creating Reusable Software" [1]. This book recommends opaque
pointers as a good design principle for designing C libraries, using the
term "interface" as the functions defined in *.h (header) files and
"implementation" as the corresponding *.c file which define the
interfaces.

The book says this about opaque pointers:

    ... clients can manipulate such pointers freely, but they can’t
    dereference them; that is, they can’t look at the innards of the
    structure pointed to by them. Only the implementation has that
    privilege. Opaque pointers hide representation details and help
    catch errors.

In our case, "struct trailer_info" is now hidden from clients, and the
ways in which this opaque pointer can be used is limited to the richness
of the trailer.h file. In other words, trailer.h exclusively controls
exactly how "trailer_info" pointers are to be used.

[1] Hanson, David R. "C Interfaces and Implementations: Techniques for
    Creating Reusable Software". Addison Wesley, 1997. p. 22

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  13 +--
 trailer.c                    | 154 +++++++++++++++++++++++------------
 trailer.h                    |  37 ++-------
 3 files changed, 117 insertions(+), 87 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 934833a4645..0838a57e157 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -141,7 +141,7 @@ static void interpret_trailers(const char *file,
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
 	struct strbuf trailer_block = STRBUF_INIT;
-	struct trailer_info info;
+	struct trailer_info *info;
 	FILE *outfile = stdout;
 
 	trailer_config_init();
@@ -151,13 +151,13 @@ static void interpret_trailers(const char *file,
 	if (opts->in_place)
 		outfile = create_in_place_tempfile(file);
 
-	parse_trailers(&info, sb.buf, &head, opts);
+	info = parse_trailers(sb.buf, &head, opts);
 
 	/* Print the lines before the trailers */
 	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
+		fwrite(sb.buf, 1, trailer_block_start(info), outfile);
 
-	if (!opts->only_trailers && !info.blank_line_before_trailer)
+	if (!opts->only_trailers && !blank_line_before_trailer_block(info))
 		fprintf(outfile, "\n");
 
 
@@ -176,11 +176,12 @@ static void interpret_trailers(const char *file,
 	strbuf_release(&trailer_block);
 
 	free_trailers(&head);
-	trailer_info_release(&info);
 
 	/* Print the lines after the trailers as is */
 	if (!opts->only_trailers)
-		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
+		fwrite(sb.buf + trailer_block_end(info), 1, sb.len - trailer_block_end(info), outfile);
+
+	trailer_info_release(info);
 
 	if (opts->in_place)
 		if (rename_tempfile(&trailers_tempfile, file))
diff --git a/trailer.c b/trailer.c
index 593717fd56c..0c66e2d3812 100644
--- a/trailer.c
+++ b/trailer.c
@@ -11,6 +11,27 @@
  * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
  */
 
+struct trailer_info {
+	/*
+	 * True if there is a blank line before the location pointed to by
+	 * trailer_block_start.
+	 */
+	int blank_line_before_trailer;
+
+	/*
+	 * Offsets to the trailer block start and end positions in the input
+	 * string. If no trailer block is found, these are both set to the
+	 * "true" end of the input (find_end_of_log_message()).
+	 */
+	size_t trailer_block_start, trailer_block_end;
+
+	/*
+	 * Array of trailers found.
+	 */
+	char **trailers;
+	size_t trailer_nr;
+};
+
 struct conf_info {
 	char *name;
 	char *key;
@@ -1025,20 +1046,72 @@ void format_trailers(struct list_head *head,
 	}
 }
 
+static struct trailer_info *trailer_info_new(void)
+{
+	struct trailer_info *info = xcalloc(1, sizeof(*info));
+	return info;
+}
+
+static struct trailer_info *trailer_info_get(const char *str,
+					     const struct process_trailer_options *opts)
+{
+	struct trailer_info *info = trailer_info_new();
+	size_t end_of_log_message = 0, trailer_block_start = 0;
+	struct strbuf **trailer_lines, **ptr;
+	char **trailer_strings = NULL;
+	size_t nr = 0, alloc = 0;
+	char **last = NULL;
+
+	trailer_config_init();
+
+	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
+	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
+
+	trailer_lines = strbuf_split_buf(str + trailer_block_start,
+					 end_of_log_message - trailer_block_start,
+					 '\n',
+					 0);
+	for (ptr = trailer_lines; *ptr; ptr++) {
+		if (last && isspace((*ptr)->buf[0])) {
+			struct strbuf sb = STRBUF_INIT;
+			strbuf_attach(&sb, *last, strlen(*last), strlen(*last));
+			strbuf_addbuf(&sb, *ptr);
+			*last = strbuf_detach(&sb, NULL);
+			continue;
+		}
+		ALLOC_GROW(trailer_strings, nr + 1, alloc);
+		trailer_strings[nr] = strbuf_detach(*ptr, NULL);
+		last = find_separator(trailer_strings[nr], separators) >= 1
+			? &trailer_strings[nr]
+			: NULL;
+		nr++;
+	}
+	strbuf_list_free(trailer_lines);
+
+	info->blank_line_before_trailer = ends_with_blank_line(str,
+							       trailer_block_start);
+	info->trailer_block_start = trailer_block_start;
+	info->trailer_block_end = end_of_log_message;
+	info->trailers = trailer_strings;
+	info->trailer_nr = nr;
+
+	return info;
+}
+
 /*
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
  */
-void parse_trailers(struct trailer_info *info,
-		    const char *str,
-		    struct list_head *head,
-		    const struct process_trailer_options *opts)
+struct trailer_info *parse_trailers(const char *str,
+				    struct list_head *head,
+				    const struct process_trailer_options *opts)
 {
+	struct trailer_info *info;
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
 	size_t i;
 
-	trailer_info_get(info, str, opts);
+	info = trailer_info_get(str, opts);
 
 	for (i = 0; i < info->trailer_nr; i++) {
 		int separator_pos;
@@ -1062,6 +1135,8 @@ void parse_trailers(struct trailer_info *info,
 					 strbuf_detach(&val, NULL));
 		}
 	}
+
+	return info;
 }
 
 void free_trailers(struct list_head *head)
@@ -1073,47 +1148,19 @@ void free_trailers(struct list_head *head)
 	}
 }
 
-void trailer_info_get(struct trailer_info *info, const char *str,
-		      const struct process_trailer_options *opts)
+size_t trailer_block_start(struct trailer_info *info)
 {
-	size_t end_of_log_message = 0, trailer_block_start = 0;
-	struct strbuf **trailer_lines, **ptr;
-	char **trailer_strings = NULL;
-	size_t nr = 0, alloc = 0;
-	char **last = NULL;
-
-	trailer_config_init();
-
-	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
-	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
+	return info->trailer_block_start;
+}
 
-	trailer_lines = strbuf_split_buf(str + trailer_block_start,
-					 end_of_log_message - trailer_block_start,
-					 '\n',
-					 0);
-	for (ptr = trailer_lines; *ptr; ptr++) {
-		if (last && isspace((*ptr)->buf[0])) {
-			struct strbuf sb = STRBUF_INIT;
-			strbuf_attach(&sb, *last, strlen(*last), strlen(*last));
-			strbuf_addbuf(&sb, *ptr);
-			*last = strbuf_detach(&sb, NULL);
-			continue;
-		}
-		ALLOC_GROW(trailer_strings, nr + 1, alloc);
-		trailer_strings[nr] = strbuf_detach(*ptr, NULL);
-		last = find_separator(trailer_strings[nr], separators) >= 1
-			? &trailer_strings[nr]
-			: NULL;
-		nr++;
-	}
-	strbuf_list_free(trailer_lines);
+size_t trailer_block_end(struct trailer_info *info)
+{
+	return info->trailer_block_end;
+}
 
-	info->blank_line_before_trailer = ends_with_blank_line(str,
-							       trailer_block_start);
-	info->trailer_block_start = trailer_block_start;
-	info->trailer_block_end = end_of_log_message;
-	info->trailers = trailer_strings;
-	info->trailer_nr = nr;
+int blank_line_before_trailer_block(struct trailer_info *info)
+{
+	return info->blank_line_before_trailer;
 }
 
 void trailer_info_release(struct trailer_info *info)
@@ -1122,6 +1169,7 @@ void trailer_info_release(struct trailer_info *info)
 	for (i = 0; i < info->trailer_nr; i++)
 		free(info->trailers[i]);
 	free(info->trailers);
+	free(info);
 }
 
 void format_trailers_from_commit(const char *msg,
@@ -1129,31 +1177,31 @@ void format_trailers_from_commit(const char *msg,
 				 struct strbuf *out)
 {
 	LIST_HEAD(head);
-	struct trailer_info info;
-
-	parse_trailers(&info, msg, &head, opts);
+	struct trailer_info *info = parse_trailers(msg, &head, opts);
 
 	/* If we want the whole block untouched, we can take the fast path. */
 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
 	    !opts->separator && !opts->key_only && !opts->value_only &&
 	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info.trailer_block_start,
-			   info.trailer_block_end - info.trailer_block_start);
+		strbuf_add(out, msg + info->trailer_block_start,
+			   info->trailer_block_end - info->trailer_block_start);
 	} else
 		format_trailers(&head, opts, out);
 
 	free_trailers(&head);
-	trailer_info_release(&info);
+	trailer_info_release(info);
 }
 
 void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
+	struct trailer_info *internal = trailer_info_new();
 	strbuf_init(&iter->key, 0);
 	strbuf_init(&iter->val, 0);
 	strbuf_init(&iter->raw, 0);
 	opts.no_divider = 1;
-	trailer_info_get(&iter->internal.info, msg, &opts);
+	iter->internal.info = internal;
+	iter->internal.info = trailer_info_get(msg, &opts);
 	iter->internal.cur = 0;
 }
 
@@ -1161,8 +1209,8 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 {
 	char *line;
 	int separator_pos;
-	if (iter->internal.cur < iter->internal.info.trailer_nr) {
-		line = iter->internal.info.trailers[iter->internal.cur++];
+	if (iter->internal.cur < iter->internal.info->trailer_nr) {
+		line = iter->internal.info->trailers[iter->internal.cur++];
 		separator_pos = find_separator(line, separators);
 		iter->is_trailer = (separator_pos > 0);
 
@@ -1180,7 +1228,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 
 void trailer_iterator_release(struct trailer_iterator *iter)
 {
-	trailer_info_release(&iter->internal.info);
+	trailer_info_release(iter->internal.info);
 	strbuf_release(&iter->val);
 	strbuf_release(&iter->key);
 	strbuf_release(&iter->raw);
diff --git a/trailer.h b/trailer.h
index d50c9fd79b2..b06da1a7d3a 100644
--- a/trailer.h
+++ b/trailer.h
@@ -4,6 +4,8 @@
 #include "list.h"
 #include "strbuf.h"
 
+struct trailer_info;
+
 enum trailer_where {
 	WHERE_DEFAULT,
 	WHERE_END,
@@ -29,27 +31,6 @@ int trailer_set_where(enum trailer_where *item, const char *value);
 int trailer_set_if_exists(enum trailer_if_exists *item, const char *value);
 int trailer_set_if_missing(enum trailer_if_missing *item, const char *value);
 
-struct trailer_info {
-	/*
-	 * True if there is a blank line before the location pointed to by
-	 * trailer_block_start.
-	 */
-	int blank_line_before_trailer;
-
-	/*
-	 * Offsets to the trailer block start and end positions in the input
-	 * string. If no trailer block is found, these are both set to the
-	 * "true" end of the input (find_end_of_log_message()).
-	 */
-	size_t trailer_block_start, trailer_block_end;
-
-	/*
-	 * Array of trailers found.
-	 */
-	char **trailers;
-	size_t trailer_nr;
-};
-
 /*
  * A list that represents newly-added trailers, such as those provided
  * with the --trailer command line option of git-interpret-trailers.
@@ -89,13 +70,13 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
-void parse_trailers(struct trailer_info *info,
-		    const char *str,
-		    struct list_head *head,
-		    const struct process_trailer_options *opts);
+struct trailer_info *parse_trailers(const char *str,
+				    struct list_head *head,
+				    const struct process_trailer_options *opts);
 
-void trailer_info_get(struct trailer_info *info, const char *str,
-		      const struct process_trailer_options *opts);
+size_t trailer_block_start(struct trailer_info *info);
+size_t trailer_block_end(struct trailer_info *info);
+int blank_line_before_trailer_block(struct trailer_info *info);
 
 void trailer_info_release(struct trailer_info *info);
 
@@ -142,7 +123,7 @@ struct trailer_iterator {
 
 	/* private */
 	struct {
-		struct trailer_info info;
+		struct trailer_info *info;
 		size_t cur;
 	} internal;
 };
-- 
gitgitgadget

[PATCH 07/10] trailer: spread usage of "trailer_block" language

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Deprecate the "trailer_info" struct name and replace it with
"trailer_block". The main reason is to help readability, because
"trailer_info" on the surface sounds like it's about a single trailer
when in reality it is a collection of contiguous lines, at least 25% of
which are trailers.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c | 26 +++++-----
 trailer.c                    | 99 ++++++++++++++++++------------------
 trailer.h                    | 18 +++----
 3 files changed, 71 insertions(+), 72 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 0838a57e157..42d9ca07a56 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -140,8 +140,8 @@ static void interpret_trailers(const char *file,
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
-	struct strbuf trailer_block = STRBUF_INIT;
-	struct trailer_info *info;
+	struct strbuf tb = STRBUF_INIT;
+	struct trailer_block *trailer_block;
 	FILE *outfile = stdout;
 
 	trailer_config_init();
@@ -151,13 +151,13 @@ static void interpret_trailers(const char *file,
 	if (opts->in_place)
 		outfile = create_in_place_tempfile(file);
 
-	info = parse_trailers(sb.buf, &head, opts);
+	trailer_block = parse_trailers(sb.buf, opts, &head);
 
-	/* Print the lines before the trailers */
+	/* Print the lines before the trailer block */
 	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, trailer_block_start(info), outfile);
+		fwrite(sb.buf, 1, trailer_block_start(trailer_block), outfile);
 
-	if (!opts->only_trailers && !blank_line_before_trailer_block(info))
+	if (!opts->only_trailers && !blank_line_before_trailer_block(trailer_block))
 		fprintf(outfile, "\n");
 
 
@@ -171,17 +171,17 @@ static void interpret_trailers(const char *file,
 	}
 
 	/* Print trailer block. */
-	format_trailers(&head, opts, &trailer_block);
-	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
-	strbuf_release(&trailer_block);
+	format_trailers(&head, opts, &tb);
+	fwrite(tb.buf, 1, tb.len, outfile);
+	strbuf_release(&tb);
 
 	free_trailers(&head);
 
-	/* Print the lines after the trailers as is */
+	/* Print the lines after the trailer block as is */
 	if (!opts->only_trailers)
-		fwrite(sb.buf + trailer_block_end(info), 1, sb.len - trailer_block_end(info), outfile);
-
-	trailer_info_release(info);
+		fwrite(sb.buf + trailer_block_end(trailer_block),
+		       1, sb.len - trailer_block_end(trailer_block), outfile);
+	trailer_block_release(trailer_block);
 
 	if (opts->in_place)
 		if (rename_tempfile(&trailers_tempfile, file))
diff --git a/trailer.c b/trailer.c
index 0c66e2d3812..360e76376b8 100644
--- a/trailer.c
+++ b/trailer.c
@@ -11,19 +11,20 @@
  * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
  */
 
-struct trailer_info {
+struct trailer_block {
 	/*
 	 * True if there is a blank line before the location pointed to by
-	 * trailer_block_start.
+	 * "start".
 	 */
 	int blank_line_before_trailer;
 
 	/*
-	 * Offsets to the trailer block start and end positions in the input
-	 * string. If no trailer block is found, these are both set to the
-	 * "true" end of the input (find_end_of_log_message()).
+	 * The locations of the start and end positions of the trailer block
+	 * found, as offsets from the beginning of the source text from which
+	 * this trailer block was parsed. If no trailer block is found, these
+	 * are both set to 0.
 	 */
-	size_t trailer_block_start, trailer_block_end;
+	size_t start, end;
 
 	/*
 	 * Array of trailers found.
@@ -1046,16 +1047,16 @@ void format_trailers(struct list_head *head,
 	}
 }
 
-static struct trailer_info *trailer_info_new(void)
+static struct trailer_block *trailer_block_new(void)
 {
-	struct trailer_info *info = xcalloc(1, sizeof(*info));
-	return info;
+	struct trailer_block *trailer_block = xcalloc(1, sizeof(*trailer_block));
+	return trailer_block;
 }
 
-static struct trailer_info *trailer_info_get(const char *str,
-					     const struct process_trailer_options *opts)
+static struct trailer_block *trailer_block_get(const char *str,
+					       const struct process_trailer_options *opts)
 {
-	struct trailer_info *info = trailer_info_new();
+	struct trailer_block *trailer_block = trailer_block_new();
 	size_t end_of_log_message = 0, trailer_block_start = 0;
 	struct strbuf **trailer_lines, **ptr;
 	char **trailer_strings = NULL;
@@ -1088,34 +1089,34 @@ static struct trailer_info *trailer_info_get(const char *str,
 	}
 	strbuf_list_free(trailer_lines);
 
-	info->blank_line_before_trailer = ends_with_blank_line(str,
-							       trailer_block_start);
-	info->trailer_block_start = trailer_block_start;
-	info->trailer_block_end = end_of_log_message;
-	info->trailers = trailer_strings;
-	info->trailer_nr = nr;
+	trailer_block->blank_line_before_trailer = ends_with_blank_line(str,
+									trailer_block_start);
+	trailer_block->start = trailer_block_start;
+	trailer_block->end = end_of_log_message;
+	trailer_block->trailers = trailer_strings;
+	trailer_block->trailer_nr = nr;
 
-	return info;
+	return trailer_block;
 }
 
 /*
- * Parse trailers in "str", populating the trailer info and "head"
- * linked list structure.
+ * Parse trailers in "str", populating the trailer_block info and "head" linked
+ * list structure.
  */
-struct trailer_info *parse_trailers(const char *str,
-				    struct list_head *head,
-				    const struct process_trailer_options *opts)
+struct trailer_block *parse_trailers(const char *str,
+				     const struct process_trailer_options *opts,
+				     struct list_head *head)
 {
-	struct trailer_info *info;
+	struct trailer_block *trailer_block;
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
 	size_t i;
 
-	info = trailer_info_get(str, opts);
+	trailer_block = trailer_block_get(str, opts);
 
-	for (i = 0; i < info->trailer_nr; i++) {
+	for (i = 0; i < trailer_block->trailer_nr; i++) {
 		int separator_pos;
-		char *trailer = info->trailers[i];
+		char *trailer = trailer_block->trailers[i];
 		if (trailer[0] == comment_line_char)
 			continue;
 		separator_pos = find_separator(trailer, separators);
@@ -1136,7 +1137,7 @@ struct trailer_info *parse_trailers(const char *str,
 		}
 	}
 
-	return info;
+	return trailer_block;
 }
 
 void free_trailers(struct list_head *head)
@@ -1148,28 +1149,28 @@ void free_trailers(struct list_head *head)
 	}
 }
 
-size_t trailer_block_start(struct trailer_info *info)
+size_t trailer_block_start(struct trailer_block *trailer_block)
 {
-	return info->trailer_block_start;
+	return trailer_block->start;
 }
 
-size_t trailer_block_end(struct trailer_info *info)
+size_t trailer_block_end(struct trailer_block *trailer_block)
 {
-	return info->trailer_block_end;
+	return trailer_block->end;
 }
 
-int blank_line_before_trailer_block(struct trailer_info *info)
+int blank_line_before_trailer_block(struct trailer_block *trailer_block)
 {
-	return info->blank_line_before_trailer;
+	return trailer_block->blank_line_before_trailer;
 }
 
-void trailer_info_release(struct trailer_info *info)
+void trailer_block_release(struct trailer_block *trailer_block)
 {
 	size_t i;
-	for (i = 0; i < info->trailer_nr; i++)
-		free(info->trailers[i]);
-	free(info->trailers);
-	free(info);
+	for (i = 0; i < trailer_block->trailer_nr; i++)
+		free(trailer_block->trailers[i]);
+	free(trailer_block->trailers);
+	free(trailer_block);
 }
 
 void format_trailers_from_commit(const char *msg,
@@ -1177,31 +1178,29 @@ void format_trailers_from_commit(const char *msg,
 				 struct strbuf *out)
 {
 	LIST_HEAD(head);
-	struct trailer_info *info = parse_trailers(msg, &head, opts);
+	struct trailer_block *trailer_block = parse_trailers(msg, opts, &head);
 
 	/* If we want the whole block untouched, we can take the fast path. */
 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
 	    !opts->separator && !opts->key_only && !opts->value_only &&
 	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info->trailer_block_start,
-			   info->trailer_block_end - info->trailer_block_start);
+		strbuf_add(out, msg + trailer_block->start,
+			   trailer_block->end - trailer_block->start);
 	} else
 		format_trailers(&head, opts, out);
 
 	free_trailers(&head);
-	trailer_info_release(info);
+	trailer_block_release(trailer_block);
 }
 
 void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
-	struct trailer_info *internal = trailer_info_new();
 	strbuf_init(&iter->key, 0);
 	strbuf_init(&iter->val, 0);
 	strbuf_init(&iter->raw, 0);
 	opts.no_divider = 1;
-	iter->internal.info = internal;
-	iter->internal.info = trailer_info_get(msg, &opts);
+	iter->internal.trailer_block = trailer_block_get(msg, &opts);
 	iter->internal.cur = 0;
 }
 
@@ -1209,8 +1208,8 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 {
 	char *line;
 	int separator_pos;
-	if (iter->internal.cur < iter->internal.info->trailer_nr) {
-		line = iter->internal.info->trailers[iter->internal.cur++];
+	if (iter->internal.cur < iter->internal.trailer_block->trailer_nr) {
+		line = iter->internal.trailer_block->trailers[iter->internal.cur++];
 		separator_pos = find_separator(line, separators);
 		iter->is_trailer = (separator_pos > 0);
 
@@ -1228,7 +1227,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 
 void trailer_iterator_release(struct trailer_iterator *iter)
 {
-	trailer_info_release(iter->internal.info);
+	trailer_block_release(iter->internal.trailer_block);
 	strbuf_release(&iter->val);
 	strbuf_release(&iter->key);
 	strbuf_release(&iter->raw);
diff --git a/trailer.h b/trailer.h
index b06da1a7d3a..5c8503ade78 100644
--- a/trailer.h
+++ b/trailer.h
@@ -4,7 +4,7 @@
 #include "list.h"
 #include "strbuf.h"
 
-struct trailer_info;
+struct trailer_block;
 
 enum trailer_where {
 	WHERE_DEFAULT,
@@ -70,15 +70,15 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
-struct trailer_info *parse_trailers(const char *str,
-				    struct list_head *head,
-				    const struct process_trailer_options *opts);
+struct trailer_block *parse_trailers(const char *str,
+				     const struct process_trailer_options *opts,
+				     struct list_head *head);
 
-size_t trailer_block_start(struct trailer_info *info);
-size_t trailer_block_end(struct trailer_info *info);
-int blank_line_before_trailer_block(struct trailer_info *info);
+size_t trailer_block_start(struct trailer_block *trailer_block);
+size_t trailer_block_end(struct trailer_block *trailer_block);
+int blank_line_before_trailer_block(struct trailer_block *trailer_block);
 
-void trailer_info_release(struct trailer_info *info);
+void trailer_block_release(struct trailer_block *trailer_block);
 
 void trailer_config_init(void);
 void free_trailers(struct list_head *trailers);
@@ -123,7 +123,7 @@ struct trailer_iterator {
 
 	/* private */
 	struct {
-		struct trailer_info *info;
+		struct trailer_block *trailer_block;
 		size_t cur;
 	} internal;
 };
-- 
gitgitgadget

[PATCH 08/10] trailer: prepare to move parse_trailers_from_command_line_args() to builtin

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Expose more functions in the trailer.h API, in preparation for moving
out

    parse_trailers_from_command_line_args()

to interpret-trailer.c, because the trailer API should not be concerned
with command line arguments (as it has nothing to do with trailers
themselves). The interpret-trailers builtin is the only user of the
above function.

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 66 +++++++++++++++++++++++++++----------------------------
 trailer.h | 10 +++++++++
 2 files changed, 42 insertions(+), 34 deletions(-)

diff --git a/trailer.c b/trailer.c
index 360e76376b8..e2d541372a3 100644
--- a/trailer.c
+++ b/trailer.c
@@ -33,7 +33,7 @@ struct trailer_block {
 	size_t trailer_nr;
 };
 
-struct conf_info {
+struct trailer_conf {
 	char *name;
 	char *key;
 	char *command;
@@ -43,7 +43,7 @@ struct conf_info {
 	enum trailer_if_missing if_missing;
 };
 
-static struct conf_info default_conf_info;
+static struct trailer_conf default_trailer_conf;
 
 struct trailer_item {
 	struct list_head list;
@@ -59,7 +59,7 @@ struct arg_item {
 	struct list_head list;
 	char *token;
 	char *value;
-	struct conf_info conf;
+	struct trailer_conf conf;
 };
 
 static LIST_HEAD(conf_head);
@@ -210,7 +210,7 @@ static int check_if_different(struct trailer_item *in_tok,
 	return 1;
 }
 
-static char *apply_command(struct conf_info *conf, const char *arg)
+static char *apply_command(struct trailer_conf *conf, const char *arg)
 {
 	struct strbuf cmd = STRBUF_INIT;
 	struct strbuf buf = STRBUF_INIT;
@@ -424,7 +424,8 @@ int trailer_set_if_missing(enum trailer_if_missing *item, const char *value)
 	return 0;
 }
 
-static void duplicate_conf(struct conf_info *dst, const struct conf_info *src)
+void duplicate_trailer_conf(struct trailer_conf *dst,
+			    const struct trailer_conf *src)
 {
 	*dst = *src;
 	dst->name = xstrdup_or_null(src->name);
@@ -447,7 +448,7 @@ static struct arg_item *get_conf_item(const char *name)
 
 	/* Item does not already exists, create it */
 	CALLOC_ARRAY(item, 1);
-	duplicate_conf(&item->conf, &default_conf_info);
+	duplicate_trailer_conf(&item->conf, &default_trailer_conf);
 	item->conf.name = xstrdup(name);
 
 	list_add_tail(&item->list, &conf_head);
@@ -482,17 +483,17 @@ static int git_trailer_default_config(const char *conf_key, const char *value,
 	variable_name = strrchr(trailer_item, '.');
 	if (!variable_name) {
 		if (!strcmp(trailer_item, "where")) {
-			if (trailer_set_where(&default_conf_info.where,
+			if (trailer_set_where(&default_trailer_conf.where,
 					      value) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
 		} else if (!strcmp(trailer_item, "ifexists")) {
-			if (trailer_set_if_exists(&default_conf_info.if_exists,
+			if (trailer_set_if_exists(&default_trailer_conf.if_exists,
 						  value) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
 		} else if (!strcmp(trailer_item, "ifmissing")) {
-			if (trailer_set_if_missing(&default_conf_info.if_missing,
+			if (trailer_set_if_missing(&default_trailer_conf.if_missing,
 						   value) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
@@ -511,7 +512,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 {
 	const char *trailer_item, *variable_name;
 	struct arg_item *item;
-	struct conf_info *conf;
+	struct trailer_conf *conf;
 	char *name = NULL;
 	enum trailer_info_type type;
 	int i;
@@ -585,9 +586,9 @@ void trailer_config_init(void)
 		return;
 
 	/* Default config must be setup first */
-	default_conf_info.where = WHERE_END;
-	default_conf_info.if_exists = EXISTS_ADD_IF_DIFFERENT_NEIGHBOR;
-	default_conf_info.if_missing = MISSING_ADD;
+	default_trailer_conf.where = WHERE_END;
+	default_trailer_conf.if_exists = EXISTS_ADD_IF_DIFFERENT_NEIGHBOR;
+	default_trailer_conf.if_missing = MISSING_ADD;
 	git_config(git_trailer_default_config, NULL);
 	git_config(git_trailer_config, NULL);
 	configured = 1;
@@ -620,7 +621,7 @@ static int token_matches_item(const char *tok, struct arg_item *item, size_t tok
  * distinguished from the non-well-formed-line case (in which this function
  * returns -1) because some callers of this function need such a distinction.
  */
-static ssize_t find_separator(const char *line, const char *separators)
+ssize_t find_separator(const char *line, const char *separators)
 {
 	int whitespace_found = 0;
 	const char *c;
@@ -645,28 +646,28 @@ static ssize_t find_separator(const char *line, const char *separators)
  *
  * If separator_pos is -1, interpret the whole trailer as a token.
  */
-static void parse_trailer(struct strbuf *tok, struct strbuf *val,
-			 const struct conf_info **conf, const char *trailer,
-			 ssize_t separator_pos)
+void parse_trailer(const char *line, ssize_t separator_pos,
+		   struct strbuf *tok, struct strbuf *val,
+		   const struct trailer_conf **conf)
 {
 	struct arg_item *item;
 	size_t tok_len;
 	struct list_head *pos;
 
 	if (separator_pos != -1) {
-		strbuf_add(tok, trailer, separator_pos);
+		strbuf_add(tok, line, separator_pos);
 		strbuf_trim(tok);
-		strbuf_addstr(val, trailer + separator_pos + 1);
+		strbuf_addstr(val, line + separator_pos + 1);
 		strbuf_trim(val);
 	} else {
-		strbuf_addstr(tok, trailer);
+		strbuf_addstr(tok, line);
 		strbuf_trim(tok);
 	}
 
 	/* Lookup if the token matches something in the config */
 	tok_len = token_len_without_separator(tok->buf, tok->len);
 	if (conf)
-		*conf = &default_conf_info;
+		*conf = &default_trailer_conf;
 	list_for_each(pos, &conf_head) {
 		item = list_entry(pos, struct arg_item, list);
 		if (token_matches_item(tok->buf, item, tok_len)) {
@@ -691,13 +692,13 @@ static struct trailer_item *add_trailer_item(struct list_head *head, char *tok,
 }
 
 static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
-			 const struct conf_info *conf,
+			 const struct trailer_conf *conf,
 			 const struct new_trailer_item *new_trailer_item)
 {
 	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
 	new_item->token = tok;
 	new_item->value = val;
-	duplicate_conf(&new_item->conf, conf);
+	duplicate_trailer_conf(&new_item->conf, conf);
 	if (new_trailer_item) {
 		if (new_trailer_item->where != WHERE_DEFAULT)
 			new_item->conf.where = new_trailer_item->where;
@@ -730,7 +731,7 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
-	const struct conf_info *conf;
+	const struct trailer_conf *conf;
 	struct list_head *pos;
 
 	/*
@@ -753,8 +754,7 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 			      (int) sb.len, sb.buf);
 			strbuf_release(&sb);
 		} else {
-			parse_trailer(&tok, &val, &conf, tr->text,
-				      separator_pos);
+			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
 			add_arg_item(arg_head,
 				     strbuf_detach(&tok, NULL),
 				     strbuf_detach(&val, NULL),
@@ -1116,20 +1116,19 @@ struct trailer_block *parse_trailers(const char *str,
 
 	for (i = 0; i < trailer_block->trailer_nr; i++) {
 		int separator_pos;
-		char *trailer = trailer_block->trailers[i];
-		if (trailer[0] == comment_line_char)
+		char *line = trailer_block->trailers[i];
+		if (line[0] == comment_line_char)
 			continue;
-		separator_pos = find_separator(trailer, separators);
+		separator_pos = find_separator(line, separators);
 		if (separator_pos >= 1) {
-			parse_trailer(&tok, &val, NULL, trailer,
-				      separator_pos);
+			parse_trailer(line, separator_pos, &tok, &val, NULL);
 			if (opts->unfold)
 				unfold_value(&val);
 			add_trailer_item(head,
 					 strbuf_detach(&tok, NULL),
 					 strbuf_detach(&val, NULL));
 		} else if (!opts->only_trailers) {
-			strbuf_addstr(&val, trailer);
+			strbuf_addstr(&val, line);
 			strbuf_strip_suffix(&val, "\n");
 			add_trailer_item(head,
 					 NULL,
@@ -1217,8 +1216,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 		strbuf_addstr(&iter->raw, line);
 		strbuf_reset(&iter->key);
 		strbuf_reset(&iter->val);
-		parse_trailer(&iter->key, &iter->val, NULL,
-			      line, separator_pos);
+		parse_trailer(line, separator_pos, &iter->key, &iter->val, NULL);
 		unfold_value(&iter->val);
 		return 1;
 	}
diff --git a/trailer.h b/trailer.h
index 5c8503ade78..fe49a9bad52 100644
--- a/trailer.h
+++ b/trailer.h
@@ -5,6 +5,7 @@
 #include "strbuf.h"
 
 struct trailer_block;
+struct trailer_conf;
 
 enum trailer_where {
 	WHERE_DEFAULT,
@@ -45,6 +46,9 @@ struct new_trailer_item {
 	enum trailer_if_missing if_missing;
 };
 
+void duplicate_trailer_conf(struct trailer_conf *dst,
+			    const struct trailer_conf *src);
+
 struct process_trailer_options {
 	int in_place;
 	int trim_empty;
@@ -70,6 +74,12 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
+ssize_t find_separator(const char *line, const char *separators);
+
+void parse_trailer(const char *line, ssize_t separator_pos,
+		   struct strbuf *tok, struct strbuf *val,
+		   const struct trailer_conf **conf);
+
 struct trailer_block *parse_trailers(const char *str,
 				     const struct process_trailer_options *opts,
 				     struct list_head *head);
-- 
gitgitgadget

[PATCH 09/10] trailer: move arg handling to interpret-trailers.c

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

We don't move the "arg_item" struct to interpret-trailers.c, because it
is now a struct that contains information about trailers that should be
injected into the input text's own trailers. We will rename this
language as such in a follow-up patch to keep the diff here small.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c | 88 ++++++++++++++++++++++--------------
 trailer.c                    | 63 +++++++++++++++++++-------
 trailer.h                    | 12 +++++
 3 files changed, 113 insertions(+), 50 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 42d9ca07a56..4da4eac3b46 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -45,23 +45,17 @@ static int option_parse_if_missing(const struct option *opt,
 	return trailer_set_if_missing(opt->value, arg);
 }
 
-static void new_trailers_clear(struct list_head *trailers)
-{
-	struct list_head *pos, *tmp;
-	struct new_trailer_item *item;
-
-	list_for_each_safe(pos, tmp, trailers) {
-		item = list_entry(pos, struct new_trailer_item, list);
-		list_del(pos);
-		free(item);
-	}
-}
+static char *cl_separators;
 
 static int option_parse_trailer(const struct option *opt,
 				   const char *arg, int unset)
 {
 	struct list_head *trailers = opt->value;
-	struct new_trailer_item *item;
+	struct strbuf tok = STRBUF_INIT;
+	struct strbuf val = STRBUF_INIT;
+	const struct trailer_conf *conf;
+	struct trailer_conf *conf_current = new_trailer_conf();
+	ssize_t separator_pos;
 
 	if (unset) {
 		new_trailers_clear(trailers);
@@ -71,12 +65,31 @@ static int option_parse_trailer(const struct option *opt,
 	if (!arg)
 		return -1;
 
-	item = xmalloc(sizeof(*item));
-	item->text = arg;
-	item->where = where;
-	item->if_exists = if_exists;
-	item->if_missing = if_missing;
-	list_add_tail(&item->list, trailers);
+	separator_pos = find_separator(arg, cl_separators);
+	if (separator_pos) {
+		parse_trailer(arg, separator_pos, &tok, &val, &conf);
+		duplicate_trailer_conf(conf_current, conf);
+
+		/*
+		 * Override conf_current with settings specified via CLI flags.
+		 */
+		trailer_conf_set(where, if_exists, if_missing, conf_current);
+
+		add_arg_item(strbuf_detach(&tok, NULL),
+			     strbuf_detach(&val, NULL),
+			     conf_current,
+			     trailers);
+	} else {
+		struct strbuf sb = STRBUF_INIT;
+		strbuf_addstr(&sb, arg);
+		strbuf_trim(&sb);
+		error(_("empty trailer token in trailer '%.*s'"),
+			(int) sb.len, sb.buf);
+		strbuf_release(&sb);
+	}
+
+	free(conf_current);
+
 	return 0;
 }
 
@@ -136,7 +149,7 @@ static void read_input_file(struct strbuf *sb, const char *file)
 
 static void interpret_trailers(const char *file,
 			       const struct process_trailer_options *opts,
-			       struct list_head *new_trailer_head)
+			       struct list_head *arg_trailers)
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
@@ -144,8 +157,6 @@ static void interpret_trailers(const char *file,
 	struct trailer_block *trailer_block;
 	FILE *outfile = stdout;
 
-	trailer_config_init();
-
 	read_input_file(&sb, file);
 
 	if (opts->in_place)
@@ -162,12 +173,7 @@ static void interpret_trailers(const char *file,
 
 
 	if (!opts->only_input) {
-		LIST_HEAD(config_head);
-		LIST_HEAD(arg_head);
-		parse_trailers_from_config(&config_head);
-		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
-		list_splice(&config_head, &arg_head);
-		process_trailers_lists(&head, &arg_head);
+		process_trailers_lists(&head, arg_trailers);
 	}
 
 	/* Print trailer block. */
@@ -193,7 +199,8 @@ static void interpret_trailers(const char *file,
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
-	LIST_HEAD(trailers);
+	LIST_HEAD(configured_trailers);
+	LIST_HEAD(arg_trailers);
 
 	struct option options[] = {
 		OPT_BOOL(0, "in-place", &opts.in_place, N_("edit files in place")),
@@ -212,33 +219,48 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 		OPT_CALLBACK_F(0, "parse", &opts, NULL, N_("alias for --only-trailers --only-input --unfold"),
 			PARSE_OPT_NOARG | PARSE_OPT_NONEG, parse_opt_parse),
 		OPT_BOOL(0, "no-divider", &opts.no_divider, N_("do not treat \"---\" as the end of input")),
-		OPT_CALLBACK(0, "trailer", &trailers, N_("trailer"),
+		OPT_CALLBACK(0, "trailer", &arg_trailers, N_("trailer"),
 				N_("trailer(s) to add"), option_parse_trailer),
 		OPT_END()
 	};
 
 	git_config(git_default_config, NULL);
+	trailer_config_init();
+
+	if (!opts.only_input) {
+		parse_trailers_from_config(&configured_trailers);
+	}
+
+	/*
+	* In command-line arguments, '=' is accepted (in addition to the
+	* separators that are defined).
+	*/
+	cl_separators = xstrfmt("=%s", default_separators());
 
 	argc = parse_options(argc, argv, prefix, options,
 			     git_interpret_trailers_usage, 0);
 
-	if (opts.only_input && !list_empty(&trailers))
+	free(cl_separators);
+
+	if (opts.only_input && !list_empty(&arg_trailers))
 		usage_msg_opt(
 			_("--trailer with --only-input does not make sense"),
 			git_interpret_trailers_usage,
 			options);
 
+	list_splice(&configured_trailers, &arg_trailers);
+
 	if (argc) {
 		int i;
 		for (i = 0; i < argc; i++)
-			interpret_trailers(argv[i], &opts, &trailers);
+			interpret_trailers(argv[i], &opts, &arg_trailers);
 	} else {
 		if (opts.in_place)
 			die(_("no input file given for in-place editing"));
-		interpret_trailers(NULL, &opts, &trailers);
+		interpret_trailers(NULL, &opts, &arg_trailers);
 	}
 
-	new_trailers_clear(&trailers);
+	new_trailers_clear(&arg_trailers);
 
 	return 0;
 }
diff --git a/trailer.c b/trailer.c
index e2d541372a3..0a86e0d5afa 100644
--- a/trailer.c
+++ b/trailer.c
@@ -66,6 +66,11 @@ static LIST_HEAD(conf_head);
 
 static char *separators = ":";
 
+const char *default_separators(void)
+{
+	return separators;
+}
+
 static int configured;
 
 #define TRAILER_ARG_STRING "$ARG"
@@ -424,6 +429,25 @@ int trailer_set_if_missing(enum trailer_if_missing *item, const char *value)
 	return 0;
 }
 
+void trailer_conf_set(enum trailer_where where,
+		      enum trailer_if_exists if_exists,
+		      enum trailer_if_missing if_missing,
+		      struct trailer_conf *conf)
+{
+	if (where != WHERE_DEFAULT)
+		conf->where = where;
+	if (if_exists != EXISTS_DEFAULT)
+		conf->if_exists = if_exists;
+	if (if_missing != MISSING_DEFAULT)
+		conf->if_missing = if_missing;
+}
+
+struct trailer_conf *new_trailer_conf(void)
+{
+	struct trailer_conf *new = xcalloc(1, sizeof(*new));
+	return new;
+}
+
 void duplicate_trailer_conf(struct trailer_conf *dst,
 			    const struct trailer_conf *src)
 {
@@ -642,6 +666,9 @@ ssize_t find_separator(const char *line, const char *separators)
 /*
  * Obtain the token, value, and conf from the given trailer.
  *
+ * The conf needs special handling. We first read hardcoded defaults, and
+ * override them if we find a matching trailer configuration in the config.
+ *
  * separator_pos must not be 0, since the token cannot be an empty string.
  *
  * If separator_pos is -1, interpret the whole trailer as a token.
@@ -691,22 +718,14 @@ static struct trailer_item *add_trailer_item(struct list_head *head, char *tok,
 	return new_item;
 }
 
-static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
-			 const struct trailer_conf *conf,
-			 const struct new_trailer_item *new_trailer_item)
+void add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
+		  struct list_head *arg_head)
+
 {
 	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
 	new_item->token = tok;
 	new_item->value = val;
 	duplicate_trailer_conf(&new_item->conf, conf);
-	if (new_trailer_item) {
-		if (new_trailer_item->where != WHERE_DEFAULT)
-			new_item->conf.where = new_trailer_item->where;
-		if (new_trailer_item->if_exists != EXISTS_DEFAULT)
-			new_item->conf.if_exists = new_trailer_item->if_exists;
-		if (new_trailer_item->if_missing != MISSING_DEFAULT)
-			new_item->conf.if_missing = new_trailer_item->if_missing;
-	}
 	list_add_tail(&new_item->list, arg_head);
 }
 
@@ -719,10 +738,10 @@ void parse_trailers_from_config(struct list_head *config_head)
 	list_for_each(pos, &conf_head) {
 		item = list_entry(pos, struct arg_item, list);
 		if (item->conf.command)
-			add_arg_item(config_head,
-				     xstrdup(token_from_item(item, NULL)),
+			add_arg_item(xstrdup(token_from_item(item, NULL)),
 				     xstrdup(""),
-				     &item->conf, NULL);
+				     &item->conf,
+				     config_head);
 	}
 }
 
@@ -755,10 +774,10 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 			strbuf_release(&sb);
 		} else {
 			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
-			add_arg_item(arg_head,
-				     strbuf_detach(&tok, NULL),
+			add_arg_item(strbuf_detach(&tok, NULL),
 				     strbuf_detach(&val, NULL),
-				     conf, tr);
+				     conf,
+				     arg_head);
 		}
 	}
 
@@ -1148,6 +1167,16 @@ void free_trailers(struct list_head *head)
 	}
 }
 
+void new_trailers_clear(struct list_head *trailers)
+{
+	struct list_head *pos, *p;
+
+	list_for_each_safe(pos, p, trailers) {
+		list_del(pos);
+		free_arg_item(list_entry(pos, struct arg_item, list));
+	}
+}
+
 size_t trailer_block_start(struct trailer_block *trailer_block)
 {
 	return trailer_block->start;
diff --git a/trailer.h b/trailer.h
index fe49a9bad52..9b86acfe2d4 100644
--- a/trailer.h
+++ b/trailer.h
@@ -46,9 +46,20 @@ struct new_trailer_item {
 	enum trailer_if_missing if_missing;
 };
 
+void trailer_conf_set(enum trailer_where where,
+		      enum trailer_if_exists if_exists,
+		      enum trailer_if_missing if_missing,
+		      struct trailer_conf *conf);
+
+struct trailer_conf *new_trailer_conf(void);
 void duplicate_trailer_conf(struct trailer_conf *dst,
 			    const struct trailer_conf *src);
 
+const char *default_separators(void);
+
+void add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
+		  struct list_head *arg_head);
+
 struct process_trailer_options {
 	int in_place;
 	int trim_empty;
@@ -92,6 +103,7 @@ void trailer_block_release(struct trailer_block *trailer_block);
 
 void trailer_config_init(void);
 void free_trailers(struct list_head *trailers);
+void new_trailers_clear(struct list_head *trailers);
 
 void format_trailers(struct list_head *head,
 		     const struct process_trailer_options *opts,
-- 
gitgitgadget

[PATCH 10/10] trailer: delete obsolete argument handling code from API

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This commit was not squashed with its parent in order to keep the diff
separate (to make the additive changes in the parent easier to read).

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 39 ---------------------------------------
 trailer.h | 17 -----------------
 2 files changed, 56 deletions(-)

diff --git a/trailer.c b/trailer.c
index 0a86e0d5afa..27bb2195f53 100644
--- a/trailer.c
+++ b/trailer.c
@@ -745,45 +745,6 @@ void parse_trailers_from_config(struct list_head *config_head)
 	}
 }
 
-void parse_trailers_from_command_line_args(struct list_head *arg_head,
-					   struct list_head *new_trailer_head)
-{
-	struct strbuf tok = STRBUF_INIT;
-	struct strbuf val = STRBUF_INIT;
-	const struct trailer_conf *conf;
-	struct list_head *pos;
-
-	/*
-	 * In command-line arguments, '=' is accepted (in addition to the
-	 * separators that are defined).
-	 */
-	char *cl_separators = xstrfmt("=%s", separators);
-
-	/* Add an arg item for each trailer on the command line */
-	list_for_each(pos, new_trailer_head) {
-		struct new_trailer_item *tr =
-			list_entry(pos, struct new_trailer_item, list);
-		ssize_t separator_pos = find_separator(tr->text, cl_separators);
-
-		if (separator_pos == 0) {
-			struct strbuf sb = STRBUF_INIT;
-			strbuf_addstr(&sb, tr->text);
-			strbuf_trim(&sb);
-			error(_("empty trailer token in trailer '%.*s'"),
-			      (int) sb.len, sb.buf);
-			strbuf_release(&sb);
-		} else {
-			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
-			add_arg_item(strbuf_detach(&tok, NULL),
-				     strbuf_detach(&val, NULL),
-				     conf,
-				     arg_head);
-		}
-	}
-
-	free(cl_separators);
-}
-
 static const char *next_line(const char *str)
 {
 	const char *nl = strchrnul(str, '\n');
diff --git a/trailer.h b/trailer.h
index 9b86acfe2d4..8a89e95c171 100644
--- a/trailer.h
+++ b/trailer.h
@@ -32,20 +32,6 @@ int trailer_set_where(enum trailer_where *item, const char *value);
 int trailer_set_if_exists(enum trailer_if_exists *item, const char *value);
 int trailer_set_if_missing(enum trailer_if_missing *item, const char *value);
 
-/*
- * A list that represents newly-added trailers, such as those provided
- * with the --trailer command line option of git-interpret-trailers.
- */
-struct new_trailer_item {
-	struct list_head list;
-
-	const char *text;
-
-	enum trailer_where where;
-	enum trailer_if_exists if_exists;
-	enum trailer_if_missing if_missing;
-};
-
 void trailer_conf_set(enum trailer_where where,
 		      enum trailer_if_exists if_exists,
 		      enum trailer_if_missing if_missing,
@@ -79,9 +65,6 @@ struct process_trailer_options {
 
 void parse_trailers_from_config(struct list_head *config_head);
 
-void parse_trailers_from_command_line_args(struct list_head *arg_head,
-					   struct list_head *new_trailer_head);
-
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
-- 
gitgitgadget

Re: [PATCH 00/10] Enrich Trailer API

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> This patch series is the first 10 patches of a much larger series I've been
> working. The main goal of this series is to enrich the API in trailer.h. The
> larger series brings a number of additional code simplifications and
> cleanups (exposing and fixing some bugs along the way), and builds on top of
> this series. The goal of the larger series is to make the trailer interface
> ready for unit testing. By "trailer API" I mean those functions exposed in
> trailer.h.

Are there places in the current code that deals with trailers but
does not use the trailer API (e.g., manually parse and/or insert the
trailer in an in-core buffer)?  Is it part of the larger goal to
update these places so that we will always use the trailer API to
touch trailers, and if so, have these places been identified?

Obviously the reason why I ask is that testing cannot be the goal by
itself.  The "alternative" ...

> As an alternative to this patch series, we could keep trailer.h intact and
> decide to unit-test the existing "trailer_info_get()" function which does
> most of the trailer parsing work.

... may allow you to "test", but it would make it more difficult in
the future to revamp the trailer API, if it is needed, in order to
cover code paths that ought to be using but currently bypassing the
trailer API.

> This series breaks up "process_trailers()" into smaller pieces, exposing
> many of the parts relevant to trailer-related processing in trailer.h. This
> forces us to start writing unit tests for these now public functions, but
> that is a good thing because those same unit tests should be easy to write
> (due to their small(er) sizes), but also, because those unit tests will now
> ensure some degree of stability across new versions of trailer.h (we will
> start noticing when the behavior of any of these API functions change).

And helper functions, each of which does one small thing well, may
be more applicable to other code paths that are currently bypassing
the API.

> Thanks to the aggressive refactoring in this series, I've been able to
> identify and fix a couple bugs in our existing implementation. Those fixes
> build on top of this series but were not included here, in order to keep
> this series small.

It would be nicer to have a concise list of these fixes (in the form
of "git shortlog") as a teaser here ;-).  That would hopefully
entice others into reviewing this part that forms the foundation.

Thanks.

Re: [PATCH 00/10] Enrich Trailer API

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> This patch series is the first 10 patches of a much larger series I've been
>> working. The main goal of this series is to enrich the API in trailer.h. The
>> larger series brings a number of additional code simplifications and
>> cleanups (exposing and fixing some bugs along the way), and builds on top of
>> this series. The goal of the larger series is to make the trailer interface
>> ready for unit testing. By "trailer API" I mean those functions exposed in
>> trailer.h.
>
> Are there places in the current code that deals with trailers but
> does not use the trailer API (e.g., manually parse and/or insert the
> trailer in an in-core buffer)?

While working on this series I unfortunately did not search for such use
cases (I limited the scope of my work to mainly the interpret-trailers
builtin). But, a quick search just now on master branch turned up
append_signoff() in sequencer.c which constructs a Signed-off-by trailer
manually with a raw strbuf [1].

This is somewhat understandable though, because AFAICS the current
trailer API does not support creating and formatting single trailer
objects.

> Is it part of the larger goal to
> update these places so that we will always use the trailer API to
> touch trailers

That sounds like The Right Thing to do.

> , and if so, have these places been identified?

Not yet, but, it should be easy to check any remaining cases other than
the one I identified up above.

> Obviously the reason why I ask is that testing cannot be the goal by
> itself.

I now seem to recall an offline discussion where I said I wanted to
enrich the API to make other parts of Git also use this new API. Somehow
I've left that motivation out of the cover letter (will include in the
next reroll).

> The "alternative" ...
>
>> As an alternative to this patch series, we could keep trailer.h intact and
>> decide to unit-test the existing "trailer_info_get()" function which does
>> most of the trailer parsing work.
>
> ... may allow you to "test", but it would make it more difficult in
> the future to revamp the trailer API, if it is needed, in order to
> cover code paths that ought to be using but currently bypassing the
> trailer API.

Agreed. I should include this rationale as well in the next cover
letter, thanks.

>> This series breaks up "process_trailers()" into smaller pieces, exposing
>> many of the parts relevant to trailer-related processing in trailer.h. This
>> forces us to start writing unit tests for these now public functions, but
>> that is a good thing because those same unit tests should be easy to write
>> (due to their small(er) sizes), but also, because those unit tests will now
>> ensure some degree of stability across new versions of trailer.h (we will
>> start noticing when the behavior of any of these API functions change).
>
> And helper functions, each of which does one small thing well, may
> be more applicable to other code paths that are currently bypassing
> the API.

Yep.

>> Thanks to the aggressive refactoring in this series, I've been able to
>> identify and fix a couple bugs in our existing implementation. Those fixes
>> build on top of this series but were not included here, in order to keep
>> this series small.
>
> It would be nicer to have a concise list of these fixes (in the form
> of "git shortlog") as a teaser here ;-).  That would hopefully
> entice others into reviewing this part that forms the foundation.

Ah, good idea. Here's a shortlog (with a short summary of each one) of
bug fixes that are forthcoming:

    trailer: trailer replacement should not change its position
      (Summary: If we found a trailer we'd like to replace, preserve its
      position relative to the other trailers found in the trailer
      block, instead of always moving it to the beginning or end of the
      entire trailer block.)

    interpret-trailers: preserve trailers coming from the input
      (Summary: Sometimes, the parsed trailers from the input will be
      formatted differently depending on whether we provide
      --only-trailers or not.  Make the trailers that were not modified
      and are coming directly from the input get formatted the same way,
      regardless of this flag.)

    interpret-trailers: fail if given unrecognized arguments
      (Summary: E.g., for "--where", only accept recognized WHERE_* enum
      values. If we get something unrecognized, fail with an error
      instead of silently doing nothing. Ditto for "--if-exists" and
      "--if-missing".)

The last one is a different class of bug than the first two, and perhaps
less interesting.

Thanks.

[1] https://github.com/git/git/blob/d4dbce1db5cd227a57074bcfc7ec9f0655961bba/sequencer.c#L5299-L5301

Re: [PATCH 00/10] Enrich Trailer API

[Linus Arver]

Linus Arver <linusa@google.com> writes:

>     interpret-trailers: fail if given unrecognized arguments
>       (Summary: E.g., for "--where", only accept recognized WHERE_* enum
>       values. If we get something unrecognized, fail with an error
>       instead of silently doing nothing. Ditto for "--if-exists" and
>       "--if-missing".)
>
> The last one is a different class of bug than the first two, and perhaps
> less interesting.

Actually, upon closer inspection I realize that we already fail if given
unrecognized arguments to --where, --if-exists, and --if-missing. But we
don't explain why to the user because no error message is printed. This
commit has been retitled to "interpret-trailers: print error if given
unrecognized arguments".

Thanks.

Re: [PATCH 01/10] trailer: move process_trailers() to interpret-trailers.c

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> diff --git a/trailer.h b/trailer.h
> index 1644cd05f60..b3e4a5e127d 100644
> --- a/trailer.h
> +++ b/trailer.h
> @@ -81,15 +81,29 @@ struct process_trailer_options {
>  
>  #define PROCESS_TRAILER_OPTIONS_INIT {0}
>  
> -void process_trailers(const char *file,
> -		      const struct process_trailer_options *opts,
> -		      struct list_head *new_trailer_head);
> +void parse_trailers_from_config(struct list_head *config_head);
> +
> +void parse_trailers_from_command_line_args(struct list_head *arg_head,
> +					   struct list_head *new_trailer_head);
> +
> +void process_trailers_lists(struct list_head *head,
> +			    struct list_head *arg_head);
> +
> +void parse_trailers(struct trailer_info *info,
> +		    const char *str,
> +		    struct list_head *head,
> +		    const struct process_trailer_options *opts);

OK.

> +void ensure_configured(void);
> +void print_all(FILE *outfile, struct list_head *head,
> +	       const struct process_trailer_options *opts);
> +void free_all(struct list_head *head);

All of these names are way overly generic to live in the global
namespace, no?  Granted, they may only be visible to folks who
include <trailer.h>, but sooner or later other subsystems would find
need to provide library-ish functions that allows their callers to
do these things to their subsystem: make sure the subsystem has been
initialized, print info on all "things" on a list that the subsystem
works on, and free all "things" on such a list.

It is sad to have a function that takes "struct list_head *" as its
parameter without having any comment on what the elements on that
list are about.  free_trailer_items() might be a more appropriate
name (or free_all_trailer_items(), if we foresee a need to just free
the first item on the list and give such a helper
free_trailer_item(), in which case, the distinction between "item"
vs "items" might become too subtle.

I am assuming (and did not verify more than seeing the "git show"
output with "--color-moved") that other changes in this step are all
pure moves?  If so, other than need to be careful about naming when
making things from static to extern pointed out above, this step
looks OK to me.

Thanks.

Re: [PATCH 02/10] trailer: include "trailer" term in API functions

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> These functions are exposed to clients and so they should include
> "trailer" in their names for easier identification, just like all the
> other functions already exposed by trailer.h.

Oh, if you were to do this and code movement in two separate
patches, doing the rename before the move in the previous step would
have made much more sense.  If we stopped after 01/10, the tree
would have been a very sorry state.  If this came first, even if we
stop after these renaming of internal functions that are not extern,
nobody wil be hurt by these new and improved names.

Re: [PATCH 03/10] trailer: unify trailer formatting machinery

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> Currently have two functions for formatting trailers exposed in
> trailer.h:
>
>     void format_trailers(FILE *outfile, struct list_head *head,
>                         const struct process_trailer_options *opts);
>
>     void format_trailers_from_commit(struct strbuf *out, const char *msg,
>                                     const struct process_trailer_options *opts);
>
> and previously these functions, although similar enough (even taking the
> same process_trailer_options struct pointer), did not build on each
> other.
>
> Make format_trailers_from_commit() rely on format_trailers(). Teach
> format_trailers() to process trailers with the additional
> process_trailer_options fields like opts->key_only which is only used by
> format_trailers_from_commit() and not builtin/interpret-trailers.c.

Yay.  It feels a bit disappointing to see the diffstat and learn
that we are not deleting substantial number of lines.

> ---
>  builtin/interpret-trailers.c |   5 +-
>  pretty.c                     |   2 +-
>  ref-filter.c                 |   2 +-
>  trailer.c                    | 105 +++++++++++++++++++++++++++++------
>  trailer.h                    |  21 +++----
>  5 files changed, 102 insertions(+), 33 deletions(-)

> diff --git a/pretty.c b/pretty.c
> index cf964b060cd..f0721a5214f 100644
> --- a/pretty.c
> +++ b/pretty.c
> @@ -1759,7 +1759,7 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */
>  				goto trailer_out;
>  		}
>  		if (*arg == ')') {
> -			format_trailers_from_commit(sb, msg + c->subject_off, &opts);
> +			format_trailers_from_commit(msg + c->subject_off, &opts, sb);

I am curious (read: no objection---merely wondering if there is a
guiding principle behind the choice of the new order) why this new
parameter ordering.  I suspect it was originally written with a
strbuf-centric worldview and having sb at the beginning may have
made sense, but if we are moving them around, wouldn't it make more
sense to have &opts as the first parameter, as this is primarily a
"trailers" function?  Unsure until I read through to the end, but
that is my gut reaction.

>  static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
>  {
>  	struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
> @@ -984,6 +971,78 @@ static void unfold_value(struct strbuf *val)
>  	strbuf_release(&out);
>  }
>  
> +void format_trailers(struct list_head *head,
> +		     const struct process_trailer_options *opts,
> +		     struct strbuf *out)
> +{
> +	struct list_head *pos;
> +	struct trailer_item *item;
> +	int need_separator = 0;
> +
> +	list_for_each(pos, head) {
> +		item = list_entry(pos, struct trailer_item, list);
> +		if (item->token) {
> +			char c;
> + ...
> +			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
> +				if (opts->unfold)
> +					unfold_value(&val);
> +
> +				if (opts->separator && need_separator)
> +					strbuf_addbuf(out, opts->separator);
> +				if (!opts->value_only)
> +					strbuf_addbuf(out, &tok);
> +				if (!opts->key_only && !opts->value_only) {
> +					if (opts->key_value_separator)
> +						strbuf_addbuf(out, opts->key_value_separator);
> +					else {
> +						c = last_non_space_char(tok.buf);
> +						if (c) {
> +							if (!strchr(separators, c))
> +								strbuf_addf(out, "%c ", separators[0]);
> +						}
> +					}

That's an overly deep nesting.  I wonder if a small file-scope
helper function is in order?

	static add_separator(struct process_trailer_options *opts,
        		     const char *token
			     struct strbuf *out)
	{
		if (opts->key_value_separator)
			strbuf_addbuf(out, opts->key_value_separator);
		else
			strbuf_addstr(out, ": ");
	}

Or perhaps inside the context of the loop to go over the list of
trailer items, one iteration of the list_for_each() loop can become
one call to a small helper function format_one_trailer() and that
may make the result easier to follow?

In any case, I didn't see anything glaringly wrong so far in this
series.  Let me keep reading.

Thanks.

Re: [PATCH 02/10] trailer: include "trailer" term in API functions

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> From: Linus Arver <linusa@google.com>
>>
>> These functions are exposed to clients and so they should include
>> "trailer" in their names for easier identification, just like all the
>> other functions already exposed by trailer.h.
>
> Oh, if you were to do this and code movement in two separate
> patches, doing the rename before the move in the previous step would
> have made much more sense.  If we stopped after 01/10, the tree
> would have been a very sorry state.  If this came first, even if we
> stop after these renaming of internal functions that are not extern,
> nobody wil be hurt by these new and improved names.

Ah very true. I think I was thinking of the series as one atomic thing
(all 10 patches or nothing), which is a bad habit I need to break. I'll
reorder these two on the next reroll, because principles matter (and
it's an easy mechanical change). Thanks.

Re: [PATCH 02/10] trailer: include "trailer" term in API functions

[Junio C Hamano]

Linus Arver <linusa@google.com> writes:

> Ah very true. I think I was thinking of the series as one atomic thing
> (all 10 patches or nothing), which is a bad habit I need to break. I'll
> reorder these two on the next reroll, because principles matter (and
> it's an easy mechanical change). Thanks.

Thanks.  I wouldn't have minded if these renames and moves were done
in a single step, but because you already have them as separate two
steps, let's keep them separate.

Re: [PATCH 01/10] trailer: move process_trailers() to interpret-trailers.c

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> I am assuming (and did not verify more than seeing the "git show"
> output with "--color-moved") that other changes in this step are all
> pure moves?

Yup.

Re: [PATCH 04/10] trailer: delete obsolete formatting functions

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> Signed-off-by: Linus Arver <linusa@google.com>
> ---
>  trailer.c | 79 -------------------------------------------------------
>  1 file changed, 79 deletions(-)

OK, these unused functions are the reason why 03/10 weren't removing
as many lines as I would have expected from the refactoring.

Looking good.

Re: [PATCH 05/10] sequencer: use the trailer iterator

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> This patch allows for the removal of "trailer_info_get()" from the
> trailer.h API, which will be in the next patch.
>
> Instead of calling "trailer_info_get()", which is a low-level function
> in the trailers implementation (trailer.c), call
> trailer_iterator_advance(), which was specifically designed for public
> consumption in f0939a0eb1 (trailer: add interface for iterating over
> commit trailers, 2020-09-27).
>
> Avoiding "trailer_info_get()" means we don't have to worry about options
> like "no_divider" (relevant for parsing trailers). We also don't have to
> check for things like "info.trailer_start == info.trailer_end" to see
> whether there were any trailers (instead we can just check to see
> whether the iterator advanced at all).

Nice.

> Also, teach the iterator about non-trailer lines, by adding a new field
> called "raw" to hold both trailer and non-trailer lines. This is
> necessary because a "trailer block" is a list of trailer lines of at
> least 25% trailers (see 146245063e (trailer: allow non-trailers in
> trailer block, 2016-10-21)), such that it may hold non-trailer lines.

OK.  This would change behaviour, wouldn't it, in the sense that we
used to yield a non-trailer line from the old iterator but the new
one skips them?  Is that something we can demonstrate and protect in
tests (presumably once we conclude these refactoring, we would be
able to call into this machinery from unit-testing framework)?

Thanks.

> Signed-off-by: Linus Arver <linusa@google.com>
> ---
>  builtin/shortlog.c |  7 +++++--
>  sequencer.c        | 35 +++++++++++++++--------------------
>  trailer.c          | 20 ++++++++++++--------
>  trailer.h          | 13 +++++++++++++
>  4 files changed, 45 insertions(+), 30 deletions(-)
>
> diff --git a/builtin/shortlog.c b/builtin/shortlog.c
> index 1307ed2b88a..dc8fd5a5532 100644
> --- a/builtin/shortlog.c
> +++ b/builtin/shortlog.c
> @@ -172,7 +172,7 @@ static void insert_records_from_trailers(struct shortlog *log,
>  					 const char *oneline)
>  {
>  	struct trailer_iterator iter;
> -	const char *commit_buffer, *body;
> +	const char *commit_buffer, *body, *value;
>  	struct strbuf ident = STRBUF_INIT;
>  
>  	if (!log->trailers.nr)
> @@ -190,7 +190,10 @@ static void insert_records_from_trailers(struct shortlog *log,
>  
>  	trailer_iterator_init(&iter, body);
>  	while (trailer_iterator_advance(&iter)) {
> -		const char *value = iter.val.buf;
> +		if (!iter.is_trailer)
> +			continue;
> +
> +		value = iter.val.buf;
>  
>  		if (!string_list_has_string(&log->trailers, iter.key.buf))
>  			continue;
> diff --git a/sequencer.c b/sequencer.c
> index 3cc88d8a800..d199869cda9 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -319,37 +319,32 @@ static const char *get_todo_path(const struct replay_opts *opts)
>  static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
>  	size_t ignore_footer)
>  {
> -	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
> -	struct trailer_info info;
> -	size_t i;
> -	int found_sob = 0, found_sob_last = 0;
> -	char saved_char;
> -
> -	opts.no_divider = 1;
> +	struct trailer_iterator iter;
> +	size_t i = 0, found_sob = 0;
> +	char saved_char = sb->buf[sb->len - ignore_footer];
>  
>  	if (ignore_footer) {
> -		saved_char = sb->buf[sb->len - ignore_footer];
>  		sb->buf[sb->len - ignore_footer] = '\0';
>  	}
>  
> -	trailer_info_get(&info, sb->buf, &opts);
> +	trailer_iterator_init(&iter, sb->buf);
> +	while (trailer_iterator_advance(&iter)) {
> +		i++;
> +		if (sob &&
> +		    iter.is_trailer &&
> +		    !strncmp(iter.raw.buf, sob->buf, sob->len)) {
> +			found_sob = i;
> +		}
> +	}
> +	trailer_iterator_release(&iter);
>  
>  	if (ignore_footer)
>  		sb->buf[sb->len - ignore_footer] = saved_char;
>  
> -	if (info.trailer_block_start == info.trailer_block_end)
> +	if (!i)
>  		return 0;
>  
> -	for (i = 0; i < info.trailer_nr; i++)
> -		if (sob && !strncmp(info.trailers[i], sob->buf, sob->len)) {
> -			found_sob = 1;
> -			if (i == info.trailer_nr - 1)
> -				found_sob_last = 1;
> -		}
> -
> -	trailer_info_release(&info);
> -
> -	if (found_sob_last)
> +	if (found_sob == i)
>  		return 3;
>  	if (found_sob)
>  		return 2;
> diff --git a/trailer.c b/trailer.c
> index 132f22b3dd7..593717fd56c 100644
> --- a/trailer.c
> +++ b/trailer.c
> @@ -1151,6 +1151,7 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
>  	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
>  	strbuf_init(&iter->key, 0);
>  	strbuf_init(&iter->val, 0);
> +	strbuf_init(&iter->raw, 0);
>  	opts.no_divider = 1;
>  	trailer_info_get(&iter->internal.info, msg, &opts);
>  	iter->internal.cur = 0;
> @@ -1158,17 +1159,19 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
>  
>  int trailer_iterator_advance(struct trailer_iterator *iter)
>  {
> -	while (iter->internal.cur < iter->internal.info.trailer_nr) {
> -		char *trailer = iter->internal.info.trailers[iter->internal.cur++];
> -		int separator_pos = find_separator(trailer, separators);
> -
> -		if (separator_pos < 1)
> -			continue; /* not a real trailer */
> -
> +	char *line;
> +	int separator_pos;
> +	if (iter->internal.cur < iter->internal.info.trailer_nr) {
> +		line = iter->internal.info.trailers[iter->internal.cur++];
> +		separator_pos = find_separator(line, separators);
> +		iter->is_trailer = (separator_pos > 0);
> +
> +		strbuf_reset(&iter->raw);
> +		strbuf_addstr(&iter->raw, line);
>  		strbuf_reset(&iter->key);
>  		strbuf_reset(&iter->val);
>  		parse_trailer(&iter->key, &iter->val, NULL,
> -			      trailer, separator_pos);
> +			      line, separator_pos);
>  		unfold_value(&iter->val);
>  		return 1;
>  	}
> @@ -1180,4 +1183,5 @@ void trailer_iterator_release(struct trailer_iterator *iter)
>  	trailer_info_release(&iter->internal.info);
>  	strbuf_release(&iter->val);
>  	strbuf_release(&iter->key);
> +	strbuf_release(&iter->raw);
>  }
> diff --git a/trailer.h b/trailer.h
> index 50f70556302..d50c9fd79b2 100644
> --- a/trailer.h
> +++ b/trailer.h
> @@ -127,6 +127,19 @@ struct trailer_iterator {
>  	struct strbuf key;
>  	struct strbuf val;
>  
> +	/*
> +	 * Raw line (e.g., "foo: bar baz") before being parsed as a trailer
> +	 * key/val pair. This field can contain non-trailer lines because it's
> +	 * valid for a trailer block to contain such lines (i.e., we only
> +	 * require 25% of the lines in a trailer block to be trailer lines).
> +	 */
> +	struct strbuf raw;
> +
> +	/*
> +	 * 1 if the raw line was parsed as a separate key/val pair.
> +	 */
> +	int is_trailer;
> +
>  	/* private */
>  	struct {
>  		struct trailer_info info;

Re: [PATCH 06/10] trailer: make trailer_info struct private

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> In 13211ae23f (trailer: separate public from internal portion of
> trailer_iterator, 2023-09-09) we moved trailer_info behind an anonymous
> struct to discourage use by trailer.h API users. However it still left
> open the possibility of external use of trailer_info itself. Now that
> there are no external users of trailer_info, we can make this struct
> private.

Surely, and looking good from a cursory glance.  I'll defer to
Christian to comment on this step as an area expert, though.

Thanks.

Re: [PATCH 07/10] trailer: spread usage of "trailer_block" language

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> Deprecate the "trailer_info" struct name and replace it with
> "trailer_block". The main reason is to help readability, because
> "trailer_info" on the surface sounds like it's about a single trailer
> when in reality it is a collection of contiguous lines, at least 25% of
> which are trailers.

Together with the introduction of trailer_block_{start,end}()
accessor functions in the previous step, this sounds like the right
thing to do.

"info" is a word with so low information contents.  At least "block"
would hint that we are talking about a block of text that makes up
the trailer attached to a single commit.

Re: [PATCH 03/10] trailer: unify trailer formatting machinery

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> From: Linus Arver <linusa@google.com>
>>
>> Currently have two functions for formatting trailers exposed in
>> trailer.h:
>>
>>     void format_trailers(FILE *outfile, struct list_head *head,
>>                         const struct process_trailer_options *opts);
>>
>>     void format_trailers_from_commit(struct strbuf *out, const char *msg,
>>                                     const struct process_trailer_options *opts);
>>
>> and previously these functions, although similar enough (even taking the
>> same process_trailer_options struct pointer), did not build on each
>> other.
>>
>> Make format_trailers_from_commit() rely on format_trailers(). Teach
>> format_trailers() to process trailers with the additional
>> process_trailer_options fields like opts->key_only which is only used by
>> format_trailers_from_commit() and not builtin/interpret-trailers.c.
>
> Yay.  It feels a bit disappointing to see the diffstat and learn
> that we are not deleting substantial number of lines.

Indeed ;)

>> ---
>>  builtin/interpret-trailers.c |   5 +-
>>  pretty.c                     |   2 +-
>>  ref-filter.c                 |   2 +-
>>  trailer.c                    | 105 +++++++++++++++++++++++++++++------
>>  trailer.h                    |  21 +++----
>>  5 files changed, 102 insertions(+), 33 deletions(-)
>
>> diff --git a/pretty.c b/pretty.c
>> index cf964b060cd..f0721a5214f 100644
>> --- a/pretty.c
>> +++ b/pretty.c
>> @@ -1759,7 +1759,7 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */
>>  				goto trailer_out;
>>  		}
>>  		if (*arg == ')') {
>> -			format_trailers_from_commit(sb, msg + c->subject_off, &opts);
>> +			format_trailers_from_commit(msg + c->subject_off, &opts, sb);
>
> I am curious (read: no objection---merely wondering if there is a
> guiding principle behind the choice of the new order) why this new
> parameter ordering.

Glen Choo told me a while ago [1] that we usually put "out parameters"
at the end, and somehow that stuck with me.

> I suspect it was originally written with a
> strbuf-centric worldview and having sb at the beginning may have
> made sense,

Having gotten more familiar with the strbuf.h functions since this patch
was originally written, I agree.

> but if we are moving them around, wouldn't it make more
> sense to have &opts as the first parameter, as this is primarily a
> "trailers" function?  Unsure until I read through to the end, but
> that is my gut reaction.

I simply (mechanically) moved "sb" from the beginning of the parameters
to the end, and didn't think much beyond that adjustment.

Your ordering seems fine to me. Noted for the next reroll, but I will
wait until you're done reviewing the other patches before going with the
new ordering (in case you change your mind).

>>  static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
>>  {
>>  	struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
>> @@ -984,6 +971,78 @@ static void unfold_value(struct strbuf *val)
>>  	strbuf_release(&out);
>>  }
>>  
>> +void format_trailers(struct list_head *head,
>> +		     const struct process_trailer_options *opts,
>> +		     struct strbuf *out)
>> +{
>> +	struct list_head *pos;
>> +	struct trailer_item *item;
>> +	int need_separator = 0;
>> +
>> +	list_for_each(pos, head) {
>> +		item = list_entry(pos, struct trailer_item, list);
>> +		if (item->token) {
>> +			char c;
>> + ...
>> +			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
>> +				if (opts->unfold)
>> +					unfold_value(&val);
>> +
>> +				if (opts->separator && need_separator)
>> +					strbuf_addbuf(out, opts->separator);
>> +				if (!opts->value_only)
>> +					strbuf_addbuf(out, &tok);
>> +				if (!opts->key_only && !opts->value_only) {
>> +					if (opts->key_value_separator)
>> +						strbuf_addbuf(out, opts->key_value_separator);
>> +					else {
>> +						c = last_non_space_char(tok.buf);
>> +						if (c) {
>> +							if (!strchr(separators, c))
>> +								strbuf_addf(out, "%c ", separators[0]);
>> +						}
>> +					}
>
> That's an overly deep nesting.  I wonder if a small file-scope
> helper function is in order?
>
> 	static add_separator(struct process_trailer_options *opts,
>         		     const char *token
> 			     struct strbuf *out)
> 	{
> 		if (opts->key_value_separator)
> 			strbuf_addbuf(out, opts->key_value_separator);
> 		else
> 			strbuf_addstr(out, ": ");
> 	}
>
> Or perhaps inside the context of the loop to go over the list of
> trailer items, one iteration of the list_for_each() loop can become
> one call to a small helper function format_one_trailer() and that
> may make the result easier to follow?

This is actually what I've already done (introducing helper functions to
make this more readable and reduce nesting), but in the larger series
(not in this patch series). I think in this patch I tried to avoid
introducing new functions in order to keep the original "shape" of the
existing refactored functions, including all of the nesting that they
originally had. I wanted to keep the original shape because I thought
that would make review simpler.

> In any case, I didn't see anything glaringly wrong so far in this
> series.  Let me keep reading.
>
> Thanks.

Thank you for reviewing!

[1] https://lore.kernel.org/git/kl6l5y5qa34v.fsf@chooglen-macbookpro.roam.corp.google.com/ 

Re: [PATCH 09/10] trailer: move arg handling to interpret-trailers.c

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> diff --git a/trailer.c b/trailer.c
> index e2d541372a3..0a86e0d5afa 100644
> --- a/trailer.c
> +++ b/trailer.c
> @@ -66,6 +66,11 @@ static LIST_HEAD(conf_head);
>  
>  static char *separators = ":";
>  
> +const char *default_separators(void)
> +{
> +	return separators;
> +}

This allows API users to peek into the current set of separator
bytes (either the default ":" or specified by the configuration
varaible "trailer.separators"), which is an improvement over
directly exposing the "separators" variable, but in a longer term,
do we need to have some "trailer context" object that holds this
and possibly other global variables like this?

I do not demand such further abstraction in this series, but I'd
prefer to see if we all have shared vision into the future.

Thanks.

Re: [PATCH 05/10] sequencer: use the trailer iterator

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

>> From: Linus Arver <linusa@google.com>
>> Also, teach the iterator about non-trailer lines, by adding a new field
>> called "raw" to hold both trailer and non-trailer lines. This is
>> necessary because a "trailer block" is a list of trailer lines of at
>> least 25% trailers (see 146245063e (trailer: allow non-trailers in
>> trailer block, 2016-10-21)), such that it may hold non-trailer lines.
>
> OK.  This would change behaviour, wouldn't it, in the sense that we
> used to yield a non-trailer line from the old iterator but the new
> one skips them?

I think it's the other way; the old iterator only iterated over trailer
lines, skipping over non-trailer lines (see the "not a real trailer"
deleted bit for trailer_iterator_advance()). The new one iterates over
all lines found in the trailer block, whether they are trailer or
non-trailer lines.

The function insert_records_from_trailers() from shortlog.c uses the new
iterator, but has to be careful because the new iterator goes over
non-trailer lines too. That's why it now does

    if (!iter.is_trailer)
            continue;

to do the skipping itself.

> Is that something we can demonstrate and protect in
> tests (presumably once we conclude these refactoring, we would be
> able to call into this machinery from unit-testing framework)?

Yup, that is exactly what I want to do once the dust around the trailer
API settles down after this series (and also the larger series). :)

Re: [PATCH 07/10] trailer: spread usage of "trailer_block" language

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "info" is a word with so low information contents.  At least "block"
> would hint that we are talking about a block of text that makes up
> the trailer attached to a single commit.

There is also "conf_info" and the whole "key"/"key alias" vs "token"
lingo which (if we are going to rename things for consistency) needs to
be updated to match documentation. In the larger series I've cleaned
this up as well.

Re: [PATCH 09/10] trailer: move arg handling to interpret-trailers.c

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> diff --git a/trailer.c b/trailer.c
>> index e2d541372a3..0a86e0d5afa 100644
>> --- a/trailer.c
>> +++ b/trailer.c
>> @@ -66,6 +66,11 @@ static LIST_HEAD(conf_head);
>>  
>>  static char *separators = ":";
>>  
>> +const char *default_separators(void)
>> +{
>> +	return separators;
>> +}
>
> This allows API users to peek into the current set of separator
> bytes (either the default ":" or specified by the configuration
> varaible "trailer.separators"), which is an improvement over
> directly exposing the "separators" variable, but in a longer term,
> do we need to have some "trailer context" object that holds this
> and possibly other global variables like this?

Yes, and I've implemented a "trailer_subsystem_conf" struct to hold all
configuration-related bits (and hardcoded defaults) relating to trailers
in one place, in the larger series.

> I do not demand such further abstraction in this series, but I'd
> prefer to see if we all have shared vision into the future.

Makes sense. Thanks.

Re: [PATCH 05/10] sequencer: use the trailer iterator

[Linus Arver]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> diff --git a/trailer.h b/trailer.h
> index 50f70556302..d50c9fd79b2 100644
> --- a/trailer.h
> +++ b/trailer.h
> @@ -127,6 +127,19 @@ struct trailer_iterator {
>  	struct strbuf key;
>  	struct strbuf val;
>  
> +	/*
> +	 * Raw line (e.g., "foo: bar baz") before being parsed as a trailer
> +	 * key/val pair. This field can contain non-trailer lines because it's
> +	 * valid for a trailer block to contain such lines (i.e., we only
> +	 * require 25% of the lines in a trailer block to be trailer lines).
> +	 */
> +	struct strbuf raw;

Originally I used a strbuf here for consistency with the other strbufs
used in the iterator for the key and val members. But now I've realized
that there's no need to make "raw" a strbuf at all, because iterator
users will never need to manipulate the string that this points to. Will
change to just "const char *" in the reroll.

Re: [PATCH 06/10] trailer: make trailer_info struct private

[Josh Steadmon]

On 2024.01.10 06:51, Linus Arver via GitGitGadget wrote:
> From: Linus Arver <linusa@google.com>
> 
> In 13211ae23f (trailer: separate public from internal portion of
> trailer_iterator, 2023-09-09) we moved trailer_info behind an anonymous
> struct to discourage use by trailer.h API users. However it still left
> open the possibility of external use of trailer_info itself. Now that
> there are no external users of trailer_info, we can make this struct
> private.
> 
> Make this struct private by putting its definition inside trailer.c.
> This has two benefits:
> 
> (1) it makes the surface area of the public facing interface (trailer.h)
>     smaller, and
> 
> (2) external API users are unable to peer inside this struct (because it
>     is only ever exposed as an opaque pointer).
> 
> This change exposes some deficiencies in the API, mainly with regard to
> information about the location of the trailer block that was parsed.
> Expose new API functions to access this information (needed by
> builtin/interpret-trailers.c).
> 
> The idea in this patch to hide implementation details behind an "opaque
> pointer" is also known as the "pimpl" (pointer to implementation) idiom
> in C++ and is a common pattern in that language (where, for example,
> abstract classes only have pointers to concrete classes).
> 
> However, the original inspiration to use this idiom does not come from
> C++, but instead the book "C Interfaces and Implementations: Techniques
> for Creating Reusable Software" [1]. This book recommends opaque
> pointers as a good design principle for designing C libraries, using the
> term "interface" as the functions defined in *.h (header) files and
> "implementation" as the corresponding *.c file which define the
> interfaces.
> 
> The book says this about opaque pointers:
> 
>     ... clients can manipulate such pointers freely, but they can’t
>     dereference them; that is, they can’t look at the innards of the
>     structure pointed to by them. Only the implementation has that
>     privilege. Opaque pointers hide representation details and help
>     catch errors.
> 
> In our case, "struct trailer_info" is now hidden from clients, and the
> ways in which this opaque pointer can be used is limited to the richness
> of the trailer.h file. In other words, trailer.h exclusively controls
> exactly how "trailer_info" pointers are to be used.
> 
> [1] Hanson, David R. "C Interfaces and Implementations: Techniques for
>     Creating Reusable Software". Addison Wesley, 1997. p. 22
> 
> Signed-off-by: Linus Arver <linusa@google.com>
> ---
>  builtin/interpret-trailers.c |  13 +--
>  trailer.c                    | 154 +++++++++++++++++++++++------------
>  trailer.h                    |  37 ++-------
>  3 files changed, 117 insertions(+), 87 deletions(-)

Looks like a pretty straightforward change. I think the only point worth
discussing is using the "pimpl" idiom. I think it's harder to see the
value because the trailer_info struct is fairly simple, but I can
definitely see this pattern being useful as we libify more complex parts
of Git where the struct internals have more complicated logic involved.
The pimpl pattern also seems like it will force us to think harder about
providing a useful interface, so I am in favor of using it here as
practice for future libification.

Re: [PATCH 06/10] trailer: make trailer_info struct private

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> The pimpl pattern also seems like it will force us to think harder about
> providing a useful interface, so I am in favor of using it here as
> practice for future libification.

Sounds good.

Re: [PATCH 08/10] trailer: prepare to move parse_trailers_from_command_line_args() to builtin

[Josh Steadmon]

On 2024.01.10 06:51, Linus Arver via GitGitGadget wrote:
> From: Linus Arver <linusa@google.com>
> 
> Expose more functions in the trailer.h API, in preparation for moving
> out
> 
>     parse_trailers_from_command_line_args()
> 
> to interpret-trailer.c, because the trailer API should not be concerned
> with command line arguments (as it has nothing to do with trailers
> themselves). The interpret-trailers builtin is the only user of the
> above function.
> 
> Signed-off-by: Linus Arver <linusa@google.com>
> ---
>  trailer.c | 66 +++++++++++++++++++++++++++----------------------------
>  trailer.h | 10 +++++++++
>  2 files changed, 42 insertions(+), 34 deletions(-)
> 
> diff --git a/trailer.c b/trailer.c
> index 360e76376b8..e2d541372a3 100644
> --- a/trailer.c
> +++ b/trailer.c
> @@ -33,7 +33,7 @@ struct trailer_block {
>  	size_t trailer_nr;
>  };
>  
> -struct conf_info {
> +struct trailer_conf {

Can you also add a note about this conf_info -> trailer_conf rename? I
agree that it's an improvement but I think it should be mentioned in the
commit message.

Re: [PATCH 00/10] Enrich Trailer API

[Josh Steadmon]

On 2024.01.10 06:51, Linus Arver via GitGitGadget wrote:
> This patch series is the first 10 patches of a much larger series I've been
> working. The main goal of this series is to enrich the API in trailer.h. The
> larger series brings a number of additional code simplifications and
> cleanups (exposing and fixing some bugs along the way), and builds on top of
> this series. The goal of the larger series is to make the trailer interface
> ready for unit testing. By "trailer API" I mean those functions exposed in
> trailer.h.

I agree with Junio's points, and I added a small nitpick about patch 8's
commit message. But apart from that, this all looks good to me and I'm
interested in seeing the followup series as well. Thanks!

Reviewed-by: Josh Steadmon <steadmon@google.com>

Re: [PATCH 08/10] trailer: prepare to move parse_trailers_from_command_line_args() to builtin

[Linus Arver]

Josh Steadmon <steadmon@google.com> writes:

> On 2024.01.10 06:51, Linus Arver via GitGitGadget wrote:
>> diff --git a/trailer.c b/trailer.c
>> index 360e76376b8..e2d541372a3 100644
>> --- a/trailer.c
>> +++ b/trailer.c
>> @@ -33,7 +33,7 @@ struct trailer_block {
>>  	size_t trailer_nr;
>>  };
>>  
>> -struct conf_info {
>> +struct trailer_conf {
>
> Can you also add a note about this conf_info -> trailer_conf rename? I
> agree that it's an improvement but I think it should be mentioned in the
> commit message.

Will do.

[PATCH v2 00/10] Enrich Trailer API

[Linus Arver via GitGitGadget]

This patch series is the first 10 patches of a larger cleanup/bugfix series
(henceforth "larger series") I've been working on. The main goal of this
series is to begin the process of "libifying" the trailer API. By "API" I
mean the interface exposed in trailer.h. The larger series brings a number
of additional cleanups (exposing and fixing some bugs along the way), and
builds on top of this series.

When the larger series is merged, we will be in a good state to additionally
pursue the following goals:

 1. "API reuse inside Git": make the API expressive enough to eliminate any
    need by other parts of Git to use the interpret-trailers builtin as a
    subprocess (instead they could just use the API directly);
 2. "API stability": add unit tests to codify the expected behavior of API
    functions; and
 3. "API documentation": create developer-focused documentation to explain
    how to use the API effectively, noting any API limitations or
    anti-patterns.

The reason why the larger series itself doesn't tackle these goals directly
is because I believe that API code should be thought from the ground up with
a libification-focused perspective. Some structs and functions exposed in
the API today should probably not be libified (read: kept in trailer.h) as
is. For example, the "trailer_iterator" struct has a "private" member and it
feels wrong to allow API users to peek inside here (and take at face value
our future API users' pinky promise that they won't depend on those private
internals not meant for public consumption).

One pattern we could use here to cleanly separate "what is the API"
(publicly exposed) and "what is the implementation" (private) is the
pointer-to-implementation ("pimpl") idiom. There may be other appropriate
patterns, but I've chosen this one because it's a simple, low-level concept
(put structs in foo.c instead of foo.h), which has far-reaching high-level
consequences (API users must rely exclusively on the API to make use of such
private structs, via opaque pointers). The pimpl idiom for C comes from the
book "C Interfaces and Implementations" (see patch "trailer: make
trailer_info struct private").

The idea of turning a public struct into a private one is a fundamental
question of libification because it forces us to reconsider all of the data
structures we have and how they're actually used by already existing users.
For the trailer API, those existing users are the "interpret-trailers"
builtin command, and anything else that includes the "trailer.h" header file
(e.g., sequencer.c). One advantage of this idiom is that even the compiler
understands it --- the compiler will loudly complain if you try to access
the innards of a private struct through an opaque pointer.

Another advantage of this idiom is that it helps to reduce the probability
of breaking changes in the API. Because a private struct's members are out
of view from our users (they only know about opaque pointers to the private
struct, not its members), we are free to modify the members of the struct at
any time, as much as we like, as long as we don't break the semantics of the
exposed API functions (which is why unit-testing these API functions will be
crucial long-term).

If this pimpl idiom turns out to be a mistake, undoing it is easy --- just
move the relevant struct definition from foo.c to the header file. So it's a
great way to try things out without digging ourselves into a pit of despair
that will be difficult to get out of.

With the libification-focused goals out of the way, let's turn to this patch
series in more detail.

Currently, we have "process_trailers()" in trailer.h which does many
different things (parse command-line arguments, create temporary files, etc)
that are independent of the concept of "trailers". Keeping this function as
an API function would make unit-testing it difficult. While there is no
technical reason why we couldn't write unit tests for the smaller functions
that are called within process_trailers(), doing so would involve testing
private ("static" in trailer.c) functions instead of API functions, which
defeats the goal of "API stability" mentioned earlier above.

As an alternative to how things are done in this patch series, we could keep
trailer.h intact and decide to unit-test the existing "trailer_info_get()"
function which does most of the trailer parsing work (and is used by
sequencer.c). However this function wouldn't be easy to test either, because
the resulting "trailer_info" struct merely contains the unparsed "trailers"
lines. So the unit test (if it wants to inspect the result of parsing these
lines) would have to invoke additional parsing functions itself. And at that
point it would no longer be a unit test in the traditional sense, because it
would be invoking multiple functions at once.

In summary this series breaks up "process_trailers()" into smaller pieces,
exposing many of the parts relevant to trailer-related processing in
trailer.h. This will force us to eventually introduce unit tests for these
API functions, but that is a good thing for API stability.

In the future after libification is "complete", users external to Git will
be able to use the same trailer processing API used by the
interpret-trailers builtin. For example, a web server may want to parse
trailers the same way that Git would parse them, without having to call
interpret-trailers as a subprocess. This use case was the original
motivation behind my work in this area.

Thanks to the aggressive refactoring in this series, I've been able to
identify and fix several bugs in our existing implementation. Those fixes
build on top of this series but were not included here, in order to keep
this series small. Below is a "shortlog" of those fixes I have locally:

 * "trailer: trailer replacement should not change its position" (If we
   found a trailer we'd like to replace, preserve its position relative to
   the other trailers found in the trailer block, instead of always moving
   it to the beginning or end of the entire trailer block.)
 * "interpret-trailers: preserve trailers coming from the input" (Sometimes,
   the parsed trailers from the input will be formatted differently
   depending on whether we provide --only-trailers or not. Make the trailers
   that were not modified and which are coming directly from the input get
   formatted the same way, regardless of this flag.)
 * "interpret-trailers: do not modify the input if NOP" (Refrain from
   subtracting or adding a newline around the patch divider "---" if we are
   not adding new trailers.)
 * "trailer formatter: split up format_trailer() monolith" (Fix a bug in
   git-log where we still printed a blank newline even if we didn't want to
   format anything.)
 * "interpret-trailers: fail if given unrecognized arguments" (E.g., for
   "--where", only accept recognized WHERE_* enum values. If we get
   something unrecognized, fail with an error instead of silently doing
   nothing. Ditto for "--if-exists" and "--if-missing".)


Notable changes in v2
=====================

 * Reorder function parameters to have trailer options at the beginning (and
   out parameters toward the end)
 * "sequencer: use the trailer iterator": prefer C string instead of strbuf
   for new "raw" field
 * Patch 1 (was Patch 2) also renames ensure_configured() to
   trailer_config_init() (forgot to rename this one previously)

Linus Arver (10):
  trailer: prepare to expose functions as part of API
  trailer: move interpret_trailers() to interpret-trailers.c
  trailer: unify trailer formatting machinery
  trailer: delete obsolete formatting functions
  sequencer: use the trailer iterator
  trailer: make trailer_info struct private
  trailer: spread usage of "trailer_block" language
  trailer: prepare to move parse_trailers_from_command_line_args() to
    builtin
  trailer: move arg handling to interpret-trailers.c
  trailer: delete obsolete argument handling code from API

 builtin/interpret-trailers.c | 169 ++++++++--
 builtin/shortlog.c           |   7 +-
 pretty.c                     |   2 +-
 ref-filter.c                 |   2 +-
 sequencer.c                  |  35 +--
 trailer.c                    | 576 ++++++++++++++++-------------------
 trailer.h                    | 105 ++++---
 7 files changed, 478 insertions(+), 418 deletions(-)


base-commit: a54a84b333adbecf7bc4483c0e36ed5878cac17b
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1632%2Flistx%2Ftrailer-api-refactor-part-1-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1632/listx/trailer-api-refactor-part-1-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/1632

Range-diff vs v1:

  2:  5f64718abfc !  1:  e2d3ed9b5b6 trailer: include "trailer" term in API functions
     @@ Metadata
      Author: Linus Arver <linusa@google.com>
      
       ## Commit message ##
     -    trailer: include "trailer" term in API functions
     +    trailer: prepare to expose functions as part of API
      
     -    These functions are exposed to clients and so they should include
     -    "trailer" in their names for easier identification, just like all the
     -    other functions already exposed by trailer.h.
     +    In the next patch, we will move "process_trailers" from trailer.c to
     +    builtin/interpret-trailers.c. That move will necessitate the growth of
     +    the trailer.h API, forcing us to expose some additional functions in
     +    trailer.h.
      
     +    Rename relevant functions so that they include the term "trailer" in
     +    their name, so that clients of the API will be able to easily identify
     +    them by their "trailer" moniker, just like all the other functions
     +    already exposed by trailer.h.
     +
     +    The the opportunity to start putting trailer processions options (opts)
     +    as the first parameter. This will be the pattern going forward in this
     +    series.
     +
     +    Helped-by: Junio C Hamano <gitster@pobox.com>
          Signed-off-by: Linus Arver <linusa@google.com>
      
       ## builtin/interpret-trailers.c ##
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
     - 	struct trailer_info info;
     - 	FILE *outfile = stdout;
     - 
     --	ensure_configured();
     -+	trailer_config_init();
     - 
     - 	read_input_file(&sb, file);
     - 
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
     - 		process_trailers_lists(&head, &arg_head);
     +@@ builtin/interpret-trailers.c: int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
     + 	if (argc) {
     + 		int i;
     + 		for (i = 0; i < argc; i++)
     +-			process_trailers(argv[i], &opts, &trailers);
     ++			interpret_trailers(&opts, &trailers, argv[i]);
     + 	} else {
     + 		if (opts.in_place)
     + 			die(_("no input file given for in-place editing"));
     +-		process_trailers(NULL, &opts, &trailers);
     ++		interpret_trailers(&opts, &trailers, NULL);
       	}
       
     --	print_all(outfile, &head, opts);
     -+	/* Print trailer block. */
     -+	format_trailers(outfile, &head, opts);
     - 
     --	free_all(&head);
     -+	free_trailers(&head);
     - 	trailer_info_release(&info);
     - 
     - 	/* Print the lines after the trailers as is */
     + 	new_trailers_clear(&trailers);
      
       ## trailer.c ##
      @@ trailer.c: static void print_tok_val(FILE *outfile, const char *tok, const char *val)
       		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
       }
       
     --void print_all(FILE *outfile, struct list_head *head,
     --	       const struct process_trailer_options *opts)
     -+void format_trailers(FILE *outfile, struct list_head *head,
     -+		     const struct process_trailer_options *opts)
     +-static void print_all(FILE *outfile, struct list_head *head,
     +-		      const struct process_trailer_options *opts)
     ++static void format_trailers(const struct process_trailer_options *opts,
     ++			    struct list_head *trailers, FILE *outfile)
       {
       	struct list_head *pos;
       	struct trailer_item *item;
     +-	list_for_each(pos, head) {
     ++	list_for_each(pos, trailers) {
     + 		item = list_entry(pos, struct trailer_item, list);
     + 		if ((!opts->trim_empty || strlen(item->value) > 0) &&
     + 		    (!opts->only_trailers || item->token))
      @@ trailer.c: static int git_trailer_config(const char *conf_key, const char *value,
       	return 0;
       }
       
     --void ensure_configured(void)
     -+void trailer_config_init(void)
     +-static void ensure_configured(void)
     ++static void trailer_config_init(void)
       {
       	if (configured)
       		return;
     -@@ trailer.c: void parse_trailers(struct trailer_info *info,
     +@@ trailer.c: static void parse_trailers(struct trailer_info *info,
       	}
       }
       
     --void free_all(struct list_head *head)
     -+void free_trailers(struct list_head *head)
     +-static void free_all(struct list_head *head)
     ++static void free_trailers(struct list_head *trailers)
       {
       	struct list_head *pos, *p;
     - 	list_for_each_safe(pos, p, head) {
     +-	list_for_each_safe(pos, p, head) {
     ++	list_for_each_safe(pos, p, trailers) {
     + 		list_del(pos);
     + 		free_trailer_item(list_entry(pos, struct trailer_item, list));
     + 	}
     +@@ trailer.c: static FILE *create_in_place_tempfile(const char *file)
     + 	return outfile;
     + }
     + 
     +-void process_trailers(const char *file,
     +-		      const struct process_trailer_options *opts,
     +-		      struct list_head *new_trailer_head)
     ++void interpret_trailers(const struct process_trailer_options *opts,
     ++			struct list_head *new_trailer_head,
     ++			const char *file)
     + {
     + 	LIST_HEAD(head);
     + 	struct strbuf sb = STRBUF_INIT;
     + 	struct trailer_info info;
     + 	FILE *outfile = stdout;
     + 
     +-	ensure_configured();
     ++	trailer_config_init();
     + 
     + 	read_input_file(&sb, file);
     + 
     +@@ trailer.c: void process_trailers(const char *file,
     + 		process_trailers_lists(&head, &arg_head);
     + 	}
     + 
     +-	print_all(outfile, &head, opts);
     ++	format_trailers(opts, &head, outfile);
     + 
     +-	free_all(&head);
     ++	free_trailers(&head);
     + 	trailer_info_release(&info);
     + 
     + 	/* Print the lines after the trailers as is */
      @@ trailer.c: void trailer_info_get(struct trailer_info *info, const char *str,
       	size_t nr = 0, alloc = 0;
       	char **last = NULL;
     @@ trailer.c: void trailer_info_get(struct trailer_info *info, const char *str,
       	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
      
       ## trailer.h ##
     -@@ trailer.h: void trailer_info_get(struct trailer_info *info, const char *str,
     - 
     - void trailer_info_release(struct trailer_info *info);
     - 
     --void ensure_configured(void);
     --void print_all(FILE *outfile, struct list_head *head,
     --	       const struct process_trailer_options *opts);
     --void free_all(struct list_head *head);
     -+void trailer_config_init(void);
     -+void free_trailers(struct list_head *trailers);
     - 
     -+void format_trailers(FILE *outfile, struct list_head *head,
     -+		     const struct process_trailer_options *opts);
     - /*
     -  * Format the trailers from the commit msg "msg" into the strbuf "out".
     -  * Note two caveats about "opts":
     -@@ trailer.h: void free_all(struct list_head *head);
     -  *   - this is primarily a helper for pretty.c, and not
     -  *     all of the flags are supported.
     -  *
     -- *   - this differs from process_trailers slightly in that we always format
     -+ *   - this differs from format_trailers slightly in that we always format
     -  *     only the trailer block itself, even if the "only_trailers" option is not
     -  *     set.
     -  */
     +@@ trailer.h: struct process_trailer_options {
     + 
     + #define PROCESS_TRAILER_OPTIONS_INIT {0}
     + 
     +-void process_trailers(const char *file,
     +-		      const struct process_trailer_options *opts,
     +-		      struct list_head *new_trailer_head);
     ++void interpret_trailers(const struct process_trailer_options *opts,
     ++			struct list_head *new_trailer_head,
     ++			const char *file);
     + 
     + void trailer_info_get(struct trailer_info *info, const char *str,
     + 		      const struct process_trailer_options *opts);
  1:  2dc3279b37f !  2:  eaca39fd7ea trailer: move process_trailers() to interpret-trailers.c
     @@ Metadata
      Author: Linus Arver <linusa@google.com>
      
       ## Commit message ##
     -    trailer: move process_trailers() to interpret-trailers.c
     +    trailer: move interpret_trailers() to interpret-trailers.c
      
          The interpret-trailers.c builtin is the only place we need to call
     -    process_trailers(). As it stands, process_trailers() is inherently tied
     -    to how the builtin behaves, so move its definition there.
     +    interpret_trailers(), so move its definition there.
      
          Delete the corresponding declaration from trailer.h, which then forces
          us to expose the working innards of that function. This enriches
     -    trailer.h to include a more granular API, which can then be unit-tested
     -    in the future (because process_trailers() by itself does too many things
     -    to be able to be easily unit-tested).
     +    trailer.h with a more granular API, which can then be unit-tested in the
     +    future (because interpret_trailers() by itself does too many things to
     +    be able to be easily unit-tested).
      
          Take this opportunity to demote some file-handling functions out of the
          trailer API implementation, as these have nothing to do with trailers.
      
     -    While we're at it, rename process_trailers() to interpret_trailers() in
     -    the builtin for consistency with the existing cmd_interpret_trailers(),
     -    which wraps around this function.
     -
          Signed-off-by: Linus Arver <linusa@google.com>
      
       ## builtin/interpret-trailers.c ##
     @@ builtin/interpret-trailers.c: static int parse_opt_parse(const struct option *op
      +	}
      +}
      +
     -+static void interpret_trailers(const char *file,
     -+			       const struct process_trailer_options *opts,
     -+			       struct list_head *new_trailer_head)
     ++static void interpret_trailers(const struct process_trailer_options *opts,
     ++			       struct list_head *new_trailer_head,
     ++			       const char *file)
      +{
      +	LIST_HEAD(head);
      +	struct strbuf sb = STRBUF_INIT;
      +	struct trailer_info info;
      +	FILE *outfile = stdout;
      +
     -+	ensure_configured();
     ++	trailer_config_init();
      +
      +	read_input_file(&sb, file);
      +
      +	if (opts->in_place)
      +		outfile = create_in_place_tempfile(file);
      +
     -+	parse_trailers(&info, sb.buf, &head, opts);
     ++	parse_trailers(opts, &info, sb.buf, &head);
      +
      +	/* Print the lines before the trailers */
      +	if (!opts->only_trailers)
     @@ builtin/interpret-trailers.c: static int parse_opt_parse(const struct option *op
      +		process_trailers_lists(&head, &arg_head);
      +	}
      +
     -+	print_all(outfile, &head, opts);
     ++	format_trailers(opts, &head, outfile);
      +
     -+	free_all(&head);
     ++	free_trailers(&head);
      +	trailer_info_release(&info);
      +
      +	/* Print the lines after the trailers as is */
     @@ builtin/interpret-trailers.c: static int parse_opt_parse(const struct option *op
       int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
       {
       	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
     -@@ builtin/interpret-trailers.c: int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
     - 	if (argc) {
     - 		int i;
     - 		for (i = 0; i < argc; i++)
     --			process_trailers(argv[i], &opts, &trailers);
     -+			interpret_trailers(argv[i], &opts, &trailers);
     - 	} else {
     - 		if (opts.in_place)
     - 			die(_("no input file given for in-place editing"));
     --		process_trailers(NULL, &opts, &trailers);
     -+		interpret_trailers(NULL, &opts, &trailers);
     - 	}
     - 
     - 	new_trailers_clear(&trailers);
      
       ## trailer.c ##
      @@
     @@ trailer.c: static void print_tok_val(FILE *outfile, const char *tok, const char
       		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
       }
       
     --static void print_all(FILE *outfile, struct list_head *head,
     --		      const struct process_trailer_options *opts)
     -+void print_all(FILE *outfile, struct list_head *head,
     -+	       const struct process_trailer_options *opts)
     +-static void format_trailers(const struct process_trailer_options *opts,
     +-			    struct list_head *trailers, FILE *outfile)
     ++void format_trailers(const struct process_trailer_options *opts,
     ++		     struct list_head *trailers, FILE *outfile)
       {
       	struct list_head *pos;
       	struct trailer_item *item;
     @@ trailer.c: static int git_trailer_config(const char *conf_key, const char *value
       	return 0;
       }
       
     --static void ensure_configured(void)
     -+void ensure_configured(void)
     +-static void trailer_config_init(void)
     ++void trailer_config_init(void)
       {
       	if (configured)
       		return;
     @@ trailer.c: static void unfold_value(struct strbuf *val)
      -			     const char *str,
      -			     struct list_head *head,
      -			     const struct process_trailer_options *opts)
     -+void parse_trailers(struct trailer_info *info,
     ++void parse_trailers(const struct process_trailer_options *opts,
     ++		    struct trailer_info *info,
      +		    const char *str,
     -+		    struct list_head *head,
     -+		    const struct process_trailer_options *opts)
     ++		    struct list_head *head)
       {
       	struct strbuf tok = STRBUF_INIT;
       	struct strbuf val = STRBUF_INIT;
     @@ trailer.c: static void parse_trailers(struct trailer_info *info,
       	}
       }
       
     --static void free_all(struct list_head *head)
     -+void free_all(struct list_head *head)
     +-static void free_trailers(struct list_head *trailers)
     ++void free_trailers(struct list_head *trailers)
       {
       	struct list_head *pos, *p;
     - 	list_for_each_safe(pos, p, head) {
     -@@ trailer.c: static void free_all(struct list_head *head)
     + 	list_for_each_safe(pos, p, trailers) {
     +@@ trailer.c: static void free_trailers(struct list_head *trailers)
       	}
       }
       
     @@ trailer.c: static void free_all(struct list_head *head)
      -	return outfile;
      -}
      -
     --void process_trailers(const char *file,
     --		      const struct process_trailer_options *opts,
     --		      struct list_head *new_trailer_head)
     +-void interpret_trailers(const struct process_trailer_options *opts,
     +-			struct list_head *new_trailer_head,
     +-			const char *file)
      -{
      -	LIST_HEAD(head);
      -	struct strbuf sb = STRBUF_INIT;
      -	struct trailer_info info;
      -	FILE *outfile = stdout;
      -
     --	ensure_configured();
     +-	trailer_config_init();
      -
      -	read_input_file(&sb, file);
      -
     @@ trailer.c: static void free_all(struct list_head *head)
      -		process_trailers_lists(&head, &arg_head);
      -	}
      -
     --	print_all(outfile, &head, opts);
     +-	format_trailers(opts, &head, outfile);
      -
     --	free_all(&head);
     +-	free_trailers(&head);
      -	trailer_info_release(&info);
      -
      -	/* Print the lines after the trailers as is */
     @@ trailer.h: struct process_trailer_options {
       
       #define PROCESS_TRAILER_OPTIONS_INIT {0}
       
     --void process_trailers(const char *file,
     --		      const struct process_trailer_options *opts,
     --		      struct list_head *new_trailer_head);
     +-void interpret_trailers(const struct process_trailer_options *opts,
     +-			struct list_head *new_trailer_head,
     +-			const char *file);
      +void parse_trailers_from_config(struct list_head *config_head);
      +
      +void parse_trailers_from_command_line_args(struct list_head *arg_head,
     @@ trailer.h: struct process_trailer_options {
      +void process_trailers_lists(struct list_head *head,
      +			    struct list_head *arg_head);
      +
     -+void parse_trailers(struct trailer_info *info,
     ++void parse_trailers(const struct process_trailer_options *opts,
     ++		    struct trailer_info *info,
      +		    const char *str,
     -+		    struct list_head *head,
     -+		    const struct process_trailer_options *opts);
     ++		    struct list_head *head);
       
       void trailer_info_get(struct trailer_info *info, const char *str,
       		      const struct process_trailer_options *opts);
       
       void trailer_info_release(struct trailer_info *info);
       
     -+void ensure_configured(void);
     -+void print_all(FILE *outfile, struct list_head *head,
     -+	       const struct process_trailer_options *opts);
     -+void free_all(struct list_head *head);
     ++void trailer_config_init(void);
     ++void format_trailers(const struct process_trailer_options *opts,
     ++		     struct list_head *trailers, FILE *outfile);
     ++void free_trailers(struct list_head *trailers);
      +
       /*
        * Format the trailers from the commit msg "msg" into the strbuf "out".
  3:  d3326021fb6 !  3:  9b7747d550e trailer: unify trailer formatting machinery
     @@ Commit message
          format_trailers() to process trailers with the additional
          process_trailer_options fields like opts->key_only which is only used by
          format_trailers_from_commit() and not builtin/interpret-trailers.c.
     +    While we're at it, reorder parameters to put the trailer processing
     +    options first, and the out parameter (strbuf we write into) at the end.
      
     -    This will allow us to delete the format_trailer_info() and
     +    This unification will allow us to delete the format_trailer_info() and
          print_tok_val() functions in the next patch. They are not deleted here
          in order to keep the diff small.
      
     +    Helped-by: Junio C Hamano <gitster@pobox.com>
          Signed-off-by: Linus Arver <linusa@google.com>
      
       ## builtin/interpret-trailers.c ##
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
       {
       	LIST_HEAD(head);
       	struct strbuf sb = STRBUF_INIT;
     @@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
       	struct trailer_info info;
       	FILE *outfile = stdout;
       
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
     + 		process_trailers_lists(&head, &arg_head);
       	}
       
     - 	/* Print trailer block. */
     --	format_trailers(outfile, &head, opts);
     -+	format_trailers(&head, opts, &trailer_block);
     +-	format_trailers(opts, &head, outfile);
     ++	/* Print trailer block. */
     ++	format_trailers(opts, &head, &trailer_block);
      +	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
      +	strbuf_release(&trailer_block);
       
     @@ pretty.c: static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */
       		}
       		if (*arg == ')') {
      -			format_trailers_from_commit(sb, msg + c->subject_off, &opts);
     -+			format_trailers_from_commit(msg + c->subject_off, &opts, sb);
     ++			format_trailers_from_commit(&opts, msg + c->subject_off, sb);
       			ret = arg - placeholder + 1;
       		}
       	trailer_out:
     @@ ref-filter.c: static void grab_sub_body_contents(struct atom_value *val, int der
       
       			/* Format the trailer info according to the trailer_opts given */
      -			format_trailers_from_commit(&s, subpos, &atom->u.contents.trailer_opts);
     -+			format_trailers_from_commit(subpos, &atom->u.contents.trailer_opts, &s);
     ++			format_trailers_from_commit(&atom->u.contents.trailer_opts, subpos, &s);
       
       			v->s = strbuf_detach(&s, NULL);
       		} else if (atom->u.contents.option == C_BARE)
     @@ trailer.c: static void print_tok_val(FILE *outfile, const char *tok, const char
       		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
       }
       
     --void format_trailers(FILE *outfile, struct list_head *head,
     --		     const struct process_trailer_options *opts)
     +-void format_trailers(const struct process_trailer_options *opts,
     +-		     struct list_head *trailers, FILE *outfile)
      -{
      -	struct list_head *pos;
      -	struct trailer_item *item;
     --	list_for_each(pos, head) {
     +-	list_for_each(pos, trailers) {
      -		item = list_entry(pos, struct trailer_item, list);
      -		if ((!opts->trim_empty || strlen(item->value) > 0) &&
      -		    (!opts->only_trailers || item->token))
     @@ trailer.c: static void unfold_value(struct strbuf *val)
       	strbuf_release(&out);
       }
       
     -+void format_trailers(struct list_head *head,
     -+		     const struct process_trailer_options *opts,
     ++void format_trailers(const struct process_trailer_options *opts,
     ++		     struct list_head *trailers,
      +		     struct strbuf *out)
      +{
      +	struct list_head *pos;
      +	struct trailer_item *item;
      +	int need_separator = 0;
      +
     -+	list_for_each(pos, head) {
     ++	list_for_each(pos, trailers) {
      +		item = list_entry(pos, struct trailer_item, list);
      +		if (item->token) {
      +			char c;
     @@ trailer.c: static void format_trailer_info(struct strbuf *out,
       
      -void format_trailers_from_commit(struct strbuf *out, const char *msg,
      -				 const struct process_trailer_options *opts)
     -+void format_trailers_from_commit(const char *msg,
     -+				 const struct process_trailer_options *opts,
     ++void format_trailers_from_commit(const struct process_trailer_options *opts,
     ++				 const char *msg,
      +				 struct strbuf *out)
       {
      +	LIST_HEAD(head);
     @@ trailer.c: static void format_trailer_info(struct strbuf *out,
       
      -	trailer_info_get(&info, msg, opts);
      -	format_trailer_info(out, &info, msg, opts);
     -+	parse_trailers(&info, msg, &head, opts);
     ++	parse_trailers(opts, &info, msg, &head);
      +
      +	/* If we want the whole block untouched, we can take the fast path. */
      +	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
     @@ trailer.c: static void format_trailer_info(struct strbuf *out,
      +		strbuf_add(out, msg + info.trailer_block_start,
      +			   info.trailer_block_end - info.trailer_block_start);
      +	} else
     -+		format_trailers(&head, opts, out);
     ++		format_trailers(opts, &head, out);
      +
      +	free_trailers(&head);
       	trailer_info_release(&info);
     @@ trailer.c: static void format_trailer_info(struct strbuf *out,
      
       ## trailer.h ##
      @@ trailer.h: void trailer_info_release(struct trailer_info *info);
     + 
       void trailer_config_init(void);
     + void format_trailers(const struct process_trailer_options *opts,
     +-		     struct list_head *trailers, FILE *outfile);
     ++		     struct list_head *trailers,
     ++		     struct strbuf *out);
       void free_trailers(struct list_head *trailers);
       
     --void format_trailers(FILE *outfile, struct list_head *head,
     --		     const struct process_trailer_options *opts);
     -+void format_trailers(struct list_head *head,
     -+		     const struct process_trailer_options *opts,
     -+		     struct strbuf *out);
       /*
      - * Format the trailers from the commit msg "msg" into the strbuf "out".
      - * Note two caveats about "opts":
     @@ trailer.h: void trailer_info_release(struct trailer_info *info);
      - *   - this is primarily a helper for pretty.c, and not
      - *     all of the flags are supported.
      - *
     -- *   - this differs from format_trailers slightly in that we always format
     +- *   - this differs from process_trailers slightly in that we always format
      - *     only the trailer block itself, even if the "only_trailers" option is not
      - *     set.
      + * Convenience function to format the trailers from the commit msg "msg" into
     @@ trailer.h: void trailer_info_release(struct trailer_info *info);
        */
      -void format_trailers_from_commit(struct strbuf *out, const char *msg,
      -				 const struct process_trailer_options *opts);
     -+void format_trailers_from_commit(const char *msg,
     -+				 const struct process_trailer_options *opts,
     ++void format_trailers_from_commit(const struct process_trailer_options *opts,
     ++				 const char *msg,
      +				 struct strbuf *out);
       
       /*
  4:  8d864614757 !  4:  f1171f5202f trailer: delete obsolete formatting functions
     @@ trailer.c: void trailer_info_release(struct trailer_info *info)
      -
      -}
      -
     - void format_trailers_from_commit(const char *msg,
     - 				 const struct process_trailer_options *opts,
     + void format_trailers_from_commit(const struct process_trailer_options *opts,
     + 				 const char *msg,
       				 struct strbuf *out)
  5:  fd4a9d54d95 !  5:  5ba842b5005 sequencer: use the trailer iterator
     @@ sequencer.c: static const char *get_todo_path(const struct replay_opts *opts)
      +		i++;
      +		if (sob &&
      +		    iter.is_trailer &&
     -+		    !strncmp(iter.raw.buf, sob->buf, sob->len)) {
     ++		    !strncmp(iter.raw, sob->buf, sob->len)) {
      +			found_sob = i;
      +		}
      +	}
     @@ sequencer.c: static const char *get_todo_path(const struct replay_opts *opts)
      
       ## trailer.c ##
      @@ trailer.c: void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
     - 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
     - 	strbuf_init(&iter->key, 0);
     - 	strbuf_init(&iter->val, 0);
     -+	strbuf_init(&iter->raw, 0);
     - 	opts.no_divider = 1;
     - 	trailer_info_get(&iter->internal.info, msg, &opts);
     - 	iter->internal.cur = 0;
     -@@ trailer.c: void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
       
       int trailer_iterator_advance(struct trailer_iterator *iter)
       {
     @@ trailer.c: void trailer_iterator_init(struct trailer_iterator *iter, const char
      +		separator_pos = find_separator(line, separators);
      +		iter->is_trailer = (separator_pos > 0);
      +
     -+		strbuf_reset(&iter->raw);
     -+		strbuf_addstr(&iter->raw, line);
     ++		iter->raw = line;
       		strbuf_reset(&iter->key);
       		strbuf_reset(&iter->val);
       		parse_trailer(&iter->key, &iter->val, NULL,
     @@ trailer.c: void trailer_iterator_init(struct trailer_iterator *iter, const char
       		unfold_value(&iter->val);
       		return 1;
       	}
     -@@ trailer.c: void trailer_iterator_release(struct trailer_iterator *iter)
     - 	trailer_info_release(&iter->internal.info);
     - 	strbuf_release(&iter->val);
     - 	strbuf_release(&iter->key);
     -+	strbuf_release(&iter->raw);
     - }
      
       ## trailer.h ##
      @@ trailer.h: struct trailer_iterator {
     @@ trailer.h: struct trailer_iterator {
       
      +	/*
      +	 * Raw line (e.g., "foo: bar baz") before being parsed as a trailer
     -+	 * key/val pair. This field can contain non-trailer lines because it's
     -+	 * valid for a trailer block to contain such lines (i.e., we only
     -+	 * require 25% of the lines in a trailer block to be trailer lines).
     ++	 * key/val pair as part of a trailer block. A trailer block can be
     ++	 * either 100% trailer lines, or mixed in with non-trailer lines (in
     ++	 * which case at least 25% must be trailer lines).
      +	 */
     -+	struct strbuf raw;
     ++	const char *raw;
      +
      +	/*
     -+	 * 1 if the raw line was parsed as a separate key/val pair.
     ++	 * 1 if the raw line was parsed as a trailer line (key/val pair).
      +	 */
      +	int is_trailer;
      +
  6:  0cbe96421c7 !  6:  f0ac2f6c4b9 trailer: make trailer_info struct private
     @@ Commit message
          Make this struct private by putting its definition inside trailer.c.
          This has two benefits:
      
     -    (1) it makes the surface area of the public facing interface (trailer.h)
     -        smaller, and
     +      (1) it makes the surface area of the public facing
     +          interface (trailer.h) smaller, and
      
     -    (2) external API users are unable to peer inside this struct (because it
     -        is only ever exposed as an opaque pointer).
     +      (2) external API users are unable to peer inside this struct (because
     +          it is only ever exposed as an opaque pointer).
      
          This change exposes some deficiencies in the API, mainly with regard to
          information about the location of the trailer block that was parsed.
     @@ Commit message
          Signed-off-by: Linus Arver <linusa@google.com>
      
       ## builtin/interpret-trailers.c ##
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
       	LIST_HEAD(head);
       	struct strbuf sb = STRBUF_INIT;
       	struct strbuf trailer_block = STRBUF_INIT;
     @@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
       	FILE *outfile = stdout;
       
       	trailer_config_init();
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
       	if (opts->in_place)
       		outfile = create_in_place_tempfile(file);
       
     --	parse_trailers(&info, sb.buf, &head, opts);
     -+	info = parse_trailers(sb.buf, &head, opts);
     +-	parse_trailers(opts, &info, sb.buf, &head);
     ++	info = parse_trailers(opts, sb.buf, &head);
       
       	/* Print the lines before the trailers */
       	if (!opts->only_trailers)
     @@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
       		fprintf(outfile, "\n");
       
       
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
       	strbuf_release(&trailer_block);
       
       	free_trailers(&head);
     @@ trailer.c
       struct conf_info {
       	char *name;
       	char *key;
     -@@ trailer.c: void format_trailers(struct list_head *head,
     +@@ trailer.c: void format_trailers(const struct process_trailer_options *opts,
       	}
       }
       
     @@ trailer.c: void format_trailers(struct list_head *head,
      +	return info;
      +}
      +
     -+static struct trailer_info *trailer_info_get(const char *str,
     -+					     const struct process_trailer_options *opts)
     ++static struct trailer_info *trailer_info_get(const struct process_trailer_options *opts,
     ++					     const char *str)
      +{
      +	struct trailer_info *info = trailer_info_new();
      +	size_t end_of_log_message = 0, trailer_block_start = 0;
     @@ trailer.c: void format_trailers(struct list_head *head,
        * Parse trailers in "str", populating the trailer info and "head"
        * linked list structure.
        */
     --void parse_trailers(struct trailer_info *info,
     +-void parse_trailers(const struct process_trailer_options *opts,
     +-		    struct trailer_info *info,
      -		    const char *str,
     --		    struct list_head *head,
     --		    const struct process_trailer_options *opts)
     -+struct trailer_info *parse_trailers(const char *str,
     -+				    struct list_head *head,
     -+				    const struct process_trailer_options *opts)
     +-		    struct list_head *head)
     ++struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
     ++				    const char *str,
     ++				    struct list_head *head)
       {
      +	struct trailer_info *info;
       	struct strbuf tok = STRBUF_INIT;
     @@ trailer.c: void format_trailers(struct list_head *head,
       	size_t i;
       
      -	trailer_info_get(info, str, opts);
     -+	info = trailer_info_get(str, opts);
     ++	info = trailer_info_get(opts, str);
       
       	for (i = 0; i < info->trailer_nr; i++) {
       		int separator_pos;
     -@@ trailer.c: void parse_trailers(struct trailer_info *info,
     +@@ trailer.c: void parse_trailers(const struct process_trailer_options *opts,
       					 strbuf_detach(&val, NULL));
       		}
       	}
     @@ trailer.c: void parse_trailers(struct trailer_info *info,
      +	return info;
       }
       
     - void free_trailers(struct list_head *head)
     -@@ trailer.c: void free_trailers(struct list_head *head)
     + void free_trailers(struct list_head *trailers)
     +@@ trailer.c: void free_trailers(struct list_head *trailers)
       	}
       }
       
     @@ trailer.c: void trailer_info_release(struct trailer_info *info)
      +	free(info);
       }
       
     - void format_trailers_from_commit(const char *msg,
     -@@ trailer.c: void format_trailers_from_commit(const char *msg,
     + void format_trailers_from_commit(const struct process_trailer_options *opts,
     +@@ trailer.c: void format_trailers_from_commit(const struct process_trailer_options *opts,
       				 struct strbuf *out)
       {
       	LIST_HEAD(head);
      -	struct trailer_info info;
      -
     --	parse_trailers(&info, msg, &head, opts);
     -+	struct trailer_info *info = parse_trailers(msg, &head, opts);
     +-	parse_trailers(opts, &info, msg, &head);
     ++	struct trailer_info *info = parse_trailers(opts, msg, &head);
       
       	/* If we want the whole block untouched, we can take the fast path. */
       	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
     @@ trailer.c: void format_trailers_from_commit(const char *msg,
      +		strbuf_add(out, msg + info->trailer_block_start,
      +			   info->trailer_block_end - info->trailer_block_start);
       	} else
     - 		format_trailers(&head, opts, out);
     + 		format_trailers(opts, &head, out);
       
       	free_trailers(&head);
      -	trailer_info_release(&info);
     @@ trailer.c: void format_trailers_from_commit(const char *msg,
      +	struct trailer_info *internal = trailer_info_new();
       	strbuf_init(&iter->key, 0);
       	strbuf_init(&iter->val, 0);
     - 	strbuf_init(&iter->raw, 0);
       	opts.no_divider = 1;
      -	trailer_info_get(&iter->internal.info, msg, &opts);
      +	iter->internal.info = internal;
     -+	iter->internal.info = trailer_info_get(msg, &opts);
     ++	iter->internal.info = trailer_info_get(&opts, msg);
       	iter->internal.cur = 0;
       }
       
     @@ trailer.c: int trailer_iterator_advance(struct trailer_iterator *iter)
      +	trailer_info_release(iter->internal.info);
       	strbuf_release(&iter->val);
       	strbuf_release(&iter->key);
     - 	strbuf_release(&iter->raw);
     + }
      
       ## trailer.h ##
      @@
     @@ trailer.h: void parse_trailers_from_command_line_args(struct list_head *arg_head
       void process_trailers_lists(struct list_head *head,
       			    struct list_head *arg_head);
       
     --void parse_trailers(struct trailer_info *info,
     +-void parse_trailers(const struct process_trailer_options *opts,
     +-		    struct trailer_info *info,
      -		    const char *str,
     --		    struct list_head *head,
     --		    const struct process_trailer_options *opts);
     -+struct trailer_info *parse_trailers(const char *str,
     -+				    struct list_head *head,
     -+				    const struct process_trailer_options *opts);
     +-		    struct list_head *head);
     ++struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
     ++				    const char *str,
     ++				    struct list_head *head);
       
      -void trailer_info_get(struct trailer_info *info, const char *str,
      -		      const struct process_trailer_options *opts);
  7:  9183990583f !  7:  291aa83af55 trailer: spread usage of "trailer_block" language
     @@ Commit message
          Signed-off-by: Linus Arver <linusa@google.com>
      
       ## builtin/interpret-trailers.c ##
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
       {
       	LIST_HEAD(head);
       	struct strbuf sb = STRBUF_INIT;
     @@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
       	FILE *outfile = stdout;
       
       	trailer_config_init();
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
       	if (opts->in_place)
       		outfile = create_in_place_tempfile(file);
       
     --	info = parse_trailers(sb.buf, &head, opts);
     -+	trailer_block = parse_trailers(sb.buf, opts, &head);
     +-	info = parse_trailers(opts, sb.buf, &head);
     ++	trailer_block = parse_trailers(opts, sb.buf, &head);
       
      -	/* Print the lines before the trailers */
      +	/* Print the lines before the trailer block */
     @@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
       		fprintf(outfile, "\n");
       
       
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
       	}
       
       	/* Print trailer block. */
     --	format_trailers(&head, opts, &trailer_block);
     +-	format_trailers(opts, &head, &trailer_block);
      -	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
      -	strbuf_release(&trailer_block);
     -+	format_trailers(&head, opts, &tb);
     ++	format_trailers(opts, &head, &tb);
      +	fwrite(tb.buf, 1, tb.len, outfile);
      +	strbuf_release(&tb);
       
     @@ trailer.c
       
       	/*
       	 * Array of trailers found.
     -@@ trailer.c: void format_trailers(struct list_head *head,
     +@@ trailer.c: void format_trailers(const struct process_trailer_options *opts,
       	}
       }
       
     @@ trailer.c: void format_trailers(struct list_head *head,
      +	return trailer_block;
       }
       
     --static struct trailer_info *trailer_info_get(const char *str,
     --					     const struct process_trailer_options *opts)
     -+static struct trailer_block *trailer_block_get(const char *str,
     -+					       const struct process_trailer_options *opts)
     +-static struct trailer_info *trailer_info_get(const struct process_trailer_options *opts,
     +-					     const char *str)
     ++static struct trailer_block *trailer_block_get(const struct process_trailer_options *opts,
     ++					       const char *str)
       {
      -	struct trailer_info *info = trailer_info_new();
      +	struct trailer_block *trailer_block = trailer_block_new();
       	size_t end_of_log_message = 0, trailer_block_start = 0;
       	struct strbuf **trailer_lines, **ptr;
       	char **trailer_strings = NULL;
     -@@ trailer.c: static struct trailer_info *trailer_info_get(const char *str,
     +@@ trailer.c: static struct trailer_info *trailer_info_get(const struct process_trailer_option
       	}
       	strbuf_list_free(trailer_lines);
       
     @@ trailer.c: static struct trailer_info *trailer_info_get(const char *str,
      + * Parse trailers in "str", populating the trailer_block info and "head" linked
      + * list structure.
        */
     --struct trailer_info *parse_trailers(const char *str,
     --				    struct list_head *head,
     --				    const struct process_trailer_options *opts)
     -+struct trailer_block *parse_trailers(const char *str,
     -+				     const struct process_trailer_options *opts,
     +-struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
     +-				    const char *str,
     +-				    struct list_head *head)
     ++struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
     ++				     const char *str,
      +				     struct list_head *head)
       {
      -	struct trailer_info *info;
     @@ trailer.c: static struct trailer_info *trailer_info_get(const char *str,
       	struct strbuf val = STRBUF_INIT;
       	size_t i;
       
     --	info = trailer_info_get(str, opts);
     -+	trailer_block = trailer_block_get(str, opts);
     +-	info = trailer_info_get(opts, str);
     ++	trailer_block = trailer_block_get(opts, str);
       
      -	for (i = 0; i < info->trailer_nr; i++) {
      +	for (i = 0; i < trailer_block->trailer_nr; i++) {
     @@ trailer.c: static struct trailer_info *trailer_info_get(const char *str,
       		if (trailer[0] == comment_line_char)
       			continue;
       		separator_pos = find_separator(trailer, separators);
     -@@ trailer.c: struct trailer_info *parse_trailers(const char *str,
     +@@ trailer.c: struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
       		}
       	}
       
     @@ trailer.c: struct trailer_info *parse_trailers(const char *str,
      +	return trailer_block;
       }
       
     - void free_trailers(struct list_head *head)
     -@@ trailer.c: void free_trailers(struct list_head *head)
     + void free_trailers(struct list_head *trailers)
     +@@ trailer.c: void free_trailers(struct list_head *trailers)
       	}
       }
       
     @@ trailer.c: void free_trailers(struct list_head *head)
      +	free(trailer_block);
       }
       
     - void format_trailers_from_commit(const char *msg,
     -@@ trailer.c: void format_trailers_from_commit(const char *msg,
     + void format_trailers_from_commit(const struct process_trailer_options *opts,
     +@@ trailer.c: void format_trailers_from_commit(const struct process_trailer_options *opts,
       				 struct strbuf *out)
       {
       	LIST_HEAD(head);
     --	struct trailer_info *info = parse_trailers(msg, &head, opts);
     -+	struct trailer_block *trailer_block = parse_trailers(msg, opts, &head);
     +-	struct trailer_info *info = parse_trailers(opts, msg, &head);
     ++	struct trailer_block *trailer_block = parse_trailers(opts, msg, &head);
       
       	/* If we want the whole block untouched, we can take the fast path. */
       	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
     @@ trailer.c: void format_trailers_from_commit(const char *msg,
      +		strbuf_add(out, msg + trailer_block->start,
      +			   trailer_block->end - trailer_block->start);
       	} else
     - 		format_trailers(&head, opts, out);
     + 		format_trailers(opts, &head, out);
       
       	free_trailers(&head);
      -	trailer_info_release(info);
     @@ trailer.c: void format_trailers_from_commit(const char *msg,
      -	struct trailer_info *internal = trailer_info_new();
       	strbuf_init(&iter->key, 0);
       	strbuf_init(&iter->val, 0);
     - 	strbuf_init(&iter->raw, 0);
       	opts.no_divider = 1;
      -	iter->internal.info = internal;
     --	iter->internal.info = trailer_info_get(msg, &opts);
     -+	iter->internal.trailer_block = trailer_block_get(msg, &opts);
     +-	iter->internal.info = trailer_info_get(&opts, msg);
     ++	iter->internal.trailer_block = trailer_block_get(&opts, msg);
       	iter->internal.cur = 0;
       }
       
     @@ trailer.c: int trailer_iterator_advance(struct trailer_iterator *iter)
      +	trailer_block_release(iter->internal.trailer_block);
       	strbuf_release(&iter->val);
       	strbuf_release(&iter->key);
     - 	strbuf_release(&iter->raw);
     + }
      
       ## trailer.h ##
      @@
     @@ trailer.h: void parse_trailers_from_command_line_args(struct list_head *arg_head
       void process_trailers_lists(struct list_head *head,
       			    struct list_head *arg_head);
       
     --struct trailer_info *parse_trailers(const char *str,
     --				    struct list_head *head,
     --				    const struct process_trailer_options *opts);
     -+struct trailer_block *parse_trailers(const char *str,
     -+				     const struct process_trailer_options *opts,
     +-struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
     +-				    const char *str,
     +-				    struct list_head *head);
     ++struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
     ++				     const char *str,
      +				     struct list_head *head);
       
      -size_t trailer_block_start(struct trailer_info *info);
     @@ trailer.h: void parse_trailers_from_command_line_args(struct list_head *arg_head
      +void trailer_block_release(struct trailer_block *trailer_block);
       
       void trailer_config_init(void);
     - void free_trailers(struct list_head *trailers);
     + void format_trailers(const struct process_trailer_options *opts,
      @@ trailer.h: struct trailer_iterator {
       
       	/* private */
  8:  406725df46a !  8:  64ee07d0b53 trailer: prepare to move parse_trailers_from_command_line_args() to builtin
     @@ Commit message
              parse_trailers_from_command_line_args()
      
          to interpret-trailer.c, because the trailer API should not be concerned
     -    with command line arguments (as it has nothing to do with trailers
     -    themselves). The interpret-trailers builtin is the only user of the
     -    above function.
     +    with command line arguments (as they have nothing to do with trailers
     +    themselves). The interpret-trailers builtin is the only caller of this
     +    function.
      
     +    Also rename "conf_info" to "trailer_conf" for readability, dropping the
     +    low-value "_info" suffix as we did earlier in this series for
     +    "trailer_info" to "trailer_block".
     +
     +    Helped-by: Josh Steadmon <steadmon@google.com>
          Signed-off-by: Linus Arver <linusa@google.com>
      
       ## trailer.c ##
     @@ trailer.c: void parse_trailers_from_command_line_args(struct list_head *arg_head
       			add_arg_item(arg_head,
       				     strbuf_detach(&tok, NULL),
       				     strbuf_detach(&val, NULL),
     -@@ trailer.c: struct trailer_block *parse_trailers(const char *str,
     +@@ trailer.c: struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
       
       	for (i = 0; i < trailer_block->trailer_nr; i++) {
       		int separator_pos;
     @@ trailer.c: struct trailer_block *parse_trailers(const char *str,
       			add_trailer_item(head,
       					 NULL,
      @@ trailer.c: int trailer_iterator_advance(struct trailer_iterator *iter)
     - 		strbuf_addstr(&iter->raw, line);
     + 		iter->raw = line;
       		strbuf_reset(&iter->key);
       		strbuf_reset(&iter->val);
      -		parse_trailer(&iter->key, &iter->val, NULL,
     @@ trailer.h: void parse_trailers_from_command_line_args(struct list_head *arg_head
      +		   struct strbuf *tok, struct strbuf *val,
      +		   const struct trailer_conf **conf);
      +
     - struct trailer_block *parse_trailers(const char *str,
     - 				     const struct process_trailer_options *opts,
     + struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
     + 				     const char *str,
       				     struct list_head *head);
  9:  8a99d0fca21 !  9:  1b4bdde65bc trailer: move arg handling to interpret-trailers.c
     @@ builtin/interpret-trailers.c: static int option_parse_trailer(const struct optio
       }
       
      @@ builtin/interpret-trailers.c: static void read_input_file(struct strbuf *sb, const char *file)
     + }
       
     - static void interpret_trailers(const char *file,
     - 			       const struct process_trailer_options *opts,
     --			       struct list_head *new_trailer_head)
     -+			       struct list_head *arg_trailers)
     + static void interpret_trailers(const struct process_trailer_options *opts,
     +-			       struct list_head *new_trailer_head,
     ++			       struct list_head *arg_trailers,
     + 			       const char *file)
       {
       	LIST_HEAD(head);
     - 	struct strbuf sb = STRBUF_INIT;
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
       	struct trailer_block *trailer_block;
       	FILE *outfile = stdout;
       
     @@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
       	read_input_file(&sb, file);
       
       	if (opts->in_place)
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
       
       
       	if (!opts->only_input) {
     @@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
       	}
       
       	/* Print trailer block. */
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const char *file,
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
       int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
       {
       	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
     @@ builtin/interpret-trailers.c: int cmd_interpret_trailers(int argc, const char **
       	if (argc) {
       		int i;
       		for (i = 0; i < argc; i++)
     --			interpret_trailers(argv[i], &opts, &trailers);
     -+			interpret_trailers(argv[i], &opts, &arg_trailers);
     +-			interpret_trailers(&opts, &trailers, argv[i]);
     ++			interpret_trailers(&opts, &arg_trailers, argv[i]);
       	} else {
       		if (opts.in_place)
       			die(_("no input file given for in-place editing"));
     --		interpret_trailers(NULL, &opts, &trailers);
     -+		interpret_trailers(NULL, &opts, &arg_trailers);
     +-		interpret_trailers(&opts, &trailers, NULL);
     ++		interpret_trailers(&opts, &arg_trailers, NULL);
       	}
       
      -	new_trailers_clear(&trailers);
     @@ trailer.c: void parse_trailers_from_command_line_args(struct list_head *arg_head
       		}
       	}
       
     -@@ trailer.c: void free_trailers(struct list_head *head)
     +@@ trailer.c: void free_trailers(struct list_head *trailers)
       	}
       }
       
     @@ trailer.h: struct new_trailer_item {
       struct process_trailer_options {
       	int in_place;
       	int trim_empty;
     -@@ trailer.h: void trailer_block_release(struct trailer_block *trailer_block);
     - 
     - void trailer_config_init(void);
     +@@ trailer.h: void format_trailers(const struct process_trailer_options *opts,
     + 		     struct list_head *trailers,
     + 		     struct strbuf *out);
       void free_trailers(struct list_head *trailers);
     -+void new_trailers_clear(struct list_head *trailers);
     - 
     - void format_trailers(struct list_head *head,
     - 		     const struct process_trailer_options *opts,
     +-
     ++void new_trailers_clear(struct list_head *new_trailers);
     + /*
     +  * Convenience function to format the trailers from the commit msg "msg" into
     +  * the strbuf "out". Reuses format_trailers internally.
 10:  243eee730e4 = 10:  ed67ebf8647 trailer: delete obsolete argument handling code from API

-- 
gitgitgadget

[PATCH v2 01/10] trailer: prepare to expose functions as part of API

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

In the next patch, we will move "process_trailers" from trailer.c to
builtin/interpret-trailers.c. That move will necessitate the growth of
the trailer.h API, forcing us to expose some additional functions in
trailer.h.

Rename relevant functions so that they include the term "trailer" in
their name, so that clients of the API will be able to easily identify
them by their "trailer" moniker, just like all the other functions
already exposed by trailer.h.

The the opportunity to start putting trailer processions options (opts)
as the first parameter. This will be the pattern going forward in this
series.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  4 ++--
 trailer.c                    | 26 +++++++++++++-------------
 trailer.h                    |  6 +++---
 3 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 033bd1556cf..85a3413baf5 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -132,11 +132,11 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 	if (argc) {
 		int i;
 		for (i = 0; i < argc; i++)
-			process_trailers(argv[i], &opts, &trailers);
+			interpret_trailers(&opts, &trailers, argv[i]);
 	} else {
 		if (opts.in_place)
 			die(_("no input file given for in-place editing"));
-		process_trailers(NULL, &opts, &trailers);
+		interpret_trailers(&opts, &trailers, NULL);
 	}
 
 	new_trailers_clear(&trailers);
diff --git a/trailer.c b/trailer.c
index 3a0710a4583..66b6660f5a4 100644
--- a/trailer.c
+++ b/trailer.c
@@ -163,12 +163,12 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-static void print_all(FILE *outfile, struct list_head *head,
-		      const struct process_trailer_options *opts)
+static void format_trailers(const struct process_trailer_options *opts,
+			    struct list_head *trailers, FILE *outfile)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
-	list_for_each(pos, head) {
+	list_for_each(pos, trailers) {
 		item = list_entry(pos, struct trailer_item, list);
 		if ((!opts->trim_empty || strlen(item->value) > 0) &&
 		    (!opts->only_trailers || item->token))
@@ -589,7 +589,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	return 0;
 }
 
-static void ensure_configured(void)
+static void trailer_config_init(void)
 {
 	if (configured)
 		return;
@@ -1035,10 +1035,10 @@ static void parse_trailers(struct trailer_info *info,
 	}
 }
 
-static void free_all(struct list_head *head)
+static void free_trailers(struct list_head *trailers)
 {
 	struct list_head *pos, *p;
-	list_for_each_safe(pos, p, head) {
+	list_for_each_safe(pos, p, trailers) {
 		list_del(pos);
 		free_trailer_item(list_entry(pos, struct trailer_item, list));
 	}
@@ -1075,16 +1075,16 @@ static FILE *create_in_place_tempfile(const char *file)
 	return outfile;
 }
 
-void process_trailers(const char *file,
-		      const struct process_trailer_options *opts,
-		      struct list_head *new_trailer_head)
+void interpret_trailers(const struct process_trailer_options *opts,
+			struct list_head *new_trailer_head,
+			const char *file)
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
 	struct trailer_info info;
 	FILE *outfile = stdout;
 
-	ensure_configured();
+	trailer_config_init();
 
 	read_input_file(&sb, file);
 
@@ -1110,9 +1110,9 @@ void process_trailers(const char *file,
 		process_trailers_lists(&head, &arg_head);
 	}
 
-	print_all(outfile, &head, opts);
+	format_trailers(opts, &head, outfile);
 
-	free_all(&head);
+	free_trailers(&head);
 	trailer_info_release(&info);
 
 	/* Print the lines after the trailers as is */
@@ -1135,7 +1135,7 @@ void trailer_info_get(struct trailer_info *info, const char *str,
 	size_t nr = 0, alloc = 0;
 	char **last = NULL;
 
-	ensure_configured();
+	trailer_config_init();
 
 	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
 	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
diff --git a/trailer.h b/trailer.h
index 1644cd05f60..37033e631a1 100644
--- a/trailer.h
+++ b/trailer.h
@@ -81,9 +81,9 @@ struct process_trailer_options {
 
 #define PROCESS_TRAILER_OPTIONS_INIT {0}
 
-void process_trailers(const char *file,
-		      const struct process_trailer_options *opts,
-		      struct list_head *new_trailer_head);
+void interpret_trailers(const struct process_trailer_options *opts,
+			struct list_head *new_trailer_head,
+			const char *file);
 
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts);
-- 
gitgitgadget

[PATCH v2 02/10] trailer: move interpret_trailers() to interpret-trailers.c

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

The interpret-trailers.c builtin is the only place we need to call
interpret_trailers(), so move its definition there.

Delete the corresponding declaration from trailer.h, which then forces
us to expose the working innards of that function. This enriches
trailer.h with a more granular API, which can then be unit-tested in the
future (because interpret_trailers() by itself does too many things to
be able to be easily unit-tested).

Take this opportunity to demote some file-handling functions out of the
trailer API implementation, as these have nothing to do with trailers.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  94 +++++++++++++++++++++++++++
 trailer.c                    | 120 ++++-------------------------------
 trailer.h                    |  20 +++++-
 3 files changed, 124 insertions(+), 110 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 85a3413baf5..8556acde4aa 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -9,6 +9,7 @@
 #include "gettext.h"
 #include "parse-options.h"
 #include "string-list.h"
+#include "tempfile.h"
 #include "trailer.h"
 #include "config.h"
 
@@ -91,6 +92,99 @@ static int parse_opt_parse(const struct option *opt, const char *arg,
 	return 0;
 }
 
+static struct tempfile *trailers_tempfile;
+
+static FILE *create_in_place_tempfile(const char *file)
+{
+	struct stat st;
+	struct strbuf filename_template = STRBUF_INIT;
+	const char *tail;
+	FILE *outfile;
+
+	if (stat(file, &st))
+		die_errno(_("could not stat %s"), file);
+	if (!S_ISREG(st.st_mode))
+		die(_("file %s is not a regular file"), file);
+	if (!(st.st_mode & S_IWUSR))
+		die(_("file %s is not writable by user"), file);
+
+	/* Create temporary file in the same directory as the original */
+	tail = strrchr(file, '/');
+	if (tail)
+		strbuf_add(&filename_template, file, tail - file + 1);
+	strbuf_addstr(&filename_template, "git-interpret-trailers-XXXXXX");
+
+	trailers_tempfile = xmks_tempfile_m(filename_template.buf, st.st_mode);
+	strbuf_release(&filename_template);
+	outfile = fdopen_tempfile(trailers_tempfile, "w");
+	if (!outfile)
+		die_errno(_("could not open temporary file"));
+
+	return outfile;
+}
+
+static void read_input_file(struct strbuf *sb, const char *file)
+{
+	if (file) {
+		if (strbuf_read_file(sb, file, 0) < 0)
+			die_errno(_("could not read input file '%s'"), file);
+	} else {
+		if (strbuf_read(sb, fileno(stdin), 0) < 0)
+			die_errno(_("could not read from stdin"));
+	}
+}
+
+static void interpret_trailers(const struct process_trailer_options *opts,
+			       struct list_head *new_trailer_head,
+			       const char *file)
+{
+	LIST_HEAD(head);
+	struct strbuf sb = STRBUF_INIT;
+	struct trailer_info info;
+	FILE *outfile = stdout;
+
+	trailer_config_init();
+
+	read_input_file(&sb, file);
+
+	if (opts->in_place)
+		outfile = create_in_place_tempfile(file);
+
+	parse_trailers(opts, &info, sb.buf, &head);
+
+	/* Print the lines before the trailers */
+	if (!opts->only_trailers)
+		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
+
+	if (!opts->only_trailers && !info.blank_line_before_trailer)
+		fprintf(outfile, "\n");
+
+
+	if (!opts->only_input) {
+		LIST_HEAD(config_head);
+		LIST_HEAD(arg_head);
+		parse_trailers_from_config(&config_head);
+		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
+		list_splice(&config_head, &arg_head);
+		process_trailers_lists(&head, &arg_head);
+	}
+
+	format_trailers(opts, &head, outfile);
+
+	free_trailers(&head);
+	trailer_info_release(&info);
+
+	/* Print the lines after the trailers as is */
+	if (!opts->only_trailers)
+		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
+
+	if (opts->in_place)
+		if (rename_tempfile(&trailers_tempfile, file))
+			die_errno(_("could not rename temporary file to %s"), file);
+
+	strbuf_release(&sb);
+}
+
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
diff --git a/trailer.c b/trailer.c
index 66b6660f5a4..d3899195876 100644
--- a/trailer.c
+++ b/trailer.c
@@ -5,7 +5,6 @@
 #include "string-list.h"
 #include "run-command.h"
 #include "commit.h"
-#include "tempfile.h"
 #include "trailer.h"
 #include "list.h"
 /*
@@ -163,8 +162,8 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-static void format_trailers(const struct process_trailer_options *opts,
-			    struct list_head *trailers, FILE *outfile)
+void format_trailers(const struct process_trailer_options *opts,
+		     struct list_head *trailers, FILE *outfile)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
@@ -366,8 +365,8 @@ static int find_same_and_apply_arg(struct list_head *head,
 	return 0;
 }
 
-static void process_trailers_lists(struct list_head *head,
-				   struct list_head *arg_head)
+void process_trailers_lists(struct list_head *head,
+			    struct list_head *arg_head)
 {
 	struct list_head *pos, *p;
 	struct arg_item *arg_tok;
@@ -589,7 +588,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	return 0;
 }
 
-static void trailer_config_init(void)
+void trailer_config_init(void)
 {
 	if (configured)
 		return;
@@ -719,7 +718,7 @@ static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
 	list_add_tail(&new_item->list, arg_head);
 }
 
-static void parse_trailers_from_config(struct list_head *config_head)
+void parse_trailers_from_config(struct list_head *config_head)
 {
 	struct arg_item *item;
 	struct list_head *pos;
@@ -735,8 +734,8 @@ static void parse_trailers_from_config(struct list_head *config_head)
 	}
 }
 
-static void parse_trailers_from_command_line_args(struct list_head *arg_head,
-						  struct list_head *new_trailer_head)
+void parse_trailers_from_command_line_args(struct list_head *arg_head,
+					   struct list_head *new_trailer_head)
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
@@ -775,17 +774,6 @@ static void parse_trailers_from_command_line_args(struct list_head *arg_head,
 	free(cl_separators);
 }
 
-static void read_input_file(struct strbuf *sb, const char *file)
-{
-	if (file) {
-		if (strbuf_read_file(sb, file, 0) < 0)
-			die_errno(_("could not read input file '%s'"), file);
-	} else {
-		if (strbuf_read(sb, fileno(stdin), 0) < 0)
-			die_errno(_("could not read from stdin"));
-	}
-}
-
 static const char *next_line(const char *str)
 {
 	const char *nl = strchrnul(str, '\n');
@@ -1000,10 +988,10 @@ static void unfold_value(struct strbuf *val)
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
  */
-static void parse_trailers(struct trailer_info *info,
-			     const char *str,
-			     struct list_head *head,
-			     const struct process_trailer_options *opts)
+void parse_trailers(const struct process_trailer_options *opts,
+		    struct trailer_info *info,
+		    const char *str,
+		    struct list_head *head)
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
@@ -1035,7 +1023,7 @@ static void parse_trailers(struct trailer_info *info,
 	}
 }
 
-static void free_trailers(struct list_head *trailers)
+void free_trailers(struct list_head *trailers)
 {
 	struct list_head *pos, *p;
 	list_for_each_safe(pos, p, trailers) {
@@ -1044,88 +1032,6 @@ static void free_trailers(struct list_head *trailers)
 	}
 }
 
-static struct tempfile *trailers_tempfile;
-
-static FILE *create_in_place_tempfile(const char *file)
-{
-	struct stat st;
-	struct strbuf filename_template = STRBUF_INIT;
-	const char *tail;
-	FILE *outfile;
-
-	if (stat(file, &st))
-		die_errno(_("could not stat %s"), file);
-	if (!S_ISREG(st.st_mode))
-		die(_("file %s is not a regular file"), file);
-	if (!(st.st_mode & S_IWUSR))
-		die(_("file %s is not writable by user"), file);
-
-	/* Create temporary file in the same directory as the original */
-	tail = strrchr(file, '/');
-	if (tail)
-		strbuf_add(&filename_template, file, tail - file + 1);
-	strbuf_addstr(&filename_template, "git-interpret-trailers-XXXXXX");
-
-	trailers_tempfile = xmks_tempfile_m(filename_template.buf, st.st_mode);
-	strbuf_release(&filename_template);
-	outfile = fdopen_tempfile(trailers_tempfile, "w");
-	if (!outfile)
-		die_errno(_("could not open temporary file"));
-
-	return outfile;
-}
-
-void interpret_trailers(const struct process_trailer_options *opts,
-			struct list_head *new_trailer_head,
-			const char *file)
-{
-	LIST_HEAD(head);
-	struct strbuf sb = STRBUF_INIT;
-	struct trailer_info info;
-	FILE *outfile = stdout;
-
-	trailer_config_init();
-
-	read_input_file(&sb, file);
-
-	if (opts->in_place)
-		outfile = create_in_place_tempfile(file);
-
-	parse_trailers(&info, sb.buf, &head, opts);
-
-	/* Print the lines before the trailers */
-	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
-
-	if (!opts->only_trailers && !info.blank_line_before_trailer)
-		fprintf(outfile, "\n");
-
-
-	if (!opts->only_input) {
-		LIST_HEAD(config_head);
-		LIST_HEAD(arg_head);
-		parse_trailers_from_config(&config_head);
-		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
-		list_splice(&config_head, &arg_head);
-		process_trailers_lists(&head, &arg_head);
-	}
-
-	format_trailers(opts, &head, outfile);
-
-	free_trailers(&head);
-	trailer_info_release(&info);
-
-	/* Print the lines after the trailers as is */
-	if (!opts->only_trailers)
-		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
-
-	if (opts->in_place)
-		if (rename_tempfile(&trailers_tempfile, file))
-			die_errno(_("could not rename temporary file to %s"), file);
-
-	strbuf_release(&sb);
-}
-
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts)
 {
diff --git a/trailer.h b/trailer.h
index 37033e631a1..4f603e03ceb 100644
--- a/trailer.h
+++ b/trailer.h
@@ -81,15 +81,29 @@ struct process_trailer_options {
 
 #define PROCESS_TRAILER_OPTIONS_INIT {0}
 
-void interpret_trailers(const struct process_trailer_options *opts,
-			struct list_head *new_trailer_head,
-			const char *file);
+void parse_trailers_from_config(struct list_head *config_head);
+
+void parse_trailers_from_command_line_args(struct list_head *arg_head,
+					   struct list_head *new_trailer_head);
+
+void process_trailers_lists(struct list_head *head,
+			    struct list_head *arg_head);
+
+void parse_trailers(const struct process_trailer_options *opts,
+		    struct trailer_info *info,
+		    const char *str,
+		    struct list_head *head);
 
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts);
 
 void trailer_info_release(struct trailer_info *info);
 
+void trailer_config_init(void);
+void format_trailers(const struct process_trailer_options *opts,
+		     struct list_head *trailers, FILE *outfile);
+void free_trailers(struct list_head *trailers);
+
 /*
  * Format the trailers from the commit msg "msg" into the strbuf "out".
  * Note two caveats about "opts":
-- 
gitgitgadget

[PATCH v2 03/10] trailer: unify trailer formatting machinery

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Currently have two functions for formatting trailers exposed in
trailer.h:

    void format_trailers(FILE *outfile, struct list_head *head,
                        const struct process_trailer_options *opts);

    void format_trailers_from_commit(struct strbuf *out, const char *msg,
                                    const struct process_trailer_options *opts);

and previously these functions, although similar enough (even taking the
same process_trailer_options struct pointer), did not build on each
other.

Make format_trailers_from_commit() rely on format_trailers(). Teach
format_trailers() to process trailers with the additional
process_trailer_options fields like opts->key_only which is only used by
format_trailers_from_commit() and not builtin/interpret-trailers.c.
While we're at it, reorder parameters to put the trailer processing
options first, and the out parameter (strbuf we write into) at the end.

This unification will allow us to delete the format_trailer_info() and
print_tok_val() functions in the next patch. They are not deleted here
in order to keep the diff small.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |   6 +-
 pretty.c                     |   2 +-
 ref-filter.c                 |   2 +-
 trailer.c                    | 105 +++++++++++++++++++++++++++++------
 trailer.h                    |  19 +++----
 5 files changed, 102 insertions(+), 32 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 8556acde4aa..5352ee65bd1 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -140,6 +140,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
+	struct strbuf trailer_block = STRBUF_INIT;
 	struct trailer_info info;
 	FILE *outfile = stdout;
 
@@ -169,7 +170,10 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 		process_trailers_lists(&head, &arg_head);
 	}
 
-	format_trailers(opts, &head, outfile);
+	/* Print trailer block. */
+	format_trailers(opts, &head, &trailer_block);
+	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
+	strbuf_release(&trailer_block);
 
 	free_trailers(&head);
 	trailer_info_release(&info);
diff --git a/pretty.c b/pretty.c
index cf964b060cd..bdbed4295aa 100644
--- a/pretty.c
+++ b/pretty.c
@@ -1759,7 +1759,7 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */
 				goto trailer_out;
 		}
 		if (*arg == ')') {
-			format_trailers_from_commit(sb, msg + c->subject_off, &opts);
+			format_trailers_from_commit(&opts, msg + c->subject_off, sb);
 			ret = arg - placeholder + 1;
 		}
 	trailer_out:
diff --git a/ref-filter.c b/ref-filter.c
index 35b989e1dfe..d358953b0ce 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -1985,7 +1985,7 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp
 			struct strbuf s = STRBUF_INIT;
 
 			/* Format the trailer info according to the trailer_opts given */
-			format_trailers_from_commit(&s, subpos, &atom->u.contents.trailer_opts);
+			format_trailers_from_commit(&atom->u.contents.trailer_opts, subpos, &s);
 
 			v->s = strbuf_detach(&s, NULL);
 		} else if (atom->u.contents.option == C_BARE)
diff --git a/trailer.c b/trailer.c
index d3899195876..7692bf9eb40 100644
--- a/trailer.c
+++ b/trailer.c
@@ -162,19 +162,6 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-void format_trailers(const struct process_trailer_options *opts,
-		     struct list_head *trailers, FILE *outfile)
-{
-	struct list_head *pos;
-	struct trailer_item *item;
-	list_for_each(pos, trailers) {
-		item = list_entry(pos, struct trailer_item, list);
-		if ((!opts->trim_empty || strlen(item->value) > 0) &&
-		    (!opts->only_trailers || item->token))
-			print_tok_val(outfile, item->token, item->value);
-	}
-}
-
 static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
 {
 	struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
@@ -984,6 +971,78 @@ static void unfold_value(struct strbuf *val)
 	strbuf_release(&out);
 }
 
+void format_trailers(const struct process_trailer_options *opts,
+		     struct list_head *trailers,
+		     struct strbuf *out)
+{
+	struct list_head *pos;
+	struct trailer_item *item;
+	int need_separator = 0;
+
+	list_for_each(pos, trailers) {
+		item = list_entry(pos, struct trailer_item, list);
+		if (item->token) {
+			char c;
+
+			struct strbuf tok = STRBUF_INIT;
+			struct strbuf val = STRBUF_INIT;
+			strbuf_addstr(&tok, item->token);
+			strbuf_addstr(&val, item->value);
+
+			/*
+			 * Skip key/value pairs where the value was empty. This
+			 * can happen from trailers specified without a
+			 * separator, like `--trailer "Reviewed-by"` (no
+			 * corresponding value).
+			 */
+			if (opts->trim_empty && !strlen(item->value))
+				continue;
+
+			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
+				if (opts->unfold)
+					unfold_value(&val);
+
+				if (opts->separator && need_separator)
+					strbuf_addbuf(out, opts->separator);
+				if (!opts->value_only)
+					strbuf_addbuf(out, &tok);
+				if (!opts->key_only && !opts->value_only) {
+					if (opts->key_value_separator)
+						strbuf_addbuf(out, opts->key_value_separator);
+					else {
+						c = last_non_space_char(tok.buf);
+						if (c) {
+							if (!strchr(separators, c))
+								strbuf_addf(out, "%c ", separators[0]);
+						}
+					}
+				}
+				if (!opts->key_only)
+					strbuf_addbuf(out, &val);
+				if (!opts->separator)
+					strbuf_addch(out, '\n');
+
+				need_separator = 1;
+			}
+
+			strbuf_release(&tok);
+			strbuf_release(&val);
+		} else if (!opts->only_trailers) {
+			if (opts->separator && need_separator) {
+				strbuf_addbuf(out, opts->separator);
+			}
+			strbuf_addstr(out, item->value);
+			if (opts->separator)
+				strbuf_rtrim(out);
+			else
+				strbuf_addch(out, '\n');
+
+			need_separator = 1;
+		}
+
+	}
+}
+
 /*
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
@@ -1144,13 +1203,25 @@ static void format_trailer_info(struct strbuf *out,
 
 }
 
-void format_trailers_from_commit(struct strbuf *out, const char *msg,
-				 const struct process_trailer_options *opts)
+void format_trailers_from_commit(const struct process_trailer_options *opts,
+				 const char *msg,
+				 struct strbuf *out)
 {
+	LIST_HEAD(head);
 	struct trailer_info info;
 
-	trailer_info_get(&info, msg, opts);
-	format_trailer_info(out, &info, msg, opts);
+	parse_trailers(opts, &info, msg, &head);
+
+	/* If we want the whole block untouched, we can take the fast path. */
+	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
+	    !opts->separator && !opts->key_only && !opts->value_only &&
+	    !opts->key_value_separator) {
+		strbuf_add(out, msg + info.trailer_block_start,
+			   info.trailer_block_end - info.trailer_block_start);
+	} else
+		format_trailers(opts, &head, out);
+
+	free_trailers(&head);
 	trailer_info_release(&info);
 }
 
diff --git a/trailer.h b/trailer.h
index 4f603e03ceb..c309b01323d 100644
--- a/trailer.h
+++ b/trailer.h
@@ -101,22 +101,17 @@ void trailer_info_release(struct trailer_info *info);
 
 void trailer_config_init(void);
 void format_trailers(const struct process_trailer_options *opts,
-		     struct list_head *trailers, FILE *outfile);
+		     struct list_head *trailers,
+		     struct strbuf *out);
 void free_trailers(struct list_head *trailers);
 
 /*
- * Format the trailers from the commit msg "msg" into the strbuf "out".
- * Note two caveats about "opts":
- *
- *   - this is primarily a helper for pretty.c, and not
- *     all of the flags are supported.
- *
- *   - this differs from process_trailers slightly in that we always format
- *     only the trailer block itself, even if the "only_trailers" option is not
- *     set.
+ * Convenience function to format the trailers from the commit msg "msg" into
+ * the strbuf "out". Reuses format_trailers internally.
  */
-void format_trailers_from_commit(struct strbuf *out, const char *msg,
-				 const struct process_trailer_options *opts);
+void format_trailers_from_commit(const struct process_trailer_options *opts,
+				 const char *msg,
+				 struct strbuf *out);
 
 /*
  * An interface for iterating over the trailers found in a particular commit
-- 
gitgitgadget

[PATCH v2 04/10] trailer: delete obsolete formatting functions

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 79 -------------------------------------------------------
 1 file changed, 79 deletions(-)

diff --git a/trailer.c b/trailer.c
index 7692bf9eb40..71ea2bb67f8 100644
--- a/trailer.c
+++ b/trailer.c
@@ -144,24 +144,6 @@ static char last_non_space_char(const char *s)
 	return '\0';
 }
 
-static void print_tok_val(FILE *outfile, const char *tok, const char *val)
-{
-	char c;
-
-	if (!tok) {
-		fprintf(outfile, "%s\n", val);
-		return;
-	}
-
-	c = last_non_space_char(tok);
-	if (!c)
-		return;
-	if (strchr(separators, c))
-		fprintf(outfile, "%s%s\n", tok, val);
-	else
-		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
-}
-
 static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
 {
 	struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
@@ -1142,67 +1124,6 @@ void trailer_info_release(struct trailer_info *info)
 	free(info->trailers);
 }
 
-static void format_trailer_info(struct strbuf *out,
-				const struct trailer_info *info,
-				const char *msg,
-				const struct process_trailer_options *opts)
-{
-	size_t origlen = out->len;
-	size_t i;
-
-	/* If we want the whole block untouched, we can take the fast path. */
-	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
-	    !opts->separator && !opts->key_only && !opts->value_only &&
-	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info->trailer_block_start,
-			   info->trailer_block_end - info->trailer_block_start);
-		return;
-	}
-
-	for (i = 0; i < info->trailer_nr; i++) {
-		char *trailer = info->trailers[i];
-		ssize_t separator_pos = find_separator(trailer, separators);
-
-		if (separator_pos >= 1) {
-			struct strbuf tok = STRBUF_INIT;
-			struct strbuf val = STRBUF_INIT;
-
-			parse_trailer(&tok, &val, NULL, trailer, separator_pos);
-			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
-				if (opts->unfold)
-					unfold_value(&val);
-
-				if (opts->separator && out->len != origlen)
-					strbuf_addbuf(out, opts->separator);
-				if (!opts->value_only)
-					strbuf_addbuf(out, &tok);
-				if (!opts->key_only && !opts->value_only) {
-					if (opts->key_value_separator)
-						strbuf_addbuf(out, opts->key_value_separator);
-					else
-						strbuf_addstr(out, ": ");
-				}
-				if (!opts->key_only)
-					strbuf_addbuf(out, &val);
-				if (!opts->separator)
-					strbuf_addch(out, '\n');
-			}
-			strbuf_release(&tok);
-			strbuf_release(&val);
-
-		} else if (!opts->only_trailers) {
-			if (opts->separator && out->len != origlen) {
-				strbuf_addbuf(out, opts->separator);
-			}
-			strbuf_addstr(out, trailer);
-			if (opts->separator) {
-				strbuf_rtrim(out);
-			}
-		}
-	}
-
-}
-
 void format_trailers_from_commit(const struct process_trailer_options *opts,
 				 const char *msg,
 				 struct strbuf *out)
-- 
gitgitgadget

[PATCH v2 05/10] sequencer: use the trailer iterator

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This patch allows for the removal of "trailer_info_get()" from the
trailer.h API, which will be in the next patch.

Instead of calling "trailer_info_get()", which is a low-level function
in the trailers implementation (trailer.c), call
trailer_iterator_advance(), which was specifically designed for public
consumption in f0939a0eb1 (trailer: add interface for iterating over
commit trailers, 2020-09-27).

Avoiding "trailer_info_get()" means we don't have to worry about options
like "no_divider" (relevant for parsing trailers). We also don't have to
check for things like "info.trailer_start == info.trailer_end" to see
whether there were any trailers (instead we can just check to see
whether the iterator advanced at all).

Also, teach the iterator about non-trailer lines, by adding a new field
called "raw" to hold both trailer and non-trailer lines. This is
necessary because a "trailer block" is a list of trailer lines of at
least 25% trailers (see 146245063e (trailer: allow non-trailers in
trailer block, 2016-10-21)), such that it may hold non-trailer lines.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/shortlog.c |  7 +++++--
 sequencer.c        | 35 +++++++++++++++--------------------
 trailer.c          | 17 +++++++++--------
 trailer.h          | 13 +++++++++++++
 4 files changed, 42 insertions(+), 30 deletions(-)

diff --git a/builtin/shortlog.c b/builtin/shortlog.c
index 1307ed2b88a..dc8fd5a5532 100644
--- a/builtin/shortlog.c
+++ b/builtin/shortlog.c
@@ -172,7 +172,7 @@ static void insert_records_from_trailers(struct shortlog *log,
 					 const char *oneline)
 {
 	struct trailer_iterator iter;
-	const char *commit_buffer, *body;
+	const char *commit_buffer, *body, *value;
 	struct strbuf ident = STRBUF_INIT;
 
 	if (!log->trailers.nr)
@@ -190,7 +190,10 @@ static void insert_records_from_trailers(struct shortlog *log,
 
 	trailer_iterator_init(&iter, body);
 	while (trailer_iterator_advance(&iter)) {
-		const char *value = iter.val.buf;
+		if (!iter.is_trailer)
+			continue;
+
+		value = iter.val.buf;
 
 		if (!string_list_has_string(&log->trailers, iter.key.buf))
 			continue;
diff --git a/sequencer.c b/sequencer.c
index 3cc88d8a800..bc7c82c5271 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -319,37 +319,32 @@ static const char *get_todo_path(const struct replay_opts *opts)
 static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	size_t ignore_footer)
 {
-	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
-	struct trailer_info info;
-	size_t i;
-	int found_sob = 0, found_sob_last = 0;
-	char saved_char;
-
-	opts.no_divider = 1;
+	struct trailer_iterator iter;
+	size_t i = 0, found_sob = 0;
+	char saved_char = sb->buf[sb->len - ignore_footer];
 
 	if (ignore_footer) {
-		saved_char = sb->buf[sb->len - ignore_footer];
 		sb->buf[sb->len - ignore_footer] = '\0';
 	}
 
-	trailer_info_get(&info, sb->buf, &opts);
+	trailer_iterator_init(&iter, sb->buf);
+	while (trailer_iterator_advance(&iter)) {
+		i++;
+		if (sob &&
+		    iter.is_trailer &&
+		    !strncmp(iter.raw, sob->buf, sob->len)) {
+			found_sob = i;
+		}
+	}
+	trailer_iterator_release(&iter);
 
 	if (ignore_footer)
 		sb->buf[sb->len - ignore_footer] = saved_char;
 
-	if (info.trailer_block_start == info.trailer_block_end)
+	if (!i)
 		return 0;
 
-	for (i = 0; i < info.trailer_nr; i++)
-		if (sob && !strncmp(info.trailers[i], sob->buf, sob->len)) {
-			found_sob = 1;
-			if (i == info.trailer_nr - 1)
-				found_sob_last = 1;
-		}
-
-	trailer_info_release(&info);
-
-	if (found_sob_last)
+	if (found_sob == i)
 		return 3;
 	if (found_sob)
 		return 2;
diff --git a/trailer.c b/trailer.c
index 71ea2bb67f8..5bcc9b0006c 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1158,17 +1158,18 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 
 int trailer_iterator_advance(struct trailer_iterator *iter)
 {
-	while (iter->internal.cur < iter->internal.info.trailer_nr) {
-		char *trailer = iter->internal.info.trailers[iter->internal.cur++];
-		int separator_pos = find_separator(trailer, separators);
-
-		if (separator_pos < 1)
-			continue; /* not a real trailer */
-
+	char *line;
+	int separator_pos;
+	if (iter->internal.cur < iter->internal.info.trailer_nr) {
+		line = iter->internal.info.trailers[iter->internal.cur++];
+		separator_pos = find_separator(line, separators);
+		iter->is_trailer = (separator_pos > 0);
+
+		iter->raw = line;
 		strbuf_reset(&iter->key);
 		strbuf_reset(&iter->val);
 		parse_trailer(&iter->key, &iter->val, NULL,
-			      trailer, separator_pos);
+			      line, separator_pos);
 		unfold_value(&iter->val);
 		return 1;
 	}
diff --git a/trailer.h b/trailer.h
index c309b01323d..a47f18b1525 100644
--- a/trailer.h
+++ b/trailer.h
@@ -127,6 +127,19 @@ struct trailer_iterator {
 	struct strbuf key;
 	struct strbuf val;
 
+	/*
+	 * Raw line (e.g., "foo: bar baz") before being parsed as a trailer
+	 * key/val pair as part of a trailer block. A trailer block can be
+	 * either 100% trailer lines, or mixed in with non-trailer lines (in
+	 * which case at least 25% must be trailer lines).
+	 */
+	const char *raw;
+
+	/*
+	 * 1 if the raw line was parsed as a trailer line (key/val pair).
+	 */
+	int is_trailer;
+
 	/* private */
 	struct {
 		struct trailer_info info;
-- 
gitgitgadget

[PATCH v2 06/10] trailer: make trailer_info struct private

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

In 13211ae23f (trailer: separate public from internal portion of
trailer_iterator, 2023-09-09) we moved trailer_info behind an anonymous
struct to discourage use by trailer.h API users. However it still left
open the possibility of external use of trailer_info itself. Now that
there are no external users of trailer_info, we can make this struct
private.

Make this struct private by putting its definition inside trailer.c.
This has two benefits:

  (1) it makes the surface area of the public facing
      interface (trailer.h) smaller, and

  (2) external API users are unable to peer inside this struct (because
      it is only ever exposed as an opaque pointer).

This change exposes some deficiencies in the API, mainly with regard to
information about the location of the trailer block that was parsed.
Expose new API functions to access this information (needed by
builtin/interpret-trailers.c).

The idea in this patch to hide implementation details behind an "opaque
pointer" is also known as the "pimpl" (pointer to implementation) idiom
in C++ and is a common pattern in that language (where, for example,
abstract classes only have pointers to concrete classes).

However, the original inspiration to use this idiom does not come from
C++, but instead the book "C Interfaces and Implementations: Techniques
for Creating Reusable Software" [1]. This book recommends opaque
pointers as a good design principle for designing C libraries, using the
term "interface" as the functions defined in *.h (header) files and
"implementation" as the corresponding *.c file which define the
interfaces.

The book says this about opaque pointers:

    ... clients can manipulate such pointers freely, but they can’t
    dereference them; that is, they can’t look at the innards of the
    structure pointed to by them. Only the implementation has that
    privilege. Opaque pointers hide representation details and help
    catch errors.

In our case, "struct trailer_info" is now hidden from clients, and the
ways in which this opaque pointer can be used is limited to the richness
of the trailer.h file. In other words, trailer.h exclusively controls
exactly how "trailer_info" pointers are to be used.

[1] Hanson, David R. "C Interfaces and Implementations: Techniques for
    Creating Reusable Software". Addison Wesley, 1997. p. 22

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  13 +--
 trailer.c                    | 154 +++++++++++++++++++++++------------
 trailer.h                    |  37 ++-------
 3 files changed, 117 insertions(+), 87 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 5352ee65bd1..9e6ed6b65e2 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -141,7 +141,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
 	struct strbuf trailer_block = STRBUF_INIT;
-	struct trailer_info info;
+	struct trailer_info *info;
 	FILE *outfile = stdout;
 
 	trailer_config_init();
@@ -151,13 +151,13 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	if (opts->in_place)
 		outfile = create_in_place_tempfile(file);
 
-	parse_trailers(opts, &info, sb.buf, &head);
+	info = parse_trailers(opts, sb.buf, &head);
 
 	/* Print the lines before the trailers */
 	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
+		fwrite(sb.buf, 1, trailer_block_start(info), outfile);
 
-	if (!opts->only_trailers && !info.blank_line_before_trailer)
+	if (!opts->only_trailers && !blank_line_before_trailer_block(info))
 		fprintf(outfile, "\n");
 
 
@@ -176,11 +176,12 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	strbuf_release(&trailer_block);
 
 	free_trailers(&head);
-	trailer_info_release(&info);
 
 	/* Print the lines after the trailers as is */
 	if (!opts->only_trailers)
-		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
+		fwrite(sb.buf + trailer_block_end(info), 1, sb.len - trailer_block_end(info), outfile);
+
+	trailer_info_release(info);
 
 	if (opts->in_place)
 		if (rename_tempfile(&trailers_tempfile, file))
diff --git a/trailer.c b/trailer.c
index 5bcc9b0006c..63774cd068d 100644
--- a/trailer.c
+++ b/trailer.c
@@ -11,6 +11,27 @@
  * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
  */
 
+struct trailer_info {
+	/*
+	 * True if there is a blank line before the location pointed to by
+	 * trailer_block_start.
+	 */
+	int blank_line_before_trailer;
+
+	/*
+	 * Offsets to the trailer block start and end positions in the input
+	 * string. If no trailer block is found, these are both set to the
+	 * "true" end of the input (find_end_of_log_message()).
+	 */
+	size_t trailer_block_start, trailer_block_end;
+
+	/*
+	 * Array of trailers found.
+	 */
+	char **trailers;
+	size_t trailer_nr;
+};
+
 struct conf_info {
 	char *name;
 	char *key;
@@ -1025,20 +1046,72 @@ void format_trailers(const struct process_trailer_options *opts,
 	}
 }
 
+static struct trailer_info *trailer_info_new(void)
+{
+	struct trailer_info *info = xcalloc(1, sizeof(*info));
+	return info;
+}
+
+static struct trailer_info *trailer_info_get(const struct process_trailer_options *opts,
+					     const char *str)
+{
+	struct trailer_info *info = trailer_info_new();
+	size_t end_of_log_message = 0, trailer_block_start = 0;
+	struct strbuf **trailer_lines, **ptr;
+	char **trailer_strings = NULL;
+	size_t nr = 0, alloc = 0;
+	char **last = NULL;
+
+	trailer_config_init();
+
+	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
+	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
+
+	trailer_lines = strbuf_split_buf(str + trailer_block_start,
+					 end_of_log_message - trailer_block_start,
+					 '\n',
+					 0);
+	for (ptr = trailer_lines; *ptr; ptr++) {
+		if (last && isspace((*ptr)->buf[0])) {
+			struct strbuf sb = STRBUF_INIT;
+			strbuf_attach(&sb, *last, strlen(*last), strlen(*last));
+			strbuf_addbuf(&sb, *ptr);
+			*last = strbuf_detach(&sb, NULL);
+			continue;
+		}
+		ALLOC_GROW(trailer_strings, nr + 1, alloc);
+		trailer_strings[nr] = strbuf_detach(*ptr, NULL);
+		last = find_separator(trailer_strings[nr], separators) >= 1
+			? &trailer_strings[nr]
+			: NULL;
+		nr++;
+	}
+	strbuf_list_free(trailer_lines);
+
+	info->blank_line_before_trailer = ends_with_blank_line(str,
+							       trailer_block_start);
+	info->trailer_block_start = trailer_block_start;
+	info->trailer_block_end = end_of_log_message;
+	info->trailers = trailer_strings;
+	info->trailer_nr = nr;
+
+	return info;
+}
+
 /*
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
  */
-void parse_trailers(const struct process_trailer_options *opts,
-		    struct trailer_info *info,
-		    const char *str,
-		    struct list_head *head)
+struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
+				    const char *str,
+				    struct list_head *head)
 {
+	struct trailer_info *info;
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
 	size_t i;
 
-	trailer_info_get(info, str, opts);
+	info = trailer_info_get(opts, str);
 
 	for (i = 0; i < info->trailer_nr; i++) {
 		int separator_pos;
@@ -1062,6 +1135,8 @@ void parse_trailers(const struct process_trailer_options *opts,
 					 strbuf_detach(&val, NULL));
 		}
 	}
+
+	return info;
 }
 
 void free_trailers(struct list_head *trailers)
@@ -1073,47 +1148,19 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
-void trailer_info_get(struct trailer_info *info, const char *str,
-		      const struct process_trailer_options *opts)
+size_t trailer_block_start(struct trailer_info *info)
 {
-	size_t end_of_log_message = 0, trailer_block_start = 0;
-	struct strbuf **trailer_lines, **ptr;
-	char **trailer_strings = NULL;
-	size_t nr = 0, alloc = 0;
-	char **last = NULL;
-
-	trailer_config_init();
-
-	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
-	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
+	return info->trailer_block_start;
+}
 
-	trailer_lines = strbuf_split_buf(str + trailer_block_start,
-					 end_of_log_message - trailer_block_start,
-					 '\n',
-					 0);
-	for (ptr = trailer_lines; *ptr; ptr++) {
-		if (last && isspace((*ptr)->buf[0])) {
-			struct strbuf sb = STRBUF_INIT;
-			strbuf_attach(&sb, *last, strlen(*last), strlen(*last));
-			strbuf_addbuf(&sb, *ptr);
-			*last = strbuf_detach(&sb, NULL);
-			continue;
-		}
-		ALLOC_GROW(trailer_strings, nr + 1, alloc);
-		trailer_strings[nr] = strbuf_detach(*ptr, NULL);
-		last = find_separator(trailer_strings[nr], separators) >= 1
-			? &trailer_strings[nr]
-			: NULL;
-		nr++;
-	}
-	strbuf_list_free(trailer_lines);
+size_t trailer_block_end(struct trailer_info *info)
+{
+	return info->trailer_block_end;
+}
 
-	info->blank_line_before_trailer = ends_with_blank_line(str,
-							       trailer_block_start);
-	info->trailer_block_start = trailer_block_start;
-	info->trailer_block_end = end_of_log_message;
-	info->trailers = trailer_strings;
-	info->trailer_nr = nr;
+int blank_line_before_trailer_block(struct trailer_info *info)
+{
+	return info->blank_line_before_trailer;
 }
 
 void trailer_info_release(struct trailer_info *info)
@@ -1122,6 +1169,7 @@ void trailer_info_release(struct trailer_info *info)
 	for (i = 0; i < info->trailer_nr; i++)
 		free(info->trailers[i]);
 	free(info->trailers);
+	free(info);
 }
 
 void format_trailers_from_commit(const struct process_trailer_options *opts,
@@ -1129,30 +1177,30 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 				 struct strbuf *out)
 {
 	LIST_HEAD(head);
-	struct trailer_info info;
-
-	parse_trailers(opts, &info, msg, &head);
+	struct trailer_info *info = parse_trailers(opts, msg, &head);
 
 	/* If we want the whole block untouched, we can take the fast path. */
 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
 	    !opts->separator && !opts->key_only && !opts->value_only &&
 	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info.trailer_block_start,
-			   info.trailer_block_end - info.trailer_block_start);
+		strbuf_add(out, msg + info->trailer_block_start,
+			   info->trailer_block_end - info->trailer_block_start);
 	} else
 		format_trailers(opts, &head, out);
 
 	free_trailers(&head);
-	trailer_info_release(&info);
+	trailer_info_release(info);
 }
 
 void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
+	struct trailer_info *internal = trailer_info_new();
 	strbuf_init(&iter->key, 0);
 	strbuf_init(&iter->val, 0);
 	opts.no_divider = 1;
-	trailer_info_get(&iter->internal.info, msg, &opts);
+	iter->internal.info = internal;
+	iter->internal.info = trailer_info_get(&opts, msg);
 	iter->internal.cur = 0;
 }
 
@@ -1160,8 +1208,8 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 {
 	char *line;
 	int separator_pos;
-	if (iter->internal.cur < iter->internal.info.trailer_nr) {
-		line = iter->internal.info.trailers[iter->internal.cur++];
+	if (iter->internal.cur < iter->internal.info->trailer_nr) {
+		line = iter->internal.info->trailers[iter->internal.cur++];
 		separator_pos = find_separator(line, separators);
 		iter->is_trailer = (separator_pos > 0);
 
@@ -1178,7 +1226,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 
 void trailer_iterator_release(struct trailer_iterator *iter)
 {
-	trailer_info_release(&iter->internal.info);
+	trailer_info_release(iter->internal.info);
 	strbuf_release(&iter->val);
 	strbuf_release(&iter->key);
 }
diff --git a/trailer.h b/trailer.h
index a47f18b1525..fff16a29a55 100644
--- a/trailer.h
+++ b/trailer.h
@@ -4,6 +4,8 @@
 #include "list.h"
 #include "strbuf.h"
 
+struct trailer_info;
+
 enum trailer_where {
 	WHERE_DEFAULT,
 	WHERE_END,
@@ -29,27 +31,6 @@ int trailer_set_where(enum trailer_where *item, const char *value);
 int trailer_set_if_exists(enum trailer_if_exists *item, const char *value);
 int trailer_set_if_missing(enum trailer_if_missing *item, const char *value);
 
-struct trailer_info {
-	/*
-	 * True if there is a blank line before the location pointed to by
-	 * trailer_block_start.
-	 */
-	int blank_line_before_trailer;
-
-	/*
-	 * Offsets to the trailer block start and end positions in the input
-	 * string. If no trailer block is found, these are both set to the
-	 * "true" end of the input (find_end_of_log_message()).
-	 */
-	size_t trailer_block_start, trailer_block_end;
-
-	/*
-	 * Array of trailers found.
-	 */
-	char **trailers;
-	size_t trailer_nr;
-};
-
 /*
  * A list that represents newly-added trailers, such as those provided
  * with the --trailer command line option of git-interpret-trailers.
@@ -89,13 +70,13 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
-void parse_trailers(const struct process_trailer_options *opts,
-		    struct trailer_info *info,
-		    const char *str,
-		    struct list_head *head);
+struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
+				    const char *str,
+				    struct list_head *head);
 
-void trailer_info_get(struct trailer_info *info, const char *str,
-		      const struct process_trailer_options *opts);
+size_t trailer_block_start(struct trailer_info *info);
+size_t trailer_block_end(struct trailer_info *info);
+int blank_line_before_trailer_block(struct trailer_info *info);
 
 void trailer_info_release(struct trailer_info *info);
 
@@ -142,7 +123,7 @@ struct trailer_iterator {
 
 	/* private */
 	struct {
-		struct trailer_info info;
+		struct trailer_info *info;
 		size_t cur;
 	} internal;
 };
-- 
gitgitgadget

[PATCH v2 07/10] trailer: spread usage of "trailer_block" language

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Deprecate the "trailer_info" struct name and replace it with
"trailer_block". The main reason is to help readability, because
"trailer_info" on the surface sounds like it's about a single trailer
when in reality it is a collection of contiguous lines, at least 25% of
which are trailers.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c | 26 +++++-----
 trailer.c                    | 99 ++++++++++++++++++------------------
 trailer.h                    | 18 +++----
 3 files changed, 71 insertions(+), 72 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 9e6ed6b65e2..9e41fa20b5f 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -140,8 +140,8 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
-	struct strbuf trailer_block = STRBUF_INIT;
-	struct trailer_info *info;
+	struct strbuf tb = STRBUF_INIT;
+	struct trailer_block *trailer_block;
 	FILE *outfile = stdout;
 
 	trailer_config_init();
@@ -151,13 +151,13 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	if (opts->in_place)
 		outfile = create_in_place_tempfile(file);
 
-	info = parse_trailers(opts, sb.buf, &head);
+	trailer_block = parse_trailers(opts, sb.buf, &head);
 
-	/* Print the lines before the trailers */
+	/* Print the lines before the trailer block */
 	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, trailer_block_start(info), outfile);
+		fwrite(sb.buf, 1, trailer_block_start(trailer_block), outfile);
 
-	if (!opts->only_trailers && !blank_line_before_trailer_block(info))
+	if (!opts->only_trailers && !blank_line_before_trailer_block(trailer_block))
 		fprintf(outfile, "\n");
 
 
@@ -171,17 +171,17 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	}
 
 	/* Print trailer block. */
-	format_trailers(opts, &head, &trailer_block);
-	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
-	strbuf_release(&trailer_block);
+	format_trailers(opts, &head, &tb);
+	fwrite(tb.buf, 1, tb.len, outfile);
+	strbuf_release(&tb);
 
 	free_trailers(&head);
 
-	/* Print the lines after the trailers as is */
+	/* Print the lines after the trailer block as is */
 	if (!opts->only_trailers)
-		fwrite(sb.buf + trailer_block_end(info), 1, sb.len - trailer_block_end(info), outfile);
-
-	trailer_info_release(info);
+		fwrite(sb.buf + trailer_block_end(trailer_block),
+		       1, sb.len - trailer_block_end(trailer_block), outfile);
+	trailer_block_release(trailer_block);
 
 	if (opts->in_place)
 		if (rename_tempfile(&trailers_tempfile, file))
diff --git a/trailer.c b/trailer.c
index 63774cd068d..e2a48bea0ae 100644
--- a/trailer.c
+++ b/trailer.c
@@ -11,19 +11,20 @@
  * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
  */
 
-struct trailer_info {
+struct trailer_block {
 	/*
 	 * True if there is a blank line before the location pointed to by
-	 * trailer_block_start.
+	 * "start".
 	 */
 	int blank_line_before_trailer;
 
 	/*
-	 * Offsets to the trailer block start and end positions in the input
-	 * string. If no trailer block is found, these are both set to the
-	 * "true" end of the input (find_end_of_log_message()).
+	 * The locations of the start and end positions of the trailer block
+	 * found, as offsets from the beginning of the source text from which
+	 * this trailer block was parsed. If no trailer block is found, these
+	 * are both set to 0.
 	 */
-	size_t trailer_block_start, trailer_block_end;
+	size_t start, end;
 
 	/*
 	 * Array of trailers found.
@@ -1046,16 +1047,16 @@ void format_trailers(const struct process_trailer_options *opts,
 	}
 }
 
-static struct trailer_info *trailer_info_new(void)
+static struct trailer_block *trailer_block_new(void)
 {
-	struct trailer_info *info = xcalloc(1, sizeof(*info));
-	return info;
+	struct trailer_block *trailer_block = xcalloc(1, sizeof(*trailer_block));
+	return trailer_block;
 }
 
-static struct trailer_info *trailer_info_get(const struct process_trailer_options *opts,
-					     const char *str)
+static struct trailer_block *trailer_block_get(const struct process_trailer_options *opts,
+					       const char *str)
 {
-	struct trailer_info *info = trailer_info_new();
+	struct trailer_block *trailer_block = trailer_block_new();
 	size_t end_of_log_message = 0, trailer_block_start = 0;
 	struct strbuf **trailer_lines, **ptr;
 	char **trailer_strings = NULL;
@@ -1088,34 +1089,34 @@ static struct trailer_info *trailer_info_get(const struct process_trailer_option
 	}
 	strbuf_list_free(trailer_lines);
 
-	info->blank_line_before_trailer = ends_with_blank_line(str,
-							       trailer_block_start);
-	info->trailer_block_start = trailer_block_start;
-	info->trailer_block_end = end_of_log_message;
-	info->trailers = trailer_strings;
-	info->trailer_nr = nr;
+	trailer_block->blank_line_before_trailer = ends_with_blank_line(str,
+									trailer_block_start);
+	trailer_block->start = trailer_block_start;
+	trailer_block->end = end_of_log_message;
+	trailer_block->trailers = trailer_strings;
+	trailer_block->trailer_nr = nr;
 
-	return info;
+	return trailer_block;
 }
 
 /*
- * Parse trailers in "str", populating the trailer info and "head"
- * linked list structure.
+ * Parse trailers in "str", populating the trailer_block info and "head" linked
+ * list structure.
  */
-struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
-				    const char *str,
-				    struct list_head *head)
+struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
+				     const char *str,
+				     struct list_head *head)
 {
-	struct trailer_info *info;
+	struct trailer_block *trailer_block;
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
 	size_t i;
 
-	info = trailer_info_get(opts, str);
+	trailer_block = trailer_block_get(opts, str);
 
-	for (i = 0; i < info->trailer_nr; i++) {
+	for (i = 0; i < trailer_block->trailer_nr; i++) {
 		int separator_pos;
-		char *trailer = info->trailers[i];
+		char *trailer = trailer_block->trailers[i];
 		if (trailer[0] == comment_line_char)
 			continue;
 		separator_pos = find_separator(trailer, separators);
@@ -1136,7 +1137,7 @@ struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
 		}
 	}
 
-	return info;
+	return trailer_block;
 }
 
 void free_trailers(struct list_head *trailers)
@@ -1148,28 +1149,28 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
-size_t trailer_block_start(struct trailer_info *info)
+size_t trailer_block_start(struct trailer_block *trailer_block)
 {
-	return info->trailer_block_start;
+	return trailer_block->start;
 }
 
-size_t trailer_block_end(struct trailer_info *info)
+size_t trailer_block_end(struct trailer_block *trailer_block)
 {
-	return info->trailer_block_end;
+	return trailer_block->end;
 }
 
-int blank_line_before_trailer_block(struct trailer_info *info)
+int blank_line_before_trailer_block(struct trailer_block *trailer_block)
 {
-	return info->blank_line_before_trailer;
+	return trailer_block->blank_line_before_trailer;
 }
 
-void trailer_info_release(struct trailer_info *info)
+void trailer_block_release(struct trailer_block *trailer_block)
 {
 	size_t i;
-	for (i = 0; i < info->trailer_nr; i++)
-		free(info->trailers[i]);
-	free(info->trailers);
-	free(info);
+	for (i = 0; i < trailer_block->trailer_nr; i++)
+		free(trailer_block->trailers[i]);
+	free(trailer_block->trailers);
+	free(trailer_block);
 }
 
 void format_trailers_from_commit(const struct process_trailer_options *opts,
@@ -1177,30 +1178,28 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 				 struct strbuf *out)
 {
 	LIST_HEAD(head);
-	struct trailer_info *info = parse_trailers(opts, msg, &head);
+	struct trailer_block *trailer_block = parse_trailers(opts, msg, &head);
 
 	/* If we want the whole block untouched, we can take the fast path. */
 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
 	    !opts->separator && !opts->key_only && !opts->value_only &&
 	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info->trailer_block_start,
-			   info->trailer_block_end - info->trailer_block_start);
+		strbuf_add(out, msg + trailer_block->start,
+			   trailer_block->end - trailer_block->start);
 	} else
 		format_trailers(opts, &head, out);
 
 	free_trailers(&head);
-	trailer_info_release(info);
+	trailer_block_release(trailer_block);
 }
 
 void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
-	struct trailer_info *internal = trailer_info_new();
 	strbuf_init(&iter->key, 0);
 	strbuf_init(&iter->val, 0);
 	opts.no_divider = 1;
-	iter->internal.info = internal;
-	iter->internal.info = trailer_info_get(&opts, msg);
+	iter->internal.trailer_block = trailer_block_get(&opts, msg);
 	iter->internal.cur = 0;
 }
 
@@ -1208,8 +1207,8 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 {
 	char *line;
 	int separator_pos;
-	if (iter->internal.cur < iter->internal.info->trailer_nr) {
-		line = iter->internal.info->trailers[iter->internal.cur++];
+	if (iter->internal.cur < iter->internal.trailer_block->trailer_nr) {
+		line = iter->internal.trailer_block->trailers[iter->internal.cur++];
 		separator_pos = find_separator(line, separators);
 		iter->is_trailer = (separator_pos > 0);
 
@@ -1226,7 +1225,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 
 void trailer_iterator_release(struct trailer_iterator *iter)
 {
-	trailer_info_release(iter->internal.info);
+	trailer_block_release(iter->internal.trailer_block);
 	strbuf_release(&iter->val);
 	strbuf_release(&iter->key);
 }
diff --git a/trailer.h b/trailer.h
index fff16a29a55..be14dd17185 100644
--- a/trailer.h
+++ b/trailer.h
@@ -4,7 +4,7 @@
 #include "list.h"
 #include "strbuf.h"
 
-struct trailer_info;
+struct trailer_block;
 
 enum trailer_where {
 	WHERE_DEFAULT,
@@ -70,15 +70,15 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
-struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
-				    const char *str,
-				    struct list_head *head);
+struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
+				     const char *str,
+				     struct list_head *head);
 
-size_t trailer_block_start(struct trailer_info *info);
-size_t trailer_block_end(struct trailer_info *info);
-int blank_line_before_trailer_block(struct trailer_info *info);
+size_t trailer_block_start(struct trailer_block *trailer_block);
+size_t trailer_block_end(struct trailer_block *trailer_block);
+int blank_line_before_trailer_block(struct trailer_block *trailer_block);
 
-void trailer_info_release(struct trailer_info *info);
+void trailer_block_release(struct trailer_block *trailer_block);
 
 void trailer_config_init(void);
 void format_trailers(const struct process_trailer_options *opts,
@@ -123,7 +123,7 @@ struct trailer_iterator {
 
 	/* private */
 	struct {
-		struct trailer_info *info;
+		struct trailer_block *trailer_block;
 		size_t cur;
 	} internal;
 };
-- 
gitgitgadget

[PATCH v2 08/10] trailer: prepare to move parse_trailers_from_command_line_args() to builtin

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Expose more functions in the trailer.h API, in preparation for moving
out

    parse_trailers_from_command_line_args()

to interpret-trailer.c, because the trailer API should not be concerned
with command line arguments (as they have nothing to do with trailers
themselves). The interpret-trailers builtin is the only caller of this
function.

Also rename "conf_info" to "trailer_conf" for readability, dropping the
low-value "_info" suffix as we did earlier in this series for
"trailer_info" to "trailer_block".

Helped-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 66 +++++++++++++++++++++++++++----------------------------
 trailer.h | 10 +++++++++
 2 files changed, 42 insertions(+), 34 deletions(-)

diff --git a/trailer.c b/trailer.c
index e2a48bea0ae..d362b9a604e 100644
--- a/trailer.c
+++ b/trailer.c
@@ -33,7 +33,7 @@ struct trailer_block {
 	size_t trailer_nr;
 };
 
-struct conf_info {
+struct trailer_conf {
 	char *name;
 	char *key;
 	char *command;
@@ -43,7 +43,7 @@ struct conf_info {
 	enum trailer_if_missing if_missing;
 };
 
-static struct conf_info default_conf_info;
+static struct trailer_conf default_trailer_conf;
 
 struct trailer_item {
 	struct list_head list;
@@ -59,7 +59,7 @@ struct arg_item {
 	struct list_head list;
 	char *token;
 	char *value;
-	struct conf_info conf;
+	struct trailer_conf conf;
 };
 
 static LIST_HEAD(conf_head);
@@ -210,7 +210,7 @@ static int check_if_different(struct trailer_item *in_tok,
 	return 1;
 }
 
-static char *apply_command(struct conf_info *conf, const char *arg)
+static char *apply_command(struct trailer_conf *conf, const char *arg)
 {
 	struct strbuf cmd = STRBUF_INIT;
 	struct strbuf buf = STRBUF_INIT;
@@ -424,7 +424,8 @@ int trailer_set_if_missing(enum trailer_if_missing *item, const char *value)
 	return 0;
 }
 
-static void duplicate_conf(struct conf_info *dst, const struct conf_info *src)
+void duplicate_trailer_conf(struct trailer_conf *dst,
+			    const struct trailer_conf *src)
 {
 	*dst = *src;
 	dst->name = xstrdup_or_null(src->name);
@@ -447,7 +448,7 @@ static struct arg_item *get_conf_item(const char *name)
 
 	/* Item does not already exists, create it */
 	CALLOC_ARRAY(item, 1);
-	duplicate_conf(&item->conf, &default_conf_info);
+	duplicate_trailer_conf(&item->conf, &default_trailer_conf);
 	item->conf.name = xstrdup(name);
 
 	list_add_tail(&item->list, &conf_head);
@@ -482,17 +483,17 @@ static int git_trailer_default_config(const char *conf_key, const char *value,
 	variable_name = strrchr(trailer_item, '.');
 	if (!variable_name) {
 		if (!strcmp(trailer_item, "where")) {
-			if (trailer_set_where(&default_conf_info.where,
+			if (trailer_set_where(&default_trailer_conf.where,
 					      value) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
 		} else if (!strcmp(trailer_item, "ifexists")) {
-			if (trailer_set_if_exists(&default_conf_info.if_exists,
+			if (trailer_set_if_exists(&default_trailer_conf.if_exists,
 						  value) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
 		} else if (!strcmp(trailer_item, "ifmissing")) {
-			if (trailer_set_if_missing(&default_conf_info.if_missing,
+			if (trailer_set_if_missing(&default_trailer_conf.if_missing,
 						   value) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
@@ -511,7 +512,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 {
 	const char *trailer_item, *variable_name;
 	struct arg_item *item;
-	struct conf_info *conf;
+	struct trailer_conf *conf;
 	char *name = NULL;
 	enum trailer_info_type type;
 	int i;
@@ -585,9 +586,9 @@ void trailer_config_init(void)
 		return;
 
 	/* Default config must be setup first */
-	default_conf_info.where = WHERE_END;
-	default_conf_info.if_exists = EXISTS_ADD_IF_DIFFERENT_NEIGHBOR;
-	default_conf_info.if_missing = MISSING_ADD;
+	default_trailer_conf.where = WHERE_END;
+	default_trailer_conf.if_exists = EXISTS_ADD_IF_DIFFERENT_NEIGHBOR;
+	default_trailer_conf.if_missing = MISSING_ADD;
 	git_config(git_trailer_default_config, NULL);
 	git_config(git_trailer_config, NULL);
 	configured = 1;
@@ -620,7 +621,7 @@ static int token_matches_item(const char *tok, struct arg_item *item, size_t tok
  * distinguished from the non-well-formed-line case (in which this function
  * returns -1) because some callers of this function need such a distinction.
  */
-static ssize_t find_separator(const char *line, const char *separators)
+ssize_t find_separator(const char *line, const char *separators)
 {
 	int whitespace_found = 0;
 	const char *c;
@@ -645,28 +646,28 @@ static ssize_t find_separator(const char *line, const char *separators)
  *
  * If separator_pos is -1, interpret the whole trailer as a token.
  */
-static void parse_trailer(struct strbuf *tok, struct strbuf *val,
-			 const struct conf_info **conf, const char *trailer,
-			 ssize_t separator_pos)
+void parse_trailer(const char *line, ssize_t separator_pos,
+		   struct strbuf *tok, struct strbuf *val,
+		   const struct trailer_conf **conf)
 {
 	struct arg_item *item;
 	size_t tok_len;
 	struct list_head *pos;
 
 	if (separator_pos != -1) {
-		strbuf_add(tok, trailer, separator_pos);
+		strbuf_add(tok, line, separator_pos);
 		strbuf_trim(tok);
-		strbuf_addstr(val, trailer + separator_pos + 1);
+		strbuf_addstr(val, line + separator_pos + 1);
 		strbuf_trim(val);
 	} else {
-		strbuf_addstr(tok, trailer);
+		strbuf_addstr(tok, line);
 		strbuf_trim(tok);
 	}
 
 	/* Lookup if the token matches something in the config */
 	tok_len = token_len_without_separator(tok->buf, tok->len);
 	if (conf)
-		*conf = &default_conf_info;
+		*conf = &default_trailer_conf;
 	list_for_each(pos, &conf_head) {
 		item = list_entry(pos, struct arg_item, list);
 		if (token_matches_item(tok->buf, item, tok_len)) {
@@ -691,13 +692,13 @@ static struct trailer_item *add_trailer_item(struct list_head *head, char *tok,
 }
 
 static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
-			 const struct conf_info *conf,
+			 const struct trailer_conf *conf,
 			 const struct new_trailer_item *new_trailer_item)
 {
 	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
 	new_item->token = tok;
 	new_item->value = val;
-	duplicate_conf(&new_item->conf, conf);
+	duplicate_trailer_conf(&new_item->conf, conf);
 	if (new_trailer_item) {
 		if (new_trailer_item->where != WHERE_DEFAULT)
 			new_item->conf.where = new_trailer_item->where;
@@ -730,7 +731,7 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
-	const struct conf_info *conf;
+	const struct trailer_conf *conf;
 	struct list_head *pos;
 
 	/*
@@ -753,8 +754,7 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 			      (int) sb.len, sb.buf);
 			strbuf_release(&sb);
 		} else {
-			parse_trailer(&tok, &val, &conf, tr->text,
-				      separator_pos);
+			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
 			add_arg_item(arg_head,
 				     strbuf_detach(&tok, NULL),
 				     strbuf_detach(&val, NULL),
@@ -1116,20 +1116,19 @@ struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
 
 	for (i = 0; i < trailer_block->trailer_nr; i++) {
 		int separator_pos;
-		char *trailer = trailer_block->trailers[i];
-		if (trailer[0] == comment_line_char)
+		char *line = trailer_block->trailers[i];
+		if (line[0] == comment_line_char)
 			continue;
-		separator_pos = find_separator(trailer, separators);
+		separator_pos = find_separator(line, separators);
 		if (separator_pos >= 1) {
-			parse_trailer(&tok, &val, NULL, trailer,
-				      separator_pos);
+			parse_trailer(line, separator_pos, &tok, &val, NULL);
 			if (opts->unfold)
 				unfold_value(&val);
 			add_trailer_item(head,
 					 strbuf_detach(&tok, NULL),
 					 strbuf_detach(&val, NULL));
 		} else if (!opts->only_trailers) {
-			strbuf_addstr(&val, trailer);
+			strbuf_addstr(&val, line);
 			strbuf_strip_suffix(&val, "\n");
 			add_trailer_item(head,
 					 NULL,
@@ -1215,8 +1214,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 		iter->raw = line;
 		strbuf_reset(&iter->key);
 		strbuf_reset(&iter->val);
-		parse_trailer(&iter->key, &iter->val, NULL,
-			      line, separator_pos);
+		parse_trailer(line, separator_pos, &iter->key, &iter->val, NULL);
 		unfold_value(&iter->val);
 		return 1;
 	}
diff --git a/trailer.h b/trailer.h
index be14dd17185..c6aa96dd6be 100644
--- a/trailer.h
+++ b/trailer.h
@@ -5,6 +5,7 @@
 #include "strbuf.h"
 
 struct trailer_block;
+struct trailer_conf;
 
 enum trailer_where {
 	WHERE_DEFAULT,
@@ -45,6 +46,9 @@ struct new_trailer_item {
 	enum trailer_if_missing if_missing;
 };
 
+void duplicate_trailer_conf(struct trailer_conf *dst,
+			    const struct trailer_conf *src);
+
 struct process_trailer_options {
 	int in_place;
 	int trim_empty;
@@ -70,6 +74,12 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
+ssize_t find_separator(const char *line, const char *separators);
+
+void parse_trailer(const char *line, ssize_t separator_pos,
+		   struct strbuf *tok, struct strbuf *val,
+		   const struct trailer_conf **conf);
+
 struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
 				     const char *str,
 				     struct list_head *head);
-- 
gitgitgadget

[PATCH v2 09/10] trailer: move arg handling to interpret-trailers.c

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

We don't move the "arg_item" struct to interpret-trailers.c, because it
is now a struct that contains information about trailers that should be
injected into the input text's own trailers. We will rename this
language as such in a follow-up patch to keep the diff here small.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c | 88 ++++++++++++++++++++++--------------
 trailer.c                    | 63 +++++++++++++++++++-------
 trailer.h                    | 13 +++++-
 3 files changed, 113 insertions(+), 51 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 9e41fa20b5f..ad9b9a9f9ef 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -45,23 +45,17 @@ static int option_parse_if_missing(const struct option *opt,
 	return trailer_set_if_missing(opt->value, arg);
 }
 
-static void new_trailers_clear(struct list_head *trailers)
-{
-	struct list_head *pos, *tmp;
-	struct new_trailer_item *item;
-
-	list_for_each_safe(pos, tmp, trailers) {
-		item = list_entry(pos, struct new_trailer_item, list);
-		list_del(pos);
-		free(item);
-	}
-}
+static char *cl_separators;
 
 static int option_parse_trailer(const struct option *opt,
 				   const char *arg, int unset)
 {
 	struct list_head *trailers = opt->value;
-	struct new_trailer_item *item;
+	struct strbuf tok = STRBUF_INIT;
+	struct strbuf val = STRBUF_INIT;
+	const struct trailer_conf *conf;
+	struct trailer_conf *conf_current = new_trailer_conf();
+	ssize_t separator_pos;
 
 	if (unset) {
 		new_trailers_clear(trailers);
@@ -71,12 +65,31 @@ static int option_parse_trailer(const struct option *opt,
 	if (!arg)
 		return -1;
 
-	item = xmalloc(sizeof(*item));
-	item->text = arg;
-	item->where = where;
-	item->if_exists = if_exists;
-	item->if_missing = if_missing;
-	list_add_tail(&item->list, trailers);
+	separator_pos = find_separator(arg, cl_separators);
+	if (separator_pos) {
+		parse_trailer(arg, separator_pos, &tok, &val, &conf);
+		duplicate_trailer_conf(conf_current, conf);
+
+		/*
+		 * Override conf_current with settings specified via CLI flags.
+		 */
+		trailer_conf_set(where, if_exists, if_missing, conf_current);
+
+		add_arg_item(strbuf_detach(&tok, NULL),
+			     strbuf_detach(&val, NULL),
+			     conf_current,
+			     trailers);
+	} else {
+		struct strbuf sb = STRBUF_INIT;
+		strbuf_addstr(&sb, arg);
+		strbuf_trim(&sb);
+		error(_("empty trailer token in trailer '%.*s'"),
+			(int) sb.len, sb.buf);
+		strbuf_release(&sb);
+	}
+
+	free(conf_current);
+
 	return 0;
 }
 
@@ -135,7 +148,7 @@ static void read_input_file(struct strbuf *sb, const char *file)
 }
 
 static void interpret_trailers(const struct process_trailer_options *opts,
-			       struct list_head *new_trailer_head,
+			       struct list_head *arg_trailers,
 			       const char *file)
 {
 	LIST_HEAD(head);
@@ -144,8 +157,6 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	struct trailer_block *trailer_block;
 	FILE *outfile = stdout;
 
-	trailer_config_init();
-
 	read_input_file(&sb, file);
 
 	if (opts->in_place)
@@ -162,12 +173,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 
 
 	if (!opts->only_input) {
-		LIST_HEAD(config_head);
-		LIST_HEAD(arg_head);
-		parse_trailers_from_config(&config_head);
-		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
-		list_splice(&config_head, &arg_head);
-		process_trailers_lists(&head, &arg_head);
+		process_trailers_lists(&head, arg_trailers);
 	}
 
 	/* Print trailer block. */
@@ -193,7 +199,8 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
-	LIST_HEAD(trailers);
+	LIST_HEAD(configured_trailers);
+	LIST_HEAD(arg_trailers);
 
 	struct option options[] = {
 		OPT_BOOL(0, "in-place", &opts.in_place, N_("edit files in place")),
@@ -212,33 +219,48 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 		OPT_CALLBACK_F(0, "parse", &opts, NULL, N_("alias for --only-trailers --only-input --unfold"),
 			PARSE_OPT_NOARG | PARSE_OPT_NONEG, parse_opt_parse),
 		OPT_BOOL(0, "no-divider", &opts.no_divider, N_("do not treat \"---\" as the end of input")),
-		OPT_CALLBACK(0, "trailer", &trailers, N_("trailer"),
+		OPT_CALLBACK(0, "trailer", &arg_trailers, N_("trailer"),
 				N_("trailer(s) to add"), option_parse_trailer),
 		OPT_END()
 	};
 
 	git_config(git_default_config, NULL);
+	trailer_config_init();
+
+	if (!opts.only_input) {
+		parse_trailers_from_config(&configured_trailers);
+	}
+
+	/*
+	* In command-line arguments, '=' is accepted (in addition to the
+	* separators that are defined).
+	*/
+	cl_separators = xstrfmt("=%s", default_separators());
 
 	argc = parse_options(argc, argv, prefix, options,
 			     git_interpret_trailers_usage, 0);
 
-	if (opts.only_input && !list_empty(&trailers))
+	free(cl_separators);
+
+	if (opts.only_input && !list_empty(&arg_trailers))
 		usage_msg_opt(
 			_("--trailer with --only-input does not make sense"),
 			git_interpret_trailers_usage,
 			options);
 
+	list_splice(&configured_trailers, &arg_trailers);
+
 	if (argc) {
 		int i;
 		for (i = 0; i < argc; i++)
-			interpret_trailers(&opts, &trailers, argv[i]);
+			interpret_trailers(&opts, &arg_trailers, argv[i]);
 	} else {
 		if (opts.in_place)
 			die(_("no input file given for in-place editing"));
-		interpret_trailers(&opts, &trailers, NULL);
+		interpret_trailers(&opts, &arg_trailers, NULL);
 	}
 
-	new_trailers_clear(&trailers);
+	new_trailers_clear(&arg_trailers);
 
 	return 0;
 }
diff --git a/trailer.c b/trailer.c
index d362b9a604e..1865d04bdf2 100644
--- a/trailer.c
+++ b/trailer.c
@@ -66,6 +66,11 @@ static LIST_HEAD(conf_head);
 
 static char *separators = ":";
 
+const char *default_separators(void)
+{
+	return separators;
+}
+
 static int configured;
 
 #define TRAILER_ARG_STRING "$ARG"
@@ -424,6 +429,25 @@ int trailer_set_if_missing(enum trailer_if_missing *item, const char *value)
 	return 0;
 }
 
+void trailer_conf_set(enum trailer_where where,
+		      enum trailer_if_exists if_exists,
+		      enum trailer_if_missing if_missing,
+		      struct trailer_conf *conf)
+{
+	if (where != WHERE_DEFAULT)
+		conf->where = where;
+	if (if_exists != EXISTS_DEFAULT)
+		conf->if_exists = if_exists;
+	if (if_missing != MISSING_DEFAULT)
+		conf->if_missing = if_missing;
+}
+
+struct trailer_conf *new_trailer_conf(void)
+{
+	struct trailer_conf *new = xcalloc(1, sizeof(*new));
+	return new;
+}
+
 void duplicate_trailer_conf(struct trailer_conf *dst,
 			    const struct trailer_conf *src)
 {
@@ -642,6 +666,9 @@ ssize_t find_separator(const char *line, const char *separators)
 /*
  * Obtain the token, value, and conf from the given trailer.
  *
+ * The conf needs special handling. We first read hardcoded defaults, and
+ * override them if we find a matching trailer configuration in the config.
+ *
  * separator_pos must not be 0, since the token cannot be an empty string.
  *
  * If separator_pos is -1, interpret the whole trailer as a token.
@@ -691,22 +718,14 @@ static struct trailer_item *add_trailer_item(struct list_head *head, char *tok,
 	return new_item;
 }
 
-static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
-			 const struct trailer_conf *conf,
-			 const struct new_trailer_item *new_trailer_item)
+void add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
+		  struct list_head *arg_head)
+
 {
 	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
 	new_item->token = tok;
 	new_item->value = val;
 	duplicate_trailer_conf(&new_item->conf, conf);
-	if (new_trailer_item) {
-		if (new_trailer_item->where != WHERE_DEFAULT)
-			new_item->conf.where = new_trailer_item->where;
-		if (new_trailer_item->if_exists != EXISTS_DEFAULT)
-			new_item->conf.if_exists = new_trailer_item->if_exists;
-		if (new_trailer_item->if_missing != MISSING_DEFAULT)
-			new_item->conf.if_missing = new_trailer_item->if_missing;
-	}
 	list_add_tail(&new_item->list, arg_head);
 }
 
@@ -719,10 +738,10 @@ void parse_trailers_from_config(struct list_head *config_head)
 	list_for_each(pos, &conf_head) {
 		item = list_entry(pos, struct arg_item, list);
 		if (item->conf.command)
-			add_arg_item(config_head,
-				     xstrdup(token_from_item(item, NULL)),
+			add_arg_item(xstrdup(token_from_item(item, NULL)),
 				     xstrdup(""),
-				     &item->conf, NULL);
+				     &item->conf,
+				     config_head);
 	}
 }
 
@@ -755,10 +774,10 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 			strbuf_release(&sb);
 		} else {
 			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
-			add_arg_item(arg_head,
-				     strbuf_detach(&tok, NULL),
+			add_arg_item(strbuf_detach(&tok, NULL),
 				     strbuf_detach(&val, NULL),
-				     conf, tr);
+				     conf,
+				     arg_head);
 		}
 	}
 
@@ -1148,6 +1167,16 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
+void new_trailers_clear(struct list_head *trailers)
+{
+	struct list_head *pos, *p;
+
+	list_for_each_safe(pos, p, trailers) {
+		list_del(pos);
+		free_arg_item(list_entry(pos, struct arg_item, list));
+	}
+}
+
 size_t trailer_block_start(struct trailer_block *trailer_block)
 {
 	return trailer_block->start;
diff --git a/trailer.h b/trailer.h
index c6aa96dd6be..e01437160cf 100644
--- a/trailer.h
+++ b/trailer.h
@@ -46,9 +46,20 @@ struct new_trailer_item {
 	enum trailer_if_missing if_missing;
 };
 
+void trailer_conf_set(enum trailer_where where,
+		      enum trailer_if_exists if_exists,
+		      enum trailer_if_missing if_missing,
+		      struct trailer_conf *conf);
+
+struct trailer_conf *new_trailer_conf(void);
 void duplicate_trailer_conf(struct trailer_conf *dst,
 			    const struct trailer_conf *src);
 
+const char *default_separators(void);
+
+void add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
+		  struct list_head *arg_head);
+
 struct process_trailer_options {
 	int in_place;
 	int trim_empty;
@@ -95,7 +106,7 @@ void format_trailers(const struct process_trailer_options *opts,
 		     struct list_head *trailers,
 		     struct strbuf *out);
 void free_trailers(struct list_head *trailers);
-
+void new_trailers_clear(struct list_head *new_trailers);
 /*
  * Convenience function to format the trailers from the commit msg "msg" into
  * the strbuf "out". Reuses format_trailers internally.
-- 
gitgitgadget

[PATCH v2 10/10] trailer: delete obsolete argument handling code from API

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This commit was not squashed with its parent in order to keep the diff
separate (to make the additive changes in the parent easier to read).

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 39 ---------------------------------------
 trailer.h | 17 -----------------
 2 files changed, 56 deletions(-)

diff --git a/trailer.c b/trailer.c
index 1865d04bdf2..8016908039f 100644
--- a/trailer.c
+++ b/trailer.c
@@ -745,45 +745,6 @@ void parse_trailers_from_config(struct list_head *config_head)
 	}
 }
 
-void parse_trailers_from_command_line_args(struct list_head *arg_head,
-					   struct list_head *new_trailer_head)
-{
-	struct strbuf tok = STRBUF_INIT;
-	struct strbuf val = STRBUF_INIT;
-	const struct trailer_conf *conf;
-	struct list_head *pos;
-
-	/*
-	 * In command-line arguments, '=' is accepted (in addition to the
-	 * separators that are defined).
-	 */
-	char *cl_separators = xstrfmt("=%s", separators);
-
-	/* Add an arg item for each trailer on the command line */
-	list_for_each(pos, new_trailer_head) {
-		struct new_trailer_item *tr =
-			list_entry(pos, struct new_trailer_item, list);
-		ssize_t separator_pos = find_separator(tr->text, cl_separators);
-
-		if (separator_pos == 0) {
-			struct strbuf sb = STRBUF_INIT;
-			strbuf_addstr(&sb, tr->text);
-			strbuf_trim(&sb);
-			error(_("empty trailer token in trailer '%.*s'"),
-			      (int) sb.len, sb.buf);
-			strbuf_release(&sb);
-		} else {
-			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
-			add_arg_item(strbuf_detach(&tok, NULL),
-				     strbuf_detach(&val, NULL),
-				     conf,
-				     arg_head);
-		}
-	}
-
-	free(cl_separators);
-}
-
 static const char *next_line(const char *str)
 {
 	const char *nl = strchrnul(str, '\n');
diff --git a/trailer.h b/trailer.h
index e01437160cf..a71b770c564 100644
--- a/trailer.h
+++ b/trailer.h
@@ -32,20 +32,6 @@ int trailer_set_where(enum trailer_where *item, const char *value);
 int trailer_set_if_exists(enum trailer_if_exists *item, const char *value);
 int trailer_set_if_missing(enum trailer_if_missing *item, const char *value);
 
-/*
- * A list that represents newly-added trailers, such as those provided
- * with the --trailer command line option of git-interpret-trailers.
- */
-struct new_trailer_item {
-	struct list_head list;
-
-	const char *text;
-
-	enum trailer_where where;
-	enum trailer_if_exists if_exists;
-	enum trailer_if_missing if_missing;
-};
-
 void trailer_conf_set(enum trailer_where where,
 		      enum trailer_if_exists if_exists,
 		      enum trailer_if_missing if_missing,
@@ -79,9 +65,6 @@ struct process_trailer_options {
 
 void parse_trailers_from_config(struct list_head *config_head);
 
-void parse_trailers_from_command_line_args(struct list_head *arg_head,
-					   struct list_head *new_trailer_head);
-
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
-- 
gitgitgadget

Re: [PATCH v2 09/10] trailer: move arg handling to interpret-trailers.c

[Linus Arver]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> diff --git a/trailer.h b/trailer.h
> index c6aa96dd6be..e01437160cf 100644
> --- a/trailer.h
> +++ b/trailer.h
> @@ -46,9 +46,20 @@ struct new_trailer_item {
>  	enum trailer_if_missing if_missing;
>  };
>
> +void trailer_conf_set(enum trailer_where where,
> +		      enum trailer_if_exists if_exists,
> +		      enum trailer_if_missing if_missing,
> +		      struct trailer_conf *conf);
> +
> +struct trailer_conf *new_trailer_conf(void);
>  void duplicate_trailer_conf(struct trailer_conf *dst,
>  			    const struct trailer_conf *src);
>
> +const char *default_separators(void);
> +
> +void add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
> +		  struct list_head *arg_head);

Looks like I forgot to add a "trailer" keyword for these API functions.
Will rename to "trailer_default_separators()" and
"trailer_add_arg_item()" in next reroll.

Re: [PATCH v2 09/10] trailer: move arg handling to interpret-trailers.c

[Linus Arver]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> We don't move the "arg_item" struct to interpret-trailers.c, because it
> is now a struct that contains information about trailers that should be
> injected into the input text's own trailers. We will rename this
> language as such in a follow-up patch to keep the diff here small.

I forgot to add this follow-up patch in this series. That will be patch
11 in v3, which renames "arg_item" to "trailer_template" (the idea being
that trailer templates are used to create new trailers, using the
template as a guide).

> diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
> index 9e41fa20b5f..ad9b9a9f9ef 100644
> --- a/builtin/interpret-trailers.c
> +++ b/builtin/interpret-trailers.c
> @@ -45,23 +45,17 @@ static int option_parse_if_missing(const struct option *opt,
>  	return trailer_set_if_missing(opt->value, arg);
>  }
>  
> -static void new_trailers_clear(struct list_head *trailers)
> -{
> -	struct list_head *pos, *tmp;
> -	struct new_trailer_item *item;
> -
> -	list_for_each_safe(pos, tmp, trailers) {
> -		item = list_entry(pos, struct new_trailer_item, list);
> -		list_del(pos);
> -		free(item);
> -	}
> -}

This function will also get promoted to the API, and will be renamed to
"free_new_trailers()" initially in patch 9, but renamed to
"free_trailer_templates()" in patch 11 (we wait until patch 11 to
introduce the term "template").

Re: [PATCH v2 03/10] trailer: unify trailer formatting machinery

[Josh Steadmon]

On 2024.01.26 22:38, Linus Arver via GitGitGadget wrote:
> From: Linus Arver <linusa@google.com>
> 
> Currently have two functions for formatting trailers exposed in
> trailer.h:
> 
>     void format_trailers(FILE *outfile, struct list_head *head,
>                         const struct process_trailer_options *opts);
> 
>     void format_trailers_from_commit(struct strbuf *out, const char *msg,
>                                     const struct process_trailer_options *opts);
> 
> and previously these functions, although similar enough (even taking the
> same process_trailer_options struct pointer), did not build on each
> other.
> 
> Make format_trailers_from_commit() rely on format_trailers(). Teach
> format_trailers() to process trailers with the additional
> process_trailer_options fields like opts->key_only which is only used by
> format_trailers_from_commit() and not builtin/interpret-trailers.c.
> While we're at it, reorder parameters to put the trailer processing
> options first, and the out parameter (strbuf we write into) at the end.
> 
> This unification will allow us to delete the format_trailer_info() and
> print_tok_val() functions in the next patch. They are not deleted here
> in order to keep the diff small.

Unfortunately this breaks the build:

trailer.c:1145:13: error: ‘format_trailer_info’ defined but not used [-Werror=unused-function]

and

trailer.c:147:13: error: ‘print_tok_val’ defined but not used [-Werror=unused-function]

While separating this patch from the deletion does make it easier to
review, it may make bisection more difficult.

Re: [PATCH v2 01/10] trailer: prepare to expose functions as part of API

[Josh Steadmon]

On 2024.01.26 22:38, Linus Arver via GitGitGadget wrote:
> From: Linus Arver <linusa@google.com>
> 
> In the next patch, we will move "process_trailers" from trailer.c to
> builtin/interpret-trailers.c. That move will necessitate the growth of
> the trailer.h API, forcing us to expose some additional functions in
> trailer.h.
> 
> Rename relevant functions so that they include the term "trailer" in
> their name, so that clients of the API will be able to easily identify
> them by their "trailer" moniker, just like all the other functions
> already exposed by trailer.h.
> 
> The the opportunity to start putting trailer processions options (opts)

Nitpick: typo in the commit message.
s/The the opportunity/Take the opportunity/ ?

> as the first parameter. This will be the pattern going forward in this
> series.
> 
> Helped-by: Junio C Hamano <gitster@pobox.com>
> Signed-off-by: Linus Arver <linusa@google.com>
> ---
>  builtin/interpret-trailers.c |  4 ++--
>  trailer.c                    | 26 +++++++++++++-------------
>  trailer.h                    |  6 +++---
>  3 files changed, 18 insertions(+), 18 deletions(-)

Re: [PATCH v2 01/10] trailer: prepare to expose functions as part of API

[Linus Arver]

Josh Steadmon <steadmon@google.com> writes:

> On 2024.01.26 22:38, Linus Arver via GitGitGadget wrote:
>> From: Linus Arver <linusa@google.com>
>> 
>> In the next patch, we will move "process_trailers" from trailer.c to
>> builtin/interpret-trailers.c. That move will necessitate the growth of
>> the trailer.h API, forcing us to expose some additional functions in
>> trailer.h.
>> 
>> Rename relevant functions so that they include the term "trailer" in
>> their name, so that clients of the API will be able to easily identify
>> them by their "trailer" moniker, just like all the other functions
>> already exposed by trailer.h.
>> 
>> The the opportunity to start putting trailer processions options (opts)
>
> Nitpick: typo in the commit message.
> s/The the opportunity/Take the opportunity/ ?

Ack, will update in reroll.

Re: [PATCH v2 03/10] trailer: unify trailer formatting machinery

[Linus Arver]

Josh Steadmon <steadmon@google.com> writes:

> On 2024.01.26 22:38, Linus Arver via GitGitGadget wrote:
>> From: Linus Arver <linusa@google.com>
>> 
>> Currently have two functions for formatting trailers exposed in
>> trailer.h:
>> 
>>     void format_trailers(FILE *outfile, struct list_head *head,
>>                         const struct process_trailer_options *opts);
>> 
>>     void format_trailers_from_commit(struct strbuf *out, const char *msg,
>>                                     const struct process_trailer_options *opts);
>> 
>> and previously these functions, although similar enough (even taking the
>> same process_trailer_options struct pointer), did not build on each
>> other.
>> 
>> Make format_trailers_from_commit() rely on format_trailers(). Teach
>> format_trailers() to process trailers with the additional
>> process_trailer_options fields like opts->key_only which is only used by
>> format_trailers_from_commit() and not builtin/interpret-trailers.c.
>> While we're at it, reorder parameters to put the trailer processing
>> options first, and the out parameter (strbuf we write into) at the end.
>> 
>> This unification will allow us to delete the format_trailer_info() and
>> print_tok_val() functions in the next patch. They are not deleted here
>> in order to keep the diff small.
>
> Unfortunately this breaks the build:
>
> trailer.c:1145:13: error: ‘format_trailer_info’ defined but not used [-Werror=unused-function]
>
> and
>
> trailer.c:147:13: error: ‘print_tok_val’ defined but not used [-Werror=unused-function]
>
> While separating this patch from the deletion does make it easier to
> review, it may make bisection more difficult.

FTR I've tried a preview version of squashing the deletion into this
patch with "/preview" in GitGitGadget, and it generated a clean-enough
diff where the deletions weren't intermixed with additions (maybe it
uses the patience diff algorithm). But I didn't squash them for v2
because I was concerned about the range diff becoming even more
difficult to read for reviewers.

I'm OK with squashing them for v3, but I'm also not sure that's
necessary. For example, during bisection you could use DEVOPTS=no-error
(or similar) in config.mak to skip over harmless errors like
"unused-function". Personally I'd prefer to keep the patches separate
because they started separately on the list.

Ultimately, I don't have a strong opinion on this. Maybe Junio or
someone else can cast the tie-breaking vote? To squash or not to squash?
I will take lazy consensus to mean "squash" for v3 if no one has
objections. Thanks.

[PATCH v3 00/10] Enrich Trailer API

[Linus Arver via GitGitGadget]

This patch series is the first 10 patches of a larger cleanup/bugfix series
(henceforth "larger series") I've been working on. The main goal of this
series is to begin the process of "libifying" the trailer API. By "API" I
mean the interface exposed in trailer.h. The larger series brings a number
of additional cleanups (exposing and fixing some bugs along the way), and
builds on top of this series.

When the larger series is merged, we will be in a good state to additionally
pursue the following goals:

 1. "API reuse inside Git": make the API expressive enough to eliminate any
    need by other parts of Git to use the interpret-trailers builtin as a
    subprocess (instead they could just use the API directly);
 2. "API stability": add unit tests to codify the expected behavior of API
    functions; and
 3. "API documentation": create developer-focused documentation to explain
    how to use the API effectively, noting any API limitations or
    anti-patterns.

The reason why the larger series itself doesn't tackle these goals directly
is because I believe that API code should be thought from the ground up with
a libification-focused perspective. Some structs and functions exposed in
the API today should probably not be libified (read: kept in trailer.h) as
is. For example, the "trailer_iterator" struct has a "private" member and it
feels wrong to allow API users to peek inside here (and take at face value
our future API users' pinky promise that they won't depend on those private
internals not meant for public consumption).

One pattern we could use here to cleanly separate "what is the API"
(publicly exposed) and "what is the implementation" (private) is the
pointer-to-implementation ("pimpl") idiom. There may be other appropriate
patterns, but I've chosen this one because it's a simple, low-level concept
(put structs in foo.c instead of foo.h), which has far-reaching high-level
consequences (API users must rely exclusively on the API to make use of such
private structs, via opaque pointers). The pimpl idiom for C comes from the
book "C Interfaces and Implementations" (see patch "trailer: make
trailer_info struct private").

The idea of turning a public struct into a private one is a fundamental
question of libification because it forces us to reconsider all of the data
structures we have and how they're actually used by already existing users.
For the trailer API, those existing users are the "interpret-trailers"
builtin command, and anything else that includes the "trailer.h" header file
(e.g., sequencer.c). One advantage of this idiom is that even the compiler
understands it --- the compiler will loudly complain if you try to access
the innards of a private struct through an opaque pointer.

Another advantage of this idiom is that it helps to reduce the probability
of breaking changes in the API. Because a private struct's members are out
of view from our users (they only know about opaque pointers to the private
struct, not its members), we are free to modify the members of the struct at
any time, as much as we like, as long as we don't break the semantics of the
exposed API functions (which is why unit-testing these API functions will be
crucial long-term).

If this pimpl idiom turns out to be a mistake, undoing it is easy --- just
move the relevant struct definition from foo.c to the header file. So it's a
great way to try things out without digging ourselves into a pit of despair
that will be difficult to get out of.

With the libification-focused goals out of the way, let's turn to this patch
series in more detail.

Currently, we have "process_trailers()" in trailer.h which does many
different things (parse command-line arguments, create temporary files, etc)
that are independent of the concept of "trailers". Keeping this function as
an API function would make unit-testing it difficult. While there is no
technical reason why we couldn't write unit tests for the smaller functions
that are called within process_trailers(), doing so would involve testing
private ("static" in trailer.c) functions instead of API functions, which
defeats the goal of "API stability" mentioned earlier above.

As an alternative to how things are done in this patch series, we could keep
trailer.h intact and decide to unit-test the existing "trailer_info_get()"
function which does most of the trailer parsing work (and is used by
sequencer.c). However this function wouldn't be easy to test either, because
the resulting "trailer_info" struct merely contains the unparsed "trailers"
lines. So the unit test (if it wants to inspect the result of parsing these
lines) would have to invoke additional parsing functions itself. And at that
point it would no longer be a unit test in the traditional sense, because it
would be invoking multiple functions at once.

In summary this series breaks up "process_trailers()" into smaller pieces,
exposing many of the parts relevant to trailer-related processing in
trailer.h. This will force us to eventually introduce unit tests for these
API functions, but that is a good thing for API stability.

In the future after libification is "complete", users external to Git will
be able to use the same trailer processing API used by the
interpret-trailers builtin. For example, a web server may want to parse
trailers the same way that Git would parse them, without having to call
interpret-trailers as a subprocess. This use case was the original
motivation behind my work in this area.

Thanks to the aggressive refactoring in this series, I've been able to
identify and fix several bugs in our existing implementation. Those fixes
build on top of this series but were not included here, in order to keep
this series small. Below is a "shortlog" of those fixes I have locally:

 * "trailer: trailer replacement should not change its position" (If we
   found a trailer we'd like to replace, preserve its position relative to
   the other trailers found in the trailer block, instead of always moving
   it to the beginning or end of the entire trailer block.)
 * "interpret-trailers: preserve trailers coming from the input" (Sometimes,
   the parsed trailers from the input will be formatted differently
   depending on whether we provide --only-trailers or not. Make the trailers
   that were not modified and which are coming directly from the input get
   formatted the same way, regardless of this flag.)
 * "interpret-trailers: do not modify the input if NOP" (Refrain from
   subtracting or adding a newline around the patch divider "---" if we are
   not adding new trailers.)
 * "trailer formatter: split up format_trailer() monolith" (Fix a bug in
   git-log where we still printed a blank newline even if we didn't want to
   format anything.)
 * "interpret-trailers: fail if given unrecognized arguments" (E.g., for
   "--where", only accept recognized WHERE_* enum values. If we get
   something unrecognized, fail with an error instead of silently doing
   nothing. Ditto for "--if-exists" and "--if-missing".)


Notable changes in v3
=====================

 * Squashed Patch 4 into Patch 3 ("trailer: unify trailer formatting
   machinery"), to avoid breaking the build ("-Werror=unused-function"
   violations)
 * NEW (Patch 10): Introduce "trailer template" terminology for readability
   (no behavioral change)
 * (API function) Rename default_separators() to
   trailer_default_separators()
 * (API function) Rename new_trailers_clear() to free_trailer_templates()
 * trailer.h: for single-parameter functions, anonymize the parameter name
   to reduce verbosity


Notable changes in v2
=====================

 * (cover letter) Discuss goals of the larger series in more detail,
   especially the pimpl idiom
 * (cover letter) List bug fixes pending in the larger series that depend on
   this series
 * Reorder function parameters to have trailer options at the beginning (and
   out parameters toward the end)
 * "sequencer: use the trailer iterator": prefer C string instead of strbuf
   for new "raw" field
 * Patch 1 (was Patch 2) also renames ensure_configured() to
   trailer_config_init() (forgot to rename this one previously)

Linus Arver (10):
  trailer: prepare to expose functions as part of API
  trailer: move interpret_trailers() to interpret-trailers.c
  trailer: unify trailer formatting machinery
  sequencer: use the trailer iterator
  trailer: make trailer_info struct private
  trailer: spread usage of "trailer_block" language
  trailer: prepare to move parse_trailers_from_command_line_args() to
    builtin
  trailer: move arg handling to interpret-trailers.c
  trailer: delete obsolete argument handling code from API
  trailer: introduce "template" term for readability

 builtin/interpret-trailers.c | 183 ++++++--
 builtin/shortlog.c           |   7 +-
 pretty.c                     |   2 +-
 ref-filter.c                 |   2 +-
 sequencer.c                  |  35 +-
 trailer.c                    | 835 +++++++++++++++++------------------
 trailer.h                    | 105 +++--
 7 files changed, 631 insertions(+), 538 deletions(-)


base-commit: a54a84b333adbecf7bc4483c0e36ed5878cac17b
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1632%2Flistx%2Ftrailer-api-refactor-part-1-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1632/listx/trailer-api-refactor-part-1-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/1632

Range-diff vs v2:

  1:  e2d3ed9b5b6 !  1:  72cc12a3066 trailer: prepare to expose functions as part of API
     @@ Commit message
          them by their "trailer" moniker, just like all the other functions
          already exposed by trailer.h.
      
     -    The the opportunity to start putting trailer processions options (opts)
     +    Take the opportunity to start putting trailer processions options (opts)
          as the first parameter. This will be the pattern going forward in this
          series.
      
  2:  eaca39fd7ea !  2:  cafa39d1048 trailer: move interpret_trailers() to interpret-trailers.c
     @@ trailer.h: struct process_trailer_options {
      +void trailer_config_init(void);
      +void format_trailers(const struct process_trailer_options *opts,
      +		     struct list_head *trailers, FILE *outfile);
     -+void free_trailers(struct list_head *trailers);
     ++void free_trailers(struct list_head *);
      +
       /*
        * Format the trailers from the commit msg "msg" into the strbuf "out".
  3:  9b7747d550e !  3:  5c7a2354df0 trailer: unify trailer formatting machinery
     @@ ref-filter.c: static void grab_sub_body_contents(struct atom_value *val, int der
       		} else if (atom->u.contents.option == C_BARE)
      
       ## trailer.c ##
     -@@ trailer.c: static void print_tok_val(FILE *outfile, const char *tok, const char *val)
     - 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
     +@@ trailer.c: static char last_non_space_char(const char *s)
     + 	return '\0';
       }
       
     +-static void print_tok_val(FILE *outfile, const char *tok, const char *val)
     +-{
     +-	char c;
     +-
     +-	if (!tok) {
     +-		fprintf(outfile, "%s\n", val);
     +-		return;
     +-	}
     +-
     +-	c = last_non_space_char(tok);
     +-	if (!c)
     +-		return;
     +-	if (strchr(separators, c))
     +-		fprintf(outfile, "%s%s\n", tok, val);
     +-	else
     +-		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
     +-}
     +-
      -void format_trailers(const struct process_trailer_options *opts,
      -		     struct list_head *trailers, FILE *outfile)
      -{
     @@ trailer.c: static void unfold_value(struct strbuf *val)
       /*
        * Parse trailers in "str", populating the trailer info and "head"
        * linked list structure.
     -@@ trailer.c: static void format_trailer_info(struct strbuf *out,
     - 
     +@@ trailer.c: void trailer_info_release(struct trailer_info *info)
     + 	free(info->trailers);
       }
       
     --void format_trailers_from_commit(struct strbuf *out, const char *msg,
     --				 const struct process_trailer_options *opts)
     +-static void format_trailer_info(struct strbuf *out,
     +-				const struct trailer_info *info,
     +-				const char *msg,
     +-				const struct process_trailer_options *opts)
      +void format_trailers_from_commit(const struct process_trailer_options *opts,
      +				 const char *msg,
      +				 struct strbuf *out)
       {
     +-	size_t origlen = out->len;
     +-	size_t i;
      +	LIST_HEAD(head);
     - 	struct trailer_info info;
     - 
     --	trailer_info_get(&info, msg, opts);
     --	format_trailer_info(out, &info, msg, opts);
     -+	parse_trailers(opts, &info, msg, &head);
     ++	struct trailer_info info;
      +
     -+	/* If we want the whole block untouched, we can take the fast path. */
     -+	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
     -+	    !opts->separator && !opts->key_only && !opts->value_only &&
     -+	    !opts->key_value_separator) {
     ++	parse_trailers(opts, &info, msg, &head);
     + 
     + 	/* If we want the whole block untouched, we can take the fast path. */
     + 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
     + 	    !opts->separator && !opts->key_only && !opts->value_only &&
     + 	    !opts->key_value_separator) {
     +-		strbuf_add(out, msg + info->trailer_block_start,
     +-			   info->trailer_block_end - info->trailer_block_start);
     +-		return;
     +-	}
     +-
     +-	for (i = 0; i < info->trailer_nr; i++) {
     +-		char *trailer = info->trailers[i];
     +-		ssize_t separator_pos = find_separator(trailer, separators);
     +-
     +-		if (separator_pos >= 1) {
     +-			struct strbuf tok = STRBUF_INIT;
     +-			struct strbuf val = STRBUF_INIT;
     +-
     +-			parse_trailer(&tok, &val, NULL, trailer, separator_pos);
     +-			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
     +-				if (opts->unfold)
     +-					unfold_value(&val);
     +-
     +-				if (opts->separator && out->len != origlen)
     +-					strbuf_addbuf(out, opts->separator);
     +-				if (!opts->value_only)
     +-					strbuf_addbuf(out, &tok);
     +-				if (!opts->key_only && !opts->value_only) {
     +-					if (opts->key_value_separator)
     +-						strbuf_addbuf(out, opts->key_value_separator);
     +-					else
     +-						strbuf_addstr(out, ": ");
     +-				}
     +-				if (!opts->key_only)
     +-					strbuf_addbuf(out, &val);
     +-				if (!opts->separator)
     +-					strbuf_addch(out, '\n');
     +-			}
     +-			strbuf_release(&tok);
     +-			strbuf_release(&val);
     +-
     +-		} else if (!opts->only_trailers) {
     +-			if (opts->separator && out->len != origlen) {
     +-				strbuf_addbuf(out, opts->separator);
     +-			}
     +-			strbuf_addstr(out, trailer);
     +-			if (opts->separator) {
     +-				strbuf_rtrim(out);
     +-			}
     +-		}
     +-	}
     +-
     +-}
     +-
     +-void format_trailers_from_commit(struct strbuf *out, const char *msg,
     +-				 const struct process_trailer_options *opts)
     +-{
     +-	struct trailer_info info;
      +		strbuf_add(out, msg + info.trailer_block_start,
      +			   info.trailer_block_end - info.trailer_block_start);
      +	} else
      +		format_trailers(opts, &head, out);
     -+
     + 
     +-	trailer_info_get(&info, msg, opts);
     +-	format_trailer_info(out, &info, msg, opts);
      +	free_trailers(&head);
       	trailer_info_release(&info);
       }
     @@ trailer.h: void trailer_info_release(struct trailer_info *info);
      -		     struct list_head *trailers, FILE *outfile);
      +		     struct list_head *trailers,
      +		     struct strbuf *out);
     - void free_trailers(struct list_head *trailers);
     + void free_trailers(struct list_head *);
       
       /*
      - * Format the trailers from the commit msg "msg" into the strbuf "out".
  4:  f1171f5202f <  -:  ----------- trailer: delete obsolete formatting functions
  5:  5ba842b5005 =  4:  bf2b8e1a3c4 sequencer: use the trailer iterator
  6:  f0ac2f6c4b9 =  5:  c19c1dcc592 trailer: make trailer_info struct private
  7:  291aa83af55 =  6:  0a9a7438c3f trailer: spread usage of "trailer_block" language
  8:  64ee07d0b53 !  7:  97e5d86ddf0 trailer: prepare to move parse_trailers_from_command_line_args() to builtin
     @@ Commit message
          themselves). The interpret-trailers builtin is the only caller of this
          function.
      
     -    Also rename "conf_info" to "trailer_conf" for readability, dropping the
     -    low-value "_info" suffix as we did earlier in this series for
     -    "trailer_info" to "trailer_block".
     +    Rename add_arg_item() to trailer_add_arg_item() because it will have to
     +    be exposed as an API function in the next patch. Rename
     +    new_trailers_clear() to free_new_trailers() because it will be promoted
     +    into an API function; the API already has free_trailers(), so using the
     +    "free_*" naming style will keep it consistent. Also rename "conf_info"
     +    to "trailer_conf" for readability, dropping the low-value "_info" suffix
     +    as we did earlier in this series for "trailer_info" to "trailer_block".
      
          Helped-by: Josh Steadmon <steadmon@google.com>
          Signed-off-by: Linus Arver <linusa@google.com>
      
     + ## builtin/interpret-trailers.c ##
     +@@ builtin/interpret-trailers.c: static int option_parse_if_missing(const struct option *opt,
     + 	return trailer_set_if_missing(opt->value, arg);
     + }
     + 
     +-static void new_trailers_clear(struct list_head *trailers)
     ++static void free_new_trailers(struct list_head *trailers)
     + {
     + 	struct list_head *pos, *tmp;
     + 	struct new_trailer_item *item;
     +@@ builtin/interpret-trailers.c: static int option_parse_trailer(const struct option *opt,
     + 	struct new_trailer_item *item;
     + 
     + 	if (unset) {
     +-		new_trailers_clear(trailers);
     ++		free_new_trailers(trailers);
     + 		return 0;
     + 	}
     + 
     +@@ builtin/interpret-trailers.c: int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
     + 		interpret_trailers(&opts, &trailers, NULL);
     + 	}
     + 
     +-	new_trailers_clear(&trailers);
     ++	free_new_trailers(&trailers);
     + 
     + 	return 0;
     + }
     +
       ## trailer.c ##
      @@ trailer.c: struct trailer_block {
       	size_t trailer_nr;
     @@ trailer.c: static ssize_t find_separator(const char *line, const char *separator
       		item = list_entry(pos, struct arg_item, list);
       		if (token_matches_item(tok->buf, item, tok_len)) {
      @@ trailer.c: static struct trailer_item *add_trailer_item(struct list_head *head, char *tok,
     + 	return new_item;
       }
       
     - static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
     +-static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
      -			 const struct conf_info *conf,
     -+			 const struct trailer_conf *conf,
     - 			 const struct new_trailer_item *new_trailer_item)
     +-			 const struct new_trailer_item *new_trailer_item)
     ++static void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
     ++				 const struct trailer_conf *conf,
     ++				 const struct new_trailer_item *new_trailer_item)
       {
       	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
       	new_item->token = tok;
     @@ trailer.c: static struct trailer_item *add_trailer_item(struct list_head *head,
       	if (new_trailer_item) {
       		if (new_trailer_item->where != WHERE_DEFAULT)
       			new_item->conf.where = new_trailer_item->where;
     +@@ trailer.c: void parse_trailers_from_config(struct list_head *config_head)
     + 	list_for_each(pos, &conf_head) {
     + 		item = list_entry(pos, struct arg_item, list);
     + 		if (item->conf.command)
     +-			add_arg_item(config_head,
     +-				     xstrdup(token_from_item(item, NULL)),
     +-				     xstrdup(""),
     +-				     &item->conf, NULL);
     ++			trailer_add_arg_item(config_head,
     ++					     xstrdup(token_from_item(item, NULL)),
     ++					     xstrdup(""),
     ++					     &item->conf, NULL);
     + 	}
     + }
     + 
      @@ trailer.c: void parse_trailers_from_command_line_args(struct list_head *arg_head,
       {
       	struct strbuf tok = STRBUF_INIT;
     @@ trailer.c: void parse_trailers_from_command_line_args(struct list_head *arg_head
       		} else {
      -			parse_trailer(&tok, &val, &conf, tr->text,
      -				      separator_pos);
     +-			add_arg_item(arg_head,
     +-				     strbuf_detach(&tok, NULL),
     +-				     strbuf_detach(&val, NULL),
     +-				     conf, tr);
      +			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
     - 			add_arg_item(arg_head,
     - 				     strbuf_detach(&tok, NULL),
     - 				     strbuf_detach(&val, NULL),
     ++			trailer_add_arg_item(arg_head,
     ++					     strbuf_detach(&tok, NULL),
     ++					     strbuf_detach(&val, NULL),
     ++					     conf, tr);
     + 		}
     + 	}
     + 
      @@ trailer.c: struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
       
       	for (i = 0; i < trailer_block->trailer_nr; i++) {
  9:  1b4bdde65bc !  8:  465dc51cdcb trailer: move arg handling to interpret-trailers.c
     @@ Commit message
          trailer: move arg handling to interpret-trailers.c
      
          We don't move the "arg_item" struct to interpret-trailers.c, because it
     -    is now a struct that contains information about trailers that should be
     -    injected into the input text's own trailers. We will rename this
     -    language as such in a follow-up patch to keep the diff here small.
     +    is now a struct that contains information about trailers that could be
     +    added into the input text's own trailers. This is a generic concept that
     +    extends beyond trailers defined as CLI arguments (it applies to trailers
     +    defined in configuration as well). We will rename "arg_item" to
     +    "trailer_template" in a follow-up patch to keep the diff here small.
      
          Signed-off-by: Linus Arver <linusa@google.com>
      
     @@ builtin/interpret-trailers.c: static int option_parse_if_missing(const struct op
       	return trailer_set_if_missing(opt->value, arg);
       }
       
     --static void new_trailers_clear(struct list_head *trailers)
     +-static void free_new_trailers(struct list_head *trailers)
      -{
      -	struct list_head *pos, *tmp;
      -	struct new_trailer_item *item;
     @@ builtin/interpret-trailers.c: static int option_parse_if_missing(const struct op
      +	ssize_t separator_pos;
       
       	if (unset) {
     - 		new_trailers_clear(trailers);
     + 		free_new_trailers(trailers);
      @@ builtin/interpret-trailers.c: static int option_parse_trailer(const struct option *opt,
       	if (!arg)
       		return -1;
     @@ builtin/interpret-trailers.c: static int option_parse_trailer(const struct optio
      +		 */
      +		trailer_conf_set(where, if_exists, if_missing, conf_current);
      +
     -+		add_arg_item(strbuf_detach(&tok, NULL),
     -+			     strbuf_detach(&val, NULL),
     -+			     conf_current,
     -+			     trailers);
     ++		trailer_add_arg_item(strbuf_detach(&tok, NULL),
     ++				     strbuf_detach(&val, NULL),
     ++				     conf_current,
     ++				     trailers);
      +	} else {
      +		struct strbuf sb = STRBUF_INIT;
      +		strbuf_addstr(&sb, arg);
     @@ builtin/interpret-trailers.c: int cmd_interpret_trailers(int argc, const char **
      +	* In command-line arguments, '=' is accepted (in addition to the
      +	* separators that are defined).
      +	*/
     -+	cl_separators = xstrfmt("=%s", default_separators());
     ++	cl_separators = xstrfmt("=%s", trailer_default_separators());
       
       	argc = parse_options(argc, argv, prefix, options,
       			     git_interpret_trailers_usage, 0);
     @@ builtin/interpret-trailers.c: int cmd_interpret_trailers(int argc, const char **
      +		interpret_trailers(&opts, &arg_trailers, NULL);
       	}
       
     --	new_trailers_clear(&trailers);
     -+	new_trailers_clear(&arg_trailers);
     +-	free_new_trailers(&trailers);
     ++	free_new_trailers(&arg_trailers);
       
       	return 0;
       }
     @@ trailer.c: static LIST_HEAD(conf_head);
       
       static char *separators = ":";
       
     -+const char *default_separators(void)
     ++const char *trailer_default_separators(void)
      +{
      +	return separators;
      +}
     @@ trailer.c: ssize_t find_separator(const char *line, const char *separators)
        * Obtain the token, value, and conf from the given trailer.
        *
      + * The conf needs special handling. We first read hardcoded defaults, and
     -+ * override them if we find a matching trailer configuration in the config.
     ++ * override them if we find a matching trailer configuration.
      + *
        * separator_pos must not be 0, since the token cannot be an empty string.
        *
     @@ trailer.c: static struct trailer_item *add_trailer_item(struct list_head *head,
       	return new_item;
       }
       
     --static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
     --			 const struct trailer_conf *conf,
     --			 const struct new_trailer_item *new_trailer_item)
     -+void add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
     -+		  struct list_head *arg_head)
     -+
     +-static void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
     +-				 const struct trailer_conf *conf,
     +-				 const struct new_trailer_item *new_trailer_item)
     ++void trailer_add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
     ++			  struct list_head *arg_head)
       {
       	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
       	new_item->token = tok;
     @@ trailer.c: void parse_trailers_from_config(struct list_head *config_head)
       	list_for_each(pos, &conf_head) {
       		item = list_entry(pos, struct arg_item, list);
       		if (item->conf.command)
     --			add_arg_item(config_head,
     --				     xstrdup(token_from_item(item, NULL)),
     -+			add_arg_item(xstrdup(token_from_item(item, NULL)),
     - 				     xstrdup(""),
     --				     &item->conf, NULL);
     -+				     &item->conf,
     -+				     config_head);
     +-			trailer_add_arg_item(config_head,
     +-					     xstrdup(token_from_item(item, NULL)),
     ++			trailer_add_arg_item(xstrdup(token_from_item(item, NULL)),
     + 					     xstrdup(""),
     +-					     &item->conf, NULL);
     ++					     &item->conf,
     ++					     config_head);
       	}
       }
       
     @@ trailer.c: void parse_trailers_from_command_line_args(struct list_head *arg_head
       			strbuf_release(&sb);
       		} else {
       			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
     --			add_arg_item(arg_head,
     --				     strbuf_detach(&tok, NULL),
     -+			add_arg_item(strbuf_detach(&tok, NULL),
     - 				     strbuf_detach(&val, NULL),
     --				     conf, tr);
     -+				     conf,
     -+				     arg_head);
     +-			trailer_add_arg_item(arg_head,
     +-					     strbuf_detach(&tok, NULL),
     ++			trailer_add_arg_item(strbuf_detach(&tok, NULL),
     + 					     strbuf_detach(&val, NULL),
     +-					     conf, tr);
     ++					     conf,
     ++					     arg_head);
       		}
       	}
       
     @@ trailer.c: void free_trailers(struct list_head *trailers)
       	}
       }
       
     -+void new_trailers_clear(struct list_head *trailers)
     ++void free_new_trailers(struct list_head *trailers)
      +{
      +	struct list_head *pos, *p;
      +
     @@ trailer.h: struct new_trailer_item {
       void duplicate_trailer_conf(struct trailer_conf *dst,
       			    const struct trailer_conf *src);
       
     -+const char *default_separators(void);
     ++const char *trailer_default_separators(void);
      +
     -+void add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
     -+		  struct list_head *arg_head);
     ++void trailer_add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
     ++			  struct list_head *arg_head);
      +
       struct process_trailer_options {
       	int in_place;
     @@ trailer.h: struct new_trailer_item {
      @@ trailer.h: void format_trailers(const struct process_trailer_options *opts,
       		     struct list_head *trailers,
       		     struct strbuf *out);
     - void free_trailers(struct list_head *trailers);
     --
     -+void new_trailers_clear(struct list_head *new_trailers);
     + void free_trailers(struct list_head *);
     ++void free_new_trailers(struct list_head *);
     + 
       /*
        * Convenience function to format the trailers from the commit msg "msg" into
     -  * the strbuf "out". Reuses format_trailers internally.
 10:  ed67ebf8647 !  9:  885ac87a544 trailer: delete obsolete argument handling code from API
     @@ Commit message
          This commit was not squashed with its parent in order to keep the diff
          separate (to make the additive changes in the parent easier to read).
      
     +    Note that we remove the "new_trailer_item" struct, because it has some
     +    overlap with "arg_item" struct that also deals with trailers coming from
     +    the command line. The latter will be renamed to "trailer_template" in
     +    the next patch.
     +
          Signed-off-by: Linus Arver <linusa@google.com>
      
       ## trailer.c ##
     @@ trailer.c: void parse_trailers_from_config(struct list_head *config_head)
      -			strbuf_release(&sb);
      -		} else {
      -			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
     --			add_arg_item(strbuf_detach(&tok, NULL),
     --				     strbuf_detach(&val, NULL),
     --				     conf,
     --				     arg_head);
     +-			trailer_add_arg_item(strbuf_detach(&tok, NULL),
     +-					     strbuf_detach(&val, NULL),
     +-					     conf,
     +-					     arg_head);
      -		}
      -	}
      -
  -:  ----------- > 10:  bcd3fc9660e trailer: introduce "template" term for readability

-- 
gitgitgadget

[PATCH v3 01/10] trailer: prepare to expose functions as part of API

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

In the next patch, we will move "process_trailers" from trailer.c to
builtin/interpret-trailers.c. That move will necessitate the growth of
the trailer.h API, forcing us to expose some additional functions in
trailer.h.

Rename relevant functions so that they include the term "trailer" in
their name, so that clients of the API will be able to easily identify
them by their "trailer" moniker, just like all the other functions
already exposed by trailer.h.

Take the opportunity to start putting trailer processions options (opts)
as the first parameter. This will be the pattern going forward in this
series.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  4 ++--
 trailer.c                    | 26 +++++++++++++-------------
 trailer.h                    |  6 +++---
 3 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 033bd1556cf..85a3413baf5 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -132,11 +132,11 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 	if (argc) {
 		int i;
 		for (i = 0; i < argc; i++)
-			process_trailers(argv[i], &opts, &trailers);
+			interpret_trailers(&opts, &trailers, argv[i]);
 	} else {
 		if (opts.in_place)
 			die(_("no input file given for in-place editing"));
-		process_trailers(NULL, &opts, &trailers);
+		interpret_trailers(&opts, &trailers, NULL);
 	}
 
 	new_trailers_clear(&trailers);
diff --git a/trailer.c b/trailer.c
index 3a0710a4583..66b6660f5a4 100644
--- a/trailer.c
+++ b/trailer.c
@@ -163,12 +163,12 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-static void print_all(FILE *outfile, struct list_head *head,
-		      const struct process_trailer_options *opts)
+static void format_trailers(const struct process_trailer_options *opts,
+			    struct list_head *trailers, FILE *outfile)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
-	list_for_each(pos, head) {
+	list_for_each(pos, trailers) {
 		item = list_entry(pos, struct trailer_item, list);
 		if ((!opts->trim_empty || strlen(item->value) > 0) &&
 		    (!opts->only_trailers || item->token))
@@ -589,7 +589,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	return 0;
 }
 
-static void ensure_configured(void)
+static void trailer_config_init(void)
 {
 	if (configured)
 		return;
@@ -1035,10 +1035,10 @@ static void parse_trailers(struct trailer_info *info,
 	}
 }
 
-static void free_all(struct list_head *head)
+static void free_trailers(struct list_head *trailers)
 {
 	struct list_head *pos, *p;
-	list_for_each_safe(pos, p, head) {
+	list_for_each_safe(pos, p, trailers) {
 		list_del(pos);
 		free_trailer_item(list_entry(pos, struct trailer_item, list));
 	}
@@ -1075,16 +1075,16 @@ static FILE *create_in_place_tempfile(const char *file)
 	return outfile;
 }
 
-void process_trailers(const char *file,
-		      const struct process_trailer_options *opts,
-		      struct list_head *new_trailer_head)
+void interpret_trailers(const struct process_trailer_options *opts,
+			struct list_head *new_trailer_head,
+			const char *file)
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
 	struct trailer_info info;
 	FILE *outfile = stdout;
 
-	ensure_configured();
+	trailer_config_init();
 
 	read_input_file(&sb, file);
 
@@ -1110,9 +1110,9 @@ void process_trailers(const char *file,
 		process_trailers_lists(&head, &arg_head);
 	}
 
-	print_all(outfile, &head, opts);
+	format_trailers(opts, &head, outfile);
 
-	free_all(&head);
+	free_trailers(&head);
 	trailer_info_release(&info);
 
 	/* Print the lines after the trailers as is */
@@ -1135,7 +1135,7 @@ void trailer_info_get(struct trailer_info *info, const char *str,
 	size_t nr = 0, alloc = 0;
 	char **last = NULL;
 
-	ensure_configured();
+	trailer_config_init();
 
 	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
 	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
diff --git a/trailer.h b/trailer.h
index 1644cd05f60..37033e631a1 100644
--- a/trailer.h
+++ b/trailer.h
@@ -81,9 +81,9 @@ struct process_trailer_options {
 
 #define PROCESS_TRAILER_OPTIONS_INIT {0}
 
-void process_trailers(const char *file,
-		      const struct process_trailer_options *opts,
-		      struct list_head *new_trailer_head);
+void interpret_trailers(const struct process_trailer_options *opts,
+			struct list_head *new_trailer_head,
+			const char *file);
 
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts);
-- 
gitgitgadget

[PATCH v3 02/10] trailer: move interpret_trailers() to interpret-trailers.c

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

The interpret-trailers.c builtin is the only place we need to call
interpret_trailers(), so move its definition there.

Delete the corresponding declaration from trailer.h, which then forces
us to expose the working innards of that function. This enriches
trailer.h with a more granular API, which can then be unit-tested in the
future (because interpret_trailers() by itself does too many things to
be able to be easily unit-tested).

Take this opportunity to demote some file-handling functions out of the
trailer API implementation, as these have nothing to do with trailers.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  94 +++++++++++++++++++++++++++
 trailer.c                    | 120 ++++-------------------------------
 trailer.h                    |  20 +++++-
 3 files changed, 124 insertions(+), 110 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 85a3413baf5..8556acde4aa 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -9,6 +9,7 @@
 #include "gettext.h"
 #include "parse-options.h"
 #include "string-list.h"
+#include "tempfile.h"
 #include "trailer.h"
 #include "config.h"
 
@@ -91,6 +92,99 @@ static int parse_opt_parse(const struct option *opt, const char *arg,
 	return 0;
 }
 
+static struct tempfile *trailers_tempfile;
+
+static FILE *create_in_place_tempfile(const char *file)
+{
+	struct stat st;
+	struct strbuf filename_template = STRBUF_INIT;
+	const char *tail;
+	FILE *outfile;
+
+	if (stat(file, &st))
+		die_errno(_("could not stat %s"), file);
+	if (!S_ISREG(st.st_mode))
+		die(_("file %s is not a regular file"), file);
+	if (!(st.st_mode & S_IWUSR))
+		die(_("file %s is not writable by user"), file);
+
+	/* Create temporary file in the same directory as the original */
+	tail = strrchr(file, '/');
+	if (tail)
+		strbuf_add(&filename_template, file, tail - file + 1);
+	strbuf_addstr(&filename_template, "git-interpret-trailers-XXXXXX");
+
+	trailers_tempfile = xmks_tempfile_m(filename_template.buf, st.st_mode);
+	strbuf_release(&filename_template);
+	outfile = fdopen_tempfile(trailers_tempfile, "w");
+	if (!outfile)
+		die_errno(_("could not open temporary file"));
+
+	return outfile;
+}
+
+static void read_input_file(struct strbuf *sb, const char *file)
+{
+	if (file) {
+		if (strbuf_read_file(sb, file, 0) < 0)
+			die_errno(_("could not read input file '%s'"), file);
+	} else {
+		if (strbuf_read(sb, fileno(stdin), 0) < 0)
+			die_errno(_("could not read from stdin"));
+	}
+}
+
+static void interpret_trailers(const struct process_trailer_options *opts,
+			       struct list_head *new_trailer_head,
+			       const char *file)
+{
+	LIST_HEAD(head);
+	struct strbuf sb = STRBUF_INIT;
+	struct trailer_info info;
+	FILE *outfile = stdout;
+
+	trailer_config_init();
+
+	read_input_file(&sb, file);
+
+	if (opts->in_place)
+		outfile = create_in_place_tempfile(file);
+
+	parse_trailers(opts, &info, sb.buf, &head);
+
+	/* Print the lines before the trailers */
+	if (!opts->only_trailers)
+		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
+
+	if (!opts->only_trailers && !info.blank_line_before_trailer)
+		fprintf(outfile, "\n");
+
+
+	if (!opts->only_input) {
+		LIST_HEAD(config_head);
+		LIST_HEAD(arg_head);
+		parse_trailers_from_config(&config_head);
+		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
+		list_splice(&config_head, &arg_head);
+		process_trailers_lists(&head, &arg_head);
+	}
+
+	format_trailers(opts, &head, outfile);
+
+	free_trailers(&head);
+	trailer_info_release(&info);
+
+	/* Print the lines after the trailers as is */
+	if (!opts->only_trailers)
+		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
+
+	if (opts->in_place)
+		if (rename_tempfile(&trailers_tempfile, file))
+			die_errno(_("could not rename temporary file to %s"), file);
+
+	strbuf_release(&sb);
+}
+
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
diff --git a/trailer.c b/trailer.c
index 66b6660f5a4..d3899195876 100644
--- a/trailer.c
+++ b/trailer.c
@@ -5,7 +5,6 @@
 #include "string-list.h"
 #include "run-command.h"
 #include "commit.h"
-#include "tempfile.h"
 #include "trailer.h"
 #include "list.h"
 /*
@@ -163,8 +162,8 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-static void format_trailers(const struct process_trailer_options *opts,
-			    struct list_head *trailers, FILE *outfile)
+void format_trailers(const struct process_trailer_options *opts,
+		     struct list_head *trailers, FILE *outfile)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
@@ -366,8 +365,8 @@ static int find_same_and_apply_arg(struct list_head *head,
 	return 0;
 }
 
-static void process_trailers_lists(struct list_head *head,
-				   struct list_head *arg_head)
+void process_trailers_lists(struct list_head *head,
+			    struct list_head *arg_head)
 {
 	struct list_head *pos, *p;
 	struct arg_item *arg_tok;
@@ -589,7 +588,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	return 0;
 }
 
-static void trailer_config_init(void)
+void trailer_config_init(void)
 {
 	if (configured)
 		return;
@@ -719,7 +718,7 @@ static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
 	list_add_tail(&new_item->list, arg_head);
 }
 
-static void parse_trailers_from_config(struct list_head *config_head)
+void parse_trailers_from_config(struct list_head *config_head)
 {
 	struct arg_item *item;
 	struct list_head *pos;
@@ -735,8 +734,8 @@ static void parse_trailers_from_config(struct list_head *config_head)
 	}
 }
 
-static void parse_trailers_from_command_line_args(struct list_head *arg_head,
-						  struct list_head *new_trailer_head)
+void parse_trailers_from_command_line_args(struct list_head *arg_head,
+					   struct list_head *new_trailer_head)
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
@@ -775,17 +774,6 @@ static void parse_trailers_from_command_line_args(struct list_head *arg_head,
 	free(cl_separators);
 }
 
-static void read_input_file(struct strbuf *sb, const char *file)
-{
-	if (file) {
-		if (strbuf_read_file(sb, file, 0) < 0)
-			die_errno(_("could not read input file '%s'"), file);
-	} else {
-		if (strbuf_read(sb, fileno(stdin), 0) < 0)
-			die_errno(_("could not read from stdin"));
-	}
-}
-
 static const char *next_line(const char *str)
 {
 	const char *nl = strchrnul(str, '\n');
@@ -1000,10 +988,10 @@ static void unfold_value(struct strbuf *val)
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
  */
-static void parse_trailers(struct trailer_info *info,
-			     const char *str,
-			     struct list_head *head,
-			     const struct process_trailer_options *opts)
+void parse_trailers(const struct process_trailer_options *opts,
+		    struct trailer_info *info,
+		    const char *str,
+		    struct list_head *head)
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
@@ -1035,7 +1023,7 @@ static void parse_trailers(struct trailer_info *info,
 	}
 }
 
-static void free_trailers(struct list_head *trailers)
+void free_trailers(struct list_head *trailers)
 {
 	struct list_head *pos, *p;
 	list_for_each_safe(pos, p, trailers) {
@@ -1044,88 +1032,6 @@ static void free_trailers(struct list_head *trailers)
 	}
 }
 
-static struct tempfile *trailers_tempfile;
-
-static FILE *create_in_place_tempfile(const char *file)
-{
-	struct stat st;
-	struct strbuf filename_template = STRBUF_INIT;
-	const char *tail;
-	FILE *outfile;
-
-	if (stat(file, &st))
-		die_errno(_("could not stat %s"), file);
-	if (!S_ISREG(st.st_mode))
-		die(_("file %s is not a regular file"), file);
-	if (!(st.st_mode & S_IWUSR))
-		die(_("file %s is not writable by user"), file);
-
-	/* Create temporary file in the same directory as the original */
-	tail = strrchr(file, '/');
-	if (tail)
-		strbuf_add(&filename_template, file, tail - file + 1);
-	strbuf_addstr(&filename_template, "git-interpret-trailers-XXXXXX");
-
-	trailers_tempfile = xmks_tempfile_m(filename_template.buf, st.st_mode);
-	strbuf_release(&filename_template);
-	outfile = fdopen_tempfile(trailers_tempfile, "w");
-	if (!outfile)
-		die_errno(_("could not open temporary file"));
-
-	return outfile;
-}
-
-void interpret_trailers(const struct process_trailer_options *opts,
-			struct list_head *new_trailer_head,
-			const char *file)
-{
-	LIST_HEAD(head);
-	struct strbuf sb = STRBUF_INIT;
-	struct trailer_info info;
-	FILE *outfile = stdout;
-
-	trailer_config_init();
-
-	read_input_file(&sb, file);
-
-	if (opts->in_place)
-		outfile = create_in_place_tempfile(file);
-
-	parse_trailers(&info, sb.buf, &head, opts);
-
-	/* Print the lines before the trailers */
-	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
-
-	if (!opts->only_trailers && !info.blank_line_before_trailer)
-		fprintf(outfile, "\n");
-
-
-	if (!opts->only_input) {
-		LIST_HEAD(config_head);
-		LIST_HEAD(arg_head);
-		parse_trailers_from_config(&config_head);
-		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
-		list_splice(&config_head, &arg_head);
-		process_trailers_lists(&head, &arg_head);
-	}
-
-	format_trailers(opts, &head, outfile);
-
-	free_trailers(&head);
-	trailer_info_release(&info);
-
-	/* Print the lines after the trailers as is */
-	if (!opts->only_trailers)
-		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
-
-	if (opts->in_place)
-		if (rename_tempfile(&trailers_tempfile, file))
-			die_errno(_("could not rename temporary file to %s"), file);
-
-	strbuf_release(&sb);
-}
-
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts)
 {
diff --git a/trailer.h b/trailer.h
index 37033e631a1..ca701c04f3b 100644
--- a/trailer.h
+++ b/trailer.h
@@ -81,15 +81,29 @@ struct process_trailer_options {
 
 #define PROCESS_TRAILER_OPTIONS_INIT {0}
 
-void interpret_trailers(const struct process_trailer_options *opts,
-			struct list_head *new_trailer_head,
-			const char *file);
+void parse_trailers_from_config(struct list_head *config_head);
+
+void parse_trailers_from_command_line_args(struct list_head *arg_head,
+					   struct list_head *new_trailer_head);
+
+void process_trailers_lists(struct list_head *head,
+			    struct list_head *arg_head);
+
+void parse_trailers(const struct process_trailer_options *opts,
+		    struct trailer_info *info,
+		    const char *str,
+		    struct list_head *head);
 
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts);
 
 void trailer_info_release(struct trailer_info *info);
 
+void trailer_config_init(void);
+void format_trailers(const struct process_trailer_options *opts,
+		     struct list_head *trailers, FILE *outfile);
+void free_trailers(struct list_head *);
+
 /*
  * Format the trailers from the commit msg "msg" into the strbuf "out".
  * Note two caveats about "opts":
-- 
gitgitgadget

[PATCH v3 03/10] trailer: unify trailer formatting machinery

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Currently have two functions for formatting trailers exposed in
trailer.h:

    void format_trailers(FILE *outfile, struct list_head *head,
                        const struct process_trailer_options *opts);

    void format_trailers_from_commit(struct strbuf *out, const char *msg,
                                    const struct process_trailer_options *opts);

and previously these functions, although similar enough (even taking the
same process_trailer_options struct pointer), did not build on each
other.

Make format_trailers_from_commit() rely on format_trailers(). Teach
format_trailers() to process trailers with the additional
process_trailer_options fields like opts->key_only which is only used by
format_trailers_from_commit() and not builtin/interpret-trailers.c.
While we're at it, reorder parameters to put the trailer processing
options first, and the out parameter (strbuf we write into) at the end.

This unification will allow us to delete the format_trailer_info() and
print_tok_val() functions in the next patch. They are not deleted here
in order to keep the diff small.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |   6 +-
 pretty.c                     |   2 +-
 ref-filter.c                 |   2 +-
 trailer.c                    | 176 +++++++++++++++++------------------
 trailer.h                    |  19 ++--
 5 files changed, 98 insertions(+), 107 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 8556acde4aa..5352ee65bd1 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -140,6 +140,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
+	struct strbuf trailer_block = STRBUF_INIT;
 	struct trailer_info info;
 	FILE *outfile = stdout;
 
@@ -169,7 +170,10 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 		process_trailers_lists(&head, &arg_head);
 	}
 
-	format_trailers(opts, &head, outfile);
+	/* Print trailer block. */
+	format_trailers(opts, &head, &trailer_block);
+	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
+	strbuf_release(&trailer_block);
 
 	free_trailers(&head);
 	trailer_info_release(&info);
diff --git a/pretty.c b/pretty.c
index cf964b060cd..bdbed4295aa 100644
--- a/pretty.c
+++ b/pretty.c
@@ -1759,7 +1759,7 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */
 				goto trailer_out;
 		}
 		if (*arg == ')') {
-			format_trailers_from_commit(sb, msg + c->subject_off, &opts);
+			format_trailers_from_commit(&opts, msg + c->subject_off, sb);
 			ret = arg - placeholder + 1;
 		}
 	trailer_out:
diff --git a/ref-filter.c b/ref-filter.c
index 35b989e1dfe..d358953b0ce 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -1985,7 +1985,7 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp
 			struct strbuf s = STRBUF_INIT;
 
 			/* Format the trailer info according to the trailer_opts given */
-			format_trailers_from_commit(&s, subpos, &atom->u.contents.trailer_opts);
+			format_trailers_from_commit(&atom->u.contents.trailer_opts, subpos, &s);
 
 			v->s = strbuf_detach(&s, NULL);
 		} else if (atom->u.contents.option == C_BARE)
diff --git a/trailer.c b/trailer.c
index d3899195876..71ea2bb67f8 100644
--- a/trailer.c
+++ b/trailer.c
@@ -144,37 +144,6 @@ static char last_non_space_char(const char *s)
 	return '\0';
 }
 
-static void print_tok_val(FILE *outfile, const char *tok, const char *val)
-{
-	char c;
-
-	if (!tok) {
-		fprintf(outfile, "%s\n", val);
-		return;
-	}
-
-	c = last_non_space_char(tok);
-	if (!c)
-		return;
-	if (strchr(separators, c))
-		fprintf(outfile, "%s%s\n", tok, val);
-	else
-		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
-}
-
-void format_trailers(const struct process_trailer_options *opts,
-		     struct list_head *trailers, FILE *outfile)
-{
-	struct list_head *pos;
-	struct trailer_item *item;
-	list_for_each(pos, trailers) {
-		item = list_entry(pos, struct trailer_item, list);
-		if ((!opts->trim_empty || strlen(item->value) > 0) &&
-		    (!opts->only_trailers || item->token))
-			print_tok_val(outfile, item->token, item->value);
-	}
-}
-
 static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
 {
 	struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
@@ -984,6 +953,78 @@ static void unfold_value(struct strbuf *val)
 	strbuf_release(&out);
 }
 
+void format_trailers(const struct process_trailer_options *opts,
+		     struct list_head *trailers,
+		     struct strbuf *out)
+{
+	struct list_head *pos;
+	struct trailer_item *item;
+	int need_separator = 0;
+
+	list_for_each(pos, trailers) {
+		item = list_entry(pos, struct trailer_item, list);
+		if (item->token) {
+			char c;
+
+			struct strbuf tok = STRBUF_INIT;
+			struct strbuf val = STRBUF_INIT;
+			strbuf_addstr(&tok, item->token);
+			strbuf_addstr(&val, item->value);
+
+			/*
+			 * Skip key/value pairs where the value was empty. This
+			 * can happen from trailers specified without a
+			 * separator, like `--trailer "Reviewed-by"` (no
+			 * corresponding value).
+			 */
+			if (opts->trim_empty && !strlen(item->value))
+				continue;
+
+			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
+				if (opts->unfold)
+					unfold_value(&val);
+
+				if (opts->separator && need_separator)
+					strbuf_addbuf(out, opts->separator);
+				if (!opts->value_only)
+					strbuf_addbuf(out, &tok);
+				if (!opts->key_only && !opts->value_only) {
+					if (opts->key_value_separator)
+						strbuf_addbuf(out, opts->key_value_separator);
+					else {
+						c = last_non_space_char(tok.buf);
+						if (c) {
+							if (!strchr(separators, c))
+								strbuf_addf(out, "%c ", separators[0]);
+						}
+					}
+				}
+				if (!opts->key_only)
+					strbuf_addbuf(out, &val);
+				if (!opts->separator)
+					strbuf_addch(out, '\n');
+
+				need_separator = 1;
+			}
+
+			strbuf_release(&tok);
+			strbuf_release(&val);
+		} else if (!opts->only_trailers) {
+			if (opts->separator && need_separator) {
+				strbuf_addbuf(out, opts->separator);
+			}
+			strbuf_addstr(out, item->value);
+			if (opts->separator)
+				strbuf_rtrim(out);
+			else
+				strbuf_addch(out, '\n');
+
+			need_separator = 1;
+		}
+
+	}
+}
+
 /*
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
@@ -1083,74 +1124,25 @@ void trailer_info_release(struct trailer_info *info)
 	free(info->trailers);
 }
 
-static void format_trailer_info(struct strbuf *out,
-				const struct trailer_info *info,
-				const char *msg,
-				const struct process_trailer_options *opts)
+void format_trailers_from_commit(const struct process_trailer_options *opts,
+				 const char *msg,
+				 struct strbuf *out)
 {
-	size_t origlen = out->len;
-	size_t i;
+	LIST_HEAD(head);
+	struct trailer_info info;
+
+	parse_trailers(opts, &info, msg, &head);
 
 	/* If we want the whole block untouched, we can take the fast path. */
 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
 	    !opts->separator && !opts->key_only && !opts->value_only &&
 	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info->trailer_block_start,
-			   info->trailer_block_end - info->trailer_block_start);
-		return;
-	}
-
-	for (i = 0; i < info->trailer_nr; i++) {
-		char *trailer = info->trailers[i];
-		ssize_t separator_pos = find_separator(trailer, separators);
-
-		if (separator_pos >= 1) {
-			struct strbuf tok = STRBUF_INIT;
-			struct strbuf val = STRBUF_INIT;
-
-			parse_trailer(&tok, &val, NULL, trailer, separator_pos);
-			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
-				if (opts->unfold)
-					unfold_value(&val);
-
-				if (opts->separator && out->len != origlen)
-					strbuf_addbuf(out, opts->separator);
-				if (!opts->value_only)
-					strbuf_addbuf(out, &tok);
-				if (!opts->key_only && !opts->value_only) {
-					if (opts->key_value_separator)
-						strbuf_addbuf(out, opts->key_value_separator);
-					else
-						strbuf_addstr(out, ": ");
-				}
-				if (!opts->key_only)
-					strbuf_addbuf(out, &val);
-				if (!opts->separator)
-					strbuf_addch(out, '\n');
-			}
-			strbuf_release(&tok);
-			strbuf_release(&val);
-
-		} else if (!opts->only_trailers) {
-			if (opts->separator && out->len != origlen) {
-				strbuf_addbuf(out, opts->separator);
-			}
-			strbuf_addstr(out, trailer);
-			if (opts->separator) {
-				strbuf_rtrim(out);
-			}
-		}
-	}
-
-}
-
-void format_trailers_from_commit(struct strbuf *out, const char *msg,
-				 const struct process_trailer_options *opts)
-{
-	struct trailer_info info;
+		strbuf_add(out, msg + info.trailer_block_start,
+			   info.trailer_block_end - info.trailer_block_start);
+	} else
+		format_trailers(opts, &head, out);
 
-	trailer_info_get(&info, msg, opts);
-	format_trailer_info(out, &info, msg, opts);
+	free_trailers(&head);
 	trailer_info_release(&info);
 }
 
diff --git a/trailer.h b/trailer.h
index ca701c04f3b..244f29fc91f 100644
--- a/trailer.h
+++ b/trailer.h
@@ -101,22 +101,17 @@ void trailer_info_release(struct trailer_info *info);
 
 void trailer_config_init(void);
 void format_trailers(const struct process_trailer_options *opts,
-		     struct list_head *trailers, FILE *outfile);
+		     struct list_head *trailers,
+		     struct strbuf *out);
 void free_trailers(struct list_head *);
 
 /*
- * Format the trailers from the commit msg "msg" into the strbuf "out".
- * Note two caveats about "opts":
- *
- *   - this is primarily a helper for pretty.c, and not
- *     all of the flags are supported.
- *
- *   - this differs from process_trailers slightly in that we always format
- *     only the trailer block itself, even if the "only_trailers" option is not
- *     set.
+ * Convenience function to format the trailers from the commit msg "msg" into
+ * the strbuf "out". Reuses format_trailers internally.
  */
-void format_trailers_from_commit(struct strbuf *out, const char *msg,
-				 const struct process_trailer_options *opts);
+void format_trailers_from_commit(const struct process_trailer_options *opts,
+				 const char *msg,
+				 struct strbuf *out);
 
 /*
  * An interface for iterating over the trailers found in a particular commit
-- 
gitgitgadget

[PATCH v3 04/10] sequencer: use the trailer iterator

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This patch allows for the removal of "trailer_info_get()" from the
trailer.h API, which will be in the next patch.

Instead of calling "trailer_info_get()", which is a low-level function
in the trailers implementation (trailer.c), call
trailer_iterator_advance(), which was specifically designed for public
consumption in f0939a0eb1 (trailer: add interface for iterating over
commit trailers, 2020-09-27).

Avoiding "trailer_info_get()" means we don't have to worry about options
like "no_divider" (relevant for parsing trailers). We also don't have to
check for things like "info.trailer_start == info.trailer_end" to see
whether there were any trailers (instead we can just check to see
whether the iterator advanced at all).

Also, teach the iterator about non-trailer lines, by adding a new field
called "raw" to hold both trailer and non-trailer lines. This is
necessary because a "trailer block" is a list of trailer lines of at
least 25% trailers (see 146245063e (trailer: allow non-trailers in
trailer block, 2016-10-21)), such that it may hold non-trailer lines.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/shortlog.c |  7 +++++--
 sequencer.c        | 35 +++++++++++++++--------------------
 trailer.c          | 17 +++++++++--------
 trailer.h          | 13 +++++++++++++
 4 files changed, 42 insertions(+), 30 deletions(-)

diff --git a/builtin/shortlog.c b/builtin/shortlog.c
index 1307ed2b88a..dc8fd5a5532 100644
--- a/builtin/shortlog.c
+++ b/builtin/shortlog.c
@@ -172,7 +172,7 @@ static void insert_records_from_trailers(struct shortlog *log,
 					 const char *oneline)
 {
 	struct trailer_iterator iter;
-	const char *commit_buffer, *body;
+	const char *commit_buffer, *body, *value;
 	struct strbuf ident = STRBUF_INIT;
 
 	if (!log->trailers.nr)
@@ -190,7 +190,10 @@ static void insert_records_from_trailers(struct shortlog *log,
 
 	trailer_iterator_init(&iter, body);
 	while (trailer_iterator_advance(&iter)) {
-		const char *value = iter.val.buf;
+		if (!iter.is_trailer)
+			continue;
+
+		value = iter.val.buf;
 
 		if (!string_list_has_string(&log->trailers, iter.key.buf))
 			continue;
diff --git a/sequencer.c b/sequencer.c
index 3cc88d8a800..bc7c82c5271 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -319,37 +319,32 @@ static const char *get_todo_path(const struct replay_opts *opts)
 static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	size_t ignore_footer)
 {
-	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
-	struct trailer_info info;
-	size_t i;
-	int found_sob = 0, found_sob_last = 0;
-	char saved_char;
-
-	opts.no_divider = 1;
+	struct trailer_iterator iter;
+	size_t i = 0, found_sob = 0;
+	char saved_char = sb->buf[sb->len - ignore_footer];
 
 	if (ignore_footer) {
-		saved_char = sb->buf[sb->len - ignore_footer];
 		sb->buf[sb->len - ignore_footer] = '\0';
 	}
 
-	trailer_info_get(&info, sb->buf, &opts);
+	trailer_iterator_init(&iter, sb->buf);
+	while (trailer_iterator_advance(&iter)) {
+		i++;
+		if (sob &&
+		    iter.is_trailer &&
+		    !strncmp(iter.raw, sob->buf, sob->len)) {
+			found_sob = i;
+		}
+	}
+	trailer_iterator_release(&iter);
 
 	if (ignore_footer)
 		sb->buf[sb->len - ignore_footer] = saved_char;
 
-	if (info.trailer_block_start == info.trailer_block_end)
+	if (!i)
 		return 0;
 
-	for (i = 0; i < info.trailer_nr; i++)
-		if (sob && !strncmp(info.trailers[i], sob->buf, sob->len)) {
-			found_sob = 1;
-			if (i == info.trailer_nr - 1)
-				found_sob_last = 1;
-		}
-
-	trailer_info_release(&info);
-
-	if (found_sob_last)
+	if (found_sob == i)
 		return 3;
 	if (found_sob)
 		return 2;
diff --git a/trailer.c b/trailer.c
index 71ea2bb67f8..5bcc9b0006c 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1158,17 +1158,18 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 
 int trailer_iterator_advance(struct trailer_iterator *iter)
 {
-	while (iter->internal.cur < iter->internal.info.trailer_nr) {
-		char *trailer = iter->internal.info.trailers[iter->internal.cur++];
-		int separator_pos = find_separator(trailer, separators);
-
-		if (separator_pos < 1)
-			continue; /* not a real trailer */
-
+	char *line;
+	int separator_pos;
+	if (iter->internal.cur < iter->internal.info.trailer_nr) {
+		line = iter->internal.info.trailers[iter->internal.cur++];
+		separator_pos = find_separator(line, separators);
+		iter->is_trailer = (separator_pos > 0);
+
+		iter->raw = line;
 		strbuf_reset(&iter->key);
 		strbuf_reset(&iter->val);
 		parse_trailer(&iter->key, &iter->val, NULL,
-			      trailer, separator_pos);
+			      line, separator_pos);
 		unfold_value(&iter->val);
 		return 1;
 	}
diff --git a/trailer.h b/trailer.h
index 244f29fc91f..a7599067acc 100644
--- a/trailer.h
+++ b/trailer.h
@@ -127,6 +127,19 @@ struct trailer_iterator {
 	struct strbuf key;
 	struct strbuf val;
 
+	/*
+	 * Raw line (e.g., "foo: bar baz") before being parsed as a trailer
+	 * key/val pair as part of a trailer block. A trailer block can be
+	 * either 100% trailer lines, or mixed in with non-trailer lines (in
+	 * which case at least 25% must be trailer lines).
+	 */
+	const char *raw;
+
+	/*
+	 * 1 if the raw line was parsed as a trailer line (key/val pair).
+	 */
+	int is_trailer;
+
 	/* private */
 	struct {
 		struct trailer_info info;
-- 
gitgitgadget

[PATCH v3 05/10] trailer: make trailer_info struct private

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

In 13211ae23f (trailer: separate public from internal portion of
trailer_iterator, 2023-09-09) we moved trailer_info behind an anonymous
struct to discourage use by trailer.h API users. However it still left
open the possibility of external use of trailer_info itself. Now that
there are no external users of trailer_info, we can make this struct
private.

Make this struct private by putting its definition inside trailer.c.
This has two benefits:

  (1) it makes the surface area of the public facing
      interface (trailer.h) smaller, and

  (2) external API users are unable to peer inside this struct (because
      it is only ever exposed as an opaque pointer).

This change exposes some deficiencies in the API, mainly with regard to
information about the location of the trailer block that was parsed.
Expose new API functions to access this information (needed by
builtin/interpret-trailers.c).

The idea in this patch to hide implementation details behind an "opaque
pointer" is also known as the "pimpl" (pointer to implementation) idiom
in C++ and is a common pattern in that language (where, for example,
abstract classes only have pointers to concrete classes).

However, the original inspiration to use this idiom does not come from
C++, but instead the book "C Interfaces and Implementations: Techniques
for Creating Reusable Software" [1]. This book recommends opaque
pointers as a good design principle for designing C libraries, using the
term "interface" as the functions defined in *.h (header) files and
"implementation" as the corresponding *.c file which define the
interfaces.

The book says this about opaque pointers:

    ... clients can manipulate such pointers freely, but they can’t
    dereference them; that is, they can’t look at the innards of the
    structure pointed to by them. Only the implementation has that
    privilege. Opaque pointers hide representation details and help
    catch errors.

In our case, "struct trailer_info" is now hidden from clients, and the
ways in which this opaque pointer can be used is limited to the richness
of the trailer.h file. In other words, trailer.h exclusively controls
exactly how "trailer_info" pointers are to be used.

[1] Hanson, David R. "C Interfaces and Implementations: Techniques for
    Creating Reusable Software". Addison Wesley, 1997. p. 22

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  13 +--
 trailer.c                    | 154 +++++++++++++++++++++++------------
 trailer.h                    |  37 ++-------
 3 files changed, 117 insertions(+), 87 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 5352ee65bd1..9e6ed6b65e2 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -141,7 +141,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
 	struct strbuf trailer_block = STRBUF_INIT;
-	struct trailer_info info;
+	struct trailer_info *info;
 	FILE *outfile = stdout;
 
 	trailer_config_init();
@@ -151,13 +151,13 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	if (opts->in_place)
 		outfile = create_in_place_tempfile(file);
 
-	parse_trailers(opts, &info, sb.buf, &head);
+	info = parse_trailers(opts, sb.buf, &head);
 
 	/* Print the lines before the trailers */
 	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
+		fwrite(sb.buf, 1, trailer_block_start(info), outfile);
 
-	if (!opts->only_trailers && !info.blank_line_before_trailer)
+	if (!opts->only_trailers && !blank_line_before_trailer_block(info))
 		fprintf(outfile, "\n");
 
 
@@ -176,11 +176,12 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	strbuf_release(&trailer_block);
 
 	free_trailers(&head);
-	trailer_info_release(&info);
 
 	/* Print the lines after the trailers as is */
 	if (!opts->only_trailers)
-		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
+		fwrite(sb.buf + trailer_block_end(info), 1, sb.len - trailer_block_end(info), outfile);
+
+	trailer_info_release(info);
 
 	if (opts->in_place)
 		if (rename_tempfile(&trailers_tempfile, file))
diff --git a/trailer.c b/trailer.c
index 5bcc9b0006c..63774cd068d 100644
--- a/trailer.c
+++ b/trailer.c
@@ -11,6 +11,27 @@
  * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
  */
 
+struct trailer_info {
+	/*
+	 * True if there is a blank line before the location pointed to by
+	 * trailer_block_start.
+	 */
+	int blank_line_before_trailer;
+
+	/*
+	 * Offsets to the trailer block start and end positions in the input
+	 * string. If no trailer block is found, these are both set to the
+	 * "true" end of the input (find_end_of_log_message()).
+	 */
+	size_t trailer_block_start, trailer_block_end;
+
+	/*
+	 * Array of trailers found.
+	 */
+	char **trailers;
+	size_t trailer_nr;
+};
+
 struct conf_info {
 	char *name;
 	char *key;
@@ -1025,20 +1046,72 @@ void format_trailers(const struct process_trailer_options *opts,
 	}
 }
 
+static struct trailer_info *trailer_info_new(void)
+{
+	struct trailer_info *info = xcalloc(1, sizeof(*info));
+	return info;
+}
+
+static struct trailer_info *trailer_info_get(const struct process_trailer_options *opts,
+					     const char *str)
+{
+	struct trailer_info *info = trailer_info_new();
+	size_t end_of_log_message = 0, trailer_block_start = 0;
+	struct strbuf **trailer_lines, **ptr;
+	char **trailer_strings = NULL;
+	size_t nr = 0, alloc = 0;
+	char **last = NULL;
+
+	trailer_config_init();
+
+	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
+	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
+
+	trailer_lines = strbuf_split_buf(str + trailer_block_start,
+					 end_of_log_message - trailer_block_start,
+					 '\n',
+					 0);
+	for (ptr = trailer_lines; *ptr; ptr++) {
+		if (last && isspace((*ptr)->buf[0])) {
+			struct strbuf sb = STRBUF_INIT;
+			strbuf_attach(&sb, *last, strlen(*last), strlen(*last));
+			strbuf_addbuf(&sb, *ptr);
+			*last = strbuf_detach(&sb, NULL);
+			continue;
+		}
+		ALLOC_GROW(trailer_strings, nr + 1, alloc);
+		trailer_strings[nr] = strbuf_detach(*ptr, NULL);
+		last = find_separator(trailer_strings[nr], separators) >= 1
+			? &trailer_strings[nr]
+			: NULL;
+		nr++;
+	}
+	strbuf_list_free(trailer_lines);
+
+	info->blank_line_before_trailer = ends_with_blank_line(str,
+							       trailer_block_start);
+	info->trailer_block_start = trailer_block_start;
+	info->trailer_block_end = end_of_log_message;
+	info->trailers = trailer_strings;
+	info->trailer_nr = nr;
+
+	return info;
+}
+
 /*
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
  */
-void parse_trailers(const struct process_trailer_options *opts,
-		    struct trailer_info *info,
-		    const char *str,
-		    struct list_head *head)
+struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
+				    const char *str,
+				    struct list_head *head)
 {
+	struct trailer_info *info;
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
 	size_t i;
 
-	trailer_info_get(info, str, opts);
+	info = trailer_info_get(opts, str);
 
 	for (i = 0; i < info->trailer_nr; i++) {
 		int separator_pos;
@@ -1062,6 +1135,8 @@ void parse_trailers(const struct process_trailer_options *opts,
 					 strbuf_detach(&val, NULL));
 		}
 	}
+
+	return info;
 }
 
 void free_trailers(struct list_head *trailers)
@@ -1073,47 +1148,19 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
-void trailer_info_get(struct trailer_info *info, const char *str,
-		      const struct process_trailer_options *opts)
+size_t trailer_block_start(struct trailer_info *info)
 {
-	size_t end_of_log_message = 0, trailer_block_start = 0;
-	struct strbuf **trailer_lines, **ptr;
-	char **trailer_strings = NULL;
-	size_t nr = 0, alloc = 0;
-	char **last = NULL;
-
-	trailer_config_init();
-
-	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
-	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
+	return info->trailer_block_start;
+}
 
-	trailer_lines = strbuf_split_buf(str + trailer_block_start,
-					 end_of_log_message - trailer_block_start,
-					 '\n',
-					 0);
-	for (ptr = trailer_lines; *ptr; ptr++) {
-		if (last && isspace((*ptr)->buf[0])) {
-			struct strbuf sb = STRBUF_INIT;
-			strbuf_attach(&sb, *last, strlen(*last), strlen(*last));
-			strbuf_addbuf(&sb, *ptr);
-			*last = strbuf_detach(&sb, NULL);
-			continue;
-		}
-		ALLOC_GROW(trailer_strings, nr + 1, alloc);
-		trailer_strings[nr] = strbuf_detach(*ptr, NULL);
-		last = find_separator(trailer_strings[nr], separators) >= 1
-			? &trailer_strings[nr]
-			: NULL;
-		nr++;
-	}
-	strbuf_list_free(trailer_lines);
+size_t trailer_block_end(struct trailer_info *info)
+{
+	return info->trailer_block_end;
+}
 
-	info->blank_line_before_trailer = ends_with_blank_line(str,
-							       trailer_block_start);
-	info->trailer_block_start = trailer_block_start;
-	info->trailer_block_end = end_of_log_message;
-	info->trailers = trailer_strings;
-	info->trailer_nr = nr;
+int blank_line_before_trailer_block(struct trailer_info *info)
+{
+	return info->blank_line_before_trailer;
 }
 
 void trailer_info_release(struct trailer_info *info)
@@ -1122,6 +1169,7 @@ void trailer_info_release(struct trailer_info *info)
 	for (i = 0; i < info->trailer_nr; i++)
 		free(info->trailers[i]);
 	free(info->trailers);
+	free(info);
 }
 
 void format_trailers_from_commit(const struct process_trailer_options *opts,
@@ -1129,30 +1177,30 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 				 struct strbuf *out)
 {
 	LIST_HEAD(head);
-	struct trailer_info info;
-
-	parse_trailers(opts, &info, msg, &head);
+	struct trailer_info *info = parse_trailers(opts, msg, &head);
 
 	/* If we want the whole block untouched, we can take the fast path. */
 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
 	    !opts->separator && !opts->key_only && !opts->value_only &&
 	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info.trailer_block_start,
-			   info.trailer_block_end - info.trailer_block_start);
+		strbuf_add(out, msg + info->trailer_block_start,
+			   info->trailer_block_end - info->trailer_block_start);
 	} else
 		format_trailers(opts, &head, out);
 
 	free_trailers(&head);
-	trailer_info_release(&info);
+	trailer_info_release(info);
 }
 
 void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
+	struct trailer_info *internal = trailer_info_new();
 	strbuf_init(&iter->key, 0);
 	strbuf_init(&iter->val, 0);
 	opts.no_divider = 1;
-	trailer_info_get(&iter->internal.info, msg, &opts);
+	iter->internal.info = internal;
+	iter->internal.info = trailer_info_get(&opts, msg);
 	iter->internal.cur = 0;
 }
 
@@ -1160,8 +1208,8 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 {
 	char *line;
 	int separator_pos;
-	if (iter->internal.cur < iter->internal.info.trailer_nr) {
-		line = iter->internal.info.trailers[iter->internal.cur++];
+	if (iter->internal.cur < iter->internal.info->trailer_nr) {
+		line = iter->internal.info->trailers[iter->internal.cur++];
 		separator_pos = find_separator(line, separators);
 		iter->is_trailer = (separator_pos > 0);
 
@@ -1178,7 +1226,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 
 void trailer_iterator_release(struct trailer_iterator *iter)
 {
-	trailer_info_release(&iter->internal.info);
+	trailer_info_release(iter->internal.info);
 	strbuf_release(&iter->val);
 	strbuf_release(&iter->key);
 }
diff --git a/trailer.h b/trailer.h
index a7599067acc..e19ddf84e64 100644
--- a/trailer.h
+++ b/trailer.h
@@ -4,6 +4,8 @@
 #include "list.h"
 #include "strbuf.h"
 
+struct trailer_info;
+
 enum trailer_where {
 	WHERE_DEFAULT,
 	WHERE_END,
@@ -29,27 +31,6 @@ int trailer_set_where(enum trailer_where *item, const char *value);
 int trailer_set_if_exists(enum trailer_if_exists *item, const char *value);
 int trailer_set_if_missing(enum trailer_if_missing *item, const char *value);
 
-struct trailer_info {
-	/*
-	 * True if there is a blank line before the location pointed to by
-	 * trailer_block_start.
-	 */
-	int blank_line_before_trailer;
-
-	/*
-	 * Offsets to the trailer block start and end positions in the input
-	 * string. If no trailer block is found, these are both set to the
-	 * "true" end of the input (find_end_of_log_message()).
-	 */
-	size_t trailer_block_start, trailer_block_end;
-
-	/*
-	 * Array of trailers found.
-	 */
-	char **trailers;
-	size_t trailer_nr;
-};
-
 /*
  * A list that represents newly-added trailers, such as those provided
  * with the --trailer command line option of git-interpret-trailers.
@@ -89,13 +70,13 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
-void parse_trailers(const struct process_trailer_options *opts,
-		    struct trailer_info *info,
-		    const char *str,
-		    struct list_head *head);
+struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
+				    const char *str,
+				    struct list_head *head);
 
-void trailer_info_get(struct trailer_info *info, const char *str,
-		      const struct process_trailer_options *opts);
+size_t trailer_block_start(struct trailer_info *info);
+size_t trailer_block_end(struct trailer_info *info);
+int blank_line_before_trailer_block(struct trailer_info *info);
 
 void trailer_info_release(struct trailer_info *info);
 
@@ -142,7 +123,7 @@ struct trailer_iterator {
 
 	/* private */
 	struct {
-		struct trailer_info info;
+		struct trailer_info *info;
 		size_t cur;
 	} internal;
 };
-- 
gitgitgadget

[PATCH v3 06/10] trailer: spread usage of "trailer_block" language

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Deprecate the "trailer_info" struct name and replace it with
"trailer_block". The main reason is to help readability, because
"trailer_info" on the surface sounds like it's about a single trailer
when in reality it is a collection of contiguous lines, at least 25% of
which are trailers.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c | 26 +++++-----
 trailer.c                    | 99 ++++++++++++++++++------------------
 trailer.h                    | 18 +++----
 3 files changed, 71 insertions(+), 72 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 9e6ed6b65e2..9e41fa20b5f 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -140,8 +140,8 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
-	struct strbuf trailer_block = STRBUF_INIT;
-	struct trailer_info *info;
+	struct strbuf tb = STRBUF_INIT;
+	struct trailer_block *trailer_block;
 	FILE *outfile = stdout;
 
 	trailer_config_init();
@@ -151,13 +151,13 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	if (opts->in_place)
 		outfile = create_in_place_tempfile(file);
 
-	info = parse_trailers(opts, sb.buf, &head);
+	trailer_block = parse_trailers(opts, sb.buf, &head);
 
-	/* Print the lines before the trailers */
+	/* Print the lines before the trailer block */
 	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, trailer_block_start(info), outfile);
+		fwrite(sb.buf, 1, trailer_block_start(trailer_block), outfile);
 
-	if (!opts->only_trailers && !blank_line_before_trailer_block(info))
+	if (!opts->only_trailers && !blank_line_before_trailer_block(trailer_block))
 		fprintf(outfile, "\n");
 
 
@@ -171,17 +171,17 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	}
 
 	/* Print trailer block. */
-	format_trailers(opts, &head, &trailer_block);
-	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
-	strbuf_release(&trailer_block);
+	format_trailers(opts, &head, &tb);
+	fwrite(tb.buf, 1, tb.len, outfile);
+	strbuf_release(&tb);
 
 	free_trailers(&head);
 
-	/* Print the lines after the trailers as is */
+	/* Print the lines after the trailer block as is */
 	if (!opts->only_trailers)
-		fwrite(sb.buf + trailer_block_end(info), 1, sb.len - trailer_block_end(info), outfile);
-
-	trailer_info_release(info);
+		fwrite(sb.buf + trailer_block_end(trailer_block),
+		       1, sb.len - trailer_block_end(trailer_block), outfile);
+	trailer_block_release(trailer_block);
 
 	if (opts->in_place)
 		if (rename_tempfile(&trailers_tempfile, file))
diff --git a/trailer.c b/trailer.c
index 63774cd068d..e2a48bea0ae 100644
--- a/trailer.c
+++ b/trailer.c
@@ -11,19 +11,20 @@
  * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
  */
 
-struct trailer_info {
+struct trailer_block {
 	/*
 	 * True if there is a blank line before the location pointed to by
-	 * trailer_block_start.
+	 * "start".
 	 */
 	int blank_line_before_trailer;
 
 	/*
-	 * Offsets to the trailer block start and end positions in the input
-	 * string. If no trailer block is found, these are both set to the
-	 * "true" end of the input (find_end_of_log_message()).
+	 * The locations of the start and end positions of the trailer block
+	 * found, as offsets from the beginning of the source text from which
+	 * this trailer block was parsed. If no trailer block is found, these
+	 * are both set to 0.
 	 */
-	size_t trailer_block_start, trailer_block_end;
+	size_t start, end;
 
 	/*
 	 * Array of trailers found.
@@ -1046,16 +1047,16 @@ void format_trailers(const struct process_trailer_options *opts,
 	}
 }
 
-static struct trailer_info *trailer_info_new(void)
+static struct trailer_block *trailer_block_new(void)
 {
-	struct trailer_info *info = xcalloc(1, sizeof(*info));
-	return info;
+	struct trailer_block *trailer_block = xcalloc(1, sizeof(*trailer_block));
+	return trailer_block;
 }
 
-static struct trailer_info *trailer_info_get(const struct process_trailer_options *opts,
-					     const char *str)
+static struct trailer_block *trailer_block_get(const struct process_trailer_options *opts,
+					       const char *str)
 {
-	struct trailer_info *info = trailer_info_new();
+	struct trailer_block *trailer_block = trailer_block_new();
 	size_t end_of_log_message = 0, trailer_block_start = 0;
 	struct strbuf **trailer_lines, **ptr;
 	char **trailer_strings = NULL;
@@ -1088,34 +1089,34 @@ static struct trailer_info *trailer_info_get(const struct process_trailer_option
 	}
 	strbuf_list_free(trailer_lines);
 
-	info->blank_line_before_trailer = ends_with_blank_line(str,
-							       trailer_block_start);
-	info->trailer_block_start = trailer_block_start;
-	info->trailer_block_end = end_of_log_message;
-	info->trailers = trailer_strings;
-	info->trailer_nr = nr;
+	trailer_block->blank_line_before_trailer = ends_with_blank_line(str,
+									trailer_block_start);
+	trailer_block->start = trailer_block_start;
+	trailer_block->end = end_of_log_message;
+	trailer_block->trailers = trailer_strings;
+	trailer_block->trailer_nr = nr;
 
-	return info;
+	return trailer_block;
 }
 
 /*
- * Parse trailers in "str", populating the trailer info and "head"
- * linked list structure.
+ * Parse trailers in "str", populating the trailer_block info and "head" linked
+ * list structure.
  */
-struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
-				    const char *str,
-				    struct list_head *head)
+struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
+				     const char *str,
+				     struct list_head *head)
 {
-	struct trailer_info *info;
+	struct trailer_block *trailer_block;
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
 	size_t i;
 
-	info = trailer_info_get(opts, str);
+	trailer_block = trailer_block_get(opts, str);
 
-	for (i = 0; i < info->trailer_nr; i++) {
+	for (i = 0; i < trailer_block->trailer_nr; i++) {
 		int separator_pos;
-		char *trailer = info->trailers[i];
+		char *trailer = trailer_block->trailers[i];
 		if (trailer[0] == comment_line_char)
 			continue;
 		separator_pos = find_separator(trailer, separators);
@@ -1136,7 +1137,7 @@ struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
 		}
 	}
 
-	return info;
+	return trailer_block;
 }
 
 void free_trailers(struct list_head *trailers)
@@ -1148,28 +1149,28 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
-size_t trailer_block_start(struct trailer_info *info)
+size_t trailer_block_start(struct trailer_block *trailer_block)
 {
-	return info->trailer_block_start;
+	return trailer_block->start;
 }
 
-size_t trailer_block_end(struct trailer_info *info)
+size_t trailer_block_end(struct trailer_block *trailer_block)
 {
-	return info->trailer_block_end;
+	return trailer_block->end;
 }
 
-int blank_line_before_trailer_block(struct trailer_info *info)
+int blank_line_before_trailer_block(struct trailer_block *trailer_block)
 {
-	return info->blank_line_before_trailer;
+	return trailer_block->blank_line_before_trailer;
 }
 
-void trailer_info_release(struct trailer_info *info)
+void trailer_block_release(struct trailer_block *trailer_block)
 {
 	size_t i;
-	for (i = 0; i < info->trailer_nr; i++)
-		free(info->trailers[i]);
-	free(info->trailers);
-	free(info);
+	for (i = 0; i < trailer_block->trailer_nr; i++)
+		free(trailer_block->trailers[i]);
+	free(trailer_block->trailers);
+	free(trailer_block);
 }
 
 void format_trailers_from_commit(const struct process_trailer_options *opts,
@@ -1177,30 +1178,28 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 				 struct strbuf *out)
 {
 	LIST_HEAD(head);
-	struct trailer_info *info = parse_trailers(opts, msg, &head);
+	struct trailer_block *trailer_block = parse_trailers(opts, msg, &head);
 
 	/* If we want the whole block untouched, we can take the fast path. */
 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
 	    !opts->separator && !opts->key_only && !opts->value_only &&
 	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info->trailer_block_start,
-			   info->trailer_block_end - info->trailer_block_start);
+		strbuf_add(out, msg + trailer_block->start,
+			   trailer_block->end - trailer_block->start);
 	} else
 		format_trailers(opts, &head, out);
 
 	free_trailers(&head);
-	trailer_info_release(info);
+	trailer_block_release(trailer_block);
 }
 
 void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
-	struct trailer_info *internal = trailer_info_new();
 	strbuf_init(&iter->key, 0);
 	strbuf_init(&iter->val, 0);
 	opts.no_divider = 1;
-	iter->internal.info = internal;
-	iter->internal.info = trailer_info_get(&opts, msg);
+	iter->internal.trailer_block = trailer_block_get(&opts, msg);
 	iter->internal.cur = 0;
 }
 
@@ -1208,8 +1207,8 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 {
 	char *line;
 	int separator_pos;
-	if (iter->internal.cur < iter->internal.info->trailer_nr) {
-		line = iter->internal.info->trailers[iter->internal.cur++];
+	if (iter->internal.cur < iter->internal.trailer_block->trailer_nr) {
+		line = iter->internal.trailer_block->trailers[iter->internal.cur++];
 		separator_pos = find_separator(line, separators);
 		iter->is_trailer = (separator_pos > 0);
 
@@ -1226,7 +1225,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 
 void trailer_iterator_release(struct trailer_iterator *iter)
 {
-	trailer_info_release(iter->internal.info);
+	trailer_block_release(iter->internal.trailer_block);
 	strbuf_release(&iter->val);
 	strbuf_release(&iter->key);
 }
diff --git a/trailer.h b/trailer.h
index e19ddf84e64..e74f9189c0a 100644
--- a/trailer.h
+++ b/trailer.h
@@ -4,7 +4,7 @@
 #include "list.h"
 #include "strbuf.h"
 
-struct trailer_info;
+struct trailer_block;
 
 enum trailer_where {
 	WHERE_DEFAULT,
@@ -70,15 +70,15 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
-struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
-				    const char *str,
-				    struct list_head *head);
+struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
+				     const char *str,
+				     struct list_head *head);
 
-size_t trailer_block_start(struct trailer_info *info);
-size_t trailer_block_end(struct trailer_info *info);
-int blank_line_before_trailer_block(struct trailer_info *info);
+size_t trailer_block_start(struct trailer_block *trailer_block);
+size_t trailer_block_end(struct trailer_block *trailer_block);
+int blank_line_before_trailer_block(struct trailer_block *trailer_block);
 
-void trailer_info_release(struct trailer_info *info);
+void trailer_block_release(struct trailer_block *trailer_block);
 
 void trailer_config_init(void);
 void format_trailers(const struct process_trailer_options *opts,
@@ -123,7 +123,7 @@ struct trailer_iterator {
 
 	/* private */
 	struct {
-		struct trailer_info *info;
+		struct trailer_block *trailer_block;
 		size_t cur;
 	} internal;
 };
-- 
gitgitgadget

[PATCH v3 07/10] trailer: prepare to move parse_trailers_from_command_line_args() to builtin

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Expose more functions in the trailer.h API, in preparation for moving
out

    parse_trailers_from_command_line_args()

to interpret-trailer.c, because the trailer API should not be concerned
with command line arguments (as they have nothing to do with trailers
themselves). The interpret-trailers builtin is the only caller of this
function.

Rename add_arg_item() to trailer_add_arg_item() because it will have to
be exposed as an API function in the next patch. Rename
new_trailers_clear() to free_new_trailers() because it will be promoted
into an API function; the API already has free_trailers(), so using the
"free_*" naming style will keep it consistent. Also rename "conf_info"
to "trailer_conf" for readability, dropping the low-value "_info" suffix
as we did earlier in this series for "trailer_info" to "trailer_block".

Helped-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  6 +--
 trailer.c                    | 86 ++++++++++++++++++------------------
 trailer.h                    | 10 +++++
 3 files changed, 55 insertions(+), 47 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 9e41fa20b5f..9f0ba39b317 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -45,7 +45,7 @@ static int option_parse_if_missing(const struct option *opt,
 	return trailer_set_if_missing(opt->value, arg);
 }
 
-static void new_trailers_clear(struct list_head *trailers)
+static void free_new_trailers(struct list_head *trailers)
 {
 	struct list_head *pos, *tmp;
 	struct new_trailer_item *item;
@@ -64,7 +64,7 @@ static int option_parse_trailer(const struct option *opt,
 	struct new_trailer_item *item;
 
 	if (unset) {
-		new_trailers_clear(trailers);
+		free_new_trailers(trailers);
 		return 0;
 	}
 
@@ -238,7 +238,7 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 		interpret_trailers(&opts, &trailers, NULL);
 	}
 
-	new_trailers_clear(&trailers);
+	free_new_trailers(&trailers);
 
 	return 0;
 }
diff --git a/trailer.c b/trailer.c
index e2a48bea0ae..c16f552b463 100644
--- a/trailer.c
+++ b/trailer.c
@@ -33,7 +33,7 @@ struct trailer_block {
 	size_t trailer_nr;
 };
 
-struct conf_info {
+struct trailer_conf {
 	char *name;
 	char *key;
 	char *command;
@@ -43,7 +43,7 @@ struct conf_info {
 	enum trailer_if_missing if_missing;
 };
 
-static struct conf_info default_conf_info;
+static struct trailer_conf default_trailer_conf;
 
 struct trailer_item {
 	struct list_head list;
@@ -59,7 +59,7 @@ struct arg_item {
 	struct list_head list;
 	char *token;
 	char *value;
-	struct conf_info conf;
+	struct trailer_conf conf;
 };
 
 static LIST_HEAD(conf_head);
@@ -210,7 +210,7 @@ static int check_if_different(struct trailer_item *in_tok,
 	return 1;
 }
 
-static char *apply_command(struct conf_info *conf, const char *arg)
+static char *apply_command(struct trailer_conf *conf, const char *arg)
 {
 	struct strbuf cmd = STRBUF_INIT;
 	struct strbuf buf = STRBUF_INIT;
@@ -424,7 +424,8 @@ int trailer_set_if_missing(enum trailer_if_missing *item, const char *value)
 	return 0;
 }
 
-static void duplicate_conf(struct conf_info *dst, const struct conf_info *src)
+void duplicate_trailer_conf(struct trailer_conf *dst,
+			    const struct trailer_conf *src)
 {
 	*dst = *src;
 	dst->name = xstrdup_or_null(src->name);
@@ -447,7 +448,7 @@ static struct arg_item *get_conf_item(const char *name)
 
 	/* Item does not already exists, create it */
 	CALLOC_ARRAY(item, 1);
-	duplicate_conf(&item->conf, &default_conf_info);
+	duplicate_trailer_conf(&item->conf, &default_trailer_conf);
 	item->conf.name = xstrdup(name);
 
 	list_add_tail(&item->list, &conf_head);
@@ -482,17 +483,17 @@ static int git_trailer_default_config(const char *conf_key, const char *value,
 	variable_name = strrchr(trailer_item, '.');
 	if (!variable_name) {
 		if (!strcmp(trailer_item, "where")) {
-			if (trailer_set_where(&default_conf_info.where,
+			if (trailer_set_where(&default_trailer_conf.where,
 					      value) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
 		} else if (!strcmp(trailer_item, "ifexists")) {
-			if (trailer_set_if_exists(&default_conf_info.if_exists,
+			if (trailer_set_if_exists(&default_trailer_conf.if_exists,
 						  value) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
 		} else if (!strcmp(trailer_item, "ifmissing")) {
-			if (trailer_set_if_missing(&default_conf_info.if_missing,
+			if (trailer_set_if_missing(&default_trailer_conf.if_missing,
 						   value) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
@@ -511,7 +512,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 {
 	const char *trailer_item, *variable_name;
 	struct arg_item *item;
-	struct conf_info *conf;
+	struct trailer_conf *conf;
 	char *name = NULL;
 	enum trailer_info_type type;
 	int i;
@@ -585,9 +586,9 @@ void trailer_config_init(void)
 		return;
 
 	/* Default config must be setup first */
-	default_conf_info.where = WHERE_END;
-	default_conf_info.if_exists = EXISTS_ADD_IF_DIFFERENT_NEIGHBOR;
-	default_conf_info.if_missing = MISSING_ADD;
+	default_trailer_conf.where = WHERE_END;
+	default_trailer_conf.if_exists = EXISTS_ADD_IF_DIFFERENT_NEIGHBOR;
+	default_trailer_conf.if_missing = MISSING_ADD;
 	git_config(git_trailer_default_config, NULL);
 	git_config(git_trailer_config, NULL);
 	configured = 1;
@@ -620,7 +621,7 @@ static int token_matches_item(const char *tok, struct arg_item *item, size_t tok
  * distinguished from the non-well-formed-line case (in which this function
  * returns -1) because some callers of this function need such a distinction.
  */
-static ssize_t find_separator(const char *line, const char *separators)
+ssize_t find_separator(const char *line, const char *separators)
 {
 	int whitespace_found = 0;
 	const char *c;
@@ -645,28 +646,28 @@ static ssize_t find_separator(const char *line, const char *separators)
  *
  * If separator_pos is -1, interpret the whole trailer as a token.
  */
-static void parse_trailer(struct strbuf *tok, struct strbuf *val,
-			 const struct conf_info **conf, const char *trailer,
-			 ssize_t separator_pos)
+void parse_trailer(const char *line, ssize_t separator_pos,
+		   struct strbuf *tok, struct strbuf *val,
+		   const struct trailer_conf **conf)
 {
 	struct arg_item *item;
 	size_t tok_len;
 	struct list_head *pos;
 
 	if (separator_pos != -1) {
-		strbuf_add(tok, trailer, separator_pos);
+		strbuf_add(tok, line, separator_pos);
 		strbuf_trim(tok);
-		strbuf_addstr(val, trailer + separator_pos + 1);
+		strbuf_addstr(val, line + separator_pos + 1);
 		strbuf_trim(val);
 	} else {
-		strbuf_addstr(tok, trailer);
+		strbuf_addstr(tok, line);
 		strbuf_trim(tok);
 	}
 
 	/* Lookup if the token matches something in the config */
 	tok_len = token_len_without_separator(tok->buf, tok->len);
 	if (conf)
-		*conf = &default_conf_info;
+		*conf = &default_trailer_conf;
 	list_for_each(pos, &conf_head) {
 		item = list_entry(pos, struct arg_item, list);
 		if (token_matches_item(tok->buf, item, tok_len)) {
@@ -690,14 +691,14 @@ static struct trailer_item *add_trailer_item(struct list_head *head, char *tok,
 	return new_item;
 }
 
-static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
-			 const struct conf_info *conf,
-			 const struct new_trailer_item *new_trailer_item)
+static void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
+				 const struct trailer_conf *conf,
+				 const struct new_trailer_item *new_trailer_item)
 {
 	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
 	new_item->token = tok;
 	new_item->value = val;
-	duplicate_conf(&new_item->conf, conf);
+	duplicate_trailer_conf(&new_item->conf, conf);
 	if (new_trailer_item) {
 		if (new_trailer_item->where != WHERE_DEFAULT)
 			new_item->conf.where = new_trailer_item->where;
@@ -718,10 +719,10 @@ void parse_trailers_from_config(struct list_head *config_head)
 	list_for_each(pos, &conf_head) {
 		item = list_entry(pos, struct arg_item, list);
 		if (item->conf.command)
-			add_arg_item(config_head,
-				     xstrdup(token_from_item(item, NULL)),
-				     xstrdup(""),
-				     &item->conf, NULL);
+			trailer_add_arg_item(config_head,
+					     xstrdup(token_from_item(item, NULL)),
+					     xstrdup(""),
+					     &item->conf, NULL);
 	}
 }
 
@@ -730,7 +731,7 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
-	const struct conf_info *conf;
+	const struct trailer_conf *conf;
 	struct list_head *pos;
 
 	/*
@@ -753,12 +754,11 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 			      (int) sb.len, sb.buf);
 			strbuf_release(&sb);
 		} else {
-			parse_trailer(&tok, &val, &conf, tr->text,
-				      separator_pos);
-			add_arg_item(arg_head,
-				     strbuf_detach(&tok, NULL),
-				     strbuf_detach(&val, NULL),
-				     conf, tr);
+			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
+			trailer_add_arg_item(arg_head,
+					     strbuf_detach(&tok, NULL),
+					     strbuf_detach(&val, NULL),
+					     conf, tr);
 		}
 	}
 
@@ -1116,20 +1116,19 @@ struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
 
 	for (i = 0; i < trailer_block->trailer_nr; i++) {
 		int separator_pos;
-		char *trailer = trailer_block->trailers[i];
-		if (trailer[0] == comment_line_char)
+		char *line = trailer_block->trailers[i];
+		if (line[0] == comment_line_char)
 			continue;
-		separator_pos = find_separator(trailer, separators);
+		separator_pos = find_separator(line, separators);
 		if (separator_pos >= 1) {
-			parse_trailer(&tok, &val, NULL, trailer,
-				      separator_pos);
+			parse_trailer(line, separator_pos, &tok, &val, NULL);
 			if (opts->unfold)
 				unfold_value(&val);
 			add_trailer_item(head,
 					 strbuf_detach(&tok, NULL),
 					 strbuf_detach(&val, NULL));
 		} else if (!opts->only_trailers) {
-			strbuf_addstr(&val, trailer);
+			strbuf_addstr(&val, line);
 			strbuf_strip_suffix(&val, "\n");
 			add_trailer_item(head,
 					 NULL,
@@ -1215,8 +1214,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 		iter->raw = line;
 		strbuf_reset(&iter->key);
 		strbuf_reset(&iter->val);
-		parse_trailer(&iter->key, &iter->val, NULL,
-			      line, separator_pos);
+		parse_trailer(line, separator_pos, &iter->key, &iter->val, NULL);
 		unfold_value(&iter->val);
 		return 1;
 	}
diff --git a/trailer.h b/trailer.h
index e74f9189c0a..d724263e4f6 100644
--- a/trailer.h
+++ b/trailer.h
@@ -5,6 +5,7 @@
 #include "strbuf.h"
 
 struct trailer_block;
+struct trailer_conf;
 
 enum trailer_where {
 	WHERE_DEFAULT,
@@ -45,6 +46,9 @@ struct new_trailer_item {
 	enum trailer_if_missing if_missing;
 };
 
+void duplicate_trailer_conf(struct trailer_conf *dst,
+			    const struct trailer_conf *src);
+
 struct process_trailer_options {
 	int in_place;
 	int trim_empty;
@@ -70,6 +74,12 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
+ssize_t find_separator(const char *line, const char *separators);
+
+void parse_trailer(const char *line, ssize_t separator_pos,
+		   struct strbuf *tok, struct strbuf *val,
+		   const struct trailer_conf **conf);
+
 struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
 				     const char *str,
 				     struct list_head *head);
-- 
gitgitgadget

[PATCH v3 08/10] trailer: move arg handling to interpret-trailers.c

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

We don't move the "arg_item" struct to interpret-trailers.c, because it
is now a struct that contains information about trailers that could be
added into the input text's own trailers. This is a generic concept that
extends beyond trailers defined as CLI arguments (it applies to trailers
defined in configuration as well). We will rename "arg_item" to
"trailer_template" in a follow-up patch to keep the diff here small.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c | 88 ++++++++++++++++++++++--------------
 trailer.c                    | 62 ++++++++++++++++++-------
 trailer.h                    | 12 +++++
 3 files changed, 112 insertions(+), 50 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 9f0ba39b317..9a902012912 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -45,23 +45,17 @@ static int option_parse_if_missing(const struct option *opt,
 	return trailer_set_if_missing(opt->value, arg);
 }
 
-static void free_new_trailers(struct list_head *trailers)
-{
-	struct list_head *pos, *tmp;
-	struct new_trailer_item *item;
-
-	list_for_each_safe(pos, tmp, trailers) {
-		item = list_entry(pos, struct new_trailer_item, list);
-		list_del(pos);
-		free(item);
-	}
-}
+static char *cl_separators;
 
 static int option_parse_trailer(const struct option *opt,
 				   const char *arg, int unset)
 {
 	struct list_head *trailers = opt->value;
-	struct new_trailer_item *item;
+	struct strbuf tok = STRBUF_INIT;
+	struct strbuf val = STRBUF_INIT;
+	const struct trailer_conf *conf;
+	struct trailer_conf *conf_current = new_trailer_conf();
+	ssize_t separator_pos;
 
 	if (unset) {
 		free_new_trailers(trailers);
@@ -71,12 +65,31 @@ static int option_parse_trailer(const struct option *opt,
 	if (!arg)
 		return -1;
 
-	item = xmalloc(sizeof(*item));
-	item->text = arg;
-	item->where = where;
-	item->if_exists = if_exists;
-	item->if_missing = if_missing;
-	list_add_tail(&item->list, trailers);
+	separator_pos = find_separator(arg, cl_separators);
+	if (separator_pos) {
+		parse_trailer(arg, separator_pos, &tok, &val, &conf);
+		duplicate_trailer_conf(conf_current, conf);
+
+		/*
+		 * Override conf_current with settings specified via CLI flags.
+		 */
+		trailer_conf_set(where, if_exists, if_missing, conf_current);
+
+		trailer_add_arg_item(strbuf_detach(&tok, NULL),
+				     strbuf_detach(&val, NULL),
+				     conf_current,
+				     trailers);
+	} else {
+		struct strbuf sb = STRBUF_INIT;
+		strbuf_addstr(&sb, arg);
+		strbuf_trim(&sb);
+		error(_("empty trailer token in trailer '%.*s'"),
+			(int) sb.len, sb.buf);
+		strbuf_release(&sb);
+	}
+
+	free(conf_current);
+
 	return 0;
 }
 
@@ -135,7 +148,7 @@ static void read_input_file(struct strbuf *sb, const char *file)
 }
 
 static void interpret_trailers(const struct process_trailer_options *opts,
-			       struct list_head *new_trailer_head,
+			       struct list_head *arg_trailers,
 			       const char *file)
 {
 	LIST_HEAD(head);
@@ -144,8 +157,6 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	struct trailer_block *trailer_block;
 	FILE *outfile = stdout;
 
-	trailer_config_init();
-
 	read_input_file(&sb, file);
 
 	if (opts->in_place)
@@ -162,12 +173,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 
 
 	if (!opts->only_input) {
-		LIST_HEAD(config_head);
-		LIST_HEAD(arg_head);
-		parse_trailers_from_config(&config_head);
-		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
-		list_splice(&config_head, &arg_head);
-		process_trailers_lists(&head, &arg_head);
+		process_trailers_lists(&head, arg_trailers);
 	}
 
 	/* Print trailer block. */
@@ -193,7 +199,8 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
-	LIST_HEAD(trailers);
+	LIST_HEAD(configured_trailers);
+	LIST_HEAD(arg_trailers);
 
 	struct option options[] = {
 		OPT_BOOL(0, "in-place", &opts.in_place, N_("edit files in place")),
@@ -212,33 +219,48 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 		OPT_CALLBACK_F(0, "parse", &opts, NULL, N_("alias for --only-trailers --only-input --unfold"),
 			PARSE_OPT_NOARG | PARSE_OPT_NONEG, parse_opt_parse),
 		OPT_BOOL(0, "no-divider", &opts.no_divider, N_("do not treat \"---\" as the end of input")),
-		OPT_CALLBACK(0, "trailer", &trailers, N_("trailer"),
+		OPT_CALLBACK(0, "trailer", &arg_trailers, N_("trailer"),
 				N_("trailer(s) to add"), option_parse_trailer),
 		OPT_END()
 	};
 
 	git_config(git_default_config, NULL);
+	trailer_config_init();
+
+	if (!opts.only_input) {
+		parse_trailers_from_config(&configured_trailers);
+	}
+
+	/*
+	* In command-line arguments, '=' is accepted (in addition to the
+	* separators that are defined).
+	*/
+	cl_separators = xstrfmt("=%s", trailer_default_separators());
 
 	argc = parse_options(argc, argv, prefix, options,
 			     git_interpret_trailers_usage, 0);
 
-	if (opts.only_input && !list_empty(&trailers))
+	free(cl_separators);
+
+	if (opts.only_input && !list_empty(&arg_trailers))
 		usage_msg_opt(
 			_("--trailer with --only-input does not make sense"),
 			git_interpret_trailers_usage,
 			options);
 
+	list_splice(&configured_trailers, &arg_trailers);
+
 	if (argc) {
 		int i;
 		for (i = 0; i < argc; i++)
-			interpret_trailers(&opts, &trailers, argv[i]);
+			interpret_trailers(&opts, &arg_trailers, argv[i]);
 	} else {
 		if (opts.in_place)
 			die(_("no input file given for in-place editing"));
-		interpret_trailers(&opts, &trailers, NULL);
+		interpret_trailers(&opts, &arg_trailers, NULL);
 	}
 
-	free_new_trailers(&trailers);
+	free_new_trailers(&arg_trailers);
 
 	return 0;
 }
diff --git a/trailer.c b/trailer.c
index c16f552b463..19637ff295d 100644
--- a/trailer.c
+++ b/trailer.c
@@ -66,6 +66,11 @@ static LIST_HEAD(conf_head);
 
 static char *separators = ":";
 
+const char *trailer_default_separators(void)
+{
+	return separators;
+}
+
 static int configured;
 
 #define TRAILER_ARG_STRING "$ARG"
@@ -424,6 +429,25 @@ int trailer_set_if_missing(enum trailer_if_missing *item, const char *value)
 	return 0;
 }
 
+void trailer_conf_set(enum trailer_where where,
+		      enum trailer_if_exists if_exists,
+		      enum trailer_if_missing if_missing,
+		      struct trailer_conf *conf)
+{
+	if (where != WHERE_DEFAULT)
+		conf->where = where;
+	if (if_exists != EXISTS_DEFAULT)
+		conf->if_exists = if_exists;
+	if (if_missing != MISSING_DEFAULT)
+		conf->if_missing = if_missing;
+}
+
+struct trailer_conf *new_trailer_conf(void)
+{
+	struct trailer_conf *new = xcalloc(1, sizeof(*new));
+	return new;
+}
+
 void duplicate_trailer_conf(struct trailer_conf *dst,
 			    const struct trailer_conf *src)
 {
@@ -642,6 +666,9 @@ ssize_t find_separator(const char *line, const char *separators)
 /*
  * Obtain the token, value, and conf from the given trailer.
  *
+ * The conf needs special handling. We first read hardcoded defaults, and
+ * override them if we find a matching trailer configuration.
+ *
  * separator_pos must not be 0, since the token cannot be an empty string.
  *
  * If separator_pos is -1, interpret the whole trailer as a token.
@@ -691,22 +718,13 @@ static struct trailer_item *add_trailer_item(struct list_head *head, char *tok,
 	return new_item;
 }
 
-static void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
-				 const struct trailer_conf *conf,
-				 const struct new_trailer_item *new_trailer_item)
+void trailer_add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
+			  struct list_head *arg_head)
 {
 	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
 	new_item->token = tok;
 	new_item->value = val;
 	duplicate_trailer_conf(&new_item->conf, conf);
-	if (new_trailer_item) {
-		if (new_trailer_item->where != WHERE_DEFAULT)
-			new_item->conf.where = new_trailer_item->where;
-		if (new_trailer_item->if_exists != EXISTS_DEFAULT)
-			new_item->conf.if_exists = new_trailer_item->if_exists;
-		if (new_trailer_item->if_missing != MISSING_DEFAULT)
-			new_item->conf.if_missing = new_trailer_item->if_missing;
-	}
 	list_add_tail(&new_item->list, arg_head);
 }
 
@@ -719,10 +737,10 @@ void parse_trailers_from_config(struct list_head *config_head)
 	list_for_each(pos, &conf_head) {
 		item = list_entry(pos, struct arg_item, list);
 		if (item->conf.command)
-			trailer_add_arg_item(config_head,
-					     xstrdup(token_from_item(item, NULL)),
+			trailer_add_arg_item(xstrdup(token_from_item(item, NULL)),
 					     xstrdup(""),
-					     &item->conf, NULL);
+					     &item->conf,
+					     config_head);
 	}
 }
 
@@ -755,10 +773,10 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 			strbuf_release(&sb);
 		} else {
 			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
-			trailer_add_arg_item(arg_head,
-					     strbuf_detach(&tok, NULL),
+			trailer_add_arg_item(strbuf_detach(&tok, NULL),
 					     strbuf_detach(&val, NULL),
-					     conf, tr);
+					     conf,
+					     arg_head);
 		}
 	}
 
@@ -1148,6 +1166,16 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
+void free_new_trailers(struct list_head *trailers)
+{
+	struct list_head *pos, *p;
+
+	list_for_each_safe(pos, p, trailers) {
+		list_del(pos);
+		free_arg_item(list_entry(pos, struct arg_item, list));
+	}
+}
+
 size_t trailer_block_start(struct trailer_block *trailer_block)
 {
 	return trailer_block->start;
diff --git a/trailer.h b/trailer.h
index d724263e4f6..8fcf1969a3c 100644
--- a/trailer.h
+++ b/trailer.h
@@ -46,9 +46,20 @@ struct new_trailer_item {
 	enum trailer_if_missing if_missing;
 };
 
+void trailer_conf_set(enum trailer_where where,
+		      enum trailer_if_exists if_exists,
+		      enum trailer_if_missing if_missing,
+		      struct trailer_conf *conf);
+
+struct trailer_conf *new_trailer_conf(void);
 void duplicate_trailer_conf(struct trailer_conf *dst,
 			    const struct trailer_conf *src);
 
+const char *trailer_default_separators(void);
+
+void trailer_add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
+			  struct list_head *arg_head);
+
 struct process_trailer_options {
 	int in_place;
 	int trim_empty;
@@ -95,6 +106,7 @@ void format_trailers(const struct process_trailer_options *opts,
 		     struct list_head *trailers,
 		     struct strbuf *out);
 void free_trailers(struct list_head *);
+void free_new_trailers(struct list_head *);
 
 /*
  * Convenience function to format the trailers from the commit msg "msg" into
-- 
gitgitgadget

[PATCH v3 09/10] trailer: delete obsolete argument handling code from API

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This commit was not squashed with its parent in order to keep the diff
separate (to make the additive changes in the parent easier to read).

Note that we remove the "new_trailer_item" struct, because it has some
overlap with "arg_item" struct that also deals with trailers coming from
the command line. The latter will be renamed to "trailer_template" in
the next patch.

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 39 ---------------------------------------
 trailer.h | 17 -----------------
 2 files changed, 56 deletions(-)

diff --git a/trailer.c b/trailer.c
index 19637ff295d..bf1d2eee550 100644
--- a/trailer.c
+++ b/trailer.c
@@ -744,45 +744,6 @@ void parse_trailers_from_config(struct list_head *config_head)
 	}
 }
 
-void parse_trailers_from_command_line_args(struct list_head *arg_head,
-					   struct list_head *new_trailer_head)
-{
-	struct strbuf tok = STRBUF_INIT;
-	struct strbuf val = STRBUF_INIT;
-	const struct trailer_conf *conf;
-	struct list_head *pos;
-
-	/*
-	 * In command-line arguments, '=' is accepted (in addition to the
-	 * separators that are defined).
-	 */
-	char *cl_separators = xstrfmt("=%s", separators);
-
-	/* Add an arg item for each trailer on the command line */
-	list_for_each(pos, new_trailer_head) {
-		struct new_trailer_item *tr =
-			list_entry(pos, struct new_trailer_item, list);
-		ssize_t separator_pos = find_separator(tr->text, cl_separators);
-
-		if (separator_pos == 0) {
-			struct strbuf sb = STRBUF_INIT;
-			strbuf_addstr(&sb, tr->text);
-			strbuf_trim(&sb);
-			error(_("empty trailer token in trailer '%.*s'"),
-			      (int) sb.len, sb.buf);
-			strbuf_release(&sb);
-		} else {
-			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
-			trailer_add_arg_item(strbuf_detach(&tok, NULL),
-					     strbuf_detach(&val, NULL),
-					     conf,
-					     arg_head);
-		}
-	}
-
-	free(cl_separators);
-}
-
 static const char *next_line(const char *str)
 {
 	const char *nl = strchrnul(str, '\n');
diff --git a/trailer.h b/trailer.h
index 8fcf1969a3c..5d4bacd9931 100644
--- a/trailer.h
+++ b/trailer.h
@@ -32,20 +32,6 @@ int trailer_set_where(enum trailer_where *item, const char *value);
 int trailer_set_if_exists(enum trailer_if_exists *item, const char *value);
 int trailer_set_if_missing(enum trailer_if_missing *item, const char *value);
 
-/*
- * A list that represents newly-added trailers, such as those provided
- * with the --trailer command line option of git-interpret-trailers.
- */
-struct new_trailer_item {
-	struct list_head list;
-
-	const char *text;
-
-	enum trailer_where where;
-	enum trailer_if_exists if_exists;
-	enum trailer_if_missing if_missing;
-};
-
 void trailer_conf_set(enum trailer_where where,
 		      enum trailer_if_exists if_exists,
 		      enum trailer_if_missing if_missing,
@@ -79,9 +65,6 @@ struct process_trailer_options {
 
 void parse_trailers_from_config(struct list_head *config_head);
 
-void parse_trailers_from_command_line_args(struct list_head *arg_head,
-					   struct list_head *new_trailer_head);
-
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
-- 
gitgitgadget

[PATCH v3 10/10] trailer: introduce "template" term for readability

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

The term "arg_item" is ambiguous because we use it to hold data for

  (1) trailers specified as command line arguments (in
      builtin/interpret-trailers.c), and

  (2) trailers specified in configuration,

and these are both used to ultimately insert new trailers (based
on the contents of arg_item, acting as a kind of template) into some
other set of existing trailers (such as those found in a trailer block
inside a log message) that have already been parsed.

Rename "arg_item" to "trailer_template". This necessitates further
renames to make the functions that act on these templates match the data
structures (parameters) they act on:

  - [*] add_arg_to_input_list()      to apply_template_to_trailers()
  - [*] apply_arg_if_exists()        to maybe_add_if_exists()
  - [*] apply_arg_if_missing()       to maybe_add_if_missing()
  -     apply_command()              to run_command_from_template()
  - [*] apply_item_command()         to populate_template_value()
  -     free_arg_item()              to free_template() (non-API function)
  -     free_new_trailers()          to free_trailer_templates() (API function)
  -     get_conf_item()              to get_or_add_template_by()
  -     option_parse_trailer()       to option_parse_trailer_template()
  -     parse_trailers_from_config() to parse_trailer_templates_from_config()
  - [*] process_trailers_lists()     to apply_trailer_templates()
  -     token_from_item()            to token_from_template()
  -     token_matches_item           to token_matches_template
  - [*] trailer_add_arg_item()       to add_trailer_template()
  -     trailer_from_arg()           to trailer_from()
  - [*] check_if_different()         (reorder parameters only)
  - [*] find_same_and_apply_arg()    (reorder parameters only)

Reorder parameters where appropriate; these functions have been marked
with an asterisk ([*]).

This removes the "arg" terminology (standing for "CLI arguments") from
the trailer implementation, which makes sense because trailers
themselves have nothing to do with CLI argument handling.

Also note that these renames expose the previously liberal use of
"trailer" to mean both trailers we read from the input text (trailer
block) and trailer templates that are defined as CLI args or
configurations. Some functions implied a single action when they could
do two different things, so introduce words like "maybe" and "or" to
unmask their behavior.

In summary this patch renames and reorders parameters for readability,
without any behavioral change. We don't rename
find_same_and_apply_arg(), because it will be refactored soon. As an
added benefit, we no longer use the term "arg" to mean "CLI arguments"
in the trailer implementation, because trailers themselves should not be
concerned about CLI arguments (this is the domain of the
interpret-trailers builtin).

For parse_trailers_from_config() (renamed to
parse_trailer_templates_from_config()), add a NEEDSWORK discussion about
how the deprecated trailer.*.command configuration option is oddly more
featureful than trailer.*.cmd (if we were to remove support for
trailer.*.command, users would not be able to replicate the behavior
with trailer.*.cmd and lose out on functionality).

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  48 +++---
 trailer.c                    | 292 +++++++++++++++++++----------------
 trailer.h                    |  11 +-
 3 files changed, 192 insertions(+), 159 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 9a902012912..aae3f3119df 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -47,10 +47,14 @@ static int option_parse_if_missing(const struct option *opt,
 
 static char *cl_separators;
 
-static int option_parse_trailer(const struct option *opt,
-				   const char *arg, int unset)
+/*
+ * Interpret "--trailer ..." as trailer templates (trailers we want to add into
+ * the input text).
+ */
+static int option_parse_trailer_template(const struct option *opt,
+					 const char *arg, int unset)
 {
-	struct list_head *trailers = opt->value;
+	struct list_head *templates = opt->value;
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
 	const struct trailer_conf *conf;
@@ -58,7 +62,7 @@ static int option_parse_trailer(const struct option *opt,
 	ssize_t separator_pos;
 
 	if (unset) {
-		free_new_trailers(trailers);
+		free_trailer_templates(templates);
 		return 0;
 	}
 
@@ -75,10 +79,10 @@ static int option_parse_trailer(const struct option *opt,
 		 */
 		trailer_conf_set(where, if_exists, if_missing, conf_current);
 
-		trailer_add_arg_item(strbuf_detach(&tok, NULL),
+		add_trailer_template(strbuf_detach(&tok, NULL),
 				     strbuf_detach(&val, NULL),
 				     conf_current,
-				     trailers);
+				     templates);
 	} else {
 		struct strbuf sb = STRBUF_INIT;
 		strbuf_addstr(&sb, arg);
@@ -148,10 +152,10 @@ static void read_input_file(struct strbuf *sb, const char *file)
 }
 
 static void interpret_trailers(const struct process_trailer_options *opts,
-			       struct list_head *arg_trailers,
+			       struct list_head *templates,
 			       const char *file)
 {
-	LIST_HEAD(head);
+	LIST_HEAD(trailers_from_sb);
 	struct strbuf sb = STRBUF_INIT;
 	struct strbuf tb = STRBUF_INIT;
 	struct trailer_block *trailer_block;
@@ -162,7 +166,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	if (opts->in_place)
 		outfile = create_in_place_tempfile(file);
 
-	trailer_block = parse_trailers(opts, sb.buf, &head);
+	trailer_block = parse_trailers(opts, sb.buf, &trailers_from_sb);
 
 	/* Print the lines before the trailer block */
 	if (!opts->only_trailers)
@@ -173,15 +177,15 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 
 
 	if (!opts->only_input) {
-		process_trailers_lists(&head, arg_trailers);
+		apply_trailer_templates(templates, &trailers_from_sb);
 	}
 
 	/* Print trailer block. */
-	format_trailers(opts, &head, &tb);
+	format_trailers(opts, &trailers_from_sb, &tb);
 	fwrite(tb.buf, 1, tb.len, outfile);
 	strbuf_release(&tb);
 
-	free_trailers(&head);
+	free_trailers(&trailers_from_sb);
 
 	/* Print the lines after the trailer block as is */
 	if (!opts->only_trailers)
@@ -199,8 +203,8 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
-	LIST_HEAD(configured_trailers);
-	LIST_HEAD(arg_trailers);
+	LIST_HEAD(configured_templates);
+	LIST_HEAD(templates);
 
 	struct option options[] = {
 		OPT_BOOL(0, "in-place", &opts.in_place, N_("edit files in place")),
@@ -219,8 +223,8 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 		OPT_CALLBACK_F(0, "parse", &opts, NULL, N_("alias for --only-trailers --only-input --unfold"),
 			PARSE_OPT_NOARG | PARSE_OPT_NONEG, parse_opt_parse),
 		OPT_BOOL(0, "no-divider", &opts.no_divider, N_("do not treat \"---\" as the end of input")),
-		OPT_CALLBACK(0, "trailer", &arg_trailers, N_("trailer"),
-				N_("trailer(s) to add"), option_parse_trailer),
+		OPT_CALLBACK(0, "trailer", &templates, N_("trailer"),
+				N_("trailer(s) to add"), option_parse_trailer_template),
 		OPT_END()
 	};
 
@@ -228,7 +232,7 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 	trailer_config_init();
 
 	if (!opts.only_input) {
-		parse_trailers_from_config(&configured_trailers);
+		parse_trailer_templates_from_config(&configured_templates);
 	}
 
 	/*
@@ -242,25 +246,25 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 
 	free(cl_separators);
 
-	if (opts.only_input && !list_empty(&arg_trailers))
+	if (opts.only_input && !list_empty(&templates))
 		usage_msg_opt(
 			_("--trailer with --only-input does not make sense"),
 			git_interpret_trailers_usage,
 			options);
 
-	list_splice(&configured_trailers, &arg_trailers);
+	list_splice(&configured_templates, &templates);
 
 	if (argc) {
 		int i;
 		for (i = 0; i < argc; i++)
-			interpret_trailers(&opts, &arg_trailers, argv[i]);
+			interpret_trailers(&opts, &templates, argv[i]);
 	} else {
 		if (opts.in_place)
 			die(_("no input file given for in-place editing"));
-		interpret_trailers(&opts, &arg_trailers, NULL);
+		interpret_trailers(&opts, &templates, NULL);
 	}
 
-	free_new_trailers(&arg_trailers);
+	free_trailer_templates(&templates);
 
 	return 0;
 }
diff --git a/trailer.c b/trailer.c
index bf1d2eee550..a739e2ada86 100644
--- a/trailer.c
+++ b/trailer.c
@@ -55,14 +55,14 @@ struct trailer_item {
 	char *value;
 };
 
-struct arg_item {
+struct trailer_template {
 	struct list_head list;
 	char *token;
 	char *value;
 	struct trailer_conf conf;
 };
 
-static LIST_HEAD(conf_head);
+static LIST_HEAD(templates_from_conf);
 
 static char *separators = ":";
 
@@ -105,7 +105,7 @@ static size_t token_len_without_separator(const char *token, size_t len)
 	return len;
 }
 
-static int same_token(struct trailer_item *a, struct arg_item *b)
+static int same_token(struct trailer_item *a, struct trailer_template *b)
 {
 	size_t a_len, b_len, min_len;
 
@@ -119,12 +119,12 @@ static int same_token(struct trailer_item *a, struct arg_item *b)
 	return !strncasecmp(a->token, b->token, min_len);
 }
 
-static int same_value(struct trailer_item *a, struct arg_item *b)
+static int same_value(struct trailer_item *a, struct trailer_template *b)
 {
 	return !strcasecmp(a->value, b->value);
 }
 
-static int same_trailer(struct trailer_item *a, struct arg_item *b)
+static int same_trailer(struct trailer_item *a, struct trailer_template *b)
 {
 	return same_token(a, b) && same_value(a, b);
 }
@@ -151,15 +151,15 @@ static void free_trailer_item(struct trailer_item *item)
 	free(item);
 }
 
-static void free_arg_item(struct arg_item *item)
+static void free_template(struct trailer_template *template)
 {
-	free(item->conf.name);
-	free(item->conf.key);
-	free(item->conf.command);
-	free(item->conf.cmd);
-	free(item->token);
-	free(item->value);
-	free(item);
+	free(template->conf.name);
+	free(template->conf.key);
+	free(template->conf.command);
+	free(template->conf.cmd);
+	free(template->token);
+	free(template->value);
+	free(template);
 }
 
 static char last_non_space_char(const char *s)
@@ -171,36 +171,36 @@ static char last_non_space_char(const char *s)
 	return '\0';
 }
 
-static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
+static struct trailer_item *trailer_from(struct trailer_template *template)
 {
 	struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
-	new_item->token = arg_tok->token;
-	new_item->value = arg_tok->value;
-	arg_tok->token = arg_tok->value = NULL;
-	free_arg_item(arg_tok);
+	new_item->token = template->token;
+	new_item->value = template->value;
+	template->token = template->value = NULL;
+	free_template(template);
 	return new_item;
 }
 
-static void add_arg_to_input_list(struct trailer_item *on_tok,
-				  struct arg_item *arg_tok)
+static void apply_template_to_trailers(struct trailer_template *template,
+				       struct trailer_item *on_tok)
 {
-	int aoe = after_or_end(arg_tok->conf.where);
-	struct trailer_item *to_add = trailer_from_arg(arg_tok);
+	int aoe = after_or_end(template->conf.where);
+	struct trailer_item *to_add = trailer_from(template);
 	if (aoe)
 		list_add(&to_add->list, &on_tok->list);
 	else
 		list_add_tail(&to_add->list, &on_tok->list);
 }
 
-static int check_if_different(struct trailer_item *in_tok,
-			      struct arg_item *arg_tok,
-			      int check_all,
-			      struct list_head *head)
+static int check_if_different(struct trailer_template *template,
+			      struct trailer_item *in_tok,
+			      struct list_head *head,
+			      int check_all)
 {
-	enum trailer_where where = arg_tok->conf.where;
+	enum trailer_where where = template->conf.where;
 	struct list_head *next_head;
 	do {
-		if (same_trailer(in_tok, arg_tok))
+		if (same_trailer(in_tok, template))
 			return 0;
 		/*
 		 * if we want to add a trailer after another one,
@@ -215,7 +215,8 @@ static int check_if_different(struct trailer_item *in_tok,
 	return 1;
 }
 
-static char *apply_command(struct trailer_conf *conf, const char *arg)
+static char *run_command_from_template(struct trailer_conf *conf,
+				       const char *arg)
 {
 	struct strbuf cmd = STRBUF_INIT;
 	struct strbuf buf = STRBUF_INIT;
@@ -250,133 +251,142 @@ static char *apply_command(struct trailer_conf *conf, const char *arg)
 	return result;
 }
 
-static void apply_item_command(struct trailer_item *in_tok, struct arg_item *arg_tok)
+/*
+ * Prepare the template by running the command (if any) requested by the
+ * template in order to populate the template's value field.
+ */
+static void populate_template_value(struct trailer_template *template,
+				    struct trailer_item *in_tok)
 {
-	if (arg_tok->conf.command || arg_tok->conf.cmd) {
+	if (template->conf.command || template->conf.cmd) {
+		/*
+		 * Determine argument to pass into the command.
+		 */
 		const char *arg;
-		if (arg_tok->value && arg_tok->value[0]) {
-			arg = arg_tok->value;
+		if (template->value && template->value[0]) {
+			arg = template->value;
 		} else {
 			if (in_tok && in_tok->value)
 				arg = xstrdup(in_tok->value);
 			else
 				arg = xstrdup("");
 		}
-		arg_tok->value = apply_command(&arg_tok->conf, arg);
+		template->value = run_command_from_template(&template->conf,
+							    arg);
 		free((char *)arg);
 	}
 }
 
-static void apply_arg_if_exists(struct trailer_item *in_tok,
-				struct arg_item *arg_tok,
+static void maybe_add_if_exists(struct trailer_template *template,
+				struct trailer_item *in_tok,
 				struct trailer_item *on_tok,
-				struct list_head *head)
+				struct list_head *trailers)
 {
-	switch (arg_tok->conf.if_exists) {
+	switch (template->conf.if_exists) {
 	case EXISTS_DO_NOTHING:
-		free_arg_item(arg_tok);
+		free_template(template);
 		break;
 	case EXISTS_REPLACE:
-		apply_item_command(in_tok, arg_tok);
-		add_arg_to_input_list(on_tok, arg_tok);
+		populate_template_value(template, in_tok);
+		apply_template_to_trailers(template, on_tok);
 		list_del(&in_tok->list);
 		free_trailer_item(in_tok);
 		break;
 	case EXISTS_ADD:
-		apply_item_command(in_tok, arg_tok);
-		add_arg_to_input_list(on_tok, arg_tok);
+		populate_template_value(template, in_tok);
+		apply_template_to_trailers(template, on_tok);
 		break;
 	case EXISTS_ADD_IF_DIFFERENT:
-		apply_item_command(in_tok, arg_tok);
-		if (check_if_different(in_tok, arg_tok, 1, head))
-			add_arg_to_input_list(on_tok, arg_tok);
+		populate_template_value(template, in_tok);
+		if (check_if_different(template, in_tok, trailers, 1))
+			apply_template_to_trailers(template, on_tok);
 		else
-			free_arg_item(arg_tok);
+			free_template(template);
 		break;
 	case EXISTS_ADD_IF_DIFFERENT_NEIGHBOR:
-		apply_item_command(in_tok, arg_tok);
-		if (check_if_different(on_tok, arg_tok, 0, head))
-			add_arg_to_input_list(on_tok, arg_tok);
+		populate_template_value(template, in_tok);
+		if (check_if_different(template, on_tok, trailers, 0))
+			apply_template_to_trailers(template, on_tok);
 		else
-			free_arg_item(arg_tok);
+			free_template(template);
 		break;
 	default:
 		BUG("trailer.c: unhandled value %d",
-		    arg_tok->conf.if_exists);
+		    template->conf.if_exists);
 	}
 }
 
-static void apply_arg_if_missing(struct list_head *head,
-				 struct arg_item *arg_tok)
+static void maybe_add_if_missing(struct trailer_template *template,
+				 struct list_head *trailers)
 {
 	enum trailer_where where;
 	struct trailer_item *to_add;
 
-	switch (arg_tok->conf.if_missing) {
+	switch (template->conf.if_missing) {
 	case MISSING_DO_NOTHING:
-		free_arg_item(arg_tok);
+		free_template(template);
 		break;
 	case MISSING_ADD:
-		where = arg_tok->conf.where;
-		apply_item_command(NULL, arg_tok);
-		to_add = trailer_from_arg(arg_tok);
+		where = template->conf.where;
+		populate_template_value(template, NULL);
+		to_add = trailer_from(template);
 		if (after_or_end(where))
-			list_add_tail(&to_add->list, head);
+			list_add_tail(&to_add->list, trailers);
 		else
-			list_add(&to_add->list, head);
+			list_add(&to_add->list, trailers);
 		break;
 	default:
 		BUG("trailer.c: unhandled value %d",
-		    arg_tok->conf.if_missing);
+		    template->conf.if_missing);
 	}
 }
 
-static int find_same_and_apply_arg(struct list_head *head,
-				   struct arg_item *arg_tok)
+static int find_same_and_apply_arg(struct trailer_template *template,
+				   struct list_head *trailers)
 {
 	struct list_head *pos;
 	struct trailer_item *in_tok;
 	struct trailer_item *on_tok;
 
-	enum trailer_where where = arg_tok->conf.where;
+	enum trailer_where where = template->conf.where;
 	int middle = (where == WHERE_AFTER) || (where == WHERE_BEFORE);
 	int backwards = after_or_end(where);
 	struct trailer_item *start_tok;
 
-	if (list_empty(head))
+	if (list_empty(trailers))
 		return 0;
 
-	start_tok = list_entry(backwards ? head->prev : head->next,
+	start_tok = list_entry(backwards ? trailers->prev : trailers->next,
 			       struct trailer_item,
 			       list);
 
-	list_for_each_dir(pos, head, backwards) {
+	list_for_each_dir(pos, trailers, backwards) {
 		in_tok = list_entry(pos, struct trailer_item, list);
-		if (!same_token(in_tok, arg_tok))
+		if (!same_token(in_tok, template))
 			continue;
 		on_tok = middle ? in_tok : start_tok;
-		apply_arg_if_exists(in_tok, arg_tok, on_tok, head);
+		maybe_add_if_exists(template, in_tok, on_tok, trailers);
 		return 1;
 	}
 	return 0;
 }
 
-void process_trailers_lists(struct list_head *head,
-			    struct list_head *arg_head)
+void apply_trailer_templates(struct list_head *templates,
+			     struct list_head *trailers)
 {
 	struct list_head *pos, *p;
-	struct arg_item *arg_tok;
+	struct trailer_template *template;
 
-	list_for_each_safe(pos, p, arg_head) {
+	list_for_each_safe(pos, p, templates) {
 		int applied = 0;
-		arg_tok = list_entry(pos, struct arg_item, list);
+		template = list_entry(pos, struct trailer_template, list);
 
 		list_del(pos);
 
-		applied = find_same_and_apply_arg(head, arg_tok);
+		applied = find_same_and_apply_arg(template, trailers);
 
 		if (!applied)
-			apply_arg_if_missing(head, arg_tok);
+			maybe_add_if_missing(template, trailers);
 	}
 }
 
@@ -458,26 +468,26 @@ void duplicate_trailer_conf(struct trailer_conf *dst,
 	dst->cmd = xstrdup_or_null(src->cmd);
 }
 
-static struct arg_item *get_conf_item(const char *name)
+static struct trailer_template *get_or_add_template_by(const char *name)
 {
 	struct list_head *pos;
-	struct arg_item *item;
+	struct trailer_template *template;
 
-	/* Look up item with same name */
-	list_for_each(pos, &conf_head) {
-		item = list_entry(pos, struct arg_item, list);
-		if (!strcasecmp(item->conf.name, name))
-			return item;
+	/* Look up template with same name. */
+	list_for_each(pos, &templates_from_conf) {
+		template = list_entry(pos, struct trailer_template, list);
+		if (!strcasecmp(template->conf.name, name))
+			return template;
 	}
 
-	/* Item does not already exists, create it */
-	CALLOC_ARRAY(item, 1);
-	duplicate_trailer_conf(&item->conf, &default_trailer_conf);
-	item->conf.name = xstrdup(name);
+	/* Template does not already exist; create it. */
+	CALLOC_ARRAY(template, 1);
+	duplicate_trailer_conf(&template->conf, &default_trailer_conf);
+	template->conf.name = xstrdup(name);
 
-	list_add_tail(&item->list, &conf_head);
+	list_add_tail(&template->list, &templates_from_conf);
 
-	return item;
+	return template;
 }
 
 enum trailer_info_type { TRAILER_KEY, TRAILER_COMMAND, TRAILER_CMD,
@@ -535,7 +545,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 			      void *cb UNUSED)
 {
 	const char *trailer_item, *variable_name;
-	struct arg_item *item;
+	struct trailer_template *template;
 	struct trailer_conf *conf;
 	char *name = NULL;
 	enum trailer_info_type type;
@@ -560,8 +570,8 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	if (!name)
 		return 0;
 
-	item = get_conf_item(name);
-	conf = &item->conf;
+	template = get_or_add_template_by(name);
+	conf = &template->conf;
 	free(name);
 
 	switch (type) {
@@ -618,20 +628,22 @@ void trailer_config_init(void)
 	configured = 1;
 }
 
-static const char *token_from_item(struct arg_item *item, char *tok)
+static const char *token_from_template(struct trailer_template *template, char *tok)
 {
-	if (item->conf.key)
-		return item->conf.key;
+	if (template->conf.key)
+		return template->conf.key;
 	if (tok)
 		return tok;
-	return item->conf.name;
+	return template->conf.name;
 }
 
-static int token_matches_item(const char *tok, struct arg_item *item, size_t tok_len)
+static int token_matches_template(const char *tok,
+				  struct trailer_template *template,
+				  size_t tok_len)
 {
-	if (!strncasecmp(tok, item->conf.name, tok_len))
+	if (!strncasecmp(tok, template->conf.name, tok_len))
 		return 1;
-	return item->conf.key ? !strncasecmp(tok, item->conf.key, tok_len) : 0;
+	return template->conf.key ? !strncasecmp(tok, template->conf.key, tok_len) : 0;
 }
 
 /*
@@ -677,7 +689,7 @@ void parse_trailer(const char *line, ssize_t separator_pos,
 		   struct strbuf *tok, struct strbuf *val,
 		   const struct trailer_conf **conf)
 {
-	struct arg_item *item;
+	struct trailer_template *template;
 	size_t tok_len;
 	struct list_head *pos;
 
@@ -695,13 +707,13 @@ void parse_trailer(const char *line, ssize_t separator_pos,
 	tok_len = token_len_without_separator(tok->buf, tok->len);
 	if (conf)
 		*conf = &default_trailer_conf;
-	list_for_each(pos, &conf_head) {
-		item = list_entry(pos, struct arg_item, list);
-		if (token_matches_item(tok->buf, item, tok_len)) {
+	list_for_each(pos, &templates_from_conf) {
+		template = list_entry(pos, struct trailer_template, list);
+		if (token_matches_template(tok->buf, template, tok_len)) {
 			char *tok_buf = strbuf_detach(tok, NULL);
 			if (conf)
-				*conf = &item->conf;
-			strbuf_addstr(tok, token_from_item(item, tok_buf));
+				*conf = &template->conf;
+			strbuf_addstr(tok, token_from_template(template, tok_buf));
 			free(tok_buf);
 			break;
 		}
@@ -718,29 +730,41 @@ static struct trailer_item *add_trailer_item(struct list_head *head, char *tok,
 	return new_item;
 }
 
-void trailer_add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
-			  struct list_head *arg_head)
+void add_trailer_template(char *tok, char *val, const struct trailer_conf *conf,
+			  struct list_head *templates)
 {
-	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
-	new_item->token = tok;
-	new_item->value = val;
-	duplicate_trailer_conf(&new_item->conf, conf);
-	list_add_tail(&new_item->list, arg_head);
+	struct trailer_template *template = xcalloc(1, sizeof(*template));
+	template->token = tok;
+	template->value = val;
+	duplicate_trailer_conf(&template->conf, conf);
+	list_add_tail(&template->list, templates);
 }
 
-void parse_trailers_from_config(struct list_head *config_head)
+void parse_trailer_templates_from_config(struct list_head *config_head)
 {
-	struct arg_item *item;
+	struct trailer_template *template;
 	struct list_head *pos;
 
-	/* Add an arg item for each configured trailer with a command */
-	list_for_each(pos, &conf_head) {
-		item = list_entry(pos, struct arg_item, list);
-		if (item->conf.command)
-			trailer_add_arg_item(xstrdup(token_from_item(item, NULL)),
-					     xstrdup(""),
-					     &item->conf,
-					     config_head);
+	/*
+	 * Get configured templates with a ".command" option.
+	 *
+	 * NEEDSWORK: If the interpret-trailers builtin sees a
+	 * "trailer.foo.command = ..." setting, then the "foo" trailer will
+	 * always be inserted, even if "--trailer foo" is not provided.
+	 * Considering how ".command" is deprecated, it is a bit strange to see
+	 * it getting special treatment like this over ".cmd". Instead, we
+	 * should add a new option that explicitly lets the user decide if the
+	 * configured trailer should always be added automatically, or if it
+	 * should only be added if "--trailer foo" is provided (default).
+	 * Then we can collect configured trailers that have either ".command"
+	 * or ".cmd" below, instead of just ".command".
+	 */
+	list_for_each(pos, &templates_from_conf) {
+		template = list_entry(pos, struct trailer_template, list);
+		if (template->conf.command)
+			add_trailer_template(xstrdup(token_from_template(template,
+									 NULL)),
+					     xstrdup(""), &template->conf, config_head);
 	}
 }
 
@@ -852,7 +876,7 @@ static size_t find_trailer_block_start(const char *buf, size_t len)
 	 * Get the start of the trailers by looking starting from the end for a
 	 * blank line before a set of non-blank lines that (i) are all
 	 * trailers, or (ii) contains at least one Git-generated trailer and
-	 * consists of at least 25% trailers.
+	 * consists of at least 25% configured trailers.
 	 */
 	for (l = last_line(buf, len);
 	     l >= end_of_title;
@@ -896,10 +920,16 @@ static size_t find_trailer_block_start(const char *buf, size_t len)
 			possible_continuation_lines = 0;
 			if (recognized_prefix)
 				continue;
-			list_for_each(pos, &conf_head) {
-				struct arg_item *item;
-				item = list_entry(pos, struct arg_item, list);
-				if (token_matches_item(bol, item,
+			/*
+			 * The templates here are not used for actually
+			 * adding trailers anywhere, but instead to help us
+			 * identify trailer lines by comparing their keys with
+			 * those found in configured templates.
+			 */
+			list_for_each(pos, &templates_from_conf) {
+				struct trailer_template *template;
+				template = list_entry(pos, struct trailer_template, list);
+				if (token_matches_template(bol, template,
 						       separator_pos)) {
 					recognized_prefix = 1;
 					break;
@@ -1127,13 +1157,13 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
-void free_new_trailers(struct list_head *trailers)
+void free_trailer_templates(struct list_head *trailer_templates)
 {
 	struct list_head *pos, *p;
 
-	list_for_each_safe(pos, p, trailers) {
+	list_for_each_safe(pos, p, trailer_templates) {
 		list_del(pos);
-		free_arg_item(list_entry(pos, struct arg_item, list));
+		free_template(list_entry(pos, struct trailer_template, list));
 	}
 }
 
diff --git a/trailer.h b/trailer.h
index 5d4bacd9931..79aa123478d 100644
--- a/trailer.h
+++ b/trailer.h
@@ -43,8 +43,8 @@ void duplicate_trailer_conf(struct trailer_conf *dst,
 
 const char *trailer_default_separators(void);
 
-void trailer_add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
-			  struct list_head *arg_head);
+void add_trailer_template(char *tok, char *val, const struct trailer_conf *conf,
+			  struct list_head *templates);
 
 struct process_trailer_options {
 	int in_place;
@@ -63,10 +63,9 @@ struct process_trailer_options {
 
 #define PROCESS_TRAILER_OPTIONS_INIT {0}
 
-void parse_trailers_from_config(struct list_head *config_head);
+void parse_trailer_templates_from_config(struct list_head *config_head);
 
-void process_trailers_lists(struct list_head *head,
-			    struct list_head *arg_head);
+void apply_trailer_templates(struct list_head *templates, struct list_head *trailers_head);
 
 ssize_t find_separator(const char *line, const char *separators);
 
@@ -89,7 +88,7 @@ void format_trailers(const struct process_trailer_options *opts,
 		     struct list_head *trailers,
 		     struct strbuf *out);
 void free_trailers(struct list_head *);
-void free_new_trailers(struct list_head *);
+void free_trailer_templates(struct list_head *);
 
 /*
  * Convenience function to format the trailers from the commit msg "msg" into
-- 
gitgitgadget

Re: [PATCH v3 02/10] trailer: move interpret_trailers() to interpret-trailers.c

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> The interpret-trailers.c builtin is the only place we need to call
> interpret_trailers(), so move its definition there.

A few helper functions that are only called by interpret_trailers()
are also moved, naturally.  I would have less surprised to see
the addtion near the beginning of builtin/interpret-trailers.c if
this part said:

    ..., so move its definition there, together with a few helper
    functions called only by it, and remove its external declaration
    from <trailer.h>.

> Delete the corresponding declaration from trailer.h, which then forces
> us to expose the working innards of that function.

This was a bit confusing, at least to me.  The reason why several
other helper functions that are called by interpret_trailers() need
to be declared in trailer.h is not because the declaration of
interpret_trailers() is deleted from trailer.h but that is how I
read the above.

    Several helper functions that are called by interpret_trailers()
    remain in trailer.c because other callers in the same file still
    call them, so add declaration for them to <trailer.h>.

> This enriches
> trailer.h with a more granular API, which can then be unit-tested in the
> future (because interpret_trailers() by itself does too many things to
> be able to be easily unit-tested).
>
> Take this opportunity to demote some file-handling functions out of the
> trailer API implementation, as these have nothing to do with trailers.
>
> Signed-off-by: Linus Arver <linusa@google.com>
> ---
>  builtin/interpret-trailers.c |  94 +++++++++++++++++++++++++++
>  trailer.c                    | 120 ++++-------------------------------
>  trailer.h                    |  20 +++++-

Together with the preparatory renaming in the previous step, this
made quite a pleasant read.  Thanks.

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Josh Steadmon]

On 2024.01.31 01:22, Linus Arver via GitGitGadget wrote:
> This unification will allow us to delete the format_trailer_info() and
> print_tok_val() functions in the next patch. They are not deleted here
> in order to keep the diff small.

Needs to be removed after squashing v2 patch 4 :)

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> Currently have two functions for formatting trailers exposed in
> trailer.h:
>
>     void format_trailers(FILE *outfile, struct list_head *head,
>                         const struct process_trailer_options *opts);
>
>     void format_trailers_from_commit(struct strbuf *out, const char *msg,
>                                     const struct process_trailer_options *opts);
>
> and previously these functions, although similar enough (even taking the
> same process_trailer_options struct pointer), did not build on each
> other.
>
> Make format_trailers_from_commit() rely on format_trailers(). Teach
> format_trailers() to process trailers with the additional
> process_trailer_options fields like opts->key_only which is only used by
> format_trailers_from_commit() and not builtin/interpret-trailers.c.
> While we're at it, reorder parameters to put the trailer processing
> options first, and the out parameter (strbuf we write into) at the end.
>
> This unification will allow us to delete the format_trailer_info() and
> print_tok_val() functions in the next patch. They are not deleted here
> in order to keep the diff small.
>
> Helped-by: Junio C Hamano <gitster@pobox.com>
> Signed-off-by: Linus Arver <linusa@google.com>
> ---

I am not sure how I helped this particularly (say, compared to all
the other patches on the list from everybody), though.

The updated format_trailers() does so much more than the original
and does so quite differently, it is hard to tell if the existing
caller is getting the same behaviour (modulo obviously it now needs
to prepare a strbuf and print the resulting string in the strbuf,
which is expected) as before.

The conversion for the sole existing caller looks like this, ...

>  	LIST_HEAD(head);
>  	struct strbuf sb = STRBUF_INIT;
> +	struct strbuf trailer_block = STRBUF_INIT;
>  	struct trailer_info info;
> ...
> -	format_trailers(opts, &head, outfile);
> +	/* Print trailer block. */
> +	format_trailers(opts, &head, &trailer_block);
> +	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
> +	strbuf_release(&trailer_block);

... and it is very straight-forward.


But what it called was this:

> -static void print_tok_val(FILE *outfile, const char *tok, const char *val)
> -{
> -	char c;
> -
> -	if (!tok) {
> -		fprintf(outfile, "%s\n", val);
> -		return;
> -	}
> -
> -	c = last_non_space_char(tok);
> -	if (!c)
> -		return;
> -	if (strchr(separators, c))
> -		fprintf(outfile, "%s%s\n", tok, val);
> -	else
> -		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
> -}
> -
> -void format_trailers(const struct process_trailer_options *opts,
> -		     struct list_head *trailers, FILE *outfile)
> -{
> -	struct list_head *pos;
> -	struct trailer_item *item;
> -	list_for_each(pos, trailers) {
> -		item = list_entry(pos, struct trailer_item, list);
> -		if ((!opts->trim_empty || strlen(item->value) > 0) &&
> -		    (!opts->only_trailers || item->token))
> -			print_tok_val(outfile, item->token, item->value);
> -	}
> -}

i.e. iterate over trailers, and sometimes call print_tok_val(),
which does not do all that much.  Print only the val, or when tok is
not empty, show "tok: val" while taking into account the possibility
that the last byte of tok may already be the separator.

But the new thing is way more complex.

> +void format_trailers(const struct process_trailer_options *opts,
> +		     struct list_head *trailers,
> +		     struct strbuf *out)
> +{
> +	struct list_head *pos;
> +	struct trailer_item *item;
> +	int need_separator = 0;
> +
> +	list_for_each(pos, trailers) {
> +		item = list_entry(pos, struct trailer_item, list);
> +		if (item->token) {
> +			char c;
> +
> +			struct strbuf tok = STRBUF_INIT;
> +			struct strbuf val = STRBUF_INIT;
> +			strbuf_addstr(&tok, item->token);
> +			strbuf_addstr(&val, item->value);
> +
> +			/*
> +			 * Skip key/value pairs where the value was empty. This
> +			 * can happen from trailers specified without a
> +			 * separator, like `--trailer "Reviewed-by"` (no
> +			 * corresponding value).
> +			 */
> +			if (opts->trim_empty && !strlen(item->value))
> +				continue;

OK, this matches the first condition to stay silent in the original.

> +			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {

For the original caller of format_trailers(), which is the
interpret_trailers(), .filter member is never set, so we always come
here, I presume.  cmd_interpret_trailers(), before calling
interpret_trailers() could affect the following members:

    .in_place .trim_empty .only_trailers .only_input .unfold .no_divider

so we should make sure this new implementation does not change the
behaviour from the original with various combination of these
options.

> +				if (opts->unfold)
> +					unfold_value(&val);

So, ... how would this affect an invocation of

    $ git interpret-trailers --unfold

which would have set opts.unfold to true in cmd_interpret_trailers()
and eventually called process_trailers() with it, which eventually
called print_all(), which conditionally called print_tok_val() but
never gave the val to unfold_value()?

 ... goes and digs a bit more ...

Ahhhh, original process_trailers() did this by calling
parse_trailers() that munged the value.  So its print_all()
codepath only had to print what has already been munged.

But then in this new code, do we unfold only here?  I thought that
in the new code you moved around, the caller, whose name is now
interpret_trailers(), still calls parse_trailers() and that calls
the unfold_value().  So, are we doing redundant work that may merely
be unneeded (if we are lucky) or could be harmful (if things, other
than the unfold thing I just noticed, that are done both in
parse_trailers() and this function are not idempotent)?

It could be that a bit more preparatory refactoring would clarify.
I dunno.

> +				if (opts->separator && need_separator)
> +					strbuf_addbuf(out, opts->separator);
> +				if (!opts->value_only)
> +					strbuf_addbuf(out, &tok);
> +				if (!opts->key_only && !opts->value_only) {
> +					if (opts->key_value_separator)
> +						strbuf_addbuf(out, opts->key_value_separator);
> +					else {
> +						c = last_non_space_char(tok.buf);
> +						if (c) {
> +							if (!strchr(separators, c))
> +								strbuf_addf(out, "%c ", separators[0]);
> +						}
> +					}
> +				}
> +				if (!opts->key_only)
> +					strbuf_addbuf(out, &val);
> +				if (!opts->separator)
> +					strbuf_addch(out, '\n');
> +
> +				need_separator = 1;
> +			}
> +
> +			strbuf_release(&tok);
> +			strbuf_release(&val);
> +		} else if (!opts->only_trailers) {
> +			if (opts->separator && need_separator) {
> +				strbuf_addbuf(out, opts->separator);
> +			}
> +			strbuf_addstr(out, item->value);
> +			if (opts->separator)
> +				strbuf_rtrim(out);
> +			else
> +				strbuf_addch(out, '\n');
> +
> +			need_separator = 1;
> +		}
> +
> +	}
> +}

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

>> +				if (opts->unfold)
>> +					unfold_value(&val);
>
> So, ... how would this affect an invocation of
>
>     $ git interpret-trailers --unfold
>
> which would have set opts.unfold to true in cmd_interpret_trailers()
> and eventually called process_trailers() with it, which eventually
> called print_all(), which conditionally called print_tok_val() but
> never gave the val to unfold_value()?
>
>  ... goes and digs a bit more ...
>
> Ahhhh, original process_trailers() did this by calling
> parse_trailers() that munged the value.  So its print_all()
> codepath only had to print what has already been munged.
>
> But then in this new code, do we unfold only here?  I thought that
> in the new code you moved around, the caller, whose name is now
> interpret_trailers(), still calls parse_trailers() and that calls
> the unfold_value().  So, are we doing redundant work that may merely
> be unneeded (if we are lucky) or could be harmful (if things, other
> than the unfold thing I just noticed, that are done both in
> parse_trailers() and this function are not idempotent)?

I was too confused by the code flow and resorted to tracing what
happens when "git interpret-trailers --unfold" is run in a way
similar to how t7513 does in gdb.  Shame shame.

In any case, the updated code does try to call unfold_value() in
format_trailers() on "val" that has already been unfolded in
parse_trailers().  This may not produce an incorrect result if
unfold_value() is truly idempotent (I do not know), but even if it
is correct for its handling of .unfold bit, this episode lowered my
confidence level on the new code significantly, as I do not know
what unintended effects [*] all the other new things done in this
function have, or if none of these unintended effects are
error-free.

> It could be that a bit more preparatory refactoring would clarify.
> I dunno.

Thanks.

[Footnote] 

 * I am assuming that calling unfold twice on the same value and
   relying for its idempotent-ness for correctness was not your
   intention.

Re: [PATCH v3 02/10] trailer: move interpret_trailers() to interpret-trailers.c

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> From: Linus Arver <linusa@google.com>
>>
>> The interpret-trailers.c builtin is the only place we need to call
>> interpret_trailers(), so move its definition there.
>
> A few helper functions that are only called by interpret_trailers()
> are also moved, naturally.  I would have less surprised to see
> the addtion near the beginning of builtin/interpret-trailers.c if
> this part said:
>
>     ..., so move its definition there, together with a few helper
>     functions called only by it, and remove its external declaration
>     from <trailer.h>.

Will add in next reroll.

>> Delete the corresponding declaration from trailer.h, which then forces
>> us to expose the working innards of that function.
>
> This was a bit confusing, at least to me.  The reason why several
> other helper functions that are called by interpret_trailers() need
> to be declared in trailer.h is not because the declaration of
> interpret_trailers() is deleted from trailer.h but that is how I
> read the above.
>
>     Several helper functions that are called by interpret_trailers()
>     remain in trailer.c because other callers in the same file still
>     call them, so add declaration for them to <trailer.h>.

I've adopted most of this language in the next reroll to make the
reasoning a bit easier to follow. Thanks.

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Linus Arver]

Josh Steadmon <steadmon@google.com> writes:

> On 2024.01.31 01:22, Linus Arver via GitGitGadget wrote:
>> This unification will allow us to delete the format_trailer_info() and
>> print_tok_val() functions in the next patch. They are not deleted here
>> in order to keep the diff small.
>
> Needs to be removed after squashing v2 patch 4 :)

Oops. Will update in next reroll, thanks.

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> From: Linus Arver <linusa@google.com>
>>
>> Currently have two functions for formatting trailers exposed in
>> trailer.h:
>>
>>     void format_trailers(FILE *outfile, struct list_head *head,
>>                         const struct process_trailer_options *opts);
>>
>>     void format_trailers_from_commit(struct strbuf *out, const char *msg,
>>                                     const struct process_trailer_options *opts);
>>
>> and previously these functions, although similar enough (even taking the
>> same process_trailer_options struct pointer), did not build on each
>> other.
>>
>> Make format_trailers_from_commit() rely on format_trailers(). Teach
>> format_trailers() to process trailers with the additional
>> process_trailer_options fields like opts->key_only which is only used by
>> format_trailers_from_commit() and not builtin/interpret-trailers.c.
>> While we're at it, reorder parameters to put the trailer processing
>> options first, and the out parameter (strbuf we write into) at the end.
>>
>> This unification will allow us to delete the format_trailer_info() and
>> print_tok_val() functions in the next patch. They are not deleted here
>> in order to keep the diff small.
>>
>> Helped-by: Junio C Hamano <gitster@pobox.com>
>> Signed-off-by: Linus Arver <linusa@google.com>
>> ---
>
> I am not sure how I helped this particularly (say, compared to all
> the other patches on the list from everybody), though.

You made a suggestion to put trailer processing options first for
function parameters, which I've adopted in this series (but also in the
larger series).

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> Junio C Hamano <gitster@pobox.com> writes:
>
>>> +				if (opts->unfold)
>>> +					unfold_value(&val);
>>
>> So, ... how would this affect an invocation of
>>
>>     $ git interpret-trailers --unfold
>>
>> which would have set opts.unfold to true in cmd_interpret_trailers()
>> and eventually called process_trailers() with it, which eventually
>> called print_all(), which conditionally called print_tok_val() but
>> never gave the val to unfold_value()?
>>
>>  ... goes and digs a bit more ...
>>
>> Ahhhh, original process_trailers() did this by calling
>> parse_trailers() that munged the value.  So its print_all()
>> codepath only had to print what has already been munged.
>>
>> But then in this new code, do we unfold only here?  I thought that
>> in the new code you moved around, the caller, whose name is now
>> interpret_trailers(), still calls parse_trailers() and that calls
>> the unfold_value().  So, are we doing redundant work that may merely
>> be unneeded (if we are lucky) or could be harmful (if things, other
>> than the unfold thing I just noticed, that are done both in
>> parse_trailers() and this function are not idempotent)?
>
> I was too confused by the code flow and resorted to tracing what
> happens when "git interpret-trailers --unfold" is run in a way
> similar to how t7513 does in gdb.  Shame shame.

In my larger series I've made the parser and formater much simpler
(removing nesting, calling helper functions, _only parsing once and only
once_, etc) to make both parsing and formatting much easier to
understand. While I am biased as the author of these refactors, I do
think they make the code simpler.

> In any case, the updated code does try to call unfold_value() in
> format_trailers() on "val" that has already been unfolded in
> parse_trailers().

Correct. But this was already the case before this series. IOW the
existing code assumes that this function is idempotent: we call
unfold_value() in parse_trailers() and again in format_trailer_info().

In my tests if I remove the redundant call to unfold_value() in
the new formatter (so that unfolding only happens during
parse_trailers()), all trailer-related tests still pass:

    prove -j47 t{7513,3507,7502,4205,3511,3428,6300}*.sh

FWIW in the larger series we prohibit the parser from making mutations
to the input (unfolding is one such mutation), and allow multiline
(folded) strings to be stored as is in "val", only unfolding the value
if we need to during formatting.

> This may not produce an incorrect result if
> unfold_value() is truly idempotent (I do not know), but even if it
> is correct for its handling of .unfold bit, this episode lowered my
> confidence level on the new code significantly, as I do not know
> what unintended effects [*] all the other new things done in this
> function have, or if none of these unintended effects are
> error-free.

I think dropping the redundant call to unfold_value() would help address
some of your concerns. Of course, all of the existing test cases pass
(with and without the redundant call). As for addressing _all_ (not just
some) concerns, I am not confident I can deliver that as an additional
patch or two to this series because ...

>> It could be that a bit more preparatory refactoring would clarify.
>> I dunno.

... this is basically what I've done in the larger series, because there
are many areas that could be simplified (not to mention addressing some
bugs some bugs are lurking around). Granted, those are more aggressive
refactorings, and are not "preparatory refactoring" at all.

I could just grow this series with another ~22 patches to include those
additional refactors, but I am hesitant about doing so, simply due to
the sheer number of them.

How would you like me to proceed, other than deleting the redundant
unfold_value() call in the next reroll?

For reference, here's a list of those 22 commits that build on this
series as a preview (roughly grouped on themes):

    trailer formatter: deprecate format_trailers()
    trailer formatter: introduce format_trailer_block()
    trailer parser: parse_trailer_v2() for '--trailer <...>' CLI arguments
    trailer parser: parse_trailer_v2() for configuration
    trailer parser: parse_trailer_v2() for trailer blocks
    trailer parser: introduce parse_trailer_v2()
    interpret-trailers: preserve trailers coming from the input
    trailer_block: remove trailer_strings member
    trailer_block: parse trailers in one place
    trailer_block: prepare to remove trailer_strings member
    trailer: make find_same_and_apply_arg() do one thing
    trailer: unify global configuration
    interpret-trailers: print error if given unrecognized arguments
    trailer: make entire iterator struct private
    trailer: rename terms for consistency
    trailer: use create_new_target_for() everywhere
    trailer: capture behavior of addIfDifferentNeighbor
    trailer: EXISTS_REPLACE preserves original position
    trailer: capture replacement behavior for multiple matches
    trailer: split template application into 2 steps
    trailer: free templates in one place
    trailer: template's final value does not depend on in_tok

and "trailer formatter: introduce format_trailer_block()" is the one
that refactors the unified formatter introduced in this patch to have
multiple smaller helper functions.

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Junio C Hamano]

Linus Arver <linusa@google.com> writes:

>> In any case, the updated code does try to call unfold_value() in
>> format_trailers() on "val" that has already been unfolded in
>> parse_trailers().
>
> Correct. But this was already the case before this series. IOW the
> existing code assumes that this function is idempotent: we call
> unfold_value() in parse_trailers() and again in format_trailer_info().

But not in format_trailers(), which is the theme of this step.  In
other words, the behaviour before and after this step of the
function are not the same (modulo that the new version stores the
output in a strbuf), and as the way the changes are presented here,
it is almost impossible to make sure that we are not introducing
regressions, without making such assumptions like "unfold_value() is
idempotent and we already rely on that fact elsewhere in the code".

It is not just "unfold", which was the first thing I happened to
notice, by the way.  There are many more lines in the new lines of
that function, doing things that the original version did not appear
to do, or doing in very different ways.

> I could just grow this series with another ~22 patches to include those
> additional refactors, but I am hesitant about doing so, simply due to
> the sheer number of them.

I actually do not mind at all if you started with a preliminary
clean-up series, and stopped the first batch somewhere in the middle
of the 20+ patches before even reaching any of these 10 patches we
see here, if that gives us a readable set of bite-sized changes that
prepare a solid foundation to rebuild things on top.  I am having a
feeling that not even a single person has reviewed them on list even
though we are already at the third iteration, which is quite
frustrating (and I would imagine that it would be frustrating for
you, too), and I suspect that the step like [v3 03/10] that makes
too large a change with too little explanation (and perhaps a bit of
"trust me, this does not change the behaviour") is one contributing
factor why people are afraid of touching it.

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> ..., if that gives us a readable set of bite-sized changes that
> prepare a solid foundation to rebuild things on top.  I am having a
> feeling that not even a single person has reviewed them on list even
> though we are already at the third iteration, which is quite
> frustrating (and I would imagine that it would be frustrating for
> you, too), ...

I guess "not even a single person" was a bit unfair, given that we
did see some minor comments from me and also Josh during the first
round.

But neither of two rounds saw any in-depth reviews that question "is
the code doing the right thing?", which would take a real reading of
and a comparison between the code before and after the patches, with
some understanding of how things have been working, how things were
envisioned to evolve, and how the patch author would want to change
the course of the evolution of the code in the longer term.  

Christian, you've been touching the code in this area the longuest.
Can we have some of your time reviewing these?

Thanks.

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Junio C Hamano]

Linus Arver <linusa@google.com> writes:

> Josh Steadmon <steadmon@google.com> writes:
>
>> On 2024.01.31 01:22, Linus Arver via GitGitGadget wrote:
>>> This unification will allow us to delete the format_trailer_info() and
>>> print_tok_val() functions in the next patch. They are not deleted here
>>> in order to keep the diff small.
>>
>> Needs to be removed after squashing v2 patch 4 :)
>
> Oops. Will update in next reroll, thanks.

FWIW, by the way, having them in the same patch made it a lot easier
to compare what the original did (with these removed functions) and
what the updated code would do.  When a change is supposed to be a
clean-up of an existing code without changing the behaviour, it helps
to make the before and after versions visible in the patch.

Thanks.

Re: [PATCH v3 04/10] sequencer: use the trailer iterator

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> This patch allows for the removal of "trailer_info_get()" from the
> trailer.h API, which will be in the next patch.

Hmph, do you mean "shortlog" and the sequencer were the only two
external callers and with this we can make it file-scope static to
trailer.c?  Or do you mean the next step will be more than a removal
of a declaration from trailer.h plus adding "static" in front of its
definition in trailer.c, because there need other adjustments before
that happens?

> Instead of calling "trailer_info_get()", which is a low-level function
> in the trailers implementation (trailer.c), call
> trailer_iterator_advance(), which was specifically designed for public
> consumption in f0939a0eb1 (trailer: add interface for iterating over
> commit trailers, 2020-09-27).

;-).

> Also, teach the iterator about non-trailer lines, by adding a new field
> called "raw" to hold both trailer and non-trailer lines. This is
> necessary because a "trailer block" is a list of trailer lines of at
> least 25% trailers (see 146245063e (trailer: allow non-trailers in
> trailer block, 2016-10-21)), such that it may hold non-trailer lines.

That sounds like a task larger than something we would want in a
patch that focuses on another task (e.g. update sequencer not to
call trailer_info_get()) while at it.  It seems from a casual glance
that the change to shortlog.c is to accomodate this change in the
semantics of what the iterator could return?  It smells that this
patch does two more or less unrelated things at the same time?

> Signed-off-by: Linus Arver <linusa@google.com>
> ---
>  builtin/shortlog.c |  7 +++++--
>  sequencer.c        | 35 +++++++++++++++--------------------
>  trailer.c          | 17 +++++++++--------
>  trailer.h          | 13 +++++++++++++
>  4 files changed, 42 insertions(+), 30 deletions(-)
>
> diff --git a/builtin/shortlog.c b/builtin/shortlog.c
> index 1307ed2b88a..dc8fd5a5532 100644
> --- a/builtin/shortlog.c
> +++ b/builtin/shortlog.c
> @@ -172,7 +172,7 @@ static void insert_records_from_trailers(struct shortlog *log,
>  					 const char *oneline)
>  {
>  	struct trailer_iterator iter;
> -	const char *commit_buffer, *body;
> +	const char *commit_buffer, *body, *value;
>  	struct strbuf ident = STRBUF_INIT;
>  
>  	if (!log->trailers.nr)
> @@ -190,7 +190,10 @@ static void insert_records_from_trailers(struct shortlog *log,
>  
>  	trailer_iterator_init(&iter, body);
>  	while (trailer_iterator_advance(&iter)) {
> -		const char *value = iter.val.buf;
> +		if (!iter.is_trailer)
> +			continue;
> +
> +		value = iter.val.buf;
>  
>  		if (!string_list_has_string(&log->trailers, iter.key.buf))
>  			continue;
> diff --git a/sequencer.c b/sequencer.c
> index 3cc88d8a800..bc7c82c5271 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -319,37 +319,32 @@ static const char *get_todo_path(const struct replay_opts *opts)
>  static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
>  	size_t ignore_footer)
>  {
> -	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
> -	struct trailer_info info;
> -	size_t i;
> -	int found_sob = 0, found_sob_last = 0;
> -	char saved_char;
> -
> -	opts.no_divider = 1;
> +	struct trailer_iterator iter;
> +	size_t i = 0, found_sob = 0;
> +	char saved_char = sb->buf[sb->len - ignore_footer];
>  
>  	if (ignore_footer) {
> -		saved_char = sb->buf[sb->len - ignore_footer];
>  		sb->buf[sb->len - ignore_footer] = '\0';
>  	}
>  
> -	trailer_info_get(&info, sb->buf, &opts);
> +	trailer_iterator_init(&iter, sb->buf);
> +	while (trailer_iterator_advance(&iter)) {
> +		i++;
> +		if (sob &&
> +		    iter.is_trailer &&
> +		    !strncmp(iter.raw, sob->buf, sob->len)) {
> +			found_sob = i;
> +		}
> +	}
> +	trailer_iterator_release(&iter);
>  
>  	if (ignore_footer)
>  		sb->buf[sb->len - ignore_footer] = saved_char;
>  
> -	if (info.trailer_block_start == info.trailer_block_end)
> +	if (!i)
>  		return 0;
>  
> -	for (i = 0; i < info.trailer_nr; i++)
> -		if (sob && !strncmp(info.trailers[i], sob->buf, sob->len)) {
> -			found_sob = 1;
> -			if (i == info.trailer_nr - 1)
> -				found_sob_last = 1;
> -		}
> -
> -	trailer_info_release(&info);
> -
> -	if (found_sob_last)
> +	if (found_sob == i)
>  		return 3;
>  	if (found_sob)
>  		return 2;
> diff --git a/trailer.c b/trailer.c
> index 71ea2bb67f8..5bcc9b0006c 100644
> --- a/trailer.c
> +++ b/trailer.c
> @@ -1158,17 +1158,18 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
>  
>  int trailer_iterator_advance(struct trailer_iterator *iter)
>  {
> -	while (iter->internal.cur < iter->internal.info.trailer_nr) {
> -		char *trailer = iter->internal.info.trailers[iter->internal.cur++];
> -		int separator_pos = find_separator(trailer, separators);
> -
> -		if (separator_pos < 1)
> -			continue; /* not a real trailer */
> -
> +	char *line;
> +	int separator_pos;
> +	if (iter->internal.cur < iter->internal.info.trailer_nr) {
> +		line = iter->internal.info.trailers[iter->internal.cur++];
> +		separator_pos = find_separator(line, separators);
> +		iter->is_trailer = (separator_pos > 0);
> +
> +		iter->raw = line;
>  		strbuf_reset(&iter->key);
>  		strbuf_reset(&iter->val);
>  		parse_trailer(&iter->key, &iter->val, NULL,
> -			      trailer, separator_pos);
> +			      line, separator_pos);
>  		unfold_value(&iter->val);
>  		return 1;
>  	}
> diff --git a/trailer.h b/trailer.h
> index 244f29fc91f..a7599067acc 100644
> --- a/trailer.h
> +++ b/trailer.h
> @@ -127,6 +127,19 @@ struct trailer_iterator {
>  	struct strbuf key;
>  	struct strbuf val;
>  
> +	/*
> +	 * Raw line (e.g., "foo: bar baz") before being parsed as a trailer
> +	 * key/val pair as part of a trailer block. A trailer block can be
> +	 * either 100% trailer lines, or mixed in with non-trailer lines (in
> +	 * which case at least 25% must be trailer lines).
> +	 */
> +	const char *raw;
> +
> +	/*
> +	 * 1 if the raw line was parsed as a trailer line (key/val pair).
> +	 */
> +	int is_trailer;
> +
>  	/* private */
>  	struct {
>  		struct trailer_info info;

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> Linus Arver <linusa@google.com> writes:
>
>> Josh Steadmon <steadmon@google.com> writes:
>>
>>> On 2024.01.31 01:22, Linus Arver via GitGitGadget wrote:
>>>> This unification will allow us to delete the format_trailer_info() and
>>>> print_tok_val() functions in the next patch. They are not deleted here
>>>> in order to keep the diff small.
>>>
>>> Needs to be removed after squashing v2 patch 4 :)
>>
>> Oops. Will update in next reroll, thanks.
>
> FWIW, by the way, having them in the same patch made it a lot easier
> to compare what the original did (with these removed functions) and
> what the updated code would do.  When a change is supposed to be a
> clean-up of an existing code without changing the behaviour, it helps
> to make the before and after versions visible in the patch.
>
> Thanks.

Ack, will try to keep that in mind.

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> Linus Arver <linusa@google.com> writes:
>
>> I could just grow this series with another ~22 patches to include those
>> additional refactors, but I am hesitant about doing so, simply due to
>> the sheer number of them.
>
> I actually do not mind at all if you started with a preliminary
> clean-up series, and stopped the first batch somewhere in the middle
> of the 20+ patches before even reaching any of these 10 patches we
> see here, if that gives us a readable set of bite-sized changes that
> prepare a solid foundation to rebuild things on top.  I am having a
> feeling that not even a single person has reviewed them on list even
> though we are already at the third iteration, which is quite
> frustrating (and I would imagine that it would be frustrating for
> you, too), and I suspect that the step like [v3 03/10] that makes
> too large a change with too little explanation (and perhaps a bit of
> "trust me, this does not change the behaviour") is one contributing
> factor why people are afraid of touching it.

I am planning to spend today trying to break up this patch into smaller
preparatory chunks that still end up at the same state as this patch.

Will post another update on how this goes by EOD. Thanks.

Re: [PATCH v3 05/10] trailer: make trailer_info struct private

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> Make this struct private by putting its definition inside trailer.c.
> This has two benefits:
>
>   (1) it makes the surface area of the public facing
>       interface (trailer.h) smaller, and
>
>   (2) external API users are unable to peer inside this struct (because
>       it is only ever exposed as an opaque pointer).

At the cost of an extra pointer dereference every time the member of
the struct is accessed, plus the cost of allocation/deallocation.

Which may not be a huge deal, but I wonder if the approach to name
the member of the outer struct with "private" that seems to be used
in other parts of the code (e.g. the .private_size member in the
hashmap structure, the .refs_private member in the repository
structure) or even a documented convention (e.g. raw_object_store),
might be more appropriate here.  If Coccinelle works well (which we
may be having some trouble with --- cf. <xmqq1q9ybsnl.fsf@gitster.g>),
we should be able to catch external accesses without having to hide
the implementation details via an extra pointer dereference.

> @@ -176,11 +176,12 @@ static void interpret_trailers(const struct process_trailer_options *opts,
>  	strbuf_release(&trailer_block);
>  
>  	free_trailers(&head);
> -	trailer_info_release(&info);
>  
>  	/* Print the lines after the trailers as is */
>  	if (!opts->only_trailers)
> -		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
> +		fwrite(sb.buf + trailer_block_end(info), 1, sb.len - trailer_block_end(info), outfile);
> +
> +	trailer_info_release(info);

Interesting.  Is this an indenendent bugfix even if we decided not
to take this patch?  No, I have not fully decided to be negative on
the move this entire patch makes (even though I am leaning towards
saying so).  Just hypothetically, even if we wanted to keep "info"
here as a structure and not a pointer to an opaque structure,
doesn't this hunk fix a real bug?

Well, technically, not quite, because the members referenced in that
if (.only_trailers) block are still live in the info struct.  But it
still smells wrong to access info.* after calling _release() on it,
and this fix should come before "info" is turned from an instance to
a pointer, I would say.

> diff --git a/trailer.h b/trailer.h
> index a7599067acc..e19ddf84e64 100644
> --- a/trailer.h
> +++ b/trailer.h
> @@ -4,6 +4,8 @@
>  #include "list.h"
>  #include "strbuf.h"
>  
> +struct trailer_info;
> +
>  enum trailer_where {
>  	WHERE_DEFAULT,
>  	WHERE_END,
> @@ -29,27 +31,6 @@ int trailer_set_where(enum trailer_where *item, const char *value);
>  int trailer_set_if_exists(enum trailer_if_exists *item, const char *value);
>  int trailer_set_if_missing(enum trailer_if_missing *item, const char *value);
>  
> +size_t trailer_block_start(struct trailer_info *info);
> +size_t trailer_block_end(struct trailer_info *info);
> +int blank_line_before_trailer_block(struct trailer_info *info);

And we need new accessors, which is a good change regardless of the
answer to the "do we really want an extra pointer dereference?
Shouldn't the existing 'private' and 'internal' label we see below
sufficient?" question.

> @@ -142,7 +123,7 @@ struct trailer_iterator {
>  
>  	/* private */
>  	struct {
> -		struct trailer_info info;
> +		struct trailer_info *info;
>  		size_t cur;
>  	} internal;
>  };

Re: [PATCH v3 06/10] trailer: spread usage of "trailer_block" language

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> Deprecate the "trailer_info" struct name and replace it with
> "trailer_block". The main reason is to help readability, because
> "trailer_info" on the surface sounds like it's about a single trailer
> when in reality it is a collection of contiguous lines, at least 25% of
> which are trailers.

Yup, "info" is usually a fairly meaningless word.  At least "block"
may imply it is a collection of trailers.

The naming would not matter as much to the API users, if the thing
is now opaque, though.

Re: [PATCH v3 07/10] trailer: prepare to move parse_trailers_from_command_line_args() to builtin

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> Rename add_arg_item() to trailer_add_arg_item() because it will have to
> be exposed as an API function in the next patch. Rename
> new_trailers_clear() to free_new_trailers() because it will be promoted
> into an API function; the API already has free_trailers(), so using the
> "free_*" naming style will keep it consistent. Also rename "conf_info"
> to "trailer_conf" for readability, dropping the low-value "_info" suffix
> as we did earlier in this series for "trailer_info" to "trailer_block".

Makes sense.

Thanks.

Re: [PATCH v3 04/10] sequencer: use the trailer iterator

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> From: Linus Arver <linusa@google.com>
>>
>> This patch allows for the removal of "trailer_info_get()" from the
>> trailer.h API, which will be in the next patch.
>
> Hmph, do you mean "shortlog" and the sequencer were the only two
> external callers and with this we can make it file-scope static to
> trailer.c?

This was what I meant (originally ...

> Or do you mean the next step will be more than a removal
> of a declaration from trailer.h plus adding "static" in front of its
> definition in trailer.c, because there need other adjustments before
> that happens?

... but now I realize that the operation adds a few small tweaks, such
as tweaking the parameters it expects and also what it returns). In the
spirit of breaking up patch 3, I will also break this up into
preparatory patches as well.

>> Also, teach the iterator about non-trailer lines, by adding a new field
>> called "raw" to hold both trailer and non-trailer lines. This is
>> necessary because a "trailer block" is a list of trailer lines of at
>> least 25% trailers (see 146245063e (trailer: allow non-trailers in
>> trailer block, 2016-10-21)), such that it may hold non-trailer lines.
>
> That sounds like a task larger than something we would want in a
> patch that focuses on another task (e.g. update sequencer not to
> call trailer_info_get()) while at it.  It seems from a casual glance
> that the change to shortlog.c is to accomodate this change in the
> semantics of what the iterator could return?  It smells that this
> patch does two more or less unrelated things at the same time?

I think you're correct. Hopefully breaking this up will make things
easier to review.

I am learning very quickly from your review comments in patch 3 and in
here that, in the absence of area experts, the existing tests/CI tests
cannot be trusted alone (after all some tests could be missing), which
makes it more important to do so-called "micro-commits".

But overall, breaking things up is a good thing anyway as a general
practice, so, I think this is a good lesson. TBH I have a (bad) habit of
saying "is the diff ~100 lines?" and if so I don't spend any time
thinking of breaking these up. X-<

Thanks.

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Junio C Hamano]

Linus Arver <linusa@google.com> writes:

> I am planning to spend today trying to break up this patch into smaller
> preparatory chunks that still end up at the same state as this patch.
>
> Will post another update on how this goes by EOD. Thanks.

I stopped reading the function after noticing the double unfolding,
so there may be similar "why do we do this unexplained new thing in
the function that the original didn't?" issues in the "same state",

In any case, if I understood your plan I heard from you in the
discussion yesterday correctly, the unfolding should not be added to
format (to make it double), but would be moved from parse to format
in a single step.  It would avoid making it double, and would make
the parse step about purely parsing without modification, which is a
very worthwhile thing to do.  So I am not sure if we want to end up
with the same state in the first place, though.

THanks.

Re: [PATCH v3 08/10] trailer: move arg handling to interpret-trailers.c

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> We don't move the "arg_item" struct to interpret-trailers.c, because it
> is now a struct that contains information about trailers that could be
> added into the input text's own trailers. This is a generic concept that
> extends beyond trailers defined as CLI arguments (it applies to trailers
> defined in configuration as well). We will rename "arg_item" to
> "trailer_template" in a follow-up patch to keep the diff here small.
>
> Signed-off-by: Linus Arver <linusa@google.com>
> ---
>  builtin/interpret-trailers.c | 88 ++++++++++++++++++++++--------------
>  trailer.c                    | 62 ++++++++++++++++++-------
>  trailer.h                    | 12 +++++
>  3 files changed, 112 insertions(+), 50 deletions(-)
>
> diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
> index 9f0ba39b317..9a902012912 100644
> --- a/builtin/interpret-trailers.c
> +++ b/builtin/interpret-trailers.c
> @@ -45,23 +45,17 @@ static int option_parse_if_missing(const struct option *opt,
>  	return trailer_set_if_missing(opt->value, arg);
>  }
>  
> -static void free_new_trailers(struct list_head *trailers)
> -{
> -	struct list_head *pos, *tmp;
> -	struct new_trailer_item *item;
> -
> -	list_for_each_safe(pos, tmp, trailers) {
> -		item = list_entry(pos, struct new_trailer_item, list);
> -		list_del(pos);
> -		free(item);
> -	}
> -}
> +static char *cl_separators;

It somehow feels ugly and goes backward to depend on a new global,
especially when you are aiming to libify more things.  If we are
doing something like ...

>  static int option_parse_trailer(const struct option *opt,
>  				   const char *arg, int unset)
>  {
>  	struct list_head *trailers = opt->value;
> -	struct new_trailer_item *item;
> +	struct strbuf tok = STRBUF_INIT;
> +	struct strbuf val = STRBUF_INIT;
> +	const struct trailer_conf *conf;
> +	struct trailer_conf *conf_current = new_trailer_conf();
> +	ssize_t separator_pos;
>  
>  	if (unset) {
>  		free_new_trailers(trailers);
> @@ -71,12 +65,31 @@ static int option_parse_trailer(const struct option *opt,
>  	if (!arg)
>  		return -1;
>  
> -	item = xmalloc(sizeof(*item));
> -	item->text = arg;
> -	item->where = where;
> -	item->if_exists = if_exists;
> -	item->if_missing = if_missing;
> -	list_add_tail(&item->list, trailers);
> +	separator_pos = find_separator(arg, cl_separators);
> +	if (separator_pos) {
> +		parse_trailer(arg, separator_pos, &tok, &val, &conf);
> +		duplicate_trailer_conf(conf_current, conf);
> +
> +		/*
> +		 * Override conf_current with settings specified via CLI flags.
> +		 */

... this, which apparently needs to know that we are dealing with
input from CLI, shouldn't we be able to do the "CLI allows '=' as
well as whatever is the default" here, instead of mucking with the
global variable in the caller?

Even if you plan to later make this function callable from outside
the context of parse_options() callback, and if you do not want "CLI
allows '=' as well" for such new callers, we should be able to have
a shared helper function that is the bulk of this function but takes
additional parameter to tweak its behaviour slightly depending on
the needs of the callers?

> +		trailer_conf_set(where, if_exists, if_missing, conf_current);

I am getting an impression that the use of and the introduction of
the new helper, mixed with movement of the responsibility, makes
reviewing the change unnecessarily more cumbersome.  An equivalent
series with a few more steps, each of them focusing on a small
change (like, "updating the conf_current members is done with
assignments that appear as a pattern very often---introduce a helper
to reduce boilerplate") would have been more enticing to reviewers.

> +		trailer_add_arg_item(strbuf_detach(&tok, NULL),
> +				     strbuf_detach(&val, NULL),
> +				     conf_current,
> +				     trailers);
> +	} else {
> +		struct strbuf sb = STRBUF_INIT;
> +		strbuf_addstr(&sb, arg);
> +		strbuf_trim(&sb);
> +		error(_("empty trailer token in trailer '%.*s'"),
> +			(int) sb.len, sb.buf);
> +		strbuf_release(&sb);
> +	}
> +
> +	free(conf_current);
> +
>  	return 0;
>  }
>  
> @@ -135,7 +148,7 @@ static void read_input_file(struct strbuf *sb, const char *file)
>  }
>  
>  static void interpret_trailers(const struct process_trailer_options *opts,
> -			       struct list_head *new_trailer_head,
> +			       struct list_head *arg_trailers,
>  			       const char *file)
>  {
>  	LIST_HEAD(head);
> @@ -144,8 +157,6 @@ static void interpret_trailers(const struct process_trailer_options *opts,
>  	struct trailer_block *trailer_block;
>  	FILE *outfile = stdout;
>  
> -	trailer_config_init();
> -
>  	read_input_file(&sb, file);
>  
>  	if (opts->in_place)
> @@ -162,12 +173,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
>  
>  
>  	if (!opts->only_input) {
> -		LIST_HEAD(config_head);
> -		LIST_HEAD(arg_head);
> -		parse_trailers_from_config(&config_head);
> -		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
> -		list_splice(&config_head, &arg_head);
> -		process_trailers_lists(&head, &arg_head);
> +		process_trailers_lists(&head, arg_trailers);
>  	}

So, the bulk of the parsing is no longer responsibility of this
function.  Instead, the caller (e.g., cmd_interpret_trailers()) is
expected to do that before they call us.

> ...
>  	git_config(git_default_config, NULL);
> +	trailer_config_init();
> +
> +	if (!opts.only_input) {
> +		parse_trailers_from_config(&configured_trailers);
> +	}

By the way, until we add more statements in this block, lose the
unnecessary {} around it.

> +	/*
> +	* In command-line arguments, '=' is accepted (in addition to the
> +	* separators that are defined).
> +	*/
> +	cl_separators = xstrfmt("=%s", trailer_default_separators());
>  
>  	argc = parse_options(argc, argv, prefix, options,
>  			     git_interpret_trailers_usage, 0);
>  
> -	if (opts.only_input && !list_empty(&trailers))
> +	free(cl_separators);
> +
> +	if (opts.only_input && !list_empty(&arg_trailers))
>  		usage_msg_opt(
>  			_("--trailer with --only-input does not make sense"),
>  			git_interpret_trailers_usage,
>  			options);
>  
> +	list_splice(&configured_trailers, &arg_trailers);

This corresponds to what the old code did in interpret_trailers(),
OK.

The move of the responsibility may make sense.

>  	if (argc) {
>  		int i;
>  		for (i = 0; i < argc; i++)
> -			interpret_trailers(&opts, &trailers, argv[i]);
> +			interpret_trailers(&opts, &arg_trailers, argv[i]);
>  	} else {
>  		if (opts.in_place)
>  			die(_("no input file given for in-place editing"));
> -		interpret_trailers(&opts, &trailers, NULL);
> +		interpret_trailers(&opts, &arg_trailers, NULL);
>  	}
>  
> -	free_new_trailers(&trailers);
> +	free_new_trailers(&arg_trailers);
>  
>  	return 0;
>  }
> diff --git a/trailer.c b/trailer.c
> index c16f552b463..19637ff295d 100644
> --- a/trailer.c
> +++ b/trailer.c
> @@ -66,6 +66,11 @@ static LIST_HEAD(conf_head);
>  
>  static char *separators = ":";
>  
> +const char *trailer_default_separators(void)
> +{
> +	return separators;
> +}
> +
>  static int configured;
>  
>  #define TRAILER_ARG_STRING "$ARG"
> @@ -424,6 +429,25 @@ int trailer_set_if_missing(enum trailer_if_missing *item, const char *value)
>  	return 0;
>  }
>  
> +void trailer_conf_set(enum trailer_where where,
> +		      enum trailer_if_exists if_exists,
> +		      enum trailer_if_missing if_missing,
> +		      struct trailer_conf *conf)
> +{
> +	if (where != WHERE_DEFAULT)
> +		conf->where = where;
> +	if (if_exists != EXISTS_DEFAULT)
> +		conf->if_exists = if_exists;
> +	if (if_missing != MISSING_DEFAULT)
> +		conf->if_missing = if_missing;
> +}

So, this is such a helper function. It is curious that it
deliberately lacks the ability to reset what has already been
configured back to the default.

For example, we earlier saw this original code ...

> -	item = xmalloc(sizeof(*item));
> -	item->text = arg;
> -	item->where = where;
> -	item->if_exists = if_exists;
> -	item->if_missing = if_missing;

... gets replaced with this call.

> +	struct trailer_conf *conf_current = new_trailer_conf();
> ...
> +		trailer_conf_set(where, if_exists, if_missing, conf_current);

For this conversion to be correct, we require that the value of the
*_DEFAULT macro to be 0 and/or these three values getting assigned
are not *_DEFAULT values.  Maybe that may happen to be the case in
today's code, but such an unwritten assumption makes me feel nervous.

I am not sure if it is worth the confusion factor to make a function
whose name is $anything_set() to be making a conditional assignment.
If such a conditional assignment *also* happens often and deserves
to have its own helper, it is fine to have such a helper for
conditional assignment, but calling it $anything_set() is probably a
mistake.

Other than that, the main thrust of this step seems sensible.

Thanks.

Re: [PATCH v3 09/10] trailer: delete obsolete argument handling code from API

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> This commit was not squashed with its parent in order to keep the diff
> separate (to make the additive changes in the parent easier to read).

Hopefully we won't see such an artificial separation in the new
iteration.  As we saw, being able to compare before and after images
in a single patch is useful while reviewing a change that is supposed
to be making things cleaner without altering what they do.

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> Linus Arver <linusa@google.com> writes:
>
>> I am planning to spend today trying to break up this patch into smaller
>> preparatory chunks that still end up at the same state as this patch.
>>
>> Will post another update on how this goes by EOD. Thanks.

I'm happy to report that this breaking-up of things is going a lot
smoother than expected. The hard part is not so much the rearranging of
code but rather thinking up of good commit messages to explain the
intent of each smaller step. I have most of this patch broken up into
much smaller steps with all tests passing.

> I stopped reading the function after noticing the double unfolding,
> so there may be similar "why do we do this unexplained new thing in
> the function that the original didn't?" issues in the "same state",

Yep, and these unexplained new things are the things I am recalling
(from a few weeks back when I first started this series) and am now
trying to find the right commit messages for.

> In any case, if I understood your plan I heard from you in the
> discussion yesterday correctly, the unfolding should not be added to
> format (to make it double),

Yes, it will be done during parsing for now. This is because of
additional refactors that are coming in the later series, which ...

> but would be moved from parse to format
> in a single step.  It would avoid making it double, and would make
> the parse step about purely parsing without modification, which is a
> very worthwhile thing to do.

... achieve this exact thing (unfolding is done "lazily" during
formatting only, not "eagerly" as we do now during parsing).

> So I am not sure if we want to end up
> with the same state in the first place, though.
>
> THanks.

So far the breaking-up of this patch has not revealed a need to change
the end result significantly from what I have here. But I admit I am not
sure this will be the case for the other patches that need similar
treatment in this series.

Considering how it took me most of the day to break up this single
patch, I am not sure I can get a v4 ready by tomorrow. Might need a few
days for that.

Thank you for your very helpful review comments for this series so far.
They have been quite illuminating. :)

Re: [PATCH v3 03/10] trailer: unify trailer formatting machinery

[Junio C Hamano]

Linus Arver <linusa@google.com> writes:

> The hard part is not so much the rearranging of
> code but rather thinking up of good commit messages to explain the
> intent of each smaller step.

Good.

Coming up with a good explanation of what you did forces you to
think it thorough again.  People often discover bugs in the code
they just wrote, or different and better ways to solve the problem,
while explaining the change to their colleagues, and I found it has
very similar effect to try to write a good explanation in the commit
log.  Time to write such a good explanation is time worth investing
in.

Incidentally, this is why we value clearly written proposed log
message while reviewing.  A commit that is a product of "I tried
this, which did not work, and then tried something else, which did
not work either, and after some more random tries, this seems to
work" would not have a clear thinking behind it, but if such
iterations of random tries happened to hit the best solution,
thinking backwards from the result to the original problem and
explaining how the solution works would result in a good
description.

Thanks.

Re: [PATCH v3 04/10] sequencer: use the trailer iterator

[Linus Arver]

Linus Arver <linusa@google.com> writes:

> Junio C Hamano <gitster@pobox.com> writes:
>
>> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>>
>>> From: Linus Arver <linusa@google.com>
>>>
>>> Also, teach the iterator about non-trailer lines, by adding a new field
>>> called "raw" to hold both trailer and non-trailer lines. This is
>>> necessary because a "trailer block" is a list of trailer lines of at
>>> least 25% trailers (see 146245063e (trailer: allow non-trailers in
>>> trailer block, 2016-10-21)), such that it may hold non-trailer lines.
>>
>> That sounds like a task larger than something we would want in a
>> patch that focuses on another task (e.g. update sequencer not to
>> call trailer_info_get()) while at it.  It seems from a casual glance
>> that the change to shortlog.c is to accomodate this change in the
>> semantics of what the iterator could return?  It smells that this
>> patch does two more or less unrelated things at the same time?
>
> I think you're correct. Hopefully breaking this up will make things
> easier to review.

And now that I've broken it up locally, I can say that the change I made
to shortlog was unnecessary (shortlog already has a check to see if the
trailer key is empty) which makes the "is_trailer" check I added to it
here redundant (because non-trailer lines, which the new iterator can
iterate over, have empty keys).

Will remove the shortlog change in v4. Thanks.

Re: [PATCH v3 05/10] trailer: make trailer_info struct private

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> Make this struct private by putting its definition inside trailer.c.
>> This has two benefits:
>>
>>   (1) it makes the surface area of the public facing
>>       interface (trailer.h) smaller, and
>>
>>   (2) external API users are unable to peer inside this struct (because
>>       it is only ever exposed as an opaque pointer).
>
> At the cost of an extra pointer dereference every time the member of
> the struct is accessed, plus the cost of allocation/deallocation.

Added to commit message for v4, thanks.

> Which may not be a huge deal, but I wonder if the approach to name
> the member of the outer struct with "private" that seems to be used
> in other parts of the code (e.g. the .private_size member in the
> hashmap structure, the .refs_private member in the repository
> structure) or even a documented convention (e.g. raw_object_store),
> might be more appropriate here.

Having gotten my hands dirty with the pimpl idiom, I think possibly the
best thing about it is that the compiler can tell me exactly where
every Git subsystem outside of trailer.c is accessing members inside the
(newly private) struct. So it's great for checking existing usage of how
things are used already, day-to-day. Not being well-versed in shortlog
or sequencer internals, these warnings from the compiler for these
functions (trying to peer through the opaque pointer) have been very
informative.

I think it would be reasonable to drop the idiom if the additional
performance costs (pointer dereference, allocation/deallocation) are too
much to bear. For the trailer subsystem I don't see an immediate need to
focus on performance though, so I think it's fine for now.

> If Coccinelle works well (which we
> may be having some trouble with --- cf. <xmqq1q9ybsnl.fsf@gitster.g>),
> we should be able to catch external accesses without having to hide
> the implementation details via an extra pointer dereference.

That's true. But that would only work for trailer API users inside the
Git codebase. Eventually I envision projects external to Git using
<trailer.h>, and in that scenario we would _really_ not want to let
these external users peek inside structs with members named "private" or
"internal" in name only. Murphy's law suggests those external users will
(naughtily) depend on the internals, and that would place an undue
burden on us if we ever wanted to make large changes to these (public)
structs because we might break them.

Another thing to consider is that if our API users ever want to get more
flexibility out of it, they would immediately come to us if we used the
pimpl idiom (our users are bound by exactly how we define the API). If
the structs were public, then they would be free to implement new
functionality to extend the API on their own, without ever letting us
know. If there are folks making small (unofficial) extensions to the
API, I want that to happen _in_ this project, not outside.

In summary what I'm saying is that this idiom gives very strong guidance
for desining an API for both Git-internal and external-to-Git usage. I
think for at least the trailer subsystem it's worth trying out. I've
pointed out some of the same points in the cover letter also.

>> @@ -176,11 +176,12 @@ static void interpret_trailers(const struct process_trailer_options *opts,
>>  	strbuf_release(&trailer_block);
>>  
>>  	free_trailers(&head);
>> -	trailer_info_release(&info);
>>  
>>  	/* Print the lines after the trailers as is */
>>  	if (!opts->only_trailers)
>> -		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
>> +		fwrite(sb.buf + trailer_block_end(info), 1, sb.len - trailer_block_end(info), outfile);
>> +
>> +	trailer_info_release(info);
>
> Interesting.  Is this an indenendent bugfix even if we decided not
> to take this patch?

Nice catch!

> No, I have not fully decided to be negative on
> the move this entire patch makes (even though I am leaning towards
> saying so).  Just hypothetically, even if we wanted to keep "info"
> here as a structure and not a pointer to an opaque structure,
> doesn't this hunk fix a real bug?
>
> Well, technically, not quite, because the members referenced in that
> if (.only_trailers) block are still live in the info struct.  But it
> still smells wrong to access info.* after calling _release() on it,
> and this fix should come before "info" is turned from an instance to
> a pointer, I would say.

I've promoted this change into a bugfix patch as the very first patch
for the next reroll. Thanks.

>> diff --git a/trailer.h b/trailer.h
>> index a7599067acc..e19ddf84e64 100644
>> --- a/trailer.h
>> +++ b/trailer.h
>> @@ -4,6 +4,8 @@
>>  #include "list.h"
>>  #include "strbuf.h"
>>  
>> +struct trailer_info;
>> +
>>  enum trailer_where {
>>  	WHERE_DEFAULT,
>>  	WHERE_END,
>> @@ -29,27 +31,6 @@ int trailer_set_where(enum trailer_where *item, const char *value);
>>  int trailer_set_if_exists(enum trailer_if_exists *item, const char *value);
>>  int trailer_set_if_missing(enum trailer_if_missing *item, const char *value);
>>  
>> +size_t trailer_block_start(struct trailer_info *info);
>> +size_t trailer_block_end(struct trailer_info *info);
>> +int blank_line_before_trailer_block(struct trailer_info *info);
>
> And we need new accessors, which is a good change regardless of the
> answer to the "do we really want an extra pointer dereference?
> Shouldn't the existing 'private' and 'internal' label we see below
> sufficient?" question.

I am very surprised with your response here, because I was expecting the
complete opposite reaction from reviewers --- something like

    Hmph, we have to write accessor functions now when before we could
    just reach inside the struct directly? Isn't this just adding
    needless verbosity?

(perhaps with more dissatisfaction). Something tells me you meant "bad
change" but accidentally wrote "good change". Am I correct?

Re: [PATCH v3 06/10] trailer: spread usage of "trailer_block" language

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> From: Linus Arver <linusa@google.com>
>>
>> Deprecate the "trailer_info" struct name and replace it with
>> "trailer_block". The main reason is to help readability, because
>> "trailer_info" on the surface sounds like it's about a single trailer
>> when in reality it is a collection of contiguous lines, at least 25% of
>> which are trailers.
>
> Yup, "info" is usually a fairly meaningless word.  At least "block"
> may imply it is a collection of trailers.
>
> The naming would not matter as much to the API users, if the thing
> is now opaque, though.

You make an interesting point (and I agree).

If we were to provide additional private structs named "trailer_<foo>"
at that point we should think about how these <foo> parts "interact"
with each other (if you will) in terms of names. For example we probably
would never want to name a struct "trailer_block_parser" if it has no
relationship to the existing "trailer_block" struct. More elaborate APIs
might design a particular naming scheme (with some rules about certain
suffixes, perhaps, for opaque pointers that all behave a certain way).

But such naming considerations would naturally come into view
independent of this public vs public struct discussion, so I think the
above paragraph is moot.

Re: [PATCH v3 09/10] trailer: delete obsolete argument handling code from API

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> From: Linus Arver <linusa@google.com>
>>
>> This commit was not squashed with its parent in order to keep the diff
>> separate (to make the additive changes in the parent easier to read).
>
> Hopefully we won't see such an artificial separation in the new
> iteration.  As we saw, being able to compare before and after images
> in a single patch is useful while reviewing a change that is supposed
> to be making things cleaner without altering what they do.

Totally agree (and I'm reaping the benefits of this as well in the
smaller preparatory refactoring changes I've made for the next reroll).

Re: [PATCH v3 08/10] trailer: move arg handling to interpret-trailers.c

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> [...]
>
> Other than that, the main thrust of this step seems sensible.
>
> Thanks.

Quick update: I've broken down patches 3 and 4 into smaller pieces, and
am now ready to break this one down, too.

Hopefully sometime over the weekend I'll have some answers ready for the
many good questions you've raised. Thanks.

Re: [PATCH v3 05/10] trailer: make trailer_info struct private

[Junio C Hamano]

Linus Arver <linusa@google.com> writes:

>>> +size_t trailer_block_start(struct trailer_info *info);
>>> +size_t trailer_block_end(struct trailer_info *info);
>>> +int blank_line_before_trailer_block(struct trailer_info *info);
>>
>> And we need new accessors, which is a good change regardless of the
>> answer to the "do we really want an extra pointer dereference?
>> Shouldn't the existing 'private' and 'internal' label we see below
>> sufficient?" question.
>
> I am very surprised with your response here, because I was expecting the
> complete opposite reaction from reviewers --- something like
>
>     Hmph, we have to write accessor functions now when before we could
>     just reach inside the struct directly? Isn't this just adding
>     needless verbosity?
>
> (perhaps with more dissatisfaction). Something tells me you meant "bad
> change" but accidentally wrote "good change". Am I correct?

I often make typoes and screw up my double negations, but not this
time.  While I still suspect that the extra secrecy afforded by
using a pointer to an opaque structure is something unnecessary
between friends, as coding convention, it would be a good change to
introduce accessors and have them used by the API users, i.e. code
outside the API implementation.

You can still "git grep" for references to the ".trailer_private"
(or whatever the "private" members are named by convention) member
outside the trailer "client code" to see if there are people who are
too intimate than they should be that way, as they should all be
using the published accessors.

Re: [PATCH v3 08/10] trailer: move arg handling to interpret-trailers.c

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
>> index 9f0ba39b317..9a902012912 100644
>> --- a/builtin/interpret-trailers.c
>> +++ b/builtin/interpret-trailers.c
>> @@ -45,23 +45,17 @@ static int option_parse_if_missing(const struct option *opt,
>>  	return trailer_set_if_missing(opt->value, arg);
>>  }
>>  
>> -static void free_new_trailers(struct list_head *trailers)
>> -{
>> -	struct list_head *pos, *tmp;
>> -	struct new_trailer_item *item;
>> -
>> -	list_for_each_safe(pos, tmp, trailers) {
>> -		item = list_entry(pos, struct new_trailer_item, list);
>> -		list_del(pos);
>> -		free(item);
>> -	}
>> -}
>> +static char *cl_separators;
>
> It somehow feels ugly and goes backward to depend on a new global,
> especially when you are aiming to libify more things.

Removed in the next reroll (we can just use a regular stack variable,
not a global).

> Even if you plan to later make this function callable from outside
> the context of parse_options() callback, and if you do not want "CLI
> allows '=' as well" for such new callers, we should be able to have
> a shared helper function that is the bulk of this function but takes
> additional parameter to tweak its behaviour slightly depending on
> the needs of the callers?

I'll refrain from exploring this possibility in this series (and just
make do with the non-global variable as described above) because of the
additional pending changes around trailer configuration handling in the
larger series. FWIW even in the larger series I don't think there's a
need to have such a helper function.

>> +		trailer_conf_set(where, if_exists, if_missing, conf_current);
>
> I am getting an impression that the use of and the introduction of
> the new helper, mixed with movement of the responsibility, makes
> reviewing the change unnecessarily more cumbersome.  An equivalent
> series with a few more steps, each of them focusing on a small
> change (like, "updating the conf_current members is done with
> assignments that appear as a pattern very often---introduce a helper
> to reduce boilerplate") would have been more enticing to reviewers.

I've broken this patch down into smaller steps (along with the other
notable patches in this series).

>> +		trailer_add_arg_item(strbuf_detach(&tok, NULL),
>> +				     strbuf_detach(&val, NULL),
>> +				     conf_current,
>> +				     trailers);
>> +	} else {
>> +		struct strbuf sb = STRBUF_INIT;
>> +		strbuf_addstr(&sb, arg);
>> +		strbuf_trim(&sb);
>> +		error(_("empty trailer token in trailer '%.*s'"),
>> +			(int) sb.len, sb.buf);
>> +		strbuf_release(&sb);
>> +	}
>> +
>> +	free(conf_current);
>> +
>>  	return 0;
>>  }
>>  
>> @@ -135,7 +148,7 @@ static void read_input_file(struct strbuf *sb, const char *file)
>>  }
>>  
>>  static void interpret_trailers(const struct process_trailer_options *opts,
>> -			       struct list_head *new_trailer_head,
>> +			       struct list_head *arg_trailers,
>>  			       const char *file)
>>  {
>>  	LIST_HEAD(head);
>> @@ -144,8 +157,6 @@ static void interpret_trailers(const struct process_trailer_options *opts,
>>  	struct trailer_block *trailer_block;
>>  	FILE *outfile = stdout;
>>  
>> -	trailer_config_init();
>> -
>>  	read_input_file(&sb, file);
>>  
>>  	if (opts->in_place)
>> @@ -162,12 +173,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
>>  
>>  
>>  	if (!opts->only_input) {
>> -		LIST_HEAD(config_head);
>> -		LIST_HEAD(arg_head);
>> -		parse_trailers_from_config(&config_head);
>> -		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
>> -		list_splice(&config_head, &arg_head);
>> -		process_trailers_lists(&head, &arg_head);
>> +		process_trailers_lists(&head, arg_trailers);
>>  	}
>
> So, the bulk of the parsing is no longer responsibility of this
> function.  Instead, the caller (e.g., cmd_interpret_trailers()) is
> expected to do that before they call us.

Yup. This movement of "parsing responsibility" logic should be easy to
see in the next reroll (as it will be broken out separately).

>> ...
>>  	git_config(git_default_config, NULL);
>> +	trailer_config_init();
>> +
>> +	if (!opts.only_input) {
>> +		parse_trailers_from_config(&configured_trailers);
>> +	}
>
> By the way, until we add more statements in this block, lose the
> unnecessary {} around it.

Oops. Done.

>> @@ -424,6 +429,25 @@ int trailer_set_if_missing(enum trailer_if_missing *item, const char *value)
>>  	return 0;
>>  }
>>  
>> +void trailer_conf_set(enum trailer_where where,
>> +		      enum trailer_if_exists if_exists,
>> +		      enum trailer_if_missing if_missing,
>> +		      struct trailer_conf *conf)
>> +{
>> +	if (where != WHERE_DEFAULT)
>> +		conf->where = where;
>> +	if (if_exists != EXISTS_DEFAULT)
>> +		conf->if_exists = if_exists;
>> +	if (if_missing != MISSING_DEFAULT)
>> +		conf->if_missing = if_missing;
>> +}
>
> So, this is such a helper function. It is curious that it
> deliberately lacks the ability to reset what has already been
> configured back to the default.
>
> For example, we earlier saw this original code ...
>
>> -	item = xmalloc(sizeof(*item));
>> -	item->text = arg;
>> -	item->where = where;
>> -	item->if_exists = if_exists;
>> -	item->if_missing = if_missing;
>
> ... gets replaced with this call.
>
>> +	struct trailer_conf *conf_current = new_trailer_conf();
>> ...
>> +		trailer_conf_set(where, if_exists, if_missing, conf_current);
>
> For this conversion to be correct, we require that the value of the
> *_DEFAULT macro to be 0 and/or these three values getting assigned
> are not *_DEFAULT values.  Maybe that may happen to be the case in
> today's code, but such an unwritten assumption makes me feel nervous.
>
> I am not sure if it is worth the confusion factor to make a function
> whose name is $anything_set() to be making a conditional assignment.
> If such a conditional assignment *also* happens often and deserves
> to have its own helper, it is fine to have such a helper for
> conditional assignment, but calling it $anything_set() is probably a
> mistake.

Agreed. I've split this out into separate helpers that do not have
conditionals built into them.

> Other than that, the main thrust of this step seems sensible.

Ack.

Overall, for the next reroll I have the commits broken down and it is
mostly the same. I just need to wait for CI to pass and update the cover
letter, before sending it (as v4). Should be ready later tonight.
Thanks.

[PATCH v4 00/28] Enrich Trailer API

[Linus Arver via GitGitGadget]

This patch series is the first 10 patches of a larger cleanup/bugfix series
(henceforth "larger series") I've been working on. The main goal of this
series is to begin the process of "libifying" the trailer API. By "API" I
mean the interface exposed in trailer.h. The larger series brings a number
of additional cleanups (exposing and fixing some bugs along the way), and
builds on top of this series.

When the larger series is merged, we will be in a good state to additionally
pursue the following goals:

 1. "API reuse inside Git": make the API expressive enough to eliminate any
    need by other parts of Git to use the interpret-trailers builtin as a
    subprocess (instead they could just use the API directly);
 2. "API stability": add unit tests to codify the expected behavior of API
    functions; and
 3. "API documentation": create developer-focused documentation to explain
    how to use the API effectively, noting any API limitations or
    anti-patterns.

The reason why the larger series itself doesn't tackle these goals directly
is because I believe that API code should be thought from the ground up with
a libification-focused perspective. Some structs and functions exposed in
the API today should probably not be libified (read: kept in trailer.h) as
is. For example, the "trailer_iterator" struct has a "private" member and it
feels wrong to allow API users to peek inside here (and take at face value
our future API users' pinky promise that they won't depend on those private
internals not meant for public consumption).

One pattern we could use here to cleanly separate "what is the API"
(publicly exposed) and "what is the implementation" (private) is the
pointer-to-implementation ("pimpl") idiom. There may be other appropriate
patterns, but I've chosen this one because it's a simple, low-level concept
(put structs in foo.c instead of foo.h), which has far-reaching high-level
consequences (API users must rely exclusively on the API to make use of such
private structs, via opaque pointers). The pimpl idiom for C comes from the
book "C Interfaces and Implementations" (see patch "trailer: make
trailer_info struct private").

The idea of turning a public struct into a private one is a fundamental
question of libification because it forces us to reconsider all of the data
structures we have and how they're actually used by already existing users.
For the trailer API, those existing users are the "interpret-trailers"
builtin command, and anything else that includes the "trailer.h" header file
(e.g., sequencer.c). One advantage of this idiom is that even the compiler
understands it --- the compiler will loudly complain if you try to access
the innards of a private struct through an opaque pointer.

Another advantage of this idiom is that it helps to reduce the probability
of breaking changes in the API. Because a private struct's members are out
of view from our users (they only know about opaque pointers to the private
struct, not its members), we are free to modify the members of the struct at
any time, as much as we like, as long as we don't break the semantics of the
exposed API functions (which is why unit-testing these API functions will be
crucial long-term).

If this pimpl idiom turns out to be a mistake, undoing it is easy --- just
move the relevant struct definition from foo.c to the header file. So it's a
great way to try things out without digging ourselves into a pit of despair
that will be difficult to get out of.

With the libification-focused goals out of the way, let's turn to this patch
series in more detail.

Currently, we have "process_trailers()" in trailer.h which does many
different things (parse command-line arguments, create temporary files, etc)
that are independent of the concept of "trailers". Keeping this function as
an API function would make unit-testing it difficult. While there is no
technical reason why we couldn't write unit tests for the smaller functions
that are called within process_trailers(), doing so would involve testing
private ("static" in trailer.c) functions instead of API functions, which
defeats the goal of "API stability" mentioned earlier above.

As an alternative to how things are done in this patch series, we could keep
trailer.h intact and decide to unit-test the existing "trailer_info_get()"
function which does most of the trailer parsing work (and is used by
sequencer.c). However this function wouldn't be easy to test either, because
the resulting "trailer_info" struct merely contains the unparsed "trailers"
lines. So the unit test (if it wants to inspect the result of parsing these
lines) would have to invoke additional parsing functions itself. And at that
point it would no longer be a unit test in the traditional sense, because it
would be invoking multiple functions at once.

In summary this series breaks up "process_trailers()" into smaller pieces,
exposing many of the parts relevant to trailer-related processing in
trailer.h. This will force us to eventually introduce unit tests for these
API functions, but that is a good thing for API stability.

In the future after libification is "complete", users external to Git will
be able to use the same trailer processing API used by the
interpret-trailers builtin. For example, a web server may want to parse
trailers the same way that Git would parse them, without having to call
interpret-trailers as a subprocess. This use case was the original
motivation behind my work in this area.

Thanks to the aggressive refactoring in this series, I've been able to
identify and fix several bugs in our existing implementation. Those fixes
build on top of this series but were not included here, in order to keep
this series small. Below is a "shortlog" of those fixes I have locally:

 * "trailer: trailer replacement should not change its position" (If we
   found a trailer we'd like to replace, preserve its position relative to
   the other trailers found in the trailer block, instead of always moving
   it to the beginning or end of the entire trailer block.)
 * "interpret-trailers: preserve trailers coming from the input" (Sometimes,
   the parsed trailers from the input will be formatted differently
   depending on whether we provide --only-trailers or not. Make the trailers
   that were not modified and which are coming directly from the input get
   formatted the same way, regardless of this flag.)
 * "interpret-trailers: do not modify the input if NOP" (Refrain from
   subtracting or adding a newline around the patch divider "---" if we are
   not adding new trailers.)
 * "trailer formatter: split up format_trailer() monolith" (Fix a bug in
   git-log where we still printed a blank newline even if we didn't want to
   format anything.)
 * "interpret-trailers: fail if given unrecognized arguments" (E.g., for
   "--where", only accept recognized WHERE_* enum values. If we get
   something unrecognized, fail with an error instead of silently doing
   nothing. Ditto for "--if-exists" and "--if-missing".)


Notable changes in v4
=====================

 * Patches 3, 4, 5, and 8 have been broken up into smaller steps. There are
   28 instead of 10 patches now, but these 28 should be much easier to
   review than the (previously condensed) 10.
 * NEW Patch 1: "trailer: free trailer_info after all related usage" fixes
   awkward use-after-free coding style
 * NEW Patch 2: "shortlog: add test for de-duplicating folded trailers"
   increases test coverage related to trailer iterators and "unfold_value()"
 * NEW Patch 27: "trailer_set_*(): put out parameter at the end" is a small
   refactor to reorder parameters.
 * Patches 5-16: These smaller patches make up Patch 3 from v3.
 * Patches 17-18: These smaller patches make up Patch 4 from v3.
 * Patches 19-20: These smaller patches make up Patch 5 from v3.
 * Patches 23-26: These smaller patches make up Patch 8 from v3.
 * Anonymize unambiguous parameters in <trailer.h>.


Notable changes in v3
=====================

 * Squashed Patch 4 into Patch 3 ("trailer: unify trailer formatting
   machinery"), to avoid breaking the build ("-Werror=unused-function"
   violations)
 * NEW (Patch 10): Introduce "trailer template" terminology for readability
   (no behavioral change)
 * (API function) Rename default_separators() to
   trailer_default_separators()
 * (API function) Rename new_trailers_clear() to free_trailer_templates()
 * trailer.h: for single-parameter functions, anonymize the parameter name
   to reduce verbosity


Notable changes in v2
=====================

 * (cover letter) Discuss goals of the larger series in more detail,
   especially the pimpl idiom
 * (cover letter) List bug fixes pending in the larger series that depend on
   this series
 * Reorder function parameters to have trailer options at the beginning (and
   out parameters toward the end)
 * "sequencer: use the trailer iterator": prefer C string instead of strbuf
   for new "raw" field
 * Patch 1 (was Patch 2) also renames ensure_configured() to
   trailer_config_init() (forgot to rename this one previously)

Linus Arver (28):
  trailer: free trailer_info _after_ all related usage
  shortlog: add test for de-duplicating folded trailers
  trailer: prepare to expose functions as part of API
  trailer: move interpret_trailers() to interpret-trailers.c
  trailer: start preparing for formatting unification
  trailer_info_get(): reorder parameters
  format_trailers(): use strbuf instead of FILE
  format_trailer_info(): move "fast path" to caller
  format_trailers_from_commit(): indirectly call trailer_info_get()
  format_trailer_info(): use trailer_item objects
  format_trailer_info(): drop redundant unfold_value()
  format_trailer_info(): append newline for non-trailer lines
  trailer: begin formatting unification
  format_trailer_info(): teach it about opts->trim_empty
  format_trailer_info(): avoid double-printing the separator
  trailer: finish formatting unification
  trailer: teach iterator about non-trailer lines
  sequencer: use the trailer iterator
  trailer: make trailer_info struct private
  trailer: retire trailer_info_get() from API
  trailer: spread usage of "trailer_block" language
  trailer: prepare to delete "parse_trailers_from_command_line_args()"
  trailer: add new helper functions to API
  trailer_add_arg_item(): drop new_trailer_item usage
  trailer: deprecate "new_trailer_item" struct from API
  trailer: unify "--trailer ..." arg handling
  trailer_set_*(): put out parameter at the end
  trailer: introduce "template" term for readability

 builtin/interpret-trailers.c | 189 ++++++--
 pretty.c                     |   2 +-
 ref-filter.c                 |   2 +-
 sequencer.c                  |  27 +-
 t/t4201-shortlog.sh          |  32 ++
 trailer.c                    | 811 +++++++++++++++++------------------
 trailer.h                    | 109 ++---
 7 files changed, 642 insertions(+), 530 deletions(-)


base-commit: a54a84b333adbecf7bc4483c0e36ed5878cac17b
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1632%2Flistx%2Ftrailer-api-refactor-part-1-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1632/listx/trailer-api-refactor-part-1-v4
Pull-Request: https://github.com/gitgitgadget/git/pull/1632

Range-diff vs v3:

  -:  ----------- >  1:  652df25f30e trailer: free trailer_info _after_ all related usage
  -:  ----------- >  2:  fdccaca2ba0 shortlog: add test for de-duplicating folded trailers
  1:  72cc12a3066 !  3:  4372af244f0 trailer: prepare to expose functions as part of API
     @@ Commit message
          them by their "trailer" moniker, just like all the other functions
          already exposed by trailer.h.
      
     -    Take the opportunity to start putting trailer processions options (opts)
     +    Take the opportunity to start putting trailer processing options (opts)
          as the first parameter. This will be the pattern going forward in this
          series.
      
     @@ trailer.c: void process_trailers(const char *file,
       	}
       
      -	print_all(outfile, &head, opts);
     -+	format_trailers(opts, &head, outfile);
     - 
      -	free_all(&head);
     ++	format_trailers(opts, &head, outfile);
      +	free_trailers(&head);
     - 	trailer_info_release(&info);
       
       	/* Print the lines after the trailers as is */
     + 	if (!opts->only_trailers)
      @@ trailer.c: void trailer_info_get(struct trailer_info *info, const char *str,
       	size_t nr = 0, alloc = 0;
       	char **last = NULL;
  2:  cafa39d1048 !  4:  4073b8eb510 trailer: move interpret_trailers() to interpret-trailers.c
     @@ Commit message
          trailer: move interpret_trailers() to interpret-trailers.c
      
          The interpret-trailers.c builtin is the only place we need to call
     -    interpret_trailers(), so move its definition there.
     +    interpret_trailers(), so move its definition there (together with a few
     +    helper functions called only by it) and remove its external declaration
     +    from <trailer.h>.
      
     -    Delete the corresponding declaration from trailer.h, which then forces
     -    us to expose the working innards of that function. This enriches
     -    trailer.h with a more granular API, which can then be unit-tested in the
     -    future (because interpret_trailers() by itself does too many things to
     -    be able to be easily unit-tested).
     +    Several helper functions that are called by interpret_trailers() remain
     +    in trailer.c because other callers in the same file still call them.
     +    Declare them in <trailer.h> so that interpret_trailers() (now in
     +    builtin/interpret-trailers.c) can continue calling them as a trailer API
     +    user.
     +
     +    This enriches <trailer.h> with a more granular API, which can then be
     +    unit-tested in the future (because interpret_trailers() by itself does
     +    too many things to be able to be easily unit-tested).
      
          Take this opportunity to demote some file-handling functions out of the
          trailer API implementation, as these have nothing to do with trailers.
      
     +    Helped-by: Junio C Hamano <gitster@pobox.com>
          Signed-off-by: Linus Arver <linusa@google.com>
      
       ## builtin/interpret-trailers.c ##
     @@ builtin/interpret-trailers.c: static int parse_opt_parse(const struct option *op
      +	}
      +
      +	format_trailers(opts, &head, outfile);
     -+
      +	free_trailers(&head);
     -+	trailer_info_release(&info);
      +
      +	/* Print the lines after the trailers as is */
      +	if (!opts->only_trailers)
      +		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
     ++	trailer_info_release(&info);
      +
      +	if (opts->in_place)
      +		if (rename_tempfile(&trailers_tempfile, file))
     @@ trailer.c: static void free_trailers(struct list_head *trailers)
      -	}
      -
      -	format_trailers(opts, &head, outfile);
     --
      -	free_trailers(&head);
     --	trailer_info_release(&info);
      -
      -	/* Print the lines after the trailers as is */
      -	if (!opts->only_trailers)
      -		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
     +-	trailer_info_release(&info);
      -
      -	if (opts->in_place)
      -		if (rename_tempfile(&trailers_tempfile, file))
     @@ trailer.h: struct process_trailer_options {
      +void process_trailers_lists(struct list_head *head,
      +			    struct list_head *arg_head);
      +
     -+void parse_trailers(const struct process_trailer_options *opts,
     -+		    struct trailer_info *info,
     ++void parse_trailers(const struct process_trailer_options *,
     ++		    struct trailer_info *,
      +		    const char *str,
      +		    struct list_head *head);
       
     @@ trailer.h: struct process_trailer_options {
       void trailer_info_release(struct trailer_info *info);
       
      +void trailer_config_init(void);
     -+void format_trailers(const struct process_trailer_options *opts,
     ++void format_trailers(const struct process_trailer_options *,
      +		     struct list_head *trailers, FILE *outfile);
      +void free_trailers(struct list_head *);
      +
  3:  5c7a2354df0 !  5:  b2a0f7829a1 trailer: unify trailer formatting machinery
     @@ Metadata
      Author: Linus Arver <linusa@google.com>
      
       ## Commit message ##
     -    trailer: unify trailer formatting machinery
     +    trailer: start preparing for formatting unification
      
     -    Currently have two functions for formatting trailers exposed in
     -    trailer.h:
     +    Currently there are two functions for formatting trailers in
     +    <trailer.h>:
      
     -        void format_trailers(FILE *outfile, struct list_head *head,
     -                            const struct process_trailer_options *opts);
     +        void format_trailers(const struct process_trailer_options *,
     +                             struct list_head *trailers, FILE *outfile);
      
              void format_trailers_from_commit(struct strbuf *out, const char *msg,
     -                                        const struct process_trailer_options *opts);
     +                                         const struct process_trailer_options *opts);
      
     -    and previously these functions, although similar enough (even taking the
     -    same process_trailer_options struct pointer), did not build on each
     -    other.
     +    and although they are similar enough (even taking the same
     +    process_trailer_options struct pointer) they are used quite differently.
     +    One might intuitively think that format_trailers_from_commit() builds on
     +    top of format_trailers(), but this is not the case. Instead
     +    format_trailers_from_commit() calls format_trailer_info() and
     +    format_trailers() is never called in that codepath.
      
     -    Make format_trailers_from_commit() rely on format_trailers(). Teach
     -    format_trailers() to process trailers with the additional
     -    process_trailer_options fields like opts->key_only which is only used by
     -    format_trailers_from_commit() and not builtin/interpret-trailers.c.
     -    While we're at it, reorder parameters to put the trailer processing
     -    options first, and the out parameter (strbuf we write into) at the end.
     +    This is a preparatory refactor to help us deprecate format_trailers() in
     +    favor of format_trailer_info() (at which point we can rename the latter
     +    to the former). When the deprecation is complete, both
     +    format_trailers_from_commit(), and the interpret-trailers builtin will
     +    be able to call into the same helper function (instead of
     +    format_trailers() and format_trailer_info(), respectively). Unifying the
     +    formatters is desirable because it simplifies the API.
      
     -    This unification will allow us to delete the format_trailer_info() and
     -    print_tok_val() functions in the next patch. They are not deleted here
     -    in order to keep the diff small.
     +    Reorder parameters for format_trailers_from_commit() to prefer
      
     -    Helped-by: Junio C Hamano <gitster@pobox.com>
     -    Signed-off-by: Linus Arver <linusa@google.com>
     +        const struct process_trailer_options *opts
      
     - ## builtin/interpret-trailers.c ##
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
     - {
     - 	LIST_HEAD(head);
     - 	struct strbuf sb = STRBUF_INIT;
     -+	struct strbuf trailer_block = STRBUF_INIT;
     - 	struct trailer_info info;
     - 	FILE *outfile = stdout;
     - 
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
     - 		process_trailers_lists(&head, &arg_head);
     - 	}
     - 
     --	format_trailers(opts, &head, outfile);
     -+	/* Print trailer block. */
     -+	format_trailers(opts, &head, &trailer_block);
     -+	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
     -+	strbuf_release(&trailer_block);
     - 
     - 	free_trailers(&head);
     - 	trailer_info_release(&info);
     +    as the first parameter, because these options are intimately tied to
     +    formatting trailers. And take
     +
     +        struct strbuf *out
     +
     +    last, because it's an "out parameter" (something that the caller wants
     +    to use as the output of this function).
     +
     +    Signed-off-by: Linus Arver <linusa@google.com>
      
       ## pretty.c ##
      @@ pretty.c: static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */
     @@ ref-filter.c: static void grab_sub_body_contents(struct atom_value *val, int der
       		} else if (atom->u.contents.option == C_BARE)
      
       ## trailer.c ##
     -@@ trailer.c: static char last_non_space_char(const char *s)
     - 	return '\0';
     - }
     - 
     --static void print_tok_val(FILE *outfile, const char *tok, const char *val)
     --{
     --	char c;
     --
     --	if (!tok) {
     --		fprintf(outfile, "%s\n", val);
     --		return;
     --	}
     --
     --	c = last_non_space_char(tok);
     --	if (!c)
     --		return;
     --	if (strchr(separators, c))
     --		fprintf(outfile, "%s%s\n", tok, val);
     --	else
     --		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
     --}
     --
     --void format_trailers(const struct process_trailer_options *opts,
     --		     struct list_head *trailers, FILE *outfile)
     --{
     --	struct list_head *pos;
     --	struct trailer_item *item;
     --	list_for_each(pos, trailers) {
     --		item = list_entry(pos, struct trailer_item, list);
     --		if ((!opts->trim_empty || strlen(item->value) > 0) &&
     --		    (!opts->only_trailers || item->token))
     --			print_tok_val(outfile, item->token, item->value);
     --	}
     --}
     --
     - static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
     - {
     - 	struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
     -@@ trailer.c: static void unfold_value(struct strbuf *val)
     - 	strbuf_release(&out);
     - }
     - 
     -+void format_trailers(const struct process_trailer_options *opts,
     -+		     struct list_head *trailers,
     -+		     struct strbuf *out)
     -+{
     -+	struct list_head *pos;
     -+	struct trailer_item *item;
     -+	int need_separator = 0;
     -+
     -+	list_for_each(pos, trailers) {
     -+		item = list_entry(pos, struct trailer_item, list);
     -+		if (item->token) {
     -+			char c;
     -+
     -+			struct strbuf tok = STRBUF_INIT;
     -+			struct strbuf val = STRBUF_INIT;
     -+			strbuf_addstr(&tok, item->token);
     -+			strbuf_addstr(&val, item->value);
     -+
     -+			/*
     -+			 * Skip key/value pairs where the value was empty. This
     -+			 * can happen from trailers specified without a
     -+			 * separator, like `--trailer "Reviewed-by"` (no
     -+			 * corresponding value).
     -+			 */
     -+			if (opts->trim_empty && !strlen(item->value))
     -+				continue;
     -+
     -+			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
     -+				if (opts->unfold)
     -+					unfold_value(&val);
     -+
     -+				if (opts->separator && need_separator)
     -+					strbuf_addbuf(out, opts->separator);
     -+				if (!opts->value_only)
     -+					strbuf_addbuf(out, &tok);
     -+				if (!opts->key_only && !opts->value_only) {
     -+					if (opts->key_value_separator)
     -+						strbuf_addbuf(out, opts->key_value_separator);
     -+					else {
     -+						c = last_non_space_char(tok.buf);
     -+						if (c) {
     -+							if (!strchr(separators, c))
     -+								strbuf_addf(out, "%c ", separators[0]);
     -+						}
     -+					}
     -+				}
     -+				if (!opts->key_only)
     -+					strbuf_addbuf(out, &val);
     -+				if (!opts->separator)
     -+					strbuf_addch(out, '\n');
     -+
     -+				need_separator = 1;
     -+			}
     -+
     -+			strbuf_release(&tok);
     -+			strbuf_release(&val);
     -+		} else if (!opts->only_trailers) {
     -+			if (opts->separator && need_separator) {
     -+				strbuf_addbuf(out, opts->separator);
     -+			}
     -+			strbuf_addstr(out, item->value);
     -+			if (opts->separator)
     -+				strbuf_rtrim(out);
     -+			else
     -+				strbuf_addch(out, '\n');
     -+
     -+			need_separator = 1;
     -+		}
     -+
     -+	}
     -+}
     -+
     - /*
     -  * Parse trailers in "str", populating the trailer info and "head"
     -  * linked list structure.
      @@ trailer.c: void trailer_info_release(struct trailer_info *info)
       	free(info->trailers);
       }
       
      -static void format_trailer_info(struct strbuf *out,
     --				const struct trailer_info *info,
     --				const char *msg,
     ++static void format_trailer_info(const struct process_trailer_options *opts,
     + 				const struct trailer_info *info,
     + 				const char *msg,
      -				const struct process_trailer_options *opts)
     -+void format_trailers_from_commit(const struct process_trailer_options *opts,
     -+				 const char *msg,
     -+				 struct strbuf *out)
     ++				struct strbuf *out)
       {
     --	size_t origlen = out->len;
     --	size_t i;
     -+	LIST_HEAD(head);
     -+	struct trailer_info info;
     -+
     -+	parse_trailers(opts, &info, msg, &head);
     + 	size_t origlen = out->len;
     + 	size_t i;
     +@@ trailer.c: static void format_trailer_info(struct strbuf *out,
     + 
     + }
       
     - 	/* If we want the whole block untouched, we can take the fast path. */
     - 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
     - 	    !opts->separator && !opts->key_only && !opts->value_only &&
     - 	    !opts->key_value_separator) {
     --		strbuf_add(out, msg + info->trailer_block_start,
     --			   info->trailer_block_end - info->trailer_block_start);
     --		return;
     --	}
     --
     --	for (i = 0; i < info->trailer_nr; i++) {
     --		char *trailer = info->trailers[i];
     --		ssize_t separator_pos = find_separator(trailer, separators);
     --
     --		if (separator_pos >= 1) {
     --			struct strbuf tok = STRBUF_INIT;
     --			struct strbuf val = STRBUF_INIT;
     --
     --			parse_trailer(&tok, &val, NULL, trailer, separator_pos);
     --			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
     --				if (opts->unfold)
     --					unfold_value(&val);
     --
     --				if (opts->separator && out->len != origlen)
     --					strbuf_addbuf(out, opts->separator);
     --				if (!opts->value_only)
     --					strbuf_addbuf(out, &tok);
     --				if (!opts->key_only && !opts->value_only) {
     --					if (opts->key_value_separator)
     --						strbuf_addbuf(out, opts->key_value_separator);
     --					else
     --						strbuf_addstr(out, ": ");
     --				}
     --				if (!opts->key_only)
     --					strbuf_addbuf(out, &val);
     --				if (!opts->separator)
     --					strbuf_addch(out, '\n');
     --			}
     --			strbuf_release(&tok);
     --			strbuf_release(&val);
     --
     --		} else if (!opts->only_trailers) {
     --			if (opts->separator && out->len != origlen) {
     --				strbuf_addbuf(out, opts->separator);
     --			}
     --			strbuf_addstr(out, trailer);
     --			if (opts->separator) {
     --				strbuf_rtrim(out);
     --			}
     --		}
     --	}
     --
     --}
     --
      -void format_trailers_from_commit(struct strbuf *out, const char *msg,
      -				 const struct process_trailer_options *opts)
     --{
     --	struct trailer_info info;
     -+		strbuf_add(out, msg + info.trailer_block_start,
     -+			   info.trailer_block_end - info.trailer_block_start);
     -+	} else
     -+		format_trailers(opts, &head, out);
     ++void format_trailers_from_commit(const struct process_trailer_options *opts,
     ++				 const char *msg,
     ++				 struct strbuf *out)
     + {
     + 	struct trailer_info info;
       
     --	trailer_info_get(&info, msg, opts);
     + 	trailer_info_get(&info, msg, opts);
      -	format_trailer_info(out, &info, msg, opts);
     -+	free_trailers(&head);
     ++	format_trailer_info(opts, &info, msg, out);
       	trailer_info_release(&info);
       }
       
      
       ## trailer.h ##
     -@@ trailer.h: void trailer_info_release(struct trailer_info *info);
     - 
     - void trailer_config_init(void);
     - void format_trailers(const struct process_trailer_options *opts,
     --		     struct list_head *trailers, FILE *outfile);
     -+		     struct list_head *trailers,
     -+		     struct strbuf *out);
     - void free_trailers(struct list_head *);
     - 
     - /*
     -- * Format the trailers from the commit msg "msg" into the strbuf "out".
     -- * Note two caveats about "opts":
     -- *
     -- *   - this is primarily a helper for pretty.c, and not
     -- *     all of the flags are supported.
     -- *
     -- *   - this differs from process_trailers slightly in that we always format
     -- *     only the trailer block itself, even if the "only_trailers" option is not
     -- *     set.
     -+ * Convenience function to format the trailers from the commit msg "msg" into
     -+ * the strbuf "out". Reuses format_trailers internally.
     +@@ trailer.h: void free_trailers(struct list_head *);
     +  *     only the trailer block itself, even if the "only_trailers" option is not
     +  *     set.
        */
      -void format_trailers_from_commit(struct strbuf *out, const char *msg,
      -				 const struct process_trailer_options *opts);
  -:  ----------- >  6:  c1760f80356 trailer_info_get(): reorder parameters
  -:  ----------- >  7:  9dc912b5bc5 format_trailers(): use strbuf instead of FILE
  -:  ----------- >  8:  b97c06d8bc3 format_trailer_info(): move "fast path" to caller
  -:  ----------- >  9:  6906910417a format_trailers_from_commit(): indirectly call trailer_info_get()
  -:  ----------- > 10:  f5b7ba08aa7 format_trailer_info(): use trailer_item objects
  -:  ----------- > 11:  457f2a839d5 format_trailer_info(): drop redundant unfold_value()
  -:  ----------- > 12:  a72eca301f7 format_trailer_info(): append newline for non-trailer lines
  -:  ----------- > 13:  ad77c33e457 trailer: begin formatting unification
  -:  ----------- > 14:  11f854399db format_trailer_info(): teach it about opts->trim_empty
  -:  ----------- > 15:  ba1f387747b format_trailer_info(): avoid double-printing the separator
  -:  ----------- > 16:  31725832224 trailer: finish formatting unification
  4:  bf2b8e1a3c4 ! 17:  6f17c022b15 sequencer: use the trailer iterator
     @@ Metadata
      Author: Linus Arver <linusa@google.com>
      
       ## Commit message ##
     -    sequencer: use the trailer iterator
     +    trailer: teach iterator about non-trailer lines
      
     -    This patch allows for the removal of "trailer_info_get()" from the
     -    trailer.h API, which will be in the next patch.
     +    Previously the iterator did not iterate over non-trailer lines. This was
     +    somewhat unfortunate, because trailer blocks could have non-trailer
     +    lines in them since 146245063e (trailer: allow non-trailers in trailer
     +    block, 2016-10-21), which was before the iterator was created in
     +    f0939a0eb1 (trailer: add interface for iterating over commit trailers,
     +    2020-09-27).
      
     -    Instead of calling "trailer_info_get()", which is a low-level function
     -    in the trailers implementation (trailer.c), call
     -    trailer_iterator_advance(), which was specifically designed for public
     -    consumption in f0939a0eb1 (trailer: add interface for iterating over
     -    commit trailers, 2020-09-27).
     +    So if trailer API users wanted to iterate over all lines in a trailer
     +    block (including non-trailer lines), they could not use the iterator and
     +    were forced to use the lower-level trailer_info struct directly (which
     +    provides a raw string array that includes all lines in the trailer
     +    block).
      
     -    Avoiding "trailer_info_get()" means we don't have to worry about options
     -    like "no_divider" (relevant for parsing trailers). We also don't have to
     -    check for things like "info.trailer_start == info.trailer_end" to see
     -    whether there were any trailers (instead we can just check to see
     -    whether the iterator advanced at all).
     +    Change the iterator's behavior so that we also iterate over non-trailer
     +    lines, instead of skipping over them. The new "raw" member of the
     +    iterator allows API users to access previously inaccessible non-trailer
     +    lines. Reword the variable "trailer" to just "line" because this
     +    variable can now hold both trailer lines _and_ non-trailer lines.
      
     -    Also, teach the iterator about non-trailer lines, by adding a new field
     -    called "raw" to hold both trailer and non-trailer lines. This is
     -    necessary because a "trailer block" is a list of trailer lines of at
     -    least 25% trailers (see 146245063e (trailer: allow non-trailers in
     -    trailer block, 2016-10-21)), such that it may hold non-trailer lines.
     +    The new "raw" member is important because anyone currently not using the
     +    iterator is using trailer_info's raw string array directly to access
     +    lines to check what the combined key + value looks like. If we didn't
     +    provide a "raw" member here, iterator users would have to re-construct
     +    the unparsed line by concatenating the key and value back together again
     +    --- which places an undue burden for iterator users.
      
     -    Signed-off-by: Linus Arver <linusa@google.com>
     +    The next patch demonstrates the use of the iterator in sequencer.c as an
     +    example of where "raw" will be useful, so that it can start using the
     +    iterator.
      
     - ## builtin/shortlog.c ##
     -@@ builtin/shortlog.c: static void insert_records_from_trailers(struct shortlog *log,
     - 					 const char *oneline)
     - {
     - 	struct trailer_iterator iter;
     --	const char *commit_buffer, *body;
     -+	const char *commit_buffer, *body, *value;
     - 	struct strbuf ident = STRBUF_INIT;
     - 
     - 	if (!log->trailers.nr)
     -@@ builtin/shortlog.c: static void insert_records_from_trailers(struct shortlog *log,
     - 
     - 	trailer_iterator_init(&iter, body);
     - 	while (trailer_iterator_advance(&iter)) {
     --		const char *value = iter.val.buf;
     -+		if (!iter.is_trailer)
     -+			continue;
     -+
     -+		value = iter.val.buf;
     - 
     - 		if (!string_list_has_string(&log->trailers, iter.key.buf))
     - 			continue;
     +    For the existing use of the iterator in builtin/shortlog.c, we don't
     +    have to change the code there because that code does
      
     - ## sequencer.c ##
     -@@ sequencer.c: static const char *get_todo_path(const struct replay_opts *opts)
     - static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
     - 	size_t ignore_footer)
     - {
     --	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
     --	struct trailer_info info;
     --	size_t i;
     --	int found_sob = 0, found_sob_last = 0;
     --	char saved_char;
     --
     --	opts.no_divider = 1;
     -+	struct trailer_iterator iter;
     -+	size_t i = 0, found_sob = 0;
     -+	char saved_char = sb->buf[sb->len - ignore_footer];
     - 
     - 	if (ignore_footer) {
     --		saved_char = sb->buf[sb->len - ignore_footer];
     - 		sb->buf[sb->len - ignore_footer] = '\0';
     - 	}
     - 
     --	trailer_info_get(&info, sb->buf, &opts);
     -+	trailer_iterator_init(&iter, sb->buf);
     -+	while (trailer_iterator_advance(&iter)) {
     -+		i++;
     -+		if (sob &&
     -+		    iter.is_trailer &&
     -+		    !strncmp(iter.raw, sob->buf, sob->len)) {
     -+			found_sob = i;
     -+		}
     -+	}
     -+	trailer_iterator_release(&iter);
     - 
     - 	if (ignore_footer)
     - 		sb->buf[sb->len - ignore_footer] = saved_char;
     - 
     --	if (info.trailer_block_start == info.trailer_block_end)
     -+	if (!i)
     - 		return 0;
     - 
     --	for (i = 0; i < info.trailer_nr; i++)
     --		if (sob && !strncmp(info.trailers[i], sob->buf, sob->len)) {
     --			found_sob = 1;
     --			if (i == info.trailer_nr - 1)
     --				found_sob_last = 1;
     --		}
     --
     --	trailer_info_release(&info);
     --
     --	if (found_sob_last)
     -+	if (found_sob == i)
     - 		return 3;
     - 	if (found_sob)
     - 		return 2;
     +        trailer_iterator_init(&iter, body);
     +        while (trailer_iterator_advance(&iter)) {
     +            const char *value = iter.val.buf;
     +
     +            if (!string_list_has_string(&log->trailers, iter.key.buf))
     +                continue;
     +
     +            ...
     +
     +    and the
     +
     +            if (!string_list_has_string(&log->trailers, iter.key.buf))
     +
     +    condition already skips over non-trailer lines (iter.key.buf is empty
     +    for non-trailer lines, making the comparison still work unmodified even
     +    with this patch).
     +
     +    Signed-off-by: Linus Arver <linusa@google.com>
      
       ## trailer.c ##
      @@ trailer.c: void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
     @@ trailer.c: void trailer_iterator_init(struct trailer_iterator *iter, const char
      -
      -		if (separator_pos < 1)
      -			continue; /* not a real trailer */
     --
     -+	char *line;
     -+	int separator_pos;
      +	if (iter->internal.cur < iter->internal.info.trailer_nr) {
     -+		line = iter->internal.info.trailers[iter->internal.cur++];
     -+		separator_pos = find_separator(line, separators);
     -+		iter->is_trailer = (separator_pos > 0);
     -+
     ++		char *line = iter->internal.info.trailers[iter->internal.cur++];
     ++		int separator_pos = find_separator(line, separators);
     + 
      +		iter->raw = line;
       		strbuf_reset(&iter->key);
       		strbuf_reset(&iter->val);
       		parse_trailer(&iter->key, &iter->val, NULL,
      -			      trailer, separator_pos);
      +			      line, separator_pos);
     + 		/* Always unfold values during iteration. */
       		unfold_value(&iter->val);
       		return 1;
     - 	}
      
       ## trailer.h ##
     -@@ trailer.h: struct trailer_iterator {
     - 	struct strbuf key;
     - 	struct strbuf val;
     - 
     +@@ trailer.h: void format_trailers_from_commit(const struct process_trailer_options *,
     +  *   trailer_iterator_release(&iter);
     +  */
     + struct trailer_iterator {
      +	/*
      +	 * Raw line (e.g., "foo: bar baz") before being parsed as a trailer
      +	 * key/val pair as part of a trailer block. A trailer block can be
     @@ trailer.h: struct trailer_iterator {
      +	 */
      +	const char *raw;
      +
     -+	/*
     -+	 * 1 if the raw line was parsed as a trailer line (key/val pair).
     -+	 */
     -+	int is_trailer;
     -+
     - 	/* private */
     - 	struct {
     - 		struct trailer_info info;
     + 	struct strbuf key;
     + 	struct strbuf val;
     + 
  -:  ----------- > 18:  cc92dfb0bda sequencer: use the trailer iterator
  5:  c19c1dcc592 ! 19:  f5f0d06613f trailer: make trailer_info struct private
     @@ Commit message
            (2) external API users are unable to peer inside this struct (because
                it is only ever exposed as an opaque pointer).
      
     -    This change exposes some deficiencies in the API, mainly with regard to
     -    information about the location of the trailer block that was parsed.
     -    Expose new API functions to access this information (needed by
     -    builtin/interpret-trailers.c).
     +    There are a couple disadvantages:
     +
     +      (A) every time the member of the struct is accessed an extra pointer
     +          dereference must be done, and
     +
     +      (B) for users of trailer_info outside trailer.c, this struct can no
     +          longer be allocated on the stack and may only be allocated on the
     +          heap (because its definition is hidden away in trailer.c) and
     +          appropriately deallocated by the user.
     +
     +    This patch believes that the benefits outweight the advantages for
     +    designing APIs, as explained below.
     +
     +    Making trailer_info private exposes existing deficiencies in the API.
     +    This is because users of this struct had full access to its internals,
     +    so there wasn't much need to actually design it to be "complete" in the
     +    sense that API users only needed to use what was provided by the API.
     +    For example, the location of the trailer block (start/end offsets
     +    relative to the start of the input text) was accessible by looking at
     +    these struct members directly. Now that the struct is private, we have
     +    to expose new API functions to allow clients to access this
     +    information (see builtin/interpret-trailers.c).
      
          The idea in this patch to hide implementation details behind an "opaque
          pointer" is also known as the "pimpl" (pointer to implementation) idiom
     @@ Commit message
      
          In our case, "struct trailer_info" is now hidden from clients, and the
          ways in which this opaque pointer can be used is limited to the richness
     -    of the trailer.h file. In other words, trailer.h exclusively controls
     -    exactly how "trailer_info" pointers are to be used.
     +    of <trailer.h>. In other words, <trailer.h> exclusively controls exactly
     +    how "trailer_info" pointers are to be used.
      
          [1] Hanson, David R. "C Interfaces and Implementations: Techniques for
              Creating Reusable Software". Addison Wesley, 1997. p. 22
      
     +    Helped-by: Junio C Hamano <gitster@pobox.com>
          Signed-off-by: Linus Arver <linusa@google.com>
      
       ## builtin/interpret-trailers.c ##
     @@ builtin/interpret-trailers.c: static void interpret_trailers(const struct proces
       
       
      @@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
     - 	strbuf_release(&trailer_block);
     - 
     - 	free_trailers(&head);
     --	trailer_info_release(&info);
       
       	/* Print the lines after the trailers as is */
       	if (!opts->only_trailers)
      -		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
     +-	trailer_info_release(&info);
      +		fwrite(sb.buf + trailer_block_end(info), 1, sb.len - trailer_block_end(info), outfile);
     -+
      +	trailer_info_release(info);
       
       	if (opts->in_place)
     @@ trailer.c
       struct conf_info {
       	char *name;
       	char *key;
     -@@ trailer.c: void format_trailers(const struct process_trailer_options *opts,
     - 	}
     +@@ trailer.c: static void unfold_value(struct strbuf *val)
     + 	strbuf_release(&out);
       }
       
      +static struct trailer_info *trailer_info_new(void)
     @@ trailer.c: void format_trailers(const struct process_trailer_options *opts,
      +	struct trailer_info *info = xcalloc(1, sizeof(*info));
      +	return info;
      +}
     -+
     -+static struct trailer_info *trailer_info_get(const struct process_trailer_options *opts,
     -+					     const char *str)
     -+{
     -+	struct trailer_info *info = trailer_info_new();
     -+	size_t end_of_log_message = 0, trailer_block_start = 0;
     -+	struct strbuf **trailer_lines, **ptr;
     -+	char **trailer_strings = NULL;
     -+	size_t nr = 0, alloc = 0;
     -+	char **last = NULL;
     -+
     -+	trailer_config_init();
     -+
     -+	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
     -+	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
     -+
     -+	trailer_lines = strbuf_split_buf(str + trailer_block_start,
     -+					 end_of_log_message - trailer_block_start,
     -+					 '\n',
     -+					 0);
     -+	for (ptr = trailer_lines; *ptr; ptr++) {
     -+		if (last && isspace((*ptr)->buf[0])) {
     -+			struct strbuf sb = STRBUF_INIT;
     -+			strbuf_attach(&sb, *last, strlen(*last), strlen(*last));
     -+			strbuf_addbuf(&sb, *ptr);
     -+			*last = strbuf_detach(&sb, NULL);
     -+			continue;
     -+		}
     -+		ALLOC_GROW(trailer_strings, nr + 1, alloc);
     -+		trailer_strings[nr] = strbuf_detach(*ptr, NULL);
     -+		last = find_separator(trailer_strings[nr], separators) >= 1
     -+			? &trailer_strings[nr]
     -+			: NULL;
     -+		nr++;
     -+	}
     -+	strbuf_list_free(trailer_lines);
     -+
     -+	info->blank_line_before_trailer = ends_with_blank_line(str,
     -+							       trailer_block_start);
     -+	info->trailer_block_start = trailer_block_start;
     -+	info->trailer_block_end = end_of_log_message;
     -+	info->trailers = trailer_strings;
     -+	info->trailer_nr = nr;
     -+
     -+	return info;
     -+}
      +
       /*
        * Parse trailers in "str", populating the trailer info and "head"
     @@ trailer.c: void format_trailers(const struct process_trailer_options *opts,
       	struct strbuf val = STRBUF_INIT;
       	size_t i;
       
     --	trailer_info_get(info, str, opts);
     +-	trailer_info_get(opts, str, info);
      +	info = trailer_info_get(opts, str);
       
       	for (i = 0; i < info->trailer_nr; i++) {
     @@ trailer.c: void free_trailers(struct list_head *trailers)
       	}
       }
       
     --void trailer_info_get(struct trailer_info *info, const char *str,
     --		      const struct process_trailer_options *opts)
     +-void trailer_info_get(const struct process_trailer_options *opts,
     +-		      const char *str,
     +-		      struct trailer_info *info)
      +size_t trailer_block_start(struct trailer_info *info)
     - {
     --	size_t end_of_log_message = 0, trailer_block_start = 0;
     --	struct strbuf **trailer_lines, **ptr;
     --	char **trailer_strings = NULL;
     --	size_t nr = 0, alloc = 0;
     --	char **last = NULL;
     --
     --	trailer_config_init();
     --
     --	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
     --	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
     ++{
      +	return info->trailer_block_start;
      +}
     - 
     --	trailer_lines = strbuf_split_buf(str + trailer_block_start,
     --					 end_of_log_message - trailer_block_start,
     --					 '\n',
     --					 0);
     --	for (ptr = trailer_lines; *ptr; ptr++) {
     --		if (last && isspace((*ptr)->buf[0])) {
     --			struct strbuf sb = STRBUF_INIT;
     --			strbuf_attach(&sb, *last, strlen(*last), strlen(*last));
     --			strbuf_addbuf(&sb, *ptr);
     --			*last = strbuf_detach(&sb, NULL);
     --			continue;
     --		}
     --		ALLOC_GROW(trailer_strings, nr + 1, alloc);
     --		trailer_strings[nr] = strbuf_detach(*ptr, NULL);
     --		last = find_separator(trailer_strings[nr], separators) >= 1
     --			? &trailer_strings[nr]
     --			: NULL;
     --		nr++;
     --	}
     --	strbuf_list_free(trailer_lines);
     ++
      +size_t trailer_block_end(struct trailer_info *info)
      +{
      +	return info->trailer_block_end;
      +}
     - 
     --	info->blank_line_before_trailer = ends_with_blank_line(str,
     --							       trailer_block_start);
     --	info->trailer_block_start = trailer_block_start;
     --	info->trailer_block_end = end_of_log_message;
     --	info->trailers = trailer_strings;
     --	info->trailer_nr = nr;
     ++
      +int blank_line_before_trailer_block(struct trailer_info *info)
      +{
      +	return info->blank_line_before_trailer;
     ++}
     ++
     ++struct trailer_info *trailer_info_get(const struct process_trailer_options *opts,
     ++				      const char *str)
     + {
     ++	struct trailer_info *info = trailer_info_new();
     + 	size_t end_of_log_message = 0, trailer_block_start = 0;
     + 	struct strbuf **trailer_lines, **ptr;
     + 	char **trailer_strings = NULL;
     +@@ trailer.c: void trailer_info_get(const struct process_trailer_options *opts,
     + 	info->trailer_block_end = end_of_log_message;
     + 	info->trailers = trailer_strings;
     + 	info->trailer_nr = nr;
     ++
     ++	return info;
       }
       
       void trailer_info_release(struct trailer_info *info)
     @@ trailer.c: void trailer_info_release(struct trailer_info *info)
      +	free(info);
       }
       
     - void format_trailers_from_commit(const struct process_trailer_options *opts,
     + void format_trailers(const struct process_trailer_options *opts,
      @@ trailer.c: void format_trailers_from_commit(const struct process_trailer_options *opts,
       				 struct strbuf *out)
       {
     - 	LIST_HEAD(head);
     + 	LIST_HEAD(trailers);
      -	struct trailer_info info;
      -
     --	parse_trailers(opts, &info, msg, &head);
     -+	struct trailer_info *info = parse_trailers(opts, msg, &head);
     +-	parse_trailers(opts, &info, msg, &trailers);
     ++	struct trailer_info *info = parse_trailers(opts, msg, &trailers);
       
       	/* If we want the whole block untouched, we can take the fast path. */
       	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
     @@ trailer.c: void format_trailers_from_commit(const struct process_trailer_options
      +		strbuf_add(out, msg + info->trailer_block_start,
      +			   info->trailer_block_end - info->trailer_block_start);
       	} else
     - 		format_trailers(opts, &head, out);
     + 		format_trailers(opts, &trailers, out);
       
     - 	free_trailers(&head);
     + 	free_trailers(&trailers);
      -	trailer_info_release(&info);
      +	trailer_info_release(info);
       }
       
       void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
     - {
     - 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
     -+	struct trailer_info *internal = trailer_info_new();
     +@@ trailer.c: void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
       	strbuf_init(&iter->key, 0);
       	strbuf_init(&iter->val, 0);
       	opts.no_divider = 1;
     --	trailer_info_get(&iter->internal.info, msg, &opts);
     -+	iter->internal.info = internal;
     +-	trailer_info_get(&opts, msg, &iter->internal.info);
      +	iter->internal.info = trailer_info_get(&opts, msg);
       	iter->internal.cur = 0;
       }
       
     -@@ trailer.c: int trailer_iterator_advance(struct trailer_iterator *iter)
     + int trailer_iterator_advance(struct trailer_iterator *iter)
       {
     - 	char *line;
     - 	int separator_pos;
      -	if (iter->internal.cur < iter->internal.info.trailer_nr) {
     --		line = iter->internal.info.trailers[iter->internal.cur++];
     +-		char *line = iter->internal.info.trailers[iter->internal.cur++];
      +	if (iter->internal.cur < iter->internal.info->trailer_nr) {
     -+		line = iter->internal.info->trailers[iter->internal.cur++];
     - 		separator_pos = find_separator(line, separators);
     - 		iter->is_trailer = (separator_pos > 0);
     ++		char *line = iter->internal.info->trailers[iter->internal.cur++];
     + 		int separator_pos = find_separator(line, separators);
       
     + 		iter->raw = line;
      @@ trailer.c: int trailer_iterator_advance(struct trailer_iterator *iter)
       
       void trailer_iterator_release(struct trailer_iterator *iter)
     @@ trailer.h: void parse_trailers_from_command_line_args(struct list_head *arg_head
       void process_trailers_lists(struct list_head *head,
       			    struct list_head *arg_head);
       
     --void parse_trailers(const struct process_trailer_options *opts,
     --		    struct trailer_info *info,
     +-void parse_trailers(const struct process_trailer_options *,
     +-		    struct trailer_info *,
      -		    const char *str,
      -		    struct list_head *head);
     -+struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
     ++struct trailer_info *parse_trailers(const struct process_trailer_options *,
      +				    const char *str,
      +				    struct list_head *head);
     - 
     --void trailer_info_get(struct trailer_info *info, const char *str,
     --		      const struct process_trailer_options *opts);
     -+size_t trailer_block_start(struct trailer_info *info);
     -+size_t trailer_block_end(struct trailer_info *info);
     -+int blank_line_before_trailer_block(struct trailer_info *info);
     ++struct trailer_info *trailer_info_get(const struct process_trailer_options *,
     ++				      const char *str);
     + 
     +-void trailer_info_get(const struct process_trailer_options *,
     +-		      const char *str,
     +-		      struct trailer_info *);
     ++size_t trailer_block_start(struct trailer_info *);
     ++size_t trailer_block_end(struct trailer_info *);
     ++int blank_line_before_trailer_block(struct trailer_info *);
       
       void trailer_info_release(struct trailer_info *info);
       
  -:  ----------- > 20:  607ae7a90cd trailer: retire trailer_info_get() from API
  6:  0a9a7438c3f ! 21:  38f4b4c4135 trailer: spread usage of "trailer_block" language
     @@ builtin/interpret-trailers.c: static void interpret_trailers(const struct proces
       
       	/* Print trailer block. */
      -	format_trailers(opts, &head, &trailer_block);
     ++	format_trailers(opts, &head, &tb);
     + 	free_trailers(&head);
      -	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
      -	strbuf_release(&trailer_block);
     -+	format_trailers(opts, &head, &tb);
      +	fwrite(tb.buf, 1, tb.len, outfile);
      +	strbuf_release(&tb);
       
     - 	free_trailers(&head);
     - 
      -	/* Print the lines after the trailers as is */
     -+	/* Print the lines after the trailer block as is */
     ++	/* Print the lines after the trailer block as is. */
       	if (!opts->only_trailers)
      -		fwrite(sb.buf + trailer_block_end(info), 1, sb.len - trailer_block_end(info), outfile);
     --
      -	trailer_info_release(info);
     -+		fwrite(sb.buf + trailer_block_end(trailer_block),
     -+		       1, sb.len - trailer_block_end(trailer_block), outfile);
     ++		fwrite(sb.buf + trailer_block_end(trailer_block), 1,
     ++		       sb.len - trailer_block_end(trailer_block), outfile);
      +	trailer_block_release(trailer_block);
       
       	if (opts->in_place)
     @@ trailer.c
       
       	/*
       	 * Array of trailers found.
     -@@ trailer.c: void format_trailers(const struct process_trailer_options *opts,
     - 	}
     +@@ trailer.c: static void unfold_value(struct strbuf *val)
     + 	strbuf_release(&out);
       }
       
      -static struct trailer_info *trailer_info_new(void)
     @@ trailer.c: void free_trailers(struct list_head *trailers)
      +	free(trailer_block);
       }
       
     - void format_trailers_from_commit(const struct process_trailer_options *opts,
     + void format_trailers(const struct process_trailer_options *opts,
      @@ trailer.c: void format_trailers_from_commit(const struct process_trailer_options *opts,
       				 struct strbuf *out)
       {
     - 	LIST_HEAD(head);
     --	struct trailer_info *info = parse_trailers(opts, msg, &head);
     -+	struct trailer_block *trailer_block = parse_trailers(opts, msg, &head);
     + 	LIST_HEAD(trailers);
     +-	struct trailer_info *info = parse_trailers(opts, msg, &trailers);
     ++	struct trailer_block *trailer_block = parse_trailers(opts, msg, &trailers);
       
       	/* If we want the whole block untouched, we can take the fast path. */
       	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
     @@ trailer.c: void format_trailers_from_commit(const struct process_trailer_options
      +		strbuf_add(out, msg + trailer_block->start,
      +			   trailer_block->end - trailer_block->start);
       	} else
     - 		format_trailers(opts, &head, out);
     + 		format_trailers(opts, &trailers, out);
       
     - 	free_trailers(&head);
     + 	free_trailers(&trailers);
      -	trailer_info_release(info);
      +	trailer_block_release(trailer_block);
       }
       
       void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
     - {
     - 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
     --	struct trailer_info *internal = trailer_info_new();
     +@@ trailer.c: void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
       	strbuf_init(&iter->key, 0);
       	strbuf_init(&iter->val, 0);
       	opts.no_divider = 1;
     --	iter->internal.info = internal;
      -	iter->internal.info = trailer_info_get(&opts, msg);
      +	iter->internal.trailer_block = trailer_block_get(&opts, msg);
       	iter->internal.cur = 0;
       }
       
     -@@ trailer.c: int trailer_iterator_advance(struct trailer_iterator *iter)
     + int trailer_iterator_advance(struct trailer_iterator *iter)
       {
     - 	char *line;
     - 	int separator_pos;
      -	if (iter->internal.cur < iter->internal.info->trailer_nr) {
     --		line = iter->internal.info->trailers[iter->internal.cur++];
     +-		char *line = iter->internal.info->trailers[iter->internal.cur++];
      +	if (iter->internal.cur < iter->internal.trailer_block->trailer_nr) {
     -+		line = iter->internal.trailer_block->trailers[iter->internal.cur++];
     - 		separator_pos = find_separator(line, separators);
     - 		iter->is_trailer = (separator_pos > 0);
     ++		char *line = iter->internal.trailer_block->trailers[iter->internal.cur++];
     + 		int separator_pos = find_separator(line, separators);
       
     + 		iter->raw = line;
      @@ trailer.c: int trailer_iterator_advance(struct trailer_iterator *iter)
       
       void trailer_iterator_release(struct trailer_iterator *iter)
     @@ trailer.h: void parse_trailers_from_command_line_args(struct list_head *arg_head
       void process_trailers_lists(struct list_head *head,
       			    struct list_head *arg_head);
       
     --struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
     +-struct trailer_info *parse_trailers(const struct process_trailer_options *,
      -				    const char *str,
      -				    struct list_head *head);
     -+struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
     ++struct trailer_block *parse_trailers(const struct process_trailer_options *,
      +				     const char *str,
      +				     struct list_head *head);
       
     --size_t trailer_block_start(struct trailer_info *info);
     --size_t trailer_block_end(struct trailer_info *info);
     --int blank_line_before_trailer_block(struct trailer_info *info);
     -+size_t trailer_block_start(struct trailer_block *trailer_block);
     -+size_t trailer_block_end(struct trailer_block *trailer_block);
     -+int blank_line_before_trailer_block(struct trailer_block *trailer_block);
     +-size_t trailer_block_start(struct trailer_info *);
     +-size_t trailer_block_end(struct trailer_info *);
     +-int blank_line_before_trailer_block(struct trailer_info *);
     ++size_t trailer_block_start(struct trailer_block *);
     ++size_t trailer_block_end(struct trailer_block *);
     ++int blank_line_before_trailer_block(struct trailer_block *);
       
      -void trailer_info_release(struct trailer_info *info);
     -+void trailer_block_release(struct trailer_block *trailer_block);
     ++void trailer_block_release(struct trailer_block *);
       
       void trailer_config_init(void);
     - void format_trailers(const struct process_trailer_options *opts,
     + void format_trailers(const struct process_trailer_options *,
      @@ trailer.h: struct trailer_iterator {
       
       	/* private */
  7:  97e5d86ddf0 ! 22:  94bf182e3ff trailer: prepare to move parse_trailers_from_command_line_args() to builtin
     @@ Metadata
      Author: Linus Arver <linusa@google.com>
      
       ## Commit message ##
     -    trailer: prepare to move parse_trailers_from_command_line_args() to builtin
     +    trailer: prepare to delete "parse_trailers_from_command_line_args()"
      
     -    Expose more functions in the trailer.h API, in preparation for moving
     -    out
     +    Expose more functions in the trailer.h API, in preparation for deleting
      
              parse_trailers_from_command_line_args()
      
     -    to interpret-trailer.c, because the trailer API should not be concerned
     +    from the trailers implementation, because it should not be concerned
          with command line arguments (as they have nothing to do with trailers
     -    themselves). The interpret-trailers builtin is the only caller of this
     -    function.
     +    themselves). Indeed, the interpret-trailers builtin is the only caller
     +    of this function inside Git.
      
     -    Rename add_arg_item() to trailer_add_arg_item() because it will have to
     -    be exposed as an API function in the next patch. Rename
     -    new_trailers_clear() to free_new_trailers() because it will be promoted
     -    into an API function; the API already has free_trailers(), so using the
     -    "free_*" naming style will keep it consistent. Also rename "conf_info"
     -    to "trailer_conf" for readability, dropping the low-value "_info" suffix
     -    as we did earlier in this series for "trailer_info" to "trailer_block".
     +    Rename add_arg_item() to trailer_add_arg_item() to expose it as an API
     +    function. Rename new_trailers_clear() to free_new_trailers() because it
     +    will be promoted into an API function; the API already has
     +    free_trailers(), so using the "free_*" naming style will keep it
     +    consistent. Also rename "conf_info" to "trailer_conf" for readability,
     +    dropping the low-value "_info" suffix as we did earlier in this series
     +    for "trailer_info" to "trailer_block".
      
          Helped-by: Josh Steadmon <steadmon@google.com>
          Signed-off-by: Linus Arver <linusa@google.com>
     @@ trailer.c: static struct trailer_item *add_trailer_item(struct list_head *head,
      -static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
      -			 const struct conf_info *conf,
      -			 const struct new_trailer_item *new_trailer_item)
     -+static void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
     -+				 const struct trailer_conf *conf,
     -+				 const struct new_trailer_item *new_trailer_item)
     ++void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
     ++			  const struct trailer_conf *conf,
     ++			  const struct new_trailer_item *new_trailer_item)
       {
       	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
       	new_item->token = tok;
     @@ trailer.c: int trailer_iterator_advance(struct trailer_iterator *iter)
      -		parse_trailer(&iter->key, &iter->val, NULL,
      -			      line, separator_pos);
      +		parse_trailer(line, separator_pos, &iter->key, &iter->val, NULL);
     + 		/* Always unfold values during iteration. */
       		unfold_value(&iter->val);
       		return 1;
     - 	}
      
       ## trailer.h ##
      @@
     @@ trailer.h: struct new_trailer_item {
       
      +void duplicate_trailer_conf(struct trailer_conf *dst,
      +			    const struct trailer_conf *src);
     ++void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
     ++			  const struct trailer_conf *conf,
     ++			  const struct new_trailer_item *new_trailer_item);
      +
       struct process_trailer_options {
       	int in_place;
     @@ trailer.h: void parse_trailers_from_command_line_args(struct list_head *arg_head
      +
      +void parse_trailer(const char *line, ssize_t separator_pos,
      +		   struct strbuf *tok, struct strbuf *val,
     -+		   const struct trailer_conf **conf);
     ++		   const struct trailer_conf **);
      +
     - struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
     + struct trailer_block *parse_trailers(const struct process_trailer_options *,
       				     const char *str,
       				     struct list_head *head);
  8:  465dc51cdcb ! 23:  3bfe4809ecb trailer: move arg handling to interpret-trailers.c
     @@ Metadata
      Author: Linus Arver <linusa@google.com>
      
       ## Commit message ##
     -    trailer: move arg handling to interpret-trailers.c
     +    trailer: add new helper functions to API
      
     -    We don't move the "arg_item" struct to interpret-trailers.c, because it
     -    is now a struct that contains information about trailers that could be
     -    added into the input text's own trailers. This is a generic concept that
     -    extends beyond trailers defined as CLI arguments (it applies to trailers
     -    defined in configuration as well). We will rename "arg_item" to
     -    "trailer_template" in a follow-up patch to keep the diff here small.
     +    This is a preparatory refactor for deprecating "new_trailer_item" from
     +    the API (which will let us deprecate
     +    parse_trailers_from_command_line_args()).
     +
     +    Expose new helper functions from the API, because we'll be calling them
     +    from interpret-trailers.c soon when we move
     +    parse_trailers_from_command_line_args() there.
     +
     +    Move free_new_trailers() from the builtin to trailer.c because later on
     +    we will adjust it to free arg_item structs, which are private to
     +    trailer.c.
      
          Signed-off-by: Linus Arver <linusa@google.com>
      
     @@ builtin/interpret-trailers.c: static int option_parse_if_missing(const struct op
      -		free(item);
      -	}
      -}
     -+static char *cl_separators;
     - 
     +-
       static int option_parse_trailer(const struct option *opt,
       				   const char *arg, int unset)
       {
     - 	struct list_head *trailers = opt->value;
     --	struct new_trailer_item *item;
     -+	struct strbuf tok = STRBUF_INIT;
     -+	struct strbuf val = STRBUF_INIT;
     -+	const struct trailer_conf *conf;
     -+	struct trailer_conf *conf_current = new_trailer_conf();
     -+	ssize_t separator_pos;
     - 
     - 	if (unset) {
     - 		free_new_trailers(trailers);
     -@@ builtin/interpret-trailers.c: static int option_parse_trailer(const struct option *opt,
     - 	if (!arg)
     - 		return -1;
     - 
     --	item = xmalloc(sizeof(*item));
     --	item->text = arg;
     --	item->where = where;
     --	item->if_exists = if_exists;
     --	item->if_missing = if_missing;
     --	list_add_tail(&item->list, trailers);
     -+	separator_pos = find_separator(arg, cl_separators);
     -+	if (separator_pos) {
     -+		parse_trailer(arg, separator_pos, &tok, &val, &conf);
     -+		duplicate_trailer_conf(conf_current, conf);
     -+
     -+		/*
     -+		 * Override conf_current with settings specified via CLI flags.
     -+		 */
     -+		trailer_conf_set(where, if_exists, if_missing, conf_current);
     -+
     -+		trailer_add_arg_item(strbuf_detach(&tok, NULL),
     -+				     strbuf_detach(&val, NULL),
     -+				     conf_current,
     -+				     trailers);
     -+	} else {
     -+		struct strbuf sb = STRBUF_INIT;
     -+		strbuf_addstr(&sb, arg);
     -+		strbuf_trim(&sb);
     -+		error(_("empty trailer token in trailer '%.*s'"),
     -+			(int) sb.len, sb.buf);
     -+		strbuf_release(&sb);
     -+	}
     -+
     -+	free(conf_current);
     -+
     - 	return 0;
     - }
     - 
     -@@ builtin/interpret-trailers.c: static void read_input_file(struct strbuf *sb, const char *file)
     - }
     - 
     - static void interpret_trailers(const struct process_trailer_options *opts,
     --			       struct list_head *new_trailer_head,
     -+			       struct list_head *arg_trailers,
     - 			       const char *file)
     - {
     - 	LIST_HEAD(head);
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
     - 	struct trailer_block *trailer_block;
     - 	FILE *outfile = stdout;
     - 
     --	trailer_config_init();
     --
     - 	read_input_file(&sb, file);
     - 
     - 	if (opts->in_place)
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
     - 
     - 
     - 	if (!opts->only_input) {
     --		LIST_HEAD(config_head);
     --		LIST_HEAD(arg_head);
     --		parse_trailers_from_config(&config_head);
     --		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
     --		list_splice(&config_head, &arg_head);
     --		process_trailers_lists(&head, &arg_head);
     -+		process_trailers_lists(&head, arg_trailers);
     - 	}
     - 
     - 	/* Print trailer block. */
     -@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
     - int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
     - {
     - 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
     --	LIST_HEAD(trailers);
     -+	LIST_HEAD(configured_trailers);
     -+	LIST_HEAD(arg_trailers);
     - 
     - 	struct option options[] = {
     - 		OPT_BOOL(0, "in-place", &opts.in_place, N_("edit files in place")),
     -@@ builtin/interpret-trailers.c: int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
     - 		OPT_CALLBACK_F(0, "parse", &opts, NULL, N_("alias for --only-trailers --only-input --unfold"),
     - 			PARSE_OPT_NOARG | PARSE_OPT_NONEG, parse_opt_parse),
     - 		OPT_BOOL(0, "no-divider", &opts.no_divider, N_("do not treat \"---\" as the end of input")),
     --		OPT_CALLBACK(0, "trailer", &trailers, N_("trailer"),
     -+		OPT_CALLBACK(0, "trailer", &arg_trailers, N_("trailer"),
     - 				N_("trailer(s) to add"), option_parse_trailer),
     - 		OPT_END()
     - 	};
     - 
     - 	git_config(git_default_config, NULL);
     -+	trailer_config_init();
     -+
     -+	if (!opts.only_input) {
     -+		parse_trailers_from_config(&configured_trailers);
     -+	}
     -+
     -+	/*
     -+	* In command-line arguments, '=' is accepted (in addition to the
     -+	* separators that are defined).
     -+	*/
     -+	cl_separators = xstrfmt("=%s", trailer_default_separators());
     - 
     - 	argc = parse_options(argc, argv, prefix, options,
     - 			     git_interpret_trailers_usage, 0);
     - 
     --	if (opts.only_input && !list_empty(&trailers))
     -+	free(cl_separators);
     -+
     -+	if (opts.only_input && !list_empty(&arg_trailers))
     - 		usage_msg_opt(
     - 			_("--trailer with --only-input does not make sense"),
     - 			git_interpret_trailers_usage,
     - 			options);
     - 
     -+	list_splice(&configured_trailers, &arg_trailers);
     -+
     - 	if (argc) {
     - 		int i;
     - 		for (i = 0; i < argc; i++)
     --			interpret_trailers(&opts, &trailers, argv[i]);
     -+			interpret_trailers(&opts, &arg_trailers, argv[i]);
     - 	} else {
     - 		if (opts.in_place)
     - 			die(_("no input file given for in-place editing"));
     --		interpret_trailers(&opts, &trailers, NULL);
     -+		interpret_trailers(&opts, &arg_trailers, NULL);
     - 	}
     - 
     --	free_new_trailers(&trailers);
     -+	free_new_trailers(&arg_trailers);
     - 
     - 	return 0;
     - }
      
       ## trailer.c ##
      @@ trailer.c: static LIST_HEAD(conf_head);
     @@ trailer.c: int trailer_set_if_missing(enum trailer_if_missing *item, const char
       	return 0;
       }
       
     -+void trailer_conf_set(enum trailer_where where,
     -+		      enum trailer_if_exists if_exists,
     -+		      enum trailer_if_missing if_missing,
     -+		      struct trailer_conf *conf)
     ++void trailer_set_conf_where(enum trailer_where where,
     ++			    struct trailer_conf *conf)
     ++{
     ++	conf->where = where;
     ++}
     ++
     ++void trailer_set_conf_if_exists(enum trailer_if_exists if_exists,
     ++				struct trailer_conf *conf)
     ++{
     ++	conf->if_exists = if_exists;
     ++}
     ++
     ++void trailer_set_conf_if_missing(enum trailer_if_missing if_missing,
     ++				 struct trailer_conf *conf)
      +{
     -+	if (where != WHERE_DEFAULT)
     -+		conf->where = where;
     -+	if (if_exists != EXISTS_DEFAULT)
     -+		conf->if_exists = if_exists;
     -+	if (if_missing != MISSING_DEFAULT)
     -+		conf->if_missing = if_missing;
     ++	conf->if_missing = if_missing;
      +}
      +
      +struct trailer_conf *new_trailer_conf(void)
      +{
     -+	struct trailer_conf *new = xcalloc(1, sizeof(*new));
     -+	return new;
     ++	 return xcalloc(1, sizeof(struct trailer_conf));
      +}
      +
       void duplicate_trailer_conf(struct trailer_conf *dst,
       			    const struct trailer_conf *src)
       {
     -@@ trailer.c: ssize_t find_separator(const char *line, const char *separators)
     - /*
     -  * Obtain the token, value, and conf from the given trailer.
     -  *
     -+ * The conf needs special handling. We first read hardcoded defaults, and
     -+ * override them if we find a matching trailer configuration.
     -+ *
     -  * separator_pos must not be 0, since the token cannot be an empty string.
     -  *
     -  * If separator_pos is -1, interpret the whole trailer as a token.
     -@@ trailer.c: static struct trailer_item *add_trailer_item(struct list_head *head, char *tok,
     - 	return new_item;
     +@@ trailer.c: void duplicate_trailer_conf(struct trailer_conf *dst,
     + 	dst->cmd = xstrdup_or_null(src->cmd);
       }
       
     --static void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
     --				 const struct trailer_conf *conf,
     --				 const struct new_trailer_item *new_trailer_item)
     -+void trailer_add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
     -+			  struct list_head *arg_head)
     ++void free_trailer_conf(struct trailer_conf *conf)
     ++{
     ++	free(conf->name);
     ++	free(conf->key);
     ++	free(conf->command);
     ++	free(conf->cmd);
     ++	free(conf);
     ++}
     ++
     + static struct arg_item *get_conf_item(const char *name)
       {
     - 	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
     - 	new_item->token = tok;
     - 	new_item->value = val;
     - 	duplicate_trailer_conf(&new_item->conf, conf);
     --	if (new_trailer_item) {
     --		if (new_trailer_item->where != WHERE_DEFAULT)
     --			new_item->conf.where = new_trailer_item->where;
     --		if (new_trailer_item->if_exists != EXISTS_DEFAULT)
     --			new_item->conf.if_exists = new_trailer_item->if_exists;
     --		if (new_trailer_item->if_missing != MISSING_DEFAULT)
     --			new_item->conf.if_missing = new_trailer_item->if_missing;
     --	}
     - 	list_add_tail(&new_item->list, arg_head);
     - }
     - 
     -@@ trailer.c: void parse_trailers_from_config(struct list_head *config_head)
     - 	list_for_each(pos, &conf_head) {
     - 		item = list_entry(pos, struct arg_item, list);
     - 		if (item->conf.command)
     --			trailer_add_arg_item(config_head,
     --					     xstrdup(token_from_item(item, NULL)),
     -+			trailer_add_arg_item(xstrdup(token_from_item(item, NULL)),
     - 					     xstrdup(""),
     --					     &item->conf, NULL);
     -+					     &item->conf,
     -+					     config_head);
     - 	}
     - }
     - 
     -@@ trailer.c: void parse_trailers_from_command_line_args(struct list_head *arg_head,
     - 			strbuf_release(&sb);
     - 		} else {
     - 			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
     --			trailer_add_arg_item(arg_head,
     --					     strbuf_detach(&tok, NULL),
     -+			trailer_add_arg_item(strbuf_detach(&tok, NULL),
     - 					     strbuf_detach(&val, NULL),
     --					     conf, tr);
     -+					     conf,
     -+					     arg_head);
     - 		}
     - 	}
     - 
     + 	struct list_head *pos;
      @@ trailer.c: void free_trailers(struct list_head *trailers)
       	}
       }
       
      +void free_new_trailers(struct list_head *trailers)
      +{
     -+	struct list_head *pos, *p;
     ++	struct list_head *pos, *tmp;
     ++	struct new_trailer_item *item;
      +
     -+	list_for_each_safe(pos, p, trailers) {
     ++	list_for_each_safe(pos, tmp, trailers) {
     ++		item = list_entry(pos, struct new_trailer_item, list);
      +		list_del(pos);
     -+		free_arg_item(list_entry(pos, struct arg_item, list));
     ++		free(item);
      +	}
      +}
      +
     @@ trailer.h: struct new_trailer_item {
       	enum trailer_if_missing if_missing;
       };
       
     -+void trailer_conf_set(enum trailer_where where,
     -+		      enum trailer_if_exists if_exists,
     -+		      enum trailer_if_missing if_missing,
     -+		      struct trailer_conf *conf);
     ++void trailer_set_conf_where(enum trailer_where, struct trailer_conf *);
     ++void trailer_set_conf_if_exists(enum trailer_if_exists, struct trailer_conf *);
     ++void trailer_set_conf_if_missing(enum trailer_if_missing, struct trailer_conf *);
      +
      +struct trailer_conf *new_trailer_conf(void);
       void duplicate_trailer_conf(struct trailer_conf *dst,
       			    const struct trailer_conf *src);
     - 
      +const char *trailer_default_separators(void);
     -+
     -+void trailer_add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
     -+			  struct list_head *arg_head);
     -+
     - struct process_trailer_options {
     - 	int in_place;
     - 	int trim_empty;
     -@@ trailer.h: void format_trailers(const struct process_trailer_options *opts,
     + void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
     + 			  const struct trailer_conf *conf,
     + 			  const struct new_trailer_item *new_trailer_item);
     +@@ trailer.h: void format_trailers(const struct process_trailer_options *,
       		     struct list_head *trailers,
       		     struct strbuf *out);
       void free_trailers(struct list_head *);
      +void free_new_trailers(struct list_head *);
     ++void free_trailer_conf(struct trailer_conf *);
       
       /*
        * Convenience function to format the trailers from the commit msg "msg" into
  -:  ----------- > 24:  80e1958bb8d trailer_add_arg_item(): drop new_trailer_item usage
  9:  885ac87a544 ! 25:  a9080597a28 trailer: delete obsolete argument handling code from API
     @@ Metadata
      Author: Linus Arver <linusa@google.com>
      
       ## Commit message ##
     -    trailer: delete obsolete argument handling code from API
     +    trailer: deprecate "new_trailer_item" struct from API
      
     -    This commit was not squashed with its parent in order to keep the diff
     -    separate (to make the additive changes in the parent easier to read).
     +    Previously, the "new_trailer_item" struct served only one purpose --- to
     +    capture the unparsed raw string <RAW> in "--trailer <RAW>", as well as
     +    the current state of the "where", "if_exists", and "if_missing" global
     +    variables at the time that the "--trailer <RAW>" CLI argument was
     +    encountered.
      
     -    Note that we remove the "new_trailer_item" struct, because it has some
     -    overlap with "arg_item" struct that also deals with trailers coming from
     -    the command line. The latter will be renamed to "trailer_template" in
     -    the next patch.
     +    In addition, the previous CLI argument handling behavior was to capture
     +    the <RAW> string in all "--trailer <RAW>" arguments and to collect
     +    them (via option_parse_trailer()) into the "new_trailer_head" list. We
     +    would then iterate over this list again in
     +    parse_trailers_from_command_line_args() and convert these
     +    "new_trailer_item" objects into "arg_item" objects.
     +
     +    Skip this intermediate storage of "new_trailer_item" objects in favor of
     +    just storing "arg_item" objects. Remove the looping behavior of
     +    parse_trailers_from_command_line_args() so that it parses a single
     +    "--trailer ..." argument at a time. Rename it to
     +    parse_trailer_from_command_line_arg() to reflect this new behavior of
     +    only looking at one string (not multiple strings) at a time. Make
     +    option_parse_trailer() call parse_trailer_from_command_line_arg() so
     +    that the CLI arguments it sees are parsed immediately without the need
     +    for intermediate storage.
     +
     +    Delete "new_trailer_item", because we don't need it any more.
     +
     +    In the next patch we will retire parse_trailer_from_command_line_arg()
     +    as well, combining it with option_parse_trailer().
      
          Signed-off-by: Linus Arver <linusa@google.com>
      
     + ## builtin/interpret-trailers.c ##
     +@@ builtin/interpret-trailers.c: static int option_parse_trailer(const struct option *opt,
     + 				   const char *arg, int unset)
     + {
     + 	struct list_head *trailers = opt->value;
     +-	struct new_trailer_item *item;
     + 
     + 	if (unset) {
     + 		free_new_trailers(trailers);
     +@@ builtin/interpret-trailers.c: static int option_parse_trailer(const struct option *opt,
     + 	if (!arg)
     + 		return -1;
     + 
     +-	item = xmalloc(sizeof(*item));
     +-	item->text = arg;
     +-	item->where = where;
     +-	item->if_exists = if_exists;
     +-	item->if_missing = if_missing;
     +-	list_add_tail(&item->list, trailers);
     ++	parse_trailer_from_command_line_arg(arg, where, if_exists, if_missing, trailers);
     ++
     + 	return 0;
     + }
     + 
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
     + 	struct trailer_block *trailer_block;
     + 	FILE *outfile = stdout;
     + 
     +-	trailer_config_init();
     +-
     + 	read_input_file(&sb, file);
     + 
     + 	if (opts->in_place)
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
     + 	if (!opts->only_trailers && !blank_line_before_trailer_block(trailer_block))
     + 		fprintf(outfile, "\n");
     + 
     +-
     +-	if (!opts->only_input) {
     +-		LIST_HEAD(config_head);
     +-		LIST_HEAD(arg_head);
     +-		parse_trailers_from_config(&config_head);
     +-		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
     +-		list_splice(&config_head, &arg_head);
     +-		process_trailers_lists(&head, &arg_head);
     +-	}
     ++	if (!opts->only_input)
     ++		process_trailers_lists(&head, new_trailer_head);
     + 
     + 	/* Print trailer block. */
     + 	format_trailers(opts, &head, &tb);
     +@@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
     + int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
     + {
     + 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
     ++	LIST_HEAD(configured_trailers);
     + 	LIST_HEAD(trailers);
     + 
     + 	struct option options[] = {
     +@@ builtin/interpret-trailers.c: int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
     + 	};
     + 
     + 	git_config(git_default_config, NULL);
     ++	trailer_config_init();
     ++
     ++	if (!opts.only_input)
     ++		parse_trailers_from_config(&configured_trailers);
     + 
     + 	argc = parse_options(argc, argv, prefix, options,
     + 			     git_interpret_trailers_usage, 0);
     +@@ builtin/interpret-trailers.c: int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
     + 			git_interpret_trailers_usage,
     + 			options);
     + 
     ++	list_splice(&configured_trailers, &trailers);
     ++
     + 	if (argc) {
     + 		int i;
     + 		for (i = 0; i < argc; i++)
     +
       ## trailer.c ##
      @@ trailer.c: void parse_trailers_from_config(struct list_head *config_head)
       	}
     @@ trailer.c: void parse_trailers_from_config(struct list_head *config_head)
       
      -void parse_trailers_from_command_line_args(struct list_head *arg_head,
      -					   struct list_head *new_trailer_head)
     --{
     --	struct strbuf tok = STRBUF_INIT;
     --	struct strbuf val = STRBUF_INIT;
     --	const struct trailer_conf *conf;
     ++void parse_trailer_from_command_line_arg(const char *line,
     ++					 enum trailer_where where,
     ++					 enum trailer_if_exists if_exists,
     ++					 enum trailer_if_missing if_missing,
     ++					 struct list_head *arg_head)
     + {
     + 	struct strbuf tok = STRBUF_INIT;
     + 	struct strbuf val = STRBUF_INIT;
     + 	const struct trailer_conf *conf;
      -	struct list_head *pos;
     --
     --	/*
     --	 * In command-line arguments, '=' is accepted (in addition to the
     --	 * separators that are defined).
     --	 */
     + 
     + 	/*
     + 	 * In command-line arguments, '=' is accepted (in addition to the
     + 	 * separators that are defined).
     + 	 */
      -	char *cl_separators = xstrfmt("=%s", separators);
     --
     ++	char *cl_separators = xstrfmt("=%s", trailer_default_separators());
     + 
      -	/* Add an arg item for each trailer on the command line */
      -	list_for_each(pos, new_trailer_head) {
      -		struct new_trailer_item *tr =
      -			list_entry(pos, struct new_trailer_item, list);
      -		ssize_t separator_pos = find_separator(tr->text, cl_separators);
     --
     ++	/* Add an arg item for a trailer from the command line */
     ++	ssize_t separator_pos = find_separator(line, cl_separators);
     ++	free(cl_separators);
     + 
      -		if (separator_pos == 0) {
      -			struct strbuf sb = STRBUF_INIT;
      -			strbuf_addstr(&sb, tr->text);
     @@ trailer.c: void parse_trailers_from_config(struct list_head *config_head)
      -			      (int) sb.len, sb.buf);
      -			strbuf_release(&sb);
      -		} else {
     +-			struct trailer_conf *conf_current = new_trailer_conf();
      -			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
     --			trailer_add_arg_item(strbuf_detach(&tok, NULL),
     +-			duplicate_trailer_conf(conf_current, conf);
     ++	if (separator_pos == 0) {
     ++		struct strbuf sb = STRBUF_INIT;
     ++		strbuf_addstr(&sb, line);
     ++		strbuf_trim(&sb);
     ++		error(_("empty trailer token in trailer '%.*s'"),
     ++		      (int) sb.len, sb.buf);
     ++		strbuf_release(&sb);
     ++	} else {
     ++		struct trailer_conf *conf_current = new_trailer_conf();
     ++		parse_trailer(line, separator_pos, &tok, &val, &conf);
     ++		duplicate_trailer_conf(conf_current, conf);
     + 
     +-			/*
     +-			 * Override conf_current with settings specified via CLI flags.
     +-			 */
     +-			if (tr->where != WHERE_DEFAULT)
     +-				trailer_set_conf_where(tr->where, conf_current);
     +-			if (tr->if_exists != EXISTS_DEFAULT)
     +-				trailer_set_conf_if_exists(tr->if_exists, conf_current);
     +-			if (tr->if_missing != MISSING_DEFAULT)
     +-				trailer_set_conf_if_missing(tr->if_missing, conf_current);
     +-
     +-			trailer_add_arg_item(arg_head,
     +-					     strbuf_detach(&tok, NULL),
      -					     strbuf_detach(&val, NULL),
     --					     conf,
     --					     arg_head);
     +-					     conf_current);
     +-			free_trailer_conf(conf_current);
      -		}
     --	}
     ++		/*
     ++		 * Override conf_current with settings specified via CLI flags.
     ++		 */
     ++		if (where != WHERE_DEFAULT)
     ++			trailer_set_conf_where(where, conf_current);
     ++		if (if_exists != EXISTS_DEFAULT)
     ++			trailer_set_conf_if_exists(if_exists, conf_current);
     ++		if (if_missing != MISSING_DEFAULT)
     ++			trailer_set_conf_if_missing(if_missing, conf_current);
     ++
     ++		trailer_add_arg_item(arg_head,
     ++				     strbuf_detach(&tok, NULL),
     ++				     strbuf_detach(&val, NULL),
     ++				     conf_current);
     ++		free_trailer_conf(conf_current);
     + 	}
      -
      -	free(cl_separators);
     --}
     --
     + }
     + 
       static const char *next_line(const char *str)
     +@@ trailer.c: void free_trailers(struct list_head *trailers)
     + 
     + void free_new_trailers(struct list_head *trailers)
       {
     - 	const char *nl = strchrnul(str, '\n');
     +-	struct list_head *pos, *tmp;
     +-	struct new_trailer_item *item;
     ++	struct list_head *pos, *p;
     + 
     +-	list_for_each_safe(pos, tmp, trailers) {
     +-		item = list_entry(pos, struct new_trailer_item, list);
     ++	list_for_each_safe(pos, p, trailers) {
     + 		list_del(pos);
     +-		free(item);
     ++		free_arg_item(list_entry(pos, struct arg_item, list));
     + 	}
     + }
     + 
      
       ## trailer.h ##
      @@ trailer.h: int trailer_set_where(enum trailer_where *item, const char *value);
     @@ trailer.h: int trailer_set_where(enum trailer_where *item, const char *value);
      -	enum trailer_if_missing if_missing;
      -};
      -
     - void trailer_conf_set(enum trailer_where where,
     - 		      enum trailer_if_exists if_exists,
     - 		      enum trailer_if_missing if_missing,
     + void trailer_set_conf_where(enum trailer_where, struct trailer_conf *);
     + void trailer_set_conf_if_exists(enum trailer_if_exists, struct trailer_conf *);
     + void trailer_set_conf_if_missing(enum trailer_if_missing, struct trailer_conf *);
      @@ trailer.h: struct process_trailer_options {
       
       void parse_trailers_from_config(struct list_head *config_head);
       
      -void parse_trailers_from_command_line_args(struct list_head *arg_head,
      -					   struct list_head *new_trailer_head);
     --
     ++void parse_trailer_from_command_line_arg(const char *line,
     ++					 enum trailer_where where,
     ++					 enum trailer_if_exists if_exists,
     ++					 enum trailer_if_missing if_missing,
     ++					 struct list_head *arg_head);
     + 
       void process_trailers_lists(struct list_head *head,
       			    struct list_head *arg_head);
     - 
  -:  ----------- > 26:  9720526dd8a trailer: unify "--trailer ..." arg handling
  -:  ----------- > 27:  26df2514acb trailer_set_*(): put out parameter at the end
 10:  bcd3fc9660e ! 28:  14927038d85 trailer: introduce "template" term for readability
     @@ Commit message
            - [*] check_if_different()         (reorder parameters only)
            - [*] find_same_and_apply_arg()    (reorder parameters only)
      
     -    Reorder parameters where appropriate; these functions have been marked
     -    with an asterisk ([*]).
     +    Reorder parameters to prefer input parameters toward the beginning and
     +    out parameters at the end; these functions have been marked with an
     +    asterisk ([*]).
      
          This removes the "arg" terminology (standing for "CLI arguments") from
          the trailer implementation, which makes sense because trailers
     @@ Commit message
          block) and trailer templates that are defined as CLI args or
          configurations. Some functions implied a single action when they could
          do two different things, so introduce words like "maybe" and "or" to
     -    unmask their behavior.
     +    make their behavior more explicit.
      
          In summary this patch renames and reorders parameters for readability,
          without any behavioral change. We don't rename
     -    find_same_and_apply_arg(), because it will be refactored soon. As an
     -    added benefit, we no longer use the term "arg" to mean "CLI arguments"
     -    in the trailer implementation, because trailers themselves should not be
     -    concerned about CLI arguments (this is the domain of the
     -    interpret-trailers builtin).
     +    find_same_and_apply_arg(), because it will be refactored soon.
      
          For parse_trailers_from_config() (renamed to
          parse_trailer_templates_from_config()), add a NEEDSWORK discussion about
          how the deprecated trailer.*.command configuration option is oddly more
          featureful than trailer.*.cmd (if we were to remove support for
          trailer.*.command, users would not be able to replicate the behavior
     -    with trailer.*.cmd and lose out on functionality).
     +    with trailer.*.cmd and would lose out on functionality).
      
          Signed-off-by: Linus Arver <linusa@google.com>
      
       ## builtin/interpret-trailers.c ##
      @@ builtin/interpret-trailers.c: static int option_parse_if_missing(const struct option *opt,
     - 
     - static char *cl_separators;
     + 	return trailer_set_if_missing(arg, opt->value);
     + }
       
      -static int option_parse_trailer(const struct option *opt,
      -				   const char *arg, int unset)
     @@ builtin/interpret-trailers.c: static int option_parse_if_missing(const struct op
       	struct strbuf val = STRBUF_INIT;
       	const struct trailer_conf *conf;
      @@ builtin/interpret-trailers.c: static int option_parse_trailer(const struct option *opt,
     - 	ssize_t separator_pos;
     + 	static char *cl_separators;
       
       	if (unset) {
      -		free_new_trailers(trailers);
     @@ builtin/interpret-trailers.c: static int option_parse_trailer(const struct optio
       	}
       
      @@ builtin/interpret-trailers.c: static int option_parse_trailer(const struct option *opt,
     - 		 */
     - 		trailer_conf_set(where, if_exists, if_missing, conf_current);
     + 		if (if_missing != MISSING_DEFAULT)
     + 			trailer_set_conf_if_missing(if_missing, conf_current);
       
     --		trailer_add_arg_item(strbuf_detach(&tok, NULL),
     +-		trailer_add_arg_item(trailers,
     +-				     strbuf_detach(&tok, NULL),
      +		add_trailer_template(strbuf_detach(&tok, NULL),
       				     strbuf_detach(&val, NULL),
     - 				     conf_current,
     --				     trailers);
     +-				     conf_current);
     ++				     conf_current,
      +				     templates);
     - 	} else {
     - 		struct strbuf sb = STRBUF_INIT;
     - 		strbuf_addstr(&sb, arg);
     + 		free_trailer_conf(conf_current);
     + 	}
     + 
      @@ builtin/interpret-trailers.c: static void read_input_file(struct strbuf *sb, const char *file)
       }
       
       static void interpret_trailers(const struct process_trailer_options *opts,
     --			       struct list_head *arg_trailers,
     +-			       struct list_head *new_trailer_head,
      +			       struct list_head *templates,
       			       const char *file)
       {
     @@ builtin/interpret-trailers.c: static void interpret_trailers(const struct proces
       	/* Print the lines before the trailer block */
       	if (!opts->only_trailers)
      @@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
     + 		fprintf(outfile, "\n");
       
     - 
     - 	if (!opts->only_input) {
     --		process_trailers_lists(&head, arg_trailers);
     + 	if (!opts->only_input)
     +-		process_trailers_lists(&head, new_trailer_head);
      +		apply_trailer_templates(templates, &trailers_from_sb);
     - 	}
       
       	/* Print trailer block. */
      -	format_trailers(opts, &head, &tb);
     +-	free_trailers(&head);
      +	format_trailers(opts, &trailers_from_sb, &tb);
     ++	free_trailers(&trailers_from_sb);
       	fwrite(tb.buf, 1, tb.len, outfile);
       	strbuf_release(&tb);
       
     --	free_trailers(&head);
     -+	free_trailers(&trailers_from_sb);
     - 
     - 	/* Print the lines after the trailer block as is */
     - 	if (!opts->only_trailers)
      @@ builtin/interpret-trailers.c: static void interpret_trailers(const struct process_trailer_options *opts,
       int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
       {
       	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
      -	LIST_HEAD(configured_trailers);
     --	LIST_HEAD(arg_trailers);
     +-	LIST_HEAD(trailers);
      +	LIST_HEAD(configured_templates);
      +	LIST_HEAD(templates);
       
     @@ builtin/interpret-trailers.c: int cmd_interpret_trailers(int argc, const char **
       		OPT_CALLBACK_F(0, "parse", &opts, NULL, N_("alias for --only-trailers --only-input --unfold"),
       			PARSE_OPT_NOARG | PARSE_OPT_NONEG, parse_opt_parse),
       		OPT_BOOL(0, "no-divider", &opts.no_divider, N_("do not treat \"---\" as the end of input")),
     --		OPT_CALLBACK(0, "trailer", &arg_trailers, N_("trailer"),
     +-		OPT_CALLBACK(0, "trailer", &trailers, N_("trailer"),
      -				N_("trailer(s) to add"), option_parse_trailer),
      +		OPT_CALLBACK(0, "trailer", &templates, N_("trailer"),
      +				N_("trailer(s) to add"), option_parse_trailer_template),
     @@ builtin/interpret-trailers.c: int cmd_interpret_trailers(int argc, const char **
      @@ builtin/interpret-trailers.c: int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
       	trailer_config_init();
       
     - 	if (!opts.only_input) {
     + 	if (!opts.only_input)
      -		parse_trailers_from_config(&configured_trailers);
      +		parse_trailer_templates_from_config(&configured_templates);
     - 	}
     - 
     - 	/*
     -@@ builtin/interpret-trailers.c: int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
       
     - 	free(cl_separators);
     + 	argc = parse_options(argc, argv, prefix, options,
     + 			     git_interpret_trailers_usage, 0);
       
     --	if (opts.only_input && !list_empty(&arg_trailers))
     +-	if (opts.only_input && !list_empty(&trailers))
      +	if (opts.only_input && !list_empty(&templates))
       		usage_msg_opt(
       			_("--trailer with --only-input does not make sense"),
       			git_interpret_trailers_usage,
       			options);
       
     --	list_splice(&configured_trailers, &arg_trailers);
     +-	list_splice(&configured_trailers, &trailers);
      +	list_splice(&configured_templates, &templates);
       
       	if (argc) {
       		int i;
       		for (i = 0; i < argc; i++)
     --			interpret_trailers(&opts, &arg_trailers, argv[i]);
     +-			interpret_trailers(&opts, &trailers, argv[i]);
      +			interpret_trailers(&opts, &templates, argv[i]);
       	} else {
       		if (opts.in_place)
       			die(_("no input file given for in-place editing"));
     --		interpret_trailers(&opts, &arg_trailers, NULL);
     +-		interpret_trailers(&opts, &trailers, NULL);
      +		interpret_trailers(&opts, &templates, NULL);
       	}
       
     --	free_new_trailers(&arg_trailers);
     +-	free_new_trailers(&trailers);
      +	free_trailer_templates(&templates);
       
       	return 0;
     @@ trailer.c: static char *apply_command(struct trailer_conf *conf, const char *arg
       	}
       }
       
     -@@ trailer.c: void duplicate_trailer_conf(struct trailer_conf *dst,
     - 	dst->cmd = xstrdup_or_null(src->cmd);
     +@@ trailer.c: void free_trailer_conf(struct trailer_conf *conf)
     + 	free(conf);
       }
       
      -static struct arg_item *get_conf_item(const char *name)
     @@ trailer.c: static struct trailer_item *add_trailer_item(struct list_head *head,
       	return new_item;
       }
       
     --void trailer_add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
     --			  struct list_head *arg_head)
     +-void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
     +-			  const struct trailer_conf *conf)
      +void add_trailer_template(char *tok, char *val, const struct trailer_conf *conf,
      +			  struct list_head *templates)
       {
     @@ trailer.c: static struct trailer_item *add_trailer_item(struct list_head *head,
      -	list_for_each(pos, &conf_head) {
      -		item = list_entry(pos, struct arg_item, list);
      -		if (item->conf.command)
     --			trailer_add_arg_item(xstrdup(token_from_item(item, NULL)),
     +-			trailer_add_arg_item(config_head,
     +-					     xstrdup(token_from_item(item, NULL)),
      -					     xstrdup(""),
     --					     &item->conf,
     --					     config_head);
     +-					     &item->conf);
      +	/*
      +	 * Get configured templates with a ".command" option.
      +	 *
     @@ trailer.c: static struct trailer_item *add_trailer_item(struct list_head *head,
       	}
       }
       
     -@@ trailer.c: static size_t find_trailer_block_start(const char *buf, size_t len)
     - 	 * Get the start of the trailers by looking starting from the end for a
     - 	 * blank line before a set of non-blank lines that (i) are all
     - 	 * trailers, or (ii) contains at least one Git-generated trailer and
     --	 * consists of at least 25% trailers.
     -+	 * consists of at least 25% configured trailers.
     - 	 */
     - 	for (l = last_line(buf, len);
     - 	     l >= end_of_title;
      @@ trailer.c: static size_t find_trailer_block_start(const char *buf, size_t len)
       			possible_continuation_lines = 0;
       			if (recognized_prefix)
     @@ trailer.c: void free_trailers(struct list_head *trailers)
       
      
       ## trailer.h ##
     -@@ trailer.h: void duplicate_trailer_conf(struct trailer_conf *dst,
     - 
     +@@ trailer.h: struct trailer_conf *new_trailer_conf(void);
     + void duplicate_trailer_conf(struct trailer_conf *dst,
     + 			    const struct trailer_conf *src);
       const char *trailer_default_separators(void);
     - 
     --void trailer_add_arg_item(char *tok, char *val, const struct trailer_conf *conf,
     --			  struct list_head *arg_head);
     -+void add_trailer_template(char *tok, char *val, const struct trailer_conf *conf,
     +-void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
     +-			  const struct trailer_conf *conf);
     ++void add_trailer_template(char *tok, char *val, const struct trailer_conf *,
      +			  struct list_head *templates);
       
       struct process_trailer_options {
     @@ trailer.h: struct process_trailer_options {
       
      -void process_trailers_lists(struct list_head *head,
      -			    struct list_head *arg_head);
     -+void apply_trailer_templates(struct list_head *templates, struct list_head *trailers_head);
     ++void apply_trailer_templates(struct list_head *templates,
     ++			     struct list_head *trailers_head);
       
       ssize_t find_separator(const char *line, const char *separators);
       
     -@@ trailer.h: void format_trailers(const struct process_trailer_options *opts,
     +@@ trailer.h: void format_trailers(const struct process_trailer_options *,
       		     struct list_head *trailers,
       		     struct strbuf *out);
       void free_trailers(struct list_head *);
      -void free_new_trailers(struct list_head *);
     + void free_trailer_conf(struct trailer_conf *);
      +void free_trailer_templates(struct list_head *);
       
       /*

-- 
gitgitgadget

[PATCH v4 01/28] trailer: free trailer_info _after_ all related usage

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

In de7c27a186 (trailer: use offsets for trailer_start/trailer_end,
2023-10-20), we started using trailer block offsets in trailer_info. In
particular, we dropped the use of a separate stack variable "size_t
trailer_end", in favor of accessing the new "trailer_block_end" member
of trailer_info (as "info.trailer_block_end").

At that time, we forgot to also move the

   trailer_info_release(&info);

line to be _after_ this new use of the trailer_info struct. Move it now.

Note that even without this patch, we didn't have leaks or any other
problems because trailer_info_release() only frees memory allocated on
the heap. The "trailer_block_end" member was allocated on the stack back
then (as it is now) so it was still safe to use for all this time.

Reported-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/trailer.c b/trailer.c
index 3a0710a4583..e1d83390b66 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1111,13 +1111,12 @@ void process_trailers(const char *file,
 	}
 
 	print_all(outfile, &head, opts);
-
 	free_all(&head);
-	trailer_info_release(&info);
 
 	/* Print the lines after the trailers as is */
 	if (!opts->only_trailers)
 		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
+	trailer_info_release(&info);
 
 	if (opts->in_place)
 		if (rename_tempfile(&trailers_tempfile, file))
-- 
gitgitgadget

[PATCH v4 02/28] shortlog: add test for de-duplicating folded trailers

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

The shortlog builtin was taught to use the trailer iterator interface in
47beb37bc6 (shortlog: match commit trailers with --group, 2020-09-27).
The iterator always unfolds values and this has always been the case
since the time the iterator was first introduced in f0939a0eb1 (trailer:
add interface for iterating over commit trailers, 2020-09-27). Add a
comment line to remind readers of this behavior.

The fact that the iterator always unfolds values is important
(at least for shortlog) because unfolding allows it to recognize both
folded and unfolded versions of the same trailer for de-duplication.

Capture the existing behavior in a new test case to guard against
regressions in this area. This test case is based off of the existing
"shortlog de-duplicates trailers in a single commit" just above it. Now
if we were to remove the call to

    unfold_value(&iter->val);

inside the iterator, this new test case will break.

Signed-off-by: Linus Arver <linusa@google.com>
---
 t/t4201-shortlog.sh | 32 ++++++++++++++++++++++++++++++++
 trailer.c           |  1 +
 2 files changed, 33 insertions(+)

diff --git a/t/t4201-shortlog.sh b/t/t4201-shortlog.sh
index d7382709fc1..f698d0c9ad2 100755
--- a/t/t4201-shortlog.sh
+++ b/t/t4201-shortlog.sh
@@ -312,6 +312,38 @@ test_expect_success 'shortlog de-duplicates trailers in a single commit' '
 	test_cmp expect actual
 '
 
+# Trailers that have unfolded (single line) and folded (multiline) values which
+# are otherwise identical are treated as the same trailer for de-duplication.
+test_expect_success 'shortlog de-duplicates trailers in a single commit (folded/unfolded values)' '
+	git commit --allow-empty -F - <<-\EOF &&
+	subject one
+
+	this message has two distinct values, plus a repeat (folded)
+
+	Repeated-trailer: Foo foo foo
+	Repeated-trailer: Bar
+	Repeated-trailer: Foo
+	  foo foo
+	EOF
+
+	git commit --allow-empty -F - <<-\EOF &&
+	subject two
+
+	similar to the previous, but without the second distinct value
+
+	Repeated-trailer: Foo foo foo
+	Repeated-trailer: Foo
+	  foo foo
+	EOF
+
+	cat >expect <<-\EOF &&
+	     2	Foo foo foo
+	     1	Bar
+	EOF
+	git shortlog -ns --group=trailer:repeated-trailer -2 HEAD >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'shortlog can match multiple groups' '
 	git commit --allow-empty -F - <<-\EOF &&
 	subject one
diff --git a/trailer.c b/trailer.c
index e1d83390b66..f74915bd8cd 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1270,6 +1270,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 		strbuf_reset(&iter->val);
 		parse_trailer(&iter->key, &iter->val, NULL,
 			      trailer, separator_pos);
+		/* Always unfold values during iteration. */
 		unfold_value(&iter->val);
 		return 1;
 	}
-- 
gitgitgadget

[PATCH v4 03/28] trailer: prepare to expose functions as part of API

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

In the next patch, we will move "process_trailers" from trailer.c to
builtin/interpret-trailers.c. That move will necessitate the growth of
the trailer.h API, forcing us to expose some additional functions in
trailer.h.

Rename relevant functions so that they include the term "trailer" in
their name, so that clients of the API will be able to easily identify
them by their "trailer" moniker, just like all the other functions
already exposed by trailer.h.

Take the opportunity to start putting trailer processing options (opts)
as the first parameter. This will be the pattern going forward in this
series.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  4 ++--
 trailer.c                    | 26 +++++++++++++-------------
 trailer.h                    |  6 +++---
 3 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 033bd1556cf..85a3413baf5 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -132,11 +132,11 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 	if (argc) {
 		int i;
 		for (i = 0; i < argc; i++)
-			process_trailers(argv[i], &opts, &trailers);
+			interpret_trailers(&opts, &trailers, argv[i]);
 	} else {
 		if (opts.in_place)
 			die(_("no input file given for in-place editing"));
-		process_trailers(NULL, &opts, &trailers);
+		interpret_trailers(&opts, &trailers, NULL);
 	}
 
 	new_trailers_clear(&trailers);
diff --git a/trailer.c b/trailer.c
index f74915bd8cd..916175707d8 100644
--- a/trailer.c
+++ b/trailer.c
@@ -163,12 +163,12 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-static void print_all(FILE *outfile, struct list_head *head,
-		      const struct process_trailer_options *opts)
+static void format_trailers(const struct process_trailer_options *opts,
+			    struct list_head *trailers, FILE *outfile)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
-	list_for_each(pos, head) {
+	list_for_each(pos, trailers) {
 		item = list_entry(pos, struct trailer_item, list);
 		if ((!opts->trim_empty || strlen(item->value) > 0) &&
 		    (!opts->only_trailers || item->token))
@@ -589,7 +589,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	return 0;
 }
 
-static void ensure_configured(void)
+static void trailer_config_init(void)
 {
 	if (configured)
 		return;
@@ -1035,10 +1035,10 @@ static void parse_trailers(struct trailer_info *info,
 	}
 }
 
-static void free_all(struct list_head *head)
+static void free_trailers(struct list_head *trailers)
 {
 	struct list_head *pos, *p;
-	list_for_each_safe(pos, p, head) {
+	list_for_each_safe(pos, p, trailers) {
 		list_del(pos);
 		free_trailer_item(list_entry(pos, struct trailer_item, list));
 	}
@@ -1075,16 +1075,16 @@ static FILE *create_in_place_tempfile(const char *file)
 	return outfile;
 }
 
-void process_trailers(const char *file,
-		      const struct process_trailer_options *opts,
-		      struct list_head *new_trailer_head)
+void interpret_trailers(const struct process_trailer_options *opts,
+			struct list_head *new_trailer_head,
+			const char *file)
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
 	struct trailer_info info;
 	FILE *outfile = stdout;
 
-	ensure_configured();
+	trailer_config_init();
 
 	read_input_file(&sb, file);
 
@@ -1110,8 +1110,8 @@ void process_trailers(const char *file,
 		process_trailers_lists(&head, &arg_head);
 	}
 
-	print_all(outfile, &head, opts);
-	free_all(&head);
+	format_trailers(opts, &head, outfile);
+	free_trailers(&head);
 
 	/* Print the lines after the trailers as is */
 	if (!opts->only_trailers)
@@ -1134,7 +1134,7 @@ void trailer_info_get(struct trailer_info *info, const char *str,
 	size_t nr = 0, alloc = 0;
 	char **last = NULL;
 
-	ensure_configured();
+	trailer_config_init();
 
 	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
 	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
diff --git a/trailer.h b/trailer.h
index 1644cd05f60..37033e631a1 100644
--- a/trailer.h
+++ b/trailer.h
@@ -81,9 +81,9 @@ struct process_trailer_options {
 
 #define PROCESS_TRAILER_OPTIONS_INIT {0}
 
-void process_trailers(const char *file,
-		      const struct process_trailer_options *opts,
-		      struct list_head *new_trailer_head);
+void interpret_trailers(const struct process_trailer_options *opts,
+			struct list_head *new_trailer_head,
+			const char *file);
 
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts);
-- 
gitgitgadget

[PATCH v4 04/28] trailer: move interpret_trailers() to interpret-trailers.c

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

The interpret-trailers.c builtin is the only place we need to call
interpret_trailers(), so move its definition there (together with a few
helper functions called only by it) and remove its external declaration
from <trailer.h>.

Several helper functions that are called by interpret_trailers() remain
in trailer.c because other callers in the same file still call them.
Declare them in <trailer.h> so that interpret_trailers() (now in
builtin/interpret-trailers.c) can continue calling them as a trailer API
user.

This enriches <trailer.h> with a more granular API, which can then be
unit-tested in the future (because interpret_trailers() by itself does
too many things to be able to be easily unit-tested).

Take this opportunity to demote some file-handling functions out of the
trailer API implementation, as these have nothing to do with trailers.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  93 +++++++++++++++++++++++++++
 trailer.c                    | 119 ++++-------------------------------
 trailer.h                    |  20 +++++-
 3 files changed, 123 insertions(+), 109 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 85a3413baf5..d1cf0aa33a2 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -9,6 +9,7 @@
 #include "gettext.h"
 #include "parse-options.h"
 #include "string-list.h"
+#include "tempfile.h"
 #include "trailer.h"
 #include "config.h"
 
@@ -91,6 +92,98 @@ static int parse_opt_parse(const struct option *opt, const char *arg,
 	return 0;
 }
 
+static struct tempfile *trailers_tempfile;
+
+static FILE *create_in_place_tempfile(const char *file)
+{
+	struct stat st;
+	struct strbuf filename_template = STRBUF_INIT;
+	const char *tail;
+	FILE *outfile;
+
+	if (stat(file, &st))
+		die_errno(_("could not stat %s"), file);
+	if (!S_ISREG(st.st_mode))
+		die(_("file %s is not a regular file"), file);
+	if (!(st.st_mode & S_IWUSR))
+		die(_("file %s is not writable by user"), file);
+
+	/* Create temporary file in the same directory as the original */
+	tail = strrchr(file, '/');
+	if (tail)
+		strbuf_add(&filename_template, file, tail - file + 1);
+	strbuf_addstr(&filename_template, "git-interpret-trailers-XXXXXX");
+
+	trailers_tempfile = xmks_tempfile_m(filename_template.buf, st.st_mode);
+	strbuf_release(&filename_template);
+	outfile = fdopen_tempfile(trailers_tempfile, "w");
+	if (!outfile)
+		die_errno(_("could not open temporary file"));
+
+	return outfile;
+}
+
+static void read_input_file(struct strbuf *sb, const char *file)
+{
+	if (file) {
+		if (strbuf_read_file(sb, file, 0) < 0)
+			die_errno(_("could not read input file '%s'"), file);
+	} else {
+		if (strbuf_read(sb, fileno(stdin), 0) < 0)
+			die_errno(_("could not read from stdin"));
+	}
+}
+
+static void interpret_trailers(const struct process_trailer_options *opts,
+			       struct list_head *new_trailer_head,
+			       const char *file)
+{
+	LIST_HEAD(head);
+	struct strbuf sb = STRBUF_INIT;
+	struct trailer_info info;
+	FILE *outfile = stdout;
+
+	trailer_config_init();
+
+	read_input_file(&sb, file);
+
+	if (opts->in_place)
+		outfile = create_in_place_tempfile(file);
+
+	parse_trailers(opts, &info, sb.buf, &head);
+
+	/* Print the lines before the trailers */
+	if (!opts->only_trailers)
+		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
+
+	if (!opts->only_trailers && !info.blank_line_before_trailer)
+		fprintf(outfile, "\n");
+
+
+	if (!opts->only_input) {
+		LIST_HEAD(config_head);
+		LIST_HEAD(arg_head);
+		parse_trailers_from_config(&config_head);
+		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
+		list_splice(&config_head, &arg_head);
+		process_trailers_lists(&head, &arg_head);
+	}
+
+	format_trailers(opts, &head, outfile);
+	free_trailers(&head);
+
+	/* Print the lines after the trailers as is */
+	if (!opts->only_trailers)
+		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
+	trailer_info_release(&info);
+
+	if (opts->in_place)
+		if (rename_tempfile(&trailers_tempfile, file))
+			die_errno(_("could not rename temporary file to %s"), file);
+
+	strbuf_release(&sb);
+}
+
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
diff --git a/trailer.c b/trailer.c
index 916175707d8..d23afa0a65c 100644
--- a/trailer.c
+++ b/trailer.c
@@ -5,7 +5,6 @@
 #include "string-list.h"
 #include "run-command.h"
 #include "commit.h"
-#include "tempfile.h"
 #include "trailer.h"
 #include "list.h"
 /*
@@ -163,8 +162,8 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-static void format_trailers(const struct process_trailer_options *opts,
-			    struct list_head *trailers, FILE *outfile)
+void format_trailers(const struct process_trailer_options *opts,
+		     struct list_head *trailers, FILE *outfile)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
@@ -366,8 +365,8 @@ static int find_same_and_apply_arg(struct list_head *head,
 	return 0;
 }
 
-static void process_trailers_lists(struct list_head *head,
-				   struct list_head *arg_head)
+void process_trailers_lists(struct list_head *head,
+			    struct list_head *arg_head)
 {
 	struct list_head *pos, *p;
 	struct arg_item *arg_tok;
@@ -589,7 +588,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	return 0;
 }
 
-static void trailer_config_init(void)
+void trailer_config_init(void)
 {
 	if (configured)
 		return;
@@ -719,7 +718,7 @@ static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
 	list_add_tail(&new_item->list, arg_head);
 }
 
-static void parse_trailers_from_config(struct list_head *config_head)
+void parse_trailers_from_config(struct list_head *config_head)
 {
 	struct arg_item *item;
 	struct list_head *pos;
@@ -735,8 +734,8 @@ static void parse_trailers_from_config(struct list_head *config_head)
 	}
 }
 
-static void parse_trailers_from_command_line_args(struct list_head *arg_head,
-						  struct list_head *new_trailer_head)
+void parse_trailers_from_command_line_args(struct list_head *arg_head,
+					   struct list_head *new_trailer_head)
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
@@ -775,17 +774,6 @@ static void parse_trailers_from_command_line_args(struct list_head *arg_head,
 	free(cl_separators);
 }
 
-static void read_input_file(struct strbuf *sb, const char *file)
-{
-	if (file) {
-		if (strbuf_read_file(sb, file, 0) < 0)
-			die_errno(_("could not read input file '%s'"), file);
-	} else {
-		if (strbuf_read(sb, fileno(stdin), 0) < 0)
-			die_errno(_("could not read from stdin"));
-	}
-}
-
 static const char *next_line(const char *str)
 {
 	const char *nl = strchrnul(str, '\n');
@@ -1000,10 +988,10 @@ static void unfold_value(struct strbuf *val)
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
  */
-static void parse_trailers(struct trailer_info *info,
-			     const char *str,
-			     struct list_head *head,
-			     const struct process_trailer_options *opts)
+void parse_trailers(const struct process_trailer_options *opts,
+		    struct trailer_info *info,
+		    const char *str,
+		    struct list_head *head)
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
@@ -1035,7 +1023,7 @@ static void parse_trailers(struct trailer_info *info,
 	}
 }
 
-static void free_trailers(struct list_head *trailers)
+void free_trailers(struct list_head *trailers)
 {
 	struct list_head *pos, *p;
 	list_for_each_safe(pos, p, trailers) {
@@ -1044,87 +1032,6 @@ static void free_trailers(struct list_head *trailers)
 	}
 }
 
-static struct tempfile *trailers_tempfile;
-
-static FILE *create_in_place_tempfile(const char *file)
-{
-	struct stat st;
-	struct strbuf filename_template = STRBUF_INIT;
-	const char *tail;
-	FILE *outfile;
-
-	if (stat(file, &st))
-		die_errno(_("could not stat %s"), file);
-	if (!S_ISREG(st.st_mode))
-		die(_("file %s is not a regular file"), file);
-	if (!(st.st_mode & S_IWUSR))
-		die(_("file %s is not writable by user"), file);
-
-	/* Create temporary file in the same directory as the original */
-	tail = strrchr(file, '/');
-	if (tail)
-		strbuf_add(&filename_template, file, tail - file + 1);
-	strbuf_addstr(&filename_template, "git-interpret-trailers-XXXXXX");
-
-	trailers_tempfile = xmks_tempfile_m(filename_template.buf, st.st_mode);
-	strbuf_release(&filename_template);
-	outfile = fdopen_tempfile(trailers_tempfile, "w");
-	if (!outfile)
-		die_errno(_("could not open temporary file"));
-
-	return outfile;
-}
-
-void interpret_trailers(const struct process_trailer_options *opts,
-			struct list_head *new_trailer_head,
-			const char *file)
-{
-	LIST_HEAD(head);
-	struct strbuf sb = STRBUF_INIT;
-	struct trailer_info info;
-	FILE *outfile = stdout;
-
-	trailer_config_init();
-
-	read_input_file(&sb, file);
-
-	if (opts->in_place)
-		outfile = create_in_place_tempfile(file);
-
-	parse_trailers(&info, sb.buf, &head, opts);
-
-	/* Print the lines before the trailers */
-	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
-
-	if (!opts->only_trailers && !info.blank_line_before_trailer)
-		fprintf(outfile, "\n");
-
-
-	if (!opts->only_input) {
-		LIST_HEAD(config_head);
-		LIST_HEAD(arg_head);
-		parse_trailers_from_config(&config_head);
-		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
-		list_splice(&config_head, &arg_head);
-		process_trailers_lists(&head, &arg_head);
-	}
-
-	format_trailers(opts, &head, outfile);
-	free_trailers(&head);
-
-	/* Print the lines after the trailers as is */
-	if (!opts->only_trailers)
-		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
-	trailer_info_release(&info);
-
-	if (opts->in_place)
-		if (rename_tempfile(&trailers_tempfile, file))
-			die_errno(_("could not rename temporary file to %s"), file);
-
-	strbuf_release(&sb);
-}
-
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts)
 {
diff --git a/trailer.h b/trailer.h
index 37033e631a1..c292d44b62f 100644
--- a/trailer.h
+++ b/trailer.h
@@ -81,15 +81,29 @@ struct process_trailer_options {
 
 #define PROCESS_TRAILER_OPTIONS_INIT {0}
 
-void interpret_trailers(const struct process_trailer_options *opts,
-			struct list_head *new_trailer_head,
-			const char *file);
+void parse_trailers_from_config(struct list_head *config_head);
+
+void parse_trailers_from_command_line_args(struct list_head *arg_head,
+					   struct list_head *new_trailer_head);
+
+void process_trailers_lists(struct list_head *head,
+			    struct list_head *arg_head);
+
+void parse_trailers(const struct process_trailer_options *,
+		    struct trailer_info *,
+		    const char *str,
+		    struct list_head *head);
 
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts);
 
 void trailer_info_release(struct trailer_info *info);
 
+void trailer_config_init(void);
+void format_trailers(const struct process_trailer_options *,
+		     struct list_head *trailers, FILE *outfile);
+void free_trailers(struct list_head *);
+
 /*
  * Format the trailers from the commit msg "msg" into the strbuf "out".
  * Note two caveats about "opts":
-- 
gitgitgadget

[PATCH v4 05/28] trailer: start preparing for formatting unification

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Currently there are two functions for formatting trailers in
<trailer.h>:

    void format_trailers(const struct process_trailer_options *,
                         struct list_head *trailers, FILE *outfile);

    void format_trailers_from_commit(struct strbuf *out, const char *msg,
                                     const struct process_trailer_options *opts);

and although they are similar enough (even taking the same
process_trailer_options struct pointer) they are used quite differently.
One might intuitively think that format_trailers_from_commit() builds on
top of format_trailers(), but this is not the case. Instead
format_trailers_from_commit() calls format_trailer_info() and
format_trailers() is never called in that codepath.

This is a preparatory refactor to help us deprecate format_trailers() in
favor of format_trailer_info() (at which point we can rename the latter
to the former). When the deprecation is complete, both
format_trailers_from_commit(), and the interpret-trailers builtin will
be able to call into the same helper function (instead of
format_trailers() and format_trailer_info(), respectively). Unifying the
formatters is desirable because it simplifies the API.

Reorder parameters for format_trailers_from_commit() to prefer

    const struct process_trailer_options *opts

as the first parameter, because these options are intimately tied to
formatting trailers. And take

    struct strbuf *out

last, because it's an "out parameter" (something that the caller wants
to use as the output of this function).

Signed-off-by: Linus Arver <linusa@google.com>
---
 pretty.c     |  2 +-
 ref-filter.c |  2 +-
 trailer.c    | 11 ++++++-----
 trailer.h    |  5 +++--
 4 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/pretty.c b/pretty.c
index cf964b060cd..bdbed4295aa 100644
--- a/pretty.c
+++ b/pretty.c
@@ -1759,7 +1759,7 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */
 				goto trailer_out;
 		}
 		if (*arg == ')') {
-			format_trailers_from_commit(sb, msg + c->subject_off, &opts);
+			format_trailers_from_commit(&opts, msg + c->subject_off, sb);
 			ret = arg - placeholder + 1;
 		}
 	trailer_out:
diff --git a/ref-filter.c b/ref-filter.c
index 35b989e1dfe..d358953b0ce 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -1985,7 +1985,7 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp
 			struct strbuf s = STRBUF_INIT;
 
 			/* Format the trailer info according to the trailer_opts given */
-			format_trailers_from_commit(&s, subpos, &atom->u.contents.trailer_opts);
+			format_trailers_from_commit(&atom->u.contents.trailer_opts, subpos, &s);
 
 			v->s = strbuf_detach(&s, NULL);
 		} else if (atom->u.contents.option == C_BARE)
diff --git a/trailer.c b/trailer.c
index d23afa0a65c..5025be97899 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1083,10 +1083,10 @@ void trailer_info_release(struct trailer_info *info)
 	free(info->trailers);
 }
 
-static void format_trailer_info(struct strbuf *out,
+static void format_trailer_info(const struct process_trailer_options *opts,
 				const struct trailer_info *info,
 				const char *msg,
-				const struct process_trailer_options *opts)
+				struct strbuf *out)
 {
 	size_t origlen = out->len;
 	size_t i;
@@ -1144,13 +1144,14 @@ static void format_trailer_info(struct strbuf *out,
 
 }
 
-void format_trailers_from_commit(struct strbuf *out, const char *msg,
-				 const struct process_trailer_options *opts)
+void format_trailers_from_commit(const struct process_trailer_options *opts,
+				 const char *msg,
+				 struct strbuf *out)
 {
 	struct trailer_info info;
 
 	trailer_info_get(&info, msg, opts);
-	format_trailer_info(out, &info, msg, opts);
+	format_trailer_info(opts, &info, msg, out);
 	trailer_info_release(&info);
 }
 
diff --git a/trailer.h b/trailer.h
index c292d44b62f..c6d3ee49bbf 100644
--- a/trailer.h
+++ b/trailer.h
@@ -115,8 +115,9 @@ void free_trailers(struct list_head *);
  *     only the trailer block itself, even if the "only_trailers" option is not
  *     set.
  */
-void format_trailers_from_commit(struct strbuf *out, const char *msg,
-				 const struct process_trailer_options *opts);
+void format_trailers_from_commit(const struct process_trailer_options *opts,
+				 const char *msg,
+				 struct strbuf *out);
 
 /*
  * An interface for iterating over the trailers found in a particular commit
-- 
gitgitgadget

[PATCH v4 06/28] trailer_info_get(): reorder parameters

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

Take

    const struct process_trailer_options *opts

as the first parameter, because these options are required for
parsing trailers (e.g., whether to treat "---" as the end of the log
message). And take

    struct trailer_info *info

last, because it's an "out parameter" (something that the caller wants
to use as the output of this function).

Signed-off-by: Linus Arver <linusa@google.com>
---
 sequencer.c |  2 +-
 trailer.c   | 11 ++++++-----
 trailer.h   |  5 +++--
 3 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 3cc88d8a800..8e199fc8a47 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -332,7 +332,7 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 		sb->buf[sb->len - ignore_footer] = '\0';
 	}
 
-	trailer_info_get(&info, sb->buf, &opts);
+	trailer_info_get(&opts, sb->buf, &info);
 
 	if (ignore_footer)
 		sb->buf[sb->len - ignore_footer] = saved_char;
diff --git a/trailer.c b/trailer.c
index 5025be97899..f92d844361a 100644
--- a/trailer.c
+++ b/trailer.c
@@ -997,7 +997,7 @@ void parse_trailers(const struct process_trailer_options *opts,
 	struct strbuf val = STRBUF_INIT;
 	size_t i;
 
-	trailer_info_get(info, str, opts);
+	trailer_info_get(opts, str, info);
 
 	for (i = 0; i < info->trailer_nr; i++) {
 		int separator_pos;
@@ -1032,8 +1032,9 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
-void trailer_info_get(struct trailer_info *info, const char *str,
-		      const struct process_trailer_options *opts)
+void trailer_info_get(const struct process_trailer_options *opts,
+		      const char *str,
+		      struct trailer_info *info)
 {
 	size_t end_of_log_message = 0, trailer_block_start = 0;
 	struct strbuf **trailer_lines, **ptr;
@@ -1150,7 +1151,7 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 {
 	struct trailer_info info;
 
-	trailer_info_get(&info, msg, opts);
+	trailer_info_get(opts, msg, &info);
 	format_trailer_info(opts, &info, msg, out);
 	trailer_info_release(&info);
 }
@@ -1161,7 +1162,7 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 	strbuf_init(&iter->key, 0);
 	strbuf_init(&iter->val, 0);
 	opts.no_divider = 1;
-	trailer_info_get(&iter->internal.info, msg, &opts);
+	trailer_info_get(&opts, msg, &iter->internal.info);
 	iter->internal.cur = 0;
 }
 
diff --git a/trailer.h b/trailer.h
index c6d3ee49bbf..410c61b62be 100644
--- a/trailer.h
+++ b/trailer.h
@@ -94,8 +94,9 @@ void parse_trailers(const struct process_trailer_options *,
 		    const char *str,
 		    struct list_head *head);
 
-void trailer_info_get(struct trailer_info *info, const char *str,
-		      const struct process_trailer_options *opts);
+void trailer_info_get(const struct process_trailer_options *,
+		      const char *str,
+		      struct trailer_info *);
 
 void trailer_info_release(struct trailer_info *info);
 
-- 
gitgitgadget

[PATCH v4 07/28] format_trailers(): use strbuf instead of FILE

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

Make format_trailers() also write to a strbuf, to align with
format_trailers_from_commit() which also does the same. Doing this makes
format_trailers() behave similar to format_trailer_info() (which will
soon help us replace one with the other).

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  6 +++++-
 trailer.c                    | 13 +++++++------
 trailer.h                    |  3 ++-
 3 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index d1cf0aa33a2..11f4ce9e4a2 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -140,6 +140,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
+	struct strbuf trailer_block = STRBUF_INIT;
 	struct trailer_info info;
 	FILE *outfile = stdout;
 
@@ -169,8 +170,11 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 		process_trailers_lists(&head, &arg_head);
 	}
 
-	format_trailers(opts, &head, outfile);
+	/* Print trailer block. */
+	format_trailers(opts, &head, &trailer_block);
 	free_trailers(&head);
+	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
+	strbuf_release(&trailer_block);
 
 	/* Print the lines after the trailers as is */
 	if (!opts->only_trailers)
diff --git a/trailer.c b/trailer.c
index f92d844361a..cbd643cd1fe 100644
--- a/trailer.c
+++ b/trailer.c
@@ -144,12 +144,12 @@ static char last_non_space_char(const char *s)
 	return '\0';
 }
 
-static void print_tok_val(FILE *outfile, const char *tok, const char *val)
+static void print_tok_val(struct strbuf *out, const char *tok, const char *val)
 {
 	char c;
 
 	if (!tok) {
-		fprintf(outfile, "%s\n", val);
+		strbuf_addf(out, "%s\n", val);
 		return;
 	}
 
@@ -157,13 +157,14 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 	if (!c)
 		return;
 	if (strchr(separators, c))
-		fprintf(outfile, "%s%s\n", tok, val);
+		strbuf_addf(out, "%s%s\n", tok, val);
 	else
-		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
+		strbuf_addf(out, "%s%c %s\n", tok, separators[0], val);
 }
 
 void format_trailers(const struct process_trailer_options *opts,
-		     struct list_head *trailers, FILE *outfile)
+		     struct list_head *trailers,
+		     struct strbuf *out)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
@@ -171,7 +172,7 @@ void format_trailers(const struct process_trailer_options *opts,
 		item = list_entry(pos, struct trailer_item, list);
 		if ((!opts->trim_empty || strlen(item->value) > 0) &&
 		    (!opts->only_trailers || item->token))
-			print_tok_val(outfile, item->token, item->value);
+			print_tok_val(out, item->token, item->value);
 	}
 }
 
diff --git a/trailer.h b/trailer.h
index 410c61b62be..1d106b6dd40 100644
--- a/trailer.h
+++ b/trailer.h
@@ -102,7 +102,8 @@ void trailer_info_release(struct trailer_info *info);
 
 void trailer_config_init(void);
 void format_trailers(const struct process_trailer_options *,
-		     struct list_head *trailers, FILE *outfile);
+		     struct list_head *trailers,
+		     struct strbuf *out);
 void free_trailers(struct list_head *);
 
 /*
-- 
gitgitgadget

[PATCH v4 08/28] format_trailer_info(): move "fast path" to caller

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

This allows us to drop the "msg" parameter from format_trailer_info(),
so that it take 3 parameters, similar to format_trailers() which also
takes 3 parameters:

    void format_trailers(const struct process_trailer_options *opts,
                         struct list_head *trailers,
                         struct strbuf *out)

The short-term goal is to make format_trailer_info() be smart enough to
deprecate format_trailers(). And then ultimately we will rename
format_trailer_info() to format_trailers().

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/trailer.c b/trailer.c
index cbd643cd1fe..e92d0154d90 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1087,21 +1087,11 @@ void trailer_info_release(struct trailer_info *info)
 
 static void format_trailer_info(const struct process_trailer_options *opts,
 				const struct trailer_info *info,
-				const char *msg,
 				struct strbuf *out)
 {
 	size_t origlen = out->len;
 	size_t i;
 
-	/* If we want the whole block untouched, we can take the fast path. */
-	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
-	    !opts->separator && !opts->key_only && !opts->value_only &&
-	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info->trailer_block_start,
-			   info->trailer_block_end - info->trailer_block_start);
-		return;
-	}
-
 	for (i = 0; i < info->trailer_nr; i++) {
 		char *trailer = info->trailers[i];
 		ssize_t separator_pos = find_separator(trailer, separators);
@@ -1153,7 +1143,15 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 	struct trailer_info info;
 
 	trailer_info_get(opts, msg, &info);
-	format_trailer_info(opts, &info, msg, out);
+	/* If we want the whole block untouched, we can take the fast path. */
+	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
+	    !opts->separator && !opts->key_only && !opts->value_only &&
+	    !opts->key_value_separator) {
+		strbuf_add(out, msg + info.trailer_block_start,
+			   info.trailer_block_end - info.trailer_block_start);
+	} else
+		format_trailer_info(opts, &info, out);
+
 	trailer_info_release(&info);
 }
 
-- 
gitgitgadget

[PATCH v4 09/28] format_trailers_from_commit(): indirectly call trailer_info_get()

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

Instead of calling trailer_info_get() directly, call parse_trailers()
which already calls trailer_info_get(). This change is a NOP because
format_trailer_info() only looks at the "trailers" string array, not the
trailer_item objects which parse_trailers() populates.

In the next patch, we'll change format_trailer_info() to use the parsed
trailer_item objects instead of the string array.

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/trailer.c b/trailer.c
index e92d0154d90..e6665c99cc3 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1140,9 +1140,11 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 				 const char *msg,
 				 struct strbuf *out)
 {
+	LIST_HEAD(trailers);
 	struct trailer_info info;
 
-	trailer_info_get(opts, msg, &info);
+	parse_trailers(opts, &info, msg, &trailers);
+
 	/* If we want the whole block untouched, we can take the fast path. */
 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
 	    !opts->separator && !opts->key_only && !opts->value_only &&
@@ -1152,6 +1154,7 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 	} else
 		format_trailer_info(opts, &info, out);
 
+	free_trailers(&trailers);
 	trailer_info_release(&info);
 }
 
-- 
gitgitgadget

[PATCH v4 10/28] format_trailer_info(): use trailer_item objects

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

Make format_trailer_info() operate on trailer_item objects, not the raw
string array.

This breaks t4205 and t6300. We will continue to make improvements until
the test suite passes again, ultimately renaming format_trailer_info()
to format_trailers(), at which point the unification of these formatters
will be complete.

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 21 ++++++++++-----------
 1 file changed, 10 insertions(+), 11 deletions(-)

diff --git a/trailer.c b/trailer.c
index e6665c99cc3..6333dfe1c11 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1086,21 +1086,21 @@ void trailer_info_release(struct trailer_info *info)
 }
 
 static void format_trailer_info(const struct process_trailer_options *opts,
-				const struct trailer_info *info,
+				struct list_head *trailers,
 				struct strbuf *out)
 {
 	size_t origlen = out->len;
-	size_t i;
-
-	for (i = 0; i < info->trailer_nr; i++) {
-		char *trailer = info->trailers[i];
-		ssize_t separator_pos = find_separator(trailer, separators);
+	struct list_head *pos;
+	struct trailer_item *item;
 
-		if (separator_pos >= 1) {
+	list_for_each(pos, trailers) {
+		item = list_entry(pos, struct trailer_item, list);
+		if (item->token) {
 			struct strbuf tok = STRBUF_INIT;
 			struct strbuf val = STRBUF_INIT;
+			strbuf_addstr(&tok, item->token);
+			strbuf_addstr(&val, item->value);
 
-			parse_trailer(&tok, &val, NULL, trailer, separator_pos);
 			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
 				if (opts->unfold)
 					unfold_value(&val);
@@ -1127,13 +1127,12 @@ static void format_trailer_info(const struct process_trailer_options *opts,
 			if (opts->separator && out->len != origlen) {
 				strbuf_addbuf(out, opts->separator);
 			}
-			strbuf_addstr(out, trailer);
+			strbuf_addstr(out, item->value);
 			if (opts->separator) {
 				strbuf_rtrim(out);
 			}
 		}
 	}
-
 }
 
 void format_trailers_from_commit(const struct process_trailer_options *opts,
@@ -1152,7 +1151,7 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 		strbuf_add(out, msg + info.trailer_block_start,
 			   info.trailer_block_end - info.trailer_block_start);
 	} else
-		format_trailer_info(opts, &info, out);
+		format_trailer_info(opts, &trailers, out);
 
 	free_trailers(&trailers);
 	trailer_info_release(&info);
-- 
gitgitgadget

[PATCH v4 11/28] format_trailer_info(): drop redundant unfold_value()

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

In the last patch we made format_trailer_info() use trailer_item objects
instead of the "trailers" string array. This means that the call to
unfold_value() here is redundant because the trailer_item objects are
already unfolded in parse_trailers() which is a dependency of our
caller, format_trailers_from_commit().

Remove the redundant call.

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/trailer.c b/trailer.c
index 6333dfe1c11..12cae5b73d2 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1102,9 +1102,6 @@ static void format_trailer_info(const struct process_trailer_options *opts,
 			strbuf_addstr(&val, item->value);
 
 			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
-				if (opts->unfold)
-					unfold_value(&val);
-
 				if (opts->separator && out->len != origlen)
 					strbuf_addbuf(out, opts->separator);
 				if (!opts->value_only)
-- 
gitgitgadget

[PATCH v4 12/28] format_trailer_info(): append newline for non-trailer lines

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This wraps up the preparatory refactors to unify the trailer formatters.

Two patches ago we made format_trailer_info() use trailer_item objects
instead of the "trailers" string array. The strings in the array
include trailing newlines, because the string array is split up with

    trailer_lines = strbuf_split_buf(str + trailer_block_start,
                                     end_of_log_message - trailer_block_start,
                                     '\n',
                                     0);

in trailer_info_get() and strbuf_split_buf() includes the terminator (in
this case the newline character '\n') for each split-up substring.

And before we made the transition to use trailer_item objects for it,
format_trailer_info() called parse_trailer() (which trims newlines) for
trailer lines but did _not_ call parse_trailer() for non-trailer lines.
So for trailer lines it had to add back the trimmed newline like this

    if (!opts->separator)
        strbuf_addch(out, '\n');

But for non-trailer lines it didn't have to add back the newline because
it could just reuse same string in the "trailers" string array (which
again, already included the trailing newline).

Now that format_trailer_info() uses trailer_item objects for all cases,
it can't rely on "trailers" string array anymore.  And so it must be
taught to add a newline back when printing non-trailer lines, just like
it already does for trailer lines. Do so now.

The test suite passes again, so format_trailer_info() is in better shape
supersede format_trailers(), which we'll do in the next patch.

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/trailer.c b/trailer.c
index 12cae5b73d2..0774a544c4f 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1125,9 +1125,10 @@ static void format_trailer_info(const struct process_trailer_options *opts,
 				strbuf_addbuf(out, opts->separator);
 			}
 			strbuf_addstr(out, item->value);
-			if (opts->separator) {
+			if (opts->separator)
 				strbuf_rtrim(out);
-			}
+			else
+				strbuf_addch(out, '\n');
 		}
 	}
 }
-- 
gitgitgadget

[PATCH v4 13/28] trailer: begin formatting unification

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Now that the preparatory refactors are over, we can replace the call to
format_trailers() in interpret-trailers with format_trailer_info(). This
unifies the trailer formatting machinery, but at the cost of breaking
tests.

More specifically, this patch breaks t7502 and t7513, but only because
we haven't fully stolen the features present in format_trailers() (which
knows about opts->trim_empty) and print_tok_val() (which has
non-hardcoded printing of the separator and space).

We will teach format_trailer_info() these features in the next two
patches to make all tests pass again.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  2 +-
 trailer.c                    | 12 ++++++------
 trailer.h                    |  2 +-
 3 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 11f4ce9e4a2..f57af0db37b 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -171,7 +171,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	}
 
 	/* Print trailer block. */
-	format_trailers(opts, &head, &trailer_block);
+	format_trailer_info(opts, &head, &trailer_block);
 	free_trailers(&head);
 	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
 	strbuf_release(&trailer_block);
diff --git a/trailer.c b/trailer.c
index 0774a544c4f..f4defad3dae 100644
--- a/trailer.c
+++ b/trailer.c
@@ -162,9 +162,9 @@ static void print_tok_val(struct strbuf *out, const char *tok, const char *val)
 		strbuf_addf(out, "%s%c %s\n", tok, separators[0], val);
 }
 
-void format_trailers(const struct process_trailer_options *opts,
-		     struct list_head *trailers,
-		     struct strbuf *out)
+static void format_trailers(const struct process_trailer_options *opts,
+			    struct list_head *trailers,
+			    struct strbuf *out)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
@@ -1085,9 +1085,9 @@ void trailer_info_release(struct trailer_info *info)
 	free(info->trailers);
 }
 
-static void format_trailer_info(const struct process_trailer_options *opts,
-				struct list_head *trailers,
-				struct strbuf *out)
+void format_trailer_info(const struct process_trailer_options *opts,
+			 struct list_head *trailers,
+			 struct strbuf *out)
 {
 	size_t origlen = out->len;
 	struct list_head *pos;
diff --git a/trailer.h b/trailer.h
index 1d106b6dd40..3c13006a4c1 100644
--- a/trailer.h
+++ b/trailer.h
@@ -101,7 +101,7 @@ void trailer_info_get(const struct process_trailer_options *,
 void trailer_info_release(struct trailer_info *info);
 
 void trailer_config_init(void);
-void format_trailers(const struct process_trailer_options *,
+void format_trailer_info(const struct process_trailer_options *,
 		     struct list_head *trailers,
 		     struct strbuf *out);
 void free_trailers(struct list_head *);
-- 
gitgitgadget

[PATCH v4 14/28] format_trailer_info(): teach it about opts->trim_empty

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This fixes 4 tests in t7513 to go from

    t7513-interpret-trailers.sh  (Wstat: 256 (exited 1) Tests: 94 Failed: 55)
      Failed tests:  2-5, 8, 14, 24-28, 31-37, 43-62, 66-74
                    77-80, 82-85

to

    t7513-interpret-trailers.sh  (Wstat: 256 (exited 1) Tests: 94 Failed: 51)
      Failed tests:  2-5, 14, 24-28, 31-32, 36-37, 43-62, 66-74
                    77-80, 82-85

. The next patch will fix the remaining broken test cases in t7513 and
t7502.

Even though the next patch fixes the vast majority of these test cases,
we have to position that patch after this one to avoid breaking the
build because of the way these patches delete relevant (and obsolete)
code.

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 23 +++++++++--------------
 1 file changed, 9 insertions(+), 14 deletions(-)

diff --git a/trailer.c b/trailer.c
index f4defad3dae..c28b6c11cc5 100644
--- a/trailer.c
+++ b/trailer.c
@@ -162,20 +162,6 @@ static void print_tok_val(struct strbuf *out, const char *tok, const char *val)
 		strbuf_addf(out, "%s%c %s\n", tok, separators[0], val);
 }
 
-static void format_trailers(const struct process_trailer_options *opts,
-			    struct list_head *trailers,
-			    struct strbuf *out)
-{
-	struct list_head *pos;
-	struct trailer_item *item;
-	list_for_each(pos, trailers) {
-		item = list_entry(pos, struct trailer_item, list);
-		if ((!opts->trim_empty || strlen(item->value) > 0) &&
-		    (!opts->only_trailers || item->token))
-			print_tok_val(out, item->token, item->value);
-	}
-}
-
 static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
 {
 	struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
@@ -1101,6 +1087,15 @@ void format_trailer_info(const struct process_trailer_options *opts,
 			strbuf_addstr(&tok, item->token);
 			strbuf_addstr(&val, item->value);
 
+			/*
+			 * Skip key/value pairs where the value was empty. This
+			 * can happen from trailers specified without a
+			 * separator, like `--trailer "Reviewed-by"` (no
+			 * corresponding value).
+			 */
+			if (opts->trim_empty && !strlen(item->value))
+				continue;
+
 			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
 				if (opts->separator && out->len != origlen)
 					strbuf_addbuf(out, opts->separator);
-- 
gitgitgadget

[PATCH v4 15/28] format_trailer_info(): avoid double-printing the separator

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Do not hardcode the printing of ": " as the separator and space (which
can result in double-printing these characters); instead only
print the separator and space if we cannot find any recognized separator
somewhere in the key (yes, keys may have a trailing separator in it ---
we will eventually fix this design but not now). Do so by copying the
code out of print_tok_val(), and deleting the same function.

The test suite passes again with this change.

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 25 +++++--------------------
 1 file changed, 5 insertions(+), 20 deletions(-)

diff --git a/trailer.c b/trailer.c
index c28b6c11cc5..5c42a19943a 100644
--- a/trailer.c
+++ b/trailer.c
@@ -144,24 +144,6 @@ static char last_non_space_char(const char *s)
 	return '\0';
 }
 
-static void print_tok_val(struct strbuf *out, const char *tok, const char *val)
-{
-	char c;
-
-	if (!tok) {
-		strbuf_addf(out, "%s\n", val);
-		return;
-	}
-
-	c = last_non_space_char(tok);
-	if (!c)
-		return;
-	if (strchr(separators, c))
-		strbuf_addf(out, "%s%s\n", tok, val);
-	else
-		strbuf_addf(out, "%s%c %s\n", tok, separators[0], val);
-}
-
 static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
 {
 	struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
@@ -1104,8 +1086,11 @@ void format_trailer_info(const struct process_trailer_options *opts,
 				if (!opts->key_only && !opts->value_only) {
 					if (opts->key_value_separator)
 						strbuf_addbuf(out, opts->key_value_separator);
-					else
-						strbuf_addstr(out, ": ");
+					else {
+						char c = last_non_space_char(tok.buf);
+						if (c && !strchr(separators, c))
+							strbuf_addf(out, "%c ", separators[0]);
+					}
 				}
 				if (!opts->key_only)
 					strbuf_addbuf(out, &val);
-- 
gitgitgadget

[PATCH v4 16/28] trailer: finish formatting unification

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Rename format_trailer_info() to format_trailers(). Finally, both
interpret-trailers and format_trailers_from_commit() can call
"format_trailers()"!

Update the comment in <trailer.h> to remove the (now obsolete) caveats
about format_trailers_from_commit().

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  2 +-
 trailer.c                    |  8 ++++----
 trailer.h                    | 15 ++++-----------
 3 files changed, 9 insertions(+), 16 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index f57af0db37b..11f4ce9e4a2 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -171,7 +171,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	}
 
 	/* Print trailer block. */
-	format_trailer_info(opts, &head, &trailer_block);
+	format_trailers(opts, &head, &trailer_block);
 	free_trailers(&head);
 	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
 	strbuf_release(&trailer_block);
diff --git a/trailer.c b/trailer.c
index 5c42a19943a..4f3318802d1 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1053,9 +1053,9 @@ void trailer_info_release(struct trailer_info *info)
 	free(info->trailers);
 }
 
-void format_trailer_info(const struct process_trailer_options *opts,
-			 struct list_head *trailers,
-			 struct strbuf *out)
+void format_trailers(const struct process_trailer_options *opts,
+		     struct list_head *trailers,
+		     struct strbuf *out)
 {
 	size_t origlen = out->len;
 	struct list_head *pos;
@@ -1129,7 +1129,7 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 		strbuf_add(out, msg + info.trailer_block_start,
 			   info.trailer_block_end - info.trailer_block_start);
 	} else
-		format_trailer_info(opts, &trailers, out);
+		format_trailers(opts, &trailers, out);
 
 	free_trailers(&trailers);
 	trailer_info_release(&info);
diff --git a/trailer.h b/trailer.h
index 3c13006a4c1..9f42aa75994 100644
--- a/trailer.h
+++ b/trailer.h
@@ -101,23 +101,16 @@ void trailer_info_get(const struct process_trailer_options *,
 void trailer_info_release(struct trailer_info *info);
 
 void trailer_config_init(void);
-void format_trailer_info(const struct process_trailer_options *,
+void format_trailers(const struct process_trailer_options *,
 		     struct list_head *trailers,
 		     struct strbuf *out);
 void free_trailers(struct list_head *);
 
 /*
- * Format the trailers from the commit msg "msg" into the strbuf "out".
- * Note two caveats about "opts":
- *
- *   - this is primarily a helper for pretty.c, and not
- *     all of the flags are supported.
- *
- *   - this differs from process_trailers slightly in that we always format
- *     only the trailer block itself, even if the "only_trailers" option is not
- *     set.
+ * Convenience function to format the trailers from the commit msg "msg" into
+ * the strbuf "out". Reuses format_trailers() internally.
  */
-void format_trailers_from_commit(const struct process_trailer_options *opts,
+void format_trailers_from_commit(const struct process_trailer_options *,
 				 const char *msg,
 				 struct strbuf *out);
 
-- 
gitgitgadget

[PATCH v4 17/28] trailer: teach iterator about non-trailer lines

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Previously the iterator did not iterate over non-trailer lines. This was
somewhat unfortunate, because trailer blocks could have non-trailer
lines in them since 146245063e (trailer: allow non-trailers in trailer
block, 2016-10-21), which was before the iterator was created in
f0939a0eb1 (trailer: add interface for iterating over commit trailers,
2020-09-27).

So if trailer API users wanted to iterate over all lines in a trailer
block (including non-trailer lines), they could not use the iterator and
were forced to use the lower-level trailer_info struct directly (which
provides a raw string array that includes all lines in the trailer
block).

Change the iterator's behavior so that we also iterate over non-trailer
lines, instead of skipping over them. The new "raw" member of the
iterator allows API users to access previously inaccessible non-trailer
lines. Reword the variable "trailer" to just "line" because this
variable can now hold both trailer lines _and_ non-trailer lines.

The new "raw" member is important because anyone currently not using the
iterator is using trailer_info's raw string array directly to access
lines to check what the combined key + value looks like. If we didn't
provide a "raw" member here, iterator users would have to re-construct
the unparsed line by concatenating the key and value back together again
--- which places an undue burden for iterator users.

The next patch demonstrates the use of the iterator in sequencer.c as an
example of where "raw" will be useful, so that it can start using the
iterator.

For the existing use of the iterator in builtin/shortlog.c, we don't
have to change the code there because that code does

    trailer_iterator_init(&iter, body);
    while (trailer_iterator_advance(&iter)) {
        const char *value = iter.val.buf;

        if (!string_list_has_string(&log->trailers, iter.key.buf))
            continue;

        ...

and the

        if (!string_list_has_string(&log->trailers, iter.key.buf))

condition already skips over non-trailer lines (iter.key.buf is empty
for non-trailer lines, making the comparison still work unmodified even
with this patch).

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 12 +++++-------
 trailer.h |  8 ++++++++
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/trailer.c b/trailer.c
index 4f3318802d1..2cc4a910411 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1147,17 +1147,15 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 
 int trailer_iterator_advance(struct trailer_iterator *iter)
 {
-	while (iter->internal.cur < iter->internal.info.trailer_nr) {
-		char *trailer = iter->internal.info.trailers[iter->internal.cur++];
-		int separator_pos = find_separator(trailer, separators);
-
-		if (separator_pos < 1)
-			continue; /* not a real trailer */
+	if (iter->internal.cur < iter->internal.info.trailer_nr) {
+		char *line = iter->internal.info.trailers[iter->internal.cur++];
+		int separator_pos = find_separator(line, separators);
 
+		iter->raw = line;
 		strbuf_reset(&iter->key);
 		strbuf_reset(&iter->val);
 		parse_trailer(&iter->key, &iter->val, NULL,
-			      trailer, separator_pos);
+			      line, separator_pos);
 		/* Always unfold values during iteration. */
 		unfold_value(&iter->val);
 		return 1;
diff --git a/trailer.h b/trailer.h
index 9f42aa75994..ebafa3657e4 100644
--- a/trailer.h
+++ b/trailer.h
@@ -125,6 +125,14 @@ void format_trailers_from_commit(const struct process_trailer_options *,
  *   trailer_iterator_release(&iter);
  */
 struct trailer_iterator {
+	/*
+	 * Raw line (e.g., "foo: bar baz") before being parsed as a trailer
+	 * key/val pair as part of a trailer block. A trailer block can be
+	 * either 100% trailer lines, or mixed in with non-trailer lines (in
+	 * which case at least 25% must be trailer lines).
+	 */
+	const char *raw;
+
 	struct strbuf key;
 	struct strbuf val;
 
-- 
gitgitgadget

[PATCH v4 18/28] sequencer: use the trailer iterator

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This patch allows for the removal of "trailer_info_get()" from the
trailer.h API, which will be in the next patch.

Instead of calling "trailer_info_get()", which is a low-level function
in the trailers implementation (trailer.c), call
trailer_iterator_advance(), which was specifically designed for public
consumption in f0939a0eb1 (trailer: add interface for iterating over
commit trailers, 2020-09-27).

Avoiding "trailer_info_get()" means we don't have to worry about options
like "no_divider" (relevant for parsing trailers). We also don't have to
check for things like "info.trailer_start == info.trailer_end" to see
whether there were any trailers (instead we can just check to see
whether the iterator advanced at all).

Note how we have to use "iter.raw" in order to get the same behavior as
before when we iterated over the unparsed string array (trailers[]) in
trailer_info.

Signed-off-by: Linus Arver <linusa@google.com>
---
 sequencer.c | 27 ++++++++++++---------------
 1 file changed, 12 insertions(+), 15 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 8e199fc8a47..35462a6a9d9 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -319,35 +319,32 @@ static const char *get_todo_path(const struct replay_opts *opts)
 static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	size_t ignore_footer)
 {
-	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
-	struct trailer_info info;
-	size_t i;
+	struct trailer_iterator iter;
+	size_t i = 0;
 	int found_sob = 0, found_sob_last = 0;
 	char saved_char;
 
-	opts.no_divider = 1;
-
 	if (ignore_footer) {
 		saved_char = sb->buf[sb->len - ignore_footer];
 		sb->buf[sb->len - ignore_footer] = '\0';
 	}
 
-	trailer_info_get(&opts, sb->buf, &info);
+	trailer_iterator_init(&iter, sb->buf);
 
 	if (ignore_footer)
 		sb->buf[sb->len - ignore_footer] = saved_char;
 
-	if (info.trailer_block_start == info.trailer_block_end)
-		return 0;
+	while (trailer_iterator_advance(&iter)) {
+		i++;
+		if (sob && !strncmp(iter.raw, sob->buf, sob->len))
+			found_sob = i;
+	}
+	trailer_iterator_release(&iter);
 
-	for (i = 0; i < info.trailer_nr; i++)
-		if (sob && !strncmp(info.trailers[i], sob->buf, sob->len)) {
-			found_sob = 1;
-			if (i == info.trailer_nr - 1)
-				found_sob_last = 1;
-		}
+	if (!i)
+		return 0;
 
-	trailer_info_release(&info);
+	found_sob_last = (int)i == found_sob;
 
 	if (found_sob_last)
 		return 3;
-- 
gitgitgadget

[PATCH v4 19/28] trailer: make trailer_info struct private

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

In 13211ae23f (trailer: separate public from internal portion of
trailer_iterator, 2023-09-09) we moved trailer_info behind an anonymous
struct to discourage use by trailer.h API users. However it still left
open the possibility of external use of trailer_info itself. Now that
there are no external users of trailer_info, we can make this struct
private.

Make this struct private by putting its definition inside trailer.c.
This has two benefits:

  (1) it makes the surface area of the public facing
      interface (trailer.h) smaller, and

  (2) external API users are unable to peer inside this struct (because
      it is only ever exposed as an opaque pointer).

There are a couple disadvantages:

  (A) every time the member of the struct is accessed an extra pointer
      dereference must be done, and

  (B) for users of trailer_info outside trailer.c, this struct can no
      longer be allocated on the stack and may only be allocated on the
      heap (because its definition is hidden away in trailer.c) and
      appropriately deallocated by the user.

This patch believes that the benefits outweight the advantages for
designing APIs, as explained below.

Making trailer_info private exposes existing deficiencies in the API.
This is because users of this struct had full access to its internals,
so there wasn't much need to actually design it to be "complete" in the
sense that API users only needed to use what was provided by the API.
For example, the location of the trailer block (start/end offsets
relative to the start of the input text) was accessible by looking at
these struct members directly. Now that the struct is private, we have
to expose new API functions to allow clients to access this
information (see builtin/interpret-trailers.c).

The idea in this patch to hide implementation details behind an "opaque
pointer" is also known as the "pimpl" (pointer to implementation) idiom
in C++ and is a common pattern in that language (where, for example,
abstract classes only have pointers to concrete classes).

However, the original inspiration to use this idiom does not come from
C++, but instead the book "C Interfaces and Implementations: Techniques
for Creating Reusable Software" [1]. This book recommends opaque
pointers as a good design principle for designing C libraries, using the
term "interface" as the functions defined in *.h (header) files and
"implementation" as the corresponding *.c file which define the
interfaces.

The book says this about opaque pointers:

    ... clients can manipulate such pointers freely, but they can’t
    dereference them; that is, they can’t look at the innards of the
    structure pointed to by them. Only the implementation has that
    privilege. Opaque pointers hide representation details and help
    catch errors.

In our case, "struct trailer_info" is now hidden from clients, and the
ways in which this opaque pointer can be used is limited to the richness
of <trailer.h>. In other words, <trailer.h> exclusively controls exactly
how "trailer_info" pointers are to be used.

[1] Hanson, David R. "C Interfaces and Implementations: Techniques for
    Creating Reusable Software". Addison Wesley, 1997. p. 22

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c | 12 +++---
 trailer.c                    | 81 ++++++++++++++++++++++++++++--------
 trailer.h                    | 40 +++++-------------
 3 files changed, 80 insertions(+), 53 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 11f4ce9e4a2..6bf8cec005a 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -141,7 +141,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
 	struct strbuf trailer_block = STRBUF_INIT;
-	struct trailer_info info;
+	struct trailer_info *info;
 	FILE *outfile = stdout;
 
 	trailer_config_init();
@@ -151,13 +151,13 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	if (opts->in_place)
 		outfile = create_in_place_tempfile(file);
 
-	parse_trailers(opts, &info, sb.buf, &head);
+	info = parse_trailers(opts, sb.buf, &head);
 
 	/* Print the lines before the trailers */
 	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
+		fwrite(sb.buf, 1, trailer_block_start(info), outfile);
 
-	if (!opts->only_trailers && !info.blank_line_before_trailer)
+	if (!opts->only_trailers && !blank_line_before_trailer_block(info))
 		fprintf(outfile, "\n");
 
 
@@ -178,8 +178,8 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 
 	/* Print the lines after the trailers as is */
 	if (!opts->only_trailers)
-		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
-	trailer_info_release(&info);
+		fwrite(sb.buf + trailer_block_end(info), 1, sb.len - trailer_block_end(info), outfile);
+	trailer_info_release(info);
 
 	if (opts->in_place)
 		if (rename_tempfile(&trailers_tempfile, file))
diff --git a/trailer.c b/trailer.c
index 2cc4a910411..cc211dfeeae 100644
--- a/trailer.c
+++ b/trailer.c
@@ -11,6 +11,27 @@
  * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
  */
 
+struct trailer_info {
+	/*
+	 * True if there is a blank line before the location pointed to by
+	 * trailer_block_start.
+	 */
+	int blank_line_before_trailer;
+
+	/*
+	 * Offsets to the trailer block start and end positions in the input
+	 * string. If no trailer block is found, these are both set to the
+	 * "true" end of the input (find_end_of_log_message()).
+	 */
+	size_t trailer_block_start, trailer_block_end;
+
+	/*
+	 * Array of trailers found.
+	 */
+	char **trailers;
+	size_t trailer_nr;
+};
+
 struct conf_info {
 	char *name;
 	char *key;
@@ -953,20 +974,26 @@ static void unfold_value(struct strbuf *val)
 	strbuf_release(&out);
 }
 
+static struct trailer_info *trailer_info_new(void)
+{
+	struct trailer_info *info = xcalloc(1, sizeof(*info));
+	return info;
+}
+
 /*
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
  */
-void parse_trailers(const struct process_trailer_options *opts,
-		    struct trailer_info *info,
-		    const char *str,
-		    struct list_head *head)
+struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
+				    const char *str,
+				    struct list_head *head)
 {
+	struct trailer_info *info;
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
 	size_t i;
 
-	trailer_info_get(opts, str, info);
+	info = trailer_info_get(opts, str);
 
 	for (i = 0; i < info->trailer_nr; i++) {
 		int separator_pos;
@@ -990,6 +1017,8 @@ void parse_trailers(const struct process_trailer_options *opts,
 					 strbuf_detach(&val, NULL));
 		}
 	}
+
+	return info;
 }
 
 void free_trailers(struct list_head *trailers)
@@ -1001,10 +1030,25 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
-void trailer_info_get(const struct process_trailer_options *opts,
-		      const char *str,
-		      struct trailer_info *info)
+size_t trailer_block_start(struct trailer_info *info)
+{
+	return info->trailer_block_start;
+}
+
+size_t trailer_block_end(struct trailer_info *info)
+{
+	return info->trailer_block_end;
+}
+
+int blank_line_before_trailer_block(struct trailer_info *info)
+{
+	return info->blank_line_before_trailer;
+}
+
+struct trailer_info *trailer_info_get(const struct process_trailer_options *opts,
+				      const char *str)
 {
+	struct trailer_info *info = trailer_info_new();
 	size_t end_of_log_message = 0, trailer_block_start = 0;
 	struct strbuf **trailer_lines, **ptr;
 	char **trailer_strings = NULL;
@@ -1043,6 +1087,8 @@ void trailer_info_get(const struct process_trailer_options *opts,
 	info->trailer_block_end = end_of_log_message;
 	info->trailers = trailer_strings;
 	info->trailer_nr = nr;
+
+	return info;
 }
 
 void trailer_info_release(struct trailer_info *info)
@@ -1051,6 +1097,7 @@ void trailer_info_release(struct trailer_info *info)
 	for (i = 0; i < info->trailer_nr; i++)
 		free(info->trailers[i]);
 	free(info->trailers);
+	free(info);
 }
 
 void format_trailers(const struct process_trailer_options *opts,
@@ -1118,21 +1165,19 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 				 struct strbuf *out)
 {
 	LIST_HEAD(trailers);
-	struct trailer_info info;
-
-	parse_trailers(opts, &info, msg, &trailers);
+	struct trailer_info *info = parse_trailers(opts, msg, &trailers);
 
 	/* If we want the whole block untouched, we can take the fast path. */
 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
 	    !opts->separator && !opts->key_only && !opts->value_only &&
 	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info.trailer_block_start,
-			   info.trailer_block_end - info.trailer_block_start);
+		strbuf_add(out, msg + info->trailer_block_start,
+			   info->trailer_block_end - info->trailer_block_start);
 	} else
 		format_trailers(opts, &trailers, out);
 
 	free_trailers(&trailers);
-	trailer_info_release(&info);
+	trailer_info_release(info);
 }
 
 void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
@@ -1141,14 +1186,14 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 	strbuf_init(&iter->key, 0);
 	strbuf_init(&iter->val, 0);
 	opts.no_divider = 1;
-	trailer_info_get(&opts, msg, &iter->internal.info);
+	iter->internal.info = trailer_info_get(&opts, msg);
 	iter->internal.cur = 0;
 }
 
 int trailer_iterator_advance(struct trailer_iterator *iter)
 {
-	if (iter->internal.cur < iter->internal.info.trailer_nr) {
-		char *line = iter->internal.info.trailers[iter->internal.cur++];
+	if (iter->internal.cur < iter->internal.info->trailer_nr) {
+		char *line = iter->internal.info->trailers[iter->internal.cur++];
 		int separator_pos = find_separator(line, separators);
 
 		iter->raw = line;
@@ -1165,7 +1210,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 
 void trailer_iterator_release(struct trailer_iterator *iter)
 {
-	trailer_info_release(&iter->internal.info);
+	trailer_info_release(iter->internal.info);
 	strbuf_release(&iter->val);
 	strbuf_release(&iter->key);
 }
diff --git a/trailer.h b/trailer.h
index ebafa3657e4..a63e97a2663 100644
--- a/trailer.h
+++ b/trailer.h
@@ -4,6 +4,8 @@
 #include "list.h"
 #include "strbuf.h"
 
+struct trailer_info;
+
 enum trailer_where {
 	WHERE_DEFAULT,
 	WHERE_END,
@@ -29,27 +31,6 @@ int trailer_set_where(enum trailer_where *item, const char *value);
 int trailer_set_if_exists(enum trailer_if_exists *item, const char *value);
 int trailer_set_if_missing(enum trailer_if_missing *item, const char *value);
 
-struct trailer_info {
-	/*
-	 * True if there is a blank line before the location pointed to by
-	 * trailer_block_start.
-	 */
-	int blank_line_before_trailer;
-
-	/*
-	 * Offsets to the trailer block start and end positions in the input
-	 * string. If no trailer block is found, these are both set to the
-	 * "true" end of the input (find_end_of_log_message()).
-	 */
-	size_t trailer_block_start, trailer_block_end;
-
-	/*
-	 * Array of trailers found.
-	 */
-	char **trailers;
-	size_t trailer_nr;
-};
-
 /*
  * A list that represents newly-added trailers, such as those provided
  * with the --trailer command line option of git-interpret-trailers.
@@ -89,14 +70,15 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
-void parse_trailers(const struct process_trailer_options *,
-		    struct trailer_info *,
-		    const char *str,
-		    struct list_head *head);
+struct trailer_info *parse_trailers(const struct process_trailer_options *,
+				    const char *str,
+				    struct list_head *head);
+struct trailer_info *trailer_info_get(const struct process_trailer_options *,
+				      const char *str);
 
-void trailer_info_get(const struct process_trailer_options *,
-		      const char *str,
-		      struct trailer_info *);
+size_t trailer_block_start(struct trailer_info *);
+size_t trailer_block_end(struct trailer_info *);
+int blank_line_before_trailer_block(struct trailer_info *);
 
 void trailer_info_release(struct trailer_info *info);
 
@@ -138,7 +120,7 @@ struct trailer_iterator {
 
 	/* private */
 	struct {
-		struct trailer_info info;
+		struct trailer_info *info;
 		size_t cur;
 	} internal;
 };
-- 
gitgitgadget

[PATCH v4 20/28] trailer: retire trailer_info_get() from API

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Make it "static" to be file-scoped to trailer.c, because no one outside
of trailer.c uses it. Remove its declaration from <trailer.h>.

We have to also reposition it to be above parse_trailers(), which
depends on it.

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 92 +++++++++++++++++++++++++++----------------------------
 trailer.h |  2 --
 2 files changed, 46 insertions(+), 48 deletions(-)

diff --git a/trailer.c b/trailer.c
index cc211dfeeae..36e49ab7cf5 100644
--- a/trailer.c
+++ b/trailer.c
@@ -980,6 +980,52 @@ static struct trailer_info *trailer_info_new(void)
 	return info;
 }
 
+static struct trailer_info *trailer_info_get(const struct process_trailer_options *opts,
+					     const char *str)
+{
+	struct trailer_info *info = trailer_info_new();
+	size_t end_of_log_message = 0, trailer_block_start = 0;
+	struct strbuf **trailer_lines, **ptr;
+	char **trailer_strings = NULL;
+	size_t nr = 0, alloc = 0;
+	char **last = NULL;
+
+	trailer_config_init();
+
+	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
+	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
+
+	trailer_lines = strbuf_split_buf(str + trailer_block_start,
+					 end_of_log_message - trailer_block_start,
+					 '\n',
+					 0);
+	for (ptr = trailer_lines; *ptr; ptr++) {
+		if (last && isspace((*ptr)->buf[0])) {
+			struct strbuf sb = STRBUF_INIT;
+			strbuf_attach(&sb, *last, strlen(*last), strlen(*last));
+			strbuf_addbuf(&sb, *ptr);
+			*last = strbuf_detach(&sb, NULL);
+			continue;
+		}
+		ALLOC_GROW(trailer_strings, nr + 1, alloc);
+		trailer_strings[nr] = strbuf_detach(*ptr, NULL);
+		last = find_separator(trailer_strings[nr], separators) >= 1
+			? &trailer_strings[nr]
+			: NULL;
+		nr++;
+	}
+	strbuf_list_free(trailer_lines);
+
+	info->blank_line_before_trailer = ends_with_blank_line(str,
+							       trailer_block_start);
+	info->trailer_block_start = trailer_block_start;
+	info->trailer_block_end = end_of_log_message;
+	info->trailers = trailer_strings;
+	info->trailer_nr = nr;
+
+	return info;
+}
+
 /*
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
@@ -1045,52 +1091,6 @@ int blank_line_before_trailer_block(struct trailer_info *info)
 	return info->blank_line_before_trailer;
 }
 
-struct trailer_info *trailer_info_get(const struct process_trailer_options *opts,
-				      const char *str)
-{
-	struct trailer_info *info = trailer_info_new();
-	size_t end_of_log_message = 0, trailer_block_start = 0;
-	struct strbuf **trailer_lines, **ptr;
-	char **trailer_strings = NULL;
-	size_t nr = 0, alloc = 0;
-	char **last = NULL;
-
-	trailer_config_init();
-
-	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
-	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
-
-	trailer_lines = strbuf_split_buf(str + trailer_block_start,
-					 end_of_log_message - trailer_block_start,
-					 '\n',
-					 0);
-	for (ptr = trailer_lines; *ptr; ptr++) {
-		if (last && isspace((*ptr)->buf[0])) {
-			struct strbuf sb = STRBUF_INIT;
-			strbuf_attach(&sb, *last, strlen(*last), strlen(*last));
-			strbuf_addbuf(&sb, *ptr);
-			*last = strbuf_detach(&sb, NULL);
-			continue;
-		}
-		ALLOC_GROW(trailer_strings, nr + 1, alloc);
-		trailer_strings[nr] = strbuf_detach(*ptr, NULL);
-		last = find_separator(trailer_strings[nr], separators) >= 1
-			? &trailer_strings[nr]
-			: NULL;
-		nr++;
-	}
-	strbuf_list_free(trailer_lines);
-
-	info->blank_line_before_trailer = ends_with_blank_line(str,
-							       trailer_block_start);
-	info->trailer_block_start = trailer_block_start;
-	info->trailer_block_end = end_of_log_message;
-	info->trailers = trailer_strings;
-	info->trailer_nr = nr;
-
-	return info;
-}
-
 void trailer_info_release(struct trailer_info *info)
 {
 	size_t i;
diff --git a/trailer.h b/trailer.h
index a63e97a2663..1b7422fa2b0 100644
--- a/trailer.h
+++ b/trailer.h
@@ -73,8 +73,6 @@ void process_trailers_lists(struct list_head *head,
 struct trailer_info *parse_trailers(const struct process_trailer_options *,
 				    const char *str,
 				    struct list_head *head);
-struct trailer_info *trailer_info_get(const struct process_trailer_options *,
-				      const char *str);
 
 size_t trailer_block_start(struct trailer_info *);
 size_t trailer_block_end(struct trailer_info *);
-- 
gitgitgadget

[PATCH v4 21/28] trailer: spread usage of "trailer_block" language

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Deprecate the "trailer_info" struct name and replace it with
"trailer_block". The main reason is to help readability, because
"trailer_info" on the surface sounds like it's about a single trailer
when in reality it is a collection of contiguous lines, at least 25% of
which are trailers.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c | 25 +++++-----
 trailer.c                    | 97 ++++++++++++++++++------------------
 trailer.h                    | 18 +++----
 3 files changed, 71 insertions(+), 69 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 6bf8cec005a..f76841c5280 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -140,8 +140,8 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
-	struct strbuf trailer_block = STRBUF_INIT;
-	struct trailer_info *info;
+	struct strbuf tb = STRBUF_INIT;
+	struct trailer_block *trailer_block;
 	FILE *outfile = stdout;
 
 	trailer_config_init();
@@ -151,13 +151,13 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	if (opts->in_place)
 		outfile = create_in_place_tempfile(file);
 
-	info = parse_trailers(opts, sb.buf, &head);
+	trailer_block = parse_trailers(opts, sb.buf, &head);
 
-	/* Print the lines before the trailers */
+	/* Print the lines before the trailer block */
 	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, trailer_block_start(info), outfile);
+		fwrite(sb.buf, 1, trailer_block_start(trailer_block), outfile);
 
-	if (!opts->only_trailers && !blank_line_before_trailer_block(info))
+	if (!opts->only_trailers && !blank_line_before_trailer_block(trailer_block))
 		fprintf(outfile, "\n");
 
 
@@ -171,15 +171,16 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	}
 
 	/* Print trailer block. */
-	format_trailers(opts, &head, &trailer_block);
+	format_trailers(opts, &head, &tb);
 	free_trailers(&head);
-	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
-	strbuf_release(&trailer_block);
+	fwrite(tb.buf, 1, tb.len, outfile);
+	strbuf_release(&tb);
 
-	/* Print the lines after the trailers as is */
+	/* Print the lines after the trailer block as is. */
 	if (!opts->only_trailers)
-		fwrite(sb.buf + trailer_block_end(info), 1, sb.len - trailer_block_end(info), outfile);
-	trailer_info_release(info);
+		fwrite(sb.buf + trailer_block_end(trailer_block), 1,
+		       sb.len - trailer_block_end(trailer_block), outfile);
+	trailer_block_release(trailer_block);
 
 	if (opts->in_place)
 		if (rename_tempfile(&trailers_tempfile, file))
diff --git a/trailer.c b/trailer.c
index 36e49ab7cf5..49bf26e3211 100644
--- a/trailer.c
+++ b/trailer.c
@@ -11,19 +11,20 @@
  * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
  */
 
-struct trailer_info {
+struct trailer_block {
 	/*
 	 * True if there is a blank line before the location pointed to by
-	 * trailer_block_start.
+	 * "start".
 	 */
 	int blank_line_before_trailer;
 
 	/*
-	 * Offsets to the trailer block start and end positions in the input
-	 * string. If no trailer block is found, these are both set to the
-	 * "true" end of the input (find_end_of_log_message()).
+	 * The locations of the start and end positions of the trailer block
+	 * found, as offsets from the beginning of the source text from which
+	 * this trailer block was parsed. If no trailer block is found, these
+	 * are both set to 0.
 	 */
-	size_t trailer_block_start, trailer_block_end;
+	size_t start, end;
 
 	/*
 	 * Array of trailers found.
@@ -974,16 +975,16 @@ static void unfold_value(struct strbuf *val)
 	strbuf_release(&out);
 }
 
-static struct trailer_info *trailer_info_new(void)
+static struct trailer_block *trailer_block_new(void)
 {
-	struct trailer_info *info = xcalloc(1, sizeof(*info));
-	return info;
+	struct trailer_block *trailer_block = xcalloc(1, sizeof(*trailer_block));
+	return trailer_block;
 }
 
-static struct trailer_info *trailer_info_get(const struct process_trailer_options *opts,
-					     const char *str)
+static struct trailer_block *trailer_block_get(const struct process_trailer_options *opts,
+					       const char *str)
 {
-	struct trailer_info *info = trailer_info_new();
+	struct trailer_block *trailer_block = trailer_block_new();
 	size_t end_of_log_message = 0, trailer_block_start = 0;
 	struct strbuf **trailer_lines, **ptr;
 	char **trailer_strings = NULL;
@@ -1016,34 +1017,34 @@ static struct trailer_info *trailer_info_get(const struct process_trailer_option
 	}
 	strbuf_list_free(trailer_lines);
 
-	info->blank_line_before_trailer = ends_with_blank_line(str,
-							       trailer_block_start);
-	info->trailer_block_start = trailer_block_start;
-	info->trailer_block_end = end_of_log_message;
-	info->trailers = trailer_strings;
-	info->trailer_nr = nr;
+	trailer_block->blank_line_before_trailer = ends_with_blank_line(str,
+									trailer_block_start);
+	trailer_block->start = trailer_block_start;
+	trailer_block->end = end_of_log_message;
+	trailer_block->trailers = trailer_strings;
+	trailer_block->trailer_nr = nr;
 
-	return info;
+	return trailer_block;
 }
 
 /*
- * Parse trailers in "str", populating the trailer info and "head"
- * linked list structure.
+ * Parse trailers in "str", populating the trailer_block info and "head" linked
+ * list structure.
  */
-struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
-				    const char *str,
-				    struct list_head *head)
+struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
+				     const char *str,
+				     struct list_head *head)
 {
-	struct trailer_info *info;
+	struct trailer_block *trailer_block;
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
 	size_t i;
 
-	info = trailer_info_get(opts, str);
+	trailer_block = trailer_block_get(opts, str);
 
-	for (i = 0; i < info->trailer_nr; i++) {
+	for (i = 0; i < trailer_block->trailer_nr; i++) {
 		int separator_pos;
-		char *trailer = info->trailers[i];
+		char *trailer = trailer_block->trailers[i];
 		if (trailer[0] == comment_line_char)
 			continue;
 		separator_pos = find_separator(trailer, separators);
@@ -1064,7 +1065,7 @@ struct trailer_info *parse_trailers(const struct process_trailer_options *opts,
 		}
 	}
 
-	return info;
+	return trailer_block;
 }
 
 void free_trailers(struct list_head *trailers)
@@ -1076,28 +1077,28 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
-size_t trailer_block_start(struct trailer_info *info)
+size_t trailer_block_start(struct trailer_block *trailer_block)
 {
-	return info->trailer_block_start;
+	return trailer_block->start;
 }
 
-size_t trailer_block_end(struct trailer_info *info)
+size_t trailer_block_end(struct trailer_block *trailer_block)
 {
-	return info->trailer_block_end;
+	return trailer_block->end;
 }
 
-int blank_line_before_trailer_block(struct trailer_info *info)
+int blank_line_before_trailer_block(struct trailer_block *trailer_block)
 {
-	return info->blank_line_before_trailer;
+	return trailer_block->blank_line_before_trailer;
 }
 
-void trailer_info_release(struct trailer_info *info)
+void trailer_block_release(struct trailer_block *trailer_block)
 {
 	size_t i;
-	for (i = 0; i < info->trailer_nr; i++)
-		free(info->trailers[i]);
-	free(info->trailers);
-	free(info);
+	for (i = 0; i < trailer_block->trailer_nr; i++)
+		free(trailer_block->trailers[i]);
+	free(trailer_block->trailers);
+	free(trailer_block);
 }
 
 void format_trailers(const struct process_trailer_options *opts,
@@ -1165,19 +1166,19 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 				 struct strbuf *out)
 {
 	LIST_HEAD(trailers);
-	struct trailer_info *info = parse_trailers(opts, msg, &trailers);
+	struct trailer_block *trailer_block = parse_trailers(opts, msg, &trailers);
 
 	/* If we want the whole block untouched, we can take the fast path. */
 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
 	    !opts->separator && !opts->key_only && !opts->value_only &&
 	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info->trailer_block_start,
-			   info->trailer_block_end - info->trailer_block_start);
+		strbuf_add(out, msg + trailer_block->start,
+			   trailer_block->end - trailer_block->start);
 	} else
 		format_trailers(opts, &trailers, out);
 
 	free_trailers(&trailers);
-	trailer_info_release(info);
+	trailer_block_release(trailer_block);
 }
 
 void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
@@ -1186,14 +1187,14 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 	strbuf_init(&iter->key, 0);
 	strbuf_init(&iter->val, 0);
 	opts.no_divider = 1;
-	iter->internal.info = trailer_info_get(&opts, msg);
+	iter->internal.trailer_block = trailer_block_get(&opts, msg);
 	iter->internal.cur = 0;
 }
 
 int trailer_iterator_advance(struct trailer_iterator *iter)
 {
-	if (iter->internal.cur < iter->internal.info->trailer_nr) {
-		char *line = iter->internal.info->trailers[iter->internal.cur++];
+	if (iter->internal.cur < iter->internal.trailer_block->trailer_nr) {
+		char *line = iter->internal.trailer_block->trailers[iter->internal.cur++];
 		int separator_pos = find_separator(line, separators);
 
 		iter->raw = line;
@@ -1210,7 +1211,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 
 void trailer_iterator_release(struct trailer_iterator *iter)
 {
-	trailer_info_release(iter->internal.info);
+	trailer_block_release(iter->internal.trailer_block);
 	strbuf_release(&iter->val);
 	strbuf_release(&iter->key);
 }
diff --git a/trailer.h b/trailer.h
index 1b7422fa2b0..76e6d941a07 100644
--- a/trailer.h
+++ b/trailer.h
@@ -4,7 +4,7 @@
 #include "list.h"
 #include "strbuf.h"
 
-struct trailer_info;
+struct trailer_block;
 
 enum trailer_where {
 	WHERE_DEFAULT,
@@ -70,15 +70,15 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
-struct trailer_info *parse_trailers(const struct process_trailer_options *,
-				    const char *str,
-				    struct list_head *head);
+struct trailer_block *parse_trailers(const struct process_trailer_options *,
+				     const char *str,
+				     struct list_head *head);
 
-size_t trailer_block_start(struct trailer_info *);
-size_t trailer_block_end(struct trailer_info *);
-int blank_line_before_trailer_block(struct trailer_info *);
+size_t trailer_block_start(struct trailer_block *);
+size_t trailer_block_end(struct trailer_block *);
+int blank_line_before_trailer_block(struct trailer_block *);
 
-void trailer_info_release(struct trailer_info *info);
+void trailer_block_release(struct trailer_block *);
 
 void trailer_config_init(void);
 void format_trailers(const struct process_trailer_options *,
@@ -118,7 +118,7 @@ struct trailer_iterator {
 
 	/* private */
 	struct {
-		struct trailer_info *info;
+		struct trailer_block *trailer_block;
 		size_t cur;
 	} internal;
 };
-- 
gitgitgadget

[PATCH v4 22/28] trailer: prepare to delete "parse_trailers_from_command_line_args()"

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Expose more functions in the trailer.h API, in preparation for deleting

    parse_trailers_from_command_line_args()

from the trailers implementation, because it should not be concerned
with command line arguments (as they have nothing to do with trailers
themselves). Indeed, the interpret-trailers builtin is the only caller
of this function inside Git.

Rename add_arg_item() to trailer_add_arg_item() to expose it as an API
function. Rename new_trailers_clear() to free_new_trailers() because it
will be promoted into an API function; the API already has
free_trailers(), so using the "free_*" naming style will keep it
consistent. Also rename "conf_info" to "trailer_conf" for readability,
dropping the low-value "_info" suffix as we did earlier in this series
for "trailer_info" to "trailer_block".

Helped-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  6 +--
 trailer.c                    | 86 ++++++++++++++++++------------------
 trailer.h                    | 13 ++++++
 3 files changed, 58 insertions(+), 47 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index f76841c5280..f674b5f4b9e 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -45,7 +45,7 @@ static int option_parse_if_missing(const struct option *opt,
 	return trailer_set_if_missing(opt->value, arg);
 }
 
-static void new_trailers_clear(struct list_head *trailers)
+static void free_new_trailers(struct list_head *trailers)
 {
 	struct list_head *pos, *tmp;
 	struct new_trailer_item *item;
@@ -64,7 +64,7 @@ static int option_parse_trailer(const struct option *opt,
 	struct new_trailer_item *item;
 
 	if (unset) {
-		new_trailers_clear(trailers);
+		free_new_trailers(trailers);
 		return 0;
 	}
 
@@ -237,7 +237,7 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 		interpret_trailers(&opts, &trailers, NULL);
 	}
 
-	new_trailers_clear(&trailers);
+	free_new_trailers(&trailers);
 
 	return 0;
 }
diff --git a/trailer.c b/trailer.c
index 49bf26e3211..3b8f0ba103c 100644
--- a/trailer.c
+++ b/trailer.c
@@ -33,7 +33,7 @@ struct trailer_block {
 	size_t trailer_nr;
 };
 
-struct conf_info {
+struct trailer_conf {
 	char *name;
 	char *key;
 	char *command;
@@ -43,7 +43,7 @@ struct conf_info {
 	enum trailer_if_missing if_missing;
 };
 
-static struct conf_info default_conf_info;
+static struct trailer_conf default_trailer_conf;
 
 struct trailer_item {
 	struct list_head list;
@@ -59,7 +59,7 @@ struct arg_item {
 	struct list_head list;
 	char *token;
 	char *value;
-	struct conf_info conf;
+	struct trailer_conf conf;
 };
 
 static LIST_HEAD(conf_head);
@@ -210,7 +210,7 @@ static int check_if_different(struct trailer_item *in_tok,
 	return 1;
 }
 
-static char *apply_command(struct conf_info *conf, const char *arg)
+static char *apply_command(struct trailer_conf *conf, const char *arg)
 {
 	struct strbuf cmd = STRBUF_INIT;
 	struct strbuf buf = STRBUF_INIT;
@@ -424,7 +424,8 @@ int trailer_set_if_missing(enum trailer_if_missing *item, const char *value)
 	return 0;
 }
 
-static void duplicate_conf(struct conf_info *dst, const struct conf_info *src)
+void duplicate_trailer_conf(struct trailer_conf *dst,
+			    const struct trailer_conf *src)
 {
 	*dst = *src;
 	dst->name = xstrdup_or_null(src->name);
@@ -447,7 +448,7 @@ static struct arg_item *get_conf_item(const char *name)
 
 	/* Item does not already exists, create it */
 	CALLOC_ARRAY(item, 1);
-	duplicate_conf(&item->conf, &default_conf_info);
+	duplicate_trailer_conf(&item->conf, &default_trailer_conf);
 	item->conf.name = xstrdup(name);
 
 	list_add_tail(&item->list, &conf_head);
@@ -482,17 +483,17 @@ static int git_trailer_default_config(const char *conf_key, const char *value,
 	variable_name = strrchr(trailer_item, '.');
 	if (!variable_name) {
 		if (!strcmp(trailer_item, "where")) {
-			if (trailer_set_where(&default_conf_info.where,
+			if (trailer_set_where(&default_trailer_conf.where,
 					      value) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
 		} else if (!strcmp(trailer_item, "ifexists")) {
-			if (trailer_set_if_exists(&default_conf_info.if_exists,
+			if (trailer_set_if_exists(&default_trailer_conf.if_exists,
 						  value) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
 		} else if (!strcmp(trailer_item, "ifmissing")) {
-			if (trailer_set_if_missing(&default_conf_info.if_missing,
+			if (trailer_set_if_missing(&default_trailer_conf.if_missing,
 						   value) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
@@ -511,7 +512,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 {
 	const char *trailer_item, *variable_name;
 	struct arg_item *item;
-	struct conf_info *conf;
+	struct trailer_conf *conf;
 	char *name = NULL;
 	enum trailer_info_type type;
 	int i;
@@ -585,9 +586,9 @@ void trailer_config_init(void)
 		return;
 
 	/* Default config must be setup first */
-	default_conf_info.where = WHERE_END;
-	default_conf_info.if_exists = EXISTS_ADD_IF_DIFFERENT_NEIGHBOR;
-	default_conf_info.if_missing = MISSING_ADD;
+	default_trailer_conf.where = WHERE_END;
+	default_trailer_conf.if_exists = EXISTS_ADD_IF_DIFFERENT_NEIGHBOR;
+	default_trailer_conf.if_missing = MISSING_ADD;
 	git_config(git_trailer_default_config, NULL);
 	git_config(git_trailer_config, NULL);
 	configured = 1;
@@ -620,7 +621,7 @@ static int token_matches_item(const char *tok, struct arg_item *item, size_t tok
  * distinguished from the non-well-formed-line case (in which this function
  * returns -1) because some callers of this function need such a distinction.
  */
-static ssize_t find_separator(const char *line, const char *separators)
+ssize_t find_separator(const char *line, const char *separators)
 {
 	int whitespace_found = 0;
 	const char *c;
@@ -645,28 +646,28 @@ static ssize_t find_separator(const char *line, const char *separators)
  *
  * If separator_pos is -1, interpret the whole trailer as a token.
  */
-static void parse_trailer(struct strbuf *tok, struct strbuf *val,
-			 const struct conf_info **conf, const char *trailer,
-			 ssize_t separator_pos)
+void parse_trailer(const char *line, ssize_t separator_pos,
+		   struct strbuf *tok, struct strbuf *val,
+		   const struct trailer_conf **conf)
 {
 	struct arg_item *item;
 	size_t tok_len;
 	struct list_head *pos;
 
 	if (separator_pos != -1) {
-		strbuf_add(tok, trailer, separator_pos);
+		strbuf_add(tok, line, separator_pos);
 		strbuf_trim(tok);
-		strbuf_addstr(val, trailer + separator_pos + 1);
+		strbuf_addstr(val, line + separator_pos + 1);
 		strbuf_trim(val);
 	} else {
-		strbuf_addstr(tok, trailer);
+		strbuf_addstr(tok, line);
 		strbuf_trim(tok);
 	}
 
 	/* Lookup if the token matches something in the config */
 	tok_len = token_len_without_separator(tok->buf, tok->len);
 	if (conf)
-		*conf = &default_conf_info;
+		*conf = &default_trailer_conf;
 	list_for_each(pos, &conf_head) {
 		item = list_entry(pos, struct arg_item, list);
 		if (token_matches_item(tok->buf, item, tok_len)) {
@@ -690,14 +691,14 @@ static struct trailer_item *add_trailer_item(struct list_head *head, char *tok,
 	return new_item;
 }
 
-static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
-			 const struct conf_info *conf,
-			 const struct new_trailer_item *new_trailer_item)
+void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
+			  const struct trailer_conf *conf,
+			  const struct new_trailer_item *new_trailer_item)
 {
 	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
 	new_item->token = tok;
 	new_item->value = val;
-	duplicate_conf(&new_item->conf, conf);
+	duplicate_trailer_conf(&new_item->conf, conf);
 	if (new_trailer_item) {
 		if (new_trailer_item->where != WHERE_DEFAULT)
 			new_item->conf.where = new_trailer_item->where;
@@ -718,10 +719,10 @@ void parse_trailers_from_config(struct list_head *config_head)
 	list_for_each(pos, &conf_head) {
 		item = list_entry(pos, struct arg_item, list);
 		if (item->conf.command)
-			add_arg_item(config_head,
-				     xstrdup(token_from_item(item, NULL)),
-				     xstrdup(""),
-				     &item->conf, NULL);
+			trailer_add_arg_item(config_head,
+					     xstrdup(token_from_item(item, NULL)),
+					     xstrdup(""),
+					     &item->conf, NULL);
 	}
 }
 
@@ -730,7 +731,7 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
-	const struct conf_info *conf;
+	const struct trailer_conf *conf;
 	struct list_head *pos;
 
 	/*
@@ -753,12 +754,11 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 			      (int) sb.len, sb.buf);
 			strbuf_release(&sb);
 		} else {
-			parse_trailer(&tok, &val, &conf, tr->text,
-				      separator_pos);
-			add_arg_item(arg_head,
-				     strbuf_detach(&tok, NULL),
-				     strbuf_detach(&val, NULL),
-				     conf, tr);
+			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
+			trailer_add_arg_item(arg_head,
+					     strbuf_detach(&tok, NULL),
+					     strbuf_detach(&val, NULL),
+					     conf, tr);
 		}
 	}
 
@@ -1044,20 +1044,19 @@ struct trailer_block *parse_trailers(const struct process_trailer_options *opts,
 
 	for (i = 0; i < trailer_block->trailer_nr; i++) {
 		int separator_pos;
-		char *trailer = trailer_block->trailers[i];
-		if (trailer[0] == comment_line_char)
+		char *line = trailer_block->trailers[i];
+		if (line[0] == comment_line_char)
 			continue;
-		separator_pos = find_separator(trailer, separators);
+		separator_pos = find_separator(line, separators);
 		if (separator_pos >= 1) {
-			parse_trailer(&tok, &val, NULL, trailer,
-				      separator_pos);
+			parse_trailer(line, separator_pos, &tok, &val, NULL);
 			if (opts->unfold)
 				unfold_value(&val);
 			add_trailer_item(head,
 					 strbuf_detach(&tok, NULL),
 					 strbuf_detach(&val, NULL));
 		} else if (!opts->only_trailers) {
-			strbuf_addstr(&val, trailer);
+			strbuf_addstr(&val, line);
 			strbuf_strip_suffix(&val, "\n");
 			add_trailer_item(head,
 					 NULL,
@@ -1200,8 +1199,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 		iter->raw = line;
 		strbuf_reset(&iter->key);
 		strbuf_reset(&iter->val);
-		parse_trailer(&iter->key, &iter->val, NULL,
-			      line, separator_pos);
+		parse_trailer(line, separator_pos, &iter->key, &iter->val, NULL);
 		/* Always unfold values during iteration. */
 		unfold_value(&iter->val);
 		return 1;
diff --git a/trailer.h b/trailer.h
index 76e6d941a07..f80f8f7e63f 100644
--- a/trailer.h
+++ b/trailer.h
@@ -5,6 +5,7 @@
 #include "strbuf.h"
 
 struct trailer_block;
+struct trailer_conf;
 
 enum trailer_where {
 	WHERE_DEFAULT,
@@ -45,6 +46,12 @@ struct new_trailer_item {
 	enum trailer_if_missing if_missing;
 };
 
+void duplicate_trailer_conf(struct trailer_conf *dst,
+			    const struct trailer_conf *src);
+void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
+			  const struct trailer_conf *conf,
+			  const struct new_trailer_item *new_trailer_item);
+
 struct process_trailer_options {
 	int in_place;
 	int trim_empty;
@@ -70,6 +77,12 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
+ssize_t find_separator(const char *line, const char *separators);
+
+void parse_trailer(const char *line, ssize_t separator_pos,
+		   struct strbuf *tok, struct strbuf *val,
+		   const struct trailer_conf **);
+
 struct trailer_block *parse_trailers(const struct process_trailer_options *,
 				     const char *str,
 				     struct list_head *head);
-- 
gitgitgadget

[PATCH v4 23/28] trailer: add new helper functions to API

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is a preparatory refactor for deprecating "new_trailer_item" from
the API (which will let us deprecate
parse_trailers_from_command_line_args()).

Expose new helper functions from the API, because we'll be calling them
from interpret-trailers.c soon when we move
parse_trailers_from_command_line_args() there.

Move free_new_trailers() from the builtin to trailer.c because later on
we will adjust it to free arg_item structs, which are private to
trailer.c.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c | 12 ---------
 trailer.c                    | 49 ++++++++++++++++++++++++++++++++++++
 trailer.h                    |  8 ++++++
 3 files changed, 57 insertions(+), 12 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index f674b5f4b9e..9169c320921 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -45,18 +45,6 @@ static int option_parse_if_missing(const struct option *opt,
 	return trailer_set_if_missing(opt->value, arg);
 }
 
-static void free_new_trailers(struct list_head *trailers)
-{
-	struct list_head *pos, *tmp;
-	struct new_trailer_item *item;
-
-	list_for_each_safe(pos, tmp, trailers) {
-		item = list_entry(pos, struct new_trailer_item, list);
-		list_del(pos);
-		free(item);
-	}
-}
-
 static int option_parse_trailer(const struct option *opt,
 				   const char *arg, int unset)
 {
diff --git a/trailer.c b/trailer.c
index 3b8f0ba103c..9b8cb94c021 100644
--- a/trailer.c
+++ b/trailer.c
@@ -66,6 +66,11 @@ static LIST_HEAD(conf_head);
 
 static char *separators = ":";
 
+const char *trailer_default_separators(void)
+{
+	return separators;
+}
+
 static int configured;
 
 #define TRAILER_ARG_STRING "$ARG"
@@ -424,6 +429,29 @@ int trailer_set_if_missing(enum trailer_if_missing *item, const char *value)
 	return 0;
 }
 
+void trailer_set_conf_where(enum trailer_where where,
+			    struct trailer_conf *conf)
+{
+	conf->where = where;
+}
+
+void trailer_set_conf_if_exists(enum trailer_if_exists if_exists,
+				struct trailer_conf *conf)
+{
+	conf->if_exists = if_exists;
+}
+
+void trailer_set_conf_if_missing(enum trailer_if_missing if_missing,
+				 struct trailer_conf *conf)
+{
+	conf->if_missing = if_missing;
+}
+
+struct trailer_conf *new_trailer_conf(void)
+{
+	 return xcalloc(1, sizeof(struct trailer_conf));
+}
+
 void duplicate_trailer_conf(struct trailer_conf *dst,
 			    const struct trailer_conf *src)
 {
@@ -434,6 +462,15 @@ void duplicate_trailer_conf(struct trailer_conf *dst,
 	dst->cmd = xstrdup_or_null(src->cmd);
 }
 
+void free_trailer_conf(struct trailer_conf *conf)
+{
+	free(conf->name);
+	free(conf->key);
+	free(conf->command);
+	free(conf->cmd);
+	free(conf);
+}
+
 static struct arg_item *get_conf_item(const char *name)
 {
 	struct list_head *pos;
@@ -1076,6 +1113,18 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
+void free_new_trailers(struct list_head *trailers)
+{
+	struct list_head *pos, *tmp;
+	struct new_trailer_item *item;
+
+	list_for_each_safe(pos, tmp, trailers) {
+		item = list_entry(pos, struct new_trailer_item, list);
+		list_del(pos);
+		free(item);
+	}
+}
+
 size_t trailer_block_start(struct trailer_block *trailer_block)
 {
 	return trailer_block->start;
diff --git a/trailer.h b/trailer.h
index f80f8f7e63f..a2569c10451 100644
--- a/trailer.h
+++ b/trailer.h
@@ -46,8 +46,14 @@ struct new_trailer_item {
 	enum trailer_if_missing if_missing;
 };
 
+void trailer_set_conf_where(enum trailer_where, struct trailer_conf *);
+void trailer_set_conf_if_exists(enum trailer_if_exists, struct trailer_conf *);
+void trailer_set_conf_if_missing(enum trailer_if_missing, struct trailer_conf *);
+
+struct trailer_conf *new_trailer_conf(void);
 void duplicate_trailer_conf(struct trailer_conf *dst,
 			    const struct trailer_conf *src);
+const char *trailer_default_separators(void);
 void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
 			  const struct trailer_conf *conf,
 			  const struct new_trailer_item *new_trailer_item);
@@ -98,6 +104,8 @@ void format_trailers(const struct process_trailer_options *,
 		     struct list_head *trailers,
 		     struct strbuf *out);
 void free_trailers(struct list_head *);
+void free_new_trailers(struct list_head *);
+void free_trailer_conf(struct trailer_conf *);
 
 /*
  * Convenience function to format the trailers from the commit msg "msg" into
-- 
gitgitgadget

[PATCH v4 24/28] trailer_add_arg_item(): drop new_trailer_item usage

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is a preparatory refactor for deprecating "new_trailer_item" from
the API.

Instead of preserving the where/if_exists/if_missing information inside
the new_trailer_item struct in parse_trailers_from_command_line_args()
(only to look inside it again later on in trailer_add_arg_item()), pass
this information directly as a trailer_conf which trailer_add_arg_item()
already knows how to handle. This reduces the number of parameters we
have to pass to trailer_add_arg_item() without any behavioral change.

In the next patch we'll be able to delete "new_trailer_item" altogether.

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 29 +++++++++++++++++------------
 trailer.h |  3 +--
 2 files changed, 18 insertions(+), 14 deletions(-)

diff --git a/trailer.c b/trailer.c
index 9b8cb94c021..6ab5cf7e5d7 100644
--- a/trailer.c
+++ b/trailer.c
@@ -729,21 +729,12 @@ static struct trailer_item *add_trailer_item(struct list_head *head, char *tok,
 }
 
 void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
-			  const struct trailer_conf *conf,
-			  const struct new_trailer_item *new_trailer_item)
+			  const struct trailer_conf *conf)
 {
 	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
 	new_item->token = tok;
 	new_item->value = val;
 	duplicate_trailer_conf(&new_item->conf, conf);
-	if (new_trailer_item) {
-		if (new_trailer_item->where != WHERE_DEFAULT)
-			new_item->conf.where = new_trailer_item->where;
-		if (new_trailer_item->if_exists != EXISTS_DEFAULT)
-			new_item->conf.if_exists = new_trailer_item->if_exists;
-		if (new_trailer_item->if_missing != MISSING_DEFAULT)
-			new_item->conf.if_missing = new_trailer_item->if_missing;
-	}
 	list_add_tail(&new_item->list, arg_head);
 }
 
@@ -759,7 +750,7 @@ void parse_trailers_from_config(struct list_head *config_head)
 			trailer_add_arg_item(config_head,
 					     xstrdup(token_from_item(item, NULL)),
 					     xstrdup(""),
-					     &item->conf, NULL);
+					     &item->conf);
 	}
 }
 
@@ -791,11 +782,25 @@ void parse_trailers_from_command_line_args(struct list_head *arg_head,
 			      (int) sb.len, sb.buf);
 			strbuf_release(&sb);
 		} else {
+			struct trailer_conf *conf_current = new_trailer_conf();
 			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
+			duplicate_trailer_conf(conf_current, conf);
+
+			/*
+			 * Override conf_current with settings specified via CLI flags.
+			 */
+			if (tr->where != WHERE_DEFAULT)
+				trailer_set_conf_where(tr->where, conf_current);
+			if (tr->if_exists != EXISTS_DEFAULT)
+				trailer_set_conf_if_exists(tr->if_exists, conf_current);
+			if (tr->if_missing != MISSING_DEFAULT)
+				trailer_set_conf_if_missing(tr->if_missing, conf_current);
+
 			trailer_add_arg_item(arg_head,
 					     strbuf_detach(&tok, NULL),
 					     strbuf_detach(&val, NULL),
-					     conf, tr);
+					     conf_current);
+			free_trailer_conf(conf_current);
 		}
 	}
 
diff --git a/trailer.h b/trailer.h
index a2569c10451..32fc93beb33 100644
--- a/trailer.h
+++ b/trailer.h
@@ -55,8 +55,7 @@ void duplicate_trailer_conf(struct trailer_conf *dst,
 			    const struct trailer_conf *src);
 const char *trailer_default_separators(void);
 void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
-			  const struct trailer_conf *conf,
-			  const struct new_trailer_item *new_trailer_item);
+			  const struct trailer_conf *conf);
 
 struct process_trailer_options {
 	int in_place;
-- 
gitgitgadget

[PATCH v4 25/28] trailer: deprecate "new_trailer_item" struct from API

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Previously, the "new_trailer_item" struct served only one purpose --- to
capture the unparsed raw string <RAW> in "--trailer <RAW>", as well as
the current state of the "where", "if_exists", and "if_missing" global
variables at the time that the "--trailer <RAW>" CLI argument was
encountered.

In addition, the previous CLI argument handling behavior was to capture
the <RAW> string in all "--trailer <RAW>" arguments and to collect
them (via option_parse_trailer()) into the "new_trailer_head" list. We
would then iterate over this list again in
parse_trailers_from_command_line_args() and convert these
"new_trailer_item" objects into "arg_item" objects.

Skip this intermediate storage of "new_trailer_item" objects in favor of
just storing "arg_item" objects. Remove the looping behavior of
parse_trailers_from_command_line_args() so that it parses a single
"--trailer ..." argument at a time. Rename it to
parse_trailer_from_command_line_arg() to reflect this new behavior of
only looking at one string (not multiple strings) at a time. Make
option_parse_trailer() call parse_trailer_from_command_line_arg() so
that the CLI arguments it sees are parsed immediately without the need
for intermediate storage.

Delete "new_trailer_item", because we don't need it any more.

In the next patch we will retire parse_trailer_from_command_line_arg()
as well, combining it with option_parse_trailer().

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c | 29 +++++--------
 trailer.c                    | 81 +++++++++++++++++-------------------
 trailer.h                    | 21 +++-------
 3 files changed, 54 insertions(+), 77 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 9169c320921..943be5b360e 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -49,7 +49,6 @@ static int option_parse_trailer(const struct option *opt,
 				   const char *arg, int unset)
 {
 	struct list_head *trailers = opt->value;
-	struct new_trailer_item *item;
 
 	if (unset) {
 		free_new_trailers(trailers);
@@ -59,12 +58,8 @@ static int option_parse_trailer(const struct option *opt,
 	if (!arg)
 		return -1;
 
-	item = xmalloc(sizeof(*item));
-	item->text = arg;
-	item->where = where;
-	item->if_exists = if_exists;
-	item->if_missing = if_missing;
-	list_add_tail(&item->list, trailers);
+	parse_trailer_from_command_line_arg(arg, where, if_exists, if_missing, trailers);
+
 	return 0;
 }
 
@@ -132,8 +127,6 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	struct trailer_block *trailer_block;
 	FILE *outfile = stdout;
 
-	trailer_config_init();
-
 	read_input_file(&sb, file);
 
 	if (opts->in_place)
@@ -148,15 +141,8 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	if (!opts->only_trailers && !blank_line_before_trailer_block(trailer_block))
 		fprintf(outfile, "\n");
 
-
-	if (!opts->only_input) {
-		LIST_HEAD(config_head);
-		LIST_HEAD(arg_head);
-		parse_trailers_from_config(&config_head);
-		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
-		list_splice(&config_head, &arg_head);
-		process_trailers_lists(&head, &arg_head);
-	}
+	if (!opts->only_input)
+		process_trailers_lists(&head, new_trailer_head);
 
 	/* Print trailer block. */
 	format_trailers(opts, &head, &tb);
@@ -180,6 +166,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
+	LIST_HEAD(configured_trailers);
 	LIST_HEAD(trailers);
 
 	struct option options[] = {
@@ -205,6 +192,10 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 	};
 
 	git_config(git_default_config, NULL);
+	trailer_config_init();
+
+	if (!opts.only_input)
+		parse_trailers_from_config(&configured_trailers);
 
 	argc = parse_options(argc, argv, prefix, options,
 			     git_interpret_trailers_usage, 0);
@@ -215,6 +206,8 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 			git_interpret_trailers_usage,
 			options);
 
+	list_splice(&configured_trailers, &trailers);
+
 	if (argc) {
 		int i;
 		for (i = 0; i < argc; i++)
diff --git a/trailer.c b/trailer.c
index 6ab5cf7e5d7..0893175553a 100644
--- a/trailer.c
+++ b/trailer.c
@@ -754,57 +754,54 @@ void parse_trailers_from_config(struct list_head *config_head)
 	}
 }
 
-void parse_trailers_from_command_line_args(struct list_head *arg_head,
-					   struct list_head *new_trailer_head)
+void parse_trailer_from_command_line_arg(const char *line,
+					 enum trailer_where where,
+					 enum trailer_if_exists if_exists,
+					 enum trailer_if_missing if_missing,
+					 struct list_head *arg_head)
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
 	const struct trailer_conf *conf;
-	struct list_head *pos;
 
 	/*
 	 * In command-line arguments, '=' is accepted (in addition to the
 	 * separators that are defined).
 	 */
-	char *cl_separators = xstrfmt("=%s", separators);
+	char *cl_separators = xstrfmt("=%s", trailer_default_separators());
 
-	/* Add an arg item for each trailer on the command line */
-	list_for_each(pos, new_trailer_head) {
-		struct new_trailer_item *tr =
-			list_entry(pos, struct new_trailer_item, list);
-		ssize_t separator_pos = find_separator(tr->text, cl_separators);
+	/* Add an arg item for a trailer from the command line */
+	ssize_t separator_pos = find_separator(line, cl_separators);
+	free(cl_separators);
 
-		if (separator_pos == 0) {
-			struct strbuf sb = STRBUF_INIT;
-			strbuf_addstr(&sb, tr->text);
-			strbuf_trim(&sb);
-			error(_("empty trailer token in trailer '%.*s'"),
-			      (int) sb.len, sb.buf);
-			strbuf_release(&sb);
-		} else {
-			struct trailer_conf *conf_current = new_trailer_conf();
-			parse_trailer(tr->text, separator_pos, &tok, &val, &conf);
-			duplicate_trailer_conf(conf_current, conf);
+	if (separator_pos == 0) {
+		struct strbuf sb = STRBUF_INIT;
+		strbuf_addstr(&sb, line);
+		strbuf_trim(&sb);
+		error(_("empty trailer token in trailer '%.*s'"),
+		      (int) sb.len, sb.buf);
+		strbuf_release(&sb);
+	} else {
+		struct trailer_conf *conf_current = new_trailer_conf();
+		parse_trailer(line, separator_pos, &tok, &val, &conf);
+		duplicate_trailer_conf(conf_current, conf);
 
-			/*
-			 * Override conf_current with settings specified via CLI flags.
-			 */
-			if (tr->where != WHERE_DEFAULT)
-				trailer_set_conf_where(tr->where, conf_current);
-			if (tr->if_exists != EXISTS_DEFAULT)
-				trailer_set_conf_if_exists(tr->if_exists, conf_current);
-			if (tr->if_missing != MISSING_DEFAULT)
-				trailer_set_conf_if_missing(tr->if_missing, conf_current);
-
-			trailer_add_arg_item(arg_head,
-					     strbuf_detach(&tok, NULL),
-					     strbuf_detach(&val, NULL),
-					     conf_current);
-			free_trailer_conf(conf_current);
-		}
+		/*
+		 * Override conf_current with settings specified via CLI flags.
+		 */
+		if (where != WHERE_DEFAULT)
+			trailer_set_conf_where(where, conf_current);
+		if (if_exists != EXISTS_DEFAULT)
+			trailer_set_conf_if_exists(if_exists, conf_current);
+		if (if_missing != MISSING_DEFAULT)
+			trailer_set_conf_if_missing(if_missing, conf_current);
+
+		trailer_add_arg_item(arg_head,
+				     strbuf_detach(&tok, NULL),
+				     strbuf_detach(&val, NULL),
+				     conf_current);
+		free_trailer_conf(conf_current);
 	}
-
-	free(cl_separators);
 }
 
 static const char *next_line(const char *str)
@@ -1120,13 +1117,11 @@ void free_trailers(struct list_head *trailers)
 
 void free_new_trailers(struct list_head *trailers)
 {
-	struct list_head *pos, *tmp;
-	struct new_trailer_item *item;
+	struct list_head *pos, *p;
 
-	list_for_each_safe(pos, tmp, trailers) {
-		item = list_entry(pos, struct new_trailer_item, list);
+	list_for_each_safe(pos, p, trailers) {
 		list_del(pos);
-		free(item);
+		free_arg_item(list_entry(pos, struct arg_item, list));
 	}
 }
 
diff --git a/trailer.h b/trailer.h
index 32fc93beb33..2848a0d086c 100644
--- a/trailer.h
+++ b/trailer.h
@@ -32,20 +32,6 @@ int trailer_set_where(enum trailer_where *item, const char *value);
 int trailer_set_if_exists(enum trailer_if_exists *item, const char *value);
 int trailer_set_if_missing(enum trailer_if_missing *item, const char *value);
 
-/*
- * A list that represents newly-added trailers, such as those provided
- * with the --trailer command line option of git-interpret-trailers.
- */
-struct new_trailer_item {
-	struct list_head list;
-
-	const char *text;
-
-	enum trailer_where where;
-	enum trailer_if_exists if_exists;
-	enum trailer_if_missing if_missing;
-};
-
 void trailer_set_conf_where(enum trailer_where, struct trailer_conf *);
 void trailer_set_conf_if_exists(enum trailer_if_exists, struct trailer_conf *);
 void trailer_set_conf_if_missing(enum trailer_if_missing, struct trailer_conf *);
@@ -76,8 +62,11 @@ struct process_trailer_options {
 
 void parse_trailers_from_config(struct list_head *config_head);
 
-void parse_trailers_from_command_line_args(struct list_head *arg_head,
-					   struct list_head *new_trailer_head);
+void parse_trailer_from_command_line_arg(const char *line,
+					 enum trailer_where where,
+					 enum trailer_if_exists if_exists,
+					 enum trailer_if_missing if_missing,
+					 struct list_head *arg_head);
 
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
-- 
gitgitgadget

[PATCH v4 26/28] trailer: unify "--trailer ..." arg handling

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Move the logic of parse_trailer_from_command_line_arg() into
option_parse_trailer(), because that is the only caller and there's no
benefit in keeping these two separate.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c | 42 +++++++++++++++++++++++++++++-
 trailer.c                    | 50 ------------------------------------
 trailer.h                    |  6 -----
 3 files changed, 41 insertions(+), 57 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 943be5b360e..9657b0d067c 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -49,6 +49,11 @@ static int option_parse_trailer(const struct option *opt,
 				   const char *arg, int unset)
 {
 	struct list_head *trailers = opt->value;
+	struct strbuf tok = STRBUF_INIT;
+	struct strbuf val = STRBUF_INIT;
+	const struct trailer_conf *conf;
+	ssize_t separator_pos;
+	static char *cl_separators;
 
 	if (unset) {
 		free_new_trailers(trailers);
@@ -58,7 +63,42 @@ static int option_parse_trailer(const struct option *opt,
 	if (!arg)
 		return -1;
 
-	parse_trailer_from_command_line_arg(arg, where, if_exists, if_missing, trailers);
+	/*
+	 * In command-line arguments, '=' is accepted (in addition to the
+	 * separators that are defined).
+	 */
+	cl_separators = xstrfmt("=%s", trailer_default_separators());
+	separator_pos = find_separator(arg, cl_separators);
+	free(cl_separators);
+
+	if (separator_pos == 0) {
+		struct strbuf sb = STRBUF_INIT;
+		strbuf_addstr(&sb, arg);
+		strbuf_trim(&sb);
+		error(_("empty trailer token in trailer '%.*s'"),
+			(int) sb.len, sb.buf);
+		strbuf_release(&sb);
+	} else {
+		struct trailer_conf *conf_current = new_trailer_conf();
+		parse_trailer(arg, separator_pos, &tok, &val, &conf);
+		duplicate_trailer_conf(conf_current, conf);
+
+		/*
+		 * Override conf_current with settings specified via CLI flags.
+		 */
+		if (where != WHERE_DEFAULT)
+			trailer_set_conf_where(where, conf_current);
+		if (if_exists != EXISTS_DEFAULT)
+			trailer_set_conf_if_exists(if_exists, conf_current);
+		if (if_missing != MISSING_DEFAULT)
+			trailer_set_conf_if_missing(if_missing, conf_current);
+
+		trailer_add_arg_item(trailers,
+				     strbuf_detach(&tok, NULL),
+				     strbuf_detach(&val, NULL),
+				     conf_current);
+		free_trailer_conf(conf_current);
+	}
 
 	return 0;
 }
diff --git a/trailer.c b/trailer.c
index 0893175553a..b0b067ab12c 100644
--- a/trailer.c
+++ b/trailer.c
@@ -754,56 +754,6 @@ void parse_trailers_from_config(struct list_head *config_head)
 	}
 }
 
-void parse_trailer_from_command_line_arg(const char *line,
-					 enum trailer_where where,
-					 enum trailer_if_exists if_exists,
-					 enum trailer_if_missing if_missing,
-					 struct list_head *arg_head)
-{
-	struct strbuf tok = STRBUF_INIT;
-	struct strbuf val = STRBUF_INIT;
-	const struct trailer_conf *conf;
-
-	/*
-	 * In command-line arguments, '=' is accepted (in addition to the
-	 * separators that are defined).
-	 */
-	char *cl_separators = xstrfmt("=%s", trailer_default_separators());
-
-	/* Add an arg item for a trailer from the command line */
-	ssize_t separator_pos = find_separator(line, cl_separators);
-	free(cl_separators);
-
-	if (separator_pos == 0) {
-		struct strbuf sb = STRBUF_INIT;
-		strbuf_addstr(&sb, line);
-		strbuf_trim(&sb);
-		error(_("empty trailer token in trailer '%.*s'"),
-		      (int) sb.len, sb.buf);
-		strbuf_release(&sb);
-	} else {
-		struct trailer_conf *conf_current = new_trailer_conf();
-		parse_trailer(line, separator_pos, &tok, &val, &conf);
-		duplicate_trailer_conf(conf_current, conf);
-
-		/*
-		 * Override conf_current with settings specified via CLI flags.
-		 */
-		if (where != WHERE_DEFAULT)
-			trailer_set_conf_where(where, conf_current);
-		if (if_exists != EXISTS_DEFAULT)
-			trailer_set_conf_if_exists(if_exists, conf_current);
-		if (if_missing != MISSING_DEFAULT)
-			trailer_set_conf_if_missing(if_missing, conf_current);
-
-		trailer_add_arg_item(arg_head,
-				     strbuf_detach(&tok, NULL),
-				     strbuf_detach(&val, NULL),
-				     conf_current);
-		free_trailer_conf(conf_current);
-	}
-}
-
 static const char *next_line(const char *str)
 {
 	const char *nl = strchrnul(str, '\n');
diff --git a/trailer.h b/trailer.h
index 2848a0d086c..af55032625d 100644
--- a/trailer.h
+++ b/trailer.h
@@ -62,12 +62,6 @@ struct process_trailer_options {
 
 void parse_trailers_from_config(struct list_head *config_head);
 
-void parse_trailer_from_command_line_arg(const char *line,
-					 enum trailer_where where,
-					 enum trailer_if_exists if_exists,
-					 enum trailer_if_missing if_missing,
-					 struct list_head *arg_head);
-
 void process_trailers_lists(struct list_head *head,
 			    struct list_head *arg_head);
 
-- 
gitgitgadget

[PATCH v4 27/28] trailer_set_*(): put out parameter at the end

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

The new trailer_config_set_*() functions which were introduced a few
patches ago put the out parameter (the variable being mutated) at the
end of the parameter list.

Put the out parameter at the end for these functions for these existing
trailer_set_*() functions for consistency. This also avoids confusion
for API users because otherwise these two sets of functions look rather
similar in <trailer.h> even though they have completely different out
parameters.

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  6 +++---
 trailer.c                    | 24 ++++++++++++------------
 trailer.h                    |  6 +++---
 3 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 9657b0d067c..d0c09d1d73b 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -28,21 +28,21 @@ static int option_parse_where(const struct option *opt,
 			      const char *arg, int unset UNUSED)
 {
 	/* unset implies NULL arg, which is handled in our helper */
-	return trailer_set_where(opt->value, arg);
+	return trailer_set_where(arg, opt->value);
 }
 
 static int option_parse_if_exists(const struct option *opt,
 				  const char *arg, int unset UNUSED)
 {
 	/* unset implies NULL arg, which is handled in our helper */
-	return trailer_set_if_exists(opt->value, arg);
+	return trailer_set_if_exists(arg, opt->value);
 }
 
 static int option_parse_if_missing(const struct option *opt,
 				   const char *arg, int unset UNUSED)
 {
 	/* unset implies NULL arg, which is handled in our helper */
-	return trailer_set_if_missing(opt->value, arg);
+	return trailer_set_if_missing(arg, opt->value);
 }
 
 static int option_parse_trailer(const struct option *opt,
diff --git a/trailer.c b/trailer.c
index b0b067ab12c..7b0bdfcb27e 100644
--- a/trailer.c
+++ b/trailer.c
@@ -380,7 +380,7 @@ void process_trailers_lists(struct list_head *head,
 	}
 }
 
-int trailer_set_where(enum trailer_where *item, const char *value)
+int trailer_set_where(const char *value, enum trailer_where *item)
 {
 	if (!value)
 		*item = WHERE_DEFAULT;
@@ -397,7 +397,7 @@ int trailer_set_where(enum trailer_where *item, const char *value)
 	return 0;
 }
 
-int trailer_set_if_exists(enum trailer_if_exists *item, const char *value)
+int trailer_set_if_exists(const char *value, enum trailer_if_exists *item)
 {
 	if (!value)
 		*item = EXISTS_DEFAULT;
@@ -416,7 +416,7 @@ int trailer_set_if_exists(enum trailer_if_exists *item, const char *value)
 	return 0;
 }
 
-int trailer_set_if_missing(enum trailer_if_missing *item, const char *value)
+int trailer_set_if_missing(const char *value, enum trailer_if_missing *item)
 {
 	if (!value)
 		*item = MISSING_DEFAULT;
@@ -520,18 +520,18 @@ static int git_trailer_default_config(const char *conf_key, const char *value,
 	variable_name = strrchr(trailer_item, '.');
 	if (!variable_name) {
 		if (!strcmp(trailer_item, "where")) {
-			if (trailer_set_where(&default_trailer_conf.where,
-					      value) < 0)
+			if (trailer_set_where(value,
+					      &default_trailer_conf.where) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
 		} else if (!strcmp(trailer_item, "ifexists")) {
-			if (trailer_set_if_exists(&default_trailer_conf.if_exists,
-						  value) < 0)
+			if (trailer_set_if_exists(value,
+						  &default_trailer_conf.if_exists) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
 		} else if (!strcmp(trailer_item, "ifmissing")) {
-			if (trailer_set_if_missing(&default_trailer_conf.if_missing,
-						   value) < 0)
+			if (trailer_set_if_missing(value,
+						   &default_trailer_conf.if_missing) < 0)
 				warning(_("unknown value '%s' for key '%s'"),
 					value, conf_key);
 		} else if (!strcmp(trailer_item, "separators")) {
@@ -600,15 +600,15 @@ static int git_trailer_config(const char *conf_key, const char *value,
 		conf->cmd = xstrdup(value);
 		break;
 	case TRAILER_WHERE:
-		if (trailer_set_where(&conf->where, value))
+		if (trailer_set_where(value, &conf->where))
 			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
 		break;
 	case TRAILER_IF_EXISTS:
-		if (trailer_set_if_exists(&conf->if_exists, value))
+		if (trailer_set_if_exists(value, &conf->if_exists))
 			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
 		break;
 	case TRAILER_IF_MISSING:
-		if (trailer_set_if_missing(&conf->if_missing, value))
+		if (trailer_set_if_missing(value, &conf->if_missing))
 			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
 		break;
 	default:
diff --git a/trailer.h b/trailer.h
index af55032625d..4193bedbae4 100644
--- a/trailer.h
+++ b/trailer.h
@@ -28,9 +28,9 @@ enum trailer_if_missing {
 	MISSING_DO_NOTHING
 };
 
-int trailer_set_where(enum trailer_where *item, const char *value);
-int trailer_set_if_exists(enum trailer_if_exists *item, const char *value);
-int trailer_set_if_missing(enum trailer_if_missing *item, const char *value);
+int trailer_set_where(const char *, enum trailer_where *);
+int trailer_set_if_exists(const char *, enum trailer_if_exists *);
+int trailer_set_if_missing(const char *, enum trailer_if_missing *);
 
 void trailer_set_conf_where(enum trailer_where, struct trailer_conf *);
 void trailer_set_conf_if_exists(enum trailer_if_exists, struct trailer_conf *);
-- 
gitgitgadget

[PATCH v4 28/28] trailer: introduce "template" term for readability

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

The term "arg_item" is ambiguous because we use it to hold data for

  (1) trailers specified as command line arguments (in
      builtin/interpret-trailers.c), and

  (2) trailers specified in configuration,

and these are both used to ultimately insert new trailers (based
on the contents of arg_item, acting as a kind of template) into some
other set of existing trailers (such as those found in a trailer block
inside a log message) that have already been parsed.

Rename "arg_item" to "trailer_template". This necessitates further
renames to make the functions that act on these templates match the data
structures (parameters) they act on:

  - [*] add_arg_to_input_list()      to apply_template_to_trailers()
  - [*] apply_arg_if_exists()        to maybe_add_if_exists()
  - [*] apply_arg_if_missing()       to maybe_add_if_missing()
  -     apply_command()              to run_command_from_template()
  - [*] apply_item_command()         to populate_template_value()
  -     free_arg_item()              to free_template() (non-API function)
  -     free_new_trailers()          to free_trailer_templates() (API function)
  -     get_conf_item()              to get_or_add_template_by()
  -     option_parse_trailer()       to option_parse_trailer_template()
  -     parse_trailers_from_config() to parse_trailer_templates_from_config()
  - [*] process_trailers_lists()     to apply_trailer_templates()
  -     token_from_item()            to token_from_template()
  -     token_matches_item           to token_matches_template
  - [*] trailer_add_arg_item()       to add_trailer_template()
  -     trailer_from_arg()           to trailer_from()
  - [*] check_if_different()         (reorder parameters only)
  - [*] find_same_and_apply_arg()    (reorder parameters only)

Reorder parameters to prefer input parameters toward the beginning and
out parameters at the end; these functions have been marked with an
asterisk ([*]).

This removes the "arg" terminology (standing for "CLI arguments") from
the trailer implementation, which makes sense because trailers
themselves have nothing to do with CLI argument handling.

Also note that these renames expose the previously liberal use of
"trailer" to mean both trailers we read from the input text (trailer
block) and trailer templates that are defined as CLI args or
configurations. Some functions implied a single action when they could
do two different things, so introduce words like "maybe" and "or" to
make their behavior more explicit.

In summary this patch renames and reorders parameters for readability,
without any behavioral change. We don't rename
find_same_and_apply_arg(), because it will be refactored soon.

For parse_trailers_from_config() (renamed to
parse_trailer_templates_from_config()), add a NEEDSWORK discussion about
how the deprecated trailer.*.command configuration option is oddly more
featureful than trailer.*.cmd (if we were to remove support for
trailer.*.command, users would not be able to replicate the behavior
with trailer.*.cmd and would lose out on functionality).

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  50 +++---
 trailer.c                    | 290 +++++++++++++++++++----------------
 trailer.h                    |  12 +-
 3 files changed, 193 insertions(+), 159 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index d0c09d1d73b..d3d2544c5f1 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -45,10 +45,14 @@ static int option_parse_if_missing(const struct option *opt,
 	return trailer_set_if_missing(arg, opt->value);
 }
 
-static int option_parse_trailer(const struct option *opt,
-				   const char *arg, int unset)
+/*
+ * Interpret "--trailer ..." as trailer templates (trailers we want to add into
+ * the input text).
+ */
+static int option_parse_trailer_template(const struct option *opt,
+					 const char *arg, int unset)
 {
-	struct list_head *trailers = opt->value;
+	struct list_head *templates = opt->value;
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
 	const struct trailer_conf *conf;
@@ -56,7 +60,7 @@ static int option_parse_trailer(const struct option *opt,
 	static char *cl_separators;
 
 	if (unset) {
-		free_new_trailers(trailers);
+		free_trailer_templates(templates);
 		return 0;
 	}
 
@@ -93,10 +97,10 @@ static int option_parse_trailer(const struct option *opt,
 		if (if_missing != MISSING_DEFAULT)
 			trailer_set_conf_if_missing(if_missing, conf_current);
 
-		trailer_add_arg_item(trailers,
-				     strbuf_detach(&tok, NULL),
+		add_trailer_template(strbuf_detach(&tok, NULL),
 				     strbuf_detach(&val, NULL),
-				     conf_current);
+				     conf_current,
+				     templates);
 		free_trailer_conf(conf_current);
 	}
 
@@ -158,10 +162,10 @@ static void read_input_file(struct strbuf *sb, const char *file)
 }
 
 static void interpret_trailers(const struct process_trailer_options *opts,
-			       struct list_head *new_trailer_head,
+			       struct list_head *templates,
 			       const char *file)
 {
-	LIST_HEAD(head);
+	LIST_HEAD(trailers_from_sb);
 	struct strbuf sb = STRBUF_INIT;
 	struct strbuf tb = STRBUF_INIT;
 	struct trailer_block *trailer_block;
@@ -172,7 +176,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 	if (opts->in_place)
 		outfile = create_in_place_tempfile(file);
 
-	trailer_block = parse_trailers(opts, sb.buf, &head);
+	trailer_block = parse_trailers(opts, sb.buf, &trailers_from_sb);
 
 	/* Print the lines before the trailer block */
 	if (!opts->only_trailers)
@@ -182,11 +186,11 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 		fprintf(outfile, "\n");
 
 	if (!opts->only_input)
-		process_trailers_lists(&head, new_trailer_head);
+		apply_trailer_templates(templates, &trailers_from_sb);
 
 	/* Print trailer block. */
-	format_trailers(opts, &head, &tb);
-	free_trailers(&head);
+	format_trailers(opts, &trailers_from_sb, &tb);
+	free_trailers(&trailers_from_sb);
 	fwrite(tb.buf, 1, tb.len, outfile);
 	strbuf_release(&tb);
 
@@ -206,8 +210,8 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
-	LIST_HEAD(configured_trailers);
-	LIST_HEAD(trailers);
+	LIST_HEAD(configured_templates);
+	LIST_HEAD(templates);
 
 	struct option options[] = {
 		OPT_BOOL(0, "in-place", &opts.in_place, N_("edit files in place")),
@@ -226,8 +230,8 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 		OPT_CALLBACK_F(0, "parse", &opts, NULL, N_("alias for --only-trailers --only-input --unfold"),
 			PARSE_OPT_NOARG | PARSE_OPT_NONEG, parse_opt_parse),
 		OPT_BOOL(0, "no-divider", &opts.no_divider, N_("do not treat \"---\" as the end of input")),
-		OPT_CALLBACK(0, "trailer", &trailers, N_("trailer"),
-				N_("trailer(s) to add"), option_parse_trailer),
+		OPT_CALLBACK(0, "trailer", &templates, N_("trailer"),
+				N_("trailer(s) to add"), option_parse_trailer_template),
 		OPT_END()
 	};
 
@@ -235,30 +239,30 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 	trailer_config_init();
 
 	if (!opts.only_input)
-		parse_trailers_from_config(&configured_trailers);
+		parse_trailer_templates_from_config(&configured_templates);
 
 	argc = parse_options(argc, argv, prefix, options,
 			     git_interpret_trailers_usage, 0);
 
-	if (opts.only_input && !list_empty(&trailers))
+	if (opts.only_input && !list_empty(&templates))
 		usage_msg_opt(
 			_("--trailer with --only-input does not make sense"),
 			git_interpret_trailers_usage,
 			options);
 
-	list_splice(&configured_trailers, &trailers);
+	list_splice(&configured_templates, &templates);
 
 	if (argc) {
 		int i;
 		for (i = 0; i < argc; i++)
-			interpret_trailers(&opts, &trailers, argv[i]);
+			interpret_trailers(&opts, &templates, argv[i]);
 	} else {
 		if (opts.in_place)
 			die(_("no input file given for in-place editing"));
-		interpret_trailers(&opts, &trailers, NULL);
+		interpret_trailers(&opts, &templates, NULL);
 	}
 
-	free_new_trailers(&trailers);
+	free_trailer_templates(&templates);
 
 	return 0;
 }
diff --git a/trailer.c b/trailer.c
index 7b0bdfcb27e..cdb235c27b5 100644
--- a/trailer.c
+++ b/trailer.c
@@ -55,14 +55,14 @@ struct trailer_item {
 	char *value;
 };
 
-struct arg_item {
+struct trailer_template {
 	struct list_head list;
 	char *token;
 	char *value;
 	struct trailer_conf conf;
 };
 
-static LIST_HEAD(conf_head);
+static LIST_HEAD(templates_from_conf);
 
 static char *separators = ":";
 
@@ -105,7 +105,7 @@ static size_t token_len_without_separator(const char *token, size_t len)
 	return len;
 }
 
-static int same_token(struct trailer_item *a, struct arg_item *b)
+static int same_token(struct trailer_item *a, struct trailer_template *b)
 {
 	size_t a_len, b_len, min_len;
 
@@ -119,12 +119,12 @@ static int same_token(struct trailer_item *a, struct arg_item *b)
 	return !strncasecmp(a->token, b->token, min_len);
 }
 
-static int same_value(struct trailer_item *a, struct arg_item *b)
+static int same_value(struct trailer_item *a, struct trailer_template *b)
 {
 	return !strcasecmp(a->value, b->value);
 }
 
-static int same_trailer(struct trailer_item *a, struct arg_item *b)
+static int same_trailer(struct trailer_item *a, struct trailer_template *b)
 {
 	return same_token(a, b) && same_value(a, b);
 }
@@ -151,15 +151,15 @@ static void free_trailer_item(struct trailer_item *item)
 	free(item);
 }
 
-static void free_arg_item(struct arg_item *item)
+static void free_template(struct trailer_template *template)
 {
-	free(item->conf.name);
-	free(item->conf.key);
-	free(item->conf.command);
-	free(item->conf.cmd);
-	free(item->token);
-	free(item->value);
-	free(item);
+	free(template->conf.name);
+	free(template->conf.key);
+	free(template->conf.command);
+	free(template->conf.cmd);
+	free(template->token);
+	free(template->value);
+	free(template);
 }
 
 static char last_non_space_char(const char *s)
@@ -171,36 +171,36 @@ static char last_non_space_char(const char *s)
 	return '\0';
 }
 
-static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
+static struct trailer_item *trailer_from(struct trailer_template *template)
 {
 	struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
-	new_item->token = arg_tok->token;
-	new_item->value = arg_tok->value;
-	arg_tok->token = arg_tok->value = NULL;
-	free_arg_item(arg_tok);
+	new_item->token = template->token;
+	new_item->value = template->value;
+	template->token = template->value = NULL;
+	free_template(template);
 	return new_item;
 }
 
-static void add_arg_to_input_list(struct trailer_item *on_tok,
-				  struct arg_item *arg_tok)
+static void apply_template_to_trailers(struct trailer_template *template,
+				       struct trailer_item *on_tok)
 {
-	int aoe = after_or_end(arg_tok->conf.where);
-	struct trailer_item *to_add = trailer_from_arg(arg_tok);
+	int aoe = after_or_end(template->conf.where);
+	struct trailer_item *to_add = trailer_from(template);
 	if (aoe)
 		list_add(&to_add->list, &on_tok->list);
 	else
 		list_add_tail(&to_add->list, &on_tok->list);
 }
 
-static int check_if_different(struct trailer_item *in_tok,
-			      struct arg_item *arg_tok,
-			      int check_all,
-			      struct list_head *head)
+static int check_if_different(struct trailer_template *template,
+			      struct trailer_item *in_tok,
+			      struct list_head *head,
+			      int check_all)
 {
-	enum trailer_where where = arg_tok->conf.where;
+	enum trailer_where where = template->conf.where;
 	struct list_head *next_head;
 	do {
-		if (same_trailer(in_tok, arg_tok))
+		if (same_trailer(in_tok, template))
 			return 0;
 		/*
 		 * if we want to add a trailer after another one,
@@ -215,7 +215,8 @@ static int check_if_different(struct trailer_item *in_tok,
 	return 1;
 }
 
-static char *apply_command(struct trailer_conf *conf, const char *arg)
+static char *run_command_from_template(struct trailer_conf *conf,
+				       const char *arg)
 {
 	struct strbuf cmd = STRBUF_INIT;
 	struct strbuf buf = STRBUF_INIT;
@@ -250,133 +251,142 @@ static char *apply_command(struct trailer_conf *conf, const char *arg)
 	return result;
 }
 
-static void apply_item_command(struct trailer_item *in_tok, struct arg_item *arg_tok)
+/*
+ * Prepare the template by running the command (if any) requested by the
+ * template in order to populate the template's value field.
+ */
+static void populate_template_value(struct trailer_template *template,
+				    struct trailer_item *in_tok)
 {
-	if (arg_tok->conf.command || arg_tok->conf.cmd) {
+	if (template->conf.command || template->conf.cmd) {
+		/*
+		 * Determine argument to pass into the command.
+		 */
 		const char *arg;
-		if (arg_tok->value && arg_tok->value[0]) {
-			arg = arg_tok->value;
+		if (template->value && template->value[0]) {
+			arg = template->value;
 		} else {
 			if (in_tok && in_tok->value)
 				arg = xstrdup(in_tok->value);
 			else
 				arg = xstrdup("");
 		}
-		arg_tok->value = apply_command(&arg_tok->conf, arg);
+		template->value = run_command_from_template(&template->conf,
+							    arg);
 		free((char *)arg);
 	}
 }
 
-static void apply_arg_if_exists(struct trailer_item *in_tok,
-				struct arg_item *arg_tok,
+static void maybe_add_if_exists(struct trailer_template *template,
+				struct trailer_item *in_tok,
 				struct trailer_item *on_tok,
-				struct list_head *head)
+				struct list_head *trailers)
 {
-	switch (arg_tok->conf.if_exists) {
+	switch (template->conf.if_exists) {
 	case EXISTS_DO_NOTHING:
-		free_arg_item(arg_tok);
+		free_template(template);
 		break;
 	case EXISTS_REPLACE:
-		apply_item_command(in_tok, arg_tok);
-		add_arg_to_input_list(on_tok, arg_tok);
+		populate_template_value(template, in_tok);
+		apply_template_to_trailers(template, on_tok);
 		list_del(&in_tok->list);
 		free_trailer_item(in_tok);
 		break;
 	case EXISTS_ADD:
-		apply_item_command(in_tok, arg_tok);
-		add_arg_to_input_list(on_tok, arg_tok);
+		populate_template_value(template, in_tok);
+		apply_template_to_trailers(template, on_tok);
 		break;
 	case EXISTS_ADD_IF_DIFFERENT:
-		apply_item_command(in_tok, arg_tok);
-		if (check_if_different(in_tok, arg_tok, 1, head))
-			add_arg_to_input_list(on_tok, arg_tok);
+		populate_template_value(template, in_tok);
+		if (check_if_different(template, in_tok, trailers, 1))
+			apply_template_to_trailers(template, on_tok);
 		else
-			free_arg_item(arg_tok);
+			free_template(template);
 		break;
 	case EXISTS_ADD_IF_DIFFERENT_NEIGHBOR:
-		apply_item_command(in_tok, arg_tok);
-		if (check_if_different(on_tok, arg_tok, 0, head))
-			add_arg_to_input_list(on_tok, arg_tok);
+		populate_template_value(template, in_tok);
+		if (check_if_different(template, on_tok, trailers, 0))
+			apply_template_to_trailers(template, on_tok);
 		else
-			free_arg_item(arg_tok);
+			free_template(template);
 		break;
 	default:
 		BUG("trailer.c: unhandled value %d",
-		    arg_tok->conf.if_exists);
+		    template->conf.if_exists);
 	}
 }
 
-static void apply_arg_if_missing(struct list_head *head,
-				 struct arg_item *arg_tok)
+static void maybe_add_if_missing(struct trailer_template *template,
+				 struct list_head *trailers)
 {
 	enum trailer_where where;
 	struct trailer_item *to_add;
 
-	switch (arg_tok->conf.if_missing) {
+	switch (template->conf.if_missing) {
 	case MISSING_DO_NOTHING:
-		free_arg_item(arg_tok);
+		free_template(template);
 		break;
 	case MISSING_ADD:
-		where = arg_tok->conf.where;
-		apply_item_command(NULL, arg_tok);
-		to_add = trailer_from_arg(arg_tok);
+		where = template->conf.where;
+		populate_template_value(template, NULL);
+		to_add = trailer_from(template);
 		if (after_or_end(where))
-			list_add_tail(&to_add->list, head);
+			list_add_tail(&to_add->list, trailers);
 		else
-			list_add(&to_add->list, head);
+			list_add(&to_add->list, trailers);
 		break;
 	default:
 		BUG("trailer.c: unhandled value %d",
-		    arg_tok->conf.if_missing);
+		    template->conf.if_missing);
 	}
 }
 
-static int find_same_and_apply_arg(struct list_head *head,
-				   struct arg_item *arg_tok)
+static int find_same_and_apply_arg(struct trailer_template *template,
+				   struct list_head *trailers)
 {
 	struct list_head *pos;
 	struct trailer_item *in_tok;
 	struct trailer_item *on_tok;
 
-	enum trailer_where where = arg_tok->conf.where;
+	enum trailer_where where = template->conf.where;
 	int middle = (where == WHERE_AFTER) || (where == WHERE_BEFORE);
 	int backwards = after_or_end(where);
 	struct trailer_item *start_tok;
 
-	if (list_empty(head))
+	if (list_empty(trailers))
 		return 0;
 
-	start_tok = list_entry(backwards ? head->prev : head->next,
+	start_tok = list_entry(backwards ? trailers->prev : trailers->next,
 			       struct trailer_item,
 			       list);
 
-	list_for_each_dir(pos, head, backwards) {
+	list_for_each_dir(pos, trailers, backwards) {
 		in_tok = list_entry(pos, struct trailer_item, list);
-		if (!same_token(in_tok, arg_tok))
+		if (!same_token(in_tok, template))
 			continue;
 		on_tok = middle ? in_tok : start_tok;
-		apply_arg_if_exists(in_tok, arg_tok, on_tok, head);
+		maybe_add_if_exists(template, in_tok, on_tok, trailers);
 		return 1;
 	}
 	return 0;
 }
 
-void process_trailers_lists(struct list_head *head,
-			    struct list_head *arg_head)
+void apply_trailer_templates(struct list_head *templates,
+			     struct list_head *trailers)
 {
 	struct list_head *pos, *p;
-	struct arg_item *arg_tok;
+	struct trailer_template *template;
 
-	list_for_each_safe(pos, p, arg_head) {
+	list_for_each_safe(pos, p, templates) {
 		int applied = 0;
-		arg_tok = list_entry(pos, struct arg_item, list);
+		template = list_entry(pos, struct trailer_template, list);
 
 		list_del(pos);
 
-		applied = find_same_and_apply_arg(head, arg_tok);
+		applied = find_same_and_apply_arg(template, trailers);
 
 		if (!applied)
-			apply_arg_if_missing(head, arg_tok);
+			maybe_add_if_missing(template, trailers);
 	}
 }
 
@@ -471,26 +481,26 @@ void free_trailer_conf(struct trailer_conf *conf)
 	free(conf);
 }
 
-static struct arg_item *get_conf_item(const char *name)
+static struct trailer_template *get_or_add_template_by(const char *name)
 {
 	struct list_head *pos;
-	struct arg_item *item;
+	struct trailer_template *template;
 
-	/* Look up item with same name */
-	list_for_each(pos, &conf_head) {
-		item = list_entry(pos, struct arg_item, list);
-		if (!strcasecmp(item->conf.name, name))
-			return item;
+	/* Look up template with same name. */
+	list_for_each(pos, &templates_from_conf) {
+		template = list_entry(pos, struct trailer_template, list);
+		if (!strcasecmp(template->conf.name, name))
+			return template;
 	}
 
-	/* Item does not already exists, create it */
-	CALLOC_ARRAY(item, 1);
-	duplicate_trailer_conf(&item->conf, &default_trailer_conf);
-	item->conf.name = xstrdup(name);
+	/* Template does not already exist; create it. */
+	CALLOC_ARRAY(template, 1);
+	duplicate_trailer_conf(&template->conf, &default_trailer_conf);
+	template->conf.name = xstrdup(name);
 
-	list_add_tail(&item->list, &conf_head);
+	list_add_tail(&template->list, &templates_from_conf);
 
-	return item;
+	return template;
 }
 
 enum trailer_info_type { TRAILER_KEY, TRAILER_COMMAND, TRAILER_CMD,
@@ -548,7 +558,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 			      void *cb UNUSED)
 {
 	const char *trailer_item, *variable_name;
-	struct arg_item *item;
+	struct trailer_template *template;
 	struct trailer_conf *conf;
 	char *name = NULL;
 	enum trailer_info_type type;
@@ -573,8 +583,8 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	if (!name)
 		return 0;
 
-	item = get_conf_item(name);
-	conf = &item->conf;
+	template = get_or_add_template_by(name);
+	conf = &template->conf;
 	free(name);
 
 	switch (type) {
@@ -631,20 +641,22 @@ void trailer_config_init(void)
 	configured = 1;
 }
 
-static const char *token_from_item(struct arg_item *item, char *tok)
+static const char *token_from_template(struct trailer_template *template, char *tok)
 {
-	if (item->conf.key)
-		return item->conf.key;
+	if (template->conf.key)
+		return template->conf.key;
 	if (tok)
 		return tok;
-	return item->conf.name;
+	return template->conf.name;
 }
 
-static int token_matches_item(const char *tok, struct arg_item *item, size_t tok_len)
+static int token_matches_template(const char *tok,
+				  struct trailer_template *template,
+				  size_t tok_len)
 {
-	if (!strncasecmp(tok, item->conf.name, tok_len))
+	if (!strncasecmp(tok, template->conf.name, tok_len))
 		return 1;
-	return item->conf.key ? !strncasecmp(tok, item->conf.key, tok_len) : 0;
+	return template->conf.key ? !strncasecmp(tok, template->conf.key, tok_len) : 0;
 }
 
 /*
@@ -687,7 +699,7 @@ void parse_trailer(const char *line, ssize_t separator_pos,
 		   struct strbuf *tok, struct strbuf *val,
 		   const struct trailer_conf **conf)
 {
-	struct arg_item *item;
+	struct trailer_template *template;
 	size_t tok_len;
 	struct list_head *pos;
 
@@ -705,13 +717,13 @@ void parse_trailer(const char *line, ssize_t separator_pos,
 	tok_len = token_len_without_separator(tok->buf, tok->len);
 	if (conf)
 		*conf = &default_trailer_conf;
-	list_for_each(pos, &conf_head) {
-		item = list_entry(pos, struct arg_item, list);
-		if (token_matches_item(tok->buf, item, tok_len)) {
+	list_for_each(pos, &templates_from_conf) {
+		template = list_entry(pos, struct trailer_template, list);
+		if (token_matches_template(tok->buf, template, tok_len)) {
 			char *tok_buf = strbuf_detach(tok, NULL);
 			if (conf)
-				*conf = &item->conf;
-			strbuf_addstr(tok, token_from_item(item, tok_buf));
+				*conf = &template->conf;
+			strbuf_addstr(tok, token_from_template(template, tok_buf));
 			free(tok_buf);
 			break;
 		}
@@ -728,29 +740,41 @@ static struct trailer_item *add_trailer_item(struct list_head *head, char *tok,
 	return new_item;
 }
 
-void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
-			  const struct trailer_conf *conf)
+void add_trailer_template(char *tok, char *val, const struct trailer_conf *conf,
+			  struct list_head *templates)
 {
-	struct arg_item *new_item = xcalloc(1, sizeof(*new_item));
-	new_item->token = tok;
-	new_item->value = val;
-	duplicate_trailer_conf(&new_item->conf, conf);
-	list_add_tail(&new_item->list, arg_head);
+	struct trailer_template *template = xcalloc(1, sizeof(*template));
+	template->token = tok;
+	template->value = val;
+	duplicate_trailer_conf(&template->conf, conf);
+	list_add_tail(&template->list, templates);
 }
 
-void parse_trailers_from_config(struct list_head *config_head)
+void parse_trailer_templates_from_config(struct list_head *config_head)
 {
-	struct arg_item *item;
+	struct trailer_template *template;
 	struct list_head *pos;
 
-	/* Add an arg item for each configured trailer with a command */
-	list_for_each(pos, &conf_head) {
-		item = list_entry(pos, struct arg_item, list);
-		if (item->conf.command)
-			trailer_add_arg_item(config_head,
-					     xstrdup(token_from_item(item, NULL)),
-					     xstrdup(""),
-					     &item->conf);
+	/*
+	 * Get configured templates with a ".command" option.
+	 *
+	 * NEEDSWORK: If the interpret-trailers builtin sees a
+	 * "trailer.foo.command = ..." setting, then the "foo" trailer will
+	 * always be inserted, even if "--trailer foo" is not provided.
+	 * Considering how ".command" is deprecated, it is a bit strange to see
+	 * it getting special treatment like this over ".cmd". Instead, we
+	 * should add a new option that explicitly lets the user decide if the
+	 * configured trailer should always be added automatically, or if it
+	 * should only be added if "--trailer foo" is provided (default).
+	 * Then we can collect configured trailers that have either ".command"
+	 * or ".cmd" below, instead of just ".command".
+	 */
+	list_for_each(pos, &templates_from_conf) {
+		template = list_entry(pos, struct trailer_template, list);
+		if (template->conf.command)
+			add_trailer_template(xstrdup(token_from_template(template,
+									 NULL)),
+					     xstrdup(""), &template->conf, config_head);
 	}
 }
 
@@ -906,10 +930,16 @@ static size_t find_trailer_block_start(const char *buf, size_t len)
 			possible_continuation_lines = 0;
 			if (recognized_prefix)
 				continue;
-			list_for_each(pos, &conf_head) {
-				struct arg_item *item;
-				item = list_entry(pos, struct arg_item, list);
-				if (token_matches_item(bol, item,
+			/*
+			 * The templates here are not used for actually
+			 * adding trailers anywhere, but instead to help us
+			 * identify trailer lines by comparing their keys with
+			 * those found in configured templates.
+			 */
+			list_for_each(pos, &templates_from_conf) {
+				struct trailer_template *template;
+				template = list_entry(pos, struct trailer_template, list);
+				if (token_matches_template(bol, template,
 						       separator_pos)) {
 					recognized_prefix = 1;
 					break;
@@ -1065,13 +1095,13 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
-void free_new_trailers(struct list_head *trailers)
+void free_trailer_templates(struct list_head *trailer_templates)
 {
 	struct list_head *pos, *p;
 
-	list_for_each_safe(pos, p, trailers) {
+	list_for_each_safe(pos, p, trailer_templates) {
 		list_del(pos);
-		free_arg_item(list_entry(pos, struct arg_item, list));
+		free_template(list_entry(pos, struct trailer_template, list));
 	}
 }
 
diff --git a/trailer.h b/trailer.h
index 4193bedbae4..2a21d74c263 100644
--- a/trailer.h
+++ b/trailer.h
@@ -40,8 +40,8 @@ struct trailer_conf *new_trailer_conf(void);
 void duplicate_trailer_conf(struct trailer_conf *dst,
 			    const struct trailer_conf *src);
 const char *trailer_default_separators(void);
-void trailer_add_arg_item(struct list_head *arg_head, char *tok, char *val,
-			  const struct trailer_conf *conf);
+void add_trailer_template(char *tok, char *val, const struct trailer_conf *,
+			  struct list_head *templates);
 
 struct process_trailer_options {
 	int in_place;
@@ -60,10 +60,10 @@ struct process_trailer_options {
 
 #define PROCESS_TRAILER_OPTIONS_INIT {0}
 
-void parse_trailers_from_config(struct list_head *config_head);
+void parse_trailer_templates_from_config(struct list_head *config_head);
 
-void process_trailers_lists(struct list_head *head,
-			    struct list_head *arg_head);
+void apply_trailer_templates(struct list_head *templates,
+			     struct list_head *trailers_head);
 
 ssize_t find_separator(const char *line, const char *separators);
 
@@ -86,8 +86,8 @@ void format_trailers(const struct process_trailer_options *,
 		     struct list_head *trailers,
 		     struct strbuf *out);
 void free_trailers(struct list_head *);
-void free_new_trailers(struct list_head *);
 void free_trailer_conf(struct trailer_conf *);
+void free_trailer_templates(struct list_head *);
 
 /*
  * Convenience function to format the trailers from the commit msg "msg" into
-- 
gitgitgadget

Re: [PATCH v4 19/28] trailer: make trailer_info struct private

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> This patch believes that the benefits outweight the advantages for
> designing APIs, as explained below.

"outweigh the disadvantages"?

Re: [PATCH v4 16/28] trailer: finish formatting unification

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> Rename format_trailer_info() to format_trailers(). Finally, both
> interpret-trailers and format_trailers_from_commit() can call
> "format_trailers()"!
>
> Update the comment in <trailer.h> to remove the (now obsolete) caveats
> about format_trailers_from_commit().

Nice.

Re: [PATCH v4 12/28] format_trailer_info(): append newline for non-trailer lines

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> And before we made the transition to use trailer_item objects for it,
> format_trailer_info() called parse_trailer() (which trims newlines) for
> trailer lines but did _not_ call parse_trailer() for non-trailer lines.
> So for trailer lines it had to add back the trimmed newline like this
>
>     if (!opts->separator)
>         strbuf_addch(out, '\n');
>
> But for non-trailer lines it didn't have to add back the newline because
> it could just reuse same string in the "trailers" string array (which
> again, already included the trailing newline).
>
> Now that format_trailer_info() uses trailer_item objects for all cases,
> it can't rely on "trailers" string array anymore.  And so it must be
> taught to add a newline back when printing non-trailer lines, just like
> it already does for trailer lines. Do so now.

Very nicely explained.

> The test suite passes again, so format_trailer_info() is in better shape
> supersede format_trailers(), which we'll do in the next patch.
>
> Signed-off-by: Linus Arver <linusa@google.com>
> ---
>  trailer.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/trailer.c b/trailer.c
> index 12cae5b73d2..0774a544c4f 100644
> --- a/trailer.c
> +++ b/trailer.c
> @@ -1125,9 +1125,10 @@ static void format_trailer_info(const struct process_trailer_options *opts,
>  				strbuf_addbuf(out, opts->separator);
>  			}
>  			strbuf_addstr(out, item->value);
> -			if (opts->separator) {
> +			if (opts->separator)
>  				strbuf_rtrim(out);
> -			}
> +			else
> +				strbuf_addch(out, '\n');
>  		}
>  	}
>  }

Re: [PATCH v4 10/28] format_trailer_info(): use trailer_item objects

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> This is another preparatory refactor to unify the trailer formatters.
>
> Make format_trailer_info() operate on trailer_item objects, not the raw
> string array.
>
> This breaks t4205 and t6300. We will continue to make improvements until
> the test suite passes again, ultimately renaming format_trailer_info()
> to format_trailers(), at which point the unification of these formatters
> will be complete.
>
> Signed-off-by: Linus Arver <linusa@google.com>
> ---
>  trailer.c | 21 ++++++++++-----------
>  1 file changed, 10 insertions(+), 11 deletions(-)

I would have expected a (tentative) flip from test_expect_success to
test_expect_failure in the affected tests, to illustrate what behaviour
change this step introduces and why.

But the huge single step in the previous round broken out into
smaller steps like this round makes them much easier to follow.

Thanks.

> diff --git a/trailer.c b/trailer.c
> index e6665c99cc3..6333dfe1c11 100644
> --- a/trailer.c
> +++ b/trailer.c
> @@ -1086,21 +1086,21 @@ void trailer_info_release(struct trailer_info *info)
>  }
>  
>  static void format_trailer_info(const struct process_trailer_options *opts,
> -				const struct trailer_info *info,
> +				struct list_head *trailers,
>  				struct strbuf *out)
>  {
>  	size_t origlen = out->len;
> -	size_t i;
> -
> -	for (i = 0; i < info->trailer_nr; i++) {
> -		char *trailer = info->trailers[i];
> -		ssize_t separator_pos = find_separator(trailer, separators);
> +	struct list_head *pos;
> +	struct trailer_item *item;
>  
> -		if (separator_pos >= 1) {
> +	list_for_each(pos, trailers) {
> +		item = list_entry(pos, struct trailer_item, list);
> +		if (item->token) {
>  			struct strbuf tok = STRBUF_INIT;
>  			struct strbuf val = STRBUF_INIT;
> +			strbuf_addstr(&tok, item->token);
> +			strbuf_addstr(&val, item->value);
>  
> -			parse_trailer(&tok, &val, NULL, trailer, separator_pos);
>  			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
>  				if (opts->unfold)
>  					unfold_value(&val);
> @@ -1127,13 +1127,12 @@ static void format_trailer_info(const struct process_trailer_options *opts,
>  			if (opts->separator && out->len != origlen) {
>  				strbuf_addbuf(out, opts->separator);
>  			}
> -			strbuf_addstr(out, trailer);
> +			strbuf_addstr(out, item->value);
>  			if (opts->separator) {
>  				strbuf_rtrim(out);
>  			}
>  		}
>  	}
> -
>  }
>  
>  void format_trailers_from_commit(const struct process_trailer_options *opts,
> @@ -1152,7 +1151,7 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
>  		strbuf_add(out, msg + info.trailer_block_start,
>  			   info.trailer_block_end - info.trailer_block_start);
>  	} else
> -		format_trailer_info(opts, &info, out);
> +		format_trailer_info(opts, &trailers, out);
>  
>  	free_trailers(&trailers);
>  	trailer_info_release(&info);

Re: [PATCH v4 11/28] format_trailer_info(): drop redundant unfold_value()

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Linus Arver <linusa@google.com>
>
> This is another preparatory refactor to unify the trailer formatters.
>
> In the last patch we made format_trailer_info() use trailer_item objects
> instead of the "trailers" string array. This means that the call to
> unfold_value() here is redundant because the trailer_item objects are
> already unfolded in parse_trailers() which is a dependency of our
> caller, format_trailers_from_commit().
>
> Remove the redundant call.

OK.  The previous step had this hunk:

-			parse_trailer(&tok, &val, NULL, trailer, separator_pos);
 			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
 				if (opts->unfold)
 					unfold_value(&val);

where parse_trailers() already has a call to parse_trailer()
followed by a call to unfold_value(), so in a sense, switching
to use the result of calling parse_trailers() by the caller instead
of duplicating our own parsing in format_trailer_info() that started
at step [09/28] made both parse_trailer() call (removed in step [10/28])
and unfold_value() call (removed in this step [11/28]).

So it would have also made sense if this were done as part of
[10/28], but it is also OK to keep them separated.

In either way, breaking the transition into these steps does make
them easier to follow.

> Signed-off-by: Linus Arver <linusa@google.com>
> ---
>  trailer.c | 3 ---
>  1 file changed, 3 deletions(-)
>
> diff --git a/trailer.c b/trailer.c
> index 6333dfe1c11..12cae5b73d2 100644
> --- a/trailer.c
> +++ b/trailer.c
> @@ -1102,9 +1102,6 @@ static void format_trailer_info(const struct process_trailer_options *opts,
>  			strbuf_addstr(&val, item->value);
>  
>  			if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
> -				if (opts->unfold)
> -					unfold_value(&val);
> -
>  				if (opts->separator && out->len != origlen)
>  					strbuf_addbuf(out, opts->separator);
>  				if (!opts->value_only)

Re: [PATCH v4 00/28] Enrich Trailer API

[Christian Couder]

On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> This patch series is the first 10 patches of a larger cleanup/bugfix series
> (henceforth "larger series") I've been working on.

There are now 28 patches in this series.

I took a look at all of them, and I think that this series should be
split into 4 or more series.

For example perhaps one series until the "trailer: move
interpret_trailers() to interpret-trailers.c" patch, then another one
until "trailer: finish formatting unification". etc.

Also I think it might be possible to avoid some test failures
introduced by some patches. If it's not possible, I agree with Junio
that it would be nice if the failing tests were changed to use
'test_expect_failure'.

Also it seems to me that many patches towards the end of this series
should be split.

> The main goal of this
> series is to begin the process of "libifying" the trailer API. By "API" I
> mean the interface exposed in trailer.h. The larger series brings a number
> of additional cleanups (exposing and fixing some bugs along the way), and
> builds on top of this series.

[...]

> With the libification-focused goals out of the way, let's turn to this patch
> series in more detail.

I like the goal of libifying Git the trailer API, and the way you want
to do it seems reasonable to me.

[...]

> In summary this series breaks up "process_trailers()" into smaller pieces,
> exposing many of the parts relevant to trailer-related processing in
> trailer.h. This will force us to eventually introduce unit tests for these
> API functions, but that is a good thing for API stability.

I am a bit sad that this series doesn't introduce unit tests using the
new test framework in C yet. I understand that this series is mostly a
big refactoring and maybe it's better to introduce unit tests only
when the refactoring is finished though. Anyway I hope the next series
will introduce such tests.

> In the future after libification is "complete", users external to Git will
> be able to use the same trailer processing API used by the
> interpret-trailers builtin. For example, a web server may want to parse
> trailers the same way that Git would parse them, without having to call
> interpret-trailers as a subprocess. This use case was the original
> motivation behind my work in this area.

Thanks for telling us about this use case.

> Thanks to the aggressive refactoring in this series, I've been able to
> identify and fix several bugs in our existing implementation. Those fixes
> build on top of this series but were not included here, in order to keep
> this series small. Below is a "shortlog" of those fixes I have locally:
>
>  * "trailer: trailer replacement should not change its position" (If we
>    found a trailer we'd like to replace, preserve its position relative to
>    the other trailers found in the trailer block, instead of always moving
>    it to the beginning or end of the entire trailer block.)

I believe there was a reason why it was done this way. I don't
remember what it was though.

>  * "interpret-trailers: preserve trailers coming from the input" (Sometimes,
>    the parsed trailers from the input will be formatted differently
>    depending on whether we provide --only-trailers or not. Make the trailers
>    that were not modified and which are coming directly from the input get
>    formatted the same way, regardless of this flag.)

It could be a feature to be able to normalize trailers in a certain way.

>  * "interpret-trailers: do not modify the input if NOP" (Refrain from
>    subtracting or adding a newline around the patch divider "---" if we are
>    not adding new trailers.)

It could be a feature to be able to normalize this too.

>  * "trailer formatter: split up format_trailer() monolith" (Fix a bug in
>    git-log where we still printed a blank newline even if we didn't want to
>    format anything.)

I am not sure this is a bug fix either. It could perhaps be a normalization too.

>  * "interpret-trailers: fail if given unrecognized arguments" (E.g., for
>    "--where", only accept recognized WHERE_* enum values. If we get
>    something unrecognized, fail with an error instead of silently doing
>    nothing. Ditto for "--if-exists" and "--if-missing".)

It's possible that there was a reason why it was done this way.

I think you might want to take a look at the discussions on the
mailing list when "interpret-trailers" was developed. There were a lot
of discussions over a long time, and there were a lot of requests and
suggestions about what it should do.

Re: [PATCH v4 12/28] format_trailer_info(): append newline for non-trailer lines

[Christian Couder]

On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:

[...]

> The test suite passes again, so format_trailer_info() is in better shape
> supersede format_trailers(), which we'll do in the next patch.

s/supersede/to supersede/

Re: [PATCH v4 15/28] format_trailer_info(): avoid double-printing the separator

[Christian Couder]

On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Linus Arver <linusa@google.com>
>
> Do not hardcode the printing of ": " as the separator and space (which
> can result in double-printing these characters); instead only
> print the separator and space if we cannot find any recognized separator
> somewhere in the key (yes, keys may have a trailing separator in it ---
> we will eventually fix this design but not now). Do so by copying the
> code out of print_tok_val(), and deleting the same function.
>
> The test suite passes again with this change.

I think it should be clearer above that this fixes a bug that was
introduced earlier in the series.

Also I wonder why it was not possible to modify format_trailer_info()
like it is done in this patch before using it to replace
format_trailers().

Re: [PATCH v4 14/28] format_trailer_info(): teach it about opts->trim_empty

[Christian Couder]

> diff --git a/trailer.c b/trailer.c
> index f4defad3dae..c28b6c11cc5 100644
> --- a/trailer.c
> +++ b/trailer.c
> @@ -162,20 +162,6 @@ static void print_tok_val(struct strbuf *out, const char *tok, const char *val)
>                 strbuf_addf(out, "%s%c %s\n", tok, separators[0], val);
>  }
>
> -static void format_trailers(const struct process_trailer_options *opts,
> -                           struct list_head *trailers,
> -                           struct strbuf *out)
> -{
> -       struct list_head *pos;
> -       struct trailer_item *item;
> -       list_for_each(pos, trailers) {
> -               item = list_entry(pos, struct trailer_item, list);
> -               if ((!opts->trim_empty || strlen(item->value) > 0) &&
> -                   (!opts->only_trailers || item->token))
> -                       print_tok_val(out, item->token, item->value);
> -       }
> -}

It seems to me that this function could and should have been removed
in the previous patch. If there is a reason why it is better to do it
in this patch, I think it should be explained more clearly in the
commit message.

>  static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
>  {
>         struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
> @@ -1101,6 +1087,15 @@ void format_trailer_info(const struct process_trailer_options *opts,
>                         strbuf_addstr(&tok, item->token);
>                         strbuf_addstr(&val, item->value);
>
> +                       /*
> +                        * Skip key/value pairs where the value was empty. This
> +                        * can happen from trailers specified without a
> +                        * separator, like `--trailer "Reviewed-by"` (no
> +                        * corresponding value).
> +                        */
> +                       if (opts->trim_empty && !strlen(item->value))
> +                               continue;
> +

Wasn't it possible to make this change in format_trailer_info() before
using format_trailer_info() to replace format_trailers()?


>                         if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
>                                 if (opts->separator && out->len != origlen)
>                                         strbuf_addbuf(out, opts->separator);

Re: [PATCH v4 16/28] trailer: finish formatting unification

[Christian Couder]

On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:

>  /*
> - * Format the trailers from the commit msg "msg" into the strbuf "out".
> - * Note two caveats about "opts":
> - *
> - *   - this is primarily a helper for pretty.c, and not
> - *     all of the flags are supported.
> - *
> - *   - this differs from process_trailers slightly in that we always format
> - *     only the trailer block itself, even if the "only_trailers" option is not
> - *     set.

This makes me wonder if there was actually a good reason why
format_trailers() and format_trailer_info() were 2 different
functions. Is there info about this in the commit message of the
commit which introduced this comment?

Re: [PATCH v4 19/28] trailer: make trailer_info struct private

[Christian Couder]

On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:

> diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
> index 11f4ce9e4a2..6bf8cec005a 100644
> --- a/builtin/interpret-trailers.c
> +++ b/builtin/interpret-trailers.c
> @@ -141,7 +141,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
>         LIST_HEAD(head);
>         struct strbuf sb = STRBUF_INIT;
>         struct strbuf trailer_block = STRBUF_INIT;
> -       struct trailer_info info;
> +       struct trailer_info *info;
>         FILE *outfile = stdout;
>
>         trailer_config_init();
> @@ -151,13 +151,13 @@ static void interpret_trailers(const struct process_trailer_options *opts,
>         if (opts->in_place)
>                 outfile = create_in_place_tempfile(file);
>
> -       parse_trailers(opts, &info, sb.buf, &head);
> +       info = parse_trailers(opts, sb.buf, &head);

I think this patch might be doing too much at once and could have been
split into 3 or more patches to make reviews easier.

For example the first patch could introduce trailer_info_new() and
make interpret_trailers() use it. Then the second patch could modify
parse_trailers() so that it returns a 'struct trailer_info *'. Then
the third patch could make the trailer_info struct private.

Re: [PATCH v4 21/28] trailer: spread usage of "trailer_block" language

[Christian Couder]

On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Linus Arver <linusa@google.com>
>
> Deprecate the "trailer_info" struct name and replace it with
> "trailer_block". The main reason is to help readability, because
> "trailer_info" on the surface sounds like it's about a single trailer
> when in reality it is a collection of contiguous lines, at least 25% of
> which are trailers.
>
> Signed-off-by: Linus Arver <linusa@google.com>
> ---
>  builtin/interpret-trailers.c | 25 +++++-----
>  trailer.c                    | 97 ++++++++++++++++++------------------
>  trailer.h                    | 18 +++----
>  3 files changed, 71 insertions(+), 69 deletions(-)
>
> diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
> index 6bf8cec005a..f76841c5280 100644
> --- a/builtin/interpret-trailers.c
> +++ b/builtin/interpret-trailers.c
> @@ -140,8 +140,8 @@ static void interpret_trailers(const struct process_trailer_options *opts,
>  {
>         LIST_HEAD(head);
>         struct strbuf sb = STRBUF_INIT;
> -       struct strbuf trailer_block = STRBUF_INIT;
> -       struct trailer_info *info;
> +       struct strbuf tb = STRBUF_INIT;
> +       struct trailer_block *trailer_block;

I understand that using 'trailer_block' for a 'struct trailer_block *'
makes sense and I like the idea behind this patch, but it's
unfortunate that 'struct strbuf trailer_block' becomes 'struct strbuf
tb'. Also the name change for 'struct strbuf trailer_block' could be
in a separate patch.

Re: [PATCH v4 22/28] trailer: prepare to delete "parse_trailers_from_command_line_args()"

[Christian Couder]

On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Linus Arver <linusa@google.com>
>
> Expose more functions in the trailer.h API, in preparation for deleting
>
>     parse_trailers_from_command_line_args()
>
> from the trailers implementation, because it should not be concerned
> with command line arguments (as they have nothing to do with trailers
> themselves). Indeed, the interpret-trailers builtin is the only caller
> of this function inside Git.
>
> Rename add_arg_item() to trailer_add_arg_item() to expose it as an API
> function. Rename new_trailers_clear() to free_new_trailers() because it
> will be promoted into an API function; the API already has
> free_trailers(), so using the "free_*" naming style will keep it
> consistent. Also rename "conf_info" to "trailer_conf" for readability,
> dropping the low-value "_info" suffix as we did earlier in this series
> for "trailer_info" to "trailer_block".

That's a lot done in a single patch. I think splitting it into 3 or
more patches would be nice for reviewers.

Re: [PATCH v4 23/28] trailer: add new helper functions to API

[Christian Couder]

On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Linus Arver <linusa@google.com>
>
> This is a preparatory refactor for deprecating "new_trailer_item" from
> the API (which will let us deprecate
> parse_trailers_from_command_line_args()).
>
> Expose new helper functions from the API, because we'll be calling them
> from interpret-trailers.c soon when we move
> parse_trailers_from_command_line_args() there.
>
> Move free_new_trailers() from the builtin to trailer.c because later on
> we will adjust it to free arg_item structs, which are private to
> trailer.c.

This patch seems to be also doing too much.

Re: [PATCH v4 26/28] trailer: unify "--trailer ..." arg handling

[Christian Couder]

On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Linus Arver <linusa@google.com>
>
> Move the logic of parse_trailer_from_command_line_arg() into
> option_parse_trailer(), because that is the only caller and there's no
> benefit in keeping these two separate.

Well one benefit could be that 2 small functions might be easier to
understand than a big one. So perhaps
parse_trailer_from_command_line_arg() could just have been made
static?

Re: [PATCH v4 27/28] trailer_set_*(): put out parameter at the end

[Christian Couder]

On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Linus Arver <linusa@google.com>
>
> The new trailer_config_set_*() functions which were introduced a few
> patches ago put the out parameter (the variable being mutated) at the
> end of the parameter list.
>
> Put the out parameter at the end for these functions for these existing

s/for these functions for these/for the/


> trailer_set_*() functions for consistency. This also avoids confusion
> for API users because otherwise these two sets of functions look rather
> similar in <trailer.h> even though they have completely different out
> parameters.

Re: [PATCH v4 28/28] trailer: introduce "template" term for readability

[Christian Couder]

On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Linus Arver <linusa@google.com>
>
> The term "arg_item" is ambiguous because we use it to hold data for
>
>   (1) trailers specified as command line arguments (in
>       builtin/interpret-trailers.c), and
>
>   (2) trailers specified in configuration,
>
> and these are both used to ultimately insert new trailers (based
> on the contents of arg_item, acting as a kind of template) into some
> other set of existing trailers (such as those found in a trailer block
> inside a log message) that have already been parsed.
>
> Rename "arg_item" to "trailer_template". This necessitates further
> renames to make the functions that act on these templates match the data
> structures (parameters) they act on:
>
>   - [*] add_arg_to_input_list()      to apply_template_to_trailers()
>   - [*] apply_arg_if_exists()        to maybe_add_if_exists()
>   - [*] apply_arg_if_missing()       to maybe_add_if_missing()
>   -     apply_command()              to run_command_from_template()
>   - [*] apply_item_command()         to populate_template_value()
>   -     free_arg_item()              to free_template() (non-API function)
>   -     free_new_trailers()          to free_trailer_templates() (API function)
>   -     get_conf_item()              to get_or_add_template_by()
>   -     option_parse_trailer()       to option_parse_trailer_template()
>   -     parse_trailers_from_config() to parse_trailer_templates_from_config()
>   - [*] process_trailers_lists()     to apply_trailer_templates()
>   -     token_from_item()            to token_from_template()
>   -     token_matches_item           to token_matches_template
>   - [*] trailer_add_arg_item()       to add_trailer_template()
>   -     trailer_from_arg()           to trailer_from()
>   - [*] check_if_different()         (reorder parameters only)
>   - [*] find_same_and_apply_arg()    (reorder parameters only)
>
> Reorder parameters to prefer input parameters toward the beginning and
> out parameters at the end; these functions have been marked with an
> asterisk ([*]).

That's a lot of changes in a single patch.

> This removes the "arg" terminology (standing for "CLI arguments") from
> the trailer implementation, which makes sense because trailers
> themselves have nothing to do with CLI argument handling.
>
> Also note that these renames expose the previously liberal use of
> "trailer" to mean both trailers we read from the input text (trailer
> block) and trailer templates that are defined as CLI args or
> configurations. Some functions implied a single action when they could
> do two different things, so introduce words like "maybe" and "or" to
> make their behavior more explicit.
>
> In summary this patch renames and reorders parameters for readability,
> without any behavioral change. We don't rename
> find_same_and_apply_arg(), because it will be refactored soon.
>
> For parse_trailers_from_config() (renamed to
> parse_trailer_templates_from_config()), add a NEEDSWORK discussion about
> how the deprecated trailer.*.command configuration option is oddly more
> featureful than trailer.*.cmd (if we were to remove support for
> trailer.*.command, users would not be able to replicate the behavior
> with trailer.*.cmd and would lose out on functionality).

This change could be in a separate patch. Also there were discussions
when trailer.*.command was deprecated and trailer.*.cmd introduced. I
think it might be useful to talk about them in the commit message of
the separate patch introducing the NEEDSWORK.

Re: [PATCH v4 00/28] Enrich Trailer API

[Junio C Hamano]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>>
>> This patch series is the first 10 patches of a larger cleanup/bugfix series
>> (henceforth "larger series") I've been working on.
>
> There are now 28 patches in this series.
>
> I took a look at all of them, and I think that this series should be
> split into 4 or more series.

I presume that [01-09/28] would be the first part, nothing
controversial and consisting of obvious clean-ups?  I do not mind
merging that part down to remove the future review load if everybody
agrees.

Thanks.

Re: [PATCH v4 00/28] Enrich Trailer API

[Christian Couder]

On Tue, Feb 13, 2024 at 1:11 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> Christian Couder <christian.couder@gmail.com> writes:
>
> > On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> > <gitgitgadget@gmail.com> wrote:
> >>
> >> This patch series is the first 10 patches of a larger cleanup/bugfix series
> >> (henceforth "larger series") I've been working on.
> >
> > There are now 28 patches in this series.
> >
> > I took a look at all of them, and I think that this series should be
> > split into 4 or more series.
>
> I presume that [01-09/28] would be the first part, nothing
> controversial and consisting of obvious clean-ups?  I do not mind
> merging that part down to remove the future review load if everybody
> agrees.

Yeah, patches [01-09/28] look good to me. If we are merging them
without the rest, we might want to change a bit the last sentence in
09/28 which says:

"In the next patch, we'll change format_trailer_info() to use the
parsed trailer_item objects instead of the string array."

Maybe just: s/In the next patch/In a future patch/

Re: [PATCH v4 10/28] format_trailer_info(): use trailer_item objects

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> From: Linus Arver <linusa@google.com>
>>
>> This is another preparatory refactor to unify the trailer formatters.
>>
>> Make format_trailer_info() operate on trailer_item objects, not the raw
>> string array.
>>
>> This breaks t4205 and t6300. We will continue to make improvements until
>> the test suite passes again, ultimately renaming format_trailer_info()
>> to format_trailers(), at which point the unification of these formatters
>> will be complete.
>>
>> Signed-off-by: Linus Arver <linusa@google.com>
>> ---
>>  trailer.c | 21 ++++++++++-----------
>>  1 file changed, 10 insertions(+), 11 deletions(-)
>
> I would have expected a (tentative) flip from test_expect_success to
> test_expect_failure in the affected tests, to illustrate what behaviour
> change this step introduces and why.

Somehow, such a simple idea did not cross my mind (even though I admit I
didn't like how I was breaking tests, albeit temporarily). Will update
on next reroll.

> But the huge single step in the previous round broken out into
> smaller steps like this round makes them much easier to follow.

Great, I'm doing something right for once! ;)

Re: [PATCH v4 12/28] format_trailer_info(): append newline for non-trailer lines

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>
> [...]
>
>> The test suite passes again, so format_trailer_info() is in better shape
>> supersede format_trailers(), which we'll do in the next patch.
>
> s/supersede/to supersede/

Nice catch! Will update for next reroll, thanks.

Re: [PATCH v4 14/28] format_trailer_info(): teach it about opts->trim_empty

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

>> diff --git a/trailer.c b/trailer.c
>> index f4defad3dae..c28b6c11cc5 100644
>> --- a/trailer.c
>> +++ b/trailer.c
>> @@ -162,20 +162,6 @@ static void print_tok_val(struct strbuf *out, const char *tok, const char *val)
>>                 strbuf_addf(out, "%s%c %s\n", tok, separators[0], val);
>>  }
>>
>> -static void format_trailers(const struct process_trailer_options *opts,
>> -                           struct list_head *trailers,
>> -                           struct strbuf *out)
>> -{
>> -       struct list_head *pos;
>> -       struct trailer_item *item;
>> -       list_for_each(pos, trailers) {
>> -               item = list_entry(pos, struct trailer_item, list);
>> -               if ((!opts->trim_empty || strlen(item->value) > 0) &&
>> -                   (!opts->only_trailers || item->token))
>> -                       print_tok_val(out, item->token, item->value);
>> -       }
>> -}
>
> It seems to me that this function could and should have been removed
> in the previous patch. If there is a reason why it is better to do it
> in this patch, I think it should be explained more clearly in the
> commit message.

Ah yes, the decision to delay the deletion like this was deliberate.
Will update the commit message to add something like:

    Although we could have deleted format_trailers() in the previous patch,
    we perform the deletion here like this in order to isolate
    (and highlight) in this patch the salvaging of the logic in the deleted
    code

        if ((!opts->trim_empty || strlen(item->value) > 0) && ...)
            print_tok_val(...)

    as

        if (opts->trim_empty && !strlen(item->value))
            continue;

    in the new code, which has the same effect (because we are skipping the
    formatting in the new code).

>>  static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
>>  {
>>         struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
>> @@ -1101,6 +1087,15 @@ void format_trailer_info(const struct process_trailer_options *opts,
>>                         strbuf_addstr(&tok, item->token);
>>                         strbuf_addstr(&val, item->value);
>>
>> +                       /*
>> +                        * Skip key/value pairs where the value was empty. This
>> +                        * can happen from trailers specified without a
>> +                        * separator, like `--trailer "Reviewed-by"` (no
>> +                        * corresponding value).
>> +                        */
>> +                       if (opts->trim_empty && !strlen(item->value))
>> +                               continue;
>> +
>
> Wasn't it possible to make this change in format_trailer_info() before
> using format_trailer_info() to replace format_trailers()?

It was certainly possible, but the choice to purposely time the
addition/deletion of code like this was deliberate (see my comment
above).

>>                         if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
>>                                 if (opts->separator && out->len != origlen)
>>                                         strbuf_addbuf(out, opts->separator);

Re: [PATCH v4 15/28] format_trailer_info(): avoid double-printing the separator

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>>
>> From: Linus Arver <linusa@google.com>
>>
>> Do not hardcode the printing of ": " as the separator and space (which
>> can result in double-printing these characters); instead only
>> print the separator and space if we cannot find any recognized separator
>> somewhere in the key (yes, keys may have a trailing separator in it ---
>> we will eventually fix this design but not now). Do so by copying the
>> code out of print_tok_val(), and deleting the same function.
>>
>> The test suite passes again with this change.
>
> I think it should be clearer above that this fixes a bug that was
> introduced earlier in the series.

Ack, will add something like

    This double printing is from a bug introduced earlier when we
    started using format_trailer_info() everywhere.

to this patch's description, but also add explicit language in
"trailer: begin formatting unification" to say that the change is
introducing temporary bugs (and that this is why the tests break).

> Also I wonder why it was not possible to modify format_trailer_info()
> like it is done in this patch before using it to replace
> format_trailers().

The artificial organization apparent in this patch was deliberate, in
order to make it painfully obvious exactly what was being replaced and
how. See https://lore.kernel.org/git/xmqqjzno13ev.fsf@gitster.g/

Re: [PATCH v4 14/28] format_trailer_info(): teach it about opts->trim_empty

[Christian Couder]

On Tue, Feb 13, 2024 at 6:05 PM Linus Arver <linusa@google.com> wrote:
> Christian Couder <christian.couder@gmail.com> writes:

> >>  static struct trailer_item *trailer_from_arg(struct arg_item *arg_tok)
> >>  {
> >>         struct trailer_item *new_item = xcalloc(1, sizeof(*new_item));
> >> @@ -1101,6 +1087,15 @@ void format_trailer_info(const struct process_trailer_options *opts,
> >>                         strbuf_addstr(&tok, item->token);
> >>                         strbuf_addstr(&val, item->value);
> >>
> >> +                       /*
> >> +                        * Skip key/value pairs where the value was empty. This
> >> +                        * can happen from trailers specified without a
> >> +                        * separator, like `--trailer "Reviewed-by"` (no
> >> +                        * corresponding value).
> >> +                        */
> >> +                       if (opts->trim_empty && !strlen(item->value))
> >> +                               continue;
> >> +
> >
> > Wasn't it possible to make this change in format_trailer_info() before
> > using format_trailer_info() to replace format_trailers()?
>
> It was certainly possible, but the choice to purposely time the
> addition/deletion of code like this was deliberate (see my comment
> above).

I would have thought that it would be better to make this change
earlier to avoid breaking tests.

> >>                         if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
> >>                                 if (opts->separator && out->len != origlen)
> >>                                         strbuf_addbuf(out, opts->separator);

Re: [PATCH v4 15/28] format_trailer_info(): avoid double-printing the separator

[Christian Couder]

On Tue, Feb 13, 2024 at 6:21 PM Linus Arver <linusa@google.com> wrote:
>
> Christian Couder <christian.couder@gmail.com> writes:
>
> > On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> > <gitgitgadget@gmail.com> wrote:

> > Also I wonder why it was not possible to modify format_trailer_info()
> > like it is done in this patch before using it to replace
> > format_trailers().
>
> The artificial organization apparent in this patch was deliberate, in
> order to make it painfully obvious exactly what was being replaced and
> how. See https://lore.kernel.org/git/xmqqjzno13ev.fsf@gitster.g/

As for the previous patch, I would have thought that it would be
better not to break the tests.

Re: [PATCH v4 16/28] trailer: finish formatting unification

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>
>>  /*
>> - * Format the trailers from the commit msg "msg" into the strbuf "out".
>> - * Note two caveats about "opts":
>> - *
>> - *   - this is primarily a helper for pretty.c, and not
>> - *     all of the flags are supported.
>> - *
>> - *   - this differs from process_trailers slightly in that we always format
>> - *     only the trailer block itself, even if the "only_trailers" option is not
>> - *     set.
>
> This makes me wonder if there was actually a good reason why
> format_trailers() and format_trailer_info() were 2 different
> functions. Is there info about this in the commit message of the
> commit which introduced this comment?

Good question. I see a388b10fc1 (pretty: move trailer formatting to
trailer.c, 2017-08-15) and there it says:

    pretty: move trailer formatting to trailer.c
    
    The next commit will add many features to the %(trailer)
    placeholder in pretty.c. We'll need to access some internal
    functions of trailer.c for that, so our options are either:
    
      1. expose those functions publicly
    
    or
    
      2. make an entry point into trailer.c to do the formatting
    
    Doing (2) ends up exposing less surface area, though do note
    that caveats in the docstring of the new function.

so it looks like this function started out from pretty.c and did not
have access to all of the trailer implementation internals, and was
never intended to replace (unify) the formatting machinery in trailer.c
both before and after that commit. This seems like good information to
include in the commit message, so I will do so in the next reroll.

Aside: it is interesting that the current patch series is taking the
direction of (1) instead of (2) as was done in the past (we had a choice
to "libify" at that time but did not do so, in order to "expos[e] less
surface area [in <trailer.h>]").

Re: [PATCH v4 00/28] Enrich Trailer API

[Junio C Hamano]

Christian Couder <christian.couder@gmail.com> writes:

>> > I took a look at all of them, and I think that this series should be
>> > split into 4 or more series.
>>
>> I presume that [01-09/28] would be the first part, nothing
>> controversial and consisting of obvious clean-ups?  I do not mind
>> merging that part down to remove the future review load if everybody
>> agrees.
>
> Yeah, patches [01-09/28] look good to me.

I was hoping that you'll give us more details of what the other 3 or
more you would envision the series to be, actually.

Re: [PATCH v4 19/28] trailer: make trailer_info struct private

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> "Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> This patch believes that the benefits outweight the advantages for
>> designing APIs, as explained below.
>
> "outweigh the disadvantages"?

Will update, thanks.

Re: [PATCH v4 19/28] trailer: make trailer_info struct private

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>
>> diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
>> index 11f4ce9e4a2..6bf8cec005a 100644
>> --- a/builtin/interpret-trailers.c
>> +++ b/builtin/interpret-trailers.c
>> @@ -141,7 +141,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
>>         LIST_HEAD(head);
>>         struct strbuf sb = STRBUF_INIT;
>>         struct strbuf trailer_block = STRBUF_INIT;
>> -       struct trailer_info info;
>> +       struct trailer_info *info;
>>         FILE *outfile = stdout;
>>
>>         trailer_config_init();
>> @@ -151,13 +151,13 @@ static void interpret_trailers(const struct process_trailer_options *opts,
>>         if (opts->in_place)
>>                 outfile = create_in_place_tempfile(file);
>>
>> -       parse_trailers(opts, &info, sb.buf, &head);
>> +       info = parse_trailers(opts, sb.buf, &head);
>
> I think this patch might be doing too much at once and could have been
> split into 3 or more patches to make reviews easier.

Ack, will do.

> For example the first patch could introduce trailer_info_new() and
> make interpret_trailers() use it. Then the second patch could modify
> parse_trailers() so that it returns a 'struct trailer_info *'. Then
> the third patch could make the trailer_info struct private.

Thanks, I will try to use this breakdown in the next reroll.

Re: [PATCH v4 21/28] trailer: spread usage of "trailer_block" language

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>>
>> From: Linus Arver <linusa@google.com>
>>
>> Deprecate the "trailer_info" struct name and replace it with
>> "trailer_block". The main reason is to help readability, because
>> "trailer_info" on the surface sounds like it's about a single trailer
>> when in reality it is a collection of contiguous lines, at least 25% of
>> which are trailers.
>>
>> Signed-off-by: Linus Arver <linusa@google.com>
>> ---
>>  builtin/interpret-trailers.c | 25 +++++-----
>>  trailer.c                    | 97 ++++++++++++++++++------------------
>>  trailer.h                    | 18 +++----
>>  3 files changed, 71 insertions(+), 69 deletions(-)
>>
>> diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
>> index 6bf8cec005a..f76841c5280 100644
>> --- a/builtin/interpret-trailers.c
>> +++ b/builtin/interpret-trailers.c
>> @@ -140,8 +140,8 @@ static void interpret_trailers(const struct process_trailer_options *opts,
>>  {
>>         LIST_HEAD(head);
>>         struct strbuf sb = STRBUF_INIT;
>> -       struct strbuf trailer_block = STRBUF_INIT;
>> -       struct trailer_info *info;
>> +       struct strbuf tb = STRBUF_INIT;
>> +       struct trailer_block *trailer_block;
>
> I understand that using 'trailer_block' for a 'struct trailer_block *'
> makes sense and I like the idea behind this patch, but it's
> unfortunate that 'struct strbuf trailer_block' becomes 'struct strbuf
> tb'.

I confess I did not like the name "tb" either. It does have "symmetry in
naming with 'sb' above" going for it, but that symmetry is only on the
surface because the "b" in "sb" stands for "buf" (buffer),
not "block" as is the case for "tb".

Let me try to clean up the naming scheme a bit around here.

> Also the name change for 'struct strbuf trailer_block' could be
> in a separate patch.

Will do, thanks.

Re: [PATCH v4 22/28] trailer: prepare to delete "parse_trailers_from_command_line_args()"

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>>
>> From: Linus Arver <linusa@google.com>
>>
>> Expose more functions in the trailer.h API, in preparation for deleting
>>
>>     parse_trailers_from_command_line_args()
>>
>> from the trailers implementation, because it should not be concerned
>> with command line arguments (as they have nothing to do with trailers
>> themselves). Indeed, the interpret-trailers builtin is the only caller
>> of this function inside Git.
>>
>> Rename add_arg_item() to trailer_add_arg_item() to expose it as an API
>> function. Rename new_trailers_clear() to free_new_trailers() because it
>> will be promoted into an API function; the API already has
>> free_trailers(), so using the "free_*" naming style will keep it
>> consistent. Also rename "conf_info" to "trailer_conf" for readability,
>> dropping the low-value "_info" suffix as we did earlier in this series
>> for "trailer_info" to "trailer_block".
>
> That's a lot done in a single patch. I think splitting it into 3 or
> more patches would be nice for reviewers.

Thank you for pointing this out (and, for everyone else reading, please
don't hesitate to ask for this from me); will do in the next reroll.

Re: [PATCH v4 23/28] trailer: add new helper functions to API

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>>
>> From: Linus Arver <linusa@google.com>
>>
>> This is a preparatory refactor for deprecating "new_trailer_item" from
>> the API (which will let us deprecate
>> parse_trailers_from_command_line_args()).
>>
>> Expose new helper functions from the API, because we'll be calling them
>> from interpret-trailers.c soon when we move
>> parse_trailers_from_command_line_args() there.
>>
>> Move free_new_trailers() from the builtin to trailer.c because later on
>> we will adjust it to free arg_item structs, which are private to
>> trailer.c.
>
> This patch seems to be also doing too much.

I assume you mean that you'd like for the movement of
free_new_trailers() to be separated into its own patch, separate from
the introduction of new helper functions. I agree.

Will update.

Re: [PATCH v4 26/28] trailer: unify "--trailer ..." arg handling

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>>
>> From: Linus Arver <linusa@google.com>
>>
>> Move the logic of parse_trailer_from_command_line_arg() into
>> option_parse_trailer(), because that is the only caller and there's no
>> benefit in keeping these two separate.
>
> Well one benefit could be that 2 small functions might be easier to
> understand than a big one.

True.

> So perhaps
> parse_trailer_from_command_line_arg() could just have been made
> static?

In this case I don't think keeping these two functions separate would
make sense because parse_trailer_from_command_line_arg() is much more
heavyweight than option_parse_trailer() (one is just a thin wrapper
around the other). And I didn't like the thought of having 2 function
names that look very different:

    parse_trailer_from_command_line_arg()
    option_parse_trailer()

be so closely related in behavior.

And I already have some more patches (not in this series) that refactors
this area a bit also, so I wanted to wait until the dust settled down a
bit before deciding (esp. when unit tests are added) whether keep this
function separate. It's not clear to me yet whether we do want to add
unit tests for parse_trailer_from_command_line_arg() (if we do end up
"resurrecting it" in this patch or later), so IDK. The main reason is
because I think the first set of unit tests should be for the exposed
functions in <trailer.h>, not so much the helper functions that only the
builtin uses.

Re: [PATCH v4 27/28] trailer_set_*(): put out parameter at the end

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>>
>> From: Linus Arver <linusa@google.com>
>>
>> The new trailer_config_set_*() functions which were introduced a few
>> patches ago put the out parameter (the variable being mutated) at the
>> end of the parameter list.
>>
>> Put the out parameter at the end for these functions for these existing
>
> s/for these functions for these/for the/

That wording is much better; will update.

>> trailer_set_*() functions for consistency. This also avoids confusion
>> for API users because otherwise these two sets of functions look rather
>> similar in <trailer.h> even though they have completely different out
>> parameters.

Re: [PATCH v4 28/28] trailer: introduce "template" term for readability

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>>
>> From: Linus Arver <linusa@google.com>
>>
>> The term "arg_item" is ambiguous because we use it to hold data for
>>
>>   (1) trailers specified as command line arguments (in
>>       builtin/interpret-trailers.c), and
>>
>>   (2) trailers specified in configuration,
>>
>> and these are both used to ultimately insert new trailers (based
>> on the contents of arg_item, acting as a kind of template) into some
>> other set of existing trailers (such as those found in a trailer block
>> inside a log message) that have already been parsed.
>>
>> Rename "arg_item" to "trailer_template". This necessitates further
>> renames to make the functions that act on these templates match the data
>> structures (parameters) they act on:
>>
>>   - [*] add_arg_to_input_list()      to apply_template_to_trailers()
>>   - [*] apply_arg_if_exists()        to maybe_add_if_exists()
>>   - [*] apply_arg_if_missing()       to maybe_add_if_missing()
>>   -     apply_command()              to run_command_from_template()
>>   - [*] apply_item_command()         to populate_template_value()
>>   -     free_arg_item()              to free_template() (non-API function)
>>   -     free_new_trailers()          to free_trailer_templates() (API function)
>>   -     get_conf_item()              to get_or_add_template_by()
>>   -     option_parse_trailer()       to option_parse_trailer_template()
>>   -     parse_trailers_from_config() to parse_trailer_templates_from_config()
>>   - [*] process_trailers_lists()     to apply_trailer_templates()
>>   -     token_from_item()            to token_from_template()
>>   -     token_matches_item           to token_matches_template
>>   - [*] trailer_add_arg_item()       to add_trailer_template()
>>   -     trailer_from_arg()           to trailer_from()
>>   - [*] check_if_different()         (reorder parameters only)
>>   - [*] find_same_and_apply_arg()    (reorder parameters only)
>>
>> Reorder parameters to prefer input parameters toward the beginning and
>> out parameters at the end; these functions have been marked with an
>> asterisk ([*]).
>
> That's a lot of changes in a single patch.

I confess I was not happy with the volume of the change either. I
suppose I could break things down into renames vs parameter reorderings.
Will update.

>> This removes the "arg" terminology (standing for "CLI arguments") from
>> the trailer implementation, which makes sense because trailers
>> themselves have nothing to do with CLI argument handling.
>>
>> Also note that these renames expose the previously liberal use of
>> "trailer" to mean both trailers we read from the input text (trailer
>> block) and trailer templates that are defined as CLI args or
>> configurations. Some functions implied a single action when they could
>> do two different things, so introduce words like "maybe" and "or" to
>> make their behavior more explicit.
>>
>> In summary this patch renames and reorders parameters for readability,
>> without any behavioral change. We don't rename
>> find_same_and_apply_arg(), because it will be refactored soon.
>>
>> For parse_trailers_from_config() (renamed to
>> parse_trailer_templates_from_config()), add a NEEDSWORK discussion about
>> how the deprecated trailer.*.command configuration option is oddly more
>> featureful than trailer.*.cmd (if we were to remove support for
>> trailer.*.command, users would not be able to replicate the behavior
>> with trailer.*.cmd and would lose out on functionality).
>
> This change could be in a separate patch. Also there were discussions
> when trailer.*.command was deprecated and trailer.*.cmd introduced. I
> think it might be useful to talk about them in the commit message of
> the separate patch introducing the NEEDSWORK.

Will do (will reference the commit that introduced trailer.*.cmd also),
thanks.

Re: [PATCH v4 00/28] Enrich Trailer API

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>>
>> This patch series is the first 10 patches of a larger cleanup/bugfix series
>> (henceforth "larger series") I've been working on.
>
> There are now 28 patches in this series.
>
> I took a look at all of them, and I think that this series should be
> split into 4 or more series.

This sounds fine to me. IIRC this means for the 2nd, 3rd+ series I have
to remember to say "this series builds on top of the other topic
branches '...'" in the cover letter. Now that I've written this out I
will hopefully not forget to do this...

Or, I suppose I could just introduce the 1st sub-series, wait for that
to get queued to next, then (re)introduce the 2nd sub-series, etc, in
order. Hmm. I think this will be simpler.

> For example perhaps one series until the "trailer: move
> interpret_trailers() to interpret-trailers.c" patch, then another one
> until "trailer: finish formatting unification". etc.
>
> Also I think it might be possible to avoid some test failures
> introduced by some patches. If it's not possible, I agree with Junio
> that it would be nice if the failing tests were changed to use
> 'test_expect_failure'.

As the breakages are deliberate, I will have to go with using
"test_expect_failure".

> Also it seems to me that many patches towards the end of this series
> should be split.

In hindsight, I fully agree.

Aside: I am delighted with the quality of reviews on this project. It's
not something I am used to, so please bear with me while I try to break
old habits. Thanks.

>> The main goal of this
>> series is to begin the process of "libifying" the trailer API. By "API" I
>> mean the interface exposed in trailer.h. The larger series brings a number
>> of additional cleanups (exposing and fixing some bugs along the way), and
>> builds on top of this series.
>
> [...]
>
>> With the libification-focused goals out of the way, let's turn to this patch
>> series in more detail.
>
> I like the goal of libifying Git the trailer API, and the way you want
> to do it seems reasonable to me.
>
> [...]
>
>> In summary this series breaks up "process_trailers()" into smaller pieces,
>> exposing many of the parts relevant to trailer-related processing in
>> trailer.h. This will force us to eventually introduce unit tests for these
>> API functions, but that is a good thing for API stability.
>
> I am a bit sad that this series doesn't introduce unit tests using the
> new test framework in C yet. I understand that this series is mostly a
> big refactoring and maybe it's better to introduce unit tests only
> when the refactoring is finished though.

This was my original goal as well, and still is.

> Anyway I hope the next series will introduce such tests.

I will see which API functions are stable enough, and add tests
accordingly (in a patch series sooner than later).

Probably the "biggest" (?) thing that is coming from the larger series
is the introduction of a complete separation between parsing (without
any modification of the input) and formatting. The parser/formatter is
a large chunk of the trailer implementation, so I would expect unit
tests for those bits to have to wait until those improvements are merged
into "next".

> [...]
>
>> Thanks to the aggressive refactoring in this series, I've been able to
>> identify and fix several bugs in our existing implementation. Those fixes
>> build on top of this series but were not included here, in order to keep
>> this series small. Below is a "shortlog" of those fixes I have locally:
>>
>>  * "trailer: trailer replacement should not change its position" (If we
>>    found a trailer we'd like to replace, preserve its position relative to
>>    the other trailers found in the trailer block, instead of always moving
>>    it to the beginning or end of the entire trailer block.)
>
> I believe there was a reason why it was done this way. I don't
> remember what it was though.

Noted. I'll see what I can find in our commit history.

>>  * "interpret-trailers: preserve trailers coming from the input" (Sometimes,
>>    the parsed trailers from the input will be formatted differently
>>    depending on whether we provide --only-trailers or not. Make the trailers
>>    that were not modified and which are coming directly from the input get
>>    formatted the same way, regardless of this flag.)
>
> It could be a feature to be able to normalize trailers in a certain way.

True. But doing such normalization silently is undocumented behavior,
and we should provide explicit flags for this sort of thing. Adding such
flags might be the right thing to do (let's discuss more when this patch
gets proposed). FWIW the behavior I observed is that this normalization
only happens for *some* trailers that have configuration options, not
all trailers in the input. So it's a special kind of (limited)
normalization.

>>  * "interpret-trailers: do not modify the input if NOP" (Refrain from
>>    subtracting or adding a newline around the patch divider "---" if we are
>>    not adding new trailers.)
>
> It could be a feature to be able to normalize this too.

OK, but it would again be undocumented behavior.

>>  * "trailer formatter: split up format_trailer() monolith" (Fix a bug in
>>    git-log where we still printed a blank newline even if we didn't want to
>>    format anything.)
>
> I am not sure this is a bug fix either. It could perhaps be a normalization too.

See my comment above.

>>  * "interpret-trailers: fail if given unrecognized arguments" (E.g., for
>>    "--where", only accept recognized WHERE_* enum values. If we get
>>    something unrecognized, fail with an error instead of silently doing
>>    nothing. Ditto for "--if-exists" and "--if-missing".)
>
> It's possible that there was a reason why it was done this way.
>
> I think you might want to take a look at the discussions on the
> mailing list when "interpret-trailers" was developed. There were a lot
> of discussions over a long time, and there were a lot of requests and
> suggestions about what it should do.

I confess I haven't looked too deeply into the original threads
surrounding the introduction of "interpret-trailers". But all of the
changes which I categorize as bugfixes above have a theme of
undocumented modifications.

While working on this (and the larger) series around trailers, I only
looked into some (not all) of the discussions on the mailing list in
this area. Instead, I deferred to
Documentation/git-interpret-trailers.txt as the official (authoritative)
source of truth. This is partly why I first started out on this project
last year by making improvements to that doc. And, this is why seeing so
many edge cases where we perform such undocumented modifications smelled
more like bugs to me than features.

That being said, I cannot disagree with your position that the so-called
bugfixes I've reported above could be misguided (and should rather be
just updates to missing documentation). Let's not try to decide that
here, but instead later when I get to introduce those changes on the
list, one at a time. Thanks.

Re: [PATCH v4 15/28] format_trailer_info(): avoid double-printing the separator

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 13, 2024 at 6:21 PM Linus Arver <linusa@google.com> wrote:
>>
>> Christian Couder <christian.couder@gmail.com> writes:
>>
>> > On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
>> > <gitgitgadget@gmail.com> wrote:
>
>> > Also I wonder why it was not possible to modify format_trailer_info()
>> > like it is done in this patch before using it to replace
>> > format_trailers().
>>
>> The artificial organization apparent in this patch was deliberate, in
>> order to make it painfully obvious exactly what was being replaced and
>> how. See https://lore.kernel.org/git/xmqqjzno13ev.fsf@gitster.g/
>
> As for the previous patch, I would have thought that it would be
> better not to break the tests.

I could just squash these patches together to avoid breaking tests (and
also avoid doing the flipping of expect_success to expect_fail and back
again). I don't mind at all which way we go, but now that we have these
patches broken out I wonder if it's better to just keep them that way.

Junio, do you mind if I squash the relevant changes together into just
one patch?  I'd like your input because you requested the current style
(modulo test breakages which was my error). Thanks.

Re: [PATCH v4 00/28] Enrich Trailer API

[Junio C Hamano]

Linus Arver <linusa@google.com> writes:

> This sounds fine to me. IIRC this means for the 2nd, 3rd+ series I have
> to remember to say "this series builds on top of the other topic
> branches '...'" in the cover letter. Now that I've written this out I
> will hopefully not forget to do this...
>
> Or, I suppose I could just introduce the 1st sub-series, wait for that
> to get queued to next, then (re)introduce the 2nd sub-series, etc, in
> order. Hmm. I think this will be simpler.

FWIW, which was how the recent flurry of topics related to the
reftable backend were done by Patrick, which was quite nice.  People
certainly need the feel of larger picture to get motivated to review
an earlier part of the series, but now that they saw the projected
end-game, splitting these into 4 series, each with materials in 7 or
so patches in v4, and presenting in not-so-quick succession one by
one, would make it less distracting and less daunting.

>> I am a bit sad that this series doesn't introduce unit tests using the
>> new test framework in C yet. I understand that this series is mostly a
>> big refactoring and maybe it's better to introduce unit tests only
>> when the refactoring is finished though.
>
> This was my original goal as well, and still is.

That's OK.  If v4 were 40-patch series instead of 28, I am sure you
would have had the unit-test part near the end.  So a bite sized
series of serieses may not show the unit-tests while the earlier
batches are still cooking, but as long as we all are aiming for that
same goal, we are fine.  Let's help ourselves get there soon by
reviewing each other's patches ;-).

Thanks, both.

Re: [PATCH v4 00/28] Enrich Trailer API

[Kristoffer Haugsbakk]

On Tue, Feb 13, 2024, at 20:57, Junio C Hamano wrote:
> Linus Arver <linusa@google.com> writes:
>
>> This sounds fine to me. IIRC this means for the 2nd, 3rd+ series I have
>> to remember to say "this series builds on top of the other topic
>> branches '...'" in the cover letter. Now that I've written this out I
>> will hopefully not forget to do this...
>>
>> Or, I suppose I could just introduce the 1st sub-series, wait for that
>> to get queued to next, then (re)introduce the 2nd sub-series, etc, in
>> order. Hmm. I think this will be simpler.
>
> FWIW, which was how the recent flurry of topics related to the
> reftable backend were done by Patrick, which was quite nice.  People
> certainly need the feel of larger picture to get motivated to review
> an earlier part of the series, but now that they saw the projected
> end-game, splitting these into 4 series, each with materials in 7 or
> so patches in v4, and presenting in not-so-quick succession one by
> one, would make it less distracting and less daunting.

I feel like I’m learning some things about how version control programs
(and accompanying review software (here email)) can be vital—not just
helpful, not just a nice-to-have—for structuring reviews of large
changes from observing these conversations. Thanks to both/all.

-- 
Kristoffer Haugsbakk

Re: [PATCH v4 00/28] Enrich Trailer API

[Christian Couder]

(Sorry for sending this previously to Junio only)

On Tue, Feb 13, 2024 at 6:30 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Christian Couder <christian.couder@gmail.com> writes:

> >> I presume that [01-09/28] would be the first part, nothing
> >> controversial and consisting of obvious clean-ups?  I do not mind
> >> merging that part down to remove the future review load if everybody
> >> agrees.
> >
> > Yeah, patches [01-09/28] look good to me.
>
> I was hoping that you'll give us more details of what the other 3 or
> more you would envision the series to be, actually.

I think the next one could be [10-16/28], so until "trailer: finish
formatting unification".

Then I am not sure about the next one, perhaps [17-20/28] or [17-21/28].

The rest would depend on the splitting of the big patches towards the
end of the series.

Re: [PATCH v4 15/28] format_trailer_info(): avoid double-printing the separator

[Kristoffer Haugsbakk]

On Tue, Feb 6, 2024, at 06:12, Linus Arver via GitGitGadget wrote:
> From: Linus Arver <linusa@google.com>
>
> Do not hardcode the printing of ": " as the separator and space (which
> can result in double-printing these characters); instead only
> print the separator and space if we cannot find any recognized separator
> somewhere in the key (yes, keys may have a trailing separator in it ---
> we will eventually fix this design but not now). Do so by copying the
> code out of print_tok_val(), and deleting the same function.
>
> The test suite passes again with this change.
>
> Signed-off-by: Linus Arver <linusa@google.com>

Nice find and a great commit message!

-- 
Kristoffer Haugsbakk

Re: [PATCH v4 00/28] Enrich Trailer API

[Christian Couder]

On Tue, Feb 13, 2024 at 8:39 PM Linus Arver <linusa@google.com> wrote:
>
> Christian Couder <christian.couder@gmail.com> writes:

> > I took a look at all of them, and I think that this series should be
> > split into 4 or more series.
>
> This sounds fine to me. IIRC this means for the 2nd, 3rd+ series I have
> to remember to say "this series builds on top of the other topic
> branches '...'" in the cover letter. Now that I've written this out I
> will hopefully not forget to do this...
>
> Or, I suppose I could just introduce the 1st sub-series, wait for that
> to get queued to next, then (re)introduce the 2nd sub-series, etc, in
> order. Hmm. I think this will be simpler.

Yeah, sure.

> > Also it seems to me that many patches towards the end of this series
> > should be split.
>
> In hindsight, I fully agree.
>
> Aside: I am delighted with the quality of reviews on this project. It's
> not something I am used to, so please bear with me while I try to break
> old habits.

Sure no worries.

> Thanks.

[...]

> > I am a bit sad that this series doesn't introduce unit tests using the
> > new test framework in C yet. I understand that this series is mostly a
> > big refactoring and maybe it's better to introduce unit tests only
> > when the refactoring is finished though.
>
> This was my original goal as well, and still is.
>
> > Anyway I hope the next series will introduce such tests.
>
> I will see which API functions are stable enough, and add tests
> accordingly (in a patch series sooner than later).
>
> Probably the "biggest" (?) thing that is coming from the larger series
> is the introduction of a complete separation between parsing (without
> any modification of the input) and formatting. The parser/formatter is
> a large chunk of the trailer implementation, so I would expect unit
> tests for those bits to have to wait until those improvements are merged
> into "next".

Ok.

> >> Thanks to the aggressive refactoring in this series, I've been able to
> >> identify and fix several bugs in our existing implementation. Those fixes
> >> build on top of this series but were not included here, in order to keep
> >> this series small. Below is a "shortlog" of those fixes I have locally:
> >>
> >>  * "trailer: trailer replacement should not change its position" (If we
> >>    found a trailer we'd like to replace, preserve its position relative to
> >>    the other trailers found in the trailer block, instead of always moving
> >>    it to the beginning or end of the entire trailer block.)
> >
> > I believe there was a reason why it was done this way. I don't
> > remember what it was though.
>
> Noted. I'll see what I can find in our commit history.
>
> >>  * "interpret-trailers: preserve trailers coming from the input" (Sometimes,
> >>    the parsed trailers from the input will be formatted differently
> >>    depending on whether we provide --only-trailers or not. Make the trailers
> >>    that were not modified and which are coming directly from the input get
> >>    formatted the same way, regardless of this flag.)
> >
> > It could be a feature to be able to normalize trailers in a certain way.
>
> True. But doing such normalization silently is undocumented behavior,
> and we should provide explicit flags for this sort of thing. Adding such
> flags might be the right thing to do (let's discuss more when this patch
> gets proposed). FWIW the behavior I observed is that this normalization
> only happens for *some* trailers that have configuration options, not
> all trailers in the input. So it's a special kind of (limited)
> normalization.

Perhaps because we consider that having some configuration means that
the user consistently expects things in a certain way.

> >>  * "interpret-trailers: fail if given unrecognized arguments" (E.g., for
> >>    "--where", only accept recognized WHERE_* enum values. If we get
> >>    something unrecognized, fail with an error instead of silently doing
> >>    nothing. Ditto for "--if-exists" and "--if-missing".)
> >
> > It's possible that there was a reason why it was done this way.
> >
> > I think you might want to take a look at the discussions on the
> > mailing list when "interpret-trailers" was developed. There were a lot
> > of discussions over a long time, and there were a lot of requests and
> > suggestions about what it should do.
>
> I confess I haven't looked too deeply into the original threads
> surrounding the introduction of "interpret-trailers". But all of the
> changes which I categorize as bugfixes above have a theme of
> undocumented modifications.
>
> While working on this (and the larger) series around trailers, I only
> looked into some (not all) of the discussions on the mailing list in
> this area. Instead, I deferred to
> Documentation/git-interpret-trailers.txt as the official (authoritative)
> source of truth. This is partly why I first started out on this project
> last year by making improvements to that doc. And, this is why seeing so
> many edge cases where we perform such undocumented modifications smelled
> more like bugs to me than features.
>
> That being said, I cannot disagree with your position that the so-called
> bugfixes I've reported above could be misguided (and should rather be
> just updates to missing documentation). Let's not try to decide that
> here, but instead later when I get to introduce those changes on the
> list, one at a time. Thanks.

Yeah, it might seem like undocumented features are bad, and I agree
that reading original discussions can be tiring. But if the latter
makes it possible to fix undocumented features by just properly
documenting them, then I think it might just be the best thing to do.
Ok not to decide about it now though.

Re: [PATCH v4 00/28] Enrich Trailer API

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Feb 13, 2024 at 8:39 PM Linus Arver <linusa@google.com> wrote:
>>
>> Christian Couder <christian.couder@gmail.com> writes:
>
> [...]
> 
>> >> Thanks to the aggressive refactoring in this series, I've been able to
>> >> identify and fix several bugs in our existing implementation. Those fixes
>> >> build on top of this series but were not included here, in order to keep
>> >> this series small. Below is a "shortlog" of those fixes I have locally:
>> >>
>> >>  * "trailer: trailer replacement should not change its position" (If we
>> >>    found a trailer we'd like to replace, preserve its position relative to
>> >>    the other trailers found in the trailer block, instead of always moving
>> >>    it to the beginning or end of the entire trailer block.)
>> >
>> > I believe there was a reason why it was done this way. I don't
>> > remember what it was though.
>>
>> Noted. I'll see what I can find in our commit history.
>>
>> >>  * "interpret-trailers: preserve trailers coming from the input" (Sometimes,
>> >>    the parsed trailers from the input will be formatted differently
>> >>    depending on whether we provide --only-trailers or not. Make the trailers
>> >>    that were not modified and which are coming directly from the input get
>> >>    formatted the same way, regardless of this flag.)
>> >
>> > It could be a feature to be able to normalize trailers in a certain way.
>>
>> True. But doing such normalization silently is undocumented behavior,
>> and we should provide explicit flags for this sort of thing. Adding such
>> flags might be the right thing to do (let's discuss more when this patch
>> gets proposed). FWIW the behavior I observed is that this normalization
>> only happens for *some* trailers that have configuration options, not
>> all trailers in the input. So it's a special kind of (limited)
>> normalization.
>
> Perhaps because we consider that having some configuration means that
> the user consistently expects things in a certain way.

Yes, this was one possibility I considered after sending my reply. If a
user has gone out of their way to configure something, maybe they do
want things (for those bits) to be normalized.

And adding a flag to disable normalization seems like a good feature to
have also (while keeping the behavior of the interpret-trailers that has
been relatively untouched since its introduction). But anyway I'm
getting a little bit ahead of myself.

>> >>  * "interpret-trailers: fail if given unrecognized arguments" (E.g., for
>> >>    "--where", only accept recognized WHERE_* enum values. If we get
>> >>    something unrecognized, fail with an error instead of silently doing
>> >>    nothing. Ditto for "--if-exists" and "--if-missing".)
>> >
>> > It's possible that there was a reason why it was done this way.
>> >
>> > I think you might want to take a look at the discussions on the
>> > mailing list when "interpret-trailers" was developed. There were a lot
>> > of discussions over a long time, and there were a lot of requests and
>> > suggestions about what it should do.
>>
>> I confess I haven't looked too deeply into the original threads
>> surrounding the introduction of "interpret-trailers". But all of the
>> changes which I categorize as bugfixes above have a theme of
>> undocumented modifications.
>>
>> While working on this (and the larger) series around trailers, I only
>> looked into some (not all) of the discussions on the mailing list in
>> this area. Instead, I deferred to
>> Documentation/git-interpret-trailers.txt as the official (authoritative)
>> source of truth. This is partly why I first started out on this project
>> last year by making improvements to that doc. And, this is why seeing so
>> many edge cases where we perform such undocumented modifications smelled
>> more like bugs to me than features.
>>
>> That being said, I cannot disagree with your position that the so-called
>> bugfixes I've reported above could be misguided (and should rather be
>> just updates to missing documentation). Let's not try to decide that
>> here, but instead later when I get to introduce those changes on the
>> list, one at a time. Thanks.
>
> Yeah, it might seem like undocumented features are bad, and I agree
> that reading original discussions can be tiring. But if the latter
> makes it possible to fix undocumented features by just properly
> documenting them, then I think it might just be the best thing to do.
> Ok not to decide about it now though.

Thanks!

Re: [PATCH v4 00/28] Enrich Trailer API

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> (Sorry for sending this previously to Junio only)
>
> On Tue, Feb 13, 2024 at 6:30 PM Junio C Hamano <gitster@pobox.com> wrote:
>>
>> Christian Couder <christian.couder@gmail.com> writes:
>
>> >> I presume that [01-09/28] would be the first part, nothing
>> >> controversial and consisting of obvious clean-ups?  I do not mind
>> >> merging that part down to remove the future review load if everybody
>> >> agrees.
>> >
>> > Yeah, patches [01-09/28] look good to me.
>>
>> I was hoping that you'll give us more details of what the other 3 or
>> more you would envision the series to be, actually.
>
> I think the next one could be [10-16/28], so until "trailer: finish
> formatting unification".
>
> Then I am not sure about the next one, perhaps [17-20/28] or [17-21/28].
>
> The rest would depend on the splitting of the big patches towards the
> end of the series.

Ack, I'll try to group them like this. Thanks.

[PATCH v5 0/9] Enrich Trailer API

[Linus Arver via GitGitGadget]

This patch series is the first 9 patches of a larger cleanup/bugfix series
(henceforth "larger series") I've been working on. The main goal of this
series is to begin the process of "libifying" the trailer API. By "API" I
mean the interface exposed in trailer.h. The larger series brings a number
of additional cleanups (exposing and fixing some bugs along the way), and
builds on top of this series.

When the larger series is merged, we will be in a good state to additionally
pursue the following goals:

 1. "API reuse inside Git": make the API expressive enough to eliminate any
    need by other parts of Git to use the interpret-trailers builtin as a
    subprocess (instead they could just use the API directly);
 2. "API stability": add unit tests to codify the expected behavior of API
    functions; and
 3. "API documentation": create developer-focused documentation to explain
    how to use the API effectively, noting any API limitations or
    anti-patterns.

In the future after libification is "complete", users external to Git will
be able to use the same trailer processing API used by the
interpret-trailers builtin. For example, a web server may want to parse
trailers the same way that Git would parse them, without having to call
interpret-trailers as a subprocess. This use case was the original
motivation behind my work in this area.

With the libification-focused goals out of the way, let's turn to this patch
series in more detail.

In summary this series breaks up "process_trailers()" into smaller pieces,
exposing many of the parts relevant to trailer-related processing in
trailer.h. This will force us to eventually introduce unit tests for these
API functions, but that is a good thing for API stability. We also perform
some preparatory refactors in order to help us unify the trailer formatting
machinery toward the end of this series.


Notable changes in v5
=====================

 * Removed patches 10+ from this series. Thanks to Christian for the
   suggestion.
 * Reworded the log message of patch 09 to reflect the above arrangement, as
   suggested by Christian.


Notable changes in v4
=====================

 * Patches 3, 4, 5, and 8 have been broken up into smaller steps. There are
   28 instead of 10 patches now, but these 28 should be much easier to
   review than the (previously condensed) 10.
 * NEW Patch 1: "trailer: free trailer_info after all related usage" fixes
   awkward use-after-free coding style
 * NEW Patch 2: "shortlog: add test for de-duplicating folded trailers"
   increases test coverage related to trailer iterators and "unfold_value()"
 * NEW Patch 27: "trailer_set_*(): put out parameter at the end" is a small
   refactor to reorder parameters.
 * Patches 5-16: These smaller patches make up Patch 3 from v3.
 * Patches 17-18: These smaller patches make up Patch 4 from v3.
 * Patches 19-20: These smaller patches make up Patch 5 from v3.
 * Patches 23-26: These smaller patches make up Patch 8 from v3.
 * Anonymize unambiguous parameters in <trailer.h>.


Notable changes in v3
=====================

 * Squashed Patch 4 into Patch 3 ("trailer: unify trailer formatting
   machinery"), to avoid breaking the build ("-Werror=unused-function"
   violations)
 * NEW (Patch 10): Introduce "trailer template" terminology for readability
   (no behavioral change)
 * (API function) Rename default_separators() to
   trailer_default_separators()
 * (API function) Rename new_trailers_clear() to free_trailer_templates()
 * trailer.h: for single-parameter functions, anonymize the parameter name
   to reduce verbosity


Notable changes in v2
=====================

 * (cover letter) Discuss goals of the larger series in more detail,
   especially the pimpl idiom
 * (cover letter) List bug fixes pending in the larger series that depend on
   this series
 * Reorder function parameters to have trailer options at the beginning (and
   out parameters toward the end)
 * "sequencer: use the trailer iterator": prefer C string instead of strbuf
   for new "raw" field
 * Patch 1 (was Patch 2) also renames ensure_configured() to
   trailer_config_init() (forgot to rename this one previously)

Linus Arver (9):
  trailer: free trailer_info _after_ all related usage
  shortlog: add test for de-duplicating folded trailers
  trailer: prepare to expose functions as part of API
  trailer: move interpret_trailers() to interpret-trailers.c
  trailer: start preparing for formatting unification
  trailer_info_get(): reorder parameters
  format_trailers(): use strbuf instead of FILE
  format_trailer_info(): move "fast path" to caller
  format_trailers_from_commit(): indirectly call trailer_info_get()

 builtin/interpret-trailers.c | 101 ++++++++++++++++++-
 pretty.c                     |   2 +-
 ref-filter.c                 |   2 +-
 sequencer.c                  |   2 +-
 t/t4201-shortlog.sh          |  32 +++++++
 trailer.c                    | 181 +++++++++--------------------------
 trailer.h                    |  31 ++++--
 7 files changed, 204 insertions(+), 147 deletions(-)


base-commit: a54a84b333adbecf7bc4483c0e36ed5878cac17b
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1632%2Flistx%2Ftrailer-api-refactor-part-1-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1632/listx/trailer-api-refactor-part-1-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/1632

Range-diff vs v4:

  1:  652df25f30e =  1:  652df25f30e trailer: free trailer_info _after_ all related usage
  2:  fdccaca2ba0 =  2:  fdccaca2ba0 shortlog: add test for de-duplicating folded trailers
  3:  4372af244f0 =  3:  4372af244f0 trailer: prepare to expose functions as part of API
  4:  4073b8eb510 =  4:  4073b8eb510 trailer: move interpret_trailers() to interpret-trailers.c
  5:  b2a0f7829a1 =  5:  b2a0f7829a1 trailer: start preparing for formatting unification
  6:  c1760f80356 =  6:  c1760f80356 trailer_info_get(): reorder parameters
  7:  9dc912b5bc5 =  7:  9dc912b5bc5 format_trailers(): use strbuf instead of FILE
  8:  b97c06d8bc3 =  8:  b97c06d8bc3 format_trailer_info(): move "fast path" to caller
  9:  6906910417a !  9:  7c656b3f775 format_trailers_from_commit(): indirectly call trailer_info_get()
     @@ Commit message
          format_trailer_info() only looks at the "trailers" string array, not the
          trailer_item objects which parse_trailers() populates.
      
     -    In the next patch, we'll change format_trailer_info() to use the parsed
     +    In a future patch, we'll change format_trailer_info() to use the parsed
          trailer_item objects instead of the string array.
      
          Signed-off-by: Linus Arver <linusa@google.com>
 10:  f5b7ba08aa7 <  -:  ----------- format_trailer_info(): use trailer_item objects
 11:  457f2a839d5 <  -:  ----------- format_trailer_info(): drop redundant unfold_value()
 12:  a72eca301f7 <  -:  ----------- format_trailer_info(): append newline for non-trailer lines
 13:  ad77c33e457 <  -:  ----------- trailer: begin formatting unification
 14:  11f854399db <  -:  ----------- format_trailer_info(): teach it about opts->trim_empty
 15:  ba1f387747b <  -:  ----------- format_trailer_info(): avoid double-printing the separator
 16:  31725832224 <  -:  ----------- trailer: finish formatting unification
 17:  6f17c022b15 <  -:  ----------- trailer: teach iterator about non-trailer lines
 18:  cc92dfb0bda <  -:  ----------- sequencer: use the trailer iterator
 19:  f5f0d06613f <  -:  ----------- trailer: make trailer_info struct private
 20:  607ae7a90cd <  -:  ----------- trailer: retire trailer_info_get() from API
 21:  38f4b4c4135 <  -:  ----------- trailer: spread usage of "trailer_block" language
 22:  94bf182e3ff <  -:  ----------- trailer: prepare to delete "parse_trailers_from_command_line_args()"
 23:  3bfe4809ecb <  -:  ----------- trailer: add new helper functions to API
 24:  80e1958bb8d <  -:  ----------- trailer_add_arg_item(): drop new_trailer_item usage
 25:  a9080597a28 <  -:  ----------- trailer: deprecate "new_trailer_item" struct from API
 26:  9720526dd8a <  -:  ----------- trailer: unify "--trailer ..." arg handling
 27:  26df2514acb <  -:  ----------- trailer_set_*(): put out parameter at the end
 28:  14927038d85 <  -:  ----------- trailer: introduce "template" term for readability

-- 
gitgitgadget

[PATCH v5 1/9] trailer: free trailer_info _after_ all related usage

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

In de7c27a186 (trailer: use offsets for trailer_start/trailer_end,
2023-10-20), we started using trailer block offsets in trailer_info. In
particular, we dropped the use of a separate stack variable "size_t
trailer_end", in favor of accessing the new "trailer_block_end" member
of trailer_info (as "info.trailer_block_end").

At that time, we forgot to also move the

   trailer_info_release(&info);

line to be _after_ this new use of the trailer_info struct. Move it now.

Note that even without this patch, we didn't have leaks or any other
problems because trailer_info_release() only frees memory allocated on
the heap. The "trailer_block_end" member was allocated on the stack back
then (as it is now) so it was still safe to use for all this time.

Reported-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/trailer.c b/trailer.c
index 3a0710a4583..e1d83390b66 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1111,13 +1111,12 @@ void process_trailers(const char *file,
 	}
 
 	print_all(outfile, &head, opts);
-
 	free_all(&head);
-	trailer_info_release(&info);
 
 	/* Print the lines after the trailers as is */
 	if (!opts->only_trailers)
 		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
+	trailer_info_release(&info);
 
 	if (opts->in_place)
 		if (rename_tempfile(&trailers_tempfile, file))
-- 
gitgitgadget

[PATCH v5 2/9] shortlog: add test for de-duplicating folded trailers

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

The shortlog builtin was taught to use the trailer iterator interface in
47beb37bc6 (shortlog: match commit trailers with --group, 2020-09-27).
The iterator always unfolds values and this has always been the case
since the time the iterator was first introduced in f0939a0eb1 (trailer:
add interface for iterating over commit trailers, 2020-09-27). Add a
comment line to remind readers of this behavior.

The fact that the iterator always unfolds values is important
(at least for shortlog) because unfolding allows it to recognize both
folded and unfolded versions of the same trailer for de-duplication.

Capture the existing behavior in a new test case to guard against
regressions in this area. This test case is based off of the existing
"shortlog de-duplicates trailers in a single commit" just above it. Now
if we were to remove the call to

    unfold_value(&iter->val);

inside the iterator, this new test case will break.

Signed-off-by: Linus Arver <linusa@google.com>
---
 t/t4201-shortlog.sh | 32 ++++++++++++++++++++++++++++++++
 trailer.c           |  1 +
 2 files changed, 33 insertions(+)

diff --git a/t/t4201-shortlog.sh b/t/t4201-shortlog.sh
index d7382709fc1..f698d0c9ad2 100755
--- a/t/t4201-shortlog.sh
+++ b/t/t4201-shortlog.sh
@@ -312,6 +312,38 @@ test_expect_success 'shortlog de-duplicates trailers in a single commit' '
 	test_cmp expect actual
 '
 
+# Trailers that have unfolded (single line) and folded (multiline) values which
+# are otherwise identical are treated as the same trailer for de-duplication.
+test_expect_success 'shortlog de-duplicates trailers in a single commit (folded/unfolded values)' '
+	git commit --allow-empty -F - <<-\EOF &&
+	subject one
+
+	this message has two distinct values, plus a repeat (folded)
+
+	Repeated-trailer: Foo foo foo
+	Repeated-trailer: Bar
+	Repeated-trailer: Foo
+	  foo foo
+	EOF
+
+	git commit --allow-empty -F - <<-\EOF &&
+	subject two
+
+	similar to the previous, but without the second distinct value
+
+	Repeated-trailer: Foo foo foo
+	Repeated-trailer: Foo
+	  foo foo
+	EOF
+
+	cat >expect <<-\EOF &&
+	     2	Foo foo foo
+	     1	Bar
+	EOF
+	git shortlog -ns --group=trailer:repeated-trailer -2 HEAD >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'shortlog can match multiple groups' '
 	git commit --allow-empty -F - <<-\EOF &&
 	subject one
diff --git a/trailer.c b/trailer.c
index e1d83390b66..f74915bd8cd 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1270,6 +1270,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 		strbuf_reset(&iter->val);
 		parse_trailer(&iter->key, &iter->val, NULL,
 			      trailer, separator_pos);
+		/* Always unfold values during iteration. */
 		unfold_value(&iter->val);
 		return 1;
 	}
-- 
gitgitgadget

[PATCH v5 3/9] trailer: prepare to expose functions as part of API

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

In the next patch, we will move "process_trailers" from trailer.c to
builtin/interpret-trailers.c. That move will necessitate the growth of
the trailer.h API, forcing us to expose some additional functions in
trailer.h.

Rename relevant functions so that they include the term "trailer" in
their name, so that clients of the API will be able to easily identify
them by their "trailer" moniker, just like all the other functions
already exposed by trailer.h.

Take the opportunity to start putting trailer processing options (opts)
as the first parameter. This will be the pattern going forward in this
series.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  4 ++--
 trailer.c                    | 26 +++++++++++++-------------
 trailer.h                    |  6 +++---
 3 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 033bd1556cf..85a3413baf5 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -132,11 +132,11 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 	if (argc) {
 		int i;
 		for (i = 0; i < argc; i++)
-			process_trailers(argv[i], &opts, &trailers);
+			interpret_trailers(&opts, &trailers, argv[i]);
 	} else {
 		if (opts.in_place)
 			die(_("no input file given for in-place editing"));
-		process_trailers(NULL, &opts, &trailers);
+		interpret_trailers(&opts, &trailers, NULL);
 	}
 
 	new_trailers_clear(&trailers);
diff --git a/trailer.c b/trailer.c
index f74915bd8cd..916175707d8 100644
--- a/trailer.c
+++ b/trailer.c
@@ -163,12 +163,12 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-static void print_all(FILE *outfile, struct list_head *head,
-		      const struct process_trailer_options *opts)
+static void format_trailers(const struct process_trailer_options *opts,
+			    struct list_head *trailers, FILE *outfile)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
-	list_for_each(pos, head) {
+	list_for_each(pos, trailers) {
 		item = list_entry(pos, struct trailer_item, list);
 		if ((!opts->trim_empty || strlen(item->value) > 0) &&
 		    (!opts->only_trailers || item->token))
@@ -589,7 +589,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	return 0;
 }
 
-static void ensure_configured(void)
+static void trailer_config_init(void)
 {
 	if (configured)
 		return;
@@ -1035,10 +1035,10 @@ static void parse_trailers(struct trailer_info *info,
 	}
 }
 
-static void free_all(struct list_head *head)
+static void free_trailers(struct list_head *trailers)
 {
 	struct list_head *pos, *p;
-	list_for_each_safe(pos, p, head) {
+	list_for_each_safe(pos, p, trailers) {
 		list_del(pos);
 		free_trailer_item(list_entry(pos, struct trailer_item, list));
 	}
@@ -1075,16 +1075,16 @@ static FILE *create_in_place_tempfile(const char *file)
 	return outfile;
 }
 
-void process_trailers(const char *file,
-		      const struct process_trailer_options *opts,
-		      struct list_head *new_trailer_head)
+void interpret_trailers(const struct process_trailer_options *opts,
+			struct list_head *new_trailer_head,
+			const char *file)
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
 	struct trailer_info info;
 	FILE *outfile = stdout;
 
-	ensure_configured();
+	trailer_config_init();
 
 	read_input_file(&sb, file);
 
@@ -1110,8 +1110,8 @@ void process_trailers(const char *file,
 		process_trailers_lists(&head, &arg_head);
 	}
 
-	print_all(outfile, &head, opts);
-	free_all(&head);
+	format_trailers(opts, &head, outfile);
+	free_trailers(&head);
 
 	/* Print the lines after the trailers as is */
 	if (!opts->only_trailers)
@@ -1134,7 +1134,7 @@ void trailer_info_get(struct trailer_info *info, const char *str,
 	size_t nr = 0, alloc = 0;
 	char **last = NULL;
 
-	ensure_configured();
+	trailer_config_init();
 
 	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
 	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
diff --git a/trailer.h b/trailer.h
index 1644cd05f60..37033e631a1 100644
--- a/trailer.h
+++ b/trailer.h
@@ -81,9 +81,9 @@ struct process_trailer_options {
 
 #define PROCESS_TRAILER_OPTIONS_INIT {0}
 
-void process_trailers(const char *file,
-		      const struct process_trailer_options *opts,
-		      struct list_head *new_trailer_head);
+void interpret_trailers(const struct process_trailer_options *opts,
+			struct list_head *new_trailer_head,
+			const char *file);
 
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts);
-- 
gitgitgadget

[PATCH v5 4/9] trailer: move interpret_trailers() to interpret-trailers.c

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

The interpret-trailers.c builtin is the only place we need to call
interpret_trailers(), so move its definition there (together with a few
helper functions called only by it) and remove its external declaration
from <trailer.h>.

Several helper functions that are called by interpret_trailers() remain
in trailer.c because other callers in the same file still call them.
Declare them in <trailer.h> so that interpret_trailers() (now in
builtin/interpret-trailers.c) can continue calling them as a trailer API
user.

This enriches <trailer.h> with a more granular API, which can then be
unit-tested in the future (because interpret_trailers() by itself does
too many things to be able to be easily unit-tested).

Take this opportunity to demote some file-handling functions out of the
trailer API implementation, as these have nothing to do with trailers.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  93 +++++++++++++++++++++++++++
 trailer.c                    | 119 ++++-------------------------------
 trailer.h                    |  20 +++++-
 3 files changed, 123 insertions(+), 109 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 85a3413baf5..d1cf0aa33a2 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -9,6 +9,7 @@
 #include "gettext.h"
 #include "parse-options.h"
 #include "string-list.h"
+#include "tempfile.h"
 #include "trailer.h"
 #include "config.h"
 
@@ -91,6 +92,98 @@ static int parse_opt_parse(const struct option *opt, const char *arg,
 	return 0;
 }
 
+static struct tempfile *trailers_tempfile;
+
+static FILE *create_in_place_tempfile(const char *file)
+{
+	struct stat st;
+	struct strbuf filename_template = STRBUF_INIT;
+	const char *tail;
+	FILE *outfile;
+
+	if (stat(file, &st))
+		die_errno(_("could not stat %s"), file);
+	if (!S_ISREG(st.st_mode))
+		die(_("file %s is not a regular file"), file);
+	if (!(st.st_mode & S_IWUSR))
+		die(_("file %s is not writable by user"), file);
+
+	/* Create temporary file in the same directory as the original */
+	tail = strrchr(file, '/');
+	if (tail)
+		strbuf_add(&filename_template, file, tail - file + 1);
+	strbuf_addstr(&filename_template, "git-interpret-trailers-XXXXXX");
+
+	trailers_tempfile = xmks_tempfile_m(filename_template.buf, st.st_mode);
+	strbuf_release(&filename_template);
+	outfile = fdopen_tempfile(trailers_tempfile, "w");
+	if (!outfile)
+		die_errno(_("could not open temporary file"));
+
+	return outfile;
+}
+
+static void read_input_file(struct strbuf *sb, const char *file)
+{
+	if (file) {
+		if (strbuf_read_file(sb, file, 0) < 0)
+			die_errno(_("could not read input file '%s'"), file);
+	} else {
+		if (strbuf_read(sb, fileno(stdin), 0) < 0)
+			die_errno(_("could not read from stdin"));
+	}
+}
+
+static void interpret_trailers(const struct process_trailer_options *opts,
+			       struct list_head *new_trailer_head,
+			       const char *file)
+{
+	LIST_HEAD(head);
+	struct strbuf sb = STRBUF_INIT;
+	struct trailer_info info;
+	FILE *outfile = stdout;
+
+	trailer_config_init();
+
+	read_input_file(&sb, file);
+
+	if (opts->in_place)
+		outfile = create_in_place_tempfile(file);
+
+	parse_trailers(opts, &info, sb.buf, &head);
+
+	/* Print the lines before the trailers */
+	if (!opts->only_trailers)
+		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
+
+	if (!opts->only_trailers && !info.blank_line_before_trailer)
+		fprintf(outfile, "\n");
+
+
+	if (!opts->only_input) {
+		LIST_HEAD(config_head);
+		LIST_HEAD(arg_head);
+		parse_trailers_from_config(&config_head);
+		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
+		list_splice(&config_head, &arg_head);
+		process_trailers_lists(&head, &arg_head);
+	}
+
+	format_trailers(opts, &head, outfile);
+	free_trailers(&head);
+
+	/* Print the lines after the trailers as is */
+	if (!opts->only_trailers)
+		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
+	trailer_info_release(&info);
+
+	if (opts->in_place)
+		if (rename_tempfile(&trailers_tempfile, file))
+			die_errno(_("could not rename temporary file to %s"), file);
+
+	strbuf_release(&sb);
+}
+
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
diff --git a/trailer.c b/trailer.c
index 916175707d8..d23afa0a65c 100644
--- a/trailer.c
+++ b/trailer.c
@@ -5,7 +5,6 @@
 #include "string-list.h"
 #include "run-command.h"
 #include "commit.h"
-#include "tempfile.h"
 #include "trailer.h"
 #include "list.h"
 /*
@@ -163,8 +162,8 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-static void format_trailers(const struct process_trailer_options *opts,
-			    struct list_head *trailers, FILE *outfile)
+void format_trailers(const struct process_trailer_options *opts,
+		     struct list_head *trailers, FILE *outfile)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
@@ -366,8 +365,8 @@ static int find_same_and_apply_arg(struct list_head *head,
 	return 0;
 }
 
-static void process_trailers_lists(struct list_head *head,
-				   struct list_head *arg_head)
+void process_trailers_lists(struct list_head *head,
+			    struct list_head *arg_head)
 {
 	struct list_head *pos, *p;
 	struct arg_item *arg_tok;
@@ -589,7 +588,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	return 0;
 }
 
-static void trailer_config_init(void)
+void trailer_config_init(void)
 {
 	if (configured)
 		return;
@@ -719,7 +718,7 @@ static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
 	list_add_tail(&new_item->list, arg_head);
 }
 
-static void parse_trailers_from_config(struct list_head *config_head)
+void parse_trailers_from_config(struct list_head *config_head)
 {
 	struct arg_item *item;
 	struct list_head *pos;
@@ -735,8 +734,8 @@ static void parse_trailers_from_config(struct list_head *config_head)
 	}
 }
 
-static void parse_trailers_from_command_line_args(struct list_head *arg_head,
-						  struct list_head *new_trailer_head)
+void parse_trailers_from_command_line_args(struct list_head *arg_head,
+					   struct list_head *new_trailer_head)
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
@@ -775,17 +774,6 @@ static void parse_trailers_from_command_line_args(struct list_head *arg_head,
 	free(cl_separators);
 }
 
-static void read_input_file(struct strbuf *sb, const char *file)
-{
-	if (file) {
-		if (strbuf_read_file(sb, file, 0) < 0)
-			die_errno(_("could not read input file '%s'"), file);
-	} else {
-		if (strbuf_read(sb, fileno(stdin), 0) < 0)
-			die_errno(_("could not read from stdin"));
-	}
-}
-
 static const char *next_line(const char *str)
 {
 	const char *nl = strchrnul(str, '\n');
@@ -1000,10 +988,10 @@ static void unfold_value(struct strbuf *val)
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
  */
-static void parse_trailers(struct trailer_info *info,
-			     const char *str,
-			     struct list_head *head,
-			     const struct process_trailer_options *opts)
+void parse_trailers(const struct process_trailer_options *opts,
+		    struct trailer_info *info,
+		    const char *str,
+		    struct list_head *head)
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
@@ -1035,7 +1023,7 @@ static void parse_trailers(struct trailer_info *info,
 	}
 }
 
-static void free_trailers(struct list_head *trailers)
+void free_trailers(struct list_head *trailers)
 {
 	struct list_head *pos, *p;
 	list_for_each_safe(pos, p, trailers) {
@@ -1044,87 +1032,6 @@ static void free_trailers(struct list_head *trailers)
 	}
 }
 
-static struct tempfile *trailers_tempfile;
-
-static FILE *create_in_place_tempfile(const char *file)
-{
-	struct stat st;
-	struct strbuf filename_template = STRBUF_INIT;
-	const char *tail;
-	FILE *outfile;
-
-	if (stat(file, &st))
-		die_errno(_("could not stat %s"), file);
-	if (!S_ISREG(st.st_mode))
-		die(_("file %s is not a regular file"), file);
-	if (!(st.st_mode & S_IWUSR))
-		die(_("file %s is not writable by user"), file);
-
-	/* Create temporary file in the same directory as the original */
-	tail = strrchr(file, '/');
-	if (tail)
-		strbuf_add(&filename_template, file, tail - file + 1);
-	strbuf_addstr(&filename_template, "git-interpret-trailers-XXXXXX");
-
-	trailers_tempfile = xmks_tempfile_m(filename_template.buf, st.st_mode);
-	strbuf_release(&filename_template);
-	outfile = fdopen_tempfile(trailers_tempfile, "w");
-	if (!outfile)
-		die_errno(_("could not open temporary file"));
-
-	return outfile;
-}
-
-void interpret_trailers(const struct process_trailer_options *opts,
-			struct list_head *new_trailer_head,
-			const char *file)
-{
-	LIST_HEAD(head);
-	struct strbuf sb = STRBUF_INIT;
-	struct trailer_info info;
-	FILE *outfile = stdout;
-
-	trailer_config_init();
-
-	read_input_file(&sb, file);
-
-	if (opts->in_place)
-		outfile = create_in_place_tempfile(file);
-
-	parse_trailers(&info, sb.buf, &head, opts);
-
-	/* Print the lines before the trailers */
-	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
-
-	if (!opts->only_trailers && !info.blank_line_before_trailer)
-		fprintf(outfile, "\n");
-
-
-	if (!opts->only_input) {
-		LIST_HEAD(config_head);
-		LIST_HEAD(arg_head);
-		parse_trailers_from_config(&config_head);
-		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
-		list_splice(&config_head, &arg_head);
-		process_trailers_lists(&head, &arg_head);
-	}
-
-	format_trailers(opts, &head, outfile);
-	free_trailers(&head);
-
-	/* Print the lines after the trailers as is */
-	if (!opts->only_trailers)
-		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
-	trailer_info_release(&info);
-
-	if (opts->in_place)
-		if (rename_tempfile(&trailers_tempfile, file))
-			die_errno(_("could not rename temporary file to %s"), file);
-
-	strbuf_release(&sb);
-}
-
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts)
 {
diff --git a/trailer.h b/trailer.h
index 37033e631a1..c292d44b62f 100644
--- a/trailer.h
+++ b/trailer.h
@@ -81,15 +81,29 @@ struct process_trailer_options {
 
 #define PROCESS_TRAILER_OPTIONS_INIT {0}
 
-void interpret_trailers(const struct process_trailer_options *opts,
-			struct list_head *new_trailer_head,
-			const char *file);
+void parse_trailers_from_config(struct list_head *config_head);
+
+void parse_trailers_from_command_line_args(struct list_head *arg_head,
+					   struct list_head *new_trailer_head);
+
+void process_trailers_lists(struct list_head *head,
+			    struct list_head *arg_head);
+
+void parse_trailers(const struct process_trailer_options *,
+		    struct trailer_info *,
+		    const char *str,
+		    struct list_head *head);
 
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts);
 
 void trailer_info_release(struct trailer_info *info);
 
+void trailer_config_init(void);
+void format_trailers(const struct process_trailer_options *,
+		     struct list_head *trailers, FILE *outfile);
+void free_trailers(struct list_head *);
+
 /*
  * Format the trailers from the commit msg "msg" into the strbuf "out".
  * Note two caveats about "opts":
-- 
gitgitgadget

[PATCH v5 5/9] trailer: start preparing for formatting unification

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Currently there are two functions for formatting trailers in
<trailer.h>:

    void format_trailers(const struct process_trailer_options *,
                         struct list_head *trailers, FILE *outfile);

    void format_trailers_from_commit(struct strbuf *out, const char *msg,
                                     const struct process_trailer_options *opts);

and although they are similar enough (even taking the same
process_trailer_options struct pointer) they are used quite differently.
One might intuitively think that format_trailers_from_commit() builds on
top of format_trailers(), but this is not the case. Instead
format_trailers_from_commit() calls format_trailer_info() and
format_trailers() is never called in that codepath.

This is a preparatory refactor to help us deprecate format_trailers() in
favor of format_trailer_info() (at which point we can rename the latter
to the former). When the deprecation is complete, both
format_trailers_from_commit(), and the interpret-trailers builtin will
be able to call into the same helper function (instead of
format_trailers() and format_trailer_info(), respectively). Unifying the
formatters is desirable because it simplifies the API.

Reorder parameters for format_trailers_from_commit() to prefer

    const struct process_trailer_options *opts

as the first parameter, because these options are intimately tied to
formatting trailers. And take

    struct strbuf *out

last, because it's an "out parameter" (something that the caller wants
to use as the output of this function).

Signed-off-by: Linus Arver <linusa@google.com>
---
 pretty.c     |  2 +-
 ref-filter.c |  2 +-
 trailer.c    | 11 ++++++-----
 trailer.h    |  5 +++--
 4 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/pretty.c b/pretty.c
index cf964b060cd..bdbed4295aa 100644
--- a/pretty.c
+++ b/pretty.c
@@ -1759,7 +1759,7 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */
 				goto trailer_out;
 		}
 		if (*arg == ')') {
-			format_trailers_from_commit(sb, msg + c->subject_off, &opts);
+			format_trailers_from_commit(&opts, msg + c->subject_off, sb);
 			ret = arg - placeholder + 1;
 		}
 	trailer_out:
diff --git a/ref-filter.c b/ref-filter.c
index 35b989e1dfe..d358953b0ce 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -1985,7 +1985,7 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp
 			struct strbuf s = STRBUF_INIT;
 
 			/* Format the trailer info according to the trailer_opts given */
-			format_trailers_from_commit(&s, subpos, &atom->u.contents.trailer_opts);
+			format_trailers_from_commit(&atom->u.contents.trailer_opts, subpos, &s);
 
 			v->s = strbuf_detach(&s, NULL);
 		} else if (atom->u.contents.option == C_BARE)
diff --git a/trailer.c b/trailer.c
index d23afa0a65c..5025be97899 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1083,10 +1083,10 @@ void trailer_info_release(struct trailer_info *info)
 	free(info->trailers);
 }
 
-static void format_trailer_info(struct strbuf *out,
+static void format_trailer_info(const struct process_trailer_options *opts,
 				const struct trailer_info *info,
 				const char *msg,
-				const struct process_trailer_options *opts)
+				struct strbuf *out)
 {
 	size_t origlen = out->len;
 	size_t i;
@@ -1144,13 +1144,14 @@ static void format_trailer_info(struct strbuf *out,
 
 }
 
-void format_trailers_from_commit(struct strbuf *out, const char *msg,
-				 const struct process_trailer_options *opts)
+void format_trailers_from_commit(const struct process_trailer_options *opts,
+				 const char *msg,
+				 struct strbuf *out)
 {
 	struct trailer_info info;
 
 	trailer_info_get(&info, msg, opts);
-	format_trailer_info(out, &info, msg, opts);
+	format_trailer_info(opts, &info, msg, out);
 	trailer_info_release(&info);
 }
 
diff --git a/trailer.h b/trailer.h
index c292d44b62f..c6d3ee49bbf 100644
--- a/trailer.h
+++ b/trailer.h
@@ -115,8 +115,9 @@ void free_trailers(struct list_head *);
  *     only the trailer block itself, even if the "only_trailers" option is not
  *     set.
  */
-void format_trailers_from_commit(struct strbuf *out, const char *msg,
-				 const struct process_trailer_options *opts);
+void format_trailers_from_commit(const struct process_trailer_options *opts,
+				 const char *msg,
+				 struct strbuf *out);
 
 /*
  * An interface for iterating over the trailers found in a particular commit
-- 
gitgitgadget

[PATCH v5 6/9] trailer_info_get(): reorder parameters

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

Take

    const struct process_trailer_options *opts

as the first parameter, because these options are required for
parsing trailers (e.g., whether to treat "---" as the end of the log
message). And take

    struct trailer_info *info

last, because it's an "out parameter" (something that the caller wants
to use as the output of this function).

Signed-off-by: Linus Arver <linusa@google.com>
---
 sequencer.c |  2 +-
 trailer.c   | 11 ++++++-----
 trailer.h   |  5 +++--
 3 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 3cc88d8a800..8e199fc8a47 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -332,7 +332,7 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 		sb->buf[sb->len - ignore_footer] = '\0';
 	}
 
-	trailer_info_get(&info, sb->buf, &opts);
+	trailer_info_get(&opts, sb->buf, &info);
 
 	if (ignore_footer)
 		sb->buf[sb->len - ignore_footer] = saved_char;
diff --git a/trailer.c b/trailer.c
index 5025be97899..f92d844361a 100644
--- a/trailer.c
+++ b/trailer.c
@@ -997,7 +997,7 @@ void parse_trailers(const struct process_trailer_options *opts,
 	struct strbuf val = STRBUF_INIT;
 	size_t i;
 
-	trailer_info_get(info, str, opts);
+	trailer_info_get(opts, str, info);
 
 	for (i = 0; i < info->trailer_nr; i++) {
 		int separator_pos;
@@ -1032,8 +1032,9 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
-void trailer_info_get(struct trailer_info *info, const char *str,
-		      const struct process_trailer_options *opts)
+void trailer_info_get(const struct process_trailer_options *opts,
+		      const char *str,
+		      struct trailer_info *info)
 {
 	size_t end_of_log_message = 0, trailer_block_start = 0;
 	struct strbuf **trailer_lines, **ptr;
@@ -1150,7 +1151,7 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 {
 	struct trailer_info info;
 
-	trailer_info_get(&info, msg, opts);
+	trailer_info_get(opts, msg, &info);
 	format_trailer_info(opts, &info, msg, out);
 	trailer_info_release(&info);
 }
@@ -1161,7 +1162,7 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 	strbuf_init(&iter->key, 0);
 	strbuf_init(&iter->val, 0);
 	opts.no_divider = 1;
-	trailer_info_get(&iter->internal.info, msg, &opts);
+	trailer_info_get(&opts, msg, &iter->internal.info);
 	iter->internal.cur = 0;
 }
 
diff --git a/trailer.h b/trailer.h
index c6d3ee49bbf..410c61b62be 100644
--- a/trailer.h
+++ b/trailer.h
@@ -94,8 +94,9 @@ void parse_trailers(const struct process_trailer_options *,
 		    const char *str,
 		    struct list_head *head);
 
-void trailer_info_get(struct trailer_info *info, const char *str,
-		      const struct process_trailer_options *opts);
+void trailer_info_get(const struct process_trailer_options *,
+		      const char *str,
+		      struct trailer_info *);
 
 void trailer_info_release(struct trailer_info *info);
 
-- 
gitgitgadget

[PATCH v5 7/9] format_trailers(): use strbuf instead of FILE

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

Make format_trailers() also write to a strbuf, to align with
format_trailers_from_commit() which also does the same. Doing this makes
format_trailers() behave similar to format_trailer_info() (which will
soon help us replace one with the other).

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  6 +++++-
 trailer.c                    | 13 +++++++------
 trailer.h                    |  3 ++-
 3 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index d1cf0aa33a2..11f4ce9e4a2 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -140,6 +140,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
+	struct strbuf trailer_block = STRBUF_INIT;
 	struct trailer_info info;
 	FILE *outfile = stdout;
 
@@ -169,8 +170,11 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 		process_trailers_lists(&head, &arg_head);
 	}
 
-	format_trailers(opts, &head, outfile);
+	/* Print trailer block. */
+	format_trailers(opts, &head, &trailer_block);
 	free_trailers(&head);
+	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
+	strbuf_release(&trailer_block);
 
 	/* Print the lines after the trailers as is */
 	if (!opts->only_trailers)
diff --git a/trailer.c b/trailer.c
index f92d844361a..cbd643cd1fe 100644
--- a/trailer.c
+++ b/trailer.c
@@ -144,12 +144,12 @@ static char last_non_space_char(const char *s)
 	return '\0';
 }
 
-static void print_tok_val(FILE *outfile, const char *tok, const char *val)
+static void print_tok_val(struct strbuf *out, const char *tok, const char *val)
 {
 	char c;
 
 	if (!tok) {
-		fprintf(outfile, "%s\n", val);
+		strbuf_addf(out, "%s\n", val);
 		return;
 	}
 
@@ -157,13 +157,14 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 	if (!c)
 		return;
 	if (strchr(separators, c))
-		fprintf(outfile, "%s%s\n", tok, val);
+		strbuf_addf(out, "%s%s\n", tok, val);
 	else
-		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
+		strbuf_addf(out, "%s%c %s\n", tok, separators[0], val);
 }
 
 void format_trailers(const struct process_trailer_options *opts,
-		     struct list_head *trailers, FILE *outfile)
+		     struct list_head *trailers,
+		     struct strbuf *out)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
@@ -171,7 +172,7 @@ void format_trailers(const struct process_trailer_options *opts,
 		item = list_entry(pos, struct trailer_item, list);
 		if ((!opts->trim_empty || strlen(item->value) > 0) &&
 		    (!opts->only_trailers || item->token))
-			print_tok_val(outfile, item->token, item->value);
+			print_tok_val(out, item->token, item->value);
 	}
 }
 
diff --git a/trailer.h b/trailer.h
index 410c61b62be..1d106b6dd40 100644
--- a/trailer.h
+++ b/trailer.h
@@ -102,7 +102,8 @@ void trailer_info_release(struct trailer_info *info);
 
 void trailer_config_init(void);
 void format_trailers(const struct process_trailer_options *,
-		     struct list_head *trailers, FILE *outfile);
+		     struct list_head *trailers,
+		     struct strbuf *out);
 void free_trailers(struct list_head *);
 
 /*
-- 
gitgitgadget

[PATCH v5 8/9] format_trailer_info(): move "fast path" to caller

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

This allows us to drop the "msg" parameter from format_trailer_info(),
so that it take 3 parameters, similar to format_trailers() which also
takes 3 parameters:

    void format_trailers(const struct process_trailer_options *opts,
                         struct list_head *trailers,
                         struct strbuf *out)

The short-term goal is to make format_trailer_info() be smart enough to
deprecate format_trailers(). And then ultimately we will rename
format_trailer_info() to format_trailers().

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/trailer.c b/trailer.c
index cbd643cd1fe..e92d0154d90 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1087,21 +1087,11 @@ void trailer_info_release(struct trailer_info *info)
 
 static void format_trailer_info(const struct process_trailer_options *opts,
 				const struct trailer_info *info,
-				const char *msg,
 				struct strbuf *out)
 {
 	size_t origlen = out->len;
 	size_t i;
 
-	/* If we want the whole block untouched, we can take the fast path. */
-	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
-	    !opts->separator && !opts->key_only && !opts->value_only &&
-	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info->trailer_block_start,
-			   info->trailer_block_end - info->trailer_block_start);
-		return;
-	}
-
 	for (i = 0; i < info->trailer_nr; i++) {
 		char *trailer = info->trailers[i];
 		ssize_t separator_pos = find_separator(trailer, separators);
@@ -1153,7 +1143,15 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 	struct trailer_info info;
 
 	trailer_info_get(opts, msg, &info);
-	format_trailer_info(opts, &info, msg, out);
+	/* If we want the whole block untouched, we can take the fast path. */
+	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
+	    !opts->separator && !opts->key_only && !opts->value_only &&
+	    !opts->key_value_separator) {
+		strbuf_add(out, msg + info.trailer_block_start,
+			   info.trailer_block_end - info.trailer_block_start);
+	} else
+		format_trailer_info(opts, &info, out);
+
 	trailer_info_release(&info);
 }
 
-- 
gitgitgadget

[PATCH v5 9/9] format_trailers_from_commit(): indirectly call trailer_info_get()

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

Instead of calling trailer_info_get() directly, call parse_trailers()
which already calls trailer_info_get(). This change is a NOP because
format_trailer_info() only looks at the "trailers" string array, not the
trailer_item objects which parse_trailers() populates.

In a future patch, we'll change format_trailer_info() to use the parsed
trailer_item objects instead of the string array.

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/trailer.c b/trailer.c
index e92d0154d90..e6665c99cc3 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1140,9 +1140,11 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 				 const char *msg,
 				 struct strbuf *out)
 {
+	LIST_HEAD(trailers);
 	struct trailer_info info;
 
-	trailer_info_get(opts, msg, &info);
+	parse_trailers(opts, &info, msg, &trailers);
+
 	/* If we want the whole block untouched, we can take the fast path. */
 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
 	    !opts->separator && !opts->key_only && !opts->value_only &&
@@ -1152,6 +1154,7 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 	} else
 		format_trailer_info(opts, &info, out);
 
+	free_trailers(&trailers);
 	trailer_info_release(&info);
 }
 
-- 
gitgitgadget

Re: [PATCH v5 3/9] trailer: prepare to expose functions as part of API

[Christian Couder]

On Sat, Feb 17, 2024 at 12:09 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Linus Arver <linusa@google.com>
>
> In the next patch, we will move "process_trailers" from trailer.c to
> builtin/interpret-trailers.c. That move will necessitate the growth of
> the trailer.h API, forcing us to expose some additional functions in
> trailer.h.

Nit: actually this patch renames process_trailers() to
interpret_trailers() so the function that will be moved will be
interpret_trailers().

Nit: this patch and the next one will become commits, so perhaps:

s/In the next patch/In a following commit/

> Rename relevant functions so that they include the term "trailer" in
> their name, so that clients of the API will be able to easily identify
> them by their "trailer" moniker, just like all the other functions
> already exposed by trailer.h.

Except that "process_trailers()" already contains "trailer" but will
still be renamed by this patch to "interpret_trailers()". So I think
it might be nice to explain a bit why renaming process_trailers() to
interpret_trailers() makes sense too.

Also I think the subject, "trailer: prepare to expose functions as
part of API" could be more explicit about what the patch is actually
doing, like perhaps "trailer: rename functions to use 'trailer'".

In general, when there is a patch called "prepare to do X", then we
might expect a following patch called something like "actually do X".
But there isn't any patch in the series named like "trailer: expose
functions as part of API".

> Take the opportunity to start putting trailer processing options (opts)
> as the first parameter. This will be the pattern going forward in this
> series.

It's interesting to know that this will be the pattern going forward
in the series, but that doesn't quite tell why it's a good idea to do
it.

So I think it might be nice to repeat an explanation similar to the
one you give in "trailer: start preparing for formatting unification"
for format_trailers_from_commit():

"Reorder parameters for format_trailers_from_commit() to prefer

    const struct process_trailer_options *opts

as the first parameter, because these options are intimately tied to
formatting trailers."

And maybe also say that parameters like `FILE *outfile` should be last
because they are some kind of 'out' parameters.

> diff --git a/trailer.c b/trailer.c
> index f74915bd8cd..916175707d8 100644
> --- a/trailer.c
> +++ b/trailer.c
> @@ -163,12 +163,12 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
>                 fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
>  }
>
> -static void print_all(FILE *outfile, struct list_head *head,
> -                     const struct process_trailer_options *opts)
> +static void format_trailers(const struct process_trailer_options *opts,
> +                           struct list_head *trailers, FILE *outfile)

This also renames `struct list_head *head` to `struct list_head
*trailers`. I think it would be nice if the commit message could talk
a bit about these renames too.

Re: [PATCH v5 5/9] trailer: start preparing for formatting unification

[Christian Couder]

On Sat, Feb 17, 2024 at 12:09 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Linus Arver <linusa@google.com>
>
> Currently there are two functions for formatting trailers in
> <trailer.h>:
>
>     void format_trailers(const struct process_trailer_options *,
>                          struct list_head *trailers, FILE *outfile);
>
>     void format_trailers_from_commit(struct strbuf *out, const char *msg,
>                                      const struct process_trailer_options *opts);
>
> and although they are similar enough (even taking the same
> process_trailer_options struct pointer) they are used quite differently.
> One might intuitively think that format_trailers_from_commit() builds on
> top of format_trailers(), but this is not the case. Instead
> format_trailers_from_commit() calls format_trailer_info() and
> format_trailers() is never called in that codepath.
>
> This is a preparatory refactor to help us deprecate format_trailers() in
> favor of format_trailer_info() (at which point we can rename the latter
> to the former). When the deprecation is complete, both
> format_trailers_from_commit(), and the interpret-trailers builtin will
> be able to call into the same helper function (instead of
> format_trailers() and format_trailer_info(), respectively). Unifying the
> formatters is desirable because it simplifies the API.
>
> Reorder parameters for format_trailers_from_commit() to prefer
>
>     const struct process_trailer_options *opts
>
> as the first parameter, because these options are intimately tied to
> formatting trailers. And take
>
>     struct strbuf *out
>
> last, because it's an "out parameter" (something that the caller wants
> to use as the output of this function).

Here also I think the subject could be more specific like for example:

"trailer: reorder format_trailers_from_commit() parameters"

> diff --git a/trailer.c b/trailer.c
> index d23afa0a65c..5025be97899 100644
> --- a/trailer.c
> +++ b/trailer.c
> @@ -1083,10 +1083,10 @@ void trailer_info_release(struct trailer_info *info)
>         free(info->trailers);
>  }
>
> -static void format_trailer_info(struct strbuf *out,
> +static void format_trailer_info(const struct process_trailer_options *opts,
>                                 const struct trailer_info *info,
>                                 const char *msg,
> -                               const struct process_trailer_options *opts)
> +                               struct strbuf *out)

Ok, so it's not just format_trailers_from_commit() parameters that are
reordered, but also format_trailer_info() parameters. It would be nice
if the commit message mentioned it.

Re: [PATCH v5 9/9] format_trailers_from_commit(): indirectly call trailer_info_get()

[Christian Couder]

On Sat, Feb 17, 2024 at 12:09 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Linus Arver <linusa@google.com>
>
> This is another preparatory refactor to unify the trailer formatters.
>
> Instead of calling trailer_info_get() directly, call parse_trailers()
> which already calls trailer_info_get(). This change is a NOP because
> format_trailer_info() only looks at the "trailers" string array, not the
> trailer_item objects which parse_trailers() populates.

Is the extra processing done by parse_trailers() compared to
trailer_info_get() impacting performance?

Also when looking only at the patch, it's a bit difficult to
understand that the "trailers" string array is the `char **trailers`
field in `struct trailer_info` and that the trailer_item objects are
the elements of the `struct list_head *head` linked list. It could
also be confusing because the patch is adding a new 'trailers'
variable with `LIST_HEAD(trailers);`. So a few more details could help
understand what's going on.

> In a future patch, we'll change format_trailer_info() to use the parsed
> trailer_item objects instead of the string array.

Ok, so I guess the possible performance issue would disappear then, as
populating the trailer_item objects will be useful.

Re: [PATCH v5 0/9] Enrich Trailer API

[Christian Couder]

On Sat, Feb 17, 2024 at 12:09 AM Linus Arver via GitGitGadget
<gitgitgadget@gmail.com> wrote:

> In summary this series breaks up "process_trailers()" into smaller pieces,
> exposing many of the parts relevant to trailer-related processing in
> trailer.h. This will force us to eventually introduce unit tests for these
> API functions, but that is a good thing for API stability. We also perform
> some preparatory refactors in order to help us unify the trailer formatting
> machinery toward the end of this series.

I took another look and suggested some improvements to commit messages.

Thanks!

Re: [PATCH v5 3/9] trailer: prepare to expose functions as part of API

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Sat, Feb 17, 2024 at 12:09 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>>
>> From: Linus Arver <linusa@google.com>
>>
>> In the next patch, we will move "process_trailers" from trailer.c to
>> builtin/interpret-trailers.c. That move will necessitate the growth of
>> the trailer.h API, forcing us to expose some additional functions in
>> trailer.h.
>
> Nit: actually this patch renames process_trailers() to
> interpret_trailers() so the function that will be moved will be
> interpret_trailers().

Oops, fixed locally.

> Nit: this patch and the next one will become commits, so perhaps:
>
> s/In the next patch/In a following commit/

TBH I've always wondered whether "patch" or "commit" matters --- I've
seen examples of patch series that referred to "commits" instead of
"patches", and vice versa. I was hoping to hear an opinion on this, so
I'm happy to see (and apply) your suggestion. Thanks.

>> Rename relevant functions so that they include the term "trailer" in
>> their name, so that clients of the API will be able to easily identify
>> them by their "trailer" moniker, just like all the other functions
>> already exposed by trailer.h.
>
> Except that "process_trailers()" already contains "trailer" but will
> still be renamed by this patch to "interpret_trailers()". So I think
> it might be nice to explain a bit why renaming process_trailers() to
> interpret_trailers() makes sense too.

I will add something like:

    Rename process_trailers() to interpret_trailers(), because it
    matches the name for the builtin command of the same name
    (git-interpret-trailers), which is the sole user of
    process_trailers().

> Also I think the subject, "trailer: prepare to expose functions as
> part of API" could be more explicit about what the patch is actually
> doing, like perhaps "trailer: rename functions to use 'trailer'".

Applied.

> In general, when there is a patch called "prepare to do X", then we
> might expect a following patch called something like "actually do X".
> But there isn't any patch in the series named like "trailer: expose
> functions as part of API".

Sounds like a very sensible rule. IOW, leave the detailed explanation to
the commit message and use the subject line only for the most obvious
explanation of what's going on in the patch. +1

>> Take the opportunity to start putting trailer processing options (opts)
>> as the first parameter. This will be the pattern going forward in this
>> series.
>
> It's interesting to know that this will be the pattern going forward
> in the series, but that doesn't quite tell why it's a good idea to do
> it.
>
> So I think it might be nice to repeat an explanation similar to the
> one you give in "trailer: start preparing for formatting unification"
> for format_trailers_from_commit():
>
> "Reorder parameters for format_trailers_from_commit() to prefer
>
>     const struct process_trailer_options *opts
>
> as the first parameter, because these options are intimately tied to
> formatting trailers."
>
> And maybe also say that parameters like `FILE *outfile` should be last
> because they are some kind of 'out' parameters.

SGTM, will do.

>> diff --git a/trailer.c b/trailer.c
>> index f74915bd8cd..916175707d8 100644
>> --- a/trailer.c
>> +++ b/trailer.c
>> @@ -163,12 +163,12 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
>>                 fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
>>  }
>>
>> -static void print_all(FILE *outfile, struct list_head *head,
>> -                     const struct process_trailer_options *opts)
>> +static void format_trailers(const struct process_trailer_options *opts,
>> +                           struct list_head *trailers, FILE *outfile)
>
> This also renames `struct list_head *head` to `struct list_head
> *trailers`. I think it would be nice if the commit message could talk
> a bit about these renames too.

Ah nice catch. Will do.

Really appreciate the quality of your reviews, thanks so much! :)

Re: [PATCH v5 5/9] trailer: start preparing for formatting unification

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Sat, Feb 17, 2024 at 12:09 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>>
>> From: Linus Arver <linusa@google.com>
>>
>> Currently there are two functions for formatting trailers in
>> <trailer.h>:
>>
>>     void format_trailers(const struct process_trailer_options *,
>>                          struct list_head *trailers, FILE *outfile);
>>
>>     void format_trailers_from_commit(struct strbuf *out, const char *msg,
>>                                      const struct process_trailer_options *opts);
>>
>> and although they are similar enough (even taking the same
>> process_trailer_options struct pointer) they are used quite differently.
>> One might intuitively think that format_trailers_from_commit() builds on
>> top of format_trailers(), but this is not the case. Instead
>> format_trailers_from_commit() calls format_trailer_info() and
>> format_trailers() is never called in that codepath.
>>
>> This is a preparatory refactor to help us deprecate format_trailers() in
>> favor of format_trailer_info() (at which point we can rename the latter
>> to the former). When the deprecation is complete, both
>> format_trailers_from_commit(), and the interpret-trailers builtin will
>> be able to call into the same helper function (instead of
>> format_trailers() and format_trailer_info(), respectively). Unifying the
>> formatters is desirable because it simplifies the API.
>>
>> Reorder parameters for format_trailers_from_commit() to prefer
>>
>>     const struct process_trailer_options *opts
>>
>> as the first parameter, because these options are intimately tied to
>> formatting trailers. And take
>>
>>     struct strbuf *out
>>
>> last, because it's an "out parameter" (something that the caller wants
>> to use as the output of this function).
>
> Here also I think the subject could be more specific like for example:
>
> "trailer: reorder format_trailers_from_commit() parameters"

Applied, thanks.

>> diff --git a/trailer.c b/trailer.c
>> index d23afa0a65c..5025be97899 100644
>> --- a/trailer.c
>> +++ b/trailer.c
>> @@ -1083,10 +1083,10 @@ void trailer_info_release(struct trailer_info *info)
>>         free(info->trailers);
>>  }
>>
>> -static void format_trailer_info(struct strbuf *out,
>> +static void format_trailer_info(const struct process_trailer_options *opts,
>>                                 const struct trailer_info *info,
>>                                 const char *msg,
>> -                               const struct process_trailer_options *opts)
>> +                               struct strbuf *out)
>
> Ok, so it's not just format_trailers_from_commit() parameters that are
> reordered, but also format_trailer_info() parameters. It would be nice
> if the commit message mentioned it.

Agreed. Will do.

Re: [PATCH v5 9/9] format_trailers_from_commit(): indirectly call trailer_info_get()

[Linus Arver]

Christian Couder <christian.couder@gmail.com> writes:

> On Sat, Feb 17, 2024 at 12:09 AM Linus Arver via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>>
>> From: Linus Arver <linusa@google.com>
>>
>> This is another preparatory refactor to unify the trailer formatters.
>>
>> Instead of calling trailer_info_get() directly, call parse_trailers()
>> which already calls trailer_info_get(). This change is a NOP because
>> format_trailer_info() only looks at the "trailers" string array, not the
>> trailer_item objects which parse_trailers() populates.
>
> Is the extra processing done by parse_trailers() compared to
> trailer_info_get() impacting performance?

I was going to answer this now but I see that you've already reached the
same conclusion as me further below. ;)

> Also when looking only at the patch, it's a bit difficult to
> understand that the "trailers" string array is the `char **trailers`
> field in `struct trailer_info` and that the trailer_item objects are
> the elements of the `struct list_head *head` linked list. It could
> also be confusing because the patch is adding a new 'trailers'
> variable with `LIST_HEAD(trailers);`. So a few more details could help
> understand what's going on.

Makes sense. Admittedly, this is one thing that did bother me at some
point but which I forgot about as I got more familiar with my own patch.

I will avoid shadowing the "trailers" word (and maybe at least add a
suffix or prefix to disambiguate it).

>> In a future patch, we'll change format_trailer_info() to use the parsed
>> trailer_item objects instead of the string array.
>
> Ok, so I guess the possible performance issue would disappear then, as
> populating the trailer_item objects will be useful.

Yep, exactly. In the larger series (some 20? 30?) commits down the road
in my local tree, I have it so that we remove `char **trailers`
entirely (because we should be using the (smarter) trailer_item objects,
not raw strings, where possible).

Re: [PATCH v5 3/9] trailer: prepare to expose functions as part of API

[Junio C Hamano]

Linus Arver <linusa@google.com> writes:

>> Nit: this patch and the next one will become commits, so perhaps:
>>
>> s/In the next patch/In a following commit/
>
> TBH I've always wondered whether "patch" or "commit" matters --- I've
> seen examples of patch series that referred to "commits" instead of
> "patches", and vice versa. I was hoping to hear an opinion on this, so
> I'm happy to see (and apply) your suggestion. Thanks.

I think it is just fine to use either; sticking to one you pick
consistently in the same series would have value.  If you prefer
commit, then fine.  If you like patch, that's fine too.

Re: [PATCH v5 3/9] trailer: prepare to expose functions as part of API

[Linus Arver]

Junio C Hamano <gitster@pobox.com> writes:

> Linus Arver <linusa@google.com> writes:
>
>>> Nit: this patch and the next one will become commits, so perhaps:
>>>
>>> s/In the next patch/In a following commit/
>>
>> TBH I've always wondered whether "patch" or "commit" matters --- I've
>> seen examples of patch series that referred to "commits" instead of
>> "patches", and vice versa. I was hoping to hear an opinion on this, so
>> I'm happy to see (and apply) your suggestion. Thanks.
>
> I think it is just fine to use either; sticking to one you pick
> consistently in the same series would have value.  If you prefer
> commit, then fine.  If you like patch, that's fine too.

Makes sense, thanks.

[PATCH v6 0/9] Enrich Trailer API

[Linus Arver via GitGitGadget]

This patch series is the first 9 patches of a larger cleanup/bugfix series
(henceforth "larger series") I've been working on. The main goal of this
series is to begin the process of "libifying" the trailer API. By "API" I
mean the interface exposed in trailer.h. The larger series brings a number
of additional cleanups (exposing and fixing some bugs along the way), and
builds on top of this series.

When the larger series is merged, we will be in a good state to additionally
pursue the following goals:

 1. "API reuse inside Git": make the API expressive enough to eliminate any
    need by other parts of Git to use the interpret-trailers builtin as a
    subprocess (instead they could just use the API directly);
 2. "API stability": add unit tests to codify the expected behavior of API
    functions; and
 3. "API documentation": create developer-focused documentation to explain
    how to use the API effectively, noting any API limitations or
    anti-patterns.

In the future after libification is "complete", users external to Git will
be able to use the same trailer processing API used by the
interpret-trailers builtin. For example, a web server may want to parse
trailers the same way that Git would parse them, without having to call
interpret-trailers as a subprocess. This use case was the original
motivation behind my work in this area.

With the libification-focused goals out of the way, let's turn to this patch
series in more detail.

In summary this series breaks up "process_trailers()" into smaller pieces,
exposing many of the parts relevant to trailer-related processing in
trailer.h. This will force us to eventually introduce unit tests for these
API functions, but that is a good thing for API stability. We also perform
some preparatory refactors in order to help us unify the trailer formatting
machinery toward the end of this series.


Notable changes in v6
=====================

 * Mainly wording changes to commit messages. Thanks to Christian for the
   suggestions.


Notable changes in v5
=====================

 * Removed patches 10+ from this series. Thanks to Christian for the
   suggestion.
 * Reworded the log message of patch 09 to reflect the above arrangement, as
   suggested by Christian.


Notable changes in v4
=====================

 * Patches 3, 4, 5, and 8 have been broken up into smaller steps. There are
   28 instead of 10 patches now, but these 28 should be much easier to
   review than the (previously condensed) 10.
 * NEW Patch 1: "trailer: free trailer_info after all related usage" fixes
   awkward use-after-free coding style
 * NEW Patch 2: "shortlog: add test for de-duplicating folded trailers"
   increases test coverage related to trailer iterators and "unfold_value()"
 * NEW Patch 27: "trailer_set_*(): put out parameter at the end" is a small
   refactor to reorder parameters.
 * Patches 5-16: These smaller patches make up Patch 3 from v3.
 * Patches 17-18: These smaller patches make up Patch 4 from v3.
 * Patches 19-20: These smaller patches make up Patch 5 from v3.
 * Patches 23-26: These smaller patches make up Patch 8 from v3.
 * Anonymize unambiguous parameters in <trailer.h>.


Notable changes in v3
=====================

 * Squashed Patch 4 into Patch 3 ("trailer: unify trailer formatting
   machinery"), to avoid breaking the build ("-Werror=unused-function"
   violations)
 * NEW (Patch 10): Introduce "trailer template" terminology for readability
   (no behavioral change)
 * (API function) Rename default_separators() to
   trailer_default_separators()
 * (API function) Rename new_trailers_clear() to free_trailer_templates()
 * trailer.h: for single-parameter functions, anonymize the parameter name
   to reduce verbosity


Notable changes in v2
=====================

 * (cover letter) Discuss goals of the larger series in more detail,
   especially the pimpl idiom
 * (cover letter) List bug fixes pending in the larger series that depend on
   this series
 * Reorder function parameters to have trailer options at the beginning (and
   out parameters toward the end)
 * "sequencer: use the trailer iterator": prefer C string instead of strbuf
   for new "raw" field
 * Patch 1 (was Patch 2) also renames ensure_configured() to
   trailer_config_init() (forgot to rename this one previously)

Linus Arver (9):
  trailer: free trailer_info _after_ all related usage
  shortlog: add test for de-duplicating folded trailers
  trailer: rename functions to use 'trailer'
  trailer: move interpret_trailers() to interpret-trailers.c
  trailer: reorder format_trailers_from_commit() parameters
  trailer_info_get(): reorder parameters
  format_trailers(): use strbuf instead of FILE
  format_trailer_info(): move "fast path" to caller
  format_trailers_from_commit(): indirectly call trailer_info_get()

 builtin/interpret-trailers.c | 101 ++++++++++++++++++-
 pretty.c                     |   2 +-
 ref-filter.c                 |   2 +-
 sequencer.c                  |   2 +-
 t/t4201-shortlog.sh          |  32 +++++++
 trailer.c                    | 181 +++++++++--------------------------
 trailer.h                    |  31 ++++--
 7 files changed, 204 insertions(+), 147 deletions(-)


base-commit: a54a84b333adbecf7bc4483c0e36ed5878cac17b
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1632%2Flistx%2Ftrailer-api-refactor-part-1-v6
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1632/listx/trailer-api-refactor-part-1-v6
Pull-Request: https://github.com/gitgitgadget/git/pull/1632

Range-diff vs v5:

  1:  652df25f30e =  1:  652df25f30e trailer: free trailer_info _after_ all related usage
  2:  fdccaca2ba0 =  2:  fdccaca2ba0 shortlog: add test for de-duplicating folded trailers
  3:  4372af244f0 !  3:  7b1d739cddb trailer: prepare to expose functions as part of API
     @@ Metadata
      Author: Linus Arver <linusa@google.com>
      
       ## Commit message ##
     -    trailer: prepare to expose functions as part of API
     +    trailer: rename functions to use 'trailer'
      
     -    In the next patch, we will move "process_trailers" from trailer.c to
     -    builtin/interpret-trailers.c. That move will necessitate the growth of
     -    the trailer.h API, forcing us to expose some additional functions in
     +    Rename process_trailers() to interpret_trailers(), because it matches
     +    the name for the builtin command of the same name
     +    (git-interpret-trailers), which is the sole user of process_trailers().
     +
     +    In a following commit, we will move "interpret_trailers" from trailer.c
     +    to builtin/interpret-trailers.c. That move will necessitate the growth
     +    of the trailer.h API, forcing us to expose some additional functions in
          trailer.h.
      
          Rename relevant functions so that they include the term "trailer" in
     @@ Commit message
          them by their "trailer" moniker, just like all the other functions
          already exposed by trailer.h.
      
     -    Take the opportunity to start putting trailer processing options (opts)
     -    as the first parameter. This will be the pattern going forward in this
     -    series.
     +    Rename `struct list_head *head` to `struct list_head *trailers` because
     +    "head" conveys no additional information beyond the "list_head" type.
     +
     +    Reorder parameters for format_trailers_from_commit() to prefer
     +
     +        const struct process_trailer_options *opts
     +
     +    as the first parameter, because these options are intimately tied to
     +    formatting trailers. Parameters like `FILE *outfile` should be last
     +    because they are a kind of 'out' parameter, so put such parameters at
     +    the end. This will be the pattern going forward in this series.
      
          Helped-by: Junio C Hamano <gitster@pobox.com>
     +    Helped-by: Christian Couder <chriscool@tuxfamily.org>
          Signed-off-by: Linus Arver <linusa@google.com>
      
       ## builtin/interpret-trailers.c ##
  4:  4073b8eb510 =  4:  7ac4da3019a trailer: move interpret_trailers() to interpret-trailers.c
  5:  b2a0f7829a1 !  5:  47c994ce025 trailer: start preparing for formatting unification
     @@ Metadata
      Author: Linus Arver <linusa@google.com>
      
       ## Commit message ##
     -    trailer: start preparing for formatting unification
     +    trailer: reorder format_trailers_from_commit() parameters
      
          Currently there are two functions for formatting trailers in
          <trailer.h>:
     @@ Commit message
          last, because it's an "out parameter" (something that the caller wants
          to use as the output of this function).
      
     +    Similarly, reorder parameters for format_trailer_info(), because later
     +    on we will unify the two together.
     +
          Signed-off-by: Linus Arver <linusa@google.com>
      
       ## pretty.c ##
  6:  c1760f80356 =  6:  7a565580167 trailer_info_get(): reorder parameters
  7:  9dc912b5bc5 =  7:  46c7f4c0e81 format_trailers(): use strbuf instead of FILE
  8:  b97c06d8bc3 =  8:  26b1f19d0e1 format_trailer_info(): move "fast path" to caller
  9:  7c656b3f775 !  9:  0e884d870c8 format_trailers_from_commit(): indirectly call trailer_info_get()
     @@ Commit message
      
          This is another preparatory refactor to unify the trailer formatters.
      
     -    Instead of calling trailer_info_get() directly, call parse_trailers()
     -    which already calls trailer_info_get(). This change is a NOP because
     -    format_trailer_info() only looks at the "trailers" string array, not the
     -    trailer_item objects which parse_trailers() populates.
     +    For background, note that the "trailers" string array is the
     +    `char **trailers` member in `struct trailer_info` and that the
     +    trailer_item objects are the elements of the `struct list_head *head`
     +    linked list.
     +
     +    Currently trailer_info_get() only populates `char **trailers`. And
     +    parse_trailers() first calls trailer_info_get() so that it can use the
     +    `char **trailers` to populate a list of `struct trailer_item` objects
     +
     +    Instead of calling trailer_info_get() directly from
     +    format_trailers_from_commit(), make it call parse_trailers() instead
     +    because parse_trailers() already calls trailer_info_get().
     +
     +    This change is a NOP because format_trailer_info() (which
     +    format_trailers_from_commit() wraps around) only looks at the "trailers"
     +    string array, not the trailer_item objects which parse_trailers()
     +    populates. For now we do need to create a dummy
     +
     +        LIST_HEAD(trailer_objects);
     +
     +    because parse_trailers() expects it in its signature.
      
          In a future patch, we'll change format_trailer_info() to use the parsed
     -    trailer_item objects instead of the string array.
     +    trailer_item objects (trailer_objects) instead of the `char **trailers`
     +    array.
      
          Signed-off-by: Linus Arver <linusa@google.com>
      
     @@ trailer.c: void format_trailers_from_commit(const struct process_trailer_options
       				 const char *msg,
       				 struct strbuf *out)
       {
     -+	LIST_HEAD(trailers);
     ++	LIST_HEAD(trailer_objects);
       	struct trailer_info info;
       
      -	trailer_info_get(opts, msg, &info);
     -+	parse_trailers(opts, &info, msg, &trailers);
     ++	parse_trailers(opts, &info, msg, &trailer_objects);
      +
       	/* If we want the whole block untouched, we can take the fast path. */
       	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
     @@ trailer.c: void format_trailers_from_commit(const struct process_trailer_options
       	} else
       		format_trailer_info(opts, &info, out);
       
     -+	free_trailers(&trailers);
     ++	free_trailers(&trailer_objects);
       	trailer_info_release(&info);
       }
       

-- 
gitgitgadget

[PATCH v6 1/9] trailer: free trailer_info _after_ all related usage

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

In de7c27a186 (trailer: use offsets for trailer_start/trailer_end,
2023-10-20), we started using trailer block offsets in trailer_info. In
particular, we dropped the use of a separate stack variable "size_t
trailer_end", in favor of accessing the new "trailer_block_end" member
of trailer_info (as "info.trailer_block_end").

At that time, we forgot to also move the

   trailer_info_release(&info);

line to be _after_ this new use of the trailer_info struct. Move it now.

Note that even without this patch, we didn't have leaks or any other
problems because trailer_info_release() only frees memory allocated on
the heap. The "trailer_block_end" member was allocated on the stack back
then (as it is now) so it was still safe to use for all this time.

Reported-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/trailer.c b/trailer.c
index 3a0710a4583..e1d83390b66 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1111,13 +1111,12 @@ void process_trailers(const char *file,
 	}
 
 	print_all(outfile, &head, opts);
-
 	free_all(&head);
-	trailer_info_release(&info);
 
 	/* Print the lines after the trailers as is */
 	if (!opts->only_trailers)
 		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
+	trailer_info_release(&info);
 
 	if (opts->in_place)
 		if (rename_tempfile(&trailers_tempfile, file))
-- 
gitgitgadget

[PATCH v6 2/9] shortlog: add test for de-duplicating folded trailers

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

The shortlog builtin was taught to use the trailer iterator interface in
47beb37bc6 (shortlog: match commit trailers with --group, 2020-09-27).
The iterator always unfolds values and this has always been the case
since the time the iterator was first introduced in f0939a0eb1 (trailer:
add interface for iterating over commit trailers, 2020-09-27). Add a
comment line to remind readers of this behavior.

The fact that the iterator always unfolds values is important
(at least for shortlog) because unfolding allows it to recognize both
folded and unfolded versions of the same trailer for de-duplication.

Capture the existing behavior in a new test case to guard against
regressions in this area. This test case is based off of the existing
"shortlog de-duplicates trailers in a single commit" just above it. Now
if we were to remove the call to

    unfold_value(&iter->val);

inside the iterator, this new test case will break.

Signed-off-by: Linus Arver <linusa@google.com>
---
 t/t4201-shortlog.sh | 32 ++++++++++++++++++++++++++++++++
 trailer.c           |  1 +
 2 files changed, 33 insertions(+)

diff --git a/t/t4201-shortlog.sh b/t/t4201-shortlog.sh
index d7382709fc1..f698d0c9ad2 100755
--- a/t/t4201-shortlog.sh
+++ b/t/t4201-shortlog.sh
@@ -312,6 +312,38 @@ test_expect_success 'shortlog de-duplicates trailers in a single commit' '
 	test_cmp expect actual
 '
 
+# Trailers that have unfolded (single line) and folded (multiline) values which
+# are otherwise identical are treated as the same trailer for de-duplication.
+test_expect_success 'shortlog de-duplicates trailers in a single commit (folded/unfolded values)' '
+	git commit --allow-empty -F - <<-\EOF &&
+	subject one
+
+	this message has two distinct values, plus a repeat (folded)
+
+	Repeated-trailer: Foo foo foo
+	Repeated-trailer: Bar
+	Repeated-trailer: Foo
+	  foo foo
+	EOF
+
+	git commit --allow-empty -F - <<-\EOF &&
+	subject two
+
+	similar to the previous, but without the second distinct value
+
+	Repeated-trailer: Foo foo foo
+	Repeated-trailer: Foo
+	  foo foo
+	EOF
+
+	cat >expect <<-\EOF &&
+	     2	Foo foo foo
+	     1	Bar
+	EOF
+	git shortlog -ns --group=trailer:repeated-trailer -2 HEAD >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'shortlog can match multiple groups' '
 	git commit --allow-empty -F - <<-\EOF &&
 	subject one
diff --git a/trailer.c b/trailer.c
index e1d83390b66..f74915bd8cd 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1270,6 +1270,7 @@ int trailer_iterator_advance(struct trailer_iterator *iter)
 		strbuf_reset(&iter->val);
 		parse_trailer(&iter->key, &iter->val, NULL,
 			      trailer, separator_pos);
+		/* Always unfold values during iteration. */
 		unfold_value(&iter->val);
 		return 1;
 	}
-- 
gitgitgadget

[PATCH v6 3/9] trailer: rename functions to use 'trailer'

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Rename process_trailers() to interpret_trailers(), because it matches
the name for the builtin command of the same name
(git-interpret-trailers), which is the sole user of process_trailers().

In a following commit, we will move "interpret_trailers" from trailer.c
to builtin/interpret-trailers.c. That move will necessitate the growth
of the trailer.h API, forcing us to expose some additional functions in
trailer.h.

Rename relevant functions so that they include the term "trailer" in
their name, so that clients of the API will be able to easily identify
them by their "trailer" moniker, just like all the other functions
already exposed by trailer.h.

Rename `struct list_head *head` to `struct list_head *trailers` because
"head" conveys no additional information beyond the "list_head" type.

Reorder parameters for format_trailers_from_commit() to prefer

    const struct process_trailer_options *opts

as the first parameter, because these options are intimately tied to
formatting trailers. Parameters like `FILE *outfile` should be last
because they are a kind of 'out' parameter, so put such parameters at
the end. This will be the pattern going forward in this series.

Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  4 ++--
 trailer.c                    | 26 +++++++++++++-------------
 trailer.h                    |  6 +++---
 3 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 033bd1556cf..85a3413baf5 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -132,11 +132,11 @@ int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 	if (argc) {
 		int i;
 		for (i = 0; i < argc; i++)
-			process_trailers(argv[i], &opts, &trailers);
+			interpret_trailers(&opts, &trailers, argv[i]);
 	} else {
 		if (opts.in_place)
 			die(_("no input file given for in-place editing"));
-		process_trailers(NULL, &opts, &trailers);
+		interpret_trailers(&opts, &trailers, NULL);
 	}
 
 	new_trailers_clear(&trailers);
diff --git a/trailer.c b/trailer.c
index f74915bd8cd..916175707d8 100644
--- a/trailer.c
+++ b/trailer.c
@@ -163,12 +163,12 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-static void print_all(FILE *outfile, struct list_head *head,
-		      const struct process_trailer_options *opts)
+static void format_trailers(const struct process_trailer_options *opts,
+			    struct list_head *trailers, FILE *outfile)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
-	list_for_each(pos, head) {
+	list_for_each(pos, trailers) {
 		item = list_entry(pos, struct trailer_item, list);
 		if ((!opts->trim_empty || strlen(item->value) > 0) &&
 		    (!opts->only_trailers || item->token))
@@ -589,7 +589,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	return 0;
 }
 
-static void ensure_configured(void)
+static void trailer_config_init(void)
 {
 	if (configured)
 		return;
@@ -1035,10 +1035,10 @@ static void parse_trailers(struct trailer_info *info,
 	}
 }
 
-static void free_all(struct list_head *head)
+static void free_trailers(struct list_head *trailers)
 {
 	struct list_head *pos, *p;
-	list_for_each_safe(pos, p, head) {
+	list_for_each_safe(pos, p, trailers) {
 		list_del(pos);
 		free_trailer_item(list_entry(pos, struct trailer_item, list));
 	}
@@ -1075,16 +1075,16 @@ static FILE *create_in_place_tempfile(const char *file)
 	return outfile;
 }
 
-void process_trailers(const char *file,
-		      const struct process_trailer_options *opts,
-		      struct list_head *new_trailer_head)
+void interpret_trailers(const struct process_trailer_options *opts,
+			struct list_head *new_trailer_head,
+			const char *file)
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
 	struct trailer_info info;
 	FILE *outfile = stdout;
 
-	ensure_configured();
+	trailer_config_init();
 
 	read_input_file(&sb, file);
 
@@ -1110,8 +1110,8 @@ void process_trailers(const char *file,
 		process_trailers_lists(&head, &arg_head);
 	}
 
-	print_all(outfile, &head, opts);
-	free_all(&head);
+	format_trailers(opts, &head, outfile);
+	free_trailers(&head);
 
 	/* Print the lines after the trailers as is */
 	if (!opts->only_trailers)
@@ -1134,7 +1134,7 @@ void trailer_info_get(struct trailer_info *info, const char *str,
 	size_t nr = 0, alloc = 0;
 	char **last = NULL;
 
-	ensure_configured();
+	trailer_config_init();
 
 	end_of_log_message = find_end_of_log_message(str, opts->no_divider);
 	trailer_block_start = find_trailer_block_start(str, end_of_log_message);
diff --git a/trailer.h b/trailer.h
index 1644cd05f60..37033e631a1 100644
--- a/trailer.h
+++ b/trailer.h
@@ -81,9 +81,9 @@ struct process_trailer_options {
 
 #define PROCESS_TRAILER_OPTIONS_INIT {0}
 
-void process_trailers(const char *file,
-		      const struct process_trailer_options *opts,
-		      struct list_head *new_trailer_head);
+void interpret_trailers(const struct process_trailer_options *opts,
+			struct list_head *new_trailer_head,
+			const char *file);
 
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts);
-- 
gitgitgadget

[PATCH v6 4/9] trailer: move interpret_trailers() to interpret-trailers.c

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

The interpret-trailers.c builtin is the only place we need to call
interpret_trailers(), so move its definition there (together with a few
helper functions called only by it) and remove its external declaration
from <trailer.h>.

Several helper functions that are called by interpret_trailers() remain
in trailer.c because other callers in the same file still call them.
Declare them in <trailer.h> so that interpret_trailers() (now in
builtin/interpret-trailers.c) can continue calling them as a trailer API
user.

This enriches <trailer.h> with a more granular API, which can then be
unit-tested in the future (because interpret_trailers() by itself does
too many things to be able to be easily unit-tested).

Take this opportunity to demote some file-handling functions out of the
trailer API implementation, as these have nothing to do with trailers.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  93 +++++++++++++++++++++++++++
 trailer.c                    | 119 ++++-------------------------------
 trailer.h                    |  20 +++++-
 3 files changed, 123 insertions(+), 109 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index 85a3413baf5..d1cf0aa33a2 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -9,6 +9,7 @@
 #include "gettext.h"
 #include "parse-options.h"
 #include "string-list.h"
+#include "tempfile.h"
 #include "trailer.h"
 #include "config.h"
 
@@ -91,6 +92,98 @@ static int parse_opt_parse(const struct option *opt, const char *arg,
 	return 0;
 }
 
+static struct tempfile *trailers_tempfile;
+
+static FILE *create_in_place_tempfile(const char *file)
+{
+	struct stat st;
+	struct strbuf filename_template = STRBUF_INIT;
+	const char *tail;
+	FILE *outfile;
+
+	if (stat(file, &st))
+		die_errno(_("could not stat %s"), file);
+	if (!S_ISREG(st.st_mode))
+		die(_("file %s is not a regular file"), file);
+	if (!(st.st_mode & S_IWUSR))
+		die(_("file %s is not writable by user"), file);
+
+	/* Create temporary file in the same directory as the original */
+	tail = strrchr(file, '/');
+	if (tail)
+		strbuf_add(&filename_template, file, tail - file + 1);
+	strbuf_addstr(&filename_template, "git-interpret-trailers-XXXXXX");
+
+	trailers_tempfile = xmks_tempfile_m(filename_template.buf, st.st_mode);
+	strbuf_release(&filename_template);
+	outfile = fdopen_tempfile(trailers_tempfile, "w");
+	if (!outfile)
+		die_errno(_("could not open temporary file"));
+
+	return outfile;
+}
+
+static void read_input_file(struct strbuf *sb, const char *file)
+{
+	if (file) {
+		if (strbuf_read_file(sb, file, 0) < 0)
+			die_errno(_("could not read input file '%s'"), file);
+	} else {
+		if (strbuf_read(sb, fileno(stdin), 0) < 0)
+			die_errno(_("could not read from stdin"));
+	}
+}
+
+static void interpret_trailers(const struct process_trailer_options *opts,
+			       struct list_head *new_trailer_head,
+			       const char *file)
+{
+	LIST_HEAD(head);
+	struct strbuf sb = STRBUF_INIT;
+	struct trailer_info info;
+	FILE *outfile = stdout;
+
+	trailer_config_init();
+
+	read_input_file(&sb, file);
+
+	if (opts->in_place)
+		outfile = create_in_place_tempfile(file);
+
+	parse_trailers(opts, &info, sb.buf, &head);
+
+	/* Print the lines before the trailers */
+	if (!opts->only_trailers)
+		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
+
+	if (!opts->only_trailers && !info.blank_line_before_trailer)
+		fprintf(outfile, "\n");
+
+
+	if (!opts->only_input) {
+		LIST_HEAD(config_head);
+		LIST_HEAD(arg_head);
+		parse_trailers_from_config(&config_head);
+		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
+		list_splice(&config_head, &arg_head);
+		process_trailers_lists(&head, &arg_head);
+	}
+
+	format_trailers(opts, &head, outfile);
+	free_trailers(&head);
+
+	/* Print the lines after the trailers as is */
+	if (!opts->only_trailers)
+		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
+	trailer_info_release(&info);
+
+	if (opts->in_place)
+		if (rename_tempfile(&trailers_tempfile, file))
+			die_errno(_("could not rename temporary file to %s"), file);
+
+	strbuf_release(&sb);
+}
+
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
 {
 	struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
diff --git a/trailer.c b/trailer.c
index 916175707d8..d23afa0a65c 100644
--- a/trailer.c
+++ b/trailer.c
@@ -5,7 +5,6 @@
 #include "string-list.h"
 #include "run-command.h"
 #include "commit.h"
-#include "tempfile.h"
 #include "trailer.h"
 #include "list.h"
 /*
@@ -163,8 +162,8 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
 }
 
-static void format_trailers(const struct process_trailer_options *opts,
-			    struct list_head *trailers, FILE *outfile)
+void format_trailers(const struct process_trailer_options *opts,
+		     struct list_head *trailers, FILE *outfile)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
@@ -366,8 +365,8 @@ static int find_same_and_apply_arg(struct list_head *head,
 	return 0;
 }
 
-static void process_trailers_lists(struct list_head *head,
-				   struct list_head *arg_head)
+void process_trailers_lists(struct list_head *head,
+			    struct list_head *arg_head)
 {
 	struct list_head *pos, *p;
 	struct arg_item *arg_tok;
@@ -589,7 +588,7 @@ static int git_trailer_config(const char *conf_key, const char *value,
 	return 0;
 }
 
-static void trailer_config_init(void)
+void trailer_config_init(void)
 {
 	if (configured)
 		return;
@@ -719,7 +718,7 @@ static void add_arg_item(struct list_head *arg_head, char *tok, char *val,
 	list_add_tail(&new_item->list, arg_head);
 }
 
-static void parse_trailers_from_config(struct list_head *config_head)
+void parse_trailers_from_config(struct list_head *config_head)
 {
 	struct arg_item *item;
 	struct list_head *pos;
@@ -735,8 +734,8 @@ static void parse_trailers_from_config(struct list_head *config_head)
 	}
 }
 
-static void parse_trailers_from_command_line_args(struct list_head *arg_head,
-						  struct list_head *new_trailer_head)
+void parse_trailers_from_command_line_args(struct list_head *arg_head,
+					   struct list_head *new_trailer_head)
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
@@ -775,17 +774,6 @@ static void parse_trailers_from_command_line_args(struct list_head *arg_head,
 	free(cl_separators);
 }
 
-static void read_input_file(struct strbuf *sb, const char *file)
-{
-	if (file) {
-		if (strbuf_read_file(sb, file, 0) < 0)
-			die_errno(_("could not read input file '%s'"), file);
-	} else {
-		if (strbuf_read(sb, fileno(stdin), 0) < 0)
-			die_errno(_("could not read from stdin"));
-	}
-}
-
 static const char *next_line(const char *str)
 {
 	const char *nl = strchrnul(str, '\n');
@@ -1000,10 +988,10 @@ static void unfold_value(struct strbuf *val)
  * Parse trailers in "str", populating the trailer info and "head"
  * linked list structure.
  */
-static void parse_trailers(struct trailer_info *info,
-			     const char *str,
-			     struct list_head *head,
-			     const struct process_trailer_options *opts)
+void parse_trailers(const struct process_trailer_options *opts,
+		    struct trailer_info *info,
+		    const char *str,
+		    struct list_head *head)
 {
 	struct strbuf tok = STRBUF_INIT;
 	struct strbuf val = STRBUF_INIT;
@@ -1035,7 +1023,7 @@ static void parse_trailers(struct trailer_info *info,
 	}
 }
 
-static void free_trailers(struct list_head *trailers)
+void free_trailers(struct list_head *trailers)
 {
 	struct list_head *pos, *p;
 	list_for_each_safe(pos, p, trailers) {
@@ -1044,87 +1032,6 @@ static void free_trailers(struct list_head *trailers)
 	}
 }
 
-static struct tempfile *trailers_tempfile;
-
-static FILE *create_in_place_tempfile(const char *file)
-{
-	struct stat st;
-	struct strbuf filename_template = STRBUF_INIT;
-	const char *tail;
-	FILE *outfile;
-
-	if (stat(file, &st))
-		die_errno(_("could not stat %s"), file);
-	if (!S_ISREG(st.st_mode))
-		die(_("file %s is not a regular file"), file);
-	if (!(st.st_mode & S_IWUSR))
-		die(_("file %s is not writable by user"), file);
-
-	/* Create temporary file in the same directory as the original */
-	tail = strrchr(file, '/');
-	if (tail)
-		strbuf_add(&filename_template, file, tail - file + 1);
-	strbuf_addstr(&filename_template, "git-interpret-trailers-XXXXXX");
-
-	trailers_tempfile = xmks_tempfile_m(filename_template.buf, st.st_mode);
-	strbuf_release(&filename_template);
-	outfile = fdopen_tempfile(trailers_tempfile, "w");
-	if (!outfile)
-		die_errno(_("could not open temporary file"));
-
-	return outfile;
-}
-
-void interpret_trailers(const struct process_trailer_options *opts,
-			struct list_head *new_trailer_head,
-			const char *file)
-{
-	LIST_HEAD(head);
-	struct strbuf sb = STRBUF_INIT;
-	struct trailer_info info;
-	FILE *outfile = stdout;
-
-	trailer_config_init();
-
-	read_input_file(&sb, file);
-
-	if (opts->in_place)
-		outfile = create_in_place_tempfile(file);
-
-	parse_trailers(&info, sb.buf, &head, opts);
-
-	/* Print the lines before the trailers */
-	if (!opts->only_trailers)
-		fwrite(sb.buf, 1, info.trailer_block_start, outfile);
-
-	if (!opts->only_trailers && !info.blank_line_before_trailer)
-		fprintf(outfile, "\n");
-
-
-	if (!opts->only_input) {
-		LIST_HEAD(config_head);
-		LIST_HEAD(arg_head);
-		parse_trailers_from_config(&config_head);
-		parse_trailers_from_command_line_args(&arg_head, new_trailer_head);
-		list_splice(&config_head, &arg_head);
-		process_trailers_lists(&head, &arg_head);
-	}
-
-	format_trailers(opts, &head, outfile);
-	free_trailers(&head);
-
-	/* Print the lines after the trailers as is */
-	if (!opts->only_trailers)
-		fwrite(sb.buf + info.trailer_block_end, 1, sb.len - info.trailer_block_end, outfile);
-	trailer_info_release(&info);
-
-	if (opts->in_place)
-		if (rename_tempfile(&trailers_tempfile, file))
-			die_errno(_("could not rename temporary file to %s"), file);
-
-	strbuf_release(&sb);
-}
-
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts)
 {
diff --git a/trailer.h b/trailer.h
index 37033e631a1..c292d44b62f 100644
--- a/trailer.h
+++ b/trailer.h
@@ -81,15 +81,29 @@ struct process_trailer_options {
 
 #define PROCESS_TRAILER_OPTIONS_INIT {0}
 
-void interpret_trailers(const struct process_trailer_options *opts,
-			struct list_head *new_trailer_head,
-			const char *file);
+void parse_trailers_from_config(struct list_head *config_head);
+
+void parse_trailers_from_command_line_args(struct list_head *arg_head,
+					   struct list_head *new_trailer_head);
+
+void process_trailers_lists(struct list_head *head,
+			    struct list_head *arg_head);
+
+void parse_trailers(const struct process_trailer_options *,
+		    struct trailer_info *,
+		    const char *str,
+		    struct list_head *head);
 
 void trailer_info_get(struct trailer_info *info, const char *str,
 		      const struct process_trailer_options *opts);
 
 void trailer_info_release(struct trailer_info *info);
 
+void trailer_config_init(void);
+void format_trailers(const struct process_trailer_options *,
+		     struct list_head *trailers, FILE *outfile);
+void free_trailers(struct list_head *);
+
 /*
  * Format the trailers from the commit msg "msg" into the strbuf "out".
  * Note two caveats about "opts":
-- 
gitgitgadget

[PATCH v6 5/9] trailer: reorder format_trailers_from_commit() parameters

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

Currently there are two functions for formatting trailers in
<trailer.h>:

    void format_trailers(const struct process_trailer_options *,
                         struct list_head *trailers, FILE *outfile);

    void format_trailers_from_commit(struct strbuf *out, const char *msg,
                                     const struct process_trailer_options *opts);

and although they are similar enough (even taking the same
process_trailer_options struct pointer) they are used quite differently.
One might intuitively think that format_trailers_from_commit() builds on
top of format_trailers(), but this is not the case. Instead
format_trailers_from_commit() calls format_trailer_info() and
format_trailers() is never called in that codepath.

This is a preparatory refactor to help us deprecate format_trailers() in
favor of format_trailer_info() (at which point we can rename the latter
to the former). When the deprecation is complete, both
format_trailers_from_commit(), and the interpret-trailers builtin will
be able to call into the same helper function (instead of
format_trailers() and format_trailer_info(), respectively). Unifying the
formatters is desirable because it simplifies the API.

Reorder parameters for format_trailers_from_commit() to prefer

    const struct process_trailer_options *opts

as the first parameter, because these options are intimately tied to
formatting trailers. And take

    struct strbuf *out

last, because it's an "out parameter" (something that the caller wants
to use as the output of this function).

Similarly, reorder parameters for format_trailer_info(), because later
on we will unify the two together.

Signed-off-by: Linus Arver <linusa@google.com>
---
 pretty.c     |  2 +-
 ref-filter.c |  2 +-
 trailer.c    | 11 ++++++-----
 trailer.h    |  5 +++--
 4 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/pretty.c b/pretty.c
index cf964b060cd..bdbed4295aa 100644
--- a/pretty.c
+++ b/pretty.c
@@ -1759,7 +1759,7 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */
 				goto trailer_out;
 		}
 		if (*arg == ')') {
-			format_trailers_from_commit(sb, msg + c->subject_off, &opts);
+			format_trailers_from_commit(&opts, msg + c->subject_off, sb);
 			ret = arg - placeholder + 1;
 		}
 	trailer_out:
diff --git a/ref-filter.c b/ref-filter.c
index 35b989e1dfe..d358953b0ce 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -1985,7 +1985,7 @@ static void grab_sub_body_contents(struct atom_value *val, int deref, struct exp
 			struct strbuf s = STRBUF_INIT;
 
 			/* Format the trailer info according to the trailer_opts given */
-			format_trailers_from_commit(&s, subpos, &atom->u.contents.trailer_opts);
+			format_trailers_from_commit(&atom->u.contents.trailer_opts, subpos, &s);
 
 			v->s = strbuf_detach(&s, NULL);
 		} else if (atom->u.contents.option == C_BARE)
diff --git a/trailer.c b/trailer.c
index d23afa0a65c..5025be97899 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1083,10 +1083,10 @@ void trailer_info_release(struct trailer_info *info)
 	free(info->trailers);
 }
 
-static void format_trailer_info(struct strbuf *out,
+static void format_trailer_info(const struct process_trailer_options *opts,
 				const struct trailer_info *info,
 				const char *msg,
-				const struct process_trailer_options *opts)
+				struct strbuf *out)
 {
 	size_t origlen = out->len;
 	size_t i;
@@ -1144,13 +1144,14 @@ static void format_trailer_info(struct strbuf *out,
 
 }
 
-void format_trailers_from_commit(struct strbuf *out, const char *msg,
-				 const struct process_trailer_options *opts)
+void format_trailers_from_commit(const struct process_trailer_options *opts,
+				 const char *msg,
+				 struct strbuf *out)
 {
 	struct trailer_info info;
 
 	trailer_info_get(&info, msg, opts);
-	format_trailer_info(out, &info, msg, opts);
+	format_trailer_info(opts, &info, msg, out);
 	trailer_info_release(&info);
 }
 
diff --git a/trailer.h b/trailer.h
index c292d44b62f..c6d3ee49bbf 100644
--- a/trailer.h
+++ b/trailer.h
@@ -115,8 +115,9 @@ void free_trailers(struct list_head *);
  *     only the trailer block itself, even if the "only_trailers" option is not
  *     set.
  */
-void format_trailers_from_commit(struct strbuf *out, const char *msg,
-				 const struct process_trailer_options *opts);
+void format_trailers_from_commit(const struct process_trailer_options *opts,
+				 const char *msg,
+				 struct strbuf *out);
 
 /*
  * An interface for iterating over the trailers found in a particular commit
-- 
gitgitgadget

[PATCH v6 6/9] trailer_info_get(): reorder parameters

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

Take

    const struct process_trailer_options *opts

as the first parameter, because these options are required for
parsing trailers (e.g., whether to treat "---" as the end of the log
message). And take

    struct trailer_info *info

last, because it's an "out parameter" (something that the caller wants
to use as the output of this function).

Signed-off-by: Linus Arver <linusa@google.com>
---
 sequencer.c |  2 +-
 trailer.c   | 11 ++++++-----
 trailer.h   |  5 +++--
 3 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 3cc88d8a800..8e199fc8a47 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -332,7 +332,7 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 		sb->buf[sb->len - ignore_footer] = '\0';
 	}
 
-	trailer_info_get(&info, sb->buf, &opts);
+	trailer_info_get(&opts, sb->buf, &info);
 
 	if (ignore_footer)
 		sb->buf[sb->len - ignore_footer] = saved_char;
diff --git a/trailer.c b/trailer.c
index 5025be97899..f92d844361a 100644
--- a/trailer.c
+++ b/trailer.c
@@ -997,7 +997,7 @@ void parse_trailers(const struct process_trailer_options *opts,
 	struct strbuf val = STRBUF_INIT;
 	size_t i;
 
-	trailer_info_get(info, str, opts);
+	trailer_info_get(opts, str, info);
 
 	for (i = 0; i < info->trailer_nr; i++) {
 		int separator_pos;
@@ -1032,8 +1032,9 @@ void free_trailers(struct list_head *trailers)
 	}
 }
 
-void trailer_info_get(struct trailer_info *info, const char *str,
-		      const struct process_trailer_options *opts)
+void trailer_info_get(const struct process_trailer_options *opts,
+		      const char *str,
+		      struct trailer_info *info)
 {
 	size_t end_of_log_message = 0, trailer_block_start = 0;
 	struct strbuf **trailer_lines, **ptr;
@@ -1150,7 +1151,7 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 {
 	struct trailer_info info;
 
-	trailer_info_get(&info, msg, opts);
+	trailer_info_get(opts, msg, &info);
 	format_trailer_info(opts, &info, msg, out);
 	trailer_info_release(&info);
 }
@@ -1161,7 +1162,7 @@ void trailer_iterator_init(struct trailer_iterator *iter, const char *msg)
 	strbuf_init(&iter->key, 0);
 	strbuf_init(&iter->val, 0);
 	opts.no_divider = 1;
-	trailer_info_get(&iter->internal.info, msg, &opts);
+	trailer_info_get(&opts, msg, &iter->internal.info);
 	iter->internal.cur = 0;
 }
 
diff --git a/trailer.h b/trailer.h
index c6d3ee49bbf..410c61b62be 100644
--- a/trailer.h
+++ b/trailer.h
@@ -94,8 +94,9 @@ void parse_trailers(const struct process_trailer_options *,
 		    const char *str,
 		    struct list_head *head);
 
-void trailer_info_get(struct trailer_info *info, const char *str,
-		      const struct process_trailer_options *opts);
+void trailer_info_get(const struct process_trailer_options *,
+		      const char *str,
+		      struct trailer_info *);
 
 void trailer_info_release(struct trailer_info *info);
 
-- 
gitgitgadget

[PATCH v6 7/9] format_trailers(): use strbuf instead of FILE

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

Make format_trailers() also write to a strbuf, to align with
format_trailers_from_commit() which also does the same. Doing this makes
format_trailers() behave similar to format_trailer_info() (which will
soon help us replace one with the other).

Signed-off-by: Linus Arver <linusa@google.com>
---
 builtin/interpret-trailers.c |  6 +++++-
 trailer.c                    | 13 +++++++------
 trailer.h                    |  3 ++-
 3 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
index d1cf0aa33a2..11f4ce9e4a2 100644
--- a/builtin/interpret-trailers.c
+++ b/builtin/interpret-trailers.c
@@ -140,6 +140,7 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 {
 	LIST_HEAD(head);
 	struct strbuf sb = STRBUF_INIT;
+	struct strbuf trailer_block = STRBUF_INIT;
 	struct trailer_info info;
 	FILE *outfile = stdout;
 
@@ -169,8 +170,11 @@ static void interpret_trailers(const struct process_trailer_options *opts,
 		process_trailers_lists(&head, &arg_head);
 	}
 
-	format_trailers(opts, &head, outfile);
+	/* Print trailer block. */
+	format_trailers(opts, &head, &trailer_block);
 	free_trailers(&head);
+	fwrite(trailer_block.buf, 1, trailer_block.len, outfile);
+	strbuf_release(&trailer_block);
 
 	/* Print the lines after the trailers as is */
 	if (!opts->only_trailers)
diff --git a/trailer.c b/trailer.c
index f92d844361a..cbd643cd1fe 100644
--- a/trailer.c
+++ b/trailer.c
@@ -144,12 +144,12 @@ static char last_non_space_char(const char *s)
 	return '\0';
 }
 
-static void print_tok_val(FILE *outfile, const char *tok, const char *val)
+static void print_tok_val(struct strbuf *out, const char *tok, const char *val)
 {
 	char c;
 
 	if (!tok) {
-		fprintf(outfile, "%s\n", val);
+		strbuf_addf(out, "%s\n", val);
 		return;
 	}
 
@@ -157,13 +157,14 @@ static void print_tok_val(FILE *outfile, const char *tok, const char *val)
 	if (!c)
 		return;
 	if (strchr(separators, c))
-		fprintf(outfile, "%s%s\n", tok, val);
+		strbuf_addf(out, "%s%s\n", tok, val);
 	else
-		fprintf(outfile, "%s%c %s\n", tok, separators[0], val);
+		strbuf_addf(out, "%s%c %s\n", tok, separators[0], val);
 }
 
 void format_trailers(const struct process_trailer_options *opts,
-		     struct list_head *trailers, FILE *outfile)
+		     struct list_head *trailers,
+		     struct strbuf *out)
 {
 	struct list_head *pos;
 	struct trailer_item *item;
@@ -171,7 +172,7 @@ void format_trailers(const struct process_trailer_options *opts,
 		item = list_entry(pos, struct trailer_item, list);
 		if ((!opts->trim_empty || strlen(item->value) > 0) &&
 		    (!opts->only_trailers || item->token))
-			print_tok_val(outfile, item->token, item->value);
+			print_tok_val(out, item->token, item->value);
 	}
 }
 
diff --git a/trailer.h b/trailer.h
index 410c61b62be..1d106b6dd40 100644
--- a/trailer.h
+++ b/trailer.h
@@ -102,7 +102,8 @@ void trailer_info_release(struct trailer_info *info);
 
 void trailer_config_init(void);
 void format_trailers(const struct process_trailer_options *,
-		     struct list_head *trailers, FILE *outfile);
+		     struct list_head *trailers,
+		     struct strbuf *out);
 void free_trailers(struct list_head *);
 
 /*
-- 
gitgitgadget

[PATCH v6 8/9] format_trailer_info(): move "fast path" to caller

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

This allows us to drop the "msg" parameter from format_trailer_info(),
so that it take 3 parameters, similar to format_trailers() which also
takes 3 parameters:

    void format_trailers(const struct process_trailer_options *opts,
                         struct list_head *trailers,
                         struct strbuf *out)

The short-term goal is to make format_trailer_info() be smart enough to
deprecate format_trailers(). And then ultimately we will rename
format_trailer_info() to format_trailers().

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/trailer.c b/trailer.c
index cbd643cd1fe..e92d0154d90 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1087,21 +1087,11 @@ void trailer_info_release(struct trailer_info *info)
 
 static void format_trailer_info(const struct process_trailer_options *opts,
 				const struct trailer_info *info,
-				const char *msg,
 				struct strbuf *out)
 {
 	size_t origlen = out->len;
 	size_t i;
 
-	/* If we want the whole block untouched, we can take the fast path. */
-	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
-	    !opts->separator && !opts->key_only && !opts->value_only &&
-	    !opts->key_value_separator) {
-		strbuf_add(out, msg + info->trailer_block_start,
-			   info->trailer_block_end - info->trailer_block_start);
-		return;
-	}
-
 	for (i = 0; i < info->trailer_nr; i++) {
 		char *trailer = info->trailers[i];
 		ssize_t separator_pos = find_separator(trailer, separators);
@@ -1153,7 +1143,15 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 	struct trailer_info info;
 
 	trailer_info_get(opts, msg, &info);
-	format_trailer_info(opts, &info, msg, out);
+	/* If we want the whole block untouched, we can take the fast path. */
+	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
+	    !opts->separator && !opts->key_only && !opts->value_only &&
+	    !opts->key_value_separator) {
+		strbuf_add(out, msg + info.trailer_block_start,
+			   info.trailer_block_end - info.trailer_block_start);
+	} else
+		format_trailer_info(opts, &info, out);
+
 	trailer_info_release(&info);
 }
 
-- 
gitgitgadget

[PATCH v6 9/9] format_trailers_from_commit(): indirectly call trailer_info_get()

[Linus Arver via GitGitGadget]

From: Linus Arver <linusa@google.com>

This is another preparatory refactor to unify the trailer formatters.

For background, note that the "trailers" string array is the
`char **trailers` member in `struct trailer_info` and that the
trailer_item objects are the elements of the `struct list_head *head`
linked list.

Currently trailer_info_get() only populates `char **trailers`. And
parse_trailers() first calls trailer_info_get() so that it can use the
`char **trailers` to populate a list of `struct trailer_item` objects

Instead of calling trailer_info_get() directly from
format_trailers_from_commit(), make it call parse_trailers() instead
because parse_trailers() already calls trailer_info_get().

This change is a NOP because format_trailer_info() (which
format_trailers_from_commit() wraps around) only looks at the "trailers"
string array, not the trailer_item objects which parse_trailers()
populates. For now we do need to create a dummy

    LIST_HEAD(trailer_objects);

because parse_trailers() expects it in its signature.

In a future patch, we'll change format_trailer_info() to use the parsed
trailer_item objects (trailer_objects) instead of the `char **trailers`
array.

Signed-off-by: Linus Arver <linusa@google.com>
---
 trailer.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/trailer.c b/trailer.c
index e92d0154d90..798388cbf29 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1140,9 +1140,11 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 				 const char *msg,
 				 struct strbuf *out)
 {
+	LIST_HEAD(trailer_objects);
 	struct trailer_info info;
 
-	trailer_info_get(opts, msg, &info);
+	parse_trailers(opts, &info, msg, &trailer_objects);
+
 	/* If we want the whole block untouched, we can take the fast path. */
 	if (!opts->only_trailers && !opts->unfold && !opts->filter &&
 	    !opts->separator && !opts->key_only && !opts->value_only &&
@@ -1152,6 +1154,7 @@ void format_trailers_from_commit(const struct process_trailer_options *opts,
 	} else
 		format_trailer_info(opts, &info, out);
 
+	free_trailers(&trailer_objects);
 	trailer_info_release(&info);
 }
 
-- 
gitgitgadget

Re: [PATCH v6 0/9] Enrich Trailer API

[Junio C Hamano]

"Linus Arver via GitGitGadget" <gitgitgadget@gmail.com> writes:

> This patch series is the first 9 patches of a larger cleanup/bugfix series
> (henceforth "larger series") I've been working on. The main goal of this
> series is to begin the process of "libifying" the trailer API. By "API" I
> mean the interface exposed in trailer.h. The larger series brings a number
> of additional cleanups (exposing and fixing some bugs along the way), and
> builds on top of this series.
>
> When the larger series is merged, we will be in a good state to additionally
> pursue the following goals:
>
>  1. "API reuse inside Git": make the API expressive enough to eliminate any
>     need by other parts of Git to use the interpret-trailers builtin as a
>     subprocess (instead they could just use the API directly);
>  2. "API stability": add unit tests to codify the expected behavior of API
>     functions; and
>  3. "API documentation": create developer-focused documentation to explain
>     how to use the API effectively, noting any API limitations or
>     anti-patterns.
>
> In the future after libification is "complete", users external to Git will
> be able to use the same trailer processing API used by the
> interpret-trailers builtin. For example, a web server may want to parse
> trailers the same way that Git would parse them, without having to call
> interpret-trailers as a subprocess. This use case was the original
> motivation behind my work in this area.
>
> With the libification-focused goals out of the way, let's turn to this patch
> series in more detail.
>
> In summary this series breaks up "process_trailers()" into smaller pieces,
> exposing many of the parts relevant to trailer-related processing in
> trailer.h. This will force us to eventually introduce unit tests for these
> API functions, but that is a good thing for API stability. We also perform
> some preparatory refactors in order to help us unify the trailer formatting
> machinery toward the end of this series.
>
>
> Notable changes in v6
> =====================
>
>  * Mainly wording changes to commit messages. Thanks to Christian for the
>    suggestions.

It's been nearly a week since this was posted.  Any more comments,
or is everybody happy with this iteration?  Otherwise I am tempted
to mark the topic for 'next' soon.

Thanks.

Re: [PATCH v6 0/9] Enrich Trailer API

[Josh Steadmon]

On 2024.03.05 10:03, Junio C Hamano wrote:
> 
> It's been nearly a week since this was posted.  Any more comments,
> or is everybody happy with this iteration?  Otherwise I am tempted
> to mark the topic for 'next' soon.
> 
> Thanks.

I scanned through v6 yesterday and have nothing new to add. LGTM.

Re: [PATCH v6 0/9] Enrich Trailer API

[Junio C Hamano]

Josh Steadmon <steadmon@google.com> writes:

> On 2024.03.05 10:03, Junio C Hamano wrote:
>> 
>> It's been nearly a week since this was posted.  Any more comments,
>> or is everybody happy with this iteration?  Otherwise I am tempted
>> to mark the topic for 'next' soon.
>> 
>> Thanks.
>
> I scanned through v6 yesterday and have nothing new to add. LGTM.

Thanks.

Re: [PATCH v6 0/9] Enrich Trailer API

[Christian Couder]

On Tue, Mar 5, 2024 at 8:07 PM Josh Steadmon <steadmon@google.com> wrote:
>
> On 2024.03.05 10:03, Junio C Hamano wrote:
> >
> > It's been nearly a week since this was posted.  Any more comments,
> > or is everybody happy with this iteration?  Otherwise I am tempted
> > to mark the topic for 'next' soon.
> >
> > Thanks.
>
> I scanned through v6 yesterday and have nothing new to add. LGTM.

I took another look at it, and I am fine with it now too. Acked.

Re: [PATCH v6 0/9] Enrich Trailer API

[Junio C Hamano]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Mar 5, 2024 at 8:07 PM Josh Steadmon <steadmon@google.com> wrote:
>>
>> On 2024.03.05 10:03, Junio C Hamano wrote:
>> >
>> > It's been nearly a week since this was posted.  Any more comments,
>> > or is everybody happy with this iteration?  Otherwise I am tempted
>> > to mark the topic for 'next' soon.
>> >
>> > Thanks.
>>
>> I scanned through v6 yesterday and have nothing new to add. LGTM.
>
> I took another look at it, and I am fine with it now too. Acked.

Thanks.

Re: [PATCH v6 0/9] Enrich Trailer API

[Junio C Hamano]

Christian Couder <christian.couder@gmail.com> writes:

> On Tue, Mar 5, 2024 at 8:07 PM Josh Steadmon <steadmon@google.com> wrote:
>>
>> On 2024.03.05 10:03, Junio C Hamano wrote:
>> >
>> > It's been nearly a week since this was posted.  Any more comments,
>> > or is everybody happy with this iteration?  Otherwise I am tempted
>> > to mark the topic for 'next' soon.
>> >
>> > Thanks.
>>
>> I scanned through v6 yesterday and have nothing new to add. LGTM.
>
> I took another look at it, and I am fine with it now too. Acked.

Thanks.

Re: [PATCH v4 15/28] format_trailer_info(): avoid double-printing the separator

[Linus Arver]

Linus Arver <linusa@google.com> writes:

> Christian Couder <christian.couder@gmail.com> writes:
>
>> On Tue, Feb 13, 2024 at 6:21 PM Linus Arver <linusa@google.com> wrote:
>>>
>>> Christian Couder <christian.couder@gmail.com> writes:
>>>
>>> > On Tue, Feb 6, 2024 at 6:12 AM Linus Arver via GitGitGadget
>>> > <gitgitgadget@gmail.com> wrote:
>>
>>> > Also I wonder why it was not possible to modify format_trailer_info()
>>> > like it is done in this patch before using it to replace
>>> > format_trailers().
>>>
>>> The artificial organization apparent in this patch was deliberate, in
>>> order to make it painfully obvious exactly what was being replaced and
>>> how. See https://lore.kernel.org/git/xmqqjzno13ev.fsf@gitster.g/
>>
>> As for the previous patch, I would have thought that it would be
>> better not to break the tests.
>
> I could just squash these patches together to avoid breaking tests (and
> also avoid doing the flipping of expect_success to expect_fail and back
> again). I don't mind at all which way we go, but now that we have these
> patches broken out I wonder if it's better to just keep them that way.
>
> Junio, do you mind if I squash the relevant changes together into just
> one patch?  I'd like your input because you requested the current style
> (modulo test breakages which was my error). Thanks.

When I asked this question, I forgot that the number of test cases that
break are around ~50. This is a very large number. So I think it
would be cleaner to squash this and the previous patch down to avoid having
to flip test_expect_{success,failure} for 50+ individual test cases.

For the earlier patch

    [PATCH v4 10/28] format_trailer_info(): use trailer_item objects

there are only 8 failures so I think doing the *_{success,failure} flip
is reasonable.