[PATCH v10 00/12] Add interpret-trailers builtin

[PATCH v10 00/12] Add interpret-trailers builtin

[Christian Couder]

This patch series implements a new command:

        git interpret-trailers

and an infrastructure to process trailers that can be reused,
for example in "commit.c".

1) Rationale:

This command should help with RFC 822 style headers, called
"trailers", that are found at the end of commit messages.

(Note that these headers do not follow and are not intended to
follow many rules that are in RFC 822. For example they do not
follow the line breaking rules, the encoding rules and probably
many other rules.)

For a long time, these trailers have become a de facto standard
way to add helpful information into commit messages.

Until now git commit has only supported the well known
"Signed-off-by: " trailer, that is used by many projects like
the Linux kernel and Git.

It is better to keep builtin/commit.c uncontaminated by any more
hard-wired logic, like what we have for the signed-off-by line.  Any
new things can and should be doable in hooks, and this filter would
help writing these hooks.

And that is why the design goal of the filter is to make it at least
as powerful as the built-in logic we have for signed-off-by lines;
that would allow us to later eject the hard-wired logic for
signed-off-by line from the main codepath, if/when we wanted to.

Alternatively, we could build a library-ish API around this filter
code and replace the hard-wired logic for signed-off-by line with a
call into that API, if/when we wanted to, but that requires (in
addition to the "at least as powerful as the built-in logic") that
the implementation of this stand-alone filter can be cleanly made
into a reusable library, so that is a bit higher bar to cross than
"everything can be doable with hooks" alternative.

2) Current state:

Currently the usage string of this command is:

git interpret-trailers [--trim-empty] [(<token>[(=|:)<value>])...]

The following features are implemented:

        - the result is printed on stdout
        - the [<token>[=<value>]>] arguments are interpreted
        - a commit message read from stdin is interpreted
        - the "trailer.<token>.key" options in the config are interpreted
        - the "trailer.<token>.where" options are interpreted
        - the "trailer.<token>.ifExist" options are interpreted
        - the "trailer.<token>.ifMissing" options are interpreted
        - the "trailer.<token>.command" config works
        - $ARG can be used in commands
        - there are 31 tests (4 more than in version 9)
        - there is some documentation

The following features are planned but not yet implemented:
        - add examples in documentation

Possible improvements:
        - integration with "git commit"
        - support GIT_COMMIT_PROTO env variable in commands

3) Changes since version 9, thanks to Jonathan and Junio:

* added 1 test with empty trailers in patch 10/12
* fixed bugs when there was no 'key' in the config in patch
  4/12 and added 2 related tests in patch 10/12
* fixed bug when command failed in patch 9/12 and added 1
  related test in patch 10/12
* added patch 12/12 which add one blank line before the
  trailers if there is not one already

This means code changes only in patches 4/12, 9/12, 10/12
and 12/12.


Christian Couder (12):
  trailer: add data structures and basic functions
  trailer: process trailers from stdin and arguments
  trailer: read and process config information
  trailer: process command line trailer arguments
  trailer: parse trailers from stdin
  trailer: put all the processing together and print
  trailer: add interpret-trailers command
  trailer: add tests for "git interpret-trailers"
  trailer: execute command from 'trailer.<name>.command'
  trailer: add tests for commands in config file
  Documentation: add documentation for 'git interpret-trailers'
  trailer: add blank line before the trailers if needed

 .gitignore                               |   1 +
 Documentation/git-interpret-trailers.txt | 123 ++++++
 Makefile                                 |   2 +
 builtin.h                                |   1 +
 builtin/interpret-trailers.c             |  33 ++
 git.c                                    |   1 +
 t/t7513-interpret-trailers.sh            | 477 +++++++++++++++++++++
 trailer.c                                | 709 +++++++++++++++++++++++++++++++
 trailer.h                                |   6 +
 9 files changed, 1353 insertions(+)
 create mode 100644 Documentation/git-interpret-trailers.txt
 create mode 100644 builtin/interpret-trailers.c
 create mode 100755 t/t7513-interpret-trailers.sh
 create mode 100644 trailer.c
 create mode 100644 trailer.h

-- 
1.9.0.163.g8ca203c

[PATCH v10 01/12] trailer: add data structures and basic functions

[Christian Couder]

We will use a doubly linked list to store all information
about trailers and their configuration.

This way we can easily remove or add trailers to or from
trailer lists while traversing the lists in either direction.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Makefile  |  1 +
 trailer.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)
 create mode 100644 trailer.c

diff --git a/Makefile b/Makefile
index c5316a3..179be0a 100644
--- a/Makefile
+++ b/Makefile
@@ -879,6 +879,7 @@ LIB_OBJS += submodule.o
 LIB_OBJS += symlinks.o
 LIB_OBJS += tag.o
 LIB_OBJS += trace.o
+LIB_OBJS += trailer.o
 LIB_OBJS += transport.o
 LIB_OBJS += transport-helper.o
 LIB_OBJS += tree-diff.o
diff --git a/trailer.c b/trailer.c
new file mode 100644
index 0000000..db93a63
--- /dev/null
+++ b/trailer.c
@@ -0,0 +1,49 @@
+#include "cache.h"
+/*
+ * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
+ */
+
+enum action_where { WHERE_AFTER, WHERE_BEFORE };
+enum action_if_exists { EXISTS_ADD_IF_DIFFERENT, EXISTS_ADD_IF_DIFFERENT_NEIGHBOR,
+			EXISTS_ADD, EXISTS_OVERWRITE, EXISTS_DO_NOTHING };
+enum action_if_missing { MISSING_ADD, MISSING_DO_NOTHING };
+
+struct conf_info {
+	char *name;
+	char *key;
+	char *command;
+	enum action_where where;
+	enum action_if_exists if_exists;
+	enum action_if_missing if_missing;
+};
+
+struct trailer_item {
+	struct trailer_item *previous;
+	struct trailer_item *next;
+	const char *token;
+	const char *value;
+	struct conf_info conf;
+};
+
+static int same_token(struct trailer_item *a, struct trailer_item *b, int alnum_len)
+{
+	return !strncasecmp(a->token, b->token, alnum_len);
+}
+
+static int same_value(struct trailer_item *a, struct trailer_item *b)
+{
+	return !strcasecmp(a->value, b->value);
+}
+
+static int same_trailer(struct trailer_item *a, struct trailer_item *b, int alnum_len)
+{
+	return same_token(a, b, alnum_len) && same_value(a, b);
+}
+
+/* Get the length of buf from its beginning until its last alphanumeric character */
+static size_t alnum_len(const char *buf, size_t len)
+{
+	while (len > 0 && !isalnum(buf[len - 1]))
+		len--;
+	return len;
+}
-- 
1.9.0.163.g8ca203c

[PATCH v10 02/12] trailer: process trailers from stdin and arguments

[Christian Couder]

Implement the logic to process trailers from stdin and arguments.

At the beginning trailers from stdin are in their own in_tok
doubly linked list, and trailers from arguments are in their own
arg_tok doubly linked list.

The lists are traversed and when an arg_tok should be "applied",
it is removed from its list and inserted into the in_tok list.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 trailer.c | 198 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 198 insertions(+)

diff --git a/trailer.c b/trailer.c
index db93a63..52108c2 100644
--- a/trailer.c
+++ b/trailer.c
@@ -47,3 +47,201 @@ static size_t alnum_len(const char *buf, size_t len)
 		len--;
 	return len;
 }
+
+static void free_trailer_item(struct trailer_item *item)
+{
+	free(item->conf.name);
+	free(item->conf.key);
+	free(item->conf.command);
+	free((char *)item->token);
+	free((char *)item->value);
+	free(item);
+}
+
+static void add_arg_to_input_list(struct trailer_item *in_tok,
+				  struct trailer_item *arg_tok)
+{
+	if (arg_tok->conf.where == WHERE_AFTER) {
+		arg_tok->next = in_tok->next;
+		in_tok->next = arg_tok;
+		arg_tok->previous = in_tok;
+		if (arg_tok->next)
+			arg_tok->next->previous = arg_tok;
+	} else {
+		arg_tok->previous = in_tok->previous;
+		in_tok->previous = arg_tok;
+		arg_tok->next = in_tok;
+		if (arg_tok->previous)
+			arg_tok->previous->next = arg_tok;
+	}
+}
+
+static int check_if_different(struct trailer_item *in_tok,
+			      struct trailer_item *arg_tok,
+			      int alnum_len, int check_all)
+{
+	enum action_where where = arg_tok->conf.where;
+	do {
+		if (!in_tok)
+			return 1;
+		if (same_trailer(in_tok, arg_tok, alnum_len))
+			return 0;
+		/*
+		 * if we want to add a trailer after another one,
+		 * we have to check those before this one
+		 */
+		in_tok = (where == WHERE_AFTER) ? in_tok->previous : in_tok->next;
+	} while (check_all);
+	return 1;
+}
+
+static void apply_arg_if_exists(struct trailer_item *in_tok,
+				struct trailer_item *arg_tok,
+				int alnum_len)
+{
+	switch (arg_tok->conf.if_exists) {
+	case EXISTS_DO_NOTHING:
+		free_trailer_item(arg_tok);
+		break;
+	case EXISTS_OVERWRITE:
+		free((char *)in_tok->value);
+		in_tok->value = xstrdup(arg_tok->value);
+		free_trailer_item(arg_tok);
+		break;
+	case EXISTS_ADD:
+		add_arg_to_input_list(in_tok, arg_tok);
+		break;
+	case EXISTS_ADD_IF_DIFFERENT:
+		if (check_if_different(in_tok, arg_tok, alnum_len, 1))
+			add_arg_to_input_list(in_tok, arg_tok);
+		else
+			free_trailer_item(arg_tok);
+		break;
+	case EXISTS_ADD_IF_DIFFERENT_NEIGHBOR:
+		if (check_if_different(in_tok, arg_tok, alnum_len, 0))
+			add_arg_to_input_list(in_tok, arg_tok);
+		else
+			free_trailer_item(arg_tok);
+		break;
+	}
+}
+
+static void remove_from_list(struct trailer_item *item,
+			     struct trailer_item **first)
+{
+	if (item->next)
+		item->next->previous = item->previous;
+	if (item->previous)
+		item->previous->next = item->next;
+	else
+		*first = item->next;
+}
+
+static struct trailer_item *remove_first(struct trailer_item **first)
+{
+	struct trailer_item *item = *first;
+	*first = item->next;
+	if (item->next) {
+		item->next->previous = NULL;
+		item->next = NULL;
+	}
+	return item;
+}
+
+static void process_input_token(struct trailer_item *in_tok,
+				struct trailer_item **arg_tok_first,
+				enum action_where where)
+{
+	struct trailer_item *arg_tok;
+	struct trailer_item *next_arg;
+
+	int after = where == WHERE_AFTER;
+	int tok_alnum_len = alnum_len(in_tok->token, strlen(in_tok->token));
+
+	for (arg_tok = *arg_tok_first; arg_tok; arg_tok = next_arg) {
+		next_arg = arg_tok->next;
+		if (!same_token(in_tok, arg_tok, tok_alnum_len))
+			continue;
+		if (arg_tok->conf.where != where)
+			continue;
+		remove_from_list(arg_tok, arg_tok_first);
+		apply_arg_if_exists(in_tok, arg_tok, tok_alnum_len);
+		/*
+		 * If arg has been added to input,
+		 * then we need to process it too now.
+		 */
+		if ((after ? in_tok->next : in_tok->previous) == arg_tok)
+			in_tok = arg_tok;
+	}
+}
+
+static void update_last(struct trailer_item **last)
+{
+	if (*last)
+		while ((*last)->next != NULL)
+			*last = (*last)->next;
+}
+
+static void update_first(struct trailer_item **first)
+{
+	if (*first)
+		while ((*first)->previous != NULL)
+			*first = (*first)->previous;
+}
+
+static void apply_arg_if_missing(struct trailer_item **in_tok_first,
+				 struct trailer_item **in_tok_last,
+				 struct trailer_item *arg_tok)
+{
+	struct trailer_item **in_tok;
+	enum action_where where;
+
+	switch (arg_tok->conf.if_missing) {
+	case MISSING_DO_NOTHING:
+		free_trailer_item(arg_tok);
+		break;
+	case MISSING_ADD:
+		where = arg_tok->conf.where;
+		in_tok = (where == WHERE_AFTER) ? in_tok_last : in_tok_first;
+		if (*in_tok) {
+			add_arg_to_input_list(*in_tok, arg_tok);
+			*in_tok = arg_tok;
+		} else {
+			*in_tok_first = arg_tok;
+			*in_tok_last = arg_tok;
+		}
+		break;
+	}
+}
+
+static void process_trailers_lists(struct trailer_item **in_tok_first,
+				   struct trailer_item **in_tok_last,
+				   struct trailer_item **arg_tok_first)
+{
+	struct trailer_item *in_tok;
+	struct trailer_item *arg_tok;
+
+	if (!*arg_tok_first)
+		return;
+
+	/* Process input from end to start */
+	for (in_tok = *in_tok_last; in_tok; in_tok = in_tok->previous)
+		process_input_token(in_tok, arg_tok_first, WHERE_AFTER);
+
+	update_last(in_tok_last);
+
+	if (!*arg_tok_first)
+		return;
+
+	/* Process input from start to end */
+	for (in_tok = *in_tok_first; in_tok; in_tok = in_tok->next)
+		process_input_token(in_tok, arg_tok_first, WHERE_BEFORE);
+
+	update_first(in_tok_first);
+
+	/* Process args left */
+	while (*arg_tok_first) {
+		arg_tok = remove_first(arg_tok_first);
+		apply_arg_if_missing(in_tok_first, in_tok_last, arg_tok);
+	}
+}
-- 
1.9.0.163.g8ca203c

[PATCH v10 03/12] trailer: read and process config information

[Christian Couder]

Read the configuration to get trailer information, and then process
it and storing it in a doubly linked list.

The config information is stored in the list whose first item is
pointed to by:

static struct trailer_item *first_conf_item;

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 trailer.c | 146 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 146 insertions(+)

diff --git a/trailer.c b/trailer.c
index 52108c2..c7c0f54 100644
--- a/trailer.c
+++ b/trailer.c
@@ -25,6 +25,8 @@ struct trailer_item {
 	struct conf_info conf;
 };
 
+static struct trailer_item *first_conf_item;
+
 static int same_token(struct trailer_item *a, struct trailer_item *b, int alnum_len)
 {
 	return !strncasecmp(a->token, b->token, alnum_len);
@@ -245,3 +247,147 @@ static void process_trailers_lists(struct trailer_item **in_tok_first,
 		apply_arg_if_missing(in_tok_first, in_tok_last, arg_tok);
 	}
 }
+
+static int set_where(struct conf_info *item, const char *value)
+{
+	if (!strcmp("after", value))
+		item->where = WHERE_AFTER;
+	else if (!strcmp("before", value))
+		item->where = WHERE_BEFORE;
+	else
+		return -1;
+	return 0;
+}
+
+static int set_if_exists(struct conf_info *item, const char *value)
+{
+	if (!strcmp("addIfDifferent", value))
+		item->if_exists = EXISTS_ADD_IF_DIFFERENT;
+	else if (!strcmp("addIfDifferentNeighbor", value))
+		item->if_exists = EXISTS_ADD_IF_DIFFERENT_NEIGHBOR;
+	else if (!strcmp("add", value))
+		item->if_exists = EXISTS_ADD;
+	else if (!strcmp("overwrite", value))
+		item->if_exists = EXISTS_OVERWRITE;
+	else if (!strcmp("doNothing", value))
+		item->if_exists = EXISTS_DO_NOTHING;
+	else
+		return -1;
+	return 0;
+}
+
+static int set_if_missing(struct conf_info *item, const char *value)
+{
+	if (!strcmp("doNothing", value))
+		item->if_missing = MISSING_DO_NOTHING;
+	else if (!strcmp("add", value))
+		item->if_missing = MISSING_ADD;
+	else
+		return -1;
+	return 0;
+}
+
+static struct trailer_item *get_conf_item(const char *name)
+{
+	struct trailer_item *item;
+	struct trailer_item *previous;
+
+	/* Look up item with same name */
+	for (previous = NULL, item = first_conf_item;
+	     item;
+	     previous = item, item = item->next) {
+		if (!strcasecmp(item->conf.name, name))
+			return item;
+	}
+
+	/* Item does not already exists, create it */
+	item = xcalloc(sizeof(struct trailer_item), 1);
+	item->conf.name = xstrdup(name);
+
+	if (!previous)
+		first_conf_item = item;
+	else {
+		previous->next = item;
+		item->previous = previous;
+	}
+
+	return item;
+}
+
+enum trailer_info_type { TRAILER_KEY, TRAILER_COMMAND, TRAILER_WHERE,
+			 TRAILER_IF_EXISTS, TRAILER_IF_MISSING };
+
+static struct {
+	const char *name;
+	enum trailer_info_type type;
+} trailer_config_items[] = {
+	{ "key", TRAILER_KEY },
+	{ "command", TRAILER_COMMAND },
+	{ "where", TRAILER_WHERE },
+	{ "ifexists", TRAILER_IF_EXISTS },
+	{ "ifmissing", TRAILER_IF_MISSING }
+};
+
+static int git_trailer_config(const char *conf_key, const char *value, void *cb)
+{
+	const char *trailer_item, *variable_name;
+	struct trailer_item *item;
+	struct conf_info *conf;
+	char *name = NULL;
+	enum trailer_info_type type;
+	int i;
+
+	trailer_item = skip_prefix(conf_key, "trailer.");
+	if (!trailer_item)
+		return 0;
+
+	variable_name = strrchr(trailer_item, '.');
+	if (!variable_name) {
+		warning(_("two level trailer config variable %s"), conf_key);
+		return 0;
+	}
+
+	variable_name++;
+	for (i = 0; i < ARRAY_SIZE(trailer_config_items); i++) {
+		if (strcmp(trailer_config_items[i].name, variable_name))
+			continue;
+		name = xstrndup(trailer_item,  variable_name - trailer_item - 1);
+		type = trailer_config_items[i].type;
+		break;
+	}
+
+	if (!name)
+		return 0;
+
+	item = get_conf_item(name);
+	conf = &item->conf;
+	free(name);
+
+	switch (type) {
+	case TRAILER_KEY:
+		if (conf->key)
+			warning(_("more than one %s"), conf_key);
+		conf->key = xstrdup(value);
+		break;
+	case TRAILER_COMMAND:
+		if (conf->command)
+			warning(_("more than one %s"), conf_key);
+		conf->command = xstrdup(value);
+		break;
+	case TRAILER_WHERE:
+		if (set_where(conf, value))
+			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
+		break;
+	case TRAILER_IF_EXISTS:
+		if (set_if_exists(conf, value))
+			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
+		break;
+	case TRAILER_IF_MISSING:
+		if (set_if_missing(conf, value))
+			warning(_("unknown value '%s' for key '%s'"), value, conf_key);
+		break;
+	default:
+		die("internal bug in trailer.c");
+	}
+	return 0;
+}
-- 
1.9.0.163.g8ca203c

[PATCH v10 04/12] trailer: process command line trailer arguments

[Christian Couder]

Parse the trailer command line arguments and put
the result into an arg_tok doubly linked list.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 trailer.c | 117 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 117 insertions(+)

diff --git a/trailer.c b/trailer.c
index c7c0f54..89ebff1 100644
--- a/trailer.c
+++ b/trailer.c
@@ -391,3 +391,120 @@ static int git_trailer_config(const char *conf_key, const char *value, void *cb)
 	}
 	return 0;
 }
+
+static int parse_trailer(struct strbuf *tok, struct strbuf *val, const char *trailer)
+{
+	size_t len = strcspn(trailer, "=:");
+	if (len == 0)
+		return error(_("empty trailer token in trailer '%s'"), trailer);
+	if (len < strlen(trailer)) {
+		strbuf_add(tok, trailer, len);
+		strbuf_trim(tok);
+		strbuf_addstr(val, trailer + len + 1);
+		strbuf_trim(val);
+	} else {
+		strbuf_addstr(tok, trailer);
+		strbuf_trim(tok);
+	}
+	return 0;
+}
+
+
+static void duplicate_conf(struct conf_info *dst, struct conf_info *src)
+{
+	*dst = *src;
+	if (src->name)
+		dst->name = xstrdup(src->name);
+	if (src->key)
+		dst->key = xstrdup(src->key);
+	if (src->command)
+		dst->command = xstrdup(src->command);
+}
+
+static const char *token_from_item(struct trailer_item *item)
+{
+	if (item->conf.key)
+		return item->conf.key;
+
+	return item->conf.name;
+}
+
+static struct trailer_item *new_trailer_item(struct trailer_item *conf_item,
+					     char *tok, char *val)
+{
+	struct trailer_item *new = xcalloc(sizeof(*new), 1);
+	new->value = val;
+
+	if (conf_item) {
+		duplicate_conf(&new->conf, &conf_item->conf);
+		new->token = xstrdup(token_from_item(conf_item));
+		free(tok);
+	} else
+		new->token = tok;
+
+	return new;
+}
+
+static int token_matches_item(const char *tok, struct trailer_item *item, int alnum_len)
+{
+	if (!strncasecmp(tok, item->conf.name, alnum_len))
+		return 1;
+	return item->conf.key ? !strncasecmp(tok, item->conf.key, alnum_len) : 0;
+}
+
+static struct trailer_item *create_trailer_item(const char *string)
+{
+	struct strbuf tok = STRBUF_INIT;
+	struct strbuf val = STRBUF_INIT;
+	struct trailer_item *item;
+	int tok_alnum_len;
+
+	if (parse_trailer(&tok, &val, string))
+		return NULL;
+
+	tok_alnum_len = alnum_len(tok.buf, tok.len);
+
+	/* Lookup if the token matches something in the config */
+	for (item = first_conf_item; item; item = item->next) {
+		if (token_matches_item(tok.buf, item, tok_alnum_len)) {
+			strbuf_release(&tok);
+			return new_trailer_item(item,
+						NULL,
+						strbuf_detach(&val, NULL));
+		}
+	}
+
+	return new_trailer_item(NULL,
+				strbuf_detach(&tok, NULL),
+				strbuf_detach(&val, NULL));
+}
+
+static void add_trailer_item(struct trailer_item **first,
+			     struct trailer_item **last,
+			     struct trailer_item *new)
+{
+	if (!new)
+		return;
+	if (!*last) {
+		*first = new;
+		*last = new;
+	} else {
+		(*last)->next = new;
+		new->previous = *last;
+		*last = new;
+	}
+}
+
+static struct trailer_item *process_command_line_args(int argc, const char **argv)
+{
+	int i;
+	struct trailer_item *arg_tok_first = NULL;
+	struct trailer_item *arg_tok_last = NULL;
+
+	for (i = 0; i < argc; i++) {
+		struct trailer_item *new = create_trailer_item(argv[i]);
+		add_trailer_item(&arg_tok_first, &arg_tok_last, new);
+	}
+
+	return arg_tok_first;
+}
-- 
1.9.0.163.g8ca203c

[PATCH v10 05/12] trailer: parse trailers from stdin

[Christian Couder]

Read trailers from stdin, parse them and put the result into a doubly linked
list.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 trailer.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 76 insertions(+)

diff --git a/trailer.c b/trailer.c
index 89ebff1..6d2da32 100644
--- a/trailer.c
+++ b/trailer.c
@@ -50,6 +50,14 @@ static size_t alnum_len(const char *buf, size_t len)
 	return len;
 }
 
+static inline int contains_only_spaces(const char *str)
+{
+	const char *s = str;
+	while (*s && isspace(*s))
+		s++;
+	return !*s;
+}
+
 static void free_trailer_item(struct trailer_item *item)
 {
 	free(item->conf.name);
@@ -508,3 +516,71 @@ static struct trailer_item *process_command_line_args(int argc, const char **arg
 
 	return arg_tok_first;
 }
+
+static struct strbuf **read_stdin(void)
+{
+	struct strbuf **lines;
+	struct strbuf sb = STRBUF_INIT;
+
+	if (strbuf_read(&sb, fileno(stdin), 0) < 0)
+		die_errno(_("could not read from stdin"));
+
+	lines = strbuf_split(&sb, '\n');
+
+	strbuf_release(&sb);
+
+	return lines;
+}
+
+/*
+ * Return the the (0 based) index of the first trailer line
+ * or the line count if there are no trailers.
+ */
+static int find_trailer_start(struct strbuf **lines)
+{
+	int start, empty = 1, count = 0;
+
+	/* Get the line count */
+	while (lines[count])
+		count++;
+
+	/*
+	 * Get the start of the trailers by looking starting from the end
+	 * for a line with only spaces before lines with one ':'.
+	 */
+	for (start = count - 1; start >= 0; start--) {
+		if (contains_only_spaces(lines[start]->buf)) {
+			if (empty)
+				continue;
+			return start + 1;
+		}
+		if (strchr(lines[start]->buf, ':')) {
+			if (empty)
+				empty = 0;
+			continue;
+		}
+		return count;
+	}
+
+	return empty ? count : start + 1;
+}
+
+static void process_stdin(struct trailer_item **in_tok_first,
+			  struct trailer_item **in_tok_last)
+{
+	struct strbuf **lines = read_stdin();
+	int start = find_trailer_start(lines);
+	int i;
+
+	/* Print non trailer lines as is */
+	for (i = 0; lines[i] && i < start; i++)
+		printf("%s", lines[i]->buf);
+
+	/* Parse trailer lines */
+	for (i = start; lines[i]; i++) {
+		struct trailer_item *new = create_trailer_item(lines[i]->buf);
+		add_trailer_item(in_tok_first, in_tok_last, new);
+	}
+
+	strbuf_list_free(lines);
+}
-- 
1.9.0.163.g8ca203c

[PATCH v10 06/12] trailer: put all the processing together and print

[Christian Couder]

This patch adds the process_trailers() function that
calls all the previously added processing functions
and then prints the results on the standard output.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 trailer.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 trailer.h |  6 ++++++
 2 files changed, 55 insertions(+)
 create mode 100644 trailer.h

diff --git a/trailer.c b/trailer.c
index 6d2da32..16465e5 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "trailer.h"
 /*
  * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
  */
@@ -68,6 +69,26 @@ static void free_trailer_item(struct trailer_item *item)
 	free(item);
 }
 
+static void print_tok_val(const char *tok, const char *val)
+{
+	char c = tok[strlen(tok) - 1];
+	if (isalnum(c))
+		printf("%s: %s\n", tok, val);
+	else if (isspace(c) || c == '#')
+		printf("%s%s\n", tok, val);
+	else
+		printf("%s %s\n", tok, val);
+}
+
+static void print_all(struct trailer_item *first, int trim_empty)
+{
+	struct trailer_item *item;
+	for (item = first; item; item = item->next) {
+		if (!trim_empty || strlen(item->value) > 0)
+			print_tok_val(item->token, item->value);
+	}
+}
+
 static void add_arg_to_input_list(struct trailer_item *in_tok,
 				  struct trailer_item *arg_tok)
 {
@@ -584,3 +605,31 @@ static void process_stdin(struct trailer_item **in_tok_first,
 
 	strbuf_list_free(lines);
 }
+
+static void free_all(struct trailer_item **first)
+{
+	while (*first) {
+		struct trailer_item *item = remove_first(first);
+		free_trailer_item(item);
+	}
+}
+
+void process_trailers(int trim_empty, int argc, const char **argv)
+{
+	struct trailer_item *in_tok_first = NULL;
+	struct trailer_item *in_tok_last = NULL;
+	struct trailer_item *arg_tok_first;
+
+	git_config(git_trailer_config, NULL);
+
+	/* Print the non trailer part of stdin */
+	process_stdin(&in_tok_first, &in_tok_last);
+
+	arg_tok_first = process_command_line_args(argc, argv);
+
+	process_trailers_lists(&in_tok_first, &in_tok_last, &arg_tok_first);
+
+	print_all(in_tok_first, trim_empty);
+
+	free_all(&in_tok_first);
+}
diff --git a/trailer.h b/trailer.h
new file mode 100644
index 0000000..9323b1e
--- /dev/null
+++ b/trailer.h
@@ -0,0 +1,6 @@
+#ifndef TRAILER_H
+#define TRAILER_H
+
+void process_trailers(int trim_empty, int argc, const char **argv);
+
+#endif /* TRAILER_H */
-- 
1.9.0.163.g8ca203c

[PATCH v10 07/12] trailer: add interpret-trailers command

[Christian Couder]

This patch adds the "git interpret-trailers" command.
This command uses the previously added process_trailers()
function in trailer.c.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 .gitignore                   |  1 +
 Makefile                     |  1 +
 builtin.h                    |  1 +
 builtin/interpret-trailers.c | 33 +++++++++++++++++++++++++++++++++
 git.c                        |  1 +
 5 files changed, 37 insertions(+)
 create mode 100644 builtin/interpret-trailers.c

diff --git a/.gitignore b/.gitignore
index dc600f9..c2a0b19 100644
--- a/.gitignore
+++ b/.gitignore
@@ -74,6 +74,7 @@
 /git-index-pack
 /git-init
 /git-init-db
+/git-interpret-trailers
 /git-instaweb
 /git-log
 /git-ls-files
diff --git a/Makefile b/Makefile
index 179be0a..499ca30 100644
--- a/Makefile
+++ b/Makefile
@@ -944,6 +944,7 @@ BUILTIN_OBJS += builtin/hash-object.o
 BUILTIN_OBJS += builtin/help.o
 BUILTIN_OBJS += builtin/index-pack.o
 BUILTIN_OBJS += builtin/init-db.o
+BUILTIN_OBJS += builtin/interpret-trailers.o
 BUILTIN_OBJS += builtin/log.o
 BUILTIN_OBJS += builtin/ls-files.o
 BUILTIN_OBJS += builtin/ls-remote.o
diff --git a/builtin.h b/builtin.h
index c47c110..8ca0065 100644
--- a/builtin.h
+++ b/builtin.h
@@ -73,6 +73,7 @@ extern int cmd_hash_object(int argc, const char **argv, const char *prefix);
 extern int cmd_help(int argc, const char **argv, const char *prefix);
 extern int cmd_index_pack(int argc, const char **argv, const char *prefix);
 extern int cmd_init_db(int argc, const char **argv, const char *prefix);
+extern int cmd_interpret_trailers(int argc, const char **argv, const char *prefix);
 extern int cmd_log(int argc, const char **argv, const char *prefix);
 extern int cmd_log_reflog(int argc, const char **argv, const char *prefix);
 extern int cmd_ls_files(int argc, const char **argv, const char *prefix);
diff --git a/builtin/interpret-trailers.c b/builtin/interpret-trailers.c
new file mode 100644
index 0000000..0c8ca72
--- /dev/null
+++ b/builtin/interpret-trailers.c
@@ -0,0 +1,33 @@
+/*
+ * Builtin "git interpret-trailers"
+ *
+ * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
+ *
+ */
+
+#include "cache.h"
+#include "builtin.h"
+#include "parse-options.h"
+#include "trailer.h"
+
+static const char * const git_interpret_trailers_usage[] = {
+	N_("git interpret-trailers [--trim-empty] [(<token>[(=|:)<value>])...]"),
+	NULL
+};
+
+int cmd_interpret_trailers(int argc, const char **argv, const char *prefix)
+{
+	int trim_empty = 0;
+
+	struct option options[] = {
+		OPT_BOOL(0, "trim-empty", &trim_empty, N_("trim empty trailers")),
+		OPT_END()
+	};
+
+	argc = parse_options(argc, argv, prefix, options,
+			     git_interpret_trailers_usage, 0);
+
+	process_trailers(trim_empty, argc, argv);
+
+	return 0;
+}
diff --git a/git.c b/git.c
index 9efd1a3..d432f11 100644
--- a/git.c
+++ b/git.c
@@ -380,6 +380,7 @@ static struct cmd_struct commands[] = {
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
+	{ "interpret-trailers", cmd_interpret_trailers, RUN_SETUP },
 	{ "log", cmd_log, RUN_SETUP },
 	{ "ls-files", cmd_ls_files, RUN_SETUP },
 	{ "ls-remote", cmd_ls_remote, RUN_SETUP_GENTLY },
-- 
1.9.0.163.g8ca203c

[PATCH v10 08/12] trailer: add tests for "git interpret-trailers"

[Christian Couder]

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/t7513-interpret-trailers.sh | 351 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 351 insertions(+)
 create mode 100755 t/t7513-interpret-trailers.sh

diff --git a/t/t7513-interpret-trailers.sh b/t/t7513-interpret-trailers.sh
new file mode 100755
index 0000000..0e5d57f
--- /dev/null
+++ b/t/t7513-interpret-trailers.sh
@@ -0,0 +1,351 @@
+#!/bin/sh
+#
+# Copyright (c) 2013 Christian Couder
+#
+
+test_description='git interpret-trailers'
+
+. ./test-lib.sh
+
+# When we want one trailing space at the end of each line, let's use sed
+# to make sure that these spaces are not removed by any automatic tool.
+
+test_expect_success 'setup' '
+	cat >basic_message <<-\EOF &&
+		subject
+
+		body
+	EOF
+	cat >complex_message_body <<-\EOF &&
+		my subject
+
+		my body which is long
+		and contains some special
+		chars like : = ? !
+
+	EOF
+	sed -e "s/ Z\$/ /" >complex_message_trailers <<-\EOF
+		Fixes: Z
+		Acked-by: Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+'
+
+test_expect_success 'without config' '
+	sed -e "s/ Z\$/ /" >expected <<-\EOF &&
+		ack: Peff
+		Reviewed-by: Z
+		Acked-by: Johan
+	EOF
+	git interpret-trailers "ack = Peff" "Reviewed-by" "Acked-by: Johan" >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success '--trim-empty without config' '
+	cat >expected <<-\EOF &&
+		ack: Peff
+		Acked-by: Johan
+	EOF
+	git interpret-trailers --trim-empty "ack = Peff" \
+		"Reviewed-by" "Acked-by: Johan" "sob:" >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with config setup' '
+	git config trailer.ack.key "Acked-by: " &&
+	cat >expected <<-\EOF &&
+		Acked-by: Peff
+	EOF
+	git interpret-trailers --trim-empty "ack = Peff" >actual &&
+	test_cmp expected actual &&
+	git interpret-trailers --trim-empty "Acked-by = Peff" >actual &&
+	test_cmp expected actual &&
+	git interpret-trailers --trim-empty "Acked-by :Peff" >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with config setup and = sign' '
+	git config trailer.ack.key "Acked-by= " &&
+	cat >expected <<-\EOF &&
+		Acked-by= Peff
+	EOF
+	git interpret-trailers --trim-empty "ack = Peff" >actual &&
+	test_cmp expected actual &&
+	git interpret-trailers --trim-empty "Acked-by= Peff" >actual &&
+	test_cmp expected actual &&
+	git interpret-trailers --trim-empty "Acked-by : Peff" >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with config setup and # sign' '
+	git config trailer.bug.key "Bug #" &&
+	cat >expected <<-\EOF &&
+		Bug #42
+	EOF
+	git interpret-trailers --trim-empty "bug = 42" >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with commit basic message' '
+	git interpret-trailers <basic_message >actual &&
+	test_cmp basic_message actual
+'
+
+test_expect_success 'with commit complex message' '
+	cat complex_message_body complex_message_trailers >complex_message &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Fixes: Z
+		Acked-by= Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with commit complex message and args' '
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+		Bug #42
+	EOF
+	git interpret-trailers "ack: Peff" "bug: 42" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with commit complex message, args and --trim-empty' '
+	cat complex_message_body >expected &&
+	cat >>expected <<-\EOF &&
+		Acked-by= Peff
+		Bug #42
+	EOF
+	git interpret-trailers --trim-empty "ack: Peff" "bug: 42" \
+		<complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "where = before"' '
+	git config trailer.bug.where "before" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "ack: Peff" "bug: 42" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "where = before" for a token in the middle of the message' '
+	git config trailer.review.key "Reviewed-by:" &&
+	git config trailer.review.where "before" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Reviewed-by: Johan
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "ack: Peff" "bug: 42" "review: Johan" \
+		<complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "where = before" and --trim-empty' '
+	cat complex_message_body >expected &&
+	cat >>expected <<-\EOF &&
+		Bug #46
+		Bug #42
+		Acked-by= Peff
+		Reviewed-by: Johan
+	EOF
+	git interpret-trailers --trim-empty "ack: Peff" "bug: 42" \
+		"review: Johan" "Bug: 46" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'the default is "ifExists = addIfDifferent"' '
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "ack: Peff" "review:" "bug: 42" \
+		"ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifExists = addIfDifferent"' '
+	git config trailer.review.ifExists "addIfDifferent" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "ack: Peff" "review:" "bug: 42" \
+		"ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifExists = addIfDifferentNeighbor"' '
+	git config trailer.ack.ifExists "addIfDifferentNeighbor" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "ack: Peff" "review:" "ack: Junio" "bug: 42" \
+		"ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifExists = addIfDifferentNeighbor" and --trim-empty' '
+	git config trailer.ack.ifExists "addIfDifferentNeighbor" &&
+	cat complex_message_body >expected &&
+	cat >>expected <<-\EOF &&
+		Bug #42
+		Acked-by= Peff
+		Acked-by= Junio
+		Acked-by= Peff
+	EOF
+	git interpret-trailers --trim-empty "ack: Peff" "Acked-by= Peff" \
+		"review:" "ack: Junio" "bug: 42" "ack: Peff" \
+		<complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifExists = add"' '
+	git config trailer.ack.ifExists "add" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Peff
+		Acked-by= Peff
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "ack: Peff" "Acked-by= Peff" "review:" \
+		"ack: Junio" "bug: 42" "ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifExists = overwrite"' '
+	git config trailer.fix.key "Fixes:" &&
+	git config trailer.fix.ifExists "overwrite" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: 22
+		Acked-by= Z
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "review:" "fix=53" "ack: Junio" "fix=22" \
+		"bug: 42" "ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifExists = doNothing"' '
+	git config trailer.fix.ifExists "doNothing" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "review:" "fix=53" "ack: Junio" "fix=22" \
+		"bug: 42" "ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'the default is "ifMissing = add"' '
+	git config trailer.cc.key "Cc: " &&
+	git config trailer.cc.where "before" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Cc: Linus
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "review:" "fix=53" "cc=Linus" "ack: Junio" \
+		"fix=22" "bug: 42" "ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifMissing = add"' '
+	git config trailer.cc.ifMissing "add" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Cc: Linus
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "review:" "fix=53" "ack: Junio" "fix=22" \
+		"bug: 42" "cc=Linus" "ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'using "ifMissing = doNothing"' '
+	git config trailer.cc.ifMissing "doNothing" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Bug #42
+		Fixes: Z
+		Acked-by= Z
+		Acked-by= Junio
+		Acked-by= Peff
+		Reviewed-by: Z
+		Signed-off-by: Z
+	EOF
+	git interpret-trailers "review:" "fix=53" "cc=Linus" "ack: Junio" \
+		"fix=22" "bug: 42" "ack: Peff" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_done
-- 
1.9.0.163.g8ca203c

[PATCH v10 09/12] trailer: execute command from 'trailer.<name>.command'

[Christian Couder]

Let the user specify a command that will give on its standard output
the value to use for the specified trailer.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 trailer.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/trailer.c b/trailer.c
index 16465e5..09db2c2 100644
--- a/trailer.c
+++ b/trailer.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "run-command.h"
 #include "trailer.h"
 /*
  * Copyright (c) 2013, 2014 Christian Couder <chriscool@tuxfamily.org>
@@ -13,11 +14,14 @@ struct conf_info {
 	char *name;
 	char *key;
 	char *command;
+	unsigned command_uses_arg : 1;
 	enum action_where where;
 	enum action_if_exists if_exists;
 	enum action_if_missing if_missing;
 };
 
+#define TRAILER_ARG_STRING "$ARG"
+
 struct trailer_item {
 	struct trailer_item *previous;
 	struct trailer_item *next;
@@ -59,6 +63,13 @@ static inline int contains_only_spaces(const char *str)
 	return !*s;
 }
 
+static inline void strbuf_replace(struct strbuf *sb, const char *a, const char *b)
+{
+	const char *ptr = strstr(sb->buf, a);
+	if (ptr)
+		strbuf_splice(sb, ptr - sb->buf, strlen(a), b, strlen(b));
+}
+
 static void free_trailer_item(struct trailer_item *item)
 {
 	free(item->conf.name);
@@ -402,6 +413,7 @@ static int git_trailer_config(const char *conf_key, const char *value, void *cb)
 		if (conf->command)
 			warning(_("more than one %s"), conf_key);
 		conf->command = xstrdup(value);
+		conf->command_uses_arg = !!strstr(conf->command, TRAILER_ARG_STRING);
 		break;
 	case TRAILER_WHERE:
 		if (set_where(conf, value))
@@ -438,6 +450,45 @@ static int parse_trailer(struct strbuf *tok, struct strbuf *val, const char *tra
 	return 0;
 }
 
+static int read_from_command(struct child_process *cp, struct strbuf *buf)
+{
+	if (run_command(cp))
+		return error("running trailer command '%s' failed", cp->argv[0]);
+	if (strbuf_read(buf, cp->out, 1024) < 1)
+		return error("reading from trailer command '%s' failed", cp->argv[0]);
+	strbuf_trim(buf);
+	return 0;
+}
+
+static const char *apply_command(const char *command, const char *arg)
+{
+	struct strbuf cmd = STRBUF_INIT;
+	struct strbuf buf = STRBUF_INIT;
+	struct child_process cp;
+	const char *argv[] = {NULL, NULL};
+	const char *result;
+
+	strbuf_addstr(&cmd, command);
+	if (arg)
+		strbuf_replace(&cmd, TRAILER_ARG_STRING, arg);
+
+	argv[0] = cmd.buf;
+	memset(&cp, 0, sizeof(cp));
+	cp.argv = argv;
+	cp.env = local_repo_env;
+	cp.no_stdin = 1;
+	cp.out = -1;
+	cp.use_shell = 1;
+
+	if (read_from_command(&cp, &buf)) {
+		strbuf_release(&buf);
+		result = xstrdup("");
+	} else
+		result = strbuf_detach(&buf, NULL);
+
+	strbuf_release(&cmd);
+	return result;
+}
 
 static void duplicate_conf(struct conf_info *dst, struct conf_info *src)
 {
@@ -468,6 +519,10 @@ static struct trailer_item *new_trailer_item(struct trailer_item *conf_item,
 		duplicate_conf(&new->conf, &conf_item->conf);
 		new->token = xstrdup(token_from_item(conf_item));
 		free(tok);
+		if (conf_item->conf.command_uses_arg || !val) {
+			new->value = apply_command(conf_item->conf.command, val);
+			free(val);
+		}
 	} else
 		new->token = tok;
 
@@ -529,12 +584,21 @@ static struct trailer_item *process_command_line_args(int argc, const char **arg
 	int i;
 	struct trailer_item *arg_tok_first = NULL;
 	struct trailer_item *arg_tok_last = NULL;
+	struct trailer_item *item;
 
 	for (i = 0; i < argc; i++) {
 		struct trailer_item *new = create_trailer_item(argv[i]);
 		add_trailer_item(&arg_tok_first, &arg_tok_last, new);
 	}
 
+	/* Add conf commands that don't use $ARG */
+	for (item = first_conf_item; item; item = item->next) {
+		if (item->conf.command && !item->conf.command_uses_arg) {
+			struct trailer_item *new = new_trailer_item(item, NULL, NULL);
+			add_trailer_item(&arg_tok_first, &arg_tok_last, new);
+		}
+	}
+
 	return arg_tok_first;
 }
 
-- 
1.9.0.163.g8ca203c

[PATCH v10 10/12] trailer: add tests for commands in config file

[Christian Couder]

And add a few other tests for some special cases.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/t7513-interpret-trailers.sh | 116 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 116 insertions(+)

diff --git a/t/t7513-interpret-trailers.sh b/t/t7513-interpret-trailers.sh
index 0e5d57f..262f7bf 100755
--- a/t/t7513-interpret-trailers.sh
+++ b/t/t7513-interpret-trailers.sh
@@ -348,4 +348,120 @@ test_expect_success 'using "ifMissing = doNothing"' '
 	test_cmp expected actual
 '
 
+test_expect_success 'with simple command' '
+	git config trailer.sign.key "Signed-off-by: " &&
+	git config trailer.sign.where "after" &&
+	git config trailer.sign.ifExists "addIfDifferentNeighbor" &&
+	git config trailer.sign.command "echo \"A U Thor <author@example.com>\"" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Fixes: Z
+		Acked-by= Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+		Signed-off-by: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers "review:" "fix=22" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with command using commiter information' '
+	git config trailer.sign.ifExists "addIfDifferent" &&
+	git config trailer.sign.command "echo \"\$GIT_COMMITTER_NAME <\$GIT_COMMITTER_EMAIL>\"" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Fixes: Z
+		Acked-by= Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+		Signed-off-by: C O Mitter <committer@example.com>
+	EOF
+	git interpret-trailers "review:" "fix=22" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with command using author information' '
+	git config trailer.sign.key "Signed-off-by: " &&
+	git config trailer.sign.where "after" &&
+	git config trailer.sign.ifExists "addIfDifferentNeighbor" &&
+	git config trailer.sign.command "echo \"\$GIT_AUTHOR_NAME <\$GIT_AUTHOR_EMAIL>\"" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-\EOF &&
+		Fixes: Z
+		Acked-by= Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+		Signed-off-by: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers "review:" "fix=22" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'setup a commit' '
+	echo "Content of the first commit." > a.txt &&
+	git add a.txt &&
+	git commit -m "Add file a.txt"
+'
+
+test_expect_success 'with command using $ARG' '
+	git config trailer.fix.ifExists "overwrite" &&
+	git config trailer.fix.command "git log -1 --oneline --format=\"%h (%s)\" --abbrev-commit --abbrev=14 \$ARG" &&
+	FIXED=$(git log -1 --oneline --format="%h (%s)" --abbrev-commit --abbrev=14 HEAD) &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-EOF &&
+		Fixes: $FIXED
+		Acked-by= Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+		Signed-off-by: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers "review:" "fix=HEAD" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with failing command using $ARG' '
+	git config trailer.fix.ifExists "overwrite" &&
+	git config trailer.fix.command "false \$ARG" &&
+	cat complex_message_body >expected &&
+	sed -e "s/ Z\$/ /" >>expected <<-EOF &&
+		Fixes: Z
+		Acked-by= Z
+		Reviewed-by: Z
+		Signed-off-by: Z
+		Signed-off-by: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers "review:" "fix=HEAD" <complex_message >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'with empty tokens' '
+	cat >expected <<-EOF &&
+		Signed-off-by: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers ":" ":test" >actual <<-EOF &&
+	EOF
+	test_cmp expected actual
+'
+
+test_expect_success 'with command but no key' '
+	git config --unset trailer.sign.key &&
+	cat >expected <<-EOF &&
+		sign: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers >actual <<-EOF &&
+	EOF
+	test_cmp expected actual
+'
+
+test_expect_success 'with no command and no key' '
+	git config --unset trailer.review.key &&
+	cat >expected <<-EOF &&
+		review: Junio
+		sign: A U Thor <author@example.com>
+	EOF
+	git interpret-trailers "review:Junio" >actual <<-EOF &&
+	EOF
+	test_cmp expected actual
+'
+
 test_done
-- 
1.9.0.163.g8ca203c

[PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Christian Couder]

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-interpret-trailers.txt | 123 +++++++++++++++++++++++++++++++
 1 file changed, 123 insertions(+)
 create mode 100644 Documentation/git-interpret-trailers.txt

diff --git a/Documentation/git-interpret-trailers.txt b/Documentation/git-interpret-trailers.txt
new file mode 100644
index 0000000..75ae386
--- /dev/null
+++ b/Documentation/git-interpret-trailers.txt
@@ -0,0 +1,123 @@
+git-interpret-trailers(1)
+=========================
+
+NAME
+----
+git-interpret-trailers - help add stuctured information into commit messages
+
+SYNOPSIS
+--------
+[verse]
+'git interpret-trailers' [--trim-empty] [(<token>[(=|:)<value>])...]
+
+DESCRIPTION
+-----------
+Help add RFC 822-like headers, called 'trailers', at the end of the
+otherwise free-form part of a commit message.
+
+This command is a filter. It reads the standard input for a commit
+message and applies the `token` arguments, if any, to this
+message. The resulting message is emited on the standard output.
+
+Some configuration variables control the way the `token` arguments are
+applied to the message and the way any existing trailer in the message
+is changed. They also make it possible to automatically add some
+trailers.
+
+By default, a 'token=value' or 'token:value' argument will be added
+only if no trailer with the same (token, value) pair is already in the
+message. The 'token' and 'value' parts will be trimmed to remove
+starting and trailing whitespace, and the resulting trimmed 'token'
+and 'value' will appear in the message like this:
+
+------------------------------------------------
+token: value
+------------------------------------------------
+
+By default, if there are already trailers with the same 'token', the
+new trailer will appear just after the last trailer with the same
+'token'. Otherwise it will appear at the end of the message.
+
+Note that 'trailers' do not follow and are not intended to follow many
+rules that are in RFC 822. For example they do not follow the line
+breaking rules, the encoding rules and probably many other rules.
+
+OPTIONS
+-------
+--trim-empty::
+	If the 'value' part of any trailer contains only whitespace,
+	the whole trailer will be removed from the resulting message.
+
+CONFIGURATION VARIABLES
+-----------------------
+
+trailer.<token>.key::
+	This 'key' will be used instead of 'token' in the
+	trailer. After some alphanumeric characters, it can contain
+	some non alphanumeric characters like ':', '=' or '#' that will
+	be used instead of ':' to separate the token from the value in
+	the trailer, though the default ':' is more standard.
+
+trailer.<token>.where::
+	This can be either `after`, which is the default, or
+	`before`. If it is `before`, then a trailer with the specified
+	token, will appear before, instead of after, other trailers
+	with the same token, or otherwise at the beginning, instead of
+	at the end, of all the trailers.
+
+trailer.<token>.ifexist::
+	This option makes it possible to choose what action will be
+	performed when there is already at least one trailer with the
+	same token in the message.
++
+The valid values for this option are: `addIfDifferent` (this is the
+default), `addIfDifferentNeighbor`, `add`, `overwrite` or `doNothing`.
++
+With `addIfDifferent`, a new trailer will be added only if no trailer
+with the same (token, value) pair is already in the message.
++
+With `addIfDifferentNeighbor`, a new trailer will be added only if no
+trailer with the same (token, value) pair is above or below the line
+where the new trailer will be added.
++
+With `add`, a new trailer will be added, even if some trailers with
+the same (token, value) pair are already in the message.
++
+With `overwrite`, the new trailer will overwrite an existing trailer
+with the same token.
++
+With `doNothing`, nothing will be done, that is no new trailer will be
+added if there is already one with the same token in the message.
+
+trailer.<token>.ifmissing::
+	This option makes it possible to choose what action will be
+	performed when there is not yet any trailer with the same
+	token in the message.
++
+The valid values for this option are: `add` (this is the default) and
+`doNothing`.
++
+With `add`, a new trailer will be added.
++
+With `doNothing`, nothing will be done.
+
+trailer.<token>.command::
+	This option can be used to specify a shell command that will
+	be used to automatically add or modify a trailer with the
+	specified 'token'.
++
+When this option is specified, it is like if a special 'token=value'
+argument is added at the end of the command line, where 'value' will
+be given by the standard output of the specified command.
++
+If the command contains the `$ARG` string, this string will be
+replaced with the 'value' part of an existing trailer with the same
+token, if any, before the command is launched.
+
+SEE ALSO
+--------
+linkgit:git-commit[1]
+
+GIT
+---
+Part of the linkgit:git[1] suite
-- 
1.9.0.163.g8ca203c

[PATCH v10 12/12] trailer: add blank line before the trailers if needed

[Christian Couder]

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 t/t7513-interpret-trailers.sh | 12 +++++++++++-
 trailer.c                     | 26 ++++++++++++++++++--------
 2 files changed, 29 insertions(+), 9 deletions(-)

diff --git a/t/t7513-interpret-trailers.sh b/t/t7513-interpret-trailers.sh
index 262f7bf..44a7131 100755
--- a/t/t7513-interpret-trailers.sh
+++ b/t/t7513-interpret-trailers.sh
@@ -34,6 +34,7 @@ test_expect_success 'setup' '
 
 test_expect_success 'without config' '
 	sed -e "s/ Z\$/ /" >expected <<-\EOF &&
+
 		ack: Peff
 		Reviewed-by: Z
 		Acked-by: Johan
@@ -44,6 +45,7 @@ test_expect_success 'without config' '
 
 test_expect_success '--trim-empty without config' '
 	cat >expected <<-\EOF &&
+
 		ack: Peff
 		Acked-by: Johan
 	EOF
@@ -55,6 +57,7 @@ test_expect_success '--trim-empty without config' '
 test_expect_success 'with config setup' '
 	git config trailer.ack.key "Acked-by: " &&
 	cat >expected <<-\EOF &&
+
 		Acked-by: Peff
 	EOF
 	git interpret-trailers --trim-empty "ack = Peff" >actual &&
@@ -68,6 +71,7 @@ test_expect_success 'with config setup' '
 test_expect_success 'with config setup and = sign' '
 	git config trailer.ack.key "Acked-by= " &&
 	cat >expected <<-\EOF &&
+
 		Acked-by= Peff
 	EOF
 	git interpret-trailers --trim-empty "ack = Peff" >actual &&
@@ -81,6 +85,7 @@ test_expect_success 'with config setup and = sign' '
 test_expect_success 'with config setup and # sign' '
 	git config trailer.bug.key "Bug #" &&
 	cat >expected <<-\EOF &&
+
 		Bug #42
 	EOF
 	git interpret-trailers --trim-empty "bug = 42" >actual &&
@@ -88,8 +93,10 @@ test_expect_success 'with config setup and # sign' '
 '
 
 test_expect_success 'with commit basic message' '
+	cat basic_message >expected &&
+	echo >>expected &&
 	git interpret-trailers <basic_message >actual &&
-	test_cmp basic_message actual
+	test_cmp expected actual
 '
 
 test_expect_success 'with commit complex message' '
@@ -436,6 +443,7 @@ test_expect_success 'with failing command using $ARG' '
 
 test_expect_success 'with empty tokens' '
 	cat >expected <<-EOF &&
+
 		Signed-off-by: A U Thor <author@example.com>
 	EOF
 	git interpret-trailers ":" ":test" >actual <<-EOF &&
@@ -446,6 +454,7 @@ test_expect_success 'with empty tokens' '
 test_expect_success 'with command but no key' '
 	git config --unset trailer.sign.key &&
 	cat >expected <<-EOF &&
+
 		sign: A U Thor <author@example.com>
 	EOF
 	git interpret-trailers >actual <<-EOF &&
@@ -456,6 +465,7 @@ test_expect_success 'with command but no key' '
 test_expect_success 'with no command and no key' '
 	git config --unset trailer.review.key &&
 	cat >expected <<-EOF &&
+
 		review: Junio
 		sign: A U Thor <author@example.com>
 	EOF
diff --git a/trailer.c b/trailer.c
index 09db2c2..639f657 100644
--- a/trailer.c
+++ b/trailer.c
@@ -618,12 +618,14 @@ static struct strbuf **read_stdin(void)
 }
 
 /*
- * Return the the (0 based) index of the first trailer line
+ * Return the (0 based) index of the first trailer line
  * or the line count if there are no trailers.
+ * The has_blank_line parameter tells if there is a blank
+ * line before the trailers.
  */
-static int find_trailer_start(struct strbuf **lines)
+static int find_trailer_start(struct strbuf **lines, int *has_blank_line)
 {
-	int start, empty = 1, count = 0;
+	int start, only_spaces = 1, count = 0;
 
 	/* Get the line count */
 	while (lines[count])
@@ -635,32 +637,40 @@ static int find_trailer_start(struct strbuf **lines)
 	 */
 	for (start = count - 1; start >= 0; start--) {
 		if (contains_only_spaces(lines[start]->buf)) {
-			if (empty)
+			if (only_spaces)
 				continue;
+			*has_blank_line = 1;
 			return start + 1;
 		}
 		if (strchr(lines[start]->buf, ':')) {
-			if (empty)
-				empty = 0;
+			if (only_spaces)
+				only_spaces = 0;
 			continue;
 		}
+		*has_blank_line = start == count - 1 ?
+		  0 : contains_only_spaces(lines[start + 1]->buf);
 		return count;
 	}
 
-	return empty ? count : start + 1;
+	*has_blank_line = only_spaces ? count > 0 : 0;
+	return only_spaces ? count : start + 1;
 }
 
 static void process_stdin(struct trailer_item **in_tok_first,
 			  struct trailer_item **in_tok_last)
 {
 	struct strbuf **lines = read_stdin();
-	int start = find_trailer_start(lines);
+	int has_blank_line;
+	int start = find_trailer_start(lines, &has_blank_line);
 	int i;
 
 	/* Print non trailer lines as is */
 	for (i = 0; lines[i] && i < start; i++)
 		printf("%s", lines[i]->buf);
 
+	if (!has_blank_line)
+		printf("\n");
+
 	/* Parse trailer lines */
 	for (i = start; lines[i]; i++) {
 		struct trailer_item *new = create_trailer_item(lines[i]->buf);
-- 
1.9.0.163.g8ca203c

Re: [PATCH v10 12/12] trailer: add blank line before the trailers if needed

[Junio C Hamano]

Christian Couder <chriscool@tuxfamily.org> writes:

> Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
> ---

Hmph, this is more fixing a mistake made earlier in the series at
the end than adding a new feature or something.  Can you start from
a version that does not have the mistake from the beginning?

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Michael Haggerty]

Sorry for reappearing in this thread after such a long absence.  I
wanted to see what is coming up (I think this interpret-trailers command
will be handy!) so I read this documentation patch carefully, and added
some questions and suggestions below.

On 04/06/2014 07:02 PM, Christian Couder wrote:
> Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  Documentation/git-interpret-trailers.txt | 123 +++++++++++++++++++++++++++++++
>  1 file changed, 123 insertions(+)
>  create mode 100644 Documentation/git-interpret-trailers.txt
> 
> diff --git a/Documentation/git-interpret-trailers.txt b/Documentation/git-interpret-trailers.txt
> new file mode 100644
> index 0000000..75ae386
> --- /dev/null
> +++ b/Documentation/git-interpret-trailers.txt
> @@ -0,0 +1,123 @@
> +git-interpret-trailers(1)
> +=========================
> +
> +NAME
> +----
> +git-interpret-trailers - help add stuctured information into commit messages
> +
> +SYNOPSIS
> +--------
> +[verse]
> +'git interpret-trailers' [--trim-empty] [(<token>[(=|:)<value>])...]
> +
> +DESCRIPTION
> +-----------
> +Help add RFC 822-like headers, called 'trailers', at the end of the
> +otherwise free-form part of a commit message.
> +
> +This command is a filter. It reads the standard input for a commit
> +message and applies the `token` arguments, if any, to this
> +message. The resulting message is emited on the standard output.

s/emited/emitted/

> +
> +Some configuration variables control the way the `token` arguments are
> +applied to the message and the way any existing trailer in the message
> +is changed. They also make it possible to automatically add some
> +trailers.
> +
> +By default, a 'token=value' or 'token:value' argument will be added
> +only if no trailer with the same (token, value) pair is already in the
> +message. The 'token' and 'value' parts will be trimmed to remove
> +starting and trailing whitespace, and the resulting trimmed 'token'
> +and 'value' will appear in the message like this:
> +
> +------------------------------------------------
> +token: value
> +------------------------------------------------
> +
> +By default, if there are already trailers with the same 'token', the
> +new trailer will appear just after the last trailer with the same
> +'token'. Otherwise it will appear at the end of the message.

How are existing trailers recognized in the input commit message?  Do
trailers have to be configured to be recognized?  Or are all lines
matching a specific pattern considered trailers?  If so, it might be
helpful to include a regexp here that describes the trailer "syntax".

What about blank lines?  I see that you try to add a blank line before
new trailers.  But what about on input?  Do the trailer lines have to be
separated from the free-form comment by a blank line to be recognized?
What if there are blank lines between trailer lines, or after them?  Is
it allowed to have non-trailer lines between or after trailer lines?

> +
> +Note that 'trailers' do not follow and are not intended to follow many
> +rules that are in RFC 822. For example they do not follow the line
> +breaking rules, the encoding rules and probably many other rules.
> +
> +OPTIONS
> +-------
> +--trim-empty::
> +	If the 'value' part of any trailer contains only whitespace,
> +	the whole trailer will be removed from the resulting message.

Does this apply to existing trailers, new trailers, or both?  If it
applies to existing trailers, then it seems a bit dangerous, in the
sense that the command might end up changing trailers that are unrelated
to the one that the command is trying to add.

> +
> +CONFIGURATION VARIABLES
> +-----------------------
> +
> +trailer.<token>.key::
> +	This 'key' will be used instead of 'token' in the
> +	trailer. After some alphanumeric characters, it can contain

Trailer keys can also contain '-', right?

> +	some non alphanumeric characters like ':', '=' or '#' that will
> +	be used instead of ':' to separate the token from the value in
> +	the trailer, though the default ':' is more standard.

Above it looks like the default separator is not ':' but rather ': '
(with a space).  Is the space always added regardless of the value of
this configuration variable, or should the configuration value include
the trailing space if it is desired?  Is there any way to get a trailer
that doesn't include a space, like

    foo=bar

?  (Changing this to "foo= bar" would look pretty ugly.)

If a commit message containing trailer lines with separators other than
':' is input to the program, will it recognize them as trailer lines?
Do such trailer lines have to have the same separator as the one listed
in this configuration setting to be recognized?

I suppose that there is some compelling reason to allow non-colon
separators here.  If not, it seems like it adds a lot of complexity and
should maybe be omitted, or limited to only a few specific separators.

> +
> +trailer.<token>.where::
> +	This can be either `after`, which is the default, or
> +	`before`. If it is `before`, then a trailer with the specified
> +	token, will appear before, instead of after, other trailers
> +	with the same token, or otherwise at the beginning, instead of
> +	at the end, of all the trailers.

Brainstorming: some other options that might make sense here someday:

`end`: add new trailer after all existing trailers (even those with
different keys).  This would allow trailers to be kept in chronological
order.

`beginning`: add new trailer before the first existing trailer (allows
reverse chronological order).

`sorted`: add new trailer among the existing trailers with the same key
so as to keep their values in lexicographic order.

> +
> +trailer.<token>.ifexist::
> +	This option makes it possible to choose what action will be
> +	performed when there is already at least one trailer with the
> +	same token in the message.
> ++
> +The valid values for this option are: `addIfDifferent` (this is the
> +default), `addIfDifferentNeighbor`, `add`, `overwrite` or `doNothing`.

Are these option values case sensitive?  If so, it might be a little bit
confusing because the same camel-case is often used in documentation for
configuration *keys*, which are not case sensitive [1], and users might
have gotten used to thinking of strings that look like this to be
non-case-sensitive.

> ++
> +With `addIfDifferent`, a new trailer will be added only if no trailer
> +with the same (token, value) pair is already in the message.
> ++
> +With `addIfDifferentNeighbor`, a new trailer will be added only if no
> +trailer with the same (token, value) pair is above or below the line
> +where the new trailer will be added.
> ++
> +With `add`, a new trailer will be added, even if some trailers with
> +the same (token, value) pair are already in the message.
> ++
> +With `overwrite`, the new trailer will overwrite an existing trailer
> +with the same token.

What if there are multiple existing trailers with the same token?  Are
they all overwritten?

> ++
> +With `doNothing`, nothing will be done, that is no new trailer will be
> +added if there is already one with the same token in the message.
> +
> +trailer.<token>.ifmissing::
> +	This option makes it possible to choose what action will be
> +	performed when there is not yet any trailer with the same
> +	token in the message.
> ++
> +The valid values for this option are: `add` (this is the default) and
> +`doNothing`.
> ++
> +With `add`, a new trailer will be added.
> ++
> +With `doNothing`, nothing will be done.
> +
> +trailer.<token>.command::
> +	This option can be used to specify a shell command that will
> +	be used to automatically add or modify a trailer with the
> +	specified 'token'.
> ++
> +When this option is specified, it is like if a special 'token=value'
> +argument is added at the end of the command line, where 'value' will
> +be given by the standard output of the specified command.

Maybe reword to

    When this option is specified, the behavior is as if a special
    'token=value' argument were added at the end of the command line,
    where 'value' is taken to be the standard output of the specified
    command.

And if it is the case, maybe add "with leading and trailing whitespace
trimmed off" at the end of the sentence.

> ++
> +If the command contains the `$ARG` string, this string will be
> +replaced with the 'value' part of an existing trailer with the same
> +token, if any, before the command is launched.

What if the key appears multiple times in existing trailers?

> +
> +SEE ALSO
> +--------
> +linkgit:git-commit[1]
> +
> +GIT
> +---
> +Part of the linkgit:git[1] suite
> 

Doesn't this command have to be added to command-list.txt?

Michael

[1] Anti-nitpick declaration: yes, I know that the middle part of
configuration keys is case-sensitive.

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Christian Couder]

On Tue, Apr 8, 2014 at 9:30 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
>
>> +This command is a filter. It reads the standard input for a commit
>> +message and applies the `token` arguments, if any, to this
>> +message. The resulting message is emited on the standard output.
>
> s/emited/emitted/

Ok.

>> +Some configuration variables control the way the `token` arguments are
>> +applied to the message and the way any existing trailer in the message
>> +is changed. They also make it possible to automatically add some
>> +trailers.
>> +
>> +By default, a 'token=value' or 'token:value' argument will be added
>> +only if no trailer with the same (token, value) pair is already in the
>> +message. The 'token' and 'value' parts will be trimmed to remove
>> +starting and trailing whitespace, and the resulting trimmed 'token'
>> +and 'value' will appear in the message like this:
>> +
>> +------------------------------------------------
>> +token: value
>> +------------------------------------------------
>> +
>> +By default, if there are already trailers with the same 'token', the
>> +new trailer will appear just after the last trailer with the same
>> +'token'. Otherwise it will appear at the end of the message.
>
> How are existing trailers recognized in the input commit message?  Do
> trailers have to be configured to be recognized?  Or are all lines
> matching a specific pattern considered trailers?  If so, it might be
> helpful to include a regexp here that describes the trailer "syntax".

The trailers are recognized in the input commit message using the
following rules:
 - only lines that contains a ':' are considered trailers,
 - the trailer lines must all be next to each other,
 - after them it's only possible to have some lines that contain only spaces,
 - before them there must be at least one line with only spaces

> What about blank lines?  I see that you try to add a blank line before
> new trailers.  But what about on input?

One line with only spaces has to be before the trailers. Some can be
after the trailers.

> Do the trailer lines have to be
> separated from the free-form comment by a blank line to be recognized?

Yes.

> What if there are blank lines between trailer lines, or after them?

After them is ok. Between is not ok (only the trailers after the blank
lines will be recognized).

> Is it allowed to have non-trailer lines between or after trailer lines?

No except lines with spaces after the trailers lines.

>> +Note that 'trailers' do not follow and are not intended to follow many
>> +rules that are in RFC 822. For example they do not follow the line
>> +breaking rules, the encoding rules and probably many other rules.
>> +
>> +OPTIONS
>> +-------
>> +--trim-empty::
>> +     If the 'value' part of any trailer contains only whitespace,
>> +     the whole trailer will be removed from the resulting message.
>
> Does this apply to existing trailers, new trailers, or both?

Both.

> If it applies to existing trailers, then it seems a bit dangerous, in the
> sense that the command might end up changing trailers that are unrelated
> to the one that the command is trying to add.

The command is not just for adding trailers.
But there could be an option to just trim trailers that are added.

>> +CONFIGURATION VARIABLES
>> +-----------------------
>> +
>> +trailer.<token>.key::
>> +     This 'key' will be used instead of 'token' in the
>> +     trailer. After some alphanumeric characters, it can contain
>
> Trailer keys can also contain '-', right?

Yes.
I should have written "after the last alphanumeric character".
I will fix that.

>> +     some non alphanumeric characters like ':', '=' or '#' that will
>> +     be used instead of ':' to separate the token from the value in
>> +     the trailer, though the default ':' is more standard.
>
> Above it looks like the default separator is not ':' but rather ': '
> (with a space).  Is the space always added regardless of the value of
> this configuration variable, or should the configuration value include
> the trailing space if it is desired?  Is there any way to get a trailer
> that doesn't include a space, like
>
>     foo=bar
>
> ?  (Changing this to "foo= bar" would look pretty ugly.)

I will have a look, but I think that:

- a space is always added after ':' or '=',
- a space is never added after '#',
- it doesn't matter if there is a space or not in the configured key.

> If a commit message containing trailer lines with separators other than
> ':' is input to the program, will it recognize them as trailer lines?

No, '=' and '#' are not supported in the input message, only in the output.

> Do such trailer lines have to have the same separator as the one listed
> in this configuration setting to be recognized?

No they need to have ':' as a separator.

The reason why only ':' is supported is because it is the cannonical
trailer separator and it could create problems with many input
messages if other separators where supported.

Maybe we could detect a special line like the following:

# TRAILERS START

in the input message and consider everyhting after that line as trailers.
In this case it would be ok to accept other separators.

> I suppose that there is some compelling reason to allow non-colon
> separators here.  If not, it seems like it adds a lot of complexity and
> should maybe be omitted, or limited to only a few specific separators.

Yeah, but in the early threads concerning this subject, someone said
that GitHub for example uses "bug #XXX".
I will have a look again.

>> +trailer.<token>.where::
>> +     This can be either `after`, which is the default, or
>> +     `before`. If it is `before`, then a trailer with the specified
>> +     token, will appear before, instead of after, other trailers
>> +     with the same token, or otherwise at the beginning, instead of
>> +     at the end, of all the trailers.
>
> Brainstorming: some other options that might make sense here someday:
>
> `end`: add new trailer after all existing trailers (even those with
> different keys).  This would allow trailers to be kept in chronological
> order.
>
> `beginning`: add new trailer before the first existing trailer (allows
> reverse chronological order).
>
> `sorted`: add new trailer among the existing trailers with the same key
> so as to keep their values in lexicographic order.

Yeah, I thought about these, but I don't think there is a need for
them right now.

>> +trailer.<token>.ifexist::
>> +     This option makes it possible to choose what action will be
>> +     performed when there is already at least one trailer with the
>> +     same token in the message.
>> ++
>> +The valid values for this option are: `addIfDifferent` (this is the
>> +default), `addIfDifferentNeighbor`, `add`, `overwrite` or `doNothing`.
>
> Are these option values case sensitive?  If so, it might be a little bit
> confusing because the same camel-case is often used in documentation for
> configuration *keys*, which are not case sensitive [1], and users might
> have gotten used to thinking of strings that look like this to be
> non-case-sensitive.

There were some discussions a few versions of this series ago with
Peff, Junio and perhaps others about this.
I thought that being case insensitive was better and Peff kind of
agreed with that, but as Junio disagreed it is now case sensitive.

>> +With `addIfDifferent`, a new trailer will be added only if no trailer
>> +with the same (token, value) pair is already in the message.
>> ++
>> +With `addIfDifferentNeighbor`, a new trailer will be added only if no
>> +trailer with the same (token, value) pair is above or below the line
>> +where the new trailer will be added.
>> ++
>> +With `add`, a new trailer will be added, even if some trailers with
>> +the same (token, value) pair are already in the message.
>> ++
>> +With `overwrite`, the new trailer will overwrite an existing trailer
>> +with the same token.
>
> What if there are multiple existing trailers with the same token?  Are
> they all overwritten?

No, if where == after, only the last one is overwritten, and if where
== before, only the first one is overwritten.

I could add an "overwriteAll" option. It could be interesting to use
when a command using "$ARG" is configured, as this way the command
would apply to all the trailers with the given token instead of just
the last or first one.

>> +With `doNothing`, nothing will be done, that is no new trailer will be
>> +added if there is already one with the same token in the message.
>> +
>> +trailer.<token>.ifmissing::
>> +     This option makes it possible to choose what action will be
>> +     performed when there is not yet any trailer with the same
>> +     token in the message.
>> ++
>> +The valid values for this option are: `add` (this is the default) and
>> +`doNothing`.
>> ++
>> +With `add`, a new trailer will be added.
>> ++
>> +With `doNothing`, nothing will be done.
>> +
>> +trailer.<token>.command::
>> +     This option can be used to specify a shell command that will
>> +     be used to automatically add or modify a trailer with the
>> +     specified 'token'.
>> ++
>> +When this option is specified, it is like if a special 'token=value'
>> +argument is added at the end of the command line, where 'value' will
>> +be given by the standard output of the specified command.
>
> Maybe reword to
>
>     When this option is specified, the behavior is as if a special
>     'token=value' argument were added at the end of the command line,
>     where 'value' is taken to be the standard output of the specified
>     command.
>
> And if it is the case, maybe add "with leading and trailing whitespace
> trimmed off" at the end of the sentence.

Ok.

>> +If the command contains the `$ARG` string, this string will be
>> +replaced with the 'value' part of an existing trailer with the same
>> +token, if any, before the command is launched.
>
> What if the key appears multiple times in existing trailers?

It will be done only once for the last or first trailer with the key
depending on "where".

>> +
>> +SEE ALSO
>> +--------
>> +linkgit:git-commit[1]
>> +
>> +GIT
>> +---
>> +Part of the linkgit:git[1] suite
>>
>
> Doesn't this command have to be added to command-list.txt?

Maybe, I will have a look.

Thanks,
Christian.

Re: [PATCH v10 12/12] trailer: add blank line before the trailers if needed

[Christian Couder]

On Mon, Apr 7, 2014 at 11:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
> Hmph, this is more fixing a mistake made earlier in the series at
> the end than adding a new feature or something.  Can you start from
> a version that does not have the mistake from the beginning?

Ok, I will squash this patch in other previous patches.

Thanks,
Christian.

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Michael Haggerty]

On 04/08/2014 01:35 PM, Christian Couder wrote:
> On Tue, Apr 8, 2014 at 9:30 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
>> How are existing trailers recognized in the input commit message?  Do
>> trailers have to be configured to be recognized?  Or are all lines
>> matching a specific pattern considered trailers?  If so, it might be
>> helpful to include a regexp here that describes the trailer "syntax".
> 
> The trailers are recognized in the input commit message using the
> following rules:
>  - only lines that contains a ':' are considered trailers,
>  - the trailer lines must all be next to each other,
>  - after them it's only possible to have some lines that contain only spaces,
>  - before them there must be at least one line with only spaces

Thanks for all the explanation.  I think that most/all of this
information should be included in the documentation.

>>> +OPTIONS
>>> +-------
>>> +--trim-empty::
>>> +     If the 'value' part of any trailer contains only whitespace,
>>> +     the whole trailer will be removed from the resulting message.
>>
>> Does this apply to existing trailers, new trailers, or both?
> 
> Both.
> 
>> If it applies to existing trailers, then it seems a bit dangerous, in the
>> sense that the command might end up changing trailers that are unrelated
>> to the one that the command is trying to add.
> 
> The command is not just for adding trailers.
> But there could be an option to just trim trailers that are added.

Maybe that should be the *only* behavior of this option.

Maybe there should be a trailer.<token>.trimEmpty config option.

>>> +CONFIGURATION VARIABLES
>>> +-----------------------
>>> +
>>> +trailer.<token>.key::
>>> +     This 'key' will be used instead of 'token' in the
>>> +     trailer. After some alphanumeric characters, it can contain
>>
>> Trailer keys can also contain '-', right?
> 
> Yes.
> I should have written "after the last alphanumeric character".
> I will fix that.
> 
>>> +     some non alphanumeric characters like ':', '=' or '#' that will
>>> +     be used instead of ':' to separate the token from the value in
>>> +     the trailer, though the default ':' is more standard.
>>
>> Above it looks like the default separator is not ':' but rather ': '
>> (with a space).  Is the space always added regardless of the value of
>> this configuration variable, or should the configuration value include
>> the trailing space if it is desired?  Is there any way to get a trailer
>> that doesn't include a space, like
>>
>>     foo=bar
>>
>> ?  (Changing this to "foo= bar" would look pretty ugly.)
> 
> I will have a look, but I think that:
> 
> - a space is always added after ':' or '=',
> - a space is never added after '#',
> - it doesn't matter if there is a space or not in the configured key.
> 
>> If a commit message containing trailer lines with separators other than
>> ':' is input to the program, will it recognize them as trailer lines?
> 
> No, '=' and '#' are not supported in the input message, only in the output.
> 
>> Do such trailer lines have to have the same separator as the one listed
>> in this configuration setting to be recognized?
> 
> No they need to have ':' as a separator.
> 
> The reason why only ':' is supported is because it is the cannonical
> trailer separator and it could create problems with many input
> messages if other separators where supported.
> 
> Maybe we could detect a special line like the following:
> 
> # TRAILERS START
> 
> in the input message and consider everyhting after that line as trailers.
> In this case it would be ok to accept other separators.

It would be ugly to have to use such a line.  I think it would be
preferable to be more restrictive about trailer separators than to
require something like this.

>From what you've said above, it sounds like your code might get confused
with the following input commit message:

    This is the human-readable comment

    Foo: bar
    Fixes #123
    Plugh: xyzzy

It seems to me that none of these lines would be accepted as trailers,
because they include a non-trailer "Fixes" line (non-trailer in the
sense that it doesn't use a colon separator).

>> I suppose that there is some compelling reason to allow non-colon
>> separators here.  If not, it seems like it adds a lot of complexity and
>> should maybe be omitted, or limited to only a few specific separators.
> 
> Yeah, but in the early threads concerning this subject, someone said
> that GitHub for example uses "bug #XXX".
> I will have a look again.

Yes, that's true: GitHub recognizes strings like "fixes #33" but not if
there is an intervening colon like in "fixes: #33".  OTOH GitHub
recognizes such strings wherever they appear in the commit message (they
don't have to be in "trailer" lines).  So I'm not sure that the added
complication is worth it if GitHub is the only use case.  (And maybe we
could convince GitHub to recognize "Fixes: #33" if such syntax becomes
the de-facto Git standard for trailers.)

>>> +trailer.<token>.where::
>>> +     This can be either `after`, which is the default, or
>>> +     `before`. If it is `before`, then a trailer with the specified
>>> +     token, will appear before, instead of after, other trailers
>>> +     with the same token, or otherwise at the beginning, instead of
>>> +     at the end, of all the trailers.
>>
>> Brainstorming: some other options that might make sense here someday:
>>
>> `end`: add new trailer after all existing trailers (even those with
>> different keys).  This would allow trailers to be kept in chronological
>> order.
>>
>> `beginning`: add new trailer before the first existing trailer (allows
>> reverse chronological order).
>>
>> `sorted`: add new trailer among the existing trailers with the same key
>> so as to keep their values in lexicographic order.
> 
> Yeah, I thought about these, but I don't think there is a need for
> them right now.

Yes, I didn't mean to imply that any of these options have to be in the
first version.

>>> +trailer.<token>.ifexist::
>>> +     This option makes it possible to choose what action will be
>>> +     performed when there is already at least one trailer with the
>>> +     same token in the message.
>>> ++
>>> +The valid values for this option are: `addIfDifferent` (this is the
>>> +default), `addIfDifferentNeighbor`, `add`, `overwrite` or `doNothing`.
>>
>> Are these option values case sensitive?  If so, it might be a little bit
>> confusing because the same camel-case is often used in documentation for
>> configuration *keys*, which are not case sensitive [1], and users might
>> have gotten used to thinking of strings that look like this to be
>> non-case-sensitive.
> 
> There were some discussions a few versions of this series ago with
> Peff, Junio and perhaps others about this.
> I thought that being case insensitive was better and Peff kind of
> agreed with that, but as Junio disagreed it is now case sensitive.

OK, it's my fault for not having followed along with the history of this
patch series.

>>> +With `addIfDifferent`, a new trailer will be added only if no trailer
>>> +with the same (token, value) pair is already in the message.
>>> ++
>>> +With `addIfDifferentNeighbor`, a new trailer will be added only if no
>>> +trailer with the same (token, value) pair is above or below the line
>>> +where the new trailer will be added.
>>> ++
>>> +With `add`, a new trailer will be added, even if some trailers with
>>> +the same (token, value) pair are already in the message.
>>> ++
>>> +With `overwrite`, the new trailer will overwrite an existing trailer
>>> +with the same token.
>>
>> What if there are multiple existing trailers with the same token?  Are
>> they all overwritten?
> 
> No, if where == after, only the last one is overwritten, and if where
> == before, only the first one is overwritten.
> 
> I could add an "overwriteAll" option. It could be interesting to use
> when a command using "$ARG" is configured, as this way the command
> would apply to all the trailers with the given token instead of just
> the last or first one.

It seems to me that the current behavior (rewriting exactly one existing
line) is not that useful.  Why not make "overwrite" overwrite *all*
existing matching lines?

>>> +With `doNothing`, nothing will be done, that is no new trailer will be
>>> +added if there is already one with the same token in the message.

I just noticed a punctuation problem (comma -> semicolon and add comma)
in the sentence above:

    With `doNothing`, nothing will be done; that is, no new trailer
    will be added if there is already one with the same token in the
    message.

>>> +
>>> +trailer.<token>.ifmissing::
>>> +     This option makes it possible to choose what action will be
>>> +     performed when there is not yet any trailer with the same
>>> +     token in the message.
>>> ++
>>> +The valid values for this option are: `add` (this is the default) and
>>> +`doNothing`.
>>> ++
>>> +With `add`, a new trailer will be added.
>>> ++
>>> +With `doNothing`, nothing will be done.
>>> +
>>> +trailer.<token>.command::
>>> +     This option can be used to specify a shell command that will
>>> +     be used to automatically add or modify a trailer with the
>>> +     specified 'token'.
>>> ++
>>> +When this option is specified, it is like if a special 'token=value'
>>> +argument is added at the end of the command line, where 'value' will
>>> +be given by the standard output of the specified command.
>>
>> Maybe reword to
>>
>>     When this option is specified, the behavior is as if a special
>>     'token=value' argument were added at the end of the command line,
>>     where 'value' is taken to be the standard output of the specified
>>     command.
>>
>> And if it is the case, maybe add "with leading and trailing whitespace
>> trimmed off" at the end of the sentence.
> 
> Ok.
> 
>>> +If the command contains the `$ARG` string, this string will be
>>> +replaced with the 'value' part of an existing trailer with the same
>>> +token, if any, before the command is launched.
>>
>> What if the key appears multiple times in existing trailers?
> 
> It will be done only once for the last or first trailer with the key
> depending on "where".

It seems like the UI for "git interpret-trailers" is optimized for
trailers that appear only once.  That's a bit limiting.  Maybe it would
be better to pipe the existing values to the command's standard input,
one per line?  For example, suppose we run

    git interpret-trailers \
        Signed-off-by='Christian Couder <christian.couder@gmail.com>'

with the following input:

    Human-readable subject

    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    Signed-off-by: Christian Couder <christian.couder@gmail.com>

Then the following three lines could be piped to the command (i.e., one
value per line, without the key):

    Junio C Hamano <gitster@pobox.com>
    Christian Couder <christian.couder@gmail.com>
    Christian Couder <christian.couder@gmail.com>

Then, supposing the command were "sort --unique", the command's output
would be

    Christian Couder <christian.couder@gmail.com>
    Junio C Hamano <gitster@pobox.com>

which would be converted back into trailer lines by prepending
"Signed-off-by: ", resulting in the modified commit message

    Human-readable subject

    Signed-off-by: Christian Couder <christian.couder@gmail.com>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>

(Not that we would want to do with "Signed-off-by" trailers, but you get
the idea.)


I'm really sorry for coming so late to the show.  Feel free to ignore
any of my comments with the justification "too late".

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Junio C Hamano]

Michael Haggerty <mhagger@alum.mit.edu> writes:

> Sorry for reappearing in this thread after such a long absence.  I
> wanted to see what is coming up (I think this interpret-trailers command
> will be handy!) so I read this documentation patch carefully, and added
> some questions and suggestions below.

Thanks for reading the patch carefully.  It helps to have fresh set
of eyes that are not contaminated by the preconception formed by
previous discussions, especially when reviewing the documentation
whose primary target audiences are those who do not care about these
previous back-and-forth.

>> +trailer.<token>.where::
>> +	This can be either `after`, which is the default, or
>> +	`before`. If it is `before`, then a trailer with the specified
>> +	token, will appear before, instead of after, other trailers
>> +	with the same token, or otherwise at the beginning, instead of
>> +	at the end, of all the trailers.
>
> Brainstorming: some other options that might make sense here someday:
> ...
>> +trailer.<token>.ifexist::
>> +	This option makes it possible to choose what action will be
>> +	performed when there is already at least one trailer with the
>> +	same token in the message.
>> ++
>> +The valid values for this option are: `addIfDifferent` (this is the
>> +default), `addIfDifferentNeighbor`, `add`, `overwrite` or `doNothing`.
>
> Are these option values case sensitive?

It is interesting and somewhat sad that it all has to come back
together inter-twined.  From the very beginning, I was opposed to
having logical complexity that requires multi-words in both variable
names (e.g. "if-exist") and values (e.g. "add-if-different"), and
after $gmane/241929 where I let the devil's advocate "how about
making the variable simpler without logical operation and put all
the conditional on the value side?" suggestion shot down, I somehow
was hoping that the value part got a lot simpler not to require
multi-words, which would have meant that we would not have to worry
about "Is it addIfDifferent? add-if-different? or Add_If_Different?"
at all.  Sadly that is not what we have ended up with.

So, with that realization...

> If so, it might be a little bit
> confusing because the same camel-case is often used in documentation for
> configuration *keys*, which are not case sensitive [1], and users might
> have gotten used to thinking of strings that look like this to be
> non-case-sensitive.

... very true.  Having to have these enum values as so complex to
require multi-words is probably the root cause of the confusion, and
we might probably be better off if we did not have to, but it would
be helpful to allow various different spellings (i.e. make them case
insensitive to allow random camel spellings, and also accept things
like "add-if-different" as well) if we absolutely have to have these
complex values.

But you had a lot of good questions and suggestions for possible
future enhancements that we would need to take into account while
designing the overall scheme to later allow them to fit into.  Maybe
a value that is a single-token that consists of just a few words
(e.g. "addIfDifferent") may not be the best way to go after all.

I dunno.

> What if there are multiple existing trailers with the same token?  Are
> they all overwritten?
> ...
> What if the key appears multiple times in existing trailers?

All good questions, I would think.

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Junio C Hamano]

Christian Couder <chriscool@tuxfamily.org> writes:

> +Help add RFC 822-like headers, called 'trailers', at the end of the
> +otherwise free-form part of a commit message.

I think it is somewhat misleading to use the word "headers" like
that.  'trailers' look similar to RFC-822-headers but they come at
the end.  The sentence however reads as if they are "headers" that
look like RFC 822.  Perhaps shuffling words like so:

	Help adding 'trailers' lines, that look similar to RFC 822
	e-mail headers, at the end of the ...

would make it less confusing.

> +Some configuration variables control the way the `token` arguments are
> +applied to the message and the way any existing trailer in the message
> +is changed. They also make it possible to automatically add some
> +trailers.
> +
> +By default, a 'token=value' or 'token:value' argument will be added
> +only if no trailer with the same (token, value) pair is already in the
> +message. The 'token' and 'value' parts will be trimmed to remove
> +starting and trailing whitespace, and the resulting trimmed 'token'
> +and 'value' will appear in the message like this:
> +
> +------------------------------------------------
> +token: value
> +------------------------------------------------

Mental note: this does assume that the final output for the 'token'
is to have a line <label> that is followed by a colon ":", SP and
the value.

And the natural way to express that on the command line would be to
say "token: value", I would think, but let's just read on.

> +Note that 'trailers' do not follow and are not intended to follow many
> +rules that are in RFC 822. For example they do not follow the line
> +breaking rules, the encoding rules and probably many other rules.

s/that are in RFC 822/for RFC 822 headers/.
s/line breaking/line folding/. (see RFC 822, 3.1.1)

> +OPTIONS
> +-------
> +--trim-empty::
> +	If the 'value' part of any trailer contains only whitespace,
> +	the whole trailer will be removed from the resulting message.
> +
> +CONFIGURATION VARIABLES
> +-----------------------
> +
> +trailer.<token>.key::
> +	This 'key' will be used instead of 'token' in the

As `key` is something that is typed literally, it should be typeset
as `key` in the descriptive text.  I think other manpages spell the
placeholder as `<token>` (or '<token>', I am not sure which...).

> +	trailer. After some alphanumeric characters, it can contain
> +	some non alphanumeric characters like ':', '=' or '#' that will
> +	be used instead of ':' to separate the token from the value in
> +	the trailer, though the default ':' is more standard.

I assume that this is for things like

	bug #538

and the configuration would say something like:

	[trailer "bug"]
        	key = "bug #"

For completeness (of this example), the bog-standard s-o-b would
look like

	Signed-off-by: Christian Couder <chriscool@tuxfamily.org>

and the configuration for it that spell the redundant "key" would
be:

	[trailer "Signed-off-by"]
        	key = "Signed-off-by: "

Am I reading the intention correctly?

That is, when trailer.<token>.key is not defined, the value defaults
to "<token>: " (with one SP after the label and colon), and when it
is defined, the value can come directly after it.

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Christian Couder]

From: Junio C Hamano <gitster@pobox.com>
>
> Christian Couder <chriscool@tuxfamily.org> writes:
> 
>> +Help add RFC 822-like headers, called 'trailers', at the end of the
>> +otherwise free-form part of a commit message.
> 
> I think it is somewhat misleading to use the word "headers" like
> that.  'trailers' look similar to RFC-822-headers but they come at
> the end.  The sentence however reads as if they are "headers" that
> look like RFC 822.  Perhaps shuffling words like so:
> 
> 	Help adding 'trailers' lines, that look similar to RFC 822
> 	e-mail headers, at the end of the ...
> 
> would make it less confusing.

Ok, I made this change in v11.

>> +Some configuration variables control the way the `token` arguments are
>> +applied to the message and the way any existing trailer in the message
>> +is changed. They also make it possible to automatically add some
>> +trailers.
>> +
>> +By default, a 'token=value' or 'token:value' argument will be added
>> +only if no trailer with the same (token, value) pair is already in the
>> +message. The 'token' and 'value' parts will be trimmed to remove
>> +starting and trailing whitespace, and the resulting trimmed 'token'
>> +and 'value' will appear in the message like this:
>> +
>> +------------------------------------------------
>> +token: value
>> +------------------------------------------------
> 
> Mental note: this does assume that the final output for the 'token'
> is to have a line <label> that is followed by a colon ":", SP and
> the value.
> 
> And the natural way to express that on the command line would be to
> say "token: value", I would think, but let's just read on.
> 
>> +Note that 'trailers' do not follow and are not intended to follow many
>> +rules that are in RFC 822. For example they do not follow the line
>> +breaking rules, the encoding rules and probably many other rules.
> 
> s/that are in RFC 822/for RFC 822 headers/.
> s/line breaking/line folding/. (see RFC 822, 3.1.1)

Ok, it's in v11 too.

>> +OPTIONS
>> +-------
>> +--trim-empty::
>> +	If the 'value' part of any trailer contains only whitespace,
>> +	the whole trailer will be removed from the resulting message.
>> +
>> +CONFIGURATION VARIABLES
>> +-----------------------
>> +
>> +trailer.<token>.key::
>> +	This 'key' will be used instead of 'token' in the
> 
> As `key` is something that is typed literally, it should be typeset
> as `key` in the descriptive text.

Ok, I used `key` in v11.

> I think other manpages spell the
> placeholder as `<token>` (or '<token>', I am not sure which...).

I found mostly <token>, so I used that in v11.

>> +	trailer. After some alphanumeric characters, it can contain
>> +	some non alphanumeric characters like ':', '=' or '#' that will
>> +	be used instead of ':' to separate the token from the value in
>> +	the trailer, though the default ':' is more standard.
> 
> I assume that this is for things like
> 
> 	bug #538
> 
> and the configuration would say something like:
> 
> 	[trailer "bug"]
>         	key = "bug #"
> 
> For completeness (of this example), the bog-standard s-o-b would
> look like
> 
> 	Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
> 
> and the configuration for it that spell the redundant "key" would
> be:
> 
> 	[trailer "Signed-off-by"]
>         	key = "Signed-off-by: "

Yeah, but you can use the following instead:

 	[trailer "s-o-b"]
         	key = "Signed-off-by: "

The <token> and the key can be different.

> Am I reading the intention correctly?

Yeah, I think so.

> That is, when trailer.<token>.key is not defined, the value defaults
> to "<token>: " (with one SP after the label and colon),

Yes.

> and when it
> is defined, the value can come directly after it.

The value can come directly after the key, only if the key ends with '#'.

If it ends with something else, except spaces, one SP will be added
between the key and the value.

Yeah, I made '#' special in the hope that it would be more compatible
with GitHub and other services that might also use '#'.

Thanks,
Christian.

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Christian Couder]

From: Michael Haggerty <mhagger@alum.mit.edu>
>
> On 04/08/2014 01:35 PM, Christian Couder wrote:
>> 
>> The trailers are recognized in the input commit message using the
>> following rules:
>>  - only lines that contains a ':' are considered trailers,
>>  - the trailer lines must all be next to each other,
>>  - after them it's only possible to have some lines that contain only spaces,
>>  - before them there must be at least one line with only spaces
> 
> Thanks for all the explanation.  I think that most/all of this
> information should be included in the documentation.

Ok, I included the above rules in v11, but maybe not other pieces of
information that you might have wanted.

>>>> +OPTIONS
>>>> +-------
>>>> +--trim-empty::
>>>> +     If the 'value' part of any trailer contains only whitespace,
>>>> +     the whole trailer will be removed from the resulting message.
>>>
>>> Does this apply to existing trailers, new trailers, or both?
>> 
>> Both.
>> 
>>> If it applies to existing trailers, then it seems a bit dangerous, in the
>>> sense that the command might end up changing trailers that are unrelated
>>> to the one that the command is trying to add.
>> 
>> The command is not just for adding trailers.
>> But there could be an option to just trim trailers that are added.
> 
> Maybe that should be the *only* behavior of this option.
> 
> Maybe there should be a trailer.<token>.trimEmpty config option.

One possible usage of the "git interpret-trailers" command that was
discussed in the early threads was the following:

1) You have a commit message template like the following:

-----------------
***SUBJECT***

***MESSAGE***

Fixes: 
Cc: 
Cc: 
Reviewed-by: 
Reviewed-by: 
Signed-off-by: 
-----------------

2) The user add some information when committing:

$ git commit --trailer "Fixes:534" --trailer "Signed-off-by: Michael <mhagger@alum.mit.edu>"

3) "git interpret-trailers" is used automatically by "git commit"
without --trim-empty, and it is passed the --trailer arguments and the
commit message template, so the user is shown the result which is for
example the following:

-----------------
***SUBJECT***

***MESSAGE***

Fixes: 534
Cc: 
Cc: 
Reviewed-by: 
Reviewed-by: 
Signed-off-by: Michael <mhagger@alum.mit.edu>
-----------------

4) The user adds some information and the resulting message is for
example:

-----------------
Doing foo and bar

And also baz.

Fixes: 534
Cc: 
Cc: Peff <peff@peff.net>
Reviewed-by: Junio <gitster@pobox.com>
Reviewed-by: 
Signed-off-by: Michael <mhagger@alum.mit.edu>
-----------------

5) Then a post commit hook automatically uses "git interpret-trailers
--trim-empty" on the result, so the commit message is eventually the
following:

-----------------
Doing foo and bar

And also baz.

Fixes: 534
Cc: Peff <peff@peff.net>
Reviewed-by: Junio <gitster@pobox.com>
Signed-off-by: Michael <mhagger@alum.mit.edu>
-----------------

So I think it could be very useful to have --trim-empty work on all
the trailers, not just those passed as arguments.

>>> If a commit message containing trailer lines with separators other than
>>> ':' is input to the program, will it recognize them as trailer lines?
>> 
>> No, '=' and '#' are not supported in the input message, only in the output.
>> 
>>> Do such trailer lines have to have the same separator as the one listed
>>> in this configuration setting to be recognized?
>> 
>> No they need to have ':' as a separator.
>> 
>> The reason why only ':' is supported is because it is the cannonical
>> trailer separator and it could create problems with many input
>> messages if other separators where supported.
>> 
>> Maybe we could detect a special line like the following:
>> 
>> # TRAILERS START
>> 
>> in the input message and consider everyhting after that line as trailers.
>> In this case it would be ok to accept other separators.
> 
> It would be ugly to have to use such a line.  I think it would be
> preferable to be more restrictive about trailer separators than to
> require something like this.

The code is already very restrictive about trailer separators.

> From what you've said above, it sounds like your code might get confused
> with the following input commit message:
> 
>     This is the human-readable comment
> 
>     Foo: bar
>     Fixes #123
>     Plugh: xyzzy
> 
> It seems to me that none of these lines would be accepted as trailers,
> because they include a non-trailer "Fixes" line (non-trailer in the
> sense that it doesn't use a colon separator).

Yeah, they would not be accepted because the code is very restrictive.

The following would be accepted:

     Foo: bar
     Fixes: 123
     Plugh: xyzzy

>>> I suppose that there is some compelling reason to allow non-colon
>>> separators here.  If not, it seems like it adds a lot of complexity and
>>> should maybe be omitted, or limited to only a few specific separators.
>> 
>> Yeah, but in the early threads concerning this subject, someone said
>> that GitHub for example uses "bug #XXX".
>> I will have a look again.
> 
> Yes, that's true: GitHub recognizes strings like "fixes #33" but not if
> there is an intervening colon like in "fixes: #33".  OTOH GitHub
> recognizes such strings wherever they appear in the commit message (they
> don't have to be in "trailer" lines).  So I'm not sure that the added
> complication is worth it if GitHub is the only use case.  (And maybe we
> could convince GitHub to recognize "Fixes: #33" if such syntax becomes
> the de-facto Git standard for trailers.)

I don't think there is a lot of complexity.
But maybe I need to explain how it works better.
Feel free to suggest me sentences I could add.

>>>> +With `addIfDifferent`, a new trailer will be added only if no trailer
>>>> +with the same (token, value) pair is already in the message.
>>>> ++
>>>> +With `addIfDifferentNeighbor`, a new trailer will be added only if no
>>>> +trailer with the same (token, value) pair is above or below the line
>>>> +where the new trailer will be added.
>>>> ++
>>>> +With `add`, a new trailer will be added, even if some trailers with
>>>> +the same (token, value) pair are already in the message.
>>>> ++
>>>> +With `overwrite`, the new trailer will overwrite an existing trailer
>>>> +with the same token.
>>>
>>> What if there are multiple existing trailers with the same token?  Are
>>> they all overwritten?
>> 
>> No, if where == after, only the last one is overwritten, and if where
>> == before, only the first one is overwritten.
>> 
>> I could add an "overwriteAll" option. It could be interesting to use
>> when a command using "$ARG" is configured, as this way the command
>> would apply to all the trailers with the given token instead of just
>> the last or first one.
> 
> It seems to me that the current behavior (rewriting exactly one existing
> line) is not that useful.  Why not make "overwrite" overwrite *all*
> existing matching lines?

I was thinking that people could use the following template message:

---------------
Signed-off-by: 
Signed-off-by: YOU-WILL-BE-AUTOMATICALLY-ADDED-HERE
---------------

and the following config:

---------------
[trailer "s-o-b"]
	 key = "Signed-off-by: "
	 ifexist = overwrite
	 command = echo \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\"
---------------

This way the user can add other people's s-o-b before the last one
which will always contain his own s-o-b.

>>>> +If the command contains the `$ARG` string, this string will be
>>>> +replaced with the 'value' part of an existing trailer with the same
>>>> +token, if any, before the command is launched.
>>>
>>> What if the key appears multiple times in existing trailers?
>> 
>> It will be done only once for the last or first trailer with the key
>> depending on "where".
> 
> It seems like the UI for "git interpret-trailers" is optimized for
> trailers that appear only once.  That's a bit limiting.

As I said, it is possible to add an overwriteAll option.
I think that would fix the current limitations.

> Maybe it would
> be better to pipe the existing values to the command's standard input,
> one per line?  For example, suppose we run
> 
>     git interpret-trailers \
>         Signed-off-by='Christian Couder <christian.couder@gmail.com>'
> 
> with the following input:
> 
>     Human-readable subject
> 
>     Signed-off-by: Junio C Hamano <gitster@pobox.com>
>     Signed-off-by: Christian Couder <christian.couder@gmail.com>
> 
> Then the following three lines could be piped to the command (i.e., one
> value per line, without the key):
> 
>     Junio C Hamano <gitster@pobox.com>
>     Christian Couder <christian.couder@gmail.com>
>     Christian Couder <christian.couder@gmail.com>
> 
> Then, supposing the command were "sort --unique", the command's output
> would be
> 
>     Christian Couder <christian.couder@gmail.com>
>     Junio C Hamano <gitster@pobox.com>
> 
> which would be converted back into trailer lines by prepending
> "Signed-off-by: ", resulting in the modified commit message
> 
>     Human-readable subject
> 
>     Signed-off-by: Christian Couder <christian.couder@gmail.com>
>     Signed-off-by: Junio C Hamano <gitster@pobox.com>
> 
> (Not that we would want to do with "Signed-off-by" trailers, but you get
> the idea.)

Yeah, but I think the existing options like addIfDifferent and
addIfDifferentNeighbor are simpler to do these kind of things.

And it is also possible to add a "where = sorted" option.

And if later we realize that people are still not happy, we can still
add a special "trailer.<token>.filter" that could do what you suggest.

Thanks,
Christian.

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Michael Haggerty]

On 04/25/2014 11:07 PM, Christian Couder wrote:
> From: Michael Haggerty <mhagger@alum.mit.edu>
>>>>> +OPTIONS
>>>>> +-------
>>>>> +--trim-empty::
>>>>> +     If the 'value' part of any trailer contains only whitespace,
>>>>> +     the whole trailer will be removed from the resulting message.
>>>>
>>>> Does this apply to existing trailers, new trailers, or both?
>>>
>>> Both.
>>>
>>>> If it applies to existing trailers, then it seems a bit dangerous, in the
>>>> sense that the command might end up changing trailers that are unrelated
>>>> to the one that the command is trying to add.
>>>
>>> The command is not just for adding trailers.
>>> But there could be an option to just trim trailers that are added.
>>
>> Maybe that should be the *only* behavior of this option.
>>
>> Maybe there should be a trailer.<token>.trimEmpty config option.
> 
> One possible usage of the "git interpret-trailers" command that was
> discussed in the early threads was the following:
> 
> 1) You have a commit message template like the following:
> 
> -----------------
> ***SUBJECT***
> 
> ***MESSAGE***
> 
> Fixes: 
> Cc: 
> Cc: 
> Reviewed-by: 
> Reviewed-by: 
> Signed-off-by: 
> -----------------
> [...etc...]

Thanks for the explanation.  Now the --trim-empty option makes a lot
more sense.

>>>> If a commit message containing trailer lines with separators other than
>>>> ':' is input to the program, will it recognize them as trailer lines?
>>>
>>> No, '=' and '#' are not supported in the input message, only in the output.
>>>
>>>> Do such trailer lines have to have the same separator as the one listed
>>>> in this configuration setting to be recognized?
>>>
>>> No they need to have ':' as a separator.
>>>
>>> The reason why only ':' is supported is because it is the cannonical
>>> trailer separator and it could create problems with many input
>>> messages if other separators where supported.
>>>
>>> Maybe we could detect a special line like the following:
>>>
>>> # TRAILERS START
>>>
>>> in the input message and consider everyhting after that line as trailers.
>>> In this case it would be ok to accept other separators.
>>
>> It would be ugly to have to use such a line.  I think it would be
>> preferable to be more restrictive about trailer separators than to
>> require something like this.
> 
> The code is already very restrictive about trailer separators.
> 
>> From what you've said above, it sounds like your code might get confused
>> with the following input commit message:
>>
>>     This is the human-readable comment
>>
>>     Foo: bar
>>     Fixes #123
>>     Plugh: xyzzy
>>
>> It seems to me that none of these lines would be accepted as trailers,
>> because they include a non-trailer "Fixes" line (non-trailer in the
>> sense that it doesn't use a colon separator).
> 
> Yeah, they would not be accepted because the code is very restrictive.
> 
> The following would be accepted:
> 
>      Foo: bar
>      Fixes: 123
>      Plugh: xyzzy
> 
>>>> I suppose that there is some compelling reason to allow non-colon
>>>> separators here.  If not, it seems like it adds a lot of complexity and
>>>> should maybe be omitted, or limited to only a few specific separators.
>>>
>>> Yeah, but in the early threads concerning this subject, someone said
>>> that GitHub for example uses "bug #XXX".
>>> I will have a look again.
>>
>> Yes, that's true: GitHub recognizes strings like "fixes #33" but not if
>> there is an intervening colon like in "fixes: #33".  OTOH GitHub
>> recognizes such strings wherever they appear in the commit message (they
>> don't have to be in "trailer" lines).  So I'm not sure that the added
>> complication is worth it if GitHub is the only use case.  (And maybe we
>> could convince GitHub to recognize "Fixes: #33" if such syntax becomes
>> the de-facto Git standard for trailers.)
> 
> I don't think there is a lot of complexity.
> But maybe I need to explain how it works better.
> Feel free to suggest me sentences I could add.

I am really excited about having better support for trailers in Git, and
I want to thank you for your work.  For me the promise of trailers is

* A way for users to add information to commits for whatever purpose
  they want, without having to convince upstream to built support in.

* A standard format for that information, so that all tools can agree
  how to read/write trailers without being confused by or breaking
  trailers that they didn't know about in advance.

* A format that is straightforward enough that it can be machine-
  readable with minimum ambiguity.

* Some command-line tools to make it easy for scripts to work with
  trailers, and that serve as a reference implementation that other
  Git implementations can imitate.  For example, I totally expect that
  we will soon want a command-line tool for inquiring about the
  presence and contents of trailers, for use in scripting.  Eventually
  we will want to be able to do stuff like

      git trailers --get-all s-o-b origin/master..origin/next
      git rev-list --trailer=s-o-b:gitster@pobox.com master
      git trailers --pipe --draft \
          --add-first fixes \
          --append '# You can delete the following line:' \
          --append s-o-b:"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>" \
          --unset private
      git trailers --pipe --verify --tidy-up

I think it is really important to nail down the format of trailers
tightly enough that everybody who reads/writes a commit message agrees
about exactly what trailers are there.  For example the specification
might look something like:

    A commit message can optionally end with a block of trailers.
    The trailers, if present, must be separated from the rest of the
    commit message by one or more blank lines (lines that contain only
    whitespace).  There must be no blank lines within the list of
    trailers.  It is allowed to have blank lines after the trailers.

    Each trailer line must match the following Perl regular
    expression:

        ^([A-Za-z0-9_-]+)\s*:\s*(.*[^\s])\s*$

    The string matching the first group is called the key and the string
    matching the second is called the value.  Keys are considered to be
    case-insensitive [or should they be case-sensitive?].  The
    interpretation of values is left entirely up to the application.
    Values must not be empty.

    However, in --draft and --cleanup modes, empty values *are*
    allowed, as are comments (lines starting with `core.commentchar`)
    within the trailer block.  In --draft mode such lines are passed
    through unchanged, and in --cleanup mode such lines are removed.

I'm not saying this is the exact definition that we want; I'm just
providing an example of the level of precision that I think is needed.

With regard to the separator character, my concern is not about how to
document the rules for this one tool.  It's more about having really
well-defined rules that are consistent between reading and writing.  For
me it seems silly to let "git interpret-trailers" output trailers that
it doesn't know how to read back in, and pretty much be a show-stopper
if the presence of such trailers makes the tool unable to read other
trailers in the same commit message.

So my preference would be to make the format of trailers really strict;
for example, only allowing colon separators as in the regexp above.
People who want to work with trailers using Git tools will just have to
conform to this format.

But if we must support flexibility in the separator characters, then I
think it is important that we be able to read whatever we can write.
For me this means:

* Enumerating a list of allowed separators (e.g., [:=#])

* Specifying how it is decided what separator to use when generating
  new trailers

* Specifying what appends when a trailer is read and then written again:
  is its separator preserved, or is the trailer converted to use the
  separator configured for that particular key in the config.  And if
  the latter, what happens if a key's separator is not configured?

* Specifying whether whitespace around a separator is adjusted when
  reading then writing a trailer.  For example, is

      Foo SP SP : HT bar SP

  canonicalized to

      Foo: SP bar

  (SP=space, HT=tab)?  What about

      Fixes SP #33

  ?  What if the separator for the "fixes" key is not configured?

The reason that I prefer supporting only colons is that more flexibility
inevitably raises lots of questions like this, makes the documentation
and implementation more complicated, and makes it harder for other
implementations to be sure they agree 100% with the reference
implementation.

>>>>> +With `addIfDifferent`, a new trailer will be added only if no trailer
>>>>> +with the same (token, value) pair is already in the message.
>>>>> ++
>>>>> +With `addIfDifferentNeighbor`, a new trailer will be added only if no
>>>>> +trailer with the same (token, value) pair is above or below the line
>>>>> +where the new trailer will be added.
>>>>> ++
>>>>> +With `add`, a new trailer will be added, even if some trailers with
>>>>> +the same (token, value) pair are already in the message.
>>>>> ++
>>>>> +With `overwrite`, the new trailer will overwrite an existing trailer
>>>>> +with the same token.
>>>>
>>>> What if there are multiple existing trailers with the same token?  Are
>>>> they all overwritten?
>>>
>>> No, if where == after, only the last one is overwritten, and if where
>>> == before, only the first one is overwritten.
>>>
>>> I could add an "overwriteAll" option. It could be interesting to use
>>> when a command using "$ARG" is configured, as this way the command
>>> would apply to all the trailers with the given token instead of just
>>> the last or first one.
>>
>> It seems to me that the current behavior (rewriting exactly one existing
>> line) is not that useful.  Why not make "overwrite" overwrite *all*
>> existing matching lines?
> 
> I was thinking that people could use the following template message:
> 
> ---------------
> Signed-off-by: 
> Signed-off-by: YOU-WILL-BE-AUTOMATICALLY-ADDED-HERE
> ---------------
> 
> and the following config:
> 
> ---------------
> [trailer "s-o-b"]
> 	 key = "Signed-off-by: "
> 	 ifexist = overwrite
> 	 command = echo \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\"
> ---------------
> 
> This way the user can add other people's s-o-b before the last one
> which will always contain his own s-o-b.

That seems fragile.  For example, if the user changes the lines to

    Signed-off-by: Somebody Else <...>

(deleting the "YOU-WILL-BE" line, maybe because they don't want to sign
off the commit) then not only will their wish be contradicted, but also
Somebody Else would be deleted.

What about allowing a --draft option, which allows blank trailer values
plus comments interspersed in the trailer lines?  (I.e., the equivalent
of --trim-empty could be the default and --draft would turn it off plus
allow interspersed comments.)  Then the template could be

    # You can add one or more Signed-off-by lines for other people here.
    # A Signed-off-by line for you will be appended automatically when
    # you commit.
    Signed-off-by:

Or, even better:

    # You can add one or more Signed-off-by lines here:
    Signed-off-by:
    Signed-off-by:
    # You can delete the following line if you don't want it:
    Signed-off-by: Me <me@example.com>

; i.e., the Signed-off-by line for the author could be filled in
*before* the user is asked to edit the commit message.  There could also
be a --cleanup mode that allows blank values and comments on input but
removes them from the output.

> [...]

Given Git's requirements for backwards compatibility, a specification
that we release now will have to be supported forever (because it will
be baked into commits and can *never* be changed), and any
trailer-handling tools that we release now will have to be supported for
years (until at least Git 3.0).

All in all, I think that there has been a lot of discussion about the
interface of this one command, "git interpret-trailers", including its
quite complicated configuration and a command-line behavior.  And yet it
seems to me that not many Git developers have been very engaged in the
conversation, and Junio (who has) still doesn't seem satisfied with it.
 I (though among the too-little engaged) have the feeling that it is
still a ways from maturity.

On the contrary, the data format and semantics of the finished trailers
seem to have gotten too little attention, even though they are simpler
to define and even more important than the interface of the command used
to manipulate trailers.

I think it would be really helpful to have a careful specification of
the data format, and make sure that everybody agrees on what we want.
For example, I think it is crucial that the trailers can be read and
written unambiguously.

Once that's clear, it will be a lot easier to be sure that the tool(s)
for working with trailers conform to the specification.

Even then, I think it might be prudent to mark "git interpret-trailers"
as "experimental" and/or put it under "contrib" rather than among the
main Git commands for a couple of releases.  Luckily, it is very loosely
coupled to the rest of Git, so I don't see any urgency to having it in
core [1].  After people have had time to experiment with it, then it
could be moved to core.

Michael

[1] Having the script in contrib would also make it possible to
implement it use a scripting language to make it easier to iterate on
the design.  When the details are agreed it could have been
reimplemented in C.  But I guess that ship has already sailed.

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Junio C Hamano]

Christian Couder <chriscool@tuxfamily.org> writes:

> From: Junio C Hamano <gitster@pobox.com>
>>
>> Christian Couder <chriscool@tuxfamily.org> writes:
>> ...
>
>>> +	trailer. After some alphanumeric characters, it can contain
>>> +	some non alphanumeric characters like ':', '=' or '#' that will
>>> +	be used instead of ':' to separate the token from the value in
>>> +	the trailer, though the default ':' is more standard.
>> 
>> I assume that this is for things like
>> 
>> 	bug #538
>> 
>> and the configuration would say something like:
>> 
>> 	[trailer "bug"]
>>         	key = "bug #"
>> 
>> For completeness (of this example), the bog-standard s-o-b would
>> look like
>> 
>> 	Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
>> 
>> and the configuration for it that spell the redundant "key" would
>> be:
>> 
>> 	[trailer "Signed-off-by"]
>>         	key = "Signed-off-by: "
>
> Yeah, but you can use the following instead:
>
>  	[trailer "s-o-b"]
>          	key = "Signed-off-by: "

Sure, but note that both of these have a SP at the end in the value
part (which I think is a sensible thing to do).

> The <token> and the key can be different.
>
>> Am I reading the intention correctly?
>
> Yeah, I think so.
>
>> That is, when trailer.<token>.key is not defined, the value defaults
>> to "<token>: " (with one SP after the label and colon),
>
> Yes.
>
>> and when it
>> is defined, the value can come directly after it.
>
> The value can come directly after the key, only if the key ends with '#'.
>
> If it ends with something else, except spaces, one SP will be added
> between the key and the value.

And I do not think we want (or even need) this "only when it ends
with #" special casing in the code at all.  When the project's
convention is to say "frotz# value-of-frotz", the users will specify
that with 'key = "frotz# "' (with a trailing SP in the value part),
and in a project that wants 'nitfol %value-of-nitfol', your parser
will find 'key = "nitfol %"'.  The users will obtain the result they
want for either case, and a hard-coded special casing in the code
that only has incomplete knowledge on the project convention will
actively harm them.  I'd suggest dropping that special case.

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Jeremy Morton]

On 28/04/2014 17:37, Junio C Hamano wrote:
> Christian Couder<chriscool@tuxfamily.org>  writes:
>
>> From: Junio C Hamano<gitster@pobox.com>
>>>
>>> Christian Couder<chriscool@tuxfamily.org>  writes:
>>> ...
>>
>>>> +	trailer. After some alphanumeric characters, it can contain
>>>> +	some non alphanumeric characters like ':', '=' or '#' that will
>>>> +	be used instead of ':' to separate the token from the value in
>>>> +	the trailer, though the default ':' is more standard.
>>>
>>> I assume that this is for things like
>>>
>>> 	bug #538
>>>
>>> and the configuration would say something like:
>>>
>>> 	[trailer "bug"]
>>>          	key = "bug #"
>>>
>>> For completeness (of this example), the bog-standard s-o-b would
>>> look like
>>>
>>> 	Signed-off-by: Christian Couder<chriscool@tuxfamily.org>
>>>
>>> and the configuration for it that spell the redundant "key" would
>>> be:
>>>
>>> 	[trailer "Signed-off-by"]
>>>          	key = "Signed-off-by: "
>>
>> Yeah, but you can use the following instead:
>>
>>   	[trailer "s-o-b"]
>>           	key = "Signed-off-by: "

One thing I'm not quite understanding is where the "Christian 
Couder<chriscool@tuxfamily.org>" bit comes from.  So you've defined the 
trailer token and key, but interpret-trailers then needs to get the 
value it will give for the key from somewhere.  Does it have to just be 
hardcoded in?  We probably want some way to get various variables like 
current branch name, current git version, etc.  So in the case of always 
adding a trailer for the branch that the commit was checked in to at the 
time (Developed-on, Made-on-branch, Author-branch, etc. [I think my 
favourite is Made-on-branch]), you'd want something like:

	[trailer "m-o-b"]
		key = "Made-on-branch: "
		value = "$currentBranch"

... resulting in the trailer (for example):
	Made-on-branch: pacman-minigame

Also, if there were no current branch name because you're committing in 
a detached head state, it would be nice if you could have some logic to 
determine that, and instead write the trailer as:
	Made-on-branch: (detached HEAD: AB12CD34)

... or whatever.  And also how about some logic to be able to say that 
if you're committing to the "master" branch, the trailer doesn't get 
inserted at all?

-- 
Best regards,
Jeremy Morton (Jez)

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Christian Couder]

On Tue, Apr 29, 2014 at 1:05 PM, Jeremy Morton <admin@game-point.net> wrote:
> On 28/04/2014 17:37, Junio C Hamano wrote:
>>
>> Christian Couder<chriscool@tuxfamily.org>  writes:
>>
>>> From: Junio C Hamano<gitster@pobox.com>
>>>>
>>>>
>>>> Christian Couder<chriscool@tuxfamily.org>  writes:
>>>> ...
>>>
>>>
>>>>> +       trailer. After some alphanumeric characters, it can contain
>>>>> +       some non alphanumeric characters like ':', '=' or '#' that will
>>>>> +       be used instead of ':' to separate the token from the value in
>>>>> +       the trailer, though the default ':' is more standard.
>>>>
>>>>
>>>> I assume that this is for things like
>>>>
>>>>         bug #538
>>>>
>>>> and the configuration would say something like:
>>>>
>>>>         [trailer "bug"]
>>>>                 key = "bug #"
>>>>
>>>> For completeness (of this example), the bog-standard s-o-b would
>>>> look like
>>>>
>>>>         Signed-off-by: Christian Couder<chriscool@tuxfamily.org>
>>>>
>>>> and the configuration for it that spell the redundant "key" would
>>>> be:
>>>>
>>>>         [trailer "Signed-off-by"]
>>>>                 key = "Signed-off-by: "
>>>
>>>
>>> Yeah, but you can use the following instead:
>>>
>>>         [trailer "s-o-b"]
>>>                 key = "Signed-off-by: "
>
>
> One thing I'm not quite understanding is where the "Christian
> Couder<chriscool@tuxfamily.org>" bit comes from.  So you've defined the
> trailer token and key, but interpret-trailers then needs to get the value it
> will give for the key from somewhere.  Does it have to just be hardcoded in?
> We probably want some way to get various variables like current branch name,
> current git version, etc.  So in the case of always adding a trailer for the
> branch that the commit was checked in to at the time (Developed-on,
> Made-on-branch, Author-branch, etc. [I think my favourite is
> Made-on-branch]), you'd want something like:
>
>         [trailer "m-o-b"]
>                 key = "Made-on-branch: "
>                 value = "$currentBranch"
>
> ... resulting in the trailer (for example):
>         Made-on-branch: pacman-minigame

In the documentation patch, there is:

trailer.<token>.command::
       This option can be used to specify a shell command that will
       be used to automatically add or modify a trailer with the
       specified 'token'.

       When this option is specified, it is like if a special 'token=value'
       argument is added at the end of the command line, where 'value' will
       be given by the standard output of the specified command.

       If the command contains the `$ARG` string, this string will be
       replaced with the 'value' part of an existing trailer with the same
       token, if any, before the command is launched.

That's why Something like the following should work if "git commit"
automitically runs "git interpret-trailers":

         [trailer "m-o-b"]
                 key = "Made-on-branch: "
                 command = "git name-rev --name-only HEAD"


> Also, if there were no current branch name because you're committing in a
> detached head state, it would be nice if you could have some logic to
> determine that, and instead write the trailer as:
>         Made-on-branch: (detached HEAD: AB12CD34)

You may need to write a small script for that.
Then you just need the "trailer.m-o-b.command" config value to point
to your script.

> ... or whatever.  And also how about some logic to be able to say that if
> you're committing to the "master" branch, the trailer doesn't get inserted
> at all?

You can script that too.

Best,
Christian.

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Jeremy Morton]

On 29/04/2014 12:47, Christian Couder wrote:
>> Also, if there were no current branch name because you're committing in a
>> detached head state, it would be nice if you could have some logic to
>> determine that, and instead write the trailer as:
>>          Made-on-branch: (detached HEAD: AB12CD34)
>
> You may need to write a small script for that.
> Then you just need the "trailer.m-o-b.command" config value to point
> to your script.
>
>> ... or whatever.  And also how about some logic to be able to say that if
>> you're committing to the "master" branch, the trailer doesn't get inserted
>> at all?
>
> You can script that too.

But it would be nicer if the logic were built-in, then you wouldn't have 
to share some script with your work colleagues. :-)

-- 
Best regards,
Jeremy Morton (Jez)

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Jeremy Morton]

On 29/04/2014 12:47, Christian Couder wrote:
>> Also, if there were no current branch name because you're committing in a
>> detached head state, it would be nice if you could have some logic to
>> determine that, and instead write the trailer as:
>>          Made-on-branch: (detached HEAD: AB12CD34)
>
> You may need to write a small script for that.
> Then you just need the "trailer.m-o-b.command" config value to point
> to your script.
>
>> ... or whatever.  And also how about some logic to be able to say that if
>> you're committing to the "master" branch, the trailer doesn't get inserted
>> at all?
>
> You can script that too.

But it would be nicer if the logic were built-in, then you wouldn't have 
to share some script with your work colleagues. :-)

-- 
Best regards,
Jeremy Morton (Jez)

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Christian Couder]

From: Jeremy Morton <admin@game-point.net>

> On 29/04/2014 12:47, Christian Couder wrote:
>>> Also, if there were no current branch name because you're committing
>>> in a
>>> detached head state, it would be nice if you could have some logic to
>>> determine that, and instead write the trailer as:
>>>          Made-on-branch: (detached HEAD: AB12CD34)
>>
>> You may need to write a small script for that.
>> Then you just need the "trailer.m-o-b.command" config value to point
>> to your script.
>>
>>> ... or whatever.  And also how about some logic to be able to say that
>>> if
>>> you're committing to the "master" branch, the trailer doesn't get
>>> inserted
>>> at all?
>>
>> You can script that too.
> 
> But it would be nicer if the logic were built-in, then you wouldn't
> have to share some script with your work colleagues. :-)

The above logic is very specific to your workflow. For example some
people might a "Made-on-branch: " trailer only when they are on real
branches except "dev" and "master".

Best,
Christian.

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Christian Couder]

From: Michael Haggerty <mhagger@alum.mit.edu>

> On 04/25/2014 11:07 PM, Christian Couder wrote:
>> 
>> I don't think there is a lot of complexity.
>> But maybe I need to explain how it works better.
>> Feel free to suggest me sentences I could add.
> 
> I am really excited about having better support for trailers in Git, and
> I want to thank you for your work.  For me the promise of trailers is
> 
> * A way for users to add information to commits for whatever purpose
>   they want, without having to convince upstream to built support in.

Yeah, I agree this is the main purpose of trailers.

> * A standard format for that information, so that all tools can agree
>   how to read/write trailers without being confused by or breaking
>   trailers that they didn't know about in advance.

Yeah, but don't you think this goal can sometimes go against the
previous goal?

I mean, if some users for their project think that it's better, for
example, if they use trailers like "Fix #42" instead of "Fix: 42",
because their bug tracking system supports "Fix #42" better, we should
let them do what suits them better, even if Git supports them not as
well as if they used "Fix: 42".

> * A format that is straightforward enough that it can be machine-
>   readable with minimum ambiguity.

Yeah, but again this could go against the main purpose of trailers
above.

> * Some command-line tools to make it easy for scripts to work with
>   trailers, and that serve as a reference implementation that other
>   Git implementations can imitate.

Yeah, ok, as long as we keep in mind the main purpose.

> For example, I totally expect that
>   we will soon want a command-line tool for inquiring about the
>   presence and contents of trailers, for use in scripting.  Eventually
>   we will want to be able to do stuff like
> 
>       git trailers --get-all s-o-b origin/master..origin/next
>       git rev-list --trailer=s-o-b:gitster@pobox.com master
>       git trailers --pipe --draft \
>           --add-first fixes \
>           --append '# You can delete the following line:' \
>           --append s-o-b:"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>" \
>           --unset private
>       git trailers --pipe --verify --tidy-up

Yeah, feel free to help make this kind of things possible :-)

> I think it is really important to nail down the format of trailers
> tightly enough that everybody who reads/writes a commit message agrees
> about exactly what trailers are there.

I think we should have a default format for trailers that is clear,
but we should not force users to use this format. Because forcing it
would go against the main goal of trailers that you listed first
above.

> For example the specification
> might look something like:
> 
>     A commit message can optionally end with a block of trailers.
>     The trailers, if present, must be separated from the rest of the
>     commit message by one or more blank lines (lines that contain only
>     whitespace).  There must be no blank lines within the list of
>     trailers.  It is allowed to have blank lines after the trailers.
> 
>     Each trailer line must match the following Perl regular
>     expression:
> 
>         ^([A-Za-z0-9_-]+)\s*:\s*(.*[^\s])\s*$
> 
>     The string matching the first group is called the key and the string
>     matching the second is called the value.  Keys are considered to be
>     case-insensitive [or should they be case-sensitive?].  The
>     interpretation of values is left entirely up to the application.
>     Values must not be empty.

I tried to be clearer in the v12 I just posted, and I think it should
be enough to be very clear. We might want to tweak a little the
specifications later, so being too strict might be counter productive.

And as other tools might already use trailers in a case-sensitive way
and yet other tools in a case-insensitive way, I am not sure we would
gain anything by specifying if keys or values should be interpreted in
a case-sensitive or case-insensitive way. On the contrary we might
upset people already using some of these tools for no good reason.

>     However, in --draft and --cleanup modes, empty values *are*
>     allowed, as are comments (lines starting with `core.commentchar`)
>     within the trailer block.  In --draft mode such lines are passed
>     through unchanged, and in --cleanup mode such lines are removed.

I am not sure we should use modes. I think options like
"--trim-empty", "--allow-comments", "--allow-empty" might be clearer.

> I'm not saying this is the exact definition that we want; I'm just
> providing an example of the level of precision that I think is needed.

Yeah, but I think too much precision can be counter productive.

> With regard to the separator character, my concern is not about how to
> document the rules for this one tool.  It's more about having really
> well-defined rules that are consistent between reading and writing.  For
> me it seems silly to let "git interpret-trailers" output trailers that
> it doesn't know how to read back in, and pretty much be a show-stopper
> if the presence of such trailers makes the tool unable to read other
> trailers in the same commit message.

We might allow an option to specify witch separator(s) should be
allowed in the input messages for example. Right now I think it is
enough if we support well the default separator, ':' in the input
message.

> So my preference would be to make the format of trailers really strict;
> for example, only allowing colon separators as in the regexp above.
> People who want to work with trailers using Git tools will just have to
> conform to this format.

I don't think we should cast in stone the format for trailers, because
of the main purpose of trailers.

The format of the commit header for example is cast in stone, but
that's ok because it is mostly for Git internal use. Trailers are
mostly for external use by users who already have tools expecting
different formats.

There are already users who are not happy that they cannot easily have
other commit headers, and we point them to trailers. If we specify
trailers too strictly, where will we point them to?

> But if we must support flexibility in the separator characters, then I
> think it is important that we be able to read whatever we can write.

An option like --input-separator might be enough to support this.

> For me this means:
> 
> * Enumerating a list of allowed separators (e.g., [:=#])

Junio suggested in a message that users might use different separators
like '%'.
 
> * Specifying how it is decided what separator to use when generating
>   new trailers

This is already possible with the 'trailer.<token>.key' config
variable.

> * Specifying what appends when a trailer is read and then written again:
>   is its separator preserved, or is the trailer converted to use the
>   separator configured for that particular key in the config.  And if
>   the latter, what happens if a key's separator is not configured?

Right now we only accept ':' as input separator for the messages and
':' and '=' for the --trailer option, and the default output separator
is ':'. If the user specify a different separator in a key, this
separator will be used only in the output for this key.

If this is not clear in the documentation, please susggest specific
improvements.

> * Specifying whether whitespace around a separator is adjusted when
>   reading then writing a trailer.  For example, is
> 
>       Foo SP SP : HT bar SP
> 
>   canonicalized to
> 
>       Foo: SP bar
> 
>   (SP=space, HT=tab)?  What about
> 
>       Fixes SP #33
> 
>   ?  What if the separator for the "fixes" key is not configured?

I tried to be very clear in the doc in v12.

> The reason that I prefer supporting only colons is that more flexibility
> inevitably raises lots of questions like this, makes the documentation
> and implementation more complicated, and makes it harder for other
> implementations to be sure they agree 100% with the reference
> implementation.

Yeah, but we should not forget the main purpose of trailers.

>>> It seems to me that the current behavior (rewriting exactly one existing
>>> line) is not that useful.  Why not make "overwrite" overwrite *all*
>>> existing matching lines?
>> 
>> I was thinking that people could use the following template message:
>> 
>> ---------------
>> Signed-off-by: 
>> Signed-off-by: YOU-WILL-BE-AUTOMATICALLY-ADDED-HERE
>> ---------------
>> 
>> and the following config:
>> 
>> ---------------
>> [trailer "s-o-b"]
>> 	 key = "Signed-off-by: "
>> 	 ifexist = overwrite
>> 	 command = echo \"$GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL>\"
>> ---------------
>> 
>> This way the user can add other people's s-o-b before the last one
>> which will always contain his own s-o-b.
> 
> That seems fragile.  For example, if the user changes the lines to
> 
>     Signed-off-by: Somebody Else <...>
> 
> (deleting the "YOU-WILL-BE" line, maybe because they don't want to sign
> off the commit) then not only will their wish be contradicted, but also
> Somebody Else would be deleted.

I agree that it is fragile, but we can add an overwriteAll option if
that suits people needs better. "overwrite" is needed anyway for
trailers where there should be only one trailer with a given key.

> What about allowing a --draft option, which allows blank trailer values
> plus comments interspersed in the trailer lines?  (I.e., the equivalent
> of --trim-empty could be the default and --draft would turn it off plus
> allow interspersed comments.)

Yeah, or --allow-comments. As I said above prefer many orthogonal
options, rather than some general options like --draft.

> Then the template could be
> 
>     # You can add one or more Signed-off-by lines for other people here.
>     # A Signed-off-by line for you will be appended automatically when
>     # you commit.
>     Signed-off-by:
> 
> Or, even better:
> 
>     # You can add one or more Signed-off-by lines here:
>     Signed-off-by:
>     Signed-off-by:
>     # You can delete the following line if you don't want it:
>     Signed-off-by: Me <me@example.com>
> 
> ; i.e., the Signed-off-by line for the author could be filled in
> *before* the user is asked to edit the commit message.  There could also
> be a --cleanup mode that allows blank values and comments on input but
> removes them from the output.

I would prefer to add --trim-comments rather than a --cleanup mode. 

>> [...]
> 
> Given Git's requirements for backwards compatibility, a specification
> that we release now will have to be supported forever (because it will
> be baked into commits and can *never* be changed), and any
> trailer-handling tools that we release now will have to be supported for
> years (until at least Git 3.0).

Yeah, I know that. So if we are too strict in the specification will
be stuck for a long time.

> All in all, I think that there has been a lot of discussion about the
> interface of this one command, "git interpret-trailers", including its
> quite complicated configuration and a command-line behavior.  And yet it
> seems to me that not many Git developers have been very engaged in the
> conversation, and Junio (who has) still doesn't seem satisfied with it.
>  I (though among the too-little engaged) have the feeling that it is
> still a ways from maturity.

My opinion is that many Git developers have been engaged and you can
see that in the Cc.

I cannot tell if they are all very happy or not but I suppose that if
they were very unhappy they would tell it.

> On the contrary, the data format and semantics of the finished trailers
> seem to have gotten too little attention, even though they are simpler
> to define and even more important than the interface of the command used
> to manipulate trailers.
> 
> I think it would be really helpful to have a careful specification of
> the data format, and make sure that everybody agrees on what we want.
> For example, I think it is crucial that the trailers can be read and
> written unambiguously.
> 
> Once that's clear, it will be a lot easier to be sure that the tool(s)
> for working with trailers conform to the specification.

Please realize that too much specification is not always good and that
it cuts both ways...

> Even then, I think it might be prudent to mark "git interpret-trailers"
> as "experimental" and/or put it under "contrib" rather than among the
> main Git commands for a couple of releases.  Luckily, it is very loosely
> coupled to the rest of Git, so I don't see any urgency to having it in
> core [1].  After people have had time to experiment with it, then it
> could be moved to core.

Yeah, it is very loosely coupled to the rest of Git by design. I
posted the first version of this series around last November. So
people have had a very long time to review it, comment on it,
experiment with it, bikeshed many details... And in the same time many
users have asked for some features that "git interpret-trailers"
provides...

Regards,
Christian.

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Michael Haggerty]

tl;dr: This patch series wants to introduce a permanent new Git data
format.  The current version can write trailers in formats that it is
incapable of reading, which I consider broken.  I advocate a stricter
specification of the format of trailers, at least until we get feedback
from users that they need more flexibility.

On 05/25/2014 10:37 AM, Christian Couder wrote:
> From: Michael Haggerty <mhagger@alum.mit.edu>
> [...]
>> * A way for users to add information to commits for whatever purpose
>>   they want, without having to convince upstream to built support in.
> 
> Yeah, I agree this is the main purpose of trailers.
> 
>> * A standard format for that information, so that all tools can agree
>>   how to read/write trailers without being confused by or breaking
>>   trailers that they didn't know about in advance.
> 
> Yeah, but don't you think this goal can sometimes go against the
> previous goal?
> 
> I mean, if some users for their project think that it's better, for
> example, if they use trailers like "Fix #42" instead of "Fix: 42",
> because their bug tracking system supports "Fix #42" better, we should
> let them do what suits them better, even if Git supports them not as
> well as if they used "Fix: 42".

The flexibility that comes from offering our users a more-or-less
general key/value store already accomplishes the first goal.  With that
the users *could* store their data as "Fix: 42" or "Fixes: #42" and
satisfy their functional requirements.

Giving them the option to use "Fix #42" doesn't make *any* new
functionality possible.  It is pure eye-candy.  And it would come at the
IMO high cost of making it harder for *everybody* to work with the
metadata.  It makes the specification more complicated.  It makes the
code more complicated.  It makes the configuration more complicated.  It
makes it more likely that there will be "false positives" (text in a
commit message that our code recognizes as key/value data even though it
was not meant to be).  And in my opinion it makes the k/v data itself
harder for a human to read because it is not in a uniform format.

The only justification I can think of for allowing more flexible formats
would be to "retroactively" support metadata that people already have in
their history.  Are there "famous" or "important" existing metadata
formats that are incompatible with "Key: Value"?

More to the point: do you have a concrete reason for wanting to support
alternative formats like "Fix #42", or is it based more on the feeling
that users will want it?

Remember, it would be really easy to release v1 of this feature with a
strict format, then wait and see if users clamor for more flexibility.
We can always add more flexibility later.  Whereas if v1 already
supports more flexible formats, we pretty much have to support them forever.

>> * A format that is straightforward enough that it can be machine-
>>   readable with minimum ambiguity.
> 
> Yeah, but again this could go against the main purpose of trailers
> above.

No, the users have all the flexibility they need if they can choose
their own key/value schema.  Allowing alternative formats adds very little.

I feel strongly that it would be a bad mistake to leave the
specification of trailers so loose that they cannot be machine-readable
with a good degree of confidence.  Tools that add trailers should be
composable.  The current scheme is not.  For example, suppose one tool
wants to add a "Fix #42" line and another one wants to add "Signed-off-by":

    git config trailer.fix.key "Fix #"
    git config trailer.sign.key "Signed-off-by: "
    git config trailer.sign.ifexists doNothing

    echo "subject

    Signed-off-by: Alice <alice@example.com>" |
        git interpret-trailers --trailer fix=42 |
        git interpret-trailers --trailer sign="Bob <bob@example.com>"
    --------- output ------------------------------------------
    subject

    Signed-off-by: Alice <alice@example.com>
    Fix #42

    Signed-off-by: Bob <bob@example.com>
    -----------------------------------------------------------

The result is that the trailers end up not in one block but in two
(meaning that the first block is no longer recognized as a trailer
block), and the second "Signed-off-by" line, which should have been
omitted because of ifexists=doNothing, was incorrectly added.

Or let's do something like the "commit template" example from the
documentation, but using "Fix #" instead of "Fixes: ":

    echo "***subject***

    ***message***

    Fix #
    Cc:
    Reviewed-by:
    Signed-off-by:
    " |
        sed -Ee 's/(Reviewed-by.*)/\1me/' |
        git interpret-trailers --trim-empty --trailer "git-version: foo"
    ---------- output ------------------------------------------
    ***subject***

    ***message***

    Fix #
    Cc:
    Reviewed-by: me
    Signed-off-by:

    git-version: foo
    ------------------------------------------------------------

Not only haven't the empty lines been stripped off, but a new trailer
block has been created for "git-version".

I consider this broken.

> [...]
>> For example the specification
>> might look something like:
>>
>>     A commit message can optionally end with a block of trailers.
>>     The trailers, if present, must be separated from the rest of the
>>     commit message by one or more blank lines (lines that contain only
>>     whitespace).  There must be no blank lines within the list of
>>     trailers.  It is allowed to have blank lines after the trailers.
>>
>>     Each trailer line must match the following Perl regular
>>     expression:
>>
>>         ^([A-Za-z0-9_-]+)\s*:\s*(.*[^\s])\s*$
>>
>>     The string matching the first group is called the key and the string
>>     matching the second is called the value.  Keys are considered to be
>>     case-insensitive [or should they be case-sensitive?].  The
>>     interpretation of values is left entirely up to the application.
>>     Values must not be empty.
> 
> I tried to be clearer in the v12 I just posted, and I think it should
> be enough to be very clear. We might want to tweak a little the
> specifications later, so being too strict might be counter productive.
> 
> And as other tools might already use trailers in a case-sensitive way
> and yet other tools in a case-insensitive way, I am not sure we would
> gain anything by specifying if keys or values should be interpreted in
> a case-sensitive or case-insensitive way. On the contrary we might
> upset people already using some of these tools for no good reason.

No sane tool would interpret a trailer differently depending on how the
key is capitalized.  So I think that if we ourselves treat the keys
case-insensitively but we preserve case when processing metadata, there
won't be any problems.

>>     However, in --draft and --cleanup modes, empty values *are*
>>     allowed, as are comments (lines starting with `core.commentchar`)
>>     within the trailer block.  In --draft mode such lines are passed
>>     through unchanged, and in --cleanup mode such lines are removed.
> 
> I am not sure we should use modes. I think options like
> "--trim-empty", "--allow-comments", "--allow-empty" might be clearer.

I think we want to train users to think of trailers-with-no-values as
temporary helpers that shouldn't end up in commits, just like the
"#"-commented lines in commit message templates.  Why?  Because you
can't rely on their being preserved.  As soon as your trailer goes
through a "--cleanup" (which might be there because of an unrelated
tool) it will disappear.  For the user it is a simpler mental model to
think of modes: as long as I am in draft mode, there might be comment
lines and trailer templates in my commit message, but when I commit they
will go away.  I would go so far as to say that deleting trailer lines
with no values should be a standard part of cleaning commit messages and
should maybe be an option offered by "git stripspace".

> [...]
>> So my preference would be to make the format of trailers really strict;
>> for example, only allowing colon separators as in the regexp above.
>> People who want to work with trailers using Git tools will just have to
>> conform to this format.
> 
> I don't think we should cast in stone the format for trailers, because
> of the main purpose of trailers.
> 
> The format of the commit header for example is cast in stone, but
> that's ok because it is mostly for Git internal use. Trailers are
> mostly for external use by users who already have tools expecting
> different formats.
> 
> There are already users who are not happy that they cannot easily have
> other commit headers, and we point them to trailers. If we specify
> trailers too strictly, where will we point them to?

Nobody is disagreeing that users should be allowed to choose their own
keys and values and assign their own interpretations to them.  If we let
users add their own commit headers, we certainly wouldn't let them
define a header formatted like "Fix #42", would we?

Again, the *only* justification for more flexible formats would be if
there are a lot of tools that *already* exist out there that don't
conform to "Key: value".  Are there?  New tools will certainly be
written to use whatever format we define, and in exchange they will be
able to use your awesome new tool and hopefully other upcoming tools for
dealing with trailers.

>> But if we must support flexibility in the separator characters, then I
>> think it is important that we be able to read whatever we can write.
> 
> An option like --input-separator might be enough to support this.

This would not be adequate because a single commit message might have
multiple trailers with different formats:

    Signed-off-by: me
    Fix #42

Either the input separator would have to be specified for every single
trailer (which is impractical because you can't dictate centrally how
git clients are configured) or the parsing code would have to be taught
to read any allowed format.

>> For me this means:
>>
>> * Enumerating a list of allowed separators (e.g., [:=#])
> 
> Junio suggested in a message that users might use different separators
> like '%'.

The fewer and stricter the better, to avoid false positive matches to
non-trailer text.

>> * Specifying how it is decided what separator to use when generating
>>   new trailers
> 
> This is already possible with the 'trailer.<token>.key' config
> variable.

This means that if one developer has forgotten to configure the "Fix"
trailer in one clone, then he will generate trailers that are considered
malformed by a colleague who has configured "Fix #".

>> * Specifying what appends when a trailer is read and then written again:
>>   is its separator preserved, or is the trailer converted to use the
>>   separator configured for that particular key in the config.  And if
>>   the latter, what happens if a key's separator is not configured?
> 
> Right now we only accept ':' as input separator for the messages and
> ':' and '=' for the --trailer option, and the default output separator
> is ':'. If the user specify a different separator in a key, this
> separator will be used only in the output for this key.

I consider trailers that can only be written but not read to be broken.
 My questions are to probe one possible alternate reality, namely:
"Suppose we want to allow alternative trailer formats.  What would it
take to make them readable *and* writable?"

For example:

* What if I try to add a Signed-off-by trailer to a message that already
contains "Fix #42"?  In what form should the "Fix #42" line be written
to the output

  * if I don't have the separator for "Fix" configured?
  * if I have the separator for "Fix" configured to be "="?

* What if I *do* have the separator for "Fix" set to " #", but the input
contains "Fix: 42"?  How should that line be formatted on output?  Does
the answer change if I have "token.fix.ifexist" set to "overwrite" and
run "git interpret-trailers fix=43"?

By asking these questions, I hope to hint that supporting alternative
formats increases the complexity of the specification and the
implementation to an unjustifiable extent and/or it unrealistically
relies all developers on a project having their trailer configuration
set up correctly.

> If this is not clear in the documentation, please susggest specific
> improvements.
> [...]
> I tried to be very clear in the doc in v12.

The doc only covers writing new trailers in "alternative" formats, not
reading them.

> [...]
>> Given Git's requirements for backwards compatibility, a specification
>> that we release now will have to be supported forever (because it will
>> be baked into commits and can *never* be changed), and any
>> trailer-handling tools that we release now will have to be supported for
>> years (until at least Git 3.0).
> 
> Yeah, I know that. So if we are too strict in the specification will
> be stuck for a long time.

No, that's exactly backwards!  We can easily loosen the format in the
future.  Projects don't need to output trailers in the looser format
until everybody involved is using the new Git version.

But the format can never be made *more* strict, because then the
metadata that users have added to their history will stop being readable.

>> All in all, I think that there has been a lot of discussion about the
>> interface of this one command, "git interpret-trailers", including its
>> quite complicated configuration and a command-line behavior.  And yet it
>> seems to me that not many Git developers have been very engaged in the
>> conversation, and Junio (who has) still doesn't seem satisfied with it.
>>  I (though among the too-little engaged) have the feeling that it is
>> still a ways from maturity.
> 
> My opinion is that many Git developers have been engaged and you can
> see that in the Cc.
> 
> I cannot tell if they are all very happy or not but I suppose that if
> they were very unhappy they would tell it.
> [...]

It was unfair of me to try to characterize the opinions of other
developers.  On the other hand, even though many people have commented
on this proposal over its long lifetime, I didn't get the feeling that
it has won a consensus of +1s in its current form.

I'd love to hear the opinion of others because maybe I'm totally out in
left field.

And I want to reiterate that the reason I'm so emphatic about this
proposal is because I think it will be such a great new feature.  I just
think that some tweaks would make it a more solid foundation for
building even more functionality onto.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Johan Herland]

On Tue, May 27, 2014 at 10:21 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
> tl;dr: This patch series wants to introduce a permanent new Git data
> format.  The current version can write trailers in formats that it is
> incapable of reading, which I consider broken.  I advocate a stricter
> specification of the format of trailers, at least until we get feedback
> from users that they need more flexibility.
>
> On 05/25/2014 10:37 AM, Christian Couder wrote:

[...]

>> My opinion is that many Git developers have been engaged and you can
>> see that in the Cc.
>>
>> I cannot tell if they are all very happy or not but I suppose that if
>> they were very unhappy they would tell it.
>> [...]
>
> It was unfair of me to try to characterize the opinions of other
> developers.  On the other hand, even though many people have commented
> on this proposal over its long lifetime, I didn't get the feeling that
> it has won a consensus of +1s in its current form.
>
> I'd love to hear the opinion of others because maybe I'm totally out in
> left field.

FWIW, after a quick read, I find myself agreeing very much with
Michael's arguments for a stricter format (at least in its initial
version).

We are formalizing and applying tools/automation to a part of the
commit message that has so far been ad hoc and very informal. There is
no expectation that _every_ _single_ existing use of (informal)
trailers (except the somewhat-formalized support for --signoff) must
be supported by git-interpret-trailers.

However, there _is_ an expectation that git-interpret-trailers is
self-consistent and does not stumble over its own trailers. Therefore,
it makes perfect sense to make v1 very strict in what formats it
produce (i.e. a strict "key: value" format is enough for now).

> And I want to reiterate that the reason I'm so emphatic about this
> proposal is because I think it will be such a great new feature.  I just
> think that some tweaks would make it a more solid foundation for
> building even more functionality onto.

Fully agreed. git-interpret-trailers have come up in several other
discussion, both on the git mailing list and elsewhere, and I have no
doubt that this will be a very useful feature that will be put to good
use in many projects.


...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: [PATCH v10 11/12] Documentation: add documentation for 'git interpret-trailers'

[Junio C Hamano]

Christian Couder <chriscool@tuxfamily.org> writes:

> From: Michael Haggerty <mhagger@alum.mit.edu>
> ...
> An option like --input-separator might be enough to support this.
>
>> For me this means:
>> 
>> * Enumerating a list of allowed separators (e.g., [:=#])
>
> Junio suggested in a message that users might use different separators
> like '%'.

I actually think we shouldn't go any fancier than ":" and nothing
else, not even "#".

I was hoping that you would eventually realize that there are only
two viable extremes when I suggested "the users may want to use
other random characters like '%'" and also "the users can specify
the 'key' with colon and trailing SP" (in $gmane/245960).

 - If you want to give the projects greater control of the format,
   then you cannot rely on "separators" anyway.  Your users can list
   all possible footer "keys" the particular project would use, so
   that they are recognized by Git, be that "Fixes: 4a28f16", "Bug
   #12354", without hard-coding what "separator" Git must pay
   attention to.  You can easily find a run of lines that begin with
   any of the "key" (e.g. "Fixes: ", "Signed-off-by: ", "Bug #",
   ...) starting from the tail-end of the log message and that is
   your footer block.  No need for "separators" at all.

 - If you want to give the projects freedom to come up with random
   new kinds of footers without pre-arrangement, then you need to
   have a reliable way to say if any line you have never seen could
   be a footer material.  A colon has been used everywhere, and used
   even in the "Fixes: 4a28f16" example you took from the kernel
   circle.  I think you presented it with '#' but I do not think
   they even want that, looking at:

   http://lists.linuxfoundation.org/pipermail/ksummit-discuss/2014-May/000618.html

I also think that bug tracking system using "Bug #12345" is an
unrelated issue, as log viewers would want to highlight and make
links out of them anywhere in the log message text, not limited
to the log footer part.

As to which one of these two we should take, I tend to think that we
should start small and limited; loosening the syntax later is much
easier than going the other way, i.e. ":" and nothing else.