All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/8] rebase -i: offer to recreate merge commits
@ 2018-01-18 15:35 Johannes Schindelin
  2018-01-18 15:35 ` [PATCH 1/8] sequencer: introduce new commands to reset the revision Johannes Schindelin
                   ` (12 more replies)
  0 siblings, 13 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 15:35 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Jacob Keller

Once upon a time, I dreamt of an interactive rebase that would not
flatten branch structure, but instead recreate the commit topology
faithfully.

My original attempt was --preserve-merges, but that design was so
limited that I did not even enable it in interactive mode.

Subsequently, it *was* enabled in interactive mode, with the predictable
consequences: as the --preserve-merges design does not allow for
specifying the parents of merge commits explicitly, all the new commits'
parents are defined *implicitly* by the previous commit history, and
hence it is *not possible to even reorder commits*.

This design flaw cannot be fixed. Not without a complete re-design, at
least. This patch series offers such a re-design.

Think of --recreate-merges as "--preserve-merges done right". It
introduces new verbs for the todo list, `label`, `reset` and `merge`.
For a commit topology like this:

            A - B - C
              \   /
                D

the generated todo list would look like this:

            # branch D
            pick 0123 A
            label branch-point
            pick 1234 D
            label D

            reset branch-point
            pick 2345 B
            merge 3456 D C

There are more patches in the pipeline, based on this patch series, but
left for later in the interest of reviewable patch series: one mini
series to use the sequencer even for `git rebase -i --root`, and another
one to add support for octopus merges to --recreate-merges.


Johannes Schindelin (8):
  sequencer: introduce new commands to reset the revision
  sequencer: introduce the `merge` command
  sequencer: fast-forward merge commits, if possible
  rebase-helper --make-script: introduce a flag to recreate merges
  rebase: introduce the --recreate-merges option
  sequencer: handle autosquash and post-rewrite for merge commands
  pull: accept --rebase=recreate to recreate the branch topology
  rebase -i: introduce --recreate-merges=no-rebase-cousins

 Documentation/config.txt               |   8 +
 Documentation/git-pull.txt             |   5 +-
 Documentation/git-rebase.txt           |  13 +-
 builtin/pull.c                         |  14 +-
 builtin/rebase--helper.c               |  13 +-
 builtin/remote.c                       |   2 +
 contrib/completion/git-completion.bash |   4 +-
 git-rebase--interactive.sh             |   6 +
 git-rebase.sh                          |  16 +
 refs.c                                 |   3 +-
 sequencer.c                            | 697 ++++++++++++++++++++++++++++++++-
 sequencer.h                            |   9 +
 t/t3430-rebase-recreate-merges.sh      | 208 ++++++++++
 13 files changed, 977 insertions(+), 21 deletions(-)
 create mode 100755 t/t3430-rebase-recreate-merges.sh


base-commit: 2512f15446149235156528dafbe75930c712b29e
Published-As: https://github.com/dscho/git/releases/tag/recreate-merges-v1
Fetch-It-Via: git fetch https://github.com/dscho/git recreate-merges-v1
-- 
2.15.1.windows.2.1430.ga56c4f9e2a9


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
@ 2018-01-18 15:35 ` Johannes Schindelin
  2018-01-18 16:25   ` Jacob Keller
                     ` (2 more replies)
  2018-01-18 15:35 ` [PATCH 2/8] sequencer: introduce the `merge` command Johannes Schindelin
                   ` (11 subsequent siblings)
  12 siblings, 3 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 15:35 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Jacob Keller

In the upcoming commits, we will teach the sequencer to recreate merges.
This will be done in a very different way from the unfortunate design of
`git rebase --preserve-merges` (which does not allow for reordering
commits, or changing the branch topology).

The main idea is to introduce new todo list commands, to support
labeling the current revision with a given name, resetting the current
revision to a previous state, merging labeled revisions.

This idea was developed in Git for Windows' Git garden shears (that are
used to maintain the "thicket of branches" on top of upstream Git), and
this patch is part of the effort to make it available to a wider
audience, as well as to make the entire process more robust (by
implementing it in a safe and portable language rather than a Unix shell
script).

This commit implements the commands to label, and to reset to, given
revisions. The syntax is:

	label <name>
	reset <name>

As a convenience shortcut, also to improve readability of the generated
todo list, a third command is introduced: bud. It simply resets to the
"onto" revision, i.e. the commit onto which we currently rebase.

Internally, the `label <name>` command creates the ref
`refs/rewritten/<name>`. This makes it possible to work with the labeled
revisions interactively, or in a scripted fashion (e.g. via the todo
list command `exec`).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   3 +
 sequencer.c                | 181 ++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 180 insertions(+), 4 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index d47bd29593a..3d2cd19d65a 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -162,6 +162,9 @@ s, squash = use commit, but meld into previous commit
 f, fixup = like \"squash\", but discard this commit's log message
 x, exec = run command (the rest of the line) using shell
 d, drop = remove commit
+l, label = label current HEAD with a name
+t, reset = reset HEAD to a label
+b, bud = reset HEAD to the revision labeled 'onto'
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index 4d3f60594cb..91cc55a002f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -21,6 +21,8 @@
 #include "log-tree.h"
 #include "wt-status.h"
 #include "hashmap.h"
+#include "unpack-trees.h"
+#include "worktree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -116,6 +118,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
 static GIT_PATH_FUNC(rebase_path_rewritten_pending,
 	"rebase-merge/rewritten-pending")
+
+/*
+ * The path of the file listing refs that need to be deleted after the rebase
+ * finishes. This is used by the `merge` command.
+ */
+static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
+
 /*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
@@ -767,6 +776,9 @@ enum todo_command {
 	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
+	TODO_LABEL,
+	TODO_RESET,
+	TODO_BUD,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -785,6 +797,9 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
+	{ 'l', "label" },
+	{ 't', "reset" },
+	{ 'b', "bud" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1253,7 +1268,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
 			item->command = i;
 			break;
-		} else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
+		} else if ((bol + 1 == eol || bol[1] == ' ') &&
+			   *bol == todo_command_info[i].c) {
 			bol++;
 			item->command = i;
 			break;
@@ -1265,7 +1281,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	padding = strspn(bol, " \t");
 	bol += padding;
 
-	if (item->command == TODO_NOOP) {
+	if (item->command == TODO_NOOP || item->command == TODO_BUD) {
 		if (bol != eol)
 			return error(_("%s does not accept arguments: '%s'"),
 				     command_to_string(item->command), bol);
@@ -1279,7 +1295,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return error(_("missing arguments for %s"),
 			     command_to_string(item->command));
 
-	if (item->command == TODO_EXEC) {
+	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
+	    item->command == TODO_RESET) {
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = (int)(eol - bol);
@@ -1919,6 +1936,139 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int safe_append(const char *filename, const char *fmt, ...)
+{
+	va_list ap;
+	struct lock_file lock = LOCK_INIT;
+	int fd = hold_lock_file_for_update(&lock, filename, 0);
+	struct strbuf buf = STRBUF_INIT;
+
+	if (fd < 0)
+		return error_errno(_("could not lock '%s'"), filename);
+
+	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT)
+		return error_errno(_("could not read '%s'"), filename);
+	strbuf_complete(&buf, '\n');
+	va_start(ap, fmt);
+	strbuf_vaddf(&buf, fmt, ap);
+	va_end(ap);
+
+	if (write_in_full(fd, buf.buf, buf.len) < 0) {
+		rollback_lock_file(&lock);
+		return error_errno(_("could not write to '%s'"), filename);
+	}
+	if (commit_lock_file(&lock) < 0) {
+		rollback_lock_file(&lock);
+		return error(_("failed to finalize '%s'."), filename);
+	}
+
+	return 0;
+}
+
+static int do_label(const char *name, int len)
+{
+	struct ref_store *refs = get_main_ref_store();
+	struct ref_transaction *transaction;
+	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
+	struct strbuf msg = STRBUF_INIT;
+	int ret = 0;
+	struct object_id head_oid;
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	strbuf_addf(&msg, "label '%.*s'", len, name);
+
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction ||
+	    get_oid("HEAD", &head_oid) ||
+	    ref_transaction_update(transaction, ref_name.buf, &head_oid, NULL,
+				   0, msg.buf, &err) < 0 ||
+	    ref_transaction_commit(transaction, &err)) {
+		error("%s", err.buf);
+		ret = -1;
+	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
+	strbuf_release(&msg);
+
+	if (!ret)
+		ret = safe_append(rebase_path_refs_to_delete(),
+				  "%s\n", ref_name.buf);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
+static int do_reset(const char *name, int len)
+{
+	struct strbuf ref_name = STRBUF_INIT;
+	struct object_id oid;
+	struct lock_file lock = LOCK_INIT;
+	struct tree_desc desc;
+	struct tree *tree;
+	struct unpack_trees_options opts;
+	int ret = 0, i;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	for (i = 0; i < len; i++)
+		if (isspace(name[i]))
+			len = i;
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	if (get_oid(ref_name.buf, &oid) &&
+	    get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
+		error(_("could not read '%s'"), ref_name.buf);
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	memset(&opts, 0, sizeof(opts));
+	opts.head_idx = 1;
+	opts.src_index = &the_index;
+	opts.dst_index = &the_index;
+	opts.fn = oneway_merge;
+	opts.merge = 1;
+	opts.update = 1;
+	opts.reset = 1;
+
+	read_cache_unmerged();
+	if (!fill_tree_descriptor(&desc, &oid)) {
+		error(_("Failed to find tree of %s."), oid_to_hex(&oid));
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	if (unpack_trees(1, &desc, &opts)) {
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	tree = parse_tree_indirect(&oid);
+	prime_cache_tree(&the_index, tree);
+
+	if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
+		ret = error(_("could not write index"));
+	free((void *)desc.buffer);
+
+	if (!ret) {
+		struct strbuf msg = STRBUF_INIT;
+
+		strbuf_addf(&msg, "(rebase -i) reset '%.*s'", len, name);
+		ret = update_ref(msg.buf, "HEAD", &oid, NULL, 0,
+				 UPDATE_REFS_MSG_ON_ERR);
+		strbuf_release(&msg);
+	}
+
+	strbuf_release(&ref_name);
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2102,7 +2252,13 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				/* `current` will be incremented below */
 				todo_list->current = -1;
 			}
-		} else if (!is_noop(item->command))
+		} else if (item->command == TODO_LABEL)
+			res = do_label(item->arg, item->arg_len);
+		else if (item->command == TODO_RESET)
+			res = do_reset(item->arg, item->arg_len);
+		else if (item->command == TODO_BUD)
+			res = do_reset("onto", 4);
+		else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
@@ -2207,6 +2363,23 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		}
 		apply_autostash(opts);
 
+		strbuf_reset(&buf);
+		if (strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0)
+		    > 0) {
+			char *p = buf.buf;
+			while (*p) {
+				char *eol = strchr(p, '\n');
+				if (eol)
+					*eol = '\0';
+				if (delete_ref("(rebase -i) cleanup",
+					       p, NULL, 0) < 0)
+					warning(_("could not delete '%s'"), p);
+				if (!eol)
+					break;
+				p = eol + 1;
+			}
+		}
+
 		fprintf(stderr, "Successfully rebased and updated %s.\n",
 			head_ref.buf);
 
-- 
2.15.1.windows.2.1430.ga56c4f9e2a9



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH 2/8] sequencer: introduce the `merge` command
  2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
  2018-01-18 15:35 ` [PATCH 1/8] sequencer: introduce new commands to reset the revision Johannes Schindelin
@ 2018-01-18 15:35 ` Johannes Schindelin
  2018-01-18 16:31   ` Jacob Keller
                     ` (3 more replies)
  2018-01-18 15:35 ` [PATCH 3/8] sequencer: fast-forward merge commits, if possible Johannes Schindelin
                   ` (10 subsequent siblings)
  12 siblings, 4 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 15:35 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Jacob Keller

This patch is part of the effort to reimplement `--preserve-merges` with
a substantially improved design, a design that has been developed in the
Git for Windows project to maintain the dozens of Windows-specific patch
series on top of upstream Git.

The previous patch implemented the `label`, `bud` and `reset` commands
to label commits and to reset to a labeled commits. This patch adds the
`merge` command, with the following syntax:

	merge <commit> <rev> <oneline>

The <commit> parameter in this instance is the *original* merge commit,
whose author and message will be used for the to-be-created merge
commit.

The <rev> parameter refers to the (possibly rewritten) revision to
merge. Let's see an example of a todo list:

	label onto

	# Branch abc
	bud
	pick deadbeef Hello, world!
	label abc

	bud
	pick cafecafe And now for something completely different
	merge baaabaaa abc Merge the branch 'abc' into master

To support creating *new* merges, i.e. without copying the commit
message from an existing commit, use the special value `-` as <commit>
parameter (in which case the text after the <rev> parameter is used as
commit message):

	merge - abc This will be the actual commit message of the merge

This comes in handy when splitting a branch into two or more branches.

Note: this patch only adds support for recursive merges, to keep things
simple. Support for octopus merges will be added later in this patch
series, support for merges using strategies other than the recursive
merge is left for future contributions.

The design of the `merge` command as introduced by this patch only
supports creating new merge commits with exactly two parents, i.e. it
adds no support for octopus merges.

We will introduce support for octopus merges in a later commit.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   1 +
 sequencer.c                | 146 +++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 143 insertions(+), 4 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 3d2cd19d65a..5bf1ea3781f 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -165,6 +165,7 @@ d, drop = remove commit
 l, label = label current HEAD with a name
 t, reset = reset HEAD to a label
 b, bud = reset HEAD to the revision labeled 'onto'
+m, merge = create a merge commit using a given commit's message
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index 91cc55a002f..567cfcbbe8b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -779,6 +779,7 @@ enum todo_command {
 	TODO_LABEL,
 	TODO_RESET,
 	TODO_BUD,
+	TODO_MERGE,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -800,6 +801,7 @@ static struct {
 	{ 'l', "label" },
 	{ 't', "reset" },
 	{ 'b', "bud" },
+	{ 'm', "merge" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1304,14 +1306,20 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	}
 
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
+	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
+	item->arg_len = (int)(eol - item->arg);
+
 	saved = *end_of_object_name;
+	if (item->command == TODO_MERGE && *bol == '-' &&
+	    bol + 1 == end_of_object_name) {
+		item->commit = NULL;
+		return 0;
+	}
+
 	*end_of_object_name = '\0';
 	status = get_oid(bol, &commit_oid);
 	*end_of_object_name = saved;
 
-	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
-	item->arg_len = (int)(eol - item->arg);
-
 	if (status < 0)
 		return -1;
 
@@ -2069,6 +2077,132 @@ static int do_reset(const char *name, int len)
 	return ret;
 }
 
+static int do_merge(struct commit *commit, const char *arg, int arg_len,
+		    struct replay_opts *opts)
+{
+	int merge_arg_len;
+	struct strbuf ref_name = STRBUF_INIT;
+	struct commit *head_commit, *merge_commit, *i;
+	struct commit_list *common, *j, *reversed = NULL;
+	struct merge_options o;
+	int ret;
+	static struct lock_file lock;
+
+	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
+		if (isspace(arg[merge_arg_len]))
+			break;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	if (commit) {
+		const char *message = get_commit_buffer(commit, NULL);
+		const char *body;
+		int len;
+
+		if (!message) {
+			rollback_lock_file(&lock);
+			return error(_("could not get commit message of '%s'"),
+				     oid_to_hex(&commit->object.oid));
+		}
+		write_author_script(message);
+		find_commit_subject(message, &body);
+		len = strlen(body);
+		if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
+			error_errno(_("Could not write '%s'"),
+				    git_path_merge_msg());
+			unuse_commit_buffer(commit, message);
+			rollback_lock_file(&lock);
+			return -1;
+		}
+		unuse_commit_buffer(commit, message);
+	} else {
+		const char *p = arg + merge_arg_len;
+		struct strbuf buf = STRBUF_INIT;
+		int len;
+
+		strbuf_addf(&buf, "author %s", git_author_info(0));
+		write_author_script(buf.buf);
+		strbuf_reset(&buf);
+
+		p += strspn(p, " \t");
+		if (*p)
+			len = strlen(p);
+		else {
+			strbuf_addf(&buf, "Merge branch '%.*s'",
+				    merge_arg_len, arg);
+			p = buf.buf;
+			len = buf.len;
+		}
+
+		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
+			error_errno(_("Could not write '%s'"),
+				    git_path_merge_msg());
+			strbuf_release(&buf);
+			rollback_lock_file(&lock);
+			return -1;
+		}
+		strbuf_release(&buf);
+	}
+
+	head_commit = lookup_commit_reference_by_name("HEAD");
+	if (!head_commit) {
+		rollback_lock_file(&lock);
+		return error(_("Cannot merge without a current revision"));
+	}
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
+	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	if (!merge_commit) {
+		/* fall back to non-rewritten ref or commit */
+		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
+		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	}
+	if (!merge_commit) {
+		error(_("could not resolve '%s'"), ref_name.buf);
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
+		      git_path_merge_head(), 0);
+	write_message("no-ff", 5, git_path_merge_mode(), 0);
+
+	common = get_merge_bases(head_commit, merge_commit);
+	for (j = common; j; j = j->next)
+		commit_list_insert(j->item, &reversed);
+	free_commit_list(common);
+
+	read_cache();
+	init_merge_options(&o);
+	o.branch1 = "HEAD";
+	o.branch2 = ref_name.buf;
+	o.buffer_output = 2;
+
+	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
+	if (ret <= 0)
+		fputs(o.obuf.buf, stdout);
+	strbuf_release(&o.obuf);
+	if (ret < 0) {
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return error(_("conflicts while merging '%.*s'"),
+			     merge_arg_len, arg);
+	}
+
+	if (active_cache_changed &&
+	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
+		strbuf_release(&ref_name);
+		return error(_("merge: Unable to write new index file"));
+	}
+	rollback_lock_file(&lock);
+
+	ret = run_git_commit(git_path_merge_msg(), opts, 0);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2258,6 +2392,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_reset(item->arg, item->arg_len);
 		else if (item->command == TODO_BUD)
 			res = do_reset("onto", 4);
+		else if (item->command == TODO_MERGE)
+			res = do_merge(item->commit,
+				       item->arg, item->arg_len, opts);
 		else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
@@ -2757,7 +2894,8 @@ int transform_todos(unsigned flags)
 					  oid_to_hex(&item->commit->object.oid);
 
 			strbuf_addf(&buf, " %s", oid);
-		}
+		} else if (item->command == TODO_MERGE)
+			strbuf_addstr(&buf, " -");
 		/* add all the rest */
 		if (!item->arg_len)
 			strbuf_addch(&buf, '\n');
-- 
2.15.1.windows.2.1430.ga56c4f9e2a9



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH 3/8] sequencer: fast-forward merge commits, if possible
  2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
  2018-01-18 15:35 ` [PATCH 1/8] sequencer: introduce new commands to reset the revision Johannes Schindelin
  2018-01-18 15:35 ` [PATCH 2/8] sequencer: introduce the `merge` command Johannes Schindelin
@ 2018-01-18 15:35 ` Johannes Schindelin
  2018-01-19 14:53   ` Phillip Wood
  2018-01-23 18:51   ` Junio C Hamano
  2018-01-18 15:35 ` [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
                   ` (9 subsequent siblings)
  12 siblings, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 15:35 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Jacob Keller

Just like with regular `pick` commands, if we are trying to recreate a
merge commit, we now test whether the parents of said commit match HEAD
and the commits to be merged, and fast-forward if possible.

This is not only faster, but also avoids unnecessary proliferation of
new objects.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 567cfcbbe8b..a96255426e7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2085,7 +2085,7 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	struct commit *head_commit, *merge_commit, *i;
 	struct commit_list *common, *j, *reversed = NULL;
 	struct merge_options o;
-	int ret;
+	int can_fast_forward, ret;
 	static struct lock_file lock;
 
 	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
@@ -2151,6 +2151,14 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		return error(_("Cannot merge without a current revision"));
 	}
 
+	/*
+	 * If HEAD is not identical to the parent of the original merge commit,
+	 * we cannot fast-forward.
+	 */
+	can_fast_forward = commit && commit->parents &&
+		!oidcmp(&commit->parents->item->object.oid,
+			&head_commit->object.oid);
+
 	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
 	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
 	if (!merge_commit) {
@@ -2164,6 +2172,17 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		rollback_lock_file(&lock);
 		return -1;
 	}
+
+	if (can_fast_forward && commit->parents->next &&
+	    !commit->parents->next->next &&
+	    !oidcmp(&commit->parents->next->item->object.oid,
+		    &merge_commit->object.oid)) {
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return fast_forward_to(&commit->object.oid,
+				       &head_commit->object.oid, 0, opts);
+	}
+
 	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
 		      git_path_merge_head(), 0);
 	write_message("no-ff", 5, git_path_merge_mode(), 0);
-- 
2.15.1.windows.2.1430.ga56c4f9e2a9



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges
  2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
                   ` (2 preceding siblings ...)
  2018-01-18 15:35 ` [PATCH 3/8] sequencer: fast-forward merge commits, if possible Johannes Schindelin
@ 2018-01-18 15:35 ` Johannes Schindelin
  2018-01-18 21:39   ` Philip Oakley
                     ` (2 more replies)
  2018-01-18 15:35 ` [PATCH 5/8] rebase: introduce the --recreate-merges option Johannes Schindelin
                   ` (8 subsequent siblings)
  12 siblings, 3 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 15:35 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Jacob Keller

The sequencer just learned a new commands intended to recreate branch
structure (similar in spirit to --preserve-merges, but with a
substantially less-broken design).

Let's allow the rebase--helper to generate todo lists making use of
these commands, triggered by the new --recreate-merges option. For a
commit topology like this:

	A - B - C
	  \   /
	    D

the generated todo list would look like this:

	# branch D
	pick 0123 A
	label branch-point
	pick 1234 D
	label D

	reset branch-point
	pick 2345 B
	merge 3456 D C

To keep things simple, we first only implement support for merge commits
with exactly two parents, leaving support for octopus merges to a later
patch in this patch series.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/rebase--helper.c |   4 +-
 sequencer.c              | 343 ++++++++++++++++++++++++++++++++++++++++++++++-
 sequencer.h              |   1 +
 3 files changed, 345 insertions(+), 3 deletions(-)

diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index 7daee544b7b..a34ab5c0655 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -12,7 +12,7 @@ static const char * const builtin_rebase_helper_usage[] = {
 int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
-	unsigned flags = 0, keep_empty = 0;
+	unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
 	int abbreviate_commands = 0;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
@@ -22,6 +22,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 	struct option options[] = {
 		OPT_BOOL(0, "ff", &opts.allow_ff, N_("allow fast-forward")),
 		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
+		OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge commits")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -55,6 +56,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
+	flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
 	if (command == CONTINUE && argc == 1)
diff --git a/sequencer.c b/sequencer.c
index a96255426e7..1bef16647b4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -23,6 +23,8 @@
 #include "hashmap.h"
 #include "unpack-trees.h"
 #include "worktree.h"
+#include "oidmap.h"
+#include "oidset.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -2785,6 +2787,335 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
 	strbuf_release(&sob);
 }
 
+struct labels_entry {
+	struct hashmap_entry entry;
+	char label[FLEX_ARRAY];
+};
+
+static int labels_cmp(const void *fndata, const struct labels_entry *a,
+		      const struct labels_entry *b, const void *key)
+{
+	return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
+}
+
+struct string_entry {
+	struct oidmap_entry entry;
+	char string[FLEX_ARRAY];
+};
+
+struct label_state {
+	struct oidmap commit2label;
+	struct hashmap labels;
+	struct strbuf buf;
+};
+
+static const char *label_oid(struct object_id *oid, const char *label,
+			     struct label_state *state)
+{
+	struct labels_entry *labels_entry;
+	struct string_entry *string_entry;
+	struct object_id dummy;
+	size_t len;
+	int i;
+
+	string_entry = oidmap_get(&state->commit2label, oid);
+	if (string_entry)
+		return string_entry->string;
+
+	/*
+	 * For "uninteresting" commits, i.e. commits that are not to be
+	 * rebased, and which can therefore not be labeled, we use a unique
+	 * abbreviation of the commit name. This is slightly more complicated
+	 * than calling find_unique_abbrev() because we also need to make
+	 * sure that the abbreviation does not conflict with any other
+	 * label.
+	 *
+	 * We disallow "interesting" commits to be labeled by a string that
+	 * is a valid full-length hash, to ensure that we always can find an
+	 * abbreviation for any uninteresting commit's names that does not
+	 * clash with any other label.
+	 */
+	if (!label) {
+		char *p;
+
+		strbuf_reset(&state->buf);
+		strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
+		label = p = state->buf.buf;
+
+		find_unique_abbrev_r(p, oid->hash, default_abbrev);
+
+		/*
+		 * We may need to extend the abbreviated hash so that there is
+		 * no conflicting label.
+		 */
+		if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
+			size_t i = strlen(p) + 1;
+
+			oid_to_hex_r(p, oid);
+			for (; i < GIT_SHA1_HEXSZ; i++) {
+				char save = p[i];
+				p[i] = '\0';
+				if (!hashmap_get_from_hash(&state->labels,
+							   strihash(p), p))
+					break;
+				p[i] = save;
+			}
+		}
+	} else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
+		    !get_oid_hex(label, &dummy)) ||
+		   hashmap_get_from_hash(&state->labels,
+					 strihash(label), label)) {
+		/*
+		 * If the label already exists, or if the label is a valid full
+		 * OID, we append a dash and a number to make it unique.
+		 */
+		struct strbuf *buf = &state->buf;
+
+		strbuf_reset(buf);
+		strbuf_add(buf, label, len);
+
+		for (i = 2; ; i++) {
+			strbuf_setlen(buf, len);
+			strbuf_addf(buf, "-%d", i);
+			if (!hashmap_get_from_hash(&state->labels,
+						   strihash(buf->buf),
+						   buf->buf))
+				break;
+		}
+
+		label = buf->buf;
+	}
+
+	FLEX_ALLOC_STR(labels_entry, label, label);
+	hashmap_entry_init(labels_entry, strihash(label));
+	hashmap_add(&state->labels, labels_entry);
+
+	FLEX_ALLOC_STR(string_entry, string, label);
+	oidcpy(&string_entry->entry.oid, oid);
+	oidmap_put(&state->commit2label, string_entry);
+
+	return string_entry->string;
+}
+
+static int make_script_with_merges(struct pretty_print_context *pp,
+				   struct rev_info *revs, FILE *out,
+				   unsigned flags)
+{
+	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
+	struct strbuf label = STRBUF_INIT;
+	struct commit_list *commits = NULL, **tail = &commits, *iter;
+	struct commit_list *tips = NULL, **tips_tail = &tips;
+	struct commit *commit;
+	struct oidmap commit2todo = OIDMAP_INIT;
+	struct string_entry *entry;
+	struct oidset interesting = OIDSET_INIT, child_seen = OIDSET_INIT,
+		shown = OIDSET_INIT;
+	struct label_state state = { OIDMAP_INIT, { NULL }, STRBUF_INIT };
+
+	int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
+	const char *p = abbr ? "p" : "pick", *l = abbr ? "l" : "label",
+		 *t = abbr ? "t" : "reset", *b = abbr ? "b" : "bud",
+		 *m = abbr ? "m" : "merge";
+
+	oidmap_init(&commit2todo, 0);
+	oidmap_init(&state.commit2label, 0);
+	hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
+	strbuf_init(&state.buf, 32);
+
+	if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
+		struct object_id *oid = &revs->cmdline.rev[0].item->oid;
+		FLEX_ALLOC_STR(entry, string, "onto");
+		oidcpy(&entry->entry.oid, oid);
+		oidmap_put(&state.commit2label, entry);
+	}
+
+	/*
+	 * First phase:
+	 * - get onelines for all commits
+	 * - gather all branch tips (i.e. 2nd or later parents of merges)
+	 * - label all branch tips
+	 */
+	while ((commit = get_revision(revs))) {
+		struct commit_list *to_merge;
+		int is_octopus;
+		const char *p1, *p2;
+		struct object_id *oid;
+
+		tail = &commit_list_insert(commit, tail)->next;
+		oidset_insert(&interesting, &commit->object.oid);
+
+		if ((commit->object.flags & PATCHSAME))
+			continue;
+
+		strbuf_reset(&oneline);
+		pretty_print_commit(pp, commit, &oneline);
+
+		to_merge = commit->parents ? commit->parents->next : NULL;
+		if (!to_merge) {
+			/* non-merge commit: easy case */
+			strbuf_reset(&buf);
+			if (!keep_empty && is_original_commit_empty(commit))
+				strbuf_addf(&buf, "%c ", comment_line_char);
+			strbuf_addf(&buf, "%s %s %s", p,
+				    oid_to_hex(&commit->object.oid),
+				    oneline.buf);
+
+			FLEX_ALLOC_STR(entry, string, buf.buf);
+			oidcpy(&entry->entry.oid, &commit->object.oid);
+			oidmap_put(&commit2todo, entry);
+
+			continue;
+		}
+
+		is_octopus = to_merge && to_merge->next;
+
+		if (is_octopus)
+			BUG("Octopus merges not yet supported");
+
+		/* Create a label */
+		strbuf_reset(&label);
+		if (skip_prefix(oneline.buf, "Merge ", &p1) &&
+		    (p1 = strchr(p1, '\'')) &&
+		    (p2 = strchr(++p1, '\'')))
+			strbuf_add(&label, p1, p2 - p1);
+		else if (skip_prefix(oneline.buf, "Merge pull request ",
+				     &p1) &&
+			 (p1 = strstr(p1, " from ")))
+			strbuf_addstr(&label, p1 + strlen(" from "));
+		else
+			strbuf_addbuf(&label, &oneline);
+
+		for (p1 = label.buf; *p1; p1++)
+			if (isspace(*p1))
+				*(char *)p1 = '-';
+
+		strbuf_reset(&buf);
+		strbuf_addf(&buf, "%s %s", m, oid_to_hex(&commit->object.oid));
+
+		/* label the tip of merged branch */
+		oid = &to_merge->item->object.oid;
+		strbuf_addch(&buf, ' ');
+
+		if (!oidset_contains(&interesting, oid))
+			strbuf_addstr(&buf, label_oid(oid, NULL, &state));
+		else {
+			tips_tail = &commit_list_insert(to_merge->item,
+							tips_tail)->next;
+
+			strbuf_addstr(&buf, label_oid(oid, label.buf, &state));
+		}
+		strbuf_addf(&buf, " %s", oneline.buf);
+
+		FLEX_ALLOC_STR(entry, string, buf.buf);
+		oidcpy(&entry->entry.oid, &commit->object.oid);
+		oidmap_put(&commit2todo, entry);
+	}
+
+	/*
+	 * Second phase:
+	 * - label branch points
+	 * - add HEAD to the branch tips
+	 */
+	for (iter = commits; iter; iter = iter->next) {
+		struct commit_list *parent = iter->item->parents;
+		for (; parent; parent = parent->next) {
+			struct object_id *oid = &parent->item->object.oid;
+			if (!oidset_contains(&interesting, oid))
+				continue;
+			if (!oidset_contains(&child_seen, oid))
+				oidset_insert(&child_seen, oid);
+			else
+				label_oid(oid, "branch-point", &state);
+		}
+
+		/* Add HEAD as implict "tip of branch" */
+		if (!iter->next)
+			tips_tail = &commit_list_insert(iter->item,
+							tips_tail)->next;
+	}
+
+	/*
+	 * Third phase: output the todo list. This is a bit tricky, as we
+	 * want to avoid jumping back and forth between revisions. To
+	 * accomplish that goal, we walk backwards from the branch tips,
+	 * gathering commits not yet shown, reversing the list on the fly,
+	 * then outputting that list (labeling revisions as needed).
+	 */
+	fprintf(out, "%s onto\n", l);
+	for (iter = tips; iter; iter = iter->next) {
+		struct commit_list *list = NULL, *iter2;
+
+		commit = iter->item;
+		if (oidset_contains(&shown, &commit->object.oid))
+			continue;
+		entry = oidmap_get(&state.commit2label, &commit->object.oid);
+
+		if (entry)
+			fprintf(out, "\n# Branch %s\n", entry->string);
+		else
+			fprintf(out, "\n");
+
+		while (oidset_contains(&interesting, &commit->object.oid) &&
+		       !oidset_contains(&shown, &commit->object.oid)) {
+			commit_list_insert(commit, &list);
+			if (!commit->parents) {
+				commit = NULL;
+				break;
+			}
+			commit = commit->parents->item;
+		}
+
+		if (!commit)
+			fprintf(out, "%s\n", b);
+		else {
+			const char *to = NULL;
+
+			entry = oidmap_get(&state.commit2label,
+					   &commit->object.oid);
+			if (entry)
+				to = entry->string;
+
+			if (!to || !strcmp("onto", to))
+				fprintf(out, "%s\n", b);
+			else {
+				strbuf_reset(&oneline);
+				pretty_print_commit(pp, commit, &oneline);
+				fprintf(out, "%s %s %s\n",
+					t, to, oneline.buf);
+			}
+		}
+
+		for (iter2 = list; iter2; iter2 = iter2->next) {
+			struct object_id *oid = &iter2->item->object.oid;
+			entry = oidmap_get(&commit2todo, oid);
+			/* only show if not already upstream */
+			if (entry)
+				fprintf(out, "%s\n", entry->string);
+			entry = oidmap_get(&state.commit2label, oid);
+			if (entry)
+				fprintf(out, "%s %s\n", l, entry->string);
+			oidset_insert(&shown, oid);
+		}
+
+		free_commit_list(list);
+	}
+
+	free_commit_list(commits);
+	free_commit_list(tips);
+
+	strbuf_release(&label);
+	strbuf_release(&oneline);
+	strbuf_release(&buf);
+
+	oidmap_free(&commit2todo, 1);
+	oidmap_free(&state.commit2label, 1);
+	hashmap_free(&state.labels, 1);
+	strbuf_release(&state.buf);
+
+	return 0;
+}
+
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags)
 {
@@ -2795,11 +3126,16 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	struct commit *commit;
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
 	const char *insn = flags & TODO_LIST_ABBREVIATE_CMDS ? "p" : "pick";
+	int recreate_merges = flags & TODO_LIST_RECREATE_MERGES;
 
 	init_revisions(&revs, NULL);
 	revs.verbose_header = 1;
-	revs.max_parents = 1;
-	revs.cherry_pick = 1;
+	if (recreate_merges)
+		revs.cherry_mark = 1;
+	else {
+		revs.max_parents = 1;
+		revs.cherry_pick = 1;
+	}
 	revs.limited = 1;
 	revs.reverse = 1;
 	revs.right_only = 1;
@@ -2823,6 +3159,9 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	if (prepare_revision_walk(&revs) < 0)
 		return error(_("make_script: error preparing revisions"));
 
+	if (recreate_merges)
+		return make_script_with_merges(&pp, &revs, out, flags);
+
 	while ((commit = get_revision(&revs))) {
 		strbuf_reset(&buf);
 		if (!keep_empty && is_original_commit_empty(commit))
diff --git a/sequencer.h b/sequencer.h
index 81f6d7d393f..11d1ac925ef 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -48,6 +48,7 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_KEEP_EMPTY (1U << 0)
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
+#define TODO_LIST_RECREATE_MERGES (1U << 3)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
-- 
2.15.1.windows.2.1430.ga56c4f9e2a9



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
                   ` (3 preceding siblings ...)
  2018-01-18 15:35 ` [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
@ 2018-01-18 15:35 ` Johannes Schindelin
  2018-01-19 10:55   ` Eric Sunshine
                     ` (3 more replies)
  2018-01-18 15:35 ` [PATCH 6/8] sequencer: handle autosquash and post-rewrite for merge commands Johannes Schindelin
                   ` (7 subsequent siblings)
  12 siblings, 4 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 15:35 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Jacob Keller

Once upon a time, this here developer thought: wouldn't it be nice if,
say, Git for Windows' patches on top of core Git could be represented as
a thicket of branches, and be rebased on top of core Git in order to
maintain a cherry-pick'able set of patch series?

The original attempt at an answer was: git rebase --preserve-merges.

However, that experiment was never intended as an interactive option,
and it only piggy-backed on git rebase --interactive because that
command's implementation looked already very, very familiar: it was
designed by the same person who designed --preserve-merges: yours truly.

Some time later, some other developer (I am looking at you, Andreas!
;-)) decided that it would be a good idea to allow --preserve-merges to
be combined with --interactive (with caveats!) and the Git maintainer
(well, the interim Git maintainer during Junio's absence, that is)
agreed, and that is when the glamor of the --preserve-merges design
started to fall apart rather quickly and unglamorously.

The reason? In --preserve-merges mode, the parents of a merge commit (or
for that matter, of *any* commit) were not stated explicitly, but were
*implied* by the commit name passed to the `pick` command.

This made it impossible, for example, to reorder commits. Not to mention
to flatten the branch topology or, deity forbid, to split topic branches
into two.

Alas, these shortcomings also prevented that mode (whose original
purpose was to serve Git for Windows' needs, with the additional hope
that it may be useful to others, too) from serving Git for Windows'
needs.

Five years later, when it became really untenable to have one unwieldy,
big hodge-podge patch series of partly related, partly unrelated patches
in Git for Windows that was rebased onto core Git's tags from time to
time (earning the undeserved wrath of the developer of the ill-fated
git-remote-hg series that first obsoleted Git for Windows' competing
approach, only to be abandoned without maintainer later) was really
untenable, the "Git garden shears" were born [*1*/*2*]: a script,
piggy-backing on top of the interactive rebase, that would first
determine the branch topology of the patches to be rebased, create a
pseudo todo list for further editing, transform the result into a real
todo list (making heavy use of the `exec` command to "implement" the
missing todo list commands) and finally recreate the patch series on
top of the new base commit.

That was in 2013. And it took about three weeks to come up with the
design and implement it as an out-of-tree script. Needless to say, the
implementation needed quite a few years to stabilize, all the while the
design itself proved itself sound.

With this patch, the goodness of the Git garden shears comes to `git
rebase -i` itself. Passing the `--recreate-merges` option will generate
a todo list that can be understood readily, and where it is obvious
how to reorder commits. New branches can be introduced by inserting
`label` commands and calling `merge - <label> <oneline>`. And once this
mode has become stable and universally accepted, we can deprecate the
design mistake that was `--preserve-merges`.

Link *1*:
https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
Link *2*:
https://github.com/git-for-windows/build-extra/blob/master/shears.sh

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt           |   8 +-
 contrib/completion/git-completion.bash |   2 +-
 git-rebase--interactive.sh             |   1 +
 git-rebase.sh                          |   6 ++
 t/t3430-rebase-recreate-merges.sh      | 146 +++++++++++++++++++++++++++++++++
 5 files changed, 161 insertions(+), 2 deletions(-)
 create mode 100755 t/t3430-rebase-recreate-merges.sh

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 8a861c1e0d6..1d061373288 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -368,6 +368,11 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
+--recreate-merges::
+	Recreate merge commits instead of flattening the history by replaying
+	merges. Merge conflict resolutions or manual amendments to merge
+	commits are not preserved.
+
 -p::
 --preserve-merges::
 	Recreate merge commits instead of flattening the history by replaying
@@ -770,7 +775,8 @@ BUGS
 The todo list presented by `--preserve-merges --interactive` does not
 represent the topology of the revision graph.  Editing commits and
 rewording their commit messages should work fine, but attempts to
-reorder commits tend to produce counterintuitive results.
+reorder commits tend to produce counterintuitive results. Use
+--recreate-merges for a more faithful representation.
 
 For example, an attempt to rearrange
 ------------
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 3683c772c55..6893c3adabc 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2008,7 +2008,7 @@ _git_rebase ()
 	--*)
 		__gitcomp "
 			--onto --merge --strategy --interactive
-			--preserve-merges --stat --no-stat
+			--recreate-merges --preserve-merges --stat --no-stat
 			--committer-date-is-author-date --ignore-date
 			--ignore-whitespace --whitespace=
 			--autosquash --no-autosquash
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 5bf1ea3781f..3459ec5a018 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -900,6 +900,7 @@ fi
 if test t != "$preserve_merges"
 then
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
+		${recreate_merges:+--recreate-merges} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 else
diff --git a/git-rebase.sh b/git-rebase.sh
index fd72a35c65b..d69bc7d0e0d 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,6 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
+recreate-merges!   try to recreate merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -86,6 +87,7 @@ type=
 state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
+recreate_merges=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -262,6 +264,10 @@ do
 	--keep-empty)
 		keep_empty=yes
 		;;
+	--recreate-merges)
+		recreate_merges=t
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
new file mode 100755
index 00000000000..46ae52f88b3
--- /dev/null
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -0,0 +1,146 @@
+#!/bin/sh
+#
+# Copyright (c) 2017 Johannes E. Schindelin
+#
+
+test_description='git rebase -i --recreate-merges
+
+This test runs git rebase "interactively", retaining the branch structure by
+recreating merge commits.
+
+Initial setup:
+
+    -- B --                   (first)
+   /       \
+ A - C - D - E - H            (master)
+       \       /
+         F - G                (second)
+'
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
+
+test_expect_success 'setup' '
+	write_script replace-editor.sh <<-\EOF &&
+	mv "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	cp script-from-scratch "$1"
+	EOF
+
+	test_commit A &&
+	git checkout -b first &&
+	test_commit B &&
+	git checkout master &&
+	test_commit C &&
+	test_commit D &&
+	git merge --no-commit B &&
+	test_tick &&
+	git commit -m E &&
+	git tag -m E E &&
+	git checkout -b second C &&
+	test_commit F &&
+	test_commit G &&
+	git checkout master &&
+	git merge --no-commit G &&
+	test_tick &&
+	git commit -m H &&
+	git tag -m H H
+'
+
+cat >script-from-scratch <<\EOF
+label onto
+
+# onebranch
+pick G
+pick D
+label onebranch
+
+# second
+bud
+pick B
+label second
+
+bud
+merge H second
+merge - onebranch Merge the topic branch 'onebranch'
+EOF
+
+test_cmp_graph () {
+	cat >expect &&
+	git log --graph --boundary --format=%s "$@" >output &&
+	sed "s/ *$//" <output >output.trimmed &&
+	test_cmp expect output.trimmed
+}
+
+test_expect_success 'create completely different structure' '
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_tick &&
+	git rebase -i --recreate-merges A &&
+	test_cmp_graph <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	* |   H
+	|\ \
+	| |/
+	|/|
+	| * B
+	|/
+	* A
+	EOF
+'
+
+test_expect_success 'generate correct todo list' '
+	cat >expect <<-\EOF &&
+	label onto
+
+	bud
+	pick d9df450 B
+	label E
+
+	bud
+	pick 5dee784 C
+	label branch-point
+	pick ca2c861 F
+	pick 088b00a G
+	label H
+
+	reset branch-point C
+	pick 12bd07b D
+	merge 2051b56 E E
+	merge 233d48a H H
+
+	EOF
+
+	grep -v "^#" <.git/ORIGINAL-TODO >output &&
+	test_cmp expect output
+'
+
+test_expect_success 'with a branch tip that was cherry-picked already' '
+	git checkout -b already-upstream master &&
+	base="$(git rev-parse --verify HEAD)" &&
+
+	test_commit A1 &&
+	test_commit A2 &&
+	git reset --hard $base &&
+	test_commit B1 &&
+	test_tick &&
+	git merge -m "Merge branch A" A2 &&
+
+	git checkout -b upstream-with-a2 $base &&
+	test_tick &&
+	git cherry-pick A2 &&
+
+	git checkout already-upstream &&
+	test_tick &&
+	git rebase -i --recreate-merges upstream-with-a2 &&
+	test_cmp_graph upstream-with-a2.. <<-\EOF
+	*   Merge branch A
+	|\
+	| * A1
+	* | B1
+	|/
+	o A2
+	EOF
+'
+
+test_done
-- 
2.15.1.windows.2.1430.ga56c4f9e2a9



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH 6/8] sequencer: handle autosquash and post-rewrite for merge commands
  2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
                   ` (4 preceding siblings ...)
  2018-01-18 15:35 ` [PATCH 5/8] rebase: introduce the --recreate-merges option Johannes Schindelin
@ 2018-01-18 15:35 ` Johannes Schindelin
  2018-01-18 16:43   ` Jacob Keller
  2018-01-18 15:36 ` [PATCH 7/8] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
                   ` (6 subsequent siblings)
  12 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 15:35 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Jacob Keller

In the previous patches, we implemented the basic functionality of the
`git rebase -i --recreate-merges` command, in particular the `merge`
command to create merge commits in the sequencer.

The interactive rebase is a lot more these days, though, than a simple
cherry-pick in a loop. For example, it calls the post-rewrite hook (if
any) after rebasing with a mapping of the old->new commits. And the
interactive rebase also supports the autosquash mode, where commits
whose oneline is of the form `fixup! <oneline>` or `squash! <oneline>`
are rearranged to amend commits whose oneline they match.

This patch implements the post-rewrite and autosquash handling for the
`merge` command we just introduced. The other commands that were added
recently (`label`, `reset` and `bud`) do not create new commits,
therefore post-rewrite & autosquash do not need to handle them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 refs.c                            |  3 ++-
 sequencer.c                       | 10 +++++++---
 t/t3430-rebase-recreate-merges.sh | 39 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 48 insertions(+), 4 deletions(-)

diff --git a/refs.c b/refs.c
index 20ba82b4343..e8b84c189ff 100644
--- a/refs.c
+++ b/refs.c
@@ -600,7 +600,8 @@ int dwim_log(const char *str, int len, struct object_id *oid, char **log)
 static int is_per_worktree_ref(const char *refname)
 {
 	return !strcmp(refname, "HEAD") ||
-		starts_with(refname, "refs/bisect/");
+		starts_with(refname, "refs/bisect/") ||
+		starts_with(refname, "refs/rewritten/");
 }
 
 static int is_pseudoref_syntax(const char *refname)
diff --git a/sequencer.c b/sequencer.c
index 1bef16647b4..b63bfb9a141 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2413,10 +2413,13 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_reset(item->arg, item->arg_len);
 		else if (item->command == TODO_BUD)
 			res = do_reset("onto", 4);
-		else if (item->command == TODO_MERGE)
+		else if (item->command == TODO_MERGE) {
 			res = do_merge(item->commit,
 				       item->arg, item->arg_len, opts);
-		else if (!is_noop(item->command))
+			if (item->commit)
+				record_in_rewritten(&item->commit->object.oid,
+						    peek_command(todo_list, 1));
+		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
@@ -3556,7 +3559,8 @@ int rearrange_squash(void)
 		struct subject2item_entry *entry;
 
 		next[i] = tail[i] = -1;
-		if (item->command >= TODO_EXEC) {
+		if (item->command >= TODO_EXEC &&
+		    (item->command != TODO_MERGE || !item->commit)) {
 			subjects[i] = NULL;
 			continue;
 		}
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index 46ae52f88b3..76e615bd7c1 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -143,4 +143,43 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'refs/rewritten/* is worktree-local' '
+	git worktree add wt &&
+	cat >wt/script-from-scratch <<-\EOF &&
+	label xyz
+	exec GIT_DIR=../.git git rev-parse --verify refs/rewritten/xyz >a || :
+	exec git rev-parse --verify refs/rewritten/xyz >b
+	EOF
+
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	git -C wt rebase -i HEAD &&
+	test_must_be_empty wt/a &&
+	test_cmp_rev HEAD "$(cat wt/b)"
+'
+
+test_expect_success 'post-rewrite hook and fixups work for merges' '
+	git checkout -b post-rewrite &&
+	test_commit same1 &&
+	git reset --hard HEAD^ &&
+	test_commit same2 &&
+	git merge -m "to fix up" same1 &&
+	echo same old same old >same2.t &&
+	test_tick &&
+	git commit --fixup HEAD same2.t &&
+	fixup="$(git rev-parse HEAD)" &&
+
+	mkdir -p .git/hooks &&
+	test_when_finished "rm .git/hooks/post-rewrite" &&
+	echo "cat >actual" | write_script .git/hooks/post-rewrite &&
+
+	test_tick &&
+	git rebase -i --autosquash --recreate-merges HEAD^^^ &&
+	printf "%s %s\n%s %s\n%s %s\n%s %s\n" >expect $(git rev-parse \
+		$fixup^^2 HEAD^2 \
+		$fixup^^ HEAD^ \
+		$fixup^ HEAD \
+		$fixup HEAD) &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.15.1.windows.2.1430.ga56c4f9e2a9



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH 7/8] pull: accept --rebase=recreate to recreate the branch topology
  2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
                   ` (5 preceding siblings ...)
  2018-01-18 15:35 ` [PATCH 6/8] sequencer: handle autosquash and post-rewrite for merge commands Johannes Schindelin
@ 2018-01-18 15:36 ` Johannes Schindelin
  2018-01-18 15:36 ` [PATCH 8/8] rebase -i: introduce --recreate-merges=no-rebase-cousins Johannes Schindelin
                   ` (5 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 15:36 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Jacob Keller

Similar to the `preserve` mode simply passing the `--preserve-merges`
option to the `rebase` command, the `recreate` mode simply passes the
`--recreate-merges` option.

This will allow users to conveniently rebase non-trivial commit
topologies when pulling new commits, without flattening them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/config.txt               |  8 ++++++++
 Documentation/git-pull.txt             |  5 ++++-
 builtin/pull.c                         | 14 ++++++++++----
 builtin/remote.c                       |  2 ++
 contrib/completion/git-completion.bash |  2 +-
 5 files changed, 25 insertions(+), 6 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0e25b2c92b3..da41ab246dc 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1058,6 +1058,10 @@ branch.<name>.rebase::
 	"git pull" is run. See "pull.rebase" for doing this in a non
 	branch-specific manner.
 +
+When recreate, also pass `--recreate-merges` along to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
@@ -2607,6 +2611,10 @@ pull.rebase::
 	pull" is run. See "branch.<name>.rebase" for setting this on a
 	per-branch basis.
 +
+When recreate, also pass `--recreate-merges` along to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
diff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt
index ce05b7a5b13..b4f9f057ea9 100644
--- a/Documentation/git-pull.txt
+++ b/Documentation/git-pull.txt
@@ -101,13 +101,16 @@ Options related to merging
 include::merge-options.txt[]
 
 -r::
---rebase[=false|true|preserve|interactive]::
+--rebase[=false|true|recreate|preserve|interactive]::
 	When true, rebase the current branch on top of the upstream
 	branch after fetching. If there is a remote-tracking branch
 	corresponding to the upstream branch and the upstream branch
 	was rebased since last fetched, the rebase uses that information
 	to avoid rebasing non-local changes.
 +
+When set to recreate, rebase with the `--recreate-merges` option passed
+to `git rebase` so that locally created merge commits will not be flattened.
++
 When set to preserve, rebase with the `--preserve-merges` option passed
 to `git rebase` so that locally created merge commits will not be flattened.
 +
diff --git a/builtin/pull.c b/builtin/pull.c
index 511dbbe0f6e..e33c84e0345 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -27,14 +27,16 @@ enum rebase_type {
 	REBASE_FALSE = 0,
 	REBASE_TRUE,
 	REBASE_PRESERVE,
+	REBASE_RECREATE,
 	REBASE_INTERACTIVE
 };
 
 /**
  * Parses the value of --rebase. If value is a false value, returns
  * REBASE_FALSE. If value is a true value, returns REBASE_TRUE. If value is
- * "preserve", returns REBASE_PRESERVE. If value is a invalid value, dies with
- * a fatal error if fatal is true, otherwise returns REBASE_INVALID.
+ * "recreate", returns REBASE_RECREATE. If value is "preserve", returns
+ * REBASE_PRESERVE. If value is a invalid value, dies with a fatal error if
+ * fatal is true, otherwise returns REBASE_INVALID.
  */
 static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		int fatal)
@@ -47,6 +49,8 @@ static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		return REBASE_TRUE;
 	else if (!strcmp(value, "preserve"))
 		return REBASE_PRESERVE;
+	else if (!strcmp(value, "recreate"))
+		return REBASE_RECREATE;
 	else if (!strcmp(value, "interactive"))
 		return REBASE_INTERACTIVE;
 
@@ -130,7 +134,7 @@ static struct option pull_options[] = {
 	/* Options passed to git-merge or git-rebase */
 	OPT_GROUP(N_("Options related to merging")),
 	{ OPTION_CALLBACK, 'r', "rebase", &opt_rebase,
-	  "false|true|preserve|interactive",
+	  "false|true|recreate|preserve|interactive",
 	  N_("incorporate changes by rebasing rather than merging"),
 	  PARSE_OPT_OPTARG, parse_opt_rebase },
 	OPT_PASSTHRU('n', NULL, &opt_diffstat, NULL,
@@ -798,7 +802,9 @@ static int run_rebase(const struct object_id *curr_head,
 	argv_push_verbosity(&args);
 
 	/* Options passed to git-rebase */
-	if (opt_rebase == REBASE_PRESERVE)
+	if (opt_rebase == REBASE_RECREATE)
+		argv_array_push(&args, "--recreate-merges");
+	else if (opt_rebase == REBASE_PRESERVE)
 		argv_array_push(&args, "--preserve-merges");
 	else if (opt_rebase == REBASE_INTERACTIVE)
 		argv_array_push(&args, "--interactive");
diff --git a/builtin/remote.c b/builtin/remote.c
index d95bf904c3b..b7d0f7ce596 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -306,6 +306,8 @@ static int config_read_branches(const char *key, const char *value, void *cb)
 				info->rebase = v;
 			else if (!strcmp(value, "preserve"))
 				info->rebase = NORMAL_REBASE;
+			else if (!strcmp(value, "recreate"))
+				info->rebase = NORMAL_REBASE;
 			else if (!strcmp(value, "interactive"))
 				info->rebase = INTERACTIVE_REBASE;
 		}
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 6893c3adabc..6f98c96fee9 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2182,7 +2182,7 @@ _git_config ()
 		return
 		;;
 	branch.*.rebase)
-		__gitcomp "false true preserve interactive"
+		__gitcomp "false true recreate preserve interactive"
 		return
 		;;
 	remote.pushdefault)
-- 
2.15.1.windows.2.1430.ga56c4f9e2a9



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH 8/8] rebase -i: introduce --recreate-merges=no-rebase-cousins
  2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
                   ` (6 preceding siblings ...)
  2018-01-18 15:36 ` [PATCH 7/8] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
@ 2018-01-18 15:36 ` Johannes Schindelin
  2018-01-18 22:00   ` Philip Oakley
  2018-01-20  1:09   ` Eric Sunshine
  2018-01-18 16:49 ` [PATCH 0/8] rebase -i: offer to recreate merge commits Jacob Keller
                   ` (4 subsequent siblings)
  12 siblings, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 15:36 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Jacob Keller

This one is a bit tricky to explain, so let's try with a diagram:

        C
      /   \
A - B - E - F
  \   /
    D

To illustrate what this new mode is all about, let's consider what
happens upon `git rebase -i --recreate-merges B`, in particular to
the commit `D`. In the default mode, the new branch structure is:

       --- C' --
      /         \
A - B ------ E' - F'
      \    /
        D'

This is not really preserving the branch topology from before! The
reason is that the commit `D` does not have `B` as ancestor, and
therefore it gets rebased onto `B`.

However, when recreating branch structure, there are legitimate use
cases where one might want to preserve the branch points of commits that
do not descend from the <upstream> commit that was passed to the rebase
command, e.g. when a branch from core Git's `next` was merged into Git
for Windows' master we will not want to rebase those commits on top of a
Windows-specific commit. In the example above, the desired outcome would
look like this:

       --- C' --
      /         \
A - B ------ E' - F'
  \        /
   -- D' --

Let's introduce the term "cousins" for such commits ("D" in the
example), and the "no-rebase-cousins" mode of the merge-recreating
rebase, to help those use cases.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt      |  7 ++++++-
 builtin/rebase--helper.c          |  9 ++++++++-
 git-rebase--interactive.sh        |  1 +
 git-rebase.sh                     | 12 +++++++++++-
 sequencer.c                       |  4 ++++
 sequencer.h                       |  8 ++++++++
 t/t3430-rebase-recreate-merges.sh | 23 +++++++++++++++++++++++
 7 files changed, 61 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 1d061373288..ac07a5c3fc9 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -368,10 +368,15 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
---recreate-merges::
+--recreate-merges[=(rebase-cousins|no-rebase-cousins)]::
 	Recreate merge commits instead of flattening the history by replaying
 	merges. Merge conflict resolutions or manual amendments to merge
 	commits are not preserved.
++
+By default, or when `rebase-cousins` was specified, commits which do not have
+`<upstream>` as direct ancestor are rebased onto `<upstream>` (or `<onto>`,
+if specified). If the `rebase-cousins` mode is turned off, such commits will
+retain their original branch point.
 
 -p::
 --preserve-merges::
diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index a34ab5c0655..ef08fef4d14 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
 	unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
-	int abbreviate_commands = 0;
+	int abbreviate_commands = 0, no_rebase_cousins = -1;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
 		CHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH,
@@ -23,6 +23,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "ff", &opts.allow_ff, N_("allow fast-forward")),
 		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
 		OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge commits")),
+		OPT_BOOL(0, "no-rebase-cousins", &no_rebase_cousins,
+			 N_("keep original branch points of cousins")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -57,8 +59,13 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
 	flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
+	flags |= no_rebase_cousins > 0 ? TODO_LIST_NO_REBASE_COUSINS : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
+	if (no_rebase_cousins >= 0&& !recreate_merges)
+		warning(_("--[no-]rebase-cousins has no effect without "
+			  "--recreate-merges"));
+
 	if (command == CONTINUE && argc == 1)
 		return !!sequencer_continue(&opts);
 	if (command == ABORT && argc == 1)
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 3459ec5a018..23184c77e88 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -901,6 +901,7 @@ if test t != "$preserve_merges"
 then
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
 		${recreate_merges:+--recreate-merges} \
+		${no_rebase_cousins:+--no-rebase-cousins} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 else
diff --git a/git-rebase.sh b/git-rebase.sh
index d69bc7d0e0d..3403b1416a8 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,7 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
-recreate-merges!   try to recreate merges instead of skipping them
+recreate-merges?   try to recreate merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -88,6 +88,7 @@ state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
 recreate_merges=
+no_rebase_cousins=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -268,6 +269,15 @@ do
 		recreate_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
 		;;
+	--recreate-merges=*)
+		recreate_merges=t
+		case "${1#*=}" in
+		rebase-cousins) no_rebase_cousins=;;
+		no-rebase-cousins) no_rebase_cousins=t;;
+		*) die "Unknown mode: $1";;
+		esac
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/sequencer.c b/sequencer.c
index b63bfb9a141..2b4e6b12321 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2905,6 +2905,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 				   unsigned flags)
 {
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	int no_rebase_cousins = flags & TODO_LIST_NO_REBASE_COUSINS;
 	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
 	struct strbuf label = STRBUF_INIT;
 	struct commit_list *commits = NULL, **tail = &commits, *iter;
@@ -3078,6 +3079,9 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 					   &commit->object.oid);
 			if (entry)
 				to = entry->string;
+			else if (no_rebase_cousins)
+				to = label_oid(&commit->object.oid, NULL,
+					       &state);
 
 			if (!to || !strcmp("onto", to))
 				fprintf(out, "%s\n", b);
diff --git a/sequencer.h b/sequencer.h
index 11d1ac925ef..9530dba3cba 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -49,6 +49,14 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
 #define TODO_LIST_RECREATE_MERGES (1U << 3)
+/*
+ * When recreating merges, commits that do have the base commit as ancestor
+ * ("cousins") are rebased onto the new base by default. If those commits
+ * should keep their original branch point, this flag needs to be passed.
+ *
+ * This flag only makes sense when <base> and <onto> are different.
+ */
+#define TODO_LIST_NO_REBASE_COUSINS (1U << 4)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index 76e615bd7c1..22930e470a4 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -143,6 +143,29 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'rebase cousins unless told not to' '
+	write_script copy-editor.sh <<-\EOF &&
+	cp "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	EOF
+
+	test_config sequence.editor \""$PWD"/copy-editor.sh\" &&
+	git checkout -b cousins master &&
+	before="$(git rev-parse --verify HEAD)" &&
+	test_tick &&
+	git rebase -i --recreate-merges=no-rebase-cousins HEAD^ &&
+	test_cmp_rev HEAD $before &&
+	test_tick &&
+	git rebase -i --recreate-merges HEAD^ &&
+	test_cmp_graph HEAD^.. <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	|/
+	o H
+	EOF
+'
+
 test_expect_success 'refs/rewritten/* is worktree-local' '
 	git worktree add wt &&
 	cat >wt/script-from-scratch <<-\EOF &&
-- 
2.15.1.windows.2.1430.ga56c4f9e2a9

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-18 15:35 ` [PATCH 1/8] sequencer: introduce new commands to reset the revision Johannes Schindelin
@ 2018-01-18 16:25   ` Jacob Keller
  2018-01-18 21:13     ` Johannes Schindelin
                       ` (2 more replies)
  2018-01-19  8:59   ` Eric Sunshine
  2018-01-19 12:24   ` [PATCH 1/8] sequencer: introduce new commands to resettherevision Phillip Wood
  2 siblings, 3 replies; 412+ messages in thread
From: Jacob Keller @ 2018-01-18 16:25 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Git mailing list, Junio C Hamano

On Thu, Jan 18, 2018 at 7:35 AM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> This commit implements the commands to label, and to reset to, given
> revisions. The syntax is:
>
>         label <name>
>         reset <name>
>
> As a convenience shortcut, also to improve readability of the generated
> todo list, a third command is introduced: bud. It simply resets to the
> "onto" revision, i.e. the commit onto which we currently rebase.
>

The code looks good, but I'm a little wary of adding bud which
hard-codes a specific label. I suppose it does grant a bit of
readability to the resulting script... ? It doesn't seem that
important compared to use using "reset onto"? At least when
documenting this it should be made clear that the "onto" label is
special.

Thanks,
Jake.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/8] sequencer: introduce the `merge` command
  2018-01-18 15:35 ` [PATCH 2/8] sequencer: introduce the `merge` command Johannes Schindelin
@ 2018-01-18 16:31   ` Jacob Keller
  2018-01-18 21:22     ` Johannes Schindelin
  2018-01-19  9:54   ` Eric Sunshine
                     ` (2 subsequent siblings)
  3 siblings, 1 reply; 412+ messages in thread
From: Jacob Keller @ 2018-01-18 16:31 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Git mailing list, Junio C Hamano

On Thu, Jan 18, 2018 at 7:35 AM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> This patch is part of the effort to reimplement `--preserve-merges` with
> a substantially improved design, a design that has been developed in the
> Git for Windows project to maintain the dozens of Windows-specific patch
> series on top of upstream Git.
>
> The previous patch implemented the `label`, `bud` and `reset` commands
> to label commits and to reset to a labeled commits. This patch adds the
> `merge` command, with the following syntax:
>
>         merge <commit> <rev> <oneline>
>
> The <commit> parameter in this instance is the *original* merge commit,
> whose author and message will be used for the to-be-created merge
> commit.
>
> The <rev> parameter refers to the (possibly rewritten) revision to
> merge. Let's see an example of a todo list:
>
>         label onto
>
>         # Branch abc
>         bud
>         pick deadbeef Hello, world!
>         label abc
>
>         bud
>         pick cafecafe And now for something completely different
>         merge baaabaaa abc Merge the branch 'abc' into master
>
> To support creating *new* merges, i.e. without copying the commit
> message from an existing commit, use the special value `-` as <commit>
> parameter (in which case the text after the <rev> parameter is used as
> commit message):
>
>         merge - abc This will be the actual commit message of the merge
>
> This comes in handy when splitting a branch into two or more branches.
>

Would it be possible to open the editor with the supplied text when
there's no commit? The text after <rev> must be oneline only..

It's difficult to reword merges because of the nature of rebase
interactive, you can't just re-run the rebase command and use
"reword".

I suppose you could cheat by putting in an "edit" command that let you
create an empty commit with a message...

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 6/8] sequencer: handle autosquash and post-rewrite for merge commands
  2018-01-18 15:35 ` [PATCH 6/8] sequencer: handle autosquash and post-rewrite for merge commands Johannes Schindelin
@ 2018-01-18 16:43   ` Jacob Keller
  2018-01-18 21:27     ` Johannes Schindelin
  2018-01-23 20:27     ` Junio C Hamano
  0 siblings, 2 replies; 412+ messages in thread
From: Jacob Keller @ 2018-01-18 16:43 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Git mailing list, Junio C Hamano

On Thu, Jan 18, 2018 at 7:35 AM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> In the previous patches, we implemented the basic functionality of the
> `git rebase -i --recreate-merges` command, in particular the `merge`
> command to create merge commits in the sequencer.
>
> The interactive rebase is a lot more these days, though, than a simple
> cherry-pick in a loop. For example, it calls the post-rewrite hook (if
> any) after rebasing with a mapping of the old->new commits. And the
> interactive rebase also supports the autosquash mode, where commits
> whose oneline is of the form `fixup! <oneline>` or `squash! <oneline>`
> are rearranged to amend commits whose oneline they match.
>
> This patch implements the post-rewrite and autosquash handling for the
> `merge` command we just introduced. The other commands that were added
> recently (`label`, `reset` and `bud`) do not create new commits,
> therefore post-rewrite & autosquash do not need to handle them.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  refs.c                            |  3 ++-
>  sequencer.c                       | 10 +++++++---
>  t/t3430-rebase-recreate-merges.sh | 39 +++++++++++++++++++++++++++++++++++++++
>  3 files changed, 48 insertions(+), 4 deletions(-)
>
> diff --git a/refs.c b/refs.c
> index 20ba82b4343..e8b84c189ff 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -600,7 +600,8 @@ int dwim_log(const char *str, int len, struct object_id *oid, char **log)
>  static int is_per_worktree_ref(const char *refname)
>  {
>         return !strcmp(refname, "HEAD") ||
> -               starts_with(refname, "refs/bisect/");
> +               starts_with(refname, "refs/bisect/") ||
> +               starts_with(refname, "refs/rewritten/");
>  }

Would this part make more sense to move into the commit that
introduces writing these refs, or does it only matter once you start
this step here?

>
>  static int is_pseudoref_syntax(const char *refname)
> diff --git a/sequencer.c b/sequencer.c
> index 1bef16647b4..b63bfb9a141 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -2413,10 +2413,13 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>                         res = do_reset(item->arg, item->arg_len);
>                 else if (item->command == TODO_BUD)
>                         res = do_reset("onto", 4);
> -               else if (item->command == TODO_MERGE)
> +               else if (item->command == TODO_MERGE) {
>                         res = do_merge(item->commit,
>                                        item->arg, item->arg_len, opts);
> -               else if (!is_noop(item->command))
> +                       if (item->commit)
> +                               record_in_rewritten(&item->commit->object.oid,
> +                                                   peek_command(todo_list, 1));
> +               } else if (!is_noop(item->command))
>                         return error(_("unknown command %d"), item->command);
>
>                 todo_list->current++;
> @@ -3556,7 +3559,8 @@ int rearrange_squash(void)
>                 struct subject2item_entry *entry;
>
>                 next[i] = tail[i] = -1;
> -               if (item->command >= TODO_EXEC) {
> +               if (item->command >= TODO_EXEC &&
> +                   (item->command != TODO_MERGE || !item->commit)) {
>                         subjects[i] = NULL;
>                         continue;
>                 }
> diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
> index 46ae52f88b3..76e615bd7c1 100755
> --- a/t/t3430-rebase-recreate-merges.sh
> +++ b/t/t3430-rebase-recreate-merges.sh
> @@ -143,4 +143,43 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
>         EOF
>  '
>
> +test_expect_success 'refs/rewritten/* is worktree-local' '
> +       git worktree add wt &&
> +       cat >wt/script-from-scratch <<-\EOF &&
> +       label xyz
> +       exec GIT_DIR=../.git git rev-parse --verify refs/rewritten/xyz >a || :
> +       exec git rev-parse --verify refs/rewritten/xyz >b
> +       EOF
> +
> +       test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
> +       git -C wt rebase -i HEAD &&
> +       test_must_be_empty wt/a &&
> +       test_cmp_rev HEAD "$(cat wt/b)"
> +'
> +

Same for the test here, I can't figure out why this is necessary in
this patch as opposed to the patch which first introduced the
refs/rewritten/<label> refs.

> +test_expect_success 'post-rewrite hook and fixups work for merges' '
> +       git checkout -b post-rewrite &&
> +       test_commit same1 &&
> +       git reset --hard HEAD^ &&
> +       test_commit same2 &&
> +       git merge -m "to fix up" same1 &&
> +       echo same old same old >same2.t &&
> +       test_tick &&
> +       git commit --fixup HEAD same2.t &&
> +       fixup="$(git rev-parse HEAD)" &&
> +
> +       mkdir -p .git/hooks &&
> +       test_when_finished "rm .git/hooks/post-rewrite" &&
> +       echo "cat >actual" | write_script .git/hooks/post-rewrite &&
> +
> +       test_tick &&
> +       git rebase -i --autosquash --recreate-merges HEAD^^^ &&
> +       printf "%s %s\n%s %s\n%s %s\n%s %s\n" >expect $(git rev-parse \
> +               $fixup^^2 HEAD^2 \
> +               $fixup^^ HEAD^ \
> +               $fixup^ HEAD \
> +               $fixup HEAD) &&
> +       test_cmp expect actual
> +'
> +
>  test_done
> --
> 2.15.1.windows.2.1430.ga56c4f9e2a9
>
>

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 0/8] rebase -i: offer to recreate merge commits
  2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
                   ` (7 preceding siblings ...)
  2018-01-18 15:36 ` [PATCH 8/8] rebase -i: introduce --recreate-merges=no-rebase-cousins Johannes Schindelin
@ 2018-01-18 16:49 ` Jacob Keller
  2018-01-18 18:36 ` [PATCH 9, 10/8] interactive rebase feedback Stefan Beller
                   ` (3 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-01-18 16:49 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Git mailing list, Junio C Hamano

On Thu, Jan 18, 2018 at 7:35 AM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> Once upon a time, I dreamt of an interactive rebase that would not
> flatten branch structure, but instead recreate the commit topology
> faithfully.
>
> My original attempt was --preserve-merges, but that design was so
> limited that I did not even enable it in interactive mode.
>
> Subsequently, it *was* enabled in interactive mode, with the predictable
> consequences: as the --preserve-merges design does not allow for
> specifying the parents of merge commits explicitly, all the new commits'
> parents are defined *implicitly* by the previous commit history, and
> hence it is *not possible to even reorder commits*.
>
> This design flaw cannot be fixed. Not without a complete re-design, at
> least. This patch series offers such a re-design.
>
> Think of --recreate-merges as "--preserve-merges done right". It
> introduces new verbs for the todo list, `label`, `reset` and `merge`.
> For a commit topology like this:
>
>             A - B - C
>               \   /
>                 D
>
> the generated todo list would look like this:
>
>             # branch D
>             pick 0123 A
>             label branch-point
>             pick 1234 D
>             label D
>
>             reset branch-point
>             pick 2345 B
>             merge 3456 D C
>
> There are more patches in the pipeline, based on this patch series, but
> left for later in the interest of reviewable patch series: one mini
> series to use the sequencer even for `git rebase -i --root`, and another
> one to add support for octopus merges to --recreate-merges.
>
>

I've been looking forward to seeing this hit the list! Overall I don't
think I have any major complaints, except to make sure the special
label "onto" is documented.

I think it's possible to "reword" or "edit" the merge commit by
inserting "x false" after the merge commnd to halt the rebase and let
the user perform commands to edit, so that should work well.

Thanks for taking the time to make this a reality!

Thanks,
Jake

> Johannes Schindelin (8):
>   sequencer: introduce new commands to reset the revision
>   sequencer: introduce the `merge` command
>   sequencer: fast-forward merge commits, if possible
>   rebase-helper --make-script: introduce a flag to recreate merges
>   rebase: introduce the --recreate-merges option
>   sequencer: handle autosquash and post-rewrite for merge commands
>   pull: accept --rebase=recreate to recreate the branch topology
>   rebase -i: introduce --recreate-merges=no-rebase-cousins
>
>  Documentation/config.txt               |   8 +
>  Documentation/git-pull.txt             |   5 +-
>  Documentation/git-rebase.txt           |  13 +-
>  builtin/pull.c                         |  14 +-
>  builtin/rebase--helper.c               |  13 +-
>  builtin/remote.c                       |   2 +
>  contrib/completion/git-completion.bash |   4 +-
>  git-rebase--interactive.sh             |   6 +
>  git-rebase.sh                          |  16 +
>  refs.c                                 |   3 +-
>  sequencer.c                            | 697 ++++++++++++++++++++++++++++++++-
>  sequencer.h                            |   9 +
>  t/t3430-rebase-recreate-merges.sh      | 208 ++++++++++
>  13 files changed, 977 insertions(+), 21 deletions(-)
>  create mode 100755 t/t3430-rebase-recreate-merges.sh
>
>
> base-commit: 2512f15446149235156528dafbe75930c712b29e
> Published-As: https://github.com/dscho/git/releases/tag/recreate-merges-v1
> Fetch-It-Via: git fetch https://github.com/dscho/git recreate-merges-v1
> --
> 2.15.1.windows.2.1430.ga56c4f9e2a9
>

^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH 9, 10/8] interactive rebase feedback
  2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
                   ` (8 preceding siblings ...)
  2018-01-18 16:49 ` [PATCH 0/8] rebase -i: offer to recreate merge commits Jacob Keller
@ 2018-01-18 18:36 ` Stefan Beller
  2018-01-18 18:36   ` [PATCH 9/8] [DO NOT APPLY, but squash?] git-rebase--interactive: clarify arguments Stefan Beller
  2018-01-18 18:36   ` [PATCH 10/8] [DO NOT APPLY, but improve?] rebase--interactive: introduce "stop" command Stefan Beller
  2018-01-19 20:25 ` [PATCH 0/8] rebase -i: offer to recreate merge commits Junio C Hamano
                   ` (2 subsequent siblings)
  12 siblings, 2 replies; 412+ messages in thread
From: Stefan Beller @ 2018-01-18 18:36 UTC (permalink / raw)
  To: johannes.schindelin; +Cc: git, gitster, jacob.keller, Stefan Beller

I thought of writing a reply, but instead I wrote it in patch form.

Stefan Beller (2):
  [DO NOT APPLY, but squash?] git-rebase--interactive: clarify arguments
  [DO NOT APPLY, but improve?] rebase--interactive: introduce "stop"
    command

 git-rebase--interactive.sh | 23 ++++++++++++-----------
 sequencer.c                | 10 ++++++++++
 2 files changed, 22 insertions(+), 11 deletions(-)

-- 
2.16.0.rc1.238.g530d649a79-goog


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH 9/8] [DO NOT APPLY, but squash?] git-rebase--interactive: clarify arguments
  2018-01-18 18:36 ` [PATCH 9, 10/8] interactive rebase feedback Stefan Beller
@ 2018-01-18 18:36   ` Stefan Beller
  2018-01-18 21:18     ` Jacob Keller
  2018-01-18 21:36     ` Johannes Schindelin
  2018-01-18 18:36   ` [PATCH 10/8] [DO NOT APPLY, but improve?] rebase--interactive: introduce "stop" command Stefan Beller
  1 sibling, 2 replies; 412+ messages in thread
From: Stefan Beller @ 2018-01-18 18:36 UTC (permalink / raw)
  To: johannes.schindelin; +Cc: git, gitster, jacob.keller, Stefan Beller

Up to now each command took a commit as its first argument and ignored
the rest of the line (usually the subject of the commit)

Now that we have commands that take different arguments, clarify each
command by giving the argument list.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 git-rebase--interactive.sh | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 23184c77e8..3cd7446d0b 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -155,17 +155,17 @@ reschedule_last_action () {
 append_todo_help () {
 	gettext "
 Commands:
-p, pick = use commit
-r, reword = use commit, but edit the commit message
-e, edit = use commit, but stop for amending
-s, squash = use commit, but meld into previous commit
-f, fixup = like \"squash\", but discard this commit's log message
-x, exec = run command (the rest of the line) using shell
-d, drop = remove commit
-l, label = label current HEAD with a name
-t, reset = reset HEAD to a label
-b, bud = reset HEAD to the revision labeled 'onto'
-m, merge = create a merge commit using a given commit's message
+p, pick <commit> = use commit
+r, reword <commit> = use commit, but edit the commit message
+e, edit <commit> = use commit, but stop for amending
+s, squash <commit> = use commit, but meld into previous commit
+f, fixup <commit> = like \"squash\", but discard this commit's log message
+x, exec <commit> = run command (the rest of the line) using shell
+d, drop <commit> = remove commit
+l, label <label>= label current HEAD with a name
+t, reset <label> = reset HEAD to a label
+b, bud = reset HEAD to the revision labeled 'onto', no arguments
+m, merge [<label-or-commit>]* = create a merge commit using a given commit's message
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
-- 
2.16.0.rc1.238.g530d649a79-goog


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH 10/8] [DO NOT APPLY, but improve?] rebase--interactive: introduce "stop" command
  2018-01-18 18:36 ` [PATCH 9, 10/8] interactive rebase feedback Stefan Beller
  2018-01-18 18:36   ` [PATCH 9/8] [DO NOT APPLY, but squash?] git-rebase--interactive: clarify arguments Stefan Beller
@ 2018-01-18 18:36   ` Stefan Beller
  2018-01-18 21:20     ` Jacob Keller
  2018-01-18 22:00     ` Johannes Schindelin
  1 sibling, 2 replies; 412+ messages in thread
From: Stefan Beller @ 2018-01-18 18:36 UTC (permalink / raw)
  To: johannes.schindelin; +Cc: git, gitster, jacob.keller, Stefan Beller

Jake suggested using "x false" instead of "edit" for some corner cases.

I do prefer using "x false" for all kinds of things such as stopping
before a commit (edit only let's you stop after a commit), and the
knowledge that "x false" does the least amount of actions behind my back.

We should have that command as well, maybe?

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 git-rebase--interactive.sh |  1 +
 sequencer.c                | 10 ++++++++++
 2 files changed, 11 insertions(+)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 3cd7446d0b..9eac53f0c5 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -166,6 +166,7 @@ l, label <label>= label current HEAD with a name
 t, reset <label> = reset HEAD to a label
 b, bud = reset HEAD to the revision labeled 'onto', no arguments
 m, merge [<label-or-commit>]* = create a merge commit using a given commit's message
+y, stay = stop for  shortcut for
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index 2b4e6b1232..4b3b9fe59d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -782,6 +782,7 @@ enum todo_command {
 	TODO_RESET,
 	TODO_BUD,
 	TODO_MERGE,
+	TODO_STOP,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -803,6 +804,7 @@ static struct {
 	{ 'l', "label" },
 	{ 't', "reset" },
 	{ 'b', "bud" },
+	{ 'y', "stay" },
 	{ 'm', "merge" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
@@ -1307,6 +1309,12 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return 0;
 	}
 
+	if (item->command == TODO_STOP) {
+		item->commit = NULL;
+		item->arg = "";
+		item->arg_len = 0;
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
 	item->arg_len = (int)(eol - item->arg);
@@ -2407,6 +2415,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				/* `current` will be incremented below */
 				todo_list->current = -1;
 			}
+		} else if (item->command == TODO_STOP) {
+			todo_list->current = -1;
 		} else if (item->command == TODO_LABEL)
 			res = do_label(item->arg, item->arg_len);
 		else if (item->command == TODO_RESET)
-- 
2.16.0.rc1.238.g530d649a79-goog


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-18 16:25   ` Jacob Keller
@ 2018-01-18 21:13     ` Johannes Schindelin
  2018-01-18 21:21       ` Jacob Keller
  2018-01-18 21:24     ` Philip Oakley
  2018-01-22 21:25     ` Junio C Hamano
  2 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 21:13 UTC (permalink / raw)
  To: Jacob Keller; +Cc: Git mailing list, Junio C Hamano

Hi Jake,

On Thu, 18 Jan 2018, Jacob Keller wrote:

> On Thu, Jan 18, 2018 at 7:35 AM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > This commit implements the commands to label, and to reset to, given
> > revisions. The syntax is:
> >
> >         label <name>
> >         reset <name>
> >
> > As a convenience shortcut, also to improve readability of the generated
> > todo list, a third command is introduced: bud. It simply resets to the
> > "onto" revision, i.e. the commit onto which we currently rebase.
> >
> 
> The code looks good, but I'm a little wary of adding bud which
> hard-codes a specific label. I suppose it does grant a bit of
> readability to the resulting script... ? It doesn't seem that
> important compared to use using "reset onto"? At least when
> documenting this it should be made clear that the "onto" label is
> special.

Indeed, `bud` helped me more in the earlier times of the Git garden shears
when I had to *introduce* branch structure into the long list of Git for
Windows' patches.

If there are no voices in favor of `bud` other than mine, I will remove
support for it in the next iteration.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 9/8] [DO NOT APPLY, but squash?] git-rebase--interactive: clarify arguments
  2018-01-18 18:36   ` [PATCH 9/8] [DO NOT APPLY, but squash?] git-rebase--interactive: clarify arguments Stefan Beller
@ 2018-01-18 21:18     ` Jacob Keller
  2018-01-18 21:36     ` Johannes Schindelin
  1 sibling, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-01-18 21:18 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Johannes Schindelin, Git mailing list, Junio C Hamano

On Thu, Jan 18, 2018 at 10:36 AM, Stefan Beller <sbeller@google.com> wrote:
> Up to now each command took a commit as its first argument and ignored
> the rest of the line (usually the subject of the commit)
>
> Now that we have commands that take different arguments, clarify each
> command by giving the argument list.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  git-rebase--interactive.sh | 22 +++++++++++-----------
>  1 file changed, 11 insertions(+), 11 deletions(-)
>
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index 23184c77e8..3cd7446d0b 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -155,17 +155,17 @@ reschedule_last_action () {
>  append_todo_help () {
>         gettext "
>  Commands:
> -p, pick = use commit
> -r, reword = use commit, but edit the commit message
> -e, edit = use commit, but stop for amending
> -s, squash = use commit, but meld into previous commit
> -f, fixup = like \"squash\", but discard this commit's log message
> -x, exec = run command (the rest of the line) using shell
> -d, drop = remove commit
> -l, label = label current HEAD with a name
> -t, reset = reset HEAD to a label
> -b, bud = reset HEAD to the revision labeled 'onto'
> -m, merge = create a merge commit using a given commit's message
> +p, pick <commit> = use commit
> +r, reword <commit> = use commit, but edit the commit message
> +e, edit <commit> = use commit, but stop for amending
> +s, squash <commit> = use commit, but meld into previous commit
> +f, fixup <commit> = like \"squash\", but discard this commit's log message
> +x, exec <commit> = run command (the rest of the line) using shell
> +d, drop <commit> = remove commit
> +l, label <label>= label current HEAD with a name
> +t, reset <label> = reset HEAD to a label
> +b, bud = reset HEAD to the revision labeled 'onto', no arguments
> +m, merge [<label-or-commit>]* = create a merge commit using a given commit's message

The merge arguments are complicated, i'm not sure how best to add
clarification here.. I think it might need its own paragraph, since it
takes something like: <commit> <parents to merge>* <possibly message>?

Thanks,
Jake

>
>  These lines can be re-ordered; they are executed from top to bottom.
>  " | git stripspace --comment-lines >>"$todo"
> --
> 2.16.0.rc1.238.g530d649a79-goog
>

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 10/8] [DO NOT APPLY, but improve?] rebase--interactive: introduce "stop" command
  2018-01-18 18:36   ` [PATCH 10/8] [DO NOT APPLY, but improve?] rebase--interactive: introduce "stop" command Stefan Beller
@ 2018-01-18 21:20     ` Jacob Keller
  2018-01-18 22:08       ` Philip Oakley
  2018-01-18 22:00     ` Johannes Schindelin
  1 sibling, 1 reply; 412+ messages in thread
From: Jacob Keller @ 2018-01-18 21:20 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Johannes Schindelin, Git mailing list, Junio C Hamano

On Thu, Jan 18, 2018 at 10:36 AM, Stefan Beller <sbeller@google.com> wrote:
> Jake suggested using "x false" instead of "edit" for some corner cases.
>
> I do prefer using "x false" for all kinds of things such as stopping
> before a commit (edit only let's you stop after a commit), and the
> knowledge that "x false" does the least amount of actions behind my back.
>
> We should have that command as well, maybe?
>


I agree. I use "x false" very often, and I think stop is probably a
better solution since it avoids spawning an extra shell that will just
fail. Not sure if stop implies too much about "stop the whole thing"
as opposed to "stop here and let me do something manual", but I think
it's clear enough.

Thanks,
Jake

> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  git-rebase--interactive.sh |  1 +
>  sequencer.c                | 10 ++++++++++
>  2 files changed, 11 insertions(+)
>
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index 3cd7446d0b..9eac53f0c5 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -166,6 +166,7 @@ l, label <label>= label current HEAD with a name
>  t, reset <label> = reset HEAD to a label
>  b, bud = reset HEAD to the revision labeled 'onto', no arguments
>  m, merge [<label-or-commit>]* = create a merge commit using a given commit's message
> +y, stay = stop for  shortcut for
>
>  These lines can be re-ordered; they are executed from top to bottom.
>  " | git stripspace --comment-lines >>"$todo"
> diff --git a/sequencer.c b/sequencer.c
> index 2b4e6b1232..4b3b9fe59d 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -782,6 +782,7 @@ enum todo_command {
>         TODO_RESET,
>         TODO_BUD,
>         TODO_MERGE,
> +       TODO_STOP,
>         /* commands that do nothing but are counted for reporting progress */
>         TODO_NOOP,
>         TODO_DROP,
> @@ -803,6 +804,7 @@ static struct {
>         { 'l', "label" },
>         { 't', "reset" },
>         { 'b', "bud" },
> +       { 'y', "stay" },
>         { 'm', "merge" },
>         { 0,   "noop" },
>         { 'd', "drop" },
> @@ -1307,6 +1309,12 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>                 return 0;
>         }
>
> +       if (item->command == TODO_STOP) {
> +               item->commit = NULL;
> +               item->arg = "";
> +               item->arg_len = 0;
> +       }
> +
>         end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
>         item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
>         item->arg_len = (int)(eol - item->arg);
> @@ -2407,6 +2415,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>                                 /* `current` will be incremented below */
>                                 todo_list->current = -1;
>                         }
> +               } else if (item->command == TODO_STOP) {
> +                       todo_list->current = -1;
>                 } else if (item->command == TODO_LABEL)
>                         res = do_label(item->arg, item->arg_len);
>                 else if (item->command == TODO_RESET)
> --
> 2.16.0.rc1.238.g530d649a79-goog
>

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-18 21:13     ` Johannes Schindelin
@ 2018-01-18 21:21       ` Jacob Keller
  0 siblings, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-01-18 21:21 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Git mailing list, Junio C Hamano

On Thu, Jan 18, 2018 at 1:13 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Jake,
>
> On Thu, 18 Jan 2018, Jacob Keller wrote:
>
>> On Thu, Jan 18, 2018 at 7:35 AM, Johannes Schindelin
>> <johannes.schindelin@gmx.de> wrote:
>> > This commit implements the commands to label, and to reset to, given
>> > revisions. The syntax is:
>> >
>> >         label <name>
>> >         reset <name>
>> >
>> > As a convenience shortcut, also to improve readability of the generated
>> > todo list, a third command is introduced: bud. It simply resets to the
>> > "onto" revision, i.e. the commit onto which we currently rebase.
>> >
>>
>> The code looks good, but I'm a little wary of adding bud which
>> hard-codes a specific label. I suppose it does grant a bit of
>> readability to the resulting script... ? It doesn't seem that
>> important compared to use using "reset onto"? At least when
>> documenting this it should be made clear that the "onto" label is
>> special.
>
> Indeed, `bud` helped me more in the earlier times of the Git garden shears
> when I had to *introduce* branch structure into the long list of Git for
> Windows' patches.
>
> If there are no voices in favor of `bud` other than mine, I will remove
> support for it in the next iteration.
>
> Ciao,
> Dscho

After I saw more examples, I don't mind it. I do think it's not
strictly necessary, but it seemed to help me with the readability. My
only main concern is that it's limited to a special label.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/8] sequencer: introduce the `merge` command
  2018-01-18 16:31   ` Jacob Keller
@ 2018-01-18 21:22     ` Johannes Schindelin
  2018-01-18 21:26       ` Jacob Keller
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 21:22 UTC (permalink / raw)
  To: Jacob Keller; +Cc: Git mailing list, Junio C Hamano

Hi Jake,

On Thu, 18 Jan 2018, Jacob Keller wrote:

> On Thu, Jan 18, 2018 at 7:35 AM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > This patch is part of the effort to reimplement `--preserve-merges` with
> > a substantially improved design, a design that has been developed in the
> > Git for Windows project to maintain the dozens of Windows-specific patch
> > series on top of upstream Git.
> >
> > The previous patch implemented the `label`, `bud` and `reset` commands
> > to label commits and to reset to a labeled commits. This patch adds the
> > `merge` command, with the following syntax:
> >
> >         merge <commit> <rev> <oneline>
> >
> > The <commit> parameter in this instance is the *original* merge commit,
> > whose author and message will be used for the to-be-created merge
> > commit.
> >
> > The <rev> parameter refers to the (possibly rewritten) revision to
> > merge. Let's see an example of a todo list:
> >
> >         label onto
> >
> >         # Branch abc
> >         bud
> >         pick deadbeef Hello, world!
> >         label abc
> >
> >         bud
> >         pick cafecafe And now for something completely different
> >         merge baaabaaa abc Merge the branch 'abc' into master
> >
> > To support creating *new* merges, i.e. without copying the commit
> > message from an existing commit, use the special value `-` as <commit>
> > parameter (in which case the text after the <rev> parameter is used as
> > commit message):
> >
> >         merge - abc This will be the actual commit message of the merge
> >
> > This comes in handy when splitting a branch into two or more branches.
> >
> 
> Would it be possible to open the editor with the supplied text when
> there's no commit?  The text after <rev> must be oneline only..

I actually want to avoid that because my main use case is fire-and-forget,
i.e. I want to edit only the todo list and then (barring any merge
conflicts) I do not want to edit anything anymore.

But I guess we could special-case the thing where `-` is specified as
"merge commit message provider" and an empty oneline is provided?

> It's difficult to reword merges because of the nature of rebase
> interactive, you can't just re-run the rebase command and use
> "reword".
> 
> I suppose you could cheat by putting in an "edit" command that let you
> create an empty commit with a message...

Or you could "cheat" by adding `exec git commit --amend`...

Seriously again, I have no good idea how to provide an equivalent to the
`reword` verb that would work on merge commits...

Anyone?

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-18 16:25   ` Jacob Keller
  2018-01-18 21:13     ` Johannes Schindelin
@ 2018-01-18 21:24     ` Philip Oakley
  2018-01-18 21:28       ` Jacob Keller
  2018-01-29 20:28       ` Johannes Schindelin
  2018-01-22 21:25     ` Junio C Hamano
  2 siblings, 2 replies; 412+ messages in thread
From: Philip Oakley @ 2018-01-18 21:24 UTC (permalink / raw)
  To: Jacob Keller, Johannes Schindelin; +Cc: Git mailing list, Junio C Hamano

From: "Jacob Keller" <jacob.keller@gmail.com>
> On Thu, Jan 18, 2018 at 7:35 AM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
>> This commit implements the commands to label, and to reset to, given
>> revisions. The syntax is:
>>
>>         label <name>
>>         reset <name>
>>
>> As a convenience shortcut, also to improve readability of the generated
>> todo list, a third command is introduced: bud. It simply resets to the
>> "onto" revision, i.e. the commit onto which we currently rebase.
>>
>
> The code looks good, but I'm a little wary of adding bud which
> hard-codes a specific label. I suppose it does grant a bit of
> readability to the resulting script... ? It doesn't seem that
> important compared to use using "reset onto"? At least when
> documenting this it should be made clear that the "onto" label is
> special.
>
> Thanks,
> Jake.

I'd agree.

The special 'onto' label should be fully documented, and the commit message 
should indicate which patch actually defines it (and all its corner cases 
and fall backs if --onto isn't explicitly given..)

Likewise the choice of 'bud' should be explained with some nice phraseology 
indicating that we are growing the new flowering from the bud, otherwise the 
word is a bit too short and sudden for easy explanation.

Philip 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/8] sequencer: introduce the `merge` command
  2018-01-18 21:22     ` Johannes Schindelin
@ 2018-01-18 21:26       ` Jacob Keller
  0 siblings, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-01-18 21:26 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Git mailing list, Junio C Hamano

On Thu, Jan 18, 2018 at 1:22 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>> Would it be possible to open the editor with the supplied text when
>> there's no commit?  The text after <rev> must be oneline only..
>
> I actually want to avoid that because my main use case is fire-and-forget,
> i.e. I want to edit only the todo list and then (barring any merge
> conflicts) I do not want to edit anything anymore.
>

Agreed, for the case where we copy a commit message, I do not want the
editor either.

> But I guess we could special-case the thing where `-` is specified as
> "merge commit message provider" and an empty oneline is provided?
>

It's for when there is a new merge, for when we are creating a new one
using "-", yes.

>> It's difficult to reword merges because of the nature of rebase
>> interactive, you can't just re-run the rebase command and use
>> "reword".
>>
>> I suppose you could cheat by putting in an "edit" command that let you
>> create an empty commit with a message...
>
> Or you could "cheat" by adding `exec git commit --amend`...
>
> Seriously again, I have no good idea how to provide an equivalent to the
> `reword` verb that would work on merge commits...
>

Given that there is a work around, and I doubt it's that common, I'm
not sure we need one, plus i have no idea what verb to use....

We could allow reword on its own to simply reword the top commit?

That being said, since there's a simple-ish workaruond using "stop",
or "exec git commit --amend" I don't see this as being important
enough to worry about now.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 6/8] sequencer: handle autosquash and post-rewrite for merge commands
  2018-01-18 16:43   ` Jacob Keller
@ 2018-01-18 21:27     ` Johannes Schindelin
  2018-01-18 21:29       ` Jacob Keller
  2018-01-23 20:27     ` Junio C Hamano
  1 sibling, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 21:27 UTC (permalink / raw)
  To: Jacob Keller; +Cc: Git mailing list, Junio C Hamano

Hi Jake,

On Thu, 18 Jan 2018, Jacob Keller wrote:

> On Thu, Jan 18, 2018 at 7:35 AM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > In the previous patches, we implemented the basic functionality of the
> > `git rebase -i --recreate-merges` command, in particular the `merge`
> > command to create merge commits in the sequencer.
> >
> > The interactive rebase is a lot more these days, though, than a simple
> > cherry-pick in a loop. For example, it calls the post-rewrite hook (if
> > any) after rebasing with a mapping of the old->new commits. And the
> > interactive rebase also supports the autosquash mode, where commits
> > whose oneline is of the form `fixup! <oneline>` or `squash! <oneline>`
> > are rearranged to amend commits whose oneline they match.
> >
> > This patch implements the post-rewrite and autosquash handling for the
> > `merge` command we just introduced. The other commands that were added
> > recently (`label`, `reset` and `bud`) do not create new commits,
> > therefore post-rewrite & autosquash do not need to handle them.
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  refs.c                            |  3 ++-
> >  sequencer.c                       | 10 +++++++---
> >  t/t3430-rebase-recreate-merges.sh | 39 +++++++++++++++++++++++++++++++++++++++
> >  3 files changed, 48 insertions(+), 4 deletions(-)
> >
> > diff --git a/refs.c b/refs.c
> > index 20ba82b4343..e8b84c189ff 100644
> > --- a/refs.c
> > +++ b/refs.c
> > @@ -600,7 +600,8 @@ int dwim_log(const char *str, int len, struct object_id *oid, char **log)
> >  static int is_per_worktree_ref(const char *refname)
> >  {
> >         return !strcmp(refname, "HEAD") ||
> > -               starts_with(refname, "refs/bisect/");
> > +               starts_with(refname, "refs/bisect/") ||
> > +               starts_with(refname, "refs/rewritten/");
> >  }
> 
> Would this part make more sense to move into the commit that
> introduces writing these refs, or does it only matter once you start
> this step here?
> 
> >
> >  static int is_pseudoref_syntax(const char *refname)
> > diff --git a/sequencer.c b/sequencer.c
> > index 1bef16647b4..b63bfb9a141 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -2413,10 +2413,13 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
> >                         res = do_reset(item->arg, item->arg_len);
> >                 else if (item->command == TODO_BUD)
> >                         res = do_reset("onto", 4);
> > -               else if (item->command == TODO_MERGE)
> > +               else if (item->command == TODO_MERGE) {
> >                         res = do_merge(item->commit,
> >                                        item->arg, item->arg_len, opts);
> > -               else if (!is_noop(item->command))
> > +                       if (item->commit)
> > +                               record_in_rewritten(&item->commit->object.oid,
> > +                                                   peek_command(todo_list, 1));
> > +               } else if (!is_noop(item->command))
> >                         return error(_("unknown command %d"), item->command);
> >
> >                 todo_list->current++;
> > @@ -3556,7 +3559,8 @@ int rearrange_squash(void)
> >                 struct subject2item_entry *entry;
> >
> >                 next[i] = tail[i] = -1;
> > -               if (item->command >= TODO_EXEC) {
> > +               if (item->command >= TODO_EXEC &&
> > +                   (item->command != TODO_MERGE || !item->commit)) {
> >                         subjects[i] = NULL;
> >                         continue;
> >                 }
> > diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
> > index 46ae52f88b3..76e615bd7c1 100755
> > --- a/t/t3430-rebase-recreate-merges.sh
> > +++ b/t/t3430-rebase-recreate-merges.sh
> > @@ -143,4 +143,43 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
> >         EOF
> >  '
> >
> > +test_expect_success 'refs/rewritten/* is worktree-local' '
> > +       git worktree add wt &&
> > +       cat >wt/script-from-scratch <<-\EOF &&
> > +       label xyz
> > +       exec GIT_DIR=../.git git rev-parse --verify refs/rewritten/xyz >a || :
> > +       exec git rev-parse --verify refs/rewritten/xyz >b
> > +       EOF
> > +
> > +       test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
> > +       git -C wt rebase -i HEAD &&
> > +       test_must_be_empty wt/a &&
> > +       test_cmp_rev HEAD "$(cat wt/b)"
> > +'
> > +
> 
> Same for the test here, I can't figure out why this is necessary in
> this patch as opposed to the patch which first introduced the
> refs/rewritten/<label> refs.

Woops. This was its own commit, and must have been accidentally squashed
during one of my rebases. Will re-introduce it; this is roughly what it
will look like:

-- snipsnap --
Author: Johannes Schindelin <johannes.schindelin@gmx.de>
Subject: sequencer: make refs generated by the `label` command worktree-local

This allows for rebases to be run in parallel in separate worktrees
(think: interrupted in the middle of one rebase, being asked to perform
a different rebase, adding a separate worktree just for that job).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

diff --git a/refs.c b/refs.c
index 20ba82b4343..e8b84c189ff 100644
--- a/refs.c
+++ b/refs.c
@@ -600,7 +600,8 @@ int dwim_log(const char *str, int len, struct object_id *oid, char **log)
 static int is_per_worktree_ref(const char *refname)
 {
 	return !strcmp(refname, "HEAD") ||
-		starts_with(refname, "refs/bisect/");
+		starts_with(refname, "refs/bisect/") ||
+		starts_with(refname, "refs/rewritten/");
 }
 
 static int is_pseudoref_syntax(const char *refname)
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index f5e476d948b..50a8eefc81e 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -115,4 +115,18 @@ test_expect_success 'generate correct todo list' '
 	test_cmp expect output
 '
 
+test_expect_success 'refs/rewritten/* is worktree-local' '
+	git worktree add wt &&
+	cat >wt/script-from-scratch <<-\EOF &&
+	label xyz
+	exec GIT_DIR=../.git git rev-parse --verify refs/rewritten/xyz >a || :
+	exec git rev-parse --verify refs/rewritten/xyz >b
+	EOF
+
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	git -C wt rebase -i HEAD &&
+	test_must_be_empty wt/a &&
+	test_cmp_rev HEAD "$(cat wt/b)"
+'
+
 test_done

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-18 21:24     ` Philip Oakley
@ 2018-01-18 21:28       ` Jacob Keller
  2018-01-29 20:28       ` Johannes Schindelin
  1 sibling, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-01-18 21:28 UTC (permalink / raw)
  To: Philip Oakley; +Cc: Johannes Schindelin, Git mailing list, Junio C Hamano

On Thu, Jan 18, 2018 at 1:24 PM, Philip Oakley <philipoakley@iee.org> wrote:
> From: "Jacob Keller" <jacob.keller@gmail.com>
>
>> On Thu, Jan 18, 2018 at 7:35 AM, Johannes Schindelin
>> <johannes.schindelin@gmx.de> wrote:
>>>
>>> This commit implements the commands to label, and to reset to, given
>>> revisions. The syntax is:
>>>
>>>         label <name>
>>>         reset <name>
>>>
>>> As a convenience shortcut, also to improve readability of the generated
>>> todo list, a third command is introduced: bud. It simply resets to the
>>> "onto" revision, i.e. the commit onto which we currently rebase.
>>>
>>
>> The code looks good, but I'm a little wary of adding bud which
>> hard-codes a specific label. I suppose it does grant a bit of
>> readability to the resulting script... ? It doesn't seem that
>> important compared to use using "reset onto"? At least when
>> documenting this it should be made clear that the "onto" label is
>> special.
>>
>> Thanks,
>> Jake.
>
>
> I'd agree.
>
> The special 'onto' label should be fully documented, and the commit message
> should indicate which patch actually defines it (and all its corner cases
> and fall backs if --onto isn't explicitly given..)

I don't think it actually relates to "--onto" but rather to simply
using "label onto" in your sequencer script allows bud to work, and
simply shortens the overall work necessary. It's equivalent to "reset
onto" if I understand.

>
> Likewise the choice of 'bud' should be explained with some nice phraseology
> indicating that we are growing the new flowering from the bud, otherwise the
> word is a bit too short and sudden for easy explanation.
>
> Philip

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 6/8] sequencer: handle autosquash and post-rewrite for merge commands
  2018-01-18 21:27     ` Johannes Schindelin
@ 2018-01-18 21:29       ` Jacob Keller
  0 siblings, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-01-18 21:29 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Git mailing list, Junio C Hamano

>> Same for the test here, I can't figure out why this is necessary in
>> this patch as opposed to the patch which first introduced the
>> refs/rewritten/<label> refs.
>
> Woops. This was its own commit, and must have been accidentally squashed
> during one of my rebases. Will re-introduce it;

Yep that makes sense!

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 9/8] [DO NOT APPLY, but squash?] git-rebase--interactive: clarify arguments
  2018-01-18 18:36   ` [PATCH 9/8] [DO NOT APPLY, but squash?] git-rebase--interactive: clarify arguments Stefan Beller
  2018-01-18 21:18     ` Jacob Keller
@ 2018-01-18 21:36     ` Johannes Schindelin
  2018-01-18 21:58       ` Stefan Beller
  2018-01-19 20:30       ` Junio C Hamano
  1 sibling, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 21:36 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, gitster, jacob.keller

Hi Stefan,

On Thu, 18 Jan 2018, Stefan Beller wrote:

> Up to now each command took a commit as its first argument and ignored
> the rest of the line (usually the subject of the commit)
> 
> Now that we have commands that take different arguments, clarify each
> command by giving the argument list.
> 
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  git-rebase--interactive.sh | 22 +++++++++++-----------
>  1 file changed, 11 insertions(+), 11 deletions(-)
> 
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index 23184c77e8..3cd7446d0b 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -155,17 +155,17 @@ reschedule_last_action () {
>  append_todo_help () {
>  	gettext "
>  Commands:
> -p, pick = use commit
> -r, reword = use commit, but edit the commit message
> -e, edit = use commit, but stop for amending
> -s, squash = use commit, but meld into previous commit
> -f, fixup = like \"squash\", but discard this commit's log message
> -x, exec = run command (the rest of the line) using shell
> -d, drop = remove commit
> -l, label = label current HEAD with a name
> -t, reset = reset HEAD to a label
> -b, bud = reset HEAD to the revision labeled 'onto'
> -m, merge = create a merge commit using a given commit's message
> +p, pick <commit> = use commit
> +r, reword <commit> = use commit, but edit the commit message
> +e, edit <commit> = use commit, but stop for amending
> +s, squash <commit> = use commit, but meld into previous commit
> +f, fixup <commit> = like \"squash\", but discard this commit's log message
> +x, exec <commit> = run command (the rest of the line) using shell
> +d, drop <commit> = remove commit
> +l, label <label>= label current HEAD with a name
> +t, reset <label> = reset HEAD to a label
> +b, bud = reset HEAD to the revision labeled 'onto', no arguments
> +m, merge [<label-or-commit>]* = create a merge commit using a given commit's message

Good idea! I would rather do it as an introductory patch (that only
converts the existing list).

As to `merge`: it is a bit more complicated ;-)

	m, merge <original-merge-commit> ( <label> | "<label>..." ) [<oneline>]
		create a merge commit using the original merge commit's
		message (or the oneline, if "-" is given). Use a quoted
		list of commits to be merged for octopus merges.

Thanks,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges
  2018-01-18 15:35 ` [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
@ 2018-01-18 21:39   ` Philip Oakley
  2018-01-19 10:34   ` Eric Sunshine
  2018-01-23 20:03   ` Junio C Hamano
  2 siblings, 0 replies; 412+ messages in thread
From: Philip Oakley @ 2018-01-18 21:39 UTC (permalink / raw)
  To: Johannes Schindelin, git; +Cc: Junio C Hamano, Jacob Keller

From: "Johannes Schindelin" <johannes.schindelin@gmx.de>
> The sequencer just learned a new commands intended to recreate branch
> structure (similar in spirit to --preserve-merges, but with a
> substantially less-broken design).
>
> Let's allow the rebase--helper to generate todo lists making use of
> these commands, triggered by the new --recreate-merges option. For a
> commit topology like this:
>
> A - B - C
>   \   /
>     D

Could the topology include the predecessor for context. Alo it is easy for 
readers to become confused between the arcs of the graphs and the nodes of 
the graphs, such that we confuse 'commits as patches' with 'commits as 
snapshots'. It might need an 'Aa' distinction between the two types, 
especially around merges and potential evilness.

>
> the generated todo list would look like this:
>
> # branch D
> pick 0123 A
> label branch-point
> pick 1234 D
> label D
>
> reset branch-point
> pick 2345 B
> merge 3456 D C
>
> To keep things simple, we first only implement support for merge commits
> with exactly two parents, leaving support for octopus merges to a later
> patch in this patch series.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> builtin/rebase--helper.c |   4 +-
> sequencer.c              | 343 
> ++++++++++++++++++++++++++++++++++++++++++++++-
> sequencer.h              |   1 +
> 3 files changed, 345 insertions(+), 3 deletions(-)
>
> diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
> index 7daee544b7b..a34ab5c0655 100644
> --- a/builtin/rebase--helper.c
> +++ b/builtin/rebase--helper.c
> @@ -12,7 +12,7 @@ static const char * const builtin_rebase_helper_usage[] 
> = {
> int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
> {
>  struct replay_opts opts = REPLAY_OPTS_INIT;
> - unsigned flags = 0, keep_empty = 0;
> + unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
>  int abbreviate_commands = 0;
>  enum {
>  CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
> @@ -22,6 +22,7 @@ int cmd_rebase__helper(int argc, const char **argv, 
> const char *prefix)
>  struct option options[] = {
>  OPT_BOOL(0, "ff", &opts.allow_ff, N_("allow fast-forward")),
>  OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
> + OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge 
> commits")),
>  OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
>  CONTINUE),
>  OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
> @@ -55,6 +56,7 @@ int cmd_rebase__helper(int argc, const char **argv, 
> const char *prefix)
>
>  flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
>  flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
> + flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
>  flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
>
>  if (command == CONTINUE && argc == 1)
> diff --git a/sequencer.c b/sequencer.c
> index a96255426e7..1bef16647b4 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -23,6 +23,8 @@
> #include "hashmap.h"
> #include "unpack-trees.h"
> #include "worktree.h"
> +#include "oidmap.h"
> +#include "oidset.h"
>
> #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
>
> @@ -2785,6 +2787,335 @@ void append_signoff(struct strbuf *msgbuf, int 
> ignore_footer, unsigned flag)
>  strbuf_release(&sob);
> }
>
> +struct labels_entry {
> + struct hashmap_entry entry;
> + char label[FLEX_ARRAY];
> +};
> +
> +static int labels_cmp(const void *fndata, const struct labels_entry *a,
> +       const struct labels_entry *b, const void *key)
> +{
> + return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
> +}
> +
> +struct string_entry {
> + struct oidmap_entry entry;
> + char string[FLEX_ARRAY];
> +};
> +
> +struct label_state {
> + struct oidmap commit2label;
> + struct hashmap labels;
> + struct strbuf buf;
> +};
> +
> +static const char *label_oid(struct object_id *oid, const char *label,
> +      struct label_state *state)
> +{
> + struct labels_entry *labels_entry;
> + struct string_entry *string_entry;
> + struct object_id dummy;
> + size_t len;
> + int i;
> +
> + string_entry = oidmap_get(&state->commit2label, oid);
> + if (string_entry)
> + return string_entry->string;
> +
> + /*
> + * For "uninteresting" commits, i.e. commits that are not to be
> + * rebased, and which can therefore not be labeled, we use a unique
> + * abbreviation of the commit name. This is slightly more complicated
> + * than calling find_unique_abbrev() because we also need to make
> + * sure that the abbreviation does not conflict with any other
> + * label.
> + *
> + * We disallow "interesting" commits to be labeled by a string that
> + * is a valid full-length hash, to ensure that we always can find an
> + * abbreviation for any uninteresting commit's names that does not
> + * clash with any other label.
> + */
> + if (!label) {
> + char *p;
> +
> + strbuf_reset(&state->buf);
> + strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
> + label = p = state->buf.buf;
> +
> + find_unique_abbrev_r(p, oid->hash, default_abbrev);
> +
> + /*
> + * We may need to extend the abbreviated hash so that there is
> + * no conflicting label.
> + */
> + if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
> + size_t i = strlen(p) + 1;
> +
> + oid_to_hex_r(p, oid);
> + for (; i < GIT_SHA1_HEXSZ; i++) {
> + char save = p[i];
> + p[i] = '\0';
> + if (!hashmap_get_from_hash(&state->labels,
> +    strihash(p), p))
> + break;
> + p[i] = save;
> + }
> + }
> + } else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
> +     !get_oid_hex(label, &dummy)) ||
> +    hashmap_get_from_hash(&state->labels,
> + strihash(label), label)) {
> + /*
> + * If the label already exists, or if the label is a valid full
> + * OID, we append a dash and a number to make it unique.
> + */
> + struct strbuf *buf = &state->buf;
> +
> + strbuf_reset(buf);
> + strbuf_add(buf, label, len);
> +
> + for (i = 2; ; i++) {
> + strbuf_setlen(buf, len);
> + strbuf_addf(buf, "-%d", i);
> + if (!hashmap_get_from_hash(&state->labels,
> +    strihash(buf->buf),
> +    buf->buf))
> + break;
> + }
> +
> + label = buf->buf;
> + }
> +
> + FLEX_ALLOC_STR(labels_entry, label, label);
> + hashmap_entry_init(labels_entry, strihash(label));
> + hashmap_add(&state->labels, labels_entry);
> +
> + FLEX_ALLOC_STR(string_entry, string, label);
> + oidcpy(&string_entry->entry.oid, oid);
> + oidmap_put(&state->commit2label, string_entry);
> +
> + return string_entry->string;
> +}
> +
> +static int make_script_with_merges(struct pretty_print_context *pp,
> +    struct rev_info *revs, FILE *out,
> +    unsigned flags)
> +{
> + int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
> + struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
> + struct strbuf label = STRBUF_INIT;
> + struct commit_list *commits = NULL, **tail = &commits, *iter;
> + struct commit_list *tips = NULL, **tips_tail = &tips;
> + struct commit *commit;
> + struct oidmap commit2todo = OIDMAP_INIT;
> + struct string_entry *entry;
> + struct oidset interesting = OIDSET_INIT, child_seen = OIDSET_INIT,
> + shown = OIDSET_INIT;
> + struct label_state state = { OIDMAP_INIT, { NULL }, STRBUF_INIT };
> +
> + int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
> + const char *p = abbr ? "p" : "pick", *l = abbr ? "l" : "label",
> + *t = abbr ? "t" : "reset", *b = abbr ? "b" : "bud",
> + *m = abbr ? "m" : "merge";
> +
> + oidmap_init(&commit2todo, 0);
> + oidmap_init(&state.commit2label, 0);
> + hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
> + strbuf_init(&state.buf, 32);
> +
> + if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
> + struct object_id *oid = &revs->cmdline.rev[0].item->oid;
> + FLEX_ALLOC_STR(entry, string, "onto");

"onto" Needs documentation / commit message?


> + oidcpy(&entry->entry.oid, oid);
> + oidmap_put(&state.commit2label, entry);
> + }
> +
> + /*
> + * First phase:
> + * - get onelines for all commits
> + * - gather all branch tips (i.e. 2nd or later parents of merges)
> + * - label all branch tips
> + */
> + while ((commit = get_revision(revs))) {
> + struct commit_list *to_merge;
> + int is_octopus;
> + const char *p1, *p2;
> + struct object_id *oid;
> +
> + tail = &commit_list_insert(commit, tail)->next;
> + oidset_insert(&interesting, &commit->object.oid);
> +
> + if ((commit->object.flags & PATCHSAME))
> + continue;
> +
> + strbuf_reset(&oneline);
> + pretty_print_commit(pp, commit, &oneline);
> +
> + to_merge = commit->parents ? commit->parents->next : NULL;
> + if (!to_merge) {
> + /* non-merge commit: easy case */
> + strbuf_reset(&buf);
> + if (!keep_empty && is_original_commit_empty(commit))
> + strbuf_addf(&buf, "%c ", comment_line_char);
> + strbuf_addf(&buf, "%s %s %s", p,
> +     oid_to_hex(&commit->object.oid),
> +     oneline.buf);
> +
> + FLEX_ALLOC_STR(entry, string, buf.buf);
> + oidcpy(&entry->entry.oid, &commit->object.oid);
> + oidmap_put(&commit2todo, entry);
> +
> + continue;
> + }
> +
> + is_octopus = to_merge && to_merge->next;
> +
> + if (is_octopus)
> + BUG("Octopus merges not yet supported");
> +
> + /* Create a label */
> + strbuf_reset(&label);
> + if (skip_prefix(oneline.buf, "Merge ", &p1) &&
> +     (p1 = strchr(p1, '\'')) &&
> +     (p2 = strchr(++p1, '\'')))
> + strbuf_add(&label, p1, p2 - p1);
> + else if (skip_prefix(oneline.buf, "Merge pull request ",
> +      &p1) &&
> + (p1 = strstr(p1, " from ")))
> + strbuf_addstr(&label, p1 + strlen(" from "));
> + else
> + strbuf_addbuf(&label, &oneline);
> +
> + for (p1 = label.buf; *p1; p1++)
> + if (isspace(*p1))
> + *(char *)p1 = '-';
> +
> + strbuf_reset(&buf);
> + strbuf_addf(&buf, "%s %s", m, oid_to_hex(&commit->object.oid));
> +
> + /* label the tip of merged branch */
> + oid = &to_merge->item->object.oid;
> + strbuf_addch(&buf, ' ');
> +
> + if (!oidset_contains(&interesting, oid))
> + strbuf_addstr(&buf, label_oid(oid, NULL, &state));
> + else {
> + tips_tail = &commit_list_insert(to_merge->item,
> + tips_tail)->next;
> +
> + strbuf_addstr(&buf, label_oid(oid, label.buf, &state));
> + }
> + strbuf_addf(&buf, " %s", oneline.buf);
> +
> + FLEX_ALLOC_STR(entry, string, buf.buf);
> + oidcpy(&entry->entry.oid, &commit->object.oid);
> + oidmap_put(&commit2todo, entry);
> + }
> +
> + /*
> + * Second phase:
> + * - label branch points
> + * - add HEAD to the branch tips
> + */
> + for (iter = commits; iter; iter = iter->next) {
> + struct commit_list *parent = iter->item->parents;
> + for (; parent; parent = parent->next) {
> + struct object_id *oid = &parent->item->object.oid;
> + if (!oidset_contains(&interesting, oid))
> + continue;
> + if (!oidset_contains(&child_seen, oid))
> + oidset_insert(&child_seen, oid);
> + else
> + label_oid(oid, "branch-point", &state);
> + }
> +
> + /* Add HEAD as implict "tip of branch" */
> + if (!iter->next)
> + tips_tail = &commit_list_insert(iter->item,
> + tips_tail)->next;
> + }
> +
> + /*
> + * Third phase: output the todo list. This is a bit tricky, as we
> + * want to avoid jumping back and forth between revisions. To
> + * accomplish that goal, we walk backwards from the branch tips,
> + * gathering commits not yet shown, reversing the list on the fly,
> + * then outputting that list (labeling revisions as needed).
> + */
> + fprintf(out, "%s onto\n", l);
> + for (iter = tips; iter; iter = iter->next) {
> + struct commit_list *list = NULL, *iter2;
> +
> + commit = iter->item;
> + if (oidset_contains(&shown, &commit->object.oid))
> + continue;
> + entry = oidmap_get(&state.commit2label, &commit->object.oid);
> +
> + if (entry)
> + fprintf(out, "\n# Branch %s\n", entry->string);
> + else
> + fprintf(out, "\n");
> +
> + while (oidset_contains(&interesting, &commit->object.oid) &&
> +        !oidset_contains(&shown, &commit->object.oid)) {
> + commit_list_insert(commit, &list);
> + if (!commit->parents) {
> + commit = NULL;
> + break;
> + }
> + commit = commit->parents->item;
> + }
> +
> + if (!commit)
> + fprintf(out, "%s\n", b);
> + else {
> + const char *to = NULL;
> +
> + entry = oidmap_get(&state.commit2label,
> +    &commit->object.oid);
> + if (entry)
> + to = entry->string;
> +
> + if (!to || !strcmp("onto", to))
> + fprintf(out, "%s\n", b);
> + else {
> + strbuf_reset(&oneline);
> + pretty_print_commit(pp, commit, &oneline);
> + fprintf(out, "%s %s %s\n",
> + t, to, oneline.buf);
> + }
> + }
> +
> + for (iter2 = list; iter2; iter2 = iter2->next) {
> + struct object_id *oid = &iter2->item->object.oid;
> + entry = oidmap_get(&commit2todo, oid);
> + /* only show if not already upstream */
> + if (entry)
> + fprintf(out, "%s\n", entry->string);
> + entry = oidmap_get(&state.commit2label, oid);
> + if (entry)
> + fprintf(out, "%s %s\n", l, entry->string);
> + oidset_insert(&shown, oid);
> + }
> +
> + free_commit_list(list);
> + }
> +
> + free_commit_list(commits);
> + free_commit_list(tips);
> +
> + strbuf_release(&label);
> + strbuf_release(&oneline);
> + strbuf_release(&buf);
> +
> + oidmap_free(&commit2todo, 1);
> + oidmap_free(&state.commit2label, 1);
> + hashmap_free(&state.labels, 1);
> + strbuf_release(&state.buf);
> +
> + return 0;
> +}
> +
> int sequencer_make_script(FILE *out, int argc, const char **argv,
>    unsigned flags)
> {
> @@ -2795,11 +3126,16 @@ int sequencer_make_script(FILE *out, int argc, 
> const char **argv,
>  struct commit *commit;
>  int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
>  const char *insn = flags & TODO_LIST_ABBREVIATE_CMDS ? "p" : "pick";
> + int recreate_merges = flags & TODO_LIST_RECREATE_MERGES;
>
>  init_revisions(&revs, NULL);
>  revs.verbose_header = 1;
> - revs.max_parents = 1;
> - revs.cherry_pick = 1;
> + if (recreate_merges)
> + revs.cherry_mark = 1;
> + else {
> + revs.max_parents = 1;
> + revs.cherry_pick = 1;
> + }
>  revs.limited = 1;
>  revs.reverse = 1;
>  revs.right_only = 1;
> @@ -2823,6 +3159,9 @@ int sequencer_make_script(FILE *out, int argc, const 
> char **argv,
>  if (prepare_revision_walk(&revs) < 0)
>  return error(_("make_script: error preparing revisions"));
>
> + if (recreate_merges)
> + return make_script_with_merges(&pp, &revs, out, flags);
> +
>  while ((commit = get_revision(&revs))) {
>  strbuf_reset(&buf);
>  if (!keep_empty && is_original_commit_empty(commit))
> diff --git a/sequencer.h b/sequencer.h
> index 81f6d7d393f..11d1ac925ef 100644
> --- a/sequencer.h
> +++ b/sequencer.h
> @@ -48,6 +48,7 @@ int sequencer_remove_state(struct replay_opts *opts);
> #define TODO_LIST_KEEP_EMPTY (1U << 0)
> #define TODO_LIST_SHORTEN_IDS (1U << 1)
> #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
> +#define TODO_LIST_RECREATE_MERGES (1U << 3)
> int sequencer_make_script(FILE *out, int argc, const char **argv,
>    unsigned flags);
>
> -- 
> 2.15.1.windows.2.1430.ga56c4f9e2a9
>
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 9/8] [DO NOT APPLY, but squash?] git-rebase--interactive: clarify arguments
  2018-01-18 21:36     ` Johannes Schindelin
@ 2018-01-18 21:58       ` Stefan Beller
  2018-01-19 20:30       ` Junio C Hamano
  1 sibling, 0 replies; 412+ messages in thread
From: Stefan Beller @ 2018-01-18 21:58 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Jacob Keller

On Thu, Jan 18, 2018 at 1:36 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:

> Good idea! I would rather do it as an introductory patch (that only
> converts the existing list).

That makes sense.

>
> As to `merge`: it is a bit more complicated ;-)
>
>         m, merge <original-merge-commit> ( <label> | "<label>..." ) [<oneline>]
>                 create a merge commit using the original merge commit's
>                 message (or the oneline, if "-" is given). Use a quoted
>                 list of commits to be merged for octopus merges.

That makes sense. Thanks for being precise at the the user facing UX here.

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 10/8] [DO NOT APPLY, but improve?] rebase--interactive: introduce "stop" command
  2018-01-18 18:36   ` [PATCH 10/8] [DO NOT APPLY, but improve?] rebase--interactive: introduce "stop" command Stefan Beller
  2018-01-18 21:20     ` Jacob Keller
@ 2018-01-18 22:00     ` Johannes Schindelin
  2018-01-18 22:09       ` Stefan Beller
  1 sibling, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-18 22:00 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, gitster, jacob.keller

Hi Stefan,

On Thu, 18 Jan 2018, Stefan Beller wrote:

> Jake suggested using "x false" instead of "edit" for some corner cases.
> 
> I do prefer using "x false" for all kinds of things such as stopping
> before a commit (edit only let's you stop after a commit), and the
> knowledge that "x false" does the least amount of actions behind my back.
> 
> We should have that command as well, maybe?
> 
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  git-rebase--interactive.sh |  1 +
>  sequencer.c                | 10 ++++++++++
>  2 files changed, 11 insertions(+)
> 
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index 3cd7446d0b..9eac53f0c5 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -166,6 +166,7 @@ l, label <label>= label current HEAD with a name
>  t, reset <label> = reset HEAD to a label
>  b, bud = reset HEAD to the revision labeled 'onto', no arguments
>  m, merge [<label-or-commit>]* = create a merge commit using a given commit's message
> +y, stay = stop for  shortcut for
>  
>  These lines can be re-ordered; they are executed from top to bottom.
>  " | git stripspace --comment-lines >>"$todo"
> diff --git a/sequencer.c b/sequencer.c
> index 2b4e6b1232..4b3b9fe59d 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -782,6 +782,7 @@ enum todo_command {
>  	TODO_RESET,
>  	TODO_BUD,
>  	TODO_MERGE,
> +	TODO_STOP,

I see that your original idea was "stop", but then you probably realized
that there would be no good abbreviation for that, and changed your mind.

Personally, I would have called it `break`...

> @@ -2407,6 +2415,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  				/* `current` will be incremented below */
>  				todo_list->current = -1;
>  			}
> +		} else if (item->command == TODO_STOP) {
> +			todo_list->current = -1;

That is incorrect, it will most likely write an unexpected `done` file.

Did you mean `return 0` instead?

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 8/8] rebase -i: introduce --recreate-merges=no-rebase-cousins
  2018-01-18 15:36 ` [PATCH 8/8] rebase -i: introduce --recreate-merges=no-rebase-cousins Johannes Schindelin
@ 2018-01-18 22:00   ` Philip Oakley
  2018-01-29 20:42     ` Johannes Schindelin
  2018-01-20  1:09   ` Eric Sunshine
  1 sibling, 1 reply; 412+ messages in thread
From: Philip Oakley @ 2018-01-18 22:00 UTC (permalink / raw)
  To: Johannes Schindelin, git; +Cc: Junio C Hamano, Jacob Keller

From: "Johannes Schindelin" <johannes.schindelin@gmx.de>
> This one is a bit tricky to explain, so let's try with a diagram:
>
>        C
>      /   \
> A - B - E - F
>  \   /
>    D
>
> To illustrate what this new mode is all about, let's consider what
> happens upon `git rebase -i --recreate-merges B`, in particular to
> the commit `D`. In the default mode, the new branch structure is:
>
>       --- C' --
>      /         \
> A - B ------ E' - F'
>      \    /
>        D'
>
> This is not really preserving the branch topology from before! The
> reason is that the commit `D` does not have `B` as ancestor, and
> therefore it gets rebased onto `B`.
>
> However, when recreating branch structure, there are legitimate use
> cases where one might want to preserve the branch points of commits that
> do not descend from the <upstream> commit that was passed to the rebase
> command, e.g. when a branch from core Git's `next` was merged into Git
> for Windows' master we will not want to rebase those commits on top of a
> Windows-specific commit. In the example above, the desired outcome would
> look like this:
>
>       --- C' --
>      /         \
> A - B ------ E' - F'
>  \        /
>   -- D' --

I'm not understanding this. I see that D properly starts from A, but don't 
see why it is now D'. Surely it's unchanged.
Maybe it's the arc/node confusion. Maybe even spell out that the rebased 
commits from the command are B..HEAD, but that includes D, which may not be 
what folk had expected. (not even sure if the reflog comes into determining 
merge-bases here..)

I do think an exact definition is needed (e.g. via --ancestry-path or its 
equivalent?).

>
> Let's introduce the term "cousins" for such commits ("D" in the
> example), and the "no-rebase-cousins" mode of the merge-recreating
> rebase, to help those use cases.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> Documentation/git-rebase.txt      |  7 ++++++-
> builtin/rebase--helper.c          |  9 ++++++++-
> git-rebase--interactive.sh        |  1 +
> git-rebase.sh                     | 12 +++++++++++-
> sequencer.c                       |  4 ++++
> sequencer.h                       |  8 ++++++++
> t/t3430-rebase-recreate-merges.sh | 23 +++++++++++++++++++++++
> 7 files changed, 61 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> index 1d061373288..ac07a5c3fc9 100644
> --- a/Documentation/git-rebase.txt
> +++ b/Documentation/git-rebase.txt
> @@ -368,10 +368,15 @@ The commit list format can be changed by setting the 
> configuration option
> rebase.instructionFormat.  A customized instruction format will 
> automatically
> have the long commit hash prepended to the format.
>
> ---recreate-merges::
> +--recreate-merges[=(rebase-cousins|no-rebase-cousins)]::
>  Recreate merge commits instead of flattening the history by replaying
>  merges. Merge conflict resolutions or manual amendments to merge
>  commits are not preserved.
> ++
> +By default, or when `rebase-cousins` was specified, commits which do not 
> have
> +`<upstream>` as direct ancestor are rebased onto `<upstream>` (or 
> `<onto>`,
> +if specified). If the `rebase-cousins` mode is turned off, such commits 
> will
> +retain their original branch point.
>
> -p::
> --preserve-merges::
> diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
> index a34ab5c0655..ef08fef4d14 100644
> --- a/builtin/rebase--helper.c
> +++ b/builtin/rebase--helper.c
> @@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, 
> const char *prefix)
> {
>  struct replay_opts opts = REPLAY_OPTS_INIT;
>  unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
> - int abbreviate_commands = 0;
> + int abbreviate_commands = 0, no_rebase_cousins = -1;
>  enum {
>  CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
>  CHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH,
> @@ -23,6 +23,8 @@ int cmd_rebase__helper(int argc, const char **argv, 
> const char *prefix)
>  OPT_BOOL(0, "ff", &opts.allow_ff, N_("allow fast-forward")),
>  OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
>  OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge 
> commits")),
> + OPT_BOOL(0, "no-rebase-cousins", &no_rebase_cousins,
> + N_("keep original branch points of cousins")),
>  OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
>  CONTINUE),
>  OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
> @@ -57,8 +59,13 @@ int cmd_rebase__helper(int argc, const char **argv, 
> const char *prefix)
>  flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
>  flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
>  flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
> + flags |= no_rebase_cousins > 0 ? TODO_LIST_NO_REBASE_COUSINS : 0;
>  flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
>
> + if (no_rebase_cousins >= 0&& !recreate_merges)
> + warning(_("--[no-]rebase-cousins has no effect without "
> +   "--recreate-merges"));
> +
>  if (command == CONTINUE && argc == 1)
>  return !!sequencer_continue(&opts);
>  if (command == ABORT && argc == 1)
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index 3459ec5a018..23184c77e88 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -901,6 +901,7 @@ if test t != "$preserve_merges"
> then
>  git rebase--helper --make-script ${keep_empty:+--keep-empty} \
>  ${recreate_merges:+--recreate-merges} \
> + ${no_rebase_cousins:+--no-rebase-cousins} \
>  $revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
>  die "$(gettext "Could not generate todo list")"
> else
> diff --git a/git-rebase.sh b/git-rebase.sh
> index d69bc7d0e0d..3403b1416a8 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -17,7 +17,7 @@ q,quiet!           be quiet. implies --no-stat
> autostash          automatically stash/stash pop before and after
> fork-point         use 'merge-base --fork-point' to refine upstream
> onto=!             rebase onto given branch instead of upstream
> -recreate-merges!   try to recreate merges instead of skipping them
> +recreate-merges?   try to recreate merges instead of skipping them
> p,preserve-merges! try to recreate merges instead of ignoring them
> s,strategy=!       use the given merge strategy
> no-ff!             cherry-pick all commits, even if unchanged
> @@ -88,6 +88,7 @@ state_dir=
> # One of {'', continue, skip, abort}, as parsed from command line
> action=
> recreate_merges=
> +no_rebase_cousins=
> preserve_merges=
> autosquash=
> keep_empty=
> @@ -268,6 +269,15 @@ do
>  recreate_merges=t
>  test -z "$interactive_rebase" && interactive_rebase=implied
>  ;;
> + --recreate-merges=*)
> + recreate_merges=t
> + case "${1#*=}" in
> + rebase-cousins) no_rebase_cousins=;;
> + no-rebase-cousins) no_rebase_cousins=t;;
> + *) die "Unknown mode: $1";;
> + esac
> + test -z "$interactive_rebase" && interactive_rebase=implied
> + ;;
>  --preserve-merges)
>  preserve_merges=t
>  test -z "$interactive_rebase" && interactive_rebase=implied
> diff --git a/sequencer.c b/sequencer.c
> index b63bfb9a141..2b4e6b12321 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -2905,6 +2905,7 @@ static int make_script_with_merges(struct 
> pretty_print_context *pp,
>     unsigned flags)
> {
>  int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
> + int no_rebase_cousins = flags & TODO_LIST_NO_REBASE_COUSINS;
>  struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
>  struct strbuf label = STRBUF_INIT;
>  struct commit_list *commits = NULL, **tail = &commits, *iter;
> @@ -3078,6 +3079,9 @@ static int make_script_with_merges(struct 
> pretty_print_context *pp,
>     &commit->object.oid);
>  if (entry)
>  to = entry->string;
> + else if (no_rebase_cousins)
> + to = label_oid(&commit->object.oid, NULL,
> +        &state);
>
>  if (!to || !strcmp("onto", to))
>  fprintf(out, "%s\n", b);
> diff --git a/sequencer.h b/sequencer.h
> index 11d1ac925ef..9530dba3cba 100644
> --- a/sequencer.h
> +++ b/sequencer.h
> @@ -49,6 +49,14 @@ int sequencer_remove_state(struct replay_opts *opts);
> #define TODO_LIST_SHORTEN_IDS (1U << 1)
> #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
> #define TODO_LIST_RECREATE_MERGES (1U << 3)
> +/*
> + * When recreating merges, commits that do have the base commit as 
> ancestor
> + * ("cousins") are rebased onto the new base by default. If those commits
> + * should keep their original branch point, this flag needs to be passed.
> + *
> + * This flag only makes sense when <base> and <onto> are different.
> + */
> +#define TODO_LIST_NO_REBASE_COUSINS (1U << 4)
> int sequencer_make_script(FILE *out, int argc, const char **argv,
>    unsigned flags);
>
> diff --git a/t/t3430-rebase-recreate-merges.sh 
> b/t/t3430-rebase-recreate-merges.sh
> index 76e615bd7c1..22930e470a4 100755
> --- a/t/t3430-rebase-recreate-merges.sh
> +++ b/t/t3430-rebase-recreate-merges.sh
> @@ -143,6 +143,29 @@ test_expect_success 'with a branch tip that was 
> cherry-picked already' '
>  EOF
> '
>
> +test_expect_success 'rebase cousins unless told not to' '
> + write_script copy-editor.sh <<-\EOF &&
> + cp "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
> + EOF
> +
> + test_config sequence.editor \""$PWD"/copy-editor.sh\" &&
> + git checkout -b cousins master &&
> + before="$(git rev-parse --verify HEAD)" &&
> + test_tick &&
> + git rebase -i --recreate-merges=no-rebase-cousins HEAD^ &&
> + test_cmp_rev HEAD $before &&
> + test_tick &&
> + git rebase -i --recreate-merges HEAD^ &&
> + test_cmp_graph HEAD^.. <<-\EOF
> + *   Merge the topic branch '\''onebranch'\''
> + |\
> + | * D
> + | * G
> + |/
> + o H
> + EOF
> +'
> +
> test_expect_success 'refs/rewritten/* is worktree-local' '
>  git worktree add wt &&
>  cat >wt/script-from-scratch <<-\EOF &&
> -- 
> 2.15.1.windows.2.1430.ga56c4f9e2a9 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 10/8] [DO NOT APPLY, but improve?] rebase--interactive: introduce "stop" command
  2018-01-18 21:20     ` Jacob Keller
@ 2018-01-18 22:08       ` Philip Oakley
  2018-01-18 22:09         ` Jacob Keller
  0 siblings, 1 reply; 412+ messages in thread
From: Philip Oakley @ 2018-01-18 22:08 UTC (permalink / raw)
  To: Jacob Keller, Stefan Beller
  Cc: Johannes Schindelin, Git mailing list, Junio C Hamano

From: "Jacob Keller" <jacob.keller@gmail.com>
> On Thu, Jan 18, 2018 at 10:36 AM, Stefan Beller <sbeller@google.com> 
> wrote:
>> Jake suggested using "x false" instead of "edit" for some corner cases.
>>
>> I do prefer using "x false" for all kinds of things such as stopping
>> before a commit (edit only let's you stop after a commit), and the
>> knowledge that "x false" does the least amount of actions behind my back.
>>
>> We should have that command as well, maybe?
>>
>
>
> I agree. I use "x false" very often, and I think stop is probably a
> better solution since it avoids spawning an extra shell that will just
> fail. Not sure if stop implies too much about "stop the whole thing"
> as opposed to "stop here and let me do something manual", but I think
> it's clear enough.
>
'hold' or 'pause' maybe options (leads to 
http://www.thesaurus.com/browse/put+on+hold offering procastinate etc.)
'adjourn'.

>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>>  git-rebase--interactive.sh |  1 +
>>  sequencer.c                | 10 ++++++++++
>>  2 files changed, 11 insertions(+)
>>
>> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
>> index 3cd7446d0b..9eac53f0c5 100644
>> --- a/git-rebase--interactive.sh
>> +++ b/git-rebase--interactive.sh
>> @@ -166,6 +166,7 @@ l, label <label>= label current HEAD with a name
>>  t, reset <label> = reset HEAD to a label
>>  b, bud = reset HEAD to the revision labeled 'onto', no arguments
>>  m, merge [<label-or-commit>]* = create a merge commit using a given 
>> commit's message
>> +y, stay = stop for  shortcut for
>>
>>  These lines can be re-ordered; they are executed from top to bottom.
>>  " | git stripspace --comment-lines >>"$todo"
>> diff --git a/sequencer.c b/sequencer.c
>> index 2b4e6b1232..4b3b9fe59d 100644
>> --- a/sequencer.c
>> +++ b/sequencer.c
>> @@ -782,6 +782,7 @@ enum todo_command {
>>         TODO_RESET,
>>         TODO_BUD,
>>         TODO_MERGE,
>> +       TODO_STOP,
>>         /* commands that do nothing but are counted for reporting 
>> progress */
>>         TODO_NOOP,
>>         TODO_DROP,
>> @@ -803,6 +804,7 @@ static struct {
>>         { 'l', "label" },
>>         { 't', "reset" },
>>         { 'b', "bud" },
>> +       { 'y', "stay" },
>>         { 'm', "merge" },
>>         { 0,   "noop" },
>>         { 'd', "drop" },
>> @@ -1307,6 +1309,12 @@ static int parse_insn_line(struct todo_item *item, 
>> const char *bol, char *eol)
>>                 return 0;
>>         }
>>
>> +       if (item->command == TODO_STOP) {
>> +               item->commit = NULL;
>> +               item->arg = "";
>> +               item->arg_len = 0;
>> +       }
>> +
>>         end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
>>         item->arg = end_of_object_name + strspn(end_of_object_name, " 
>> \t");
>>         item->arg_len = (int)(eol - item->arg);
>> @@ -2407,6 +2415,8 @@ static int pick_commits(struct todo_list 
>> *todo_list, struct replay_opts *opts)
>>                                 /* `current` will be incremented below */
>>                                 todo_list->current = -1;
>>                         }
>> +               } else if (item->command == TODO_STOP) {
>> +                       todo_list->current = -1;
>>                 } else if (item->command == TODO_LABEL)
>>                         res = do_label(item->arg, item->arg_len);
>>                 else if (item->command == TODO_RESET)
>> --
>> 2.16.0.rc1.238.g530d649a79-goog
>> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 10/8] [DO NOT APPLY, but improve?] rebase--interactive: introduce "stop" command
  2018-01-18 22:00     ` Johannes Schindelin
@ 2018-01-18 22:09       ` Stefan Beller
  0 siblings, 0 replies; 412+ messages in thread
From: Stefan Beller @ 2018-01-18 22:09 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Jacob Keller

On Thu, Jan 18, 2018 at 2:00 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:

>> +     TODO_STOP,
>
> I see that your original idea was "stop", but then you probably realized
> that there would be no good abbreviation for that, and changed your mind.
>
> Personally, I would have called it `break`...

I was looking at a synonym list of stop to find a word that contained a letter
which was not already taken. 'break' would allow for 'a', or 'k', assuming 'bud'
takes 'b' (or can that go to 'u'? Are there people out there with muscle memory
on these letters already?)

Any word (of stop, break, stay, control) sounds good to me, though 'break' might
be the clearest.

>
>> @@ -2407,6 +2415,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>>                               /* `current` will be incremented below */
>>                               todo_list->current = -1;
>>                       }
>> +             } else if (item->command == TODO_STOP) {
>> +                     todo_list->current = -1;
>
> That is incorrect, it will most likely write an unexpected `done` file.
>
> Did you mean `return 0` instead?

I guess. I did not compile or test the patch, I was merely writing down enough
to convey the idea, hopefully.

While talking about this idea of exploding the number of keywords,
maybe we can also have 'abort', which does the same as deleting all lines
(every time I want to abort I still get shivers if I just drop all
patches instead
of aborting, so maybe typing 'abort-and-restore' as the first thing in the file
would convey a safer feeling to users?)

Thanks for taking these additional considerations into mind while I don't
review the actual patches,

Stefan

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 10/8] [DO NOT APPLY, but improve?] rebase--interactive: introduce "stop" command
  2018-01-18 22:08       ` Philip Oakley
@ 2018-01-18 22:09         ` Jacob Keller
  0 siblings, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-01-18 22:09 UTC (permalink / raw)
  To: Philip Oakley
  Cc: Stefan Beller, Johannes Schindelin, Git mailing list, Junio C Hamano

On Thu, Jan 18, 2018 at 2:08 PM, Philip Oakley <philipoakley@iee.org> wrote:
> From: "Jacob Keller" <jacob.keller@gmail.com>
>>
>> On Thu, Jan 18, 2018 at 10:36 AM, Stefan Beller <sbeller@google.com>
>> wrote:
>>>
>>> Jake suggested using "x false" instead of "edit" for some corner cases.
>>>
>>> I do prefer using "x false" for all kinds of things such as stopping
>>> before a commit (edit only let's you stop after a commit), and the
>>> knowledge that "x false" does the least amount of actions behind my back.
>>>
>>> We should have that command as well, maybe?
>>>
>>
>>
>> I agree. I use "x false" very often, and I think stop is probably a
>> better solution since it avoids spawning an extra shell that will just
>> fail. Not sure if stop implies too much about "stop the whole thing"
>> as opposed to "stop here and let me do something manual", but I think
>> it's clear enough.
>>
> 'hold' or 'pause' maybe options (leads to
> http://www.thesaurus.com/browse/put+on+hold offering procastinate etc.)
> 'adjourn'.
>
>

I like break, as suggested by Dscho. That also works well for
abbreviation if we drop the "bud" command.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-18 15:35 ` [PATCH 1/8] sequencer: introduce new commands to reset the revision Johannes Schindelin
  2018-01-18 16:25   ` Jacob Keller
@ 2018-01-19  8:59   ` Eric Sunshine
  2018-01-24 22:01     ` Junio C Hamano
  2018-01-29 20:50     ` Johannes Schindelin
  2018-01-19 12:24   ` [PATCH 1/8] sequencer: introduce new commands to resettherevision Phillip Wood
  2 siblings, 2 replies; 412+ messages in thread
From: Eric Sunshine @ 2018-01-19  8:59 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Git List, Junio C Hamano, Jacob Keller

On Thu, Jan 18, 2018 at 10:35 AM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> [...]
> This commit implements the commands to label, and to reset to, given
> revisions. The syntax is:
>
>         label <name>
>         reset <name>
> [...]
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> diff --git a/sequencer.c b/sequencer.c
> @@ -1919,6 +1936,139 @@ static int do_exec(const char *command_line)
> +static int safe_append(const char *filename, const char *fmt, ...)
> +{
> +       [...]
> +       if (commit_lock_file(&lock) < 0) {
> +               rollback_lock_file(&lock);
> +               return error(_("failed to finalize '%s'."), filename);

s/\.//

> +       }
> +
> +       return 0;
> +}
> +
> +static int do_reset(const char *name, int len)
> +{
> +       [...]
> +       if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> +               return -1;
> +
> +       for (i = 0; i < len; i++)
> +               if (isspace(name[i]))
> +                       len = i;

What is the purpose of this loop? I could imagine that it's trying to
strip all whitespace from the end of 'name', however, to do that it
would iterate backward, not forward. (Or perhaps it's trying to
truncate at the first space, but then it would need to invert the
condition or use 'break'.) Am I missing something obvious?

> +       read_cache_unmerged();
> +       if (!fill_tree_descriptor(&desc, &oid)) {
> +               error(_("Failed to find tree of %s."), oid_to_hex(&oid));

s/Failed/failed/
s/\.//

> +               rollback_lock_file(&lock);
> +               free((void *)desc.buffer);
> +               strbuf_release(&ref_name);
> +               return -1;
> +       }

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/8] sequencer: introduce the `merge` command
  2018-01-18 15:35 ` [PATCH 2/8] sequencer: introduce the `merge` command Johannes Schindelin
  2018-01-18 16:31   ` Jacob Keller
@ 2018-01-19  9:54   ` Eric Sunshine
  2018-01-19 14:45   ` Phillip Wood
  2018-01-22 22:12   ` Junio C Hamano
  3 siblings, 0 replies; 412+ messages in thread
From: Eric Sunshine @ 2018-01-19  9:54 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Git List, Junio C Hamano, Jacob Keller

On Thu, Jan 18, 2018 at 10:35 AM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> [...]
> Note: this patch only adds support for recursive merges, to keep things
> simple. Support for octopus merges will be added later in this patch
> series, support for merges using strategies other than the recursive
> merge is left for future contributions.

The above paragraph...

> The design of the `merge` command as introduced by this patch only
> supports creating new merge commits with exactly two parents, i.e. it
> adds no support for octopus merges.
>
> We will introduce support for octopus merges in a later commit.

and these two sentences say the same thing. I suppose one or the other
was meant to be dropped(?).

> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> diff --git a/sequencer.c b/sequencer.c
> @@ -2069,6 +2077,132 @@ static int do_reset(const char *name, int len)
> +static int do_merge(struct commit *commit, const char *arg, int arg_len,
> +                   struct replay_opts *opts)
> +{
> +       [...]
> +               if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
> +                       error_errno(_("Could not write '%s'"),

s/Could/could/

> +               if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
> +                       error_errno(_("Could not write '%s'"),

Ditto.

> +       if (!head_commit) {
> +               rollback_lock_file(&lock);
> +               return error(_("Cannot merge without a current revision"));

s/Cannot/cannot/

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges
  2018-01-18 15:35 ` [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
  2018-01-18 21:39   ` Philip Oakley
@ 2018-01-19 10:34   ` Eric Sunshine
  2018-01-23 20:13     ` Junio C Hamano
  2018-01-29 21:05     ` Johannes Schindelin
  2018-01-23 20:03   ` Junio C Hamano
  2 siblings, 2 replies; 412+ messages in thread
From: Eric Sunshine @ 2018-01-19 10:34 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Git List, Junio C Hamano, Jacob Keller

On Thu, Jan 18, 2018 at 10:35 AM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> The sequencer just learned a new commands intended to recreate branch

s/a //

> structure (similar in spirit to --preserve-merges, but with a
> substantially less-broken design).
> [...]
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> diff --git a/sequencer.c b/sequencer.c
> @@ -2785,6 +2787,335 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
> +static const char *label_oid(struct object_id *oid, const char *label,
> +                            struct label_state *state)
> +{
> +       [...]
> +       } else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
> +                   !get_oid_hex(label, &dummy)) ||
> +                  hashmap_get_from_hash(&state->labels,
> +                                        strihash(label), label)) {
> +               /*
> +                * If the label already exists, or if the label is a valid full
> +                * OID, we append a dash and a number to make it unique.
> +                */
> +               [...]
> +               for (i = 2; ; i++) {

Why '2'? Is there some non-obvious significance to this value?

> +                       strbuf_setlen(buf, len);
> +                       strbuf_addf(buf, "-%d", i);
> +                       if (!hashmap_get_from_hash(&state->labels,
> +                                                  strihash(buf->buf),
> +                                                  buf->buf))
> +                               break;
> +               }
> +
> +static int make_script_with_merges(struct pretty_print_context *pp,
> +                                  struct rev_info *revs, FILE *out,
> +                                  unsigned flags)
> +{
> +       [...]
> +               is_octopus = to_merge && to_merge->next;
> +
> +               if (is_octopus)
> +                       BUG("Octopus merges not yet supported");

Is this a situation which the end-user can trigger by specifying a
merge with more than two parents? If so, shouldn't this be just a
normal error message rather than a (developer) bug message? Or, am I
misunderstanding?

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-01-18 15:35 ` [PATCH 5/8] rebase: introduce the --recreate-merges option Johannes Schindelin
@ 2018-01-19 10:55   ` Eric Sunshine
  2018-01-29 21:09     ` Johannes Schindelin
  2018-01-23 20:22   ` Junio C Hamano
                     ` (2 subsequent siblings)
  3 siblings, 1 reply; 412+ messages in thread
From: Eric Sunshine @ 2018-01-19 10:55 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Git List, Junio C Hamano, Jacob Keller

On Thu, Jan 18, 2018 at 10:35 AM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> [...]
> With this patch, the goodness of the Git garden shears comes to `git
> rebase -i` itself. Passing the `--recreate-merges` option will generate
> a todo list that can be understood readily, and where it is obvious
> how to reorder commits. New branches can be introduced by inserting
> `label` commands and calling `merge - <label> <oneline>`. And once this
> mode has become stable and universally accepted, we can deprecate the
> design mistake that was `--preserve-merges`.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> @@ -900,6 +900,7 @@ fi
>  if test t != "$preserve_merges"
>  then
>         git rebase--helper --make-script ${keep_empty:+--keep-empty} \
> +               ${recreate_merges:+--recreate-merges} \

If the user specifies both --preserve-merges and --recreate-merges, it
looks like --preserve-merges will take precedence.

Should git-rebase.sh have a mutual-exclusion check and error out if
both are specified?

>                 $revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
>         die "$(gettext "Could not generate todo list")"
> diff --git a/git-rebase.sh b/git-rebase.sh
> @@ -262,6 +264,10 @@ do
> +       --recreate-merges)
> +               recreate_merges=t
> +               test -z "$interactive_rebase" && interactive_rebase=implied
> +               ;;
>         --preserve-merges)
>                 preserve_merges=t
>                 test -z "$interactive_rebase" && interactive_rebase=implied

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to resettherevision
  2018-01-18 15:35 ` [PATCH 1/8] sequencer: introduce new commands to reset the revision Johannes Schindelin
  2018-01-18 16:25   ` Jacob Keller
  2018-01-19  8:59   ` Eric Sunshine
@ 2018-01-19 12:24   ` Phillip Wood
  2018-01-19 18:55     ` Phillip Wood
  2018-01-29 21:23     ` Johannes Schindelin
  2 siblings, 2 replies; 412+ messages in thread
From: Phillip Wood @ 2018-01-19 12:24 UTC (permalink / raw)
  To: Johannes Schindelin, git
  Cc: Junio C Hamano, Jacob Keller, Philip Oakley, Eric Sunshine


On 18/01/18 15:35, Johannes Schindelin wrote:
> 
> In the upcoming commits, we will teach the sequencer to recreate merges.
> This will be done in a very different way from the unfortunate design of
> `git rebase --preserve-merges` (which does not allow for reordering
> commits, or changing the branch topology).
> 
> The main idea is to introduce new todo list commands, to support
> labeling the current revision with a given name, resetting the current
> revision to a previous state, merging labeled revisions.

I think this would be a great improvement to rebase -i, thanks for
working on it.

> This idea was developed in Git for Windows' Git garden shears (that are
> used to maintain the "thicket of branches" on top of upstream Git), and
> this patch is part of the effort to make it available to a wider
> audience, as well as to make the entire process more robust (by
> implementing it in a safe and portable language rather than a Unix shell
> script).
> 
> This commit implements the commands to label, and to reset to, given
> revisions. The syntax is:
> 
> 	label <name>
> 	reset <name>

If I've understood the code below correctly then reset will clobber
untracked files, this is the opposite behaviour to what happens when
tries to checkout <onto> at the start of a rebase - then it will fail if
untracked files would be overwritten.

> As a convenience shortcut, also to improve readability of the generated
> todo list, a third command is introduced: bud. It simply resets to the
> "onto" revision, i.e. the commit onto which we currently rebase.

I found the whole bud business bewildering at first, reading the other
replies it seems I wasn't the only one to be befuddled by it. Having
seen an example I can see what it's trying to do but I still think it
adds more confusion than value.

> Internally, the `label <name>` command creates the ref
> `refs/rewritten/<name>`. This makes it possible to work with the labeled
> revisions interactively, or in a scripted fashion (e.g. via the todo
> list command `exec`).

If a user has two work trees and runs a rebase in each with the same
label name, they'll clobber each other. I'd suggest storing them under
refs/rewritten/<branch-name or detached HEAD SHA> instead. If the user
tries to rebase a second worktree with the same detached HEAD as an
existing rebase then refuse to start.

> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  git-rebase--interactive.sh |   3 +
>  sequencer.c                | 181 ++++++++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 180 insertions(+), 4 deletions(-)
> 
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index d47bd29593a..3d2cd19d65a 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -162,6 +162,9 @@ s, squash = use commit, but meld into previous commit
>  f, fixup = like \"squash\", but discard this commit's log message
>  x, exec = run command (the rest of the line) using shell
>  d, drop = remove commit
> +l, label = label current HEAD with a name
> +t, reset = reset HEAD to a label
> +b, bud = reset HEAD to the revision labeled 'onto'
>  
>  These lines can be re-ordered; they are executed from top to bottom.
>  " | git stripspace --comment-lines >>"$todo"
> diff --git a/sequencer.c b/sequencer.c
> index 4d3f60594cb..91cc55a002f 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -21,6 +21,8 @@
>  #include "log-tree.h"
>  #include "wt-status.h"
>  #include "hashmap.h"
> +#include "unpack-trees.h"
> +#include "worktree.h"
>  
>  #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
>  
> @@ -116,6 +118,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
>  static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
>  static GIT_PATH_FUNC(rebase_path_rewritten_pending,
>  	"rebase-merge/rewritten-pending")
> +
> +/*
> + * The path of the file listing refs that need to be deleted after the rebase
> + * finishes. This is used by the `merge` command.
> + */
> +static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
> +
>  /*
>   * The following files are written by git-rebase just after parsing the
>   * command-line (and are only consumed, not modified, by the sequencer).
> @@ -767,6 +776,9 @@ enum todo_command {
>  	TODO_SQUASH,
>  	/* commands that do something else than handling a single commit */
>  	TODO_EXEC,
> +	TODO_LABEL,
> +	TODO_RESET,
> +	TODO_BUD,
>  	/* commands that do nothing but are counted for reporting progress */
>  	TODO_NOOP,
>  	TODO_DROP,
> @@ -785,6 +797,9 @@ static struct {
>  	{ 'f', "fixup" },
>  	{ 's', "squash" },
>  	{ 'x', "exec" },
> +	{ 'l', "label" },
> +	{ 't', "reset" },
> +	{ 'b', "bud" },
>  	{ 0,   "noop" },
>  	{ 'd', "drop" },
>  	{ 0,   NULL }
> @@ -1253,7 +1268,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>  		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
>  			item->command = i;
>  			break;
> -		} else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
> +		} else if ((bol + 1 == eol || bol[1] == ' ') &&
> +			   *bol == todo_command_info[i].c) {
>  			bol++;
>  			item->command = i;
>  			break;
> @@ -1265,7 +1281,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>  	padding = strspn(bol, " \t");
>  	bol += padding;
>  
> -	if (item->command == TODO_NOOP) {
> +	if (item->command == TODO_NOOP || item->command == TODO_BUD) {
>  		if (bol != eol)
>  			return error(_("%s does not accept arguments: '%s'"),
>  				     command_to_string(item->command), bol);
> @@ -1279,7 +1295,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>  		return error(_("missing arguments for %s"),
>  			     command_to_string(item->command));
>  
> -	if (item->command == TODO_EXEC) {
> +	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
> +	    item->command == TODO_RESET) {
>  		item->commit = NULL;
>  		item->arg = bol;
>  		item->arg_len = (int)(eol - bol);
> @@ -1919,6 +1936,139 @@ static int do_exec(const char *command_line)
>  	return status;
>  }
>  
> +static int safe_append(const char *filename, const char *fmt, ...)
> +{
> +	va_list ap;
> +	struct lock_file lock = LOCK_INIT;
> +	int fd = hold_lock_file_for_update(&lock, filename, 0);
> +	struct strbuf buf = STRBUF_INIT;
> +
> +	if (fd < 0)
> +		return error_errno(_("could not lock '%s'"), filename);
> +
> +	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT)
> +		return error_errno(_("could not read '%s'"), filename);
> +	strbuf_complete(&buf, '\n');
> +	va_start(ap, fmt);
> +	strbuf_vaddf(&buf, fmt, ap);
> +	va_end(ap);
> +
> +	if (write_in_full(fd, buf.buf, buf.len) < 0) {
> +		rollback_lock_file(&lock);
> +		return error_errno(_("could not write to '%s'"), filename);
> +	}
> +	if (commit_lock_file(&lock) < 0) {
> +		rollback_lock_file(&lock);
> +		return error(_("failed to finalize '%s'."), filename);
> +	}
> +
> +	return 0;
> +}
> +
> +static int do_label(const char *name, int len)
> +{
> +	struct ref_store *refs = get_main_ref_store();
> +	struct ref_transaction *transaction;
> +	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
> +	struct strbuf msg = STRBUF_INIT;
> +	int ret = 0;
> +	struct object_id head_oid;
> +
> +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> +	strbuf_addf(&msg, "label '%.*s'", len, name);

The other reflog messages below have a (rebase -i) prefix

> +
> +	transaction = ref_store_transaction_begin(refs, &err);
> +	if (!transaction ||
> +	    get_oid("HEAD", &head_oid) ||
> +	    ref_transaction_update(transaction, ref_name.buf, &head_oid, NULL,
> +				   0, msg.buf, &err) < 0 ||
> +	    ref_transaction_commit(transaction, &err)) {
> +		error("%s", err.buf);

if get_oid() fails then err is empty so there wont be an message after
the 'error: '

> +		ret = -1;
> +	}
> +	ref_transaction_free(transaction);
> +	strbuf_release(&err);
> +	strbuf_release(&msg);
> +
> +	if (!ret)
> +		ret = safe_append(rebase_path_refs_to_delete(),
> +				  "%s\n", ref_name.buf);
> +	strbuf_release(&ref_name);
> +
> +	return ret;
> +}
> +
> +static int do_reset(const char *name, int len)
> +{
> +	struct strbuf ref_name = STRBUF_INIT;
> +	struct object_id oid;
> +	struct lock_file lock = LOCK_INIT;
> +	struct tree_desc desc;
> +	struct tree *tree;
> +	struct unpack_trees_options opts;
> +	int ret = 0, i;
> +
> +	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> +		return -1;
> +
> +	for (i = 0; i < len; i++)
> +		if (isspace(name[i]))
> +			len = i;

If name starts with any white space then I think this effectively
truncates name to a bunch of white space which doesn't sound right. I'm
not sure how this is being called, but it might be better to clean up
name when the to-do list is parsed instead.

> +
> +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> +	if (get_oid(ref_name.buf, &oid) &&
> +	    get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
> +		error(_("could not read '%s'"), ref_name.buf);
> +		rollback_lock_file(&lock);
> +		strbuf_release(&ref_name);
> +		return -1;
> +	}
> +
> +	memset(&opts, 0, sizeof(opts));
> +	opts.head_idx = 1;
> +	opts.src_index = &the_index;
> +	opts.dst_index = &the_index;
> +	opts.fn = oneway_merge;
> +	opts.merge = 1;
> +	opts.update = 1;
> +	opts.reset = 1;
> +
> +	read_cache_unmerged();
> +	if (!fill_tree_descriptor(&desc, &oid)) {
> +		error(_("Failed to find tree of %s."), oid_to_hex(&oid));
> +		rollback_lock_file(&lock);
> +		free((void *)desc.buffer);
> +		strbuf_release(&ref_name);
> +		return -1;
> +	}
> +
> +	if (unpack_trees(1, &desc, &opts)) {
> +		rollback_lock_file(&lock);
> +		free((void *)desc.buffer);
> +		strbuf_release(&ref_name);
> +		return -1;
> +	}
> +
> +	tree = parse_tree_indirect(&oid);
> +	prime_cache_tree(&the_index, tree);
> +
> +	if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
> +		ret = error(_("could not write index"));
> +	free((void *)desc.buffer);
> +
> +	if (!ret) {
> +		struct strbuf msg = STRBUF_INIT;
> +
> +		strbuf_addf(&msg, "(rebase -i) reset '%.*s'", len, name);
> +		ret = update_ref(msg.buf, "HEAD", &oid, NULL, 0,
> +				 UPDATE_REFS_MSG_ON_ERR);
> +		strbuf_release(&msg);
> +	}
> +
> +	strbuf_release(&ref_name);
> +	return ret;
> +}
> +
>  static int is_final_fixup(struct todo_list *todo_list)
>  {
>  	int i = todo_list->current;
> @@ -2102,7 +2252,13 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  				/* `current` will be incremented below */
>  				todo_list->current = -1;
>  			}
> -		} else if (!is_noop(item->command))
> +		} else if (item->command == TODO_LABEL)
> +			res = do_label(item->arg, item->arg_len);
> +		else if (item->command == TODO_RESET)
> +			res = do_reset(item->arg, item->arg_len);
> +		else if (item->command == TODO_BUD)
> +			res = do_reset("onto", 4);
> +		else if (!is_noop(item->command))
>  			return error(_("unknown command %d"), item->command);
>  
>  		todo_list->current++;
> @@ -2207,6 +2363,23 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  		}
>  		apply_autostash(opts);
>  
> +		strbuf_reset(&buf);
> +		if (strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0)
> +		    > 0) {
> +			char *p = buf.buf;
> +			while (*p) {
> +				char *eol = strchr(p, '\n');
> +				if (eol)
> +					*eol = '\0';
> +				if (delete_ref("(rebase -i) cleanup",
> +					       p, NULL, 0) < 0)
> +					warning(_("could not delete '%s'"), p);
> +				if (!eol)
> +					break;
> +				p = eol + 1;
> +			}
> +		}
> +
>  		fprintf(stderr, "Successfully rebased and updated %s.\n",
>  			head_ref.buf);
>  
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/8] sequencer: introduce the `merge` command
  2018-01-18 15:35 ` [PATCH 2/8] sequencer: introduce the `merge` command Johannes Schindelin
  2018-01-18 16:31   ` Jacob Keller
  2018-01-19  9:54   ` Eric Sunshine
@ 2018-01-19 14:45   ` Phillip Wood
  2018-01-20  9:18     ` Jacob Keller
  2018-01-22 22:12   ` Junio C Hamano
  3 siblings, 1 reply; 412+ messages in thread
From: Phillip Wood @ 2018-01-19 14:45 UTC (permalink / raw)
  To: Johannes Schindelin, git; +Cc: Junio C Hamano, Jacob Keller, Eric Sunshine

On 18/01/18 15:35, Johannes Schindelin wrote:
> 
> This patch is part of the effort to reimplement `--preserve-merges` with
> a substantially improved design, a design that has been developed in the
> Git for Windows project to maintain the dozens of Windows-specific patch
> series on top of upstream Git.
> 
> The previous patch implemented the `label`, `bud` and `reset` commands
> to label commits and to reset to a labeled commits. This patch adds the
> `merge` command, with the following syntax:
> 
> 	merge <commit> <rev> <oneline>

I'm concerned that this will be confusing for users. All of the other
rebase commands replay the changes in the commit hash immediately
following the command name. This command instead uses the first commit
to specify the message which is different to both 'git merge' and the
existing rebase commands. I wonder if it would be clearer to have 'merge
-C <commit> <rev> ...' instead so it's clear which argument specifies
the message and which the remote head to merge. It would also allow for
'merge -c <commit> <rev> ...' in the future for rewording an existing
merge message and also avoid the slightly odd 'merge - <rev> ...'. Where
it's creating new merges I'm not sure it's a good idea to encourage
people to only have oneline commit messages by making it harder to edit
them, perhaps it could take another argument to mean open the editor or
not, though as Jake said I guess it's not that common.

One thought that just struck me - if a merge or reset command specifies
an invalid label is it rescheduled so that it's still in the to-do list
when the user edits it after rebase stops?

In the future it might be nice if the label, reset and merge commands
were validated when the to-do list is parsed so that the user gets
immediate feedback if they try to create a label that is not a valid ref
name or that they have a typo in a name given to reset or merge rather
than the rebase stopping later.

> The <commit> parameter in this instance is the *original* merge commit,
> whose author and message will be used for the to-be-created merge
> commit.
> 
> The <rev> parameter refers to the (possibly rewritten) revision to
> merge. Let's see an example of a todo list:
> 
> 	label onto
> 
> 	# Branch abc
> 	bud
> 	pick deadbeef Hello, world!
> 	label abc
> 
> 	bud
> 	pick cafecafe And now for something completely different
> 	merge baaabaaa abc Merge the branch 'abc' into master
> 
> To support creating *new* merges, i.e. without copying the commit
> message from an existing commit, use the special value `-` as <commit>
> parameter (in which case the text after the <rev> parameter is used as
> commit message):
> 
> 	merge - abc This will be the actual commit message of the merge
> 
> This comes in handy when splitting a branch into two or more branches.
> 
> Note: this patch only adds support for recursive merges, to keep things
> simple. Support for octopus merges will be added later in this patch
> series, support for merges using strategies other than the recursive
> merge is left for future contributions.
> 
> The design of the `merge` command as introduced by this patch only
> supports creating new merge commits with exactly two parents, i.e. it
> adds no support for octopus merges.
> 
> We will introduce support for octopus merges in a later commit.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  git-rebase--interactive.sh |   1 +
>  sequencer.c                | 146 +++++++++++++++++++++++++++++++++++++++++++--
>  2 files changed, 143 insertions(+), 4 deletions(-)
> 
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index 3d2cd19d65a..5bf1ea3781f 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -165,6 +165,7 @@ d, drop = remove commit
>  l, label = label current HEAD with a name
>  t, reset = reset HEAD to a label
>  b, bud = reset HEAD to the revision labeled 'onto'
> +m, merge = create a merge commit using a given commit's message
>  
>  These lines can be re-ordered; they are executed from top to bottom.
>  " | git stripspace --comment-lines >>"$todo"
> diff --git a/sequencer.c b/sequencer.c
> index 91cc55a002f..567cfcbbe8b 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -779,6 +779,7 @@ enum todo_command {
>  	TODO_LABEL,
>  	TODO_RESET,
>  	TODO_BUD,
> +	TODO_MERGE,
>  	/* commands that do nothing but are counted for reporting progress */
>  	TODO_NOOP,
>  	TODO_DROP,
> @@ -800,6 +801,7 @@ static struct {
>  	{ 'l', "label" },
>  	{ 't', "reset" },
>  	{ 'b', "bud" },
> +	{ 'm', "merge" },
>  	{ 0,   "noop" },
>  	{ 'd', "drop" },
>  	{ 0,   NULL }
> @@ -1304,14 +1306,20 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>  	}
>  
>  	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
> +	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
> +	item->arg_len = (int)(eol - item->arg);
> +
>  	saved = *end_of_object_name;
> +	if (item->command == TODO_MERGE && *bol == '-' &&
> +	    bol + 1 == end_of_object_name) {
> +		item->commit = NULL;
> +		return 0;
> +	}
> +
>  	*end_of_object_name = '\0';
>  	status = get_oid(bol, &commit_oid);
>  	*end_of_object_name = saved;
>  
> -	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
> -	item->arg_len = (int)(eol - item->arg);
> -
>  	if (status < 0)
>  		return -1;
>  
> @@ -2069,6 +2077,132 @@ static int do_reset(const char *name, int len)
>  	return ret;
>  }
>  
> +static int do_merge(struct commit *commit, const char *arg, int arg_len,
> +		    struct replay_opts *opts)
> +{
> +	int merge_arg_len;
> +	struct strbuf ref_name = STRBUF_INIT;
> +	struct commit *head_commit, *merge_commit, *i;
> +	struct commit_list *common, *j, *reversed = NULL;
> +	struct merge_options o;
> +	int ret;
> +	static struct lock_file lock;
> +
> +	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
> +		if (isspace(arg[merge_arg_len]))
> +			break;
> +
> +	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> +		return -1;
> +
> +	if (commit) {
> +		const char *message = get_commit_buffer(commit, NULL);
> +		const char *body;
> +		int len;
> +
> +		if (!message) {
> +			rollback_lock_file(&lock);
> +			return error(_("could not get commit message of '%s'"),
> +				     oid_to_hex(&commit->object.oid));
> +		}
> +		write_author_script(message);
> +		find_commit_subject(message, &body);
> +		len = strlen(body);
> +		if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
> +			error_errno(_("Could not write '%s'"),
> +				    git_path_merge_msg());
> +			unuse_commit_buffer(commit, message);
> +			rollback_lock_file(&lock);
> +			return -1;
> +		}
> +		unuse_commit_buffer(commit, message);
> +	} else {
> +		const char *p = arg + merge_arg_len;
> +		struct strbuf buf = STRBUF_INIT;
> +		int len;
> +
> +		strbuf_addf(&buf, "author %s", git_author_info(0));
> +		write_author_script(buf.buf);
> +		strbuf_reset(&buf);
> +
> +		p += strspn(p, " \t");
> +		if (*p)
> +			len = strlen(p);
> +		else {
> +			strbuf_addf(&buf, "Merge branch '%.*s'",
> +				    merge_arg_len, arg);
> +			p = buf.buf;
> +			len = buf.len;
> +		}
> +
> +		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
> +			error_errno(_("Could not write '%s'"),
> +				    git_path_merge_msg());
> +			strbuf_release(&buf);
> +			rollback_lock_file(&lock);
> +			return -1;
> +		}
> +		strbuf_release(&buf);
> +	}
> +
> +	head_commit = lookup_commit_reference_by_name("HEAD");
> +	if (!head_commit) {
> +		rollback_lock_file(&lock);
> +		return error(_("Cannot merge without a current revision"));
> +	}
> +
> +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
> +	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> +	if (!merge_commit) {
> +		/* fall back to non-rewritten ref or commit */
> +		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
> +		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> +	}
> +	if (!merge_commit) {
> +		error(_("could not resolve '%s'"), ref_name.buf);
> +		strbuf_release(&ref_name);
> +		rollback_lock_file(&lock);
> +		return -1;
> +	}
> +	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
> +		      git_path_merge_head(), 0);
> +	write_message("no-ff", 5, git_path_merge_mode(), 0);
> +
> +	common = get_merge_bases(head_commit, merge_commit);
> +	for (j = common; j; j = j->next)
> +		commit_list_insert(j->item, &reversed);
> +	free_commit_list(common);
> +
> +	read_cache();
> +	init_merge_options(&o);
> +	o.branch1 = "HEAD";
> +	o.branch2 = ref_name.buf;
> +	o.buffer_output = 2;
> +
> +	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
> +	if (ret <= 0)
> +		fputs(o.obuf.buf, stdout);
> +	strbuf_release(&o.obuf);
> +	if (ret < 0) {
> +		strbuf_release(&ref_name);
> +		rollback_lock_file(&lock);
> +		return error(_("conflicts while merging '%.*s'"),
> +			     merge_arg_len, arg);
> +	}
> +
> +	if (active_cache_changed &&
> +	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
> +		strbuf_release(&ref_name);
> +		return error(_("merge: Unable to write new index file"));
> +	}
> +	rollback_lock_file(&lock);
> +
> +	ret = run_git_commit(git_path_merge_msg(), opts, 0);
> +	strbuf_release(&ref_name);
> +
> +	return ret;
> +}
> +
>  static int is_final_fixup(struct todo_list *todo_list)
>  {
>  	int i = todo_list->current;
> @@ -2258,6 +2392,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  			res = do_reset(item->arg, item->arg_len);
>  		else if (item->command == TODO_BUD)
>  			res = do_reset("onto", 4);
> +		else if (item->command == TODO_MERGE)
> +			res = do_merge(item->commit,
> +				       item->arg, item->arg_len, opts);
>  		else if (!is_noop(item->command))
>  			return error(_("unknown command %d"), item->command);
>  
> @@ -2757,7 +2894,8 @@ int transform_todos(unsigned flags)
>  					  oid_to_hex(&item->commit->object.oid);
>  
>  			strbuf_addf(&buf, " %s", oid);
> -		}
> +		} else if (item->command == TODO_MERGE)
> +			strbuf_addstr(&buf, " -");
>  		/* add all the rest */
>  		if (!item->arg_len)
>  			strbuf_addch(&buf, '\n');
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 3/8] sequencer: fast-forward merge commits, if possible
  2018-01-18 15:35 ` [PATCH 3/8] sequencer: fast-forward merge commits, if possible Johannes Schindelin
@ 2018-01-19 14:53   ` Phillip Wood
  2018-01-23 19:12     ` Junio C Hamano
  2018-01-29 21:47     ` Johannes Schindelin
  2018-01-23 18:51   ` Junio C Hamano
  1 sibling, 2 replies; 412+ messages in thread
From: Phillip Wood @ 2018-01-19 14:53 UTC (permalink / raw)
  To: Johannes Schindelin, git; +Cc: Junio C Hamano, Jacob Keller

On 18/01/18 15:35, Johannes Schindelin wrote:
> 
> Just like with regular `pick` commands, if we are trying to recreate a
> merge commit, we now test whether the parents of said commit match HEAD
> and the commits to be merged, and fast-forward if possible.
> 
> This is not only faster, but also avoids unnecessary proliferation of
> new objects.

I might have missed something but shouldn't this be checking opts->allow_ff?

Another possible optimization is that if the parent branches have only
reworded commits or some commits that have been squashed but no other
changes then their trees will be the same as in the original merge
commit and so could be reused without calling merge_recursive().

> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 21 ++++++++++++++++++++-
>  1 file changed, 20 insertions(+), 1 deletion(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index 567cfcbbe8b..a96255426e7 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -2085,7 +2085,7 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
>  	struct commit *head_commit, *merge_commit, *i;
>  	struct commit_list *common, *j, *reversed = NULL;
>  	struct merge_options o;
> -	int ret;
> +	int can_fast_forward, ret;
>  	static struct lock_file lock;
>  
>  	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
> @@ -2151,6 +2151,14 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
>  		return error(_("Cannot merge without a current revision"));
>  	}
>  
> +	/*
> +	 * If HEAD is not identical to the parent of the original merge commit,
> +	 * we cannot fast-forward.
> +	 */
> +	can_fast_forward = commit && commit->parents &&
> +		!oidcmp(&commit->parents->item->object.oid,
> +			&head_commit->object.oid);
> +
>  	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
>  	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
>  	if (!merge_commit) {
> @@ -2164,6 +2172,17 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
>  		rollback_lock_file(&lock);
>  		return -1;
>  	}
> +
> +	if (can_fast_forward && commit->parents->next &&
> +	    !commit->parents->next->next &&
> +	    !oidcmp(&commit->parents->next->item->object.oid,
> +		    &merge_commit->object.oid)) {
> +		strbuf_release(&ref_name);
> +		rollback_lock_file(&lock);
> +		return fast_forward_to(&commit->object.oid,
> +				       &head_commit->object.oid, 0, opts);
> +	}
> +
>  	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
>  		      git_path_merge_head(), 0);
>  	write_message("no-ff", 5, git_path_merge_mode(), 0);
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to resettherevision
  2018-01-19 12:24   ` [PATCH 1/8] sequencer: introduce new commands to resettherevision Phillip Wood
@ 2018-01-19 18:55     ` Phillip Wood
  2018-01-19 18:59       ` Jacob Keller
  2018-01-29 21:23     ` Johannes Schindelin
  1 sibling, 1 reply; 412+ messages in thread
From: Phillip Wood @ 2018-01-19 18:55 UTC (permalink / raw)
  To: Johannes Schindelin, git
  Cc: Junio C Hamano, Jacob Keller, Philip Oakley, Eric Sunshine

On 19/01/18 12:24, Phillip Wood wrote:
> 
> On 18/01/18 15:35, Johannes Schindelin wrote:
>>
>> Internally, the `label <name>` command creates the ref
>> `refs/rewritten/<name>`. This makes it possible to work with the labeled
>> revisions interactively, or in a scripted fashion (e.g. via the todo
>> list command `exec`).
> 
> If a user has two work trees and runs a rebase in each with the same
> label name, they'll clobber each other. I'd suggest storing them under
> refs/rewritten/<branch-name or detached HEAD SHA> instead. If the user
> tries to rebase a second worktree with the same detached HEAD as an
> existing rebase then refuse to start.
> 

Ah this isn't a concern after all as patch 5 makes refs/rewritten local
to the worktree. Perhaps you could move that part of patch 5 here or add
a note to the commit message that it will become worktree local later in
the series

Best Wishes

Phillip

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to resettherevision
  2018-01-19 18:55     ` Phillip Wood
@ 2018-01-19 18:59       ` Jacob Keller
  2018-01-29 21:25         ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Jacob Keller @ 2018-01-19 18:59 UTC (permalink / raw)
  To: Phillip Wood
  Cc: Johannes Schindelin, Git mailing list, Junio C Hamano,
	Philip Oakley, Eric Sunshine

On Fri, Jan 19, 2018 at 10:55 AM, Phillip Wood
<phillip.wood@talktalk.net> wrote:
> On 19/01/18 12:24, Phillip Wood wrote:
>>
>> On 18/01/18 15:35, Johannes Schindelin wrote:
>>>
>>> Internally, the `label <name>` command creates the ref
>>> `refs/rewritten/<name>`. This makes it possible to work with the labeled
>>> revisions interactively, or in a scripted fashion (e.g. via the todo
>>> list command `exec`).
>>
>> If a user has two work trees and runs a rebase in each with the same
>> label name, they'll clobber each other. I'd suggest storing them under
>> refs/rewritten/<branch-name or detached HEAD SHA> instead. If the user
>> tries to rebase a second worktree with the same detached HEAD as an
>> existing rebase then refuse to start.
>>
>
> Ah this isn't a concern after all as patch 5 makes refs/rewritten local
> to the worktree. Perhaps you could move that part of patch 5 here or add
> a note to the commit message that it will become worktree local later in
> the series
>
> Best Wishes
>
> Phillip

I'd rather it be included here as well.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 0/8] rebase -i: offer to recreate merge commits
  2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
                   ` (9 preceding siblings ...)
  2018-01-18 18:36 ` [PATCH 9, 10/8] interactive rebase feedback Stefan Beller
@ 2018-01-19 20:25 ` Junio C Hamano
  2018-01-29 21:53   ` Johannes Schindelin
  2018-01-23 20:29 ` Junio C Hamano
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
  12 siblings, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-01-19 20:25 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Jacob Keller

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Think of --recreate-merges as "--preserve-merges done right". It
> introduces new verbs for the todo list, `label`, `reset` and `merge`.
> For a commit topology like this:
>
>             A - B - C
>               \   /
>                 D
>
> the generated todo list would look like this:
>
>             # branch D
>             pick 0123 A
>             label branch-point
>             pick 1234 D
>             label D
>
>             reset branch-point
>             pick 2345 B
>             merge 3456 D C

Yup.  I've seen this design talked about on list in the past, and
I've always felt that this is "sequencer done right".

At the first glance, it may feel somewhat unsatisfying that "merge"
has to say effects of which commits should be reflected in the
result and which commot to take the log message from, i.e.
(recreated)D is merged to form the resulting tree, and 3456=C is
used for the log, to recreate C in the above example, while "pick"
always uses the same commit for both, i.e. recreated B inherits both
the changes and log message from the original B=2345 (or depending
on the readers' point of view, "merge" is allowed to use two
different commits, while "pick" is always limited to the same one).

But I think this distinction is probably fundamental and I am not
opposed to it at all.  The result of "pick" has only one parent, and
the parent is determined only by the previous actions and not by
anything on the "pick" line in the todo list.  But the result of
"merge" has to record all the other parents, and only the first
parent is determined implicitly by the previous actions.  We need to
tell the "merge" command about "3456=C" in order to recreate the
effect of original merge commit (i.e. changes between B and C) as
well as its log message, and we also need to tell it about label "D"
that it is the "other parent" that need to be recorded.

Obviously "merge" command syntax should allow recreating an octopus,
so whenever I said "two" in the above, I meant "N".  The original
merge commit is needed so that the effect to replay (roughly: a
patch going to the original merge result from its first parent) can
be learned from the existing history, and all the other "N-1"
parents needs to be given (and they must have been already created
in the todo list) so that the resulting recreated merge can be
recorded with them as parents (in addition to the first parent that
is implicitly given as the result of all the previous steps).

One interesting (and probably useful) thing to notice is that if A
were not rebased in the above sample picture, and only B were the
one that was tweaked, then a recreated C may use the same original D
as its side parent, and the mechanism outlined above naturally can
support it by allowing an un-rewritten commit to be given as a side
parent when "merge" is redoing C.

I probably won't have time to actually look at the code for a few
days, but I am reasonably excited about the topic ;-)

Thanks.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 9/8] [DO NOT APPLY, but squash?] git-rebase--interactive: clarify arguments
  2018-01-18 21:36     ` Johannes Schindelin
  2018-01-18 21:58       ` Stefan Beller
@ 2018-01-19 20:30       ` Junio C Hamano
  2018-01-20  9:14         ` Jacob Keller
  1 sibling, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-01-19 20:30 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Stefan Beller, git, jacob.keller

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Good idea! I would rather do it as an introductory patch (that only
> converts the existing list).
>
> As to `merge`: it is a bit more complicated ;-)
>
> 	m, merge <original-merge-commit> ( <label> | "<label>..." ) [<oneline>]
> 		create a merge commit using the original merge commit's
> 		message (or the oneline, if "-" is given). Use a quoted
> 		list of commits to be merged for octopus merges.

Is it just the message that is being reused?  

Aren't the trees of the original commit and its parents participate
in creating the tree of the recreated merge?  One way to preserve an
originally evil merge is to notice how it was made by taking the
difference between the result of mechanical merge of original merge
parents and the original merge result, and carry it forward when
recreating the merge across new parents.  Just being curious.


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 8/8] rebase -i: introduce --recreate-merges=no-rebase-cousins
  2018-01-18 15:36 ` [PATCH 8/8] rebase -i: introduce --recreate-merges=no-rebase-cousins Johannes Schindelin
  2018-01-18 22:00   ` Philip Oakley
@ 2018-01-20  1:09   ` Eric Sunshine
  1 sibling, 0 replies; 412+ messages in thread
From: Eric Sunshine @ 2018-01-20  1:09 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Git List, Junio C Hamano, Jacob Keller

On Thu, Jan 18, 2018 at 10:36 AM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> This one is a bit tricky to explain, so let's try with a diagram:
> [...]
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
> @@ -57,8 +59,13 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
> +       if (no_rebase_cousins >= 0&& !recreate_merges)

Style: space before &&

> +               warning(_("--[no-]rebase-cousins has no effect without "
> +                         "--recreate-merges"));

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 9/8] [DO NOT APPLY, but squash?] git-rebase--interactive: clarify arguments
  2018-01-19 20:30       ` Junio C Hamano
@ 2018-01-20  9:14         ` Jacob Keller
  2018-01-29 17:02           ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Jacob Keller @ 2018-01-20  9:14 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Johannes Schindelin, Stefan Beller, Git mailing list

On Fri, Jan 19, 2018 at 12:30 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
>> Good idea! I would rather do it as an introductory patch (that only
>> converts the existing list).
>>
>> As to `merge`: it is a bit more complicated ;-)
>>
>>       m, merge <original-merge-commit> ( <label> | "<label>..." ) [<oneline>]
>>               create a merge commit using the original merge commit's
>>               message (or the oneline, if "-" is given). Use a quoted
>>               list of commits to be merged for octopus merges.
>
> Is it just the message that is being reused?
>
> Aren't the trees of the original commit and its parents participate
> in creating the tree of the recreated merge?  One way to preserve an
> originally evil merge is to notice how it was made by taking the
> difference between the result of mechanical merge of original merge
> parents and the original merge result, and carry it forward when
> recreating the merge across new parents.  Just being curious.
>

It looks like currently that only the commit is kept, with no attempt
to recreate evil merges.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/8] sequencer: introduce the `merge` command
  2018-01-19 14:45   ` Phillip Wood
@ 2018-01-20  9:18     ` Jacob Keller
  2018-01-29 21:41       ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Jacob Keller @ 2018-01-20  9:18 UTC (permalink / raw)
  To: Phillip Wood
  Cc: Johannes Schindelin, Git mailing list, Junio C Hamano, Eric Sunshine

On Fri, Jan 19, 2018 at 6:45 AM, Phillip Wood <phillip.wood@talktalk.net> wrote:
> On 18/01/18 15:35, Johannes Schindelin wrote:
>>
>> This patch is part of the effort to reimplement `--preserve-merges` with
>> a substantially improved design, a design that has been developed in the
>> Git for Windows project to maintain the dozens of Windows-specific patch
>> series on top of upstream Git.
>>
>> The previous patch implemented the `label`, `bud` and `reset` commands
>> to label commits and to reset to a labeled commits. This patch adds the
>> `merge` command, with the following syntax:
>>
>>       merge <commit> <rev> <oneline>
>
> I'm concerned that this will be confusing for users. All of the other
> rebase commands replay the changes in the commit hash immediately
> following the command name. This command instead uses the first commit
> to specify the message which is different to both 'git merge' and the
> existing rebase commands. I wonder if it would be clearer to have 'merge
> -C <commit> <rev> ...' instead so it's clear which argument specifies
> the message and which the remote head to merge. It would also allow for
> 'merge -c <commit> <rev> ...' in the future for rewording an existing
> merge message and also avoid the slightly odd 'merge - <rev> ...'. Where
> it's creating new merges I'm not sure it's a good idea to encourage
> people to only have oneline commit messages by making it harder to edit
> them, perhaps it could take another argument to mean open the editor or
> not, though as Jake said I guess it's not that common.

I actually like the idea of re-using commit message options like -C,
-c,  and -m, so we could do:

merge -C <commit> ... to take message from commit
merge -c <commit> ...  to take the message from commit and open editor to edit
merge -m "<message>" ... to take the message from the quoted test
merge ... to merge and open commit editor with default message

This also, I think, allows us to not need to put the oneline on the
end, meaning we wouldn't have to quote the parent commit arguments
since we could use option semantics?

>
> One thought that just struck me - if a merge or reset command specifies
> an invalid label is it rescheduled so that it's still in the to-do list
> when the user edits it after rebase stops?
>
> In the future it might be nice if the label, reset and merge commands
> were validated when the to-do list is parsed so that the user gets
> immediate feedback if they try to create a label that is not a valid ref
> name or that they have a typo in a name given to reset or merge rather
> than the rebase stopping later.
>

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-18 16:25   ` Jacob Keller
  2018-01-18 21:13     ` Johannes Schindelin
  2018-01-18 21:24     ` Philip Oakley
@ 2018-01-22 21:25     ` Junio C Hamano
  2018-01-29 22:00       ` Johannes Schindelin
  2 siblings, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-01-22 21:25 UTC (permalink / raw)
  To: Jacob Keller; +Cc: Johannes Schindelin, Git mailing list

Jacob Keller <jacob.keller@gmail.com> writes:

> The code looks good, but I'm a little wary of adding bud which
> hard-codes a specific label. I suppose it does grant a bit of
> readability to the resulting script... ? It doesn't seem that
> important compared to use using "reset onto"? At least when
> documenting this it should be made clear that the "onto" label is
> special.

I do not think we would mind "bud" too much in the end result, but
the change in 1/8 is made harder to read than necessary with it.  It
is the only thing that needs "a single-letter command name may now
not have any argument after it" change to the parser among the three
things being added here, and it also needs to be added to the list
of special commands without arguments.

It would have been easier to reason about if addition of "bud" was
in its own patch done after label and reset are added.  And if done
as a separate step, perhaps it would have been easier to realize
that it would be a more future-proof solution for handling the
"special" ness of BUD to add a new "unsigned flags" word to
todo_command_info[] structure and using a bit that says "this does
not take an arg" than to hardcode "noop and bud are the commands
without args" in the code.  That hardcode was good enough when there
was only one thing in that special case.  Now it has two.

In a similar way, the code to special case label and reset just like
exec may also want to become more table driven, perhaps using
another bit in the same new flags word to say "this does not refer
to commit".  I think that can become [v2 1/N] while addition of "bud"
can be [v2 2/N] (after all, "bud" just does the same do_reset() with
hardcoded argument, so "label/reset" must come first).





^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/8] sequencer: introduce the `merge` command
  2018-01-18 15:35 ` [PATCH 2/8] sequencer: introduce the `merge` command Johannes Schindelin
                     ` (2 preceding siblings ...)
  2018-01-19 14:45   ` Phillip Wood
@ 2018-01-22 22:12   ` Junio C Hamano
  2018-01-29 22:15     ` Johannes Schindelin
  3 siblings, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-01-22 22:12 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Jacob Keller

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

>  	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
> +	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
> +	item->arg_len = (int)(eol - item->arg);
> +
>  	saved = *end_of_object_name;
> +	if (item->command == TODO_MERGE && *bol == '-' &&
> +	    bol + 1 == end_of_object_name) {
> +		item->commit = NULL;
> +		return 0;
> +	}
> +
>  	*end_of_object_name = '\0';
>  	status = get_oid(bol, &commit_oid);
>  	*end_of_object_name = saved;
>  
> -	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
> -	item->arg_len = (int)(eol - item->arg);
> -

Assigning to "saved" before the added "if we are doing merge and see
'-', do this special thing" is not only unnecessary, but makes the
logic in the non-special case harder to read.  The four things
"saved = *eol; *eol = 0; do_thing_using(bol); *eol = saved;" is a
single logical unit; keep them together.

This hunk may have been the most expedient way to coax "-" into the
location where a commit object name is expected; it looks ugly, but
for the limited purpose of this series it should do.

> @@ -2069,6 +2077,132 @@ static int do_reset(const char *name, int len)
>  	return ret;
>  }
>  
> +static int do_merge(struct commit *commit, const char *arg, int arg_len,
> +		    struct replay_opts *opts)
> +{
> +	int merge_arg_len;
> +	struct strbuf ref_name = STRBUF_INIT;
> +	struct commit *head_commit, *merge_commit, *i;
> +	struct commit_list *common, *j, *reversed = NULL;
> +	struct merge_options o;
> +	int ret;
> +	static struct lock_file lock;
> +
> +	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
> +		if (isspace(arg[merge_arg_len]))
> +			break;

Mental note: this scans for a whitespace, and tab is accepted
instead of SP, which presumably is to allow human typed string.

> +	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> +		return -1;
> +
> +	if (commit) {
> +		const char *message = get_commit_buffer(commit, NULL);
> +		const char *body;
> +		int len;
> +
> +		if (!message) {
> +			rollback_lock_file(&lock);
> +			return error(_("could not get commit message of '%s'"),
> +				     oid_to_hex(&commit->object.oid));
> +		}
> +		write_author_script(message);
> +		find_commit_subject(message, &body);
> +		len = strlen(body);
> +		if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
> +			error_errno(_("Could not write '%s'"),
> +				    git_path_merge_msg());
> +			unuse_commit_buffer(commit, message);
> +			rollback_lock_file(&lock);
> +			return -1;
> +		}
> +		unuse_commit_buffer(commit, message);
> +	} else {
> +		const char *p = arg + merge_arg_len;
> +		struct strbuf buf = STRBUF_INIT;
> +		int len;
> +
> +		strbuf_addf(&buf, "author %s", git_author_info(0));
> +		write_author_script(buf.buf);
> +		strbuf_reset(&buf);
> +
> +		p += strspn(p, " \t");

... and this matches the above mental note.  It allows consecutive
whitespaces as a separator, which is sensible behaviour.

> +		if (*p)
> +			len = strlen(p);
> +		else {
> +			strbuf_addf(&buf, "Merge branch '%.*s'",
> +				    merge_arg_len, arg);
> +			p = buf.buf;
> +			len = buf.len;
> +		}

So... "arg" received by this function can be a single non-whitespace
token, which is taken as the name of the branch being merged (in
this else clause).  Or it can also be followed by a single liner
message for the merge commit.  Presumably, this is for creating a
new merge (i.e. "commit==NULL" case), and preparing a proper log
message in the todo list is unrealistic, so this would be a
reasonable compromise.  Those users who want to write proper log
message could presumably follow such "merge" insn with a "x git
commit --amend" or something, I presume, if they really wanted to.

> +		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
> +			error_errno(_("Could not write '%s'"),
> +				    git_path_merge_msg());
> +			strbuf_release(&buf);
> +			rollback_lock_file(&lock);
> +			return -1;
> +		}
> +		strbuf_release(&buf);
> +	}

OK.  Now we have prepared the MERGE_MSG file and are ready to commit.

> +	head_commit = lookup_commit_reference_by_name("HEAD");
> +	if (!head_commit) {
> +		rollback_lock_file(&lock);
> +		return error(_("Cannot merge without a current revision"));
> +	}

Hmph, I would have expected to see this a lot earlier, before
dealing with the log message.  Leftover MERGE_MSG file after an
error will cause unexpected fallout to the end-user experience
(including what is shown by the shell prompt scripts), but if we do
this before the MERGE_MSG thing, we do not have to worry about
error codepath having to remove it.

> +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
> +	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> +	if (!merge_commit) {
> +		/* fall back to non-rewritten ref or commit */
> +		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
> +		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> +	}

OK, so "abc" in the example in the log message is looked up first as
a label and then we take a fallback to interpret as an object name.

Hopefully allowed names in "label" would be documented clearly in
later steps (I am guessing that "a name that can be used as a branch
name can be used as a label name and vice versa" or something like
that).

> +	if (!merge_commit) {
> +		error(_("could not resolve '%s'"), ref_name.buf);
> +		strbuf_release(&ref_name);
> +		rollback_lock_file(&lock);
> +		return -1;
> +	}
> +	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
> +		      git_path_merge_head(), 0);
> +	write_message("no-ff", 5, git_path_merge_mode(), 0);

These two calls gave me a "Huh?" moment; write_message() sounds like
it is allowed to be later updated with frills suitable for *_MSG
files we place in .git/ directory (iow, it is in principle OK if
commented out instructions common to these files are added to the
output by the function), but these want exact bytes passed in the
result, for which wrapper.c::write_file() is more appropriate.

Alternatively, perhaps write_message() can be dropped and its
callers can call wrapper.c::write_file() instead?  Such a clean-up
may require teaching the append-eol thing that write_message() wants
to wrapper.c::write_file(), but it shouldn't be a rocket science.

> +	common = get_merge_bases(head_commit, merge_commit);
> +	for (j = common; j; j = j->next)
> +		commit_list_insert(j->item, &reversed);
> +	free_commit_list(common);

I know this is copy&pasted code from "builtin/merge.c", but is there
a reason to reverse the common ancestor list here?

> +	read_cache();
> +	init_merge_options(&o);
> +	o.branch1 = "HEAD";
> +	o.branch2 = ref_name.buf;
> +	o.buffer_output = 2;
> +
> +	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
> +	if (ret <= 0)
> +		fputs(o.obuf.buf, stdout);
> +...

Other than these minor nits, looks quite promising.  Nicely done.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 3/8] sequencer: fast-forward merge commits, if possible
  2018-01-18 15:35 ` [PATCH 3/8] sequencer: fast-forward merge commits, if possible Johannes Schindelin
  2018-01-19 14:53   ` Phillip Wood
@ 2018-01-23 18:51   ` Junio C Hamano
  2018-01-29 22:18     ` Johannes Schindelin
  1 sibling, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-01-23 18:51 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Jacob Keller

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> +	/*
> +	 * If HEAD is not identical to the parent of the original merge commit,
> +	 * we cannot fast-forward.
> +	 */
> +	can_fast_forward = commit && commit->parents &&
> +		!oidcmp(&commit->parents->item->object.oid,
> +			&head_commit->object.oid);
> +

I think this expression and assignment should better be done much
later.  Are you going to update commit, commit->parents, etc. that
are involved in the computation in the meantime???

>  	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
>  	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
>  	if (!merge_commit) {
> @@ -2164,6 +2172,17 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
>  		rollback_lock_file(&lock);
>  		return -1;
>  	}
> +
> +	if (can_fast_forward && commit->parents->next &&
> +	    !commit->parents->next->next &&
> +	    !oidcmp(&commit->parents->next->item->object.oid,
> +		    &merge_commit->object.oid)) {

... Namely, here.  Because the earlier one is computing "are we
replaying exactly the same commit on top of exactly the same
state?", which is merely one half of "can we fast-forward", and
storing it in a variable whose name is over-promising way before it
becomes necessary.  The other half of "can we fast-forward?" logic
is the remainder of the if() condition we see above.  IOW, when
fully spelled, this code can fast-forward when we are replaying a
commit on top of exactly the same first-parent and the commit being
replayed is a single parent merge.

We may even want to get rid of can_fast_forward variable.

> +		strbuf_release(&ref_name);
> +		rollback_lock_file(&lock);
> +		return fast_forward_to(&commit->object.oid,
> +				       &head_commit->object.oid, 0, opts);
> +	}
> +
>  	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
>  		      git_path_merge_head(), 0);
>  	write_message("no-ff", 5, git_path_merge_mode(), 0);

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 3/8] sequencer: fast-forward merge commits, if possible
  2018-01-19 14:53   ` Phillip Wood
@ 2018-01-23 19:12     ` Junio C Hamano
  2018-01-24 10:32       ` Phillip Wood
  2018-01-29 21:47     ` Johannes Schindelin
  1 sibling, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-01-23 19:12 UTC (permalink / raw)
  To: Phillip Wood; +Cc: Johannes Schindelin, git, Jacob Keller

Phillip Wood <phillip.wood@talktalk.net> writes:

> On 18/01/18 15:35, Johannes Schindelin wrote:
>> 
>> Just like with regular `pick` commands, if we are trying to recreate a
>> merge commit, we now test whether the parents of said commit match HEAD
>> and the commits to be merged, and fast-forward if possible.
>> 
>> This is not only faster, but also avoids unnecessary proliferation of
>> new objects.
>
> I might have missed something but shouldn't this be checking opts->allow_ff?

Because the whole point of this mechanism is to recreate the
topology faithfully to the original, even if the original was a
redundant merge (which has a side parent that is an ancestor or a
descendant of the first parent), we should just point at the
original merge when the condition allows it, regardless of
opts->allow_ff.

I think it is a different matter if an insn to create a new merge
(i.e. "merge - <parent> <message>", not "merge <commit> <parent>")
should honor opts->allow_ff; because it is not about recreating an
existing history but is a way to create what did not exist before,
I think it is sensible if allow_ff option is honored.


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges
  2018-01-18 15:35 ` [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
  2018-01-18 21:39   ` Philip Oakley
  2018-01-19 10:34   ` Eric Sunshine
@ 2018-01-23 20:03   ` Junio C Hamano
  2018-01-29 22:37     ` Johannes Schindelin
  2 siblings, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-01-23 20:03 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Jacob Keller

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> The sequencer just learned a new commands intended to recreate branch

s/a //;

> structure (similar in spirit to --preserve-merges, but with a
> substantially less-broken design).
> ...
> @@ -2785,6 +2787,335 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
>  	strbuf_release(&sob);
>  }
>  
> +struct labels_entry {
> +	struct hashmap_entry entry;
> +	char label[FLEX_ARRAY];
> +};
> +
> +static int labels_cmp(const void *fndata, const struct labels_entry *a,
> +		      const struct labels_entry *b, const void *key)
> +{
> +	return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
> +}

label_oid() accesses state->labels hash using strihash() as the hash
function, but the final comparison between the entries in the same
hash buckets are done with case sensitivity.  It is unclear to me if
that is what was intended, and why.

> +struct string_entry {
> +	struct oidmap_entry entry;
> +	char string[FLEX_ARRAY];
> +};
> +
> +struct label_state {
> +	struct oidmap commit2label;
> +	struct hashmap labels;
> +	struct strbuf buf;
> +};
> +
> +static const char *label_oid(struct object_id *oid, const char *label,
> +			     struct label_state *state)
> +{
> +	struct labels_entry *labels_entry;
> +	struct string_entry *string_entry;
> +	struct object_id dummy;
> +	size_t len;
> +	int i;
> +
> +	string_entry = oidmap_get(&state->commit2label, oid);
> +	if (string_entry)
> +		return string_entry->string;
> +
> +	/*
> +	 * For "uninteresting" commits, i.e. commits that are not to be
> +	 * rebased, and which can therefore not be labeled, we use a unique
> +	 * abbreviation of the commit name. This is slightly more complicated
> +	 * than calling find_unique_abbrev() because we also need to make
> +	 * sure that the abbreviation does not conflict with any other
> +	 * label.
> +	 *
> +	 * We disallow "interesting" commits to be labeled by a string that
> +	 * is a valid full-length hash, to ensure that we always can find an
> +	 * abbreviation for any uninteresting commit's names that does not
> +	 * clash with any other label.
> +	 */
> +	if (!label) {
> +		char *p;
> +
> +		strbuf_reset(&state->buf);
> +		strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
> +		label = p = state->buf.buf;
> +
> +		find_unique_abbrev_r(p, oid->hash, default_abbrev);
> +
> +		/*
> +		 * We may need to extend the abbreviated hash so that there is
> +		 * no conflicting label.
> +		 */
> +		if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
> +			size_t i = strlen(p) + 1;
> +
> +			oid_to_hex_r(p, oid);
> +			for (; i < GIT_SHA1_HEXSZ; i++) {
> +				char save = p[i];
> +				p[i] = '\0';
> +				if (!hashmap_get_from_hash(&state->labels,
> +							   strihash(p), p))
> +					break;
> +				p[i] = save;
> +			}
> +		}

If oid->hash required full 40-hex to disambiguate, then
find-unique-abbrev would give 40-hex and we'd want the same "-<num>"
suffix technique employed below to make it consistently unique.  I
wonder if organizing the function this way ...

	if (!label)
		label = oid-to-hex(oid);

	if (label already exists or full oid) {
		make it unambiguous;
	}

... allows the resulting code easier to understand and manage.

A related tangent.  Does an auto-label given to "uninteresting"
commit need to be visible to end users?  I doubted it and that is
why I said oid-to-hex in the above, but if it is given to end users,
use of find-unique-abbrev-r is perfectly fine.

> +static int make_script_with_merges(struct pretty_print_context *pp,
> +				   struct rev_info *revs, FILE *out,
> +				   unsigned flags)
> +{
> + ...
> +	int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
> +	const char *p = abbr ? "p" : "pick", *l = abbr ? "l" : "label",
> +		 *t = abbr ? "t" : "reset", *b = abbr ? "b" : "bud",
> +		 *m = abbr ? "m" : "merge";

It would be easier to understand if these short named variables are
reserved only for temporary use, not as constants.  It is not too
much to spell 

	fprintf(out, "%s onto\n", cmd_label);

than

	fprintf(out, "%s onto\n", l);

and would save readers from head-scratching, wondering where the
last assignment to variable "l" is.

> +
> +	oidmap_init(&commit2todo, 0);
> +	oidmap_init(&state.commit2label, 0);
> +	hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
> +	strbuf_init(&state.buf, 32);
> +
> +	if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
> +		struct object_id *oid = &revs->cmdline.rev[0].item->oid;
> +		FLEX_ALLOC_STR(entry, string, "onto");
> +		oidcpy(&entry->entry.oid, oid);
> +		oidmap_put(&state.commit2label, entry);
> +	}
> +
> +	/*
> +	 * First phase:
> +	 * - get onelines for all commits
> +	 * - gather all branch tips (i.e. 2nd or later parents of merges)
> +	 * - label all branch tips
> +	 */

When an early part of a branch is merged and then the remaining part
of the same branch is merged again, "branch tip" and "2nd or later
parents of merges" would become different concepts.  The 2nd parent
of an early merge is not among the branch tips.

For the purpose of the "recreate the topology" algorithm, I am
imagining that you would need not just the tips but all the 2nd and
subsequent parents of merges, and my quick skimming tells me that
the following code grabs them correctly.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges
  2018-01-19 10:34   ` Eric Sunshine
@ 2018-01-23 20:13     ` Junio C Hamano
  2018-01-29 21:07       ` Johannes Schindelin
  2018-01-29 21:05     ` Johannes Schindelin
  1 sibling, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-01-23 20:13 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: Johannes Schindelin, Git List, Jacob Keller

Eric Sunshine <sunshine@sunshineco.com> writes:

>> +               is_octopus = to_merge && to_merge->next;
>> +
>> +               if (is_octopus)
>> +                       BUG("Octopus merges not yet supported");
>
> Is this a situation which the end-user can trigger by specifying a
> merge with more than two parents? If so, shouldn't this be just a
> normal error message rather than a (developer) bug message? Or, am I
> misunderstanding?

BUG() is "we wrote code carefully so that this should not trigger;
we do not _expect_ the code to reach here".  This one is expected to
trigger, and I agree with you that it should be die(), if the series
is meant to be released to the general public in the current form
(i.e. until the limitation is lifted so that it can handle an
octopus).

If the callers are made more careful to check if there is an octopus
involved and reject the request early, then seeing an octopus in
this location in a loop will become a BUG().

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-01-18 15:35 ` [PATCH 5/8] rebase: introduce the --recreate-merges option Johannes Schindelin
  2018-01-19 10:55   ` Eric Sunshine
@ 2018-01-23 20:22   ` Junio C Hamano
  2018-02-10 19:31     ` Johannes Schindelin
  2018-02-07  6:16   ` Sergey Organov
  2018-02-09  6:50   ` Sergey Organov
  3 siblings, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-01-23 20:22 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Jacob Keller

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> index 8a861c1e0d6..1d061373288 100644
> --- a/Documentation/git-rebase.txt
> +++ b/Documentation/git-rebase.txt
> @@ -368,6 +368,11 @@ The commit list format can be changed by setting the configuration option
>  rebase.instructionFormat.  A customized instruction format will automatically
>  have the long commit hash prepended to the format.
>  
> +--recreate-merges::
> +	Recreate merge commits instead of flattening the history by replaying
> +	merges. Merge conflict resolutions or manual amendments to merge
> +	commits are not preserved.
> +

It is sensible to postpone tackling "evil merges" in this initial
iteration of the series, and "manual amendments ... not preserved"
is a reasonable thing to document.  But do we want to say a bit more
about conflicting merges?  "conflict resolutions ... not preserved"
sounds as if it does not stop and instead record the result with
conflict markers without even letting rerere to kick in, which
certainly is not the impression you wanted to give to the readers.

I am imagining that it will stop and give control back to the end
user just like a conflicted "pick" would, and allow "rebase
--continue" to record resolution from the working tree, and just
like conflicted "pick", it would allow rerere() to help end users
recall previous resolution.


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 6/8] sequencer: handle autosquash and post-rewrite for merge commands
  2018-01-18 16:43   ` Jacob Keller
  2018-01-18 21:27     ` Johannes Schindelin
@ 2018-01-23 20:27     ` Junio C Hamano
  1 sibling, 0 replies; 412+ messages in thread
From: Junio C Hamano @ 2018-01-23 20:27 UTC (permalink / raw)
  To: Jacob Keller; +Cc: Johannes Schindelin, Git mailing list

Jacob Keller <jacob.keller@gmail.com> writes:

>>  static int is_per_worktree_ref(const char *refname)
>>  {
>>         return !strcmp(refname, "HEAD") ||
>> -               starts_with(refname, "refs/bisect/");
>> +               starts_with(refname, "refs/bisect/") ||
>> +               starts_with(refname, "refs/rewritten/");
>>  }
>
> Would this part make more sense to move into the commit that
> introduces writing these refs, or does it only matter once you start
> this step here?

Good spotting.  I too was wondering about multiple worktrees when I
saw the "label" thing introduced.  It probably makes sense to move
this to that step.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 0/8] rebase -i: offer to recreate merge commits
  2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
                   ` (10 preceding siblings ...)
  2018-01-19 20:25 ` [PATCH 0/8] rebase -i: offer to recreate merge commits Junio C Hamano
@ 2018-01-23 20:29 ` Junio C Hamano
  2018-01-29 22:53   ` Johannes Schindelin
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
  12 siblings, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-01-23 20:29 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Jacob Keller

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> My original attempt was --preserve-merges, but that design was so
> limited that I did not even enable it in interactive mode.
> ...
> There are more patches in the pipeline, based on this patch series, but
> left for later in the interest of reviewable patch series: one mini
> series to use the sequencer even for `git rebase -i --root`, and another
> one to add support for octopus merges to --recreate-merges.

I left comments on a handful of them, but I do not think any of them
spotted a grave design issue to be a show stopper.  Overall, the
series was quite a pleasant read, even with those minor nits and
rooms for improvements.

Thanks.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 3/8] sequencer: fast-forward merge commits, if possible
  2018-01-23 19:12     ` Junio C Hamano
@ 2018-01-24 10:32       ` Phillip Wood
  2018-01-24 18:51         ` Junio C Hamano
  0 siblings, 1 reply; 412+ messages in thread
From: Phillip Wood @ 2018-01-24 10:32 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Johannes Schindelin, git, Jacob Keller

On 23/01/18 19:12, Junio C Hamano wrote:
> Phillip Wood <phillip.wood@talktalk.net> writes:
> 
>> On 18/01/18 15:35, Johannes Schindelin wrote:
>>>
>>> Just like with regular `pick` commands, if we are trying to recreate a
>>> merge commit, we now test whether the parents of said commit match HEAD
>>> and the commits to be merged, and fast-forward if possible.
>>>
>>> This is not only faster, but also avoids unnecessary proliferation of
>>> new objects.
>>
>> I might have missed something but shouldn't this be checking opts->allow_ff?
> 
> Because the whole point of this mechanism is to recreate the
> topology faithfully to the original, even if the original was a
> redundant merge (which has a side parent that is an ancestor or a
> descendant of the first parent), we should just point at the
> original merge when the condition allows it, regardless of
> opts->allow_ff.

I agree that the merge should be recreated, but I was thinking of 
something slightly different. Currently the sequencer uses 
opts->allow_ff to control whether a new commit with the same contents 
should be created even if the existing one could be reused. So I was 
querying whether we should recreate the commit when the user run 'git 
rebase --recreate-merges --no-ff' rather than just reusing it. As merges 
also have another meaning for fast-forward the terminology gets confusing.

> I think it is a different matter if an insn to create a new merge
> (i.e. "merge - <parent> <message>", not "merge <commit> <parent>")
> should honor opts->allow_ff; because it is not about recreating an
> existing history but is a way to create what did not exist before,
> I think it is sensible if allow_ff option is honored.

This is the merge sense of 'fast-forward' not the existing sequencer 
sense, without thinking about it more I'm not sure if one command line 
option for rebase is sufficient to cover both uses.

Best Wishes

Phillip


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 3/8] sequencer: fast-forward merge commits, if possible
  2018-01-24 10:32       ` Phillip Wood
@ 2018-01-24 18:51         ` Junio C Hamano
  0 siblings, 0 replies; 412+ messages in thread
From: Junio C Hamano @ 2018-01-24 18:51 UTC (permalink / raw)
  To: Phillip Wood; +Cc: Johannes Schindelin, git, Jacob Keller

Phillip Wood <phillip.wood@talktalk.net> writes:

> I agree that the merge should be recreated, but I was thinking of
> something slightly different. Currently the sequencer uses
> opts->allow_ff to control whether a new commit with the same contents
> should be created even if the existing one could be reused.

Ahh, OK.  I misunderstood what you meant.  Yes, what you said makes
sense to me.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-19  8:59   ` Eric Sunshine
@ 2018-01-24 22:01     ` Junio C Hamano
  2018-01-29 20:55       ` Johannes Schindelin
  2018-01-29 20:50     ` Johannes Schindelin
  1 sibling, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-01-24 22:01 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: Johannes Schindelin, Git List, Jacob Keller

Eric Sunshine <sunshine@sunshineco.com> writes:

>> +static int do_reset(const char *name, int len)
>> +{
>> +       [...]
>> +       if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
>> +               return -1;
>> +
>> +       for (i = 0; i < len; i++)
>> +               if (isspace(name[i]))
>> +                       len = i;
>
> What is the purpose of this loop? I could imagine that it's trying to
> strip all whitespace from the end of 'name', however, to do that it
> would iterate backward, not forward. (Or perhaps it's trying to
> truncate at the first space, but then it would need to invert the
> condition or use 'break'.) Am I missing something obvious?

I must be missing the same thing.  Given that the callers of
do_reset(), other than the "bug" thing that passes the hard coded
"onto", uses item->arg/item->arg_len which includes everything after
the insn word on the line in the todo list, I do suspect that the
intention is to stop at the first whitespace char to avoid creating
a ref with whitespace in it, i.e. it is a bug that can be fixed with
s/len = i/break/.

The code probably should further check the resulting string with
check_ref_format() to detect strange chars and char sequences that
make the resulting refname invalid.  For example, you would not want
to allow a label with two consecutive periods in it.



^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 9/8] [DO NOT APPLY, but squash?] git-rebase--interactive: clarify arguments
  2018-01-20  9:14         ` Jacob Keller
@ 2018-01-29 17:02           ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 17:02 UTC (permalink / raw)
  To: Jacob Keller; +Cc: Junio C Hamano, Stefan Beller, Git mailing list

Hi,

On Sat, 20 Jan 2018, Jacob Keller wrote:

> On Fri, Jan 19, 2018 at 12:30 PM, Junio C Hamano <gitster@pobox.com> wrote:
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >
> >> Good idea! I would rather do it as an introductory patch (that only
> >> converts the existing list).
> >>
> >> As to `merge`: it is a bit more complicated ;-)
> >>
> >>       m, merge <original-merge-commit> ( <label> | "<label>..." ) [<oneline>]
> >>               create a merge commit using the original merge commit's
> >>               message (or the oneline, if "-" is given). Use a quoted
> >>               list of commits to be merged for octopus merges.
> >
> > Is it just the message that is being reused?
> >
> > Aren't the trees of the original commit and its parents participate
> > in creating the tree of the recreated merge?  One way to preserve an
> > originally evil merge is to notice how it was made by taking the
> > difference between the result of mechanical merge of original merge
> > parents and the original merge result, and carry it forward when
> > recreating the merge across new parents.  Just being curious.
> >
> 
> It looks like currently that only the commit is kept, with no attempt
> to recreate evil merges.

Yep. I even documented that somewhere ;-)

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-18 21:24     ` Philip Oakley
  2018-01-18 21:28       ` Jacob Keller
@ 2018-01-29 20:28       ` Johannes Schindelin
  1 sibling, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 20:28 UTC (permalink / raw)
  To: Philip Oakley; +Cc: Jacob Keller, Git mailing list, Junio C Hamano

Hi Philip,

On Thu, 18 Jan 2018, Philip Oakley wrote:

> From: "Jacob Keller" <jacob.keller@gmail.com>
> > On Thu, Jan 18, 2018 at 7:35 AM, Johannes Schindelin
> > <johannes.schindelin@gmx.de> wrote:
> > > This commit implements the commands to label, and to reset to, given
> > > revisions. The syntax is:
> > >
> > >         label <name>
> > >         reset <name>
> > >
> > > As a convenience shortcut, also to improve readability of the generated
> > > todo list, a third command is introduced: bud. It simply resets to the
> > > "onto" revision, i.e. the commit onto which we currently rebase.
> > >
> >
> > The code looks good, but I'm a little wary of adding bud which
> > hard-codes a specific label. I suppose it does grant a bit of
> > readability to the resulting script... ? It doesn't seem that
> > important compared to use using "reset onto"? At least when
> > documenting this it should be made clear that the "onto" label is
> > special.
> >
> > Thanks,
> > Jake.
> 
> I'd agree.
> 
> The special 'onto' label should be fully documented, and the commit message
> should indicate which patch actually defines it (and all its corner cases and
> fall backs if --onto isn't explicitly given..)

I hoped that the example todo lists would clarify that 'onto' is just the
name of the first label:

	label onto

is *literally* how all of those todo lists start. And that is why there
are no possible concerns about any missing `--onto` argument: that
argument is irrelevant for that label.

Maybe it is just a bad name. Maybe `START` would be a better label.

What do you think?

> Likewise the choice of 'bud' should be explained with some nice
> phraseology indicating that we are growing the new flowering from the
> bud, otherwise the word is a bit too short and sudden for easy
> explanation.

I dropped the `bud` command.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 8/8] rebase -i: introduce --recreate-merges=no-rebase-cousins
  2018-01-18 22:00   ` Philip Oakley
@ 2018-01-29 20:42     ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 20:42 UTC (permalink / raw)
  To: Philip Oakley; +Cc: git, Junio C Hamano, Jacob Keller

Hi Philip,

On Thu, 18 Jan 2018, Philip Oakley wrote:

> From: "Johannes Schindelin" <johannes.schindelin@gmx.de>
> > This one is a bit tricky to explain, so let's try with a diagram:
> >
> >        C
> >      /   \
> > A - B - E - F
> >  \   /
> >    D
> >
> > To illustrate what this new mode is all about, let's consider what
> > happens upon `git rebase -i --recreate-merges B`, in particular to
> > the commit `D`. In the default mode, the new branch structure is:
> >
> >      --- C' --
> >      /         \
> > A - B ------ E' - F'
> >      \    /
> >        D'
> >
> > This is not really preserving the branch topology from before! The
> > reason is that the commit `D` does not have `B` as ancestor, and
> > therefore it gets rebased onto `B`.
> >
> > However, when recreating branch structure, there are legitimate use
> > cases where one might want to preserve the branch points of commits that
> > do not descend from the <upstream> commit that was passed to the rebase
> > command, e.g. when a branch from core Git's `next` was merged into Git
> > for Windows' master we will not want to rebase those commits on top of a
> > Windows-specific commit. In the example above, the desired outcome would
> > look like this:
> >
> >      --- C' --
> >      /         \
> > A - B ------ E' - F'
> >  \        /
> >   -- D' --
> 
> I'm not understanding this. I see that D properly starts from A, but
> don't see why it is now D'. Surely it's unchanged.

It is not necessarily unchanged, because this is an *interactive* rebase.
If you mark `D` for `reword`, for example, it may be changed.

I use the label D' in the mathematical sense, to indicate that D' is
derived from D. It may even be identical to D, but the point is that it is
in the todo list of the interactive rebase, so it can be changed. As
opposed to, say, A and B. Those cannot be changed in this interactive
rebase.

> Maybe it's the arc/node confusion. Maybe even spell out that the rebased
> commits from the command are B..HEAD, but that includes D, which may not
> be what folk had expected. (not even sure if the reflog comes into
> determining merge-bases here..)
> 
> I do think an exact definition is needed (e.g. via --ancestry-path or
> its equivalent?).

I don't find "ancestry path" any more intuitive a term than the
mathematically correct "uncomparable".

If you have a better way to explain this (without devolving into
mathematical terminology), please let's hear it.

Don't get me wrong, as a mathematician I am comfortable with very precise
descriptions involving plenty of Greek symbols.

But this documentation, and these commit messages do not target myself. I
know perfectly well what I am talking about here. The target audience are
software developers who may not have a background in mathematics, who do
not even want to fully understand what the heck constitutes a Directed
Acyclic Graph.

So what we need here is plain English. And I had thought that the analogy
with the family tree would be intuitive enough for even math haters to
understand easily and quickly...

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-19  8:59   ` Eric Sunshine
  2018-01-24 22:01     ` Junio C Hamano
@ 2018-01-29 20:50     ` Johannes Schindelin
  2018-01-30  7:12       ` Eric Sunshine
  1 sibling, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 20:50 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: Git List, Junio C Hamano, Jacob Keller

Hi Eric,

On Fri, 19 Jan 2018, Eric Sunshine wrote:

> On Thu, Jan 18, 2018 at 10:35 AM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > [...]
> > +static int do_reset(const char *name, int len)
> > +{
> > +       [...]
> > +       if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> > +               return -1;
> > +
> > +       for (i = 0; i < len; i++)
> > +               if (isspace(name[i]))
> > +                       len = i;
> 
> What is the purpose of this loop? I could imagine that it's trying to
> strip all whitespace from the end of 'name', however, to do that it
> would iterate backward, not forward. (Or perhaps it's trying to
> truncate at the first space, but then it would need to invert the
> condition or use 'break'.) Am I missing something obvious?

Yes, you are missing something obvious. The idea of the `reset` command is
that it not only has a label, but also the oneline of the original commit:

	reset branch-point sequencer: prepare for cleanup

In this instance, `branch-point` is the label. And for convenience of the
person editing, it also has the oneline. This came in *extremely* handy
when editing the commit topology in Git for Windows, i.e. when introducing
topic branches or flattening them.

In the Git garden shears, I separated the two arguments via `#`:

	reset branch-point # sequencer: prepare for cleanup

I guess that is actually more readable, so I will introduce that into this
patch series, too.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-24 22:01     ` Junio C Hamano
@ 2018-01-29 20:55       ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 20:55 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Eric Sunshine, Git List, Jacob Keller

Hi Junio,

On Wed, 24 Jan 2018, Junio C Hamano wrote:

> Eric Sunshine <sunshine@sunshineco.com> writes:
> 
> >> +static int do_reset(const char *name, int len)
> >> +{
> >> +       [...]
> >> +       if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> >> +               return -1;
> >> +
> >> +       for (i = 0; i < len; i++)
> >> +               if (isspace(name[i]))
> >> +                       len = i;
> >
> > What is the purpose of this loop? I could imagine that it's trying to
> > strip all whitespace from the end of 'name', however, to do that it
> > would iterate backward, not forward. (Or perhaps it's trying to
> > truncate at the first space, but then it would need to invert the
> > condition or use 'break'.) Am I missing something obvious?
> 
> I must be missing the same thing.  Given that the callers of
> do_reset(), other than the "bug" thing that passes the hard coded
> "onto", uses item->arg/item->arg_len which includes everything after
> the insn word on the line in the todo list, I do suspect that the
> intention is to stop at the first whitespace char to avoid creating
> a ref with whitespace in it, i.e. it is a bug that can be fixed with
> s/len = i/break/.
> 
> The code probably should further check the resulting string with
> check_ref_format() to detect strange chars and char sequences that
> make the resulting refname invalid.  For example, you would not want
> to allow a label with two consecutive periods in it.

The code already checks that by creating a ref.

No need to go crazy and validate ref names every time we parse the todo
list (which is quite often), eh?

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges
  2018-01-19 10:34   ` Eric Sunshine
  2018-01-23 20:13     ` Junio C Hamano
@ 2018-01-29 21:05     ` Johannes Schindelin
  1 sibling, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 21:05 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: Git List, Junio C Hamano, Jacob Keller

Hi Eric,

On Fri, 19 Jan 2018, Eric Sunshine wrote:

> On Thu, Jan 18, 2018 at 10:35 AM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> 
> > structure (similar in spirit to --preserve-merges, but with a
> > substantially less-broken design).
> > [...]
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> > diff --git a/sequencer.c b/sequencer.c
> > @@ -2785,6 +2787,335 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
> > +static const char *label_oid(struct object_id *oid, const char *label,
> > +                            struct label_state *state)
> > +{
> > +       [...]
> > +       } else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
> > +                   !get_oid_hex(label, &dummy)) ||
> > +                  hashmap_get_from_hash(&state->labels,
> > +                                        strihash(label), label)) {
> > +               /*
> > +                * If the label already exists, or if the label is a valid full
> > +                * OID, we append a dash and a number to make it unique.
> > +                */
> > +               [...]
> > +               for (i = 2; ; i++) {
> 
> Why '2'? Is there some non-obvious significance to this value?

I personally found it irritating to have labels "sequencer",
"sequencer-1". It sounds *wrong* to have a "-1". Because it is the second
label referring to the term "sequencer". So if there are two labels that
both want to be named "sequencer", the first one wins, and the second one
will be called "sequencer-2".

Hence the 2.

> > +static int make_script_with_merges(struct pretty_print_context *pp,
> > +                                  struct rev_info *revs, FILE *out,
> > +                                  unsigned flags)
> > +{
> > +       [...]
> > +               is_octopus = to_merge && to_merge->next;
> > +
> > +               if (is_octopus)
> > +                       BUG("Octopus merges not yet supported");
> 
> Is this a situation which the end-user can trigger by specifying a
> merge with more than two parents? If so, shouldn't this be just a
> normal error message rather than a (developer) bug message? Or, am I
> misunderstanding?

You are misunderstanding.

This is just a place-holder here. The patches to introduce support for
octopus merges are already written. They are lined up after this here
patch series, is all.

As such, please do not occupy your mind on the specifics or even the
upper-case of the "Octopus". This line is here only as a hint for the
reviewer that this is not yet implemented. And BUG(...) was chosen because
that way, we are not even tempted to waste the time of translators.

Speaking of wasting time... let's move on to further interesting code
reviews.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges
  2018-01-23 20:13     ` Junio C Hamano
@ 2018-01-29 21:07       ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 21:07 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Eric Sunshine, Git List, Jacob Keller

Hi Junio,

On Tue, 23 Jan 2018, Junio C Hamano wrote:

> Eric Sunshine <sunshine@sunshineco.com> writes:
> 
> >> +               is_octopus = to_merge && to_merge->next;
> >> +
> >> +               if (is_octopus)
> >> +                       BUG("Octopus merges not yet supported");
> >
> > Is this a situation which the end-user can trigger by specifying a
> > merge with more than two parents? If so, shouldn't this be just a
> > normal error message rather than a (developer) bug message? Or, am I
> > misunderstanding?
> 
> BUG() is "we wrote code carefully so that this should not trigger;
> we do not _expect_ the code to reach here".  This one is expected to
> trigger, and I agree with you that it should be die(), if the series
> is meant to be released to the general public in the current form
> (i.e. until the limitation is lifted so that it can handle an
> octopus).
> 
> If the callers are made more careful to check if there is an octopus
> involved and reject the request early, then seeing an octopus in
> this location in a loop will become a BUG().

This has occupied both of you for way too long.

It is *not interesting*. What *is* interesting is for example the
discussion about the "cousin commits". And maybe both of you gentle
persons can spend your brain cycles splendidly by trying to come up with a
better term. Or by trying to beat out obvious or not-so-obvious bugs in
the code.

Seriously, I am not interested in a discussion about BUG() vs die() as
long as there may be real bugs hiding.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-01-19 10:55   ` Eric Sunshine
@ 2018-01-29 21:09     ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 21:09 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: Git List, Junio C Hamano, Jacob Keller

Hi Eric,

On Fri, 19 Jan 2018, Eric Sunshine wrote:

> On Thu, Jan 18, 2018 at 10:35 AM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > [...]
> > With this patch, the goodness of the Git garden shears comes to `git
> > rebase -i` itself. Passing the `--recreate-merges` option will generate
> > a todo list that can be understood readily, and where it is obvious
> > how to reorder commits. New branches can be introduced by inserting
> > `label` commands and calling `merge - <label> <oneline>`. And once this
> > mode has become stable and universally accepted, we can deprecate the
> > design mistake that was `--preserve-merges`.
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> > diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> > @@ -900,6 +900,7 @@ fi
> >  if test t != "$preserve_merges"
> >  then
> >         git rebase--helper --make-script ${keep_empty:+--keep-empty} \
> > +               ${recreate_merges:+--recreate-merges} \
> 
> If the user specifies both --preserve-merges and --recreate-merges, it
> looks like --preserve-merges will take precedence.
> 
> Should git-rebase.sh have a mutual-exclusion check and error out if
> both are specified?

Maybe. I welcome you to contribute such a patch once recreate-merges made
it into the code base.

In other words: this would be premature optimization. We're not at that
stage yet.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to resettherevision
  2018-01-19 12:24   ` [PATCH 1/8] sequencer: introduce new commands to resettherevision Phillip Wood
  2018-01-19 18:55     ` Phillip Wood
@ 2018-01-29 21:23     ` Johannes Schindelin
  1 sibling, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 21:23 UTC (permalink / raw)
  To: phillip.wood
  Cc: git, Junio C Hamano, Jacob Keller, Philip Oakley, Eric Sunshine

Hi Phillip,

On Fri, 19 Jan 2018, Phillip Wood wrote:

> 
> On 18/01/18 15:35, Johannes Schindelin wrote:
> 
> > This idea was developed in Git for Windows' Git garden shears (that
> > are used to maintain the "thicket of branches" on top of upstream
> > Git), and this patch is part of the effort to make it available to a
> > wider audience, as well as to make the entire process more robust (by
> > implementing it in a safe and portable language rather than a Unix
> > shell script).
> > 
> > This commit implements the commands to label, and to reset to, given
> > revisions. The syntax is:
> > 
> > 	label <name>
> > 	reset <name>
> 
> If I've understood the code below correctly then reset will clobber
> untracked files, this is the opposite behaviour to what happens when
> tries to checkout <onto> at the start of a rebase - then it will fail if
> untracked files would be overwritten.

This would be completely unintentional, I will verify that untracked files
are not clobbered.

However, in practice this should not happen because the intended use case
is for revisions to be labeled *before* checking them out at a later
stage. Therefore, the files that would be clobbered would already have
been tracked in the revision when it was labeled, and I do not quite see
how those files could become untracked without playing sloppy exec games
in between.

> > Internally, the `label <name>` command creates the ref
> > `refs/rewritten/<name>`. This makes it possible to work with the labeled
> > revisions interactively, or in a scripted fashion (e.g. via the todo
> > list command `exec`).
> 
> If a user has two work trees and runs a rebase in each with the same
> label name, they'll clobber each other. I'd suggest storing them under
> refs/rewritten/<branch-name or detached HEAD SHA> instead. If the user
> tries to rebase a second worktree with the same detached HEAD as an
> existing rebase then refuse to start.

That is why a later patch marks those refs/rewritten/ refs as
worktree-local.

> > +static int do_label(const char *name, int len)
> > +{
> > +	struct ref_store *refs = get_main_ref_store();
> > +	struct ref_transaction *transaction;
> > +	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
> > +	struct strbuf msg = STRBUF_INIT;
> > +	int ret = 0;
> > +	struct object_id head_oid;
> > +
> > +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> > +	strbuf_addf(&msg, "label '%.*s'", len, name);
> 
> The other reflog messages below have a (rebase -i) prefix

Good point. I changed it to "rebase -i (label)".

> > +	transaction = ref_store_transaction_begin(refs, &err);
> > +	if (!transaction ||
> > +	    get_oid("HEAD", &head_oid) ||
> > +	    ref_transaction_update(transaction, ref_name.buf, &head_oid, NULL,
> > +				   0, msg.buf, &err) < 0 ||
> > +	    ref_transaction_commit(transaction, &err)) {
> > +		error("%s", err.buf);
> 
> if get_oid() fails then err is empty so there wont be an message after
> the 'error: '

Yep, that would be nasty. Fixed.

> > +static int do_reset(const char *name, int len)
> > +{
> > +	struct strbuf ref_name = STRBUF_INIT;
> > +	struct object_id oid;
> > +	struct lock_file lock = LOCK_INIT;
> > +	struct tree_desc desc;
> > +	struct tree *tree;
> > +	struct unpack_trees_options opts;
> > +	int ret = 0, i;
> > +
> > +	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> > +		return -1;
> > +
> > +	for (i = 0; i < len; i++)
> > +		if (isspace(name[i]))
> > +			len = i;
> 
> If name starts with any white space then I think this effectively
> truncates name to a bunch of white space which doesn't sound right. I'm
> not sure how this is being called, but it might be better to clean up
> name when the to-do list is parsed instead.

The left-trimming of the name was already performed as part of the todo
list parsing.

And we are not really right-trimming here. We are splitting a line of the
form

	reset <label> <oneline>

In fact, after reflecting about it, I changed the code so that it would
now even read:

	reset <label> # <oneline>

So the code really is doing the intended thing here.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to resettherevision
  2018-01-19 18:59       ` Jacob Keller
@ 2018-01-29 21:25         ` Johannes Schindelin
  2018-01-29 21:29           ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 21:25 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Phillip Wood, Git mailing list, Junio C Hamano, Philip Oakley,
	Eric Sunshine

Hi Jake,

On Fri, 19 Jan 2018, Jacob Keller wrote:

> On Fri, Jan 19, 2018 at 10:55 AM, Phillip Wood
> <phillip.wood@talktalk.net> wrote:
> > On 19/01/18 12:24, Phillip Wood wrote:
> >>
> >> On 18/01/18 15:35, Johannes Schindelin wrote:
> >>>
> >>> Internally, the `label <name>` command creates the ref
> >>> `refs/rewritten/<name>`. This makes it possible to work with the labeled
> >>> revisions interactively, or in a scripted fashion (e.g. via the todo
> >>> list command `exec`).
> >>
> >> If a user has two work trees and runs a rebase in each with the same
> >> label name, they'll clobber each other. I'd suggest storing them under
> >> refs/rewritten/<branch-name or detached HEAD SHA> instead. If the user
> >> tries to rebase a second worktree with the same detached HEAD as an
> >> existing rebase then refuse to start.
> >>
> >
> > Ah this isn't a concern after all as patch 5 makes refs/rewritten local
> > to the worktree. Perhaps you could move that part of patch 5 here or add
> > a note to the commit message that it will become worktree local later in
> > the series
> >
> > Best Wishes
> >
> > Phillip
> 
> I'd rather it be included here as well.

But it would have been really easy to overlook in here. I really want this
to be a separate commit, also to have a chance to get this done
*differently* if somebody comes up with a splendid idea how to do that
(because hard-coding feels quite dirty).

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to resettherevision
  2018-01-29 21:25         ` Johannes Schindelin
@ 2018-01-29 21:29           ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 21:29 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Phillip Wood, Git mailing list, Junio C Hamano, Philip Oakley,
	Eric Sunshine

Hi,

On Mon, 29 Jan 2018, Johannes Schindelin wrote:

> On Fri, 19 Jan 2018, Jacob Keller wrote:
> 
> > On Fri, Jan 19, 2018 at 10:55 AM, Phillip Wood
> > <phillip.wood@talktalk.net> wrote:
> > > On 19/01/18 12:24, Phillip Wood wrote:
> > >>
> > >> On 18/01/18 15:35, Johannes Schindelin wrote:
> > >>>
> > >>> Internally, the `label <name>` command creates the ref
> > >>> `refs/rewritten/<name>`. This makes it possible to work with the labeled
> > >>> revisions interactively, or in a scripted fashion (e.g. via the todo
> > >>> list command `exec`).
> > >>
> > >> If a user has two work trees and runs a rebase in each with the same
> > >> label name, they'll clobber each other. I'd suggest storing them under
> > >> refs/rewritten/<branch-name or detached HEAD SHA> instead. If the user
> > >> tries to rebase a second worktree with the same detached HEAD as an
> > >> existing rebase then refuse to start.
> > >>
> > >
> > > Ah this isn't a concern after all as patch 5 makes refs/rewritten local
> > > to the worktree. Perhaps you could move that part of patch 5 here or add
> > > a note to the commit message that it will become worktree local later in
> > > the series
> > >
> > > Best Wishes
> > >
> > > Phillip
> > 
> > I'd rather it be included here as well.
> 
> But it would have been really easy to overlook in here. I really want this
> to be a separate commit, also to have a chance to get this done
> *differently* if somebody comes up with a splendid idea how to do that
> (because hard-coding feels quite dirty).

BTW there is an additional good reason why the patch to make
refs/rewritten/* worktree-local is so far away: that is the first time in
the patch series when we can test this really effectively; at that stage
we can easily just add to t3430 because all the building blocks for
`rebase -i --recreate-merges` are in place.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/8] sequencer: introduce the `merge` command
  2018-01-20  9:18     ` Jacob Keller
@ 2018-01-29 21:41       ` Johannes Schindelin
  2018-01-31 13:48         ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 21:41 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Phillip Wood, Git mailing list, Junio C Hamano, Eric Sunshine

Hi Jake & Phillip,

On Sat, 20 Jan 2018, Jacob Keller wrote:

> On Fri, Jan 19, 2018 at 6:45 AM, Phillip Wood <phillip.wood@talktalk.net> wrote:
> > On 18/01/18 15:35, Johannes Schindelin wrote:
> >>
> >> This patch is part of the effort to reimplement `--preserve-merges` with
> >> a substantially improved design, a design that has been developed in the
> >> Git for Windows project to maintain the dozens of Windows-specific patch
> >> series on top of upstream Git.
> >>
> >> The previous patch implemented the `label`, `bud` and `reset` commands
> >> to label commits and to reset to a labeled commits. This patch adds the
> >> `merge` command, with the following syntax:
> >>
> >>       merge <commit> <rev> <oneline>
> >
> > I'm concerned that this will be confusing for users. All of the other
> > rebase commands replay the changes in the commit hash immediately
> > following the command name. This command instead uses the first commit
> > to specify the message which is different to both 'git merge' and the
> > existing rebase commands. I wonder if it would be clearer to have 'merge
> > -C <commit> <rev> ...' instead so it's clear which argument specifies
> > the message and which the remote head to merge. It would also allow for
> > 'merge -c <commit> <rev> ...' in the future for rewording an existing
> > merge message and also avoid the slightly odd 'merge - <rev> ...'. Where
> > it's creating new merges I'm not sure it's a good idea to encourage
> > people to only have oneline commit messages by making it harder to edit
> > them, perhaps it could take another argument to mean open the editor or
> > not, though as Jake said I guess it's not that common.
> 
> I actually like the idea of re-using commit message options like -C,
> -c,  and -m, so we could do:
> 
> merge -C <commit> ... to take message from commit

That is exactly how the Git garden shears do it.

I found it not very readable. That is why I wanted to get away from it in
--recreate-merges.

> merge -c <commit> ...  to take the message from commit and open editor to edit
> merge -m "<message>" ... to take the message from the quoted test
> merge ... to merge and open commit editor with default message
> 
> This also, I think, allows us to not need to put the oneline on the
> end, meaning we wouldn't have to quote the parent commit arguments
> since we could use option semantics?

The oneline is there primarily to give you, the reader, a clue when
reading and editing the todo list.

Reusing it for the `merge -` command was only an afterthought.

> > One thought that just struck me - if a merge or reset command specifies
> > an invalid label is it rescheduled so that it's still in the to-do list
> > when the user edits it after rebase stops?

It is not rescheduled, because the command already failed, so we know it
is bad.

You have to go edit the todo list, possibly after copying the faulty
command from `git status`' output.

> > In the future it might be nice if the label, reset and merge commands
> > were validated when the to-do list is parsed so that the user gets
> > immediate feedback if they try to create a label that is not a valid
> > ref name or that they have a typo in a name given to reset or merge
> > rather than the rebase stopping later.

There are too many possible errors to make this fool-proof. What if the
ref name is valid, but there was no `label` command yet?  What if there
*has* been a `label` command but it is now stuck in the `done` file (which
we do not parse, ever)? What if the user specified two label commands with
the same label?

It sounds like an exercise in futility to try to catch these things in the
parser.

And keep in mind that the parser is used

- when shortening the commit names
- when checking the todo list for accidentally dropped picks
- when skipping unnecessary picks
- when rearranging fixup!/squash! commands
- when adding `exec` commands specified via `-x`

I am not sure that I would come out in favor of trying to catch ref name
errors during parsing time if I wanted to balance bang vs buck.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 3/8] sequencer: fast-forward merge commits, if possible
  2018-01-19 14:53   ` Phillip Wood
  2018-01-23 19:12     ` Junio C Hamano
@ 2018-01-29 21:47     ` Johannes Schindelin
  1 sibling, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 21:47 UTC (permalink / raw)
  To: phillip.wood; +Cc: git, Junio C Hamano, Jacob Keller

Hi Phillip,

On Fri, 19 Jan 2018, Phillip Wood wrote:

> On 18/01/18 15:35, Johannes Schindelin wrote:
> > 
> > Just like with regular `pick` commands, if we are trying to recreate a
> > merge commit, we now test whether the parents of said commit match HEAD
> > and the commits to be merged, and fast-forward if possible.
> > 
> > This is not only faster, but also avoids unnecessary proliferation of
> > new objects.
> 
> I might have missed something but shouldn't this be checking
> opts->allow_ff?

Good point. This is the type of review for which I was hoping.

> Another possible optimization is that if the parent branches have only
> reworded commits or some commits that have been squashed but no other
> changes then their trees will be the same as in the original merge
> commit and so could be reused without calling merge_recursive().

True. It is also a bit involved to check this condition, and I am not sure
that it is worth the effort for my use case.

So I would invite you to work on this after this patch series settles, if
you are interested.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 0/8] rebase -i: offer to recreate merge commits
  2018-01-19 20:25 ` [PATCH 0/8] rebase -i: offer to recreate merge commits Junio C Hamano
@ 2018-01-29 21:53   ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 21:53 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Jacob Keller

Hi Junio,

On Fri, 19 Jan 2018, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Think of --recreate-merges as "--preserve-merges done right". It
> > introduces new verbs for the todo list, `label`, `reset` and `merge`.
> > For a commit topology like this:
> >
> >             A - B - C
> >               \   /
> >                 D
> >
> > the generated todo list would look like this:
> >
> >             # branch D
> >             pick 0123 A
> >             label branch-point
> >             pick 1234 D
> >             label D
> >
> >             reset branch-point
> >             pick 2345 B
> >             merge 3456 D C
> 
> Yup.  I've seen this design talked about on list in the past, and
> I've always felt that this is "sequencer done right".
> 
> At the first glance, it may feel somewhat unsatisfying that "merge"
> has to say effects of which commits should be reflected in the
> result and which commot to take the log message from, i.e.
> (recreated)D is merged to form the resulting tree, and 3456=C is
> used for the log, to recreate C in the above example, while "pick"
> always uses the same commit for both, i.e. recreated B inherits both
> the changes and log message from the original B=2345 (or depending
> on the readers' point of view, "merge" is allowed to use two
> different commits, while "pick" is always limited to the same one).
> 
> But I think this distinction is probably fundamental and I am not
> opposed to it at all.  The result of "pick" has only one parent, and
> the parent is determined only by the previous actions and not by
> anything on the "pick" line in the todo list.  But the result of
> "merge" has to record all the other parents, and only the first
> parent is determined implicitly by the previous actions.  We need to
> tell the "merge" command about "3456=C" in order to recreate the
> effect of original merge commit (i.e. changes between B and C) as
> well as its log message, and we also need to tell it about label "D"
> that it is the "other parent" that need to be recorded.

Yes, this was the hard lesson of the failed preserve-merges design.

> Obviously "merge" command syntax should allow recreating an octopus,
> so whenever I said "two" in the above, I meant "N".  The original
> merge commit is needed so that the effect to replay (roughly: a
> patch going to the original merge result from its first parent) can
> be learned from the existing history, and all the other "N-1"
> parents needs to be given (and they must have been already created
> in the todo list) so that the resulting recreated merge can be
> recorded with them as parents (in addition to the first parent that
> is implicitly given as the result of all the previous steps).

I have two more patch series lined up after this one, the first one
implements --root via the sequencer, and the second one indeed extends
`merge` to handle octopus commits.

> One interesting (and probably useful) thing to notice is that if A
> were not rebased in the above sample picture, and only B were the
> one that was tweaked, then a recreated C may use the same original D
> as its side parent, and the mechanism outlined above naturally can
> support it by allowing an un-rewritten commit to be given as a side
> parent when "merge" is redoing C.

I think that you will get a kick out of reading the commit message of the
last commit, as it does talk about the problematic C: it *would* be
rebased by default.

In the next iteration I will actually switch around the default from
rebase-cousins to no-rebase-cousins for that reason.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-22 21:25     ` Junio C Hamano
@ 2018-01-29 22:00       ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:00 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Jacob Keller, Git mailing list

Hi Junio,

On Mon, 22 Jan 2018, Junio C Hamano wrote:

> Jacob Keller <jacob.keller@gmail.com> writes:
> 
> > The code looks good, but I'm a little wary of adding bud which
> > hard-codes a specific label. I suppose it does grant a bit of
> > readability to the resulting script... ? It doesn't seem that
> > important compared to use using "reset onto"? At least when
> > documenting this it should be made clear that the "onto" label is
> > special.
> 
> I do not think we would mind "bud" too much in the end result, but
> the change in 1/8 is made harder to read than necessary with it.  It
> is the only thing that needs "a single-letter command name may now
> not have any argument after it" change to the parser among the three
> things being added here, and it also needs to be added to the list
> of special commands without arguments.
> 
> It would have been easier to reason about if addition of "bud" was
> in its own patch done after label and reset are added.  And if done
> as a separate step, perhaps it would have been easier to realize
> that it would be a more future-proof solution for handling the
> "special" ness of BUD to add a new "unsigned flags" word to
> todo_command_info[] structure and using a bit that says "this does
> not take an arg" than to hardcode "noop and bud are the commands
> without args" in the code.  That hardcode was good enough when there
> was only one thing in that special case.  Now it has two.

I dropped the `bud` command. It did come in handy when I truly recreated
branch structure from a way-too-long topic branch, but that is probably a
rare use case, and not worth spending so much air time on.

> In a similar way, the code to special case label and reset just like
> exec may also want to become more table driven, perhaps using
> another bit in the same new flags word to say "this does not refer
> to commit".  I think that can become [v2 1/N] while addition of "bud"
> can be [v2 2/N] (after all, "bud" just does the same do_reset() with
> hardcoded argument, so "label/reset" must come first).

The downside of such a table-driven approach is readability, of course. It
becomes *less* readable because all of a sudden you have to jump back and
forth between the parsing code and the table (and then you also have to
keep the table header in sight).

I have had enough problems with such table-driven approaches in the past,
even in Git's own source code. Exhibit A: t0027. I do not wish upon my
worst enemies having to investigate problems in that script, for in those
tables despair awaits ye who dare enter.

So I respectfully decline to go into that direction in the sequencer.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/8] sequencer: introduce the `merge` command
  2018-01-22 22:12   ` Junio C Hamano
@ 2018-01-29 22:15     ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:15 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Jacob Keller

Hi Junio,

On Mon, 22 Jan 2018, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> >  	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
> > +	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
> > +	item->arg_len = (int)(eol - item->arg);
> > +
> >  	saved = *end_of_object_name;
> > +	if (item->command == TODO_MERGE && *bol == '-' &&
> > +	    bol + 1 == end_of_object_name) {
> > +		item->commit = NULL;
> > +		return 0;
> > +	}
> > +
> >  	*end_of_object_name = '\0';
> >  	status = get_oid(bol, &commit_oid);
> >  	*end_of_object_name = saved;
> >  
> > -	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
> > -	item->arg_len = (int)(eol - item->arg);
> > -
> 
> Assigning to "saved" before the added "if we are doing merge and see
> '-', do this special thing" is not only unnecessary, but makes the
> logic in the non-special case harder to read.  The four things
> "saved = *eol; *eol = 0; do_thing_using(bol); *eol = saved;" is a
> single logical unit; keep them together.

True. This was a sloppily resolved merge conflict in one of the many
rewrites, I guess.

> > +		if (*p)
> > +			len = strlen(p);
> > +		else {
> > +			strbuf_addf(&buf, "Merge branch '%.*s'",
> > +				    merge_arg_len, arg);
> > +			p = buf.buf;
> > +			len = buf.len;
> > +		}
> 
> So... "arg" received by this function can be a single non-whitespace
> token, which is taken as the name of the branch being merged (in
> this else clause).  Or it can also be followed by a single liner
> message for the merge commit.  Presumably, this is for creating a
> new merge (i.e. "commit==NULL" case), and preparing a proper log
> message in the todo list is unrealistic, so this would be a
> reasonable compromise.  Those users who want to write proper log
> message could presumably follow such "merge" insn with a "x git
> commit --amend" or something, I presume, if they really wanted to.

Precisely.

> > +		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
> > +			error_errno(_("Could not write '%s'"),
> > +				    git_path_merge_msg());
> > +			strbuf_release(&buf);
> > +			rollback_lock_file(&lock);
> > +			return -1;
> > +		}
> > +		strbuf_release(&buf);
> > +	}
> 
> OK.  Now we have prepared the MERGE_MSG file and are ready to commit.
> 
> > +	head_commit = lookup_commit_reference_by_name("HEAD");
> > +	if (!head_commit) {
> > +		rollback_lock_file(&lock);
> > +		return error(_("Cannot merge without a current revision"));
> > +	}
> 
> Hmph, I would have expected to see this a lot earlier, before
> dealing with the log message.  Leftover MERGE_MSG file after an
> error will cause unexpected fallout to the end-user experience
> (including what is shown by the shell prompt scripts), but if we do
> this before the MERGE_MSG thing, we do not have to worry about
> error codepath having to remove it.

Fixed.

> > +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
> > +	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> > +	if (!merge_commit) {
> > +		/* fall back to non-rewritten ref or commit */
> > +		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
> > +		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> > +	}
> 
> OK, so "abc" in the example in the log message is looked up first as
> a label and then we take a fallback to interpret as an object name.

Yes. And auto-generated labels are guaranteed not to be full hex hashes
for that reason.

> Hopefully allowed names in "label" would be documented clearly in
> later steps (I am guessing that "a name that can be used as a branch
> name can be used as a label name and vice versa" or something like
> that).

Well, I thought that it would suffice to say that these labels are
available as refs/rewritten/<label>. It kind of goes without saying that
those need to be valid ref names, then?

> > +	if (!merge_commit) {
> > +		error(_("could not resolve '%s'"), ref_name.buf);
> > +		strbuf_release(&ref_name);
> > +		rollback_lock_file(&lock);
> > +		return -1;
> > +	}
> > +	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
> > +		      git_path_merge_head(), 0);
> > +	write_message("no-ff", 5, git_path_merge_mode(), 0);
> 
> These two calls gave me a "Huh?" moment; write_message() sounds like
> it is allowed to be later updated with frills suitable for *_MSG
> files we place in .git/ directory (iow, it is in principle OK if
> commented out instructions common to these files are added to the
> output by the function), but these want exact bytes passed in the
> result, for which wrapper.c::write_file() is more appropriate.

I agree that write_message() is not a good name, but write_file() was
already taken. I do not think wrapper.c:write_file() is more appropriate,
as it has *also* misnamed: it *always completes the line*. In other words,
it is not write_file(), it is write_line_to_file(). And not even that, as
it takes a printf()-type format.

No, write_message(), even if not named appropriately, is the functionality
I want.

> Alternatively, perhaps write_message() can be dropped and its
> callers can call wrapper.c::write_file() instead?  Such a clean-up
> may require teaching the append-eol thing that write_message() wants
> to wrapper.c::write_file(), but it shouldn't be a rocket science.

write_file() does different things than write_message(). I think this
suggestion to congeal them into one is quite overzealous.

And would also lead us astray.

> > +	common = get_merge_bases(head_commit, merge_commit);
> > +	for (j = common; j; j = j->next)
> > +		commit_list_insert(j->item, &reversed);
> > +	free_commit_list(common);
> 
> I know this is copy&pasted code from "builtin/merge.c", but is there
> a reason to reverse the common ancestor list here?

Yes. You explained the reason yourself: this is `git merge`'s behavior. To
recreate it in the sequencer, the list of common ancestors has to be
reversed here, too.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 3/8] sequencer: fast-forward merge commits, if possible
  2018-01-23 18:51   ` Junio C Hamano
@ 2018-01-29 22:18     ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:18 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Jacob Keller

Hi Junio,

On Tue, 23 Jan 2018, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > +	/*
> > +	 * If HEAD is not identical to the parent of the original merge commit,
> > +	 * we cannot fast-forward.
> > +	 */
> > +	can_fast_forward = commit && commit->parents &&
> > +		!oidcmp(&commit->parents->item->object.oid,
> > +			&head_commit->object.oid);
> > +
> 
> I think this expression and assignment should better be done much
> later.  Are you going to update commit, commit->parents, etc. that
> are involved in the computation in the meantime???

No, it is in the exact right spot.

What you are missing is that there will be some more code here, to support
octopus merges.

That's why it is exactly the right spot.

> > @@ -2164,6 +2172,17 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
> >  		rollback_lock_file(&lock);
> >  		return -1;
> >  	}
> > +
> > +	if (can_fast_forward && commit->parents->next &&
> > +	    !commit->parents->next->next &&
> > +	    !oidcmp(&commit->parents->next->item->object.oid,
> > +		    &merge_commit->object.oid)) {
> 
> ... Namely, here.

... which the octopus merge code will never reach. That's why this is the
wrong spot for that initial can_fast_forward assignment.

> Because the earlier one is computing "are we
> replaying exactly the same commit on top of exactly the same
> state?", which is merely one half of "can we fast-forward", and
> storing it in a variable whose name is over-promising way before it
> becomes necessary.  The other half of "can we fast-forward?" logic
> is the remainder of the if() condition we see above.  IOW, when
> fully spelled, this code can fast-forward when we are replaying a
> commit on top of exactly the same first-parent and the commit being
> replayed is a single parent merge.
> 
> We may even want to get rid of can_fast_forward variable.

Absolutely not.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges
  2018-01-23 20:03   ` Junio C Hamano
@ 2018-01-29 22:37     ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:37 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Jacob Keller

Hi Junio,

On Tue, 23 Jan 2018, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > structure (similar in spirit to --preserve-merges, but with a
> > substantially less-broken design).
> > ...
> > @@ -2785,6 +2787,335 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
> >  	strbuf_release(&sob);
> >  }
> >  
> > +struct labels_entry {
> > +	struct hashmap_entry entry;
> > +	char label[FLEX_ARRAY];
> > +};
> > +
> > +static int labels_cmp(const void *fndata, const struct labels_entry *a,
> > +		      const struct labels_entry *b, const void *key)
> > +{
> > +	return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
> > +}
> 
> label_oid() accesses state->labels hash using strihash() as the hash
> function, but the final comparison between the entries in the same
> hash buckets are done with case sensitivity.  It is unclear to me if
> that is what was intended, and why.

Heh... you were almost there with your analysis. strihash() is needed for
case-insensitive comparisons, and labels are... ref names.

So the idea (which I implemented only partially) was to make the labels
case-insensitive based on `ignore_case`.

I think the best way forward will be to copy
merge-recursive.c:path_hash().

> > +struct string_entry {
> > +	struct oidmap_entry entry;
> > +	char string[FLEX_ARRAY];
> > +};
> > +
> > +struct label_state {
> > +	struct oidmap commit2label;
> > +	struct hashmap labels;
> > +	struct strbuf buf;
> > +};
> > +
> > +static const char *label_oid(struct object_id *oid, const char *label,
> > +			     struct label_state *state)
> > +{
> > +	struct labels_entry *labels_entry;
> > +	struct string_entry *string_entry;
> > +	struct object_id dummy;
> > +	size_t len;
> > +	int i;
> > +
> > +	string_entry = oidmap_get(&state->commit2label, oid);
> > +	if (string_entry)
> > +		return string_entry->string;
> > +
> > +	/*
> > +	 * For "uninteresting" commits, i.e. commits that are not to be
> > +	 * rebased, and which can therefore not be labeled, we use a unique
> > +	 * abbreviation of the commit name. This is slightly more complicated
> > +	 * than calling find_unique_abbrev() because we also need to make
> > +	 * sure that the abbreviation does not conflict with any other
> > +	 * label.
> > +	 *
> > +	 * We disallow "interesting" commits to be labeled by a string that
> > +	 * is a valid full-length hash, to ensure that we always can find an
> > +	 * abbreviation for any uninteresting commit's names that does not
> > +	 * clash with any other label.
> > +	 */
> > +	if (!label) {
> > +		char *p;
> > +
> > +		strbuf_reset(&state->buf);
> > +		strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
> > +		label = p = state->buf.buf;
> > +
> > +		find_unique_abbrev_r(p, oid->hash, default_abbrev);
> > +
> > +		/*
> > +		 * We may need to extend the abbreviated hash so that there is
> > +		 * no conflicting label.
> > +		 */
> > +		if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
> > +			size_t i = strlen(p) + 1;
> > +
> > +			oid_to_hex_r(p, oid);
> > +			for (; i < GIT_SHA1_HEXSZ; i++) {
> > +				char save = p[i];
> > +				p[i] = '\0';
> > +				if (!hashmap_get_from_hash(&state->labels,
> > +							   strihash(p), p))
> > +					break;
> > +				p[i] = save;
> > +			}
> > +		}
> 
> If oid->hash required full 40-hex to disambiguate, then
> find-unique-abbrev would give 40-hex and we'd want the same "-<num>"
> suffix technique employed below to make it consistently unique.  I
> wonder if organizing the function this way ...
> 
> 	if (!label)
> 		label = oid-to-hex(oid);
> 
> 	if (label already exists or full oid) {
> 		make it unambiguous;
> 	}

It was hard to miss, I agree. The first arm of the if() is not trying to
make a label unambiguous by adding `-<num>`, but by extending the
abbreviated 40-hex until it is unique. (And we guarantee that there is a
solution, as we do not allow valid 40-hex strings to be used as label.)

The reason: if no `label` is provided, we want a valid raw (possibly
abbreviated) commit name. If `label` is provided, we want a valid ref name
(possibly made unique by appending `-<num>`).

Different beasts. Cannot be put into the same if() arm.

> A related tangent.  Does an auto-label given to "uninteresting"
> commit need to be visible to end users?

Yes. The reason for this is the `merge`/`reset` commands when *not*
rebasing cousins. Because in that case, we have to refer to commits that
we could not possibly have `label`ed, because they were never the current
revision. Therefore we cannot assign <name>-<num> labels, but have to use
the abbreviated commit hash.

> > +static int make_script_with_merges(struct pretty_print_context *pp,
> > +				   struct rev_info *revs, FILE *out,
> > +				   unsigned flags)
> > +{
> > + ...
> > +	int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
> > +	const char *p = abbr ? "p" : "pick", *l = abbr ? "l" : "label",
> > +		 *t = abbr ? "t" : "reset", *b = abbr ? "b" : "bud",
> > +		 *m = abbr ? "m" : "merge";
> 
> It would be easier to understand if these short named variables are
> reserved only for temporary use, not as constants.  It is not too
> much to spell 
> 
> 	fprintf(out, "%s onto\n", cmd_label);
> 
> than
> 
> 	fprintf(out, "%s onto\n", l);
> 
> and would save readers from head-scratching, wondering where the
> last assignment to variable "l" is.

Sure.

> > +	oidmap_init(&commit2todo, 0);
> > +	oidmap_init(&state.commit2label, 0);
> > +	hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
> > +	strbuf_init(&state.buf, 32);
> > +
> > +	if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
> > +		struct object_id *oid = &revs->cmdline.rev[0].item->oid;
> > +		FLEX_ALLOC_STR(entry, string, "onto");
> > +		oidcpy(&entry->entry.oid, oid);
> > +		oidmap_put(&state.commit2label, entry);
> > +	}
> > +
> > +	/*
> > +	 * First phase:
> > +	 * - get onelines for all commits
> > +	 * - gather all branch tips (i.e. 2nd or later parents of merges)
> > +	 * - label all branch tips
> > +	 */
> 
> When an early part of a branch is merged and then the remaining part
> of the same branch is merged again, "branch tip" and "2nd or later
> parents of merges" would become different concepts.  The 2nd parent
> of an early merge is not among the branch tips.
> 
> For the purpose of the "recreate the topology" algorithm, I am
> imagining that you would need not just the tips but all the 2nd and
> subsequent parents of merges, and my quick skimming tells me that
> the following code grabs them correctly.

Yes. I have used an earlier iteration of this patch series to create the
merging-rebase of Git for Windows v2.16.0 and that has a couple of really
good corner cases to exercise this code.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 0/8] rebase -i: offer to recreate merge commits
  2018-01-23 20:29 ` Junio C Hamano
@ 2018-01-29 22:53   ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:53 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Jacob Keller

Hi Junio,

On Tue, 23 Jan 2018, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > My original attempt was --preserve-merges, but that design was so
> > limited that I did not even enable it in interactive mode.
> > ...
> > There are more patches in the pipeline, based on this patch series, but
> > left for later in the interest of reviewable patch series: one mini
> > series to use the sequencer even for `git rebase -i --root`, and another
> > one to add support for octopus merges to --recreate-merges.
> 
> I left comments on a handful of them, but I do not think any of them
> spotted a grave design issue to be a show stopper.  Overall, the
> series was quite a pleasant read, even with those minor nits and
> rooms for improvements.

I objected to a couple obviously problematic suggestions, and I
implemented all others (even those to which I did not respond
specifically).

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v2 00/10] rebase -i: offer to recreate merge commits
  2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
                   ` (11 preceding siblings ...)
  2018-01-23 20:29 ` Junio C Hamano
@ 2018-01-29 22:54 ` Johannes Schindelin
  2018-01-29 22:54   ` [PATCH v2 01/10] git-rebase--interactive: clarify arguments Johannes Schindelin
                     ` (12 more replies)
  12 siblings, 13 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:54 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Once upon a time, I dreamt of an interactive rebase that would not
flatten branch structure, but instead recreate the commit topology
faithfully.

My original attempt was --preserve-merges, but that design was so
limited that I did not even enable it in interactive mode.

Subsequently, it *was* enabled in interactive mode, with the predictable
consequences: as the --preserve-merges design does not allow for
specifying the parents of merge commits explicitly, all the new commits'
parents are defined *implicitly* by the previous commit history, and
hence it is *not possible to even reorder commits*.

This design flaw cannot be fixed. Not without a complete re-design, at
least. This patch series offers such a re-design.

Think of --recreate-merges as "--preserve-merges done right". It
introduces new verbs for the todo list, `label`, `reset` and `merge`.
For a commit topology like this:

            A - B - C
              \   /
                D

the generated todo list would look like this:

            # branch D
            pick 0123 A
            label branch-point
            pick 1234 D
            label D

            reset branch-point
            pick 2345 B
            merge 3456 D C

There are more patches in the pipeline, based on this patch series, but
left for later in the interest of reviewable patch series: one mini
series to use the sequencer even for `git rebase -i --root`, and another
one to add support for octopus merges to --recreate-merges.

Changes since v1:

- reintroduced "sequencer: make refs generated by the `label` command
  worktree-local" (which was squashed into "sequencer: handle autosquash
  and post-rewrite for merge commands" by accident)

- got rid of the universally-hated `bud` command

- as per Stefan's suggestion, the help blurb at the end of the todo list
  now lists the syntax

- the no-rebase-cousins mode was made the default; This not only reflects
  the experience won from those years of using the Git garden shears, but
  was also deemed the better default in the discussion on the PR at
  https://github.com/git/git/pull/447

- I tried to clarify the role of the `onto` label in the commit message of
  `rebase-helper --make-script: introduce a flag to recreate merges`

- fixed punctuation at the end of error(...) messages, and incorrect
  upper-case at the start

- changed the generated todo lists to separate the label and the oneline in
  the `reset` command with a `#`, for readability

- dropped redundant paragraph in the commit message that talked about
  support for octopus merges

- avoided empty error message when HEAD could not be read during do_label()

- merge commits are fast-forwarded only unless --force-rebase was passed

- do_merge() now errors out a lot earlier when HEAD could not be parsed

- the one-letter variables to hold either abbreviated or full todo list
  instructions in make_script_recreating_merges() were renamed to clearer
  names

- The description of rebase's --recreate-merge option has been reworded;
  Hopefully it is a lot more clear now.


Johannes Schindelin (9):
  sequencer: introduce new commands to reset the revision
  sequencer: introduce the `merge` command
  sequencer: fast-forward merge commits, if possible
  rebase-helper --make-script: introduce a flag to recreate merges
  rebase: introduce the --recreate-merges option
  sequencer: make refs generated by the `label` command worktree-local
  sequencer: handle autosquash and post-rewrite for merge commands
  pull: accept --rebase=recreate to recreate the branch topology
  rebase -i: introduce --recreate-merges=[no-]rebase-cousins

Stefan Beller (1):
  git-rebase--interactive: clarify arguments

 Documentation/config.txt               |   8 +
 Documentation/git-pull.txt             |   5 +-
 Documentation/git-rebase.txt           |  14 +-
 builtin/pull.c                         |  14 +-
 builtin/rebase--helper.c               |  13 +-
 builtin/remote.c                       |   2 +
 contrib/completion/git-completion.bash |   4 +-
 git-rebase--interactive.sh             |  22 +-
 git-rebase.sh                          |  16 +
 refs.c                                 |   3 +-
 sequencer.c                            | 699 ++++++++++++++++++++++++++++++++-
 sequencer.h                            |   7 +
 t/t3430-rebase-recreate-merges.sh      | 208 ++++++++++
 13 files changed, 988 insertions(+), 27 deletions(-)
 create mode 100755 t/t3430-rebase-recreate-merges.sh


base-commit: 5be1f00a9a701532232f57958efab4be8c959a29
Published-As: https://github.com/dscho/git/releases/tag/recreate-merges-v2
Fetch-It-Via: git fetch https://github.com/dscho/git recreate-merges-v2

Interdiff vs v1:
 diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
 index ac07a5c3fc9..0e6d020d924 100644
 --- a/Documentation/git-rebase.txt
 +++ b/Documentation/git-rebase.txt
 @@ -371,12 +371,13 @@ have the long commit hash prepended to the format.
  --recreate-merges[=(rebase-cousins|no-rebase-cousins)]::
  	Recreate merge commits instead of flattening the history by replaying
  	merges. Merge conflict resolutions or manual amendments to merge
 -	commits are not preserved.
 +	commits are not recreated automatically, but have to be recreated
 +	manually.
  +
 -By default, or when `rebase-cousins` was specified, commits which do not have
 -`<upstream>` as direct ancestor are rebased onto `<upstream>` (or `<onto>`,
 -if specified). If the `rebase-cousins` mode is turned off, such commits will
 -retain their original branch point.
 +By default, or when `no-rebase-cousins` was specified, commits which do not
 +have `<upstream>` as direct ancestor keep their original branch point.
 +If the `rebase-cousins` mode is turned on, such commits are rebased onto
 +`<upstream>` (or `<onto>`, if specified).
  
  -p::
  --preserve-merges::
 diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
 index ef08fef4d14..cea99cb3235 100644
 --- a/builtin/rebase--helper.c
 +++ b/builtin/rebase--helper.c
 @@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
  {
  	struct replay_opts opts = REPLAY_OPTS_INIT;
  	unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
 -	int abbreviate_commands = 0, no_rebase_cousins = -1;
 +	int abbreviate_commands = 0, rebase_cousins = -1;
  	enum {
  		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
  		CHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH,
 @@ -23,7 +23,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
  		OPT_BOOL(0, "ff", &opts.allow_ff, N_("allow fast-forward")),
  		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
  		OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge commits")),
 -		OPT_BOOL(0, "no-rebase-cousins", &no_rebase_cousins,
 +		OPT_BOOL(0, "rebase-cousins", &rebase_cousins,
  			 N_("keep original branch points of cousins")),
  		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
  				CONTINUE),
 @@ -59,10 +59,10 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
  	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
  	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
  	flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
 -	flags |= no_rebase_cousins > 0 ? TODO_LIST_NO_REBASE_COUSINS : 0;
 +	flags |= rebase_cousins > 0 ? TODO_LIST_REBASE_COUSINS : 0;
  	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
  
 -	if (no_rebase_cousins >= 0&& !recreate_merges)
 +	if (rebase_cousins >= 0 && !recreate_merges)
  		warning(_("--[no-]rebase-cousins has no effect without "
  			  "--recreate-merges"));
  
 diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
 index 23184c77e88..5e21e4cf269 100644
 --- a/git-rebase--interactive.sh
 +++ b/git-rebase--interactive.sh
 @@ -155,17 +155,19 @@ reschedule_last_action () {
  append_todo_help () {
  	gettext "
  Commands:
 -p, pick = use commit
 -r, reword = use commit, but edit the commit message
 -e, edit = use commit, but stop for amending
 -s, squash = use commit, but meld into previous commit
 -f, fixup = like \"squash\", but discard this commit's log message
 -x, exec = run command (the rest of the line) using shell
 -d, drop = remove commit
 -l, label = label current HEAD with a name
 -t, reset = reset HEAD to a label
 -b, bud = reset HEAD to the revision labeled 'onto'
 -m, merge = create a merge commit using a given commit's message
 +p, pick <commit> = use commit
 +r, reword <commit> = use commit, but edit the commit message
 +e, edit <commit> = use commit, but stop for amending
 +s, squash <commit> = use commit, but meld into previous commit
 +f, fixup <commit> = like \"squash\", but discard this commit's log message
 +x, exec <commit> = run command (the rest of the line) using shell
 +d, drop <commit> = remove commit
 +l, label <label> = label current HEAD with a name
 +t, reset <label> = reset HEAD to a label
 +m, merge <original-merge-commit> ( <label> | \"<label>...\" ) [<oneline>]
 +.       create a merge commit using the original merge commit's
 +.       message (or the oneline, if "-" is given). Use a quoted
 +.       list of commits to be merged for octopus merges.
  
  These lines can be re-ordered; they are executed from top to bottom.
  " | git stripspace --comment-lines >>"$todo"
 @@ -901,7 +903,7 @@ if test t != "$preserve_merges"
  then
  	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
  		${recreate_merges:+--recreate-merges} \
 -		${no_rebase_cousins:+--no-rebase-cousins} \
 +		${rebase_cousins:+--rebase-cousins} \
  		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
  	die "$(gettext "Could not generate todo list")"
  else
 diff --git a/git-rebase.sh b/git-rebase.sh
 index 3403b1416a8..58d778a2da0 100755
 --- a/git-rebase.sh
 +++ b/git-rebase.sh
 @@ -88,7 +88,7 @@ state_dir=
  # One of {'', continue, skip, abort}, as parsed from command line
  action=
  recreate_merges=
 -no_rebase_cousins=
 +rebase_cousins=
  preserve_merges=
  autosquash=
  keep_empty=
 @@ -272,8 +272,8 @@ do
  	--recreate-merges=*)
  		recreate_merges=t
  		case "${1#*=}" in
 -		rebase-cousins) no_rebase_cousins=;;
 -		no-rebase-cousins) no_rebase_cousins=t;;
 +		rebase-cousins) rebase_cousins=t;;
 +		no-rebase-cousins) rebase_cousins=;;
  		*) die "Unknown mode: $1";;
  		esac
  		test -z "$interactive_rebase" && interactive_rebase=implied
 diff --git a/sequencer.c b/sequencer.c
 index 2b4e6b12321..cd2f2ae5d53 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -780,7 +780,6 @@ enum todo_command {
  	TODO_EXEC,
  	TODO_LABEL,
  	TODO_RESET,
 -	TODO_BUD,
  	TODO_MERGE,
  	/* commands that do nothing but are counted for reporting progress */
  	TODO_NOOP,
 @@ -802,7 +801,6 @@ static struct {
  	{ 'x', "exec" },
  	{ 'l', "label" },
  	{ 't', "reset" },
 -	{ 'b', "bud" },
  	{ 'm', "merge" },
  	{ 0,   "noop" },
  	{ 'd', "drop" },
 @@ -1285,7 +1283,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
  	padding = strspn(bol, " \t");
  	bol += padding;
  
 -	if (item->command == TODO_NOOP || item->command == TODO_BUD) {
 +	if (item->command == TODO_NOOP) {
  		if (bol != eol)
  			return error(_("%s does not accept arguments: '%s'"),
  				     command_to_string(item->command), bol);
 @@ -1311,13 +1309,13 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
  	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
  	item->arg_len = (int)(eol - item->arg);
  
 -	saved = *end_of_object_name;
  	if (item->command == TODO_MERGE && *bol == '-' &&
  	    bol + 1 == end_of_object_name) {
  		item->commit = NULL;
  		return 0;
  	}
  
 +	saved = *end_of_object_name;
  	*end_of_object_name = '\0';
  	status = get_oid(bol, &commit_oid);
  	*end_of_object_name = saved;
 @@ -1969,7 +1967,7 @@ static int safe_append(const char *filename, const char *fmt, ...)
  	}
  	if (commit_lock_file(&lock) < 0) {
  		rollback_lock_file(&lock);
 -		return error(_("failed to finalize '%s'."), filename);
 +		return error(_("failed to finalize '%s'"), filename);
  	}
  
  	return 0;
 @@ -1985,14 +1983,18 @@ static int do_label(const char *name, int len)
  	struct object_id head_oid;
  
  	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
 -	strbuf_addf(&msg, "label '%.*s'", len, name);
 +	strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
  
  	transaction = ref_store_transaction_begin(refs, &err);
 -	if (!transaction ||
 -	    get_oid("HEAD", &head_oid) ||
 -	    ref_transaction_update(transaction, ref_name.buf, &head_oid, NULL,
 -				   0, msg.buf, &err) < 0 ||
 -	    ref_transaction_commit(transaction, &err)) {
 +	if (!transaction) {
 +		error("%s", err.buf);
 +		ret = -1;
 +	} else if (get_oid("HEAD", &head_oid)) {
 +		error(_("could not read HEAD"));
 +		ret = -1;
 +	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
 +					  NULL, 0, msg.buf, &err) < 0 ||
 +		   ref_transaction_commit(transaction, &err)) {
  		error("%s", err.buf);
  		ret = -1;
  	}
 @@ -2021,6 +2023,7 @@ static int do_reset(const char *name, int len)
  	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
  		return -1;
  
 +	/* Determine the length of the label */
  	for (i = 0; i < len; i++)
  		if (isspace(name[i]))
  			len = i;
 @@ -2045,7 +2048,7 @@ static int do_reset(const char *name, int len)
  
  	read_cache_unmerged();
  	if (!fill_tree_descriptor(&desc, &oid)) {
 -		error(_("Failed to find tree of %s."), oid_to_hex(&oid));
 +		error(_("failed to find tree of %s"), oid_to_hex(&oid));
  		rollback_lock_file(&lock);
  		free((void *)desc.buffer);
  		strbuf_release(&ref_name);
 @@ -2097,6 +2100,12 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
  		return -1;
  
 +	head_commit = lookup_commit_reference_by_name("HEAD");
 +	if (!head_commit) {
 +		rollback_lock_file(&lock);
 +		return error(_("cannot merge without a current revision"));
 +	}
 +
  	if (commit) {
  		const char *message = get_commit_buffer(commit, NULL);
  		const char *body;
 @@ -2111,7 +2120,7 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  		find_commit_subject(message, &body);
  		len = strlen(body);
  		if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
 -			error_errno(_("Could not write '%s'"),
 +			error_errno(_("could not write '%s'"),
  				    git_path_merge_msg());
  			unuse_commit_buffer(commit, message);
  			rollback_lock_file(&lock);
 @@ -2138,7 +2147,7 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  		}
  
  		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
 -			error_errno(_("Could not write '%s'"),
 +			error_errno(_("could not write '%s'"),
  				    git_path_merge_msg());
  			strbuf_release(&buf);
  			rollback_lock_file(&lock);
 @@ -2147,17 +2156,11 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  		strbuf_release(&buf);
  	}
  
 -	head_commit = lookup_commit_reference_by_name("HEAD");
 -	if (!head_commit) {
 -		rollback_lock_file(&lock);
 -		return error(_("Cannot merge without a current revision"));
 -	}
 -
  	/*
  	 * If HEAD is not identical to the parent of the original merge commit,
  	 * we cannot fast-forward.
  	 */
 -	can_fast_forward = commit && commit->parents &&
 +	can_fast_forward = opts->allow_ff && commit && commit->parents &&
  		!oidcmp(&commit->parents->item->object.oid,
  			&head_commit->object.oid);
  
 @@ -2411,8 +2414,6 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			res = do_label(item->arg, item->arg_len);
  		else if (item->command == TODO_RESET)
  			res = do_reset(item->arg, item->arg_len);
 -		else if (item->command == TODO_BUD)
 -			res = do_reset("onto", 4);
  		else if (item->command == TODO_MERGE) {
  			res = do_merge(item->commit,
  				       item->arg, item->arg_len, opts);
 @@ -2905,7 +2906,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
  				   unsigned flags)
  {
  	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
 -	int no_rebase_cousins = flags & TODO_LIST_NO_REBASE_COUSINS;
 +	int rebase_cousins = flags & TODO_LIST_REBASE_COUSINS;
  	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
  	struct strbuf label = STRBUF_INIT;
  	struct commit_list *commits = NULL, **tail = &commits, *iter;
 @@ -2918,9 +2919,10 @@ static int make_script_with_merges(struct pretty_print_context *pp,
  	struct label_state state = { OIDMAP_INIT, { NULL }, STRBUF_INIT };
  
  	int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
 -	const char *p = abbr ? "p" : "pick", *l = abbr ? "l" : "label",
 -		 *t = abbr ? "t" : "reset", *b = abbr ? "b" : "bud",
 -		 *m = abbr ? "m" : "merge";
 +	const char *cmd_pick = abbr ? "p" : "pick",
 +		*cmd_label = abbr ? "l" : "label",
 +		*cmd_reset = abbr ? "t" : "reset",
 +		*cmd_merge = abbr ? "m" : "merge";
  
  	oidmap_init(&commit2todo, 0);
  	oidmap_init(&state.commit2label, 0);
 @@ -2961,7 +2963,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
  			strbuf_reset(&buf);
  			if (!keep_empty && is_original_commit_empty(commit))
  				strbuf_addf(&buf, "%c ", comment_line_char);
 -			strbuf_addf(&buf, "%s %s %s", p,
 +			strbuf_addf(&buf, "%s %s %s", cmd_pick,
  				    oid_to_hex(&commit->object.oid),
  				    oneline.buf);
  
 @@ -2995,7 +2997,8 @@ static int make_script_with_merges(struct pretty_print_context *pp,
  				*(char *)p1 = '-';
  
  		strbuf_reset(&buf);
 -		strbuf_addf(&buf, "%s %s", m, oid_to_hex(&commit->object.oid));
 +		strbuf_addf(&buf, "%s %s",
 +			    cmd_merge, oid_to_hex(&commit->object.oid));
  
  		/* label the tip of merged branch */
  		oid = &to_merge->item->object.oid;
 @@ -3046,7 +3049,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
  	 * gathering commits not yet shown, reversing the list on the fly,
  	 * then outputting that list (labeling revisions as needed).
  	 */
 -	fprintf(out, "%s onto\n", l);
 +	fprintf(out, "%s onto\n", cmd_label);
  	for (iter = tips; iter; iter = iter->next) {
  		struct commit_list *list = NULL, *iter2;
  
 @@ -3071,7 +3074,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
  		}
  
  		if (!commit)
 -			fprintf(out, "%s\n", b);
 +			fprintf(out, "%s onto\n", cmd_reset);
  		else {
  			const char *to = NULL;
  
 @@ -3079,17 +3082,17 @@ static int make_script_with_merges(struct pretty_print_context *pp,
  					   &commit->object.oid);
  			if (entry)
  				to = entry->string;
 -			else if (no_rebase_cousins)
 +			else if (!rebase_cousins)
  				to = label_oid(&commit->object.oid, NULL,
  					       &state);
  
 -			if (!to || !strcmp("onto", to))
 -				fprintf(out, "%s\n", b);
 +			if (!to || !strcmp(to, "onto"))
 +				fprintf(out, "%s onto\n", cmd_reset);
  			else {
  				strbuf_reset(&oneline);
  				pretty_print_commit(pp, commit, &oneline);
 -				fprintf(out, "%s %s %s\n",
 -					t, to, oneline.buf);
 +				fprintf(out, "%s %s # %s\n",
 +					cmd_reset, to, oneline.buf);
  			}
  		}
  
 @@ -3101,7 +3104,8 @@ static int make_script_with_merges(struct pretty_print_context *pp,
  				fprintf(out, "%s\n", entry->string);
  			entry = oidmap_get(&state.commit2label, oid);
  			if (entry)
 -				fprintf(out, "%s %s\n", l, entry->string);
 +				fprintf(out, "%s %s\n",
 +					cmd_label, entry->string);
  			oidset_insert(&shown, oid);
  		}
  
 diff --git a/sequencer.h b/sequencer.h
 index 9530dba3cba..deebc6e3258 100644
 --- a/sequencer.h
 +++ b/sequencer.h
 @@ -51,12 +51,10 @@ int sequencer_remove_state(struct replay_opts *opts);
  #define TODO_LIST_RECREATE_MERGES (1U << 3)
  /*
   * When recreating merges, commits that do have the base commit as ancestor
 - * ("cousins") are rebased onto the new base by default. If those commits
 - * should keep their original branch point, this flag needs to be passed.
 - *
 - * This flag only makes sense when <base> and <onto> are different.
 + * ("cousins") are *not* rebased onto the new base by default. If those
 + * commits should be rebased onto the new base, this flag needs to be passed.
   */
 -#define TODO_LIST_NO_REBASE_COUSINS (1U << 4)
 +#define TODO_LIST_REBASE_COUSINS (1U << 4)
  int sequencer_make_script(FILE *out, int argc, const char **argv,
  			  unsigned flags);
  
 diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
 index 22930e470a4..ab51b584ff9 100755
 --- a/t/t3430-rebase-recreate-merges.sh
 +++ b/t/t3430-rebase-recreate-merges.sh
 @@ -54,11 +54,11 @@ pick D
  label onebranch
  
  # second
 -bud
 +reset onto
  pick B
  label second
  
 -bud
 +reset onto
  merge H second
  merge - onebranch Merge the topic branch 'onebranch'
  EOF
 @@ -93,18 +93,18 @@ test_expect_success 'generate correct todo list' '
  	cat >expect <<-\EOF &&
  	label onto
  
 -	bud
 +	reset onto
  	pick d9df450 B
  	label E
  
 -	bud
 +	reset onto
  	pick 5dee784 C
  	label branch-point
  	pick ca2c861 F
  	pick 088b00a G
  	label H
  
 -	reset branch-point C
 +	reset branch-point # C
  	pick 12bd07b D
  	merge 2051b56 E E
  	merge 233d48a H H
 @@ -143,7 +143,7 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
  	EOF
  '
  
 -test_expect_success 'rebase cousins unless told not to' '
 +test_expect_success 'do not rebase cousins unless asked for' '
  	write_script copy-editor.sh <<-\EOF &&
  	cp "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
  	EOF
 @@ -152,10 +152,10 @@ test_expect_success 'rebase cousins unless told not to' '
  	git checkout -b cousins master &&
  	before="$(git rev-parse --verify HEAD)" &&
  	test_tick &&
 -	git rebase -i --recreate-merges=no-rebase-cousins HEAD^ &&
 +	git rebase -i --recreate-merges HEAD^ &&
  	test_cmp_rev HEAD $before &&
  	test_tick &&
 -	git rebase -i --recreate-merges HEAD^ &&
 +	git rebase -i --recreate-merges=rebase-cousins HEAD^ &&
  	test_cmp_graph HEAD^.. <<-\EOF
  	*   Merge the topic branch '\''onebranch'\''
  	|\
-- 
2.16.1.windows.1


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v2 01/10] git-rebase--interactive: clarify arguments
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
@ 2018-01-29 22:54   ` Johannes Schindelin
  2018-01-29 22:54   ` [PATCH v2 02/10] sequencer: introduce new commands to reset the revision Johannes Schindelin
                     ` (11 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:54 UTC (permalink / raw)
  To: git
  Cc: Stefan Beller, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood

From: Stefan Beller <stefanbeller@gmail.com>

Up to now each command took a commit as its first argument and ignored
the rest of the line (usually the subject of the commit)

Now that we are about to introduce commands that take different
arguments, clarify each command by giving the argument list.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index d47bd29593a..fcedece1860 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -155,13 +155,13 @@ reschedule_last_action () {
 append_todo_help () {
 	gettext "
 Commands:
-p, pick = use commit
-r, reword = use commit, but edit the commit message
-e, edit = use commit, but stop for amending
-s, squash = use commit, but meld into previous commit
-f, fixup = like \"squash\", but discard this commit's log message
-x, exec = run command (the rest of the line) using shell
-d, drop = remove commit
+p, pick <commit> = use commit
+r, reword <commit> = use commit, but edit the commit message
+e, edit <commit> = use commit, but stop for amending
+s, squash <commit> = use commit, but meld into previous commit
+f, fixup <commit> = like \"squash\", but discard this commit's log message
+x, exec <commit> = run command (the rest of the line) using shell
+d, drop <commit> = remove commit
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 02/10] sequencer: introduce new commands to reset the revision
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
  2018-01-29 22:54   ` [PATCH v2 01/10] git-rebase--interactive: clarify arguments Johannes Schindelin
@ 2018-01-29 22:54   ` Johannes Schindelin
  2018-01-30  8:06     ` Eric Sunshine
  2018-01-30 20:17     ` Stefan Beller
  2018-01-29 22:54   ` [PATCH v2 03/10] sequencer: introduce the `merge` command Johannes Schindelin
                     ` (10 subsequent siblings)
  12 siblings, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:54 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

In the upcoming commits, we will teach the sequencer to recreate merges.
This will be done in a very different way from the unfortunate design of
`git rebase --preserve-merges` (which does not allow for reordering
commits, or changing the branch topology).

The main idea is to introduce new todo list commands, to support
labeling the current revision with a given name, resetting the current
revision to a previous state, merging labeled revisions.

This idea was developed in Git for Windows' Git garden shears (that are
used to maintain the "thicket of branches" on top of upstream Git), and
this patch is part of the effort to make it available to a wider
audience, as well as to make the entire process more robust (by
implementing it in a safe and portable language rather than a Unix shell
script).

This commit implements the commands to label, and to reset to, given
revisions. The syntax is:

	label <name>
	reset <name>

Internally, the `label <name>` command creates the ref
`refs/rewritten/<name>`. This makes it possible to work with the labeled
revisions interactively, or in a scripted fashion (e.g. via the todo
list command `exec`).

Later in this patch series, we will mark the `refs/rewritten/` refs as
worktree-local, to allow for interactive rebases to be run in parallel in
worktrees linked to the same repository.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   2 +
 sequencer.c                | 180 ++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 179 insertions(+), 3 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index fcedece1860..7e5281e74aa 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -162,6 +162,8 @@ s, squash <commit> = use commit, but meld into previous commit
 f, fixup <commit> = like \"squash\", but discard this commit's log message
 x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
+l, label <label> = label current HEAD with a name
+t, reset <label> = reset HEAD to a label
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index 4d3f60594cb..92ca8d2adee 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -21,6 +21,8 @@
 #include "log-tree.h"
 #include "wt-status.h"
 #include "hashmap.h"
+#include "unpack-trees.h"
+#include "worktree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -116,6 +118,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
 static GIT_PATH_FUNC(rebase_path_rewritten_pending,
 	"rebase-merge/rewritten-pending")
+
+/*
+ * The path of the file listing refs that need to be deleted after the rebase
+ * finishes. This is used by the `merge` command.
+ */
+static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
+
 /*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
@@ -767,6 +776,8 @@ enum todo_command {
 	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
+	TODO_LABEL,
+	TODO_RESET,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -785,6 +796,8 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
+	{ 'l', "label" },
+	{ 't', "reset" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1253,7 +1266,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
 			item->command = i;
 			break;
-		} else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
+		} else if ((bol + 1 == eol || bol[1] == ' ') &&
+			   *bol == todo_command_info[i].c) {
 			bol++;
 			item->command = i;
 			break;
@@ -1279,7 +1293,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return error(_("missing arguments for %s"),
 			     command_to_string(item->command));
 
-	if (item->command == TODO_EXEC) {
+	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
+	    item->command == TODO_RESET) {
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = (int)(eol - bol);
@@ -1919,6 +1934,144 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int safe_append(const char *filename, const char *fmt, ...)
+{
+	va_list ap;
+	struct lock_file lock = LOCK_INIT;
+	int fd = hold_lock_file_for_update(&lock, filename, 0);
+	struct strbuf buf = STRBUF_INIT;
+
+	if (fd < 0)
+		return error_errno(_("could not lock '%s'"), filename);
+
+	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT)
+		return error_errno(_("could not read '%s'"), filename);
+	strbuf_complete(&buf, '\n');
+	va_start(ap, fmt);
+	strbuf_vaddf(&buf, fmt, ap);
+	va_end(ap);
+
+	if (write_in_full(fd, buf.buf, buf.len) < 0) {
+		rollback_lock_file(&lock);
+		return error_errno(_("could not write to '%s'"), filename);
+	}
+	if (commit_lock_file(&lock) < 0) {
+		rollback_lock_file(&lock);
+		return error(_("failed to finalize '%s'"), filename);
+	}
+
+	return 0;
+}
+
+static int do_label(const char *name, int len)
+{
+	struct ref_store *refs = get_main_ref_store();
+	struct ref_transaction *transaction;
+	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
+	struct strbuf msg = STRBUF_INIT;
+	int ret = 0;
+	struct object_id head_oid;
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
+
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction) {
+		error("%s", err.buf);
+		ret = -1;
+	} else if (get_oid("HEAD", &head_oid)) {
+		error(_("could not read HEAD"));
+		ret = -1;
+	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
+					  NULL, 0, msg.buf, &err) < 0 ||
+		   ref_transaction_commit(transaction, &err)) {
+		error("%s", err.buf);
+		ret = -1;
+	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
+	strbuf_release(&msg);
+
+	if (!ret)
+		ret = safe_append(rebase_path_refs_to_delete(),
+				  "%s\n", ref_name.buf);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
+static int do_reset(const char *name, int len)
+{
+	struct strbuf ref_name = STRBUF_INIT;
+	struct object_id oid;
+	struct lock_file lock = LOCK_INIT;
+	struct tree_desc desc;
+	struct tree *tree;
+	struct unpack_trees_options opts;
+	int ret = 0, i;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	/* Determine the length of the label */
+	for (i = 0; i < len; i++)
+		if (isspace(name[i]))
+			len = i;
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	if (get_oid(ref_name.buf, &oid) &&
+	    get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
+		error(_("could not read '%s'"), ref_name.buf);
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	memset(&opts, 0, sizeof(opts));
+	opts.head_idx = 1;
+	opts.src_index = &the_index;
+	opts.dst_index = &the_index;
+	opts.fn = oneway_merge;
+	opts.merge = 1;
+	opts.update = 1;
+	opts.reset = 1;
+
+	read_cache_unmerged();
+	if (!fill_tree_descriptor(&desc, &oid)) {
+		error(_("failed to find tree of %s"), oid_to_hex(&oid));
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	if (unpack_trees(1, &desc, &opts)) {
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	tree = parse_tree_indirect(&oid);
+	prime_cache_tree(&the_index, tree);
+
+	if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
+		ret = error(_("could not write index"));
+	free((void *)desc.buffer);
+
+	if (!ret) {
+		struct strbuf msg = STRBUF_INIT;
+
+		strbuf_addf(&msg, "(rebase -i) reset '%.*s'", len, name);
+		ret = update_ref(msg.buf, "HEAD", &oid, NULL, 0,
+				 UPDATE_REFS_MSG_ON_ERR);
+		strbuf_release(&msg);
+	}
+
+	strbuf_release(&ref_name);
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2102,7 +2255,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				/* `current` will be incremented below */
 				todo_list->current = -1;
 			}
-		} else if (!is_noop(item->command))
+		} else if (item->command == TODO_LABEL)
+			res = do_label(item->arg, item->arg_len);
+		else if (item->command == TODO_RESET)
+			res = do_reset(item->arg, item->arg_len);
+		else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
@@ -2207,6 +2364,23 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		}
 		apply_autostash(opts);
 
+		strbuf_reset(&buf);
+		if (strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0)
+		    > 0) {
+			char *p = buf.buf;
+			while (*p) {
+				char *eol = strchr(p, '\n');
+				if (eol)
+					*eol = '\0';
+				if (delete_ref("(rebase -i) cleanup",
+					       p, NULL, 0) < 0)
+					warning(_("could not delete '%s'"), p);
+				if (!eol)
+					break;
+				p = eol + 1;
+			}
+		}
+
 		fprintf(stderr, "Successfully rebased and updated %s.\n",
 			head_ref.buf);
 
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 03/10] sequencer: introduce the `merge` command
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
  2018-01-29 22:54   ` [PATCH v2 01/10] git-rebase--interactive: clarify arguments Johannes Schindelin
  2018-01-29 22:54   ` [PATCH v2 02/10] sequencer: introduce new commands to reset the revision Johannes Schindelin
@ 2018-01-29 22:54   ` Johannes Schindelin
  2018-01-29 22:54   ` [PATCH v2 04/10] sequencer: fast-forward merge commits, if possible Johannes Schindelin
                     ` (9 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:54 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

This patch is part of the effort to reimplement `--preserve-merges` with
a substantially improved design, a design that has been developed in the
Git for Windows project to maintain the dozens of Windows-specific patch
series on top of upstream Git.

The previous patch implemented the `label` and `reset` commands to label
commits and to reset to a labeled commits. This patch adds the `merge`
command, with the following syntax:

	merge <commit> <rev> <oneline>

The <commit> parameter in this instance is the *original* merge commit,
whose author and message will be used for the to-be-created merge
commit.

The <rev> parameter refers to the (possibly rewritten) revision to
merge. Let's see an example of a todo list:

	label onto

	# Branch abc
	reset onto
	pick deadbeef Hello, world!
	label abc

	reset onto
	pick cafecafe And now for something completely different
	merge baaabaaa abc Merge the branch 'abc' into master

To support creating *new* merges, i.e. without copying the commit
message from an existing commit, use the special value `-` as <commit>
parameter (in which case the text after the <rev> parameter is used as
commit message):

	merge - abc This will be the actual commit message of the merge

This comes in handy when splitting a branch into two or more branches.

Note: this patch only adds support for recursive merges, to keep things
simple. Support for octopus merges will be added later in a separate
patch series, support for merges using strategies other than the
recursive merge is left for the future.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   4 ++
 sequencer.c                | 146 +++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 146 insertions(+), 4 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 7e5281e74aa..d6fd30f6c09 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -164,6 +164,10 @@ x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
 l, label <label> = label current HEAD with a name
 t, reset <label> = reset HEAD to a label
+m, merge <original-merge-commit> ( <label> | \"<label>...\" ) [<oneline>]
+.       create a merge commit using the original merge commit's
+.       message (or the oneline, if "-" is given). Use a quoted
+.       list of commits to be merged for octopus merges.
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index 92ca8d2adee..dfc9f9e13cd 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -778,6 +778,7 @@ enum todo_command {
 	TODO_EXEC,
 	TODO_LABEL,
 	TODO_RESET,
+	TODO_MERGE,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -798,6 +799,7 @@ static struct {
 	{ 'x', "exec" },
 	{ 'l', "label" },
 	{ 't', "reset" },
+	{ 'm', "merge" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1302,14 +1304,20 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	}
 
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
+	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
+	item->arg_len = (int)(eol - item->arg);
+
+	if (item->command == TODO_MERGE && *bol == '-' &&
+	    bol + 1 == end_of_object_name) {
+		item->commit = NULL;
+		return 0;
+	}
+
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
 	status = get_oid(bol, &commit_oid);
 	*end_of_object_name = saved;
 
-	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
-	item->arg_len = (int)(eol - item->arg);
-
 	if (status < 0)
 		return -1;
 
@@ -2072,6 +2080,132 @@ static int do_reset(const char *name, int len)
 	return ret;
 }
 
+static int do_merge(struct commit *commit, const char *arg, int arg_len,
+		    struct replay_opts *opts)
+{
+	int merge_arg_len;
+	struct strbuf ref_name = STRBUF_INIT;
+	struct commit *head_commit, *merge_commit, *i;
+	struct commit_list *common, *j, *reversed = NULL;
+	struct merge_options o;
+	int ret;
+	static struct lock_file lock;
+
+	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
+		if (isspace(arg[merge_arg_len]))
+			break;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	head_commit = lookup_commit_reference_by_name("HEAD");
+	if (!head_commit) {
+		rollback_lock_file(&lock);
+		return error(_("cannot merge without a current revision"));
+	}
+
+	if (commit) {
+		const char *message = get_commit_buffer(commit, NULL);
+		const char *body;
+		int len;
+
+		if (!message) {
+			rollback_lock_file(&lock);
+			return error(_("could not get commit message of '%s'"),
+				     oid_to_hex(&commit->object.oid));
+		}
+		write_author_script(message);
+		find_commit_subject(message, &body);
+		len = strlen(body);
+		if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			unuse_commit_buffer(commit, message);
+			rollback_lock_file(&lock);
+			return -1;
+		}
+		unuse_commit_buffer(commit, message);
+	} else {
+		const char *p = arg + merge_arg_len;
+		struct strbuf buf = STRBUF_INIT;
+		int len;
+
+		strbuf_addf(&buf, "author %s", git_author_info(0));
+		write_author_script(buf.buf);
+		strbuf_reset(&buf);
+
+		p += strspn(p, " \t");
+		if (*p)
+			len = strlen(p);
+		else {
+			strbuf_addf(&buf, "Merge branch '%.*s'",
+				    merge_arg_len, arg);
+			p = buf.buf;
+			len = buf.len;
+		}
+
+		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			strbuf_release(&buf);
+			rollback_lock_file(&lock);
+			return -1;
+		}
+		strbuf_release(&buf);
+	}
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
+	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	if (!merge_commit) {
+		/* fall back to non-rewritten ref or commit */
+		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
+		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	}
+	if (!merge_commit) {
+		error(_("could not resolve '%s'"), ref_name.buf);
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
+		      git_path_merge_head(), 0);
+	write_message("no-ff", 5, git_path_merge_mode(), 0);
+
+	common = get_merge_bases(head_commit, merge_commit);
+	for (j = common; j; j = j->next)
+		commit_list_insert(j->item, &reversed);
+	free_commit_list(common);
+
+	read_cache();
+	init_merge_options(&o);
+	o.branch1 = "HEAD";
+	o.branch2 = ref_name.buf;
+	o.buffer_output = 2;
+
+	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
+	if (ret <= 0)
+		fputs(o.obuf.buf, stdout);
+	strbuf_release(&o.obuf);
+	if (ret < 0) {
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return error(_("conflicts while merging '%.*s'"),
+			     merge_arg_len, arg);
+	}
+
+	if (active_cache_changed &&
+	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
+		strbuf_release(&ref_name);
+		return error(_("merge: Unable to write new index file"));
+	}
+	rollback_lock_file(&lock);
+
+	ret = run_git_commit(git_path_merge_msg(), opts, 0);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2259,6 +2393,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_label(item->arg, item->arg_len);
 		else if (item->command == TODO_RESET)
 			res = do_reset(item->arg, item->arg_len);
+		else if (item->command == TODO_MERGE)
+			res = do_merge(item->commit,
+				       item->arg, item->arg_len, opts);
 		else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
@@ -2758,7 +2895,8 @@ int transform_todos(unsigned flags)
 					  oid_to_hex(&item->commit->object.oid);
 
 			strbuf_addf(&buf, " %s", oid);
-		}
+		} else if (item->command == TODO_MERGE)
+			strbuf_addstr(&buf, " -");
 		/* add all the rest */
 		if (!item->arg_len)
 			strbuf_addch(&buf, '\n');
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 04/10] sequencer: fast-forward merge commits, if possible
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
                     ` (2 preceding siblings ...)
  2018-01-29 22:54   ` [PATCH v2 03/10] sequencer: introduce the `merge` command Johannes Schindelin
@ 2018-01-29 22:54   ` Johannes Schindelin
  2018-01-29 22:54   ` [PATCH v2 05/10] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
                     ` (8 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:54 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Just like with regular `pick` commands, if we are trying to recreate a
merge commit, we now test whether the parents of said commit match HEAD
and the commits to be merged, and fast-forward if possible.

This is not only faster, but also avoids unnecessary proliferation of
new objects.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index dfc9f9e13cd..df61e7883b3 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2088,7 +2088,7 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	struct commit *head_commit, *merge_commit, *i;
 	struct commit_list *common, *j, *reversed = NULL;
 	struct merge_options o;
-	int ret;
+	int can_fast_forward, ret;
 	static struct lock_file lock;
 
 	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
@@ -2154,6 +2154,14 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		strbuf_release(&buf);
 	}
 
+	/*
+	 * If HEAD is not identical to the parent of the original merge commit,
+	 * we cannot fast-forward.
+	 */
+	can_fast_forward = opts->allow_ff && commit && commit->parents &&
+		!oidcmp(&commit->parents->item->object.oid,
+			&head_commit->object.oid);
+
 	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
 	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
 	if (!merge_commit) {
@@ -2167,6 +2175,17 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		rollback_lock_file(&lock);
 		return -1;
 	}
+
+	if (can_fast_forward && commit->parents->next &&
+	    !commit->parents->next->next &&
+	    !oidcmp(&commit->parents->next->item->object.oid,
+		    &merge_commit->object.oid)) {
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return fast_forward_to(&commit->object.oid,
+				       &head_commit->object.oid, 0, opts);
+	}
+
 	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
 		      git_path_merge_head(), 0);
 	write_message("no-ff", 5, git_path_merge_mode(), 0);
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 05/10] rebase-helper --make-script: introduce a flag to recreate merges
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
                     ` (3 preceding siblings ...)
  2018-01-29 22:54   ` [PATCH v2 04/10] sequencer: fast-forward merge commits, if possible Johannes Schindelin
@ 2018-01-29 22:54   ` Johannes Schindelin
  2018-01-29 22:54   ` [PATCH v2 06/10] rebase: introduce the --recreate-merges option Johannes Schindelin
                     ` (7 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:54 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

The sequencer just learned new commands intended to recreate branch
structure (similar in spirit to --preserve-merges, but with a
substantially less-broken design).

Let's allow the rebase--helper to generate todo lists making use of
these commands, triggered by the new --recreate-merges option. For a
commit topology like this (where the HEAD points to C):

	- A - B - C
	    \   /
	      D

the generated todo list would look like this:

	# branch D
	pick 0123 A
	label branch-point
	pick 1234 D
	label D

	reset branch-point
	pick 2345 B
	merge 3456 D C

To keep things simple, we first only implement support for merge commits
with exactly two parents, leaving support for octopus merges to a later
patch in this patch series.

As a special, hard-coded label, all merge-recreating todo lists start with
the command `label onto` so that we can later always refer to the revision
onto which everything is rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/rebase--helper.c |   4 +-
 sequencer.c              | 346 ++++++++++++++++++++++++++++++++++++++++++++++-
 sequencer.h              |   1 +
 3 files changed, 348 insertions(+), 3 deletions(-)

diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index 7daee544b7b..a34ab5c0655 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -12,7 +12,7 @@ static const char * const builtin_rebase_helper_usage[] = {
 int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
-	unsigned flags = 0, keep_empty = 0;
+	unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
 	int abbreviate_commands = 0;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
@@ -22,6 +22,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 	struct option options[] = {
 		OPT_BOOL(0, "ff", &opts.allow_ff, N_("allow fast-forward")),
 		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
+		OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge commits")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -55,6 +56,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
+	flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
 	if (command == CONTINUE && argc == 1)
diff --git a/sequencer.c b/sequencer.c
index df61e7883b3..d5af315a440 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -23,6 +23,8 @@
 #include "hashmap.h"
 #include "unpack-trees.h"
 #include "worktree.h"
+#include "oidmap.h"
+#include "oidset.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -2786,6 +2788,338 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
 	strbuf_release(&sob);
 }
 
+struct labels_entry {
+	struct hashmap_entry entry;
+	char label[FLEX_ARRAY];
+};
+
+static int labels_cmp(const void *fndata, const struct labels_entry *a,
+		      const struct labels_entry *b, const void *key)
+{
+	return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
+}
+
+struct string_entry {
+	struct oidmap_entry entry;
+	char string[FLEX_ARRAY];
+};
+
+struct label_state {
+	struct oidmap commit2label;
+	struct hashmap labels;
+	struct strbuf buf;
+};
+
+static const char *label_oid(struct object_id *oid, const char *label,
+			     struct label_state *state)
+{
+	struct labels_entry *labels_entry;
+	struct string_entry *string_entry;
+	struct object_id dummy;
+	size_t len;
+	int i;
+
+	string_entry = oidmap_get(&state->commit2label, oid);
+	if (string_entry)
+		return string_entry->string;
+
+	/*
+	 * For "uninteresting" commits, i.e. commits that are not to be
+	 * rebased, and which can therefore not be labeled, we use a unique
+	 * abbreviation of the commit name. This is slightly more complicated
+	 * than calling find_unique_abbrev() because we also need to make
+	 * sure that the abbreviation does not conflict with any other
+	 * label.
+	 *
+	 * We disallow "interesting" commits to be labeled by a string that
+	 * is a valid full-length hash, to ensure that we always can find an
+	 * abbreviation for any uninteresting commit's names that does not
+	 * clash with any other label.
+	 */
+	if (!label) {
+		char *p;
+
+		strbuf_reset(&state->buf);
+		strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
+		label = p = state->buf.buf;
+
+		find_unique_abbrev_r(p, oid->hash, default_abbrev);
+
+		/*
+		 * We may need to extend the abbreviated hash so that there is
+		 * no conflicting label.
+		 */
+		if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
+			size_t i = strlen(p) + 1;
+
+			oid_to_hex_r(p, oid);
+			for (; i < GIT_SHA1_HEXSZ; i++) {
+				char save = p[i];
+				p[i] = '\0';
+				if (!hashmap_get_from_hash(&state->labels,
+							   strihash(p), p))
+					break;
+				p[i] = save;
+			}
+		}
+	} else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
+		    !get_oid_hex(label, &dummy)) ||
+		   hashmap_get_from_hash(&state->labels,
+					 strihash(label), label)) {
+		/*
+		 * If the label already exists, or if the label is a valid full
+		 * OID, we append a dash and a number to make it unique.
+		 */
+		struct strbuf *buf = &state->buf;
+
+		strbuf_reset(buf);
+		strbuf_add(buf, label, len);
+
+		for (i = 2; ; i++) {
+			strbuf_setlen(buf, len);
+			strbuf_addf(buf, "-%d", i);
+			if (!hashmap_get_from_hash(&state->labels,
+						   strihash(buf->buf),
+						   buf->buf))
+				break;
+		}
+
+		label = buf->buf;
+	}
+
+	FLEX_ALLOC_STR(labels_entry, label, label);
+	hashmap_entry_init(labels_entry, strihash(label));
+	hashmap_add(&state->labels, labels_entry);
+
+	FLEX_ALLOC_STR(string_entry, string, label);
+	oidcpy(&string_entry->entry.oid, oid);
+	oidmap_put(&state->commit2label, string_entry);
+
+	return string_entry->string;
+}
+
+static int make_script_with_merges(struct pretty_print_context *pp,
+				   struct rev_info *revs, FILE *out,
+				   unsigned flags)
+{
+	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
+	struct strbuf label = STRBUF_INIT;
+	struct commit_list *commits = NULL, **tail = &commits, *iter;
+	struct commit_list *tips = NULL, **tips_tail = &tips;
+	struct commit *commit;
+	struct oidmap commit2todo = OIDMAP_INIT;
+	struct string_entry *entry;
+	struct oidset interesting = OIDSET_INIT, child_seen = OIDSET_INIT,
+		shown = OIDSET_INIT;
+	struct label_state state = { OIDMAP_INIT, { NULL }, STRBUF_INIT };
+
+	int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
+	const char *cmd_pick = abbr ? "p" : "pick",
+		*cmd_label = abbr ? "l" : "label",
+		*cmd_reset = abbr ? "t" : "reset",
+		*cmd_merge = abbr ? "m" : "merge";
+
+	oidmap_init(&commit2todo, 0);
+	oidmap_init(&state.commit2label, 0);
+	hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
+	strbuf_init(&state.buf, 32);
+
+	if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
+		struct object_id *oid = &revs->cmdline.rev[0].item->oid;
+		FLEX_ALLOC_STR(entry, string, "onto");
+		oidcpy(&entry->entry.oid, oid);
+		oidmap_put(&state.commit2label, entry);
+	}
+
+	/*
+	 * First phase:
+	 * - get onelines for all commits
+	 * - gather all branch tips (i.e. 2nd or later parents of merges)
+	 * - label all branch tips
+	 */
+	while ((commit = get_revision(revs))) {
+		struct commit_list *to_merge;
+		int is_octopus;
+		const char *p1, *p2;
+		struct object_id *oid;
+
+		tail = &commit_list_insert(commit, tail)->next;
+		oidset_insert(&interesting, &commit->object.oid);
+
+		if ((commit->object.flags & PATCHSAME))
+			continue;
+
+		strbuf_reset(&oneline);
+		pretty_print_commit(pp, commit, &oneline);
+
+		to_merge = commit->parents ? commit->parents->next : NULL;
+		if (!to_merge) {
+			/* non-merge commit: easy case */
+			strbuf_reset(&buf);
+			if (!keep_empty && is_original_commit_empty(commit))
+				strbuf_addf(&buf, "%c ", comment_line_char);
+			strbuf_addf(&buf, "%s %s %s", cmd_pick,
+				    oid_to_hex(&commit->object.oid),
+				    oneline.buf);
+
+			FLEX_ALLOC_STR(entry, string, buf.buf);
+			oidcpy(&entry->entry.oid, &commit->object.oid);
+			oidmap_put(&commit2todo, entry);
+
+			continue;
+		}
+
+		is_octopus = to_merge && to_merge->next;
+
+		if (is_octopus)
+			BUG("Octopus merges not yet supported");
+
+		/* Create a label */
+		strbuf_reset(&label);
+		if (skip_prefix(oneline.buf, "Merge ", &p1) &&
+		    (p1 = strchr(p1, '\'')) &&
+		    (p2 = strchr(++p1, '\'')))
+			strbuf_add(&label, p1, p2 - p1);
+		else if (skip_prefix(oneline.buf, "Merge pull request ",
+				     &p1) &&
+			 (p1 = strstr(p1, " from ")))
+			strbuf_addstr(&label, p1 + strlen(" from "));
+		else
+			strbuf_addbuf(&label, &oneline);
+
+		for (p1 = label.buf; *p1; p1++)
+			if (isspace(*p1))
+				*(char *)p1 = '-';
+
+		strbuf_reset(&buf);
+		strbuf_addf(&buf, "%s %s",
+			    cmd_merge, oid_to_hex(&commit->object.oid));
+
+		/* label the tip of merged branch */
+		oid = &to_merge->item->object.oid;
+		strbuf_addch(&buf, ' ');
+
+		if (!oidset_contains(&interesting, oid))
+			strbuf_addstr(&buf, label_oid(oid, NULL, &state));
+		else {
+			tips_tail = &commit_list_insert(to_merge->item,
+							tips_tail)->next;
+
+			strbuf_addstr(&buf, label_oid(oid, label.buf, &state));
+		}
+		strbuf_addf(&buf, " %s", oneline.buf);
+
+		FLEX_ALLOC_STR(entry, string, buf.buf);
+		oidcpy(&entry->entry.oid, &commit->object.oid);
+		oidmap_put(&commit2todo, entry);
+	}
+
+	/*
+	 * Second phase:
+	 * - label branch points
+	 * - add HEAD to the branch tips
+	 */
+	for (iter = commits; iter; iter = iter->next) {
+		struct commit_list *parent = iter->item->parents;
+		for (; parent; parent = parent->next) {
+			struct object_id *oid = &parent->item->object.oid;
+			if (!oidset_contains(&interesting, oid))
+				continue;
+			if (!oidset_contains(&child_seen, oid))
+				oidset_insert(&child_seen, oid);
+			else
+				label_oid(oid, "branch-point", &state);
+		}
+
+		/* Add HEAD as implict "tip of branch" */
+		if (!iter->next)
+			tips_tail = &commit_list_insert(iter->item,
+							tips_tail)->next;
+	}
+
+	/*
+	 * Third phase: output the todo list. This is a bit tricky, as we
+	 * want to avoid jumping back and forth between revisions. To
+	 * accomplish that goal, we walk backwards from the branch tips,
+	 * gathering commits not yet shown, reversing the list on the fly,
+	 * then outputting that list (labeling revisions as needed).
+	 */
+	fprintf(out, "%s onto\n", cmd_label);
+	for (iter = tips; iter; iter = iter->next) {
+		struct commit_list *list = NULL, *iter2;
+
+		commit = iter->item;
+		if (oidset_contains(&shown, &commit->object.oid))
+			continue;
+		entry = oidmap_get(&state.commit2label, &commit->object.oid);
+
+		if (entry)
+			fprintf(out, "\n# Branch %s\n", entry->string);
+		else
+			fprintf(out, "\n");
+
+		while (oidset_contains(&interesting, &commit->object.oid) &&
+		       !oidset_contains(&shown, &commit->object.oid)) {
+			commit_list_insert(commit, &list);
+			if (!commit->parents) {
+				commit = NULL;
+				break;
+			}
+			commit = commit->parents->item;
+		}
+
+		if (!commit)
+			fprintf(out, "%s onto\n", cmd_reset);
+		else {
+			const char *to = NULL;
+
+			entry = oidmap_get(&state.commit2label,
+					   &commit->object.oid);
+			if (entry)
+				to = entry->string;
+
+			if (!to || !strcmp(to, "onto"))
+				fprintf(out, "%s onto\n", cmd_reset);
+			else {
+				strbuf_reset(&oneline);
+				pretty_print_commit(pp, commit, &oneline);
+				fprintf(out, "%s %s # %s\n",
+					cmd_reset, to, oneline.buf);
+			}
+		}
+
+		for (iter2 = list; iter2; iter2 = iter2->next) {
+			struct object_id *oid = &iter2->item->object.oid;
+			entry = oidmap_get(&commit2todo, oid);
+			/* only show if not already upstream */
+			if (entry)
+				fprintf(out, "%s\n", entry->string);
+			entry = oidmap_get(&state.commit2label, oid);
+			if (entry)
+				fprintf(out, "%s %s\n",
+					cmd_label, entry->string);
+			oidset_insert(&shown, oid);
+		}
+
+		free_commit_list(list);
+	}
+
+	free_commit_list(commits);
+	free_commit_list(tips);
+
+	strbuf_release(&label);
+	strbuf_release(&oneline);
+	strbuf_release(&buf);
+
+	oidmap_free(&commit2todo, 1);
+	oidmap_free(&state.commit2label, 1);
+	hashmap_free(&state.labels, 1);
+	strbuf_release(&state.buf);
+
+	return 0;
+}
+
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags)
 {
@@ -2796,11 +3130,16 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	struct commit *commit;
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
 	const char *insn = flags & TODO_LIST_ABBREVIATE_CMDS ? "p" : "pick";
+	int recreate_merges = flags & TODO_LIST_RECREATE_MERGES;
 
 	init_revisions(&revs, NULL);
 	revs.verbose_header = 1;
-	revs.max_parents = 1;
-	revs.cherry_pick = 1;
+	if (recreate_merges)
+		revs.cherry_mark = 1;
+	else {
+		revs.max_parents = 1;
+		revs.cherry_pick = 1;
+	}
 	revs.limited = 1;
 	revs.reverse = 1;
 	revs.right_only = 1;
@@ -2824,6 +3163,9 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	if (prepare_revision_walk(&revs) < 0)
 		return error(_("make_script: error preparing revisions"));
 
+	if (recreate_merges)
+		return make_script_with_merges(&pp, &revs, out, flags);
+
 	while ((commit = get_revision(&revs))) {
 		strbuf_reset(&buf);
 		if (!keep_empty && is_original_commit_empty(commit))
diff --git a/sequencer.h b/sequencer.h
index 81f6d7d393f..11d1ac925ef 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -48,6 +48,7 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_KEEP_EMPTY (1U << 0)
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
+#define TODO_LIST_RECREATE_MERGES (1U << 3)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 06/10] rebase: introduce the --recreate-merges option
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
                     ` (4 preceding siblings ...)
  2018-01-29 22:54   ` [PATCH v2 05/10] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
@ 2018-01-29 22:54   ` Johannes Schindelin
  2018-01-29 22:54   ` [PATCH v2 07/10] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
                     ` (6 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:54 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Once upon a time, this here developer thought: wouldn't it be nice if,
say, Git for Windows' patches on top of core Git could be represented as
a thicket of branches, and be rebased on top of core Git in order to
maintain a cherry-pick'able set of patch series?

The original attempt at an answer was: git rebase --preserve-merges.

However, that experiment was never intended as an interactive option,
and it only piggy-backed on git rebase --interactive because that
command's implementation looked already very, very familiar: it was
designed by the same person who designed --preserve-merges: yours truly.

Some time later, some other developer (I am looking at you, Andreas!
;-)) decided that it would be a good idea to allow --preserve-merges to
be combined with --interactive (with caveats!) and the Git maintainer
(well, the interim Git maintainer during Junio's absence, that is)
agreed, and that is when the glamor of the --preserve-merges design
started to fall apart rather quickly and unglamorously.

The reason? In --preserve-merges mode, the parents of a merge commit (or
for that matter, of *any* commit) were not stated explicitly, but were
*implied* by the commit name passed to the `pick` command.

This made it impossible, for example, to reorder commits. Not to mention
to flatten the branch topology or, deity forbid, to split topic branches
into two.

Alas, these shortcomings also prevented that mode (whose original
purpose was to serve Git for Windows' needs, with the additional hope
that it may be useful to others, too) from serving Git for Windows'
needs.

Five years later, when it became really untenable to have one unwieldy,
big hodge-podge patch series of partly related, partly unrelated patches
in Git for Windows that was rebased onto core Git's tags from time to
time (earning the undeserved wrath of the developer of the ill-fated
git-remote-hg series that first obsoleted Git for Windows' competing
approach, only to be abandoned without maintainer later) was really
untenable, the "Git garden shears" were born [*1*/*2*]: a script,
piggy-backing on top of the interactive rebase, that would first
determine the branch topology of the patches to be rebased, create a
pseudo todo list for further editing, transform the result into a real
todo list (making heavy use of the `exec` command to "implement" the
missing todo list commands) and finally recreate the patch series on
top of the new base commit.

That was in 2013. And it took about three weeks to come up with the
design and implement it as an out-of-tree script. Needless to say, the
implementation needed quite a few years to stabilize, all the while the
design itself proved itself sound.

With this patch, the goodness of the Git garden shears comes to `git
rebase -i` itself. Passing the `--recreate-merges` option will generate
a todo list that can be understood readily, and where it is obvious
how to reorder commits. New branches can be introduced by inserting
`label` commands and calling `merge - <label> <oneline>`. And once this
mode has become stable and universally accepted, we can deprecate the
design mistake that was `--preserve-merges`.

Link *1*:
https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
Link *2*:
https://github.com/git-for-windows/build-extra/blob/master/shears.sh

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt           |   9 +-
 contrib/completion/git-completion.bash |   2 +-
 git-rebase--interactive.sh             |   1 +
 git-rebase.sh                          |   6 ++
 t/t3430-rebase-recreate-merges.sh      | 146 +++++++++++++++++++++++++++++++++
 5 files changed, 162 insertions(+), 2 deletions(-)
 create mode 100755 t/t3430-rebase-recreate-merges.sh

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 8a861c1e0d6..e9da7e26329 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -368,6 +368,12 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
+--recreate-merges::
+	Recreate merge commits instead of flattening the history by replaying
+	merges. Merge conflict resolutions or manual amendments to merge
+	commits are not recreated automatically, but have to be recreated
+	manually.
+
 -p::
 --preserve-merges::
 	Recreate merge commits instead of flattening the history by replaying
@@ -770,7 +776,8 @@ BUGS
 The todo list presented by `--preserve-merges --interactive` does not
 represent the topology of the revision graph.  Editing commits and
 rewording their commit messages should work fine, but attempts to
-reorder commits tend to produce counterintuitive results.
+reorder commits tend to produce counterintuitive results. Use
+--recreate-merges for a more faithful representation.
 
 For example, an attempt to rearrange
 ------------
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 3683c772c55..6893c3adabc 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2008,7 +2008,7 @@ _git_rebase ()
 	--*)
 		__gitcomp "
 			--onto --merge --strategy --interactive
-			--preserve-merges --stat --no-stat
+			--recreate-merges --preserve-merges --stat --no-stat
 			--committer-date-is-author-date --ignore-date
 			--ignore-whitespace --whitespace=
 			--autosquash --no-autosquash
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index d6fd30f6c09..97b7954f7d3 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -902,6 +902,7 @@ fi
 if test t != "$preserve_merges"
 then
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
+		${recreate_merges:+--recreate-merges} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 else
diff --git a/git-rebase.sh b/git-rebase.sh
index fd72a35c65b..d69bc7d0e0d 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,6 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
+recreate-merges!   try to recreate merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -86,6 +87,7 @@ type=
 state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
+recreate_merges=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -262,6 +264,10 @@ do
 	--keep-empty)
 		keep_empty=yes
 		;;
+	--recreate-merges)
+		recreate_merges=t
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
new file mode 100755
index 00000000000..b5ea4130bb5
--- /dev/null
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -0,0 +1,146 @@
+#!/bin/sh
+#
+# Copyright (c) 2017 Johannes E. Schindelin
+#
+
+test_description='git rebase -i --recreate-merges
+
+This test runs git rebase "interactively", retaining the branch structure by
+recreating merge commits.
+
+Initial setup:
+
+    -- B --                   (first)
+   /       \
+ A - C - D - E - H            (master)
+       \       /
+         F - G                (second)
+'
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
+
+test_expect_success 'setup' '
+	write_script replace-editor.sh <<-\EOF &&
+	mv "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	cp script-from-scratch "$1"
+	EOF
+
+	test_commit A &&
+	git checkout -b first &&
+	test_commit B &&
+	git checkout master &&
+	test_commit C &&
+	test_commit D &&
+	git merge --no-commit B &&
+	test_tick &&
+	git commit -m E &&
+	git tag -m E E &&
+	git checkout -b second C &&
+	test_commit F &&
+	test_commit G &&
+	git checkout master &&
+	git merge --no-commit G &&
+	test_tick &&
+	git commit -m H &&
+	git tag -m H H
+'
+
+cat >script-from-scratch <<\EOF
+label onto
+
+# onebranch
+pick G
+pick D
+label onebranch
+
+# second
+reset onto
+pick B
+label second
+
+reset onto
+merge H second
+merge - onebranch Merge the topic branch 'onebranch'
+EOF
+
+test_cmp_graph () {
+	cat >expect &&
+	git log --graph --boundary --format=%s "$@" >output &&
+	sed "s/ *$//" <output >output.trimmed &&
+	test_cmp expect output.trimmed
+}
+
+test_expect_success 'create completely different structure' '
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_tick &&
+	git rebase -i --recreate-merges A &&
+	test_cmp_graph <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	* |   H
+	|\ \
+	| |/
+	|/|
+	| * B
+	|/
+	* A
+	EOF
+'
+
+test_expect_success 'generate correct todo list' '
+	cat >expect <<-\EOF &&
+	label onto
+
+	reset onto
+	pick d9df450 B
+	label E
+
+	reset onto
+	pick 5dee784 C
+	label branch-point
+	pick ca2c861 F
+	pick 088b00a G
+	label H
+
+	reset branch-point # C
+	pick 12bd07b D
+	merge 2051b56 E E
+	merge 233d48a H H
+
+	EOF
+
+	grep -v "^#" <.git/ORIGINAL-TODO >output &&
+	test_cmp expect output
+'
+
+test_expect_success 'with a branch tip that was cherry-picked already' '
+	git checkout -b already-upstream master &&
+	base="$(git rev-parse --verify HEAD)" &&
+
+	test_commit A1 &&
+	test_commit A2 &&
+	git reset --hard $base &&
+	test_commit B1 &&
+	test_tick &&
+	git merge -m "Merge branch A" A2 &&
+
+	git checkout -b upstream-with-a2 $base &&
+	test_tick &&
+	git cherry-pick A2 &&
+
+	git checkout already-upstream &&
+	test_tick &&
+	git rebase -i --recreate-merges upstream-with-a2 &&
+	test_cmp_graph upstream-with-a2.. <<-\EOF
+	*   Merge branch A
+	|\
+	| * A1
+	* | B1
+	|/
+	o A2
+	EOF
+'
+
+test_done
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 07/10] sequencer: make refs generated by the `label` command worktree-local
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
                     ` (5 preceding siblings ...)
  2018-01-29 22:54   ` [PATCH v2 06/10] rebase: introduce the --recreate-merges option Johannes Schindelin
@ 2018-01-29 22:54   ` Johannes Schindelin
  2018-01-29 22:55   ` [PATCH v2 08/10] sequencer: handle autosquash and post-rewrite for merge commands Johannes Schindelin
                     ` (5 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:54 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

This allows for rebases to be run in parallel in separate worktrees
(think: interrupted in the middle of one rebase, being asked to perform
a different rebase, adding a separate worktree just for that job).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 refs.c                            |  3 ++-
 t/t3430-rebase-recreate-merges.sh | 14 ++++++++++++++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 20ba82b4343..e8b84c189ff 100644
--- a/refs.c
+++ b/refs.c
@@ -600,7 +600,8 @@ int dwim_log(const char *str, int len, struct object_id *oid, char **log)
 static int is_per_worktree_ref(const char *refname)
 {
 	return !strcmp(refname, "HEAD") ||
-		starts_with(refname, "refs/bisect/");
+		starts_with(refname, "refs/bisect/") ||
+		starts_with(refname, "refs/rewritten/");
 }
 
 static int is_pseudoref_syntax(const char *refname)
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index b5ea4130bb5..5295bb03dc0 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -143,4 +143,18 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'refs/rewritten/* is worktree-local' '
+	git worktree add wt &&
+	cat >wt/script-from-scratch <<-\EOF &&
+	label xyz
+	exec GIT_DIR=../.git git rev-parse --verify refs/rewritten/xyz >a || :
+	exec git rev-parse --verify refs/rewritten/xyz >b
+	EOF
+
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	git -C wt rebase -i HEAD &&
+	test_must_be_empty wt/a &&
+	test_cmp_rev HEAD "$(cat wt/b)"
+'
+
 test_done
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 08/10] sequencer: handle autosquash and post-rewrite for merge commands
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
                     ` (6 preceding siblings ...)
  2018-01-29 22:54   ` [PATCH v2 07/10] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
@ 2018-01-29 22:55   ` Johannes Schindelin
  2018-01-29 22:55   ` [PATCH v2 09/10] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
                     ` (4 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:55 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

In the previous patches, we implemented the basic functionality of the
`git rebase -i --recreate-merges` command, in particular the `merge`
command to create merge commits in the sequencer.

The interactive rebase is a lot more these days, though, than a simple
cherry-pick in a loop. For example, it calls the post-rewrite hook (if
any) after rebasing with a mapping of the old->new commits. And the
interactive rebase also supports the autosquash mode, where commits
whose oneline is of the form `fixup! <oneline>` or `squash! <oneline>`
are rearranged to amend commits whose oneline they match.

This patch implements the post-rewrite and autosquash handling for the
`merge` command we just introduced. The other commands that were added
recently (`label` and `reset`) do not create new commits, therefore
post-rewrite & autosquash do not need to handle them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c                       | 10 +++++++---
 t/t3430-rebase-recreate-merges.sh | 25 +++++++++++++++++++++++++
 2 files changed, 32 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index d5af315a440..4cc73775394 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2414,10 +2414,13 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_label(item->arg, item->arg_len);
 		else if (item->command == TODO_RESET)
 			res = do_reset(item->arg, item->arg_len);
-		else if (item->command == TODO_MERGE)
+		else if (item->command == TODO_MERGE) {
 			res = do_merge(item->commit,
 				       item->arg, item->arg_len, opts);
-		else if (!is_noop(item->command))
+			if (item->commit)
+				record_in_rewritten(&item->commit->object.oid,
+						    peek_command(todo_list, 1));
+		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
@@ -3560,7 +3563,8 @@ int rearrange_squash(void)
 		struct subject2item_entry *entry;
 
 		next[i] = tail[i] = -1;
-		if (item->command >= TODO_EXEC) {
+		if (item->command >= TODO_EXEC &&
+		    (item->command != TODO_MERGE || !item->commit)) {
 			subjects[i] = NULL;
 			continue;
 		}
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index 5295bb03dc0..2eeda0c512b 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -157,4 +157,29 @@ test_expect_success 'refs/rewritten/* is worktree-local' '
 	test_cmp_rev HEAD "$(cat wt/b)"
 '
 
+test_expect_success 'post-rewrite hook and fixups work for merges' '
+	git checkout -b post-rewrite &&
+	test_commit same1 &&
+	git reset --hard HEAD^ &&
+	test_commit same2 &&
+	git merge -m "to fix up" same1 &&
+	echo same old same old >same2.t &&
+	test_tick &&
+	git commit --fixup HEAD same2.t &&
+	fixup="$(git rev-parse HEAD)" &&
+
+	mkdir -p .git/hooks &&
+	test_when_finished "rm .git/hooks/post-rewrite" &&
+	echo "cat >actual" | write_script .git/hooks/post-rewrite &&
+
+	test_tick &&
+	git rebase -i --autosquash --recreate-merges HEAD^^^ &&
+	printf "%s %s\n%s %s\n%s %s\n%s %s\n" >expect $(git rev-parse \
+		$fixup^^2 HEAD^2 \
+		$fixup^^ HEAD^ \
+		$fixup^ HEAD \
+		$fixup HEAD) &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 09/10] pull: accept --rebase=recreate to recreate the branch topology
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
                     ` (7 preceding siblings ...)
  2018-01-29 22:55   ` [PATCH v2 08/10] sequencer: handle autosquash and post-rewrite for merge commands Johannes Schindelin
@ 2018-01-29 22:55   ` Johannes Schindelin
  2018-01-29 22:55   ` [PATCH v2 10/10] rebase -i: introduce --recreate-merges=[no-]rebase-cousins Johannes Schindelin
                     ` (3 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:55 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Similar to the `preserve` mode simply passing the `--preserve-merges`
option to the `rebase` command, the `recreate` mode simply passes the
`--recreate-merges` option.

This will allow users to conveniently rebase non-trivial commit
topologies when pulling new commits, without flattening them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/config.txt               |  8 ++++++++
 Documentation/git-pull.txt             |  5 ++++-
 builtin/pull.c                         | 14 ++++++++++----
 builtin/remote.c                       |  2 ++
 contrib/completion/git-completion.bash |  2 +-
 5 files changed, 25 insertions(+), 6 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0e25b2c92b3..da41ab246dc 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1058,6 +1058,10 @@ branch.<name>.rebase::
 	"git pull" is run. See "pull.rebase" for doing this in a non
 	branch-specific manner.
 +
+When recreate, also pass `--recreate-merges` along to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
@@ -2607,6 +2611,10 @@ pull.rebase::
 	pull" is run. See "branch.<name>.rebase" for setting this on a
 	per-branch basis.
 +
+When recreate, also pass `--recreate-merges` along to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
diff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt
index ce05b7a5b13..b4f9f057ea9 100644
--- a/Documentation/git-pull.txt
+++ b/Documentation/git-pull.txt
@@ -101,13 +101,16 @@ Options related to merging
 include::merge-options.txt[]
 
 -r::
---rebase[=false|true|preserve|interactive]::
+--rebase[=false|true|recreate|preserve|interactive]::
 	When true, rebase the current branch on top of the upstream
 	branch after fetching. If there is a remote-tracking branch
 	corresponding to the upstream branch and the upstream branch
 	was rebased since last fetched, the rebase uses that information
 	to avoid rebasing non-local changes.
 +
+When set to recreate, rebase with the `--recreate-merges` option passed
+to `git rebase` so that locally created merge commits will not be flattened.
++
 When set to preserve, rebase with the `--preserve-merges` option passed
 to `git rebase` so that locally created merge commits will not be flattened.
 +
diff --git a/builtin/pull.c b/builtin/pull.c
index 511dbbe0f6e..e33c84e0345 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -27,14 +27,16 @@ enum rebase_type {
 	REBASE_FALSE = 0,
 	REBASE_TRUE,
 	REBASE_PRESERVE,
+	REBASE_RECREATE,
 	REBASE_INTERACTIVE
 };
 
 /**
  * Parses the value of --rebase. If value is a false value, returns
  * REBASE_FALSE. If value is a true value, returns REBASE_TRUE. If value is
- * "preserve", returns REBASE_PRESERVE. If value is a invalid value, dies with
- * a fatal error if fatal is true, otherwise returns REBASE_INVALID.
+ * "recreate", returns REBASE_RECREATE. If value is "preserve", returns
+ * REBASE_PRESERVE. If value is a invalid value, dies with a fatal error if
+ * fatal is true, otherwise returns REBASE_INVALID.
  */
 static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		int fatal)
@@ -47,6 +49,8 @@ static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		return REBASE_TRUE;
 	else if (!strcmp(value, "preserve"))
 		return REBASE_PRESERVE;
+	else if (!strcmp(value, "recreate"))
+		return REBASE_RECREATE;
 	else if (!strcmp(value, "interactive"))
 		return REBASE_INTERACTIVE;
 
@@ -130,7 +134,7 @@ static struct option pull_options[] = {
 	/* Options passed to git-merge or git-rebase */
 	OPT_GROUP(N_("Options related to merging")),
 	{ OPTION_CALLBACK, 'r', "rebase", &opt_rebase,
-	  "false|true|preserve|interactive",
+	  "false|true|recreate|preserve|interactive",
 	  N_("incorporate changes by rebasing rather than merging"),
 	  PARSE_OPT_OPTARG, parse_opt_rebase },
 	OPT_PASSTHRU('n', NULL, &opt_diffstat, NULL,
@@ -798,7 +802,9 @@ static int run_rebase(const struct object_id *curr_head,
 	argv_push_verbosity(&args);
 
 	/* Options passed to git-rebase */
-	if (opt_rebase == REBASE_PRESERVE)
+	if (opt_rebase == REBASE_RECREATE)
+		argv_array_push(&args, "--recreate-merges");
+	else if (opt_rebase == REBASE_PRESERVE)
 		argv_array_push(&args, "--preserve-merges");
 	else if (opt_rebase == REBASE_INTERACTIVE)
 		argv_array_push(&args, "--interactive");
diff --git a/builtin/remote.c b/builtin/remote.c
index d95bf904c3b..b7d0f7ce596 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -306,6 +306,8 @@ static int config_read_branches(const char *key, const char *value, void *cb)
 				info->rebase = v;
 			else if (!strcmp(value, "preserve"))
 				info->rebase = NORMAL_REBASE;
+			else if (!strcmp(value, "recreate"))
+				info->rebase = NORMAL_REBASE;
 			else if (!strcmp(value, "interactive"))
 				info->rebase = INTERACTIVE_REBASE;
 		}
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 6893c3adabc..6f98c96fee9 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2182,7 +2182,7 @@ _git_config ()
 		return
 		;;
 	branch.*.rebase)
-		__gitcomp "false true preserve interactive"
+		__gitcomp "false true recreate preserve interactive"
 		return
 		;;
 	remote.pushdefault)
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 10/10] rebase -i: introduce --recreate-merges=[no-]rebase-cousins
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
                     ` (8 preceding siblings ...)
  2018-01-29 22:55   ` [PATCH v2 09/10] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
@ 2018-01-29 22:55   ` Johannes Schindelin
  2018-01-30 18:47   ` [PATCH v2 00/10] rebase -i: offer to recreate merge commits Stefan Beller
                     ` (2 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-29 22:55 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

This one is a bit tricky to explain, so let's try with a diagram:

        C
      /   \
A - B - E - F
  \   /
    D

To illustrate what this new mode is all about, let's consider what
happens upon `git rebase -i --recreate-merges B`, in particular to
the commit `D`. So far, the new branch structure would be:

       --- C' --
      /         \
A - B ------ E' - F'
      \    /
        D'

This is not really preserving the branch topology from before! The
reason is that the commit `D` does not have `B` as ancestor, and
therefore it gets rebased onto `B`.

This is unintuitive behavior. Even worse, when recreating branch
structure, most use cases would appear to want cousins *not* to be
rebased onto the new base commit. For example, Git for Windows (the
heaviest user of the Git garden shears, which served as the blueprint
for --recreate-merges) frequently merges branches from `next` early, and
these branches certainly do *not* want to be rebased. In the example
above, the desired outcome would look like this:

       --- C' --
      /         \
A - B ------ E' - F'
  \        /
   -- D' --

Let's introduce the term "cousins" for such commits ("D" in the
example), and let's not rebase them by default, introducing the new
"rebase-cousins" mode for use cases where they should be rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt      |  7 ++++++-
 builtin/rebase--helper.c          |  9 ++++++++-
 git-rebase--interactive.sh        |  1 +
 git-rebase.sh                     | 12 +++++++++++-
 sequencer.c                       |  4 ++++
 sequencer.h                       |  6 ++++++
 t/t3430-rebase-recreate-merges.sh | 23 +++++++++++++++++++++++
 7 files changed, 59 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index e9da7e26329..0e6d020d924 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -368,11 +368,16 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
---recreate-merges::
+--recreate-merges[=(rebase-cousins|no-rebase-cousins)]::
 	Recreate merge commits instead of flattening the history by replaying
 	merges. Merge conflict resolutions or manual amendments to merge
 	commits are not recreated automatically, but have to be recreated
 	manually.
++
+By default, or when `no-rebase-cousins` was specified, commits which do not
+have `<upstream>` as direct ancestor keep their original branch point.
+If the `rebase-cousins` mode is turned on, such commits are rebased onto
+`<upstream>` (or `<onto>`, if specified).
 
 -p::
 --preserve-merges::
diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index a34ab5c0655..cea99cb3235 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
 	unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
-	int abbreviate_commands = 0;
+	int abbreviate_commands = 0, rebase_cousins = -1;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
 		CHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH,
@@ -23,6 +23,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "ff", &opts.allow_ff, N_("allow fast-forward")),
 		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
 		OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge commits")),
+		OPT_BOOL(0, "rebase-cousins", &rebase_cousins,
+			 N_("keep original branch points of cousins")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -57,8 +59,13 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
 	flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
+	flags |= rebase_cousins > 0 ? TODO_LIST_REBASE_COUSINS : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
+	if (rebase_cousins >= 0 && !recreate_merges)
+		warning(_("--[no-]rebase-cousins has no effect without "
+			  "--recreate-merges"));
+
 	if (command == CONTINUE && argc == 1)
 		return !!sequencer_continue(&opts);
 	if (command == ABORT && argc == 1)
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 97b7954f7d3..5e21e4cf269 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -903,6 +903,7 @@ if test t != "$preserve_merges"
 then
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
 		${recreate_merges:+--recreate-merges} \
+		${rebase_cousins:+--rebase-cousins} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 else
diff --git a/git-rebase.sh b/git-rebase.sh
index d69bc7d0e0d..58d778a2da0 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,7 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
-recreate-merges!   try to recreate merges instead of skipping them
+recreate-merges?   try to recreate merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -88,6 +88,7 @@ state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
 recreate_merges=
+rebase_cousins=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -268,6 +269,15 @@ do
 		recreate_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
 		;;
+	--recreate-merges=*)
+		recreate_merges=t
+		case "${1#*=}" in
+		rebase-cousins) rebase_cousins=t;;
+		no-rebase-cousins) rebase_cousins=;;
+		*) die "Unknown mode: $1";;
+		esac
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/sequencer.c b/sequencer.c
index 4cc73775394..cd2f2ae5d53 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2906,6 +2906,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 				   unsigned flags)
 {
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	int rebase_cousins = flags & TODO_LIST_REBASE_COUSINS;
 	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
 	struct strbuf label = STRBUF_INIT;
 	struct commit_list *commits = NULL, **tail = &commits, *iter;
@@ -3081,6 +3082,9 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 					   &commit->object.oid);
 			if (entry)
 				to = entry->string;
+			else if (!rebase_cousins)
+				to = label_oid(&commit->object.oid, NULL,
+					       &state);
 
 			if (!to || !strcmp(to, "onto"))
 				fprintf(out, "%s onto\n", cmd_reset);
diff --git a/sequencer.h b/sequencer.h
index 11d1ac925ef..deebc6e3258 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -49,6 +49,12 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
 #define TODO_LIST_RECREATE_MERGES (1U << 3)
+/*
+ * When recreating merges, commits that do have the base commit as ancestor
+ * ("cousins") are *not* rebased onto the new base by default. If those
+ * commits should be rebased onto the new base, this flag needs to be passed.
+ */
+#define TODO_LIST_REBASE_COUSINS (1U << 4)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index 2eeda0c512b..ab51b584ff9 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -143,6 +143,29 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'do not rebase cousins unless asked for' '
+	write_script copy-editor.sh <<-\EOF &&
+	cp "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	EOF
+
+	test_config sequence.editor \""$PWD"/copy-editor.sh\" &&
+	git checkout -b cousins master &&
+	before="$(git rev-parse --verify HEAD)" &&
+	test_tick &&
+	git rebase -i --recreate-merges HEAD^ &&
+	test_cmp_rev HEAD $before &&
+	test_tick &&
+	git rebase -i --recreate-merges=rebase-cousins HEAD^ &&
+	test_cmp_graph HEAD^.. <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	|/
+	o H
+	EOF
+'
+
 test_expect_success 'refs/rewritten/* is worktree-local' '
 	git worktree add wt &&
 	cat >wt/script-from-scratch <<-\EOF &&
-- 
2.16.1.windows.1

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/8] sequencer: introduce new commands to reset the revision
  2018-01-29 20:50     ` Johannes Schindelin
@ 2018-01-30  7:12       ` Eric Sunshine
  0 siblings, 0 replies; 412+ messages in thread
From: Eric Sunshine @ 2018-01-30  7:12 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Git List, Junio C Hamano, Jacob Keller

On Mon, Jan 29, 2018 at 3:50 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> On Fri, 19 Jan 2018, Eric Sunshine wrote:
>> On Thu, Jan 18, 2018 at 10:35 AM, Johannes Schindelin
>> <johannes.schindelin@gmx.de> wrote:
>> > +static int do_reset(const char *name, int len)
>> > +{
>> > +       for (i = 0; i < len; i++)
>> > +               if (isspace(name[i]))
>> > +                       len = i;
>>
>> What is the purpose of this loop? I could imagine that it's trying to
>> strip all whitespace from the end of 'name', however, to do that it
>> would iterate backward, not forward. (Or perhaps it's trying to
>> truncate at the first space, but then it would need to invert the
>> condition or use 'break'.) Am I missing something obvious?
>
> Yes, you are missing something obvious. The idea of the `reset` command is
> that it not only has a label, but also the oneline of the original commit:
>
>         reset branch-point sequencer: prepare for cleanup
>
> In this instance, `branch-point` is the label. And for convenience of the
> person editing, it also has the oneline.

No, that's not what I was missing. What I was missing was that
assigning 'i' to 'len' also causes the loop to terminate. It's
embarrassing how long I had to stare at this loop to see that, and I
suspect that's what fooled a couple other reviewers, as well, since
idiomatic loops don't normally muck with the termination condition in
quite that fashion (and is why I suggested that a 'break' might be
missing).

Had the loop been a bit more idiomatic:

    for (i = 0; i < len; i++)
        if (isspace(name[i]))
            break;
    len = i;

then the question would never have arisen. Anyhow, it's a minor point
in the greater scheme of the patch series.

> In the Git garden shears, I separated the two arguments via `#`:
>
>         reset branch-point # sequencer: prepare for cleanup
>
> I guess that is actually more readable, so I will introduce that into this
> patch series, too.

Given my termination-condition blindness, the extra "#" would not have
helped me understand the loop any better.

Having now played with the feature a tiny bit, I don't have a strong
opinion about the "#" other than to note that it seems inconsistent
with other commands which don't use "#" as a separator.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 02/10] sequencer: introduce new commands to reset the revision
  2018-01-29 22:54   ` [PATCH v2 02/10] sequencer: introduce new commands to reset the revision Johannes Schindelin
@ 2018-01-30  8:06     ` Eric Sunshine
  2018-02-10 20:58       ` Johannes Schindelin
  2018-01-30 20:17     ` Stefan Beller
  1 sibling, 1 reply; 412+ messages in thread
From: Eric Sunshine @ 2018-01-30  8:06 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Phillip Wood

On Mon, Jan 29, 2018 at 5:54 PM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> [...]
> This commit implements the commands to label, and to reset to, given
> revisions. The syntax is:
>
>         label <name>
>         reset <name>
> [...]
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> diff --git a/sequencer.c b/sequencer.c
> @@ -1253,7 +1266,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>                 if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
>                         item->command = i;
>                         break;
> -               } else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
> +               } else if ((bol + 1 == eol || bol[1] == ' ') &&
> +                          *bol == todo_command_info[i].c) {

This adds support for commands which have no arguments, however, now
that the "bud" command has been retired, this can go away too, right?

>                         bol++;
>                         item->command = i;
>                         break;
> @@ -1919,6 +1934,144 @@ static int do_exec(const char *command_line)
> +static int safe_append(const char *filename, const char *fmt, ...)
> +{
> +       va_list ap;
> +       struct lock_file lock = LOCK_INIT;
> +       int fd = hold_lock_file_for_update(&lock, filename, 0);
> +       struct strbuf buf = STRBUF_INIT;
> +
> +       if (fd < 0)
> +               return error_errno(_("could not lock '%s'"), filename);

Minor: unable_to_lock_message() can provide a more detailed
explanation of the failure.

> +
> +       if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT)
> +               return error_errno(_("could not read '%s'"), filename);
> +       strbuf_complete(&buf, '\n');
> +       va_start(ap, fmt);
> +       strbuf_vaddf(&buf, fmt, ap);
> +       va_end(ap);

Would it make sense to also

    strbuf_complete(&buf, '\n')

here, as well, to be a bit more robust against lazy callers?

> +
> +       if (write_in_full(fd, buf.buf, buf.len) < 0) {
> +               rollback_lock_file(&lock);
> +               return error_errno(_("could not write to '%s'"), filename);

Reading lockfile.h & tempfile.c, I see that rollback_lock_file()
clobbers write_in_full()'s errno before error_errno() is called.

> +       }
> +       if (commit_lock_file(&lock) < 0) {
> +               rollback_lock_file(&lock);
> +               return error(_("failed to finalize '%s'"), filename);
> +       }
> +
> +       return 0;
> +}
> +
> +static int do_reset(const char *name, int len)
> +{
> +       [...]
> +       strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> +       if (get_oid(ref_name.buf, &oid) &&
> +           get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
> +               error(_("could not read '%s'"), ref_name.buf);

Checking my understanding: The two get_oid() calls allow the argument
to 'reset' to be a label created with the 'label' command or any other
way to name an object, right? If so, then I wonder if the error
invocation should instead be:

    error(_("could not read '%.*s'"), len, name);

> +               rollback_lock_file(&lock);
> +               strbuf_release(&ref_name);
> +               return -1;
> +       }

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 00/10] rebase -i: offer to recreate merge commits
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
                     ` (9 preceding siblings ...)
  2018-01-29 22:55   ` [PATCH v2 10/10] rebase -i: introduce --recreate-merges=[no-]rebase-cousins Johannes Schindelin
@ 2018-01-30 18:47   ` Stefan Beller
  2018-01-31 13:08     ` Johannes Schindelin
  2018-01-30 21:36   ` Junio C Hamano
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
  12 siblings, 1 reply; 412+ messages in thread
From: Stefan Beller @ 2018-01-30 18:47 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Junio C Hamano, Jacob Keller, Philip Oakley, Eric Sunshine,
	Phillip Wood

On Mon, Jan 29, 2018 at 2:54 PM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> Once upon a time, I dreamt of an interactive rebase that would not
> flatten branch structure, but instead recreate the commit topology
> faithfully.
>
> My original attempt was --preserve-merges, but that design was so
> limited that I did not even enable it in interactive mode.
>
> Subsequently, it *was* enabled in interactive mode, with the predictable
> consequences: as the --preserve-merges design does not allow for
> specifying the parents of merge commits explicitly, all the new commits'
> parents are defined *implicitly* by the previous commit history, and
> hence it is *not possible to even reorder commits*.
>
> This design flaw cannot be fixed. Not without a complete re-design, at
> least. This patch series offers such a re-design.
>
> Think of --recreate-merges as "--preserve-merges done right". It
> introduces new verbs for the todo list, `label`, `reset` and `merge`.
> For a commit topology like this:
>
>             A - B - C
>               \   /
>                 D
>
> the generated todo list would look like this:
>
>             # branch D
>             pick 0123 A
>             label branch-point
>             pick 1234 D
>             label D
>
>             reset branch-point
>             pick 2345 B
>             merge 3456 D C
>
> There are more patches in the pipeline, based on this patch series, but
> left for later in the interest of reviewable patch series: one mini
> series to use the sequencer even for `git rebase -i --root`, and another
> one to add support for octopus merges to --recreate-merges.
>
> Changes since v1:
>
> - reintroduced "sequencer: make refs generated by the `label` command
>   worktree-local" (which was squashed into "sequencer: handle autosquash
>   and post-rewrite for merge commands" by accident)
>
> - got rid of the universally-hated `bud` command

Sorry if you got the impression for that. Maybe I was imprecise.
I had no strong opinion one way or another, I merely pointed out the
collision in abbreviation letters with the potential new 'break', IIRC.

'bud' was a special case for resetting to a specific revision
(and labeling it?)

Maybe we can have default labels, such that there is no need to reset
to the first revision manually, but can just use these defaults in the merge.
(I haven't thought about this in the big picture, just food for thought)


>
> - as per Stefan's suggestion, the help blurb at the end of the todo list
>   now lists the syntax
>
> - the no-rebase-cousins mode was made the default; This not only reflects
>   the experience won from those years of using the Git garden shears, but
>   was also deemed the better default in the discussion on the PR at
>   https://github.com/git/git/pull/447
>
> - I tried to clarify the role of the `onto` label in the commit message of
>   `rebase-helper --make-script: introduce a flag to recreate merges`
>
> - fixed punctuation at the end of error(...) messages, and incorrect
>   upper-case at the start
>
> - changed the generated todo lists to separate the label and the oneline in
>   the `reset` command with a `#`, for readability
>
> - dropped redundant paragraph in the commit message that talked about
>   support for octopus merges
>
> - avoided empty error message when HEAD could not be read during do_label()
>
> - merge commits are fast-forwarded only unless --force-rebase was passed
>
> - do_merge() now errors out a lot earlier when HEAD could not be parsed
>
> - the one-letter variables to hold either abbreviated or full todo list
>   instructions in make_script_recreating_merges() were renamed to clearer
>   names
>
> - The description of rebase's --recreate-merge option has been reworded;
>   Hopefully it is a lot more clear now.
>
>
> Johannes Schindelin (9):
>   sequencer: introduce new commands to reset the revision
>   sequencer: introduce the `merge` command
>   sequencer: fast-forward merge commits, if possible
>   rebase-helper --make-script: introduce a flag to recreate merges
>   rebase: introduce the --recreate-merges option
>   sequencer: make refs generated by the `label` command worktree-local
>   sequencer: handle autosquash and post-rewrite for merge commands
>   pull: accept --rebase=recreate to recreate the branch topology
>   rebase -i: introduce --recreate-merges=[no-]rebase-cousins
>
> Stefan Beller (1):
>   git-rebase--interactive: clarify arguments

No need to honor me with authorship, as I just wrote
that patch in a quick hurry to express the idea.
But this is fine, too.

The interdiff looks good to me, I'll review the patches now.

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 02/10] sequencer: introduce new commands to reset the revision
  2018-01-29 22:54   ` [PATCH v2 02/10] sequencer: introduce new commands to reset the revision Johannes Schindelin
  2018-01-30  8:06     ` Eric Sunshine
@ 2018-01-30 20:17     ` Stefan Beller
  2018-01-31 13:21       ` Johannes Schindelin
  1 sibling, 1 reply; 412+ messages in thread
From: Stefan Beller @ 2018-01-30 20:17 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Junio C Hamano, Jacob Keller, Philip Oakley, Eric Sunshine,
	Phillip Wood

On Mon, Jan 29, 2018 at 2:54 PM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> In the upcoming commits, we will teach the sequencer to recreate merges.
> This will be done in a very different way from the unfortunate design of
> `git rebase --preserve-merges` (which does not allow for reordering
> commits, or changing the branch topology).
>
> The main idea is to introduce new todo list commands, to support
> labeling the current revision with a given name, resetting the current
> revision to a previous state, merging labeled revisions.
>
> This idea was developed in Git for Windows' Git garden shears (that are
> used to maintain the "thicket of branches" on top of upstream Git), and
> this patch is part of the effort to make it available to a wider
> audience, as well as to make the entire process more robust (by
> implementing it in a safe and portable language rather than a Unix shell
> script).
>
> This commit implements the commands to label, and to reset to, given
> revisions. The syntax is:
>
>         label <name>
>         reset <name>
>
> Internally, the `label <name>` command creates the ref
> `refs/rewritten/<name>`. This makes it possible to work with the labeled
> revisions interactively, or in a scripted fashion (e.g. via the todo
> list command `exec`).
>
> Later in this patch series, we will mark the `refs/rewritten/` refs as
> worktree-local, to allow for interactive rebases to be run in parallel in
> worktrees linked to the same repository.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  git-rebase--interactive.sh |   2 +
>  sequencer.c                | 180 ++++++++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 179 insertions(+), 3 deletions(-)
>
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index fcedece1860..7e5281e74aa 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -162,6 +162,8 @@ s, squash <commit> = use commit, but meld into previous commit
>  f, fixup <commit> = like \"squash\", but discard this commit's log message
>  x, exec <commit> = run command (the rest of the line) using shell
>  d, drop <commit> = remove commit
> +l, label <label> = label current HEAD with a name
> +t, reset <label> = reset HEAD to a label
>
>  These lines can be re-ordered; they are executed from top to bottom.
>  " | git stripspace --comment-lines >>"$todo"
> diff --git a/sequencer.c b/sequencer.c
> index 4d3f60594cb..92ca8d2adee 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -21,6 +21,8 @@
>  #include "log-tree.h"
>  #include "wt-status.h"
>  #include "hashmap.h"
> +#include "unpack-trees.h"
> +#include "worktree.h"
>
>  #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
>
> @@ -116,6 +118,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
>  static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
>  static GIT_PATH_FUNC(rebase_path_rewritten_pending,
>         "rebase-merge/rewritten-pending")
> +
> +/*
> + * The path of the file listing refs that need to be deleted after the rebase
> + * finishes. This is used by the `merge` command.
> + */

So this file contains (label -> commit), which is appended in do_label,
it uses refs to store the commits in refs/rewritten.
We do not have to worry about the contents of that file getting too long,
or label re-use, because the directory containing all these helper files will
be deleted upon successful rebase in `sequencer_remove_state()`.



> +static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
> +
>  /*
>   * The following files are written by git-rebase just after parsing the
>   * command-line (and are only consumed, not modified, by the sequencer).
> @@ -767,6 +776,8 @@ enum todo_command {
>         TODO_SQUASH,
>         /* commands that do something else than handling a single commit */
>         TODO_EXEC,
> +       TODO_LABEL,
> +       TODO_RESET,
>         /* commands that do nothing but are counted for reporting progress */
>         TODO_NOOP,
>         TODO_DROP,
> @@ -785,6 +796,8 @@ static struct {
>         { 'f', "fixup" },
>         { 's', "squash" },
>         { 'x', "exec" },
> +       { 'l', "label" },
> +       { 't', "reset" },
>         { 0,   "noop" },
>         { 'd', "drop" },
>         { 0,   NULL }
> @@ -1253,7 +1266,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>                 if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
>                         item->command = i;
>                         break;
> -               } else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
> +               } else if ((bol + 1 == eol || bol[1] == ' ') &&
> +                          *bol == todo_command_info[i].c) {
>                         bol++;
>                         item->command = i;
>                         break;
> @@ -1279,7 +1293,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>                 return error(_("missing arguments for %s"),
>                              command_to_string(item->command));
>
> -       if (item->command == TODO_EXEC) {
> +       if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
> +           item->command == TODO_RESET) {
>                 item->commit = NULL;
>                 item->arg = bol;
>                 item->arg_len = (int)(eol - bol);
> @@ -1919,6 +1934,144 @@ static int do_exec(const char *command_line)
>         return status;
>  }
>
> +static int safe_append(const char *filename, const char *fmt, ...)
> +{
> +       va_list ap;
> +       struct lock_file lock = LOCK_INIT;
> +       int fd = hold_lock_file_for_update(&lock, filename, 0);
> +       struct strbuf buf = STRBUF_INIT;
> +
> +       if (fd < 0)
> +               return error_errno(_("could not lock '%s'"), filename);
> +
> +       if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT)
> +               return error_errno(_("could not read '%s'"), filename);
> +       strbuf_complete(&buf, '\n');
> +       va_start(ap, fmt);
> +       strbuf_vaddf(&buf, fmt, ap);
> +       va_end(ap);
> +
> +       if (write_in_full(fd, buf.buf, buf.len) < 0) {
> +               rollback_lock_file(&lock);
> +               return error_errno(_("could not write to '%s'"), filename);
> +       }
> +       if (commit_lock_file(&lock) < 0) {
> +               rollback_lock_file(&lock);
> +               return error(_("failed to finalize '%s'"), filename);
> +       }
> +
> +       return 0;
> +}
> +
> +static int do_label(const char *name, int len)
> +{
> +       struct ref_store *refs = get_main_ref_store();
> +       struct ref_transaction *transaction;
> +       struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
> +       struct strbuf msg = STRBUF_INIT;
> +       int ret = 0;
> +       struct object_id head_oid;
> +
> +       strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> +       strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
> +
> +       transaction = ref_store_transaction_begin(refs, &err);
> +       if (!transaction) {
> +               error("%s", err.buf);
> +               ret = -1;
> +       } else if (get_oid("HEAD", &head_oid)) {
> +               error(_("could not read HEAD"));
> +               ret = -1;
> +       } else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
> +                                         NULL, 0, msg.buf, &err) < 0 ||
> +                  ref_transaction_commit(transaction, &err)) {
> +               error("%s", err.buf);
> +               ret = -1;
> +       }
> +       ref_transaction_free(transaction);
> +       strbuf_release(&err);
> +       strbuf_release(&msg);
> +
> +       if (!ret)
> +               ret = safe_append(rebase_path_refs_to_delete(),
> +                                 "%s\n", ref_name.buf);
> +       strbuf_release(&ref_name);
> +
> +       return ret;
> +}
> +
> +static int do_reset(const char *name, int len)
> +{
> +       struct strbuf ref_name = STRBUF_INIT;
> +       struct object_id oid;
> +       struct lock_file lock = LOCK_INIT;
> +       struct tree_desc desc;
> +       struct tree *tree;
> +       struct unpack_trees_options opts;
> +       int ret = 0, i;
> +
> +       if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> +               return -1;
> +
> +       /* Determine the length of the label */
> +       for (i = 0; i < len; i++)
> +               if (isspace(name[i]))
> +                       len = i;
> +
> +       strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> +       if (get_oid(ref_name.buf, &oid) &&
> +           get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
> +               error(_("could not read '%s'"), ref_name.buf);
> +               rollback_lock_file(&lock);
> +               strbuf_release(&ref_name);
> +               return -1;
> +       }
> +
> +       memset(&opts, 0, sizeof(opts));
> +       opts.head_idx = 1;
> +       opts.src_index = &the_index;
> +       opts.dst_index = &the_index;
> +       opts.fn = oneway_merge;
> +       opts.merge = 1;
> +       opts.update = 1;
> +       opts.reset = 1;
> +
> +       read_cache_unmerged();

In read-tree.c merge.c and pull.c we guard this conditionally
and use die_resolve_conflict to bail out. In am.c we do not.

I think we'd want to guard it here, too?

Constructing an instruction sheet that produces a merge
conflict just before the reset is a bit artificial, but still:

    label onto
    pick abc
    exec false # run "git merge out-of-rebase-merge"
        # manually to produce a conflict
    reset onto # we want to stop here telling the user to fix it.

> +       if (!fill_tree_descriptor(&desc, &oid)) {
> +               error(_("failed to find tree of %s"), oid_to_hex(&oid));
> +               rollback_lock_file(&lock);
> +               free((void *)desc.buffer);
> +               strbuf_release(&ref_name);
> +               return -1;
> +       }
> +
> +       if (unpack_trees(1, &desc, &opts)) {
> +               rollback_lock_file(&lock);
> +               free((void *)desc.buffer);
> +               strbuf_release(&ref_name);
> +               return -1;
> +       }
> +
> +       tree = parse_tree_indirect(&oid);
> +       prime_cache_tree(&the_index, tree);
> +
> +       if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
> +               ret = error(_("could not write index"));
> +       free((void *)desc.buffer);

For most newer structs we have a {release, clear, free}_X,
but for tree_desc's this seems to be the convention to avoid memleaks.

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 00/10] rebase -i: offer to recreate merge commits
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
                     ` (10 preceding siblings ...)
  2018-01-30 18:47   ` [PATCH v2 00/10] rebase -i: offer to recreate merge commits Stefan Beller
@ 2018-01-30 21:36   ` Junio C Hamano
  2018-01-31 13:29     ` Johannes Schindelin
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
  12 siblings, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-01-30 21:36 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Jacob Keller, Stefan Beller, Philip Oakley, Eric Sunshine,
	Phillip Wood

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Changes since v1:
>
> - reintroduced "sequencer: make refs generated by the `label` command
>   worktree-local" (which was squashed into "sequencer: handle autosquash
>   and post-rewrite for merge commands" by accident)

Good.

> - got rid of the universally-hated `bud` command

Universally is a bit too strong a word, unless you want to hint that
you are specifically ignoring my input ;-).

> - the no-rebase-cousins mode was made the default

Although I lack first-hand experience with this implementation, this
design decision matches my instinct.

May comment on individual patches separately, later.

Thanks.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 00/10] rebase -i: offer to recreate merge commits
  2018-01-30 18:47   ` [PATCH v2 00/10] rebase -i: offer to recreate merge commits Stefan Beller
@ 2018-01-31 13:08     ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-31 13:08 UTC (permalink / raw)
  To: Stefan Beller
  Cc: git, Junio C Hamano, Jacob Keller, Philip Oakley, Eric Sunshine,
	Phillip Wood

Hi Stefan,

On Tue, 30 Jan 2018, Stefan Beller wrote:

> > - got rid of the universally-hated `bud` command
> 
> Sorry if you got the impression for that. Maybe I was imprecise.

You were not the most vocal voice. Anyway, `bud` is gone now.

> > Stefan Beller (1):
> >   git-rebase--interactive: clarify arguments
> 
> No need to honor me with authorship, as I just wrote
> that patch in a quick hurry to express the idea.

You wrote it.

> The interdiff looks good to me, I'll review the patches now.

Thanks,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 02/10] sequencer: introduce new commands to reset the revision
  2018-01-30 20:17     ` Stefan Beller
@ 2018-01-31 13:21       ` Johannes Schindelin
  2018-01-31 18:02         ` [PATCH v2 02/10] sequencer: introduce new commands to reset therevision Phillip Wood
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-31 13:21 UTC (permalink / raw)
  To: Stefan Beller
  Cc: git, Junio C Hamano, Jacob Keller, Philip Oakley, Eric Sunshine,
	Phillip Wood

Hi Stefan,

On Tue, 30 Jan 2018, Stefan Beller wrote:

> On Mon, Jan 29, 2018 at 2:54 PM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > @@ -116,6 +118,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
> >  static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
> >  static GIT_PATH_FUNC(rebase_path_rewritten_pending,
> >         "rebase-merge/rewritten-pending")
> > +
> > +/*
> > + * The path of the file listing refs that need to be deleted after the rebase
> > + * finishes. This is used by the `merge` command.
> > + */

Whoops. The comment "This is used by the `merge` command`" is completely
wrong. Will fix.

> So this file contains (label -> commit),

Only `label`. No `commit`.

> which is appended in do_label, it uses refs to store the commits in
> refs/rewritten.  We do not have to worry about the contents of that file
> getting too long, or label re-use, because the directory containing all
> these helper files will be deleted upon successful rebase in
> `sequencer_remove_state()`.

Yes.

> > +static int do_reset(const char *name, int len)
> > +{
> > +       struct strbuf ref_name = STRBUF_INIT;
> > +       struct object_id oid;
> > +       struct lock_file lock = LOCK_INIT;
> > +       struct tree_desc desc;
> > +       struct tree *tree;
> > +       struct unpack_trees_options opts;
> > +       int ret = 0, i;
> > +
> > +       if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> > +               return -1;
> > +
> > +       /* Determine the length of the label */
> > +       for (i = 0; i < len; i++)
> > +               if (isspace(name[i]))
> > +                       len = i;
> > +
> > +       strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> > +       if (get_oid(ref_name.buf, &oid) &&
> > +           get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
> > +               error(_("could not read '%s'"), ref_name.buf);
> > +               rollback_lock_file(&lock);
> > +               strbuf_release(&ref_name);
> > +               return -1;
> > +       }
> > +
> > +       memset(&opts, 0, sizeof(opts));
> > +       opts.head_idx = 1;
> > +       opts.src_index = &the_index;
> > +       opts.dst_index = &the_index;
> > +       opts.fn = oneway_merge;
> > +       opts.merge = 1;
> > +       opts.update = 1;
> > +       opts.reset = 1;
> > +
> > +       read_cache_unmerged();
> 
> In read-tree.c merge.c and pull.c we guard this conditionally
> and use die_resolve_conflict to bail out. In am.c we do not.
> 
> I think we'd want to guard it here, too?

Yes.

> Constructing an instruction sheet that produces a merge
> conflict just before the reset is a bit artificial, but still:
> 
>     label onto
>     pick abc
>     exec false # run "git merge out-of-rebase-merge"
>         # manually to produce a conflict

This would fail already, as `exec` tests for a clean index after the
operation ran.

>     reset onto # we want to stop here telling the user to fix it.

But you are absolutely right that we still need to fix it.

> > +       if (!fill_tree_descriptor(&desc, &oid)) {
> > +               error(_("failed to find tree of %s"), oid_to_hex(&oid));
> > +               rollback_lock_file(&lock);
> > +               free((void *)desc.buffer);
> > +               strbuf_release(&ref_name);
> > +               return -1;
> > +       }
> > +
> > +       if (unpack_trees(1, &desc, &opts)) {
> > +               rollback_lock_file(&lock);
> > +               free((void *)desc.buffer);
> > +               strbuf_release(&ref_name);
> > +               return -1;
> > +       }
> > +
> > +       tree = parse_tree_indirect(&oid);
> > +       prime_cache_tree(&the_index, tree);
> > +
> > +       if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
> > +               ret = error(_("could not write index"));
> > +       free((void *)desc.buffer);
> 
> For most newer structs we have a {release, clear, free}_X,
> but for tree_desc's this seems to be the convention to avoid memleaks.

Yep, this code is just copy-edited from elsewhere. It seemed to be
different enough from the (very generic) use case in builtin/reset.c that
I did not think refactoring this into a convenience function in
unpack-trees.[ch] would make sense.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 00/10] rebase -i: offer to recreate merge commits
  2018-01-30 21:36   ` Junio C Hamano
@ 2018-01-31 13:29     ` Johannes Schindelin
  2018-02-01  6:37       ` Jacob Keller
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-31 13:29 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jacob Keller, Stefan Beller, Philip Oakley, Eric Sunshine,
	Phillip Wood

Hi Junio,

On Tue, 30 Jan 2018, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Changes since v1:
> >
> > - reintroduced "sequencer: make refs generated by the `label` command
> >   worktree-local" (which was squashed into "sequencer: handle autosquash
> >   and post-rewrite for merge commands" by accident)
> 
> Good.
> 
> > - got rid of the universally-hated `bud` command
> 
> Universally is a bit too strong a word, unless you want to hint that
> you are specifically ignoring my input ;-).

In the interest of comic effect, I exaggerated a little.

> > - the no-rebase-cousins mode was made the default
> 
> Although I lack first-hand experience with this implementation, this
> design decision matches my instinct.

Excellent.

> May comment on individual patches separately, later.

I think I may want to introduce a bigger change, still. I forgot who
exactly came up with the suggestion to use `merge -C <original-commit>
<to-merge>` (I think it was Jake), and I reacted too forcefully in
rejecting it.

This design had been my original design in the Git garden shears, and I
did not like it because it felt clunky and it also broke the style of
<command> <commit>.

But the longer I think about this, the more I come to the conclusion that
I was wrong, and that the -C way is the way that leaves the door open to
the pretty elegant `-c <commit>` (imitating `git commit`'s option to
borrow the commit message from elsewhere but still allowing to edit it).

And it also leaves open the door to just write `merge <to-merge>` and have
the sequencer come up with a default merge message that the user can then
edit.

I'll probably refrain from implementing support for -m because I do not
want to implement the dequoting.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/8] sequencer: introduce the `merge` command
  2018-01-29 21:41       ` Johannes Schindelin
@ 2018-01-31 13:48         ` Johannes Schindelin
  2018-01-31 17:58           ` Phillip Wood
  2018-02-01  6:40           ` Jacob Keller
  0 siblings, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-01-31 13:48 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Phillip Wood, Git mailing list, Junio C Hamano, Eric Sunshine

Hi Jake & Phillip,

On Mon, 29 Jan 2018, Johannes Schindelin wrote:

> On Sat, 20 Jan 2018, Jacob Keller wrote:
> 
> > On Fri, Jan 19, 2018 at 6:45 AM, Phillip Wood <phillip.wood@talktalk.net> wrote:
> > > On 18/01/18 15:35, Johannes Schindelin wrote:
> > >>
> > >> This patch adds the `merge` command, with the following syntax:
> > >>
> > >>       merge <commit> <rev> <oneline>
> > >
> > > I'm concerned that this will be confusing for users. All of the other
> > > rebase commands replay the changes in the commit hash immediately
> > > following the command name. This command instead uses the first
> > > commit to specify the message which is different to both 'git merge'
> > > and the existing rebase commands. I wonder if it would be clearer to
> > > have 'merge -C <commit> <rev> ...' instead so it's clear which
> > > argument specifies the message and which the remote head to merge.
> > > It would also allow for 'merge -c <commit> <rev> ...' in the future
> > > for rewording an existing merge message and also avoid the slightly
> > > odd 'merge - <rev> ...'. Where it's creating new merges I'm not sure
> > > it's a good idea to encourage people to only have oneline commit
> > > messages by making it harder to edit them, perhaps it could take
> > > another argument to mean open the editor or not, though as Jake said
> > > I guess it's not that common.
> > 
> > I actually like the idea of re-using commit message options like -C,
> > -c,  and -m, so we could do:
> > 
> > merge -C <commit> ... to take message from commit
> 
> That is exactly how the Git garden shears do it.
> 
> I found it not very readable. That is why I wanted to get away from it in
> --recreate-merges.

I made up my mind. Even if it is not very readable, it is still better
than the `merge A B` where the order of A and B magically determines their
respective roles.

> > merge -c <commit> ...  to take the message from commit and open editor to edit
> > merge -m "<message>" ... to take the message from the quoted test
> > merge ... to merge and open commit editor with default message

I will probably implement -c, but not -m, and will handle the absence of
the -C and -c options to construct a default merge message which can then
be edited.

The -m option just opens such a can of worms with dequoting, that's why I
do not want to do that.

BTW I am still trying to figure out how to present the oneline of the
commit to merge (which is sometimes really helpful because the label might
be less than meaningful) while *still* allowing for octopus merges.

So far, what I have is this:

	merge <original> <to-merge> <oneline>

and for octopus:

	merge <original> "<to-merge> <to-merge2>..." <oneline>...

I think with the -C syntax, it would become something like

	merge -C <original> <to-merge> # <oneline>

and

	merge -C <original> <to-merge> <to-merge2>...
	# Merging: <oneline>
	# Merging: <oneline2>
	# ...

The only qualm I have about this is that `#` really *is* a valid ref name.
(Seriously, it is...). So that would mean that I'd have to disallow `#`
as a label specificially.

Thoughts?

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/8] sequencer: introduce the `merge` command
  2018-01-31 13:48         ` Johannes Schindelin
@ 2018-01-31 17:58           ` Phillip Wood
  2018-02-01  6:40           ` Jacob Keller
  1 sibling, 0 replies; 412+ messages in thread
From: Phillip Wood @ 2018-01-31 17:58 UTC (permalink / raw)
  To: Johannes Schindelin, Jacob Keller
  Cc: Phillip Wood, Git mailing list, Junio C Hamano, Eric Sunshine

On 31/01/18 13:48, Johannes Schindelin wrote:
> Hi Jake & Phillip,
> 
> On Mon, 29 Jan 2018, Johannes Schindelin wrote:
> 
>> On Sat, 20 Jan 2018, Jacob Keller wrote:
>>
>>> On Fri, Jan 19, 2018 at 6:45 AM, Phillip Wood <phillip.wood@talktalk.net> wrote:
>>>> On 18/01/18 15:35, Johannes Schindelin wrote:
>>>>>
>>>>> This patch adds the `merge` command, with the following syntax:
>>>>>
>>>>>       merge <commit> <rev> <oneline>
>>>>
>>>> I'm concerned that this will be confusing for users. All of the other
>>>> rebase commands replay the changes in the commit hash immediately
>>>> following the command name. This command instead uses the first
>>>> commit to specify the message which is different to both 'git merge'
>>>> and the existing rebase commands. I wonder if it would be clearer to
>>>> have 'merge -C <commit> <rev> ...' instead so it's clear which
>>>> argument specifies the message and which the remote head to merge.
>>>> It would also allow for 'merge -c <commit> <rev> ...' in the future
>>>> for rewording an existing merge message and also avoid the slightly
>>>> odd 'merge - <rev> ...'. Where it's creating new merges I'm not sure
>>>> it's a good idea to encourage people to only have oneline commit
>>>> messages by making it harder to edit them, perhaps it could take
>>>> another argument to mean open the editor or not, though as Jake said
>>>> I guess it's not that common.
>>>
>>> I actually like the idea of re-using commit message options like -C,
>>> -c,  and -m, so we could do:
>>>
>>> merge -C <commit> ... to take message from commit
>>
>> That is exactly how the Git garden shears do it.
>>
>> I found it not very readable. That is why I wanted to get away from it in
>> --recreate-merges.
> 
> I made up my mind. Even if it is not very readable, it is still better
> than the `merge A B` where the order of A and B magically determines their
> respective roles.
> 
>>> merge -c <commit> ...  to take the message from commit and open editor to edit
>>> merge -m "<message>" ... to take the message from the quoted test
>>> merge ... to merge and open commit editor with default message
> 
> I will probably implement -c, but not -m, and will handle the absence of
> the -C and -c options to construct a default merge message which can then
> be edited.

That sounds like a good plan (-c can always be added later), I'm really
pleased you changed your mind on this, having the -C may be a bit ugly
but I think it is valuable to have some way of distinguishing the
message commit from the merge heads.

> The -m option just opens such a can of worms with dequoting, that's why I
> do not want to do that.
> 
> BTW I am still trying to figure out how to present the oneline of the
> commit to merge (which is sometimes really helpful because the label might
> be less than meaningful) while *still* allowing for octopus merges.
> 
> So far, what I have is this:
> 
> 	merge <original> <to-merge> <oneline>
> 
> and for octopus:
> 
> 	merge <original> "<to-merge> <to-merge2>..." <oneline>...
> 
> I think with the -C syntax, it would become something like
> 
> 	merge -C <original> <to-merge> # <oneline>
> 
> and
> 
> 	merge -C <original> <to-merge> <to-merge2>...
> 	# Merging: <oneline>
> 	# Merging: <oneline2>
> 	# ...
> 
> The only qualm I have about this is that `#` really *is* a valid ref name.
> (Seriously, it is...). So that would mean that I'd have to disallow `#`
> as a label specificially.
> 
> Thoughts?

As ':' is not a valid ref if you want a separator you could have

	merge -C <original> <to-merge> : <oneline>

personally I'm not sure what value having a separator adds in this case.
I think in the octopus case have a separate comment line for the subject
of each merge head is a good idea - maybe the two head merge could just
have the subject of the remote head in a comment below. I wonder if
having the subject of the commit that is going to be used for the
message may be a useful prompt in some cases but that's just making
things more complicated.

Best Wishes

Phillip

> Ciao,
> Dscho
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 02/10] sequencer: introduce new commands to reset therevision
  2018-01-31 13:21       ` Johannes Schindelin
@ 2018-01-31 18:02         ` Phillip Wood
  2018-02-10 21:49           ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Phillip Wood @ 2018-01-31 18:02 UTC (permalink / raw)
  To: Johannes Schindelin, Stefan Beller
  Cc: git, Junio C Hamano, Jacob Keller, Philip Oakley, Eric Sunshine,
	Phillip Wood

On 31/01/18 13:21, Johannes Schindelin wrote:
> 
> Hi Stefan,
> 
> On Tue, 30 Jan 2018, Stefan Beller wrote:
> 
>> On Mon, Jan 29, 2018 at 2:54 PM, Johannes Schindelin
>> <johannes.schindelin@gmx.de> wrote:
>>> @@ -116,6 +118,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
>>>  static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
>>>  static GIT_PATH_FUNC(rebase_path_rewritten_pending,
>>>         "rebase-merge/rewritten-pending")
>>> +
>>> +/*
>>> + * The path of the file listing refs that need to be deleted after the rebase
>>> + * finishes. This is used by the `merge` command.
>>> + */
> 
> Whoops. The comment "This is used by the `merge` command`" is completely
> wrong. Will fix.
> 
>> So this file contains (label -> commit),
> 
> Only `label`. No `commit`.
> 
>> which is appended in do_label, it uses refs to store the commits in
>> refs/rewritten.  We do not have to worry about the contents of that file
>> getting too long, or label re-use, because the directory containing all
>> these helper files will be deleted upon successful rebase in
>> `sequencer_remove_state()`.
> 
> Yes.
>
It might be a good idea to have 'git rebase --abort' delete the refs as
well as the file though

Best Wishes

Phillip

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 00/10] rebase -i: offer to recreate merge commits
  2018-01-31 13:29     ` Johannes Schindelin
@ 2018-02-01  6:37       ` Jacob Keller
  0 siblings, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-02-01  6:37 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Junio C Hamano, Git mailing list, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

On Wed, Jan 31, 2018 at 5:29 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> I think I may want to introduce a bigger change, still. I forgot who
> exactly came up with the suggestion to use `merge -C <original-commit>
> <to-merge>` (I think it was Jake), and I reacted too forcefully in
> rejecting it.
>

I believe someone else suggested it, but I replied that I liked it.

> This design had been my original design in the Git garden shears, and I
> did not like it because it felt clunky and it also broke the style of
> <command> <commit>.
>

I agree it's a bit weird it breaks the style of "<command> <commit>",
but on some level merge does this anyways as it's the first one to
take more than one argument.

> But the longer I think about this, the more I come to the conclusion that
> I was wrong, and that the -C way is the way that leaves the door open to
> the pretty elegant `-c <commit>` (imitating `git commit`'s option to
> borrow the commit message from elsewhere but still allowing to edit it).
>

The other reason I liked this, is that it matches merge syntax on the
command line, so users don't need to learn a special new syntax for
the todo file.

> And it also leaves open the door to just write `merge <to-merge>` and have
> the sequencer come up with a default merge message that the user can then
> edit.

I like that we could completely forgo the -C and -c in order to allow
the normal default merge commit message that is auto generated as
well.

>
> I'll probably refrain from implementing support for -m because I do not
> want to implement the dequoting.
>

Yea, I don't think that is necessary either.

Thanks,
Jake

> Ciao,
> Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/8] sequencer: introduce the `merge` command
  2018-01-31 13:48         ` Johannes Schindelin
  2018-01-31 17:58           ` Phillip Wood
@ 2018-02-01  6:40           ` Jacob Keller
  1 sibling, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-02-01  6:40 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Phillip Wood, Git mailing list, Junio C Hamano, Eric Sunshine

On Wed, Jan 31, 2018 at 5:48 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Jake & Phillip,
>
> On Mon, 29 Jan 2018, Johannes Schindelin wrote:
>
>> On Sat, 20 Jan 2018, Jacob Keller wrote:
>>
>> > On Fri, Jan 19, 2018 at 6:45 AM, Phillip Wood <phillip.wood@talktalk.net> wrote:
>> > > On 18/01/18 15:35, Johannes Schindelin wrote:
>> > >>
>> > >> This patch adds the `merge` command, with the following syntax:
>> > >>
>> > >>       merge <commit> <rev> <oneline>
>> > >
>> > > I'm concerned that this will be confusing for users. All of the other
>> > > rebase commands replay the changes in the commit hash immediately
>> > > following the command name. This command instead uses the first
>> > > commit to specify the message which is different to both 'git merge'
>> > > and the existing rebase commands. I wonder if it would be clearer to
>> > > have 'merge -C <commit> <rev> ...' instead so it's clear which
>> > > argument specifies the message and which the remote head to merge.
>> > > It would also allow for 'merge -c <commit> <rev> ...' in the future
>> > > for rewording an existing merge message and also avoid the slightly
>> > > odd 'merge - <rev> ...'. Where it's creating new merges I'm not sure
>> > > it's a good idea to encourage people to only have oneline commit
>> > > messages by making it harder to edit them, perhaps it could take
>> > > another argument to mean open the editor or not, though as Jake said
>> > > I guess it's not that common.
>> >
>> > I actually like the idea of re-using commit message options like -C,
>> > -c,  and -m, so we could do:
>> >
>> > merge -C <commit> ... to take message from commit
>>
>> That is exactly how the Git garden shears do it.
>>
>> I found it not very readable. That is why I wanted to get away from it in
>> --recreate-merges.
>
> I made up my mind. Even if it is not very readable, it is still better
> than the `merge A B` where the order of A and B magically determines their
> respective roles.
>
>> > merge -c <commit> ...  to take the message from commit and open editor to edit
>> > merge -m "<message>" ... to take the message from the quoted test
>> > merge ... to merge and open commit editor with default message
>
> I will probably implement -c, but not -m, and will handle the absence of
> the -C and -c options to construct a default merge message which can then
> be edited.
>
> The -m option just opens such a can of worms with dequoting, that's why I
> do not want to do that.
>

I agree, I don't see a need for "-m".

> BTW I am still trying to figure out how to present the oneline of the
> commit to merge (which is sometimes really helpful because the label might
> be less than meaningful) while *still* allowing for octopus merges.
>
> So far, what I have is this:
>
>         merge <original> <to-merge> <oneline>
>
> and for octopus:
>
>         merge <original> "<to-merge> <to-merge2>..." <oneline>...
>
> I think with the -C syntax, it would become something like
>
>         merge -C <original> <to-merge> # <oneline>
>

I like this, especially given you added the "#" for one of the other
new commands as well, (reset I think?)

> and
>
>         merge -C <original> <to-merge> <to-merge2>...
>         # Merging: <oneline>
>         # Merging: <oneline2>
>         # ...
>

I really like this, since you can show each oneline for all the
to-merges for an octopus.

> The only qualm I have about this is that `#` really *is* a valid ref name.
> (Seriously, it is...). So that would mean that I'd have to disallow `#`
> as a label specifically.
>
> Thoughts?
>

I think it's fine to disallow # as a label.

Thanks,
Jake

> Ciao,
> Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-01-18 15:35 ` [PATCH 5/8] rebase: introduce the --recreate-merges option Johannes Schindelin
  2018-01-19 10:55   ` Eric Sunshine
  2018-01-23 20:22   ` Junio C Hamano
@ 2018-02-07  6:16   ` Sergey Organov
  2018-02-07  7:26     ` Jacob Keller
                       ` (2 more replies)
  2018-02-09  6:50   ` Sergey Organov
  3 siblings, 3 replies; 412+ messages in thread
From: Sergey Organov @ 2018-02-07  6:16 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Jacob Keller, Johannes Sixt

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

[...]

> +--recreate-merges::
> +	Recreate merge commits instead of flattening the history by replaying
> +	merges. Merge conflict resolutions or manual amendments to merge
> +	commits are not preserved.

I wonder why you guys still hold on replaying "merge-the-operation"
instead of replaying "merge-the-result"? The latter, the merge commit
itself, no matter how exactly it was created in the first place, is the
most valuable thing git keeps about the merge, and you silently drop it
entirely! OTOH, git keeps almost no information about
"merge-the-operation", so it's virtually impossible to reliably replay
the operation automatically, and yet you try to.

IMHO that was severe mistake in the original --preserve-merges, and you
bring with you to this new --recreate-merges... It's sad. Even more sad
as solution is already known for years:

    bc00341838a8faddcd101da9e746902994eef38a
    Author: Johannes Sixt <j6t@kdbg.org>
    Date:   Sun Jun 16 15:50:42 2013 +0200
    
        rebase -p --first-parent: redo merge by cherry-picking first-parent change

and it works like a charm.

-- Sergey


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-07  6:16   ` Sergey Organov
@ 2018-02-07  7:26     ` Jacob Keller
  2018-02-07  9:47       ` Sergey Organov
  2018-02-07  7:27     ` Johannes Sixt
  2018-02-07 17:36     ` Johannes Schindelin
  2 siblings, 1 reply; 412+ messages in thread
From: Jacob Keller @ 2018-02-07  7:26 UTC (permalink / raw)
  To: Sergey Organov
  Cc: Johannes Schindelin, Git mailing list, Junio C Hamano, Johannes Sixt

On Tue, Feb 6, 2018 at 10:16 PM, Sergey Organov <sorganov@gmail.com> wrote:
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>
> [...]
>
>> +--recreate-merges::
>> +     Recreate merge commits instead of flattening the history by replaying
>> +     merges. Merge conflict resolutions or manual amendments to merge
>> +     commits are not preserved.
>
> I wonder why you guys still hold on replaying "merge-the-operation"
> instead of replaying "merge-the-result"? The latter, the merge commit
> itself, no matter how exactly it was created in the first place, is the
> most valuable thing git keeps about the merge, and you silently drop it
> entirely! OTOH, git keeps almost no information about
> "merge-the-operation", so it's virtually impossible to reliably replay
> the operation automatically, and yet you try to.
>

I'm not sure I follow what you mean here?

You mean that you'd want this to actually attempt to re-create the
original merge including conflict resolutions by taking the contents
of the result?

How do you handle if that result has conflicts? What UX do you present
to the user to handle such conflicts? I don't think the normal 3-way
conflicts would even be possible in this case?

Thanks,
Jake

> IMHO that was severe mistake in the original --preserve-merges, and you
> bring with you to this new --recreate-merges... It's sad. Even more sad
> as solution is already known for years:
>
>     bc00341838a8faddcd101da9e746902994eef38a
>     Author: Johannes Sixt <j6t@kdbg.org>
>     Date:   Sun Jun 16 15:50:42 2013 +0200
>
>         rebase -p --first-parent: redo merge by cherry-picking first-parent change
>
> and it works like a charm.
>
> -- Sergey
>

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-07  6:16   ` Sergey Organov
  2018-02-07  7:26     ` Jacob Keller
@ 2018-02-07  7:27     ` Johannes Sixt
  2018-02-07 17:36     ` Johannes Schindelin
  2 siblings, 0 replies; 412+ messages in thread
From: Johannes Sixt @ 2018-02-07  7:27 UTC (permalink / raw)
  To: Sergey Organov, Johannes Schindelin; +Cc: git, Junio C Hamano, Jacob Keller

Am 07.02.2018 um 07:16 schrieb Sergey Organov:
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> [...]
> 
>> +--recreate-merges::
>> +	Recreate merge commits instead of flattening the history by replaying
>> +	merges. Merge conflict resolutions or manual amendments to merge
>> +	commits are not preserved.
> 
> I wonder why you guys still hold on replaying "merge-the-operation"
> instead of replaying "merge-the-result"? The latter, the merge commit
> itself, no matter how exactly it was created in the first place, is the
> most valuable thing git keeps about the merge, and you silently drop it
> entirely! OTOH, git keeps almost no information about
> "merge-the-operation", so it's virtually impossible to reliably replay
> the operation automatically, and yet you try to.

Very well put. I share your concerns.

-- Hannes

> IMHO that was severe mistake in the original --preserve-merges, and you
> bring with you to this new --recreate-merges... It's sad. Even more sad
> as solution is already known for years:
> 
>      bc00341838a8faddcd101da9e746902994eef38a
>      Author: Johannes Sixt <j6t@kdbg.org>
>      Date:   Sun Jun 16 15:50:42 2013 +0200
>      
>          rebase -p --first-parent: redo merge by cherry-picking first-parent change
> 
> and it works like a charm.
> 
> -- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-07  7:26     ` Jacob Keller
@ 2018-02-07  9:47       ` Sergey Organov
  0 siblings, 0 replies; 412+ messages in thread
From: Sergey Organov @ 2018-02-07  9:47 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Johannes Schindelin, Git mailing list, Junio C Hamano, Johannes Sixt

Jacob Keller <jacob.keller@gmail.com> writes:

> On Tue, Feb 6, 2018 at 10:16 PM, Sergey Organov <sorganov@gmail.com> wrote:
>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>>
>> [...]
>>
>>> +--recreate-merges::
>>> +     Recreate merge commits instead of flattening the history by replaying
>>> +     merges. Merge conflict resolutions or manual amendments to merge
>>> +     commits are not preserved.
>>
>> I wonder why you guys still hold on replaying "merge-the-operation"
>> instead of replaying "merge-the-result"? The latter, the merge commit
>> itself, no matter how exactly it was created in the first place, is the
>> most valuable thing git keeps about the merge, and you silently drop it
>> entirely! OTOH, git keeps almost no information about
>> "merge-the-operation", so it's virtually impossible to reliably replay
>> the operation automatically, and yet you try to.
>>
>
> I'm not sure I follow what you mean here?
>
> You mean that you'd want this to actually attempt to re-create the
> original merge including conflict resolutions by taking the contents
> of the result?

I mean just cherry-pick the merge the same way all other commits are
essentially cherry-picked during rebase. That's what Johannes Sixt did
in his patch I was reffering to.

> How do you handle if that result has conflicts? What UX do you present
> to the user to handle such conflicts? I don't think the normal 3-way
> conflicts would even be possible in this case?

No problem here. It goes exactly the same way as for non-merge commits
that are being rebased. You can try it right now using

$ git cherry-pick -m1 <merge_commit>

that will induce conflicts.

The (somewhat tricky) functional difference is only in recording correct
additional parents to the final commit, but that part is hidden from the
user.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-07  6:16   ` Sergey Organov
  2018-02-07  7:26     ` Jacob Keller
  2018-02-07  7:27     ` Johannes Sixt
@ 2018-02-07 17:36     ` Johannes Schindelin
  2018-02-07 22:58       ` Øyvind Rønningstad
  2018-02-09  6:11       ` Sergey Organov
  2 siblings, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-07 17:36 UTC (permalink / raw)
  To: Sergey Organov; +Cc: git, Junio C Hamano, Jacob Keller, Johannes Sixt

Hi,

On Wed, 7 Feb 2018, Sergey Organov wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> [...]
> 
> > +--recreate-merges::
> > +	Recreate merge commits instead of flattening the history by replaying
> > +	merges. Merge conflict resolutions or manual amendments to merge
> > +	commits are not preserved.
> 
> I wonder why you guys still hold on replaying "merge-the-operation"
> instead of replaying "merge-the-result"?

This misses the point of rebasing: you want to replay the changes.

> The latter, the merge commit itself, no matter how exactly it was
> created in the first place, is the most valuable thing git keeps about
> the merge, and you silently drop it entirely!

You miss another very crucial point. I don't blame you, as you certainly
have not used the Git garden shears for years.

Let me explain the scenario which comes up plenty of times in my work with
Git for Windows. We have a thicket of some 70 branches on top of git.git's
latest release. These branches often include fixup! and squash! commits
and even more complicated constructs that rebase cannot handle at all at
the moment, such as reorder-before! and reorder-after! (for commits that
really need to go into a different branch).

Even if you do not have such a complicated setup, it is quite possible
that you need to include a commit in your development that needs to be
dropped before contributing your work. Think e.g. removing the `-O2` flag
when compiling with GCC because GDB gets utterly confused with executables
compiled with `-O2` while single-stepping. This could be an initial commit
called `TO-DROP` or some such.

And guess what happens if you drop that `pick` line in your todo list and
then the `merge` command simply tries to re-create the original merge
commit's changes?

Exactly. The merge will become an evil merge, and will introduce that very
much not-wanted and therefore-dropped changes.

> OTOH, git keeps almost no information about "merge-the-operation", so
> it's virtually impossible to reliably replay the operation
> automatically, and yet you try to.

That is true. However, the intended use case is not to allow you to
recreate funny merges. Its use case is to allow you to recreate merges.

At a later stage, I might introduce support to detect `-s ours` merges,
because they are easy to detect. But even then, it will be an opt-in.

> IMHO that was severe mistake in the original --preserve-merges, and you
> bring with you to this new --recreate-merges... It's sad.

Please refrain from drawing this discussion into an emotional direction.
That is definitely not helpful.

> Even more sad as solution is already known for years:
> 
>     bc00341838a8faddcd101da9e746902994eef38a
>     Author: Johannes Sixt <j6t@kdbg.org>
>     Date:   Sun Jun 16 15:50:42 2013 +0200
>     
>         rebase -p --first-parent: redo merge by cherry-picking first-parent change
> 
> and it works like a charm.

It might work for you, as you probably used --preserve-merges, and dealt
with the fact that you could neither drop nor reorder commits.

So --preserve-merges --first-parent is probably what you were looking for.

Instead, --recreate-merges is all about allowing the same level of freedom
as with regular interactive rebases, but recreating the original commit
topology (and allowing to change it, too).

Therefore, I think that it would be even harmful to allow
--recreate-merges --first-parent *because it would cause evil merges*!

And I totally could see myself being vexed again about options that worked
perfectly well (just like --preserve-merges) being completely messed up by
allowing it to be combined with options *that they cannot work with* (just
like --preserve-merges --interactive, a *huge* mistake causing so many
annoying "bug" reports: I *never intended it that way because I knew it
would not work as users expect*).

So no, I do not think that --recreate-merges --first-parent is a good idea
at all. Unless you try to do that non-interactively only, *and disallow it
in interactive mode*. Because the entire point of the interactive rebase
is to allow reordering and dropping commits, in --recreate-merges even
moving, introducing and dropping merge commits. The --first-parent option
flies in the face of this idea.

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-07 17:36     ` Johannes Schindelin
@ 2018-02-07 22:58       ` Øyvind Rønningstad
  2018-02-07 23:31         ` Junio C Hamano
  2018-02-09  6:11       ` Sergey Organov
  1 sibling, 1 reply; 412+ messages in thread
From: Øyvind Rønningstad @ 2018-02-07 22:58 UTC (permalink / raw)
  To: Johannes Schindelin, Sergey Organov
  Cc: git, Junio C Hamano, Jacob Keller, Johannes Sixt

edit: Sending again, hopefully without HTML :). Sorry for spamming.

Hi, I think --recreate-merges is a very exciting feature.

I've also been puzzled by why we can't just pick merge commits directly
including
conflict resolutions, so allow me to join the discussion.

On Wed, Feb 7, 2018 at 6:36 PM, Johannes Schindelin <Johannes.Schindeli
n@gmx.de> wrote:
>
> Hi,
>
> [...]
>
> And guess what happens if you drop that `pick` line in your todo list
and
> then the `merge` command simply tries to re-create the original merge
> commit's changes?
>
> Exactly. The merge will become an evil merge, and will introduce that
very
> much not-wanted and therefore-dropped changes.

I think I understand. Evil merges happen when we change the branch
that is not the mainline..? Is there any reason why the following
wouldn't work?

Imagine rebase is about to pick a merge commit, and we have edited at
least one
commit in each branch to be merged.

1. apply patch mainline_orig..merge_orig
2. apply patch branch1_orig..branch1
...
N. apply patch branchN_orig..branchN
N+1. Commit merge

I do see complications, like the fact that steps 2-N can be done in any
order, with
possibly quite different results. Moving commits from one branch to
another might
not work very well. And what to do when you remove branches or create
new ones?

These problems might be prohibitive, but picking merge commits seems
like
something that should be possible to do.

>
> [...]
>
> So --preserve-merges --first-parent is probably what you were looking
for.

I want this as well :). I don't quite see the risk if it's not used
with --interactive.

> [...]
>
> So no, I do not think that --recreate-merges --first-parent is a good
idea
> at all. Unless you try to do that non-interactively only, *and
disallow it
> in interactive mode*. Because the entire point of the interactive
rebase
> is to allow reordering and dropping commits, in --recreate-merges
even
> moving, introducing and dropping merge commits. The --first-parent
option
> flies in the face of this idea.

FWIW I'd be totally fine with disallowing it in --interactive. It would
be incredibly useful 
e.g. with pull --rebase in merge-based workflows.

BTW what is the difference between --recreate-merges and --preserve-
merges when
--interactive is not present? I apologize if you have explained this
somewhere
else in the patch series.

>
> Ciao,
> Johannes

Thanks,
Øyvind

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-07 22:58       ` Øyvind Rønningstad
@ 2018-02-07 23:31         ` Junio C Hamano
  2018-02-08 12:34           ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-02-07 23:31 UTC (permalink / raw)
  To: Øyvind Rønningstad
  Cc: Johannes Schindelin, Sergey Organov, git, Jacob Keller, Johannes Sixt

Øyvind Rønningstad <ronningstad@gmail.com> writes:

>> So no, I do not think that --recreate-merges --first-parent is a good
> idea
>> at all. Unless you try to do that non-interactively only, *and
> disallow it
>> in interactive mode*.

Correct.  If the original side branch has commits A, B and C, you
are rebuilding the topic to have only A and C but not B and then
recreate the merge of that rebuilt topic, then you absolutely do not
want "cherry-pick -m1" of the original merge when recreating the
merge, as that would resurrect the effect of having B.  The same
argument applies if you rebuilt the topic with A and C and then a
new commit D.  "cherry-pick -m1" of the original would do a wrong
thing.

When there is no such fixing up, "cherry-pick -m1" is the right
thing to do, though, so it probably makes sense to pick merges that
way when the side topic being merged consists of the same commits as
the original.  I do not think that the code structure in the topic
as posted makes it impossible (or unnecessarily hard) to give an
enhancement like that in the future as a follow-up series.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-07 23:31         ` Junio C Hamano
@ 2018-02-08 12:34           ` Johannes Schindelin
  2018-02-14  5:41             ` Sergey Organov
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-08 12:34 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Øyvind Rønningstad, Sergey Organov, git, Jacob Keller,
	Johannes Sixt

[-- Attachment #1: Type: text/plain, Size: 4062 bytes --]

Hi Junio,

On Wed, 7 Feb 2018, Junio C Hamano wrote:

> Øyvind Rønningstad <ronningstad@gmail.com> writes:
> 
> >> So no, I do not think that --recreate-merges --first-parent is a good
> > idea
> >> at all. Unless you try to do that non-interactively only, *and
> > disallow it
> >> in interactive mode*.
> 
> Correct.  If the original side branch has commits A, B and C, you
> are rebuilding the topic to have only A and C but not B and then
> recreate the merge of that rebuilt topic, then you absolutely do not
> want "cherry-pick -m1" of the original merge when recreating the
> merge, as that would resurrect the effect of having B.  The same
> argument applies if you rebuilt the topic with A and C and then a
> new commit D.  "cherry-pick -m1" of the original would do a wrong
> thing.
> 
> When there is no such fixing up, "cherry-pick -m1" is the right
> thing to do, though, so it probably makes sense to pick merges that
> way when the side topic being merged consists of the same commits as
> the original.

Please note that there are a lot of conditions of "fixing up". A lot more
than just dropping, reordering or adding `pick`s.

> I do not think that the code structure in the topic as posted makes it
> impossible (or unnecessarily hard) to give an enhancement like that in
> the future as a follow-up series.

Just to give you one concrete example: when I recently rebased some
patches (no reording or dropping involved here!) and one of the picks
failed with merge conflicts, I realized that that particular commit
introduced incorrect formatting and fixed that right away (verifying that
no other commits introduced incorrect formatting, of course).

With your new cute idea to magically cherry-pick -m1, this change would
have been magically dropped from the subsequent merge commits!

And let me pick a bit on the statement "I do not think that ... makes it
impossible (or unnecessarily hard) ...": I absolutely agree. I absolutely
agree that it is not impossible or unnecessarily hard to introduce
features *that are confusing the users because they are inconsistent with
the expectations how such a command should operate*.

So the question is not so much whether we can introduce a feature that
makes no sense. Of course we can, we are decent software developers.

The question is: will this be confusing, inconsistent behavior that
violates the Principle of Least Surprise?

In that respect, introducing conditional code that would `cherry-pick -m1`
when the todo list is unchanged so far (and only then) is an absolute no,
no, no. It would be all three: confusing, inconsistent and violating the
Principle of Least Surprise.

So how about introducing support for `--recreate-merges --first-parent`
and allowing to combine it with `--interactive`? Also violating all three.

I can see how you *could* argue that `--recreate-merges --first-parent` is
a Good Thing. I really can. It would even recreate evil merges.

But in interactive mode?

Nope. It would cause all kind of pain, not the least on me, because I know
how many people ask me about `--preserve-merges --interactive` and its
confusing and inconsistent behavior that violates the Principle of Least
Surprise.

So as long as y'all don't go anywhere near "oh, let's just introduce
--recreate-merges --first-parent and *then also support it in
--interactive because we can even if it hurts the user experience", I
agree that it could be a good follow-up patch series.

Taking a step back, I have to wonder, though, why we stumble over our feet
trying to cross bridges that are one farther than the one we currently
have to cross.

By now, it should be clear why the default mode of --recreate-merges
*cannot* be --first-parent.

And before --recreate-merges is said and done, we should maybe work on
getting it said and done, rather than musing about what comes next? I, for
one, would really like to focus my time on getting *this* patch series
reviewed and included.

Thanks,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-07 17:36     ` Johannes Schindelin
  2018-02-07 22:58       ` Øyvind Rønningstad
@ 2018-02-09  6:11       ` Sergey Organov
  2018-02-09  7:13         ` Johannes Sixt
  1 sibling, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-02-09  6:11 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Jacob Keller, Johannes Sixt

Hi,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> On Wed, 7 Feb 2018, Sergey Organov wrote:
>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> > +--recreate-merges::
>> > +	Recreate merge commits instead of flattening the history by replaying
>> > +	merges. Merge conflict resolutions or manual amendments to merge
>> > +	commits are not preserved.
>> 
>> I wonder why you guys still hold on replaying "merge-the-operation"
>> instead of replaying "merge-the-result"?
>
> This misses the point of rebasing: you want to replay the changes.

What this comment has to do with the statement to which it's supposed to
be a reply? Sounds like topic change to me. Please clarify if it isn't.

>
>> The latter, the merge commit itself, no matter how exactly it was
>> created in the first place, is the most valuable thing git keeps about
>> the merge, and you silently drop it entirely!
>
> You miss another very crucial point.

What was the first crucial point I miss? Do you rather agree that the
point you are replying to with this is very crucial one as well?

> I don't blame you, as you certainly have not used the Git garden
> shears for years.

Thanks a lot!

> Let me explain the scenario which comes up plenty of times in my work with
> Git for Windows. We have a thicket of some 70 branches on top of git.git's
> latest release. These branches often include fixup! and squash! commits
> and even more complicated constructs that rebase cannot handle at all at
> the moment, such as reorder-before! and reorder-after! (for commits that
> really need to go into a different branch).

I sympathize, but a solution that breaks even in simple cases can't be
used reliably to solve more complex problems, sorry. Being so deep
into your problems, I think you maybe just aren't seeing forest for the
trees [1].

> Even if you do not have such a complicated setup, it is quite possible
> that you need to include a commit in your development that needs to be
> dropped before contributing your work. Think e.g. removing the `-O2` flag
> when compiling with GCC because GDB gets utterly confused with executables
> compiled with `-O2` while single-stepping. This could be an initial commit
> called `TO-DROP` or some such.
>
> And guess what happens if you drop that `pick` line in your todo list and
> then the `merge` command simply tries to re-create the original merge
> commit's changes?
>
> Exactly. The merge will become an evil merge, and will introduce that very
> much not-wanted and therefore-dropped changes.

Okay, Houston, we've had a problem here.

I'm sure you'll be able to come-up with suitable solution once you start
to think about it positively, but automatic unguided silent re-merge is
still not the right answer, for the same reason of distortion of user
changes.

As for "evil merges"... I don't want to get too far from original
subject to even start discussing this.

>> OTOH, git keeps almost no information about "merge-the-operation", so
>> it's virtually impossible to reliably replay the operation
>> automatically, and yet you try to.
>
> That is true. However, the intended use case is not to allow you to
> recreate funny merges. Its use case is to allow you to recreate
> merges.

Then it at least should behave accordingly, e.g., stop after every such
occurrence, for user assistance. As an example, see what rerere does
when it fires, even though it's much more reliable than this blind
re-merge.

But the actual problem here is that almost any merge but those made with
pure "git merge", no options, no conflicts, no edits, no nothing,
becomes "funny" and is being destroyed, sometimes in a weird way, by
silently creating something different instead of original.

> At a later stage, I might introduce support to detect `-s ours` merges,
> because they are easy to detect. But even then, it will be an opt-in.

So you are going to fix one particular case that is "easy to detect"
(and fix). Does it mean you do realize it's a problem, but fail to see that
it's _fundamental_ problem with current approach?

I think you start from the wrong end. I think that any merge should be
made reproducible first (with possible guidance from the user when
required, as usual), and then advanced features for complex history
tweaking should come, not the other way around.

I feel that any solution that fails to exactly reproduce original
history, unless it is to be actually changed, is flawed, and we will
continue to hit our heads on sharp corners like "merge -s ours", most of
which will be not that simple to detect and fix, for an unforeseeable
future.

>
>> IMHO that was severe mistake in the original --preserve-merges, and you
>> bring with you to this new --recreate-merges... It's sad.
>
> Please refrain from drawing this discussion into an emotional direction.
> That is definitely not helpful.

I don't actually blame anybody for that original implementation in the
first place. It was made based on the actual needs, and that's perfectly
fine with me, however quick-and-dirty it was. Keeping it quick-and-dirty
for years isn't that fine though.

And I do get somewhat emotional seeing this, all right. Look what you
did. We had one pet that strikes badly (with corresponding warning label
put on it in the manual), and now we have another one that would strike
as bad! I'm even afraid that I'm unfortunately not that young anymore to
get /properly/ emotional about it.

>> Even more sad as solution is already known for years:
>> 
>>     bc00341838a8faddcd101da9e746902994eef38a
>>     Author: Johannes Sixt <j6t@kdbg.org>
>>     Date:   Sun Jun 16 15:50:42 2013 +0200
>>     
>>         rebase -p --first-parent: redo merge by cherry-picking first-parent change
>> 
>> and it works like a charm.
>
> It might work for you, as you probably used --preserve-merges, and dealt
> with the fact that you could neither drop nor reorder commits.
>
> So --preserve-merges --first-parent is probably what you were looking
> for.

No. What I'm looking for is for my history to be kept as intact as
possible during rebase, unless I explicitly ask to change it, be it
--preserve-merges, or --recreate-merges, interactive or not. Is it too
much to ask for?

And no, I don't think --preserve-merges --first-parent is the right
answer either, in general, even though it did suit most of my purposes
indeed (in fact I simply patched --preserve-merges in my local git).

> Instead, --recreate-merges is all about allowing the same level of
> freedom as with regular interactive rebases, but recreating the
> original commit topology (and allowing to change it, too).

That's a very good thing and a very nice job as a whole, sure! If it
weren't I'd not even bother to raise this topic. But provided you
realize what problem "--preserve-merges --first-parent" would solve for
non-interactive use, you should realize that you have exactly the same
problem unsolved with the new --recreate-merges.

> Therefore, I think that it would be even harmful to allow
> --recreate-merges --first-parent *because it would cause evil merges*!

Once again, for me it seems you are thinking about it from the wrong
end, and this indeed won't work, but for different reasons than you
think.

[And please, stop frightening us with those "evil merge" thingy!]

> And I totally could see myself being vexed again about options that worked
> perfectly well (just like --preserve-merges) being completely messed up by
> allowing it to be combined with options *that they cannot work with* (just
> like --preserve-merges --interactive, a *huge* mistake causing so many
> annoying "bug" reports: I *never intended it that way because I knew it
> would not work as users expect*).

IMHO it's a minor problem. At least there was a warning there in the
manual, and nobody actually claimed reliable support for the feature.

> So no, I do not think that --recreate-merges --first-parent is a good idea
> at all. Unless you try to do that non-interactively only, *and disallow it
> in interactive mode*. Because the entire point of the interactive rebase
> is to allow reordering and dropping commits, in --recreate-merges even
> moving, introducing and dropping merge commits. The --first-parent option
> flies in the face of this idea.

Forget about --first-parent, please! The patch that introduced it was
only meant to show the general way of replaying a merge. Actual
suggestion is to implement proper support right with --preserve-merges,
or call it --recreate-merges, or --keep-topology, or
--no-flatten-history, whatever [2].

-- Sergey

[1] Actually I wonder how, with such a complex setup, you seem to never
step onto the problem I regularly have? Don't you ever have sub-topic
branches on top of those 70? How do you ensure none of the merges you
are rebasing are "funny", as you call them?

[2] Proper implementation of this will likely need some tweaks to the
todo list, such as "rm" command for a commit to be removed, explicit
distinguish between "merge-redo" and "merge-pick" (or even "merge-dwim")
in the list, or something like that. But to start actually thinking
about implementation, we need to agree that silent dropping user changes
is a huge problem, be it "evil merge" or something else that sounds even
more "dangerous".

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-01-18 15:35 ` [PATCH 5/8] rebase: introduce the --recreate-merges option Johannes Schindelin
                     ` (2 preceding siblings ...)
  2018-02-07  6:16   ` Sergey Organov
@ 2018-02-09  6:50   ` Sergey Organov
  2018-02-10 23:06     ` Johannes Schindelin
  3 siblings, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-02-09  6:50 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Jacob Keller

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

[...]

> With this patch, the goodness of the Git garden shears comes to `git
> rebase -i` itself. Passing the `--recreate-merges` option will generate
> a todo list that can be understood readily, and where it is obvious
> how to reorder commits. New branches can be introduced by inserting
> `label` commands and calling `merge - <label> <oneline>`. And once this
> mode has become stable and universally accepted, we can deprecate the
> design mistake that was `--preserve-merges`.

This doesn't explain why you introduced this new --recreate-merges. Why
didn't you rather fix --preserve-merges to generate and use new todo
list format?

It doesn't seem likely that todo list created by one Git version is to
be ever used by another, right? Is there some hidden reason here? Some
tools outside of Git that use old todo list format, maybe?

Then, if new option indeed required, please look at the resulting manual:

--recreate-merges::
	Recreate merge commits instead of flattening the history by replaying
	merges. Merge conflict resolutions or manual amendments to merge
	commits are not preserved.

-p::
--preserve-merges::
	Recreate merge commits instead of flattening the history by replaying
	commits a merge commit introduces. Merge conflict resolutions or manual
	amendments to merge commits are not preserved.


Don't you think more explanations are needed there in the manual on
why do we have 2 separate options with almost the same yet subtly
different description? Is this subtle difference even important? How?

I also have trouble making sense of "Recreate merge commits instead of
flattening the history by replaying merges." Is it "<Recreate merge
commits by replaying merges> instead of <flattening the history>" or is it
rather "<Recreate merge commits> instead of <flattening the history by
replaying merges>?

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-09  6:11       ` Sergey Organov
@ 2018-02-09  7:13         ` Johannes Sixt
  2018-02-11 10:16           ` Jacob Keller
  2018-02-12  7:38           ` Sergey Organov
  0 siblings, 2 replies; 412+ messages in thread
From: Johannes Sixt @ 2018-02-09  7:13 UTC (permalink / raw)
  To: Sergey Organov; +Cc: Johannes Schindelin, git, Junio C Hamano, Jacob Keller

Am 09.02.2018 um 07:11 schrieb Sergey Organov:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> Let me explain the scenario which comes up plenty of times in my work with
>> Git for Windows. We have a thicket of some 70 branches on top of git.git's
>> latest release. These branches often include fixup! and squash! commits
>> and even more complicated constructs that rebase cannot handle at all at
>> the moment, such as reorder-before! and reorder-after! (for commits that
>> really need to go into a different branch).
> 
> I sympathize, but a solution that breaks even in simple cases can't be
> used reliably to solve more complex problems, sorry. Being so deep
> into your problems, I think you maybe just aren't seeing forest for the
> trees [1].

Hold your horses! Dscho has a point here. --preserve-merges 
--first-parent works only as long as you don't tamper with the side 
branches. If you make changes in the side branches during the same 
rebase operation, this --first-parent mode would undo that change. (And, 
yes, its result would be called an "evil merge", and that scary name 
_should_ frighten you!)

-- Hannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-01-23 20:22   ` Junio C Hamano
@ 2018-02-10 19:31     ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-10 19:31 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Jacob Keller

Hi Junio,

On Tue, 23 Jan 2018, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> > index 8a861c1e0d6..1d061373288 100644
> > --- a/Documentation/git-rebase.txt
> > +++ b/Documentation/git-rebase.txt
> > @@ -368,6 +368,11 @@ The commit list format can be changed by setting the configuration option
> >  rebase.instructionFormat.  A customized instruction format will automatically
> >  have the long commit hash prepended to the format.
> >  
> > +--recreate-merges::
> > +	Recreate merge commits instead of flattening the history by replaying
> > +	merges. Merge conflict resolutions or manual amendments to merge
> > +	commits are not preserved.
> > +
> 
> It is sensible to postpone tackling "evil merges" in this initial
> iteration of the series, and "manual amendments ... not preserved"
> is a reasonable thing to document.  But do we want to say a bit more
> about conflicting merges?  "conflict resolutions ... not preserved"
> sounds as if it does not stop and instead record the result with
> conflict markers without even letting rerere to kick in, which
> certainly is not the impression you wanted to give to the readers.
> 
> I am imagining that it will stop and give control back to the end
> user just like a conflicted "pick" would, and allow "rebase
> --continue" to record resolution from the working tree, and just
> like conflicted "pick", it would allow rerere() to help end users
> recall previous resolution.

This is my current version:

--recreate-merges[=(rebase-cousins|no-rebase-cousins)]::
        Recreate merge commits instead of flattening the history by replaying
        merges. Merge conflict resolutions or manual amendments to merge
        commits are not recreated automatically, but have to be recreated
        manually.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 02/10] sequencer: introduce new commands to reset the revision
  2018-01-30  8:06     ` Eric Sunshine
@ 2018-02-10 20:58       ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-10 20:58 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Phillip Wood

Hi Eric,

On Tue, 30 Jan 2018, Eric Sunshine wrote:

> On Mon, Jan 29, 2018 at 5:54 PM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > [...]
> > This commit implements the commands to label, and to reset to, given
> > revisions. The syntax is:
> >
> >         label <name>
> >         reset <name>
> > [...]
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> > diff --git a/sequencer.c b/sequencer.c
> > @@ -1253,7 +1266,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
> >                 if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
> >                         item->command = i;
> >                         break;
> > -               } else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
> > +               } else if ((bol + 1 == eol || bol[1] == ' ') &&
> > +                          *bol == todo_command_info[i].c) {
> 
> This adds support for commands which have no arguments, however, now
> that the "bud" command has been retired, this can go away too, right?

Good point. Fixed.

> >                         bol++;
> >                         item->command = i;
> >                         break;
> > @@ -1919,6 +1934,144 @@ static int do_exec(const char *command_line)
> > +static int safe_append(const char *filename, const char *fmt, ...)
> > +{
> > +       va_list ap;
> > +       struct lock_file lock = LOCK_INIT;
> > +       int fd = hold_lock_file_for_update(&lock, filename, 0);
> > +       struct strbuf buf = STRBUF_INIT;
> > +
> > +       if (fd < 0)
> > +               return error_errno(_("could not lock '%s'"), filename);
> 
> Minor: unable_to_lock_message() can provide a more detailed
> explanation of the failure.

That is true. Due to its awkward signature (returning void, using a
strbuf), it would add a whopping 4 lines, too.

There is a better solution, though, adding only one line: passing
LOCK_REPORT_ON_ERROR as flag to hold_lock_file_for_update().

> > +
> > +       if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT)
> > +               return error_errno(_("could not read '%s'"), filename);
> > +       strbuf_complete(&buf, '\n');
> > +       va_start(ap, fmt);
> > +       strbuf_vaddf(&buf, fmt, ap);
> > +       va_end(ap);
> 
> Would it make sense to also
> 
>     strbuf_complete(&buf, '\n')
> 
> here, as well, to be a bit more robust against lazy callers?

I'd rather not make that assumption. It *may* be true that the current
sole user wants the last line of the file to end in a newline. I try to
design my code for maximum reusability, though. And who is to say whether
my next use case for the safe_append() function wants the semantics you
suggest, if it wants to append less than entire lines at a time, maybe?
Let's not optimize prematurely, okay?

> > +
> > +       if (write_in_full(fd, buf.buf, buf.len) < 0) {
> > +               rollback_lock_file(&lock);
> > +               return error_errno(_("could not write to '%s'"), filename);
> 
> Reading lockfile.h & tempfile.c, I see that rollback_lock_file()
> clobbers write_in_full()'s errno before error_errno() is called.

True. Fixed.

I also fixed the code from where I copy-edited this pattern (increasing
the patch series by yet another patch).

> > +       }
> > +       if (commit_lock_file(&lock) < 0) {
> > +               rollback_lock_file(&lock);
> > +               return error(_("failed to finalize '%s'"), filename);
> > +       }
> > +
> > +       return 0;
> > +}
> > +
> > +static int do_reset(const char *name, int len)
> > +{
> > +       [...]
> > +       strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> > +       if (get_oid(ref_name.buf, &oid) &&
> > +           get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
> > +               error(_("could not read '%s'"), ref_name.buf);
> 
> Checking my understanding: The two get_oid() calls allow the argument
> to 'reset' to be a label created with the 'label' command or any other
> way to name an object, right? If so, then I wonder if the error
> invocation should instead be:
> 
>     error(_("could not read '%.*s'"), len, name);

I would rather give the preferred form: refs/rewritten/<label>.

The main reason this code falls back to getting the OID of `<label>`
directly is to support the `no-rebase-cousins` code: in that mode, topic
branches may be based on commits other than the one labeled `onto`, but
the original, unchanged one. In this case, we have no way of labeling the
base commit, and therefore use a unique abbreviation of that base commit's
OID.

But this is really a very special use case, and the more common use case
should be the one using refs/rewritten/<label>.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 02/10] sequencer: introduce new commands to reset therevision
  2018-01-31 18:02         ` [PATCH v2 02/10] sequencer: introduce new commands to reset therevision Phillip Wood
@ 2018-02-10 21:49           ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-10 21:49 UTC (permalink / raw)
  To: Phillip Wood
  Cc: Stefan Beller, git, Junio C Hamano, Jacob Keller, Philip Oakley,
	Eric Sunshine

Hi Phillip,

On Wed, 31 Jan 2018, Phillip Wood wrote:

> On 31/01/18 13:21, Johannes Schindelin wrote:
> > 
> > On Tue, 30 Jan 2018, Stefan Beller wrote:
> > 
> >> On Mon, Jan 29, 2018 at 2:54 PM, Johannes Schindelin
> >> <johannes.schindelin@gmx.de> wrote:
> >>> @@ -116,6 +118,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
> >>>  static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
> >>>  static GIT_PATH_FUNC(rebase_path_rewritten_pending,
> >>>         "rebase-merge/rewritten-pending")
> >>> +
> >>> +/*
> >>> + * The path of the file listing refs that need to be deleted after the rebase
> >>> + * finishes. This is used by the `merge` command.
> >>> + */
> > 
> > Whoops. The comment "This is used by the `merge` command`" is completely
> > wrong. Will fix.
> > 
> >> So this file contains (label -> commit),
> > 
> > Only `label`. No `commit`.
> > 
> >> which is appended in do_label, it uses refs to store the commits in
> >> refs/rewritten.  We do not have to worry about the contents of that file
> >> getting too long, or label re-use, because the directory containing all
> >> these helper files will be deleted upon successful rebase in
> >> `sequencer_remove_state()`.
> > 
> > Yes.
> >
> It might be a good idea to have 'git rebase --abort' delete the refs as
> well as the file though

That makes sense. I made it so.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-09  6:50   ` Sergey Organov
@ 2018-02-10 23:06     ` Johannes Schindelin
  2018-02-12  4:58       ` Sergey Organov
  2018-02-12  5:22       ` Sergey Organov
  0 siblings, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-10 23:06 UTC (permalink / raw)
  To: Sergey Organov; +Cc: git, Junio C Hamano, Jacob Keller

Hi Sergey,

On Fri, 9 Feb 2018, Sergey Organov wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> [...]
> 
> > With this patch, the goodness of the Git garden shears comes to `git
> > rebase -i` itself. Passing the `--recreate-merges` option will generate
> > a todo list that can be understood readily, and where it is obvious
> > how to reorder commits. New branches can be introduced by inserting
> > `label` commands and calling `merge - <label> <oneline>`. And once this
> > mode has become stable and universally accepted, we can deprecate the
> > design mistake that was `--preserve-merges`.
> 
> This doesn't explain why you introduced this new --recreate-merges. Why
> didn't you rather fix --preserve-merges to generate and use new todo
> list format?

Because that would of course break existing users of --preserve-merges.

So why not --preserve-merges=v2? Because that would force me to maintain
--preserve-merges forever. And I don't want to.

> It doesn't seem likely that todo list created by one Git version is to
> be ever used by another, right?

No. But by scripts based on `git rebase -p`.

> Is there some hidden reason here? Some tools outside of Git that use old
> todo list format, maybe?

Exactly.

I did mention such a tool: the Git garden shears:

	https://github.com/git-for-windows/build-extra/blob/master/shears.sh

Have a look at it. It will inform the discussion.

> Then, if new option indeed required, please look at the resulting manual:
> 
> --recreate-merges::
> 	Recreate merge commits instead of flattening the history by replaying
> 	merges. Merge conflict resolutions or manual amendments to merge
> 	commits are not preserved.
> 
> -p::
> --preserve-merges::
> 	Recreate merge commits instead of flattening the history by replaying
> 	commits a merge commit introduces. Merge conflict resolutions or manual
> 	amendments to merge commits are not preserved.

As I stated in the cover letter, there are more patches lined up after
this patch series.

Have a look at https://github.com/git/git/pull/447, especially the latest
commit in there which is an early version of the deprecation I intend to
bring about.

Also, please refrain from saying things like... "Don't you think ..."

If you don't like the wording, I wold much more appreciate it if a better
alternative was suggested.

> Don't you think more explanations are needed there in the manual on
> why do we have 2 separate options with almost the same yet subtly
> different description? Is this subtle difference even important? How?
> 
> I also have trouble making sense of "Recreate merge commits instead of
> flattening the history by replaying merges." Is it "<Recreate merge
> commits by replaying merges> instead of <flattening the history>" or is it
> rather "<Recreate merge commits> instead of <flattening the history by
> replaying merges>?

The documentation of the --recreate-merges option is not meant to explain
the difference to --preserve-merges. It is meant to explain the difference
to regular `git rebase -i`, which flattens the commit history into a
single branch without merge commits (in fact, all merge commits are simply
ignored).

And I would rather not start to describe the difference between
--recreate-merges and --preserve-merges because I want to deprecate the
latter, and describing the difference as I get the sense is your wish
would simply mean more work because it would have to be added and then
removed again.

If you still think it would be a good idea to describe the difference
between --recreate-merges and --preserve-merges, then please provide a
suggestion, preferably in the form of a patch, that adds appropriate
paragraphs to *both* options' documentation, so that your proposal can be
discussed properly.

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v3 00/12] rebase -i: offer to recreate merge commits
  2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
                     ` (11 preceding siblings ...)
  2018-01-30 21:36   ` Junio C Hamano
@ 2018-02-11  0:09   ` Johannes Schindelin
  2018-02-11  0:10     ` [PATCH v3 01/12] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
                       ` (13 more replies)
  12 siblings, 14 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-11  0:09 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Once upon a time, I dreamt of an interactive rebase that would not
flatten branch structure, but instead recreate the commit topology
faithfully.

My original attempt was --preserve-merges, but that design was so
limited that I did not even enable it in interactive mode.

Subsequently, it *was* enabled in interactive mode, with the predictable
consequences: as the --preserve-merges design does not allow for
specifying the parents of merge commits explicitly, all the new commits'
parents are defined *implicitly* by the previous commit history, and
hence it is *not possible to even reorder commits*.

This design flaw cannot be fixed. Not without a complete re-design, at
least. This patch series offers such a re-design.

Think of --recreate-merges as "--preserve-merges done right". It
introduces new verbs for the todo list, `label`, `reset` and `merge`.
For a commit topology like this:

            A - B - C
              \   /
                D

the generated todo list would look like this:

            # branch D
            pick 0123 A
            label branch-point
            pick 1234 D
            label D

            reset branch-point
            pick 2345 B
            merge -C 3456 D # C

There are more patches in the pipeline, based on this patch series, but
left for later in the interest of reviewable patch series: one mini
series to use the sequencer even for `git rebase -i --root`, and another
one to add support for octopus merges to --recreate-merges.

Changes since v2:

- fixed the incorrect comment for rebase_path_refs_to_delete.

- we now error out properly if read_cache_unmerged() fails.

- if there are unresolved merge conflicts, the `reset` command now errors out
  (even if the current design should not allow for such a scenario to occur).

- a diff hunk that was necessary to support `bud` was dropped from 2/10.

- changed all `rollback_lock_file(); return error_errno(...);` patterns to
  first show the errors (i.e. using the correct errno). This added 1/11.

- The temporary refs are now also cleaned up upon `git rebase --abort`.

- Reworked the entire patch series to support

	merge -C <commit> <tip> # <oneline>

  instead of the previous `merge <commit> <tip> <oneline>`.

- Dropped the octopus part of the description of the `merge` command in
  the usage at the bottom of the todo list, as it is subject to change.

- The autosquash handling was not elegant, and cuddled into the same
  commit as the post-rewrite changes. Now, the autosquash handling is a
  lot more elegant, and a separate introductory patch (as it arguably
  improves the current code on its own).


Johannes Schindelin (11):
  sequencer: avoid using errno clobbered by rollback_lock_file()
  sequencer: make rearrange_squash() a bit more obvious
  sequencer: introduce new commands to reset the revision
  sequencer: introduce the `merge` command
  sequencer: fast-forward merge commits, if possible
  rebase-helper --make-script: introduce a flag to recreate merges
  rebase: introduce the --recreate-merges option
  sequencer: make refs generated by the `label` command worktree-local
  sequencer: handle post-rewrite for merge commands
  pull: accept --rebase=recreate to recreate the branch topology
  rebase -i: introduce --recreate-merges=[no-]rebase-cousins

Stefan Beller (1):
  git-rebase--interactive: clarify arguments

 Documentation/config.txt               |   8 +
 Documentation/git-pull.txt             |   5 +-
 Documentation/git-rebase.txt           |  14 +-
 builtin/pull.c                         |  14 +-
 builtin/rebase--helper.c               |  13 +-
 builtin/remote.c                       |   2 +
 contrib/completion/git-completion.bash |   4 +-
 git-rebase--interactive.sh             |  22 +-
 git-rebase.sh                          |  16 +
 refs.c                                 |   3 +-
 sequencer.c                            | 736 ++++++++++++++++++++++++++++++++-
 sequencer.h                            |   7 +
 t/t3430-rebase-recreate-merges.sh      | 208 ++++++++++
 13 files changed, 1021 insertions(+), 31 deletions(-)
 create mode 100755 t/t3430-rebase-recreate-merges.sh


base-commit: 5be1f00a9a701532232f57958efab4be8c959a29
Published-As: https://github.com/dscho/git/releases/tag/recreate-merges-v3
Fetch-It-Via: git fetch https://github.com/dscho/git recreate-merges-v3

Interdiff vs v2:
 diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
 index 5e21e4cf269..e199fe1cca5 100644
 --- a/git-rebase--interactive.sh
 +++ b/git-rebase--interactive.sh
 @@ -164,10 +164,10 @@ x, exec <commit> = run command (the rest of the line) using shell
  d, drop <commit> = remove commit
  l, label <label> = label current HEAD with a name
  t, reset <label> = reset HEAD to a label
 -m, merge <original-merge-commit> ( <label> | \"<label>...\" ) [<oneline>]
 +m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
  .       create a merge commit using the original merge commit's
 -.       message (or the oneline, if "-" is given). Use a quoted
 -.       list of commits to be merged for octopus merges.
 +.       message (or the oneline, if no original merge commit was
 +.       specified). Use -c <commit> to reword the commit message.
  
  These lines can be re-ordered; they are executed from top to bottom.
  " | git stripspace --comment-lines >>"$todo"
 diff --git a/sequencer.c b/sequencer.c
 index cd2f2ae5d53..c877432d7b4 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -123,7 +123,7 @@ static GIT_PATH_FUNC(rebase_path_rewritten_pending,
  
  /*
   * The path of the file listing refs that need to be deleted after the rebase
 - * finishes. This is used by the `merge` command.
 + * finishes. This is used by the `label` command to record the need for cleanup.
   */
  static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
  
 @@ -206,18 +206,33 @@ static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
  
  int sequencer_remove_state(struct replay_opts *opts)
  {
 -	struct strbuf dir = STRBUF_INIT;
 +	struct strbuf buf = STRBUF_INIT;
  	int i;
  
 +	if (strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
 +		char *p = buf.buf;
 +		while (*p) {
 +			char *eol = strchr(p, '\n');
 +			if (eol)
 +				*eol = '\0';
 +			if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
 +				warning(_("could not delete '%s'"), p);
 +			if (!eol)
 +				break;
 +			p = eol + 1;
 +		}
 +	}
 +
  	free(opts->gpg_sign);
  	free(opts->strategy);
  	for (i = 0; i < opts->xopts_nr; i++)
  		free(opts->xopts[i]);
  	free(opts->xopts);
  
 -	strbuf_addstr(&dir, get_dir(opts));
 -	remove_dir_recursively(&dir, 0);
 -	strbuf_release(&dir);
 +	strbuf_reset(&buf);
 +	strbuf_addstr(&buf, get_dir(opts));
 +	remove_dir_recursively(&buf, 0);
 +	strbuf_release(&buf);
  
  	return 0;
  }
 @@ -307,12 +322,14 @@ static int write_message(const void *buf, size_t len, const char *filename,
  	if (msg_fd < 0)
  		return error_errno(_("could not lock '%s'"), filename);
  	if (write_in_full(msg_fd, buf, len) < 0) {
 +		error_errno(_("could not write to '%s'"), filename);
  		rollback_lock_file(&msg_file);
 -		return error_errno(_("could not write to '%s'"), filename);
 +		return -1;
  	}
  	if (append_eol && write(msg_fd, "\n", 1) < 0) {
 +		error_errno(_("could not write eol to '%s'"), filename);
  		rollback_lock_file(&msg_file);
 -		return error_errno(_("could not write eol to '%s'"), filename);
 +		return -1;
  	}
  	if (commit_lock_file(&msg_file) < 0) {
  		rollback_lock_file(&msg_file);
 @@ -781,6 +798,7 @@ enum todo_command {
  	TODO_LABEL,
  	TODO_RESET,
  	TODO_MERGE,
 +	TODO_MERGE_AND_EDIT,
  	/* commands that do nothing but are counted for reporting progress */
  	TODO_NOOP,
  	TODO_DROP,
 @@ -802,6 +820,7 @@ static struct {
  	{ 'l', "label" },
  	{ 't', "reset" },
  	{ 'm', "merge" },
 +	{ 0, "merge" }, /* MERGE_AND_EDIT */
  	{ 0,   "noop" },
  	{ 'd', "drop" },
  	{ 0,   NULL }
 @@ -1270,8 +1289,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
  		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
  			item->command = i;
  			break;
 -		} else if ((bol + 1 == eol || bol[1] == ' ') &&
 -			   *bol == todo_command_info[i].c) {
 +		} else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
  			bol++;
  			item->command = i;
  			break;
 @@ -1305,21 +1323,30 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
  		return 0;
  	}
  
 -	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 -	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
 -	item->arg_len = (int)(eol - item->arg);
 -
 -	if (item->command == TODO_MERGE && *bol == '-' &&
 -	    bol + 1 == end_of_object_name) {
 -		item->commit = NULL;
 -		return 0;
 +	if (item->command == TODO_MERGE) {
 +		if (skip_prefix(bol, "-C", &bol))
 +			bol += strspn(bol, " \t");
 +		else if (skip_prefix(bol, "-c", &bol)) {
 +			bol += strspn(bol, " \t");
 +			item->command = TODO_MERGE_AND_EDIT;
 +		} else {
 +			item->command = TODO_MERGE_AND_EDIT;
 +			item->commit = NULL;
 +			item->arg = bol;
 +			item->arg_len = (int)(eol - bol);
 +			return 0;
 +		}
  	}
  
 +	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
  	saved = *end_of_object_name;
  	*end_of_object_name = '\0';
  	status = get_oid(bol, &commit_oid);
  	*end_of_object_name = saved;
  
 +	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
 +	item->arg_len = (int)(eol - item->arg);
 +
  	if (status < 0)
  		return -1;
  
 @@ -1609,16 +1636,17 @@ static int save_head(const char *head)
  
  	fd = hold_lock_file_for_update(&head_lock, git_path_head_file(), 0);
  	if (fd < 0) {
 +		error_errno(_("could not lock HEAD"));
  		rollback_lock_file(&head_lock);
 -		return error_errno(_("could not lock HEAD"));
 +		return -1;
  	}
  	strbuf_addf(&buf, "%s\n", head);
  	written = write_in_full(fd, buf.buf, buf.len);
  	strbuf_release(&buf);
  	if (written < 0) {
 +		error_errno(_("could not write to '%s'"), git_path_head_file());
  		rollback_lock_file(&head_lock);
 -		return error_errno(_("could not write to '%s'"),
 -				   git_path_head_file());
 +		return -1;
  	}
  	if (commit_lock_file(&head_lock) < 0) {
  		rollback_lock_file(&head_lock);
 @@ -1948,11 +1976,12 @@ static int safe_append(const char *filename, const char *fmt, ...)
  {
  	va_list ap;
  	struct lock_file lock = LOCK_INIT;
 -	int fd = hold_lock_file_for_update(&lock, filename, 0);
 +	int fd = hold_lock_file_for_update(&lock, filename,
 +					   LOCK_REPORT_ON_ERROR);
  	struct strbuf buf = STRBUF_INIT;
  
  	if (fd < 0)
 -		return error_errno(_("could not lock '%s'"), filename);
 +		return -1;
  
  	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT)
  		return error_errno(_("could not read '%s'"), filename);
 @@ -1962,8 +1991,9 @@ static int safe_append(const char *filename, const char *fmt, ...)
  	va_end(ap);
  
  	if (write_in_full(fd, buf.buf, buf.len) < 0) {
 +		error_errno(_("could not write to '%s'"), filename);
  		rollback_lock_file(&lock);
 -		return error_errno(_("could not write to '%s'"), filename);
 +		return -1;
  	}
  	if (commit_lock_file(&lock) < 0) {
  		rollback_lock_file(&lock);
 @@ -1982,6 +2012,9 @@ static int do_label(const char *name, int len)
  	int ret = 0;
  	struct object_id head_oid;
  
 +	if (len == 1 && *name == '#')
 +		return error("Illegal label name: '%.*s'", len, name);
 +
  	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
  	strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
  
 @@ -2010,14 +2043,14 @@ static int do_label(const char *name, int len)
  	return ret;
  }
  
 -static int do_reset(const char *name, int len)
 +static int do_reset(const char *name, int len, struct replay_opts *opts)
  {
  	struct strbuf ref_name = STRBUF_INIT;
  	struct object_id oid;
  	struct lock_file lock = LOCK_INIT;
  	struct tree_desc desc;
  	struct tree *tree;
 -	struct unpack_trees_options opts;
 +	struct unpack_trees_options unpack_tree_opts;
  	int ret = 0, i;
  
  	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
 @@ -2037,16 +2070,18 @@ static int do_reset(const char *name, int len)
  		return -1;
  	}
  
 -	memset(&opts, 0, sizeof(opts));
 -	opts.head_idx = 1;
 -	opts.src_index = &the_index;
 -	opts.dst_index = &the_index;
 -	opts.fn = oneway_merge;
 -	opts.merge = 1;
 -	opts.update = 1;
 -	opts.reset = 1;
 +	memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
 +	unpack_tree_opts.head_idx = 1;
 +	unpack_tree_opts.src_index = &the_index;
 +	unpack_tree_opts.dst_index = &the_index;
 +	unpack_tree_opts.fn = oneway_merge;
 +	unpack_tree_opts.merge = 1;
 +	unpack_tree_opts.update = 1;
 +	unpack_tree_opts.reset = 1;
 +
 +	if (read_cache_unmerged())
 +		return error_resolve_conflict(_(action_name(opts)));
  
 -	read_cache_unmerged();
  	if (!fill_tree_descriptor(&desc, &oid)) {
  		error(_("failed to find tree of %s"), oid_to_hex(&oid));
  		rollback_lock_file(&lock);
 @@ -2055,7 +2090,7 @@ static int do_reset(const char *name, int len)
  		return -1;
  	}
  
 -	if (unpack_trees(1, &desc, &opts)) {
 +	if (unpack_trees(1, &desc, &unpack_tree_opts)) {
  		rollback_lock_file(&lock);
  		free((void *)desc.buffer);
  		strbuf_release(&ref_name);
 @@ -2083,7 +2118,7 @@ static int do_reset(const char *name, int len)
  }
  
  static int do_merge(struct commit *commit, const char *arg, int arg_len,
 -		    struct replay_opts *opts)
 +		    int run_commit_flags, struct replay_opts *opts)
  {
  	int merge_arg_len;
  	struct strbuf ref_name = STRBUF_INIT;
 @@ -2137,6 +2172,8 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  		strbuf_reset(&buf);
  
  		p += strspn(p, " \t");
 +		if (*p == '#' && isspace(p[1]))
 +			p += 1 + strspn(p + 1, " \t");
  		if (*p)
  			len = strlen(p);
  		else {
 @@ -2221,7 +2258,7 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  	}
  	rollback_lock_file(&lock);
  
 -	ret = run_git_commit(git_path_merge_msg(), opts, 0);
 +	ret = run_git_commit(git_path_merge_msg(), opts, run_commit_flags);
  	strbuf_release(&ref_name);
  
  	return ret;
 @@ -2413,10 +2450,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  		} else if (item->command == TODO_LABEL)
  			res = do_label(item->arg, item->arg_len);
  		else if (item->command == TODO_RESET)
 -			res = do_reset(item->arg, item->arg_len);
 -		else if (item->command == TODO_MERGE) {
 -			res = do_merge(item->commit,
 -				       item->arg, item->arg_len, opts);
 +			res = do_reset(item->arg, item->arg_len, opts);
 +		else if (item->command == TODO_MERGE ||
 +			 item->command == TODO_MERGE_AND_EDIT) {
 +			res = do_merge(item->commit, item->arg, item->arg_len,
 +				       item->command == TODO_MERGE_AND_EDIT ?
 +				       EDIT_MSG | VERIFY_MSG : 0, opts);
  			if (item->commit)
  				record_in_rewritten(&item->commit->object.oid,
  						    peek_command(todo_list, 1));
 @@ -2525,23 +2564,6 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  		}
  		apply_autostash(opts);
  
 -		strbuf_reset(&buf);
 -		if (strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0)
 -		    > 0) {
 -			char *p = buf.buf;
 -			while (*p) {
 -				char *eol = strchr(p, '\n');
 -				if (eol)
 -					*eol = '\0';
 -				if (delete_ref("(rebase -i) cleanup",
 -					       p, NULL, 0) < 0)
 -					warning(_("could not delete '%s'"), p);
 -				if (!eol)
 -					break;
 -				p = eol + 1;
 -			}
 -		}
 -
  		fprintf(stderr, "Successfully rebased and updated %s.\n",
  			head_ref.buf);
  
 @@ -2867,11 +2889,14 @@ static const char *label_oid(struct object_id *oid, const char *label,
  		}
  	} else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
  		    !get_oid_hex(label, &dummy)) ||
 +		   (len == 1 && *label == '#') ||
  		   hashmap_get_from_hash(&state->labels,
  					 strihash(label), label)) {
  		/*
  		 * If the label already exists, or if the label is a valid full
 -		 * OID, we append a dash and a number to make it unique.
 +		 * OID, or the label is a '#' (which we use as a separator
 +		 * between merge heads and oneline), we append a dash and a
 +		 * number to make it unique.
  		 */
  		struct strbuf *buf = &state->buf;
  
 @@ -2997,7 +3022,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
  				*(char *)p1 = '-';
  
  		strbuf_reset(&buf);
 -		strbuf_addf(&buf, "%s %s",
 +		strbuf_addf(&buf, "%s -C %s",
  			    cmd_merge, oid_to_hex(&commit->object.oid));
  
  		/* label the tip of merged branch */
 @@ -3012,7 +3037,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
  
  			strbuf_addstr(&buf, label_oid(oid, label.buf, &state));
  		}
 -		strbuf_addf(&buf, " %s", oneline.buf);
 +		strbuf_addf(&buf, " # %s", oneline.buf);
  
  		FLEX_ALLOC_STR(entry, string, buf.buf);
  		oidcpy(&entry->entry.oid, &commit->object.oid);
 @@ -3262,9 +3287,14 @@ int transform_todos(unsigned flags)
  					  short_commit_name(item->commit) :
  					  oid_to_hex(&item->commit->object.oid);
  
 +			if (item->command == TODO_MERGE)
 +				strbuf_addstr(&buf, " -C");
 +			else if (item->command == TODO_MERGE_AND_EDIT)
 +				strbuf_addstr(&buf, " -c");
 +
  			strbuf_addf(&buf, " %s", oid);
 -		} else if (item->command == TODO_MERGE)
 -			strbuf_addstr(&buf, " -");
 +		}
 +
  		/* add all the rest */
  		if (!item->arg_len)
  			strbuf_addch(&buf, '\n');
 @@ -3567,8 +3597,7 @@ int rearrange_squash(void)
  		struct subject2item_entry *entry;
  
  		next[i] = tail[i] = -1;
 -		if (item->command >= TODO_EXEC &&
 -		    (item->command != TODO_MERGE || !item->commit)) {
 +		if (!item->commit || item->command == TODO_DROP) {
  			subjects[i] = NULL;
  			continue;
  		}
 diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
 index ab51b584ff9..9a59f12b670 100755
 --- a/t/t3430-rebase-recreate-merges.sh
 +++ b/t/t3430-rebase-recreate-merges.sh
 @@ -59,8 +59,8 @@ pick B
  label second
  
  reset onto
 -merge H second
 -merge - onebranch Merge the topic branch 'onebranch'
 +merge -C H second
 +merge onebranch # Merge the topic branch 'onebranch'
  EOF
  
  test_cmp_graph () {
 @@ -106,8 +106,8 @@ test_expect_success 'generate correct todo list' '
  
  	reset branch-point # C
  	pick 12bd07b D
 -	merge 2051b56 E E
 -	merge 233d48a H H
 +	merge -C 2051b56 E # E
 +	merge -C 233d48a H # H
  
  	EOF
  
-- 
2.16.1.windows.1


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v3 01/12] sequencer: avoid using errno clobbered by rollback_lock_file()
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
@ 2018-02-11  0:10     ` Johannes Schindelin
  2018-02-11  0:10     ` [PATCH v3 02/12] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
                       ` (12 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-11  0:10 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

As pointed out in a review of the `--recreate-merges` patch series,
`rollback_lock_file()` clobbers errno. Therefore, we have to report the
error message that uses errno before calling said function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 4d3f60594cb..114db3b2775 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -296,12 +296,14 @@ static int write_message(const void *buf, size_t len, const char *filename,
 	if (msg_fd < 0)
 		return error_errno(_("could not lock '%s'"), filename);
 	if (write_in_full(msg_fd, buf, len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write to '%s'"), filename);
+		return -1;
 	}
 	if (append_eol && write(msg_fd, "\n", 1) < 0) {
+		error_errno(_("could not write eol to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write eol to '%s'"), filename);
+		return -1;
 	}
 	if (commit_lock_file(&msg_file) < 0) {
 		rollback_lock_file(&msg_file);
@@ -1584,16 +1586,17 @@ static int save_head(const char *head)
 
 	fd = hold_lock_file_for_update(&head_lock, git_path_head_file(), 0);
 	if (fd < 0) {
+		error_errno(_("could not lock HEAD"));
 		rollback_lock_file(&head_lock);
-		return error_errno(_("could not lock HEAD"));
+		return -1;
 	}
 	strbuf_addf(&buf, "%s\n", head);
 	written = write_in_full(fd, buf.buf, buf.len);
 	strbuf_release(&buf);
 	if (written < 0) {
+		error_errno(_("could not write to '%s'"), git_path_head_file());
 		rollback_lock_file(&head_lock);
-		return error_errno(_("could not write to '%s'"),
-				   git_path_head_file());
+		return -1;
 	}
 	if (commit_lock_file(&head_lock) < 0) {
 		rollback_lock_file(&head_lock);
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 02/12] sequencer: make rearrange_squash() a bit more obvious
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
  2018-02-11  0:10     ` [PATCH v3 01/12] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
@ 2018-02-11  0:10     ` Johannes Schindelin
  2018-02-11  0:10     ` [PATCH v3 03/12] git-rebase--interactive: clarify arguments Johannes Schindelin
                       ` (11 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-11  0:10 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

There are some commands that have to be skipped from rearranging by virtue
of not handling any commits.

However, the logic was not quite obvious: it skipped commands based on
their position in the enum todo_command.

Instead, let's make it explicit that we skip all commands that do not
handle any commit. With one exception: the `drop` command, because it,
well, drops the commit and is therefore not eligible to rearranging.

Note: this is a bit academic at the moment because the only time we call
`rearrange_squash()` is directly after generating the todo list, when we
have nothing but `pick` commands anyway.

However, the upcoming `merge` command *will* want to be handled by that
function, and it *can* handle commits.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 114db3b2775..764ad43388f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2890,7 +2890,7 @@ int rearrange_squash(void)
 		struct subject2item_entry *entry;
 
 		next[i] = tail[i] = -1;
-		if (item->command >= TODO_EXEC) {
+		if (!item->commit || item->command == TODO_DROP) {
 			subjects[i] = NULL;
 			continue;
 		}
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 03/12] git-rebase--interactive: clarify arguments
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
  2018-02-11  0:10     ` [PATCH v3 01/12] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
  2018-02-11  0:10     ` [PATCH v3 02/12] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
@ 2018-02-11  0:10     ` Johannes Schindelin
  2018-02-11  0:10     ` [PATCH v3 04/12] sequencer: introduce new commands to reset the revision Johannes Schindelin
                       ` (10 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-11  0:10 UTC (permalink / raw)
  To: git
  Cc: Stefan Beller, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood

From: Stefan Beller <stefanbeller@gmail.com>

Up to now each command took a commit as its first argument and ignored
the rest of the line (usually the subject of the commit)

Now that we are about to introduce commands that take different
arguments, clarify each command by giving the argument list.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index d47bd29593a..fcedece1860 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -155,13 +155,13 @@ reschedule_last_action () {
 append_todo_help () {
 	gettext "
 Commands:
-p, pick = use commit
-r, reword = use commit, but edit the commit message
-e, edit = use commit, but stop for amending
-s, squash = use commit, but meld into previous commit
-f, fixup = like \"squash\", but discard this commit's log message
-x, exec = run command (the rest of the line) using shell
-d, drop = remove commit
+p, pick <commit> = use commit
+r, reword <commit> = use commit, but edit the commit message
+e, edit <commit> = use commit, but stop for amending
+s, squash <commit> = use commit, but meld into previous commit
+f, fixup <commit> = like \"squash\", but discard this commit's log message
+x, exec <commit> = run command (the rest of the line) using shell
+d, drop <commit> = remove commit
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 04/12] sequencer: introduce new commands to reset the revision
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
                       ` (2 preceding siblings ...)
  2018-02-11  0:10     ` [PATCH v3 03/12] git-rebase--interactive: clarify arguments Johannes Schindelin
@ 2018-02-11  0:10     ` Johannes Schindelin
  2018-02-12 19:26       ` Eric Sunshine
  2018-02-11  0:10     ` [PATCH v3 05/12] sequencer: introduce the `merge` command Johannes Schindelin
                       ` (9 subsequent siblings)
  13 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-11  0:10 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

In the upcoming commits, we will teach the sequencer to recreate merges.
This will be done in a very different way from the unfortunate design of
`git rebase --preserve-merges` (which does not allow for reordering
commits, or changing the branch topology).

The main idea is to introduce new todo list commands, to support
labeling the current revision with a given name, resetting the current
revision to a previous state, and  merging labeled revisions.

This idea was developed in Git for Windows' Git garden shears (that are
used to maintain the "thicket of branches" on top of upstream Git), and
this patch is part of the effort to make it available to a wider
audience, as well as to make the entire process more robust (by
implementing it in a safe and portable language rather than a Unix shell
script).

This commit implements the commands to label, and to reset to, given
revisions. The syntax is:

	label <name>
	reset <name>

Internally, the `label <name>` command creates the ref
`refs/rewritten/<name>`. This makes it possible to work with the labeled
revisions interactively, or in a scripted fashion (e.g. via the todo
list command `exec`).

These temporary refs are removed upon sequencer_remove_state(), so that
even a `git rebase --abort` cleans them up.

We disallow '#' as label because that character will be used as separator
in the upcoming `merge` command.

Later in this patch series, we will mark the `refs/rewritten/` refs as
worktree-local, to allow for interactive rebases to be run in parallel in
worktrees linked to the same repository.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   2 +
 sequencer.c                | 190 +++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 186 insertions(+), 6 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index fcedece1860..7e5281e74aa 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -162,6 +162,8 @@ s, squash <commit> = use commit, but meld into previous commit
 f, fixup <commit> = like \"squash\", but discard this commit's log message
 x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
+l, label <label> = label current HEAD with a name
+t, reset <label> = reset HEAD to a label
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index 764ad43388f..8638086f667 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -21,6 +21,8 @@
 #include "log-tree.h"
 #include "wt-status.h"
 #include "hashmap.h"
+#include "unpack-trees.h"
+#include "worktree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -116,6 +118,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
 static GIT_PATH_FUNC(rebase_path_rewritten_pending,
 	"rebase-merge/rewritten-pending")
+
+/*
+ * The path of the file listing refs that need to be deleted after the rebase
+ * finishes. This is used by the `label` command to record the need for cleanup.
+ */
+static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
+
 /*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
@@ -195,18 +204,33 @@ static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
 
 int sequencer_remove_state(struct replay_opts *opts)
 {
-	struct strbuf dir = STRBUF_INIT;
+	struct strbuf buf = STRBUF_INIT;
 	int i;
 
+	if (strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
+		char *p = buf.buf;
+		while (*p) {
+			char *eol = strchr(p, '\n');
+			if (eol)
+				*eol = '\0';
+			if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
+				warning(_("could not delete '%s'"), p);
+			if (!eol)
+				break;
+			p = eol + 1;
+		}
+	}
+
 	free(opts->gpg_sign);
 	free(opts->strategy);
 	for (i = 0; i < opts->xopts_nr; i++)
 		free(opts->xopts[i]);
 	free(opts->xopts);
 
-	strbuf_addstr(&dir, get_dir(opts));
-	remove_dir_recursively(&dir, 0);
-	strbuf_release(&dir);
+	strbuf_reset(&buf);
+	strbuf_addstr(&buf, get_dir(opts));
+	remove_dir_recursively(&buf, 0);
+	strbuf_release(&buf);
 
 	return 0;
 }
@@ -769,6 +793,8 @@ enum todo_command {
 	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
+	TODO_LABEL,
+	TODO_RESET,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -787,6 +813,8 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
+	{ 'l', "label" },
+	{ 't', "reset" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1281,7 +1309,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return error(_("missing arguments for %s"),
 			     command_to_string(item->command));
 
-	if (item->command == TODO_EXEC) {
+	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
+	    item->command == TODO_RESET) {
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = (int)(eol - bol);
@@ -1922,6 +1951,151 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int safe_append(const char *filename, const char *fmt, ...)
+{
+	va_list ap;
+	struct lock_file lock = LOCK_INIT;
+	int fd = hold_lock_file_for_update(&lock, filename,
+					   LOCK_REPORT_ON_ERROR);
+	struct strbuf buf = STRBUF_INIT;
+
+	if (fd < 0)
+		return -1;
+
+	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT)
+		return error_errno(_("could not read '%s'"), filename);
+	strbuf_complete(&buf, '\n');
+	va_start(ap, fmt);
+	strbuf_vaddf(&buf, fmt, ap);
+	va_end(ap);
+
+	if (write_in_full(fd, buf.buf, buf.len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	if (commit_lock_file(&lock) < 0) {
+		rollback_lock_file(&lock);
+		return error(_("failed to finalize '%s'"), filename);
+	}
+
+	return 0;
+}
+
+static int do_label(const char *name, int len)
+{
+	struct ref_store *refs = get_main_ref_store();
+	struct ref_transaction *transaction;
+	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
+	struct strbuf msg = STRBUF_INIT;
+	int ret = 0;
+	struct object_id head_oid;
+
+	if (len == 1 && *name == '#')
+		return error("Illegal label name: '%.*s'", len, name);
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
+
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction) {
+		error("%s", err.buf);
+		ret = -1;
+	} else if (get_oid("HEAD", &head_oid)) {
+		error(_("could not read HEAD"));
+		ret = -1;
+	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
+					  NULL, 0, msg.buf, &err) < 0 ||
+		   ref_transaction_commit(transaction, &err)) {
+		error("%s", err.buf);
+		ret = -1;
+	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
+	strbuf_release(&msg);
+
+	if (!ret)
+		ret = safe_append(rebase_path_refs_to_delete(),
+				  "%s\n", ref_name.buf);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
+static int do_reset(const char *name, int len, struct replay_opts *opts)
+{
+	struct strbuf ref_name = STRBUF_INIT;
+	struct object_id oid;
+	struct lock_file lock = LOCK_INIT;
+	struct tree_desc desc;
+	struct tree *tree;
+	struct unpack_trees_options unpack_tree_opts;
+	int ret = 0, i;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	/* Determine the length of the label */
+	for (i = 0; i < len; i++)
+		if (isspace(name[i]))
+			len = i;
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	if (get_oid(ref_name.buf, &oid) &&
+	    get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
+		error(_("could not read '%s'"), ref_name.buf);
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
+	unpack_tree_opts.head_idx = 1;
+	unpack_tree_opts.src_index = &the_index;
+	unpack_tree_opts.dst_index = &the_index;
+	unpack_tree_opts.fn = oneway_merge;
+	unpack_tree_opts.merge = 1;
+	unpack_tree_opts.update = 1;
+	unpack_tree_opts.reset = 1;
+
+	if (read_cache_unmerged())
+		return error_resolve_conflict(_(action_name(opts)));
+
+	if (!fill_tree_descriptor(&desc, &oid)) {
+		error(_("failed to find tree of %s"), oid_to_hex(&oid));
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	if (unpack_trees(1, &desc, &unpack_tree_opts)) {
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	tree = parse_tree_indirect(&oid);
+	prime_cache_tree(&the_index, tree);
+
+	if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
+		ret = error(_("could not write index"));
+	free((void *)desc.buffer);
+
+	if (!ret) {
+		struct strbuf msg = STRBUF_INIT;
+
+		strbuf_addf(&msg, "(rebase -i) reset '%.*s'", len, name);
+		ret = update_ref(msg.buf, "HEAD", &oid, NULL, 0,
+				 UPDATE_REFS_MSG_ON_ERR);
+		strbuf_release(&msg);
+	}
+
+	strbuf_release(&ref_name);
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2105,7 +2279,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				/* `current` will be incremented below */
 				todo_list->current = -1;
 			}
-		} else if (!is_noop(item->command))
+		} else if (item->command == TODO_LABEL)
+			res = do_label(item->arg, item->arg_len);
+		else if (item->command == TODO_RESET)
+			res = do_reset(item->arg, item->arg_len, opts);
+		else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 05/12] sequencer: introduce the `merge` command
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
                       ` (3 preceding siblings ...)
  2018-02-11  0:10     ` [PATCH v3 04/12] sequencer: introduce new commands to reset the revision Johannes Schindelin
@ 2018-02-11  0:10     ` Johannes Schindelin
  2018-02-12  8:48       ` Eric Sunshine
  2018-02-11  0:10     ` [PATCH v3 06/12] sequencer: fast-forward merge commits, if possible Johannes Schindelin
                       ` (8 subsequent siblings)
  13 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-11  0:10 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

This patch is part of the effort to reimplement `--preserve-merges` with
a substantially improved design, a design that has been developed in the
Git for Windows project to maintain the dozens of Windows-specific patch
series on top of upstream Git.

The previous patch implemented the `label` and `reset` commands to label
commits and to reset to a labeled commits. This patch adds the `merge`
command, with the following syntax:

	merge [-C <commit>] <rev> # <oneline>

The <commit> parameter in this instance is the *original* merge commit,
whose author and message will be used for the merge commit that is about
to be created.

The <rev> parameter refers to the (possibly rewritten) revision to
merge. Let's see an example of a todo list:

	label onto

	# Branch abc
	reset onto
	pick deadbeef Hello, world!
	label abc

	reset onto
	pick cafecafe And now for something completely different
	merge -C baaabaaa abc # Merge the branch 'abc' into master

To edit the merge commit's message (a "reword" for merges, if you will),
use `-c` (lower-case) instead of `-C`; this convention was borrowed from
`git commit` that also supports `-c` and `-C` with similar meanings.

To create *new* merges, i.e. without copying the commit message from an
existing commit, simply omit the `-C <commit>` parameter (which will
open an editor for the merge message):

	merge abc

This comes in handy when splitting a branch into two or more branches.

Note: this patch only adds support for recursive merges, to keep things
simple. Support for octopus merges will be added later in a separate
patch series, support for merges using strategies other than the
recursive merge is left for the future.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   4 ++
 sequencer.c                | 158 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 162 insertions(+)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 7e5281e74aa..9d9d91f25e3 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -164,6 +164,10 @@ x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
 l, label <label> = label current HEAD with a name
 t, reset <label> = reset HEAD to a label
+m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
+.       create a merge commit using the original merge commit's
+.       message (or the oneline, if no original merge commit was
+.       specified). Use -c <commit> to reword the commit message.
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index 8638086f667..e577c213494 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -795,6 +795,8 @@ enum todo_command {
 	TODO_EXEC,
 	TODO_LABEL,
 	TODO_RESET,
+	TODO_MERGE,
+	TODO_MERGE_AND_EDIT,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -815,6 +817,8 @@ static struct {
 	{ 'x', "exec" },
 	{ 'l', "label" },
 	{ 't', "reset" },
+	{ 'm', "merge" },
+	{ 0, "merge" }, /* MERGE_AND_EDIT */
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1317,6 +1321,21 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return 0;
 	}
 
+	if (item->command == TODO_MERGE) {
+		if (skip_prefix(bol, "-C", &bol))
+			bol += strspn(bol, " \t");
+		else if (skip_prefix(bol, "-c", &bol)) {
+			bol += strspn(bol, " \t");
+			item->command = TODO_MERGE_AND_EDIT;
+		} else {
+			item->command = TODO_MERGE_AND_EDIT;
+			item->commit = NULL;
+			item->arg = bol;
+			item->arg_len = (int)(eol - bol);
+			return 0;
+		}
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
@@ -2096,6 +2115,134 @@ static int do_reset(const char *name, int len, struct replay_opts *opts)
 	return ret;
 }
 
+static int do_merge(struct commit *commit, const char *arg, int arg_len,
+		    int run_commit_flags, struct replay_opts *opts)
+{
+	int merge_arg_len;
+	struct strbuf ref_name = STRBUF_INIT;
+	struct commit *head_commit, *merge_commit, *i;
+	struct commit_list *common, *j, *reversed = NULL;
+	struct merge_options o;
+	int ret;
+	static struct lock_file lock;
+
+	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
+		if (isspace(arg[merge_arg_len]))
+			break;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	head_commit = lookup_commit_reference_by_name("HEAD");
+	if (!head_commit) {
+		rollback_lock_file(&lock);
+		return error(_("cannot merge without a current revision"));
+	}
+
+	if (commit) {
+		const char *message = get_commit_buffer(commit, NULL);
+		const char *body;
+		int len;
+
+		if (!message) {
+			rollback_lock_file(&lock);
+			return error(_("could not get commit message of '%s'"),
+				     oid_to_hex(&commit->object.oid));
+		}
+		write_author_script(message);
+		find_commit_subject(message, &body);
+		len = strlen(body);
+		if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			unuse_commit_buffer(commit, message);
+			rollback_lock_file(&lock);
+			return -1;
+		}
+		unuse_commit_buffer(commit, message);
+	} else {
+		const char *p = arg + merge_arg_len;
+		struct strbuf buf = STRBUF_INIT;
+		int len;
+
+		strbuf_addf(&buf, "author %s", git_author_info(0));
+		write_author_script(buf.buf);
+		strbuf_reset(&buf);
+
+		p += strspn(p, " \t");
+		if (*p == '#' && isspace(p[1]))
+			p += 1 + strspn(p + 1, " \t");
+		if (*p)
+			len = strlen(p);
+		else {
+			strbuf_addf(&buf, "Merge branch '%.*s'",
+				    merge_arg_len, arg);
+			p = buf.buf;
+			len = buf.len;
+		}
+
+		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			strbuf_release(&buf);
+			rollback_lock_file(&lock);
+			return -1;
+		}
+		strbuf_release(&buf);
+	}
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
+	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	if (!merge_commit) {
+		/* fall back to non-rewritten ref or commit */
+		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
+		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	}
+	if (!merge_commit) {
+		error(_("could not resolve '%s'"), ref_name.buf);
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
+		      git_path_merge_head(), 0);
+	write_message("no-ff", 5, git_path_merge_mode(), 0);
+
+	common = get_merge_bases(head_commit, merge_commit);
+	for (j = common; j; j = j->next)
+		commit_list_insert(j->item, &reversed);
+	free_commit_list(common);
+
+	read_cache();
+	init_merge_options(&o);
+	o.branch1 = "HEAD";
+	o.branch2 = ref_name.buf;
+	o.buffer_output = 2;
+
+	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
+	if (ret <= 0)
+		fputs(o.obuf.buf, stdout);
+	strbuf_release(&o.obuf);
+	if (ret < 0) {
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return error(_("conflicts while merging '%.*s'"),
+			     merge_arg_len, arg);
+	}
+
+	if (active_cache_changed &&
+	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
+		strbuf_release(&ref_name);
+		return error(_("merge: Unable to write new index file"));
+	}
+	rollback_lock_file(&lock);
+
+	ret = run_git_commit(git_path_merge_msg(), opts, run_commit_flags);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2283,6 +2430,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_label(item->arg, item->arg_len);
 		else if (item->command == TODO_RESET)
 			res = do_reset(item->arg, item->arg_len, opts);
+		else if (item->command == TODO_MERGE ||
+			 item->command == TODO_MERGE_AND_EDIT)
+			res = do_merge(item->commit, item->arg, item->arg_len,
+				       item->command == TODO_MERGE_AND_EDIT ?
+				       EDIT_MSG | VERIFY_MSG : 0, opts);
 		else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
@@ -2764,8 +2916,14 @@ int transform_todos(unsigned flags)
 					  short_commit_name(item->commit) :
 					  oid_to_hex(&item->commit->object.oid);
 
+			if (item->command == TODO_MERGE)
+				strbuf_addstr(&buf, " -C");
+			else if (item->command == TODO_MERGE_AND_EDIT)
+				strbuf_addstr(&buf, " -c");
+
 			strbuf_addf(&buf, " %s", oid);
 		}
+
 		/* add all the rest */
 		if (!item->arg_len)
 			strbuf_addch(&buf, '\n');
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 06/12] sequencer: fast-forward merge commits, if possible
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
                       ` (4 preceding siblings ...)
  2018-02-11  0:10     ` [PATCH v3 05/12] sequencer: introduce the `merge` command Johannes Schindelin
@ 2018-02-11  0:10     ` Johannes Schindelin
  2018-02-11  0:10     ` [PATCH v3 07/12] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
                       ` (7 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-11  0:10 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Just like with regular `pick` commands, if we are trying to recreate a
merge commit, we now test whether the parents of said commit match HEAD
and the commits to be merged, and fast-forward if possible.

This is not only faster, but also avoids unnecessary proliferation of
new objects.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index e577c213494..27d582479d1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2123,7 +2123,7 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	struct commit *head_commit, *merge_commit, *i;
 	struct commit_list *common, *j, *reversed = NULL;
 	struct merge_options o;
-	int ret;
+	int can_fast_forward, ret;
 	static struct lock_file lock;
 
 	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
@@ -2191,6 +2191,14 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		strbuf_release(&buf);
 	}
 
+	/*
+	 * If HEAD is not identical to the parent of the original merge commit,
+	 * we cannot fast-forward.
+	 */
+	can_fast_forward = opts->allow_ff && commit && commit->parents &&
+		!oidcmp(&commit->parents->item->object.oid,
+			&head_commit->object.oid);
+
 	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
 	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
 	if (!merge_commit) {
@@ -2204,6 +2212,17 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		rollback_lock_file(&lock);
 		return -1;
 	}
+
+	if (can_fast_forward && commit->parents->next &&
+	    !commit->parents->next->next &&
+	    !oidcmp(&commit->parents->next->item->object.oid,
+		    &merge_commit->object.oid)) {
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return fast_forward_to(&commit->object.oid,
+				       &head_commit->object.oid, 0, opts);
+	}
+
 	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
 		      git_path_merge_head(), 0);
 	write_message("no-ff", 5, git_path_merge_mode(), 0);
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 07/12] rebase-helper --make-script: introduce a flag to recreate merges
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
                       ` (5 preceding siblings ...)
  2018-02-11  0:10     ` [PATCH v3 06/12] sequencer: fast-forward merge commits, if possible Johannes Schindelin
@ 2018-02-11  0:10     ` Johannes Schindelin
  2018-02-11  0:10     ` [PATCH v3 08/12] rebase: introduce the --recreate-merges option Johannes Schindelin
                       ` (6 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-11  0:10 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

The sequencer just learned new commands intended to recreate branch
structure (similar in spirit to --preserve-merges, but with a
substantially less-broken design).

Let's allow the rebase--helper to generate todo lists making use of
these commands, triggered by the new --recreate-merges option. For a
commit topology like this (where the HEAD points to C):

	- A - B - C
	    \   /
	      D

the generated todo list would look like this:

	# branch D
	pick 0123 A
	label branch-point
	pick 1234 D
	label D

	reset branch-point
	pick 2345 B
	merge -C 3456 D # C

To keep things simple, we first only implement support for merge commits
with exactly two parents, leaving support for octopus merges to a later
patch in this patch series.

As a special, hard-coded label, all merge-recreating todo lists start with
the command `label onto` so that we can later always refer to the revision
onto which everything is rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/rebase--helper.c |   4 +-
 sequencer.c              | 349 ++++++++++++++++++++++++++++++++++++++++++++++-
 sequencer.h              |   1 +
 3 files changed, 351 insertions(+), 3 deletions(-)

diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index 7daee544b7b..a34ab5c0655 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -12,7 +12,7 @@ static const char * const builtin_rebase_helper_usage[] = {
 int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
-	unsigned flags = 0, keep_empty = 0;
+	unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
 	int abbreviate_commands = 0;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
@@ -22,6 +22,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 	struct option options[] = {
 		OPT_BOOL(0, "ff", &opts.allow_ff, N_("allow fast-forward")),
 		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
+		OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge commits")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -55,6 +56,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
+	flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
 	if (command == CONTINUE && argc == 1)
diff --git a/sequencer.c b/sequencer.c
index 27d582479d1..7cd091a9fd6 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -23,6 +23,8 @@
 #include "hashmap.h"
 #include "unpack-trees.h"
 #include "worktree.h"
+#include "oidmap.h"
+#include "oidset.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -2808,6 +2810,341 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
 	strbuf_release(&sob);
 }
 
+struct labels_entry {
+	struct hashmap_entry entry;
+	char label[FLEX_ARRAY];
+};
+
+static int labels_cmp(const void *fndata, const struct labels_entry *a,
+		      const struct labels_entry *b, const void *key)
+{
+	return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
+}
+
+struct string_entry {
+	struct oidmap_entry entry;
+	char string[FLEX_ARRAY];
+};
+
+struct label_state {
+	struct oidmap commit2label;
+	struct hashmap labels;
+	struct strbuf buf;
+};
+
+static const char *label_oid(struct object_id *oid, const char *label,
+			     struct label_state *state)
+{
+	struct labels_entry *labels_entry;
+	struct string_entry *string_entry;
+	struct object_id dummy;
+	size_t len;
+	int i;
+
+	string_entry = oidmap_get(&state->commit2label, oid);
+	if (string_entry)
+		return string_entry->string;
+
+	/*
+	 * For "uninteresting" commits, i.e. commits that are not to be
+	 * rebased, and which can therefore not be labeled, we use a unique
+	 * abbreviation of the commit name. This is slightly more complicated
+	 * than calling find_unique_abbrev() because we also need to make
+	 * sure that the abbreviation does not conflict with any other
+	 * label.
+	 *
+	 * We disallow "interesting" commits to be labeled by a string that
+	 * is a valid full-length hash, to ensure that we always can find an
+	 * abbreviation for any uninteresting commit's names that does not
+	 * clash with any other label.
+	 */
+	if (!label) {
+		char *p;
+
+		strbuf_reset(&state->buf);
+		strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
+		label = p = state->buf.buf;
+
+		find_unique_abbrev_r(p, oid->hash, default_abbrev);
+
+		/*
+		 * We may need to extend the abbreviated hash so that there is
+		 * no conflicting label.
+		 */
+		if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
+			size_t i = strlen(p) + 1;
+
+			oid_to_hex_r(p, oid);
+			for (; i < GIT_SHA1_HEXSZ; i++) {
+				char save = p[i];
+				p[i] = '\0';
+				if (!hashmap_get_from_hash(&state->labels,
+							   strihash(p), p))
+					break;
+				p[i] = save;
+			}
+		}
+	} else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
+		    !get_oid_hex(label, &dummy)) ||
+		   (len == 1 && *label == '#') ||
+		   hashmap_get_from_hash(&state->labels,
+					 strihash(label), label)) {
+		/*
+		 * If the label already exists, or if the label is a valid full
+		 * OID, or the label is a '#' (which we use as a separator
+		 * between merge heads and oneline), we append a dash and a
+		 * number to make it unique.
+		 */
+		struct strbuf *buf = &state->buf;
+
+		strbuf_reset(buf);
+		strbuf_add(buf, label, len);
+
+		for (i = 2; ; i++) {
+			strbuf_setlen(buf, len);
+			strbuf_addf(buf, "-%d", i);
+			if (!hashmap_get_from_hash(&state->labels,
+						   strihash(buf->buf),
+						   buf->buf))
+				break;
+		}
+
+		label = buf->buf;
+	}
+
+	FLEX_ALLOC_STR(labels_entry, label, label);
+	hashmap_entry_init(labels_entry, strihash(label));
+	hashmap_add(&state->labels, labels_entry);
+
+	FLEX_ALLOC_STR(string_entry, string, label);
+	oidcpy(&string_entry->entry.oid, oid);
+	oidmap_put(&state->commit2label, string_entry);
+
+	return string_entry->string;
+}
+
+static int make_script_with_merges(struct pretty_print_context *pp,
+				   struct rev_info *revs, FILE *out,
+				   unsigned flags)
+{
+	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
+	struct strbuf label = STRBUF_INIT;
+	struct commit_list *commits = NULL, **tail = &commits, *iter;
+	struct commit_list *tips = NULL, **tips_tail = &tips;
+	struct commit *commit;
+	struct oidmap commit2todo = OIDMAP_INIT;
+	struct string_entry *entry;
+	struct oidset interesting = OIDSET_INIT, child_seen = OIDSET_INIT,
+		shown = OIDSET_INIT;
+	struct label_state state = { OIDMAP_INIT, { NULL }, STRBUF_INIT };
+
+	int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
+	const char *cmd_pick = abbr ? "p" : "pick",
+		*cmd_label = abbr ? "l" : "label",
+		*cmd_reset = abbr ? "t" : "reset",
+		*cmd_merge = abbr ? "m" : "merge";
+
+	oidmap_init(&commit2todo, 0);
+	oidmap_init(&state.commit2label, 0);
+	hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
+	strbuf_init(&state.buf, 32);
+
+	if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
+		struct object_id *oid = &revs->cmdline.rev[0].item->oid;
+		FLEX_ALLOC_STR(entry, string, "onto");
+		oidcpy(&entry->entry.oid, oid);
+		oidmap_put(&state.commit2label, entry);
+	}
+
+	/*
+	 * First phase:
+	 * - get onelines for all commits
+	 * - gather all branch tips (i.e. 2nd or later parents of merges)
+	 * - label all branch tips
+	 */
+	while ((commit = get_revision(revs))) {
+		struct commit_list *to_merge;
+		int is_octopus;
+		const char *p1, *p2;
+		struct object_id *oid;
+
+		tail = &commit_list_insert(commit, tail)->next;
+		oidset_insert(&interesting, &commit->object.oid);
+
+		if ((commit->object.flags & PATCHSAME))
+			continue;
+
+		strbuf_reset(&oneline);
+		pretty_print_commit(pp, commit, &oneline);
+
+		to_merge = commit->parents ? commit->parents->next : NULL;
+		if (!to_merge) {
+			/* non-merge commit: easy case */
+			strbuf_reset(&buf);
+			if (!keep_empty && is_original_commit_empty(commit))
+				strbuf_addf(&buf, "%c ", comment_line_char);
+			strbuf_addf(&buf, "%s %s %s", cmd_pick,
+				    oid_to_hex(&commit->object.oid),
+				    oneline.buf);
+
+			FLEX_ALLOC_STR(entry, string, buf.buf);
+			oidcpy(&entry->entry.oid, &commit->object.oid);
+			oidmap_put(&commit2todo, entry);
+
+			continue;
+		}
+
+		is_octopus = to_merge && to_merge->next;
+
+		if (is_octopus)
+			BUG("Octopus merges not yet supported");
+
+		/* Create a label */
+		strbuf_reset(&label);
+		if (skip_prefix(oneline.buf, "Merge ", &p1) &&
+		    (p1 = strchr(p1, '\'')) &&
+		    (p2 = strchr(++p1, '\'')))
+			strbuf_add(&label, p1, p2 - p1);
+		else if (skip_prefix(oneline.buf, "Merge pull request ",
+				     &p1) &&
+			 (p1 = strstr(p1, " from ")))
+			strbuf_addstr(&label, p1 + strlen(" from "));
+		else
+			strbuf_addbuf(&label, &oneline);
+
+		for (p1 = label.buf; *p1; p1++)
+			if (isspace(*p1))
+				*(char *)p1 = '-';
+
+		strbuf_reset(&buf);
+		strbuf_addf(&buf, "%s -C %s",
+			    cmd_merge, oid_to_hex(&commit->object.oid));
+
+		/* label the tip of merged branch */
+		oid = &to_merge->item->object.oid;
+		strbuf_addch(&buf, ' ');
+
+		if (!oidset_contains(&interesting, oid))
+			strbuf_addstr(&buf, label_oid(oid, NULL, &state));
+		else {
+			tips_tail = &commit_list_insert(to_merge->item,
+							tips_tail)->next;
+
+			strbuf_addstr(&buf, label_oid(oid, label.buf, &state));
+		}
+		strbuf_addf(&buf, " # %s", oneline.buf);
+
+		FLEX_ALLOC_STR(entry, string, buf.buf);
+		oidcpy(&entry->entry.oid, &commit->object.oid);
+		oidmap_put(&commit2todo, entry);
+	}
+
+	/*
+	 * Second phase:
+	 * - label branch points
+	 * - add HEAD to the branch tips
+	 */
+	for (iter = commits; iter; iter = iter->next) {
+		struct commit_list *parent = iter->item->parents;
+		for (; parent; parent = parent->next) {
+			struct object_id *oid = &parent->item->object.oid;
+			if (!oidset_contains(&interesting, oid))
+				continue;
+			if (!oidset_contains(&child_seen, oid))
+				oidset_insert(&child_seen, oid);
+			else
+				label_oid(oid, "branch-point", &state);
+		}
+
+		/* Add HEAD as implict "tip of branch" */
+		if (!iter->next)
+			tips_tail = &commit_list_insert(iter->item,
+							tips_tail)->next;
+	}
+
+	/*
+	 * Third phase: output the todo list. This is a bit tricky, as we
+	 * want to avoid jumping back and forth between revisions. To
+	 * accomplish that goal, we walk backwards from the branch tips,
+	 * gathering commits not yet shown, reversing the list on the fly,
+	 * then outputting that list (labeling revisions as needed).
+	 */
+	fprintf(out, "%s onto\n", cmd_label);
+	for (iter = tips; iter; iter = iter->next) {
+		struct commit_list *list = NULL, *iter2;
+
+		commit = iter->item;
+		if (oidset_contains(&shown, &commit->object.oid))
+			continue;
+		entry = oidmap_get(&state.commit2label, &commit->object.oid);
+
+		if (entry)
+			fprintf(out, "\n# Branch %s\n", entry->string);
+		else
+			fprintf(out, "\n");
+
+		while (oidset_contains(&interesting, &commit->object.oid) &&
+		       !oidset_contains(&shown, &commit->object.oid)) {
+			commit_list_insert(commit, &list);
+			if (!commit->parents) {
+				commit = NULL;
+				break;
+			}
+			commit = commit->parents->item;
+		}
+
+		if (!commit)
+			fprintf(out, "%s onto\n", cmd_reset);
+		else {
+			const char *to = NULL;
+
+			entry = oidmap_get(&state.commit2label,
+					   &commit->object.oid);
+			if (entry)
+				to = entry->string;
+
+			if (!to || !strcmp(to, "onto"))
+				fprintf(out, "%s onto\n", cmd_reset);
+			else {
+				strbuf_reset(&oneline);
+				pretty_print_commit(pp, commit, &oneline);
+				fprintf(out, "%s %s # %s\n",
+					cmd_reset, to, oneline.buf);
+			}
+		}
+
+		for (iter2 = list; iter2; iter2 = iter2->next) {
+			struct object_id *oid = &iter2->item->object.oid;
+			entry = oidmap_get(&commit2todo, oid);
+			/* only show if not already upstream */
+			if (entry)
+				fprintf(out, "%s\n", entry->string);
+			entry = oidmap_get(&state.commit2label, oid);
+			if (entry)
+				fprintf(out, "%s %s\n",
+					cmd_label, entry->string);
+			oidset_insert(&shown, oid);
+		}
+
+		free_commit_list(list);
+	}
+
+	free_commit_list(commits);
+	free_commit_list(tips);
+
+	strbuf_release(&label);
+	strbuf_release(&oneline);
+	strbuf_release(&buf);
+
+	oidmap_free(&commit2todo, 1);
+	oidmap_free(&state.commit2label, 1);
+	hashmap_free(&state.labels, 1);
+	strbuf_release(&state.buf);
+
+	return 0;
+}
+
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags)
 {
@@ -2818,11 +3155,16 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	struct commit *commit;
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
 	const char *insn = flags & TODO_LIST_ABBREVIATE_CMDS ? "p" : "pick";
+	int recreate_merges = flags & TODO_LIST_RECREATE_MERGES;
 
 	init_revisions(&revs, NULL);
 	revs.verbose_header = 1;
-	revs.max_parents = 1;
-	revs.cherry_pick = 1;
+	if (recreate_merges)
+		revs.cherry_mark = 1;
+	else {
+		revs.max_parents = 1;
+		revs.cherry_pick = 1;
+	}
 	revs.limited = 1;
 	revs.reverse = 1;
 	revs.right_only = 1;
@@ -2846,6 +3188,9 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	if (prepare_revision_walk(&revs) < 0)
 		return error(_("make_script: error preparing revisions"));
 
+	if (recreate_merges)
+		return make_script_with_merges(&pp, &revs, out, flags);
+
 	while ((commit = get_revision(&revs))) {
 		strbuf_reset(&buf);
 		if (!keep_empty && is_original_commit_empty(commit))
diff --git a/sequencer.h b/sequencer.h
index 81f6d7d393f..11d1ac925ef 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -48,6 +48,7 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_KEEP_EMPTY (1U << 0)
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
+#define TODO_LIST_RECREATE_MERGES (1U << 3)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 08/12] rebase: introduce the --recreate-merges option
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
                       ` (6 preceding siblings ...)
  2018-02-11  0:10     ` [PATCH v3 07/12] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
@ 2018-02-11  0:10     ` Johannes Schindelin
  2018-02-11  0:10     ` [PATCH v3 09/12] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
                       ` (5 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-11  0:10 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Once upon a time, this here developer thought: wouldn't it be nice if,
say, Git for Windows' patches on top of core Git could be represented as
a thicket of branches, and be rebased on top of core Git in order to
maintain a cherry-pick'able set of patch series?

The original attempt to answer this was: git rebase --preserve-merges.

However, that experiment was never intended as an interactive option,
and it only piggy-backed on git rebase --interactive because that
command's implementation looked already very, very familiar: it was
designed by the same person who designed --preserve-merges: yours truly.

Some time later, some other developer (I am looking at you, Andreas!
;-)) decided that it would be a good idea to allow --preserve-merges to
be combined with --interactive (with caveats!) and the Git maintainer
(well, the interim Git maintainer during Junio's absence, that is)
agreed, and that is when the glamor of the --preserve-merges design
started to fall apart rather quickly and unglamorously.

The reason? In --preserve-merges mode, the parents of a merge commit (or
for that matter, of *any* commit) were not stated explicitly, but were
*implied* by the commit name passed to the `pick` command.

This made it impossible, for example, to reorder commits. Not to mention
to flatten the branch topology or, deity forbid, to split topic branches
into two.

Alas, these shortcomings also prevented that mode (whose original
purpose was to serve Git for Windows' needs, with the additional hope
that it may be useful to others, too) from serving Git for Windows'
needs.

Five years later, when it became really untenable to have one unwieldy,
big hodge-podge patch series of partly related, partly unrelated patches
in Git for Windows that was rebased onto core Git's tags from time to
time (earning the undeserved wrath of the developer of the ill-fated
git-remote-hg series that first obsoleted Git for Windows' competing
approach, only to be abandoned without maintainer later) was really
untenable, the "Git garden shears" were born [*1*/*2*]: a script,
piggy-backing on top of the interactive rebase, that would first
determine the branch topology of the patches to be rebased, create a
pseudo todo list for further editing, transform the result into a real
todo list (making heavy use of the `exec` command to "implement" the
missing todo list commands) and finally recreate the patch series on
top of the new base commit.

That was in 2013. And it took about three weeks to come up with the
design and implement it as an out-of-tree script. Needless to say, the
implementation needed quite a few years to stabilize, all the while the
design itself proved itself sound.

With this patch, the goodness of the Git garden shears comes to `git
rebase -i` itself. Passing the `--recreate-merges` option will generate
a todo list that can be understood readily, and where it is obvious
how to reorder commits. New branches can be introduced by inserting
`label` commands and calling `merge <label>`. And once this mode will
have become stable and universally accepted, we can deprecate the design
mistake that was `--preserve-merges`.

Link *1*:
https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
Link *2*:
https://github.com/git-for-windows/build-extra/blob/master/shears.sh

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt           |   9 +-
 contrib/completion/git-completion.bash |   2 +-
 git-rebase--interactive.sh             |   1 +
 git-rebase.sh                          |   6 ++
 t/t3430-rebase-recreate-merges.sh      | 146 +++++++++++++++++++++++++++++++++
 5 files changed, 162 insertions(+), 2 deletions(-)
 create mode 100755 t/t3430-rebase-recreate-merges.sh

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 8a861c1e0d6..e9da7e26329 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -368,6 +368,12 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
+--recreate-merges::
+	Recreate merge commits instead of flattening the history by replaying
+	merges. Merge conflict resolutions or manual amendments to merge
+	commits are not recreated automatically, but have to be recreated
+	manually.
+
 -p::
 --preserve-merges::
 	Recreate merge commits instead of flattening the history by replaying
@@ -770,7 +776,8 @@ BUGS
 The todo list presented by `--preserve-merges --interactive` does not
 represent the topology of the revision graph.  Editing commits and
 rewording their commit messages should work fine, but attempts to
-reorder commits tend to produce counterintuitive results.
+reorder commits tend to produce counterintuitive results. Use
+--recreate-merges for a more faithful representation.
 
 For example, an attempt to rearrange
 ------------
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 3683c772c55..6893c3adabc 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2008,7 +2008,7 @@ _git_rebase ()
 	--*)
 		__gitcomp "
 			--onto --merge --strategy --interactive
-			--preserve-merges --stat --no-stat
+			--recreate-merges --preserve-merges --stat --no-stat
 			--committer-date-is-author-date --ignore-date
 			--ignore-whitespace --whitespace=
 			--autosquash --no-autosquash
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 9d9d91f25e3..cfe3a537ac2 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -902,6 +902,7 @@ fi
 if test t != "$preserve_merges"
 then
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
+		${recreate_merges:+--recreate-merges} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 else
diff --git a/git-rebase.sh b/git-rebase.sh
index fd72a35c65b..d69bc7d0e0d 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,6 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
+recreate-merges!   try to recreate merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -86,6 +87,7 @@ type=
 state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
+recreate_merges=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -262,6 +264,10 @@ do
 	--keep-empty)
 		keep_empty=yes
 		;;
+	--recreate-merges)
+		recreate_merges=t
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
new file mode 100755
index 00000000000..0073601a206
--- /dev/null
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -0,0 +1,146 @@
+#!/bin/sh
+#
+# Copyright (c) 2017 Johannes E. Schindelin
+#
+
+test_description='git rebase -i --recreate-merges
+
+This test runs git rebase "interactively", retaining the branch structure by
+recreating merge commits.
+
+Initial setup:
+
+    -- B --                   (first)
+   /       \
+ A - C - D - E - H            (master)
+       \       /
+         F - G                (second)
+'
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
+
+test_expect_success 'setup' '
+	write_script replace-editor.sh <<-\EOF &&
+	mv "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	cp script-from-scratch "$1"
+	EOF
+
+	test_commit A &&
+	git checkout -b first &&
+	test_commit B &&
+	git checkout master &&
+	test_commit C &&
+	test_commit D &&
+	git merge --no-commit B &&
+	test_tick &&
+	git commit -m E &&
+	git tag -m E E &&
+	git checkout -b second C &&
+	test_commit F &&
+	test_commit G &&
+	git checkout master &&
+	git merge --no-commit G &&
+	test_tick &&
+	git commit -m H &&
+	git tag -m H H
+'
+
+cat >script-from-scratch <<\EOF
+label onto
+
+# onebranch
+pick G
+pick D
+label onebranch
+
+# second
+reset onto
+pick B
+label second
+
+reset onto
+merge -C H second
+merge onebranch # Merge the topic branch 'onebranch'
+EOF
+
+test_cmp_graph () {
+	cat >expect &&
+	git log --graph --boundary --format=%s "$@" >output &&
+	sed "s/ *$//" <output >output.trimmed &&
+	test_cmp expect output.trimmed
+}
+
+test_expect_success 'create completely different structure' '
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_tick &&
+	git rebase -i --recreate-merges A &&
+	test_cmp_graph <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	* |   H
+	|\ \
+	| |/
+	|/|
+	| * B
+	|/
+	* A
+	EOF
+'
+
+test_expect_success 'generate correct todo list' '
+	cat >expect <<-\EOF &&
+	label onto
+
+	reset onto
+	pick d9df450 B
+	label E
+
+	reset onto
+	pick 5dee784 C
+	label branch-point
+	pick ca2c861 F
+	pick 088b00a G
+	label H
+
+	reset branch-point # C
+	pick 12bd07b D
+	merge -C 2051b56 E # E
+	merge -C 233d48a H # H
+
+	EOF
+
+	grep -v "^#" <.git/ORIGINAL-TODO >output &&
+	test_cmp expect output
+'
+
+test_expect_success 'with a branch tip that was cherry-picked already' '
+	git checkout -b already-upstream master &&
+	base="$(git rev-parse --verify HEAD)" &&
+
+	test_commit A1 &&
+	test_commit A2 &&
+	git reset --hard $base &&
+	test_commit B1 &&
+	test_tick &&
+	git merge -m "Merge branch A" A2 &&
+
+	git checkout -b upstream-with-a2 $base &&
+	test_tick &&
+	git cherry-pick A2 &&
+
+	git checkout already-upstream &&
+	test_tick &&
+	git rebase -i --recreate-merges upstream-with-a2 &&
+	test_cmp_graph upstream-with-a2.. <<-\EOF
+	*   Merge branch A
+	|\
+	| * A1
+	* | B1
+	|/
+	o A2
+	EOF
+'
+
+test_done
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 09/12] sequencer: make refs generated by the `label` command worktree-local
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
                       ` (7 preceding siblings ...)
  2018-02-11  0:10     ` [PATCH v3 08/12] rebase: introduce the --recreate-merges option Johannes Schindelin
@ 2018-02-11  0:10     ` Johannes Schindelin
  2018-02-11  0:10     ` [PATCH v3 10/12] sequencer: handle post-rewrite for merge commands Johannes Schindelin
                       ` (4 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-11  0:10 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

This allows for rebases to be run in parallel in separate worktrees
(think: interrupted in the middle of one rebase, being asked to perform
a different rebase, adding a separate worktree just for that job).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 refs.c                            |  3 ++-
 t/t3430-rebase-recreate-merges.sh | 14 ++++++++++++++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 20ba82b4343..e8b84c189ff 100644
--- a/refs.c
+++ b/refs.c
@@ -600,7 +600,8 @@ int dwim_log(const char *str, int len, struct object_id *oid, char **log)
 static int is_per_worktree_ref(const char *refname)
 {
 	return !strcmp(refname, "HEAD") ||
-		starts_with(refname, "refs/bisect/");
+		starts_with(refname, "refs/bisect/") ||
+		starts_with(refname, "refs/rewritten/");
 }
 
 static int is_pseudoref_syntax(const char *refname)
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index 0073601a206..1a3e43d66ff 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -143,4 +143,18 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'refs/rewritten/* is worktree-local' '
+	git worktree add wt &&
+	cat >wt/script-from-scratch <<-\EOF &&
+	label xyz
+	exec GIT_DIR=../.git git rev-parse --verify refs/rewritten/xyz >a || :
+	exec git rev-parse --verify refs/rewritten/xyz >b
+	EOF
+
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	git -C wt rebase -i HEAD &&
+	test_must_be_empty wt/a &&
+	test_cmp_rev HEAD "$(cat wt/b)"
+'
+
 test_done
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 10/12] sequencer: handle post-rewrite for merge commands
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
                       ` (8 preceding siblings ...)
  2018-02-11  0:10     ` [PATCH v3 09/12] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
@ 2018-02-11  0:10     ` Johannes Schindelin
  2018-02-11  0:10     ` [PATCH v3 11/12] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
                       ` (3 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-11  0:10 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

In the previous patches, we implemented the basic functionality of the
`git rebase -i --recreate-merges` command, in particular the `merge`
command to create merge commits in the sequencer.

The interactive rebase is a lot more these days, though, than a simple
cherry-pick in a loop. For example, it calls the post-rewrite hook (if
any) after rebasing with a mapping of the old->new commits.

This patch implements the post-rewrite handling for the `merge` command
we just introduced. The other commands that were added recently (`label`
and `reset`) do not create new commits, therefore post-rewrite do not
need to handle them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c                       |  7 +++++--
 t/t3430-rebase-recreate-merges.sh | 25 +++++++++++++++++++++++++
 2 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 7cd091a9fd6..306ae014311 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2452,11 +2452,14 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		else if (item->command == TODO_RESET)
 			res = do_reset(item->arg, item->arg_len, opts);
 		else if (item->command == TODO_MERGE ||
-			 item->command == TODO_MERGE_AND_EDIT)
+			 item->command == TODO_MERGE_AND_EDIT) {
 			res = do_merge(item->commit, item->arg, item->arg_len,
 				       item->command == TODO_MERGE_AND_EDIT ?
 				       EDIT_MSG | VERIFY_MSG : 0, opts);
-		else if (!is_noop(item->command))
+			if (item->commit)
+				record_in_rewritten(&item->commit->object.oid,
+						    peek_command(todo_list, 1));
+		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index 1a3e43d66ff..35a61ce90bb 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -157,4 +157,29 @@ test_expect_success 'refs/rewritten/* is worktree-local' '
 	test_cmp_rev HEAD "$(cat wt/b)"
 '
 
+test_expect_success 'post-rewrite hook and fixups work for merges' '
+	git checkout -b post-rewrite &&
+	test_commit same1 &&
+	git reset --hard HEAD^ &&
+	test_commit same2 &&
+	git merge -m "to fix up" same1 &&
+	echo same old same old >same2.t &&
+	test_tick &&
+	git commit --fixup HEAD same2.t &&
+	fixup="$(git rev-parse HEAD)" &&
+
+	mkdir -p .git/hooks &&
+	test_when_finished "rm .git/hooks/post-rewrite" &&
+	echo "cat >actual" | write_script .git/hooks/post-rewrite &&
+
+	test_tick &&
+	git rebase -i --autosquash --recreate-merges HEAD^^^ &&
+	printf "%s %s\n%s %s\n%s %s\n%s %s\n" >expect $(git rev-parse \
+		$fixup^^2 HEAD^2 \
+		$fixup^^ HEAD^ \
+		$fixup^ HEAD \
+		$fixup HEAD) &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 11/12] pull: accept --rebase=recreate to recreate the branch topology
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
                       ` (9 preceding siblings ...)
  2018-02-11  0:10     ` [PATCH v3 10/12] sequencer: handle post-rewrite for merge commands Johannes Schindelin
@ 2018-02-11  0:10     ` Johannes Schindelin
  2018-02-11  0:10     ` [PATCH v3 12/12] rebase -i: introduce --recreate-merges=[no-]rebase-cousins Johannes Schindelin
                       ` (2 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-11  0:10 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Similar to the `preserve` mode simply passing the `--preserve-merges`
option to the `rebase` command, the `recreate` mode simply passes the
`--recreate-merges` option.

This will allow users to conveniently rebase non-trivial commit
topologies when pulling new commits, without flattening them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/config.txt               |  8 ++++++++
 Documentation/git-pull.txt             |  5 ++++-
 builtin/pull.c                         | 14 ++++++++++----
 builtin/remote.c                       |  2 ++
 contrib/completion/git-completion.bash |  2 +-
 5 files changed, 25 insertions(+), 6 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0e25b2c92b3..da41ab246dc 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1058,6 +1058,10 @@ branch.<name>.rebase::
 	"git pull" is run. See "pull.rebase" for doing this in a non
 	branch-specific manner.
 +
+When recreate, also pass `--recreate-merges` along to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
@@ -2607,6 +2611,10 @@ pull.rebase::
 	pull" is run. See "branch.<name>.rebase" for setting this on a
 	per-branch basis.
 +
+When recreate, also pass `--recreate-merges` along to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
diff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt
index ce05b7a5b13..b4f9f057ea9 100644
--- a/Documentation/git-pull.txt
+++ b/Documentation/git-pull.txt
@@ -101,13 +101,16 @@ Options related to merging
 include::merge-options.txt[]
 
 -r::
---rebase[=false|true|preserve|interactive]::
+--rebase[=false|true|recreate|preserve|interactive]::
 	When true, rebase the current branch on top of the upstream
 	branch after fetching. If there is a remote-tracking branch
 	corresponding to the upstream branch and the upstream branch
 	was rebased since last fetched, the rebase uses that information
 	to avoid rebasing non-local changes.
 +
+When set to recreate, rebase with the `--recreate-merges` option passed
+to `git rebase` so that locally created merge commits will not be flattened.
++
 When set to preserve, rebase with the `--preserve-merges` option passed
 to `git rebase` so that locally created merge commits will not be flattened.
 +
diff --git a/builtin/pull.c b/builtin/pull.c
index 511dbbe0f6e..e33c84e0345 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -27,14 +27,16 @@ enum rebase_type {
 	REBASE_FALSE = 0,
 	REBASE_TRUE,
 	REBASE_PRESERVE,
+	REBASE_RECREATE,
 	REBASE_INTERACTIVE
 };
 
 /**
  * Parses the value of --rebase. If value is a false value, returns
  * REBASE_FALSE. If value is a true value, returns REBASE_TRUE. If value is
- * "preserve", returns REBASE_PRESERVE. If value is a invalid value, dies with
- * a fatal error if fatal is true, otherwise returns REBASE_INVALID.
+ * "recreate", returns REBASE_RECREATE. If value is "preserve", returns
+ * REBASE_PRESERVE. If value is a invalid value, dies with a fatal error if
+ * fatal is true, otherwise returns REBASE_INVALID.
  */
 static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		int fatal)
@@ -47,6 +49,8 @@ static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		return REBASE_TRUE;
 	else if (!strcmp(value, "preserve"))
 		return REBASE_PRESERVE;
+	else if (!strcmp(value, "recreate"))
+		return REBASE_RECREATE;
 	else if (!strcmp(value, "interactive"))
 		return REBASE_INTERACTIVE;
 
@@ -130,7 +134,7 @@ static struct option pull_options[] = {
 	/* Options passed to git-merge or git-rebase */
 	OPT_GROUP(N_("Options related to merging")),
 	{ OPTION_CALLBACK, 'r', "rebase", &opt_rebase,
-	  "false|true|preserve|interactive",
+	  "false|true|recreate|preserve|interactive",
 	  N_("incorporate changes by rebasing rather than merging"),
 	  PARSE_OPT_OPTARG, parse_opt_rebase },
 	OPT_PASSTHRU('n', NULL, &opt_diffstat, NULL,
@@ -798,7 +802,9 @@ static int run_rebase(const struct object_id *curr_head,
 	argv_push_verbosity(&args);
 
 	/* Options passed to git-rebase */
-	if (opt_rebase == REBASE_PRESERVE)
+	if (opt_rebase == REBASE_RECREATE)
+		argv_array_push(&args, "--recreate-merges");
+	else if (opt_rebase == REBASE_PRESERVE)
 		argv_array_push(&args, "--preserve-merges");
 	else if (opt_rebase == REBASE_INTERACTIVE)
 		argv_array_push(&args, "--interactive");
diff --git a/builtin/remote.c b/builtin/remote.c
index d95bf904c3b..b7d0f7ce596 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -306,6 +306,8 @@ static int config_read_branches(const char *key, const char *value, void *cb)
 				info->rebase = v;
 			else if (!strcmp(value, "preserve"))
 				info->rebase = NORMAL_REBASE;
+			else if (!strcmp(value, "recreate"))
+				info->rebase = NORMAL_REBASE;
 			else if (!strcmp(value, "interactive"))
 				info->rebase = INTERACTIVE_REBASE;
 		}
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 6893c3adabc..6f98c96fee9 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2182,7 +2182,7 @@ _git_config ()
 		return
 		;;
 	branch.*.rebase)
-		__gitcomp "false true preserve interactive"
+		__gitcomp "false true recreate preserve interactive"
 		return
 		;;
 	remote.pushdefault)
-- 
2.16.1.windows.1



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 12/12] rebase -i: introduce --recreate-merges=[no-]rebase-cousins
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
                       ` (10 preceding siblings ...)
  2018-02-11  0:10     ` [PATCH v3 11/12] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
@ 2018-02-11  0:10     ` Johannes Schindelin
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-11  0:10 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

This one is a bit tricky to explain, so let's try with a diagram:

        C
      /   \
A - B - E - F
  \   /
    D

To illustrate what this new mode is all about, let's consider what
happens upon `git rebase -i --recreate-merges B`, in particular to
the commit `D`. So far, the new branch structure would be:

       --- C' --
      /         \
A - B ------ E' - F'
      \    /
        D'

This is not really preserving the branch topology from before! The
reason is that the commit `D` does not have `B` as ancestor, and
therefore it gets rebased onto `B`.

This is unintuitive behavior. Even worse, when recreating branch
structure, most use cases would appear to want cousins *not* to be
rebased onto the new base commit. For example, Git for Windows (the
heaviest user of the Git garden shears, which served as the blueprint
for --recreate-merges) frequently merges branches from `next` early, and
these branches certainly do *not* want to be rebased. In the example
above, the desired outcome would look like this:

       --- C' --
      /         \
A - B ------ E' - F'
  \        /
   -- D' --

Let's introduce the term "cousins" for such commits ("D" in the
example), and let's not rebase them by default, introducing the new
"rebase-cousins" mode for use cases where they should be rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt      |  7 ++++++-
 builtin/rebase--helper.c          |  9 ++++++++-
 git-rebase--interactive.sh        |  1 +
 git-rebase.sh                     | 12 +++++++++++-
 sequencer.c                       |  4 ++++
 sequencer.h                       |  6 ++++++
 t/t3430-rebase-recreate-merges.sh | 23 +++++++++++++++++++++++
 7 files changed, 59 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index e9da7e26329..0e6d020d924 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -368,11 +368,16 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
---recreate-merges::
+--recreate-merges[=(rebase-cousins|no-rebase-cousins)]::
 	Recreate merge commits instead of flattening the history by replaying
 	merges. Merge conflict resolutions or manual amendments to merge
 	commits are not recreated automatically, but have to be recreated
 	manually.
++
+By default, or when `no-rebase-cousins` was specified, commits which do not
+have `<upstream>` as direct ancestor keep their original branch point.
+If the `rebase-cousins` mode is turned on, such commits are rebased onto
+`<upstream>` (or `<onto>`, if specified).
 
 -p::
 --preserve-merges::
diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index a34ab5c0655..cea99cb3235 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
 	unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
-	int abbreviate_commands = 0;
+	int abbreviate_commands = 0, rebase_cousins = -1;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
 		CHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH,
@@ -23,6 +23,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "ff", &opts.allow_ff, N_("allow fast-forward")),
 		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
 		OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge commits")),
+		OPT_BOOL(0, "rebase-cousins", &rebase_cousins,
+			 N_("keep original branch points of cousins")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -57,8 +59,13 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
 	flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
+	flags |= rebase_cousins > 0 ? TODO_LIST_REBASE_COUSINS : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
+	if (rebase_cousins >= 0 && !recreate_merges)
+		warning(_("--[no-]rebase-cousins has no effect without "
+			  "--recreate-merges"));
+
 	if (command == CONTINUE && argc == 1)
 		return !!sequencer_continue(&opts);
 	if (command == ABORT && argc == 1)
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index cfe3a537ac2..e199fe1cca5 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -903,6 +903,7 @@ if test t != "$preserve_merges"
 then
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
 		${recreate_merges:+--recreate-merges} \
+		${rebase_cousins:+--rebase-cousins} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 else
diff --git a/git-rebase.sh b/git-rebase.sh
index d69bc7d0e0d..58d778a2da0 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,7 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
-recreate-merges!   try to recreate merges instead of skipping them
+recreate-merges?   try to recreate merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -88,6 +88,7 @@ state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
 recreate_merges=
+rebase_cousins=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -268,6 +269,15 @@ do
 		recreate_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
 		;;
+	--recreate-merges=*)
+		recreate_merges=t
+		case "${1#*=}" in
+		rebase-cousins) rebase_cousins=t;;
+		no-rebase-cousins) rebase_cousins=;;
+		*) die "Unknown mode: $1";;
+		esac
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/sequencer.c b/sequencer.c
index 306ae014311..c877432d7b4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2931,6 +2931,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 				   unsigned flags)
 {
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	int rebase_cousins = flags & TODO_LIST_REBASE_COUSINS;
 	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
 	struct strbuf label = STRBUF_INIT;
 	struct commit_list *commits = NULL, **tail = &commits, *iter;
@@ -3106,6 +3107,9 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 					   &commit->object.oid);
 			if (entry)
 				to = entry->string;
+			else if (!rebase_cousins)
+				to = label_oid(&commit->object.oid, NULL,
+					       &state);
 
 			if (!to || !strcmp(to, "onto"))
 				fprintf(out, "%s onto\n", cmd_reset);
diff --git a/sequencer.h b/sequencer.h
index 11d1ac925ef..deebc6e3258 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -49,6 +49,12 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
 #define TODO_LIST_RECREATE_MERGES (1U << 3)
+/*
+ * When recreating merges, commits that do have the base commit as ancestor
+ * ("cousins") are *not* rebased onto the new base by default. If those
+ * commits should be rebased onto the new base, this flag needs to be passed.
+ */
+#define TODO_LIST_REBASE_COUSINS (1U << 4)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index 35a61ce90bb..9a59f12b670 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -143,6 +143,29 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'do not rebase cousins unless asked for' '
+	write_script copy-editor.sh <<-\EOF &&
+	cp "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	EOF
+
+	test_config sequence.editor \""$PWD"/copy-editor.sh\" &&
+	git checkout -b cousins master &&
+	before="$(git rev-parse --verify HEAD)" &&
+	test_tick &&
+	git rebase -i --recreate-merges HEAD^ &&
+	test_cmp_rev HEAD $before &&
+	test_tick &&
+	git rebase -i --recreate-merges=rebase-cousins HEAD^ &&
+	test_cmp_graph HEAD^.. <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	|/
+	o H
+	EOF
+'
+
 test_expect_success 'refs/rewritten/* is worktree-local' '
 	git worktree add wt &&
 	cat >wt/script-from-scratch <<-\EOF &&
-- 
2.16.1.windows.1

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-09  7:13         ` Johannes Sixt
@ 2018-02-11 10:16           ` Jacob Keller
  2018-02-12  7:38           ` Sergey Organov
  1 sibling, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-02-11 10:16 UTC (permalink / raw)
  To: Johannes Sixt
  Cc: Sergey Organov, Johannes Schindelin, Git mailing list, Junio C Hamano

On Thu, Feb 8, 2018 at 11:13 PM, Johannes Sixt <j6t@kdbg.org> wrote:
> Am 09.02.2018 um 07:11 schrieb Sergey Organov:
>>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>>>
>>> Let me explain the scenario which comes up plenty of times in my work
>>> with
>>> Git for Windows. We have a thicket of some 70 branches on top of
>>> git.git's
>>> latest release. These branches often include fixup! and squash! commits
>>> and even more complicated constructs that rebase cannot handle at all at
>>> the moment, such as reorder-before! and reorder-after! (for commits that
>>> really need to go into a different branch).
>>
>>
>> I sympathize, but a solution that breaks even in simple cases can't be
>> used reliably to solve more complex problems, sorry. Being so deep
>> into your problems, I think you maybe just aren't seeing forest for the
>> trees [1].
>
>
> Hold your horses! Dscho has a point here. --preserve-merges --first-parent
> works only as long as you don't tamper with the side branches. If you make
> changes in the side branches during the same rebase operation, this
> --first-parent mode would undo that change. (And, yes, its result would be
> called an "evil merge", and that scary name _should_ frighten you!)
>
> -- Hannes

This is the reason I agree with Johannes, in regards to why
recreate-merges approach is correct.

Yes, an ideal system would be one which correctly, automatically
re-creates the merge *as if* a human had re-merged the two newly
re-created side branches, and preserves any changes in the result of
the merge, such as cases we call "evil merges" which includes
necessary changes to resolve conflicts properly.

However, I would state that such a system, in order to cause the least
surprise to a user must be correct against arbitrary removal, reorder,
and addition of new commits on both the main and topic side branches
for which we are re-creating the merges.

This is problematic, because something like how --preserve-merges
--first-parent does not work under this case.

As a user of the tool, I may be biased because I already read and
understand how recreate-merges is expected to work, but it makes sense
to me that the re-creation of the merge merely re-does the merge and
any modfications in the original would have to be carried over.

I don't know what process we could use to essentially move the changes
from the original merge into the new copy. What ever solution we have
would need to have a coherent user interface and be presentable in
some manner.

One way to think about the contents we're wanting to keep, rather than
the full tree result of the merge which we had before, what we
actually want to keep in some sense is the resulting "diff" as shown
by something like the condensed --combined output. This is obviously
not really a diff that we can apply.

And even if we could apply it, since the merge is occurring, we can't
exactly use 3-way merge conflict in order to actually apply the old
changes into the new merged setup? Could something like rerere logic
work here to track what was done and then re-apply it to the new merge
we create? And once we apply it, we need to be able to handle any
conflicts that occur because of deleting, adding, or re-ordering
commits on the branches we're merging... so in some sense we could
have "conflicts of conflicts" which is a scenario that I don't yet
have a good handle on how this would be presented to the user.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-10 23:06     ` Johannes Schindelin
@ 2018-02-12  4:58       ` Sergey Organov
  2018-02-12 20:21         ` Johannes Schindelin
  2018-02-12  5:22       ` Sergey Organov
  1 sibling, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-02-12  4:58 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Jacob Keller

Hi Johannes,

Thanks for explanations, and could you please answer this one:

[...]

>> I also have trouble making sense of "Recreate merge commits instead of
>> flattening the history by replaying merges." Is it "<Recreate merge
>> commits by replaying merges> instead of <flattening the history>" or is it
>> rather "<Recreate merge commits> instead of <flattening the history by
>> replaying merges>?

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-10 23:06     ` Johannes Schindelin
  2018-02-12  4:58       ` Sergey Organov
@ 2018-02-12  5:22       ` Sergey Organov
  2018-02-12 20:39         ` Johannes Schindelin
  1 sibling, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-02-12  5:22 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Jacob Keller

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi Sergey,
>
> On Fri, 9 Feb 2018, Sergey Organov wrote:
>
>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> 
>> [...]
>> 
>> > With this patch, the goodness of the Git garden shears comes to `git
>> > rebase -i` itself. Passing the `--recreate-merges` option will generate
>> > a todo list that can be understood readily, and where it is obvious
>> > how to reorder commits. New branches can be introduced by inserting
>> > `label` commands and calling `merge - <label> <oneline>`. And once this
>> > mode has become stable and universally accepted, we can deprecate the
>> > design mistake that was `--preserve-merges`.
>> 
>> This doesn't explain why you introduced this new --recreate-merges. Why
>> didn't you rather fix --preserve-merges to generate and use new todo
>> list format?
>
> Because that would of course break existing users of
> --preserve-merges.

How exactly? Doesn't "--recreate-merges" produce the same result as
"--preserve-merges" if run non-interactively?

> So why not --preserve-merges=v2? Because that would force me to maintain
> --preserve-merges forever. And I don't want to.
>
>> It doesn't seem likely that todo list created by one Git version is to
>> be ever used by another, right?
>
> No. But by scripts based on `git rebase -p`.
>
>> Is there some hidden reason here? Some tools outside of Git that use old
>> todo list format, maybe?
>
> Exactly.
>
> I did mention such a tool: the Git garden shears:
>
> 	https://github.com/git-for-windows/build-extra/blob/master/shears.sh
>
> Have a look at it. It will inform the discussion.

I've searched for "-p" in the script, but didn't find positives for
either "-p" or "--preserve-merges". How it would break if it doesn't use
them? What am I missing?

>
>> Then, if new option indeed required, please look at the resulting manual:
>> 
>> --recreate-merges::
>> 	Recreate merge commits instead of flattening the history by replaying
>> 	merges. Merge conflict resolutions or manual amendments to merge
>> 	commits are not preserved.
>> 
>> -p::
>> --preserve-merges::
>> 	Recreate merge commits instead of flattening the history by replaying
>> 	commits a merge commit introduces. Merge conflict resolutions or manual
>> 	amendments to merge commits are not preserved.
>
> As I stated in the cover letter, there are more patches lined up after
> this patch series.

Good, but I thought this one should better be self-consistent anyway.
What if those that come later aren't included?

>
> Have a look at https://github.com/git/git/pull/447, especially the latest
> commit in there which is an early version of the deprecation I intend to
> bring about.

You shouldn't want a deprecation at all should you have re-used
--preserve-merges in the first place, and I still don't see why you
haven't. 

>
> Also, please refrain from saying things like... "Don't you think ..."
>
> If you don't like the wording, I wold much more appreciate it if a better
> alternative was suggested.

Sorry, but how can I suggest one if I don't understand what you are
doing here in the first place? That's why I ask you.

>
>> Don't you think more explanations are needed there in the manual on
>> why do we have 2 separate options with almost the same yet subtly
>> different description? Is this subtle difference even important? How?
>> 
>> I also have trouble making sense of "Recreate merge commits instead of
>> flattening the history by replaying merges." Is it "<Recreate merge
>> commits by replaying merges> instead of <flattening the history>" or is it
>> rather "<Recreate merge commits> instead of <flattening the history by
>> replaying merges>?
>
> The documentation of the --recreate-merges option is not meant to explain
> the difference to --preserve-merges. It is meant to explain the difference
> to regular `git rebase -i`, which flattens the commit history into a
> single branch without merge commits (in fact, all merge commits are simply
> ignored).

Yeah, that's obvious, but the point is that resulting manual is ended
up being confusing.

> And I would rather not start to describe the difference between
> --recreate-merges and --preserve-merges because I want to deprecate the
> latter, and describing the difference as I get the sense is your wish
> would simply mean more work because it would have to be added and then
> removed again.

I suspect you actually didn't need those new option in the first place,
and that's the core reason of these troubles.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-09  7:13         ` Johannes Sixt
  2018-02-11 10:16           ` Jacob Keller
@ 2018-02-12  7:38           ` Sergey Organov
  1 sibling, 0 replies; 412+ messages in thread
From: Sergey Organov @ 2018-02-12  7:38 UTC (permalink / raw)
  To: Johannes Sixt; +Cc: Johannes Schindelin, git, Junio C Hamano, Jacob Keller

Johannes Sixt <j6t@kdbg.org> writes:

> Am 09.02.2018 um 07:11 schrieb Sergey Organov:
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>>> Let me explain the scenario which comes up plenty of times in my work with
>>> Git for Windows. We have a thicket of some 70 branches on top of git.git's
>>> latest release. These branches often include fixup! and squash! commits
>>> and even more complicated constructs that rebase cannot handle at all at
>>> the moment, such as reorder-before! and reorder-after! (for commits that
>>> really need to go into a different branch).
>>
>> I sympathize, but a solution that breaks even in simple cases can't be
>> used reliably to solve more complex problems, sorry. Being so deep
>> into your problems, I think you maybe just aren't seeing forest for the
>> trees [1].
>
> Hold your horses! Dscho has a point here. --preserve-merges
> --first-parent works only as long as you don't tamper with the side
> branches. If you make changes in the side branches during the same
> rebase operation, this --first-parent mode would undo that change.

He has a point indeed, but it must not be used as an excuse to silently
damage user data, as if there are no other options!

Simple --first-parent won't always fit, it's obvious. I used
--first-parent patch as mere illustration of concept, it's rather
"rebase [-i] --keep-the-f*g-shape" itself that should behave. There
should be no need for actual --first-parent that only fits
no-manual-editing use-cases.

Look at it as if it's a scale where --first-parent is on one side, and
"blind re-merge" is on the other. The right answer(s) lie somewhere
in-between, but I think they are much closer to --first-parent than they
are to "blind re-merge".

> (And, yes, its result would be called an "evil merge", and that scary
> name _should_ frighten you!)

(It won't always be "evil merge", and it still doesn't frighten even if
it will, provided git stops making them more evil then they actually
deserve, and it isn't an excuse to silently distort user data anyway!)

-- Sergey

[1] The "--first-parent" here would rather keep that change from
propagation to the main-line, not undo it, and sometimes it's even the
right thing to do ("-x ours" for the original merge being one example).
Frequently though it is needed on main-line indeed, and there should be
a way to tell git to propagate the change to the main-line, but even
then automatic blind unattended re-merge is wrong answer and I'm sure
git can be made to do better than that.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 05/12] sequencer: introduce the `merge` command
  2018-02-11  0:10     ` [PATCH v3 05/12] sequencer: introduce the `merge` command Johannes Schindelin
@ 2018-02-12  8:48       ` Eric Sunshine
  2018-02-12 20:17         ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Eric Sunshine @ 2018-02-12  8:48 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Phillip Wood

On Sat, Feb 10, 2018 at 7:10 PM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> This patch is part of the effort to reimplement `--preserve-merges` with
> a substantially improved design, a design that has been developed in the
> Git for Windows project to maintain the dozens of Windows-specific patch
> series on top of upstream Git.
>
> The previous patch implemented the `label` and `reset` commands to label
> commits and to reset to a labeled commits. This patch adds the `merge`

s/to a/to/

> command, with the following syntax:
>
>         merge [-C <commit>] <rev> # <oneline>
>
> The <commit> parameter in this instance is the *original* merge commit,
> whose author and message will be used for the merge commit that is about
> to be created.
>
> The <rev> parameter refers to the (possibly rewritten) revision to
> merge. Let's see an example of a todo list:
>
>         label onto
>
>         # Branch abc
>         reset onto
>         pick deadbeef Hello, world!
>         label abc
>
>         reset onto
>         pick cafecafe And now for something completely different
>         merge -C baaabaaa abc # Merge the branch 'abc' into master
>
> To edit the merge commit's message (a "reword" for merges, if you will),
> use `-c` (lower-case) instead of `-C`; this convention was borrowed from
> `git commit` that also supports `-c` and `-C` with similar meanings.
>
> To create *new* merges, i.e. without copying the commit message from an
> existing commit, simply omit the `-C <commit>` parameter (which will
> open an editor for the merge message):
>
>         merge abc
>
> This comes in handy when splitting a branch into two or more branches.
>
> Note: this patch only adds support for recursive merges, to keep things
> simple. Support for octopus merges will be added later in a separate
> patch series, support for merges using strategies other than the
> recursive merge is left for the future.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 04/12] sequencer: introduce new commands to reset the revision
  2018-02-11  0:10     ` [PATCH v3 04/12] sequencer: introduce new commands to reset the revision Johannes Schindelin
@ 2018-02-12 19:26       ` Eric Sunshine
  2018-02-12 20:46         ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Eric Sunshine @ 2018-02-12 19:26 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Phillip Wood

On Sat, Feb 10, 2018 at 7:10 PM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> [...]
> This commit implements the commands to label, and to reset to, given
> revisions. The syntax is:
>
>         label <name>
>         reset <name>
> [...]
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> diff --git a/sequencer.c b/sequencer.c
> @@ -1922,6 +1951,151 @@ static int do_exec(const char *command_line)
> +static int safe_append(const char *filename, const char *fmt, ...)
> +{
> +       [...]
> +       if (write_in_full(fd, buf.buf, buf.len) < 0) {
> +               error_errno(_("could not write to '%s'"), filename);
> +               rollback_lock_file(&lock);

strbuf_release(&buf);

> +               return -1;
> +       }
> +       if (commit_lock_file(&lock) < 0) {
> +               rollback_lock_file(&lock);

strbuf_release(&buf);

> +               return error(_("failed to finalize '%s'"), filename);
> +       }
> +

strbuf_release(&buf);

> +       return 0;
> +}
> +
> +static int do_reset(const char *name, int len, struct replay_opts *opts)
> +{
> +       [...]
> +       unpack_tree_opts.reset = 1;
> +
> +       if (read_cache_unmerged())

rollback_lock_file(&lock);
strbuf_release(&ref_name);

> +               return error_resolve_conflict(_(action_name(opts)));
> +
> +       if (!fill_tree_descriptor(&desc, &oid)) {
> +               error(_("failed to find tree of %s"), oid_to_hex(&oid));
> +               rollback_lock_file(&lock);
> +               free((void *)desc.buffer);
> +               strbuf_release(&ref_name);
> +               return -1;
> +       }

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 05/12] sequencer: introduce the `merge` command
  2018-02-12  8:48       ` Eric Sunshine
@ 2018-02-12 20:17         ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-12 20:17 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Phillip Wood

Hi Eric,

On Mon, 12 Feb 2018, Eric Sunshine wrote:

> On Sat, Feb 10, 2018 at 7:10 PM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > This patch is part of the effort to reimplement `--preserve-merges` with
> > a substantially improved design, a design that has been developed in the
> > Git for Windows project to maintain the dozens of Windows-specific patch
> > series on top of upstream Git.
> >
> > The previous patch implemented the `label` and `reset` commands to label
> > commits and to reset to a labeled commits. This patch adds the `merge`
> 
> s/to a/to/

Fixed locally. Will be part of the next iteration, if one is necessary.
Otherwise I will first ask Junio whether he can touch up the commit
message before applying.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-12  4:58       ` Sergey Organov
@ 2018-02-12 20:21         ` Johannes Schindelin
  2018-02-13  6:44           ` Sergey Organov
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-12 20:21 UTC (permalink / raw)
  To: Sergey Organov; +Cc: git, Junio C Hamano, Jacob Keller

Hi Sergey,

On Mon, 12 Feb 2018, Sergey Organov wrote:

> Thanks for explanations, and could you please answer this one:
> 
> [...]
> 
> >> I also have trouble making sense of "Recreate merge commits instead of
> >> flattening the history by replaying merges." Is it "<Recreate merge
> >> commits by replaying merges> instead of <flattening the history>" or is it
> >> rather "<Recreate merge commits> instead of <flattening the history by
> >> replaying merges>?

I thought I had answered that one.

Flattening the history is what happens in regular rebase (i.e. without
--recreate-merges and without --preserve-merges).

The idea to recreate merges is of course to *not* flatten the history.

Maybe there should have been a comma after "history" to clarify what the
sentence means.

The wording is poor either way, but you are also not a native speaker so
we have to rely on, say, Eric to help us out here.

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-12  5:22       ` Sergey Organov
@ 2018-02-12 20:39         ` Johannes Schindelin
  2018-02-13  4:39           ` Jacob Keller
  2018-02-13  6:43           ` Sergey Organov
  0 siblings, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-12 20:39 UTC (permalink / raw)
  To: Sergey Organov; +Cc: git, Junio C Hamano, Jacob Keller

Hi Sergey,

On Mon, 12 Feb 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >
> > On Fri, 9 Feb 2018, Sergey Organov wrote:
> >
> >> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> >> 
> >> [...]
> >> 
> >> > With this patch, the goodness of the Git garden shears comes to `git
> >> > rebase -i` itself. Passing the `--recreate-merges` option will generate
> >> > a todo list that can be understood readily, and where it is obvious
> >> > how to reorder commits. New branches can be introduced by inserting
> >> > `label` commands and calling `merge - <label> <oneline>`. And once this
> >> > mode has become stable and universally accepted, we can deprecate the
> >> > design mistake that was `--preserve-merges`.
> >> 
> >> This doesn't explain why you introduced this new --recreate-merges. Why
> >> didn't you rather fix --preserve-merges to generate and use new todo
> >> list format?
> >
> > Because that would of course break existing users of
> > --preserve-merges.
> 
> How exactly?

Power users of interactive rebase use scripting to augment Git's
functionality. One particularly powerful trick is to override
GIT_SEQUENCER_EDITOR with an invocation of such a script, to perform
automated edits. Such a script breaks when we change the format of the
content to edit. If we change the format of the todo list generated in
--preserve-merges mode, that is exactly what happens. We break existing
users.

BTW it seems that you did not really read my previous reply carefully
because I referenced such a use case: the Git garden shears. They do
override the sequencer editor, and while they do not exactly edit the todo
list (they simply through the generated one away), they generate a new
todo list and would break if that format changes. Of course, the shears do
not use the --preserve-merges mode, but from just reading about the way
how the Git garden shears work, it is quite obvious how similar users of
--preserve-merges are likely to exist?

> Doesn't "--recreate-merges" produce the same result as
> "--preserve-merges" if run non-interactively?

The final result of a rebase where you do not edit the todo list? Should
be identical, indeed.

But that is the most boring, most uninteresting, and least important use
case. So we might just as well forget about it when we focus on keeping
Git's usage stable.

> > So why not --preserve-merges=v2? Because that would force me to
> > maintain --preserve-merges forever. And I don't want to.
> >
> >> It doesn't seem likely that todo list created by one Git version is
> >> to be ever used by another, right?
> >
> > No. But by scripts based on `git rebase -p`.
> >
> >> Is there some hidden reason here? Some tools outside of Git that use
> >> old todo list format, maybe?
> >
> > Exactly.
> >
> > I did mention such a tool: the Git garden shears:
> >
> > 	https://github.com/git-for-windows/build-extra/blob/master/shears.sh
> >
> > Have a look at it. It will inform the discussion.
> 
> I've searched for "-p" in the script, but didn't find positives for
> either "-p" or "--preserve-merges". How it would break if it doesn't use
> them? What am I missing?

*This* particular script does not use -p.

But it is not *this* particular script that I do not want to break! It is
*all* scripts that use interactive rebase! Don't you also care about not
breaking existing users?

> >> Then, if new option indeed required, please look at the resulting manual:
> >> 
> >> --recreate-merges::
> >> 	Recreate merge commits instead of flattening the history by replaying
> >> 	merges. Merge conflict resolutions or manual amendments to merge
> >> 	commits are not preserved.
> >> 
> >> -p::
> >> --preserve-merges::
> >> 	Recreate merge commits instead of flattening the history by replaying
> >> 	commits a merge commit introduces. Merge conflict resolutions or manual
> >> 	amendments to merge commits are not preserved.
> >
> > As I stated in the cover letter, there are more patches lined up after
> > this patch series.
> 
> Good, but I thought this one should better be self-consistent anyway.
> What if those that come later aren't included?

Right, let's just rip apart the partial progress because the latter
patches might not make it in?

I cannot work on that basis, and I also do not want to work on that basis.

If you do not like how the documentation is worded, fine, suggest a better
alternative.

> > Have a look at https://github.com/git/git/pull/447, especially the
> > latest commit in there which is an early version of the deprecation I
> > intend to bring about.
> 
> You shouldn't want a deprecation at all should you have re-used
> --preserve-merges in the first place, and I still don't see why you
> haven't. 

Keep repeating it, and it won't become truer.

If you break formats, you break scripts. Git has *so* many users, there
are very likely some who script *every* part of it.

We simply cannot do that.

What we can is deprecate designs which we learned on the way were not only
incomplete from the get-go, but bad overall and hard (or impossible) to
fix. Like --preserve-merges.

Or for that matter like the design you proposed, to use --first-parent for
--recreate-merges. Or to use --first-parent for some --recreate-merges,
surprising users in very bad ways when it is not used (or when it is
used). I get the impression that you still think it would be a good idea,
even if it should be obvious that it is not.

> > Also, please refrain from saying things like... "Don't you think ..."
> >
> > If you don't like the wording, I wold much more appreciate it if a better
> > alternative was suggested.
> 
> Sorry, but how can I suggest one if I don't understand what you are
> doing here in the first place? That's why I ask you.

There are ways to put the person you ask on trial. And there are ways to
genuinely show interest and seek education.

I am a really poor example how to communicate properly, of course, so
don't try to learn from me. I am trying myself to learn better ways to
express what I mean clearly, and to express it in a direct yet kind
manner.

> >> Don't you think more explanations are needed there in the manual on
> >> why do we have 2 separate options with almost the same yet subtly
> >> different description? Is this subtle difference even important? How?
> >> 
> >> I also have trouble making sense of "Recreate merge commits instead of
> >> flattening the history by replaying merges." Is it "<Recreate merge
> >> commits by replaying merges> instead of <flattening the history>" or is it
> >> rather "<Recreate merge commits> instead of <flattening the history by
> >> replaying merges>?
> >
> > The documentation of the --recreate-merges option is not meant to explain
> > the difference to --preserve-merges. It is meant to explain the difference
> > to regular `git rebase -i`, which flattens the commit history into a
> > single branch without merge commits (in fact, all merge commits are simply
> > ignored).
> 
> Yeah, that's obvious, but the point is that resulting manual is ended
> up being confusing.

Again, just saying something is bad, is bad. Saying something leaves room
for improvement and then suggesting how to improve it, is good.

> > And I would rather not start to describe the difference between
> > --recreate-merges and --preserve-merges because I want to deprecate the
> > latter, and describing the difference as I get the sense is your wish
> > would simply mean more work because it would have to be added and then
> > removed again.
> 
> I suspect you actually didn't need those new option in the first place,
> and that's the core reason of these troubles.

Are you suspecting that I, myself, do not use --recreate-merges?

If so, please read the cover letter again, in particular the part where I
describe how this entire series of patch series arose from the Git garden
shears, which I invented myself to help with maintaining Git for Windows,
and which I use for five years now. This should help disperse that
suspicion rather quickly: the intent of --recreate-merges is to allow me
to simplify the shears by quite a bit, and maybe eventually even get rid
of the script altogether (if I ever manage to convince myself that the
concept of a merging-rebase should be official enough to enter core Git).

I am a heavy user of --recreate-merges, even if it does not really exist
yet. I have five years of experience with it, which is the reason why I am
so confident about its design, and why I can tell you a lot about typical
use cases and common pitfalls, and where the original design had to be
adjusted.

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 04/12] sequencer: introduce new commands to reset the revision
  2018-02-12 19:26       ` Eric Sunshine
@ 2018-02-12 20:46         ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-12 20:46 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Phillip Wood

Hi Eric,

On Mon, 12 Feb 2018, Eric Sunshine wrote:

> On Sat, Feb 10, 2018 at 7:10 PM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > [...]
> > This commit implements the commands to label, and to reset to, given
> > revisions. The syntax is:
> >
> >         label <name>
> >         reset <name>
> > [...]
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> > diff --git a/sequencer.c b/sequencer.c
> > @@ -1922,6 +1951,151 @@ static int do_exec(const char *command_line)
> > +static int safe_append(const char *filename, const char *fmt, ...)
> > +{
> > +       [...]
> > +       if (write_in_full(fd, buf.buf, buf.len) < 0) {
> > +               error_errno(_("could not write to '%s'"), filename);
> > +               rollback_lock_file(&lock);
> 
> strbuf_release(&buf);
> 
> > +               return -1;
> > +       }
> > +       if (commit_lock_file(&lock) < 0) {
> > +               rollback_lock_file(&lock);
> 
> strbuf_release(&buf);
> 
> > +               return error(_("failed to finalize '%s'"), filename);
> > +       }
> > +
> 
> strbuf_release(&buf);
> 
> > +       return 0;
> > +}
> > +
> > +static int do_reset(const char *name, int len, struct replay_opts *opts)
> > +{
> > +       [...]
> > +       unpack_tree_opts.reset = 1;
> > +
> > +       if (read_cache_unmerged())
> 
> rollback_lock_file(&lock);
> strbuf_release(&ref_name);

Thank you very much! I fixed these locally and force-pushed the
recreate-merges branch to https://github.com/dscho/git. These fixes will
be part of v4.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-12 20:39         ` Johannes Schindelin
@ 2018-02-13  4:39           ` Jacob Keller
  2018-02-13  7:15             ` Sergey Organov
  2018-02-13  6:43           ` Sergey Organov
  1 sibling, 1 reply; 412+ messages in thread
From: Jacob Keller @ 2018-02-13  4:39 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Sergey Organov, Git mailing list, Junio C Hamano

On Mon, Feb 12, 2018 at 12:39 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Sergey,
>
> On Mon, 12 Feb 2018, Sergey Organov wrote:
>> > Have a look at https://github.com/git/git/pull/447, especially the
>> > latest commit in there which is an early version of the deprecation I
>> > intend to bring about.
>>
>> You shouldn't want a deprecation at all should you have re-used
>> --preserve-merges in the first place, and I still don't see why you
>> haven't.
>
> Keep repeating it, and it won't become truer.
>
> If you break formats, you break scripts. Git has *so* many users, there
> are very likely some who script *every* part of it.
>
> We simply cannot do that.
>
> What we can is deprecate designs which we learned on the way were not only
> incomplete from the get-go, but bad overall and hard (or impossible) to
> fix. Like --preserve-merges.
>
> Or for that matter like the design you proposed, to use --first-parent for
> --recreate-merges. Or to use --first-parent for some --recreate-merges,
> surprising users in very bad ways when it is not used (or when it is
> used). I get the impression that you still think it would be a good idea,
> even if it should be obvious that it is not.

If we consider the addition of new todo list elements as "user
breaking", then yes this change would be user-script breaking.

Since we did not originally spell out that todo-list items are subject
to enhancement by addition of operations in the future, scripts are
likely not designed to allow addition of new elements.

Thus, adding recreate-merges, and deprecating preserve-merges, seems
to me to be the correct action to take here.

One could argue that users should have expected new todo list elements
to be added in the future and thus design their scripts to cope with
such a thing. If you can convincingly argue this, then I don't
necessarily see it as a complete user breaking change to fix
preserve-merges in order to allow it to handle re-ordering properly..

I think I lean towards agreeing with Johannes, and that adding
recreate-merges and removing preserve-merges is the better solution.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-12 20:39         ` Johannes Schindelin
  2018-02-13  4:39           ` Jacob Keller
@ 2018-02-13  6:43           ` Sergey Organov
  2018-02-15  1:40             ` Johannes Schindelin
  1 sibling, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-02-13  6:43 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Jacob Keller

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi Sergey,
>
> On Mon, 12 Feb 2018, Sergey Organov wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> >
>> > On Fri, 9 Feb 2018, Sergey Organov wrote:
>> >
>> >> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> >> 
>> >> [...]
>> >> 
>> >> > With this patch, the goodness of the Git garden shears comes to `git
>> >> > rebase -i` itself. Passing the `--recreate-merges` option will generate
>> >> > a todo list that can be understood readily, and where it is obvious
>> >> > how to reorder commits. New branches can be introduced by inserting
>> >> > `label` commands and calling `merge - <label> <oneline>`. And once this
>> >> > mode has become stable and universally accepted, we can deprecate the
>> >> > design mistake that was `--preserve-merges`.
>> >> 
>> >> This doesn't explain why you introduced this new --recreate-merges. Why
>> >> didn't you rather fix --preserve-merges to generate and use new todo
>> >> list format?
>> >
>> > Because that would of course break existing users of
>> > --preserve-merges.
>> 
>> How exactly?
>
> Power users of interactive rebase use scripting to augment Git's
> functionality. One particularly powerful trick is to override
> GIT_SEQUENCER_EDITOR with an invocation of such a script, to perform
> automated edits. Such a script breaks when we change the format of the
> content to edit. If we change the format of the todo list generated in
> --preserve-merges mode, that is exactly what happens. We break existing
> users.

I didn't say a word against "--preserve-merges mode", whatever it is,
only about re-using "--preserve-merges" command-line option to "git
rebase", the git user interface. I'm sure you see the difference? Unless
there are out-of-git scripts that do use "git rebase --preserve-merges"
and simultaneously do rely on the todo list format this exact command
generates, there should be no breakage of existing users caused by
changing todo list format generated by  "git rebase --preserve-merges".

Old broken "--preserve-merges mode" could be then kept in the
implementation for ages, unused by the new fixed "git rebase
--preserve-merge", for the sake of compatibility.

> BTW it seems that you did not really read my previous reply carefully
> because I referenced such a use case: the Git garden shears.

I thought I did. You confirm below that this script doesn't use "git
rebase --preserve-merges" in the first place, nor will it break if "git
rebase --preserve-merges" starts to generate new todo format, yet you
expected I'd readily see how it's relevant? No, I'm not that clever, nor
am I a mind-reader.

> They do override the sequencer editor, and while they do not exactly
> edit the todo list (they simply through the generated one away), they
> generate a new todo list and would break if that format changes. Of
> course, the shears do not use the --preserve-merges mode,
> but from just reading about the way how the Git garden shears work, it
> is quite obvious how similar users of --preserve-merges are likely to
> exist?

Maybe, I dunno. If even "garden shears" won't break, then what will? Do
you know an example?

Anyway, as it seems it's too late already for such a change, let me stop
this and assume there are indeed such scripts that will break and that
it's indeed a good idea to introduce new option. Case closed. The manual
should still be fixed though, I think.

>> Doesn't "--recreate-merges" produce the same result as
>> "--preserve-merges" if run non-interactively?
>
> The final result of a rebase where you do not edit the todo list? Should
> be identical, indeed.

That's good to hear.

> But that is the most boring, most uninteresting, and least important use
> case.

For you. Do you suddenly stop caring about compatibility?

> So we might just as well forget about it when we focus on keeping
> Git's usage stable.

Why? It's good it behaves the same, so --preserve-merges could indeed be
deprecated, as you apparently intend.

>> > So why not --preserve-merges=v2? Because that would force me to
>> > maintain --preserve-merges forever. And I don't want to.
>> >
>> >> It doesn't seem likely that todo list created by one Git version is
>> >> to be ever used by another, right?
>> >
>> > No. But by scripts based on `git rebase -p`.
>> >
>> >> Is there some hidden reason here? Some tools outside of Git that use
>> >> old todo list format, maybe?
>> >
>> > Exactly.
>> >
>> > I did mention such a tool: the Git garden shears:
>> >
>> > 	https://github.com/git-for-windows/build-extra/blob/master/shears.sh
>> >
>> > Have a look at it. It will inform the discussion.
>> 
>> I've searched for "-p" in the script, but didn't find positives for
>> either "-p" or "--preserve-merges". How it would break if it doesn't use
>> them? What am I missing?
>
> *This* particular script does not use -p.
>
> But it is not *this* particular script that I do not want to break!

I thought that was an example of a tool that would break. Well, it won't
break. Good.

> It is *all* scripts that use interactive rebase!

I'm really interested, and here I *do* ask for education. What are
those? As I now only ask this out of curiosity, and don't argue
--recreate-merges anymore, are you finally willing to reveal the
information?

> Don't you also care about not breaking existing users?

I do care. I just suspected they are very unlikely to exist, and I do
want to be educated in this matter indeed, as they could be rather
interesting.

[Please notice violation of your own standard of not using "Don't
you...", not that I care myself.]

>> >> Then, if new option indeed required, please look at the resulting manual:
>> >> 
>> >> --recreate-merges::
>> >> 	Recreate merge commits instead of flattening the history by replaying
>> >> 	merges. Merge conflict resolutions or manual amendments to merge
>> >> 	commits are not preserved.
>> >> 
>> >> -p::
>> >> --preserve-merges::
>> >> 	Recreate merge commits instead of flattening the history by replaying
>> >> 	commits a merge commit introduces. Merge conflict resolutions or manual
>> >> 	amendments to merge commits are not preserved.
>> >
>> > As I stated in the cover letter, there are more patches lined up after
>> > this patch series.
>> 
>> Good, but I thought this one should better be self-consistent anyway.
>> What if those that come later aren't included?
>
> Right, let's just rip apart the partial progress because the latter
> patches might not make it in?

No, let's fix it instead.

>
> I cannot work on that basis, and I also do not want to work on that basis.
>
> If you do not like how the documentation is worded, fine, suggest a better
> alternative.

I suggested to re-use --preserve-merges command-line option to "git
rebase", unless there are actual users that would break. But as you
believe that's wrong idea, then it could be something like this in the
manual:

--recreate-merges::
	Recreate merge commits instead of flattening the history. Merge
	conflict resolutions or manual amendments to merge commits are
	not preserved. 

-p::
--preserve-merges::
	This option is similar to --recreate-merges, but doesn't
        support interactive mode properly. This option is deprecated,
        use --recreate-merges instead.

>
>> > Have a look at https://github.com/git/git/pull/447, especially the
>> > latest commit in there which is an early version of the deprecation I
>> > intend to bring about.
>> 
>> You shouldn't want a deprecation at all should you have re-used
>> --preserve-merges in the first place, and I still don't see why you
>> haven't. 
>
> Keep repeating it, and it won't become truer.

It is just my point that I repeat, and you gave no evidence it is false,
so I assume it's true, unless proved otherwise.

[...]

> Or for that matter like the design you proposed, to use --first-parent for
> --recreate-merges. Or to use --first-parent for some --recreate-merges,
> surprising users in very bad ways when it is not used (or when it is
> used). I get the impression that you still think it would be a good idea,
> even if it should be obvious that it is not.

What you describe here is bad idea indeed, but it has little to do with
what I actually have in mind and what you apparently don't want to even
try to understand.

>> > Also, please refrain from saying things like... "Don't you think ..."
>> >
>> > If you don't like the wording, I wold much more appreciate it if a better
>> > alternative was suggested.
>> 
>> Sorry, but how can I suggest one if I don't understand what you are
>> doing here in the first place? That's why I ask you.
>
> There are ways to put the person you ask on trial. And there are ways to
> genuinely show interest and seek education.

I didn't seek education, nor did I intend any trial. I asked for
clarification of the patch to the manual page that you wrote in a way
that made resulting manual page confusing for me. Confusing manual is
often indication of some additional problem(s) elsewhere, that's what
I've learned for sure, from multiple occasions, so I did reveal my
doubts.

> I am a really poor example how to communicate properly, of course, so
> don't try to learn from me. I am trying myself to learn better ways to
> express what I mean clearly, and to express it in a direct yet kind
> manner.
>
>> >> Don't you think more explanations are needed there in the manual on
>> >> why do we have 2 separate options with almost the same yet subtly
>> >> different description? Is this subtle difference even important? How?
>> >> 
>> >> I also have trouble making sense of "Recreate merge commits instead of
>> >> flattening the history by replaying merges." Is it "<Recreate merge
>> >> commits by replaying merges> instead of <flattening the history>" or is it
>> >> rather "<Recreate merge commits> instead of <flattening the history by
>> >> replaying merges>?
>> >
>> > The documentation of the --recreate-merges option is not meant to explain
>> > the difference to --preserve-merges. It is meant to explain the difference
>> > to regular `git rebase -i`, which flattens the commit history into a
>> > single branch without merge commits (in fact, all merge commits are simply
>> > ignored).
>> 
>> Yeah, that's obvious, but the point is that resulting manual is ended
>> up being confusing.
>
> Again, just saying something is bad, is bad. Saying something leaves room
> for improvement and then suggesting how to improve it, is good.

Please see wording suggestion above.

>> > And I would rather not start to describe the difference between
>> > --recreate-merges and --preserve-merges because I want to deprecate the
>> > latter, and describing the difference as I get the sense is your wish
>> > would simply mean more work because it would have to be added and then
>> > removed again.
>> 
>> I suspect you actually didn't need those new option in the first place,
>> and that's the core reason of these troubles.
>
> Are you suspecting that I, myself, do not use --recreate-merges?

I suspect that if you've had rather changed --preserve-merges, you'd
happily use it and no --recreate-merges were ever necessary. You did
what you did, and it seems to be too late to ask for changing it back,
exactly due to heavy use of this new option.

> If so, please read the cover letter again, in particular the part where I
> describe how this entire series of patch series arose from the Git garden
> shears, which I invented myself to help with maintaining Git for Windows,
> and which I use for five years now. This should help disperse that
> suspicion rather quickly: the intent of --recreate-merges is to allow me
> to simplify the shears by quite a bit, and maybe eventually even get rid
> of the script altogether (if I ever manage to convince myself that the
> concept of a merging-rebase should be official enough to enter core Git).
>
> I am a heavy user of --recreate-merges, even if it does not really exist
> yet. I have five years of experience with it, which is the reason why I am
> so confident about its design, and why I can tell you a lot about typical
> use cases and common pitfalls, and where the original design had to be
> adjusted.

I fail to see how anything of the above would change should
--recreate-merges be still called --preserve-merges, but I do see why
you don't want that to happen now, so please only consider fixing of the
manual page.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-12 20:21         ` Johannes Schindelin
@ 2018-02-13  6:44           ` Sergey Organov
  2018-02-15  1:08             ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-02-13  6:44 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Jacob Keller

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi Sergey,
>
> On Mon, 12 Feb 2018, Sergey Organov wrote:
>
>> Thanks for explanations, and could you please answer this one:
>> 
>> [...]
>> 
>> >> I also have trouble making sense of "Recreate merge commits instead of
>> >> flattening the history by replaying merges." Is it "<Recreate merge
>> >> commits by replaying merges> instead of <flattening the history>" or is it
>> >> rather "<Recreate merge commits> instead of <flattening the history by
>> >> replaying merges>?
>
> I thought I had answered that one.

No, not really, but now you did, please see below.

>
> Flattening the history is what happens in regular rebase (i.e. without
> --recreate-merges and without --preserve-merges).
>
> The idea to recreate merges is of course to *not* flatten the history.

Sure. Never supposed it is.

> Maybe there should have been a comma after "history" to clarify what the
> sentence means.

That's the actual answer to my question, but it in turn raises another
one: why did you change wording of --preserve-merges description for
this new option?

> The wording is poor either way, but you are also not a native speaker so
> we have to rely on, say, Eric to help us out here.

Likely, but why didn't you keep original wording from --preserve-merges?
Do you feel it's somehow poor either?

Anyway, please also refer to wording suggestion in the another (lengthy)
answer in this thread.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-13  4:39           ` Jacob Keller
@ 2018-02-13  7:15             ` Sergey Organov
  2018-02-14  1:35               ` Jacob Keller
  0 siblings, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-02-13  7:15 UTC (permalink / raw)
  To: Jacob Keller; +Cc: Johannes Schindelin, Git mailing list, Junio C Hamano

Hi Jake,

Jacob Keller <jacob.keller@gmail.com> writes:

> On Mon, Feb 12, 2018 at 12:39 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
>> Hi Sergey,
>>
>> On Mon, 12 Feb 2018, Sergey Organov wrote:
>>> > Have a look at https://github.com/git/git/pull/447, especially the
>>> > latest commit in there which is an early version of the deprecation I
>>> > intend to bring about.
>>>
>>> You shouldn't want a deprecation at all should you have re-used
>>> --preserve-merges in the first place, and I still don't see why you
>>> haven't.
>>
>> Keep repeating it, and it won't become truer.
>>
>> If you break formats, you break scripts. Git has *so* many users, there
>> are very likely some who script *every* part of it.
>>
>> We simply cannot do that.
>>
>> What we can is deprecate designs which we learned on the way were not only
>> incomplete from the get-go, but bad overall and hard (or impossible) to
>> fix. Like --preserve-merges.
>>
>> Or for that matter like the design you proposed, to use --first-parent for
>> --recreate-merges. Or to use --first-parent for some --recreate-merges,
>> surprising users in very bad ways when it is not used (or when it is
>> used). I get the impression that you still think it would be a good idea,
>> even if it should be obvious that it is not.
>
> If we consider the addition of new todo list elements as "user
> breaking", then yes this change would be user-script breaking.

It _is_ user script breaking, provided such script exists. Has anybody
actually seen one? Not that it's wrong to be extra-cautious about it,
just curios. Note that to be actually affected, such a script must
invoke "git rebase -p" _command_ and then tweak its todo output to
produce outcome.

> Since we did not originally spell out that todo-list items are subject
> to enhancement by addition of operations in the future, scripts are
> likely not designed to allow addition of new elements.

Out of curiosity, are you going to spell it now, for the new todo
format?

> Thus, adding recreate-merges, and deprecating preserve-merges, seems
> to me to be the correct action to take here.

Yes, sure, provided there is actual breakage, or at least informed
suspicion there is one.

> One could argue that users should have expected new todo list elements
> to be added in the future and thus design their scripts to cope with
> such a thing. If you can convincingly argue this, then I don't
> necessarily see it as a complete user breaking change to fix
> preserve-merges in order to allow it to handle re-ordering properly..

I'd not argue this way myself. If there are out-of-git-tree non-human
users that accept and tweak todo _generated_ by current "git rebase -p"
_command_, I also vote for a new option.

> I think I lean towards agreeing with Johannes, and that adding
> recreate-merges and removing preserve-merges is the better solution.

On these grounds it is, no objections.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-13  7:15             ` Sergey Organov
@ 2018-02-14  1:35               ` Jacob Keller
  2018-02-15  1:14                 ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Jacob Keller @ 2018-02-14  1:35 UTC (permalink / raw)
  To: Sergey Organov; +Cc: Johannes Schindelin, Git mailing list, Junio C Hamano

On Mon, Feb 12, 2018 at 11:15 PM, Sergey Organov <sorganov@gmail.com> wrote:
> Hi Jake,
>
> Jacob Keller <jacob.keller@gmail.com> writes:
>
>> On Mon, Feb 12, 2018 at 12:39 PM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>>> Hi Sergey,
>>>
>>> On Mon, 12 Feb 2018, Sergey Organov wrote:
>>>> > Have a look at https://github.com/git/git/pull/447, especially the
>>>> > latest commit in there which is an early version of the deprecation I
>>>> > intend to bring about.
>>>>
>>>> You shouldn't want a deprecation at all should you have re-used
>>>> --preserve-merges in the first place, and I still don't see why you
>>>> haven't.
>>>
>>> Keep repeating it, and it won't become truer.
>>>
>>> If you break formats, you break scripts. Git has *so* many users, there
>>> are very likely some who script *every* part of it.
>>>
>>> We simply cannot do that.
>>>
>>> What we can is deprecate designs which we learned on the way were not only
>>> incomplete from the get-go, but bad overall and hard (or impossible) to
>>> fix. Like --preserve-merges.
>>>
>>> Or for that matter like the design you proposed, to use --first-parent for
>>> --recreate-merges. Or to use --first-parent for some --recreate-merges,
>>> surprising users in very bad ways when it is not used (or when it is
>>> used). I get the impression that you still think it would be a good idea,
>>> even if it should be obvious that it is not.
>>
>> If we consider the addition of new todo list elements as "user
>> breaking", then yes this change would be user-script breaking.
>
> It _is_ user script breaking, provided such script exists. Has anybody
> actually seen one? Not that it's wrong to be extra-cautious about it,
> just curios. Note that to be actually affected, such a script must
> invoke "git rebase -p" _command_ and then tweak its todo output to
> produce outcome.
>
>> Since we did not originally spell out that todo-list items are subject
>> to enhancement by addition of operations in the future, scripts are
>> likely not designed to allow addition of new elements.
>
> Out of curiosity, are you going to spell it now, for the new todo
> format?
>
>> Thus, adding recreate-merges, and deprecating preserve-merges, seems
>> to me to be the correct action to take here.
>
> Yes, sure, provided there is actual breakage, or at least informed
> suspicion there is one.
>
>> One could argue that users should have expected new todo list elements
>> to be added in the future and thus design their scripts to cope with
>> such a thing. If you can convincingly argue this, then I don't
>> necessarily see it as a complete user breaking change to fix
>> preserve-merges in order to allow it to handle re-ordering properly..
>
> I'd not argue this way myself. If there are out-of-git-tree non-human
> users that accept and tweak todo _generated_ by current "git rebase -p"
> _command_, I also vote for a new option.
>

To be fair, I have not seen anything that actually reads the todo list
and tweaks it in such a manner. The closest example is the git garden
shears script, which simply replaces the todo list.

It's certainly *possible* that such a script would exist though,

Thanks,
Jake

>> I think I lean towards agreeing with Johannes, and that adding
>> recreate-merges and removing preserve-merges is the better solution.
>
> On these grounds it is, no objections.
>
> -- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-08 12:34           ` Johannes Schindelin
@ 2018-02-14  5:41             ` Sergey Organov
  0 siblings, 0 replies; 412+ messages in thread
From: Sergey Organov @ 2018-02-14  5:41 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Junio C Hamano, Øyvind Rønningstad, git, Jacob Keller,
	Johannes Sixt

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
[...]
> Just to give you one concrete example: when I recently rebased some
> patches (no reording or dropping involved here!) and one of the picks
> failed with merge conflicts, I realized that that particular commit
> introduced incorrect formatting and fixed that right away (verifying that
> no other commits introduced incorrect formatting, of course).
>
> With your new cute idea to magically cherry-pick -m1, this change would
> have been magically dropped from the subsequent merge commits!

You put it as if the problem you describe is unsolvable short of getting
back to your favorite blind re-merge. Do you really believe it?

I thought it's obvious that I originally meant "cherry-pick -m1" to be
an explanation facility, a proof of concept, not the final answer to all
the problems of history editing. It's a nice base for actually
approaching these problems though, unlike blind re-merge currently being
used, the latter having no potential.

The fact that bare naked "cherry-pick -m1" doesn't do what is often[1]
required in such cases neither voids the general idea of reproducing
merge-the-result, nor does it make current re-merge approach less
broken.

[1] Please take into consideration that it's _not always_ the case that
one needs a change made to a side-branch to actually propagate to the
main-line over the merge (think "merge -x ours", or something similar
but not that simple), and then it's rather the cute idea to blindly
re-merge that will wreak havoc, as in a lot of other cases.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-13  6:44           ` Sergey Organov
@ 2018-02-15  1:08             ` Johannes Schindelin
  2018-02-15  4:28               ` Sergey Organov
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-15  1:08 UTC (permalink / raw)
  To: Sergey Organov; +Cc: git, Junio C Hamano, Jacob Keller

Hi,

On Tue, 13 Feb 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > The wording is poor either way, but you are also not a native speaker so
> > we have to rely on, say, Eric to help us out here.
> 
> Likely, but why didn't you keep original wording from --preserve-merges?
> Do you feel it's somehow poor either?

Yes, I felt it is poor, especially when --recreate-merges is present, that
is indeed why I changed it.

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-14  1:35               ` Jacob Keller
@ 2018-02-15  1:14                 ` Johannes Schindelin
  2018-02-15  4:35                   ` Sergey Organov
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-15  1:14 UTC (permalink / raw)
  To: Jacob Keller; +Cc: Sergey Organov, Git mailing list, Junio C Hamano

Hi Jake,

On Tue, 13 Feb 2018, Jacob Keller wrote:

> On Mon, Feb 12, 2018 at 11:15 PM, Sergey Organov <sorganov@gmail.com> wrote:
> >
> > Jacob Keller <jacob.keller@gmail.com> writes:
> >
> >> On Mon, Feb 12, 2018 at 12:39 PM, Johannes Schindelin
> >> <Johannes.Schindelin@gmx.de> wrote:
> >>>
> >>> On Mon, 12 Feb 2018, Sergey Organov wrote:
> >>>> > Have a look at https://github.com/git/git/pull/447, especially the
> >>>> > latest commit in there which is an early version of the deprecation I
> >>>> > intend to bring about.
> >>>>
> >>>> You shouldn't want a deprecation at all should you have re-used
> >>>> --preserve-merges in the first place, and I still don't see why you
> >>>> haven't.
> >>>
> >>> Keep repeating it, and it won't become truer.
> >>>
> >>> If you break formats, you break scripts. Git has *so* many users, there
> >>> are very likely some who script *every* part of it.
> >>>
> >>> We simply cannot do that.
> >>>
> >>> What we can is deprecate designs which we learned on the way were not only
> >>> incomplete from the get-go, but bad overall and hard (or impossible) to
> >>> fix. Like --preserve-merges.
> >>>
> >>> Or for that matter like the design you proposed, to use --first-parent for
> >>> --recreate-merges. Or to use --first-parent for some --recreate-merges,
> >>> surprising users in very bad ways when it is not used (or when it is
> >>> used). I get the impression that you still think it would be a good idea,
> >>> even if it should be obvious that it is not.
> >>
> >> If we consider the addition of new todo list elements as "user
> >> breaking", then yes this change would be user-script breaking.
> >
> > It _is_ user script breaking, provided such script exists. Has anybody
> > actually seen one? Not that it's wrong to be extra-cautious about it,
> > just curios. Note that to be actually affected, such a script must
> > invoke "git rebase -p" _command_ and then tweak its todo output to
> > produce outcome.
> >
> >> Since we did not originally spell out that todo-list items are subject
> >> to enhancement by addition of operations in the future, scripts are
> >> likely not designed to allow addition of new elements.
> >
> > Out of curiosity, are you going to spell it now, for the new todo
> > format?
> >
> >> Thus, adding recreate-merges, and deprecating preserve-merges, seems
> >> to me to be the correct action to take here.
> >
> > Yes, sure, provided there is actual breakage, or at least informed
> > suspicion there is one.
> >
> >> One could argue that users should have expected new todo list elements
> >> to be added in the future and thus design their scripts to cope with
> >> such a thing. If you can convincingly argue this, then I don't
> >> necessarily see it as a complete user breaking change to fix
> >> preserve-merges in order to allow it to handle re-ordering properly..
> >
> > I'd not argue this way myself. If there are out-of-git-tree non-human
> > users that accept and tweak todo _generated_ by current "git rebase -p"
> > _command_, I also vote for a new option.
> >
> 
> To be fair, I have not seen anything that actually reads the todo list
> and tweaks it in such a manner. The closest example is the git garden
> shears script, which simply replaces the todo list.
> 
> It's certainly *possible* that such a script would exist though,

We actually know of such scripts.

Remember how rewriting parts of rebase -i in C broke somebody's script
because the todo list was not re-read after a successful `exec`?

Guess three times why that script was broken? Precisely: it modified the
todo list!

To see the fix (and the explanation) in all its glory, just have a look at
54fd3243dae (rebase -i: reread the todo list if `exec` touched it,
2017-04-26).

And even if we did not know about any user. What does that mean? Does it
mean that there is no such user? Or does it not rather mean that our
imagination is rather limited, but we *still* should practice safe
software development and use the totally appropriate vehicle of
deprecating, rather than replacing, functionality?

Obviously, the latter option is what I favor, that's why I suggested it in
the first place.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-13  6:43           ` Sergey Organov
@ 2018-02-15  1:40             ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-15  1:40 UTC (permalink / raw)
  To: Sergey Organov; +Cc: git, Junio C Hamano, Jacob Keller

Hi Sergey,

On Tue, 13 Feb 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >
> > On Mon, 12 Feb 2018, Sergey Organov wrote:
> >
> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >> >
> >> > On Fri, 9 Feb 2018, Sergey Organov wrote:
> >> >
> >> >> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> >> >> 
> >> >> [...]
> >> >> 
> >> >> > With this patch, the goodness of the Git garden shears comes to `git
> >> >> > rebase -i` itself. Passing the `--recreate-merges` option will generate
> >> >> > a todo list that can be understood readily, and where it is obvious
> >> >> > how to reorder commits. New branches can be introduced by inserting
> >> >> > `label` commands and calling `merge - <label> <oneline>`. And once this
> >> >> > mode has become stable and universally accepted, we can deprecate the
> >> >> > design mistake that was `--preserve-merges`.
> >> >> 
> >> >> This doesn't explain why you introduced this new --recreate-merges. Why
> >> >> didn't you rather fix --preserve-merges to generate and use new todo
> >> >> list format?
> >> >
> >> > Because that would of course break existing users of
> >> > --preserve-merges.
> >> 
> >> How exactly?
> >
> > Power users of interactive rebase use scripting to augment Git's
> > functionality. One particularly powerful trick is to override
> > GIT_SEQUENCER_EDITOR with an invocation of such a script, to perform
> > automated edits. Such a script breaks when we change the format of the
> > content to edit. If we change the format of the todo list generated in
> > --preserve-merges mode, that is exactly what happens. We break existing
> > users.
> 
> I didn't say a word against "--preserve-merges mode", whatever it is,
> only about re-using "--preserve-merges" command-line option to "git
> rebase", the git user interface.

*I* said something against --preserve-merges. You did not even need to. I
know fully well its limitations.

I also said something agains the suggestion to replace the functionality
of a previously well-defined (although misdesigned) feature.

I do not know how often I have to repeat that your suggestion would break
backwards-compatibility?

> I'm sure you see the difference?

Yes, of course I do, and you do not even have to suggest otherwise by
asking such a question.

I already demonstrated plenty of times that I do understand what you wish
for, and that I see serious problems with it.

> Unless there are out-of-git scripts that do use "git rebase
> --preserve-merges" and simultaneously do rely on the todo list format
> this exact command generates, there should be no breakage of existing
> users caused by changing todo list format generated by  "git rebase
> --preserve-merges".

So. Just because you cannot imagine that anybody uses rebase in such a
powerful way means you are willing to break their setups?

Git is used by millions of users. Many of them are power users. It would
be quite naive to assume that nobody uses rebase -p in a scripted manner
that modifies the todo list.

Changing the behavior of --preserve-merges would be simply irresponsible,
and that's why we won't do it.

Even if that was not so, there is yet another really good reason not to
reuse the name --preserve-merges: The name itself suggests that this mode
is about preserving all merges in the specified commit range. That was its
original intention, too, as I never designed it to be user with rebase -i.
If the todo list of rebase -p is not modified (preserving the entire
commit topology as well as possible), it works quite well.

The new mode is not so much about preserving, though. It is about
interactively modifying the todo list, to change the order of the commits,
even to change the branch topology. That means that we do not necessarily
preserve the merges. We recreate them. So you see, I did try to be careful
about the naming, too. I thought about this.

> Old broken "--preserve-merges mode" could be then kept in the
> implementation for ages, unused by the new fixed "git rebase
> --preserve-merge", for the sake of compatibility.

This sentence contradicts itself. Either you keep the code unused, or you
keep it used for backwards-compatibility.

> > BTW it seems that you did not really read my previous reply carefully
> > because I referenced such a use case: the Git garden shears.
> 
> I thought I did. You confirm below that this script doesn't use "git
> rebase --preserve-merges" in the first place, nor will it break if "git
> rebase --preserve-merges" starts to generate new todo format, yet you
> expected I'd readily see how it's relevant? No, I'm not that clever, nor
> am I a mind-reader.

You caught me. I am not a user of --preserve-merges. Not anymore.

Does that mean that by extension nobody is a user of that feature?

Certainly not.

And does my example of (ab-)using interactive rebase by scripting on top
of it maybe suggest that others do the same? Maybe even with
--preserve-merges? Most likely. Git is used by many, many users. It would
be foolish to make any assumption about how Git is used by others.

> > They do override the sequencer editor, and while they do not exactly
> > edit the todo list (they simply through the generated one away), they
> > generate a new todo list and would break if that format changes. Of
> > course, the shears do not use the --preserve-merges mode, but from
> > just reading about the way how the Git garden shears work, it is quite
> > obvious how similar users of --preserve-merges are likely to exist?
> 
> Maybe, I dunno. If even "garden shears" won't break, then what will? Do
> you know an example?

You are not seriously suggesting that we should assume that there is no
such Git user, just because neither you nor I personally know such a user?
Seriously?

> Anyway, as it seems it's too late already for such a change, let me stop
> this and assume there are indeed such scripts that will break and that
> it's indeed a good idea to introduce new option. Case closed. The manual
> should still be fixed though, I think.

Finally I got through. Yes, we cannot break backwards-compatibility.

> >> Doesn't "--recreate-merges" produce the same result as
> >> "--preserve-merges" if run non-interactively?
> >
> > The final result of a rebase where you do not edit the todo list? Should
> > be identical, indeed.
> 
> That's good to hear.
> 
> > But that is the most boring, most uninteresting, and least important use
> > case.
> 
> For you. Do you suddenly stop caring about compatibility?

What does "fun" and "interesting" have to do with compatibility?

Yes, to me, this case is boring. And yes, I took pains to make it work
(for compatibility).

And yes, I did not stop after that. After the boring case, I still wanted
to think things through, to come up with a design that would not be too
limited to be useful. With a design that is consistent.

If you find holes in the consistency or usability, please do call them
out.

But please stop suggesting to break backwards-compatibility, or to
introduce features that are inconsistent and/or can produce "surprising"
results (as --first-parent would, and multiple contributors had to argue
in concert, pointing out how it is not extensible to the general case, and
is hence consistent).

> > It is *all* scripts that use interactive rebase!
> 
> I'm really interested, and here I *do* ask for education. What are
> those? As I now only ask this out of curiosity, and don't argue
> --recreate-merges anymore, are you finally willing to reveal the
> information?

If you are interested, why don't you go about asking people for their
power scripts.

In the context of this patch series, I am not interested in such a
collection. What I had to do was to convince myself that they could not
exist, in which case I could just do away with backwards-compatibility. In
the alternative, the safe play is to go the deprecation route.

A mere "highly unlikely" made up from thin air does not convince me,
though. So deprecation route it is.

> > Don't you also care about not breaking existing users?
> 
> I do care. I just suspected they are very unlikely to exist, and I do
> want to be educated in this matter indeed, as they could be rather
> interesting.

Okay, "very unlikely". Not "highly unlikely". Still, it is an unconvincing
argument that suffers very seriously from lack of any robust evidence.

> [Please notice violation of your own standard of not using "Don't
> you...", not that I care myself.]

True. My apologies.

And as the rest of the mail seems to reiterate the idea that the
--recreate-merges code should override --preserve-merges (breaking
backwards-compatibility), despite my repeated efforts to educate you why
this would be a bad idea, I guess the best course of action to avoid
telling you "Don't you ..." is to just stop here.

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-15  1:08             ` Johannes Schindelin
@ 2018-02-15  4:28               ` Sergey Organov
  2018-02-15 16:51                 ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-02-15  4:28 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Jacob Keller

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi,
>
> On Tue, 13 Feb 2018, Sergey Organov wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> 
>> > The wording is poor either way, but you are also not a native speaker so
>> > we have to rely on, say, Eric to help us out here.
>> 
>> Likely, but why didn't you keep original wording from --preserve-merges?
>> Do you feel it's somehow poor either?
>
> Yes, I felt it is poor, especially when --recreate-merges is present, that
> is indeed why I changed it.

So, how about this (yeah, I noticed the option now got arguments, but
please, tweak this to the new implementation yourself):

--recreate-merges::
	Recreate merge commits instead of flattening the history. Merge
	conflict resolutions or manual amendments to merge commits are
	not preserved. 

-p::
--preserve-merges::
	(deprecated) This option is similar to --recreate-merges. It has
        no proper support for interactive mode and thus is deprecated.
        Use '--recreate-merges' instead.


-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-15  1:14                 ` Johannes Schindelin
@ 2018-02-15  4:35                   ` Sergey Organov
  2018-02-15 16:50                     ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-02-15  4:35 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Jacob Keller, Git mailing list, Junio C Hamano

Hi Johannes,
Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

[...]

>> > I'd not argue this way myself. If there are out-of-git-tree non-human
>> > users that accept and tweak todo _generated_ by current "git rebase -p"
>> > _command_, I also vote for a new option.
>> >
>> 
>> To be fair, I have not seen anything that actually reads the todo list
>> and tweaks it in such a manner. The closest example is the git garden
>> shears script, which simply replaces the todo list.
>> 
>> It's certainly *possible* that such a script would exist though,
>
> We actually know of such scripts.

Please consider to explain this in the description of the change. I
believe readers deserve an explanation of why you decided to invent new
option instead of fixing the old one, even if it were only a suspicion,
more so if it is confidence.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-15  4:35                   ` Sergey Organov
@ 2018-02-15 16:50                     ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-15 16:50 UTC (permalink / raw)
  To: Sergey Organov; +Cc: Jacob Keller, Git mailing list, Junio C Hamano

Hi,

On Thu, 15 Feb 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> [...]
> 
> >> > I'd not argue this way myself. If there are out-of-git-tree non-human
> >> > users that accept and tweak todo _generated_ by current "git rebase -p"
> >> > _command_, I also vote for a new option.
> >> >
> >> 
> >> To be fair, I have not seen anything that actually reads the todo list
> >> and tweaks it in such a manner. The closest example is the git garden
> >> shears script, which simply replaces the todo list.
> >> 
> >> It's certainly *possible* that such a script would exist though,
> >
> > We actually know of such scripts.
> 
> Please consider to explain this in the description of the change. I
> believe readers deserve an explanation of why you decided to invent new
> option instead of fixing the old one, even if it were only a suspicion,
> more so if it is confidence.

I considered.

And since even the absence of this use case would *still* not be a
convincing case against keeping --preserve-merges backwards-compatible, I
will not mention it.

Just saying that --preserve-merges is not changed, in order to keep
backwards-compatibility, is plenty enough.

It probably already convinced the Git maintainer, who is very careful
about backwards-compatibility, and rightfully so.

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 5/8] rebase: introduce the --recreate-merges option
  2018-02-15  4:28               ` Sergey Organov
@ 2018-02-15 16:51                 ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-15 16:51 UTC (permalink / raw)
  To: Sergey Organov; +Cc: git, Junio C Hamano, Jacob Keller

Hi,

On Thu, 15 Feb 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Tue, 13 Feb 2018, Sergey Organov wrote:
> >
> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >> 
> >> > The wording is poor either way, but you are also not a native speaker so
> >> > we have to rely on, say, Eric to help us out here.
> >> 
> >> Likely, but why didn't you keep original wording from --preserve-merges?
> >> Do you feel it's somehow poor either?
> >
> > Yes, I felt it is poor, especially when --recreate-merges is present, that
> > is indeed why I changed it.
> 
> So, how about this (yeah, I noticed the option now got arguments, but
> please, tweak this to the new implementation yourself):
> 
> --recreate-merges::
> 	Recreate merge commits instead of flattening the history. Merge
> 	conflict resolutions or manual amendments to merge commits are
> 	not preserved. 
> 
> -p::
> --preserve-merges::
> 	(deprecated) This option is similar to --recreate-merges. It has
>         no proper support for interactive mode and thus is deprecated.
>         Use '--recreate-merges' instead.

I still don't like either.

I want something different there: descriptions that are a bit more
self-contained, and only describe the differences to -i or
--preserve-merges in a second paragraph.

Don't worry about it, though, I don't think you or me are capable of a
good explanation. I will ask some native speakers I trust.

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v4 00/12] rebase -i: offer to recreate merge commits
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
                       ` (11 preceding siblings ...)
  2018-02-11  0:10     ` [PATCH v3 12/12] rebase -i: introduce --recreate-merges=[no-]rebase-cousins Johannes Schindelin
@ 2018-02-23 12:35     ` Johannes Schindelin
  2018-02-23 12:35       ` [PATCH v4 01/12] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
                         ` (12 more replies)
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
  13 siblings, 13 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-23 12:35 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Once upon a time, I dreamt of an interactive rebase that would not
flatten branch structure, but instead recreate the commit topology
faithfully.

My original attempt was --preserve-merges, but that design was so
limited that I did not even enable it in interactive mode.

Subsequently, it *was* enabled in interactive mode, with the predictable
consequences: as the --preserve-merges design does not allow for
specifying the parents of merge commits explicitly, all the new commits'
parents are defined *implicitly* by the previous commit history, and
hence it is *not possible to even reorder commits*.

This design flaw cannot be fixed. Not without a complete re-design, at
least. This patch series offers such a re-design.

Think of --recreate-merges as "--preserve-merges done right". It
introduces new verbs for the todo list, `label`, `reset` and `merge`.
For a commit topology like this:

            A - B - C
              \   /
                D

the generated todo list would look like this:

            # branch D
            pick 0123 A
            label branch-point
            pick 1234 D
            label D

            reset branch-point
            pick 2345 B
            merge -C 3456 D # C

There are more patches in the pipeline, based on this patch series, but
left for later in the interest of reviewable patch series: one mini
series to use the sequencer even for `git rebase -i --root`, and another
one to add support for octopus merges to --recreate-merges.

Changes since v3:

- fixed a grammar error in "introduce the `merge` command"'s commit message.

- fixed a couple of resource leaks in safe_append() and do_reset(), pointed
  out by Eric Sunshine.


Johannes Schindelin (11):
  sequencer: avoid using errno clobbered by rollback_lock_file()
  sequencer: make rearrange_squash() a bit more obvious
  sequencer: introduce new commands to reset the revision
  sequencer: introduce the `merge` command
  sequencer: fast-forward merge commits, if possible
  rebase-helper --make-script: introduce a flag to recreate merges
  rebase: introduce the --recreate-merges option
  sequencer: make refs generated by the `label` command worktree-local
  sequencer: handle post-rewrite for merge commands
  pull: accept --rebase=recreate to recreate the branch topology
  rebase -i: introduce --recreate-merges=[no-]rebase-cousins

Stefan Beller (1):
  git-rebase--interactive: clarify arguments

 Documentation/config.txt               |   8 +
 Documentation/git-pull.txt             |   5 +-
 Documentation/git-rebase.txt           |  14 +-
 builtin/pull.c                         |  14 +-
 builtin/rebase--helper.c               |  13 +-
 builtin/remote.c                       |   2 +
 contrib/completion/git-completion.bash |   4 +-
 git-rebase--interactive.sh             |  22 +-
 git-rebase.sh                          |  16 +
 refs.c                                 |   3 +-
 sequencer.c                            | 742 ++++++++++++++++++++++++++++++++-
 sequencer.h                            |   7 +
 t/t3430-rebase-recreate-merges.sh      | 208 +++++++++
 13 files changed, 1027 insertions(+), 31 deletions(-)
 create mode 100755 t/t3430-rebase-recreate-merges.sh


base-commit: e3a80781f5932f5fea12a49eb06f3ade4ed8945c
Published-As: https://github.com/dscho/git/releases/tag/recreate-merges-v4
Fetch-It-Via: git fetch https://github.com/dscho/git recreate-merges-v4

Interdiff vs v3:
 diff --git a/Documentation/config.txt b/Documentation/config.txt
 index f57e9cf10ca..8c9adea0d0c 100644
 --- a/Documentation/config.txt
 +++ b/Documentation/config.txt
 @@ -1058,6 +1058,10 @@ branch.<name>.rebase::
  	"git pull" is run. See "pull.rebase" for doing this in a non
  	branch-specific manner.
  +
 +When recreate, also pass `--recreate-merges` along to 'git rebase'
 +so that locally committed merge commits will not be flattened
 +by running 'git pull'.
 ++
  When preserve, also pass `--preserve-merges` along to 'git rebase'
  so that locally committed merge commits will not be flattened
  by running 'git pull'.
 @@ -2607,6 +2611,10 @@ pull.rebase::
  	pull" is run. See "branch.<name>.rebase" for setting this on a
  	per-branch basis.
  +
 +When recreate, also pass `--recreate-merges` along to 'git rebase'
 +so that locally committed merge commits will not be flattened
 +by running 'git pull'.
 ++
  When preserve, also pass `--preserve-merges` along to 'git rebase'
  so that locally committed merge commits will not be flattened
  by running 'git pull'.
 diff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt
 index ce05b7a5b13..b4f9f057ea9 100644
 --- a/Documentation/git-pull.txt
 +++ b/Documentation/git-pull.txt
 @@ -101,13 +101,16 @@ Options related to merging
  include::merge-options.txt[]
  
  -r::
 ---rebase[=false|true|preserve|interactive]::
 +--rebase[=false|true|recreate|preserve|interactive]::
  	When true, rebase the current branch on top of the upstream
  	branch after fetching. If there is a remote-tracking branch
  	corresponding to the upstream branch and the upstream branch
  	was rebased since last fetched, the rebase uses that information
  	to avoid rebasing non-local changes.
  +
 +When set to recreate, rebase with the `--recreate-merges` option passed
 +to `git rebase` so that locally created merge commits will not be flattened.
 ++
  When set to preserve, rebase with the `--preserve-merges` option passed
  to `git rebase` so that locally created merge commits will not be flattened.
  +
 diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
 index d713951b86a..c5a77599c47 100644
 --- a/Documentation/git-rebase.txt
 +++ b/Documentation/git-rebase.txt
 @@ -373,6 +373,17 @@ The commit list format can be changed by setting the configuration option
  rebase.instructionFormat.  A customized instruction format will automatically
  have the long commit hash prepended to the format.
  
 +--recreate-merges[=(rebase-cousins|no-rebase-cousins)]::
 +	Recreate merge commits instead of flattening the history by replaying
 +	merges. Merge conflict resolutions or manual amendments to merge
 +	commits are not recreated automatically, but have to be recreated
 +	manually.
 ++
 +By default, or when `no-rebase-cousins` was specified, commits which do not
 +have `<upstream>` as direct ancestor keep their original branch point.
 +If the `rebase-cousins` mode is turned on, such commits are rebased onto
 +`<upstream>` (or `<onto>`, if specified).
 +
  -p::
  --preserve-merges::
  	Recreate merge commits instead of flattening the history by replaying
 @@ -775,7 +786,8 @@ BUGS
  The todo list presented by `--preserve-merges --interactive` does not
  represent the topology of the revision graph.  Editing commits and
  rewording their commit messages should work fine, but attempts to
 -reorder commits tend to produce counterintuitive results.
 +reorder commits tend to produce counterintuitive results. Use
 +--recreate-merges for a more faithful representation.
  
  For example, an attempt to rearrange
  ------------
 diff --git a/builtin/pull.c b/builtin/pull.c
 index 1876271af94..9da2cfa0bd3 100644
 --- a/builtin/pull.c
 +++ b/builtin/pull.c
 @@ -27,14 +27,16 @@ enum rebase_type {
  	REBASE_FALSE = 0,
  	REBASE_TRUE,
  	REBASE_PRESERVE,
 +	REBASE_RECREATE,
  	REBASE_INTERACTIVE
  };
  
  /**
   * Parses the value of --rebase. If value is a false value, returns
   * REBASE_FALSE. If value is a true value, returns REBASE_TRUE. If value is
 - * "preserve", returns REBASE_PRESERVE. If value is a invalid value, dies with
 - * a fatal error if fatal is true, otherwise returns REBASE_INVALID.
 + * "recreate", returns REBASE_RECREATE. If value is "preserve", returns
 + * REBASE_PRESERVE. If value is a invalid value, dies with a fatal error if
 + * fatal is true, otherwise returns REBASE_INVALID.
   */
  static enum rebase_type parse_config_rebase(const char *key, const char *value,
  		int fatal)
 @@ -47,6 +49,8 @@ static enum rebase_type parse_config_rebase(const char *key, const char *value,
  		return REBASE_TRUE;
  	else if (!strcmp(value, "preserve"))
  		return REBASE_PRESERVE;
 +	else if (!strcmp(value, "recreate"))
 +		return REBASE_RECREATE;
  	else if (!strcmp(value, "interactive"))
  		return REBASE_INTERACTIVE;
  
 @@ -130,7 +134,7 @@ static struct option pull_options[] = {
  	/* Options passed to git-merge or git-rebase */
  	OPT_GROUP(N_("Options related to merging")),
  	{ OPTION_CALLBACK, 'r', "rebase", &opt_rebase,
 -	  "false|true|preserve|interactive",
 +	  "false|true|recreate|preserve|interactive",
  	  N_("incorporate changes by rebasing rather than merging"),
  	  PARSE_OPT_OPTARG, parse_opt_rebase },
  	OPT_PASSTHRU('n', NULL, &opt_diffstat, NULL,
 @@ -800,7 +804,9 @@ static int run_rebase(const struct object_id *curr_head,
  	argv_push_verbosity(&args);
  
  	/* Options passed to git-rebase */
 -	if (opt_rebase == REBASE_PRESERVE)
 +	if (opt_rebase == REBASE_RECREATE)
 +		argv_array_push(&args, "--recreate-merges");
 +	else if (opt_rebase == REBASE_PRESERVE)
  		argv_array_push(&args, "--preserve-merges");
  	else if (opt_rebase == REBASE_INTERACTIVE)
  		argv_array_push(&args, "--interactive");
 diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
 index ad074705bb5..5d1f12de57b 100644
 --- a/builtin/rebase--helper.c
 +++ b/builtin/rebase--helper.c
 @@ -12,8 +12,8 @@ static const char * const builtin_rebase_helper_usage[] = {
  int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
  {
  	struct replay_opts opts = REPLAY_OPTS_INIT;
 -	unsigned flags = 0, keep_empty = 0;
 -	int abbreviate_commands = 0;
 +	unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
 +	int abbreviate_commands = 0, rebase_cousins = -1;
  	enum {
  		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
  		CHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH,
 @@ -24,6 +24,9 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
  		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
  		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
  			N_("allow commits with empty messages")),
 +		OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge commits")),
 +		OPT_BOOL(0, "rebase-cousins", &rebase_cousins,
 +			 N_("keep original branch points of cousins")),
  		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
  				CONTINUE),
  		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
 @@ -57,8 +60,14 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
  
  	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
  	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
 +	flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
 +	flags |= rebase_cousins > 0 ? TODO_LIST_REBASE_COUSINS : 0;
  	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
  
 +	if (rebase_cousins >= 0 && !recreate_merges)
 +		warning(_("--[no-]rebase-cousins has no effect without "
 +			  "--recreate-merges"));
 +
  	if (command == CONTINUE && argc == 1)
  		return !!sequencer_continue(&opts);
  	if (command == ABORT && argc == 1)
 diff --git a/builtin/remote.c b/builtin/remote.c
 index d95bf904c3b..b7d0f7ce596 100644
 --- a/builtin/remote.c
 +++ b/builtin/remote.c
 @@ -306,6 +306,8 @@ static int config_read_branches(const char *key, const char *value, void *cb)
  				info->rebase = v;
  			else if (!strcmp(value, "preserve"))
  				info->rebase = NORMAL_REBASE;
 +			else if (!strcmp(value, "recreate"))
 +				info->rebase = NORMAL_REBASE;
  			else if (!strcmp(value, "interactive"))
  				info->rebase = INTERACTIVE_REBASE;
  		}
 diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
 index 88813e91244..3d44cb6890c 100644
 --- a/contrib/completion/git-completion.bash
 +++ b/contrib/completion/git-completion.bash
 @@ -2008,7 +2008,7 @@ _git_rebase ()
  	--*)
  		__gitcomp "
  			--onto --merge --strategy --interactive
 -			--preserve-merges --stat --no-stat
 +			--recreate-merges --preserve-merges --stat --no-stat
  			--committer-date-is-author-date --ignore-date
  			--ignore-whitespace --whitespace=
  			--autosquash --no-autosquash
 @@ -2182,7 +2182,7 @@ _git_config ()
  		return
  		;;
  	branch.*.rebase)
 -		__gitcomp "false true preserve interactive"
 +		__gitcomp "false true recreate preserve interactive"
  		return
  		;;
  	remote.pushdefault)
 diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
 index a2659fea982..679d79e0d17 100644
 --- a/git-rebase--interactive.sh
 +++ b/git-rebase--interactive.sh
 @@ -162,6 +162,12 @@ s, squash <commit> = use commit, but meld into previous commit
  f, fixup <commit> = like \"squash\", but discard this commit's log message
  x, exec <commit> = run command (the rest of the line) using shell
  d, drop <commit> = remove commit
 +l, label <label> = label current HEAD with a name
 +t, reset <label> = reset HEAD to a label
 +m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
 +.       create a merge commit using the original merge commit's
 +.       message (or the oneline, if no original merge commit was
 +.       specified). Use -c <commit> to reword the commit message.
  
  These lines can be re-ordered; they are executed from top to bottom.
  " | git stripspace --comment-lines >>"$todo"
 @@ -900,6 +906,8 @@ fi
  if test t != "$preserve_merges"
  then
  	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
 +		${recreate_merges:+--recreate-merges} \
 +		${rebase_cousins:+--rebase-cousins} \
  		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
  	die "$(gettext "Could not generate todo list")"
  else
 diff --git a/git-rebase.sh b/git-rebase.sh
 index b353c33d417..9487e543bec 100755
 --- a/git-rebase.sh
 +++ b/git-rebase.sh
 @@ -17,6 +17,7 @@ q,quiet!           be quiet. implies --no-stat
  autostash          automatically stash/stash pop before and after
  fork-point         use 'merge-base --fork-point' to refine upstream
  onto=!             rebase onto given branch instead of upstream
 +recreate-merges?   try to recreate merges instead of skipping them
  p,preserve-merges! try to recreate merges instead of ignoring them
  s,strategy=!       use the given merge strategy
  no-ff!             cherry-pick all commits, even if unchanged
 @@ -87,6 +88,8 @@ type=
  state_dir=
  # One of {'', continue, skip, abort}, as parsed from command line
  action=
 +recreate_merges=
 +rebase_cousins=
  preserve_merges=
  autosquash=
  keep_empty=
 @@ -267,6 +270,19 @@ do
  	--allow-empty-message)
  		allow_empty_message=--allow-empty-message
  		;;
 +	--recreate-merges)
 +		recreate_merges=t
 +		test -z "$interactive_rebase" && interactive_rebase=implied
 +		;;
 +	--recreate-merges=*)
 +		recreate_merges=t
 +		case "${1#*=}" in
 +		rebase-cousins) rebase_cousins=t;;
 +		no-rebase-cousins) rebase_cousins=;;
 +		*) die "Unknown mode: $1";;
 +		esac
 +		test -z "$interactive_rebase" && interactive_rebase=implied
 +		;;
  	--preserve-merges)
  		preserve_merges=t
  		test -z "$interactive_rebase" && interactive_rebase=implied
 diff --git a/refs.c b/refs.c
 index 20ba82b4343..e8b84c189ff 100644
 --- a/refs.c
 +++ b/refs.c
 @@ -600,7 +600,8 @@ int dwim_log(const char *str, int len, struct object_id *oid, char **log)
  static int is_per_worktree_ref(const char *refname)
  {
  	return !strcmp(refname, "HEAD") ||
 -		starts_with(refname, "refs/bisect/");
 +		starts_with(refname, "refs/bisect/") ||
 +		starts_with(refname, "refs/rewritten/");
  }
  
  static int is_pseudoref_syntax(const char *refname)
 diff --git a/sequencer.c b/sequencer.c
 index cfa01d3bdd2..b2bf63029d4 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -23,6 +23,10 @@
  #include "hashmap.h"
  #include "notes-utils.h"
  #include "sigchain.h"
 +#include "unpack-trees.h"
 +#include "worktree.h"
 +#include "oidmap.h"
 +#include "oidset.h"
  
  #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
  
 @@ -120,6 +124,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
  static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
  static GIT_PATH_FUNC(rebase_path_rewritten_pending,
  	"rebase-merge/rewritten-pending")
 +
 +/*
 + * The path of the file listing refs that need to be deleted after the rebase
 + * finishes. This is used by the `label` command to record the need for cleanup.
 + */
 +static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
 +
  /*
   * The following files are written by git-rebase just after parsing the
   * command-line (and are only consumed, not modified, by the sequencer).
 @@ -244,18 +255,33 @@ static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
  
  int sequencer_remove_state(struct replay_opts *opts)
  {
 -	struct strbuf dir = STRBUF_INIT;
 +	struct strbuf buf = STRBUF_INIT;
  	int i;
  
 +	if (strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
 +		char *p = buf.buf;
 +		while (*p) {
 +			char *eol = strchr(p, '\n');
 +			if (eol)
 +				*eol = '\0';
 +			if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
 +				warning(_("could not delete '%s'"), p);
 +			if (!eol)
 +				break;
 +			p = eol + 1;
 +		}
 +	}
 +
  	free(opts->gpg_sign);
  	free(opts->strategy);
  	for (i = 0; i < opts->xopts_nr; i++)
  		free(opts->xopts[i]);
  	free(opts->xopts);
  
 -	strbuf_addstr(&dir, get_dir(opts));
 -	remove_dir_recursively(&dir, 0);
 -	strbuf_release(&dir);
 +	strbuf_reset(&buf);
 +	strbuf_addstr(&buf, get_dir(opts));
 +	remove_dir_recursively(&buf, 0);
 +	strbuf_release(&buf);
  
  	return 0;
  }
 @@ -1280,6 +1306,10 @@ enum todo_command {
  	TODO_SQUASH,
  	/* commands that do something else than handling a single commit */
  	TODO_EXEC,
 +	TODO_LABEL,
 +	TODO_RESET,
 +	TODO_MERGE,
 +	TODO_MERGE_AND_EDIT,
  	/* commands that do nothing but are counted for reporting progress */
  	TODO_NOOP,
  	TODO_DROP,
 @@ -1298,6 +1328,10 @@ static struct {
  	{ 'f', "fixup" },
  	{ 's', "squash" },
  	{ 'x', "exec" },
 +	{ 'l', "label" },
 +	{ 't', "reset" },
 +	{ 'm', "merge" },
 +	{ 0, "merge" }, /* MERGE_AND_EDIT */
  	{ 0,   "noop" },
  	{ 'd', "drop" },
  	{ 0,   NULL }
 @@ -1803,13 +1837,29 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
  		return error(_("missing arguments for %s"),
  			     command_to_string(item->command));
  
 -	if (item->command == TODO_EXEC) {
 +	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
 +	    item->command == TODO_RESET) {
  		item->commit = NULL;
  		item->arg = bol;
  		item->arg_len = (int)(eol - bol);
  		return 0;
  	}
  
 +	if (item->command == TODO_MERGE) {
 +		if (skip_prefix(bol, "-C", &bol))
 +			bol += strspn(bol, " \t");
 +		else if (skip_prefix(bol, "-c", &bol)) {
 +			bol += strspn(bol, " \t");
 +			item->command = TODO_MERGE_AND_EDIT;
 +		} else {
 +			item->command = TODO_MERGE_AND_EDIT;
 +			item->commit = NULL;
 +			item->arg = bol;
 +			item->arg_len = (int)(eol - bol);
 +			return 0;
 +		}
 +	}
 +
  	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
  	saved = *end_of_object_name;
  	*end_of_object_name = '\0';
 @@ -2444,6 +2494,304 @@ static int do_exec(const char *command_line)
  	return status;
  }
  
 +static int safe_append(const char *filename, const char *fmt, ...)
 +{
 +	va_list ap;
 +	struct lock_file lock = LOCK_INIT;
 +	int fd = hold_lock_file_for_update(&lock, filename,
 +					   LOCK_REPORT_ON_ERROR);
 +	struct strbuf buf = STRBUF_INIT;
 +
 +	if (fd < 0)
 +		return -1;
 +
 +	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT)
 +		return error_errno(_("could not read '%s'"), filename);
 +	strbuf_complete(&buf, '\n');
 +	va_start(ap, fmt);
 +	strbuf_vaddf(&buf, fmt, ap);
 +	va_end(ap);
 +
 +	if (write_in_full(fd, buf.buf, buf.len) < 0) {
 +		error_errno(_("could not write to '%s'"), filename);
 +		strbuf_release(&buf);
 +		rollback_lock_file(&lock);
 +		return -1;
 +	}
 +	if (commit_lock_file(&lock) < 0) {
 +		strbuf_release(&buf);
 +		rollback_lock_file(&lock);
 +		return error(_("failed to finalize '%s'"), filename);
 +	}
 +
 +	strbuf_release(&buf);
 +	return 0;
 +}
 +
 +static int do_label(const char *name, int len)
 +{
 +	struct ref_store *refs = get_main_ref_store();
 +	struct ref_transaction *transaction;
 +	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
 +	struct strbuf msg = STRBUF_INIT;
 +	int ret = 0;
 +	struct object_id head_oid;
 +
 +	if (len == 1 && *name == '#')
 +		return error("Illegal label name: '%.*s'", len, name);
 +
 +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
 +	strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
 +
 +	transaction = ref_store_transaction_begin(refs, &err);
 +	if (!transaction) {
 +		error("%s", err.buf);
 +		ret = -1;
 +	} else if (get_oid("HEAD", &head_oid)) {
 +		error(_("could not read HEAD"));
 +		ret = -1;
 +	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
 +					  NULL, 0, msg.buf, &err) < 0 ||
 +		   ref_transaction_commit(transaction, &err)) {
 +		error("%s", err.buf);
 +		ret = -1;
 +	}
 +	ref_transaction_free(transaction);
 +	strbuf_release(&err);
 +	strbuf_release(&msg);
 +
 +	if (!ret)
 +		ret = safe_append(rebase_path_refs_to_delete(),
 +				  "%s\n", ref_name.buf);
 +	strbuf_release(&ref_name);
 +
 +	return ret;
 +}
 +
 +static int do_reset(const char *name, int len, struct replay_opts *opts)
 +{
 +	struct strbuf ref_name = STRBUF_INIT;
 +	struct object_id oid;
 +	struct lock_file lock = LOCK_INIT;
 +	struct tree_desc desc;
 +	struct tree *tree;
 +	struct unpack_trees_options unpack_tree_opts;
 +	int ret = 0, i;
 +
 +	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
 +		return -1;
 +
 +	/* Determine the length of the label */
 +	for (i = 0; i < len; i++)
 +		if (isspace(name[i]))
 +			len = i;
 +
 +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
 +	if (get_oid(ref_name.buf, &oid) &&
 +	    get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
 +		error(_("could not read '%s'"), ref_name.buf);
 +		rollback_lock_file(&lock);
 +		strbuf_release(&ref_name);
 +		return -1;
 +	}
 +
 +	memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
 +	unpack_tree_opts.head_idx = 1;
 +	unpack_tree_opts.src_index = &the_index;
 +	unpack_tree_opts.dst_index = &the_index;
 +	unpack_tree_opts.fn = oneway_merge;
 +	unpack_tree_opts.merge = 1;
 +	unpack_tree_opts.update = 1;
 +	unpack_tree_opts.reset = 1;
 +
 +	if (read_cache_unmerged()) {
 +		rollback_lock_file(&lock);
 +		strbuf_release(&ref_name);
 +		return error_resolve_conflict(_(action_name(opts)));
 +	}
 +
 +	if (!fill_tree_descriptor(&desc, &oid)) {
 +		error(_("failed to find tree of %s"), oid_to_hex(&oid));
 +		rollback_lock_file(&lock);
 +		free((void *)desc.buffer);
 +		strbuf_release(&ref_name);
 +		return -1;
 +	}
 +
 +	if (unpack_trees(1, &desc, &unpack_tree_opts)) {
 +		rollback_lock_file(&lock);
 +		free((void *)desc.buffer);
 +		strbuf_release(&ref_name);
 +		return -1;
 +	}
 +
 +	tree = parse_tree_indirect(&oid);
 +	prime_cache_tree(&the_index, tree);
 +
 +	if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
 +		ret = error(_("could not write index"));
 +	free((void *)desc.buffer);
 +
 +	if (!ret) {
 +		struct strbuf msg = STRBUF_INIT;
 +
 +		strbuf_addf(&msg, "(rebase -i) reset '%.*s'", len, name);
 +		ret = update_ref(msg.buf, "HEAD", &oid, NULL, 0,
 +				 UPDATE_REFS_MSG_ON_ERR);
 +		strbuf_release(&msg);
 +	}
 +
 +	strbuf_release(&ref_name);
 +	return ret;
 +}
 +
 +static int do_merge(struct commit *commit, const char *arg, int arg_len,
 +		    int run_commit_flags, struct replay_opts *opts)
 +{
 +	int merge_arg_len;
 +	struct strbuf ref_name = STRBUF_INIT;
 +	struct commit *head_commit, *merge_commit, *i;
 +	struct commit_list *common, *j, *reversed = NULL;
 +	struct merge_options o;
 +	int can_fast_forward, ret;
 +	static struct lock_file lock;
 +
 +	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
 +		if (isspace(arg[merge_arg_len]))
 +			break;
 +
 +	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
 +		return -1;
 +
 +	head_commit = lookup_commit_reference_by_name("HEAD");
 +	if (!head_commit) {
 +		rollback_lock_file(&lock);
 +		return error(_("cannot merge without a current revision"));
 +	}
 +
 +	if (commit) {
 +		const char *message = get_commit_buffer(commit, NULL);
 +		const char *body;
 +		int len;
 +
 +		if (!message) {
 +			rollback_lock_file(&lock);
 +			return error(_("could not get commit message of '%s'"),
 +				     oid_to_hex(&commit->object.oid));
 +		}
 +		write_author_script(message);
 +		find_commit_subject(message, &body);
 +		len = strlen(body);
 +		if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
 +			error_errno(_("could not write '%s'"),
 +				    git_path_merge_msg());
 +			unuse_commit_buffer(commit, message);
 +			rollback_lock_file(&lock);
 +			return -1;
 +		}
 +		unuse_commit_buffer(commit, message);
 +	} else {
 +		const char *p = arg + merge_arg_len;
 +		struct strbuf buf = STRBUF_INIT;
 +		int len;
 +
 +		strbuf_addf(&buf, "author %s", git_author_info(0));
 +		write_author_script(buf.buf);
 +		strbuf_reset(&buf);
 +
 +		p += strspn(p, " \t");
 +		if (*p == '#' && isspace(p[1]))
 +			p += 1 + strspn(p + 1, " \t");
 +		if (*p)
 +			len = strlen(p);
 +		else {
 +			strbuf_addf(&buf, "Merge branch '%.*s'",
 +				    merge_arg_len, arg);
 +			p = buf.buf;
 +			len = buf.len;
 +		}
 +
 +		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
 +			error_errno(_("could not write '%s'"),
 +				    git_path_merge_msg());
 +			strbuf_release(&buf);
 +			rollback_lock_file(&lock);
 +			return -1;
 +		}
 +		strbuf_release(&buf);
 +	}
 +
 +	/*
 +	 * If HEAD is not identical to the parent of the original merge commit,
 +	 * we cannot fast-forward.
 +	 */
 +	can_fast_forward = opts->allow_ff && commit && commit->parents &&
 +		!oidcmp(&commit->parents->item->object.oid,
 +			&head_commit->object.oid);
 +
 +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
 +	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
 +	if (!merge_commit) {
 +		/* fall back to non-rewritten ref or commit */
 +		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
 +		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
 +	}
 +	if (!merge_commit) {
 +		error(_("could not resolve '%s'"), ref_name.buf);
 +		strbuf_release(&ref_name);
 +		rollback_lock_file(&lock);
 +		return -1;
 +	}
 +
 +	if (can_fast_forward && commit->parents->next &&
 +	    !commit->parents->next->next &&
 +	    !oidcmp(&commit->parents->next->item->object.oid,
 +		    &merge_commit->object.oid)) {
 +		strbuf_release(&ref_name);
 +		rollback_lock_file(&lock);
 +		return fast_forward_to(&commit->object.oid,
 +				       &head_commit->object.oid, 0, opts);
 +	}
 +
 +	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
 +		      git_path_merge_head(), 0);
 +	write_message("no-ff", 5, git_path_merge_mode(), 0);
 +
 +	common = get_merge_bases(head_commit, merge_commit);
 +	for (j = common; j; j = j->next)
 +		commit_list_insert(j->item, &reversed);
 +	free_commit_list(common);
 +
 +	read_cache();
 +	init_merge_options(&o);
 +	o.branch1 = "HEAD";
 +	o.branch2 = ref_name.buf;
 +	o.buffer_output = 2;
 +
 +	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
 +	if (ret <= 0)
 +		fputs(o.obuf.buf, stdout);
 +	strbuf_release(&o.obuf);
 +	if (ret < 0) {
 +		strbuf_release(&ref_name);
 +		rollback_lock_file(&lock);
 +		return error(_("conflicts while merging '%.*s'"),
 +			     merge_arg_len, arg);
 +	}
 +
 +	if (active_cache_changed &&
 +	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
 +		strbuf_release(&ref_name);
 +		return error(_("merge: Unable to write new index file"));
 +	}
 +	rollback_lock_file(&lock);
 +
 +	ret = run_git_commit(git_path_merge_msg(), opts, run_commit_flags);
 +	strbuf_release(&ref_name);
 +
 +	return ret;
 +}
 +
  static int is_final_fixup(struct todo_list *todo_list)
  {
  	int i = todo_list->current;
 @@ -2627,6 +2975,18 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  				/* `current` will be incremented below */
  				todo_list->current = -1;
  			}
 +		} else if (item->command == TODO_LABEL)
 +			res = do_label(item->arg, item->arg_len);
 +		else if (item->command == TODO_RESET)
 +			res = do_reset(item->arg, item->arg_len, opts);
 +		else if (item->command == TODO_MERGE ||
 +			 item->command == TODO_MERGE_AND_EDIT) {
 +			res = do_merge(item->commit, item->arg, item->arg_len,
 +				       item->command == TODO_MERGE_AND_EDIT ?
 +				       EDIT_MSG | VERIFY_MSG : 0, opts);
 +			if (item->commit)
 +				record_in_rewritten(&item->commit->object.oid,
 +						    peek_command(todo_list, 1));
  		} else if (!is_noop(item->command))
  			return error(_("unknown command %d"), item->command);
  
 @@ -2981,6 +3341,345 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
  	strbuf_release(&sob);
  }
  
 +struct labels_entry {
 +	struct hashmap_entry entry;
 +	char label[FLEX_ARRAY];
 +};
 +
 +static int labels_cmp(const void *fndata, const struct labels_entry *a,
 +		      const struct labels_entry *b, const void *key)
 +{
 +	return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
 +}
 +
 +struct string_entry {
 +	struct oidmap_entry entry;
 +	char string[FLEX_ARRAY];
 +};
 +
 +struct label_state {
 +	struct oidmap commit2label;
 +	struct hashmap labels;
 +	struct strbuf buf;
 +};
 +
 +static const char *label_oid(struct object_id *oid, const char *label,
 +			     struct label_state *state)
 +{
 +	struct labels_entry *labels_entry;
 +	struct string_entry *string_entry;
 +	struct object_id dummy;
 +	size_t len;
 +	int i;
 +
 +	string_entry = oidmap_get(&state->commit2label, oid);
 +	if (string_entry)
 +		return string_entry->string;
 +
 +	/*
 +	 * For "uninteresting" commits, i.e. commits that are not to be
 +	 * rebased, and which can therefore not be labeled, we use a unique
 +	 * abbreviation of the commit name. This is slightly more complicated
 +	 * than calling find_unique_abbrev() because we also need to make
 +	 * sure that the abbreviation does not conflict with any other
 +	 * label.
 +	 *
 +	 * We disallow "interesting" commits to be labeled by a string that
 +	 * is a valid full-length hash, to ensure that we always can find an
 +	 * abbreviation for any uninteresting commit's names that does not
 +	 * clash with any other label.
 +	 */
 +	if (!label) {
 +		char *p;
 +
 +		strbuf_reset(&state->buf);
 +		strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
 +		label = p = state->buf.buf;
 +
 +		find_unique_abbrev_r(p, oid->hash, default_abbrev);
 +
 +		/*
 +		 * We may need to extend the abbreviated hash so that there is
 +		 * no conflicting label.
 +		 */
 +		if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
 +			size_t i = strlen(p) + 1;
 +
 +			oid_to_hex_r(p, oid);
 +			for (; i < GIT_SHA1_HEXSZ; i++) {
 +				char save = p[i];
 +				p[i] = '\0';
 +				if (!hashmap_get_from_hash(&state->labels,
 +							   strihash(p), p))
 +					break;
 +				p[i] = save;
 +			}
 +		}
 +	} else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
 +		    !get_oid_hex(label, &dummy)) ||
 +		   (len == 1 && *label == '#') ||
 +		   hashmap_get_from_hash(&state->labels,
 +					 strihash(label), label)) {
 +		/*
 +		 * If the label already exists, or if the label is a valid full
 +		 * OID, or the label is a '#' (which we use as a separator
 +		 * between merge heads and oneline), we append a dash and a
 +		 * number to make it unique.
 +		 */
 +		struct strbuf *buf = &state->buf;
 +
 +		strbuf_reset(buf);
 +		strbuf_add(buf, label, len);
 +
 +		for (i = 2; ; i++) {
 +			strbuf_setlen(buf, len);
 +			strbuf_addf(buf, "-%d", i);
 +			if (!hashmap_get_from_hash(&state->labels,
 +						   strihash(buf->buf),
 +						   buf->buf))
 +				break;
 +		}
 +
 +		label = buf->buf;
 +	}
 +
 +	FLEX_ALLOC_STR(labels_entry, label, label);
 +	hashmap_entry_init(labels_entry, strihash(label));
 +	hashmap_add(&state->labels, labels_entry);
 +
 +	FLEX_ALLOC_STR(string_entry, string, label);
 +	oidcpy(&string_entry->entry.oid, oid);
 +	oidmap_put(&state->commit2label, string_entry);
 +
 +	return string_entry->string;
 +}
 +
 +static int make_script_with_merges(struct pretty_print_context *pp,
 +				   struct rev_info *revs, FILE *out,
 +				   unsigned flags)
 +{
 +	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
 +	int rebase_cousins = flags & TODO_LIST_REBASE_COUSINS;
 +	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
 +	struct strbuf label = STRBUF_INIT;
 +	struct commit_list *commits = NULL, **tail = &commits, *iter;
 +	struct commit_list *tips = NULL, **tips_tail = &tips;
 +	struct commit *commit;
 +	struct oidmap commit2todo = OIDMAP_INIT;
 +	struct string_entry *entry;
 +	struct oidset interesting = OIDSET_INIT, child_seen = OIDSET_INIT,
 +		shown = OIDSET_INIT;
 +	struct label_state state = { OIDMAP_INIT, { NULL }, STRBUF_INIT };
 +
 +	int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
 +	const char *cmd_pick = abbr ? "p" : "pick",
 +		*cmd_label = abbr ? "l" : "label",
 +		*cmd_reset = abbr ? "t" : "reset",
 +		*cmd_merge = abbr ? "m" : "merge";
 +
 +	oidmap_init(&commit2todo, 0);
 +	oidmap_init(&state.commit2label, 0);
 +	hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
 +	strbuf_init(&state.buf, 32);
 +
 +	if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
 +		struct object_id *oid = &revs->cmdline.rev[0].item->oid;
 +		FLEX_ALLOC_STR(entry, string, "onto");
 +		oidcpy(&entry->entry.oid, oid);
 +		oidmap_put(&state.commit2label, entry);
 +	}
 +
 +	/*
 +	 * First phase:
 +	 * - get onelines for all commits
 +	 * - gather all branch tips (i.e. 2nd or later parents of merges)
 +	 * - label all branch tips
 +	 */
 +	while ((commit = get_revision(revs))) {
 +		struct commit_list *to_merge;
 +		int is_octopus;
 +		const char *p1, *p2;
 +		struct object_id *oid;
 +
 +		tail = &commit_list_insert(commit, tail)->next;
 +		oidset_insert(&interesting, &commit->object.oid);
 +
 +		if ((commit->object.flags & PATCHSAME))
 +			continue;
 +
 +		strbuf_reset(&oneline);
 +		pretty_print_commit(pp, commit, &oneline);
 +
 +		to_merge = commit->parents ? commit->parents->next : NULL;
 +		if (!to_merge) {
 +			/* non-merge commit: easy case */
 +			strbuf_reset(&buf);
 +			if (!keep_empty && is_original_commit_empty(commit))
 +				strbuf_addf(&buf, "%c ", comment_line_char);
 +			strbuf_addf(&buf, "%s %s %s", cmd_pick,
 +				    oid_to_hex(&commit->object.oid),
 +				    oneline.buf);
 +
 +			FLEX_ALLOC_STR(entry, string, buf.buf);
 +			oidcpy(&entry->entry.oid, &commit->object.oid);
 +			oidmap_put(&commit2todo, entry);
 +
 +			continue;
 +		}
 +
 +		is_octopus = to_merge && to_merge->next;
 +
 +		if (is_octopus)
 +			BUG("Octopus merges not yet supported");
 +
 +		/* Create a label */
 +		strbuf_reset(&label);
 +		if (skip_prefix(oneline.buf, "Merge ", &p1) &&
 +		    (p1 = strchr(p1, '\'')) &&
 +		    (p2 = strchr(++p1, '\'')))
 +			strbuf_add(&label, p1, p2 - p1);
 +		else if (skip_prefix(oneline.buf, "Merge pull request ",
 +				     &p1) &&
 +			 (p1 = strstr(p1, " from ")))
 +			strbuf_addstr(&label, p1 + strlen(" from "));
 +		else
 +			strbuf_addbuf(&label, &oneline);
 +
 +		for (p1 = label.buf; *p1; p1++)
 +			if (isspace(*p1))
 +				*(char *)p1 = '-';
 +
 +		strbuf_reset(&buf);
 +		strbuf_addf(&buf, "%s -C %s",
 +			    cmd_merge, oid_to_hex(&commit->object.oid));
 +
 +		/* label the tip of merged branch */
 +		oid = &to_merge->item->object.oid;
 +		strbuf_addch(&buf, ' ');
 +
 +		if (!oidset_contains(&interesting, oid))
 +			strbuf_addstr(&buf, label_oid(oid, NULL, &state));
 +		else {
 +			tips_tail = &commit_list_insert(to_merge->item,
 +							tips_tail)->next;
 +
 +			strbuf_addstr(&buf, label_oid(oid, label.buf, &state));
 +		}
 +		strbuf_addf(&buf, " # %s", oneline.buf);
 +
 +		FLEX_ALLOC_STR(entry, string, buf.buf);
 +		oidcpy(&entry->entry.oid, &commit->object.oid);
 +		oidmap_put(&commit2todo, entry);
 +	}
 +
 +	/*
 +	 * Second phase:
 +	 * - label branch points
 +	 * - add HEAD to the branch tips
 +	 */
 +	for (iter = commits; iter; iter = iter->next) {
 +		struct commit_list *parent = iter->item->parents;
 +		for (; parent; parent = parent->next) {
 +			struct object_id *oid = &parent->item->object.oid;
 +			if (!oidset_contains(&interesting, oid))
 +				continue;
 +			if (!oidset_contains(&child_seen, oid))
 +				oidset_insert(&child_seen, oid);
 +			else
 +				label_oid(oid, "branch-point", &state);
 +		}
 +
 +		/* Add HEAD as implict "tip of branch" */
 +		if (!iter->next)
 +			tips_tail = &commit_list_insert(iter->item,
 +							tips_tail)->next;
 +	}
 +
 +	/*
 +	 * Third phase: output the todo list. This is a bit tricky, as we
 +	 * want to avoid jumping back and forth between revisions. To
 +	 * accomplish that goal, we walk backwards from the branch tips,
 +	 * gathering commits not yet shown, reversing the list on the fly,
 +	 * then outputting that list (labeling revisions as needed).
 +	 */
 +	fprintf(out, "%s onto\n", cmd_label);
 +	for (iter = tips; iter; iter = iter->next) {
 +		struct commit_list *list = NULL, *iter2;
 +
 +		commit = iter->item;
 +		if (oidset_contains(&shown, &commit->object.oid))
 +			continue;
 +		entry = oidmap_get(&state.commit2label, &commit->object.oid);
 +
 +		if (entry)
 +			fprintf(out, "\n# Branch %s\n", entry->string);
 +		else
 +			fprintf(out, "\n");
 +
 +		while (oidset_contains(&interesting, &commit->object.oid) &&
 +		       !oidset_contains(&shown, &commit->object.oid)) {
 +			commit_list_insert(commit, &list);
 +			if (!commit->parents) {
 +				commit = NULL;
 +				break;
 +			}
 +			commit = commit->parents->item;
 +		}
 +
 +		if (!commit)
 +			fprintf(out, "%s onto\n", cmd_reset);
 +		else {
 +			const char *to = NULL;
 +
 +			entry = oidmap_get(&state.commit2label,
 +					   &commit->object.oid);
 +			if (entry)
 +				to = entry->string;
 +			else if (!rebase_cousins)
 +				to = label_oid(&commit->object.oid, NULL,
 +					       &state);
 +
 +			if (!to || !strcmp(to, "onto"))
 +				fprintf(out, "%s onto\n", cmd_reset);
 +			else {
 +				strbuf_reset(&oneline);
 +				pretty_print_commit(pp, commit, &oneline);
 +				fprintf(out, "%s %s # %s\n",
 +					cmd_reset, to, oneline.buf);
 +			}
 +		}
 +
 +		for (iter2 = list; iter2; iter2 = iter2->next) {
 +			struct object_id *oid = &iter2->item->object.oid;
 +			entry = oidmap_get(&commit2todo, oid);
 +			/* only show if not already upstream */
 +			if (entry)
 +				fprintf(out, "%s\n", entry->string);
 +			entry = oidmap_get(&state.commit2label, oid);
 +			if (entry)
 +				fprintf(out, "%s %s\n",
 +					cmd_label, entry->string);
 +			oidset_insert(&shown, oid);
 +		}
 +
 +		free_commit_list(list);
 +	}
 +
 +	free_commit_list(commits);
 +	free_commit_list(tips);
 +
 +	strbuf_release(&label);
 +	strbuf_release(&oneline);
 +	strbuf_release(&buf);
 +
 +	oidmap_free(&commit2todo, 1);
 +	oidmap_free(&state.commit2label, 1);
 +	hashmap_free(&state.labels, 1);
 +	strbuf_release(&state.buf);
 +
 +	return 0;
 +}
 +
  int sequencer_make_script(FILE *out, int argc, const char **argv,
  			  unsigned flags)
  {
 @@ -2991,11 +3690,16 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
  	struct commit *commit;
  	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
  	const char *insn = flags & TODO_LIST_ABBREVIATE_CMDS ? "p" : "pick";
 +	int recreate_merges = flags & TODO_LIST_RECREATE_MERGES;
  
  	init_revisions(&revs, NULL);
  	revs.verbose_header = 1;
 -	revs.max_parents = 1;
 -	revs.cherry_pick = 1;
 +	if (recreate_merges)
 +		revs.cherry_mark = 1;
 +	else {
 +		revs.max_parents = 1;
 +		revs.cherry_pick = 1;
 +	}
  	revs.limited = 1;
  	revs.reverse = 1;
  	revs.right_only = 1;
 @@ -3019,6 +3723,9 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
  	if (prepare_revision_walk(&revs) < 0)
  		return error(_("make_script: error preparing revisions"));
  
 +	if (recreate_merges)
 +		return make_script_with_merges(&pp, &revs, out, flags);
 +
  	while ((commit = get_revision(&revs))) {
  		strbuf_reset(&buf);
  		if (!keep_empty && is_original_commit_empty(commit))
 @@ -3108,8 +3815,14 @@ int transform_todos(unsigned flags)
  					  short_commit_name(item->commit) :
  					  oid_to_hex(&item->commit->object.oid);
  
 +			if (item->command == TODO_MERGE)
 +				strbuf_addstr(&buf, " -C");
 +			else if (item->command == TODO_MERGE_AND_EDIT)
 +				strbuf_addstr(&buf, " -c");
 +
  			strbuf_addf(&buf, " %s", oid);
  		}
 +
  		/* add all the rest */
  		if (!item->arg_len)
  			strbuf_addch(&buf, '\n');
 diff --git a/sequencer.h b/sequencer.h
 index e45b178dfc4..739dd0fa92b 100644
 --- a/sequencer.h
 +++ b/sequencer.h
 @@ -59,6 +59,13 @@ int sequencer_remove_state(struct replay_opts *opts);
  #define TODO_LIST_KEEP_EMPTY (1U << 0)
  #define TODO_LIST_SHORTEN_IDS (1U << 1)
  #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
 +#define TODO_LIST_RECREATE_MERGES (1U << 3)
 +/*
 + * When recreating merges, commits that do have the base commit as ancestor
 + * ("cousins") are *not* rebased onto the new base by default. If those
 + * commits should be rebased onto the new base, this flag needs to be passed.
 + */
 +#define TODO_LIST_REBASE_COUSINS (1U << 4)
  int sequencer_make_script(FILE *out, int argc, const char **argv,
  			  unsigned flags);
  
 diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
 new file mode 100755
 index 00000000000..9a59f12b670
 --- /dev/null
 +++ b/t/t3430-rebase-recreate-merges.sh
 @@ -0,0 +1,208 @@
 +#!/bin/sh
 +#
 +# Copyright (c) 2017 Johannes E. Schindelin
 +#
 +
 +test_description='git rebase -i --recreate-merges
 +
 +This test runs git rebase "interactively", retaining the branch structure by
 +recreating merge commits.
 +
 +Initial setup:
 +
 +    -- B --                   (first)
 +   /       \
 + A - C - D - E - H            (master)
 +       \       /
 +         F - G                (second)
 +'
 +. ./test-lib.sh
 +. "$TEST_DIRECTORY"/lib-rebase.sh
 +
 +test_expect_success 'setup' '
 +	write_script replace-editor.sh <<-\EOF &&
 +	mv "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
 +	cp script-from-scratch "$1"
 +	EOF
 +
 +	test_commit A &&
 +	git checkout -b first &&
 +	test_commit B &&
 +	git checkout master &&
 +	test_commit C &&
 +	test_commit D &&
 +	git merge --no-commit B &&
 +	test_tick &&
 +	git commit -m E &&
 +	git tag -m E E &&
 +	git checkout -b second C &&
 +	test_commit F &&
 +	test_commit G &&
 +	git checkout master &&
 +	git merge --no-commit G &&
 +	test_tick &&
 +	git commit -m H &&
 +	git tag -m H H
 +'
 +
 +cat >script-from-scratch <<\EOF
 +label onto
 +
 +# onebranch
 +pick G
 +pick D
 +label onebranch
 +
 +# second
 +reset onto
 +pick B
 +label second
 +
 +reset onto
 +merge -C H second
 +merge onebranch # Merge the topic branch 'onebranch'
 +EOF
 +
 +test_cmp_graph () {
 +	cat >expect &&
 +	git log --graph --boundary --format=%s "$@" >output &&
 +	sed "s/ *$//" <output >output.trimmed &&
 +	test_cmp expect output.trimmed
 +}
 +
 +test_expect_success 'create completely different structure' '
 +	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
 +	test_tick &&
 +	git rebase -i --recreate-merges A &&
 +	test_cmp_graph <<-\EOF
 +	*   Merge the topic branch '\''onebranch'\''
 +	|\
 +	| * D
 +	| * G
 +	* |   H
 +	|\ \
 +	| |/
 +	|/|
 +	| * B
 +	|/
 +	* A
 +	EOF
 +'
 +
 +test_expect_success 'generate correct todo list' '
 +	cat >expect <<-\EOF &&
 +	label onto
 +
 +	reset onto
 +	pick d9df450 B
 +	label E
 +
 +	reset onto
 +	pick 5dee784 C
 +	label branch-point
 +	pick ca2c861 F
 +	pick 088b00a G
 +	label H
 +
 +	reset branch-point # C
 +	pick 12bd07b D
 +	merge -C 2051b56 E # E
 +	merge -C 233d48a H # H
 +
 +	EOF
 +
 +	grep -v "^#" <.git/ORIGINAL-TODO >output &&
 +	test_cmp expect output
 +'
 +
 +test_expect_success 'with a branch tip that was cherry-picked already' '
 +	git checkout -b already-upstream master &&
 +	base="$(git rev-parse --verify HEAD)" &&
 +
 +	test_commit A1 &&
 +	test_commit A2 &&
 +	git reset --hard $base &&
 +	test_commit B1 &&
 +	test_tick &&
 +	git merge -m "Merge branch A" A2 &&
 +
 +	git checkout -b upstream-with-a2 $base &&
 +	test_tick &&
 +	git cherry-pick A2 &&
 +
 +	git checkout already-upstream &&
 +	test_tick &&
 +	git rebase -i --recreate-merges upstream-with-a2 &&
 +	test_cmp_graph upstream-with-a2.. <<-\EOF
 +	*   Merge branch A
 +	|\
 +	| * A1
 +	* | B1
 +	|/
 +	o A2
 +	EOF
 +'
 +
 +test_expect_success 'do not rebase cousins unless asked for' '
 +	write_script copy-editor.sh <<-\EOF &&
 +	cp "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
 +	EOF
 +
 +	test_config sequence.editor \""$PWD"/copy-editor.sh\" &&
 +	git checkout -b cousins master &&
 +	before="$(git rev-parse --verify HEAD)" &&
 +	test_tick &&
 +	git rebase -i --recreate-merges HEAD^ &&
 +	test_cmp_rev HEAD $before &&
 +	test_tick &&
 +	git rebase -i --recreate-merges=rebase-cousins HEAD^ &&
 +	test_cmp_graph HEAD^.. <<-\EOF
 +	*   Merge the topic branch '\''onebranch'\''
 +	|\
 +	| * D
 +	| * G
 +	|/
 +	o H
 +	EOF
 +'
 +
 +test_expect_success 'refs/rewritten/* is worktree-local' '
 +	git worktree add wt &&
 +	cat >wt/script-from-scratch <<-\EOF &&
 +	label xyz
 +	exec GIT_DIR=../.git git rev-parse --verify refs/rewritten/xyz >a || :
 +	exec git rev-parse --verify refs/rewritten/xyz >b
 +	EOF
 +
 +	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
 +	git -C wt rebase -i HEAD &&
 +	test_must_be_empty wt/a &&
 +	test_cmp_rev HEAD "$(cat wt/b)"
 +'
 +
 +test_expect_success 'post-rewrite hook and fixups work for merges' '
 +	git checkout -b post-rewrite &&
 +	test_commit same1 &&
 +	git reset --hard HEAD^ &&
 +	test_commit same2 &&
 +	git merge -m "to fix up" same1 &&
 +	echo same old same old >same2.t &&
 +	test_tick &&
 +	git commit --fixup HEAD same2.t &&
 +	fixup="$(git rev-parse HEAD)" &&
 +
 +	mkdir -p .git/hooks &&
 +	test_when_finished "rm .git/hooks/post-rewrite" &&
 +	echo "cat >actual" | write_script .git/hooks/post-rewrite &&
 +
 +	test_tick &&
 +	git rebase -i --autosquash --recreate-merges HEAD^^^ &&
 +	printf "%s %s\n%s %s\n%s %s\n%s %s\n" >expect $(git rev-parse \
 +		$fixup^^2 HEAD^2 \
 +		$fixup^^ HEAD^ \
 +		$fixup^ HEAD \
 +		$fixup HEAD) &&
 +	test_cmp expect actual
 +'
 +
 +test_done
-- 
2.16.1.windows.4


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v4 01/12] sequencer: avoid using errno clobbered by rollback_lock_file()
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
@ 2018-02-23 12:35       ` Johannes Schindelin
  2018-02-23 12:36       ` [PATCH v4 02/12] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
                         ` (11 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-23 12:35 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

As pointed out in a review of the `--recreate-merges` patch series,
`rollback_lock_file()` clobbers errno. Therefore, we have to report the
error message that uses errno before calling said function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index e9baaf59bd9..5aa3dc3c95c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -345,12 +345,14 @@ static int write_message(const void *buf, size_t len, const char *filename,
 	if (msg_fd < 0)
 		return error_errno(_("could not lock '%s'"), filename);
 	if (write_in_full(msg_fd, buf, len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write to '%s'"), filename);
+		return -1;
 	}
 	if (append_eol && write(msg_fd, "\n", 1) < 0) {
+		error_errno(_("could not write eol to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write eol to '%s'"), filename);
+		return -1;
 	}
 	if (commit_lock_file(&msg_file) < 0) {
 		rollback_lock_file(&msg_file);
@@ -2106,16 +2108,17 @@ static int save_head(const char *head)
 
 	fd = hold_lock_file_for_update(&head_lock, git_path_head_file(), 0);
 	if (fd < 0) {
+		error_errno(_("could not lock HEAD"));
 		rollback_lock_file(&head_lock);
-		return error_errno(_("could not lock HEAD"));
+		return -1;
 	}
 	strbuf_addf(&buf, "%s\n", head);
 	written = write_in_full(fd, buf.buf, buf.len);
 	strbuf_release(&buf);
 	if (written < 0) {
+		error_errno(_("could not write to '%s'"), git_path_head_file());
 		rollback_lock_file(&head_lock);
-		return error_errno(_("could not write to '%s'"),
-				   git_path_head_file());
+		return -1;
 	}
 	if (commit_lock_file(&head_lock) < 0) {
 		rollback_lock_file(&head_lock);
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 02/12] sequencer: make rearrange_squash() a bit more obvious
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
  2018-02-23 12:35       ` [PATCH v4 01/12] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
@ 2018-02-23 12:36       ` Johannes Schindelin
  2018-02-23 12:36       ` [PATCH v4 03/12] git-rebase--interactive: clarify arguments Johannes Schindelin
                         ` (10 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-23 12:36 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

There are some commands that have to be skipped from rearranging by virtue
of not handling any commits.

However, the logic was not quite obvious: it skipped commands based on
their position in the enum todo_command.

Instead, let's make it explicit that we skip all commands that do not
handle any commit. With one exception: the `drop` command, because it,
well, drops the commit and is therefore not eligible to rearranging.

Note: this is a bit academic at the moment because the only time we call
`rearrange_squash()` is directly after generating the todo list, when we
have nothing but `pick` commands anyway.

However, the upcoming `merge` command *will* want to be handled by that
function, and it *can* handle commits.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 5aa3dc3c95c..cfa01d3bdd2 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3412,7 +3412,7 @@ int rearrange_squash(void)
 		struct subject2item_entry *entry;
 
 		next[i] = tail[i] = -1;
-		if (item->command >= TODO_EXEC) {
+		if (!item->commit || item->command == TODO_DROP) {
 			subjects[i] = NULL;
 			continue;
 		}
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 03/12] git-rebase--interactive: clarify arguments
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
  2018-02-23 12:35       ` [PATCH v4 01/12] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
  2018-02-23 12:36       ` [PATCH v4 02/12] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
@ 2018-02-23 12:36       ` Johannes Schindelin
  2018-02-23 12:37       ` [PATCH v4 04/12] sequencer: introduce new commands to reset the revision Johannes Schindelin
                         ` (9 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-23 12:36 UTC (permalink / raw)
  To: git
  Cc: Stefan Beller, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood

From: Stefan Beller <stefanbeller@gmail.com>

Up to now each command took a commit as its first argument and ignored
the rest of the line (usually the subject of the commit)

Now that we are about to introduce commands that take different
arguments, clarify each command by giving the argument list.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 81c5b428757..a2659fea982 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -155,13 +155,13 @@ reschedule_last_action () {
 append_todo_help () {
 	gettext "
 Commands:
-p, pick = use commit
-r, reword = use commit, but edit the commit message
-e, edit = use commit, but stop for amending
-s, squash = use commit, but meld into previous commit
-f, fixup = like \"squash\", but discard this commit's log message
-x, exec = run command (the rest of the line) using shell
-d, drop = remove commit
+p, pick <commit> = use commit
+r, reword <commit> = use commit, but edit the commit message
+e, edit <commit> = use commit, but stop for amending
+s, squash <commit> = use commit, but meld into previous commit
+f, fixup <commit> = like \"squash\", but discard this commit's log message
+x, exec <commit> = run command (the rest of the line) using shell
+d, drop <commit> = remove commit
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 04/12] sequencer: introduce new commands to reset the revision
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
                         ` (2 preceding siblings ...)
  2018-02-23 12:36       ` [PATCH v4 03/12] git-rebase--interactive: clarify arguments Johannes Schindelin
@ 2018-02-23 12:37       ` Johannes Schindelin
  2018-02-23 12:37       ` [PATCH v4 05/12] sequencer: introduce the `merge` command Johannes Schindelin
                         ` (8 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-23 12:37 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

In the upcoming commits, we will teach the sequencer to recreate merges.
This will be done in a very different way from the unfortunate design of
`git rebase --preserve-merges` (which does not allow for reordering
commits, or changing the branch topology).

The main idea is to introduce new todo list commands, to support
labeling the current revision with a given name, resetting the current
revision to a previous state, and  merging labeled revisions.

This idea was developed in Git for Windows' Git garden shears (that are
used to maintain the "thicket of branches" on top of upstream Git), and
this patch is part of the effort to make it available to a wider
audience, as well as to make the entire process more robust (by
implementing it in a safe and portable language rather than a Unix shell
script).

This commit implements the commands to label, and to reset to, given
revisions. The syntax is:

	label <name>
	reset <name>

Internally, the `label <name>` command creates the ref
`refs/rewritten/<name>`. This makes it possible to work with the labeled
revisions interactively, or in a scripted fashion (e.g. via the todo
list command `exec`).

These temporary refs are removed upon sequencer_remove_state(), so that
even a `git rebase --abort` cleans them up.

We disallow '#' as label because that character will be used as separator
in the upcoming `merge` command.

Later in this patch series, we will mark the `refs/rewritten/` refs as
worktree-local, to allow for interactive rebases to be run in parallel in
worktrees linked to the same repository.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   2 +
 sequencer.c                | 196 +++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 192 insertions(+), 6 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index a2659fea982..501f09b28c4 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -162,6 +162,8 @@ s, squash <commit> = use commit, but meld into previous commit
 f, fixup <commit> = like \"squash\", but discard this commit's log message
 x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
+l, label <label> = label current HEAD with a name
+t, reset <label> = reset HEAD to a label
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index cfa01d3bdd2..e25522ecdf1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -23,6 +23,8 @@
 #include "hashmap.h"
 #include "notes-utils.h"
 #include "sigchain.h"
+#include "unpack-trees.h"
+#include "worktree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -120,6 +122,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
 static GIT_PATH_FUNC(rebase_path_rewritten_pending,
 	"rebase-merge/rewritten-pending")
+
+/*
+ * The path of the file listing refs that need to be deleted after the rebase
+ * finishes. This is used by the `label` command to record the need for cleanup.
+ */
+static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
+
 /*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
@@ -244,18 +253,33 @@ static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
 
 int sequencer_remove_state(struct replay_opts *opts)
 {
-	struct strbuf dir = STRBUF_INIT;
+	struct strbuf buf = STRBUF_INIT;
 	int i;
 
+	if (strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
+		char *p = buf.buf;
+		while (*p) {
+			char *eol = strchr(p, '\n');
+			if (eol)
+				*eol = '\0';
+			if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
+				warning(_("could not delete '%s'"), p);
+			if (!eol)
+				break;
+			p = eol + 1;
+		}
+	}
+
 	free(opts->gpg_sign);
 	free(opts->strategy);
 	for (i = 0; i < opts->xopts_nr; i++)
 		free(opts->xopts[i]);
 	free(opts->xopts);
 
-	strbuf_addstr(&dir, get_dir(opts));
-	remove_dir_recursively(&dir, 0);
-	strbuf_release(&dir);
+	strbuf_reset(&buf);
+	strbuf_addstr(&buf, get_dir(opts));
+	remove_dir_recursively(&buf, 0);
+	strbuf_release(&buf);
 
 	return 0;
 }
@@ -1280,6 +1304,8 @@ enum todo_command {
 	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
+	TODO_LABEL,
+	TODO_RESET,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -1298,6 +1324,8 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
+	{ 'l', "label" },
+	{ 't', "reset" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1803,7 +1831,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return error(_("missing arguments for %s"),
 			     command_to_string(item->command));
 
-	if (item->command == TODO_EXEC) {
+	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
+	    item->command == TODO_RESET) {
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = (int)(eol - bol);
@@ -2444,6 +2473,157 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int safe_append(const char *filename, const char *fmt, ...)
+{
+	va_list ap;
+	struct lock_file lock = LOCK_INIT;
+	int fd = hold_lock_file_for_update(&lock, filename,
+					   LOCK_REPORT_ON_ERROR);
+	struct strbuf buf = STRBUF_INIT;
+
+	if (fd < 0)
+		return -1;
+
+	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT)
+		return error_errno(_("could not read '%s'"), filename);
+	strbuf_complete(&buf, '\n');
+	va_start(ap, fmt);
+	strbuf_vaddf(&buf, fmt, ap);
+	va_end(ap);
+
+	if (write_in_full(fd, buf.buf, buf.len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
+		strbuf_release(&buf);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	if (commit_lock_file(&lock) < 0) {
+		strbuf_release(&buf);
+		rollback_lock_file(&lock);
+		return error(_("failed to finalize '%s'"), filename);
+	}
+
+	strbuf_release(&buf);
+	return 0;
+}
+
+static int do_label(const char *name, int len)
+{
+	struct ref_store *refs = get_main_ref_store();
+	struct ref_transaction *transaction;
+	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
+	struct strbuf msg = STRBUF_INIT;
+	int ret = 0;
+	struct object_id head_oid;
+
+	if (len == 1 && *name == '#')
+		return error("Illegal label name: '%.*s'", len, name);
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
+
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction) {
+		error("%s", err.buf);
+		ret = -1;
+	} else if (get_oid("HEAD", &head_oid)) {
+		error(_("could not read HEAD"));
+		ret = -1;
+	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
+					  NULL, 0, msg.buf, &err) < 0 ||
+		   ref_transaction_commit(transaction, &err)) {
+		error("%s", err.buf);
+		ret = -1;
+	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
+	strbuf_release(&msg);
+
+	if (!ret)
+		ret = safe_append(rebase_path_refs_to_delete(),
+				  "%s\n", ref_name.buf);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
+static int do_reset(const char *name, int len, struct replay_opts *opts)
+{
+	struct strbuf ref_name = STRBUF_INIT;
+	struct object_id oid;
+	struct lock_file lock = LOCK_INIT;
+	struct tree_desc desc;
+	struct tree *tree;
+	struct unpack_trees_options unpack_tree_opts;
+	int ret = 0, i;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	/* Determine the length of the label */
+	for (i = 0; i < len; i++)
+		if (isspace(name[i]))
+			len = i;
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	if (get_oid(ref_name.buf, &oid) &&
+	    get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
+		error(_("could not read '%s'"), ref_name.buf);
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
+	unpack_tree_opts.head_idx = 1;
+	unpack_tree_opts.src_index = &the_index;
+	unpack_tree_opts.dst_index = &the_index;
+	unpack_tree_opts.fn = oneway_merge;
+	unpack_tree_opts.merge = 1;
+	unpack_tree_opts.update = 1;
+	unpack_tree_opts.reset = 1;
+
+	if (read_cache_unmerged()) {
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return error_resolve_conflict(_(action_name(opts)));
+	}
+
+	if (!fill_tree_descriptor(&desc, &oid)) {
+		error(_("failed to find tree of %s"), oid_to_hex(&oid));
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	if (unpack_trees(1, &desc, &unpack_tree_opts)) {
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	tree = parse_tree_indirect(&oid);
+	prime_cache_tree(&the_index, tree);
+
+	if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
+		ret = error(_("could not write index"));
+	free((void *)desc.buffer);
+
+	if (!ret) {
+		struct strbuf msg = STRBUF_INIT;
+
+		strbuf_addf(&msg, "(rebase -i) reset '%.*s'", len, name);
+		ret = update_ref(msg.buf, "HEAD", &oid, NULL, 0,
+				 UPDATE_REFS_MSG_ON_ERR);
+		strbuf_release(&msg);
+	}
+
+	strbuf_release(&ref_name);
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2627,7 +2807,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				/* `current` will be incremented below */
 				todo_list->current = -1;
 			}
-		} else if (!is_noop(item->command))
+		} else if (item->command == TODO_LABEL)
+			res = do_label(item->arg, item->arg_len);
+		else if (item->command == TODO_RESET)
+			res = do_reset(item->arg, item->arg_len, opts);
+		else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 05/12] sequencer: introduce the `merge` command
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
                         ` (3 preceding siblings ...)
  2018-02-23 12:37       ` [PATCH v4 04/12] sequencer: introduce new commands to reset the revision Johannes Schindelin
@ 2018-02-23 12:37       ` Johannes Schindelin
  2018-02-23 12:37       ` [PATCH v4 06/12] sequencer: fast-forward merge commits, if possible Johannes Schindelin
                         ` (7 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-23 12:37 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

This patch is part of the effort to reimplement `--preserve-merges` with
a substantially improved design, a design that has been developed in the
Git for Windows project to maintain the dozens of Windows-specific patch
series on top of upstream Git.

The previous patch implemented the `label` and `reset` commands to label
commits and to reset to labeled commits. This patch adds the `merge`
command, with the following syntax:

	merge [-C <commit>] <rev> # <oneline>

The <commit> parameter in this instance is the *original* merge commit,
whose author and message will be used for the merge commit that is about
to be created.

The <rev> parameter refers to the (possibly rewritten) revision to
merge. Let's see an example of a todo list:

	label onto

	# Branch abc
	reset onto
	pick deadbeef Hello, world!
	label abc

	reset onto
	pick cafecafe And now for something completely different
	merge -C baaabaaa abc # Merge the branch 'abc' into master

To edit the merge commit's message (a "reword" for merges, if you will),
use `-c` (lower-case) instead of `-C`; this convention was borrowed from
`git commit` that also supports `-c` and `-C` with similar meanings.

To create *new* merges, i.e. without copying the commit message from an
existing commit, simply omit the `-C <commit>` parameter (which will
open an editor for the merge message):

	merge abc

This comes in handy when splitting a branch into two or more branches.

Note: this patch only adds support for recursive merges, to keep things
simple. Support for octopus merges will be added later in a separate
patch series, support for merges using strategies other than the
recursive merge is left for the future.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   4 ++
 sequencer.c                | 158 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 162 insertions(+)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 501f09b28c4..2d8bbe20b74 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -164,6 +164,10 @@ x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
 l, label <label> = label current HEAD with a name
 t, reset <label> = reset HEAD to a label
+m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
+.       create a merge commit using the original merge commit's
+.       message (or the oneline, if no original merge commit was
+.       specified). Use -c <commit> to reword the commit message.
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index e25522ecdf1..64dbd1d3e2e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1306,6 +1306,8 @@ enum todo_command {
 	TODO_EXEC,
 	TODO_LABEL,
 	TODO_RESET,
+	TODO_MERGE,
+	TODO_MERGE_AND_EDIT,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -1326,6 +1328,8 @@ static struct {
 	{ 'x', "exec" },
 	{ 'l', "label" },
 	{ 't', "reset" },
+	{ 'm', "merge" },
+	{ 0, "merge" }, /* MERGE_AND_EDIT */
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1839,6 +1843,21 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return 0;
 	}
 
+	if (item->command == TODO_MERGE) {
+		if (skip_prefix(bol, "-C", &bol))
+			bol += strspn(bol, " \t");
+		else if (skip_prefix(bol, "-c", &bol)) {
+			bol += strspn(bol, " \t");
+			item->command = TODO_MERGE_AND_EDIT;
+		} else {
+			item->command = TODO_MERGE_AND_EDIT;
+			item->commit = NULL;
+			item->arg = bol;
+			item->arg_len = (int)(eol - bol);
+			return 0;
+		}
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
@@ -2624,6 +2643,134 @@ static int do_reset(const char *name, int len, struct replay_opts *opts)
 	return ret;
 }
 
+static int do_merge(struct commit *commit, const char *arg, int arg_len,
+		    int run_commit_flags, struct replay_opts *opts)
+{
+	int merge_arg_len;
+	struct strbuf ref_name = STRBUF_INIT;
+	struct commit *head_commit, *merge_commit, *i;
+	struct commit_list *common, *j, *reversed = NULL;
+	struct merge_options o;
+	int ret;
+	static struct lock_file lock;
+
+	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
+		if (isspace(arg[merge_arg_len]))
+			break;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	head_commit = lookup_commit_reference_by_name("HEAD");
+	if (!head_commit) {
+		rollback_lock_file(&lock);
+		return error(_("cannot merge without a current revision"));
+	}
+
+	if (commit) {
+		const char *message = get_commit_buffer(commit, NULL);
+		const char *body;
+		int len;
+
+		if (!message) {
+			rollback_lock_file(&lock);
+			return error(_("could not get commit message of '%s'"),
+				     oid_to_hex(&commit->object.oid));
+		}
+		write_author_script(message);
+		find_commit_subject(message, &body);
+		len = strlen(body);
+		if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			unuse_commit_buffer(commit, message);
+			rollback_lock_file(&lock);
+			return -1;
+		}
+		unuse_commit_buffer(commit, message);
+	} else {
+		const char *p = arg + merge_arg_len;
+		struct strbuf buf = STRBUF_INIT;
+		int len;
+
+		strbuf_addf(&buf, "author %s", git_author_info(0));
+		write_author_script(buf.buf);
+		strbuf_reset(&buf);
+
+		p += strspn(p, " \t");
+		if (*p == '#' && isspace(p[1]))
+			p += 1 + strspn(p + 1, " \t");
+		if (*p)
+			len = strlen(p);
+		else {
+			strbuf_addf(&buf, "Merge branch '%.*s'",
+				    merge_arg_len, arg);
+			p = buf.buf;
+			len = buf.len;
+		}
+
+		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			strbuf_release(&buf);
+			rollback_lock_file(&lock);
+			return -1;
+		}
+		strbuf_release(&buf);
+	}
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
+	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	if (!merge_commit) {
+		/* fall back to non-rewritten ref or commit */
+		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
+		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	}
+	if (!merge_commit) {
+		error(_("could not resolve '%s'"), ref_name.buf);
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
+		      git_path_merge_head(), 0);
+	write_message("no-ff", 5, git_path_merge_mode(), 0);
+
+	common = get_merge_bases(head_commit, merge_commit);
+	for (j = common; j; j = j->next)
+		commit_list_insert(j->item, &reversed);
+	free_commit_list(common);
+
+	read_cache();
+	init_merge_options(&o);
+	o.branch1 = "HEAD";
+	o.branch2 = ref_name.buf;
+	o.buffer_output = 2;
+
+	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
+	if (ret <= 0)
+		fputs(o.obuf.buf, stdout);
+	strbuf_release(&o.obuf);
+	if (ret < 0) {
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return error(_("conflicts while merging '%.*s'"),
+			     merge_arg_len, arg);
+	}
+
+	if (active_cache_changed &&
+	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
+		strbuf_release(&ref_name);
+		return error(_("merge: Unable to write new index file"));
+	}
+	rollback_lock_file(&lock);
+
+	ret = run_git_commit(git_path_merge_msg(), opts, run_commit_flags);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2811,6 +2958,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_label(item->arg, item->arg_len);
 		else if (item->command == TODO_RESET)
 			res = do_reset(item->arg, item->arg_len, opts);
+		else if (item->command == TODO_MERGE ||
+			 item->command == TODO_MERGE_AND_EDIT)
+			res = do_merge(item->commit, item->arg, item->arg_len,
+				       item->command == TODO_MERGE_AND_EDIT ?
+				       EDIT_MSG | VERIFY_MSG : 0, opts);
 		else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
@@ -3292,8 +3444,14 @@ int transform_todos(unsigned flags)
 					  short_commit_name(item->commit) :
 					  oid_to_hex(&item->commit->object.oid);
 
+			if (item->command == TODO_MERGE)
+				strbuf_addstr(&buf, " -C");
+			else if (item->command == TODO_MERGE_AND_EDIT)
+				strbuf_addstr(&buf, " -c");
+
 			strbuf_addf(&buf, " %s", oid);
 		}
+
 		/* add all the rest */
 		if (!item->arg_len)
 			strbuf_addch(&buf, '\n');
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 06/12] sequencer: fast-forward merge commits, if possible
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
                         ` (4 preceding siblings ...)
  2018-02-23 12:37       ` [PATCH v4 05/12] sequencer: introduce the `merge` command Johannes Schindelin
@ 2018-02-23 12:37       ` Johannes Schindelin
  2018-02-23 12:38       ` [PATCH v4 07/12] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
                         ` (6 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-23 12:37 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Just like with regular `pick` commands, if we are trying to recreate a
merge commit, we now test whether the parents of said commit match HEAD
and the commits to be merged, and fast-forward if possible.

This is not only faster, but also avoids unnecessary proliferation of
new objects.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 64dbd1d3e2e..361ec98f764 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2651,7 +2651,7 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	struct commit *head_commit, *merge_commit, *i;
 	struct commit_list *common, *j, *reversed = NULL;
 	struct merge_options o;
-	int ret;
+	int can_fast_forward, ret;
 	static struct lock_file lock;
 
 	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
@@ -2719,6 +2719,14 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		strbuf_release(&buf);
 	}
 
+	/*
+	 * If HEAD is not identical to the parent of the original merge commit,
+	 * we cannot fast-forward.
+	 */
+	can_fast_forward = opts->allow_ff && commit && commit->parents &&
+		!oidcmp(&commit->parents->item->object.oid,
+			&head_commit->object.oid);
+
 	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
 	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
 	if (!merge_commit) {
@@ -2732,6 +2740,17 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		rollback_lock_file(&lock);
 		return -1;
 	}
+
+	if (can_fast_forward && commit->parents->next &&
+	    !commit->parents->next->next &&
+	    !oidcmp(&commit->parents->next->item->object.oid,
+		    &merge_commit->object.oid)) {
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return fast_forward_to(&commit->object.oid,
+				       &head_commit->object.oid, 0, opts);
+	}
+
 	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
 		      git_path_merge_head(), 0);
 	write_message("no-ff", 5, git_path_merge_mode(), 0);
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 07/12] rebase-helper --make-script: introduce a flag to recreate merges
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
                         ` (5 preceding siblings ...)
  2018-02-23 12:37       ` [PATCH v4 06/12] sequencer: fast-forward merge commits, if possible Johannes Schindelin
@ 2018-02-23 12:38       ` Johannes Schindelin
  2018-02-23 12:38       ` [PATCH v4 08/12] rebase: introduce the --recreate-merges option Johannes Schindelin
                         ` (5 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-23 12:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

The sequencer just learned new commands intended to recreate branch
structure (similar in spirit to --preserve-merges, but with a
substantially less-broken design).

Let's allow the rebase--helper to generate todo lists making use of
these commands, triggered by the new --recreate-merges option. For a
commit topology like this (where the HEAD points to C):

	- A - B - C
	    \   /
	      D

the generated todo list would look like this:

	# branch D
	pick 0123 A
	label branch-point
	pick 1234 D
	label D

	reset branch-point
	pick 2345 B
	merge -C 3456 D # C

To keep things simple, we first only implement support for merge commits
with exactly two parents, leaving support for octopus merges to a later
patch in this patch series.

As a special, hard-coded label, all merge-recreating todo lists start with
the command `label onto` so that we can later always refer to the revision
onto which everything is rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/rebase--helper.c |   4 +-
 sequencer.c              | 349 ++++++++++++++++++++++++++++++++++++++++++++++-
 sequencer.h              |   1 +
 3 files changed, 351 insertions(+), 3 deletions(-)

diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index ad074705bb5..a5b07c43c96 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -12,7 +12,7 @@ static const char * const builtin_rebase_helper_usage[] = {
 int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
-	unsigned flags = 0, keep_empty = 0;
+	unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
 	int abbreviate_commands = 0;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
@@ -24,6 +24,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
 		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
 			N_("allow commits with empty messages")),
+		OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge commits")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -57,6 +58,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
+	flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
 	if (command == CONTINUE && argc == 1)
diff --git a/sequencer.c b/sequencer.c
index 361ec98f764..01bafe2fe47 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -25,6 +25,8 @@
 #include "sigchain.h"
 #include "unpack-trees.h"
 #include "worktree.h"
+#include "oidmap.h"
+#include "oidset.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -3336,6 +3338,341 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
 	strbuf_release(&sob);
 }
 
+struct labels_entry {
+	struct hashmap_entry entry;
+	char label[FLEX_ARRAY];
+};
+
+static int labels_cmp(const void *fndata, const struct labels_entry *a,
+		      const struct labels_entry *b, const void *key)
+{
+	return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
+}
+
+struct string_entry {
+	struct oidmap_entry entry;
+	char string[FLEX_ARRAY];
+};
+
+struct label_state {
+	struct oidmap commit2label;
+	struct hashmap labels;
+	struct strbuf buf;
+};
+
+static const char *label_oid(struct object_id *oid, const char *label,
+			     struct label_state *state)
+{
+	struct labels_entry *labels_entry;
+	struct string_entry *string_entry;
+	struct object_id dummy;
+	size_t len;
+	int i;
+
+	string_entry = oidmap_get(&state->commit2label, oid);
+	if (string_entry)
+		return string_entry->string;
+
+	/*
+	 * For "uninteresting" commits, i.e. commits that are not to be
+	 * rebased, and which can therefore not be labeled, we use a unique
+	 * abbreviation of the commit name. This is slightly more complicated
+	 * than calling find_unique_abbrev() because we also need to make
+	 * sure that the abbreviation does not conflict with any other
+	 * label.
+	 *
+	 * We disallow "interesting" commits to be labeled by a string that
+	 * is a valid full-length hash, to ensure that we always can find an
+	 * abbreviation for any uninteresting commit's names that does not
+	 * clash with any other label.
+	 */
+	if (!label) {
+		char *p;
+
+		strbuf_reset(&state->buf);
+		strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
+		label = p = state->buf.buf;
+
+		find_unique_abbrev_r(p, oid->hash, default_abbrev);
+
+		/*
+		 * We may need to extend the abbreviated hash so that there is
+		 * no conflicting label.
+		 */
+		if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
+			size_t i = strlen(p) + 1;
+
+			oid_to_hex_r(p, oid);
+			for (; i < GIT_SHA1_HEXSZ; i++) {
+				char save = p[i];
+				p[i] = '\0';
+				if (!hashmap_get_from_hash(&state->labels,
+							   strihash(p), p))
+					break;
+				p[i] = save;
+			}
+		}
+	} else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
+		    !get_oid_hex(label, &dummy)) ||
+		   (len == 1 && *label == '#') ||
+		   hashmap_get_from_hash(&state->labels,
+					 strihash(label), label)) {
+		/*
+		 * If the label already exists, or if the label is a valid full
+		 * OID, or the label is a '#' (which we use as a separator
+		 * between merge heads and oneline), we append a dash and a
+		 * number to make it unique.
+		 */
+		struct strbuf *buf = &state->buf;
+
+		strbuf_reset(buf);
+		strbuf_add(buf, label, len);
+
+		for (i = 2; ; i++) {
+			strbuf_setlen(buf, len);
+			strbuf_addf(buf, "-%d", i);
+			if (!hashmap_get_from_hash(&state->labels,
+						   strihash(buf->buf),
+						   buf->buf))
+				break;
+		}
+
+		label = buf->buf;
+	}
+
+	FLEX_ALLOC_STR(labels_entry, label, label);
+	hashmap_entry_init(labels_entry, strihash(label));
+	hashmap_add(&state->labels, labels_entry);
+
+	FLEX_ALLOC_STR(string_entry, string, label);
+	oidcpy(&string_entry->entry.oid, oid);
+	oidmap_put(&state->commit2label, string_entry);
+
+	return string_entry->string;
+}
+
+static int make_script_with_merges(struct pretty_print_context *pp,
+				   struct rev_info *revs, FILE *out,
+				   unsigned flags)
+{
+	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
+	struct strbuf label = STRBUF_INIT;
+	struct commit_list *commits = NULL, **tail = &commits, *iter;
+	struct commit_list *tips = NULL, **tips_tail = &tips;
+	struct commit *commit;
+	struct oidmap commit2todo = OIDMAP_INIT;
+	struct string_entry *entry;
+	struct oidset interesting = OIDSET_INIT, child_seen = OIDSET_INIT,
+		shown = OIDSET_INIT;
+	struct label_state state = { OIDMAP_INIT, { NULL }, STRBUF_INIT };
+
+	int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
+	const char *cmd_pick = abbr ? "p" : "pick",
+		*cmd_label = abbr ? "l" : "label",
+		*cmd_reset = abbr ? "t" : "reset",
+		*cmd_merge = abbr ? "m" : "merge";
+
+	oidmap_init(&commit2todo, 0);
+	oidmap_init(&state.commit2label, 0);
+	hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
+	strbuf_init(&state.buf, 32);
+
+	if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
+		struct object_id *oid = &revs->cmdline.rev[0].item->oid;
+		FLEX_ALLOC_STR(entry, string, "onto");
+		oidcpy(&entry->entry.oid, oid);
+		oidmap_put(&state.commit2label, entry);
+	}
+
+	/*
+	 * First phase:
+	 * - get onelines for all commits
+	 * - gather all branch tips (i.e. 2nd or later parents of merges)
+	 * - label all branch tips
+	 */
+	while ((commit = get_revision(revs))) {
+		struct commit_list *to_merge;
+		int is_octopus;
+		const char *p1, *p2;
+		struct object_id *oid;
+
+		tail = &commit_list_insert(commit, tail)->next;
+		oidset_insert(&interesting, &commit->object.oid);
+
+		if ((commit->object.flags & PATCHSAME))
+			continue;
+
+		strbuf_reset(&oneline);
+		pretty_print_commit(pp, commit, &oneline);
+
+		to_merge = commit->parents ? commit->parents->next : NULL;
+		if (!to_merge) {
+			/* non-merge commit: easy case */
+			strbuf_reset(&buf);
+			if (!keep_empty && is_original_commit_empty(commit))
+				strbuf_addf(&buf, "%c ", comment_line_char);
+			strbuf_addf(&buf, "%s %s %s", cmd_pick,
+				    oid_to_hex(&commit->object.oid),
+				    oneline.buf);
+
+			FLEX_ALLOC_STR(entry, string, buf.buf);
+			oidcpy(&entry->entry.oid, &commit->object.oid);
+			oidmap_put(&commit2todo, entry);
+
+			continue;
+		}
+
+		is_octopus = to_merge && to_merge->next;
+
+		if (is_octopus)
+			BUG("Octopus merges not yet supported");
+
+		/* Create a label */
+		strbuf_reset(&label);
+		if (skip_prefix(oneline.buf, "Merge ", &p1) &&
+		    (p1 = strchr(p1, '\'')) &&
+		    (p2 = strchr(++p1, '\'')))
+			strbuf_add(&label, p1, p2 - p1);
+		else if (skip_prefix(oneline.buf, "Merge pull request ",
+				     &p1) &&
+			 (p1 = strstr(p1, " from ")))
+			strbuf_addstr(&label, p1 + strlen(" from "));
+		else
+			strbuf_addbuf(&label, &oneline);
+
+		for (p1 = label.buf; *p1; p1++)
+			if (isspace(*p1))
+				*(char *)p1 = '-';
+
+		strbuf_reset(&buf);
+		strbuf_addf(&buf, "%s -C %s",
+			    cmd_merge, oid_to_hex(&commit->object.oid));
+
+		/* label the tip of merged branch */
+		oid = &to_merge->item->object.oid;
+		strbuf_addch(&buf, ' ');
+
+		if (!oidset_contains(&interesting, oid))
+			strbuf_addstr(&buf, label_oid(oid, NULL, &state));
+		else {
+			tips_tail = &commit_list_insert(to_merge->item,
+							tips_tail)->next;
+
+			strbuf_addstr(&buf, label_oid(oid, label.buf, &state));
+		}
+		strbuf_addf(&buf, " # %s", oneline.buf);
+
+		FLEX_ALLOC_STR(entry, string, buf.buf);
+		oidcpy(&entry->entry.oid, &commit->object.oid);
+		oidmap_put(&commit2todo, entry);
+	}
+
+	/*
+	 * Second phase:
+	 * - label branch points
+	 * - add HEAD to the branch tips
+	 */
+	for (iter = commits; iter; iter = iter->next) {
+		struct commit_list *parent = iter->item->parents;
+		for (; parent; parent = parent->next) {
+			struct object_id *oid = &parent->item->object.oid;
+			if (!oidset_contains(&interesting, oid))
+				continue;
+			if (!oidset_contains(&child_seen, oid))
+				oidset_insert(&child_seen, oid);
+			else
+				label_oid(oid, "branch-point", &state);
+		}
+
+		/* Add HEAD as implict "tip of branch" */
+		if (!iter->next)
+			tips_tail = &commit_list_insert(iter->item,
+							tips_tail)->next;
+	}
+
+	/*
+	 * Third phase: output the todo list. This is a bit tricky, as we
+	 * want to avoid jumping back and forth between revisions. To
+	 * accomplish that goal, we walk backwards from the branch tips,
+	 * gathering commits not yet shown, reversing the list on the fly,
+	 * then outputting that list (labeling revisions as needed).
+	 */
+	fprintf(out, "%s onto\n", cmd_label);
+	for (iter = tips; iter; iter = iter->next) {
+		struct commit_list *list = NULL, *iter2;
+
+		commit = iter->item;
+		if (oidset_contains(&shown, &commit->object.oid))
+			continue;
+		entry = oidmap_get(&state.commit2label, &commit->object.oid);
+
+		if (entry)
+			fprintf(out, "\n# Branch %s\n", entry->string);
+		else
+			fprintf(out, "\n");
+
+		while (oidset_contains(&interesting, &commit->object.oid) &&
+		       !oidset_contains(&shown, &commit->object.oid)) {
+			commit_list_insert(commit, &list);
+			if (!commit->parents) {
+				commit = NULL;
+				break;
+			}
+			commit = commit->parents->item;
+		}
+
+		if (!commit)
+			fprintf(out, "%s onto\n", cmd_reset);
+		else {
+			const char *to = NULL;
+
+			entry = oidmap_get(&state.commit2label,
+					   &commit->object.oid);
+			if (entry)
+				to = entry->string;
+
+			if (!to || !strcmp(to, "onto"))
+				fprintf(out, "%s onto\n", cmd_reset);
+			else {
+				strbuf_reset(&oneline);
+				pretty_print_commit(pp, commit, &oneline);
+				fprintf(out, "%s %s # %s\n",
+					cmd_reset, to, oneline.buf);
+			}
+		}
+
+		for (iter2 = list; iter2; iter2 = iter2->next) {
+			struct object_id *oid = &iter2->item->object.oid;
+			entry = oidmap_get(&commit2todo, oid);
+			/* only show if not already upstream */
+			if (entry)
+				fprintf(out, "%s\n", entry->string);
+			entry = oidmap_get(&state.commit2label, oid);
+			if (entry)
+				fprintf(out, "%s %s\n",
+					cmd_label, entry->string);
+			oidset_insert(&shown, oid);
+		}
+
+		free_commit_list(list);
+	}
+
+	free_commit_list(commits);
+	free_commit_list(tips);
+
+	strbuf_release(&label);
+	strbuf_release(&oneline);
+	strbuf_release(&buf);
+
+	oidmap_free(&commit2todo, 1);
+	oidmap_free(&state.commit2label, 1);
+	hashmap_free(&state.labels, 1);
+	strbuf_release(&state.buf);
+
+	return 0;
+}
+
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags)
 {
@@ -3346,11 +3683,16 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	struct commit *commit;
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
 	const char *insn = flags & TODO_LIST_ABBREVIATE_CMDS ? "p" : "pick";
+	int recreate_merges = flags & TODO_LIST_RECREATE_MERGES;
 
 	init_revisions(&revs, NULL);
 	revs.verbose_header = 1;
-	revs.max_parents = 1;
-	revs.cherry_pick = 1;
+	if (recreate_merges)
+		revs.cherry_mark = 1;
+	else {
+		revs.max_parents = 1;
+		revs.cherry_pick = 1;
+	}
 	revs.limited = 1;
 	revs.reverse = 1;
 	revs.right_only = 1;
@@ -3374,6 +3716,9 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	if (prepare_revision_walk(&revs) < 0)
 		return error(_("make_script: error preparing revisions"));
 
+	if (recreate_merges)
+		return make_script_with_merges(&pp, &revs, out, flags);
+
 	while ((commit = get_revision(&revs))) {
 		strbuf_reset(&buf);
 		if (!keep_empty && is_original_commit_empty(commit))
diff --git a/sequencer.h b/sequencer.h
index e45b178dfc4..7c7c67d623c 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -59,6 +59,7 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_KEEP_EMPTY (1U << 0)
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
+#define TODO_LIST_RECREATE_MERGES (1U << 3)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 08/12] rebase: introduce the --recreate-merges option
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
                         ` (6 preceding siblings ...)
  2018-02-23 12:38       ` [PATCH v4 07/12] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
@ 2018-02-23 12:38       ` Johannes Schindelin
  2018-02-23 12:38       ` [PATCH v4 09/12] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
                         ` (4 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-23 12:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Once upon a time, this here developer thought: wouldn't it be nice if,
say, Git for Windows' patches on top of core Git could be represented as
a thicket of branches, and be rebased on top of core Git in order to
maintain a cherry-pick'able set of patch series?

The original attempt to answer this was: git rebase --preserve-merges.

However, that experiment was never intended as an interactive option,
and it only piggy-backed on git rebase --interactive because that
command's implementation looked already very, very familiar: it was
designed by the same person who designed --preserve-merges: yours truly.

Some time later, some other developer (I am looking at you, Andreas!
;-)) decided that it would be a good idea to allow --preserve-merges to
be combined with --interactive (with caveats!) and the Git maintainer
(well, the interim Git maintainer during Junio's absence, that is)
agreed, and that is when the glamor of the --preserve-merges design
started to fall apart rather quickly and unglamorously.

The reason? In --preserve-merges mode, the parents of a merge commit (or
for that matter, of *any* commit) were not stated explicitly, but were
*implied* by the commit name passed to the `pick` command.

This made it impossible, for example, to reorder commits. Not to mention
to flatten the branch topology or, deity forbid, to split topic branches
into two.

Alas, these shortcomings also prevented that mode (whose original
purpose was to serve Git for Windows' needs, with the additional hope
that it may be useful to others, too) from serving Git for Windows'
needs.

Five years later, when it became really untenable to have one unwieldy,
big hodge-podge patch series of partly related, partly unrelated patches
in Git for Windows that was rebased onto core Git's tags from time to
time (earning the undeserved wrath of the developer of the ill-fated
git-remote-hg series that first obsoleted Git for Windows' competing
approach, only to be abandoned without maintainer later) was really
untenable, the "Git garden shears" were born [*1*/*2*]: a script,
piggy-backing on top of the interactive rebase, that would first
determine the branch topology of the patches to be rebased, create a
pseudo todo list for further editing, transform the result into a real
todo list (making heavy use of the `exec` command to "implement" the
missing todo list commands) and finally recreate the patch series on
top of the new base commit.

That was in 2013. And it took about three weeks to come up with the
design and implement it as an out-of-tree script. Needless to say, the
implementation needed quite a few years to stabilize, all the while the
design itself proved itself sound.

With this patch, the goodness of the Git garden shears comes to `git
rebase -i` itself. Passing the `--recreate-merges` option will generate
a todo list that can be understood readily, and where it is obvious
how to reorder commits. New branches can be introduced by inserting
`label` commands and calling `merge <label>`. And once this mode will
have become stable and universally accepted, we can deprecate the design
mistake that was `--preserve-merges`.

Link *1*:
https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
Link *2*:
https://github.com/git-for-windows/build-extra/blob/master/shears.sh

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt           |   9 +-
 contrib/completion/git-completion.bash |   2 +-
 git-rebase--interactive.sh             |   1 +
 git-rebase.sh                          |   6 ++
 t/t3430-rebase-recreate-merges.sh      | 146 +++++++++++++++++++++++++++++++++
 5 files changed, 162 insertions(+), 2 deletions(-)
 create mode 100755 t/t3430-rebase-recreate-merges.sh

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index d713951b86a..5e056c8ab6b 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -373,6 +373,12 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
+--recreate-merges::
+	Recreate merge commits instead of flattening the history by replaying
+	merges. Merge conflict resolutions or manual amendments to merge
+	commits are not recreated automatically, but have to be recreated
+	manually.
+
 -p::
 --preserve-merges::
 	Recreate merge commits instead of flattening the history by replaying
@@ -775,7 +781,8 @@ BUGS
 The todo list presented by `--preserve-merges --interactive` does not
 represent the topology of the revision graph.  Editing commits and
 rewording their commit messages should work fine, but attempts to
-reorder commits tend to produce counterintuitive results.
+reorder commits tend to produce counterintuitive results. Use
+--recreate-merges for a more faithful representation.
 
 For example, an attempt to rearrange
 ------------
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 88813e91244..38bba3835c6 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2008,7 +2008,7 @@ _git_rebase ()
 	--*)
 		__gitcomp "
 			--onto --merge --strategy --interactive
-			--preserve-merges --stat --no-stat
+			--recreate-merges --preserve-merges --stat --no-stat
 			--committer-date-is-author-date --ignore-date
 			--ignore-whitespace --whitespace=
 			--autosquash --no-autosquash
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 2d8bbe20b74..f5c8db2fdf8 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -906,6 +906,7 @@ fi
 if test t != "$preserve_merges"
 then
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
+		${recreate_merges:+--recreate-merges} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 else
diff --git a/git-rebase.sh b/git-rebase.sh
index b353c33d417..528fa0073ac 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,6 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
+recreate-merges!   try to recreate merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -87,6 +88,7 @@ type=
 state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
+recreate_merges=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -267,6 +269,10 @@ do
 	--allow-empty-message)
 		allow_empty_message=--allow-empty-message
 		;;
+	--recreate-merges)
+		recreate_merges=t
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
new file mode 100755
index 00000000000..0073601a206
--- /dev/null
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -0,0 +1,146 @@
+#!/bin/sh
+#
+# Copyright (c) 2017 Johannes E. Schindelin
+#
+
+test_description='git rebase -i --recreate-merges
+
+This test runs git rebase "interactively", retaining the branch structure by
+recreating merge commits.
+
+Initial setup:
+
+    -- B --                   (first)
+   /       \
+ A - C - D - E - H            (master)
+       \       /
+         F - G                (second)
+'
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
+
+test_expect_success 'setup' '
+	write_script replace-editor.sh <<-\EOF &&
+	mv "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	cp script-from-scratch "$1"
+	EOF
+
+	test_commit A &&
+	git checkout -b first &&
+	test_commit B &&
+	git checkout master &&
+	test_commit C &&
+	test_commit D &&
+	git merge --no-commit B &&
+	test_tick &&
+	git commit -m E &&
+	git tag -m E E &&
+	git checkout -b second C &&
+	test_commit F &&
+	test_commit G &&
+	git checkout master &&
+	git merge --no-commit G &&
+	test_tick &&
+	git commit -m H &&
+	git tag -m H H
+'
+
+cat >script-from-scratch <<\EOF
+label onto
+
+# onebranch
+pick G
+pick D
+label onebranch
+
+# second
+reset onto
+pick B
+label second
+
+reset onto
+merge -C H second
+merge onebranch # Merge the topic branch 'onebranch'
+EOF
+
+test_cmp_graph () {
+	cat >expect &&
+	git log --graph --boundary --format=%s "$@" >output &&
+	sed "s/ *$//" <output >output.trimmed &&
+	test_cmp expect output.trimmed
+}
+
+test_expect_success 'create completely different structure' '
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_tick &&
+	git rebase -i --recreate-merges A &&
+	test_cmp_graph <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	* |   H
+	|\ \
+	| |/
+	|/|
+	| * B
+	|/
+	* A
+	EOF
+'
+
+test_expect_success 'generate correct todo list' '
+	cat >expect <<-\EOF &&
+	label onto
+
+	reset onto
+	pick d9df450 B
+	label E
+
+	reset onto
+	pick 5dee784 C
+	label branch-point
+	pick ca2c861 F
+	pick 088b00a G
+	label H
+
+	reset branch-point # C
+	pick 12bd07b D
+	merge -C 2051b56 E # E
+	merge -C 233d48a H # H
+
+	EOF
+
+	grep -v "^#" <.git/ORIGINAL-TODO >output &&
+	test_cmp expect output
+'
+
+test_expect_success 'with a branch tip that was cherry-picked already' '
+	git checkout -b already-upstream master &&
+	base="$(git rev-parse --verify HEAD)" &&
+
+	test_commit A1 &&
+	test_commit A2 &&
+	git reset --hard $base &&
+	test_commit B1 &&
+	test_tick &&
+	git merge -m "Merge branch A" A2 &&
+
+	git checkout -b upstream-with-a2 $base &&
+	test_tick &&
+	git cherry-pick A2 &&
+
+	git checkout already-upstream &&
+	test_tick &&
+	git rebase -i --recreate-merges upstream-with-a2 &&
+	test_cmp_graph upstream-with-a2.. <<-\EOF
+	*   Merge branch A
+	|\
+	| * A1
+	* | B1
+	|/
+	o A2
+	EOF
+'
+
+test_done
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 09/12] sequencer: make refs generated by the `label` command worktree-local
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
                         ` (7 preceding siblings ...)
  2018-02-23 12:38       ` [PATCH v4 08/12] rebase: introduce the --recreate-merges option Johannes Schindelin
@ 2018-02-23 12:38       ` Johannes Schindelin
  2018-02-23 12:39       ` [PATCH v4 10/12] sequencer: handle post-rewrite for merge commands Johannes Schindelin
                         ` (3 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-23 12:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

This allows for rebases to be run in parallel in separate worktrees
(think: interrupted in the middle of one rebase, being asked to perform
a different rebase, adding a separate worktree just for that job).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 refs.c                            |  3 ++-
 t/t3430-rebase-recreate-merges.sh | 14 ++++++++++++++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 20ba82b4343..e8b84c189ff 100644
--- a/refs.c
+++ b/refs.c
@@ -600,7 +600,8 @@ int dwim_log(const char *str, int len, struct object_id *oid, char **log)
 static int is_per_worktree_ref(const char *refname)
 {
 	return !strcmp(refname, "HEAD") ||
-		starts_with(refname, "refs/bisect/");
+		starts_with(refname, "refs/bisect/") ||
+		starts_with(refname, "refs/rewritten/");
 }
 
 static int is_pseudoref_syntax(const char *refname)
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index 0073601a206..1a3e43d66ff 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -143,4 +143,18 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'refs/rewritten/* is worktree-local' '
+	git worktree add wt &&
+	cat >wt/script-from-scratch <<-\EOF &&
+	label xyz
+	exec GIT_DIR=../.git git rev-parse --verify refs/rewritten/xyz >a || :
+	exec git rev-parse --verify refs/rewritten/xyz >b
+	EOF
+
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	git -C wt rebase -i HEAD &&
+	test_must_be_empty wt/a &&
+	test_cmp_rev HEAD "$(cat wt/b)"
+'
+
 test_done
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 10/12] sequencer: handle post-rewrite for merge commands
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
                         ` (8 preceding siblings ...)
  2018-02-23 12:38       ` [PATCH v4 09/12] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
@ 2018-02-23 12:39       ` Johannes Schindelin
  2018-02-23 12:39       ` [PATCH v4 11/12] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
                         ` (2 subsequent siblings)
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-23 12:39 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

In the previous patches, we implemented the basic functionality of the
`git rebase -i --recreate-merges` command, in particular the `merge`
command to create merge commits in the sequencer.

The interactive rebase is a lot more these days, though, than a simple
cherry-pick in a loop. For example, it calls the post-rewrite hook (if
any) after rebasing with a mapping of the old->new commits.

This patch implements the post-rewrite handling for the `merge` command
we just introduced. The other commands that were added recently (`label`
and `reset`) do not create new commits, therefore post-rewrite do not
need to handle them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c                       |  7 +++++--
 t/t3430-rebase-recreate-merges.sh | 25 +++++++++++++++++++++++++
 2 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 01bafe2fe47..85ce37cb99f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2980,11 +2980,14 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		else if (item->command == TODO_RESET)
 			res = do_reset(item->arg, item->arg_len, opts);
 		else if (item->command == TODO_MERGE ||
-			 item->command == TODO_MERGE_AND_EDIT)
+			 item->command == TODO_MERGE_AND_EDIT) {
 			res = do_merge(item->commit, item->arg, item->arg_len,
 				       item->command == TODO_MERGE_AND_EDIT ?
 				       EDIT_MSG | VERIFY_MSG : 0, opts);
-		else if (!is_noop(item->command))
+			if (item->commit)
+				record_in_rewritten(&item->commit->object.oid,
+						    peek_command(todo_list, 1));
+		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index 1a3e43d66ff..35a61ce90bb 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -157,4 +157,29 @@ test_expect_success 'refs/rewritten/* is worktree-local' '
 	test_cmp_rev HEAD "$(cat wt/b)"
 '
 
+test_expect_success 'post-rewrite hook and fixups work for merges' '
+	git checkout -b post-rewrite &&
+	test_commit same1 &&
+	git reset --hard HEAD^ &&
+	test_commit same2 &&
+	git merge -m "to fix up" same1 &&
+	echo same old same old >same2.t &&
+	test_tick &&
+	git commit --fixup HEAD same2.t &&
+	fixup="$(git rev-parse HEAD)" &&
+
+	mkdir -p .git/hooks &&
+	test_when_finished "rm .git/hooks/post-rewrite" &&
+	echo "cat >actual" | write_script .git/hooks/post-rewrite &&
+
+	test_tick &&
+	git rebase -i --autosquash --recreate-merges HEAD^^^ &&
+	printf "%s %s\n%s %s\n%s %s\n%s %s\n" >expect $(git rev-parse \
+		$fixup^^2 HEAD^2 \
+		$fixup^^ HEAD^ \
+		$fixup^ HEAD \
+		$fixup HEAD) &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 11/12] pull: accept --rebase=recreate to recreate the branch topology
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
                         ` (9 preceding siblings ...)
  2018-02-23 12:39       ` [PATCH v4 10/12] sequencer: handle post-rewrite for merge commands Johannes Schindelin
@ 2018-02-23 12:39       ` Johannes Schindelin
  2018-02-23 12:39       ` [PATCH v4 12/12] rebase -i: introduce --recreate-merges=[no-]rebase-cousins Johannes Schindelin
  2018-02-25 10:54       ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Jacob Keller
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-23 12:39 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Similar to the `preserve` mode simply passing the `--preserve-merges`
option to the `rebase` command, the `recreate` mode simply passes the
`--recreate-merges` option.

This will allow users to conveniently rebase non-trivial commit
topologies when pulling new commits, without flattening them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/config.txt               |  8 ++++++++
 Documentation/git-pull.txt             |  5 ++++-
 builtin/pull.c                         | 14 ++++++++++----
 builtin/remote.c                       |  2 ++
 contrib/completion/git-completion.bash |  2 +-
 5 files changed, 25 insertions(+), 6 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index f57e9cf10ca..8c9adea0d0c 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1058,6 +1058,10 @@ branch.<name>.rebase::
 	"git pull" is run. See "pull.rebase" for doing this in a non
 	branch-specific manner.
 +
+When recreate, also pass `--recreate-merges` along to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
@@ -2607,6 +2611,10 @@ pull.rebase::
 	pull" is run. See "branch.<name>.rebase" for setting this on a
 	per-branch basis.
 +
+When recreate, also pass `--recreate-merges` along to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
diff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt
index ce05b7a5b13..b4f9f057ea9 100644
--- a/Documentation/git-pull.txt
+++ b/Documentation/git-pull.txt
@@ -101,13 +101,16 @@ Options related to merging
 include::merge-options.txt[]
 
 -r::
---rebase[=false|true|preserve|interactive]::
+--rebase[=false|true|recreate|preserve|interactive]::
 	When true, rebase the current branch on top of the upstream
 	branch after fetching. If there is a remote-tracking branch
 	corresponding to the upstream branch and the upstream branch
 	was rebased since last fetched, the rebase uses that information
 	to avoid rebasing non-local changes.
 +
+When set to recreate, rebase with the `--recreate-merges` option passed
+to `git rebase` so that locally created merge commits will not be flattened.
++
 When set to preserve, rebase with the `--preserve-merges` option passed
 to `git rebase` so that locally created merge commits will not be flattened.
 +
diff --git a/builtin/pull.c b/builtin/pull.c
index 1876271af94..9da2cfa0bd3 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -27,14 +27,16 @@ enum rebase_type {
 	REBASE_FALSE = 0,
 	REBASE_TRUE,
 	REBASE_PRESERVE,
+	REBASE_RECREATE,
 	REBASE_INTERACTIVE
 };
 
 /**
  * Parses the value of --rebase. If value is a false value, returns
  * REBASE_FALSE. If value is a true value, returns REBASE_TRUE. If value is
- * "preserve", returns REBASE_PRESERVE. If value is a invalid value, dies with
- * a fatal error if fatal is true, otherwise returns REBASE_INVALID.
+ * "recreate", returns REBASE_RECREATE. If value is "preserve", returns
+ * REBASE_PRESERVE. If value is a invalid value, dies with a fatal error if
+ * fatal is true, otherwise returns REBASE_INVALID.
  */
 static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		int fatal)
@@ -47,6 +49,8 @@ static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		return REBASE_TRUE;
 	else if (!strcmp(value, "preserve"))
 		return REBASE_PRESERVE;
+	else if (!strcmp(value, "recreate"))
+		return REBASE_RECREATE;
 	else if (!strcmp(value, "interactive"))
 		return REBASE_INTERACTIVE;
 
@@ -130,7 +134,7 @@ static struct option pull_options[] = {
 	/* Options passed to git-merge or git-rebase */
 	OPT_GROUP(N_("Options related to merging")),
 	{ OPTION_CALLBACK, 'r', "rebase", &opt_rebase,
-	  "false|true|preserve|interactive",
+	  "false|true|recreate|preserve|interactive",
 	  N_("incorporate changes by rebasing rather than merging"),
 	  PARSE_OPT_OPTARG, parse_opt_rebase },
 	OPT_PASSTHRU('n', NULL, &opt_diffstat, NULL,
@@ -800,7 +804,9 @@ static int run_rebase(const struct object_id *curr_head,
 	argv_push_verbosity(&args);
 
 	/* Options passed to git-rebase */
-	if (opt_rebase == REBASE_PRESERVE)
+	if (opt_rebase == REBASE_RECREATE)
+		argv_array_push(&args, "--recreate-merges");
+	else if (opt_rebase == REBASE_PRESERVE)
 		argv_array_push(&args, "--preserve-merges");
 	else if (opt_rebase == REBASE_INTERACTIVE)
 		argv_array_push(&args, "--interactive");
diff --git a/builtin/remote.c b/builtin/remote.c
index d95bf904c3b..b7d0f7ce596 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -306,6 +306,8 @@ static int config_read_branches(const char *key, const char *value, void *cb)
 				info->rebase = v;
 			else if (!strcmp(value, "preserve"))
 				info->rebase = NORMAL_REBASE;
+			else if (!strcmp(value, "recreate"))
+				info->rebase = NORMAL_REBASE;
 			else if (!strcmp(value, "interactive"))
 				info->rebase = INTERACTIVE_REBASE;
 		}
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 38bba3835c6..3d44cb6890c 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2182,7 +2182,7 @@ _git_config ()
 		return
 		;;
 	branch.*.rebase)
-		__gitcomp "false true preserve interactive"
+		__gitcomp "false true recreate preserve interactive"
 		return
 		;;
 	remote.pushdefault)
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 12/12] rebase -i: introduce --recreate-merges=[no-]rebase-cousins
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
                         ` (10 preceding siblings ...)
  2018-02-23 12:39       ` [PATCH v4 11/12] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
@ 2018-02-23 12:39       ` Johannes Schindelin
  2018-02-25 10:54       ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Jacob Keller
  12 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-23 12:39 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

This one is a bit tricky to explain, so let's try with a diagram:

        C
      /   \
A - B - E - F
  \   /
    D

To illustrate what this new mode is all about, let's consider what
happens upon `git rebase -i --recreate-merges B`, in particular to
the commit `D`. So far, the new branch structure would be:

       --- C' --
      /         \
A - B ------ E' - F'
      \    /
        D'

This is not really preserving the branch topology from before! The
reason is that the commit `D` does not have `B` as ancestor, and
therefore it gets rebased onto `B`.

This is unintuitive behavior. Even worse, when recreating branch
structure, most use cases would appear to want cousins *not* to be
rebased onto the new base commit. For example, Git for Windows (the
heaviest user of the Git garden shears, which served as the blueprint
for --recreate-merges) frequently merges branches from `next` early, and
these branches certainly do *not* want to be rebased. In the example
above, the desired outcome would look like this:

       --- C' --
      /         \
A - B ------ E' - F'
  \        /
   -- D' --

Let's introduce the term "cousins" for such commits ("D" in the
example), and let's not rebase them by default, introducing the new
"rebase-cousins" mode for use cases where they should be rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt      |  7 ++++++-
 builtin/rebase--helper.c          |  9 ++++++++-
 git-rebase--interactive.sh        |  1 +
 git-rebase.sh                     | 12 +++++++++++-
 sequencer.c                       |  4 ++++
 sequencer.h                       |  6 ++++++
 t/t3430-rebase-recreate-merges.sh | 23 +++++++++++++++++++++++
 7 files changed, 59 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 5e056c8ab6b..c5a77599c47 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -373,11 +373,16 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
---recreate-merges::
+--recreate-merges[=(rebase-cousins|no-rebase-cousins)]::
 	Recreate merge commits instead of flattening the history by replaying
 	merges. Merge conflict resolutions or manual amendments to merge
 	commits are not recreated automatically, but have to be recreated
 	manually.
++
+By default, or when `no-rebase-cousins` was specified, commits which do not
+have `<upstream>` as direct ancestor keep their original branch point.
+If the `rebase-cousins` mode is turned on, such commits are rebased onto
+`<upstream>` (or `<onto>`, if specified).
 
 -p::
 --preserve-merges::
diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index a5b07c43c96..5d1f12de57b 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
 	unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
-	int abbreviate_commands = 0;
+	int abbreviate_commands = 0, rebase_cousins = -1;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
 		CHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH,
@@ -25,6 +25,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
 			N_("allow commits with empty messages")),
 		OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge commits")),
+		OPT_BOOL(0, "rebase-cousins", &rebase_cousins,
+			 N_("keep original branch points of cousins")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -59,8 +61,13 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
 	flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
+	flags |= rebase_cousins > 0 ? TODO_LIST_REBASE_COUSINS : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
+	if (rebase_cousins >= 0 && !recreate_merges)
+		warning(_("--[no-]rebase-cousins has no effect without "
+			  "--recreate-merges"));
+
 	if (command == CONTINUE && argc == 1)
 		return !!sequencer_continue(&opts);
 	if (command == ABORT && argc == 1)
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index f5c8db2fdf8..679d79e0d17 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -907,6 +907,7 @@ if test t != "$preserve_merges"
 then
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
 		${recreate_merges:+--recreate-merges} \
+		${rebase_cousins:+--rebase-cousins} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 else
diff --git a/git-rebase.sh b/git-rebase.sh
index 528fa0073ac..9487e543bec 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,7 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
-recreate-merges!   try to recreate merges instead of skipping them
+recreate-merges?   try to recreate merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -89,6 +89,7 @@ state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
 recreate_merges=
+rebase_cousins=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -273,6 +274,15 @@ do
 		recreate_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
 		;;
+	--recreate-merges=*)
+		recreate_merges=t
+		case "${1#*=}" in
+		rebase-cousins) rebase_cousins=t;;
+		no-rebase-cousins) rebase_cousins=;;
+		*) die "Unknown mode: $1";;
+		esac
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/sequencer.c b/sequencer.c
index 85ce37cb99f..b2bf63029d4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3459,6 +3459,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 				   unsigned flags)
 {
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	int rebase_cousins = flags & TODO_LIST_REBASE_COUSINS;
 	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
 	struct strbuf label = STRBUF_INIT;
 	struct commit_list *commits = NULL, **tail = &commits, *iter;
@@ -3634,6 +3635,9 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 					   &commit->object.oid);
 			if (entry)
 				to = entry->string;
+			else if (!rebase_cousins)
+				to = label_oid(&commit->object.oid, NULL,
+					       &state);
 
 			if (!to || !strcmp(to, "onto"))
 				fprintf(out, "%s onto\n", cmd_reset);
diff --git a/sequencer.h b/sequencer.h
index 7c7c67d623c..739dd0fa92b 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -60,6 +60,12 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
 #define TODO_LIST_RECREATE_MERGES (1U << 3)
+/*
+ * When recreating merges, commits that do have the base commit as ancestor
+ * ("cousins") are *not* rebased onto the new base by default. If those
+ * commits should be rebased onto the new base, this flag needs to be passed.
+ */
+#define TODO_LIST_REBASE_COUSINS (1U << 4)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index 35a61ce90bb..9a59f12b670 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -143,6 +143,29 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'do not rebase cousins unless asked for' '
+	write_script copy-editor.sh <<-\EOF &&
+	cp "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	EOF
+
+	test_config sequence.editor \""$PWD"/copy-editor.sh\" &&
+	git checkout -b cousins master &&
+	before="$(git rev-parse --verify HEAD)" &&
+	test_tick &&
+	git rebase -i --recreate-merges HEAD^ &&
+	test_cmp_rev HEAD $before &&
+	test_tick &&
+	git rebase -i --recreate-merges=rebase-cousins HEAD^ &&
+	test_cmp_graph HEAD^.. <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	|/
+	o H
+	EOF
+'
+
 test_expect_success 'refs/rewritten/* is worktree-local' '
 	git worktree add wt &&
 	cat >wt/script-from-scratch <<-\EOF &&
-- 
2.16.1.windows.4

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v4 00/12] rebase -i: offer to recreate merge commits
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
                         ` (11 preceding siblings ...)
  2018-02-23 12:39       ` [PATCH v4 12/12] rebase -i: introduce --recreate-merges=[no-]rebase-cousins Johannes Schindelin
@ 2018-02-25 10:54       ` Jacob Keller
  2018-02-26 20:49         ` Johannes Schindelin
  12 siblings, 1 reply; 412+ messages in thread
From: Jacob Keller @ 2018-02-25 10:54 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Git mailing list, Junio C Hamano, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

On Fri, Feb 23, 2018 at 4:35 AM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> Changes since v3:
>
> - fixed a grammar error in "introduce the `merge` command"'s commit message.
>
> - fixed a couple of resource leaks in safe_append() and do_reset(), pointed
>   out by Eric Sunshine.
>

The interdiff seems incorrect for such a small list of changes, it
appears like large sections of code added by this series appear in the
interdiff without subtractions from the previous versions? Is all that
code new to v3? If not, I'd suspect you accidentally diffed between
the wrong points.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v4 00/12] rebase -i: offer to recreate merge commits
  2018-02-25 10:54       ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Jacob Keller
@ 2018-02-26 20:49         ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 20:49 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Git mailing list, Junio C Hamano, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Hi Jake,

On Sun, 25 Feb 2018, Jacob Keller wrote:

> On Fri, Feb 23, 2018 at 4:35 AM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > Changes since v3:
> >
> > - fixed a grammar error in "introduce the `merge` command"'s commit message.
> >
> > - fixed a couple of resource leaks in safe_append() and do_reset(), pointed
> >   out by Eric Sunshine.
> >
> 
> The interdiff seems incorrect for such a small list of changes, it
> appears like large sections of code added by this series appear in the
> interdiff without subtractions from the previous versions? Is all that
> code new to v3? If not, I'd suspect you accidentally diffed between
> the wrong points.

Indeed, it seems that I messed this iteration up rather well. Will redo.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v5 00/12] rebase -i: offer to recreate merge commits
  2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
                       ` (12 preceding siblings ...)
  2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
@ 2018-02-26 21:29     ` Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 01/12] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
                         ` (13 more replies)
  13 siblings, 14 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 21:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Once upon a time, I dreamt of an interactive rebase that would not
flatten branch structure, but instead recreate the commit topology
faithfully.

My original attempt was --preserve-merges, but that design was so
limited that I did not even enable it in interactive mode.

Subsequently, it *was* enabled in interactive mode, with the predictable
consequences: as the --preserve-merges design does not allow for
specifying the parents of merge commits explicitly, all the new commits'
parents are defined *implicitly* by the previous commit history, and
hence it is *not possible to even reorder commits*.

This design flaw cannot be fixed. Not without a complete re-design, at
least. This patch series offers such a re-design.

Think of --recreate-merges as "--preserve-merges done right". It
introduces new verbs for the todo list, `label`, `reset` and `merge`.
For a commit topology like this:

            A - B - C
              \   /
                D

the generated todo list would look like this:

            # branch D
            pick 0123 A
            label branch-point
            pick 1234 D
            label D

            reset branch-point
            pick 2345 B
            merge -C 3456 D # C

There are more patches in the pipeline, based on this patch series, but
left for later in the interest of reviewable patch series: one mini
series to use the sequencer even for `git rebase -i --root`, and another
one to add support for octopus merges to --recreate-merges.

Changes since v3:

- (sorry for the broken iteration v4)

- fixed a grammar error in "introduce the `merge` command"'s commit message.

- fixed a couple of resource leaks in safe_append() and do_reset(), pointed
  out by Eric Sunshine.


Johannes Schindelin (11):
  sequencer: avoid using errno clobbered by rollback_lock_file()
  sequencer: make rearrange_squash() a bit more obvious
  sequencer: introduce new commands to reset the revision
  sequencer: introduce the `merge` command
  sequencer: fast-forward merge commits, if possible
  rebase-helper --make-script: introduce a flag to recreate merges
  rebase: introduce the --recreate-merges option
  sequencer: make refs generated by the `label` command worktree-local
  sequencer: handle post-rewrite for merge commands
  pull: accept --rebase=recreate to recreate the branch topology
  rebase -i: introduce --recreate-merges=[no-]rebase-cousins

Stefan Beller (1):
  git-rebase--interactive: clarify arguments

 Documentation/config.txt               |   8 +
 Documentation/git-pull.txt             |   5 +-
 Documentation/git-rebase.txt           |  14 +-
 builtin/pull.c                         |  14 +-
 builtin/rebase--helper.c               |  13 +-
 builtin/remote.c                       |   2 +
 contrib/completion/git-completion.bash |   4 +-
 git-rebase--interactive.sh             |  22 +-
 git-rebase.sh                          |  16 +
 refs.c                                 |   3 +-
 sequencer.c                            | 742 ++++++++++++++++++++++++++++++++-
 sequencer.h                            |   7 +
 t/t3430-rebase-recreate-merges.sh      | 208 +++++++++
 13 files changed, 1027 insertions(+), 31 deletions(-)
 create mode 100755 t/t3430-rebase-recreate-merges.sh


base-commit: e3a80781f5932f5fea12a49eb06f3ade4ed8945c
Published-As: https://github.com/dscho/git/releases/tag/recreate-merges-v5
Fetch-It-Via: git fetch https://github.com/dscho/git recreate-merges-v5

Interdiff vs v4:
 diff --git a/sequencer.c b/sequencer.c
 index 63ae71a7512..b2bf63029d4 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -2514,14 +2514,17 @@ static int safe_append(const char *filename, const char *fmt, ...)
  
  	if (write_in_full(fd, buf.buf, buf.len) < 0) {
  		error_errno(_("could not write to '%s'"), filename);
 +		strbuf_release(&buf);
  		rollback_lock_file(&lock);
  		return -1;
  	}
  	if (commit_lock_file(&lock) < 0) {
 +		strbuf_release(&buf);
  		rollback_lock_file(&lock);
  		return error(_("failed to finalize '%s'"), filename);
  	}
  
 +	strbuf_release(&buf);
  	return 0;
  }
  
 @@ -2601,8 +2604,11 @@ static int do_reset(const char *name, int len, struct replay_opts *opts)
  	unpack_tree_opts.update = 1;
  	unpack_tree_opts.reset = 1;
  
 -	if (read_cache_unmerged())
 +	if (read_cache_unmerged()) {
 +		rollback_lock_file(&lock);
 +		strbuf_release(&ref_name);
  		return error_resolve_conflict(_(action_name(opts)));
 +	}
  
  	if (!fill_tree_descriptor(&desc, &oid)) {
  		error(_("failed to find tree of %s"), oid_to_hex(&oid));
-- 
2.16.1.windows.4


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v5 01/12] sequencer: avoid using errno clobbered by rollback_lock_file()
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
@ 2018-02-26 21:29       ` Johannes Schindelin
  2018-02-27 21:33         ` Martin Ågren
  2018-02-26 21:29       ` [PATCH v5 02/12] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
                         ` (12 subsequent siblings)
  13 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 21:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

As pointed out in a review of the `--recreate-merges` patch series,
`rollback_lock_file()` clobbers errno. Therefore, we have to report the
error message that uses errno before calling said function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index e9baaf59bd9..5aa3dc3c95c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -345,12 +345,14 @@ static int write_message(const void *buf, size_t len, const char *filename,
 	if (msg_fd < 0)
 		return error_errno(_("could not lock '%s'"), filename);
 	if (write_in_full(msg_fd, buf, len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write to '%s'"), filename);
+		return -1;
 	}
 	if (append_eol && write(msg_fd, "\n", 1) < 0) {
+		error_errno(_("could not write eol to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write eol to '%s'"), filename);
+		return -1;
 	}
 	if (commit_lock_file(&msg_file) < 0) {
 		rollback_lock_file(&msg_file);
@@ -2106,16 +2108,17 @@ static int save_head(const char *head)
 
 	fd = hold_lock_file_for_update(&head_lock, git_path_head_file(), 0);
 	if (fd < 0) {
+		error_errno(_("could not lock HEAD"));
 		rollback_lock_file(&head_lock);
-		return error_errno(_("could not lock HEAD"));
+		return -1;
 	}
 	strbuf_addf(&buf, "%s\n", head);
 	written = write_in_full(fd, buf.buf, buf.len);
 	strbuf_release(&buf);
 	if (written < 0) {
+		error_errno(_("could not write to '%s'"), git_path_head_file());
 		rollback_lock_file(&head_lock);
-		return error_errno(_("could not write to '%s'"),
-				   git_path_head_file());
+		return -1;
 	}
 	if (commit_lock_file(&head_lock) < 0) {
 		rollback_lock_file(&head_lock);
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 02/12] sequencer: make rearrange_squash() a bit more obvious
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 01/12] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
@ 2018-02-26 21:29       ` Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 03/12] git-rebase--interactive: clarify arguments Johannes Schindelin
                         ` (11 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 21:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

There are some commands that have to be skipped from rearranging by virtue
of not handling any commits.

However, the logic was not quite obvious: it skipped commands based on
their position in the enum todo_command.

Instead, let's make it explicit that we skip all commands that do not
handle any commit. With one exception: the `drop` command, because it,
well, drops the commit and is therefore not eligible to rearranging.

Note: this is a bit academic at the moment because the only time we call
`rearrange_squash()` is directly after generating the todo list, when we
have nothing but `pick` commands anyway.

However, the upcoming `merge` command *will* want to be handled by that
function, and it *can* handle commits.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 5aa3dc3c95c..cfa01d3bdd2 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3412,7 +3412,7 @@ int rearrange_squash(void)
 		struct subject2item_entry *entry;
 
 		next[i] = tail[i] = -1;
-		if (item->command >= TODO_EXEC) {
+		if (!item->commit || item->command == TODO_DROP) {
 			subjects[i] = NULL;
 			continue;
 		}
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 03/12] git-rebase--interactive: clarify arguments
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 01/12] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 02/12] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
@ 2018-02-26 21:29       ` Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 04/12] sequencer: introduce new commands to reset the revision Johannes Schindelin
                         ` (10 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 21:29 UTC (permalink / raw)
  To: git
  Cc: Stefan Beller, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood

From: Stefan Beller <stefanbeller@gmail.com>

Up to now each command took a commit as its first argument and ignored
the rest of the line (usually the subject of the commit)

Now that we are about to introduce commands that take different
arguments, clarify each command by giving the argument list.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 81c5b428757..a2659fea982 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -155,13 +155,13 @@ reschedule_last_action () {
 append_todo_help () {
 	gettext "
 Commands:
-p, pick = use commit
-r, reword = use commit, but edit the commit message
-e, edit = use commit, but stop for amending
-s, squash = use commit, but meld into previous commit
-f, fixup = like \"squash\", but discard this commit's log message
-x, exec = run command (the rest of the line) using shell
-d, drop = remove commit
+p, pick <commit> = use commit
+r, reword <commit> = use commit, but edit the commit message
+e, edit <commit> = use commit, but stop for amending
+s, squash <commit> = use commit, but meld into previous commit
+f, fixup <commit> = like \"squash\", but discard this commit's log message
+x, exec <commit> = run command (the rest of the line) using shell
+d, drop <commit> = remove commit
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 04/12] sequencer: introduce new commands to reset the revision
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
                         ` (2 preceding siblings ...)
  2018-02-26 21:29       ` [PATCH v5 03/12] git-rebase--interactive: clarify arguments Johannes Schindelin
@ 2018-02-26 21:29       ` Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 05/12] sequencer: introduce the `merge` command Johannes Schindelin
                         ` (9 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 21:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

In the upcoming commits, we will teach the sequencer to recreate merges.
This will be done in a very different way from the unfortunate design of
`git rebase --preserve-merges` (which does not allow for reordering
commits, or changing the branch topology).

The main idea is to introduce new todo list commands, to support
labeling the current revision with a given name, resetting the current
revision to a previous state, and  merging labeled revisions.

This idea was developed in Git for Windows' Git garden shears (that are
used to maintain the "thicket of branches" on top of upstream Git), and
this patch is part of the effort to make it available to a wider
audience, as well as to make the entire process more robust (by
implementing it in a safe and portable language rather than a Unix shell
script).

This commit implements the commands to label, and to reset to, given
revisions. The syntax is:

	label <name>
	reset <name>

Internally, the `label <name>` command creates the ref
`refs/rewritten/<name>`. This makes it possible to work with the labeled
revisions interactively, or in a scripted fashion (e.g. via the todo
list command `exec`).

These temporary refs are removed upon sequencer_remove_state(), so that
even a `git rebase --abort` cleans them up.

We disallow '#' as label because that character will be used as separator
in the upcoming `merge` command.

Later in this patch series, we will mark the `refs/rewritten/` refs as
worktree-local, to allow for interactive rebases to be run in parallel in
worktrees linked to the same repository.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   2 +
 sequencer.c                | 196 +++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 192 insertions(+), 6 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index a2659fea982..501f09b28c4 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -162,6 +162,8 @@ s, squash <commit> = use commit, but meld into previous commit
 f, fixup <commit> = like \"squash\", but discard this commit's log message
 x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
+l, label <label> = label current HEAD with a name
+t, reset <label> = reset HEAD to a label
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index cfa01d3bdd2..e25522ecdf1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -23,6 +23,8 @@
 #include "hashmap.h"
 #include "notes-utils.h"
 #include "sigchain.h"
+#include "unpack-trees.h"
+#include "worktree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -120,6 +122,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
 static GIT_PATH_FUNC(rebase_path_rewritten_pending,
 	"rebase-merge/rewritten-pending")
+
+/*
+ * The path of the file listing refs that need to be deleted after the rebase
+ * finishes. This is used by the `label` command to record the need for cleanup.
+ */
+static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
+
 /*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
@@ -244,18 +253,33 @@ static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
 
 int sequencer_remove_state(struct replay_opts *opts)
 {
-	struct strbuf dir = STRBUF_INIT;
+	struct strbuf buf = STRBUF_INIT;
 	int i;
 
+	if (strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
+		char *p = buf.buf;
+		while (*p) {
+			char *eol = strchr(p, '\n');
+			if (eol)
+				*eol = '\0';
+			if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
+				warning(_("could not delete '%s'"), p);
+			if (!eol)
+				break;
+			p = eol + 1;
+		}
+	}
+
 	free(opts->gpg_sign);
 	free(opts->strategy);
 	for (i = 0; i < opts->xopts_nr; i++)
 		free(opts->xopts[i]);
 	free(opts->xopts);
 
-	strbuf_addstr(&dir, get_dir(opts));
-	remove_dir_recursively(&dir, 0);
-	strbuf_release(&dir);
+	strbuf_reset(&buf);
+	strbuf_addstr(&buf, get_dir(opts));
+	remove_dir_recursively(&buf, 0);
+	strbuf_release(&buf);
 
 	return 0;
 }
@@ -1280,6 +1304,8 @@ enum todo_command {
 	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
+	TODO_LABEL,
+	TODO_RESET,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -1298,6 +1324,8 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
+	{ 'l', "label" },
+	{ 't', "reset" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1803,7 +1831,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return error(_("missing arguments for %s"),
 			     command_to_string(item->command));
 
-	if (item->command == TODO_EXEC) {
+	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
+	    item->command == TODO_RESET) {
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = (int)(eol - bol);
@@ -2444,6 +2473,157 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int safe_append(const char *filename, const char *fmt, ...)
+{
+	va_list ap;
+	struct lock_file lock = LOCK_INIT;
+	int fd = hold_lock_file_for_update(&lock, filename,
+					   LOCK_REPORT_ON_ERROR);
+	struct strbuf buf = STRBUF_INIT;
+
+	if (fd < 0)
+		return -1;
+
+	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT)
+		return error_errno(_("could not read '%s'"), filename);
+	strbuf_complete(&buf, '\n');
+	va_start(ap, fmt);
+	strbuf_vaddf(&buf, fmt, ap);
+	va_end(ap);
+
+	if (write_in_full(fd, buf.buf, buf.len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
+		strbuf_release(&buf);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	if (commit_lock_file(&lock) < 0) {
+		strbuf_release(&buf);
+		rollback_lock_file(&lock);
+		return error(_("failed to finalize '%s'"), filename);
+	}
+
+	strbuf_release(&buf);
+	return 0;
+}
+
+static int do_label(const char *name, int len)
+{
+	struct ref_store *refs = get_main_ref_store();
+	struct ref_transaction *transaction;
+	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
+	struct strbuf msg = STRBUF_INIT;
+	int ret = 0;
+	struct object_id head_oid;
+
+	if (len == 1 && *name == '#')
+		return error("Illegal label name: '%.*s'", len, name);
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
+
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction) {
+		error("%s", err.buf);
+		ret = -1;
+	} else if (get_oid("HEAD", &head_oid)) {
+		error(_("could not read HEAD"));
+		ret = -1;
+	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
+					  NULL, 0, msg.buf, &err) < 0 ||
+		   ref_transaction_commit(transaction, &err)) {
+		error("%s", err.buf);
+		ret = -1;
+	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
+	strbuf_release(&msg);
+
+	if (!ret)
+		ret = safe_append(rebase_path_refs_to_delete(),
+				  "%s\n", ref_name.buf);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
+static int do_reset(const char *name, int len, struct replay_opts *opts)
+{
+	struct strbuf ref_name = STRBUF_INIT;
+	struct object_id oid;
+	struct lock_file lock = LOCK_INIT;
+	struct tree_desc desc;
+	struct tree *tree;
+	struct unpack_trees_options unpack_tree_opts;
+	int ret = 0, i;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	/* Determine the length of the label */
+	for (i = 0; i < len; i++)
+		if (isspace(name[i]))
+			len = i;
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	if (get_oid(ref_name.buf, &oid) &&
+	    get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
+		error(_("could not read '%s'"), ref_name.buf);
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
+	unpack_tree_opts.head_idx = 1;
+	unpack_tree_opts.src_index = &the_index;
+	unpack_tree_opts.dst_index = &the_index;
+	unpack_tree_opts.fn = oneway_merge;
+	unpack_tree_opts.merge = 1;
+	unpack_tree_opts.update = 1;
+	unpack_tree_opts.reset = 1;
+
+	if (read_cache_unmerged()) {
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return error_resolve_conflict(_(action_name(opts)));
+	}
+
+	if (!fill_tree_descriptor(&desc, &oid)) {
+		error(_("failed to find tree of %s"), oid_to_hex(&oid));
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	if (unpack_trees(1, &desc, &unpack_tree_opts)) {
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	tree = parse_tree_indirect(&oid);
+	prime_cache_tree(&the_index, tree);
+
+	if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
+		ret = error(_("could not write index"));
+	free((void *)desc.buffer);
+
+	if (!ret) {
+		struct strbuf msg = STRBUF_INIT;
+
+		strbuf_addf(&msg, "(rebase -i) reset '%.*s'", len, name);
+		ret = update_ref(msg.buf, "HEAD", &oid, NULL, 0,
+				 UPDATE_REFS_MSG_ON_ERR);
+		strbuf_release(&msg);
+	}
+
+	strbuf_release(&ref_name);
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2627,7 +2807,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				/* `current` will be incremented below */
 				todo_list->current = -1;
 			}
-		} else if (!is_noop(item->command))
+		} else if (item->command == TODO_LABEL)
+			res = do_label(item->arg, item->arg_len);
+		else if (item->command == TODO_RESET)
+			res = do_reset(item->arg, item->arg_len, opts);
+		else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 05/12] sequencer: introduce the `merge` command
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
                         ` (3 preceding siblings ...)
  2018-02-26 21:29       ` [PATCH v5 04/12] sequencer: introduce new commands to reset the revision Johannes Schindelin
@ 2018-02-26 21:29       ` Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 06/12] sequencer: fast-forward merge commits, if possible Johannes Schindelin
                         ` (8 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 21:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

This patch is part of the effort to reimplement `--preserve-merges` with
a substantially improved design, a design that has been developed in the
Git for Windows project to maintain the dozens of Windows-specific patch
series on top of upstream Git.

The previous patch implemented the `label` and `reset` commands to label
commits and to reset to labeled commits. This patch adds the `merge`
command, with the following syntax:

	merge [-C <commit>] <rev> # <oneline>

The <commit> parameter in this instance is the *original* merge commit,
whose author and message will be used for the merge commit that is about
to be created.

The <rev> parameter refers to the (possibly rewritten) revision to
merge. Let's see an example of a todo list:

	label onto

	# Branch abc
	reset onto
	pick deadbeef Hello, world!
	label abc

	reset onto
	pick cafecafe And now for something completely different
	merge -C baaabaaa abc # Merge the branch 'abc' into master

To edit the merge commit's message (a "reword" for merges, if you will),
use `-c` (lower-case) instead of `-C`; this convention was borrowed from
`git commit` that also supports `-c` and `-C` with similar meanings.

To create *new* merges, i.e. without copying the commit message from an
existing commit, simply omit the `-C <commit>` parameter (which will
open an editor for the merge message):

	merge abc

This comes in handy when splitting a branch into two or more branches.

Note: this patch only adds support for recursive merges, to keep things
simple. Support for octopus merges will be added later in a separate
patch series, support for merges using strategies other than the
recursive merge is left for the future.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   4 ++
 sequencer.c                | 158 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 162 insertions(+)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 501f09b28c4..2d8bbe20b74 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -164,6 +164,10 @@ x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
 l, label <label> = label current HEAD with a name
 t, reset <label> = reset HEAD to a label
+m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
+.       create a merge commit using the original merge commit's
+.       message (or the oneline, if no original merge commit was
+.       specified). Use -c <commit> to reword the commit message.
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index e25522ecdf1..64dbd1d3e2e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1306,6 +1306,8 @@ enum todo_command {
 	TODO_EXEC,
 	TODO_LABEL,
 	TODO_RESET,
+	TODO_MERGE,
+	TODO_MERGE_AND_EDIT,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -1326,6 +1328,8 @@ static struct {
 	{ 'x', "exec" },
 	{ 'l', "label" },
 	{ 't', "reset" },
+	{ 'm', "merge" },
+	{ 0, "merge" }, /* MERGE_AND_EDIT */
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1839,6 +1843,21 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return 0;
 	}
 
+	if (item->command == TODO_MERGE) {
+		if (skip_prefix(bol, "-C", &bol))
+			bol += strspn(bol, " \t");
+		else if (skip_prefix(bol, "-c", &bol)) {
+			bol += strspn(bol, " \t");
+			item->command = TODO_MERGE_AND_EDIT;
+		} else {
+			item->command = TODO_MERGE_AND_EDIT;
+			item->commit = NULL;
+			item->arg = bol;
+			item->arg_len = (int)(eol - bol);
+			return 0;
+		}
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
@@ -2624,6 +2643,134 @@ static int do_reset(const char *name, int len, struct replay_opts *opts)
 	return ret;
 }
 
+static int do_merge(struct commit *commit, const char *arg, int arg_len,
+		    int run_commit_flags, struct replay_opts *opts)
+{
+	int merge_arg_len;
+	struct strbuf ref_name = STRBUF_INIT;
+	struct commit *head_commit, *merge_commit, *i;
+	struct commit_list *common, *j, *reversed = NULL;
+	struct merge_options o;
+	int ret;
+	static struct lock_file lock;
+
+	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
+		if (isspace(arg[merge_arg_len]))
+			break;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	head_commit = lookup_commit_reference_by_name("HEAD");
+	if (!head_commit) {
+		rollback_lock_file(&lock);
+		return error(_("cannot merge without a current revision"));
+	}
+
+	if (commit) {
+		const char *message = get_commit_buffer(commit, NULL);
+		const char *body;
+		int len;
+
+		if (!message) {
+			rollback_lock_file(&lock);
+			return error(_("could not get commit message of '%s'"),
+				     oid_to_hex(&commit->object.oid));
+		}
+		write_author_script(message);
+		find_commit_subject(message, &body);
+		len = strlen(body);
+		if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			unuse_commit_buffer(commit, message);
+			rollback_lock_file(&lock);
+			return -1;
+		}
+		unuse_commit_buffer(commit, message);
+	} else {
+		const char *p = arg + merge_arg_len;
+		struct strbuf buf = STRBUF_INIT;
+		int len;
+
+		strbuf_addf(&buf, "author %s", git_author_info(0));
+		write_author_script(buf.buf);
+		strbuf_reset(&buf);
+
+		p += strspn(p, " \t");
+		if (*p == '#' && isspace(p[1]))
+			p += 1 + strspn(p + 1, " \t");
+		if (*p)
+			len = strlen(p);
+		else {
+			strbuf_addf(&buf, "Merge branch '%.*s'",
+				    merge_arg_len, arg);
+			p = buf.buf;
+			len = buf.len;
+		}
+
+		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			strbuf_release(&buf);
+			rollback_lock_file(&lock);
+			return -1;
+		}
+		strbuf_release(&buf);
+	}
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
+	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	if (!merge_commit) {
+		/* fall back to non-rewritten ref or commit */
+		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
+		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	}
+	if (!merge_commit) {
+		error(_("could not resolve '%s'"), ref_name.buf);
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
+		      git_path_merge_head(), 0);
+	write_message("no-ff", 5, git_path_merge_mode(), 0);
+
+	common = get_merge_bases(head_commit, merge_commit);
+	for (j = common; j; j = j->next)
+		commit_list_insert(j->item, &reversed);
+	free_commit_list(common);
+
+	read_cache();
+	init_merge_options(&o);
+	o.branch1 = "HEAD";
+	o.branch2 = ref_name.buf;
+	o.buffer_output = 2;
+
+	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
+	if (ret <= 0)
+		fputs(o.obuf.buf, stdout);
+	strbuf_release(&o.obuf);
+	if (ret < 0) {
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return error(_("conflicts while merging '%.*s'"),
+			     merge_arg_len, arg);
+	}
+
+	if (active_cache_changed &&
+	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
+		strbuf_release(&ref_name);
+		return error(_("merge: Unable to write new index file"));
+	}
+	rollback_lock_file(&lock);
+
+	ret = run_git_commit(git_path_merge_msg(), opts, run_commit_flags);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2811,6 +2958,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_label(item->arg, item->arg_len);
 		else if (item->command == TODO_RESET)
 			res = do_reset(item->arg, item->arg_len, opts);
+		else if (item->command == TODO_MERGE ||
+			 item->command == TODO_MERGE_AND_EDIT)
+			res = do_merge(item->commit, item->arg, item->arg_len,
+				       item->command == TODO_MERGE_AND_EDIT ?
+				       EDIT_MSG | VERIFY_MSG : 0, opts);
 		else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
@@ -3292,8 +3444,14 @@ int transform_todos(unsigned flags)
 					  short_commit_name(item->commit) :
 					  oid_to_hex(&item->commit->object.oid);
 
+			if (item->command == TODO_MERGE)
+				strbuf_addstr(&buf, " -C");
+			else if (item->command == TODO_MERGE_AND_EDIT)
+				strbuf_addstr(&buf, " -c");
+
 			strbuf_addf(&buf, " %s", oid);
 		}
+
 		/* add all the rest */
 		if (!item->arg_len)
 			strbuf_addch(&buf, '\n');
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 06/12] sequencer: fast-forward merge commits, if possible
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
                         ` (4 preceding siblings ...)
  2018-02-26 21:29       ` [PATCH v5 05/12] sequencer: introduce the `merge` command Johannes Schindelin
@ 2018-02-26 21:29       ` Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 07/12] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
                         ` (7 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 21:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Just like with regular `pick` commands, if we are trying to recreate a
merge commit, we now test whether the parents of said commit match HEAD
and the commits to be merged, and fast-forward if possible.

This is not only faster, but also avoids unnecessary proliferation of
new objects.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 64dbd1d3e2e..361ec98f764 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2651,7 +2651,7 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	struct commit *head_commit, *merge_commit, *i;
 	struct commit_list *common, *j, *reversed = NULL;
 	struct merge_options o;
-	int ret;
+	int can_fast_forward, ret;
 	static struct lock_file lock;
 
 	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
@@ -2719,6 +2719,14 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		strbuf_release(&buf);
 	}
 
+	/*
+	 * If HEAD is not identical to the parent of the original merge commit,
+	 * we cannot fast-forward.
+	 */
+	can_fast_forward = opts->allow_ff && commit && commit->parents &&
+		!oidcmp(&commit->parents->item->object.oid,
+			&head_commit->object.oid);
+
 	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
 	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
 	if (!merge_commit) {
@@ -2732,6 +2740,17 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		rollback_lock_file(&lock);
 		return -1;
 	}
+
+	if (can_fast_forward && commit->parents->next &&
+	    !commit->parents->next->next &&
+	    !oidcmp(&commit->parents->next->item->object.oid,
+		    &merge_commit->object.oid)) {
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return fast_forward_to(&commit->object.oid,
+				       &head_commit->object.oid, 0, opts);
+	}
+
 	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
 		      git_path_merge_head(), 0);
 	write_message("no-ff", 5, git_path_merge_mode(), 0);
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 07/12] rebase-helper --make-script: introduce a flag to recreate merges
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
                         ` (5 preceding siblings ...)
  2018-02-26 21:29       ` [PATCH v5 06/12] sequencer: fast-forward merge commits, if possible Johannes Schindelin
@ 2018-02-26 21:29       ` Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 08/12] rebase: introduce the --recreate-merges option Johannes Schindelin
                         ` (6 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 21:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

The sequencer just learned new commands intended to recreate branch
structure (similar in spirit to --preserve-merges, but with a
substantially less-broken design).

Let's allow the rebase--helper to generate todo lists making use of
these commands, triggered by the new --recreate-merges option. For a
commit topology like this (where the HEAD points to C):

	- A - B - C
	    \   /
	      D

the generated todo list would look like this:

	# branch D
	pick 0123 A
	label branch-point
	pick 1234 D
	label D

	reset branch-point
	pick 2345 B
	merge -C 3456 D # C

To keep things simple, we first only implement support for merge commits
with exactly two parents, leaving support for octopus merges to a later
patch in this patch series.

As a special, hard-coded label, all merge-recreating todo lists start with
the command `label onto` so that we can later always refer to the revision
onto which everything is rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/rebase--helper.c |   4 +-
 sequencer.c              | 349 ++++++++++++++++++++++++++++++++++++++++++++++-
 sequencer.h              |   1 +
 3 files changed, 351 insertions(+), 3 deletions(-)

diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index ad074705bb5..a5b07c43c96 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -12,7 +12,7 @@ static const char * const builtin_rebase_helper_usage[] = {
 int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
-	unsigned flags = 0, keep_empty = 0;
+	unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
 	int abbreviate_commands = 0;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
@@ -24,6 +24,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
 		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
 			N_("allow commits with empty messages")),
+		OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge commits")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -57,6 +58,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
+	flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
 	if (command == CONTINUE && argc == 1)
diff --git a/sequencer.c b/sequencer.c
index 361ec98f764..01bafe2fe47 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -25,6 +25,8 @@
 #include "sigchain.h"
 #include "unpack-trees.h"
 #include "worktree.h"
+#include "oidmap.h"
+#include "oidset.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -3336,6 +3338,341 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
 	strbuf_release(&sob);
 }
 
+struct labels_entry {
+	struct hashmap_entry entry;
+	char label[FLEX_ARRAY];
+};
+
+static int labels_cmp(const void *fndata, const struct labels_entry *a,
+		      const struct labels_entry *b, const void *key)
+{
+	return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
+}
+
+struct string_entry {
+	struct oidmap_entry entry;
+	char string[FLEX_ARRAY];
+};
+
+struct label_state {
+	struct oidmap commit2label;
+	struct hashmap labels;
+	struct strbuf buf;
+};
+
+static const char *label_oid(struct object_id *oid, const char *label,
+			     struct label_state *state)
+{
+	struct labels_entry *labels_entry;
+	struct string_entry *string_entry;
+	struct object_id dummy;
+	size_t len;
+	int i;
+
+	string_entry = oidmap_get(&state->commit2label, oid);
+	if (string_entry)
+		return string_entry->string;
+
+	/*
+	 * For "uninteresting" commits, i.e. commits that are not to be
+	 * rebased, and which can therefore not be labeled, we use a unique
+	 * abbreviation of the commit name. This is slightly more complicated
+	 * than calling find_unique_abbrev() because we also need to make
+	 * sure that the abbreviation does not conflict with any other
+	 * label.
+	 *
+	 * We disallow "interesting" commits to be labeled by a string that
+	 * is a valid full-length hash, to ensure that we always can find an
+	 * abbreviation for any uninteresting commit's names that does not
+	 * clash with any other label.
+	 */
+	if (!label) {
+		char *p;
+
+		strbuf_reset(&state->buf);
+		strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
+		label = p = state->buf.buf;
+
+		find_unique_abbrev_r(p, oid->hash, default_abbrev);
+
+		/*
+		 * We may need to extend the abbreviated hash so that there is
+		 * no conflicting label.
+		 */
+		if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
+			size_t i = strlen(p) + 1;
+
+			oid_to_hex_r(p, oid);
+			for (; i < GIT_SHA1_HEXSZ; i++) {
+				char save = p[i];
+				p[i] = '\0';
+				if (!hashmap_get_from_hash(&state->labels,
+							   strihash(p), p))
+					break;
+				p[i] = save;
+			}
+		}
+	} else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
+		    !get_oid_hex(label, &dummy)) ||
+		   (len == 1 && *label == '#') ||
+		   hashmap_get_from_hash(&state->labels,
+					 strihash(label), label)) {
+		/*
+		 * If the label already exists, or if the label is a valid full
+		 * OID, or the label is a '#' (which we use as a separator
+		 * between merge heads and oneline), we append a dash and a
+		 * number to make it unique.
+		 */
+		struct strbuf *buf = &state->buf;
+
+		strbuf_reset(buf);
+		strbuf_add(buf, label, len);
+
+		for (i = 2; ; i++) {
+			strbuf_setlen(buf, len);
+			strbuf_addf(buf, "-%d", i);
+			if (!hashmap_get_from_hash(&state->labels,
+						   strihash(buf->buf),
+						   buf->buf))
+				break;
+		}
+
+		label = buf->buf;
+	}
+
+	FLEX_ALLOC_STR(labels_entry, label, label);
+	hashmap_entry_init(labels_entry, strihash(label));
+	hashmap_add(&state->labels, labels_entry);
+
+	FLEX_ALLOC_STR(string_entry, string, label);
+	oidcpy(&string_entry->entry.oid, oid);
+	oidmap_put(&state->commit2label, string_entry);
+
+	return string_entry->string;
+}
+
+static int make_script_with_merges(struct pretty_print_context *pp,
+				   struct rev_info *revs, FILE *out,
+				   unsigned flags)
+{
+	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
+	struct strbuf label = STRBUF_INIT;
+	struct commit_list *commits = NULL, **tail = &commits, *iter;
+	struct commit_list *tips = NULL, **tips_tail = &tips;
+	struct commit *commit;
+	struct oidmap commit2todo = OIDMAP_INIT;
+	struct string_entry *entry;
+	struct oidset interesting = OIDSET_INIT, child_seen = OIDSET_INIT,
+		shown = OIDSET_INIT;
+	struct label_state state = { OIDMAP_INIT, { NULL }, STRBUF_INIT };
+
+	int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
+	const char *cmd_pick = abbr ? "p" : "pick",
+		*cmd_label = abbr ? "l" : "label",
+		*cmd_reset = abbr ? "t" : "reset",
+		*cmd_merge = abbr ? "m" : "merge";
+
+	oidmap_init(&commit2todo, 0);
+	oidmap_init(&state.commit2label, 0);
+	hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
+	strbuf_init(&state.buf, 32);
+
+	if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
+		struct object_id *oid = &revs->cmdline.rev[0].item->oid;
+		FLEX_ALLOC_STR(entry, string, "onto");
+		oidcpy(&entry->entry.oid, oid);
+		oidmap_put(&state.commit2label, entry);
+	}
+
+	/*
+	 * First phase:
+	 * - get onelines for all commits
+	 * - gather all branch tips (i.e. 2nd or later parents of merges)
+	 * - label all branch tips
+	 */
+	while ((commit = get_revision(revs))) {
+		struct commit_list *to_merge;
+		int is_octopus;
+		const char *p1, *p2;
+		struct object_id *oid;
+
+		tail = &commit_list_insert(commit, tail)->next;
+		oidset_insert(&interesting, &commit->object.oid);
+
+		if ((commit->object.flags & PATCHSAME))
+			continue;
+
+		strbuf_reset(&oneline);
+		pretty_print_commit(pp, commit, &oneline);
+
+		to_merge = commit->parents ? commit->parents->next : NULL;
+		if (!to_merge) {
+			/* non-merge commit: easy case */
+			strbuf_reset(&buf);
+			if (!keep_empty && is_original_commit_empty(commit))
+				strbuf_addf(&buf, "%c ", comment_line_char);
+			strbuf_addf(&buf, "%s %s %s", cmd_pick,
+				    oid_to_hex(&commit->object.oid),
+				    oneline.buf);
+
+			FLEX_ALLOC_STR(entry, string, buf.buf);
+			oidcpy(&entry->entry.oid, &commit->object.oid);
+			oidmap_put(&commit2todo, entry);
+
+			continue;
+		}
+
+		is_octopus = to_merge && to_merge->next;
+
+		if (is_octopus)
+			BUG("Octopus merges not yet supported");
+
+		/* Create a label */
+		strbuf_reset(&label);
+		if (skip_prefix(oneline.buf, "Merge ", &p1) &&
+		    (p1 = strchr(p1, '\'')) &&
+		    (p2 = strchr(++p1, '\'')))
+			strbuf_add(&label, p1, p2 - p1);
+		else if (skip_prefix(oneline.buf, "Merge pull request ",
+				     &p1) &&
+			 (p1 = strstr(p1, " from ")))
+			strbuf_addstr(&label, p1 + strlen(" from "));
+		else
+			strbuf_addbuf(&label, &oneline);
+
+		for (p1 = label.buf; *p1; p1++)
+			if (isspace(*p1))
+				*(char *)p1 = '-';
+
+		strbuf_reset(&buf);
+		strbuf_addf(&buf, "%s -C %s",
+			    cmd_merge, oid_to_hex(&commit->object.oid));
+
+		/* label the tip of merged branch */
+		oid = &to_merge->item->object.oid;
+		strbuf_addch(&buf, ' ');
+
+		if (!oidset_contains(&interesting, oid))
+			strbuf_addstr(&buf, label_oid(oid, NULL, &state));
+		else {
+			tips_tail = &commit_list_insert(to_merge->item,
+							tips_tail)->next;
+
+			strbuf_addstr(&buf, label_oid(oid, label.buf, &state));
+		}
+		strbuf_addf(&buf, " # %s", oneline.buf);
+
+		FLEX_ALLOC_STR(entry, string, buf.buf);
+		oidcpy(&entry->entry.oid, &commit->object.oid);
+		oidmap_put(&commit2todo, entry);
+	}
+
+	/*
+	 * Second phase:
+	 * - label branch points
+	 * - add HEAD to the branch tips
+	 */
+	for (iter = commits; iter; iter = iter->next) {
+		struct commit_list *parent = iter->item->parents;
+		for (; parent; parent = parent->next) {
+			struct object_id *oid = &parent->item->object.oid;
+			if (!oidset_contains(&interesting, oid))
+				continue;
+			if (!oidset_contains(&child_seen, oid))
+				oidset_insert(&child_seen, oid);
+			else
+				label_oid(oid, "branch-point", &state);
+		}
+
+		/* Add HEAD as implict "tip of branch" */
+		if (!iter->next)
+			tips_tail = &commit_list_insert(iter->item,
+							tips_tail)->next;
+	}
+
+	/*
+	 * Third phase: output the todo list. This is a bit tricky, as we
+	 * want to avoid jumping back and forth between revisions. To
+	 * accomplish that goal, we walk backwards from the branch tips,
+	 * gathering commits not yet shown, reversing the list on the fly,
+	 * then outputting that list (labeling revisions as needed).
+	 */
+	fprintf(out, "%s onto\n", cmd_label);
+	for (iter = tips; iter; iter = iter->next) {
+		struct commit_list *list = NULL, *iter2;
+
+		commit = iter->item;
+		if (oidset_contains(&shown, &commit->object.oid))
+			continue;
+		entry = oidmap_get(&state.commit2label, &commit->object.oid);
+
+		if (entry)
+			fprintf(out, "\n# Branch %s\n", entry->string);
+		else
+			fprintf(out, "\n");
+
+		while (oidset_contains(&interesting, &commit->object.oid) &&
+		       !oidset_contains(&shown, &commit->object.oid)) {
+			commit_list_insert(commit, &list);
+			if (!commit->parents) {
+				commit = NULL;
+				break;
+			}
+			commit = commit->parents->item;
+		}
+
+		if (!commit)
+			fprintf(out, "%s onto\n", cmd_reset);
+		else {
+			const char *to = NULL;
+
+			entry = oidmap_get(&state.commit2label,
+					   &commit->object.oid);
+			if (entry)
+				to = entry->string;
+
+			if (!to || !strcmp(to, "onto"))
+				fprintf(out, "%s onto\n", cmd_reset);
+			else {
+				strbuf_reset(&oneline);
+				pretty_print_commit(pp, commit, &oneline);
+				fprintf(out, "%s %s # %s\n",
+					cmd_reset, to, oneline.buf);
+			}
+		}
+
+		for (iter2 = list; iter2; iter2 = iter2->next) {
+			struct object_id *oid = &iter2->item->object.oid;
+			entry = oidmap_get(&commit2todo, oid);
+			/* only show if not already upstream */
+			if (entry)
+				fprintf(out, "%s\n", entry->string);
+			entry = oidmap_get(&state.commit2label, oid);
+			if (entry)
+				fprintf(out, "%s %s\n",
+					cmd_label, entry->string);
+			oidset_insert(&shown, oid);
+		}
+
+		free_commit_list(list);
+	}
+
+	free_commit_list(commits);
+	free_commit_list(tips);
+
+	strbuf_release(&label);
+	strbuf_release(&oneline);
+	strbuf_release(&buf);
+
+	oidmap_free(&commit2todo, 1);
+	oidmap_free(&state.commit2label, 1);
+	hashmap_free(&state.labels, 1);
+	strbuf_release(&state.buf);
+
+	return 0;
+}
+
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags)
 {
@@ -3346,11 +3683,16 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	struct commit *commit;
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
 	const char *insn = flags & TODO_LIST_ABBREVIATE_CMDS ? "p" : "pick";
+	int recreate_merges = flags & TODO_LIST_RECREATE_MERGES;
 
 	init_revisions(&revs, NULL);
 	revs.verbose_header = 1;
-	revs.max_parents = 1;
-	revs.cherry_pick = 1;
+	if (recreate_merges)
+		revs.cherry_mark = 1;
+	else {
+		revs.max_parents = 1;
+		revs.cherry_pick = 1;
+	}
 	revs.limited = 1;
 	revs.reverse = 1;
 	revs.right_only = 1;
@@ -3374,6 +3716,9 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	if (prepare_revision_walk(&revs) < 0)
 		return error(_("make_script: error preparing revisions"));
 
+	if (recreate_merges)
+		return make_script_with_merges(&pp, &revs, out, flags);
+
 	while ((commit = get_revision(&revs))) {
 		strbuf_reset(&buf);
 		if (!keep_empty && is_original_commit_empty(commit))
diff --git a/sequencer.h b/sequencer.h
index e45b178dfc4..7c7c67d623c 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -59,6 +59,7 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_KEEP_EMPTY (1U << 0)
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
+#define TODO_LIST_RECREATE_MERGES (1U << 3)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 08/12] rebase: introduce the --recreate-merges option
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
                         ` (6 preceding siblings ...)
  2018-02-26 21:29       ` [PATCH v5 07/12] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
@ 2018-02-26 21:29       ` Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 09/12] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
                         ` (5 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 21:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Once upon a time, this here developer thought: wouldn't it be nice if,
say, Git for Windows' patches on top of core Git could be represented as
a thicket of branches, and be rebased on top of core Git in order to
maintain a cherry-pick'able set of patch series?

The original attempt to answer this was: git rebase --preserve-merges.

However, that experiment was never intended as an interactive option,
and it only piggy-backed on git rebase --interactive because that
command's implementation looked already very, very familiar: it was
designed by the same person who designed --preserve-merges: yours truly.

Some time later, some other developer (I am looking at you, Andreas!
;-)) decided that it would be a good idea to allow --preserve-merges to
be combined with --interactive (with caveats!) and the Git maintainer
(well, the interim Git maintainer during Junio's absence, that is)
agreed, and that is when the glamor of the --preserve-merges design
started to fall apart rather quickly and unglamorously.

The reason? In --preserve-merges mode, the parents of a merge commit (or
for that matter, of *any* commit) were not stated explicitly, but were
*implied* by the commit name passed to the `pick` command.

This made it impossible, for example, to reorder commits. Not to mention
to flatten the branch topology or, deity forbid, to split topic branches
into two.

Alas, these shortcomings also prevented that mode (whose original
purpose was to serve Git for Windows' needs, with the additional hope
that it may be useful to others, too) from serving Git for Windows'
needs.

Five years later, when it became really untenable to have one unwieldy,
big hodge-podge patch series of partly related, partly unrelated patches
in Git for Windows that was rebased onto core Git's tags from time to
time (earning the undeserved wrath of the developer of the ill-fated
git-remote-hg series that first obsoleted Git for Windows' competing
approach, only to be abandoned without maintainer later) was really
untenable, the "Git garden shears" were born [*1*/*2*]: a script,
piggy-backing on top of the interactive rebase, that would first
determine the branch topology of the patches to be rebased, create a
pseudo todo list for further editing, transform the result into a real
todo list (making heavy use of the `exec` command to "implement" the
missing todo list commands) and finally recreate the patch series on
top of the new base commit.

That was in 2013. And it took about three weeks to come up with the
design and implement it as an out-of-tree script. Needless to say, the
implementation needed quite a few years to stabilize, all the while the
design itself proved itself sound.

With this patch, the goodness of the Git garden shears comes to `git
rebase -i` itself. Passing the `--recreate-merges` option will generate
a todo list that can be understood readily, and where it is obvious
how to reorder commits. New branches can be introduced by inserting
`label` commands and calling `merge <label>`. And once this mode will
have become stable and universally accepted, we can deprecate the design
mistake that was `--preserve-merges`.

Link *1*:
https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
Link *2*:
https://github.com/git-for-windows/build-extra/blob/master/shears.sh

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt           |   9 +-
 contrib/completion/git-completion.bash |   2 +-
 git-rebase--interactive.sh             |   1 +
 git-rebase.sh                          |   6 ++
 t/t3430-rebase-recreate-merges.sh      | 146 +++++++++++++++++++++++++++++++++
 5 files changed, 162 insertions(+), 2 deletions(-)
 create mode 100755 t/t3430-rebase-recreate-merges.sh

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index d713951b86a..5e056c8ab6b 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -373,6 +373,12 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
+--recreate-merges::
+	Recreate merge commits instead of flattening the history by replaying
+	merges. Merge conflict resolutions or manual amendments to merge
+	commits are not recreated automatically, but have to be recreated
+	manually.
+
 -p::
 --preserve-merges::
 	Recreate merge commits instead of flattening the history by replaying
@@ -775,7 +781,8 @@ BUGS
 The todo list presented by `--preserve-merges --interactive` does not
 represent the topology of the revision graph.  Editing commits and
 rewording their commit messages should work fine, but attempts to
-reorder commits tend to produce counterintuitive results.
+reorder commits tend to produce counterintuitive results. Use
+--recreate-merges for a more faithful representation.
 
 For example, an attempt to rearrange
 ------------
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 88813e91244..38bba3835c6 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2008,7 +2008,7 @@ _git_rebase ()
 	--*)
 		__gitcomp "
 			--onto --merge --strategy --interactive
-			--preserve-merges --stat --no-stat
+			--recreate-merges --preserve-merges --stat --no-stat
 			--committer-date-is-author-date --ignore-date
 			--ignore-whitespace --whitespace=
 			--autosquash --no-autosquash
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 2d8bbe20b74..f5c8db2fdf8 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -906,6 +906,7 @@ fi
 if test t != "$preserve_merges"
 then
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
+		${recreate_merges:+--recreate-merges} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 else
diff --git a/git-rebase.sh b/git-rebase.sh
index b353c33d417..528fa0073ac 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,6 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
+recreate-merges!   try to recreate merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -87,6 +88,7 @@ type=
 state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
+recreate_merges=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -267,6 +269,10 @@ do
 	--allow-empty-message)
 		allow_empty_message=--allow-empty-message
 		;;
+	--recreate-merges)
+		recreate_merges=t
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
new file mode 100755
index 00000000000..0073601a206
--- /dev/null
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -0,0 +1,146 @@
+#!/bin/sh
+#
+# Copyright (c) 2017 Johannes E. Schindelin
+#
+
+test_description='git rebase -i --recreate-merges
+
+This test runs git rebase "interactively", retaining the branch structure by
+recreating merge commits.
+
+Initial setup:
+
+    -- B --                   (first)
+   /       \
+ A - C - D - E - H            (master)
+       \       /
+         F - G                (second)
+'
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
+
+test_expect_success 'setup' '
+	write_script replace-editor.sh <<-\EOF &&
+	mv "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	cp script-from-scratch "$1"
+	EOF
+
+	test_commit A &&
+	git checkout -b first &&
+	test_commit B &&
+	git checkout master &&
+	test_commit C &&
+	test_commit D &&
+	git merge --no-commit B &&
+	test_tick &&
+	git commit -m E &&
+	git tag -m E E &&
+	git checkout -b second C &&
+	test_commit F &&
+	test_commit G &&
+	git checkout master &&
+	git merge --no-commit G &&
+	test_tick &&
+	git commit -m H &&
+	git tag -m H H
+'
+
+cat >script-from-scratch <<\EOF
+label onto
+
+# onebranch
+pick G
+pick D
+label onebranch
+
+# second
+reset onto
+pick B
+label second
+
+reset onto
+merge -C H second
+merge onebranch # Merge the topic branch 'onebranch'
+EOF
+
+test_cmp_graph () {
+	cat >expect &&
+	git log --graph --boundary --format=%s "$@" >output &&
+	sed "s/ *$//" <output >output.trimmed &&
+	test_cmp expect output.trimmed
+}
+
+test_expect_success 'create completely different structure' '
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_tick &&
+	git rebase -i --recreate-merges A &&
+	test_cmp_graph <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	* |   H
+	|\ \
+	| |/
+	|/|
+	| * B
+	|/
+	* A
+	EOF
+'
+
+test_expect_success 'generate correct todo list' '
+	cat >expect <<-\EOF &&
+	label onto
+
+	reset onto
+	pick d9df450 B
+	label E
+
+	reset onto
+	pick 5dee784 C
+	label branch-point
+	pick ca2c861 F
+	pick 088b00a G
+	label H
+
+	reset branch-point # C
+	pick 12bd07b D
+	merge -C 2051b56 E # E
+	merge -C 233d48a H # H
+
+	EOF
+
+	grep -v "^#" <.git/ORIGINAL-TODO >output &&
+	test_cmp expect output
+'
+
+test_expect_success 'with a branch tip that was cherry-picked already' '
+	git checkout -b already-upstream master &&
+	base="$(git rev-parse --verify HEAD)" &&
+
+	test_commit A1 &&
+	test_commit A2 &&
+	git reset --hard $base &&
+	test_commit B1 &&
+	test_tick &&
+	git merge -m "Merge branch A" A2 &&
+
+	git checkout -b upstream-with-a2 $base &&
+	test_tick &&
+	git cherry-pick A2 &&
+
+	git checkout already-upstream &&
+	test_tick &&
+	git rebase -i --recreate-merges upstream-with-a2 &&
+	test_cmp_graph upstream-with-a2.. <<-\EOF
+	*   Merge branch A
+	|\
+	| * A1
+	* | B1
+	|/
+	o A2
+	EOF
+'
+
+test_done
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 09/12] sequencer: make refs generated by the `label` command worktree-local
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
                         ` (7 preceding siblings ...)
  2018-02-26 21:29       ` [PATCH v5 08/12] rebase: introduce the --recreate-merges option Johannes Schindelin
@ 2018-02-26 21:29       ` Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 10/12] sequencer: handle post-rewrite for merge commands Johannes Schindelin
                         ` (4 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 21:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

This allows for rebases to be run in parallel in separate worktrees
(think: interrupted in the middle of one rebase, being asked to perform
a different rebase, adding a separate worktree just for that job).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 refs.c                            |  3 ++-
 t/t3430-rebase-recreate-merges.sh | 14 ++++++++++++++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 20ba82b4343..e8b84c189ff 100644
--- a/refs.c
+++ b/refs.c
@@ -600,7 +600,8 @@ int dwim_log(const char *str, int len, struct object_id *oid, char **log)
 static int is_per_worktree_ref(const char *refname)
 {
 	return !strcmp(refname, "HEAD") ||
-		starts_with(refname, "refs/bisect/");
+		starts_with(refname, "refs/bisect/") ||
+		starts_with(refname, "refs/rewritten/");
 }
 
 static int is_pseudoref_syntax(const char *refname)
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index 0073601a206..1a3e43d66ff 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -143,4 +143,18 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'refs/rewritten/* is worktree-local' '
+	git worktree add wt &&
+	cat >wt/script-from-scratch <<-\EOF &&
+	label xyz
+	exec GIT_DIR=../.git git rev-parse --verify refs/rewritten/xyz >a || :
+	exec git rev-parse --verify refs/rewritten/xyz >b
+	EOF
+
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	git -C wt rebase -i HEAD &&
+	test_must_be_empty wt/a &&
+	test_cmp_rev HEAD "$(cat wt/b)"
+'
+
 test_done
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 10/12] sequencer: handle post-rewrite for merge commands
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
                         ` (8 preceding siblings ...)
  2018-02-26 21:29       ` [PATCH v5 09/12] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
@ 2018-02-26 21:29       ` Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 11/12] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
                         ` (3 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 21:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

In the previous patches, we implemented the basic functionality of the
`git rebase -i --recreate-merges` command, in particular the `merge`
command to create merge commits in the sequencer.

The interactive rebase is a lot more these days, though, than a simple
cherry-pick in a loop. For example, it calls the post-rewrite hook (if
any) after rebasing with a mapping of the old->new commits.

This patch implements the post-rewrite handling for the `merge` command
we just introduced. The other commands that were added recently (`label`
and `reset`) do not create new commits, therefore post-rewrite do not
need to handle them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c                       |  7 +++++--
 t/t3430-rebase-recreate-merges.sh | 25 +++++++++++++++++++++++++
 2 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 01bafe2fe47..85ce37cb99f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2980,11 +2980,14 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		else if (item->command == TODO_RESET)
 			res = do_reset(item->arg, item->arg_len, opts);
 		else if (item->command == TODO_MERGE ||
-			 item->command == TODO_MERGE_AND_EDIT)
+			 item->command == TODO_MERGE_AND_EDIT) {
 			res = do_merge(item->commit, item->arg, item->arg_len,
 				       item->command == TODO_MERGE_AND_EDIT ?
 				       EDIT_MSG | VERIFY_MSG : 0, opts);
-		else if (!is_noop(item->command))
+			if (item->commit)
+				record_in_rewritten(&item->commit->object.oid,
+						    peek_command(todo_list, 1));
+		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index 1a3e43d66ff..35a61ce90bb 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -157,4 +157,29 @@ test_expect_success 'refs/rewritten/* is worktree-local' '
 	test_cmp_rev HEAD "$(cat wt/b)"
 '
 
+test_expect_success 'post-rewrite hook and fixups work for merges' '
+	git checkout -b post-rewrite &&
+	test_commit same1 &&
+	git reset --hard HEAD^ &&
+	test_commit same2 &&
+	git merge -m "to fix up" same1 &&
+	echo same old same old >same2.t &&
+	test_tick &&
+	git commit --fixup HEAD same2.t &&
+	fixup="$(git rev-parse HEAD)" &&
+
+	mkdir -p .git/hooks &&
+	test_when_finished "rm .git/hooks/post-rewrite" &&
+	echo "cat >actual" | write_script .git/hooks/post-rewrite &&
+
+	test_tick &&
+	git rebase -i --autosquash --recreate-merges HEAD^^^ &&
+	printf "%s %s\n%s %s\n%s %s\n%s %s\n" >expect $(git rev-parse \
+		$fixup^^2 HEAD^2 \
+		$fixup^^ HEAD^ \
+		$fixup^ HEAD \
+		$fixup HEAD) &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 11/12] pull: accept --rebase=recreate to recreate the branch topology
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
                         ` (9 preceding siblings ...)
  2018-02-26 21:29       ` [PATCH v5 10/12] sequencer: handle post-rewrite for merge commands Johannes Schindelin
@ 2018-02-26 21:29       ` Johannes Schindelin
  2018-02-26 21:29       ` [PATCH v5 12/12] rebase -i: introduce --recreate-merges=[no-]rebase-cousins Johannes Schindelin
                         ` (2 subsequent siblings)
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 21:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Similar to the `preserve` mode simply passing the `--preserve-merges`
option to the `rebase` command, the `recreate` mode simply passes the
`--recreate-merges` option.

This will allow users to conveniently rebase non-trivial commit
topologies when pulling new commits, without flattening them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/config.txt               |  8 ++++++++
 Documentation/git-pull.txt             |  5 ++++-
 builtin/pull.c                         | 14 ++++++++++----
 builtin/remote.c                       |  2 ++
 contrib/completion/git-completion.bash |  2 +-
 5 files changed, 25 insertions(+), 6 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index f57e9cf10ca..8c9adea0d0c 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1058,6 +1058,10 @@ branch.<name>.rebase::
 	"git pull" is run. See "pull.rebase" for doing this in a non
 	branch-specific manner.
 +
+When recreate, also pass `--recreate-merges` along to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
@@ -2607,6 +2611,10 @@ pull.rebase::
 	pull" is run. See "branch.<name>.rebase" for setting this on a
 	per-branch basis.
 +
+When recreate, also pass `--recreate-merges` along to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
diff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt
index ce05b7a5b13..b4f9f057ea9 100644
--- a/Documentation/git-pull.txt
+++ b/Documentation/git-pull.txt
@@ -101,13 +101,16 @@ Options related to merging
 include::merge-options.txt[]
 
 -r::
---rebase[=false|true|preserve|interactive]::
+--rebase[=false|true|recreate|preserve|interactive]::
 	When true, rebase the current branch on top of the upstream
 	branch after fetching. If there is a remote-tracking branch
 	corresponding to the upstream branch and the upstream branch
 	was rebased since last fetched, the rebase uses that information
 	to avoid rebasing non-local changes.
 +
+When set to recreate, rebase with the `--recreate-merges` option passed
+to `git rebase` so that locally created merge commits will not be flattened.
++
 When set to preserve, rebase with the `--preserve-merges` option passed
 to `git rebase` so that locally created merge commits will not be flattened.
 +
diff --git a/builtin/pull.c b/builtin/pull.c
index 1876271af94..9da2cfa0bd3 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -27,14 +27,16 @@ enum rebase_type {
 	REBASE_FALSE = 0,
 	REBASE_TRUE,
 	REBASE_PRESERVE,
+	REBASE_RECREATE,
 	REBASE_INTERACTIVE
 };
 
 /**
  * Parses the value of --rebase. If value is a false value, returns
  * REBASE_FALSE. If value is a true value, returns REBASE_TRUE. If value is
- * "preserve", returns REBASE_PRESERVE. If value is a invalid value, dies with
- * a fatal error if fatal is true, otherwise returns REBASE_INVALID.
+ * "recreate", returns REBASE_RECREATE. If value is "preserve", returns
+ * REBASE_PRESERVE. If value is a invalid value, dies with a fatal error if
+ * fatal is true, otherwise returns REBASE_INVALID.
  */
 static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		int fatal)
@@ -47,6 +49,8 @@ static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		return REBASE_TRUE;
 	else if (!strcmp(value, "preserve"))
 		return REBASE_PRESERVE;
+	else if (!strcmp(value, "recreate"))
+		return REBASE_RECREATE;
 	else if (!strcmp(value, "interactive"))
 		return REBASE_INTERACTIVE;
 
@@ -130,7 +134,7 @@ static struct option pull_options[] = {
 	/* Options passed to git-merge or git-rebase */
 	OPT_GROUP(N_("Options related to merging")),
 	{ OPTION_CALLBACK, 'r', "rebase", &opt_rebase,
-	  "false|true|preserve|interactive",
+	  "false|true|recreate|preserve|interactive",
 	  N_("incorporate changes by rebasing rather than merging"),
 	  PARSE_OPT_OPTARG, parse_opt_rebase },
 	OPT_PASSTHRU('n', NULL, &opt_diffstat, NULL,
@@ -800,7 +804,9 @@ static int run_rebase(const struct object_id *curr_head,
 	argv_push_verbosity(&args);
 
 	/* Options passed to git-rebase */
-	if (opt_rebase == REBASE_PRESERVE)
+	if (opt_rebase == REBASE_RECREATE)
+		argv_array_push(&args, "--recreate-merges");
+	else if (opt_rebase == REBASE_PRESERVE)
 		argv_array_push(&args, "--preserve-merges");
 	else if (opt_rebase == REBASE_INTERACTIVE)
 		argv_array_push(&args, "--interactive");
diff --git a/builtin/remote.c b/builtin/remote.c
index d95bf904c3b..b7d0f7ce596 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -306,6 +306,8 @@ static int config_read_branches(const char *key, const char *value, void *cb)
 				info->rebase = v;
 			else if (!strcmp(value, "preserve"))
 				info->rebase = NORMAL_REBASE;
+			else if (!strcmp(value, "recreate"))
+				info->rebase = NORMAL_REBASE;
 			else if (!strcmp(value, "interactive"))
 				info->rebase = INTERACTIVE_REBASE;
 		}
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 38bba3835c6..3d44cb6890c 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2182,7 +2182,7 @@ _git_config ()
 		return
 		;;
 	branch.*.rebase)
-		__gitcomp "false true preserve interactive"
+		__gitcomp "false true recreate preserve interactive"
 		return
 		;;
 	remote.pushdefault)
-- 
2.16.1.windows.4



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 12/12] rebase -i: introduce --recreate-merges=[no-]rebase-cousins
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
                         ` (10 preceding siblings ...)
  2018-02-26 21:29       ` [PATCH v5 11/12] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
@ 2018-02-26 21:29       ` Johannes Schindelin
  2018-03-06  4:02       ` [PATCH v5 00/12] rebase -i: offer to recreate merge commits Igor Djordjevic
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
  13 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-02-26 21:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

This one is a bit tricky to explain, so let's try with a diagram:

        C
      /   \
A - B - E - F
  \   /
    D

To illustrate what this new mode is all about, let's consider what
happens upon `git rebase -i --recreate-merges B`, in particular to
the commit `D`. So far, the new branch structure would be:

       --- C' --
      /         \
A - B ------ E' - F'
      \    /
        D'

This is not really preserving the branch topology from before! The
reason is that the commit `D` does not have `B` as ancestor, and
therefore it gets rebased onto `B`.

This is unintuitive behavior. Even worse, when recreating branch
structure, most use cases would appear to want cousins *not* to be
rebased onto the new base commit. For example, Git for Windows (the
heaviest user of the Git garden shears, which served as the blueprint
for --recreate-merges) frequently merges branches from `next` early, and
these branches certainly do *not* want to be rebased. In the example
above, the desired outcome would look like this:

       --- C' --
      /         \
A - B ------ E' - F'
  \        /
   -- D' --

Let's introduce the term "cousins" for such commits ("D" in the
example), and let's not rebase them by default, introducing the new
"rebase-cousins" mode for use cases where they should be rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt      |  7 ++++++-
 builtin/rebase--helper.c          |  9 ++++++++-
 git-rebase--interactive.sh        |  1 +
 git-rebase.sh                     | 12 +++++++++++-
 sequencer.c                       |  4 ++++
 sequencer.h                       |  6 ++++++
 t/t3430-rebase-recreate-merges.sh | 23 +++++++++++++++++++++++
 7 files changed, 59 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 5e056c8ab6b..c5a77599c47 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -373,11 +373,16 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
---recreate-merges::
+--recreate-merges[=(rebase-cousins|no-rebase-cousins)]::
 	Recreate merge commits instead of flattening the history by replaying
 	merges. Merge conflict resolutions or manual amendments to merge
 	commits are not recreated automatically, but have to be recreated
 	manually.
++
+By default, or when `no-rebase-cousins` was specified, commits which do not
+have `<upstream>` as direct ancestor keep their original branch point.
+If the `rebase-cousins` mode is turned on, such commits are rebased onto
+`<upstream>` (or `<onto>`, if specified).
 
 -p::
 --preserve-merges::
diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index a5b07c43c96..5d1f12de57b 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
 	unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
-	int abbreviate_commands = 0;
+	int abbreviate_commands = 0, rebase_cousins = -1;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
 		CHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH,
@@ -25,6 +25,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
 			N_("allow commits with empty messages")),
 		OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge commits")),
+		OPT_BOOL(0, "rebase-cousins", &rebase_cousins,
+			 N_("keep original branch points of cousins")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -59,8 +61,13 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
 	flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
+	flags |= rebase_cousins > 0 ? TODO_LIST_REBASE_COUSINS : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
+	if (rebase_cousins >= 0 && !recreate_merges)
+		warning(_("--[no-]rebase-cousins has no effect without "
+			  "--recreate-merges"));
+
 	if (command == CONTINUE && argc == 1)
 		return !!sequencer_continue(&opts);
 	if (command == ABORT && argc == 1)
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index f5c8db2fdf8..679d79e0d17 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -907,6 +907,7 @@ if test t != "$preserve_merges"
 then
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
 		${recreate_merges:+--recreate-merges} \
+		${rebase_cousins:+--rebase-cousins} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 else
diff --git a/git-rebase.sh b/git-rebase.sh
index 528fa0073ac..9487e543bec 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,7 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
-recreate-merges!   try to recreate merges instead of skipping them
+recreate-merges?   try to recreate merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -89,6 +89,7 @@ state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
 recreate_merges=
+rebase_cousins=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -273,6 +274,15 @@ do
 		recreate_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
 		;;
+	--recreate-merges=*)
+		recreate_merges=t
+		case "${1#*=}" in
+		rebase-cousins) rebase_cousins=t;;
+		no-rebase-cousins) rebase_cousins=;;
+		*) die "Unknown mode: $1";;
+		esac
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/sequencer.c b/sequencer.c
index 85ce37cb99f..b2bf63029d4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3459,6 +3459,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 				   unsigned flags)
 {
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	int rebase_cousins = flags & TODO_LIST_REBASE_COUSINS;
 	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
 	struct strbuf label = STRBUF_INIT;
 	struct commit_list *commits = NULL, **tail = &commits, *iter;
@@ -3634,6 +3635,9 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 					   &commit->object.oid);
 			if (entry)
 				to = entry->string;
+			else if (!rebase_cousins)
+				to = label_oid(&commit->object.oid, NULL,
+					       &state);
 
 			if (!to || !strcmp(to, "onto"))
 				fprintf(out, "%s onto\n", cmd_reset);
diff --git a/sequencer.h b/sequencer.h
index 7c7c67d623c..739dd0fa92b 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -60,6 +60,12 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
 #define TODO_LIST_RECREATE_MERGES (1U << 3)
+/*
+ * When recreating merges, commits that do have the base commit as ancestor
+ * ("cousins") are *not* rebased onto the new base by default. If those
+ * commits should be rebased onto the new base, this flag needs to be passed.
+ */
+#define TODO_LIST_REBASE_COUSINS (1U << 4)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-recreate-merges.sh
index 35a61ce90bb..9a59f12b670 100755
--- a/t/t3430-rebase-recreate-merges.sh
+++ b/t/t3430-rebase-recreate-merges.sh
@@ -143,6 +143,29 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'do not rebase cousins unless asked for' '
+	write_script copy-editor.sh <<-\EOF &&
+	cp "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	EOF
+
+	test_config sequence.editor \""$PWD"/copy-editor.sh\" &&
+	git checkout -b cousins master &&
+	before="$(git rev-parse --verify HEAD)" &&
+	test_tick &&
+	git rebase -i --recreate-merges HEAD^ &&
+	test_cmp_rev HEAD $before &&
+	test_tick &&
+	git rebase -i --recreate-merges=rebase-cousins HEAD^ &&
+	test_cmp_graph HEAD^.. <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	|/
+	o H
+	EOF
+'
+
 test_expect_success 'refs/rewritten/* is worktree-local' '
 	git worktree add wt &&
 	cat >wt/script-from-scratch <<-\EOF &&
-- 
2.16.1.windows.4

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v5 01/12] sequencer: avoid using errno clobbered by rollback_lock_file()
  2018-02-26 21:29       ` [PATCH v5 01/12] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
@ 2018-02-27 21:33         ` Martin Ågren
  2018-03-02 20:33           ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Martin Ågren @ 2018-02-27 21:33 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Git Mailing List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood

On 26 February 2018 at 22:29, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> As pointed out in a review of the `--recreate-merges` patch series,
> `rollback_lock_file()` clobbers errno. Therefore, we have to report the
> error message that uses errno before calling said function.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index e9baaf59bd9..5aa3dc3c95c 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -345,12 +345,14 @@ static int write_message(const void *buf, size_t len, const char *filename,
>         if (msg_fd < 0)
>                 return error_errno(_("could not lock '%s'"), filename);
>         if (write_in_full(msg_fd, buf, len) < 0) {
> +               error_errno(_("could not write to '%s'"), filename);
>                 rollback_lock_file(&msg_file);
> -               return error_errno(_("could not write to '%s'"), filename);
> +               return -1;
>         }
>         if (append_eol && write(msg_fd, "\n", 1) < 0) {
> +               error_errno(_("could not write eol to '%s'"), filename);
>                 rollback_lock_file(&msg_file);
> -               return error_errno(_("could not write eol to '%s'"), filename);
> +               return -1;
>         }
>         if (commit_lock_file(&msg_file) < 0) {
>                 rollback_lock_file(&msg_file);
> @@ -2106,16 +2108,17 @@ static int save_head(const char *head)
>
>         fd = hold_lock_file_for_update(&head_lock, git_path_head_file(), 0);
>         if (fd < 0) {
> +               error_errno(_("could not lock HEAD"));
>                 rollback_lock_file(&head_lock);
> -               return error_errno(_("could not lock HEAD"));
> +               return -1;
>         }

I just noticed this when test-merging my series of lockfile-fixes to pu.
This `rollback_lock_file()` is not needed, since failure to take the
lock leaves it unlocked. If one wants to roll back the lock "for
clarity" or "just to be safe", then the same should arguably be done in
`write_message()`, just barely visible at the top of this diff.

Perhaps not worth a reroll. The conflict resolution between this and my
patch would be to take my hunk.

https://public-inbox.org/git/cover.1519763396.git.martin.agren@gmail.com/T/#t

Martin

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v5 01/12] sequencer: avoid using errno clobbered by rollback_lock_file()
  2018-02-27 21:33         ` Martin Ågren
@ 2018-03-02 20:33           ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-03-02 20:33 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Git Mailing List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood

[-- Attachment #1: Type: text/plain, Size: 2633 bytes --]

Hi Martin,

On Tue, 27 Feb 2018, Martin Ågren wrote:

> On 26 February 2018 at 22:29, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > As pointed out in a review of the `--recreate-merges` patch series,
> > `rollback_lock_file()` clobbers errno. Therefore, we have to report the
> > error message that uses errno before calling said function.
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  sequencer.c | 13 ++++++++-----
> >  1 file changed, 8 insertions(+), 5 deletions(-)
> >
> > diff --git a/sequencer.c b/sequencer.c
> > index e9baaf59bd9..5aa3dc3c95c 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -345,12 +345,14 @@ static int write_message(const void *buf, size_t len, const char *filename,
> >         if (msg_fd < 0)
> >                 return error_errno(_("could not lock '%s'"), filename);
> >         if (write_in_full(msg_fd, buf, len) < 0) {
> > +               error_errno(_("could not write to '%s'"), filename);
> >                 rollback_lock_file(&msg_file);
> > -               return error_errno(_("could not write to '%s'"), filename);
> > +               return -1;
> >         }
> >         if (append_eol && write(msg_fd, "\n", 1) < 0) {
> > +               error_errno(_("could not write eol to '%s'"), filename);
> >                 rollback_lock_file(&msg_file);
> > -               return error_errno(_("could not write eol to '%s'"), filename);
> > +               return -1;
> >         }
> >         if (commit_lock_file(&msg_file) < 0) {
> >                 rollback_lock_file(&msg_file);
> > @@ -2106,16 +2108,17 @@ static int save_head(const char *head)
> >
> >         fd = hold_lock_file_for_update(&head_lock, git_path_head_file(), 0);
> >         if (fd < 0) {
> > +               error_errno(_("could not lock HEAD"));
> >                 rollback_lock_file(&head_lock);
> > -               return error_errno(_("could not lock HEAD"));
> > +               return -1;
> >         }
> 
> I just noticed this when test-merging my series of lockfile-fixes to pu.
> This `rollback_lock_file()` is not needed, since failure to take the
> lock leaves it unlocked. If one wants to roll back the lock "for
> clarity" or "just to be safe", then the same should arguably be done in
> `write_message()`, just barely visible at the top of this diff.
> 
> Perhaps not worth a reroll. The conflict resolution between this and my
> patch would be to take my hunk.
> 
> https://public-inbox.org/git/cover.1519763396.git.martin.agren@gmail.com/T/#t

Thank you for working on this!
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v5 00/12] rebase -i: offer to recreate merge commits
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
                         ` (11 preceding siblings ...)
  2018-02-26 21:29       ` [PATCH v5 12/12] rebase -i: introduce --recreate-merges=[no-]rebase-cousins Johannes Schindelin
@ 2018-03-06  4:02       ` Igor Djordjevic
  2018-03-07 13:50         ` Johannes Schindelin
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
  13 siblings, 1 reply; 412+ messages in thread
From: Igor Djordjevic @ 2018-03-06  4:02 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Hi Johannes,

On 26/02/2018 22:29, Johannes Schindelin wrote:
> 
> Once upon a time, I dreamt of an interactive rebase that would not
> flatten branch structure, but instead recreate the commit topology
> faithfully.
> 
> My original attempt was --preserve-merges, but that design was so
> limited that I did not even enable it in interactive mode.
> 
> Subsequently, it *was* enabled in interactive mode, with the predictable
> consequences: as the --preserve-merges design does not allow for
> specifying the parents of merge commits explicitly, all the new commits'
> parents are defined *implicitly* by the previous commit history, and
> hence it is *not possible to even reorder commits*.
> 
> This design flaw cannot be fixed. Not without a complete re-design, at
> least. This patch series offers such a re-design.
> 
> Think of --recreate-merges as "--preserve-merges done right".

First of all, thanks for this wonderful improvement to existing `git 
rebase` functionality, I`m really excited to have this in the mainline! :)

But in the light of "--preserve-merges done right", I would like to 
hear your opinion on a topic that might be considered more or less 
important, and thus tackled in a few different ways... :$

Rebasing amended merges :( Even though documentation is quite clear 
about merge conflicts and manual amendments not recreated 
automatically, this might be considered quite an issue (a bug, even), 
as even in case of non-interactive rebase, amended content will be 
dropped - and even worse, it all happens silently, without alerting 
the user (for whom we presume to know what he`s doing, I guess).

Now, might be this is considered the least interesting use case, in 
comparison to all the power of more serious interactive rebases, but 
I would argue it could be the one most used by less adventurous users 
that would simply like to stay up-to-date with upstream, rebasing their 
current work on top of it (think `git pull --rebase=recreate`, even).

As it currently is, and that was the case with `--preserve-merges`, 
too, this will cause them to silently lose their work (amended merge 
content). And while documentation is clear about it, these might be 
less knowledgeable users, too, and thus potentially be the ones we 
should (try to) protect even more, if possible.

Now, in the light of that other, ongoing "merge rebasing" topic[1], 
it seems we really might be able to do much better, actually 
_rebasing_ merges (and keeping manual conflict resolutions/amendments), 
instead of _recreating_ them (and silently loosing content), and doing 
so reliably (or stopping for possible user inspection, but not silently 
doing the wrong thing, even if documented).

This concerns non-interactive rebase the most, but I have ideas on 
making it aligned with interactive one, too, where user could 
actually decide whether to rebase or (re)create the merge (rebase 
becoming the default, intuitively aligned with non-interactive rebase).

But before elaborating, I would like to hear your opinion on whether 
you find it worth to pursue that goal here, before `--recreate-merges` 
hits the mainstream, or it might be just fine as a possible later
improvement, too (if accepted, that is).

My main concern, and why I raised the question inside this topic in 
the first place, is default behavior. With `--recreate-merges` just 
being introduced, we have no backwards compatibility to think about, 
being a unique chance to make default behave as needed (not to say 
"correct", even), and might be really ticking one more of 
"--preserve-merges done right" boxes, and could be a pretty important 
one, too.

But once this becomes widely available, I guess it will be hard to 
improve (fix?) this merge rebasing silent content losing behavior 
(even if we would acknowledge it as a bug), without introducing 
additional options - and without a possibility to make possibly 
"right" behavior a default one, thus further complicating user 
experience.

So, I wanted to hear your stance on this :(

Knowing how much this means to you, it is really not my wish to drag 
this topic further, and if you find it that we`re good here as it is, 
I wouldn`t have any objections - I guess later new `--rebase-merges` 
option is a possibility, too, might be a wrapper around 
`--recreate-merges`, but with actual merge rebasing being a default 
(where merge recreation would still be possible, too)...

Otherwise, if you have any interest in it now, I can further elaborate 
what I`m thinking about, where it might help improve both user 
experience and rebase possibilities, for what might not be too much 
extra work... hopefully :P

Whatever ends up being your response, I`m really grateful for your 
work on this matter so far, and thank you for everything you did.

p.s. lol, now that I said it, and after writing all this, I might 
actually even like the idea of (later) having `--rebase-merges` 
alongside `--recreate-merges`, too, each one clearly communicating 
its default mode of operation - rebase merges vs. recreate merges... 
as one might rightfully expect ;) Eh :P

Regards, Buga

[1] https://public-inbox.org/git/87y3jtqdyg.fsf@javad.com/

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v5 00/12] rebase -i: offer to recreate merge commits
  2018-03-06  4:02       ` [PATCH v5 00/12] rebase -i: offer to recreate merge commits Igor Djordjevic
@ 2018-03-07 13:50         ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-03-07 13:50 UTC (permalink / raw)
  To: Igor Djordjevic
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood

Hi Buga,

On Tue, 6 Mar 2018, Igor Djordjevic wrote:

> [...]
>
> But before elaborating, I would like to hear your opinion on whether you
> find it worth to pursue that goal here, before `--recreate-merges` hits
> the mainstream, or it might be just fine as a possible later
> improvement, too (if accepted, that is).

As I suggested in another sub-thread, I think the best way forward is to
use your idea to make the 'rebase original merge commits' strategy
explicit.

That would not actually hold up the current --recreate-merges patch
series, but would mean to provide an add-on patch series to add support
for `merge -R` and then use that from the generated todo list.

For implementation detail reasons, it may actually make sense to integrate
those patches into the --recreate-merges patch series, though. Should not
be hard (except during GitMerge).

> p.s. lol, now that I said it, and after writing all this, I might 
> actually even like the idea of (later) having `--rebase-merges` 
> alongside `--recreate-merges`, too, each one clearly communicating 
> its default mode of operation - rebase merges vs. recreate merges... 
> as one might rightfully expect ;) Eh :P

Hehe...

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
                         ` (12 preceding siblings ...)
  2018-03-06  4:02       ` [PATCH v5 00/12] rebase -i: offer to recreate merge commits Igor Djordjevic
@ 2018-04-10 12:29       ` Johannes Schindelin
  2018-04-10 12:29         ` [PATCH v6 01/15] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
                           ` (16 more replies)
  13 siblings, 17 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

Once upon a time, I dreamt of an interactive rebase that would not
flatten branch structure, but instead recreate the commit topology
faithfully.

My original attempt was --preserve-merges, but that design was so
limited that I did not even enable it in interactive mode.

Subsequently, it *was* enabled in interactive mode, with the predictable
consequences: as the --preserve-merges design does not allow for
specifying the parents of merge commits explicitly, all the new commits'
parents are defined *implicitly* by the previous commit history, and
hence it is *not possible to even reorder commits*.

This design flaw cannot be fixed. Not without a complete re-design, at
least. This patch series offers such a re-design.

Think of --rebase-merges as "--preserve-merges done right". It
introduces new verbs for the todo list, `label`, `reset` and `merge`.
For a commit topology like this:

            A - B - C
              \   /
                D

the generated todo list would look like this:

            # branch D
            pick 0123 A
            label branch-point
            pick 1234 D
            label D

            reset branch-point
            pick 2345 B
            merge -C 3456 D # C

There are more patches in the pipeline, based on this patch series, but
left for later in the interest of reviewable patch series: one mini
series to use the sequencer even for `git rebase -i --root`, and another
one to add support for octopus merges to --rebase-merges. And then one
to allow for rebasing merge commits in a smarter way (this one will need
a bit more work, though, as it can result in very complicated, nested
merge conflicts *very* easily).

Changes since v5 (sorry, this one is big, and so is the interdiff):

- rebased to `master`, resolving conflicts with `ws/rebase-p` and
  `pw/rebase-keep-empty-fixes` (these changes are not reflected in the
  interdiff because I still did not find a good way to represent such
  fixups).

- just like `git merge` refuses to merge ancestors of HEAD, so does now
  the todo command `merge`.

- `git remote -v`'s output now differs when pulling with --rebase-merges
  vs pulling with --interactive.

- the `merge` command now also gives rerere a chance (just like `pick`
  already does).

- simplified test for rebase-cousins (no need to run --rebase-merges
  interactively, so there is no need to override the editor either).

- fixed `safe_append()` to roll back the lock file even when *reading*
  failed.

- used `reflog_message()` in `do_reset()` rather than duplicating the
  logic.

- reworded misleading commit message talking about fast-forwarding merge
  commits, when we just fast-forward `merge` commands *to* those merge commits
  whenever possible.

- removed duplicate `if (can_fast_forward)` clause.

- stopped promising support for octopus merges in --make-script in this patch
  series (it will be added in a later patch series).

- fixed grammar error in the message of the commit adding support for
  post-rewrite hooks to handle commits processed via the `merge` command.

- folded TODO_MERGE_AND_EDIT into TODO_MERGE by using a new `flags`
  field.

- the code of `do_merge()` has been made more obvious by using a variable
  `oneline_offset` instead of the non-descriptive `p`.

- renamed the option to `rebase-merges`, in preparation for doing it
  smarter using Phillip Wood's strategy (this will be contributed in a
  follow-up patch series after two others that add support for octopus
  merges and for handling --root via the sequencer).

- included Phillip Wood's test for --keep-empty with the new mode (and folded
  in a fix into the code of the `merge` command).

- added -r as shortcut for --rebase-merges

- added an entire section about "REBASING MERGES" to git-rebase's man page.


Johannes Schindelin (13):
  sequencer: avoid using errno clobbered by rollback_lock_file()
  sequencer: make rearrange_squash() a bit more obvious
  sequencer: introduce new commands to reset the revision
  sequencer: introduce the `merge` command
  sequencer: fast-forward `merge` commands, if possible
  rebase-helper --make-script: introduce a flag to rebase merges
  rebase: introduce the --rebase-merges option
  sequencer: make refs generated by the `label` command worktree-local
  sequencer: handle post-rewrite for merge commands
  rebase --rebase-merges: avoid "empty merges"
  pull: accept --rebase=merges to recreate the branch topology
  rebase -i: introduce --rebase-merges=[no-]rebase-cousins
  rebase -i --rebase-merges: add a section to the man page

Phillip Wood (1):
  rebase --rebase-merges: add test for --keep-empty

Stefan Beller (1):
  git-rebase--interactive: clarify arguments

 Documentation/config.txt               |   8 +
 Documentation/git-pull.txt             |   5 +-
 Documentation/git-rebase.txt           | 140 ++++-
 builtin/pull.c                         |  14 +-
 builtin/rebase--helper.c               |  13 +-
 builtin/remote.c                       |  18 +-
 contrib/completion/git-completion.bash |   4 +-
 git-rebase--interactive.sh             |  22 +-
 git-rebase.sh                          |  16 +
 refs.c                                 |   3 +-
 sequencer.c                            | 775 ++++++++++++++++++++++++-
 sequencer.h                            |   7 +
 t/t3421-rebase-topology-linear.sh      |   1 +
 t/t3430-rebase-merges.sh               | 211 +++++++
 14 files changed, 1203 insertions(+), 34 deletions(-)
 create mode 100755 t/t3430-rebase-merges.sh


base-commit: 0b0cc9f86731f894cff8dd25299a9b38c254569e
Published-As: https://github.com/dscho/git/releases/tag/recreate-merges-v6
Fetch-It-Via: git fetch https://github.com/dscho/git recreate-merges-v6

Interdiff vs v5:
 diff --git a/Documentation/config.txt b/Documentation/config.txt
 index 85dc3a0c429..45916ea8104 100644
 --- a/Documentation/config.txt
 +++ b/Documentation/config.txt
 @@ -1058,7 +1058,7 @@ branch.<name>.rebase::
  	"git pull" is run. See "pull.rebase" for doing this in a non
  	branch-specific manner.
  +
 -When recreate, also pass `--recreate-merges` along to 'git rebase'
 +When `merges`, pass the `--rebase-merges` option to 'git rebase'
  so that locally committed merge commits will not be flattened
  by running 'git pull'.
  +
 @@ -2620,7 +2620,7 @@ pull.rebase::
  	pull" is run. See "branch.<name>.rebase" for setting this on a
  	per-branch basis.
  +
 -When recreate, also pass `--recreate-merges` along to 'git rebase'
 +When `merges`, pass the `--rebase-merges` option to 'git rebase'
  so that locally committed merge commits will not be flattened
  by running 'git pull'.
  +
 diff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt
 index b4f9f057ea9..6f76d815dd3 100644
 --- a/Documentation/git-pull.txt
 +++ b/Documentation/git-pull.txt
 @@ -101,15 +101,15 @@ Options related to merging
  include::merge-options.txt[]
  
  -r::
 ---rebase[=false|true|recreate|preserve|interactive]::
 +--rebase[=false|true|merges|preserve|interactive]::
  	When true, rebase the current branch on top of the upstream
  	branch after fetching. If there is a remote-tracking branch
  	corresponding to the upstream branch and the upstream branch
  	was rebased since last fetched, the rebase uses that information
  	to avoid rebasing non-local changes.
  +
 -When set to recreate, rebase with the `--recreate-merges` option passed
 -to `git rebase` so that locally created merge commits will not be flattened.
 +When set to `merges`, rebase using `git rebase --rebase-merges` so that
 +locally created merge commits will not be flattened.
  +
  When set to preserve, rebase with the `--preserve-merges` option passed
  to `git rebase` so that locally created merge commits will not be flattened.
 diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
 index 2b85416f969..be946de2efb 100644
 --- a/Documentation/git-rebase.txt
 +++ b/Documentation/git-rebase.txt
 @@ -378,16 +378,19 @@ The commit list format can be changed by setting the configuration option
  rebase.instructionFormat.  A customized instruction format will automatically
  have the long commit hash prepended to the format.
  
 ---recreate-merges[=(rebase-cousins|no-rebase-cousins)]::
 -	Recreate merge commits instead of flattening the history by replaying
 +-r::
 +--rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
 +	Rebase merge commits instead of flattening the history by replaying
  	merges. Merge conflict resolutions or manual amendments to merge
 -	commits are not recreated automatically, but have to be recreated
 +	commits are not rebased automatically, but have to be applied
  	manually.
  +
  By default, or when `no-rebase-cousins` was specified, commits which do not
 -have `<upstream>` as direct ancestor keep their original branch point.
 +have `<upstream>` as direct ancestor will keep their original branch point.
  If the `rebase-cousins` mode is turned on, such commits are rebased onto
  `<upstream>` (or `<onto>`, if specified).
 ++
 +See also REBASING MERGES below.
  
  -p::
  --preserve-merges::
 @@ -786,13 +789,136 @@ The ripple effect of a "hard case" recovery is especially bad:
  'everyone' downstream from 'topic' will now have to perform a "hard
  case" recovery too!
  
 +REBASING MERGES
 +-----------------
 +
 +The interactive rebase command was originally designed to handle
 +individual patch series. As such, it makes sense to exclude merge
 +commits from the todo list, as the developer may have merged the
 +current `master` while working on the branch, only to eventually
 +rebase all the commits onto `master` (skipping the merge commits).
 +
 +However, there are legitimate reasons why a developer may want to
 +recreate merge commits: to keep the branch structure (or "commit
 +topology") when working on multiple, inter-related branches.
 +
 +In the following example, the developer works on a topic branch that
 +refactors the way buttons are defined, and on another topic branch
 +that uses that refactoring to implement a "Report a bug" button. The
 +output of `git log --graph --format=%s -5` may look like this:
 +
 +------------
 +*   Merge branch 'report-a-bug'
 +|\
 +| * Add the feedback button
 +* | Merge branch 'refactor-button'
 +|\ \
 +| |/
 +| * Use the Button class for all buttons
 +| * Extract a generic Button class from the DownloadButton one
 +------------
 +
 +The developer might want to rebase those commits to a newer `master`
 +while keeping the branch topology, for example when the first topic
 +branch is expected to be integrated into `master` much earlier than the
 +second one, say, to resolve merge conflicts with changes to the
 +DownloadButton class that made it into `master`.
 +
 +This rebase can be performed using the `--rebase-merges` option.
 +It will generate a todo list looking like this:
 +
 +------------
 +label onto
 +
 +# Branch: refactor-button
 +reset onto
 +pick 123456 Extract a generic Button class from the DownloadButton one
 +pick 654321 Use the Button class for all buttons
 +label refactor-button
 +
 +# Branch: report-a-bug
 +reset refactor-button # Use the Button class for all buttons
 +pick abcdef Add the feedback button
 +label report-a-bug
 +
 +reset onto
 +merge -C a1b2c3 refactor-button # Merge 'refactor-button'
 +merge -C 6f5e4d report-a-bug # Merge 'report-a-bug'
 +------------
 +
 +In contrast to a regular interactive rebase, there are `label`, `reset` and
 +`merge` commands in addition to `pick` ones.
 +
 +The `label` command puts a label to whatever will be the current
 +revision when that command is executed. Internally, these labels are
 +worktree-local refs that will be deleted when the rebase finishes or
 +when it is aborted. That way, rebase operations in multiple worktrees
 +linked to the same repository do not interfere with one another.
 +
 +The `reset` command is essentially a `git reset --hard` to the specified
 +revision (typically a previously-labeled one).
 +
 +The `merge` command will merge the specified revision into whatever is
 +HEAD at that time. With `-C <original-commit>`, the commit message of
 +the specified merge commit will be used. When the `-C` is changed to
 +a lower-case `-c`, the message will be opened in an editor after a
 +successful merge so that the user can edit the message.
 +
 +At this time, the `merge` command will *always* use the `recursive`
 +merge strategy, with no way to choose a different one. To work around
 +this, an `exec` command can be used to call `git merge` explicitly,
 +using the fact that the labels are worktree-local refs (the ref
 +`refs/rewritten/onto` would correspond to the label `onto`).
 +
 +Note: the first command (`reset onto`) labels the revision onto which
 +the commits are rebased; The name `onto` is just a convention, as a nod
 +to the `--onto` option.
 +
 +It is also possible to introduce completely new merge commits from scratch
 +by adding a command of the form `merge <merge-head>`. This form will
 +generate a tentative commit message and always open an editor to let the
 +user edit it. This can be useful e.g. when a topic branch turns out to
 +address more than a single concern and wants to be split into two or
 +even more topic branches. Consider this todo list:
 +
 +------------
 +pick 192837 Switch from GNU Makefiles to CMake
 +pick 5a6c7e Document the switch to CMake
 +pick 918273 Fix detection of OpenSSL in CMake
 +pick afbecd http: add support for TLS v1.3
 +pick fdbaec Fix detection of cURL in CMake on Windows
 +------------
 +
 +The one commit in this list that is not related to CMake may very well
 +have been motivated by working on fixing all those bugs introduced by
 +switching to CMake, but it addresses a different concern. To split this
 +branch into two topic branches, the todo list could be edited like this:
 +
 +------------
 +label onto
 +
 +pick afbecd http: add support for TLS v1.3
 +label tlsv1.3
 +
 +reset onto
 +pick 192837 Switch from GNU Makefiles to CMake
 +pick 918273 Fix detection of OpenSSL in CMake
 +pick fdbaec Fix detection of cURL in CMake on Windows
 +pick 5a6c7e Document the switch to CMake
 +label cmake
 +
 +reset onto
 +merge tlsv1.3
 +merge cmake
 +------------
 +
  BUGS
  ----
  The todo list presented by `--preserve-merges --interactive` does not
  represent the topology of the revision graph.  Editing commits and
  rewording their commit messages should work fine, but attempts to
  reorder commits tend to produce counterintuitive results. Use
 ---recreate-merges for a more faithful representation.
 +--rebase-merges for a more faithful representation.
  
  For example, an attempt to rearrange
  ------------
 diff --git a/builtin/pull.c b/builtin/pull.c
 index 3d1cc60eed6..70b44146ce4 100644
 --- a/builtin/pull.c
 +++ b/builtin/pull.c
 @@ -27,14 +27,14 @@ enum rebase_type {
  	REBASE_FALSE = 0,
  	REBASE_TRUE,
  	REBASE_PRESERVE,
 -	REBASE_RECREATE,
 +	REBASE_MERGES,
  	REBASE_INTERACTIVE
  };
  
  /**
   * Parses the value of --rebase. If value is a false value, returns
   * REBASE_FALSE. If value is a true value, returns REBASE_TRUE. If value is
 - * "recreate", returns REBASE_RECREATE. If value is "preserve", returns
 + * "merges", returns REBASE_MERGES. If value is "preserve", returns
   * REBASE_PRESERVE. If value is a invalid value, dies with a fatal error if
   * fatal is true, otherwise returns REBASE_INVALID.
   */
 @@ -49,8 +49,8 @@ static enum rebase_type parse_config_rebase(const char *key, const char *value,
  		return REBASE_TRUE;
  	else if (!strcmp(value, "preserve"))
  		return REBASE_PRESERVE;
 -	else if (!strcmp(value, "recreate"))
 -		return REBASE_RECREATE;
 +	else if (!strcmp(value, "merges"))
 +		return REBASE_MERGES;
  	else if (!strcmp(value, "interactive"))
  		return REBASE_INTERACTIVE;
  
 @@ -134,7 +134,7 @@ static struct option pull_options[] = {
  	/* Options passed to git-merge or git-rebase */
  	OPT_GROUP(N_("Options related to merging")),
  	{ OPTION_CALLBACK, 'r', "rebase", &opt_rebase,
 -	  "false|true|recreate|preserve|interactive",
 +	  "false|true|merges|preserve|interactive",
  	  N_("incorporate changes by rebasing rather than merging"),
  	  PARSE_OPT_OPTARG, parse_opt_rebase },
  	OPT_PASSTHRU('n', NULL, &opt_diffstat, NULL,
 @@ -804,8 +804,8 @@ static int run_rebase(const struct object_id *curr_head,
  	argv_push_verbosity(&args);
  
  	/* Options passed to git-rebase */
 -	if (opt_rebase == REBASE_RECREATE)
 -		argv_array_push(&args, "--recreate-merges");
 +	if (opt_rebase == REBASE_MERGES)
 +		argv_array_push(&args, "--rebase-merges");
  	else if (opt_rebase == REBASE_PRESERVE)
  		argv_array_push(&args, "--preserve-merges");
  	else if (opt_rebase == REBASE_INTERACTIVE)
 diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
 index 5d1f12de57b..f7c2a5fdc81 100644
 --- a/builtin/rebase--helper.c
 +++ b/builtin/rebase--helper.c
 @@ -12,7 +12,7 @@ static const char * const builtin_rebase_helper_usage[] = {
  int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
  {
  	struct replay_opts opts = REPLAY_OPTS_INIT;
 -	unsigned flags = 0, keep_empty = 0, recreate_merges = 0;
 +	unsigned flags = 0, keep_empty = 0, rebase_merges = 0;
  	int abbreviate_commands = 0, rebase_cousins = -1;
  	enum {
  		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
 @@ -24,7 +24,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
  		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
  		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
  			N_("allow commits with empty messages")),
 -		OPT_BOOL(0, "recreate-merges", &recreate_merges, N_("recreate merge commits")),
 +		OPT_BOOL(0, "rebase-merges", &rebase_merges, N_("rebase merge commits")),
  		OPT_BOOL(0, "rebase-cousins", &rebase_cousins,
  			 N_("keep original branch points of cousins")),
  		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 @@ -60,13 +60,13 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
  
  	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
  	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
 -	flags |= recreate_merges ? TODO_LIST_RECREATE_MERGES : 0;
 +	flags |= rebase_merges ? TODO_LIST_REBASE_MERGES : 0;
  	flags |= rebase_cousins > 0 ? TODO_LIST_REBASE_COUSINS : 0;
  	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
  
 -	if (rebase_cousins >= 0 && !recreate_merges)
 +	if (rebase_cousins >= 0 && !rebase_merges)
  		warning(_("--[no-]rebase-cousins has no effect without "
 -			  "--recreate-merges"));
 +			  "--rebase-merges"));
  
  	if (command == CONTINUE && argc == 1)
  		return !!sequencer_continue(&opts);
 diff --git a/builtin/remote.c b/builtin/remote.c
 index 210890c8a8e..45c9219e07a 100644
 --- a/builtin/remote.c
 +++ b/builtin/remote.c
 @@ -245,7 +245,9 @@ static int add(int argc, const char **argv)
  struct branch_info {
  	char *remote_name;
  	struct string_list merge;
 -	enum { NO_REBASE, NORMAL_REBASE, INTERACTIVE_REBASE } rebase;
 +	enum {
 +		NO_REBASE, NORMAL_REBASE, INTERACTIVE_REBASE, REBASE_MERGES
 +	} rebase;
  };
  
  static struct string_list branch_list = STRING_LIST_INIT_NODUP;
 @@ -306,8 +308,8 @@ static int config_read_branches(const char *key, const char *value, void *cb)
  				info->rebase = v;
  			else if (!strcmp(value, "preserve"))
  				info->rebase = NORMAL_REBASE;
 -			else if (!strcmp(value, "recreate"))
 -				info->rebase = NORMAL_REBASE;
 +			else if (!strcmp(value, "merges"))
 +				info->rebase = REBASE_MERGES;
  			else if (!strcmp(value, "interactive"))
  				info->rebase = INTERACTIVE_REBASE;
  		}
 @@ -965,9 +967,15 @@ static int show_local_info_item(struct string_list_item *item, void *cb_data)
  
  	printf("    %-*s ", show_info->width, item->string);
  	if (branch_info->rebase) {
 -		printf_ln(branch_info->rebase == INTERACTIVE_REBASE
 -			  ? _("rebases interactively onto remote %s")
 -			  : _("rebases onto remote %s"), merge->items[0].string);
 +		const char *msg;
 +		if (branch_info->rebase == INTERACTIVE_REBASE)
 +			msg = _("rebases interactively onto remote %s");
 +		else if (branch_info->rebase == REBASE_MERGES)
 +			msg = _("rebases interactively (with merges) onto "
 +				"remote %s");
 +		else
 +			msg = _("rebases onto remote %s");
 +		printf_ln(msg, merge->items[0].string);
  		return 0;
  	} else if (show_info->any_rebase) {
  		printf_ln(_(" merges with remote %s"), merge->items[0].string);
 diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
 index 7d2e7062919..6af65155c59 100644
 --- a/contrib/completion/git-completion.bash
 +++ b/contrib/completion/git-completion.bash
 @@ -1949,7 +1949,7 @@ _git_rebase ()
  	--*)
  		__gitcomp "
  			--onto --merge --strategy --interactive
 -			--recreate-merges --preserve-merges --stat --no-stat
 +			--rebase-merges --preserve-merges --stat --no-stat
  			--committer-date-is-author-date --ignore-date
  			--ignore-whitespace --whitespace=
  			--autosquash --no-autosquash
 @@ -2120,7 +2120,7 @@ _git_config ()
  		return
  		;;
  	branch.*.rebase)
 -		__gitcomp "false true recreate preserve interactive"
 +		__gitcomp "false true merges preserve interactive"
  		return
  		;;
  	remote.pushdefault)
 diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
 index 4c21faaccb1..b4ad130e8b1 100644
 --- a/git-rebase--interactive.sh
 +++ b/git-rebase--interactive.sh
 @@ -970,7 +970,7 @@ git_rebase__interactive () {
  	init_revisions_and_shortrevisions
  
  	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
 -		${recreate_merges:+--recreate-merges} \
 +		${rebase_merges:+--rebase-merges} \
  		${rebase_cousins:+--rebase-cousins} \
  		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
  	die "$(gettext "Could not generate todo list")"
 diff --git a/git-rebase.sh b/git-rebase.sh
 index dd39dfb1112..157705d2a72 100755
 --- a/git-rebase.sh
 +++ b/git-rebase.sh
 @@ -17,7 +17,7 @@ q,quiet!           be quiet. implies --no-stat
  autostash          automatically stash/stash pop before and after
  fork-point         use 'merge-base --fork-point' to refine upstream
  onto=!             rebase onto given branch instead of upstream
 -recreate-merges?   try to recreate merges instead of skipping them
 +r,rebase-merges?   try to rebase merges instead of skipping them
  p,preserve-merges! try to recreate merges instead of ignoring them
  s,strategy=!       use the given merge strategy
  no-ff!             cherry-pick all commits, even if unchanged
 @@ -89,7 +89,7 @@ type=
  state_dir=
  # One of {'', continue, skip, abort}, as parsed from command line
  action=
 -recreate_merges=
 +rebase_merges=
  rebase_cousins=
  preserve_merges=
  autosquash=
 @@ -273,12 +273,12 @@ do
  	--allow-empty-message)
  		allow_empty_message=--allow-empty-message
  		;;
 -	--recreate-merges)
 -		recreate_merges=t
 +	--rebase-merges)
 +		rebase_merges=t
  		test -z "$interactive_rebase" && interactive_rebase=implied
  		;;
 -	--recreate-merges=*)
 -		recreate_merges=t
 +	--rebase-merges=*)
 +		rebase_merges=t
  		case "${1#*=}" in
  		rebase-cousins) rebase_cousins=t;;
  		no-rebase-cousins) rebase_cousins=;;
 diff --git a/sequencer.c b/sequencer.c
 index 0b6aaced9a5..809df1ce484 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -1308,7 +1308,6 @@ enum todo_command {
  	TODO_LABEL,
  	TODO_RESET,
  	TODO_MERGE,
 -	TODO_MERGE_AND_EDIT,
  	/* commands that do nothing but are counted for reporting progress */
  	TODO_NOOP,
  	TODO_DROP,
 @@ -1330,7 +1329,6 @@ static struct {
  	{ 'l', "label" },
  	{ 't', "reset" },
  	{ 'm', "merge" },
 -	{ 0, "merge" }, /* MERGE_AND_EDIT */
  	{ 0,   "noop" },
  	{ 'd', "drop" },
  	{ 0,   NULL }
 @@ -1758,9 +1756,14 @@ static int read_and_refresh_cache(struct replay_opts *opts)
  	return 0;
  }
  
 +enum todo_item_flags {
 +	TODO_EDIT_MERGE_MSG = 1
 +};
 +
  struct todo_item {
  	enum todo_command command;
  	struct commit *commit;
 +	unsigned int flags;
  	const char *arg;
  	int arg_len;
  	size_t offset_in_buf;
 @@ -1795,6 +1798,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
  	char *end_of_object_name;
  	int i, saved, status, padding;
  
 +	item->flags = 0;
 +
  	/* left-trim */
  	bol += strspn(bol, " \t");
  
 @@ -1849,9 +1854,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
  			bol += strspn(bol, " \t");
  		else if (skip_prefix(bol, "-c", &bol)) {
  			bol += strspn(bol, " \t");
 -			item->command = TODO_MERGE_AND_EDIT;
 +			item->flags |= TODO_EDIT_MERGE_MSG;
  		} else {
 -			item->command = TODO_MERGE_AND_EDIT;
 +			item->flags |= TODO_EDIT_MERGE_MSG;
  			item->commit = NULL;
  			item->arg = bol;
  			item->arg_len = (int)(eol - bol);
 @@ -2511,8 +2516,11 @@ static int safe_append(const char *filename, const char *fmt, ...)
  	if (fd < 0)
  		return -1;
  
 -	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT)
 -		return error_errno(_("could not read '%s'"), filename);
 +	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT) {
 +		error_errno(_("could not read '%s'"), filename);
 +		rollback_lock_file(&lock);
 +		return -1;
 +	}
  	strbuf_complete(&buf, '\n');
  	va_start(ap, fmt);
  	strbuf_vaddf(&buf, fmt, ap);
 @@ -2574,6 +2582,9 @@ static int do_label(const char *name, int len)
  	return ret;
  }
  
 +static const char *reflog_message(struct replay_opts *opts,
 +	const char *sub_action, const char *fmt, ...);
 +
  static int do_reset(const char *name, int len, struct replay_opts *opts)
  {
  	struct strbuf ref_name = STRBUF_INIT;
 @@ -2638,33 +2649,50 @@ static int do_reset(const char *name, int len, struct replay_opts *opts)
  		ret = error(_("could not write index"));
  	free((void *)desc.buffer);
  
 -	if (!ret) {
 -		struct strbuf msg = STRBUF_INIT;
 -
 -		strbuf_addf(&msg, "(rebase -i) reset '%.*s'", len, name);
 -		ret = update_ref(msg.buf, "HEAD", &oid, NULL, 0,
 -				 UPDATE_REFS_MSG_ON_ERR);
 -		strbuf_release(&msg);
 -	}
 +	if (!ret)
 +		ret = update_ref(reflog_message(opts, "reset", "'%.*s'",
 +						len, name), "HEAD", &oid,
 +				 NULL, 0, UPDATE_REFS_MSG_ON_ERR);
  
  	strbuf_release(&ref_name);
  	return ret;
  }
  
  static int do_merge(struct commit *commit, const char *arg, int arg_len,
 -		    int run_commit_flags, struct replay_opts *opts)
 +		    int flags, struct replay_opts *opts)
  {
 -	int merge_arg_len;
 +	int run_commit_flags = (flags & TODO_EDIT_MERGE_MSG) ?
 +		EDIT_MSG | VERIFY_MSG : 0;
  	struct strbuf ref_name = STRBUF_INIT;
  	struct commit *head_commit, *merge_commit, *i;
 -	struct commit_list *common, *j, *reversed = NULL;
 +	struct commit_list *bases, *j, *reversed = NULL;
  	struct merge_options o;
 -	int can_fast_forward, ret;
 +	int merge_arg_len, oneline_offset, can_fast_forward, ret;
  	static struct lock_file lock;
 +	const char *p;
  
 -	for (merge_arg_len = 0; merge_arg_len < arg_len; merge_arg_len++)
 -		if (isspace(arg[merge_arg_len]))
 -			break;
 +	oneline_offset = arg_len;
 +	merge_arg_len = strcspn(arg, " \t\n");
 +	p = arg + merge_arg_len;
 +	p += strspn(p, " \t\n");
 +	if (*p == '#' && (!p[1] || isspace(p[1]))) {
 +		p += 1 + strspn(p + 1, " \t\n");
 +		oneline_offset = p - arg;
 +	} else if (p - arg < arg_len)
 +		BUG("octopus merges are not supported yet: '%s'", p);
 +
 +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
 +	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
 +	if (!merge_commit) {
 +		/* fall back to non-rewritten ref or commit */
 +		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
 +		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
 +	}
 +	if (!merge_commit) {
 +		error(_("could not resolve '%s'"), ref_name.buf);
 +		strbuf_release(&ref_name);
 +		return -1;
 +	}
  
  	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
  		return -1;
 @@ -2697,7 +2725,6 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  		}
  		unuse_commit_buffer(commit, message);
  	} else {
 -		const char *p = arg + merge_arg_len;
  		struct strbuf buf = STRBUF_INIT;
  		int len;
  
 @@ -2705,12 +2732,10 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  		write_author_script(buf.buf);
  		strbuf_reset(&buf);
  
 -		p += strspn(p, " \t");
 -		if (*p == '#' && isspace(p[1]))
 -			p += 1 + strspn(p + 1, " \t");
 -		if (*p)
 -			len = strlen(p);
 -		else {
 +		if (oneline_offset < arg_len) {
 +			p = arg + oneline_offset;
 +			len = arg_len - oneline_offset;
 +		} else {
  			strbuf_addf(&buf, "Merge branch '%.*s'",
  				    merge_arg_len, arg);
  			p = buf.buf;
 @@ -2728,25 +2753,24 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  	}
  
  	/*
 -	 * If HEAD is not identical to the parent of the original merge commit,
 -	 * we cannot fast-forward.
 +	 * If HEAD is not identical to the first parent of the original merge
 +	 * commit, we cannot fast-forward.
  	 */
  	can_fast_forward = opts->allow_ff && commit && commit->parents &&
  		!oidcmp(&commit->parents->item->object.oid,
  			&head_commit->object.oid);
  
 -	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
 -	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
 -	if (!merge_commit) {
 -		/* fall back to non-rewritten ref or commit */
 -		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
 -		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
 -	}
 -	if (!merge_commit) {
 -		error(_("could not resolve '%s'"), ref_name.buf);
 -		strbuf_release(&ref_name);
 -		rollback_lock_file(&lock);
 -		return -1;
 +	/*
 +	 * If the merge head is different from the original one, we cannot
 +	 * fast-forward.
 +	 */
 +	if (can_fast_forward) {
 +		struct commit_list *second_parent = commit->parents->next;
 +
 +		if (second_parent && !second_parent->next &&
 +		    oidcmp(&merge_commit->object.oid,
 +			   &second_parent->item->object.oid))
 +			can_fast_forward = 0;
  	}
  
  	if (can_fast_forward && commit->parents->next &&
 @@ -2763,10 +2787,18 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  		      git_path_merge_head(), 0);
  	write_message("no-ff", 5, git_path_merge_mode(), 0);
  
 -	common = get_merge_bases(head_commit, merge_commit);
 -	for (j = common; j; j = j->next)
 +	bases = get_merge_bases(head_commit, merge_commit);
 +	if (bases && !oidcmp(&merge_commit->object.oid,
 +			     &bases->item->object.oid)) {
 +		strbuf_release(&ref_name);
 +		rollback_lock_file(&lock);
 +		/* skip merging an ancestor of HEAD */
 +		return 0;
 +	}
 +
 +	for (j = bases; j; j = j->next)
  		commit_list_insert(j->item, &reversed);
 -	free_commit_list(common);
 +	free_commit_list(bases);
  
  	read_cache();
  	init_merge_options(&o);
 @@ -2775,6 +2807,8 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  	o.buffer_output = 2;
  
  	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
 +	if (!ret)
 +		rerere(opts->allow_rerere_auto);
  	if (ret <= 0)
  		fputs(o.obuf.buf, stdout);
  	strbuf_release(&o.obuf);
 @@ -2986,11 +3020,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			res = do_label(item->arg, item->arg_len);
  		else if (item->command == TODO_RESET)
  			res = do_reset(item->arg, item->arg_len, opts);
 -		else if (item->command == TODO_MERGE ||
 -			 item->command == TODO_MERGE_AND_EDIT) {
 +		else if (item->command == TODO_MERGE) {
  			res = do_merge(item->commit, item->arg, item->arg_len,
 -				       item->command == TODO_MERGE_AND_EDIT ?
 -				       EDIT_MSG | VERIFY_MSG : 0, opts);
 +				       item->flags, opts);
  			if (item->commit)
  				record_in_rewritten(&item->commit->object.oid,
  						    peek_command(todo_list, 1));
 @@ -3404,7 +3436,7 @@ static const char *label_oid(struct object_id *oid, const char *label,
  		strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
  		label = p = state->buf.buf;
  
 -		find_unique_abbrev_r(p, oid->hash, default_abbrev);
 +		find_unique_abbrev_r(p, oid, default_abbrev);
  
  		/*
  		 * We may need to extend the abbreviated hash so that there is
 @@ -3508,11 +3540,13 @@ static int make_script_with_merges(struct pretty_print_context *pp,
  		int is_octopus;
  		const char *p1, *p2;
  		struct object_id *oid;
 +		int is_empty;
  
  		tail = &commit_list_insert(commit, tail)->next;
  		oidset_insert(&interesting, &commit->object.oid);
  
 -		if ((commit->object.flags & PATCHSAME))
 +		is_empty = is_original_commit_empty(commit);
 +		if (!is_empty && (commit->object.flags & PATCHSAME))
  			continue;
  
  		strbuf_reset(&oneline);
 @@ -3522,7 +3556,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
  		if (!to_merge) {
  			/* non-merge commit: easy case */
  			strbuf_reset(&buf);
 -			if (!keep_empty && is_original_commit_empty(commit))
 +			if (!keep_empty && is_empty)
  				strbuf_addf(&buf, "%c ", comment_line_char);
  			strbuf_addf(&buf, "%s %s %s", cmd_pick,
  				    oid_to_hex(&commit->object.oid),
 @@ -3698,11 +3732,11 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
  	struct commit *commit;
  	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
  	const char *insn = flags & TODO_LIST_ABBREVIATE_CMDS ? "p" : "pick";
 -	int recreate_merges = flags & TODO_LIST_RECREATE_MERGES;
 +	int rebase_merges = flags & TODO_LIST_REBASE_MERGES;
  
  	init_revisions(&revs, NULL);
  	revs.verbose_header = 1;
 -	if (recreate_merges)
 +	if (rebase_merges)
  		revs.cherry_mark = 1;
  	else {
  		revs.max_parents = 1;
 @@ -3731,7 +3765,7 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
  	if (prepare_revision_walk(&revs) < 0)
  		return error(_("make_script: error preparing revisions"));
  
 -	if (recreate_merges)
 +	if (rebase_merges)
  		return make_script_with_merges(&pp, &revs, out, flags);
  
  	while ((commit = get_revision(&revs))) {
 @@ -3823,10 +3857,12 @@ int transform_todos(unsigned flags)
  					  short_commit_name(item->commit) :
  					  oid_to_hex(&item->commit->object.oid);
  
 -			if (item->command == TODO_MERGE)
 -				strbuf_addstr(&buf, " -C");
 -			else if (item->command == TODO_MERGE_AND_EDIT)
 -				strbuf_addstr(&buf, " -c");
 +			if (item->command == TODO_MERGE) {
 +				if (item->flags & TODO_EDIT_MERGE_MSG)
 +					strbuf_addstr(&buf, " -c");
 +				else
 +					strbuf_addstr(&buf, " -C");
 +			}
  
  			strbuf_addf(&buf, " %s", oid);
  		}
 diff --git a/sequencer.h b/sequencer.h
 index 739dd0fa92b..d9570d92b11 100644
 --- a/sequencer.h
 +++ b/sequencer.h
 @@ -59,9 +59,9 @@ int sequencer_remove_state(struct replay_opts *opts);
  #define TODO_LIST_KEEP_EMPTY (1U << 0)
  #define TODO_LIST_SHORTEN_IDS (1U << 1)
  #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
 -#define TODO_LIST_RECREATE_MERGES (1U << 3)
 +#define TODO_LIST_REBASE_MERGES (1U << 3)
  /*
 - * When recreating merges, commits that do have the base commit as ancestor
 + * When rebasing merges, commits that do have the base commit as ancestor
   * ("cousins") are *not* rebased onto the new base by default. If those
   * commits should be rebased onto the new base, this flag needs to be passed.
   */
 diff --git a/t/t3421-rebase-topology-linear.sh b/t/t3421-rebase-topology-linear.sh
 index 68fe2003ef5..fbae5dab7e2 100755
 --- a/t/t3421-rebase-topology-linear.sh
 +++ b/t/t3421-rebase-topology-linear.sh
 @@ -217,6 +217,7 @@ test_run_rebase success ''
  test_run_rebase failure -m
  test_run_rebase failure -i
  test_run_rebase failure -p
 +test_run_rebase success --rebase-merges
  
  #       m
  #      /
 diff --git a/t/t3430-rebase-recreate-merges.sh b/t/t3430-rebase-merges.sh
 similarity index 87%
 rename from t/t3430-rebase-recreate-merges.sh
 rename to t/t3430-rebase-merges.sh
 index 9a59f12b670..ee006810573 100755
 --- a/t/t3430-rebase-recreate-merges.sh
 +++ b/t/t3430-rebase-merges.sh
 @@ -1,9 +1,9 @@
  #!/bin/sh
  #
 -# Copyright (c) 2017 Johannes E. Schindelin
 +# Copyright (c) 2018 Johannes E. Schindelin
  #
  
 -test_description='git rebase -i --recreate-merges
 +test_description='git rebase -i --rebase-merges
  
  This test runs git rebase "interactively", retaining the branch structure by
  recreating merge commits.
 @@ -19,6 +19,13 @@ Initial setup:
  . ./test-lib.sh
  . "$TEST_DIRECTORY"/lib-rebase.sh
  
 +test_cmp_graph () {
 +	cat >expect &&
 +	git log --graph --boundary --format=%s "$@" >output &&
 +	sed "s/ *$//" <output >output.trimmed &&
 +	test_cmp expect output.trimmed
 +}
 +
  test_expect_success 'setup' '
  	write_script replace-editor.sh <<-\EOF &&
  	mv "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
 @@ -63,17 +70,10 @@ merge -C H second
  merge onebranch # Merge the topic branch 'onebranch'
  EOF
  
 -test_cmp_graph () {
 -	cat >expect &&
 -	git log --graph --boundary --format=%s "$@" >output &&
 -	sed "s/ *$//" <output >output.trimmed &&
 -	test_cmp expect output.trimmed
 -}
 -
  test_expect_success 'create completely different structure' '
  	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
  	test_tick &&
 -	git rebase -i --recreate-merges A &&
 +	git rebase -i -r A &&
  	test_cmp_graph <<-\EOF
  	*   Merge the topic branch '\''onebranch'\''
  	|\
 @@ -132,7 +132,7 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
  
  	git checkout already-upstream &&
  	test_tick &&
 -	git rebase -i --recreate-merges upstream-with-a2 &&
 +	git rebase -i -r upstream-with-a2 &&
  	test_cmp_graph upstream-with-a2.. <<-\EOF
  	*   Merge branch A
  	|\
 @@ -144,18 +144,13 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
  '
  
  test_expect_success 'do not rebase cousins unless asked for' '
 -	write_script copy-editor.sh <<-\EOF &&
 -	cp "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
 -	EOF
 -
 -	test_config sequence.editor \""$PWD"/copy-editor.sh\" &&
  	git checkout -b cousins master &&
  	before="$(git rev-parse --verify HEAD)" &&
  	test_tick &&
 -	git rebase -i --recreate-merges HEAD^ &&
 +	git rebase -r HEAD^ &&
  	test_cmp_rev HEAD $before &&
  	test_tick &&
 -	git rebase -i --recreate-merges=rebase-cousins HEAD^ &&
 +	git rebase --rebase-merges=rebase-cousins HEAD^ &&
  	test_cmp_graph HEAD^.. <<-\EOF
  	*   Merge the topic branch '\''onebranch'\''
  	|\
 @@ -196,7 +191,7 @@ test_expect_success 'post-rewrite hook and fixups work for merges' '
  	echo "cat >actual" | write_script .git/hooks/post-rewrite &&
  
  	test_tick &&
 -	git rebase -i --autosquash --recreate-merges HEAD^^^ &&
 +	git rebase -i --autosquash -r HEAD^^^ &&
  	printf "%s %s\n%s %s\n%s %s\n%s %s\n" >expect $(git rev-parse \
  		$fixup^^2 HEAD^2 \
  		$fixup^^ HEAD^ \
 @@ -205,4 +200,12 @@ test_expect_success 'post-rewrite hook and fixups work for merges' '
  	test_cmp expect actual
  '
  
 +test_expect_success 'refuse to merge ancestors of HEAD' '
 +	echo "merge HEAD^" >script-from-scratch &&
 +	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
 +	before="$(git rev-parse HEAD)" &&
 +	git rebase -i HEAD &&
 +	test_cmp_rev HEAD $before
 +'
 +
  test_done
-- 
2.17.0.windows.1.4.g7e4058d72e3


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v6 01/15] sequencer: avoid using errno clobbered by rollback_lock_file()
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
@ 2018-04-10 12:29         ` Johannes Schindelin
  2018-04-10 12:29         ` [PATCH v6 02/15] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
                           ` (15 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

As pointed out in a review of the `--rebase-merges` patch series,
`rollback_lock_file()` clobbers errno. Therefore, we have to report the
error message that uses errno before calling said function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 667f35ebdff..096e6d241e0 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -345,12 +345,14 @@ static int write_message(const void *buf, size_t len, const char *filename,
 	if (msg_fd < 0)
 		return error_errno(_("could not lock '%s'"), filename);
 	if (write_in_full(msg_fd, buf, len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write to '%s'"), filename);
+		return -1;
 	}
 	if (append_eol && write(msg_fd, "\n", 1) < 0) {
+		error_errno(_("could not write eol to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write eol to '%s'"), filename);
+		return -1;
 	}
 	if (commit_lock_file(&msg_file) < 0)
 		return error(_("failed to finalize '%s'"), filename);
@@ -2119,9 +2121,9 @@ static int save_head(const char *head)
 	written = write_in_full(fd, buf.buf, buf.len);
 	strbuf_release(&buf);
 	if (written < 0) {
+		error_errno(_("could not write to '%s'"), git_path_head_file());
 		rollback_lock_file(&head_lock);
-		return error_errno(_("could not write to '%s'"),
-				   git_path_head_file());
+		return -1;
 	}
 	if (commit_lock_file(&head_lock) < 0)
 		return error(_("failed to finalize '%s'"), git_path_head_file());
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 02/15] sequencer: make rearrange_squash() a bit more obvious
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
  2018-04-10 12:29         ` [PATCH v6 01/15] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
@ 2018-04-10 12:29         ` Johannes Schindelin
  2018-04-10 12:29         ` [PATCH v6 03/15] git-rebase--interactive: clarify arguments Johannes Schindelin
                           ` (14 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

There are some commands that have to be skipped from rearranging by virtue
of not handling any commits.

However, the logic was not quite obvious: it skipped commands based on
their position in the enum todo_command.

Instead, let's make it explicit that we skip all commands that do not
handle any commit. With one exception: the `drop` command, because it,
well, drops the commit and is therefore not eligible to rearranging.

Note: this is a bit academic at the moment because the only time we call
`rearrange_squash()` is directly after generating the todo list, when we
have nothing but `pick` commands anyway.

However, the upcoming `merge` command *will* want to be handled by that
function, and it *can* handle commits.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 096e6d241e0..1ee70d843c1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3393,7 +3393,7 @@ int rearrange_squash(void)
 		struct subject2item_entry *entry;
 
 		next[i] = tail[i] = -1;
-		if (item->command >= TODO_EXEC) {
+		if (!item->commit || item->command == TODO_DROP) {
 			subjects[i] = NULL;
 			continue;
 		}
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 03/15] git-rebase--interactive: clarify arguments
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
  2018-04-10 12:29         ` [PATCH v6 01/15] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
  2018-04-10 12:29         ` [PATCH v6 02/15] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
@ 2018-04-10 12:29         ` Johannes Schindelin
  2018-04-10 12:29         ` [PATCH v6 04/15] sequencer: introduce new commands to reset the revision Johannes Schindelin
                           ` (13 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:29 UTC (permalink / raw)
  To: git
  Cc: Stefan Beller, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov

From: Stefan Beller <stefanbeller@gmail.com>

Up to now each command took a commit as its first argument and ignored
the rest of the line (usually the subject of the commit)

Now that we are about to introduce commands that take different
arguments, clarify each command by giving the argument list.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 50323fc2735..e1b865f43f2 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -155,13 +155,13 @@ reschedule_last_action () {
 append_todo_help () {
 	gettext "
 Commands:
-p, pick = use commit
-r, reword = use commit, but edit the commit message
-e, edit = use commit, but stop for amending
-s, squash = use commit, but meld into previous commit
-f, fixup = like \"squash\", but discard this commit's log message
-x, exec = run command (the rest of the line) using shell
-d, drop = remove commit
+p, pick <commit> = use commit
+r, reword <commit> = use commit, but edit the commit message
+e, edit <commit> = use commit, but stop for amending
+s, squash <commit> = use commit, but meld into previous commit
+f, fixup <commit> = like \"squash\", but discard this commit's log message
+x, exec <commit> = run command (the rest of the line) using shell
+d, drop <commit> = remove commit
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 04/15] sequencer: introduce new commands to reset the revision
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (2 preceding siblings ...)
  2018-04-10 12:29         ` [PATCH v6 03/15] git-rebase--interactive: clarify arguments Johannes Schindelin
@ 2018-04-10 12:29         ` Johannes Schindelin
  2018-04-11  6:17           ` Sergey Organov
  2018-04-13 10:03           ` Phillip Wood
  2018-04-10 12:29         ` [PATCH v6 05/15] sequencer: introduce the `merge` command Johannes Schindelin
                           ` (12 subsequent siblings)
  16 siblings, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

In the upcoming commits, we will teach the sequencer to rebase merges.
This will be done in a very different way from the unfortunate design of
`git rebase --preserve-merges` (which does not allow for reordering
commits, or changing the branch topology).

The main idea is to introduce new todo list commands, to support
labeling the current revision with a given name, resetting the current
revision to a previous state, and  merging labeled revisions.

This idea was developed in Git for Windows' Git garden shears (that are
used to maintain Git for Windows' "thicket of branches" on top of
upstream Git), and this patch is part of the effort to make it available
to a wider audience, as well as to make the entire process more robust
(by implementing it in a safe and portable language rather than a Unix
shell script).

This commit implements the commands to label, and to reset to, given
revisions. The syntax is:

	label <name>
	reset <name>

Internally, the `label <name>` command creates the ref
`refs/rewritten/<name>`. This makes it possible to work with the labeled
revisions interactively, or in a scripted fashion (e.g. via the todo
list command `exec`).

These temporary refs are removed upon sequencer_remove_state(), so that
even a `git rebase --abort` cleans them up.

We disallow '#' as label because that character will be used as separator
in the upcoming `merge` command.

Later in this patch series, we will mark the `refs/rewritten/` refs as
worktree-local, to allow for interactive rebases to be run in parallel in
worktrees linked to the same repository.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   2 +
 sequencer.c                | 198 +++++++++++++++++++++++++++++++++++--
 2 files changed, 194 insertions(+), 6 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index e1b865f43f2..e8d3a7d7588 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -162,6 +162,8 @@ s, squash <commit> = use commit, but meld into previous commit
 f, fixup <commit> = like \"squash\", but discard this commit's log message
 x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
+l, label <label> = label current HEAD with a name
+t, reset <label> = reset HEAD to a label
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index 1ee70d843c1..c63d47f5e09 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -23,6 +23,8 @@
 #include "hashmap.h"
 #include "notes-utils.h"
 #include "sigchain.h"
+#include "unpack-trees.h"
+#include "worktree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -120,6 +122,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
 static GIT_PATH_FUNC(rebase_path_rewritten_pending,
 	"rebase-merge/rewritten-pending")
+
+/*
+ * The path of the file listing refs that need to be deleted after the rebase
+ * finishes. This is used by the `label` command to record the need for cleanup.
+ */
+static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
+
 /*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
@@ -244,18 +253,33 @@ static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
 
 int sequencer_remove_state(struct replay_opts *opts)
 {
-	struct strbuf dir = STRBUF_INIT;
+	struct strbuf buf = STRBUF_INIT;
 	int i;
 
+	if (strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
+		char *p = buf.buf;
+		while (*p) {
+			char *eol = strchr(p, '\n');
+			if (eol)
+				*eol = '\0';
+			if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
+				warning(_("could not delete '%s'"), p);
+			if (!eol)
+				break;
+			p = eol + 1;
+		}
+	}
+
 	free(opts->gpg_sign);
 	free(opts->strategy);
 	for (i = 0; i < opts->xopts_nr; i++)
 		free(opts->xopts[i]);
 	free(opts->xopts);
 
-	strbuf_addstr(&dir, get_dir(opts));
-	remove_dir_recursively(&dir, 0);
-	strbuf_release(&dir);
+	strbuf_reset(&buf);
+	strbuf_addstr(&buf, get_dir(opts));
+	remove_dir_recursively(&buf, 0);
+	strbuf_release(&buf);
 
 	return 0;
 }
@@ -1279,6 +1303,8 @@ enum todo_command {
 	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
+	TODO_LABEL,
+	TODO_RESET,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -1297,6 +1323,8 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
+	{ 'l', "label" },
+	{ 't', "reset" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1802,7 +1830,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return error(_("missing arguments for %s"),
 			     command_to_string(item->command));
 
-	if (item->command == TODO_EXEC) {
+	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
+	    item->command == TODO_RESET) {
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = (int)(eol - bol);
@@ -2450,6 +2479,159 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int safe_append(const char *filename, const char *fmt, ...)
+{
+	va_list ap;
+	struct lock_file lock = LOCK_INIT;
+	int fd = hold_lock_file_for_update(&lock, filename,
+					   LOCK_REPORT_ON_ERROR);
+	struct strbuf buf = STRBUF_INIT;
+
+	if (fd < 0)
+		return -1;
+
+	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT) {
+		error_errno(_("could not read '%s'"), filename);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	strbuf_complete(&buf, '\n');
+	va_start(ap, fmt);
+	strbuf_vaddf(&buf, fmt, ap);
+	va_end(ap);
+
+	if (write_in_full(fd, buf.buf, buf.len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
+		strbuf_release(&buf);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	if (commit_lock_file(&lock) < 0) {
+		strbuf_release(&buf);
+		rollback_lock_file(&lock);
+		return error(_("failed to finalize '%s'"), filename);
+	}
+
+	strbuf_release(&buf);
+	return 0;
+}
+
+static int do_label(const char *name, int len)
+{
+	struct ref_store *refs = get_main_ref_store();
+	struct ref_transaction *transaction;
+	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
+	struct strbuf msg = STRBUF_INIT;
+	int ret = 0;
+	struct object_id head_oid;
+
+	if (len == 1 && *name == '#')
+		return error("Illegal label name: '%.*s'", len, name);
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
+
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction) {
+		error("%s", err.buf);
+		ret = -1;
+	} else if (get_oid("HEAD", &head_oid)) {
+		error(_("could not read HEAD"));
+		ret = -1;
+	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
+					  NULL, 0, msg.buf, &err) < 0 ||
+		   ref_transaction_commit(transaction, &err)) {
+		error("%s", err.buf);
+		ret = -1;
+	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
+	strbuf_release(&msg);
+
+	if (!ret)
+		ret = safe_append(rebase_path_refs_to_delete(),
+				  "%s\n", ref_name.buf);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
+static const char *reflog_message(struct replay_opts *opts,
+	const char *sub_action, const char *fmt, ...);
+
+static int do_reset(const char *name, int len, struct replay_opts *opts)
+{
+	struct strbuf ref_name = STRBUF_INIT;
+	struct object_id oid;
+	struct lock_file lock = LOCK_INIT;
+	struct tree_desc desc;
+	struct tree *tree;
+	struct unpack_trees_options unpack_tree_opts;
+	int ret = 0, i;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	/* Determine the length of the label */
+	for (i = 0; i < len; i++)
+		if (isspace(name[i]))
+			len = i;
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	if (get_oid(ref_name.buf, &oid) &&
+	    get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
+		error(_("could not read '%s'"), ref_name.buf);
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
+	unpack_tree_opts.head_idx = 1;
+	unpack_tree_opts.src_index = &the_index;
+	unpack_tree_opts.dst_index = &the_index;
+	unpack_tree_opts.fn = oneway_merge;
+	unpack_tree_opts.merge = 1;
+	unpack_tree_opts.update = 1;
+	unpack_tree_opts.reset = 1;
+
+	if (read_cache_unmerged()) {
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return error_resolve_conflict(_(action_name(opts)));
+	}
+
+	if (!fill_tree_descriptor(&desc, &oid)) {
+		error(_("failed to find tree of %s"), oid_to_hex(&oid));
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	if (unpack_trees(1, &desc, &unpack_tree_opts)) {
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	tree = parse_tree_indirect(&oid);
+	prime_cache_tree(&the_index, tree);
+
+	if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
+		ret = error(_("could not write index"));
+	free((void *)desc.buffer);
+
+	if (!ret)
+		ret = update_ref(reflog_message(opts, "reset", "'%.*s'",
+						len, name), "HEAD", &oid,
+				 NULL, 0, UPDATE_REFS_MSG_ON_ERR);
+
+	strbuf_release(&ref_name);
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2634,7 +2816,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				/* `current` will be incremented below */
 				todo_list->current = -1;
 			}
-		} else if (!is_noop(item->command))
+		} else if (item->command == TODO_LABEL)
+			res = do_label(item->arg, item->arg_len);
+		else if (item->command == TODO_RESET)
+			res = do_reset(item->arg, item->arg_len, opts);
+		else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 05/15] sequencer: introduce the `merge` command
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (3 preceding siblings ...)
  2018-04-10 12:29         ` [PATCH v6 04/15] sequencer: introduce new commands to reset the revision Johannes Schindelin
@ 2018-04-10 12:29         ` Johannes Schindelin
  2018-04-13 10:12           ` Phillip Wood
  2018-04-10 12:29         ` [PATCH v6 06/15] sequencer: fast-forward `merge` commands, if possible Johannes Schindelin
                           ` (11 subsequent siblings)
  16 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

This patch is part of the effort to reimplement `--preserve-merges` with
a substantially improved design, a design that has been developed in the
Git for Windows project to maintain the dozens of Windows-specific patch
series on top of upstream Git.

The previous patch implemented the `label` and `reset` commands to label
commits and to reset to labeled commits. This patch adds the `merge`
command, with the following syntax:

	merge [-C <commit>] <rev> # <oneline>

The <commit> parameter in this instance is the *original* merge commit,
whose author and message will be used for the merge commit that is about
to be created.

The <rev> parameter refers to the (possibly rewritten) revision to
merge. Let's see an example of a todo list:

	label onto

	# Branch abc
	reset onto
	pick deadbeef Hello, world!
	label abc

	reset onto
	pick cafecafe And now for something completely different
	merge -C baaabaaa abc # Merge the branch 'abc' into master

To edit the merge commit's message (a "reword" for merges, if you will),
use `-c` (lower-case) instead of `-C`; this convention was borrowed from
`git commit` that also supports `-c` and `-C` with similar meanings.

To create *new* merges, i.e. without copying the commit message from an
existing commit, simply omit the `-C <commit>` parameter (which will
open an editor for the merge message):

	merge abc

This comes in handy when splitting a branch into two or more branches.

Note: this patch only adds support for recursive merges, to keep things
simple. Support for octopus merges will be added later in a separate
patch series, support for merges using strategies other than the
recursive merge is left for the future.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   4 +
 sequencer.c                | 170 +++++++++++++++++++++++++++++++++++++
 2 files changed, 174 insertions(+)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index e8d3a7d7588..ccd5254d1c9 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -164,6 +164,10 @@ x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
 l, label <label> = label current HEAD with a name
 t, reset <label> = reset HEAD to a label
+m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
+.       create a merge commit using the original merge commit's
+.       message (or the oneline, if no original merge commit was
+.       specified). Use -c <commit> to reword the commit message.
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index c63d47f5e09..1b5f1441102 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1305,6 +1305,7 @@ enum todo_command {
 	TODO_EXEC,
 	TODO_LABEL,
 	TODO_RESET,
+	TODO_MERGE,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -1325,6 +1326,7 @@ static struct {
 	{ 'x', "exec" },
 	{ 'l', "label" },
 	{ 't', "reset" },
+	{ 'm', "merge" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1752,9 +1754,14 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 	return 0;
 }
 
+enum todo_item_flags {
+	TODO_EDIT_MERGE_MSG = 1
+};
+
 struct todo_item {
 	enum todo_command command;
 	struct commit *commit;
+	unsigned int flags;
 	const char *arg;
 	int arg_len;
 	size_t offset_in_buf;
@@ -1789,6 +1796,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	char *end_of_object_name;
 	int i, saved, status, padding;
 
+	item->flags = 0;
+
 	/* left-trim */
 	bol += strspn(bol, " \t");
 
@@ -1838,6 +1847,21 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return 0;
 	}
 
+	if (item->command == TODO_MERGE) {
+		if (skip_prefix(bol, "-C", &bol))
+			bol += strspn(bol, " \t");
+		else if (skip_prefix(bol, "-c", &bol)) {
+			bol += strspn(bol, " \t");
+			item->flags |= TODO_EDIT_MERGE_MSG;
+		} else {
+			item->flags |= TODO_EDIT_MERGE_MSG;
+			item->commit = NULL;
+			item->arg = bol;
+			item->arg_len = (int)(eol - bol);
+			return 0;
+		}
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
@@ -2632,6 +2656,141 @@ static int do_reset(const char *name, int len, struct replay_opts *opts)
 	return ret;
 }
 
+static int do_merge(struct commit *commit, const char *arg, int arg_len,
+		    int flags, struct replay_opts *opts)
+{
+	int run_commit_flags = (flags & TODO_EDIT_MERGE_MSG) ?
+		EDIT_MSG | VERIFY_MSG : 0;
+	struct strbuf ref_name = STRBUF_INIT;
+	struct commit *head_commit, *merge_commit, *i;
+	struct commit_list *bases, *j, *reversed = NULL;
+	struct merge_options o;
+	int merge_arg_len, oneline_offset, ret;
+	static struct lock_file lock;
+	const char *p;
+
+	oneline_offset = arg_len;
+	merge_arg_len = strcspn(arg, " \t\n");
+	p = arg + merge_arg_len;
+	p += strspn(p, " \t\n");
+	if (*p == '#' && (!p[1] || isspace(p[1]))) {
+		p += 1 + strspn(p + 1, " \t\n");
+		oneline_offset = p - arg;
+	} else if (p - arg < arg_len)
+		BUG("octopus merges are not supported yet: '%s'", p);
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
+	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	if (!merge_commit) {
+		/* fall back to non-rewritten ref or commit */
+		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
+		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	}
+	if (!merge_commit) {
+		error(_("could not resolve '%s'"), ref_name.buf);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	head_commit = lookup_commit_reference_by_name("HEAD");
+	if (!head_commit) {
+		rollback_lock_file(&lock);
+		return error(_("cannot merge without a current revision"));
+	}
+
+	if (commit) {
+		const char *message = get_commit_buffer(commit, NULL);
+		const char *body;
+		int len;
+
+		if (!message) {
+			rollback_lock_file(&lock);
+			return error(_("could not get commit message of '%s'"),
+				     oid_to_hex(&commit->object.oid));
+		}
+		write_author_script(message);
+		find_commit_subject(message, &body);
+		len = strlen(body);
+		if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			unuse_commit_buffer(commit, message);
+			rollback_lock_file(&lock);
+			return -1;
+		}
+		unuse_commit_buffer(commit, message);
+	} else {
+		struct strbuf buf = STRBUF_INIT;
+		int len;
+
+		strbuf_addf(&buf, "author %s", git_author_info(0));
+		write_author_script(buf.buf);
+		strbuf_reset(&buf);
+
+		if (oneline_offset < arg_len) {
+			p = arg + oneline_offset;
+			len = arg_len - oneline_offset;
+		} else {
+			strbuf_addf(&buf, "Merge branch '%.*s'",
+				    merge_arg_len, arg);
+			p = buf.buf;
+			len = buf.len;
+		}
+
+		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			strbuf_release(&buf);
+			rollback_lock_file(&lock);
+			return -1;
+		}
+		strbuf_release(&buf);
+	}
+
+	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
+		      git_path_merge_head(), 0);
+	write_message("no-ff", 5, git_path_merge_mode(), 0);
+
+	bases = get_merge_bases(head_commit, merge_commit);
+	for (j = bases; j; j = j->next)
+		commit_list_insert(j->item, &reversed);
+	free_commit_list(bases);
+
+	read_cache();
+	init_merge_options(&o);
+	o.branch1 = "HEAD";
+	o.branch2 = ref_name.buf;
+	o.buffer_output = 2;
+
+	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
+	if (!ret)
+		rerere(opts->allow_rerere_auto);
+	if (ret <= 0)
+		fputs(o.obuf.buf, stdout);
+	strbuf_release(&o.obuf);
+	if (ret < 0) {
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return error(_("conflicts while merging '%.*s'"),
+			     merge_arg_len, arg);
+	}
+
+	if (active_cache_changed &&
+	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
+		strbuf_release(&ref_name);
+		return error(_("merge: Unable to write new index file"));
+	}
+	rollback_lock_file(&lock);
+
+	ret = run_git_commit(git_path_merge_msg(), opts, run_commit_flags);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2820,6 +2979,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_label(item->arg, item->arg_len);
 		else if (item->command == TODO_RESET)
 			res = do_reset(item->arg, item->arg_len, opts);
+		else if (item->command == TODO_MERGE)
+			res = do_merge(item->commit, item->arg, item->arg_len,
+				       item->flags, opts);
 		else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
@@ -3302,8 +3464,16 @@ int transform_todos(unsigned flags)
 					  short_commit_name(item->commit) :
 					  oid_to_hex(&item->commit->object.oid);
 
+			if (item->command == TODO_MERGE) {
+				if (item->flags & TODO_EDIT_MERGE_MSG)
+					strbuf_addstr(&buf, " -c");
+				else
+					strbuf_addstr(&buf, " -C");
+			}
+
 			strbuf_addf(&buf, " %s", oid);
 		}
+
 		/* add all the rest */
 		if (!item->arg_len)
 			strbuf_addch(&buf, '\n');
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 06/15] sequencer: fast-forward `merge` commands, if possible
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (4 preceding siblings ...)
  2018-04-10 12:29         ` [PATCH v6 05/15] sequencer: introduce the `merge` command Johannes Schindelin
@ 2018-04-10 12:29         ` Johannes Schindelin
  2018-04-10 12:29         ` [PATCH v6 07/15] rebase-helper --make-script: introduce a flag to rebase merges Johannes Schindelin
                           ` (10 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

Just like with regular `pick` commands, if we are trying to rebase a
merge commit, we now test whether the parents of said commit match HEAD
and the commits to be merged, and fast-forward if possible.

This is not only faster, but also avoids unnecessary proliferation of
new objects.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 ++++++++++++++++++++++++++++++++-
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 1b5f1441102..fb362fafaa7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2665,7 +2665,7 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	struct commit *head_commit, *merge_commit, *i;
 	struct commit_list *bases, *j, *reversed = NULL;
 	struct merge_options o;
-	int merge_arg_len, oneline_offset, ret;
+	int merge_arg_len, oneline_offset, can_fast_forward, ret;
 	static struct lock_file lock;
 	const char *p;
 
@@ -2750,6 +2750,37 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		strbuf_release(&buf);
 	}
 
+	/*
+	 * If HEAD is not identical to the first parent of the original merge
+	 * commit, we cannot fast-forward.
+	 */
+	can_fast_forward = opts->allow_ff && commit && commit->parents &&
+		!oidcmp(&commit->parents->item->object.oid,
+			&head_commit->object.oid);
+
+	/*
+	 * If the merge head is different from the original one, we cannot
+	 * fast-forward.
+	 */
+	if (can_fast_forward) {
+		struct commit_list *second_parent = commit->parents->next;
+
+		if (second_parent && !second_parent->next &&
+		    oidcmp(&merge_commit->object.oid,
+			   &second_parent->item->object.oid))
+			can_fast_forward = 0;
+	}
+
+	if (can_fast_forward && commit->parents->next &&
+	    !commit->parents->next->next &&
+	    !oidcmp(&commit->parents->next->item->object.oid,
+		    &merge_commit->object.oid)) {
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		return fast_forward_to(&commit->object.oid,
+				       &head_commit->object.oid, 0, opts);
+	}
+
 	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
 		      git_path_merge_head(), 0);
 	write_message("no-ff", 5, git_path_merge_mode(), 0);
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 07/15] rebase-helper --make-script: introduce a flag to rebase merges
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (5 preceding siblings ...)
  2018-04-10 12:29         ` [PATCH v6 06/15] sequencer: fast-forward `merge` commands, if possible Johannes Schindelin
@ 2018-04-10 12:29         ` Johannes Schindelin
  2018-04-10 12:29         ` [PATCH v6 08/15] rebase: introduce the --rebase-merges option Johannes Schindelin
                           ` (9 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

The sequencer just learned new commands intended to recreate branch
structure (similar in spirit to --preserve-merges, but with a
substantially less-broken design).

Let's allow the rebase--helper to generate todo lists making use of
these commands, triggered by the new --rebase-merges option. For a
commit topology like this (where the HEAD points to C):

	- A - B - C
	    \   /
	      D

the generated todo list would look like this:

	# branch D
	pick 0123 A
	label branch-point
	pick 1234 D
	label D

	reset branch-point
	pick 2345 B
	merge -C 3456 D # C

To keep things simple, we first only implement support for merge commits
with exactly two parents, leaving support for octopus merges to a later
patch series.

As a special, hard-coded label, all merge-rebasing todo lists start with
the command `label onto` so that we can later always refer to the revision
onto which everything is rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/rebase--helper.c |   4 +-
 sequencer.c              | 351 ++++++++++++++++++++++++++++++++++++++-
 sequencer.h              |   1 +
 3 files changed, 353 insertions(+), 3 deletions(-)

diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index ad074705bb5..781782e7272 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -12,7 +12,7 @@ static const char * const builtin_rebase_helper_usage[] = {
 int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
-	unsigned flags = 0, keep_empty = 0;
+	unsigned flags = 0, keep_empty = 0, rebase_merges = 0;
 	int abbreviate_commands = 0;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
@@ -24,6 +24,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
 		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
 			N_("allow commits with empty messages")),
+		OPT_BOOL(0, "rebase-merges", &rebase_merges, N_("rebase merge commits")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -57,6 +58,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
+	flags |= rebase_merges ? TODO_LIST_REBASE_MERGES : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
 	if (command == CONTINUE && argc == 1)
diff --git a/sequencer.c b/sequencer.c
index fb362fafaa7..422c71db975 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -25,6 +25,8 @@
 #include "sigchain.h"
 #include "unpack-trees.h"
 #include "worktree.h"
+#include "oidmap.h"
+#include "oidset.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -3368,6 +3370,343 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
 	strbuf_release(&sob);
 }
 
+struct labels_entry {
+	struct hashmap_entry entry;
+	char label[FLEX_ARRAY];
+};
+
+static int labels_cmp(const void *fndata, const struct labels_entry *a,
+		      const struct labels_entry *b, const void *key)
+{
+	return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
+}
+
+struct string_entry {
+	struct oidmap_entry entry;
+	char string[FLEX_ARRAY];
+};
+
+struct label_state {
+	struct oidmap commit2label;
+	struct hashmap labels;
+	struct strbuf buf;
+};
+
+static const char *label_oid(struct object_id *oid, const char *label,
+			     struct label_state *state)
+{
+	struct labels_entry *labels_entry;
+	struct string_entry *string_entry;
+	struct object_id dummy;
+	size_t len;
+	int i;
+
+	string_entry = oidmap_get(&state->commit2label, oid);
+	if (string_entry)
+		return string_entry->string;
+
+	/*
+	 * For "uninteresting" commits, i.e. commits that are not to be
+	 * rebased, and which can therefore not be labeled, we use a unique
+	 * abbreviation of the commit name. This is slightly more complicated
+	 * than calling find_unique_abbrev() because we also need to make
+	 * sure that the abbreviation does not conflict with any other
+	 * label.
+	 *
+	 * We disallow "interesting" commits to be labeled by a string that
+	 * is a valid full-length hash, to ensure that we always can find an
+	 * abbreviation for any uninteresting commit's names that does not
+	 * clash with any other label.
+	 */
+	if (!label) {
+		char *p;
+
+		strbuf_reset(&state->buf);
+		strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
+		label = p = state->buf.buf;
+
+		find_unique_abbrev_r(p, oid, default_abbrev);
+
+		/*
+		 * We may need to extend the abbreviated hash so that there is
+		 * no conflicting label.
+		 */
+		if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
+			size_t i = strlen(p) + 1;
+
+			oid_to_hex_r(p, oid);
+			for (; i < GIT_SHA1_HEXSZ; i++) {
+				char save = p[i];
+				p[i] = '\0';
+				if (!hashmap_get_from_hash(&state->labels,
+							   strihash(p), p))
+					break;
+				p[i] = save;
+			}
+		}
+	} else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
+		    !get_oid_hex(label, &dummy)) ||
+		   (len == 1 && *label == '#') ||
+		   hashmap_get_from_hash(&state->labels,
+					 strihash(label), label)) {
+		/*
+		 * If the label already exists, or if the label is a valid full
+		 * OID, or the label is a '#' (which we use as a separator
+		 * between merge heads and oneline), we append a dash and a
+		 * number to make it unique.
+		 */
+		struct strbuf *buf = &state->buf;
+
+		strbuf_reset(buf);
+		strbuf_add(buf, label, len);
+
+		for (i = 2; ; i++) {
+			strbuf_setlen(buf, len);
+			strbuf_addf(buf, "-%d", i);
+			if (!hashmap_get_from_hash(&state->labels,
+						   strihash(buf->buf),
+						   buf->buf))
+				break;
+		}
+
+		label = buf->buf;
+	}
+
+	FLEX_ALLOC_STR(labels_entry, label, label);
+	hashmap_entry_init(labels_entry, strihash(label));
+	hashmap_add(&state->labels, labels_entry);
+
+	FLEX_ALLOC_STR(string_entry, string, label);
+	oidcpy(&string_entry->entry.oid, oid);
+	oidmap_put(&state->commit2label, string_entry);
+
+	return string_entry->string;
+}
+
+static int make_script_with_merges(struct pretty_print_context *pp,
+				   struct rev_info *revs, FILE *out,
+				   unsigned flags)
+{
+	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
+	struct strbuf label = STRBUF_INIT;
+	struct commit_list *commits = NULL, **tail = &commits, *iter;
+	struct commit_list *tips = NULL, **tips_tail = &tips;
+	struct commit *commit;
+	struct oidmap commit2todo = OIDMAP_INIT;
+	struct string_entry *entry;
+	struct oidset interesting = OIDSET_INIT, child_seen = OIDSET_INIT,
+		shown = OIDSET_INIT;
+	struct label_state state = { OIDMAP_INIT, { NULL }, STRBUF_INIT };
+
+	int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
+	const char *cmd_pick = abbr ? "p" : "pick",
+		*cmd_label = abbr ? "l" : "label",
+		*cmd_reset = abbr ? "t" : "reset",
+		*cmd_merge = abbr ? "m" : "merge";
+
+	oidmap_init(&commit2todo, 0);
+	oidmap_init(&state.commit2label, 0);
+	hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
+	strbuf_init(&state.buf, 32);
+
+	if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
+		struct object_id *oid = &revs->cmdline.rev[0].item->oid;
+		FLEX_ALLOC_STR(entry, string, "onto");
+		oidcpy(&entry->entry.oid, oid);
+		oidmap_put(&state.commit2label, entry);
+	}
+
+	/*
+	 * First phase:
+	 * - get onelines for all commits
+	 * - gather all branch tips (i.e. 2nd or later parents of merges)
+	 * - label all branch tips
+	 */
+	while ((commit = get_revision(revs))) {
+		struct commit_list *to_merge;
+		int is_octopus;
+		const char *p1, *p2;
+		struct object_id *oid;
+		int is_empty;
+
+		tail = &commit_list_insert(commit, tail)->next;
+		oidset_insert(&interesting, &commit->object.oid);
+
+		is_empty = is_original_commit_empty(commit);
+		if (!is_empty && (commit->object.flags & PATCHSAME))
+			continue;
+
+		strbuf_reset(&oneline);
+		pretty_print_commit(pp, commit, &oneline);
+
+		to_merge = commit->parents ? commit->parents->next : NULL;
+		if (!to_merge) {
+			/* non-merge commit: easy case */
+			strbuf_reset(&buf);
+			if (!keep_empty && is_empty)
+				strbuf_addf(&buf, "%c ", comment_line_char);
+			strbuf_addf(&buf, "%s %s %s", cmd_pick,
+				    oid_to_hex(&commit->object.oid),
+				    oneline.buf);
+
+			FLEX_ALLOC_STR(entry, string, buf.buf);
+			oidcpy(&entry->entry.oid, &commit->object.oid);
+			oidmap_put(&commit2todo, entry);
+
+			continue;
+		}
+
+		is_octopus = to_merge && to_merge->next;
+
+		if (is_octopus)
+			BUG("Octopus merges not yet supported");
+
+		/* Create a label */
+		strbuf_reset(&label);
+		if (skip_prefix(oneline.buf, "Merge ", &p1) &&
+		    (p1 = strchr(p1, '\'')) &&
+		    (p2 = strchr(++p1, '\'')))
+			strbuf_add(&label, p1, p2 - p1);
+		else if (skip_prefix(oneline.buf, "Merge pull request ",
+				     &p1) &&
+			 (p1 = strstr(p1, " from ")))
+			strbuf_addstr(&label, p1 + strlen(" from "));
+		else
+			strbuf_addbuf(&label, &oneline);
+
+		for (p1 = label.buf; *p1; p1++)
+			if (isspace(*p1))
+				*(char *)p1 = '-';
+
+		strbuf_reset(&buf);
+		strbuf_addf(&buf, "%s -C %s",
+			    cmd_merge, oid_to_hex(&commit->object.oid));
+
+		/* label the tip of merged branch */
+		oid = &to_merge->item->object.oid;
+		strbuf_addch(&buf, ' ');
+
+		if (!oidset_contains(&interesting, oid))
+			strbuf_addstr(&buf, label_oid(oid, NULL, &state));
+		else {
+			tips_tail = &commit_list_insert(to_merge->item,
+							tips_tail)->next;
+
+			strbuf_addstr(&buf, label_oid(oid, label.buf, &state));
+		}
+		strbuf_addf(&buf, " # %s", oneline.buf);
+
+		FLEX_ALLOC_STR(entry, string, buf.buf);
+		oidcpy(&entry->entry.oid, &commit->object.oid);
+		oidmap_put(&commit2todo, entry);
+	}
+
+	/*
+	 * Second phase:
+	 * - label branch points
+	 * - add HEAD to the branch tips
+	 */
+	for (iter = commits; iter; iter = iter->next) {
+		struct commit_list *parent = iter->item->parents;
+		for (; parent; parent = parent->next) {
+			struct object_id *oid = &parent->item->object.oid;
+			if (!oidset_contains(&interesting, oid))
+				continue;
+			if (!oidset_contains(&child_seen, oid))
+				oidset_insert(&child_seen, oid);
+			else
+				label_oid(oid, "branch-point", &state);
+		}
+
+		/* Add HEAD as implict "tip of branch" */
+		if (!iter->next)
+			tips_tail = &commit_list_insert(iter->item,
+							tips_tail)->next;
+	}
+
+	/*
+	 * Third phase: output the todo list. This is a bit tricky, as we
+	 * want to avoid jumping back and forth between revisions. To
+	 * accomplish that goal, we walk backwards from the branch tips,
+	 * gathering commits not yet shown, reversing the list on the fly,
+	 * then outputting that list (labeling revisions as needed).
+	 */
+	fprintf(out, "%s onto\n", cmd_label);
+	for (iter = tips; iter; iter = iter->next) {
+		struct commit_list *list = NULL, *iter2;
+
+		commit = iter->item;
+		if (oidset_contains(&shown, &commit->object.oid))
+			continue;
+		entry = oidmap_get(&state.commit2label, &commit->object.oid);
+
+		if (entry)
+			fprintf(out, "\n# Branch %s\n", entry->string);
+		else
+			fprintf(out, "\n");
+
+		while (oidset_contains(&interesting, &commit->object.oid) &&
+		       !oidset_contains(&shown, &commit->object.oid)) {
+			commit_list_insert(commit, &list);
+			if (!commit->parents) {
+				commit = NULL;
+				break;
+			}
+			commit = commit->parents->item;
+		}
+
+		if (!commit)
+			fprintf(out, "%s onto\n", cmd_reset);
+		else {
+			const char *to = NULL;
+
+			entry = oidmap_get(&state.commit2label,
+					   &commit->object.oid);
+			if (entry)
+				to = entry->string;
+
+			if (!to || !strcmp(to, "onto"))
+				fprintf(out, "%s onto\n", cmd_reset);
+			else {
+				strbuf_reset(&oneline);
+				pretty_print_commit(pp, commit, &oneline);
+				fprintf(out, "%s %s # %s\n",
+					cmd_reset, to, oneline.buf);
+			}
+		}
+
+		for (iter2 = list; iter2; iter2 = iter2->next) {
+			struct object_id *oid = &iter2->item->object.oid;
+			entry = oidmap_get(&commit2todo, oid);
+			/* only show if not already upstream */
+			if (entry)
+				fprintf(out, "%s\n", entry->string);
+			entry = oidmap_get(&state.commit2label, oid);
+			if (entry)
+				fprintf(out, "%s %s\n",
+					cmd_label, entry->string);
+			oidset_insert(&shown, oid);
+		}
+
+		free_commit_list(list);
+	}
+
+	free_commit_list(commits);
+	free_commit_list(tips);
+
+	strbuf_release(&label);
+	strbuf_release(&oneline);
+	strbuf_release(&buf);
+
+	oidmap_free(&commit2todo, 1);
+	oidmap_free(&state.commit2label, 1);
+	hashmap_free(&state.labels, 1);
+	strbuf_release(&state.buf);
+
+	return 0;
+}
+
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags)
 {
@@ -3378,11 +3717,16 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	struct commit *commit;
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
 	const char *insn = flags & TODO_LIST_ABBREVIATE_CMDS ? "p" : "pick";
+	int rebase_merges = flags & TODO_LIST_REBASE_MERGES;
 
 	init_revisions(&revs, NULL);
 	revs.verbose_header = 1;
-	revs.max_parents = 1;
-	revs.cherry_pick = 1;
+	if (rebase_merges)
+		revs.cherry_mark = 1;
+	else {
+		revs.max_parents = 1;
+		revs.cherry_pick = 1;
+	}
 	revs.limited = 1;
 	revs.reverse = 1;
 	revs.right_only = 1;
@@ -3406,6 +3750,9 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	if (prepare_revision_walk(&revs) < 0)
 		return error(_("make_script: error preparing revisions"));
 
+	if (rebase_merges)
+		return make_script_with_merges(&pp, &revs, out, flags);
+
 	while ((commit = get_revision(&revs))) {
 		strbuf_reset(&buf);
 		if (!keep_empty && is_original_commit_empty(commit))
diff --git a/sequencer.h b/sequencer.h
index e45b178dfc4..6bc4da17243 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -59,6 +59,7 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_KEEP_EMPTY (1U << 0)
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
+#define TODO_LIST_REBASE_MERGES (1U << 3)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 08/15] rebase: introduce the --rebase-merges option
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (6 preceding siblings ...)
  2018-04-10 12:29         ` [PATCH v6 07/15] rebase-helper --make-script: introduce a flag to rebase merges Johannes Schindelin
@ 2018-04-10 12:29         ` Johannes Schindelin
  2018-04-10 12:30         ` [PATCH v6 09/15] rebase --rebase-merges: add test for --keep-empty Johannes Schindelin
                           ` (8 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

Once upon a time, this here developer thought: wouldn't it be nice if,
say, Git for Windows' patches on top of core Git could be represented as
a thicket of branches, and be rebased on top of core Git in order to
maintain a cherry-pick'able set of patch series?

The original attempt to answer this was: git rebase --preserve-merges.

However, that experiment was never intended as an interactive option,
and it only piggy-backed on git rebase --interactive because that
command's implementation looked already very, very familiar: it was
designed by the same person who designed --preserve-merges: yours truly.

Some time later, some other developer (I am looking at you, Andreas!
;-)) decided that it would be a good idea to allow --preserve-merges to
be combined with --interactive (with caveats!) and the Git maintainer
(well, the interim Git maintainer during Junio's absence, that is)
agreed, and that is when the glamor of the --preserve-merges design
started to fall apart rather quickly and unglamorously.

The reason? In --preserve-merges mode, the parents of a merge commit (or
for that matter, of *any* commit) were not stated explicitly, but were
*implied* by the commit name passed to the `pick` command.

This made it impossible, for example, to reorder commits. Not to mention
to flatten the branch topology or, deity forbid, to split topic branches
into two.

Alas, these shortcomings also prevented that mode (whose original
purpose was to serve Git for Windows' needs, with the additional hope
that it may be useful to others, too) from serving Git for Windows'
needs.

Five years later, when it became really untenable to have one unwieldy,
big hodge-podge patch series of partly related, partly unrelated patches
in Git for Windows that was rebased onto core Git's tags from time to
time (earning the undeserved wrath of the developer of the ill-fated
git-remote-hg series that first obsoleted Git for Windows' competing
approach, only to be abandoned without maintainer later) was really
untenable, the "Git garden shears" were born [*1*/*2*]: a script,
piggy-backing on top of the interactive rebase, that would first
determine the branch topology of the patches to be rebased, create a
pseudo todo list for further editing, transform the result into a real
todo list (making heavy use of the `exec` command to "implement" the
missing todo list commands) and finally recreate the patch series on
top of the new base commit.

That was in 2013. And it took about three weeks to come up with the
design and implement it as an out-of-tree script. Needless to say, the
implementation needed quite a few years to stabilize, all the while the
design itself proved itself sound.

With this patch, the goodness of the Git garden shears comes to `git
rebase -i` itself. Passing the `--rebase-merges` option will generate
a todo list that can be understood readily, and where it is obvious
how to reorder commits. New branches can be introduced by inserting
`label` commands and calling `merge <label>`. And once this mode will
have become stable and universally accepted, we can deprecate the design
mistake that was `--preserve-merges`.

Link *1*:
https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
Link *2*:
https://github.com/git-for-windows/build-extra/blob/master/shears.sh

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt           |  10 +-
 contrib/completion/git-completion.bash |   2 +-
 git-rebase--interactive.sh             |   1 +
 git-rebase.sh                          |   6 +
 t/t3430-rebase-merges.sh               | 146 +++++++++++++++++++++++++
 5 files changed, 163 insertions(+), 2 deletions(-)
 create mode 100755 t/t3430-rebase-merges.sh

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 3277ca14327..936c5619b42 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -378,6 +378,13 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
+-r::
+--rebase-merges::
+	Rebase merge commits instead of flattening the history by replaying
+	merges. Merge conflict resolutions or manual amendments to merge
+	commits are not rebased automatically, but have to be applied
+	manually.
+
 -p::
 --preserve-merges::
 	Recreate merge commits instead of flattening the history by replaying
@@ -780,7 +787,8 @@ BUGS
 The todo list presented by `--preserve-merges --interactive` does not
 represent the topology of the revision graph.  Editing commits and
 rewording their commit messages should work fine, but attempts to
-reorder commits tend to produce counterintuitive results.
+reorder commits tend to produce counterintuitive results. Use
+--rebase-merges for a more faithful representation.
 
 For example, an attempt to rearrange
 ------------
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index a7570739454..d4c0a995c39 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1949,7 +1949,7 @@ _git_rebase ()
 	--*)
 		__gitcomp "
 			--onto --merge --strategy --interactive
-			--preserve-merges --stat --no-stat
+			--rebase-merges --preserve-merges --stat --no-stat
 			--committer-date-is-author-date --ignore-date
 			--ignore-whitespace --whitespace=
 			--autosquash --no-autosquash
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index ccd5254d1c9..7a3daf3e40c 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -970,6 +970,7 @@ git_rebase__interactive () {
 	init_revisions_and_shortrevisions
 
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
+		${rebase_merges:+--rebase-merges} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 
diff --git a/git-rebase.sh b/git-rebase.sh
index fb64ee1fe42..a64460fd25a 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,6 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
+r,rebase-merges!   try to rebase merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -88,6 +89,7 @@ type=
 state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
+rebase_merges=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -270,6 +272,10 @@ do
 	--allow-empty-message)
 		allow_empty_message=--allow-empty-message
 		;;
+	--rebase-merges)
+		rebase_merges=t
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
new file mode 100755
index 00000000000..1965005778b
--- /dev/null
+++ b/t/t3430-rebase-merges.sh
@@ -0,0 +1,146 @@
+#!/bin/sh
+#
+# Copyright (c) 2018 Johannes E. Schindelin
+#
+
+test_description='git rebase -i --rebase-merges
+
+This test runs git rebase "interactively", retaining the branch structure by
+recreating merge commits.
+
+Initial setup:
+
+    -- B --                   (first)
+   /       \
+ A - C - D - E - H            (master)
+       \       /
+         F - G                (second)
+'
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
+
+test_cmp_graph () {
+	cat >expect &&
+	git log --graph --boundary --format=%s "$@" >output &&
+	sed "s/ *$//" <output >output.trimmed &&
+	test_cmp expect output.trimmed
+}
+
+test_expect_success 'setup' '
+	write_script replace-editor.sh <<-\EOF &&
+	mv "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	cp script-from-scratch "$1"
+	EOF
+
+	test_commit A &&
+	git checkout -b first &&
+	test_commit B &&
+	git checkout master &&
+	test_commit C &&
+	test_commit D &&
+	git merge --no-commit B &&
+	test_tick &&
+	git commit -m E &&
+	git tag -m E E &&
+	git checkout -b second C &&
+	test_commit F &&
+	test_commit G &&
+	git checkout master &&
+	git merge --no-commit G &&
+	test_tick &&
+	git commit -m H &&
+	git tag -m H H
+'
+
+cat >script-from-scratch <<\EOF
+label onto
+
+# onebranch
+pick G
+pick D
+label onebranch
+
+# second
+reset onto
+pick B
+label second
+
+reset onto
+merge -C H second
+merge onebranch # Merge the topic branch 'onebranch'
+EOF
+
+test_expect_success 'create completely different structure' '
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_tick &&
+	git rebase -i -r A &&
+	test_cmp_graph <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	* |   H
+	|\ \
+	| |/
+	|/|
+	| * B
+	|/
+	* A
+	EOF
+'
+
+test_expect_success 'generate correct todo list' '
+	cat >expect <<-\EOF &&
+	label onto
+
+	reset onto
+	pick d9df450 B
+	label E
+
+	reset onto
+	pick 5dee784 C
+	label branch-point
+	pick ca2c861 F
+	pick 088b00a G
+	label H
+
+	reset branch-point # C
+	pick 12bd07b D
+	merge -C 2051b56 E # E
+	merge -C 233d48a H # H
+
+	EOF
+
+	grep -v "^#" <.git/ORIGINAL-TODO >output &&
+	test_cmp expect output
+'
+
+test_expect_success 'with a branch tip that was cherry-picked already' '
+	git checkout -b already-upstream master &&
+	base="$(git rev-parse --verify HEAD)" &&
+
+	test_commit A1 &&
+	test_commit A2 &&
+	git reset --hard $base &&
+	test_commit B1 &&
+	test_tick &&
+	git merge -m "Merge branch A" A2 &&
+
+	git checkout -b upstream-with-a2 $base &&
+	test_tick &&
+	git cherry-pick A2 &&
+
+	git checkout already-upstream &&
+	test_tick &&
+	git rebase -i -r upstream-with-a2 &&
+	test_cmp_graph upstream-with-a2.. <<-\EOF
+	*   Merge branch A
+	|\
+	| * A1
+	* | B1
+	|/
+	o A2
+	EOF
+'
+
+test_done
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 09/15] rebase --rebase-merges: add test for --keep-empty
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (7 preceding siblings ...)
  2018-04-10 12:29         ` [PATCH v6 08/15] rebase: introduce the --rebase-merges option Johannes Schindelin
@ 2018-04-10 12:30         ` Johannes Schindelin
  2018-04-10 12:30         ` [PATCH v6 10/15] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
                           ` (7 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:30 UTC (permalink / raw)
  To: git
  Cc: Phillip Wood, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov

From: Phillip Wood <phillip.wood@dunelm.org.uk>

If there are empty commits on the left hand side of $upstream...HEAD
then the empty commits on the right hand side that we want to keep are
being pruned.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
---
 t/t3421-rebase-topology-linear.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/t3421-rebase-topology-linear.sh b/t/t3421-rebase-topology-linear.sh
index 68fe2003ef5..fbae5dab7e2 100755
--- a/t/t3421-rebase-topology-linear.sh
+++ b/t/t3421-rebase-topology-linear.sh
@@ -217,6 +217,7 @@ test_run_rebase success ''
 test_run_rebase failure -m
 test_run_rebase failure -i
 test_run_rebase failure -p
+test_run_rebase success --rebase-merges
 
 #       m
 #      /
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 10/15] sequencer: make refs generated by the `label` command worktree-local
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (8 preceding siblings ...)
  2018-04-10 12:30         ` [PATCH v6 09/15] rebase --rebase-merges: add test for --keep-empty Johannes Schindelin
@ 2018-04-10 12:30         ` Johannes Schindelin
  2018-04-10 12:30         ` [PATCH v6 11/15] sequencer: handle post-rewrite for merge commands Johannes Schindelin
                           ` (6 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

This allows for rebases to be run in parallel in separate worktrees
(think: interrupted in the middle of one rebase, being asked to perform
a different rebase, adding a separate worktree just for that job).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 refs.c                   |  3 ++-
 t/t3430-rebase-merges.sh | 14 ++++++++++++++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 8b7a77fe5ee..f61ec58d1df 100644
--- a/refs.c
+++ b/refs.c
@@ -600,7 +600,8 @@ int dwim_log(const char *str, int len, struct object_id *oid, char **log)
 static int is_per_worktree_ref(const char *refname)
 {
 	return !strcmp(refname, "HEAD") ||
-		starts_with(refname, "refs/bisect/");
+		starts_with(refname, "refs/bisect/") ||
+		starts_with(refname, "refs/rewritten/");
 }
 
 static int is_pseudoref_syntax(const char *refname)
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index 1965005778b..95f99d965c1 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -143,4 +143,18 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'refs/rewritten/* is worktree-local' '
+	git worktree add wt &&
+	cat >wt/script-from-scratch <<-\EOF &&
+	label xyz
+	exec GIT_DIR=../.git git rev-parse --verify refs/rewritten/xyz >a || :
+	exec git rev-parse --verify refs/rewritten/xyz >b
+	EOF
+
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	git -C wt rebase -i HEAD &&
+	test_must_be_empty wt/a &&
+	test_cmp_rev HEAD "$(cat wt/b)"
+'
+
 test_done
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 11/15] sequencer: handle post-rewrite for merge commands
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (9 preceding siblings ...)
  2018-04-10 12:30         ` [PATCH v6 10/15] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
@ 2018-04-10 12:30         ` Johannes Schindelin
  2018-04-10 12:30         ` [PATCH v6 12/15] rebase --rebase-merges: avoid "empty merges" Johannes Schindelin
                           ` (5 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

In the previous patches, we implemented the basic functionality of the
`git rebase -i --rebase-merges` command, in particular the `merge`
command to create merge commits in the sequencer.

The interactive rebase is a lot more these days, though, than a simple
cherry-pick in a loop. For example, it calls the post-rewrite hook (if
any) after rebasing with a mapping of the old->new commits.

This patch implements the post-rewrite handling for the `merge` command
we just introduced. The other commands that were added recently (`label`
and `reset`) do not create new commits, therefore post-rewrite hooks do
not need to handle them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c              |  7 +++++--
 t/t3430-rebase-merges.sh | 25 +++++++++++++++++++++++++
 2 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 422c71db975..878ff449fe8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3012,10 +3012,13 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_label(item->arg, item->arg_len);
 		else if (item->command == TODO_RESET)
 			res = do_reset(item->arg, item->arg_len, opts);
-		else if (item->command == TODO_MERGE)
+		else if (item->command == TODO_MERGE) {
 			res = do_merge(item->commit, item->arg, item->arg_len,
 				       item->flags, opts);
-		else if (!is_noop(item->command))
+			if (item->commit)
+				record_in_rewritten(&item->commit->object.oid,
+						    peek_command(todo_list, 1));
+		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index 95f99d965c1..392c1136973 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -157,4 +157,29 @@ test_expect_success 'refs/rewritten/* is worktree-local' '
 	test_cmp_rev HEAD "$(cat wt/b)"
 '
 
+test_expect_success 'post-rewrite hook and fixups work for merges' '
+	git checkout -b post-rewrite &&
+	test_commit same1 &&
+	git reset --hard HEAD^ &&
+	test_commit same2 &&
+	git merge -m "to fix up" same1 &&
+	echo same old same old >same2.t &&
+	test_tick &&
+	git commit --fixup HEAD same2.t &&
+	fixup="$(git rev-parse HEAD)" &&
+
+	mkdir -p .git/hooks &&
+	test_when_finished "rm .git/hooks/post-rewrite" &&
+	echo "cat >actual" | write_script .git/hooks/post-rewrite &&
+
+	test_tick &&
+	git rebase -i --autosquash -r HEAD^^^ &&
+	printf "%s %s\n%s %s\n%s %s\n%s %s\n" >expect $(git rev-parse \
+		$fixup^^2 HEAD^2 \
+		$fixup^^ HEAD^ \
+		$fixup^ HEAD \
+		$fixup HEAD) &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 12/15] rebase --rebase-merges: avoid "empty merges"
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (10 preceding siblings ...)
  2018-04-10 12:30         ` [PATCH v6 11/15] sequencer: handle post-rewrite for merge commands Johannes Schindelin
@ 2018-04-10 12:30         ` Johannes Schindelin
  2018-04-10 12:30         ` [PATCH v6 13/15] pull: accept --rebase=merges to recreate the branch topology Johannes Schindelin
                           ` (4 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

The `git merge` command does not allow merging commits that are already
reachable from HEAD: `git merge HEAD^`, for example, will report that we
are already up to date and not change a thing.

In an interactive rebase, such a merge could occur previously, e.g. when
competing (or slightly modified) versions of a patch series were applied
upstream, and the user had to `git rebase --skip` all of the local
commits, and the topic branch becomes "empty" as a consequence.

Let's teach the todo command `merge` to behave the same as `git merge`.

Seeing as it requires some low-level trickery to create such merges with
Git's commands in the first place, we do not even have to bother to
introduce an option to force `merge` to create such merge commits.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c              | 8 ++++++++
 t/t3430-rebase-merges.sh | 8 ++++++++
 2 files changed, 16 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 878ff449fe8..60bad5708c6 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2788,6 +2788,14 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	write_message("no-ff", 5, git_path_merge_mode(), 0);
 
 	bases = get_merge_bases(head_commit, merge_commit);
+	if (bases && !oidcmp(&merge_commit->object.oid,
+			     &bases->item->object.oid)) {
+		strbuf_release(&ref_name);
+		rollback_lock_file(&lock);
+		/* skip merging an ancestor of HEAD */
+		return 0;
+	}
+
 	for (j = bases; j; j = j->next)
 		commit_list_insert(j->item, &reversed);
 	free_commit_list(bases);
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index 392c1136973..63faf7c2246 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -182,4 +182,12 @@ test_expect_success 'post-rewrite hook and fixups work for merges' '
 	test_cmp expect actual
 '
 
+test_expect_success 'refuse to merge ancestors of HEAD' '
+	echo "merge HEAD^" >script-from-scratch &&
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	before="$(git rev-parse HEAD)" &&
+	git rebase -i HEAD &&
+	test_cmp_rev HEAD $before
+'
+
 test_done
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 13/15] pull: accept --rebase=merges to recreate the branch topology
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (11 preceding siblings ...)
  2018-04-10 12:30         ` [PATCH v6 12/15] rebase --rebase-merges: avoid "empty merges" Johannes Schindelin
@ 2018-04-10 12:30         ` Johannes Schindelin
  2018-04-10 12:30         ` [PATCH v6 14/15] rebase -i: introduce --rebase-merges=[no-]rebase-cousins Johannes Schindelin
                           ` (3 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

Similar to the `preserve` mode simply passing the `--preserve-merges`
option to the `rebase` command, the `merges` mode simply passes the
`--rebase-merges` option.

This will allow users to conveniently rebase non-trivial commit
topologies when pulling new commits, without flattening them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/config.txt               |  8 ++++++++
 Documentation/git-pull.txt             |  5 ++++-
 builtin/pull.c                         | 14 ++++++++++----
 builtin/remote.c                       | 18 ++++++++++++++----
 contrib/completion/git-completion.bash |  2 +-
 5 files changed, 37 insertions(+), 10 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 4e0cff87f62..45916ea8104 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1058,6 +1058,10 @@ branch.<name>.rebase::
 	"git pull" is run. See "pull.rebase" for doing this in a non
 	branch-specific manner.
 +
+When `merges`, pass the `--rebase-merges` option to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
@@ -2616,6 +2620,10 @@ pull.rebase::
 	pull" is run. See "branch.<name>.rebase" for setting this on a
 	per-branch basis.
 +
+When `merges`, pass the `--rebase-merges` option to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
diff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt
index ce05b7a5b13..6f76d815dd3 100644
--- a/Documentation/git-pull.txt
+++ b/Documentation/git-pull.txt
@@ -101,13 +101,16 @@ Options related to merging
 include::merge-options.txt[]
 
 -r::
---rebase[=false|true|preserve|interactive]::
+--rebase[=false|true|merges|preserve|interactive]::
 	When true, rebase the current branch on top of the upstream
 	branch after fetching. If there is a remote-tracking branch
 	corresponding to the upstream branch and the upstream branch
 	was rebased since last fetched, the rebase uses that information
 	to avoid rebasing non-local changes.
 +
+When set to `merges`, rebase using `git rebase --rebase-merges` so that
+locally created merge commits will not be flattened.
++
 When set to preserve, rebase with the `--preserve-merges` option passed
 to `git rebase` so that locally created merge commits will not be flattened.
 +
diff --git a/builtin/pull.c b/builtin/pull.c
index e32d6cd5b4c..70b44146ce4 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -27,14 +27,16 @@ enum rebase_type {
 	REBASE_FALSE = 0,
 	REBASE_TRUE,
 	REBASE_PRESERVE,
+	REBASE_MERGES,
 	REBASE_INTERACTIVE
 };
 
 /**
  * Parses the value of --rebase. If value is a false value, returns
  * REBASE_FALSE. If value is a true value, returns REBASE_TRUE. If value is
- * "preserve", returns REBASE_PRESERVE. If value is a invalid value, dies with
- * a fatal error if fatal is true, otherwise returns REBASE_INVALID.
+ * "merges", returns REBASE_MERGES. If value is "preserve", returns
+ * REBASE_PRESERVE. If value is a invalid value, dies with a fatal error if
+ * fatal is true, otherwise returns REBASE_INVALID.
  */
 static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		int fatal)
@@ -47,6 +49,8 @@ static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		return REBASE_TRUE;
 	else if (!strcmp(value, "preserve"))
 		return REBASE_PRESERVE;
+	else if (!strcmp(value, "merges"))
+		return REBASE_MERGES;
 	else if (!strcmp(value, "interactive"))
 		return REBASE_INTERACTIVE;
 
@@ -130,7 +134,7 @@ static struct option pull_options[] = {
 	/* Options passed to git-merge or git-rebase */
 	OPT_GROUP(N_("Options related to merging")),
 	{ OPTION_CALLBACK, 'r', "rebase", &opt_rebase,
-	  "false|true|preserve|interactive",
+	  "false|true|merges|preserve|interactive",
 	  N_("incorporate changes by rebasing rather than merging"),
 	  PARSE_OPT_OPTARG, parse_opt_rebase },
 	OPT_PASSTHRU('n', NULL, &opt_diffstat, NULL,
@@ -800,7 +804,9 @@ static int run_rebase(const struct object_id *curr_head,
 	argv_push_verbosity(&args);
 
 	/* Options passed to git-rebase */
-	if (opt_rebase == REBASE_PRESERVE)
+	if (opt_rebase == REBASE_MERGES)
+		argv_array_push(&args, "--rebase-merges");
+	else if (opt_rebase == REBASE_PRESERVE)
 		argv_array_push(&args, "--preserve-merges");
 	else if (opt_rebase == REBASE_INTERACTIVE)
 		argv_array_push(&args, "--interactive");
diff --git a/builtin/remote.c b/builtin/remote.c
index 805ffc05cdb..45c9219e07a 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -245,7 +245,9 @@ static int add(int argc, const char **argv)
 struct branch_info {
 	char *remote_name;
 	struct string_list merge;
-	enum { NO_REBASE, NORMAL_REBASE, INTERACTIVE_REBASE } rebase;
+	enum {
+		NO_REBASE, NORMAL_REBASE, INTERACTIVE_REBASE, REBASE_MERGES
+	} rebase;
 };
 
 static struct string_list branch_list = STRING_LIST_INIT_NODUP;
@@ -306,6 +308,8 @@ static int config_read_branches(const char *key, const char *value, void *cb)
 				info->rebase = v;
 			else if (!strcmp(value, "preserve"))
 				info->rebase = NORMAL_REBASE;
+			else if (!strcmp(value, "merges"))
+				info->rebase = REBASE_MERGES;
 			else if (!strcmp(value, "interactive"))
 				info->rebase = INTERACTIVE_REBASE;
 		}
@@ -963,9 +967,15 @@ static int show_local_info_item(struct string_list_item *item, void *cb_data)
 
 	printf("    %-*s ", show_info->width, item->string);
 	if (branch_info->rebase) {
-		printf_ln(branch_info->rebase == INTERACTIVE_REBASE
-			  ? _("rebases interactively onto remote %s")
-			  : _("rebases onto remote %s"), merge->items[0].string);
+		const char *msg;
+		if (branch_info->rebase == INTERACTIVE_REBASE)
+			msg = _("rebases interactively onto remote %s");
+		else if (branch_info->rebase == REBASE_MERGES)
+			msg = _("rebases interactively (with merges) onto "
+				"remote %s");
+		else
+			msg = _("rebases onto remote %s");
+		printf_ln(msg, merge->items[0].string);
 		return 0;
 	} else if (show_info->any_rebase) {
 		printf_ln(_(" merges with remote %s"), merge->items[0].string);
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index d4c0a995c39..6af65155c59 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2120,7 +2120,7 @@ _git_config ()
 		return
 		;;
 	branch.*.rebase)
-		__gitcomp "false true preserve interactive"
+		__gitcomp "false true merges preserve interactive"
 		return
 		;;
 	remote.pushdefault)
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 14/15] rebase -i: introduce --rebase-merges=[no-]rebase-cousins
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (12 preceding siblings ...)
  2018-04-10 12:30         ` [PATCH v6 13/15] pull: accept --rebase=merges to recreate the branch topology Johannes Schindelin
@ 2018-04-10 12:30         ` Johannes Schindelin
  2018-04-12 11:30           ` Sergey Organov
  2018-04-10 12:30         ` [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
                           ` (2 subsequent siblings)
  16 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

This one is a bit tricky to explain, so let's try with a diagram:

        C
      /   \
A - B - E - F
  \   /
    D

To illustrate what this new mode is all about, let's consider what
happens upon `git rebase -i --rebase-merges B`, in particular to
the commit `D`. So far, the new branch structure would be:

       --- C' --
      /         \
A - B ------ E' - F'
      \    /
        D'

This is not really preserving the branch topology from before! The
reason is that the commit `D` does not have `B` as ancestor, and
therefore it gets rebased onto `B`.

This is unintuitive behavior. Even worse, when recreating branch
structure, most use cases would appear to want cousins *not* to be
rebased onto the new base commit. For example, Git for Windows (the
heaviest user of the Git garden shears, which served as the blueprint
for --rebase-merges) frequently merges branches from `next` early, and
these branches certainly do *not* want to be rebased. In the example
above, the desired outcome would look like this:

       --- C' --
      /         \
A - B ------ E' - F'
  \        /
   -- D' --

Let's introduce the term "cousins" for such commits ("D" in the
example), and let's not rebase them by default, introducing the new
"rebase-cousins" mode for use cases where they should be rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt |  7 ++++++-
 builtin/rebase--helper.c     |  9 ++++++++-
 git-rebase--interactive.sh   |  1 +
 git-rebase.sh                | 12 +++++++++++-
 sequencer.c                  |  4 ++++
 sequencer.h                  |  6 ++++++
 t/t3430-rebase-merges.sh     | 18 ++++++++++++++++++
 7 files changed, 54 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 936c5619b42..8feadf6e663 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -379,11 +379,16 @@ rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
 -r::
---rebase-merges::
+--rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
 	Rebase merge commits instead of flattening the history by replaying
 	merges. Merge conflict resolutions or manual amendments to merge
 	commits are not rebased automatically, but have to be applied
 	manually.
++
+By default, or when `no-rebase-cousins` was specified, commits which do not
+have `<upstream>` as direct ancestor will keep their original branch point.
+If the `rebase-cousins` mode is turned on, such commits are rebased onto
+`<upstream>` (or `<onto>`, if specified).
 
 -p::
 --preserve-merges::
diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index 781782e7272..f7c2a5fdc81 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
 	unsigned flags = 0, keep_empty = 0, rebase_merges = 0;
-	int abbreviate_commands = 0;
+	int abbreviate_commands = 0, rebase_cousins = -1;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
 		CHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH,
@@ -25,6 +25,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
 			N_("allow commits with empty messages")),
 		OPT_BOOL(0, "rebase-merges", &rebase_merges, N_("rebase merge commits")),
+		OPT_BOOL(0, "rebase-cousins", &rebase_cousins,
+			 N_("keep original branch points of cousins")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -59,8 +61,13 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
 	flags |= rebase_merges ? TODO_LIST_REBASE_MERGES : 0;
+	flags |= rebase_cousins > 0 ? TODO_LIST_REBASE_COUSINS : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
+	if (rebase_cousins >= 0 && !rebase_merges)
+		warning(_("--[no-]rebase-cousins has no effect without "
+			  "--rebase-merges"));
+
 	if (command == CONTINUE && argc == 1)
 		return !!sequencer_continue(&opts);
 	if (command == ABORT && argc == 1)
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 7a3daf3e40c..b4ad130e8b1 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -971,6 +971,7 @@ git_rebase__interactive () {
 
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
 		${rebase_merges:+--rebase-merges} \
+		${rebase_cousins:+--rebase-cousins} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 
diff --git a/git-rebase.sh b/git-rebase.sh
index a64460fd25a..157705d2a72 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,7 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
-r,rebase-merges!   try to rebase merges instead of skipping them
+r,rebase-merges?   try to rebase merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -90,6 +90,7 @@ state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
 rebase_merges=
+rebase_cousins=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -276,6 +277,15 @@ do
 		rebase_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
 		;;
+	--rebase-merges=*)
+		rebase_merges=t
+		case "${1#*=}" in
+		rebase-cousins) rebase_cousins=t;;
+		no-rebase-cousins) rebase_cousins=;;
+		*) die "Unknown mode: $1";;
+		esac
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/sequencer.c b/sequencer.c
index 60bad5708c6..809df1ce484 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3499,6 +3499,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 				   unsigned flags)
 {
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	int rebase_cousins = flags & TODO_LIST_REBASE_COUSINS;
 	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
 	struct strbuf label = STRBUF_INIT;
 	struct commit_list *commits = NULL, **tail = &commits, *iter;
@@ -3676,6 +3677,9 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 					   &commit->object.oid);
 			if (entry)
 				to = entry->string;
+			else if (!rebase_cousins)
+				to = label_oid(&commit->object.oid, NULL,
+					       &state);
 
 			if (!to || !strcmp(to, "onto"))
 				fprintf(out, "%s onto\n", cmd_reset);
diff --git a/sequencer.h b/sequencer.h
index 6bc4da17243..d9570d92b11 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -60,6 +60,12 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
 #define TODO_LIST_REBASE_MERGES (1U << 3)
+/*
+ * When rebasing merges, commits that do have the base commit as ancestor
+ * ("cousins") are *not* rebased onto the new base by default. If those
+ * commits should be rebased onto the new base, this flag needs to be passed.
+ */
+#define TODO_LIST_REBASE_COUSINS (1U << 4)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index 63faf7c2246..ee006810573 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -143,6 +143,24 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'do not rebase cousins unless asked for' '
+	git checkout -b cousins master &&
+	before="$(git rev-parse --verify HEAD)" &&
+	test_tick &&
+	git rebase -r HEAD^ &&
+	test_cmp_rev HEAD $before &&
+	test_tick &&
+	git rebase --rebase-merges=rebase-cousins HEAD^ &&
+	test_cmp_graph HEAD^.. <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	|/
+	o H
+	EOF
+'
+
 test_expect_success 'refs/rewritten/* is worktree-local' '
 	git worktree add wt &&
 	cat >wt/script-from-scratch <<-\EOF &&
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (13 preceding siblings ...)
  2018-04-10 12:30         ` [PATCH v6 14/15] rebase -i: introduce --rebase-merges=[no-]rebase-cousins Johannes Schindelin
@ 2018-04-10 12:30         ` Johannes Schindelin
  2018-04-10 18:49           ` Martin Ågren
                             ` (2 more replies)
  2018-04-10 14:52         ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Sergey Organov
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
  16 siblings, 3 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 12:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

The --rebase-merges mode is probably not half as intuitive to use as
its inventor hopes, so let's document it some.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt | 125 +++++++++++++++++++++++++++++++++++
 1 file changed, 125 insertions(+)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 8feadf6e663..be946de2efb 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -389,6 +389,8 @@ By default, or when `no-rebase-cousins` was specified, commits which do not
 have `<upstream>` as direct ancestor will keep their original branch point.
 If the `rebase-cousins` mode is turned on, such commits are rebased onto
 `<upstream>` (or `<onto>`, if specified).
++
+See also REBASING MERGES below.
 
 -p::
 --preserve-merges::
@@ -787,6 +789,129 @@ The ripple effect of a "hard case" recovery is especially bad:
 'everyone' downstream from 'topic' will now have to perform a "hard
 case" recovery too!
 
+REBASING MERGES
+-----------------
+
+The interactive rebase command was originally designed to handle
+individual patch series. As such, it makes sense to exclude merge
+commits from the todo list, as the developer may have merged the
+current `master` while working on the branch, only to eventually
+rebase all the commits onto `master` (skipping the merge commits).
+
+However, there are legitimate reasons why a developer may want to
+recreate merge commits: to keep the branch structure (or "commit
+topology") when working on multiple, inter-related branches.
+
+In the following example, the developer works on a topic branch that
+refactors the way buttons are defined, and on another topic branch
+that uses that refactoring to implement a "Report a bug" button. The
+output of `git log --graph --format=%s -5` may look like this:
+
+------------
+*   Merge branch 'report-a-bug'
+|\
+| * Add the feedback button
+* | Merge branch 'refactor-button'
+|\ \
+| |/
+| * Use the Button class for all buttons
+| * Extract a generic Button class from the DownloadButton one
+------------
+
+The developer might want to rebase those commits to a newer `master`
+while keeping the branch topology, for example when the first topic
+branch is expected to be integrated into `master` much earlier than the
+second one, say, to resolve merge conflicts with changes to the
+DownloadButton class that made it into `master`.
+
+This rebase can be performed using the `--rebase-merges` option.
+It will generate a todo list looking like this:
+
+------------
+label onto
+
+# Branch: refactor-button
+reset onto
+pick 123456 Extract a generic Button class from the DownloadButton one
+pick 654321 Use the Button class for all buttons
+label refactor-button
+
+# Branch: report-a-bug
+reset refactor-button # Use the Button class for all buttons
+pick abcdef Add the feedback button
+label report-a-bug
+
+reset onto
+merge -C a1b2c3 refactor-button # Merge 'refactor-button'
+merge -C 6f5e4d report-a-bug # Merge 'report-a-bug'
+------------
+
+In contrast to a regular interactive rebase, there are `label`, `reset` and
+`merge` commands in addition to `pick` ones.
+
+The `label` command puts a label to whatever will be the current
+revision when that command is executed. Internally, these labels are
+worktree-local refs that will be deleted when the rebase finishes or
+when it is aborted. That way, rebase operations in multiple worktrees
+linked to the same repository do not interfere with one another.
+
+The `reset` command is essentially a `git reset --hard` to the specified
+revision (typically a previously-labeled one).
+
+The `merge` command will merge the specified revision into whatever is
+HEAD at that time. With `-C <original-commit>`, the commit message of
+the specified merge commit will be used. When the `-C` is changed to
+a lower-case `-c`, the message will be opened in an editor after a
+successful merge so that the user can edit the message.
+
+At this time, the `merge` command will *always* use the `recursive`
+merge strategy, with no way to choose a different one. To work around
+this, an `exec` command can be used to call `git merge` explicitly,
+using the fact that the labels are worktree-local refs (the ref
+`refs/rewritten/onto` would correspond to the label `onto`).
+
+Note: the first command (`reset onto`) labels the revision onto which
+the commits are rebased; The name `onto` is just a convention, as a nod
+to the `--onto` option.
+
+It is also possible to introduce completely new merge commits from scratch
+by adding a command of the form `merge <merge-head>`. This form will
+generate a tentative commit message and always open an editor to let the
+user edit it. This can be useful e.g. when a topic branch turns out to
+address more than a single concern and wants to be split into two or
+even more topic branches. Consider this todo list:
+
+------------
+pick 192837 Switch from GNU Makefiles to CMake
+pick 5a6c7e Document the switch to CMake
+pick 918273 Fix detection of OpenSSL in CMake
+pick afbecd http: add support for TLS v1.3
+pick fdbaec Fix detection of cURL in CMake on Windows
+------------
+
+The one commit in this list that is not related to CMake may very well
+have been motivated by working on fixing all those bugs introduced by
+switching to CMake, but it addresses a different concern. To split this
+branch into two topic branches, the todo list could be edited like this:
+
+------------
+label onto
+
+pick afbecd http: add support for TLS v1.3
+label tlsv1.3
+
+reset onto
+pick 192837 Switch from GNU Makefiles to CMake
+pick 918273 Fix detection of OpenSSL in CMake
+pick fdbaec Fix detection of cURL in CMake on Windows
+pick 5a6c7e Document the switch to CMake
+label cmake
+
+reset onto
+merge tlsv1.3
+merge cmake
+------------
+
 BUGS
 ----
 The todo list presented by `--preserve-merges --interactive` does not
-- 
2.17.0.windows.1.4.g7e4058d72e3

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (14 preceding siblings ...)
  2018-04-10 12:30         ` [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
@ 2018-04-10 14:52         ` Sergey Organov
  2018-04-10 22:11           ` Johannes Schindelin
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
  16 siblings, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-04-10 14:52 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt

Hi Johannes,

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Once upon a time, I dreamt of an interactive rebase that would not
> flatten branch structure, but instead recreate the commit topology
> faithfully.

[...]

> Think of --rebase-merges as "--preserve-merges done right".

Both option names seem to miss the primary point of the mode of
operation that you've formulated in the first sentence. I suggest to
rather call the new option in accordance to your description, say,
--no-flatten, --keep-topology, or --preserve-shape.

Besides, this way the option name will only specify one thing: _what_ it
is about, leaving out the _how_ part, that could vary and could then be
specified as option value or as another companion option(s), that is
usually considered to be an indication of a good design.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page
  2018-04-10 12:30         ` [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
@ 2018-04-10 18:49           ` Martin Ågren
  2018-04-10 21:56             ` Johannes Schindelin
  2018-04-11 15:35           ` Phillip Wood
  2018-04-12 11:52           ` Sergey Organov
  2 siblings, 1 reply; 412+ messages in thread
From: Martin Ågren @ 2018-04-10 18:49 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Git Mailing List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov

On 10 April 2018 at 14:30, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> The --rebase-merges mode is probably not half as intuitive to use as
> its inventor hopes, so let's document it some.

I quite like this documentation. Well-structured and well-paced.
Already after the first reading, I believe I understand how to use this.

> +The `label` command puts a label to whatever will be the current
> +revision when that command is executed. Internally, these labels are
> +worktree-local refs that will be deleted when the rebase finishes or
> +when it is aborted. That way, rebase operations in multiple worktrees
> +linked to the same repository do not interfere with one another.

In the above paragraph, you say "internally".

> +At this time, the `merge` command will *always* use the `recursive`
> +merge strategy, with no way to choose a different one. To work around
> +this, an `exec` command can be used to call `git merge` explicitly,
> +using the fact that the labels are worktree-local refs (the ref
> +`refs/rewritten/onto` would correspond to the label `onto`).

This sort of encourages use of that "internal" detail, which made me a
little bit surprised at first. But if we can't come up with a reason why
we would want to change the "refs/rewritten/<label>"-concept later (I
can't) and if we think the gain this paragraph gives is significant (it
basically gives access to `git merge` in its entirety), then providing
this hint might be the correct thing to do.

> +Note: the first command (`reset onto`) labels the revision onto which
> +the commits are rebased; The name `onto` is just a convention, as a nod
> +to the `--onto` option.

s/reset onto/label onto/

Martin

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page
  2018-04-10 18:49           ` Martin Ågren
@ 2018-04-10 21:56             ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 21:56 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Git Mailing List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov

[-- Attachment #1: Type: text/plain, Size: 2554 bytes --]

Hi Martin,

On Tue, 10 Apr 2018, Martin Ågren wrote:

> On 10 April 2018 at 14:30, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > The --rebase-merges mode is probably not half as intuitive to use as
> > its inventor hopes, so let's document it some.
> 
> I quite like this documentation. Well-structured and well-paced.
> Already after the first reading, I believe I understand how to use this.

Thanks!

> > +The `label` command puts a label to whatever will be the current
> > +revision when that command is executed. Internally, these labels are
> > +worktree-local refs that will be deleted when the rebase finishes or
> > +when it is aborted. That way, rebase operations in multiple worktrees
> > +linked to the same repository do not interfere with one another.
> 
> In the above paragraph, you say "internally".

I guess that I should reword this to say "These labels are created as
worktree-local refs (`refs/rewritten/<label>`) that will be ..."

I'll do that, thanks for the sanity check!

> > +At this time, the `merge` command will *always* use the `recursive`
> > +merge strategy, with no way to choose a different one. To work around
> > +this, an `exec` command can be used to call `git merge` explicitly,
> > +using the fact that the labels are worktree-local refs (the ref
> > +`refs/rewritten/onto` would correspond to the label `onto`).
> 
> This sort of encourages use of that "internal" detail, which made me a
> little bit surprised at first. But if we can't come up with a reason why
> we would want to change the "refs/rewritten/<label>"-concept later (I
> can't) and if we think the gain this paragraph gives is significant (it
> basically gives access to `git merge` in its entirety), then providing
> this hint might be the correct thing to do.

You are right. I made it sound as if this was an implementation detail
that you should not rely on, when I wanted to say that this is how it is
implemented and you are free to use it in your scripts.

> > +Note: the first command (`reset onto`) labels the revision onto which
> > +the commits are rebased; The name `onto` is just a convention, as a nod
> > +to the `--onto` option.
> 
> s/reset onto/label onto/

D'oh!

Thanks, fixed. Current state is in `sequencer-shears` in
https://github.com/dscho/git (I will update the `recreate-merges` branch,
which needs to keep its name so that my scripts will connect the mail
threads for the patch submissions, once I called `git rebase -kir @{u}`).

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-10 14:52         ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Sergey Organov
@ 2018-04-10 22:11           ` Johannes Schindelin
  2018-04-11  4:54             ` Sergey Organov
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-10 22:11 UTC (permalink / raw)
  To: Sergey Organov
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt

Hi Sergey,

On Tue, 10 Apr 2018, Sergey Organov wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Once upon a time, I dreamt of an interactive rebase that would not
> > flatten branch structure, but instead recreate the commit topology
> > faithfully.
> 
> [...]
> 
> > Think of --rebase-merges as "--preserve-merges done right".
> 
> Both option names seem to miss the primary point of the mode of
> operation that you've formulated in the first sentence. I suggest to
> rather call the new option in accordance to your description, say,
> --no-flatten, --keep-topology, or --preserve-shape.

A very quick A/B test shows that neither --no-flatten nor --keep-topology
and certainly not --preserve-shape conveys to Git users what those options
are supposed to do.

But --rebase-merges did convey the purpose of my patch series. So there.

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-10 22:11           ` Johannes Schindelin
@ 2018-04-11  4:54             ` Sergey Organov
  2018-04-11 11:28               ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-04-11  4:54 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Sergey,
>
> On Tue, 10 Apr 2018, Sergey Organov wrote:
>
>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> 
>> > Once upon a time, I dreamt of an interactive rebase that would not
>> > flatten branch structure, but instead recreate the commit topology
>> > faithfully.
>> 
>> [...]
>> 
>> > Think of --rebase-merges as "--preserve-merges done right".
>> 
>> Both option names seem to miss the primary point of the mode of
>> operation that you've formulated in the first sentence. I suggest to
>> rather call the new option in accordance to your description, say,
>> --no-flatten, --keep-topology, or --preserve-shape.
>
> A very quick A/B test shows that neither --no-flatten nor --keep-topology
> and certainly not --preserve-shape conveys to Git users what those options
> are supposed to do.

In fact, my preference would be --[no-]flatten, exactly because the
default mode of rebase operation flattens the history, and thus what I'm
talking about is:

git rebase --no-flatten

vs 

git rebase --rebase-merges

I honestly fail to see how the latter conveys the purpose of the option
better than the former, sorry. To tell the truth, the latter also looks
plain ugly to me.

> But --rebase-merges did convey the purpose of my patch series. So
> there.

- Except that your primary description of the series (that I find pretty
solid) doesn't mention _merges_ at all and still conveys the purpose?

- Except that this patch series _don't_ actually _rebase_ merges?
Yeah, I remember a follow-up is to be expected, but anyway.

I'm still unconvinced.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 04/15] sequencer: introduce new commands to reset the revision
  2018-04-10 12:29         ` [PATCH v6 04/15] sequencer: introduce new commands to reset the revision Johannes Schindelin
@ 2018-04-11  6:17           ` Sergey Organov
  2018-04-11 11:36             ` Johannes Schindelin
  2018-04-13 10:03           ` Phillip Wood
  1 sibling, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-04-11  6:17 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt

Hi Johannes,

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

[...]

> We disallow '#' as label because that character will be used as separator
> in the upcoming `merge` command.

Please consider to use # not only in `merge` and `reset`, but in the rest
of the commands as well, to unify this new syntax. I.e., right now it
seems to be:

pick  abcd A commit message
merge beaf # B commit message

I suggest to turn it to:

pick  abcd # A commit message
merge beaf # B commit message

So that the # is finally universally the start of comment.

While we are at this, I couldn't find any even semi-formal syntax
description of the entire todo list. Is there one already? Are you going
to provide one?

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-11  4:54             ` Sergey Organov
@ 2018-04-11 11:28               ` Johannes Schindelin
  2018-04-11 13:13                 ` Sergey Organov
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-11 11:28 UTC (permalink / raw)
  To: Sergey Organov
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt

Hi Sergey,

On Wed, 11 Apr 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Tue, 10 Apr 2018, Sergey Organov wrote:
> >
> >> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> >> 
> >> > Once upon a time, I dreamt of an interactive rebase that would not
> >> > flatten branch structure, but instead recreate the commit topology
> >> > faithfully.
> >> 
> >> [...]
> >> 
> >> > Think of --rebase-merges as "--preserve-merges done right".
> >> 
> >> Both option names seem to miss the primary point of the mode of
> >> operation that you've formulated in the first sentence. I suggest to
> >> rather call the new option in accordance to your description, say,
> >> --no-flatten, --keep-topology, or --preserve-shape.
> >
> > A very quick A/B test shows that neither --no-flatten nor --keep-topology
> > and certainly not --preserve-shape conveys to Git users what those options
> > are supposed to do.
> 
> In fact, my preference would be --[no-]flatten, exactly because the
> default mode of rebase operation flattens the history, and thus what I'm
> talking about is:
> 
> git rebase --no-flatten

And this is the option out of the four that fared *worst* in the A/B
testing. Not even experts in Git internals were able to figure out what
the heck you are talking about.

Now, you can beat that dead horse until it is pulp. Your choice. I'd
rather go on to more interesting things, because as far as I am concerned,
the naming issue has been settled, with you being the only person in
disfavor of --rebase-merges.

What you *could* do is finally take your RFC to the test. Run it with the
concrete example I showed you in
https://public-inbox.org/git/nycvar.QRO.7.76.6.1803261405170.77@ZVAVAG-6OXH6DA.rhebcr.pbec.zvpebfbsg.pbz/

It is high time that you demonstrated on this concrete case study how your
proposed solution performs. And then tally that up with Phillip's
strategy.

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 04/15] sequencer: introduce new commands to reset the revision
  2018-04-11  6:17           ` Sergey Organov
@ 2018-04-11 11:36             ` Johannes Schindelin
  2018-04-11 16:07               ` Sergey Organov
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-11 11:36 UTC (permalink / raw)
  To: Sergey Organov
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt

Hi Sergey,

On Wed, 11 Apr 2018, Sergey Organov wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> [...]
> 
> > We disallow '#' as label because that character will be used as
> > separator in the upcoming `merge` command.
> 
> Please consider to use # not only in `merge` and `reset`, but in the
> rest of the commands as well, to unify this new syntax. I.e., right now
> it seems to be:
> 
> pick  abcd A commit message
> merge beaf # B commit message
> 
> I suggest to turn it to:
> 
> pick  abcd # A commit message
> merge beaf # B commit message

First of all, that alignment of pick's and merge's first arguments? That
does not exist. If you want aligned arguments, you have to use the
rebase.abbreviateCommands feature.

Second: this change would break backwards-compatibility. For almost eleven
years, we generated `pick abcdef0123 A commit message`. Even if there are
no scripts that rely on this form, power users have gotten used to it, and
I can tell you from experience how unsettling even minor visual changes
are in everyday operations.

In short: no, we cannot do that. Just like your proposal to conflate the
`merge` and `pick` commands for some perception of consistency: The user
experience is more important than individual persons' sense of elegance
(that might not even be shared with the majority).

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-11 11:28               ` Johannes Schindelin
@ 2018-04-11 13:13                 ` Sergey Organov
  2018-04-11 20:40                   ` Johannes Schindelin
  2018-04-11 23:52                   ` Jacob Keller
  0 siblings, 2 replies; 412+ messages in thread
From: Sergey Organov @ 2018-04-11 13:13 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Sergey,
>
> On Wed, 11 Apr 2018, Sergey Organov wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>>
>> > On Tue, 10 Apr 2018, Sergey Organov wrote:
>> >
>> >> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> >>
>> >> > Once upon a time, I dreamt of an interactive rebase that would not
>> >> > flatten branch structure, but instead recreate the commit topology
>> >> > faithfully.
>> >>
>> >> [...]
>> >>
>> >> > Think of --rebase-merges as "--preserve-merges done right".
>> >>
>> >> Both option names seem to miss the primary point of the mode of
>> >> operation that you've formulated in the first sentence. I suggest to
>> >> rather call the new option in accordance to your description, say,
>> >> --no-flatten, --keep-topology, or --preserve-shape.
>> >
>> > A very quick A/B test shows that neither --no-flatten nor --keep-topology
>> > and certainly not --preserve-shape conveys to Git users what those options
>> > are supposed to do.
>>
>> In fact, my preference would be --[no-]flatten, exactly because the
>> default mode of rebase operation flattens the history, and thus what I'm
>> talking about is:
>>
>> git rebase --no-flatten
>
> And this is the option out of the four that fared *worst* in the A/B
> testing. Not even experts in Git internals were able to figure out what
> the heck you are talking about.

It was you who introduced the "flatten" term, not me. I took it from
your descriptions.

So they are able to make sense of your own:

>>> Once upon a time, I dreamt of an interactive rebase that would not
>>> flatten branch structure, but instead recreate the commit topology
>>> faithfully.

Yet they can't get:

--no-flatten::
	Instead of flattening branch structure, recreate the commit
	topology faithfully

Are you kidding?

Well, suppose for a moment that nobody could even guess what "flatten"
means indeed. Then are you willing to remove the "flatten" from both the
description of our patch series and from the proposed patch to the Git
manual:

-r::
--rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
	Rebase merge commits instead of _flattening_ the history by replaying
	merges.

???

>
> Now, you can beat that dead horse until it is pulp. Your choice. I'd
> rather go on to more interesting things, because as far as I am concerned,
> the naming issue has been settled, with you being the only person in
> disfavor of --rebase-merges.

It was rather --recreate-merges just a few weeks ago, and I've seen
nobody actually commented either in favor or against the
--rebase-merges.

git rebase --rebase-merges

_is_ plain simple ugly.

>
> What you *could* do is finally take your RFC to the test. Run it with the
> concrete example I showed you in
> https://public-inbox.org/git/nycvar.QRO.7.76.6.1803261405170.77@ZVAVAG-6OXH6DA.rhebcr.pbec.zvpebfbsg.pbz/
>
> It is high time that you demonstrated on this concrete case study how your
> proposed solution performs. And then tally that up with Phillip's
> strategy.

What you could do is to stop shifting the subject of discussion.

The RFC v2 and Phillip's strategy are essentially the same, as has been
already shown multiple times, both theoretically and by testing. Ask
Bugga for details.

One way or another, this doesn't make

git rebase --rebase-merges

even a bit less ugly.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page
  2018-04-10 12:30         ` [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
  2018-04-10 18:49           ` Martin Ågren
@ 2018-04-11 15:35           ` Phillip Wood
  2018-04-11 19:10             ` Eric Sunshine
  2018-04-12  9:30             ` Johannes Schindelin
  2018-04-12 11:52           ` Sergey Organov
  2 siblings, 2 replies; 412+ messages in thread
From: Phillip Wood @ 2018-04-11 15:35 UTC (permalink / raw)
  To: Johannes Schindelin, git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

On 10/04/18 13:30, Johannes Schindelin wrote:

Firstly let me say that I think expanding the documentation and having 
an example is an excellent idea.

> +
> +------------
> +label onto
> +
> +# Branch: refactor-button
> +reset onto
> +pick 123456 Extract a generic Button class from the DownloadButton one
> +pick 654321 Use the Button class for all buttons
> +label refactor-button
> +
> +# Branch: report-a-bug
> +reset refactor-button # Use the Button class for all buttons
> +pick abcdef Add the feedback button
> +label report-a-bug
> +
> +reset onto
> +merge -C a1b2c3 refactor-button # Merge 'refactor-button'
> +merge -C 6f5e4d report-a-bug # Merge 'report-a-bug'
> +------------
> +
> +In contrast to a regular interactive rebase, there are `label`, `reset` and
> +`merge` commands in addition to `pick` ones.
> +
> +The `label` command puts a label to whatever will be the current

s/puts a label to/associates a label with/ would be clearer I think. 
Maybe s/whatever will be the current revision/the current HEAD/ an well?

> +revision when that command is executed. Internally, these labels are
> +worktree-local refs that will be deleted when the rebase finishes or
> +when it is aborted.

I agree they should be deleted when the rebase is aborted but I cannot 
see any changes to git-rebase.sh to make that happen. I think they 
should also be deleted by 'rebase --quit'.

> That way, rebase operations in multiple worktrees
> +linked to the same repository do not interfere with one another.
> +
> +The `reset` command is essentially a `git reset --hard` to the specified
> +revision (typically a previously-labeled one).

s/labeled/labelled/

I think it would be worthwhile to point out that unlike the other 
commands this will not preserve untracked files. Maybe something like
"Note that unlike the `pick` or `merge` commands or initial checkout 
when the rebase starts the `reset` command will overwrite any untracked 
files."


Best Wishes

Phillip

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 04/15] sequencer: introduce new commands to reset the revision
  2018-04-11 11:36             ` Johannes Schindelin
@ 2018-04-11 16:07               ` Sergey Organov
  0 siblings, 0 replies; 412+ messages in thread
From: Sergey Organov @ 2018-04-11 16:07 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Sergey,
>
> On Wed, 11 Apr 2018, Sergey Organov wrote:
>
>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> 
>> [...]
>> 
>> > We disallow '#' as label because that character will be used as
>> > separator in the upcoming `merge` command.
>> 
>> Please consider to use # not only in `merge` and `reset`, but in the
>> rest of the commands as well, to unify this new syntax. I.e., right now
>> it seems to be:
>> 
>> pick  abcd A commit message
>> merge beaf # B commit message
>> 
>> I suggest to turn it to:
>> 
>> pick  abcd # A commit message
>> merge beaf # B commit message
>
> First of all, that alignment of pick's and merge's first arguments?

As if it has anything to do with the topic of the issue!

Just a nice look. Let it be:

pick abcd # A commit message
merge beaf # B commit message

if it's that essential indeed.

> That does not exist. If you want aligned arguments, you have to use the
> rebase.abbreviateCommands feature.

It's changing the subject.

> Second: this change would break backwards-compatibility. For almost eleven
> years, we generated `pick abcdef0123 A commit message`.

I thought we already agreed that you have no backward compatibility
issues with this new feature, as it's a new feature, complete re-design,
as you put it yourself:

"This design flaw cannot be fixed. Not without a complete re-design, at
least. This patch series offers such a re-design."

At least could you please answer plain yes/no to this simple question: is
this feature a complete re-design or not? yes/no, please!

> Even if there are no scripts that rely on this form, power users have
> gotten used to it, and I can tell you from experience how unsettling
> even minor visual changes are in everyday operations.
> In short: no, we cannot do that.

You can do that, provided it's complete re-design indeed. You don't wish
to, but you can. Nothing will break and things will be at least a little
bit cleaner.

Each directive having its own dedicated syntax... gosh! No luck getting
syntax description, I'm afraid.

> Just like your proposal to conflate the `merge` and `pick` commands

There was never such proposal. The proposal was not to introduce new
`merge` command when there is already `pick` that could simply be
extended to pick any commit, whatever number of parents it happens to
have.

But provided you decline to even put a # before the commit message...
that proposal is simply a pie in the sky.

> for some perception of consistency: The user experience is more
> important than individual persons' sense of elegance (that might not
> even be shared with the majority).

It's about consistency indeed. Consistent handling of commits is
essential. Consistency is one of the things that bring positive user
experience. You disagree?

Besides, it was bad user experience that forced you to re-design, isn't
it? I'm afraid you miss good opportunity to fix some of your former
mistakes and you make some new. As the discussion goes, it seems you'd
never admit it, the design is set in stone, and my attempts are in fact
pointless.

Overall, I hereby withdraw all my pending suggestions to improve this
patch series.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page
  2018-04-11 15:35           ` Phillip Wood
@ 2018-04-11 19:10             ` Eric Sunshine
  2018-04-12  9:00               ` Johannes Schindelin
  2018-04-13 15:21               ` Phillip Wood
  2018-04-12  9:30             ` Johannes Schindelin
  1 sibling, 2 replies; 412+ messages in thread
From: Eric Sunshine @ 2018-04-11 19:10 UTC (permalink / raw)
  To: Phillip Wood
  Cc: Johannes Schindelin, Git List, Junio C Hamano, Jacob Keller,
	Stefan Beller, Philip Oakley, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

On Wed, Apr 11, 2018 at 11:35 AM, Phillip Wood
<phillip.wood@talktalk.net> wrote:
> On 10/04/18 13:30, Johannes Schindelin wrote:
>> +The `reset` command is essentially a `git reset --hard` to the specified
>> +revision (typically a previously-labeled one).
>
> s/labeled/labelled/

American vs. British English spelling.

CodingGuidelines and SubmittingPatches talk about this. Junio
summarizes the issue well in [1]. The TL;DR is to lean toward the
American English spelling.

[1]: https://public-inbox.org/git/xmqq4m9gpebm.fsf@gitster.mtv.corp.google.com/

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-11 13:13                 ` Sergey Organov
@ 2018-04-11 20:40                   ` Johannes Schindelin
  2018-04-12  8:34                     ` Sergey Organov
  2018-04-11 23:52                   ` Jacob Keller
  1 sibling, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-11 20:40 UTC (permalink / raw)
  To: Sergey Organov
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt

Hi Sergey,

On Wed, 11 Apr 2018, Sergey Organov wrote:

> The RFC v2 and Phillip's strategy are essentially the same, as has been
> already shown multiple times, both theoretically and by testing.

No, they are not.

I am really tired of repeating myself here, as I have demonstrated it at
length, at least half a dozen times, that they are *not* in practice the
same.

If you had played through the example as I suggested, you would actually
see where the differences are, and that your proposal is simply
impractical.

And you would see that Phillip's strategy is better, but I get the
impression that you want to avoid that insight at all cost.

Ciao,
Johannes



^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-11 13:13                 ` Sergey Organov
  2018-04-11 20:40                   ` Johannes Schindelin
@ 2018-04-11 23:52                   ` Jacob Keller
  2018-04-12  5:42                     ` Sergey Organov
  1 sibling, 1 reply; 412+ messages in thread
From: Jacob Keller @ 2018-04-11 23:52 UTC (permalink / raw)
  To: Sergey Organov
  Cc: Johannes Schindelin, Git mailing list, Junio C Hamano,
	Stefan Beller, Philip Oakley, Eric Sunshine, Phillip Wood,
	Igor Djordjevic, Johannes Sixt

On Wed, Apr 11, 2018 at 6:13 AM, Sergey Organov <sorganov@gmail.com> wrote:
> It was rather --recreate-merges just a few weeks ago, and I've seen
> nobody actually commented either in favor or against the
> --rebase-merges.
>
> git rebase --rebase-merges
>

I'm going to jump in here and say that *I* prefer --rebase-merges, as
it clearly mentions merge commits (which is the thing that changes).

I hadn't mentioned this before, because it was a suggestion that
someone else made and it seemed that Johannes liked it, so I didn't
think further discussion was worthwhile.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-11 23:52                   ` Jacob Keller
@ 2018-04-12  5:42                     ` Sergey Organov
  2018-04-12 17:03                       ` Jacob Keller
  0 siblings, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-04-12  5:42 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Johannes Schindelin, Git mailing list, Junio C Hamano,
	Stefan Beller, Philip Oakley, Eric Sunshine, Phillip Wood,
	Igor Djordjevic, Johannes Sixt

Hi Jacob,

Jacob Keller <jacob.keller@gmail.com> writes:
> On Wed, Apr 11, 2018 at 6:13 AM, Sergey Organov <sorganov@gmail.com> wrote:
>> It was rather --recreate-merges just a few weeks ago, and I've seen
>> nobody actually commented either in favor or against the
>> --rebase-merges.
>>
>> git rebase --rebase-merges
>>
>
> I'm going to jump in here and say that *I* prefer --rebase-merges, as
> it clearly mentions merge commits (which is the thing that changes).

OK, thanks, it's fair and the first argument in favor of --rebase-merges
I see.

I don't get why this detail matters so much it should be reflected in
the option name, and if it is what matters most, why the patch series
are not headed:

<twisted quote>
rebase -i: offer to rebase merge commits.

Once upon a time, I dreamt of an interactive rebase that would not
drop merge commits, but instead rebase them.
</twisted quote>

> I hadn't mentioned this before, because it was a suggestion that
> someone else made and it seemed that Johannes liked it, so I didn't
> think further discussion was worthwhile.

So you guys seem to be winning 2:1, or even 3:1, counting the guy who
made the suggestion. Except it was Buga's suggestion [1], and I believe
I was able to convince him that something like --no-flatten would be
better [2]:

<quote>
> I hope he'd be pleased to be able to say --no-flatten=remerge and get
> back his current mode of operation, that he obviously has a good use
> for.

Makes sense, I like it, thanks for elaborating. [ Especially that you 
used "(no) flatten" phrasing, where original `--preserve-merges` 
documentation says it`s used "not to flatten the history", nice touch
;) ]
</quote>

So I assume it's 2:2 by now, with the author of original suggestion on
my side.

I still find

git rebase --rebase-merges

both being ugly and missing the point.

When I look at it with a fresh eye, the questions that immediately rise
are: "What the hell else could 'git _rebase_' do with (merge) commits
but _rebase_ them? Why do I even need to specify this option? Should I
also specify --rebase-non-merges to rebase the rest of commits?"

Well, if it was called something like --[no-]keep-merges, it'd make more
sense as it'd be obvious that alternative is to drop merges (notice how
the old --preserve-merges does match this criteria). However, it'd still
miss to reflect the generic intent of the patch series, -- to preserve
history shape as much as possible, -- now citing author's head message
non-twisted: 

<quote>
rebase -i: offer to recreate commit topology

Once upon a time, I dreamt of an interactive rebase that would not
flatten branch structure, but instead recreate the commit topology
faithfully.
</quote>

-- Sergey

[1] https://public-inbox.org/git/bc9f82fb-fd18-ee45-36a4-921a1381b32e@gmail.com/
[2] https://public-inbox.org/git/a3d40dca-f508-5853-89bc-1f9ab393416b@gmail.com/

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-11 20:40                   ` Johannes Schindelin
@ 2018-04-12  8:34                     ` Sergey Organov
  2018-04-12 12:31                       ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-04-12  8:34 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi Sergey,
>
> On Wed, 11 Apr 2018, Sergey Organov wrote:
>
>> The RFC v2 and Phillip's strategy are essentially the same, as has been
>> already shown multiple times, both theoretically and by testing.
>
> No, they are not.

It's off-topic here. If you _really_ want to discuss it further, you are
still welcome to come back to where you ran away from and continue:

https://public-inbox.org/git/87po3oddl1.fsf@javad.com/

Abrupt change of the topic of discussion indicates your intention to
take attention off the apparent ugliness of 

git rebase --rebase-merges

I also get it as an indication that there are no more arguments in favor
of --rebase-merges on your side, at least for now.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page
  2018-04-11 19:10             ` Eric Sunshine
@ 2018-04-12  9:00               ` Johannes Schindelin
  2018-04-13 15:21               ` Phillip Wood
  1 sibling, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-12  9:00 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Phillip Wood, Git List, Junio C Hamano, Jacob Keller,
	Stefan Beller, Philip Oakley, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

Hi Eric & Phillip,

On Wed, 11 Apr 2018, Eric Sunshine wrote:

> On Wed, Apr 11, 2018 at 11:35 AM, Phillip Wood
> <phillip.wood@talktalk.net> wrote:
> > On 10/04/18 13:30, Johannes Schindelin wrote:
> >> +The `reset` command is essentially a `git reset --hard` to the specified
> >> +revision (typically a previously-labeled one).
> >
> > s/labeled/labelled/
> 
> American vs. British English spelling.
> 
> CodingGuidelines and SubmittingPatches talk about this. Junio
> summarizes the issue well in [1]. The TL;DR is to lean toward the
> American English spelling.
> 
> [1]: https://public-inbox.org/git/xmqq4m9gpebm.fsf@gitster.mtv.corp.google.com/

Thanks, I meant to look that up because I was not sure, and now I do not
have to ;-)

No worries, Phillip, I will keep spelling your name with two `l`s. :0)

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page
  2018-04-11 15:35           ` Phillip Wood
  2018-04-11 19:10             ` Eric Sunshine
@ 2018-04-12  9:30             ` Johannes Schindelin
  2018-04-12 18:29               ` Jacob Keller
  2018-04-13 15:27               ` Phillip Wood
  1 sibling, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-12  9:30 UTC (permalink / raw)
  To: Phillip Wood
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Igor Djordjevic, Johannes Sixt, Sergey Organov

Hi Phillip,

On Wed, 11 Apr 2018, Phillip Wood wrote:

> On 10/04/18 13:30, Johannes Schindelin wrote:
> 
> Firstly let me say that I think expanding the documentation and having an
> example is an excellent idea.

Thanks! At first, I meant to leave this for others to contribute, but I
think it makes sense for me to describe it, as I do have a little bit of
experience with rebasing merges.

> > +
> > +------------
> > +label onto
> > +
> > +# Branch: refactor-button
> > +reset onto
> > +pick 123456 Extract a generic Button class from the DownloadButton one
> > +pick 654321 Use the Button class for all buttons
> > +label refactor-button
> > +
> > +# Branch: report-a-bug
> > +reset refactor-button # Use the Button class for all buttons
> > +pick abcdef Add the feedback button
> > +label report-a-bug
> > +
> > +reset onto
> > +merge -C a1b2c3 refactor-button # Merge 'refactor-button'
> > +merge -C 6f5e4d report-a-bug # Merge 'report-a-bug'
> > +------------
> > +
> > +In contrast to a regular interactive rebase, there are `label`, `reset` and
> > +`merge` commands in addition to `pick` ones.
> > +
> > +The `label` command puts a label to whatever will be the current
> 
> s/puts a label to/associates a label with/ would be clearer I think. Maybe
> s/whatever will be the current revision/the current HEAD/ an well?

Thanks, I incorporated both changes here.

> > +revision when that command is executed. Internally, these labels are
> > +worktree-local refs that will be deleted when the rebase finishes or
> > +when it is aborted.
> 
> I agree they should be deleted when the rebase is aborted but I cannot see any
> changes to git-rebase.sh to make that happen. I think they should also be
> deleted by 'rebase --quit'.

Oh right! For some reason I thought I already hooked up rebase--helper
--abort when rebase was called with --abort or quit, but I had not managed
yet. I think I will leave this for later, or for GSoC, or something.

In the meantime, I'll just drop the "or when it is aborted.".

> > That way, rebase operations in multiple worktrees
> > +linked to the same repository do not interfere with one another.
> > +
> > +The `reset` command is essentially a `git reset --hard` to the specified
> > +revision (typically a previously-labeled one).
> 
> s/labeled/labelled/

As Eric pointed out, I am using 'murricane spelling here (or is it
speling? Ya never know these days).

> I think it would be worthwhile to point out that unlike the other commands
> this will not preserve untracked files. Maybe something like
> "Note that unlike the `pick` or `merge` commands or initial checkout when the
> rebase starts the `reset` command will overwrite any untracked files."

You know what? You just pointed out a bug in my thinking. Previously, I
thought that this is impossible, that you cannot overwrite untracked files
because we labeled this revision previously, so the only new files to
write by `reset` were tracked files previous. But that forgets `exec` and
`reset` with unlabeled revisions (e.g. for cousins).

So I changed the `reset` command to refuse overwriting untracked files...

Thank you for improving this patch series!
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 14/15] rebase -i: introduce --rebase-merges=[no-]rebase-cousins
  2018-04-10 12:30         ` [PATCH v6 14/15] rebase -i: introduce --rebase-merges=[no-]rebase-cousins Johannes Schindelin
@ 2018-04-12 11:30           ` Sergey Organov
  0 siblings, 0 replies; 412+ messages in thread
From: Sergey Organov @ 2018-04-12 11:30 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

[...]

> ++
> +By default, or when `no-rebase-cousins` was specified, commits which do not
> +have `<upstream>` as direct ancestor will keep their original branch point.

<upstream>

sans quotes, <...> are used without them throughout the manual page.

> +If the `rebase-cousins` mode is turned on, such commits are rebased onto
> +`<upstream>` (or `<onto>`, if specified).

<upstream> (or <newbase>, if --onto is specified).

sans quotes, and there is no <onto> defined.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page
  2018-04-10 12:30         ` [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
  2018-04-10 18:49           ` Martin Ågren
  2018-04-11 15:35           ` Phillip Wood
@ 2018-04-12 11:52           ` Sergey Organov
  2 siblings, 0 replies; 412+ messages in thread
From: Sergey Organov @ 2018-04-12 11:52 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> +------------
> +*   Merge branch 'report-a-bug'
> +|\
> +| * Add the feedback button
> +* | Merge branch 'refactor-button'
> +|\ \
> +| |/
> +| * Use the Button class for all buttons
> +| * Extract a generic Button class from the DownloadButton one
> +------------

Consider to put SHA1s into the diagram, as they are then used in
explanaitions. Hopefully I got them right here:

------------
*   6f5e4d Merge branch 'report-a-bug'
|\
| * abcdef Add the feedback button
* | a1b2c3 Merge branch 'refactor-button'
|\ \
| |/
| * 654321 Use the Button class for all buttons
| * 123456 Extract a generic Button class from the DownloadButton one
------------

Original explanation, just for reference, unchanged:

> +------------
> +label onto
> +
> +# Branch: refactor-button
> +reset onto
> +pick 123456 Extract a generic Button class from the DownloadButton one
> +pick 654321 Use the Button class for all buttons
> +label refactor-button
> +
> +# Branch: report-a-bug
> +reset refactor-button # Use the Button class for all buttons
> +pick abcdef Add the feedback button
> +label report-a-bug
> +
> +reset onto
> +merge -C a1b2c3 refactor-button # Merge 'refactor-button'
> +merge -C 6f5e4d report-a-bug # Merge 'report-a-bug'
> +------------

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-12  8:34                     ` Sergey Organov
@ 2018-04-12 12:31                       ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-12 12:31 UTC (permalink / raw)
  To: Sergey Organov
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt

Hi Sergey,

On Thu, 12 Apr 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > On Wed, 11 Apr 2018, Sergey Organov wrote:
> >
> >> The RFC v2 and Phillip's strategy are essentially the same, as has been
> >> already shown multiple times, both theoretically and by testing.
> >
> > No, they are not.
> 
> It's off-topic here.

Well, you directed the discussion there. So there.

> If you _really_ want to discuss it further [...]

I am always interested in a constructive discussion toward the goal of
making Git better, to improve its user experience, to give users more
powerful options, and to make things easier to use.

I'll let you know when I detect a change in this discussion in that vague
direction.

> Abrupt change of the topic of discussion indicates your intention to
> take attention off the apparent ugliness of 
> 
> git rebase --rebase-merges

If you want to discuss ugly things in Git, that is really an abrupt
diversion, but I would not fault you: there is plenty of that in Git.

As to `git rebase --rebase-merges`? I do not actually find that really
ugly. I find that it says what I want it to say. And after how many people
agreed, I find it rather pointless and time-wasting to discuss this
further. Naming is hard, and you seem to have a knack for coming up with
names that are really terrible. That is why I stopped discussing this with
you.

> I also get it as an indication that there are no more arguments in favor
> of --rebase-merges on your side, at least for now.

You seem to misinterpret your own arguments against --rebase-merges to be
anywhere in the realm of convincing. They are not.

Did I say "flatten history" to you in this discussion? Sure I did. We also
talked about Darcs. About the theory of patches. About the inner workings
of recursive merges. About commit graphs. And topologies. And we threw
around many terms that we know people understand who are deep into the
inner workings of merges and cherry-picks.

Does this mean that we should expose all the terms we used in this
technical discussion to the user interface?

No, it does not. We should not absolutely not do that.

So it is not at all a convincing argument to say "but you said XYZ". *In
this mail thread*. Which is necessarily full of technical lingo.

Also, I am still waiting for something tangible from your side. Something
non-theoretic. Something practical. Something like taking that FAKE_INIT
example at heart, studying it, deducing from it what weaknesses we cannot
tolerate in strategies to "cherry-pick merge commits" or "forward-port
merges" or "re-apply amendments in merge commits" or whatever you want to
call it.

Your suggestions so far are heavily biased by your own preferences, based
in theoretical musings, not in practical examples. I do not see any focus
on the Git user base at large. "What? They don't know what a topology is?"
is a question I could see you asking.

There has been a lot of talk in this mail thread, and the only actual
outcome I see is my own work, and Buga's tireless efforts to test
approaches for their practicality. There is zilch concrete testing from
your side. No implementation of anything. No demonstration what kinds of
merge conflicts are produced, how often they would have to be resolved by
the user. None.

The important thing to keep in mind is that all my efforts here are spent
in order to come up with a feature in Git that empowers users. And I want
this feature to be as usable as possible. And I want it to use as simple
language and option names as possible. That is what I will keep focusing
on, like it or not.

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-12  5:42                     ` Sergey Organov
@ 2018-04-12 17:03                       ` Jacob Keller
  2018-04-12 22:02                         ` Johannes Schindelin
  2018-04-18  5:23                         ` Sergey Organov
  0 siblings, 2 replies; 412+ messages in thread
From: Jacob Keller @ 2018-04-12 17:03 UTC (permalink / raw)
  To: Sergey Organov
  Cc: Johannes Schindelin, Git mailing list, Junio C Hamano,
	Stefan Beller, Philip Oakley, Eric Sunshine, Phillip Wood,
	Igor Djordjevic, Johannes Sixt

On Wed, Apr 11, 2018 at 10:42 PM, Sergey Organov <sorganov@gmail.com> wrote:
> Hi Jacob,
>
> Jacob Keller <jacob.keller@gmail.com> writes:
>> On Wed, Apr 11, 2018 at 6:13 AM, Sergey Organov <sorganov@gmail.com> wrote:
>>> It was rather --recreate-merges just a few weeks ago, and I've seen
>>> nobody actually commented either in favor or against the
>>> --rebase-merges.
>>>
>>> git rebase --rebase-merges
>>>
>>
>> I'm going to jump in here and say that *I* prefer --rebase-merges, as
>> it clearly mentions merge commits (which is the thing that changes).
>
> OK, thanks, it's fair and the first argument in favor of --rebase-merges
> I see.
>

I'd be ok with "--keep-merges" also. I don't like the idea of
"flatten" as it, to me, means that anyone who wants to understand the
option without prior knowledge must immediately read the man page or
they will be confused. Something like "--rebase-merges" at least my
coworkers got it instantly. The same could be said for "--keep-merges"
too, but so far no one I asked said the immediately understood
"--no-flatten".

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page
  2018-04-12  9:30             ` Johannes Schindelin
@ 2018-04-12 18:29               ` Jacob Keller
  2018-04-13 15:27               ` Phillip Wood
  1 sibling, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-04-12 18:29 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Phillip Wood, Git mailing list, Junio C Hamano, Stefan Beller,
	Philip Oakley, Eric Sunshine, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

On Thu, Apr 12, 2018 at 2:30 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>> I think it would be worthwhile to point out that unlike the other commands
>> this will not preserve untracked files. Maybe something like
>> "Note that unlike the `pick` or `merge` commands or initial checkout when the
>> rebase starts the `reset` command will overwrite any untracked files."
>
> You know what? You just pointed out a bug in my thinking. Previously, I
> thought that this is impossible, that you cannot overwrite untracked files
> because we labeled this revision previously, so the only new files to
> write by `reset` were tracked files previous. But that forgets `exec` and
> `reset` with unlabeled revisions (e.g. for cousins).
>
> So I changed the `reset` command to refuse overwriting untracked files...
>
> Thank you for improving this patch series!
> Dscho

Good catch! This could possibly have bitten someone badly.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-12 17:03                       ` Jacob Keller
@ 2018-04-12 22:02                         ` Johannes Schindelin
  2018-04-12 22:14                           ` Jacob Keller
  2018-04-13 15:43                           ` Phillip Wood
  2018-04-18  5:23                         ` Sergey Organov
  1 sibling, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-12 22:02 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Sergey Organov, Git mailing list, Junio C Hamano, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt

Hi Jake,

On Thu, 12 Apr 2018, Jacob Keller wrote:

> On Wed, Apr 11, 2018 at 10:42 PM, Sergey Organov <sorganov@gmail.com> wrote:
> >
> > Jacob Keller <jacob.keller@gmail.com> writes:
> >> On Wed, Apr 11, 2018 at 6:13 AM, Sergey Organov <sorganov@gmail.com> wrote:
> >>> It was rather --recreate-merges just a few weeks ago, and I've seen
> >>> nobody actually commented either in favor or against the
> >>> --rebase-merges.
> >>>
> >>> git rebase --rebase-merges
> >>
> >> I'm going to jump in here and say that *I* prefer --rebase-merges, as
> >> it clearly mentions merge commits (which is the thing that changes).
> >
> > OK, thanks, it's fair and the first argument in favor of
> > --rebase-merges I see.
> 
> I'd be ok with "--keep-merges" also.

My main argument against --keep-merges is that there is no good short
option for it: -k and -m are already taken. And I really like my `git
rebase -kir` now...

A minor argument in favor of `--rebase-merges` vs `--keep-merges` is that
we do not really keep the merge commits, we rewrite them. In the version
as per this here patch series, we really create recursive merges from
scratch.

In the later patch series on which I am working, we use a variation of
Phillip's strategy which can be construed as a generalization of the
cherry-pick to include merges: for a cherry-pick, we perform a 3-way merge
between the commit and HEAD, with the commit's parent commit as merge
base. With Phillip's strategy, we perform a 3-way merge between the merge
commit and HEAD (i.e. the rebased first parent), with the merge commit's
first parent as merge base, followed by a 3-way merge with the rebased
2nd parent (with the original 2nd parent as merge base), etc

However. This strategy, while it performed well in my initial tests (and
in Buga's initial tests, too), *does* involve more than one 3-way merge,
and therefore it risks something very, very nasty: *nested* merge
conflicts.

Now, I did see nested merge conflicts in the past, very rarely, but that
can happen, when two developers criss-cross merge each others' `master`
branch and are really happy to perform invasive changes that our merge
does not deal well with, such as indentation changes.

When rebasing a merge conflict, however, such nested conflicts can happen
relatively easily. Not rare at all.

I found out about this by doing what I keep preaching in this thred:
theory is often very nice *right* until the point where it hits reality,
and then frequently turns really useless, real quickly. Theoretical
musings can therefore be an utter waste of time, unless accompanied by
concrete examples.

To start, I built on the example for an "evil merge" that I gave already
in the very beginning of this insanely chatty thread: if one branch
changes the signature of a function, and a second branch adds a caller to
that function, then by necessity a merge between those two branches has to
change the caller to accommodate the signature change. Otherwise it would
end up in a broken state.

In my `sequencer-shears` branch at https://github.com/dscho/git, I added
this as a test case, where I start out with a main.c containing a single
function called core(). I then create one branch where this function is
renamed to hi(), and another branch where the function caller() is added
that calls core(). Then I merge both, amending the merge commit so that
caller() now calls hi(). So this is the main.c after merging:

	int hi(void) {
		printf("Hello, world!\n");
	}
	/* caller */
	void caller(void) {
		hi();
	}

To create the kind of problems that are all too common in my daily work
(seemingly every time some stable patch in Git for Windows gets
upstreamed, it changes into an incompatible version, causing merge
conflicts, and sometimes not only that... but I digress...), I then added
an "upstream" where some maintainer decided that core() is better called
greeting(), and also that a placeholder function for an event loop should
be added. So in upstream, main.c looks like this:

	int greeting(void) {
		printf("Hello, world!\n");
	}
	/* main event loop */
	void event_loop(void) {
		/* TODO: place holder for now */
	}

Keep in mind: while this is a minimal example of disagreeing changes that
may look unrealistic, in practice this is the exact type of problem I am
dealing with on a daily basis, in Git for Windows as well as in GVFS Git
(which adds a thicket of branches on top of Git for Windows) and with the
MSYS2 runtime (where Git for Windows stacks patches on top of MSYs2, which
in turn maintains their set of patches on top of the Cygwin runtime), and
with BusyBox, and probably other projects I forgot spontaneously. This
makes me convinced that this is the exact type of problem that will
challenge whatever --rebase-merges has to deal with, or better put: what
the user of --rebase-merges will have to deal with.

(If I got a penny for every merge conflict I resolved, where test cases
were appended to files in t/, I'd probably be rich by now. Likewise, the
`const char *` -> `struct object_oid *` conflicts have gotten to a point
where I can resolve them while chatting to somebody.)

Now, rebasing the original patches above (renaming core() to hi(), and
adding caller()) will obviously conflict with those upstream patches
(renaming core() to greeting(), and adding event_loop()). That cannot be
avoided. In the example above, I decided to override upstream's decision
by insisting on the name hi(), and resolving the other merge conflict by
adding *both* event_loop() and caller().

The big trick, now, is to avoid forcing the user to resolve the same
conflicts *again* when the merge commit is rebased. The better we can help
the user here, the more powerful will this mode be.

But here, Phillip's strategy (as implemented by yours truly) runs this
problem:

	int hi(void) {
		printf("Hello, world!\n");
	}
	<<<<<<< intermediate merge
	<<<<<<< HEAD
	/* main event loop */
	void event_loop(void) {
		/* TODO: place holder for now */
	=======
	=======
	}
	>>>>>>> <HASH>... merge head #1
	/* caller */
	void caller(void) {
		hi();
	>>>>>>> <HASH>... original merge
	}

Now, no matter who I ask, everybody so far agreed with me that this looks
bad. Like, really bad. There are two merge conflicts, obviously, but it is
not even clear which conflict markers belong together!

It gets a little better when I take a page out of recursive merge's
playbook, which uses different marker sizes for nested merge conflicts
(which I of course implemented and pushed to `sequencer-shears`, currently
still in an unpolished state):

	int hi(void) {
		printf("Hello, world!\n");
	}
	<<<<<<< intermediate merge
	<<<<<<<< HEAD
	/* main event loop */
	void event_loop(void) {
		/* TODO: place holder for now */
	========
	=======
	}
	>>>>>>> <HASH>... merge head #1
	/* caller */
	void caller(void) {
		hi();
	>>>>>>>> <HASH>... original merge
	}

At least now we understand which conflict markers belong together. But I
still needed to inspect the intermediate states to understand what is
going on:

After the first 3-way merge (the one between the original merge commit and
HEAD), we have the conflict markers around event_loop() and caller(),
because they had been added into the same spot.

The second 3-way merge would also want to add the event_loop(), but not
caller(), so ideally it should see that event_loop() is already there and
not add any conflict markers. But that is not the case: event_loop() was
added *with conflict markers*.

So those conflict markers in the first 3-way merge *cause* the conflicts
in the second 3-way merge!

And indeed, if we merge the other way round (original merge with 2nd
parent, then with 1st parent), the result looks much better:

	int hi(void) {
		printf("Hello, world!\n");
	}
	/* main event loop */
	void event_loop(void) {
		/* TODO: place holder for now */
	}
	<<<<<<<< HEAD
	========
	/* caller */
	void caller(void) {
		hi();
	}
	>>>>>>>> <HASH>... intermediate merge

So: the order of the 3-way merges does matter.

I did implement this, too, in the `sequencer_shears` branch: if the first
3-way merge causes conflicts, attempt the second one, and if that one is
clean, try merging that merge result into HEAD (forgetting about the first
attempted 3-way merge).

That is still unsatisfying, though, as it is easy to come up with a
main2.c in the above example that requires the *opposite* merge order to
avoid nested conflicts.

The only way out I can see is to implement some sort of "W merge" or
"chandelier merge" that can perform an N-way merge between one revision
and N-1 other revisions (each of the N-1 bringing its own merge base). I
call them "W" or "chandelier" because such a merge can be visualized by
the original merge commit being the center of a chandelier, and each arm
representing one of the N-1 merge heads with their own merge bases.

Similar to the 3-way merge we have implemented in xdiff/xmerge.c, this
"chandelier merge" would then generate the two diffs between merge base
and both merge heads, except not only one time, but N-1 times. It would
then iterate through all hunks ordered by file name and line range. Any
hunk without conflicting changes would be applied as-is, and the remaining
ones be turned into conflicts (handling those chandelier arms first where
both diffs' hunks look identical).

Have I missed any simpler alternative?

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-12 22:02                         ` Johannes Schindelin
@ 2018-04-12 22:14                           ` Jacob Keller
  2018-04-13 12:08                             ` Johannes Schindelin
  2018-04-13 15:43                           ` Phillip Wood
  1 sibling, 1 reply; 412+ messages in thread
From: Jacob Keller @ 2018-04-12 22:14 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Sergey Organov, Git mailing list, Junio C Hamano, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt

On Thu, Apr 12, 2018 at 3:02 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Jake,
>
> On Thu, 12 Apr 2018, Jacob Keller wrote:
>
>> On Wed, Apr 11, 2018 at 10:42 PM, Sergey Organov <sorganov@gmail.com> wrote:
>> >
>> > Jacob Keller <jacob.keller@gmail.com> writes:
>> >> On Wed, Apr 11, 2018 at 6:13 AM, Sergey Organov <sorganov@gmail.com> wrote:
>> >>> It was rather --recreate-merges just a few weeks ago, and I've seen
>> >>> nobody actually commented either in favor or against the
>> >>> --rebase-merges.
>> >>>
>> >>> git rebase --rebase-merges
>> >>
>> >> I'm going to jump in here and say that *I* prefer --rebase-merges, as
>> >> it clearly mentions merge commits (which is the thing that changes).
>> >
>> > OK, thanks, it's fair and the first argument in favor of
>> > --rebase-merges I see.
>>
>> I'd be ok with "--keep-merges" also.
>
> My main argument against --keep-merges is that there is no good short
> option for it: -k and -m are already taken. And I really like my `git
> rebase -kir` now...
>

Right, that's unfortunate.

> A minor argument in favor of `--rebase-merges` vs `--keep-merges` is that
> we do not really keep the merge commits, we rewrite them. In the version
> as per this here patch series, we really create recursive merges from
> scratch.

I also don't have a strong opinion in regards to --keep vs --rebase..

>
> In the later patch series on which I am working, we use a variation of
> Phillip's strategy which can be construed as a generalization of the
> cherry-pick to include merges: for a cherry-pick, we perform a 3-way merge
> between the commit and HEAD, with the commit's parent commit as merge
> base. With Phillip's strategy, we perform a 3-way merge between the merge
> commit and HEAD (i.e. the rebased first parent), with the merge commit's
> first parent as merge base, followed by a 3-way merge with the rebased
> 2nd parent (with the original 2nd parent as merge base), etc
>
> However. This strategy, while it performed well in my initial tests (and
> in Buga's initial tests, too), *does* involve more than one 3-way merge,
> and therefore it risks something very, very nasty: *nested* merge
> conflicts.

Yea, it could. Finding an elegant solution around this would be ideal!
(By elegant, I mean a solution which produces merge conflicts that
users can resolve relatively easily).

>
> Now, I did see nested merge conflicts in the past, very rarely, but that
> can happen, when two developers criss-cross merge each others' `master`
> branch and are really happy to perform invasive changes that our merge
> does not deal well with, such as indentation changes.
>
> When rebasing a merge conflict, however, such nested conflicts can happen
> relatively easily. Not rare at all.

Right. This would be true regardless of what strategy we choose, I think.

>
> I found out about this by doing what I keep preaching in this thred:
> theory is often very nice *right* until the point where it hits reality,
> and then frequently turns really useless, real quickly. Theoretical
> musings can therefore be an utter waste of time, unless accompanied by
> concrete examples.

Agreed.

>
> To start, I built on the example for an "evil merge" that I gave already
> in the very beginning of this insanely chatty thread: if one branch
> changes the signature of a function, and a second branch adds a caller to
> that function, then by necessity a merge between those two branches has to
> change the caller to accommodate the signature change. Otherwise it would
> end up in a broken state.
>
> In my `sequencer-shears` branch at https://github.com/dscho/git, I added
> this as a test case, where I start out with a main.c containing a single
> function called core(). I then create one branch where this function is
> renamed to hi(), and another branch where the function caller() is added
> that calls core(). Then I merge both, amending the merge commit so that
> caller() now calls hi(). So this is the main.c after merging:
>
>         int hi(void) {
>                 printf("Hello, world!\n");
>         }
>         /* caller */
>         void caller(void) {
>                 hi();
>         }
>
> To create the kind of problems that are all too common in my daily work
> (seemingly every time some stable patch in Git for Windows gets
> upstreamed, it changes into an incompatible version, causing merge
> conflicts, and sometimes not only that... but I digress...), I then added
> an "upstream" where some maintainer decided that core() is better called
> greeting(), and also that a placeholder function for an event loop should
> be added. So in upstream, main.c looks like this:
>
>         int greeting(void) {
>                 printf("Hello, world!\n");
>         }
>         /* main event loop */
>         void event_loop(void) {
>                 /* TODO: place holder for now */
>         }
>
> Keep in mind: while this is a minimal example of disagreeing changes that
> may look unrealistic, in practice this is the exact type of problem I am
> dealing with on a daily basis, in Git for Windows as well as in GVFS Git
> (which adds a thicket of branches on top of Git for Windows) and with the
> MSYS2 runtime (where Git for Windows stacks patches on top of MSYs2, which
> in turn maintains their set of patches on top of the Cygwin runtime), and
> with BusyBox, and probably other projects I forgot spontaneously. This
> makes me convinced that this is the exact type of problem that will
> challenge whatever --rebase-merges has to deal with, or better put: what
> the user of --rebase-merges will have to deal with.
>
> (If I got a penny for every merge conflict I resolved, where test cases
> were appended to files in t/, I'd probably be rich by now. Likewise, the
> `const char *` -> `struct object_oid *` conflicts have gotten to a point
> where I can resolve them while chatting to somebody.)
>
> Now, rebasing the original patches above (renaming core() to hi(), and
> adding caller()) will obviously conflict with those upstream patches
> (renaming core() to greeting(), and adding event_loop()). That cannot be
> avoided. In the example above, I decided to override upstream's decision
> by insisting on the name hi(), and resolving the other merge conflict by
> adding *both* event_loop() and caller().
>
> The big trick, now, is to avoid forcing the user to resolve the same
> conflicts *again* when the merge commit is rebased. The better we can help
> the user here, the more powerful will this mode be.
>
> But here, Phillip's strategy (as implemented by yours truly) runs this
> problem:
>
>         int hi(void) {
>                 printf("Hello, world!\n");
>         }
>         <<<<<<< intermediate merge
>         <<<<<<< HEAD
>         /* main event loop */
>         void event_loop(void) {
>                 /* TODO: place holder for now */
>         =======
>         =======
>         }
>         >>>>>>> <HASH>... merge head #1
>         /* caller */
>         void caller(void) {
>                 hi();
>         >>>>>>> <HASH>... original merge
>         }
>
> Now, no matter who I ask, everybody so far agreed with me that this looks
> bad. Like, really bad. There are two merge conflicts, obviously, but it is
> not even clear which conflict markers belong together!
>
> It gets a little better when I take a page out of recursive merge's
> playbook, which uses different marker sizes for nested merge conflicts
> (which I of course implemented and pushed to `sequencer-shears`, currently
> still in an unpolished state):
>
>         int hi(void) {
>                 printf("Hello, world!\n");
>         }
>         <<<<<<< intermediate merge
>         <<<<<<<< HEAD
>         /* main event loop */
>         void event_loop(void) {
>                 /* TODO: place holder for now */
>         ========
>         =======
>         }
>         >>>>>>> <HASH>... merge head #1
>         /* caller */
>         void caller(void) {
>                 hi();
>         >>>>>>>> <HASH>... original merge
>         }
>
> At least now we understand which conflict markers belong together. But I
> still needed to inspect the intermediate states to understand what is
> going on:
>
> After the first 3-way merge (the one between the original merge commit and
> HEAD), we have the conflict markers around event_loop() and caller(),
> because they had been added into the same spot.
>
> The second 3-way merge would also want to add the event_loop(), but not
> caller(), so ideally it should see that event_loop() is already there and
> not add any conflict markers. But that is not the case: event_loop() was
> added *with conflict markers*.
>
> So those conflict markers in the first 3-way merge *cause* the conflicts
> in the second 3-way merge!
>
> And indeed, if we merge the other way round (original merge with 2nd
> parent, then with 1st parent), the result looks much better:
>
>         int hi(void) {
>                 printf("Hello, world!\n");
>         }
>         /* main event loop */
>         void event_loop(void) {
>                 /* TODO: place holder for now */
>         }
>         <<<<<<<< HEAD
>         ========
>         /* caller */
>         void caller(void) {
>                 hi();
>         }
>         >>>>>>>> <HASH>... intermediate merge
>
> So: the order of the 3-way merges does matter.
>
> I did implement this, too, in the `sequencer_shears` branch: if the first
> 3-way merge causes conflicts, attempt the second one, and if that one is
> clean, try merging that merge result into HEAD (forgetting about the first
> attempted 3-way merge).
>
> That is still unsatisfying, though, as it is easy to come up with a
> main2.c in the above example that requires the *opposite* merge order to
> avoid nested conflicts.

I agree, this solution won't work reliably because we can show
examples which fail the opposite way.

>
> The only way out I can see is to implement some sort of "W merge" or
> "chandelier merge" that can perform an N-way merge between one revision
> and N-1 other revisions (each of the N-1 bringing its own merge base). I
> call them "W" or "chandelier" because such a merge can be visualized by
> the original merge commit being the center of a chandelier, and each arm
> representing one of the N-1 merge heads with their own merge bases.
>

I think this approach sounds reasonable.

> Similar to the 3-way merge we have implemented in xdiff/xmerge.c, this
> "chandelier merge" would then generate the two diffs between merge base
> and both merge heads, except not only one time, but N-1 times. It would
> then iterate through all hunks ordered by file name and line range. Any
> hunk without conflicting changes would be applied as-is, and the remaining
> ones be turned into conflicts (handling those chandelier arms first where
> both diffs' hunks look identical).
>
> Have I missed any simpler alternative?

I *think* this would work well if I understand it, but it's difficult
to process without examples.

>
> Ciao,
> Johannes

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 04/15] sequencer: introduce new commands to reset the revision
  2018-04-10 12:29         ` [PATCH v6 04/15] sequencer: introduce new commands to reset the revision Johannes Schindelin
  2018-04-11  6:17           ` Sergey Organov
@ 2018-04-13 10:03           ` Phillip Wood
  2018-04-15 17:17             ` Philip Oakley
  1 sibling, 1 reply; 412+ messages in thread
From: Phillip Wood @ 2018-04-13 10:03 UTC (permalink / raw)
  To: Johannes Schindelin, git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

On 10/04/18 13:29, Johannes Schindelin wrote:
> In the upcoming commits, we will teach the sequencer to rebase merges.
> This will be done in a very different way from the unfortunate design of
> `git rebase --preserve-merges` (which does not allow for reordering
> commits, or changing the branch topology).
> 
> The main idea is to introduce new todo list commands, to support
> labeling the current revision with a given name, resetting the current
> revision to a previous state, and  merging labeled revisions.
> 
> This idea was developed in Git for Windows' Git garden shears (that are
> used to maintain Git for Windows' "thicket of branches" on top of
> upstream Git), and this patch is part of the effort to make it available
> to a wider audience, as well as to make the entire process more robust
> (by implementing it in a safe and portable language rather than a Unix
> shell script).
> 
> This commit implements the commands to label, and to reset to, given
> revisions. The syntax is:
> 
> 	label <name>
> 	reset <name>
> 
> Internally, the `label <name>` command creates the ref
> `refs/rewritten/<name>`. This makes it possible to work with the labeled
> revisions interactively, or in a scripted fashion (e.g. via the todo
> list command `exec`).
> 
> These temporary refs are removed upon sequencer_remove_state(), so that
> even a `git rebase --abort` cleans them up.
> 
> We disallow '#' as label because that character will be used as separator
> in the upcoming `merge` command.
> 
> Later in this patch series, we will mark the `refs/rewritten/` refs as
> worktree-local, to allow for interactive rebases to be run in parallel in
> worktrees linked to the same repository.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

If a label or reset command fails it is likely to be due to a
typo. Rescheduling the command would make it easier for the user to fix
the problem as they can just run 'git rebase --edit-todo'. It also
ensures that the problem has actually been fixed when the rebase
continues. I think you could do it like this

--->8---
From: Phillip Wood <phillip.wood@dunelm.org.uk>
Subject: [PATCH] fixup! sequencer: introduce new commands to reset the revision

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
---
 sequencer.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 809df1ce48..e1b9be7327 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3029,6 +3029,13 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
+		if (res < 0 && (item->command == TODO_LABEL ||
+				item->command == TODO_RESET)) {
+			/* Reschedule */
+			todo_list->current--;
+			save_todo(todo_list, opts);
+			return res;
+		}
 		todo_list->current++;
 		if (res)
 			return res;
-- 
2.17.0

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 05/15] sequencer: introduce the `merge` command
  2018-04-10 12:29         ` [PATCH v6 05/15] sequencer: introduce the `merge` command Johannes Schindelin
@ 2018-04-13 10:12           ` Phillip Wood
  2018-04-13 17:57             ` Phillip Wood
  0 siblings, 1 reply; 412+ messages in thread
From: Phillip Wood @ 2018-04-13 10:12 UTC (permalink / raw)
  To: Johannes Schindelin, git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

On 10/04/18 13:29, Johannes Schindelin wrote:
> +static int do_merge(struct commit *commit, const char *arg, int arg_len,
> +		    int flags, struct replay_opts *opts)
> +{
> +	int run_commit_flags = (flags & TODO_EDIT_MERGE_MSG) ?
> +		EDIT_MSG | VERIFY_MSG : 0;
> +	struct strbuf ref_name = STRBUF_INIT;
> +	struct commit *head_commit, *merge_commit, *i;
> +	struct commit_list *bases, *j, *reversed = NULL;
> +	struct merge_options o;
> +	int merge_arg_len, oneline_offset, ret;
> +	static struct lock_file lock;
> +	const char *p;
> +
> +	oneline_offset = arg_len;
> +	merge_arg_len = strcspn(arg, " \t\n");
> +	p = arg + merge_arg_len;
> +	p += strspn(p, " \t\n");
> +	if (*p == '#' && (!p[1] || isspace(p[1]))) {
> +		p += 1 + strspn(p + 1, " \t\n");
> +		oneline_offset = p - arg;
> +	} else if (p - arg < arg_len)
> +		BUG("octopus merges are not supported yet: '%s'", p);
> +
> +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
> +	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> +	if (!merge_commit) {
> +		/* fall back to non-rewritten ref or commit */
> +		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
> +		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> +	}
> +	if (!merge_commit) {
> +		error(_("could not resolve '%s'"), ref_name.buf);
> +		strbuf_release(&ref_name);
> +		return -1;
> +	}
> +
> +	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> +		return -1;
> +
> +	head_commit = lookup_commit_reference_by_name("HEAD");
> +	if (!head_commit) {
> +		rollback_lock_file(&lock);
> +		return error(_("cannot merge without a current revision"));
> +	}
> +
> +	if (commit) {
> +		const char *message = get_commit_buffer(commit, NULL);
> +		const char *body;
> +		int len;
> +
> +		if (!message) {
> +			rollback_lock_file(&lock);
> +			return error(_("could not get commit message of '%s'"),
> +				     oid_to_hex(&commit->object.oid));
> +		}
> +		write_author_script(message);
> +		find_commit_subject(message, &body);
> +		len = strlen(body);
> +		if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
> +			error_errno(_("could not write '%s'"),
> +				    git_path_merge_msg());
> +			unuse_commit_buffer(commit, message);
> +			rollback_lock_file(&lock);
> +			return -1;
> +		}
> +		unuse_commit_buffer(commit, message);
> +	} else {
> +		struct strbuf buf = STRBUF_INIT;
> +		int len;
> +
> +		strbuf_addf(&buf, "author %s", git_author_info(0));
> +		write_author_script(buf.buf);
> +		strbuf_reset(&buf);
> +
> +		if (oneline_offset < arg_len) {
> +			p = arg + oneline_offset;
> +			len = arg_len - oneline_offset;
> +		} else {
> +			strbuf_addf(&buf, "Merge branch '%.*s'",
> +				    merge_arg_len, arg);
> +			p = buf.buf;
> +			len = buf.len;
> +		}
> +
> +		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
> +			error_errno(_("could not write '%s'"),
> +				    git_path_merge_msg());
> +			strbuf_release(&buf);
> +			rollback_lock_file(&lock);
> +			return -1;
> +		}
> +		strbuf_release(&buf);
> +	}
> +
> +	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
> +		      git_path_merge_head(), 0);
> +	write_message("no-ff", 5, git_path_merge_mode(), 0);
> +
> +	bases = get_merge_bases(head_commit, merge_commit);
> +	for (j = bases; j; j = j->next)
> +		commit_list_insert(j->item, &reversed);
> +	free_commit_list(bases);
> +
> +	read_cache();
> +	init_merge_options(&o);
> +	o.branch1 = "HEAD";
> +	o.branch2 = ref_name.buf;
> +	o.buffer_output = 2;
> +
> +	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
> +	if (!ret)
> +		rerere(opts->allow_rerere_auto);
> +	if (ret <= 0)
> +		fputs(o.obuf.buf, stdout);
> +	strbuf_release(&o.obuf);
> +	if (ret < 0) {
> +		strbuf_release(&ref_name);
> +		rollback_lock_file(&lock);
> +		return error(_("conflicts while merging '%.*s'"),
> +			     merge_arg_len, arg);
> +	}

If there are conflicts then ret == 0 rather than -1

> +
> +	if (active_cache_changed &&
> +	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
> +		strbuf_release(&ref_name);
> +		return error(_("merge: Unable to write new index file"));
> +	}
> +	rollback_lock_file(&lock);
> +
> +	ret = run_git_commit(git_path_merge_msg(), opts, run_commit_flags);

If there were conflicts this will try and run git commit with unmerged
cache entries

> +	strbuf_release(&ref_name);
> +
> +	return ret;
> +}
> +

If the merge fails with an error rather than conflicts then I think it
should be rescheduled as we do for picks that fail with an error. The
patch below does that and also adjusts the logic following the merge so
that it does not call 'git commit' when there are conflicts. I think we
may want to say something about fixing the conflicts and running
'git rebase --continue' as we do for conflicts when picking.

Best Wishes

Phillip

--->8---
From: Phillip Wood <phillip.wood@dunelm.org.uk>
Subject: [PATCH] fixup! sequencer: introduce the `merge` command


Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
---
 sequencer.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index e1b9be7327..511b7fddca 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2807,27 +2807,26 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	o.buffer_output = 2;
 
 	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
-	if (!ret)
-		rerere(opts->allow_rerere_auto);
+	strbuf_release(&ref_name);
 	if (ret <= 0)
 		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
 	if (ret < 0) {
-		strbuf_release(&ref_name);
 		rollback_lock_file(&lock);
-		return error(_("conflicts while merging '%.*s'"),
-			     merge_arg_len, arg);
+		return ret;
 	}
 
 	if (active_cache_changed &&
-	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
-		strbuf_release(&ref_name);
+	    write_locked_index(&the_index, &lock, COMMIT_LOCK))
 		return error(_("merge: Unable to write new index file"));
-	}
 	rollback_lock_file(&lock);
+	if (!ret) {
+		rerere(opts->allow_rerere_auto);
+		error(_("conflicts while merging '%.*s'"), merge_arg_len, arg);
+		return 1;
+	}
 
 	ret = run_git_commit(git_path_merge_msg(), opts, run_commit_flags);
-	strbuf_release(&ref_name);
 
 	return ret;
 }
@@ -3030,7 +3029,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			return error(_("unknown command %d"), item->command);
 
 		if (res < 0 && (item->command == TODO_LABEL ||
-				item->command == TODO_RESET)) {
+				item->command == TODO_RESET ||
+				item->command == TODO_MERGE)) {
 			/* Reschedule */
 			todo_list->current--;
 			save_todo(todo_list, opts);
-- 
2.17.0

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-12 22:14                           ` Jacob Keller
@ 2018-04-13 12:08                             ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-13 12:08 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Sergey Organov, Git mailing list, Junio C Hamano, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt

Hi Jake,

On Thu, 12 Apr 2018, Jacob Keller wrote:

> On Thu, Apr 12, 2018 at 3:02 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> 
> > [... talking about nested merge conflicts ...]
> >
> > The only way out I can see is to implement some sort of "W merge" or
> > "chandelier merge" that can perform an N-way merge between one revision
> > and N-1 other revisions (each of the N-1 bringing its own merge base). I
> > call them "W" or "chandelier" because such a merge can be visualized by
> > the original merge commit being the center of a chandelier, and each arm
> > representing one of the N-1 merge heads with their own merge bases.
> >
> 
> I think this approach sounds reasonable.

... and it would incidentally also offer a saner way to do octopus merges
(so far, an octopus merge that causes merge conflicts causes... huge
pains, as it usually stops in the middle of everything, without a UI to
help with concluding the merge).

> > Similar to the 3-way merge we have implemented in xdiff/xmerge.c, this
> > "chandelier merge" would then generate the two diffs between merge base
> > and both merge heads, except not only one time, but N-1 times. It would
> > then iterate through all hunks ordered by file name and line range. Any
> > hunk without conflicting changes would be applied as-is, and the remaining
> > ones be turned into conflicts (handling those chandelier arms first where
> > both diffs' hunks look identical).
> >
> > Have I missed any simpler alternative?
> 
> I *think* this would work well if I understand it, but it's difficult
> to process without examples.

Well, I am fairly certain about the implementation details (it's been a
while since I contributed xdiff/xmerge.c, and if you ever want to hear the
horrible story how I wrote the initial version in a stopped train in the
middle of the night, just buy me a beer or three, my memory is fresh on
the "simultaneous walking" of the diff hunks).

So it goes somewhat like this. You have two diffs, and for the matter of
the discussion, let's just look at the hunk headers (with 0 context lines,
i.e. -U0):

diff base..HEAD
@@ -10,1 +10,2 @@
@@ -40,2 +41,0 @@

diff base..branch
@@ -8,4 +8,3 @@

So on one side of the merge, we changed line 10 (e.g. wrapping a long
line), and we removed lines 40 and 41.

In the branch we want to merge, lines 8--11 were edited (removing one
line).

The 3-way merge as implemented in xdiff/xmerge.c handles only one file,
and first uses the diff machinery to figure out the hunk headers of both
diffs, then iterates through both diffs. This is the `while (xscr1 &&
xscr2)` loop in `xdl_do_merge()`, and the "scr" stands for "script" as in
"edit script". In other words, `xscr1` refers to the current hunk in the
first diff, and `xscr2` to the one in the second diff.

Inside the loop, we look whether they overlap. If not, the one with the
smaller line numbers is "applied" and we iterate to the next hunk after
that.

If the hunks overlap, we have a look at the respective post images to see
whether both sides of the merge modified that part identically; if they
don't, we create a conflict (and later, we will try to reduce the conflict
by trimming identially-changed lines at both ends of the line range).

Lather, rinse & repeat.

Now, what I have in mind is that we will have not only two diffs' hunks to
look through, but (N-1)*2 (note that if N == 2, it is the exact same thing
as before).

Again, at each iteration, we look for the next hunk among all available
ones, then determine whether it overlaps with any other hunk. If it does
not, we apply it. If it does, we first look whether all overlapping hunks
agree on the post image and if they do: apply the change, otherwise create
a conflict.

How to present such conflicts to the user, though?

The worst case, I think, would be N diverging changes with N-1 agreeing on
a large part of the post image and the remaining post image being
completely different. Imagine, for example, that the original merge
contains a long function hi() that was renamed to greeting() in HEAD, but
replaced by a completely different implementation in the rebased
branch-to-merge. In such a case, this nested conflict would be most
intuitive, methinks:

	<<< intermediate merge
	<<<< HEAD
	greeting()
	====
	hi()
	>>>> original merge
	... /* original function body */
	===
	hi()
	... /* complete rewrite */
	>>> branch

But now that I look at it, it is still hard to parse. *Is* there any good
way to present this conflict?

And then there is the problem that our index really is only prepared for
*three* stages, but we would need N*2-1.

So maybe I am overthinking this and we should stick with the
implementation I have right now (try to merge HEAD and the original merge
first, then merge the rebased 2nd parent if there are no conflicts,
otherwise try the other way round), and simply come up with a *very good*
message to the unfortunate user who encounters this situation?

I am thinking about something along these lines:

	There were conflicts merging the original merge
		deadbee (Merge 'side-branch')
	with its rebased first parent
		b1ab1ab (Rename 'core()' to 'hi()')
	and its rebased second parent
		ceeceec (Call core() in the event loop)

	The intermediate merge(s) are available as
		abcdef6 (intermediate merge)

Maybe that is good enough? Then the user could always try to glean which
amendments in the original merge (if any) were responsible for the
conflicts, and maybe even try to recreate the merge and then apply the
amendments manually... or something else...

I could even imagine that we could come up with more clever fall-back
strategies, such as recreating the original merge with a regular
merge_trees() to see whether it resulted in the same tree, i.e. find out
whether there *were* amendments, and in that case simply recreate a new
merge from scratch.

At some point, though, I should stop spending so much time on something
that may not even happen all that much in practice, I guess... ;-)

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page
  2018-04-11 19:10             ` Eric Sunshine
  2018-04-12  9:00               ` Johannes Schindelin
@ 2018-04-13 15:21               ` Phillip Wood
  1 sibling, 0 replies; 412+ messages in thread
From: Phillip Wood @ 2018-04-13 15:21 UTC (permalink / raw)
  To: Eric Sunshine, Phillip Wood
  Cc: Johannes Schindelin, Git List, Junio C Hamano, Jacob Keller,
	Stefan Beller, Philip Oakley, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

On 11/04/18 20:10, Eric Sunshine wrote:
> On Wed, Apr 11, 2018 at 11:35 AM, Phillip Wood
> <phillip.wood@talktalk.net> wrote:
>> On 10/04/18 13:30, Johannes Schindelin wrote:
>>> +The `reset` command is essentially a `git reset --hard` to the specified
>>> +revision (typically a previously-labeled one).
>>
>> s/labeled/labelled/
> 
> American vs. British English spelling.

Ah, I'd forgotten that the American version only had one 'l'

Thanks

Phillip

> CodingGuidelines and SubmittingPatches talk about this. Junio
> summarizes the issue well in [1]. The TL;DR is to lean toward the
> American English spelling.
> 
> [1]: https://public-inbox.org/git/xmqq4m9gpebm.fsf@gitster.mtv.corp.google.com/
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page
  2018-04-12  9:30             ` Johannes Schindelin
  2018-04-12 18:29               ` Jacob Keller
@ 2018-04-13 15:27               ` Phillip Wood
  1 sibling, 0 replies; 412+ messages in thread
From: Phillip Wood @ 2018-04-13 15:27 UTC (permalink / raw)
  To: Johannes Schindelin, Phillip Wood
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Igor Djordjevic, Johannes Sixt, Sergey Organov

On 12/04/18 10:30, Johannes Schindelin wrote:
> Hi Phillip,
> 
> On Wed, 11 Apr 2018, Phillip Wood wrote:
> 
>> On 10/04/18 13:30, Johannes Schindelin wrote:
>>
>> Firstly let me say that I think expanding the documentation and having an
>> example is an excellent idea.
> 
> Thanks! At first, I meant to leave this for others to contribute, but I
> think it makes sense for me to describe it, as I do have a little bit of
> experience with rebasing merges.
> 
>>> +
>>> +------------
>>> +label onto
>>> +
>>> +# Branch: refactor-button
>>> +reset onto
>>> +pick 123456 Extract a generic Button class from the DownloadButton one
>>> +pick 654321 Use the Button class for all buttons
>>> +label refactor-button
>>> +
>>> +# Branch: report-a-bug
>>> +reset refactor-button # Use the Button class for all buttons
>>> +pick abcdef Add the feedback button
>>> +label report-a-bug
>>> +
>>> +reset onto
>>> +merge -C a1b2c3 refactor-button # Merge 'refactor-button'
>>> +merge -C 6f5e4d report-a-bug # Merge 'report-a-bug'
>>> +------------
>>> +
>>> +In contrast to a regular interactive rebase, there are `label`, `reset` and
>>> +`merge` commands in addition to `pick` ones.
>>> +
>>> +The `label` command puts a label to whatever will be the current
>>
>> s/puts a label to/associates a label with/ would be clearer I think. Maybe
>> s/whatever will be the current revision/the current HEAD/ an well?
> 
> Thanks, I incorporated both changes here.
> 
>>> +revision when that command is executed. Internally, these labels are
>>> +worktree-local refs that will be deleted when the rebase finishes or
>>> +when it is aborted.
>>
>> I agree they should be deleted when the rebase is aborted but I cannot see any
>> changes to git-rebase.sh to make that happen. I think they should also be
>> deleted by 'rebase --quit'.
> 
> Oh right! For some reason I thought I already hooked up rebase--helper
> --abort when rebase was called with --abort or quit, but I had not managed
> yet. I think I will leave this for later, or for GSoC, or something.
> 
> In the meantime, I'll just drop the "or when it is aborted.".
> 
>>> That way, rebase operations in multiple worktrees
>>> +linked to the same repository do not interfere with one another.
>>> +
>>> +The `reset` command is essentially a `git reset --hard` to the specified
>>> +revision (typically a previously-labeled one).
>>
>> s/labeled/labelled/
> 
> As Eric pointed out, I am using 'murricane spelling here (or is it
> speling? Ya never know these days).

:-)

>> I think it would be worthwhile to point out that unlike the other commands
>> this will not preserve untracked files. Maybe something like
>> "Note that unlike the `pick` or `merge` commands or initial checkout when the
>> rebase starts the `reset` command will overwrite any untracked files."
> 
> You know what? You just pointed out a bug in my thinking. Previously, I
> thought that this is impossible, that you cannot overwrite untracked files
> because we labeled this revision previously, so the only new files to
> write by `reset` were tracked files previous. But that forgets `exec` and
> `reset` with unlabeled revisions (e.g. for cousins).
> 
> So I changed the `reset` command to refuse overwriting untracked files...

That sounds like the safest plan

Thanks

Phillip
> 
> Thank you for improving this patch series!
> Dscho
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-12 22:02                         ` Johannes Schindelin
  2018-04-12 22:14                           ` Jacob Keller
@ 2018-04-13 15:43                           ` Phillip Wood
  2018-04-13 23:48                             ` Johannes Schindelin
  1 sibling, 1 reply; 412+ messages in thread
From: Phillip Wood @ 2018-04-13 15:43 UTC (permalink / raw)
  To: Johannes Schindelin, Jacob Keller
  Cc: Sergey Organov, Git mailing list, Junio C Hamano, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt

On 12/04/18 23:02, Johannes Schindelin wrote:
> Hi Jake,
> 
> On Thu, 12 Apr 2018, Jacob Keller wrote:
> 
>> On Wed, Apr 11, 2018 at 10:42 PM, Sergey Organov <sorganov@gmail.com> wrote:
>>>
>>> Jacob Keller <jacob.keller@gmail.com> writes:
>>>> On Wed, Apr 11, 2018 at 6:13 AM, Sergey Organov <sorganov@gmail.com> wrote:
>>>>> It was rather --recreate-merges just a few weeks ago, and I've seen
>>>>> nobody actually commented either in favor or against the
>>>>> --rebase-merges.
>>>>>
>>>>> git rebase --rebase-merges
>>>>
>>>> I'm going to jump in here and say that *I* prefer --rebase-merges, as
>>>> it clearly mentions merge commits (which is the thing that changes).
>>>
>>> OK, thanks, it's fair and the first argument in favor of
>>> --rebase-merges I see.
>>
>> I'd be ok with "--keep-merges" also.
> 
> My main argument against --keep-merges is that there is no good short
> option for it: -k and -m are already taken. And I really like my `git
> rebase -kir` now...
> 
> A minor argument in favor of `--rebase-merges` vs `--keep-merges` is that
> we do not really keep the merge commits, we rewrite them. In the version
> as per this here patch series, we really create recursive merges from
> scratch.
> 
> In the later patch series on which I am working, we use a variation of
> Phillip's strategy which can be construed as a generalization of the
> cherry-pick to include merges: for a cherry-pick, we perform a 3-way merge
> between the commit and HEAD, with the commit's parent commit as merge
> base. With Phillip's strategy, we perform a 3-way merge between the merge
> commit and HEAD (i.e. the rebased first parent), with the merge commit's
> first parent as merge base, followed by a 3-way merge with the rebased
> 2nd parent (with the original 2nd parent as merge base), etc
> 
> However. This strategy, while it performed well in my initial tests (and
> in Buga's initial tests, too), *does* involve more than one 3-way merge,
> and therefore it risks something very, very nasty: *nested* merge
> conflicts.
> 
> Now, I did see nested merge conflicts in the past, very rarely, but that
> can happen, when two developers criss-cross merge each others' `master`
> branch and are really happy to perform invasive changes that our merge
> does not deal well with, such as indentation changes.
> 
> When rebasing a merge conflict, however, such nested conflicts can happen
> relatively easily. Not rare at all.
> 
> I found out about this by doing what I keep preaching in this thred:
> theory is often very nice *right* until the point where it hits reality,
> and then frequently turns really useless, real quickly. Theoretical
> musings can therefore be an utter waste of time, unless accompanied by
> concrete examples.

Exactly (that's one reason I've been keeping a low profile on this 
thread since my initial suggestion - I haven't had time to test out any 
examples). Thanks for taking the time to test out the theory

> To start, I built on the example for an "evil merge" that I gave already
> in the very beginning of this insanely chatty thread: if one branch
> changes the signature of a function, and a second branch adds a caller to
> that function, then by necessity a merge between those two branches has to
> change the caller to accommodate the signature change. Otherwise it would
> end up in a broken state.
> 
> In my `sequencer-shears` branch at https://github.com/dscho/git, I added
> this as a test case, where I start out with a main.c containing a single
> function called core(). I then create one branch where this function is
> renamed to hi(), and another branch where the function caller() is added
> that calls core(). Then I merge both, amending the merge commit so that
> caller() now calls hi(). So this is the main.c after merging:
> 
> 	int hi(void) {
> 		printf("Hello, world!\n");
> 	}
> 	/* caller */
> 	void caller(void) {
> 		hi();
> 	}
> 
> To create the kind of problems that are all too common in my daily work
> (seemingly every time some stable patch in Git for Windows gets
> upstreamed, it changes into an incompatible version, causing merge
> conflicts, and sometimes not only that... but I digress...), I then added
> an "upstream" where some maintainer decided that core() is better called
> greeting(), and also that a placeholder function for an event loop should
> be added. So in upstream, main.c looks like this:
> 
> 	int greeting(void) {
> 		printf("Hello, world!\n");
> 	}
> 	/* main event loop */
> 	void event_loop(void) {
> 		/* TODO: place holder for now */
> 	}
> 
> Keep in mind: while this is a minimal example of disagreeing changes that
> may look unrealistic, in practice this is the exact type of problem I am
> dealing with on a daily basis, in Git for Windows as well as in GVFS Git
> (which adds a thicket of branches on top of Git for Windows) and with the
> MSYS2 runtime (where Git for Windows stacks patches on top of MSYs2, which
> in turn maintains their set of patches on top of the Cygwin runtime), and
> with BusyBox, and probably other projects I forgot spontaneously. This
> makes me convinced that this is the exact type of problem that will
> challenge whatever --rebase-merges has to deal with, or better put: what
> the user of --rebase-merges will have to deal with.
> 
> (If I got a penny for every merge conflict I resolved, where test cases
> were appended to files in t/, I'd probably be rich by now. Likewise, the
> `const char *` -> `struct object_oid *` conflicts have gotten to a point
> where I can resolve them while chatting to somebody.)
> 
> Now, rebasing the original patches above (renaming core() to hi(), and
> adding caller()) will obviously conflict with those upstream patches
> (renaming core() to greeting(), and adding event_loop()). That cannot be
> avoided. In the example above, I decided to override upstream's decision
> by insisting on the name hi(), and resolving the other merge conflict by
> adding *both* event_loop() and caller().
> 
> The big trick, now, is to avoid forcing the user to resolve the same
> conflicts *again* when the merge commit is rebased. The better we can help
> the user here, the more powerful will this mode be.
> 
> But here, Phillip's strategy (as implemented by yours truly) runs this
> problem:
> 
> 	int hi(void) {
> 		printf("Hello, world!\n");
> 	}
> 	<<<<<<< intermediate merge
> 	<<<<<<< HEAD
> 	/* main event loop */
> 	void event_loop(void) {
> 		/* TODO: place holder for now */
> 	=======
> 	=======
> 	}
> 	>>>>>>> <HASH>... merge head #1
> 	/* caller */
> 	void caller(void) {
> 		hi();
> 	>>>>>>> <HASH>... original merge
> 	}
> 
> Now, no matter who I ask, everybody so far agreed with me that this looks
> bad. Like, really bad. There are two merge conflicts, obviously, but it is
> not even clear which conflict markers belong together!
> 
> It gets a little better when I take a page out of recursive merge's
> playbook, which uses different marker sizes for nested merge conflicts
> (which I of course implemented and pushed to `sequencer-shears`, currently
> still in an unpolished state):
> 
> 	int hi(void) {
> 		printf("Hello, world!\n");
> 	}
> 	<<<<<<< intermediate merge
> 	<<<<<<<< HEAD
> 	/* main event loop */
> 	void event_loop(void) {
> 		/* TODO: place holder for now */
> 	========
> 	=======
> 	}
> 	>>>>>>> <HASH>... merge head #1
> 	/* caller */
> 	void caller(void) {
> 		hi();
> 	>>>>>>>> <HASH>... original merge
> 	}
> 
> At least now we understand which conflict markers belong together. But I
> still needed to inspect the intermediate states to understand what is
> going on:
> 
> After the first 3-way merge (the one between the original merge commit and
> HEAD), we have the conflict markers around event_loop() and caller(),
> because they had been added into the same spot.
> 
> The second 3-way merge would also want to add the event_loop(), but not
> caller(), so ideally it should see that event_loop() is already there and
> not add any conflict markers. But that is not the case: event_loop() was
> added *with conflict markers*.
> 
> So those conflict markers in the first 3-way merge *cause* the conflicts
> in the second 3-way merge!
> 
> And indeed, if we merge the other way round (original merge with 2nd
> parent, then with 1st parent), the result looks much better:
> 
> 	int hi(void) {
> 		printf("Hello, world!\n");
> 	}
> 	/* main event loop */
> 	void event_loop(void) {
> 		/* TODO: place holder for now */
> 	}
> 	<<<<<<<< HEAD
> 	========
> 	/* caller */
> 	void caller(void) {
> 		hi();
> 	}
> 	>>>>>>>> <HASH>... intermediate merge
> 
> So: the order of the 3-way merges does matter.
> 
> I did implement this, too, in the `sequencer_shears` branch: if the first
> 3-way merge causes conflicts, attempt the second one, and if that one is
> clean, try merging that merge result into HEAD (forgetting about the first
> attempted 3-way merge).
> 
> That is still unsatisfying, though, as it is easy to come up with a
> main2.c in the above example that requires the *opposite* merge order to
> avoid nested conflicts.
> 
> The only way out I can see is to implement some sort of "W merge" or
> "chandelier merge" that can perform an N-way merge between one revision
> and N-1 other revisions (each of the N-1 bringing its own merge base). I
> call them "W" or "chandelier" because such a merge can be visualized by
> the original merge commit being the center of a chandelier, and each arm
> representing one of the N-1 merge heads with their own merge bases.
> 
> Similar to the 3-way merge we have implemented in xdiff/xmerge.c, this
> "chandelier merge" would then generate the two diffs between merge base
> and both merge heads, except not only one time, but N-1 times. It would
> then iterate through all hunks ordered by file name and line range. Any
> hunk without conflicting changes would be applied as-is, and the remaining
> ones be turned into conflicts (handling those chandelier arms first where
> both diffs' hunks look identical).
> 
> Have I missed any simpler alternative?

Those conflicts certainly look intimidating (and the ones in your later 
reply with the N way merge example still look quite complicated). One 
option would be just to stop and have the user resolve the conflicts 
after each conflicting 3-way merge rather than at the end of all the 
merges. There are some downsides: there would need to be a way to 
explain to the user that this is an intermediate step (and what that 
step was); the code would have to do some book keeping to know where it 
had got to; and it would stop and prompt the user to resolve conflicts 
more often which could be annoying but hopefully they'd be clearer to 
resolve because they weren't nested.

Best Wishes

Phillip

> 
> Ciao,
> Johannes
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 05/15] sequencer: introduce the `merge` command
  2018-04-13 10:12           ` Phillip Wood
@ 2018-04-13 17:57             ` Phillip Wood
  2018-04-14  0:51               ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Phillip Wood @ 2018-04-13 17:57 UTC (permalink / raw)
  To: Johannes Schindelin, git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

On 13/04/18 11:12, Phillip Wood wrote:
> On 10/04/18 13:29, Johannes Schindelin wrote:
>> +static int do_merge(struct commit *commit, const char *arg, int arg_len,
>> +		    int flags, struct replay_opts *opts)
>> +{
>> +	int run_commit_flags = (flags & TODO_EDIT_MERGE_MSG) ?
>> +		EDIT_MSG | VERIFY_MSG : 0;
>> +	struct strbuf ref_name = STRBUF_INIT;
>> +	struct commit *head_commit, *merge_commit, *i;
>> +	struct commit_list *bases, *j, *reversed = NULL;
>> +	struct merge_options o;
>> +	int merge_arg_len, oneline_offset, ret;
>> +	static struct lock_file lock;
>> +	const char *p;
>> +
>> +	oneline_offset = arg_len;
>> +	merge_arg_len = strcspn(arg, " \t\n");
>> +	p = arg + merge_arg_len;
>> +	p += strspn(p, " \t\n");
>> +	if (*p == '#' && (!p[1] || isspace(p[1]))) {
>> +		p += 1 + strspn(p + 1, " \t\n");
>> +		oneline_offset = p - arg;
>> +	} else if (p - arg < arg_len)
>> +		BUG("octopus merges are not supported yet: '%s'", p);
>> +
>> +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
>> +	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
>> +	if (!merge_commit) {
>> +		/* fall back to non-rewritten ref or commit */
>> +		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
>> +		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
>> +	}
>> +	if (!merge_commit) {
>> +		error(_("could not resolve '%s'"), ref_name.buf);
>> +		strbuf_release(&ref_name);
>> +		return -1;
>> +	}
>> +
>> +	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
>> +		return -1;
>> +
>> +	head_commit = lookup_commit_reference_by_name("HEAD");
>> +	if (!head_commit) {
>> +		rollback_lock_file(&lock);
>> +		return error(_("cannot merge without a current revision"));
>> +	}
>> +
>> +	if (commit) {
>> +		const char *message = get_commit_buffer(commit, NULL);
>> +		const char *body;
>> +		int len;
>> +
>> +		if (!message) {
>> +			rollback_lock_file(&lock);
>> +			return error(_("could not get commit message of '%s'"),
>> +				     oid_to_hex(&commit->object.oid));
>> +		}
>> +		write_author_script(message);
>> +		find_commit_subject(message, &body);
>> +		len = strlen(body);
>> +		if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
>> +			error_errno(_("could not write '%s'"),
>> +				    git_path_merge_msg());
>> +			unuse_commit_buffer(commit, message);
>> +			rollback_lock_file(&lock);
>> +			return -1;
>> +		}
>> +		unuse_commit_buffer(commit, message);
>> +	} else {
>> +		struct strbuf buf = STRBUF_INIT;
>> +		int len;
>> +
>> +		strbuf_addf(&buf, "author %s", git_author_info(0));
>> +		write_author_script(buf.buf);
>> +		strbuf_reset(&buf);
>> +
>> +		if (oneline_offset < arg_len) {
>> +			p = arg + oneline_offset;
>> +			len = arg_len - oneline_offset;
>> +		} else {
>> +			strbuf_addf(&buf, "Merge branch '%.*s'",
>> +				    merge_arg_len, arg);
>> +			p = buf.buf;
>> +			len = buf.len;
>> +		}
>> +
>> +		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
>> +			error_errno(_("could not write '%s'"),
>> +				    git_path_merge_msg());
>> +			strbuf_release(&buf);
>> +			rollback_lock_file(&lock);
>> +			return -1;
>> +		}
>> +		strbuf_release(&buf);
>> +	}
>> +
>> +	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
>> +		      git_path_merge_head(), 0);
>> +	write_message("no-ff", 5, git_path_merge_mode(), 0);
>> +
>> +	bases = get_merge_bases(head_commit, merge_commit);
>> +	for (j = bases; j; j = j->next)
>> +		commit_list_insert(j->item, &reversed);
>> +	free_commit_list(bases);
>> +
>> +	read_cache();
>> +	init_merge_options(&o);
>> +	o.branch1 = "HEAD";
>> +	o.branch2 = ref_name.buf;
>> +	o.buffer_output = 2;
>> +
>> +	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
>> +	if (!ret)
>> +		rerere(opts->allow_rerere_auto);
>> +	if (ret <= 0)
>> +		fputs(o.obuf.buf, stdout);
>> +	strbuf_release(&o.obuf);
>> +	if (ret < 0) {
>> +		strbuf_release(&ref_name);
>> +		rollback_lock_file(&lock);
>> +		return error(_("conflicts while merging '%.*s'"),
>> +			     merge_arg_len, arg);
>> +	}
> 
> If there are conflicts then ret == 0 rather than -1
> 
>> +
>> +	if (active_cache_changed &&
>> +	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
>> +		strbuf_release(&ref_name);
>> +		return error(_("merge: Unable to write new index file"));
>> +	}
>> +	rollback_lock_file(&lock);
>> +
>> +	ret = run_git_commit(git_path_merge_msg(), opts, run_commit_flags);
> 
> If there were conflicts this will try and run git commit with unmerged
> cache entries
> 
>> +	strbuf_release(&ref_name);
>> +
>> +	return ret;
>> +}
>> +
> 
> If the merge fails with an error rather than conflicts then I think it
> should be rescheduled as we do for picks that fail with an error. The
> patch below does that and also adjusts the logic following the merge so
> that it does not call 'git commit' when there are conflicts. I think we
> may want to say something about fixing the conflicts and running
> 'git rebase --continue' as we do for conflicts when picking.
> 
> Best Wishes
> 
> Phillip
> 
> --->8---
> From: Phillip Wood <phillip.wood@dunelm.org.uk>
> Subject: [PATCH] fixup! sequencer: introduce the `merge` command
> 
> 
> Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
> ---
>  sequencer.c | 20 ++++++++++----------
>  1 file changed, 10 insertions(+), 10 deletions(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index e1b9be7327..511b7fddca 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -2807,27 +2807,26 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
>  	o.buffer_output = 2;
>  
>  	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
> -	if (!ret)
> -		rerere(opts->allow_rerere_auto);
> +	strbuf_release(&ref_name);
>  	if (ret <= 0)
>  		fputs(o.obuf.buf, stdout);
>  	strbuf_release(&o.obuf);
>  	if (ret < 0) {
> -		strbuf_release(&ref_name);
>  		rollback_lock_file(&lock);
> -		return error(_("conflicts while merging '%.*s'"),
> -			     merge_arg_len, arg);
> +		return ret;
>  	}
>  
>  	if (active_cache_changed &&
> -	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
> -		strbuf_release(&ref_name);
> +	    write_locked_index(&the_index, &lock, COMMIT_LOCK))
>  		return error(_("merge: Unable to write new index file"));
> -	}
>  	rollback_lock_file(&lock);
> +	if (!ret) {
> +		rerere(opts->allow_rerere_auto);
> +		error(_("conflicts while merging '%.*s'"), merge_arg_len, arg);
> +		return 1;
> +	}
>  
>  	ret = run_git_commit(git_path_merge_msg(), opts, run_commit_flags);
> -	strbuf_release(&ref_name);
>  
>  	return ret;
>  }
> @@ -3030,7 +3029,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  			return error(_("unknown command %d"), item->command);
>  
>  		if (res < 0 && (item->command == TODO_LABEL ||
> -				item->command == TODO_RESET)) {
> +				item->command == TODO_RESET ||
> +				item->command == TODO_MERGE)) {

Unfortunately it's not as simple as that - we only want to reschedule if
merge_recursive() fails, not if run_git_commit() does.


>  			/* Reschedule */
>  			todo_list->current--;
>  			save_todo(todo_list, opts);
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-13 15:43                           ` Phillip Wood
@ 2018-04-13 23:48                             ` Johannes Schindelin
  2018-04-19  4:24                               ` Sergey Organov
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-13 23:48 UTC (permalink / raw)
  To: Phillip Wood
  Cc: Jacob Keller, Sergey Organov, Git mailing list, Junio C Hamano,
	Stefan Beller, Philip Oakley, Eric Sunshine, Igor Djordjevic,
	Johannes Sixt

Hi Phillip,

On Fri, 13 Apr 2018, Phillip Wood wrote:

> On 12/04/18 23:02, Johannes Schindelin wrote:
> > 
> > [...]
> > 
> > So: the order of the 3-way merges does matter.
> >
> > [...]
> 
> Those conflicts certainly look intimidating (and the ones in your later
> reply with the N way merge example still look quite complicated). One
> option would be just to stop and have the user resolve the conflicts
> after each conflicting 3-way merge rather than at the end of all the
> merges. There are some downsides: there would need to be a way to
> explain to the user that this is an intermediate step (and what that
> step was); the code would have to do some book keeping to know where it
> had got to; and it would stop and prompt the user to resolve conflicts
> more often which could be annoying but hopefully they'd be clearer to
> resolve because they weren't nested.

I thought about that. But as I pointed out: the order of the merges *does*
matter. Otherwise we force the user to resolve conflicts that they
*already* resolved during this rebase...

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 05/15] sequencer: introduce the `merge` command
  2018-04-13 17:57             ` Phillip Wood
@ 2018-04-14  0:51               ` Johannes Schindelin
  2018-04-18 18:04                 ` Phillip Wood
  2018-04-19 12:03                 ` Johannes Schindelin
  0 siblings, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-14  0:51 UTC (permalink / raw)
  To: Phillip Wood
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Igor Djordjevic, Johannes Sixt, Sergey Organov

Hi Phillip,

On Fri, 13 Apr 2018, Phillip Wood wrote:

> On 13/04/18 11:12, Phillip Wood wrote:
> > @@ -3030,7 +3029,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
> >  			return error(_("unknown command %d"), item->command);
> >  
> >  		if (res < 0 && (item->command == TODO_LABEL ||
> > -				item->command == TODO_RESET)) {
> > +				item->command == TODO_RESET ||
> > +				item->command == TODO_MERGE)) {
> 
> Unfortunately it's not as simple as that - we only want to reschedule if
> merge_recursive() fails, not if run_git_commit() does.

Correct. How about introducing a flag `reschedule` that is passed to
do_label(), do_reset() and do_merge()?

Seeing as do_reset() and do_merge() already have a replay_opts parameter,
we could add a field `needs_rescheduling` and pass the replay_opts also to
do_label().

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 04/15] sequencer: introduce new commands to reset the revision
  2018-04-13 10:03           ` Phillip Wood
@ 2018-04-15 17:17             ` Philip Oakley
  2018-04-18 18:00               ` Phillip Wood
  0 siblings, 1 reply; 412+ messages in thread
From: Philip Oakley @ 2018-04-15 17:17 UTC (permalink / raw)
  To: phillip.wood, Johannes Schindelin, Git List
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Eric Sunshine,
	Phillip Wood, Igor Djordjevic, Johannes Sixt, Sergey Organov

From: "Phillip Wood" <phillip.wood@talktalk.net>
: Friday, April 13, 2018 11:03 AM
> If a label or reset command fails it is likely to be due to a
> typo. Rescheduling the command would make it easier for the user to fix
> the problem as they can just run 'git rebase --edit-todo'. 

Is this worth noting in the command documentation? 
"If the label or reset command fails then fix
the problem by runnning 'git rebase --edit-todo'." ?

Just a thought.

> It also
> ensures that the problem has actually been fixed when the rebase
> continues. I think you could do it like this
> 
<snip>
--
Philip
(also @dunelm, 73-79..)

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-12 17:03                       ` Jacob Keller
  2018-04-12 22:02                         ` Johannes Schindelin
@ 2018-04-18  5:23                         ` Sergey Organov
  1 sibling, 0 replies; 412+ messages in thread
From: Sergey Organov @ 2018-04-18  5:23 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Johannes Schindelin, Git mailing list, Junio C Hamano,
	Stefan Beller, Philip Oakley, Eric Sunshine, Phillip Wood,
	Igor Djordjevic, Johannes Sixt

Hi Jacob,

Jacob Keller <jacob.keller@gmail.com> writes:

> On Wed, Apr 11, 2018 at 10:42 PM, Sergey Organov <sorganov@gmail.com> wrote:
>> Hi Jacob,
>>
>> Jacob Keller <jacob.keller@gmail.com> writes:
>>> On Wed, Apr 11, 2018 at 6:13 AM, Sergey Organov <sorganov@gmail.com> wrote:
>>>> It was rather --recreate-merges just a few weeks ago, and I've seen
>>>> nobody actually commented either in favor or against the
>>>> --rebase-merges.
>>>>
>>>> git rebase --rebase-merges
>>>>
>>>
>>> I'm going to jump in here and say that *I* prefer --rebase-merges, as
>>> it clearly mentions merge commits (which is the thing that changes).
>>
>> OK, thanks, it's fair and the first argument in favor of --rebase-merges
>> I see.
>>
>
> I'd be ok with "--keep-merges" also. I don't like the idea of
> "flatten" as it, to me, means that anyone who wants to understand the
> option without prior knowledge must immediately read the man page or
> they will be confused. Something like "--rebase-merges" at least my
> coworkers got it instantly. The same could be said for "--keep-merges"
> too, but so far no one I asked said the immediately understood
> "--no-flatten".

If they got --rebase-merges instantly, they should already have known
what "rebase" and "merge" mean. If so, they are likely Git users that
are already familiar with "git rebase" and thus at least heard about a
buddy called --preserve-merges. If it's the case indeed, the outcome
you've got was rather predictable, me thinks.

Now, what are the consequences?

When pleasing maximum number of users of --preserve-merges (and probably
--recreate-merges) is number one target of design, while the rest of
issues are secondary, being in favor of --rebase-merges, --keep-merges,
or --<whatever>-merges is only natural indeed.

However, I don't believe meeting user expectations should be the number
one criteria of a good design. Sound technical design should come first,
and meeting user expectations, provided they don't contradict the
design, only second. That's how Git was born, that's how it should
continue to evolve. Going in reverse direction, from user expectations
to design, will give us Bzr, not Git.

In discussing of these patch series though I rather see care for user
expectations or preferences being used as an excuse for questionable
design all the time. That's what actually bothers me much more than
choosing particular names for particular options.

Narrowing back to the topic, don't you see, honestly, that there is
something wrong with:

git rebase --rebase-merges

that is supposedly easy to understand even without referring to the
manual, yet when you do happen to refer to the manual, you suddenly
realize it's not that easy to understand:

--rebase-merges[=(rebase-cousins|no-rebase-cousins)]
	Rebase merge commits instead of flattening the history by replaying
	merges. Merge conflict resolutions or manual amendments to merge
	commits are not rebased automatically, but have to be applied
	manually.

???

Please read the description. Actually read as if you never knew what's
all this about.

Why does it use "flattening the history" that is supposedly hard to
understand to explain "--rebase-merges" that is supposedly easy to
understand? How comes? And if it's actually a good explanation, why
didn't author just call the option --no-flatten-history, to match its
description?

Next, what is "replaying merges", exactly? That's explaining one term
with another that has not being explained and sounds being even more
vague.

Further, "Merge conflict resolutions or manual amendments to merge
commits are not rebased automatically, but have to be applied manually."
is mutually exclusive with "Rebase merge commits", making all this even
more messy. A merge commit is just content with multiple parents, and
`git rebase`, by definition, reapplies the changes the content
introduces. Any "amendments" or "resolutions" that could have been
happening (or not) when that commit was being created are entirely
irrelevant.

Further yet it goes with:

"By default, or when `no-rebase-cousins` was specified, commits which do
not have `<upstream>` as direct ancestor will keep their original branch
point."

Really? What does it actually mean? What is "commit branch point",
exactly? What "direct ancestor" means in this context, exactly? Provided
even when I do know what the option actually does, the description looks
wrong, how it could explain anything?

Having all this right here in the patch series, you guys try to convince
me that it should not be fixed? That it meets user expectations? You
know what? I, as a user, have somewhat higher expectations.

Below is my final attempt at actually defining a sane alternative. If
you still find this approach inferior, please feel free to ignore it. I
added "history" at the end of original --no-flatten, as a courtesy to
user expectations, as you seem to prefer more verbose names:

----------

--flatten-history
	Flatten rebased history by reapplying non-merge commits only.
        This is the default.

--no-flatten-history[=<options>]
	Do not flatten rebased history. When this option is specified,
	the original shape of the history being rebased will be
	preserved. <options> is comma-separated list of supported
	options.

The following options are supported:

'merge-heads' - perform a merge of rebased merge heads instead of
rebasing original merge commits. Only commit messages will be taken from
original merge commits in this mode.

'rebase-cousins' - commits which do not have <upstream> as
direct ancestor will not keep their original branch point.

------------

In fact I think that 'rebase-cousins' should be removed as making no
sense, but I've borrowed it from the original anyway, to show how the
concept of this option itself works to support multiple additional
options.

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 04/15] sequencer: introduce new commands to reset the revision
  2018-04-15 17:17             ` Philip Oakley
@ 2018-04-18 18:00               ` Phillip Wood
  0 siblings, 0 replies; 412+ messages in thread
From: Phillip Wood @ 2018-04-18 18:00 UTC (permalink / raw)
  To: Philip Oakley, phillip.wood, Johannes Schindelin, Git List
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Eric Sunshine,
	Igor Djordjevic, Johannes Sixt, Sergey Organov

On 15/04/18 18:17, Philip Oakley wrote:
> From: "Phillip Wood" <phillip.wood@talktalk.net>
> : Friday, April 13, 2018 11:03 AM
>> If a label or reset command fails it is likely to be due to a
>> typo. Rescheduling the command would make it easier for the user to fix
>> the problem as they can just run 'git rebase --edit-todo'. 
> 
> Is this worth noting in the command documentation? "If the label or
> reset command fails then fix
> the problem by runnning 'git rebase --edit-todo'." ?
> 
> Just a thought.

Yes that's a good idea, thanks

>> It also
>> ensures that the problem has actually been fixed when the rebase
>> continues. I think you could do it like this
>>
> <snip>
> -- 
> Philip
> (also @dunelm, 73-79..)
That's a bit before me (94-00) were you there when they were building
the hill colleges and some of the science site?

Best Wishes

Phillip

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 05/15] sequencer: introduce the `merge` command
  2018-04-14  0:51               ` Johannes Schindelin
@ 2018-04-18 18:04                 ` Phillip Wood
  2018-04-19 12:03                 ` Johannes Schindelin
  1 sibling, 0 replies; 412+ messages in thread
From: Phillip Wood @ 2018-04-18 18:04 UTC (permalink / raw)
  To: Johannes Schindelin, Phillip Wood
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Igor Djordjevic, Johannes Sixt, Sergey Organov

On 14/04/18 01:51, Johannes Schindelin wrote:
> Hi Phillip,
> 
> On Fri, 13 Apr 2018, Phillip Wood wrote:
> 
>> On 13/04/18 11:12, Phillip Wood wrote:
>>> @@ -3030,7 +3029,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>>>  			return error(_("unknown command %d"), item->command);
>>>  
>>>  		if (res < 0 && (item->command == TODO_LABEL ||
>>> -				item->command == TODO_RESET)) {
>>> +				item->command == TODO_RESET ||
>>> +				item->command == TODO_MERGE)) {
>>
>> Unfortunately it's not as simple as that - we only want to reschedule if
>> merge_recursive() fails, not if run_git_commit() does.
> 
> Correct. How about introducing a flag `reschedule` that is passed to
> do_label(), do_reset() and do_merge()?

That would work (I was thinking about using return codes but having a
parameter is a better idea). Do you want me to re-roll the fixups or are
you happy to make the changes in your next version?

> 
> Seeing as do_reset() and do_merge() already have a replay_opts parameter,
> we could add a field `needs_rescheduling` and pass the replay_opts also to
> do_label().

I'm slightly wary of putting state in an options structure but maybe it
doesn't matter.

Best Wishes

Phillip

> Ciao,
> Dscho
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-13 23:48                             ` Johannes Schindelin
@ 2018-04-19  4:24                               ` Sergey Organov
  2018-04-19  8:23                                 ` Jacob Keller
  0 siblings, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-04-19  4:24 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Phillip Wood, Jacob Keller, Git mailing list, Junio C Hamano,
	Stefan Beller, Philip Oakley, Eric Sunshine, Igor Djordjevic,
	Johannes Sixt

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Phillip,
>
> On Fri, 13 Apr 2018, Phillip Wood wrote:
>
>> On 12/04/18 23:02, Johannes Schindelin wrote:
>> > 
>> > [...]
>> > 
>> > So: the order of the 3-way merges does matter.
>> >
>> > [...]
>> 
>> Those conflicts certainly look intimidating (and the ones in your later
>> reply with the N way merge example still look quite complicated). One
>> option would be just to stop and have the user resolve the conflicts
>> after each conflicting 3-way merge rather than at the end of all the
>> merges. There are some downsides: there would need to be a way to
>> explain to the user that this is an intermediate step (and what that
>> step was); the code would have to do some book keeping to know where it
>> had got to; and it would stop and prompt the user to resolve conflicts
>> more often which could be annoying but hopefully they'd be clearer to
>> resolve because they weren't nested.
>
> I thought about that. But as I pointed out: the order of the merges *does*
> matter. Otherwise we force the user to resolve conflicts that they
> *already* resolved during this rebase...

How it's relevant to what Phillip suggested? How the order of taking 2
steps, A and B, affects an ability to stop after the first step? It's
still either "A,stop,B" or "B,stop,A", depending on the chosen order.

What's the _actual_ problem here, if any?

-- Sergey

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-19  4:24                               ` Sergey Organov
@ 2018-04-19  8:23                                 ` Jacob Keller
  2018-04-19 11:28                                   ` Sergey Organov
  2018-04-20  8:26                                   ` Johannes Schindelin
  0 siblings, 2 replies; 412+ messages in thread
From: Jacob Keller @ 2018-04-19  8:23 UTC (permalink / raw)
  To: Sergey Organov
  Cc: Johannes Schindelin, Phillip Wood, Git mailing list,
	Junio C Hamano, Stefan Beller, Philip Oakley, Eric Sunshine,
	Igor Djordjevic, Johannes Sixt

On Wed, Apr 18, 2018 at 9:24 PM, Sergey Organov <sorganov@gmail.com> wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
>> Hi Phillip,
>>
>> On Fri, 13 Apr 2018, Phillip Wood wrote:
>>
>>> On 12/04/18 23:02, Johannes Schindelin wrote:
>>> >
>>> > [...]
>>> >
>>> > So: the order of the 3-way merges does matter.
>>> >
>>> > [...]
>>>
>>> Those conflicts certainly look intimidating (and the ones in your later
>>> reply with the N way merge example still look quite complicated). One
>>> option would be just to stop and have the user resolve the conflicts
>>> after each conflicting 3-way merge rather than at the end of all the
>>> merges. There are some downsides: there would need to be a way to
>>> explain to the user that this is an intermediate step (and what that
>>> step was); the code would have to do some book keeping to know where it
>>> had got to; and it would stop and prompt the user to resolve conflicts
>>> more often which could be annoying but hopefully they'd be clearer to
>>> resolve because they weren't nested.
>>
>> I thought about that. But as I pointed out: the order of the merges *does*
>> matter. Otherwise we force the user to resolve conflicts that they
>> *already* resolved during this rebase...
>
> How it's relevant to what Phillip suggested? How the order of taking 2
> steps, A and B, affects an ability to stop after the first step? It's
> still either "A,stop,B" or "B,stop,A", depending on the chosen order.
>
> What's the _actual_ problem here, if any?
>
> -- Sergey

I believe the order of the merges changes which ones cause conflicts,
but it's possible to generate pre-images (i.e. a set of parents to
merge) which cause conflicts regardless of which ordering we pick, so
I'm not sure there is a "best ordering".

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-19  8:23                                 ` Jacob Keller
@ 2018-04-19 11:28                                   ` Sergey Organov
  2018-04-20  8:26                                   ` Johannes Schindelin
  1 sibling, 0 replies; 412+ messages in thread
From: Sergey Organov @ 2018-04-19 11:28 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Johannes Schindelin, Phillip Wood, Git mailing list,
	Junio C Hamano, Stefan Beller, Philip Oakley, Eric Sunshine,
	Igor Djordjevic, Johannes Sixt

Hi Jacob,

Jacob Keller <jacob.keller@gmail.com> writes:

> On Wed, Apr 18, 2018 at 9:24 PM, Sergey Organov <sorganov@gmail.com> wrote:
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>>
>>> Hi Phillip,
>>>
>>> On Fri, 13 Apr 2018, Phillip Wood wrote:
>>>
>>>> On 12/04/18 23:02, Johannes Schindelin wrote:
>>>> >
>>>> > [...]
>>>> >
>>>> > So: the order of the 3-way merges does matter.
>>>> >
>>>> > [...]
>>>>
>>>> Those conflicts certainly look intimidating (and the ones in your later
>>>> reply with the N way merge example still look quite complicated). One
>>>> option would be just to stop and have the user resolve the conflicts
>>>> after each conflicting 3-way merge rather than at the end of all the
>>>> merges. There are some downsides: there would need to be a way to
>>>> explain to the user that this is an intermediate step (and what that
>>>> step was); the code would have to do some book keeping to know where it
>>>> had got to; and it would stop and prompt the user to resolve conflicts
>>>> more often which could be annoying but hopefully they'd be clearer to
>>>> resolve because they weren't nested.
>>>
>>> I thought about that. But as I pointed out: the order of the merges *does*
>>> matter. Otherwise we force the user to resolve conflicts that they
>>> *already* resolved during this rebase...
>>
>> How it's relevant to what Phillip suggested? How the order of taking 2
>> steps, A and B, affects an ability to stop after the first step? It's
>> still either "A,stop,B" or "B,stop,A", depending on the chosen order.
>>
>> What's the _actual_ problem here, if any?
>>
>> -- Sergey
>
> I believe the order of the merges changes which ones cause conflicts,
> but it's possible to generate pre-images (i.e. a set of parents to
> merge) which cause conflicts regardless of which ordering we pick, so
> I'm not sure there is a "best ordering".

I totally agree, but this still does not address the problem of
recursive conflicts, and it's this particular problem that Phillip's
suggestion addresses. Just stop after _every_ conflict and let user
resolve it, whatever the order is. Recursive conflicts are simply
showstoppers. Whatever cleverness is invented to represent them, it will
still outsmart most of the users.

As for your statement, it should be clear the absolute "best ordering"
simply can't exist, as merges are inherently symmetric in the DAG. One
can try all orders in turn and select one that brings less conflicts
though. Comparing conflicts is a problem by itself here. Recursive vs
non-recursive and conflict vs no-conflict are obvious and could be the
only checks adopted, all other cases being considered equal.

If we do select fixed order method, or can't find the best order, the
default order should simply match the natural one, first parent first.
Besides, it's the change to the mainline that is most important for an
actual Git merge, so letting it come first sounds most reasonable.

-- Sergey


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 05/15] sequencer: introduce the `merge` command
  2018-04-14  0:51               ` Johannes Schindelin
  2018-04-18 18:04                 ` Phillip Wood
@ 2018-04-19 12:03                 ` Johannes Schindelin
  1 sibling, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:03 UTC (permalink / raw)
  To: Phillip Wood
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Igor Djordjevic, Johannes Sixt, Sergey Organov

Hi Phillip,

On Sat, 14 Apr 2018, Johannes Schindelin wrote:

> On Fri, 13 Apr 2018, Phillip Wood wrote:
> 
> > On 13/04/18 11:12, Phillip Wood wrote:
> > > @@ -3030,7 +3029,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
> > >  			return error(_("unknown command %d"), item->command);
> > >  
> > >  		if (res < 0 && (item->command == TODO_LABEL ||
> > > -				item->command == TODO_RESET)) {
> > > +				item->command == TODO_RESET ||
> > > +				item->command == TODO_MERGE)) {
> > 
> > Unfortunately it's not as simple as that - we only want to reschedule if
> > merge_recursive() fails, not if run_git_commit() does.
> 
> Correct. How about introducing a flag `reschedule` that is passed to
> do_label(), do_reset() and do_merge()?
> 
> Seeing as do_reset() and do_merge() already have a replay_opts parameter,
> we could add a field `needs_rescheduling` and pass the replay_opts also to
> do_label().

Nevermind, we already use the trick in do_pick_commit() that -1 means:
reschedule, 0 means: success, and 1 means: merge conflicts (don't bother
rescheduling).

It just had slipped my mind; I use the same convention in do_merge() now.

Thank you so much for your review and suggestions. I *think* I
incorporated it all.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v7 00/17] rebase -i: offer to recreate commit topology
  2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
                           ` (15 preceding siblings ...)
  2018-04-10 14:52         ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Sergey Organov
@ 2018-04-19 12:12         ` Johannes Schindelin
  2018-04-19 12:15           ` [PATCH v7 01/17] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
                             ` (17 more replies)
  16 siblings, 18 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:12 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Once upon a time, I dreamt of an interactive rebase that would not
flatten branch structure, but instead recreate the commit topology
faithfully.

My original attempt was --preserve-merges, but that design was so
limited that I did not even enable it in interactive mode.

Subsequently, it *was* enabled in interactive mode, with the predictable
consequences: as the --preserve-merges design does not allow for
specifying the parents of merge commits explicitly, all the new commits'
parents are defined *implicitly* by the previous commit history, and
hence it is *not possible to even reorder commits*.

This design flaw cannot be fixed. Not without a complete re-design, at
least. This patch series offers such a re-design.

Think of --rebase-merges as "--preserve-merges done right". It
introduces new verbs for the todo list, `label`, `reset` and `merge`.
For a commit topology like this:

            A - B - C
              \   /
                D

the generated todo list would look like this:

            # branch D
            pick 0123 A
            label branch-point
            pick 1234 D
            label D

            reset branch-point
            pick 2345 B
            merge -C 3456 D # C

There are more patches in the pipeline, based on this patch series, but
left for later in the interest of reviewable patch series: one mini
series to use the sequencer even for `git rebase -i --root`, and another
one to add support for octopus merges to --rebase-merges. And then one
to allow for rebasing merge commits in a smarter way (this one will need
a bit more work, though, as it can result in very complicated, nested
merge conflicts *very* easily).

Changes since v6:

- Reworded the REBASING MERGES section of the man page a bit (thanks, Martin &
  Phillip!).

- The `reset` todo command now refuses to overwrite untracked files (thanks
  Phillip!).

- The do_merge() function was prevented from leaking memory left and right.

- Added a nice advice for the case when todo commands were rescheduled.

- Refactored the way we get to the original line of any given todo command in
  the todo list, simplifying even existing code to make it a lot more readable.

- Failed `label` and `reset` commands, as well as `merge` that failed before
  even attempting to merge, are now rescheduled automatically (thanks
  Phillip and Philip!).

- The do_merge() function no longer tries to commit when there are merge
  conflicts (thanks Phillip!).

- When do_merge() failed to run the recursive merge, it no longer claims that
  there were conflicts (thanks Phillip!).

- When the merge failed, we now write out the index before giving `rerere` a
  chance (d'oh!).


Johannes Schindelin (15):
  sequencer: avoid using errno clobbered by rollback_lock_file()
  sequencer: make rearrange_squash() a bit more obvious
  sequencer: refactor how original todo list lines are accessed
  sequencer: offer helpful advice when a command was rescheduled
  sequencer: introduce new commands to reset the revision
  # This is a combination of 2 commits. # This is the 1st commit
    message:
  sequencer: fast-forward `merge` commands, if possible
  rebase-helper --make-script: introduce a flag to rebase merges
  rebase: introduce the --rebase-merges option
  sequencer: make refs generated by the `label` command worktree-local
  sequencer: handle post-rewrite for merge commands
  rebase --rebase-merges: avoid "empty merges"
  pull: accept --rebase=merges to recreate the branch topology
  rebase -i: introduce --rebase-merges=[no-]rebase-cousins
  rebase -i --rebase-merges: add a section to the man page

Phillip Wood (1):
  rebase --rebase-merges: add test for --keep-empty

Stefan Beller (1):
  git-rebase--interactive: clarify arguments

 Documentation/config.txt               |   8 +
 Documentation/git-pull.txt             |   5 +-
 Documentation/git-rebase.txt           | 147 ++++-
 builtin/pull.c                         |  14 +-
 builtin/rebase--helper.c               |  13 +-
 builtin/remote.c                       |  18 +-
 contrib/completion/git-completion.bash |   4 +-
 git-rebase--interactive.sh             |  22 +-
 git-rebase.sh                          |  16 +
 refs.c                                 |   3 +-
 sequencer.c                            | 869 +++++++++++++++++++++++--
 sequencer.h                            |   7 +
 t/t3421-rebase-topology-linear.sh      |   1 +
 t/t3430-rebase-merges.sh               | 221 +++++++
 14 files changed, 1288 insertions(+), 60 deletions(-)
 create mode 100755 t/t3430-rebase-merges.sh


base-commit: fe0a9eaf31dd0c349ae4308498c33a5c3794b293
Published-As: https://github.com/dscho/git/releases/tag/recreate-merges-v7
Fetch-It-Via: git fetch https://github.com/dscho/git recreate-merges-v7

Interdiff vs v6:
 diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
 index be946de2efb..0ff83b62821 100644
 --- a/Documentation/git-rebase.txt
 +++ b/Documentation/git-rebase.txt
 @@ -849,14 +849,18 @@ merge -C 6f5e4d report-a-bug # Merge 'report-a-bug'
  In contrast to a regular interactive rebase, there are `label`, `reset` and
  `merge` commands in addition to `pick` ones.
  
 -The `label` command puts a label to whatever will be the current
 -revision when that command is executed. Internally, these labels are
 -worktree-local refs that will be deleted when the rebase finishes or
 -when it is aborted. That way, rebase operations in multiple worktrees
 -linked to the same repository do not interfere with one another.
 -
 -The `reset` command is essentially a `git reset --hard` to the specified
 -revision (typically a previously-labeled one).
 +The `label` command associates a label with the current HEAD when that
 +command is executed. These labels are created as worktree-local refs
 +(`refs/rewritten/<label>`) that will be deleted when the rebase
 +finishes. That way, rebase operations in multiple worktrees linked to
 +the same repository do not interfere with one another. If the `label` command
 +fails, it is rescheduled immediately, with a helpful message how to proceed.
 +
 +The `reset` command is essentially a `git read-tree -m -u` (think: `git
 +reset --hard`, but refusing to overwrite untracked files) to the
 +specified revision (typically a previously-labeled one). If the `reset`
 +command fails, it is rescheduled immediately, with a helpful message how to
 +proceed.
  
  The `merge` command will merge the specified revision into whatever is
  HEAD at that time. With `-C <original-commit>`, the commit message of
 @@ -864,13 +868,16 @@ the specified merge commit will be used. When the `-C` is changed to
  a lower-case `-c`, the message will be opened in an editor after a
  successful merge so that the user can edit the message.
  
 +If a `merge` command fails for any reason other than merge conflicts (i.e.
 +when the merge operation did not even start), it is rescheduled immediately.
 +
  At this time, the `merge` command will *always* use the `recursive`
  merge strategy, with no way to choose a different one. To work around
  this, an `exec` command can be used to call `git merge` explicitly,
  using the fact that the labels are worktree-local refs (the ref
  `refs/rewritten/onto` would correspond to the label `onto`).
  
 -Note: the first command (`reset onto`) labels the revision onto which
 +Note: the first command (`label onto`) labels the revision onto which
  the commits are rebased; The name `onto` is just a convention, as a nod
  to the `--onto` option.
  
 diff --git a/sequencer.c b/sequencer.c
 index 809df1ce484..3c7bb5d3fd8 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -1925,6 +1925,23 @@ static int count_commands(struct todo_list *todo_list)
  	return count;
  }
  
 +static int get_item_line_offset(struct todo_list *todo_list, int index)
 +{
 +	return index < todo_list->nr ?
 +		todo_list->items[index].offset_in_buf : todo_list->buf.len;
 +}
 +
 +static const char *get_item_line(struct todo_list *todo_list, int index)
 +{
 +	return todo_list->buf.buf + get_item_line_offset(todo_list, index);
 +}
 +
 +static int get_item_line_length(struct todo_list *todo_list, int index)
 +{
 +	return get_item_line_offset(todo_list, index + 1)
 +		-  get_item_line_offset(todo_list, index);
 +}
 +
  static ssize_t strbuf_read_file_or_whine(struct strbuf *sb, const char *path)
  {
  	int fd;
 @@ -2299,29 +2316,27 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
  	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
  	if (fd < 0)
  		return error_errno(_("could not lock '%s'"), todo_path);
 -	offset = next < todo_list->nr ?
 -		todo_list->items[next].offset_in_buf : todo_list->buf.len;
 +	offset = get_item_line_offset(todo_list, next);
  	if (write_in_full(fd, todo_list->buf.buf + offset,
  			todo_list->buf.len - offset) < 0)
  		return error_errno(_("could not write to '%s'"), todo_path);
  	if (commit_lock_file(&todo_lock) < 0)
  		return error(_("failed to finalize '%s'"), todo_path);
  
 -	if (is_rebase_i(opts)) {
 -		const char *done_path = rebase_path_done();
 -		int fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
 -		int prev_offset = !next ? 0 :
 -			todo_list->items[next - 1].offset_in_buf;
 +	if (is_rebase_i(opts) && next > 0) {
 +		const char *done = rebase_path_done();
 +		int fd = open(done, O_CREAT | O_WRONLY | O_APPEND, 0666);
 +		int ret = 0;
  
 -		if (fd >= 0 && offset > prev_offset &&
 -		    write_in_full(fd, todo_list->buf.buf + prev_offset,
 -				  offset - prev_offset) < 0) {
 -			close(fd);
 -			return error_errno(_("could not write to '%s'"),
 -					   done_path);
 -		}
 -		if (fd >= 0)
 -			close(fd);
 +		if (fd < 0)
 +			return 0;
 +		if (write_in_full(fd, get_item_line(todo_list, next - 1),
 +				  get_item_line_length(todo_list, next - 1))
 +		    < 0)
 +			ret = error_errno(_("could not write to '%s'"), done);
 +		if (close(fd) < 0)
 +			ret = error_errno(_("failed to finalize '%s'"), done);
 +		return ret;
  	}
  	return 0;
  }
 @@ -2619,7 +2634,6 @@ static int do_reset(const char *name, int len, struct replay_opts *opts)
  	unpack_tree_opts.fn = oneway_merge;
  	unpack_tree_opts.merge = 1;
  	unpack_tree_opts.update = 1;
 -	unpack_tree_opts.reset = 1;
  
  	if (read_cache_unmerged()) {
  		rollback_lock_file(&lock);
 @@ -2671,6 +2685,17 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  	static struct lock_file lock;
  	const char *p;
  
 +	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0) {
 +		ret = -1;
 +		goto leave_merge;
 +	}
 +
 +	head_commit = lookup_commit_reference_by_name("HEAD");
 +	if (!head_commit) {
 +		ret = error(_("cannot merge without a current revision"));
 +		goto leave_merge;
 +	}
 +
  	oneline_offset = arg_len;
  	merge_arg_len = strcspn(arg, " \t\n");
  	p = arg + merge_arg_len;
 @@ -2688,19 +2713,10 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
  		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
  	}
 -	if (!merge_commit) {
 -		error(_("could not resolve '%s'"), ref_name.buf);
 -		strbuf_release(&ref_name);
 -		return -1;
 -	}
  
 -	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
 -		return -1;
 -
 -	head_commit = lookup_commit_reference_by_name("HEAD");
 -	if (!head_commit) {
 -		rollback_lock_file(&lock);
 -		return error(_("cannot merge without a current revision"));
 +	if (!merge_commit) {
 +		ret = error(_("could not resolve '%s'"), ref_name.buf);
 +		goto leave_merge;
  	}
  
  	if (commit) {
 @@ -2709,21 +2725,20 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  		int len;
  
  		if (!message) {
 -			rollback_lock_file(&lock);
 -			return error(_("could not get commit message of '%s'"),
 -				     oid_to_hex(&commit->object.oid));
 +			ret = error(_("could not get commit message of '%s'"),
 +				    oid_to_hex(&commit->object.oid));
 +			goto leave_merge;
  		}
  		write_author_script(message);
  		find_commit_subject(message, &body);
  		len = strlen(body);
 -		if (write_message(body, len, git_path_merge_msg(), 0) < 0) {
 +		ret = write_message(body, len, git_path_merge_msg(), 0);
 +		unuse_commit_buffer(commit, message);
 +		if (ret) {
  			error_errno(_("could not write '%s'"),
  				    git_path_merge_msg());
 -			unuse_commit_buffer(commit, message);
 -			rollback_lock_file(&lock);
 -			return -1;
 +			goto leave_merge;
  		}
 -		unuse_commit_buffer(commit, message);
  	} else {
  		struct strbuf buf = STRBUF_INIT;
  		int len;
 @@ -2742,14 +2757,13 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  			len = buf.len;
  		}
  
 -		if (write_message(p, len, git_path_merge_msg(), 0) < 0) {
 +		ret = write_message(p, len, git_path_merge_msg(), 0);
 +		strbuf_release(&buf);
 +		if (ret) {
  			error_errno(_("could not write '%s'"),
  				    git_path_merge_msg());
 -			strbuf_release(&buf);
 -			rollback_lock_file(&lock);
 -			return -1;
 +			goto leave_merge;
  		}
 -		strbuf_release(&buf);
  	}
  
  	/*
 @@ -2777,10 +2791,10 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  	    !commit->parents->next->next &&
  	    !oidcmp(&commit->parents->next->item->object.oid,
  		    &merge_commit->object.oid)) {
 -		strbuf_release(&ref_name);
  		rollback_lock_file(&lock);
 -		return fast_forward_to(&commit->object.oid,
 -				       &head_commit->object.oid, 0, opts);
 +		ret = fast_forward_to(&commit->object.oid,
 +				      &head_commit->object.oid, 0, opts);
 +		goto leave_merge;
  	}
  
  	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
 @@ -2790,10 +2804,9 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  	bases = get_merge_bases(head_commit, merge_commit);
  	if (bases && !oidcmp(&merge_commit->object.oid,
  			     &bases->item->object.oid)) {
 -		strbuf_release(&ref_name);
 -		rollback_lock_file(&lock);
 +		ret = 0;
  		/* skip merging an ancestor of HEAD */
 -		return 0;
 +		goto leave_merge;
  	}
  
  	for (j = bases; j; j = j->next)
 @@ -2807,28 +2820,40 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  	o.buffer_output = 2;
  
  	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
 -	if (!ret)
 -		rerere(opts->allow_rerere_auto);
  	if (ret <= 0)
  		fputs(o.obuf.buf, stdout);
  	strbuf_release(&o.obuf);
  	if (ret < 0) {
 -		strbuf_release(&ref_name);
 -		rollback_lock_file(&lock);
 -		return error(_("conflicts while merging '%.*s'"),
 -			     merge_arg_len, arg);
 +		error(_("could not even attempt to merge '%.*s'"),
 +		      merge_arg_len, arg);
 +		goto leave_merge;
  	}
 +	/*
 +	 * The return value of merge_recursive() is 1 on clean, and 0 on
 +	 * unclean merge.
 +	 *
 +	 * Let's reverse that, so that do_merge() returns 0 upon success and
 +	 * 1 upon failed merge (keeping the return value -1 for the cases where
 +	 * we will want to reschedule the `merge` command).
 +	 */
 +	ret = !ret;
  
  	if (active_cache_changed &&
  	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
 -		strbuf_release(&ref_name);
 -		return error(_("merge: Unable to write new index file"));
 +		ret = error(_("merge: Unable to write new index file"));
 +		goto leave_merge;
  	}
 +
  	rollback_lock_file(&lock);
 +	if (ret)
 +		rerere(opts->allow_rerere_auto);
 +	else
 +		ret = run_git_commit(git_path_merge_msg(), opts,
 +				     run_commit_flags);
  
 -	ret = run_git_commit(git_path_merge_msg(), opts, run_commit_flags);
 +leave_merge:
  	strbuf_release(&ref_name);
 -
 +	rollback_lock_file(&lock);
  	return ret;
  }
  
 @@ -2922,6 +2947,17 @@ static const char *reflog_message(struct replay_opts *opts,
  	return buf.buf;
  }
  
 +static const char rescheduled_advice[] =
 +N_("Could not execute the todo command\n"
 +"\n"
 +"    %.*s"
 +"\n"
 +"It has been rescheduled; To edit the command before continuing, please\n"
 +"edit the todo list first:\n"
 +"\n"
 +"    git rebase --edit-todo\n"
 +"    git rebase --continue\n");
 +
  static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  {
  	int res = 0;
 @@ -2966,7 +3002,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			res = do_pick_commit(item->command, item->commit,
  					opts, is_final_fixup(todo_list));
  			if (is_rebase_i(opts) && res < 0) {
 -				/* Reschedule */
 +reschedule:
 +				advise(_(rescheduled_advice),
 +				       get_item_line_length(todo_list,
 +							    todo_list->current),
 +				       get_item_line(todo_list,
 +						     todo_list->current));
  				todo_list->current--;
  				if (save_todo(todo_list, opts))
  					return -1;
 @@ -2990,7 +3031,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  					intend_to_amend();
  				return error_failed_squash(item->commit, opts,
  					item->arg_len, item->arg);
 -			} else if (res && is_rebase_i(opts))
 +			} else if (res && is_rebase_i(opts) && item->commit)
  				return res | error_with_patch(item->commit,
  					item->arg, item->arg_len, opts, res,
  					item->command == TODO_REWORD);
 @@ -3016,13 +3057,17 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  				/* `current` will be incremented below */
  				todo_list->current = -1;
  			}
 -		} else if (item->command == TODO_LABEL)
 -			res = do_label(item->arg, item->arg_len);
 -		else if (item->command == TODO_RESET)
 -			res = do_reset(item->arg, item->arg_len, opts);
 -		else if (item->command == TODO_MERGE) {
 +		} else if (item->command == TODO_LABEL) {
 +			if ((res = do_label(item->arg, item->arg_len)))
 +				goto reschedule;
 +		} else if (item->command == TODO_RESET) {
 +			if ((res = do_reset(item->arg, item->arg_len, opts)))
 +				goto reschedule;
 +		} else if (item->command == TODO_MERGE) {
  			res = do_merge(item->commit, item->arg, item->arg_len,
  				       item->flags, opts);
 +			if (res < 0)
 +				goto reschedule;
  			if (item->commit)
  				record_in_rewritten(&item->commit->object.oid,
  						    peek_command(todo_list, 1));
 @@ -4046,8 +4091,7 @@ int skip_unnecessary_picks(void)
  		oid = &item->commit->object.oid;
  	}
  	if (i > 0) {
 -		int offset = i < todo_list.nr ?
 -			todo_list.items[i].offset_in_buf : todo_list.buf.len;
 +		int offset = get_item_line_offset(&todo_list, i);
  		const char *done_path = rebase_path_done();
  
  		fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
 @@ -4227,12 +4271,10 @@ int rearrange_squash(void)
  				continue;
  
  			while (cur >= 0) {
 -				int offset = todo_list.items[cur].offset_in_buf;
 -				int end_offset = cur + 1 < todo_list.nr ?
 -					todo_list.items[cur + 1].offset_in_buf :
 -					todo_list.buf.len;
 -				char *bol = todo_list.buf.buf + offset;
 -				char *eol = todo_list.buf.buf + end_offset;
 +				const char *bol =
 +					get_item_line(&todo_list, cur);
 +				const char *eol =
 +					get_item_line(&todo_list, cur + 1);
  
  				/* replace 'pick', by 'fixup' or 'squash' */
  				command = todo_list.items[cur].command;
 diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
 index ee006810573..f2de7059830 100755
 --- a/t/t3430-rebase-merges.sh
 +++ b/t/t3430-rebase-merges.sh
 @@ -52,25 +52,24 @@ test_expect_success 'setup' '
  	git tag -m H H
  '
  
 -cat >script-from-scratch <<\EOF
 -label onto
 -
 -# onebranch
 -pick G
 -pick D
 -label onebranch
 +test_expect_success 'create completely different structure' '
 +	cat >script-from-scratch <<-\EOF &&
 +	label onto
  
 -# second
 -reset onto
 -pick B
 -label second
 +	# onebranch
 +	pick G
 +	pick D
 +	label onebranch
  
 -reset onto
 -merge -C H second
 -merge onebranch # Merge the topic branch 'onebranch'
 -EOF
 +	# second
 +	reset onto
 +	pick B
 +	label second
  
 -test_expect_success 'create completely different structure' '
 +	reset onto
 +	merge -C H second
 +	merge onebranch # Merge the topic branch '\''onebranch'\''
 +	EOF
  	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
  	test_tick &&
  	git rebase -i -r A &&
 @@ -115,6 +114,17 @@ test_expect_success 'generate correct todo list' '
  	test_cmp expect output
  '
  
 +test_expect_success '`reset` refuses to overwrite untracked files' '
 +	git checkout -b refuse-to-reset &&
 +	test_commit dont-overwrite-untracked &&
 +	git checkout @{-1} &&
 +	: >dont-overwrite-untracked.t &&
 +	echo "reset refs/tags/dont-overwrite-untracked" >script-from-scratch &&
 +	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
 +	test_must_fail git rebase -r HEAD &&
 +	git rebase --abort
 +'
 +
  test_expect_success 'with a branch tip that was cherry-picked already' '
  	git checkout -b already-upstream master &&
  	base="$(git rev-parse --verify HEAD)" &&
-- 
2.17.0.windows.1.4.g7e4058d72e3


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v7 01/17] sequencer: avoid using errno clobbered by rollback_lock_file()
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
@ 2018-04-19 12:15           ` Johannes Schindelin
  2018-04-19 12:18           ` [PATCH v7 02/17] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
                             ` (16 subsequent siblings)
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:15 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

As pointed out in a review of the `--rebase-merges` patch series,
`rollback_lock_file()` clobbers errno. Therefore, we have to report the
error message that uses errno before calling said function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 667f35ebdff..096e6d241e0 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -345,12 +345,14 @@ static int write_message(const void *buf, size_t len, const char *filename,
 	if (msg_fd < 0)
 		return error_errno(_("could not lock '%s'"), filename);
 	if (write_in_full(msg_fd, buf, len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write to '%s'"), filename);
+		return -1;
 	}
 	if (append_eol && write(msg_fd, "\n", 1) < 0) {
+		error_errno(_("could not write eol to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write eol to '%s'"), filename);
+		return -1;
 	}
 	if (commit_lock_file(&msg_file) < 0)
 		return error(_("failed to finalize '%s'"), filename);
@@ -2119,9 +2121,9 @@ static int save_head(const char *head)
 	written = write_in_full(fd, buf.buf, buf.len);
 	strbuf_release(&buf);
 	if (written < 0) {
+		error_errno(_("could not write to '%s'"), git_path_head_file());
 		rollback_lock_file(&head_lock);
-		return error_errno(_("could not write to '%s'"),
-				   git_path_head_file());
+		return -1;
 	}
 	if (commit_lock_file(&head_lock) < 0)
 		return error(_("failed to finalize '%s'"), git_path_head_file());
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 02/17] sequencer: make rearrange_squash() a bit more obvious
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
  2018-04-19 12:15           ` [PATCH v7 01/17] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
@ 2018-04-19 12:18           ` Johannes Schindelin
  2018-04-19 12:19           ` [PATCH v7 03/17] sequencer: refactor how original todo list lines are accessed Johannes Schindelin
                             ` (15 subsequent siblings)
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:18 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

There are some commands that have to be skipped from rearranging by virtue
of not handling any commits.

However, the logic was not quite obvious: it skipped commands based on
their position in the enum todo_command.

Instead, let's make it explicit that we skip all commands that do not
handle any commit. With one exception: the `drop` command, because it,
well, drops the commit and is therefore not eligible to rearranging.

Note: this is a bit academic at the moment because the only time we call
`rearrange_squash()` is directly after generating the todo list, when we
have nothing but `pick` commands anyway.

However, the upcoming `merge` command *will* want to be handled by that
function, and it *can* handle commits.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 096e6d241e0..1ee70d843c1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3393,7 +3393,7 @@ int rearrange_squash(void)
 		struct subject2item_entry *entry;
 
 		next[i] = tail[i] = -1;
-		if (item->command >= TODO_EXEC) {
+		if (!item->commit || item->command == TODO_DROP) {
 			subjects[i] = NULL;
 			continue;
 		}
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 03/17] sequencer: refactor how original todo list lines are accessed
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
  2018-04-19 12:15           ` [PATCH v7 01/17] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
  2018-04-19 12:18           ` [PATCH v7 02/17] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
@ 2018-04-19 12:19           ` Johannes Schindelin
  2018-04-19 12:19           ` [PATCH v7 04/17] sequencer: offer helpful advice when a command was rescheduled Johannes Schindelin
                             ` (14 subsequent siblings)
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:19 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Previously, we did a lot of arithmetic gymnastics to get at the line in
the todo list (as stored in todo_list.buf). This might have been fast,
but only in terms of execution speed, not in terms of developer time.

Let's refactor this to make it a lot easier to read, and hence to
reason about the correctness of the code. It is not performance-critical
code anyway.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 60 ++++++++++++++++++++++++++++++++---------------------
 1 file changed, 36 insertions(+), 24 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 1ee70d843c1..3d0a45ab25a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1870,6 +1870,23 @@ static int count_commands(struct todo_list *todo_list)
 	return count;
 }
 
+static int get_item_line_offset(struct todo_list *todo_list, int index)
+{
+	return index < todo_list->nr ?
+		todo_list->items[index].offset_in_buf : todo_list->buf.len;
+}
+
+static const char *get_item_line(struct todo_list *todo_list, int index)
+{
+	return todo_list->buf.buf + get_item_line_offset(todo_list, index);
+}
+
+static int get_item_line_length(struct todo_list *todo_list, int index)
+{
+	return get_item_line_offset(todo_list, index + 1)
+		-  get_item_line_offset(todo_list, index);
+}
+
 static ssize_t strbuf_read_file_or_whine(struct strbuf *sb, const char *path)
 {
 	int fd;
@@ -2244,29 +2261,27 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
 		return error_errno(_("could not lock '%s'"), todo_path);
-	offset = next < todo_list->nr ?
-		todo_list->items[next].offset_in_buf : todo_list->buf.len;
+	offset = get_item_line_offset(todo_list, next);
 	if (write_in_full(fd, todo_list->buf.buf + offset,
 			todo_list->buf.len - offset) < 0)
 		return error_errno(_("could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
 		return error(_("failed to finalize '%s'"), todo_path);
 
-	if (is_rebase_i(opts)) {
-		const char *done_path = rebase_path_done();
-		int fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
-		int prev_offset = !next ? 0 :
-			todo_list->items[next - 1].offset_in_buf;
+	if (is_rebase_i(opts) && next > 0) {
+		const char *done = rebase_path_done();
+		int fd = open(done, O_CREAT | O_WRONLY | O_APPEND, 0666);
+		int ret = 0;
 
-		if (fd >= 0 && offset > prev_offset &&
-		    write_in_full(fd, todo_list->buf.buf + prev_offset,
-				  offset - prev_offset) < 0) {
-			close(fd);
-			return error_errno(_("could not write to '%s'"),
-					   done_path);
-		}
-		if (fd >= 0)
-			close(fd);
+		if (fd < 0)
+			return 0;
+		if (write_in_full(fd, get_item_line(todo_list, next - 1),
+				  get_item_line_length(todo_list, next - 1))
+		    < 0)
+			ret = error_errno(_("could not write to '%s'"), done);
+		if (close(fd) < 0)
+			ret = error_errno(_("failed to finalize '%s'"), done);
+		return ret;
 	}
 	return 0;
 }
@@ -3297,8 +3312,7 @@ int skip_unnecessary_picks(void)
 		oid = &item->commit->object.oid;
 	}
 	if (i > 0) {
-		int offset = i < todo_list.nr ?
-			todo_list.items[i].offset_in_buf : todo_list.buf.len;
+		int offset = get_item_line_offset(&todo_list, i);
 		const char *done_path = rebase_path_done();
 
 		fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
@@ -3478,12 +3492,10 @@ int rearrange_squash(void)
 				continue;
 
 			while (cur >= 0) {
-				int offset = todo_list.items[cur].offset_in_buf;
-				int end_offset = cur + 1 < todo_list.nr ?
-					todo_list.items[cur + 1].offset_in_buf :
-					todo_list.buf.len;
-				char *bol = todo_list.buf.buf + offset;
-				char *eol = todo_list.buf.buf + end_offset;
+				const char *bol =
+					get_item_line(&todo_list, cur);
+				const char *eol =
+					get_item_line(&todo_list, cur + 1);
 
 				/* replace 'pick', by 'fixup' or 'squash' */
 				command = todo_list.items[cur].command;
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 04/17] sequencer: offer helpful advice when a command was rescheduled
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (2 preceding siblings ...)
  2018-04-19 12:19           ` [PATCH v7 03/17] sequencer: refactor how original todo list lines are accessed Johannes Schindelin
@ 2018-04-19 12:19           ` Johannes Schindelin
  2018-04-19 12:20           ` [PATCH v7 05/17] git-rebase--interactive: clarify arguments Johannes Schindelin
                             ` (13 subsequent siblings)
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:19 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Previously, we did that just magically, and potentially left some users
quite puzzled. Let's err on the safe side instead, telling the user what
is happening, and how they are supposed to continue.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 3d0a45ab25a..01443e0f245 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2555,6 +2555,17 @@ static const char *reflog_message(struct replay_opts *opts,
 	return buf.buf;
 }
 
+static const char rescheduled_advice[] =
+N_("Could not execute the todo command\n"
+"\n"
+"    %.*s"
+"\n"
+"It has been rescheduled; To edit the command before continuing, please\n"
+"edit the todo list first:\n"
+"\n"
+"    git rebase --edit-todo\n"
+"    git rebase --continue\n");
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -2600,6 +2611,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					opts, is_final_fixup(todo_list));
 			if (is_rebase_i(opts) && res < 0) {
 				/* Reschedule */
+				advise(_(rescheduled_advice),
+				       get_item_line_length(todo_list,
+							    todo_list->current),
+				       get_item_line(todo_list,
+						     todo_list->current));
 				todo_list->current--;
 				if (save_todo(todo_list, opts))
 					return -1;
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 05/17] git-rebase--interactive: clarify arguments
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (3 preceding siblings ...)
  2018-04-19 12:19           ` [PATCH v7 04/17] sequencer: offer helpful advice when a command was rescheduled Johannes Schindelin
@ 2018-04-19 12:20           ` Johannes Schindelin
  2018-04-19 12:20           ` [PATCH v7 06/17] sequencer: introduce new commands to reset the revision Johannes Schindelin
                             ` (12 subsequent siblings)
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:20 UTC (permalink / raw)
  To: git
  Cc: Stefan Beller, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

From: Stefan Beller <stefanbeller@gmail.com>

Up to now each command took a commit as its first argument and ignored
the rest of the line (usually the subject of the commit)

Now that we are about to introduce commands that take different
arguments, clarify each command by giving the argument list.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 50323fc2735..e1b865f43f2 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -155,13 +155,13 @@ reschedule_last_action () {
 append_todo_help () {
 	gettext "
 Commands:
-p, pick = use commit
-r, reword = use commit, but edit the commit message
-e, edit = use commit, but stop for amending
-s, squash = use commit, but meld into previous commit
-f, fixup = like \"squash\", but discard this commit's log message
-x, exec = run command (the rest of the line) using shell
-d, drop = remove commit
+p, pick <commit> = use commit
+r, reword <commit> = use commit, but edit the commit message
+e, edit <commit> = use commit, but stop for amending
+s, squash <commit> = use commit, but meld into previous commit
+f, fixup <commit> = like \"squash\", but discard this commit's log message
+x, exec <commit> = run command (the rest of the line) using shell
+d, drop <commit> = remove commit
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 06/17] sequencer: introduce new commands to reset the revision
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (4 preceding siblings ...)
  2018-04-19 12:20           ` [PATCH v7 05/17] git-rebase--interactive: clarify arguments Johannes Schindelin
@ 2018-04-19 12:20           ` Johannes Schindelin
  2018-04-20  9:39             ` Phillip Wood
  2018-04-19 12:20           ` [PATCH v7 07/17] # This is a combination of 2 commits. # This is the 1st commit message: Johannes Schindelin
                             ` (11 subsequent siblings)
  17 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:20 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

In the upcoming commits, we will teach the sequencer to rebase merges.
This will be done in a very different way from the unfortunate design of
`git rebase --preserve-merges` (which does not allow for reordering
commits, or changing the branch topology).

The main idea is to introduce new todo list commands, to support
labeling the current revision with a given name, resetting the current
revision to a previous state, and  merging labeled revisions.

This idea was developed in Git for Windows' Git garden shears (that are
used to maintain Git for Windows' "thicket of branches" on top of
upstream Git), and this patch is part of the effort to make it available
to a wider audience, as well as to make the entire process more robust
(by implementing it in a safe and portable language rather than a Unix
shell script).

This commit implements the commands to label, and to reset to, given
revisions. The syntax is:

	label <name>
	reset <name>

Internally, the `label <name>` command creates the ref
`refs/rewritten/<name>`. This makes it possible to work with the labeled
revisions interactively, or in a scripted fashion (e.g. via the todo
list command `exec`).

These temporary refs are removed upon sequencer_remove_state(), so that
even a `git rebase --abort` cleans them up.

We disallow '#' as label because that character will be used as separator
in the upcoming `merge` command.

Later in this patch series, we will mark the `refs/rewritten/` refs as
worktree-local, to allow for interactive rebases to be run in parallel in
worktrees linked to the same repository.

As typos happen, a failed `label` or `reset` command will be rescheduled
immediately. Note that this needs a little change in the original code to
perform a reschedule: there is no commit from which to generate a patch
here (and we will simply fall through to the regular `return res`). We
keep that code path, though, because we will use it for the upcoming
`merge` command, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   2 +
 sequencer.c                | 201 +++++++++++++++++++++++++++++++++++--
 2 files changed, 196 insertions(+), 7 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index e1b865f43f2..e8d3a7d7588 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -162,6 +162,8 @@ s, squash <commit> = use commit, but meld into previous commit
 f, fixup <commit> = like \"squash\", but discard this commit's log message
 x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
+l, label <label> = label current HEAD with a name
+t, reset <label> = reset HEAD to a label
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index 01443e0f245..9e09026b594 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -23,6 +23,8 @@
 #include "hashmap.h"
 #include "notes-utils.h"
 #include "sigchain.h"
+#include "unpack-trees.h"
+#include "worktree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -120,6 +122,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
 static GIT_PATH_FUNC(rebase_path_rewritten_pending,
 	"rebase-merge/rewritten-pending")
+
+/*
+ * The path of the file listing refs that need to be deleted after the rebase
+ * finishes. This is used by the `label` command to record the need for cleanup.
+ */
+static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
+
 /*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
@@ -244,18 +253,33 @@ static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
 
 int sequencer_remove_state(struct replay_opts *opts)
 {
-	struct strbuf dir = STRBUF_INIT;
+	struct strbuf buf = STRBUF_INIT;
 	int i;
 
+	if (strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
+		char *p = buf.buf;
+		while (*p) {
+			char *eol = strchr(p, '\n');
+			if (eol)
+				*eol = '\0';
+			if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
+				warning(_("could not delete '%s'"), p);
+			if (!eol)
+				break;
+			p = eol + 1;
+		}
+	}
+
 	free(opts->gpg_sign);
 	free(opts->strategy);
 	for (i = 0; i < opts->xopts_nr; i++)
 		free(opts->xopts[i]);
 	free(opts->xopts);
 
-	strbuf_addstr(&dir, get_dir(opts));
-	remove_dir_recursively(&dir, 0);
-	strbuf_release(&dir);
+	strbuf_reset(&buf);
+	strbuf_addstr(&buf, get_dir(opts));
+	remove_dir_recursively(&buf, 0);
+	strbuf_release(&buf);
 
 	return 0;
 }
@@ -1279,6 +1303,8 @@ enum todo_command {
 	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
+	TODO_LABEL,
+	TODO_RESET,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -1297,6 +1323,8 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
+	{ 'l', "label" },
+	{ 't', "reset" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1802,7 +1830,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return error(_("missing arguments for %s"),
 			     command_to_string(item->command));
 
-	if (item->command == TODO_EXEC) {
+	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
+	    item->command == TODO_RESET) {
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = (int)(eol - bol);
@@ -2465,6 +2494,158 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int safe_append(const char *filename, const char *fmt, ...)
+{
+	va_list ap;
+	struct lock_file lock = LOCK_INIT;
+	int fd = hold_lock_file_for_update(&lock, filename,
+					   LOCK_REPORT_ON_ERROR);
+	struct strbuf buf = STRBUF_INIT;
+
+	if (fd < 0)
+		return -1;
+
+	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT) {
+		error_errno(_("could not read '%s'"), filename);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	strbuf_complete(&buf, '\n');
+	va_start(ap, fmt);
+	strbuf_vaddf(&buf, fmt, ap);
+	va_end(ap);
+
+	if (write_in_full(fd, buf.buf, buf.len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
+		strbuf_release(&buf);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	if (commit_lock_file(&lock) < 0) {
+		strbuf_release(&buf);
+		rollback_lock_file(&lock);
+		return error(_("failed to finalize '%s'"), filename);
+	}
+
+	strbuf_release(&buf);
+	return 0;
+}
+
+static int do_label(const char *name, int len)
+{
+	struct ref_store *refs = get_main_ref_store();
+	struct ref_transaction *transaction;
+	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
+	struct strbuf msg = STRBUF_INIT;
+	int ret = 0;
+	struct object_id head_oid;
+
+	if (len == 1 && *name == '#')
+		return error("Illegal label name: '%.*s'", len, name);
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
+
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction) {
+		error("%s", err.buf);
+		ret = -1;
+	} else if (get_oid("HEAD", &head_oid)) {
+		error(_("could not read HEAD"));
+		ret = -1;
+	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
+					  NULL, 0, msg.buf, &err) < 0 ||
+		   ref_transaction_commit(transaction, &err)) {
+		error("%s", err.buf);
+		ret = -1;
+	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
+	strbuf_release(&msg);
+
+	if (!ret)
+		ret = safe_append(rebase_path_refs_to_delete(),
+				  "%s\n", ref_name.buf);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
+static const char *reflog_message(struct replay_opts *opts,
+	const char *sub_action, const char *fmt, ...);
+
+static int do_reset(const char *name, int len, struct replay_opts *opts)
+{
+	struct strbuf ref_name = STRBUF_INIT;
+	struct object_id oid;
+	struct lock_file lock = LOCK_INIT;
+	struct tree_desc desc;
+	struct tree *tree;
+	struct unpack_trees_options unpack_tree_opts;
+	int ret = 0, i;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	/* Determine the length of the label */
+	for (i = 0; i < len; i++)
+		if (isspace(name[i]))
+			len = i;
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	if (get_oid(ref_name.buf, &oid) &&
+	    get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
+		error(_("could not read '%s'"), ref_name.buf);
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
+	unpack_tree_opts.head_idx = 1;
+	unpack_tree_opts.src_index = &the_index;
+	unpack_tree_opts.dst_index = &the_index;
+	unpack_tree_opts.fn = oneway_merge;
+	unpack_tree_opts.merge = 1;
+	unpack_tree_opts.update = 1;
+
+	if (read_cache_unmerged()) {
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return error_resolve_conflict(_(action_name(opts)));
+	}
+
+	if (!fill_tree_descriptor(&desc, &oid)) {
+		error(_("failed to find tree of %s"), oid_to_hex(&oid));
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	if (unpack_trees(1, &desc, &unpack_tree_opts)) {
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	tree = parse_tree_indirect(&oid);
+	prime_cache_tree(&the_index, tree);
+
+	if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
+		ret = error(_("could not write index"));
+	free((void *)desc.buffer);
+
+	if (!ret)
+		ret = update_ref(reflog_message(opts, "reset", "'%.*s'",
+						len, name), "HEAD", &oid,
+				 NULL, 0, UPDATE_REFS_MSG_ON_ERR);
+
+	strbuf_release(&ref_name);
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2610,7 +2791,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_pick_commit(item->command, item->commit,
 					opts, is_final_fixup(todo_list));
 			if (is_rebase_i(opts) && res < 0) {
-				/* Reschedule */
+reschedule:
 				advise(_(rescheduled_advice),
 				       get_item_line_length(todo_list,
 							    todo_list->current),
@@ -2639,7 +2820,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					intend_to_amend();
 				return error_failed_squash(item->commit, opts,
 					item->arg_len, item->arg);
-			} else if (res && is_rebase_i(opts))
+			} else if (res && is_rebase_i(opts) && item->commit)
 				return res | error_with_patch(item->commit,
 					item->arg, item->arg_len, opts, res,
 					item->command == TODO_REWORD);
@@ -2665,6 +2846,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				/* `current` will be incremented below */
 				todo_list->current = -1;
 			}
+		} else if (item->command == TODO_LABEL) {
+			if ((res = do_label(item->arg, item->arg_len)))
+				goto reschedule;
+		} else if (item->command == TODO_RESET) {
+			if ((res = do_reset(item->arg, item->arg_len, opts)))
+				goto reschedule;
 		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 07/17] # This is a combination of 2 commits. # This is the 1st commit message:
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (5 preceding siblings ...)
  2018-04-19 12:20           ` [PATCH v7 06/17] sequencer: introduce new commands to reset the revision Johannes Schindelin
@ 2018-04-19 12:20           ` Johannes Schindelin
  2018-04-20  5:38             ` Eric Sunshine
  2018-04-19 12:21           ` [PATCH v7 08/17] sequencer: fast-forward `merge` commands, if possible Johannes Schindelin
                             ` (10 subsequent siblings)
  17 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:20 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

sequencer: introduce the `merge` command

This patch is part of the effort to reimplement `--preserve-merges` with
a substantially improved design, a design that has been developed in the
Git for Windows project to maintain the dozens of Windows-specific patch
series on top of upstream Git.

The previous patch implemented the `label` and `reset` commands to label
commits and to reset to labeled commits. This patch adds the `merge`
command, with the following syntax:

	merge [-C <commit>] <rev> # <oneline>

The <commit> parameter in this instance is the *original* merge commit,
whose author and message will be used for the merge commit that is about
to be created.

The <rev> parameter refers to the (possibly rewritten) revision to
merge. Let's see an example of a todo list:

	label onto

	# Branch abc
	reset onto
	pick deadbeef Hello, world!
	label abc

	reset onto
	pick cafecafe And now for something completely different
	merge -C baaabaaa abc # Merge the branch 'abc' into master

To edit the merge commit's message (a "reword" for merges, if you will),
use `-c` (lower-case) instead of `-C`; this convention was borrowed from
`git commit` that also supports `-c` and `-C` with similar meanings.

To create *new* merges, i.e. without copying the commit message from an
existing commit, simply omit the `-C <commit>` parameter (which will
open an editor for the merge message):

	merge abc

This comes in handy when splitting a branch into two or more branches.

Note: this patch only adds support for recursive merges, to keep things
simple. Support for octopus merges will be added later in a separate
patch series, support for merges using strategies other than the
recursive merge is left for the future.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

# The commit message #2 will be skipped:

# fixup! sequencer: introduce the `merge` command
---
 git-rebase--interactive.sh |   4 +
 sequencer.c                | 184 +++++++++++++++++++++++++++++++++++++
 2 files changed, 188 insertions(+)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index e8d3a7d7588..ccd5254d1c9 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -164,6 +164,10 @@ x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
 l, label <label> = label current HEAD with a name
 t, reset <label> = reset HEAD to a label
+m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
+.       create a merge commit using the original merge commit's
+.       message (or the oneline, if no original merge commit was
+.       specified). Use -c <commit> to reword the commit message.
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index 9e09026b594..90b2fac96b1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1305,6 +1305,7 @@ enum todo_command {
 	TODO_EXEC,
 	TODO_LABEL,
 	TODO_RESET,
+	TODO_MERGE,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -1325,6 +1326,7 @@ static struct {
 	{ 'x', "exec" },
 	{ 'l', "label" },
 	{ 't', "reset" },
+	{ 'm', "merge" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1752,9 +1754,14 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 	return 0;
 }
 
+enum todo_item_flags {
+	TODO_EDIT_MERGE_MSG = 1
+};
+
 struct todo_item {
 	enum todo_command command;
 	struct commit *commit;
+	unsigned int flags;
 	const char *arg;
 	int arg_len;
 	size_t offset_in_buf;
@@ -1789,6 +1796,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	char *end_of_object_name;
 	int i, saved, status, padding;
 
+	item->flags = 0;
+
 	/* left-trim */
 	bol += strspn(bol, " \t");
 
@@ -1838,6 +1847,21 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return 0;
 	}
 
+	if (item->command == TODO_MERGE) {
+		if (skip_prefix(bol, "-C", &bol))
+			bol += strspn(bol, " \t");
+		else if (skip_prefix(bol, "-c", &bol)) {
+			bol += strspn(bol, " \t");
+			item->flags |= TODO_EDIT_MERGE_MSG;
+		} else {
+			item->flags |= TODO_EDIT_MERGE_MSG;
+			item->commit = NULL;
+			item->arg = bol;
+			item->arg_len = (int)(eol - bol);
+			return 0;
+		}
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
@@ -2646,6 +2670,153 @@ static int do_reset(const char *name, int len, struct replay_opts *opts)
 	return ret;
 }
 
+static int do_merge(struct commit *commit, const char *arg, int arg_len,
+		    int flags, struct replay_opts *opts)
+{
+	int run_commit_flags = (flags & TODO_EDIT_MERGE_MSG) ?
+		EDIT_MSG | VERIFY_MSG : 0;
+	struct strbuf ref_name = STRBUF_INIT;
+	struct commit *head_commit, *merge_commit, *i;
+	struct commit_list *bases, *j, *reversed = NULL;
+	struct merge_options o;
+	int merge_arg_len, oneline_offset, ret;
+	static struct lock_file lock;
+	const char *p;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0) {
+		ret = -1;
+		goto leave_merge;
+	}
+
+	head_commit = lookup_commit_reference_by_name("HEAD");
+	if (!head_commit) {
+		ret = error(_("cannot merge without a current revision"));
+		goto leave_merge;
+	}
+
+	oneline_offset = arg_len;
+	merge_arg_len = strcspn(arg, " \t\n");
+	p = arg + merge_arg_len;
+	p += strspn(p, " \t\n");
+	if (*p == '#' && (!p[1] || isspace(p[1]))) {
+		p += 1 + strspn(p + 1, " \t\n");
+		oneline_offset = p - arg;
+	} else if (p - arg < arg_len)
+		BUG("octopus merges are not supported yet: '%s'", p);
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
+	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	if (!merge_commit) {
+		/* fall back to non-rewritten ref or commit */
+		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
+		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	}
+
+	if (!merge_commit) {
+		ret = error(_("could not resolve '%s'"), ref_name.buf);
+		goto leave_merge;
+	}
+
+	if (commit) {
+		const char *message = get_commit_buffer(commit, NULL);
+		const char *body;
+		int len;
+
+		if (!message) {
+			ret = error(_("could not get commit message of '%s'"),
+				    oid_to_hex(&commit->object.oid));
+			goto leave_merge;
+		}
+		write_author_script(message);
+		find_commit_subject(message, &body);
+		len = strlen(body);
+		ret = write_message(body, len, git_path_merge_msg(), 0);
+		unuse_commit_buffer(commit, message);
+		if (ret) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			goto leave_merge;
+		}
+	} else {
+		struct strbuf buf = STRBUF_INIT;
+		int len;
+
+		strbuf_addf(&buf, "author %s", git_author_info(0));
+		write_author_script(buf.buf);
+		strbuf_reset(&buf);
+
+		if (oneline_offset < arg_len) {
+			p = arg + oneline_offset;
+			len = arg_len - oneline_offset;
+		} else {
+			strbuf_addf(&buf, "Merge branch '%.*s'",
+				    merge_arg_len, arg);
+			p = buf.buf;
+			len = buf.len;
+		}
+
+		ret = write_message(p, len, git_path_merge_msg(), 0);
+		strbuf_release(&buf);
+		if (ret) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			goto leave_merge;
+		}
+	}
+
+	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
+		      git_path_merge_head(), 0);
+	write_message("no-ff", 5, git_path_merge_mode(), 0);
+
+	bases = get_merge_bases(head_commit, merge_commit);
+	for (j = bases; j; j = j->next)
+		commit_list_insert(j->item, &reversed);
+	free_commit_list(bases);
+
+	read_cache();
+	init_merge_options(&o);
+	o.branch1 = "HEAD";
+	o.branch2 = ref_name.buf;
+	o.buffer_output = 2;
+
+	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
+	if (ret <= 0)
+		fputs(o.obuf.buf, stdout);
+	strbuf_release(&o.obuf);
+	if (ret < 0) {
+		error(_("could not even attempt to merge '%.*s'"),
+		      merge_arg_len, arg);
+		goto leave_merge;
+	}
+	/*
+	 * The return value of merge_recursive() is 1 on clean, and 0 on
+	 * unclean merge.
+	 *
+	 * Let's reverse that, so that do_merge() returns 0 upon success and
+	 * 1 upon failed merge (keeping the return value -1 for the cases where
+	 * we will want to reschedule the `merge` command).
+	 */
+	ret = !ret;
+
+	if (active_cache_changed &&
+	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
+		ret = error(_("merge: Unable to write new index file"));
+		goto leave_merge;
+	}
+
+	rollback_lock_file(&lock);
+	if (ret)
+		rerere(opts->allow_rerere_auto);
+	else
+		ret = run_git_commit(git_path_merge_msg(), opts,
+				     run_commit_flags);
+
+leave_merge:
+	strbuf_release(&ref_name);
+	rollback_lock_file(&lock);
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2852,6 +3023,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		} else if (item->command == TODO_RESET) {
 			if ((res = do_reset(item->arg, item->arg_len, opts)))
 				goto reschedule;
+		} else if (item->command == TODO_MERGE) {
+			res = do_merge(item->commit, item->arg, item->arg_len,
+				       item->flags, opts);
+			if (res < 0)
+				goto reschedule;
 		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
@@ -3334,8 +3510,16 @@ int transform_todos(unsigned flags)
 					  short_commit_name(item->commit) :
 					  oid_to_hex(&item->commit->object.oid);
 
+			if (item->command == TODO_MERGE) {
+				if (item->flags & TODO_EDIT_MERGE_MSG)
+					strbuf_addstr(&buf, " -c");
+				else
+					strbuf_addstr(&buf, " -C");
+			}
+
 			strbuf_addf(&buf, " %s", oid);
 		}
+
 		/* add all the rest */
 		if (!item->arg_len)
 			strbuf_addch(&buf, '\n');
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 08/17] sequencer: fast-forward `merge` commands, if possible
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (6 preceding siblings ...)
  2018-04-19 12:20           ` [PATCH v7 07/17] # This is a combination of 2 commits. # This is the 1st commit message: Johannes Schindelin
@ 2018-04-19 12:21           ` Johannes Schindelin
  2018-04-19 12:21           ` [PATCH v7 09/17] rebase-helper --make-script: introduce a flag to rebase merges Johannes Schindelin
                             ` (9 subsequent siblings)
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:21 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Just like with regular `pick` commands, if we are trying to rebase a
merge commit, we now test whether the parents of said commit match HEAD
and the commits to be merged, and fast-forward if possible.

This is not only faster, but also avoids unnecessary proliferation of
new objects.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 ++++++++++++++++++++++++++++++++-
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 90b2fac96b1..2ae2294272b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2679,7 +2679,7 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	struct commit *head_commit, *merge_commit, *i;
 	struct commit_list *bases, *j, *reversed = NULL;
 	struct merge_options o;
-	int merge_arg_len, oneline_offset, ret;
+	int merge_arg_len, oneline_offset, can_fast_forward, ret;
 	static struct lock_file lock;
 	const char *p;
 
@@ -2764,6 +2764,37 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		}
 	}
 
+	/*
+	 * If HEAD is not identical to the first parent of the original merge
+	 * commit, we cannot fast-forward.
+	 */
+	can_fast_forward = opts->allow_ff && commit && commit->parents &&
+		!oidcmp(&commit->parents->item->object.oid,
+			&head_commit->object.oid);
+
+	/*
+	 * If the merge head is different from the original one, we cannot
+	 * fast-forward.
+	 */
+	if (can_fast_forward) {
+		struct commit_list *second_parent = commit->parents->next;
+
+		if (second_parent && !second_parent->next &&
+		    oidcmp(&merge_commit->object.oid,
+			   &second_parent->item->object.oid))
+			can_fast_forward = 0;
+	}
+
+	if (can_fast_forward && commit->parents->next &&
+	    !commit->parents->next->next &&
+	    !oidcmp(&commit->parents->next->item->object.oid,
+		    &merge_commit->object.oid)) {
+		rollback_lock_file(&lock);
+		ret = fast_forward_to(&commit->object.oid,
+				      &head_commit->object.oid, 0, opts);
+		goto leave_merge;
+	}
+
 	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
 		      git_path_merge_head(), 0);
 	write_message("no-ff", 5, git_path_merge_mode(), 0);
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 09/17] rebase-helper --make-script: introduce a flag to rebase merges
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (7 preceding siblings ...)
  2018-04-19 12:21           ` [PATCH v7 08/17] sequencer: fast-forward `merge` commands, if possible Johannes Schindelin
@ 2018-04-19 12:21           ` Johannes Schindelin
  2018-04-19 12:21           ` [PATCH v7 10/17] rebase: introduce the --rebase-merges option Johannes Schindelin
                             ` (8 subsequent siblings)
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:21 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

The sequencer just learned new commands intended to recreate branch
structure (similar in spirit to --preserve-merges, but with a
substantially less-broken design).

Let's allow the rebase--helper to generate todo lists making use of
these commands, triggered by the new --rebase-merges option. For a
commit topology like this (where the HEAD points to C):

	- A - B - C
	    \   /
	      D

the generated todo list would look like this:

	# branch D
	pick 0123 A
	label branch-point
	pick 1234 D
	label D

	reset branch-point
	pick 2345 B
	merge -C 3456 D # C

To keep things simple, we first only implement support for merge commits
with exactly two parents, leaving support for octopus merges to a later
patch series.

As a special, hard-coded label, all merge-rebasing todo lists start with
the command `label onto` so that we can later always refer to the revision
onto which everything is rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/rebase--helper.c |   4 +-
 sequencer.c              | 351 ++++++++++++++++++++++++++++++++++++++-
 sequencer.h              |   1 +
 3 files changed, 353 insertions(+), 3 deletions(-)

diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index ad074705bb5..781782e7272 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -12,7 +12,7 @@ static const char * const builtin_rebase_helper_usage[] = {
 int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
-	unsigned flags = 0, keep_empty = 0;
+	unsigned flags = 0, keep_empty = 0, rebase_merges = 0;
 	int abbreviate_commands = 0;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
@@ -24,6 +24,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
 		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
 			N_("allow commits with empty messages")),
+		OPT_BOOL(0, "rebase-merges", &rebase_merges, N_("rebase merge commits")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -57,6 +58,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
+	flags |= rebase_merges ? TODO_LIST_REBASE_MERGES : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
 	if (command == CONTINUE && argc == 1)
diff --git a/sequencer.c b/sequencer.c
index 2ae2294272b..32ebbc002c1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -25,6 +25,8 @@
 #include "sigchain.h"
 #include "unpack-trees.h"
 #include "worktree.h"
+#include "oidmap.h"
+#include "oidset.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -3414,6 +3416,343 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
 	strbuf_release(&sob);
 }
 
+struct labels_entry {
+	struct hashmap_entry entry;
+	char label[FLEX_ARRAY];
+};
+
+static int labels_cmp(const void *fndata, const struct labels_entry *a,
+		      const struct labels_entry *b, const void *key)
+{
+	return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
+}
+
+struct string_entry {
+	struct oidmap_entry entry;
+	char string[FLEX_ARRAY];
+};
+
+struct label_state {
+	struct oidmap commit2label;
+	struct hashmap labels;
+	struct strbuf buf;
+};
+
+static const char *label_oid(struct object_id *oid, const char *label,
+			     struct label_state *state)
+{
+	struct labels_entry *labels_entry;
+	struct string_entry *string_entry;
+	struct object_id dummy;
+	size_t len;
+	int i;
+
+	string_entry = oidmap_get(&state->commit2label, oid);
+	if (string_entry)
+		return string_entry->string;
+
+	/*
+	 * For "uninteresting" commits, i.e. commits that are not to be
+	 * rebased, and which can therefore not be labeled, we use a unique
+	 * abbreviation of the commit name. This is slightly more complicated
+	 * than calling find_unique_abbrev() because we also need to make
+	 * sure that the abbreviation does not conflict with any other
+	 * label.
+	 *
+	 * We disallow "interesting" commits to be labeled by a string that
+	 * is a valid full-length hash, to ensure that we always can find an
+	 * abbreviation for any uninteresting commit's names that does not
+	 * clash with any other label.
+	 */
+	if (!label) {
+		char *p;
+
+		strbuf_reset(&state->buf);
+		strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
+		label = p = state->buf.buf;
+
+		find_unique_abbrev_r(p, oid, default_abbrev);
+
+		/*
+		 * We may need to extend the abbreviated hash so that there is
+		 * no conflicting label.
+		 */
+		if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
+			size_t i = strlen(p) + 1;
+
+			oid_to_hex_r(p, oid);
+			for (; i < GIT_SHA1_HEXSZ; i++) {
+				char save = p[i];
+				p[i] = '\0';
+				if (!hashmap_get_from_hash(&state->labels,
+							   strihash(p), p))
+					break;
+				p[i] = save;
+			}
+		}
+	} else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
+		    !get_oid_hex(label, &dummy)) ||
+		   (len == 1 && *label == '#') ||
+		   hashmap_get_from_hash(&state->labels,
+					 strihash(label), label)) {
+		/*
+		 * If the label already exists, or if the label is a valid full
+		 * OID, or the label is a '#' (which we use as a separator
+		 * between merge heads and oneline), we append a dash and a
+		 * number to make it unique.
+		 */
+		struct strbuf *buf = &state->buf;
+
+		strbuf_reset(buf);
+		strbuf_add(buf, label, len);
+
+		for (i = 2; ; i++) {
+			strbuf_setlen(buf, len);
+			strbuf_addf(buf, "-%d", i);
+			if (!hashmap_get_from_hash(&state->labels,
+						   strihash(buf->buf),
+						   buf->buf))
+				break;
+		}
+
+		label = buf->buf;
+	}
+
+	FLEX_ALLOC_STR(labels_entry, label, label);
+	hashmap_entry_init(labels_entry, strihash(label));
+	hashmap_add(&state->labels, labels_entry);
+
+	FLEX_ALLOC_STR(string_entry, string, label);
+	oidcpy(&string_entry->entry.oid, oid);
+	oidmap_put(&state->commit2label, string_entry);
+
+	return string_entry->string;
+}
+
+static int make_script_with_merges(struct pretty_print_context *pp,
+				   struct rev_info *revs, FILE *out,
+				   unsigned flags)
+{
+	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
+	struct strbuf label = STRBUF_INIT;
+	struct commit_list *commits = NULL, **tail = &commits, *iter;
+	struct commit_list *tips = NULL, **tips_tail = &tips;
+	struct commit *commit;
+	struct oidmap commit2todo = OIDMAP_INIT;
+	struct string_entry *entry;
+	struct oidset interesting = OIDSET_INIT, child_seen = OIDSET_INIT,
+		shown = OIDSET_INIT;
+	struct label_state state = { OIDMAP_INIT, { NULL }, STRBUF_INIT };
+
+	int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
+	const char *cmd_pick = abbr ? "p" : "pick",
+		*cmd_label = abbr ? "l" : "label",
+		*cmd_reset = abbr ? "t" : "reset",
+		*cmd_merge = abbr ? "m" : "merge";
+
+	oidmap_init(&commit2todo, 0);
+	oidmap_init(&state.commit2label, 0);
+	hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
+	strbuf_init(&state.buf, 32);
+
+	if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
+		struct object_id *oid = &revs->cmdline.rev[0].item->oid;
+		FLEX_ALLOC_STR(entry, string, "onto");
+		oidcpy(&entry->entry.oid, oid);
+		oidmap_put(&state.commit2label, entry);
+	}
+
+	/*
+	 * First phase:
+	 * - get onelines for all commits
+	 * - gather all branch tips (i.e. 2nd or later parents of merges)
+	 * - label all branch tips
+	 */
+	while ((commit = get_revision(revs))) {
+		struct commit_list *to_merge;
+		int is_octopus;
+		const char *p1, *p2;
+		struct object_id *oid;
+		int is_empty;
+
+		tail = &commit_list_insert(commit, tail)->next;
+		oidset_insert(&interesting, &commit->object.oid);
+
+		is_empty = is_original_commit_empty(commit);
+		if (!is_empty && (commit->object.flags & PATCHSAME))
+			continue;
+
+		strbuf_reset(&oneline);
+		pretty_print_commit(pp, commit, &oneline);
+
+		to_merge = commit->parents ? commit->parents->next : NULL;
+		if (!to_merge) {
+			/* non-merge commit: easy case */
+			strbuf_reset(&buf);
+			if (!keep_empty && is_empty)
+				strbuf_addf(&buf, "%c ", comment_line_char);
+			strbuf_addf(&buf, "%s %s %s", cmd_pick,
+				    oid_to_hex(&commit->object.oid),
+				    oneline.buf);
+
+			FLEX_ALLOC_STR(entry, string, buf.buf);
+			oidcpy(&entry->entry.oid, &commit->object.oid);
+			oidmap_put(&commit2todo, entry);
+
+			continue;
+		}
+
+		is_octopus = to_merge && to_merge->next;
+
+		if (is_octopus)
+			BUG("Octopus merges not yet supported");
+
+		/* Create a label */
+		strbuf_reset(&label);
+		if (skip_prefix(oneline.buf, "Merge ", &p1) &&
+		    (p1 = strchr(p1, '\'')) &&
+		    (p2 = strchr(++p1, '\'')))
+			strbuf_add(&label, p1, p2 - p1);
+		else if (skip_prefix(oneline.buf, "Merge pull request ",
+				     &p1) &&
+			 (p1 = strstr(p1, " from ")))
+			strbuf_addstr(&label, p1 + strlen(" from "));
+		else
+			strbuf_addbuf(&label, &oneline);
+
+		for (p1 = label.buf; *p1; p1++)
+			if (isspace(*p1))
+				*(char *)p1 = '-';
+
+		strbuf_reset(&buf);
+		strbuf_addf(&buf, "%s -C %s",
+			    cmd_merge, oid_to_hex(&commit->object.oid));
+
+		/* label the tip of merged branch */
+		oid = &to_merge->item->object.oid;
+		strbuf_addch(&buf, ' ');
+
+		if (!oidset_contains(&interesting, oid))
+			strbuf_addstr(&buf, label_oid(oid, NULL, &state));
+		else {
+			tips_tail = &commit_list_insert(to_merge->item,
+							tips_tail)->next;
+
+			strbuf_addstr(&buf, label_oid(oid, label.buf, &state));
+		}
+		strbuf_addf(&buf, " # %s", oneline.buf);
+
+		FLEX_ALLOC_STR(entry, string, buf.buf);
+		oidcpy(&entry->entry.oid, &commit->object.oid);
+		oidmap_put(&commit2todo, entry);
+	}
+
+	/*
+	 * Second phase:
+	 * - label branch points
+	 * - add HEAD to the branch tips
+	 */
+	for (iter = commits; iter; iter = iter->next) {
+		struct commit_list *parent = iter->item->parents;
+		for (; parent; parent = parent->next) {
+			struct object_id *oid = &parent->item->object.oid;
+			if (!oidset_contains(&interesting, oid))
+				continue;
+			if (!oidset_contains(&child_seen, oid))
+				oidset_insert(&child_seen, oid);
+			else
+				label_oid(oid, "branch-point", &state);
+		}
+
+		/* Add HEAD as implict "tip of branch" */
+		if (!iter->next)
+			tips_tail = &commit_list_insert(iter->item,
+							tips_tail)->next;
+	}
+
+	/*
+	 * Third phase: output the todo list. This is a bit tricky, as we
+	 * want to avoid jumping back and forth between revisions. To
+	 * accomplish that goal, we walk backwards from the branch tips,
+	 * gathering commits not yet shown, reversing the list on the fly,
+	 * then outputting that list (labeling revisions as needed).
+	 */
+	fprintf(out, "%s onto\n", cmd_label);
+	for (iter = tips; iter; iter = iter->next) {
+		struct commit_list *list = NULL, *iter2;
+
+		commit = iter->item;
+		if (oidset_contains(&shown, &commit->object.oid))
+			continue;
+		entry = oidmap_get(&state.commit2label, &commit->object.oid);
+
+		if (entry)
+			fprintf(out, "\n# Branch %s\n", entry->string);
+		else
+			fprintf(out, "\n");
+
+		while (oidset_contains(&interesting, &commit->object.oid) &&
+		       !oidset_contains(&shown, &commit->object.oid)) {
+			commit_list_insert(commit, &list);
+			if (!commit->parents) {
+				commit = NULL;
+				break;
+			}
+			commit = commit->parents->item;
+		}
+
+		if (!commit)
+			fprintf(out, "%s onto\n", cmd_reset);
+		else {
+			const char *to = NULL;
+
+			entry = oidmap_get(&state.commit2label,
+					   &commit->object.oid);
+			if (entry)
+				to = entry->string;
+
+			if (!to || !strcmp(to, "onto"))
+				fprintf(out, "%s onto\n", cmd_reset);
+			else {
+				strbuf_reset(&oneline);
+				pretty_print_commit(pp, commit, &oneline);
+				fprintf(out, "%s %s # %s\n",
+					cmd_reset, to, oneline.buf);
+			}
+		}
+
+		for (iter2 = list; iter2; iter2 = iter2->next) {
+			struct object_id *oid = &iter2->item->object.oid;
+			entry = oidmap_get(&commit2todo, oid);
+			/* only show if not already upstream */
+			if (entry)
+				fprintf(out, "%s\n", entry->string);
+			entry = oidmap_get(&state.commit2label, oid);
+			if (entry)
+				fprintf(out, "%s %s\n",
+					cmd_label, entry->string);
+			oidset_insert(&shown, oid);
+		}
+
+		free_commit_list(list);
+	}
+
+	free_commit_list(commits);
+	free_commit_list(tips);
+
+	strbuf_release(&label);
+	strbuf_release(&oneline);
+	strbuf_release(&buf);
+
+	oidmap_free(&commit2todo, 1);
+	oidmap_free(&state.commit2label, 1);
+	hashmap_free(&state.labels, 1);
+	strbuf_release(&state.buf);
+
+	return 0;
+}
+
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags)
 {
@@ -3424,11 +3763,16 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	struct commit *commit;
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
 	const char *insn = flags & TODO_LIST_ABBREVIATE_CMDS ? "p" : "pick";
+	int rebase_merges = flags & TODO_LIST_REBASE_MERGES;
 
 	init_revisions(&revs, NULL);
 	revs.verbose_header = 1;
-	revs.max_parents = 1;
-	revs.cherry_pick = 1;
+	if (rebase_merges)
+		revs.cherry_mark = 1;
+	else {
+		revs.max_parents = 1;
+		revs.cherry_pick = 1;
+	}
 	revs.limited = 1;
 	revs.reverse = 1;
 	revs.right_only = 1;
@@ -3452,6 +3796,9 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	if (prepare_revision_walk(&revs) < 0)
 		return error(_("make_script: error preparing revisions"));
 
+	if (rebase_merges)
+		return make_script_with_merges(&pp, &revs, out, flags);
+
 	while ((commit = get_revision(&revs))) {
 		strbuf_reset(&buf);
 		if (!keep_empty && is_original_commit_empty(commit))
diff --git a/sequencer.h b/sequencer.h
index e45b178dfc4..6bc4da17243 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -59,6 +59,7 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_KEEP_EMPTY (1U << 0)
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
+#define TODO_LIST_REBASE_MERGES (1U << 3)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 10/17] rebase: introduce the --rebase-merges option
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (8 preceding siblings ...)
  2018-04-19 12:21           ` [PATCH v7 09/17] rebase-helper --make-script: introduce a flag to rebase merges Johannes Schindelin
@ 2018-04-19 12:21           ` Johannes Schindelin
  2018-04-19 12:22           ` [PATCH v7 11/17] rebase --rebase-merges: add test for --keep-empty Johannes Schindelin
                             ` (7 subsequent siblings)
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:21 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Once upon a time, this here developer thought: wouldn't it be nice if,
say, Git for Windows' patches on top of core Git could be represented as
a thicket of branches, and be rebased on top of core Git in order to
maintain a cherry-pick'able set of patch series?

The original attempt to answer this was: git rebase --preserve-merges.

However, that experiment was never intended as an interactive option,
and it only piggy-backed on git rebase --interactive because that
command's implementation looked already very, very familiar: it was
designed by the same person who designed --preserve-merges: yours truly.

Some time later, some other developer (I am looking at you, Andreas!
;-)) decided that it would be a good idea to allow --preserve-merges to
be combined with --interactive (with caveats!) and the Git maintainer
(well, the interim Git maintainer during Junio's absence, that is)
agreed, and that is when the glamor of the --preserve-merges design
started to fall apart rather quickly and unglamorously.

The reason? In --preserve-merges mode, the parents of a merge commit (or
for that matter, of *any* commit) were not stated explicitly, but were
*implied* by the commit name passed to the `pick` command.

This made it impossible, for example, to reorder commits. Not to mention
to flatten the branch topology or, deity forbid, to split topic branches
into two.

Alas, these shortcomings also prevented that mode (whose original
purpose was to serve Git for Windows' needs, with the additional hope
that it may be useful to others, too) from serving Git for Windows'
needs.

Five years later, when it became really untenable to have one unwieldy,
big hodge-podge patch series of partly related, partly unrelated patches
in Git for Windows that was rebased onto core Git's tags from time to
time (earning the undeserved wrath of the developer of the ill-fated
git-remote-hg series that first obsoleted Git for Windows' competing
approach, only to be abandoned without maintainer later) was really
untenable, the "Git garden shears" were born [*1*/*2*]: a script,
piggy-backing on top of the interactive rebase, that would first
determine the branch topology of the patches to be rebased, create a
pseudo todo list for further editing, transform the result into a real
todo list (making heavy use of the `exec` command to "implement" the
missing todo list commands) and finally recreate the patch series on
top of the new base commit.

That was in 2013. And it took about three weeks to come up with the
design and implement it as an out-of-tree script. Needless to say, the
implementation needed quite a few years to stabilize, all the while the
design itself proved itself sound.

With this patch, the goodness of the Git garden shears comes to `git
rebase -i` itself. Passing the `--rebase-merges` option will generate
a todo list that can be understood readily, and where it is obvious
how to reorder commits. New branches can be introduced by inserting
`label` commands and calling `merge <label>`. And once this mode will
have become stable and universally accepted, we can deprecate the design
mistake that was `--preserve-merges`.

Link *1*:
https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
Link *2*:
https://github.com/git-for-windows/build-extra/blob/master/shears.sh

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt           |  10 +-
 contrib/completion/git-completion.bash |   2 +-
 git-rebase--interactive.sh             |   1 +
 git-rebase.sh                          |   6 +
 t/t3430-rebase-merges.sh               | 156 +++++++++++++++++++++++++
 5 files changed, 173 insertions(+), 2 deletions(-)
 create mode 100755 t/t3430-rebase-merges.sh

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 3277ca14327..936c5619b42 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -378,6 +378,13 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
+-r::
+--rebase-merges::
+	Rebase merge commits instead of flattening the history by replaying
+	merges. Merge conflict resolutions or manual amendments to merge
+	commits are not rebased automatically, but have to be applied
+	manually.
+
 -p::
 --preserve-merges::
 	Recreate merge commits instead of flattening the history by replaying
@@ -780,7 +787,8 @@ BUGS
 The todo list presented by `--preserve-merges --interactive` does not
 represent the topology of the revision graph.  Editing commits and
 rewording their commit messages should work fine, but attempts to
-reorder commits tend to produce counterintuitive results.
+reorder commits tend to produce counterintuitive results. Use
+--rebase-merges for a more faithful representation.
 
 For example, an attempt to rearrange
 ------------
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index a7570739454..d4c0a995c39 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1949,7 +1949,7 @@ _git_rebase ()
 	--*)
 		__gitcomp "
 			--onto --merge --strategy --interactive
-			--preserve-merges --stat --no-stat
+			--rebase-merges --preserve-merges --stat --no-stat
 			--committer-date-is-author-date --ignore-date
 			--ignore-whitespace --whitespace=
 			--autosquash --no-autosquash
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index ccd5254d1c9..7a3daf3e40c 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -970,6 +970,7 @@ git_rebase__interactive () {
 	init_revisions_and_shortrevisions
 
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
+		${rebase_merges:+--rebase-merges} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 
diff --git a/git-rebase.sh b/git-rebase.sh
index fb64ee1fe42..a64460fd25a 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,6 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
+r,rebase-merges!   try to rebase merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -88,6 +89,7 @@ type=
 state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
+rebase_merges=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -270,6 +272,10 @@ do
 	--allow-empty-message)
 		allow_empty_message=--allow-empty-message
 		;;
+	--rebase-merges)
+		rebase_merges=t
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
new file mode 100755
index 00000000000..e80fa068d05
--- /dev/null
+++ b/t/t3430-rebase-merges.sh
@@ -0,0 +1,156 @@
+#!/bin/sh
+#
+# Copyright (c) 2018 Johannes E. Schindelin
+#
+
+test_description='git rebase -i --rebase-merges
+
+This test runs git rebase "interactively", retaining the branch structure by
+recreating merge commits.
+
+Initial setup:
+
+    -- B --                   (first)
+   /       \
+ A - C - D - E - H            (master)
+       \       /
+         F - G                (second)
+'
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
+
+test_cmp_graph () {
+	cat >expect &&
+	git log --graph --boundary --format=%s "$@" >output &&
+	sed "s/ *$//" <output >output.trimmed &&
+	test_cmp expect output.trimmed
+}
+
+test_expect_success 'setup' '
+	write_script replace-editor.sh <<-\EOF &&
+	mv "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	cp script-from-scratch "$1"
+	EOF
+
+	test_commit A &&
+	git checkout -b first &&
+	test_commit B &&
+	git checkout master &&
+	test_commit C &&
+	test_commit D &&
+	git merge --no-commit B &&
+	test_tick &&
+	git commit -m E &&
+	git tag -m E E &&
+	git checkout -b second C &&
+	test_commit F &&
+	test_commit G &&
+	git checkout master &&
+	git merge --no-commit G &&
+	test_tick &&
+	git commit -m H &&
+	git tag -m H H
+'
+
+test_expect_success 'create completely different structure' '
+	cat >script-from-scratch <<-\EOF &&
+	label onto
+
+	# onebranch
+	pick G
+	pick D
+	label onebranch
+
+	# second
+	reset onto
+	pick B
+	label second
+
+	reset onto
+	merge -C H second
+	merge onebranch # Merge the topic branch '\''onebranch'\''
+	EOF
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_tick &&
+	git rebase -i -r A &&
+	test_cmp_graph <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	* |   H
+	|\ \
+	| |/
+	|/|
+	| * B
+	|/
+	* A
+	EOF
+'
+
+test_expect_success 'generate correct todo list' '
+	cat >expect <<-\EOF &&
+	label onto
+
+	reset onto
+	pick d9df450 B
+	label E
+
+	reset onto
+	pick 5dee784 C
+	label branch-point
+	pick ca2c861 F
+	pick 088b00a G
+	label H
+
+	reset branch-point # C
+	pick 12bd07b D
+	merge -C 2051b56 E # E
+	merge -C 233d48a H # H
+
+	EOF
+
+	grep -v "^#" <.git/ORIGINAL-TODO >output &&
+	test_cmp expect output
+'
+
+test_expect_success '`reset` refuses to overwrite untracked files' '
+	git checkout -b refuse-to-reset &&
+	test_commit dont-overwrite-untracked &&
+	git checkout @{-1} &&
+	: >dont-overwrite-untracked.t &&
+	echo "reset refs/tags/dont-overwrite-untracked" >script-from-scratch &&
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_must_fail git rebase -r HEAD &&
+	git rebase --abort
+'
+
+test_expect_success 'with a branch tip that was cherry-picked already' '
+	git checkout -b already-upstream master &&
+	base="$(git rev-parse --verify HEAD)" &&
+
+	test_commit A1 &&
+	test_commit A2 &&
+	git reset --hard $base &&
+	test_commit B1 &&
+	test_tick &&
+	git merge -m "Merge branch A" A2 &&
+
+	git checkout -b upstream-with-a2 $base &&
+	test_tick &&
+	git cherry-pick A2 &&
+
+	git checkout already-upstream &&
+	test_tick &&
+	git rebase -i -r upstream-with-a2 &&
+	test_cmp_graph upstream-with-a2.. <<-\EOF
+	*   Merge branch A
+	|\
+	| * A1
+	* | B1
+	|/
+	o A2
+	EOF
+'
+
+test_done
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 11/17] rebase --rebase-merges: add test for --keep-empty
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (9 preceding siblings ...)
  2018-04-19 12:21           ` [PATCH v7 10/17] rebase: introduce the --rebase-merges option Johannes Schindelin
@ 2018-04-19 12:22           ` Johannes Schindelin
  2018-04-19 12:22           ` [PATCH v7 12/17] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
                             ` (6 subsequent siblings)
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:22 UTC (permalink / raw)
  To: git
  Cc: Phillip Wood, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

From: Phillip Wood <phillip.wood@dunelm.org.uk>

If there are empty commits on the left hand side of $upstream...HEAD
then the empty commits on the right hand side that we want to keep are
being pruned.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
---
 t/t3421-rebase-topology-linear.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/t3421-rebase-topology-linear.sh b/t/t3421-rebase-topology-linear.sh
index 68fe2003ef5..fbae5dab7e2 100755
--- a/t/t3421-rebase-topology-linear.sh
+++ b/t/t3421-rebase-topology-linear.sh
@@ -217,6 +217,7 @@ test_run_rebase success ''
 test_run_rebase failure -m
 test_run_rebase failure -i
 test_run_rebase failure -p
+test_run_rebase success --rebase-merges
 
 #       m
 #      /
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 12/17] sequencer: make refs generated by the `label` command worktree-local
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (10 preceding siblings ...)
  2018-04-19 12:22           ` [PATCH v7 11/17] rebase --rebase-merges: add test for --keep-empty Johannes Schindelin
@ 2018-04-19 12:22           ` Johannes Schindelin
  2018-04-19 12:23           ` [PATCH v7 13/17] sequencer: handle post-rewrite for merge commands Johannes Schindelin
                             ` (5 subsequent siblings)
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

This allows for rebases to be run in parallel in separate worktrees
(think: interrupted in the middle of one rebase, being asked to perform
a different rebase, adding a separate worktree just for that job).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 refs.c                   |  3 ++-
 t/t3430-rebase-merges.sh | 14 ++++++++++++++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 8b7a77fe5ee..f61ec58d1df 100644
--- a/refs.c
+++ b/refs.c
@@ -600,7 +600,8 @@ int dwim_log(const char *str, int len, struct object_id *oid, char **log)
 static int is_per_worktree_ref(const char *refname)
 {
 	return !strcmp(refname, "HEAD") ||
-		starts_with(refname, "refs/bisect/");
+		starts_with(refname, "refs/bisect/") ||
+		starts_with(refname, "refs/rewritten/");
 }
 
 static int is_pseudoref_syntax(const char *refname)
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index e80fa068d05..4b553fc78b1 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -153,4 +153,18 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'refs/rewritten/* is worktree-local' '
+	git worktree add wt &&
+	cat >wt/script-from-scratch <<-\EOF &&
+	label xyz
+	exec GIT_DIR=../.git git rev-parse --verify refs/rewritten/xyz >a || :
+	exec git rev-parse --verify refs/rewritten/xyz >b
+	EOF
+
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	git -C wt rebase -i HEAD &&
+	test_must_be_empty wt/a &&
+	test_cmp_rev HEAD "$(cat wt/b)"
+'
+
 test_done
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 13/17] sequencer: handle post-rewrite for merge commands
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (11 preceding siblings ...)
  2018-04-19 12:22           ` [PATCH v7 12/17] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
@ 2018-04-19 12:23           ` Johannes Schindelin
  2018-04-19 12:23           ` [PATCH v7 14/17] rebase --rebase-merges: avoid "empty merges" Johannes Schindelin
                             ` (4 subsequent siblings)
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:23 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

In the previous patches, we implemented the basic functionality of the
`git rebase -i --rebase-merges` command, in particular the `merge`
command to create merge commits in the sequencer.

The interactive rebase is a lot more these days, though, than a simple
cherry-pick in a loop. For example, it calls the post-rewrite hook (if
any) after rebasing with a mapping of the old->new commits.

This patch implements the post-rewrite handling for the `merge` command
we just introduced. The other commands that were added recently (`label`
and `reset`) do not create new commits, therefore post-rewrite hooks do
not need to handle them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c              |  3 +++
 t/t3430-rebase-merges.sh | 25 +++++++++++++++++++++++++
 2 files changed, 28 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 32ebbc002c1..620a4c3a506 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3061,6 +3061,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				       item->flags, opts);
 			if (res < 0)
 				goto reschedule;
+			if (item->commit)
+				record_in_rewritten(&item->commit->object.oid,
+						    peek_command(todo_list, 1));
 		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index 4b553fc78b1..dedfa09d761 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -167,4 +167,29 @@ test_expect_success 'refs/rewritten/* is worktree-local' '
 	test_cmp_rev HEAD "$(cat wt/b)"
 '
 
+test_expect_success 'post-rewrite hook and fixups work for merges' '
+	git checkout -b post-rewrite &&
+	test_commit same1 &&
+	git reset --hard HEAD^ &&
+	test_commit same2 &&
+	git merge -m "to fix up" same1 &&
+	echo same old same old >same2.t &&
+	test_tick &&
+	git commit --fixup HEAD same2.t &&
+	fixup="$(git rev-parse HEAD)" &&
+
+	mkdir -p .git/hooks &&
+	test_when_finished "rm .git/hooks/post-rewrite" &&
+	echo "cat >actual" | write_script .git/hooks/post-rewrite &&
+
+	test_tick &&
+	git rebase -i --autosquash -r HEAD^^^ &&
+	printf "%s %s\n%s %s\n%s %s\n%s %s\n" >expect $(git rev-parse \
+		$fixup^^2 HEAD^2 \
+		$fixup^^ HEAD^ \
+		$fixup^ HEAD \
+		$fixup HEAD) &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 14/17] rebase --rebase-merges: avoid "empty merges"
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (12 preceding siblings ...)
  2018-04-19 12:23           ` [PATCH v7 13/17] sequencer: handle post-rewrite for merge commands Johannes Schindelin
@ 2018-04-19 12:23           ` Johannes Schindelin
  2018-04-19 12:23           ` [PATCH v7 15/17] pull: accept --rebase=merges to recreate the branch topology Johannes Schindelin
                             ` (3 subsequent siblings)
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:23 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

The `git merge` command does not allow merging commits that are already
reachable from HEAD: `git merge HEAD^`, for example, will report that we
are already up to date and not change a thing.

In an interactive rebase, such a merge could occur previously, e.g. when
competing (or slightly modified) versions of a patch series were applied
upstream, and the user had to `git rebase --skip` all of the local
commits, and the topic branch becomes "empty" as a consequence.

Let's teach the todo command `merge` to behave the same as `git merge`.

Seeing as it requires some low-level trickery to create such merges with
Git's commands in the first place, we do not even have to bother to
introduce an option to force `merge` to create such merge commits.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c              | 7 +++++++
 t/t3430-rebase-merges.sh | 8 ++++++++
 2 files changed, 15 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 620a4c3a506..708b8719965 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2802,6 +2802,13 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	write_message("no-ff", 5, git_path_merge_mode(), 0);
 
 	bases = get_merge_bases(head_commit, merge_commit);
+	if (bases && !oidcmp(&merge_commit->object.oid,
+			     &bases->item->object.oid)) {
+		ret = 0;
+		/* skip merging an ancestor of HEAD */
+		goto leave_merge;
+	}
+
 	for (j = bases; j; j = j->next)
 		commit_list_insert(j->item, &reversed);
 	free_commit_list(bases);
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index dedfa09d761..37c3f73784a 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -192,4 +192,12 @@ test_expect_success 'post-rewrite hook and fixups work for merges' '
 	test_cmp expect actual
 '
 
+test_expect_success 'refuse to merge ancestors of HEAD' '
+	echo "merge HEAD^" >script-from-scratch &&
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	before="$(git rev-parse HEAD)" &&
+	git rebase -i HEAD &&
+	test_cmp_rev HEAD $before
+'
+
 test_done
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 15/17] pull: accept --rebase=merges to recreate the branch topology
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (13 preceding siblings ...)
  2018-04-19 12:23           ` [PATCH v7 14/17] rebase --rebase-merges: avoid "empty merges" Johannes Schindelin
@ 2018-04-19 12:23           ` Johannes Schindelin
  2018-04-19 12:24           ` [PATCH v7 16/17] rebase -i: introduce --rebase-merges=[no-]rebase-cousins Johannes Schindelin
                             ` (2 subsequent siblings)
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:23 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Similar to the `preserve` mode simply passing the `--preserve-merges`
option to the `rebase` command, the `merges` mode simply passes the
`--rebase-merges` option.

This will allow users to conveniently rebase non-trivial commit
topologies when pulling new commits, without flattening them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/config.txt               |  8 ++++++++
 Documentation/git-pull.txt             |  5 ++++-
 builtin/pull.c                         | 14 ++++++++++----
 builtin/remote.c                       | 18 ++++++++++++++----
 contrib/completion/git-completion.bash |  2 +-
 5 files changed, 37 insertions(+), 10 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 2659153cb37..da46f154bb3 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1058,6 +1058,10 @@ branch.<name>.rebase::
 	"git pull" is run. See "pull.rebase" for doing this in a non
 	branch-specific manner.
 +
+When `merges`, pass the `--rebase-merges` option to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
@@ -2617,6 +2621,10 @@ pull.rebase::
 	pull" is run. See "branch.<name>.rebase" for setting this on a
 	per-branch basis.
 +
+When `merges`, pass the `--rebase-merges` option to 'git rebase'
+so that locally committed merge commits will not be flattened
+by running 'git pull'.
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
diff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt
index ce05b7a5b13..6f76d815dd3 100644
--- a/Documentation/git-pull.txt
+++ b/Documentation/git-pull.txt
@@ -101,13 +101,16 @@ Options related to merging
 include::merge-options.txt[]
 
 -r::
---rebase[=false|true|preserve|interactive]::
+--rebase[=false|true|merges|preserve|interactive]::
 	When true, rebase the current branch on top of the upstream
 	branch after fetching. If there is a remote-tracking branch
 	corresponding to the upstream branch and the upstream branch
 	was rebased since last fetched, the rebase uses that information
 	to avoid rebasing non-local changes.
 +
+When set to `merges`, rebase using `git rebase --rebase-merges` so that
+locally created merge commits will not be flattened.
++
 When set to preserve, rebase with the `--preserve-merges` option passed
 to `git rebase` so that locally created merge commits will not be flattened.
 +
diff --git a/builtin/pull.c b/builtin/pull.c
index e32d6cd5b4c..70b44146ce4 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -27,14 +27,16 @@ enum rebase_type {
 	REBASE_FALSE = 0,
 	REBASE_TRUE,
 	REBASE_PRESERVE,
+	REBASE_MERGES,
 	REBASE_INTERACTIVE
 };
 
 /**
  * Parses the value of --rebase. If value is a false value, returns
  * REBASE_FALSE. If value is a true value, returns REBASE_TRUE. If value is
- * "preserve", returns REBASE_PRESERVE. If value is a invalid value, dies with
- * a fatal error if fatal is true, otherwise returns REBASE_INVALID.
+ * "merges", returns REBASE_MERGES. If value is "preserve", returns
+ * REBASE_PRESERVE. If value is a invalid value, dies with a fatal error if
+ * fatal is true, otherwise returns REBASE_INVALID.
  */
 static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		int fatal)
@@ -47,6 +49,8 @@ static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		return REBASE_TRUE;
 	else if (!strcmp(value, "preserve"))
 		return REBASE_PRESERVE;
+	else if (!strcmp(value, "merges"))
+		return REBASE_MERGES;
 	else if (!strcmp(value, "interactive"))
 		return REBASE_INTERACTIVE;
 
@@ -130,7 +134,7 @@ static struct option pull_options[] = {
 	/* Options passed to git-merge or git-rebase */
 	OPT_GROUP(N_("Options related to merging")),
 	{ OPTION_CALLBACK, 'r', "rebase", &opt_rebase,
-	  "false|true|preserve|interactive",
+	  "false|true|merges|preserve|interactive",
 	  N_("incorporate changes by rebasing rather than merging"),
 	  PARSE_OPT_OPTARG, parse_opt_rebase },
 	OPT_PASSTHRU('n', NULL, &opt_diffstat, NULL,
@@ -800,7 +804,9 @@ static int run_rebase(const struct object_id *curr_head,
 	argv_push_verbosity(&args);
 
 	/* Options passed to git-rebase */
-	if (opt_rebase == REBASE_PRESERVE)
+	if (opt_rebase == REBASE_MERGES)
+		argv_array_push(&args, "--rebase-merges");
+	else if (opt_rebase == REBASE_PRESERVE)
 		argv_array_push(&args, "--preserve-merges");
 	else if (opt_rebase == REBASE_INTERACTIVE)
 		argv_array_push(&args, "--interactive");
diff --git a/builtin/remote.c b/builtin/remote.c
index 805ffc05cdb..45c9219e07a 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -245,7 +245,9 @@ static int add(int argc, const char **argv)
 struct branch_info {
 	char *remote_name;
 	struct string_list merge;
-	enum { NO_REBASE, NORMAL_REBASE, INTERACTIVE_REBASE } rebase;
+	enum {
+		NO_REBASE, NORMAL_REBASE, INTERACTIVE_REBASE, REBASE_MERGES
+	} rebase;
 };
 
 static struct string_list branch_list = STRING_LIST_INIT_NODUP;
@@ -306,6 +308,8 @@ static int config_read_branches(const char *key, const char *value, void *cb)
 				info->rebase = v;
 			else if (!strcmp(value, "preserve"))
 				info->rebase = NORMAL_REBASE;
+			else if (!strcmp(value, "merges"))
+				info->rebase = REBASE_MERGES;
 			else if (!strcmp(value, "interactive"))
 				info->rebase = INTERACTIVE_REBASE;
 		}
@@ -963,9 +967,15 @@ static int show_local_info_item(struct string_list_item *item, void *cb_data)
 
 	printf("    %-*s ", show_info->width, item->string);
 	if (branch_info->rebase) {
-		printf_ln(branch_info->rebase == INTERACTIVE_REBASE
-			  ? _("rebases interactively onto remote %s")
-			  : _("rebases onto remote %s"), merge->items[0].string);
+		const char *msg;
+		if (branch_info->rebase == INTERACTIVE_REBASE)
+			msg = _("rebases interactively onto remote %s");
+		else if (branch_info->rebase == REBASE_MERGES)
+			msg = _("rebases interactively (with merges) onto "
+				"remote %s");
+		else
+			msg = _("rebases onto remote %s");
+		printf_ln(msg, merge->items[0].string);
 		return 0;
 	} else if (show_info->any_rebase) {
 		printf_ln(_(" merges with remote %s"), merge->items[0].string);
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index d4c0a995c39..6af65155c59 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2120,7 +2120,7 @@ _git_config ()
 		return
 		;;
 	branch.*.rebase)
-		__gitcomp "false true preserve interactive"
+		__gitcomp "false true merges preserve interactive"
 		return
 		;;
 	remote.pushdefault)
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 16/17] rebase -i: introduce --rebase-merges=[no-]rebase-cousins
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (14 preceding siblings ...)
  2018-04-19 12:23           ` [PATCH v7 15/17] pull: accept --rebase=merges to recreate the branch topology Johannes Schindelin
@ 2018-04-19 12:24           ` Johannes Schindelin
  2018-04-19 12:24           ` [PATCH v7 17/17] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:24 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

This one is a bit tricky to explain, so let's try with a diagram:

        C
      /   \
A - B - E - F
  \   /
    D

To illustrate what this new mode is all about, let's consider what
happens upon `git rebase -i --rebase-merges B`, in particular to
the commit `D`. So far, the new branch structure would be:

       --- C' --
      /         \
A - B ------ E' - F'
      \    /
        D'

This is not really preserving the branch topology from before! The
reason is that the commit `D` does not have `B` as ancestor, and
therefore it gets rebased onto `B`.

This is unintuitive behavior. Even worse, when recreating branch
structure, most use cases would appear to want cousins *not* to be
rebased onto the new base commit. For example, Git for Windows (the
heaviest user of the Git garden shears, which served as the blueprint
for --rebase-merges) frequently merges branches from `next` early, and
these branches certainly do *not* want to be rebased. In the example
above, the desired outcome would look like this:

       --- C' --
      /         \
A - B ------ E' - F'
  \        /
   -- D' --

Let's introduce the term "cousins" for such commits ("D" in the
example), and let's not rebase them by default, introducing the new
"rebase-cousins" mode for use cases where they should be rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt |  7 ++++++-
 builtin/rebase--helper.c     |  9 ++++++++-
 git-rebase--interactive.sh   |  1 +
 git-rebase.sh                | 12 +++++++++++-
 sequencer.c                  |  4 ++++
 sequencer.h                  |  6 ++++++
 t/t3430-rebase-merges.sh     | 18 ++++++++++++++++++
 7 files changed, 54 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 936c5619b42..8feadf6e663 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -379,11 +379,16 @@ rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
 -r::
---rebase-merges::
+--rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
 	Rebase merge commits instead of flattening the history by replaying
 	merges. Merge conflict resolutions or manual amendments to merge
 	commits are not rebased automatically, but have to be applied
 	manually.
++
+By default, or when `no-rebase-cousins` was specified, commits which do not
+have `<upstream>` as direct ancestor will keep their original branch point.
+If the `rebase-cousins` mode is turned on, such commits are rebased onto
+`<upstream>` (or `<onto>`, if specified).
 
 -p::
 --preserve-merges::
diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index 781782e7272..f7c2a5fdc81 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
 	unsigned flags = 0, keep_empty = 0, rebase_merges = 0;
-	int abbreviate_commands = 0;
+	int abbreviate_commands = 0, rebase_cousins = -1;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
 		CHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH,
@@ -25,6 +25,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
 			N_("allow commits with empty messages")),
 		OPT_BOOL(0, "rebase-merges", &rebase_merges, N_("rebase merge commits")),
+		OPT_BOOL(0, "rebase-cousins", &rebase_cousins,
+			 N_("keep original branch points of cousins")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -59,8 +61,13 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
 	flags |= rebase_merges ? TODO_LIST_REBASE_MERGES : 0;
+	flags |= rebase_cousins > 0 ? TODO_LIST_REBASE_COUSINS : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
+	if (rebase_cousins >= 0 && !rebase_merges)
+		warning(_("--[no-]rebase-cousins has no effect without "
+			  "--rebase-merges"));
+
 	if (command == CONTINUE && argc == 1)
 		return !!sequencer_continue(&opts);
 	if (command == ABORT && argc == 1)
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 7a3daf3e40c..b4ad130e8b1 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -971,6 +971,7 @@ git_rebase__interactive () {
 
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
 		${rebase_merges:+--rebase-merges} \
+		${rebase_cousins:+--rebase-cousins} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 
diff --git a/git-rebase.sh b/git-rebase.sh
index a64460fd25a..157705d2a72 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,7 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
-r,rebase-merges!   try to rebase merges instead of skipping them
+r,rebase-merges?   try to rebase merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -90,6 +90,7 @@ state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
 rebase_merges=
+rebase_cousins=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -276,6 +277,15 @@ do
 		rebase_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
 		;;
+	--rebase-merges=*)
+		rebase_merges=t
+		case "${1#*=}" in
+		rebase-cousins) rebase_cousins=t;;
+		no-rebase-cousins) rebase_cousins=;;
+		*) die "Unknown mode: $1";;
+		esac
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/sequencer.c b/sequencer.c
index 708b8719965..3c7bb5d3fd8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3544,6 +3544,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 				   unsigned flags)
 {
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	int rebase_cousins = flags & TODO_LIST_REBASE_COUSINS;
 	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
 	struct strbuf label = STRBUF_INIT;
 	struct commit_list *commits = NULL, **tail = &commits, *iter;
@@ -3721,6 +3722,9 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 					   &commit->object.oid);
 			if (entry)
 				to = entry->string;
+			else if (!rebase_cousins)
+				to = label_oid(&commit->object.oid, NULL,
+					       &state);
 
 			if (!to || !strcmp(to, "onto"))
 				fprintf(out, "%s onto\n", cmd_reset);
diff --git a/sequencer.h b/sequencer.h
index 6bc4da17243..d9570d92b11 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -60,6 +60,12 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
 #define TODO_LIST_REBASE_MERGES (1U << 3)
+/*
+ * When rebasing merges, commits that do have the base commit as ancestor
+ * ("cousins") are *not* rebased onto the new base by default. If those
+ * commits should be rebased onto the new base, this flag needs to be passed.
+ */
+#define TODO_LIST_REBASE_COUSINS (1U << 4)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index 37c3f73784a..f2de7059830 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -153,6 +153,24 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'do not rebase cousins unless asked for' '
+	git checkout -b cousins master &&
+	before="$(git rev-parse --verify HEAD)" &&
+	test_tick &&
+	git rebase -r HEAD^ &&
+	test_cmp_rev HEAD $before &&
+	test_tick &&
+	git rebase --rebase-merges=rebase-cousins HEAD^ &&
+	test_cmp_graph HEAD^.. <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	|/
+	o H
+	EOF
+'
+
 test_expect_success 'refs/rewritten/* is worktree-local' '
 	git worktree add wt &&
 	cat >wt/script-from-scratch <<-\EOF &&
-- 
2.17.0.windows.1.4.g7e4058d72e3



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v7 17/17] rebase -i --rebase-merges: add a section to the man page
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (15 preceding siblings ...)
  2018-04-19 12:24           ` [PATCH v7 16/17] rebase -i: introduce --rebase-merges=[no-]rebase-cousins Johannes Schindelin
@ 2018-04-19 12:24           ` Johannes Schindelin
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
  17 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-19 12:24 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

The --rebase-merges mode is probably not half as intuitive to use as
its inventor hopes, so let's document it some.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt | 132 +++++++++++++++++++++++++++++++++++
 1 file changed, 132 insertions(+)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 8feadf6e663..0ff83b62821 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -389,6 +389,8 @@ By default, or when `no-rebase-cousins` was specified, commits which do not
 have `<upstream>` as direct ancestor will keep their original branch point.
 If the `rebase-cousins` mode is turned on, such commits are rebased onto
 `<upstream>` (or `<onto>`, if specified).
++
+See also REBASING MERGES below.
 
 -p::
 --preserve-merges::
@@ -787,6 +789,136 @@ The ripple effect of a "hard case" recovery is especially bad:
 'everyone' downstream from 'topic' will now have to perform a "hard
 case" recovery too!
 
+REBASING MERGES
+-----------------
+
+The interactive rebase command was originally designed to handle
+individual patch series. As such, it makes sense to exclude merge
+commits from the todo list, as the developer may have merged the
+current `master` while working on the branch, only to eventually
+rebase all the commits onto `master` (skipping the merge commits).
+
+However, there are legitimate reasons why a developer may want to
+recreate merge commits: to keep the branch structure (or "commit
+topology") when working on multiple, inter-related branches.
+
+In the following example, the developer works on a topic branch that
+refactors the way buttons are defined, and on another topic branch
+that uses that refactoring to implement a "Report a bug" button. The
+output of `git log --graph --format=%s -5` may look like this:
+
+------------
+*   Merge branch 'report-a-bug'
+|\
+| * Add the feedback button
+* | Merge branch 'refactor-button'
+|\ \
+| |/
+| * Use the Button class for all buttons
+| * Extract a generic Button class from the DownloadButton one
+------------
+
+The developer might want to rebase those commits to a newer `master`
+while keeping the branch topology, for example when the first topic
+branch is expected to be integrated into `master` much earlier than the
+second one, say, to resolve merge conflicts with changes to the
+DownloadButton class that made it into `master`.
+
+This rebase can be performed using the `--rebase-merges` option.
+It will generate a todo list looking like this:
+
+------------
+label onto
+
+# Branch: refactor-button
+reset onto
+pick 123456 Extract a generic Button class from the DownloadButton one
+pick 654321 Use the Button class for all buttons
+label refactor-button
+
+# Branch: report-a-bug
+reset refactor-button # Use the Button class for all buttons
+pick abcdef Add the feedback button
+label report-a-bug
+
+reset onto
+merge -C a1b2c3 refactor-button # Merge 'refactor-button'
+merge -C 6f5e4d report-a-bug # Merge 'report-a-bug'
+------------
+
+In contrast to a regular interactive rebase, there are `label`, `reset` and
+`merge` commands in addition to `pick` ones.
+
+The `label` command associates a label with the current HEAD when that
+command is executed. These labels are created as worktree-local refs
+(`refs/rewritten/<label>`) that will be deleted when the rebase
+finishes. That way, rebase operations in multiple worktrees linked to
+the same repository do not interfere with one another. If the `label` command
+fails, it is rescheduled immediately, with a helpful message how to proceed.
+
+The `reset` command is essentially a `git read-tree -m -u` (think: `git
+reset --hard`, but refusing to overwrite untracked files) to the
+specified revision (typically a previously-labeled one). If the `reset`
+command fails, it is rescheduled immediately, with a helpful message how to
+proceed.
+
+The `merge` command will merge the specified revision into whatever is
+HEAD at that time. With `-C <original-commit>`, the commit message of
+the specified merge commit will be used. When the `-C` is changed to
+a lower-case `-c`, the message will be opened in an editor after a
+successful merge so that the user can edit the message.
+
+If a `merge` command fails for any reason other than merge conflicts (i.e.
+when the merge operation did not even start), it is rescheduled immediately.
+
+At this time, the `merge` command will *always* use the `recursive`
+merge strategy, with no way to choose a different one. To work around
+this, an `exec` command can be used to call `git merge` explicitly,
+using the fact that the labels are worktree-local refs (the ref
+`refs/rewritten/onto` would correspond to the label `onto`).
+
+Note: the first command (`label onto`) labels the revision onto which
+the commits are rebased; The name `onto` is just a convention, as a nod
+to the `--onto` option.
+
+It is also possible to introduce completely new merge commits from scratch
+by adding a command of the form `merge <merge-head>`. This form will
+generate a tentative commit message and always open an editor to let the
+user edit it. This can be useful e.g. when a topic branch turns out to
+address more than a single concern and wants to be split into two or
+even more topic branches. Consider this todo list:
+
+------------
+pick 192837 Switch from GNU Makefiles to CMake
+pick 5a6c7e Document the switch to CMake
+pick 918273 Fix detection of OpenSSL in CMake
+pick afbecd http: add support for TLS v1.3
+pick fdbaec Fix detection of cURL in CMake on Windows
+------------
+
+The one commit in this list that is not related to CMake may very well
+have been motivated by working on fixing all those bugs introduced by
+switching to CMake, but it addresses a different concern. To split this
+branch into two topic branches, the todo list could be edited like this:
+
+------------
+label onto
+
+pick afbecd http: add support for TLS v1.3
+label tlsv1.3
+
+reset onto
+pick 192837 Switch from GNU Makefiles to CMake
+pick 918273 Fix detection of OpenSSL in CMake
+pick fdbaec Fix detection of cURL in CMake on Windows
+pick 5a6c7e Document the switch to CMake
+label cmake
+
+reset onto
+merge tlsv1.3
+merge cmake
+------------
+
 BUGS
 ----
 The todo list presented by `--preserve-merges --interactive` does not
-- 
2.17.0.windows.1.4.g7e4058d72e3

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v7 07/17] # This is a combination of 2 commits. # This is the 1st commit message:
  2018-04-19 12:20           ` [PATCH v7 07/17] # This is a combination of 2 commits. # This is the 1st commit message: Johannes Schindelin
@ 2018-04-20  5:38             ` Eric Sunshine
  2018-04-20  8:34               ` Johannes Schindelin
                                 ` (3 more replies)
  0 siblings, 4 replies; 412+ messages in thread
From: Eric Sunshine @ 2018-04-20  5:38 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

On Thu, Apr 19, 2018 at 8:20 AM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> # This is a combination of 2 commits. # This is the 1st commit message:

Botched squash/fixup?

> sequencer: introduce the `merge` command
>
> This patch is part of the effort to reimplement `--preserve-merges` with
> a substantially improved design, a design that has been developed in the
> Git for Windows project to maintain the dozens of Windows-specific patch
> series on top of upstream Git.
> [...]
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> # The commit message #2 will be skipped:
>
> # fixup! sequencer: introduce the `merge` command

Bloop.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-19  8:23                                 ` Jacob Keller
  2018-04-19 11:28                                   ` Sergey Organov
@ 2018-04-20  8:26                                   ` Johannes Schindelin
  2018-04-20 20:39                                     ` Jacob Keller
  1 sibling, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-20  8:26 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Sergey Organov, Phillip Wood, Git mailing list, Junio C Hamano,
	Stefan Beller, Philip Oakley, Eric Sunshine, Igor Djordjevic,
	Johannes Sixt

Hi Jake,

On Thu, 19 Apr 2018, Jacob Keller wrote:

> On Wed, Apr 18, 2018 at 9:24 PM, Sergey Organov <sorganov@gmail.com> wrote:
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >
> >> On Fri, 13 Apr 2018, Phillip Wood wrote:
> >>
> >>> On 12/04/18 23:02, Johannes Schindelin wrote:
> >>> >
> >>> > [...]
> >>> >
> >>> > So: the order of the 3-way merges does matter.
> >>> >
> >>> > [...]
> >>>
> >>> Those conflicts certainly look intimidating (and the ones in your later
> >>> reply with the N way merge example still look quite complicated). One
> >>> option would be just to stop and have the user resolve the conflicts
> >>> after each conflicting 3-way merge rather than at the end of all the
> >>> merges. There are some downsides: there would need to be a way to
> >>> explain to the user that this is an intermediate step (and what that
> >>> step was); the code would have to do some book keeping to know where it
> >>> had got to; and it would stop and prompt the user to resolve conflicts
> >>> more often which could be annoying but hopefully they'd be clearer to
> >>> resolve because they weren't nested.
> >>
> >> I thought about that. But as I pointed out: the order of the merges *does*
> >> matter. Otherwise we force the user to resolve conflicts that they
> >> *already* resolved during this rebase...
> >
> > How it's relevant to what Phillip suggested? How the order of taking 2
> > steps, A and B, affects an ability to stop after the first step? It's
> > still either "A,stop,B" or "B,stop,A", depending on the chosen order.
> >
> > What's the _actual_ problem here, if any?
> >
> > -- Sergey
> 
> I believe the order of the merges changes which ones cause conflicts,

That is a correct interpretation of the example I showed.

> but it's possible to generate pre-images (i.e. a set of parents to
> merge) which cause conflicts regardless of which ordering we pick, so
> I'm not sure there is a "best ordering".

In general, there is no best ordering, you are right. There is no silver
bullet.

I am not satisfied with stating that and then leaving it at that.

In the example I presented, you can see that there are common cases where
there *is* a best ordering. In the wrong order, even if you would force
the user to resolve the merge conflict in an intermediate merge (which
would introduce a nightmare for the user interface, I am sure you see
that), then the next merge would *again* show merge conflicts.

And I, for one, am *really* certain what my decision would be when offered
the two options 1) force the user to resolve merge conflicts *twice*, or
2) reorder the intermediate merges and present the user with exactly one
set of merge conflicts.

So it is irrelevant that there might not be a "best order" in the general
case, when in the common cases quite frequently there is.

It is just another example where theory disagrees with practice. Don't get
me wrong: it is good to start with theory. And likewise it is simply
necessary to continue from there, and put your theory to the test. And
then you need to turn this into something practical.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v7 07/17] # This is a combination of 2 commits. # This is the 1st commit message:
  2018-04-20  5:38             ` Eric Sunshine
@ 2018-04-20  8:34               ` Johannes Schindelin
  2018-04-20 21:06               ` [PATCH v2 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
                                 ` (2 subsequent siblings)
  3 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-20  8:34 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Hi Eric,

On Fri, 20 Apr 2018, Eric Sunshine wrote:

> On Thu, Apr 19, 2018 at 8:20 AM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > # This is a combination of 2 commits. # This is the 1st commit message:
> 
> Botched squash/fixup?

Sure was!

> > sequencer: introduce the `merge` command
> >
> > This patch is part of the effort to reimplement `--preserve-merges` with
> > a substantially improved design, a design that has been developed in the
> > Git for Windows project to maintain the dozens of Windows-specific patch
> > series on top of upstream Git.
> > [...]
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> >
> > # The commit message #2 will be skipped:
> >
> > # fixup! sequencer: introduce the `merge` command
> 
> Bloop.

Obviously, this was not processed properly (I certainly did not have this
in my editor, otherwise it would have been stripped). This is not the
first time this happened. I think there is a bug in sequencer.c (i.e. in
my partial reimplementation of git-rebase--interactive in C), but I do not
have time right now to investigate further... Will try to find some time
soon.

Or maybe I'll just install a post-rewrite hook that notifies me of this
problem so that I can investigate right when it happened...

Oh scratch that. Now I am too annoyed with what I perceive a bug in *my*
code. I'll hunt it down.

Thanks,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v7 06/17] sequencer: introduce new commands to reset the revision
  2018-04-19 12:20           ` [PATCH v7 06/17] sequencer: introduce new commands to reset the revision Johannes Schindelin
@ 2018-04-20  9:39             ` Phillip Wood
  2018-04-20 22:39               ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Phillip Wood @ 2018-04-20  9:39 UTC (permalink / raw)
  To: Johannes Schindelin, git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

On 19/04/18 13:20, Johannes Schindelin wrote:
> In the upcoming commits, we will teach the sequencer to rebase merges.
> This will be done in a very different way from the unfortunate design of
> `git rebase --preserve-merges` (which does not allow for reordering
> commits, or changing the branch topology).
> 
> The main idea is to introduce new todo list commands, to support
> labeling the current revision with a given name, resetting the current
> revision to a previous state, and  merging labeled revisions.
> 
> This idea was developed in Git for Windows' Git garden shears (that are
> used to maintain Git for Windows' "thicket of branches" on top of
> upstream Git), and this patch is part of the effort to make it available
> to a wider audience, as well as to make the entire process more robust
> (by implementing it in a safe and portable language rather than a Unix
> shell script).
> 
> This commit implements the commands to label, and to reset to, given
> revisions. The syntax is:
> 
> 	label <name>
> 	reset <name>
> 
> Internally, the `label <name>` command creates the ref
> `refs/rewritten/<name>`. This makes it possible to work with the labeled
> revisions interactively, or in a scripted fashion (e.g. via the todo
> list command `exec`).
> 
> These temporary refs are removed upon sequencer_remove_state(), so that
> even a `git rebase --abort` cleans them up.
> 
> We disallow '#' as label because that character will be used as separator
> in the upcoming `merge` command.
> 
> Later in this patch series, we will mark the `refs/rewritten/` refs as
> worktree-local, to allow for interactive rebases to be run in parallel in
> worktrees linked to the same repository.
> 
> As typos happen, a failed `label` or `reset` command will be rescheduled
> immediately. Note that this needs a little change in the original code to
> perform a reschedule: there is no commit from which to generate a patch
> here (and we will simply fall through to the regular `return res`). We
> keep that code path, though, because we will use it for the upcoming
> `merge` command, too.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  git-rebase--interactive.sh |   2 +
>  sequencer.c                | 201 +++++++++++++++++++++++++++++++++++--
>  2 files changed, 196 insertions(+), 7 deletions(-)
> 
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index e1b865f43f2..e8d3a7d7588 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -162,6 +162,8 @@ s, squash <commit> = use commit, but meld into previous commit
>  f, fixup <commit> = like \"squash\", but discard this commit's log message
>  x, exec <commit> = run command (the rest of the line) using shell
>  d, drop <commit> = remove commit
> +l, label <label> = label current HEAD with a name
> +t, reset <label> = reset HEAD to a label
>  
>  These lines can be re-ordered; they are executed from top to bottom.
>  " | git stripspace --comment-lines >>"$todo"
> diff --git a/sequencer.c b/sequencer.c
> index 01443e0f245..9e09026b594 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -23,6 +23,8 @@
>  #include "hashmap.h"
>  #include "notes-utils.h"
>  #include "sigchain.h"
> +#include "unpack-trees.h"
> +#include "worktree.h"
>  
>  #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
>  
> @@ -120,6 +122,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
>  static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
>  static GIT_PATH_FUNC(rebase_path_rewritten_pending,
>  	"rebase-merge/rewritten-pending")
> +
> +/*
> + * The path of the file listing refs that need to be deleted after the rebase
> + * finishes. This is used by the `label` command to record the need for cleanup.
> + */
> +static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
> +
>  /*
>   * The following files are written by git-rebase just after parsing the
>   * command-line (and are only consumed, not modified, by the sequencer).
> @@ -244,18 +253,33 @@ static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
>  
>  int sequencer_remove_state(struct replay_opts *opts)
>  {
> -	struct strbuf dir = STRBUF_INIT;
> +	struct strbuf buf = STRBUF_INIT;
>  	int i;
>  
> +	if (strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
> +		char *p = buf.buf;
> +		while (*p) {
> +			char *eol = strchr(p, '\n');
> +			if (eol)
> +				*eol = '\0';
> +			if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
> +				warning(_("could not delete '%s'"), p);
> +			if (!eol)
> +				break;
> +			p = eol + 1;
> +		}
> +	}
> +
>  	free(opts->gpg_sign);
>  	free(opts->strategy);
>  	for (i = 0; i < opts->xopts_nr; i++)
>  		free(opts->xopts[i]);
>  	free(opts->xopts);
>  
> -	strbuf_addstr(&dir, get_dir(opts));
> -	remove_dir_recursively(&dir, 0);
> -	strbuf_release(&dir);
> +	strbuf_reset(&buf);
> +	strbuf_addstr(&buf, get_dir(opts));
> +	remove_dir_recursively(&buf, 0);
> +	strbuf_release(&buf);
>  
>  	return 0;
>  }
> @@ -1279,6 +1303,8 @@ enum todo_command {
>  	TODO_SQUASH,
>  	/* commands that do something else than handling a single commit */
>  	TODO_EXEC,
> +	TODO_LABEL,
> +	TODO_RESET,
>  	/* commands that do nothing but are counted for reporting progress */
>  	TODO_NOOP,
>  	TODO_DROP,
> @@ -1297,6 +1323,8 @@ static struct {
>  	{ 'f', "fixup" },
>  	{ 's', "squash" },
>  	{ 'x', "exec" },
> +	{ 'l', "label" },
> +	{ 't', "reset" },
>  	{ 0,   "noop" },
>  	{ 'd', "drop" },
>  	{ 0,   NULL }
> @@ -1802,7 +1830,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>  		return error(_("missing arguments for %s"),
>  			     command_to_string(item->command));
>  
> -	if (item->command == TODO_EXEC) {
> +	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
> +	    item->command == TODO_RESET) {
>  		item->commit = NULL;
>  		item->arg = bol;
>  		item->arg_len = (int)(eol - bol);
> @@ -2465,6 +2494,158 @@ static int do_exec(const char *command_line)
>  	return status;
>  }
>  
> +static int safe_append(const char *filename, const char *fmt, ...)
> +{
> +	va_list ap;
> +	struct lock_file lock = LOCK_INIT;
> +	int fd = hold_lock_file_for_update(&lock, filename,
> +					   LOCK_REPORT_ON_ERROR);
> +	struct strbuf buf = STRBUF_INIT;
> +
> +	if (fd < 0)
> +		return -1;
> +
> +	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT) {
> +		error_errno(_("could not read '%s'"), filename);
> +		rollback_lock_file(&lock);
> +		return -1;
> +	}
> +	strbuf_complete(&buf, '\n');
> +	va_start(ap, fmt);
> +	strbuf_vaddf(&buf, fmt, ap);
> +	va_end(ap);
> +
> +	if (write_in_full(fd, buf.buf, buf.len) < 0) {
> +		error_errno(_("could not write to '%s'"), filename);
> +		strbuf_release(&buf);
> +		rollback_lock_file(&lock);
> +		return -1;
> +	}
> +	if (commit_lock_file(&lock) < 0) {
> +		strbuf_release(&buf);
> +		rollback_lock_file(&lock);
> +		return error(_("failed to finalize '%s'"), filename);
> +	}
> +
> +	strbuf_release(&buf);
> +	return 0;
> +}
> +
> +static int do_label(const char *name, int len)
> +{
> +	struct ref_store *refs = get_main_ref_store();
> +	struct ref_transaction *transaction;
> +	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
> +	struct strbuf msg = STRBUF_INIT;
> +	int ret = 0;
> +	struct object_id head_oid;
> +
> +	if (len == 1 && *name == '#')
> +		return error("Illegal label name: '%.*s'", len, name);
> +
> +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> +	strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
> +
> +	transaction = ref_store_transaction_begin(refs, &err);
> +	if (!transaction) {
> +		error("%s", err.buf);
> +		ret = -1;
> +	} else if (get_oid("HEAD", &head_oid)) {
> +		error(_("could not read HEAD"));
> +		ret = -1;
> +	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
> +					  NULL, 0, msg.buf, &err) < 0 ||
> +		   ref_transaction_commit(transaction, &err)) {
> +		error("%s", err.buf);
> +		ret = -1;
> +	}
> +	ref_transaction_free(transaction);
> +	strbuf_release(&err);
> +	strbuf_release(&msg);
> +
> +	if (!ret)
> +		ret = safe_append(rebase_path_refs_to_delete(),
> +				  "%s\n", ref_name.buf);
> +	strbuf_release(&ref_name);
> +
> +	return ret;
> +}
> +
> +static const char *reflog_message(struct replay_opts *opts,
> +	const char *sub_action, const char *fmt, ...);
> +
> +static int do_reset(const char *name, int len, struct replay_opts *opts)
> +{
> +	struct strbuf ref_name = STRBUF_INIT;
> +	struct object_id oid;
> +	struct lock_file lock = LOCK_INIT;
> +	struct tree_desc desc;
> +	struct tree *tree;
> +	struct unpack_trees_options unpack_tree_opts;
> +	int ret = 0, i;
> +
> +	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> +		return -1;
> +
> +	/* Determine the length of the label */
> +	for (i = 0; i < len; i++)
> +		if (isspace(name[i]))
> +			len = i;
> +
> +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> +	if (get_oid(ref_name.buf, &oid) &&
> +	    get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
> +		error(_("could not read '%s'"), ref_name.buf);
> +		rollback_lock_file(&lock);
> +		strbuf_release(&ref_name);
> +		return -1;
> +	}
> +
> +	memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
> +	unpack_tree_opts.head_idx = 1;
> +	unpack_tree_opts.src_index = &the_index;
> +	unpack_tree_opts.dst_index = &the_index;
> +	unpack_tree_opts.fn = oneway_merge;
> +	unpack_tree_opts.merge = 1;
> +	unpack_tree_opts.update = 1;
> +
> +	if (read_cache_unmerged()) {
> +		rollback_lock_file(&lock);
> +		strbuf_release(&ref_name);
> +		return error_resolve_conflict(_(action_name(opts)));
> +	}
> +
> +	if (!fill_tree_descriptor(&desc, &oid)) {
> +		error(_("failed to find tree of %s"), oid_to_hex(&oid));
> +		rollback_lock_file(&lock);
> +		free((void *)desc.buffer);
> +		strbuf_release(&ref_name);
> +		return -1;
> +	}
> +
> +	if (unpack_trees(1, &desc, &unpack_tree_opts)) {
> +		rollback_lock_file(&lock);
> +		free((void *)desc.buffer);
> +		strbuf_release(&ref_name);
> +		return -1;
> +	}
> +
> +	tree = parse_tree_indirect(&oid);
> +	prime_cache_tree(&the_index, tree);
> +
> +	if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
> +		ret = error(_("could not write index"));
> +	free((void *)desc.buffer);
> +
> +	if (!ret)
> +		ret = update_ref(reflog_message(opts, "reset", "'%.*s'",
> +						len, name), "HEAD", &oid,
> +				 NULL, 0, UPDATE_REFS_MSG_ON_ERR);
> +
> +	strbuf_release(&ref_name);
> +	return ret;
> +}
> +
>  static int is_final_fixup(struct todo_list *todo_list)
>  {
>  	int i = todo_list->current;
> @@ -2610,7 +2791,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  			res = do_pick_commit(item->command, item->commit,
>  					opts, is_final_fixup(todo_list));
>  			if (is_rebase_i(opts) && res < 0) {
> -				/* Reschedule */
> +reschedule:
>  				advise(_(rescheduled_advice),
>  				       get_item_line_length(todo_list,
>  							    todo_list->current),
> @@ -2639,7 +2820,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  					intend_to_amend();
>  				return error_failed_squash(item->commit, opts,
>  					item->arg_len, item->arg);
> -			} else if (res && is_rebase_i(opts))
> +			} else if (res && is_rebase_i(opts) && item->commit)
>  				return res | error_with_patch(item->commit,
>  					item->arg, item->arg_len, opts, res,
>  					item->command == TODO_REWORD);
> @@ -2665,6 +2846,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  				/* `current` will be incremented below */
>  				todo_list->current = -1;
>  			}
> +		} else if (item->command == TODO_LABEL) {
> +			if ((res = do_label(item->arg, item->arg_len)))
> +				goto reschedule;

I can see why you've implemented like this but I'm uneasy with jumping
into a block guarded with "if (item->command <= TODO_SQUASH)" when
item->command > TODO_SQUASH. I think it works OK at the moment but it's
possible that in the future someone will edit that block of code and add
something like

if (item->command == TODO_PICK)
	do_something()
else
	do_something_else()

assuming that item->command <= TODO_SQUASH because they haven't noticed
the goto jumping back into that block.

Best Wishes

Phillip


> +		} else if (item->command == TODO_RESET) {
> +			if ((res = do_reset(item->arg, item->arg_len, opts)))
> +				goto reschedule;
>  		} else if (!is_noop(item->command))
>  			return error(_("unknown command %d"), item->command);
>  
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v6 00/15] rebase -i: offer to recreate commit topology
  2018-04-20  8:26                                   ` Johannes Schindelin
@ 2018-04-20 20:39                                     ` Jacob Keller
  0 siblings, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-04-20 20:39 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Sergey Organov, Phillip Wood, Git mailing list, Junio C Hamano,
	Stefan Beller, Philip Oakley, Eric Sunshine, Igor Djordjevic,
	Johannes Sixt

On Fri, Apr 20, 2018 at 1:26 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Jake,
>
> On Thu, 19 Apr 2018, Jacob Keller wrote:
>
>> On Wed, Apr 18, 2018 at 9:24 PM, Sergey Organov <sorganov@gmail.com> wrote:
>> > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> >
>> >> On Fri, 13 Apr 2018, Phillip Wood wrote:
>> >>
>> >>> On 12/04/18 23:02, Johannes Schindelin wrote:
>> >>> >
>> >>> > [...]
>> >>> >
>> >>> > So: the order of the 3-way merges does matter.
>> >>> >
>> >>> > [...]
>> >>>
>> >>> Those conflicts certainly look intimidating (and the ones in your later
>> >>> reply with the N way merge example still look quite complicated). One
>> >>> option would be just to stop and have the user resolve the conflicts
>> >>> after each conflicting 3-way merge rather than at the end of all the
>> >>> merges. There are some downsides: there would need to be a way to
>> >>> explain to the user that this is an intermediate step (and what that
>> >>> step was); the code would have to do some book keeping to know where it
>> >>> had got to; and it would stop and prompt the user to resolve conflicts
>> >>> more often which could be annoying but hopefully they'd be clearer to
>> >>> resolve because they weren't nested.
>> >>
>> >> I thought about that. But as I pointed out: the order of the merges *does*
>> >> matter. Otherwise we force the user to resolve conflicts that they
>> >> *already* resolved during this rebase...
>> >
>> > How it's relevant to what Phillip suggested? How the order of taking 2
>> > steps, A and B, affects an ability to stop after the first step? It's
>> > still either "A,stop,B" or "B,stop,A", depending on the chosen order.
>> >
>> > What's the _actual_ problem here, if any?
>> >
>> > -- Sergey
>>
>> I believe the order of the merges changes which ones cause conflicts,
>
> That is a correct interpretation of the example I showed.
>
>> but it's possible to generate pre-images (i.e. a set of parents to
>> merge) which cause conflicts regardless of which ordering we pick, so
>> I'm not sure there is a "best ordering".
>
> In general, there is no best ordering, you are right. There is no silver
> bullet.
>
> I am not satisfied with stating that and then leaving it at that.
>
> In the example I presented, you can see that there are common cases where
> there *is* a best ordering. In the wrong order, even if you would force
> the user to resolve the merge conflict in an intermediate merge (which
> would introduce a nightmare for the user interface, I am sure you see
> that), then the next merge would *again* show merge conflicts.
>
> And I, for one, am *really* certain what my decision would be when offered
> the two options 1) force the user to resolve merge conflicts *twice*, or
> 2) reorder the intermediate merges and present the user with exactly one
> set of merge conflicts.
>
> So it is irrelevant that there might not be a "best order" in the general
> case, when in the common cases quite frequently there is.
>
> It is just another example where theory disagrees with practice. Don't get
> me wrong: it is good to start with theory. And likewise it is simply
> necessary to continue from there, and put your theory to the test. And
> then you need to turn this into something practical.
>
> Ciao,
> Dscho

I recall you suggested an approach of "try one way, if there are
conflicts, check the other way and see if it had conflicts".

And I also agree that forcing the user to resolve conflicts in the
middle of the operation is a huge nightmare of a user interface,
probably worse than the issues with nested merge conflicts.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v2 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages
  2018-04-20  5:38             ` Eric Sunshine
  2018-04-20  8:34               ` Johannes Schindelin
@ 2018-04-20 21:06               ` Johannes Schindelin
  2018-04-20 21:07                 ` [PATCH v2 1/4] rebase -i: demonstrate bugs with fixup!/squash! " Johannes Schindelin
                                   ` (3 more replies)
  2018-04-21  7:34               ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
  2018-04-27 20:48               ` [PATCH v4 " Johannes Schindelin
  3 siblings, 4 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-20 21:06 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Eric Sunshine, Stefan Beller

Eric Sunshine pointed out that I had such a commit message in
https://public-inbox.org/git/CAPig+cRrS0_nYJJY=O6cboV630sNQHPV5QGrQdD8MW-sYzNFGQ@mail.gmail.com/
and I went on a hunt to figure out how the heck this happened.

Turns out that if there is a fixup/squash chain where the *last* command
fails with merge conflicts, and we either --skip ahead or resolve the
conflict to a clean tree and then --continue, our code does not do a
final cleanup.

Contrary to my initial gut feeling, this bug was not introduced by my
rewrite in C of the core parts of rebase -i, but it looks to me as if
that bug was with us for a very long time (at least the --skip part).

The developer (read: user of rebase -i) in me says that we would want to
fast-track this, but the author of rebase -i in me says that we should
be cautious and cook this in `next` for a while.

Fixes since v1:

- Using test_i18ngrep instead of grep, because "This is a combination of
  <N> commits" is marked for translation.

- Added a patch to actually fix `rebase -i` when building with
  GETTEXT_POISON, because we used to assume that numbers are encoded as
  ASCII so that we can increment it when writing the next commit message
  in the fixup/squash chain. This also seems to be a long-standing bug
  that has been with us since the the beginning of the localization of
  rebase -i's commit messages.

- The test case now starts with test_when_finished "test_might_fail git rebase
  --abort" to be allow for failing more gently.

- Fixed grammar of 2/3 (now 3/4): thanks, Eric!

- Fixed the description of the new test case (it purported to test --continue,
  but it really tests --skip).


Johannes Schindelin (4):
  rebase -i: demonstrate bugs with fixup!/squash! commit messages
  rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON
  sequencer: leave a tell-tale when a fixup/squash failed
  rebase --skip: clean up commit message after a failed fixup/squash

 sequencer.c                | 93 ++++++++++++++++++++++++++++----------
 t/t3418-rebase-continue.sh | 22 +++++++++
 2 files changed, 92 insertions(+), 23 deletions(-)


base-commit: fe0a9eaf31dd0c349ae4308498c33a5c3794b293
Published-As: https://github.com/dscho/git/releases/tag/clean-msg-after-fixup-continue-v2
Fetch-It-Via: git fetch https://github.com/dscho/git clean-msg-after-fixup-continue-v2

Interdiff vs v1:
 diff --git a/sequencer.c b/sequencer.c
 index f067b7b24c5..881503a6463 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -1350,19 +1350,18 @@ static int update_squash_messages(enum todo_command command,
  		eol = strchrnul(buf.buf, '\n');
  		if (buf.buf[0] != comment_line_char ||
  		    (p += strcspn(p, "0123456789\n")) == eol)
 -			return error(_("unexpected 1st line of squash message:"
 -				       "\n\n\t%.*s"),
 -				     (int)(eol - buf.buf), buf.buf);
 -		count = strtol(p, NULL, 10);
 -
 -		if (count < 1)
 -			return error(_("invalid 1st line of squash message:\n"
 -				       "\n\t%.*s"),
 -				     (int)(eol - buf.buf), buf.buf);
 +			count = -1;
 +		else
 +			count = strtol(p, NULL, 10);
  
  		strbuf_addf(&header, "%c ", comment_line_char);
 -		strbuf_addf(&header,
 -			    _("This is a combination of %d commits."), ++count);
 +		if (count < 1)
 +			strbuf_addf(&header, _("This is a combination of "
 +					       "several commits."));
 +		else
 +			strbuf_addf(&header,
 +				    _("This is a combination of %d commits."),
 +				    ++count);
  		strbuf_splice(&buf, 0, eol - buf.buf, header.buf, header.len);
  		strbuf_release(&header);
  	} else {
 @@ -1405,13 +1404,22 @@ static int update_squash_messages(enum todo_command command,
  	if (command == TODO_SQUASH) {
  		unlink(rebase_path_fixup_msg());
  		strbuf_addf(&buf, "\n%c ", comment_line_char);
 -		strbuf_addf(&buf, _("This is the commit message #%d:"), count);
 +		if (count < 2)
 +			strbuf_addf(&buf, _("This is the next commit "
 +					    "message:"));
 +		else
 +			strbuf_addf(&buf, _("This is the commit message #%d:"),
 +				    count);
  		strbuf_addstr(&buf, "\n\n");
  		strbuf_addstr(&buf, body);
  	} else if (command == TODO_FIXUP) {
  		strbuf_addf(&buf, "\n%c ", comment_line_char);
 -		strbuf_addf(&buf, _("The commit message #%d will be skipped:"),
 -			    count);
 +		if (count < 2)
 +			strbuf_addf(&buf, _("The next commit message will be "
 +					    "skipped:"));
 +		else
 +			strbuf_addf(&buf, _("The commit message #%d will be "
 +					    "skipped:"), count);
  		strbuf_addstr(&buf, "\n\n");
  		strbuf_add_commented_lines(&buf, body, strlen(body));
  	} else
 diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
 index 4880bff82ff..693f92409ec 100755
 --- a/t/t3418-rebase-continue.sh
 +++ b/t/t3418-rebase-continue.sh
 @@ -88,7 +88,8 @@ test_expect_success 'rebase passes merge strategy options correctly' '
  	git rebase --continue
  '
  
 -test_expect_success '--continue after failed fixup cleans commit message' '
 +test_expect_success '--skip after failed fixup cleans commit message' '
 +	test_when_finished "test_might_fail git rebase --abort" &&
  	git checkout -b with-conflicting-fixup &&
  	test_commit wants-fixup &&
  	test_commit "fixup! wants-fixup" wants-fixup.t 1 wants-fixup-1 &&
 @@ -99,14 +100,14 @@ test_expect_success '--continue after failed fixup cleans commit message' '
  
  	: now there is a conflict, and comments in the commit message &&
  	git show HEAD >out &&
 -	grep "This is a combination of" out &&
 +	test_i18ngrep "This is a combination of" out &&
  
  	: skip and continue &&
  	git rebase --skip &&
  
  	: now the comments in the commit message should have been cleaned up &&
  	git show HEAD >out &&
 -	! grep "This is a combination of" out
 +	test_i18ngrep ! "This is a combination of" out
  '
  
  test_expect_success 'setup rerere database' '
-- 
2.17.0.windows.1.15.gaa56ade3205


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v2 1/4] rebase -i: demonstrate bugs with fixup!/squash! commit messages
  2018-04-20 21:06               ` [PATCH v2 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
@ 2018-04-20 21:07                 ` Johannes Schindelin
  2018-04-20 21:07                 ` [PATCH v2 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON Johannes Schindelin
                                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-20 21:07 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Eric Sunshine, Stefan Beller

When multiple fixup/squash commands are processed and the last one
causes merge conflicts and is skipped, we leave the "This is a
combination of ..." comments in the commit message.

Noticed by Eric Sunshine.

This regression test also demonstrates that we rely on the localized
version of

	# This is a combination of <number> commits

to contain the <number> in ASCII, which breaks under GETTEXT_POISON.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t3418-rebase-continue.sh | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
index 9214d0bb511..6ddf952b7b9 100755
--- a/t/t3418-rebase-continue.sh
+++ b/t/t3418-rebase-continue.sh
@@ -88,6 +88,28 @@ test_expect_success 'rebase passes merge strategy options correctly' '
 	git rebase --continue
 '
 
+test_expect_failure '--skip after failed fixup cleans commit message' '
+	test_when_finished "test_might_fail git rebase --abort" &&
+	git checkout -b with-conflicting-fixup &&
+	test_commit wants-fixup &&
+	test_commit "fixup! wants-fixup" wants-fixup.t 1 wants-fixup-1 &&
+	test_commit "fixup! wants-fixup" wants-fixup.t 2 wants-fixup-2 &&
+	test_commit "fixup! wants-fixup" wants-fixup.t 3 wants-fixup-3 &&
+	test_must_fail env FAKE_LINES="1 fixup 2 fixup 4" \
+		git rebase -i HEAD~4 &&
+
+	: now there is a conflict, and comments in the commit message &&
+	git show HEAD >out &&
+	test_i18ngrep "This is a combination of" out &&
+
+	: skip and continue &&
+	git rebase --skip &&
+
+	: now the comments in the commit message should have been cleaned up &&
+	git show HEAD >out &&
+	test_i18ngrep ! "This is a combination of" out
+'
+
 test_expect_success 'setup rerere database' '
 	rm -fr .git/rebase-* &&
 	git reset --hard commit-new-file-F3-on-topic-branch &&
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON
  2018-04-20 21:06               ` [PATCH v2 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
  2018-04-20 21:07                 ` [PATCH v2 1/4] rebase -i: demonstrate bugs with fixup!/squash! " Johannes Schindelin
@ 2018-04-20 21:07                 ` Johannes Schindelin
  2018-04-20 21:16                   ` Stefan Beller
  2018-04-20 21:07                 ` [PATCH v2 3/4] sequencer: leave a tell-tale when a fixup/squash failed Johannes Schindelin
  2018-04-20 21:08                 ` [PATCH v2 4/4] rebase --skip: clean up commit message after a failed fixup/squash Johannes Schindelin
  3 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-20 21:07 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Eric Sunshine, Stefan Beller

We previously relied on the localized versions of

	# This is a combination of <N> commits

(which we write into the commit messages during fixup/squash chains)
to contain <N> as ASCII.

Thisis not true in general, and certainly not in GETTEXT_POISON, as
demonstrated by the regression test we just introduced in t3418.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 36 ++++++++++++++++++++++--------------
 1 file changed, 22 insertions(+), 14 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 667f35ebdff..dc482e76a28 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1343,19 +1343,18 @@ static int update_squash_messages(enum todo_command command,
 		eol = strchrnul(buf.buf, '\n');
 		if (buf.buf[0] != comment_line_char ||
 		    (p += strcspn(p, "0123456789\n")) == eol)
-			return error(_("unexpected 1st line of squash message:"
-				       "\n\n\t%.*s"),
-				     (int)(eol - buf.buf), buf.buf);
-		count = strtol(p, NULL, 10);
-
-		if (count < 1)
-			return error(_("invalid 1st line of squash message:\n"
-				       "\n\t%.*s"),
-				     (int)(eol - buf.buf), buf.buf);
+			count = -1;
+		else
+			count = strtol(p, NULL, 10);
 
 		strbuf_addf(&header, "%c ", comment_line_char);
-		strbuf_addf(&header,
-			    _("This is a combination of %d commits."), ++count);
+		if (count < 1)
+			strbuf_addf(&header, _("This is a combination of "
+					       "several commits."));
+		else
+			strbuf_addf(&header,
+				    _("This is a combination of %d commits."),
+				    ++count);
 		strbuf_splice(&buf, 0, eol - buf.buf, header.buf, header.len);
 		strbuf_release(&header);
 	} else {
@@ -1398,13 +1397,22 @@ static int update_squash_messages(enum todo_command command,
 	if (command == TODO_SQUASH) {
 		unlink(rebase_path_fixup_msg());
 		strbuf_addf(&buf, "\n%c ", comment_line_char);
-		strbuf_addf(&buf, _("This is the commit message #%d:"), count);
+		if (count < 2)
+			strbuf_addf(&buf, _("This is the next commit "
+					    "message:"));
+		else
+			strbuf_addf(&buf, _("This is the commit message #%d:"),
+				    count);
 		strbuf_addstr(&buf, "\n\n");
 		strbuf_addstr(&buf, body);
 	} else if (command == TODO_FIXUP) {
 		strbuf_addf(&buf, "\n%c ", comment_line_char);
-		strbuf_addf(&buf, _("The commit message #%d will be skipped:"),
-			    count);
+		if (count < 2)
+			strbuf_addf(&buf, _("The next commit message will be "
+					    "skipped:"));
+		else
+			strbuf_addf(&buf, _("The commit message #%d will be "
+					    "skipped:"), count);
 		strbuf_addstr(&buf, "\n\n");
 		strbuf_add_commented_lines(&buf, body, strlen(body));
 	} else
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 3/4] sequencer: leave a tell-tale when a fixup/squash failed
  2018-04-20 21:06               ` [PATCH v2 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
  2018-04-20 21:07                 ` [PATCH v2 1/4] rebase -i: demonstrate bugs with fixup!/squash! " Johannes Schindelin
  2018-04-20 21:07                 ` [PATCH v2 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON Johannes Schindelin
@ 2018-04-20 21:07                 ` Johannes Schindelin
  2018-04-20 21:25                   ` Stefan Beller
  2018-04-20 21:08                 ` [PATCH v2 4/4] rebase --skip: clean up commit message after a failed fixup/squash Johannes Schindelin
  3 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-20 21:07 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Eric Sunshine, Stefan Beller

In the upcoming patch to clean up fixup/squash commit messages even when
skipping a final fixup/squash that failed with merge conflicts, we will
need to have some indicator what happened.

As we need to remove the message-fixup and message-squash files upon
failure, we cannot use those. So let's just write an explicit amend-type
file, containing either `fixup` or `squash`. The absence of that file
indicates that we were not in the middle of a fixup or squash when merge
conflicts were happening.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index dc482e76a28..a6a4efeaae2 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -106,6 +106,13 @@ static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
  * command is processed, this file is deleted.
  */
 static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
+/*
+ * If there was a merge conflict in a fixup/squash series, we need to
+ * record the type so that a `git rebase --skip` can clean up the commit
+ * message as appropriate. This file will contain that type (`fixup` or
+ * `squash`), and not exist otherwise.
+ */
+static GIT_PATH_FUNC(rebase_path_amend_type, "rebase-merge/amend-type")
 /*
  * When we stop at a given patch via the "edit" command, this file contains
  * the abbreviated commit name of the corresponding patch.
@@ -2400,10 +2407,20 @@ static int error_with_patch(struct commit *commit,
 static int error_failed_squash(struct commit *commit,
 	struct replay_opts *opts, int subject_len, const char *subject)
 {
+	const char *amend_type = "squash";
+
+	if (file_exists(rebase_path_fixup_msg())) {
+		unlink(rebase_path_fixup_msg());
+		amend_type = "fixup";
+	}
+	if (write_message(amend_type, strlen(amend_type),
+		       rebase_path_amend_type(), 0))
+		return error(_("could not write '%s'"),
+			     rebase_path_amend_type());
+
 	if (rename(rebase_path_squash_msg(), rebase_path_message()))
 		return error(_("could not rename '%s' to '%s'"),
 			rebase_path_squash_msg(), rebase_path_message());
-	unlink(rebase_path_fixup_msg());
 	unlink(git_path_merge_msg());
 	if (copy_file(git_path_merge_msg(), rebase_path_message(), 0666))
 		return error(_("could not copy '%s' to '%s'"),
@@ -2580,6 +2597,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
 			unlink(rebase_path_amend());
+			unlink(rebase_path_amend_type());
 			delete_ref(NULL, "REBASE_HEAD", NULL, REF_NO_DEREF);
 		}
 		if (item->command <= TODO_SQUASH) {
@@ -2807,6 +2825,7 @@ static int commit_staged_changes(struct replay_opts *opts)
 	if (run_git_commit(rebase_path_message(), opts, flags))
 		return error(_("could not commit staged changes."));
 	unlink(rebase_path_amend());
+	unlink(rebase_path_amend_type());
 	return 0;
 }
 
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 4/4] rebase --skip: clean up commit message after a failed fixup/squash
  2018-04-20 21:06               ` [PATCH v2 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
                                   ` (2 preceding siblings ...)
  2018-04-20 21:07                 ` [PATCH v2 3/4] sequencer: leave a tell-tale when a fixup/squash failed Johannes Schindelin
@ 2018-04-20 21:08                 ` Johannes Schindelin
  3 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-20 21:08 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Eric Sunshine, Stefan Beller

During a series of fixup/squash commands, the interactive rebase builds
up a commit message with comments. This will be presented to the user in
the editor if at least one of those commands was a `squash`.

However, if the last of these fixup/squash commands fails with merge
conflicts, and if the user then decides to skip it (or resolve it to a
clean worktree and then continue the rebase), the current code fails to
clean up the commit message.

This commit fixes that behavior.

The diff is best viewed with --color-moved.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c                | 36 ++++++++++++++++++++++++++++--------
 t/t3418-rebase-continue.sh |  2 +-
 2 files changed, 29 insertions(+), 9 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index a6a4efeaae2..881503a6463 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2789,17 +2789,12 @@ static int continue_single_pick(void)
 
 static int commit_staged_changes(struct replay_opts *opts)
 {
-	unsigned int flags = ALLOW_EMPTY | EDIT_MSG;
+	unsigned int flags = ALLOW_EMPTY | EDIT_MSG, is_fixup = 0, is_clean;
 
 	if (has_unstaged_changes(1))
 		return error(_("cannot rebase: You have unstaged changes."));
-	if (!has_uncommitted_changes(0)) {
-		const char *cherry_pick_head = git_path_cherry_pick_head();
 
-		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
-			return error(_("could not remove CHERRY_PICK_HEAD"));
-		return 0;
-	}
+	is_clean = !has_uncommitted_changes(0);
 
 	if (file_exists(rebase_path_amend())) {
 		struct strbuf rev = STRBUF_INIT;
@@ -2812,16 +2807,41 @@ static int commit_staged_changes(struct replay_opts *opts)
 		if (get_oid_hex(rev.buf, &to_amend))
 			return error(_("invalid contents: '%s'"),
 				rebase_path_amend());
-		if (oidcmp(&head, &to_amend))
+		if (!is_clean && oidcmp(&head, &to_amend))
 			return error(_("\nYou have uncommitted changes in your "
 				       "working tree. Please, commit them\n"
 				       "first and then run 'git rebase "
 				       "--continue' again."));
+		if (is_clean && !oidcmp(&head, &to_amend)) {
+			strbuf_reset(&rev);
+			/*
+			 * Clean tree, but we may need to finalize a
+			 * fixup/squash chain. A failed fixup/squash leaves the
+			 * file amend-type in rebase-merge/; It is okay if that
+			 * file is missing, in which case there is no such
+			 * chain to finalize.
+			 */
+			read_oneliner(&rev, rebase_path_amend_type(), 0);
+			if (!strcmp("squash", rev.buf))
+				is_fixup = TODO_SQUASH;
+			else if (!strcmp("fixup", rev.buf)) {
+				is_fixup = TODO_FIXUP;
+				flags = (flags & ~EDIT_MSG) | CLEANUP_MSG;
+			}
+		}
 
 		strbuf_release(&rev);
 		flags |= AMEND_MSG;
 	}
 
+	if (is_clean && !is_fixup) {
+		const char *cherry_pick_head = git_path_cherry_pick_head();
+
+		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
+			return error(_("could not remove CHERRY_PICK_HEAD"));
+		return 0;
+	}
+
 	if (run_git_commit(rebase_path_message(), opts, flags))
 		return error(_("could not commit staged changes."));
 	unlink(rebase_path_amend());
diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
index 6ddf952b7b9..693f92409ec 100755
--- a/t/t3418-rebase-continue.sh
+++ b/t/t3418-rebase-continue.sh
@@ -88,7 +88,7 @@ test_expect_success 'rebase passes merge strategy options correctly' '
 	git rebase --continue
 '
 
-test_expect_failure '--skip after failed fixup cleans commit message' '
+test_expect_success '--skip after failed fixup cleans commit message' '
 	test_when_finished "test_might_fail git rebase --abort" &&
 	git checkout -b with-conflicting-fixup &&
 	test_commit wants-fixup &&
-- 
2.17.0.windows.1.15.gaa56ade3205

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON
  2018-04-20 21:07                 ` [PATCH v2 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON Johannes Schindelin
@ 2018-04-20 21:16                   ` Stefan Beller
  2018-04-21  7:20                     ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Stefan Beller @ 2018-04-20 21:16 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Eric Sunshine

On Fri, Apr 20, 2018 at 2:07 PM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> We previously relied on the localized versions of
>
>         # This is a combination of <N> commits
>
> (which we write into the commit messages during fixup/squash chains)
> to contain <N> as ASCII.
>
> Thisis not true in general, and certainly not in GETTEXT_POISON, as

This is

Apart from this typo, this patch looks good.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 3/4] sequencer: leave a tell-tale when a fixup/squash failed
  2018-04-20 21:07                 ` [PATCH v2 3/4] sequencer: leave a tell-tale when a fixup/squash failed Johannes Schindelin
@ 2018-04-20 21:25                   ` Stefan Beller
  2018-04-21  7:24                     ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Stefan Beller @ 2018-04-20 21:25 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Eric Sunshine

>  static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
> +/*
> + * If there was a merge conflict in a fixup/squash series, we need to
> + * record the type so that a `git rebase --skip` can clean up the commit
> + * message as appropriate. This file will contain that type (`fixup` or
> + * `squash`), and not exist otherwise.
> + */

Thanks for the documentation here, is there some other high level doc that
describes all things to know about the internals of the rebase-merge dir
or is this the definitive guide?

> +static GIT_PATH_FUNC(rebase_path_amend_type, "rebase-merge/amend-type")
>  /*
>   * When we stop at a given patch via the "edit" command, this file contains
>   * the abbreviated commit name of the corresponding patch.
> @@ -2400,10 +2407,20 @@ static int error_with_patch(struct commit *commit,
>  static int error_failed_squash(struct commit *commit,
>         struct replay_opts *opts, int subject_len, const char *subject)
>  {
> +       const char *amend_type = "squash";
> +
> +       if (file_exists(rebase_path_fixup_msg())) {
> +               unlink(rebase_path_fixup_msg());
> +               amend_type = "fixup";
> +       }
> +       if (write_message(amend_type, strlen(amend_type),
> +                      rebase_path_amend_type(), 0))
> +               return error(_("could not write '%s'"),
> +                            rebase_path_amend_type());

Do we want to wait with unlinking rebase_path_fixup_msg()
until after we are sure there is no error returned?
I first thought so as to preserve the state as before, but
then it only signals the amend type. But we're downgrading the
amend type from "squash" to "fixup", which means that if
this error happens and the user just retries the git command
we'll end up with a "fixup", i.e. not opening their editor?

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v7 06/17] sequencer: introduce new commands to reset the revision
  2018-04-20  9:39             ` Phillip Wood
@ 2018-04-20 22:39               ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-20 22:39 UTC (permalink / raw)
  To: Phillip Wood
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Igor Djordjevic, Johannes Sixt, Sergey Organov,
	Martin Ågren

Hi Phillip,

On Fri, 20 Apr 2018, Phillip Wood wrote:

> On 19/04/18 13:20, Johannes Schindelin wrote:
>
> [... please cull long stretches of quoted mail that is not responded to ...]
>
> > @@ -2665,6 +2846,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
> >  				/* `current` will be incremented below */
> >  				todo_list->current = -1;
> >  			}
> > +		} else if (item->command == TODO_LABEL) {
> > +			if ((res = do_label(item->arg, item->arg_len)))
> > +				goto reschedule;
> 
> I can see why you've implemented like this but I'm uneasy with jumping
> into a block guarded with "if (item->command <= TODO_SQUASH)" when
> item->command > TODO_SQUASH. I think it works OK at the moment but it's
> possible that in the future someone will edit that block of code and add
> something like
> 
> if (item->command == TODO_PICK)
> 	do_something()
> else
> 	do_something_else()
> 
> assuming that item->command <= TODO_SQUASH because they haven't noticed
> the goto jumping back into that block.

I changed it by duplicating the rescheduling, as I agree that it is
somewhat dangerous what with all the code going on after the rescheduling
of a pick/fixup/squash/reword.

My plan is to go over the documentation changes once more tomorrow, with a
fresh set of eyes, and then submit the hopefully final iteration of this
patch series.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON
  2018-04-20 21:16                   ` Stefan Beller
@ 2018-04-21  7:20                     ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21  7:20 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, Junio C Hamano, Eric Sunshine

Hi Stefan,

On Fri, 20 Apr 2018, Stefan Beller wrote:

> On Fri, Apr 20, 2018 at 2:07 PM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > We previously relied on the localized versions of
> >
> >         # This is a combination of <N> commits
> >
> > (which we write into the commit messages during fixup/squash chains)
> > to contain <N> as ASCII.
> >
> > Thisis not true in general, and certainly not in GETTEXT_POISON, as
> 
> This is
> 
> Apart from this typo, this patch looks good.

Ouch. Fixed, locally.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 3/4] sequencer: leave a tell-tale when a fixup/squash failed
  2018-04-20 21:25                   ` Stefan Beller
@ 2018-04-21  7:24                     ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21  7:24 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, Junio C Hamano, Eric Sunshine

Hi Stefan,

On Fri, 20 Apr 2018, Stefan Beller wrote:

> >  static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
> > +/*
> > + * If there was a merge conflict in a fixup/squash series, we need to
> > + * record the type so that a `git rebase --skip` can clean up the commit
> > + * message as appropriate. This file will contain that type (`fixup` or
> > + * `squash`), and not exist otherwise.
> > + */
> 
> Thanks for the documentation here, is there some other high level doc that
> describes all things to know about the internals of the rebase-merge dir
> or is this the definitive guide?
> 
> > +static GIT_PATH_FUNC(rebase_path_amend_type, "rebase-merge/amend-type")
> >  /*
> >   * When we stop at a given patch via the "edit" command, this file contains
> >   * the abbreviated commit name of the corresponding patch.
> > @@ -2400,10 +2407,20 @@ static int error_with_patch(struct commit *commit,
> >  static int error_failed_squash(struct commit *commit,
> >         struct replay_opts *opts, int subject_len, const char *subject)
> >  {
> > +       const char *amend_type = "squash";
> > +
> > +       if (file_exists(rebase_path_fixup_msg())) {
> > +               unlink(rebase_path_fixup_msg());
> > +               amend_type = "fixup";
> > +       }
> > +       if (write_message(amend_type, strlen(amend_type),
> > +                      rebase_path_amend_type(), 0))
> > +               return error(_("could not write '%s'"),
> > +                            rebase_path_amend_type());
> 
> Do we want to wait with unlinking rebase_path_fixup_msg()
> until after we are sure there is no error returned?

Actually until after the rename() of `rebase_path_squash_msg()` succeeded,
you are right. I had changed the behavior unintentionally.

> I first thought so as to preserve the state as before, but
> then it only signals the amend type. But we're downgrading the
> amend type from "squash" to "fixup", which means that if
> this error happens and the user just retries the git command
> we'll end up with a "fixup", i.e. not opening their editor?

I am actually more worried about the rename() call failing... ;-) I
changed the order back to where it was before.

Thanks,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages
  2018-04-20  5:38             ` Eric Sunshine
  2018-04-20  8:34               ` Johannes Schindelin
  2018-04-20 21:06               ` [PATCH v2 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
@ 2018-04-21  7:34               ` Johannes Schindelin
  2018-04-21  7:35                 ` [PATCH v3 1/4] rebase -i: demonstrate bugs with fixup!/squash! " Johannes Schindelin
                                   ` (5 more replies)
  2018-04-27 20:48               ` [PATCH v4 " Johannes Schindelin
  3 siblings, 6 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21  7:34 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Eric Sunshine, Stefan Beller

Eric Sunshine pointed out that I had such a commit message in
https://public-inbox.org/git/CAPig+cRrS0_nYJJY=O6cboV630sNQHPV5QGrQdD8MW-sYzNFGQ@mail.gmail.com/
and I went on a hunt to figure out how the heck this happened.

Turns out that if there is a fixup/squash chain where the *last* command
fails with merge conflicts, and we either --skip ahead or resolve the
conflict to a clean tree and then --continue, our code does not do a
final cleanup.

Contrary to my initial gut feeling, this bug was not introduced by my
rewrite in C of the core parts of rebase -i, but it looks to me as if
that bug was with us for a very long time (at least the --skip part).

The developer (read: user of rebase -i) in me says that we would want to
fast-track this, but the author of rebase -i in me says that we should
be cautious and cook this in `next` for a while.

Fixes since v2 (thanks, Stefan!):

- Fixed commit message of 2/4: "Thisis" -> "This is".

- Reinstated the order where the `message-squash` file is renamed to
  `message` first, and only if that succeeded, we delete the
  `message-fixup` file.


Johannes Schindelin (4):
  rebase -i: demonstrate bugs with fixup!/squash! commit messages
  rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON
  sequencer: leave a tell-tale when a fixup/squash failed
  rebase --skip: clean up commit message after a failed fixup/squash

 sequencer.c                | 94 ++++++++++++++++++++++++++++----------
 t/t3418-rebase-continue.sh | 22 +++++++++
 2 files changed, 93 insertions(+), 23 deletions(-)


base-commit: fe0a9eaf31dd0c349ae4308498c33a5c3794b293
Published-As: https://github.com/dscho/git/releases/tag/clean-msg-after-fixup-continue-v3
Fetch-It-Via: git fetch https://github.com/dscho/git clean-msg-after-fixup-continue-v3

Interdiff vs v2:
 diff --git a/sequencer.c b/sequencer.c
 index 881503a6463..b8b72fd540f 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -2409,6 +2409,10 @@ static int error_failed_squash(struct commit *commit,
  {
  	const char *amend_type = "squash";
  
 +	if (rename(rebase_path_squash_msg(), rebase_path_message()))
 +		return error(_("could not rename '%s' to '%s'"),
 +			rebase_path_squash_msg(), rebase_path_message());
 +
  	if (file_exists(rebase_path_fixup_msg())) {
  		unlink(rebase_path_fixup_msg());
  		amend_type = "fixup";
 @@ -2418,9 +2422,6 @@ static int error_failed_squash(struct commit *commit,
  		return error(_("could not write '%s'"),
  			     rebase_path_amend_type());
  
 -	if (rename(rebase_path_squash_msg(), rebase_path_message()))
 -		return error(_("could not rename '%s' to '%s'"),
 -			rebase_path_squash_msg(), rebase_path_message());
  	unlink(git_path_merge_msg());
  	if (copy_file(git_path_merge_msg(), rebase_path_message(), 0666))
  		return error(_("could not copy '%s' to '%s'"),
-- 
2.17.0.windows.1.15.gaa56ade3205


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v3 1/4] rebase -i: demonstrate bugs with fixup!/squash! commit messages
  2018-04-21  7:34               ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
@ 2018-04-21  7:35                 ` Johannes Schindelin
  2018-04-21  7:35                 ` [PATCH v3 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON Johannes Schindelin
                                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21  7:35 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Eric Sunshine, Stefan Beller

When multiple fixup/squash commands are processed and the last one
causes merge conflicts and is skipped, we leave the "This is a
combination of ..." comments in the commit message.

Noticed by Eric Sunshine.

This regression test also demonstrates that we rely on the localized
version of

	# This is a combination of <number> commits

to contain the <number> in ASCII, which breaks under GETTEXT_POISON.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t3418-rebase-continue.sh | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
index 9214d0bb511..6ddf952b7b9 100755
--- a/t/t3418-rebase-continue.sh
+++ b/t/t3418-rebase-continue.sh
@@ -88,6 +88,28 @@ test_expect_success 'rebase passes merge strategy options correctly' '
 	git rebase --continue
 '
 
+test_expect_failure '--skip after failed fixup cleans commit message' '
+	test_when_finished "test_might_fail git rebase --abort" &&
+	git checkout -b with-conflicting-fixup &&
+	test_commit wants-fixup &&
+	test_commit "fixup! wants-fixup" wants-fixup.t 1 wants-fixup-1 &&
+	test_commit "fixup! wants-fixup" wants-fixup.t 2 wants-fixup-2 &&
+	test_commit "fixup! wants-fixup" wants-fixup.t 3 wants-fixup-3 &&
+	test_must_fail env FAKE_LINES="1 fixup 2 fixup 4" \
+		git rebase -i HEAD~4 &&
+
+	: now there is a conflict, and comments in the commit message &&
+	git show HEAD >out &&
+	test_i18ngrep "This is a combination of" out &&
+
+	: skip and continue &&
+	git rebase --skip &&
+
+	: now the comments in the commit message should have been cleaned up &&
+	git show HEAD >out &&
+	test_i18ngrep ! "This is a combination of" out
+'
+
 test_expect_success 'setup rerere database' '
 	rm -fr .git/rebase-* &&
 	git reset --hard commit-new-file-F3-on-topic-branch &&
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON
  2018-04-21  7:34               ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
  2018-04-21  7:35                 ` [PATCH v3 1/4] rebase -i: demonstrate bugs with fixup!/squash! " Johannes Schindelin
@ 2018-04-21  7:35                 ` Johannes Schindelin
  2018-04-21  7:35                 ` [PATCH v3 3/4] sequencer: leave a tell-tale when a fixup/squash failed Johannes Schindelin
                                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21  7:35 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Eric Sunshine, Stefan Beller

We previously relied on the localized versions of

	# This is a combination of <N> commits

(which we write into the commit messages during fixup/squash chains)
to contain <N> as ASCII.

This is not true in general, and certainly not in GETTEXT_POISON, as
demonstrated by the regression test we just introduced in t3418.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 36 ++++++++++++++++++++++--------------
 1 file changed, 22 insertions(+), 14 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 667f35ebdff..dc482e76a28 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1343,19 +1343,18 @@ static int update_squash_messages(enum todo_command command,
 		eol = strchrnul(buf.buf, '\n');
 		if (buf.buf[0] != comment_line_char ||
 		    (p += strcspn(p, "0123456789\n")) == eol)
-			return error(_("unexpected 1st line of squash message:"
-				       "\n\n\t%.*s"),
-				     (int)(eol - buf.buf), buf.buf);
-		count = strtol(p, NULL, 10);
-
-		if (count < 1)
-			return error(_("invalid 1st line of squash message:\n"
-				       "\n\t%.*s"),
-				     (int)(eol - buf.buf), buf.buf);
+			count = -1;
+		else
+			count = strtol(p, NULL, 10);
 
 		strbuf_addf(&header, "%c ", comment_line_char);
-		strbuf_addf(&header,
-			    _("This is a combination of %d commits."), ++count);
+		if (count < 1)
+			strbuf_addf(&header, _("This is a combination of "
+					       "several commits."));
+		else
+			strbuf_addf(&header,
+				    _("This is a combination of %d commits."),
+				    ++count);
 		strbuf_splice(&buf, 0, eol - buf.buf, header.buf, header.len);
 		strbuf_release(&header);
 	} else {
@@ -1398,13 +1397,22 @@ static int update_squash_messages(enum todo_command command,
 	if (command == TODO_SQUASH) {
 		unlink(rebase_path_fixup_msg());
 		strbuf_addf(&buf, "\n%c ", comment_line_char);
-		strbuf_addf(&buf, _("This is the commit message #%d:"), count);
+		if (count < 2)
+			strbuf_addf(&buf, _("This is the next commit "
+					    "message:"));
+		else
+			strbuf_addf(&buf, _("This is the commit message #%d:"),
+				    count);
 		strbuf_addstr(&buf, "\n\n");
 		strbuf_addstr(&buf, body);
 	} else if (command == TODO_FIXUP) {
 		strbuf_addf(&buf, "\n%c ", comment_line_char);
-		strbuf_addf(&buf, _("The commit message #%d will be skipped:"),
-			    count);
+		if (count < 2)
+			strbuf_addf(&buf, _("The next commit message will be "
+					    "skipped:"));
+		else
+			strbuf_addf(&buf, _("The commit message #%d will be "
+					    "skipped:"), count);
 		strbuf_addstr(&buf, "\n\n");
 		strbuf_add_commented_lines(&buf, body, strlen(body));
 	} else
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 3/4] sequencer: leave a tell-tale when a fixup/squash failed
  2018-04-21  7:34               ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
  2018-04-21  7:35                 ` [PATCH v3 1/4] rebase -i: demonstrate bugs with fixup!/squash! " Johannes Schindelin
  2018-04-21  7:35                 ` [PATCH v3 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON Johannes Schindelin
@ 2018-04-21  7:35                 ` Johannes Schindelin
  2018-04-21  7:39                 ` [PATCH v3 4/4] rebase --skip: clean up commit message after a failed fixup/squash Johannes Schindelin
                                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21  7:35 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Eric Sunshine, Stefan Beller

In the upcoming patch to clean up fixup/squash commit messages even when
skipping a final fixup/squash that failed with merge conflicts, we will
need to have some indicator what happened.

As we need to remove the message-fixup and message-squash files upon
failure, we cannot use those. So let's just write an explicit amend-type
file, containing either `fixup` or `squash`. The absence of that file
indicates that we were not in the middle of a fixup or squash when merge
conflicts were happening.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index dc482e76a28..9a85b705a84 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -106,6 +106,13 @@ static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
  * command is processed, this file is deleted.
  */
 static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
+/*
+ * If there was a merge conflict in a fixup/squash series, we need to
+ * record the type so that a `git rebase --skip` can clean up the commit
+ * message as appropriate. This file will contain that type (`fixup` or
+ * `squash`), and not exist otherwise.
+ */
+static GIT_PATH_FUNC(rebase_path_amend_type, "rebase-merge/amend-type")
 /*
  * When we stop at a given patch via the "edit" command, this file contains
  * the abbreviated commit name of the corresponding patch.
@@ -2400,10 +2407,21 @@ static int error_with_patch(struct commit *commit,
 static int error_failed_squash(struct commit *commit,
 	struct replay_opts *opts, int subject_len, const char *subject)
 {
+	const char *amend_type = "squash";
+
 	if (rename(rebase_path_squash_msg(), rebase_path_message()))
 		return error(_("could not rename '%s' to '%s'"),
 			rebase_path_squash_msg(), rebase_path_message());
-	unlink(rebase_path_fixup_msg());
+
+	if (file_exists(rebase_path_fixup_msg())) {
+		unlink(rebase_path_fixup_msg());
+		amend_type = "fixup";
+	}
+	if (write_message(amend_type, strlen(amend_type),
+		       rebase_path_amend_type(), 0))
+		return error(_("could not write '%s'"),
+			     rebase_path_amend_type());
+
 	unlink(git_path_merge_msg());
 	if (copy_file(git_path_merge_msg(), rebase_path_message(), 0666))
 		return error(_("could not copy '%s' to '%s'"),
@@ -2580,6 +2598,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
 			unlink(rebase_path_amend());
+			unlink(rebase_path_amend_type());
 			delete_ref(NULL, "REBASE_HEAD", NULL, REF_NO_DEREF);
 		}
 		if (item->command <= TODO_SQUASH) {
@@ -2807,6 +2826,7 @@ static int commit_staged_changes(struct replay_opts *opts)
 	if (run_git_commit(rebase_path_message(), opts, flags))
 		return error(_("could not commit staged changes."));
 	unlink(rebase_path_amend());
+	unlink(rebase_path_amend_type());
 	return 0;
 }
 
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 4/4] rebase --skip: clean up commit message after a failed fixup/squash
  2018-04-21  7:34               ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
                                   ` (2 preceding siblings ...)
  2018-04-21  7:35                 ` [PATCH v3 3/4] sequencer: leave a tell-tale when a fixup/squash failed Johannes Schindelin
@ 2018-04-21  7:39                 ` Johannes Schindelin
  2018-04-23 18:11                 ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Stefan Beller
  2018-04-24  1:28                 ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" " Junio C Hamano
  5 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21  7:39 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Eric Sunshine, Stefan Beller

During a series of fixup/squash commands, the interactive rebase builds
up a commit message with comments. This will be presented to the user in
the editor if at least one of those commands was a `squash`.

However, if the last of these fixup/squash commands fails with merge
conflicts, and if the user then decides to skip it (or resolve it to a
clean worktree and then continue the rebase), the current code fails to
clean up the commit message.

This commit fixes that behavior.

The diff is best viewed with --color-moved.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c                | 36 ++++++++++++++++++++++++++++--------
 t/t3418-rebase-continue.sh |  2 +-
 2 files changed, 29 insertions(+), 9 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 9a85b705a84..b8b72fd540f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2790,17 +2790,12 @@ static int continue_single_pick(void)
 
 static int commit_staged_changes(struct replay_opts *opts)
 {
-	unsigned int flags = ALLOW_EMPTY | EDIT_MSG;
+	unsigned int flags = ALLOW_EMPTY | EDIT_MSG, is_fixup = 0, is_clean;
 
 	if (has_unstaged_changes(1))
 		return error(_("cannot rebase: You have unstaged changes."));
-	if (!has_uncommitted_changes(0)) {
-		const char *cherry_pick_head = git_path_cherry_pick_head();
 
-		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
-			return error(_("could not remove CHERRY_PICK_HEAD"));
-		return 0;
-	}
+	is_clean = !has_uncommitted_changes(0);
 
 	if (file_exists(rebase_path_amend())) {
 		struct strbuf rev = STRBUF_INIT;
@@ -2813,16 +2808,41 @@ static int commit_staged_changes(struct replay_opts *opts)
 		if (get_oid_hex(rev.buf, &to_amend))
 			return error(_("invalid contents: '%s'"),
 				rebase_path_amend());
-		if (oidcmp(&head, &to_amend))
+		if (!is_clean && oidcmp(&head, &to_amend))
 			return error(_("\nYou have uncommitted changes in your "
 				       "working tree. Please, commit them\n"
 				       "first and then run 'git rebase "
 				       "--continue' again."));
+		if (is_clean && !oidcmp(&head, &to_amend)) {
+			strbuf_reset(&rev);
+			/*
+			 * Clean tree, but we may need to finalize a
+			 * fixup/squash chain. A failed fixup/squash leaves the
+			 * file amend-type in rebase-merge/; It is okay if that
+			 * file is missing, in which case there is no such
+			 * chain to finalize.
+			 */
+			read_oneliner(&rev, rebase_path_amend_type(), 0);
+			if (!strcmp("squash", rev.buf))
+				is_fixup = TODO_SQUASH;
+			else if (!strcmp("fixup", rev.buf)) {
+				is_fixup = TODO_FIXUP;
+				flags = (flags & ~EDIT_MSG) | CLEANUP_MSG;
+			}
+		}
 
 		strbuf_release(&rev);
 		flags |= AMEND_MSG;
 	}
 
+	if (is_clean && !is_fixup) {
+		const char *cherry_pick_head = git_path_cherry_pick_head();
+
+		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
+			return error(_("could not remove CHERRY_PICK_HEAD"));
+		return 0;
+	}
+
 	if (run_git_commit(rebase_path_message(), opts, flags))
 		return error(_("could not commit staged changes."));
 	unlink(rebase_path_amend());
diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
index 6ddf952b7b9..693f92409ec 100755
--- a/t/t3418-rebase-continue.sh
+++ b/t/t3418-rebase-continue.sh
@@ -88,7 +88,7 @@ test_expect_success 'rebase passes merge strategy options correctly' '
 	git rebase --continue
 '
 
-test_expect_failure '--skip after failed fixup cleans commit message' '
+test_expect_success '--skip after failed fixup cleans commit message' '
 	test_when_finished "test_might_fail git rebase --abort" &&
 	git checkout -b with-conflicting-fixup &&
 	test_commit wants-fixup &&
-- 
2.17.0.windows.1.15.gaa56ade3205

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges
  2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
                             ` (16 preceding siblings ...)
  2018-04-19 12:24           ` [PATCH v7 17/17] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
@ 2018-04-21 10:29           ` Johannes Schindelin
  2018-04-21 10:30             ` [PATCH v8 01/16] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
                               ` (16 more replies)
  17 siblings, 17 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:29 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Junio, I think this is now ready for `next`. Thank you for your patience
and help with this.

Once upon a time, I dreamed of an interactive rebase that would not
linearize all patches and drop all merge commits, but instead recreate
the commit topology faithfully.

My original attempt was --preserve-merges, but that design was so
limited that I did not even enable it in interactive mode.

Subsequently, it *was* enabled in interactive mode, with the predictable
consequences: as the --preserve-merges design does not allow for
specifying the parents of merge commits explicitly, all the new commits'
parents are defined *implicitly* by the previous commit history, and
hence it is *not possible to even reorder commits*.

This design flaw cannot be fixed. Not without a complete re-design, at
least. This patch series offers such a re-design.

Think of --rebase-merges as "--preserve-merges done right". It
introduces new verbs for the todo list, `label`, `reset` and `merge`.
For a commit topology like this:

            A - B - C
              \   /
                D

the generated todo list would look like this:

            # branch D
            pick 0123 A
            label branch-point
            pick 1234 D
            label D

            reset branch-point
            pick 2345 B
            merge -C 3456 D # C

There are more patches in the pipeline, based on this patch series, but
left for later in the interest of reviewable patch series: one mini
series to use the sequencer even for `git rebase -i --root`, and another
one to add support for octopus merges to --rebase-merges. And then one
to allow for rebasing merge commits in a smarter way (this one will need
a bit more work, though, as it can result in very complicated, nested
merge conflicts *very* easily).

Changes since v7:

- Touched up all the documentation (it was a mistake to copy-edit the
  --preserve-merges description, for example).

- Disentangled the rescheduling of label/reset/merge from the one of the
  pick/fixup/squash code path (thanks Phillip!).

- When the merge failed, we now write out .git/rebase-merge/patch.

- An `exec git cherry-pick` or `exec git revert` will no longer mess
  with refs/rewritten/ in sequencer_remove_state() (d'oh....).


Johannes Schindelin (14):
  sequencer: avoid using errno clobbered by rollback_lock_file()
  sequencer: make rearrange_squash() a bit more obvious
  sequencer: refactor how original todo list lines are accessed
  sequencer: offer helpful advice when a command was rescheduled
  sequencer: introduce the `merge` command
  sequencer: fast-forward `merge` commands, if possible
  rebase-helper --make-script: introduce a flag to rebase merges
  rebase: introduce the --rebase-merges option
  sequencer: make refs generated by the `label` command worktree-local
  sequencer: handle post-rewrite for merge commands
  rebase --rebase-merges: avoid "empty merges"
  pull: accept --rebase=merges to recreate the branch topology
  rebase -i: introduce --rebase-merges=[no-]rebase-cousins
  rebase -i --rebase-merges: add a section to the man page

Phillip Wood (1):
  rebase --rebase-merges: add test for --keep-empty

Stefan Beller (1):
  git-rebase--interactive: clarify arguments

 Documentation/config.txt               |   8 +
 Documentation/git-pull.txt             |   6 +-
 Documentation/git-rebase.txt           | 160 ++++-
 builtin/pull.c                         |  14 +-
 builtin/rebase--helper.c               |  13 +-
 builtin/remote.c                       |  18 +-
 contrib/completion/git-completion.bash |   4 +-
 git-rebase--interactive.sh             |  22 +-
 git-rebase.sh                          |  16 +
 refs.c                                 |   3 +-
 sequencer.c                            | 891 +++++++++++++++++++++++--
 sequencer.h                            |   7 +
 t/t3421-rebase-topology-linear.sh      |   1 +
 t/t3430-rebase-merges.sh               | 244 +++++++
 14 files changed, 1347 insertions(+), 60 deletions(-)
 create mode 100755 t/t3430-rebase-merges.sh


base-commit: fe0a9eaf31dd0c349ae4308498c33a5c3794b293
Published-As: https://github.com/dscho/git/releases/tag/recreate-merges-v8
Fetch-It-Via: git fetch https://github.com/dscho/git recreate-merges-v8

Interdiff vs v7:
 diff --git a/Documentation/config.txt b/Documentation/config.txt
 index da46f154bb3..d6bcb5dcb67 100644
 --- a/Documentation/config.txt
 +++ b/Documentation/config.txt
 @@ -1059,8 +1059,8 @@ branch.<name>.rebase::
  	branch-specific manner.
  +
  When `merges`, pass the `--rebase-merges` option to 'git rebase'
 -so that locally committed merge commits will not be flattened
 -by running 'git pull'.
 +so that the local merge commits are included in the rebase (see
 +linkgit:git-rebase[1] for details).
  +
  When preserve, also pass `--preserve-merges` along to 'git rebase'
  so that locally committed merge commits will not be flattened
 @@ -2622,8 +2622,8 @@ pull.rebase::
  	per-branch basis.
  +
  When `merges`, pass the `--rebase-merges` option to 'git rebase'
 -so that locally committed merge commits will not be flattened
 -by running 'git pull'.
 +so that the local merge commits are included in the rebase (see
 +linkgit:git-rebase[1] for details).
  +
  When preserve, also pass `--preserve-merges` along to 'git rebase'
  so that locally committed merge commits will not be flattened
 diff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt
 index 6f76d815dd3..4e0ad6fd8e0 100644
 --- a/Documentation/git-pull.txt
 +++ b/Documentation/git-pull.txt
 @@ -109,7 +109,8 @@ include::merge-options.txt[]
  	to avoid rebasing non-local changes.
  +
  When set to `merges`, rebase using `git rebase --rebase-merges` so that
 -locally created merge commits will not be flattened.
 +the local merge commits are included in the rebase (see
 +linkgit:git-rebase[1] for details).
  +
  When set to preserve, rebase with the `--preserve-merges` option passed
  to `git rebase` so that locally created merge commits will not be flattened.
 diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
 index 0ff83b62821..3b996e46d6a 100644
 --- a/Documentation/git-rebase.txt
 +++ b/Documentation/git-rebase.txt
 @@ -380,15 +380,25 @@ have the long commit hash prepended to the format.
  
  -r::
  --rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
 -	Rebase merge commits instead of flattening the history by replaying
 -	merges. Merge conflict resolutions or manual amendments to merge
 -	commits are not rebased automatically, but have to be applied
 +	By default, a rebase will simply drop merge commits and only rebase
 +	the non-merge commits. With this option, it will try to preserve
 +	the branching structure within the commits that are to be rebased,
 +	by recreating the merge commits. If a merge commit resolved any merge
 +	or contained manual amendments, then they will have to be re-applied
  	manually.
  +
  By default, or when `no-rebase-cousins` was specified, commits which do not
  have `<upstream>` as direct ancestor will keep their original branch point.
 -If the `rebase-cousins` mode is turned on, such commits are rebased onto
 -`<upstream>` (or `<onto>`, if specified).
 +If the `rebase-cousins` mode is turned on, such commits are instead rebased
 +onto `<upstream>` (or `<onto>`, if specified).
 ++
 +This mode is similar in spirit to `--preserve-merges`, but in contrast to
 +that option works well in interactive rebases: commits can be reordered,
 +inserted and dropped at will.
 ++
 +It is currently only possible to recreate the merge commits using the
 +`recursive` merge strategy; Different merge strategies can be used only via
 +explicit `exec git merge -s <strategy> [...]` commands.
  +
  See also REBASING MERGES below.
  
 @@ -795,8 +805,9 @@ REBASING MERGES
  The interactive rebase command was originally designed to handle
  individual patch series. As such, it makes sense to exclude merge
  commits from the todo list, as the developer may have merged the
 -current `master` while working on the branch, only to eventually
 -rebase all the commits onto `master` (skipping the merge commits).
 +then-current `master` while working on the branch, only to rebase
 +all the commits onto `master` eventually (skipping the merge
 +commits).
  
  However, there are legitimate reasons why a developer may want to
  recreate merge commits: to keep the branch structure (or "commit
 @@ -846,21 +857,23 @@ merge -C a1b2c3 refactor-button # Merge 'refactor-button'
  merge -C 6f5e4d report-a-bug # Merge 'report-a-bug'
  ------------
  
 -In contrast to a regular interactive rebase, there are `label`, `reset` and
 -`merge` commands in addition to `pick` ones.
 +In contrast to a regular interactive rebase, there are `label`, `reset`
 +and `merge` commands in addition to `pick` ones.
  
  The `label` command associates a label with the current HEAD when that
  command is executed. These labels are created as worktree-local refs
  (`refs/rewritten/<label>`) that will be deleted when the rebase
  finishes. That way, rebase operations in multiple worktrees linked to
 -the same repository do not interfere with one another. If the `label` command
 -fails, it is rescheduled immediately, with a helpful message how to proceed.
 +the same repository do not interfere with one another. If the `label`
 +command fails, it is rescheduled immediately, with a helpful message how
 +to proceed.
  
 -The `reset` command is essentially a `git read-tree -m -u` (think: `git
 -reset --hard`, but refusing to overwrite untracked files) to the
 -specified revision (typically a previously-labeled one). If the `reset`
 -command fails, it is rescheduled immediately, with a helpful message how to
 -proceed.
 +The `reset` command resets the HEAD, index and worktree to the specified
 +revision. It is isimilar to an `exec git reset --hard <label>`, but
 +refuses to overwrite untracked files. If the `reset` command fails, it is
 +rescheduled immediately, with a helpful message how to edit the todo list
 +(this typically happens when a `reset` command was inserted into the todo
 +list manually and contains a typo).
  
  The `merge` command will merge the specified revision into whatever is
  HEAD at that time. With `-C <original-commit>`, the commit message of
 @@ -875,7 +888,7 @@ At this time, the `merge` command will *always* use the `recursive`
  merge strategy, with no way to choose a different one. To work around
  this, an `exec` command can be used to call `git merge` explicitly,
  using the fact that the labels are worktree-local refs (the ref
 -`refs/rewritten/onto` would correspond to the label `onto`).
 +`refs/rewritten/onto` would correspond to the label `onto`, for example).
  
  Note: the first command (`label onto`) labels the revision onto which
  the commits are rebased; The name `onto` is just a convention, as a nod
 @@ -925,7 +938,7 @@ The todo list presented by `--preserve-merges --interactive` does not
  represent the topology of the revision graph.  Editing commits and
  rewording their commit messages should work fine, but attempts to
  reorder commits tend to produce counterintuitive results. Use
 ---rebase-merges for a more faithful representation.
 +`--rebase-merges` in such scenarios instead.
  
  For example, an attempt to rearrange
  ------------
 diff --git a/sequencer.c b/sequencer.c
 index 3c7bb5d3fd8..9ffadbb3d3c 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -258,7 +258,8 @@ int sequencer_remove_state(struct replay_opts *opts)
  	struct strbuf buf = STRBUF_INIT;
  	int i;
  
 -	if (strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
 +	if (is_rebase_i(opts) &&
 +	    strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
  		char *p = buf.buf;
  		while (*p) {
  			char *eol = strchr(p, '\n');
 @@ -2960,7 +2961,7 @@ N_("Could not execute the todo command\n"
  
  static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  {
 -	int res = 0;
 +	int res = 0, reschedule = 0;
  
  	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
  	if (opts->allow_ff)
 @@ -3002,7 +3003,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			res = do_pick_commit(item->command, item->commit,
  					opts, is_final_fixup(todo_list));
  			if (is_rebase_i(opts) && res < 0) {
 -reschedule:
 +				/* Reschedule */
  				advise(_(rescheduled_advice),
  				       get_item_line_length(todo_list,
  							    todo_list->current),
 @@ -3059,21 +3060,42 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			}
  		} else if (item->command == TODO_LABEL) {
  			if ((res = do_label(item->arg, item->arg_len)))
 -				goto reschedule;
 +				reschedule = 1;
  		} else if (item->command == TODO_RESET) {
  			if ((res = do_reset(item->arg, item->arg_len, opts)))
 -				goto reschedule;
 +				reschedule = 1;
  		} else if (item->command == TODO_MERGE) {
 -			res = do_merge(item->commit, item->arg, item->arg_len,
 -				       item->flags, opts);
 -			if (res < 0)
 -				goto reschedule;
 -			if (item->commit)
 +			if ((res = do_merge(item->commit,
 +					    item->arg, item->arg_len,
 +					    item->flags, opts)) < 0)
 +				reschedule = 1;
 +			else if (item->commit)
  				record_in_rewritten(&item->commit->object.oid,
  						    peek_command(todo_list, 1));
 +			if (res > 0)
 +				/* failed with merge conflicts */
 +				return error_with_patch(item->commit,
 +							item->arg,
 +							item->arg_len, opts,
 +							res, 0);
  		} else if (!is_noop(item->command))
  			return error(_("unknown command %d"), item->command);
  
 +		if (reschedule) {
 +			advise(_(rescheduled_advice),
 +			       get_item_line_length(todo_list,
 +						    todo_list->current),
 +			       get_item_line(todo_list, todo_list->current));
 +			todo_list->current--;
 +			if (save_todo(todo_list, opts))
 +				return -1;
 +			if (item->commit)
 +				return error_with_patch(item->commit,
 +							item->arg,
 +							item->arg_len, opts,
 +							res, 0);
 +		}
 +
  		todo_list->current++;
  		if (res)
  			return res;
 diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
 index f2de7059830..3d4dfdf7bec 100755
 --- a/t/t3430-rebase-merges.sh
 +++ b/t/t3430-rebase-merges.sh
 @@ -125,6 +125,29 @@ test_expect_success '`reset` refuses to overwrite untracked files' '
  	git rebase --abort
  '
  
 +test_expect_success 'failed `merge` writes patch (may be rescheduled, too)' '
 +	test_when_finished "test_might_fail git rebase --abort" &&
 +	git checkout -b conflicting-merge A &&
 +
 +	: fail because of conflicting untracked file &&
 +	>G.t &&
 +	echo "merge -C H G" >script-from-scratch &&
 +	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
 +	test_tick &&
 +	test_must_fail git rebase -ir HEAD &&
 +	grep "^merge -C .* G$" .git/rebase-merge/done &&
 +	grep "^merge -C .* G$" .git/rebase-merge/git-rebase-todo &&
 +	test_path_is_file .git/rebase-merge/patch &&
 +
 +	: fail because of merge conflict &&
 +	rm G.t .git/rebase-merge/patch &&
 +	git reset --hard &&
 +	test_commit conflicting-G G.t not-G conflicting-G &&
 +	test_must_fail git rebase --continue &&
 +	! grep "^merge -C .* G$" .git/rebase-merge/git-rebase-todo &&
 +	test_path_is_file .git/rebase-merge/patch
 +'
 +
  test_expect_success 'with a branch tip that was cherry-picked already' '
  	git checkout -b already-upstream master &&
  	base="$(git rev-parse --verify HEAD)" &&
-- 
2.17.0.windows.1.15.gaa56ade3205


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v8 01/16] sequencer: avoid using errno clobbered by rollback_lock_file()
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
@ 2018-04-21 10:30             ` Johannes Schindelin
  2018-04-21 10:30             ` [PATCH v8 02/16] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
                               ` (15 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

As pointed out in a review of the `--rebase-merges` patch series,
`rollback_lock_file()` clobbers errno. Therefore, we have to report the
error message that uses errno before calling said function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 667f35ebdff..096e6d241e0 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -345,12 +345,14 @@ static int write_message(const void *buf, size_t len, const char *filename,
 	if (msg_fd < 0)
 		return error_errno(_("could not lock '%s'"), filename);
 	if (write_in_full(msg_fd, buf, len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write to '%s'"), filename);
+		return -1;
 	}
 	if (append_eol && write(msg_fd, "\n", 1) < 0) {
+		error_errno(_("could not write eol to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write eol to '%s'"), filename);
+		return -1;
 	}
 	if (commit_lock_file(&msg_file) < 0)
 		return error(_("failed to finalize '%s'"), filename);
@@ -2119,9 +2121,9 @@ static int save_head(const char *head)
 	written = write_in_full(fd, buf.buf, buf.len);
 	strbuf_release(&buf);
 	if (written < 0) {
+		error_errno(_("could not write to '%s'"), git_path_head_file());
 		rollback_lock_file(&head_lock);
-		return error_errno(_("could not write to '%s'"),
-				   git_path_head_file());
+		return -1;
 	}
 	if (commit_lock_file(&head_lock) < 0)
 		return error(_("failed to finalize '%s'"), git_path_head_file());
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 02/16] sequencer: make rearrange_squash() a bit more obvious
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
  2018-04-21 10:30             ` [PATCH v8 01/16] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
@ 2018-04-21 10:30             ` Johannes Schindelin
  2018-04-21 10:31             ` [PATCH v8 03/16] sequencer: refactor how original todo list lines are accessed Johannes Schindelin
                               ` (14 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:30 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

There are some commands that have to be skipped from rearranging by virtue
of not handling any commits.

However, the logic was not quite obvious: it skipped commands based on
their position in the enum todo_command.

Instead, let's make it explicit that we skip all commands that do not
handle any commit. With one exception: the `drop` command, because it,
well, drops the commit and is therefore not eligible to rearranging.

Note: this is a bit academic at the moment because the only time we call
`rearrange_squash()` is directly after generating the todo list, when we
have nothing but `pick` commands anyway.

However, the upcoming `merge` command *will* want to be handled by that
function, and it *can* handle commits.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 096e6d241e0..1ee70d843c1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3393,7 +3393,7 @@ int rearrange_squash(void)
 		struct subject2item_entry *entry;
 
 		next[i] = tail[i] = -1;
-		if (item->command >= TODO_EXEC) {
+		if (!item->commit || item->command == TODO_DROP) {
 			subjects[i] = NULL;
 			continue;
 		}
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 03/16] sequencer: refactor how original todo list lines are accessed
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
  2018-04-21 10:30             ` [PATCH v8 01/16] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
  2018-04-21 10:30             ` [PATCH v8 02/16] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
@ 2018-04-21 10:31             ` Johannes Schindelin
  2018-04-21 10:31             ` [PATCH v8 04/16] sequencer: offer helpful advice when a command was rescheduled Johannes Schindelin
                               ` (13 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:31 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Previously, we did a lot of arithmetic gymnastics to get at the line in
the todo list (as stored in todo_list.buf). This might have been fast,
but only in terms of execution speed, not in terms of developer time.

Let's refactor this to make it a lot easier to read, and hence to
reason about the correctness of the code. It is not performance-critical
code anyway.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 60 ++++++++++++++++++++++++++++++++---------------------
 1 file changed, 36 insertions(+), 24 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 1ee70d843c1..3d0a45ab25a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1870,6 +1870,23 @@ static int count_commands(struct todo_list *todo_list)
 	return count;
 }
 
+static int get_item_line_offset(struct todo_list *todo_list, int index)
+{
+	return index < todo_list->nr ?
+		todo_list->items[index].offset_in_buf : todo_list->buf.len;
+}
+
+static const char *get_item_line(struct todo_list *todo_list, int index)
+{
+	return todo_list->buf.buf + get_item_line_offset(todo_list, index);
+}
+
+static int get_item_line_length(struct todo_list *todo_list, int index)
+{
+	return get_item_line_offset(todo_list, index + 1)
+		-  get_item_line_offset(todo_list, index);
+}
+
 static ssize_t strbuf_read_file_or_whine(struct strbuf *sb, const char *path)
 {
 	int fd;
@@ -2244,29 +2261,27 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
 		return error_errno(_("could not lock '%s'"), todo_path);
-	offset = next < todo_list->nr ?
-		todo_list->items[next].offset_in_buf : todo_list->buf.len;
+	offset = get_item_line_offset(todo_list, next);
 	if (write_in_full(fd, todo_list->buf.buf + offset,
 			todo_list->buf.len - offset) < 0)
 		return error_errno(_("could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
 		return error(_("failed to finalize '%s'"), todo_path);
 
-	if (is_rebase_i(opts)) {
-		const char *done_path = rebase_path_done();
-		int fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
-		int prev_offset = !next ? 0 :
-			todo_list->items[next - 1].offset_in_buf;
+	if (is_rebase_i(opts) && next > 0) {
+		const char *done = rebase_path_done();
+		int fd = open(done, O_CREAT | O_WRONLY | O_APPEND, 0666);
+		int ret = 0;
 
-		if (fd >= 0 && offset > prev_offset &&
-		    write_in_full(fd, todo_list->buf.buf + prev_offset,
-				  offset - prev_offset) < 0) {
-			close(fd);
-			return error_errno(_("could not write to '%s'"),
-					   done_path);
-		}
-		if (fd >= 0)
-			close(fd);
+		if (fd < 0)
+			return 0;
+		if (write_in_full(fd, get_item_line(todo_list, next - 1),
+				  get_item_line_length(todo_list, next - 1))
+		    < 0)
+			ret = error_errno(_("could not write to '%s'"), done);
+		if (close(fd) < 0)
+			ret = error_errno(_("failed to finalize '%s'"), done);
+		return ret;
 	}
 	return 0;
 }
@@ -3297,8 +3312,7 @@ int skip_unnecessary_picks(void)
 		oid = &item->commit->object.oid;
 	}
 	if (i > 0) {
-		int offset = i < todo_list.nr ?
-			todo_list.items[i].offset_in_buf : todo_list.buf.len;
+		int offset = get_item_line_offset(&todo_list, i);
 		const char *done_path = rebase_path_done();
 
 		fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
@@ -3478,12 +3492,10 @@ int rearrange_squash(void)
 				continue;
 
 			while (cur >= 0) {
-				int offset = todo_list.items[cur].offset_in_buf;
-				int end_offset = cur + 1 < todo_list.nr ?
-					todo_list.items[cur + 1].offset_in_buf :
-					todo_list.buf.len;
-				char *bol = todo_list.buf.buf + offset;
-				char *eol = todo_list.buf.buf + end_offset;
+				const char *bol =
+					get_item_line(&todo_list, cur);
+				const char *eol =
+					get_item_line(&todo_list, cur + 1);
 
 				/* replace 'pick', by 'fixup' or 'squash' */
 				command = todo_list.items[cur].command;
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 04/16] sequencer: offer helpful advice when a command was rescheduled
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (2 preceding siblings ...)
  2018-04-21 10:31             ` [PATCH v8 03/16] sequencer: refactor how original todo list lines are accessed Johannes Schindelin
@ 2018-04-21 10:31             ` Johannes Schindelin
  2018-04-21 10:32             ` [PATCH v8 05/16] git-rebase--interactive: clarify arguments Johannes Schindelin
                               ` (12 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:31 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Previously, we did that just magically, and potentially left some users
quite puzzled. Let's err on the safe side instead, telling the user what
is happening, and how they are supposed to continue.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 3d0a45ab25a..01443e0f245 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2555,6 +2555,17 @@ static const char *reflog_message(struct replay_opts *opts,
 	return buf.buf;
 }
 
+static const char rescheduled_advice[] =
+N_("Could not execute the todo command\n"
+"\n"
+"    %.*s"
+"\n"
+"It has been rescheduled; To edit the command before continuing, please\n"
+"edit the todo list first:\n"
+"\n"
+"    git rebase --edit-todo\n"
+"    git rebase --continue\n");
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -2600,6 +2611,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					opts, is_final_fixup(todo_list));
 			if (is_rebase_i(opts) && res < 0) {
 				/* Reschedule */
+				advise(_(rescheduled_advice),
+				       get_item_line_length(todo_list,
+							    todo_list->current),
+				       get_item_line(todo_list,
+						     todo_list->current));
 				todo_list->current--;
 				if (save_todo(todo_list, opts))
 					return -1;
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 05/16] git-rebase--interactive: clarify arguments
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (3 preceding siblings ...)
  2018-04-21 10:31             ` [PATCH v8 04/16] sequencer: offer helpful advice when a command was rescheduled Johannes Schindelin
@ 2018-04-21 10:32             ` Johannes Schindelin
  2018-04-21 10:33             ` [PATCH v8 06/16] sequencer: introduce the `merge` command Johannes Schindelin
                               ` (11 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:32 UTC (permalink / raw)
  To: git
  Cc: Stefan Beller, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

From: Stefan Beller <stefanbeller@gmail.com>

Up to now each command took a commit as its first argument and ignored
the rest of the line (usually the subject of the commit)

Now that we are about to introduce commands that take different
arguments, clarify each command by giving the argument list.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 50323fc2735..e1b865f43f2 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -155,13 +155,13 @@ reschedule_last_action () {
 append_todo_help () {
 	gettext "
 Commands:
-p, pick = use commit
-r, reword = use commit, but edit the commit message
-e, edit = use commit, but stop for amending
-s, squash = use commit, but meld into previous commit
-f, fixup = like \"squash\", but discard this commit's log message
-x, exec = run command (the rest of the line) using shell
-d, drop = remove commit
+p, pick <commit> = use commit
+r, reword <commit> = use commit, but edit the commit message
+e, edit <commit> = use commit, but stop for amending
+s, squash <commit> = use commit, but meld into previous commit
+f, fixup <commit> = like \"squash\", but discard this commit's log message
+x, exec <commit> = run command (the rest of the line) using shell
+d, drop <commit> = remove commit
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (4 preceding siblings ...)
  2018-04-21 10:32             ` [PATCH v8 05/16] git-rebase--interactive: clarify arguments Johannes Schindelin
@ 2018-04-21 10:33             ` Johannes Schindelin
  2018-04-21 15:56               ` Phillip Wood
                                 ` (2 more replies)
  2018-04-21 10:33             ` [PATCH v8 07/16] sequencer: fast-forward `merge` commands, if possible Johannes Schindelin
                               ` (10 subsequent siblings)
  16 siblings, 3 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

This patch is part of the effort to reimplement `--preserve-merges` with
a substantially improved design, a design that has been developed in the
Git for Windows project to maintain the dozens of Windows-specific patch
series on top of upstream Git.

The previous patch implemented the `label` and `reset` commands to label
commits and to reset to labeled commits. This patch adds the `merge`
command, with the following syntax:

	merge [-C <commit>] <rev> # <oneline>

The <commit> parameter in this instance is the *original* merge commit,
whose author and message will be used for the merge commit that is about
to be created.

The <rev> parameter refers to the (possibly rewritten) revision to
merge. Let's see an example of a todo list:

	label onto

	# Branch abc
	reset onto
	pick deadbeef Hello, world!
	label abc

	reset onto
	pick cafecafe And now for something completely different
	merge -C baaabaaa abc # Merge the branch 'abc' into master

To edit the merge commit's message (a "reword" for merges, if you will),
use `-c` (lower-case) instead of `-C`; this convention was borrowed from
`git commit` that also supports `-c` and `-C` with similar meanings.

To create *new* merges, i.e. without copying the commit message from an
existing commit, simply omit the `-C <commit>` parameter (which will
open an editor for the merge message):

	merge abc

This comes in handy when splitting a branch into two or more branches.

Note: this patch only adds support for recursive merges, to keep things
simple. Support for octopus merges will be added later in a separate
patch series, support for merges using strategies other than the
recursive merge is left for the future.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   6 +
 sequencer.c                | 407 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 406 insertions(+), 7 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index e1b865f43f2..ccd5254d1c9 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -162,6 +162,12 @@ s, squash <commit> = use commit, but meld into previous commit
 f, fixup <commit> = like \"squash\", but discard this commit's log message
 x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
+l, label <label> = label current HEAD with a name
+t, reset <label> = reset HEAD to a label
+m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
+.       create a merge commit using the original merge commit's
+.       message (or the oneline, if no original merge commit was
+.       specified). Use -c <commit> to reword the commit message.
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index 01443e0f245..35fcacbdf0f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -23,6 +23,8 @@
 #include "hashmap.h"
 #include "notes-utils.h"
 #include "sigchain.h"
+#include "unpack-trees.h"
+#include "worktree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -120,6 +122,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
 static GIT_PATH_FUNC(rebase_path_rewritten_pending,
 	"rebase-merge/rewritten-pending")
+
+/*
+ * The path of the file listing refs that need to be deleted after the rebase
+ * finishes. This is used by the `label` command to record the need for cleanup.
+ */
+static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
+
 /*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
@@ -244,18 +253,34 @@ static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
 
 int sequencer_remove_state(struct replay_opts *opts)
 {
-	struct strbuf dir = STRBUF_INIT;
+	struct strbuf buf = STRBUF_INIT;
 	int i;
 
+	if (is_rebase_i(opts) &&
+	    strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
+		char *p = buf.buf;
+		while (*p) {
+			char *eol = strchr(p, '\n');
+			if (eol)
+				*eol = '\0';
+			if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
+				warning(_("could not delete '%s'"), p);
+			if (!eol)
+				break;
+			p = eol + 1;
+		}
+	}
+
 	free(opts->gpg_sign);
 	free(opts->strategy);
 	for (i = 0; i < opts->xopts_nr; i++)
 		free(opts->xopts[i]);
 	free(opts->xopts);
 
-	strbuf_addstr(&dir, get_dir(opts));
-	remove_dir_recursively(&dir, 0);
-	strbuf_release(&dir);
+	strbuf_reset(&buf);
+	strbuf_addstr(&buf, get_dir(opts));
+	remove_dir_recursively(&buf, 0);
+	strbuf_release(&buf);
 
 	return 0;
 }
@@ -1279,6 +1304,9 @@ enum todo_command {
 	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
+	TODO_LABEL,
+	TODO_RESET,
+	TODO_MERGE,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -1297,6 +1325,9 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
+	{ 'l', "label" },
+	{ 't', "reset" },
+	{ 'm', "merge" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1724,9 +1755,14 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 	return 0;
 }
 
+enum todo_item_flags {
+	TODO_EDIT_MERGE_MSG = 1
+};
+
 struct todo_item {
 	enum todo_command command;
 	struct commit *commit;
+	unsigned int flags;
 	const char *arg;
 	int arg_len;
 	size_t offset_in_buf;
@@ -1761,6 +1797,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	char *end_of_object_name;
 	int i, saved, status, padding;
 
+	item->flags = 0;
+
 	/* left-trim */
 	bol += strspn(bol, " \t");
 
@@ -1802,13 +1840,29 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return error(_("missing arguments for %s"),
 			     command_to_string(item->command));
 
-	if (item->command == TODO_EXEC) {
+	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
+	    item->command == TODO_RESET) {
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = (int)(eol - bol);
 		return 0;
 	}
 
+	if (item->command == TODO_MERGE) {
+		if (skip_prefix(bol, "-C", &bol))
+			bol += strspn(bol, " \t");
+		else if (skip_prefix(bol, "-c", &bol)) {
+			bol += strspn(bol, " \t");
+			item->flags |= TODO_EDIT_MERGE_MSG;
+		} else {
+			item->flags |= TODO_EDIT_MERGE_MSG;
+			item->commit = NULL;
+			item->arg = bol;
+			item->arg_len = (int)(eol - bol);
+			return 0;
+		}
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
@@ -2465,6 +2519,305 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int safe_append(const char *filename, const char *fmt, ...)
+{
+	va_list ap;
+	struct lock_file lock = LOCK_INIT;
+	int fd = hold_lock_file_for_update(&lock, filename,
+					   LOCK_REPORT_ON_ERROR);
+	struct strbuf buf = STRBUF_INIT;
+
+	if (fd < 0)
+		return -1;
+
+	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT) {
+		error_errno(_("could not read '%s'"), filename);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	strbuf_complete(&buf, '\n');
+	va_start(ap, fmt);
+	strbuf_vaddf(&buf, fmt, ap);
+	va_end(ap);
+
+	if (write_in_full(fd, buf.buf, buf.len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
+		strbuf_release(&buf);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	if (commit_lock_file(&lock) < 0) {
+		strbuf_release(&buf);
+		rollback_lock_file(&lock);
+		return error(_("failed to finalize '%s'"), filename);
+	}
+
+	strbuf_release(&buf);
+	return 0;
+}
+
+static int do_label(const char *name, int len)
+{
+	struct ref_store *refs = get_main_ref_store();
+	struct ref_transaction *transaction;
+	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
+	struct strbuf msg = STRBUF_INIT;
+	int ret = 0;
+	struct object_id head_oid;
+
+	if (len == 1 && *name == '#')
+		return error("Illegal label name: '%.*s'", len, name);
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
+
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction) {
+		error("%s", err.buf);
+		ret = -1;
+	} else if (get_oid("HEAD", &head_oid)) {
+		error(_("could not read HEAD"));
+		ret = -1;
+	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
+					  NULL, 0, msg.buf, &err) < 0 ||
+		   ref_transaction_commit(transaction, &err)) {
+		error("%s", err.buf);
+		ret = -1;
+	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
+	strbuf_release(&msg);
+
+	if (!ret)
+		ret = safe_append(rebase_path_refs_to_delete(),
+				  "%s\n", ref_name.buf);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
+static const char *reflog_message(struct replay_opts *opts,
+	const char *sub_action, const char *fmt, ...);
+
+static int do_reset(const char *name, int len, struct replay_opts *opts)
+{
+	struct strbuf ref_name = STRBUF_INIT;
+	struct object_id oid;
+	struct lock_file lock = LOCK_INIT;
+	struct tree_desc desc;
+	struct tree *tree;
+	struct unpack_trees_options unpack_tree_opts;
+	int ret = 0, i;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	/* Determine the length of the label */
+	for (i = 0; i < len; i++)
+		if (isspace(name[i]))
+			len = i;
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	if (get_oid(ref_name.buf, &oid) &&
+	    get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
+		error(_("could not read '%s'"), ref_name.buf);
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
+	unpack_tree_opts.head_idx = 1;
+	unpack_tree_opts.src_index = &the_index;
+	unpack_tree_opts.dst_index = &the_index;
+	unpack_tree_opts.fn = oneway_merge;
+	unpack_tree_opts.merge = 1;
+	unpack_tree_opts.update = 1;
+
+	if (read_cache_unmerged()) {
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return error_resolve_conflict(_(action_name(opts)));
+	}
+
+	if (!fill_tree_descriptor(&desc, &oid)) {
+		error(_("failed to find tree of %s"), oid_to_hex(&oid));
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	if (unpack_trees(1, &desc, &unpack_tree_opts)) {
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	tree = parse_tree_indirect(&oid);
+	prime_cache_tree(&the_index, tree);
+
+	if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
+		ret = error(_("could not write index"));
+	free((void *)desc.buffer);
+
+	if (!ret)
+		ret = update_ref(reflog_message(opts, "reset", "'%.*s'",
+						len, name), "HEAD", &oid,
+				 NULL, 0, UPDATE_REFS_MSG_ON_ERR);
+
+	strbuf_release(&ref_name);
+	return ret;
+}
+
+static int do_merge(struct commit *commit, const char *arg, int arg_len,
+		    int flags, struct replay_opts *opts)
+{
+	int run_commit_flags = (flags & TODO_EDIT_MERGE_MSG) ?
+		EDIT_MSG | VERIFY_MSG : 0;
+	struct strbuf ref_name = STRBUF_INIT;
+	struct commit *head_commit, *merge_commit, *i;
+	struct commit_list *bases, *j, *reversed = NULL;
+	struct merge_options o;
+	int merge_arg_len, oneline_offset, ret;
+	static struct lock_file lock;
+	const char *p;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0) {
+		ret = -1;
+		goto leave_merge;
+	}
+
+	head_commit = lookup_commit_reference_by_name("HEAD");
+	if (!head_commit) {
+		ret = error(_("cannot merge without a current revision"));
+		goto leave_merge;
+	}
+
+	oneline_offset = arg_len;
+	merge_arg_len = strcspn(arg, " \t\n");
+	p = arg + merge_arg_len;
+	p += strspn(p, " \t\n");
+	if (*p == '#' && (!p[1] || isspace(p[1]))) {
+		p += 1 + strspn(p + 1, " \t\n");
+		oneline_offset = p - arg;
+	} else if (p - arg < arg_len)
+		BUG("octopus merges are not supported yet: '%s'", p);
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
+	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	if (!merge_commit) {
+		/* fall back to non-rewritten ref or commit */
+		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
+		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	}
+
+	if (!merge_commit) {
+		ret = error(_("could not resolve '%s'"), ref_name.buf);
+		goto leave_merge;
+	}
+
+	if (commit) {
+		const char *message = get_commit_buffer(commit, NULL);
+		const char *body;
+		int len;
+
+		if (!message) {
+			ret = error(_("could not get commit message of '%s'"),
+				    oid_to_hex(&commit->object.oid));
+			goto leave_merge;
+		}
+		write_author_script(message);
+		find_commit_subject(message, &body);
+		len = strlen(body);
+		ret = write_message(body, len, git_path_merge_msg(), 0);
+		unuse_commit_buffer(commit, message);
+		if (ret) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			goto leave_merge;
+		}
+	} else {
+		struct strbuf buf = STRBUF_INIT;
+		int len;
+
+		strbuf_addf(&buf, "author %s", git_author_info(0));
+		write_author_script(buf.buf);
+		strbuf_reset(&buf);
+
+		if (oneline_offset < arg_len) {
+			p = arg + oneline_offset;
+			len = arg_len - oneline_offset;
+		} else {
+			strbuf_addf(&buf, "Merge branch '%.*s'",
+				    merge_arg_len, arg);
+			p = buf.buf;
+			len = buf.len;
+		}
+
+		ret = write_message(p, len, git_path_merge_msg(), 0);
+		strbuf_release(&buf);
+		if (ret) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			goto leave_merge;
+		}
+	}
+
+	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
+		      git_path_merge_head(), 0);
+	write_message("no-ff", 5, git_path_merge_mode(), 0);
+
+	bases = get_merge_bases(head_commit, merge_commit);
+	for (j = bases; j; j = j->next)
+		commit_list_insert(j->item, &reversed);
+	free_commit_list(bases);
+
+	read_cache();
+	init_merge_options(&o);
+	o.branch1 = "HEAD";
+	o.branch2 = ref_name.buf;
+	o.buffer_output = 2;
+
+	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
+	if (ret <= 0)
+		fputs(o.obuf.buf, stdout);
+	strbuf_release(&o.obuf);
+	if (ret < 0) {
+		error(_("could not even attempt to merge '%.*s'"),
+		      merge_arg_len, arg);
+		goto leave_merge;
+	}
+	/*
+	 * The return value of merge_recursive() is 1 on clean, and 0 on
+	 * unclean merge.
+	 *
+	 * Let's reverse that, so that do_merge() returns 0 upon success and
+	 * 1 upon failed merge (keeping the return value -1 for the cases where
+	 * we will want to reschedule the `merge` command).
+	 */
+	ret = !ret;
+
+	if (active_cache_changed &&
+	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
+		ret = error(_("merge: Unable to write new index file"));
+		goto leave_merge;
+	}
+
+	rollback_lock_file(&lock);
+	if (ret)
+		rerere(opts->allow_rerere_auto);
+	else
+		ret = run_git_commit(git_path_merge_msg(), opts,
+				     run_commit_flags);
+
+leave_merge:
+	strbuf_release(&ref_name);
+	rollback_lock_file(&lock);
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2568,7 +2921,7 @@ N_("Could not execute the todo command\n"
 
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
-	int res = 0;
+	int res = 0, reschedule = 0;
 
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	if (opts->allow_ff)
@@ -2639,7 +2992,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					intend_to_amend();
 				return error_failed_squash(item->commit, opts,
 					item->arg_len, item->arg);
-			} else if (res && is_rebase_i(opts))
+			} else if (res && is_rebase_i(opts) && item->commit)
 				return res | error_with_patch(item->commit,
 					item->arg, item->arg_len, opts, res,
 					item->command == TODO_REWORD);
@@ -2665,9 +3018,41 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				/* `current` will be incremented below */
 				todo_list->current = -1;
 			}
+		} else if (item->command == TODO_LABEL) {
+			if ((res = do_label(item->arg, item->arg_len)))
+				reschedule = 1;
+		} else if (item->command == TODO_RESET) {
+			if ((res = do_reset(item->arg, item->arg_len, opts)))
+				reschedule = 1;
+		} else if (item->command == TODO_MERGE) {
+			if ((res = do_merge(item->commit,
+					    item->arg, item->arg_len,
+					    item->flags, opts)) < 0)
+				reschedule = 1;
+			else if (res > 0)
+				/* failed with merge conflicts */
+				return error_with_patch(item->commit,
+							item->arg,
+							item->arg_len, opts,
+							res, 0);
 		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
+		if (reschedule) {
+			advise(_(rescheduled_advice),
+			       get_item_line_length(todo_list,
+						    todo_list->current),
+			       get_item_line(todo_list, todo_list->current));
+			todo_list->current--;
+			if (save_todo(todo_list, opts))
+				return -1;
+			if (item->commit)
+				return error_with_patch(item->commit,
+							item->arg,
+							item->arg_len, opts,
+							res, 0);
+		}
+
 		todo_list->current++;
 		if (res)
 			return res;
@@ -3147,8 +3532,16 @@ int transform_todos(unsigned flags)
 					  short_commit_name(item->commit) :
 					  oid_to_hex(&item->commit->object.oid);
 
+			if (item->command == TODO_MERGE) {
+				if (item->flags & TODO_EDIT_MERGE_MSG)
+					strbuf_addstr(&buf, " -c");
+				else
+					strbuf_addstr(&buf, " -C");
+			}
+
 			strbuf_addf(&buf, " %s", oid);
 		}
+
 		/* add all the rest */
 		if (!item->arg_len)
 			strbuf_addch(&buf, '\n');
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 07/16] sequencer: fast-forward `merge` commands, if possible
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (5 preceding siblings ...)
  2018-04-21 10:33             ` [PATCH v8 06/16] sequencer: introduce the `merge` command Johannes Schindelin
@ 2018-04-21 10:33             ` Johannes Schindelin
  2018-04-21 10:34             ` [PATCH v8 08/16] rebase-helper --make-script: introduce a flag to rebase merges Johannes Schindelin
                               ` (9 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Just like with regular `pick` commands, if we are trying to rebase a
merge commit, we now test whether the parents of said commit match HEAD
and the commits to be merged, and fast-forward if possible.

This is not only faster, but also avoids unnecessary proliferation of
new objects.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 ++++++++++++++++++++++++++++++++-
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 35fcacbdf0f..5944d3a34eb 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2680,7 +2680,7 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	struct commit *head_commit, *merge_commit, *i;
 	struct commit_list *bases, *j, *reversed = NULL;
 	struct merge_options o;
-	int merge_arg_len, oneline_offset, ret;
+	int merge_arg_len, oneline_offset, can_fast_forward, ret;
 	static struct lock_file lock;
 	const char *p;
 
@@ -2765,6 +2765,37 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		}
 	}
 
+	/*
+	 * If HEAD is not identical to the first parent of the original merge
+	 * commit, we cannot fast-forward.
+	 */
+	can_fast_forward = opts->allow_ff && commit && commit->parents &&
+		!oidcmp(&commit->parents->item->object.oid,
+			&head_commit->object.oid);
+
+	/*
+	 * If the merge head is different from the original one, we cannot
+	 * fast-forward.
+	 */
+	if (can_fast_forward) {
+		struct commit_list *second_parent = commit->parents->next;
+
+		if (second_parent && !second_parent->next &&
+		    oidcmp(&merge_commit->object.oid,
+			   &second_parent->item->object.oid))
+			can_fast_forward = 0;
+	}
+
+	if (can_fast_forward && commit->parents->next &&
+	    !commit->parents->next->next &&
+	    !oidcmp(&commit->parents->next->item->object.oid,
+		    &merge_commit->object.oid)) {
+		rollback_lock_file(&lock);
+		ret = fast_forward_to(&commit->object.oid,
+				      &head_commit->object.oid, 0, opts);
+		goto leave_merge;
+	}
+
 	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
 		      git_path_merge_head(), 0);
 	write_message("no-ff", 5, git_path_merge_mode(), 0);
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 08/16] rebase-helper --make-script: introduce a flag to rebase merges
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (6 preceding siblings ...)
  2018-04-21 10:33             ` [PATCH v8 07/16] sequencer: fast-forward `merge` commands, if possible Johannes Schindelin
@ 2018-04-21 10:34             ` Johannes Schindelin
  2018-04-22 13:42               ` Philip Oakley
  2018-04-21 10:34             ` [PATCH v8 09/16] rebase: introduce the --rebase-merges option Johannes Schindelin
                               ` (8 subsequent siblings)
  16 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:34 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

The sequencer just learned new commands intended to recreate branch
structure (similar in spirit to --preserve-merges, but with a
substantially less-broken design).

Let's allow the rebase--helper to generate todo lists making use of
these commands, triggered by the new --rebase-merges option. For a
commit topology like this (where the HEAD points to C):

	- A - B - C
	    \   /
	      D

the generated todo list would look like this:

	# branch D
	pick 0123 A
	label branch-point
	pick 1234 D
	label D

	reset branch-point
	pick 2345 B
	merge -C 3456 D # C

To keep things simple, we first only implement support for merge commits
with exactly two parents, leaving support for octopus merges to a later
patch series.

As a special, hard-coded label, all merge-rebasing todo lists start with
the command `label onto` so that we can later always refer to the revision
onto which everything is rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/rebase--helper.c |   4 +-
 sequencer.c              | 351 ++++++++++++++++++++++++++++++++++++++-
 sequencer.h              |   1 +
 3 files changed, 353 insertions(+), 3 deletions(-)

diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index ad074705bb5..781782e7272 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -12,7 +12,7 @@ static const char * const builtin_rebase_helper_usage[] = {
 int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
-	unsigned flags = 0, keep_empty = 0;
+	unsigned flags = 0, keep_empty = 0, rebase_merges = 0;
 	int abbreviate_commands = 0;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
@@ -24,6 +24,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
 		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
 			N_("allow commits with empty messages")),
+		OPT_BOOL(0, "rebase-merges", &rebase_merges, N_("rebase merge commits")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -57,6 +58,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
+	flags |= rebase_merges ? TODO_LIST_REBASE_MERGES : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
 	if (command == CONTINUE && argc == 1)
diff --git a/sequencer.c b/sequencer.c
index 5944d3a34eb..1e17a11ca32 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -25,6 +25,8 @@
 #include "sigchain.h"
 #include "unpack-trees.h"
 #include "worktree.h"
+#include "oidmap.h"
+#include "oidset.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -3436,6 +3438,343 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
 	strbuf_release(&sob);
 }
 
+struct labels_entry {
+	struct hashmap_entry entry;
+	char label[FLEX_ARRAY];
+};
+
+static int labels_cmp(const void *fndata, const struct labels_entry *a,
+		      const struct labels_entry *b, const void *key)
+{
+	return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
+}
+
+struct string_entry {
+	struct oidmap_entry entry;
+	char string[FLEX_ARRAY];
+};
+
+struct label_state {
+	struct oidmap commit2label;
+	struct hashmap labels;
+	struct strbuf buf;
+};
+
+static const char *label_oid(struct object_id *oid, const char *label,
+			     struct label_state *state)
+{
+	struct labels_entry *labels_entry;
+	struct string_entry *string_entry;
+	struct object_id dummy;
+	size_t len;
+	int i;
+
+	string_entry = oidmap_get(&state->commit2label, oid);
+	if (string_entry)
+		return string_entry->string;
+
+	/*
+	 * For "uninteresting" commits, i.e. commits that are not to be
+	 * rebased, and which can therefore not be labeled, we use a unique
+	 * abbreviation of the commit name. This is slightly more complicated
+	 * than calling find_unique_abbrev() because we also need to make
+	 * sure that the abbreviation does not conflict with any other
+	 * label.
+	 *
+	 * We disallow "interesting" commits to be labeled by a string that
+	 * is a valid full-length hash, to ensure that we always can find an
+	 * abbreviation for any uninteresting commit's names that does not
+	 * clash with any other label.
+	 */
+	if (!label) {
+		char *p;
+
+		strbuf_reset(&state->buf);
+		strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
+		label = p = state->buf.buf;
+
+		find_unique_abbrev_r(p, oid, default_abbrev);
+
+		/*
+		 * We may need to extend the abbreviated hash so that there is
+		 * no conflicting label.
+		 */
+		if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
+			size_t i = strlen(p) + 1;
+
+			oid_to_hex_r(p, oid);
+			for (; i < GIT_SHA1_HEXSZ; i++) {
+				char save = p[i];
+				p[i] = '\0';
+				if (!hashmap_get_from_hash(&state->labels,
+							   strihash(p), p))
+					break;
+				p[i] = save;
+			}
+		}
+	} else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
+		    !get_oid_hex(label, &dummy)) ||
+		   (len == 1 && *label == '#') ||
+		   hashmap_get_from_hash(&state->labels,
+					 strihash(label), label)) {
+		/*
+		 * If the label already exists, or if the label is a valid full
+		 * OID, or the label is a '#' (which we use as a separator
+		 * between merge heads and oneline), we append a dash and a
+		 * number to make it unique.
+		 */
+		struct strbuf *buf = &state->buf;
+
+		strbuf_reset(buf);
+		strbuf_add(buf, label, len);
+
+		for (i = 2; ; i++) {
+			strbuf_setlen(buf, len);
+			strbuf_addf(buf, "-%d", i);
+			if (!hashmap_get_from_hash(&state->labels,
+						   strihash(buf->buf),
+						   buf->buf))
+				break;
+		}
+
+		label = buf->buf;
+	}
+
+	FLEX_ALLOC_STR(labels_entry, label, label);
+	hashmap_entry_init(labels_entry, strihash(label));
+	hashmap_add(&state->labels, labels_entry);
+
+	FLEX_ALLOC_STR(string_entry, string, label);
+	oidcpy(&string_entry->entry.oid, oid);
+	oidmap_put(&state->commit2label, string_entry);
+
+	return string_entry->string;
+}
+
+static int make_script_with_merges(struct pretty_print_context *pp,
+				   struct rev_info *revs, FILE *out,
+				   unsigned flags)
+{
+	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
+	struct strbuf label = STRBUF_INIT;
+	struct commit_list *commits = NULL, **tail = &commits, *iter;
+	struct commit_list *tips = NULL, **tips_tail = &tips;
+	struct commit *commit;
+	struct oidmap commit2todo = OIDMAP_INIT;
+	struct string_entry *entry;
+	struct oidset interesting = OIDSET_INIT, child_seen = OIDSET_INIT,
+		shown = OIDSET_INIT;
+	struct label_state state = { OIDMAP_INIT, { NULL }, STRBUF_INIT };
+
+	int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
+	const char *cmd_pick = abbr ? "p" : "pick",
+		*cmd_label = abbr ? "l" : "label",
+		*cmd_reset = abbr ? "t" : "reset",
+		*cmd_merge = abbr ? "m" : "merge";
+
+	oidmap_init(&commit2todo, 0);
+	oidmap_init(&state.commit2label, 0);
+	hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
+	strbuf_init(&state.buf, 32);
+
+	if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
+		struct object_id *oid = &revs->cmdline.rev[0].item->oid;
+		FLEX_ALLOC_STR(entry, string, "onto");
+		oidcpy(&entry->entry.oid, oid);
+		oidmap_put(&state.commit2label, entry);
+	}
+
+	/*
+	 * First phase:
+	 * - get onelines for all commits
+	 * - gather all branch tips (i.e. 2nd or later parents of merges)
+	 * - label all branch tips
+	 */
+	while ((commit = get_revision(revs))) {
+		struct commit_list *to_merge;
+		int is_octopus;
+		const char *p1, *p2;
+		struct object_id *oid;
+		int is_empty;
+
+		tail = &commit_list_insert(commit, tail)->next;
+		oidset_insert(&interesting, &commit->object.oid);
+
+		is_empty = is_original_commit_empty(commit);
+		if (!is_empty && (commit->object.flags & PATCHSAME))
+			continue;
+
+		strbuf_reset(&oneline);
+		pretty_print_commit(pp, commit, &oneline);
+
+		to_merge = commit->parents ? commit->parents->next : NULL;
+		if (!to_merge) {
+			/* non-merge commit: easy case */
+			strbuf_reset(&buf);
+			if (!keep_empty && is_empty)
+				strbuf_addf(&buf, "%c ", comment_line_char);
+			strbuf_addf(&buf, "%s %s %s", cmd_pick,
+				    oid_to_hex(&commit->object.oid),
+				    oneline.buf);
+
+			FLEX_ALLOC_STR(entry, string, buf.buf);
+			oidcpy(&entry->entry.oid, &commit->object.oid);
+			oidmap_put(&commit2todo, entry);
+
+			continue;
+		}
+
+		is_octopus = to_merge && to_merge->next;
+
+		if (is_octopus)
+			BUG("Octopus merges not yet supported");
+
+		/* Create a label */
+		strbuf_reset(&label);
+		if (skip_prefix(oneline.buf, "Merge ", &p1) &&
+		    (p1 = strchr(p1, '\'')) &&
+		    (p2 = strchr(++p1, '\'')))
+			strbuf_add(&label, p1, p2 - p1);
+		else if (skip_prefix(oneline.buf, "Merge pull request ",
+				     &p1) &&
+			 (p1 = strstr(p1, " from ")))
+			strbuf_addstr(&label, p1 + strlen(" from "));
+		else
+			strbuf_addbuf(&label, &oneline);
+
+		for (p1 = label.buf; *p1; p1++)
+			if (isspace(*p1))
+				*(char *)p1 = '-';
+
+		strbuf_reset(&buf);
+		strbuf_addf(&buf, "%s -C %s",
+			    cmd_merge, oid_to_hex(&commit->object.oid));
+
+		/* label the tip of merged branch */
+		oid = &to_merge->item->object.oid;
+		strbuf_addch(&buf, ' ');
+
+		if (!oidset_contains(&interesting, oid))
+			strbuf_addstr(&buf, label_oid(oid, NULL, &state));
+		else {
+			tips_tail = &commit_list_insert(to_merge->item,
+							tips_tail)->next;
+
+			strbuf_addstr(&buf, label_oid(oid, label.buf, &state));
+		}
+		strbuf_addf(&buf, " # %s", oneline.buf);
+
+		FLEX_ALLOC_STR(entry, string, buf.buf);
+		oidcpy(&entry->entry.oid, &commit->object.oid);
+		oidmap_put(&commit2todo, entry);
+	}
+
+	/*
+	 * Second phase:
+	 * - label branch points
+	 * - add HEAD to the branch tips
+	 */
+	for (iter = commits; iter; iter = iter->next) {
+		struct commit_list *parent = iter->item->parents;
+		for (; parent; parent = parent->next) {
+			struct object_id *oid = &parent->item->object.oid;
+			if (!oidset_contains(&interesting, oid))
+				continue;
+			if (!oidset_contains(&child_seen, oid))
+				oidset_insert(&child_seen, oid);
+			else
+				label_oid(oid, "branch-point", &state);
+		}
+
+		/* Add HEAD as implict "tip of branch" */
+		if (!iter->next)
+			tips_tail = &commit_list_insert(iter->item,
+							tips_tail)->next;
+	}
+
+	/*
+	 * Third phase: output the todo list. This is a bit tricky, as we
+	 * want to avoid jumping back and forth between revisions. To
+	 * accomplish that goal, we walk backwards from the branch tips,
+	 * gathering commits not yet shown, reversing the list on the fly,
+	 * then outputting that list (labeling revisions as needed).
+	 */
+	fprintf(out, "%s onto\n", cmd_label);
+	for (iter = tips; iter; iter = iter->next) {
+		struct commit_list *list = NULL, *iter2;
+
+		commit = iter->item;
+		if (oidset_contains(&shown, &commit->object.oid))
+			continue;
+		entry = oidmap_get(&state.commit2label, &commit->object.oid);
+
+		if (entry)
+			fprintf(out, "\n# Branch %s\n", entry->string);
+		else
+			fprintf(out, "\n");
+
+		while (oidset_contains(&interesting, &commit->object.oid) &&
+		       !oidset_contains(&shown, &commit->object.oid)) {
+			commit_list_insert(commit, &list);
+			if (!commit->parents) {
+				commit = NULL;
+				break;
+			}
+			commit = commit->parents->item;
+		}
+
+		if (!commit)
+			fprintf(out, "%s onto\n", cmd_reset);
+		else {
+			const char *to = NULL;
+
+			entry = oidmap_get(&state.commit2label,
+					   &commit->object.oid);
+			if (entry)
+				to = entry->string;
+
+			if (!to || !strcmp(to, "onto"))
+				fprintf(out, "%s onto\n", cmd_reset);
+			else {
+				strbuf_reset(&oneline);
+				pretty_print_commit(pp, commit, &oneline);
+				fprintf(out, "%s %s # %s\n",
+					cmd_reset, to, oneline.buf);
+			}
+		}
+
+		for (iter2 = list; iter2; iter2 = iter2->next) {
+			struct object_id *oid = &iter2->item->object.oid;
+			entry = oidmap_get(&commit2todo, oid);
+			/* only show if not already upstream */
+			if (entry)
+				fprintf(out, "%s\n", entry->string);
+			entry = oidmap_get(&state.commit2label, oid);
+			if (entry)
+				fprintf(out, "%s %s\n",
+					cmd_label, entry->string);
+			oidset_insert(&shown, oid);
+		}
+
+		free_commit_list(list);
+	}
+
+	free_commit_list(commits);
+	free_commit_list(tips);
+
+	strbuf_release(&label);
+	strbuf_release(&oneline);
+	strbuf_release(&buf);
+
+	oidmap_free(&commit2todo, 1);
+	oidmap_free(&state.commit2label, 1);
+	hashmap_free(&state.labels, 1);
+	strbuf_release(&state.buf);
+
+	return 0;
+}
+
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags)
 {
@@ -3446,11 +3785,16 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	struct commit *commit;
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
 	const char *insn = flags & TODO_LIST_ABBREVIATE_CMDS ? "p" : "pick";
+	int rebase_merges = flags & TODO_LIST_REBASE_MERGES;
 
 	init_revisions(&revs, NULL);
 	revs.verbose_header = 1;
-	revs.max_parents = 1;
-	revs.cherry_pick = 1;
+	if (rebase_merges)
+		revs.cherry_mark = 1;
+	else {
+		revs.max_parents = 1;
+		revs.cherry_pick = 1;
+	}
 	revs.limited = 1;
 	revs.reverse = 1;
 	revs.right_only = 1;
@@ -3474,6 +3818,9 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	if (prepare_revision_walk(&revs) < 0)
 		return error(_("make_script: error preparing revisions"));
 
+	if (rebase_merges)
+		return make_script_with_merges(&pp, &revs, out, flags);
+
 	while ((commit = get_revision(&revs))) {
 		strbuf_reset(&buf);
 		if (!keep_empty && is_original_commit_empty(commit))
diff --git a/sequencer.h b/sequencer.h
index e45b178dfc4..6bc4da17243 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -59,6 +59,7 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_KEEP_EMPTY (1U << 0)
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
+#define TODO_LIST_REBASE_MERGES (1U << 3)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 09/16] rebase: introduce the --rebase-merges option
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (7 preceding siblings ...)
  2018-04-21 10:34             ` [PATCH v8 08/16] rebase-helper --make-script: introduce a flag to rebase merges Johannes Schindelin
@ 2018-04-21 10:34             ` Johannes Schindelin
  2018-04-22 14:15               ` Philip Oakley
  2018-04-22 14:37               ` Philip Oakley
  2018-04-21 10:35             ` [PATCH v8 10/16] rebase --rebase-merges: add test for --keep-empty Johannes Schindelin
                               ` (7 subsequent siblings)
  16 siblings, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:34 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Once upon a time, this here developer thought: wouldn't it be nice if,
say, Git for Windows' patches on top of core Git could be represented as
a thicket of branches, and be rebased on top of core Git in order to
maintain a cherry-pick'able set of patch series?

The original attempt to answer this was: git rebase --preserve-merges.

However, that experiment was never intended as an interactive option,
and it only piggy-backed on git rebase --interactive because that
command's implementation looked already very, very familiar: it was
designed by the same person who designed --preserve-merges: yours truly.

Some time later, some other developer (I am looking at you, Andreas!
;-)) decided that it would be a good idea to allow --preserve-merges to
be combined with --interactive (with caveats!) and the Git maintainer
(well, the interim Git maintainer during Junio's absence, that is)
agreed, and that is when the glamor of the --preserve-merges design
started to fall apart rather quickly and unglamorously.

The reason? In --preserve-merges mode, the parents of a merge commit (or
for that matter, of *any* commit) were not stated explicitly, but were
*implied* by the commit name passed to the `pick` command.

This made it impossible, for example, to reorder commits. Not to mention
to flatten the branch topology or, deity forbid, to split topic branches
into two.

Alas, these shortcomings also prevented that mode (whose original
purpose was to serve Git for Windows' needs, with the additional hope
that it may be useful to others, too) from serving Git for Windows'
needs.

Five years later, when it became really untenable to have one unwieldy,
big hodge-podge patch series of partly related, partly unrelated patches
in Git for Windows that was rebased onto core Git's tags from time to
time (earning the undeserved wrath of the developer of the ill-fated
git-remote-hg series that first obsoleted Git for Windows' competing
approach, only to be abandoned without maintainer later) was really
untenable, the "Git garden shears" were born [*1*/*2*]: a script,
piggy-backing on top of the interactive rebase, that would first
determine the branch topology of the patches to be rebased, create a
pseudo todo list for further editing, transform the result into a real
todo list (making heavy use of the `exec` command to "implement" the
missing todo list commands) and finally recreate the patch series on
top of the new base commit.

That was in 2013. And it took about three weeks to come up with the
design and implement it as an out-of-tree script. Needless to say, the
implementation needed quite a few years to stabilize, all the while the
design itself proved itself sound.

With this patch, the goodness of the Git garden shears comes to `git
rebase -i` itself. Passing the `--rebase-merges` option will generate
a todo list that can be understood readily, and where it is obvious
how to reorder commits. New branches can be introduced by inserting
`label` commands and calling `merge <label>`. And once this mode will
have become stable and universally accepted, we can deprecate the design
mistake that was `--preserve-merges`.

Link *1*:
https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
Link *2*:
https://github.com/git-for-windows/build-extra/blob/master/shears.sh

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt           |  20 ++-
 contrib/completion/git-completion.bash |   2 +-
 git-rebase--interactive.sh             |   1 +
 git-rebase.sh                          |   6 +
 t/t3430-rebase-merges.sh               | 179 +++++++++++++++++++++++++
 5 files changed, 206 insertions(+), 2 deletions(-)
 create mode 100755 t/t3430-rebase-merges.sh

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 3277ca14327..34e0f6a69c1 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -378,6 +378,23 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
+-r::
+--rebase-merges::
+	By default, a rebase will simply drop merge commits and only rebase
+	the non-merge commits. With this option, it will try to preserve
+	the branching structure within the commits that are to be rebased,
+	by recreating the merge commits. If a merge commit resolved any merge
+	or contained manual amendments, then they will have to be re-applied
+	manually.
++
+This mode is similar in spirit to `--preserve-merges`, but in contrast to
+that option works well in interactive rebases: commits can be reordered,
+inserted and dropped at will.
++
+It is currently only possible to recreate the merge commits using the
+`recursive` merge strategy; Different merge strategies can be used only via
+explicit `exec git merge -s <strategy> [...]` commands.
+
 -p::
 --preserve-merges::
 	Recreate merge commits instead of flattening the history by replaying
@@ -780,7 +797,8 @@ BUGS
 The todo list presented by `--preserve-merges --interactive` does not
 represent the topology of the revision graph.  Editing commits and
 rewording their commit messages should work fine, but attempts to
-reorder commits tend to produce counterintuitive results.
+reorder commits tend to produce counterintuitive results. Use
+`--rebase-merges` in such scenarios instead.
 
 For example, an attempt to rearrange
 ------------
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index a7570739454..d4c0a995c39 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1949,7 +1949,7 @@ _git_rebase ()
 	--*)
 		__gitcomp "
 			--onto --merge --strategy --interactive
-			--preserve-merges --stat --no-stat
+			--rebase-merges --preserve-merges --stat --no-stat
 			--committer-date-is-author-date --ignore-date
 			--ignore-whitespace --whitespace=
 			--autosquash --no-autosquash
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index ccd5254d1c9..7a3daf3e40c 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -970,6 +970,7 @@ git_rebase__interactive () {
 	init_revisions_and_shortrevisions
 
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
+		${rebase_merges:+--rebase-merges} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 
diff --git a/git-rebase.sh b/git-rebase.sh
index fb64ee1fe42..a64460fd25a 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,6 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
+r,rebase-merges!   try to rebase merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -88,6 +89,7 @@ type=
 state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
+rebase_merges=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -270,6 +272,10 @@ do
 	--allow-empty-message)
 		allow_empty_message=--allow-empty-message
 		;;
+	--rebase-merges)
+		rebase_merges=t
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
new file mode 100755
index 00000000000..5f0febb9970
--- /dev/null
+++ b/t/t3430-rebase-merges.sh
@@ -0,0 +1,179 @@
+#!/bin/sh
+#
+# Copyright (c) 2018 Johannes E. Schindelin
+#
+
+test_description='git rebase -i --rebase-merges
+
+This test runs git rebase "interactively", retaining the branch structure by
+recreating merge commits.
+
+Initial setup:
+
+    -- B --                   (first)
+   /       \
+ A - C - D - E - H            (master)
+       \       /
+         F - G                (second)
+'
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
+
+test_cmp_graph () {
+	cat >expect &&
+	git log --graph --boundary --format=%s "$@" >output &&
+	sed "s/ *$//" <output >output.trimmed &&
+	test_cmp expect output.trimmed
+}
+
+test_expect_success 'setup' '
+	write_script replace-editor.sh <<-\EOF &&
+	mv "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	cp script-from-scratch "$1"
+	EOF
+
+	test_commit A &&
+	git checkout -b first &&
+	test_commit B &&
+	git checkout master &&
+	test_commit C &&
+	test_commit D &&
+	git merge --no-commit B &&
+	test_tick &&
+	git commit -m E &&
+	git tag -m E E &&
+	git checkout -b second C &&
+	test_commit F &&
+	test_commit G &&
+	git checkout master &&
+	git merge --no-commit G &&
+	test_tick &&
+	git commit -m H &&
+	git tag -m H H
+'
+
+test_expect_success 'create completely different structure' '
+	cat >script-from-scratch <<-\EOF &&
+	label onto
+
+	# onebranch
+	pick G
+	pick D
+	label onebranch
+
+	# second
+	reset onto
+	pick B
+	label second
+
+	reset onto
+	merge -C H second
+	merge onebranch # Merge the topic branch '\''onebranch'\''
+	EOF
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_tick &&
+	git rebase -i -r A &&
+	test_cmp_graph <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	* |   H
+	|\ \
+	| |/
+	|/|
+	| * B
+	|/
+	* A
+	EOF
+'
+
+test_expect_success 'generate correct todo list' '
+	cat >expect <<-\EOF &&
+	label onto
+
+	reset onto
+	pick d9df450 B
+	label E
+
+	reset onto
+	pick 5dee784 C
+	label branch-point
+	pick ca2c861 F
+	pick 088b00a G
+	label H
+
+	reset branch-point # C
+	pick 12bd07b D
+	merge -C 2051b56 E # E
+	merge -C 233d48a H # H
+
+	EOF
+
+	grep -v "^#" <.git/ORIGINAL-TODO >output &&
+	test_cmp expect output
+'
+
+test_expect_success '`reset` refuses to overwrite untracked files' '
+	git checkout -b refuse-to-reset &&
+	test_commit dont-overwrite-untracked &&
+	git checkout @{-1} &&
+	: >dont-overwrite-untracked.t &&
+	echo "reset refs/tags/dont-overwrite-untracked" >script-from-scratch &&
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_must_fail git rebase -r HEAD &&
+	git rebase --abort
+'
+
+test_expect_success 'failed `merge` writes patch (may be rescheduled, too)' '
+	test_when_finished "test_might_fail git rebase --abort" &&
+	git checkout -b conflicting-merge A &&
+
+	: fail because of conflicting untracked file &&
+	>G.t &&
+	echo "merge -C H G" >script-from-scratch &&
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_tick &&
+	test_must_fail git rebase -ir HEAD &&
+	grep "^merge -C .* G$" .git/rebase-merge/done &&
+	grep "^merge -C .* G$" .git/rebase-merge/git-rebase-todo &&
+	test_path_is_file .git/rebase-merge/patch &&
+
+	: fail because of merge conflict &&
+	rm G.t .git/rebase-merge/patch &&
+	git reset --hard &&
+	test_commit conflicting-G G.t not-G conflicting-G &&
+	test_must_fail git rebase --continue &&
+	! grep "^merge -C .* G$" .git/rebase-merge/git-rebase-todo &&
+	test_path_is_file .git/rebase-merge/patch
+'
+
+test_expect_success 'with a branch tip that was cherry-picked already' '
+	git checkout -b already-upstream master &&
+	base="$(git rev-parse --verify HEAD)" &&
+
+	test_commit A1 &&
+	test_commit A2 &&
+	git reset --hard $base &&
+	test_commit B1 &&
+	test_tick &&
+	git merge -m "Merge branch A" A2 &&
+
+	git checkout -b upstream-with-a2 $base &&
+	test_tick &&
+	git cherry-pick A2 &&
+
+	git checkout already-upstream &&
+	test_tick &&
+	git rebase -i -r upstream-with-a2 &&
+	test_cmp_graph upstream-with-a2.. <<-\EOF
+	*   Merge branch A
+	|\
+	| * A1
+	* | B1
+	|/
+	o A2
+	EOF
+'
+
+test_done
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 10/16] rebase --rebase-merges: add test for --keep-empty
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (8 preceding siblings ...)
  2018-04-21 10:34             ` [PATCH v8 09/16] rebase: introduce the --rebase-merges option Johannes Schindelin
@ 2018-04-21 10:35             ` Johannes Schindelin
  2018-04-21 10:43             ` [PATCH v8 11/16] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
                               ` (6 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:35 UTC (permalink / raw)
  To: git
  Cc: Phillip Wood, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

From: Phillip Wood <phillip.wood@dunelm.org.uk>

If there are empty commits on the left hand side of $upstream...HEAD
then the empty commits on the right hand side that we want to keep are
being pruned.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
---
 t/t3421-rebase-topology-linear.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/t3421-rebase-topology-linear.sh b/t/t3421-rebase-topology-linear.sh
index 68fe2003ef5..fbae5dab7e2 100755
--- a/t/t3421-rebase-topology-linear.sh
+++ b/t/t3421-rebase-topology-linear.sh
@@ -217,6 +217,7 @@ test_run_rebase success ''
 test_run_rebase failure -m
 test_run_rebase failure -i
 test_run_rebase failure -p
+test_run_rebase success --rebase-merges
 
 #       m
 #      /
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 11/16] sequencer: make refs generated by the `label` command worktree-local
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (9 preceding siblings ...)
  2018-04-21 10:35             ` [PATCH v8 10/16] rebase --rebase-merges: add test for --keep-empty Johannes Schindelin
@ 2018-04-21 10:43             ` Johannes Schindelin
  2018-04-21 10:46             ` [PATCH v8 12/16] sequencer: handle post-rewrite for merge commands Johannes Schindelin
                               ` (5 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:43 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

This allows for rebases to be run in parallel in separate worktrees
(think: interrupted in the middle of one rebase, being asked to perform
a different rebase, adding a separate worktree just for that job).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 refs.c                   |  3 ++-
 t/t3430-rebase-merges.sh | 14 ++++++++++++++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 8b7a77fe5ee..f61ec58d1df 100644
--- a/refs.c
+++ b/refs.c
@@ -600,7 +600,8 @@ int dwim_log(const char *str, int len, struct object_id *oid, char **log)
 static int is_per_worktree_ref(const char *refname)
 {
 	return !strcmp(refname, "HEAD") ||
-		starts_with(refname, "refs/bisect/");
+		starts_with(refname, "refs/bisect/") ||
+		starts_with(refname, "refs/rewritten/");
 }
 
 static int is_pseudoref_syntax(const char *refname)
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index 5f0febb9970..96853784ec0 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -176,4 +176,18 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'refs/rewritten/* is worktree-local' '
+	git worktree add wt &&
+	cat >wt/script-from-scratch <<-\EOF &&
+	label xyz
+	exec GIT_DIR=../.git git rev-parse --verify refs/rewritten/xyz >a || :
+	exec git rev-parse --verify refs/rewritten/xyz >b
+	EOF
+
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	git -C wt rebase -i HEAD &&
+	test_must_be_empty wt/a &&
+	test_cmp_rev HEAD "$(cat wt/b)"
+'
+
 test_done
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 12/16] sequencer: handle post-rewrite for merge commands
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (10 preceding siblings ...)
  2018-04-21 10:43             ` [PATCH v8 11/16] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
@ 2018-04-21 10:46             ` Johannes Schindelin
  2018-04-21 10:47             ` [PATCH v8 13/16] rebase --rebase-merges: avoid "empty merges" Johannes Schindelin
                               ` (4 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:46 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

In the previous patches, we implemented the basic functionality of the
`git rebase -i --rebase-merges` command, in particular the `merge`
command to create merge commits in the sequencer.

The interactive rebase is a lot more these days, though, than a simple
cherry-pick in a loop. For example, it calls the post-rewrite hook (if
any) after rebasing with a mapping of the old->new commits.

This patch implements the post-rewrite handling for the `merge` command
we just introduced. The other commands that were added recently (`label`
and `reset`) do not create new commits, therefore post-rewrite hooks do
not need to handle them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c              |  5 ++++-
 t/t3430-rebase-merges.sh | 25 +++++++++++++++++++++++++
 2 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 1e17a11ca32..f3b4fe4d75f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3062,7 +3062,10 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					    item->arg, item->arg_len,
 					    item->flags, opts)) < 0)
 				reschedule = 1;
-			else if (res > 0)
+			else if (item->commit)
+				record_in_rewritten(&item->commit->object.oid,
+						    peek_command(todo_list, 1));
+			if (res > 0)
 				/* failed with merge conflicts */
 				return error_with_patch(item->commit,
 							item->arg,
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index 96853784ec0..e9c5dc1cd95 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -190,4 +190,29 @@ test_expect_success 'refs/rewritten/* is worktree-local' '
 	test_cmp_rev HEAD "$(cat wt/b)"
 '
 
+test_expect_success 'post-rewrite hook and fixups work for merges' '
+	git checkout -b post-rewrite &&
+	test_commit same1 &&
+	git reset --hard HEAD^ &&
+	test_commit same2 &&
+	git merge -m "to fix up" same1 &&
+	echo same old same old >same2.t &&
+	test_tick &&
+	git commit --fixup HEAD same2.t &&
+	fixup="$(git rev-parse HEAD)" &&
+
+	mkdir -p .git/hooks &&
+	test_when_finished "rm .git/hooks/post-rewrite" &&
+	echo "cat >actual" | write_script .git/hooks/post-rewrite &&
+
+	test_tick &&
+	git rebase -i --autosquash -r HEAD^^^ &&
+	printf "%s %s\n%s %s\n%s %s\n%s %s\n" >expect $(git rev-parse \
+		$fixup^^2 HEAD^2 \
+		$fixup^^ HEAD^ \
+		$fixup^ HEAD \
+		$fixup HEAD) &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 13/16] rebase --rebase-merges: avoid "empty merges"
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (11 preceding siblings ...)
  2018-04-21 10:46             ` [PATCH v8 12/16] sequencer: handle post-rewrite for merge commands Johannes Schindelin
@ 2018-04-21 10:47             ` Johannes Schindelin
  2018-04-21 10:49             ` [PATCH v8 14/16] pull: accept --rebase=merges to recreate the branch topology Johannes Schindelin
                               ` (3 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:47 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

The `git merge` command does not allow merging commits that are already
reachable from HEAD: `git merge HEAD^`, for example, will report that we
are already up to date and not change a thing.

In an interactive rebase, such a merge could occur previously, e.g. when
competing (or slightly modified) versions of a patch series were applied
upstream, and the user had to `git rebase --skip` all of the local
commits, and the topic branch becomes "empty" as a consequence.

Let's teach the todo command `merge` to behave the same as `git merge`.

Seeing as it requires some low-level trickery to create such merges with
Git's commands in the first place, we do not even have to bother to
introduce an option to force `merge` to create such merge commits.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c              | 7 +++++++
 t/t3430-rebase-merges.sh | 8 ++++++++
 2 files changed, 15 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index f3b4fe4d75f..443a0a0ee87 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2803,6 +2803,13 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	write_message("no-ff", 5, git_path_merge_mode(), 0);
 
 	bases = get_merge_bases(head_commit, merge_commit);
+	if (bases && !oidcmp(&merge_commit->object.oid,
+			     &bases->item->object.oid)) {
+		ret = 0;
+		/* skip merging an ancestor of HEAD */
+		goto leave_merge;
+	}
+
 	for (j = bases; j; j = j->next)
 		commit_list_insert(j->item, &reversed);
 	free_commit_list(bases);
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index e9c5dc1cd95..1628c8dcc20 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -215,4 +215,12 @@ test_expect_success 'post-rewrite hook and fixups work for merges' '
 	test_cmp expect actual
 '
 
+test_expect_success 'refuse to merge ancestors of HEAD' '
+	echo "merge HEAD^" >script-from-scratch &&
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	before="$(git rev-parse HEAD)" &&
+	git rebase -i HEAD &&
+	test_cmp_rev HEAD $before
+'
+
 test_done
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 14/16] pull: accept --rebase=merges to recreate the branch topology
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (12 preceding siblings ...)
  2018-04-21 10:47             ` [PATCH v8 13/16] rebase --rebase-merges: avoid "empty merges" Johannes Schindelin
@ 2018-04-21 10:49             ` Johannes Schindelin
  2018-04-21 10:57             ` [PATCH v8 15/16] rebase -i: introduce --rebase-merges=[no-]rebase-cousins Johannes Schindelin
                               ` (2 subsequent siblings)
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:49 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Similar to the `preserve` mode simply passing the `--preserve-merges`
option to the `rebase` command, the `merges` mode simply passes the
`--rebase-merges` option.

This will allow users to conveniently rebase non-trivial commit
topologies when pulling new commits, without flattening them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/config.txt               |  8 ++++++++
 Documentation/git-pull.txt             |  6 +++++-
 builtin/pull.c                         | 14 ++++++++++----
 builtin/remote.c                       | 18 ++++++++++++++----
 contrib/completion/git-completion.bash |  2 +-
 5 files changed, 38 insertions(+), 10 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 2659153cb37..d6bcb5dcb67 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1058,6 +1058,10 @@ branch.<name>.rebase::
 	"git pull" is run. See "pull.rebase" for doing this in a non
 	branch-specific manner.
 +
+When `merges`, pass the `--rebase-merges` option to 'git rebase'
+so that the local merge commits are included in the rebase (see
+linkgit:git-rebase[1] for details).
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
@@ -2617,6 +2621,10 @@ pull.rebase::
 	pull" is run. See "branch.<name>.rebase" for setting this on a
 	per-branch basis.
 +
+When `merges`, pass the `--rebase-merges` option to 'git rebase'
+so that the local merge commits are included in the rebase (see
+linkgit:git-rebase[1] for details).
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
diff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt
index ce05b7a5b13..4e0ad6fd8e0 100644
--- a/Documentation/git-pull.txt
+++ b/Documentation/git-pull.txt
@@ -101,13 +101,17 @@ Options related to merging
 include::merge-options.txt[]
 
 -r::
---rebase[=false|true|preserve|interactive]::
+--rebase[=false|true|merges|preserve|interactive]::
 	When true, rebase the current branch on top of the upstream
 	branch after fetching. If there is a remote-tracking branch
 	corresponding to the upstream branch and the upstream branch
 	was rebased since last fetched, the rebase uses that information
 	to avoid rebasing non-local changes.
 +
+When set to `merges`, rebase using `git rebase --rebase-merges` so that
+the local merge commits are included in the rebase (see
+linkgit:git-rebase[1] for details).
++
 When set to preserve, rebase with the `--preserve-merges` option passed
 to `git rebase` so that locally created merge commits will not be flattened.
 +
diff --git a/builtin/pull.c b/builtin/pull.c
index e32d6cd5b4c..70b44146ce4 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -27,14 +27,16 @@ enum rebase_type {
 	REBASE_FALSE = 0,
 	REBASE_TRUE,
 	REBASE_PRESERVE,
+	REBASE_MERGES,
 	REBASE_INTERACTIVE
 };
 
 /**
  * Parses the value of --rebase. If value is a false value, returns
  * REBASE_FALSE. If value is a true value, returns REBASE_TRUE. If value is
- * "preserve", returns REBASE_PRESERVE. If value is a invalid value, dies with
- * a fatal error if fatal is true, otherwise returns REBASE_INVALID.
+ * "merges", returns REBASE_MERGES. If value is "preserve", returns
+ * REBASE_PRESERVE. If value is a invalid value, dies with a fatal error if
+ * fatal is true, otherwise returns REBASE_INVALID.
  */
 static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		int fatal)
@@ -47,6 +49,8 @@ static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		return REBASE_TRUE;
 	else if (!strcmp(value, "preserve"))
 		return REBASE_PRESERVE;
+	else if (!strcmp(value, "merges"))
+		return REBASE_MERGES;
 	else if (!strcmp(value, "interactive"))
 		return REBASE_INTERACTIVE;
 
@@ -130,7 +134,7 @@ static struct option pull_options[] = {
 	/* Options passed to git-merge or git-rebase */
 	OPT_GROUP(N_("Options related to merging")),
 	{ OPTION_CALLBACK, 'r', "rebase", &opt_rebase,
-	  "false|true|preserve|interactive",
+	  "false|true|merges|preserve|interactive",
 	  N_("incorporate changes by rebasing rather than merging"),
 	  PARSE_OPT_OPTARG, parse_opt_rebase },
 	OPT_PASSTHRU('n', NULL, &opt_diffstat, NULL,
@@ -800,7 +804,9 @@ static int run_rebase(const struct object_id *curr_head,
 	argv_push_verbosity(&args);
 
 	/* Options passed to git-rebase */
-	if (opt_rebase == REBASE_PRESERVE)
+	if (opt_rebase == REBASE_MERGES)
+		argv_array_push(&args, "--rebase-merges");
+	else if (opt_rebase == REBASE_PRESERVE)
 		argv_array_push(&args, "--preserve-merges");
 	else if (opt_rebase == REBASE_INTERACTIVE)
 		argv_array_push(&args, "--interactive");
diff --git a/builtin/remote.c b/builtin/remote.c
index 805ffc05cdb..45c9219e07a 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -245,7 +245,9 @@ static int add(int argc, const char **argv)
 struct branch_info {
 	char *remote_name;
 	struct string_list merge;
-	enum { NO_REBASE, NORMAL_REBASE, INTERACTIVE_REBASE } rebase;
+	enum {
+		NO_REBASE, NORMAL_REBASE, INTERACTIVE_REBASE, REBASE_MERGES
+	} rebase;
 };
 
 static struct string_list branch_list = STRING_LIST_INIT_NODUP;
@@ -306,6 +308,8 @@ static int config_read_branches(const char *key, const char *value, void *cb)
 				info->rebase = v;
 			else if (!strcmp(value, "preserve"))
 				info->rebase = NORMAL_REBASE;
+			else if (!strcmp(value, "merges"))
+				info->rebase = REBASE_MERGES;
 			else if (!strcmp(value, "interactive"))
 				info->rebase = INTERACTIVE_REBASE;
 		}
@@ -963,9 +967,15 @@ static int show_local_info_item(struct string_list_item *item, void *cb_data)
 
 	printf("    %-*s ", show_info->width, item->string);
 	if (branch_info->rebase) {
-		printf_ln(branch_info->rebase == INTERACTIVE_REBASE
-			  ? _("rebases interactively onto remote %s")
-			  : _("rebases onto remote %s"), merge->items[0].string);
+		const char *msg;
+		if (branch_info->rebase == INTERACTIVE_REBASE)
+			msg = _("rebases interactively onto remote %s");
+		else if (branch_info->rebase == REBASE_MERGES)
+			msg = _("rebases interactively (with merges) onto "
+				"remote %s");
+		else
+			msg = _("rebases onto remote %s");
+		printf_ln(msg, merge->items[0].string);
 		return 0;
 	} else if (show_info->any_rebase) {
 		printf_ln(_(" merges with remote %s"), merge->items[0].string);
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index d4c0a995c39..6af65155c59 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2120,7 +2120,7 @@ _git_config ()
 		return
 		;;
 	branch.*.rebase)
-		__gitcomp "false true preserve interactive"
+		__gitcomp "false true merges preserve interactive"
 		return
 		;;
 	remote.pushdefault)
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 15/16] rebase -i: introduce --rebase-merges=[no-]rebase-cousins
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (13 preceding siblings ...)
  2018-04-21 10:49             ` [PATCH v8 14/16] pull: accept --rebase=merges to recreate the branch topology Johannes Schindelin
@ 2018-04-21 10:57             ` Johannes Schindelin
  2018-04-21 11:09             ` [PATCH v8 16/16] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 10:57 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

This one is a bit tricky to explain, so let's try with a diagram:

        C
      /   \
A - B - E - F
  \   /
    D

To illustrate what this new mode is all about, let's consider what
happens upon `git rebase -i --rebase-merges B`, in particular to
the commit `D`. So far, the new branch structure would be:

       --- C' --
      /         \
A - B ------ E' - F'
      \    /
        D'

This is not really preserving the branch topology from before! The
reason is that the commit `D` does not have `B` as ancestor, and
therefore it gets rebased onto `B`.

This is unintuitive behavior. Even worse, when recreating branch
structure, most use cases would appear to want cousins *not* to be
rebased onto the new base commit. For example, Git for Windows (the
heaviest user of the Git garden shears, which served as the blueprint
for --rebase-merges) frequently merges branches from `next` early, and
these branches certainly do *not* want to be rebased. In the example
above, the desired outcome would look like this:

       --- C' --
      /         \
A - B ------ E' - F'
  \        /
   -- D' --

Let's introduce the term "cousins" for such commits ("D" in the
example), and let's not rebase them by default, introducing the new
"rebase-cousins" mode for use cases where they should be rebased.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt |  7 ++++++-
 builtin/rebase--helper.c     |  9 ++++++++-
 git-rebase--interactive.sh   |  1 +
 git-rebase.sh                | 12 +++++++++++-
 sequencer.c                  |  4 ++++
 sequencer.h                  |  6 ++++++
 t/t3430-rebase-merges.sh     | 18 ++++++++++++++++++
 7 files changed, 54 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 34e0f6a69c1..841cf9cf385 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -379,7 +379,7 @@ rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
 -r::
---rebase-merges::
+--rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
 	By default, a rebase will simply drop merge commits and only rebase
 	the non-merge commits. With this option, it will try to preserve
 	the branching structure within the commits that are to be rebased,
@@ -387,6 +387,11 @@ have the long commit hash prepended to the format.
 	or contained manual amendments, then they will have to be re-applied
 	manually.
 +
+By default, or when `no-rebase-cousins` was specified, commits which do not
+have `<upstream>` as direct ancestor will keep their original branch point.
+If the `rebase-cousins` mode is turned on, such commits are instead rebased
+onto `<upstream>` (or `<onto>`, if specified).
++
 This mode is similar in spirit to `--preserve-merges`, but in contrast to
 that option works well in interactive rebases: commits can be reordered,
 inserted and dropped at will.
diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index 781782e7272..f7c2a5fdc81 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
 	unsigned flags = 0, keep_empty = 0, rebase_merges = 0;
-	int abbreviate_commands = 0;
+	int abbreviate_commands = 0, rebase_cousins = -1;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
 		CHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH,
@@ -25,6 +25,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
 			N_("allow commits with empty messages")),
 		OPT_BOOL(0, "rebase-merges", &rebase_merges, N_("rebase merge commits")),
+		OPT_BOOL(0, "rebase-cousins", &rebase_cousins,
+			 N_("keep original branch points of cousins")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -59,8 +61,13 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
 	flags |= rebase_merges ? TODO_LIST_REBASE_MERGES : 0;
+	flags |= rebase_cousins > 0 ? TODO_LIST_REBASE_COUSINS : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
+	if (rebase_cousins >= 0 && !rebase_merges)
+		warning(_("--[no-]rebase-cousins has no effect without "
+			  "--rebase-merges"));
+
 	if (command == CONTINUE && argc == 1)
 		return !!sequencer_continue(&opts);
 	if (command == ABORT && argc == 1)
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 7a3daf3e40c..b4ad130e8b1 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -971,6 +971,7 @@ git_rebase__interactive () {
 
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
 		${rebase_merges:+--rebase-merges} \
+		${rebase_cousins:+--rebase-cousins} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 
diff --git a/git-rebase.sh b/git-rebase.sh
index a64460fd25a..157705d2a72 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,7 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
-r,rebase-merges!   try to rebase merges instead of skipping them
+r,rebase-merges?   try to rebase merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -90,6 +90,7 @@ state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
 rebase_merges=
+rebase_cousins=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -276,6 +277,15 @@ do
 		rebase_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
 		;;
+	--rebase-merges=*)
+		rebase_merges=t
+		case "${1#*=}" in
+		rebase-cousins) rebase_cousins=t;;
+		no-rebase-cousins) rebase_cousins=;;
+		*) die "Unknown mode: $1";;
+		esac
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/sequencer.c b/sequencer.c
index 443a0a0ee87..9ffadbb3d3c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3566,6 +3566,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 				   unsigned flags)
 {
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	int rebase_cousins = flags & TODO_LIST_REBASE_COUSINS;
 	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
 	struct strbuf label = STRBUF_INIT;
 	struct commit_list *commits = NULL, **tail = &commits, *iter;
@@ -3743,6 +3744,9 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 					   &commit->object.oid);
 			if (entry)
 				to = entry->string;
+			else if (!rebase_cousins)
+				to = label_oid(&commit->object.oid, NULL,
+					       &state);
 
 			if (!to || !strcmp(to, "onto"))
 				fprintf(out, "%s onto\n", cmd_reset);
diff --git a/sequencer.h b/sequencer.h
index 6bc4da17243..d9570d92b11 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -60,6 +60,12 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
 #define TODO_LIST_REBASE_MERGES (1U << 3)
+/*
+ * When rebasing merges, commits that do have the base commit as ancestor
+ * ("cousins") are *not* rebased onto the new base by default. If those
+ * commits should be rebased onto the new base, this flag needs to be passed.
+ */
+#define TODO_LIST_REBASE_COUSINS (1U << 4)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index 1628c8dcc20..3d4dfdf7bec 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -176,6 +176,24 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'do not rebase cousins unless asked for' '
+	git checkout -b cousins master &&
+	before="$(git rev-parse --verify HEAD)" &&
+	test_tick &&
+	git rebase -r HEAD^ &&
+	test_cmp_rev HEAD $before &&
+	test_tick &&
+	git rebase --rebase-merges=rebase-cousins HEAD^ &&
+	test_cmp_graph HEAD^.. <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	|/
+	o H
+	EOF
+'
+
 test_expect_success 'refs/rewritten/* is worktree-local' '
 	git worktree add wt &&
 	cat >wt/script-from-scratch <<-\EOF &&
-- 
2.17.0.windows.1.15.gaa56ade3205



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v8 16/16] rebase -i --rebase-merges: add a section to the man page
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (14 preceding siblings ...)
  2018-04-21 10:57             ` [PATCH v8 15/16] rebase -i: introduce --rebase-merges=[no-]rebase-cousins Johannes Schindelin
@ 2018-04-21 11:09             ` Johannes Schindelin
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
  16 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-21 11:09 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

The --rebase-merges mode is probably not half as intuitive to use as
its inventor hopes, so let's document it some.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt | 135 +++++++++++++++++++++++++++++++++++
 1 file changed, 135 insertions(+)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 841cf9cf385..3b996e46d6a 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -399,6 +399,8 @@ inserted and dropped at will.
 It is currently only possible to recreate the merge commits using the
 `recursive` merge strategy; Different merge strategies can be used only via
 explicit `exec git merge -s <strategy> [...]` commands.
++
+See also REBASING MERGES below.
 
 -p::
 --preserve-merges::
@@ -797,6 +799,139 @@ The ripple effect of a "hard case" recovery is especially bad:
 'everyone' downstream from 'topic' will now have to perform a "hard
 case" recovery too!
 
+REBASING MERGES
+-----------------
+
+The interactive rebase command was originally designed to handle
+individual patch series. As such, it makes sense to exclude merge
+commits from the todo list, as the developer may have merged the
+then-current `master` while working on the branch, only to rebase
+all the commits onto `master` eventually (skipping the merge
+commits).
+
+However, there are legitimate reasons why a developer may want to
+recreate merge commits: to keep the branch structure (or "commit
+topology") when working on multiple, inter-related branches.
+
+In the following example, the developer works on a topic branch that
+refactors the way buttons are defined, and on another topic branch
+that uses that refactoring to implement a "Report a bug" button. The
+output of `git log --graph --format=%s -5` may look like this:
+
+------------
+*   Merge branch 'report-a-bug'
+|\
+| * Add the feedback button
+* | Merge branch 'refactor-button'
+|\ \
+| |/
+| * Use the Button class for all buttons
+| * Extract a generic Button class from the DownloadButton one
+------------
+
+The developer might want to rebase those commits to a newer `master`
+while keeping the branch topology, for example when the first topic
+branch is expected to be integrated into `master` much earlier than the
+second one, say, to resolve merge conflicts with changes to the
+DownloadButton class that made it into `master`.
+
+This rebase can be performed using the `--rebase-merges` option.
+It will generate a todo list looking like this:
+
+------------
+label onto
+
+# Branch: refactor-button
+reset onto
+pick 123456 Extract a generic Button class from the DownloadButton one
+pick 654321 Use the Button class for all buttons
+label refactor-button
+
+# Branch: report-a-bug
+reset refactor-button # Use the Button class for all buttons
+pick abcdef Add the feedback button
+label report-a-bug
+
+reset onto
+merge -C a1b2c3 refactor-button # Merge 'refactor-button'
+merge -C 6f5e4d report-a-bug # Merge 'report-a-bug'
+------------
+
+In contrast to a regular interactive rebase, there are `label`, `reset`
+and `merge` commands in addition to `pick` ones.
+
+The `label` command associates a label with the current HEAD when that
+command is executed. These labels are created as worktree-local refs
+(`refs/rewritten/<label>`) that will be deleted when the rebase
+finishes. That way, rebase operations in multiple worktrees linked to
+the same repository do not interfere with one another. If the `label`
+command fails, it is rescheduled immediately, with a helpful message how
+to proceed.
+
+The `reset` command resets the HEAD, index and worktree to the specified
+revision. It is isimilar to an `exec git reset --hard <label>`, but
+refuses to overwrite untracked files. If the `reset` command fails, it is
+rescheduled immediately, with a helpful message how to edit the todo list
+(this typically happens when a `reset` command was inserted into the todo
+list manually and contains a typo).
+
+The `merge` command will merge the specified revision into whatever is
+HEAD at that time. With `-C <original-commit>`, the commit message of
+the specified merge commit will be used. When the `-C` is changed to
+a lower-case `-c`, the message will be opened in an editor after a
+successful merge so that the user can edit the message.
+
+If a `merge` command fails for any reason other than merge conflicts (i.e.
+when the merge operation did not even start), it is rescheduled immediately.
+
+At this time, the `merge` command will *always* use the `recursive`
+merge strategy, with no way to choose a different one. To work around
+this, an `exec` command can be used to call `git merge` explicitly,
+using the fact that the labels are worktree-local refs (the ref
+`refs/rewritten/onto` would correspond to the label `onto`, for example).
+
+Note: the first command (`label onto`) labels the revision onto which
+the commits are rebased; The name `onto` is just a convention, as a nod
+to the `--onto` option.
+
+It is also possible to introduce completely new merge commits from scratch
+by adding a command of the form `merge <merge-head>`. This form will
+generate a tentative commit message and always open an editor to let the
+user edit it. This can be useful e.g. when a topic branch turns out to
+address more than a single concern and wants to be split into two or
+even more topic branches. Consider this todo list:
+
+------------
+pick 192837 Switch from GNU Makefiles to CMake
+pick 5a6c7e Document the switch to CMake
+pick 918273 Fix detection of OpenSSL in CMake
+pick afbecd http: add support for TLS v1.3
+pick fdbaec Fix detection of cURL in CMake on Windows
+------------
+
+The one commit in this list that is not related to CMake may very well
+have been motivated by working on fixing all those bugs introduced by
+switching to CMake, but it addresses a different concern. To split this
+branch into two topic branches, the todo list could be edited like this:
+
+------------
+label onto
+
+pick afbecd http: add support for TLS v1.3
+label tlsv1.3
+
+reset onto
+pick 192837 Switch from GNU Makefiles to CMake
+pick 918273 Fix detection of OpenSSL in CMake
+pick fdbaec Fix detection of cURL in CMake on Windows
+pick 5a6c7e Document the switch to CMake
+label cmake
+
+reset onto
+merge tlsv1.3
+merge cmake
+------------
+
 BUGS
 ----
 The todo list presented by `--preserve-merges --interactive` does not
-- 
2.17.0.windows.1.15.gaa56ade3205

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-21 10:33             ` [PATCH v8 06/16] sequencer: introduce the `merge` command Johannes Schindelin
@ 2018-04-21 15:56               ` Phillip Wood
  2018-04-22 17:17                 ` Phillip Wood
  2018-04-23 12:20                 ` Johannes Schindelin
  2018-04-22 12:01               ` Philip Oakley
  2018-04-22 13:55               ` Philip Oakley
  2 siblings, 2 replies; 412+ messages in thread
From: Phillip Wood @ 2018-04-21 15:56 UTC (permalink / raw)
  To: Johannes Schindelin, git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

On 21/04/18 11:33, Johannes Schindelin wrote:
> This patch is part of the effort to reimplement `--preserve-merges` with
> a substantially improved design, a design that has been developed in the
> Git for Windows project to maintain the dozens of Windows-specific patch
> series on top of upstream Git.
> 
> The previous patch implemented the `label` and `reset` commands to label
> commits and to reset to labeled commits. This patch adds the `merge`
> command, with the following syntax:

The two patches seem to have been fused together in this series.

If the reset command fails because it would overwrite untracked files it 
says

error: Untracked working tree file 'b' would be overwritten by merge.

Followed by the hint to edit the todo file. Saying 'merge' rather 
'reset' is possibly confusing to users. Perhaps it could call 
setup_unpack_trees_porcelain(), though that would need to be extended to 
handle 'reset'. Also it currently refuses to overwrite ignored files 
which is either annoying or safe depending on one's point of view.

Best Wishes

Phillip

> 
> 	merge [-C <commit>] <rev> # <oneline>
> 
> The <commit> parameter in this instance is the *original* merge commit,
> whose author and message will be used for the merge commit that is about
> to be created.
> 
> The <rev> parameter refers to the (possibly rewritten) revision to
> merge. Let's see an example of a todo list:
> 
> 	label onto
> 
> 	# Branch abc
> 	reset onto
> 	pick deadbeef Hello, world!
> 	label abc
> 
> 	reset onto
> 	pick cafecafe And now for something completely different
> 	merge -C baaabaaa abc # Merge the branch 'abc' into master
> 
> To edit the merge commit's message (a "reword" for merges, if you will),
> use `-c` (lower-case) instead of `-C`; this convention was borrowed from
> `git commit` that also supports `-c` and `-C` with similar meanings.
> 
> To create *new* merges, i.e. without copying the commit message from an
> existing commit, simply omit the `-C <commit>` parameter (which will
> open an editor for the merge message):
> 
> 	merge abc
> 
> This comes in handy when splitting a branch into two or more branches.
> 
> Note: this patch only adds support for recursive merges, to keep things
> simple. Support for octopus merges will be added later in a separate
> patch series, support for merges using strategies other than the
> recursive merge is left for the future.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>   git-rebase--interactive.sh |   6 +
>   sequencer.c                | 407 ++++++++++++++++++++++++++++++++++++-
>   2 files changed, 406 insertions(+), 7 deletions(-)
> 
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index e1b865f43f2..ccd5254d1c9 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -162,6 +162,12 @@ s, squash <commit> = use commit, but meld into previous commit
>   f, fixup <commit> = like \"squash\", but discard this commit's log message
>   x, exec <commit> = run command (the rest of the line) using shell
>   d, drop <commit> = remove commit
> +l, label <label> = label current HEAD with a name
> +t, reset <label> = reset HEAD to a label
> +m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
> +.       create a merge commit using the original merge commit's
> +.       message (or the oneline, if no original merge commit was
> +.       specified). Use -c <commit> to reword the commit message.
>   
>   These lines can be re-ordered; they are executed from top to bottom.
>   " | git stripspace --comment-lines >>"$todo"
> diff --git a/sequencer.c b/sequencer.c
> index 01443e0f245..35fcacbdf0f 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -23,6 +23,8 @@
>   #include "hashmap.h"
>   #include "notes-utils.h"
>   #include "sigchain.h"
> +#include "unpack-trees.h"
> +#include "worktree.h"
>   
>   #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
>   
> @@ -120,6 +122,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
>   static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
>   static GIT_PATH_FUNC(rebase_path_rewritten_pending,
>   	"rebase-merge/rewritten-pending")
> +
> +/*
> + * The path of the file listing refs that need to be deleted after the rebase
> + * finishes. This is used by the `label` command to record the need for cleanup.
> + */
> +static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
> +
>   /*
>    * The following files are written by git-rebase just after parsing the
>    * command-line (and are only consumed, not modified, by the sequencer).
> @@ -244,18 +253,34 @@ static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
>   
>   int sequencer_remove_state(struct replay_opts *opts)
>   {
> -	struct strbuf dir = STRBUF_INIT;
> +	struct strbuf buf = STRBUF_INIT;
>   	int i;
>   
> +	if (is_rebase_i(opts) &&
> +	    strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
> +		char *p = buf.buf;
> +		while (*p) {
> +			char *eol = strchr(p, '\n');
> +			if (eol)
> +				*eol = '\0';
> +			if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
> +				warning(_("could not delete '%s'"), p);
> +			if (!eol)
> +				break;
> +			p = eol + 1;
> +		}
> +	}
> +
>   	free(opts->gpg_sign);
>   	free(opts->strategy);
>   	for (i = 0; i < opts->xopts_nr; i++)
>   		free(opts->xopts[i]);
>   	free(opts->xopts);
>   
> -	strbuf_addstr(&dir, get_dir(opts));
> -	remove_dir_recursively(&dir, 0);
> -	strbuf_release(&dir);
> +	strbuf_reset(&buf);
> +	strbuf_addstr(&buf, get_dir(opts));
> +	remove_dir_recursively(&buf, 0);
> +	strbuf_release(&buf);
>   
>   	return 0;
>   }
> @@ -1279,6 +1304,9 @@ enum todo_command {
>   	TODO_SQUASH,
>   	/* commands that do something else than handling a single commit */
>   	TODO_EXEC,
> +	TODO_LABEL,
> +	TODO_RESET,
> +	TODO_MERGE,
>   	/* commands that do nothing but are counted for reporting progress */
>   	TODO_NOOP,
>   	TODO_DROP,
> @@ -1297,6 +1325,9 @@ static struct {
>   	{ 'f', "fixup" },
>   	{ 's', "squash" },
>   	{ 'x', "exec" },
> +	{ 'l', "label" },
> +	{ 't', "reset" },
> +	{ 'm', "merge" },
>   	{ 0,   "noop" },
>   	{ 'd', "drop" },
>   	{ 0,   NULL }
> @@ -1724,9 +1755,14 @@ static int read_and_refresh_cache(struct replay_opts *opts)
>   	return 0;
>   }
>   
> +enum todo_item_flags {
> +	TODO_EDIT_MERGE_MSG = 1
> +};
> +
>   struct todo_item {
>   	enum todo_command command;
>   	struct commit *commit;
> +	unsigned int flags;
>   	const char *arg;
>   	int arg_len;
>   	size_t offset_in_buf;
> @@ -1761,6 +1797,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>   	char *end_of_object_name;
>   	int i, saved, status, padding;
>   
> +	item->flags = 0;
> +
>   	/* left-trim */
>   	bol += strspn(bol, " \t");
>   
> @@ -1802,13 +1840,29 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>   		return error(_("missing arguments for %s"),
>   			     command_to_string(item->command));
>   
> -	if (item->command == TODO_EXEC) {
> +	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
> +	    item->command == TODO_RESET) {
>   		item->commit = NULL;
>   		item->arg = bol;
>   		item->arg_len = (int)(eol - bol);
>   		return 0;
>   	}
>   
> +	if (item->command == TODO_MERGE) {
> +		if (skip_prefix(bol, "-C", &bol))
> +			bol += strspn(bol, " \t");
> +		else if (skip_prefix(bol, "-c", &bol)) {
> +			bol += strspn(bol, " \t");
> +			item->flags |= TODO_EDIT_MERGE_MSG;
> +		} else {
> +			item->flags |= TODO_EDIT_MERGE_MSG;
> +			item->commit = NULL;
> +			item->arg = bol;
> +			item->arg_len = (int)(eol - bol);
> +			return 0;
> +		}
> +	}
> +
>   	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
>   	saved = *end_of_object_name;
>   	*end_of_object_name = '\0';
> @@ -2465,6 +2519,305 @@ static int do_exec(const char *command_line)
>   	return status;
>   }
>   
> +static int safe_append(const char *filename, const char *fmt, ...)
> +{
> +	va_list ap;
> +	struct lock_file lock = LOCK_INIT;
> +	int fd = hold_lock_file_for_update(&lock, filename,
> +					   LOCK_REPORT_ON_ERROR);
> +	struct strbuf buf = STRBUF_INIT;
> +
> +	if (fd < 0)
> +		return -1;
> +
> +	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT) {
> +		error_errno(_("could not read '%s'"), filename);
> +		rollback_lock_file(&lock);
> +		return -1;
> +	}
> +	strbuf_complete(&buf, '\n');
> +	va_start(ap, fmt);
> +	strbuf_vaddf(&buf, fmt, ap);
> +	va_end(ap);
> +
> +	if (write_in_full(fd, buf.buf, buf.len) < 0) {
> +		error_errno(_("could not write to '%s'"), filename);
> +		strbuf_release(&buf);
> +		rollback_lock_file(&lock);
> +		return -1;
> +	}
> +	if (commit_lock_file(&lock) < 0) {
> +		strbuf_release(&buf);
> +		rollback_lock_file(&lock);
> +		return error(_("failed to finalize '%s'"), filename);
> +	}
> +
> +	strbuf_release(&buf);
> +	return 0;
> +}
> +
> +static int do_label(const char *name, int len)
> +{
> +	struct ref_store *refs = get_main_ref_store();
> +	struct ref_transaction *transaction;
> +	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
> +	struct strbuf msg = STRBUF_INIT;
> +	int ret = 0;
> +	struct object_id head_oid;
> +
> +	if (len == 1 && *name == '#')
> +		return error("Illegal label name: '%.*s'", len, name);
> +
> +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> +	strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
> +
> +	transaction = ref_store_transaction_begin(refs, &err);
> +	if (!transaction) {
> +		error("%s", err.buf);
> +		ret = -1;
> +	} else if (get_oid("HEAD", &head_oid)) {
> +		error(_("could not read HEAD"));
> +		ret = -1;
> +	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
> +					  NULL, 0, msg.buf, &err) < 0 ||
> +		   ref_transaction_commit(transaction, &err)) {
> +		error("%s", err.buf);
> +		ret = -1;
> +	}
> +	ref_transaction_free(transaction);
> +	strbuf_release(&err);
> +	strbuf_release(&msg);
> +
> +	if (!ret)
> +		ret = safe_append(rebase_path_refs_to_delete(),
> +				  "%s\n", ref_name.buf);
> +	strbuf_release(&ref_name);
> +
> +	return ret;
> +}
> +
> +static const char *reflog_message(struct replay_opts *opts,
> +	const char *sub_action, const char *fmt, ...);
> +
> +static int do_reset(const char *name, int len, struct replay_opts *opts)
> +{
> +	struct strbuf ref_name = STRBUF_INIT;
> +	struct object_id oid;
> +	struct lock_file lock = LOCK_INIT;
> +	struct tree_desc desc;
> +	struct tree *tree;
> +	struct unpack_trees_options unpack_tree_opts;
> +	int ret = 0, i;
> +
> +	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> +		return -1;
> +
> +	/* Determine the length of the label */
> +	for (i = 0; i < len; i++)
> +		if (isspace(name[i]))
> +			len = i;
> +
> +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> +	if (get_oid(ref_name.buf, &oid) &&
> +	    get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
> +		error(_("could not read '%s'"), ref_name.buf);
> +		rollback_lock_file(&lock);
> +		strbuf_release(&ref_name);
> +		return -1;
> +	}
> +
> +	memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
> +	unpack_tree_opts.head_idx = 1;
> +	unpack_tree_opts.src_index = &the_index;
> +	unpack_tree_opts.dst_index = &the_index;
> +	unpack_tree_opts.fn = oneway_merge;
> +	unpack_tree_opts.merge = 1;
> +	unpack_tree_opts.update = 1;
> +
> +	if (read_cache_unmerged()) {
> +		rollback_lock_file(&lock);
> +		strbuf_release(&ref_name);
> +		return error_resolve_conflict(_(action_name(opts)));
> +	}
> +
> +	if (!fill_tree_descriptor(&desc, &oid)) {
> +		error(_("failed to find tree of %s"), oid_to_hex(&oid));
> +		rollback_lock_file(&lock);
> +		free((void *)desc.buffer);
> +		strbuf_release(&ref_name);
> +		return -1;
> +	}
> +
> +	if (unpack_trees(1, &desc, &unpack_tree_opts)) {
> +		rollback_lock_file(&lock);
> +		free((void *)desc.buffer);
> +		strbuf_release(&ref_name);
> +		return -1;
> +	}
> +
> +	tree = parse_tree_indirect(&oid);
> +	prime_cache_tree(&the_index, tree);
> +
> +	if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
> +		ret = error(_("could not write index"));
> +	free((void *)desc.buffer);
> +
> +	if (!ret)
> +		ret = update_ref(reflog_message(opts, "reset", "'%.*s'",
> +						len, name), "HEAD", &oid,
> +				 NULL, 0, UPDATE_REFS_MSG_ON_ERR);
> +
> +	strbuf_release(&ref_name);
> +	return ret;
> +}
> +
> +static int do_merge(struct commit *commit, const char *arg, int arg_len,
> +		    int flags, struct replay_opts *opts)
> +{
> +	int run_commit_flags = (flags & TODO_EDIT_MERGE_MSG) ?
> +		EDIT_MSG | VERIFY_MSG : 0;
> +	struct strbuf ref_name = STRBUF_INIT;
> +	struct commit *head_commit, *merge_commit, *i;
> +	struct commit_list *bases, *j, *reversed = NULL;
> +	struct merge_options o;
> +	int merge_arg_len, oneline_offset, ret;
> +	static struct lock_file lock;
> +	const char *p;
> +
> +	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0) {
> +		ret = -1;
> +		goto leave_merge;
> +	}
> +
> +	head_commit = lookup_commit_reference_by_name("HEAD");
> +	if (!head_commit) {
> +		ret = error(_("cannot merge without a current revision"));
> +		goto leave_merge;
> +	}
> +
> +	oneline_offset = arg_len;
> +	merge_arg_len = strcspn(arg, " \t\n");
> +	p = arg + merge_arg_len;
> +	p += strspn(p, " \t\n");
> +	if (*p == '#' && (!p[1] || isspace(p[1]))) {
> +		p += 1 + strspn(p + 1, " \t\n");
> +		oneline_offset = p - arg;
> +	} else if (p - arg < arg_len)
> +		BUG("octopus merges are not supported yet: '%s'", p);
> +
> +	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
> +	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> +	if (!merge_commit) {
> +		/* fall back to non-rewritten ref or commit */
> +		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
> +		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> +	}
> +
> +	if (!merge_commit) {
> +		ret = error(_("could not resolve '%s'"), ref_name.buf);
> +		goto leave_merge;
> +	}
> +
> +	if (commit) {
> +		const char *message = get_commit_buffer(commit, NULL);
> +		const char *body;
> +		int len;
> +
> +		if (!message) {
> +			ret = error(_("could not get commit message of '%s'"),
> +				    oid_to_hex(&commit->object.oid));
> +			goto leave_merge;
> +		}
> +		write_author_script(message);
> +		find_commit_subject(message, &body);
> +		len = strlen(body);
> +		ret = write_message(body, len, git_path_merge_msg(), 0);
> +		unuse_commit_buffer(commit, message);
> +		if (ret) {
> +			error_errno(_("could not write '%s'"),
> +				    git_path_merge_msg());
> +			goto leave_merge;
> +		}
> +	} else {
> +		struct strbuf buf = STRBUF_INIT;
> +		int len;
> +
> +		strbuf_addf(&buf, "author %s", git_author_info(0));
> +		write_author_script(buf.buf);
> +		strbuf_reset(&buf);
> +
> +		if (oneline_offset < arg_len) {
> +			p = arg + oneline_offset;
> +			len = arg_len - oneline_offset;
> +		} else {
> +			strbuf_addf(&buf, "Merge branch '%.*s'",
> +				    merge_arg_len, arg);
> +			p = buf.buf;
> +			len = buf.len;
> +		}
> +
> +		ret = write_message(p, len, git_path_merge_msg(), 0);
> +		strbuf_release(&buf);
> +		if (ret) {
> +			error_errno(_("could not write '%s'"),
> +				    git_path_merge_msg());
> +			goto leave_merge;
> +		}
> +	}
> +
> +	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
> +		      git_path_merge_head(), 0);
> +	write_message("no-ff", 5, git_path_merge_mode(), 0);
> +
> +	bases = get_merge_bases(head_commit, merge_commit);
> +	for (j = bases; j; j = j->next)
> +		commit_list_insert(j->item, &reversed);
> +	free_commit_list(bases);
> +
> +	read_cache();
> +	init_merge_options(&o);
> +	o.branch1 = "HEAD";
> +	o.branch2 = ref_name.buf;
> +	o.buffer_output = 2;
> +
> +	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
> +	if (ret <= 0)
> +		fputs(o.obuf.buf, stdout);
> +	strbuf_release(&o.obuf);
> +	if (ret < 0) {
> +		error(_("could not even attempt to merge '%.*s'"),
> +		      merge_arg_len, arg);
> +		goto leave_merge;
> +	}
> +	/*
> +	 * The return value of merge_recursive() is 1 on clean, and 0 on
> +	 * unclean merge.
> +	 *
> +	 * Let's reverse that, so that do_merge() returns 0 upon success and
> +	 * 1 upon failed merge (keeping the return value -1 for the cases where
> +	 * we will want to reschedule the `merge` command).
> +	 */
> +	ret = !ret;
> +
> +	if (active_cache_changed &&
> +	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
> +		ret = error(_("merge: Unable to write new index file"));
> +		goto leave_merge;
> +	}
> +
> +	rollback_lock_file(&lock);
> +	if (ret)
> +		rerere(opts->allow_rerere_auto);
> +	else
> +		ret = run_git_commit(git_path_merge_msg(), opts,
> +				     run_commit_flags);
> +
> +leave_merge:
> +	strbuf_release(&ref_name);
> +	rollback_lock_file(&lock);
> +	return ret;
> +}
> +
>   static int is_final_fixup(struct todo_list *todo_list)
>   {
>   	int i = todo_list->current;
> @@ -2568,7 +2921,7 @@ N_("Could not execute the todo command\n"
>   
>   static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>   {
> -	int res = 0;
> +	int res = 0, reschedule = 0;
>   
>   	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
>   	if (opts->allow_ff)
> @@ -2639,7 +2992,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>   					intend_to_amend();
>   				return error_failed_squash(item->commit, opts,
>   					item->arg_len, item->arg);
> -			} else if (res && is_rebase_i(opts))
> +			} else if (res && is_rebase_i(opts) && item->commit)
>   				return res | error_with_patch(item->commit,
>   					item->arg, item->arg_len, opts, res,
>   					item->command == TODO_REWORD);
> @@ -2665,9 +3018,41 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>   				/* `current` will be incremented below */
>   				todo_list->current = -1;
>   			}
> +		} else if (item->command == TODO_LABEL) {
> +			if ((res = do_label(item->arg, item->arg_len)))
> +				reschedule = 1;
> +		} else if (item->command == TODO_RESET) {
> +			if ((res = do_reset(item->arg, item->arg_len, opts)))
> +				reschedule = 1;
> +		} else if (item->command == TODO_MERGE) {
> +			if ((res = do_merge(item->commit,
> +					    item->arg, item->arg_len,
> +					    item->flags, opts)) < 0)
> +				reschedule = 1;
> +			else if (res > 0)
> +				/* failed with merge conflicts */
> +				return error_with_patch(item->commit,
> +							item->arg,
> +							item->arg_len, opts,
> +							res, 0);
>   		} else if (!is_noop(item->command))
>   			return error(_("unknown command %d"), item->command);
>   
> +		if (reschedule) {
> +			advise(_(rescheduled_advice),
> +			       get_item_line_length(todo_list,
> +						    todo_list->current),
> +			       get_item_line(todo_list, todo_list->current));
> +			todo_list->current--;
> +			if (save_todo(todo_list, opts))
> +				return -1;
> +			if (item->commit)
> +				return error_with_patch(item->commit,
> +							item->arg,
> +							item->arg_len, opts,
> +							res, 0);
> +		}
> +
>   		todo_list->current++;
>   		if (res)
>   			return res;
> @@ -3147,8 +3532,16 @@ int transform_todos(unsigned flags)
>   					  short_commit_name(item->commit) :
>   					  oid_to_hex(&item->commit->object.oid);
>   
> +			if (item->command == TODO_MERGE) {
> +				if (item->flags & TODO_EDIT_MERGE_MSG)
> +					strbuf_addstr(&buf, " -c");
> +				else
> +					strbuf_addstr(&buf, " -C");
> +			}
> +
>   			strbuf_addf(&buf, " %s", oid);
>   		}
> +
>   		/* add all the rest */
>   		if (!item->arg_len)
>   			strbuf_addch(&buf, '\n');
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-21 10:33             ` [PATCH v8 06/16] sequencer: introduce the `merge` command Johannes Schindelin
  2018-04-21 15:56               ` Phillip Wood
@ 2018-04-22 12:01               ` Philip Oakley
  2018-04-23 12:03                 ` Johannes Schindelin
  2018-04-22 13:55               ` Philip Oakley
  2 siblings, 1 reply; 412+ messages in thread
From: Philip Oakley @ 2018-04-22 12:01 UTC (permalink / raw)
  To: Johannes Schindelin, Git List
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Eric Sunshine,
	Phillip Wood, Igor Djordjevic, Johannes Sixt, Sergey Organov,
	Martin Ågren

From: "Johannes Schindelin" <johannes.schindelin@gmx.de>
> This patch is part of the effort to reimplement `--preserve-merges` with
> a substantially improved design, a design that has been developed in the
> Git for Windows project to maintain the dozens of Windows-specific patch
> series on top of upstream Git.
>
> The previous patch implemented the `label` and `reset` commands to label

The previous patch was [Patch 05/16] git-rebase--interactive: clarify
arguments, so this statement doesn't appear to be true. Has a patch been
missed or re-ordered? Or should it be simply "This patch implements" ?
Likewise the patch subject would be updated.

> commits and to reset to labeled commits. This patch adds the `merge`

s/adds/also adds/ ?

> command, with the following syntax:
>
> merge [-C <commit>] <rev> # <oneline>
>
> The <commit> parameter in this instance is the *original* merge commit,
> whose author and message will be used for the merge commit that is about
> to be created.
>
> The <rev> parameter refers to the (possibly rewritten) revision to
> merge. Let's see an example of a todo list:
>
The example ought to also note that `label onto` is to
`# label current HEAD with a name`, seeing as this is the first occurance.
It may be obvious in retrospect, but not at first reading.

> label onto
>
> # Branch abc
> reset onto

Is this reset strictly necessary. We are already there @head.

> pick deadbeef Hello, world!
> label abc
>
> reset onto
> pick cafecafe And now for something completely different
> merge -C baaabaaa abc # Merge the branch 'abc' into master
>
> To edit the merge commit's message (a "reword" for merges, if you will),
> use `-c` (lower-case) instead of `-C`; this convention was borrowed from
> `git commit` that also supports `-c` and `-C` with similar meanings.
>
> To create *new* merges, i.e. without copying the commit message from an
> existing commit, simply omit the `-C <commit>` parameter (which will
> open an editor for the merge message):
>
> merge abc
>
> This comes in handy when splitting a branch into two or more branches.
>
> Note: this patch only adds support for recursive merges, to keep things
> simple. Support for octopus merges will be added later in a separate
> patch series, support for merges using strategies other than the
> recursive merge is left for the future.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> git-rebase--interactive.sh |   6 +
> sequencer.c                | 407 ++++++++++++++++++++++++++++++++++++-
> 2 files changed, 406 insertions(+), 7 deletions(-)
>
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index e1b865f43f2..ccd5254d1c9 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -162,6 +162,12 @@ s, squash <commit> = use commit, but meld into
> previous commit
> f, fixup <commit> = like \"squash\", but discard this commit's log message
> x, exec <commit> = run command (the rest of the line) using shell
> d, drop <commit> = remove commit
> +l, label <label> = label current HEAD with a name
> +t, reset <label> = reset HEAD to a label
> +m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
> +.       create a merge commit using the original merge commit's
> +.       message (or the oneline, if no original merge commit was
> +.       specified). Use -c <commit> to reword the commit message.
>
> These lines can be re-ordered; they are executed from top to bottom.
> " | git stripspace --comment-lines >>"$todo"
> diff --git a/sequencer.c b/sequencer.c
> index 01443e0f245..35fcacbdf0f 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -23,6 +23,8 @@
> #include "hashmap.h"
> #include "notes-utils.h"
> #include "sigchain.h"
> +#include "unpack-trees.h"
> +#include "worktree.h"
>
> #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
>
> @@ -120,6 +122,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha,
> "rebase-merge/stopped-sha")
> static GIT_PATH_FUNC(rebase_path_rewritten_list,
> "rebase-merge/rewritten-list")
> static GIT_PATH_FUNC(rebase_path_rewritten_pending,
>  "rebase-merge/rewritten-pending")
> +
> +/*
> + * The path of the file listing refs that need to be deleted after the
> rebase
> + * finishes. This is used by the `label` command to record the need for
> cleanup.
> + */
> +static GIT_PATH_FUNC(rebase_path_refs_to_delete,
> "rebase-merge/refs-to-delete")
> +
> /*
>  * The following files are written by git-rebase just after parsing the
>  * command-line (and are only consumed, not modified, by the sequencer).
> @@ -244,18 +253,34 @@ static const char *gpg_sign_opt_quoted(struct
> replay_opts *opts)
>
> int sequencer_remove_state(struct replay_opts *opts)
> {
> - struct strbuf dir = STRBUF_INIT;
> + struct strbuf buf = STRBUF_INIT;
>  int i;
>
> + if (is_rebase_i(opts) &&
> +     strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
> + char *p = buf.buf;
> + while (*p) {
> + char *eol = strchr(p, '\n');
> + if (eol)
> + *eol = '\0';
> + if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
> + warning(_("could not delete '%s'"), p);
> + if (!eol)
> + break;
> + p = eol + 1;
> + }
> + }
> +
>  free(opts->gpg_sign);
>  free(opts->strategy);
>  for (i = 0; i < opts->xopts_nr; i++)
>  free(opts->xopts[i]);
>  free(opts->xopts);
>
> - strbuf_addstr(&dir, get_dir(opts));
> - remove_dir_recursively(&dir, 0);
> - strbuf_release(&dir);
> + strbuf_reset(&buf);
> + strbuf_addstr(&buf, get_dir(opts));
> + remove_dir_recursively(&buf, 0);
> + strbuf_release(&buf);
>
>  return 0;
> }
> @@ -1279,6 +1304,9 @@ enum todo_command {
>  TODO_SQUASH,
>  /* commands that do something else than handling a single commit */
>  TODO_EXEC,
> + TODO_LABEL,
> + TODO_RESET,
> + TODO_MERGE,
>  /* commands that do nothing but are counted for reporting progress */
>  TODO_NOOP,
>  TODO_DROP,
> @@ -1297,6 +1325,9 @@ static struct {
>  { 'f', "fixup" },
>  { 's', "squash" },
>  { 'x', "exec" },
> + { 'l', "label" },
> + { 't', "reset" },
> + { 'm', "merge" },
>  { 0,   "noop" },
>  { 'd', "drop" },
>  { 0,   NULL }
> @@ -1724,9 +1755,14 @@ static int read_and_refresh_cache(struct
> replay_opts *opts)
>  return 0;
> }
>
> +enum todo_item_flags {
> + TODO_EDIT_MERGE_MSG = 1
> +};
> +
> struct todo_item {
>  enum todo_command command;
>  struct commit *commit;
> + unsigned int flags;
>  const char *arg;
>  int arg_len;
>  size_t offset_in_buf;
> @@ -1761,6 +1797,8 @@ static int parse_insn_line(struct todo_item *item,
> const char *bol, char *eol)
>  char *end_of_object_name;
>  int i, saved, status, padding;
>
> + item->flags = 0;
> +
>  /* left-trim */
>  bol += strspn(bol, " \t");
>
> @@ -1802,13 +1840,29 @@ static int parse_insn_line(struct todo_item *item,
> const char *bol, char *eol)
>  return error(_("missing arguments for %s"),
>       command_to_string(item->command));
>
> - if (item->command == TODO_EXEC) {
> + if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
> +     item->command == TODO_RESET) {
>  item->commit = NULL;
>  item->arg = bol;
>  item->arg_len = (int)(eol - bol);
>  return 0;
>  }
>
> + if (item->command == TODO_MERGE) {
> + if (skip_prefix(bol, "-C", &bol))
> + bol += strspn(bol, " \t");
> + else if (skip_prefix(bol, "-c", &bol)) {
> + bol += strspn(bol, " \t");
> + item->flags |= TODO_EDIT_MERGE_MSG;
> + } else {
> + item->flags |= TODO_EDIT_MERGE_MSG;
> + item->commit = NULL;
> + item->arg = bol;
> + item->arg_len = (int)(eol - bol);
> + return 0;
> + }
> + }
> +
>  end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
>  saved = *end_of_object_name;
>  *end_of_object_name = '\0';
> @@ -2465,6 +2519,305 @@ static int do_exec(const char *command_line)
>  return status;
> }
>
> +static int safe_append(const char *filename, const char *fmt, ...)
> +{
> + va_list ap;
> + struct lock_file lock = LOCK_INIT;
> + int fd = hold_lock_file_for_update(&lock, filename,
> +    LOCK_REPORT_ON_ERROR);
> + struct strbuf buf = STRBUF_INIT;
> +
> + if (fd < 0)
> + return -1;
> +
> + if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT) {
> + error_errno(_("could not read '%s'"), filename);
> + rollback_lock_file(&lock);
> + return -1;
> + }
> + strbuf_complete(&buf, '\n');
> + va_start(ap, fmt);
> + strbuf_vaddf(&buf, fmt, ap);
> + va_end(ap);
> +
> + if (write_in_full(fd, buf.buf, buf.len) < 0) {
> + error_errno(_("could not write to '%s'"), filename);
> + strbuf_release(&buf);
> + rollback_lock_file(&lock);
> + return -1;
> + }
> + if (commit_lock_file(&lock) < 0) {
> + strbuf_release(&buf);
> + rollback_lock_file(&lock);
> + return error(_("failed to finalize '%s'"), filename);
> + }
> +
> + strbuf_release(&buf);
> + return 0;
> +}
> +
> +static int do_label(const char *name, int len)
> +{
> + struct ref_store *refs = get_main_ref_store();
> + struct ref_transaction *transaction;
> + struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
> + struct strbuf msg = STRBUF_INIT;
> + int ret = 0;
> + struct object_id head_oid;
> +
> + if (len == 1 && *name == '#')
> + return error("Illegal label name: '%.*s'", len, name);
> +
> + strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> + strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
> +
> + transaction = ref_store_transaction_begin(refs, &err);
> + if (!transaction) {
> + error("%s", err.buf);
> + ret = -1;
> + } else if (get_oid("HEAD", &head_oid)) {
> + error(_("could not read HEAD"));
> + ret = -1;
> + } else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
> +   NULL, 0, msg.buf, &err) < 0 ||
> +    ref_transaction_commit(transaction, &err)) {
> + error("%s", err.buf);
> + ret = -1;
> + }
> + ref_transaction_free(transaction);
> + strbuf_release(&err);
> + strbuf_release(&msg);
> +
> + if (!ret)
> + ret = safe_append(rebase_path_refs_to_delete(),
> +   "%s\n", ref_name.buf);
> + strbuf_release(&ref_name);
> +
> + return ret;
> +}
> +
> +static const char *reflog_message(struct replay_opts *opts,
> + const char *sub_action, const char *fmt, ...);
> +
> +static int do_reset(const char *name, int len, struct replay_opts *opts)
> +{
> + struct strbuf ref_name = STRBUF_INIT;
> + struct object_id oid;
> + struct lock_file lock = LOCK_INIT;
> + struct tree_desc desc;
> + struct tree *tree;
> + struct unpack_trees_options unpack_tree_opts;
> + int ret = 0, i;
> +
> + if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> + return -1;
> +
> + /* Determine the length of the label */
> + for (i = 0; i < len; i++)
> + if (isspace(name[i]))
> + len = i;
> +
> + strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> + if (get_oid(ref_name.buf, &oid) &&
> +     get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
> + error(_("could not read '%s'"), ref_name.buf);
> + rollback_lock_file(&lock);
> + strbuf_release(&ref_name);
> + return -1;
> + }
> +
> + memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
> + unpack_tree_opts.head_idx = 1;
> + unpack_tree_opts.src_index = &the_index;
> + unpack_tree_opts.dst_index = &the_index;
> + unpack_tree_opts.fn = oneway_merge;
> + unpack_tree_opts.merge = 1;
> + unpack_tree_opts.update = 1;
> +
> + if (read_cache_unmerged()) {
> + rollback_lock_file(&lock);
> + strbuf_release(&ref_name);
> + return error_resolve_conflict(_(action_name(opts)));
> + }
> +
> + if (!fill_tree_descriptor(&desc, &oid)) {
> + error(_("failed to find tree of %s"), oid_to_hex(&oid));
> + rollback_lock_file(&lock);
> + free((void *)desc.buffer);
> + strbuf_release(&ref_name);
> + return -1;
> + }
> +
> + if (unpack_trees(1, &desc, &unpack_tree_opts)) {
> + rollback_lock_file(&lock);
> + free((void *)desc.buffer);
> + strbuf_release(&ref_name);
> + return -1;
> + }
> +
> + tree = parse_tree_indirect(&oid);
> + prime_cache_tree(&the_index, tree);
> +
> + if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
> + ret = error(_("could not write index"));
> + free((void *)desc.buffer);
> +
> + if (!ret)
> + ret = update_ref(reflog_message(opts, "reset", "'%.*s'",
> + len, name), "HEAD", &oid,
> + NULL, 0, UPDATE_REFS_MSG_ON_ERR);
> +
> + strbuf_release(&ref_name);
> + return ret;
> +}
> +
> +static int do_merge(struct commit *commit, const char *arg, int arg_len,
> +     int flags, struct replay_opts *opts)
> +{
> + int run_commit_flags = (flags & TODO_EDIT_MERGE_MSG) ?
> + EDIT_MSG | VERIFY_MSG : 0;
> + struct strbuf ref_name = STRBUF_INIT;
> + struct commit *head_commit, *merge_commit, *i;
> + struct commit_list *bases, *j, *reversed = NULL;
> + struct merge_options o;
> + int merge_arg_len, oneline_offset, ret;
> + static struct lock_file lock;
> + const char *p;
> +
> + if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0) {
> + ret = -1;
> + goto leave_merge;
> + }
> +
> + head_commit = lookup_commit_reference_by_name("HEAD");
> + if (!head_commit) {
> + ret = error(_("cannot merge without a current revision"));
> + goto leave_merge;
> + }
> +
> + oneline_offset = arg_len;
> + merge_arg_len = strcspn(arg, " \t\n");
> + p = arg + merge_arg_len;
> + p += strspn(p, " \t\n");
> + if (*p == '#' && (!p[1] || isspace(p[1]))) {
> + p += 1 + strspn(p + 1, " \t\n");
> + oneline_offset = p - arg;
> + } else if (p - arg < arg_len)
> + BUG("octopus merges are not supported yet: '%s'", p);
> +
> + strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
> + merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> + if (!merge_commit) {
> + /* fall back to non-rewritten ref or commit */
> + strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
> + merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> + }
> +
> + if (!merge_commit) {
> + ret = error(_("could not resolve '%s'"), ref_name.buf);
> + goto leave_merge;
> + }
> +
> + if (commit) {
> + const char *message = get_commit_buffer(commit, NULL);
> + const char *body;
> + int len;
> +
> + if (!message) {
> + ret = error(_("could not get commit message of '%s'"),
> +     oid_to_hex(&commit->object.oid));
> + goto leave_merge;
> + }
> + write_author_script(message);
> + find_commit_subject(message, &body);
> + len = strlen(body);
> + ret = write_message(body, len, git_path_merge_msg(), 0);
> + unuse_commit_buffer(commit, message);
> + if (ret) {
> + error_errno(_("could not write '%s'"),
> +     git_path_merge_msg());
> + goto leave_merge;
> + }
> + } else {
> + struct strbuf buf = STRBUF_INIT;
> + int len;
> +
> + strbuf_addf(&buf, "author %s", git_author_info(0));
> + write_author_script(buf.buf);
> + strbuf_reset(&buf);
> +
> + if (oneline_offset < arg_len) {
> + p = arg + oneline_offset;
> + len = arg_len - oneline_offset;
> + } else {
> + strbuf_addf(&buf, "Merge branch '%.*s'",
> +     merge_arg_len, arg);
> + p = buf.buf;
> + len = buf.len;
> + }
> +
> + ret = write_message(p, len, git_path_merge_msg(), 0);
> + strbuf_release(&buf);
> + if (ret) {
> + error_errno(_("could not write '%s'"),
> +     git_path_merge_msg());
> + goto leave_merge;
> + }
> + }
> +
> + write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
> +       git_path_merge_head(), 0);
> + write_message("no-ff", 5, git_path_merge_mode(), 0);
> +
> + bases = get_merge_bases(head_commit, merge_commit);
> + for (j = bases; j; j = j->next)
> + commit_list_insert(j->item, &reversed);
> + free_commit_list(bases);
> +
> + read_cache();
> + init_merge_options(&o);
> + o.branch1 = "HEAD";
> + o.branch2 = ref_name.buf;
> + o.buffer_output = 2;
> +
> + ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
> + if (ret <= 0)
> + fputs(o.obuf.buf, stdout);
> + strbuf_release(&o.obuf);
> + if (ret < 0) {
> + error(_("could not even attempt to merge '%.*s'"),
> +       merge_arg_len, arg);
> + goto leave_merge;
> + }
> + /*
> + * The return value of merge_recursive() is 1 on clean, and 0 on
> + * unclean merge.
> + *
> + * Let's reverse that, so that do_merge() returns 0 upon success and
> + * 1 upon failed merge (keeping the return value -1 for the cases where
> + * we will want to reschedule the `merge` command).
> + */
> + ret = !ret;
> +
> + if (active_cache_changed &&
> +     write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
> + ret = error(_("merge: Unable to write new index file"));
> + goto leave_merge;
> + }
> +
> + rollback_lock_file(&lock);
> + if (ret)
> + rerere(opts->allow_rerere_auto);
> + else
> + ret = run_git_commit(git_path_merge_msg(), opts,
> +      run_commit_flags);
> +
> +leave_merge:
> + strbuf_release(&ref_name);
> + rollback_lock_file(&lock);
> + return ret;
> +}
> +
> static int is_final_fixup(struct todo_list *todo_list)
> {
>  int i = todo_list->current;
> @@ -2568,7 +2921,7 @@ N_("Could not execute the todo command\n"
>
> static int pick_commits(struct todo_list *todo_list, struct replay_opts
> *opts)
> {
> - int res = 0;
> + int res = 0, reschedule = 0;
>
>  setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
>  if (opts->allow_ff)
> @@ -2639,7 +2992,7 @@ static int pick_commits(struct todo_list *todo_list,
> struct replay_opts *opts)
>  intend_to_amend();
>  return error_failed_squash(item->commit, opts,
>  item->arg_len, item->arg);
> - } else if (res && is_rebase_i(opts))
> + } else if (res && is_rebase_i(opts) && item->commit)
>  return res | error_with_patch(item->commit,
>  item->arg, item->arg_len, opts, res,
>  item->command == TODO_REWORD);
> @@ -2665,9 +3018,41 @@ static int pick_commits(struct todo_list
> *todo_list, struct replay_opts *opts)
>  /* `current` will be incremented below */
>  todo_list->current = -1;
>  }
> + } else if (item->command == TODO_LABEL) {
> + if ((res = do_label(item->arg, item->arg_len)))
> + reschedule = 1;
> + } else if (item->command == TODO_RESET) {
> + if ((res = do_reset(item->arg, item->arg_len, opts)))
> + reschedule = 1;
> + } else if (item->command == TODO_MERGE) {
> + if ((res = do_merge(item->commit,
> +     item->arg, item->arg_len,
> +     item->flags, opts)) < 0)
> + reschedule = 1;
> + else if (res > 0)
> + /* failed with merge conflicts */
> + return error_with_patch(item->commit,
> + item->arg,
> + item->arg_len, opts,
> + res, 0);
>  } else if (!is_noop(item->command))
>  return error(_("unknown command %d"), item->command);
>
> + if (reschedule) {
> + advise(_(rescheduled_advice),
> +        get_item_line_length(todo_list,
> +     todo_list->current),
> +        get_item_line(todo_list, todo_list->current));
> + todo_list->current--;
> + if (save_todo(todo_list, opts))
> + return -1;
> + if (item->commit)
> + return error_with_patch(item->commit,
> + item->arg,
> + item->arg_len, opts,
> + res, 0);
> + }
> +
>  todo_list->current++;
>  if (res)
>  return res;
> @@ -3147,8 +3532,16 @@ int transform_todos(unsigned flags)
>    short_commit_name(item->commit) :
>    oid_to_hex(&item->commit->object.oid);
>
> + if (item->command == TODO_MERGE) {
> + if (item->flags & TODO_EDIT_MERGE_MSG)
> + strbuf_addstr(&buf, " -c");
> + else
> + strbuf_addstr(&buf, " -C");
> + }
> +
>  strbuf_addf(&buf, " %s", oid);
>  }
> +
>  /* add all the rest */
>  if (!item->arg_len)
>  strbuf_addch(&buf, '\n');
> -- 
> 2.17.0.windows.1.15.gaa56ade3205
>
>
>


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 08/16] rebase-helper --make-script: introduce a flag to rebase merges
  2018-04-21 10:34             ` [PATCH v8 08/16] rebase-helper --make-script: introduce a flag to rebase merges Johannes Schindelin
@ 2018-04-22 13:42               ` Philip Oakley
  2018-04-24  8:33                 ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Philip Oakley @ 2018-04-22 13:42 UTC (permalink / raw)
  To: Johannes Schindelin, Git List
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Eric Sunshine,
	Phillip Wood, Igor Djordjevic, Johannes Sixt, Sergey Organov,
	Martin Ågren

From: "Johannes Schindelin" <johannes.schindelin@gmx.de>

Sorry for the very late in the series comments..

> The sequencer just learned new commands intended to recreate branch
> structure (similar in spirit to --preserve-merges, but with a
> substantially less-broken design).
>
> Let's allow the rebase--helper to generate todo lists making use of
> these commands, triggered by the new --rebase-merges option. For a
> commit topology like this (where the HEAD points to C):
>
> - A - B - C
>     \   /
>       D
>
> the generated todo list would look like this:
>
> # branch D
> pick 0123 A
> label branch-point
> pick 1234 D
> label D
>
> reset branch-point
> pick 2345 B
> merge -C 3456 D # C
>
> To keep things simple, we first only implement support for merge commits
> with exactly two parents, leaving support for octopus merges to a later
> patch series.
>
For the first time reader this (below) isn't as obvious as may be thought.
maybe we should be a little more explicit here.

> As a special, hard-coded label, all merge-rebasing todo lists start with
> the command `label onto`

.. which labels the start point head with the name 'onto' ...

Maybe even:
"All merge-rebasing todo lists start with, as a convenience, a hard-coded
`label onto` line which will label the start point's head" ...

>                so that we can later always refer to the revision
> onto which everything is rebased.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> builtin/rebase--helper.c |   4 +-
> sequencer.c              | 351 ++++++++++++++++++++++++++++++++++++++-
> sequencer.h              |   1 +
> 3 files changed, 353 insertions(+), 3 deletions(-)
>
> diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
> index ad074705bb5..781782e7272 100644
> --- a/builtin/rebase--helper.c
> +++ b/builtin/rebase--helper.c
> @@ -12,7 +12,7 @@ static const char * const builtin_rebase_helper_usage[]
> = {
> int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
> {
>  struct replay_opts opts = REPLAY_OPTS_INIT;
> - unsigned flags = 0, keep_empty = 0;
> + unsigned flags = 0, keep_empty = 0, rebase_merges = 0;
>  int abbreviate_commands = 0;
>  enum {
>  CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
> @@ -24,6 +24,7 @@ int cmd_rebase__helper(int argc, const char **argv,
> const char *prefix)
>  OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
>  OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
>  N_("allow commits with empty messages")),
> + OPT_BOOL(0, "rebase-merges", &rebase_merges, N_("rebase merge
> commits")),
>  OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
>  CONTINUE),
>  OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
> @@ -57,6 +58,7 @@ int cmd_rebase__helper(int argc, const char **argv,
> const char *prefix)
>
>  flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
>  flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
> + flags |= rebase_merges ? TODO_LIST_REBASE_MERGES : 0;
>  flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
>
>  if (command == CONTINUE && argc == 1)
> diff --git a/sequencer.c b/sequencer.c
> index 5944d3a34eb..1e17a11ca32 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -25,6 +25,8 @@
> #include "sigchain.h"
> #include "unpack-trees.h"
> #include "worktree.h"
> +#include "oidmap.h"
> +#include "oidset.h"
>
> #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
>
> @@ -3436,6 +3438,343 @@ void append_signoff(struct strbuf *msgbuf, int
> ignore_footer, unsigned flag)
>  strbuf_release(&sob);
> }
>
> +struct labels_entry {
> + struct hashmap_entry entry;
> + char label[FLEX_ARRAY];
> +};
> +
> +static int labels_cmp(const void *fndata, const struct labels_entry *a,
> +       const struct labels_entry *b, const void *key)
> +{
> + return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
> +}
> +
> +struct string_entry {
> + struct oidmap_entry entry;
> + char string[FLEX_ARRAY];
> +};
> +
> +struct label_state {
> + struct oidmap commit2label;
> + struct hashmap labels;
> + struct strbuf buf;
> +};
> +
> +static const char *label_oid(struct object_id *oid, const char *label,
> +      struct label_state *state)
> +{
> + struct labels_entry *labels_entry;
> + struct string_entry *string_entry;
> + struct object_id dummy;
> + size_t len;
> + int i;
> +
> + string_entry = oidmap_get(&state->commit2label, oid);
> + if (string_entry)
> + return string_entry->string;
> +
> + /*
> + * For "uninteresting" commits, i.e. commits that are not to be
> + * rebased, and which can therefore not be labeled, we use a unique
> + * abbreviation of the commit name. This is slightly more complicated
> + * than calling find_unique_abbrev() because we also need to make
> + * sure that the abbreviation does not conflict with any other
> + * label.
> + *
> + * We disallow "interesting" commits to be labeled by a string that
> + * is a valid full-length hash, to ensure that we always can find an
> + * abbreviation for any uninteresting commit's names that does not
> + * clash with any other label.
> + */
> + if (!label) {
> + char *p;
> +
> + strbuf_reset(&state->buf);
> + strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
> + label = p = state->buf.buf;
> +
> + find_unique_abbrev_r(p, oid, default_abbrev);
> +
> + /*
> + * We may need to extend the abbreviated hash so that there is
> + * no conflicting label.
> + */
> + if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
> + size_t i = strlen(p) + 1;
> +
> + oid_to_hex_r(p, oid);
> + for (; i < GIT_SHA1_HEXSZ; i++) {
> + char save = p[i];
> + p[i] = '\0';
> + if (!hashmap_get_from_hash(&state->labels,
> +    strihash(p), p))
> + break;
> + p[i] = save;
> + }
> + }
> + } else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
> +     !get_oid_hex(label, &dummy)) ||
> +    (len == 1 && *label == '#') ||
> +    hashmap_get_from_hash(&state->labels,
> + strihash(label), label)) {
> + /*
> + * If the label already exists, or if the label is a valid full
> + * OID, or the label is a '#' (which we use as a separator
> + * between merge heads and oneline), we append a dash and a
> + * number to make it unique.
> + */
> + struct strbuf *buf = &state->buf;
> +
> + strbuf_reset(buf);
> + strbuf_add(buf, label, len);
> +
> + for (i = 2; ; i++) {
> + strbuf_setlen(buf, len);
> + strbuf_addf(buf, "-%d", i);
> + if (!hashmap_get_from_hash(&state->labels,
> +    strihash(buf->buf),
> +    buf->buf))
> + break;
> + }
> +
> + label = buf->buf;
> + }
> +
> + FLEX_ALLOC_STR(labels_entry, label, label);
> + hashmap_entry_init(labels_entry, strihash(label));
> + hashmap_add(&state->labels, labels_entry);
> +
> + FLEX_ALLOC_STR(string_entry, string, label);
> + oidcpy(&string_entry->entry.oid, oid);
> + oidmap_put(&state->commit2label, string_entry);
> +
> + return string_entry->string;
> +}
> +
> +static int make_script_with_merges(struct pretty_print_context *pp,
> +    struct rev_info *revs, FILE *out,
> +    unsigned flags)
> +{
> + int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
> + struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
> + struct strbuf label = STRBUF_INIT;
> + struct commit_list *commits = NULL, **tail = &commits, *iter;
> + struct commit_list *tips = NULL, **tips_tail = &tips;
> + struct commit *commit;
> + struct oidmap commit2todo = OIDMAP_INIT;
> + struct string_entry *entry;
> + struct oidset interesting = OIDSET_INIT, child_seen = OIDSET_INIT,
> + shown = OIDSET_INIT;
> + struct label_state state = { OIDMAP_INIT, { NULL }, STRBUF_INIT };
> +
> + int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
> + const char *cmd_pick = abbr ? "p" : "pick",
> + *cmd_label = abbr ? "l" : "label",
> + *cmd_reset = abbr ? "t" : "reset",
> + *cmd_merge = abbr ? "m" : "merge";
> +
> + oidmap_init(&commit2todo, 0);
> + oidmap_init(&state.commit2label, 0);
> + hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
> + strbuf_init(&state.buf, 32);
> +
> + if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
> + struct object_id *oid = &revs->cmdline.rev[0].item->oid;
> + FLEX_ALLOC_STR(entry, string, "onto");
> + oidcpy(&entry->entry.oid, oid);
> + oidmap_put(&state.commit2label, entry);
> + }
> +
> + /*
> + * First phase:
> + * - get onelines for all commits
> + * - gather all branch tips (i.e. 2nd or later parents of merges)
> + * - label all branch tips
> + */
> + while ((commit = get_revision(revs))) {
> + struct commit_list *to_merge;
> + int is_octopus;
> + const char *p1, *p2;
> + struct object_id *oid;
> + int is_empty;
> +
> + tail = &commit_list_insert(commit, tail)->next;
> + oidset_insert(&interesting, &commit->object.oid);
> +
> + is_empty = is_original_commit_empty(commit);
> + if (!is_empty && (commit->object.flags & PATCHSAME))
> + continue;
> +
> + strbuf_reset(&oneline);
> + pretty_print_commit(pp, commit, &oneline);
> +
> + to_merge = commit->parents ? commit->parents->next : NULL;
> + if (!to_merge) {
> + /* non-merge commit: easy case */
> + strbuf_reset(&buf);
> + if (!keep_empty && is_empty)
> + strbuf_addf(&buf, "%c ", comment_line_char);
> + strbuf_addf(&buf, "%s %s %s", cmd_pick,
> +     oid_to_hex(&commit->object.oid),
> +     oneline.buf);
> +
> + FLEX_ALLOC_STR(entry, string, buf.buf);
> + oidcpy(&entry->entry.oid, &commit->object.oid);
> + oidmap_put(&commit2todo, entry);
> +
> + continue;
> + }
> +
> + is_octopus = to_merge && to_merge->next;
> +
> + if (is_octopus)
> + BUG("Octopus merges not yet supported");
> +
> + /* Create a label */
> + strbuf_reset(&label);
> + if (skip_prefix(oneline.buf, "Merge ", &p1) &&
> +     (p1 = strchr(p1, '\'')) &&
> +     (p2 = strchr(++p1, '\'')))
> + strbuf_add(&label, p1, p2 - p1);
> + else if (skip_prefix(oneline.buf, "Merge pull request ",
> +      &p1) &&
> + (p1 = strstr(p1, " from ")))
> + strbuf_addstr(&label, p1 + strlen(" from "));
> + else
> + strbuf_addbuf(&label, &oneline);
> +
> + for (p1 = label.buf; *p1; p1++)
> + if (isspace(*p1))
> + *(char *)p1 = '-';
> +
> + strbuf_reset(&buf);
> + strbuf_addf(&buf, "%s -C %s",
> +     cmd_merge, oid_to_hex(&commit->object.oid));
> +
> + /* label the tip of merged branch */
> + oid = &to_merge->item->object.oid;
> + strbuf_addch(&buf, ' ');
> +
> + if (!oidset_contains(&interesting, oid))
> + strbuf_addstr(&buf, label_oid(oid, NULL, &state));
> + else {
> + tips_tail = &commit_list_insert(to_merge->item,
> + tips_tail)->next;
> +
> + strbuf_addstr(&buf, label_oid(oid, label.buf, &state));
> + }
> + strbuf_addf(&buf, " # %s", oneline.buf);
> +
> + FLEX_ALLOC_STR(entry, string, buf.buf);
> + oidcpy(&entry->entry.oid, &commit->object.oid);
> + oidmap_put(&commit2todo, entry);
> + }
> +
> + /*
> + * Second phase:
> + * - label branch points
> + * - add HEAD to the branch tips
> + */
> + for (iter = commits; iter; iter = iter->next) {
> + struct commit_list *parent = iter->item->parents;
> + for (; parent; parent = parent->next) {
> + struct object_id *oid = &parent->item->object.oid;
> + if (!oidset_contains(&interesting, oid))
> + continue;
> + if (!oidset_contains(&child_seen, oid))
> + oidset_insert(&child_seen, oid);
> + else
> + label_oid(oid, "branch-point", &state);
> + }
> +
> + /* Add HEAD as implict "tip of branch" */
> + if (!iter->next)
> + tips_tail = &commit_list_insert(iter->item,
> + tips_tail)->next;
> + }
> +
> + /*
> + * Third phase: output the todo list. This is a bit tricky, as we
> + * want to avoid jumping back and forth between revisions. To
> + * accomplish that goal, we walk backwards from the branch tips,
> + * gathering commits not yet shown, reversing the list on the fly,
> + * then outputting that list (labeling revisions as needed).
> + */
> + fprintf(out, "%s onto\n", cmd_label);
> + for (iter = tips; iter; iter = iter->next) {
> + struct commit_list *list = NULL, *iter2;
> +
> + commit = iter->item;
> + if (oidset_contains(&shown, &commit->object.oid))
> + continue;
> + entry = oidmap_get(&state.commit2label, &commit->object.oid);
> +
> + if (entry)
> + fprintf(out, "\n# Branch %s\n", entry->string);
> + else
> + fprintf(out, "\n");
> +
> + while (oidset_contains(&interesting, &commit->object.oid) &&
> +        !oidset_contains(&shown, &commit->object.oid)) {
> + commit_list_insert(commit, &list);
> + if (!commit->parents) {
> + commit = NULL;
> + break;
> + }
> + commit = commit->parents->item;
> + }
> +
> + if (!commit)
> + fprintf(out, "%s onto\n", cmd_reset);
> + else {
> + const char *to = NULL;
> +
> + entry = oidmap_get(&state.commit2label,
> +    &commit->object.oid);
> + if (entry)
> + to = entry->string;
> +
> + if (!to || !strcmp(to, "onto"))
> + fprintf(out, "%s onto\n", cmd_reset);
> + else {
> + strbuf_reset(&oneline);
> + pretty_print_commit(pp, commit, &oneline);
> + fprintf(out, "%s %s # %s\n",
> + cmd_reset, to, oneline.buf);
> + }
> + }
> +
> + for (iter2 = list; iter2; iter2 = iter2->next) {
> + struct object_id *oid = &iter2->item->object.oid;
> + entry = oidmap_get(&commit2todo, oid);
> + /* only show if not already upstream */
> + if (entry)
> + fprintf(out, "%s\n", entry->string);
> + entry = oidmap_get(&state.commit2label, oid);
> + if (entry)
> + fprintf(out, "%s %s\n",
> + cmd_label, entry->string);
> + oidset_insert(&shown, oid);
> + }
> +
> + free_commit_list(list);
> + }
> +
> + free_commit_list(commits);
> + free_commit_list(tips);
> +
> + strbuf_release(&label);
> + strbuf_release(&oneline);
> + strbuf_release(&buf);
> +
> + oidmap_free(&commit2todo, 1);
> + oidmap_free(&state.commit2label, 1);
> + hashmap_free(&state.labels, 1);
> + strbuf_release(&state.buf);
> +
> + return 0;
> +}
> +
> int sequencer_make_script(FILE *out, int argc, const char **argv,
>    unsigned flags)
> {
> @@ -3446,11 +3785,16 @@ int sequencer_make_script(FILE *out, int argc,
> const char **argv,
>  struct commit *commit;
>  int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
>  const char *insn = flags & TODO_LIST_ABBREVIATE_CMDS ? "p" : "pick";
> + int rebase_merges = flags & TODO_LIST_REBASE_MERGES;
>
>  init_revisions(&revs, NULL);
>  revs.verbose_header = 1;
> - revs.max_parents = 1;
> - revs.cherry_pick = 1;
> + if (rebase_merges)
> + revs.cherry_mark = 1;
> + else {
> + revs.max_parents = 1;
> + revs.cherry_pick = 1;
> + }
>  revs.limited = 1;
>  revs.reverse = 1;
>  revs.right_only = 1;
> @@ -3474,6 +3818,9 @@ int sequencer_make_script(FILE *out, int argc, const
> char **argv,
>  if (prepare_revision_walk(&revs) < 0)
>  return error(_("make_script: error preparing revisions"));
>
> + if (rebase_merges)
> + return make_script_with_merges(&pp, &revs, out, flags);
> +
>  while ((commit = get_revision(&revs))) {
>  strbuf_reset(&buf);
>  if (!keep_empty && is_original_commit_empty(commit))
> diff --git a/sequencer.h b/sequencer.h
> index e45b178dfc4..6bc4da17243 100644
> --- a/sequencer.h
> +++ b/sequencer.h
> @@ -59,6 +59,7 @@ int sequencer_remove_state(struct replay_opts *opts);
> #define TODO_LIST_KEEP_EMPTY (1U << 0)
> #define TODO_LIST_SHORTEN_IDS (1U << 1)
> #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
> +#define TODO_LIST_REBASE_MERGES (1U << 3)
> int sequencer_make_script(FILE *out, int argc, const char **argv,
>    unsigned flags);
>
> -- 
> 2.17.0.windows.1.15.gaa56ade3205
>
>
>


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-21 10:33             ` [PATCH v8 06/16] sequencer: introduce the `merge` command Johannes Schindelin
  2018-04-21 15:56               ` Phillip Wood
  2018-04-22 12:01               ` Philip Oakley
@ 2018-04-22 13:55               ` Philip Oakley
  2 siblings, 0 replies; 412+ messages in thread
From: Philip Oakley @ 2018-04-22 13:55 UTC (permalink / raw)
  To: Johannes Schindelin, git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Eric Sunshine,
	Phillip Wood, Igor Djordjevic, Johannes Sixt, Sergey Organov,
	Martin Ågren

From: "Johannes Schindelin" <johannes.schindelin@gmx.de>
> This patch is part of the effort to reimplement `--preserve-merges` with
> a substantially improved design, a design that has been developed in the
> Git for Windows project to maintain the dozens of Windows-specific patch
> series on top of upstream Git.
>
> The previous patch implemented the `label` and `reset` commands to label

The previous patch was [Patch 05/16] git-rebase--interactive: clarify 
arguments, so this statement doesn't appear to be true. Has a patch been 
missed or re-ordered? Or should it be simply "This patch implements" ? 
Likewise the patch subject would be updated.

> commits and to reset to labeled commits. This patch adds the `merge`

s/adds/also adds/ ?

> command, with the following syntax:
>
> merge [-C <commit>] <rev> # <oneline>
>
> The <commit> parameter in this instance is the *original* merge commit,
> whose author and message will be used for the merge commit that is about
> to be created.
>
> The <rev> parameter refers to the (possibly rewritten) revision to
> merge. Let's see an example of a todo list:
>
The example ought to also note that `label onto` is to
`# label current HEAD with a name`, seeing as this is the first occurance.
It may be obvious in retrospect, but not at first reading.

> label onto
>
> # Branch abc
> reset onto

Is this reset strictly necessary. We are already there @head.

> pick deadbeef Hello, world!
> label abc
>
> reset onto
> pick cafecafe And now for something completely different
> merge -C baaabaaa abc # Merge the branch 'abc' into master
>
> To edit the merge commit's message (a "reword" for merges, if you will),
> use `-c` (lower-case) instead of `-C`; this convention was borrowed from
> `git commit` that also supports `-c` and `-C` with similar meanings.
>
> To create *new* merges, i.e. without copying the commit message from an
> existing commit, simply omit the `-C <commit>` parameter (which will
> open an editor for the merge message):
>
> merge abc
>
> This comes in handy when splitting a branch into two or more branches.
>
> Note: this patch only adds support for recursive merges, to keep things
> simple. Support for octopus merges will be added later in a separate
> patch series, support for merges using strategies other than the
> recursive merge is left for the future.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> git-rebase--interactive.sh |   6 +
> sequencer.c                | 407 ++++++++++++++++++++++++++++++++++++-
> 2 files changed, 406 insertions(+), 7 deletions(-)
>
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index e1b865f43f2..ccd5254d1c9 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -162,6 +162,12 @@ s, squash <commit> = use commit, but meld into 
> previous commit
> f, fixup <commit> = like \"squash\", but discard this commit's log message
> x, exec <commit> = run command (the rest of the line) using shell
> d, drop <commit> = remove commit
> +l, label <label> = label current HEAD with a name
> +t, reset <label> = reset HEAD to a label
> +m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
> +.       create a merge commit using the original merge commit's
> +.       message (or the oneline, if no original merge commit was
> +.       specified). Use -c <commit> to reword the commit message.
>
> These lines can be re-ordered; they are executed from top to bottom.
> " | git stripspace --comment-lines >>"$todo"
> diff --git a/sequencer.c b/sequencer.c
> index 01443e0f245..35fcacbdf0f 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -23,6 +23,8 @@
> #include "hashmap.h"
> #include "notes-utils.h"
> #include "sigchain.h"
> +#include "unpack-trees.h"
> +#include "worktree.h"
>
> #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
>
> @@ -120,6 +122,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, 
> "rebase-merge/stopped-sha")
> static GIT_PATH_FUNC(rebase_path_rewritten_list, 
> "rebase-merge/rewritten-list")
> static GIT_PATH_FUNC(rebase_path_rewritten_pending,
>  "rebase-merge/rewritten-pending")
> +
> +/*
> + * The path of the file listing refs that need to be deleted after the 
> rebase
> + * finishes. This is used by the `label` command to record the need for 
> cleanup.
> + */
> +static GIT_PATH_FUNC(rebase_path_refs_to_delete, 
> "rebase-merge/refs-to-delete")
> +
> /*
>  * The following files are written by git-rebase just after parsing the
>  * command-line (and are only consumed, not modified, by the sequencer).
> @@ -244,18 +253,34 @@ static const char *gpg_sign_opt_quoted(struct 
> replay_opts *opts)
>
> int sequencer_remove_state(struct replay_opts *opts)
> {
> - struct strbuf dir = STRBUF_INIT;
> + struct strbuf buf = STRBUF_INIT;
>  int i;
>
> + if (is_rebase_i(opts) &&
> +     strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
> + char *p = buf.buf;
> + while (*p) {
> + char *eol = strchr(p, '\n');
> + if (eol)
> + *eol = '\0';
> + if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
> + warning(_("could not delete '%s'"), p);
> + if (!eol)
> + break;
> + p = eol + 1;
> + }
> + }
> +
>  free(opts->gpg_sign);
>  free(opts->strategy);
>  for (i = 0; i < opts->xopts_nr; i++)
>  free(opts->xopts[i]);
>  free(opts->xopts);
>
> - strbuf_addstr(&dir, get_dir(opts));
> - remove_dir_recursively(&dir, 0);
> - strbuf_release(&dir);
> + strbuf_reset(&buf);
> + strbuf_addstr(&buf, get_dir(opts));
> + remove_dir_recursively(&buf, 0);
> + strbuf_release(&buf);
>
>  return 0;
> }
> @@ -1279,6 +1304,9 @@ enum todo_command {
>  TODO_SQUASH,
>  /* commands that do something else than handling a single commit */
>  TODO_EXEC,
> + TODO_LABEL,
> + TODO_RESET,
> + TODO_MERGE,
>  /* commands that do nothing but are counted for reporting progress */
>  TODO_NOOP,
>  TODO_DROP,
> @@ -1297,6 +1325,9 @@ static struct {
>  { 'f', "fixup" },
>  { 's', "squash" },
>  { 'x', "exec" },
> + { 'l', "label" },
> + { 't', "reset" },
> + { 'm', "merge" },
>  { 0,   "noop" },
>  { 'd', "drop" },
>  { 0,   NULL }
> @@ -1724,9 +1755,14 @@ static int read_and_refresh_cache(struct 
> replay_opts *opts)
>  return 0;
> }
>
> +enum todo_item_flags {
> + TODO_EDIT_MERGE_MSG = 1
> +};
> +
> struct todo_item {
>  enum todo_command command;
>  struct commit *commit;
> + unsigned int flags;
>  const char *arg;
>  int arg_len;
>  size_t offset_in_buf;
> @@ -1761,6 +1797,8 @@ static int parse_insn_line(struct todo_item *item, 
> const char *bol, char *eol)
>  char *end_of_object_name;
>  int i, saved, status, padding;
>
> + item->flags = 0;
> +
>  /* left-trim */
>  bol += strspn(bol, " \t");
>
> @@ -1802,13 +1840,29 @@ static int parse_insn_line(struct todo_item *item, 
> const char *bol, char *eol)
>  return error(_("missing arguments for %s"),
>       command_to_string(item->command));
>
> - if (item->command == TODO_EXEC) {
> + if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
> +     item->command == TODO_RESET) {
>  item->commit = NULL;
>  item->arg = bol;
>  item->arg_len = (int)(eol - bol);
>  return 0;
>  }
>
> + if (item->command == TODO_MERGE) {
> + if (skip_prefix(bol, "-C", &bol))
> + bol += strspn(bol, " \t");
> + else if (skip_prefix(bol, "-c", &bol)) {
> + bol += strspn(bol, " \t");
> + item->flags |= TODO_EDIT_MERGE_MSG;
> + } else {
> + item->flags |= TODO_EDIT_MERGE_MSG;
> + item->commit = NULL;
> + item->arg = bol;
> + item->arg_len = (int)(eol - bol);
> + return 0;
> + }
> + }
> +
>  end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
>  saved = *end_of_object_name;
>  *end_of_object_name = '\0';
> @@ -2465,6 +2519,305 @@ static int do_exec(const char *command_line)
>  return status;
> }
>
> +static int safe_append(const char *filename, const char *fmt, ...)
> +{
> + va_list ap;
> + struct lock_file lock = LOCK_INIT;
> + int fd = hold_lock_file_for_update(&lock, filename,
> +    LOCK_REPORT_ON_ERROR);
> + struct strbuf buf = STRBUF_INIT;
> +
> + if (fd < 0)
> + return -1;
> +
> + if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT) {
> + error_errno(_("could not read '%s'"), filename);
> + rollback_lock_file(&lock);
> + return -1;
> + }
> + strbuf_complete(&buf, '\n');
> + va_start(ap, fmt);
> + strbuf_vaddf(&buf, fmt, ap);
> + va_end(ap);
> +
> + if (write_in_full(fd, buf.buf, buf.len) < 0) {
> + error_errno(_("could not write to '%s'"), filename);
> + strbuf_release(&buf);
> + rollback_lock_file(&lock);
> + return -1;
> + }
> + if (commit_lock_file(&lock) < 0) {
> + strbuf_release(&buf);
> + rollback_lock_file(&lock);
> + return error(_("failed to finalize '%s'"), filename);
> + }
> +
> + strbuf_release(&buf);
> + return 0;
> +}
> +
> +static int do_label(const char *name, int len)
> +{
> + struct ref_store *refs = get_main_ref_store();
> + struct ref_transaction *transaction;
> + struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
> + struct strbuf msg = STRBUF_INIT;
> + int ret = 0;
> + struct object_id head_oid;
> +
> + if (len == 1 && *name == '#')
> + return error("Illegal label name: '%.*s'", len, name);
> +
> + strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> + strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
> +
> + transaction = ref_store_transaction_begin(refs, &err);
> + if (!transaction) {
> + error("%s", err.buf);
> + ret = -1;
> + } else if (get_oid("HEAD", &head_oid)) {
> + error(_("could not read HEAD"));
> + ret = -1;
> + } else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
> +   NULL, 0, msg.buf, &err) < 0 ||
> +    ref_transaction_commit(transaction, &err)) {
> + error("%s", err.buf);
> + ret = -1;
> + }
> + ref_transaction_free(transaction);
> + strbuf_release(&err);
> + strbuf_release(&msg);
> +
> + if (!ret)
> + ret = safe_append(rebase_path_refs_to_delete(),
> +   "%s\n", ref_name.buf);
> + strbuf_release(&ref_name);
> +
> + return ret;
> +}
> +
> +static const char *reflog_message(struct replay_opts *opts,
> + const char *sub_action, const char *fmt, ...);
> +
> +static int do_reset(const char *name, int len, struct replay_opts *opts)
> +{
> + struct strbuf ref_name = STRBUF_INIT;
> + struct object_id oid;
> + struct lock_file lock = LOCK_INIT;
> + struct tree_desc desc;
> + struct tree *tree;
> + struct unpack_trees_options unpack_tree_opts;
> + int ret = 0, i;
> +
> + if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
> + return -1;
> +
> + /* Determine the length of the label */
> + for (i = 0; i < len; i++)
> + if (isspace(name[i]))
> + len = i;
> +
> + strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
> + if (get_oid(ref_name.buf, &oid) &&
> +     get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
> + error(_("could not read '%s'"), ref_name.buf);
> + rollback_lock_file(&lock);
> + strbuf_release(&ref_name);
> + return -1;
> + }
> +
> + memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
> + unpack_tree_opts.head_idx = 1;
> + unpack_tree_opts.src_index = &the_index;
> + unpack_tree_opts.dst_index = &the_index;
> + unpack_tree_opts.fn = oneway_merge;
> + unpack_tree_opts.merge = 1;
> + unpack_tree_opts.update = 1;
> +
> + if (read_cache_unmerged()) {
> + rollback_lock_file(&lock);
> + strbuf_release(&ref_name);
> + return error_resolve_conflict(_(action_name(opts)));
> + }
> +
> + if (!fill_tree_descriptor(&desc, &oid)) {
> + error(_("failed to find tree of %s"), oid_to_hex(&oid));
> + rollback_lock_file(&lock);
> + free((void *)desc.buffer);
> + strbuf_release(&ref_name);
> + return -1;
> + }
> +
> + if (unpack_trees(1, &desc, &unpack_tree_opts)) {
> + rollback_lock_file(&lock);
> + free((void *)desc.buffer);
> + strbuf_release(&ref_name);
> + return -1;
> + }
> +
> + tree = parse_tree_indirect(&oid);
> + prime_cache_tree(&the_index, tree);
> +
> + if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
> + ret = error(_("could not write index"));
> + free((void *)desc.buffer);
> +
> + if (!ret)
> + ret = update_ref(reflog_message(opts, "reset", "'%.*s'",
> + len, name), "HEAD", &oid,
> + NULL, 0, UPDATE_REFS_MSG_ON_ERR);
> +
> + strbuf_release(&ref_name);
> + return ret;
> +}
> +
> +static int do_merge(struct commit *commit, const char *arg, int arg_len,
> +     int flags, struct replay_opts *opts)
> +{
> + int run_commit_flags = (flags & TODO_EDIT_MERGE_MSG) ?
> + EDIT_MSG | VERIFY_MSG : 0;
> + struct strbuf ref_name = STRBUF_INIT;
> + struct commit *head_commit, *merge_commit, *i;
> + struct commit_list *bases, *j, *reversed = NULL;
> + struct merge_options o;
> + int merge_arg_len, oneline_offset, ret;
> + static struct lock_file lock;
> + const char *p;
> +
> + if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0) {
> + ret = -1;
> + goto leave_merge;
> + }
> +
> + head_commit = lookup_commit_reference_by_name("HEAD");
> + if (!head_commit) {
> + ret = error(_("cannot merge without a current revision"));
> + goto leave_merge;
> + }
> +
> + oneline_offset = arg_len;
> + merge_arg_len = strcspn(arg, " \t\n");
> + p = arg + merge_arg_len;
> + p += strspn(p, " \t\n");
> + if (*p == '#' && (!p[1] || isspace(p[1]))) {
> + p += 1 + strspn(p + 1, " \t\n");
> + oneline_offset = p - arg;
> + } else if (p - arg < arg_len)
> + BUG("octopus merges are not supported yet: '%s'", p);
> +
> + strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
> + merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> + if (!merge_commit) {
> + /* fall back to non-rewritten ref or commit */
> + strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
> + merge_commit = lookup_commit_reference_by_name(ref_name.buf);
> + }
> +
> + if (!merge_commit) {
> + ret = error(_("could not resolve '%s'"), ref_name.buf);
> + goto leave_merge;
> + }
> +
> + if (commit) {
> + const char *message = get_commit_buffer(commit, NULL);
> + const char *body;
> + int len;
> +
> + if (!message) {
> + ret = error(_("could not get commit message of '%s'"),
> +     oid_to_hex(&commit->object.oid));
> + goto leave_merge;
> + }
> + write_author_script(message);
> + find_commit_subject(message, &body);
> + len = strlen(body);
> + ret = write_message(body, len, git_path_merge_msg(), 0);
> + unuse_commit_buffer(commit, message);
> + if (ret) {
> + error_errno(_("could not write '%s'"),
> +     git_path_merge_msg());
> + goto leave_merge;
> + }
> + } else {
> + struct strbuf buf = STRBUF_INIT;
> + int len;
> +
> + strbuf_addf(&buf, "author %s", git_author_info(0));
> + write_author_script(buf.buf);
> + strbuf_reset(&buf);
> +
> + if (oneline_offset < arg_len) {
> + p = arg + oneline_offset;
> + len = arg_len - oneline_offset;
> + } else {
> + strbuf_addf(&buf, "Merge branch '%.*s'",
> +     merge_arg_len, arg);
> + p = buf.buf;
> + len = buf.len;
> + }
> +
> + ret = write_message(p, len, git_path_merge_msg(), 0);
> + strbuf_release(&buf);
> + if (ret) {
> + error_errno(_("could not write '%s'"),
> +     git_path_merge_msg());
> + goto leave_merge;
> + }
> + }
> +
> + write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
> +       git_path_merge_head(), 0);
> + write_message("no-ff", 5, git_path_merge_mode(), 0);
> +
> + bases = get_merge_bases(head_commit, merge_commit);
> + for (j = bases; j; j = j->next)
> + commit_list_insert(j->item, &reversed);
> + free_commit_list(bases);
> +
> + read_cache();
> + init_merge_options(&o);
> + o.branch1 = "HEAD";
> + o.branch2 = ref_name.buf;
> + o.buffer_output = 2;
> +
> + ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
> + if (ret <= 0)
> + fputs(o.obuf.buf, stdout);
> + strbuf_release(&o.obuf);
> + if (ret < 0) {
> + error(_("could not even attempt to merge '%.*s'"),
> +       merge_arg_len, arg);
> + goto leave_merge;
> + }
> + /*
> + * The return value of merge_recursive() is 1 on clean, and 0 on
> + * unclean merge.
> + *
> + * Let's reverse that, so that do_merge() returns 0 upon success and
> + * 1 upon failed merge (keeping the return value -1 for the cases where
> + * we will want to reschedule the `merge` command).
> + */
> + ret = !ret;
> +
> + if (active_cache_changed &&
> +     write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
> + ret = error(_("merge: Unable to write new index file"));
> + goto leave_merge;
> + }
> +
> + rollback_lock_file(&lock);
> + if (ret)
> + rerere(opts->allow_rerere_auto);
> + else
> + ret = run_git_commit(git_path_merge_msg(), opts,
> +      run_commit_flags);
> +
> +leave_merge:
> + strbuf_release(&ref_name);
> + rollback_lock_file(&lock);
> + return ret;
> +}
> +
> static int is_final_fixup(struct todo_list *todo_list)
> {
>  int i = todo_list->current;
> @@ -2568,7 +2921,7 @@ N_("Could not execute the todo command\n"
>
> static int pick_commits(struct todo_list *todo_list, struct replay_opts 
> *opts)
> {
> - int res = 0;
> + int res = 0, reschedule = 0;
>
>  setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
>  if (opts->allow_ff)
> @@ -2639,7 +2992,7 @@ static int pick_commits(struct todo_list *todo_list, 
> struct replay_opts *opts)
>  intend_to_amend();
>  return error_failed_squash(item->commit, opts,
>  item->arg_len, item->arg);
> - } else if (res && is_rebase_i(opts))
> + } else if (res && is_rebase_i(opts) && item->commit)
>  return res | error_with_patch(item->commit,
>  item->arg, item->arg_len, opts, res,
>  item->command == TODO_REWORD);
> @@ -2665,9 +3018,41 @@ static int pick_commits(struct todo_list 
> *todo_list, struct replay_opts *opts)
>  /* `current` will be incremented below */
>  todo_list->current = -1;
>  }
> + } else if (item->command == TODO_LABEL) {
> + if ((res = do_label(item->arg, item->arg_len)))
> + reschedule = 1;
> + } else if (item->command == TODO_RESET) {
> + if ((res = do_reset(item->arg, item->arg_len, opts)))
> + reschedule = 1;
> + } else if (item->command == TODO_MERGE) {
> + if ((res = do_merge(item->commit,
> +     item->arg, item->arg_len,
> +     item->flags, opts)) < 0)
> + reschedule = 1;
> + else if (res > 0)
> + /* failed with merge conflicts */
> + return error_with_patch(item->commit,
> + item->arg,
> + item->arg_len, opts,
> + res, 0);
>  } else if (!is_noop(item->command))
>  return error(_("unknown command %d"), item->command);
>
> + if (reschedule) {
> + advise(_(rescheduled_advice),
> +        get_item_line_length(todo_list,
> +     todo_list->current),
> +        get_item_line(todo_list, todo_list->current));
> + todo_list->current--;
> + if (save_todo(todo_list, opts))
> + return -1;
> + if (item->commit)
> + return error_with_patch(item->commit,
> + item->arg,
> + item->arg_len, opts,
> + res, 0);
> + }
> +
>  todo_list->current++;
>  if (res)
>  return res;
> @@ -3147,8 +3532,16 @@ int transform_todos(unsigned flags)
>    short_commit_name(item->commit) :
>    oid_to_hex(&item->commit->object.oid);
>
> + if (item->command == TODO_MERGE) {
> + if (item->flags & TODO_EDIT_MERGE_MSG)
> + strbuf_addstr(&buf, " -c");
> + else
> + strbuf_addstr(&buf, " -C");
> + }
> +
>  strbuf_addf(&buf, " %s", oid);
>  }
> +
>  /* add all the rest */
>  if (!item->arg_len)
>  strbuf_addch(&buf, '\n');
> -- 
> 2.17.0.windows.1.15.gaa56ade3205
>
>
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 09/16] rebase: introduce the --rebase-merges option
  2018-04-21 10:34             ` [PATCH v8 09/16] rebase: introduce the --rebase-merges option Johannes Schindelin
@ 2018-04-22 14:15               ` Philip Oakley
  2018-04-24  5:01                 ` Junio C Hamano
  2018-04-24  8:40                 ` Johannes Schindelin
  2018-04-22 14:37               ` Philip Oakley
  1 sibling, 2 replies; 412+ messages in thread
From: Philip Oakley @ 2018-04-22 14:15 UTC (permalink / raw)
  To: Johannes Schindelin, Git List
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Eric Sunshine,
	Phillip Wood, Igor Djordjevic, Johannes Sixt, Sergey Organov,
	Martin Ågren

From: "Johannes Schindelin" <johannes.schindelin@gmx.de>
> Once upon a time, this here developer thought: wouldn't it be nice if,
> say, Git for Windows' patches on top of core Git could be represented as
> a thicket of branches, and be rebased on top of core Git in order to
> maintain a cherry-pick'able set of patch series?
>
> The original attempt to answer this was: git rebase --preserve-merges.
>
> However, that experiment was never intended as an interactive option,
> and it only piggy-backed on git rebase --interactive because that
> command's implementation looked already very, very familiar: it was
> designed by the same person who designed --preserve-merges: yours truly.
>
> Some time later, some other developer (I am looking at you, Andreas!
> ;-)) decided that it would be a good idea to allow --preserve-merges to
> be combined with --interactive (with caveats!) and the Git maintainer
> (well, the interim Git maintainer during Junio's absence, that is)
> agreed, and that is when the glamor of the --preserve-merges design
> started to fall apart rather quickly and unglamorously.
>
> The reason? In --preserve-merges mode, the parents of a merge commit (or
> for that matter, of *any* commit) were not stated explicitly, but were
> *implied* by the commit name passed to the `pick` command.
>
> This made it impossible, for example, to reorder commits. Not to mention
> to flatten the branch topology or, deity forbid, to split topic branches

Aside: The idea of a "flattened" topology is, to my mind, not actually
defined though may be understood by devs working in the area. Hopefully it's
going away as a term, though the new 'cousins' will need clarification
(there's no dot notation for that area of topology).

> into two.
>
> Alas, these shortcomings also prevented that mode (whose original
> purpose was to serve Git for Windows' needs, with the additional hope
> that it may be useful to others, too) from serving Git for Windows'
> needs.
>
> Five years later, when it became really untenable to have one unwieldy,
> big hodge-podge patch series of partly related, partly unrelated patches
> in Git for Windows that was rebased onto core Git's tags from time to
> time (earning the undeserved wrath of the developer of the ill-fated
> git-remote-hg series that first obsoleted Git for Windows' competing
> approach, only to be abandoned without maintainer later) was really
> untenable, the "Git garden shears" were born [*1*/*2*]: a script,
> piggy-backing on top of the interactive rebase, that would first
> determine the branch topology of the patches to be rebased, create a
> pseudo todo list for further editing, transform the result into a real
> todo list (making heavy use of the `exec` command to "implement" the
> missing todo list commands) and finally recreate the patch series on
> top of the new base commit.
>
> That was in 2013. And it took about three weeks to come up with the
> design and implement it as an out-of-tree script. Needless to say, the
> implementation needed quite a few years to stabilize, all the while the
> design itself proved itself sound.
>
> With this patch, the goodness of the Git garden shears comes to `git
> rebase -i` itself. Passing the `--rebase-merges` option will generate
> a todo list that can be understood readily, and where it is obvious
> how to reorder commits. New branches can be introduced by inserting
> `label` commands and calling `merge <label>`. And once this mode will
> have become stable and universally accepted, we can deprecate the design
> mistake that was `--preserve-merges`.
>
> Link *1*:
> https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
> Link *2*:
> https://github.com/git-for-windows/build-extra/blob/master/shears.sh
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> Documentation/git-rebase.txt           |  20 ++-
> contrib/completion/git-completion.bash |   2 +-
> git-rebase--interactive.sh             |   1 +
> git-rebase.sh                          |   6 +
> t/t3430-rebase-merges.sh               | 179 +++++++++++++++++++++++++
> 5 files changed, 206 insertions(+), 2 deletions(-)
> create mode 100755 t/t3430-rebase-merges.sh
>
> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> index 3277ca14327..34e0f6a69c1 100644
> --- a/Documentation/git-rebase.txt
> +++ b/Documentation/git-rebase.txt
> @@ -378,6 +378,23 @@ The commit list format can be changed by setting the
> configuration option
> rebase.instructionFormat.  A customized instruction format will
> automatically
> have the long commit hash prepended to the format.
>
> +-r::
> +--rebase-merges::
> + By default, a rebase will simply drop merge commits and only rebase
> + the non-merge commits. With this option, it will try to preserve
> + the branching structure within the commits that are to be rebased,
> + by recreating the merge commits. If a merge commit resolved any merge
> + or contained manual amendments, then they will have to be re-applied
> + manually.
> ++
> +This mode is similar in spirit to `--preserve-merges`, but in contrast to
> +that option works well in interactive rebases: commits can be reordered,
> +inserted and dropped at will.
> ++
> +It is currently only possible to recreate the merge commits using the
> +`recursive` merge strategy; Different merge strategies can be used only
> via
> +explicit `exec git merge -s <strategy> [...]` commands.
> +
> -p::
> --preserve-merges::
>  Recreate merge commits instead of flattening the history by replaying

Flatten is here in the context lines but its just a blunt statement that 'it
is what it is'...

> @@ -780,7 +797,8 @@ BUGS
> The todo list presented by `--preserve-merges --interactive` does not
> represent the topology of the revision graph.  Editing commits and
> rewording their commit messages should work fine, but attempts to
> -reorder commits tend to produce counterintuitive results.
> +reorder commits tend to produce counterintuitive results. Use
> +`--rebase-merges` in such scenarios instead.
>
> For example, an attempt to rearrange
> ------------
> diff --git a/contrib/completion/git-completion.bash
> b/contrib/completion/git-completion.bash
> index a7570739454..d4c0a995c39 100644
> --- a/contrib/completion/git-completion.bash
> +++ b/contrib/completion/git-completion.bash
> @@ -1949,7 +1949,7 @@ _git_rebase ()
>  --*)
>  __gitcomp "
>  --onto --merge --strategy --interactive
> - --preserve-merges --stat --no-stat
> + --rebase-merges --preserve-merges --stat --no-stat
>  --committer-date-is-author-date --ignore-date
>  --ignore-whitespace --whitespace=
>  --autosquash --no-autosquash
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index ccd5254d1c9..7a3daf3e40c 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -970,6 +970,7 @@ git_rebase__interactive () {
>  init_revisions_and_shortrevisions
>
>  git rebase--helper --make-script ${keep_empty:+--keep-empty} \
> + ${rebase_merges:+--rebase-merges} \
>  $revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
>  die "$(gettext "Could not generate todo list")"
>
> diff --git a/git-rebase.sh b/git-rebase.sh
> index fb64ee1fe42..a64460fd25a 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -17,6 +17,7 @@ q,quiet!           be quiet. implies --no-stat
> autostash          automatically stash/stash pop before and after
> fork-point         use 'merge-base --fork-point' to refine upstream
> onto=!             rebase onto given branch instead of upstream
> +r,rebase-merges!   try to rebase merges instead of skipping them
> p,preserve-merges! try to recreate merges instead of ignoring them
> s,strategy=!       use the given merge strategy
> no-ff!             cherry-pick all commits, even if unchanged
> @@ -88,6 +89,7 @@ type=
> state_dir=
> # One of {'', continue, skip, abort}, as parsed from command line
> action=
> +rebase_merges=
> preserve_merges=
> autosquash=
> keep_empty=
> @@ -270,6 +272,10 @@ do
>  --allow-empty-message)
>  allow_empty_message=--allow-empty-message
>  ;;
> + --rebase-merges)
> + rebase_merges=t
> + test -z "$interactive_rebase" && interactive_rebase=implied
> + ;;
>  --preserve-merges)
>  preserve_merges=t
>  test -z "$interactive_rebase" && interactive_rebase=implied
> diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
> new file mode 100755
> index 00000000000..5f0febb9970
> --- /dev/null
> +++ b/t/t3430-rebase-merges.sh
> @@ -0,0 +1,179 @@
> +#!/bin/sh
> +#
> +# Copyright (c) 2018 Johannes E. Schindelin
> +#
> +
> +test_description='git rebase -i --rebase-merges
> +
> +This test runs git rebase "interactively", retaining the branch structure
> by
> +recreating merge commits.
> +
> +Initial setup:
> +
> +    -- B --                   (first)
> +   /       \
> + A - C - D - E - H            (master)
> +       \       /
> +         F - G                (second)
> +'
> +. ./test-lib.sh
> +. "$TEST_DIRECTORY"/lib-rebase.sh
> +
> +test_cmp_graph () {
> + cat >expect &&
> + git log --graph --boundary --format=%s "$@" >output &&
> + sed "s/ *$//" <output >output.trimmed &&
> + test_cmp expect output.trimmed
> +}
> +
> +test_expect_success 'setup' '
> + write_script replace-editor.sh <<-\EOF &&
> + mv "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
> + cp script-from-scratch "$1"
> + EOF
> +
> + test_commit A &&
> + git checkout -b first &&
> + test_commit B &&
> + git checkout master &&
> + test_commit C &&
> + test_commit D &&
> + git merge --no-commit B &&
> + test_tick &&
> + git commit -m E &&
> + git tag -m E E &&
> + git checkout -b second C &&
> + test_commit F &&
> + test_commit G &&
> + git checkout master &&
> + git merge --no-commit G &&
> + test_tick &&
> + git commit -m H &&
> + git tag -m H H
> +'
> +
> +test_expect_success 'create completely different structure' '
> + cat >script-from-scratch <<-\EOF &&
> + label onto
> +
> + # onebranch
> + pick G
> + pick D
> + label onebranch
> +
> + # second
> + reset onto
> + pick B
> + label second
> +
> + reset onto
> + merge -C H second
> + merge onebranch # Merge the topic branch '\''onebranch'\''
> + EOF
> + test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
> + test_tick &&
> + git rebase -i -r A &&
> + test_cmp_graph <<-\EOF
> + *   Merge the topic branch '\''onebranch'\''
> + |\
> + | * D
> + | * G
> + * |   H
> + |\ \
> + | |/
> + |/|
> + | * B
> + |/
> + * A
> + EOF
> +'
> +
> +test_expect_success 'generate correct todo list' '
> + cat >expect <<-\EOF &&
> + label onto
> +
> + reset onto
> + pick d9df450 B
> + label E
> +
> + reset onto
> + pick 5dee784 C
> + label branch-point
> + pick ca2c861 F
> + pick 088b00a G
> + label H
> +
> + reset branch-point # C
> + pick 12bd07b D
> + merge -C 2051b56 E # E
> + merge -C 233d48a H # H
> +
> + EOF
> +
> + grep -v "^#" <.git/ORIGINAL-TODO >output &&
> + test_cmp expect output
> +'
> +
> +test_expect_success '`reset` refuses to overwrite untracked files' '
> + git checkout -b refuse-to-reset &&
> + test_commit dont-overwrite-untracked &&
> + git checkout @{-1} &&
> + : >dont-overwrite-untracked.t &&
> + echo "reset refs/tags/dont-overwrite-untracked" >script-from-scratch &&
> + test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
> + test_must_fail git rebase -r HEAD &&
> + git rebase --abort
> +'
> +
> +test_expect_success 'failed `merge` writes patch (may be rescheduled,
> too)' '
> + test_when_finished "test_might_fail git rebase --abort" &&
> + git checkout -b conflicting-merge A &&
> +
> + : fail because of conflicting untracked file &&
> + >G.t &&
> + echo "merge -C H G" >script-from-scratch &&
> + test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
> + test_tick &&
> + test_must_fail git rebase -ir HEAD &&
> + grep "^merge -C .* G$" .git/rebase-merge/done &&
> + grep "^merge -C .* G$" .git/rebase-merge/git-rebase-todo &&
> + test_path_is_file .git/rebase-merge/patch &&
> +
> + : fail because of merge conflict &&
> + rm G.t .git/rebase-merge/patch &&
> + git reset --hard &&
> + test_commit conflicting-G G.t not-G conflicting-G &&
> + test_must_fail git rebase --continue &&
> + ! grep "^merge -C .* G$" .git/rebase-merge/git-rebase-todo &&
> + test_path_is_file .git/rebase-merge/patch
> +'
> +
> +test_expect_success 'with a branch tip that was cherry-picked already' '
> + git checkout -b already-upstream master &&
> + base="$(git rev-parse --verify HEAD)" &&
> +
> + test_commit A1 &&
> + test_commit A2 &&
> + git reset --hard $base &&
> + test_commit B1 &&
> + test_tick &&
> + git merge -m "Merge branch A" A2 &&
> +
> + git checkout -b upstream-with-a2 $base &&
> + test_tick &&
> + git cherry-pick A2 &&
> +
> + git checkout already-upstream &&
> + test_tick &&
> + git rebase -i -r upstream-with-a2 &&
> + test_cmp_graph upstream-with-a2.. <<-\EOF
> + *   Merge branch A
> + |\
> + | * A1
> + * | B1
> + |/
> + o A2
> + EOF
> +'
> +
> +test_done
> -- 
> 2.17.0.windows.1.15.gaa56ade3205
>
>
>


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 09/16] rebase: introduce the --rebase-merges option
  2018-04-21 10:34             ` [PATCH v8 09/16] rebase: introduce the --rebase-merges option Johannes Schindelin
  2018-04-22 14:15               ` Philip Oakley
@ 2018-04-22 14:37               ` Philip Oakley
  2018-04-24 10:52                 ` Johannes Schindelin
  1 sibling, 1 reply; 412+ messages in thread
From: Philip Oakley @ 2018-04-22 14:37 UTC (permalink / raw)
  To: Johannes Schindelin, git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Eric Sunshine,
	Phillip Wood, Igor Djordjevic, Johannes Sixt, Sergey Organov,
	Martin Ågren

From: "Johannes Schindelin" <johannes.schindelin@gmx.de>
> Once upon a time, this here developer thought: wouldn't it be nice if,
> say, Git for Windows' patches on top of core Git could be represented as
> a thicket of branches, and be rebased on top of core Git in order to
> maintain a cherry-pick'able set of patch series?
>
> The original attempt to answer this was: git rebase --preserve-merges.
>
> However, that experiment was never intended as an interactive option,
> and it only piggy-backed on git rebase --interactive because that
> command's implementation looked already very, very familiar: it was
> designed by the same person who designed --preserve-merges: yours truly.
>
> Some time later, some other developer (I am looking at you, Andreas!
> ;-)) decided that it would be a good idea to allow --preserve-merges to
> be combined with --interactive (with caveats!) and the Git maintainer
> (well, the interim Git maintainer during Junio's absence, that is)
> agreed, and that is when the glamor of the --preserve-merges design
> started to fall apart rather quickly and unglamorously.
>
> The reason? In --preserve-merges mode, the parents of a merge commit (or
> for that matter, of *any* commit) were not stated explicitly, but were
> *implied* by the commit name passed to the `pick` command.
>
Aside: I think this para should be extracted to the --preserve-merges 
documentation to highlight what it does / why it is 'wrong' (not what would 
be expected in some case). It may also need to discuss the (figurative) 
Cousins vs. Siblings distinction [merge of branches external, or internal, 
to the rebase.

"In --preserve-merges, the commit being selected for merging is implied by 
the commit name  passed to the `pick` command (i.e. of the original merge 
commit), not that of the rebased version of that parent."

A similar issue occurs with (figuratively) '--ancestry-path --first parent' 
searches which lacks the alternate '--lead parent' post-walk selection. [1]. 
I don't think there is a dot notation to select the merge cousins, nor merge 
siblings either A.,B ? (that's dot-comma ;-)

> This made it impossible, for example, to reorder commits. Not to mention
> to flatten the branch topology or, deity forbid, to split topic branches
> into two.
>
> Alas, these shortcomings also prevented that mode (whose original
> purpose was to serve Git for Windows' needs, with the additional hope
> that it may be useful to others, too) from serving Git for Windows'
> needs.
>
> Five years later, when it became really untenable to have one unwieldy,
> big hodge-podge patch series of partly related, partly unrelated patches
> in Git for Windows that was rebased onto core Git's tags from time to
> time (earning the undeserved wrath of the developer of the ill-fated
> git-remote-hg series that first obsoleted Git for Windows' competing
> approach, only to be abandoned without maintainer later) was really
> untenable, the "Git garden shears" were born [*1*/*2*]: a script,
> piggy-backing on top of the interactive rebase, that would first
> determine the branch topology of the patches to be rebased, create a
> pseudo todo list for further editing, transform the result into a real
> todo list (making heavy use of the `exec` command to "implement" the
> missing todo list commands) and finally recreate the patch series on
> top of the new base commit.
>
> That was in 2013. And it took about three weeks to come up with the
> design and implement it as an out-of-tree script. Needless to say, the
> implementation needed quite a few years to stabilize, all the while the
> design itself proved itself sound.
>
> With this patch, the goodness of the Git garden shears comes to `git
> rebase -i` itself. Passing the `--rebase-merges` option will generate
> a todo list that can be understood readily, and where it is obvious
> how to reorder commits. New branches can be introduced by inserting
> `label` commands and calling `merge <label>`. And once this mode will
> have become stable and universally accepted, we can deprecate the design
> mistake that was `--preserve-merges`.
>
> Link *1*:
> https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
> Link *2*:
> https://github.com/git-for-windows/build-extra/blob/master/shears.sh
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> Documentation/git-rebase.txt           |  20 ++-
> contrib/completion/git-completion.bash |   2 +-
> git-rebase--interactive.sh             |   1 +
> git-rebase.sh                          |   6 +
> t/t3430-rebase-merges.sh               | 179 +++++++++++++++++++++++++
> 5 files changed, 206 insertions(+), 2 deletions(-)
> create mode 100755 t/t3430-rebase-merges.sh
>
> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> index 3277ca14327..34e0f6a69c1 100644
> --- a/Documentation/git-rebase.txt
> +++ b/Documentation/git-rebase.txt
> @@ -378,6 +378,23 @@ The commit list format can be changed by setting the 
> configuration option
> rebase.instructionFormat.  A customized instruction format will 
> automatically
> have the long commit hash prepended to the format.
>
> +-r::
> +--rebase-merges::
> + By default, a rebase will simply drop merge commits and only rebase
> + the non-merge commits. With this option, it will try to preserve
> + the branching structure within the commits that are to be rebased,
> + by recreating the merge commits. If a merge commit resolved any merge
> + or contained manual amendments, then they will have to be re-applied
> + manually.
> ++
> +This mode is similar in spirit to `--preserve-merges`, but in contrast to
> +that option works well in interactive rebases: commits can be reordered,
> +inserted and dropped at will.
> ++
> +It is currently only possible to recreate the merge commits using the
> +`recursive` merge strategy; Different merge strategies can be used only 
> via
> +explicit `exec git merge -s <strategy> [...]` commands.
> +
> -p::
> --preserve-merges::
>  Recreate merge commits instead of flattening the history by replaying
> @@ -780,7 +797,8 @@ BUGS
> The todo list presented by `--preserve-merges --interactive` does not
> represent the topology of the revision graph.  Editing commits and
> rewording their commit messages should work fine, but attempts to
> -reorder commits tend to produce counterintuitive results.
> +reorder commits tend to produce counterintuitive results. Use
> +`--rebase-merges` in such scenarios instead.
>
> For example, an attempt to rearrange
> ------------
> diff --git a/contrib/completion/git-completion.bash 
> b/contrib/completion/git-completion.bash
> index a7570739454..d4c0a995c39 100644
> --- a/contrib/completion/git-completion.bash
> +++ b/contrib/completion/git-completion.bash
> @@ -1949,7 +1949,7 @@ _git_rebase ()
>  --*)
>  __gitcomp "
>  --onto --merge --strategy --interactive
> - --preserve-merges --stat --no-stat
> + --rebase-merges --preserve-merges --stat --no-stat
>  --committer-date-is-author-date --ignore-date
>  --ignore-whitespace --whitespace=
>  --autosquash --no-autosquash
> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
> index ccd5254d1c9..7a3daf3e40c 100644
> --- a/git-rebase--interactive.sh
> +++ b/git-rebase--interactive.sh
> @@ -970,6 +970,7 @@ git_rebase__interactive () {
>  init_revisions_and_shortrevisions
>
>  git rebase--helper --make-script ${keep_empty:+--keep-empty} \
> + ${rebase_merges:+--rebase-merges} \
>  $revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
>  die "$(gettext "Could not generate todo list")"
>
> diff --git a/git-rebase.sh b/git-rebase.sh
> index fb64ee1fe42..a64460fd25a 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -17,6 +17,7 @@ q,quiet!           be quiet. implies --no-stat
> autostash          automatically stash/stash pop before and after
> fork-point         use 'merge-base --fork-point' to refine upstream
> onto=!             rebase onto given branch instead of upstream
> +r,rebase-merges!   try to rebase merges instead of skipping them
> p,preserve-merges! try to recreate merges instead of ignoring them
> s,strategy=!       use the given merge strategy
> no-ff!             cherry-pick all commits, even if unchanged
> @@ -88,6 +89,7 @@ type=
> state_dir=
> # One of {'', continue, skip, abort}, as parsed from command line
> action=
> +rebase_merges=
> preserve_merges=
> autosquash=
> keep_empty=
> @@ -270,6 +272,10 @@ do
>  --allow-empty-message)
>  allow_empty_message=--allow-empty-message
>  ;;
> + --rebase-merges)
> + rebase_merges=t
> + test -z "$interactive_rebase" && interactive_rebase=implied
> + ;;
>  --preserve-merges)
>  preserve_merges=t
>  test -z "$interactive_rebase" && interactive_rebase=implied
> diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
> new file mode 100755
> index 00000000000..5f0febb9970
> --- /dev/null
> +++ b/t/t3430-rebase-merges.sh
> @@ -0,0 +1,179 @@
> +#!/bin/sh
> +#
> +# Copyright (c) 2018 Johannes E. Schindelin
> +#
> +
> +test_description='git rebase -i --rebase-merges
> +
> +This test runs git rebase "interactively", retaining the branch structure 
> by
> +recreating merge commits.
> +
> +Initial setup:
> +
> +    -- B --                   (first)
> +   /       \
> + A - C - D - E - H            (master)
> +       \       /
> +         F - G                (second)
> +'
> +. ./test-lib.sh
> +. "$TEST_DIRECTORY"/lib-rebase.sh
> +
> +test_cmp_graph () {
> + cat >expect &&
> + git log --graph --boundary --format=%s "$@" >output &&
> + sed "s/ *$//" <output >output.trimmed &&
> + test_cmp expect output.trimmed
> +}
> +
> +test_expect_success 'setup' '
> + write_script replace-editor.sh <<-\EOF &&
> + mv "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
> + cp script-from-scratch "$1"
> + EOF
> +
> + test_commit A &&
> + git checkout -b first &&
> + test_commit B &&
> + git checkout master &&
> + test_commit C &&
> + test_commit D &&
> + git merge --no-commit B &&
> + test_tick &&
> + git commit -m E &&
> + git tag -m E E &&
> + git checkout -b second C &&
> + test_commit F &&
> + test_commit G &&
> + git checkout master &&
> + git merge --no-commit G &&
> + test_tick &&
> + git commit -m H &&
> + git tag -m H H
> +'
> +
> +test_expect_success 'create completely different structure' '
> + cat >script-from-scratch <<-\EOF &&
> + label onto
> +
> + # onebranch
> + pick G
> + pick D
> + label onebranch
> +
> + # second
> + reset onto
> + pick B
> + label second
> +
> + reset onto
> + merge -C H second
> + merge onebranch # Merge the topic branch '\''onebranch'\''
> + EOF
> + test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
> + test_tick &&
> + git rebase -i -r A &&
> + test_cmp_graph <<-\EOF
> + *   Merge the topic branch '\''onebranch'\''
> + |\
> + | * D
> + | * G
> + * |   H
> + |\ \
> + | |/
> + |/|
> + | * B
> + |/
> + * A
> + EOF
> +'
> +
> +test_expect_success 'generate correct todo list' '
> + cat >expect <<-\EOF &&
> + label onto
> +
> + reset onto
> + pick d9df450 B
> + label E
> +
> + reset onto
> + pick 5dee784 C
> + label branch-point
> + pick ca2c861 F
> + pick 088b00a G
> + label H
> +
> + reset branch-point # C
> + pick 12bd07b D
> + merge -C 2051b56 E # E
> + merge -C 233d48a H # H
> +
> + EOF
> +
> + grep -v "^#" <.git/ORIGINAL-TODO >output &&
> + test_cmp expect output
> +'
> +
> +test_expect_success '`reset` refuses to overwrite untracked files' '
> + git checkout -b refuse-to-reset &&
> + test_commit dont-overwrite-untracked &&
> + git checkout @{-1} &&
> + : >dont-overwrite-untracked.t &&
> + echo "reset refs/tags/dont-overwrite-untracked" >script-from-scratch &&
> + test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
> + test_must_fail git rebase -r HEAD &&
> + git rebase --abort
> +'
> +
> +test_expect_success 'failed `merge` writes patch (may be rescheduled, 
> too)' '
> + test_when_finished "test_might_fail git rebase --abort" &&
> + git checkout -b conflicting-merge A &&
> +
> + : fail because of conflicting untracked file &&
> + >G.t &&
> + echo "merge -C H G" >script-from-scratch &&
> + test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
> + test_tick &&
> + test_must_fail git rebase -ir HEAD &&
> + grep "^merge -C .* G$" .git/rebase-merge/done &&
> + grep "^merge -C .* G$" .git/rebase-merge/git-rebase-todo &&
> + test_path_is_file .git/rebase-merge/patch &&
> +
> + : fail because of merge conflict &&
> + rm G.t .git/rebase-merge/patch &&
> + git reset --hard &&
> + test_commit conflicting-G G.t not-G conflicting-G &&
> + test_must_fail git rebase --continue &&
> + ! grep "^merge -C .* G$" .git/rebase-merge/git-rebase-todo &&
> + test_path_is_file .git/rebase-merge/patch
> +'
> +
> +test_expect_success 'with a branch tip that was cherry-picked already' '
> + git checkout -b already-upstream master &&
> + base="$(git rev-parse --verify HEAD)" &&
> +
> + test_commit A1 &&
> + test_commit A2 &&
> + git reset --hard $base &&
> + test_commit B1 &&
> + test_tick &&
> + git merge -m "Merge branch A" A2 &&
> +
> + git checkout -b upstream-with-a2 $base &&
> + test_tick &&
> + git cherry-pick A2 &&
> +
> + git checkout already-upstream &&
> + test_tick &&
> + git rebase -i -r upstream-with-a2 &&
> + test_cmp_graph upstream-with-a2.. <<-\EOF
> + *   Merge branch A
> + |\
> + | * A1
> + * | B1
> + |/
> + o A2
> + EOF
> +'
> +
> +test_done
> -- 
> 2.17.0.windows.1.15.gaa56ade3205
>
>
>
[1]https://public-inbox.org/git/2FA1998250474E76A386B82AD635E56A@PhilipOakley/ 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-21 15:56               ` Phillip Wood
@ 2018-04-22 17:17                 ` Phillip Wood
  2018-04-23 12:22                   ` Johannes Schindelin
  2018-04-23 12:20                 ` Johannes Schindelin
  1 sibling, 1 reply; 412+ messages in thread
From: Phillip Wood @ 2018-04-22 17:17 UTC (permalink / raw)
  To: Johannes Schindelin, git
  Cc: Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

On 21/04/18 16:56, Phillip Wood wrote:
> On 21/04/18 11:33, Johannes Schindelin wrote:
>> This patch is part of the effort to reimplement `--preserve-merges` with
>> a substantially improved design, a design that has been developed in the
>> Git for Windows project to maintain the dozens of Windows-specific patch
>> series on top of upstream Git.
>>
>> The previous patch implemented the `label` and `reset` commands to label
>> commits and to reset to labeled commits. This patch adds the `merge`
>> command, with the following syntax:
> 
> The two patches seem to have been fused together in this series.
> 
> If the reset command fails because it would overwrite untracked files it
> says
> 
> error: Untracked working tree file 'b' would be overwritten by merge.
> 
> Followed by the hint to edit the todo file. Saying 'merge' rather
> 'reset' is possibly confusing to users. Perhaps it could call
> setup_unpack_trees_porcelain(), though that would need to be extended to
> handle 'reset'.


> Also it currently refuses to overwrite ignored files
> which is either annoying or safe depending on one's point of view.

Looking at the existing code this is consistent with (most) of the rest
of the sequencer. The code to fast-forward commits will overwrite
ignored files, and I think the initial checkout will as well but the
rest (picking commits and the new merge command) will not.

> Best Wishes
> 
> Phillip
> 
>>
>>     merge [-C <commit>] <rev> # <oneline>
>>
>> The <commit> parameter in this instance is the *original* merge commit,
>> whose author and message will be used for the merge commit that is about
>> to be created.
>>
>> The <rev> parameter refers to the (possibly rewritten) revision to
>> merge. Let's see an example of a todo list:
>>
>>     label onto
>>
>>     # Branch abc
>>     reset onto
>>     pick deadbeef Hello, world!
>>     label abc
>>
>>     reset onto
>>     pick cafecafe And now for something completely different
>>     merge -C baaabaaa abc # Merge the branch 'abc' into master
>>
>> To edit the merge commit's message (a "reword" for merges, if you will),
>> use `-c` (lower-case) instead of `-C`; this convention was borrowed from
>> `git commit` that also supports `-c` and `-C` with similar meanings.
>>
>> To create *new* merges, i.e. without copying the commit message from an
>> existing commit, simply omit the `-C <commit>` parameter (which will
>> open an editor for the merge message):
>>
>>     merge abc
>>
>> This comes in handy when splitting a branch into two or more branches.
>>
>> Note: this patch only adds support for recursive merges, to keep things
>> simple. Support for octopus merges will be added later in a separate
>> patch series, support for merges using strategies other than the
>> recursive merge is left for the future.
>>
>> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
>> ---
>>   git-rebase--interactive.sh |   6 +
>>   sequencer.c                | 407 ++++++++++++++++++++++++++++++++++++-
>>   2 files changed, 406 insertions(+), 7 deletions(-)
>>
>> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
>> index e1b865f43f2..ccd5254d1c9 100644
>> --- a/git-rebase--interactive.sh
>> +++ b/git-rebase--interactive.sh
>> @@ -162,6 +162,12 @@ s, squash <commit> = use commit, but meld into
>> previous commit
>>   f, fixup <commit> = like \"squash\", but discard this commit's log
>> message
>>   x, exec <commit> = run command (the rest of the line) using shell
>>   d, drop <commit> = remove commit
>> +l, label <label> = label current HEAD with a name
>> +t, reset <label> = reset HEAD to a label
>> +m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
>> +.       create a merge commit using the original merge commit's
>> +.       message (or the oneline, if no original merge commit was
>> +.       specified). Use -c <commit> to reword the commit message.
>>     These lines can be re-ordered; they are executed from top to bottom.
>>   " | git stripspace --comment-lines >>"$todo"
>> diff --git a/sequencer.c b/sequencer.c
>> index 01443e0f245..35fcacbdf0f 100644
>> --- a/sequencer.c
>> +++ b/sequencer.c
>> @@ -23,6 +23,8 @@
>>   #include "hashmap.h"
>>   #include "notes-utils.h"
>>   #include "sigchain.h"
>> +#include "unpack-trees.h"
>> +#include "worktree.h"
>>     #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
>>   @@ -120,6 +122,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha,
>> "rebase-merge/stopped-sha")
>>   static GIT_PATH_FUNC(rebase_path_rewritten_list,
>> "rebase-merge/rewritten-list")
>>   static GIT_PATH_FUNC(rebase_path_rewritten_pending,
>>       "rebase-merge/rewritten-pending")
>> +
>> +/*
>> + * The path of the file listing refs that need to be deleted after
>> the rebase
>> + * finishes. This is used by the `label` command to record the need
>> for cleanup.
>> + */
>> +static GIT_PATH_FUNC(rebase_path_refs_to_delete,
>> "rebase-merge/refs-to-delete")
>> +
>>   /*
>>    * The following files are written by git-rebase just after parsing the
>>    * command-line (and are only consumed, not modified, by the
>> sequencer).
>> @@ -244,18 +253,34 @@ static const char *gpg_sign_opt_quoted(struct
>> replay_opts *opts)
>>     int sequencer_remove_state(struct replay_opts *opts)
>>   {
>> -    struct strbuf dir = STRBUF_INIT;
>> +    struct strbuf buf = STRBUF_INIT;
>>       int i;
>>   +    if (is_rebase_i(opts) &&
>> +        strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
>> +        char *p = buf.buf;
>> +        while (*p) {
>> +            char *eol = strchr(p, '\n');
>> +            if (eol)
>> +                *eol = '\0';
>> +            if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
>> +                warning(_("could not delete '%s'"), p);
>> +            if (!eol)
>> +                break;
>> +            p = eol + 1;
>> +        }
>> +    }
>> +
>>       free(opts->gpg_sign);
>>       free(opts->strategy);
>>       for (i = 0; i < opts->xopts_nr; i++)
>>           free(opts->xopts[i]);
>>       free(opts->xopts);
>>   -    strbuf_addstr(&dir, get_dir(opts));
>> -    remove_dir_recursively(&dir, 0);
>> -    strbuf_release(&dir);
>> +    strbuf_reset(&buf);
>> +    strbuf_addstr(&buf, get_dir(opts));
>> +    remove_dir_recursively(&buf, 0);
>> +    strbuf_release(&buf);
>>         return 0;
>>   }
>> @@ -1279,6 +1304,9 @@ enum todo_command {
>>       TODO_SQUASH,
>>       /* commands that do something else than handling a single commit */
>>       TODO_EXEC,
>> +    TODO_LABEL,
>> +    TODO_RESET,
>> +    TODO_MERGE,
>>       /* commands that do nothing but are counted for reporting
>> progress */
>>       TODO_NOOP,
>>       TODO_DROP,
>> @@ -1297,6 +1325,9 @@ static struct {
>>       { 'f', "fixup" },
>>       { 's', "squash" },
>>       { 'x', "exec" },
>> +    { 'l', "label" },
>> +    { 't', "reset" },
>> +    { 'm', "merge" },
>>       { 0,   "noop" },
>>       { 'd', "drop" },
>>       { 0,   NULL }
>> @@ -1724,9 +1755,14 @@ static int read_and_refresh_cache(struct
>> replay_opts *opts)
>>       return 0;
>>   }
>>   +enum todo_item_flags {
>> +    TODO_EDIT_MERGE_MSG = 1
>> +};
>> +
>>   struct todo_item {
>>       enum todo_command command;
>>       struct commit *commit;
>> +    unsigned int flags;
>>       const char *arg;
>>       int arg_len;
>>       size_t offset_in_buf;
>> @@ -1761,6 +1797,8 @@ static int parse_insn_line(struct todo_item
>> *item, const char *bol, char *eol)
>>       char *end_of_object_name;
>>       int i, saved, status, padding;
>>   +    item->flags = 0;
>> +
>>       /* left-trim */
>>       bol += strspn(bol, " \t");
>>   @@ -1802,13 +1840,29 @@ static int parse_insn_line(struct todo_item
>> *item, const char *bol, char *eol)
>>           return error(_("missing arguments for %s"),
>>                    command_to_string(item->command));
>>   -    if (item->command == TODO_EXEC) {
>> +    if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
>> +        item->command == TODO_RESET) {
>>           item->commit = NULL;
>>           item->arg = bol;
>>           item->arg_len = (int)(eol - bol);
>>           return 0;
>>       }
>>   +    if (item->command == TODO_MERGE) {
>> +        if (skip_prefix(bol, "-C", &bol))
>> +            bol += strspn(bol, " \t");
>> +        else if (skip_prefix(bol, "-c", &bol)) {
>> +            bol += strspn(bol, " \t");
>> +            item->flags |= TODO_EDIT_MERGE_MSG;
>> +        } else {
>> +            item->flags |= TODO_EDIT_MERGE_MSG;
>> +            item->commit = NULL;
>> +            item->arg = bol;
>> +            item->arg_len = (int)(eol - bol);
>> +            return 0;
>> +        }
>> +    }
>> +
>>       end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
>>       saved = *end_of_object_name;
>>       *end_of_object_name = '\0';
>> @@ -2465,6 +2519,305 @@ static int do_exec(const char *command_line)
>>       return status;
>>   }
>>   +static int safe_append(const char *filename, const char *fmt, ...)
>> +{
>> +    va_list ap;
>> +    struct lock_file lock = LOCK_INIT;
>> +    int fd = hold_lock_file_for_update(&lock, filename,
>> +                       LOCK_REPORT_ON_ERROR);
>> +    struct strbuf buf = STRBUF_INIT;
>> +
>> +    if (fd < 0)
>> +        return -1;
>> +
>> +    if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT) {
>> +        error_errno(_("could not read '%s'"), filename);
>> +        rollback_lock_file(&lock);
>> +        return -1;
>> +    }
>> +    strbuf_complete(&buf, '\n');
>> +    va_start(ap, fmt);
>> +    strbuf_vaddf(&buf, fmt, ap);
>> +    va_end(ap);
>> +
>> +    if (write_in_full(fd, buf.buf, buf.len) < 0) {
>> +        error_errno(_("could not write to '%s'"), filename);
>> +        strbuf_release(&buf);
>> +        rollback_lock_file(&lock);
>> +        return -1;
>> +    }
>> +    if (commit_lock_file(&lock) < 0) {
>> +        strbuf_release(&buf);
>> +        rollback_lock_file(&lock);
>> +        return error(_("failed to finalize '%s'"), filename);
>> +    }
>> +
>> +    strbuf_release(&buf);
>> +    return 0;
>> +}
>> +
>> +static int do_label(const char *name, int len)
>> +{
>> +    struct ref_store *refs = get_main_ref_store();
>> +    struct ref_transaction *transaction;
>> +    struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
>> +    struct strbuf msg = STRBUF_INIT;
>> +    int ret = 0;
>> +    struct object_id head_oid;
>> +
>> +    if (len == 1 && *name == '#')
>> +        return error("Illegal label name: '%.*s'", len, name);
>> +
>> +    strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
>> +    strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
>> +
>> +    transaction = ref_store_transaction_begin(refs, &err);
>> +    if (!transaction) {
>> +        error("%s", err.buf);
>> +        ret = -1;
>> +    } else if (get_oid("HEAD", &head_oid)) {
>> +        error(_("could not read HEAD"));
>> +        ret = -1;
>> +    } else if (ref_transaction_update(transaction, ref_name.buf,
>> &head_oid,
>> +                      NULL, 0, msg.buf, &err) < 0 ||
>> +           ref_transaction_commit(transaction, &err)) {
>> +        error("%s", err.buf);
>> +        ret = -1;
>> +    }
>> +    ref_transaction_free(transaction);
>> +    strbuf_release(&err);
>> +    strbuf_release(&msg);
>> +
>> +    if (!ret)
>> +        ret = safe_append(rebase_path_refs_to_delete(),
>> +                  "%s\n", ref_name.buf);
>> +    strbuf_release(&ref_name);
>> +
>> +    return ret;
>> +}
>> +
>> +static const char *reflog_message(struct replay_opts *opts,
>> +    const char *sub_action, const char *fmt, ...);
>> +
>> +static int do_reset(const char *name, int len, struct replay_opts *opts)
>> +{
>> +    struct strbuf ref_name = STRBUF_INIT;
>> +    struct object_id oid;
>> +    struct lock_file lock = LOCK_INIT;
>> +    struct tree_desc desc;
>> +    struct tree *tree;
>> +    struct unpack_trees_options unpack_tree_opts;
>> +    int ret = 0, i;
>> +
>> +    if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
>> +        return -1;
>> +
>> +    /* Determine the length of the label */
>> +    for (i = 0; i < len; i++)
>> +        if (isspace(name[i]))
>> +            len = i;
>> +
>> +    strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
>> +    if (get_oid(ref_name.buf, &oid) &&
>> +        get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
>> +        error(_("could not read '%s'"), ref_name.buf);
>> +        rollback_lock_file(&lock);
>> +        strbuf_release(&ref_name);
>> +        return -1;
>> +    }
>> +
>> +    memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
>> +    unpack_tree_opts.head_idx = 1;
>> +    unpack_tree_opts.src_index = &the_index;
>> +    unpack_tree_opts.dst_index = &the_index;
>> +    unpack_tree_opts.fn = oneway_merge;
>> +    unpack_tree_opts.merge = 1;
>> +    unpack_tree_opts.update = 1;
>> +
>> +    if (read_cache_unmerged()) {
>> +        rollback_lock_file(&lock);
>> +        strbuf_release(&ref_name);
>> +        return error_resolve_conflict(_(action_name(opts)));
>> +    }
>> +
>> +    if (!fill_tree_descriptor(&desc, &oid)) {
>> +        error(_("failed to find tree of %s"), oid_to_hex(&oid));
>> +        rollback_lock_file(&lock);
>> +        free((void *)desc.buffer);
>> +        strbuf_release(&ref_name);
>> +        return -1;
>> +    }
>> +
>> +    if (unpack_trees(1, &desc, &unpack_tree_opts)) {
>> +        rollback_lock_file(&lock);
>> +        free((void *)desc.buffer);
>> +        strbuf_release(&ref_name);
>> +        return -1;
>> +    }
>> +
>> +    tree = parse_tree_indirect(&oid);
>> +    prime_cache_tree(&the_index, tree);
>> +
>> +    if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
>> +        ret = error(_("could not write index"));
>> +    free((void *)desc.buffer);
>> +
>> +    if (!ret)
>> +        ret = update_ref(reflog_message(opts, "reset", "'%.*s'",
>> +                        len, name), "HEAD", &oid,
>> +                 NULL, 0, UPDATE_REFS_MSG_ON_ERR);
>> +
>> +    strbuf_release(&ref_name);
>> +    return ret;
>> +}
>> +
>> +static int do_merge(struct commit *commit, const char *arg, int arg_len,
>> +            int flags, struct replay_opts *opts)
>> +{
>> +    int run_commit_flags = (flags & TODO_EDIT_MERGE_MSG) ?
>> +        EDIT_MSG | VERIFY_MSG : 0;
>> +    struct strbuf ref_name = STRBUF_INIT;
>> +    struct commit *head_commit, *merge_commit, *i;
>> +    struct commit_list *bases, *j, *reversed = NULL;
>> +    struct merge_options o;
>> +    int merge_arg_len, oneline_offset, ret;
>> +    static struct lock_file lock;
>> +    const char *p;
>> +
>> +    if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0) {
>> +        ret = -1;
>> +        goto leave_merge;
>> +    }
>> +
>> +    head_commit = lookup_commit_reference_by_name("HEAD");
>> +    if (!head_commit) {
>> +        ret = error(_("cannot merge without a current revision"));
>> +        goto leave_merge;
>> +    }
>> +
>> +    oneline_offset = arg_len;
>> +    merge_arg_len = strcspn(arg, " \t\n");
>> +    p = arg + merge_arg_len;
>> +    p += strspn(p, " \t\n");
>> +    if (*p == '#' && (!p[1] || isspace(p[1]))) {
>> +        p += 1 + strspn(p + 1, " \t\n");
>> +        oneline_offset = p - arg;
>> +    } else if (p - arg < arg_len)
>> +        BUG("octopus merges are not supported yet: '%s'", p);
>> +
>> +    strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
>> +    merge_commit = lookup_commit_reference_by_name(ref_name.buf);
>> +    if (!merge_commit) {
>> +        /* fall back to non-rewritten ref or commit */
>> +        strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
>> +        merge_commit = lookup_commit_reference_by_name(ref_name.buf);
>> +    }
>> +
>> +    if (!merge_commit) {
>> +        ret = error(_("could not resolve '%s'"), ref_name.buf);
>> +        goto leave_merge;
>> +    }
>> +
>> +    if (commit) {
>> +        const char *message = get_commit_buffer(commit, NULL);
>> +        const char *body;
>> +        int len;
>> +
>> +        if (!message) {
>> +            ret = error(_("could not get commit message of '%s'"),
>> +                    oid_to_hex(&commit->object.oid));
>> +            goto leave_merge;
>> +        }
>> +        write_author_script(message);
>> +        find_commit_subject(message, &body);
>> +        len = strlen(body);
>> +        ret = write_message(body, len, git_path_merge_msg(), 0);
>> +        unuse_commit_buffer(commit, message);
>> +        if (ret) {
>> +            error_errno(_("could not write '%s'"),
>> +                    git_path_merge_msg());
>> +            goto leave_merge;
>> +        }
>> +    } else {
>> +        struct strbuf buf = STRBUF_INIT;
>> +        int len;
>> +
>> +        strbuf_addf(&buf, "author %s", git_author_info(0));
>> +        write_author_script(buf.buf);
>> +        strbuf_reset(&buf);
>> +
>> +        if (oneline_offset < arg_len) {
>> +            p = arg + oneline_offset;
>> +            len = arg_len - oneline_offset;
>> +        } else {
>> +            strbuf_addf(&buf, "Merge branch '%.*s'",
>> +                    merge_arg_len, arg);
>> +            p = buf.buf;
>> +            len = buf.len;
>> +        }
>> +
>> +        ret = write_message(p, len, git_path_merge_msg(), 0);
>> +        strbuf_release(&buf);
>> +        if (ret) {
>> +            error_errno(_("could not write '%s'"),
>> +                    git_path_merge_msg());
>> +            goto leave_merge;
>> +        }
>> +    }
>> +
>> +    write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
>> +              git_path_merge_head(), 0);
>> +    write_message("no-ff", 5, git_path_merge_mode(), 0);
>> +
>> +    bases = get_merge_bases(head_commit, merge_commit);
>> +    for (j = bases; j; j = j->next)
>> +        commit_list_insert(j->item, &reversed);
>> +    free_commit_list(bases);
>> +
>> +    read_cache();
>> +    init_merge_options(&o);
>> +    o.branch1 = "HEAD";
>> +    o.branch2 = ref_name.buf;
>> +    o.buffer_output = 2;
>> +
>> +    ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
>> +    if (ret <= 0)
>> +        fputs(o.obuf.buf, stdout);
>> +    strbuf_release(&o.obuf);
>> +    if (ret < 0) {
>> +        error(_("could not even attempt to merge '%.*s'"),
>> +              merge_arg_len, arg);
>> +        goto leave_merge;
>> +    }
>> +    /*
>> +     * The return value of merge_recursive() is 1 on clean, and 0 on
>> +     * unclean merge.
>> +     *
>> +     * Let's reverse that, so that do_merge() returns 0 upon success and
>> +     * 1 upon failed merge (keeping the return value -1 for the cases
>> where
>> +     * we will want to reschedule the `merge` command).
>> +     */
>> +    ret = !ret;
>> +
>> +    if (active_cache_changed &&
>> +        write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
>> +        ret = error(_("merge: Unable to write new index file"));
>> +        goto leave_merge;
>> +    }
>> +
>> +    rollback_lock_file(&lock);
>> +    if (ret)
>> +        rerere(opts->allow_rerere_auto);
>> +    else
>> +        ret = run_git_commit(git_path_merge_msg(), opts,
>> +                     run_commit_flags);
>> +
>> +leave_merge:
>> +    strbuf_release(&ref_name);
>> +    rollback_lock_file(&lock);
>> +    return ret;
>> +}
>> +
>>   static int is_final_fixup(struct todo_list *todo_list)
>>   {
>>       int i = todo_list->current;
>> @@ -2568,7 +2921,7 @@ N_("Could not execute the todo command\n"
>>     static int pick_commits(struct todo_list *todo_list, struct
>> replay_opts *opts)
>>   {
>> -    int res = 0;
>> +    int res = 0, reschedule = 0;
>>         setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
>>       if (opts->allow_ff)
>> @@ -2639,7 +2992,7 @@ static int pick_commits(struct todo_list
>> *todo_list, struct replay_opts *opts)
>>                       intend_to_amend();
>>                   return error_failed_squash(item->commit, opts,
>>                       item->arg_len, item->arg);
>> -            } else if (res && is_rebase_i(opts))
>> +            } else if (res && is_rebase_i(opts) && item->commit)
>>                   return res | error_with_patch(item->commit,
>>                       item->arg, item->arg_len, opts, res,
>>                       item->command == TODO_REWORD);
>> @@ -2665,9 +3018,41 @@ static int pick_commits(struct todo_list
>> *todo_list, struct replay_opts *opts)
>>                   /* `current` will be incremented below */
>>                   todo_list->current = -1;
>>               }
>> +        } else if (item->command == TODO_LABEL) {
>> +            if ((res = do_label(item->arg, item->arg_len)))
>> +                reschedule = 1;
>> +        } else if (item->command == TODO_RESET) {
>> +            if ((res = do_reset(item->arg, item->arg_len, opts)))
>> +                reschedule = 1;
>> +        } else if (item->command == TODO_MERGE) {
>> +            if ((res = do_merge(item->commit,
>> +                        item->arg, item->arg_len,
>> +                        item->flags, opts)) < 0)
>> +                reschedule = 1;
>> +            else if (res > 0)
>> +                /* failed with merge conflicts */
>> +                return error_with_patch(item->commit,
>> +                            item->arg,
>> +                            item->arg_len, opts,
>> +                            res, 0);
>>           } else if (!is_noop(item->command))
>>               return error(_("unknown command %d"), item->command);
>>   +        if (reschedule) {
>> +            advise(_(rescheduled_advice),
>> +                   get_item_line_length(todo_list,
>> +                            todo_list->current),
>> +                   get_item_line(todo_list, todo_list->current));
>> +            todo_list->current--;
>> +            if (save_todo(todo_list, opts))
>> +                return -1;
>> +            if (item->commit)
>> +                return error_with_patch(item->commit,
>> +                            item->arg,
>> +                            item->arg_len, opts,
>> +                            res, 0);
>> +        }
>> +
>>           todo_list->current++;
>>           if (res)
>>               return res;
>> @@ -3147,8 +3532,16 @@ int transform_todos(unsigned flags)
>>                         short_commit_name(item->commit) :
>>                         oid_to_hex(&item->commit->object.oid);
>>   +            if (item->command == TODO_MERGE) {
>> +                if (item->flags & TODO_EDIT_MERGE_MSG)
>> +                    strbuf_addstr(&buf, " -c");
>> +                else
>> +                    strbuf_addstr(&buf, " -C");
>> +            }
>> +
>>               strbuf_addf(&buf, " %s", oid);
>>           }
>> +
>>           /* add all the rest */
>>           if (!item->arg_len)
>>               strbuf_addch(&buf, '\n');
>>
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-22 12:01               ` Philip Oakley
@ 2018-04-23 12:03                 ` Johannes Schindelin
  2018-04-23 20:34                   ` Philip Oakley
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-23 12:03 UTC (permalink / raw)
  To: Philip Oakley
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Hi Philip,

On Sun, 22 Apr 2018, Philip Oakley wrote:

> From: "Johannes Schindelin" <johannes.schindelin@gmx.de>
> > This patch is part of the effort to reimplement `--preserve-merges` with
> > a substantially improved design, a design that has been developed in the
> > Git for Windows project to maintain the dozens of Windows-specific patch
> > series on top of upstream Git.
> >
> > The previous patch implemented the `label` and `reset` commands to label
> 
> The previous patch was [Patch 05/16] git-rebase--interactive: clarify
> arguments, so this statement doesn't appear to be true. Has a patch been
> missed or re-ordered? Or should it be simply "This patch implements" ?
> Likewise the patch subject would be updated.

As Phillip guessed correctly, it was a mistaken `git commit --amend`.

> > commits and to reset to labeled commits. This patch adds the `merge`
> 
> s/adds/also adds/ ?

No, as I really want to keep those two commits separate. I disentangled
them.

> > command, with the following syntax:
> >
> > merge [-C <commit>] <rev> # <oneline>
> >
> > The <commit> parameter in this instance is the *original* merge commit,
> > whose author and message will be used for the merge commit that is about
> > to be created.
> >
> > The <rev> parameter refers to the (possibly rewritten) revision to
> > merge. Let's see an example of a todo list:
> >
> The example ought to also note that `label onto` is to
> `# label current HEAD with a name`, seeing as this is the first occurance.
> It may be obvious in retrospect, but not at first reading.

I added some sentence to describe what `label onto` does and why.

> > label onto
> >
> > # Branch abc
> > reset onto
> 
> Is this reset strictly necessary. We are already there @head.

No, this is not strictly necessary, but

- it makes it easier to auto-generate (otherwise you would have to keep
  track of the "current HEAD" while generating that todo list, and

- if I keep the `reset onto` there, then it is *a lot* easier to reorder
  topic branches.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-21 15:56               ` Phillip Wood
  2018-04-22 17:17                 ` Phillip Wood
@ 2018-04-23 12:20                 ` Johannes Schindelin
  2018-04-23 15:54                   ` Phillip Wood
  1 sibling, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-23 12:20 UTC (permalink / raw)
  To: Phillip Wood
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Igor Djordjevic, Johannes Sixt, Sergey Organov,
	Martin Ågren

Hi Phillip,

On Sat, 21 Apr 2018, Phillip Wood wrote:

> On 21/04/18 11:33, Johannes Schindelin wrote:
> > This patch is part of the effort to reimplement `--preserve-merges` with
> > a substantially improved design, a design that has been developed in the
> > Git for Windows project to maintain the dozens of Windows-specific patch
> > series on top of upstream Git.
> > 
> > The previous patch implemented the `label` and `reset` commands to label
> > commits and to reset to labeled commits. This patch adds the `merge`
> > command, with the following syntax:
> 
> The two patches seem to have been fused together in this series.

Indeed. I have yet to investigate further how that happened, it could be a
bug in my series after all.

> If the reset command fails because it would overwrite untracked files it
> says
> 
> error: Untracked working tree file 'b' would be overwritten by merge.
> 
> Followed by the hint to edit the todo file. Saying 'merge' rather 'reset' is
> possibly confusing to users. Perhaps it could call
> setup_unpack_trees_porcelain(), though that would need to be extended to
> handle 'reset'.

Yes, and it changes global state :-(

Maybe we can leave it as-is for now? After all, it should be clear to the
user what is happening. The most important part is the "Untracked working
tree file"...

> Also it currently refuses to overwrite ignored files which is either
> annoying or safe depending on one's point of view.

Let me think about that. My gut feeling says: if it is easy to do, then
let's nuke ignored files, but keep untracked files. But I do not think
that the unpack-trees machinery was taught to know about .gitignore...

Seeing as `label` and `reset` really are mostly about revisions we see
along the lines, I think that the common case will *not* overwrite any
untracked files, ever. You would have to use `reset` on a
not-previously-seen commit and/or add `exec` commands designed to
interfere with the `reset`.

So I tend to want to not bother with discerning between untracked and
ignored files here.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-22 17:17                 ` Phillip Wood
@ 2018-04-23 12:22                   ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-23 12:22 UTC (permalink / raw)
  To: Phillip Wood
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Igor Djordjevic, Johannes Sixt, Sergey Organov,
	Martin Ågren

Hi Phillip,

On Sun, 22 Apr 2018, Phillip Wood wrote:

> On 21/04/18 16:56, Phillip Wood wrote:
> > On 21/04/18 11:33, Johannes Schindelin wrote:
> >> This patch is part of the effort to reimplement `--preserve-merges` with
> >> a substantially improved design, a design that has been developed in the
> >> Git for Windows project to maintain the dozens of Windows-specific patch
> >> series on top of upstream Git.
> >>
> >> The previous patch implemented the `label` and `reset` commands to label
> >> commits and to reset to labeled commits. This patch adds the `merge`
> >> command, with the following syntax:
> > 
> > The two patches seem to have been fused together in this series.
> > 
> > If the reset command fails because it would overwrite untracked files it
> > says
> > 
> > error: Untracked working tree file 'b' would be overwritten by merge.
> > 
> > Followed by the hint to edit the todo file. Saying 'merge' rather
> > 'reset' is possibly confusing to users. Perhaps it could call
> > setup_unpack_trees_porcelain(), though that would need to be extended to
> > handle 'reset'.
> 
> 
> > Also it currently refuses to overwrite ignored files
> > which is either annoying or safe depending on one's point of view.
> 
> Looking at the existing code this is consistent with (most) of the rest
> of the sequencer. The code to fast-forward commits will overwrite
> ignored files, and I think the initial checkout will as well but the
> rest (picking commits and the new merge command) will not.

I never thought about that... but then, I never came close to encountering
such an issue, as I do not typically turn ignored files into tracked ones
;-)

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-23 12:20                 ` Johannes Schindelin
@ 2018-04-23 15:54                   ` Phillip Wood
  2018-04-24  5:13                     ` Martin Ågren
  0 siblings, 1 reply; 412+ messages in thread
From: Phillip Wood @ 2018-04-23 15:54 UTC (permalink / raw)
  To: Johannes Schindelin, Phillip Wood
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Igor Djordjevic, Johannes Sixt, Sergey Organov,
	Martin Ågren

On 23/04/18 13:20, Johannes Schindelin wrote:
> Hi Phillip,
> 
> On Sat, 21 Apr 2018, Phillip Wood wrote:
> 
>> On 21/04/18 11:33, Johannes Schindelin wrote:
>>> This patch is part of the effort to reimplement `--preserve-merges` with
>>> a substantially improved design, a design that has been developed in the
>>> Git for Windows project to maintain the dozens of Windows-specific patch
>>> series on top of upstream Git.
>>>
>>> The previous patch implemented the `label` and `reset` commands to label
>>> commits and to reset to labeled commits. This patch adds the `merge`
>>> command, with the following syntax:
>>
>> The two patches seem to have been fused together in this series.
> 
> Indeed. I have yet to investigate further how that happened, it could be a
> bug in my series after all.
> 
>> If the reset command fails because it would overwrite untracked files it
>> says
>>
>> error: Untracked working tree file 'b' would be overwritten by merge.
>>
>> Followed by the hint to edit the todo file. Saying 'merge' rather 'reset' is
>> possibly confusing to users. Perhaps it could call
>> setup_unpack_trees_porcelain(), though that would need to be extended to
>> handle 'reset'.
> 
> Yes, and it changes global state :-(
> 
> Maybe we can leave it as-is for now? After all, it should be clear to the
> user what is happening. The most important part is the "Untracked working
> tree file"...

I'm fine with leaving it, I've might get round to doing a small series 
to clean things up slightly in a few weeks. At the moment 
setup_unpack_trees_porcelain() leaks memory as it is called for each 
merge and allocates new strings each time. It would also be nice if the 
error messages reflected the command, so it said 'cherry-pick', 'revert' 
or 'reset' rather than 'merge'

>> Also it currently refuses to overwrite ignored files which is either
>> annoying or safe depending on one's point of view.
> 
> Let me think about that. My gut feeling says: if it is easy to do, then
> let's nuke ignored files, but keep untracked files. But I do not think
> that the unpack-trees machinery was taught to know about .gitignore...
> 
> Seeing as `label` and `reset` really are mostly about revisions we see
> along the lines, I think that the common case will *not* overwrite any
> untracked files, ever. You would have to use `reset` on a
> not-previously-seen commit and/or add `exec` commands designed to
> interfere with the `reset`.
> 
> So I tend to want to not bother with discerning between untracked and
> ignored files here.

I don't think it's a pressing concern. In the past I once had a patch 
series that removed some tracked files in favor of having the build 
system generate them and added them to .gitignore. Each time I rebased I 
had to manually remove them at some stage which was annoying but that is 
quite a rare occurrence.

Best Wishes

Phillip


> Ciao,
> Dscho
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages
  2018-04-21  7:34               ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
                                   ` (3 preceding siblings ...)
  2018-04-21  7:39                 ` [PATCH v3 4/4] rebase --skip: clean up commit message after a failed fixup/squash Johannes Schindelin
@ 2018-04-23 18:11                 ` Stefan Beller
  2018-04-23 19:50                   ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combinationof" " Phillip Wood
  2018-04-24  1:28                 ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" " Junio C Hamano
  5 siblings, 1 reply; 412+ messages in thread
From: Stefan Beller @ 2018-04-23 18:11 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Eric Sunshine

On Sat, Apr 21, 2018 at 12:34 AM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> Eric Sunshine pointed out that I had such a commit message in
> https://public-inbox.org/git/CAPig+cRrS0_nYJJY=O6cboV630sNQHPV5QGrQdD8MW-sYzNFGQ@mail.gmail.com/
> and I went on a hunt to figure out how the heck this happened.
>
> Turns out that if there is a fixup/squash chain where the *last* command
> fails with merge conflicts, and we either --skip ahead or resolve the
> conflict to a clean tree and then --continue, our code does not do a
> final cleanup.
>
> Contrary to my initial gut feeling, this bug was not introduced by my
> rewrite in C of the core parts of rebase -i, but it looks to me as if
> that bug was with us for a very long time (at least the --skip part).
>
> The developer (read: user of rebase -i) in me says that we would want to
> fast-track this, but the author of rebase -i in me says that we should
> be cautious and cook this in `next` for a while.

I looked through the patches again and think this series is good to go.

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 0/4] rebase -i: avoid stale "# This is a combinationof" in commit messages
  2018-04-23 18:11                 ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Stefan Beller
@ 2018-04-23 19:50                   ` Phillip Wood
  2018-04-25 12:48                     ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Phillip Wood @ 2018-04-23 19:50 UTC (permalink / raw)
  To: Stefan Beller, Johannes Schindelin; +Cc: git, Junio C Hamano, Eric Sunshine

On 23/04/18 19:11, Stefan Beller wrote:
> 
> On Sat, Apr 21, 2018 at 12:34 AM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
>> Eric Sunshine pointed out that I had such a commit message in
>> https://public-inbox.org/git/CAPig+cRrS0_nYJJY=O6cboV630sNQHPV5QGrQdD8MW-sYzNFGQ@mail.gmail.com/
>> and I went on a hunt to figure out how the heck this happened.
>>
>> Turns out that if there is a fixup/squash chain where the *last* command
>> fails with merge conflicts, and we either --skip ahead or resolve the
>> conflict to a clean tree and then --continue, our code does not do a
>> final cleanup.
>>
>> Contrary to my initial gut feeling, this bug was not introduced by my
>> rewrite in C of the core parts of rebase -i, but it looks to me as if
>> that bug was with us for a very long time (at least the --skip part).
>>
>> The developer (read: user of rebase -i) in me says that we would want to
>> fast-track this, but the author of rebase -i in me says that we should
>> be cautious and cook this in `next` for a while.
> 
> I looked through the patches again and think this series is good to go.

I've just realized I commented on an outdated version as the new version 
was posted there rather than as a reply to v1. I've just looked through 
it and I'm not sure it addresses the unnecessary editing of the commit 
message of the previous commit if a single squash command is skipped as 
outlined in 
https://public-inbox.org/git/b6512eae-e214-9699-4d69-77117a0daec3@talktalk.net/

Best Wishes

Phillip

> Thanks,
> Stefan
> 

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-23 12:03                 ` Johannes Schindelin
@ 2018-04-23 20:34                   ` Philip Oakley
  2018-04-24  8:11                     ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Philip Oakley @ 2018-04-23 20:34 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

From: "Johannes Schindelin" <Johannes.Schindelin@gmx.de> : Monday, April 23, 
2018 1:03 PM
Subject: Re: [PATCH v8 06/16] sequencer: introduce the `merge` command


> Hi Philip,
>
[...]
>
>> > label onto
>> >
>> > # Branch abc
>> > reset onto
>>
>> Is this reset strictly necessary. We are already there @head.
>
> No, this is not strictly necessary, but

I've realised my misunderstanding. I was thinking this (and others) was 
equivalent to

$  git reset <thatHead'onto'> # maybe even --hard,

i.e. affecting the worktree

rather that just being a movement of the Head rev (though I may be having 
brain fade here regarding untracked files etc..)

>
> - it makes it easier to auto-generate (otherwise you would have to keep
>  track of the "current HEAD" while generating that todo list, and
>
> - if I keep the `reset onto` there, then it is *a lot* easier to reorder
>  topic branches.
>
> Ciao,
> Dscho
>
Thanks

Philip 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages
  2018-04-21  7:34               ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
                                   ` (4 preceding siblings ...)
  2018-04-23 18:11                 ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Stefan Beller
@ 2018-04-24  1:28                 ` Junio C Hamano
  5 siblings, 0 replies; 412+ messages in thread
From: Junio C Hamano @ 2018-04-24  1:28 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Eric Sunshine, Stefan Beller

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Eric Sunshine pointed out that I had such a commit message in
> https://public-inbox.org/git/CAPig+cRrS0_nYJJY=O6cboV630sNQHPV5QGrQdD8MW-sYzNFGQ@mail.gmail.com/
> and I went on a hunt to figure out how the heck this happened.
>
> Turns out that if there is a fixup/squash chain where the *last* command
> fails with merge conflicts, and we either --skip ahead or resolve the
> conflict to a clean tree and then --continue, our code does not do a
> final cleanup.
>
> Contrary to my initial gut feeling, this bug was not introduced by my
> rewrite in C of the core parts of rebase -i, but it looks to me as if
> that bug was with us for a very long time (at least the --skip part).
>
> The developer (read: user of rebase -i) in me says that we would want to
> fast-track this, but the author of rebase -i in me says that we should
> be cautious and cook this in `next` for a while.
>
> Fixes since v2 (thanks, Stefan!):
>
> - Fixed commit message of 2/4: "Thisis" -> "This is".
>
> - Reinstated the order where the `message-squash` file is renamed to
>   `message` first, and only if that succeeded, we delete the
>   `message-fixup` file.
>
> base-commit: fe0a9eaf31dd0c349ae4308498c33a5c3794b293

This round looks reasonable (the last one was already so, though
;-).  As this is not a recent regression, however, I think we would
want to have it regardless of recent updates to rebase-i that is
happening on the 'next down to master' front.

I've queued this round using base-commit of d32eb83c ("Git 2.16.3",
2018-03-22), the same base as the previous round, for that reason.

Merging it to the tip of 'master' and applying these patches
directly on top of 'master' result in identical trees, of course
(otherwise we wouldn't be able to maintain the stable releases and
make forward progress on the 'master' front at the same time ;-).


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 09/16] rebase: introduce the --rebase-merges option
  2018-04-22 14:15               ` Philip Oakley
@ 2018-04-24  5:01                 ` Junio C Hamano
  2018-04-24  9:03                   ` Johannes Schindelin
  2018-04-24  8:40                 ` Johannes Schindelin
  1 sibling, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-04-24  5:01 UTC (permalink / raw)
  To: Philip Oakley
  Cc: Johannes Schindelin, Git List, Jacob Keller, Stefan Beller,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

"Philip Oakley" <philipoakley@iee.org> writes:

>> +-r::
>> +--rebase-merges::
>> + By default, a rebase will simply drop merge commits and only rebase
>> + the non-merge commits. With this option, it will try to preserve
>> + the branching structure within the commits that are to be rebased,
>> + by recreating the merge commits. If a merge commit resolved any merge
>> + or contained manual amendments, then they will have to be re-applied
>> + manually.
>> ++
>> +This mode is similar in spirit to `--preserve-merges`, but in contrast to
>> +that option works well in interactive rebases: commits can be reordered,
>> +inserted and dropped at will.
>> ++
>> +It is currently only possible to recreate the merge commits using the
>> +`recursive` merge strategy; Different merge strategies can be used only
>> via
>> +explicit `exec git merge -s <strategy> [...]` commands.
>> +
>> -p::
>> --preserve-merges::
>>  Recreate merge commits instead of flattening the history by replaying
>
> Flatten is here in the context lines but its just a blunt statement that 'it
> is what it is'...

The first paragraph that explains --rebase-merges talks about what
happens when the option is not given, and says "drop merge commits
and only rebase the non-merge commits", which is not incorrect
per-se but does not make it explicit how the resulting topology
looks like.  I think it is easier to understand if it mentioned
"flattening" as well.  If flatten is not the word you want, perhaps
"make it linear" or something like that?

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-23 15:54                   ` Phillip Wood
@ 2018-04-24  5:13                     ` Martin Ågren
  2018-04-24  5:13                       ` [PATCH 1/2] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
                                         ` (2 more replies)
  0 siblings, 3 replies; 412+ messages in thread
From: Martin Ågren @ 2018-04-24  5:13 UTC (permalink / raw)
  To: Phillip Wood
  Cc: git, Johannes Schindelin, Junio C Hamano, Jacob Keller,
	Stefan Beller, Philip Oakley, Eric Sunshine, Igor Djordjevic,
	Johannes Sixt, Sergey Organov

Hi Phillip,

On 23 April 2018 at 17:54, Phillip Wood <phillip.wood@talktalk.net> wrote:
> I'm fine with leaving it, I've might get round to doing a small series to
> clean things up slightly in a few weeks. At the moment
> setup_unpack_trees_porcelain() leaks memory as it is called for each merge
> and allocates new strings each time. It would also be nice if the error
> messages reflected the command, so it said 'cherry-pick', 'revert' or
> 'reset' rather than 'merge'

This is a small patch series to introduce and use
`clear_unpack_trees_porcelain()`. Since Elijah is doing substantial
rewrites to one of the users of `setup_unpack_trees_porcelain()` [1], I
think we should hold off on these for now to avoid a quite evil merge.
(I haven't studied the details enough to be confident, but I think the
calls to `setup_...()` and `clear_...()` might need to be moved up the
call-chain.)

I'm posting this so we don't repeat each other's work. If you feel like
picking these up and running with them, go ahead. Otherwise, I will try
to get them in as soon as Elijah's series lands. I'll keep you posted.

[1] https://public-inbox.org/git/CAN0heSquJboMMgay+5XomqXCGoHtXxf1mJBmY_L7y+AA4eG0KA@mail.gmail.com/#t

Martin

Martin Ågren (2):
  merge: setup `opts` later in `checkout_fast_forward()`
  unpack_trees_options: free messages when done

 unpack-trees.h     |  5 +++++
 builtin/checkout.c |  1 +
 merge-recursive.c  |  1 +
 merge.c            | 35 ++++++++++++++++++++---------------
 unpack-trees.c     | 11 +++++++++++
 5 files changed, 38 insertions(+), 15 deletions(-)

-- 
2.17.0


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH 1/2] merge: setup `opts` later in `checkout_fast_forward()`
  2018-04-24  5:13                     ` Martin Ågren
@ 2018-04-24  5:13                       ` Martin Ågren
  2018-04-24  6:20                         ` Jacob Keller
  2018-04-24  5:13                       ` [PATCH 2/2] unpack_trees_options: free messages when done Martin Ågren
  2018-04-24  8:22                       ` [PATCH v8 06/16] sequencer: introduce the `merge` command Johannes Schindelin
  2 siblings, 1 reply; 412+ messages in thread
From: Martin Ågren @ 2018-04-24  5:13 UTC (permalink / raw)
  To: Phillip Wood
  Cc: git, Johannes Schindelin, Junio C Hamano, Jacob Keller,
	Stefan Beller, Philip Oakley, Eric Sunshine, Igor Djordjevic,
	Johannes Sixt, Sergey Organov

After we initialize the various fields in `opts` but before we actually
use them, we might return early. Move the initialization further down,
to immediately before we use `opts`.

This limits the scope of `opts` and will help a subsequent commit fix a
memory leak without having to worry about those early returns.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
---
 merge.c | 32 +++++++++++++++++---------------
 1 file changed, 17 insertions(+), 15 deletions(-)

diff --git a/merge.c b/merge.c
index f06a4773d4..f123658e58 100644
--- a/merge.c
+++ b/merge.c
@@ -94,8 +94,24 @@ int checkout_fast_forward(const struct object_id *head,
 		return -1;
 
 	memset(&trees, 0, sizeof(trees));
-	memset(&opts, 0, sizeof(opts));
 	memset(&t, 0, sizeof(t));
+
+	trees[nr_trees] = parse_tree_indirect(head);
+	if (!trees[nr_trees++]) {
+		rollback_lock_file(&lock_file);
+		return -1;
+	}
+	trees[nr_trees] = parse_tree_indirect(remote);
+	if (!trees[nr_trees++]) {
+		rollback_lock_file(&lock_file);
+		return -1;
+	}
+	for (i = 0; i < nr_trees; i++) {
+		parse_tree(trees[i]);
+		init_tree_desc(t+i, trees[i]->buffer, trees[i]->size);
+	}
+
+	memset(&opts, 0, sizeof(opts));
 	if (overwrite_ignore) {
 		memset(&dir, 0, sizeof(dir));
 		dir.flags |= DIR_SHOW_IGNORED;
@@ -112,20 +128,6 @@ int checkout_fast_forward(const struct object_id *head,
 	opts.fn = twoway_merge;
 	setup_unpack_trees_porcelain(&opts, "merge");
 
-	trees[nr_trees] = parse_tree_indirect(head);
-	if (!trees[nr_trees++]) {
-		rollback_lock_file(&lock_file);
-		return -1;
-	}
-	trees[nr_trees] = parse_tree_indirect(remote);
-	if (!trees[nr_trees++]) {
-		rollback_lock_file(&lock_file);
-		return -1;
-	}
-	for (i = 0; i < nr_trees; i++) {
-		parse_tree(trees[i]);
-		init_tree_desc(t+i, trees[i]->buffer, trees[i]->size);
-	}
 	if (unpack_trees(nr_trees, t, &opts)) {
 		rollback_lock_file(&lock_file);
 		return -1;
-- 
2.17.0


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH 2/2] unpack_trees_options: free messages when done
  2018-04-24  5:13                     ` Martin Ågren
  2018-04-24  5:13                       ` [PATCH 1/2] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
@ 2018-04-24  5:13                       ` Martin Ågren
  2018-04-24 16:29                         ` Elijah Newren
  2018-04-24  8:22                       ` [PATCH v8 06/16] sequencer: introduce the `merge` command Johannes Schindelin
  2 siblings, 1 reply; 412+ messages in thread
From: Martin Ågren @ 2018-04-24  5:13 UTC (permalink / raw)
  To: Phillip Wood
  Cc: git, Johannes Schindelin, Junio C Hamano, Jacob Keller,
	Stefan Beller, Philip Oakley, Eric Sunshine, Igor Djordjevic,
	Johannes Sixt, Sergey Organov

The strings allocated in `setup_unpack_trees_porcelain()` are never
freed. Provide a function `clear_unpack_trees_porcelain()` to do so and
call it in the functions which use `setup_unpack_trees_porcelain()`.

In all current callers, the pointers are about to go out of scope, so we
do not need to set them to NULL. Let's do so anyway so that a future
caller or restructured code doesn't suddenly start accessing dangling
pointers.

Note that we only take responsibility for the memory allocated in
`setup_unpack_trees_porcelain()` and not any other members of the
`struct unpack_trees_options`.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
---
 unpack-trees.h     |  5 +++++
 builtin/checkout.c |  1 +
 merge-recursive.c  |  1 +
 merge.c            |  3 +++
 unpack-trees.c     | 11 +++++++++++
 5 files changed, 21 insertions(+)

diff --git a/unpack-trees.h b/unpack-trees.h
index 6c48117b84..8c56cf0150 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -32,6 +32,11 @@ enum unpack_trees_error_types {
 void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 				  const char *cmd);
 
+/*
+ * Frees resources allocated by setup_unpack_trees_porcelain().
+ */
+extern void clear_unpack_trees_porcelain(struct unpack_trees_options *opts);
+
 struct unpack_trees_options {
 	unsigned int reset,
 		     merge,
diff --git a/builtin/checkout.c b/builtin/checkout.c
index b49b582071..5cebe170fc 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -526,6 +526,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 		init_tree_desc(&trees[1], tree->buffer, tree->size);
 
 		ret = unpack_trees(2, trees, &topts);
+		clear_unpack_trees_porcelain(&topts);
 		if (ret == -1) {
 			/*
 			 * Unpack couldn't do a trivial merge; either
diff --git a/merge-recursive.c b/merge-recursive.c
index 0c0d48624d..8229b91e2f 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -301,6 +301,7 @@ static int git_merge_trees(int index_only,
 	init_tree_desc_from_tree(t+2, merge);
 
 	rc = unpack_trees(3, t, &opts);
+	clear_unpack_trees_porcelain(&opts);
 	cache_tree_free(&active_cache_tree);
 	return rc;
 }
diff --git a/merge.c b/merge.c
index f123658e58..b433291d0c 100644
--- a/merge.c
+++ b/merge.c
@@ -130,8 +130,11 @@ int checkout_fast_forward(const struct object_id *head,
 
 	if (unpack_trees(nr_trees, t, &opts)) {
 		rollback_lock_file(&lock_file);
+		clear_unpack_trees_porcelain(&opts);
 		return -1;
 	}
+	clear_unpack_trees_porcelain(&opts);
+
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		return error(_("unable to write new index file"));
 	return 0;
diff --git a/unpack-trees.c b/unpack-trees.c
index e73745051e..4c76a29241 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -179,6 +179,17 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 		opts->unpack_rejects[i].strdup_strings = 1;
 }
 
+void clear_unpack_trees_porcelain(struct unpack_trees_options *opts)
+{
+	char **msgs = (char **)opts->msgs;
+
+	free(msgs[ERROR_WOULD_OVERWRITE]);
+	free(msgs[ERROR_WOULD_LOSE_UNTRACKED_REMOVED]);
+	free(msgs[ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN]);
+
+	memset(opts->msgs, 0, sizeof(opts->msgs));
+}
+
 static int do_add_entry(struct unpack_trees_options *o, struct cache_entry *ce,
 			 unsigned int set, unsigned int clear)
 {
-- 
2.17.0


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/2] merge: setup `opts` later in `checkout_fast_forward()`
  2018-04-24  5:13                       ` [PATCH 1/2] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
@ 2018-04-24  6:20                         ` Jacob Keller
  2018-04-24  9:36                           ` Martin Ågren
  0 siblings, 1 reply; 412+ messages in thread
From: Jacob Keller @ 2018-04-24  6:20 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Phillip Wood, Git mailing list, Johannes Schindelin,
	Junio C Hamano, Stefan Beller, Philip Oakley, Eric Sunshine,
	Igor Djordjevic, Johannes Sixt, Sergey Organov

On Mon, Apr 23, 2018 at 10:13 PM, Martin Ågren <martin.agren@gmail.com> wrote:
> After we initialize the various fields in `opts` but before we actually
> use them, we might return early. Move the initialization further down,
> to immediately before we use `opts`.
>
> This limits the scope of `opts` and will help a subsequent commit fix a
> memory leak without having to worry about those early returns.
>
> Signed-off-by: Martin Ågren <martin.agren@gmail.com>
> ---
>  merge.c | 32 +++++++++++++++++---------------
>  1 file changed, 17 insertions(+), 15 deletions(-)
>
> diff --git a/merge.c b/merge.c
> index f06a4773d4..f123658e58 100644
> --- a/merge.c
> +++ b/merge.c
> @@ -94,8 +94,24 @@ int checkout_fast_forward(const struct object_id *head,
>                 return -1;
>
>         memset(&trees, 0, sizeof(trees));
> -       memset(&opts, 0, sizeof(opts));
>         memset(&t, 0, sizeof(t));
> +
> +       trees[nr_trees] = parse_tree_indirect(head);
> +       if (!trees[nr_trees++]) {
> +               rollback_lock_file(&lock_file);
> +               return -1;
> +       }
> +       trees[nr_trees] = parse_tree_indirect(remote);
> +       if (!trees[nr_trees++]) {
> +               rollback_lock_file(&lock_file);
> +               return -1;
> +       }
> +       for (i = 0; i < nr_trees; i++) {
> +               parse_tree(trees[i]);
> +               init_tree_desc(t+i, trees[i]->buffer, trees[i]->size);
> +       }
> +
> +       memset(&opts, 0, sizeof(opts));
>         if (overwrite_ignore) {
>                 memset(&dir, 0, sizeof(dir));

I'm guessing the diff algorithm simply found that this was a more
compact representation of the change? It's a bit confusing when your
description indicates you "moved" some code down, but it looks like
you moved code up.

Thanks,
Jake

>                 dir.flags |= DIR_SHOW_IGNORED;
> @@ -112,20 +128,6 @@ int checkout_fast_forward(const struct object_id *head,
>         opts.fn = twoway_merge;
>         setup_unpack_trees_porcelain(&opts, "merge");
>
> -       trees[nr_trees] = parse_tree_indirect(head);
> -       if (!trees[nr_trees++]) {
> -               rollback_lock_file(&lock_file);
> -               return -1;
> -       }
> -       trees[nr_trees] = parse_tree_indirect(remote);
> -       if (!trees[nr_trees++]) {
> -               rollback_lock_file(&lock_file);
> -               return -1;
> -       }
> -       for (i = 0; i < nr_trees; i++) {
> -               parse_tree(trees[i]);
> -               init_tree_desc(t+i, trees[i]->buffer, trees[i]->size);
> -       }
>         if (unpack_trees(nr_trees, t, &opts)) {
>                 rollback_lock_file(&lock_file);
>                 return -1;
> --
> 2.17.0
>

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-23 20:34                   ` Philip Oakley
@ 2018-04-24  8:11                     ` Johannes Schindelin
  2018-04-24 19:41                       ` Philip Oakley
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-24  8:11 UTC (permalink / raw)
  To: Philip Oakley
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Hi Philip,

On Mon, 23 Apr 2018, Philip Oakley wrote:

> From: "Johannes Schindelin" <Johannes.Schindelin@gmx.de> : Monday, April 23,
> 2018 1:03 PM
> Subject: Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
> 
> [...]
> >
> > > > label onto
> > > >
> > > > # Branch abc
> > > > reset onto
> > >
> > > Is this reset strictly necessary. We are already there @head.
> >
> > No, this is not strictly necessary, but
> 
> I've realised my misunderstanding. I was thinking this (and others) was
> equivalent to
> 
> $  git reset <thatHead'onto'> # maybe even --hard,
> 
> i.e. affecting the worktree

Oh, but it *is* affecting the worktree. In this case, since we label HEAD
and then immediately reset to the label, there is just nothing to change.

Consider this example, though:

	label onto

	# Branch: from-philip
	reset onto
	pick abcdef something
	label from-philip

	# Branch: with-love
	reset onto
	pick 012345 else
	label with-love

	reset onto
	merge -C 98765 from-philip
	merge -C 43210 with-love

Only in the first instance is the `reset onto` a no-op, an incidental one.
After picking `something` and labeling the result as `from-philip`,
though, the next `reset onto` really resets the worktree.

> rather that just being a movement of the Head rev (though I may be having
> brain fade here regarding untracked files etc..)

The current way of doing things does not allow the `reset` to overwrite
untracked, nor ignored files (I think, I only verified the former, not the
latter).

But yeah, it is not just a movement of HEAD. It does reset the worktree,
although quite a bit more gently (and safely) than `git reset --hard`. In
that respect, this patch series is a drastic improvement over the Git
garden shears (which is the shell script I use in Git for Windows which
inspired this here patch series).

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-24  5:13                     ` Martin Ågren
  2018-04-24  5:13                       ` [PATCH 1/2] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
  2018-04-24  5:13                       ` [PATCH 2/2] unpack_trees_options: free messages when done Martin Ågren
@ 2018-04-24  8:22                       ` Johannes Schindelin
  2 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-24  8:22 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Phillip Wood, git, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Igor Djordjevic, Johannes Sixt,
	Sergey Organov

[-- Attachment #1: Type: text/plain, Size: 927 bytes --]

Hi Martin,

On Tue, 24 Apr 2018, Martin Ågren wrote:

> On 23 April 2018 at 17:54, Phillip Wood <phillip.wood@talktalk.net> wrote:
> > I'm fine with leaving it, I've might get round to doing a small series to
> > clean things up slightly in a few weeks. At the moment
> > setup_unpack_trees_porcelain() leaks memory as it is called for each merge
> > and allocates new strings each time. It would also be nice if the error
> > messages reflected the command, so it said 'cherry-pick', 'revert' or
> > 'reset' rather than 'merge'
> 
> This is a small patch series to introduce and use
> `clear_unpack_trees_porcelain()`.

Great. Now I have no excuse but must change the sequencer code to output
"reset" instead of "merge" ;-)

Seriously speaking again: thank you for those patches. This is truly
exciting! I mean, we all touch the same code and move it forward, and
somehow it all works out.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 08/16] rebase-helper --make-script: introduce a flag to rebase merges
  2018-04-22 13:42               ` Philip Oakley
@ 2018-04-24  8:33                 ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-24  8:33 UTC (permalink / raw)
  To: Philip Oakley
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Hi Philip,

On Sun, 22 Apr 2018, Philip Oakley wrote:

> From: "Johannes Schindelin" <johannes.schindelin@gmx.de>
> 
> Sorry for the very late in the series comments..
> 
> > The sequencer just learned new commands intended to recreate branch
> > structure (similar in spirit to --preserve-merges, but with a
> > substantially less-broken design).
> >
> > Let's allow the rebase--helper to generate todo lists making use of
> > these commands, triggered by the new --rebase-merges option. For a
> > commit topology like this (where the HEAD points to C):
> >
> > - A - B - C
> >     \   /
> >       D
> >
> > the generated todo list would look like this:
> >
> > # branch D
> > pick 0123 A
> > label branch-point
> > pick 1234 D
> > label D
> >
> > reset branch-point
> > pick 2345 B
> > merge -C 3456 D # C
> >
> > To keep things simple, we first only implement support for merge commits
> > with exactly two parents, leaving support for octopus merges to a later
> > patch series.
> >
> For the first time reader this (below) isn't as obvious as may be thought.
> maybe we should be a little more explicit here.
> 
> > As a special, hard-coded label, all merge-rebasing todo lists start with
> > the command `label onto`
> 
> .. which labels the start point head with the name 'onto' ...
> 
> Maybe even:
> "All merge-rebasing todo lists start with, as a convenience, a hard-coded
> `label onto` line which will label the start point's head" ...

I changed it to

    All merge-rebasing todo lists start with a hard-coded `label onto` line.
    This makes it convenient to refer later on to the revision onto which
    everything is rebased, e.g. as starting point for branches other than
    the very first one.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 09/16] rebase: introduce the --rebase-merges option
  2018-04-22 14:15               ` Philip Oakley
  2018-04-24  5:01                 ` Junio C Hamano
@ 2018-04-24  8:40                 ` Johannes Schindelin
  1 sibling, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-24  8:40 UTC (permalink / raw)
  To: Philip Oakley
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Hi Philip,

On Sun, 22 Apr 2018, Philip Oakley wrote:

> From: "Johannes Schindelin" <johannes.schindelin@gmx.de>
> > Once upon a time, this here developer thought: wouldn't it be nice if,
> > say, Git for Windows' patches on top of core Git could be represented as
> > a thicket of branches, and be rebased on top of core Git in order to
> > maintain a cherry-pick'able set of patch series?
> >
> > The original attempt to answer this was: git rebase --preserve-merges.
> >
> > However, that experiment was never intended as an interactive option,
> > and it only piggy-backed on git rebase --interactive because that
> > command's implementation looked already very, very familiar: it was
> > designed by the same person who designed --preserve-merges: yours truly.
> >
> > Some time later, some other developer (I am looking at you, Andreas!
> > ;-)) decided that it would be a good idea to allow --preserve-merges to
> > be combined with --interactive (with caveats!) and the Git maintainer
> > (well, the interim Git maintainer during Junio's absence, that is)
> > agreed, and that is when the glamor of the --preserve-merges design
> > started to fall apart rather quickly and unglamorously.
> >
> > The reason? In --preserve-merges mode, the parents of a merge commit (or
> > for that matter, of *any* commit) were not stated explicitly, but were
> > *implied* by the commit name passed to the `pick` command.
> >
> > This made it impossible, for example, to reorder commits. Not to mention
> > to flatten the branch topology or, deity forbid, to split topic branches
> 
> Aside: The idea of a "flattened" topology is, to my mind, not actually
> defined though may be understood by devs working in the area. Hopefully it's
> going away as a term, though the new 'cousins' will need clarification
> (there's no dot notation for that area of topology).

Right. The point is not actually to talk about "flattening" branches. The
point is to talk about the flexibility one might expect in an
*interactive* rebase, a flexibility notably lacking from the
--preserve-merges mode.

So I changed it to

    This made it impossible, for example, to reorder commits. Not to mention
    to move commits between branches or, deity forbid, to split topic branches
    into two.

> > into two.
> >
> > Alas, these shortcomings also prevented that mode (whose original
> > purpose was to serve Git for Windows' needs, with the additional hope
> > that it may be useful to others, too) from serving Git for Windows'
> > needs.
>
> [... please feel free to save readers time by culling quoted text that
> is irrelevant to your reply...]
>
> > diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> > index 3277ca14327..34e0f6a69c1 100644
> > --- a/Documentation/git-rebase.txt
> > +++ b/Documentation/git-rebase.txt
> > @@ -378,6 +378,23 @@ The commit list format can be changed by setting the
> > configuration option
> > rebase.instructionFormat.  A customized instruction format will
> > automatically
> > have the long commit hash prepended to the format.
> >
> > +-r::
> > +--rebase-merges::
> > + By default, a rebase will simply drop merge commits and only rebase
> > + the non-merge commits. With this option, it will try to preserve
> > + the branching structure within the commits that are to be rebased,
> > + by recreating the merge commits. If a merge commit resolved any merge
> > + or contained manual amendments, then they will have to be re-applied
> > + manually.
> > ++
> > +This mode is similar in spirit to `--preserve-merges`, but in contrast to
> > +that option works well in interactive rebases: commits can be reordered,
> > +inserted and dropped at will.
> > ++
> > +It is currently only possible to recreate the merge commits using the
> > +`recursive` merge strategy; Different merge strategies can be used only
> > via
> > +explicit `exec git merge -s <strategy> [...]` commands.
> > +
> > -p::
> > --preserve-merges::
> >  Recreate merge commits instead of flattening the history by replaying
> 
> Flatten is here in the context lines but its just a blunt statement that 'it
> is what it is'...

Correct. This is where that way of expressing things came from.

I will *not* fix the documentation of `--preserve-merges`, though, as I
hope it can be instead retired soon enough.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 09/16] rebase: introduce the --rebase-merges option
  2018-04-24  5:01                 ` Junio C Hamano
@ 2018-04-24  9:03                   ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-24  9:03 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Philip Oakley, Git List, Jacob Keller, Stefan Beller,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

Hi Junio,

On Tue, 24 Apr 2018, Junio C Hamano wrote:

> "Philip Oakley" <philipoakley@iee.org> writes:
> 
> >> +-r::
> >> +--rebase-merges::
> >> + By default, a rebase will simply drop merge commits and only rebase
> >> + the non-merge commits. With this option, it will try to preserve
> >> + the branching structure within the commits that are to be rebased,
> >> + by recreating the merge commits. If a merge commit resolved any merge

It is funny how nobody caught the missing "conflicts" in "If a merge
commit resolved any merge [conflicts]"...

> >> + or contained manual amendments, then they will have to be re-applied
> >> + manually.
> >> ++
> >> +This mode is similar in spirit to `--preserve-merges`, but in contrast to
> >> +that option works well in interactive rebases: commits can be reordered,
> >> +inserted and dropped at will.
> >> ++
> >> +It is currently only possible to recreate the merge commits using the
> >> +`recursive` merge strategy; Different merge strategies can be used only
> >> via
> >> +explicit `exec git merge -s <strategy> [...]` commands.
> >> +
> >> -p::
> >> --preserve-merges::
> >>  Recreate merge commits instead of flattening the history by replaying
> >
> > Flatten is here in the context lines but its just a blunt statement that 'it
> > is what it is'...
> 
> The first paragraph that explains --rebase-merges talks about what
> happens when the option is not given, and says "drop merge commits
> and only rebase the non-merge commits", which is not incorrect
> per-se but does not make it explicit how the resulting topology
> looks like.

Correct. And it would be the wrong place to describe in detail what a
rebase *without --rebase-merges* does, right?

> I think it is easier to understand if it mentioned "flattening" as well.
> If flatten is not the word you want, perhaps "make it linear" or
> something like that?

I fear that we all here are way too deeply in "Git think". If I ask a
random Git user what it means to "make commits linear", I am sure I would
get only puzzled, nervous looks as a response.

So I am rather certain that the suggested wording is something I want to
avoid.

Besides, we really should expect the reader to know a little about the way
the interactive rebase works by the time they read the explanation of
`--rebase-merges`.

Therefore, I think I can sidestep the entire thing by saying this instead:

-r::
--rebase-merges::
        By default, a rebase will simply drop merge commits from the todo
        list, and put the rebased commits into a single, linear branch.
        With `--rebase-merges`, the rebase will instead try to preserve
        the branching structure within the commits that are to be rebased,
        by recreating the merge commits. Any resolved merge conflicts or
        manual amendments in these merge commits will have to be
        resolved/re-applied manually.

Thank you for helping me improve the documentation part of this patch,
which I think is really, really important,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/2] merge: setup `opts` later in `checkout_fast_forward()`
  2018-04-24  6:20                         ` Jacob Keller
@ 2018-04-24  9:36                           ` Martin Ågren
  2018-04-24 11:30                             ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Martin Ågren @ 2018-04-24  9:36 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Phillip Wood, Git mailing list, Johannes Schindelin,
	Junio C Hamano, Stefan Beller, Philip Oakley, Eric Sunshine,
	Igor Djordjevic, Johannes Sixt, Sergey Organov

On 24 April 2018 at 08:20, Jacob Keller <jacob.keller@gmail.com> wrote:
> I'm guessing the diff algorithm simply found that this was a more
> compact representation of the change? It's a bit confusing when your
> description indicates you "moved" some code down, but it looks like
> you moved code up.

Agreed. I'll play with --anchored and other magic stuff to see if I can
improve this. Or I could instead try to sell this patch as "move some
other stuff out of the way" ;-) That seems a bit less direct though.

Thanks
Martin

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 09/16] rebase: introduce the --rebase-merges option
  2018-04-22 14:37               ` Philip Oakley
@ 2018-04-24 10:52                 ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-24 10:52 UTC (permalink / raw)
  To: Philip Oakley
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Eric Sunshine,
	Phillip Wood, Igor Djordjevic, Johannes Sixt, Sergey Organov,
	Martin Ågren

Hi Philip,

On Sun, 22 Apr 2018, Philip Oakley wrote:

> From: "Johannes Schindelin" <johannes.schindelin@gmx.de>
> > Once upon a time, this here developer thought: wouldn't it be nice if,
> > say, Git for Windows' patches on top of core Git could be represented as
> > a thicket of branches, and be rebased on top of core Git in order to
> > maintain a cherry-pick'able set of patch series?
> >
> > The original attempt to answer this was: git rebase --preserve-merges.
> >
> > However, that experiment was never intended as an interactive option,
> > and it only piggy-backed on git rebase --interactive because that
> > command's implementation looked already very, very familiar: it was
> > designed by the same person who designed --preserve-merges: yours truly.
> >
> > Some time later, some other developer (I am looking at you, Andreas!
> > ;-)) decided that it would be a good idea to allow --preserve-merges to
> > be combined with --interactive (with caveats!) and the Git maintainer
> > (well, the interim Git maintainer during Junio's absence, that is)
> > agreed, and that is when the glamor of the --preserve-merges design
> > started to fall apart rather quickly and unglamorously.
> >
> > The reason? In --preserve-merges mode, the parents of a merge commit (or
> > for that matter, of *any* commit) were not stated explicitly, but were
> > *implied* by the commit name passed to the `pick` command.
> >
> Aside: I think this para should be extracted to the --preserve-merges
> documentation to highlight what it does / why it is 'wrong' (not what would be
> expected in some case). It may also need to discuss the (figurative) Cousins
> vs. Siblings distinction [merge of branches external, or internal, to the
> rebase.

Quite honestly, I'd much rather spend time improving --rebase-merges than
improving --preserve-merges documentation. In my mind, the latter is
pretty useless, especially once the former lands in an official Git
version.

Of course, feel free to disagree with me by sending a patch to improve the
documentation of --preserve-merges ;-)

> "In --preserve-merges, the commit being selected for merging is implied by the
> commit name  passed to the `pick` command (i.e. of the original merge commit),
> not that of the rebased version of that parent."

It is much, much worse:

	In --preserve-merges, no commit can change its ancestry. Every
	rebased commit's parents will be the rebased original parents.

Or some such. But really, why bother describing something *that* broken?
Why not work toward a solution that makes that broken option obsolete?
Like, say, --rebase-merges? ;-)

> A similar issue occurs with (figuratively) '--ancestry-path --first parent'
> searches which lacks the alternate '--lead parent' post-walk selection. [1]. I
> don't think there is a dot notation to select the merge cousins, nor merge
> siblings either A.,B ? (that's dot-comma ;-)

I actually had missed `--ancestry-path`... I should probably use it in the
description of the "cousins".

> [... lots of quoted text...]

Could I ask you to make it easier for me by cutting quoted text that is
irrelevant to your reply? The way I read mails forces me to scroll down
(sometimes on a phone) all the way to the end, just to find that that time
was spent in vain.

Thanks,
Dscho



^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 1/2] merge: setup `opts` later in `checkout_fast_forward()`
  2018-04-24  9:36                           ` Martin Ågren
@ 2018-04-24 11:30                             ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-24 11:30 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Jacob Keller, Phillip Wood, Git mailing list, Junio C Hamano,
	Stefan Beller, Philip Oakley, Eric Sunshine, Igor Djordjevic,
	Johannes Sixt, Sergey Organov

[-- Attachment #1: Type: text/plain, Size: 747 bytes --]

Hi Martin,

On Tue, 24 Apr 2018, Martin Ågren wrote:

> On 24 April 2018 at 08:20, Jacob Keller <jacob.keller@gmail.com> wrote:
> > I'm guessing the diff algorithm simply found that this was a more
> > compact representation of the change? It's a bit confusing when your
> > description indicates you "moved" some code down, but it looks like
> > you moved code up.
> 
> Agreed. I'll play with --anchored and other magic stuff to see if I can
> improve this. Or I could instead try to sell this patch as "move some
> other stuff out of the way" ;-) That seems a bit less direct though.

Or you could add a remark to the commit message along the lines "best
viewed with `--anchored=...`". This is what I would do ;-)

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/2] unpack_trees_options: free messages when done
  2018-04-24  5:13                       ` [PATCH 2/2] unpack_trees_options: free messages when done Martin Ågren
@ 2018-04-24 16:29                         ` Elijah Newren
  2018-04-28 11:32                           ` Martin Ågren
  0 siblings, 1 reply; 412+ messages in thread
From: Elijah Newren @ 2018-04-24 16:29 UTC (permalink / raw)
  To: martin.agren
  Cc: Johannes.Schindelin, git, gitster, igor.d.djordjevic, j6t,
	jacob.keller, philipoakley, phillip.wood, sbeller, sorganov,
	sunshine, Elijah Newren

On Mon, Apr 23, 2018 at 10:13 PM, Martin Ågren <martin.agren@gmail.com> wrote:
> The strings allocated in `setup_unpack_trees_porcelain()` are never
> freed. Provide a function `clear_unpack_trees_porcelain()` to do so and
> call it in the functions which use `setup_unpack_trees_porcelain()`.

This is awesome; thanks.

> diff --git a/merge-recursive.c b/merge-recursive.c
> index 0c0d48624d..8229b91e2f 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -301,6 +301,7 @@ static int git_merge_trees(int index_only,
>         init_tree_desc_from_tree(t+2, merge);
>
>         rc = unpack_trees(3, t, &opts);
> +       clear_unpack_trees_porcelain(&opts);
>         cache_tree_free(&active_cache_tree);
>         return rc;

Yeah, this could result in an evil merge.  In my series, I want to
continue to be able to call verify_uptodate() from unpack_trees.c in order
to check if files affected by renames are dirty and we need to avoid
overwriting them.  That can trigger error messages, so they need to not be
freed until later.  So, instead, I'd like to see something like the below
(built on top of my series):

-- >8 --

---
 merge-recursive.c | 25 ++++++++++++-------------
 1 file changed, 12 insertions(+), 13 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index f2cbad4f10..3491a27bf2 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -337,10 +337,10 @@ static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
 	init_tree_desc(desc, tree->buffer, tree->size);
 }
 
-static int git_merge_trees(struct merge_options *o,
-			   struct tree *common,
-			   struct tree *head,
-			   struct tree *merge)
+static int unpack_trees_start(struct merge_options *o,
+			      struct tree *common,
+			      struct tree *head,
+			      struct tree *merge)
 {
 	int rc;
 	struct tree_desc t[3];
@@ -378,6 +378,12 @@ static int git_merge_trees(struct merge_options *o,
 	return rc;
 }
 
+static void unpack_trees_finish(struct merge_options *o)
+{
+	discard_index(&o->orig_index);
+	clear_unpack_trees_porcelain(&o->unpack_opts);
+}
+
 struct tree *write_tree_from_memory(struct merge_options *o)
 {
 	struct tree *result = NULL;
@@ -3079,7 +3085,7 @@ int merge_trees(struct merge_options *o,
 		return 1;
 	}
 
-	code = git_merge_trees(o, common, head, merge);
+	code = unpack_trees_start(o, common, head, merge);
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
@@ -3144,14 +3150,7 @@ int merge_trees(struct merge_options *o,
 	else
 		clean = 1;
 
-	/* Free the extra index left from git_merge_trees() */
-	/*
-	 * FIXME: Need to also free data allocated by
-	 * setup_unpack_trees_porcelain() tucked away in o->unpack_opts.msgs,
-	 * but the problem is that only half of it refers to dynamically
-	 * allocated data, while the other half points at static strings.
-	 */
-	discard_index(&o->orig_index);
+	unpack_trees_finish(o);
 
 	if (o->call_depth && !(*result = write_tree_from_memory(o)))
 		return -1;
-- 
2.17.0.295.g791b7256b2.dirty


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
  2018-04-24  8:11                     ` Johannes Schindelin
@ 2018-04-24 19:41                       ` Philip Oakley
  0 siblings, 0 replies; 412+ messages in thread
From: Philip Oakley @ 2018-04-24 19:41 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Git List, Junio C Hamano, Jacob Keller, Stefan Beller,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Sergey Organov, Martin Ågren

From: "Johannes Schindelin" <Johannes.Schindelin@gmx.de>
> On Mon, 23 Apr 2018, Philip Oakley wrote:
>
>> From: "Johannes Schindelin" <Johannes.Schindelin@gmx.de> : Monday, April 
>> 23,
>> 2018 1:03 PM
>> Subject: Re: [PATCH v8 06/16] sequencer: introduce the `merge` command
>>
>> [...]
>> >
>> > > > label onto
>> > > >
>> > > > # Branch abc
>> > > > reset onto
>> > >
>> > > Is this reset strictly necessary. We are already there @head.
>> >
>> > No, this is not strictly necessary, but
>>
>> I've realised my misunderstanding. I was thinking this (and others) was
>> equivalent to
>>
>> $  git reset <thatHead'onto'> # maybe even --hard,
>>
>> i.e. affecting the worktree
>
> Oh, but it *is* affecting the worktree. In this case, since we label HEAD
> and then immediately reset to the label, there is just nothing to change.
>
> Consider this example, though:
>
> label onto
>
> # Branch: from-philip
> reset onto
> pick abcdef something
> label from-philip
>
> # Branch: with-love
> reset onto
> pick 012345 else
> label with-love
>
> reset onto
> merge -C 98765 from-philip
> merge -C 43210 with-love
>
> Only in the first instance is the `reset onto` a no-op, an incidental one.
> After picking `something` and labeling the result as `from-philip`,
> though, the next `reset onto` really resets the worktree.
>
>> rather that just being a movement of the Head rev (though I may be having
>> brain fade here regarding untracked files etc..)
>
> The current way of doing things does not allow the `reset` to overwrite
> untracked, nor ignored files (I think, I only verified the former, not the
> latter).
>
> But yeah, it is not just a movement of HEAD. It does reset the worktree,
> although quite a bit more gently (and safely) than `git reset --hard`. In
> that respect, this patch series is a drastic improvement over the Git
> garden shears (which is the shell script I use in Git for Windows which
> inspired this here patch series).
>
thanks for clarifying. Yes my reasoning  was a total brain fade ... Along 
with the fact that it's a soft/safe/gentle reset.
--
Philip 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges
  2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                               ` (15 preceding siblings ...)
  2018-04-21 11:09             ` [PATCH v8 16/16] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
@ 2018-04-25 12:28             ` Johannes Schindelin
  2018-04-25 12:28               ` [PATCH v9 01/17] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
                                 ` (18 more replies)
  16 siblings, 19 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:28 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

Junio, I think this is now ready for `next`. Thank you for your patience
and help with this.

Once upon a time, I dreamed of an interactive rebase that would not
linearize all patches and drop all merge commits, but instead recreate
the commit topology faithfully.

My original attempt was --preserve-merges, but that design was so
limited that I did not even enable it in interactive mode.

Subsequently, it *was* enabled in interactive mode, with the predictable
consequences: as the --preserve-merges design does not allow for
specifying the parents of merge commits explicitly, all the new commits'
parents are defined *implicitly* by the previous commit history, and
hence it is *not possible to even reorder commits*.

This design flaw cannot be fixed. Not without a complete re-design, at
least. This patch series offers such a re-design.

Think of --rebase-merges as "--preserve-merges done right". It
introduces new verbs for the todo list, `label`, `reset` and `merge`.
For a commit topology like this:

            A - B - C
              \   /
                D

the generated todo list would look like this:

            # branch D
            pick 0123 A
            label branch-point
            pick 1234 D
            label D

            reset branch-point
            pick 2345 B
            merge -C 3456 D # C

There are more patches in the pipeline, based on this patch series, but
left for later in the interest of reviewable patch series: one mini
series to use the sequencer even for `git rebase -i --root`, and another
one to add support for octopus merges to --rebase-merges. And then one
to allow for rebasing merge commits in a smarter way (this one will need
a bit more work, though, as it can result in very complicated, nested
merge conflicts *very* easily).

Changes since v8:

- Disentangled the patch introducing `label`/`reset` from the one
  introducing `merge` again (this was one stupid, tired `git commit
  --amend` too many).

- Augmented the commit message of "introduce the `merge` command" to
  describe what the `label onto` is all about.

- Fixed the error message when `reset` would overwrite untracked files to
  actually say that a "reset" failed (not a "merge").

- Clarified the rationale for `label onto` in the commit message of
  "rebase-helper --make-script: introduce a flag to rebase merges".

- Edited the description of `--rebase-merges` heavily, for clarity, in
  "rebase: introduce the --rebase-merges option".

- Edited the commit message of (and the documentation introduced by) " rebase
  -i: introduce --rebase-merges=[no-]rebase-cousins" for clarity (also
  mentioning the `--ancestry-path` option).

- When run_git_commit() fails after a successful merge, we now take pains
  not to reschedule the `merge` command.

- Rebased the patch series on top of current `master`, i.e. both
  `pw/rebase-keep-empty-fixes` and `pw/rebase-signoff`, to resolve merge
  conflicts myself.


Johannes Schindelin (15):
  sequencer: avoid using errno clobbered by rollback_lock_file()
  sequencer: make rearrange_squash() a bit more obvious
  sequencer: refactor how original todo list lines are accessed
  sequencer: offer helpful advice when a command was rescheduled
  sequencer: introduce new commands to reset the revision
  sequencer: introduce the `merge` command
  sequencer: fast-forward `merge` commands, if possible
  rebase-helper --make-script: introduce a flag to rebase merges
  rebase: introduce the --rebase-merges option
  sequencer: make refs generated by the `label` command worktree-local
  sequencer: handle post-rewrite for merge commands
  rebase --rebase-merges: avoid "empty merges"
  pull: accept --rebase=merges to recreate the branch topology
  rebase -i: introduce --rebase-merges=[no-]rebase-cousins
  rebase -i --rebase-merges: add a section to the man page

Phillip Wood (1):
  rebase --rebase-merges: add test for --keep-empty

Stefan Beller (1):
  git-rebase--interactive: clarify arguments

 Documentation/config.txt               |   8 +
 Documentation/git-pull.txt             |   6 +-
 Documentation/git-rebase.txt           | 163 ++++-
 builtin/pull.c                         |  14 +-
 builtin/rebase--helper.c               |  13 +-
 builtin/remote.c                       |  18 +-
 contrib/completion/git-completion.bash |   4 +-
 git-rebase--interactive.sh             |  22 +-
 git-rebase.sh                          |  16 +
 refs.c                                 |   3 +-
 sequencer.c                            | 892 ++++++++++++++++++++++++-
 sequencer.h                            |   7 +
 t/t3421-rebase-topology-linear.sh      |   1 +
 t/t3430-rebase-merges.sh               | 244 +++++++
 14 files changed, 1352 insertions(+), 59 deletions(-)
 create mode 100755 t/t3430-rebase-merges.sh


base-commit: 1f1cddd558b54bb0ce19c8ace353fd07b758510d
Published-As: https://github.com/dscho/git/releases/tag/recreate-merges-v9
Fetch-It-Via: git fetch https://github.com/dscho/git recreate-merges-v9

Interdiff vs v8:
 diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
 index e691b93e920..bd5ecff980e 100644
 --- a/Documentation/git-rebase.txt
 +++ b/Documentation/git-rebase.txt
 @@ -381,21 +381,24 @@ have the long commit hash prepended to the format.
  
  -r::
  --rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
 -	By default, a rebase will simply drop merge commits and only rebase
 -	the non-merge commits. With this option, it will try to preserve
 +	By default, a rebase will simply drop merge commits from the todo
 +	list, and put the rebased commits into a single, linear branch.
 +	With `--rebase-merges`, the rebase will instead try to preserve
  	the branching structure within the commits that are to be rebased,
 -	by recreating the merge commits. If a merge commit resolved any merge
 -	or contained manual amendments, then they will have to be re-applied
 -	manually.
 +	by recreating the merge commits. Any resolved merge conflicts or
 +	manual amendments in these merge commits will have to be
 +	resolved/re-applied manually.
  +
  By default, or when `no-rebase-cousins` was specified, commits which do not
 -have `<upstream>` as direct ancestor will keep their original branch point.
 -If the `rebase-cousins` mode is turned on, such commits are instead rebased
 +have `<upstream>` as direct ancestor will keep their original branch point,
 +i.e. commits that would be excluded by gitlink:git-log[1]'s
 +`--ancestry-path` option will keep their original ancestry by default. If
 +the `rebase-cousins` mode is turned on, such commits are instead rebased
  onto `<upstream>` (or `<onto>`, if specified).
  +
 -This mode is similar in spirit to `--preserve-merges`, but in contrast to
 -that option works well in interactive rebases: commits can be reordered,
 -inserted and dropped at will.
 +The `--rebase-merges` mode is similar in spirit to `--preserve-merges`, but
 +in contrast to that option works well in interactive rebases: commits can be
 +reordered, inserted and dropped at will.
  +
  It is currently only possible to recreate the merge commits using the
  `recursive` merge strategy; Different merge strategies can be used only via
 diff --git a/sequencer.c b/sequencer.c
 index b5715f69450..e2f83942843 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -2635,6 +2635,7 @@ static int do_reset(const char *name, int len, struct replay_opts *opts)
  	}
  
  	memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
 +	setup_unpack_trees_porcelain(&unpack_tree_opts, "reset");
  	unpack_tree_opts.head_idx = 1;
  	unpack_tree_opts.src_index = &the_index;
  	unpack_tree_opts.dst_index = &the_index;
 @@ -2855,7 +2856,12 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
  	if (ret)
  		rerere(opts->allow_rerere_auto);
  	else
 -		ret = run_git_commit(git_path_merge_msg(), opts,
 +		/*
 +		 * In case of problems, we now want to return a positive
 +		 * value (a negative one would indicate that the `merge`
 +		 * command needs to be rescheduled).
 +		 */
 +		ret = !!run_git_commit(git_path_merge_msg(), opts,
  				     run_commit_flags);
  
  leave_merge:
 @@ -3809,12 +3815,9 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
  
  	init_revisions(&revs, NULL);
  	revs.verbose_header = 1;
 -	if (rebase_merges)
 -		revs.cherry_mark = 1;
 -	else {
 +	if (!rebase_merges)
  		revs.max_parents = 1;
 -		revs.cherry_pick = 1;
 -	}
 +	revs.cherry_mark = 1;
  	revs.limited = 1;
  	revs.reverse = 1;
  	revs.right_only = 1;
-- 
2.17.0.windows.1.33.gfcbb1fa0445


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v9 01/17] sequencer: avoid using errno clobbered by rollback_lock_file()
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
@ 2018-04-25 12:28               ` Johannes Schindelin
  2018-04-25 12:28               ` [PATCH v9 02/17] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
                                 ` (17 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:28 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

As pointed out in a review of the `--rebase-merges` patch series,
`rollback_lock_file()` clobbers errno. Therefore, we have to report the
error message that uses errno before calling said function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 5e3a50fafc9..674e26bf826 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -346,12 +346,14 @@ static int write_message(const void *buf, size_t len, const char *filename,
 	if (msg_fd < 0)
 		return error_errno(_("could not lock '%s'"), filename);
 	if (write_in_full(msg_fd, buf, len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write to '%s'"), filename);
+		return -1;
 	}
 	if (append_eol && write(msg_fd, "\n", 1) < 0) {
+		error_errno(_("could not write eol to '%s'"), filename);
 		rollback_lock_file(&msg_file);
-		return error_errno(_("could not write eol to '%s'"), filename);
+		return -1;
 	}
 	if (commit_lock_file(&msg_file) < 0)
 		return error(_("failed to finalize '%s'"), filename);
@@ -2125,9 +2127,9 @@ static int save_head(const char *head)
 	written = write_in_full(fd, buf.buf, buf.len);
 	strbuf_release(&buf);
 	if (written < 0) {
+		error_errno(_("could not write to '%s'"), git_path_head_file());
 		rollback_lock_file(&head_lock);
-		return error_errno(_("could not write to '%s'"),
-				   git_path_head_file());
+		return -1;
 	}
 	if (commit_lock_file(&head_lock) < 0)
 		return error(_("failed to finalize '%s'"), git_path_head_file());
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 02/17] sequencer: make rearrange_squash() a bit more obvious
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
  2018-04-25 12:28               ` [PATCH v9 01/17] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
@ 2018-04-25 12:28               ` Johannes Schindelin
  2018-04-25 12:28               ` [PATCH v9 03/17] sequencer: refactor how original todo list lines are accessed Johannes Schindelin
                                 ` (16 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:28 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

There are some commands that have to be skipped from rearranging by virtue
of not handling any commits.

However, the logic was not quite obvious: it skipped commands based on
their position in the enum todo_command.

Instead, let's make it explicit that we skip all commands that do not
handle any commit. With one exception: the `drop` command, because it,
well, drops the commit and is therefore not eligible to rearranging.

Note: this is a bit academic at the moment because the only time we call
`rearrange_squash()` is directly after generating the todo list, when we
have nothing but `pick` commands anyway.

However, the upcoming `merge` command *will* want to be handled by that
function, and it *can* handle commits.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 674e26bf826..c131e39fa93 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3403,7 +3403,7 @@ int rearrange_squash(void)
 		struct subject2item_entry *entry;
 
 		next[i] = tail[i] = -1;
-		if (item->command >= TODO_EXEC) {
+		if (!item->commit || item->command == TODO_DROP) {
 			subjects[i] = NULL;
 			continue;
 		}
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 03/17] sequencer: refactor how original todo list lines are accessed
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
  2018-04-25 12:28               ` [PATCH v9 01/17] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
  2018-04-25 12:28               ` [PATCH v9 02/17] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
@ 2018-04-25 12:28               ` Johannes Schindelin
  2018-04-25 12:28               ` [PATCH v9 04/17] sequencer: offer helpful advice when a command was rescheduled Johannes Schindelin
                                 ` (15 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:28 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

Previously, we did a lot of arithmetic gymnastics to get at the line in
the todo list (as stored in todo_list.buf). This might have been fast,
but only in terms of execution speed, not in terms of developer time.

Let's refactor this to make it a lot easier to read, and hence to
reason about the correctness of the code. It is not performance-critical
code anyway.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 60 ++++++++++++++++++++++++++++++++---------------------
 1 file changed, 36 insertions(+), 24 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index c131e39fa93..eac1c341c1c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1871,6 +1871,23 @@ static int count_commands(struct todo_list *todo_list)
 	return count;
 }
 
+static int get_item_line_offset(struct todo_list *todo_list, int index)
+{
+	return index < todo_list->nr ?
+		todo_list->items[index].offset_in_buf : todo_list->buf.len;
+}
+
+static const char *get_item_line(struct todo_list *todo_list, int index)
+{
+	return todo_list->buf.buf + get_item_line_offset(todo_list, index);
+}
+
+static int get_item_line_length(struct todo_list *todo_list, int index)
+{
+	return get_item_line_offset(todo_list, index + 1)
+		-  get_item_line_offset(todo_list, index);
+}
+
 static ssize_t strbuf_read_file_or_whine(struct strbuf *sb, const char *path)
 {
 	int fd;
@@ -2250,29 +2267,27 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
 		return error_errno(_("could not lock '%s'"), todo_path);
-	offset = next < todo_list->nr ?
-		todo_list->items[next].offset_in_buf : todo_list->buf.len;
+	offset = get_item_line_offset(todo_list, next);
 	if (write_in_full(fd, todo_list->buf.buf + offset,
 			todo_list->buf.len - offset) < 0)
 		return error_errno(_("could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
 		return error(_("failed to finalize '%s'"), todo_path);
 
-	if (is_rebase_i(opts)) {
-		const char *done_path = rebase_path_done();
-		int fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
-		int prev_offset = !next ? 0 :
-			todo_list->items[next - 1].offset_in_buf;
+	if (is_rebase_i(opts) && next > 0) {
+		const char *done = rebase_path_done();
+		int fd = open(done, O_CREAT | O_WRONLY | O_APPEND, 0666);
+		int ret = 0;
 
-		if (fd >= 0 && offset > prev_offset &&
-		    write_in_full(fd, todo_list->buf.buf + prev_offset,
-				  offset - prev_offset) < 0) {
-			close(fd);
-			return error_errno(_("could not write to '%s'"),
-					   done_path);
-		}
-		if (fd >= 0)
-			close(fd);
+		if (fd < 0)
+			return 0;
+		if (write_in_full(fd, get_item_line(todo_list, next - 1),
+				  get_item_line_length(todo_list, next - 1))
+		    < 0)
+			ret = error_errno(_("could not write to '%s'"), done);
+		if (close(fd) < 0)
+			ret = error_errno(_("failed to finalize '%s'"), done);
+		return ret;
 	}
 	return 0;
 }
@@ -3307,8 +3322,7 @@ int skip_unnecessary_picks(void)
 		oid = &item->commit->object.oid;
 	}
 	if (i > 0) {
-		int offset = i < todo_list.nr ?
-			todo_list.items[i].offset_in_buf : todo_list.buf.len;
+		int offset = get_item_line_offset(&todo_list, i);
 		const char *done_path = rebase_path_done();
 
 		fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
@@ -3488,12 +3502,10 @@ int rearrange_squash(void)
 				continue;
 
 			while (cur >= 0) {
-				int offset = todo_list.items[cur].offset_in_buf;
-				int end_offset = cur + 1 < todo_list.nr ?
-					todo_list.items[cur + 1].offset_in_buf :
-					todo_list.buf.len;
-				char *bol = todo_list.buf.buf + offset;
-				char *eol = todo_list.buf.buf + end_offset;
+				const char *bol =
+					get_item_line(&todo_list, cur);
+				const char *eol =
+					get_item_line(&todo_list, cur + 1);
 
 				/* replace 'pick', by 'fixup' or 'squash' */
 				command = todo_list.items[cur].command;
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 04/17] sequencer: offer helpful advice when a command was rescheduled
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (2 preceding siblings ...)
  2018-04-25 12:28               ` [PATCH v9 03/17] sequencer: refactor how original todo list lines are accessed Johannes Schindelin
@ 2018-04-25 12:28               ` Johannes Schindelin
  2018-04-25 12:28               ` [PATCH v9 05/17] git-rebase--interactive: clarify arguments Johannes Schindelin
                                 ` (14 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:28 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

Previously, we did that just magically, and potentially left some users
quite puzzled. Let's err on the safe side instead, telling the user what
is happening, and how they are supposed to continue.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index eac1c341c1c..f9c1ddb5385 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2561,6 +2561,17 @@ static const char *reflog_message(struct replay_opts *opts,
 	return buf.buf;
 }
 
+static const char rescheduled_advice[] =
+N_("Could not execute the todo command\n"
+"\n"
+"    %.*s"
+"\n"
+"It has been rescheduled; To edit the command before continuing, please\n"
+"edit the todo list first:\n"
+"\n"
+"    git rebase --edit-todo\n"
+"    git rebase --continue\n");
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -2606,6 +2617,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					opts, is_final_fixup(todo_list));
 			if (is_rebase_i(opts) && res < 0) {
 				/* Reschedule */
+				advise(_(rescheduled_advice),
+				       get_item_line_length(todo_list,
+							    todo_list->current),
+				       get_item_line(todo_list,
+						     todo_list->current));
 				todo_list->current--;
 				if (save_todo(todo_list, opts))
 					return -1;
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 05/17] git-rebase--interactive: clarify arguments
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (3 preceding siblings ...)
  2018-04-25 12:28               ` [PATCH v9 04/17] sequencer: offer helpful advice when a command was rescheduled Johannes Schindelin
@ 2018-04-25 12:28               ` Johannes Schindelin
  2018-04-25 12:28               ` [PATCH v9 06/17] sequencer: introduce new commands to reset the revision Johannes Schindelin
                                 ` (13 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:28 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Stefan Beller, Junio C Hamano, Jacob Keller,
	Stefan Beller, Philip Oakley, Eric Sunshine, Phillip Wood,
	Igor Djordjevic, Johannes Sixt, Sergey Organov,
	Martin Ågren

From: Stefan Beller <stefanbeller@gmail.com>

Up to now each command took a commit as its first argument and ignored
the rest of the line (usually the subject of the commit)

Now that we are about to introduce commands that take different
arguments, clarify each command by giving the argument list.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 9947e6265fe..a60df2ee5a0 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -155,13 +155,13 @@ reschedule_last_action () {
 append_todo_help () {
 	gettext "
 Commands:
-p, pick = use commit
-r, reword = use commit, but edit the commit message
-e, edit = use commit, but stop for amending
-s, squash = use commit, but meld into previous commit
-f, fixup = like \"squash\", but discard this commit's log message
-x, exec = run command (the rest of the line) using shell
-d, drop = remove commit
+p, pick <commit> = use commit
+r, reword <commit> = use commit, but edit the commit message
+e, edit <commit> = use commit, but stop for amending
+s, squash <commit> = use commit, but meld into previous commit
+f, fixup <commit> = like \"squash\", but discard this commit's log message
+x, exec <commit> = run command (the rest of the line) using shell
+d, drop <commit> = remove commit
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 06/17] sequencer: introduce new commands to reset the revision
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (4 preceding siblings ...)
  2018-04-25 12:28               ` [PATCH v9 05/17] git-rebase--interactive: clarify arguments Johannes Schindelin
@ 2018-04-25 12:28               ` Johannes Schindelin
  2018-04-25 12:28               ` [PATCH v9 07/17] sequencer: introduce the `merge` command Johannes Schindelin
                                 ` (12 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:28 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

In the upcoming commits, we will teach the sequencer to rebase merges.
This will be done in a very different way from the unfortunate design of
`git rebase --preserve-merges` (which does not allow for reordering
commits, or changing the branch topology).

The main idea is to introduce new todo list commands, to support
labeling the current revision with a given name, resetting the current
revision to a previous state, and  merging labeled revisions.

This idea was developed in Git for Windows' Git garden shears (that are
used to maintain Git for Windows' "thicket of branches" on top of
upstream Git), and this patch is part of the effort to make it available
to a wider audience, as well as to make the entire process more robust
(by implementing it in a safe and portable language rather than a Unix
shell script).

This commit implements the commands to label, and to reset to, given
revisions. The syntax is:

	label <name>
	reset <name>

Internally, the `label <name>` command creates the ref
`refs/rewritten/<name>`. This makes it possible to work with the labeled
revisions interactively, or in a scripted fashion (e.g. via the todo
list command `exec`).

These temporary refs are removed upon sequencer_remove_state(), so that
even a `git rebase --abort` cleans them up.

We disallow '#' as label because that character will be used as separator
in the upcoming `merge` command.

Later in this patch series, we will mark the `refs/rewritten/` refs as
worktree-local, to allow for interactive rebases to be run in parallel in
worktrees linked to the same repository.

As typos happen, a failed `label` or `reset` command will be rescheduled
immediately. As the previous code to reschedule a command is embedded
deeply in the pick/fixup/squash code path, we simply duplicate the few
lines. This will allow us to extend the new code path easily for the
upcoming `merge` command.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   2 +
 sequencer.c                | 213 +++++++++++++++++++++++++++++++++++--
 2 files changed, 208 insertions(+), 7 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index a60df2ee5a0..d6e8958dae4 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -162,6 +162,8 @@ s, squash <commit> = use commit, but meld into previous commit
 f, fixup <commit> = like \"squash\", but discard this commit's log message
 x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
+l, label <label> = label current HEAD with a name
+t, reset <label> = reset HEAD to a label
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index f9c1ddb5385..c9655edffa5 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -23,6 +23,8 @@
 #include "hashmap.h"
 #include "notes-utils.h"
 #include "sigchain.h"
+#include "unpack-trees.h"
+#include "worktree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -120,6 +122,13 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
 static GIT_PATH_FUNC(rebase_path_rewritten_pending,
 	"rebase-merge/rewritten-pending")
+
+/*
+ * The path of the file listing refs that need to be deleted after the rebase
+ * finishes. This is used by the `label` command to record the need for cleanup.
+ */
+static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete")
+
 /*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
@@ -245,18 +254,34 @@ static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
 
 int sequencer_remove_state(struct replay_opts *opts)
 {
-	struct strbuf dir = STRBUF_INIT;
+	struct strbuf buf = STRBUF_INIT;
 	int i;
 
+	if (is_rebase_i(opts) &&
+	    strbuf_read_file(&buf, rebase_path_refs_to_delete(), 0) > 0) {
+		char *p = buf.buf;
+		while (*p) {
+			char *eol = strchr(p, '\n');
+			if (eol)
+				*eol = '\0';
+			if (delete_ref("(rebase -i) cleanup", p, NULL, 0) < 0)
+				warning(_("could not delete '%s'"), p);
+			if (!eol)
+				break;
+			p = eol + 1;
+		}
+	}
+
 	free(opts->gpg_sign);
 	free(opts->strategy);
 	for (i = 0; i < opts->xopts_nr; i++)
 		free(opts->xopts[i]);
 	free(opts->xopts);
 
-	strbuf_addstr(&dir, get_dir(opts));
-	remove_dir_recursively(&dir, 0);
-	strbuf_release(&dir);
+	strbuf_reset(&buf);
+	strbuf_addstr(&buf, get_dir(opts));
+	remove_dir_recursively(&buf, 0);
+	strbuf_release(&buf);
 
 	return 0;
 }
@@ -1280,6 +1305,8 @@ enum todo_command {
 	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
+	TODO_LABEL,
+	TODO_RESET,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -1298,6 +1325,8 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
+	{ 'l', "label" },
+	{ 't', "reset" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1803,7 +1832,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return error(_("missing arguments for %s"),
 			     command_to_string(item->command));
 
-	if (item->command == TODO_EXEC) {
+	if (item->command == TODO_EXEC || item->command == TODO_LABEL ||
+	    item->command == TODO_RESET) {
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = (int)(eol - bol);
@@ -2471,6 +2501,159 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int safe_append(const char *filename, const char *fmt, ...)
+{
+	va_list ap;
+	struct lock_file lock = LOCK_INIT;
+	int fd = hold_lock_file_for_update(&lock, filename,
+					   LOCK_REPORT_ON_ERROR);
+	struct strbuf buf = STRBUF_INIT;
+
+	if (fd < 0)
+		return -1;
+
+	if (strbuf_read_file(&buf, filename, 0) < 0 && errno != ENOENT) {
+		error_errno(_("could not read '%s'"), filename);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	strbuf_complete(&buf, '\n');
+	va_start(ap, fmt);
+	strbuf_vaddf(&buf, fmt, ap);
+	va_end(ap);
+
+	if (write_in_full(fd, buf.buf, buf.len) < 0) {
+		error_errno(_("could not write to '%s'"), filename);
+		strbuf_release(&buf);
+		rollback_lock_file(&lock);
+		return -1;
+	}
+	if (commit_lock_file(&lock) < 0) {
+		strbuf_release(&buf);
+		rollback_lock_file(&lock);
+		return error(_("failed to finalize '%s'"), filename);
+	}
+
+	strbuf_release(&buf);
+	return 0;
+}
+
+static int do_label(const char *name, int len)
+{
+	struct ref_store *refs = get_main_ref_store();
+	struct ref_transaction *transaction;
+	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
+	struct strbuf msg = STRBUF_INIT;
+	int ret = 0;
+	struct object_id head_oid;
+
+	if (len == 1 && *name == '#')
+		return error("Illegal label name: '%.*s'", len, name);
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	strbuf_addf(&msg, "rebase -i (label) '%.*s'", len, name);
+
+	transaction = ref_store_transaction_begin(refs, &err);
+	if (!transaction) {
+		error("%s", err.buf);
+		ret = -1;
+	} else if (get_oid("HEAD", &head_oid)) {
+		error(_("could not read HEAD"));
+		ret = -1;
+	} else if (ref_transaction_update(transaction, ref_name.buf, &head_oid,
+					  NULL, 0, msg.buf, &err) < 0 ||
+		   ref_transaction_commit(transaction, &err)) {
+		error("%s", err.buf);
+		ret = -1;
+	}
+	ref_transaction_free(transaction);
+	strbuf_release(&err);
+	strbuf_release(&msg);
+
+	if (!ret)
+		ret = safe_append(rebase_path_refs_to_delete(),
+				  "%s\n", ref_name.buf);
+	strbuf_release(&ref_name);
+
+	return ret;
+}
+
+static const char *reflog_message(struct replay_opts *opts,
+	const char *sub_action, const char *fmt, ...);
+
+static int do_reset(const char *name, int len, struct replay_opts *opts)
+{
+	struct strbuf ref_name = STRBUF_INIT;
+	struct object_id oid;
+	struct lock_file lock = LOCK_INIT;
+	struct tree_desc desc;
+	struct tree *tree;
+	struct unpack_trees_options unpack_tree_opts;
+	int ret = 0, i;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0)
+		return -1;
+
+	/* Determine the length of the label */
+	for (i = 0; i < len; i++)
+		if (isspace(name[i]))
+			len = i;
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", len, name);
+	if (get_oid(ref_name.buf, &oid) &&
+	    get_oid(ref_name.buf + strlen("refs/rewritten/"), &oid)) {
+		error(_("could not read '%s'"), ref_name.buf);
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
+	setup_unpack_trees_porcelain(&unpack_tree_opts, "reset");
+	unpack_tree_opts.head_idx = 1;
+	unpack_tree_opts.src_index = &the_index;
+	unpack_tree_opts.dst_index = &the_index;
+	unpack_tree_opts.fn = oneway_merge;
+	unpack_tree_opts.merge = 1;
+	unpack_tree_opts.update = 1;
+
+	if (read_cache_unmerged()) {
+		rollback_lock_file(&lock);
+		strbuf_release(&ref_name);
+		return error_resolve_conflict(_(action_name(opts)));
+	}
+
+	if (!fill_tree_descriptor(&desc, &oid)) {
+		error(_("failed to find tree of %s"), oid_to_hex(&oid));
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	if (unpack_trees(1, &desc, &unpack_tree_opts)) {
+		rollback_lock_file(&lock);
+		free((void *)desc.buffer);
+		strbuf_release(&ref_name);
+		return -1;
+	}
+
+	tree = parse_tree_indirect(&oid);
+	prime_cache_tree(&the_index, tree);
+
+	if (write_locked_index(&the_index, &lock, COMMIT_LOCK) < 0)
+		ret = error(_("could not write index"));
+	free((void *)desc.buffer);
+
+	if (!ret)
+		ret = update_ref(reflog_message(opts, "reset", "'%.*s'",
+						len, name), "HEAD", &oid,
+				 NULL, 0, UPDATE_REFS_MSG_ON_ERR);
+
+	strbuf_release(&ref_name);
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2574,7 +2757,7 @@ N_("Could not execute the todo command\n"
 
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
-	int res = 0;
+	int res = 0, reschedule = 0;
 
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	if (opts->allow_ff)
@@ -2645,7 +2828,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					intend_to_amend();
 				return error_failed_squash(item->commit, opts,
 					item->arg_len, item->arg);
-			} else if (res && is_rebase_i(opts))
+			} else if (res && is_rebase_i(opts) && item->commit)
 				return res | error_with_patch(item->commit,
 					item->arg, item->arg_len, opts, res,
 					item->command == TODO_REWORD);
@@ -2671,9 +2854,25 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				/* `current` will be incremented below */
 				todo_list->current = -1;
 			}
+		} else if (item->command == TODO_LABEL) {
+			if ((res = do_label(item->arg, item->arg_len)))
+				reschedule = 1;
+		} else if (item->command == TODO_RESET) {
+			if ((res = do_reset(item->arg, item->arg_len, opts)))
+				reschedule = 1;
 		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
+		if (reschedule) {
+			advise(_(rescheduled_advice),
+			       get_item_line_length(todo_list,
+						    todo_list->current),
+			       get_item_line(todo_list, todo_list->current));
+			todo_list->current--;
+			if (save_todo(todo_list, opts))
+				return -1;
+		}
+
 		todo_list->current++;
 		if (res)
 			return res;
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 07/17] sequencer: introduce the `merge` command
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (5 preceding siblings ...)
  2018-04-25 12:28               ` [PATCH v9 06/17] sequencer: introduce new commands to reset the revision Johannes Schindelin
@ 2018-04-25 12:28               ` Johannes Schindelin
  2018-04-25 12:28               ` [PATCH v9 08/17] sequencer: fast-forward `merge` commands, if possible Johannes Schindelin
                                 ` (11 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:28 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

This patch is part of the effort to reimplement `--preserve-merges` with
a substantially improved design, a design that has been developed in the
Git for Windows project to maintain the dozens of Windows-specific patch
series on top of upstream Git.

The previous patch implemented the `label` and `reset` commands to label
commits and to reset to labeled commits. This patch adds the `merge`
command, with the following syntax:

	merge [-C <commit>] <rev> # <oneline>

The <commit> parameter in this instance is the *original* merge commit,
whose author and message will be used for the merge commit that is about
to be created.

The <rev> parameter refers to the (possibly rewritten) revision to
merge. Let's see an example of a todo list (the initial `label onto`
command is an auto-generated convenience so that the label `onto` can be
used to refer to the revision onto which we rebase):

	label onto

	# Branch abc
	reset onto
	pick deadbeef Hello, world!
	label abc

	reset onto
	pick cafecafe And now for something completely different
	merge -C baaabaaa abc # Merge the branch 'abc' into master

To edit the merge commit's message (a "reword" for merges, if you will),
use `-c` (lower-case) instead of `-C`; this convention was borrowed from
`git commit` that also supports `-c` and `-C` with similar meanings.

To create *new* merges, i.e. without copying the commit message from an
existing commit, simply omit the `-C <commit>` parameter (which will
open an editor for the merge message):

	merge abc

This comes in handy when splitting a branch into two or more branches.

Note: this patch only adds support for recursive merges, to keep things
simple. Support for octopus merges will be added later in a separate
patch series, support for merges using strategies other than the
recursive merge is left for the future.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-rebase--interactive.sh |   4 +
 sequencer.c                | 200 +++++++++++++++++++++++++++++++++++++
 2 files changed, 204 insertions(+)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index d6e8958dae4..acb4bfd3fc8 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -164,6 +164,10 @@ x, exec <commit> = run command (the rest of the line) using shell
 d, drop <commit> = remove commit
 l, label <label> = label current HEAD with a name
 t, reset <label> = reset HEAD to a label
+m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]
+.       create a merge commit using the original merge commit's
+.       message (or the oneline, if no original merge commit was
+.       specified). Use -c <commit> to reword the commit message.
 
 These lines can be re-ordered; they are executed from top to bottom.
 " | git stripspace --comment-lines >>"$todo"
diff --git a/sequencer.c b/sequencer.c
index c9655edffa5..94f4831a0c3 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1307,6 +1307,7 @@ enum todo_command {
 	TODO_EXEC,
 	TODO_LABEL,
 	TODO_RESET,
+	TODO_MERGE,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
 	TODO_DROP,
@@ -1327,6 +1328,7 @@ static struct {
 	{ 'x', "exec" },
 	{ 'l', "label" },
 	{ 't', "reset" },
+	{ 'm', "merge" },
 	{ 0,   "noop" },
 	{ 'd', "drop" },
 	{ 0,   NULL }
@@ -1754,9 +1756,14 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 	return 0;
 }
 
+enum todo_item_flags {
+	TODO_EDIT_MERGE_MSG = 1
+};
+
 struct todo_item {
 	enum todo_command command;
 	struct commit *commit;
+	unsigned int flags;
 	const char *arg;
 	int arg_len;
 	size_t offset_in_buf;
@@ -1791,6 +1798,8 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	char *end_of_object_name;
 	int i, saved, status, padding;
 
+	item->flags = 0;
+
 	/* left-trim */
 	bol += strspn(bol, " \t");
 
@@ -1840,6 +1849,21 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return 0;
 	}
 
+	if (item->command == TODO_MERGE) {
+		if (skip_prefix(bol, "-C", &bol))
+			bol += strspn(bol, " \t");
+		else if (skip_prefix(bol, "-c", &bol)) {
+			bol += strspn(bol, " \t");
+			item->flags |= TODO_EDIT_MERGE_MSG;
+		} else {
+			item->flags |= TODO_EDIT_MERGE_MSG;
+			item->commit = NULL;
+			item->arg = bol;
+			item->arg_len = (int)(eol - bol);
+			return 0;
+		}
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
@@ -2654,6 +2678,158 @@ static int do_reset(const char *name, int len, struct replay_opts *opts)
 	return ret;
 }
 
+static int do_merge(struct commit *commit, const char *arg, int arg_len,
+		    int flags, struct replay_opts *opts)
+{
+	int run_commit_flags = (flags & TODO_EDIT_MERGE_MSG) ?
+		EDIT_MSG | VERIFY_MSG : 0;
+	struct strbuf ref_name = STRBUF_INIT;
+	struct commit *head_commit, *merge_commit, *i;
+	struct commit_list *bases, *j, *reversed = NULL;
+	struct merge_options o;
+	int merge_arg_len, oneline_offset, ret;
+	static struct lock_file lock;
+	const char *p;
+
+	if (hold_locked_index(&lock, LOCK_REPORT_ON_ERROR) < 0) {
+		ret = -1;
+		goto leave_merge;
+	}
+
+	head_commit = lookup_commit_reference_by_name("HEAD");
+	if (!head_commit) {
+		ret = error(_("cannot merge without a current revision"));
+		goto leave_merge;
+	}
+
+	oneline_offset = arg_len;
+	merge_arg_len = strcspn(arg, " \t\n");
+	p = arg + merge_arg_len;
+	p += strspn(p, " \t\n");
+	if (*p == '#' && (!p[1] || isspace(p[1]))) {
+		p += 1 + strspn(p + 1, " \t\n");
+		oneline_offset = p - arg;
+	} else if (p - arg < arg_len)
+		BUG("octopus merges are not supported yet: '%s'", p);
+
+	strbuf_addf(&ref_name, "refs/rewritten/%.*s", merge_arg_len, arg);
+	merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	if (!merge_commit) {
+		/* fall back to non-rewritten ref or commit */
+		strbuf_splice(&ref_name, 0, strlen("refs/rewritten/"), "", 0);
+		merge_commit = lookup_commit_reference_by_name(ref_name.buf);
+	}
+
+	if (!merge_commit) {
+		ret = error(_("could not resolve '%s'"), ref_name.buf);
+		goto leave_merge;
+	}
+
+	if (commit) {
+		const char *message = get_commit_buffer(commit, NULL);
+		const char *body;
+		int len;
+
+		if (!message) {
+			ret = error(_("could not get commit message of '%s'"),
+				    oid_to_hex(&commit->object.oid));
+			goto leave_merge;
+		}
+		write_author_script(message);
+		find_commit_subject(message, &body);
+		len = strlen(body);
+		ret = write_message(body, len, git_path_merge_msg(), 0);
+		unuse_commit_buffer(commit, message);
+		if (ret) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			goto leave_merge;
+		}
+	} else {
+		struct strbuf buf = STRBUF_INIT;
+		int len;
+
+		strbuf_addf(&buf, "author %s", git_author_info(0));
+		write_author_script(buf.buf);
+		strbuf_reset(&buf);
+
+		if (oneline_offset < arg_len) {
+			p = arg + oneline_offset;
+			len = arg_len - oneline_offset;
+		} else {
+			strbuf_addf(&buf, "Merge branch '%.*s'",
+				    merge_arg_len, arg);
+			p = buf.buf;
+			len = buf.len;
+		}
+
+		ret = write_message(p, len, git_path_merge_msg(), 0);
+		strbuf_release(&buf);
+		if (ret) {
+			error_errno(_("could not write '%s'"),
+				    git_path_merge_msg());
+			goto leave_merge;
+		}
+	}
+
+	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
+		      git_path_merge_head(), 0);
+	write_message("no-ff", 5, git_path_merge_mode(), 0);
+
+	bases = get_merge_bases(head_commit, merge_commit);
+	for (j = bases; j; j = j->next)
+		commit_list_insert(j->item, &reversed);
+	free_commit_list(bases);
+
+	read_cache();
+	init_merge_options(&o);
+	o.branch1 = "HEAD";
+	o.branch2 = ref_name.buf;
+	o.buffer_output = 2;
+
+	ret = merge_recursive(&o, head_commit, merge_commit, reversed, &i);
+	if (ret <= 0)
+		fputs(o.obuf.buf, stdout);
+	strbuf_release(&o.obuf);
+	if (ret < 0) {
+		error(_("could not even attempt to merge '%.*s'"),
+		      merge_arg_len, arg);
+		goto leave_merge;
+	}
+	/*
+	 * The return value of merge_recursive() is 1 on clean, and 0 on
+	 * unclean merge.
+	 *
+	 * Let's reverse that, so that do_merge() returns 0 upon success and
+	 * 1 upon failed merge (keeping the return value -1 for the cases where
+	 * we will want to reschedule the `merge` command).
+	 */
+	ret = !ret;
+
+	if (active_cache_changed &&
+	    write_locked_index(&the_index, &lock, COMMIT_LOCK)) {
+		ret = error(_("merge: Unable to write new index file"));
+		goto leave_merge;
+	}
+
+	rollback_lock_file(&lock);
+	if (ret)
+		rerere(opts->allow_rerere_auto);
+	else
+		/*
+		 * In case of problems, we now want to return a positive
+		 * value (a negative one would indicate that the `merge`
+		 * command needs to be rescheduled).
+		 */
+		ret = !!run_git_commit(git_path_merge_msg(), opts,
+				     run_commit_flags);
+
+leave_merge:
+	strbuf_release(&ref_name);
+	rollback_lock_file(&lock);
+	return ret;
+}
+
 static int is_final_fixup(struct todo_list *todo_list)
 {
 	int i = todo_list->current;
@@ -2860,6 +3036,17 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		} else if (item->command == TODO_RESET) {
 			if ((res = do_reset(item->arg, item->arg_len, opts)))
 				reschedule = 1;
+		} else if (item->command == TODO_MERGE) {
+			if ((res = do_merge(item->commit,
+					    item->arg, item->arg_len,
+					    item->flags, opts)) < 0)
+				reschedule = 1;
+			else if (res > 0)
+				/* failed with merge conflicts */
+				return error_with_patch(item->commit,
+							item->arg,
+							item->arg_len, opts,
+							res, 0);
 		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
@@ -2871,6 +3058,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			todo_list->current--;
 			if (save_todo(todo_list, opts))
 				return -1;
+			if (item->commit)
+				return error_with_patch(item->commit,
+							item->arg,
+							item->arg_len, opts,
+							res, 0);
 		}
 
 		todo_list->current++;
@@ -3356,8 +3548,16 @@ int transform_todos(unsigned flags)
 					  short_commit_name(item->commit) :
 					  oid_to_hex(&item->commit->object.oid);
 
+			if (item->command == TODO_MERGE) {
+				if (item->flags & TODO_EDIT_MERGE_MSG)
+					strbuf_addstr(&buf, " -c");
+				else
+					strbuf_addstr(&buf, " -C");
+			}
+
 			strbuf_addf(&buf, " %s", oid);
 		}
+
 		/* add all the rest */
 		if (!item->arg_len)
 			strbuf_addch(&buf, '\n');
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 08/17] sequencer: fast-forward `merge` commands, if possible
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (6 preceding siblings ...)
  2018-04-25 12:28               ` [PATCH v9 07/17] sequencer: introduce the `merge` command Johannes Schindelin
@ 2018-04-25 12:28               ` Johannes Schindelin
  2018-04-25 12:29               ` [PATCH v9 09/17] rebase-helper --make-script: introduce a flag to rebase merges Johannes Schindelin
                                 ` (10 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:28 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

Just like with regular `pick` commands, if we are trying to rebase a
merge commit, we now test whether the parents of said commit match HEAD
and the commits to be merged, and fast-forward if possible.

This is not only faster, but also avoids unnecessary proliferation of
new objects.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 ++++++++++++++++++++++++++++++++-
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 94f4831a0c3..6722095655d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2687,7 +2687,7 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	struct commit *head_commit, *merge_commit, *i;
 	struct commit_list *bases, *j, *reversed = NULL;
 	struct merge_options o;
-	int merge_arg_len, oneline_offset, ret;
+	int merge_arg_len, oneline_offset, can_fast_forward, ret;
 	static struct lock_file lock;
 	const char *p;
 
@@ -2772,6 +2772,37 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 		}
 	}
 
+	/*
+	 * If HEAD is not identical to the first parent of the original merge
+	 * commit, we cannot fast-forward.
+	 */
+	can_fast_forward = opts->allow_ff && commit && commit->parents &&
+		!oidcmp(&commit->parents->item->object.oid,
+			&head_commit->object.oid);
+
+	/*
+	 * If the merge head is different from the original one, we cannot
+	 * fast-forward.
+	 */
+	if (can_fast_forward) {
+		struct commit_list *second_parent = commit->parents->next;
+
+		if (second_parent && !second_parent->next &&
+		    oidcmp(&merge_commit->object.oid,
+			   &second_parent->item->object.oid))
+			can_fast_forward = 0;
+	}
+
+	if (can_fast_forward && commit->parents->next &&
+	    !commit->parents->next->next &&
+	    !oidcmp(&commit->parents->next->item->object.oid,
+		    &merge_commit->object.oid)) {
+		rollback_lock_file(&lock);
+		ret = fast_forward_to(&commit->object.oid,
+				      &head_commit->object.oid, 0, opts);
+		goto leave_merge;
+	}
+
 	write_message(oid_to_hex(&merge_commit->object.oid), GIT_SHA1_HEXSZ,
 		      git_path_merge_head(), 0);
 	write_message("no-ff", 5, git_path_merge_mode(), 0);
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 09/17] rebase-helper --make-script: introduce a flag to rebase merges
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (7 preceding siblings ...)
  2018-04-25 12:28               ` [PATCH v9 08/17] sequencer: fast-forward `merge` commands, if possible Johannes Schindelin
@ 2018-04-25 12:29               ` Johannes Schindelin
  2018-04-25 12:29               ` [PATCH v9 10/17] rebase: introduce the --rebase-merges option Johannes Schindelin
                                 ` (9 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:29 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

The sequencer just learned new commands intended to recreate branch
structure (similar in spirit to --preserve-merges, but with a
substantially less-broken design).

Let's allow the rebase--helper to generate todo lists making use of
these commands, triggered by the new --rebase-merges option. For a
commit topology like this (where the HEAD points to C):

	- A - B - C
	    \   /
	      D

the generated todo list would look like this:

	# branch D
	pick 0123 A
	label branch-point
	pick 1234 D
	label D

	reset branch-point
	pick 2345 B
	merge -C 3456 D # C

To keep things simple, we first only implement support for merge commits
with exactly two parents, leaving support for octopus merges to a later
patch series.

All merge-rebasing todo lists start with a hard-coded `label onto` line.
This makes it convenient to refer later on to the revision onto which
everything is rebased, e.g. as starting point for branches other than
the very first one.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/rebase--helper.c |   4 +-
 sequencer.c              | 346 ++++++++++++++++++++++++++++++++++++++-
 sequencer.h              |   1 +
 3 files changed, 349 insertions(+), 2 deletions(-)

diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index ad074705bb5..781782e7272 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -12,7 +12,7 @@ static const char * const builtin_rebase_helper_usage[] = {
 int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
-	unsigned flags = 0, keep_empty = 0;
+	unsigned flags = 0, keep_empty = 0, rebase_merges = 0;
 	int abbreviate_commands = 0;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
@@ -24,6 +24,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "keep-empty", &keep_empty, N_("keep empty commits")),
 		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
 			N_("allow commits with empty messages")),
+		OPT_BOOL(0, "rebase-merges", &rebase_merges, N_("rebase merge commits")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -57,6 +58,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
+	flags |= rebase_merges ? TODO_LIST_REBASE_MERGES : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
 	if (command == CONTINUE && argc == 1)
diff --git a/sequencer.c b/sequencer.c
index 6722095655d..e9297122633 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -25,6 +25,8 @@
 #include "sigchain.h"
 #include "unpack-trees.h"
 #include "worktree.h"
+#include "oidmap.h"
+#include "oidset.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -3448,6 +3450,343 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
 	strbuf_release(&sob);
 }
 
+struct labels_entry {
+	struct hashmap_entry entry;
+	char label[FLEX_ARRAY];
+};
+
+static int labels_cmp(const void *fndata, const struct labels_entry *a,
+		      const struct labels_entry *b, const void *key)
+{
+	return key ? strcmp(a->label, key) : strcmp(a->label, b->label);
+}
+
+struct string_entry {
+	struct oidmap_entry entry;
+	char string[FLEX_ARRAY];
+};
+
+struct label_state {
+	struct oidmap commit2label;
+	struct hashmap labels;
+	struct strbuf buf;
+};
+
+static const char *label_oid(struct object_id *oid, const char *label,
+			     struct label_state *state)
+{
+	struct labels_entry *labels_entry;
+	struct string_entry *string_entry;
+	struct object_id dummy;
+	size_t len;
+	int i;
+
+	string_entry = oidmap_get(&state->commit2label, oid);
+	if (string_entry)
+		return string_entry->string;
+
+	/*
+	 * For "uninteresting" commits, i.e. commits that are not to be
+	 * rebased, and which can therefore not be labeled, we use a unique
+	 * abbreviation of the commit name. This is slightly more complicated
+	 * than calling find_unique_abbrev() because we also need to make
+	 * sure that the abbreviation does not conflict with any other
+	 * label.
+	 *
+	 * We disallow "interesting" commits to be labeled by a string that
+	 * is a valid full-length hash, to ensure that we always can find an
+	 * abbreviation for any uninteresting commit's names that does not
+	 * clash with any other label.
+	 */
+	if (!label) {
+		char *p;
+
+		strbuf_reset(&state->buf);
+		strbuf_grow(&state->buf, GIT_SHA1_HEXSZ);
+		label = p = state->buf.buf;
+
+		find_unique_abbrev_r(p, oid, default_abbrev);
+
+		/*
+		 * We may need to extend the abbreviated hash so that there is
+		 * no conflicting label.
+		 */
+		if (hashmap_get_from_hash(&state->labels, strihash(p), p)) {
+			size_t i = strlen(p) + 1;
+
+			oid_to_hex_r(p, oid);
+			for (; i < GIT_SHA1_HEXSZ; i++) {
+				char save = p[i];
+				p[i] = '\0';
+				if (!hashmap_get_from_hash(&state->labels,
+							   strihash(p), p))
+					break;
+				p[i] = save;
+			}
+		}
+	} else if (((len = strlen(label)) == GIT_SHA1_RAWSZ &&
+		    !get_oid_hex(label, &dummy)) ||
+		   (len == 1 && *label == '#') ||
+		   hashmap_get_from_hash(&state->labels,
+					 strihash(label), label)) {
+		/*
+		 * If the label already exists, or if the label is a valid full
+		 * OID, or the label is a '#' (which we use as a separator
+		 * between merge heads and oneline), we append a dash and a
+		 * number to make it unique.
+		 */
+		struct strbuf *buf = &state->buf;
+
+		strbuf_reset(buf);
+		strbuf_add(buf, label, len);
+
+		for (i = 2; ; i++) {
+			strbuf_setlen(buf, len);
+			strbuf_addf(buf, "-%d", i);
+			if (!hashmap_get_from_hash(&state->labels,
+						   strihash(buf->buf),
+						   buf->buf))
+				break;
+		}
+
+		label = buf->buf;
+	}
+
+	FLEX_ALLOC_STR(labels_entry, label, label);
+	hashmap_entry_init(labels_entry, strihash(label));
+	hashmap_add(&state->labels, labels_entry);
+
+	FLEX_ALLOC_STR(string_entry, string, label);
+	oidcpy(&string_entry->entry.oid, oid);
+	oidmap_put(&state->commit2label, string_entry);
+
+	return string_entry->string;
+}
+
+static int make_script_with_merges(struct pretty_print_context *pp,
+				   struct rev_info *revs, FILE *out,
+				   unsigned flags)
+{
+	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
+	struct strbuf label = STRBUF_INIT;
+	struct commit_list *commits = NULL, **tail = &commits, *iter;
+	struct commit_list *tips = NULL, **tips_tail = &tips;
+	struct commit *commit;
+	struct oidmap commit2todo = OIDMAP_INIT;
+	struct string_entry *entry;
+	struct oidset interesting = OIDSET_INIT, child_seen = OIDSET_INIT,
+		shown = OIDSET_INIT;
+	struct label_state state = { OIDMAP_INIT, { NULL }, STRBUF_INIT };
+
+	int abbr = flags & TODO_LIST_ABBREVIATE_CMDS;
+	const char *cmd_pick = abbr ? "p" : "pick",
+		*cmd_label = abbr ? "l" : "label",
+		*cmd_reset = abbr ? "t" : "reset",
+		*cmd_merge = abbr ? "m" : "merge";
+
+	oidmap_init(&commit2todo, 0);
+	oidmap_init(&state.commit2label, 0);
+	hashmap_init(&state.labels, (hashmap_cmp_fn) labels_cmp, NULL, 0);
+	strbuf_init(&state.buf, 32);
+
+	if (revs->cmdline.nr && (revs->cmdline.rev[0].flags & BOTTOM)) {
+		struct object_id *oid = &revs->cmdline.rev[0].item->oid;
+		FLEX_ALLOC_STR(entry, string, "onto");
+		oidcpy(&entry->entry.oid, oid);
+		oidmap_put(&state.commit2label, entry);
+	}
+
+	/*
+	 * First phase:
+	 * - get onelines for all commits
+	 * - gather all branch tips (i.e. 2nd or later parents of merges)
+	 * - label all branch tips
+	 */
+	while ((commit = get_revision(revs))) {
+		struct commit_list *to_merge;
+		int is_octopus;
+		const char *p1, *p2;
+		struct object_id *oid;
+		int is_empty;
+
+		tail = &commit_list_insert(commit, tail)->next;
+		oidset_insert(&interesting, &commit->object.oid);
+
+		is_empty = is_original_commit_empty(commit);
+		if (!is_empty && (commit->object.flags & PATCHSAME))
+			continue;
+
+		strbuf_reset(&oneline);
+		pretty_print_commit(pp, commit, &oneline);
+
+		to_merge = commit->parents ? commit->parents->next : NULL;
+		if (!to_merge) {
+			/* non-merge commit: easy case */
+			strbuf_reset(&buf);
+			if (!keep_empty && is_empty)
+				strbuf_addf(&buf, "%c ", comment_line_char);
+			strbuf_addf(&buf, "%s %s %s", cmd_pick,
+				    oid_to_hex(&commit->object.oid),
+				    oneline.buf);
+
+			FLEX_ALLOC_STR(entry, string, buf.buf);
+			oidcpy(&entry->entry.oid, &commit->object.oid);
+			oidmap_put(&commit2todo, entry);
+
+			continue;
+		}
+
+		is_octopus = to_merge && to_merge->next;
+
+		if (is_octopus)
+			BUG("Octopus merges not yet supported");
+
+		/* Create a label */
+		strbuf_reset(&label);
+		if (skip_prefix(oneline.buf, "Merge ", &p1) &&
+		    (p1 = strchr(p1, '\'')) &&
+		    (p2 = strchr(++p1, '\'')))
+			strbuf_add(&label, p1, p2 - p1);
+		else if (skip_prefix(oneline.buf, "Merge pull request ",
+				     &p1) &&
+			 (p1 = strstr(p1, " from ")))
+			strbuf_addstr(&label, p1 + strlen(" from "));
+		else
+			strbuf_addbuf(&label, &oneline);
+
+		for (p1 = label.buf; *p1; p1++)
+			if (isspace(*p1))
+				*(char *)p1 = '-';
+
+		strbuf_reset(&buf);
+		strbuf_addf(&buf, "%s -C %s",
+			    cmd_merge, oid_to_hex(&commit->object.oid));
+
+		/* label the tip of merged branch */
+		oid = &to_merge->item->object.oid;
+		strbuf_addch(&buf, ' ');
+
+		if (!oidset_contains(&interesting, oid))
+			strbuf_addstr(&buf, label_oid(oid, NULL, &state));
+		else {
+			tips_tail = &commit_list_insert(to_merge->item,
+							tips_tail)->next;
+
+			strbuf_addstr(&buf, label_oid(oid, label.buf, &state));
+		}
+		strbuf_addf(&buf, " # %s", oneline.buf);
+
+		FLEX_ALLOC_STR(entry, string, buf.buf);
+		oidcpy(&entry->entry.oid, &commit->object.oid);
+		oidmap_put(&commit2todo, entry);
+	}
+
+	/*
+	 * Second phase:
+	 * - label branch points
+	 * - add HEAD to the branch tips
+	 */
+	for (iter = commits; iter; iter = iter->next) {
+		struct commit_list *parent = iter->item->parents;
+		for (; parent; parent = parent->next) {
+			struct object_id *oid = &parent->item->object.oid;
+			if (!oidset_contains(&interesting, oid))
+				continue;
+			if (!oidset_contains(&child_seen, oid))
+				oidset_insert(&child_seen, oid);
+			else
+				label_oid(oid, "branch-point", &state);
+		}
+
+		/* Add HEAD as implict "tip of branch" */
+		if (!iter->next)
+			tips_tail = &commit_list_insert(iter->item,
+							tips_tail)->next;
+	}
+
+	/*
+	 * Third phase: output the todo list. This is a bit tricky, as we
+	 * want to avoid jumping back and forth between revisions. To
+	 * accomplish that goal, we walk backwards from the branch tips,
+	 * gathering commits not yet shown, reversing the list on the fly,
+	 * then outputting that list (labeling revisions as needed).
+	 */
+	fprintf(out, "%s onto\n", cmd_label);
+	for (iter = tips; iter; iter = iter->next) {
+		struct commit_list *list = NULL, *iter2;
+
+		commit = iter->item;
+		if (oidset_contains(&shown, &commit->object.oid))
+			continue;
+		entry = oidmap_get(&state.commit2label, &commit->object.oid);
+
+		if (entry)
+			fprintf(out, "\n# Branch %s\n", entry->string);
+		else
+			fprintf(out, "\n");
+
+		while (oidset_contains(&interesting, &commit->object.oid) &&
+		       !oidset_contains(&shown, &commit->object.oid)) {
+			commit_list_insert(commit, &list);
+			if (!commit->parents) {
+				commit = NULL;
+				break;
+			}
+			commit = commit->parents->item;
+		}
+
+		if (!commit)
+			fprintf(out, "%s onto\n", cmd_reset);
+		else {
+			const char *to = NULL;
+
+			entry = oidmap_get(&state.commit2label,
+					   &commit->object.oid);
+			if (entry)
+				to = entry->string;
+
+			if (!to || !strcmp(to, "onto"))
+				fprintf(out, "%s onto\n", cmd_reset);
+			else {
+				strbuf_reset(&oneline);
+				pretty_print_commit(pp, commit, &oneline);
+				fprintf(out, "%s %s # %s\n",
+					cmd_reset, to, oneline.buf);
+			}
+		}
+
+		for (iter2 = list; iter2; iter2 = iter2->next) {
+			struct object_id *oid = &iter2->item->object.oid;
+			entry = oidmap_get(&commit2todo, oid);
+			/* only show if not already upstream */
+			if (entry)
+				fprintf(out, "%s\n", entry->string);
+			entry = oidmap_get(&state.commit2label, oid);
+			if (entry)
+				fprintf(out, "%s %s\n",
+					cmd_label, entry->string);
+			oidset_insert(&shown, oid);
+		}
+
+		free_commit_list(list);
+	}
+
+	free_commit_list(commits);
+	free_commit_list(tips);
+
+	strbuf_release(&label);
+	strbuf_release(&oneline);
+	strbuf_release(&buf);
+
+	oidmap_free(&commit2todo, 1);
+	oidmap_free(&state.commit2label, 1);
+	hashmap_free(&state.labels, 1);
+	strbuf_release(&state.buf);
+
+	return 0;
+}
+
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags)
 {
@@ -3458,10 +3797,12 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	struct commit *commit;
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
 	const char *insn = flags & TODO_LIST_ABBREVIATE_CMDS ? "p" : "pick";
+	int rebase_merges = flags & TODO_LIST_REBASE_MERGES;
 
 	init_revisions(&revs, NULL);
 	revs.verbose_header = 1;
-	revs.max_parents = 1;
+	if (!rebase_merges)
+		revs.max_parents = 1;
 	revs.cherry_mark = 1;
 	revs.limited = 1;
 	revs.reverse = 1;
@@ -3486,6 +3827,9 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
 	if (prepare_revision_walk(&revs) < 0)
 		return error(_("make_script: error preparing revisions"));
 
+	if (rebase_merges)
+		return make_script_with_merges(&pp, &revs, out, flags);
+
 	while ((commit = get_revision(&revs))) {
 		int is_empty  = is_original_commit_empty(commit);
 
diff --git a/sequencer.h b/sequencer.h
index e45b178dfc4..6bc4da17243 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -59,6 +59,7 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_KEEP_EMPTY (1U << 0)
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
+#define TODO_LIST_REBASE_MERGES (1U << 3)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 10/17] rebase: introduce the --rebase-merges option
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (8 preceding siblings ...)
  2018-04-25 12:29               ` [PATCH v9 09/17] rebase-helper --make-script: introduce a flag to rebase merges Johannes Schindelin
@ 2018-04-25 12:29               ` Johannes Schindelin
  2018-04-25 12:29               ` [PATCH v9 11/17] rebase --rebase-merges: add test for --keep-empty Johannes Schindelin
                                 ` (8 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:29 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

Once upon a time, this here developer thought: wouldn't it be nice if,
say, Git for Windows' patches on top of core Git could be represented as
a thicket of branches, and be rebased on top of core Git in order to
maintain a cherry-pick'able set of patch series?

The original attempt to answer this was: git rebase --preserve-merges.

However, that experiment was never intended as an interactive option,
and it only piggy-backed on git rebase --interactive because that
command's implementation looked already very, very familiar: it was
designed by the same person who designed --preserve-merges: yours truly.

Some time later, some other developer (I am looking at you, Andreas!
;-)) decided that it would be a good idea to allow --preserve-merges to
be combined with --interactive (with caveats!) and the Git maintainer
(well, the interim Git maintainer during Junio's absence, that is)
agreed, and that is when the glamor of the --preserve-merges design
started to fall apart rather quickly and unglamorously.

The reason? In --preserve-merges mode, the parents of a merge commit (or
for that matter, of *any* commit) were not stated explicitly, but were
*implied* by the commit name passed to the `pick` command.

This made it impossible, for example, to reorder commits. Not to mention
to move commits between branches or, deity forbid, to split topic branches
into two.

Alas, these shortcomings also prevented that mode (whose original
purpose was to serve Git for Windows' needs, with the additional hope
that it may be useful to others, too) from serving Git for Windows'
needs.

Five years later, when it became really untenable to have one unwieldy,
big hodge-podge patch series of partly related, partly unrelated patches
in Git for Windows that was rebased onto core Git's tags from time to
time (earning the undeserved wrath of the developer of the ill-fated
git-remote-hg series that first obsoleted Git for Windows' competing
approach, only to be abandoned without maintainer later) was really
untenable, the "Git garden shears" were born [*1*/*2*]: a script,
piggy-backing on top of the interactive rebase, that would first
determine the branch topology of the patches to be rebased, create a
pseudo todo list for further editing, transform the result into a real
todo list (making heavy use of the `exec` command to "implement" the
missing todo list commands) and finally recreate the patch series on
top of the new base commit.

That was in 2013. And it took about three weeks to come up with the
design and implement it as an out-of-tree script. Needless to say, the
implementation needed quite a few years to stabilize, all the while the
design itself proved itself sound.

With this patch, the goodness of the Git garden shears comes to `git
rebase -i` itself. Passing the `--rebase-merges` option will generate
a todo list that can be understood readily, and where it is obvious
how to reorder commits. New branches can be introduced by inserting
`label` commands and calling `merge <label>`. And once this mode will
have become stable and universally accepted, we can deprecate the design
mistake that was `--preserve-merges`.

Link *1*:
https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
Link *2*:
https://github.com/git-for-windows/build-extra/blob/master/shears.sh

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt           |  21 ++-
 contrib/completion/git-completion.bash |   2 +-
 git-rebase--interactive.sh             |   1 +
 git-rebase.sh                          |   6 +
 t/t3430-rebase-merges.sh               | 179 +++++++++++++++++++++++++
 5 files changed, 207 insertions(+), 2 deletions(-)
 create mode 100755 t/t3430-rebase-merges.sh

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index dd852068b1d..7f1756f1eba 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -379,6 +379,24 @@ The commit list format can be changed by setting the configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
+-r::
+--rebase-merges::
+	By default, a rebase will simply drop merge commits from the todo
+	list, and put the rebased commits into a single, linear branch.
+	With `--rebase-merges`, the rebase will instead try to preserve
+	the branching structure within the commits that are to be rebased,
+	by recreating the merge commits. Any resolved merge conflicts or
+	manual amendments in these merge commits will have to be
+	resolved/re-applied manually.
++
+This mode is similar in spirit to `--preserve-merges`, but in contrast to
+that option works well in interactive rebases: commits can be reordered,
+inserted and dropped at will.
++
+It is currently only possible to recreate the merge commits using the
+`recursive` merge strategy; Different merge strategies can be used only via
+explicit `exec git merge -s <strategy> [...]` commands.
+
 -p::
 --preserve-merges::
 	Recreate merge commits instead of flattening the history by replaying
@@ -781,7 +799,8 @@ BUGS
 The todo list presented by `--preserve-merges --interactive` does not
 represent the topology of the revision graph.  Editing commits and
 rewording their commit messages should work fine, but attempts to
-reorder commits tend to produce counterintuitive results.
+reorder commits tend to produce counterintuitive results. Use
+`--rebase-merges` in such scenarios instead.
 
 For example, an attempt to rearrange
 ------------
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 01dd9ff07a2..e6469004099 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1944,7 +1944,7 @@ _git_rebase ()
 	--*)
 		__gitcomp "
 			--onto --merge --strategy --interactive
-			--preserve-merges --stat --no-stat
+			--rebase-merges --preserve-merges --stat --no-stat
 			--committer-date-is-author-date --ignore-date
 			--ignore-whitespace --whitespace=
 			--autosquash --no-autosquash
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index acb4bfd3fc8..e29da634339 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -970,6 +970,7 @@ git_rebase__interactive () {
 	init_revisions_and_shortrevisions
 
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
+		${rebase_merges:+--rebase-merges} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 
diff --git a/git-rebase.sh b/git-rebase.sh
index ded5de085a8..a553f969d11 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,6 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
+r,rebase-merges!   try to rebase merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -89,6 +90,7 @@ type=
 state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
+rebase_merges=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -280,6 +282,10 @@ do
 	--no-keep-empty)
 		keep_empty=
 		;;
+	--rebase-merges)
+		rebase_merges=t
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
new file mode 100755
index 00000000000..5f0febb9970
--- /dev/null
+++ b/t/t3430-rebase-merges.sh
@@ -0,0 +1,179 @@
+#!/bin/sh
+#
+# Copyright (c) 2018 Johannes E. Schindelin
+#
+
+test_description='git rebase -i --rebase-merges
+
+This test runs git rebase "interactively", retaining the branch structure by
+recreating merge commits.
+
+Initial setup:
+
+    -- B --                   (first)
+   /       \
+ A - C - D - E - H            (master)
+       \       /
+         F - G                (second)
+'
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
+
+test_cmp_graph () {
+	cat >expect &&
+	git log --graph --boundary --format=%s "$@" >output &&
+	sed "s/ *$//" <output >output.trimmed &&
+	test_cmp expect output.trimmed
+}
+
+test_expect_success 'setup' '
+	write_script replace-editor.sh <<-\EOF &&
+	mv "$1" "$(git rev-parse --git-path ORIGINAL-TODO)"
+	cp script-from-scratch "$1"
+	EOF
+
+	test_commit A &&
+	git checkout -b first &&
+	test_commit B &&
+	git checkout master &&
+	test_commit C &&
+	test_commit D &&
+	git merge --no-commit B &&
+	test_tick &&
+	git commit -m E &&
+	git tag -m E E &&
+	git checkout -b second C &&
+	test_commit F &&
+	test_commit G &&
+	git checkout master &&
+	git merge --no-commit G &&
+	test_tick &&
+	git commit -m H &&
+	git tag -m H H
+'
+
+test_expect_success 'create completely different structure' '
+	cat >script-from-scratch <<-\EOF &&
+	label onto
+
+	# onebranch
+	pick G
+	pick D
+	label onebranch
+
+	# second
+	reset onto
+	pick B
+	label second
+
+	reset onto
+	merge -C H second
+	merge onebranch # Merge the topic branch '\''onebranch'\''
+	EOF
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_tick &&
+	git rebase -i -r A &&
+	test_cmp_graph <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	* |   H
+	|\ \
+	| |/
+	|/|
+	| * B
+	|/
+	* A
+	EOF
+'
+
+test_expect_success 'generate correct todo list' '
+	cat >expect <<-\EOF &&
+	label onto
+
+	reset onto
+	pick d9df450 B
+	label E
+
+	reset onto
+	pick 5dee784 C
+	label branch-point
+	pick ca2c861 F
+	pick 088b00a G
+	label H
+
+	reset branch-point # C
+	pick 12bd07b D
+	merge -C 2051b56 E # E
+	merge -C 233d48a H # H
+
+	EOF
+
+	grep -v "^#" <.git/ORIGINAL-TODO >output &&
+	test_cmp expect output
+'
+
+test_expect_success '`reset` refuses to overwrite untracked files' '
+	git checkout -b refuse-to-reset &&
+	test_commit dont-overwrite-untracked &&
+	git checkout @{-1} &&
+	: >dont-overwrite-untracked.t &&
+	echo "reset refs/tags/dont-overwrite-untracked" >script-from-scratch &&
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_must_fail git rebase -r HEAD &&
+	git rebase --abort
+'
+
+test_expect_success 'failed `merge` writes patch (may be rescheduled, too)' '
+	test_when_finished "test_might_fail git rebase --abort" &&
+	git checkout -b conflicting-merge A &&
+
+	: fail because of conflicting untracked file &&
+	>G.t &&
+	echo "merge -C H G" >script-from-scratch &&
+	test_config sequence.editor \""$PWD"/replace-editor.sh\" &&
+	test_tick &&
+	test_must_fail git rebase -ir HEAD &&
+	grep "^merge -C .* G$" .git/rebase-merge/done &&
+	grep "^merge -C .* G$" .git/rebase-merge/git-rebase-todo &&
+	test_path_is_file .git/rebase-merge/patch &&
+
+	: fail because of merge conflict &&
+	rm G.t .git/rebase-merge/patch &&
+	git reset --hard &&
+	test_commit conflicting-G G.t not-G conflicting-G &&
+	test_must_fail git rebase --continue &&
+	! grep "^merge -C .* G$" .git/rebase-merge/git-rebase-todo &&
+	test_path_is_file .git/rebase-merge/patch
+'
+
+test_expect_success 'with a branch tip that was cherry-picked already' '
+	git checkout -b already-upstream master &&
+	base="$(git rev-parse --verify HEAD)" &&
+
+	test_commit A1 &&
+	test_commit A2 &&
+	git reset --hard $base &&
+	test_commit B1 &&
+	test_tick &&
+	git merge -m "Merge branch A" A2 &&
+
+	git checkout -b upstream-with-a2 $base &&
+	test_tick &&
+	git cherry-pick A2 &&
+
+	git checkout already-upstream &&
+	test_tick &&
+	git rebase -i -r upstream-with-a2 &&
+	test_cmp_graph upstream-with-a2.. <<-\EOF
+	*   Merge branch A
+	|\
+	| * A1
+	* | B1
+	|/
+	o A2
+	EOF
+'
+
+test_done
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 11/17] rebase --rebase-merges: add test for --keep-empty
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (9 preceding siblings ...)
  2018-04-25 12:29               ` [PATCH v9 10/17] rebase: introduce the --rebase-merges option Johannes Schindelin
@ 2018-04-25 12:29               ` Johannes Schindelin
  2018-04-25 12:29               ` [PATCH v9 12/17] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
                                 ` (7 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:29 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Phillip Wood, Junio C Hamano, Jacob Keller,
	Stefan Beller, Philip Oakley, Eric Sunshine, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

From: Phillip Wood <phillip.wood@dunelm.org.uk>

If there are empty commits on the left hand side of $upstream...HEAD
then the empty commits on the right hand side that we want to keep are
being pruned.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
---
 t/t3421-rebase-topology-linear.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/t3421-rebase-topology-linear.sh b/t/t3421-rebase-topology-linear.sh
index b078f930462..e7438ad06ac 100755
--- a/t/t3421-rebase-topology-linear.sh
+++ b/t/t3421-rebase-topology-linear.sh
@@ -217,6 +217,7 @@ test_run_rebase success ''
 test_run_rebase success -m
 test_run_rebase success -i
 test_run_rebase failure -p
+test_run_rebase success --rebase-merges
 
 #       m
 #      /
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 12/17] sequencer: make refs generated by the `label` command worktree-local
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (10 preceding siblings ...)
  2018-04-25 12:29               ` [PATCH v9 11/17] rebase --rebase-merges: add test for --keep-empty Johannes Schindelin
@ 2018-04-25 12:29               ` Johannes Schindelin
  2018-04-25 12:29               ` [PATCH v9 13/17] sequencer: handle post-rewrite for merge commands Johannes Schindelin
                                 ` (6 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:29 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

This allows for rebases to be run in parallel in separate worktrees
(think: interrupted in the middle of one rebase, being asked to perform
a different rebase, adding a separate worktree just for that job).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 refs.c                   |  3 ++-
 t/t3430-rebase-merges.sh | 14 ++++++++++++++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 8b7a77fe5ee..f61ec58d1df 100644
--- a/refs.c
+++ b/refs.c
@@ -600,7 +600,8 @@ int dwim_log(const char *str, int len, struct object_id *oid, char **log)
 static int is_per_worktree_ref(const char *refname)
 {
 	return !strcmp(refname, "HEAD") ||
-		starts_with(refname, "refs/bisect/");
+		starts_with(refname, "refs/bisect/") ||
+		starts_with(refname, "refs/rewritten/");
 }
 
 static int is_pseudoref_syntax(const char *refname)
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index 5f0febb9970..96853784ec0 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -176,4 +176,18 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'refs/rewritten/* is worktree-local' '
+	git worktree add wt &&
+	cat >wt/script-from-scratch <<-\EOF &&
+	label xyz
+	exec GIT_DIR=../.git git rev-parse --verify refs/rewritten/xyz >a || :
+	exec git rev-parse --verify refs/rewritten/xyz >b
+	EOF
+
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	git -C wt rebase -i HEAD &&
+	test_must_be_empty wt/a &&
+	test_cmp_rev HEAD "$(cat wt/b)"
+'
+
 test_done
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 13/17] sequencer: handle post-rewrite for merge commands
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (11 preceding siblings ...)
  2018-04-25 12:29               ` [PATCH v9 12/17] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
@ 2018-04-25 12:29               ` Johannes Schindelin
  2018-04-25 12:29               ` [PATCH v9 14/17] rebase --rebase-merges: avoid "empty merges" Johannes Schindelin
                                 ` (5 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:29 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

In the previous patches, we implemented the basic functionality of the
`git rebase -i --rebase-merges` command, in particular the `merge`
command to create merge commits in the sequencer.

The interactive rebase is a lot more these days, though, than a simple
cherry-pick in a loop. For example, it calls the post-rewrite hook (if
any) after rebasing with a mapping of the old->new commits.

This patch implements the post-rewrite handling for the `merge` command
we just introduced. The other commands that were added recently (`label`
and `reset`) do not create new commits, therefore post-rewrite hooks do
not need to handle them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c              |  5 ++++-
 t/t3430-rebase-merges.sh | 25 +++++++++++++++++++++++++
 2 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index e9297122633..558efc1af6e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3074,7 +3074,10 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					    item->arg, item->arg_len,
 					    item->flags, opts)) < 0)
 				reschedule = 1;
-			else if (res > 0)
+			else if (item->commit)
+				record_in_rewritten(&item->commit->object.oid,
+						    peek_command(todo_list, 1));
+			if (res > 0)
 				/* failed with merge conflicts */
 				return error_with_patch(item->commit,
 							item->arg,
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index 96853784ec0..e9c5dc1cd95 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -190,4 +190,29 @@ test_expect_success 'refs/rewritten/* is worktree-local' '
 	test_cmp_rev HEAD "$(cat wt/b)"
 '
 
+test_expect_success 'post-rewrite hook and fixups work for merges' '
+	git checkout -b post-rewrite &&
+	test_commit same1 &&
+	git reset --hard HEAD^ &&
+	test_commit same2 &&
+	git merge -m "to fix up" same1 &&
+	echo same old same old >same2.t &&
+	test_tick &&
+	git commit --fixup HEAD same2.t &&
+	fixup="$(git rev-parse HEAD)" &&
+
+	mkdir -p .git/hooks &&
+	test_when_finished "rm .git/hooks/post-rewrite" &&
+	echo "cat >actual" | write_script .git/hooks/post-rewrite &&
+
+	test_tick &&
+	git rebase -i --autosquash -r HEAD^^^ &&
+	printf "%s %s\n%s %s\n%s %s\n%s %s\n" >expect $(git rev-parse \
+		$fixup^^2 HEAD^2 \
+		$fixup^^ HEAD^ \
+		$fixup^ HEAD \
+		$fixup HEAD) &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 14/17] rebase --rebase-merges: avoid "empty merges"
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (12 preceding siblings ...)
  2018-04-25 12:29               ` [PATCH v9 13/17] sequencer: handle post-rewrite for merge commands Johannes Schindelin
@ 2018-04-25 12:29               ` Johannes Schindelin
  2018-04-25 12:29               ` [PATCH v9 15/17] pull: accept --rebase=merges to recreate the branch topology Johannes Schindelin
                                 ` (4 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:29 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

The `git merge` command does not allow merging commits that are already
reachable from HEAD: `git merge HEAD^`, for example, will report that we
are already up to date and not change a thing.

In an interactive rebase, such a merge could occur previously, e.g. when
competing (or slightly modified) versions of a patch series were applied
upstream, and the user had to `git rebase --skip` all of the local
commits, and the topic branch becomes "empty" as a consequence.

Let's teach the todo command `merge` to behave the same as `git merge`.

Seeing as it requires some low-level trickery to create such merges with
Git's commands in the first place, we do not even have to bother to
introduce an option to force `merge` to create such merge commits.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c              | 7 +++++++
 t/t3430-rebase-merges.sh | 8 ++++++++
 2 files changed, 15 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 558efc1af6e..afa155c2829 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2810,6 +2810,13 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
 	write_message("no-ff", 5, git_path_merge_mode(), 0);
 
 	bases = get_merge_bases(head_commit, merge_commit);
+	if (bases && !oidcmp(&merge_commit->object.oid,
+			     &bases->item->object.oid)) {
+		ret = 0;
+		/* skip merging an ancestor of HEAD */
+		goto leave_merge;
+	}
+
 	for (j = bases; j; j = j->next)
 		commit_list_insert(j->item, &reversed);
 	free_commit_list(bases);
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index e9c5dc1cd95..1628c8dcc20 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -215,4 +215,12 @@ test_expect_success 'post-rewrite hook and fixups work for merges' '
 	test_cmp expect actual
 '
 
+test_expect_success 'refuse to merge ancestors of HEAD' '
+	echo "merge HEAD^" >script-from-scratch &&
+	test_config -C wt sequence.editor \""$PWD"/replace-editor.sh\" &&
+	before="$(git rev-parse HEAD)" &&
+	git rebase -i HEAD &&
+	test_cmp_rev HEAD $before
+'
+
 test_done
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 15/17] pull: accept --rebase=merges to recreate the branch topology
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (13 preceding siblings ...)
  2018-04-25 12:29               ` [PATCH v9 14/17] rebase --rebase-merges: avoid "empty merges" Johannes Schindelin
@ 2018-04-25 12:29               ` Johannes Schindelin
  2018-04-25 12:29               ` [PATCH v9 16/17] rebase -i: introduce --rebase-merges=[no-]rebase-cousins Johannes Schindelin
                                 ` (3 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:29 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

Similar to the `preserve` mode simply passing the `--preserve-merges`
option to the `rebase` command, the `merges` mode simply passes the
`--rebase-merges` option.

This will allow users to conveniently rebase non-trivial commit
topologies when pulling new commits, without flattening them.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/config.txt               |  8 ++++++++
 Documentation/git-pull.txt             |  6 +++++-
 builtin/pull.c                         | 14 ++++++++++----
 builtin/remote.c                       | 18 ++++++++++++++----
 contrib/completion/git-completion.bash |  2 +-
 5 files changed, 38 insertions(+), 10 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 2659153cb37..d6bcb5dcb67 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1058,6 +1058,10 @@ branch.<name>.rebase::
 	"git pull" is run. See "pull.rebase" for doing this in a non
 	branch-specific manner.
 +
+When `merges`, pass the `--rebase-merges` option to 'git rebase'
+so that the local merge commits are included in the rebase (see
+linkgit:git-rebase[1] for details).
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
@@ -2617,6 +2621,10 @@ pull.rebase::
 	pull" is run. See "branch.<name>.rebase" for setting this on a
 	per-branch basis.
 +
+When `merges`, pass the `--rebase-merges` option to 'git rebase'
+so that the local merge commits are included in the rebase (see
+linkgit:git-rebase[1] for details).
++
 When preserve, also pass `--preserve-merges` along to 'git rebase'
 so that locally committed merge commits will not be flattened
 by running 'git pull'.
diff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt
index ce05b7a5b13..4e0ad6fd8e0 100644
--- a/Documentation/git-pull.txt
+++ b/Documentation/git-pull.txt
@@ -101,13 +101,17 @@ Options related to merging
 include::merge-options.txt[]
 
 -r::
---rebase[=false|true|preserve|interactive]::
+--rebase[=false|true|merges|preserve|interactive]::
 	When true, rebase the current branch on top of the upstream
 	branch after fetching. If there is a remote-tracking branch
 	corresponding to the upstream branch and the upstream branch
 	was rebased since last fetched, the rebase uses that information
 	to avoid rebasing non-local changes.
 +
+When set to `merges`, rebase using `git rebase --rebase-merges` so that
+the local merge commits are included in the rebase (see
+linkgit:git-rebase[1] for details).
++
 When set to preserve, rebase with the `--preserve-merges` option passed
 to `git rebase` so that locally created merge commits will not be flattened.
 +
diff --git a/builtin/pull.c b/builtin/pull.c
index 71aac5005e0..c719a4f9d73 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -27,14 +27,16 @@ enum rebase_type {
 	REBASE_FALSE = 0,
 	REBASE_TRUE,
 	REBASE_PRESERVE,
+	REBASE_MERGES,
 	REBASE_INTERACTIVE
 };
 
 /**
  * Parses the value of --rebase. If value is a false value, returns
  * REBASE_FALSE. If value is a true value, returns REBASE_TRUE. If value is
- * "preserve", returns REBASE_PRESERVE. If value is a invalid value, dies with
- * a fatal error if fatal is true, otherwise returns REBASE_INVALID.
+ * "merges", returns REBASE_MERGES. If value is "preserve", returns
+ * REBASE_PRESERVE. If value is a invalid value, dies with a fatal error if
+ * fatal is true, otherwise returns REBASE_INVALID.
  */
 static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		int fatal)
@@ -47,6 +49,8 @@ static enum rebase_type parse_config_rebase(const char *key, const char *value,
 		return REBASE_TRUE;
 	else if (!strcmp(value, "preserve"))
 		return REBASE_PRESERVE;
+	else if (!strcmp(value, "merges"))
+		return REBASE_MERGES;
 	else if (!strcmp(value, "interactive"))
 		return REBASE_INTERACTIVE;
 
@@ -130,7 +134,7 @@ static struct option pull_options[] = {
 	/* Options passed to git-merge or git-rebase */
 	OPT_GROUP(N_("Options related to merging")),
 	{ OPTION_CALLBACK, 'r', "rebase", &opt_rebase,
-	  "false|true|preserve|interactive",
+	  "false|true|merges|preserve|interactive",
 	  N_("incorporate changes by rebasing rather than merging"),
 	  PARSE_OPT_OPTARG, parse_opt_rebase },
 	OPT_PASSTHRU('n', NULL, &opt_diffstat, NULL,
@@ -800,7 +804,9 @@ static int run_rebase(const struct object_id *curr_head,
 	argv_push_verbosity(&args);
 
 	/* Options passed to git-rebase */
-	if (opt_rebase == REBASE_PRESERVE)
+	if (opt_rebase == REBASE_MERGES)
+		argv_array_push(&args, "--rebase-merges");
+	else if (opt_rebase == REBASE_PRESERVE)
 		argv_array_push(&args, "--preserve-merges");
 	else if (opt_rebase == REBASE_INTERACTIVE)
 		argv_array_push(&args, "--interactive");
diff --git a/builtin/remote.c b/builtin/remote.c
index 805ffc05cdb..45c9219e07a 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -245,7 +245,9 @@ static int add(int argc, const char **argv)
 struct branch_info {
 	char *remote_name;
 	struct string_list merge;
-	enum { NO_REBASE, NORMAL_REBASE, INTERACTIVE_REBASE } rebase;
+	enum {
+		NO_REBASE, NORMAL_REBASE, INTERACTIVE_REBASE, REBASE_MERGES
+	} rebase;
 };
 
 static struct string_list branch_list = STRING_LIST_INIT_NODUP;
@@ -306,6 +308,8 @@ static int config_read_branches(const char *key, const char *value, void *cb)
 				info->rebase = v;
 			else if (!strcmp(value, "preserve"))
 				info->rebase = NORMAL_REBASE;
+			else if (!strcmp(value, "merges"))
+				info->rebase = REBASE_MERGES;
 			else if (!strcmp(value, "interactive"))
 				info->rebase = INTERACTIVE_REBASE;
 		}
@@ -963,9 +967,15 @@ static int show_local_info_item(struct string_list_item *item, void *cb_data)
 
 	printf("    %-*s ", show_info->width, item->string);
 	if (branch_info->rebase) {
-		printf_ln(branch_info->rebase == INTERACTIVE_REBASE
-			  ? _("rebases interactively onto remote %s")
-			  : _("rebases onto remote %s"), merge->items[0].string);
+		const char *msg;
+		if (branch_info->rebase == INTERACTIVE_REBASE)
+			msg = _("rebases interactively onto remote %s");
+		else if (branch_info->rebase == REBASE_MERGES)
+			msg = _("rebases interactively (with merges) onto "
+				"remote %s");
+		else
+			msg = _("rebases onto remote %s");
+		printf_ln(msg, merge->items[0].string);
 		return 0;
 	} else if (show_info->any_rebase) {
 		printf_ln(_(" merges with remote %s"), merge->items[0].string);
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index e6469004099..7fe2e213d1d 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2115,7 +2115,7 @@ _git_config ()
 		return
 		;;
 	branch.*.rebase)
-		__gitcomp "false true preserve interactive"
+		__gitcomp "false true merges preserve interactive"
 		return
 		;;
 	remote.pushdefault)
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 16/17] rebase -i: introduce --rebase-merges=[no-]rebase-cousins
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (14 preceding siblings ...)
  2018-04-25 12:29               ` [PATCH v9 15/17] pull: accept --rebase=merges to recreate the branch topology Johannes Schindelin
@ 2018-04-25 12:29               ` Johannes Schindelin
  2018-04-25 12:29               ` [PATCH v9 17/17] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
                                 ` (2 subsequent siblings)
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:29 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

When running `git rebase --rebase-merges` non-interactively with an
ancestor of HEAD as <upstream> (or leaving the todo list unmodified),
we would ideally recreate the exact same commits as before the rebase.

However, if there are commits in the commit range <upstream>.. that do not
have <upstream> as direct ancestor (i.e. if `git log <upstream>..` would
show commits that are omitted by `git log --ancestry-path <upstream>..`),
this is currently not the case: we would turn them into commits that have
<upstream> as direct ancestor.

Let's illustrate that with a diagram:

        C
      /   \
A - B - E - F
  \   /
    D

Currently, after running `git rebase -i --rebase-merges B`, the new branch
structure would be (pay particular attention to the commit `D`):

       --- C' --
      /         \
A - B ------ E' - F'
      \    /
        D'

This is not really preserving the branch topology from before! The
reason is that the commit `D` does not have `B` as ancestor, and
therefore it gets rebased onto `B`.

This is unintuitive behavior. Even worse, when recreating branch
structure, most use cases would appear to want cousins *not* to be
rebased onto the new base commit. For example, Git for Windows (the
heaviest user of the Git garden shears, which served as the blueprint
for --rebase-merges) frequently merges branches from `next` early, and
these branches certainly do *not* want to be rebased. In the example
above, the desired outcome would look like this:

       --- C' --
      /         \
A - B ------ E' - F'
  \        /
   -- D' --

Let's introduce the term "cousins" for such commits ("D" in the
example), and let's not rebase them by default. For hypothetical
use cases where cousins *do* need to be rebased, `git rebase
--rebase=merges=rebase-cousins` needs to be used.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt | 15 +++++++++++----
 builtin/rebase--helper.c     |  9 ++++++++-
 git-rebase--interactive.sh   |  1 +
 git-rebase.sh                | 12 +++++++++++-
 sequencer.c                  |  4 ++++
 sequencer.h                  |  6 ++++++
 t/t3430-rebase-merges.sh     | 18 ++++++++++++++++++
 7 files changed, 59 insertions(+), 6 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 7f1756f1eba..fe681d69281 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -380,7 +380,7 @@ rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
 -r::
---rebase-merges::
+--rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
 	By default, a rebase will simply drop merge commits from the todo
 	list, and put the rebased commits into a single, linear branch.
 	With `--rebase-merges`, the rebase will instead try to preserve
@@ -389,9 +389,16 @@ have the long commit hash prepended to the format.
 	manual amendments in these merge commits will have to be
 	resolved/re-applied manually.
 +
-This mode is similar in spirit to `--preserve-merges`, but in contrast to
-that option works well in interactive rebases: commits can be reordered,
-inserted and dropped at will.
+By default, or when `no-rebase-cousins` was specified, commits which do not
+have `<upstream>` as direct ancestor will keep their original branch point,
+i.e. commits that would be excluded by gitlink:git-log[1]'s
+`--ancestry-path` option will keep their original ancestry by default. If
+the `rebase-cousins` mode is turned on, such commits are instead rebased
+onto `<upstream>` (or `<onto>`, if specified).
++
+The `--rebase-merges` mode is similar in spirit to `--preserve-merges`, but
+in contrast to that option works well in interactive rebases: commits can be
+reordered, inserted and dropped at will.
 +
 It is currently only possible to recreate the merge commits using the
 `recursive` merge strategy; Different merge strategies can be used only via
diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c
index 781782e7272..f7c2a5fdc81 100644
--- a/builtin/rebase--helper.c
+++ b/builtin/rebase--helper.c
@@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 {
 	struct replay_opts opts = REPLAY_OPTS_INIT;
 	unsigned flags = 0, keep_empty = 0, rebase_merges = 0;
-	int abbreviate_commands = 0;
+	int abbreviate_commands = 0, rebase_cousins = -1;
 	enum {
 		CONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_OIDS, EXPAND_OIDS,
 		CHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH,
@@ -25,6 +25,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 		OPT_BOOL(0, "allow-empty-message", &opts.allow_empty_message,
 			N_("allow commits with empty messages")),
 		OPT_BOOL(0, "rebase-merges", &rebase_merges, N_("rebase merge commits")),
+		OPT_BOOL(0, "rebase-cousins", &rebase_cousins,
+			 N_("keep original branch points of cousins")),
 		OPT_CMDMODE(0, "continue", &command, N_("continue rebase"),
 				CONTINUE),
 		OPT_CMDMODE(0, "abort", &command, N_("abort rebase"),
@@ -59,8 +61,13 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)
 	flags |= keep_empty ? TODO_LIST_KEEP_EMPTY : 0;
 	flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0;
 	flags |= rebase_merges ? TODO_LIST_REBASE_MERGES : 0;
+	flags |= rebase_cousins > 0 ? TODO_LIST_REBASE_COUSINS : 0;
 	flags |= command == SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0;
 
+	if (rebase_cousins >= 0 && !rebase_merges)
+		warning(_("--[no-]rebase-cousins has no effect without "
+			  "--rebase-merges"));
+
 	if (command == CONTINUE && argc == 1)
 		return !!sequencer_continue(&opts);
 	if (command == ABORT && argc == 1)
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index e29da634339..cbf44f86482 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -971,6 +971,7 @@ git_rebase__interactive () {
 
 	git rebase--helper --make-script ${keep_empty:+--keep-empty} \
 		${rebase_merges:+--rebase-merges} \
+		${rebase_cousins:+--rebase-cousins} \
 		$revisions ${restrict_revision+^$restrict_revision} >"$todo" ||
 	die "$(gettext "Could not generate todo list")"
 
diff --git a/git-rebase.sh b/git-rebase.sh
index a553f969d11..40be59ecc47 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -17,7 +17,7 @@ q,quiet!           be quiet. implies --no-stat
 autostash          automatically stash/stash pop before and after
 fork-point         use 'merge-base --fork-point' to refine upstream
 onto=!             rebase onto given branch instead of upstream
-r,rebase-merges!   try to rebase merges instead of skipping them
+r,rebase-merges?   try to rebase merges instead of skipping them
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -91,6 +91,7 @@ state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
 rebase_merges=
+rebase_cousins=
 preserve_merges=
 autosquash=
 keep_empty=
@@ -286,6 +287,15 @@ do
 		rebase_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
 		;;
+	--rebase-merges=*)
+		rebase_merges=t
+		case "${1#*=}" in
+		rebase-cousins) rebase_cousins=t;;
+		no-rebase-cousins) rebase_cousins=;;
+		*) die "Unknown mode: $1";;
+		esac
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	--preserve-merges)
 		preserve_merges=t
 		test -z "$interactive_rebase" && interactive_rebase=implied
diff --git a/sequencer.c b/sequencer.c
index afa155c2829..e2f83942843 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -3578,6 +3578,7 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 				   unsigned flags)
 {
 	int keep_empty = flags & TODO_LIST_KEEP_EMPTY;
+	int rebase_cousins = flags & TODO_LIST_REBASE_COUSINS;
 	struct strbuf buf = STRBUF_INIT, oneline = STRBUF_INIT;
 	struct strbuf label = STRBUF_INIT;
 	struct commit_list *commits = NULL, **tail = &commits, *iter;
@@ -3755,6 +3756,9 @@ static int make_script_with_merges(struct pretty_print_context *pp,
 					   &commit->object.oid);
 			if (entry)
 				to = entry->string;
+			else if (!rebase_cousins)
+				to = label_oid(&commit->object.oid, NULL,
+					       &state);
 
 			if (!to || !strcmp(to, "onto"))
 				fprintf(out, "%s onto\n", cmd_reset);
diff --git a/sequencer.h b/sequencer.h
index 6bc4da17243..d9570d92b11 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -60,6 +60,12 @@ int sequencer_remove_state(struct replay_opts *opts);
 #define TODO_LIST_SHORTEN_IDS (1U << 1)
 #define TODO_LIST_ABBREVIATE_CMDS (1U << 2)
 #define TODO_LIST_REBASE_MERGES (1U << 3)
+/*
+ * When rebasing merges, commits that do have the base commit as ancestor
+ * ("cousins") are *not* rebased onto the new base by default. If those
+ * commits should be rebased onto the new base, this flag needs to be passed.
+ */
+#define TODO_LIST_REBASE_COUSINS (1U << 4)
 int sequencer_make_script(FILE *out, int argc, const char **argv,
 			  unsigned flags);
 
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index 1628c8dcc20..3d4dfdf7bec 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -176,6 +176,24 @@ test_expect_success 'with a branch tip that was cherry-picked already' '
 	EOF
 '
 
+test_expect_success 'do not rebase cousins unless asked for' '
+	git checkout -b cousins master &&
+	before="$(git rev-parse --verify HEAD)" &&
+	test_tick &&
+	git rebase -r HEAD^ &&
+	test_cmp_rev HEAD $before &&
+	test_tick &&
+	git rebase --rebase-merges=rebase-cousins HEAD^ &&
+	test_cmp_graph HEAD^.. <<-\EOF
+	*   Merge the topic branch '\''onebranch'\''
+	|\
+	| * D
+	| * G
+	|/
+	o H
+	EOF
+'
+
 test_expect_success 'refs/rewritten/* is worktree-local' '
 	git worktree add wt &&
 	cat >wt/script-from-scratch <<-\EOF &&
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v9 17/17] rebase -i --rebase-merges: add a section to the man page
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (15 preceding siblings ...)
  2018-04-25 12:29               ` [PATCH v9 16/17] rebase -i: introduce --rebase-merges=[no-]rebase-cousins Johannes Schindelin
@ 2018-04-25 12:29               ` Johannes Schindelin
  2018-04-26  3:51               ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Junio C Hamano
  2018-05-25 14:19               ` Sergey Organov
  18 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:29 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Jacob Keller, Stefan Beller,
	Philip Oakley, Eric Sunshine, Phillip Wood, Igor Djordjevic,
	Johannes Sixt, Sergey Organov, Martin Ågren

The --rebase-merges mode is probably not half as intuitive to use as
its inventor hopes, so let's document it some.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Documentation/git-rebase.txt | 135 +++++++++++++++++++++++++++++++++++
 1 file changed, 135 insertions(+)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index fe681d69281..bd5ecff980e 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -403,6 +403,8 @@ reordered, inserted and dropped at will.
 It is currently only possible to recreate the merge commits using the
 `recursive` merge strategy; Different merge strategies can be used only via
 explicit `exec git merge -s <strategy> [...]` commands.
++
+See also REBASING MERGES below.
 
 -p::
 --preserve-merges::
@@ -801,6 +803,139 @@ The ripple effect of a "hard case" recovery is especially bad:
 'everyone' downstream from 'topic' will now have to perform a "hard
 case" recovery too!
 
+REBASING MERGES
+-----------------
+
+The interactive rebase command was originally designed to handle
+individual patch series. As such, it makes sense to exclude merge
+commits from the todo list, as the developer may have merged the
+then-current `master` while working on the branch, only to rebase
+all the commits onto `master` eventually (skipping the merge
+commits).
+
+However, there are legitimate reasons why a developer may want to
+recreate merge commits: to keep the branch structure (or "commit
+topology") when working on multiple, inter-related branches.
+
+In the following example, the developer works on a topic branch that
+refactors the way buttons are defined, and on another topic branch
+that uses that refactoring to implement a "Report a bug" button. The
+output of `git log --graph --format=%s -5` may look like this:
+
+------------
+*   Merge branch 'report-a-bug'
+|\
+| * Add the feedback button
+* | Merge branch 'refactor-button'
+|\ \
+| |/
+| * Use the Button class for all buttons
+| * Extract a generic Button class from the DownloadButton one
+------------
+
+The developer might want to rebase those commits to a newer `master`
+while keeping the branch topology, for example when the first topic
+branch is expected to be integrated into `master` much earlier than the
+second one, say, to resolve merge conflicts with changes to the
+DownloadButton class that made it into `master`.
+
+This rebase can be performed using the `--rebase-merges` option.
+It will generate a todo list looking like this:
+
+------------
+label onto
+
+# Branch: refactor-button
+reset onto
+pick 123456 Extract a generic Button class from the DownloadButton one
+pick 654321 Use the Button class for all buttons
+label refactor-button
+
+# Branch: report-a-bug
+reset refactor-button # Use the Button class for all buttons
+pick abcdef Add the feedback button
+label report-a-bug
+
+reset onto
+merge -C a1b2c3 refactor-button # Merge 'refactor-button'
+merge -C 6f5e4d report-a-bug # Merge 'report-a-bug'
+------------
+
+In contrast to a regular interactive rebase, there are `label`, `reset`
+and `merge` commands in addition to `pick` ones.
+
+The `label` command associates a label with the current HEAD when that
+command is executed. These labels are created as worktree-local refs
+(`refs/rewritten/<label>`) that will be deleted when the rebase
+finishes. That way, rebase operations in multiple worktrees linked to
+the same repository do not interfere with one another. If the `label`
+command fails, it is rescheduled immediately, with a helpful message how
+to proceed.
+
+The `reset` command resets the HEAD, index and worktree to the specified
+revision. It is isimilar to an `exec git reset --hard <label>`, but
+refuses to overwrite untracked files. If the `reset` command fails, it is
+rescheduled immediately, with a helpful message how to edit the todo list
+(this typically happens when a `reset` command was inserted into the todo
+list manually and contains a typo).
+
+The `merge` command will merge the specified revision into whatever is
+HEAD at that time. With `-C <original-commit>`, the commit message of
+the specified merge commit will be used. When the `-C` is changed to
+a lower-case `-c`, the message will be opened in an editor after a
+successful merge so that the user can edit the message.
+
+If a `merge` command fails for any reason other than merge conflicts (i.e.
+when the merge operation did not even start), it is rescheduled immediately.
+
+At this time, the `merge` command will *always* use the `recursive`
+merge strategy, with no way to choose a different one. To work around
+this, an `exec` command can be used to call `git merge` explicitly,
+using the fact that the labels are worktree-local refs (the ref
+`refs/rewritten/onto` would correspond to the label `onto`, for example).
+
+Note: the first command (`label onto`) labels the revision onto which
+the commits are rebased; The name `onto` is just a convention, as a nod
+to the `--onto` option.
+
+It is also possible to introduce completely new merge commits from scratch
+by adding a command of the form `merge <merge-head>`. This form will
+generate a tentative commit message and always open an editor to let the
+user edit it. This can be useful e.g. when a topic branch turns out to
+address more than a single concern and wants to be split into two or
+even more topic branches. Consider this todo list:
+
+------------
+pick 192837 Switch from GNU Makefiles to CMake
+pick 5a6c7e Document the switch to CMake
+pick 918273 Fix detection of OpenSSL in CMake
+pick afbecd http: add support for TLS v1.3
+pick fdbaec Fix detection of cURL in CMake on Windows
+------------
+
+The one commit in this list that is not related to CMake may very well
+have been motivated by working on fixing all those bugs introduced by
+switching to CMake, but it addresses a different concern. To split this
+branch into two topic branches, the todo list could be edited like this:
+
+------------
+label onto
+
+pick afbecd http: add support for TLS v1.3
+label tlsv1.3
+
+reset onto
+pick 192837 Switch from GNU Makefiles to CMake
+pick 918273 Fix detection of OpenSSL in CMake
+pick fdbaec Fix detection of cURL in CMake on Windows
+pick 5a6c7e Document the switch to CMake
+label cmake
+
+reset onto
+merge tlsv1.3
+merge cmake
+------------
+
 BUGS
 ----
 The todo list presented by `--preserve-merges --interactive` does not
-- 
2.17.0.windows.1.33.gfcbb1fa0445

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 0/4] rebase -i: avoid stale "# This is a combinationof" in commit messages
  2018-04-23 19:50                   ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combinationof" " Phillip Wood
@ 2018-04-25 12:48                     ` Johannes Schindelin
  2018-04-25 17:09                       ` Phillip Wood
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-25 12:48 UTC (permalink / raw)
  To: phillip.wood; +Cc: Stefan Beller, git, Junio C Hamano, Eric Sunshine

Hi Phillip,

On Mon, 23 Apr 2018, Phillip Wood wrote:

> On 23/04/18 19:11, Stefan Beller wrote:
> > 
> > On Sat, Apr 21, 2018 at 12:34 AM, Johannes Schindelin
> > <johannes.schindelin@gmx.de> wrote:
> > > Eric Sunshine pointed out that I had such a commit message in
> > > https://public-inbox.org/git/CAPig+cRrS0_nYJJY=O6cboV630sNQHPV5QGrQdD8MW-sYzNFGQ@mail.gmail.com/
> > > and I went on a hunt to figure out how the heck this happened.
> > >
> > > Turns out that if there is a fixup/squash chain where the *last* command
> > > fails with merge conflicts, and we either --skip ahead or resolve the
> > > conflict to a clean tree and then --continue, our code does not do a
> > > final cleanup.
> > >
> > > Contrary to my initial gut feeling, this bug was not introduced by my
> > > rewrite in C of the core parts of rebase -i, but it looks to me as if
> > > that bug was with us for a very long time (at least the --skip part).
> > >
> > > The developer (read: user of rebase -i) in me says that we would want to
> > > fast-track this, but the author of rebase -i in me says that we should
> > > be cautious and cook this in `next` for a while.
> > 
> > I looked through the patches again and think this series is good to go.
> 
> I've just realized I commented on an outdated version as the new version was
> posted there rather than as a reply to v1. I've just looked through it and I'm
> not sure it addresses the unnecessary editing of the commit message of the
> previous commit if a single squash command is skipped as outlined in
> https://public-inbox.org/git/b6512eae-e214-9699-4d69-77117a0daec3@talktalk.net/

I have not forgotten about this! I simply did not find the time yet, is
all...

The patch series still has not been merged to `next`, but I plan on
working on your suggested changes as an add-on commit anyway. I am not
quite sure yet how I want to handle the "avoid running commit for the
first fixup/squash in the series" problem, but I think we will have to add
*yet another* file that is written (in the "we already have comments in
the commit message" conditional block in error_failed_squash())...

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 0/4] rebase -i: avoid stale "# This is a combinationof" in commit messages
  2018-04-25 12:48                     ` Johannes Schindelin
@ 2018-04-25 17:09                       ` Phillip Wood
  2018-04-26  9:51                         ` Johannes Schindelin
  0 siblings, 1 reply; 412+ messages in thread
From: Phillip Wood @ 2018-04-25 17:09 UTC (permalink / raw)
  To: Johannes Schindelin, phillip.wood
  Cc: Stefan Beller, git, Junio C Hamano, Eric Sunshine

On 25/04/18 13:48, Johannes Schindelin wrote:
> Hi Phillip,
> 
> On Mon, 23 Apr 2018, Phillip Wood wrote:
> 
>> On 23/04/18 19:11, Stefan Beller wrote:
>>>
>>> On Sat, Apr 21, 2018 at 12:34 AM, Johannes Schindelin
>>> <johannes.schindelin@gmx.de> wrote:
>>>> Eric Sunshine pointed out that I had such a commit message in
>>>> https://public-inbox.org/git/CAPig+cRrS0_nYJJY=O6cboV630sNQHPV5QGrQdD8MW-sYzNFGQ@mail.gmail.com/
>>>> and I went on a hunt to figure out how the heck this happened.
>>>>
>>>> Turns out that if there is a fixup/squash chain where the *last* command
>>>> fails with merge conflicts, and we either --skip ahead or resolve the
>>>> conflict to a clean tree and then --continue, our code does not do a
>>>> final cleanup.
>>>>
>>>> Contrary to my initial gut feeling, this bug was not introduced by my
>>>> rewrite in C of the core parts of rebase -i, but it looks to me as if
>>>> that bug was with us for a very long time (at least the --skip part).
>>>>
>>>> The developer (read: user of rebase -i) in me says that we would want to
>>>> fast-track this, but the author of rebase -i in me says that we should
>>>> be cautious and cook this in `next` for a while.
>>>
>>> I looked through the patches again and think this series is good to go.
>>
>> I've just realized I commented on an outdated version as the new version was
>> posted there rather than as a reply to v1. I've just looked through it and I'm
>> not sure it addresses the unnecessary editing of the commit message of the
>> previous commit if a single squash command is skipped as outlined in
>> https://public-inbox.org/git/b6512eae-e214-9699-4d69-77117a0daec3@talktalk.net/
> 
> I have not forgotten about this! I simply did not find the time yet, is
> all...

I wondered if that was the case but I wanted to check as I wasn't sure 
if you'd seen the original message as it was on an obsolete version of 
the series

> The patch series still has not been merged to `next`, but I plan on
> working on your suggested changes as an add-on commit anyway. I am not
> quite sure yet how I want to handle the "avoid running commit for the
> first fixup/squash in the series" problem, but I think we will have to add
> *yet another* file that is written (in the "we already have comments in
> the commit message" conditional block in error_failed_squash())...

I wonder if creating the file in update_squash_messages() rather than 
error_failed_squash() would be a better approach as then it is easy to 
only create rebase_path_amend_type() when there has already been a 
squash or fixup. The file is removed in the loop that picks commits in 
pick_commits() so it would be cleaned up at the beginning of the next 
pick if it's not needed.

Best Wishes

Phillip

> 
> Ciao,
> Dscho
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (16 preceding siblings ...)
  2018-04-25 12:29               ` [PATCH v9 17/17] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
@ 2018-04-26  3:51               ` Junio C Hamano
  2018-04-26  6:06                 ` Junio C Hamano
  2018-05-25 14:19               ` Sergey Organov
  18 siblings, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-04-26  3:51 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Jacob Keller, Stefan Beller, Philip Oakley, Eric Sunshine,
	Phillip Wood, Igor Djordjevic, Johannes Sixt, Sergey Organov,
	Martin Ågren

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Changes since v8:
>
> - Disentangled the patch introducing `label`/`reset` from the one
>   introducing `merge` again (this was one stupid, tired `git commit
>   --amend` too many).
>
> - Augmented the commit message of "introduce the `merge` command" to
>   describe what the `label onto` is all about.
>
> - Fixed the error message when `reset` would overwrite untracked files to
>   actually say that a "reset" failed (not a "merge").
>
> - Clarified the rationale for `label onto` in the commit message of
>   "rebase-helper --make-script: introduce a flag to rebase merges".
>
> - Edited the description of `--rebase-merges` heavily, for clarity, in
>   "rebase: introduce the --rebase-merges option".
>
> - Edited the commit message of (and the documentation introduced by) " rebase
>   -i: introduce --rebase-merges=[no-]rebase-cousins" for clarity (also
>   mentioning the `--ancestry-path` option).
>
> - When run_git_commit() fails after a successful merge, we now take pains
>   not to reschedule the `merge` command.
>
> - Rebased the patch series on top of current `master`, i.e. both
>   `pw/rebase-keep-empty-fixes` and `pw/rebase-signoff`, to resolve merge
>   conflicts myself.

Good to see the last item, as this gave me a chance to make sure
that the conflict resolution I've been carrying matches how you
would have resolved as the original author.  Applying these on the
old base (with minor conflict resolution) to match the old iteration
and merging the result to the new base1f1cddd5 ("The fourth batch
for 2.18", 2018-04-25) resulted in the same tree as the tree that
results from applying these on top of the new base.

That was done only to validate the result of the past resolution
(and also seeing the interdiff from the old iteration).  There is no
reason to keep this series back-portable to older tip of 'master',
so I'll queue the result of applying the patches to the new base.


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges
  2018-04-26  3:51               ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Junio C Hamano
@ 2018-04-26  6:06                 ` Junio C Hamano
  0 siblings, 0 replies; 412+ messages in thread
From: Junio C Hamano @ 2018-04-26  6:06 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Jacob Keller, Stefan Beller, Philip Oakley, Eric Sunshine,
	Phillip Wood, Igor Djordjevic, Johannes Sixt, Sergey Organov,
	Martin Ågren

Junio C Hamano <gitster@pobox.com> writes:

>> - Rebased the patch series on top of current `master`, i.e. both
>>   `pw/rebase-keep-empty-fixes` and `pw/rebase-signoff`, to resolve merge
>>   conflicts myself.
>
> Good to see the last item, as this gave me a chance to make sure
> that the conflict resolution I've been carrying matches how you
> would have resolved as the original author.  Applying these on the
> old base (with minor conflict resolution) to match the old iteration
> and merging the result to the new base1f1cddd5 ("The fourth batch
> for 2.18", 2018-04-25) resulted in the same tree as the tree that
> results from applying these on top of the new base.
>
> That was done only to validate the result of the past resolution
> (and also seeing the interdiff from the old iteration).  There is no
> reason to keep this series back-portable to older tip of 'master',
> so I'll queue the result of applying the patches to the new base.

By the way, the rebasing made the topic textually merge cleanly to
the tip of 'pu' which made it slightly more cumbersome to deal with
a semantic conflict the topic has with another topic that modifies
the function signature of get_main_ref_store().  This topic adds a
new callsite in sequencer.c to this function.

The old base that forced the integrator to resolve conflicts in
sequencer.c with some other topic, thanks to that exact textual
conflicts, gave rerere a chance to record the adjustment for this
semantic conflict.

Now because the series applied to new base does not have textual
conflicts in sequencer.c when merged to 'pu', the adjustment for the
semantic conflict needs to be carried by a different mechanism.

    Side note.  Do not take the above as a complaint.  Dealing with
    interactions among various topics in flight while keeping them
    as straight and clean topic is what I do.  It is a normal part
    of running an active project.

It may be an interesting exercise to attempt to rebase tonight's
'pu' onto something younger in 'pu', say 'pu~4', without changing
anything in "pu^2" (which is the tip of this topic) and see how well
the merge recreation feature of this topic handles the evil merge.

The gist of the evil merge looks like this:

diff --cc sequencer.c
index a428fc7db7,e2f8394284..729cf05768
--- a/sequencer.c
+++ b/sequencer.c
@@@ -2483,6 -2527,349 +2556,349 @@@ static int do_exec(const char *command_
 ...
+ 
+ static int do_label(const char *name, int len)
+ {
 -	struct ref_store *refs = get_main_ref_store();
++	struct ref_store *refs = get_main_ref_store(the_repository);
+ 	struct ref_transaction *transaction;
+ 	struct strbuf ref_name = STRBUF_INIT, err = STRBUF_INIT;
+ 	struct strbuf msg = STRBUF_INIT;
+...

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 0/4] rebase -i: avoid stale "# This is a combinationof" in commit messages
  2018-04-25 17:09                       ` Phillip Wood
@ 2018-04-26  9:51                         ` Johannes Schindelin
  2018-04-26 10:52                           ` Phillip Wood
  0 siblings, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-26  9:51 UTC (permalink / raw)
  To: phillip.wood; +Cc: Stefan Beller, git, Junio C Hamano, Eric Sunshine

Hi Phillip,

On Wed, 25 Apr 2018, Phillip Wood wrote:

> On 25/04/18 13:48, Johannes Schindelin wrote:
> > 
> > On Mon, 23 Apr 2018, Phillip Wood wrote:
> > 
> > > On 23/04/18 19:11, Stefan Beller wrote:
> > > >
> > > > On Sat, Apr 21, 2018 at 12:34 AM, Johannes Schindelin
> > > > <johannes.schindelin@gmx.de> wrote:
> > > > > Eric Sunshine pointed out that I had such a commit message in
> > > > > https://public-inbox.org/git/CAPig+cRrS0_nYJJY=O6cboV630sNQHPV5QGrQdD8MW-sYzNFGQ@mail.gmail.com/
> > > > > and I went on a hunt to figure out how the heck this happened.
> > > > >
> > > > > Turns out that if there is a fixup/squash chain where the *last*
> > > > > command fails with merge conflicts, and we either --skip ahead
> > > > > or resolve the conflict to a clean tree and then --continue, our
> > > > > code does not do a final cleanup.
> > > > >
> > > > > Contrary to my initial gut feeling, this bug was not introduced
> > > > > by my rewrite in C of the core parts of rebase -i, but it looks
> > > > > to me as if that bug was with us for a very long time (at least
> > > > > the --skip part).
> > > > >
> > > > > The developer (read: user of rebase -i) in me says that we would
> > > > > want to fast-track this, but the author of rebase -i in me says
> > > > > that we should be cautious and cook this in `next` for a while.
> > > >
> > > > I looked through the patches again and think this series is good
> > > > to go.
> > >
> > > I've just realized I commented on an outdated version as the new
> > > version was posted there rather than as a reply to v1. I've just
> > > looked through it and I'm not sure it addresses the unnecessary
> > > editing of the commit message of the previous commit if a single
> > > squash command is skipped as outlined in
> > > https://public-inbox.org/git/b6512eae-e214-9699-4d69-77117a0daec3@talktalk.net/
> > 
> > I have not forgotten about this! I simply did not find the time yet,
> > is all...
> 
> I wondered if that was the case but I wanted to check as I wasn't sure
> if you'd seen the original message as it was on an obsolete version of
> the series
> 
> > The patch series still has not been merged to `next`, but I plan on
> > working on your suggested changes as an add-on commit anyway. I am not
> > quite sure yet how I want to handle the "avoid running commit for the
> > first fixup/squash in the series" problem, but I think we will have to
> > add *yet another* file that is written (in the "we already have
> > comments in the commit message" conditional block in
> > error_failed_squash())...
> 
> I wonder if creating the file in update_squash_messages() rather than
> error_failed_squash() would be a better approach as then it is easy to
> only create rebase_path_amend_type() when there has already been a
> squash or fixup.  The file is removed in the loop that picks commits in
> pick_commits() so it would be cleaned up at the beginning of the next
> pick if it's not needed.

That would be a good idea in general, but I think we have to take care of
the following scenario:

	pick	<- succeeds
	squash	<- succeeds
	fixup	<- fails, will be skipped

In this case, we do need to open the editor. But in this scenario, we do
not:

	pick	<- succeeds
	fixup	<- succeeds
	squash	<- fails, will be skipped

If we write the amend-type file in update_squash_messages(), we would
write "squash" into it in both cases. My hope was to somehow avoid that.

I just realized that the current iteration does not fulfill that goal, as
the message-fixup file would be long gone by the time
error_failed_squash() was called in the latter example.

Also, I realized something else: my previous work-around for the
GETTEXT_POISON case (where I fail gently when a commit message does not
contain the "This is a combination of #<count> commits" count in ASCII)
would be much superior if it simply would not abuse the comment in the
commit message, but had a robust, non-l18ned way to count the fixup/squash
commits.

My current thinking is to reconcile both problems by shunning the
amend-type and instead just record the sequence of fixup/squash commits
that went into HEAD, in a new file, say, current-fixups.

To answer the question how many commit messages are combined, I then
simply need to count the lines in that file.

To answer the question whether a skipped fixup/squash requires the editor
to be launched, I can simply look whether there is a "squash" line
(ignoring the last line).

Oh, and I also forgot to test whether this is the "final fixup". If we are
skipping a "fixup" in the middle of a chain, there is no need to clean the
commit message to begin with.

This will take a while... ;-)

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 0/4] rebase -i: avoid stale "# This is a combinationof" in commit messages
  2018-04-26  9:51                         ` Johannes Schindelin
@ 2018-04-26 10:52                           ` Phillip Wood
  0 siblings, 0 replies; 412+ messages in thread
From: Phillip Wood @ 2018-04-26 10:52 UTC (permalink / raw)
  To: Johannes Schindelin, phillip.wood
  Cc: Stefan Beller, git, Junio C Hamano, Eric Sunshine

On 26/04/18 10:51, Johannes Schindelin wrote:
> Hi Phillip,
> 
> On Wed, 25 Apr 2018, Phillip Wood wrote:
> 
>> On 25/04/18 13:48, Johannes Schindelin wrote:
>>>
>>> On Mon, 23 Apr 2018, Phillip Wood wrote:
>>>
>>>> On 23/04/18 19:11, Stefan Beller wrote:
>>>>>
>>>>> On Sat, Apr 21, 2018 at 12:34 AM, Johannes Schindelin
>>>>> <johannes.schindelin@gmx.de> wrote:
>>>>>> Eric Sunshine pointed out that I had such a commit message in
>>>>>> https://public-inbox.org/git/CAPig+cRrS0_nYJJY=O6cboV630sNQHPV5QGrQdD8MW-sYzNFGQ@mail.gmail.com/
>>>>>> and I went on a hunt to figure out how the heck this happened.
>>>>>>
>>>>>> Turns out that if there is a fixup/squash chain where the *last*
>>>>>> command fails with merge conflicts, and we either --skip ahead
>>>>>> or resolve the conflict to a clean tree and then --continue, our
>>>>>> code does not do a final cleanup.
>>>>>>
>>>>>> Contrary to my initial gut feeling, this bug was not introduced
>>>>>> by my rewrite in C of the core parts of rebase -i, but it looks
>>>>>> to me as if that bug was with us for a very long time (at least
>>>>>> the --skip part).
>>>>>>
>>>>>> The developer (read: user of rebase -i) in me says that we would
>>>>>> want to fast-track this, but the author of rebase -i in me says
>>>>>> that we should be cautious and cook this in `next` for a while.
>>>>>
>>>>> I looked through the patches again and think this series is good
>>>>> to go.
>>>>
>>>> I've just realized I commented on an outdated version as the new
>>>> version was posted there rather than as a reply to v1. I've just
>>>> looked through it and I'm not sure it addresses the unnecessary
>>>> editing of the commit message of the previous commit if a single
>>>> squash command is skipped as outlined in
>>>> https://public-inbox.org/git/b6512eae-e214-9699-4d69-77117a0daec3@talktalk.net/
>>>
>>> I have not forgotten about this! I simply did not find the time yet,
>>> is all...
>>
>> I wondered if that was the case but I wanted to check as I wasn't sure
>> if you'd seen the original message as it was on an obsolete version of
>> the series
>>
>>> The patch series still has not been merged to `next`, but I plan on
>>> working on your suggested changes as an add-on commit anyway. I am not
>>> quite sure yet how I want to handle the "avoid running commit for the
>>> first fixup/squash in the series" problem, but I think we will have to
>>> add *yet another* file that is written (in the "we already have
>>> comments in the commit message" conditional block in
>>> error_failed_squash())...
>>
>> I wonder if creating the file in update_squash_messages() rather than
>> error_failed_squash() would be a better approach as then it is easy to
>> only create rebase_path_amend_type() when there has already been a
>> squash or fixup.  The file is removed in the loop that picks commits in
>> pick_commits() so it would be cleaned up at the beginning of the next
>> pick if it's not needed.
> 
> That would be a good idea in general, but I think we have to take care of
> the following scenario:
> 
> 	pick	<- succeeds
> 	squash	<- succeeds
> 	fixup	<- fails, will be skipped
> 
> In this case, we do need to open the editor. But in this scenario, we do
> not:
> 
> 	pick	<- succeeds
> 	fixup	<- succeeds
> 	squash	<- fails, will be skipped
> 
> If we write the amend-type file in update_squash_messages(), we would
> write "squash" into it in both cases. My hope was to somehow avoid that.

Good point, I'd not thought of that

> I just realized that the current iteration does not fulfill that goal, as
> the message-fixup file would be long gone by the time
> error_failed_squash() was called in the latter example.
> 
> Also, I realized something else: my previous work-around for the
> GETTEXT_POISON case (where I fail gently when a commit message does not
> contain the "This is a combination of #<count> commits" count in ASCII)
> would be much superior if it simply would not abuse the comment in the
> commit message, but had a robust, non-l18ned way to count the fixup/squash
> commits.
> 
> My current thinking is to reconcile both problems by shunning the
> amend-type and instead just record the sequence of fixup/squash commits
> that went into HEAD, in a new file, say, current-fixups.
> 
> To answer the question how many commit messages are combined, I then
> simply need to count the lines in that file.
> 
> To answer the question whether a skipped fixup/squash requires the editor
> to be launched, I can simply look whether there is a "squash" line
> (ignoring the last line).

That sounds like a good plan, keeping count of the fixup/squash without
having to parse the last message is a good idea.

> Oh, and I also forgot to test whether this is the "final fixup". If we are
> skipping a "fixup" in the middle of a chain, there is no need to clean the
> commit message to begin with.
> 
> This will take a while... ;-)

Yes, it sounds like quite a bit of work, but it will be a very
worthwhile improvement.

Thanks

Phillip

> Ciao,
> Dscho
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v4 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages
  2018-04-20  5:38             ` Eric Sunshine
                                 ` (2 preceding siblings ...)
  2018-04-21  7:34               ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
@ 2018-04-27 20:48               ` Johannes Schindelin
  2018-04-27 20:48                 ` [PATCH v4 1/4] rebase -i: demonstrate bugs with fixup!/squash! " Johannes Schindelin
                                   ` (3 more replies)
  3 siblings, 4 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-27 20:48 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Eric Sunshine,
	Stefan Beller, Phillip Wood

Eric Sunshine pointed out that I had such a commit message in
https://public-inbox.org/git/CAPig+cRrS0_nYJJY=O6cboV630sNQHPV5QGrQdD8MW-sYzNFGQ@mail.gmail.com/
and I went on a hunt to figure out how the heck this happened.

Turns out that if there is a fixup/squash chain where the *last* command
fails with merge conflicts, and we either --skip ahead or resolve the
conflict to a clean tree and then --continue, our code does not do a
final cleanup.

Contrary to my initial gut feeling, this bug was not introduced by my
rewrite in C of the core parts of rebase -i, but it looks to me as if
that bug was with us for a very long time (at least the --skip part).

The developer (read: user of rebase -i) in me says that we would want to
fast-track this, but the author of rebase -i in me says that we should
be cautious and cook this in `next` for a while.

Fixes since v3 (thanks, Phillip, for the really fruitful discussion!):

- We now avoid using the commit message prepared for the skipped
  fixup/squash.

- Replaced the "rebase -i: Handle "combination of <n> commits" with
  GETTEXT_POISON" patch by a *real* fix instead of a work-around: Instead
  of parsing the first line of the commit message and punting when it is
  missing an ASCII-encoded number, we determine <n> separately
  (independent from any localized text).

- Fixed quite a couple more corner cases, using the `current-fixups`
  file introduced for the GETTEXT_POISON fix:

  * we only need to re-commit if this was the final fixup/squash in the
    fixup/squash chain,

  * we only need to commit interactively if there was *any* non-skipped
    squash,

  * if the fixup/squash chain continues, the <N> was incorrect in the
    "This is a combination of <N> commits" comment in the intermediate
    commit message (it included the now-skipped commits), and

  * even if a filed fixup/squash in the middle of a fixup/squash chain
    failed, and its merge conflicts were resolved and committed, the
    "This is a combination of <N> commits" comment was incorrect: we
    had already deleted message-fixup and message-squash, so the next
    update_squash_message() would mistakenly assume that we were
    starting afresh. Worse: if only fixup commands were remaining, but
    there had been a squash command, we would retain the "squash!" line
    in the commit message and not give the user a chance to clean things
    up in the final fixup!


Johannes Schindelin (4):
  rebase -i: demonstrate bugs with fixup!/squash! commit messages
  rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON
  sequencer: always commit without editing when asked for
  rebase --skip: clean up commit message after a failed fixup/squash

 sequencer.c                | 193 ++++++++++++++++++++++++++++---------
 sequencer.h                |   6 +-
 t/t3418-rebase-continue.sh |  49 ++++++++++
 3 files changed, 200 insertions(+), 48 deletions(-)


base-commit: 1f1cddd558b54bb0ce19c8ace353fd07b758510d
Published-As: https://github.com/dscho/git/releases/tag/clean-msg-after-fixup-continue-v4
Fetch-It-Via: git fetch https://github.com/dscho/git clean-msg-after-fixup-continue-v4

Interdiff vs v3:
 diff --git a/sequencer.c b/sequencer.c
 index e1efb0ebf31..cec180714ef 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -74,13 +74,6 @@ static GIT_PATH_FUNC(rebase_path_message, "rebase-merge/message")
   * previous commit and from the first squash/fixup commit are written
   * to it. The commit message for each subsequent squash/fixup commit
   * is appended to the file as it is processed.
 - *
 - * The first line of the file is of the form
 - *     # This is a combination of $count commits.
 - * where $count is the number of commits whose messages have been
 - * written to the file so far (including the initial "pick" commit).
 - * Each time that a commit message is processed, this line is read and
 - * updated. It is deleted just before the combined commit is made.
   */
  static GIT_PATH_FUNC(rebase_path_squash_msg, "rebase-merge/message-squash")
  /*
 @@ -91,6 +84,11 @@ static GIT_PATH_FUNC(rebase_path_squash_msg, "rebase-merge/message-squash")
   * commit without opening the editor.)
   */
  static GIT_PATH_FUNC(rebase_path_fixup_msg, "rebase-merge/message-fixup")
 +/*
 + * This file contains the list fixup/squash commands that have been
 + * accumulated into message-fixup or message-squash so far.
 + */
 +static GIT_PATH_FUNC(rebase_path_current_fixups, "rebase-merge/current-fixups")
  /*
   * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
   * GIT_AUTHOR_DATE that will be used for the commit that is currently
 @@ -106,13 +104,6 @@ static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
   * command is processed, this file is deleted.
   */
  static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
 -/*
 - * If there was a merge conflict in a fixup/squash series, we need to
 - * record the type so that a `git rebase --skip` can clean up the commit
 - * message as appropriate. This file will contain that type (`fixup` or
 - * `squash`), and not exist otherwise.
 - */
 -static GIT_PATH_FUNC(rebase_path_amend_type, "rebase-merge/amend-type")
  /*
   * When we stop at a given patch via the "edit" command, this file contains
   * the abbreviated commit name of the corresponding patch.
 @@ -260,6 +251,7 @@ int sequencer_remove_state(struct replay_opts *opts)
  	for (i = 0; i < opts->xopts_nr; i++)
  		free(opts->xopts[i]);
  	free(opts->xopts);
 +	strbuf_release(&opts->current_fixups);
  
  	strbuf_addstr(&dir, get_dir(opts));
  	remove_dir_recursively(&dir, 0);
 @@ -725,6 +717,8 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
  		argv_array_pushf(&cmd.args, "-S%s", opts->gpg_sign);
  	if (defmsg)
  		argv_array_pushl(&cmd.args, "-F", defmsg, NULL);
 +	else if (!(flags & EDIT_MSG))
 +		argv_array_pushl(&cmd.args, "-C", "HEAD", NULL);
  	if ((flags & CLEANUP_MSG))
  		argv_array_push(&cmd.args, "--cleanup=strip");
  	if ((flags & EDIT_MSG))
 @@ -1336,33 +1330,23 @@ static int update_squash_messages(enum todo_command command,
  		struct commit *commit, struct replay_opts *opts)
  {
  	struct strbuf buf = STRBUF_INIT;
 -	int count, res;
 +	int res;
  	const char *message, *body;
  
 -	if (file_exists(rebase_path_squash_msg())) {
 +	if (opts->current_fixup_count > 0) {
  		struct strbuf header = STRBUF_INIT;
 -		char *eol, *p;
 +		char *eol;
  
 -		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
 +		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 9) <= 0)
  			return error(_("could not read '%s'"),
  				rebase_path_squash_msg());
  
 -		p = buf.buf + 1;
 -		eol = strchrnul(buf.buf, '\n');
 -		if (buf.buf[0] != comment_line_char ||
 -		    (p += strcspn(p, "0123456789\n")) == eol)
 -			count = -1;
 -		else
 -			count = strtol(p, NULL, 10);
 +		eol = buf.buf[0] != comment_line_char ?
 +			buf.buf : strchrnul(buf.buf, '\n');
  
  		strbuf_addf(&header, "%c ", comment_line_char);
 -		if (count < 1)
 -			strbuf_addf(&header, _("This is a combination of "
 -					       "several commits."));
 -		else
 -			strbuf_addf(&header,
 -				    _("This is a combination of %d commits."),
 -				    ++count);
 +		strbuf_addf(&header, _("This is a combination of %d commits."),
 +			    opts->current_fixup_count + 2);
  		strbuf_splice(&buf, 0, eol - buf.buf, header.buf, header.len);
  		strbuf_release(&header);
  	} else {
 @@ -1385,10 +1369,8 @@ static int update_squash_messages(enum todo_command command,
  				     rebase_path_fixup_msg());
  		}
  
 -		count = 2;
  		strbuf_addf(&buf, "%c ", comment_line_char);
 -		strbuf_addf(&buf, _("This is a combination of %d commits."),
 -			    count);
 +		strbuf_addf(&buf, _("This is a combination of %d commits."), 2);
  		strbuf_addf(&buf, "\n%c ", comment_line_char);
  		strbuf_addstr(&buf, _("This is the 1st commit message:"));
  		strbuf_addstr(&buf, "\n\n");
 @@ -1405,22 +1387,14 @@ static int update_squash_messages(enum todo_command command,
  	if (command == TODO_SQUASH) {
  		unlink(rebase_path_fixup_msg());
  		strbuf_addf(&buf, "\n%c ", comment_line_char);
 -		if (count < 2)
 -			strbuf_addf(&buf, _("This is the next commit "
 -					    "message:"));
 -		else
 -			strbuf_addf(&buf, _("This is the commit message #%d:"),
 -				    count);
 +		strbuf_addf(&buf, _("This is the commit message #%d:"),
 +			    ++opts->current_fixup_count);
  		strbuf_addstr(&buf, "\n\n");
  		strbuf_addstr(&buf, body);
  	} else if (command == TODO_FIXUP) {
  		strbuf_addf(&buf, "\n%c ", comment_line_char);
 -		if (count < 2)
 -			strbuf_addf(&buf, _("The next commit message will be "
 -					    "skipped:"));
 -		else
 -			strbuf_addf(&buf, _("The commit message #%d will be "
 -					    "skipped:"), count);
 +		strbuf_addf(&buf, _("The commit message #%d will be skipped:"),
 +			    ++opts->current_fixup_count);
  		strbuf_addstr(&buf, "\n\n");
  		strbuf_add_commented_lines(&buf, body, strlen(body));
  	} else
 @@ -1429,6 +1403,17 @@ static int update_squash_messages(enum todo_command command,
  
  	res = write_message(buf.buf, buf.len, rebase_path_squash_msg(), 0);
  	strbuf_release(&buf);
 +
 +	if (!res) {
 +		strbuf_addf(&opts->current_fixups, "%s%s %s",
 +			    opts->current_fixups.len ? "\n" : "",
 +			    command_to_string(command),
 +			    oid_to_hex(&commit->object.oid));
 +		res = write_message(opts->current_fixups.buf,
 +				    opts->current_fixups.len,
 +				    rebase_path_current_fixups(), 0);
 +	}
 +
  	return res;
  }
  
 @@ -1691,6 +1676,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  	if (!res && final_fixup) {
  		unlink(rebase_path_fixup_msg());
  		unlink(rebase_path_squash_msg());
 +		unlink(rebase_path_current_fixups());
 +		strbuf_reset(&opts->current_fixups);
 +		opts->current_fixup_count = 0;
  	}
  
  leave:
 @@ -2067,6 +2055,16 @@ static int read_populate_opts(struct replay_opts *opts)
  		read_strategy_opts(opts, &buf);
  		strbuf_release(&buf);
  
 +		if (read_oneliner(&opts->current_fixups,
 +				  rebase_path_current_fixups(), 1)) {
 +			const char *p = opts->current_fixups.buf;
 +			opts->current_fixup_count = 1;
 +			while ((p = strchr(p, '\n'))) {
 +				opts->current_fixup_count++;
 +				p++;
 +			}
 +		}
 +
  		return 0;
  	}
  
 @@ -2413,21 +2411,9 @@ static int error_with_patch(struct commit *commit,
  static int error_failed_squash(struct commit *commit,
  	struct replay_opts *opts, int subject_len, const char *subject)
  {
 -	const char *amend_type = "squash";
 -
 -	if (rename(rebase_path_squash_msg(), rebase_path_message()))
 -		return error(_("could not rename '%s' to '%s'"),
 +	if (copy_file(rebase_path_message(), rebase_path_squash_msg(), 0666))
 +		return error(_("could not copy '%s' to '%s'"),
  			rebase_path_squash_msg(), rebase_path_message());
 -
 -	if (file_exists(rebase_path_fixup_msg())) {
 -		unlink(rebase_path_fixup_msg());
 -		amend_type = "fixup";
 -	}
 -	if (write_message(amend_type, strlen(amend_type),
 -		       rebase_path_amend_type(), 0))
 -		return error(_("could not write '%s'"),
 -			     rebase_path_amend_type());
 -
  	unlink(git_path_merge_msg());
  	if (copy_file(git_path_merge_msg(), rebase_path_message(), 0666))
  		return error(_("could not copy '%s' to '%s'"),
 @@ -2604,7 +2590,6 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			unlink(rebase_path_author_script());
  			unlink(rebase_path_stopped_sha());
  			unlink(rebase_path_amend());
 -			unlink(rebase_path_amend_type());
  			delete_ref(NULL, "REBASE_HEAD", NULL, REF_NO_DEREF);
  		}
  		if (item->command <= TODO_SQUASH) {
 @@ -2794,9 +2779,11 @@ static int continue_single_pick(void)
  	return run_command_v_opt(argv, RUN_GIT_CMD);
  }
  
 -static int commit_staged_changes(struct replay_opts *opts)
 +static int commit_staged_changes(struct replay_opts *opts,
 +				 struct todo_list *todo_list)
  {
 -	unsigned int flags = ALLOW_EMPTY | EDIT_MSG, is_fixup = 0, is_clean;
 +	unsigned int flags = ALLOW_EMPTY | EDIT_MSG;
 +	unsigned int final_fixup = 0, is_clean;
  
  	if (has_unstaged_changes(1))
  		return error(_("cannot rebase: You have unstaged changes."));
 @@ -2819,21 +2806,69 @@ static int commit_staged_changes(struct replay_opts *opts)
  				       "working tree. Please, commit them\n"
  				       "first and then run 'git rebase "
  				       "--continue' again."));
 -		if (is_clean && !oidcmp(&head, &to_amend)) {
 -			strbuf_reset(&rev);
 +		/*
 +		 * When skipping a failed fixup/squash, we need to edit the
 +		 * commit message, the current fixup list and count, and if it
 +		 * was the last fixup/squash in the chain, we need to clean up
 +		 * the commit message and if there was a squash, let the user
 +		 * edit it.
 +		 */
 +		if (is_clean && !oidcmp(&head, &to_amend) &&
 +		    opts->current_fixup_count > 0 &&
 +		    file_exists(rebase_path_stopped_sha())) {
 +			const char *p = opts->current_fixups.buf;
 +			int len = opts->current_fixups.len;
 +
 +			opts->current_fixup_count--;
 +			if (!len)
 +				BUG("Incorrect current_fixups:\n%s", p);
 +			while (len && p[len - 1] != '\n')
 +				len--;
 +			strbuf_setlen(&opts->current_fixups, len);
 +			if (write_message(p, len, rebase_path_current_fixups(),
 +					  0) < 0)
 +				return error(_("could not write file: '%s'"),
 +					     rebase_path_current_fixups());
 +
  			/*
 -			 * Clean tree, but we may need to finalize a
 -			 * fixup/squash chain. A failed fixup/squash leaves the
 -			 * file amend-type in rebase-merge/; It is okay if that
 -			 * file is missing, in which case there is no such
 -			 * chain to finalize.
 +			 * If a fixup/squash in a fixup/squash chain failed, the
 +			 * commit message is already correct, no need to commit
 +			 * it again.
 +			 *
 +			 * Only if it is the final command in the fixup/squash
 +			 * chain, and only if the chain is longer than a single
 +			 * fixup/squash command (which was just skipped), do we
 +			 * actually need to re-commit with a cleaned up commit
 +			 * message.
  			 */
 -			read_oneliner(&rev, rebase_path_amend_type(), 0);
 -			if (!strcmp("squash", rev.buf))
 -				is_fixup = TODO_SQUASH;
 -			else if (!strcmp("fixup", rev.buf)) {
 -				is_fixup = TODO_FIXUP;
 -				flags = (flags & ~EDIT_MSG) | CLEANUP_MSG;
 +			if (opts->current_fixup_count > 0 &&
 +			    !is_fixup(peek_command(todo_list, 0))) {
 +				final_fixup = 1;
 +				/*
 +				 * If there was not a single "squash" in the
 +				 * chain, we only need to clean up the commit
 +				 * message, no need to bother the user with
 +				 * opening the commit message in the editor.
 +				 */
 +				if (!starts_with(p, "squash ") &&
 +				    !strstr(p, "\nsquash "))
 +					flags = (flags & ~EDIT_MSG) | CLEANUP_MSG;
 +			} else if (is_fixup(peek_command(todo_list, 0))) {
 +				/*
 +				 * We need to update the squash message to skip
 +				 * the latest commit message.
 +				 */
 +				struct commit *commit;
 +				const char *path = rebase_path_squash_msg();
 +
 +				if (parse_head(&commit) ||
 +				    !(p = get_commit_buffer(commit, NULL)) ||
 +				    write_message(p, strlen(p), path, 0)) {
 +					unuse_commit_buffer(commit, p);
 +					return error(_("could not write file: "
 +						       "'%s'"), path);
 +				}
 +				unuse_commit_buffer(commit, p);
  			}
  		}
  
 @@ -2841,18 +2876,32 @@ static int commit_staged_changes(struct replay_opts *opts)
  		flags |= AMEND_MSG;
  	}
  
 -	if (is_clean && !is_fixup) {
 +	if (is_clean) {
  		const char *cherry_pick_head = git_path_cherry_pick_head();
  
  		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
  			return error(_("could not remove CHERRY_PICK_HEAD"));
 -		return 0;
 +		if (!final_fixup)
 +			return 0;
  	}
  
 -	if (run_git_commit(rebase_path_message(), opts, flags))
 +	if (run_git_commit(final_fixup ? NULL : rebase_path_message(),
 +			   opts, flags))
  		return error(_("could not commit staged changes."));
  	unlink(rebase_path_amend());
 -	unlink(rebase_path_amend_type());
 +	if (final_fixup) {
 +		unlink(rebase_path_fixup_msg());
 +		unlink(rebase_path_squash_msg());
 +	}
 +	if (opts->current_fixup_count > 0) {
 +		/*
 +		 * Whether final fixup or not, we just cleaned up the commit
 +		 * message...
 +		 */
 +		unlink(rebase_path_current_fixups());
 +		strbuf_reset(&opts->current_fixups);
 +		opts->current_fixup_count = 0;
 +	}
  	return 0;
  }
  
 @@ -2864,14 +2913,16 @@ int sequencer_continue(struct replay_opts *opts)
  	if (read_and_refresh_cache(opts))
  		return -1;
  
 +	if (read_populate_opts(opts))
 +		return -1;
  	if (is_rebase_i(opts)) {
 -		if (commit_staged_changes(opts))
 +		if ((res = read_populate_todo(&todo_list, opts)))
 +			goto release_todo_list;
 +		if (commit_staged_changes(opts, &todo_list))
  			return -1;
  	} else if (!file_exists(get_todo_path(opts)))
  		return continue_single_pick();
 -	if (read_populate_opts(opts))
 -		return -1;
 -	if ((res = read_populate_todo(&todo_list, opts)))
 +	else if ((res = read_populate_todo(&todo_list, opts)))
  		goto release_todo_list;
  
  	if (!is_rebase_i(opts)) {
 diff --git a/sequencer.h b/sequencer.h
 index e45b178dfc4..1898158c52d 100644
 --- a/sequencer.h
 +++ b/sequencer.h
 @@ -44,10 +44,14 @@ struct replay_opts {
  	char **xopts;
  	size_t xopts_nr, xopts_alloc;
  
 +	/* Used by fixup/squash */
 +	struct strbuf current_fixups;
 +	int current_fixup_count;
 +
  	/* Only used by REPLAY_NONE */
  	struct rev_info *revs;
  };
 -#define REPLAY_OPTS_INIT { -1 }
 +#define REPLAY_OPTS_INIT { .action = -1, .current_fixups = STRBUF_INIT }
  
  /* Call this to setup defaults before parsing command line options */
  void sequencer_init_config(struct replay_opts *opts);
 diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
 index 693f92409ec..03bf1b8a3b3 100755
 --- a/t/t3418-rebase-continue.sh
 +++ b/t/t3418-rebase-continue.sh
 @@ -95,19 +95,46 @@ test_expect_success '--skip after failed fixup cleans commit message' '
  	test_commit "fixup! wants-fixup" wants-fixup.t 1 wants-fixup-1 &&
  	test_commit "fixup! wants-fixup" wants-fixup.t 2 wants-fixup-2 &&
  	test_commit "fixup! wants-fixup" wants-fixup.t 3 wants-fixup-3 &&
 -	test_must_fail env FAKE_LINES="1 fixup 2 fixup 4" \
 +	test_must_fail env FAKE_LINES="1 fixup 2 squash 4" \
  		git rebase -i HEAD~4 &&
  
  	: now there is a conflict, and comments in the commit message &&
  	git show HEAD >out &&
 -	test_i18ngrep "This is a combination of" out &&
 +	grep "fixup! wants-fixup" out &&
  
  	: skip and continue &&
 -	git rebase --skip &&
 +	echo "cp \"\$1\" .git/copy.txt" | write_script copy-editor.sh &&
 +	(test_set_editor "$PWD/copy-editor.sh" && git rebase --skip) &&
 +
 +	: the user should not have had to edit the commit message &&
 +	test_path_is_missing .git/copy.txt &&
  
  	: now the comments in the commit message should have been cleaned up &&
  	git show HEAD >out &&
 -	test_i18ngrep ! "This is a combination of" out
 +	! grep "fixup! wants-fixup" out &&
 +
 +	: now, let us ensure that "squash" is handled correctly &&
 +	git reset --hard wants-fixup-3 &&
 +	test_must_fail env FAKE_LINES="1 squash 4 squash 2 squash 4" \
 +		git rebase -i HEAD~4 &&
 +
 +	: the first squash failed, but there are two more in the chain &&
 +	(test_set_editor "$PWD/copy-editor.sh" &&
 +	 test_must_fail git rebase --skip) &&
 +
 +	: not the final squash, no need to edit the commit message &&
 +	test_path_is_missing .git/copy.txt &&
 +
 +	: The first squash was skipped, therefore: &&
 +	git show HEAD >out &&
 +	test_i18ngrep "# This is a combination of 2 commits" out &&
 +
 +	(test_set_editor "$PWD/copy-editor.sh" && git rebase --skip) &&
 +	git show HEAD >out &&
 +	test_i18ngrep ! "# This is a combination" out &&
 +
 +	: Final squash failed, but there was still a squash &&
 +	test_i18ngrep "# This is a combination of 2 commits" .git/copy.txt
  '
  
  test_expect_success 'setup rerere database' '
-- 
2.17.0.windows.1.33.gfcbb1fa0445


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v4 1/4] rebase -i: demonstrate bugs with fixup!/squash! commit messages
  2018-04-27 20:48               ` [PATCH v4 " Johannes Schindelin
@ 2018-04-27 20:48                 ` Johannes Schindelin
  2018-04-27 20:48                 ` [PATCH v4 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON Johannes Schindelin
                                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-27 20:48 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Eric Sunshine,
	Stefan Beller, Phillip Wood

When multiple fixup/squash commands are processed and the last one
causes merge conflicts and is skipped, we leave the "This is a
combination of ..." comments in the commit message.

Noticed by Eric Sunshine.

This regression test also demonstrates that we rely on the localized
version of

	# This is a combination of <number> commits

to contain the <number> in ASCII, which breaks under GETTEXT_POISON.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t3418-rebase-continue.sh | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
index 9214d0bb511..3874f187246 100755
--- a/t/t3418-rebase-continue.sh
+++ b/t/t3418-rebase-continue.sh
@@ -88,6 +88,28 @@ test_expect_success 'rebase passes merge strategy options correctly' '
 	git rebase --continue
 '
 
+test_expect_failure '--skip after failed fixup cleans commit message' '
+	test_when_finished "test_might_fail git rebase --abort" &&
+	git checkout -b with-conflicting-fixup &&
+	test_commit wants-fixup &&
+	test_commit "fixup! wants-fixup" wants-fixup.t 1 wants-fixup-1 &&
+	test_commit "fixup! wants-fixup" wants-fixup.t 2 wants-fixup-2 &&
+	test_commit "fixup! wants-fixup" wants-fixup.t 3 wants-fixup-3 &&
+	test_must_fail env FAKE_LINES="1 fixup 2 fixup 4" \
+		git rebase -i HEAD~4 &&
+
+	: now there is a conflict, and comments in the commit message &&
+	git show HEAD >out &&
+	grep "fixup! wants-fixup" out &&
+
+	: skip and continue &&
+	git rebase --skip &&
+
+	: now the comments in the commit message should have been cleaned up &&
+	git show HEAD >out &&
+	! grep "fixup! wants-fixup" out
+'
+
 test_expect_success 'setup rerere database' '
 	rm -fr .git/rebase-* &&
 	git reset --hard commit-new-file-F3-on-topic-branch &&
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON
  2018-04-27 20:48               ` [PATCH v4 " Johannes Schindelin
  2018-04-27 20:48                 ` [PATCH v4 1/4] rebase -i: demonstrate bugs with fixup!/squash! " Johannes Schindelin
@ 2018-04-27 20:48                 ` Johannes Schindelin
  2018-04-27 20:48                 ` [PATCH v4 3/4] sequencer: always commit without editing when asked for Johannes Schindelin
  2018-04-27 20:48                 ` [PATCH v4 4/4] rebase --skip: clean up commit message after a failed fixup/squash Johannes Schindelin
  3 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-27 20:48 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Eric Sunshine,
	Stefan Beller, Phillip Wood

We previously relied on the localized versions of

	# This is a combination of <N> commits

(which we write into the commit messages during fixup/squash chains)
to contain <N> encoded in ASCII.

This is not true in general, and certainly not true when compiled with
GETTEXT_POISON=TryToKillMe, as demonstrated by the regression test we
just introduced in t3418.

So let's decouple keeping track of the count from the (localized) commit
messages by introducing a new file called 'current-fixups' that keeps
track of the current fixup/squash chain. This file contains a bit more
than just the count (it contains a list of "fixup <commit>"/"squash
<commit>" lines). This is done on purpose, as it will come in handy for
a fix for the bug where `git rebase --skip` on a final fixup/squash will
leave the commit message in limbo.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 78 ++++++++++++++++++++++++++++++-----------------------
 sequencer.h |  6 ++++-
 2 files changed, 49 insertions(+), 35 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 5e3a50fafc9..d2e6f33023d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -74,13 +74,6 @@ static GIT_PATH_FUNC(rebase_path_message, "rebase-merge/message")
  * previous commit and from the first squash/fixup commit are written
  * to it. The commit message for each subsequent squash/fixup commit
  * is appended to the file as it is processed.
- *
- * The first line of the file is of the form
- *     # This is a combination of $count commits.
- * where $count is the number of commits whose messages have been
- * written to the file so far (including the initial "pick" commit).
- * Each time that a commit message is processed, this line is read and
- * updated. It is deleted just before the combined commit is made.
  */
 static GIT_PATH_FUNC(rebase_path_squash_msg, "rebase-merge/message-squash")
 /*
@@ -91,6 +84,11 @@ static GIT_PATH_FUNC(rebase_path_squash_msg, "rebase-merge/message-squash")
  * commit without opening the editor.)
  */
 static GIT_PATH_FUNC(rebase_path_fixup_msg, "rebase-merge/message-fixup")
+/*
+ * This file contains the list fixup/squash commands that have been
+ * accumulated into message-fixup or message-squash so far.
+ */
+static GIT_PATH_FUNC(rebase_path_current_fixups, "rebase-merge/current-fixups")
 /*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
@@ -253,6 +251,7 @@ int sequencer_remove_state(struct replay_opts *opts)
 	for (i = 0; i < opts->xopts_nr; i++)
 		free(opts->xopts[i]);
 	free(opts->xopts);
+	strbuf_release(&opts->current_fixups);
 
 	strbuf_addstr(&dir, get_dir(opts));
 	remove_dir_recursively(&dir, 0);
@@ -1329,34 +1328,23 @@ static int update_squash_messages(enum todo_command command,
 		struct commit *commit, struct replay_opts *opts)
 {
 	struct strbuf buf = STRBUF_INIT;
-	int count, res;
+	int res;
 	const char *message, *body;
 
-	if (file_exists(rebase_path_squash_msg())) {
+	if (opts->current_fixup_count > 0) {
 		struct strbuf header = STRBUF_INIT;
-		char *eol, *p;
+		char *eol;
 
-		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
+		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 9) <= 0)
 			return error(_("could not read '%s'"),
 				rebase_path_squash_msg());
 
-		p = buf.buf + 1;
-		eol = strchrnul(buf.buf, '\n');
-		if (buf.buf[0] != comment_line_char ||
-		    (p += strcspn(p, "0123456789\n")) == eol)
-			return error(_("unexpected 1st line of squash message:"
-				       "\n\n\t%.*s"),
-				     (int)(eol - buf.buf), buf.buf);
-		count = strtol(p, NULL, 10);
-
-		if (count < 1)
-			return error(_("invalid 1st line of squash message:\n"
-				       "\n\t%.*s"),
-				     (int)(eol - buf.buf), buf.buf);
+		eol = buf.buf[0] != comment_line_char ?
+			buf.buf : strchrnul(buf.buf, '\n');
 
 		strbuf_addf(&header, "%c ", comment_line_char);
-		strbuf_addf(&header,
-			    _("This is a combination of %d commits."), ++count);
+		strbuf_addf(&header, _("This is a combination of %d commits."),
+			    opts->current_fixup_count + 2);
 		strbuf_splice(&buf, 0, eol - buf.buf, header.buf, header.len);
 		strbuf_release(&header);
 	} else {
@@ -1379,10 +1367,8 @@ static int update_squash_messages(enum todo_command command,
 				     rebase_path_fixup_msg());
 		}
 
-		count = 2;
 		strbuf_addf(&buf, "%c ", comment_line_char);
-		strbuf_addf(&buf, _("This is a combination of %d commits."),
-			    count);
+		strbuf_addf(&buf, _("This is a combination of %d commits."), 2);
 		strbuf_addf(&buf, "\n%c ", comment_line_char);
 		strbuf_addstr(&buf, _("This is the 1st commit message:"));
 		strbuf_addstr(&buf, "\n\n");
@@ -1399,13 +1385,14 @@ static int update_squash_messages(enum todo_command command,
 	if (command == TODO_SQUASH) {
 		unlink(rebase_path_fixup_msg());
 		strbuf_addf(&buf, "\n%c ", comment_line_char);
-		strbuf_addf(&buf, _("This is the commit message #%d:"), count);
+		strbuf_addf(&buf, _("This is the commit message #%d:"),
+			    ++opts->current_fixup_count);
 		strbuf_addstr(&buf, "\n\n");
 		strbuf_addstr(&buf, body);
 	} else if (command == TODO_FIXUP) {
 		strbuf_addf(&buf, "\n%c ", comment_line_char);
 		strbuf_addf(&buf, _("The commit message #%d will be skipped:"),
-			    count);
+			    ++opts->current_fixup_count);
 		strbuf_addstr(&buf, "\n\n");
 		strbuf_add_commented_lines(&buf, body, strlen(body));
 	} else
@@ -1414,6 +1401,17 @@ static int update_squash_messages(enum todo_command command,
 
 	res = write_message(buf.buf, buf.len, rebase_path_squash_msg(), 0);
 	strbuf_release(&buf);
+
+	if (!res) {
+		strbuf_addf(&opts->current_fixups, "%s%s %s",
+			    opts->current_fixups.len ? "\n" : "",
+			    command_to_string(command),
+			    oid_to_hex(&commit->object.oid));
+		res = write_message(opts->current_fixups.buf,
+				    opts->current_fixups.len,
+				    rebase_path_current_fixups(), 0);
+	}
+
 	return res;
 }
 
@@ -1676,6 +1674,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	if (!res && final_fixup) {
 		unlink(rebase_path_fixup_msg());
 		unlink(rebase_path_squash_msg());
+		unlink(rebase_path_current_fixups());
+		strbuf_reset(&opts->current_fixups);
+		opts->current_fixup_count = 0;
 	}
 
 leave:
@@ -2052,6 +2053,16 @@ static int read_populate_opts(struct replay_opts *opts)
 		read_strategy_opts(opts, &buf);
 		strbuf_release(&buf);
 
+		if (read_oneliner(&opts->current_fixups,
+				  rebase_path_current_fixups(), 1)) {
+			const char *p = opts->current_fixups.buf;
+			opts->current_fixup_count = 1;
+			while ((p = strchr(p, '\n'))) {
+				opts->current_fixup_count++;
+				p++;
+			}
+		}
+
 		return 0;
 	}
 
@@ -2398,10 +2409,9 @@ static int error_with_patch(struct commit *commit,
 static int error_failed_squash(struct commit *commit,
 	struct replay_opts *opts, int subject_len, const char *subject)
 {
-	if (rename(rebase_path_squash_msg(), rebase_path_message()))
-		return error(_("could not rename '%s' to '%s'"),
+	if (copy_file(rebase_path_message(), rebase_path_squash_msg(), 0666))
+		return error(_("could not copy '%s' to '%s'"),
 			rebase_path_squash_msg(), rebase_path_message());
-	unlink(rebase_path_fixup_msg());
 	unlink(git_path_merge_msg());
 	if (copy_file(git_path_merge_msg(), rebase_path_message(), 0666))
 		return error(_("could not copy '%s' to '%s'"),
diff --git a/sequencer.h b/sequencer.h
index e45b178dfc4..1898158c52d 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -44,10 +44,14 @@ struct replay_opts {
 	char **xopts;
 	size_t xopts_nr, xopts_alloc;
 
+	/* Used by fixup/squash */
+	struct strbuf current_fixups;
+	int current_fixup_count;
+
 	/* Only used by REPLAY_NONE */
 	struct rev_info *revs;
 };
-#define REPLAY_OPTS_INIT { -1 }
+#define REPLAY_OPTS_INIT { .action = -1, .current_fixups = STRBUF_INIT }
 
 /* Call this to setup defaults before parsing command line options */
 void sequencer_init_config(struct replay_opts *opts);
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 3/4] sequencer: always commit without editing when asked for
  2018-04-27 20:48               ` [PATCH v4 " Johannes Schindelin
  2018-04-27 20:48                 ` [PATCH v4 1/4] rebase -i: demonstrate bugs with fixup!/squash! " Johannes Schindelin
  2018-04-27 20:48                 ` [PATCH v4 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON Johannes Schindelin
@ 2018-04-27 20:48                 ` Johannes Schindelin
  2018-04-27 20:48                 ` [PATCH v4 4/4] rebase --skip: clean up commit message after a failed fixup/squash Johannes Schindelin
  3 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-27 20:48 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Eric Sunshine,
	Stefan Beller, Phillip Wood

Previously, we only called run_git_commit() without EDIT_MSG when we also
passed in a default message.

However, an upcoming caller will want to commit without EDIT_MSG and
*without* a default message: to clean up fixup/squash comments in HEAD's
commit message.

Let's prepare for that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index d2e6f33023d..56166b0d6c7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -717,6 +717,8 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_pushf(&cmd.args, "-S%s", opts->gpg_sign);
 	if (defmsg)
 		argv_array_pushl(&cmd.args, "-F", defmsg, NULL);
+	else if (!(flags & EDIT_MSG))
+		argv_array_pushl(&cmd.args, "-C", "HEAD", NULL);
 	if ((flags & CLEANUP_MSG))
 		argv_array_push(&cmd.args, "--cleanup=strip");
 	if ((flags & EDIT_MSG))
-- 
2.17.0.windows.1.33.gfcbb1fa0445



^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 4/4] rebase --skip: clean up commit message after a failed fixup/squash
  2018-04-27 20:48               ` [PATCH v4 " Johannes Schindelin
                                   ` (2 preceding siblings ...)
  2018-04-27 20:48                 ` [PATCH v4 3/4] sequencer: always commit without editing when asked for Johannes Schindelin
@ 2018-04-27 20:48                 ` Johannes Schindelin
  2018-04-27 21:28                   ` Stefan Beller
  2018-05-06 17:50                   ` Phillip Wood
  3 siblings, 2 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-27 20:48 UTC (permalink / raw)
  To: git
  Cc: Johannes Schindelin, Junio C Hamano, Eric Sunshine,
	Stefan Beller, Phillip Wood

During a series of fixup/squash commands, the interactive rebase builds
up a commit message with comments. This will be presented to the user in
the editor if at least one of those commands was a `squash`.

In any case, the commit message will be cleaned up eventually, removing
all those intermediate comments, in the final step of such a
fixup/squash chain.

However, if the last fixup/squash command in such a chain fails with
merge conflicts, and if the user then decides to skip it (or resolve it
to a clean worktree and then continue the rebase), the current code
fails to clean up the commit message.

This commit fixes that behavior.

The fix is quite a bit more involved than meets the eye because it is
not only about the question whether we are `git rebase --skip`ing a
fixup or squash. It is also about removing the skipped fixup/squash's
commit message from the accumulated commit message. And it is also about
the question whether we should let the user edit the final commit
message or not ("Was there a squash in the chain *that was not
skipped*?").

For example, in this case we will want to fix the commit message, but
not open it in an editor:

	pick	<- succeeds
	fixup	<- succeeds
	squash	<- fails, will be skipped

This is where the newly-introduced `current-fixups` file comes in real
handy. A quick look and we can determine whether there was a non-skipped
squash. We only need to make sure to keep it up to date with respect to
skipped fixup/squash commands. As a bonus, we can even avoid committing
unnecessarily, e.g. when there was only one fixup, and it failed, and
was skipped.

To fix only the bug where the final commit message was not cleaned up
properly, but without fixing the rest, would have been more complicated
than fixing it all in one go, hence this commit lumps together more than
a single concern.

For the same reason, this commit also adds a bit more to the existing
test case for the regression we just fixed.

The diff is best viewed with --color-moved.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c                | 113 ++++++++++++++++++++++++++++++++-----
 t/t3418-rebase-continue.sh |  35 ++++++++++--
 2 files changed, 131 insertions(+), 17 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 56166b0d6c7..cec180714ef 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2779,19 +2779,16 @@ static int continue_single_pick(void)
 	return run_command_v_opt(argv, RUN_GIT_CMD);
 }
 
-static int commit_staged_changes(struct replay_opts *opts)
+static int commit_staged_changes(struct replay_opts *opts,
+				 struct todo_list *todo_list)
 {
 	unsigned int flags = ALLOW_EMPTY | EDIT_MSG;
+	unsigned int final_fixup = 0, is_clean;
 
 	if (has_unstaged_changes(1))
 		return error(_("cannot rebase: You have unstaged changes."));
-	if (!has_uncommitted_changes(0)) {
-		const char *cherry_pick_head = git_path_cherry_pick_head();
 
-		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
-			return error(_("could not remove CHERRY_PICK_HEAD"));
-		return 0;
-	}
+	is_clean = !has_uncommitted_changes(0);
 
 	if (file_exists(rebase_path_amend())) {
 		struct strbuf rev = STRBUF_INIT;
@@ -2804,19 +2801,107 @@ static int commit_staged_changes(struct replay_opts *opts)
 		if (get_oid_hex(rev.buf, &to_amend))
 			return error(_("invalid contents: '%s'"),
 				rebase_path_amend());
-		if (oidcmp(&head, &to_amend))
+		if (!is_clean && oidcmp(&head, &to_amend))
 			return error(_("\nYou have uncommitted changes in your "
 				       "working tree. Please, commit them\n"
 				       "first and then run 'git rebase "
 				       "--continue' again."));
+		/*
+		 * When skipping a failed fixup/squash, we need to edit the
+		 * commit message, the current fixup list and count, and if it
+		 * was the last fixup/squash in the chain, we need to clean up
+		 * the commit message and if there was a squash, let the user
+		 * edit it.
+		 */
+		if (is_clean && !oidcmp(&head, &to_amend) &&
+		    opts->current_fixup_count > 0 &&
+		    file_exists(rebase_path_stopped_sha())) {
+			const char *p = opts->current_fixups.buf;
+			int len = opts->current_fixups.len;
+
+			opts->current_fixup_count--;
+			if (!len)
+				BUG("Incorrect current_fixups:\n%s", p);
+			while (len && p[len - 1] != '\n')
+				len--;
+			strbuf_setlen(&opts->current_fixups, len);
+			if (write_message(p, len, rebase_path_current_fixups(),
+					  0) < 0)
+				return error(_("could not write file: '%s'"),
+					     rebase_path_current_fixups());
+
+			/*
+			 * If a fixup/squash in a fixup/squash chain failed, the
+			 * commit message is already correct, no need to commit
+			 * it again.
+			 *
+			 * Only if it is the final command in the fixup/squash
+			 * chain, and only if the chain is longer than a single
+			 * fixup/squash command (which was just skipped), do we
+			 * actually need to re-commit with a cleaned up commit
+			 * message.
+			 */
+			if (opts->current_fixup_count > 0 &&
+			    !is_fixup(peek_command(todo_list, 0))) {
+				final_fixup = 1;
+				/*
+				 * If there was not a single "squash" in the
+				 * chain, we only need to clean up the commit
+				 * message, no need to bother the user with
+				 * opening the commit message in the editor.
+				 */
+				if (!starts_with(p, "squash ") &&
+				    !strstr(p, "\nsquash "))
+					flags = (flags & ~EDIT_MSG) | CLEANUP_MSG;
+			} else if (is_fixup(peek_command(todo_list, 0))) {
+				/*
+				 * We need to update the squash message to skip
+				 * the latest commit message.
+				 */
+				struct commit *commit;
+				const char *path = rebase_path_squash_msg();
+
+				if (parse_head(&commit) ||
+				    !(p = get_commit_buffer(commit, NULL)) ||
+				    write_message(p, strlen(p), path, 0)) {
+					unuse_commit_buffer(commit, p);
+					return error(_("could not write file: "
+						       "'%s'"), path);
+				}
+				unuse_commit_buffer(commit, p);
+			}
+		}
 
 		strbuf_release(&rev);
 		flags |= AMEND_MSG;
 	}
 
-	if (run_git_commit(rebase_path_message(), opts, flags))
+	if (is_clean) {
+		const char *cherry_pick_head = git_path_cherry_pick_head();
+
+		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
+			return error(_("could not remove CHERRY_PICK_HEAD"));
+		if (!final_fixup)
+			return 0;
+	}
+
+	if (run_git_commit(final_fixup ? NULL : rebase_path_message(),
+			   opts, flags))
 		return error(_("could not commit staged changes."));
 	unlink(rebase_path_amend());
+	if (final_fixup) {
+		unlink(rebase_path_fixup_msg());
+		unlink(rebase_path_squash_msg());
+	}
+	if (opts->current_fixup_count > 0) {
+		/*
+		 * Whether final fixup or not, we just cleaned up the commit
+		 * message...
+		 */
+		unlink(rebase_path_current_fixups());
+		strbuf_reset(&opts->current_fixups);
+		opts->current_fixup_count = 0;
+	}
 	return 0;
 }
 
@@ -2828,14 +2913,16 @@ int sequencer_continue(struct replay_opts *opts)
 	if (read_and_refresh_cache(opts))
 		return -1;
 
+	if (read_populate_opts(opts))
+		return -1;
 	if (is_rebase_i(opts)) {
-		if (commit_staged_changes(opts))
+		if ((res = read_populate_todo(&todo_list, opts)))
+			goto release_todo_list;
+		if (commit_staged_changes(opts, &todo_list))
 			return -1;
 	} else if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
-	if (read_populate_opts(opts))
-		return -1;
-	if ((res = read_populate_todo(&todo_list, opts)))
+	else if ((res = read_populate_todo(&todo_list, opts)))
 		goto release_todo_list;
 
 	if (!is_rebase_i(opts)) {
diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
index 3874f187246..03bf1b8a3b3 100755
--- a/t/t3418-rebase-continue.sh
+++ b/t/t3418-rebase-continue.sh
@@ -88,14 +88,14 @@ test_expect_success 'rebase passes merge strategy options correctly' '
 	git rebase --continue
 '
 
-test_expect_failure '--skip after failed fixup cleans commit message' '
+test_expect_success '--skip after failed fixup cleans commit message' '
 	test_when_finished "test_might_fail git rebase --abort" &&
 	git checkout -b with-conflicting-fixup &&
 	test_commit wants-fixup &&
 	test_commit "fixup! wants-fixup" wants-fixup.t 1 wants-fixup-1 &&
 	test_commit "fixup! wants-fixup" wants-fixup.t 2 wants-fixup-2 &&
 	test_commit "fixup! wants-fixup" wants-fixup.t 3 wants-fixup-3 &&
-	test_must_fail env FAKE_LINES="1 fixup 2 fixup 4" \
+	test_must_fail env FAKE_LINES="1 fixup 2 squash 4" \
 		git rebase -i HEAD~4 &&
 
 	: now there is a conflict, and comments in the commit message &&
@@ -103,11 +103,38 @@ test_expect_failure '--skip after failed fixup cleans commit message' '
 	grep "fixup! wants-fixup" out &&
 
 	: skip and continue &&
-	git rebase --skip &&
+	echo "cp \"\$1\" .git/copy.txt" | write_script copy-editor.sh &&
+	(test_set_editor "$PWD/copy-editor.sh" && git rebase --skip) &&
+
+	: the user should not have had to edit the commit message &&
+	test_path_is_missing .git/copy.txt &&
 
 	: now the comments in the commit message should have been cleaned up &&
 	git show HEAD >out &&
-	! grep "fixup! wants-fixup" out
+	! grep "fixup! wants-fixup" out &&
+
+	: now, let us ensure that "squash" is handled correctly &&
+	git reset --hard wants-fixup-3 &&
+	test_must_fail env FAKE_LINES="1 squash 4 squash 2 squash 4" \
+		git rebase -i HEAD~4 &&
+
+	: the first squash failed, but there are two more in the chain &&
+	(test_set_editor "$PWD/copy-editor.sh" &&
+	 test_must_fail git rebase --skip) &&
+
+	: not the final squash, no need to edit the commit message &&
+	test_path_is_missing .git/copy.txt &&
+
+	: The first squash was skipped, therefore: &&
+	git show HEAD >out &&
+	test_i18ngrep "# This is a combination of 2 commits" out &&
+
+	(test_set_editor "$PWD/copy-editor.sh" && git rebase --skip) &&
+	git show HEAD >out &&
+	test_i18ngrep ! "# This is a combination" out &&
+
+	: Final squash failed, but there was still a squash &&
+	test_i18ngrep "# This is a combination of 2 commits" .git/copy.txt
 '
 
 test_expect_success 'setup rerere database' '
-- 
2.17.0.windows.1.33.gfcbb1fa0445

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v4 4/4] rebase --skip: clean up commit message after a failed fixup/squash
  2018-04-27 20:48                 ` [PATCH v4 4/4] rebase --skip: clean up commit message after a failed fixup/squash Johannes Schindelin
@ 2018-04-27 21:28                   ` Stefan Beller
  2018-04-28 13:05                     ` Johannes Schindelin
  2018-05-06 17:50                   ` Phillip Wood
  1 sibling, 1 reply; 412+ messages in thread
From: Stefan Beller @ 2018-04-27 21:28 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Junio C Hamano, Eric Sunshine, Phillip Wood

On Fri, Apr 27, 2018 at 1:48 PM, Johannes Schindelin
<johannes.schindelin@gmx.de> wrote:
> During a series of fixup/squash commands, the interactive rebase builds
> up a commit message with comments. This will be presented to the user in
> the editor if at least one of those commands was a `squash`.

This sounds as if the whole series will be presented to the user, i.e.

 pick A
 squash B
 fixup C

would present A+B+C in the editor. I always assumed the sequencer
to be linear, i.e. pick A+B, open editor and then fixup C into the
previous result?

No need to resend it reworded, I just realize that I never tested my
potentially wrong assumption.

> The diff is best viewed with --color-moved.

... and web pages are "best viewed with IE 6.0" ;-)

I found this so funny that I had to download the patches and actually
look at them
using the move detection only to find out that only very few lines are moved,
as there are only very few deleted lines.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/2] unpack_trees_options: free messages when done
  2018-04-24 16:29                         ` Elijah Newren
@ 2018-04-28 11:32                           ` Martin Ågren
  2018-04-28 12:30                             ` Johannes Schindelin
                                               ` (2 more replies)
  0 siblings, 3 replies; 412+ messages in thread
From: Martin Ågren @ 2018-04-28 11:32 UTC (permalink / raw)
  To: Elijah Newren
  Cc: git, Phillip Wood, Jacob Keller, Johannes Schindelin, Ben Peart

From: Elijah Newren <newren@gmail.com>

Hi Elijah,

[Since this is leaving the topic of rename-detection in favour of 
leak-plugging, I'm shortening the cc-list a bit.]

> So, instead, I'd like to see something like the below
> (built on top of my series):

Thanks a lot. I now have the below patch in my tree as a preparatory
part of a three-patch series on top of your series. Since the gist of
this patch is entirely your creation, is it ok if I place your Author:
and Signed-off-by: on it? Credit where credit is due.

As you noted elsewhere [1], Ben is also working in this area. I'd be
perfectly happy to sit on these patches until both of your contributions
come through to master.

[1] https://public-inbox.org/git/CABPp-BFh=gL6RnbST2bgtynkij1Z5TMgAr1Via5_VyteF5eBMg@mail.gmail.com/

Martin

-->8--
Subject: merge-recursive: provide pair of `unpack_trees_{start,finish}()`

Rename `git_merge_trees()` to `unpack_trees_start()` and extract the
call to `discard_index()` into a new function `unpack_trees_finish()`.
As a result, these are called early resp. late in `merge_trees()`,
making the resource handling clearer. The next commit will expand on
that, teaching `..._finish()` to free more memory. (So rather than
moving the TODO-comment, just drop it, since it will be addressed soon
enough.)

Also call `..._finish()` when `merge_trees()` returns early.
---
 merge-recursive.c | 29 +++++++++++++++--------------
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 1de8dc1c53..e64102004a 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -337,10 +337,10 @@ static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
 	init_tree_desc(desc, tree->buffer, tree->size);
 }
 
-static int git_merge_trees(struct merge_options *o,
-			   struct tree *common,
-			   struct tree *head,
-			   struct tree *merge)
+static int unpack_trees_start(struct merge_options *o,
+			      struct tree *common,
+			      struct tree *head,
+			      struct tree *merge)
 {
 	int rc;
 	struct tree_desc t[3];
@@ -378,6 +378,11 @@ static int git_merge_trees(struct merge_options *o,
 	return rc;
 }
 
+static void unpack_trees_finish(struct merge_options *o)
+{
+	discard_index(&o->orig_index);
+}
+
 struct tree *write_tree_from_memory(struct merge_options *o)
 {
 	struct tree *result = NULL;
@@ -3079,13 +3084,14 @@ int merge_trees(struct merge_options *o,
 		return 1;
 	}
 
-	code = git_merge_trees(o, common, head, merge);
+	code = unpack_trees_start(o, common, head, merge);
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
 			err(o, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
+		unpack_trees_finish(o);
 		return -1;
 	}
 
@@ -3138,20 +3144,15 @@ int merge_trees(struct merge_options *o,
 
 		hashmap_free(&o->current_file_dir_set, 1);
 
-		if (clean < 0)
+		if (clean < 0) {
+			unpack_trees_finish(o);
 			return clean;
+		}
 	}
 	else
 		clean = 1;
 
-	/* Free the extra index left from git_merge_trees() */
-	/*
-	 * FIXME: Need to also data allocated by setup_unpack_trees_porcelain()
-	 * tucked away in o->unpack_opts.msgs, but the problem is that only
-	 * half of it refers to dynamically allocated data, while the other
-	 * half points at static strings.
-	 */
-	discard_index(&o->orig_index);
+	unpack_trees_finish(o);
 
 	if (o->call_depth && !(*result = write_tree_from_memory(o)))
 		return -1;
-- 
2.17.0


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/2] unpack_trees_options: free messages when done
  2018-04-28 11:32                           ` Martin Ågren
@ 2018-04-28 12:30                             ` Johannes Schindelin
  2018-04-28 20:56                             ` Elijah Newren
  2018-05-16 14:32                             ` Elijah Newren
  2 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-28 12:30 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Elijah Newren, git, Phillip Wood, Jacob Keller, Ben Peart

[-- Attachment #1: Type: text/plain, Size: 888 bytes --]

Hi Martin,

On Sat, 28 Apr 2018, Martin Ågren wrote:

> -->8--
> Subject: merge-recursive: provide pair of `unpack_trees_{start,finish}()`
> 
> Rename `git_merge_trees()` to `unpack_trees_start()` and extract the
> call to `discard_index()` into a new function `unpack_trees_finish()`.
> As a result, these are called early resp. late in `merge_trees()`,
> making the resource handling clearer. The next commit will expand on
> that, teaching `..._finish()` to free more memory. (So rather than
> moving the TODO-comment, just drop it, since it will be addressed soon
> enough.)
> 
> Also call `..._finish()` when `merge_trees()` returns early.

Looks good! It is missing a Signed-off-by: line, and you probably want to
start a new thread that also includes the "next commit", but other than
that it is pretty nice and ready for contributing, methinks.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v4 4/4] rebase --skip: clean up commit message after a failed fixup/squash
  2018-04-27 21:28                   ` Stefan Beller
@ 2018-04-28 13:05                     ` Johannes Schindelin
  0 siblings, 0 replies; 412+ messages in thread
From: Johannes Schindelin @ 2018-04-28 13:05 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, Junio C Hamano, Eric Sunshine, Phillip Wood

Hi Stefan,

On Fri, 27 Apr 2018, Stefan Beller wrote:

> On Fri, Apr 27, 2018 at 1:48 PM, Johannes Schindelin
> <johannes.schindelin@gmx.de> wrote:
> > During a series of fixup/squash commands, the interactive rebase builds
> > up a commit message with comments. This will be presented to the user in
> > the editor if at least one of those commands was a `squash`.
> 
> This sounds as if the whole series will be presented to the user, i.e.
> 
>  pick A
>  squash B
>  fixup C
> 
> would present A+B+C in the editor.

And that is indeed the case. The commit message would look something like
this:

	# This is a combination of 3 commits.
	# This is commit message #1:

	Hello Stefan

	This is A.

	# This is commit message #2:

	squash! A

	Me again, Stefan. I am here to be squashed.

	# The commit message #3 will be skipped:
	#
	# fixup! A

> I always assumed the sequencer to be linear, i.e. pick A+B, open editor
> and then fixup C into the previous result?

Nope.

> No need to resend it reworded, I just realize that I never tested my
> potentially wrong assumption.

No worries, you learned something today.

> > The diff is best viewed with --color-moved.
> 
> ... and web pages are "best viewed with IE 6.0" ;-)

That is what I had in mind writing that.

> I found this so funny that I had to download the patches and actually
> look at them using the move detection only to find out that only very
> few lines are moved, as there are only very few deleted lines.

I agree that the current iteration is no longer such obvious a move. I had
to add tons of stuff to fix the extra issues I found while working on v4.

But still, I found it super-helpful to see that the code was actually
moved, and where, because I essentially had to break up the nice sequence
of "is it clean? Yes? Then nothing to be done! No? Is HEAD to be amended?
Yes? No?" and basically build a matrix what to do in all combinations of
"clean? Amend HEAD?"

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/2] unpack_trees_options: free messages when done
  2018-04-28 11:32                           ` Martin Ågren
  2018-04-28 12:30                             ` Johannes Schindelin
@ 2018-04-28 20:56                             ` Elijah Newren
  2018-05-16 14:32                             ` Elijah Newren
  2 siblings, 0 replies; 412+ messages in thread
From: Elijah Newren @ 2018-04-28 20:56 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Git Mailing List, Phillip Wood, Jacob Keller,
	Johannes Schindelin, Ben Peart

Hi Martin,

On Sat, Apr 28, 2018 at 4:32 AM, Martin Ågren <martin.agren@gmail.com> wrote:
> From: Elijah Newren <newren@gmail.com>
>
> Hi Elijah,
>
> [Since this is leaving the topic of rename-detection in favour of
> leak-plugging, I'm shortening the cc-list a bit.]
>
>> So, instead, I'd like to see something like the below
>> (built on top of my series):
>
> Thanks a lot. I now have the below patch in my tree as a preparatory
> part of a three-patch series on top of your series. Since the gist of
> this patch is entirely your creation, is it ok if I place your Author:
> and Signed-off-by: on it? Credit where credit is due.

Sure, I'm fine with either that or an Original-patch-by attribution.

Anyway, thanks for fleshing it out with the commit message and
handling the early return cases.  And for tackling the
setup_unpack_trees_porcelain() memory leak.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v4 4/4] rebase --skip: clean up commit message after a failed fixup/squash
  2018-04-27 20:48                 ` [PATCH v4 4/4] rebase --skip: clean up commit message after a failed fixup/squash Johannes Schindelin
  2018-04-27 21:28                   ` Stefan Beller
@ 2018-05-06 17:50                   ` Phillip Wood
  2018-05-09 10:50                     ` Phillip Wood
  2018-10-02 13:50                     ` Johannes Schindelin
  1 sibling, 2 replies; 412+ messages in thread
From: Phillip Wood @ 2018-05-06 17:50 UTC (permalink / raw)
  To: Johannes Schindelin, git
  Cc: Junio C Hamano, Eric Sunshine, Stefan Beller, Phillip Wood

Hi Johannes, sorry it's taken me a while to look at this. I think it
mostly makes sense to me, the code is well documented. I've got one
comment below

On 27/04/18 21:48, Johannes Schindelin wrote:
> 
> During a series of fixup/squash commands, the interactive rebase builds
> up a commit message with comments. This will be presented to the user in
> the editor if at least one of those commands was a `squash`.
> 
> In any case, the commit message will be cleaned up eventually, removing
> all those intermediate comments, in the final step of such a
> fixup/squash chain.
> 
> However, if the last fixup/squash command in such a chain fails with
> merge conflicts, and if the user then decides to skip it (or resolve it
> to a clean worktree and then continue the rebase), the current code
> fails to clean up the commit message.
> 
> This commit fixes that behavior.
> 
> The fix is quite a bit more involved than meets the eye because it is
> not only about the question whether we are `git rebase --skip`ing a
> fixup or squash. It is also about removing the skipped fixup/squash's
> commit message from the accumulated commit message. And it is also about
> the question whether we should let the user edit the final commit
> message or not ("Was there a squash in the chain *that was not
> skipped*?").
> 
> For example, in this case we will want to fix the commit message, but
> not open it in an editor:
> 
> 	pick	<- succeeds
> 	fixup	<- succeeds
> 	squash	<- fails, will be skipped
> 
> This is where the newly-introduced `current-fixups` file comes in real
> handy. A quick look and we can determine whether there was a non-skipped
> squash. We only need to make sure to keep it up to date with respect to
> skipped fixup/squash commands. As a bonus, we can even avoid committing
> unnecessarily, e.g. when there was only one fixup, and it failed, and
> was skipped.
> 
> To fix only the bug where the final commit message was not cleaned up
> properly, but without fixing the rest, would have been more complicated
> than fixing it all in one go, hence this commit lumps together more than
> a single concern.
> 
> For the same reason, this commit also adds a bit more to the existing
> test case for the regression we just fixed.
> 
> The diff is best viewed with --color-moved.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c                | 113 ++++++++++++++++++++++++++++++++-----
>  t/t3418-rebase-continue.sh |  35 ++++++++++--
>  2 files changed, 131 insertions(+), 17 deletions(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index 56166b0d6c7..cec180714ef 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -2779,19 +2779,16 @@ static int continue_single_pick(void)
>  	return run_command_v_opt(argv, RUN_GIT_CMD);
>  }
>  
> -static int commit_staged_changes(struct replay_opts *opts)
> +static int commit_staged_changes(struct replay_opts *opts,
> +				 struct todo_list *todo_list)
>  {
>  	unsigned int flags = ALLOW_EMPTY | EDIT_MSG;
> +	unsigned int final_fixup = 0, is_clean;
>  
>  	if (has_unstaged_changes(1))
>  		return error(_("cannot rebase: You have unstaged changes."));
> -	if (!has_uncommitted_changes(0)) {
> -		const char *cherry_pick_head = git_path_cherry_pick_head();
>  
> -		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
> -			return error(_("could not remove CHERRY_PICK_HEAD"));
> -		return 0;
> -	}
> +	is_clean = !has_uncommitted_changes(0);
>  
>  	if (file_exists(rebase_path_amend())) {
>  		struct strbuf rev = STRBUF_INIT;
> @@ -2804,19 +2801,107 @@ static int commit_staged_changes(struct replay_opts *opts)
>  		if (get_oid_hex(rev.buf, &to_amend))
>  			return error(_("invalid contents: '%s'"),
>  				rebase_path_amend());
> -		if (oidcmp(&head, &to_amend))
> +		if (!is_clean && oidcmp(&head, &to_amend))
>  			return error(_("\nYou have uncommitted changes in your "
>  				       "working tree. Please, commit them\n"
>  				       "first and then run 'git rebase "
>  				       "--continue' again."));
> +		/*
> +		 * When skipping a failed fixup/squash, we need to edit the
> +		 * commit message, the current fixup list and count, and if it
> +		 * was the last fixup/squash in the chain, we need to clean up
> +		 * the commit message and if there was a squash, let the user
> +		 * edit it.
> +		 */
> +		if (is_clean && !oidcmp(&head, &to_amend) &&
> +		    opts->current_fixup_count > 0 &&
> +		    file_exists(rebase_path_stopped_sha())) {
> +			const char *p = opts->current_fixups.buf;
> +			int len = opts->current_fixups.len;
> +
> +			opts->current_fixup_count--;
> +			if (!len)
> +				BUG("Incorrect current_fixups:\n%s", p);
> +			while (len && p[len - 1] != '\n')
> +				len--;
> +			strbuf_setlen(&opts->current_fixups, len);
> +			if (write_message(p, len, rebase_path_current_fixups(),
> +					  0) < 0)
> +				return error(_("could not write file: '%s'"),
> +					     rebase_path_current_fixups());
> +
> +			/*
> +			 * If a fixup/squash in a fixup/squash chain failed, the
> +			 * commit message is already correct, no need to commit
> +			 * it again.
> +			 *
> +			 * Only if it is the final command in the fixup/squash
> +			 * chain, and only if the chain is longer than a single
> +			 * fixup/squash command (which was just skipped), do we
> +			 * actually need to re-commit with a cleaned up commit
> +			 * message.
> +			 */
> +			if (opts->current_fixup_count > 0 &&
> +			    !is_fixup(peek_command(todo_list, 0))) {
> +				final_fixup = 1;
> +				/*
> +				 * If there was not a single "squash" in the
> +				 * chain, we only need to clean up the commit
> +				 * message, no need to bother the user with
> +				 * opening the commit message in the editor.
> +				 */
> +				if (!starts_with(p, "squash ") &&
> +				    !strstr(p, "\nsquash "))
> +					flags = (flags & ~EDIT_MSG) | CLEANUP_MSG;
> +			} else if (is_fixup(peek_command(todo_list, 0))) {
> +				/*
> +				 * We need to update the squash message to skip
> +				 * the latest commit message.
> +				 */
> +				struct commit *commit;
> +				const char *path = rebase_path_squash_msg();
> +
> +				if (parse_head(&commit) ||
> +				    !(p = get_commit_buffer(commit, NULL)) ||
> +				    write_message(p, strlen(p), path, 0)) {
> +					unuse_commit_buffer(commit, p);
> +					return error(_("could not write file: "
> +						       "'%s'"), path);
> +				}

I think it should probably recreate the fixup message as well. If there
is a sequence

pick commit
fixup a
fixup b
fixup c

and 'fixup b' gets skipped then when 'fixup c' is applied the user will
be prompted to edit the message unless rebase_path_fixup_msg() exists.

Best Wishes

Phillip

> +				unuse_commit_buffer(commit, p);
> +			}
> +		}
>  
>  		strbuf_release(&rev);
>  		flags |= AMEND_MSG;
>  	}
>  
> -	if (run_git_commit(rebase_path_message(), opts, flags))
> +	if (is_clean) {
> +		const char *cherry_pick_head = git_path_cherry_pick_head();
> +
> +		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
> +			return error(_("could not remove CHERRY_PICK_HEAD"));
> +		if (!final_fixup)
> +			return 0;
> +	}
> +
> +	if (run_git_commit(final_fixup ? NULL : rebase_path_message(),
> +			   opts, flags))
>  		return error(_("could not commit staged changes."));
>  	unlink(rebase_path_amend());
> +	if (final_fixup) {
> +		unlink(rebase_path_fixup_msg());
> +		unlink(rebase_path_squash_msg());
> +	}
> +	if (opts->current_fixup_count > 0) {
> +		/*
> +		 * Whether final fixup or not, we just cleaned up the commit
> +		 * message...
> +		 */
> +		unlink(rebase_path_current_fixups());
> +		strbuf_reset(&opts->current_fixups);
> +		opts->current_fixup_count = 0;
> +	}
>  	return 0;
>  }
>  
> @@ -2828,14 +2913,16 @@ int sequencer_continue(struct replay_opts *opts)
>  	if (read_and_refresh_cache(opts))
>  		return -1;
>  
> +	if (read_populate_opts(opts))
> +		return -1;
>  	if (is_rebase_i(opts)) {
> -		if (commit_staged_changes(opts))
> +		if ((res = read_populate_todo(&todo_list, opts)))
> +			goto release_todo_list;
> +		if (commit_staged_changes(opts, &todo_list))
>  			return -1;
>  	} else if (!file_exists(get_todo_path(opts)))
>  		return continue_single_pick();
> -	if (read_populate_opts(opts))
> -		return -1;
> -	if ((res = read_populate_todo(&todo_list, opts)))
> +	else if ((res = read_populate_todo(&todo_list, opts)))
>  		goto release_todo_list;
>  
>  	if (!is_rebase_i(opts)) {
> diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
> index 3874f187246..03bf1b8a3b3 100755
> --- a/t/t3418-rebase-continue.sh
> +++ b/t/t3418-rebase-continue.sh
> @@ -88,14 +88,14 @@ test_expect_success 'rebase passes merge strategy options correctly' '
>  	git rebase --continue
>  '
>  
> -test_expect_failure '--skip after failed fixup cleans commit message' '
> +test_expect_success '--skip after failed fixup cleans commit message' '
>  	test_when_finished "test_might_fail git rebase --abort" &&
>  	git checkout -b with-conflicting-fixup &&
>  	test_commit wants-fixup &&
>  	test_commit "fixup! wants-fixup" wants-fixup.t 1 wants-fixup-1 &&
>  	test_commit "fixup! wants-fixup" wants-fixup.t 2 wants-fixup-2 &&
>  	test_commit "fixup! wants-fixup" wants-fixup.t 3 wants-fixup-3 &&
> -	test_must_fail env FAKE_LINES="1 fixup 2 fixup 4" \
> +	test_must_fail env FAKE_LINES="1 fixup 2 squash 4" \
>  		git rebase -i HEAD~4 &&
>  
>  	: now there is a conflict, and comments in the commit message &&
> @@ -103,11 +103,38 @@ test_expect_failure '--skip after failed fixup cleans commit message' '
>  	grep "fixup! wants-fixup" out &&
>  
>  	: skip and continue &&
> -	git rebase --skip &&
> +	echo "cp \"\$1\" .git/copy.txt" | write_script copy-editor.sh &&
> +	(test_set_editor "$PWD/copy-editor.sh" && git rebase --skip) &&
> +
> +	: the user should not have had to edit the commit message &&
> +	test_path_is_missing .git/copy.txt &&
>  
>  	: now the comments in the commit message should have been cleaned up &&
>  	git show HEAD >out &&
> -	! grep "fixup! wants-fixup" out
> +	! grep "fixup! wants-fixup" out &&
> +
> +	: now, let us ensure that "squash" is handled correctly &&
> +	git reset --hard wants-fixup-3 &&
> +	test_must_fail env FAKE_LINES="1 squash 4 squash 2 squash 4" \
> +		git rebase -i HEAD~4 &&
> +
> +	: the first squash failed, but there are two more in the chain &&
> +	(test_set_editor "$PWD/copy-editor.sh" &&
> +	 test_must_fail git rebase --skip) &&
> +
> +	: not the final squash, no need to edit the commit message &&
> +	test_path_is_missing .git/copy.txt &&
> +
> +	: The first squash was skipped, therefore: &&
> +	git show HEAD >out &&
> +	test_i18ngrep "# This is a combination of 2 commits" out &&
> +
> +	(test_set_editor "$PWD/copy-editor.sh" && git rebase --skip) &&
> +	git show HEAD >out &&
> +	test_i18ngrep ! "# This is a combination" out &&
> +
> +	: Final squash failed, but there was still a squash &&
> +	test_i18ngrep "# This is a combination of 2 commits" .git/copy.txt
>  '
>  
>  test_expect_success 'setup rerere database' '
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v4 4/4] rebase --skip: clean up commit message after a failed fixup/squash
  2018-05-06 17:50                   ` Phillip Wood
@ 2018-05-09 10:50                     ` Phillip Wood
  2018-10-02 13:50                     ` Johannes Schindelin
  1 sibling, 0 replies; 412+ messages in thread
From: Phillip Wood @ 2018-05-09 10:50 UTC (permalink / raw)
  To: Johannes Schindelin, git
  Cc: Junio C Hamano, Eric Sunshine, Stefan Beller, Phillip Wood

On 06/05/18 18:50, Phillip Wood wrote:
> Hi Johannes, sorry it's taken me a while to look at this. I think it
> mostly makes sense to me, the code is well documented. I've got one
> comment below
> 
> On 27/04/18 21:48, Johannes Schindelin wrote:
>>
>> During a series of fixup/squash commands, the interactive rebase builds
>> up a commit message with comments. This will be presented to the user in
>> the editor if at least one of those commands was a `squash`.
>>
>> In any case, the commit message will be cleaned up eventually, removing
>> all those intermediate comments, in the final step of such a
>> fixup/squash chain.
>>
>> However, if the last fixup/squash command in such a chain fails with
>> merge conflicts, and if the user then decides to skip it (or resolve it
>> to a clean worktree and then continue the rebase), the current code
>> fails to clean up the commit message.
>>
>> This commit fixes that behavior.
>>
>> The fix is quite a bit more involved than meets the eye because it is
>> not only about the question whether we are `git rebase --skip`ing a
>> fixup or squash. It is also about removing the skipped fixup/squash's
>> commit message from the accumulated commit message. And it is also about
>> the question whether we should let the user edit the final commit
>> message or not ("Was there a squash in the chain *that was not
>> skipped*?").
>>
>> For example, in this case we will want to fix the commit message, but
>> not open it in an editor:
>>
>> 	pick	<- succeeds
>> 	fixup	<- succeeds
>> 	squash	<- fails, will be skipped
>>
>> This is where the newly-introduced `current-fixups` file comes in real
>> handy. A quick look and we can determine whether there was a non-skipped
>> squash. We only need to make sure to keep it up to date with respect to
>> skipped fixup/squash commands. As a bonus, we can even avoid committing
>> unnecessarily, e.g. when there was only one fixup, and it failed, and
>> was skipped.
>>
>> To fix only the bug where the final commit message was not cleaned up
>> properly, but without fixing the rest, would have been more complicated
>> than fixing it all in one go, hence this commit lumps together more than
>> a single concern.
>>
>> For the same reason, this commit also adds a bit more to the existing
>> test case for the regression we just fixed.
>>
>> The diff is best viewed with --color-moved.
>>
>> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
>> ---
>>   sequencer.c                | 113 ++++++++++++++++++++++++++++++++-----
>>   t/t3418-rebase-continue.sh |  35 ++++++++++--
>>   2 files changed, 131 insertions(+), 17 deletions(-)
>>
>> diff --git a/sequencer.c b/sequencer.c
>> index 56166b0d6c7..cec180714ef 100644
>> --- a/sequencer.c
>> +++ b/sequencer.c
>> @@ -2779,19 +2779,16 @@ static int continue_single_pick(void)
>>   	return run_command_v_opt(argv, RUN_GIT_CMD);
>>   }
>>   
>> -static int commit_staged_changes(struct replay_opts *opts)
>> +static int commit_staged_changes(struct replay_opts *opts,
>> +				 struct todo_list *todo_list)
>>   {
>>   	unsigned int flags = ALLOW_EMPTY | EDIT_MSG;
>> +	unsigned int final_fixup = 0, is_clean;
>>   
>>   	if (has_unstaged_changes(1))
>>   		return error(_("cannot rebase: You have unstaged changes."));
>> -	if (!has_uncommitted_changes(0)) {
>> -		const char *cherry_pick_head = git_path_cherry_pick_head();
>>   
>> -		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
>> -			return error(_("could not remove CHERRY_PICK_HEAD"));
>> -		return 0;
>> -	}
>> +	is_clean = !has_uncommitted_changes(0);
>>   
>>   	if (file_exists(rebase_path_amend())) {
>>   		struct strbuf rev = STRBUF_INIT;
>> @@ -2804,19 +2801,107 @@ static int commit_staged_changes(struct replay_opts *opts)
>>   		if (get_oid_hex(rev.buf, &to_amend))
>>   			return error(_("invalid contents: '%s'"),
>>   				rebase_path_amend());
>> -		if (oidcmp(&head, &to_amend))
>> +		if (!is_clean && oidcmp(&head, &to_amend))
>>   			return error(_("\nYou have uncommitted changes in your "
>>   				       "working tree. Please, commit them\n"
>>   				       "first and then run 'git rebase "
>>   				       "--continue' again."));
>> +		/*
>> +		 * When skipping a failed fixup/squash, we need to edit the
>> +		 * commit message, the current fixup list and count, and if it
>> +		 * was the last fixup/squash in the chain, we need to clean up
>> +		 * the commit message and if there was a squash, let the user
>> +		 * edit it.
>> +		 */
>> +		if (is_clean && !oidcmp(&head, &to_amend) &&
>> +		    opts->current_fixup_count > 0 &&
>> +		    file_exists(rebase_path_stopped_sha())) {
>> +			const char *p = opts->current_fixups.buf;
>> +			int len = opts->current_fixups.len;
>> +
>> +			opts->current_fixup_count--;
>> +			if (!len)
>> +				BUG("Incorrect current_fixups:\n%s", p);
>> +			while (len && p[len - 1] != '\n')
>> +				len--;
>> +			strbuf_setlen(&opts->current_fixups, len);
>> +			if (write_message(p, len, rebase_path_current_fixups(),
>> +					  0) < 0)
>> +				return error(_("could not write file: '%s'"),
>> +					     rebase_path_current_fixups());
>> +
>> +			/*
>> +			 * If a fixup/squash in a fixup/squash chain failed, the
>> +			 * commit message is already correct, no need to commit
>> +			 * it again.
>> +			 *
>> +			 * Only if it is the final command in the fixup/squash
>> +			 * chain, and only if the chain is longer than a single
>> +			 * fixup/squash command (which was just skipped), do we
>> +			 * actually need to re-commit with a cleaned up commit
>> +			 * message.
>> +			 */
>> +			if (opts->current_fixup_count > 0 &&
>> +			    !is_fixup(peek_command(todo_list, 0))) {
>> +				final_fixup = 1;
>> +				/*
>> +				 * If there was not a single "squash" in the
>> +				 * chain, we only need to clean up the commit
>> +				 * message, no need to bother the user with
>> +				 * opening the commit message in the editor.
>> +				 */
>> +				if (!starts_with(p, "squash ") &&
>> +				    !strstr(p, "\nsquash "))
>> +					flags = (flags & ~EDIT_MSG) | CLEANUP_MSG;
>> +			} else if (is_fixup(peek_command(todo_list, 0))) {
>> +				/*
>> +				 * We need to update the squash message to skip
>> +				 * the latest commit message.
>> +				 */
>> +				struct commit *commit;
>> +				const char *path = rebase_path_squash_msg();
>> +
>> +				if (parse_head(&commit) ||
>> +				    !(p = get_commit_buffer(commit, NULL)) ||
>> +				    write_message(p, strlen(p), path, 0)) {
>> +					unuse_commit_buffer(commit, p);

I forgot to mention last time that if parse_head() returns an error then 
commit is passed uninitialized to unuse_commit_buffer(). It also 
possible that p is not pointing to a commit buffer if 
git_commit_buffer() fails. Looking at the current implementation that is 
probably ok but it's a bit ugly and passing uninitialized variables 
around could cause problems in the future.

>> +					return error(_("could not write file: "
>> +						       "'%s'"), path);
>> +				}
> 
> I think it should probably recreate the fixup message as well. If there
> is a sequence
> 
> pick commit
> fixup a
> fixup b
> fixup c
> 
> and 'fixup b' gets skipped then when 'fixup c' is applied the user will
> be prompted to edit the message unless rebase_path_fixup_msg() exists.
> 
> Best Wishes
> 
> Phillip
> 
>> +				unuse_commit_buffer(commit, p);
>> +			}
>> +		}
>>   
>>   		strbuf_release(&rev);
>>   		flags |= AMEND_MSG;
>>   	}
>>   
>> -	if (run_git_commit(rebase_path_message(), opts, flags))
>> +	if (is_clean) {
>> +		const char *cherry_pick_head = git_path_cherry_pick_head();
>> +
>> +		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
>> +			return error(_("could not remove CHERRY_PICK_HEAD"));
>> +		if (!final_fixup)
>> +			return 0;
>> +	}
>> +
>> +	if (run_git_commit(final_fixup ? NULL : rebase_path_message(),
>> +			   opts, flags))
>>   		return error(_("could not commit staged changes."));
>>   	unlink(rebase_path_amend());
>> +	if (final_fixup) {
>> +		unlink(rebase_path_fixup_msg());
>> +		unlink(rebase_path_squash_msg());
>> +	}
>> +	if (opts->current_fixup_count > 0) {
>> +		/*
>> +		 * Whether final fixup or not, we just cleaned up the commit
>> +		 * message...
>> +		 */
>> +		unlink(rebase_path_current_fixups());
>> +		strbuf_reset(&opts->current_fixups);
>> +		opts->current_fixup_count = 0;
>> +	}
>>   	return 0;
>>   }
>>   
>> @@ -2828,14 +2913,16 @@ int sequencer_continue(struct replay_opts *opts)
>>   	if (read_and_refresh_cache(opts))
>>   		return -1;
>>   
>> +	if (read_populate_opts(opts))
>> +		return -1;
>>   	if (is_rebase_i(opts)) {
>> -		if (commit_staged_changes(opts))
>> +		if ((res = read_populate_todo(&todo_list, opts)))
>> +			goto release_todo_list;
>> +		if (commit_staged_changes(opts, &todo_list))
>>   			return -1;
>>   	} else if (!file_exists(get_todo_path(opts)))
>>   		return continue_single_pick();
>> -	if (read_populate_opts(opts))
>> -		return -1;
>> -	if ((res = read_populate_todo(&todo_list, opts)))
>> +	else if ((res = read_populate_todo(&todo_list, opts)))
>>   		goto release_todo_list;
>>   
>>   	if (!is_rebase_i(opts)) {
>> diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
>> index 3874f187246..03bf1b8a3b3 100755
>> --- a/t/t3418-rebase-continue.sh
>> +++ b/t/t3418-rebase-continue.sh
>> @@ -88,14 +88,14 @@ test_expect_success 'rebase passes merge strategy options correctly' '
>>   	git rebase --continue
>>   '
>>   
>> -test_expect_failure '--skip after failed fixup cleans commit message' '
>> +test_expect_success '--skip after failed fixup cleans commit message' '
>>   	test_when_finished "test_might_fail git rebase --abort" &&
>>   	git checkout -b with-conflicting-fixup &&
>>   	test_commit wants-fixup &&
>>   	test_commit "fixup! wants-fixup" wants-fixup.t 1 wants-fixup-1 &&
>>   	test_commit "fixup! wants-fixup" wants-fixup.t 2 wants-fixup-2 &&
>>   	test_commit "fixup! wants-fixup" wants-fixup.t 3 wants-fixup-3 &&
>> -	test_must_fail env FAKE_LINES="1 fixup 2 fixup 4" \
>> +	test_must_fail env FAKE_LINES="1 fixup 2 squash 4" \
>>   		git rebase -i HEAD~4 &&
>>   
>>   	: now there is a conflict, and comments in the commit message &&
>> @@ -103,11 +103,38 @@ test_expect_failure '--skip after failed fixup cleans commit message' '
>>   	grep "fixup! wants-fixup" out &&
>>   
>>   	: skip and continue &&
>> -	git rebase --skip &&
>> +	echo "cp \"\$1\" .git/copy.txt" | write_script copy-editor.sh &&
>> +	(test_set_editor "$PWD/copy-editor.sh" && git rebase --skip) &&
>> +
>> +	: the user should not have had to edit the commit message &&
>> +	test_path_is_missing .git/copy.txt &&
>>   
>>   	: now the comments in the commit message should have been cleaned up &&
>>   	git show HEAD >out &&
>> -	! grep "fixup! wants-fixup" out
>> +	! grep "fixup! wants-fixup" out &&
>> +
>> +	: now, let us ensure that "squash" is handled correctly &&
>> +	git reset --hard wants-fixup-3 &&
>> +	test_must_fail env FAKE_LINES="1 squash 4 squash 2 squash 4" \
>> +		git rebase -i HEAD~4 &&
>> +
>> +	: the first squash failed, but there are two more in the chain &&
>> +	(test_set_editor "$PWD/copy-editor.sh" &&
>> +	 test_must_fail git rebase --skip) &&
>> +
>> +	: not the final squash, no need to edit the commit message &&
>> +	test_path_is_missing .git/copy.txt &&
>> +
>> +	: The first squash was skipped, therefore: &&
>> +	git show HEAD >out &&
>> +	test_i18ngrep "# This is a combination of 2 commits" out &&
>> +
>> +	(test_set_editor "$PWD/copy-editor.sh" && git rebase --skip) &&
>> +	git show HEAD >out &&
>> +	test_i18ngrep ! "# This is a combination" out &&
>> +
>> +	: Final squash failed, but there was still a squash &&
>> +	test_i18ngrep "# This is a combination of 2 commits" .git/copy.txt
>>   '
>>   
>>   test_expect_success 'setup rerere database' '
>>
> 


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH 2/2] unpack_trees_options: free messages when done
  2018-04-28 11:32                           ` Martin Ågren
  2018-04-28 12:30                             ` Johannes Schindelin
  2018-04-28 20:56                             ` Elijah Newren
@ 2018-05-16 14:32                             ` Elijah Newren
  2018-05-16 16:30                               ` [PATCH v2 0/3] " Martin Ågren
  2 siblings, 1 reply; 412+ messages in thread
From: Elijah Newren @ 2018-05-16 14:32 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Git Mailing List, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin

Hi Martin,

On Sat, Apr 28, 2018 at 4:32 AM, Martin Ågren <martin.agren@gmail.com> wrote:
> As you noted elsewhere [1], Ben is also working in this area. I'd be
> perfectly happy to sit on these patches until both of your contributions
> come through to master.
>
> [1] https://public-inbox.org/git/CABPp-BFh=gL6RnbST2bgtynkij1Z5TMgAr1Via5_VyteF5eBMg@mail.gmail.com/

Instead of waiting for these to come through to master, could you just
submit based on the top of bp/merge-rename-config?  I've got several
other merge-recursive changes, some about ready to send and others in
the works.  I don't think any conflict yet, but I would rather avoid
causing you any more waiting or conflicts and would rather just have
both your and Ben's changes in pu and then I can just build mine on
top of those.  Besides, I want to see that FIXME go away and have
fewer leaks.  :-)

Thanks,
Elijah

^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v2 0/3] unpack_trees_options: free messages when done
  2018-05-16 14:32                             ` Elijah Newren
@ 2018-05-16 16:30                               ` Martin Ågren
  2018-05-16 16:30                                 ` [PATCH v2 1/3] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
                                                   ` (3 more replies)
  0 siblings, 4 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-16 16:30 UTC (permalink / raw)
  To: Elijah Newren
  Cc: git, Ben Peart, Jacob Keller, Phillip Wood, Johannes Schindelin

Hi Elijah

On 16 May 2018 at 16:32, Elijah Newren <newren@gmail.com> wrote:
> On Sat, Apr 28, 2018 at 4:32 AM, Martin Ågren <martin.agren@gmail.com> wrote:
>> As you noted elsewhere [1], Ben is also working in this area. I'd be
>> perfectly happy to sit on these patches until both of your contributions
>> come through to master.
>>
>> [1] https://public-inbox.org/git/CABPp-BFh=gL6RnbST2bgtynkij1Z5TMgAr1Via5_VyteF5eBMg@mail.gmail.com/
>
> Instead of waiting for these to come through to master, could you just
> submit based on the top of bp/merge-rename-config? 

Sure, here goes. This is based on bp/merge-rename-config, gets rid of
all leaks of memory allocated in `setup_unpack_trees_porcelain()` and
cuts the number of leaks in the test-suite (i.e., the subset of the 
tests that I run) by around 10%.

Martin

Elijah Newren (1):
  merge-recursive: provide pair of `unpack_trees_{start,finish}()`

Martin Ågren (2):
  merge: setup `opts` later in `checkout_fast_forward()`
  unpack_trees_options: free messages when done

 unpack-trees.h     |  5 +++++
 builtin/checkout.c |  1 +
 merge-recursive.c  | 30 ++++++++++++++++--------------
 merge.c            | 37 +++++++++++++++++++++----------------
 unpack-trees.c     | 11 +++++++++++
 5 files changed, 54 insertions(+), 30 deletions(-)

-- 
2.17.0.583.g9a75a153ac


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v2 1/3] merge: setup `opts` later in `checkout_fast_forward()`
  2018-05-16 16:30                               ` [PATCH v2 0/3] " Martin Ågren
@ 2018-05-16 16:30                                 ` Martin Ågren
  2018-05-16 16:41                                   ` Stefan Beller
  2018-05-17 21:48                                   ` Junio C Hamano
  2018-05-16 16:30                                 ` [PATCH v2 2/3] merge-recursive: provide pair of `unpack_trees_{start,finish}()` Martin Ågren
                                                   ` (2 subsequent siblings)
  3 siblings, 2 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-16 16:30 UTC (permalink / raw)
  To: Elijah Newren
  Cc: git, Ben Peart, Jacob Keller, Phillip Wood, Johannes Schindelin

After we initialize the various fields in `opts` but before we actually
use them, we might return early. Move the initialization further down,
to immediately before we use `opts`.

This limits the scope of `opts` and will help a later commit fix a
memory leak without having to worry about those early returns.

This patch is best viewed using something like this (note the tab!):
--color-moved --anchored="	trees[nr_trees] = parse_tree_indirect"

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
---
 merge.c | 34 ++++++++++++++++++----------------
 1 file changed, 18 insertions(+), 16 deletions(-)

diff --git a/merge.c b/merge.c
index f06a4773d4..f123658e58 100644
--- a/merge.c
+++ b/merge.c
@@ -94,23 +94,7 @@ int checkout_fast_forward(const struct object_id *head,
 		return -1;
 
 	memset(&trees, 0, sizeof(trees));
-	memset(&opts, 0, sizeof(opts));
 	memset(&t, 0, sizeof(t));
-	if (overwrite_ignore) {
-		memset(&dir, 0, sizeof(dir));
-		dir.flags |= DIR_SHOW_IGNORED;
-		setup_standard_excludes(&dir);
-		opts.dir = &dir;
-	}
-
-	opts.head_idx = 1;
-	opts.src_index = &the_index;
-	opts.dst_index = &the_index;
-	opts.update = 1;
-	opts.verbose_update = 1;
-	opts.merge = 1;
-	opts.fn = twoway_merge;
-	setup_unpack_trees_porcelain(&opts, "merge");
 
 	trees[nr_trees] = parse_tree_indirect(head);
 	if (!trees[nr_trees++]) {
@@ -126,6 +110,24 @@ int checkout_fast_forward(const struct object_id *head,
 		parse_tree(trees[i]);
 		init_tree_desc(t+i, trees[i]->buffer, trees[i]->size);
 	}
+
+	memset(&opts, 0, sizeof(opts));
+	if (overwrite_ignore) {
+		memset(&dir, 0, sizeof(dir));
+		dir.flags |= DIR_SHOW_IGNORED;
+		setup_standard_excludes(&dir);
+		opts.dir = &dir;
+	}
+
+	opts.head_idx = 1;
+	opts.src_index = &the_index;
+	opts.dst_index = &the_index;
+	opts.update = 1;
+	opts.verbose_update = 1;
+	opts.merge = 1;
+	opts.fn = twoway_merge;
+	setup_unpack_trees_porcelain(&opts, "merge");
+
 	if (unpack_trees(nr_trees, t, &opts)) {
 		rollback_lock_file(&lock_file);
 		return -1;
-- 
2.17.0.583.g9a75a153ac


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 2/3] merge-recursive: provide pair of `unpack_trees_{start,finish}()`
  2018-05-16 16:30                               ` [PATCH v2 0/3] " Martin Ågren
  2018-05-16 16:30                                 ` [PATCH v2 1/3] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
@ 2018-05-16 16:30                                 ` Martin Ågren
  2018-05-16 16:31                                 ` [PATCH v2 3/3] unpack_trees_options: free messages when done Martin Ågren
  2018-05-16 21:54                                 ` [PATCH v2 0/3] " Elijah Newren
  3 siblings, 0 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-16 16:30 UTC (permalink / raw)
  To: Elijah Newren
  Cc: git, Ben Peart, Jacob Keller, Phillip Wood, Johannes Schindelin

From: Elijah Newren <newren@gmail.com>

Rename `git_merge_trees()` to `unpack_trees_start()` and extract the
call to `discard_index()` into a new function `unpack_trees_finish()`.
As a result, these are called early resp. late in `merge_trees()`,
making the resource handling clearer. The next commit will expand on
that, teaching `..._finish()` to free more memory. (So rather than
moving the FIXME-comment, just drop it, since it will be addressed soon
enough.)

Also call `..._finish()` when `merge_trees()` returns early.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
---
 merge-recursive.c | 29 +++++++++++++++--------------
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 680e01226b..ddb0fa7369 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -337,10 +337,10 @@ static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
 	init_tree_desc(desc, tree->buffer, tree->size);
 }
 
-static int git_merge_trees(struct merge_options *o,
-			   struct tree *common,
-			   struct tree *head,
-			   struct tree *merge)
+static int unpack_trees_start(struct merge_options *o,
+			      struct tree *common,
+			      struct tree *head,
+			      struct tree *merge)
 {
 	int rc;
 	struct tree_desc t[3];
@@ -379,6 +379,11 @@ static int git_merge_trees(struct merge_options *o,
 	return rc;
 }
 
+static void unpack_trees_finish(struct merge_options *o)
+{
+	discard_index(&o->orig_index);
+}
+
 struct tree *write_tree_from_memory(struct merge_options *o)
 {
 	struct tree *result = NULL;
@@ -3088,13 +3093,14 @@ int merge_trees(struct merge_options *o,
 		return 1;
 	}
 
-	code = git_merge_trees(o, common, head, merge);
+	code = unpack_trees_start(o, common, head, merge);
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
 			err(o, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
+		unpack_trees_finish(o);
 		return -1;
 	}
 
@@ -3147,20 +3153,15 @@ int merge_trees(struct merge_options *o,
 
 		hashmap_free(&o->current_file_dir_set, 1);
 
-		if (clean < 0)
+		if (clean < 0) {
+			unpack_trees_finish(o);
 			return clean;
+		}
 	}
 	else
 		clean = 1;
 
-	/* Free the extra index left from git_merge_trees() */
-	/*
-	 * FIXME: Need to also free data allocated by
-	 * setup_unpack_trees_porcelain() tucked away in o->unpack_opts.msgs,
-	 * but the problem is that only half of it refers to dynamically
-	 * allocated data, while the other half points at static strings.
-	 */
-	discard_index(&o->orig_index);
+	unpack_trees_finish(o);
 
 	if (o->call_depth && !(*result = write_tree_from_memory(o)))
 		return -1;
-- 
2.17.0.583.g9a75a153ac


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v2 3/3] unpack_trees_options: free messages when done
  2018-05-16 16:30                               ` [PATCH v2 0/3] " Martin Ågren
  2018-05-16 16:30                                 ` [PATCH v2 1/3] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
  2018-05-16 16:30                                 ` [PATCH v2 2/3] merge-recursive: provide pair of `unpack_trees_{start,finish}()` Martin Ågren
@ 2018-05-16 16:31                                 ` Martin Ågren
  2018-05-17 22:10                                   ` Junio C Hamano
  2018-05-16 21:54                                 ` [PATCH v2 0/3] " Elijah Newren
  3 siblings, 1 reply; 412+ messages in thread
From: Martin Ågren @ 2018-05-16 16:31 UTC (permalink / raw)
  To: Elijah Newren
  Cc: git, Ben Peart, Jacob Keller, Phillip Wood, Johannes Schindelin

The strings allocated in `setup_unpack_trees_porcelain()` are never
freed. Provide a function `clear_unpack_trees_porcelain()` to do so and
call it where we use `setup_unpack_trees_porcelain()`. The only
non-trivial user is `unpack_trees_start()`, where we should place the
new call in `unpack_trees_finish()`.

The `opts` string array contains multiple copies of the same pointers.
Be careful to only free each pointer once, then zeroize the whole array
so that we do not leave any dangling pointers.

Note that we only take responsibility for the memory allocated in
`setup_unpack_trees_porcelain()` and not any other members of the
`struct unpack_trees_options`.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
---
 unpack-trees.h     |  5 +++++
 builtin/checkout.c |  1 +
 merge-recursive.c  |  1 +
 merge.c            |  3 +++
 unpack-trees.c     | 11 +++++++++++
 5 files changed, 21 insertions(+)

diff --git a/unpack-trees.h b/unpack-trees.h
index 41178ada94..70053cb3ff 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -33,6 +33,11 @@ enum unpack_trees_error_types {
 void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 				  const char *cmd);
 
+/*
+ * Frees resources allocated by setup_unpack_trees_porcelain().
+ */
+extern void clear_unpack_trees_porcelain(struct unpack_trees_options *opts);
+
 struct unpack_trees_options {
 	unsigned int reset,
 		     merge,
diff --git a/builtin/checkout.c b/builtin/checkout.c
index b49b582071..5cebe170fc 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -526,6 +526,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 		init_tree_desc(&trees[1], tree->buffer, tree->size);
 
 		ret = unpack_trees(2, trees, &topts);
+		clear_unpack_trees_porcelain(&topts);
 		if (ret == -1) {
 			/*
 			 * Unpack couldn't do a trivial merge; either
diff --git a/merge-recursive.c b/merge-recursive.c
index ddb0fa7369..338f63a952 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -382,6 +382,7 @@ static int unpack_trees_start(struct merge_options *o,
 static void unpack_trees_finish(struct merge_options *o)
 {
 	discard_index(&o->orig_index);
+	clear_unpack_trees_porcelain(&o->unpack_opts);
 }
 
 struct tree *write_tree_from_memory(struct merge_options *o)
diff --git a/merge.c b/merge.c
index f123658e58..b433291d0c 100644
--- a/merge.c
+++ b/merge.c
@@ -130,8 +130,11 @@ int checkout_fast_forward(const struct object_id *head,
 
 	if (unpack_trees(nr_trees, t, &opts)) {
 		rollback_lock_file(&lock_file);
+		clear_unpack_trees_porcelain(&opts);
 		return -1;
 	}
+	clear_unpack_trees_porcelain(&opts);
+
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		return error(_("unable to write new index file"));
 	return 0;
diff --git a/unpack-trees.c b/unpack-trees.c
index 79fd97074e..25e766d30e 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -179,6 +179,17 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 		opts->unpack_rejects[i].strdup_strings = 1;
 }
 
+void clear_unpack_trees_porcelain(struct unpack_trees_options *opts)
+{
+	char **msgs = (char **)opts->msgs;
+
+	free(msgs[ERROR_WOULD_OVERWRITE]);
+	free(msgs[ERROR_WOULD_LOSE_UNTRACKED_REMOVED]);
+	free(msgs[ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN]);
+
+	memset(opts->msgs, 0, sizeof(opts->msgs));
+}
+
 static int do_add_entry(struct unpack_trees_options *o, struct cache_entry *ce,
 			 unsigned int set, unsigned int clear)
 {
-- 
2.17.0.583.g9a75a153ac


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 1/3] merge: setup `opts` later in `checkout_fast_forward()`
  2018-05-16 16:30                                 ` [PATCH v2 1/3] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
@ 2018-05-16 16:41                                   ` Stefan Beller
  2018-05-16 19:29                                     ` Martin Ågren
  2018-05-16 21:20                                     ` Jacob Keller
  2018-05-17 21:48                                   ` Junio C Hamano
  1 sibling, 2 replies; 412+ messages in thread
From: Stefan Beller @ 2018-05-16 16:41 UTC (permalink / raw)
  To: Martin Ågren, Jonathan Tan; +Cc: git

+ Jonathan Tan for a side discussion on anchoring.

On Wed, May 16, 2018 at 9:30 AM, Martin Ågren <martin.agren@gmail.com> wrote:
>
> This patch is best viewed using something like this (note the tab!):
> --color-moved --anchored="      trees[nr_trees] = parse_tree_indirect"

Heh! Having a "is best viewed" paragraph is the new shiny thing in
commit messages as 'git log origin/pu --grep "is best viewed"' tells me.

Regarding the anchoring, I wonder if we can improve it by ignoring
whitespaces or just looking for substrings, or by allowing regexes or ...

Thanks,
Stefan

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 1/3] merge: setup `opts` later in `checkout_fast_forward()`
  2018-05-16 16:41                                   ` Stefan Beller
@ 2018-05-16 19:29                                     ` Martin Ågren
  2018-05-16 21:21                                       ` Jacob Keller
  2018-05-16 21:20                                     ` Jacob Keller
  1 sibling, 1 reply; 412+ messages in thread
From: Martin Ågren @ 2018-05-16 19:29 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Jonathan Tan, git

On 16 May 2018 at 18:41, Stefan Beller <sbeller@google.com> wrote:
> On Wed, May 16, 2018 at 9:30 AM, Martin Ågren <martin.agren@gmail.com> wrote:
>>
>> This patch is best viewed using something like this (note the tab!):
>> --color-moved --anchored="      trees[nr_trees] = parse_tree_indirect"
>
> Heh! Having a "is best viewed" paragraph is the new shiny thing in
> commit messages as 'git log origin/pu --grep "is best viewed"' tells me.

:-)

> Regarding the anchoring, I wonder if we can improve it by ignoring
> whitespaces or just looking for substrings, or by allowing regexes or ...

FWIW, because my first naive attempt failed (for some reason I did not
consider the leading tab part of the "line" so I did not provide it), I
had the same thought. Ignoring leading whitespace seemed easy enough in
the implementation.

Then I started thinking about all the ways in which whitespace can be
ignored. My reaction in the end was to not try and open that can right
there and then. I did not think about regexes.

I guess this boils down to the usage. Copying the line to anchor on from
an editor could run into these kind of whitespace-issues, and shell
escaping. Typing an anchor could become easier with regexes since one
could skip typing common substrings and just anchor on /unique-part/.

Martin

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 1/3] merge: setup `opts` later in `checkout_fast_forward()`
  2018-05-16 16:41                                   ` Stefan Beller
  2018-05-16 19:29                                     ` Martin Ågren
@ 2018-05-16 21:20                                     ` Jacob Keller
  1 sibling, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-05-16 21:20 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Martin Ågren, Jonathan Tan, git

On Wed, May 16, 2018 at 9:41 AM, Stefan Beller <sbeller@google.com> wrote:
> + Jonathan Tan for a side discussion on anchoring.
>
> On Wed, May 16, 2018 at 9:30 AM, Martin Ågren <martin.agren@gmail.com> wrote:
>>
>> This patch is best viewed using something like this (note the tab!):
>> --color-moved --anchored="      trees[nr_trees] = parse_tree_indirect"
>
> Heh! Having a "is best viewed" paragraph is the new shiny thing in
> commit messages as 'git log origin/pu --grep "is best viewed"' tells me.
>
> Regarding the anchoring, I wonder if we can improve it by ignoring
> whitespaces or just looking for substrings, or by allowing regexes or ...
>
> Thanks,
> Stefan

I think expanding it to be regexp would be nicest. To be honest, I
already thought it was substring based....

It'd be *really* cool if we had a way for a commit messages (or maybe
notes?) to indicate the anchor so that git show could (optionally)
figure out the anchor automatically. It's been REALLY useful for me
when showing diffs to be able to provide a better idea of what a human
*actually* did vs what the smallest diff was.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 1/3] merge: setup `opts` later in `checkout_fast_forward()`
  2018-05-16 19:29                                     ` Martin Ågren
@ 2018-05-16 21:21                                       ` Jacob Keller
  0 siblings, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-05-16 21:21 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Stefan Beller, Jonathan Tan, git

On Wed, May 16, 2018 at 12:29 PM, Martin Ågren <martin.agren@gmail.com> wrote:
> On 16 May 2018 at 18:41, Stefan Beller <sbeller@google.com> wrote:
>> On Wed, May 16, 2018 at 9:30 AM, Martin Ågren <martin.agren@gmail.com> wrote:
>>>
>>> This patch is best viewed using something like this (note the tab!):
>>> --color-moved --anchored="      trees[nr_trees] = parse_tree_indirect"
>>
>> Heh! Having a "is best viewed" paragraph is the new shiny thing in
>> commit messages as 'git log origin/pu --grep "is best viewed"' tells me.
>
> :-)
>
>> Regarding the anchoring, I wonder if we can improve it by ignoring
>> whitespaces or just looking for substrings, or by allowing regexes or ...
>
> FWIW, because my first naive attempt failed (for some reason I did not
> consider the leading tab part of the "line" so I did not provide it), I
> had the same thought. Ignoring leading whitespace seemed easy enough in
> the implementation.
>
> Then I started thinking about all the ways in which whitespace can be
> ignored. My reaction in the end was to not try and open that can right
> there and then. I did not think about regexes.
>
> I guess this boils down to the usage. Copying the line to anchor on from
> an editor could run into these kind of whitespace-issues, and shell
> escaping. Typing an anchor could become easier with regexes since one
> could skip typing common substrings and just anchor on /unique-part/.
>
> Martin

Simpler approach is to just match substring instead. Then, the user
can decide how much of the string is required to get the anchor they
wanted.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 0/3] unpack_trees_options: free messages when done
  2018-05-16 16:30                               ` [PATCH v2 0/3] " Martin Ågren
                                                   ` (2 preceding siblings ...)
  2018-05-16 16:31                                 ` [PATCH v2 3/3] unpack_trees_options: free messages when done Martin Ågren
@ 2018-05-16 21:54                                 ` Elijah Newren
  2018-05-17 12:09                                   ` Ben Peart
  3 siblings, 1 reply; 412+ messages in thread
From: Elijah Newren @ 2018-05-16 21:54 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Git Mailing List, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin

Hi Martin,

On Wed, May 16, 2018 at 9:30 AM, Martin Ågren <martin.agren@gmail.com> wrote:
> On 16 May 2018 at 16:32, Elijah Newren <newren@gmail.com> wrote:
>> On Sat, Apr 28, 2018 at 4:32 AM, Martin Ågren <martin.agren@gmail.com> wrote:
>>> As you noted elsewhere [1], Ben is also working in this area. I'd be
>>> perfectly happy to sit on these patches until both of your contributions
>>> come through to master.
>>>
>>> [1] https://public-inbox.org/git/CABPp-BFh=gL6RnbST2bgtynkij1Z5TMgAr1Via5_VyteF5eBMg@mail.gmail.com/
>>
>> Instead of waiting for these to come through to master, could you just
>> submit based on the top of bp/merge-rename-config?
>
> Sure, here goes. This is based on bp/merge-rename-config, gets rid of
> all leaks of memory allocated in `setup_unpack_trees_porcelain()` and
> cuts the number of leaks in the test-suite (i.e., the subset of the
> tests that I run) by around 10%.

Awesome, thanks.  I've looked over patches 2 & 3; they look good to me.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 0/3] unpack_trees_options: free messages when done
  2018-05-16 21:54                                 ` [PATCH v2 0/3] " Elijah Newren
@ 2018-05-17 12:09                                   ` Ben Peart
  0 siblings, 0 replies; 412+ messages in thread
From: Ben Peart @ 2018-05-17 12:09 UTC (permalink / raw)
  To: Elijah Newren, Martin Ågren
  Cc: Git Mailing List, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin



On 5/16/2018 5:54 PM, Elijah Newren wrote:
> Hi Martin,
> 
> On Wed, May 16, 2018 at 9:30 AM, Martin Ågren <martin.agren@gmail.com> wrote:
>> On 16 May 2018 at 16:32, Elijah Newren <newren@gmail.com> wrote:
>>> On Sat, Apr 28, 2018 at 4:32 AM, Martin Ågren <martin.agren@gmail.com> wrote:
>>>> As you noted elsewhere [1], Ben is also working in this area. I'd be
>>>> perfectly happy to sit on these patches until both of your contributions
>>>> come through to master.
>>>>
>>>> [1] https://public-inbox.org/git/CABPp-BFh=gL6RnbST2bgtynkij1Z5TMgAr1Via5_VyteF5eBMg@mail.gmail.com/
>>>
>>> Instead of waiting for these to come through to master, could you just
>>> submit based on the top of bp/merge-rename-config?
>>
>> Sure, here goes. This is based on bp/merge-rename-config, gets rid of
>> all leaks of memory allocated in `setup_unpack_trees_porcelain()` and
>> cuts the number of leaks in the test-suite (i.e., the subset of the
>> tests that I run) by around 10%.
> 
> Awesome, thanks.  I've looked over patches 2 & 3; they look good to me.
> 

I like the symmetry of the naming and locality of the functions.  Should 
help people remember to keep the xstrfmt() and associated free() in 
sync.  Patches look good to me as well.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 1/3] merge: setup `opts` later in `checkout_fast_forward()`
  2018-05-16 16:30                                 ` [PATCH v2 1/3] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
  2018-05-16 16:41                                   ` Stefan Beller
@ 2018-05-17 21:48                                   ` Junio C Hamano
  2018-05-18  1:59                                     ` Jacob Keller
  1 sibling, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-05-17 21:48 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Elijah Newren, git, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin

Martin Ågren <martin.agren@gmail.com> writes:

> After we initialize the various fields in `opts` but before we actually
> use them, we might return early. Move the initialization further down,
> to immediately before we use `opts`.
>
> This limits the scope of `opts` and will help a later commit fix a
> memory leak without having to worry about those early returns.
>
> This patch is best viewed using something like this (note the tab!):
> --color-moved --anchored="	trees[nr_trees] = parse_tree_indirect"

This side remark is interesting because it totally depends on how
you look at it.  I think "initialize opts late" and "attempt to
parse the trees first and fail early" are the sides of the same
coin, and the diff shown without the anchor matches the latter,
which is also perfectly acceptable interpretation of what this patch
does.


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 3/3] unpack_trees_options: free messages when done
  2018-05-16 16:31                                 ` [PATCH v2 3/3] unpack_trees_options: free messages when done Martin Ågren
@ 2018-05-17 22:10                                   ` Junio C Hamano
  2018-05-18  5:08                                     ` Martin Ågren
  0 siblings, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-05-17 22:10 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Elijah Newren, git, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin

Martin Ågren <martin.agren@gmail.com> writes:

> The strings allocated in `setup_unpack_trees_porcelain()` are never
> freed. Provide a function `clear_unpack_trees_porcelain()` to do so and
> call it where we use `setup_unpack_trees_porcelain()`. The only
> non-trivial user is `unpack_trees_start()`, where we should place the
> new call in `unpack_trees_finish()`.
>
> The `opts` string array contains multiple copies of the same pointers.
> Be careful to only free each pointer once, then zeroize the whole array
> so that we do not leave any dangling pointers.

The verb to make it zero or fill it with zero is "to zero", I would
think.

To be honest I am not sure if I like the way this change is done.
The clear_unpack_trees_porcelain() function has too intimate
knowledge of what happens inside the setup_unpack_trees_porcelain()
function; it not just knows which fields are always allocated but
which are duplicates, which must be double checked for updates
whenever the latter gets modified, yet there is no large warning
sign painted in red in the latter, so it is easy to change the
latter and invalidate the assumption the former makes by mistake,
leading to new leaks and/or double freeing.

I wonder if an approach that is longer-term a bit more maintainable
is to add a new string-list instance to opts, save these xstrfmt()'ed
messages to it when setup_unpack_trees_porcelain() create them, and
then make clear_unpack_trees_porcelain() pay *no* attention to msg[]
array and the positions of these allocated messages and duplicates
but just reclaim the resources held in that string-list, or
something like that.


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 1/3] merge: setup `opts` later in `checkout_fast_forward()`
  2018-05-17 21:48                                   ` Junio C Hamano
@ 2018-05-18  1:59                                     ` Jacob Keller
  0 siblings, 0 replies; 412+ messages in thread
From: Jacob Keller @ 2018-05-18  1:59 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Martin Ågren, Elijah Newren, Git mailing list, Ben Peart,
	Phillip Wood, Johannes Schindelin

On Thu, May 17, 2018 at 2:48 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Martin Ågren <martin.agren@gmail.com> writes:
>
>> After we initialize the various fields in `opts` but before we actually
>> use them, we might return early. Move the initialization further down,
>> to immediately before we use `opts`.
>>
>> This limits the scope of `opts` and will help a later commit fix a
>> memory leak without having to worry about those early returns.
>>
>> This patch is best viewed using something like this (note the tab!):
>> --color-moved --anchored="    trees[nr_trees] = parse_tree_indirect"
>
> This side remark is interesting because it totally depends on how
> you look at it.  I think "initialize opts late" and "attempt to
> parse the trees first and fail early" are the sides of the same
> coin, and the diff shown without the anchor matches the latter,
> which is also perfectly acceptable interpretation of what this patch
> does.
>

Yes. I like that we have tools available to show diffs in different
hopefully meaningful ways.

I happen to like when the diff matches my mental map of the change
after reading the commit message, so having the author indicate how
best to view it is useful, but definitely cool to see that we can get
different interpretations.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v2 3/3] unpack_trees_options: free messages when done
  2018-05-17 22:10                                   ` Junio C Hamano
@ 2018-05-18  5:08                                     ` Martin Ågren
  2018-05-18 21:23                                       ` [PATCH v3 0/3] " Martin Ågren
  0 siblings, 1 reply; 412+ messages in thread
From: Martin Ågren @ 2018-05-18  5:08 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Elijah Newren, Git Mailing List, Ben Peart, Jacob Keller,
	Phillip Wood, Johannes Schindelin

On 18 May 2018 at 00:10, Junio C Hamano <gitster@pobox.com> wrote:
> Martin Ågren <martin.agren@gmail.com> writes:
>
>> The `opts` string array contains multiple copies of the same pointers.
>> Be careful to only free each pointer once, then zeroize the whole array
>> so that we do not leave any dangling pointers.

> I wonder if an approach that is longer-term a bit more maintainable
> is to add a new string-list instance to opts, save these xstrfmt()'ed
> messages to it when setup_unpack_trees_porcelain() create them, and
> then make clear_unpack_trees_porcelain() pay *no* attention to msg[]
> array and the positions of these allocated messages and duplicates
> but just reclaim the resources held in that string-list, or
> something like that.

Thank you for thoughts and this suggestion. I will try this out,
hopefully later today.

Martin

^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v3 0/3] unpack_trees_options: free messages when done
  2018-05-18  5:08                                     ` Martin Ågren
@ 2018-05-18 21:23                                       ` Martin Ågren
  2018-05-18 21:23                                         ` [PATCH v3 1/3] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
                                                           ` (2 more replies)
  0 siblings, 3 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-18 21:23 UTC (permalink / raw)
  To: git
  Cc: Elijah Newren, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin, Junio C Hamano

This is a reroll of my attempt at freeing the memory allocated by
`setup_unpack_trees_porcelain()`. The first two patches are identical to
v2. The third patch no longer relies on rather intimate knowledge of
which strings are on the heap and which pointers are duplicates.
Instead, as suggested by Junio, I keep a separate string-list of strings
to free. That should make things more future-proof.

v2: https://public-inbox.org/git/cover.1526488122.git.martin.agren@gmail.com/

Martin

Elijah Newren (1):
  merge-recursive: provide pair of `unpack_trees_{start,finish}()`

Martin Ågren (2):
  merge: setup `opts` later in `checkout_fast_forward()`
  unpack_trees_options: free messages when done

 unpack-trees.h     |  6 ++++++
 builtin/checkout.c |  1 +
 merge-recursive.c  | 30 ++++++++++++++++--------------
 merge.c            | 35 ++++++++++++++++++++---------------
 unpack-trees.c     | 23 +++++++++++++++++++----
 5 files changed, 62 insertions(+), 33 deletions(-)

-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v3 1/3] merge: setup `opts` later in `checkout_fast_forward()`
  2018-05-18 21:23                                       ` [PATCH v3 0/3] " Martin Ågren
@ 2018-05-18 21:23                                         ` Martin Ågren
  2018-05-18 21:23                                         ` [PATCH v3 2/3] merge-recursive: provide pair of `unpack_trees_{start,finish}()` Martin Ågren
  2018-05-18 21:23                                         ` [PATCH v3 3/3] unpack_trees_options: free messages when done Martin Ågren
  2 siblings, 0 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-18 21:23 UTC (permalink / raw)
  To: git
  Cc: Elijah Newren, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin, Junio C Hamano

After we initialize the various fields in `opts` but before we actually
use them, we might return early. Move the initialization further down,
to immediately before we use `opts`.

This limits the scope of `opts` and will help a later commit fix a
memory leak without having to worry about those early returns.

This patch is best viewed using something like this (note the tab!):
--color-moved --anchored="	trees[nr_trees] = parse_tree_indirect"

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 merge.c | 32 +++++++++++++++++---------------
 1 file changed, 17 insertions(+), 15 deletions(-)

diff --git a/merge.c b/merge.c
index f06a4773d4..f123658e58 100644
--- a/merge.c
+++ b/merge.c
@@ -94,8 +94,24 @@ int checkout_fast_forward(const struct object_id *head,
 		return -1;
 
 	memset(&trees, 0, sizeof(trees));
-	memset(&opts, 0, sizeof(opts));
 	memset(&t, 0, sizeof(t));
+
+	trees[nr_trees] = parse_tree_indirect(head);
+	if (!trees[nr_trees++]) {
+		rollback_lock_file(&lock_file);
+		return -1;
+	}
+	trees[nr_trees] = parse_tree_indirect(remote);
+	if (!trees[nr_trees++]) {
+		rollback_lock_file(&lock_file);
+		return -1;
+	}
+	for (i = 0; i < nr_trees; i++) {
+		parse_tree(trees[i]);
+		init_tree_desc(t+i, trees[i]->buffer, trees[i]->size);
+	}
+
+	memset(&opts, 0, sizeof(opts));
 	if (overwrite_ignore) {
 		memset(&dir, 0, sizeof(dir));
 		dir.flags |= DIR_SHOW_IGNORED;
@@ -112,20 +128,6 @@ int checkout_fast_forward(const struct object_id *head,
 	opts.fn = twoway_merge;
 	setup_unpack_trees_porcelain(&opts, "merge");
 
-	trees[nr_trees] = parse_tree_indirect(head);
-	if (!trees[nr_trees++]) {
-		rollback_lock_file(&lock_file);
-		return -1;
-	}
-	trees[nr_trees] = parse_tree_indirect(remote);
-	if (!trees[nr_trees++]) {
-		rollback_lock_file(&lock_file);
-		return -1;
-	}
-	for (i = 0; i < nr_trees; i++) {
-		parse_tree(trees[i]);
-		init_tree_desc(t+i, trees[i]->buffer, trees[i]->size);
-	}
 	if (unpack_trees(nr_trees, t, &opts)) {
 		rollback_lock_file(&lock_file);
 		return -1;
-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 2/3] merge-recursive: provide pair of `unpack_trees_{start,finish}()`
  2018-05-18 21:23                                       ` [PATCH v3 0/3] " Martin Ågren
  2018-05-18 21:23                                         ` [PATCH v3 1/3] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
@ 2018-05-18 21:23                                         ` Martin Ågren
  2018-05-18 21:23                                         ` [PATCH v3 3/3] unpack_trees_options: free messages when done Martin Ågren
  2 siblings, 0 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-18 21:23 UTC (permalink / raw)
  To: git
  Cc: Elijah Newren, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin, Junio C Hamano

From: Elijah Newren <newren@gmail.com>

Rename `git_merge_trees()` to `unpack_trees_start()` and extract the
call to `discard_index()` into a new function `unpack_trees_finish()`.
As a result, these are called early resp. late in `merge_trees()`,
making the resource handling clearer. The next commit will expand on
that, teaching `..._finish()` to free more memory. (So rather than
moving the FIXME-comment, just drop it, since it will be addressed soon
enough.)

Also call `..._finish()` when `merge_trees()` returns early.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 merge-recursive.c | 29 +++++++++++++++--------------
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 680e01226b..ddb0fa7369 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -337,10 +337,10 @@ static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
 	init_tree_desc(desc, tree->buffer, tree->size);
 }
 
-static int git_merge_trees(struct merge_options *o,
-			   struct tree *common,
-			   struct tree *head,
-			   struct tree *merge)
+static int unpack_trees_start(struct merge_options *o,
+			      struct tree *common,
+			      struct tree *head,
+			      struct tree *merge)
 {
 	int rc;
 	struct tree_desc t[3];
@@ -379,6 +379,11 @@ static int git_merge_trees(struct merge_options *o,
 	return rc;
 }
 
+static void unpack_trees_finish(struct merge_options *o)
+{
+	discard_index(&o->orig_index);
+}
+
 struct tree *write_tree_from_memory(struct merge_options *o)
 {
 	struct tree *result = NULL;
@@ -3088,13 +3093,14 @@ int merge_trees(struct merge_options *o,
 		return 1;
 	}
 
-	code = git_merge_trees(o, common, head, merge);
+	code = unpack_trees_start(o, common, head, merge);
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
 			err(o, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
+		unpack_trees_finish(o);
 		return -1;
 	}
 
@@ -3147,20 +3153,15 @@ int merge_trees(struct merge_options *o,
 
 		hashmap_free(&o->current_file_dir_set, 1);
 
-		if (clean < 0)
+		if (clean < 0) {
+			unpack_trees_finish(o);
 			return clean;
+		}
 	}
 	else
 		clean = 1;
 
-	/* Free the extra index left from git_merge_trees() */
-	/*
-	 * FIXME: Need to also free data allocated by
-	 * setup_unpack_trees_porcelain() tucked away in o->unpack_opts.msgs,
-	 * but the problem is that only half of it refers to dynamically
-	 * allocated data, while the other half points at static strings.
-	 */
-	discard_index(&o->orig_index);
+	unpack_trees_finish(o);
 
 	if (o->call_depth && !(*result = write_tree_from_memory(o)))
 		return -1;
-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v3 3/3] unpack_trees_options: free messages when done
  2018-05-18 21:23                                       ` [PATCH v3 0/3] " Martin Ågren
  2018-05-18 21:23                                         ` [PATCH v3 1/3] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
  2018-05-18 21:23                                         ` [PATCH v3 2/3] merge-recursive: provide pair of `unpack_trees_{start,finish}()` Martin Ågren
@ 2018-05-18 21:23                                         ` Martin Ågren
  2018-05-18 21:33                                           ` Jeff King
  2 siblings, 1 reply; 412+ messages in thread
From: Martin Ågren @ 2018-05-18 21:23 UTC (permalink / raw)
  To: git
  Cc: Elijah Newren, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin, Junio C Hamano

The strings allocated in `setup_unpack_trees_porcelain()` are never
freed. Provide a function `clear_unpack_trees_porcelain()` to do so and
call it where we use `setup_unpack_trees_porcelain()`. The only
non-trivial user is `unpack_trees_start()`, where we should place the
new call in `unpack_trees_finish()`.

We keep the string pointers in an array, mixing pointers to static
memory and memory that we allocate on the heap. We also keep several
copies of the individual pointers. So we need to make sure that we do
not free what we must not free and that we do not double-free. Keep the
unique, heap-allocated pointers in a separate string list, to make the
freeing safe and future-proof.

Zero the whole array of string pointers to make sure that we do not
leave any dangling pointers.

Note that we only take responsibility for the memory allocated in
`setup_unpack_trees_porcelain()` and not any other members of the
`struct unpack_trees_options`.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
---
 unpack-trees.h     |  6 ++++++
 builtin/checkout.c |  1 +
 merge-recursive.c  |  1 +
 merge.c            |  3 +++
 unpack-trees.c     | 23 +++++++++++++++++++----
 5 files changed, 30 insertions(+), 4 deletions(-)

diff --git a/unpack-trees.h b/unpack-trees.h
index 41178ada94..5a84123a40 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -33,6 +33,11 @@ enum unpack_trees_error_types {
 void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 				  const char *cmd);
 
+/*
+ * Frees resources allocated by setup_unpack_trees_porcelain().
+ */
+void clear_unpack_trees_porcelain(struct unpack_trees_options *opts);
+
 struct unpack_trees_options {
 	unsigned int reset,
 		     merge,
@@ -57,6 +62,7 @@ struct unpack_trees_options {
 	struct pathspec *pathspec;
 	merge_fn_t fn;
 	const char *msgs[NB_UNPACK_TREES_ERROR_TYPES];
+	struct string_list msgs_to_free;
 	/*
 	 * Store error messages in an array, each case
 	 * corresponding to a error message type
diff --git a/builtin/checkout.c b/builtin/checkout.c
index b49b582071..5cebe170fc 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -526,6 +526,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 		init_tree_desc(&trees[1], tree->buffer, tree->size);
 
 		ret = unpack_trees(2, trees, &topts);
+		clear_unpack_trees_porcelain(&topts);
 		if (ret == -1) {
 			/*
 			 * Unpack couldn't do a trivial merge; either
diff --git a/merge-recursive.c b/merge-recursive.c
index ddb0fa7369..338f63a952 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -382,6 +382,7 @@ static int unpack_trees_start(struct merge_options *o,
 static void unpack_trees_finish(struct merge_options *o)
 {
 	discard_index(&o->orig_index);
+	clear_unpack_trees_porcelain(&o->unpack_opts);
 }
 
 struct tree *write_tree_from_memory(struct merge_options *o)
diff --git a/merge.c b/merge.c
index f123658e58..b433291d0c 100644
--- a/merge.c
+++ b/merge.c
@@ -130,8 +130,11 @@ int checkout_fast_forward(const struct object_id *head,
 
 	if (unpack_trees(nr_trees, t, &opts)) {
 		rollback_lock_file(&lock_file);
+		clear_unpack_trees_porcelain(&opts);
 		return -1;
 	}
+	clear_unpack_trees_porcelain(&opts);
+
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		return error(_("unable to write new index file"));
 	return 0;
diff --git a/unpack-trees.c b/unpack-trees.c
index 79fd97074e..60293ff536 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -103,6 +103,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 	const char **msgs = opts->msgs;
 	const char *msg;
 
+	opts->msgs_to_free.strdup_strings = 0;
+
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_commit_before_merge
 		      ? _("Your local changes to the following files would be overwritten by checkout:\n%%s"
@@ -118,8 +120,9 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 		      ? _("Your local changes to the following files would be overwritten by %s:\n%%s"
 			  "Please commit your changes or stash them before you %s.")
 		      : _("Your local changes to the following files would be overwritten by %s:\n%%s");
-	msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOT_UPTODATE_FILE] =
-		xstrfmt(msg, cmd, cmd);
+	msg = xstrfmt(msg, cmd, cmd);
+	msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOT_UPTODATE_FILE] = msg;
+	string_list_append(&opts->msgs_to_free, msg);
 
 	msgs[ERROR_NOT_UPTODATE_DIR] =
 		_("Updating the following directories would lose untracked files in them:\n%s");
@@ -139,7 +142,9 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 		      ? _("The following untracked working tree files would be removed by %s:\n%%s"
 			  "Please move or remove them before you %s.")
 		      : _("The following untracked working tree files would be removed by %s:\n%%s");
-	msgs[ERROR_WOULD_LOSE_UNTRACKED_REMOVED] = xstrfmt(msg, cmd, cmd);
+	msg = xstrfmt(msg, cmd, cmd);
+	msgs[ERROR_WOULD_LOSE_UNTRACKED_REMOVED] = msg;
+	string_list_append(&opts->msgs_to_free, msg);
 
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_commit_before_merge
@@ -156,7 +161,9 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 		      ? _("The following untracked working tree files would be overwritten by %s:\n%%s"
 			  "Please move or remove them before you %s.")
 		      : _("The following untracked working tree files would be overwritten by %s:\n%%s");
-	msgs[ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN] = xstrfmt(msg, cmd, cmd);
+	msg = xstrfmt(msg, cmd, cmd);
+	msgs[ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN] = msg;
+	string_list_append(&opts->msgs_to_free, msg);
 
 	/*
 	 * Special case: ERROR_BIND_OVERLAP refers to a pair of paths, we
@@ -179,6 +186,14 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 		opts->unpack_rejects[i].strdup_strings = 1;
 }
 
+void clear_unpack_trees_porcelain(struct unpack_trees_options *opts)
+{
+	opts->msgs_to_free.strdup_strings = 1;
+	string_list_clear(&opts->msgs_to_free, 0);
+
+	memset(opts->msgs, 0, sizeof(opts->msgs));
+}
+
 static int do_add_entry(struct unpack_trees_options *o, struct cache_entry *ce,
 			 unsigned int set, unsigned int clear)
 {
-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 3/3] unpack_trees_options: free messages when done
  2018-05-18 21:23                                         ` [PATCH v3 3/3] unpack_trees_options: free messages when done Martin Ågren
@ 2018-05-18 21:33                                           ` Jeff King
  2018-05-18 22:30                                             ` Elijah Newren
  0 siblings, 1 reply; 412+ messages in thread
From: Jeff King @ 2018-05-18 21:33 UTC (permalink / raw)
  To: Martin Ågren
  Cc: git, Elijah Newren, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin, Junio C Hamano

On Fri, May 18, 2018 at 11:23:27PM +0200, Martin Ågren wrote:

> diff --git a/unpack-trees.c b/unpack-trees.c
> index 79fd97074e..60293ff536 100644
> --- a/unpack-trees.c
> +++ b/unpack-trees.c
> @@ -103,6 +103,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
>  	const char **msgs = opts->msgs;
>  	const char *msg;
>  
> +	opts->msgs_to_free.strdup_strings = 0;
> +
> [...]
> +void clear_unpack_trees_porcelain(struct unpack_trees_options *opts)
> +{
> +	opts->msgs_to_free.strdup_strings = 1;
> +	string_list_clear(&opts->msgs_to_free, 0);

I like this string_list approach much better, but it's too bad we have
to go through these contortions with the strdup flag to get the memory
ownership right.

If we had a string_list_appendf(), then we could just leave that flag
alone and this:

> @@ -118,8 +120,9 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
>  		      ? _("Your local changes to the following files would be overwritten by %s:\n%%s"
>  			  "Please commit your changes or stash them before you %s.")
>  		      : _("Your local changes to the following files would be overwritten by %s:\n%%s");
> -	msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOT_UPTODATE_FILE] =
> -		xstrfmt(msg, cmd, cmd);
> +	msg = xstrfmt(msg, cmd, cmd);
> +	msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOT_UPTODATE_FILE] = msg;
> +	string_list_append(&opts->msgs_to_free, msg);

would become:

  msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOUPTODATE_FILE] =
	string_list_appendf(&opts->msgs_to_free, msg, cmd, cmd)->string;

I don't know if that's worth it or not (I suspect that there are other
places where appendf would be handy, but I didn't poke around).

-Peff

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 3/3] unpack_trees_options: free messages when done
  2018-05-18 21:33                                           ` Jeff King
@ 2018-05-18 22:30                                             ` Elijah Newren
  2018-05-19  1:02                                               ` Jeff King
  0 siblings, 1 reply; 412+ messages in thread
From: Elijah Newren @ 2018-05-18 22:30 UTC (permalink / raw)
  To: Jeff King
  Cc: Martin Ågren, Git Mailing List, Ben Peart, Jacob Keller,
	Phillip Wood, Johannes Schindelin, Junio C Hamano

On Fri, May 18, 2018 at 2:33 PM, Jeff King <peff@peff.net> wrote:
> On Fri, May 18, 2018 at 11:23:27PM +0200, Martin Ågren wrote:
>
>> diff --git a/unpack-trees.c b/unpack-trees.c
>> index 79fd97074e..60293ff536 100644
>> --- a/unpack-trees.c
>> +++ b/unpack-trees.c
>> @@ -103,6 +103,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
>>       const char **msgs = opts->msgs;
>>       const char *msg;
>>
>> +     opts->msgs_to_free.strdup_strings = 0;
>> +
>> [...]
>> +void clear_unpack_trees_porcelain(struct unpack_trees_options *opts)
>> +{
>> +     opts->msgs_to_free.strdup_strings = 1;
>> +     string_list_clear(&opts->msgs_to_free, 0);
>
> I like this string_list approach much better, but it's too bad we have
> to go through these contortions with the strdup flag to get the memory
> ownership right.
>
> If we had a string_list_appendf(), then we could just leave that flag
> alone and this:
>
>> @@ -118,8 +120,9 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
>>                     ? _("Your local changes to the following files would be overwritten by %s:\n%%s"
>>                         "Please commit your changes or stash them before you %s.")
>>                     : _("Your local changes to the following files would be overwritten by %s:\n%%s");
>> -     msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOT_UPTODATE_FILE] =
>> -             xstrfmt(msg, cmd, cmd);
>> +     msg = xstrfmt(msg, cmd, cmd);
>> +     msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOT_UPTODATE_FILE] = msg;
>> +     string_list_append(&opts->msgs_to_free, msg);
>
> would become:
>
>   msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOUPTODATE_FILE] =
>         string_list_appendf(&opts->msgs_to_free, msg, cmd, cmd)->string;
>
> I don't know if that's worth it or not (I suspect that there are other
> places where appendf would be handy, but I didn't poke around).

The strdup_strings=1 immediately before calling string_list_clear()
has been used in one other place in merge-recursive.c, and tripped up
the reviewer requiring a big code comment to explain it. (See the very
end of https://public-inbox.org/git/CABPp-BGh7QTTfu3kgH4KO5DrrXiQjtrNhx_uaQsB6fHXT+9hLQ@mail.gmail.com/
).  So there's already one other place in merge-recursive.c that might
benefit from such a change.


A quick search shows about half a dozen other sites throughout the
code that are doing something similar:

$ git grep -3 strdup_strings | grep -B 1 string_list_clear
bisect.c: refs_for_removal.strdup_strings = 1;
bisect.c- string_list_clear(&refs_for_removal, 0);
--
builtin/shortlog.c: onelines->strdup_strings = 1;
builtin/shortlog.c- string_list_clear(onelines, 0);
--
builtin/shortlog.c: log->list.strdup_strings = 1;
builtin/shortlog.c- string_list_clear(&log->list, 1);
--
mailmap.c: me->namemap.strdup_strings = 1;
mailmap.c- string_list_clear_func(&me->namemap, free_mailmap_info);
--
mailmap.c: map->strdup_strings = 1;
mailmap.c- string_list_clear_func(map, free_mailmap_entry);
--
merge-recursive.c: entry->possible_new_dirs.strdup_strings = 1;
merge-recursive.c- string_list_clear(&entry->possible_new_dirs, 1);
--
revision.c: revs->notes_opt.extra_notes_refs.strdup_strings = 1;
revision.c- string_list_clear(&revs->notes_opt.extra_notes_refs, 0);


Maybe someone wants to tackle that as a separate patch series?  (Maybe
we make it a micro-project for future GSoC'ers?)

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 3/3] unpack_trees_options: free messages when done
  2018-05-18 22:30                                             ` Elijah Newren
@ 2018-05-19  1:02                                               ` Jeff King
  2018-05-19  6:13                                                 ` Martin Ågren
  0 siblings, 1 reply; 412+ messages in thread
From: Jeff King @ 2018-05-19  1:02 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Martin Ågren, Git Mailing List, Ben Peart, Jacob Keller,
	Phillip Wood, Johannes Schindelin, Junio C Hamano

On Fri, May 18, 2018 at 03:30:44PM -0700, Elijah Newren wrote:

> > would become:
> >
> >   msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOUPTODATE_FILE] =
> >         string_list_appendf(&opts->msgs_to_free, msg, cmd, cmd)->string;
> >
> > I don't know if that's worth it or not (I suspect that there are other
> > places where appendf would be handy, but I didn't poke around).
> 
> The strdup_strings=1 immediately before calling string_list_clear()
> has been used in one other place in merge-recursive.c, and tripped up
> the reviewer requiring a big code comment to explain it. (See the very
> end of https://public-inbox.org/git/CABPp-BGh7QTTfu3kgH4KO5DrrXiQjtrNhx_uaQsB6fHXT+9hLQ@mail.gmail.com/
> ).  So there's already one other place in merge-recursive.c that might
> benefit from such a change.

Thanks. I knew I had seen such hackery before, but it's nice to have a
specific site that would benefit.

IMHO the "nodup" variant of string_list is quite often a sign that
things are more complicated than they need to be. Even in cases that are
truly pointing to existing strings, is the complication really worth
saving a few strdups? Perhaps sometimes, but I have a suspicion it's
mostly premature optimization.

> Maybe someone wants to tackle that as a separate patch series?  (Maybe
> we make it a micro-project for future GSoC'ers?)

Yeah, I'm fine with these patches if somebody wants to do it separately.
It would be a good micro-project, but I'd also be just as happy if
somebody did it before next year. :)

-Peff

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v3 3/3] unpack_trees_options: free messages when done
  2018-05-19  1:02                                               ` Jeff King
@ 2018-05-19  6:13                                                 ` Martin Ågren
  2018-05-20 10:17                                                   ` [PATCH v4 0/4] " Martin Ågren
  0 siblings, 1 reply; 412+ messages in thread
From: Martin Ågren @ 2018-05-19  6:13 UTC (permalink / raw)
  To: Jeff King
  Cc: Elijah Newren, Git Mailing List, Ben Peart, Jacob Keller,
	Phillip Wood, Johannes Schindelin, Junio C Hamano

On 19 May 2018 at 03:02, Jeff King <peff@peff.net> wrote:
> On Fri, May 18, 2018 at 03:30:44PM -0700, Elijah Newren wrote:
>
>> > would become:
>> >
>> >   msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOUPTODATE_FILE] =
>> >         string_list_appendf(&opts->msgs_to_free, msg, cmd, cmd)->string;
>> >
>> > I don't know if that's worth it or not (I suspect that there are other
>> > places where appendf would be handy, but I didn't poke around).

This does poke at the `string` member, but there is precedent for doing
that. That also feels much closer to the purpose of a string list than
the fiddling with `strdup_strings` that I do in my patch.

I'll look into this over the weekend. Thanks for the suggestion.

>> The strdup_strings=1 immediately before calling string_list_clear()
>> has been used in one other place in merge-recursive.c, and tripped up
>> the reviewer requiring a big code comment to explain it. (See the very
>> end of https://public-inbox.org/git/CABPp-BGh7QTTfu3kgH4KO5DrrXiQjtrNhx_uaQsB6fHXT+9hLQ@mail.gmail.com/
>> ).  So there's already one other place in merge-recursive.c that might
>> benefit from such a change.
>
> Thanks. I knew I had seen such hackery before, but it's nice to have a
> specific site that would benefit.
>
> IMHO the "nodup" variant of string_list is quite often a sign that
> things are more complicated than they need to be. Even in cases that are
> truly pointing to existing strings, is the complication really worth
> saving a few strdups? Perhaps sometimes, but I have a suspicion it's
> mostly premature optimization.
>
>> Maybe someone wants to tackle that as a separate patch series?  (Maybe
>> we make it a micro-project for future GSoC'ers?)
>
> Yeah, I'm fine with these patches if somebody wants to do it separately.
> It would be a good micro-project, but I'd also be just as happy if
> somebody did it before next year. :)

Obviously, I won't be tackling all of that now. I'll just look into
making this final patch better and leave any further cleaning up for
later.

Martin

^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v4 0/4] unpack_trees_options: free messages when done
  2018-05-19  6:13                                                 ` Martin Ågren
@ 2018-05-20 10:17                                                   ` Martin Ågren
  2018-05-20 10:17                                                     ` [PATCH v4 1/4] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
                                                                       ` (3 more replies)
  0 siblings, 4 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-20 10:17 UTC (permalink / raw)
  To: git
  Cc: Jeff King, Elijah Newren, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin, Junio C Hamano

This is v4 of my series for taking care of the memory allocated by
`setup_unpack_trees_porcelain()`. As before, this is based on
bp/merge-rename-config.

On 19 May 2018 at 08:13, Martin Ågren <martin.agren@gmail.com> wrote:
> On 19 May 2018 at 03:02, Jeff King <peff@peff.net> wrote:
>>
>>> > would become:
>>> >
>>> >   msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOUPTODATE_FILE] =
>>> >         string_list_appendf(&opts->msgs_to_free, msg, cmd, cmd)->string;
>>> >
>>> > I don't know if that's worth it or not (I suspect that there are other
>>> > places where appendf would be handy, but I didn't poke around).
>
> I'll look into this over the weekend. Thanks for the suggestion.

The difference to v3 is indeed the new patch 3/4, which introduces
`string_list_appendf()`. I think that makes patch 4/4 clearer and the
resulting code less surprising.

There is an obvious candidate for using this new function in bisect.c,
but I refrained from doing that conversion in this series. While
converting that user to use this new function would be trivial and safe,
such a change might not look entirely sane on its own. The reason is
that the user does the whole `strdup_strings`-dance that I did in v3.

I think it would be much better to do that conversion as a part of a
"let's not play with strdup_strings"-patch. I have one prepared and it
looks quite ok to me. I should be able to be able to collect more
`strdup_string`-cleanups soonish and submit a series later (say, when/if
this here series has matured).

Elijah Newren (1):
  merge-recursive: provide pair of `unpack_trees_{start,finish}()`

Martin Ågren (3):
  merge: setup `opts` later in `checkout_fast_forward()`
  string-list: provide `string_list_appendf()`
  unpack_trees_options: free messages when done

 string-list.h      |  9 +++++++++
 unpack-trees.h     |  6 ++++++
 builtin/checkout.c |  1 +
 merge-recursive.c  | 30 ++++++++++++++++--------------
 merge.c            | 35 ++++++++++++++++++++---------------
 string-list.c      | 13 +++++++++++++
 unpack-trees.c     | 20 +++++++++++++++++---
 7 files changed, 82 insertions(+), 32 deletions(-)

-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v4 1/4] merge: setup `opts` later in `checkout_fast_forward()`
  2018-05-20 10:17                                                   ` [PATCH v4 0/4] " Martin Ågren
@ 2018-05-20 10:17                                                     ` Martin Ågren
  2018-05-20 10:17                                                     ` [PATCH v4 2/4] merge-recursive: provide pair of `unpack_trees_{start,finish}()` Martin Ågren
                                                                       ` (2 subsequent siblings)
  3 siblings, 0 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-20 10:17 UTC (permalink / raw)
  To: git
  Cc: Jeff King, Elijah Newren, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin, Junio C Hamano

After we initialize the various fields in `opts` but before we actually
use them, we might return early. Move the initialization further down,
to immediately before we use `opts`.

This limits the scope of `opts` and will help a later commit fix a
memory leak without having to worry about those early returns.

This patch is best viewed using something like this (note the tab!):
--color-moved --anchored="	trees[nr_trees] = parse_tree_indirect"

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 merge.c | 34 ++++++++++++++++++----------------
 1 file changed, 18 insertions(+), 16 deletions(-)

diff --git a/merge.c b/merge.c
index f06a4773d4..f123658e58 100644
--- a/merge.c
+++ b/merge.c
@@ -94,23 +94,7 @@ int checkout_fast_forward(const struct object_id *head,
 		return -1;
 
 	memset(&trees, 0, sizeof(trees));
-	memset(&opts, 0, sizeof(opts));
 	memset(&t, 0, sizeof(t));
-	if (overwrite_ignore) {
-		memset(&dir, 0, sizeof(dir));
-		dir.flags |= DIR_SHOW_IGNORED;
-		setup_standard_excludes(&dir);
-		opts.dir = &dir;
-	}
-
-	opts.head_idx = 1;
-	opts.src_index = &the_index;
-	opts.dst_index = &the_index;
-	opts.update = 1;
-	opts.verbose_update = 1;
-	opts.merge = 1;
-	opts.fn = twoway_merge;
-	setup_unpack_trees_porcelain(&opts, "merge");
 
 	trees[nr_trees] = parse_tree_indirect(head);
 	if (!trees[nr_trees++]) {
@@ -126,6 +110,24 @@ int checkout_fast_forward(const struct object_id *head,
 		parse_tree(trees[i]);
 		init_tree_desc(t+i, trees[i]->buffer, trees[i]->size);
 	}
+
+	memset(&opts, 0, sizeof(opts));
+	if (overwrite_ignore) {
+		memset(&dir, 0, sizeof(dir));
+		dir.flags |= DIR_SHOW_IGNORED;
+		setup_standard_excludes(&dir);
+		opts.dir = &dir;
+	}
+
+	opts.head_idx = 1;
+	opts.src_index = &the_index;
+	opts.dst_index = &the_index;
+	opts.update = 1;
+	opts.verbose_update = 1;
+	opts.merge = 1;
+	opts.fn = twoway_merge;
+	setup_unpack_trees_porcelain(&opts, "merge");
+
 	if (unpack_trees(nr_trees, t, &opts)) {
 		rollback_lock_file(&lock_file);
 		return -1;
-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 2/4] merge-recursive: provide pair of `unpack_trees_{start,finish}()`
  2018-05-20 10:17                                                   ` [PATCH v4 0/4] " Martin Ågren
  2018-05-20 10:17                                                     ` [PATCH v4 1/4] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
@ 2018-05-20 10:17                                                     ` Martin Ågren
  2018-05-20 10:17                                                     ` [PATCH v4 3/4] string-list: provide `string_list_appendf()` Martin Ågren
  2018-05-20 10:17                                                     ` [PATCH v4 4/4] unpack_trees_options: free messages when done Martin Ågren
  3 siblings, 0 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-20 10:17 UTC (permalink / raw)
  To: git
  Cc: Jeff King, Elijah Newren, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin, Junio C Hamano

From: Elijah Newren <newren@gmail.com>

Rename `git_merge_trees()` to `unpack_trees_start()` and extract the
call to `discard_index()` into a new function `unpack_trees_finish()`.
As a result, these are called early resp. late in `merge_trees()`,
making the resource handling clearer. A later commit will expand on
that, teaching `..._finish()` to free more memory. (So rather than
moving the FIXME-comment, just drop it, since it will be addressed soon
enough.)

Also call `..._finish()` when `merge_trees()` returns early.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
---
 merge-recursive.c | 29 +++++++++++++++--------------
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 680e01226b..ddb0fa7369 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -337,10 +337,10 @@ static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
 	init_tree_desc(desc, tree->buffer, tree->size);
 }
 
-static int git_merge_trees(struct merge_options *o,
-			   struct tree *common,
-			   struct tree *head,
-			   struct tree *merge)
+static int unpack_trees_start(struct merge_options *o,
+			      struct tree *common,
+			      struct tree *head,
+			      struct tree *merge)
 {
 	int rc;
 	struct tree_desc t[3];
@@ -379,6 +379,11 @@ static int git_merge_trees(struct merge_options *o,
 	return rc;
 }
 
+static void unpack_trees_finish(struct merge_options *o)
+{
+	discard_index(&o->orig_index);
+}
+
 struct tree *write_tree_from_memory(struct merge_options *o)
 {
 	struct tree *result = NULL;
@@ -3088,13 +3093,14 @@ int merge_trees(struct merge_options *o,
 		return 1;
 	}
 
-	code = git_merge_trees(o, common, head, merge);
+	code = unpack_trees_start(o, common, head, merge);
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
 			err(o, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
+		unpack_trees_finish(o);
 		return -1;
 	}
 
@@ -3147,20 +3153,15 @@ int merge_trees(struct merge_options *o,
 
 		hashmap_free(&o->current_file_dir_set, 1);
 
-		if (clean < 0)
+		if (clean < 0) {
+			unpack_trees_finish(o);
 			return clean;
+		}
 	}
 	else
 		clean = 1;
 
-	/* Free the extra index left from git_merge_trees() */
-	/*
-	 * FIXME: Need to also free data allocated by
-	 * setup_unpack_trees_porcelain() tucked away in o->unpack_opts.msgs,
-	 * but the problem is that only half of it refers to dynamically
-	 * allocated data, while the other half points at static strings.
-	 */
-	discard_index(&o->orig_index);
+	unpack_trees_finish(o);
 
 	if (o->call_depth && !(*result = write_tree_from_memory(o)))
 		return -1;
-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 3/4] string-list: provide `string_list_appendf()`
  2018-05-20 10:17                                                   ` [PATCH v4 0/4] " Martin Ågren
  2018-05-20 10:17                                                     ` [PATCH v4 1/4] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
  2018-05-20 10:17                                                     ` [PATCH v4 2/4] merge-recursive: provide pair of `unpack_trees_{start,finish}()` Martin Ågren
@ 2018-05-20 10:17                                                     ` Martin Ågren
  2018-05-20 19:12                                                       ` Jacob Keller
  2018-05-20 10:17                                                     ` [PATCH v4 4/4] unpack_trees_options: free messages when done Martin Ågren
  3 siblings, 1 reply; 412+ messages in thread
From: Martin Ågren @ 2018-05-20 10:17 UTC (permalink / raw)
  To: git
  Cc: Jeff King, Elijah Newren, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin, Junio C Hamano

Add a function `string_list_appendf(list, fmt, ...)` to the string-list
API. The next commit will add a user.

This function naturally ignores the `strdup_strings`-setting and always
appends a freshly allocated string. Thus, using this function with
`strdup_strings = 0` risks making ownership unclear and leaking memory.
With `strdup_strings = 1` on the other hand, we can easily add formatted
strings without going through `string_list_append_nodup()` or playing
with `strdup_strings`.

Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
---
 string-list.h |  9 +++++++++
 string-list.c | 13 +++++++++++++
 2 files changed, 22 insertions(+)

diff --git a/string-list.h b/string-list.h
index ff8f6094a3..3a73b86ffa 100644
--- a/string-list.h
+++ b/string-list.h
@@ -208,6 +208,15 @@ void string_list_remove_duplicates(struct string_list *sorted_list, int free_uti
  */
 struct string_list_item *string_list_append(struct string_list *list, const char *string);
 
+/**
+ * Add formatted string to the end of `list`. This function ignores
+ * the value of `list->strdup_strings` and always appends a freshly
+ * allocated string, so you will probably not want to use it with
+ * `strdup_strings = 0`.
+ */
+struct string_list_item *string_list_appendf(struct string_list *list,
+					     const char *fmt, ...);
+
 /**
  * Like string_list_append(), except string is never copied.  When
  * list->strdup_strings is set, this function can be used to hand
diff --git a/string-list.c b/string-list.c
index a0cf0cfe88..b54d31c1cf 100644
--- a/string-list.c
+++ b/string-list.c
@@ -224,6 +224,19 @@ struct string_list_item *string_list_append(struct string_list *list,
 			list->strdup_strings ? xstrdup(string) : (char *)string);
 }
 
+struct string_list_item *string_list_appendf(struct string_list *list,
+					     const char *fmt, ...)
+{
+	struct string_list_item *retval;
+	va_list ap;
+
+	va_start(ap, fmt);
+	retval = string_list_append_nodup(list, xstrvfmt(fmt, ap));
+	va_end(ap);
+
+	return retval;
+}
+
 static int cmp_items(const void *a, const void *b, void *ctx)
 {
 	compare_strings_fn cmp = ctx;
-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v4 4/4] unpack_trees_options: free messages when done
  2018-05-20 10:17                                                   ` [PATCH v4 0/4] " Martin Ågren
                                                                       ` (2 preceding siblings ...)
  2018-05-20 10:17                                                     ` [PATCH v4 3/4] string-list: provide `string_list_appendf()` Martin Ågren
@ 2018-05-20 10:17                                                     ` Martin Ågren
  3 siblings, 0 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-20 10:17 UTC (permalink / raw)
  To: git
  Cc: Jeff King, Elijah Newren, Ben Peart, Jacob Keller, Phillip Wood,
	Johannes Schindelin, Junio C Hamano

The strings allocated in `setup_unpack_trees_porcelain()` are never
freed. Provide a function `clear_unpack_trees_porcelain()` to do so and
call it where we use `setup_unpack_trees_porcelain()`. The only
non-trivial user is `unpack_trees_start()`, where we should place the
new call in `unpack_trees_finish()`.

We keep the string pointers in an array, mixing pointers to static
memory and memory that we allocate on the heap. We also keep several
copies of the individual pointers. So we need to make sure that we do
not free what we must not free and that we do not double-free. Keep the
unique, heap-allocated pointers in a separate string list, to make the
freeing safe and future-proof.

Zero the whole array of string pointers to make sure that we do not
leave any dangling pointers.

Note that we only take responsibility for the memory allocated in
`setup_unpack_trees_porcelain()` and not any other members of the
`struct unpack_trees_options`.

Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
---
 unpack-trees.h     |  6 ++++++
 builtin/checkout.c |  1 +
 merge-recursive.c  |  1 +
 merge.c            |  3 +++
 unpack-trees.c     | 20 +++++++++++++++++---
 5 files changed, 28 insertions(+), 3 deletions(-)

diff --git a/unpack-trees.h b/unpack-trees.h
index 41178ada94..5a84123a40 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -33,6 +33,11 @@ enum unpack_trees_error_types {
 void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 				  const char *cmd);
 
+/*
+ * Frees resources allocated by setup_unpack_trees_porcelain().
+ */
+void clear_unpack_trees_porcelain(struct unpack_trees_options *opts);
+
 struct unpack_trees_options {
 	unsigned int reset,
 		     merge,
@@ -57,6 +62,7 @@ struct unpack_trees_options {
 	struct pathspec *pathspec;
 	merge_fn_t fn;
 	const char *msgs[NB_UNPACK_TREES_ERROR_TYPES];
+	struct string_list msgs_to_free;
 	/*
 	 * Store error messages in an array, each case
 	 * corresponding to a error message type
diff --git a/builtin/checkout.c b/builtin/checkout.c
index b49b582071..5cebe170fc 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -526,6 +526,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 		init_tree_desc(&trees[1], tree->buffer, tree->size);
 
 		ret = unpack_trees(2, trees, &topts);
+		clear_unpack_trees_porcelain(&topts);
 		if (ret == -1) {
 			/*
 			 * Unpack couldn't do a trivial merge; either
diff --git a/merge-recursive.c b/merge-recursive.c
index ddb0fa7369..338f63a952 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -382,6 +382,7 @@ static int unpack_trees_start(struct merge_options *o,
 static void unpack_trees_finish(struct merge_options *o)
 {
 	discard_index(&o->orig_index);
+	clear_unpack_trees_porcelain(&o->unpack_opts);
 }
 
 struct tree *write_tree_from_memory(struct merge_options *o)
diff --git a/merge.c b/merge.c
index f123658e58..b433291d0c 100644
--- a/merge.c
+++ b/merge.c
@@ -130,8 +130,11 @@ int checkout_fast_forward(const struct object_id *head,
 
 	if (unpack_trees(nr_trees, t, &opts)) {
 		rollback_lock_file(&lock_file);
+		clear_unpack_trees_porcelain(&opts);
 		return -1;
 	}
+	clear_unpack_trees_porcelain(&opts);
+
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		return error(_("unable to write new index file"));
 	return 0;
diff --git a/unpack-trees.c b/unpack-trees.c
index 79fd97074e..86046b987a 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -103,6 +103,12 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 	const char **msgs = opts->msgs;
 	const char *msg;
 
+	/*
+	 * As we add strings using `...appendf()`, this does not matter,
+	 * but when we clear the string list, we want them to be freed.
+	 */
+	opts->msgs_to_free.strdup_strings = 1;
+
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_commit_before_merge
 		      ? _("Your local changes to the following files would be overwritten by checkout:\n%%s"
@@ -119,7 +125,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 			  "Please commit your changes or stash them before you %s.")
 		      : _("Your local changes to the following files would be overwritten by %s:\n%%s");
 	msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOT_UPTODATE_FILE] =
-		xstrfmt(msg, cmd, cmd);
+		string_list_appendf(&opts->msgs_to_free, msg, cmd, cmd)->string;
 
 	msgs[ERROR_NOT_UPTODATE_DIR] =
 		_("Updating the following directories would lose untracked files in them:\n%s");
@@ -139,7 +145,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 		      ? _("The following untracked working tree files would be removed by %s:\n%%s"
 			  "Please move or remove them before you %s.")
 		      : _("The following untracked working tree files would be removed by %s:\n%%s");
-	msgs[ERROR_WOULD_LOSE_UNTRACKED_REMOVED] = xstrfmt(msg, cmd, cmd);
+	msgs[ERROR_WOULD_LOSE_UNTRACKED_REMOVED] =
+		string_list_appendf(&opts->msgs_to_free, msg, cmd, cmd)->string;
 
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_commit_before_merge
@@ -156,7 +163,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 		      ? _("The following untracked working tree files would be overwritten by %s:\n%%s"
 			  "Please move or remove them before you %s.")
 		      : _("The following untracked working tree files would be overwritten by %s:\n%%s");
-	msgs[ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN] = xstrfmt(msg, cmd, cmd);
+	msgs[ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN] =
+		string_list_appendf(&opts->msgs_to_free, msg, cmd, cmd)->string;
 
 	/*
 	 * Special case: ERROR_BIND_OVERLAP refers to a pair of paths, we
@@ -179,6 +187,12 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 		opts->unpack_rejects[i].strdup_strings = 1;
 }
 
+void clear_unpack_trees_porcelain(struct unpack_trees_options *opts)
+{
+	string_list_clear(&opts->msgs_to_free, 0);
+	memset(opts->msgs, 0, sizeof(opts->msgs));
+}
+
 static int do_add_entry(struct unpack_trees_options *o, struct cache_entry *ce,
 			 unsigned int set, unsigned int clear)
 {
-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v4 3/4] string-list: provide `string_list_appendf()`
  2018-05-20 10:17                                                     ` [PATCH v4 3/4] string-list: provide `string_list_appendf()` Martin Ågren
@ 2018-05-20 19:12                                                       ` Jacob Keller
  2018-05-21  0:01                                                         ` Re*: " Junio C Hamano
  0 siblings, 1 reply; 412+ messages in thread
From: Jacob Keller @ 2018-05-20 19:12 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Git mailing list, Jeff King, Elijah Newren, Ben Peart,
	Phillip Wood, Johannes Schindelin, Junio C Hamano

On Sun, May 20, 2018 at 3:17 AM, Martin Ågren <martin.agren@gmail.com> wrote:
> +/**
> + * Add formatted string to the end of `list`. This function ignores
> + * the value of `list->strdup_strings` and always appends a freshly
> + * allocated string, so you will probably not want to use it with
> + * `strdup_strings = 0`.
> + */
> +struct string_list_item *string_list_appendf(struct string_list *list,
> +                                            const char *fmt, ...);
> +

Would it make sense to verify that strdup_strings == 0? I guess we'd
have to use die or BUG(), but that would mean that the program could
crash..

I doubt this could be verified at compilation time....

Thanks,
Jake

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re*: [PATCH v4 3/4] string-list: provide `string_list_appendf()`
  2018-05-20 19:12                                                       ` Jacob Keller
@ 2018-05-21  0:01                                                         ` Junio C Hamano
  2018-05-21  0:25                                                           ` Junio C Hamano
  2018-05-21  2:38                                                           ` Re*: [PATCH v4 3/4] string-list: provide `string_list_appendf()` Jeff King
  0 siblings, 2 replies; 412+ messages in thread
From: Junio C Hamano @ 2018-05-21  0:01 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Martin Ågren, Git mailing list, Jeff King, Elijah Newren,
	Ben Peart, Phillip Wood, Johannes Schindelin

Jacob Keller <jacob.keller@gmail.com> writes:

> On Sun, May 20, 2018 at 3:17 AM, Martin Ågren <martin.agren@gmail.com> wrote:
>> +/**
>> + * Add formatted string to the end of `list`. This function ignores
>> + * the value of `list->strdup_strings` and always appends a freshly
>> + * allocated string, so you will probably not want to use it with
>> + * `strdup_strings = 0`.
>> + */
>> +struct string_list_item *string_list_appendf(struct string_list *list,
>> +                                            const char *fmt, ...);
>> +
>
> Would it make sense to verify that strdup_strings == 0? I guess we'd
> have to use die or BUG(), but that would mean that the program could
> crash..

It probably is clear to readers that any reasonable implementation
of *_appendf() will create a new and unique string, as the point of
*f() is to give a customized instantiation of fmt string for given
parameters.  So it would be natural to expect that the storage that
holds the generated string will belong to the list.  We _could_ make
it honor strdup_strings and make one extra copy when strdup_strings
is set to true, but the only effect such a stupid implementation has
is to unnecessarily leak ;-)

I think it is probably OK to check and BUG() when strdup_strings==0,
but such a check means that we now declare that a string list must
either borrow all of its strings from elsewhere or own all of its
strings itself, and mixture is not allowed.

The (overly) flexible string_list API could be used to mix both
borrowed and owned strings (an obvious strategy to do this without
leaking and crashing is to use the .util field to mark which ones
are owned and which ones are borrowed), so there might already be
current users of the API that violates that rule.

I have a feeling that argv_array might be a better fit for the
purpose of keeping track of to_free[] strings in the context of this
series.  Moving away from string_list would allow us to sidestep the
storage ownership issues the API has, and we do not need the .util
thing string_list gives us (which is one distinct advantage string_list
has over argv_array, if the application needs that feature).

We would need to make _pushf() and friends return "const char *" if
we go that route to make the resulting API more useful, though.

-- >8 --
Subject: argv-array: return the pushed string from argv_push*()

Such an API change allows us to use an argv_array this way:

	struct argv_array to_free = ARGV_ARRAY_INIT;
        const char *msg;

        if (some condition) {
		msg = "constant string message";
		... other logic ...
	} else {
		msg = argv_pushf(&to_free, "format %s", var);
	}
	... use "msg" ...
	... do other things ...
	argv_clear(&to_free);

Note that argv_array_pushl() and argv_array_pushv() are used to push
one or more strings with a single call, so we do not return any one
of these strings from these two functions in order to reduce the
chance to misuse the API.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 argv-array.c | 6 ++++--
 argv-array.h | 4 ++--
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/argv-array.c b/argv-array.c
index 5d370fa336..449dfc105a 100644
--- a/argv-array.c
+++ b/argv-array.c
@@ -21,12 +21,13 @@ static void argv_array_push_nodup(struct argv_array *array, const char *value)
 	array->argv[array->argc] = NULL;
 }
 
-void argv_array_push(struct argv_array *array, const char *value)
+const char *argv_array_push(struct argv_array *array, const char *value)
 {
 	argv_array_push_nodup(array, xstrdup(value));
+	return array->argv[array->argc - 1];
 }
 
-void argv_array_pushf(struct argv_array *array, const char *fmt, ...)
+const char *argv_array_pushf(struct argv_array *array, const char *fmt, ...)
 {
 	va_list ap;
 	struct strbuf v = STRBUF_INIT;
@@ -36,6 +37,7 @@ void argv_array_pushf(struct argv_array *array, const char *fmt, ...)
 	va_end(ap);
 
 	argv_array_push_nodup(array, strbuf_detach(&v, NULL));
+	return array->argv[array->argc - 1];
 }
 
 void argv_array_pushl(struct argv_array *array, ...)
diff --git a/argv-array.h b/argv-array.h
index 29056e49a1..715c93b246 100644
--- a/argv-array.h
+++ b/argv-array.h
@@ -12,9 +12,9 @@ struct argv_array {
 #define ARGV_ARRAY_INIT { empty_argv, 0, 0 }
 
 void argv_array_init(struct argv_array *);
-void argv_array_push(struct argv_array *, const char *);
+const char *argv_array_push(struct argv_array *, const char *);
 __attribute__((format (printf,2,3)))
-void argv_array_pushf(struct argv_array *, const char *fmt, ...);
+const char *argv_array_pushf(struct argv_array *, const char *fmt, ...);
 LAST_ARG_MUST_BE_NULL
 void argv_array_pushl(struct argv_array *, ...);
 void argv_array_pushv(struct argv_array *, const char **);

^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: Re*: [PATCH v4 3/4] string-list: provide `string_list_appendf()`
  2018-05-21  0:01                                                         ` Re*: " Junio C Hamano
@ 2018-05-21  0:25                                                           ` Junio C Hamano
  2018-05-21  2:39                                                             ` Jeff King
  2018-05-21 14:54                                                             ` [PATCH v5 0/4] unpack_trees_options: free messages when done Martin Ågren
  2018-05-21  2:38                                                           ` Re*: [PATCH v4 3/4] string-list: provide `string_list_appendf()` Jeff King
  1 sibling, 2 replies; 412+ messages in thread
From: Junio C Hamano @ 2018-05-21  0:25 UTC (permalink / raw)
  To: Jacob Keller
  Cc: Martin Ågren, Git mailing list, Jeff King, Elijah Newren,
	Ben Peart, Phillip Wood, Johannes Schindelin

Junio C Hamano <gitster@pobox.com> writes:

> I have a feeling that argv_array might be a better fit for the
> purpose of keeping track of to_free[] strings in the context of this
> series.  Moving away from string_list would allow us to sidestep the
> storage ownership issues the API has, and we do not need the .util
> thing string_list gives us (which is one distinct advantage string_list
> has over argv_array, if the application needs that feature).
>
> We would need to make _pushf() and friends return "const char *" if
> we go that route to make the resulting API more useful, though.

... and redoing the 4/4 patch using argv_array_pushf() makes the
result look like this, which does not look too bad.

-- >8 --
From: Junio C Hamano <gitster@pobox.com>
Subject: [PATCH] unpack_trees_options: keep track of owned messages with argv_array

Instead of the string_list API, which is overly flexible and require
callers to be careful about memory ownership issues, use the
argv_array API that always takes ownership to redo the earlier
commit.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 unpack-trees.c | 16 ++++++----------
 unpack-trees.h |  4 ++--
 2 files changed, 8 insertions(+), 12 deletions(-)

diff --git a/unpack-trees.c b/unpack-trees.c
index 86046b987a..b28f0c6e9d 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1,5 +1,6 @@
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "argv-array.h"
 #include "repository.h"
 #include "config.h"
 #include "dir.h"
@@ -103,11 +104,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 	const char **msgs = opts->msgs;
 	const char *msg;
 
-	/*
-	 * As we add strings using `...appendf()`, this does not matter,
-	 * but when we clear the string list, we want them to be freed.
-	 */
-	opts->msgs_to_free.strdup_strings = 1;
+	argv_array_init(&opts->msgs_to_free);
 
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_commit_before_merge
@@ -125,7 +122,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 			  "Please commit your changes or stash them before you %s.")
 		      : _("Your local changes to the following files would be overwritten by %s:\n%%s");
 	msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOT_UPTODATE_FILE] =
-		string_list_appendf(&opts->msgs_to_free, msg, cmd, cmd)->string;
+		argv_array_pushf(&opts->msgs_to_free, msg, cmd, cmd);
 
 	msgs[ERROR_NOT_UPTODATE_DIR] =
 		_("Updating the following directories would lose untracked files in them:\n%s");
@@ -146,7 +143,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 			  "Please move or remove them before you %s.")
 		      : _("The following untracked working tree files would be removed by %s:\n%%s");
 	msgs[ERROR_WOULD_LOSE_UNTRACKED_REMOVED] =
-		string_list_appendf(&opts->msgs_to_free, msg, cmd, cmd)->string;
+		argv_array_pushf(&opts->msgs_to_free, msg, cmd, cmd);
 
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_commit_before_merge
@@ -164,7 +161,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 			  "Please move or remove them before you %s.")
 		      : _("The following untracked working tree files would be overwritten by %s:\n%%s");
 	msgs[ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN] =
-		string_list_appendf(&opts->msgs_to_free, msg, cmd, cmd)->string;
+		argv_array_pushf(&opts->msgs_to_free, msg, cmd, cmd);
 
 	/*
 	 * Special case: ERROR_BIND_OVERLAP refers to a pair of paths, we
@@ -189,8 +186,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 
 void clear_unpack_trees_porcelain(struct unpack_trees_options *opts)
 {
-	string_list_clear(&opts->msgs_to_free, 0);
-	memset(opts->msgs, 0, sizeof(opts->msgs));
+	argv_array_clear(&opts->msgs_to_free);
 }
 
 static int do_add_entry(struct unpack_trees_options *o, struct cache_entry *ce,
diff --git a/unpack-trees.h b/unpack-trees.h
index 5a84123a40..c2b434c606 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -2,7 +2,7 @@
 #define UNPACK_TREES_H
 
 #include "tree-walk.h"
-#include "string-list.h"
+#include "argv-array.h"
 
 #define MAX_UNPACK_TREES 8
 
@@ -62,7 +62,7 @@ struct unpack_trees_options {
 	struct pathspec *pathspec;
 	merge_fn_t fn;
 	const char *msgs[NB_UNPACK_TREES_ERROR_TYPES];
-	struct string_list msgs_to_free;
+	struct argv_array msgs_to_free;
 	/*
 	 * Store error messages in an array, each case
 	 * corresponding to a error message type
-- 
2.17.0-582-gccdcbd54c4


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: Re*: [PATCH v4 3/4] string-list: provide `string_list_appendf()`
  2018-05-21  0:01                                                         ` Re*: " Junio C Hamano
  2018-05-21  0:25                                                           ` Junio C Hamano
@ 2018-05-21  2:38                                                           ` Jeff King
  1 sibling, 0 replies; 412+ messages in thread
From: Jeff King @ 2018-05-21  2:38 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Jacob Keller, Martin Ågren, Git mailing list, Elijah Newren,
	Ben Peart, Phillip Wood, Johannes Schindelin

On Mon, May 21, 2018 at 09:01:05AM +0900, Junio C Hamano wrote:

> Jacob Keller <jacob.keller@gmail.com> writes:
> 
> > On Sun, May 20, 2018 at 3:17 AM, Martin Ågren <martin.agren@gmail.com> wrote:
> >> +/**
> >> + * Add formatted string to the end of `list`. This function ignores
> >> + * the value of `list->strdup_strings` and always appends a freshly
> >> + * allocated string, so you will probably not want to use it with
> >> + * `strdup_strings = 0`.
> >> + */
> >> +struct string_list_item *string_list_appendf(struct string_list *list,
> >> +                                            const char *fmt, ...);
> >> +
> >
> > Would it make sense to verify that strdup_strings == 0? I guess we'd
> > have to use die or BUG(), but that would mean that the program could
> > crash..
> 
> It probably is clear to readers that any reasonable implementation
> of *_appendf() will create a new and unique string, as the point of
> *f() is to give a customized instantiation of fmt string for given
> parameters.  So it would be natural to expect that the storage that
> holds the generated string will belong to the list.  We _could_ make
> it honor strdup_strings and make one extra copy when strdup_strings
> is set to true, but the only effect such a stupid implementation has
> is to unnecessarily leak ;-)
> 
> I think it is probably OK to check and BUG() when strdup_strings==0,
> but such a check means that we now declare that a string list must
> either borrow all of its strings from elsewhere or own all of its
> strings itself, and mixture is not allowed.
>
> The (overly) flexible string_list API could be used to mix both
> borrowed and owned strings (an obvious strategy to do this without
> leaking and crashing is to use the .util field to mark which ones
> are owned and which ones are borrowed), so there might already be
> current users of the API that violates that rule.

IMHO such a mixed use is mildly crazy. At any rate, we would know that
anybody using appendf() would not have this problem, since we are just
introducing it now.

> I have a feeling that argv_array might be a better fit for the
> purpose of keeping track of to_free[] strings in the context of this
> series.  Moving away from string_list would allow us to sidestep the
> storage ownership issues the API has, and we do not need the .util
> thing string_list gives us (which is one distinct advantage string_list
> has over argv_array, if the application needs that feature).

I do agree that argv_array is generally a better fit for most cases.
Didn't we want to rename it to strarray or something? That's probably
too much yak-shaving for this series, though. :)

> We would need to make _pushf() and friends return "const char *" if
> we go that route to make the resulting API more useful, though.

This is the first time I think that's been suggested, but I agree it's
the only sensible thing for the functions to return.

> -- >8 --
> Subject: argv-array: return the pushed string from argv_push*()
> 
> Such an API change allows us to use an argv_array this way:
> 
> 	struct argv_array to_free = ARGV_ARRAY_INIT;
>         const char *msg;
> 
>         if (some condition) {
> 		msg = "constant string message";
> 		... other logic ...
> 	} else {
> 		msg = argv_pushf(&to_free, "format %s", var);
> 	}
> 	... use "msg" ...
> 	... do other things ...
> 	argv_clear(&to_free);
> 
> Note that argv_array_pushl() and argv_array_pushv() are used to push
> one or more strings with a single call, so we do not return any one
> of these strings from these two functions in order to reduce the
> chance to misuse the API.
> 
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---

Yup, this looks good to me.

-Peff

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: Re*: [PATCH v4 3/4] string-list: provide `string_list_appendf()`
  2018-05-21  0:25                                                           ` Junio C Hamano
@ 2018-05-21  2:39                                                             ` Jeff King
  2018-05-21 14:54                                                             ` [PATCH v5 0/4] unpack_trees_options: free messages when done Martin Ågren
  1 sibling, 0 replies; 412+ messages in thread
From: Jeff King @ 2018-05-21  2:39 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Jacob Keller, Martin Ågren, Git mailing list, Elijah Newren,
	Ben Peart, Phillip Wood, Johannes Schindelin

On Mon, May 21, 2018 at 09:25:01AM +0900, Junio C Hamano wrote:

> Junio C Hamano <gitster@pobox.com> writes:
> 
> > I have a feeling that argv_array might be a better fit for the
> > purpose of keeping track of to_free[] strings in the context of this
> > series.  Moving away from string_list would allow us to sidestep the
> > storage ownership issues the API has, and we do not need the .util
> > thing string_list gives us (which is one distinct advantage string_list
> > has over argv_array, if the application needs that feature).
> >
> > We would need to make _pushf() and friends return "const char *" if
> > we go that route to make the resulting API more useful, though.
> 
> ... and redoing the 4/4 patch using argv_array_pushf() makes the
> result look like this, which does not look too bad.

Agreed.

-Peff

^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v5 0/4] unpack_trees_options: free messages when done
  2018-05-21  0:25                                                           ` Junio C Hamano
  2018-05-21  2:39                                                             ` Jeff King
@ 2018-05-21 14:54                                                             ` Martin Ågren
  2018-05-21 14:54                                                               ` [PATCH v5 1/4] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
                                                                                 ` (5 more replies)
  1 sibling, 6 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-21 14:54 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jeff King, Jacob Keller, Elijah Newren, Ben Peart,
	Phillip Wood, Johannes Schindelin

On 21 May 2018 at 02:25, Junio C Hamano <gitster@pobox.com> wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
>> I have a feeling that argv_array might be a better fit for the
>> purpose of keeping track of to_free[] strings in the context of this
>> series.  Moving away from string_list would allow us to sidestep the
>> storage ownership issues the API has, and we do not need the .util
>> thing string_list gives us (which is one distinct advantage string_list
>> has over argv_array, if the application needs that feature).
>>
>> We would need to make _pushf() and friends return "const char *" if
>> we go that route to make the resulting API more useful, though.
>
> ... and redoing the 4/4 patch using argv_array_pushf() makes the
> result look like this, which does not look too bad.

Thanks to Jacob, Junio and Peff for comments on the previous iteration.

I've taken the six patches that Junio has queued and rebuilt the series
to get rid of the new and possibly bug-prone function that no-one uses
once the series is over.

That is, I've replaced the `string_list_appendf()`-patch with Junio's
`argv_push*()`-patch, then squashed Junio's "redoing the 4/4"-patch into
patch 4/4 -- with the exception of keeping the `memset(opts->msgs, ...)`
which I suspect was mistakenly dropped.

Again, thanks for all the helpful comments and patches pointing me in
the right direction.

Martin

Elijah Newren (1):
  merge-recursive: provide pair of `unpack_trees_{start,finish}()`

Junio C Hamano (1):
  argv-array: return the pushed string from argv_push*()

Martin Ågren (2):
  merge: setup `opts` later in `checkout_fast_forward()`
  unpack_trees_options: free messages when done

 argv-array.h       |  4 ++--
 unpack-trees.h     |  8 +++++++-
 argv-array.c       |  6 ++++--
 builtin/checkout.c |  1 +
 merge-recursive.c  | 30 ++++++++++++++++--------------
 merge.c            | 35 ++++++++++++++++++++---------------
 unpack-trees.c     | 17 ++++++++++++++---
 7 files changed, 64 insertions(+), 37 deletions(-)

-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply	[flat|nested] 412+ messages in thread

* [PATCH v5 1/4] merge: setup `opts` later in `checkout_fast_forward()`
  2018-05-21 14:54                                                             ` [PATCH v5 0/4] unpack_trees_options: free messages when done Martin Ågren
@ 2018-05-21 14:54                                                               ` Martin Ågren
  2018-05-21 14:54                                                               ` [PATCH v5 2/4] merge-recursive: provide pair of `unpack_trees_{start,finish}()` Martin Ågren
                                                                                 ` (4 subsequent siblings)
  5 siblings, 0 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-21 14:54 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jeff King, Jacob Keller, Elijah Newren, Ben Peart,
	Phillip Wood, Johannes Schindelin

After we initialize the various fields in `opts` but before we actually
use them, we might return early. Move the initialization further down,
to immediately before we use `opts`.

This limits the scope of `opts` and will help a later commit fix a
memory leak without having to worry about those early returns.

This patch is best viewed using something like this (note the tab!):
--color-moved --anchored="	trees[nr_trees] = parse_tree_indirect"

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 merge.c | 32 +++++++++++++++++---------------
 1 file changed, 17 insertions(+), 15 deletions(-)

diff --git a/merge.c b/merge.c
index f06a4773d4..f123658e58 100644
--- a/merge.c
+++ b/merge.c
@@ -94,8 +94,24 @@ int checkout_fast_forward(const struct object_id *head,
 		return -1;
 
 	memset(&trees, 0, sizeof(trees));
-	memset(&opts, 0, sizeof(opts));
 	memset(&t, 0, sizeof(t));
+
+	trees[nr_trees] = parse_tree_indirect(head);
+	if (!trees[nr_trees++]) {
+		rollback_lock_file(&lock_file);
+		return -1;
+	}
+	trees[nr_trees] = parse_tree_indirect(remote);
+	if (!trees[nr_trees++]) {
+		rollback_lock_file(&lock_file);
+		return -1;
+	}
+	for (i = 0; i < nr_trees; i++) {
+		parse_tree(trees[i]);
+		init_tree_desc(t+i, trees[i]->buffer, trees[i]->size);
+	}
+
+	memset(&opts, 0, sizeof(opts));
 	if (overwrite_ignore) {
 		memset(&dir, 0, sizeof(dir));
 		dir.flags |= DIR_SHOW_IGNORED;
@@ -112,20 +128,6 @@ int checkout_fast_forward(const struct object_id *head,
 	opts.fn = twoway_merge;
 	setup_unpack_trees_porcelain(&opts, "merge");
 
-	trees[nr_trees] = parse_tree_indirect(head);
-	if (!trees[nr_trees++]) {
-		rollback_lock_file(&lock_file);
-		return -1;
-	}
-	trees[nr_trees] = parse_tree_indirect(remote);
-	if (!trees[nr_trees++]) {
-		rollback_lock_file(&lock_file);
-		return -1;
-	}
-	for (i = 0; i < nr_trees; i++) {
-		parse_tree(trees[i]);
-		init_tree_desc(t+i, trees[i]->buffer, trees[i]->size);
-	}
 	if (unpack_trees(nr_trees, t, &opts)) {
 		rollback_lock_file(&lock_file);
 		return -1;
-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 2/4] merge-recursive: provide pair of `unpack_trees_{start,finish}()`
  2018-05-21 14:54                                                             ` [PATCH v5 0/4] unpack_trees_options: free messages when done Martin Ågren
  2018-05-21 14:54                                                               ` [PATCH v5 1/4] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
@ 2018-05-21 14:54                                                               ` Martin Ågren
  2018-05-21 14:54                                                               ` [PATCH v5 3/4] argv-array: return the pushed string from argv_push*() Martin Ågren
                                                                                 ` (3 subsequent siblings)
  5 siblings, 0 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-21 14:54 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jeff King, Jacob Keller, Elijah Newren, Ben Peart,
	Phillip Wood, Johannes Schindelin

From: Elijah Newren <newren@gmail.com>

Rename `git_merge_trees()` to `unpack_trees_start()` and extract the
call to `discard_index()` into a new function `unpack_trees_finish()`.
As a result, these are called early resp. late in `merge_trees()`,
making the resource handling clearer. A later commit will expand on
that, teaching `..._finish()` to free more memory. (So rather than
moving the FIXME-comment, just drop it, since it will be addressed soon
enough.)

Also call `..._finish()` when `merge_trees()` returns early.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 merge-recursive.c | 29 +++++++++++++++--------------
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 680e01226b..ddb0fa7369 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -337,10 +337,10 @@ static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
 	init_tree_desc(desc, tree->buffer, tree->size);
 }
 
-static int git_merge_trees(struct merge_options *o,
-			   struct tree *common,
-			   struct tree *head,
-			   struct tree *merge)
+static int unpack_trees_start(struct merge_options *o,
+			      struct tree *common,
+			      struct tree *head,
+			      struct tree *merge)
 {
 	int rc;
 	struct tree_desc t[3];
@@ -379,6 +379,11 @@ static int git_merge_trees(struct merge_options *o,
 	return rc;
 }
 
+static void unpack_trees_finish(struct merge_options *o)
+{
+	discard_index(&o->orig_index);
+}
+
 struct tree *write_tree_from_memory(struct merge_options *o)
 {
 	struct tree *result = NULL;
@@ -3088,13 +3093,14 @@ int merge_trees(struct merge_options *o,
 		return 1;
 	}
 
-	code = git_merge_trees(o, common, head, merge);
+	code = unpack_trees_start(o, common, head, merge);
 
 	if (code != 0) {
 		if (show(o, 4) || o->call_depth)
 			err(o, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
+		unpack_trees_finish(o);
 		return -1;
 	}
 
@@ -3147,20 +3153,15 @@ int merge_trees(struct merge_options *o,
 
 		hashmap_free(&o->current_file_dir_set, 1);
 
-		if (clean < 0)
+		if (clean < 0) {
+			unpack_trees_finish(o);
 			return clean;
+		}
 	}
 	else
 		clean = 1;
 
-	/* Free the extra index left from git_merge_trees() */
-	/*
-	 * FIXME: Need to also free data allocated by
-	 * setup_unpack_trees_porcelain() tucked away in o->unpack_opts.msgs,
-	 * but the problem is that only half of it refers to dynamically
-	 * allocated data, while the other half points at static strings.
-	 */
-	discard_index(&o->orig_index);
+	unpack_trees_finish(o);
 
 	if (o->call_depth && !(*result = write_tree_from_memory(o)))
 		return -1;
-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 3/4] argv-array: return the pushed string from argv_push*()
  2018-05-21 14:54                                                             ` [PATCH v5 0/4] unpack_trees_options: free messages when done Martin Ågren
  2018-05-21 14:54                                                               ` [PATCH v5 1/4] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
  2018-05-21 14:54                                                               ` [PATCH v5 2/4] merge-recursive: provide pair of `unpack_trees_{start,finish}()` Martin Ågren
@ 2018-05-21 14:54                                                               ` Martin Ågren
  2018-05-21 14:54                                                               ` [PATCH v5 4/4] unpack_trees_options: free messages when done Martin Ågren
                                                                                 ` (2 subsequent siblings)
  5 siblings, 0 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-21 14:54 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jeff King, Jacob Keller, Elijah Newren, Ben Peart,
	Phillip Wood, Johannes Schindelin

From: Junio C Hamano <gitster@pobox.com>

Such an API change allows us to use an argv_array this way:

	struct argv_array to_free = ARGV_ARRAY_INIT;
        const char *msg;

        if (some condition) {
		msg = "constant string message";
		... other logic ...
	} else {
		msg = argv_pushf(&to_free, "format %s", var);
	}
	... use "msg" ...
	... do other things ...
	argv_clear(&to_free);

Note that argv_array_pushl() and argv_array_pushv() are used to push
one or more strings with a single call, so we do not return any one
of these strings from these two functions in order to reduce the
chance to misuse the API.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
---
 argv-array.h | 4 ++--
 argv-array.c | 6 ++++--
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/argv-array.h b/argv-array.h
index 29056e49a1..715c93b246 100644
--- a/argv-array.h
+++ b/argv-array.h
@@ -12,9 +12,9 @@ struct argv_array {
 #define ARGV_ARRAY_INIT { empty_argv, 0, 0 }
 
 void argv_array_init(struct argv_array *);
-void argv_array_push(struct argv_array *, const char *);
+const char *argv_array_push(struct argv_array *, const char *);
 __attribute__((format (printf,2,3)))
-void argv_array_pushf(struct argv_array *, const char *fmt, ...);
+const char *argv_array_pushf(struct argv_array *, const char *fmt, ...);
 LAST_ARG_MUST_BE_NULL
 void argv_array_pushl(struct argv_array *, ...);
 void argv_array_pushv(struct argv_array *, const char **);
diff --git a/argv-array.c b/argv-array.c
index 5d370fa336..449dfc105a 100644
--- a/argv-array.c
+++ b/argv-array.c
@@ -21,12 +21,13 @@ static void argv_array_push_nodup(struct argv_array *array, const char *value)
 	array->argv[array->argc] = NULL;
 }
 
-void argv_array_push(struct argv_array *array, const char *value)
+const char *argv_array_push(struct argv_array *array, const char *value)
 {
 	argv_array_push_nodup(array, xstrdup(value));
+	return array->argv[array->argc - 1];
 }
 
-void argv_array_pushf(struct argv_array *array, const char *fmt, ...)
+const char *argv_array_pushf(struct argv_array *array, const char *fmt, ...)
 {
 	va_list ap;
 	struct strbuf v = STRBUF_INIT;
@@ -36,6 +37,7 @@ void argv_array_pushf(struct argv_array *array, const char *fmt, ...)
 	va_end(ap);
 
 	argv_array_push_nodup(array, strbuf_detach(&v, NULL));
+	return array->argv[array->argc - 1];
 }
 
 void argv_array_pushl(struct argv_array *array, ...)
-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* [PATCH v5 4/4] unpack_trees_options: free messages when done
  2018-05-21 14:54                                                             ` [PATCH v5 0/4] unpack_trees_options: free messages when done Martin Ågren
                                                                                 ` (2 preceding siblings ...)
  2018-05-21 14:54                                                               ` [PATCH v5 3/4] argv-array: return the pushed string from argv_push*() Martin Ågren
@ 2018-05-21 14:54                                                               ` Martin Ågren
  2018-05-21 21:38                                                               ` [PATCH v5 0/4] " Jeff King
  2018-05-22  2:46                                                               ` Junio C Hamano
  5 siblings, 0 replies; 412+ messages in thread
From: Martin Ågren @ 2018-05-21 14:54 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jeff King, Jacob Keller, Elijah Newren, Ben Peart,
	Phillip Wood, Johannes Schindelin

The strings allocated in `setup_unpack_trees_porcelain()` are never
freed. Provide a function `clear_unpack_trees_porcelain()` to do so and
call it where we use `setup_unpack_trees_porcelain()`. The only
non-trivial user is `unpack_trees_start()`, where we should place the
new call in `unpack_trees_finish()`.

We keep the string pointers in an array, mixing pointers to static
memory and memory that we allocate on the heap. We also keep several
copies of the individual pointers. So we need to make sure that we do
not free what we must not free and that we do not double-free. Let a
separate argv_array take ownership of all the strings we create so that
we can easily free them.

Zero the whole array of string pointers to make sure that we do not
leave any dangling pointers.

Note that we only take responsibility for the memory allocated in
`setup_unpack_trees_porcelain()` and not any other members of the
`struct unpack_trees_options`.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
---
 unpack-trees.h     |  8 +++++++-
 builtin/checkout.c |  1 +
 merge-recursive.c  |  1 +
 merge.c            |  3 +++
 unpack-trees.c     | 17 ++++++++++++++---
 5 files changed, 26 insertions(+), 4 deletions(-)

diff --git a/unpack-trees.h b/unpack-trees.h
index 41178ada94..c2b434c606 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -2,7 +2,7 @@
 #define UNPACK_TREES_H
 
 #include "tree-walk.h"
-#include "string-list.h"
+#include "argv-array.h"
 
 #define MAX_UNPACK_TREES 8
 
@@ -33,6 +33,11 @@ enum unpack_trees_error_types {
 void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 				  const char *cmd);
 
+/*
+ * Frees resources allocated by setup_unpack_trees_porcelain().
+ */
+void clear_unpack_trees_porcelain(struct unpack_trees_options *opts);
+
 struct unpack_trees_options {
 	unsigned int reset,
 		     merge,
@@ -57,6 +62,7 @@ struct unpack_trees_options {
 	struct pathspec *pathspec;
 	merge_fn_t fn;
 	const char *msgs[NB_UNPACK_TREES_ERROR_TYPES];
+	struct argv_array msgs_to_free;
 	/*
 	 * Store error messages in an array, each case
 	 * corresponding to a error message type
diff --git a/builtin/checkout.c b/builtin/checkout.c
index b49b582071..5cebe170fc 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -526,6 +526,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 		init_tree_desc(&trees[1], tree->buffer, tree->size);
 
 		ret = unpack_trees(2, trees, &topts);
+		clear_unpack_trees_porcelain(&topts);
 		if (ret == -1) {
 			/*
 			 * Unpack couldn't do a trivial merge; either
diff --git a/merge-recursive.c b/merge-recursive.c
index ddb0fa7369..338f63a952 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -382,6 +382,7 @@ static int unpack_trees_start(struct merge_options *o,
 static void unpack_trees_finish(struct merge_options *o)
 {
 	discard_index(&o->orig_index);
+	clear_unpack_trees_porcelain(&o->unpack_opts);
 }
 
 struct tree *write_tree_from_memory(struct merge_options *o)
diff --git a/merge.c b/merge.c
index f123658e58..b433291d0c 100644
--- a/merge.c
+++ b/merge.c
@@ -130,8 +130,11 @@ int checkout_fast_forward(const struct object_id *head,
 
 	if (unpack_trees(nr_trees, t, &opts)) {
 		rollback_lock_file(&lock_file);
+		clear_unpack_trees_porcelain(&opts);
 		return -1;
 	}
+	clear_unpack_trees_porcelain(&opts);
+
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		return error(_("unable to write new index file"));
 	return 0;
diff --git a/unpack-trees.c b/unpack-trees.c
index 79fd97074e..73a6dc1701 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1,5 +1,6 @@
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
+#include "argv-array.h"
 #include "repository.h"
 #include "config.h"
 #include "dir.h"
@@ -103,6 +104,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 	const char **msgs = opts->msgs;
 	const char *msg;
 
+	argv_array_init(&opts->msgs_to_free);
+
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_commit_before_merge
 		      ? _("Your local changes to the following files would be overwritten by checkout:\n%%s"
@@ -119,7 +122,7 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 			  "Please commit your changes or stash them before you %s.")
 		      : _("Your local changes to the following files would be overwritten by %s:\n%%s");
 	msgs[ERROR_WOULD_OVERWRITE] = msgs[ERROR_NOT_UPTODATE_FILE] =
-		xstrfmt(msg, cmd, cmd);
+		argv_array_pushf(&opts->msgs_to_free, msg, cmd, cmd);
 
 	msgs[ERROR_NOT_UPTODATE_DIR] =
 		_("Updating the following directories would lose untracked files in them:\n%s");
@@ -139,7 +142,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 		      ? _("The following untracked working tree files would be removed by %s:\n%%s"
 			  "Please move or remove them before you %s.")
 		      : _("The following untracked working tree files would be removed by %s:\n%%s");
-	msgs[ERROR_WOULD_LOSE_UNTRACKED_REMOVED] = xstrfmt(msg, cmd, cmd);
+	msgs[ERROR_WOULD_LOSE_UNTRACKED_REMOVED] =
+		argv_array_pushf(&opts->msgs_to_free, msg, cmd, cmd);
 
 	if (!strcmp(cmd, "checkout"))
 		msg = advice_commit_before_merge
@@ -156,7 +160,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 		      ? _("The following untracked working tree files would be overwritten by %s:\n%%s"
 			  "Please move or remove them before you %s.")
 		      : _("The following untracked working tree files would be overwritten by %s:\n%%s");
-	msgs[ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN] = xstrfmt(msg, cmd, cmd);
+	msgs[ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN] =
+		argv_array_pushf(&opts->msgs_to_free, msg, cmd, cmd);
 
 	/*
 	 * Special case: ERROR_BIND_OVERLAP refers to a pair of paths, we
@@ -179,6 +184,12 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,
 		opts->unpack_rejects[i].strdup_strings = 1;
 }
 
+void clear_unpack_trees_porcelain(struct unpack_trees_options *opts)
+{
+	argv_array_clear(&opts->msgs_to_free);
+	memset(opts->msgs, 0, sizeof(opts->msgs));
+}
+
 static int do_add_entry(struct unpack_trees_options *o, struct cache_entry *ce,
 			 unsigned int set, unsigned int clear)
 {
-- 
2.17.0.840.g5d83f92caf


^ permalink raw reply related	[flat|nested] 412+ messages in thread

* Re: [PATCH v5 0/4] unpack_trees_options: free messages when done
  2018-05-21 14:54                                                             ` [PATCH v5 0/4] unpack_trees_options: free messages when done Martin Ågren
                                                                                 ` (3 preceding siblings ...)
  2018-05-21 14:54                                                               ` [PATCH v5 4/4] unpack_trees_options: free messages when done Martin Ågren
@ 2018-05-21 21:38                                                               ` Jeff King
  2018-05-22  2:46                                                               ` Junio C Hamano
  5 siblings, 0 replies; 412+ messages in thread
From: Jeff King @ 2018-05-21 21:38 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Junio C Hamano, git, Jacob Keller, Elijah Newren, Ben Peart,
	Phillip Wood, Johannes Schindelin

On Mon, May 21, 2018 at 04:54:24PM +0200, Martin Ågren wrote:

> That is, I've replaced the `string_list_appendf()`-patch with Junio's
> `argv_push*()`-patch, then squashed Junio's "redoing the 4/4"-patch into
> patch 4/4 -- with the exception of keeping the `memset(opts->msgs, ...)`
> which I suspect was mistakenly dropped.
> 
> Again, thanks for all the helpful comments and patches pointing me in
> the right direction.

I like it. Thanks for seeing it through. That was a lot of
back-and-forth for a small cleanup, but I hope we've established a
pattern that can be used elsewhere.

-Peff

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v5 0/4] unpack_trees_options: free messages when done
  2018-05-21 14:54                                                             ` [PATCH v5 0/4] unpack_trees_options: free messages when done Martin Ågren
                                                                                 ` (4 preceding siblings ...)
  2018-05-21 21:38                                                               ` [PATCH v5 0/4] " Jeff King
@ 2018-05-22  2:46                                                               ` Junio C Hamano
  2018-05-22  2:54                                                                 ` Junio C Hamano
  5 siblings, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-05-22  2:46 UTC (permalink / raw)
  To: Martin Ågren
  Cc: git, Jeff King, Jacob Keller, Elijah Newren, Ben Peart,
	Phillip Wood, Johannes Schindelin

Martin Ågren <martin.agren@gmail.com> writes:

> I've taken the six patches that Junio has queued and rebuilt the series
> to get rid of the new and possibly bug-prone function that no-one uses
> once the series is over.

Hmph, this unfortunately depends on 'next', which means we cannot
merge it down to 'maint' later to fix these leaks.  I guess it is
not a huge deal, though.  We've lived with these message leaks for
quite some time now and earth still kept rotating ;-)


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v5 0/4] unpack_trees_options: free messages when done
  2018-05-22  2:46                                                               ` Junio C Hamano
@ 2018-05-22  2:54                                                                 ` Junio C Hamano
  2018-05-22 11:11                                                                   ` Martin Ågren
  0 siblings, 1 reply; 412+ messages in thread
From: Junio C Hamano @ 2018-05-22  2:54 UTC (permalink / raw)
  To: Martin Ågren
  Cc: git, Jeff King, Jacob Keller, Elijah Newren, Ben Peart,
	Phillip Wood, Johannes Schindelin

Junio C Hamano <gitster@pobox.com> writes:

> Martin Ågren <martin.agren@gmail.com> writes:
>
>> I've taken the six patches that Junio has queued and rebuilt the series
>> to get rid of the new and possibly bug-prone function that no-one uses
>> once the series is over.
>
> Hmph, this unfortunately depends on 'next', which means we cannot
> merge it down to 'maint' later to fix these leaks.  I guess it is
> not a huge deal, though.  We've lived with these message leaks for
> quite some time now and earth still kept rotating ;-)

Oh, what was I thinking.  This, just like its previous rounds, is on
top of bp/merge-rename-config^0 and it is expected *not* to be
mergeable to 'maint' (or 'master', for that matter, at least not
yet).

Will queue.  Thanks.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v5 0/4] unpack_trees_options: free messages when done
  2018-05-22  2:54                                                                 ` Junio C Hamano
@ 2018-05-22 11:11                                                                   ` Martin Ågren
  2018-05-23  0:48                                                                     ` Junio C Hamano
  0 siblings, 1 reply; 412+ messages in thread
From: Martin Ågren @ 2018-05-22 11:11 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Git Mailing List, Jeff King, Jacob Keller, Elijah Newren,
	Ben Peart, Phillip Wood, Johannes Schindelin

On 22 May 2018 at 04:54, Junio C Hamano <gitster@pobox.com> wrote:
> Junio C Hamano <gitster@pobox.com> writes:

>> Hmph, this unfortunately depends on 'next', which means we cannot
>> merge it down to 'maint' later to fix these leaks.  I guess it is
>> not a huge deal, though.  We've lived with these message leaks for
>> quite some time now and earth still kept rotating ;-)
>
> Oh, what was I thinking.  This, just like its previous rounds, is on
> top of bp/merge-rename-config^0 and it is expected *not* to be
> mergeable to 'maint' (or 'master', for that matter, at least not
> yet).

Right. The reason it depends on that topic is the user in
merge-recursive.c. Other than patch 2 and a small part of patch 4, this
should be mergeable to 'master' (as I recall) and probably also to
'maint'. I suppose this series could have been done as three patches to
fix all users except one, then one or two patches to fix
merge-recursive.c.

That would have allowed merging the first part of the series to 'maint'.
(Maybe not to fix the leaking as such, but to keep 'maint' more up to
date with 'master' for easier merging of other topics?) If you'd prefer
an ordering like that (now and/or in the future), just let me know.

Martin

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v5 0/4] unpack_trees_options: free messages when done
  2018-05-22 11:11                                                                   ` Martin Ågren
@ 2018-05-23  0:48                                                                     ` Junio C Hamano
  0 siblings, 0 replies; 412+ messages in thread
From: Junio C Hamano @ 2018-05-23  0:48 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Git Mailing List, Jeff King, Jacob Keller, Elijah Newren,
	Ben Peart, Phillip Wood, Johannes Schindelin

Martin Ågren <martin.agren@gmail.com> writes:

> (Maybe not to fix the leaking as such, but to keep 'maint' more up to
> date with 'master' for easier merging of other topics?)

I admit that I occasionally do such a "presumably no-op" merge to
'maint' out of sheer laziness, but in general I'd prefer to keep
'maint' quieter than that.  As we do not have any repeated caller
that calls unpack_trees unbounded number of times in a process, I
think it is OK to limit this update only to 'master', especially at
this late in the cycle.

Thanks.

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges
  2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
                                 ` (17 preceding siblings ...)
  2018-04-26  3:51               ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Junio C Hamano
@ 2018-05-25 14:19               ` Sergey Organov
  2018-05-25 15:44                 ` Sergey Organov
  18 siblings, 1 reply; 412+ messages in thread
From: Sergey Organov @ 2018-05-25 14:19 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Martin Ågren


Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Junio, I think this is now ready for `next`. Thank you for your patience
> and help with this.
>
> Once upon a time, I dreamed of an interactive rebase that would not
> linearize all patches and drop all merge commits, but instead recreate
> the commit topology faithfully.
>
> My original attempt was --preserve-merges, but that design was so
> limited that I did not even enable it in interactive mode.
>
> Subsequently, it *was* enabled in interactive mode, with the predictable
> consequences: as the --preserve-merges design does not allow for
> specifying the parents of merge commits explicitly, all the new commits'
> parents are defined *implicitly* by the previous commit history, and
> hence it is *not possible to even reorder commits*.
>
> This design flaw cannot be fixed. Not without a complete re-design, at
> least. This patch series offers such a re-design.
>
> Think of --rebase-merges as "--preserve-merges done right". It
> introduces new verbs for the todo list, `label`, `reset` and `merge`.
> For a commit topology like this:
>
>             A - B - C
>               \   /
>                 D
>
> the generated todo list would look like this:
>
>             # branch D
>             pick 0123 A
>             label branch-point
>             pick 1234 D
>             label D
>
>             reset branch-point
>             pick 2345 B
>             merge -C 3456 D # C
>
> There are more patches in the pipeline, based on this patch series, but
> left for later in the interest of reviewable patch series: one mini
> series to use the sequencer even for `git rebase -i --root`, and another
> one to add support for octopus merges to --rebase-merges. And then one
> to allow for rebasing merge commits in a smarter way (this one will need
> a bit more work, though, as it can result in very complicated, nested
> merge conflicts *very* easily).
>
> Changes since v8:
>
> - Disentangled the patch introducing `label`/`reset` from the one
>   introducing `merge` again (this was one stupid, tired `git commit
>   --amend` too many).
>
> - Augmented the commit message of "introduce the `merge` command" to
>   describe what the `label onto` is all about.
>
> - Fixed the error message when `reset` would overwrite untracked files to
>   actually say that a "reset" failed (not a "merge").
>
> - Clarified the rationale for `label onto` in the commit message of
>   "rebase-helper --make-script: introduce a flag to rebase merges".
>
> - Edited the description of `--rebase-merges` heavily, for clarity, in
>   "rebase: introduce the --rebase-merges option".
>
> - Edited the commit message of (and the documentation introduced by) " rebase
>   -i: introduce --rebase-merges=[no-]rebase-cousins" for clarity (also
>   mentioning the `--ancestry-path` option).
>
> - When run_git_commit() fails after a successful merge, we now take pains
>   not to reschedule the `merge` command.
>
> - Rebased the patch series on top of current `master`, i.e. both
>   `pw/rebase-keep-empty-fixes` and `pw/rebase-signoff`, to resolve merge
>   conflicts myself.
>
>
> Johannes Schindelin (15):
>   sequencer: avoid using errno clobbered by rollback_lock_file()
>   sequencer: make rearrange_squash() a bit more obvious
>   sequencer: refactor how original todo list lines are accessed
>   sequencer: offer helpful advice when a command was rescheduled
>   sequencer: introduce new commands to reset the revision
>   sequencer: introduce the `merge` command
>   sequencer: fast-forward `merge` commands, if possible
>   rebase-helper --make-script: introduce a flag to rebase merges
>   rebase: introduce the --rebase-merges option
>   sequencer: make refs generated by the `label` command worktree-local
>   sequencer: handle post-rewrite for merge commands
>   rebase --rebase-merges: avoid "empty merges"
>   pull: accept --rebase=merges to recreate the branch topology
>   rebase -i: introduce --rebase-merges=[no-]rebase-cousins
>   rebase -i --rebase-merges: add a section to the man page
>
> Phillip Wood (1):
>   rebase --rebase-merges: add test for --keep-empty
>
> Stefan Beller (1):
>   git-rebase--interactive: clarify arguments
>
>  Documentation/config.txt               |   8 +
>  Documentation/git-pull.txt             |   6 +-
>  Documentation/git-rebase.txt           | 163 ++++-
>  builtin/pull.c                         |  14 +-
>  builtin/rebase--helper.c               |  13 +-
>  builtin/remote.c                       |  18 +-
>  contrib/completion/git-completion.bash |   4 +-
>  git-rebase--interactive.sh             |  22 +-
>  git-rebase.sh                          |  16 +
>  refs.c                                 |   3 +-
>  sequencer.c                            | 892 ++++++++++++++++++++++++-
>  sequencer.h                            |   7 +
>  t/t3421-rebase-topology-linear.sh      |   1 +
>  t/t3430-rebase-merges.sh               | 244 +++++++
>  14 files changed, 1352 insertions(+), 59 deletions(-)
>  create mode 100755 t/t3430-rebase-merges.sh
>
>
> base-commit: 1f1cddd558b54bb0ce19c8ace353fd07b758510d
> Published-As: https://github.com/dscho/git/releases/tag/recreate-merges-v9
> Fetch-It-Via: git fetch https://github.com/dscho/git recreate-merges-v9
>
> Interdiff vs v8:
>  diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
>  index e691b93e920..bd5ecff980e 100644
>  --- a/Documentation/git-rebase.txt
>  +++ b/Documentation/git-rebase.txt
>  @@ -381,21 +381,24 @@ have the long commit hash prepended to the format.
>   
>   -r::
>   --rebase-merges[=(rebase-cousins|no-rebase-cousins)]::
>  -	By default, a rebase will simply drop merge commits and only rebase
>  -	the non-merge commits. With this option, it will try to preserve
>  +	By default, a rebase will simply drop merge commits from the todo
>  +	list, and put the rebased commits into a single, linear branch.
>  +	With `--rebase-merges`, the rebase will instead try to preserve
>   	the branching structure within the commits that are to be rebased,
>  -	by recreating the merge commits. If a merge commit resolved any merge
>  -	or contained manual amendments, then they will have to be re-applied
>  -	manually.
>  +	by recreating the merge commits. Any resolved merge conflicts or
>  +	manual amendments in these merge commits will have to be
>  +	resolved/re-applied manually.
>   +
>   By default, or when `no-rebase-cousins` was specified, commits which do not
>  -have `<upstream>` as direct ancestor will keep their original branch point.
>  -If the `rebase-cousins` mode is turned on, such commits are instead rebased
>  +have `<upstream>` as direct ancestor will keep their original branch point,
>  +i.e. commits that would be excluded by gitlink:git-log[1]'s
>  +`--ancestry-path` option will keep their original ancestry by default. If
>  +the `rebase-cousins` mode is turned on, such commits are instead rebased
>   onto `<upstream>` (or `<onto>`, if specified).
>   +
>  -This mode is similar in spirit to `--preserve-merges`, but in contrast to
>  -that option works well in interactive rebases: commits can be reordered,
>  -inserted and dropped at will.
>  +The `--rebase-merges` mode is similar in spirit to `--preserve-merges`, but
>  +in contrast to that option works well in interactive rebases: commits can be
>  +reordered, inserted and dropped at will.
>   +
>   It is currently only possible to recreate the merge commits using the
>   `recursive` merge strategy; Different merge strategies can be used only via
>  diff --git a/sequencer.c b/sequencer.c
>  index b5715f69450..e2f83942843 100644
>  --- a/sequencer.c
>  +++ b/sequencer.c
>  @@ -2635,6 +2635,7 @@ static int do_reset(const char *name, int len, struct replay_opts *opts)
>   	}
>   
>   	memset(&unpack_tree_opts, 0, sizeof(unpack_tree_opts));
>  +	setup_unpack_trees_porcelain(&unpack_tree_opts, "reset");
>   	unpack_tree_opts.head_idx = 1;
>   	unpack_tree_opts.src_index = &the_index;
>   	unpack_tree_opts.dst_index = &the_index;
>  @@ -2855,7 +2856,12 @@ static int do_merge(struct commit *commit, const char *arg, int arg_len,
>   	if (ret)
>   		rerere(opts->allow_rerere_auto);
>   	else
>  -		ret = run_git_commit(git_path_merge_msg(), opts,
>  +		/*
>  +		 * In case of problems, we now want to return a positive
>  +		 * value (a negative one would indicate that the `merge`
>  +		 * command needs to be rescheduled).
>  +		 */
>  +		ret = !!run_git_commit(git_path_merge_msg(), opts,
>   				     run_commit_flags);
>   
>   leave_merge:
>  @@ -3809,12 +3815,9 @@ int sequencer_make_script(FILE *out, int argc, const char **argv,
>   
>   	init_revisions(&revs, NULL);
>   	revs.verbose_header = 1;
>  -	if (rebase_merges)
>  -		revs.cherry_mark = 1;
>  -	else {
>  +	if (!rebase_merges)
>   		revs.max_parents = 1;
>  -		revs.cherry_pick = 1;
>  -	}
>  +	revs.cherry_mark = 1;
>   	revs.limited = 1;
>   	revs.reverse = 1;
>   	revs.right_only = 1;

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges
  2018-05-25 14:19               ` Sergey Organov
@ 2018-05-25 15:44                 ` Sergey Organov
  0 siblings, 0 replies; 412+ messages in thread
From: Sergey Organov @ 2018-05-25 15:44 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: git, Junio C Hamano, Jacob Keller, Stefan Beller, Philip Oakley,
	Eric Sunshine, Phillip Wood, Igor Djordjevic, Johannes Sixt,
	Martin Ågren


This has been sent by mistake, I'm sorry, please disregard.

Sergey Organov <sorganov@gmail.com> writes:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>
>> Junio, I think this is now ready for `next`. Thank you for your patience
>> and help with this.

[...]


^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v4 4/4] rebase --skip: clean up commit message after a failed fixup/squash
  2018-05-06 17:50                   ` Phillip Wood
  2018-05-09 10:50                     ` Phillip Wood
@ 2018-10-02 13:50                     ` Johannes Schindelin
  2018-10-08 13:37                       ` Phillip Wood
  1 sibling, 1 reply; 412+ messages in thread
From: Johannes Schindelin @ 2018-10-02 13:50 UTC (permalink / raw)
  To: Phillip Wood; +Cc: git, Junio C Hamano, Eric Sunshine, Stefan Beller

Hi Phillip,

[sorry, I just got to this mail now]

On Sun, 6 May 2018, Phillip Wood wrote:

> On 27/04/18 21:48, Johannes Schindelin wrote:
> > 
> > During a series of fixup/squash commands, the interactive rebase builds
> > up a commit message with comments. This will be presented to the user in
> > the editor if at least one of those commands was a `squash`.
> > 
> > In any case, the commit message will be cleaned up eventually, removing
> > all those intermediate comments, in the final step of such a
> > fixup/squash chain.
> > 
> > However, if the last fixup/squash command in such a chain fails with
> > merge conflicts, and if the user then decides to skip it (or resolve it
> > to a clean worktree and then continue the rebase), the current code
> > fails to clean up the commit message.
> > 
> > This commit fixes that behavior.
> > 
> > The fix is quite a bit more involved than meets the eye because it is
> > not only about the question whether we are `git rebase --skip`ing a
> > fixup or squash. It is also about removing the skipped fixup/squash's
> > commit message from the accumulated commit message. And it is also about
> > the question whether we should let the user edit the final commit
> > message or not ("Was there a squash in the chain *that was not
> > skipped*?").
> > 
> > For example, in this case we will want to fix the commit message, but
> > not open it in an editor:
> > 
> > 	pick	<- succeeds
> > 	fixup	<- succeeds
> > 	squash	<- fails, will be skipped
> > 
> > This is where the newly-introduced `current-fixups` file comes in real
> > handy. A quick look and we can determine whether there was a non-skipped
> > squash. We only need to make sure to keep it up to date with respect to
> > skipped fixup/squash commands. As a bonus, we can even avoid committing
> > unnecessarily, e.g. when there was only one fixup, and it failed, and
> > was skipped.
> > 
> > To fix only the bug where the final commit message was not cleaned up
> > properly, but without fixing the rest, would have been more complicated
> > than fixing it all in one go, hence this commit lumps together more than
> > a single concern.
> > 
> > For the same reason, this commit also adds a bit more to the existing
> > test case for the regression we just fixed.
> > 
> > The diff is best viewed with --color-moved.
> > 
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  sequencer.c                | 113 ++++++++++++++++++++++++++++++++-----
> >  t/t3418-rebase-continue.sh |  35 ++++++++++--
> >  2 files changed, 131 insertions(+), 17 deletions(-)
> > 
> > diff --git a/sequencer.c b/sequencer.c
> > index 56166b0d6c7..cec180714ef 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > [...]
> > @@ -2804,19 +2801,107 @@ static int commit_staged_changes(struct replay_opts *opts)
> >  		if (get_oid_hex(rev.buf, &to_amend))
> >  			return error(_("invalid contents: '%s'"),
> >  				rebase_path_amend());
> > -		if (oidcmp(&head, &to_amend))
> > +		if (!is_clean && oidcmp(&head, &to_amend))
> >  			return error(_("\nYou have uncommitted changes in your "
> >  				       "working tree. Please, commit them\n"
> >  				       "first and then run 'git rebase "
> >  				       "--continue' again."));
> > +		/*
> > +		 * When skipping a failed fixup/squash, we need to edit the
> > +		 * commit message, the current fixup list and count, and if it
> > +		 * was the last fixup/squash in the chain, we need to clean up
> > +		 * the commit message and if there was a squash, let the user
> > +		 * edit it.
> > +		 */
> > +		if (is_clean && !oidcmp(&head, &to_amend) &&
> > +		    opts->current_fixup_count > 0 &&
> > +		    file_exists(rebase_path_stopped_sha())) {
> > +			const char *p = opts->current_fixups.buf;
> > +			int len = opts->current_fixups.len;
> > +
> > +			opts->current_fixup_count--;
> > +			if (!len)
> > +				BUG("Incorrect current_fixups:\n%s", p);
> > +			while (len && p[len - 1] != '\n')
> > +				len--;
> > +			strbuf_setlen(&opts->current_fixups, len);
> > +			if (write_message(p, len, rebase_path_current_fixups(),
> > +					  0) < 0)
> > +				return error(_("could not write file: '%s'"),
> > +					     rebase_path_current_fixups());
> > +
> > +			/*
> > +			 * If a fixup/squash in a fixup/squash chain failed, the
> > +			 * commit message is already correct, no need to commit
> > +			 * it again.
> > +			 *
> > +			 * Only if it is the final command in the fixup/squash
> > +			 * chain, and only if the chain is longer than a single
> > +			 * fixup/squash command (which was just skipped), do we
> > +			 * actually need to re-commit with a cleaned up commit
> > +			 * message.
> > +			 */
> > +			if (opts->current_fixup_count > 0 &&
> > +			    !is_fixup(peek_command(todo_list, 0))) {
> > +				final_fixup = 1;
> > +				/*
> > +				 * If there was not a single "squash" in the
> > +				 * chain, we only need to clean up the commit
> > +				 * message, no need to bother the user with
> > +				 * opening the commit message in the editor.
> > +				 */
> > +				if (!starts_with(p, "squash ") &&
> > +				    !strstr(p, "\nsquash "))
> > +					flags = (flags & ~EDIT_MSG) | CLEANUP_MSG;
> > +			} else if (is_fixup(peek_command(todo_list, 0))) {
> > +				/*
> > +				 * We need to update the squash message to skip
> > +				 * the latest commit message.
> > +				 */
> > +				struct commit *commit;
> > +				const char *path = rebase_path_squash_msg();
> > +
> > +				if (parse_head(&commit) ||
> > +				    !(p = get_commit_buffer(commit, NULL)) ||
> > +				    write_message(p, strlen(p), path, 0)) {
> > +					unuse_commit_buffer(commit, p);
> > +					return error(_("could not write file: "
> > +						       "'%s'"), path);
> > +				}
> 
> I think it should probably recreate the fixup message as well. If there
> is a sequence
> 
> pick commit
> fixup a
> fixup b
> fixup c
> 
> and 'fixup b' gets skipped then when 'fixup c' is applied the user will
> be prompted to edit the message unless rebase_path_fixup_msg() exists.

I was already on my way to fix this when I encountered this mail 30
minutes ago, but when I tried to implement the regression test, I took a
step back: apart from "disk full" and "pilot error", there is really no
good reason for the `message-squash` file not being able to be written.

And in both cases, a benign `git status` will show the latest command,
i.e. the one that failed.

In other words: I now deem this such a corner case that it is not worth
spending more time on the error message than we already did...

Ciao,
Dscho

> 
> Best Wishes
> 
> Phillip
> 
> > +				unuse_commit_buffer(commit, p);
> > +			}
> > +		}
> >  
> >  		strbuf_release(&rev);
> >  		flags |= AMEND_MSG;
> >  	}
> >  
> > -	if (run_git_commit(rebase_path_message(), opts, flags))
> > +	if (is_clean) {
> > +		const char *cherry_pick_head = git_path_cherry_pick_head();
> > +
> > +		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
> > +			return error(_("could not remove CHERRY_PICK_HEAD"));
> > +		if (!final_fixup)
> > +			return 0;
> > +	}
> > +
> > +	if (run_git_commit(final_fixup ? NULL : rebase_path_message(),
> > +			   opts, flags))
> >  		return error(_("could not commit staged changes."));
> >  	unlink(rebase_path_amend());
> > +	if (final_fixup) {
> > +		unlink(rebase_path_fixup_msg());
> > +		unlink(rebase_path_squash_msg());
> > +	}
> > +	if (opts->current_fixup_count > 0) {
> > +		/*
> > +		 * Whether final fixup or not, we just cleaned up the commit
> > +		 * message...
> > +		 */
> > +		unlink(rebase_path_current_fixups());
> > +		strbuf_reset(&opts->current_fixups);
> > +		opts->current_fixup_count = 0;
> > +	}
> >  	return 0;
> >  }
> >  
> > @@ -2828,14 +2913,16 @@ int sequencer_continue(struct replay_opts *opts)
> >  	if (read_and_refresh_cache(opts))
> >  		return -1;
> >  
> > +	if (read_populate_opts(opts))
> > +		return -1;
> >  	if (is_rebase_i(opts)) {
> > -		if (commit_staged_changes(opts))
> > +		if ((res = read_populate_todo(&todo_list, opts)))
> > +			goto release_todo_list;
> > +		if (commit_staged_changes(opts, &todo_list))
> >  			return -1;
> >  	} else if (!file_exists(get_todo_path(opts)))
> >  		return continue_single_pick();
> > -	if (read_populate_opts(opts))
> > -		return -1;
> > -	if ((res = read_populate_todo(&todo_list, opts)))
> > +	else if ((res = read_populate_todo(&todo_list, opts)))
> >  		goto release_todo_list;
> >  
> >  	if (!is_rebase_i(opts)) {
> > diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
> > index 3874f187246..03bf1b8a3b3 100755
> > --- a/t/t3418-rebase-continue.sh
> > +++ b/t/t3418-rebase-continue.sh
> > @@ -88,14 +88,14 @@ test_expect_success 'rebase passes merge strategy options correctly' '
> >  	git rebase --continue
> >  '
> >  
> > -test_expect_failure '--skip after failed fixup cleans commit message' '
> > +test_expect_success '--skip after failed fixup cleans commit message' '
> >  	test_when_finished "test_might_fail git rebase --abort" &&
> >  	git checkout -b with-conflicting-fixup &&
> >  	test_commit wants-fixup &&
> >  	test_commit "fixup! wants-fixup" wants-fixup.t 1 wants-fixup-1 &&
> >  	test_commit "fixup! wants-fixup" wants-fixup.t 2 wants-fixup-2 &&
> >  	test_commit "fixup! wants-fixup" wants-fixup.t 3 wants-fixup-3 &&
> > -	test_must_fail env FAKE_LINES="1 fixup 2 fixup 4" \
> > +	test_must_fail env FAKE_LINES="1 fixup 2 squash 4" \
> >  		git rebase -i HEAD~4 &&
> >  
> >  	: now there is a conflict, and comments in the commit message &&
> > @@ -103,11 +103,38 @@ test_expect_failure '--skip after failed fixup cleans commit message' '
> >  	grep "fixup! wants-fixup" out &&
> >  
> >  	: skip and continue &&
> > -	git rebase --skip &&
> > +	echo "cp \"\$1\" .git/copy.txt" | write_script copy-editor.sh &&
> > +	(test_set_editor "$PWD/copy-editor.sh" && git rebase --skip) &&
> > +
> > +	: the user should not have had to edit the commit message &&
> > +	test_path_is_missing .git/copy.txt &&
> >  
> >  	: now the comments in the commit message should have been cleaned up &&
> >  	git show HEAD >out &&
> > -	! grep "fixup! wants-fixup" out
> > +	! grep "fixup! wants-fixup" out &&
> > +
> > +	: now, let us ensure that "squash" is handled correctly &&
> > +	git reset --hard wants-fixup-3 &&
> > +	test_must_fail env FAKE_LINES="1 squash 4 squash 2 squash 4" \
> > +		git rebase -i HEAD~4 &&
> > +
> > +	: the first squash failed, but there are two more in the chain &&
> > +	(test_set_editor "$PWD/copy-editor.sh" &&
> > +	 test_must_fail git rebase --skip) &&
> > +
> > +	: not the final squash, no need to edit the commit message &&
> > +	test_path_is_missing .git/copy.txt &&
> > +
> > +	: The first squash was skipped, therefore: &&
> > +	git show HEAD >out &&
> > +	test_i18ngrep "# This is a combination of 2 commits" out &&
> > +
> > +	(test_set_editor "$PWD/copy-editor.sh" && git rebase --skip) &&
> > +	git show HEAD >out &&
> > +	test_i18ngrep ! "# This is a combination" out &&
> > +
> > +	: Final squash failed, but there was still a squash &&
> > +	test_i18ngrep "# This is a combination of 2 commits" .git/copy.txt
> >  '
> >  
> >  test_expect_success 'setup rerere database' '
> > 
> 
> 

^ permalink raw reply	[flat|nested] 412+ messages in thread

* Re: [PATCH v4 4/4] rebase --skip: clean up commit message after a failed fixup/squash
  2018-10-02 13:50                     ` Johannes Schindelin
@ 2018-10-08 13:37                       ` Phillip Wood
  0 siblings, 0 replies; 412+ messages in thread
From: Phillip Wood @ 2018-10-08 13:37 UTC (permalink / raw)
  To: Johannes Schindelin, Phillip Wood
  Cc: git, Junio C Hamano, Eric Sunshine, Stefan Beller

Hi Johannes

On 02/10/2018 14:50, Johannes Schindelin wrote:
> Hi Phillip,
> 
> [sorry, I just got to this mail now]

Don't worry, I'm impressed you remembered it, I'd completely forgotten 
about it.

> 
> On Sun, 6 May 2018, Phillip Wood wrote:
> 
>> On 27/04/18 21:48, Johannes Schindelin wrote:
>>>
>>> During a series of fixup/squash commands, the interactive rebase builds
>>> up a commit message with comments. This will be presented to the user in
>>> the editor if at least one of those commands was a `squash`.
>>>
>>> In any case, the commit message will be cleaned up eventually, removing
>>> all those intermediate comments, in the final step of such a
>>> fixup/squash chain.
>>>
>>> However, if the last fixup/squash command in such a chain fails with
>>> merge conflicts, and if the user then decides to skip it (or resolve it
>>> to a clean worktree and then continue the rebase), the current code
>>> fails to clean up the commit message.
>>>
>>> This commit fixes that behavior.
>>>
>>> The fix is quite a bit more involved than meets the eye because it is
>>> not only about the question whether we are `git rebase --skip`ing a
>>> fixup or squash. It is also about removing the skipped fixup/squash's
>>> commit message from the accumulated commit message. And it is also about
>>> the question whether we should let the user edit the final commit
>>> message or not ("Was there a squash in the chain *that was not
>>> skipped*?").
>>>
>>> For example, in this case we will want to fix the commit message, but
>>> not open it in an editor:
>>>
>>> 	pick	<- succeeds
>>> 	fixup	<- succeeds
>>> 	squash	<- fails, will be skipped
>>>
>>> This is where the newly-introduced `current-fixups` file comes in real
>>> handy. A quick look and we can determine whether there was a non-skipped
>>> squash. We only need to make sure to keep it up to date with respect to
>>> skipped fixup/squash commands. As a bonus, we can even avoid committing
>>> unnecessarily, e.g. when there was only one fixup, and it failed, and
>>> was skipped.
>>>
>>> To fix only the bug where the final commit message was not cleaned up
>>> properly, but without fixing the rest, would have been more complicated
>>> than fixing it all in one go, hence this commit lumps together more than
>>> a single concern.
>>>
>>> For the same reason, this commit also adds a bit more to the existing
>>> test case for the regression we just fixed.
>>>
>>> The diff is best viewed with --color-moved.
>>>
>>> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
>>> ---
>>>   sequencer.c                | 113 ++++++++++++++++++++++++++++++++-----
>>>   t/t3418-rebase-continue.sh |  35 ++++++++++--
>>>   2 files changed, 131 insertions(+), 17 deletions(-)
>>>
>>> diff --git a/sequencer.c b/sequencer.c
>>> index 56166b0d6c7..cec180714ef 100644
>>> --- a/sequencer.c
>>> +++ b/sequencer.c
>>> [...]
>>> @@ -2804,19 +2801,107 @@ static int commit_staged_changes(struct replay_opts *opts)
>>>   		if (get_oid_hex(rev.buf, &to_amend))
>>>   			return error(_("invalid contents: '%s'"),
>>>   				rebase_path_amend());
>>> -		if (oidcmp(&head, &to_amend))
>>> +		if (!is_clean && oidcmp(&head, &to_amend))
>>>   			return error(_("\nYou have uncommitted changes in your "
>>>   				       "working tree. Please, commit them\n"
>>>   				       "first and then run 'git rebase "
>>>   				       "--continue' again."));
>>> +		/*
>>> +		 * When skipping a failed fixup/squash, we need to edit the
>>> +		 * commit message, the current fixup list and count, and if it
>>> +		 * was the last fixup/squash in the chain, we need to clean up
>>> +		 * the commit message and if there was a squash, let the user
>>> +		 * edit it.
>>> +		 */
>>> +		if (is_clean && !oidcmp(&head, &to_amend) &&
>>> +		    opts->current_fixup_count > 0 &&
>>> +		    file_exists(rebase_path_stopped_sha())) {
>>> +			const char *p = opts->current_fixups.buf;
>>> +			int len = opts->current_fixups.len;
>>> +
>>> +			opts->current_fixup_count--;
>>> +			if (!len)
>>> +				BUG("Incorrect current_fixups:\n%s", p);
>>> +			while (len && p[len - 1] != '\n')
>>> +				len--;
>>> +			strbuf_setlen(&opts->current_fixups, len);
>>> +			if (write_message(p, len, rebase_path_current_fixups(),
>>> +					  0) < 0)
>>> +				return error(_("could not write file: '%s'"),
>>> +					     rebase_path_current_fixups());
>>> +
>>> +			/*
>>> +			 * If a fixup/squash in a fixup/squash chain failed, the
>>> +			 * commit message is already correct, no need to commit
>>> +			 * it again.
>>> +			 *
>>> +			 * Only if it is the final command in the fixup/squash
>>> +			 * chain, and only if the chain is longer than a single
>>> +			 * fixup/squash command (which was just skipped), do we
>>> +			 * actually need to re-commit with a cleaned up commit
>>> +			 * message.
>>> +			 */
>>> +			if (opts->current_fixup_count > 0 &&
>>> +			    !is_fixup(peek_command(todo_list, 0))) {
>>> +				final_fixup = 1;
>>> +				/*
>>> +				 * If there was not a single "squash" in the
>>> +				 * chain, we only need to clean up the commit
>>> +				 * message, no need to bother the user with
>>> +				 * opening the commit message in the editor.
>>> +				 */
>>> +				if (!starts_with(p, "squash ") &&
>>> +				    !strstr(p, "\nsquash "))
>>> +					flags = (flags & ~EDIT_MSG) | CLEANUP_MSG;
>>> +			} else if (is_fixup(peek_command(todo_list, 0))) {
>>> +				/*
>>> +				 * We need to update the squash message to skip
>>> +				 * the latest commit message.
>>> +				 */
>>> +				struct commit *commit;
>>> +				const char *path = rebase_path_squash_msg();
>>> +
>>> +				if (parse_head(&commit) ||
>>> +				    !(p = get_commit_buffer(commit, NULL)) ||
>>> +				    write_message(p, strlen(p), path, 0)) {
>>> +					unuse_commit_buffer(commit, p);
>>> +					return error(_("could not write file: "
>>> +						       "'%s'"), path);
>>> +				}
>>
>> I think it should probably recreate the fixup message as well. If there
>> is a sequence
>>
>> pick commit
>> fixup a
>> fixup b
>> fixup c
>>
>> and 'fixup b' gets skipped then when 'fixup c' is applied the user will
>> be prompted to edit the message unless rebase_path_fixup_msg() exists.
> 
> I was already on my way to fix this when I encountered this mail 30
> minutes ago, but when I tried to implement the regression test, I took a
> step back: apart from "disk full" and "pilot error", there is really no
> good reason for the `message-squash` file not being able to be written.

Ah, I should perhaps have cut the patch a line higher to make it clearer 
I was not talking about the error condition. The code rewrites 
'message-squash' to remove the commit message from the skipped 
fixup/squash. However if we have skipped a fixup and there are no 
preceding squash commands then when rebase stopped for the user to fix 
the merge conflicts it removed the 'message-fixup' file. I thought (and 
still think) it would be nice if 'rebase --skip' recreated the 
'message-fixup' file so that the user is not unnecessarily prompted to 
edit the commit message at the end of the fixup chain but it's not the 
end of the world if it stays as it is now.

Best Wishes

Phillip

> 
> And in both cases, a benign `git status` will show the latest command,
> i.e. the one that failed.
> 
> In other words: I now deem this such a corner case that it is not worth
> spending more time on the error message than we already did...
> 
> Ciao,
> Dscho
> 
>>
>> Best Wishes
>>
>> Phillip
>>
>>> +				unuse_commit_buffer(commit, p);
>>> +			}
>>> +		}
>>>   
>>>   		strbuf_release(&rev);
>>>   		flags |= AMEND_MSG;
>>>   	}
>>>   
>>> -	if (run_git_commit(rebase_path_message(), opts, flags))
>>> +	if (is_clean) {
>>> +		const char *cherry_pick_head = git_path_cherry_pick_head();
>>> +
>>> +		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
>>> +			return error(_("could not remove CHERRY_PICK_HEAD"));
>>> +		if (!final_fixup)
>>> +			return 0;
>>> +	}
>>> +
>>> +	if (run_git_commit(final_fixup ? NULL : rebase_path_message(),
>>> +			   opts, flags))
>>>   		return error(_("could not commit staged changes."));
>>>   	unlink(rebase_path_amend());
>>> +	if (final_fixup) {
>>> +		unlink(rebase_path_fixup_msg());
>>> +		unlink(rebase_path_squash_msg());
>>> +	}
>>> +	if (opts->current_fixup_count > 0) {
>>> +		/*
>>> +		 * Whether final fixup or not, we just cleaned up the commit
>>> +		 * message...
>>> +		 */
>>> +		unlink(rebase_path_current_fixups());
>>> +		strbuf_reset(&opts->current_fixups);
>>> +		opts->current_fixup_count = 0;
>>> +	}
>>>   	return 0;
>>>   }
>>>   
>>> @@ -2828,14 +2913,16 @@ int sequencer_continue(struct replay_opts *opts)
>>>   	if (read_and_refresh_cache(opts))
>>>   		return -1;
>>>   
>>> +	if (read_populate_opts(opts))
>>> +		return -1;
>>>   	if (is_rebase_i(opts)) {
>>> -		if (commit_staged_changes(opts))
>>> +		if ((res = read_populate_todo(&todo_list, opts)))
>>> +			goto release_todo_list;
>>> +		if (commit_staged_changes(opts, &todo_list))
>>>   			return -1;
>>>   	} else if (!file_exists(get_todo_path(opts)))
>>>   		return continue_single_pick();
>>> -	if (read_populate_opts(opts))
>>> -		return -1;
>>> -	if ((res = read_populate_todo(&todo_list, opts)))
>>> +	else if ((res = read_populate_todo(&todo_list, opts)))
>>>   		goto release_todo_list;
>>>   
>>>   	if (!is_rebase_i(opts)) {
>>> diff --git a/t/t3418-rebase-continue.sh b/t/t3418-rebase-continue.sh
>>> index 3874f187246..03bf1b8a3b3 100755
>>> --- a/t/t3418-rebase-continue.sh
>>> +++ b/t/t3418-rebase-continue.sh
>>> @@ -88,14 +88,14 @@ test_expect_success 'rebase passes merge strategy options correctly' '
>>>   	git rebase --continue
>>>   '
>>>   
>>> -test_expect_failure '--skip after failed fixup cleans commit message' '
>>> +test_expect_success '--skip after failed fixup cleans commit message' '
>>>   	test_when_finished "test_might_fail git rebase --abort" &&
>>>   	git checkout -b with-conflicting-fixup &&
>>>   	test_commit wants-fixup &&
>>>   	test_commit "fixup! wants-fixup" wants-fixup.t 1 wants-fixup-1 &&
>>>   	test_commit "fixup! wants-fixup" wants-fixup.t 2 wants-fixup-2 &&
>>>   	test_commit "fixup! wants-fixup" wants-fixup.t 3 wants-fixup-3 &&
>>> -	test_must_fail env FAKE_LINES="1 fixup 2 fixup 4" \
>>> +	test_must_fail env FAKE_LINES="1 fixup 2 squash 4" \
>>>   		git rebase -i HEAD~4 &&
>>>   
>>>   	: now there is a conflict, and comments in the commit message &&
>>> @@ -103,11 +103,38 @@ test_expect_failure '--skip after failed fixup cleans commit message' '
>>>   	grep "fixup! wants-fixup" out &&
>>>   
>>>   	: skip and continue &&
>>> -	git rebase --skip &&
>>> +	echo "cp \"\$1\" .git/copy.txt" | write_script copy-editor.sh &&
>>> +	(test_set_editor "$PWD/copy-editor.sh" && git rebase --skip) &&
>>> +
>>> +	: the user should not have had to edit the commit message &&
>>> +	test_path_is_missing .git/copy.txt &&
>>>   
>>>   	: now the comments in the commit message should have been cleaned up &&
>>>   	git show HEAD >out &&
>>> -	! grep "fixup! wants-fixup" out
>>> +	! grep "fixup! wants-fixup" out &&
>>> +
>>> +	: now, let us ensure that "squash" is handled correctly &&
>>> +	git reset --hard wants-fixup-3 &&
>>> +	test_must_fail env FAKE_LINES="1 squash 4 squash 2 squash 4" \
>>> +		git rebase -i HEAD~4 &&
>>> +
>>> +	: the first squash failed, but there are two more in the chain &&
>>> +	(test_set_editor "$PWD/copy-editor.sh" &&
>>> +	 test_must_fail git rebase --skip) &&
>>> +
>>> +	: not the final squash, no need to edit the commit message &&
>>> +	test_path_is_missing .git/copy.txt &&
>>> +
>>> +	: The first squash was skipped, therefore: &&
>>> +	git show HEAD >out &&
>>> +	test_i18ngrep "# This is a combination of 2 commits" out &&
>>> +
>>> +	(test_set_editor "$PWD/copy-editor.sh" && git rebase --skip) &&
>>> +	git show HEAD >out &&
>>> +	test_i18ngrep ! "# This is a combination" out &&
>>> +
>>> +	: Final squash failed, but there was still a squash &&
>>> +	test_i18ngrep "# This is a combination of 2 commits" .git/copy.txt
>>>   '
>>>   
>>>   test_expect_success 'setup rerere database' '
>>>
>>
>>


^ permalink raw reply	[flat|nested] 412+ messages in thread

end of thread, other threads:[~2018-10-08 13:38 UTC | newest]

Thread overview: 412+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-01-18 15:35 [PATCH 0/8] rebase -i: offer to recreate merge commits Johannes Schindelin
2018-01-18 15:35 ` [PATCH 1/8] sequencer: introduce new commands to reset the revision Johannes Schindelin
2018-01-18 16:25   ` Jacob Keller
2018-01-18 21:13     ` Johannes Schindelin
2018-01-18 21:21       ` Jacob Keller
2018-01-18 21:24     ` Philip Oakley
2018-01-18 21:28       ` Jacob Keller
2018-01-29 20:28       ` Johannes Schindelin
2018-01-22 21:25     ` Junio C Hamano
2018-01-29 22:00       ` Johannes Schindelin
2018-01-19  8:59   ` Eric Sunshine
2018-01-24 22:01     ` Junio C Hamano
2018-01-29 20:55       ` Johannes Schindelin
2018-01-29 20:50     ` Johannes Schindelin
2018-01-30  7:12       ` Eric Sunshine
2018-01-19 12:24   ` [PATCH 1/8] sequencer: introduce new commands to resettherevision Phillip Wood
2018-01-19 18:55     ` Phillip Wood
2018-01-19 18:59       ` Jacob Keller
2018-01-29 21:25         ` Johannes Schindelin
2018-01-29 21:29           ` Johannes Schindelin
2018-01-29 21:23     ` Johannes Schindelin
2018-01-18 15:35 ` [PATCH 2/8] sequencer: introduce the `merge` command Johannes Schindelin
2018-01-18 16:31   ` Jacob Keller
2018-01-18 21:22     ` Johannes Schindelin
2018-01-18 21:26       ` Jacob Keller
2018-01-19  9:54   ` Eric Sunshine
2018-01-19 14:45   ` Phillip Wood
2018-01-20  9:18     ` Jacob Keller
2018-01-29 21:41       ` Johannes Schindelin
2018-01-31 13:48         ` Johannes Schindelin
2018-01-31 17:58           ` Phillip Wood
2018-02-01  6:40           ` Jacob Keller
2018-01-22 22:12   ` Junio C Hamano
2018-01-29 22:15     ` Johannes Schindelin
2018-01-18 15:35 ` [PATCH 3/8] sequencer: fast-forward merge commits, if possible Johannes Schindelin
2018-01-19 14:53   ` Phillip Wood
2018-01-23 19:12     ` Junio C Hamano
2018-01-24 10:32       ` Phillip Wood
2018-01-24 18:51         ` Junio C Hamano
2018-01-29 21:47     ` Johannes Schindelin
2018-01-23 18:51   ` Junio C Hamano
2018-01-29 22:18     ` Johannes Schindelin
2018-01-18 15:35 ` [PATCH 4/8] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
2018-01-18 21:39   ` Philip Oakley
2018-01-19 10:34   ` Eric Sunshine
2018-01-23 20:13     ` Junio C Hamano
2018-01-29 21:07       ` Johannes Schindelin
2018-01-29 21:05     ` Johannes Schindelin
2018-01-23 20:03   ` Junio C Hamano
2018-01-29 22:37     ` Johannes Schindelin
2018-01-18 15:35 ` [PATCH 5/8] rebase: introduce the --recreate-merges option Johannes Schindelin
2018-01-19 10:55   ` Eric Sunshine
2018-01-29 21:09     ` Johannes Schindelin
2018-01-23 20:22   ` Junio C Hamano
2018-02-10 19:31     ` Johannes Schindelin
2018-02-07  6:16   ` Sergey Organov
2018-02-07  7:26     ` Jacob Keller
2018-02-07  9:47       ` Sergey Organov
2018-02-07  7:27     ` Johannes Sixt
2018-02-07 17:36     ` Johannes Schindelin
2018-02-07 22:58       ` Øyvind Rønningstad
2018-02-07 23:31         ` Junio C Hamano
2018-02-08 12:34           ` Johannes Schindelin
2018-02-14  5:41             ` Sergey Organov
2018-02-09  6:11       ` Sergey Organov
2018-02-09  7:13         ` Johannes Sixt
2018-02-11 10:16           ` Jacob Keller
2018-02-12  7:38           ` Sergey Organov
2018-02-09  6:50   ` Sergey Organov
2018-02-10 23:06     ` Johannes Schindelin
2018-02-12  4:58       ` Sergey Organov
2018-02-12 20:21         ` Johannes Schindelin
2018-02-13  6:44           ` Sergey Organov
2018-02-15  1:08             ` Johannes Schindelin
2018-02-15  4:28               ` Sergey Organov
2018-02-15 16:51                 ` Johannes Schindelin
2018-02-12  5:22       ` Sergey Organov
2018-02-12 20:39         ` Johannes Schindelin
2018-02-13  4:39           ` Jacob Keller
2018-02-13  7:15             ` Sergey Organov
2018-02-14  1:35               ` Jacob Keller
2018-02-15  1:14                 ` Johannes Schindelin
2018-02-15  4:35                   ` Sergey Organov
2018-02-15 16:50                     ` Johannes Schindelin
2018-02-13  6:43           ` Sergey Organov
2018-02-15  1:40             ` Johannes Schindelin
2018-01-18 15:35 ` [PATCH 6/8] sequencer: handle autosquash and post-rewrite for merge commands Johannes Schindelin
2018-01-18 16:43   ` Jacob Keller
2018-01-18 21:27     ` Johannes Schindelin
2018-01-18 21:29       ` Jacob Keller
2018-01-23 20:27     ` Junio C Hamano
2018-01-18 15:36 ` [PATCH 7/8] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
2018-01-18 15:36 ` [PATCH 8/8] rebase -i: introduce --recreate-merges=no-rebase-cousins Johannes Schindelin
2018-01-18 22:00   ` Philip Oakley
2018-01-29 20:42     ` Johannes Schindelin
2018-01-20  1:09   ` Eric Sunshine
2018-01-18 16:49 ` [PATCH 0/8] rebase -i: offer to recreate merge commits Jacob Keller
2018-01-18 18:36 ` [PATCH 9, 10/8] interactive rebase feedback Stefan Beller
2018-01-18 18:36   ` [PATCH 9/8] [DO NOT APPLY, but squash?] git-rebase--interactive: clarify arguments Stefan Beller
2018-01-18 21:18     ` Jacob Keller
2018-01-18 21:36     ` Johannes Schindelin
2018-01-18 21:58       ` Stefan Beller
2018-01-19 20:30       ` Junio C Hamano
2018-01-20  9:14         ` Jacob Keller
2018-01-29 17:02           ` Johannes Schindelin
2018-01-18 18:36   ` [PATCH 10/8] [DO NOT APPLY, but improve?] rebase--interactive: introduce "stop" command Stefan Beller
2018-01-18 21:20     ` Jacob Keller
2018-01-18 22:08       ` Philip Oakley
2018-01-18 22:09         ` Jacob Keller
2018-01-18 22:00     ` Johannes Schindelin
2018-01-18 22:09       ` Stefan Beller
2018-01-19 20:25 ` [PATCH 0/8] rebase -i: offer to recreate merge commits Junio C Hamano
2018-01-29 21:53   ` Johannes Schindelin
2018-01-23 20:29 ` Junio C Hamano
2018-01-29 22:53   ` Johannes Schindelin
2018-01-29 22:54 ` [PATCH v2 00/10] " Johannes Schindelin
2018-01-29 22:54   ` [PATCH v2 01/10] git-rebase--interactive: clarify arguments Johannes Schindelin
2018-01-29 22:54   ` [PATCH v2 02/10] sequencer: introduce new commands to reset the revision Johannes Schindelin
2018-01-30  8:06     ` Eric Sunshine
2018-02-10 20:58       ` Johannes Schindelin
2018-01-30 20:17     ` Stefan Beller
2018-01-31 13:21       ` Johannes Schindelin
2018-01-31 18:02         ` [PATCH v2 02/10] sequencer: introduce new commands to reset therevision Phillip Wood
2018-02-10 21:49           ` Johannes Schindelin
2018-01-29 22:54   ` [PATCH v2 03/10] sequencer: introduce the `merge` command Johannes Schindelin
2018-01-29 22:54   ` [PATCH v2 04/10] sequencer: fast-forward merge commits, if possible Johannes Schindelin
2018-01-29 22:54   ` [PATCH v2 05/10] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
2018-01-29 22:54   ` [PATCH v2 06/10] rebase: introduce the --recreate-merges option Johannes Schindelin
2018-01-29 22:54   ` [PATCH v2 07/10] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
2018-01-29 22:55   ` [PATCH v2 08/10] sequencer: handle autosquash and post-rewrite for merge commands Johannes Schindelin
2018-01-29 22:55   ` [PATCH v2 09/10] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
2018-01-29 22:55   ` [PATCH v2 10/10] rebase -i: introduce --recreate-merges=[no-]rebase-cousins Johannes Schindelin
2018-01-30 18:47   ` [PATCH v2 00/10] rebase -i: offer to recreate merge commits Stefan Beller
2018-01-31 13:08     ` Johannes Schindelin
2018-01-30 21:36   ` Junio C Hamano
2018-01-31 13:29     ` Johannes Schindelin
2018-02-01  6:37       ` Jacob Keller
2018-02-11  0:09   ` [PATCH v3 00/12] " Johannes Schindelin
2018-02-11  0:10     ` [PATCH v3 01/12] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
2018-02-11  0:10     ` [PATCH v3 02/12] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
2018-02-11  0:10     ` [PATCH v3 03/12] git-rebase--interactive: clarify arguments Johannes Schindelin
2018-02-11  0:10     ` [PATCH v3 04/12] sequencer: introduce new commands to reset the revision Johannes Schindelin
2018-02-12 19:26       ` Eric Sunshine
2018-02-12 20:46         ` Johannes Schindelin
2018-02-11  0:10     ` [PATCH v3 05/12] sequencer: introduce the `merge` command Johannes Schindelin
2018-02-12  8:48       ` Eric Sunshine
2018-02-12 20:17         ` Johannes Schindelin
2018-02-11  0:10     ` [PATCH v3 06/12] sequencer: fast-forward merge commits, if possible Johannes Schindelin
2018-02-11  0:10     ` [PATCH v3 07/12] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
2018-02-11  0:10     ` [PATCH v3 08/12] rebase: introduce the --recreate-merges option Johannes Schindelin
2018-02-11  0:10     ` [PATCH v3 09/12] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
2018-02-11  0:10     ` [PATCH v3 10/12] sequencer: handle post-rewrite for merge commands Johannes Schindelin
2018-02-11  0:10     ` [PATCH v3 11/12] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
2018-02-11  0:10     ` [PATCH v3 12/12] rebase -i: introduce --recreate-merges=[no-]rebase-cousins Johannes Schindelin
2018-02-23 12:35     ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Johannes Schindelin
2018-02-23 12:35       ` [PATCH v4 01/12] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
2018-02-23 12:36       ` [PATCH v4 02/12] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
2018-02-23 12:36       ` [PATCH v4 03/12] git-rebase--interactive: clarify arguments Johannes Schindelin
2018-02-23 12:37       ` [PATCH v4 04/12] sequencer: introduce new commands to reset the revision Johannes Schindelin
2018-02-23 12:37       ` [PATCH v4 05/12] sequencer: introduce the `merge` command Johannes Schindelin
2018-02-23 12:37       ` [PATCH v4 06/12] sequencer: fast-forward merge commits, if possible Johannes Schindelin
2018-02-23 12:38       ` [PATCH v4 07/12] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
2018-02-23 12:38       ` [PATCH v4 08/12] rebase: introduce the --recreate-merges option Johannes Schindelin
2018-02-23 12:38       ` [PATCH v4 09/12] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
2018-02-23 12:39       ` [PATCH v4 10/12] sequencer: handle post-rewrite for merge commands Johannes Schindelin
2018-02-23 12:39       ` [PATCH v4 11/12] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
2018-02-23 12:39       ` [PATCH v4 12/12] rebase -i: introduce --recreate-merges=[no-]rebase-cousins Johannes Schindelin
2018-02-25 10:54       ` [PATCH v4 00/12] rebase -i: offer to recreate merge commits Jacob Keller
2018-02-26 20:49         ` Johannes Schindelin
2018-02-26 21:29     ` [PATCH v5 " Johannes Schindelin
2018-02-26 21:29       ` [PATCH v5 01/12] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
2018-02-27 21:33         ` Martin Ågren
2018-03-02 20:33           ` Johannes Schindelin
2018-02-26 21:29       ` [PATCH v5 02/12] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
2018-02-26 21:29       ` [PATCH v5 03/12] git-rebase--interactive: clarify arguments Johannes Schindelin
2018-02-26 21:29       ` [PATCH v5 04/12] sequencer: introduce new commands to reset the revision Johannes Schindelin
2018-02-26 21:29       ` [PATCH v5 05/12] sequencer: introduce the `merge` command Johannes Schindelin
2018-02-26 21:29       ` [PATCH v5 06/12] sequencer: fast-forward merge commits, if possible Johannes Schindelin
2018-02-26 21:29       ` [PATCH v5 07/12] rebase-helper --make-script: introduce a flag to recreate merges Johannes Schindelin
2018-02-26 21:29       ` [PATCH v5 08/12] rebase: introduce the --recreate-merges option Johannes Schindelin
2018-02-26 21:29       ` [PATCH v5 09/12] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
2018-02-26 21:29       ` [PATCH v5 10/12] sequencer: handle post-rewrite for merge commands Johannes Schindelin
2018-02-26 21:29       ` [PATCH v5 11/12] pull: accept --rebase=recreate to recreate the branch topology Johannes Schindelin
2018-02-26 21:29       ` [PATCH v5 12/12] rebase -i: introduce --recreate-merges=[no-]rebase-cousins Johannes Schindelin
2018-03-06  4:02       ` [PATCH v5 00/12] rebase -i: offer to recreate merge commits Igor Djordjevic
2018-03-07 13:50         ` Johannes Schindelin
2018-04-10 12:29       ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Johannes Schindelin
2018-04-10 12:29         ` [PATCH v6 01/15] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
2018-04-10 12:29         ` [PATCH v6 02/15] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
2018-04-10 12:29         ` [PATCH v6 03/15] git-rebase--interactive: clarify arguments Johannes Schindelin
2018-04-10 12:29         ` [PATCH v6 04/15] sequencer: introduce new commands to reset the revision Johannes Schindelin
2018-04-11  6:17           ` Sergey Organov
2018-04-11 11:36             ` Johannes Schindelin
2018-04-11 16:07               ` Sergey Organov
2018-04-13 10:03           ` Phillip Wood
2018-04-15 17:17             ` Philip Oakley
2018-04-18 18:00               ` Phillip Wood
2018-04-10 12:29         ` [PATCH v6 05/15] sequencer: introduce the `merge` command Johannes Schindelin
2018-04-13 10:12           ` Phillip Wood
2018-04-13 17:57             ` Phillip Wood
2018-04-14  0:51               ` Johannes Schindelin
2018-04-18 18:04                 ` Phillip Wood
2018-04-19 12:03                 ` Johannes Schindelin
2018-04-10 12:29         ` [PATCH v6 06/15] sequencer: fast-forward `merge` commands, if possible Johannes Schindelin
2018-04-10 12:29         ` [PATCH v6 07/15] rebase-helper --make-script: introduce a flag to rebase merges Johannes Schindelin
2018-04-10 12:29         ` [PATCH v6 08/15] rebase: introduce the --rebase-merges option Johannes Schindelin
2018-04-10 12:30         ` [PATCH v6 09/15] rebase --rebase-merges: add test for --keep-empty Johannes Schindelin
2018-04-10 12:30         ` [PATCH v6 10/15] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
2018-04-10 12:30         ` [PATCH v6 11/15] sequencer: handle post-rewrite for merge commands Johannes Schindelin
2018-04-10 12:30         ` [PATCH v6 12/15] rebase --rebase-merges: avoid "empty merges" Johannes Schindelin
2018-04-10 12:30         ` [PATCH v6 13/15] pull: accept --rebase=merges to recreate the branch topology Johannes Schindelin
2018-04-10 12:30         ` [PATCH v6 14/15] rebase -i: introduce --rebase-merges=[no-]rebase-cousins Johannes Schindelin
2018-04-12 11:30           ` Sergey Organov
2018-04-10 12:30         ` [PATCH v6 15/15] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
2018-04-10 18:49           ` Martin Ågren
2018-04-10 21:56             ` Johannes Schindelin
2018-04-11 15:35           ` Phillip Wood
2018-04-11 19:10             ` Eric Sunshine
2018-04-12  9:00               ` Johannes Schindelin
2018-04-13 15:21               ` Phillip Wood
2018-04-12  9:30             ` Johannes Schindelin
2018-04-12 18:29               ` Jacob Keller
2018-04-13 15:27               ` Phillip Wood
2018-04-12 11:52           ` Sergey Organov
2018-04-10 14:52         ` [PATCH v6 00/15] rebase -i: offer to recreate commit topology Sergey Organov
2018-04-10 22:11           ` Johannes Schindelin
2018-04-11  4:54             ` Sergey Organov
2018-04-11 11:28               ` Johannes Schindelin
2018-04-11 13:13                 ` Sergey Organov
2018-04-11 20:40                   ` Johannes Schindelin
2018-04-12  8:34                     ` Sergey Organov
2018-04-12 12:31                       ` Johannes Schindelin
2018-04-11 23:52                   ` Jacob Keller
2018-04-12  5:42                     ` Sergey Organov
2018-04-12 17:03                       ` Jacob Keller
2018-04-12 22:02                         ` Johannes Schindelin
2018-04-12 22:14                           ` Jacob Keller
2018-04-13 12:08                             ` Johannes Schindelin
2018-04-13 15:43                           ` Phillip Wood
2018-04-13 23:48                             ` Johannes Schindelin
2018-04-19  4:24                               ` Sergey Organov
2018-04-19  8:23                                 ` Jacob Keller
2018-04-19 11:28                                   ` Sergey Organov
2018-04-20  8:26                                   ` Johannes Schindelin
2018-04-20 20:39                                     ` Jacob Keller
2018-04-18  5:23                         ` Sergey Organov
2018-04-19 12:12         ` [PATCH v7 00/17] " Johannes Schindelin
2018-04-19 12:15           ` [PATCH v7 01/17] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
2018-04-19 12:18           ` [PATCH v7 02/17] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
2018-04-19 12:19           ` [PATCH v7 03/17] sequencer: refactor how original todo list lines are accessed Johannes Schindelin
2018-04-19 12:19           ` [PATCH v7 04/17] sequencer: offer helpful advice when a command was rescheduled Johannes Schindelin
2018-04-19 12:20           ` [PATCH v7 05/17] git-rebase--interactive: clarify arguments Johannes Schindelin
2018-04-19 12:20           ` [PATCH v7 06/17] sequencer: introduce new commands to reset the revision Johannes Schindelin
2018-04-20  9:39             ` Phillip Wood
2018-04-20 22:39               ` Johannes Schindelin
2018-04-19 12:20           ` [PATCH v7 07/17] # This is a combination of 2 commits. # This is the 1st commit message: Johannes Schindelin
2018-04-20  5:38             ` Eric Sunshine
2018-04-20  8:34               ` Johannes Schindelin
2018-04-20 21:06               ` [PATCH v2 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
2018-04-20 21:07                 ` [PATCH v2 1/4] rebase -i: demonstrate bugs with fixup!/squash! " Johannes Schindelin
2018-04-20 21:07                 ` [PATCH v2 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON Johannes Schindelin
2018-04-20 21:16                   ` Stefan Beller
2018-04-21  7:20                     ` Johannes Schindelin
2018-04-20 21:07                 ` [PATCH v2 3/4] sequencer: leave a tell-tale when a fixup/squash failed Johannes Schindelin
2018-04-20 21:25                   ` Stefan Beller
2018-04-21  7:24                     ` Johannes Schindelin
2018-04-20 21:08                 ` [PATCH v2 4/4] rebase --skip: clean up commit message after a failed fixup/squash Johannes Schindelin
2018-04-21  7:34               ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Johannes Schindelin
2018-04-21  7:35                 ` [PATCH v3 1/4] rebase -i: demonstrate bugs with fixup!/squash! " Johannes Schindelin
2018-04-21  7:35                 ` [PATCH v3 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON Johannes Schindelin
2018-04-21  7:35                 ` [PATCH v3 3/4] sequencer: leave a tell-tale when a fixup/squash failed Johannes Schindelin
2018-04-21  7:39                 ` [PATCH v3 4/4] rebase --skip: clean up commit message after a failed fixup/squash Johannes Schindelin
2018-04-23 18:11                 ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" in commit messages Stefan Beller
2018-04-23 19:50                   ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combinationof" " Phillip Wood
2018-04-25 12:48                     ` Johannes Schindelin
2018-04-25 17:09                       ` Phillip Wood
2018-04-26  9:51                         ` Johannes Schindelin
2018-04-26 10:52                           ` Phillip Wood
2018-04-24  1:28                 ` [PATCH v3 0/4] rebase -i: avoid stale "# This is a combination of" " Junio C Hamano
2018-04-27 20:48               ` [PATCH v4 " Johannes Schindelin
2018-04-27 20:48                 ` [PATCH v4 1/4] rebase -i: demonstrate bugs with fixup!/squash! " Johannes Schindelin
2018-04-27 20:48                 ` [PATCH v4 2/4] rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON Johannes Schindelin
2018-04-27 20:48                 ` [PATCH v4 3/4] sequencer: always commit without editing when asked for Johannes Schindelin
2018-04-27 20:48                 ` [PATCH v4 4/4] rebase --skip: clean up commit message after a failed fixup/squash Johannes Schindelin
2018-04-27 21:28                   ` Stefan Beller
2018-04-28 13:05                     ` Johannes Schindelin
2018-05-06 17:50                   ` Phillip Wood
2018-05-09 10:50                     ` Phillip Wood
2018-10-02 13:50                     ` Johannes Schindelin
2018-10-08 13:37                       ` Phillip Wood
2018-04-19 12:21           ` [PATCH v7 08/17] sequencer: fast-forward `merge` commands, if possible Johannes Schindelin
2018-04-19 12:21           ` [PATCH v7 09/17] rebase-helper --make-script: introduce a flag to rebase merges Johannes Schindelin
2018-04-19 12:21           ` [PATCH v7 10/17] rebase: introduce the --rebase-merges option Johannes Schindelin
2018-04-19 12:22           ` [PATCH v7 11/17] rebase --rebase-merges: add test for --keep-empty Johannes Schindelin
2018-04-19 12:22           ` [PATCH v7 12/17] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
2018-04-19 12:23           ` [PATCH v7 13/17] sequencer: handle post-rewrite for merge commands Johannes Schindelin
2018-04-19 12:23           ` [PATCH v7 14/17] rebase --rebase-merges: avoid "empty merges" Johannes Schindelin
2018-04-19 12:23           ` [PATCH v7 15/17] pull: accept --rebase=merges to recreate the branch topology Johannes Schindelin
2018-04-19 12:24           ` [PATCH v7 16/17] rebase -i: introduce --rebase-merges=[no-]rebase-cousins Johannes Schindelin
2018-04-19 12:24           ` [PATCH v7 17/17] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
2018-04-21 10:29           ` [PATCH v8 00/16] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
2018-04-21 10:30             ` [PATCH v8 01/16] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
2018-04-21 10:30             ` [PATCH v8 02/16] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
2018-04-21 10:31             ` [PATCH v8 03/16] sequencer: refactor how original todo list lines are accessed Johannes Schindelin
2018-04-21 10:31             ` [PATCH v8 04/16] sequencer: offer helpful advice when a command was rescheduled Johannes Schindelin
2018-04-21 10:32             ` [PATCH v8 05/16] git-rebase--interactive: clarify arguments Johannes Schindelin
2018-04-21 10:33             ` [PATCH v8 06/16] sequencer: introduce the `merge` command Johannes Schindelin
2018-04-21 15:56               ` Phillip Wood
2018-04-22 17:17                 ` Phillip Wood
2018-04-23 12:22                   ` Johannes Schindelin
2018-04-23 12:20                 ` Johannes Schindelin
2018-04-23 15:54                   ` Phillip Wood
2018-04-24  5:13                     ` Martin Ågren
2018-04-24  5:13                       ` [PATCH 1/2] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
2018-04-24  6:20                         ` Jacob Keller
2018-04-24  9:36                           ` Martin Ågren
2018-04-24 11:30                             ` Johannes Schindelin
2018-04-24  5:13                       ` [PATCH 2/2] unpack_trees_options: free messages when done Martin Ågren
2018-04-24 16:29                         ` Elijah Newren
2018-04-28 11:32                           ` Martin Ågren
2018-04-28 12:30                             ` Johannes Schindelin
2018-04-28 20:56                             ` Elijah Newren
2018-05-16 14:32                             ` Elijah Newren
2018-05-16 16:30                               ` [PATCH v2 0/3] " Martin Ågren
2018-05-16 16:30                                 ` [PATCH v2 1/3] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
2018-05-16 16:41                                   ` Stefan Beller
2018-05-16 19:29                                     ` Martin Ågren
2018-05-16 21:21                                       ` Jacob Keller
2018-05-16 21:20                                     ` Jacob Keller
2018-05-17 21:48                                   ` Junio C Hamano
2018-05-18  1:59                                     ` Jacob Keller
2018-05-16 16:30                                 ` [PATCH v2 2/3] merge-recursive: provide pair of `unpack_trees_{start,finish}()` Martin Ågren
2018-05-16 16:31                                 ` [PATCH v2 3/3] unpack_trees_options: free messages when done Martin Ågren
2018-05-17 22:10                                   ` Junio C Hamano
2018-05-18  5:08                                     ` Martin Ågren
2018-05-18 21:23                                       ` [PATCH v3 0/3] " Martin Ågren
2018-05-18 21:23                                         ` [PATCH v3 1/3] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
2018-05-18 21:23                                         ` [PATCH v3 2/3] merge-recursive: provide pair of `unpack_trees_{start,finish}()` Martin Ågren
2018-05-18 21:23                                         ` [PATCH v3 3/3] unpack_trees_options: free messages when done Martin Ågren
2018-05-18 21:33                                           ` Jeff King
2018-05-18 22:30                                             ` Elijah Newren
2018-05-19  1:02                                               ` Jeff King
2018-05-19  6:13                                                 ` Martin Ågren
2018-05-20 10:17                                                   ` [PATCH v4 0/4] " Martin Ågren
2018-05-20 10:17                                                     ` [PATCH v4 1/4] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
2018-05-20 10:17                                                     ` [PATCH v4 2/4] merge-recursive: provide pair of `unpack_trees_{start,finish}()` Martin Ågren
2018-05-20 10:17                                                     ` [PATCH v4 3/4] string-list: provide `string_list_appendf()` Martin Ågren
2018-05-20 19:12                                                       ` Jacob Keller
2018-05-21  0:01                                                         ` Re*: " Junio C Hamano
2018-05-21  0:25                                                           ` Junio C Hamano
2018-05-21  2:39                                                             ` Jeff King
2018-05-21 14:54                                                             ` [PATCH v5 0/4] unpack_trees_options: free messages when done Martin Ågren
2018-05-21 14:54                                                               ` [PATCH v5 1/4] merge: setup `opts` later in `checkout_fast_forward()` Martin Ågren
2018-05-21 14:54                                                               ` [PATCH v5 2/4] merge-recursive: provide pair of `unpack_trees_{start,finish}()` Martin Ågren
2018-05-21 14:54                                                               ` [PATCH v5 3/4] argv-array: return the pushed string from argv_push*() Martin Ågren
2018-05-21 14:54                                                               ` [PATCH v5 4/4] unpack_trees_options: free messages when done Martin Ågren
2018-05-21 21:38                                                               ` [PATCH v5 0/4] " Jeff King
2018-05-22  2:46                                                               ` Junio C Hamano
2018-05-22  2:54                                                                 ` Junio C Hamano
2018-05-22 11:11                                                                   ` Martin Ågren
2018-05-23  0:48                                                                     ` Junio C Hamano
2018-05-21  2:38                                                           ` Re*: [PATCH v4 3/4] string-list: provide `string_list_appendf()` Jeff King
2018-05-20 10:17                                                     ` [PATCH v4 4/4] unpack_trees_options: free messages when done Martin Ågren
2018-05-16 21:54                                 ` [PATCH v2 0/3] " Elijah Newren
2018-05-17 12:09                                   ` Ben Peart
2018-04-24  8:22                       ` [PATCH v8 06/16] sequencer: introduce the `merge` command Johannes Schindelin
2018-04-22 12:01               ` Philip Oakley
2018-04-23 12:03                 ` Johannes Schindelin
2018-04-23 20:34                   ` Philip Oakley
2018-04-24  8:11                     ` Johannes Schindelin
2018-04-24 19:41                       ` Philip Oakley
2018-04-22 13:55               ` Philip Oakley
2018-04-21 10:33             ` [PATCH v8 07/16] sequencer: fast-forward `merge` commands, if possible Johannes Schindelin
2018-04-21 10:34             ` [PATCH v8 08/16] rebase-helper --make-script: introduce a flag to rebase merges Johannes Schindelin
2018-04-22 13:42               ` Philip Oakley
2018-04-24  8:33                 ` Johannes Schindelin
2018-04-21 10:34             ` [PATCH v8 09/16] rebase: introduce the --rebase-merges option Johannes Schindelin
2018-04-22 14:15               ` Philip Oakley
2018-04-24  5:01                 ` Junio C Hamano
2018-04-24  9:03                   ` Johannes Schindelin
2018-04-24  8:40                 ` Johannes Schindelin
2018-04-22 14:37               ` Philip Oakley
2018-04-24 10:52                 ` Johannes Schindelin
2018-04-21 10:35             ` [PATCH v8 10/16] rebase --rebase-merges: add test for --keep-empty Johannes Schindelin
2018-04-21 10:43             ` [PATCH v8 11/16] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
2018-04-21 10:46             ` [PATCH v8 12/16] sequencer: handle post-rewrite for merge commands Johannes Schindelin
2018-04-21 10:47             ` [PATCH v8 13/16] rebase --rebase-merges: avoid "empty merges" Johannes Schindelin
2018-04-21 10:49             ` [PATCH v8 14/16] pull: accept --rebase=merges to recreate the branch topology Johannes Schindelin
2018-04-21 10:57             ` [PATCH v8 15/16] rebase -i: introduce --rebase-merges=[no-]rebase-cousins Johannes Schindelin
2018-04-21 11:09             ` [PATCH v8 16/16] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
2018-04-25 12:28             ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Johannes Schindelin
2018-04-25 12:28               ` [PATCH v9 01/17] sequencer: avoid using errno clobbered by rollback_lock_file() Johannes Schindelin
2018-04-25 12:28               ` [PATCH v9 02/17] sequencer: make rearrange_squash() a bit more obvious Johannes Schindelin
2018-04-25 12:28               ` [PATCH v9 03/17] sequencer: refactor how original todo list lines are accessed Johannes Schindelin
2018-04-25 12:28               ` [PATCH v9 04/17] sequencer: offer helpful advice when a command was rescheduled Johannes Schindelin
2018-04-25 12:28               ` [PATCH v9 05/17] git-rebase--interactive: clarify arguments Johannes Schindelin
2018-04-25 12:28               ` [PATCH v9 06/17] sequencer: introduce new commands to reset the revision Johannes Schindelin
2018-04-25 12:28               ` [PATCH v9 07/17] sequencer: introduce the `merge` command Johannes Schindelin
2018-04-25 12:28               ` [PATCH v9 08/17] sequencer: fast-forward `merge` commands, if possible Johannes Schindelin
2018-04-25 12:29               ` [PATCH v9 09/17] rebase-helper --make-script: introduce a flag to rebase merges Johannes Schindelin
2018-04-25 12:29               ` [PATCH v9 10/17] rebase: introduce the --rebase-merges option Johannes Schindelin
2018-04-25 12:29               ` [PATCH v9 11/17] rebase --rebase-merges: add test for --keep-empty Johannes Schindelin
2018-04-25 12:29               ` [PATCH v9 12/17] sequencer: make refs generated by the `label` command worktree-local Johannes Schindelin
2018-04-25 12:29               ` [PATCH v9 13/17] sequencer: handle post-rewrite for merge commands Johannes Schindelin
2018-04-25 12:29               ` [PATCH v9 14/17] rebase --rebase-merges: avoid "empty merges" Johannes Schindelin
2018-04-25 12:29               ` [PATCH v9 15/17] pull: accept --rebase=merges to recreate the branch topology Johannes Schindelin
2018-04-25 12:29               ` [PATCH v9 16/17] rebase -i: introduce --rebase-merges=[no-]rebase-cousins Johannes Schindelin
2018-04-25 12:29               ` [PATCH v9 17/17] rebase -i --rebase-merges: add a section to the man page Johannes Schindelin
2018-04-26  3:51               ` [PATCH v9 00/17] rebase -i: offer to recreate commit topology by rebasing merges Junio C Hamano
2018-04-26  6:06                 ` Junio C Hamano
2018-05-25 14:19               ` Sergey Organov
2018-05-25 15:44                 ` Sergey Organov

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.