[PATCH 00/34] Teach the sequencer to act as rebase -i's backend

[PATCH 00/34] Teach the sequencer to act as rebase -i's backend

[Johannes Schindelin]

This marks the count down to '3': two more patch series after this
(really tiny ones) and we have a faster rebase -i.

The idea of this patch series is to teach the sequencer to understand
all of the commands in `git-rebase-todo` scripts, to execute them and to
behave pretty much very the same as `git rebase -i --continue` when
called with the newly-introduced REPLAY_INTERACTIVE_REBASE setting.

Most of these patches should be pretty much straight-forward. When not,
I tried to make a point of describing enough background in the commit
message. Please feel free to point out where my explanations fall short.

Note that even after this patch series is applied, rebase -i is still
unaffected. It will require the next patch series which introduces the
rebase--helper that essentially implements `git rebase -i --continue` by
calling the sequencer with the appropriate options.

The final patch series will move a couple of pre- and post-processing
steps into the rebase--helper/sequencer (such as expanding/shrinking the
SHA-1s, reordering the fixup!/squash! lines, etc). This might sound like
a mere add-on, but it is essential for the speed improvements: those
stupid little processing steps really dominated the execution time in my
tests.

Apart from mostly cosmetic patches (and the occasional odd bug that I
fixed promptly), I used these patches since mid May to perform all of my
interactive rebases. In mid June, I had the idea to teach rebase -i to
run *both* scripted rebase and rebase--helper and to cross-validate the
results. This slowed down all my interactive rebases since, but helped
me catch three rather obscure bugs (e.g. that git commit --fixup unfolds
long onelines and rebase -i still finds the correct original commit).

This is all only to say that I am rather confident that the current code
does the job.

As stated earlier, my plan is to integrate all of these patch series
into Git for Windows v2.10.0. And then be mostly offline during the
second half of September ;-) There are multiple reasons why I want to
pull through with this: I have been working on these patches since mid
February and it is high time that they benefit users other than me.
Also: I got a bit tired of postponing the integration for a couple of
Git for Windows versions already. Originally, I had planned it for
v2.9.0, but then I wanted to give the patches some review, regretting
that idea when it took one of the 14 patch series over a month to
stabilize.

This means that I will not perform any major refactoring at this stage,
but postpone any such work, if necessary at all.

Just to reiterate why I do all this: it speeds up the interactive rebase
substantially. Even with a not yet fully builtin rebase -i, but just the
part after the user edited the `git-rebase-todo` script.

The performance test I introduced to demonstrate this (p3404) shows a
speed-up of +380% here (i.e. roughly 5x), from ~8.8 seconds to ~1.8
seconds. This is on Windows, where the performance impact of avoiding
shell scripting is most noticable.

On MacOSX and on Linux, the speed-up is less pronounced, but still
noticable, at least if you trust Travis CI, which I abused to perform
that test for me. Check for yourself (searching for "3404.2") here:
https://travis-ci.org/git/git/builds/156295227. According to those logs,
p3404 is speeded up from ~0.45 seconds to ~0.12 seconds on Linux (read:
about 3.5x) and from ~1.7 seconds to ~0.5 seconds on MacOSX (read:
almost 4x).


Johannes Schindelin (34):
  sequencer: support a new action: 'interactive rebase'
  sequencer (rebase -i): implement the 'noop' command
  sequencer (rebase -i): implement the 'edit' command
  sequencer (rebase -i): implement the 'exec' command
  sequencer (rebase -i): learn about the 'verbose' mode
  sequencer (rebase -i): write the 'done' file
  sequencer (rebase -i): add support for the 'fixup' and 'squash'
    commands
  sequencer (rebase -i): implement the short commands
  sequencer (rebase -i): write an author-script file
  sequencer (rebase -i): allow continuing with staged changes
  sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed
  sequencer (rebase -i): skip some revert/cherry-pick specific code path
  sequencer (rebase -i): the todo can be empty when continuing
  sequencer (rebase -i): update refs after a successful rebase
  sequencer (rebase -i): leave a patch upon error
  sequencer (rebase -i): implement the 'reword' command
  sequencer (rebase -i): allow fast-forwarding for edit/reword
  sequencer (rebase -i): refactor setting the reflog message
  sequencer (rebase -i): set the reflog message consistently
  sequencer (rebase -i): copy commit notes at end
  sequencer (rebase -i): record interrupted commits in rewritten, too
  sequencer (rebase -i): run the post-rewrite hook, if needed
  sequencer (rebase -i): respect the rebase.autostash setting
  sequencer (rebase -i): respect strategy/strategy_opts settings
  sequencer (rebase -i): allow rescheduling commands
  sequencer (rebase -i): implement the 'drop' command
  sequencer (rebase -i): differentiate between comments and 'noop'
  run_command_opt(): optionally hide stderr when the command succeeds
  sequencer (rebase -i): show only failed `git commit`'s output
  sequencer (rebase -i): show only failed cherry-picks' output
  sequencer (rebase -i): suggest --edit-todo upon unknown command
  sequencer (rebase -i): show the progress
  sequencer (rebase -i): write the progress into files
  sequencer (rebase -i): write out the final message

 run-command.c |  23 ++
 run-command.h |   1 +
 sequencer.c   | 990 +++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 sequencer.h   |   4 +-
 4 files changed, 971 insertions(+), 47 deletions(-)

Based-On: prepare-sequencer-and-wt_status at https://github.com/dscho/git
Fetch-Base-Via: git fetch https://github.com/dscho/git prepare-sequencer-and-wt_status
Published-As: https://github.com/dscho/git/releases/tag/sequencer-i-v1
Fetch-It-Via: git fetch https://github.com/dscho/git sequencer-i-v1

-- 
2.10.0.rc2.102.g5c102ec

base-commit: 45bfeb0c4044dfcf090ad97e839d250d62308fac

[PATCH 01/34] sequencer: support a new action: 'interactive rebase'

[Johannes Schindelin]

This patch introduces a new action for the sequencer. It really does not
do a whole lot of its own right now, but lays the ground work for
patches to come. The intention, of course, is to finally make the
sequencer the work horse of the interactive rebase (the original idea
behind the "sequencer" concept).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 32 +++++++++++++++++++++++++++++---
 sequencer.h |  3 ++-
 2 files changed, 31 insertions(+), 4 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index a89ef27..5598534 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -28,6 +28,14 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+static GIT_PATH_FUNC(rebase_path, "rebase-merge")
+/*
+ * The file containing rebase commands, comments, and empty lines.
+ * This file is created by "git rebase -i" then edited by the user. As
+ * the lines are processed, they are removed from the front of this
+ * file and written to the tail of 'done'.
+ */
+static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
 /*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
@@ -43,16 +51,20 @@ static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
-	return 0;
+	return opts->action == REPLAY_INTERACTIVE_REBASE;
 }
 
 static const char *get_dir(const struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return rebase_path();
 	return git_path_seq_dir();
 }
 
 static const char *get_todo_path(const struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return rebase_path_todo();
 	return git_path_todo_file();
 }
 
@@ -176,7 +188,15 @@ int sequencer_remove_state(struct replay_opts *opts)
 
 static const char *action_name(const struct replay_opts *opts)
 {
-	return opts->action == REPLAY_REVERT ? "revert" : "cherry-pick";
+	switch (opts->action) {
+	case REPLAY_REVERT:
+		return "revert";
+	case REPLAY_PICK:
+		return "cherry-pick";
+	case REPLAY_INTERACTIVE_REBASE:
+		return "rebase -i";
+	}
+	die("Unknown action: %d", opts->action);
 }
 
 struct commit_message {
@@ -407,7 +427,10 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
-		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
+		/*
+		 * TRANSLATORS: %s will be "revert", "cherry-pick" or
+		 * "rebase -i".
+		 */
 		return error(_("%s: Unable to write new index file"),
 			action_name(opts));
 	rollback_lock_file(&index_lock);
@@ -1186,6 +1209,9 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 	const char *todo_path = get_todo_path(opts);
 	int next = todo_list->current, offset, fd;
 
+	if (is_rebase_i(opts))
+		next++;
+
 	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
 		return error_errno(_("Could not lock '%s'"),
diff --git a/sequencer.h b/sequencer.h
index 688fff1..edd7d4a 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -7,7 +7,8 @@ const char *git_path_seq_dir(void);
 
 enum replay_action {
 	REPLAY_REVERT,
-	REPLAY_PICK
+	REPLAY_PICK,
+	REPLAY_INTERACTIVE_REBASE
 };
 
 struct replay_opts {
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 02/34] sequencer (rebase -i): implement the 'noop' command

[Johannes Schindelin]

The 'noop' command is probably the most boring of all rebase -i commands
to support in the sequencer.

Which makes it an excellent candidate for this first stab to add support
for rebase -i's commands to the sequencer.

For the moment, let's also treat empty lines and commented-out lines as
'noop'; We will refine that handling later in this patch series.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 5598534..f85dc9c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -644,12 +644,14 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 
 enum todo_command {
 	TODO_PICK,
-	TODO_REVERT
+	TODO_REVERT,
+	TODO_NOOP
 };
 
 static const char *todo_command_strings[] = {
 	"pick",
-	"revert"
+	"revert",
+	"noop"
 };
 
 static const char *command_to_string(const enum todo_command command)
@@ -916,6 +918,14 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	/* left-trim */
 	bol += strspn(bol, " \t");
 
+	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
+		item->command = TODO_NOOP;
+		item->commit = NULL;
+		item->arg = bol;
+		item->arg_len = eol - bol;
+		return 0;
+	}
+
 	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
 		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
 			item->command = i;
@@ -924,6 +934,13 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	if (i >= ARRAY_SIZE(todo_command_strings))
 		return -1;
 
+	if (item->command == TODO_NOOP) {
+		item->commit = NULL;
+		item->arg = bol;
+		item->arg_len = eol - bol;
+		return 0;
+	}
+
 	/* Eat up extra spaces/ tabs before object name */
 	padding = strspn(bol, " \t");
 	if (!padding)
@@ -1277,7 +1294,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		res = do_pick_commit(item->command, item->commit, opts);
+		if (item->command <= TODO_REVERT)
+			res = do_pick_commit(item->command, item->commit,
+					opts);
+		else if (item->command != TODO_NOOP)
+			return error("Unknown command %d", item->command);
+
 		todo_list->current++;
 		if (res)
 			return res;
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 03/34] sequencer (rebase -i): implement the 'edit' command

[Johannes Schindelin]

This patch is a straight-forward reimplementation of the `edit`
operation of the interactive rebase command.

Well, not *quite* straight-forward: when stopping, the `edit`
command wants to write the `patch` file (which is not only the
patch, but includes the commit message and author information). To
that end, this patch requires the earlier work that taught the
log-tree machinery to respect the `file` setting of
rev_info->diffopt to write to a file stream different than stdout.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 112 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index f85dc9c..fe2f6e7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -16,6 +16,7 @@
 #include "refs.h"
 #include "argv-array.h"
 #include "quote.h"
+#include "log-tree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -43,6 +44,20 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
 /*
+ * When an "edit" rebase command is being processed, the SHA1 of the
+ * commit to be edited is recorded in this file.  When "git rebase
+ * --continue" is executed, if there are any staged changes then they
+ * will be amended to the HEAD commit, but only provided the HEAD
+ * commit is still the commit to be edited.  When any other rebase
+ * command is processed, this file is deleted.
+ */
+static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
+/*
+ * When we stop at a given patch via the "edit" command, this file contains
+ * the long commit name of the corresponding patch.
+ */
+static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
+/*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
  */
@@ -645,12 +660,14 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 enum todo_command {
 	TODO_PICK,
 	TODO_REVERT,
+	TODO_EDIT,
 	TODO_NOOP
 };
 
 static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
+	"edit",
 	"noop"
 };
 
@@ -1279,9 +1296,85 @@ static int save_opts(struct replay_opts *opts)
 	return res;
 }
 
+static int make_patch(struct commit *commit, struct replay_opts *opts)
+{
+	struct strbuf buf = STRBUF_INIT;
+	struct rev_info log_tree_opt;
+	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject;
+	int res = 0;
+
+	if (write_file_gently(rebase_path_stopped_sha(),
+			      short_commit_name(commit), 1) < 0)
+		return -1;
+
+	strbuf_addf(&buf, "%s/patch", get_dir(opts));
+	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
+	init_revisions(&log_tree_opt, NULL);
+	log_tree_opt.abbrev = 0;
+	log_tree_opt.diff = 1;
+	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
+	log_tree_opt.disable_stdin = 1;
+	log_tree_opt.no_commit_id = 1;
+	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
+	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
+	if (!log_tree_opt.diffopt.file)
+		res |= error_errno("could not open '%s'", buf.buf);
+	else {
+		res |= log_tree_commit(&log_tree_opt, commit);
+		fclose(log_tree_opt.diffopt.file);
+	}
+	strbuf_reset(&buf);
+
+	strbuf_addf(&buf, "%s/message", get_dir(opts));
+	if (!file_exists(buf.buf)) {
+		find_commit_subject(commit_buffer, &subject);
+		res |= write_file_gently(buf.buf, subject, 1);
+		unuse_commit_buffer(commit, commit_buffer);
+	}
+	strbuf_release(&buf);
+
+	return res;
+}
+
+static int intend_to_amend(void)
+{
+	unsigned char head[20];
+
+	if (get_sha1("HEAD", head))
+		return error("Cannot read HEAD");
+
+	return write_file_gently(rebase_path_amend(), sha1_to_hex(head), 1);
+}
+
+static int error_with_patch(struct commit *commit,
+	const char *subject, int subject_len,
+	struct replay_opts *opts, int exit_code, int to_amend)
+{
+	if (make_patch(commit, opts))
+		return -1;
+
+	if (to_amend) {
+		if (intend_to_amend())
+			return -1;
+
+		fprintf(stderr, "You can amend the commit now, with\n"
+			"\n"
+			"  git commit --amend %s\n"
+			"\n"
+			"Once you are satisfied with your changes, run\n"
+			"\n"
+			"  git rebase --continue\n", gpg_sign_opt_quoted(opts));
+	}
+	else if (exit_code)
+		fprintf(stderr, "Could not apply %s... %.*s\n",
+			short_commit_name(commit), subject_len, subject);
+
+	return exit_code;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
-	int res;
+	int res = 0;
 
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	if (opts->allow_ff)
@@ -1294,9 +1387,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		if (item->command <= TODO_REVERT)
+		if (item->command <= TODO_EDIT) {
 			res = do_pick_commit(item->command, item->commit,
 					opts);
+			if (item->command == TODO_EDIT) {
+				struct commit *commit = item->commit;
+				if (!res)
+					warning("Stopped at %s... %.*s",
+						short_commit_name(commit),
+						item->arg_len, item->arg);
+				return error_with_patch(commit,
+					item->arg, item->arg_len, opts, res,
+					!res);
+			}
+		}
 		else if (item->command != TODO_NOOP)
 			return error("Unknown command %d", item->command);
 
@@ -1305,6 +1409,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			return res;
 	}
 
+	if (is_rebase_i(opts)) {
+		/* Stopped in the middle, as planned? */
+		if (todo_list->current < todo_list->nr)
+			return 0;
+	}
+
 	/*
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 04/34] sequencer (rebase -i): implement the 'exec' command

[Johannes Schindelin]

The 'exec' command is a little special among rebase -i's commands, as it
does *not* have a SHA-1 as first parameter. Instead, everything after the
`exec` command is treated as command-line to execute.

Let's reuse the arg/arg_len fields of the todo_item structure (which hold
the oneline for pick/edit commands) to point to the command-line.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index fe2f6e7..a58bb91 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -17,6 +17,7 @@
 #include "argv-array.h"
 #include "quote.h"
 #include "log-tree.h"
+#include "wt-status.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -661,6 +662,7 @@ enum todo_command {
 	TODO_PICK,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_EXEC,
 	TODO_NOOP
 };
 
@@ -668,6 +670,7 @@ static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
 	"edit",
+	"exec",
 	"noop"
 };
 
@@ -964,6 +967,12 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return -1;
 	bol += padding;
 
+	if (item->command == TODO_EXEC) {
+		item->arg = bol;
+		item->arg_len = (int)(eol - bol);
+		return 0;
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
@@ -1372,6 +1381,47 @@ static int error_with_patch(struct commit *commit,
 	return exit_code;
 }
 
+static int do_exec(const char *command_line)
+{
+	const char *child_argv[] = { NULL, NULL };
+	int dirty, status;
+
+	fprintf(stderr, "Executing: %s\n", command_line);
+	child_argv[0] = command_line;
+	status = run_command_v_opt(child_argv, RUN_USING_SHELL);
+
+	/* force re-reading of the cache */
+	if (discard_cache() < 0 || read_cache() < 0)
+		return error(_("Could not read index"));
+
+	dirty = require_clean_work_tree("rebase", NULL, 1, 1);
+
+	if (status) {
+		warning("Execution failed: %s\n%s"
+			"You can fix the problem, and then run\n"
+			"\n"
+			"  git rebase --continue\n"
+			"\n",
+			command_line,
+			dirty ? "and made changes to the index and/or the "
+				"working tree\n" : "");
+		if (status == 127)
+			/* command not found */
+			status = 1;
+	}
+	else if (dirty) {
+		warning("Execution succeeded: %s\nbut "
+			"left changes to the index and/or the working tree\n"
+			"Commit or stash your changes, and then run\n"
+			"\n"
+			"  git rebase --continue\n"
+			"\n", command_line);
+		status = 1;
+	}
+
+	return status;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1401,6 +1451,14 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					!res);
 			}
 		}
+		else if (item->command == TODO_EXEC) {
+			char *end_of_arg = (char *)(item->arg + item->arg_len);
+			int saved = *end_of_arg;
+
+			*end_of_arg = '\0';
+			res = do_exec(item->arg);
+			*end_of_arg = saved;
+		}
 		else if (item->command != TODO_NOOP)
 			return error("Unknown command %d", item->command);
 
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 05/34] sequencer (rebase -i): learn about the 'verbose' mode

[Johannes Schindelin]

When calling `git rebase -i -v`, the user wants to see some statistics
after the commits were rebased. Let's show some.

The strbuf we use to perform that task will be used for other things
in subsequent commits, hence it is declared and initialized in a wider
scope than strictly needed here.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 22 ++++++++++++++++++++++
 sequencer.h |  1 +
 2 files changed, 23 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index a58bb91..95c31bb 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -63,6 +63,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
  * command-line (and are only consumed, not modified, by the sequencer).
  */
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
+static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
+static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
@@ -1096,6 +1098,9 @@ static int read_populate_opts(struct replay_opts *opts)
 			}
 		}
 
+		if (file_exists(rebase_path_verbose()))
+			opts->verbose = 1;
+
 		return 0;
 	}
 
@@ -1468,9 +1473,26 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	}
 
 	if (is_rebase_i(opts)) {
+		struct strbuf buf = STRBUF_INIT;
+
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
 			return 0;
+
+		if (opts->verbose) {
+			const char *argv[] = {
+				"diff-tree", "--stat", NULL, NULL
+			};
+
+			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
+				return error("Could not read %s",
+					rebase_path_orig_head());
+			strbuf_addstr(&buf, "..HEAD");
+			argv[2] = buf.buf;
+			run_command_v_opt(argv, RUN_GIT_CMD);
+			strbuf_reset(&buf);
+		}
+		strbuf_release(&buf);
 	}
 
 	/*
diff --git a/sequencer.h b/sequencer.h
index edd7d4a..fd2a719 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -24,6 +24,7 @@ struct replay_opts {
 	int allow_empty;
 	int allow_empty_message;
 	int keep_redundant_commits;
+	int verbose;
 
 	int mainline;
 
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 06/34] sequencer (rebase -i): write the 'done' file

[Johannes Schindelin]

In the interactive rebase, commands that were successfully processed are
not simply discarded, but appended to the 'done' file instead. This is
used e.g. to display the current state to the user in the output of
`git status` or the progress.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 95c31bb..3bff3d9 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -39,6 +39,12 @@ static GIT_PATH_FUNC(rebase_path, "rebase-merge")
  */
 static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
 /*
+ * The rebase command lines that have already been processed. A line
+ * is moved here when it is first handled, before any associated user
+ * actions.
+ */
+static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
+/*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
  * being rebased.
@@ -1272,6 +1278,20 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 			todo_path, strerror(errno));
 	if (commit_lock_file(&todo_lock) < 0)
 		return error(_("Error wrapping up %s."), todo_path);
+
+	if (is_rebase_i(opts)) {
+		const char *done_path = rebase_path_done();
+		int fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
+		int prev_offset = !next ? 0 :
+			todo_list->items[next - 1].offset_in_buf;
+
+		if (offset > prev_offset && write_in_full(fd,
+				todo_list->buf.buf + prev_offset,
+				offset - prev_offset) < 0)
+			return error(_("Could not write to %s (%s)"),
+				done_path, strerror(errno));
+		close(fd);
+	}
 	return 0;
 }
 
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Johannes Schindelin]

This is a huge patch, and at the same time a huge step forward to
execute the performance-critical parts of the interactive rebase in a
builtin command.

Since 'fixup' and 'squash' are not only similar, but also need to know
about each other (we want to reduce a series of fixups/squashes into a
single, final commit message edit, from the user's point of view), we
really have to implement them both at the same time.

Most of the actual work is done by the existing code path that already
handles the "pick" and the "edit" commands; We added support for other
features (e.g. to amend the commit message) in the patches leading up to
this one, yet there are still quite a few bits in this patch that simply
would not make sense as individual patches (such as: determining whether
there was anything to "fix up" in the "todo" script, etc).

In theory, it would be possible to reuse the fast-forward code path also
for the fixup and the squash code paths, but in practice this would make
the code less readable. The end result cannot be fast-forwarded anyway,
therefore let's just extend the cherry-picking code path for now.

Since the sequencer parses the entire `git-rebase-todo` script in one go,
fixup or squash commands without a preceding pick can be reported early
(in git-rebase--interactive, we could only report such errors just before
executing the fixup/squash).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 242 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 231 insertions(+), 11 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 3bff3d9..3ae5978 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -45,6 +45,35 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
 /*
+ * The commit message that is planned to be used for any changes that
+ * need to be committed following a user interaction.
+ */
+static GIT_PATH_FUNC(rebase_path_message, "rebase-merge/message")
+/*
+ * The file into which is accumulated the suggested commit message for
+ * squash/fixup commands. When the first of a series of squash/fixups
+ * is seen, the file is created and the commit message from the
+ * previous commit and from the first squash/fixup commit are written
+ * to it. The commit message for each subsequent squash/fixup commit
+ * is appended to the file as it is processed.
+ *
+ * The first line of the file is of the form
+ *     # This is a combination of $count commits.
+ * where $count is the number of commits whose messages have been
+ * written to the file so far (including the initial "pick" commit).
+ * Each time that a commit message is processed, this line is read and
+ * updated. It is deleted just before the combined commit is made.
+ */
+static GIT_PATH_FUNC(rebase_path_squash_msg, "rebase-merge/message-squash")
+/*
+ * If the current series of squash/fixups has not yet included a squash
+ * command, then this file exists and holds the commit message of the
+ * original "pick" commit.  (If the series ends without a "squash"
+ * command, then this can be used as the commit message of the combined
+ * commit without opening the editor.)
+ */
+static GIT_PATH_FUNC(rebase_path_fixup_msg, "rebase-merge/message-fixup")
+/*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
  * being rebased.
@@ -670,6 +699,8 @@ enum todo_command {
 	TODO_PICK,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_FIXUP,
+	TODO_SQUASH,
 	TODO_EXEC,
 	TODO_NOOP
 };
@@ -678,6 +709,8 @@ static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
 	"edit",
+	"fixup",
+	"squash",
 	"exec",
 	"noop"
 };
@@ -689,16 +722,129 @@ static const char *command_to_string(const enum todo_command command)
 	die("Unknown command: %d", command);
 }
 
+static int is_fixup(enum todo_command command)
+{
+	return command == TODO_FIXUP || command == TODO_SQUASH;
+}
+
+static const char *nth_for_number(int n)
+{
+	int n1 = n % 10, n10 = n % 100;
+
+	if (n1 == 1 && n10 != 11)
+		return "st";
+	if (n1 == 2 && n10 != 12)
+		return "nd";
+	if (n1 == 3 && n10 != 13)
+		return "rd";
+	return "th";
+}
+
+static int update_squash_messages(enum todo_command command,
+		struct commit *commit, struct replay_opts *opts)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int count;
+	const char *message, *body;
+
+	if (file_exists(rebase_path_squash_msg())) {
+		char *p, *p2;
+
+		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
+			return error("Could not read %s",
+				rebase_path_squash_msg());
+
+		if (buf.buf[0] == '\n' || !skip_prefix(buf.buf + 1,
+				" This is a combination of ",
+				(const char **)&p))
+			return error("Unexpected 1st line of squash message:\n"
+				"\n\t%.*s",
+				(int)(strchrnul(buf.buf, '\n') - buf.buf),
+				buf.buf);
+		count = strtol(p, &p2, 10);
+
+		if (count < 1 || *p2 != ' ')
+			return error("Invalid 1st line of squash message:\n"
+				"\n\t%.*s",
+				(int)(strchrnul(buf.buf, '\n') - buf.buf),
+				buf.buf);
+
+		sprintf((char *)p, "%d", ++count);
+		if (!*p2)
+			*p2 = ' ';
+		else {
+			*(++p2) = 'c';
+			strbuf_insert(&buf, p2 - buf.buf, " ", 1);
+		}
+	}
+	else {
+		unsigned char head[20];
+		struct commit *head_commit;
+		const char *head_message, *body;
+
+		if (get_sha1("HEAD", head))
+			return error("Need a HEAD to fixup");
+		if (!(head_commit = lookup_commit_reference(head)))
+			return error("Could not read HEAD");
+		if (!(head_message = get_commit_buffer(head_commit, NULL)))
+			return error("Could not read HEAD's commit message");
+
+		body = strstr(head_message, "\n\n");
+		if (!body)
+			body = "";
+		else
+			body = skip_blank_lines(body + 2);
+		if (write_file_gently(rebase_path_fixup_msg(), body, 0))
+			return error("Cannot write %s",
+				rebase_path_fixup_msg());
+
+		count = 2;
+		strbuf_addf(&buf, "%c This is a combination of 2 commits.\n"
+			"%c The first commit's message is:\n\n%s",
+			comment_line_char, comment_line_char, body);
+
+		unuse_commit_buffer(head_commit, head_message);
+	}
+
+	if (!(message = get_commit_buffer(commit, NULL)))
+		return error("Could not read commit message of %s",
+			oid_to_hex(&commit->object.oid));
+	body = strstr(message, "\n\n");
+	if (!body)
+		body = "";
+	else
+		body = skip_blank_lines(body + 2);
+
+	if (command == TODO_SQUASH) {
+		unlink(rebase_path_fixup_msg());
+		strbuf_addf(&buf, "\n%c This is the %d%s commit message:\n\n%s",
+			comment_line_char,
+			count, nth_for_number(count), body);
+	}
+	else if (command == TODO_FIXUP) {
+		strbuf_addf(&buf,
+			"\n%c The %d%s commit message will be skipped:\n\n",
+			comment_line_char, count, nth_for_number(count));
+		strbuf_add_commented_lines(&buf, body, strlen(body));
+	}
+	else
+		return error("Unknown command: %d", command);
+	unuse_commit_buffer(commit, message);
+
+	return write_message(&buf, rebase_path_squash_msg());
+}
 
 static int do_pick_commit(enum todo_command command, struct commit *commit,
-		struct replay_opts *opts)
+		struct replay_opts *opts, int final_fixup)
 {
+	int edit = opts->edit, cleanup_commit_message = 0;
+	const char *msg_file = edit ? NULL : git_path_merge_msg();
 	unsigned char head[20];
 	struct commit *base, *next, *parent;
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res = 0, unborn = 0, allow;
+	int res = 0, unborn = 0, amend = 0, allow;
 
 	if (opts->no_commit) {
 		/*
@@ -744,7 +890,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	else
 		parent = commit->parents->item;
 
-	if (opts->allow_ff &&
+	if (opts->allow_ff && !is_fixup(command) &&
 	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
 	     (!parent && unborn)))
 		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
@@ -808,6 +954,28 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
+	if (is_fixup(command)) {
+		if (update_squash_messages(command, commit, opts))
+			return -1;
+		amend = 1;
+		if (!final_fixup)
+			msg_file = rebase_path_squash_msg();
+		else if (file_exists(rebase_path_fixup_msg())) {
+			cleanup_commit_message = 1;
+			msg_file = rebase_path_fixup_msg();
+		}
+		else {
+			const char *dest = git_path("SQUASH_MSG");
+			unlink(dest);
+			if (copy_file(dest, rebase_path_squash_msg(), 0666))
+				return error("Could not rename %s to "
+					"%s", rebase_path_squash_msg(), dest);
+			unlink(git_path("MERGE_MSG"));
+			msg_file = dest;
+			edit = 1;
+		}
+	}
+
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res |= do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
@@ -859,9 +1027,13 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
-		res |= sequencer_commit(opts->edit ?
-				NULL : git_path_merge_msg(),
-			opts, allow, opts->edit, 0, 0);
+		res |= sequencer_commit(msg_file, opts, allow, edit, amend,
+			cleanup_commit_message);
+
+	if (!res && final_fixup) {
+		unlink(rebase_path_fixup_msg());
+		unlink(rebase_path_squash_msg());
+	}
 
 leave:
 	free_message(commit, &msg);
@@ -1001,7 +1173,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
 	struct todo_item *item;
 	char *p = buf;
-	int i, res = 0;
+	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
 
 	for (i = 1; *p; i++) {
 		char *eol = strchrnul(p, '\n');
@@ -1011,8 +1183,15 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		if (parse_insn_line(item, p, eol)) {
 			res |= error("Invalid line %d: %.*s",
 				i, (int)(eol - p), p);
-			item->command = -1;
+			item->command = TODO_NOOP;
 		}
+		if (fixup_okay)
+			; /* do nothing */
+		else if (is_fixup(item->command))
+			return error("Cannot '%s' without a previous commit",
+				command_to_string(item->command));
+		else if (item->command != TODO_NOOP)
+			fixup_okay = 1;
 		p = *eol ? eol + 1 : eol;
 	}
 	if (!todo_list->nr)
@@ -1406,6 +1585,20 @@ static int error_with_patch(struct commit *commit,
 	return exit_code;
 }
 
+static int error_failed_squash(struct commit *commit,
+	struct replay_opts *opts, int subject_len, const char *subject)
+{
+	if (rename(rebase_path_squash_msg(), rebase_path_message()))
+		return error("Could not rename %s to %s",
+			rebase_path_squash_msg(), rebase_path_message());
+	unlink(rebase_path_fixup_msg());
+	unlink(git_path("MERGE_MSG"));
+	if (copy_file(git_path("MERGE_MSG"), rebase_path_message(), 0666))
+		return error("Could not copy %s to %s", rebase_path_message(),
+			git_path("MERGE_MSG"));
+	return error_with_patch(commit, subject, subject_len, opts, 1, 0);
+}
+
 static int do_exec(const char *command_line)
 {
 	const char *child_argv[] = { NULL, NULL };
@@ -1447,6 +1640,21 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int is_final_fixup(struct todo_list *todo_list)
+{
+	int i = todo_list->current;
+
+	if (!is_fixup(todo_list->items[i].command))
+		return 0;
+
+	while (++i < todo_list->nr)
+		if (is_fixup(todo_list->items[i].command))
+			return 0;
+		else if (todo_list->items[i].command < TODO_NOOP)
+			break;
+	return 1;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1462,9 +1670,15 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		if (item->command <= TODO_EDIT) {
+		if (is_rebase_i(opts)) {
+			unlink(rebase_path_message());
+			unlink(rebase_path_author_script());
+			unlink(rebase_path_stopped_sha());
+			unlink(rebase_path_amend());
+		}
+		if (item->command <= TODO_SQUASH) {
 			res = do_pick_commit(item->command, item->commit,
-					opts);
+					opts, is_final_fixup(todo_list));
 			if (item->command == TODO_EDIT) {
 				struct commit *commit = item->commit;
 				if (!res)
@@ -1475,6 +1689,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+			if (res && is_fixup(item->command)) {
+				if (res == 1)
+					intend_to_amend();
+				return error_failed_squash(item->commit, opts,
+					item->arg_len, item->arg);
+			}
 		}
 		else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
@@ -1565,7 +1785,7 @@ static int single_pick(struct commit *cmit, struct replay_opts *opts)
 {
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	return do_pick_commit(opts->action == REPLAY_PICK ?
-		TODO_PICK : TODO_REVERT, cmit, opts);
+		TODO_PICK : TODO_REVERT, cmit, opts, 0);
 }
 
 int sequencer_pick_revisions(struct replay_opts *opts)
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 08/34] sequencer (rebase -i): implement the short commands

[Johannes Schindelin]

For users' convenience, most rebase commands can be abbreviated, e.g.
'p' instead of 'pick' and 'x' instead of 'exec'. Let's teach the
sequencer to handle those abbreviated commands just fine.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 34 +++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 13 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 3ae5978..919698b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -705,20 +705,23 @@ enum todo_command {
 	TODO_NOOP
 };
 
-static const char *todo_command_strings[] = {
-	"pick",
-	"revert",
-	"edit",
-	"fixup",
-	"squash",
-	"exec",
-	"noop"
+static struct {
+	char c;
+	const char *str;
+} todo_command_info[] = {
+	{ 'p', "pick" },
+	{ 0,   "revert" },
+	{ 'e', "edit" },
+	{ 'f', "fixup" },
+	{ 's', "squash" },
+	{ 'x', "exec" },
+	{ 0,   "noop" }
 };
 
 static const char *command_to_string(const enum todo_command command)
 {
-	if (command < ARRAY_SIZE(todo_command_strings))
-		return todo_command_strings[command];
+	if (command < ARRAY_SIZE(todo_command_info))
+		return todo_command_info[command].str;
 	die("Unknown command: %d", command);
 }
 
@@ -1126,12 +1129,17 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return 0;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
-		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
+	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)
+		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
 			item->command = i;
 			break;
 		}
-	if (i >= ARRAY_SIZE(todo_command_strings))
+		else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
+			bol++;
+			item->command = i;
+			break;
+		}
+	if (i >= ARRAY_SIZE(todo_command_info))
 		return -1;
 
 	if (item->command == TODO_NOOP) {
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 09/34] sequencer (rebase -i): write an author-script file

[Johannes Schindelin]

When the interactive rebase aborts, it writes out an author-script file
to record the author information for the current commit. As we are about
to teach the sequencer how to perform the actions behind an interactive
rebase, it needs to write those author-script files, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 48 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 919698b..ef818a2 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -528,6 +528,51 @@ static int is_index_unchanged(void)
 	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
 }
 
+static int write_author_script(const char *message)
+{
+	struct strbuf buf = STRBUF_INIT;
+	const char *eol;
+
+	for (;;)
+		if (!*message || starts_with(message, "\n")) {
+missing_author:
+			/* Missing 'author' line? */
+			unlink(rebase_path_author_script());
+			return 0;
+		}
+		else if (skip_prefix(message, "author ", &message))
+			break;
+		else if ((eol = strchr(message, '\n')))
+			message = eol + 1;
+		else
+			goto missing_author;
+
+	strbuf_addstr(&buf, "GIT_AUTHOR_NAME='");
+	while (*message && *message != '\n' && *message != '\r')
+		if (skip_prefix(message, " <", &message))
+			break;
+		else if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\nGIT_AUTHOR_EMAIL='");
+	while (*message && *message != '\n' && *message != '\r')
+		if (skip_prefix(message, "> ", &message))
+			break;
+		else if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\nGIT_AUTHOR_DATE='@");
+	while (*message && *message != '\n' && *message != '\r')
+		if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\n");
+	return write_message(&buf, rebase_path_author_script());
+}
+
 static char **read_author_script(void)
 {
 	struct strbuf script = STRBUF_INIT;
@@ -979,7 +1024,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
-	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
+	if (is_rebase_i(opts) && write_author_script(msg.message) < 0)
+		res |= -1;
+	else if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res |= do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
 		if (res < 0)
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 10/34] sequencer (rebase -i): allow continuing with staged changes

[Johannes Schindelin]

When an interactive rebase is interrupted, the user may stage changes
before continuing, and we need to commit those changes in that case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index ef818a2..4ddb12e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1807,6 +1807,41 @@ static int continue_single_pick(void)
 	return run_command_v_opt(argv, RUN_GIT_CMD);
 }
 
+static int commit_staged_changes(struct replay_opts *opts)
+{
+	int amend = 0;
+
+	if (has_unstaged_changes(1))
+		return error(_("Cannot rebase: You have unstaged changes."));
+	if (!has_uncommitted_changes(0))
+		return 0;
+
+	if (file_exists(rebase_path_amend())) {
+		struct strbuf rev = STRBUF_INIT;
+		unsigned char head[20], to_amend[20];
+
+		if (get_sha1("HEAD", head))
+			return error("Cannot amend non-existing commit");
+		if (!read_oneliner(&rev, rebase_path_amend(), 0))
+			return error("Invalid file: %s", rebase_path_amend());
+		if (get_sha1_hex(rev.buf, to_amend))
+			return error("Invalid contents: %s",
+				rebase_path_amend());
+		if (hashcmp(head, to_amend))
+			return error("\nYou have uncommitted changes in your "
+				"working tree. Please, commit them\nfirst and "
+				"then run 'git rebase --continue' again.");
+
+		strbuf_release(&rev);
+		amend = 1;
+	}
+
+	if (sequencer_commit(rebase_path_message(), opts, 1, 1, amend, 0))
+		return error("Could not commit staged changes.");
+	unlink(rebase_path_amend());
+	return 0;
+}
+
 int sequencer_continue(struct replay_opts *opts)
 {
 	struct todo_list todo_list = TODO_LIST_INIT;
@@ -1815,6 +1850,10 @@ int sequencer_continue(struct replay_opts *opts)
 	if (read_and_refresh_cache(opts))
 		return -1;
 
+	if (is_rebase_i(opts)) {
+		if (commit_staged_changes(opts))
+			return -1;
+	}
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts) ||
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 11/34] sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed

[Johannes Schindelin]

The scripted version of the interactive rebase already does that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 4ddb12e..683b89f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1813,8 +1813,13 @@ static int commit_staged_changes(struct replay_opts *opts)
 
 	if (has_unstaged_changes(1))
 		return error(_("Cannot rebase: You have unstaged changes."));
-	if (!has_uncommitted_changes(0))
+	if (!has_uncommitted_changes(0)) {
+		const char *cherry_pick_head = git_path("CHERRY_PICK_HEAD");
+
+		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
+			return error("Could not remove CHERRY_PICK_HEAD");
 		return 0;
+	}
 
 	if (file_exists(rebase_path_amend())) {
 		struct strbuf rev = STRBUF_INIT;
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 12/34] sequencer (rebase -i): skip some revert/cherry-pick specific code path

[Johannes Schindelin]

When a cherry-pick continues without a "todo script", the intention is
simply to pick a single commit.

However, when an interactive rebase is continued without a "todo
script", it means that the last command has been completed and that we
now need to clean up.

This commit guards the revert/cherry-pick specific steps so that they
are not executed in rebase -i mode.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 23 +++++++++++++----------
 1 file changed, 13 insertions(+), 10 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 683b89f..7f017a8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1859,22 +1859,25 @@ int sequencer_continue(struct replay_opts *opts)
 		if (commit_staged_changes(opts))
 			return -1;
 	}
-	if (!file_exists(get_todo_path(opts)))
+	else if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts) ||
 			read_populate_todo(&todo_list, opts))
 		return -1;
 
-	/* Verify that the conflict has been resolved */
-	if (file_exists(git_path_cherry_pick_head()) ||
-	    file_exists(git_path_revert_head())) {
-		int ret = continue_single_pick();
-		if (ret)
-			return ret;
+	if (!is_rebase_i(opts)) {
+		/* Verify that the conflict has been resolved */
+		if (file_exists(git_path_cherry_pick_head()) ||
+		    file_exists(git_path_revert_head())) {
+			int ret = continue_single_pick();
+			if (ret)
+				return ret;
+		}
+		if (index_differs_from("HEAD", 0))
+			return error_dirty_index(opts);
+		todo_list.current++;
 	}
-	if (index_differs_from("HEAD", 0))
-		return error_dirty_index(opts);
-	todo_list.current++;
+
 	res = pick_commits(&todo_list, opts);
 	todo_list_release(&todo_list);
 	return res;
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 13/34] sequencer (rebase -i): the todo can be empty when continuing

[Johannes Schindelin]

When the last command of an interactive rebase fails, the user needs to
resolve the problem and then continue the interactive rebase. Naturally,
the todo script is empty by then. So let's not complain about that!

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 7f017a8..d4437f5 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1249,8 +1249,6 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 			fixup_okay = 1;
 		p = *eol ? eol + 1 : eol;
 	}
-	if (!todo_list->nr)
-		return error(_("No commits parsed."));
 	return res;
 }
 
@@ -1273,6 +1271,9 @@ static int read_populate_todo(struct todo_list *todo_list,
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
 	if (res)
 		return error(_("Unusable instruction sheet: %s"), todo_file);
+	if (!todo_list->nr &&
+	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
+		return error(_("No commits parsed."));
 	return 0;
 }
 
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 14/34] sequencer (rebase -i): update refs after a successful rebase

[Johannes Schindelin]

An interactive rebase operates on a detached HEAD (to keep the reflog
of the original branch relatively clean), and updates the branch only
at the end.

Now that the sequencer learns to perform interactive rebases, it also
needs to learn the trick to update the branch before removing the
directory containing the state of the interactive rebase.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 32 +++++++++++++++++++++++++++++++-
 1 file changed, 31 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index d4437f5..7662222 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -100,6 +100,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
 static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
+static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
+static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
 
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
@@ -1769,12 +1771,39 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	}
 
 	if (is_rebase_i(opts)) {
-		struct strbuf buf = STRBUF_INIT;
+		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
 
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
 			return 0;
 
+		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
+				starts_with(head_ref.buf, "refs/")) {
+			unsigned char head[20], orig[20];
+
+			if (get_sha1("HEAD", head))
+				return error("Cannot read HEAD");
+			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
+					get_sha1_hex(buf.buf, orig))
+				return error("Could not read orig-head");
+			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
+				head_ref.buf);
+			if (!read_oneliner(&buf, rebase_path_onto(), 0))
+				return error("Could not read 'onto'");
+			if (update_ref(buf.buf, head_ref.buf, head, orig,
+					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
+				return error("Could not update %s",
+					head_ref.buf);
+			strbuf_reset(&buf);
+			strbuf_addf(&buf,
+				"rebase -i (finish): returning to %s",
+				head_ref.buf);
+			if (create_symref("HEAD", head_ref.buf, buf.buf))
+				return error("Could not update HEAD to %s",
+					head_ref.buf);
+			strbuf_reset(&buf);
+		}
+
 		if (opts->verbose) {
 			const char *argv[] = {
 				"diff-tree", "--stat", NULL, NULL
@@ -1789,6 +1818,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			strbuf_reset(&buf);
 		}
 		strbuf_release(&buf);
+		strbuf_release(&head_ref);
 	}
 
 	/*
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 15/34] sequencer (rebase -i): leave a patch upon error

[Johannes Schindelin]

Just like the interactive rebase, we want to leave a 'patch' file for
further inspection by the user (even if we never tried to actually apply
that patch, since we're cherry-picking instead).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 7662222..9a06b40 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1753,6 +1753,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				return error_failed_squash(item->commit, opts,
 					item->arg_len, item->arg);
 			}
+			else if (res && is_rebase_i(opts))
+				return res | error_with_patch(item->commit,
+					item->arg, item->arg_len, opts, res, 0);
 		}
 		else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 16/34] sequencer (rebase -i): implement the 'reword' command

[Johannes Schindelin]

This is now trivial, as all the building blocks are in place: all we need
to do is to flip the "edit" switch when committing.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 9a06b40..64fec47 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -746,6 +746,7 @@ enum todo_command {
 	TODO_PICK,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_REWORD,
 	TODO_FIXUP,
 	TODO_SQUASH,
 	TODO_EXEC,
@@ -759,6 +760,7 @@ static struct {
 	{ 'p', "pick" },
 	{ 0,   "revert" },
 	{ 'e', "edit" },
+	{ 'r', "reword" },
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
@@ -1004,7 +1006,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
-	if (is_fixup(command)) {
+	if (command == TODO_REWORD)
+		edit = 1;
+	else if (is_fixup(command)) {
 		if (update_squash_messages(command, commit, opts))
 			return -1;
 		amend = 1;
@@ -1755,7 +1759,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			}
 			else if (res && is_rebase_i(opts))
 				return res | error_with_patch(item->commit,
-					item->arg, item->arg_len, opts, res, 0);
+					item->arg, item->arg_len, opts, res,
+					item->command == TODO_REWORD);
 		}
 		else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 17/34] sequencer (rebase -i): allow fast-forwarding for edit/reword

[Johannes Schindelin]

The sequencer already knew how to fast-forward instead of
cherry-picking, if possible.

We want to continue to do this, of course, but in case of the 'reword'
command, we will need to call `git commit` after fast-forwarding.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 64fec47..399fc8d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -896,7 +896,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res = 0, unborn = 0, amend = 0, allow;
+	int res = 0, unborn = 0, amend = 0, allow = 0;
 
 	if (opts->no_commit) {
 		/*
@@ -942,11 +942,23 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	else
 		parent = commit->parents->item;
 
+	if (get_message(commit, &msg) != 0)
+		return error(_("Cannot get commit message for %s"),
+			oid_to_hex(&commit->object.oid));
+
 	if (opts->allow_ff && !is_fixup(command) &&
 	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
-	     (!parent && unborn)))
-		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
-
+	     (!parent && unborn))) {
+		if (is_rebase_i(opts))
+			write_author_script(msg.message);
+		res |= fast_forward_to(commit->object.oid.hash, head, unborn,
+			opts);
+		if (res || command != TODO_REWORD)
+			goto leave;
+		edit = amend = 1;
+		msg_file = NULL;
+		goto fast_forward_edit;
+	}
 	if (parent && parse_commit(parent) < 0)
 		/* TRANSLATORS: The first %s will be "revert" or
 		   "cherry-pick", the second %s a SHA1 */
@@ -954,10 +966,6 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 			command_to_string(command),
 			oid_to_hex(&parent->object.oid));
 
-	if (get_message(commit, &msg) != 0)
-		return error(_("Cannot get commit message for %s"),
-			oid_to_hex(&commit->object.oid));
-
 	/*
 	 * "commit" is an existing commit.  We would want to apply
 	 * the difference it introduces since its first parent "prev"
@@ -1083,6 +1091,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
+fast_forward_edit:
 		res |= sequencer_commit(msg_file, opts, allow, edit, amend,
 			cleanup_commit_message);
 
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 18/34] sequencer (rebase -i): refactor setting the reflog message

[Johannes Schindelin]

This makes the code DRYer, with the obvious benefit that we can enhance
the code further in a single place.

We can also reuse the functionality elsewhere by calling this new
function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 ++++++++++++++++++++++++++-------
 1 file changed, 26 insertions(+), 7 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 399fc8d..71730e6 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1726,6 +1726,26 @@ static int is_final_fixup(struct todo_list *todo_list)
 	return 1;
 }
 
+static const char *reflog_message(struct replay_opts *opts,
+	const char *sub_action, const char *fmt, ...)
+{
+	va_list ap;
+	static struct strbuf buf = STRBUF_INIT;
+
+	va_start(ap, fmt);
+	strbuf_reset(&buf);
+	strbuf_addstr(&buf, action_name(opts));
+	if (sub_action)
+		strbuf_addf(&buf, " (%s)", sub_action);
+	if (fmt) {
+		strbuf_addstr(&buf, ": ");
+		strbuf_vaddf(&buf, fmt, ap);
+	}
+	va_end(ap);
+
+	return buf.buf;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1796,6 +1816,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 
 		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
 				starts_with(head_ref.buf, "refs/")) {
+			const char *msg;
 			unsigned char head[20], orig[20];
 
 			if (get_sha1("HEAD", head))
@@ -1803,19 +1824,17 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
 					get_sha1_hex(buf.buf, orig))
 				return error("Could not read orig-head");
-			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
-				head_ref.buf);
 			if (!read_oneliner(&buf, rebase_path_onto(), 0))
 				return error("Could not read 'onto'");
-			if (update_ref(buf.buf, head_ref.buf, head, orig,
+			msg = reflog_message(opts, "finish", "%s onto %s",
+				head_ref.buf, buf.buf);
+			if (update_ref(msg, head_ref.buf, head, orig,
 					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 				return error("Could not update %s",
 					head_ref.buf);
-			strbuf_reset(&buf);
-			strbuf_addf(&buf,
-				"rebase -i (finish): returning to %s",
+			msg = reflog_message(opts, "finish", "returning to %s",
 				head_ref.buf);
-			if (create_symref("HEAD", head_ref.buf, buf.buf))
+			if (create_symref("HEAD", head_ref.buf, msg))
 				return error("Could not update HEAD to %s",
 					head_ref.buf);
 			strbuf_reset(&buf);
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 19/34] sequencer (rebase -i): set the reflog message consistently

[Johannes Schindelin]

We already used the same reflog message as the scripted version of rebase
-i when finishing. With this commit, we do that also for all the commands
before that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 71730e6..54f654c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1768,6 +1768,10 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			unlink(rebase_path_amend());
 		}
 		if (item->command <= TODO_SQUASH) {
+			if (is_rebase_i(opts))
+				setenv("GIT_REFLOG_ACTION", reflog_message(opts,
+					command_to_string(item->command), NULL),
+					1);
 			res = do_pick_commit(item->command, item->commit,
 					opts, is_final_fixup(todo_list));
 			if (item->command == TODO_EDIT) {
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 20/34] sequencer (rebase -i): copy commit notes at end

[Johannes Schindelin]

When rebasing commits that have commit notes attached, the interactive
rebase rewrites those notes faithfully at the end. The sequencer must
do this, too, if it wishes to do interactive rebase's job.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 54f654c..a5e18f3 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -94,6 +94,15 @@ static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
  */
 static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 /*
+ * For the post-rewrite hook, we make a list of rewritten commits and
+ * their new sha1s.  The rewritten-pending list keeps the sha1s of
+ * commits that have been processed, but not committed yet,
+ * e.g. because they are waiting for a 'squash' command.
+ */
+static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
+static GIT_PATH_FUNC(rebase_path_rewritten_pending,
+	"rebase-merge/rewritten-pending")
+/*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
  */
@@ -886,6 +895,43 @@ static int update_squash_messages(enum todo_command command,
 	return write_message(&buf, rebase_path_squash_msg());
 }
 
+static void flush_rewritten_pending(void) {
+	struct strbuf buf = STRBUF_INIT;
+	unsigned char newsha1[20];
+	FILE *out;
+
+	if (strbuf_read_file(&buf, rebase_path_rewritten_pending(), 82) > 0 &&
+			!get_sha1("HEAD", newsha1) &&
+			(out = fopen(rebase_path_rewritten_list(), "a"))) {
+		char *bol = buf.buf, *eol;
+
+		while (*bol) {
+			eol = strchrnul(bol, '\n');
+			fprintf(out, "%.*s %s\n", (int)(eol - bol),
+					bol, sha1_to_hex(newsha1));
+			if (!*eol)
+				break;
+			bol = eol + 1;
+		}
+		fclose(out);
+		unlink(rebase_path_rewritten_pending());
+	}
+}
+
+static void record_in_rewritten(struct object_id *oid,
+		enum todo_command next_command) {
+	FILE *out = fopen(rebase_path_rewritten_pending(), "a");
+
+	if (!out)
+		return;
+
+	fprintf(out, "%s\n", oid_to_hex(oid));
+	fclose(out);
+
+	if (!is_fixup(next_command))
+		flush_rewritten_pending();
+}
+
 static int do_pick_commit(enum todo_command command, struct commit *commit,
 		struct replay_opts *opts, int final_fixup)
 {
@@ -1726,6 +1772,17 @@ static int is_final_fixup(struct todo_list *todo_list)
 	return 1;
 }
 
+static enum todo_command peek_command(struct todo_list *todo_list, int offset)
+{
+	int i;
+
+	for (i = todo_list->current + offset; i < todo_list->nr; i++)
+		if (todo_list->items[i].command != TODO_NOOP)
+			return todo_list->items[i].command;
+
+	return -1;
+}
+
 static const char *reflog_message(struct replay_opts *opts,
 	const char *sub_action, const char *fmt, ...)
 {
@@ -1784,6 +1841,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+			if (is_rebase_i(opts) && !res)
+				record_in_rewritten(&item->commit->object.oid,
+					peek_command(todo_list, 1));
 			if (res && is_fixup(item->command)) {
 				if (res == 1)
 					intend_to_amend();
@@ -1813,6 +1873,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 
 	if (is_rebase_i(opts)) {
 		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
+		struct stat st;
 
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
@@ -1857,6 +1918,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			run_command_v_opt(argv, RUN_GIT_CMD);
 			strbuf_reset(&buf);
 		}
+		flush_rewritten_pending();
+		if (!stat(rebase_path_rewritten_list(), &st) &&
+				st.st_size > 0) {
+			struct child_process child = CHILD_PROCESS_INIT;
+
+			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			child.git_cmd = 1;
+			argv_array_push(&child.args, "notes");
+			argv_array_push(&child.args, "copy");
+			argv_array_push(&child.args, "--for-rewrite=rebase");
+			/* we don't care if this copying failed */
+			run_command(&child);
+		}
+
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
 	}
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 21/34] sequencer (rebase -i): record interrupted commits in rewritten, too

[Johannes Schindelin]

When continuing after a `pick` command failed, we want that commit
to show up in the rewritten-list (and its notes to be rewritten), too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index a5e18f3..a00d349 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2023,6 +2023,17 @@ int sequencer_continue(struct replay_opts *opts)
 			return error_dirty_index(opts);
 		todo_list.current++;
 	}
+	else if (file_exists(rebase_path_stopped_sha())) {
+		struct strbuf buf = STRBUF_INIT;
+		struct object_id oid;
+
+		if (read_oneliner(&buf, rebase_path_stopped_sha(), 1)) {
+			if (!get_sha1_committish(buf.buf, oid.hash))
+				record_in_rewritten(&oid,
+						peek_command(&todo_list, 0));
+		}
+		strbuf_release(&buf);
+	}
 
 	res = pick_commits(&todo_list, opts);
 	todo_list_release(&todo_list);
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 22/34] sequencer (rebase -i): run the post-rewrite hook, if needed

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index a00d349..153116e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1922,6 +1922,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process child = CHILD_PROCESS_INIT;
+			const char *post_rewrite_hook =
+				find_hook("post-rewrite");
 
 			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			child.git_cmd = 1;
@@ -1930,6 +1932,17 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			argv_array_push(&child.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
 			run_command(&child);
+
+			if (post_rewrite_hook) {
+				struct child_process hook = CHILD_PROCESS_INIT;
+
+				hook.in = open(rebase_path_rewritten_list(),
+					O_RDONLY);
+				argv_array_push(&hook.args, post_rewrite_hook);
+				argv_array_push(&hook.args, "rebase");
+				/* we don't care if this hook failed */
+				run_command(&hook);
+			}
 		}
 
 		strbuf_release(&buf);
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 23/34] sequencer (rebase -i): respect the rebase.autostash setting

[Johannes Schindelin]

Git's `rebase` command inspects the `rebase.autostash` config setting
to determine whether it should stash any uncommitted changes before
rebasing and re-apply them afterwards.

As we introduce more bits and pieces to let the sequencer act as
interactive rebase's backend, here is the part that adds support for
the autostash feature.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 153116e..2922837 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -111,6 +111,7 @@ static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
 static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
 static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
+static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
 
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
@@ -1783,6 +1784,47 @@ static enum todo_command peek_command(struct todo_list *todo_list, int offset)
 	return -1;
 }
 
+static int apply_autostash(struct replay_opts *opts)
+{
+	struct strbuf stash_sha1 = STRBUF_INIT;
+	struct child_process child = CHILD_PROCESS_INIT;
+	int ret = 0;
+
+	if (!read_oneliner(&stash_sha1, rebase_path_autostash(), 1)) {
+		strbuf_release(&stash_sha1);
+		return 0;
+	}
+	strbuf_trim(&stash_sha1);
+
+	child.git_cmd = 1;
+	argv_array_push(&child.args, "stash");
+	argv_array_push(&child.args, "apply");
+	argv_array_push(&child.args, stash_sha1.buf);
+	if (!run_command(&child))
+		printf(_("Applied autostash."));
+	else {
+		struct child_process store = CHILD_PROCESS_INIT;
+
+		store.git_cmd = 1;
+		argv_array_push(&store.args, "stash");
+		argv_array_push(&store.args, "store");
+		argv_array_push(&store.args, "-m");
+		argv_array_push(&store.args, "autostash");
+		argv_array_push(&store.args, "-q");
+		argv_array_push(&store.args, stash_sha1.buf);
+		if (run_command(&store))
+			ret = error(_("Cannot store %s"), stash_sha1.buf);
+		else
+			printf(_("Applying autostash resulted in conflicts.\n"
+				"Your changes are safe in the stash.\n"
+				"You can run \"git stash pop\" or"
+				" \"git stash drop\" at any time.\n"));
+	}
+
+	strbuf_release(&stash_sha1);
+	return ret;
+}
+
 static const char *reflog_message(struct replay_opts *opts,
 	const char *sub_action, const char *fmt, ...)
 {
@@ -1944,6 +1986,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				run_command(&hook);
 			}
 		}
+		apply_autostash(opts);
 
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 24/34] sequencer (rebase -i): respect strategy/strategy_opts settings

[Johannes Schindelin]

The sequencer already has an idea about using different merge
strategies. We just piggy-back on top of that, using rebase -i's
own settings, when running the sequencer in interactive rebase mode.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 2922837..5f5fb9f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -112,6 +112,8 @@ static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
 static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
 static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
+static GIT_PATH_FUNC(rebase_path_strategy, "rebase-merge/strategy")
+static GIT_PATH_FUNC(rebase_path_strategy_opts, "rebase-merge/strategy_opts")
 
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
@@ -1404,6 +1406,26 @@ static int read_populate_opts(struct replay_opts *opts)
 		if (file_exists(rebase_path_verbose()))
 			opts->verbose = 1;
 
+		if (read_oneliner(&buf, rebase_path_strategy(), 0)) {
+			opts->strategy =
+				sequencer_entrust(opts,
+						  strbuf_detach(&buf, NULL));
+			if (read_oneliner(&buf,
+					  rebase_path_strategy_opts(), 0)) {
+				int i;
+				opts->xopts_nr = split_cmdline(buf.buf,
+					&opts->xopts);
+				for (i = 0; i < opts->xopts_nr; i++)
+					skip_prefix(opts->xopts[i], "--",
+						    &opts->xopts[i]);
+				if (opts->xopts_nr)
+					sequencer_entrust(opts,
+						strbuf_detach(&buf, NULL));
+				else
+					strbuf_release(&buf);
+			}
+		}
+
 		return 0;
 	}
 
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 25/34] sequencer (rebase -i): allow rescheduling commands

[Johannes Schindelin]

The interactive rebase has the very special magic that a cherry-pick
that exits with a status different from 0 and 1 signifies a failure to
even record that a cherry-pick was started.

This can happen e.g. when a fast-forward fails because it would
overwrite untracked files.

In that case, we must reschedule the command that we thought we already
had at least started successfully.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 5f5fb9f..366ee78 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1895,6 +1895,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					1);
 			res = do_pick_commit(item->command, item->commit,
 					opts, is_final_fixup(todo_list));
+			if (is_rebase_i(opts) && res < 0) {
+				/* Reschedule */
+				todo_list->current--;
+				if (save_todo(todo_list, opts))
+					return -1;
+			}
 			if (item->command == TODO_EDIT) {
 				struct commit *commit = item->commit;
 				if (!res)
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 26/34] sequencer (rebase -i): implement the 'drop' command

[Johannes Schindelin]

The parsing part of a 'drop' command is almost identical to parsing a
'pick', while the operation is the same as that of a 'noop'.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 366ee78..51c2f76 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -762,7 +762,8 @@ enum todo_command {
 	TODO_FIXUP,
 	TODO_SQUASH,
 	TODO_EXEC,
-	TODO_NOOP
+	TODO_NOOP,
+	TODO_DROP
 };
 
 static struct {
@@ -776,7 +777,8 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
-	{ 0,   "noop" }
+	{ 0,   "noop" },
+	{ 'd', "drop" }
 };
 
 static const char *command_to_string(const enum todo_command command)
@@ -1309,7 +1311,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		else if (is_fixup(item->command))
 			return error("Cannot '%s' without a previous commit",
 				command_to_string(item->command));
-		else if (item->command != TODO_NOOP)
+		else if (item->command < TODO_NOOP)
 			fixup_okay = 1;
 		p = *eol ? eol + 1 : eol;
 	}
@@ -1800,7 +1802,7 @@ static enum todo_command peek_command(struct todo_list *todo_list, int offset)
 	int i;
 
 	for (i = todo_list->current + offset; i < todo_list->nr; i++)
-		if (todo_list->items[i].command != TODO_NOOP)
+		if (todo_list->items[i].command < TODO_NOOP)
 			return todo_list->items[i].command;
 
 	return -1;
@@ -1933,7 +1935,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_exec(item->arg);
 			*end_of_arg = saved;
 		}
-		else if (item->command != TODO_NOOP)
+		else if (item->command < TODO_NOOP)
 			return error("Unknown command %d", item->command);
 
 		todo_list->current++;
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

[Johannes Schindelin]

In the upcoming patch, we will support rebase -i's progress
reporting. The progress skips comments but counts 'noop's.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 51c2f76..4c902e5 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -763,7 +763,8 @@ enum todo_command {
 	TODO_SQUASH,
 	TODO_EXEC,
 	TODO_NOOP,
-	TODO_DROP
+	TODO_DROP,
+	TODO_COMMENT
 };
 
 static struct {
@@ -778,12 +779,13 @@ static struct {
 	{ 's', "squash" },
 	{ 'x', "exec" },
 	{ 0,   "noop" },
-	{ 'd', "drop" }
+	{ 'd', "drop" },
+	{ 0,   NULL }
 };
 
 static const char *command_to_string(const enum todo_command command)
 {
-	if (command < ARRAY_SIZE(todo_command_info))
+	if (command < TODO_COMMENT)
 		return todo_command_info[command].str;
 	die("Unknown command: %d", command);
 }
@@ -1235,14 +1237,14 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	bol += strspn(bol, " \t");
 
 	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
-		item->command = TODO_NOOP;
+		item->command = TODO_COMMENT;
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = eol - bol;
 		return 0;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)
+	for (i = 0; i < TODO_COMMENT; i++)
 		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
 			item->command = i;
 			break;
@@ -1252,7 +1254,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 			item->command = i;
 			break;
 		}
-	if (i >= ARRAY_SIZE(todo_command_info))
+	if (i >= TODO_COMMENT)
 		return -1;
 
 	if (item->command == TODO_NOOP) {
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 28/34] run_command_opt(): optionally hide stderr when the command succeeds

[Johannes Schindelin]

This will be needed to hide the output of `git commit` when the
sequencer handles an interactive rebase's script.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 run-command.c | 23 +++++++++++++++++++++++
 run-command.h |  1 +
 2 files changed, 24 insertions(+)

diff --git a/run-command.c b/run-command.c
index 5a4dbb6..921e43c 100644
--- a/run-command.c
+++ b/run-command.c
@@ -575,6 +575,29 @@ int run_command_v_opt_cd_env(const char **argv, int opt, const char *dir, const
 	cmd.clean_on_exit = opt & RUN_CLEAN_ON_EXIT ? 1 : 0;
 	cmd.dir = dir;
 	cmd.env = env;
+
+	if (opt & RUN_HIDE_STDERR_ON_SUCCESS) {
+		struct strbuf buf = STRBUF_INIT;
+		int res;
+
+		cmd.err = -1;
+		if (start_command(&cmd) < 0)
+			return -1;
+
+		if (strbuf_read(&buf, cmd.err, 0) < 0) {
+			close(cmd.err);
+			finish_command(&cmd); /* throw away exit code */
+			return -1;
+		}
+
+		close(cmd.err);
+		res = finish_command(&cmd);
+		if (res)
+			fputs(buf.buf, stderr);
+		strbuf_release(&buf);
+		return res;
+	}
+
 	return run_command(&cmd);
 }
 
diff --git a/run-command.h b/run-command.h
index 5066649..f87d01a 100644
--- a/run-command.h
+++ b/run-command.h
@@ -70,6 +70,7 @@ extern int run_hook_ve(const char *const *env, const char *name, va_list args);
 #define RUN_SILENT_EXEC_FAILURE 8
 #define RUN_USING_SHELL 16
 #define RUN_CLEAN_ON_EXIT 32
+#define RUN_HIDE_STDERR_ON_SUCCESS 64
 int run_command_v_opt(const char **argv, int opt);
 
 /*
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 29/34] sequencer (rebase -i): show only failed `git commit`'s output

[Johannes Schindelin]

This is the behavior of the shell script version of the interactive
rebase, by using the `output` function defined in `git-rebase.sh`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 4c902e5..22568d2 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -640,10 +640,15 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 {
 	char **env = NULL;
 	struct argv_array array;
-	int rc;
+	int opt = RUN_GIT_CMD, rc;
 	const char *value;
 
 	if (is_rebase_i(opts)) {
+		if (!edit) {
+			opt |= RUN_COMMAND_STDOUT_TO_STDERR;
+			opt |= RUN_HIDE_STDERR_ON_SUCCESS;
+		}
+
 		env = read_author_script();
 		if (!env) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
@@ -688,7 +693,7 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 	if (opts->allow_empty_message)
 		argv_array_push(&array, "--allow-empty-message");
 
-	rc = run_command_v_opt_cd_env(array.argv, RUN_GIT_CMD, NULL,
+	rc = run_command_v_opt_cd_env(array.argv, opt, NULL,
 			(const char *const *)env);
 	argv_array_clear(&array);
 	free(env);
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 30/34] sequencer (rebase -i): show only failed cherry-picks' output

[Johannes Schindelin]

This is the behavior of the shell script version of the interactive
rebase, by using the `output` function defined in `git-rebase.sh`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 22568d2..b34c381 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -477,6 +477,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	o.ancestor = base ? base_label : "(empty tree)";
 	o.branch1 = "HEAD";
 	o.branch2 = next ? next_label : "(empty tree)";
+	if (is_rebase_i(opts))
+		o.buffer_output = 2;
 
 	head_tree = parse_tree_indirect(head);
 	next_tree = next ? next->tree : empty_tree();
@@ -488,6 +490,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	if (is_rebase_i(opts) && clean <= 0)
+		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
 	if (clean < 0)
 		return clean;
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 31/34] sequencer (rebase -i): suggest --edit-todo upon unknown command

[Johannes Schindelin]

This is the same behavior as known from `git rebase -i`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index b34c381..89fd625 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1346,8 +1346,12 @@ static int read_populate_todo(struct todo_list *todo_list,
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
-	if (res)
+	if (res) {
+		if (is_rebase_i(opts))
+			return error("Please fix this using "
+				"'git rebase --edit-todo'.");
 		return error(_("Unusable instruction sheet: %s"), todo_file);
+	}
 	if (!todo_list->nr &&
 	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
 		return error(_("No commits parsed."));
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 32/34] sequencer (rebase -i): show the progress

[Johannes Schindelin]

The interactive rebase keeps the user informed about its progress.
If the sequencer wants to do the grunt work of the interactive
rebase, it also needs to show that progress.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 89fd625..e8c6daf 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1218,6 +1218,7 @@ struct todo_list {
 	struct strbuf buf;
 	struct todo_item *items;
 	int nr, alloc, current;
+	int done_nr, total_nr;
 };
 
 #define TODO_LIST_INIT { STRBUF_INIT }
@@ -1329,6 +1330,17 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 	return res;
 }
 
+static int count_commands(struct todo_list *todo_list)
+{
+	int count = 0, i;
+
+	for (i = 0; i < todo_list->nr; i++)
+		if (todo_list->items[i].command != TODO_COMMENT)
+			count++;
+
+	return count;
+}
+
 static int read_populate_todo(struct todo_list *todo_list,
 			struct replay_opts *opts)
 {
@@ -1355,6 +1367,22 @@ static int read_populate_todo(struct todo_list *todo_list,
 	if (!todo_list->nr &&
 	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
 		return error(_("No commits parsed."));
+
+	if (is_rebase_i(opts)) {
+		struct todo_list done = TODO_LIST_INIT;
+
+		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
+				!parse_insn_buffer(done.buf.buf, &done))
+			todo_list->done_nr = count_commands(&done);
+		else
+			todo_list->done_nr = 0;
+
+		todo_list->total_nr = todo_list->done_nr
+			+ count_commands(todo_list);
+
+		todo_list_release(&done);
+	}
+
 	return 0;
 }
 
@@ -1900,6 +1928,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
+			if (item->command != TODO_COMMENT)
+				fprintf(stderr, "Rebasing (%d/%d)%s",
+					++(todo_list->done_nr),
+					todo_list->total_nr,
+					opts->verbose ? "\n" : "\r");
 			unlink(rebase_path_message());
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 33/34] sequencer (rebase -i): write the progress into files

[Johannes Schindelin]

For the benefit of e.g. the shell prompt, the interactive rebase not
only displays the progress for the user to see, but also writes it into
the msgnum/end files in the state directory.

Teach the sequencer this new trick.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index e8c6daf..9a8069e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -45,6 +45,16 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
 /*
+ * The file to keep track of how many commands were already processed (e.g.
+ * for the prompt).
+ */
+static GIT_PATH_FUNC(rebase_path_msgnum, "rebase-merge/msgnum");
+/*
+ * The file to keep track of how many commands are to be processed in total
+ * (e.g. for the prompt).
+ */
+static GIT_PATH_FUNC(rebase_path_msgtotal, "rebase-merge/end");
+/*
  * The commit message that is planned to be used for any changes that
  * need to be committed following a user interaction.
  */
@@ -1370,17 +1380,22 @@ static int read_populate_todo(struct todo_list *todo_list,
 
 	if (is_rebase_i(opts)) {
 		struct todo_list done = TODO_LIST_INIT;
+		FILE *f = fopen(rebase_path_msgtotal(), "w");
 
 		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
 				!parse_insn_buffer(done.buf.buf, &done))
 			todo_list->done_nr = count_commands(&done);
 		else
 			todo_list->done_nr = 0;
+		todo_list_release(&done);
 
 		todo_list->total_nr = todo_list->done_nr
 			+ count_commands(todo_list);
 
-		todo_list_release(&done);
+		if (f) {
+			fprintf(f, "%d\n", todo_list->total_nr);
+			fclose(f);
+		}
 	}
 
 	return 0;
@@ -1928,11 +1943,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
-			if (item->command != TODO_COMMENT)
+			if (item->command != TODO_COMMENT) {
+				FILE *f = fopen(rebase_path_msgnum(), "w");
+
+				todo_list->done_nr++;
+
+				if (f) {
+					fprintf(f, "%d\n", todo_list->done_nr);
+					fclose(f);
+				}
 				fprintf(stderr, "Rebasing (%d/%d)%s",
-					++(todo_list->done_nr),
+					todo_list->done_nr,
 					todo_list->total_nr,
 					opts->verbose ? "\n" : "\r");
+			}
 			unlink(rebase_path_message());
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
-- 
2.10.0.rc2.102.g5c102ec

[PATCH 34/34] sequencer (rebase -i): write out the final message

[Johannes Schindelin]

The shell script version of the interactive rebase has a very specific
final message. Teach the sequencer to print the same.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 9a8069e..6064018 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2090,6 +2090,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		}
 		apply_autostash(opts);
 
+		fprintf(stderr, "Successfully rebased and updated %s.\n",
+			head_ref.buf);
+
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
 	}
-- 
2.10.0.rc2.102.g5c102ec

Re: [PATCH 05/34] sequencer (rebase -i): learn about the 'verbose' mode

[Dennis Kaarsemaker]

On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:

> +               if (file_exists(rebase_path_verbose()))
> +                       opts->verbose = 1;

I don't see anything in this series that creates this file, will that
be part of a later series?

D.

Re: [PATCH 06/34] sequencer (rebase -i): write the 'done' file

[Dennis Kaarsemaker]

On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:
> In the interactive rebase, commands that were successfully processed are
> not simply discarded, but appended to the 'done' file instead. This is
> used e.g. to display the current state to the user in the output of
> `git status` or the progress.

Wouldn't it make more sense to have this patch before the ones that
implement the actual rebase commands?

Hmm, and after reading more of this series, I think the same applies to
some other patches too, e.g. 08/34 and 14/34, so I'm probably missing
something. So before I make a fool of myself and suggest that the
implementation of the actual commands should come at the end, maybe you
could tell me what I'm missing :) 

D.

Re: [PATCH 00/34] Teach the sequencer to act as rebase -i's backend

[Dennis Kaarsemaker]

On wo, 2016-08-31 at 10:53 +0200, Johannes Schindelin wrote:
> This marks the count down to '3': two more patch series after this
> (really tiny ones) and we have a faster rebase -i.

I got to 16/34 (and skipped 07/34), will continue tomorrow. I hope the
comments are useful.

D.

Re: [PATCH 15/34] sequencer (rebase -i): leave a patch upon error

[Dennis Kaarsemaker]

On wo, 2016-08-31 at 10:55 +0200, Johannes Schindelin wrote:
> Just like the interactive rebase, we want to leave a 'patch' file for
> further inspection by the user (even if we never tried to actually apply
> that patch, since we're cherry-picking instead).
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

This commit message confuses me. Did you mean s/Just like the/When
doing an/? 

D.

Re: [PATCH 05/34] sequencer (rebase -i): learn about the 'verbose' mode

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 494 bytes --]

Hi Dennis,

On Wed, 31 Aug 2016, Dennis Kaarsemaker wrote:

> On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:
> 
> > +               if (file_exists(rebase_path_verbose()))
> > +                       opts->verbose = 1;
> 
> I don't see anything in this series that creates this file, will that
> be part of a later series?

No. The sequencer does not write that file, but just consumes it. `git
rebase` writes it.

Ciao,
Dscho

Re: [PATCH 20/34] sequencer (rebase -i): copy commit notes at end

[Dennis Kaarsemaker]

On wo, 2016-08-31 at 10:55 +0200, Johannes Schindelin wrote:
> +               if (!stat(rebase_path_rewritten_list(), &st) &&
> +                               st.st_size > 0) {
> +                       struct child_process child = CHILD_PROCESS_INIT;
> +
> +                       child.in = open(rebase_path_rewritten_list(), O_RDONLY);
> +                       child.git_cmd = 1;
> +                       argv_array_push(&child.args, "notes");
> +                       argv_array_push(&child.args, "copy");
> +                       argv_array_push(&child.args, "--for-rewrite=rebase");
> +                       /* we don't care if this copying failed */
> +                       run_command(&child);
> +               }

I know this is a strict port of git-rebase--interactive.sh, but
shouldn't we at least warn the user that the copy failed?

D.

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Dennis Kaarsemaker]

On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:

> +static int is_fixup(enum todo_command command)
> +{
> +	return command == TODO_FIXUP || command == TODO_SQUASH;
> +}

It sounds wrong to have a function named is_fixup return true when the
command isn't a fixup but a squash. Maybe name it
changes_previous_commit or something?

> +static const char *nth_for_number(int n)
> +{
> +	int n1 = n % 10, n10 = n % 100;
> +
> +	if (n1 == 1 && n10 != 11)
> +		return "st";
> +	if (n1 == 2 && n10 != 12)
> +		return "nd";
> +	if (n1 == 3 && n10 != 13)
> +		return "rd";
> +	return "th";
> +}

>8---

> +	if (command == TODO_SQUASH) {
> +		unlink(rebase_path_fixup_msg());
> +		strbuf_addf(&buf, "\n%c This is the %d%s commit message:\n\n%s",
> +			comment_line_char,
> +			count, nth_for_number(count), body);
> +	}
> +	else if (command == TODO_FIXUP) {
> +		strbuf_addf(&buf,
> +			"\n%c The %d%s commit message will be skipped:\n\n",
> +			comment_line_char, count, nth_for_number(count));
> +		strbuf_add_commented_lines(&buf, body, strlen(body));
> +	}

This way of handling numbers is not translatable, and I really think we
should mark these strings for translation, like they are in the .sh
version.

D.

Re: [PATCH 32/34] sequencer (rebase -i): show the progress

[Dennis Kaarsemaker]

On wo, 2016-08-31 at 10:56 +0200, Johannes Schindelin wrote:
> The interactive rebase keeps the user informed about its progress.
> If the sequencer wants to do the grunt work of the interactive
> rebase, it also needs to show that progress.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 33 +++++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
> 
> diff --git a/sequencer.c b/sequencer.c
> index 89fd625..e8c6daf 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -1218,6 +1218,7 @@ struct todo_list {
>  	struct strbuf buf;
>  	struct todo_item *items;
>  	int nr, alloc, current;
> +	int done_nr, total_nr;
>  };
>  
>  #define TODO_LIST_INIT { STRBUF_INIT }
> @@ -1329,6 +1330,17 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
>  	return res;
>  }
>  
> +static int count_commands(struct todo_list *todo_list)
> +{
> +	int count = 0, i;
> +
> +	for (i = 0; i < todo_list->nr; i++)
> +		if (todo_list->items[i].command != TODO_COMMENT)
> +			count++;
> +
> +	return count;
> +}
> +
>  static int read_populate_todo(struct todo_list *todo_list,
>  			struct replay_opts *opts)
>  {
> @@ -1355,6 +1367,22 @@ static int read_populate_todo(struct todo_list *todo_list,
>  	if (!todo_list->nr &&
>  	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
>  		return error(_("No commits parsed."));
> +
> +	if (is_rebase_i(opts)) {
> +		struct todo_list done = TODO_LIST_INIT;
> +
> +		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
> +				!parse_insn_buffer(done.buf.buf, &done))
> +			todo_list->done_nr = count_commands(&done);
> +		else
> +			todo_list->done_nr = 0;
> +
> +		todo_list->total_nr = todo_list->done_nr
> +			+ count_commands(todo_list);
> +
> +		todo_list_release(&done);
> +	}
> +
>  	return 0;
>  }
>  
> @@ -1900,6 +1928,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  		if (save_todo(todo_list, opts))
>  			return -1;
>  		if (is_rebase_i(opts)) {
> +			if (item->command != TODO_COMMENT)
> +				fprintf(stderr, "Rebasing (%d/%d)%s",
> +					++(todo_list->done_nr),
> +					todo_list->total_nr,
> +					opts->verbose ? "\n" : "\r");
>  			unlink(rebase_path_message());
>  			unlink(rebase_path_author_script());
>  			unlink(rebase_path_stopped_sha());

(picking a random commit that shows this 'symptom')

You're sprinking a lot of is_rebase_i's around sequencer.c to make sure
there are no changes in behaviour. I wonder if the right balance has
been struck between 'no changes in behaviour' and 'common behaviour'.
For instance, in this case, maybe it would be a better idea for non-
rebase uses of the sequencer to also show progress.

D.

Re: [PATCH 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

[Dennis Kaarsemaker]

On wo, 2016-08-31 at 10:56 +0200, Johannes Schindelin wrote:
> diff --git a/sequencer.c b/sequencer.c
> index 51c2f76..4c902e5 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -763,7 +763,8 @@ enum todo_command {
>         TODO_SQUASH,
>         TODO_EXEC,
>         TODO_NOOP,
> -       TODO_DROP
> +       TODO_DROP,
> +       TODO_COMMENT
>  };

(picking a random commit that touches this enum)

In a few places you now make comparisons like "< TODO_NOOP", so I think
it would be good to have a comment near the definition of this enum
that says that ordering matters and why, so people don't attempt to add
a new TODO_FOOBAR at the end.

D. 

Re: [PATCH 06/34] sequencer (rebase -i): write the 'done' file

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1861 bytes --]

Hi Dennis,

On Wed, 31 Aug 2016, Dennis Kaarsemaker wrote:

> On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:
> > In the interactive rebase, commands that were successfully processed are
> > not simply discarded, but appended to the 'done' file instead. This is
> > used e.g. to display the current state to the user in the output of
> > `git status` or the progress.
> 
> Wouldn't it make more sense to have this patch before the ones that
> implement the actual rebase commands?

I waffled about the order so many times that I don't know anymore. The
thing is, while the sequencer is taught incrementally to understand all of
the rebase -i functionality, rebase -i itself is not touched, on purpose.

In the case of the "done" file, my thoughts were: the commands do not need
this file *at all*. In fact, if we did not write the "done" file at all,
the only two types of test failures in the test suite would be 1) git
status' output and 2) the prompt testing for the progress.

So you see, functionally, the "done" file is only relevant to the progress
part of the patch series.

As such, I'd rather keep this patch in the current place, just before
introducing the progress.

> Hmm, and after reading more of this series, I think the same applies to
> some other patches too, e.g. 08/34 and 14/34, so I'm probably missing
> something. So before I make a fool of myself and suggest that the
> implementation of the actual commands should come at the end, maybe you
> could tell me what I'm missing :) 

No, no, don't hesitate to suggest reorderings. I am really thankful for
the discussion we are having, so that the outcome is better than what I
have right now. If the outcome would be the very same patches, but with
more confidence, it would still be better than what I have right now ;-)

Ciao,
Dscho

Re: [PATCH 15/34] sequencer (rebase -i): leave a patch upon error

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 894 bytes --]

Hi Dennis,

On Wed, 31 Aug 2016, Dennis Kaarsemaker wrote:

> On wo, 2016-08-31 at 10:55 +0200, Johannes Schindelin wrote:
> > Just like the interactive rebase, we want to leave a 'patch' file for
> > further inspection by the user (even if we never tried to actually apply
> > that patch, since we're cherry-picking instead).
> > 
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> 
> This commit message confuses me. Did you mean s/Just like the/When
> doing an/? 

What I meant is this: when calling `git rebase -i` right now, i.e. before
any of my rebase--helper work, a failing rebase will leave a `patch` file
in the `.git/rebase-merge/` directory. Since the sequencer is in the
process of learning how to do an interactive rebase, it needs to learn the
same trick.

But I guess that your suggested edit makes things much clearer.

Thanks,
Dscho

Re: [PATCH 00/34] Teach the sequencer to act as rebase -i's backend

[Johannes Schindelin]

Hi Dennis,

On Wed, 31 Aug 2016, Dennis Kaarsemaker wrote:

> On wo, 2016-08-31 at 10:53 +0200, Johannes Schindelin wrote:
> > This marks the count down to '3': two more patch series after this
> > (really tiny ones) and we have a faster rebase -i.
> 
> I got to 16/34 (and skipped 07/34), will continue tomorrow. I hope the
> comments are useful.

Much appreciated!
Dscho

Re: [PATCH 20/34] sequencer (rebase -i): copy commit notes at end

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1499 bytes --]

Hi Dennis,

On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:

> On wo, 2016-08-31 at 10:55 +0200, Johannes Schindelin wrote:
> > +               if (!stat(rebase_path_rewritten_list(), &st) &&
> > +                               st.st_size > 0) {
> > +                       struct child_process child = CHILD_PROCESS_INIT;
> > +
> > +                       child.in = open(rebase_path_rewritten_list(), O_RDONLY);
> > +                       child.git_cmd = 1;
> > +                       argv_array_push(&child.args, "notes");
> > +                       argv_array_push(&child.args, "copy");
> > +                       argv_array_push(&child.args, "--for-rewrite=rebase");
> > +                       /* we don't care if this copying failed */
> > +                       run_command(&child);
> > +               }
> 
> I know this is a strict port of git-rebase--interactive.sh, but
> shouldn't we at least warn the user that the copy failed?

At this point, I want to have as faithful a conversion as possible (except
that I do not care for the speed of git-rebase--interactive.sh, of
course).

Besides, I am fairly certain that a failure will result in an error
message. We just do not act on the exit value.

Ciao,
Dscho

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Johannes Schindelin]

Hi Dennis,

On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:

> On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:
> 
> > +static int is_fixup(enum todo_command command)
> > +{
> > +	return command == TODO_FIXUP || command == TODO_SQUASH;
> > +}
> 
> It sounds wrong to have a function named is_fixup return true when the
> command isn't a fixup but a squash. Maybe name it
> changes_previous_commit or something?

I can see how that may sound confusing, unless you understand that a
squash is a fixup that lets the user edit the commit message, too. So
essentially squash = fixup + edit, if you will.

Maybe the name is more appropriate in that light?

> > +static const char *nth_for_number(int n)
> > +{
> > +	int n1 = n % 10, n10 = n % 100;
> > +
> > +	if (n1 == 1 && n10 != 11)
> > +		return "st";
> > +	if (n1 == 2 && n10 != 12)
> > +		return "nd";
> > +	if (n1 == 3 && n10 != 13)
> > +		return "rd";
> > +	return "th";
> > +}
> 
> >8---
> 
> > +	if (command == TODO_SQUASH) {
> > +		unlink(rebase_path_fixup_msg());
> > +		strbuf_addf(&buf, "\n%c This is the %d%s commit message:\n\n%s",
> > +			comment_line_char,
> > +			count, nth_for_number(count), body);
> > +	}
> > +	else if (command == TODO_FIXUP) {
> > +		strbuf_addf(&buf,
> > +			"\n%c The %d%s commit message will be skipped:\n\n",
> > +			comment_line_char, count, nth_for_number(count));
> > +		strbuf_add_commented_lines(&buf, body, strlen(body));
> > +	}
> 
> This way of handling numbers is not translatable, and I really think we
> should mark these strings for translation, like they are in the .sh
> version.

Ah, this is the risk of working on something as big as rebase--helper.
Back when I started with it, the relevant code in git-rebase--interactive
read like this:

	nth_string () {
		case "$1" in
		*1[0-9]|*[04-9]) echo "$1"th;;
		*1) echo "$1"st;;
		*2) echo "$1"nd;;
		*3) echo "$1"rd;;
		esac
	}

I merely did a faithful translation of that...

Now, I see that git-rebase--interactive was switched to use eval_gettext,
which in turn is handled in git-sh-i18n whose code is quite convoluted. In
the absence of gettext, it uses git-sh-i18n--envsubst, which has no C API
whatsoever.

And I see that the beautiful ordinal computation was given up in favor of
a lousy "#1", "#2", "#3", etc (it used to be "1st", "2nd", "3rd" etc).

In any case, translation is not my main concern until v2.10.0, so I'll
take care of this after that release.

Ciao,
Dscho

Re: [PATCH 32/34] sequencer (rebase -i): show the progress

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3475 bytes --]

Hi Dennis,

On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:

> On wo, 2016-08-31 at 10:56 +0200, Johannes Schindelin wrote:
> > The interactive rebase keeps the user informed about its progress.
> > If the sequencer wants to do the grunt work of the interactive
> > rebase, it also needs to show that progress.
> > 
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  sequencer.c | 33 +++++++++++++++++++++++++++++++++
> >  1 file changed, 33 insertions(+)
> > 
> > diff --git a/sequencer.c b/sequencer.c
> > index 89fd625..e8c6daf 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -1218,6 +1218,7 @@ struct todo_list {
> >  	struct strbuf buf;
> >  	struct todo_item *items;
> >  	int nr, alloc, current;
> > +	int done_nr, total_nr;
> >  };
> >  
> >  #define TODO_LIST_INIT { STRBUF_INIT }
> > @@ -1329,6 +1330,17 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
> >  	return res;
> >  }
> >  
> > +static int count_commands(struct todo_list *todo_list)
> > +{
> > +	int count = 0, i;
> > +
> > +	for (i = 0; i < todo_list->nr; i++)
> > +		if (todo_list->items[i].command != TODO_COMMENT)
> > +			count++;
> > +
> > +	return count;
> > +}
> > +
> >  static int read_populate_todo(struct todo_list *todo_list,
> >  			struct replay_opts *opts)
> >  {
> > @@ -1355,6 +1367,22 @@ static int read_populate_todo(struct todo_list *todo_list,
> >  	if (!todo_list->nr &&
> >  	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
> >  		return error(_("No commits parsed."));
> > +
> > +	if (is_rebase_i(opts)) {
> > +		struct todo_list done = TODO_LIST_INIT;
> > +
> > +		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
> > +				!parse_insn_buffer(done.buf.buf, &done))
> > +			todo_list->done_nr = count_commands(&done);
> > +		else
> > +			todo_list->done_nr = 0;
> > +
> > +		todo_list->total_nr = todo_list->done_nr
> > +			+ count_commands(todo_list);
> > +
> > +		todo_list_release(&done);
> > +	}
> > +
> >  	return 0;
> >  }
> >  
> > @@ -1900,6 +1928,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
> >  		if (save_todo(todo_list, opts))
> >  			return -1;
> >  		if (is_rebase_i(opts)) {
> > +			if (item->command != TODO_COMMENT)
> > +				fprintf(stderr, "Rebasing (%d/%d)%s",
> > +					++(todo_list->done_nr),
> > +					todo_list->total_nr,
> > +					opts->verbose ? "\n" : "\r");
> >  			unlink(rebase_path_message());
> >  			unlink(rebase_path_author_script());
> >  			unlink(rebase_path_stopped_sha());
> 
> (picking a random commit that shows this 'symptom')
> 
> You're sprinking a lot of is_rebase_i's around sequencer.c to make sure
> there are no changes in behaviour. I wonder if the right balance has
> been struck between 'no changes in behaviour' and 'common behaviour'.
> For instance, in this case, maybe it would be a better idea for non-
> rebase uses of the sequencer to also show progress.

Actually, adding progress would make for a fine add-on patch series. I
still would like to have as faithful a conversion as possible, and
everything else can come after that.

This strategy has a couple of advantages:

- we can concentrate on correctness for now,

- I get something to show for my time with Git for Windows v2.10.0, and

- the add-on patches do not have to be done by *me* ;-)

Ciao,
Dscho

Re: [PATCH 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1509 bytes --]

Hi Dennis,

On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:

> On wo, 2016-08-31 at 10:56 +0200, Johannes Schindelin wrote:
> > diff --git a/sequencer.c b/sequencer.c
> > index 51c2f76..4c902e5 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -763,7 +763,8 @@ enum todo_command {
> >         TODO_SQUASH,
> >         TODO_EXEC,
> >         TODO_NOOP,
> > -       TODO_DROP
> > +       TODO_DROP,
> > +       TODO_COMMENT
> >  };
> 
> (picking a random commit that touches this enum)
> 
> In a few places you now make comparisons like "< TODO_NOOP", so I think
> it would be good to have a comment near the definition of this enum
> that says that ordering matters and why, so people don't attempt to add
> a new TODO_FOOBAR at the end.

True.

It does not seem that we have a precedent for that. The closest is what I
had in an early iteration of the fsck message IDs, and subsequently things
were refactored so that it is not the order, but a flag, that determines
what the command does.

Not sure how to do this elegantly. Maybe like this?

	enum todo_command {
		TODO_PICK_COMMANDS = 0,
		TODO_PICK = TODO_PICK_COMMANDS,
		TODO_SQUASH,

		TODO_NON_PICK_COMMANDS,
		TODO_EXEC = TODO_NON_PICK_COMMANDS,

		TODO_NOOP_COMMANDS,
		TODO_NOOP = TODO_NOOP_COMMANDS,
		TODO_DROP
		TODO_DROP,

		TODO_LAST_COMMAND,
		TODO_COMMENT = TODO_LAST_COMMAND
	};

But that is so god-awful to read.

Still unsure,
Dscho

Re: [PATCH 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

[Dennis Kaarsemaker]

On do, 2016-09-01 at 17:32 +0200, Johannes Schindelin wrote:
> Hi Dennis,
> 
> On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:
> 
> > 
> > On wo, 2016-08-31 at 10:56 +0200, Johannes Schindelin wrote:
> > > 
> > > diff --git a/sequencer.c b/sequencer.c
> > > index 51c2f76..4c902e5 100644
> > > --- a/sequencer.c
> > > +++ b/sequencer.c
> > > @@ -763,7 +763,8 @@ enum todo_command {
> > >         TODO_SQUASH,
> > >         TODO_EXEC,
> > >         TODO_NOOP,
> > > -       TODO_DROP
> > > +       TODO_DROP,
> > > +       TODO_COMMENT
> > >  };
> > (picking a random commit that touches this enum)
> > 
> > In a few places you now make comparisons like "< TODO_NOOP", so I
> > think
> > it would be good to have a comment near the definition of this enum
> > that says that ordering matters and why, so people don't attempt to
> > add
> > a new TODO_FOOBAR at the end.
> True.
> 
> It does not seem that we have a precedent for that. The closest is
> what I
> had in an early iteration of the fsck message IDs, and subsequently
> things
> were refactored so that it is not the order, but a flag, that
> determines
> what the command does.
> 
> Not sure how to do this elegantly. Maybe like this?
> 
> 	enum todo_command {
> 		TODO_PICK_COMMANDS = 0,
> 		TODO_PICK = TODO_PICK_COMMANDS,
> 		TODO_SQUASH,
> 
> 		TODO_NON_PICK_COMMANDS,
> 		TODO_EXEC = TODO_NON_PICK_COMMANDS,
> 
> 		TODO_NOOP_COMMANDS,
> 		TODO_NOOP = TODO_NOOP_COMMANDS,
> 		TODO_DROP
> 		TODO_DROP,
> 
> 		TODO_LAST_COMMAND,
> 		TODO_COMMENT = TODO_LAST_COMMAND
> 	};
> 
> But that is so god-awful to read.

Agreed, that sure is awful.

How about something like

/*
 * Note that ordering matters in this enum. Not only must it match the
 * mapping below, it is also divided into several sections that matter.
 * When adding new commands, make sure you add it in the right section.
 */
enum todo_command {
	/* All commands that handle commits */
	TODO_PICK,
	...
	/* All commands that do something else than pick */
	TODO_EXEC,
	...
	/* All commands that do nothing but are counted for reporting progress */
	TODO_NOOP,
	...
	/* Comments, which are not counted
	TODO_COMMENT
}

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Dennis Kaarsemaker]

On do, 2016-09-01 at 17:17 +0200, Johannes Schindelin wrote:
> Hi Dennis,
> 
> On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:
> 
> > 
> > On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:
> > 
> > > 
> > > +static int is_fixup(enum todo_command command)
> > > +{
> > > +	return command == TODO_FIXUP || command == TODO_SQUASH;
> > > +}
> > It sounds wrong to have a function named is_fixup return true when
> > the
> > command isn't a fixup but a squash. Maybe name it
> > changes_previous_commit or something?
> I can see how that may sound confusing, unless you understand that a
> squash is a fixup that lets the user edit the commit message, too. So
> essentially squash = fixup + edit, if you will.
> 
> Maybe the name is more appropriate in that light?

Kinda makes sense. It's not how I use fixup/squash as a user of rebase
-i though. But we can't go there, that's bikeshed country :)

> > > +static const char *nth_for_number(int n)
> > > +{
> > > +	int n1 = n % 10, n10 = n % 100;
> > > +
> > > +	if (n1 == 1 && n10 != 11)
> > > +		return "st";
> > > +	if (n1 == 2 && n10 != 12)
> > > +		return "nd";
> > > +	if (n1 == 3 && n10 != 13)
> > > +		return "rd";
> > > +	return "th";
> > > +}
> > > 
> > > 8---
> > > 
> > > +	if (command == TODO_SQUASH) {
> > > +		unlink(rebase_path_fixup_msg());
> > > +		strbuf_addf(&buf, "\n%c This is the %d%s commit
> > > message:\n\n%s",
> > > +			comment_line_char,
> > > +			count, nth_for_number(count), body);
> > > +	}
> > > +	else if (command == TODO_FIXUP) {
> > > +		strbuf_addf(&buf,
> > > +			"\n%c The %d%s commit message will be
> > > skipped:\n\n",
> > > +			comment_line_char, count,
> > > nth_for_number(count));
> > > +		strbuf_add_commented_lines(&buf, body,
> > > strlen(body));
> > > +	}
> > This way of handling numbers is not translatable, and I really
> > think we
> > should mark these strings for translation, like they are in the .sh
> > version.
> Ah, this is the risk of working on something as big as rebase
> --helper.
> Back when I started with it, the relevant code in git-rebase
> --interactive
> read like this:
> 
> 	nth_string () {
> 		case "$1" in
> 		*1[0-9]|*[04-9]) echo "$1"th;;
> 		*1) echo "$1"st;;
> 		*2) echo "$1"nd;;
> 		*3) echo "$1"rd;;
> 		esac
> 	}
> 
> I merely did a faithful translation of that...
> 
> Now, I see that git-rebase--interactive was switched to use
> eval_gettext,
> which in turn is handled in git-sh-i18n whose code is quite
> convoluted. In
> the absence of gettext, it uses git-sh-i18n--envsubst, which has no C
> API
> whatsoever.
> 
> And I see that the beautiful ordinal computation was given up in
> favor of
> a lousy "#1", "#2", "#3", etc (it used to be "1st", "2nd", "3rd"
> etc).
> 
> In any case, translation is not my main concern until v2.10.0, so
> I'll
> take care of this after that release.

Hmm, not sure if I agree with that. I'd see it as a regression to lose
the i18n there.

D.

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Junio C Hamano]

Dennis Kaarsemaker <dennis@kaarsemaker.net> writes:

>> +static const char *nth_for_number(int n)
>> +{
>> +	int n1 = n % 10, n10 = n % 100;
>> +
>> +	if (n1 == 1 && n10 != 11)
>> +		return "st";
>> +	if (n1 == 2 && n10 != 12)
>> +		return "nd";
>> +	if (n1 == 3 && n10 != 13)
>> +		return "rd";
>> +	return "th";
>> +}
>
>>8---
>
>> +	if (command == TODO_SQUASH) {
>> +		unlink(rebase_path_fixup_msg());
>> +		strbuf_addf(&buf, "\n%c This is the %d%s commit message:\n\n%s",
>> +			comment_line_char,
>> +			count, nth_for_number(count), body);
>> +	}
>> +	else if (command == TODO_FIXUP) {
>> +		strbuf_addf(&buf,
>> +			"\n%c The %d%s commit message will be skipped:\n\n",
>> +			comment_line_char, count, nth_for_number(count));
>> +		strbuf_add_commented_lines(&buf, body, strlen(body));
>> +	}
>
> This way of handling numbers is not translatable, and I really think we
> should mark these strings for translation, like they are in the .sh
> version.

Correct.

For those who were not paying attention on the 'master' front during
this pre-release period [*1*], I have to point out that the scripted
Porcelain has been updated to lose the Anglo-centric st/nd/rd/th and
this series would want to get updated to match.


[Footnote]

*1* Why weren't you?  Repent! ;-)

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Johannes Schindelin]

Hi Dennis,

On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:

> On do, 2016-09-01 at 17:17 +0200, Johannes Schindelin wrote:
>
> > And I see that the beautiful ordinal computation was given up in favor
> > of a lousy "#1", "#2", "#3", etc (it used to be "1st", "2nd", "3rd"
> > etc).
> > 
> > In any case, translation is not my main concern until v2.10.0, so I'll
> > take care of this after that release.
> 
> Hmm, not sure if I agree with that. I'd see it as a regression to lose
> the i18n there.

As Git for Windows does not ship with translations (for multiple reasons),
it would not be a regression.

Having said that, I see that having a different text than the current
rebase -i can be seen as a regression, so I changed that.

Thanks for the review!
Dscho

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Johannes Schindelin]

Hi Junio,

On Thu, 1 Sep 2016, Junio C Hamano wrote:

> For those who were not paying attention on the 'master' front during
> this pre-release period [*1*], I have to point out that the scripted
> Porcelain has been updated to lose the Anglo-centric st/nd/rd/th and
> this series would want to get updated to match.
> 
> 
> [Footnote]
> 
> *1* Why weren't you?  Repent! ;-)

I tried to. But, you know, I was kinda busy with a couple of patch series.

In any case, I changed the code this morning. Can't say that I like those
forced last-minute changes.

Ciao,
Johannes

Re: [PATCH 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 685 bytes --]

Hi Dennis,

On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:

> /*
>  * Note that ordering matters in this enum. Not only must it match the
>  * mapping below, it is also divided into several sections that matter.
>  * When adding new commands, make sure you add it in the right section.
>  */
> enum todo_command {
> 	/* All commands that handle commits */
> 	TODO_PICK,
> 	...
> 	/* All commands that do something else than pick */
> 	TODO_EXEC,
> 	...
> 	/* All commands that do nothing but are counted for reporting progress */
> 	TODO_NOOP,
> 	...
> 	/* Comments, which are not counted
> 	TODO_COMMENT
> }

I like it! Changed accordingly.

Thanks!
Dscho

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Dennis Kaarsemaker]

On vr, 2016-09-02 at 09:13 +0200, Johannes Schindelin wrote:

> As Git for Windows does not ship with translations (for multiple
> reasons), it would not be a regression.

I'm confused, how does "git for windows does not ship with
translations" translate to "this is not a regression"? Is this patch
series only meant to be for git for windows and not go into git.git
itself?

D.

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Johannes Schindelin]

Hi Dennis,

On Fri, 2 Sep 2016, Dennis Kaarsemaker wrote:

> On vr, 2016-09-02 at 09:13 +0200, Johannes Schindelin wrote:
> 
> > As Git for Windows does not ship with translations (for multiple
> > reasons), it would not be a regression.
> 
> I'm confused, how does "git for windows does not ship with
> translations" translate to "this is not a regression"? Is this patch
> series only meant to be for git for windows and not go into git.git
> itself?

Oh, I thought I had clarified my plan... The timeline is:

- I submit the remaining rebase--helper patch series for review (last week
  and this one),

- I publish a preview of Git for Windows v2.10.0 that already uses these
  patches (done: https://github.com/git-for-windows/git/releases/tag/v2.9.3.windows.3)

- once upstream Git v2.10.0 is released (possibly today, after my work
  hours), I perform a final "Git garden shears" run (read: rebase Git for
  Windows' patches, retaining the branch structure) on top of v2.10.0 and
  release Git for Windows v2.10.0, tagged as v2.10.0.windows.1 in
  https://github.com/git-for-windows/git (due to time zone differences
  relative to Junio, the most likely time for this release would be
  some time around noon tomorrow, given that the release engineering takes
  roughly 2-4 hours, running unsupervised for the most part).

- as far as Git for Windows is concerned, l10n is not really an issue yet:
  the installer is released without any localizations.

- After releasing Git for Windows v2.10.0, I will pay a lot of attention
  to feedback. Not only to hear a lot of praise, but also to catch any
  possible regressions. Not that I expect anything dramatic to happen
  because I really tested this as thoroughly as I can: not a single one of
  my interactive rebases since mid May has been performed without
  involving the rebase--helper. In the three cases where I *did* find a
  regression, I solved it immediately, of course.

- After releasing Git for Windows v2.10.0, I will have a nice beer. Or
  three.

- Then I will leisurely try to address the l10n issues.

- Then, I will send out the current iterations of the patch series that
  are in flight.

- I have the entire week to address concerns with Git for Windows as well
  as with the patch series (where the former takes precedence, of course).

- The second half of September, I will relax from this marathon that
  started in early February. Meaning: I will be mostly offline.

I hope this clarifies why I am not so concerned about some issues such as
translation, or commit messages, or grammar, and more so about others,
such as incorrect code.

Ciao,
Dscho

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Dennis Kaarsemaker]

On vr, 2016-09-02 at 16:22 +0200, Johannes Schindelin wrote:
> I hope this clarifies why I am not so concerned about some issues
> such as translation, or commit messages, or grammar, and more so
> about others, such as incorrect code.

It does, thanks!

D.

Re: [PATCH 01/34] sequencer: support a new action: 'interactive rebase'

[Kevin Daudt]

On Wed, Aug 31, 2016 at 10:54:02AM +0200, Johannes Schindelin wrote:
> @@ -43,16 +51,20 @@ static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
>  /* We will introduce the 'interactive rebase' mode later */
>  static inline int is_rebase_i(const struct replay_opts *opts)
>  {
> -	return 0;
> +	return opts->action == REPLAY_INTERACTIVE_REBASE;
>  }
>  
>  static const char *get_dir(const struct replay_opts *opts)
>  {
> +	if (is_rebase_i(opts))
> +		return rebase_path();
>  	return git_path_seq_dir();
>  }
>  
>  static const char *get_todo_path(const struct replay_opts *opts)
>  {
> +	if (is_rebase_i(opts))
> +		return rebase_path_todo();
>  	return git_path_todo_file();
>  }

This patch fails to apply for me because function is_rebase_i has never
been introduced before (no record of it anywhere). Currently, only
IS_REBASE_I macro is present.

Re: [PATCH 01/34] sequencer: support a new action: 'interactive rebase'

[Johannes Schindelin]

Hi Kevin,

On Fri, 2 Sep 2016, Kevin Daudt wrote:

> On Wed, Aug 31, 2016 at 10:54:02AM +0200, Johannes Schindelin wrote:
> > @@ -43,16 +51,20 @@ static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
> >  /* We will introduce the 'interactive rebase' mode later */
> >  static inline int is_rebase_i(const struct replay_opts *opts)
> >  {
> > -	return 0;
> > +	return opts->action == REPLAY_INTERACTIVE_REBASE;
> >  }
> >  
> >  static const char *get_dir(const struct replay_opts *opts)
> >  {
> > +	if (is_rebase_i(opts))
> > +		return rebase_path();
> >  	return git_path_seq_dir();
> >  }
> >  
> >  static const char *get_todo_path(const struct replay_opts *opts)
> >  {
> > +	if (is_rebase_i(opts))
> > +		return rebase_path_todo();
> >  	return git_path_todo_file();
> >  }
> 
> This patch fails to apply for me because function is_rebase_i has never
> been introduced before (no record of it anywhere). Currently, only
> IS_REBASE_I macro is present.

I did not send out a new iteration of the prepare-sequencer patch series
(mostly because I wanted reviewers to look at the later patch series,
rather than re-review a 2nd iteration of something they already saw).

But I did address the concerns mentioned in the review already, of course,
because part of the reason to show those patch series was to get valuable
feedback before including the work in Git for Windows v2.10.0.

You can find the current iteration of the prepare-sequencer here:
https://github.com/dscho/git/compare/libify-sequencer...prepare-sequencer

Please note that I will most likely try to address some l10n concerns
before sending out the next iteration of the patch series.

The patch introducing is_rebase_i() (and no longer IS_REBASE_I()) is:
https://github.com/dscho/git/commit/76d272020bb72618957308f06083c807efe59aca

If you want to have the latest iteration of the entire patch thicket, just
`git fetch https://github.com/dscho/git interactive-rebase`. I update that
more frequently than I send out updates via mail.

Ciao,
Johannes

[PATCH v2 00/34] Teach the sequencer to act as rebase -i's backend

[Johannes Schindelin]

This marks the count down to '3': two more patch series after this
(really tiny ones) and we have a faster rebase -i.

The idea of this patch series is to teach the sequencer to understand
all of the commands in `git-rebase-todo` scripts, to execute them and to
behave pretty much very the same as `git rebase -i --continue` when
called with the newly-introduced REPLAY_INTERACTIVE_REBASE mode.

Most of these patches should be pretty much straight-forward. When not,
I tried to make a point of describing enough background in the commit
message. Please feel free to point out where my explanations fall short.

Note that even after this patch series is applied, rebase -i is still
unaffected. It will require the next patch series which introduces the
rebase--helper that essentially implements `git rebase -i --continue` by
calling the sequencer with the appropriate options.

The final patch series will move a couple of pre- and post-processing
steps into the rebase--helper/sequencer (such as expanding/shrinking the
SHA-1s, reordering the fixup!/squash! lines, etc). This might sound like
a mere add-on, but it is essential for the speed improvements: those
stupid little processing steps really dominated the execution time in my
tests.

Apart from mostly cosmetic patches (and the occasional odd bug that I
fixed promptly), I used these patches since mid May to perform all of my
interactive rebases. In mid June, I had the idea to teach rebase -i to
run *both* scripted rebase and rebase--helper and to cross-validate the
results. This slowed down all my interactive rebases since, but helped
me catch three rather obscure bugs (e.g. that git commit --fixup unfolds
long onelines and rebase -i still finds the correct original commit).

This is all only to say that I am rather confident that the current code
does the job.

Since sending out v1, I integrated all of these patch series
into Git for Windows v2.10.0, where they have been live ever since, and
used by myself (also in a Linux VM, as Git for Windows' master branch
always builds also on Linux and passes the test suite, too).

Just to reiterate why I do all this: it speeds up the interactive rebase
substantially. Even with a not yet fully builtin rebase -i, but just the
part after the user edited the `git-rebase-todo` script.

The performance test I introduced to demonstrate this (p3404) shows a
speed-up of +380% here (i.e. roughly 5x), from ~8.8 seconds to ~1.8
seconds. This is on Windows, where the performance impact of avoiding
shell scripting is most noticable.

On MacOSX and on Linux, the speed-up is less pronounced, but still
noticable, at least if you trust Travis CI, which I abused to perform
that test for me. Check for yourself (searching for "3404.2") here:
https://travis-ci.org/git/git/builds/156295227. According to those logs,
p3404 is speeded up from ~0.45 seconds to ~0.12 seconds on Linux (read:
about 3.5x) and from ~1.7 seconds to ~0.5 seconds on MacOSX (read:
almost 4x).

Please note that the interdiff vs v1 is only of limited use: too many
things changed in the meantime, in particular the prepare-sequencer
branch that went through a couple of iterations before it found its way
into git.git's master branch. So please take the interdiff with a
mountain range of salt.

Changes since v1:

- some grammar touch-ups.

- simplified determining the command string in
  walk_revs_populate_todo().

- removed the beautiful ordinal logic (to print out "1st", "2nd", "3rd"
  etc) and made things consistent with the current `rebase -i`.

- while at it, marked more messages for translation.

- added code-comments to clarify the order, and the sections, of the
  todo_command enum.

- replaced one error(..., strerror(...)) to an error_errno(...).

- downcased error messages

- marked error messages for translation

- adjusted the patches to account for sequencer_entrust() having been
  removed from the prepare-sequencer patch series by request of Junio.

- moved the introduction of write_message_gently() into the patch
  introducing its first usage, i.e. the support for the 'edit' command.

- adjusted some indentations

- prevented an write_in_full() from being called after a failed open()

- inserted a few forgotten strbuf_release() calls


Johannes Schindelin (34):
  sequencer: support a new action: 'interactive rebase'
  sequencer (rebase -i): implement the 'noop' command
  sequencer (rebase -i): implement the 'edit' command
  sequencer (rebase -i): implement the 'exec' command
  sequencer (rebase -i): learn about the 'verbose' mode
  sequencer (rebase -i): write the 'done' file
  sequencer (rebase -i): add support for the 'fixup' and 'squash'
    commands
  sequencer (rebase -i): implement the short commands
  sequencer (rebase -i): write an author-script file
  sequencer (rebase -i): allow continuing with staged changes
  sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed
  sequencer (rebase -i): skip some revert/cherry-pick specific code path
  sequencer (rebase -i): the todo can be empty when continuing
  sequencer (rebase -i): update refs after a successful rebase
  sequencer (rebase -i): leave a patch upon error
  sequencer (rebase -i): implement the 'reword' command
  sequencer (rebase -i): allow fast-forwarding for edit/reword
  sequencer (rebase -i): refactor setting the reflog message
  sequencer (rebase -i): set the reflog message consistently
  sequencer (rebase -i): copy commit notes at end
  sequencer (rebase -i): record interrupted commits in rewritten, too
  sequencer (rebase -i): run the post-rewrite hook, if needed
  sequencer (rebase -i): respect the rebase.autostash setting
  sequencer (rebase -i): respect strategy/strategy_opts settings
  sequencer (rebase -i): allow rescheduling commands
  sequencer (rebase -i): implement the 'drop' command
  sequencer (rebase -i): differentiate between comments and 'noop'
  run_command_opt(): optionally hide stderr when the command succeeds
  sequencer (rebase -i): show only failed `git commit`'s output
  sequencer (rebase -i): show only failed cherry-picks' output
  sequencer (rebase -i): suggest --edit-todo upon unknown command
  sequencer (rebase -i): show the progress
  sequencer (rebase -i): write the progress into files
  sequencer (rebase -i): write out the final message

 run-command.c |   23 ++
 run-command.h |    1 +
 sequencer.c   | 1003 ++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 sequencer.h   |    4 +-
 4 files changed, 983 insertions(+), 48 deletions(-)


base-commit: 8d7a455ed52e2a96debc080dfc011b6bb00db5d2
Published-As: https://github.com/dscho/git/releases/tag/sequencer-i-v2
Fetch-It-Via: git fetch https://github.com/dscho/git sequencer-i-v2

Interdiff vs v1:

 diff --git a/sequencer.c b/sequencer.c
 index 6ca9d1e09d..41be4cde16 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -125,7 +125,6 @@ static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
  static GIT_PATH_FUNC(rebase_path_strategy, "rebase-merge/strategy")
  static GIT_PATH_FUNC(rebase_path_strategy_opts, "rebase-merge/strategy_opts")
  
 -/* We will introduce the 'interactive rebase' mode later */
  static inline int is_rebase_i(const struct replay_opts *opts)
  {
  	return opts->action == REPLAY_INTERACTIVE_REBASE;
 @@ -259,13 +258,13 @@ static const char *action_name(const struct replay_opts *opts)
  {
  	switch (opts->action) {
  	case REPLAY_REVERT:
 -		return "revert";
 +		return N_("revert");
  	case REPLAY_PICK:
 -		return "cherry-pick";
 +		return N_("cherry-pick");
  	case REPLAY_INTERACTIVE_REBASE:
 -		return "rebase -i";
 +		return N_("rebase -i");
  	}
 -	die("Unknown action: %d", opts->action);
 +	die(_("Unknown action: %d"), opts->action);
  }
  
  struct commit_message {
 @@ -548,6 +547,7 @@ static int write_author_script(const char *message)
  {
  	struct strbuf buf = STRBUF_INIT;
  	const char *eol;
 +	int res;
  
  	for (;;)
  		if (!*message || starts_with(message, "\n")) {
 @@ -585,8 +585,9 @@ static int write_author_script(const char *message)
  			strbuf_addch(&buf, *(message++));
  		else
  			strbuf_addf(&buf, "'\\\\%c'", *(message++));
 -	strbuf_addstr(&buf, "'\n");
 -	return write_message(&buf, rebase_path_author_script());
 +	res = write_message(buf.buf, buf.len, rebase_path_author_script(), 1);
 +	strbuf_release(&buf);
 +	return res;
  }
  
  /*
 @@ -771,16 +772,25 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
  		return 1;
  }
  
 +/*
 + * Note that ordering matters in this enum. Not only must it match the mapping
 + * below, it is also divided into several sections that matter.  When adding
 + * new commands, make sure you add it in the right section.
 + */
  enum todo_command {
 +	/* commands that handle commits */
  	TODO_PICK = 0,
  	TODO_REVERT,
  	TODO_EDIT,
  	TODO_REWORD,
  	TODO_FIXUP,
  	TODO_SQUASH,
 +	/* commands that do something else than handling a single commit */
  	TODO_EXEC,
 +	/* commands that do nothing but are counted for reporting progress */
  	TODO_NOOP,
  	TODO_DROP,
 +	/* comments (not counted for reporting progress) */
  	TODO_COMMENT
  };
  
 @@ -812,47 +822,34 @@ static int is_fixup(enum todo_command command)
  	return command == TODO_FIXUP || command == TODO_SQUASH;
  }
  
 -static const char *nth_for_number(int n)
 -{
 -	int n1 = n % 10, n10 = n % 100;
 -
 -	if (n1 == 1 && n10 != 11)
 -		return "st";
 -	if (n1 == 2 && n10 != 12)
 -		return "nd";
 -	if (n1 == 3 && n10 != 13)
 -		return "rd";
 -	return "th";
 -}
 -
  static int update_squash_messages(enum todo_command command,
  		struct commit *commit, struct replay_opts *opts)
  {
  	struct strbuf buf = STRBUF_INIT;
 -	int count;
 +	int count, res;
  	const char *message, *body;
  
  	if (file_exists(rebase_path_squash_msg())) {
  		char *p, *p2;
  
  		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
 -			return error("Could not read %s",
 +			return error(_("could not read '%s'"),
  				rebase_path_squash_msg());
  
 -		if (buf.buf[0] == '\n' || !skip_prefix(buf.buf + 1,
 -				" This is a combination of ",
 -				(const char **)&p))
 -			return error("Unexpected 1st line of squash message:\n"
 -				"\n\t%.*s",
 -				(int)(strchrnul(buf.buf, '\n') - buf.buf),
 -				buf.buf);
 +		if (buf.buf[0] != comment_line_char ||
 +		    !skip_prefix(buf.buf + 1, " This is a combination of ",
 +				 (const char **)&p))
 +			return error(_("unexpected 1st line of squash message:"
 +				       "\n\n\t%.*s"),
 +				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
 +				     buf.buf);
  		count = strtol(p, &p2, 10);
  
  		if (count < 1 || *p2 != ' ')
 -			return error("Invalid 1st line of squash message:\n"
 -				"\n\t%.*s",
 -				(int)(strchrnul(buf.buf, '\n') - buf.buf),
 -				buf.buf);
 +			return error(_("invalid 1st line of squash message:\n"
 +				       "\n\t%.*s"),
 +				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
 +				     buf.buf);
  
  		sprintf((char *)p, "%d", ++count);
  		if (!*p2)
 @@ -868,32 +865,33 @@ static int update_squash_messages(enum todo_command command,
  		const char *head_message, *body;
  
  		if (get_sha1("HEAD", head))
 -			return error("Need a HEAD to fixup");
 +			return error(_("need a HEAD to fixup"));
  		if (!(head_commit = lookup_commit_reference(head)))
 -			return error("Could not read HEAD");
 +			return error(_("could not read HEAD"));
  		if (!(head_message = get_commit_buffer(head_commit, NULL)))
 -			return error("Could not read HEAD's commit message");
 +			return error(_("could not read HEAD's commit message"));
  
  		body = strstr(head_message, "\n\n");
  		if (!body)
  			body = "";
  		else
  			body = skip_blank_lines(body + 2);
 -		if (write_file_gently(rebase_path_fixup_msg(), body, 0))
 -			return error("Cannot write %s",
 -				rebase_path_fixup_msg());
 +		if (write_message(body, strlen(body),
 +				  rebase_path_fixup_msg(), 0))
 +			return error(_("cannot write '%s'"),
 +				     rebase_path_fixup_msg());
  
  		count = 2;
 -		strbuf_addf(&buf, "%c This is a combination of 2 commits.\n"
 -			"%c The first commit's message is:\n\n%s",
 -			comment_line_char, comment_line_char, body);
 +		strbuf_addf(&buf, _("%c This is a combination of 2 commits.\n"
 +				    "%c The first commit's message is:\n\n%s"),
 +			    comment_line_char, comment_line_char, body);
  
  		unuse_commit_buffer(head_commit, head_message);
  	}
  
  	if (!(message = get_commit_buffer(commit, NULL)))
 -		return error("Could not read commit message of %s",
 -			oid_to_hex(&commit->object.oid));
 +		return error(_("could not read commit message of %s"),
 +			     oid_to_hex(&commit->object.oid));
  	body = strstr(message, "\n\n");
  	if (!body)
  		body = "";
 @@ -902,21 +900,23 @@ static int update_squash_messages(enum todo_command command,
  
  	if (command == TODO_SQUASH) {
  		unlink(rebase_path_fixup_msg());
 -		strbuf_addf(&buf, "\n%c This is the %d%s commit message:\n\n%s",
 -			comment_line_char,
 -			count, nth_for_number(count), body);
 +		strbuf_addf(&buf, _("\n%c This is the commit message #%d:\n"
 +				    "\n%s"),
 +			    comment_line_char, count, body);
  	}
  	else if (command == TODO_FIXUP) {
 -		strbuf_addf(&buf,
 -			"\n%c The %d%s commit message will be skipped:\n\n",
 -			comment_line_char, count, nth_for_number(count));
 +		strbuf_addf(&buf, _("\n%c The commit message #%d "
 +				    "will be skipped:\n\n"),
 +			    comment_line_char, count);
  		strbuf_add_commented_lines(&buf, body, strlen(body));
  	}
  	else
 -		return error("Unknown command: %d", command);
 +		return error(_("unknown command: %d"), command);
  	unuse_commit_buffer(commit, message);
  
 -	return write_message(&buf, rebase_path_squash_msg());
 +	res = write_message(buf.buf, buf.len, rebase_path_squash_msg(), 0);
 +	strbuf_release(&buf);
 +	return res;
  }
  
  static void flush_rewritten_pending(void) {
 @@ -940,6 +940,7 @@ static void flush_rewritten_pending(void) {
  		fclose(out);
  		unlink(rebase_path_rewritten_pending());
  	}
 +	strbuf_release(&buf);
  }
  
  static void record_in_rewritten(struct object_id *oid,
 @@ -1013,7 +1014,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		parent = commit->parents->item;
  
  	if (get_message(commit, &msg) != 0)
 -		return error(_("Cannot get commit message for %s"),
 +		return error(_("cannot get commit message for %s"),
  			oid_to_hex(&commit->object.oid));
  
  	if (opts->allow_ff && !is_fixup(command) &&
 @@ -1021,7 +1022,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  	     (!parent && unborn))) {
  		if (is_rebase_i(opts))
  			write_author_script(msg.message);
 -		res |= fast_forward_to(commit->object.oid.hash, head, unborn,
 +		res = fast_forward_to(commit->object.oid.hash, head, unborn,
  			opts);
  		if (res || command != TODO_REWORD)
  			goto leave;
 @@ -1100,8 +1101,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  			const char *dest = git_path("SQUASH_MSG");
  			unlink(dest);
  			if (copy_file(dest, rebase_path_squash_msg(), 0666))
 -				return error("Could not rename %s to "
 -					"%s", rebase_path_squash_msg(), dest);
 +				return error(_("could not rename '%s' to '%s'"),
 +					     rebase_path_squash_msg(), dest);
  			unlink(git_path("MERGE_MSG"));
  			msg_file = dest;
  			edit = 1;
 @@ -1109,7 +1110,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  	}
  
  	if (is_rebase_i(opts) && write_author_script(msg.message) < 0)
 -		res |= -1;
 +		res = -1;
  	else if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
  		res = do_recursive_merge(base, next, base_label, next_label,
  					 head, &msgbuf, opts);
 @@ -1166,8 +1167,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  	}
  	if (!opts->no_commit)
  fast_forward_edit:
 -		res |= sequencer_commit(msg_file, opts, allow, edit, amend,
 -			cleanup_commit_message);
 +		res = run_git_commit(msg_file, opts, allow, edit, amend,
 +				     cleanup_commit_message);
  
  	if (!res && final_fixup) {
  		unlink(rebase_path_fixup_msg());
 @@ -1317,7 +1318,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
  static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
  {
  	struct todo_item *item;
 -	char *p = buf;
 +	char *p = buf, *next_p;
  	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
  
  	for (i = 1; *p; i++, p = next_p) {
 @@ -1335,15 +1336,16 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
  				i, (int)(eol - p), p);
  			item->command = TODO_NOOP;
  		}
 +
  		if (fixup_okay)
  			; /* do nothing */
  		else if (is_fixup(item->command))
 -			return error("Cannot '%s' without a previous commit",
 +			return error(_("cannot '%s' without a previous commit"),
  				command_to_string(item->command));
  		else if (item->command < TODO_NOOP)
  			fixup_okay = 1;
 -		p = *eol ? eol + 1 : eol;
  	}
 +
  	return res;
  }
  
 @@ -1377,13 +1379,14 @@ static int read_populate_todo(struct todo_list *todo_list,
  	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
  	if (res) {
  		if (is_rebase_i(opts))
 -			return error("Please fix this using "
 -				"'git rebase --edit-todo'.");
 -		return error(_("Unusable instruction sheet: %s"), todo_file);
 +			return error(_("please fix this using "
 +				       "'git rebase --edit-todo'."));
 +		return error(_("unusable instruction sheet: '%s'"), todo_file);
  	}
 +
  	if (!todo_list->nr &&
  	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
 -		return error(_("No commits parsed."));
 +		return error(_("no commits parsed."));
  
  	if (!is_rebase_i(opts)) {
  		enum todo_command valid =
 @@ -1408,10 +1411,10 @@ static int read_populate_todo(struct todo_list *todo_list,
  			todo_list->done_nr = count_commands(&done);
  		else
  			todo_list->done_nr = 0;
 -		todo_list_release(&done);
  
  		todo_list->total_nr = todo_list->done_nr
  			+ count_commands(todo_list);
 +		todo_list_release(&done);
  
  		if (f) {
  			fprintf(f, "%d\n", todo_list->total_nr);
 @@ -1467,6 +1470,26 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
  	return 0;
  }
  
 +static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
 +{
 +	int i;
 +
 +	strbuf_reset(buf);
 +	if (!read_oneliner(buf, rebase_path_strategy(), 0))
 +		return;
 +	opts->strategy = strbuf_detach(buf, NULL);
 +	if (!read_oneliner(buf, rebase_path_strategy_opts(), 0))
 +		return;
 +
 +	opts->xopts_nr = split_cmdline(buf->buf, (const char ***)&opts->xopts);
 +	for (i = 0; i < opts->xopts_nr; i++) {
 +		const char *arg = opts->xopts[i];
 +
 +		skip_prefix(arg, "--", &arg);
 +		opts->xopts[i] = xstrdup(arg);
 +	}
 +}
 +
  static int read_populate_opts(struct replay_opts *opts)
  {
  	if (is_rebase_i(opts)) {
 @@ -1480,30 +1503,12 @@ static int read_populate_opts(struct replay_opts *opts)
  				opts->gpg_sign = xstrdup(buf.buf + 2);
  			}
  		}
 -		strbuf_release(&buf);
  
  		if (file_exists(rebase_path_verbose()))
  			opts->verbose = 1;
  
 -		if (read_oneliner(&buf, rebase_path_strategy(), 0)) {
 -			opts->strategy =
 -				sequencer_entrust(opts,
 -						  strbuf_detach(&buf, NULL));
 -			if (read_oneliner(&buf,
 -					  rebase_path_strategy_opts(), 0)) {
 -				int i;
 -				opts->xopts_nr = split_cmdline(buf.buf,
 -					&opts->xopts);
 -				for (i = 0; i < opts->xopts_nr; i++)
 -					skip_prefix(opts->xopts[i], "--",
 -						    &opts->xopts[i]);
 -				if (opts->xopts_nr)
 -					sequencer_entrust(opts,
 -						strbuf_detach(&buf, NULL));
 -				else
 -					strbuf_release(&buf);
 -			}
 -		}
 +		read_strategy_opts(opts, &buf);
 +		strbuf_release(&buf);
  
  		return 0;
  	}
 @@ -1527,7 +1532,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
  {
  	enum todo_command command = opts->action == REPLAY_PICK ?
  		TODO_PICK : TODO_REVERT;
 -	const char *command_string = todo_command_strings[command];
 +	const char *command_string = todo_command_info[command].str;
  	struct commit *commit;
  
  	if (prepare_revs(opts))
 @@ -1681,12 +1686,15 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
  		int prev_offset = !next ? 0 :
  			todo_list->items[next - 1].offset_in_buf;
  
 -		if (offset > prev_offset && write_in_full(fd,
 -				todo_list->buf.buf + prev_offset,
 -				offset - prev_offset) < 0)
 -			return error(_("Could not write to %s (%s)"),
 -				done_path, strerror(errno));
 -		close(fd);
 +		if (fd >= 0 && offset > prev_offset &&
 +		    write_in_full(fd, todo_list->buf.buf + prev_offset,
 +				  offset - prev_offset) < 0) {
 +			close(fd);
 +			return error_errno(_("could not write to '%s'"),
 +					   done_path);
 +		}
 +		if (fd >= 0)
 +			close(fd);
  	}
  	return 0;
  }
 @@ -1730,11 +1738,11 @@ static int make_patch(struct commit *commit, struct replay_opts *opts)
  {
  	struct strbuf buf = STRBUF_INIT;
  	struct rev_info log_tree_opt;
 -	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject;
 +	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject, *p;
  	int res = 0;
  
 -	if (write_file_gently(rebase_path_stopped_sha(),
 -			      short_commit_name(commit), 1) < 0)
 +	p = short_commit_name(commit);
 +	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
  		return -1;
  
  	strbuf_addf(&buf, "%s/patch", get_dir(opts));
 @@ -1748,7 +1756,7 @@ static int make_patch(struct commit *commit, struct replay_opts *opts)
  	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
  	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
  	if (!log_tree_opt.diffopt.file)
 -		res |= error_errno("could not open '%s'", buf.buf);
 +		res |= error_errno(_("could not open '%s'"), buf.buf);
  	else {
  		res |= log_tree_commit(&log_tree_opt, commit);
  		fclose(log_tree_opt.diffopt.file);
 @@ -1758,7 +1766,7 @@ static int make_patch(struct commit *commit, struct replay_opts *opts)
  	strbuf_addf(&buf, "%s/message", get_dir(opts));
  	if (!file_exists(buf.buf)) {
  		find_commit_subject(commit_buffer, &subject);
 -		res |= write_file_gently(buf.buf, subject, 1);
 +		res |= write_message(subject, strlen(subject), buf.buf, 1);
  		unuse_commit_buffer(commit, commit_buffer);
  	}
  	strbuf_release(&buf);
 @@ -1769,11 +1777,13 @@ static int make_patch(struct commit *commit, struct replay_opts *opts)
  static int intend_to_amend(void)
  {
  	unsigned char head[20];
 +	char *p;
  
  	if (get_sha1("HEAD", head))
 -		return error("Cannot read HEAD");
 +		return error(_("cannot read HEAD"));
  
 -	return write_file_gently(rebase_path_amend(), sha1_to_hex(head), 1);
 +	p = sha1_to_hex(head);
 +	return write_message(p, strlen(p), rebase_path_amend(), 1);
  }
  
  static int error_with_patch(struct commit *commit,
 @@ -1806,13 +1816,13 @@ static int error_failed_squash(struct commit *commit,
  	struct replay_opts *opts, int subject_len, const char *subject)
  {
  	if (rename(rebase_path_squash_msg(), rebase_path_message()))
 -		return error("Could not rename %s to %s",
 +		return error(_("could not rename '%s' to '%s'"),
  			rebase_path_squash_msg(), rebase_path_message());
  	unlink(rebase_path_fixup_msg());
  	unlink(git_path("MERGE_MSG"));
  	if (copy_file(git_path("MERGE_MSG"), rebase_path_message(), 0666))
 -		return error("Could not copy %s to %s", rebase_path_message(),
 -			git_path("MERGE_MSG"));
 +		return error(_("could not copy '%s' to '%s'"),
 +			     rebase_path_message(), git_path("MERGE_MSG"));
  	return error_with_patch(commit, subject, subject_len, opts, 1, 0);
  }
  
 @@ -1827,30 +1837,30 @@ static int do_exec(const char *command_line)
  
  	/* force re-reading of the cache */
  	if (discard_cache() < 0 || read_cache() < 0)
 -		return error(_("Could not read index"));
 +		return error(_("could not read index"));
  
  	dirty = require_clean_work_tree("rebase", NULL, 1, 1);
  
  	if (status) {
 -		warning("Execution failed: %s\n%s"
 -			"You can fix the problem, and then run\n"
 -			"\n"
 -			"  git rebase --continue\n"
 -			"\n",
 +		warning(_("execution failed: %s\n%s"
 +			  "You can fix the problem, and then run\n"
 +			  "\n"
 +			  "  git rebase --continue\n"
 +			  "\n"),
  			command_line,
 -			dirty ? "and made changes to the index and/or the "
 -				"working tree\n" : "");
 +			dirty ? N_("and made changes to the index and/or the "
 +				"working tree\n") : "");
  		if (status == 127)
  			/* command not found */
  			status = 1;
  	}
  	else if (dirty) {
 -		warning("Execution succeeded: %s\nbut "
 -			"left changes to the index and/or the working tree\n"
 -			"Commit or stash your changes, and then run\n"
 -			"\n"
 -			"  git rebase --continue\n"
 -			"\n", command_line);
 +		warning(_("execution succeeded: %s\nbut "
 +			  "left changes to the index and/or the working tree\n"
 +			  "Commit or stash your changes, and then run\n"
 +			  "\n"
 +			  "  git rebase --continue\n"
 +			  "\n"), command_line);
  		status = 1;
  	}
  
 @@ -1912,7 +1922,7 @@ static int apply_autostash(struct replay_opts *opts)
  		argv_array_push(&store.args, "-q");
  		argv_array_push(&store.args, stash_sha1.buf);
  		if (run_command(&store))
 -			ret = error(_("Cannot store %s"), stash_sha1.buf);
 +			ret = error(_("cannot store %s"), stash_sha1.buf);
  		else
  			printf(_("Applying autostash resulted in conflicts.\n"
  				"Your changes are safe in the stash.\n"
 @@ -1995,7 +2005,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			if (item->command == TODO_EDIT) {
  				struct commit *commit = item->commit;
  				if (!res)
 -					warning("Stopped at %s... %.*s",
 +					warning(_("stopped at %s... %.*s"),
  						short_commit_name(commit),
  						item->arg_len, item->arg);
  				return error_with_patch(commit,
 @@ -2025,7 +2035,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			*end_of_arg = saved;
  		}
  		else if (item->command < TODO_NOOP)
 -			return error("Unknown command %d", item->command);
 +			return error(_("unknown command %d"), item->command);
  
  		todo_list->current++;
  		if (res)
 @@ -2046,22 +2056,22 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			unsigned char head[20], orig[20];
  
  			if (get_sha1("HEAD", head))
 -				return error("Cannot read HEAD");
 +				return error(_("cannot read HEAD"));
  			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
  					get_sha1_hex(buf.buf, orig))
 -				return error("Could not read orig-head");
 +				return error(_("could not read orig-head"));
  			if (!read_oneliner(&buf, rebase_path_onto(), 0))
 -				return error("Could not read 'onto'");
 +				return error(_("could not read 'onto'"));
  			msg = reflog_message(opts, "finish", "%s onto %s",
  				head_ref.buf, buf.buf);
  			if (update_ref(msg, head_ref.buf, head, orig,
  					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 -				return error("Could not update %s",
 +				return error(_("could not update %s"),
  					head_ref.buf);
  			msg = reflog_message(opts, "finish", "returning to %s",
  				head_ref.buf);
  			if (create_symref("HEAD", head_ref.buf, msg))
 -				return error("Could not update HEAD to %s",
 +				return error(_("could not update HEAD to %s"),
  					head_ref.buf);
  			strbuf_reset(&buf);
  		}
 @@ -2072,7 +2082,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			};
  
  			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
 -				return error("Could not read %s",
 +				return error(_("could not read '%s'"),
  					rebase_path_orig_head());
  			strbuf_addstr(&buf, "..HEAD");
  			argv[2] = buf.buf;
 @@ -2099,6 +2109,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  
  				hook.in = open(rebase_path_rewritten_list(),
  					O_RDONLY);
 +				hook.stdout_to_stderr = 1;
  				argv_array_push(&hook.args, post_rewrite_hook);
  				argv_array_push(&hook.args, "rebase");
  				/* we don't care if this hook failed */
 @@ -2136,12 +2147,12 @@ static int commit_staged_changes(struct replay_opts *opts)
  	int amend = 0;
  
  	if (has_unstaged_changes(1))
 -		return error(_("Cannot rebase: You have unstaged changes."));
 +		return error(_("cannot rebase: You have unstaged changes."));
  	if (!has_uncommitted_changes(0)) {
  		const char *cherry_pick_head = git_path("CHERRY_PICK_HEAD");
  
  		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
 -			return error("Could not remove CHERRY_PICK_HEAD");
 +			return error(_("could not remove CHERRY_PICK_HEAD"));
  		return 0;
  	}
  
 @@ -2150,23 +2161,24 @@ static int commit_staged_changes(struct replay_opts *opts)
  		unsigned char head[20], to_amend[20];
  
  		if (get_sha1("HEAD", head))
 -			return error("Cannot amend non-existing commit");
 +			return error(_("cannot amend non-existing commit"));
  		if (!read_oneliner(&rev, rebase_path_amend(), 0))
 -			return error("Invalid file: %s", rebase_path_amend());
 +			return error(_("invalid file: '%s'"), rebase_path_amend());
  		if (get_sha1_hex(rev.buf, to_amend))
 -			return error("Invalid contents: %s",
 +			return error(_("invalid contents: '%s'"),
  				rebase_path_amend());
  		if (hashcmp(head, to_amend))
 -			return error("\nYou have uncommitted changes in your "
 -				"working tree. Please, commit them\nfirst and "
 -				"then run 'git rebase --continue' again.");
 +			return error(_("\nYou have uncommitted changes in your "
 +				       "working tree. Please, commit them\n"
 +				       "first and then run 'git rebase "
 +				       "--continue' again."));
  
  		strbuf_release(&rev);
  		amend = 1;
  	}
  
 -	if (sequencer_commit(rebase_path_message(), opts, 1, 1, amend, 0))
 -		return error("Could not commit staged changes.");
 +	if (run_git_commit(rebase_path_message(), opts, 1, 1, amend, 0))
 +		return error(_("could not commit staged changes."));
  	unlink(rebase_path_amend());
  	return 0;
  }
 @@ -2194,23 +2206,23 @@ int sequencer_continue(struct replay_opts *opts)
  		/* Verify that the conflict has been resolved */
  		if (file_exists(git_path_cherry_pick_head()) ||
  		    file_exists(git_path_revert_head())) {
 -			int ret = continue_single_pick();
 -			if (ret)
 -				return ret;
 +			res = continue_single_pick();
 +			if (res)
 +				goto release_todo_list;
 +		}
 +		if (index_differs_from("HEAD", 0, 0)) {
 +			res = error_dirty_index(opts);
 +			goto release_todo_list;
  		}
 -		if (index_differs_from("HEAD", 0))
 -			return error_dirty_index(opts);
  		todo_list.current++;
  	}
  	else if (file_exists(rebase_path_stopped_sha())) {
  		struct strbuf buf = STRBUF_INIT;
  		struct object_id oid;
  
 -		if (read_oneliner(&buf, rebase_path_stopped_sha(), 1)) {
 -			if (!get_sha1_committish(buf.buf, oid.hash))
 -				record_in_rewritten(&oid,
 -						peek_command(&todo_list, 0));
 -		}
 +		if (read_oneliner(&buf, rebase_path_stopped_sha(), 1) &&
 +		    !get_sha1_committish(buf.buf, oid.hash))
 +			record_in_rewritten(&oid, peek_command(&todo_list, 0));
  		strbuf_release(&buf);
  	}
  

-- 
2.11.0.rc3.windows.1

[PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

[Johannes Schindelin]

This patch introduces a new action for the sequencer. It really does not
do a whole lot of its own right now, but lays the ground work for
patches to come. The intention, of course, is to finally make the
sequencer the work horse of the interactive rebase (the original idea
behind the "sequencer" concept).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 +++++++++++++++++++++++++++++----
 sequencer.h |  3 ++-
 2 files changed, 31 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 30b10ba143..21cfdacd06 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -28,6 +28,14 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+static GIT_PATH_FUNC(rebase_path, "rebase-merge")
+/*
+ * The file containing rebase commands, comments, and empty lines.
+ * This file is created by "git rebase -i" then edited by the user. As
+ * the lines are processed, they are removed from the front of this
+ * file and written to the tail of 'done'.
+ */
+static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
 /*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
@@ -40,19 +48,22 @@ static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
  */
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 
-/* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
-	return 0;
+	return opts->action == REPLAY_INTERACTIVE_REBASE;
 }
 
 static const char *get_dir(const struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return rebase_path();
 	return git_path_seq_dir();
 }
 
 static const char *get_todo_path(const struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return rebase_path_todo();
 	return git_path_todo_file();
 }
 
@@ -168,7 +179,15 @@ int sequencer_remove_state(struct replay_opts *opts)
 
 static const char *action_name(const struct replay_opts *opts)
 {
-	return opts->action == REPLAY_REVERT ? N_("revert") : N_("cherry-pick");
+	switch (opts->action) {
+	case REPLAY_REVERT:
+		return N_("revert");
+	case REPLAY_PICK:
+		return N_("cherry-pick");
+	case REPLAY_INTERACTIVE_REBASE:
+		return N_("rebase -i");
+	}
+	die(_("Unknown action: %d"), opts->action);
 }
 
 struct commit_message {
@@ -395,7 +414,10 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
-		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
+		/*
+		 * TRANSLATORS: %s will be "revert", "cherry-pick" or
+		 * "rebase -i".
+		 */
 		return error(_("%s: Unable to write new index file"),
 			_(action_name(opts)));
 	rollback_lock_file(&index_lock);
@@ -1204,6 +1226,9 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 	const char *todo_path = get_todo_path(opts);
 	int next = todo_list->current, offset, fd;
 
+	if (is_rebase_i(opts))
+		next++;
+
 	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
 		return error_errno(_("could not lock '%s'"), todo_path);
diff --git a/sequencer.h b/sequencer.h
index 7a513c576b..cb21cfddee 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -7,7 +7,8 @@ const char *git_path_seq_dir(void);
 
 enum replay_action {
 	REPLAY_REVERT,
-	REPLAY_PICK
+	REPLAY_PICK,
+	REPLAY_INTERACTIVE_REBASE
 };
 
 struct replay_opts {
-- 
2.11.0.rc3.windows.1

[PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

[Johannes Schindelin]

The 'noop' command is probably the most boring of all rebase -i commands
to support in the sequencer.

Which makes it an excellent candidate for this first stab to add support
for rebase -i's commands to the sequencer.

For the moment, let's also treat empty lines and commented-out lines as
'noop'; We will refine that handling later in this patch series.

To make it easier to identify "classes" of todo_commands (such as:
determine whether a command is pick-like, i.e. handles a single commit),
let's enforce a certain order of said commands.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 21cfdacd06..1224799286 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -639,14 +639,23 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 		return 1;
 }
 
+/*
+ * Note that ordering matters in this enum. Not only must it match the mapping
+ * below, it is also divided into several sections that matter.  When adding
+ * new commands, make sure you add it in the right section.
+ */
 enum todo_command {
+	/* commands that handle commits */
 	TODO_PICK = 0,
-	TODO_REVERT
+	TODO_REVERT,
+	/* commands that do nothing but are counted for reporting progress */
+	TODO_NOOP
 };
 
 static const char *todo_command_strings[] = {
 	"pick",
-	"revert"
+	"revert",
+	"noop"
 };
 
 static const char *command_to_string(const enum todo_command command)
@@ -916,6 +925,14 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	/* left-trim */
 	bol += strspn(bol, " \t");
 
+	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
+		item->command = TODO_NOOP;
+		item->commit = NULL;
+		item->arg = bol;
+		item->arg_len = eol - bol;
+		return 0;
+	}
+
 	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
 		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
 			item->command = i;
@@ -924,6 +941,13 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	if (i >= ARRAY_SIZE(todo_command_strings))
 		return -1;
 
+	if (item->command == TODO_NOOP) {
+		item->commit = NULL;
+		item->arg = bol;
+		item->arg_len = eol - bol;
+		return 0;
+	}
+
 	/* Eat up extra spaces/ tabs before object name */
 	padding = strspn(bol, " \t");
 	if (!padding)
@@ -1292,7 +1316,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		res = do_pick_commit(item->command, item->commit, opts);
+		if (item->command <= TODO_REVERT)
+			res = do_pick_commit(item->command, item->commit,
+					opts);
+		else if (item->command != TODO_NOOP)
+			return error(_("unknown command %d"), item->command);
+
 		todo_list->current++;
 		if (res)
 			return res;
-- 
2.11.0.rc3.windows.1

[PATCH v2 03/34] sequencer (rebase -i): implement the 'edit' command

[Johannes Schindelin]

This patch is a straight-forward reimplementation of the `edit`
operation of the interactive rebase command.

Well, not *quite* straight-forward: when stopping, the `edit`
command wants to write the `patch` file (which is not only the
patch, but includes the commit message and author information). To
that end, this patch requires the earlier work that taught the
log-tree machinery to respect the `file` setting of
rev_info->diffopt to write to a file stream different than stdout.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 114 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 1224799286..68e2c84803 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -16,6 +16,7 @@
 #include "refs.h"
 #include "argv-array.h"
 #include "quote.h"
+#include "log-tree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -43,6 +44,20 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
 /*
+ * When an "edit" rebase command is being processed, the SHA1 of the
+ * commit to be edited is recorded in this file.  When "git rebase
+ * --continue" is executed, if there are any staged changes then they
+ * will be amended to the HEAD commit, but only provided the HEAD
+ * commit is still the commit to be edited.  When any other rebase
+ * command is processed, this file is deleted.
+ */
+static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
+/*
+ * When we stop at a given patch via the "edit" command, this file contains
+ * the long commit name of the corresponding patch.
+ */
+static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
+/*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
  */
@@ -648,6 +663,7 @@ enum todo_command {
 	/* commands that handle commits */
 	TODO_PICK = 0,
 	TODO_REVERT,
+	TODO_EDIT,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP
 };
@@ -655,6 +671,7 @@ enum todo_command {
 static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
+	"edit",
 	"noop"
 };
 
@@ -1301,9 +1318,87 @@ static int save_opts(struct replay_opts *opts)
 	return res;
 }
 
+static int make_patch(struct commit *commit, struct replay_opts *opts)
+{
+	struct strbuf buf = STRBUF_INIT;
+	struct rev_info log_tree_opt;
+	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject, *p;
+	int res = 0;
+
+	p = short_commit_name(commit);
+	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
+		return -1;
+
+	strbuf_addf(&buf, "%s/patch", get_dir(opts));
+	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
+	init_revisions(&log_tree_opt, NULL);
+	log_tree_opt.abbrev = 0;
+	log_tree_opt.diff = 1;
+	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
+	log_tree_opt.disable_stdin = 1;
+	log_tree_opt.no_commit_id = 1;
+	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
+	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
+	if (!log_tree_opt.diffopt.file)
+		res |= error_errno(_("could not open '%s'"), buf.buf);
+	else {
+		res |= log_tree_commit(&log_tree_opt, commit);
+		fclose(log_tree_opt.diffopt.file);
+	}
+	strbuf_reset(&buf);
+
+	strbuf_addf(&buf, "%s/message", get_dir(opts));
+	if (!file_exists(buf.buf)) {
+		find_commit_subject(commit_buffer, &subject);
+		res |= write_message(subject, strlen(subject), buf.buf, 1);
+		unuse_commit_buffer(commit, commit_buffer);
+	}
+	strbuf_release(&buf);
+
+	return res;
+}
+
+static int intend_to_amend(void)
+{
+	unsigned char head[20];
+	char *p;
+
+	if (get_sha1("HEAD", head))
+		return error(_("cannot read HEAD"));
+
+	p = sha1_to_hex(head);
+	return write_message(p, strlen(p), rebase_path_amend(), 1);
+}
+
+static int error_with_patch(struct commit *commit,
+	const char *subject, int subject_len,
+	struct replay_opts *opts, int exit_code, int to_amend)
+{
+	if (make_patch(commit, opts))
+		return -1;
+
+	if (to_amend) {
+		if (intend_to_amend())
+			return -1;
+
+		fprintf(stderr, "You can amend the commit now, with\n"
+			"\n"
+			"  git commit --amend %s\n"
+			"\n"
+			"Once you are satisfied with your changes, run\n"
+			"\n"
+			"  git rebase --continue\n", gpg_sign_opt_quoted(opts));
+	}
+	else if (exit_code)
+		fprintf(stderr, "Could not apply %s... %.*s\n",
+			short_commit_name(commit), subject_len, subject);
+
+	return exit_code;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
-	int res;
+	int res = 0;
 
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	if (opts->allow_ff)
@@ -1316,9 +1411,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		if (item->command <= TODO_REVERT)
+		if (item->command <= TODO_EDIT) {
 			res = do_pick_commit(item->command, item->commit,
 					opts);
+			if (item->command == TODO_EDIT) {
+				struct commit *commit = item->commit;
+				if (!res)
+					warning(_("stopped at %s... %.*s"),
+						short_commit_name(commit),
+						item->arg_len, item->arg);
+				return error_with_patch(commit,
+					item->arg, item->arg_len, opts, res,
+					!res);
+			}
+		}
 		else if (item->command != TODO_NOOP)
 			return error(_("unknown command %d"), item->command);
 
@@ -1327,6 +1433,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			return res;
 	}
 
+	if (is_rebase_i(opts)) {
+		/* Stopped in the middle, as planned? */
+		if (todo_list->current < todo_list->nr)
+			return 0;
+	}
+
 	/*
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
-- 
2.11.0.rc3.windows.1

[PATCH v2 04/34] sequencer (rebase -i): implement the 'exec' command

[Johannes Schindelin]

The 'exec' command is a little special among rebase -i's commands, as it
does *not* have a SHA-1 as first parameter. Instead, everything after the
`exec` command is treated as command-line to execute.

Let's reuse the arg/arg_len fields of the todo_item structure (which hold
the oneline for pick/edit commands) to point to the command-line.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 68e2c84803..700b7575ed 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -17,6 +17,7 @@
 #include "argv-array.h"
 #include "quote.h"
 #include "log-tree.h"
+#include "wt-status.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -664,6 +665,8 @@ enum todo_command {
 	TODO_PICK = 0,
 	TODO_REVERT,
 	TODO_EDIT,
+	/* commands that do something else than handling a single commit */
+	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP
 };
@@ -672,6 +675,7 @@ static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
 	"edit",
+	"exec",
 	"noop"
 };
 
@@ -971,6 +975,12 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return -1;
 	bol += padding;
 
+	if (item->command == TODO_EXEC) {
+		item->arg = bol;
+		item->arg_len = (int)(eol - bol);
+		return 0;
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
@@ -1396,6 +1406,47 @@ static int error_with_patch(struct commit *commit,
 	return exit_code;
 }
 
+static int do_exec(const char *command_line)
+{
+	const char *child_argv[] = { NULL, NULL };
+	int dirty, status;
+
+	fprintf(stderr, "Executing: %s\n", command_line);
+	child_argv[0] = command_line;
+	status = run_command_v_opt(child_argv, RUN_USING_SHELL);
+
+	/* force re-reading of the cache */
+	if (discard_cache() < 0 || read_cache() < 0)
+		return error(_("could not read index"));
+
+	dirty = require_clean_work_tree("rebase", NULL, 1, 1);
+
+	if (status) {
+		warning(_("execution failed: %s\n%s"
+			  "You can fix the problem, and then run\n"
+			  "\n"
+			  "  git rebase --continue\n"
+			  "\n"),
+			command_line,
+			dirty ? N_("and made changes to the index and/or the "
+				"working tree\n") : "");
+		if (status == 127)
+			/* command not found */
+			status = 1;
+	}
+	else if (dirty) {
+		warning(_("execution succeeded: %s\nbut "
+			  "left changes to the index and/or the working tree\n"
+			  "Commit or stash your changes, and then run\n"
+			  "\n"
+			  "  git rebase --continue\n"
+			  "\n"), command_line);
+		status = 1;
+	}
+
+	return status;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1425,6 +1476,14 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					!res);
 			}
 		}
+		else if (item->command == TODO_EXEC) {
+			char *end_of_arg = (char *)(item->arg + item->arg_len);
+			int saved = *end_of_arg;
+
+			*end_of_arg = '\0';
+			res = do_exec(item->arg);
+			*end_of_arg = saved;
+		}
 		else if (item->command != TODO_NOOP)
 			return error(_("unknown command %d"), item->command);
 
-- 
2.11.0.rc3.windows.1

[PATCH v2 05/34] sequencer (rebase -i): learn about the 'verbose' mode

[Johannes Schindelin]

When calling `git rebase -i -v`, the user wants to see some statistics
after the commits were rebased. Let's show some.

The strbuf we use to perform that task will be used for other things
in subsequent commits, hence it is declared and initialized in a wider
scope than strictly needed here.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 22 ++++++++++++++++++++++
 sequencer.h |  1 +
 2 files changed, 23 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 700b7575ed..1ab50884bd 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -63,6 +63,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
  * command-line (and are only consumed, not modified, by the sequencer).
  */
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
+static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
+static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1121,6 +1123,9 @@ static int read_populate_opts(struct replay_opts *opts)
 		}
 		strbuf_release(&buf);
 
+		if (file_exists(rebase_path_verbose()))
+			opts->verbose = 1;
+
 		return 0;
 	}
 
@@ -1493,9 +1498,26 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	}
 
 	if (is_rebase_i(opts)) {
+		struct strbuf buf = STRBUF_INIT;
+
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
 			return 0;
+
+		if (opts->verbose) {
+			const char *argv[] = {
+				"diff-tree", "--stat", NULL, NULL
+			};
+
+			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
+				return error(_("could not read '%s'"),
+					rebase_path_orig_head());
+			strbuf_addstr(&buf, "..HEAD");
+			argv[2] = buf.buf;
+			run_command_v_opt(argv, RUN_GIT_CMD);
+			strbuf_reset(&buf);
+		}
+		strbuf_release(&buf);
 	}
 
 	/*
diff --git a/sequencer.h b/sequencer.h
index cb21cfddee..f885b68395 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -24,6 +24,7 @@ struct replay_opts {
 	int allow_empty;
 	int allow_empty_message;
 	int keep_redundant_commits;
+	int verbose;
 
 	int mainline;
 
-- 
2.11.0.rc3.windows.1

[PATCH v2 06/34] sequencer (rebase -i): write the 'done' file

[Johannes Schindelin]

In the interactive rebase, commands that were successfully processed are
not simply discarded, but appended to the 'done' file instead. This is
used e.g. to display the current state to the user in the output of
`git status` or the progress.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 1ab50884bd..f6e20b142a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -39,6 +39,12 @@ static GIT_PATH_FUNC(rebase_path, "rebase-merge")
  */
 static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
 /*
+ * The rebase command lines that have already been processed. A line
+ * is moved here when it is first handled, before any associated user
+ * actions.
+ */
+static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
+/*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
  * being rebased.
@@ -1295,6 +1301,23 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 		return error_errno(_("could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
 		return error(_("failed to finalize '%s'."), todo_path);
+
+	if (is_rebase_i(opts)) {
+		const char *done_path = rebase_path_done();
+		int fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
+		int prev_offset = !next ? 0 :
+			todo_list->items[next - 1].offset_in_buf;
+
+		if (fd >= 0 && offset > prev_offset &&
+		    write_in_full(fd, todo_list->buf.buf + prev_offset,
+				  offset - prev_offset) < 0) {
+			close(fd);
+			return error_errno(_("could not write to '%s'"),
+					   done_path);
+		}
+		if (fd >= 0)
+			close(fd);
+	}
 	return 0;
 }
 
-- 
2.11.0.rc3.windows.1

[PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Johannes Schindelin]

This is a huge patch, and at the same time a huge step forward to
execute the performance-critical parts of the interactive rebase in a
builtin command.

Since 'fixup' and 'squash' are not only similar, but also need to know
about each other (we want to reduce a series of fixups/squashes into a
single, final commit message edit, from the user's point of view), we
really have to implement them both at the same time.

Most of the actual work is done by the existing code path that already
handles the "pick" and the "edit" commands; We added support for other
features (e.g. to amend the commit message) in the patches leading up to
this one, yet there are still quite a few bits in this patch that simply
would not make sense as individual patches (such as: determining whether
there was anything to "fix up" in the "todo" script, etc).

In theory, it would be possible to reuse the fast-forward code path also
for the fixup and the squash code paths, but in practice this would make
the code less readable. The end result cannot be fast-forwarded anyway,
therefore let's just extend the cherry-picking code path for now.

Since the sequencer parses the entire `git-rebase-todo` script in one go,
fixup or squash commands without a preceding pick can be reported early
(in git-rebase--interactive, we could only report such errors just before
executing the fixup/squash).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 232 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 222 insertions(+), 10 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index f6e20b142a..271c21581d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -45,6 +45,35 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
 /*
+ * The commit message that is planned to be used for any changes that
+ * need to be committed following a user interaction.
+ */
+static GIT_PATH_FUNC(rebase_path_message, "rebase-merge/message")
+/*
+ * The file into which is accumulated the suggested commit message for
+ * squash/fixup commands. When the first of a series of squash/fixups
+ * is seen, the file is created and the commit message from the
+ * previous commit and from the first squash/fixup commit are written
+ * to it. The commit message for each subsequent squash/fixup commit
+ * is appended to the file as it is processed.
+ *
+ * The first line of the file is of the form
+ *     # This is a combination of $count commits.
+ * where $count is the number of commits whose messages have been
+ * written to the file so far (including the initial "pick" commit).
+ * Each time that a commit message is processed, this line is read and
+ * updated. It is deleted just before the combined commit is made.
+ */
+static GIT_PATH_FUNC(rebase_path_squash_msg, "rebase-merge/message-squash")
+/*
+ * If the current series of squash/fixups has not yet included a squash
+ * command, then this file exists and holds the commit message of the
+ * original "pick" commit.  (If the series ends without a "squash"
+ * command, then this can be used as the commit message of the combined
+ * commit without opening the editor.)
+ */
+static GIT_PATH_FUNC(rebase_path_fixup_msg, "rebase-merge/message-fixup")
+/*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
  * being rebased.
@@ -673,6 +702,8 @@ enum todo_command {
 	TODO_PICK = 0,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_FIXUP,
+	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
@@ -683,6 +714,8 @@ static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
 	"edit",
+	"fixup",
+	"squash",
 	"exec",
 	"noop"
 };
@@ -694,16 +727,119 @@ static const char *command_to_string(const enum todo_command command)
 	die("Unknown command: %d", command);
 }
 
+static int is_fixup(enum todo_command command)
+{
+	return command == TODO_FIXUP || command == TODO_SQUASH;
+}
+
+static int update_squash_messages(enum todo_command command,
+		struct commit *commit, struct replay_opts *opts)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int count, res;
+	const char *message, *body;
+
+	if (file_exists(rebase_path_squash_msg())) {
+		char *p, *p2;
+
+		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
+			return error(_("could not read '%s'"),
+				rebase_path_squash_msg());
+
+		if (buf.buf[0] != comment_line_char ||
+		    !skip_prefix(buf.buf + 1, " This is a combination of ",
+				 (const char **)&p))
+			return error(_("unexpected 1st line of squash message:"
+				       "\n\n\t%.*s"),
+				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
+				     buf.buf);
+		count = strtol(p, &p2, 10);
+
+		if (count < 1 || *p2 != ' ')
+			return error(_("invalid 1st line of squash message:\n"
+				       "\n\t%.*s"),
+				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
+				     buf.buf);
+
+		sprintf((char *)p, "%d", ++count);
+		if (!*p2)
+			*p2 = ' ';
+		else {
+			*(++p2) = 'c';
+			strbuf_insert(&buf, p2 - buf.buf, " ", 1);
+		}
+	}
+	else {
+		unsigned char head[20];
+		struct commit *head_commit;
+		const char *head_message, *body;
+
+		if (get_sha1("HEAD", head))
+			return error(_("need a HEAD to fixup"));
+		if (!(head_commit = lookup_commit_reference(head)))
+			return error(_("could not read HEAD"));
+		if (!(head_message = get_commit_buffer(head_commit, NULL)))
+			return error(_("could not read HEAD's commit message"));
+
+		body = strstr(head_message, "\n\n");
+		if (!body)
+			body = "";
+		else
+			body = skip_blank_lines(body + 2);
+		if (write_message(body, strlen(body),
+				  rebase_path_fixup_msg(), 0))
+			return error(_("cannot write '%s'"),
+				     rebase_path_fixup_msg());
+
+		count = 2;
+		strbuf_addf(&buf, _("%c This is a combination of 2 commits.\n"
+				    "%c The first commit's message is:\n\n%s"),
+			    comment_line_char, comment_line_char, body);
+
+		unuse_commit_buffer(head_commit, head_message);
+	}
+
+	if (!(message = get_commit_buffer(commit, NULL)))
+		return error(_("could not read commit message of %s"),
+			     oid_to_hex(&commit->object.oid));
+	body = strstr(message, "\n\n");
+	if (!body)
+		body = "";
+	else
+		body = skip_blank_lines(body + 2);
+
+	if (command == TODO_SQUASH) {
+		unlink(rebase_path_fixup_msg());
+		strbuf_addf(&buf, _("\n%c This is the commit message #%d:\n"
+				    "\n%s"),
+			    comment_line_char, count, body);
+	}
+	else if (command == TODO_FIXUP) {
+		strbuf_addf(&buf, _("\n%c The commit message #%d "
+				    "will be skipped:\n\n"),
+			    comment_line_char, count);
+		strbuf_add_commented_lines(&buf, body, strlen(body));
+	}
+	else
+		return error(_("unknown command: %d"), command);
+	unuse_commit_buffer(commit, message);
+
+	res = write_message(buf.buf, buf.len, rebase_path_squash_msg(), 0);
+	strbuf_release(&buf);
+	return res;
+}
 
 static int do_pick_commit(enum todo_command command, struct commit *commit,
-		struct replay_opts *opts)
+		struct replay_opts *opts, int final_fixup)
 {
+	int edit = opts->edit, cleanup_commit_message = 0;
+	const char *msg_file = edit ? NULL : git_path_merge_msg();
 	unsigned char head[20];
 	struct commit *base, *next, *parent;
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res, unborn = 0, allow;
+	int res, unborn = 0, amend = 0, allow;
 
 	if (opts->no_commit) {
 		/*
@@ -749,7 +885,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	else
 		parent = commit->parents->item;
 
-	if (opts->allow_ff &&
+	if (opts->allow_ff && !is_fixup(command) &&
 	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
 	     (!parent && unborn)))
 		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
@@ -813,6 +949,28 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
+	if (is_fixup(command)) {
+		if (update_squash_messages(command, commit, opts))
+			return -1;
+		amend = 1;
+		if (!final_fixup)
+			msg_file = rebase_path_squash_msg();
+		else if (file_exists(rebase_path_fixup_msg())) {
+			cleanup_commit_message = 1;
+			msg_file = rebase_path_fixup_msg();
+		}
+		else {
+			const char *dest = git_path("SQUASH_MSG");
+			unlink(dest);
+			if (copy_file(dest, rebase_path_squash_msg(), 0666))
+				return error(_("could not rename '%s' to '%s'"),
+					     rebase_path_squash_msg(), dest);
+			unlink(git_path("MERGE_MSG"));
+			msg_file = dest;
+			edit = 1;
+		}
+	}
+
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
@@ -868,8 +1026,13 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
-		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
-				     opts, allow, opts->edit, 0, 0);
+		res = run_git_commit(msg_file, opts, allow, edit, amend,
+				     cleanup_commit_message);
+
+	if (!res && final_fixup) {
+		unlink(rebase_path_fixup_msg());
+		unlink(rebase_path_squash_msg());
+	}
 
 leave:
 	free_message(commit, &msg);
@@ -1009,7 +1172,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
 	struct todo_item *item;
 	char *p = buf, *next_p;
-	int i, res = 0;
+	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
 
 	for (i = 1; *p; i++, p = next_p) {
 		char *eol = strchrnul(p, '\n');
@@ -1024,8 +1187,16 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		if (parse_insn_line(item, p, eol)) {
 			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
-			item->command = -1;
+			item->command = TODO_NOOP;
 		}
+
+		if (fixup_okay)
+			; /* do nothing */
+		else if (is_fixup(item->command))
+			return error(_("cannot '%s' without a previous commit"),
+				command_to_string(item->command));
+		else if (item->command != TODO_NOOP)
+			fixup_okay = 1;
 	}
 	if (!todo_list->nr)
 		return error(_("no commits parsed."));
@@ -1434,6 +1605,20 @@ static int error_with_patch(struct commit *commit,
 	return exit_code;
 }
 
+static int error_failed_squash(struct commit *commit,
+	struct replay_opts *opts, int subject_len, const char *subject)
+{
+	if (rename(rebase_path_squash_msg(), rebase_path_message()))
+		return error(_("could not rename '%s' to '%s'"),
+			rebase_path_squash_msg(), rebase_path_message());
+	unlink(rebase_path_fixup_msg());
+	unlink(git_path("MERGE_MSG"));
+	if (copy_file(git_path("MERGE_MSG"), rebase_path_message(), 0666))
+		return error(_("could not copy '%s' to '%s'"),
+			     rebase_path_message(), git_path("MERGE_MSG"));
+	return error_with_patch(commit, subject, subject_len, opts, 1, 0);
+}
+
 static int do_exec(const char *command_line)
 {
 	const char *child_argv[] = { NULL, NULL };
@@ -1475,6 +1660,21 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int is_final_fixup(struct todo_list *todo_list)
+{
+	int i = todo_list->current;
+
+	if (!is_fixup(todo_list->items[i].command))
+		return 0;
+
+	while (++i < todo_list->nr)
+		if (is_fixup(todo_list->items[i].command))
+			return 0;
+		else if (todo_list->items[i].command < TODO_NOOP)
+			break;
+	return 1;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1490,9 +1690,15 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		if (item->command <= TODO_EDIT) {
+		if (is_rebase_i(opts)) {
+			unlink(rebase_path_message());
+			unlink(rebase_path_author_script());
+			unlink(rebase_path_stopped_sha());
+			unlink(rebase_path_amend());
+		}
+		if (item->command <= TODO_SQUASH) {
 			res = do_pick_commit(item->command, item->commit,
-					opts);
+					opts, is_final_fixup(todo_list));
 			if (item->command == TODO_EDIT) {
 				struct commit *commit = item->commit;
 				if (!res)
@@ -1503,6 +1709,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+			if (res && is_fixup(item->command)) {
+				if (res == 1)
+					intend_to_amend();
+				return error_failed_squash(item->commit, opts,
+					item->arg_len, item->arg);
+			}
 		}
 		else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
@@ -1597,7 +1809,7 @@ static int single_pick(struct commit *cmit, struct replay_opts *opts)
 {
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	return do_pick_commit(opts->action == REPLAY_PICK ?
-		TODO_PICK : TODO_REVERT, cmit, opts);
+		TODO_PICK : TODO_REVERT, cmit, opts, 0);
 }
 
 int sequencer_pick_revisions(struct replay_opts *opts)
-- 
2.11.0.rc3.windows.1

[PATCH v2 08/34] sequencer (rebase -i): implement the short commands

[Johannes Schindelin]

For users' convenience, most rebase commands can be abbreviated, e.g.
'p' instead of 'pick' and 'x' instead of 'exec'. Let's teach the
sequencer to handle those abbreviated commands just fine.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 36 ++++++++++++++++++++++--------------
 1 file changed, 22 insertions(+), 14 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 271c21581d..e443f4765d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -710,20 +710,23 @@ enum todo_command {
 	TODO_NOOP
 };
 
-static const char *todo_command_strings[] = {
-	"pick",
-	"revert",
-	"edit",
-	"fixup",
-	"squash",
-	"exec",
-	"noop"
+static struct {
+	char c;
+	const char *str;
+} todo_command_info[] = {
+	{ 'p', "pick" },
+	{ 0,   "revert" },
+	{ 'e', "edit" },
+	{ 'f', "fixup" },
+	{ 's', "squash" },
+	{ 'x', "exec" },
+	{ 0,   "noop" }
 };
 
 static const char *command_to_string(const enum todo_command command)
 {
-	if ((size_t)command < ARRAY_SIZE(todo_command_strings))
-		return todo_command_strings[command];
+	if ((size_t)command < ARRAY_SIZE(todo_command_info))
+		return todo_command_info[command].str;
 	die("Unknown command: %d", command);
 }
 
@@ -1125,12 +1128,17 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return 0;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
-		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
+	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)
+		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
 			item->command = i;
 			break;
 		}
-	if (i >= ARRAY_SIZE(todo_command_strings))
+		else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
+			bol++;
+			item->command = i;
+			break;
+		}
+	if (i >= ARRAY_SIZE(todo_command_info))
 		return -1;
 
 	if (item->command == TODO_NOOP) {
@@ -1325,7 +1333,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 {
 	enum todo_command command = opts->action == REPLAY_PICK ?
 		TODO_PICK : TODO_REVERT;
-	const char *command_string = todo_command_strings[command];
+	const char *command_string = todo_command_info[command].str;
 	struct commit *commit;
 
 	if (prepare_revs(opts))
-- 
2.11.0.rc3.windows.1

[PATCH v2 09/34] sequencer (rebase -i): write an author-script file

[Johannes Schindelin]

When the interactive rebase aborts, it writes out an author-script file
to record the author information for the current commit. As we are about
to teach the sequencer how to perform the actions behind an interactive
rebase, it needs to write those author-script files, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 50 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index e443f4765d..80469b6954 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -515,6 +515,53 @@ static int is_index_unchanged(void)
 	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
 }
 
+static int write_author_script(const char *message)
+{
+	struct strbuf buf = STRBUF_INIT;
+	const char *eol;
+	int res;
+
+	for (;;)
+		if (!*message || starts_with(message, "\n")) {
+missing_author:
+			/* Missing 'author' line? */
+			unlink(rebase_path_author_script());
+			return 0;
+		}
+		else if (skip_prefix(message, "author ", &message))
+			break;
+		else if ((eol = strchr(message, '\n')))
+			message = eol + 1;
+		else
+			goto missing_author;
+
+	strbuf_addstr(&buf, "GIT_AUTHOR_NAME='");
+	while (*message && *message != '\n' && *message != '\r')
+		if (skip_prefix(message, " <", &message))
+			break;
+		else if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\nGIT_AUTHOR_EMAIL='");
+	while (*message && *message != '\n' && *message != '\r')
+		if (skip_prefix(message, "> ", &message))
+			break;
+		else if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\nGIT_AUTHOR_DATE='@");
+	while (*message && *message != '\n' && *message != '\r')
+		if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	res = write_message(buf.buf, buf.len, rebase_path_author_script(), 1);
+	strbuf_release(&buf);
+	return res;
+}
+
 /*
  * Read the author-script file into an environment block, ready for use in
  * run_command(), that can be free()d afterwards.
@@ -974,7 +1021,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
-	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
+	if (is_rebase_i(opts) && write_author_script(msg.message) < 0)
+		res = -1;
+	else if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
 		if (res < 0)
-- 
2.11.0.rc3.windows.1

[PATCH v2 10/34] sequencer (rebase -i): allow continuing with staged changes

[Johannes Schindelin]

When an interactive rebase is interrupted, the user may stage changes
before continuing, and we need to commit those changes in that case.

Please note that the nested "if" added to the sequencer_continue() is
not combined into a single "if" because it will be extended with an
"else" clause in a later patch in this patch series.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 80469b6954..855d3ba503 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1829,6 +1829,42 @@ static int continue_single_pick(void)
 	return run_command_v_opt(argv, RUN_GIT_CMD);
 }
 
+static int commit_staged_changes(struct replay_opts *opts)
+{
+	int amend = 0;
+
+	if (has_unstaged_changes(1))
+		return error(_("cannot rebase: You have unstaged changes."));
+	if (!has_uncommitted_changes(0))
+		return 0;
+
+	if (file_exists(rebase_path_amend())) {
+		struct strbuf rev = STRBUF_INIT;
+		unsigned char head[20], to_amend[20];
+
+		if (get_sha1("HEAD", head))
+			return error(_("cannot amend non-existing commit"));
+		if (!read_oneliner(&rev, rebase_path_amend(), 0))
+			return error(_("invalid file: '%s'"), rebase_path_amend());
+		if (get_sha1_hex(rev.buf, to_amend))
+			return error(_("invalid contents: '%s'"),
+				rebase_path_amend());
+		if (hashcmp(head, to_amend))
+			return error(_("\nYou have uncommitted changes in your "
+				       "working tree. Please, commit them\n"
+				       "first and then run 'git rebase "
+				       "--continue' again."));
+
+		strbuf_release(&rev);
+		amend = 1;
+	}
+
+	if (run_git_commit(rebase_path_message(), opts, 1, 1, amend, 0))
+		return error(_("could not commit staged changes."));
+	unlink(rebase_path_amend());
+	return 0;
+}
+
 int sequencer_continue(struct replay_opts *opts)
 {
 	struct todo_list todo_list = TODO_LIST_INIT;
@@ -1837,6 +1873,10 @@ int sequencer_continue(struct replay_opts *opts)
 	if (read_and_refresh_cache(opts))
 		return -1;
 
+	if (is_rebase_i(opts)) {
+		if (commit_staged_changes(opts))
+			return -1;
+	}
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts))
-- 
2.11.0.rc3.windows.1

[PATCH v2 11/34] sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed

[Johannes Schindelin]

The scripted version of the interactive rebase already does that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 855d3ba503..abffaf3b40 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1835,8 +1835,13 @@ static int commit_staged_changes(struct replay_opts *opts)
 
 	if (has_unstaged_changes(1))
 		return error(_("cannot rebase: You have unstaged changes."));
-	if (!has_uncommitted_changes(0))
+	if (!has_uncommitted_changes(0)) {
+		const char *cherry_pick_head = git_path("CHERRY_PICK_HEAD");
+
+		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
+			return error(_("could not remove CHERRY_PICK_HEAD"));
 		return 0;
+	}
 
 	if (file_exists(rebase_path_amend())) {
 		struct strbuf rev = STRBUF_INIT;
-- 
2.11.0.rc3.windows.1

[PATCH v2 12/34] sequencer (rebase -i): skip some revert/cherry-pick specific code path

[Johannes Schindelin]

When a cherry-pick continues without a "todo script", the intention is
simply to pick a single commit.

However, when an interactive rebase is continued without a "todo
script", it means that the last command has been completed and that we
now need to clean up.

This commit guards the revert/cherry-pick specific steps so that they
are not executed in rebase -i mode.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index abffaf3b40..4ceb6f3ac5 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1882,25 +1882,28 @@ int sequencer_continue(struct replay_opts *opts)
 		if (commit_staged_changes(opts))
 			return -1;
 	}
-	if (!file_exists(get_todo_path(opts)))
+	else if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts))
 		return -1;
 	if ((res = read_populate_todo(&todo_list, opts)))
 		goto release_todo_list;
 
-	/* Verify that the conflict has been resolved */
-	if (file_exists(git_path_cherry_pick_head()) ||
-	    file_exists(git_path_revert_head())) {
-		res = continue_single_pick();
-		if (res)
+	if (!is_rebase_i(opts)) {
+		/* Verify that the conflict has been resolved */
+		if (file_exists(git_path_cherry_pick_head()) ||
+		    file_exists(git_path_revert_head())) {
+			res = continue_single_pick();
+			if (res)
+				goto release_todo_list;
+		}
+		if (index_differs_from("HEAD", 0, 0)) {
+			res = error_dirty_index(opts);
 			goto release_todo_list;
+		}
+		todo_list.current++;
 	}
-	if (index_differs_from("HEAD", 0, 0)) {
-		res = error_dirty_index(opts);
-		goto release_todo_list;
-	}
-	todo_list.current++;
+
 	res = pick_commits(&todo_list, opts);
 release_todo_list:
 	todo_list_release(&todo_list);
-- 
2.11.0.rc3.windows.1

[PATCH v2 13/34] sequencer (rebase -i): the todo can be empty when continuing

[Johannes Schindelin]

When the last command of an interactive rebase fails, the user needs to
resolve the problem and then continue the interactive rebase. Naturally,
the todo script is empty by then. So let's not complain about that!

To that end, let's move that test out of the function that parses the
todo script, and into the more high-level function read_populate_todo().
This is also necessary by now because the lower-level parse_insn_buffer()
has no idea whether we are performing an interactive rebase or not.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 4ceb6f3ac5..a6625e765d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1255,8 +1255,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		else if (item->command != TODO_NOOP)
 			fixup_okay = 1;
 	}
-	if (!todo_list->nr)
-		return error(_("no commits parsed."));
+
 	return res;
 }
 
@@ -1280,6 +1279,10 @@ static int read_populate_todo(struct todo_list *todo_list,
 	if (res)
 		return error(_("unusable instruction sheet: '%s'"), todo_file);
 
+	if (!todo_list->nr &&
+	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
+		return error(_("no commits parsed."));
+
 	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
 			opts->action == REPLAY_PICK ? TODO_PICK : TODO_REVERT;
-- 
2.11.0.rc3.windows.1

[PATCH v2 14/34] sequencer (rebase -i): update refs after a successful rebase

[Johannes Schindelin]

An interactive rebase operates on a detached HEAD (to keep the reflog
of the original branch relatively clean), and updates the branch only
at the end.

Now that the sequencer learns to perform interactive rebases, it also
needs to learn the trick to update the branch before removing the
directory containing the state of the interactive rebase.

We introduce a new head_ref variable in a wider scope than necessary at
the moment, to allow for a later patch that prints out "Successfully
rebased and updated <ref>".

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 32 +++++++++++++++++++++++++++++++-
 1 file changed, 31 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index a6625e765d..a4e9b326ba 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -100,6 +100,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
 static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
+static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
+static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1793,12 +1795,39 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	}
 
 	if (is_rebase_i(opts)) {
-		struct strbuf buf = STRBUF_INIT;
+		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
 
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
 			return 0;
 
+		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
+				starts_with(head_ref.buf, "refs/")) {
+			unsigned char head[20], orig[20];
+
+			if (get_sha1("HEAD", head))
+				return error(_("cannot read HEAD"));
+			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
+					get_sha1_hex(buf.buf, orig))
+				return error(_("could not read orig-head"));
+			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
+				head_ref.buf);
+			if (!read_oneliner(&buf, rebase_path_onto(), 0))
+				return error(_("could not read 'onto'"));
+			if (update_ref(buf.buf, head_ref.buf, head, orig,
+					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
+				return error(_("could not update %s"),
+					head_ref.buf);
+			strbuf_reset(&buf);
+			strbuf_addf(&buf,
+				"rebase -i (finish): returning to %s",
+				head_ref.buf);
+			if (create_symref("HEAD", head_ref.buf, buf.buf))
+				return error(_("could not update HEAD to %s"),
+					head_ref.buf);
+			strbuf_reset(&buf);
+		}
+
 		if (opts->verbose) {
 			const char *argv[] = {
 				"diff-tree", "--stat", NULL, NULL
@@ -1813,6 +1842,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			strbuf_reset(&buf);
 		}
 		strbuf_release(&buf);
+		strbuf_release(&head_ref);
 	}
 
 	/*
-- 
2.11.0.rc3.windows.1

[PATCH v2 15/34] sequencer (rebase -i): leave a patch upon error

[Johannes Schindelin]

When doing an interactive rebase, we want to leave a 'patch' file for
further inspection by the user (even if we never tried to actually apply
that patch, since we're cherry-picking instead).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index a4e9b326ba..4361fe0e94 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1777,6 +1777,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				return error_failed_squash(item->commit, opts,
 					item->arg_len, item->arg);
 			}
+			else if (res && is_rebase_i(opts))
+				return res | error_with_patch(item->commit,
+					item->arg, item->arg_len, opts, res, 0);
 		}
 		else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
-- 
2.11.0.rc3.windows.1

[PATCH v2 16/34] sequencer (rebase -i): implement the 'reword' command

[Johannes Schindelin]

This is now trivial, as all the building blocks are in place: all we need
to do is to flip the "edit" switch when committing.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 4361fe0e94..5a9972fec3 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -751,6 +751,7 @@ enum todo_command {
 	TODO_PICK = 0,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_REWORD,
 	TODO_FIXUP,
 	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
@@ -766,6 +767,7 @@ static struct {
 	{ 'p', "pick" },
 	{ 0,   "revert" },
 	{ 'e', "edit" },
+	{ 'r', "reword" },
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
@@ -1001,7 +1003,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
-	if (is_fixup(command)) {
+	if (command == TODO_REWORD)
+		edit = 1;
+	else if (is_fixup(command)) {
 		if (update_squash_messages(command, commit, opts))
 			return -1;
 		amend = 1;
@@ -1779,7 +1783,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			}
 			else if (res && is_rebase_i(opts))
 				return res | error_with_patch(item->commit,
-					item->arg, item->arg_len, opts, res, 0);
+					item->arg, item->arg_len, opts, res,
+					item->command == TODO_REWORD);
 		}
 		else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
-- 
2.11.0.rc3.windows.1

[PATCH v2 17/34] sequencer (rebase -i): allow fast-forwarding for edit/reword

[Johannes Schindelin]

The sequencer already knew how to fast-forward instead of
cherry-picking, if possible.

We want to continue to do this, of course, but in case of the 'reword'
command, we will need to call `git commit` after fast-forwarding.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 5a9972fec3..33fb36dcbe 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -893,7 +893,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res, unborn = 0, amend = 0, allow;
+	int res, unborn = 0, amend = 0, allow = 0;
 
 	if (opts->no_commit) {
 		/*
@@ -939,11 +939,23 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	else
 		parent = commit->parents->item;
 
+	if (get_message(commit, &msg) != 0)
+		return error(_("cannot get commit message for %s"),
+			oid_to_hex(&commit->object.oid));
+
 	if (opts->allow_ff && !is_fixup(command) &&
 	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
-	     (!parent && unborn)))
-		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
-
+	     (!parent && unborn))) {
+		if (is_rebase_i(opts))
+			write_author_script(msg.message);
+		res = fast_forward_to(commit->object.oid.hash, head, unborn,
+			opts);
+		if (res || command != TODO_REWORD)
+			goto leave;
+		edit = amend = 1;
+		msg_file = NULL;
+		goto fast_forward_edit;
+	}
 	if (parent && parse_commit(parent) < 0)
 		/* TRANSLATORS: The first %s will be a "todo" command like
 		   "revert" or "pick", the second %s a SHA1. */
@@ -951,10 +963,6 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 			command_to_string(command),
 			oid_to_hex(&parent->object.oid));
 
-	if (get_message(commit, &msg) != 0)
-		return error(_("cannot get commit message for %s"),
-			oid_to_hex(&commit->object.oid));
-
 	/*
 	 * "commit" is an existing commit.  We would want to apply
 	 * the difference it introduces since its first parent "prev"
@@ -1084,6 +1092,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
+fast_forward_edit:
 		res = run_git_commit(msg_file, opts, allow, edit, amend,
 				     cleanup_commit_message);
 
-- 
2.11.0.rc3.windows.1

[PATCH v2 18/34] sequencer (rebase -i): refactor setting the reflog message

[Johannes Schindelin]

This makes the code DRYer, with the obvious benefit that we can enhance
the code further in a single place.

We can also reuse the functionality elsewhere by calling this new
function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 ++++++++++++++++++++++++++-------
 1 file changed, 26 insertions(+), 7 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 33fb36dcbe..d20efad562 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1750,6 +1750,26 @@ static int is_final_fixup(struct todo_list *todo_list)
 	return 1;
 }
 
+static const char *reflog_message(struct replay_opts *opts,
+	const char *sub_action, const char *fmt, ...)
+{
+	va_list ap;
+	static struct strbuf buf = STRBUF_INIT;
+
+	va_start(ap, fmt);
+	strbuf_reset(&buf);
+	strbuf_addstr(&buf, action_name(opts));
+	if (sub_action)
+		strbuf_addf(&buf, " (%s)", sub_action);
+	if (fmt) {
+		strbuf_addstr(&buf, ": ");
+		strbuf_vaddf(&buf, fmt, ap);
+	}
+	va_end(ap);
+
+	return buf.buf;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1820,6 +1840,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 
 		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
 				starts_with(head_ref.buf, "refs/")) {
+			const char *msg;
 			unsigned char head[20], orig[20];
 
 			if (get_sha1("HEAD", head))
@@ -1827,19 +1848,17 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
 					get_sha1_hex(buf.buf, orig))
 				return error(_("could not read orig-head"));
-			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
-				head_ref.buf);
 			if (!read_oneliner(&buf, rebase_path_onto(), 0))
 				return error(_("could not read 'onto'"));
-			if (update_ref(buf.buf, head_ref.buf, head, orig,
+			msg = reflog_message(opts, "finish", "%s onto %s",
+				head_ref.buf, buf.buf);
+			if (update_ref(msg, head_ref.buf, head, orig,
 					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 				return error(_("could not update %s"),
 					head_ref.buf);
-			strbuf_reset(&buf);
-			strbuf_addf(&buf,
-				"rebase -i (finish): returning to %s",
+			msg = reflog_message(opts, "finish", "returning to %s",
 				head_ref.buf);
-			if (create_symref("HEAD", head_ref.buf, buf.buf))
+			if (create_symref("HEAD", head_ref.buf, msg))
 				return error(_("could not update HEAD to %s"),
 					head_ref.buf);
 			strbuf_reset(&buf);
-- 
2.11.0.rc3.windows.1

[PATCH v2 19/34] sequencer (rebase -i): set the reflog message consistently

[Johannes Schindelin]

We already used the same reflog message as the scripted version of rebase
-i when finishing. With this commit, we do that also for all the commands
before that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index d20efad562..118696f6d3 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1792,6 +1792,10 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			unlink(rebase_path_amend());
 		}
 		if (item->command <= TODO_SQUASH) {
+			if (is_rebase_i(opts))
+				setenv("GIT_REFLOG_ACTION", reflog_message(opts,
+					command_to_string(item->command), NULL),
+					1);
 			res = do_pick_commit(item->command, item->commit,
 					opts, is_final_fixup(todo_list));
 			if (item->command == TODO_EDIT) {
-- 
2.11.0.rc3.windows.1

[PATCH v2 20/34] sequencer (rebase -i): copy commit notes at end

[Johannes Schindelin]

When rebasing commits that have commit notes attached, the interactive
rebase rewrites those notes faithfully at the end. The sequencer must
do this, too, if it wishes to do interactive rebase's job.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 76 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 118696f6d3..7ab533abd9 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -94,6 +94,15 @@ static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
  */
 static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 /*
+ * For the post-rewrite hook, we make a list of rewritten commits and
+ * their new sha1s.  The rewritten-pending list keeps the sha1s of
+ * commits that have been processed, but not committed yet,
+ * e.g. because they are waiting for a 'squash' command.
+ */
+static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
+static GIT_PATH_FUNC(rebase_path_rewritten_pending,
+	"rebase-merge/rewritten-pending")
+/*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
  */
@@ -883,6 +892,44 @@ static int update_squash_messages(enum todo_command command,
 	return res;
 }
 
+static void flush_rewritten_pending(void) {
+	struct strbuf buf = STRBUF_INIT;
+	unsigned char newsha1[20];
+	FILE *out;
+
+	if (strbuf_read_file(&buf, rebase_path_rewritten_pending(), 82) > 0 &&
+			!get_sha1("HEAD", newsha1) &&
+			(out = fopen(rebase_path_rewritten_list(), "a"))) {
+		char *bol = buf.buf, *eol;
+
+		while (*bol) {
+			eol = strchrnul(bol, '\n');
+			fprintf(out, "%.*s %s\n", (int)(eol - bol),
+					bol, sha1_to_hex(newsha1));
+			if (!*eol)
+				break;
+			bol = eol + 1;
+		}
+		fclose(out);
+		unlink(rebase_path_rewritten_pending());
+	}
+	strbuf_release(&buf);
+}
+
+static void record_in_rewritten(struct object_id *oid,
+		enum todo_command next_command) {
+	FILE *out = fopen(rebase_path_rewritten_pending(), "a");
+
+	if (!out)
+		return;
+
+	fprintf(out, "%s\n", oid_to_hex(oid));
+	fclose(out);
+
+	if (!is_fixup(next_command))
+		flush_rewritten_pending();
+}
+
 static int do_pick_commit(enum todo_command command, struct commit *commit,
 		struct replay_opts *opts, int final_fixup)
 {
@@ -1750,6 +1797,17 @@ static int is_final_fixup(struct todo_list *todo_list)
 	return 1;
 }
 
+static enum todo_command peek_command(struct todo_list *todo_list, int offset)
+{
+	int i;
+
+	for (i = todo_list->current + offset; i < todo_list->nr; i++)
+		if (todo_list->items[i].command != TODO_NOOP)
+			return todo_list->items[i].command;
+
+	return -1;
+}
+
 static const char *reflog_message(struct replay_opts *opts,
 	const char *sub_action, const char *fmt, ...)
 {
@@ -1808,6 +1866,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+			if (is_rebase_i(opts) && !res)
+				record_in_rewritten(&item->commit->object.oid,
+					peek_command(todo_list, 1));
 			if (res && is_fixup(item->command)) {
 				if (res == 1)
 					intend_to_amend();
@@ -1837,6 +1898,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 
 	if (is_rebase_i(opts)) {
 		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
+		struct stat st;
 
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
@@ -1881,6 +1943,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			run_command_v_opt(argv, RUN_GIT_CMD);
 			strbuf_reset(&buf);
 		}
+		flush_rewritten_pending();
+		if (!stat(rebase_path_rewritten_list(), &st) &&
+				st.st_size > 0) {
+			struct child_process child = CHILD_PROCESS_INIT;
+
+			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			child.git_cmd = 1;
+			argv_array_push(&child.args, "notes");
+			argv_array_push(&child.args, "copy");
+			argv_array_push(&child.args, "--for-rewrite=rebase");
+			/* we don't care if this copying failed */
+			run_command(&child);
+		}
+
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
 	}
-- 
2.11.0.rc3.windows.1

[PATCH v2 21/34] sequencer (rebase -i): record interrupted commits in rewritten, too

[Johannes Schindelin]

When continuing after a `pick` command failed, we want that commit
to show up in the rewritten-list (and its notes to be rewritten), too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 7ab533abd9..0233999389 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2052,6 +2052,15 @@ int sequencer_continue(struct replay_opts *opts)
 		}
 		todo_list.current++;
 	}
+	else if (file_exists(rebase_path_stopped_sha())) {
+		struct strbuf buf = STRBUF_INIT;
+		struct object_id oid;
+
+		if (read_oneliner(&buf, rebase_path_stopped_sha(), 1) &&
+		    !get_sha1_committish(buf.buf, oid.hash))
+			record_in_rewritten(&oid, peek_command(&todo_list, 0));
+		strbuf_release(&buf);
+	}
 
 	res = pick_commits(&todo_list, opts);
 release_todo_list:
-- 
2.11.0.rc3.windows.1

[PATCH v2 22/34] sequencer (rebase -i): run the post-rewrite hook, if needed

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 0233999389..cafd945e83 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1947,6 +1947,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process child = CHILD_PROCESS_INIT;
+			const char *post_rewrite_hook =
+				find_hook("post-rewrite");
 
 			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			child.git_cmd = 1;
@@ -1955,6 +1957,18 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			argv_array_push(&child.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
 			run_command(&child);
+
+			if (post_rewrite_hook) {
+				struct child_process hook = CHILD_PROCESS_INIT;
+
+				hook.in = open(rebase_path_rewritten_list(),
+					O_RDONLY);
+				hook.stdout_to_stderr = 1;
+				argv_array_push(&hook.args, post_rewrite_hook);
+				argv_array_push(&hook.args, "rebase");
+				/* we don't care if this hook failed */
+				run_command(&hook);
+			}
 		}
 
 		strbuf_release(&buf);
-- 
2.11.0.rc3.windows.1

[PATCH v2 23/34] sequencer (rebase -i): respect the rebase.autostash setting

[Johannes Schindelin]

Git's `rebase` command inspects the `rebase.autostash` config setting
to determine whether it should stash any uncommitted changes before
rebasing and re-apply them afterwards.

As we introduce more bits and pieces to let the sequencer act as
interactive rebase's backend, here is the part that adds support for
the autostash feature.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index cafd945e83..c11eceb70b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -111,6 +111,7 @@ static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
 static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
 static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
+static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1808,6 +1809,47 @@ static enum todo_command peek_command(struct todo_list *todo_list, int offset)
 	return -1;
 }
 
+static int apply_autostash(struct replay_opts *opts)
+{
+	struct strbuf stash_sha1 = STRBUF_INIT;
+	struct child_process child = CHILD_PROCESS_INIT;
+	int ret = 0;
+
+	if (!read_oneliner(&stash_sha1, rebase_path_autostash(), 1)) {
+		strbuf_release(&stash_sha1);
+		return 0;
+	}
+	strbuf_trim(&stash_sha1);
+
+	child.git_cmd = 1;
+	argv_array_push(&child.args, "stash");
+	argv_array_push(&child.args, "apply");
+	argv_array_push(&child.args, stash_sha1.buf);
+	if (!run_command(&child))
+		printf(_("Applied autostash."));
+	else {
+		struct child_process store = CHILD_PROCESS_INIT;
+
+		store.git_cmd = 1;
+		argv_array_push(&store.args, "stash");
+		argv_array_push(&store.args, "store");
+		argv_array_push(&store.args, "-m");
+		argv_array_push(&store.args, "autostash");
+		argv_array_push(&store.args, "-q");
+		argv_array_push(&store.args, stash_sha1.buf);
+		if (run_command(&store))
+			ret = error(_("cannot store %s"), stash_sha1.buf);
+		else
+			printf(_("Applying autostash resulted in conflicts.\n"
+				"Your changes are safe in the stash.\n"
+				"You can run \"git stash pop\" or"
+				" \"git stash drop\" at any time.\n"));
+	}
+
+	strbuf_release(&stash_sha1);
+	return ret;
+}
+
 static const char *reflog_message(struct replay_opts *opts,
 	const char *sub_action, const char *fmt, ...)
 {
@@ -1970,6 +2012,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				run_command(&hook);
 			}
 		}
+		apply_autostash(opts);
 
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
-- 
2.11.0.rc3.windows.1

[PATCH v2 24/34] sequencer (rebase -i): respect strategy/strategy_opts settings

[Johannes Schindelin]

The sequencer already has an idea about using different merge
strategies. We just piggy-back on top of that, using rebase -i's
own settings, when running the sequencer in interactive rebase mode.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index c11eceb70b..a67ecec961 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -112,6 +112,8 @@ static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
 static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
 static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
+static GIT_PATH_FUNC(rebase_path_strategy, "rebase-merge/strategy")
+static GIT_PATH_FUNC(rebase_path_strategy_opts, "rebase-merge/strategy_opts")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1408,6 +1410,26 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 	return 0;
 }
 
+static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
+{
+	int i;
+
+	strbuf_reset(buf);
+	if (!read_oneliner(buf, rebase_path_strategy(), 0))
+		return;
+	opts->strategy = strbuf_detach(buf, NULL);
+	if (!read_oneliner(buf, rebase_path_strategy_opts(), 0))
+		return;
+
+	opts->xopts_nr = split_cmdline(buf->buf, (const char ***)&opts->xopts);
+	for (i = 0; i < opts->xopts_nr; i++) {
+		const char *arg = opts->xopts[i];
+
+		skip_prefix(arg, "--", &arg);
+		opts->xopts[i] = xstrdup(arg);
+	}
+}
+
 static int read_populate_opts(struct replay_opts *opts)
 {
 	if (is_rebase_i(opts)) {
@@ -1421,11 +1443,13 @@ static int read_populate_opts(struct replay_opts *opts)
 				opts->gpg_sign = xstrdup(buf.buf + 2);
 			}
 		}
-		strbuf_release(&buf);
 
 		if (file_exists(rebase_path_verbose()))
 			opts->verbose = 1;
 
+		read_strategy_opts(opts, &buf);
+		strbuf_release(&buf);
+
 		return 0;
 	}
 
-- 
2.11.0.rc3.windows.1

[PATCH v2 25/34] sequencer (rebase -i): allow rescheduling commands

[Johannes Schindelin]

The interactive rebase has the very special magic that a cherry-pick
that exits with a status different from 0 and 1 signifies a failure to
even record that a cherry-pick was started.

This can happen e.g. when a fast-forward fails because it would
overwrite untracked files.

In that case, we must reschedule the command that we thought we already
had at least started successfully.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index a67ecec961..03256b5b1d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1922,6 +1922,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					1);
 			res = do_pick_commit(item->command, item->commit,
 					opts, is_final_fixup(todo_list));
+			if (is_rebase_i(opts) && res < 0) {
+				/* Reschedule */
+				todo_list->current--;
+				if (save_todo(todo_list, opts))
+					return -1;
+			}
 			if (item->command == TODO_EDIT) {
 				struct commit *commit = item->commit;
 				if (!res)
-- 
2.11.0.rc3.windows.1

[PATCH v2 26/34] sequencer (rebase -i): implement the 'drop' command

[Johannes Schindelin]

The parsing part of a 'drop' command is almost identical to parsing a
'pick', while the operation is the same as that of a 'noop'.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 03256b5b1d..1f314b2743 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -769,7 +769,8 @@ enum todo_command {
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
-	TODO_NOOP
+	TODO_NOOP,
+	TODO_DROP
 };
 
 static struct {
@@ -783,7 +784,8 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
-	{ 0,   "noop" }
+	{ 0,   "noop" },
+	{ 'd', "drop" }
 };
 
 static const char *command_to_string(const enum todo_command command)
@@ -1317,7 +1319,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		else if (is_fixup(item->command))
 			return error(_("cannot '%s' without a previous commit"),
 				command_to_string(item->command));
-		else if (item->command != TODO_NOOP)
+		else if (item->command < TODO_NOOP)
 			fixup_okay = 1;
 	}
 
@@ -1827,7 +1829,7 @@ static enum todo_command peek_command(struct todo_list *todo_list, int offset)
 	int i;
 
 	for (i = todo_list->current + offset; i < todo_list->nr; i++)
-		if (todo_list->items[i].command != TODO_NOOP)
+		if (todo_list->items[i].command < TODO_NOOP)
 			return todo_list->items[i].command;
 
 	return -1;
@@ -1960,7 +1962,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_exec(item->arg);
 			*end_of_arg = saved;
 		}
-		else if (item->command != TODO_NOOP)
+		else if (item->command < TODO_NOOP)
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
-- 
2.11.0.rc3.windows.1

[PATCH v2 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

[Johannes Schindelin]

In the upcoming patch, we will support rebase -i's progress
reporting. The progress skips comments but counts 'noop's.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 1f314b2743..63f6f25ced 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -770,7 +770,9 @@ enum todo_command {
 	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
-	TODO_DROP
+	TODO_DROP,
+	/* comments (not counted for reporting progress) */
+	TODO_COMMENT
 };
 
 static struct {
@@ -785,12 +787,13 @@ static struct {
 	{ 's', "squash" },
 	{ 'x', "exec" },
 	{ 0,   "noop" },
-	{ 'd', "drop" }
+	{ 'd', "drop" },
+	{ 0,   NULL }
 };
 
 static const char *command_to_string(const enum todo_command command)
 {
-	if ((size_t)command < ARRAY_SIZE(todo_command_info))
+	if (command < TODO_COMMENT)
 		return todo_command_info[command].str;
 	die("Unknown command: %d", command);
 }
@@ -1237,14 +1240,14 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	bol += strspn(bol, " \t");
 
 	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
-		item->command = TODO_NOOP;
+		item->command = TODO_COMMENT;
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = eol - bol;
 		return 0;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)
+	for (i = 0; i < TODO_COMMENT; i++)
 		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
 			item->command = i;
 			break;
@@ -1254,7 +1257,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 			item->command = i;
 			break;
 		}
-	if (i >= ARRAY_SIZE(todo_command_info))
+	if (i >= TODO_COMMENT)
 		return -1;
 
 	if (item->command == TODO_NOOP) {
-- 
2.11.0.rc3.windows.1

[PATCH v2 28/34] run_command_opt(): optionally hide stderr when the command succeeds

[Johannes Schindelin]

This will be needed to hide the output of `git commit` when the
sequencer handles an interactive rebase's script.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 run-command.c | 23 +++++++++++++++++++++++
 run-command.h |  1 +
 2 files changed, 24 insertions(+)

diff --git a/run-command.c b/run-command.c
index ca905a9e80..5bb957afdd 100644
--- a/run-command.c
+++ b/run-command.c
@@ -589,6 +589,29 @@ int run_command_v_opt_cd_env(const char **argv, int opt, const char *dir, const
 	cmd.clean_on_exit = opt & RUN_CLEAN_ON_EXIT ? 1 : 0;
 	cmd.dir = dir;
 	cmd.env = env;
+
+	if (opt & RUN_HIDE_STDERR_ON_SUCCESS) {
+		struct strbuf buf = STRBUF_INIT;
+		int res;
+
+		cmd.err = -1;
+		if (start_command(&cmd) < 0)
+			return -1;
+
+		if (strbuf_read(&buf, cmd.err, 0) < 0) {
+			close(cmd.err);
+			finish_command(&cmd); /* throw away exit code */
+			return -1;
+		}
+
+		close(cmd.err);
+		res = finish_command(&cmd);
+		if (res)
+			fputs(buf.buf, stderr);
+		strbuf_release(&buf);
+		return res;
+	}
+
 	return run_command(&cmd);
 }
 
diff --git a/run-command.h b/run-command.h
index dd1c78c28d..65a21ddd4e 100644
--- a/run-command.h
+++ b/run-command.h
@@ -72,6 +72,7 @@ extern int run_hook_ve(const char *const *env, const char *name, va_list args);
 #define RUN_SILENT_EXEC_FAILURE 8
 #define RUN_USING_SHELL 16
 #define RUN_CLEAN_ON_EXIT 32
+#define RUN_HIDE_STDERR_ON_SUCCESS 64
 int run_command_v_opt(const char **argv, int opt);
 
 /*
-- 
2.11.0.rc3.windows.1

[PATCH v2 29/34] sequencer (rebase -i): show only failed `git commit`'s output

[Johannes Schindelin]

This is the behavior of the shell script version of the interactive
rebase, by using the `output` function defined in `git-rebase.sh`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 63f6f25ced..dfa4fab98b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -647,10 +647,15 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 {
 	char **env = NULL;
 	struct argv_array array;
-	int rc;
+	int opt = RUN_GIT_CMD, rc;
 	const char *value;
 
 	if (is_rebase_i(opts)) {
+		if (!edit) {
+			opt |= RUN_COMMAND_STDOUT_TO_STDERR;
+			opt |= RUN_HIDE_STDERR_ON_SUCCESS;
+		}
+
 		env = read_author_script();
 		if (!env) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
@@ -687,7 +692,7 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	if (opts->allow_empty_message)
 		argv_array_push(&array, "--allow-empty-message");
 
-	rc = run_command_v_opt_cd_env(array.argv, RUN_GIT_CMD, NULL,
+	rc = run_command_v_opt_cd_env(array.argv, opt, NULL,
 			(const char *const *)env);
 	argv_array_clear(&array);
 	free(env);
-- 
2.11.0.rc3.windows.1

[PATCH v2 30/34] sequencer (rebase -i): show only failed cherry-picks' output

[Johannes Schindelin]

This is the behavior of the shell script version of the interactive
rebase, by using the `output` function defined in `git-rebase.sh`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index dfa4fab98b..edc213a2c8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -464,6 +464,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	o.ancestor = base ? base_label : "(empty tree)";
 	o.branch1 = "HEAD";
 	o.branch2 = next ? next_label : "(empty tree)";
+	if (is_rebase_i(opts))
+		o.buffer_output = 2;
 
 	head_tree = parse_tree_indirect(head);
 	next_tree = next ? next->tree : empty_tree();
@@ -475,6 +477,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	if (is_rebase_i(opts) && clean <= 0)
+		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
 	if (clean < 0)
 		return clean;
-- 
2.11.0.rc3.windows.1

[PATCH v2 31/34] sequencer (rebase -i): suggest --edit-todo upon unknown command

[Johannes Schindelin]

This is the same behavior as known from `git rebase -i`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index edc213a2c8..498dd028d1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1355,8 +1355,12 @@ static int read_populate_todo(struct todo_list *todo_list,
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
-	if (res)
+	if (res) {
+		if (is_rebase_i(opts))
+			return error(_("please fix this using "
+				       "'git rebase --edit-todo'."));
 		return error(_("unusable instruction sheet: '%s'"), todo_file);
+	}
 
 	if (!todo_list->nr &&
 	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
-- 
2.11.0.rc3.windows.1

[PATCH v2 32/34] sequencer (rebase -i): show the progress

[Johannes Schindelin]

The interactive rebase keeps the user informed about its progress.
If the sequencer wants to do the grunt work of the interactive
rebase, it also needs to show that progress.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 498dd028d1..35d5ef4ef6 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1221,6 +1221,7 @@ struct todo_list {
 	struct strbuf buf;
 	struct todo_item *items;
 	int nr, alloc, current;
+	int done_nr, total_nr;
 };
 
 #define TODO_LIST_INIT { STRBUF_INIT }
@@ -1338,6 +1339,17 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 	return res;
 }
 
+static int count_commands(struct todo_list *todo_list)
+{
+	int count = 0, i;
+
+	for (i = 0; i < todo_list->nr; i++)
+		if (todo_list->items[i].command != TODO_COMMENT)
+			count++;
+
+	return count;
+}
+
 static int read_populate_todo(struct todo_list *todo_list,
 			struct replay_opts *opts)
 {
@@ -1380,6 +1392,21 @@ static int read_populate_todo(struct todo_list *todo_list,
 				return error(_("cannot revert during a cherry-pick."));
 	}
 
+	if (is_rebase_i(opts)) {
+		struct todo_list done = TODO_LIST_INIT;
+
+		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
+				!parse_insn_buffer(done.buf.buf, &done))
+			todo_list->done_nr = count_commands(&done);
+		else
+			todo_list->done_nr = 0;
+
+		todo_list->total_nr = todo_list->done_nr
+			+ count_commands(todo_list);
+
+		todo_list_release(&done);
+	}
+
 	return 0;
 }
 
@@ -1928,6 +1955,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
+			if (item->command != TODO_COMMENT)
+				fprintf(stderr, "Rebasing (%d/%d)%s",
+					++(todo_list->done_nr),
+					todo_list->total_nr,
+					opts->verbose ? "\n" : "\r");
 			unlink(rebase_path_message());
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
-- 
2.11.0.rc3.windows.1

[PATCH v2 33/34] sequencer (rebase -i): write the progress into files

[Johannes Schindelin]

For the benefit of e.g. the shell prompt, the interactive rebase not
only displays the progress for the user to see, but also writes it into
the msgnum/end files in the state directory.

Teach the sequencer this new trick.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 35d5ef4ef6..cb5e7f35fc 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -45,6 +45,16 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
 /*
+ * The file to keep track of how many commands were already processed (e.g.
+ * for the prompt).
+ */
+static GIT_PATH_FUNC(rebase_path_msgnum, "rebase-merge/msgnum");
+/*
+ * The file to keep track of how many commands are to be processed in total
+ * (e.g. for the prompt).
+ */
+static GIT_PATH_FUNC(rebase_path_msgtotal, "rebase-merge/end");
+/*
  * The commit message that is planned to be used for any changes that
  * need to be committed following a user interaction.
  */
@@ -1394,6 +1404,7 @@ static int read_populate_todo(struct todo_list *todo_list,
 
 	if (is_rebase_i(opts)) {
 		struct todo_list done = TODO_LIST_INIT;
+		FILE *f = fopen(rebase_path_msgtotal(), "w");
 
 		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
 				!parse_insn_buffer(done.buf.buf, &done))
@@ -1403,8 +1414,12 @@ static int read_populate_todo(struct todo_list *todo_list,
 
 		todo_list->total_nr = todo_list->done_nr
 			+ count_commands(todo_list);
-
 		todo_list_release(&done);
+
+		if (f) {
+			fprintf(f, "%d\n", todo_list->total_nr);
+			fclose(f);
+		}
 	}
 
 	return 0;
@@ -1955,11 +1970,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
-			if (item->command != TODO_COMMENT)
+			if (item->command != TODO_COMMENT) {
+				FILE *f = fopen(rebase_path_msgnum(), "w");
+
+				todo_list->done_nr++;
+
+				if (f) {
+					fprintf(f, "%d\n", todo_list->done_nr);
+					fclose(f);
+				}
 				fprintf(stderr, "Rebasing (%d/%d)%s",
-					++(todo_list->done_nr),
+					todo_list->done_nr,
 					todo_list->total_nr,
 					opts->verbose ? "\n" : "\r");
+			}
 			unlink(rebase_path_message());
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
-- 
2.11.0.rc3.windows.1

[PATCH v2 34/34] sequencer (rebase -i): write out the final message

[Johannes Schindelin]

The shell script version of the interactive rebase has a very specific
final message. Teach the sequencer to print the same.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index cb5e7f35fc..41be4cde16 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2118,6 +2118,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		}
 		apply_autostash(opts);
 
+		fprintf(stderr, "Successfully rebased and updated %s.\n",
+			head_ref.buf);
+
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
 	}
-- 
2.11.0.rc3.windows.1

Re: [PATCH v2 00/34] Teach the sequencer to act as rebase -i's backend

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> This marks the count down to '3': two more patch series after this
> (really tiny ones) and we have a faster rebase -i.

Nice.

> Apart from mostly cosmetic patches (and the occasional odd bug that I
> fixed promptly), I used these patches since mid May to perform all of my
> interactive rebases. In mid June, I had the idea to teach rebase -i to
> run *both* scripted rebase and rebase--helper and to cross-validate the
> results. This slowed down all my interactive rebases since, but helped
> me catch three rather obscure bugs (e.g. that git commit --fixup unfolds
> long onelines and rebase -i still finds the correct original commit).
> ...
> Please note that the interdiff vs v1 is only of limited use: too many
> things changed in the meantime, in particular the prepare-sequencer
> branch that went through a couple of iterations before it found its way
> into git.git's master branch. So please take the interdiff with a
> mountain range of salt.
> ...
> Changes since v1:
> ...
> - removed the beautiful ordinal logic (to print out "1st", "2nd", "3rd"
>   etc) and made things consistent with the current `rebase -i`.

It was removed because it was too Anglo-centric and unusable in i18n
context, no?

Judging from the list above, interdiff are pretty much all cosmetic
and that is why you say it is only of limited use, I guess.

    ... goes and reads the remainder and finds that these were
    ... all minor changes, mostly cosmetic, with a helper function
    ... refactored out or two and things of that nature.

It is actually a good thing.  We do not want to see it deviate too
drastically from what you have been testing for some months.

Thanks.

Re: [PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

>  static inline int is_rebase_i(const struct replay_opts *opts)
>  {
> -	return 0;
> +	return opts->action == REPLAY_INTERACTIVE_REBASE;
>  }
>  
>  static const char *get_dir(const struct replay_opts *opts)
>  {
> +	if (is_rebase_i(opts))
> +		return rebase_path();
>  	return git_path_seq_dir();
>  }
>  
>  static const char *get_todo_path(const struct replay_opts *opts)
>  {
> +	if (is_rebase_i(opts))
> +		return rebase_path_todo();
>  	return git_path_todo_file();
>  }
>  
> @@ -168,7 +179,15 @@ int sequencer_remove_state(struct replay_opts *opts)
>  
>  static const char *action_name(const struct replay_opts *opts)
>  {
> -	return opts->action == REPLAY_REVERT ? N_("revert") : N_("cherry-pick");
> +	switch (opts->action) {
> +	case REPLAY_REVERT:
> +		return N_("revert");
> +	case REPLAY_PICK:
> +		return N_("cherry-pick");
> +	case REPLAY_INTERACTIVE_REBASE:
> +		return N_("rebase -i");
> +	}
> +	die(_("Unknown action: %d"), opts->action);
>  }

This case statement which looks perfectly sensible---it says that
there are three equal modes the subsystem operates in.  

This is just a mental note and not a suggestion to change anything
immediately, but it makes me wonder if git_dir/get_todo_path would
also want to do so, moving towards retiring is_rebase_i() which is
"everything else vs one oddball which is rebase-i" mindset.

> @@ -395,7 +414,10 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
>  
>  	if (active_cache_changed &&
>  	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
> -		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
> +		/*
> +		 * TRANSLATORS: %s will be "revert", "cherry-pick" or
> +		 * "rebase -i".
> +		 */

IIRC, the "TRANSLATORS:" comment has to deviate from our coding
style due to tool limitation and has to be done like this:

> +		/* TRANSLATORS: %s will be "revert", "cherry-pick" or
> +		 * "rebase -i".
> +		 */

> @@ -1204,6 +1226,9 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
>  	const char *todo_path = get_todo_path(opts);
>  	int next = todo_list->current, offset, fd;
>  
> +	if (is_rebase_i(opts))
> +		next++;
> +

This is because...?  Everybody else counts 0-based while rebase-i
counts from 1 or something?

>  	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
>  	if (fd < 0)
>  		return error_errno(_("could not lock '%s'"), todo_path);

Everything else in the patch is understandable.  This bit isn't
without explanation, at least to me.

Re: [PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> +/*
> + * Note that ordering matters in this enum. Not only must it match the mapping
> + * below, it is also divided into several sections that matter.  When adding
> + * new commands, make sure you add it in the right section.
> + */

Good thinking.  Makes me wish C were a better language, though ;-)

>  enum todo_command {
> +	/* commands that handle commits */
>  	TODO_PICK = 0,
> -	TODO_REVERT
> +	TODO_REVERT,
> +	/* commands that do nothing but are counted for reporting progress */
> +	TODO_NOOP
>  };
>  
>  static const char *todo_command_strings[] = {
>  	"pick",
> -	"revert"
> +	"revert",
> +	"noop"
>  };

> @@ -1292,7 +1316,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  		struct todo_item *item = todo_list->items + todo_list->current;
>  		if (save_todo(todo_list, opts))
>  			return -1;
> -		res = do_pick_commit(item->command, item->commit, opts);
> +		if (item->command <= TODO_REVERT)
> +			res = do_pick_commit(item->command, item->commit,
> +					opts);
> +		else if (item->command != TODO_NOOP)
> +			return error(_("unknown command %d"), item->command);

I wonder if making this a switch() statement is easier to read in
the longer run.  The only thing at this point we are gaining by "not
only mapping and enum must match, the orders matter" is so that this
codepath can do the same thing for PICK and REVERT, but these two
would become more and more minority as we learn more words.

>  		todo_list->current++;
>  		if (res)
>  			return res;

Re: [PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

[Linus Torvalds]

On Tue, Dec 13, 2016 at 12:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>
>> +/*
>> + * Note that ordering matters in this enum. Not only must it match the mapping
>> + * below, it is also divided into several sections that matter.  When adding
>> + * new commands, make sure you add it in the right section.
>> + */
>
> Good thinking.  Makes me wish C were a better language, though ;-)

Do this:

  static const char *todo_command_strings[] = {
      [TODO_PICK] = "pick",
      [TODO_REVERT] = "revert",
      [TODO_NOOP] = "noop:,
  };

which makes the array be order-independent. You still need to make
sure you fill in all the entries, of course, but it tends to avoid at
least one gotcha, and it makes it more obvious how the two are tied
together.

              Linus

Re: [PATCH v2 03/34] sequencer (rebase -i): implement the 'edit' command

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> @@ -43,6 +44,20 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
>   */
>  static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
>  /*

It is minor, but please have a blank line to separate the logical
blocks.  If you have "comment for thing A" before "thing A", then 
having a blank after that before "comment for thing B" and "thing B"
that follow would help.  Otherwise "thing A" immediately followed by
"comment for thing B" are (mis)read together, leading to nonsense.

> + * When an "edit" rebase command is being processed, the SHA1 of the
> + * commit to be edited is recorded in this file.  When "git rebase
> + * --continue" is executed, if there are any staged changes then they
> + * will be amended to the HEAD commit, but only provided the HEAD
> + * commit is still the commit to be edited.  When any other rebase
> + * command is processed, this file is deleted.
> + */
> +static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
> +/*
> + * When we stop at a given patch via the "edit" command, this file contains
> + * the long commit name of the corresponding patch.

If you abbreviate an object name to 38-hex that is still long but
that is not full; if you meant full 40-hex, better spell it out as
"full"---that conveys useful information to programmers (e.g. they
can just use get_sha1_hex()).

But I think you are writing short_commit_name() to it?  So perhaps
"an abbreviated commit object name"?

> @@ -1301,9 +1318,87 @@ static int save_opts(struct replay_opts *opts)
>  	return res;
>  }
>  
> +static int make_patch(struct commit *commit, struct replay_opts *opts)
> +{
> +	struct strbuf buf = STRBUF_INIT;
> +	struct rev_info log_tree_opt;
> +	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject, *p;
> +	int res = 0;
> +
> +	p = short_commit_name(commit);
> +	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
> +		return -1;
> +
> +	strbuf_addf(&buf, "%s/patch", get_dir(opts));
> +	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
> +	init_revisions(&log_tree_opt, NULL);
> +	log_tree_opt.abbrev = 0;
> +	log_tree_opt.diff = 1;
> +	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
> +	log_tree_opt.disable_stdin = 1;
> +	log_tree_opt.no_commit_id = 1;
> +	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
> +	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
> +	if (!log_tree_opt.diffopt.file)
> +		res |= error_errno(_("could not open '%s'"), buf.buf);
> +	else {
> +		res |= log_tree_commit(&log_tree_opt, commit);
> +		fclose(log_tree_opt.diffopt.file);
> +	}
> +	strbuf_reset(&buf);
> +	strbuf_addf(&buf, "%s/message", get_dir(opts));
> +	if (!file_exists(buf.buf)) {
> +		find_commit_subject(commit_buffer, &subject);
> +		res |= write_message(subject, strlen(subject), buf.buf, 1);
> +		unuse_commit_buffer(commit, commit_buffer);
> +	}
> +	strbuf_release(&buf);
> +
> +	return res;
> +}

OK.  This seems to match what scripted make_patch does in a handful
of lines.  We probably should have given you a helper to reduce
boilerplate that sets up log_tree_opt so that this function does not
have to be this long, but that is a separate topic.

Does it matter output_format is set to FORMAT_PATCH here, though?
With --no-commit-id set, I suspect there is no log message or
authorship information given to the output.

As you are only interested in seeing the patch/diff in this file and
the log is stored in a separate "message" file, as long as "patch"
file gets the patch correctly, it is not a problem, but it just
looked strange to see FORMAT_PATCH there.

Re: [PATCH v2 04/34] sequencer (rebase -i): implement the 'exec' command

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> +static int do_exec(const char *command_line)
> +{
> +	const char *child_argv[] = { NULL, NULL };
> +	int dirty, status;
> +
> +	fprintf(stderr, "Executing: %s\n", command_line);
> +	child_argv[0] = command_line;
> +	status = run_command_v_opt(child_argv, RUN_USING_SHELL);
> +
> +	/* force re-reading of the cache */
> +	if (discard_cache() < 0 || read_cache() < 0)
> +		return error(_("could not read index"));
> +
> +	dirty = require_clean_work_tree("rebase", NULL, 1, 1);
> +
> +	if (status) {
> +		warning(_("execution failed: %s\n%s"
> +			  "You can fix the problem, and then run\n"
> +			  "\n"
> +			  "  git rebase --continue\n"
> +			  "\n"),
> +			command_line,
> +			dirty ? N_("and made changes to the index and/or the "
> +				"working tree\n") : "");
> +		if (status == 127)
> +			/* command not found */
> +			status = 1;
> +	}
> +	else if (dirty) {
> +		warning(_("execution succeeded: %s\nbut "
> +			  "left changes to the index and/or the working tree\n"
> +			  "Commit or stash your changes, and then run\n"
> +			  "\n"
> +			  "  git rebase --continue\n"
> +			  "\n"), command_line);
> +		status = 1;
> +	}
> +
> +	return status;
> +}

OK, this looks like a faithful reproduction of what the scripted
version does inside do_next() helper function.

Please have "else if" on the same line as "}" that closes the
"if (...) {" in the same if/else if/else cascade.

Re: [PATCH v2 05/34] sequencer (rebase -i): learn about the 'verbose' mode

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> @@ -1493,9 +1498,26 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  	}
>  
>  	if (is_rebase_i(opts)) {
> +		struct strbuf buf = STRBUF_INIT;
> +
>  		/* Stopped in the middle, as planned? */
>  		if (todo_list->current < todo_list->nr)
>  			return 0;
> +
> +		if (opts->verbose) {
> +			const char *argv[] = {
> +				"diff-tree", "--stat", NULL, NULL
> +			};
> +
> +			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
> +				return error(_("could not read '%s'"),
> +					rebase_path_orig_head());
> +			strbuf_addstr(&buf, "..HEAD");
> +			argv[2] = buf.buf;
> +			run_command_v_opt(argv, RUN_GIT_CMD);
> +			strbuf_reset(&buf);
> +		}
> +		strbuf_release(&buf);
>  	}

It's a bit curious that the previous step avoided running a separate
process and instead did "diff-tree -p" all in C, but this one does not.

I think it is because this one is outside the loop?  The original,
being a scripted Porcelain, formulates a lazy and loose command
line, but you may want to tighten it up a bit if you spawn a
process.  If your user happens to have a file whose name is
$orig_head..HEAD, the command line you are creating (which is
identical to the scripted version) will barf with "ambiguous
argument".

One good thing about a complete C rewrite is that it won't have an
issue like this one because you'd be working with in-core objects.

> diff --git a/sequencer.h b/sequencer.h
> index cb21cfddee..f885b68395 100644
> --- a/sequencer.h
> +++ b/sequencer.h
> @@ -24,6 +24,7 @@ struct replay_opts {
>  	int allow_empty;
>  	int allow_empty_message;
>  	int keep_redundant_commits;
> +	int verbose;
>  
>  	int mainline;

Re: [PATCH v2 06/34] sequencer (rebase -i): write the 'done' file

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> In the interactive rebase, commands that were successfully processed are
> not simply discarded, but appended to the 'done' file instead. This is
> used e.g. to display the current state to the user in the output of
> `git status` or the progress.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---

Looks good.

I'd stop at this point for now, and start working on other things
for the rest of the day.  I might find time to come back to it later
tonight.

So far, looking mostly good.

Re: [PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

[Junio C Hamano]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> On Tue, Dec 13, 2016 at 12:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>>
>>> +/*
>>> + * Note that ordering matters in this enum. Not only must it match the mapping
>>> + * below, it is also divided into several sections that matter.  When adding
>>> + * new commands, make sure you add it in the right section.
>>> + */
>>
>> Good thinking.  Makes me wish C were a better language, though ;-)
>
> Do this:
>
>   static const char *todo_command_strings[] = {
>       [TODO_PICK] = "pick",
>       [TODO_REVERT] = "revert",
>       [TODO_NOOP] = "noop:,
>   };
>
> which makes the array be order-independent. You still need to make
> sure you fill in all the entries, of course, but it tends to avoid at
> least one gotcha, and it makes it more obvious how the two are tied
> together.
>
>               Linus

Yes, I know.  But I do not think the variant of C we stick to is not
new enough to have that.

Re: [PATCH v2 05/34] sequencer (rebase -i): learn about the 'verbose' mode

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> ...
>> +		if (opts->verbose) {
>> +			const char *argv[] = {
>> +				"diff-tree", "--stat", NULL, NULL
>> +			};
>> + ...
>> +			run_command_v_opt(argv, RUN_GIT_CMD);
>> +			strbuf_reset(&buf);
>> +		}
>> +		strbuf_release(&buf);
>>  	}
>
> It's a bit curious that the previous step avoided running a separate
> process and instead did "diff-tree -p" all in C, but this one does not.
>
> I think it is because this one is outside the loop?

Nah, this guess of mine "The patch file generation done in 03/34
avoids spawn because it is in a loop" is off the mark.  It is done
before "edit" gives control back to the end user and it is not like
we write one patch file per iteration of the loop we want to get
maximum speed out of.

Re: [PATCH v2 00/34] Teach the sequencer to act as rebase -i's backend

[Johannes Sixt]

Am 13.12.2016 um 16:29 schrieb Johannes Schindelin:
> base-commit: 8d7a455ed52e2a96debc080dfc011b6bb00db5d2
> Published-As: https://github.com/dscho/git/releases/tag/sequencer-i-v2
> Fetch-It-Via: git fetch https://github.com/dscho/git sequencer-i-v2

Thank you so much!

I would appreciate if you could publish a branch that contains the end 
game so that I can test it, too. Currently I am still using

  git://github.com/dscho/git interactive-rebase (fca871a3cf4d)

-- Hannes

Re: [PATCH v2 28/34] run_command_opt(): optionally hide stderr when the command succeeds

[Johannes Sixt]

Am 13.12.2016 um 16:32 schrieb Johannes Schindelin:
> This will be needed to hide the output of `git commit` when the
> sequencer handles an interactive rebase's script.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  run-command.c | 23 +++++++++++++++++++++++
>  run-command.h |  1 +
>  2 files changed, 24 insertions(+)
> 
> diff --git a/run-command.c b/run-command.c
> index ca905a9e80..5bb957afdd 100644
> --- a/run-command.c
> +++ b/run-command.c
> @@ -589,6 +589,29 @@ int run_command_v_opt_cd_env(const char **argv, int opt, const char *dir, const
>  	cmd.clean_on_exit = opt & RUN_CLEAN_ON_EXIT ? 1 : 0;
>  	cmd.dir = dir;
>  	cmd.env = env;
> +
> +	if (opt & RUN_HIDE_STDERR_ON_SUCCESS) {
> +		struct strbuf buf = STRBUF_INIT;
> +		int res;
> +
> +		cmd.err = -1;
> +		if (start_command(&cmd) < 0)
> +			return -1;
> +
> +		if (strbuf_read(&buf, cmd.err, 0) < 0) {
> +			close(cmd.err);
> +			finish_command(&cmd); /* throw away exit code */
> +			return -1;
> +		}
> +
> +		close(cmd.err);
> +		res = finish_command(&cmd);
> +		if (res)
> +			fputs(buf.buf, stderr);
> +		strbuf_release(&buf);
> +		return res;
> +	}
> +
>  	return run_command(&cmd);
>  }

Clearly, this is a bolted-on feature. It's not the worst move that you
did not advertise the flag in Documentation/technical/api-run-command.txt...

I wanted to see what it would look like if we make it the caller's
responsibility to throw away stderr. The patch is below, as fixup
of patch 29/34. The change is gross, but the end result is not that
bad, though not really a delightful read, either, mostly due to the
strange cleanup semantics of the start_command/finish_command combo,
so... I dunno.

diff --git a/sequencer.c b/sequencer.c
index 41be4cde16..f375880bd7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -660,15 +660,16 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 			  int cleanup_commit_message)
 {
 	char **env = NULL;
-	struct argv_array array;
-	int opt = RUN_GIT_CMD, rc;
+	int rc;
 	const char *value;
+	struct strbuf errout = STRBUF_INIT;
+	struct child_process cmd = CHILD_PROCESS_INIT;
+
+	cmd.git_cmd = 1;
 
 	if (is_rebase_i(opts)) {
-		if (!edit) {
-			opt |= RUN_COMMAND_STDOUT_TO_STDERR;
-			opt |= RUN_HIDE_STDERR_ON_SUCCESS;
-		}
+		if (!edit)
+			cmd.stdout_to_stderr = 1;
 
 		env = read_author_script();
 		if (!env) {
@@ -679,36 +680,58 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		}
 	}
 
-	argv_array_init(&array);
-	argv_array_push(&array, "commit");
-	argv_array_push(&array, "-n");
+	argv_array_push(&cmd.args, "commit");
+	argv_array_push(&cmd.args, "-n");
 
 	if (amend)
-		argv_array_push(&array, "--amend");
+		argv_array_push(&cmd.args, "--amend");
 	if (opts->gpg_sign)
-		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
+		argv_array_pushf(&cmd.args, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
-		argv_array_push(&array, "-s");
+		argv_array_push(&cmd.args, "-s");
 	if (defmsg)
-		argv_array_pushl(&array, "-F", defmsg, NULL);
+		argv_array_pushl(&cmd.args, "-F", defmsg, NULL);
 	if (cleanup_commit_message)
-		argv_array_push(&array, "--cleanup=strip");
+		argv_array_push(&cmd.args, "--cleanup=strip");
 	if (edit)
-		argv_array_push(&array, "-e");
+		argv_array_push(&cmd.args, "-e");
 	else if (!cleanup_commit_message &&
 		 !opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
-		argv_array_push(&array, "--cleanup=verbatim");
+		argv_array_push(&cmd.args, "--cleanup=verbatim");
 
 	if (allow_empty)
-		argv_array_push(&array, "--allow-empty");
+		argv_array_push(&cmd.args, "--allow-empty");
 
 	if (opts->allow_empty_message)
-		argv_array_push(&array, "--allow-empty-message");
+		argv_array_push(&cmd.args, "--allow-empty-message");
+
+	cmd.env = (const char *const *)env;
+
+	if (cmd.stdout_to_stderr) {
+		/* hide stderr on success */
+		cmd.err = -1;
+		rc = -1;
+		if (start_command(&cmd) < 0)
+			goto cleanup;
+
+		if (strbuf_read(&errout, cmd.err, 0) < 0) {
+			close(cmd.err);
+			finish_command(&cmd); /* throw away exit code */
+			goto cleanup;
+		}
+
+		close(cmd.err);
+		rc = finish_command(&cmd);
+		if (rc)
+			fputs(errout.buf, stderr);
+	} else {
+		rc = run_command(&cmd);
+	}
 
-	rc = run_command_v_opt_cd_env(array.argv, opt, NULL,
-			(const char *const *)env);
-	argv_array_clear(&array);
+cleanup:
+	child_process_clear(&cmd);
+	strbuf_release(&errout);
 	free(env);
 
 	return rc;
-- 
2.11.0.79.g263f27a

Re: [PATCH v2 28/34] run_command_opt(): optionally hide stderr when the command succeeds

[Jeff King]

On Wed, Dec 14, 2016 at 09:34:20AM +0100, Johannes Sixt wrote:

> I wanted to see what it would look like if we make it the caller's
> responsibility to throw away stderr. The patch is below, as fixup
> of patch 29/34. The change is gross, but the end result is not that
> bad, though not really a delightful read, either, mostly due to the
> strange cleanup semantics of the start_command/finish_command combo,
> so... I dunno.

I don't have a strong opinion on the patches under discussion, but here
are a few pointers on the run-command interface:

> +	struct child_process cmd = CHILD_PROCESS_INIT;
> [...]
>  		env = read_author_script();
>  		if (!env) {

The child_process struct comes with its own internal env array. So you
can do:

  read_author_script(&cmd.env_array);
  if (!cmd.env_array.argc)
	...

and then you don't have to worry about free-ing env, as it happens
automatically as part of the child cleanup (I suspect the refactoring
may also reduce some of the confusing memory handling in
read_author_script()).

> +	if (cmd.stdout_to_stderr) {
> +		/* hide stderr on success */
> +		cmd.err = -1;
> +		rc = -1;
> +		if (start_command(&cmd) < 0)
> +			goto cleanup;
> +
> +		if (strbuf_read(&errout, cmd.err, 0) < 0) {
> +			close(cmd.err);
> +			finish_command(&cmd); /* throw away exit code */
> +			goto cleanup;
> +		}
> +
> +		close(cmd.err);
> +		rc = finish_command(&cmd);
> +		if (rc)
> +			fputs(errout.buf, stderr);
> +	} else {
> +		rc = run_command(&cmd);
> +	}

We have a helper function for capturing output, so I think you can write
this as:

  if (cmd.err == -1) {
        struct strbuf errout = STRBUF_INIT;
        int rc = pipe_command(&cmd,
		              NULL, 0, /* stdin */
                              NULL, 0, /* stdout */
                              &errout, 0);
        if (rc)
                fputs(errout.buf, stderr);
        strbuf_release(&errout);
  } else
        rc = run_command(&cmd);

and drop the cleanup goto entirely (if you do the "env" thing above, you
could even drop "rc" and just return directly from each branch of the
conditional).

> -	rc = run_command_v_opt_cd_env(array.argv, opt, NULL,
> -			(const char *const *)env);
> -	argv_array_clear(&array);
> +cleanup:
> +	child_process_clear(&cmd);
> +	strbuf_release(&errout);
>  	free(env);

Even if you do keep the goto here, I think this child_process_clear() is
unnecessary. It should be done automatically either by finish_command(),
or by start_command() when it returns an error.

-Peff

Re: [PATCH v2 28/34] run_command_opt(): optionally hide stderr when the command succeeds

[Jeff King]

On Wed, Dec 14, 2016 at 07:53:23AM -0500, Jeff King wrote:

> On Wed, Dec 14, 2016 at 09:34:20AM +0100, Johannes Sixt wrote:
> 
> > I wanted to see what it would look like if we make it the caller's
> > responsibility to throw away stderr. The patch is below, as fixup
> > of patch 29/34. The change is gross, but the end result is not that
> > bad, though not really a delightful read, either, mostly due to the
> > strange cleanup semantics of the start_command/finish_command combo,
> > so... I dunno.
> 
> I don't have a strong opinion on the patches under discussion, but here
> are a few pointers on the run-command interface:
> [...]

And here is a patch representing my suggestions, on top of yours. Not
tested beyond "make test".

I think read_author_script() could be simplified even more by appending
to the env array in the first loop, but I didn't want to refactor the
quote handling.

---
 sequencer.c | 65 +++++++++++------------------------
 1 file changed, 20 insertions(+), 45 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index f375880bd..27a9eaf15 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -591,18 +591,17 @@ static int write_author_script(const char *message)
 }
 
 /*
- * Read the author-script file into an environment block, ready for use in
- * run_command(), that can be free()d afterwards.
+ * Read the author-script file into an environment block. Returns -1
+ * on error, 0 otherwise.
  */
-static char **read_author_script(void)
+static int read_author_script(struct argv_array *env)
 {
 	struct strbuf script = STRBUF_INIT;
 	int i, count = 0;
-	char *p, *p2, **env;
-	size_t env_size;
+	char *p, *p2;
 
 	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
-		return NULL;
+		return -1;
 
 	for (p = script.buf; *p; p++)
 		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
@@ -614,19 +613,12 @@ static char **read_author_script(void)
 			count++;
 		}
 
-	env_size = (count + 1) * sizeof(*env);
-	strbuf_grow(&script, env_size);
-	memmove(script.buf + env_size, script.buf, script.len);
-	p = script.buf + env_size;
-	env = (char **)strbuf_detach(&script, NULL);
-
 	for (i = 0; i < count; i++) {
-		env[i] = p;
+		argv_array_push(env, p);
 		p += strlen(p) + 1;
 	}
-	env[count] = NULL;
 
-	return env;
+	return 0;
 }
 
 static const char staged_changes_advice[] =
@@ -659,20 +651,18 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 			  int allow_empty, int edit, int amend,
 			  int cleanup_commit_message)
 {
-	char **env = NULL;
-	int rc;
 	const char *value;
-	struct strbuf errout = STRBUF_INIT;
 	struct child_process cmd = CHILD_PROCESS_INIT;
 
 	cmd.git_cmd = 1;
 
 	if (is_rebase_i(opts)) {
-		if (!edit)
+		if (!edit) {
 			cmd.stdout_to_stderr = 1;
+			cmd.err = -1;
+		}
 
-		env = read_author_script();
-		if (!env) {
+		if (read_author_script(&cmd.env_array)) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
 			return error(_(staged_changes_advice),
@@ -706,35 +696,20 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	if (opts->allow_empty_message)
 		argv_array_push(&cmd.args, "--allow-empty-message");
 
-	cmd.env = (const char *const *)env;
-
-	if (cmd.stdout_to_stderr) {
+	if (cmd.err == -1) {
 		/* hide stderr on success */
-		cmd.err = -1;
-		rc = -1;
-		if (start_command(&cmd) < 0)
-			goto cleanup;
-
-		if (strbuf_read(&errout, cmd.err, 0) < 0) {
-			close(cmd.err);
-			finish_command(&cmd); /* throw away exit code */
-			goto cleanup;
-		}
-
-		close(cmd.err);
-		rc = finish_command(&cmd);
+		struct strbuf errout = STRBUF_INIT;
+		int rc = pipe_command(&cmd,
+				      NULL, 0, /* stdin */
+				      NULL, 0, /* stdout */
+				      &errout, 0);
 		if (rc)
 			fputs(errout.buf, stderr);
-	} else {
-		rc = run_command(&cmd);
+		strbuf_release(&errout);
+		return rc;
 	}
 
-cleanup:
-	child_process_clear(&cmd);
-	strbuf_release(&errout);
-	free(env);
-
-	return rc;
+	return run_command(&cmd);
 }
 
 static int is_original_commit_empty(struct commit *commit)

Re: [PATCH v2 28/34] run_command_opt(): optionally hide stderr when the command succeeds

[Johannes Sixt]

Am 14.12.2016 um 14:06 schrieb Jeff King:
> On Wed, Dec 14, 2016 at 07:53:23AM -0500, Jeff King wrote:
>
>> On Wed, Dec 14, 2016 at 09:34:20AM +0100, Johannes Sixt wrote:
>>
>>> I wanted to see what it would look like if we make it the caller's
>>> responsibility to throw away stderr. The patch is below, as fixup
>>> of patch 29/34. The change is gross, but the end result is not that
>>> bad, though not really a delightful read, either, mostly due to the
>>> strange cleanup semantics of the start_command/finish_command combo,
>>> so... I dunno.

The cleanup semantics of start_command and finish_command are not that 
strange as I thought first. I just hadn't looked well enough.

>>
>> I don't have a strong opinion on the patches under discussion, but here
>> are a few pointers on the run-command interface:
>> [...]
 >
> And here is a patch representing my suggestions, on top of yours. Not
> tested beyond "make test".

Thank you, that looks way better.

If there is agreement that this approach is preferable, I think we can 
have patches on top of the series; they would be orthogonal and do not 
have to take hostage of it. (And it looks like I won't be able to follow 
up until later this week[end].)

-- Hannes

Re: [PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> -/* We will introduce the 'interactive rebase' mode later */
>  static inline int is_rebase_i(const struct replay_opts *opts)
>  {
> -	return 0;
> +	return opts->action == REPLAY_INTERACTIVE_REBASE;
>  }
>  
>  static const char *get_dir(const struct replay_opts *opts)
>  {
> +	if (is_rebase_i(opts))
> +		return rebase_path();
>  	return git_path_seq_dir();
>  }

This obviously makes the assumption made by 39784cd362 ("sequencer:
remove useless get_dir() function", 2016-12-07) invalid, where the
"remove useless" thought that the callers of this function wants a
single answer that does not depend on opts.

I'll revert that commit from the sb/sequencer-abort-safety topic (as
the topic is in 'next' already) to keep this one.  Please holler if
that is a mistaken decision.

Thanks.

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> diff --git a/sequencer.c b/sequencer.c
> index f6e20b142a..271c21581d 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -45,6 +45,35 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
>   */
>  static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
>  /*
> + * The commit message that is planned to be used for any changes that
> + * need to be committed following a user interaction.
> + */
> +static GIT_PATH_FUNC(rebase_path_message, "rebase-merge/message")
> +/*
> + * The file into which is accumulated the suggested commit message for
> + * squash/fixup commands. When the first of a series of squash/fixups

The same comment as 03/34 applies here, regarding blank line to
separate logical unit.

> +static int update_squash_messages(enum todo_command command,
> +		struct commit *commit, struct replay_opts *opts)
> +{
> +	struct strbuf buf = STRBUF_INIT;
> +	int count, res;
> +	const char *message, *body;
> +
> +	if (file_exists(rebase_path_squash_msg())) {
> +		char *p, *p2;
> +
> +		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
> +			return error(_("could not read '%s'"),
> +				rebase_path_squash_msg());
> +
> +		if (buf.buf[0] != comment_line_char ||
> +		    !skip_prefix(buf.buf + 1, " This is a combination of ",
> +				 (const char **)&p))
> +			return error(_("unexpected 1st line of squash message:"
> +				       "\n\n\t%.*s"),
> +				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
> +				     buf.buf);
> +		count = strtol(p, &p2, 10);
> +
> +		if (count < 1 || *p2 != ' ')
> +			return error(_("invalid 1st line of squash message:\n"
> +				       "\n\t%.*s"),
> +				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
> +				     buf.buf);
> +
> +		sprintf((char *)p, "%d", ++count);

Do we know the area pointed at p (which is inside buf) long enough
not to overflow?  If the original were 9 and you incremented to get
10, you would need one extra byte.

> +		if (!*p2)
> +			*p2 = ' ';
> +		else {
> +			*(++p2) = 'c';

p2 points into buf; do we know this increment does not step beyond
its end?  What is the meaning of a letter 'c' here (I do not see a
corresponding one in the scripted update_squash_messages)?

> +			strbuf_insert(&buf, p2 - buf.buf, " ", 1);
> +		}
> +	}
> +	else {

Style: "} else {" (I won't repeat this, as it will become too noisy).

> +		unsigned char head[20];
> +		struct commit *head_commit;
> +		const char *head_message, *body;
> +
> +		if (get_sha1("HEAD", head))
> +			return error(_("need a HEAD to fixup"));
> +		if (!(head_commit = lookup_commit_reference(head)))
> +			return error(_("could not read HEAD"));
> +		if (!(head_message = get_commit_buffer(head_commit, NULL)))
> +			return error(_("could not read HEAD's commit message"));
> +
> +		body = strstr(head_message, "\n\n");
> +		if (!body)
> +			body = "";
> +		else
> +			body = skip_blank_lines(body + 2);

I think I saw you used a helper function find_commit_subject() to do
the above in an earlier patch for "edit" in this series.  Would it
make this part (and another one for "commit" we have after this
if/else) shorter?

>  static int do_pick_commit(enum todo_command command, struct commit *commit,
> -		struct replay_opts *opts)
> +		struct replay_opts *opts, int final_fixup)
>  {
> +	int edit = opts->edit, cleanup_commit_message = 0;
> +	const char *msg_file = edit ? NULL : git_path_merge_msg();
>  	unsigned char head[20];
>  	struct commit *base, *next, *parent;
>  	const char *base_label, *next_label;
>  	struct commit_message msg = { NULL, NULL, NULL, NULL };
>  	struct strbuf msgbuf = STRBUF_INIT;
> -	int res, unborn = 0, allow;
> +	int res, unborn = 0, amend = 0, allow;
>  
>  	if (opts->no_commit) {
>  		/*
> @@ -749,7 +885,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  	else
>  		parent = commit->parents->item;
>  
> -	if (opts->allow_ff &&
> +	if (opts->allow_ff && !is_fixup(command) &&
>  	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
>  	     (!parent && unborn)))
>  		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
> @@ -813,6 +949,28 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  		}
>  	}
>  
> +	if (is_fixup(command)) {
> +		if (update_squash_messages(command, commit, opts))
> +			return -1;
> +		amend = 1;
> +		if (!final_fixup)
> +			msg_file = rebase_path_squash_msg();
> +		else if (file_exists(rebase_path_fixup_msg())) {
> +			cleanup_commit_message = 1;
> +			msg_file = rebase_path_fixup_msg();
> +		}
> +		else {
> +			const char *dest = git_path("SQUASH_MSG");
> +			unlink(dest);
> +			if (copy_file(dest, rebase_path_squash_msg(), 0666))
> +				return error(_("could not rename '%s' to '%s'"),
> +					     rebase_path_squash_msg(), dest);

Perhaps an error from unlink(dest) before copy_file() should also
result in an error return?  After all, that unlink() was added to
mimick the "cp" in the scripted one and copy_file() does not want
to overwrite an existing destination.

> +			unlink(git_path("MERGE_MSG"));

Errors from this and other unlink() that emulates "rm -f" were
unchecked in the scripted original, so not checking for errors is
not a regression.  I would check for an error if I were writing
this, however, because I know I would forget updating these after I
am done with the series.

> +			msg_file = dest;
> +			edit = 1;
> +		}
> +	}
> +
>  	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
>  		res = do_recursive_merge(base, next, base_label, next_label,
>  					 head, &msgbuf, opts);
> @@ -868,8 +1026,13 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  		goto leave;
>  	}
>  	if (!opts->no_commit)
> -		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
> -				     opts, allow, opts->edit, 0, 0);
> +		res = run_git_commit(msg_file, opts, allow, edit, amend,
> +				     cleanup_commit_message);

The introduction of msg_file variable made this call (actually, the
logic to decide which file is affected) much nicer to understand.

> +	if (!res && final_fixup) {
> +		unlink(rebase_path_fixup_msg());
> +		unlink(rebase_path_squash_msg());
> +	}
>  
> @@ -1475,6 +1660,21 @@ static int do_exec(const char *command_line)
>  	return status;
>  }
>  
> +static int is_final_fixup(struct todo_list *todo_list)
> +{
> +	int i = todo_list->current;
> +
> +	if (!is_fixup(todo_list->items[i].command))
> +		return 0;
> +
> +	while (++i < todo_list->nr)
> +		if (is_fixup(todo_list->items[i].command))
> +			return 0;
> +		else if (todo_list->items[i].command < TODO_NOOP)
> +			break;

What follows NOOP are comment and "drop" which is another comment in
disguise, so this one is excluding all the no-op commands in various
shapes, which makes sense but is clear only to a reader who bothered
to go back to "enum todo_command" and checked that fact.  If a check
for "is it one of the no-op commands?" appears only here, a single
liner comment may be sufficient (but necessary) to help readers.
Otherwise a single-liner helper function (similar to is_fixup() you
have) with a descriptive name would be better than a single liner
comment.

Re: [PATCH v2 08/34] sequencer (rebase -i): implement the short commands

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> For users' convenience, most rebase commands can be abbreviated, e.g.
> 'p' instead of 'pick' and 'x' instead of 'exec'. Let's teach the
> sequencer to handle those abbreviated commands just fine.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---

Sensible.

Re: [PATCH v2 09/34] sequencer (rebase -i): write an author-script file

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> +	strbuf_addstr(&buf, "GIT_AUTHOR_NAME='");
> +	while (*message && *message != '\n' && *message != '\r')
> +		if (skip_prefix(message, " <", &message))
> +			break;
> +		else if (*message != '\'')
> +			strbuf_addch(&buf, *(message++));
> +		else
> +			strbuf_addf(&buf, "'\\\\%c'", *(message++));
> +	strbuf_addstr(&buf, "'\nGIT_AUTHOR_EMAIL='");
> +	while (*message && *message != '\n' && *message != '\r')
> +		if (skip_prefix(message, "> ", &message))
> +			break;
> +		else if (*message != '\'')
> +			strbuf_addch(&buf, *(message++));
> +		else
> +			strbuf_addf(&buf, "'\\\\%c'", *(message++));

Aren't these reading from an in-core commit object?  

If so, it should use split_ident_line() for consistency with other
parts of the system to do this parsing.  We should also already have
a helper for simple shell-quoting in quote.c and you would want to
use that instead of open coding like this.

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Jeff King]

On Thu, Dec 15, 2016 at 10:42:53AM -0800, Junio C Hamano wrote:

> > +		sprintf((char *)p, "%d", ++count);
> 
> Do we know the area pointed at p (which is inside buf) long enough
> not to overflow?  If the original were 9 and you incremented to get
> 10, you would need one extra byte.

Even if it is enough, I'd ask to please use xsnprintf(). In the off
chance that there's a programming error, we'd get a nice die("BUG")
instead of a buffer overflow (and it makes the code base easier to audit
for other overflows).

-Peff

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Jeff King]

On Tue, Dec 13, 2016 at 04:30:01PM +0100, Johannes Schindelin wrote:

> +	else {
> +		unsigned char head[20];
> +		struct commit *head_commit;
> +		const char *head_message, *body;
> +
> +		if (get_sha1("HEAD", head))
> +			return error(_("need a HEAD to fixup"));
> +		if (!(head_commit = lookup_commit_reference(head)))
> +			return error(_("could not read HEAD"));
> +		if (!(head_message = get_commit_buffer(head_commit, NULL)))
> +			return error(_("could not read HEAD's commit message"));

This get_commit_buffer() may allocate a fresh buffer...

> +		body = strstr(head_message, "\n\n");
> +		if (!body)
> +			body = "";
> +		else
> +			body = skip_blank_lines(body + 2);
> +		if (write_message(body, strlen(body),
> +				  rebase_path_fixup_msg(), 0))
> +			return error(_("cannot write '%s'"),
> +				     rebase_path_fixup_msg());

...and then this return leaks the result (the other code path hits
unuse_commit_buffer(), and is fine).

This leak was noticed by Coverity. It has a _ton_ of false positives
across the whole project, but it sends out a mail with new ones every
few days, which is usually short enough that I can process it in 30
seconds or so.

I _think_ that email just goes to me and Stefan right now. You can add
yourself at:

  https://scan.coverity.com/projects/git?tab=project_settings

if you already have admin access to the project (which I think you
(Dscho) do).  I wonder if it would be helpful to send that output to the
list.

-Peff

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Stefan Beller]

On Thu, Dec 15, 2016 at 11:03 AM, Jeff King <peff@peff.net> wrote:
> wonder if it would be helpful to send that output to the list.

Sure we can try.

Another project I used to run through coverity (Gerrit), shows
similar characteristics w.r.t. false positives, so people complained
when I was force feeding them the niceties of static analysis.

I'll just try to set it up and see how the mailing list reacts.
(Not sure if you can just add emails there or if the email has
to be verified or such.)

Thanks,
Stefan

Re: [PATCH v2 10/34] sequencer (rebase -i): allow continuing with staged changes

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> When an interactive rebase is interrupted, the user may stage changes
> before continuing, and we need to commit those changes in that case.
>
> Please note that the nested "if" added to the sequencer_continue() is
> not combined into a single "if" because it will be extended with an
> "else" clause in a later patch in this patch series.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 40 ++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 40 insertions(+)
>
> diff --git a/sequencer.c b/sequencer.c
> index 80469b6954..855d3ba503 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -1829,6 +1829,42 @@ static int continue_single_pick(void)
>  	return run_command_v_opt(argv, RUN_GIT_CMD);
>  }
>  
> +static int commit_staged_changes(struct replay_opts *opts)
> +{
> +	int amend = 0;
> +
> +	if (has_unstaged_changes(1))
> +		return error(_("cannot rebase: You have unstaged changes."));

The scripted one from 'master' seems to say

	$path_to_the_file: needs update
	You must edit all merge conflicts and then
	mark them as resolved using git add

when editing an existing commit in this case.  The updated message
looks more sensible for the situation, but I wonder if the control
should even reach at this point.  

One bad thing about reviewing this series is that all the comments
are about codepaths that are not exercised, so they cannot be more
than "they look good".  A comment "If the caller does X, this will
be better than the original" (or this will regress, for that matter)
cannot be validated for its relevance because we won't know the what
the caller does in the endgame while reviewing these earlier steps.

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Jeff King]

On Thu, Dec 15, 2016 at 11:07:34AM -0800, Stefan Beller wrote:

> On Thu, Dec 15, 2016 at 11:03 AM, Jeff King <peff@peff.net> wrote:
> > wonder if it would be helpful to send that output to the list.
> 
> Sure we can try.
> 
> Another project I used to run through coverity (Gerrit), shows
> similar characteristics w.r.t. false positives, so people complained
> when I was force feeding them the niceties of static analysis.
> 
> I'll just try to set it up and see how the mailing list reacts.
> (Not sure if you can just add emails there or if the email has
> to be verified or such.)

I see you added it, but I don't see the confirmation email on the list.
I wonder if it was HTML mail and vger ate it.

-Peff

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Stefan Beller]

On Thu, Dec 15, 2016 at 11:20 AM, Jeff King <peff@peff.net> wrote:
> On Thu, Dec 15, 2016 at 11:07:34AM -0800, Stefan Beller wrote:
>
>> On Thu, Dec 15, 2016 at 11:03 AM, Jeff King <peff@peff.net> wrote:
>> > wonder if it would be helpful to send that output to the list.
>>
>> Sure we can try.
>>
>> Another project I used to run through coverity (Gerrit), shows
>> similar characteristics w.r.t. false positives, so people complained
>> when I was force feeding them the niceties of static analysis.
>>
>> I'll just try to set it up and see how the mailing list reacts.
>> (Not sure if you can just add emails there or if the email has
>> to be verified or such.)
>
> I see you added it, but I don't see the confirmation email on the list.
> I wonder if it was HTML mail and vger ate it.
>
> -Peff

I think I'll setup a dummy gmail account to use for subscription
and then I'll configure that to forward all email from coverity to the list.
(the actual complaints about memleaks etc are plain text, so that
forwarding then should work)

Re: [PATCH v2 11/34] sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> The scripted version of the interactive rebase already does that.

Sensible.  I was wondering why this wasn't there while reviewing
10/34, comparing the two (this is not a suggestion to squash this
into the previous step).

>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index 855d3ba503..abffaf3b40 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -1835,8 +1835,13 @@ static int commit_staged_changes(struct replay_opts *opts)
>  
>  	if (has_unstaged_changes(1))
>  		return error(_("cannot rebase: You have unstaged changes."));
> -	if (!has_uncommitted_changes(0))
> +	if (!has_uncommitted_changes(0)) {
> +		const char *cherry_pick_head = git_path("CHERRY_PICK_HEAD");
> +
> +		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
> +			return error(_("could not remove CHERRY_PICK_HEAD"));
>  		return 0;
> +	}
>  
>  	if (file_exists(rebase_path_amend())) {
>  		struct strbuf rev = STRBUF_INIT;

Re: [PATCH v2 14/34] sequencer (rebase -i): update refs after a successful rebase

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> An interactive rebase operates on a detached HEAD (to keep the reflog
> of the original branch relatively clean), and updates the branch only
> at the end.
>
> Now that the sequencer learns to perform interactive rebases, it also
> needs to learn the trick to update the branch before removing the
> directory containing the state of the interactive rebase.
>
> We introduce a new head_ref variable in a wider scope than necessary at
> the moment, to allow for a later patch that prints out "Successfully
> rebased and updated <ref>".
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 32 +++++++++++++++++++++++++++++++-
>  1 file changed, 31 insertions(+), 1 deletion(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index a6625e765d..a4e9b326ba 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -100,6 +100,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
>  static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
>  static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
>  static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
> +static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
> +static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
>  
>  static inline int is_rebase_i(const struct replay_opts *opts)
>  {
> @@ -1793,12 +1795,39 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  	}
>  
>  	if (is_rebase_i(opts)) {
> -		struct strbuf buf = STRBUF_INIT;
> +		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
>  
>  		/* Stopped in the middle, as planned? */
>  		if (todo_list->current < todo_list->nr)
>  			return 0;
>  
> +		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
> +				starts_with(head_ref.buf, "refs/")) {
> +			unsigned char head[20], orig[20];
> +
> +			if (get_sha1("HEAD", head))
> +				return error(_("cannot read HEAD"));
> +			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
> +					get_sha1_hex(buf.buf, orig))
> +				return error(_("could not read orig-head"));
> +			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
> +				head_ref.buf);
> +			if (!read_oneliner(&buf, rebase_path_onto(), 0))
> +				return error(_("could not read 'onto'"));
> +			if (update_ref(buf.buf, head_ref.buf, head, orig,
> +					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
> +				return error(_("could not update %s"),
> +					head_ref.buf);
> +			strbuf_reset(&buf);
> +			strbuf_addf(&buf,
> +				"rebase -i (finish): returning to %s",
> +				head_ref.buf);
> +			if (create_symref("HEAD", head_ref.buf, buf.buf))
> +				return error(_("could not update HEAD to %s"),
> +					head_ref.buf);

All of the above return error() calls leak head_ref.buf; in addition
some leak buf.buf, too.

> +			strbuf_reset(&buf);
> +		}
> +
>  		if (opts->verbose) {
>  			const char *argv[] = {
>  				"diff-tree", "--stat", NULL, NULL
> @@ -1813,6 +1842,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  			strbuf_reset(&buf);
>  		}
>  		strbuf_release(&buf);
> +		strbuf_release(&head_ref);
>  	}
>  
>  	/*

Re: [PATCH v2 15/34] sequencer (rebase -i): leave a patch upon error

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> When doing an interactive rebase, we want to leave a 'patch' file for
> further inspection by the user (even if we never tried to actually apply
> that patch, since we're cherry-picking instead).
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---

Yup.  

The other day, I was kind of surprised to see the "patch" file
produced when I tried to do "git rebase -i HEAD^^ HEAD" with the one
in current Git (not yours), marked the first one "edit", and then it
gave me control back.  Obviously there was no conflict and I could
just do "git show" if I wanted to see what the original change was,
but the "patch" file was there.  I personally never have looked at
the "patch" file in such a situation, and I kind of feel it is
wasteful, but I can see people expect to find one there whenever
"rebase -i" stops and gives control back.  It is good that you are
preserving the behaviour.

>  sequencer.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/sequencer.c b/sequencer.c
> index a4e9b326ba..4361fe0e94 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -1777,6 +1777,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  				return error_failed_squash(item->commit, opts,
>  					item->arg_len, item->arg);
>  			}
> +			else if (res && is_rebase_i(opts))
> +				return res | error_with_patch(item->commit,
> +					item->arg, item->arg_len, opts, res, 0);
>  		}
>  		else if (item->command == TODO_EXEC) {
>  			char *end_of_arg = (char *)(item->arg + item->arg_len);

Re: [PATCH v2 18/34] sequencer (rebase -i): refactor setting the reflog message

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> This makes the code DRYer, with the obvious benefit that we can enhance
> the code further in a single place.
>
> We can also reuse the functionality elsewhere by calling this new
> function.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 33 ++++++++++++++++++++++++++-------
>  1 file changed, 26 insertions(+), 7 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index 33fb36dcbe..d20efad562 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -1750,6 +1750,26 @@ static int is_final_fixup(struct todo_list *todo_list)
>  	return 1;
>  }
>  
> +static const char *reflog_message(struct replay_opts *opts,
> +	const char *sub_action, const char *fmt, ...)
> +{
> +	va_list ap;
> +	static struct strbuf buf = STRBUF_INIT;
> +
> +	va_start(ap, fmt);
> +	strbuf_reset(&buf);
> +	strbuf_addstr(&buf, action_name(opts));
> +	if (sub_action)
> +		strbuf_addf(&buf, " (%s)", sub_action);
> +	if (fmt) {
> +		strbuf_addstr(&buf, ": ");
> +		strbuf_vaddf(&buf, fmt, ap);
> +	}
> +	va_end(ap);
> +
> +	return buf.buf;
> +}

It is unlikely for a single caller to want to format another reflog
entry after formating one but before consuming it [*1*], so this
"call this, and you can use the return value until you call it the
next time without worrying about leakage" is quite a reasonable
pattern to employ.

[Footnote]

*1* And it is unlikely that this will run in a multi-threaded
    environment, either ;-)

Re: [PATCH v2 20/34] sequencer (rebase -i): copy commit notes at end

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> +static void flush_rewritten_pending(void) {
> +	struct strbuf buf = STRBUF_INIT;
> +	unsigned char newsha1[20];
> +	FILE *out;
> +
> +	if (strbuf_read_file(&buf, rebase_path_rewritten_pending(), 82) > 0 &&
> +			!get_sha1("HEAD", newsha1) &&
> +			(out = fopen(rebase_path_rewritten_list(), "a"))) {

An error in fopen() here ...

> + ...
> +	}
> +	strbuf_release(&buf);
> +}
> +
> +static void record_in_rewritten(struct object_id *oid,
> +		enum todo_command next_command) {
> +	FILE *out = fopen(rebase_path_rewritten_pending(), "a");
> +
> +	if (!out)
> +		return;

... and here are ignored as an insignificant error in the scripted
version, and this one does the same.  

> +
> +	fprintf(out, "%s\n", oid_to_hex(oid));
> +	fclose(out);
> +
> +	if (!is_fixup(next_command))
> +		flush_rewritten_pending();
> +}
> +
>  static int do_pick_commit(enum todo_command command, struct commit *commit,
>  		struct replay_opts *opts, int final_fixup)
>  {
> @@ -1750,6 +1797,17 @@ static int is_final_fixup(struct todo_list *todo_list)
>  	return 1;
>  }
>  
> +static enum todo_command peek_command(struct todo_list *todo_list, int offset)
> +{
> +	int i;
> +
> +	for (i = todo_list->current + offset; i < todo_list->nr; i++)
> +		if (todo_list->items[i].command != TODO_NOOP)
> +			return todo_list->items[i].command;

Makes me wonder, after having commented on 07/34 regarding the fact
that in the end you would end up having three variants of no-op
(i.e. NOOP, DROP and COMMENT), what definition of a "command" this
function uses to return its result, when asked to "peek".  I suspect
that this will be updated in a later patch to do "< TODO_NOOP"
instead?  If so, then that answers one question in my comment on
07/34, i.e.

    If a check for "is it one of the no-op commands?" appears only
    here, a single liner comment may be sufficient (but necessary)
    to help readers.  Otherwise a single-liner helper function
    (similar to is_fixup() you have) with a descriptive name would
    be better than a single liner comment.

The answer is "no, it is not just there" hence the conclusion is "we
want a helper with a descriptive name".

Re: [PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

[Stephan Beyer]

Hi,

On 12/14/2016 08:29 PM, Junio C Hamano wrote:
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> -/* We will introduce the 'interactive rebase' mode later */
>>  static inline int is_rebase_i(const struct replay_opts *opts)
>>  {
>> -	return 0;
>> +	return opts->action == REPLAY_INTERACTIVE_REBASE;
>>  }
>>  
>>  static const char *get_dir(const struct replay_opts *opts)
>>  {
>> +	if (is_rebase_i(opts))
>> +		return rebase_path();
>>  	return git_path_seq_dir();
>>  }
> 
> This obviously makes the assumption made by 39784cd362 ("sequencer:
> remove useless get_dir() function", 2016-12-07) invalid, where the
> "remove useless" thought that the callers of this function wants a
> single answer that does not depend on opts.
> 
> I'll revert that commit from the sb/sequencer-abort-safety topic (as
> the topic is in 'next' already) to keep this one.  Please holler if
> that is a mistaken decision.

It seems to be the right decision if one wants to keep the internal-path
backwards-compatibility of "git rebase -i" (and it seems that Dscho
wants to keep it).

However, maintaining more than one directory (like "sequencer" for
sequencer state and "rebase-merge" for rebase todo and log) can cause
the necessity to be even more careful when hacking on sequencer... For
example, the cleanup code must delete both of them, not only one of them.

Hence, I think that it's a wiser choice to keep one directory for
sequencer state *and* additional files.
If we (a) save everything in "sequencer", we break the internal-path
backwards-compatbility but we can get rid of get_dir()...
If we (b) save everything in "rebase-merge" when using rebase -i and
save everything in "sequencer" for pick/revert, we are 100%
backwards-compatible, but we have to change every occurrence of
git_path_seq_dir() to get_dir() and the GIT_PATH_FUNC definitions of
git_path_{todo,opts,head}_file must be replaced by definitions using
get_dir().

Best
  Stephan

Re: [PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

[Johannes Schindelin]

Hi,

On Tue, 13 Dec 2016, Junio C Hamano wrote:

> Linus Torvalds <torvalds@linux-foundation.org> writes:
> 
> > On Tue, Dec 13, 2016 at 12:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
> >> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> >>
> >>> +/*
> >>> + * Note that ordering matters in this enum. Not only must it match the mapping
> >>> + * below, it is also divided into several sections that matter.  When adding
> >>> + * new commands, make sure you add it in the right section.
> >>> + */
> >>
> >> Good thinking.

Not my thinking... This was in direct response to a suggestion by Dennis
Kaarsemaker, I cannot take credit for the idea.

> Makes me wish C were a better language, though ;-)
> >
> > Do this:
> >
> >   static const char *todo_command_strings[] = {
> >       [TODO_PICK] = "pick",
> >       [TODO_REVERT] = "revert",
> >       [TODO_NOOP] = "noop:,
> >   };
> >
> > which makes the array be order-independent. You still need to make
> > sure you fill in all the entries, of course, but it tends to avoid at
> > least one gotcha, and it makes it more obvious how the two are tied
> > together.
> 
> Yes, I know.  But I do not think the variant of C we stick to is not
> new enough to have that.

Let me try to express this without double negation: our coding guidelines
state very explicitly that we do not use C99 initializers, and it also
explains why: we want to support a broader range of compilers. For
details, see:

https://github.com/git/git/blob/v2.11.0/Documentation/CodingGuidelines#L179-L181

TBH I briefly considered going the same route as I did for the fsck
warnings (taking a lot of heat for the ugliness):

https://github.com/git/git/blob/v2.11.0/fsck.c#L17-L85

It would have looked somewhat like this:

	#define FOREACH_TODO(FUNC) \
		FUNC(PICK, 'p', "pick") \
		FUNC(REVERT, 0, "revert") \
		FUNC(EDIT, 'e', "edit") \
		...
		FUNC(COMMENT, 0, NULL)

	#define TODO(id, short, long) TODO_##id,
	enum todo_command {
		FOREACH_TODO(TODO)
		TODO_MAX
	};
	#undef TODO

	#define TODO(id, short, long) { short, long },
	static struct {
		char c;
		const char *str;
	} todo_command_info[] = {
		FOREACH_TODO(TODO)
		{ 0, NULL }
	};
	#undef TODO

I have to admit that even I prefer the version I provided in my
contribution over the "fsck method" outlined above.

Ciao,
Dscho

Re: [PATCH v2 00/34] Teach the sequencer to act as rebase -i's backend

[Johannes Schindelin]

Hi Junio,

On Tue, 13 Dec 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Apart from mostly cosmetic patches (and the occasional odd bug that I
> > fixed promptly), I used these patches since mid May to perform all of my
> > interactive rebases. In mid June, I had the idea to teach rebase -i to
> > run *both* scripted rebase and rebase--helper and to cross-validate the
> > results. This slowed down all my interactive rebases since, but helped
> > me catch three rather obscure bugs (e.g. that git commit --fixup unfolds
> > long onelines and rebase -i still finds the correct original commit).
> > ...
> > Please note that the interdiff vs v1 is only of limited use: too many
> > things changed in the meantime, in particular the prepare-sequencer
> > branch that went through a couple of iterations before it found its way
> > into git.git's master branch. So please take the interdiff with a
> > mountain range of salt.
> > ...
> > Changes since v1:
> > ...
> > - removed the beautiful ordinal logic (to print out "1st", "2nd", "3rd"
> >   etc) and made things consistent with the current `rebase -i`.
> 
> It was removed because it was too Anglo-centric and unusable in i18n
> context, no?

Yes, but I remember putting in a lot of time to try to come up with the
most elegant way to convert a number into an English ordinal in a shell
function. That's where all that wistfulness came from.

> Judging from the list above, interdiff are pretty much all cosmetic
> and that is why you say it is only of limited use, I guess.

No, I said that it is only of limited use because I had to fudge things to
come up with an interdiff. I had to fudge things because there is no
interdiff: it would require the previous iteration of the patch series to
apply cleanly to the current `master`, which it does not. So I rebased the
patches and left as much intact as possible, which means that the rebased
changes do not even compile because they were based on a previous
iteration of the prepare-sequencer patch series that did not make it into
`master` without substantial changes.

>     ... goes and reads the remainder and finds that these were
>     ... all minor changes, mostly cosmetic, with a helper function
>     ... refactored out or two and things of that nature.
> 
> It is actually a good thing.  We do not want to see it deviate too
> drastically from what you have been testing for some months.

Well, that ship has sailed. Neither of the patch series
"require-clean-work-tree", "libify-sequencer" and "prepare-sequencer" made
it into `master` without substantial deviations from the code I had been
testing and improving for half a year. I did not expect the code to be
accepted unchanged, anyways.

Ciao,
Dscho

Re: [PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

[Johannes Schindelin]

Hi Junio,

On Tue, 13 Dec 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > @@ -395,7 +414,10 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
> >  
> >  	if (active_cache_changed &&
> >  	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
> > -		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
> > +		/*
> > +		 * TRANSLATORS: %s will be "revert", "cherry-pick" or
> > +		 * "rebase -i".
> > +		 */
> 
> IIRC, the "TRANSLATORS:" comment has to deviate from our coding
> style due to tool limitation and has to be done like this:
> 
> > +		/* TRANSLATORS: %s will be "revert", "cherry-pick" or
> > +		 * "rebase -i".
> > +		 */

Aargh. I had fixed that in another patch series already, but failed to do
so in this here patch series. Sorry.

Fixed.

> > @@ -1204,6 +1226,9 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
> >  	const char *todo_path = get_todo_path(opts);
> >  	int next = todo_list->current, offset, fd;
> >  
> > +	if (is_rebase_i(opts))
> > +		next++;
> > +
> 
> This is because...?  Everybody else counts 0-based while rebase-i
> counts from 1 or something?

No. Since its conception, edit-patch-series.sh (the artist now known as
rebase -i) processed each line of the "todo script" by writing a new
"todo" file, skipping the first line, and appending it to the "done" file.

The sequencer chooses to write the new "insn sheet" after a successful
operation.

This is not an option for rebase -i, as it has many error paths and it was
simply impossible to express "save the todo script in case of failure" in
shell script.

I added a comment to clarify why the current command is not written to
git-rebase-todo.

Ciao,
Dscho

Re: [PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

[Johannes Schindelin]

Hi Junio,

On Tue, 13 Dec 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > @@ -1292,7 +1316,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
> >  		struct todo_item *item = todo_list->items + todo_list->current;
> >  		if (save_todo(todo_list, opts))
> >  			return -1;
> > -		res = do_pick_commit(item->command, item->commit, opts);
> > +		if (item->command <= TODO_REVERT)
> > +			res = do_pick_commit(item->command, item->commit,
> > +					opts);
> > +		else if (item->command != TODO_NOOP)
> > +			return error(_("unknown command %d"), item->command);
> 
> I wonder if making this a switch() statement is easier to read in
> the longer run.  The only thing at this point we are gaining by "not
> only mapping and enum must match, the orders matter" is so that this
> codepath can do the same thing for PICK and REVERT, but these two
> would become more and more minority as we learn more words.

I doubt that this is easier to read. There are essentially three
categories we are handling: exec, comments, and everything else. IMO the
current code is the easiest to understand.

Ciao,
Dscho

Re: [PATCH v2 03/34] sequencer (rebase -i): implement the 'edit' command

[Johannes Schindelin]

Hi Junio,

On Tue, 13 Dec 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > @@ -43,6 +44,20 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
> >   */
> >  static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
> >  /*
> 
> It is minor, but please have a blank line to separate the logical
> blocks.

Ah, but where to draw the line? These comments in front of some of the
rebase_path_* functions clarify the particular role of the corresponding
paths, but all of those functions belong to the same block of
rebase_path_* functions. That latter circumstance is what made me decide
to *not* insert blank lines here, but only a blank line before that block
(to separate from the sequencer path functions) and after that block (to
separate from the rest of the code).

> If you have "comment for thing A" before "thing A", then having a blank
> after that before "comment for thing B" and "thing B" that follow would
> help.  Otherwise "thing A" immediately followed by "comment for thing B"
> are (mis)read together, leading to nonsense.

In this case, I think it is quite obvious that the comments belong to the
immediately following line, and that all of the path functions are part of
a bigger block.

> > + * When an "edit" rebase command is being processed, the SHA1 of the
> > + * commit to be edited is recorded in this file.  When "git rebase
> > + * --continue" is executed, if there are any staged changes then they
> > + * will be amended to the HEAD commit, but only provided the HEAD
> > + * commit is still the commit to be edited.  When any other rebase
> > + * command is processed, this file is deleted.
> > + */
> > +static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
> > +/*
> > + * When we stop at a given patch via the "edit" command, this file contains
> > + * the long commit name of the corresponding patch.
> 
> If you abbreviate an object name to 38-hex that is still long but
> that is not full; if you meant full 40-hex, better spell it out as
> "full"---that conveys useful information to programmers (e.g. they
> can just use get_sha1_hex()).
> 
> But I think you are writing short_commit_name() to it?  So perhaps
> "an abbreviated commit object name"?

Fixed.

> > @@ -1301,9 +1318,87 @@ static int save_opts(struct replay_opts *opts)
> >  	return res;
> >  }
> >  
> > +static int make_patch(struct commit *commit, struct replay_opts *opts)
> > +{
> > +	struct strbuf buf = STRBUF_INIT;
> > +	struct rev_info log_tree_opt;
> > +	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject, *p;
> > +	int res = 0;
> > +
> > +	p = short_commit_name(commit);
> > +	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
> > +		return -1;
> > +
> > +	strbuf_addf(&buf, "%s/patch", get_dir(opts));
> > +	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
> > +	init_revisions(&log_tree_opt, NULL);
> > +	log_tree_opt.abbrev = 0;
> > +	log_tree_opt.diff = 1;
> > +	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
> > +	log_tree_opt.disable_stdin = 1;
> > +	log_tree_opt.no_commit_id = 1;
> > +	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
> > +	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
> > +	if (!log_tree_opt.diffopt.file)
> > +		res |= error_errno(_("could not open '%s'"), buf.buf);
> > +	else {
> > +		res |= log_tree_commit(&log_tree_opt, commit);
> > +		fclose(log_tree_opt.diffopt.file);
> > +	}
> > +	strbuf_reset(&buf);
> > +	strbuf_addf(&buf, "%s/message", get_dir(opts));
> > +	if (!file_exists(buf.buf)) {
> > +		find_commit_subject(commit_buffer, &subject);
> > +		res |= write_message(subject, strlen(subject), buf.buf, 1);
> > +		unuse_commit_buffer(commit, commit_buffer);
> > +	}
> > +	strbuf_release(&buf);
> > +
> > +	return res;
> > +}
> 
> OK.  This seems to match what scripted make_patch does in a handful
> of lines.  We probably should have given you a helper to reduce
> boilerplate that sets up log_tree_opt so that this function does not
> have to be this long, but that is a separate topic.
> 
> Does it matter output_format is set to FORMAT_PATCH here, though?
> With --no-commit-id set, I suspect there is no log message or
> authorship information given to the output.
> 
> As you are only interested in seeing the patch/diff in this file and
> the log is stored in a separate "message" file, as long as "patch"
> file gets the patch correctly, it is not a problem, but it just
> looked strange to see FORMAT_PATCH there.

I am indifferent as to FORMAT_PATCH. It is there simply as a direct
translation of the `diff-tree -p` command in
https://github.com/git/git/blob/v2.11.0/git-rebase--interactive.sh#L188

Ciao,
Dscho

Re: [PATCH v2 04/34] sequencer (rebase -i): implement the 'exec' command

[Johannes Schindelin]

Hi Junio,

On Tue, 13 Dec 2016, Junio C Hamano wrote:

> Please have "else if" on the same line as "}" that closes the
> "if (...) {" in the same if/else if/else cascade.

Fixed. This affected quite a few more patches than just this one, and
added two more to this already large patch series.

Ciao,
Dscho

Re: [PATCH v2 00/34] Teach the sequencer to act as rebase -i's backend

[Johannes Schindelin]

Hi Hannes,

On Wed, 14 Dec 2016, Johannes Sixt wrote:

> Am 13.12.2016 um 16:29 schrieb Johannes Schindelin:
> > base-commit: 8d7a455ed52e2a96debc080dfc011b6bb00db5d2
> > Published-As: https://github.com/dscho/git/releases/tag/sequencer-i-v2
> > Fetch-It-Via: git fetch https://github.com/dscho/git sequencer-i-v2
> 
> Thank you so much!
> 
> I would appreciate if you could publish a branch that contains the end game so
> that I can test it, too. Currently I am still using
> 
>  git://github.com/dscho/git interactive-rebase (fca871a3cf4d)

That's the one. Unless I forget, I use the Git garden shears to update
'interactive-rebase' whenever I update 'sequencer-i', too.

Ciao,
Dscho

Re: [PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

[Johannes Schindelin]

Hi Stephan,

On Sat, 17 Dec 2016, Stephan Beyer wrote:

> On 12/14/2016 08:29 PM, Junio C Hamano wrote:
> > Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> >> -/* We will introduce the 'interactive rebase' mode later */
> >>  static inline int is_rebase_i(const struct replay_opts *opts)
> >>  {
> >> -	return 0;
> >> +	return opts->action == REPLAY_INTERACTIVE_REBASE;
> >>  }
> >>  
> >>  static const char *get_dir(const struct replay_opts *opts)
> >>  {
> >> +	if (is_rebase_i(opts))
> >> +		return rebase_path();
> >>  	return git_path_seq_dir();
> >>  }
> > 
> > This obviously makes the assumption made by 39784cd362 ("sequencer:
> > remove useless get_dir() function", 2016-12-07) invalid, where the
> > "remove useless" thought that the callers of this function wants a
> > single answer that does not depend on opts.
> > 
> > I'll revert that commit from the sb/sequencer-abort-safety topic (as
> > the topic is in 'next' already) to keep this one.  Please holler if
> > that is a mistaken decision.
> 
> It seems to be the right decision if one wants to keep the internal-path
> backwards-compatibility of "git rebase -i" (and it seems that Dscho
> wants to keep it).

You make it sound as if I had any choice there. But you probably know as
well as I do that scripted usage of rebase -i relies on the "internal-path
backwards-compatibility", and as one of my stated goals was not to break
anybody's existing rebase -i usage, well, you know.

> However, maintaining more than one directory (like "sequencer" for
> sequencer state and "rebase-merge" for rebase todo and log) can cause
> the necessity to be even more careful when hacking on sequencer... For
> example, the cleanup code must delete both of them, not only one of them.

That is incorrect. It depends on the options which directory is used. And
it is that directory (and not both) that should be cleaned up in the end.

Otherwise you run into a ton of pain e.g. when running a rebase -i with an
`exec git cherry-pick ...` line: all of a sudden, that little innocuous
line would simply destroy the state directory of the current rebase -i.

That's a rather big no-no.

> Hence, I think that it's a wiser choice to keep one directory for
> sequencer state *and* additional files.

See above for a good reason not to choose that.

> If we (a) save everything in "sequencer", we break the internal-path
> backwards-compatbility but we can get rid of get_dir()...

... and we break power users' scripts that use rebase -i. Making those
power users very angry.

Including me.

> If we (b) save everything in "rebase-merge" when using rebase -i and
> save everything in "sequencer" for pick/revert, we are 100%
> backwards-compatible, but we have to change every occurrence of
> git_path_seq_dir() to get_dir()

Yes, that is exactly what we have to do. And that is exactly what I did,
too, in the patch series I labeled "prepare-sequencer".

The only three uses of git_path_seq_dir() I left intact are:

- in builtin/commit.c, where it is used to determine whether a cherry-pick
  is really still in progress, after testing that there is a
  CHERRY_PICK_HEAD file

- in get_dir() (obviously)

- in create_seq_dir(), where we could technically avoid calling the
  function a whopping three times and use a parameter instead (not my
  construction site, though, so I leave this alone)

> and the GIT_PATH_FUNC definitions of git_path_{todo,opts,head}_file must
> be replaced by definitions using get_dir().

That is incorrect. The git_path_todo(), git_path_opts() and
git_path_head() functions need to be used in the cherry-pick/revert code
path. So they need to be left as-are.

Ciao,
Dscho

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Johannes Schindelin]

Hi Junio,

On Thu, 15 Dec 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > diff --git a/sequencer.c b/sequencer.c
> > index f6e20b142a..271c21581d 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -45,6 +45,35 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
> >   */
> >  static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
> >  /*
> > + * The commit message that is planned to be used for any changes that
> > + * need to be committed following a user interaction.
> > + */
> > +static GIT_PATH_FUNC(rebase_path_message, "rebase-merge/message")
> > +/*
> > + * The file into which is accumulated the suggested commit message for
> > + * squash/fixup commands. When the first of a series of squash/fixups
> 
> The same comment as 03/34 applies here, regarding blank line to
> separate logical unit.

Same rationale here: the path functions are one big continuous block, with
comments obviously applying to the immediately following line only.

> > +static int update_squash_messages(enum todo_command command,
> > +		struct commit *commit, struct replay_opts *opts)
> > +{
> > +	struct strbuf buf = STRBUF_INIT;
> > +	int count, res;
> > +	const char *message, *body;
> > +
> > +	if (file_exists(rebase_path_squash_msg())) {
> > +		char *p, *p2;
> > +
> > +		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
> > +			return error(_("could not read '%s'"),
> > +				rebase_path_squash_msg());
> > +
> > +		if (buf.buf[0] != comment_line_char ||
> > +		    !skip_prefix(buf.buf + 1, " This is a combination of ",
> > +				 (const char **)&p))
> > +			return error(_("unexpected 1st line of squash message:"
> > +				       "\n\n\t%.*s"),
> > +				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
> > +				     buf.buf);
> > +		count = strtol(p, &p2, 10);
> > +
> > +		if (count < 1 || *p2 != ' ')
> > +			return error(_("invalid 1st line of squash message:\n"
> > +				       "\n\t%.*s"),
> > +				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
> > +				     buf.buf);
> > +
> > +		sprintf((char *)p, "%d", ++count);
> 
> Do we know the area pointed at p (which is inside buf) long enough
> not to overflow?

Yes, we know that: p2 points to the byte after the parsed number, and said
byte is a space (ASCII 0x20), as verified by the if() above.

> If the original were 9 and you incremented to get 10, you would need one
> extra byte.

Exactly. That extra byte (if needed) is 0x20, as verified above, and can
be overwritten.

If that extra byte (to which p2 points) is *not* overwritten, i.e. if the
new count requires the same amount of space in decimal representation as
the previous count, it is now NUL, as tested here:

> > +		if (!*p2)
> > +			*p2 = ' ';
> > +		else {
> > +			*(++p2) = 'c';
> 
> p2 points into buf; do we know this increment does not step beyond
> its end?  What is the meaning of a letter 'c' here (I do not see a
> corresponding one in the scripted update_squash_messages)?

This clause is entered only when the space needed by the previous count
was not sufficient to hold the new count, at which point we know that not
only the space after the old count was overwritten, but also the 'c' of
the "commits" string.

Therefore, this clause reinstates the 'c' and inserts the space, so that
everything is groovy again:

> > +			strbuf_insert(&buf, p2 - buf.buf, " ", 1);
> > +		}

Having said that, I just realized that this code was only safe as long as
the squash messages were not localized.

I changed the code to imitate more closely what the shell script does. It
made the code a little more verbose, but it should work better as a
consequence, and I am pretty certain you will find it easier to verify
that it is correct.

> > +	}
> > +	else {
> 
> Style: "} else {" (I won't repeat this, as it will become too noisy).

It is not necessary to repeat this, either, as I took the first such
comment as a strong hint to look at the entire patch series and fix it
appropriately.

> 
> > +		unsigned char head[20];
> > +		struct commit *head_commit;
> > +		const char *head_message, *body;
> > +
> > +		if (get_sha1("HEAD", head))
> > +			return error(_("need a HEAD to fixup"));
> > +		if (!(head_commit = lookup_commit_reference(head)))
> > +			return error(_("could not read HEAD"));
> > +		if (!(head_message = get_commit_buffer(head_commit, NULL)))
> > +			return error(_("could not read HEAD's commit message"));
> > +
> > +		body = strstr(head_message, "\n\n");
> > +		if (!body)
> > +			body = "";
> > +		else
> > +			body = skip_blank_lines(body + 2);
> 
> I think I saw you used a helper function find_commit_subject() to do
> the above in an earlier patch for "edit" in this series.  Would it
> make this part (and another one for "commit" we have after this
> if/else) shorter?

Right, and by using the helper function, I also fixed a bug handling funny
commit objects that had more than one empty line separating header from
the message.

Of course, there is a third location in sequencer.c (predating my patches)
that uses the very same idiom. Yet another patch added to this growing
patch series...

> >  static int do_pick_commit(enum todo_command command, struct commit *commit,
> > -		struct replay_opts *opts)
> > +		struct replay_opts *opts, int final_fixup)
> >  {
> > +	int edit = opts->edit, cleanup_commit_message = 0;
> > +	const char *msg_file = edit ? NULL : git_path_merge_msg();
> >  	unsigned char head[20];
> >  	struct commit *base, *next, *parent;
> >  	const char *base_label, *next_label;
> >  	struct commit_message msg = { NULL, NULL, NULL, NULL };
> >  	struct strbuf msgbuf = STRBUF_INIT;
> > -	int res, unborn = 0, allow;
> > +	int res, unborn = 0, amend = 0, allow;
> >  
> >  	if (opts->no_commit) {
> >  		/*
> > @@ -749,7 +885,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
> >  	else
> >  		parent = commit->parents->item;
> >  
> > -	if (opts->allow_ff &&
> > +	if (opts->allow_ff && !is_fixup(command) &&
> >  	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
> >  	     (!parent && unborn)))
> >  		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
> > @@ -813,6 +949,28 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
> >  		}
> >  	}
> >  
> > +	if (is_fixup(command)) {
> > +		if (update_squash_messages(command, commit, opts))
> > +			return -1;
> > +		amend = 1;
> > +		if (!final_fixup)
> > +			msg_file = rebase_path_squash_msg();
> > +		else if (file_exists(rebase_path_fixup_msg())) {
> > +			cleanup_commit_message = 1;
> > +			msg_file = rebase_path_fixup_msg();
> > +		}
> > +		else {
> > +			const char *dest = git_path("SQUASH_MSG");
> > +			unlink(dest);
> > +			if (copy_file(dest, rebase_path_squash_msg(), 0666))
> > +				return error(_("could not rename '%s' to '%s'"),
> > +					     rebase_path_squash_msg(), dest);
> 
> Perhaps an error from unlink(dest) before copy_file() should also
> result in an error return?

No, because the most likely cause for that `unlink()` to fail is that the
destination does not exist, and it is fine if it does not exist yet.

No worries, copy_file() will fail if we could not remove any existing file
and we still error out in that case.

> > +			unlink(git_path("MERGE_MSG"));
> 
> Errors from this and other unlink() that emulates "rm -f" were
> unchecked in the scripted original, so not checking for errors is
> not a regression.  I would check for an error if I were writing
> this, however, because I know I would forget updating these after I
> am done with the series.

The problem is that these files do not necessarily exist. We only unlink()
them to make sure that they do not exist afterwards.

In any case, I am still more interested in a faithful translation than
already starting to improve the code at this stage.

> > @@ -1475,6 +1660,21 @@ static int do_exec(const char *command_line)
> >  	return status;
> >  }
> >  
> > +static int is_final_fixup(struct todo_list *todo_list)
> > +{
> > +	int i = todo_list->current;
> > +
> > +	if (!is_fixup(todo_list->items[i].command))
> > +		return 0;
> > +
> > +	while (++i < todo_list->nr)
> > +		if (is_fixup(todo_list->items[i].command))
> > +			return 0;
> > +		else if (todo_list->items[i].command < TODO_NOOP)
> > +			break;
> 
> What follows NOOP are comment and "drop" which is another comment in
> disguise, so this one is excluding all the no-op commands in various
> shapes, which makes sense but is clear only to a reader who bothered
> to go back to "enum todo_command" and checked that fact.  If a check
> for "is it one of the no-op commands?" appears only here, a single
> liner comment may be sufficient (but necessary) to help readers.
> Otherwise a single-liner helper function (similar to is_fixup() you
> have) with a descriptive name would be better than a single liner
> comment.

True. I introduced is_noop().

Ciao,
Dscho

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Johannes Schindelin]

Hi Peff,

On Thu, 15 Dec 2016, Jeff King wrote:

> On Thu, Dec 15, 2016 at 10:42:53AM -0800, Junio C Hamano wrote:
> 
> > > +		sprintf((char *)p, "%d", ++count);
> > 
> > Do we know the area pointed at p (which is inside buf) long enough
> > not to overflow?  If the original were 9 and you incremented to get
> > 10, you would need one extra byte.
> 
> Even if it is enough, I'd ask to please use xsnprintf(). In the off
> chance that there's a programming error, we'd get a nice die("BUG")
> instead of a buffer overflow (and it makes the code base easier to audit
> for other overflows).

I ended up with more verbose, easier-to-read code that does not try to do
things in-place, in favor of being slightly more wasteful with strbufs.

Ciao,
Dscho

Re: [PATCH v2 09/34] sequencer (rebase -i): write an author-script file

[Johannes Schindelin]

Hi Junio,

On Thu, 15 Dec 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > +	strbuf_addstr(&buf, "GIT_AUTHOR_NAME='");
> > +	while (*message && *message != '\n' && *message != '\r')
> > +		if (skip_prefix(message, " <", &message))
> > +			break;
> > +		else if (*message != '\'')
> > +			strbuf_addch(&buf, *(message++));
> > +		else
> > +			strbuf_addf(&buf, "'\\\\%c'", *(message++));
> > +	strbuf_addstr(&buf, "'\nGIT_AUTHOR_EMAIL='");
> > +	while (*message && *message != '\n' && *message != '\r')
> > +		if (skip_prefix(message, "> ", &message))
> > +			break;
> > +		else if (*message != '\'')
> > +			strbuf_addch(&buf, *(message++));
> > +		else
> > +			strbuf_addf(&buf, "'\\\\%c'", *(message++));
> 
> Aren't these reading from an in-core commit object?  
> 
> If so, it should use split_ident_line() for consistency with other
> parts of the system to do this parsing.  We should also already have
> a helper for simple shell-quoting in quote.c and you would want to
> use that instead of open coding like this.

We keep coming back to the same argument. You want this quoting/dequoting
to be turned into a full-fledged parser. And I keep pointing out that the
code here does not *need* to parse but only construct an environment
block.

Hopefully the next iteration that integrates Peff's suggestions will find
more of your approval.

Ciao,
Dscho

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Johannes Schindelin]

Hi Peff,

On Thu, 15 Dec 2016, Jeff King wrote:

> On Tue, Dec 13, 2016 at 04:30:01PM +0100, Johannes Schindelin wrote:
> 
> > +	else {
> > +		unsigned char head[20];
> > +		struct commit *head_commit;
> > +		const char *head_message, *body;
> > +
> > +		if (get_sha1("HEAD", head))
> > +			return error(_("need a HEAD to fixup"));
> > +		if (!(head_commit = lookup_commit_reference(head)))
> > +			return error(_("could not read HEAD"));
> > +		if (!(head_message = get_commit_buffer(head_commit, NULL)))
> > +			return error(_("could not read HEAD's commit message"));
> 
> This get_commit_buffer() may allocate a fresh buffer...
> 
> > +		body = strstr(head_message, "\n\n");
> > +		if (!body)
> > +			body = "";
> > +		else
> > +			body = skip_blank_lines(body + 2);
> > +		if (write_message(body, strlen(body),
> > +				  rebase_path_fixup_msg(), 0))
> > +			return error(_("cannot write '%s'"),
> > +				     rebase_path_fixup_msg());
> 
> ...and then this return leaks the result (the other code path hits
> unuse_commit_buffer(), and is fine).

Good point.

I found another leaked commit buffer in make_patch() and fixed it, too.

> This leak was noticed by Coverity. It has a _ton_ of false positives
> across the whole project, but it sends out a mail with new ones every
> few days, which is usually short enough that I can process it in 30
> seconds or so.

Yeah, I get these mails now, thanks to Stephan adding me in response to
some issues I introduced with the builtin difftool (and hence I did not
get the warnings when I introduced the problems with sequencer-i).

Ciao,
Dscho

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Jeff King]

On Mon, Dec 19, 2016 at 05:59:06PM +0100, Johannes Schindelin wrote:

> > > > +		sprintf((char *)p, "%d", ++count);
> > > 
> > > Do we know the area pointed at p (which is inside buf) long enough
> > > not to overflow?  If the original were 9 and you incremented to get
> > > 10, you would need one extra byte.
> > 
> > Even if it is enough, I'd ask to please use xsnprintf(). In the off
> > chance that there's a programming error, we'd get a nice die("BUG")
> > instead of a buffer overflow (and it makes the code base easier to audit
> > for other overflows).
> 
> I ended up with more verbose, easier-to-read code that does not try to do
> things in-place, in favor of being slightly more wasteful with strbufs.

Great. I agree that should make the whole thing way more readable.

-Peff

Re: [PATCH v2 11/34] sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed

[Johannes Schindelin]

Hi Junio,

On Fri, 16 Dec 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > The scripted version of the interactive rebase already does that.
> 
> Sensible.  I was wondering why this wasn't there while reviewing
> 10/34, comparing the two (this is not a suggestion to squash this
> into the previous step).

It's a bit of a historical wart, as I discovered test breakages only long
after implementing the fixup/squash commands (as you may have guessed, I
implemented the commands one after another, fixing things as discovered by
the test suite; it took something like a month until I got the
rebase--helper based rebase -i to pass t3404).

In the end, I decided not to squash this into 10/34 because it seemed to
be a significant enough change to merit its own commit.

Ciao,
Dscho

Re: [PATCH v2 14/34] sequencer (rebase -i): update refs after a successful rebase

[Johannes Schindelin]

Hi Junio,

On Fri, 16 Dec 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > An interactive rebase operates on a detached HEAD (to keep the reflog
> > of the original branch relatively clean), and updates the branch only
> > at the end.
> >
> > Now that the sequencer learns to perform interactive rebases, it also
> > needs to learn the trick to update the branch before removing the
> > directory containing the state of the interactive rebase.
> >
> > We introduce a new head_ref variable in a wider scope than necessary at
> > the moment, to allow for a later patch that prints out "Successfully
> > rebased and updated <ref>".
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  sequencer.c | 32 +++++++++++++++++++++++++++++++-
> >  1 file changed, 31 insertions(+), 1 deletion(-)
> >
> > diff --git a/sequencer.c b/sequencer.c
> > index a6625e765d..a4e9b326ba 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -100,6 +100,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
> >  static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
> >  static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
> >  static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
> > +static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
> > +static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
> >  
> >  static inline int is_rebase_i(const struct replay_opts *opts)
> >  {
> > @@ -1793,12 +1795,39 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
> >  	}
> >  
> >  	if (is_rebase_i(opts)) {
> > -		struct strbuf buf = STRBUF_INIT;
> > +		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
> >  
> >  		/* Stopped in the middle, as planned? */
> >  		if (todo_list->current < todo_list->nr)
> >  			return 0;
> >  
> > +		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
> > +				starts_with(head_ref.buf, "refs/")) {
> > +			unsigned char head[20], orig[20];
> > +
> > +			if (get_sha1("HEAD", head))
> > +				return error(_("cannot read HEAD"));
> > +			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
> > +					get_sha1_hex(buf.buf, orig))
> > +				return error(_("could not read orig-head"));
> > +			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
> > +				head_ref.buf);
> > +			if (!read_oneliner(&buf, rebase_path_onto(), 0))
> > +				return error(_("could not read 'onto'"));
> > +			if (update_ref(buf.buf, head_ref.buf, head, orig,
> > +					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
> > +				return error(_("could not update %s"),
> > +					head_ref.buf);
> > +			strbuf_reset(&buf);
> > +			strbuf_addf(&buf,
> > +				"rebase -i (finish): returning to %s",
> > +				head_ref.buf);
> > +			if (create_symref("HEAD", head_ref.buf, buf.buf))
> > +				return error(_("could not update HEAD to %s"),
> > +					head_ref.buf);
> 
> All of the above return error() calls leak head_ref.buf; in addition
> some leak buf.buf, too.

Thanks, fixed.

Ciao,
Dscho

Re: [PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Tue, 13 Dec 2016, Junio C Hamano wrote:
>
>> Linus Torvalds <torvalds@linux-foundation.org> writes:
>> 
>> > On Tue, Dec 13, 2016 at 12:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> >> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> >>
>> >>> +/*
>> >>> + * Note that ordering matters in this enum. Not only must it match the mapping
>> >>> + * below, it is also divided into several sections that matter.  When adding
>> >>> + * new commands, make sure you add it in the right section.
>> >>> + */
>> >>
>> >> Good thinking.
>
> Not my thinking... This was in direct response to a suggestion by Dennis
> Kaarsemaker, I cannot take credit for the idea.

I now realize that I was unclear about what "thinking" I found good
in my comment.  I do not particularly like defining two parallel
things and having to maintain them in sync.  The "Good thinking"
praise goes to whoever thought that this burdensome fact deserves a
clear comment in front of these two things.

And ...

>> Makes me wish C were a better language, though ;-)
>> >
>> > Do this:
>> >
>> >   static const char *todo_command_strings[] = {
>> >       [TODO_PICK] = "pick",
>> >       [TODO_REVERT] = "revert",
>> >       [TODO_NOOP] = "noop:,
>> >   };
>> >
>> > which makes the array be order-independent.

... solves only one-half of the problem with the language I had.
The order of the entries in this array[] may become more flexible
in the source, but you still have to define enum separately.

I guess if we really want to, we need to resort to something "ugly
but workable" like what you did in fsck.c with FOREACH_MSG_ID(X).
That approach may be the least ugly way if we have to maintain two
or more parallel things in sync.

    ... and then realizes you wrote pretty much the same thing
    ... after writing all of the above ;-)

But it is way overkill for sequencer commands that are only handful.

Re: [PATCH v2 03/34] sequencer (rebase -i): implement the 'edit' command

[Junio C Hamano]

Re: [PATCH v2 03/34] sequencer (rebase -i): implement the 'edit' command

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> > +	strbuf_addf(&buf, "%s/patch", get_dir(opts));
>> > +	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
>> > +	init_revisions(&log_tree_opt, NULL);
>> > +	log_tree_opt.abbrev = 0;
>> > +	log_tree_opt.diff = 1;
>> > +	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
>> > +	log_tree_opt.disable_stdin = 1;
>> > +	log_tree_opt.no_commit_id = 1;
>> > +	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
>> > +	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
>> > +	if (!log_tree_opt.diffopt.file)
>> > +		res |= error_errno(_("could not open '%s'"), buf.buf);
>> > +	else {
>> > +		res |= log_tree_commit(&log_tree_opt, commit);
>> > +		fclose(log_tree_opt.diffopt.file);
>> > +	}
>> > +	strbuf_reset(&buf);
>> > +	strbuf_addf(&buf, "%s/message", get_dir(opts));
>> > +	if (!file_exists(buf.buf)) {
>> > +		find_commit_subject(commit_buffer, &subject);
>> > +		res |= write_message(subject, strlen(subject), buf.buf, 1);
>> > +		unuse_commit_buffer(commit, commit_buffer);
>> > +	}
>> > +	strbuf_release(&buf);
>> > +
>> > +	return res;
>> > +}
>> 
>> OK.  This seems to match what scripted make_patch does in a handful
>> of lines.  We probably should have given you a helper to reduce
>> boilerplate that sets up log_tree_opt so that this function does not
>> have to be this long, but that is a separate topic.
>> 
>> Does it matter output_format is set to FORMAT_PATCH here, though?
>> With --no-commit-id set, I suspect there is no log message or
>> authorship information given to the output.

Sorry, this was me being stupid.  

FORMAT_PATCH here does not have anythning to do with "git
format-patch" (and "git log --pretty=email").  The PATCH there is as
opposed to things like --stat and --raw.  We want patch text that
can be fed to "git apply" and it is absolutely the right thing to
use here.

Re: [PATCH v2 24/34] sequencer (rebase -i): respect strategy/strategy_opts settings

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> The sequencer already has an idea about using different merge
> strategies. We just piggy-back on top of that, using rebase -i's
> own settings, when running the sequencer in interactive rebase mode.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 26 +++++++++++++++++++++++++-
>  1 file changed, 25 insertions(+), 1 deletion(-)

A handful of steps before and including this one look quite faithful
port to C from the scripted one.  

Looking good.

Re: [PATCH v2 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> In the upcoming patch, we will support rebase -i's progress
> reporting. The progress skips comments but counts 'noop's.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 15 +++++++++------
>  1 file changed, 9 insertions(+), 6 deletions(-)

> diff --git a/sequencer.c b/sequencer.c
> index 1f314b2743..63f6f25ced 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -770,7 +770,9 @@ enum todo_command {
>  	TODO_EXEC,
>  	/* commands that do nothing but are counted for reporting progress */
>  	TODO_NOOP,
> -	TODO_DROP
> +	TODO_DROP,
> +	/* comments (not counted for reporting progress) */
> +	TODO_COMMENT
>  };
>  
>  static struct {

Makes sense.  I would have done this immediately after introducing
NOOP if I were doing this series, if only because by having the
unchanging last element early in enum {} definition, we can avoid
having to deal with the "last element cannot have comma", but that
is not a big issue.

> @@ -785,12 +787,13 @@ static struct {
>  	{ 's', "squash" },
>  	{ 'x', "exec" },
>  	{ 0,   "noop" },
> -	{ 'd', "drop" }
> +	{ 'd', "drop" },
> +	{ 0,   NULL }
>  };
>  
>  static const char *command_to_string(const enum todo_command command)
>  {
> -	if ((size_t)command < ARRAY_SIZE(todo_command_info))
> +	if (command < TODO_COMMENT)
>  		return todo_command_info[command].str;
>  	die("Unknown command: %d", command);
>  }

The same comment as "instead of comparing with TODO_NOOP, you would
want is_noop()" applies to three instances of comparing with
TODO_COMMENT we can see in this patch, I think.

"is_counted()" perhaps?

Re: [PATCH v2 32/34] sequencer (rebase -i): show the progress

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> The interactive rebase keeps the user informed about its progress.
> If the sequencer wants to do the grunt work of the interactive
> rebase, it also needs to show that progress.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 32 ++++++++++++++++++++++++++++++++
>  1 file changed, 32 insertions(+)

Sensible.  This further adds two comparisons with TODO_COMMENT and
makes need for is_counted() or something like that felt more.

    $ git grep TODO_COMMENT pu sequencer.c

shows that some places "x < TODO_COMMENT" is used while some other
places "y != TODO_COMMENT" is used, both for the same purpose, and
"z >= TODO_COMMENT" is also seen for the negation of the same.

I think all of them except for one can become is_counted() or
!is_counted() to convey what they want to check better while the one
used in the loop control:

	for (i = 0; i < TODO_COMMENT; i++)

should probably become:

	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)

as the parsing loop wants to check the input against all known
commands and there is no strong reason for that layer to know how
the insns are numbered.  is_xxx() implementation can take advantage
of the way the insns are numbered, but there is no point spreading
the knowledge to higher layer in the callchain.

Other than that, all 34 patches looked sensible steps explained as a
coherent story that unfolds bit by bit, which was mostly a pleasant
read.

Thanks.

Re: [PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

[Stephan Beyer]

Hi Dscho,

>> However, maintaining more than one directory (like "sequencer" for
>> sequencer state and "rebase-merge" for rebase todo and log) can cause
>> the necessity to be even more careful when hacking on sequencer... For
>> example, the cleanup code must delete both of them, not only one of them.
> 
> That is incorrect. It depends on the options which directory is used. And
> it is that directory (and not both) that should be cleaned up in the end.
> 
> Otherwise you run into a ton of pain e.g. when running a rebase -i with an
> `exec git cherry-pick ...` line: all of a sudden, that little innocuous
> line would simply destroy the state directory of the current rebase -i.
> 
> That's a rather big no-no.

Ahh, I see, there seems to be a misunderstanding on my side about how
you did it (I did not look into the other patches (obviously)).

Thanks for clarifying!

Best
  Stephan

Re: [PATCH v2 09/34] sequencer (rebase -i): write an author-script file

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Junio,
>
> On Thu, 15 Dec 2016, Junio C Hamano wrote:
>
>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> 
>> > +	strbuf_addstr(&buf, "GIT_AUTHOR_NAME='");
>> > +	while (*message && *message != '\n' && *message != '\r')
>> > +		if (skip_prefix(message, " <", &message))
>> > +			break;
>> > +		else if (*message != '\'')
>> > +			strbuf_addch(&buf, *(message++));
>> > +		else
>> > +			strbuf_addf(&buf, "'\\\\%c'", *(message++));
>> > +	strbuf_addstr(&buf, "'\nGIT_AUTHOR_EMAIL='");
>> > +	while (*message && *message != '\n' && *message != '\r')
>> > +		if (skip_prefix(message, "> ", &message))
>> > +			break;
>> > +		else if (*message != '\'')
>> > +			strbuf_addch(&buf, *(message++));
>> > +		else
>> > +			strbuf_addf(&buf, "'\\\\%c'", *(message++));
>> 
>> Aren't these reading from an in-core commit object?  
>> 
>> If so, it should use split_ident_line() for consistency with other
>> parts of the system to do this parsing.  We should also already have
>> a helper for simple shell-quoting in quote.c and you would want to
>> use that instead of open coding like this.
>
> We keep coming back to the same argument. You want this quoting/dequoting
> to be turned into a full-fledged parser. And I keep pointing out that the
> code here does not *need* to parse but only construct an environment
> block.

I am afraid you are mis-reading me.  I see a code that _READS_ some
data format, for which we already have a parser, and then write out
things based on what it read.  I do not want you to make anything
into a full-fledged parser---I just do not want to see an ad-hoc
reader and instead the code to USE existing parser.

Re: [PATCH v2 09/34] sequencer (rebase -i): write an author-script file

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

>> We keep coming back to the same argument. You want this quoting/dequoting
>> to be turned into a full-fledged parser. And I keep pointing out that the
>> code here does not *need* to parse but only construct an environment
>> block.
>
> I am afraid you are mis-reading me.  I see a code that _READS_ some
> data format, for which we already have a parser, and then write out
> things based on what it read.  I do not want you to make anything
> into a full-fledged parser---I just do not want to see an ad-hoc
> reader and instead the code to USE existing parser.

To extend this a bit.

I care rather deeply about not spreading the re-invention of parsers
and formatters throughout the code, because that would introduce
unnecesary maintenance burden.

Quoting a string so that it is acceptable inside a pair of single
quotes, occasionally stepping outside of the sq context and using
backquote to excape individual byte, for example, might feel simple
enough that anybody can just write inline instead of learning how to
call and actually calling sq_quote().  You open ', show each byte
unless it is a ', in which case you close ' and give \' and
immediately open '.  That was what sq_quote() did originally and for
a long time.  If however we allowed everybody to reinvent the code
to quote all over the place, with a lame "This is different and does
not *need* to call shared code" excuse, it would have required a lot
more effort to do a change like the one in 77d604c309 ("Enhanced
sq_quote()", 2005-10-10) that changed the quoting rules slightly to
make the output safer to accomodate different variants of shells.

The same thing can be said for split_ident() code.  The "author"
line has name, '<', email address, '>', timestring, '+' or '-', and
timezone.  Splitting them into NAME and TIME may be very simple, and
"does not *need* to parse", right?  Not really.  If you look at how
split_ident() evolved to accomodate the real world malformed lines
like duplicated closing '>' and realize that what we have there may
probably *NOT* the perfect one (i.e. there may be other malformed
input we may have to accomodate by tweaking the implementation
further), we really do not want a hand-rolled ad-hoc "splitter" that
"does not *need* to parse".

The same thing can be said for the code that reads the
author-script, too.

So please do not waste any more time arguing.  I think you spent
arguing more time than you would otherwise have had to spend if you
just used existing helper functions, both in this thread and the
older one about reading the author-script.

Re: [PATCH v2 28/34] run_command_opt(): optionally hide stderr when the command succeeds

[Johannes Schindelin]

Hi Hannes,

On Wed, 14 Dec 2016, Johannes Sixt wrote:

> Am 14.12.2016 um 14:06 schrieb Jeff King:
> > On Wed, Dec 14, 2016 at 07:53:23AM -0500, Jeff King wrote:
> >
> > > I don't have a strong opinion on the patches under discussion, but
> > > here are a few pointers on the run-command interface:
> > > [...]
> >
> > And here is a patch representing my suggestions, on top of yours. Not
> > tested beyond "make test".
> 
> Thank you, that looks way better.
> 
> If there is agreement that this approach is preferable, I think we can
> have patches on top of the series; they would be orthogonal and do not
> have to take hostage of it. (And it looks like I won't be able to follow
> up until later this week[end].)

Seeing as the original intention was to do away with the
RUN_HIDE_STDERR_ON_SUCCESS flag, and that the sequencer-i branch *must*
include that functionality somehow, it is unfortunately not really
possible to do this on top of the patch series.

I say "unfortunately" because I feel pretty uncomfortable with replacing
something that has been tried and tested by something that still awaits
the test of time.

So the only possible course of action I see is to go the really long
route: incorporate the patches to use pipe_command() instead of
introducing a new RUN_* flag (which means basically munch up your patch
and Peff's and move it somewhere into the middle of the sequencer-i patch
series, which is exactly what I already did locally), cook the patches
beyond recognition in `next`, i.e. cook it really long to give it a really
good testing before moving the patches to `master`.

Ciao,
Johannes

Re: [PATCH v2 20/34] sequencer (rebase -i): copy commit notes at end

[Johannes Schindelin]

Hi Junio,

On Fri, 16 Dec 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > @@ -1750,6 +1797,17 @@ static int is_final_fixup(struct todo_list *todo_list)
> >  	return 1;
> >  }
> >  
> > +static enum todo_command peek_command(struct todo_list *todo_list, int offset)
> > +{
> > +	int i;
> > +
> > +	for (i = todo_list->current + offset; i < todo_list->nr; i++)
> > +		if (todo_list->items[i].command != TODO_NOOP)
> > +			return todo_list->items[i].command;
> 
> Makes me wonder, after having commented on 07/34 regarding the fact
> that in the end you would end up having three variants of no-op
> (i.e. NOOP, DROP and COMMENT), what definition of a "command" this
> function uses to return its result, when asked to "peek".

Well, it uses the todo_command idea of a "command"... ;-)

The only thing we do with this for now is to look whether the next command
is a fixup/squash (so that the user gets to edit the commit message just
once, for example, and also to record rewritten commits properly).

> I suspect that this will be updated in a later patch to do "< TODO_NOOP"
> instead?

Actually, no. I introduced a new function is_noop() and that is used now.

Ciao,
Dscho

Re: [PATCH v2 05/34] sequencer (rebase -i): learn about the 'verbose' mode

[Johannes Schindelin]

Hi Junio,

On Tue, 13 Dec 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > @@ -1493,9 +1498,26 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
> >  	}
> >  
> >  	if (is_rebase_i(opts)) {
> > +		struct strbuf buf = STRBUF_INIT;
> > +
> >  		/* Stopped in the middle, as planned? */
> >  		if (todo_list->current < todo_list->nr)
> >  			return 0;
> > +
> > +		if (opts->verbose) {
> > +			const char *argv[] = {
> > +				"diff-tree", "--stat", NULL, NULL
> > +			};
> > +
> > +			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
> > +				return error(_("could not read '%s'"),
> > +					rebase_path_orig_head());
> > +			strbuf_addstr(&buf, "..HEAD");
> > +			argv[2] = buf.buf;
> > +			run_command_v_opt(argv, RUN_GIT_CMD);
> > +			strbuf_reset(&buf);
> > +		}
> > +		strbuf_release(&buf);
> >  	}
> 
> It's a bit curious that the previous step avoided running a separate
> process and instead did "diff-tree -p" all in C, but this one does not.

I guess my only defence is that I tried to be a little lazy.

Fixed.
Dscho

[PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

[Johannes Schindelin]

This marks the count down to '3': two more patch series after this
(really tiny ones) and we have a faster rebase -i.

The idea of this patch series is to teach the sequencer to understand
all of the commands in `git-rebase-todo` scripts, to execute them and to
behave pretty much very the same as `git rebase -i --continue` when
called with the newly-introduced REPLAY_INTERACTIVE_REBASE mode.

Most of these patches should be pretty much straight-forward. When not,
I tried to make a point of describing enough background in the commit
message. Please feel free to point out where my explanations fall short.

Note that even after this patch series is applied, rebase -i is still
unaffected. It will require the next patch series which introduces the
rebase--helper that essentially implements `git rebase -i --continue` by
calling the sequencer with the appropriate options.

The final patch series will move a couple of pre- and post-processing
steps into the rebase--helper/sequencer (such as expanding/shrinking the
SHA-1s, reordering the fixup!/squash! lines, etc). This might sound like
a mere add-on, but it is essential for the speed improvements: those
stupid little processing steps really dominated the execution time in my
tests.

Apart from mostly cosmetic patches (and the occasional odd bug that I
fixed promptly), I used these patches since mid May to perform all of my
interactive rebases. In mid June, I had the idea to teach rebase -i to
run *both* scripted rebase and rebase--helper and to cross-validate the
results. This slowed down all my interactive rebases since, but helped
me catch three rather obscure bugs (e.g. that git commit --fixup unfolds
long onelines and rebase -i still finds the correct original commit).

This is all only to say that I am rather confident that the current code
does the job.

Since sending out v1, I integrated all of these patch series
into Git for Windows v2.10.0, where they have been live ever since, and
used by myself (also in a Linux VM, as Git for Windows' master branch
always builds also on Linux and passes the test suite, too).

Just to reiterate why I do all this: it speeds up the interactive rebase
substantially. Even with a not yet fully builtin rebase -i, but just the
part after the user edited the `git-rebase-todo` script.

The performance test I introduced to demonstrate this (p3404) shows a
speed-up of +380% here (i.e. roughly 5x), from ~8.8 seconds to ~1.8
seconds. This is on Windows, where the performance impact of avoiding
shell scripting is most noticable.

On MacOSX and on Linux, the speed-up is less pronounced, but still
noticable, at least if you trust Travis CI, which I abused to perform
that test for me. Check for yourself (searching for "3404.2") here:
https://travis-ci.org/git/git/builds/156295227. According to those logs,
p3404 is speeded up from ~0.45 seconds to ~0.12 seconds on Linux (read:
about 3.5x) and from ~1.7 seconds to ~0.5 seconds on MacOSX (read:
almost 4x).

Changes since v2:

- fixed a TRANSLATORS: comment

- added a comment to clarify why save_todo() does not write the current
  command to git-rebase-todo

- fixed the comment for "stopped-sha" that said that a long commit name
  is stored in that file, when it is in fact an abbreviated one

- consistently put the "else" keyword on the same line as the preceding
  closing brace, if any (this adds two patches to the already large
  patch series, sorry)

- completely revamped the update_squash_messages() function to make it
  more obvious why it does not overflow its buffer, and to fix it when
  using it with localised messages

- now the sequencer code uses find_commit_subject() consistently to find
  the commit message (it does not use the subject's length returned by
  the find_commit_subject() function, of course); this adds yet another
  patch to this already large patch series, sorry.

- introduced an is_noop() function (as opposed to testing <= TODO_NOOP
  manually) to make the code less magic.

- fixed two potential leaks of get_commit_buffer()'s returned buffer

- fixed leaks in the error code paths when trying to update the ref at
  the end

- renamed read_author_script() into read_env_script() to reflect that
  it would accept any environment setting, and made it much more
  readable

- completely reworked the strategy how to suppress the output of
  successful cherry-picks: instead of introducing
  RUN_HIDE_STDERR_ON_SUCCESS, the code now uses pipe_command() to catch
  the output itself, so that it can be shown in case of error

- replaced a spawned `diff-tree` command by code using the diff functions
  directly


Johannes Schindelin (38):
  sequencer: avoid unnecessary curly braces
  sequencer: move "else" keyword onto the same line as preceding brace
  sequencer: use a helper to find the commit message
  sequencer: support a new action: 'interactive rebase'
  sequencer (rebase -i): implement the 'noop' command
  sequencer (rebase -i): implement the 'edit' command
  sequencer (rebase -i): implement the 'exec' command
  sequencer (rebase -i): learn about the 'verbose' mode
  sequencer (rebase -i): write the 'done' file
  sequencer (rebase -i): add support for the 'fixup' and 'squash'
    commands
  sequencer (rebase -i): implement the short commands
  sequencer (rebase -i): write an author-script file
  sequencer (rebase -i): allow continuing with staged changes
  sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed
  sequencer (rebase -i): skip some revert/cherry-pick specific code path
  sequencer (rebase -i): the todo can be empty when continuing
  sequencer (rebase -i): update refs after a successful rebase
  sequencer (rebase -i): leave a patch upon error
  sequencer (rebase -i): implement the 'reword' command
  sequencer (rebase -i): allow fast-forwarding for edit/reword
  sequencer (rebase -i): refactor setting the reflog message
  sequencer (rebase -i): set the reflog message consistently
  sequencer (rebase -i): copy commit notes at end
  sequencer (rebase -i): record interrupted commits in rewritten, too
  sequencer (rebase -i): run the post-rewrite hook, if needed
  sequencer (rebase -i): respect the rebase.autostash setting
  sequencer (rebase -i): respect strategy/strategy_opts settings
  sequencer (rebase -i): allow rescheduling commands
  sequencer (rebase -i): implement the 'drop' command
  sequencer (rebase -i): differentiate between comments and 'noop'
  sequencer: make reading author-script more elegant
  sequencer: use run_command() directly
  sequencer (rebase -i): show only failed `git commit`'s output
  sequencer (rebase -i): show only failed cherry-picks' output
  sequencer (rebase -i): suggest --edit-todo upon unknown command
  sequencer (rebase -i): show the progress
  sequencer (rebase -i): write the progress into files
  sequencer (rebase -i): write out the final message

 sequencer.c | 1105 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 sequencer.h |    4 +-
 2 files changed, 1013 insertions(+), 96 deletions(-)


base-commit: e05806da9ec4aff8adfed142ab2a2b3b02e33c8c
Published-As: https://github.com/dscho/git/releases/tag/sequencer-i-v3
Fetch-It-Via: git fetch https://github.com/dscho/git sequencer-i-v3

Interdiff vs v2:

 diff --git a/run-command.c b/run-command.c
 index 5bb957afdd..ca905a9e80 100644
 --- a/run-command.c
 +++ b/run-command.c
 @@ -589,29 +589,6 @@ int run_command_v_opt_cd_env(const char **argv, int opt, const char *dir, const
  	cmd.clean_on_exit = opt & RUN_CLEAN_ON_EXIT ? 1 : 0;
  	cmd.dir = dir;
  	cmd.env = env;
 -
 -	if (opt & RUN_HIDE_STDERR_ON_SUCCESS) {
 -		struct strbuf buf = STRBUF_INIT;
 -		int res;
 -
 -		cmd.err = -1;
 -		if (start_command(&cmd) < 0)
 -			return -1;
 -
 -		if (strbuf_read(&buf, cmd.err, 0) < 0) {
 -			close(cmd.err);
 -			finish_command(&cmd); /* throw away exit code */
 -			return -1;
 -		}
 -
 -		close(cmd.err);
 -		res = finish_command(&cmd);
 -		if (res)
 -			fputs(buf.buf, stderr);
 -		strbuf_release(&buf);
 -		return res;
 -	}
 -
  	return run_command(&cmd);
  }
  
 diff --git a/run-command.h b/run-command.h
 index 65a21ddd4e..dd1c78c28d 100644
 --- a/run-command.h
 +++ b/run-command.h
 @@ -72,7 +72,6 @@ extern int run_hook_ve(const char *const *env, const char *name, va_list args);
  #define RUN_SILENT_EXEC_FAILURE 8
  #define RUN_USING_SHELL 16
  #define RUN_CLEAN_ON_EXIT 32
 -#define RUN_HIDE_STDERR_ON_SUCCESS 64
  int run_command_v_opt(const char **argv, int opt);
  
  /*
 diff --git a/sequencer.c b/sequencer.c
 index c0aeb4f5f7..0e7d2ca5c8 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -102,7 +102,7 @@ static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
  static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
  /*
   * When we stop at a given patch via the "edit" command, this file contains
 - * the long commit name of the corresponding patch.
 + * the abbreviated commit name of the corresponding patch.
   */
  static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
  /*
 @@ -464,8 +464,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
  
  	if (active_cache_changed &&
  	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
 -		/*
 -		 * TRANSLATORS: %s will be "revert", "cherry-pick" or
 +		/* TRANSLATORS: %s will be "revert", "cherry-pick" or
  		 * "rebase -i".
  		 */
  		return error(_("%s: Unable to write new index file"),
 @@ -524,8 +523,7 @@ static int write_author_script(const char *message)
  			/* Missing 'author' line? */
  			unlink(rebase_path_author_script());
  			return 0;
 -		}
 -		else if (skip_prefix(message, "author ", &message))
 +		} else if (skip_prefix(message, "author ", &message))
  			break;
  		else if ((eol = strchr(message, '\n')))
  			message = eol + 1;
 @@ -560,18 +558,17 @@ static int write_author_script(const char *message)
  }
  
  /*
 - * Read the author-script file into an environment block, ready for use in
 - * run_command(), that can be free()d afterwards.
 + * Read a list of environment variable assignments (such as the author-script
 + * file) into an environment block. Returns -1 on error, 0 otherwise.
   */
 -static char **read_author_script(void)
 +static int read_env_script(struct argv_array *env)
  {
  	struct strbuf script = STRBUF_INIT;
  	int i, count = 0;
 -	char *p, *p2, **env;
 -	size_t env_size;
 +	char *p, *p2;
  
  	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
 -		return NULL;
 +		return -1;
  
  	for (p = script.buf; *p; p++)
  		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
 @@ -583,19 +580,12 @@ static char **read_author_script(void)
  			count++;
  		}
  
 -	env_size = (count + 1) * sizeof(*env);
 -	strbuf_grow(&script, env_size);
 -	memmove(script.buf + env_size, script.buf, script.len);
 -	p = script.buf + env_size;
 -	env = (char **)strbuf_detach(&script, NULL);
 -
  	for (i = 0; i < count; i++) {
 -		env[i] = p;
 +		argv_array_push(env, p);
  		p += strlen(p) + 1;
  	}
 -	env[count] = NULL;
  
 -	return env;
 +	return 0;
  }
  
  static const char staged_changes_advice[] =
 @@ -628,19 +618,18 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
  			  int allow_empty, int edit, int amend,
  			  int cleanup_commit_message)
  {
 -	char **env = NULL;
 -	struct argv_array array;
 -	int opt = RUN_GIT_CMD, rc;
 +	struct child_process cmd = CHILD_PROCESS_INIT;
  	const char *value;
  
 +	cmd.git_cmd = 1;
 +
  	if (is_rebase_i(opts)) {
  		if (!edit) {
 -			opt |= RUN_COMMAND_STDOUT_TO_STDERR;
 -			opt |= RUN_HIDE_STDERR_ON_SUCCESS;
 +			cmd.stdout_to_stderr = 1;
 +			cmd.err = -1;
  		}
  
 -		env = read_author_script();
 -		if (!env) {
 +		if (read_env_script(&cmd.env_array)) {
  			const char *gpg_opt = gpg_sign_opt_quoted(opts);
  
  			return error(_(staged_changes_advice),
 @@ -648,39 +637,47 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
  		}
  	}
  
 -	argv_array_init(&array);
 -	argv_array_push(&array, "commit");
 -	argv_array_push(&array, "-n");
 +	argv_array_push(&cmd.args, "commit");
 +	argv_array_push(&cmd.args, "-n");
  
  	if (amend)
 -		argv_array_push(&array, "--amend");
 +		argv_array_push(&cmd.args, "--amend");
  	if (opts->gpg_sign)
 -		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
 +		argv_array_pushf(&cmd.args, "-S%s", opts->gpg_sign);
  	if (opts->signoff)
 -		argv_array_push(&array, "-s");
 +		argv_array_push(&cmd.args, "-s");
  	if (defmsg)
 -		argv_array_pushl(&array, "-F", defmsg, NULL);
 +		argv_array_pushl(&cmd.args, "-F", defmsg, NULL);
  	if (cleanup_commit_message)
 -		argv_array_push(&array, "--cleanup=strip");
 +		argv_array_push(&cmd.args, "--cleanup=strip");
  	if (edit)
 -		argv_array_push(&array, "-e");
 +		argv_array_push(&cmd.args, "-e");
  	else if (!cleanup_commit_message &&
  		 !opts->signoff && !opts->record_origin &&
  		 git_config_get_value("commit.cleanup", &value))
 -		argv_array_push(&array, "--cleanup=verbatim");
 +		argv_array_push(&cmd.args, "--cleanup=verbatim");
  
  	if (allow_empty)
 -		argv_array_push(&array, "--allow-empty");
 +		argv_array_push(&cmd.args, "--allow-empty");
  
  	if (opts->allow_empty_message)
 -		argv_array_push(&array, "--allow-empty-message");
 +		argv_array_push(&cmd.args, "--allow-empty-message");
  
 -	rc = run_command_v_opt_cd_env(array.argv, opt, NULL,
 -			(const char *const *)env);
 -	argv_array_clear(&array);
 -	free(env);
 +	if (cmd.err == -1) {
 +		/* hide stderr on success */
 +		struct strbuf buf = STRBUF_INIT;
 +		int rc = pipe_command(&cmd,
 +				      NULL, 0,
 +				      /* stdout is already redirected */
 +				      NULL, 0,
 +				      &buf, 0);
 +		if (rc)
 +			fputs(buf.buf, stderr);
 +		strbuf_release(&buf);
 +		return rc;
 +	}
  
 -	return rc;
 +	return run_command(&cmd);
  }
  
  static int is_original_commit_empty(struct commit *commit)
 @@ -786,6 +783,11 @@ static const char *command_to_string(const enum todo_command command)
  	die("Unknown command: %d", command);
  }
  
 +static int is_noop(const enum todo_command command)
 +{
 +	return TODO_NOOP <= command;
 +}
 +
  static int is_fixup(enum todo_command command)
  {
  	return command == TODO_FIXUP || command == TODO_SQUASH;
 @@ -799,36 +801,33 @@ static int update_squash_messages(enum todo_command command,
  	const char *message, *body;
  
  	if (file_exists(rebase_path_squash_msg())) {
 -		char *p, *p2;
 +		struct strbuf header = STRBUF_INIT;
 +		char *eol, *p;
  
  		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
  			return error(_("could not read '%s'"),
  				rebase_path_squash_msg());
  
 +		p = buf.buf + 1;
 +		eol = strchrnul(buf.buf, '\n');
  		if (buf.buf[0] != comment_line_char ||
 -		    !skip_prefix(buf.buf + 1, " This is a combination of ",
 -				 (const char **)&p))
 +		    (p += strcspn(p, "0123456789\n")) == eol)
  			return error(_("unexpected 1st line of squash message:"
  				       "\n\n\t%.*s"),
 -				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
 -				     buf.buf);
 -		count = strtol(p, &p2, 10);
 +				     (int)(eol - buf.buf), buf.buf);
 +		count = strtol(p, NULL, 10);
  
 -		if (count < 1 || *p2 != ' ')
 +		if (count < 1)
  			return error(_("invalid 1st line of squash message:\n"
  				       "\n\t%.*s"),
 -				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
 -				     buf.buf);
 -
 -		sprintf((char *)p, "%d", ++count);
 -		if (!*p2)
 -			*p2 = ' ';
 -		else {
 -			*(++p2) = 'c';
 -			strbuf_insert(&buf, p2 - buf.buf, " ", 1);
 -		}
 -	}
 -	else {
 +				     (int)(eol - buf.buf), buf.buf);
 +
 +		strbuf_addf(&header, "%c ", comment_line_char);
 +		strbuf_addf(&header,
 +			    _("This is a combination of %d commits."), ++count);
 +		strbuf_splice(&buf, 0, eol - buf.buf, header.buf, header.len);
 +		strbuf_release(&header);
 +	} else {
  		unsigned char head[20];
  		struct commit *head_commit;
  		const char *head_message, *body;
 @@ -840,20 +839,22 @@ static int update_squash_messages(enum todo_command command,
  		if (!(head_message = get_commit_buffer(head_commit, NULL)))
  			return error(_("could not read HEAD's commit message"));
  
 -		body = strstr(head_message, "\n\n");
 -		if (!body)
 -			body = "";
 -		else
 -			body = skip_blank_lines(body + 2);
 +		find_commit_subject(head_message, &body);
  		if (write_message(body, strlen(body),
 -				  rebase_path_fixup_msg(), 0))
 +				  rebase_path_fixup_msg(), 0)) {
 +			unuse_commit_buffer(head_commit, head_message);
  			return error(_("cannot write '%s'"),
  				     rebase_path_fixup_msg());
 +		}
  
  		count = 2;
 -		strbuf_addf(&buf, _("%c This is a combination of 2 commits.\n"
 -				    "%c The first commit's message is:\n\n%s"),
 -			    comment_line_char, comment_line_char, body);
 +		strbuf_addf(&buf, "%c ", comment_line_char);
 +		strbuf_addf(&buf, _("This is a combination of %d commits."),
 +			    count);
 +		strbuf_addf(&buf, "\n%c ", comment_line_char);
 +		strbuf_addstr(&buf, _("This is the 1st commit message:"));
 +		strbuf_addstr(&buf, "\n\n");
 +		strbuf_addstr(&buf, body);
  
  		unuse_commit_buffer(head_commit, head_message);
  	}
 @@ -861,25 +862,21 @@ static int update_squash_messages(enum todo_command command,
  	if (!(message = get_commit_buffer(commit, NULL)))
  		return error(_("could not read commit message of %s"),
  			     oid_to_hex(&commit->object.oid));
 -	body = strstr(message, "\n\n");
 -	if (!body)
 -		body = "";
 -	else
 -		body = skip_blank_lines(body + 2);
 +	find_commit_subject(message, &body);
  
  	if (command == TODO_SQUASH) {
  		unlink(rebase_path_fixup_msg());
 -		strbuf_addf(&buf, _("\n%c This is the commit message #%d:\n"
 -				    "\n%s"),
 -			    comment_line_char, count, body);
 -	}
 -	else if (command == TODO_FIXUP) {
 -		strbuf_addf(&buf, _("\n%c The commit message #%d "
 -				    "will be skipped:\n\n"),
 -			    comment_line_char, count);
 +		strbuf_addf(&buf, "\n%c ", comment_line_char);
 +		strbuf_addf(&buf, _("This is the commit message #%d:"), count);
 +		strbuf_addstr(&buf, "\n\n");
 +		strbuf_addstr(&buf, body);
 +	} else if (command == TODO_FIXUP) {
 +		strbuf_addf(&buf, "\n%c ", comment_line_char);
 +		strbuf_addf(&buf, _("The commit message #%d will be skipped:"),
 +			    count);
 +		strbuf_addstr(&buf, "\n\n");
  		strbuf_add_commented_lines(&buf, body, strlen(body));
 -	}
 -	else
 +	} else
  		return error(_("unknown command: %d"), command);
  	unuse_commit_buffer(commit, message);
  
 @@ -956,9 +953,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  	}
  	discard_cache();
  
 -	if (!commit->parents) {
 +	if (!commit->parents)
  		parent = NULL;
 -	}
  	else if (commit->parents->next) {
  		/* Reverting or cherry-picking a merge commit */
  		int cnt;
 @@ -1036,14 +1032,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		next = commit;
  		next_label = msg.label;
  
 -		/*
 -		 * Append the commit log message to msgbuf; it starts
 -		 * after the tree, parent, author, committer
 -		 * information followed by "\n\n".
 -		 */
 -		p = strstr(msg.message, "\n\n");
 -		if (p)
 -			strbuf_addstr(&msgbuf, skip_blank_lines(p + 2));
 +		/* Append the commit log message to msgbuf. */
 +		if (find_commit_subject(msg.message, &p))
 +			strbuf_addstr(&msgbuf, p);
  
  		if (opts->record_origin) {
  			if (!has_conforming_footer(&msgbuf, NULL, 0))
 @@ -1065,8 +1056,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		else if (file_exists(rebase_path_fixup_msg())) {
  			cleanup_commit_message = 1;
  			msg_file = rebase_path_fixup_msg();
 -		}
 -		else {
 +		} else {
  			const char *dest = git_path("SQUASH_MSG");
  			unlink(dest);
  			if (copy_file(dest, rebase_path_squash_msg(), 0666))
 @@ -1241,8 +1231,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
  		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
  			item->command = i;
  			break;
 -		}
 -		else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
 +		} else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
  			bol++;
  			item->command = i;
  			break;
 @@ -1312,7 +1301,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
  		else if (is_fixup(item->command))
  			return error(_("cannot '%s' without a previous commit"),
  				command_to_string(item->command));
 -		else if (item->command < TODO_NOOP)
 +		else if (!is_noop(item->command))
  			fixup_okay = 1;
  	}
  
 @@ -1533,8 +1522,7 @@ static int create_seq_dir(void)
  		error(_("a cherry-pick or revert is already in progress"));
  		advise(_("try \"git cherry-pick (--continue | --quit | --abort)\""));
  		return -1;
 -	}
 -	else if (mkdir(git_path_seq_dir(), 0777) < 0)
 +	} else if (mkdir(git_path_seq_dir(), 0777) < 0)
  		return error_errno(_("could not create sequencer directory '%s'"),
  				   git_path_seq_dir());
  	return 0;
 @@ -1667,6 +1655,10 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
  	const char *todo_path = get_todo_path(opts);
  	int next = todo_list->current, offset, fd;
  
 +	/*
 +	 * rebase -i writes "git-rebase-todo" without the currently executing
 +	 * command, appending it to "done" instead.
 +	 */
  	if (is_rebase_i(opts))
  		next++;
  
 @@ -1739,7 +1731,7 @@ static int make_patch(struct commit *commit, struct replay_opts *opts)
  {
  	struct strbuf buf = STRBUF_INIT;
  	struct rev_info log_tree_opt;
 -	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject, *p;
 +	const char *subject, *p;
  	int res = 0;
  
  	p = short_commit_name(commit);
 @@ -1766,6 +1758,7 @@ static int make_patch(struct commit *commit, struct replay_opts *opts)
  
  	strbuf_addf(&buf, "%s/message", get_dir(opts));
  	if (!file_exists(buf.buf)) {
 +		const char *commit_buffer = get_commit_buffer(commit, NULL);
  		find_commit_subject(commit_buffer, &subject);
  		res |= write_message(subject, strlen(subject), buf.buf, 1);
  		unuse_commit_buffer(commit, commit_buffer);
 @@ -1805,8 +1798,7 @@ static int error_with_patch(struct commit *commit,
  			"Once you are satisfied with your changes, run\n"
  			"\n"
  			"  git rebase --continue\n", gpg_sign_opt_quoted(opts));
 -	}
 -	else if (exit_code)
 +	} else if (exit_code)
  		fprintf(stderr, "Could not apply %s... %.*s\n",
  			short_commit_name(commit), subject_len, subject);
  
 @@ -1854,8 +1846,7 @@ static int do_exec(const char *command_line)
  		if (status == 127)
  			/* command not found */
  			status = 1;
 -	}
 -	else if (dirty) {
 +	} else if (dirty) {
  		warning(_("execution succeeded: %s\nbut "
  			  "left changes to the index and/or the working tree\n"
  			  "Commit or stash your changes, and then run\n"
 @@ -1878,7 +1869,7 @@ static int is_final_fixup(struct todo_list *todo_list)
  	while (++i < todo_list->nr)
  		if (is_fixup(todo_list->items[i].command))
  			return 0;
 -		else if (todo_list->items[i].command < TODO_NOOP)
 +		else if (!is_noop(todo_list->items[i].command))
  			break;
  	return 1;
  }
 @@ -1888,7 +1879,7 @@ static enum todo_command peek_command(struct todo_list *todo_list, int offset)
  	int i;
  
  	for (i = todo_list->current + offset; i < todo_list->nr; i++)
 -		if (todo_list->items[i].command < TODO_NOOP)
 +		if (!is_noop(todo_list->items[i].command))
  			return todo_list->items[i].command;
  
  	return -1;
 @@ -2021,21 +2012,18 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  					intend_to_amend();
  				return error_failed_squash(item->commit, opts,
  					item->arg_len, item->arg);
 -			}
 -			else if (res && is_rebase_i(opts))
 +			} else if (res && is_rebase_i(opts))
  				return res | error_with_patch(item->commit,
  					item->arg, item->arg_len, opts, res,
  					item->command == TODO_REWORD);
 -		}
 -		else if (item->command == TODO_EXEC) {
 +		} else if (item->command == TODO_EXEC) {
  			char *end_of_arg = (char *)(item->arg + item->arg_len);
  			int saved = *end_of_arg;
  
  			*end_of_arg = '\0';
  			res = do_exec(item->arg);
  			*end_of_arg = saved;
 -		}
 -		else if (item->command < TODO_NOOP)
 +		} else if (!is_noop(item->command))
  			return error(_("unknown command %d"), item->command);
  
  		todo_list->current++;
 @@ -2055,40 +2043,60 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  				starts_with(head_ref.buf, "refs/")) {
  			const char *msg;
  			unsigned char head[20], orig[20];
 -
 -			if (get_sha1("HEAD", head))
 -				return error(_("cannot read HEAD"));
 +			int res;
 +
 +			if (get_sha1("HEAD", head)) {
 +				res = error(_("cannot read HEAD"));
 +cleanup_head_ref:
 +				strbuf_release(&head_ref);
 +				strbuf_release(&buf);
 +				return res;
 +			}
  			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
 -					get_sha1_hex(buf.buf, orig))
 -				return error(_("could not read orig-head"));
 -			if (!read_oneliner(&buf, rebase_path_onto(), 0))
 -				return error(_("could not read 'onto'"));
 +					get_sha1_hex(buf.buf, orig)) {
 +				res = error(_("could not read orig-head"));
 +				goto cleanup_head_ref;
 +			}
 +			if (!read_oneliner(&buf, rebase_path_onto(), 0)) {
 +				res = error(_("could not read 'onto'"));
 +				goto cleanup_head_ref;
 +			}
  			msg = reflog_message(opts, "finish", "%s onto %s",
  				head_ref.buf, buf.buf);
  			if (update_ref(msg, head_ref.buf, head, orig,
 -					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 -				return error(_("could not update %s"),
 +					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR)) {
 +				res = error(_("could not update %s"),
  					head_ref.buf);
 +				goto cleanup_head_ref;
 +			}
  			msg = reflog_message(opts, "finish", "returning to %s",
  				head_ref.buf);
 -			if (create_symref("HEAD", head_ref.buf, msg))
 -				return error(_("could not update HEAD to %s"),
 +			if (create_symref("HEAD", head_ref.buf, msg)) {
 +				res = error(_("could not update HEAD to %s"),
  					head_ref.buf);
 +				goto cleanup_head_ref;
 +			}
  			strbuf_reset(&buf);
  		}
  
  		if (opts->verbose) {
 -			const char *argv[] = {
 -				"diff-tree", "--stat", NULL, NULL
 -			};
 -
 -			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
 -				return error(_("could not read '%s'"),
 -					rebase_path_orig_head());
 -			strbuf_addstr(&buf, "..HEAD");
 -			argv[2] = buf.buf;
 -			run_command_v_opt(argv, RUN_GIT_CMD);
 -			strbuf_reset(&buf);
 +			struct rev_info log_tree_opt;
 +			struct object_id orig, head;
 +
 +			memset(&log_tree_opt, 0, sizeof(log_tree_opt));
 +			init_revisions(&log_tree_opt, NULL);
 +			log_tree_opt.diff = 1;
 +			log_tree_opt.diffopt.output_format =
 +				DIFF_FORMAT_DIFFSTAT;
 +			log_tree_opt.disable_stdin = 1;
 +
 +			if (read_oneliner(&buf, rebase_path_orig_head(), 0) &&
 +			    !get_sha1(buf.buf, orig.hash) &&
 +			    !get_sha1("HEAD", head.hash)) {
 +				diff_tree_sha1(orig.hash, head.hash,
 +					       "", &log_tree_opt.diffopt);
 +				log_tree_diff_flush(&log_tree_opt);
 +			}
  		}
  		flush_rewritten_pending();
  		if (!stat(rebase_path_rewritten_list(), &st) &&
 @@ -2195,8 +2203,7 @@ int sequencer_continue(struct replay_opts *opts)
  	if (is_rebase_i(opts)) {
  		if (commit_staged_changes(opts))
  			return -1;
 -	}
 -	else if (!file_exists(get_todo_path(opts)))
 +	} else if (!file_exists(get_todo_path(opts)))
  		return continue_single_pick();
  	if (read_populate_opts(opts))
  		return -1;
 @@ -2216,8 +2223,7 @@ int sequencer_continue(struct replay_opts *opts)
  			goto release_todo_list;
  		}
  		todo_list.current++;
 -	}
 -	else if (file_exists(rebase_path_stopped_sha())) {
 +	} else if (file_exists(rebase_path_stopped_sha())) {
  		struct strbuf buf = STRBUF_INIT;
  		struct object_id oid;
  

-- 
2.11.0.rc3.windows.1

[PATCH v3 01/38] sequencer: avoid unnecessary curly braces

[Johannes Schindelin]

This was noticed while addressing Junio Hamano's concern that some
"else" operators were on separate lines than the preceding closing
brace.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 9adb7bbf1d..23793db08b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -632,9 +632,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	discard_cache();
 
-	if (!commit->parents) {
+	if (!commit->parents)
 		parent = NULL;
-	}
 	else if (commit->parents->next) {
 		/* Reverting or cherry-picking a merge commit */
 		int cnt;
-- 
2.11.0.rc3.windows.1

[PATCH v3 02/38] sequencer: move "else" keyword onto the same line as preceding brace

[Johannes Schindelin]

It is the current coding style of the Git project to write

	if (...) {
		...
	} else {
		...
	}

instead of putting the closing brace and the "else" keyword on separate
lines.

Pointed out by Junio Hamano.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 23793db08b..3eededcb98 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1070,8 +1070,7 @@ static int create_seq_dir(void)
 		error(_("a cherry-pick or revert is already in progress"));
 		advise(_("try \"git cherry-pick (--continue | --quit | --abort)\""));
 		return -1;
-	}
-	else if (mkdir(git_path_seq_dir(), 0777) < 0)
+	} else if (mkdir(git_path_seq_dir(), 0777) < 0)
 		return error_errno(_("could not create sequencer directory '%s'"),
 				   git_path_seq_dir());
 	return 0;
-- 
2.11.0.rc3.windows.1

[PATCH v3 03/38] sequencer: use a helper to find the commit message

[Johannes Schindelin]

It is actually not safe to look for a commit message by looking for the
first empty line and skipping it.

The find_commit_subject() function looks more carefully, so let's use
it. Since we are interested in the entire commit message, we re-compute
the string length after verifying that the commit subject is not empty
(in which case the entire commit message would be empty, something that
should not happen but that we want to handle gracefully).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 3eededcb98..720857beda 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -703,14 +703,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		next = commit;
 		next_label = msg.label;
 
-		/*
-		 * Append the commit log message to msgbuf; it starts
-		 * after the tree, parent, author, committer
-		 * information followed by "\n\n".
-		 */
-		p = strstr(msg.message, "\n\n");
-		if (p)
-			strbuf_addstr(&msgbuf, skip_blank_lines(p + 2));
+		/* Append the commit log message to msgbuf. */
+		if (find_commit_subject(msg.message, &p))
+			strbuf_addstr(&msgbuf, p);
 
 		if (opts->record_origin) {
 			if (!has_conforming_footer(&msgbuf, NULL, 0))
-- 
2.11.0.rc3.windows.1

[PATCH v3 04/38] sequencer: support a new action: 'interactive rebase'

[Johannes Schindelin]

This patch introduces a new action for the sequencer. It really does not
do a whole lot of its own right now, but lays the ground work for
patches to come. The intention, of course, is to finally make the
sequencer the work horse of the interactive rebase (the original idea
behind the "sequencer" concept).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 36 ++++++++++++++++++++++++++++++++----
 sequencer.h |  3 ++-
 2 files changed, 34 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 720857beda..690460bc67 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -30,6 +30,14 @@ static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 static GIT_PATH_FUNC(git_path_abort_safety_file, "sequencer/abort-safety")
 
+static GIT_PATH_FUNC(rebase_path, "rebase-merge")
+/*
+ * The file containing rebase commands, comments, and empty lines.
+ * This file is created by "git rebase -i" then edited by the user. As
+ * the lines are processed, they are removed from the front of this
+ * file and written to the tail of 'done'.
+ */
+static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
 /*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
@@ -42,19 +50,22 @@ static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
  */
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 
-/* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
-	return 0;
+	return opts->action == REPLAY_INTERACTIVE_REBASE;
 }
 
 static const char *get_dir(const struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return rebase_path();
 	return git_path_seq_dir();
 }
 
 static const char *get_todo_path(const struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return rebase_path_todo();
 	return git_path_todo_file();
 }
 
@@ -122,7 +133,15 @@ int sequencer_remove_state(struct replay_opts *opts)
 
 static const char *action_name(const struct replay_opts *opts)
 {
-	return opts->action == REPLAY_REVERT ? N_("revert") : N_("cherry-pick");
+	switch (opts->action) {
+	case REPLAY_REVERT:
+		return N_("revert");
+	case REPLAY_PICK:
+		return N_("cherry-pick");
+	case REPLAY_INTERACTIVE_REBASE:
+		return N_("rebase -i");
+	}
+	die(_("Unknown action: %d"), opts->action);
 }
 
 struct commit_message {
@@ -364,7 +383,9 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
-		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
+		/* TRANSLATORS: %s will be "revert", "cherry-pick" or
+		 * "rebase -i".
+		 */
 		return error(_("%s: Unable to write new index file"),
 			_(action_name(opts)));
 	rollback_lock_file(&index_lock);
@@ -1198,6 +1219,13 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 	const char *todo_path = get_todo_path(opts);
 	int next = todo_list->current, offset, fd;
 
+	/*
+	 * rebase -i writes "git-rebase-todo" without the currently executing
+	 * command, appending it to "done" instead.
+	 */
+	if (is_rebase_i(opts))
+		next++;
+
 	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
 		return error_errno(_("could not lock '%s'"), todo_path);
diff --git a/sequencer.h b/sequencer.h
index 7a513c576b..cb21cfddee 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -7,7 +7,8 @@ const char *git_path_seq_dir(void);
 
 enum replay_action {
 	REPLAY_REVERT,
-	REPLAY_PICK
+	REPLAY_PICK,
+	REPLAY_INTERACTIVE_REBASE
 };
 
 struct replay_opts {
-- 
2.11.0.rc3.windows.1

[PATCH v3 05/38] sequencer (rebase -i): implement the 'noop' command

[Johannes Schindelin]

The 'noop' command is probably the most boring of all rebase -i commands
to support in the sequencer.

Which makes it an excellent candidate for this first stab to add support
for rebase -i's commands to the sequencer.

For the moment, let's also treat empty lines and commented-out lines as
'noop'; We will refine that handling later in this patch series.

To make it easier to identify "classes" of todo_commands (such as:
determine whether a command is pick-like, i.e. handles a single commit),
let's enforce a certain order of said commands.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 39 ++++++++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 690460bc67..84f18e64e9 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -607,14 +607,23 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 		return 1;
 }
 
+/*
+ * Note that ordering matters in this enum. Not only must it match the mapping
+ * below, it is also divided into several sections that matter.  When adding
+ * new commands, make sure you add it in the right section.
+ */
 enum todo_command {
+	/* commands that handle commits */
 	TODO_PICK = 0,
-	TODO_REVERT
+	TODO_REVERT,
+	/* commands that do nothing but are counted for reporting progress */
+	TODO_NOOP
 };
 
 static const char *todo_command_strings[] = {
 	"pick",
-	"revert"
+	"revert",
+	"noop"
 };
 
 static const char *command_to_string(const enum todo_command command)
@@ -624,6 +633,10 @@ static const char *command_to_string(const enum todo_command command)
 	die("Unknown command: %d", command);
 }
 
+static int is_noop(const enum todo_command command)
+{
+	return TODO_NOOP <= (size_t)command;
+}
 
 static int do_pick_commit(enum todo_command command, struct commit *commit,
 		struct replay_opts *opts)
@@ -879,6 +892,14 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	/* left-trim */
 	bol += strspn(bol, " \t");
 
+	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
+		item->command = TODO_NOOP;
+		item->commit = NULL;
+		item->arg = bol;
+		item->arg_len = eol - bol;
+		return 0;
+	}
+
 	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
 		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
 			item->command = i;
@@ -887,6 +908,13 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	if (i >= ARRAY_SIZE(todo_command_strings))
 		return -1;
 
+	if (item->command == TODO_NOOP) {
+		item->commit = NULL;
+		item->arg = bol;
+		item->arg_len = eol - bol;
+		return 0;
+	}
+
 	/* Eat up extra spaces/ tabs before object name */
 	padding = strspn(bol, " \t");
 	if (!padding)
@@ -1289,7 +1317,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		res = do_pick_commit(item->command, item->commit, opts);
+		if (item->command <= TODO_REVERT)
+			res = do_pick_commit(item->command, item->commit,
+					opts);
+		else if (!is_noop(item->command))
+			return error(_("unknown command %d"), item->command);
+
 		todo_list->current++;
 		if (res)
 			return res;
-- 
2.11.0.rc3.windows.1

[PATCH v3 06/38] sequencer (rebase -i): implement the 'edit' command

[Johannes Schindelin]

This patch is a straight-forward reimplementation of the `edit`
operation of the interactive rebase command.

Well, not *quite* straight-forward: when stopping, the `edit`
command wants to write the `patch` file (which is not only the
patch, but includes the commit message and author information). To
that end, this patch requires the earlier work that taught the
log-tree machinery to respect the `file` setting of
rev_info->diffopt to write to a file stream different than stdout.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 117 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 114 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 84f18e64e9..b138a3906c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -17,6 +17,7 @@
 #include "argv-array.h"
 #include "quote.h"
 #include "trailer.h"
+#include "log-tree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -45,6 +46,20 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
 /*
+ * When an "edit" rebase command is being processed, the SHA1 of the
+ * commit to be edited is recorded in this file.  When "git rebase
+ * --continue" is executed, if there are any staged changes then they
+ * will be amended to the HEAD commit, but only provided the HEAD
+ * commit is still the commit to be edited.  When any other rebase
+ * command is processed, this file is deleted.
+ */
+static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
+/*
+ * When we stop at a given patch via the "edit" command, this file contains
+ * the abbreviated commit name of the corresponding patch.
+ */
+static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
+/*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
  */
@@ -616,6 +631,7 @@ enum todo_command {
 	/* commands that handle commits */
 	TODO_PICK = 0,
 	TODO_REVERT,
+	TODO_EDIT,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP
 };
@@ -623,6 +639,7 @@ enum todo_command {
 static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
+	"edit",
 	"noop"
 };
 
@@ -1302,9 +1319,87 @@ static int save_opts(struct replay_opts *opts)
 	return res;
 }
 
+static int make_patch(struct commit *commit, struct replay_opts *opts)
+{
+	struct strbuf buf = STRBUF_INIT;
+	struct rev_info log_tree_opt;
+	const char *subject, *p;
+	int res = 0;
+
+	p = short_commit_name(commit);
+	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
+		return -1;
+
+	strbuf_addf(&buf, "%s/patch", get_dir(opts));
+	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
+	init_revisions(&log_tree_opt, NULL);
+	log_tree_opt.abbrev = 0;
+	log_tree_opt.diff = 1;
+	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
+	log_tree_opt.disable_stdin = 1;
+	log_tree_opt.no_commit_id = 1;
+	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
+	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
+	if (!log_tree_opt.diffopt.file)
+		res |= error_errno(_("could not open '%s'"), buf.buf);
+	else {
+		res |= log_tree_commit(&log_tree_opt, commit);
+		fclose(log_tree_opt.diffopt.file);
+	}
+	strbuf_reset(&buf);
+
+	strbuf_addf(&buf, "%s/message", get_dir(opts));
+	if (!file_exists(buf.buf)) {
+		const char *commit_buffer = get_commit_buffer(commit, NULL);
+		find_commit_subject(commit_buffer, &subject);
+		res |= write_message(subject, strlen(subject), buf.buf, 1);
+		unuse_commit_buffer(commit, commit_buffer);
+	}
+	strbuf_release(&buf);
+
+	return res;
+}
+
+static int intend_to_amend(void)
+{
+	unsigned char head[20];
+	char *p;
+
+	if (get_sha1("HEAD", head))
+		return error(_("cannot read HEAD"));
+
+	p = sha1_to_hex(head);
+	return write_message(p, strlen(p), rebase_path_amend(), 1);
+}
+
+static int error_with_patch(struct commit *commit,
+	const char *subject, int subject_len,
+	struct replay_opts *opts, int exit_code, int to_amend)
+{
+	if (make_patch(commit, opts))
+		return -1;
+
+	if (to_amend) {
+		if (intend_to_amend())
+			return -1;
+
+		fprintf(stderr, "You can amend the commit now, with\n"
+			"\n"
+			"  git commit --amend %s\n"
+			"\n"
+			"Once you are satisfied with your changes, run\n"
+			"\n"
+			"  git rebase --continue\n", gpg_sign_opt_quoted(opts));
+	} else if (exit_code)
+		fprintf(stderr, "Could not apply %s... %.*s\n",
+			short_commit_name(commit), subject_len, subject);
+
+	return exit_code;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
-	int res;
+	int res = 0;
 
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	if (opts->allow_ff)
@@ -1317,10 +1412,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		if (item->command <= TODO_REVERT)
+		if (item->command <= TODO_EDIT) {
 			res = do_pick_commit(item->command, item->commit,
 					opts);
-		else if (!is_noop(item->command))
+			if (item->command == TODO_EDIT) {
+				struct commit *commit = item->commit;
+				if (!res)
+					warning(_("stopped at %s... %.*s"),
+						short_commit_name(commit),
+						item->arg_len, item->arg);
+				return error_with_patch(commit,
+					item->arg, item->arg_len, opts, res,
+					!res);
+			}
+		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
@@ -1328,6 +1433,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			return res;
 	}
 
+	if (is_rebase_i(opts)) {
+		/* Stopped in the middle, as planned? */
+		if (todo_list->current < todo_list->nr)
+			return 0;
+	}
+
 	/*
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
-- 
2.11.0.rc3.windows.1

[PATCH v3 07/38] sequencer (rebase -i): implement the 'exec' command

[Johannes Schindelin]

The 'exec' command is a little special among rebase -i's commands, as it
does *not* have a SHA-1 as first parameter. Instead, everything after the
`exec` command is treated as command-line to execute.

Let's reuse the arg/arg_len fields of the todo_item structure (which hold
the oneline for pick/edit commands) to point to the command-line.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index b138a3906c..e9c10d7fe5 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -18,6 +18,7 @@
 #include "quote.h"
 #include "trailer.h"
 #include "log-tree.h"
+#include "wt-status.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -632,6 +633,8 @@ enum todo_command {
 	TODO_PICK = 0,
 	TODO_REVERT,
 	TODO_EDIT,
+	/* commands that do something else than handling a single commit */
+	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP
 };
@@ -640,6 +643,7 @@ static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
 	"edit",
+	"exec",
 	"noop"
 };
 
@@ -938,6 +942,12 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return -1;
 	bol += padding;
 
+	if (item->command == TODO_EXEC) {
+		item->arg = bol;
+		item->arg_len = (int)(eol - bol);
+		return 0;
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
@@ -1397,6 +1407,46 @@ static int error_with_patch(struct commit *commit,
 	return exit_code;
 }
 
+static int do_exec(const char *command_line)
+{
+	const char *child_argv[] = { NULL, NULL };
+	int dirty, status;
+
+	fprintf(stderr, "Executing: %s\n", command_line);
+	child_argv[0] = command_line;
+	status = run_command_v_opt(child_argv, RUN_USING_SHELL);
+
+	/* force re-reading of the cache */
+	if (discard_cache() < 0 || read_cache() < 0)
+		return error(_("could not read index"));
+
+	dirty = require_clean_work_tree("rebase", NULL, 1, 1);
+
+	if (status) {
+		warning(_("execution failed: %s\n%s"
+			  "You can fix the problem, and then run\n"
+			  "\n"
+			  "  git rebase --continue\n"
+			  "\n"),
+			command_line,
+			dirty ? N_("and made changes to the index and/or the "
+				"working tree\n") : "");
+		if (status == 127)
+			/* command not found */
+			status = 1;
+	} else if (dirty) {
+		warning(_("execution succeeded: %s\nbut "
+			  "left changes to the index and/or the working tree\n"
+			  "Commit or stash your changes, and then run\n"
+			  "\n"
+			  "  git rebase --continue\n"
+			  "\n"), command_line);
+		status = 1;
+	}
+
+	return status;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1425,6 +1475,13 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+		} else if (item->command == TODO_EXEC) {
+			char *end_of_arg = (char *)(item->arg + item->arg_len);
+			int saved = *end_of_arg;
+
+			*end_of_arg = '\0';
+			res = do_exec(item->arg);
+			*end_of_arg = saved;
 		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
-- 
2.11.0.rc3.windows.1

[PATCH v3 08/38] sequencer (rebase -i): learn about the 'verbose' mode

[Johannes Schindelin]

When calling `git rebase -i -v`, the user wants to see some statistics
after the commits were rebased. Let's show some.

The strbuf we use to perform that task will be used for other things
in subsequent commits, hence it is declared and initialized in a wider
scope than strictly needed here.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 28 ++++++++++++++++++++++++++++
 sequencer.h |  1 +
 2 files changed, 29 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index e9c10d7fe5..ddc4d144d7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -65,6 +65,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
  * command-line (and are only consumed, not modified, by the sequencer).
  */
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
+static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
+static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1088,6 +1090,9 @@ static int read_populate_opts(struct replay_opts *opts)
 		}
 		strbuf_release(&buf);
 
+		if (file_exists(rebase_path_verbose()))
+			opts->verbose = 1;
+
 		return 0;
 	}
 
@@ -1491,9 +1496,32 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	}
 
 	if (is_rebase_i(opts)) {
+		struct strbuf buf = STRBUF_INIT;
+
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
 			return 0;
+
+		if (opts->verbose) {
+			struct rev_info log_tree_opt;
+			struct object_id orig, head;
+
+			memset(&log_tree_opt, 0, sizeof(log_tree_opt));
+			init_revisions(&log_tree_opt, NULL);
+			log_tree_opt.diff = 1;
+			log_tree_opt.diffopt.output_format =
+				DIFF_FORMAT_DIFFSTAT;
+			log_tree_opt.disable_stdin = 1;
+
+			if (read_oneliner(&buf, rebase_path_orig_head(), 0) &&
+			    !get_sha1(buf.buf, orig.hash) &&
+			    !get_sha1("HEAD", head.hash)) {
+				diff_tree_sha1(orig.hash, head.hash,
+					       "", &log_tree_opt.diffopt);
+				log_tree_diff_flush(&log_tree_opt);
+			}
+		}
+		strbuf_release(&buf);
 	}
 
 	/*
diff --git a/sequencer.h b/sequencer.h
index cb21cfddee..f885b68395 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -24,6 +24,7 @@ struct replay_opts {
 	int allow_empty;
 	int allow_empty_message;
 	int keep_redundant_commits;
+	int verbose;
 
 	int mainline;
 
-- 
2.11.0.rc3.windows.1

[PATCH v3 09/38] sequencer (rebase -i): write the 'done' file

[Johannes Schindelin]

In the interactive rebase, commands that were successfully processed are
not simply discarded, but appended to the 'done' file instead. This is
used e.g. to display the current state to the user in the output of
`git status` or the progress.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index ddc4d144d7..8ea3d6aa94 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -41,6 +41,12 @@ static GIT_PATH_FUNC(rebase_path, "rebase-merge")
  */
 static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
 /*
+ * The rebase command lines that have already been processed. A line
+ * is moved here when it is first handled, before any associated user
+ * actions.
+ */
+static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
+/*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
  * being rebased.
@@ -1296,6 +1302,23 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 		return error_errno(_("could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
 		return error(_("failed to finalize '%s'."), todo_path);
+
+	if (is_rebase_i(opts)) {
+		const char *done_path = rebase_path_done();
+		int fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
+		int prev_offset = !next ? 0 :
+			todo_list->items[next - 1].offset_in_buf;
+
+		if (fd >= 0 && offset > prev_offset &&
+		    write_in_full(fd, todo_list->buf.buf + prev_offset,
+				  offset - prev_offset) < 0) {
+			close(fd);
+			return error_errno(_("could not write to '%s'"),
+					   done_path);
+		}
+		if (fd >= 0)
+			close(fd);
+	}
 	return 0;
 }
 
-- 
2.11.0.rc3.windows.1

[PATCH v3 10/38] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

[Johannes Schindelin]

This is a huge patch, and at the same time a huge step forward to
execute the performance-critical parts of the interactive rebase in a
builtin command.

Since 'fixup' and 'squash' are not only similar, but also need to know
about each other (we want to reduce a series of fixups/squashes into a
single, final commit message edit, from the user's point of view), we
really have to implement them both at the same time.

Most of the actual work is done by the existing code path that already
handles the "pick" and the "edit" commands; We added support for other
features (e.g. to amend the commit message) in the patches leading up to
this one, yet there are still quite a few bits in this patch that simply
would not make sense as individual patches (such as: determining whether
there was anything to "fix up" in the "todo" script, etc).

In theory, it would be possible to reuse the fast-forward code path also
for the fixup and the squash code paths, but in practice this would make
the code less readable. The end result cannot be fast-forwarded anyway,
therefore let's just extend the cherry-picking code path for now.

Since the sequencer parses the entire `git-rebase-todo` script in one go,
fixup or squash commands without a preceding pick can be reported early
(in git-rebase--interactive, we could only report such errors just before
executing the fixup/squash).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 227 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 217 insertions(+), 10 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 8ea3d6aa94..6a939a10bd 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -47,6 +47,35 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
 /*
+ * The commit message that is planned to be used for any changes that
+ * need to be committed following a user interaction.
+ */
+static GIT_PATH_FUNC(rebase_path_message, "rebase-merge/message")
+/*
+ * The file into which is accumulated the suggested commit message for
+ * squash/fixup commands. When the first of a series of squash/fixups
+ * is seen, the file is created and the commit message from the
+ * previous commit and from the first squash/fixup commit are written
+ * to it. The commit message for each subsequent squash/fixup commit
+ * is appended to the file as it is processed.
+ *
+ * The first line of the file is of the form
+ *     # This is a combination of $count commits.
+ * where $count is the number of commits whose messages have been
+ * written to the file so far (including the initial "pick" commit).
+ * Each time that a commit message is processed, this line is read and
+ * updated. It is deleted just before the combined commit is made.
+ */
+static GIT_PATH_FUNC(rebase_path_squash_msg, "rebase-merge/message-squash")
+/*
+ * If the current series of squash/fixups has not yet included a squash
+ * command, then this file exists and holds the commit message of the
+ * original "pick" commit.  (If the series ends without a "squash"
+ * command, then this can be used as the commit message of the combined
+ * commit without opening the editor.)
+ */
+static GIT_PATH_FUNC(rebase_path_fixup_msg, "rebase-merge/message-fixup")
+/*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
  * being rebased.
@@ -641,6 +670,8 @@ enum todo_command {
 	TODO_PICK = 0,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_FIXUP,
+	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
@@ -651,6 +682,8 @@ static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
 	"edit",
+	"fixup",
+	"squash",
 	"exec",
 	"noop"
 };
@@ -667,15 +700,114 @@ static int is_noop(const enum todo_command command)
 	return TODO_NOOP <= (size_t)command;
 }
 
+static int is_fixup(enum todo_command command)
+{
+	return command == TODO_FIXUP || command == TODO_SQUASH;
+}
+
+static int update_squash_messages(enum todo_command command,
+		struct commit *commit, struct replay_opts *opts)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int count, res;
+	const char *message, *body;
+
+	if (file_exists(rebase_path_squash_msg())) {
+		struct strbuf header = STRBUF_INIT;
+		char *eol, *p;
+
+		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
+			return error(_("could not read '%s'"),
+				rebase_path_squash_msg());
+
+		p = buf.buf + 1;
+		eol = strchrnul(buf.buf, '\n');
+		if (buf.buf[0] != comment_line_char ||
+		    (p += strcspn(p, "0123456789\n")) == eol)
+			return error(_("unexpected 1st line of squash message:"
+				       "\n\n\t%.*s"),
+				     (int)(eol - buf.buf), buf.buf);
+		count = strtol(p, NULL, 10);
+
+		if (count < 1)
+			return error(_("invalid 1st line of squash message:\n"
+				       "\n\t%.*s"),
+				     (int)(eol - buf.buf), buf.buf);
+
+		strbuf_addf(&header, "%c ", comment_line_char);
+		strbuf_addf(&header,
+			    _("This is a combination of %d commits."), ++count);
+		strbuf_splice(&buf, 0, eol - buf.buf, header.buf, header.len);
+		strbuf_release(&header);
+	} else {
+		unsigned char head[20];
+		struct commit *head_commit;
+		const char *head_message, *body;
+
+		if (get_sha1("HEAD", head))
+			return error(_("need a HEAD to fixup"));
+		if (!(head_commit = lookup_commit_reference(head)))
+			return error(_("could not read HEAD"));
+		if (!(head_message = get_commit_buffer(head_commit, NULL)))
+			return error(_("could not read HEAD's commit message"));
+
+		find_commit_subject(head_message, &body);
+		if (write_message(body, strlen(body),
+				  rebase_path_fixup_msg(), 0)) {
+			unuse_commit_buffer(head_commit, head_message);
+			return error(_("cannot write '%s'"),
+				     rebase_path_fixup_msg());
+		}
+
+		count = 2;
+		strbuf_addf(&buf, "%c ", comment_line_char);
+		strbuf_addf(&buf, _("This is a combination of %d commits."),
+			    count);
+		strbuf_addf(&buf, "\n%c ", comment_line_char);
+		strbuf_addstr(&buf, _("This is the 1st commit message:"));
+		strbuf_addstr(&buf, "\n\n");
+		strbuf_addstr(&buf, body);
+
+		unuse_commit_buffer(head_commit, head_message);
+	}
+
+	if (!(message = get_commit_buffer(commit, NULL)))
+		return error(_("could not read commit message of %s"),
+			     oid_to_hex(&commit->object.oid));
+	find_commit_subject(message, &body);
+
+	if (command == TODO_SQUASH) {
+		unlink(rebase_path_fixup_msg());
+		strbuf_addf(&buf, "\n%c ", comment_line_char);
+		strbuf_addf(&buf, _("This is the commit message #%d:"), count);
+		strbuf_addstr(&buf, "\n\n");
+		strbuf_addstr(&buf, body);
+	} else if (command == TODO_FIXUP) {
+		strbuf_addf(&buf, "\n%c ", comment_line_char);
+		strbuf_addf(&buf, _("The commit message #%d will be skipped:"),
+			    count);
+		strbuf_addstr(&buf, "\n\n");
+		strbuf_add_commented_lines(&buf, body, strlen(body));
+	} else
+		return error(_("unknown command: %d"), command);
+	unuse_commit_buffer(commit, message);
+
+	res = write_message(buf.buf, buf.len, rebase_path_squash_msg(), 0);
+	strbuf_release(&buf);
+	return res;
+}
+
 static int do_pick_commit(enum todo_command command, struct commit *commit,
-		struct replay_opts *opts)
+		struct replay_opts *opts, int final_fixup)
 {
+	int edit = opts->edit, cleanup_commit_message = 0;
+	const char *msg_file = edit ? NULL : git_path_merge_msg();
 	unsigned char head[20];
 	struct commit *base, *next, *parent;
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res, unborn = 0, allow;
+	int res, unborn = 0, amend = 0, allow;
 
 	if (opts->no_commit) {
 		/*
@@ -720,7 +852,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	else
 		parent = commit->parents->item;
 
-	if (opts->allow_ff &&
+	if (opts->allow_ff && !is_fixup(command) &&
 	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
 	     (!parent && unborn)))
 		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
@@ -779,6 +911,27 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
+	if (is_fixup(command)) {
+		if (update_squash_messages(command, commit, opts))
+			return -1;
+		amend = 1;
+		if (!final_fixup)
+			msg_file = rebase_path_squash_msg();
+		else if (file_exists(rebase_path_fixup_msg())) {
+			cleanup_commit_message = 1;
+			msg_file = rebase_path_fixup_msg();
+		} else {
+			const char *dest = git_path("SQUASH_MSG");
+			unlink(dest);
+			if (copy_file(dest, rebase_path_squash_msg(), 0666))
+				return error(_("could not rename '%s' to '%s'"),
+					     rebase_path_squash_msg(), dest);
+			unlink(git_path("MERGE_MSG"));
+			msg_file = dest;
+			edit = 1;
+		}
+	}
+
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
@@ -834,8 +987,13 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
-		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
-				     opts, allow, opts->edit, 0, 0);
+		res = run_git_commit(msg_file, opts, allow, edit, amend,
+				     cleanup_commit_message);
+
+	if (!res && final_fixup) {
+		unlink(rebase_path_fixup_msg());
+		unlink(rebase_path_squash_msg());
+	}
 
 leave:
 	free_message(commit, &msg);
@@ -976,7 +1134,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
 	struct todo_item *item;
 	char *p = buf, *next_p;
-	int i, res = 0;
+	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
 
 	for (i = 1; *p; i++, p = next_p) {
 		char *eol = strchrnul(p, '\n');
@@ -991,8 +1149,16 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		if (parse_insn_line(item, p, eol)) {
 			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
-			item->command = -1;
+			item->command = TODO_NOOP;
 		}
+
+		if (fixup_okay)
+			; /* do nothing */
+		else if (is_fixup(item->command))
+			return error(_("cannot '%s' without a previous commit"),
+				command_to_string(item->command));
+		else if (!is_noop(item->command))
+			fixup_okay = 1;
 	}
 	if (!todo_list->nr)
 		return error(_("no commits parsed."));
@@ -1435,6 +1601,20 @@ static int error_with_patch(struct commit *commit,
 	return exit_code;
 }
 
+static int error_failed_squash(struct commit *commit,
+	struct replay_opts *opts, int subject_len, const char *subject)
+{
+	if (rename(rebase_path_squash_msg(), rebase_path_message()))
+		return error(_("could not rename '%s' to '%s'"),
+			rebase_path_squash_msg(), rebase_path_message());
+	unlink(rebase_path_fixup_msg());
+	unlink(git_path("MERGE_MSG"));
+	if (copy_file(git_path("MERGE_MSG"), rebase_path_message(), 0666))
+		return error(_("could not copy '%s' to '%s'"),
+			     rebase_path_message(), git_path("MERGE_MSG"));
+	return error_with_patch(commit, subject, subject_len, opts, 1, 0);
+}
+
 static int do_exec(const char *command_line)
 {
 	const char *child_argv[] = { NULL, NULL };
@@ -1475,6 +1655,21 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int is_final_fixup(struct todo_list *todo_list)
+{
+	int i = todo_list->current;
+
+	if (!is_fixup(todo_list->items[i].command))
+		return 0;
+
+	while (++i < todo_list->nr)
+		if (is_fixup(todo_list->items[i].command))
+			return 0;
+		else if (!is_noop(todo_list->items[i].command))
+			break;
+	return 1;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1490,9 +1685,15 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		if (item->command <= TODO_EDIT) {
+		if (is_rebase_i(opts)) {
+			unlink(rebase_path_message());
+			unlink(rebase_path_author_script());
+			unlink(rebase_path_stopped_sha());
+			unlink(rebase_path_amend());
+		}
+		if (item->command <= TODO_SQUASH) {
 			res = do_pick_commit(item->command, item->commit,
-					opts);
+					opts, is_final_fixup(todo_list));
 			if (item->command == TODO_EDIT) {
 				struct commit *commit = item->commit;
 				if (!res)
@@ -1503,6 +1704,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+			if (res && is_fixup(item->command)) {
+				if (res == 1)
+					intend_to_amend();
+				return error_failed_squash(item->commit, opts,
+					item->arg_len, item->arg);
+			}
 		} else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
 			int saved = *end_of_arg;
@@ -1601,7 +1808,7 @@ static int single_pick(struct commit *cmit, struct replay_opts *opts)
 {
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	return do_pick_commit(opts->action == REPLAY_PICK ?
-		TODO_PICK : TODO_REVERT, cmit, opts);
+		TODO_PICK : TODO_REVERT, cmit, opts, 0);
 }
 
 int sequencer_pick_revisions(struct replay_opts *opts)
-- 
2.11.0.rc3.windows.1

[PATCH v3 11/38] sequencer (rebase -i): implement the short commands

[Johannes Schindelin]

For users' convenience, most rebase commands can be abbreviated, e.g.
'p' instead of 'pick' and 'x' instead of 'exec'. Let's teach the
sequencer to handle those abbreviated commands just fine.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 35 +++++++++++++++++++++--------------
 1 file changed, 21 insertions(+), 14 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 6a939a10bd..29b944d724 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -678,20 +678,23 @@ enum todo_command {
 	TODO_NOOP
 };
 
-static const char *todo_command_strings[] = {
-	"pick",
-	"revert",
-	"edit",
-	"fixup",
-	"squash",
-	"exec",
-	"noop"
+static struct {
+	char c;
+	const char *str;
+} todo_command_info[] = {
+	{ 'p', "pick" },
+	{ 0,   "revert" },
+	{ 'e', "edit" },
+	{ 'f', "fixup" },
+	{ 's', "squash" },
+	{ 'x', "exec" },
+	{ 0,   "noop" }
 };
 
 static const char *command_to_string(const enum todo_command command)
 {
-	if ((size_t)command < ARRAY_SIZE(todo_command_strings))
-		return todo_command_strings[command];
+	if ((size_t)command < ARRAY_SIZE(todo_command_info))
+		return todo_command_info[command].str;
 	die("Unknown command: %d", command);
 }
 
@@ -1087,12 +1090,16 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return 0;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
-		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
+	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)
+		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
+			item->command = i;
+			break;
+		} else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
+			bol++;
 			item->command = i;
 			break;
 		}
-	if (i >= ARRAY_SIZE(todo_command_strings))
+	if (i >= ARRAY_SIZE(todo_command_info))
 		return -1;
 
 	if (item->command == TODO_NOOP) {
@@ -1287,7 +1294,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 {
 	enum todo_command command = opts->action == REPLAY_PICK ?
 		TODO_PICK : TODO_REVERT;
-	const char *command_string = todo_command_strings[command];
+	const char *command_string = todo_command_info[command].str;
 	struct commit *commit;
 
 	if (prepare_revs(opts))
-- 
2.11.0.rc3.windows.1

[PATCH v3 12/38] sequencer (rebase -i): write an author-script file

[Johannes Schindelin]

When the interactive rebase aborts, it writes out an author-script file
to record the author information for the current commit. As we are about
to teach the sequencer how to perform the actions behind an interactive
rebase, it needs to write those author-script files, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 50 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 49 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 29b944d724..9913882603 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -483,6 +483,52 @@ static int is_index_unchanged(void)
 	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
 }
 
+static int write_author_script(const char *message)
+{
+	struct strbuf buf = STRBUF_INIT;
+	const char *eol;
+	int res;
+
+	for (;;)
+		if (!*message || starts_with(message, "\n")) {
+missing_author:
+			/* Missing 'author' line? */
+			unlink(rebase_path_author_script());
+			return 0;
+		} else if (skip_prefix(message, "author ", &message))
+			break;
+		else if ((eol = strchr(message, '\n')))
+			message = eol + 1;
+		else
+			goto missing_author;
+
+	strbuf_addstr(&buf, "GIT_AUTHOR_NAME='");
+	while (*message && *message != '\n' && *message != '\r')
+		if (skip_prefix(message, " <", &message))
+			break;
+		else if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\nGIT_AUTHOR_EMAIL='");
+	while (*message && *message != '\n' && *message != '\r')
+		if (skip_prefix(message, "> ", &message))
+			break;
+		else if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\nGIT_AUTHOR_DATE='@");
+	while (*message && *message != '\n' && *message != '\r')
+		if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	res = write_message(buf.buf, buf.len, rebase_path_author_script(), 1);
+	strbuf_release(&buf);
+	return res;
+}
+
 /*
  * Read the author-script file into an environment block, ready for use in
  * run_command(), that can be free()d afterwards.
@@ -935,7 +981,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
-	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
+	if (is_rebase_i(opts) && write_author_script(msg.message) < 0)
+		res = -1;
+	else if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
 		if (res < 0)
-- 
2.11.0.rc3.windows.1

[PATCH v3 13/38] sequencer (rebase -i): allow continuing with staged changes

[Johannes Schindelin]

When an interactive rebase is interrupted, the user may stage changes
before continuing, and we need to commit those changes in that case.

Please note that the nested "if" added to the sequencer_continue() is
not combined into a single "if" because it will be extended with an
"else" clause in a later patch in this patch series.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 9913882603..69301fecc6 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1826,6 +1826,42 @@ static int continue_single_pick(void)
 	return run_command_v_opt(argv, RUN_GIT_CMD);
 }
 
+static int commit_staged_changes(struct replay_opts *opts)
+{
+	int amend = 0;
+
+	if (has_unstaged_changes(1))
+		return error(_("cannot rebase: You have unstaged changes."));
+	if (!has_uncommitted_changes(0))
+		return 0;
+
+	if (file_exists(rebase_path_amend())) {
+		struct strbuf rev = STRBUF_INIT;
+		unsigned char head[20], to_amend[20];
+
+		if (get_sha1("HEAD", head))
+			return error(_("cannot amend non-existing commit"));
+		if (!read_oneliner(&rev, rebase_path_amend(), 0))
+			return error(_("invalid file: '%s'"), rebase_path_amend());
+		if (get_sha1_hex(rev.buf, to_amend))
+			return error(_("invalid contents: '%s'"),
+				rebase_path_amend());
+		if (hashcmp(head, to_amend))
+			return error(_("\nYou have uncommitted changes in your "
+				       "working tree. Please, commit them\n"
+				       "first and then run 'git rebase "
+				       "--continue' again."));
+
+		strbuf_release(&rev);
+		amend = 1;
+	}
+
+	if (run_git_commit(rebase_path_message(), opts, 1, 1, amend, 0))
+		return error(_("could not commit staged changes."));
+	unlink(rebase_path_amend());
+	return 0;
+}
+
 int sequencer_continue(struct replay_opts *opts)
 {
 	struct todo_list todo_list = TODO_LIST_INIT;
@@ -1834,6 +1870,10 @@ int sequencer_continue(struct replay_opts *opts)
 	if (read_and_refresh_cache(opts))
 		return -1;
 
+	if (is_rebase_i(opts)) {
+		if (commit_staged_changes(opts))
+			return -1;
+	}
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts))
-- 
2.11.0.rc3.windows.1

[PATCH v3 14/38] sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed

[Johannes Schindelin]

The scripted version of the interactive rebase already does that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 69301fecc6..52e17c8887 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1832,8 +1832,13 @@ static int commit_staged_changes(struct replay_opts *opts)
 
 	if (has_unstaged_changes(1))
 		return error(_("cannot rebase: You have unstaged changes."));
-	if (!has_uncommitted_changes(0))
+	if (!has_uncommitted_changes(0)) {
+		const char *cherry_pick_head = git_path("CHERRY_PICK_HEAD");
+
+		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
+			return error(_("could not remove CHERRY_PICK_HEAD"));
 		return 0;
+	}
 
 	if (file_exists(rebase_path_amend())) {
 		struct strbuf rev = STRBUF_INIT;
-- 
2.11.0.rc3.windows.1

[PATCH v3 15/38] sequencer (rebase -i): skip some revert/cherry-pick specific code path

[Johannes Schindelin]

When a cherry-pick continues without a "todo script", the intention is
simply to pick a single commit.

However, when an interactive rebase is continued without a "todo
script", it means that the last command has been completed and that we
now need to clean up.

This commit guards the revert/cherry-pick specific steps so that they
are not executed in rebase -i mode.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 52e17c8887..a7b9ee0d04 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1878,26 +1878,28 @@ int sequencer_continue(struct replay_opts *opts)
 	if (is_rebase_i(opts)) {
 		if (commit_staged_changes(opts))
 			return -1;
-	}
-	if (!file_exists(get_todo_path(opts)))
+	} else if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts))
 		return -1;
 	if ((res = read_populate_todo(&todo_list, opts)))
 		goto release_todo_list;
 
-	/* Verify that the conflict has been resolved */
-	if (file_exists(git_path_cherry_pick_head()) ||
-	    file_exists(git_path_revert_head())) {
-		res = continue_single_pick();
-		if (res)
+	if (!is_rebase_i(opts)) {
+		/* Verify that the conflict has been resolved */
+		if (file_exists(git_path_cherry_pick_head()) ||
+		    file_exists(git_path_revert_head())) {
+			res = continue_single_pick();
+			if (res)
+				goto release_todo_list;
+		}
+		if (index_differs_from("HEAD", 0, 0)) {
+			res = error_dirty_index(opts);
 			goto release_todo_list;
+		}
+		todo_list.current++;
 	}
-	if (index_differs_from("HEAD", 0, 0)) {
-		res = error_dirty_index(opts);
-		goto release_todo_list;
-	}
-	todo_list.current++;
+
 	res = pick_commits(&todo_list, opts);
 release_todo_list:
 	todo_list_release(&todo_list);
-- 
2.11.0.rc3.windows.1

[PATCH v3 16/38] sequencer (rebase -i): the todo can be empty when continuing

[Johannes Schindelin]

When the last command of an interactive rebase fails, the user needs to
resolve the problem and then continue the interactive rebase. Naturally,
the todo script is empty by then. So let's not complain about that!

To that end, let's move that test out of the function that parses the
todo script, and into the more high-level function read_populate_todo().
This is also necessary by now because the lower-level parse_insn_buffer()
has no idea whether we are performing an interactive rebase or not.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index a7b9ee0d04..6a840216b1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1215,8 +1215,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		else if (!is_noop(item->command))
 			fixup_okay = 1;
 	}
-	if (!todo_list->nr)
-		return error(_("no commits parsed."));
+
 	return res;
 }
 
@@ -1240,6 +1239,10 @@ static int read_populate_todo(struct todo_list *todo_list,
 	if (res)
 		return error(_("unusable instruction sheet: '%s'"), todo_file);
 
+	if (!todo_list->nr &&
+	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
+		return error(_("no commits parsed."));
+
 	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
 			opts->action == REPLAY_PICK ? TODO_PICK : TODO_REVERT;
-- 
2.11.0.rc3.windows.1

[PATCH v3 17/38] sequencer (rebase -i): update refs after a successful rebase

[Johannes Schindelin]

An interactive rebase operates on a detached HEAD (to keep the reflog
of the original branch relatively clean), and updates the branch only
at the end.

Now that the sequencer learns to perform interactive rebases, it also
needs to learn the trick to update the branch before removing the
directory containing the state of the interactive rebase.

We introduce a new head_ref variable in a wider scope than necessary at
the moment, to allow for a later patch that prints out "Successfully
rebased and updated <ref>".

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 46 +++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 45 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 6a840216b1..80b2b2a975 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -102,6 +102,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
 static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
+static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
+static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1784,12 +1786,53 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	}
 
 	if (is_rebase_i(opts)) {
-		struct strbuf buf = STRBUF_INIT;
+		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
 
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
 			return 0;
 
+		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
+				starts_with(head_ref.buf, "refs/")) {
+			unsigned char head[20], orig[20];
+			int res;
+
+			if (get_sha1("HEAD", head)) {
+				res = error(_("cannot read HEAD"));
+cleanup_head_ref:
+				strbuf_release(&head_ref);
+				strbuf_release(&buf);
+				return res;
+			}
+			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
+					get_sha1_hex(buf.buf, orig)) {
+				res = error(_("could not read orig-head"));
+				goto cleanup_head_ref;
+			}
+			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
+				head_ref.buf);
+			if (!read_oneliner(&buf, rebase_path_onto(), 0)) {
+				res = error(_("could not read 'onto'"));
+				goto cleanup_head_ref;
+			}
+			if (update_ref(buf.buf, head_ref.buf, head, orig,
+					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR)) {
+				res = error(_("could not update %s"),
+					head_ref.buf);
+				goto cleanup_head_ref;
+			}
+			strbuf_reset(&buf);
+			strbuf_addf(&buf,
+				"rebase -i (finish): returning to %s",
+				head_ref.buf);
+			if (create_symref("HEAD", head_ref.buf, buf.buf)) {
+				res = error(_("could not update HEAD to %s"),
+					head_ref.buf);
+				goto cleanup_head_ref;
+			}
+			strbuf_reset(&buf);
+		}
+
 		if (opts->verbose) {
 			struct rev_info log_tree_opt;
 			struct object_id orig, head;
@@ -1810,6 +1853,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			}
 		}
 		strbuf_release(&buf);
+		strbuf_release(&head_ref);
 	}
 
 	/*
-- 
2.11.0.rc3.windows.1

[PATCH v3 18/38] sequencer (rebase -i): leave a patch upon error

[Johannes Schindelin]

When doing an interactive rebase, we want to leave a 'patch' file for
further inspection by the user (even if we never tried to actually apply
that patch, since we're cherry-picking instead).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 80b2b2a975..a2002f1c12 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1769,7 +1769,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					intend_to_amend();
 				return error_failed_squash(item->commit, opts,
 					item->arg_len, item->arg);
-			}
+			} else if (res && is_rebase_i(opts))
+				return res | error_with_patch(item->commit,
+					item->arg, item->arg_len, opts, res, 0);
 		} else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
 			int saved = *end_of_arg;
-- 
2.11.0.rc3.windows.1

[PATCH v3 19/38] sequencer (rebase -i): implement the 'reword' command

[Johannes Schindelin]

This is now trivial, as all the building blocks are in place: all we need
to do is to flip the "edit" switch when committing.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index a2002f1c12..50e998acc4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -718,6 +718,7 @@ enum todo_command {
 	TODO_PICK = 0,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_REWORD,
 	TODO_FIXUP,
 	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
@@ -733,6 +734,7 @@ static struct {
 	{ 'p', "pick" },
 	{ 0,   "revert" },
 	{ 'e', "edit" },
+	{ 'r', "reword" },
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
@@ -962,7 +964,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
-	if (is_fixup(command)) {
+	if (command == TODO_REWORD)
+		edit = 1;
+	else if (is_fixup(command)) {
 		if (update_squash_messages(command, commit, opts))
 			return -1;
 		amend = 1;
@@ -1771,7 +1775,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg_len, item->arg);
 			} else if (res && is_rebase_i(opts))
 				return res | error_with_patch(item->commit,
-					item->arg, item->arg_len, opts, res, 0);
+					item->arg, item->arg_len, opts, res,
+					item->command == TODO_REWORD);
 		} else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
 			int saved = *end_of_arg;
-- 
2.11.0.rc3.windows.1

[PATCH v3 20/38] sequencer (rebase -i): allow fast-forwarding for edit/reword

[Johannes Schindelin]

The sequencer already knew how to fast-forward instead of
cherry-picking, if possible.

We want to continue to do this, of course, but in case of the 'reword'
command, we will need to call `git commit` after fast-forwarding.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 50e998acc4..23161f593e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -860,7 +860,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res, unborn = 0, amend = 0, allow;
+	int res, unborn = 0, amend = 0, allow = 0;
 
 	if (opts->no_commit) {
 		/*
@@ -905,11 +905,23 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	else
 		parent = commit->parents->item;
 
+	if (get_message(commit, &msg) != 0)
+		return error(_("cannot get commit message for %s"),
+			oid_to_hex(&commit->object.oid));
+
 	if (opts->allow_ff && !is_fixup(command) &&
 	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
-	     (!parent && unborn)))
-		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
-
+	     (!parent && unborn))) {
+		if (is_rebase_i(opts))
+			write_author_script(msg.message);
+		res = fast_forward_to(commit->object.oid.hash, head, unborn,
+			opts);
+		if (res || command != TODO_REWORD)
+			goto leave;
+		edit = amend = 1;
+		msg_file = NULL;
+		goto fast_forward_edit;
+	}
 	if (parent && parse_commit(parent) < 0)
 		/* TRANSLATORS: The first %s will be a "todo" command like
 		   "revert" or "pick", the second %s a SHA1. */
@@ -917,10 +929,6 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 			command_to_string(command),
 			oid_to_hex(&parent->object.oid));
 
-	if (get_message(commit, &msg) != 0)
-		return error(_("cannot get commit message for %s"),
-			oid_to_hex(&commit->object.oid));
-
 	/*
 	 * "commit" is an existing commit.  We would want to apply
 	 * the difference it introduces since its first parent "prev"
@@ -1044,6 +1052,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
+fast_forward_edit:
 		res = run_git_commit(msg_file, opts, allow, edit, amend,
 				     cleanup_commit_message);
 
-- 
2.11.0.rc3.windows.1

[PATCH v3 21/38] sequencer (rebase -i): refactor setting the reflog message

[Johannes Schindelin]

This makes the code DRYer, with the obvious benefit that we can enhance
the code further in a single place.

We can also reuse the functionality elsewhere by calling this new
function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 ++++++++++++++++++++++++++-------
 1 file changed, 26 insertions(+), 7 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 23161f593e..0d8e11f580 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1743,6 +1743,26 @@ static int is_final_fixup(struct todo_list *todo_list)
 	return 1;
 }
 
+static const char *reflog_message(struct replay_opts *opts,
+	const char *sub_action, const char *fmt, ...)
+{
+	va_list ap;
+	static struct strbuf buf = STRBUF_INIT;
+
+	va_start(ap, fmt);
+	strbuf_reset(&buf);
+	strbuf_addstr(&buf, action_name(opts));
+	if (sub_action)
+		strbuf_addf(&buf, " (%s)", sub_action);
+	if (fmt) {
+		strbuf_addstr(&buf, ": ");
+		strbuf_vaddf(&buf, fmt, ap);
+	}
+	va_end(ap);
+
+	return buf.buf;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1810,6 +1830,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 
 		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
 				starts_with(head_ref.buf, "refs/")) {
+			const char *msg;
 			unsigned char head[20], orig[20];
 			int res;
 
@@ -1825,23 +1846,21 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				res = error(_("could not read orig-head"));
 				goto cleanup_head_ref;
 			}
-			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
-				head_ref.buf);
 			if (!read_oneliner(&buf, rebase_path_onto(), 0)) {
 				res = error(_("could not read 'onto'"));
 				goto cleanup_head_ref;
 			}
-			if (update_ref(buf.buf, head_ref.buf, head, orig,
+			msg = reflog_message(opts, "finish", "%s onto %s",
+				head_ref.buf, buf.buf);
+			if (update_ref(msg, head_ref.buf, head, orig,
 					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR)) {
 				res = error(_("could not update %s"),
 					head_ref.buf);
 				goto cleanup_head_ref;
 			}
-			strbuf_reset(&buf);
-			strbuf_addf(&buf,
-				"rebase -i (finish): returning to %s",
+			msg = reflog_message(opts, "finish", "returning to %s",
 				head_ref.buf);
-			if (create_symref("HEAD", head_ref.buf, buf.buf)) {
+			if (create_symref("HEAD", head_ref.buf, msg)) {
 				res = error(_("could not update HEAD to %s"),
 					head_ref.buf);
 				goto cleanup_head_ref;
-- 
2.11.0.rc3.windows.1

[PATCH v3 22/38] sequencer (rebase -i): set the reflog message consistently

[Johannes Schindelin]

We already used the same reflog message as the scripted version of rebase
-i when finishing. With this commit, we do that also for all the commands
before that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 0d8e11f580..95ae4bcd1e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1785,6 +1785,10 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			unlink(rebase_path_amend());
 		}
 		if (item->command <= TODO_SQUASH) {
+			if (is_rebase_i(opts))
+				setenv("GIT_REFLOG_ACTION", reflog_message(opts,
+					command_to_string(item->command), NULL),
+					1);
 			res = do_pick_commit(item->command, item->commit,
 					opts, is_final_fixup(todo_list));
 			if (item->command == TODO_EDIT) {
-- 
2.11.0.rc3.windows.1

[PATCH v3 23/38] sequencer (rebase -i): copy commit notes at end

[Johannes Schindelin]

When rebasing commits that have commit notes attached, the interactive
rebase rewrites those notes faithfully at the end. The sequencer must
do this, too, if it wishes to do interactive rebase's job.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 76 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 95ae4bcd1e..50380a15b8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -96,6 +96,15 @@ static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
  */
 static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 /*
+ * For the post-rewrite hook, we make a list of rewritten commits and
+ * their new sha1s.  The rewritten-pending list keeps the sha1s of
+ * commits that have been processed, but not committed yet,
+ * e.g. because they are waiting for a 'squash' command.
+ */
+static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
+static GIT_PATH_FUNC(rebase_path_rewritten_pending,
+	"rebase-merge/rewritten-pending")
+/*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
  */
@@ -850,6 +859,44 @@ static int update_squash_messages(enum todo_command command,
 	return res;
 }
 
+static void flush_rewritten_pending(void) {
+	struct strbuf buf = STRBUF_INIT;
+	unsigned char newsha1[20];
+	FILE *out;
+
+	if (strbuf_read_file(&buf, rebase_path_rewritten_pending(), 82) > 0 &&
+			!get_sha1("HEAD", newsha1) &&
+			(out = fopen(rebase_path_rewritten_list(), "a"))) {
+		char *bol = buf.buf, *eol;
+
+		while (*bol) {
+			eol = strchrnul(bol, '\n');
+			fprintf(out, "%.*s %s\n", (int)(eol - bol),
+					bol, sha1_to_hex(newsha1));
+			if (!*eol)
+				break;
+			bol = eol + 1;
+		}
+		fclose(out);
+		unlink(rebase_path_rewritten_pending());
+	}
+	strbuf_release(&buf);
+}
+
+static void record_in_rewritten(struct object_id *oid,
+		enum todo_command next_command) {
+	FILE *out = fopen(rebase_path_rewritten_pending(), "a");
+
+	if (!out)
+		return;
+
+	fprintf(out, "%s\n", oid_to_hex(oid));
+	fclose(out);
+
+	if (!is_fixup(next_command))
+		flush_rewritten_pending();
+}
+
 static int do_pick_commit(enum todo_command command, struct commit *commit,
 		struct replay_opts *opts, int final_fixup)
 {
@@ -1743,6 +1790,17 @@ static int is_final_fixup(struct todo_list *todo_list)
 	return 1;
 }
 
+static enum todo_command peek_command(struct todo_list *todo_list, int offset)
+{
+	int i;
+
+	for (i = todo_list->current + offset; i < todo_list->nr; i++)
+		if (!is_noop(todo_list->items[i].command))
+			return todo_list->items[i].command;
+
+	return -1;
+}
+
 static const char *reflog_message(struct replay_opts *opts,
 	const char *sub_action, const char *fmt, ...)
 {
@@ -1801,6 +1859,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+			if (is_rebase_i(opts) && !res)
+				record_in_rewritten(&item->commit->object.oid,
+					peek_command(todo_list, 1));
 			if (res && is_fixup(item->command)) {
 				if (res == 1)
 					intend_to_amend();
@@ -1827,6 +1888,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 
 	if (is_rebase_i(opts)) {
 		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
+		struct stat st;
 
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
@@ -1891,6 +1953,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				log_tree_diff_flush(&log_tree_opt);
 			}
 		}
+		flush_rewritten_pending();
+		if (!stat(rebase_path_rewritten_list(), &st) &&
+				st.st_size > 0) {
+			struct child_process child = CHILD_PROCESS_INIT;
+
+			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			child.git_cmd = 1;
+			argv_array_push(&child.args, "notes");
+			argv_array_push(&child.args, "copy");
+			argv_array_push(&child.args, "--for-rewrite=rebase");
+			/* we don't care if this copying failed */
+			run_command(&child);
+		}
+
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
 	}
-- 
2.11.0.rc3.windows.1

[PATCH v3 24/38] sequencer (rebase -i): record interrupted commits in rewritten, too

[Johannes Schindelin]

When continuing after a `pick` command failed, we want that commit
to show up in the rewritten-list (and its notes to be rewritten), too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 50380a15b8..d7273fd1b3 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2060,6 +2060,14 @@ int sequencer_continue(struct replay_opts *opts)
 			goto release_todo_list;
 		}
 		todo_list.current++;
+	} else if (file_exists(rebase_path_stopped_sha())) {
+		struct strbuf buf = STRBUF_INIT;
+		struct object_id oid;
+
+		if (read_oneliner(&buf, rebase_path_stopped_sha(), 1) &&
+		    !get_sha1_committish(buf.buf, oid.hash))
+			record_in_rewritten(&oid, peek_command(&todo_list, 0));
+		strbuf_release(&buf);
 	}
 
 	res = pick_commits(&todo_list, opts);
-- 
2.11.0.rc3.windows.1

[PATCH v3 25/38] sequencer (rebase -i): run the post-rewrite hook, if needed

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index d7273fd1b3..43ced8db31 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1957,6 +1957,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process child = CHILD_PROCESS_INIT;
+			const char *post_rewrite_hook =
+				find_hook("post-rewrite");
 
 			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			child.git_cmd = 1;
@@ -1965,6 +1967,18 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			argv_array_push(&child.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
 			run_command(&child);
+
+			if (post_rewrite_hook) {
+				struct child_process hook = CHILD_PROCESS_INIT;
+
+				hook.in = open(rebase_path_rewritten_list(),
+					O_RDONLY);
+				hook.stdout_to_stderr = 1;
+				argv_array_push(&hook.args, post_rewrite_hook);
+				argv_array_push(&hook.args, "rebase");
+				/* we don't care if this hook failed */
+				run_command(&hook);
+			}
 		}
 
 		strbuf_release(&buf);
-- 
2.11.0.rc3.windows.1

[PATCH v3 26/38] sequencer (rebase -i): respect the rebase.autostash setting

[Johannes Schindelin]

Git's `rebase` command inspects the `rebase.autostash` config setting
to determine whether it should stash any uncommitted changes before
rebasing and re-apply them afterwards.

As we introduce more bits and pieces to let the sequencer act as
interactive rebase's backend, here is the part that adds support for
the autostash feature.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 43ced8db31..06f7cebe48 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -113,6 +113,7 @@ static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
 static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
 static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
+static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1801,6 +1802,47 @@ static enum todo_command peek_command(struct todo_list *todo_list, int offset)
 	return -1;
 }
 
+static int apply_autostash(struct replay_opts *opts)
+{
+	struct strbuf stash_sha1 = STRBUF_INIT;
+	struct child_process child = CHILD_PROCESS_INIT;
+	int ret = 0;
+
+	if (!read_oneliner(&stash_sha1, rebase_path_autostash(), 1)) {
+		strbuf_release(&stash_sha1);
+		return 0;
+	}
+	strbuf_trim(&stash_sha1);
+
+	child.git_cmd = 1;
+	argv_array_push(&child.args, "stash");
+	argv_array_push(&child.args, "apply");
+	argv_array_push(&child.args, stash_sha1.buf);
+	if (!run_command(&child))
+		printf(_("Applied autostash."));
+	else {
+		struct child_process store = CHILD_PROCESS_INIT;
+
+		store.git_cmd = 1;
+		argv_array_push(&store.args, "stash");
+		argv_array_push(&store.args, "store");
+		argv_array_push(&store.args, "-m");
+		argv_array_push(&store.args, "autostash");
+		argv_array_push(&store.args, "-q");
+		argv_array_push(&store.args, stash_sha1.buf);
+		if (run_command(&store))
+			ret = error(_("cannot store %s"), stash_sha1.buf);
+		else
+			printf(_("Applying autostash resulted in conflicts.\n"
+				"Your changes are safe in the stash.\n"
+				"You can run \"git stash pop\" or"
+				" \"git stash drop\" at any time.\n"));
+	}
+
+	strbuf_release(&stash_sha1);
+	return ret;
+}
+
 static const char *reflog_message(struct replay_opts *opts,
 	const char *sub_action, const char *fmt, ...)
 {
@@ -1980,6 +2022,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				run_command(&hook);
 			}
 		}
+		apply_autostash(opts);
 
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
-- 
2.11.0.rc3.windows.1

[PATCH v3 27/38] sequencer (rebase -i): respect strategy/strategy_opts settings

[Johannes Schindelin]

The sequencer already has an idea about using different merge
strategies. We just piggy-back on top of that, using rebase -i's
own settings, when running the sequencer in interactive rebase mode.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 06f7cebe48..04a64cf0dc 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -114,6 +114,8 @@ static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
 static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
 static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
+static GIT_PATH_FUNC(rebase_path_strategy, "rebase-merge/strategy")
+static GIT_PATH_FUNC(rebase_path_strategy_opts, "rebase-merge/strategy_opts")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1368,6 +1370,26 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 	return 0;
 }
 
+static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
+{
+	int i;
+
+	strbuf_reset(buf);
+	if (!read_oneliner(buf, rebase_path_strategy(), 0))
+		return;
+	opts->strategy = strbuf_detach(buf, NULL);
+	if (!read_oneliner(buf, rebase_path_strategy_opts(), 0))
+		return;
+
+	opts->xopts_nr = split_cmdline(buf->buf, (const char ***)&opts->xopts);
+	for (i = 0; i < opts->xopts_nr; i++) {
+		const char *arg = opts->xopts[i];
+
+		skip_prefix(arg, "--", &arg);
+		opts->xopts[i] = xstrdup(arg);
+	}
+}
+
 static int read_populate_opts(struct replay_opts *opts)
 {
 	if (is_rebase_i(opts)) {
@@ -1381,11 +1403,13 @@ static int read_populate_opts(struct replay_opts *opts)
 				opts->gpg_sign = xstrdup(buf.buf + 2);
 			}
 		}
-		strbuf_release(&buf);
 
 		if (file_exists(rebase_path_verbose()))
 			opts->verbose = 1;
 
+		read_strategy_opts(opts, &buf);
+		strbuf_release(&buf);
+
 		return 0;
 	}
 
-- 
2.11.0.rc3.windows.1

[PATCH v3 28/38] sequencer (rebase -i): allow rescheduling commands

[Johannes Schindelin]

The interactive rebase has the very special magic that a cherry-pick
that exits with a status different from 0 and 1 signifies a failure to
even record that a cherry-pick was started.

This can happen e.g. when a fast-forward fails because it would
overwrite untracked files.

In that case, we must reschedule the command that we thought we already
had at least started successfully.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 04a64cf0dc..dd5b843a84 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1915,6 +1915,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					1);
 			res = do_pick_commit(item->command, item->commit,
 					opts, is_final_fixup(todo_list));
+			if (is_rebase_i(opts) && res < 0) {
+				/* Reschedule */
+				todo_list->current--;
+				if (save_todo(todo_list, opts))
+					return -1;
+			}
 			if (item->command == TODO_EDIT) {
 				struct commit *commit = item->commit;
 				if (!res)
-- 
2.11.0.rc3.windows.1

[PATCH v3 29/38] sequencer (rebase -i): implement the 'drop' command

[Johannes Schindelin]

The parsing part of a 'drop' command is almost identical to parsing a
'pick', while the operation is the same as that of a 'noop'.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index dd5b843a84..6e92f186ae 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -736,7 +736,8 @@ enum todo_command {
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
-	TODO_NOOP
+	TODO_NOOP,
+	TODO_DROP
 };
 
 static struct {
@@ -750,7 +751,8 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
-	{ 0,   "noop" }
+	{ 0,   "noop" },
+	{ 'd', "drop" }
 };
 
 static const char *command_to_string(const enum todo_command command)
@@ -762,7 +764,7 @@ static const char *command_to_string(const enum todo_command command)
 
 static int is_noop(const enum todo_command command)
 {
-	return TODO_NOOP <= (size_t)command;
+	return TODO_NOOP <= command;
 }
 
 static int is_fixup(enum todo_command command)
-- 
2.11.0.rc3.windows.1

[PATCH v3 30/38] sequencer (rebase -i): differentiate between comments and 'noop'

[Johannes Schindelin]

In the upcoming patch, we will support rebase -i's progress
reporting. The progress skips comments but counts 'noop's.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 6e92f186ae..41f80ea2c4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -737,7 +737,9 @@ enum todo_command {
 	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
-	TODO_DROP
+	TODO_DROP,
+	/* comments (not counted for reporting progress) */
+	TODO_COMMENT
 };
 
 static struct {
@@ -752,12 +754,13 @@ static struct {
 	{ 's', "squash" },
 	{ 'x', "exec" },
 	{ 0,   "noop" },
-	{ 'd', "drop" }
+	{ 'd', "drop" },
+	{ 0,   NULL }
 };
 
 static const char *command_to_string(const enum todo_command command)
 {
-	if ((size_t)command < ARRAY_SIZE(todo_command_info))
+	if (command < TODO_COMMENT)
 		return todo_command_info[command].str;
 	die("Unknown command: %d", command);
 }
@@ -1198,14 +1201,14 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	bol += strspn(bol, " \t");
 
 	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
-		item->command = TODO_NOOP;
+		item->command = TODO_COMMENT;
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = eol - bol;
 		return 0;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)
+	for (i = 0; i < TODO_COMMENT; i++)
 		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
 			item->command = i;
 			break;
@@ -1214,7 +1217,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 			item->command = i;
 			break;
 		}
-	if (i >= ARRAY_SIZE(todo_command_info))
+	if (i >= TODO_COMMENT)
 		return -1;
 
 	if (item->command == TODO_NOOP) {
-- 
2.11.0.rc3.windows.1

[PATCH v3 31/38] sequencer: make reading author-script more elegant

[Johannes Schindelin]

Rather than abusing a strbuf to come up with an environment block, let's
just use the argv_array structure which serves the same purpose much
better.

While at it, rename the function to reflect the fact that it does not
really care exactly what environment variables are defined in said file.

Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 32 +++++++++++---------------------
 1 file changed, 11 insertions(+), 21 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 41f80ea2c4..a0d0aaeaf8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -544,18 +544,17 @@ static int write_author_script(const char *message)
 }
 
 /*
- * Read the author-script file into an environment block, ready for use in
- * run_command(), that can be free()d afterwards.
+ * Read a list of environment variable assignments (such as the author-script
+ * file) into an environment block. Returns -1 on error, 0 otherwise.
  */
-static char **read_author_script(void)
+static int read_env_script(struct argv_array *env)
 {
 	struct strbuf script = STRBUF_INIT;
 	int i, count = 0;
-	char *p, *p2, **env;
-	size_t env_size;
+	char *p, *p2;
 
 	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
-		return NULL;
+		return -1;
 
 	for (p = script.buf; *p; p++)
 		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
@@ -567,19 +566,12 @@ static char **read_author_script(void)
 			count++;
 		}
 
-	env_size = (count + 1) * sizeof(*env);
-	strbuf_grow(&script, env_size);
-	memmove(script.buf + env_size, script.buf, script.len);
-	p = script.buf + env_size;
-	env = (char **)strbuf_detach(&script, NULL);
-
 	for (i = 0; i < count; i++) {
-		env[i] = p;
+		argv_array_push(env, p);
 		p += strlen(p) + 1;
 	}
-	env[count] = NULL;
 
-	return env;
+	return 0;
 }
 
 static const char staged_changes_advice[] =
@@ -612,14 +604,12 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 			  int allow_empty, int edit, int amend,
 			  int cleanup_commit_message)
 {
-	char **env = NULL;
-	struct argv_array array;
+	struct argv_array env = ARGV_ARRAY_INIT, array;
 	int rc;
 	const char *value;
 
 	if (is_rebase_i(opts)) {
-		env = read_author_script();
-		if (!env) {
+		if (!read_env_script(&env)) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
 			return error(_(staged_changes_advice),
@@ -655,9 +645,9 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_push(&array, "--allow-empty-message");
 
 	rc = run_command_v_opt_cd_env(array.argv, RUN_GIT_CMD, NULL,
-			(const char *const *)env);
+			(const char *const *)env.argv);
 	argv_array_clear(&array);
-	free(env);
+	argv_array_clear(&env);
 
 	return rc;
 }
-- 
2.11.0.rc3.windows.1

[PATCH v3 32/38] sequencer: use run_command() directly

[Johannes Schindelin]

Instead of using the convenience function run_command_v_opt_cd_env(), we
now use the run_command() function. The former function is simply a
wrapper of the latter, trying to make it more convenient to use.

However, we already have to construct the argv and the env parameters,
and we will need even finer control e.g. over the output of the command,
so let's just stop using the convenience function.

Based on patches and suggestions by Johannes Sixt and Jeff King.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 36 ++++++++++++++++--------------------
 1 file changed, 16 insertions(+), 20 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index a0d0aaeaf8..c7dc5a2ad4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -604,12 +604,13 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 			  int allow_empty, int edit, int amend,
 			  int cleanup_commit_message)
 {
-	struct argv_array env = ARGV_ARRAY_INIT, array;
-	int rc;
+	struct child_process cmd = CHILD_PROCESS_INIT;
 	const char *value;
 
+	cmd.git_cmd = 1;
+
 	if (is_rebase_i(opts)) {
-		if (!read_env_script(&env)) {
+		if (read_env_script(&cmd.env_array)) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
 			return error(_(staged_changes_advice),
@@ -617,39 +618,34 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		}
 	}
 
-	argv_array_init(&array);
-	argv_array_push(&array, "commit");
-	argv_array_push(&array, "-n");
+	argv_array_push(&cmd.args, "commit");
+	argv_array_push(&cmd.args, "-n");
 
 	if (amend)
-		argv_array_push(&array, "--amend");
+		argv_array_push(&cmd.args, "--amend");
 	if (opts->gpg_sign)
-		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
+		argv_array_pushf(&cmd.args, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
-		argv_array_push(&array, "-s");
+		argv_array_push(&cmd.args, "-s");
 	if (defmsg)
-		argv_array_pushl(&array, "-F", defmsg, NULL);
+		argv_array_pushl(&cmd.args, "-F", defmsg, NULL);
 	if (cleanup_commit_message)
-		argv_array_push(&array, "--cleanup=strip");
+		argv_array_push(&cmd.args, "--cleanup=strip");
 	if (edit)
-		argv_array_push(&array, "-e");
+		argv_array_push(&cmd.args, "-e");
 	else if (!cleanup_commit_message &&
 		 !opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
-		argv_array_push(&array, "--cleanup=verbatim");
+		argv_array_push(&cmd.args, "--cleanup=verbatim");
 
 	if (allow_empty)
-		argv_array_push(&array, "--allow-empty");
+		argv_array_push(&cmd.args, "--allow-empty");
 
 	if (opts->allow_empty_message)
-		argv_array_push(&array, "--allow-empty-message");
+		argv_array_push(&cmd.args, "--allow-empty-message");
 
-	rc = run_command_v_opt_cd_env(array.argv, RUN_GIT_CMD, NULL,
-			(const char *const *)env.argv);
-	argv_array_clear(&array);
-	argv_array_clear(&env);
 
-	return rc;
+	return run_command(&cmd);
 }
 
 static int is_original_commit_empty(struct commit *commit)
-- 
2.11.0.rc3.windows.1

[PATCH v3 33/38] sequencer (rebase -i): show only failed `git commit`'s output

[Johannes Schindelin]

This is the behavior of the shell script version of the interactive
rebase, by using the `output` function defined in `git-rebase.sh`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index c7dc5a2ad4..a501dfce38 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -610,6 +610,11 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	cmd.git_cmd = 1;
 
 	if (is_rebase_i(opts)) {
+		if (!edit) {
+			cmd.stdout_to_stderr = 1;
+			cmd.err = -1;
+		}
+
 		if (read_env_script(&cmd.env_array)) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
@@ -644,6 +649,19 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	if (opts->allow_empty_message)
 		argv_array_push(&cmd.args, "--allow-empty-message");
 
+	if (cmd.err == -1) {
+		/* hide stderr on success */
+		struct strbuf buf = STRBUF_INIT;
+		int rc = pipe_command(&cmd,
+				      NULL, 0,
+				      /* stdout is already redirected */
+				      NULL, 0,
+				      &buf, 0);
+		if (rc)
+			fputs(buf.buf, stderr);
+		strbuf_release(&buf);
+		return rc;
+	}
 
 	return run_command(&cmd);
 }
-- 
2.11.0.rc3.windows.1

[PATCH v3 34/38] sequencer (rebase -i): show only failed cherry-picks' output

[Johannes Schindelin]

This is the behavior of the shell script version of the interactive
rebase, by using the `output` function defined in `git-rebase.sh`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index a501dfce38..4f37ba8d33 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -433,6 +433,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	o.ancestor = base ? base_label : "(empty tree)";
 	o.branch1 = "HEAD";
 	o.branch2 = next ? next_label : "(empty tree)";
+	if (is_rebase_i(opts))
+		o.buffer_output = 2;
 
 	head_tree = parse_tree_indirect(head);
 	next_tree = next ? next->tree : empty_tree();
@@ -444,6 +446,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	if (is_rebase_i(opts) && clean <= 0)
+		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
 	if (clean < 0)
 		return clean;
-- 
2.11.0.rc3.windows.1

[PATCH v3 35/38] sequencer (rebase -i): suggest --edit-todo upon unknown command

[Johannes Schindelin]

This is the same behavior as known from `git rebase -i`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 4f37ba8d33..4792a3de3b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1314,8 +1314,12 @@ static int read_populate_todo(struct todo_list *todo_list,
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
-	if (res)
+	if (res) {
+		if (is_rebase_i(opts))
+			return error(_("please fix this using "
+				       "'git rebase --edit-todo'."));
 		return error(_("unusable instruction sheet: '%s'"), todo_file);
+	}
 
 	if (!todo_list->nr &&
 	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
-- 
2.11.0.rc3.windows.1

[PATCH v3 36/38] sequencer (rebase -i): show the progress

[Johannes Schindelin]

The interactive rebase keeps the user informed about its progress.
If the sequencer wants to do the grunt work of the interactive
rebase, it also needs to show that progress.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 4792a3de3b..2c9c555ab6 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1181,6 +1181,7 @@ struct todo_list {
 	struct strbuf buf;
 	struct todo_item *items;
 	int nr, alloc, current;
+	int done_nr, total_nr;
 };
 
 #define TODO_LIST_INIT { STRBUF_INIT }
@@ -1297,6 +1298,17 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 	return res;
 }
 
+static int count_commands(struct todo_list *todo_list)
+{
+	int count = 0, i;
+
+	for (i = 0; i < todo_list->nr; i++)
+		if (todo_list->items[i].command != TODO_COMMENT)
+			count++;
+
+	return count;
+}
+
 static int read_populate_todo(struct todo_list *todo_list,
 			struct replay_opts *opts)
 {
@@ -1339,6 +1351,21 @@ static int read_populate_todo(struct todo_list *todo_list,
 				return error(_("cannot revert during a cherry-pick."));
 	}
 
+	if (is_rebase_i(opts)) {
+		struct todo_list done = TODO_LIST_INIT;
+
+		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
+				!parse_insn_buffer(done.buf.buf, &done))
+			todo_list->done_nr = count_commands(&done);
+		else
+			todo_list->done_nr = 0;
+
+		todo_list->total_nr = todo_list->done_nr
+			+ count_commands(todo_list);
+
+		todo_list_release(&done);
+	}
+
 	return 0;
 }
 
@@ -1920,6 +1947,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
+			if (item->command != TODO_COMMENT)
+				fprintf(stderr, "Rebasing (%d/%d)%s",
+					++(todo_list->done_nr),
+					todo_list->total_nr,
+					opts->verbose ? "\n" : "\r");
 			unlink(rebase_path_message());
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
-- 
2.11.0.rc3.windows.1

[PATCH v3 37/38] sequencer (rebase -i): write the progress into files

[Johannes Schindelin]

For the benefit of e.g. the shell prompt, the interactive rebase not
only displays the progress for the user to see, but also writes it into
the msgnum/end files in the state directory.

Teach the sequencer this new trick.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 2c9c555ab6..b39cd21e03 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -47,6 +47,16 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
 /*
+ * The file to keep track of how many commands were already processed (e.g.
+ * for the prompt).
+ */
+static GIT_PATH_FUNC(rebase_path_msgnum, "rebase-merge/msgnum");
+/*
+ * The file to keep track of how many commands are to be processed in total
+ * (e.g. for the prompt).
+ */
+static GIT_PATH_FUNC(rebase_path_msgtotal, "rebase-merge/end");
+/*
  * The commit message that is planned to be used for any changes that
  * need to be committed following a user interaction.
  */
@@ -1353,6 +1363,7 @@ static int read_populate_todo(struct todo_list *todo_list,
 
 	if (is_rebase_i(opts)) {
 		struct todo_list done = TODO_LIST_INIT;
+		FILE *f = fopen(rebase_path_msgtotal(), "w");
 
 		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
 				!parse_insn_buffer(done.buf.buf, &done))
@@ -1362,8 +1373,12 @@ static int read_populate_todo(struct todo_list *todo_list,
 
 		todo_list->total_nr = todo_list->done_nr
 			+ count_commands(todo_list);
-
 		todo_list_release(&done);
+
+		if (f) {
+			fprintf(f, "%d\n", todo_list->total_nr);
+			fclose(f);
+		}
 	}
 
 	return 0;
@@ -1947,11 +1962,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
-			if (item->command != TODO_COMMENT)
+			if (item->command != TODO_COMMENT) {
+				FILE *f = fopen(rebase_path_msgnum(), "w");
+
+				todo_list->done_nr++;
+
+				if (f) {
+					fprintf(f, "%d\n", todo_list->done_nr);
+					fclose(f);
+				}
 				fprintf(stderr, "Rebasing (%d/%d)%s",
-					++(todo_list->done_nr),
+					todo_list->done_nr,
 					todo_list->total_nr,
 					opts->verbose ? "\n" : "\r");
+			}
 			unlink(rebase_path_message());
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
-- 
2.11.0.rc3.windows.1

[PATCH v3 38/38] sequencer (rebase -i): write out the final message

[Johannes Schindelin]

The shell script version of the interactive rebase has a very specific
final message. Teach the sequencer to print the same.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index b39cd21e03..0e7d2ca5c8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2127,6 +2127,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		}
 		apply_autostash(opts);
 
+		fprintf(stderr, "Successfully rebased and updated %s.\n",
+			head_ref.buf);
+
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
 	}
-- 
2.11.0.rc3.windows.1

Re: [PATCH v2 05/34] sequencer (rebase -i): learn about the 'verbose' mode

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Junio,
>
> On Tue, 13 Dec 2016, Junio C Hamano wrote:
>
>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> 
>> > @@ -1493,9 +1498,26 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>> >  	}
>> >  
>> >  	if (is_rebase_i(opts)) {
>> > +		struct strbuf buf = STRBUF_INIT;
>> > +
>> >  		/* Stopped in the middle, as planned? */
>> >  		if (todo_list->current < todo_list->nr)
>> >  			return 0;
>> > +
>> > +		if (opts->verbose) {
>> > +			const char *argv[] = {
>> > +				"diff-tree", "--stat", NULL, NULL
>> > +			};
>> > +
>> > +			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
>> > +				return error(_("could not read '%s'"),
>> > +					rebase_path_orig_head());
>> > +			strbuf_addstr(&buf, "..HEAD");
>> > +			argv[2] = buf.buf;
>> > +			run_command_v_opt(argv, RUN_GIT_CMD);
>> > +			strbuf_reset(&buf);
>> > +		}
>> > +		strbuf_release(&buf);
>> >  	}
>> 
>> It's a bit curious that the previous step avoided running a separate
>> process and instead did "diff-tree -p" all in C, but this one does not.
>
> I guess my only defence is that I tried to be a little lazy.

I actually was alluding to going the other way around, spawning
"diff-tree -p" in the other codepath like this one does.

Re: [PATCH v2 05/34] sequencer (rebase -i): learn about the 'verbose' mode

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> ...
>>
>> I guess my only defence is that I tried to be a little lazy.
>
> I actually was alluding to going the other way around, spawning
> "diff-tree -p" in the other codepath like this one does.

Pre-emptively, because I expect it will take a while for me to clear
the backlog to get to your reroll already on the mailing list, I do
not mean to say that you must rewrite the other one to spawn to
match this one.  I meant that I wouldn't have minded if the series
were done that way, as this kind of being "little lazy" would reduce
the chance of regression by reducing the number of exchanged parts
and it is not necessarily a bad thing.

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Changes since v2:
>
> - fixed a TRANSLATORS: comment
> ...
> - replaced a spawned `diff-tree` command by code using the diff functions
>   directly

I just finished skimming the interdiff (the difference between the
result of merging the v2 into 'master' and the result of applying
this series on 'master').  I didn't cross reference it with all the
review comments that I saw on the list for the previous round, I
wouldn't have noticed even if there are issues that still haven't
been addressed.  But what I saw in the interdiff were mostly good
changes.

I however think that the renaming of read_author_script() is totally
backwards from maintainability's point of view.  

Both "am" and "rebase" use an external file whose name is the same
"author-script" (because they are meant to serve the same purpose),
and introduction of new code that is different from what "am" uses
to read it in v2 was bad enough.  If the rename of the function in
this round really means "author-script of 'am' can hold only author
information, but that of 'rebase' can hold anything the end user
desires", it makes things even worse.  If 'rebase' side extends the
file to hold other environment variables (in its own code), users
would wonder why the same extension will not be available in 'am'
side (and vice versa).

I ran out of time for the day and haven't looked at individual
patches yet.  I may find other issues, but from the interdiff, that
was the only thing that I found even worse than the previous round,
and everything else was either the same or has improved.

Thanks.

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> I ran out of time for the day and haven't looked at individual
> patches yet.  I may find other issues, but from the interdiff, that
> was the only thing that I found even worse than the previous round,
> and everything else was either the same or has improved.

... I forgot to say a few things:

 * In the meantime, I think this is overall better than what we had
   for the past few weeks; I'll try to replace the copy in my tree
   with this round, even though this may not be the final one.

 * There were a few style issues [*1*] pointed out by the
   checkpatch.pl script borrowed from the kernel land.

 * And finally, you wrote "This marks the count down to '3': two
   more patch series after this (really tiny ones) and we have a
   faster rebase -i."  This is seriously an exciting news.  Thanks.

I probably am still forgetting to say a few more things, but I'd
send this message out in this shape for now.  Thanks.


[Footnote]

*1*

Subject: [PATCH v3 10/38] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands
ERROR: do not use assignment in if condition
#118: FILE: sequencer.c:725:
+		if (buf.buf[0] != comment_line_char ||

ERROR: do not use assignment in if condition
#142: FILE: sequencer.c:749:
+		if (!(head_commit = lookup_commit_reference(head)))

ERROR: do not use assignment in if condition
#144: FILE: sequencer.c:751:
+		if (!(head_message = get_commit_buffer(head_commit, NULL)))

ERROR: do not use assignment in if condition
#167: FILE: sequencer.c:774:
+	if (!(message = get_commit_buffer(commit, NULL)))

total: 4 errors, 1 warnings, 320 lines checked
------------------------------------------------
Subject: [PATCH v3 12/38] sequencer (rebase -i): write an author-script file
ERROR: do not use assignment in if condition
#35: FILE: sequencer.c:500:
+		else if ((eol = strchr(message, '\n')))

total: 1 errors, 0 warnings, 62 lines checked
------------------------------------------------
Subject: [PATCH v3 23/38] sequencer (rebase -i): copy commit notes at end
ERROR: open brace '{' following function declarations go on the next line
#36: FILE: sequencer.c:862:
+static void flush_rewritten_pending(void) {

ERROR: do not use assignment in if condition
#41: FILE: sequencer.c:867:
+	if (strbuf_read_file(&buf, rebase_path_rewritten_pending(), 82) > 0 &&

total: 2 errors, 0 warnings, 112 lines checked

Re: [PATCH v3 01/38] sequencer: avoid unnecessary curly braces

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

>  
> -	if (!commit->parents) {
> +	if (!commit->parents)
>  		parent = NULL;
> -	}
>  	else if (commit->parents->next) {
>  		/* Reverting or cherry-picking a merge commit */
>  		int cnt;

The result becomes

	if (...)
		single statement;
	else if (...) {
		multiple;
                statements;
        }

which is not quite an improvement.  

The preferred style is for all arms in if/elseif/else cascade to
either use or not use brace pairs, so I think a fix toward that goal
would be more like:

	if (!commit->parents) {
		parent = NULL;
-	}                
-	else if (commit->parents->next) {
+	} else if (commit->parents->next) {
		/* Reverting ...

Re: [PATCH v3 03/38] sequencer: use a helper to find the commit message

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> It is actually not safe to look for a commit message by looking for the
> first empty line and skipping it.
>
> The find_commit_subject() function looks more carefully, so let's use
> it. Since we are interested in the entire commit message, we re-compute
> the string length after verifying that the commit subject is not empty
> (in which case the entire commit message would be empty, something that
> should not happen but that we want to handle gracefully).
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---

Very sensible.

>  sequencer.c | 11 +++--------
>  1 file changed, 3 insertions(+), 8 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index 3eededcb98..720857beda 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -703,14 +703,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  		next = commit;
>  		next_label = msg.label;
>  
> -		/*
> -		 * Append the commit log message to msgbuf; it starts
> -		 * after the tree, parent, author, committer
> -		 * information followed by "\n\n".
> -		 */
> -		p = strstr(msg.message, "\n\n");
> -		if (p)
> -			strbuf_addstr(&msgbuf, skip_blank_lines(p + 2));
> +		/* Append the commit log message to msgbuf. */
> +		if (find_commit_subject(msg.message, &p))
> +			strbuf_addstr(&msgbuf, p);
>  
>  		if (opts->record_origin) {
>  			if (!has_conforming_footer(&msgbuf, NULL, 0))

Re: [PATCH v3 02/38] sequencer: move "else" keyword onto the same line as preceding brace

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> It is the current coding style of the Git project to write
>
> 	if (...) {
> 		...
> 	} else {
> 		...
> 	}
>
> instead of putting the closing brace and the "else" keyword on separate
> lines.
>
> Pointed out by Junio Hamano.

For a small thing like this, the attribution is mostly irrelevant,
but for the record, the credit should go to checkpatch.pl from the
kernel project ;-).

I'll try to squash the part that was touched by 01/38 into this
patch locally; even though I haven't finished going through the
individual patches, I expect that my conclusion would be what I
said in <xmqqinpnuaxl.fsf@gitster.mtv.corp.google.com>, i.e. the
changes in the series are mostly good and improved relative to the
previous round except for the parts reading and writing of
author-script (from the maintainability's point of view).  

As I do not think we want to see another reroll of three-dozen
patches, I am leaning towards merging v3 (with minimum fixes like
squashing 01/ and 02/ into one) to 'next' and cook it there and
fix remaining issues incrementally while the series is in 'next'.

I may have to change my mind after I read through the remaining
patches (I've done with the first dozen or so at this moment), but I
do not expect that, judging from what I saw in the interdiff.

Thanks.

> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index 23793db08b..3eededcb98 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -1070,8 +1070,7 @@ static int create_seq_dir(void)
>  		error(_("a cherry-pick or revert is already in progress"));
>  		advise(_("try \"git cherry-pick (--continue | --quit | --abort)\""));
>  		return -1;
> -	}
> -	else if (mkdir(git_path_seq_dir(), 0777) < 0)
> +	} else if (mkdir(git_path_seq_dir(), 0777) < 0)
>  		return error_errno(_("could not create sequencer directory '%s'"),
>  				   git_path_seq_dir());
>  	return 0;

Re: [PATCH v3 06/38] sequencer (rebase -i): implement the 'edit' command

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> +static int make_patch(struct commit *commit, struct replay_opts *opts)
> +{
> +	struct strbuf buf = STRBUF_INIT;
> +	struct rev_info log_tree_opt;
> +	const char *subject, *p;
> +	int res = 0;
> +
> +	p = short_commit_name(commit);
> +	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
> +		return -1;
> +
> +	strbuf_addf(&buf, "%s/patch", get_dir(opts));
> +	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
> +	init_revisions(&log_tree_opt, NULL);
> +	log_tree_opt.abbrev = 0;
> +	log_tree_opt.diff = 1;
> +	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
> +	log_tree_opt.disable_stdin = 1;
> +	log_tree_opt.no_commit_id = 1;
> +	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
> +	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
> +	if (!log_tree_opt.diffopt.file)
> +		res |= error_errno(_("could not open '%s'"), buf.buf);
> +	else {
> +		res |= log_tree_commit(&log_tree_opt, commit);
> +		fclose(log_tree_opt.diffopt.file);
> +	}
> +	strbuf_reset(&buf);
> +
> +	strbuf_addf(&buf, "%s/message", get_dir(opts));
> +	if (!file_exists(buf.buf)) {
> +		const char *commit_buffer = get_commit_buffer(commit, NULL);
> +		find_commit_subject(commit_buffer, &subject);
> +		res |= write_message(subject, strlen(subject), buf.buf, 1);
> +		unuse_commit_buffer(commit, commit_buffer);
> +	}
> +	strbuf_release(&buf);
> +
> +	return res;
> +}

Unlike the scripted version, where a merge is shown with "diff --cc"
and a root commit is shown as "Root commit", this only deals with a
single-parent commit.  Is this because this helper, at least in its
current form, will not be used by "rebase -m" and not with "--root"?

If that is the case, that is perfectly fine, perhaps that deserves a
mention in the log message and in-code comment before the function.

Re: [PATCH v3 06/38] sequencer (rebase -i): implement the 'edit' command

[Johannes Schindelin]

Hi Junio,

On Thu, 12 Jan 2017, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > +static int make_patch(struct commit *commit, struct replay_opts *opts)
> > +{
> > +	struct strbuf buf = STRBUF_INIT;
> > +	struct rev_info log_tree_opt;
> > +	const char *subject, *p;
> > +	int res = 0;
> > +
> > +	p = short_commit_name(commit);
> > +	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
> > +		return -1;
> > +
> > +	strbuf_addf(&buf, "%s/patch", get_dir(opts));
> > +	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
> > +	init_revisions(&log_tree_opt, NULL);
> > +	log_tree_opt.abbrev = 0;
> > +	log_tree_opt.diff = 1;
> > +	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
> > +	log_tree_opt.disable_stdin = 1;
> > +	log_tree_opt.no_commit_id = 1;
> > +	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
> > +	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
> > +	if (!log_tree_opt.diffopt.file)
> > +		res |= error_errno(_("could not open '%s'"), buf.buf);
> > +	else {
> > +		res |= log_tree_commit(&log_tree_opt, commit);
> > +		fclose(log_tree_opt.diffopt.file);
> > +	}
> > +	strbuf_reset(&buf);
> > +
> > +	strbuf_addf(&buf, "%s/message", get_dir(opts));
> > +	if (!file_exists(buf.buf)) {
> > +		const char *commit_buffer = get_commit_buffer(commit, NULL);
> > +		find_commit_subject(commit_buffer, &subject);
> > +		res |= write_message(subject, strlen(subject), buf.buf, 1);
> > +		unuse_commit_buffer(commit, commit_buffer);
> > +	}
> > +	strbuf_release(&buf);
> > +
> > +	return res;
> > +}
> 
> Unlike the scripted version, where a merge is shown with "diff --cc"
> and a root commit is shown as "Root commit", this only deals with a
> single-parent commit.

Indeed. The reason is that we never encounter a merge commit (as we
explicitly do not handle --preserve-merges) nor root commits (as we
explicitly do not handle --root)

Ciao,
Johannes

Re: [PATCH v3 01/38] sequencer: avoid unnecessary curly braces

[Johannes Schindelin]

Hi Junio,

On Thu, 12 Jan 2017, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> >  
> > -	if (!commit->parents) {
> > +	if (!commit->parents)
> >  		parent = NULL;
> > -	}
> >  	else if (commit->parents->next) {
> >  		/* Reverting or cherry-picking a merge commit */
> >  		int cnt;
> 
> The result becomes
> 
> 	if (...)
> 		single statement;
> 	else if (...) {
> 		multiple;
>                 statements;
>         }
> 
> which is not quite an improvement.  

Yet, this used to be the coding style of Git, and your statement comes
quite as a surprise to me.

Ciao,
Johannes

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

[Johannes Schindelin]

Hi Junio,

On Mon, 9 Jan 2017, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Changes since v2:
> >
> > - fixed a TRANSLATORS: comment
> > ...
> > - replaced a spawned `diff-tree` command by code using the diff functions
> >   directly
> 
> I just finished skimming the interdiff (the difference between the
> result of merging the v2 into 'master' and the result of applying
> this series on 'master').

I wish you would not have skimmed it, but provided a thorough review.
There was a rather serious bug in this (not the first problem introduced
into one of my patch series *because of* code review, unhidden-git and
mmap-regexec are also very recent examples, I really should learn to
resist the prodding to replace well-tested code with code of unknown
correctness).

The problem in this instance was that the authorship is no longer retained
when continuing after resolving a conflict. Let me stress again that this
has not been a problem with v1 of sequencer-i, nor with v2. The regression
was caused by changes required by the code review.

In case you wonder: Yes, I am upset by this.

The required fixup patch is:

-- snipsnap --
Subject: [PATCH] fixup! sequencer: make reading author-script more elegant

An unfortunate regression of formerly battle-tested code sadly crept
into Git for Windows v2.11.0(2): authorship was not retained in case of
conflicts during picks.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c                   |  2 +-
 t/t3404-rebase-interactive.sh | 16 ++++++++++++++++
 2 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 73b2ec6894..8ecab02291 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -612,7 +612,7 @@ static int read_env_script(struct argv_array *env)
 			count++;
 		}
 
-	for (i = 0; i < count; i++) {
+	for (i = 0, p = script.buf; i < count; i++) {
 		argv_array_push(env, p);
 		p += strlen(p) + 1;
 	}
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 71b9c8ef8b..61113be08a 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -237,6 +237,22 @@ test_expect_success 'retain authorship' '
 	git show HEAD | grep "^Author: Twerp Snog"
 '
 
+test_expect_success 'retain authorship w/ conflicts' '
+	git reset --hard twerp &&
+	test_commit a conflict a conflict-a &&
+	git reset --hard twerp &&
+	GIT_AUTHOR_NAME=AttributeMe \
+	test_commit b conflict b conflict-b &&
+	set_fake_editor &&
+	test_must_fail git rebase -i conflict-a &&
+	echo resolved >conflict &&
+	git add conflict &&
+	git rebase --continue &&
+	test $(git rev-parse conflict-a^0) = $(git rev-parse HEAD^) &&
+	git show >out &&
+	grep AttributeMe out
+'
+
 test_expect_success 'squash' '
 	git reset --hard twerp &&
 	echo B > file7 &&
-- 
2.11.0.windows.3

Re: [PATCH v3 01/38] sequencer: avoid unnecessary curly braces

[Jeff King]

On Sat, Jan 14, 2017 at 06:57:13PM +0100, Johannes Schindelin wrote:

> On Thu, 12 Jan 2017, Junio C Hamano wrote:
> 
> > Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> > 
> > >  
> > > -	if (!commit->parents) {
> > > +	if (!commit->parents)
> > >  		parent = NULL;
> > > -	}
> > >  	else if (commit->parents->next) {
> > >  		/* Reverting or cherry-picking a merge commit */
> > >  		int cnt;
> > 
> > The result becomes
> > 
> > 	if (...)
> > 		single statement;
> > 	else if (...) {
> > 		multiple;
> >                 statements;
> >         }
> > 
> > which is not quite an improvement.  
> 
> Yet, this used to be the coding style of Git, and your statement comes
> quite as a surprise to me.

Yeah, I thought we were OK with:

  if (cond)
	single statement;
  else {
	multiple;
	statements;
  }

but not the other way around:

  if (cond) {
	multiple;
	statements;
  } else
	single statement;

I don't know if the "else if" changes that or not, but I certainly have
written things like your patch does.

-Peff

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

[Johannes Schindelin]

Hi Junio,

On Mon, 9 Jan 2017, Junio C Hamano wrote:

> I however think that the renaming of read_author_script() is totally
> backwards from maintainability's point of view.  

You stated elsewhere that converting a script into a builtin should focus
on a faithful conversion.

The original code is:

	. "$author_script"

Granted, this *cannot* be converted faithfully without reimplementing a
shell interpreter. So I did the next best thing: I converted it into code
that reads a block of environment variable settings.

What you asked for is totally unreasonable: you ask me to make this
conversion *even less faithful*.

What is worse: you argue from the "maintainability point of view", when it
is pretty obvious that *adding validation that was not there before* can
in no way make the code more maintainable, as it *adds new logic*.

And what is the worst: over all these discussions about a nothingburger
(you simply cannot convince me that I should introduce validating code
that has not been there before in the same patch series that simply tries
to recreate existing functionality), the most important part of a code
review was forgotten: to make sure that the changes are correct.

The worst direction a code review can take is to introduce regressions,
and that is exactly what happened.

Ciao,
Johannes

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> The problem in this instance was that the authorship is no longer retained
> when continuing after resolving a conflict. Let me stress again that this
> has not been a problem with v1 of sequencer-i, nor with v2. The regression
> was caused by changes required by the code review.
>
> In case you wonder: Yes, I am upset by this.
>
> -- snipsnap --
> Subject: [PATCH] fixup! sequencer: make reading author-script more elegant

I do not think anybody asked to make the code "more elegant".  Quite
frankly, I do not expect elegance in your code (or any of the code
in our codebase, for that matter).  What we want is readable code
that does not make the overall codebase less maintainable that is
correct.  Not reinveting a new codepath when there is already code
that does the thing is one of the things that we may need to do, but
that was not done between these rerolls.

Of course, when trying to share code, the existing code we have that
the new codepath needs to borrow would have to be refactored and
extended, and a new bug can sneak in during the process.  If that
were what happened, I would be a bit more sympathetic, but I suspect
that this "more elegant" thing that needed fix-up is not that.

You may be upset, but I cannot quite bring myself to feel sympathy
in this particular case.

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> You stated elsewhere that converting a script into a builtin should focus
> on a faithful conversion.
>
> The original code is:
>
> 	. "$author_script"
>
> Granted, this *cannot* be converted faithfully without reimplementing a
> shell interpreter. So I did the next best thing: I converted it into code
> that reads a block of environment variable settings.

It is unfortunate that you took "faithful" too literally.  While I
do appreciate it if your conversion aims to faithfully replicate the
original, even to be bug-to-bug compatible, but obviously we cannot
replicate everything the above original _could_ do.  

By "a faithful conversion", I meant that the behaviour of the
reimplementation should be as faithful to the original's intent as
possible.  Nothing more.

The intent of the above original is to read back what we wrote to
preserve the author identity we learned earlier when we wrote the
file.  We read the "author" line from the commit object header, and
write assignments to GIT_AUTHOR_{NAME,EMAIL,DATE} variables.  Nothing
more is intended.

The end-users COULD abuse the original code to cause it to do a lot
more than that, e.g. by adding "export FOO=BAR" at the end and have
the value of the new environment variable propagate throughout the
code and even down to subprocesses.  They can even assign to some
variables we use internally for bookkeeping and break "rebase -i"
machinery.  But that is outside the intent of the original code--we
do not need to or want to replicate that faithfully.

I also need to react to the "environment variable settings" at the
end of the quoted paragraph.

If the code in the sequencer.c reads things other than the three
variables we ourselves set, and make them into environment variables
and propagate to subprocesses (hooks and editors), it would be a
bug.  The original did not intend to do that (the dot-sourcing is
overly loose than reading three known variables and nothing else,
but is OK because we do not support the case where end users muck
with the file).  Also, writing FOO=BAR alone (not "export FOO=BAR"
or "FOO=BAR; export FOO") to the file wouldn't have exported FOO to
subprocesses anyway.

Re: [PATCH v3 01/38] sequencer: avoid unnecessary curly braces

[Jacob Keller]

On Sat, Jan 14, 2017 at 10:05 AM, Jeff King <peff@peff.net> wrote:
> On Sat, Jan 14, 2017 at 06:57:13PM +0100, Johannes Schindelin wrote:
>
>> On Thu, 12 Jan 2017, Junio C Hamano wrote:
>>
>> > Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> >
>> > >
>> > > - if (!commit->parents) {
>> > > + if (!commit->parents)
>> > >           parent = NULL;
>> > > - }
>> > >   else if (commit->parents->next) {
>> > >           /* Reverting or cherry-picking a merge commit */
>> > >           int cnt;
>> >
>> > The result becomes
>> >
>> >     if (...)
>> >             single statement;
>> >     else if (...) {
>> >             multiple;
>> >                 statements;
>> >         }
>> >
>> > which is not quite an improvement.
>>
>> Yet, this used to be the coding style of Git, and your statement comes
>> quite as a surprise to me.
>
> Yeah, I thought we were OK with:
>
>   if (cond)
>         single statement;
>   else {
>         multiple;
>         statements;
>   }
>
> but not the other way around:
>
>   if (cond) {
>         multiple;
>         statements;
>   } else
>         single statement;
>
> I don't know if the "else if" changes that or not, but I certainly have
> written things like your patch does.
>
> -Peff

Personally, I am of the faith that if any branch of the
if-then-else-if-then-else blocks needs braces, then all branches
should use braces. However, I think that

if (condition)
  <line>
else {
  <block>
}

is reasonably close to this, as the main part can still clearly see
the if condition pretty well.

Thanks,
Jake

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

[Johannes Schindelin]

Hi Junio,

On Sat, 14 Jan 2017, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > You stated elsewhere that converting a script into a builtin should focus
> > on a faithful conversion.
> >
> > The original code is:
> >
> > 	. "$author_script"
> 
> [...]
>
> If the code in the sequencer.c reads things other than the three
> variables we ourselves set, and make them into environment variables
> and propagate to subprocesses (hooks and editors), it would be a
> bug.  The original did not intend to do that (the dot-sourcing is
> overly loose than reading three known variables and nothing else,
> but is OK because we do not support the case where end users muck
> with the file).  Also, writing FOO=BAR alone (not "export FOO=BAR"
> or "FOO=BAR; export FOO") to the file wouldn't have exported FOO to
> subprocesses anyway.

That analysis cannot be completely correct, as the GIT_AUTHOR_* variables
*are* used by the `git commit` subprocess.

In any case, it is clear that we (I include all reviewers here) messed
this patch series up quite a bit. Hence I will be more careful from now on
to only act on suggestions that do, in fact, improve the patch series from
my point of view.

Ciao,
Johannes

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> > The original code is:
>> >
>> > 	. "$author_script"
>> 
>> [...]
>>
>> If the code in the sequencer.c reads things other than the three
>> variables we ourselves set, and make them into environment variables
>> and propagate to subprocesses (hooks and editors), it would be a
>> bug.  The original did not intend to do that (the dot-sourcing is
>> overly loose than reading three known variables and nothing else,
>> but is OK because we do not support the case where end users muck
>> with the file).  Also, writing FOO=BAR alone (not "export FOO=BAR"
>> or "FOO=BAR; export FOO") to the file wouldn't have exported FOO to
>> subprocesses anyway.
>
> That analysis cannot be completely correct, as the GIT_AUTHOR_* variables
> *are* used by the `git commit` subprocess.

In the scripted version, do_with_author shell function explicitly
exports GIT_AUTHOR_* variables because they are the only ones we
care about that are read from existing commit and carried forward
via the author-script mechanism.  We do not care about, and in fact
we do not intend to support, any other variables or shell commands
that appear in the author-script.