[PATCH 0/6] rebase: command "ref" and options --rewrite-{refs,heads,tags}

[PATCH 0/6] rebase: command "ref" and options --rewrite-{refs,heads,tags}

[Greg Price]

This series adds a "ref" command to rebase -i, and options
--rewrite-{refs,heads,tags} to generate "ref" commands in the original
TODO file.  This makes it easy to rebase a branch together with
intermediate markers, or a series of branches.

For example, in this situation:

      part1 part2 topic
        |     |     |
        v     v     v
  A--*--*--*--*--*--*
   \
    B <--master

we may want to rebase 'topic' onto the new master, rewriting the
intermediate branches 'part1', 'part2' to the corresponding new
commits.  This can be done with a sequence of "git rebase --onto"
commands, but it's tricky to get right.

With this series, the command
  $ git rebase --rewrite-heads master topic
suffices to produce this result:

        part1 part2 topic
  A       |     |     |
   \      v     v     v
    B--*--*--*--*--*--*
    ^
    |
    master


I originally submitted a version of this series nearly a year and a
half ago [1].  This version has been rebased to the new release 1.7.6
(Martin von Zweigbergk's cleanups gave me quite some conflicts to
resolve, but are nevertheless much appreciated!) and the interface has
been adjusted for the comments of Junio and others [2].  The general
command is --rewrite-refs=PATTERN; --rewrite-heads and --rewrite-tags
are provided as shortcuts, the former being equivalent to the --refs
of my original proposal.

 [1]  http://www.amailbox.net/mailarchive/git/2009/12/22/18967
 [2]  http://www.amailbox.net/mailarchive/git/2009/12/22/18972

I have no excuse for the year and a half of delay in submitting the
revised version, but I believe that late is better than never. =)


The series contains two preparatory patches that are independently useful:
  pretty: Add %D for script-friendly decoration
  for-each-ref: --stdin to match specified refs against pattern
Each one exposes a bit more knowledge of the repository in a
machine-readable form convenient for scripting.  I hope other people
may find them useful in writing other scripts.

Greg


---
Greg Price (6):
  rebase -i: Add the "ref" command
  pretty: Add %D for script-friendly decoration
  for-each-ref: --stdin to match specified refs against pattern
  rebase: --rewrite-{refs,heads,tags} to pull refs along with branch
  t/lib-rebase.sh: pass through ref commands
  rebase --rewrite-refs: tests

 Documentation/git-rebase.txt     |   15 ++++++++
 Documentation/pretty-formats.txt |    1 +
 builtin/for-each-ref.c           |   22 ++++++++++-
 git-rebase--interactive.sh       |   37 ++++++++++++++++++-
 git-rebase.sh                    |   28 ++++++++++++++
 pretty.c                         |   33 +++++++++++++----
 t/lib-rebase.sh                  |    2 +-
 t/t3420-rebase-ref.sh            |   75 ++++++++++++++++++++++++++++++++++++++
 8 files changed, 201 insertions(+), 12 deletions(-)
 create mode 100644 t/t3420-rebase-ref.sh

-- 
1.7.5.4

[PATCH 1/6] rebase -i: Add the "ref" command

[Greg Price]

This is useful for, e.g., rewriting a branch that has ancestor
branches along the way.

Signed-off-by: Greg Price <price@mit.edu>
---
 Documentation/git-rebase.txt |    4 ++++
 git-rebase--interactive.sh   |   12 ++++++++++++
 git-rebase.sh                |   11 +++++++++++
 3 files changed, 27 insertions(+), 0 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 9a075bc..74fda58 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -455,6 +455,10 @@ message for the folded commit is the concatenation of the commit
 messages of the first commit and of those with the "squash" command,
 but omits the commit messages of commits with the "fixup" command.
 
+If you want to update a ref to point to a rewritten commit, add a
+command "ref <refname>" after the "pick", "edit", or other command
+that produces the commit.
+
 'git rebase' will stop when "pick" has been replaced with "edit" or
 when a command fails due to merge errors. When you are done editing
 and/or resolving conflicts you can continue with `git rebase --continue`.
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 65690af..cec9cab 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -418,6 +418,17 @@ do_next () {
 		warn
 		exit 0
 		;;
+	ref)
+		mark_action_done
+		refname=$sha1
+		sha1=$(git rev-parse --quiet --verify "$refname" \
+			|| echo "(null)")
+		if ! grep -Fq " $refname" "$state_dir"/oldrefs 2>/dev/null
+		then
+			echo "$sha1 $refname" >> "$state_dir"/oldrefs
+		fi
+		git update-ref $refname HEAD
+		;;
 	squash|s|fixup|f)
 		case "$command" in
 		squash|s)
@@ -801,6 +812,7 @@ cat >> "$todo" << EOF
 #  s, squash = use commit, but meld into previous commit
 #  f, fixup = like "squash", but discard this commit's log message
 #  x, exec = run command (the rest of the line) using shell
+#  ref <refname> = update ref
 #
 # If you remove a line here THAT COMMIT WILL BE LOST.
 # However, if you remove everything, the rebase will be aborted.
diff --git a/git-rebase.sh b/git-rebase.sh
index d7855ea..1bfe6a8 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -118,6 +118,8 @@ read_basic_state () {
 		strategy_opts="$(cat "$state_dir"/strategy_opts)"
 	test -f "$state_dir"/allow_rerere_autoupdate &&
 		allow_rerere_autoupdate="$(cat "$state_dir"/allow_rerere_autoupdate)"
+	test -f "$state_dir"/oldrefs &&
+		oldrefs="$(cat "$state_dir"/oldrefs)"
 }
 
 write_basic_state () {
@@ -332,6 +334,15 @@ skip)
 abort)
 	git rerere clear
 	read_basic_state
+	[ -n "$oldrefs" ] && echo "$oldrefs" | while read sha1 ref
+	do
+		if test "(null)" = $sha1
+		then
+			git update-ref -d "$ref"
+		else
+			git update-ref "$ref" $sha1
+		fi
+	done
 	case "$head_name" in
 	refs/*)
 		git symbolic-ref -m "rebase: aborting" HEAD $head_name ||
-- 
1.7.5.4

Re: [PATCH 1/6] rebase -i: Add the "ref" command

[Junio C Hamano]

Greg Price <price@MIT.EDU> writes:

> ...
> +		if ! grep -Fq " $refname" "$state_dir"/oldrefs 2>/dev/null
> +		then
> +			echo "$sha1 $refname" >> "$state_dir"/oldrefs

(Style) Extra SP between ">>" and "$state_dir/oldrefs"

> diff --git a/git-rebase.sh b/git-rebase.sh
> index d7855ea..1bfe6a8 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -118,6 +118,8 @@ read_basic_state () {
>  		strategy_opts="$(cat "$state_dir"/strategy_opts)"
>  	test -f "$state_dir"/allow_rerere_autoupdate &&
>  		allow_rerere_autoupdate="$(cat "$state_dir"/allow_rerere_autoupdate)"
> +	test -f "$state_dir"/oldrefs &&
> +		oldrefs="$(cat "$state_dir"/oldrefs)"
>  }
>  
>  write_basic_state () {
> @@ -332,6 +334,15 @@ skip)
>  abort)
>  	git rerere clear
>  	read_basic_state
> +	[ -n "$oldrefs" ] && echo "$oldrefs" | while read sha1 ref

(Style) I think almost everybody else spells out "test".  Also please
break line before the while, like this:

	test -n "$oldrefs" &&
	echo "$oldrefs" |
	while read sha1 ref
        do
        	...

> +	do
> +		if test "(null)" = $sha1

Who is giving you "(null)"???

Re: [PATCH 1/6] rebase -i: Add the "ref" command

[Greg Price]

Thanks for the review.

On Mon, Jun 27, 2011 at 11:46:52AM -0700, Junio C Hamano wrote:
> Greg Price <price@MIT.EDU> writes:
> 
> > ...
> > +		if ! grep -Fq " $refname" "$state_dir"/oldrefs 2>/dev/null
> > +		then
> > +			echo "$sha1 $refname" >> "$state_dir"/oldrefs
> 
> (Style) Extra SP between ">>" and "$state_dir/oldrefs"

Hmm -- it looks like the prevalent style in the codebase is actually
to include the space:

greg@gouda:~/w/git$ git grep -c '>>[^ ]' v1.7.6 git-*.sh
v1.7.6:git-bisect.sh:3
v1.7.6:git-instaweb.sh:2
v1.7.6:git-rebase--interactive.sh:2
v1.7.6:git-stash.sh:1
greg@gouda:~/w/git$ git grep -c '>> ' v1.7.6 git-*.sh
v1.7.6:git-am.sh:1
v1.7.6:git-filter-branch.sh:1
v1.7.6:git-instaweb.sh:8
v1.7.6:git-rebase--interactive.sh:10
v1.7.6:git-rebase--merge.sh:1

and in particular in git-rebase--interactive.sh.  But I could do it
either way.



> > @@ -332,6 +334,15 @@ skip)
> >  abort)
> >  	git rerere clear
> >  	read_basic_state
> > +	[ -n "$oldrefs" ] && echo "$oldrefs" | while read sha1 ref
> 
> (Style) I think almost everybody else spells out "test".  Also please
> break line before the while, like this:
> 
> 	test -n "$oldrefs" &&
> 	echo "$oldrefs" |
> 	while read sha1 ref
>         do
>         	...

Sure, done.



> > +	do
> > +		if test "(null)" = $sha1
> 
> Who is giving you "(null)"???

I am, myself -- it's what the 'ref' implementation in
git-rebase--interactive.sh uses to indicate that a ref had not existed
and should be deleted on abort.

+       ref)
+               mark_action_done
+               refname=$sha1
+               sha1=$(git rev-parse --quiet --verify "$refname" \
+                       || echo "(null)")
+               if ! grep -Fq " $refname" "$state_dir"/oldrefs 2>/dev/null
+               then
+                       echo "$sha1 $refname" >> "$state_dir"/oldrefs
+               fi

I could change it to something like "-".  It needs to be something
that the 'read' builtin, as used at the top of the loop, treats as a
word.

Greg

[PATCH 2/6] pretty: Add %D for script-friendly decoration

[Greg Price]

When in a script or porcelain one wants to identify what refs point to
which commits in a series, the functionality of 'git log --decorate'
is extremely useful.  This is available with the %d format code in a
form optimized for humans, but for scripts a more raw format is better.
Make such a format available through a new format code %D.

Signed-off-by: Greg Price <price@mit.edu>
---
 Documentation/pretty-formats.txt |    1 +
 pretty.c                         |   33 +++++++++++++++++++++++++--------
 2 files changed, 26 insertions(+), 8 deletions(-)

diff --git a/Documentation/pretty-formats.txt b/Documentation/pretty-formats.txt
index 561cc9f..217f11b 100644
--- a/Documentation/pretty-formats.txt
+++ b/Documentation/pretty-formats.txt
@@ -124,6 +124,7 @@ The placeholders are:
 - '%ct': committer date, UNIX timestamp
 - '%ci': committer date, ISO 8601 format
 - '%d': ref names, like the --decorate option of linkgit:git-log[1]
+- '%D': full ref names, like the --decorate=full option of linkgit:git-log[1]
 - '%e': encoding
 - '%s': subject
 - '%f': sanitized subject line, suitable for a filename
diff --git a/pretty.c b/pretty.c
index f45eb54..0c7e723 100644
--- a/pretty.c
+++ b/pretty.c
@@ -776,21 +776,35 @@ static void parse_commit_message(struct format_commit_context *c)
 	c->commit_message_parsed = 1;
 }
 
-static void format_decoration(struct strbuf *sb, const struct commit *commit)
+
+static void format_decoration(struct strbuf *sb, const struct commit *commit,
+			      int decoration_style, const char *affixes[3])
 {
 	struct name_decoration *d;
-	const char *prefix = " (";
+	const char *affix = affixes[0];
 
-	load_ref_decorations(DECORATE_SHORT_REFS);
+	load_ref_decorations(decoration_style);
 	d = lookup_decoration(&name_decoration, &commit->object);
 	while (d) {
-		strbuf_addstr(sb, prefix);
-		prefix = ", ";
+		strbuf_addstr(sb, affix);
+		affix = affixes[1];
 		strbuf_addstr(sb, d->name);
 		d = d->next;
 	}
-	if (prefix[0] == ',')
-		strbuf_addch(sb, ')');
+	if (affix == affixes[1])
+		strbuf_addstr(sb, affixes[2]);
+}
+
+static void format_decoration_short(struct strbuf *sb, const struct commit *commit)
+{
+	const char *affixes[3] = {" (", ", ", ")"};
+	format_decoration(sb, commit, DECORATE_SHORT_REFS, affixes);
+}
+
+static void format_decoration_full(struct strbuf *sb, const struct commit *commit)
+{
+	const char *affixes[3] = {"", " ", ""};
+	format_decoration(sb, commit, DECORATE_FULL_REFS, affixes);
 }
 
 static void strbuf_wrap(struct strbuf *sb, size_t pos,
@@ -947,7 +961,10 @@ static size_t format_commit_one(struct strbuf *sb, const char *placeholder,
 		strbuf_addstr(sb, get_revision_mark(NULL, commit));
 		return 1;
 	case 'd':
-		format_decoration(sb, commit);
+		format_decoration_short(sb, commit);
+		return 1;
+	case 'D':
+		format_decoration_full(sb, commit);
 		return 1;
 	case 'g':		/* reflog info */
 		switch(placeholder[1]) {
-- 
1.7.5.4

Re: [PATCH 2/6] pretty: Add %D for script-friendly decoration

[Junio C Hamano]

Greg Price <price@MIT.EDU> writes:

> +static void format_decoration_short(struct strbuf *sb, const struct commit *commit)
> +{
> +	const char *affixes[3] = {" (", ", ", ")"};
> +	format_decoration(sb, commit, DECORATE_SHORT_REFS, affixes);
> +}
> +
> +static void format_decoration_full(struct strbuf *sb, const struct commit *commit)
> +{
> +	const char *affixes[3] = {"", " ", ""};
> +	format_decoration(sb, commit, DECORATE_FULL_REFS, affixes);
>  }

Nice and cute abstraction. I like this part of the series.

[PATCH 4/6] rebase: --rewrite-{refs,heads,tags} to pull refs along with branch

[Greg Price]

The new option --rewrite-refs causes the TODO file to contain a "ref"
command for each appropriate ref pointing to a selected commit, other
than the one we are already rebasing.  The argument to --rewrite-refs
is a ref pattern of the same type accepted by for-each-ref: a pattern
matches a ref name if it matches exactly, matches exactly up to a
slash, or matches according to fnmatch(3).  The options
--rewrite-heads and --rewrite-tags are supplied as shortcuts.

The effect of this is that when a branch contains intermediate
branches, like so:

      part1 part2 topic
        |     |     |
        v     v     v
  A--*--*--*--*--*--*
   \
    B <--master

a single command like "git rebase --rewrite-heads master topic"
suffices to rewrite all the heads that are part of the topic, like so:

        part1 part2 topic
  A       |     |     |
   \      v     v     v
    B--*--*--*--*--*--*
    ^
    |
    master

Signed-off-by: Greg Price <price@mit.edu>
---
 Documentation/git-rebase.txt |   13 ++++++++++++-
 git-rebase--interactive.sh   |   25 ++++++++++++++++++++++++-
 git-rebase.sh                |   17 +++++++++++++++++
 3 files changed, 53 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 74fda58..e4f32fc 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -347,6 +347,15 @@ idea unless you know what you are doing (see BUGS below).
 	root commits will be rewritten to have <newbase> as parent
 	instead.
 
+--rewrite-refs=<pattern>::
+--rewrite-heads::
+--rewrite-tags::
+	Rewrite refs matching <pattern> which point to the rebased
+	commits.  The options --rewrite-heads and --rewrite-tags are
+	shortcuts for --rewrite-refs=refs/heads and
+	--rewrite-refs=refs/tags respectively.  Ref patterns are
+	interpreted as in linkgit:git-for-each-ref[1].
+
 --autosquash::
 --no-autosquash::
 	When the commit log message begins with "squash! ..." (or
@@ -457,7 +466,9 @@ but omits the commit messages of commits with the "fixup" command.
 
 If you want to update a ref to point to a rewritten commit, add a
 command "ref <refname>" after the "pick", "edit", or other command
-that produces the commit.
+that produces the commit.  You can use the --rewrite-refs option to
+have the file start out with these commands for refs inside the branch
+you are rebasing.
 
 'git rebase' will stop when "pick" has been replaced with "edit" or
 when a command fails due to merge errors. When you are done editing
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index cec9cab..42ea3e7 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -741,12 +741,23 @@ else
 	revisions=$onto...$orig_head
 	shortrevisions=$shorthead
 fi
-git rev-list $merges_option --pretty=oneline --abbrev-commit \
+if test -z "rewrite_refs"
+then
+	pretty=oneline
+else
+	pretty=format:"%m%h %s%n%m%D"
+fi
+git rev-list $merges_option --pretty="$pretty" --abbrev-commit \
 	--abbrev=7 --reverse --left-right --topo-order \
 	$revisions | \
 	sed -n "s/^>//p" |
 while read -r shortsha1 rest
 do
+	if test -n "$rewrite_refs"
+	then
+		read refs
+	fi
+
 	if test t != "$preserve_merges"
 	then
 		printf '%s\n' "pick $shortsha1 $rest" >> "$todo"
@@ -771,6 +782,18 @@ do
 			printf '%s\n' "pick $shortsha1 $rest" >> "$todo"
 		fi
 	fi
+
+	if test -n "$rewrite_refs"
+	then
+		for ref in $refs; do echo "$ref"; done | \
+		git for-each-ref --stdin $rewrite_refs \
+			--format '%(refname)' | \
+		while read ref
+		do
+			test "$ref" != "$head_name" &&
+			echo "ref $ref" >> "$todo"
+		done
+	fi
 done
 
 # Watch for commits that been dropped by --cherry-pick
diff --git a/git-rebase.sh b/git-rebase.sh
index 1bfe6a8..7c365ab 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -38,6 +38,9 @@ git-rebase [-i] --continue | --abort | --skip
 v,verbose!         display a diffstat of what changed upstream
 q,quiet!           be quiet. implies --no-stat
 onto=!             rebase onto given branch instead of upstream
+rewrite-heads!     rewrite intermediate heads on branch
+rewrite-tags!      rewrite intermediate tags on branch
+rewrite-refs=!     rewrite intermediate refs matching pattern
 p,preserve-merges! try to recreate merges instead of ignoring them
 s,strategy=!       use the given merge strategy
 no-ff!             cherry-pick all commits, even if unchanged
@@ -96,6 +99,7 @@ state_dir=
 # One of {'', continue, skip, abort}, as parsed from command line
 action=
 preserve_merges=
+rebase_refs=
 autosquash=
 test "$(git config --bool rebase.autosquash)" = "true" && autosquash=t
 
@@ -246,6 +250,19 @@ do
 		strategy="$1"
 		do_merge=t
 		;;
+	--rewrite-refs)
+		shift
+		rewrite_refs="$rewrite_refs $1"
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
+	--rewrite-heads)
+		rewrite_refs="$rewrite_refs refs/heads"
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
+	--rewrite-tags)
+		rewrite_refs="$rewrite_refs refs/tags"
+		test -z "$interactive_rebase" && interactive_rebase=implied
+		;;
 	-n)
 		diffstat=
 		;;
-- 
1.7.5.4

Re: [PATCH 4/6] rebase: --rewrite-{refs,heads,tags} to pull refs along with branch

[Phil Hord]

On 11/18/2009 06:51 PM, Greg Price wrote:
> diff --git a/git-rebase.sh b/git-rebase.sh
> index 1bfe6a8..7c365ab 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -38,6 +38,9 @@ git-rebase [-i] --continue | --abort | --skip
>  v,verbose!         display a diffstat of what changed upstream
>  q,quiet!           be quiet. implies --no-stat
>  onto=!             rebase onto given branch instead of upstream
> +rewrite-heads!     rewrite intermediate heads on branch
> +rewrite-tags!      rewrite intermediate tags on branch
> +rewrite-refs=!     rewrite intermediate refs matching pattern
>  p,preserve-merges! try to recreate merges instead of ignoring them
>  s,strategy=!       use the given merge strategy
>  no-ff!             cherry-pick all commits, even if unchanged
> @@ -96,6 +99,7 @@ state_dir=
>  # One of {'', continue, skip, abort}, as parsed from command line
>  action=
>  preserve_merges=
> +rebase_refs=
>  autosquash=
>  test "$(git config --bool rebase.autosquash)" = "true" && autosquash=t
I think you meant to spell that "rewrite_refs=" instead of "rebase_refs=".

>  
> @@ -246,6 +250,19 @@ do
>  		strategy="$1"
>  		do_merge=t
>  		;;
> +	--rewrite-refs)
> +		shift
> +		rewrite_refs="$rewrite_refs $1"
> +		test -z "$interactive_rebase" && interactive_rebase=implied
> +		;;
> +	--rewrite-heads)
> +		rewrite_refs="$rewrite_refs refs/heads"
> +		test -z "$interactive_rebase" && interactive_rebase=implied
> +		;;
> +	--rewrite-tags)
> +		rewrite_refs="$rewrite_refs refs/tags"
> +		test -z "$interactive_rebase" && interactive_rebase=implied
> +		;;
>  	-n)
>  		diffstat=
>  		;;

Then it matches the rest of the commit.

Phil

Re: [PATCH 4/6] rebase: --rewrite-{refs,heads,tags} to pull refs along with branch

[Greg Price]

On Mon, Jun 27, 2011 at 12:11:39PM -0400, Phil Hord wrote:
> On 11/18/2009 06:51 PM, Greg Price wrote:
> > @@ -96,6 +99,7 @@ state_dir=
> >  # One of {'', continue, skip, abort}, as parsed from command line
> >  action=
> >  preserve_merges=
> > +rebase_refs=
> >  autosquash=
> >  test "$(git config --bool rebase.autosquash)" = "true" && autosquash=t
> I think you meant to spell that "rewrite_refs=" instead of "rebase_refs=".

Ha, so I did.  Fixed.  Thanks!

Greg

[PATCH 3/6] for-each-ref: --stdin to match specified refs against pattern

[Greg Price]

The pattern syntax of for-each-ref is handy for users, but a bit subtle:
a pattern matches a ref if it matches as an fnmatch pattern, or literally,
or literally against a prefix of the refname up to a slash.  Expose the
for-each-ref pattern-matcher for scripts to use, so that they can provide
a consistent user interface without duplicating the implementation.

Signed-off-by: Greg Price <price@mit.edu>
---
 builtin/for-each-ref.c |   22 ++++++++++++++++++++--
 1 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c
index 89e75c6..0413ec0 100644
--- a/builtin/for-each-ref.c
+++ b/builtin/for-each-ref.c
@@ -779,6 +779,20 @@ static int grab_single_ref(const char *refname, const unsigned char *sha1, int f
 	return 0;
 }
 
+static void for_each_ref_stdin(each_ref_fn fn, void *cb_data)
+{
+	struct strbuf buf;
+	unsigned char sha1[20];
+
+	strbuf_init(&buf, 0);
+	while (strbuf_getline(&buf, stdin, '\n') != EOF) {
+		if (read_ref(buf.buf, sha1))
+			continue;
+		fn(buf.buf, sha1, 0, cb_data);
+	}
+	strbuf_release(&buf);
+}
+
 static int cmp_ref_sort(struct ref_sort *s, struct refinfo *a, struct refinfo *b)
 {
 	struct atom_value *va, *vb;
@@ -943,7 +957,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix)
 	int i, num_refs;
 	const char *format = "%(objectname) %(objecttype)\t%(refname)";
 	struct ref_sort *sort = NULL, **sort_tail = &sort;
-	int maxcount = 0, quote_style = 0;
+	int maxcount = 0, quote_style = 0, stdin_refs = 0;
 	struct refinfo **refs;
 	struct grab_ref_cbdata cbdata;
 
@@ -962,6 +976,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix)
 		OPT_STRING(  0 , "format", &format, "format", "format to use for the output"),
 		OPT_CALLBACK(0 , "sort", sort_tail, "key",
 		            "field name to sort on", &opt_parse_sort),
+		OPT_BOOLEAN( 0 , "stdin", &stdin_refs, "read candidate refs from stdin, rather than all refs"),
 		OPT_END(),
 	};
 
@@ -986,7 +1001,10 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix)
 
 	memset(&cbdata, 0, sizeof(cbdata));
 	cbdata.grab_pattern = argv;
-	for_each_rawref(grab_single_ref, &cbdata);
+	if (stdin_refs)
+		for_each_ref_stdin(grab_single_ref, &cbdata);
+	else
+		for_each_rawref(grab_single_ref, &cbdata);
 	refs = cbdata.grab_array;
 	num_refs = cbdata.grab_cnt;
 
-- 
1.7.5.4

[PATCH 5/6] t/lib-rebase.sh: pass through ref commands

[Greg Price]

Signed-off-by: Greg Price <price@mit.edu>
---
 t/lib-rebase.sh |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/t/lib-rebase.sh b/t/lib-rebase.sh
index 6ccf797..656e4b6 100644
--- a/t/lib-rebase.sh
+++ b/t/lib-rebase.sh
@@ -54,7 +54,7 @@ for line in $FAKE_LINES; do
 	">")
 		echo >> "$1";;
 	*)
-		sed -n "${line}s/^pick/$action/p" < "$1".tmp >> "$1"
+		sed -n "${line}{s/^pick/$action/p; /^ref/p}" < "$1".tmp >> "$1"
 		action=pick;;
 	esac
 done
-- 
1.7.5.4

[PATCH 6/6] rebase --rewrite-refs: tests

[Greg Price]

Signed-off-by: Greg Price <price@mit.edu>
---
 t/t3420-rebase-ref.sh |   75 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 75 insertions(+), 0 deletions(-)
 create mode 100644 t/t3420-rebase-ref.sh

diff --git a/t/t3420-rebase-ref.sh b/t/t3420-rebase-ref.sh
new file mode 100644
index 0000000..601a434
--- /dev/null
+++ b/t/t3420-rebase-ref.sh
@@ -0,0 +1,75 @@
+#!/bin/sh
+
+test_description='git rebase --rewrite-refs'
+
+. ./test-lib.sh
+. "$TEST_DIRECTORY/lib-rebase.sh"
+set_fake_editor
+
+#      part1  part2 topic
+#	 |	|     |
+#	 v	v     v
+#  A-----C------D-----E
+#   \
+#    B <--master
+
+test_expect_success setup '
+	test_commit A &&
+	git branch topic &&
+	test_commit B &&
+	git checkout topic &&
+	test_commit C &&
+	git branch part1 &&
+	test_commit D
+	git branch part2 &&
+	test_commit E
+'
+
+test_expect_success 'rebase --rewrite-heads' '
+	git reset --hard &&
+	git checkout topic &&
+	git reset --hard E &&
+
+	git rebase --rewrite-heads master &&
+	git rev-parse part1 >actual &&
+	git rev-parse HEAD~2 >expected &&
+	test_cmp expected actual &&
+	git rev-parse part2 >actual &&
+	git rev-parse HEAD~1 >expected &&
+	test_cmp expected actual
+'
+
+test_expect_success 'rebase --rewrite-refs' '
+	git reset --hard &&
+	git update-ref refs/heads/part1 C &&
+	git update-ref refs/heads/part2 D &&
+	git checkout topic &&
+	git reset --hard E &&
+
+	git rebase --rewrite-refs=refs/heads/part2 master &&
+	git rev-parse part1 >actual &&
+	git rev-parse C >expected &&
+	test_cmp expected actual &&
+	git rev-parse part2 >actual &&
+	git rev-parse HEAD~1 >expected &&
+	test_cmp expected actual
+'
+
+test_expect_success 'ref in TODO followed by rebase --abort' '
+	git reset --hard &&
+	git update-ref refs/heads/part1 C &&
+	git update-ref refs/heads/part2 D &&
+	git checkout topic &&
+	git reset --hard E &&
+
+	FAKE_LINES="1 2 edit 3 4 5" git rebase -i --rewrite-heads master &&
+	git rev-parse part1 >actual &&
+	git rev-parse HEAD^ >expected &&
+	test_cmp expected actual &&
+	git rebase --abort &&
+	git rev-parse part1 >actual &&
+	git rev-parse C >expected &&
+	test_cmp expected actual
+'
+
+test_done
-- 
1.7.5.4

Re: [PATCH 6/6] rebase --rewrite-refs: tests

[Phil Hord]

On 01/24/2010 09:28 PM, Greg Price wrote:
> Signed-off-by: Greg Price <price@mit.edu>
> ---
>  t/t3420-rebase-ref.sh |   75 +++++++++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 75 insertions(+), 0 deletions(-)
>  create mode 100644 t/t3420-rebase-ref.sh
>
> diff --git a/t/t3420-rebase-ref.sh b/t/t3420-rebase-ref.sh
> new file mode 100644
> index 0000000..601a434
> --- /dev/null
> +++ b/t/t3420-rebase-ref.sh
> @@ -0,0 +1,75 @@
> +#!/bin/sh
> +
> +test_description='git rebase --rewrite-refs'
> +
> +. ./test-lib.sh
> +. "$TEST_DIRECTORY/lib-rebase.sh"
> +set_fake_editor
> +
> +#      part1  part2 topic
> +#	 |	|     |
> +#	 v	v     v
> +#  A-----C------D-----E
> +#   \
> +#    B <--master
> +
> +test_expect_success setup '
> +	test_commit A &&
> +	git branch topic &&
> +	test_commit B &&
> +	git checkout topic &&
> +	test_commit C &&
> +	git branch part1 &&
> +	test_commit D
> +	git branch part2 &&
> +	test_commit E
> +'

I think there is a missing "&&" at the end of "test_commit D".  But it's
possible I just don't understand the test machinery enough to know this
is normal.  Can you explain to me the difference in that case?

Phil

Re: [PATCH 6/6] rebase --rewrite-refs: tests

[Greg Price]

On Mon, Jun 27, 2011 at 12:07:10PM -0400, Phil Hord wrote:
> On 01/24/2010 09:28 PM, Greg Price wrote:
> > +test_expect_success setup '
> > +	test_commit A &&
> > +	git branch topic &&
> > +	test_commit B &&
> > +	git checkout topic &&
> > +	test_commit C &&
> > +	git branch part1 &&
> > +	test_commit D
> > +	git branch part2 &&
> > +	test_commit E
> > +'
> 
> I think there is a missing "&&" at the end of "test_commit D".  But it's
> possible I just don't understand the test machinery enough to know this
> is normal.  Can you explain to me the difference in that case?

Nope, that's just a mistake -- it means that if "test_commit D" were
to somehow fail, we'd move on as if nothing was wrong.  Fixed, thanks.

Greg

Re: [PATCH 6/6] rebase --rewrite-refs: tests

[Junio C Hamano]

Greg Price <price@MIT.EDU> writes:

> diff --git a/t/t3420-rebase-ref.sh b/t/t3420-rebase-ref.sh
> new file mode 100644

Forgot the executable bit?

Re: [PATCH 6/6] rebase --rewrite-refs: tests

[Greg Price]

On Mon, Jun 27, 2011 at 02:02:36PM -0700, Junio C Hamano wrote:
> Greg Price <price@MIT.EDU> writes:
> 
> > diff --git a/t/t3420-rebase-ref.sh b/t/t3420-rebase-ref.sh
> > new file mode 100644
> 
> Forgot the executable bit?

Yup, thanks, fixed.

Greg

Re: [PATCH 0/6] rebase: command "ref" and options --rewrite-{refs,heads,tags}

[Junio C Hamano]

Greg Price <greg@quora.com> writes:

> With this series, the command
>   $ git rebase --rewrite-heads master topic
> suffices to produce this result:
>
>         part1 part2 topic
>   A       |     |     |
>    \      v     v     v
>     B--*--*--*--*--*--*
>     ^
>     |
>     master

Thanks for a re-roll.

I notice that this does not address the "side branch" issue raised during
the original discussion. I do agree with Michael

  http://thread.gmane.org/gmane.comp.version-control.git/135601/focus=135617

that having some commits on these part$N side branches is far more common
use case that would benefit from a "rewrite together" feature like this,
than moving part$N side branches that just mark points in the topic
without doing anything on their own and makes me doubt if doing only the
parts that can sanely done within the limitation of the current rebase-i
implementation like this series does adds much value to the system [*1*].

It would be nice to have a clear definition of what _should_ happen in
this case, and a test that makes sure that that is the behaviour we get.

Starting from this topology

          1   2   topic
  A---X---Y---Z---W
   \
    B

where the change going from A to B is an equilvalent to the change going
from Y to Z, a rebase of A..W would reproduce this topology:


          1   2   topic
  A---X---Y---Z---W
   \
    B---X'--Y'--W'
	    1'  topic'

What should heppen to ref2? Should it be deleted? Should it point at Y'?


[Footnote]

*1* I suspect that dealing with side branches would require a much richer
implementation of the sequencer machinery that lets you go back to a
previous state, which we do not have right now.

While I think that it makes your series much less interesting than the
series could be that not being able to rewrite side branches, I do not
think it is reasonable to expect it be done within the current rebase-i
implementation/limitation.

With a richer sequencer, when you want to rebuild 'topic' along with 'side'
in this picture:

            D side
           /
  A---B---C---E---F topic
   \
    X

on top of X, I would imagine that your rebase-i insn sheet would say
something like this:

    detach at X
    replay B
    replay C
    replay E
    replay F
    update ref "topic" with HEAD
    detach at the rewritten C
    replay D
    update ref "side" with HEAD

Re: Fwd: [PATCH 0/6] rebase: command "ref" and options --rewrite-{refs,heads,tags}

[Greg Price]

On Mon, Jun 27, 2011 at 11:36 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Thanks for a re-roll.
> 
> I notice that this does not address the "side branch" issue raised during
> the original discussion. I do agree with Michael
> 
>  http://thread.gmane.org/gmane.comp.version-control.git/135601/focus=135617
> 
> that having some commits on these part$N side branches is far more common
> use case that would benefit from a "rewrite together" feature like this,
> than moving part$N side branches that just mark points in the topic
> without doing anything on their own and makes me doubt if doing only the
> parts that can sanely done within the limitation of the current rebase-i
> implementation like this series does adds much value to the system [*1*].

I would also like to support that use case.  In my personal
experience, the case where the part$N are ancestors of the topic has
actually been the more common case; typically it's that part1 is some
topic, and then part2 is a further topic that depends on the changes
in part1, so naturally it goes directly on top of it.  So I'd be
pleased to get the functionality of the present series in, even before
supporting the more general case.

I agree with your footnote -- the more general case will require a
more powerful sequencer to support properly.  And now I see that
Ramkumar Ramachandra is making progress on such a thing right now!
That's great news -- this is a project that has been attempted at
least four times, by five people (including me), in the last three
years.  I hope to see this round make it in -- I was actually thinking
about returning to the problem after seeing this series through, but I
would be glad to see Ram beat me to it.

When the more powerful sequencer comes along, it may be necessary to
rewrite some of this code in C (IIUC how the new sequencer will work),
but that shouldn't be hard, and is probably easier starting from a
shell version than from scratch.  Other than that, this code should go
in smoothly as the main ingredient, other than the sequencer itself,
needed for the general "rewrite side branches" feature.  With that and
because this series is independently useful, my suggestion would be to
merge it when it's ready in itself, without waiting for the sequencer.



> It would be nice to have a clear definition of what _should_ happen in
> this case, and a test that makes sure that that is the behaviour we get.
> 
> Starting from this topology
> 
>          1   2   topic
>  A---X---Y---Z---W
>   \
>    B
> 
> where the change going from A to B is an equilvalent to the change going
> from Y to Z, a rebase of A..W would reproduce this topology:
> 
> 
>          1   2   topic
>  A---X---Y---Z---W
>   \
>    B---X'--Y'--W'
>            1'  topic'
> 
> What should heppen to ref2? Should it be deleted? Should it point at Y'?

I would have it point at Y'.  That's what would happen if we just said
"git rebase B ref2" (or threw in --rewrite-heads to pull along ref1),
and I think it makes the most sense to be consistent.

This consistency also means we can think of the reset that rebase
performs when it finishes as equivalent to an implicit
"ref refs/heads/topic" command at the end of the TODO file, which I
wouldn't point out in explaining to a beginner but I think is a nice
property for "ref" to have.

I'll add a test for this scenario.

Greg

Re: [PATCH 0/6] rebase: command "ref" and options --rewrite-{refs,heads,tags}

[Greg Price]

On Tue, Jun 28, 2011 at 06:47:58AM -0400, Greg Price wrote:
> I agree with your footnote -- the more general case will require a
> more powerful sequencer to support properly.  And now I see that
> Ramkumar Ramachandra is making progress on such a thing right now!
> That's great news -- this is a project that has been attempted at
> least four times, by five people (including me), in the last three
> years.  I hope to see this round make it in -- I was actually thinking
> about returning to the problem after seeing this series through, but I
> would be glad to see Ram beat me to it.

[Hello, Ram!]

Hmm, on further reading I'm not sure the sequencer Ram aims to build
this summer actually extends to the "mark" and "reset" commands (or
the concepts of "the rewritten <commit>" and "detach" in the
pseudo-TODO notation of your footnote) that would be required to
implement this broader rewrite-side-branches feature.  It looks like
the focus may be on taking the existing features of rebase and
bringing them into C.

This sounds great too -- it should make rebase a lot faster -- but it
will leave open the pet project I've thought about returning to, of
making "rebase -i -p" work correctly even when the user wants to
rearrange the commits rather than just s/pick/edit/ etc.  That will
require implementing a richer sequencer very like the one required for
a general rewrite-side-branches feature.

Ram, have I correctly read your plans?  For context, see Junio's
remarks upthread about one thing a richer sequencer could be used for:
  http://thread.gmane.org/gmane.comp.version-control.git/176339/focus=176359

Greg

Re: [PATCH 0/6] rebase: command "ref" and options --rewrite-{refs,heads,tags}

[Ramkumar Ramachandra]

Hi Greg!

Greg Price writes:
> On Tue, Jun 28, 2011 at 06:47:58AM -0400, Greg Price wrote:
>> I agree with your footnote -- the more general case will require a
>> more powerful sequencer to support properly.  And now I see that
>> Ramkumar Ramachandra is making progress on such a thing right now!
>> That's great news -- this is a project that has been attempted at
>> least four times, by five people (including me), in the last three
>> years.  I hope to see this round make it in -- I was actually thinking
>> about returning to the problem after seeing this series through, but I
>> would be glad to see Ram beat me to it.

You're more than welcome to join in -- all the work is public :)
The latest iteration of the sequencer is in 'pu', and I'm currently
busy rolling out a nicer version.

> Hmm, on further reading I'm not sure the sequencer Ram aims to build
> this summer actually extends to the "mark" and "reset" commands (or
> the concepts of "the rewritten <commit>" and "detach" in the
> pseudo-TODO notation of your footnote) that would be required to
> implement this broader rewrite-side-branches feature.  It looks like
> the focus may be on taking the existing features of rebase and
> bringing them into C.

Yes, you're absolutely right.  Bringing the existing features to C is
quite a challenging task in itself, and is bound to keep me busy for
the rest of the summer.  I don't plan to implement the features your
series introduces yet, but it's certainly a great long-term goal -- I
think the current series has all the right ingredients;  it just
requires a lot of work before it can start providing more features.

> This sounds great too -- it should make rebase a lot faster -- but it
> will leave open the pet project I've thought about returning to, of
> making "rebase -i -p" work correctly even when the user wants to
> rearrange the commits rather than just s/pick/edit/ etc.  That will
> require implementing a richer sequencer very like the one required for
> a general rewrite-side-branches feature.

Right.  Since all this will take time, it'll probably be a good idea
to get your series merged in now.

-- Ram

Re: Fwd: [PATCH 0/6] rebase: command "ref" and options --rewrite-{refs,heads,tags}

[Sverre Rabbelier]

Heya,

On Tue, Jun 28, 2011 at 12:47, Greg Price <price@mit.edu> wrote:
> I would also like to support that use case.  In my personal
> experience, the case where the part$N are ancestors of the topic has
> actually been the more common case; typically it's that part1 is some
> topic, and then part2 is a further topic that depends on the changes
> in part1, so naturally it goes directly on top of it.  So I'd be
> pleased to get the functionality of the present series in, even before
> supporting the more general case.

FWIW, I'm having the exact same need while working on the remote-hg
series (which depends on my remote-helper follow-up series, which
depends on fast-export-fixes, which depends on my original
remote-helper series, which depends on peffs initial patches; luckily
the latter two have been merged to next :P). I'd really like having
this available, especially the rebase -i support.

-- 
Cheers,

Sverre Rabbelier

Re: Fwd: [PATCH 0/6] rebase: command "ref" and options --rewrite-{refs,heads,tags}

[Greg Price]

On Wed, Aug 03, 2011 at 03:32:41PM +0200, Sverre Rabbelier wrote:
> FWIW, I'm having the exact same need while working on the remote-hg
> series (which depends on my remote-helper follow-up series, which
> depends on fast-export-fixes, which depends on my original
> remote-helper series, which depends on peffs initial patches; luckily
> the latter two have been merged to next :P). I'd really like having
> this available, especially the rebase -i support.

Thanks for the encouragement and the poke.  Work is busy, but I've put
together a new version of the series addressing everyone's comments
from last time.  I'll give it the finishing touches and try to send it
out soon.

Greg

Re: [PATCH 0/6] rebase: command "ref" and options --rewrite-{refs,heads,tags}

[Junio C Hamano]

Have you ran all tests with the changes in this series, especially the
ones in t34xx series?

Re: Fwd: [PATCH 0/6] rebase: command "ref" and options --rewrite-{refs,heads,tags}

[Greg Price]

On Mon, Jun 27, 2011 at 2:23 PM, Junio C Hamano <gitster@pobox.com>
wrote:
> Have you ran all tests with the changes in this series, especially the
> ones in t34xx series?

Err, now that would be smart, now wouldn't it. =)

In fact, I had a stupid bug that this caught.  Fixed, will include in
next reroll (much sooner than this reroll was =P).  All tests now pass.

Greg



diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 42ea3e7..3b1e375 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -741,7 +741,7 @@ else
        revisions=$onto...$orig_head
        shortrevisions=$shorthead
 fi
-if test -z "rewrite_refs"
+if test -z "$rewrite_refs"
 then
        pretty=oneline
 else