re-running merge on a single file

re-running merge on a single file

[Chris Packham]

Hi List,

I'm in the middle of updating our copy of another projects repository.
So far its been OK, we haven't diverged too much so it was a fairly
simple operation to add the projects repository as a 'vendor' remote
and get git merge to do most of the heavy lifting then use git merge
tool to fix up the things that git merge couldn't resolve. So right
now I have a history something like this.

-o-o-o-o-o           master
 \           \
  \           m        update
   \         /
    \-o-o-o            vendor

Now I'm finding that when I first used git mergetool for some things I
messed up and need to fix the code. In some cases this is because some
naming schemes changed in the vendor branch so I need to make
equivalent changes to the code we've added, no problem thats just more
commits on the update branch. In other cases I'm actually thinking
that it'd be easier to ask git to try the merge again on that one
file. I can ask git difftool to show me the pre-merge differences with
'git difftool master..vendor <file>' but what I'd really like to do is
ask git mergetool to do something similar so I can re-do my manual
merging.

Alternatively I could manually drive and external merge tool like
kdiff3 if I had a recipe for getting the common ancestor revision.
Something like

git checkout <magic pathspec> -- file
mv file file.base

git checkout master -- file
mv file file.ours

git checkout vendor -- file
mv file file.theirs

kdiff3 file.base file.ours file.theirs -o file

---
Thanks,
Chris

Re: re-running merge on a single file

[Chris Packham]

Answering my own question.

On Thu, Mar 11, 2010 at 10:54 AM, Chris Packham <judge.packham@gmail.com> wrote:
> Alternatively I could manually drive and external merge tool like
> kdiff3 if I had a recipe for getting the common ancestor revision.
> Something like
>
> git checkout <magic pathspec> -- file
> mv file file.base

git checkout $(git merge-base master vendor) -- file

Still if anyone has a cleaner way of doing this I'd like to hear it.
It especially like a one-shot get file X from revision Y save as Z.

Re: re-running merge on a single file

[Markus Heidelberg]

Chris Packham, 2010-03-11 19:54:
> In other cases I'm actually thinking
> that it'd be easier to ask git to try the merge again on that one
> file. I can ask git difftool to show me the pre-merge differences with
> 'git difftool master..vendor <file>' but what I'd really like to do is
> ask git mergetool to do something similar so I can re-do my manual
> merging.

Not mergetool, but checkout:
git checkout --merge -- file

> Alternatively I could manually drive and external merge tool like
> kdiff3 if I had a recipe for getting the common ancestor revision.
> Something like
> 
> git checkout <magic pathspec> -- file
> mv file file.base

git cat-file blob :1:file > file.base

> 
> git checkout master -- file

git checkout --ours -- file

> mv file file.ours

or
git cat-file blob :2:file > file.ours

> git checkout vendor -- file

git checkout --theirs -- file

> mv file file.theirs

or
git cat-file blob :3:file > file.theirs

> kdiff3 file.base file.ours file.theirs -o file


See the manpages of git-checkout and git-merge for these syntaxes.

Markus

Re: re-running merge on a single file

[Chris Packham]

Hmm, having trouble with cat-file syntax

On Thu, Mar 11, 2010 at 12:29 PM, Markus Heidelberg
<markus.heidelberg@web.de> wrote:
> Not mergetool, but checkout:
> git checkout --merge -- file

Ok.
$ git checkout --merge -- cpu/mpc83xx/start.S

> git cat-file blob :1:file > file.base
>

$ git cat-file blob :1:cpu/mpc83xx/start.S > cpu/mpc83xx/start.S.base
fatal: Not a valid object name :1:cpu/mpc83xx/start.S

So I think I have figured out that I'm trying to get at stage 1, 2 and
3 of the file that git checkout --merge has just setup but I'm
tripping over the syntax.

Also if it helps

$ git --version
git version 1.7.0.1

Re: re-running merge on a single file

[Jakub Narebski]

Chris Packham <judge.packham@gmail.com> writes:

> Hmm, having trouble with cat-file syntax
> 
> On Thu, Mar 11, 2010 at 12:29 PM, Markus Heidelberg
> <markus.heidelberg@web.de> wrote:
> > Not mergetool, but checkout:
> > git checkout --merge -- file
> 
> Ok.
> $ git checkout --merge -- cpu/mpc83xx/start.S
> 
> > git cat-file blob :1:file > file.base
> >
> 
> $ git cat-file blob :1:cpu/mpc83xx/start.S > cpu/mpc83xx/start.S.base
> fatal: Not a valid object name :1:cpu/mpc83xx/start.S
> 
> So I think I have figured out that I'm trying to get at stage 1, 2 and
> 3 of the file that git checkout --merge has just setup but I'm
> tripping over the syntax.

First, instead of 'git cat-file blob <blob-id>' you can simply use 
'git show <blob-id>'.

Second, while in

  $ git checkout --merge -- cpu/mpc83xx/start.S

the path cpu/mpc83xx/start.S is relative to your current directory,
in

  $ git show :1:cpu/mpc83xx/start.S > start.S.base

the path has to be "absolute path in repository", i.e. path relative
to top directory of the repository.

Try

  $ git show :1:$(git rev-parse --show-prefix)cpu/mpc83xx/start.S \
    > cpu/mpc83xx/start.S.base

HTH
-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: re-running merge on a single file

[Chris Packham]

Hi,

On Thu, Mar 11, 2010 at 3:20 PM, Jakub Narebski <jnareb@gmail.com> wrote:
>
> First, instead of 'git cat-file blob <blob-id>' you can simply use
> 'git show <blob-id>'.
>
> Second, while in
>
>  $ git checkout --merge -- cpu/mpc83xx/start.S
>
> the path cpu/mpc83xx/start.S is relative to your current directory,
> in
>
>  $ git show :1:cpu/mpc83xx/start.S > start.S.base
>
> the path has to be "absolute path in repository", i.e. path relative
> to top directory of the repository.

 That is the path from the top directory (its a u-boot repository).

>
> Try
>
>  $ git show :1:$(git rev-parse --show-prefix)cpu/mpc83xx/start.S \
>    > cpu/mpc83xx/start.S.base

git show doesn't complain about the path. But I'm obviously not
setting the stage correctly

$ git checkout --merge -- cpu/mpc83xx/start.S
$ git show :1:$(git rev-parse --show-prefix)cpu/mpc83xx/start.S
fatal: Path 'cpu/mpc83xx/start.S' is in the index, but not at stage 1.
Did you mean ':0:cpu/mpc83xx/start.S'?

By now I have additional commits that touch cpu/mpc83xx/start.S so
I'll see if I can find a file that I haven't touched since the merge.

Re: re-running merge on a single file

[Johannes Sixt]

Chris Packham schrieb:
> $ git checkout --merge -- cpu/mpc83xx/start.S
> $ git show :1:$(git rev-parse --show-prefix)cpu/mpc83xx/start.S
> fatal: Path 'cpu/mpc83xx/start.S' is in the index, but not at stage 1.
> Did you mean ':0:cpu/mpc83xx/start.S'?

Both of these work only as long as the index still records the conflicted
state. If you (or one of your tools) has git-added the file, or you have
git-checked-out some version of the file, the conflict stages are lost,
and you must reset --hard and redo the entire merge.

-- Hannes

Re: re-running merge on a single file

[Junio C Hamano]

Johannes Sixt <j.sixt@viscovery.net> writes:

> Chris Packham schrieb:
>> $ git checkout --merge -- cpu/mpc83xx/start.S
>> $ git show :1:$(git rev-parse --show-prefix)cpu/mpc83xx/start.S
>> fatal: Path 'cpu/mpc83xx/start.S' is in the index, but not at stage 1.
>> Did you mean ':0:cpu/mpc83xx/start.S'?
>
> Both of these work only as long as the index still records the conflicted
> state. If you (or one of your tools) has git-added the file, or you have
> git-checked-out some version of the file, the conflict stages are lost,
> and you must reset --hard and redo the entire merge.

If the merge textually autoresolves cleanly, you might not even have any
conflicted state to begin with.  In such a case you would need to grab

	MERGE_HEAD:path-to-that-thing
        HEAD:path-to-that-thing

yourself.

Re: re-running merge on a single file

[Jakub Narebski]

On Fri, 12 Mar 2010, Chris Packham wrote:
> On Thu, Mar 11, 2010 at 3:20 PM, Jakub Narebski <jnareb@gmail.com> wrote:

> > Try
> >
> >  $ git show :1:$(git rev-parse --show-prefix)cpu/mpc83xx/start.S \
> >    > cpu/mpc83xx/start.S.base
> 
> git show doesn't complain about the path. But I'm obviously not
> setting the stage correctly
> 
> $ git checkout --merge -- cpu/mpc83xx/start.S
> $ git show :1:$(git rev-parse --show-prefix)cpu/mpc83xx/start.S
> fatal: Path 'cpu/mpc83xx/start.S' is in the index, but not at stage 1.
> Did you mean ':0:cpu/mpc83xx/start.S'?
> 
> By now I have additional commits that touch cpu/mpc83xx/start.S so
> I'll see if I can find a file that I haven't touched since the merge.

Note that "git checkout --merge <file>" re-creates <file> in the state
of textual merge conflict from the index (from stages 1, 2, 3).  It is
not able, unfortunately, to recover stages in index if you marked
<file> as resolved using "git add <file>".

Note also that you would get non-0 stages in index only if there was
a *merge conflict* for a file which was not automatically resolved.
If cpu/mpc83xx/start.S resolved cleanly, it would be in stage-0 only.

You can check what stages you have in index for a specified file using

  $ git ls-files --stage cpu/mpc83xx/start.S


If you don't have stages 1 and 2 in the index (which you can check using
command provided above), you can instead get versions of a file which
would be used in resolving conflict:

 $ git show HEAD:cpu/mpc83xx/start.S \
   >cpu/mpc83xx/start.S.ours
 $ git show MERGE_HEAD:cpu/mpc83xx/start.S \
   >cpu/mpc83xx/start.S.theirs
 $ git show $(git merge-base HEAD MERGE_HEAD):cpu/mpc83xx/start.S \
   >cpu/mpc83xx/start.S.base

Sidenote: versions in stage 1, 2, 3 differ from above versions in that
they have those conflict (chunks) that can be automatically resolved,
resolved.

HTH
-- 
Jakub Narebski
Poland

Re: re-running merge on a single file

[Chris Packham]

On Thu, Mar 11, 2010 at 10:56 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Johannes Sixt <j.sixt@viscovery.net> writes:
>
>> Chris Packham schrieb:
>>> $ git checkout --merge -- cpu/mpc83xx/start.S
>>> $ git show :1:$(git rev-parse --show-prefix)cpu/mpc83xx/start.S
>>> fatal: Path 'cpu/mpc83xx/start.S' is in the index, but not at stage 1.
>>> Did you mean ':0:cpu/mpc83xx/start.S'?
>>
>> Both of these work only as long as the index still records the conflicted
>> state. If you (or one of your tools) has git-added the file, or you have
>> git-checked-out some version of the file, the conflict stages are lost,
>> and you must reset --hard and redo the entire merge.
>
> If the merge textually autoresolves cleanly, you might not even have any
> conflicted state to begin with.  In such a case you would need to grab
>
>        MERGE_HEAD:path-to-that-thing
>        HEAD:path-to-that-thing
>
> yourself.
>

So if I understand correctly once I've committed the merge the various
stages and MERGE_HEADs go away. I don't really want to re-do the whole
merge which is probably the correct thing to do. I'm lazy and my
projects rules allow broken commits so instead I can poke about with
git show and various revisions.

Here's a simple script that does just that. I've borrowed some command
line options from git mergetool (Warning: gmail web interface). If
anyone is interested in seeing a proper version of this I can clean it
up submit it properly.

----8<----

From c6ad177712422cbba641975d9e6a5fe85fc8f1fa Mon Sep 17 00:00:00 2001
From: Chris Packham <judge.packham@gmail.com>
Date: Fri, 12 Mar 2010 12:03:02 -0800
Subject: [PATCH] Add git manual-merge command

Can be used to manually merge files from the specified revisions.
---
 git-manual-merge |   91 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 91 insertions(+), 0 deletions(-)
 create mode 100755 git-manual-merge

diff --git a/git-manual-merge b/git-manual-merge
new file mode 100755
index 0000000..f37ba88
--- /dev/null
+++ b/git-manual-merge
@@ -0,0 +1,91 @@
+#!/bin/sh
+#
+# Copyright (c) 2010 Chris Packham
+#
+# Based on git-mergetool by Theodore Y. Ts'o
+#
+# This file is licensed under the GPL v2
+#
+
+USAGE='<branch1> <branch2> <file> ...'
+. git-sh-setup
+. git-mergetool--lib
+require_work_tree
+
+merge_one_file()
+{
+    local has_base=false
+    git show $1:$4 > $4.$1 && has_base=true
+    git show $2:$4 > $4.$2 || die
+    git show $3:$4 > $4.$3 || die
+
+    if "$prompt" = true; then
+       printf "Hit return to start merge resolution tool (%s): " "$merge_tool"
+       read ans
+    fi
+
+    if "$has_base" = true; then
+        $merge_tool $4.$1 $4.$2 $4.$1 -o $4
+    else
+        $merge_tool $4.$2 $4.$1 -o $4
+    fi
+}
+
+prompt=$(git config --bool mergetool.prompt || echo true)
+
+while test $# != 0
+do
+    case "$1" in
+        -t|--tool*)
+            case "$#,$1" in
+                *,*=*)
+                    merge_tool=$(expr "z$1" : 'z-[^=]*=\(.*\)')
+                    ;;
+                1,*)
+                    usage ;;
+                *)
+                    merge_tool="$2"
+                    shift ;;
+            esac
+            ;;
+        -y|--no-prompt)
+            prompt=false
+            ;;
+        --prompt)
+            prompt=true
+            ;;
+        --)
+            shift
+            break
+            ;;
+        -*)
+            usage
+            ;;
+        *)
+            break
+            ;;
+    esac
+    shift
+done
+
+echo "$merge_tool"
+
+if test $# -lt 3; then
+    usage
+fi
+
+branch1=$(git rev-parse --short  $1)
+branch2=$(git rev-parse --short  $2)
+_base=$(git merge-base $branch1 $branch2)
+base=$(git rev-parse --short  $_base)
+
+if test -z "$merge_tool"; then
+    merge_tool=$(get_merge_tool "$merge_tool") || exit
+fi
+
+shift 2
+
+for x in $*; do
+    merge_one_file $base $branch1 $branch2 $x
+done
+
--
1.7.0.1

Re: re-running merge on a single file

[Chris Packham]

On Fri, Mar 12, 2010 at 12:16 PM, Chris Packham <judge.packham@gmail.com> wrote:
> On Thu, Mar 11, 2010 at 10:56 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Johannes Sixt <j.sixt@viscovery.net> writes:
>>
>>> Chris Packham schrieb:
>>>> $ git checkout --merge -- cpu/mpc83xx/start.S
>>>> $ git show :1:$(git rev-parse --show-prefix)cpu/mpc83xx/start.S
>>>> fatal: Path 'cpu/mpc83xx/start.S' is in the index, but not at stage 1.
>>>> Did you mean ':0:cpu/mpc83xx/start.S'?
>>>
>>> Both of these work only as long as the index still records the conflicted
>>> state. If you (or one of your tools) has git-added the file, or you have
>>> git-checked-out some version of the file, the conflict stages are lost,
>>> and you must reset --hard and redo the entire merge.
>>
>> If the merge textually autoresolves cleanly, you might not even have any
>> conflicted state to begin with.  In such a case you would need to grab
>>
>>        MERGE_HEAD:path-to-that-thing
>>        HEAD:path-to-that-thing
>>
>> yourself.
>>
>
> So if I understand correctly once I've committed the merge the various
> stages and MERGE_HEADs go away. I don't really want to re-do the whole
> merge which is probably the correct thing to do. I'm lazy and my
> projects rules allow broken commits so instead I can poke about with
> git show and various revisions.
>
> Here's a simple script that does just that. I've borrowed some command
> line options from git mergetool (Warning: gmail web interface). If
> anyone is interested in seeing a proper version of this I can clean it
> up submit it properly.
>
> ----8<----
>
> From c6ad177712422cbba641975d9e6a5fe85fc8f1fa Mon Sep 17 00:00:00 2001
> From: Chris Packham <judge.packham@gmail.com>
> Date: Fri, 12 Mar 2010 12:03:02 -0800
> Subject: [PATCH] Add git manual-merge command
>
> Can be used to manually merge files from the specified revisions.
> ---
>  git-manual-merge |   91 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 91 insertions(+), 0 deletions(-)
>  create mode 100755 git-manual-merge
>
> diff --git a/git-manual-merge b/git-manual-merge
> new file mode 100755
> index 0000000..f37ba88
> --- /dev/null
> +++ b/git-manual-merge
> @@ -0,0 +1,91 @@
> +#!/bin/sh
> +#
> +# Copyright (c) 2010 Chris Packham
> +#
> +# Based on git-mergetool by Theodore Y. Ts'o
> +#
> +# This file is licensed under the GPL v2
> +#
> +
> +USAGE='<branch1> <branch2> <file> ...'
> +. git-sh-setup
> +. git-mergetool--lib
> +require_work_tree
> +
> +merge_one_file()
> +{
> +    local has_base=false
> +    git show $1:$4 > $4.$1 && has_base=true
> +    git show $2:$4 > $4.$2 || die
> +    git show $3:$4 > $4.$3 || die
> +
> +    if "$prompt" = true; then
> +       printf "Hit return to start merge resolution tool (%s): " "$merge_tool"
> +       read ans
> +    fi
> +
> +    if "$has_base" = true; then
> +        $merge_tool $4.$1 $4.$2 $4.$1 -o $4

Oops, that should be

+        $merge_tool $4.$1 $4.$2 $4.$3 -o $4

> +    else
> +        $merge_tool $4.$2 $4.$1 -o $4
> +    fi
> +}
> +
> +prompt=$(git config --bool mergetool.prompt || echo true)
> +
> +while test $# != 0
> +do
> +    case "$1" in
> +        -t|--tool*)
> +            case "$#,$1" in
> +                *,*=*)
> +                    merge_tool=$(expr "z$1" : 'z-[^=]*=\(.*\)')
> +                    ;;
> +                1,*)
> +                    usage ;;
> +                *)
> +                    merge_tool="$2"
> +                    shift ;;
> +            esac
> +            ;;
> +        -y|--no-prompt)
> +            prompt=false
> +            ;;
> +        --prompt)
> +            prompt=true
> +            ;;
> +        --)
> +            shift
> +            break
> +            ;;
> +        -*)
> +            usage
> +            ;;
> +        *)
> +            break
> +            ;;
> +    esac
> +    shift
> +done
> +
> +echo "$merge_tool"
> +
> +if test $# -lt 3; then
> +    usage
> +fi
> +
> +branch1=$(git rev-parse --short  $1)
> +branch2=$(git rev-parse --short  $2)
> +_base=$(git merge-base $branch1 $branch2)
> +base=$(git rev-parse --short  $_base)
> +
> +if test -z "$merge_tool"; then
> +    merge_tool=$(get_merge_tool "$merge_tool") || exit
> +fi
> +
> +shift 2
> +
> +for x in $*; do
> +    merge_one_file $base $branch1 $branch2 $x
> +done
> +
> --
> 1.7.0.1
>

Re: re-running merge on a single file

[Junio C Hamano]

Chris Packham <judge.packham@gmail.com> writes:

> ... I'm lazy and my
> projects rules allow broken commits so ...

We cannot fix, nor it is our job to fix, lazy people or undisciplined
projects ;-)

Having said that you can still retry your merge, which was _mostly_ Ok.

 (1) First, check out the original commit before the merge.  You may have
     built a few more commits on top of it, so "git log --first-parent"
     may help to find that commit.  Suppose you found that merge commit
     and we call that $merge; then:

     $ git checkout $merge^1
   
 (2) Then you retry that merge:

     $ git merge $merge^2

     which would conflict the same way you saw.

 (3) If you have rerere enabled, then the conflicts would be already
     resolved in your working tree at this point, but not in the index, so
     you can reproduce the conflicted state with "checkout -m".

 (4) If you didn't have rerere enabled, then you would have a lot of
     conflicted files in the working tree.  Inspect them by comparing with
     $merge:

     $ git diff $merge

     Since we are assuming that most of the resolution was good, and you
     know what paths you want to fix your earlier resolution for, you
     would want to take what is in $merge for most of the paths.  You can
     run checkout on paths that you know were good in $merge, like so:

     $ git checkout $merge -- include/ drivers/

     and repeat this step to narrow it down to the paths you are
     interested in.

 (5) After you fix the resolution and make a commit,

     $ git diff $merge

     will show that your earlier incorrect resolution is replaced by a new
     correct resolution.

After that, you can rebase your branch on top of HEAD to conclude.

Note that recent git can use "checkout -m path" to reproduce conflicts
even after you ran "git add path".

Re: re-running merge on a single file

[Chris Packham]

On Fri, Mar 12, 2010 at 3:07 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Chris Packham <judge.packham@gmail.com> writes:
>
>> ... I'm lazy and my
>> projects rules allow broken commits so ...
>
> We cannot fix, nor it is our job to fix, lazy people or undisciplined
> projects ;-)

Fair call. I don't think the maintainers of human-nature-v1.0 are
interested in patches :)

Actually on another note wasn't there an idea of being able to say
that a particular set of commits is known to break bisect-ability?
Ideally I'd want one commit that comprises the merge from upstream, at
this point the build targets from upstream should build happily but
the targets we've added we know are broken. Then I come along and fix
the targets we've added, maybe doing commits sub system by sub system
to make review easy. Finally whatever work is left to actually make
our targets build successfully is in another commit. From a bisect
point of view I'd want to say that the merge and sub system commits
are intentional breakages and that bisect should really look at this
whole series as a single uber commit. I guess this is information that
could go into notes that bisect could then take into account when
making its next jump.

> Having said that you can still retry your merge, which was _mostly_ Ok.
>
>  (1) First, check out the original commit before the merge.  You may have
>     built a few more commits on top of it, so "git log --first-parent"
>     may help to find that commit.  Suppose you found that merge commit
>     and we call that $merge; then:
>
>     $ git checkout $merge^1
>
>  (2) Then you retry that merge:
>
>     $ git merge $merge^2
>
>     which would conflict the same way you saw.
>
>  (3) If you have rerere enabled, then the conflicts would be already
>     resolved in your working tree at this point, but not in the index, so
>     you can reproduce the conflicted state with "checkout -m".

Mental note: Need to learn more about git rerere. Sounds like what I
need to do the job.

>  (4) If you didn't have rerere enabled, then you would have a lot of
>     conflicted files in the working tree.  Inspect them by comparing with
>     $merge:
>
>     $ git diff $merge
>
>     Since we are assuming that most of the resolution was good, and you
>     know what paths you want to fix your earlier resolution for, you
>     would want to take what is in $merge for most of the paths.  You can
>     run checkout on paths that you know were good in $merge, like so:
>
>     $ git checkout $merge -- include/ drivers/
>
>     and repeat this step to narrow it down to the paths you are
>     interested in.
>
>  (5) After you fix the resolution and make a commit,
>
>     $ git diff $merge
>
>     will show that your earlier incorrect resolution is replaced by a new
>     correct resolution.
>
> After that, you can rebase your branch on top of HEAD to conclude.
>
> Note that recent git can use "checkout -m path" to reproduce conflicts
> even after you ran "git add path".

OK I'll look out for that in future. This won't be the last time I'm
doing a big merge.

Re: re-running merge on a single file

[Kris Shannon]

On 13 March 2010 10:43, Chris Packham <judge.packham@gmail.com> wrote:
> On Fri, Mar 12, 2010 at 3:07 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>  (3) If you have rerere enabled, then the conflicts would be already
>>     resolved in your working tree at this point, but not in the index, so
>>     you can reproduce the conflicted state with "checkout -m".
>
> Mental note: Need to learn more about git rerere. Sounds like what I
> need to do the job.
>

There is a script in the contrib directory written by Nanako Shiraishi called
rerere-train.sh which can be used to prime the rr-cache even if you didn't
have rerere enabled before you orginally did the merge.