Sharing merge conflict resolution between multiple developers

Sharing merge conflict resolution between multiple developers

[Chris Packham]

Hi List,

At $dayjob we maintain internal forks of the a number of upstream repositories.

Unsurprisingly updating these forks can be extremely problematic,
especially when it's only one person doing the merge. Fortunately most
of us are in the same physical location so it is possible to drag in
someone who knows more about the code than the person merging but I
can't see that scaling with remote developers.

Is there any way where we could share the conflict resolution around
but still end up with a single merge commit. I'm thinking of something
like the following workflow

developer A:
  git merge $upstream
  <conflicts>
  git mergetool ...
  <resolves some of the conflicts>
  git commit -m "WIP: Merge upstream" --something-like--all-but-not

developer B:
  git pull developer_A
  git reset HEAD^
  <at this point it's be nice if developer B's work tree and index
were in the same state as developer A's before they ran 'git commit'>
  git mergetool ...
  <resolves the conflicts that they know about>
  git commit -m "WIP: Merge upstream" --something-like--all-but-not

developer A:
  git pull developer_B
  git reset HEAD^
  git mergetool ....
  <finishes off anything left>
  git commit
  <It'd be really nice if MERGE_MSG was retained through all of this>
  git push

Any thoughts on if something like this is currently possible? Is this
something other git users would find useful?

Thanks,
Chris

Re: Sharing merge conflict resolution between multiple developers

[Junio C Hamano]

Chris Packham <judge.packham@gmail.com> writes:

> Is there any way where we could share the conflict resolution around
> but still end up with a single merge commit.

One idea that immediately comes to me is to use something like
"rerere" (not its implementation and storage, but the underlying
idea) enhanced with the trick I use to fix-up merges in my daily
integration cycle (look for "merge-fix" in howto/maintain-git.txt
in Documentation/).

> developer A:
>   git merge $upstream
>   <conflicts>

And then commit this immediately, together with conflict markers
(i.e. "commit -a"), and discard it with "reset --hard HEAD^" *after*
storing it somewhere safe.  And then redo the same merge, resolve
the conflicts and commit the usual way.

The difference between the final conflict resolution and the
original conflicted state can be used as a reference for others to
redo the same conflict resolution later elsewhere.  That can most
easily be done by creating a commit that records the final state
whose parent is the one you recorded the initial conflicted state.

So, the "recording" phase may go something like this:

    git checkout $this
    git merge $that
    git commit -a -m 'merge-fix/$this-$that preimage'
    git branch merge-fix/$this-$that
    git reset --hard HEAD^
    git merge $that
    edit
    git commit -a -m 'merge $that to $this'
    git checkout merge-fix/$this-$that
    git read-tree -m -u HEAD $this
    git commit -a -m 'merge-fix/$this-$that postimage'

The rough idea is "git show merge-fix/$this-$that" will show the
"patch" you can apply on top of the conflicted state other people
would get by running "git merge $that" while on "$this" branch.

"rerere" essentially does the above recording (and replaying)
per-path and it comes with a clever indexing scheme to identify
which previous conflict resolution would apply to the conflicts you
see in your working tree.

Re: Sharing merge conflict resolution between multiple developers

[Christian Couder]

Le 11 août 2014 07:50, "Christian Couder" <christian.couder@gmail.com> a écrit :
>
> This should be possible using "git imerge" which is separate tool.

Sorry I sent the above using the gmail app on my mobile phone and
unfortunately I can't make it send plain text emails.
(Emails which are not plain text are rejected by vger.kernel.org.)

Best,
Christian.

Re: Sharing merge conflict resolution between multiple developers

[Nico Williams]

IIUC, this might help,

http://www.mail-archive.com/git@vger.kernel.org/msg56418.html
http://www.mail-archive.com/git@vger.kernel.org/msg56468.html

Re: Sharing merge conflict resolution between multiple developers

[Chris Packham]

On Tue, Aug 12, 2014 at 6:44 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Chris Packham <judge.packham@gmail.com> writes:
>
>> Is there any way where we could share the conflict resolution around
>> but still end up with a single merge commit.
>
> One idea that immediately comes to me is to use something like
> "rerere" (not its implementation and storage, but the underlying
> idea) enhanced with the trick I use to fix-up merges in my daily
> integration cycle (look for "merge-fix" in howto/maintain-git.txt
> in Documentation/).
>
>> developer A:
>>   git merge $upstream
>>   <conflicts>
>
> And then commit this immediately, together with conflict markers
> (i.e. "commit -a"), and discard it with "reset --hard HEAD^" *after*
> storing it somewhere safe.  And then redo the same merge, resolve
> the conflicts and commit the usual way.
>
> The difference between the final conflict resolution and the
> original conflicted state can be used as a reference for others to
> redo the same conflict resolution later elsewhere.  That can most
> easily be done by creating a commit that records the final state
> whose parent is the one you recorded the initial conflicted state.
>
> So, the "recording" phase may go something like this:
>
>     git checkout $this
>     git merge $that
>     git commit -a -m 'merge-fix/$this-$that preimage'
>     git branch merge-fix/$this-$that
>     git reset --hard HEAD^
>     git merge $that
>     edit
>     git commit -a -m 'merge $that to $this'
>     git checkout merge-fix/$this-$that
>     git read-tree -m -u HEAD $this
>     git commit -a -m 'merge-fix/$this-$that postimage'
>
> The rough idea is "git show merge-fix/$this-$that" will show the
> "patch" you can apply on top of the conflicted state other people
> would get by running "git merge $that" while on "$this" branch.

So how would someone else pickup that postimage and use it?

  git checkout $this
  git merge $that
  git fetch $remote ':/merge-fix/$this-$that postimage'
  git show ':/merge-fix/$this-$that postimage' | git apply (or patch -p1)
  edit

>
> "rerere" essentially does the above recording (and replaying)
> per-path and it comes with a clever indexing scheme to identify
> which previous conflict resolution would apply to the conflicts you
> see in your working tree.

I feel a toy patch coming on.

Re: Sharing merge conflict resolution between multiple developers

[Junio C Hamano]

On Mon, Aug 11, 2014 at 4:29 PM, Chris Packham <judge.packham@gmail.com> wrote:

>> So, the "recording" phase may go something like this:
>> ...
>>     git checkout merge-fix/$this-$that
>>     git read-tree -m -u HEAD $this
>>     git commit -a -m 'merge-fix/$this-$that postimage'
>>
>> The rough idea is "git show merge-fix/$this-$that" will show the
>> "patch" you can apply on top of the conflicted state other people
>> would get by running "git merge $that" while on "$this" branch.
>
> So how would someone else pickup that postimage and use it?
>
>   git checkout $this
>   git merge $that
>   git fetch $remote ':/merge-fix/$this-$that postimage'
>   git show ':/merge-fix/$this-$that postimage' | git apply (or patch -p1)

For a simpler case that would work, but because we are not saving
just a patch but two full trees to compare (i.e. merge-fix/$this-$that
is the postimage, its ^1 is the preimage), you should be able to use
the three-way merge in a similar way cherry-pick works. In fact, that
is how rerere replays the recorded resolution, not with a "patch -p1".

Re: Sharing merge conflict resolution between multiple developers

[Jeff King]

On Mon, Aug 11, 2014 at 04:59:15PM +1200, Chris Packham wrote:

> Is there any way where we could share the conflict resolution around
> but still end up with a single merge commit. I'm thinking of something
> like the following workflow

This came up once a while back. Here's the discussion:

  http://thread.gmane.org/gmane.comp.version-control.git/169187

I proposed a solution there where developers push their partial
resolutions, and the integrator (or just "next person" if it is a chain
of developers) uses "git checkout" to pull the fixes. But note that I
never actually _used_ that in practice. It was a thought experiment, so
there may be gotchas.

-Peff

Signinig a commit with multiple signatures

[Jason Pyeron]

[-- Attachment #1: Type: text/plain, Size: 2276 bytes --]

I am working on an open source project right now where we are looking to enforce a N of M audit approval process. It turns out that git supports verifying multiple signatures because gpg supports signature merging.

My question is how can this workflow best be added into git and if not added atleast supported.

Here are the manual procedures (scripts are in the bundle too):

> Procedures:
> 
> 1. Identify a normal commit.
> 2. create a new commit file as:
> parent commit-id-of-step-1
> tree tree-id-from-git-cat-file-commit-commit-id-of-step-1
> author CipherShed Security Team <security@ciphershed.org> 
> timestamp timezone
> committer Actual Person <username@ciphershed.org> timestamp timezone
> gpgsig output-from-merge-sig-tool [1]
>  more-output
>  more-output
> 
> Comments for this commit
> ...
> ...
> 
> 3. run ruby script [2] to add commit to git db
> 4. git update-ref refs/heads/BRANCH-NAME new-commit-id

To do this most properly I feel like there needs to be a way to "share" the repository state and intterrupt the commit process.

Comments?

1: 
$ cat merge-multisigs.sh
#!/bin/bash
(
 for i in "$@"
 do
  gpg --dearmor < "$i"
 done
) | gpg --enarmor

2:
$ cat write-commit.ruby
#!/usr/bin/irb
require 'fileutils'
file = File.open(ARGV[0], "rb")
content = file.read
header = "commit #{content.length}\0"
store = header + content
require 'digest/sha1'
sha1 = Digest::SHA1.hexdigest(store)
require 'zlib'
zlib_content = Zlib::Deflate.deflate(store)
path = '.git/objects/' + sha1[0,2] + '/' + sha1[2,38]
FileUtils.mkdir_p(File.dirname(path))
File.open(path, 'w') { |f| f.write zlib_content }


P.S. This was inspired by actual events and the parent thread.

--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
-                                                               -
- Jason Pyeron                      PD Inc. http://www.pdinc.us -
- Principal Consultant              10 West 24th Street #100    -
- +1 (443) 269-1555 x333            Baltimore, Maryland 21218   -
-                                                               -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.

[-- Attachment #2: multisign.bundle --]
[-- Type: application/octet-stream, Size: 2949 bytes --]

Re: Signinig a commit with multiple signatures

[Jeff King]

On Sun, Aug 17, 2014 at 09:30:47AM -0400, Jason Pyeron wrote:

> I am working on an open source project right now where we are looking
> to enforce a N of M audit approval process. It turns out that git
> supports verifying multiple signatures because gpg supports signature
> merging.

In the scheme you propose, the commit object is actually rewritten. So
whoever made and signed it first will then need to rebase on top of the
rewritten multi-signed version.

Is there a reason not to use detached signatures, and let each person
add them after the fact? You can store them in git-notes and then push
them along with the other commits (you can even check in a pre-receive
hook that the commits meet your N of M criteria, as long as everybody
has pushed up their signature notes).

> $ cat write-commit.ruby
> #!/usr/bin/irb
> require 'fileutils'
> file = File.open(ARGV[0], "rb")
> content = file.read
> header = "commit #{content.length}\0"
> store = header + content
> require 'digest/sha1'
> sha1 = Digest::SHA1.hexdigest(store)
> require 'zlib'
> zlib_content = Zlib::Deflate.deflate(store)
> path = '.git/objects/' + sha1[0,2] + '/' + sha1[2,38]
> FileUtils.mkdir_p(File.dirname(path))
> File.open(path, 'w') { |f| f.write zlib_content }

I think this is just "git hash-object -w -t commit <file>", isn't it?

-Peff

RE: Signinig a commit with multiple signatures

[Jason Pyeron]

> -----Original Message-----
> From: Jeff King
> Sent: Tuesday, August 19, 2014 4:05
> 
> On Sun, Aug 17, 2014 at 09:30:47AM -0400, Jason Pyeron wrote:
> 
> > I am working on an open source project right now where we 
> are looking
> > to enforce a N of M audit approval process. It turns out that git
> > supports verifying multiple signatures because gpg supports 
> signature
> > merging.
> 
> In the scheme you propose, the commit object is actually rewritten. So
> whoever made and signed it first will then need to rebase on 
> top of the
> rewritten multi-signed version.

Not exactly. A known and shared commit is used as the parent of an empty, no changes commit. The "no changes" commit object is taken and passed around before being added into the repository. There is no need for a rebase.

But my scheme uses out-of-band process to accomplish this. The idea of using git to "distribute" the conflict resolution seemed like a valid use case of sharing a working copy state for a distributed commit, just like this. [1][2]

> 
> Is there a reason not to use detached signatures, and let each person

Yes. The embeded signatures provides the best user experience (UX) for verification.

> add them after the fact? You can store them in git-notes and then push
> them along with the other commits (you can even check in a pre-receive
> hook that the commits meet your N of M criteria, as long as everybody
> has pushed up their signature notes).
> 
> > $ cat write-commit.ruby
> > #!/usr/bin/irb
> > require 'fileutils'
> > file = File.open(ARGV[0], "rb")
> > content = file.read
> > header = "commit #{content.length}\0"
> > store = header + content
> > require 'digest/sha1'
> > sha1 = Digest::SHA1.hexdigest(store)
> > require 'zlib'
> > zlib_content = Zlib::Deflate.deflate(store)
> > path = '.git/objects/' + sha1[0,2] + '/' + sha1[2,38]
> > FileUtils.mkdir_p(File.dirname(path))
> > File.open(path, 'w') { |f| f.write zlib_content }
> 
> I think this is just "git hash-object -w -t commit <file>", isn't it?

Let me find the most complicated way of saying this, yes. I feel silly for that.

-Jason

[1]: http://git.661346.n2.nabble.com/Sharing-a-massive-distributed-merge-td6178696.html
[2]: http://git.661346.n2.nabble.com/Sharing-merge-conflict-resolution-between-multiple-developers-td7616700.html