merging two equivalent branches

merging two equivalent branches

[David Reitter]

Hello,

I have a problem with my git project, resulting from an upstream branch beyond my control being rewritten.

Can I specify parents for a revision whose history is hidden from git-log?

Concretely, I need to merge two branches that represent different conversions of the same original CVS branch (with >100k revisions).  
I've been working with converted branch B, but now we have a new branch A.  Revisions A150 and B145 correspond to the same tree, but there is no common ancestor:

A1 -> A2 -> A3 -> .. -> A150 (A)
B1 -> B2 -> B3 -> .. -> B145 (B)

I have a published downstream branch C with my own changes that started somewhere from B and has occasionally merged new developments from B.  I'd now like to switch it to A.     Future development will show up on A and I'd like to be able to merge it into C when that happens.

Using "git-merge -s ours" does this job nicely so that I can pull further development from the remote branch into mine.

However, git-log follows both parents of the new merge commit and thus shows many redundant commits.  This is OK from the logical perspective, but because I have used the "ours" merge strategy, we're guaranteed to have only one revisions in the final tree.  Thus, I wouldn't want to see all these revisions in the resulting branch.  Grafts/rewrites or git-replace would probably lead to the same issue, I reckon.

Thanks for your help.

Re: merging two equivalent branches

[Christian MICHON]

On Thu, Jan 7, 2010 at 7:17 PM, David Reitter <david.reitter@gmail.com> wrote:
> Hello,
>
> I have a problem with my git project, resulting from an upstream branch beyond my control being rewritten.
>
> Can I specify parents for a revision whose history is hidden from git-log?
>

I recall asking a similar question in 2008, and the answer was to look
at "git graft" and use "git filter-branch" to recreate history.

My 2 cents

-- 
Christian
--
http://detaolb.sourceforge.net/, a linux distribution for Qemu with Git inside !

git-log - hide parent (was: merging two equivalent branches)

[David Reitter]

On Jan 7, 2010, at 1:22 PM, Christian MICHON wrote:
> I recall asking a similar question in 2008, and the answer was to look
> at "git graft" and use "git filter-branch" to recreate history.

Thanks, I've tried that and I recall that filter-branch wasn't willing to rewrite just the recent history - at least in started going over all 100k revisions at a very slow pace.  

I'm still unsure how, after the filter-branch, I would have some ancestor from the B series so that future pulls from the remote work, while having an ancestor from A, to make sure I can continue merging into C.  If history is rewritten, I'll get new revisions and lose ancestors.  
I'm beginning to thing that the cutting and pasting I'd like is conceptually impossible.

So what one would need is to specify a "silent parent" for a revision that is relevant w.r.t. future three-way merges, but indicates that the history behind the silent parent is irrelevant and shouldn't be shown in a git-log.  The runaway parent would be guaranteed to _not_ contribute any content to the tree of the child revision, as is the case with a "merge ours".

This could be implemented as a way to mark a parent as silent (checked by git-log at least), but one could also allow for an empty commit that, while having a normal parent, clears out the tree. 

Let me know if this idea is completely crazy. 

Re: git-log - hide parent (was: merging two equivalent branches)

[Christian Couder]

On jeudi 07 janvier 2010, David Reitter wrote:
> On Jan 7, 2010, at 1:22 PM, Christian MICHON wrote:
> > I recall asking a similar question in 2008, and the answer was to look
> > at "git graft" and use "git filter-branch" to recreate history.
>
> Thanks, I've tried that and I recall that filter-branch wasn't willing to
> rewrite just the recent history - at least in started going over all 100k
> revisions at a very slow pace.
>
> I'm still unsure how, after the filter-branch, I would have some ancestor
> from the B series so that future pulls from the remote work, while having
> an ancestor from A, to make sure I can continue merging into C.  If
> history is rewritten, I'll get new revisions and lose ancestors. I'm
> beginning to thing that the cutting and pasting I'd like is conceptually
> impossible.
>
> So what one would need is to specify a "silent parent" for a revision
> that is relevant w.r.t. future three-way merges, but indicates that the
> history behind the silent parent is irrelevant and shouldn't be shown in
> a git-log.  The runaway parent would be guaranteed to _not_ contribute
> any content to the tree of the child revision, as is the case with a
> "merge ours".

What you could perhaps do with "git replace" or a graft is to change the 
merge commit so that it has only one parent instead of 2.

> This could be implemented as a way to mark a parent as silent (checked by
> git-log at least), but one could also allow for an empty commit that,
> while having a normal parent, clears out the tree.
>
> Let me know if this idea is completely crazy. --

This looks like the right thing to do using "git replace" or grafts.

Best regards,
Christian. 

Re: git-log - hide parent (was: merging two equivalent branches)

[David Reitter]

[-- Attachment #1.1: Type: text/plain, Size: 1234 bytes --]

On Jan 8, 2010, at 12:00 AM, Christian Couder wrote:
> What you could perhaps do with "git replace" or a graft is to change the 
> merge commit so that it has only one parent instead of 2.

Thanks, also to Avery for his idea with "git .", which works well for me. 

For the benefit of others, here's what I've done in the end in order to get rid of the extra 100,000 commits in the old upstream branch.

A very simple little script takes care of remapping the merges of the old upstream branch to the new one.
It takes the output of this

git log --since=2009-01-01  --format="%H %f"

on each of the two upstream branches and finds corresponding commits using the first commit line.  With this alignment in place, we then need the list of merge commits that need to be redirected:

export BRANCHES='b1 b2 b3'
git log --since=2009-01-01 --author="my name" --format="%H %P" $BRANCHES

This can be fairly broad, but I didn't want BRANCHES to contain the upstream branches (even then it probably doesn't matter).

I then used a little piece of trivial code to apply the alignment to the parents of potential merge commits, to generate the grafts.
This is what's attached, in case anyone will find it useful.





[-- Attachment #1.2: make-grafts.py --]
[-- Type: text/x-python-script, Size: 3901 bytes --]

#!/usr/bin/python

# This will output a Git "grafts" file to use when a (set of) downstream branch(es)
# is to switch from merging with one upstream branch to another upstream branch, without
# keeping the full history of both upstream branches in the repository.
# If the upstream branches contain the same history information (semantically speaking),
# such as when they represent different conversions from an old CVS/SVN repository, then
# this script will find an alignment between them.
# The resulting grafts file will map past merges in the downstream branch to the new
# upstream branch, as if the old upstream branch never existed.
# If this works well, a "git filter-branch" should burn in the grafts so that they can
# be removed.  (This is a history-changing operation.)

# See also:
# http://thread.gmane.org/gmane.comp.version-control.git/136377


# recommended usage:
#./make-grafts.py | sort | uniq > .git/info/grafts

# set input file names
upstream1 = "em-log"
upstream2 = "em-new-log"
mergecommits = "aq-merges"


# produce files:
# git checkout upstream1
# git log --since=2009-01-01  --format="%H %f">../emc/em-log
# git checkout upstream2
# git log --since=2009-01-01  --format="%H %f">../emc/em-new-log
# export BRANCHES='topic/b1 topic/b2 john jane master'
# git log --since=2009-01-01 --author="David Reitter" --format="%H %P" $BRANCHES >../emc/aq-merges; git log --since=2009-01-01 --merges --format="%H %P" $BRANCHES >>../emc/aq-merges

#################

import re
import sys

r1msgs = {}
r1revids = {}
# checkout emacs
# git log --since=2009-01-01  --format="%H %f">../emc/em-log
file = open(upstream1, 'r')
for l in file:
    m =    re.match("([a-f0-9]*) (.*)", l)
    if m:
       r1msgs[m.group(1)] = m.group(2)
       r1revids[m.group(2)] = m.group(1)
file.close()


r2msgs = {}
r2revids = {}
# checkout emacs23
# git log --since=2009-01-01  --format="%H %f" >../emc/em-new-log
file = open(upstream2, 'r')
for l in file:
    m = re.match("([a-f0-9]*) (.*)", l)
    if m:
       r2msgs[m.group(1)] = m.group(2)
       r2revids[m.group(2)] = m.group(1)
file.close()

# checkout master
# export BRANCHES='23.1.undone  Aquamacs22  dr-after-merge  dr/dev  dr/experimental  dr/suedit master topic/NSAlertDialogs  topic/dialogs  topic/face-remapping  topic/mac-support  topic/menu-bar  topic/minibuffer  topic/option-key-remap  topic/printing  topic/python-mode  topic/reconf  topic/smart-spacing  topic/spelling  topic/tabbar  topic/tmm  topic/toolbar'
# git log --since=2009-01-01 --author="David Reitter" --merges --format="%H %P" $BRANCHES >../emc/aq-merges
# it's better to use all commits so we don't miss anything
# git log --since=2009-01-01 --author="David Reitter" --format="%H %P" $BRANCHES >../emc/aq-merges; git log --since=2009-01-01 --merges --format="%H %P" $BRANCHES >>../emc/aq-merges
def replace (revid):
   if revid in r1msgs:
      r1m = r1msgs[revid]
      if r1m in r2revids:
         r2r = r2revids[r1m]
         if r2r:
            return r2r
         else:
            print >> sys.stderr, "can't get mapping for revid "+revid+ r1m
            
      else:
         print >> sys.stderr, "can't find message of revid "+revid
   return revid
      

# checkout master
# 

file = open(mergecommits, 'r')
for l in file:
    revids = l.rstrip().split(" ")
    if revids:

        # test:
        # revids2 = revids
        # for r in revids[1:]:
        #     r2 = replace(r)
        #     if r != r2:
        #         revids2 = revids2 + [r2]
        # correct alternative
        revids2 = [revids[0]] + map(replace, revids[1:])
        if revids != revids2:
            # make graft entry to write the merge such that it
            # looks like the merge came from the other branch
            print " ".join(revids2)
        else:
            print >> sys.stderr, "no remappings found for parents of merge revid "+" ".join(revids)

file.close()



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 203 bytes --]

Re: git-log - hide parent (was: merging two equivalent branches)

[Avery Pennarun]

On Thu, Jan 7, 2010 at 4:16 PM, David Reitter <david.reitter@gmail.com> wrote:
> I'm still unsure how, after the filter-branch, I would have some ancestor from the
> B series so that future pulls from the remote work, while having an ancestor from
> A, to make sure I can continue merging into C.  If history is rewritten, I'll get new
> revisions and lose ancestors.
> I'm beginning to thing that the cutting and pasting I'd like is conceptually impossible.

Hmm, this is pretty nasty.  Essentially, you want your repo to include
both sets of commits (so that it doesn't try to re-merge in the other
commits later), but you don't want to *see* them in git log.
Basically, you want git log to lie to you :)

Luckily, it already has this ability: it's called history simplification :)

Try this in your merged repo:

   git log .

(note the '.').

Without the dot, git log doesn't simplify any history, and you get
every change.  With the dot, it shows only commits that had a tangible
effect on the file in question (in this case, the top directory, which
includes *everything*).  Thus, a "git merge -s ours" gets eliminated.

Beware that it might eliminate other equally non-impactful commits, however.

The same trick also works with gitk.

Have fun,

Avery