How to merge from newer branch to older branches?

How to merge from newer branch to older branches?

[skillzero]

I'm not sure how to merge a topic branch onto multiple branches
without conflicts or unwanted changes. My workflow probably isn't
ideal, but I often run into cases like this and wondered if there's a
way to solve it with git.

We have a "future" branch for new development. I wanted to add a new
feature so I branched from "future" to create a branch named
"feature". I completed the work, but now they've decided they need
that on two older branches. It looks like this:

future o-o-o-o-o-o-o
        \   \       \
1.2          o-o-o   \
          \           \
1.1        o-o-o       \
                        \
feature                  o-o-o
                         a b c

I've read I should have started from the oldest common denominator
branch that I might ever want, but at the time I created the branch,
"future" was it because we thought 1.1 and 1.2 were locked down and
not going to add new features. But things change and I need to roll my
feature into 1.1 and 1.2.

What's the best way to merge changes on the "feature" branch into the
1.1 and 1.2 branches? If I try to checkout 1.1 then 'git merge
feature', it (obviously) merges in a bunch of changes from "future"
that I don't want (and has a bunch of conflicts). I also tried just
cherry-picking commits a, b, and c from the "feature" branch. That
mostly worked, but there were some surprising conflicts. Text that had
been inserted into a couple files (which had been untouched in 1.1 and
1.2) caused a conflict when it seemed like it should have just added
the lines without conflict. I realize it's difficult to know why it
conflicted without seeing the changes, but I'm just asking in a more
general way how I should be doing merges like this.

Is cherry-picking the best way? It seems pretty tedious because I have
to do each commit in reverse order (although I could script it). I was
hoping there was a way to say "apply all the changes from when
"feature" branched up to its tip". I would also like to preserve the
commit history rather than just making making one big commit of all
the changes.

Re: How to merge from newer branch to older branches?

[Jeff King]

On Tue, Apr 21, 2009 at 12:24:26PM -0700, skillzero@gmail.com wrote:

> We have a "future" branch for new development. I wanted to add a new
> feature so I branched from "future" to create a branch named
> "feature". I completed the work, but now they've decided they need
> that on two older branches. It looks like this:
> [...]
> I've read I should have started from the oldest common denominator
> branch that I might ever want, but at the time I created the branch,

Right. Junio branches all topics in git from "master", but then merges
promising ones into "next". When the topic is ready for master, he can
then just merge the topic branch, pulling in that topic but not the rest
of next.

> "future" was it because we thought 1.1 and 1.2 were locked down and
> not going to add new features. But things change and I need to roll my
> feature into 1.1 and 1.2.

Yep. Advanced planning is not always possible. :)

> What's the best way to merge changes on the "feature" branch into the
> 1.1 and 1.2 branches? If I try to checkout 1.1 then 'git merge

If it is just one or two commits, you probably just want to "git
cherry-pick" them onto your older branches.

If it is a larger series, or one which might see more development, you
might want to use "git rebase" to rewrite the branch as if it were based
on 1.1, and then merge that into 1.1, 1.2, and feature.

> feature', it (obviously) merges in a bunch of changes from "future"
> that I don't want (and has a bunch of conflicts). I also tried just
> cherry-picking commits a, b, and c from the "feature" branch. That
> mostly worked, but there were some surprising conflicts. Text that had
> been inserted into a couple files (which had been untouched in 1.1 and
> 1.2) caused a conflict when it seemed like it should have just added
> the lines without conflict. I realize it's difficult to know why it
> conflicted without seeing the changes, but I'm just asking in a more
> general way how I should be doing merges like this.

Yes, without seeing the conflicts it's hard to say. But presumably the
added lines provide context, so git was having a hard time trying to
figure out where in the old file your changes would go (you can think of
cherry-pick as the moral equivalent of "git show $COMMIT | git apply";
so you are basically applying a patch on top of something that it is
not quite the same).

If you are having trouble resolving conflicts, you might try using "git
mergetool" to fire up an external resolver that may provide a richer
interface (I use xxdiff with mergetool, myself).

> Is cherry-picking the best way? It seems pretty tedious because I have
> to do each commit in reverse order (although I could script it). I was
> hoping there was a way to say "apply all the changes from when
> "feature" branched up to its tip". I would also like to preserve the
> commit history rather than just making making one big commit of all
> the changes.

In its simplest form, rebase is basically cherry-picking a series of
commits. So that would make things a bit less tedious for you, but you
will probably still end up with the same set of conflicts.

-Peff

Re: How to merge from newer branch to older branches?

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Tue, Apr 21, 2009 at 12:24:26PM -0700, skillzero@gmail.com wrote:
>
>> We have a "future" branch for new development. I wanted to add a new
>> feature so I branched from "future" to create a branch named
>> "feature". I completed the work, but now they've decided they need
>> that on two older branches. It looks like this:
>> [...]
>> I've read I should have started from the oldest common denominator
>> branch that I might ever want, but at the time I created the branch,
>
> Right. Junio branches all topics in git from "master", but then merges
> promising ones into "next". When the topic is ready for master, he can
> then just merge the topic branch, pulling in that topic but not the rest
> of next.

No, I don't.

I often branch from somewhere older than 'master', often tip from 'maint'
sometimes even older.  And that is not necessarily because I am better
than other people in planning ahead.  There are branches that forked from
older series (like 1.6.0.X) that are merged to next and then master,
without ever getting merged to produce 1.6.0.(X+1).

> Yep. Advanced planning is not always possible. :)
>
>> What's the best way to merge changes on the "feature" branch into the
>> 1.1 and 1.2 branches? If I try to checkout 1.1 then 'git merge
>
> If it is just one or two commits, you probably just want to "git
> cherry-pick" them onto your older branches.

I would suggest against it.  Just like you did not manage to plan in
advance for these two patches to fork from older branches, you will regret
later that these two were still full of bugs and need to be fixed up with
more patches on top.  Even if they are only a few right now, keeping a
cleaned up series that applies to the oldest would be easier to manage in
the long run.

I.e., rebase feature onto somewhere older (like 1.1):

	git rebase --onto v1.1 feature~2 feature

then let q&a people merge that to v1.1 and v1.2, while keeping the tip of
the feature still open.  You will queue further fix-ups on that branch and
tell q&a people to merge again to get the fixed to the feature.

Re: How to merge from newer branch to older branches?

[Jeff King]

On Tue, Apr 21, 2009 at 10:13:23PM -0700, Junio C Hamano wrote:

> > Right. Junio branches all topics in git from "master", but then merges
> > promising ones into "next". When the topic is ready for master, he can
> > then just merge the topic branch, pulling in that topic but not the rest
> > of next.
> 
> No, I don't.
> 
> I often branch from somewhere older than 'master', often tip from 'maint'
> sometimes even older.  And that is not necessarily because I am better
> than other people in planning ahead.  There are branches that forked from
> older series (like 1.6.0.X) that are merged to next and then master,
> without ever getting merged to produce 1.6.0.(X+1).

Sure, but I assumed a new topic that came in would get branched from
'master', since that is presumably what the person developing it based
it on. Is that not true?

That is also what is in Documentation/howto/maintain-git.txt:

   - Anything unobvious that is applicable to 'master' (in other
     words, does not depend on anything that is still in 'next'
     and not in 'master') is applied to a new topic branch that
     is forked from the tip of 'master'.  This includes both
     enhancements and unobvious fixes to 'master'.  A topic
     branch is named as ai/topic where "ai" is typically
     author's initial and "topic" is a descriptive name of the
     topic (in other words, "what's the series is about").

   - An unobvious fix meant for 'maint' is applied to a new
     topic branch that is forked from the tip of 'maint'.  The
     topic is named as ai/maint-topic.

-Peff

Re: How to merge from newer branch to older branches?

[skillzero]

Based on the help from this discussion and the rebase documenation,
what I ended up doing that seemed to require the fewest conflicts is
to use git merge-base to find the merge-base of master and 1.1 (the
oldest branch I wanted the feature) then I found the commit
immediately before the first commit on the feature branch (feature~3
in this case or commit a^). Then I rebased feature:

git rebase --onto merge-base-of-1.1-and-master feature-before-first feature

Turning this (hopefully not mangled by gmail):

future o-o-o-o-o-o-o
        \   \       \
1.2      \   o-o-o   \
          \           \
1.1        o-o-o       \
                        \
feature                  o-o-o
                         a b c

into this:

future  o-o-o-o-o-o-o
        |\   \
1.2     | \   o-o-o
        |  \
1.1     |   o-o-o
         \
feature   o-o-o
          a b c

I can merge 'feature' into 1.1, 1.2, etc. then theoretically, I should
be able to apply bug fixes to 'feature' later and re-merge into 1.1,
1.2, etc. to pick up just the bug fixes, right?

When I merged 'feature' into 'v1.1', it resulted in the same conflicts
I originally ran into when cherry-picking, but when I looked at it, I
realized this was because I had manually integrated some changes into
1.1 from master (I'm trying to move our tree from CVS to git so I had
to manually move in some stuff from CVS instead of doing a proper git
merge).

It seems like this merge conflict is going to cause problems if I try
to re-merge to pick up a bug fix to 'feature' in the future. Looking
at the documentation, it sounds like 'git rerere' can help by
re-applying the manual conflict resolution I did?

BTW...Is there a way to find the first commit of a branch? I had to
just look at the log manually to see what I remembered to be the first
commit.

Re: How to merge from newer branch to older branches?

[Jeff King]

On Wed, Apr 22, 2009 at 10:44:06AM -0700, skillzero@gmail.com wrote:

> Then I rebased feature:
> 
> git rebase --onto merge-base-of-1.1-and-master feature-before-first feature
>
> [...]
> 
> I can merge 'feature' into 1.1, 1.2, etc. then theoretically, I should
> be able to apply bug fixes to 'feature' later and re-merge into 1.1,
> 1.2, etc. to pick up just the bug fixes, right?

Exactly.

> It seems like this merge conflict is going to cause problems if I try
> to re-merge to pick up a bug fix to 'feature' in the future. Looking
> at the documentation, it sounds like 'git rerere' can help by
> re-applying the manual conflict resolution I did?

It shouldn't. Remember that your merges into 1.1, 1.2, etc, created a
new merge base. So when you have bug fixes to make to this topic, you
don't put them on the "feature" branch, but onto this topic branch. Then
you re-merge the topic branch into 1.1., 1.2, feature, etc, and it will
look only at the new bugfix.

> BTW...Is there a way to find the first commit of a branch? I had to
> just look at the log manually to see what I remembered to be the first
> commit.

Sort of. Branches in git are really just refs pointing to the tip of the
branch. So once the tip advances, the ref no longer remembers "oh, this
is where I branched from". If there are other commits along the thing
you branched off of, you can calculate the merge-base of the thing you
branched from, and your branch.

However, the first entry in the reflog for that branch ref should
indicate branch creation (unless it is so old that it has expired).
Something like:

  $ git branch foo origin/master
  $ git reflog show foo
  0be9bc0 foo@{0}: branch: Created from origin/master

So 0be9bc0 would be the creation point in this case. But you might have
totally rewritten the branch after that point, so 0be9bc0 might not be a
useful value anymore.

-Peff

Re: How to merge from newer branch to older branches?

[skillzero]

On Wed, Apr 22, 2009 at 1:15 PM, Jeff King <peff@peff.net> wrote:
>
> On Wed, Apr 22, 2009 at 10:44:06AM -0700, skillzero@gmail.com wrote:
>
>> It seems like this merge conflict is going to cause problems if I try
>> to re-merge to pick up a bug fix to 'feature' in the future. Looking
>> at the documentation, it sounds like 'git rerere' can help by
>> re-applying the manual conflict resolution I did?
>
> It shouldn't. Remember that your merges into 1.1, 1.2, etc, created a
> new merge base. So when you have bug fixes to make to this topic, you
> don't put them on the "feature" branch, but onto this topic branch. Then
> you re-merge the topic branch into 1.1., 1.2, feature, etc, and it will
> look only at the new bugfix.

I'm not sure I understand. When I did the original rebase of "feature"
onto the merge-base of all the branches I wanted to merge to (v1.1 and
v1.2 in this case), the end result was that "feature" is now based on
the merge-base. When I merged "feature" into 1.1, I had to fix some
conflicts so in the log I see my conflict fix commit then a merge
commit, but "feature" wasn't changed (only v1.1 was).

I was thinking that if I find a bug in my original "feature" branch, I
would commit the fix to the "feature" branch then merge that into
v1.1, v1.2, master, etc. But I was thinking that when I tried to merge
"feature" into v1.1 (which previously needed a commit to fix
conflicts), I'd need to re-fix those same conflicts.

When I look at the log for v1.1 though, maybe I just misunderstood the
way the conflicts are resolved in git? I was thinking the conflicting
merge would end up as one big commit that's a combination of the
"feature"'s commits and my conflict fixes.

But in the log for v1.1, it looks like my conflict fixing commit may
have just "fixed" the v1.1 branch such that it could accept a
"feature" merge without conflict. If that's true then a future merge
of "feature" containing new commits should just merge without
conflict. That would be awesome if that's the way conflicting
resolving in git works.

> However, the first entry in the reflog for that branch ref should
> indicate branch creation (unless it is so old that it has expired).
> Something like:
>
>  $ git branch foo origin/master
>  $ git reflog show foo
>  0be9bc0 foo@{0}: branch: Created from origin/master
>
> So 0be9bc0 would be the creation point in this case. But you might have
> totally rewritten the branch after that point, so 0be9bc0 might not be a
> useful value anymore.

Thanks, that shows what I need. For my case, it happened to be @{58}.
Is there a way to reverse the order or a special syntax to say "0 from
the end"? Trying --reverse said "cannot combine --reverse with
--walk-reflogs" and trying @{-1} show me something else.

Re: How to merge from newer branch to older branches?

[Jeff King]

On Wed, Apr 22, 2009 at 02:01:01PM -0700, skillzero@gmail.com wrote:

> I'm not sure I understand. When I did the original rebase of "feature"
> onto the merge-base of all the branches I wanted to merge to (v1.1 and
> v1.2 in this case), the end result was that "feature" is now based on

Err, sorry, I was confusing your "future" branch and your "feature"
branch. Wherever I said "feature", I meant "future", and "topic" I meant
"feature". Yikes.

So you would make bug-fixes on "feature", and then just re-merge it to
1.1, 1.2, and future.

> the merge-base. When I merged "feature" into 1.1, I had to fix some
> conflicts so in the log I see my conflict fix commit then a merge
> commit, but "feature" wasn't changed (only v1.1 was).

Right. So now the merge-base between feature and 1.1 is the new merge
commit. And when you re-merge them, you will only look at things that
happened on the feature branch since that merge-base.

> I was thinking that if I find a bug in my original "feature" branch, I
> would commit the fix to the "feature" branch then merge that into
> v1.1, v1.2, master, etc. But I was thinking that when I tried to merge
> "feature" into v1.1 (which previously needed a commit to fix
> conflicts), I'd need to re-fix those same conflicts.

Nope, because the merge commit already records the state of the tree
once those conflicts are fixed. Now it's possible that the _bugfix_ may
have its own conflicts. But you shouldn't see the same conflicts again.

> When I look at the log for v1.1 though, maybe I just misunderstood the
> way the conflicts are resolved in git? I was thinking the conflicting
> merge would end up as one big commit that's a combination of the
> "feature"'s commits and my conflict fixes.

Sort of.  It is a new commit with two parents: the previous tip of v1.1,
and the tip of "feature". But its tree contains the state with all of
v1.1, all of feature's commits, and your fixes.

-Peff

Re: How to merge from newer branch to older branches?

[John M. Dlugosz]

skillzero@gmail.com wrote:
> 
> future o-o-o-o-o-o-o
>         \   \       \
> 1.2          o-o-o   \
>           \           \
> 1.1        o-o-o       \
>                         \
> feature                  o-o-o
>                          a b c
> 
> Is cherry-picking the best way? It seems pretty tedious because I have
> to do each commit in reverse order (although I could script it). I was
> hoping there was a way to say "apply all the changes from when
> "feature" branched up to its tip". I would also like to preserve the
> commit history rather than just making making one big commit of all
> the changes.

I think any other command that "just" moves those deltas would be the exact equivalent of 
cherry picking in terms of the conflicts it sees.

You can, however, collapse a,b,c into one commit and pick that into each place it is 
needed, calling it a good comment for the whole feature.  That might be more difficult to 
merge than doing one at a time, but if not, and the whole string is tedious (i.e. 50 
instead of 3, mostly incremental typo fixes and stuff) I can see the benefit of making it 
a gestalt.

Furthermore... (thinking out loud)... the conflicts are caused by things after the other 
branches, right?  After you do the resolving for (say) 1.2, then instead of picking a,b,c 
again (with the same merge issues), pick the one you put onto 1.2 already.  Work your way 
that way from branches that came off nearest to the original feature, back down toward the 
  root.  Each time pick the one you just planted, because it already contains merges part 
way down.  Then you only have to deal with the changes between those two branches.

So that's what I'd do:  pick a, b, and c onto the most recient other branch (1.2 in your 
drawing), fixing merge as I go.  Then reset mixed back to the original tip before the 
picks, and commit everything with a good name for the feature.  Then, pick that onto the 
next newest branch (1,1 in your drawing), etc.

--John