Commiting changes onto more than one branch

Commiting changes onto more than one branch

[Mike Jarmy]

Hi,

At my day job, I'm doing the groundwork for recommending that we
switch to a DVCS from a proprietary centralized VCS.  I find git's
branching ability very compelling, and I think we would use it
extensively.  I work on a very large project (many thousands of files)
that has been around for many years, and has had several different
releases.  Right now, each release has its own top level directory,
but I'd like to change that so we use branches instead, out of one
great big git repository.  My plan is to set up the repository such
that the initial state at switchover will have a branch for the
current state of each of our releases.  Lets say that the branches for
each release are called v1, v2, etc.

My question is this:  How do I manage a checkin for a bugfix that
affects, say, only branches v3, v4, and v5?

Suppose that I checkout the v3 branch, and fix the bug by editing
several different files.  (Lets assume for now that the files in
question have not diverged between any of the 3 branches, even though
tons of other files have changed).  How do I commit the bugfix into
all of v3, v4 and v5?  Clearly, merging the branches together would be
bad.  So I think what I should do is perform 3 different commits, but
I'm not quite sure how to juggle the git index (or stash or whatever)
to accomplish this.  This may be a really obvious question, but I'm a
confused git newbie.

Also, even though I may need to do 3 commits, it would be nice if the
commits were related together in some way, since in a sense they
represent only one action (namely, fixing the bug).  Is there a way to
do that, so that its clear in gitk that it was really one unified
thing?  The very worst thing about our current VCS is that it has no
concept of a 'commit', only individual file histories.  Git would fix
that for us, but it would be nirvana if we could group commits for a
given bugfix across branches somehow, while not merging the branches
together.

One last question -- lets make the problem slightly more complicated
by specify that some of the edited files changed between, say, v4 and
v5.  I know how to handle a simple merge conflict in git, but is there
anything different about my multi-branched, grouped-together case
here?  The answer to this question may be obvious once I understand
how to do the simpler, unconflicting checkin.

Thanks,
Mike Jarmy

Re: Commiting changes onto more than one branch

[Eugene Sajine]

>
> Suppose that I checkout the v3 branch, and fix the bug by editing
> several different files.  (Lets assume for now that the files in
> question have not diverged between any of the 3 branches, even though
> tons of other files have changed).  How do I commit the bugfix into
> all of v3, v4 and v5?  Clearly, merging the branches together would be
> bad.  So I think what I should do is perform 3 different commits, but
> I'm not quite sure how to juggle the git index (or stash or whatever)
> to accomplish this.  This may be a really obvious question, but I'm a
> confused git newbie.

It as not as clear for me why you think merge will be bad?
If you commit your changes to the v3, then merging to v4 and v5 (which
are not changed) should be very simple fast forward merge. Which means
just move the pointer to the last commit from v3

Am i missing something?

Thanks,
Eugene

Re: Commiting changes onto more than one branch

[Mike Jarmy]

On Wed, Nov 25, 2009 at 11:38 AM, Eugene Sajine <euguess@gmail.com> wrote:
>>
>> Suppose that I checkout the v3 branch, and fix the bug by editing
>> several different files.  (Lets assume for now that the files in
>> question have not diverged between any of the 3 branches, even though
>> tons of other files have changed).  How do I commit the bugfix into
>> all of v3, v4 and v5?  Clearly, merging the branches together would be
>> bad.  So I think what I should do is perform 3 different commits, but
>> I'm not quite sure how to juggle the git index (or stash or whatever)
>> to accomplish this.  This may be a really obvious question, but I'm a
>> confused git newbie.
>
> It as not as clear for me why you think merge will be bad?
> If you commit your changes to the v3, then merging to v4 and v5 (which
> are not changed) should be very simple fast forward merge. Which means
> just move the pointer to the last commit from v3
>
> Am i missing something?

I guess I didn't explain it too well, I made it sound like v3, v4 and
v5 were still more-or-less the same.  What I'm thinking about here is
a case where we have switched to git a while back, and then done a lot
of work on the various different branches, so that v3, v4 and v5 have
diverged very far from each other.  In that case, we would never want
to merge them together.

Re: Commiting changes onto more than one branch

[Thomas Rast]

Mike Jarmy wrote:
> Suppose that I checkout the v3 branch, and fix the bug by editing
> several different files.  (Lets assume for now that the files in
> question have not diverged between any of the 3 branches, even though
> tons of other files have changed).  How do I commit the bugfix into
> all of v3, v4 and v5?  Clearly, merging the branches together would be
> bad.  So I think what I should do is perform 3 different commits, but
> I'm not quite sure how to juggle the git index (or stash or whatever)
> to accomplish this.  This may be a really obvious question, but I'm a
> confused git newbie.

You can build the fix on top of the merge-base of v3, v4 and v5, i.e.

  git checkout -b myfix $(git merge-base v3 $(git merge-base v4 v5))
  # work
  git commit

and then merge it to each of the version branches:

  for b in v3 v4 v5; do
    git checkout $b
    git merge myfix
  done

So much for the theory.  In the model suggested in the gitworkflows(7)
manpage and used in git.git, v3 is contained in v4 and similar for v5.
This means that after merging (possibly several) fixes to v3, you can
merge v3 into v4 and v4 into v5 (and so on, through all versions) to
propagate the fixes.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Re: Commiting changes onto more than one branch

[Jakub Narebski]

Mike Jarmy <mjarmy@gmail.com> writes:

> My question is this:  How do I manage a checkin for a bugfix that
> affects, say, only branches v3, v4, and v5?

Take a look at "Resolving conflicts/dependencies between topic
branches early" blog post by Junio C Hamano (git maintainer) at 
http://gitster.livejournal.com/27297.html

In short the solution is to create separate topic branch for a bugfix,
branching off earliest place where it would be relevant, then merge
this bugfix branch into all development branches you need
(e.g. maint-v3, maint-v4, maint-v5, master).

This means:

  $ git checkout -b fix-frobulator--issue-1235 maint-v3
  <create commit or series of commits>
  
  $ git checkout maint-v3
  $ git merge fix-frobulator--issue-1235
  <resolve conflicts if any>

  $ git checkout maint-v4
  $ git merge fix-frobulator--issue-1235
  <resolve conflicts if any>

  [...]

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: Commiting changes onto more than one branch

[Avery Pennarun]

On Wed, Nov 25, 2009 at 11:47 AM, Mike Jarmy <mjarmy@gmail.com> wrote:
> I guess I didn't explain it too well, I made it sound like v3, v4 and
> v5 were still more-or-less the same.  What I'm thinking about here is
> a case where we have switched to git a while back, and then done a lot
> of work on the various different branches, so that v3, v4 and v5 have
> diverged very far from each other.  In that case, we would never want
> to merge them together.

What many people do is, in fact, to merge v3->v4->v5.

This isn't as crazy as it sounds.  Once upon a time, v4 was just an
earlier version of v3, right?  And when you fix a bug in v3, it was
usually also a bug in v4, right?  So in fact, for many projects, it's
safe to say that "after we created v4, all further changes to v3
should be propagated to v4."  And likewise from v4 to v5.

In that case, you'd simply do

   git checkout v3
     # commit your fix
   git checkout v4
   git merge v3

Now, setting that up in the *first* place can be a bit tricky, since
the way your imported history probably currently works, git doesn't
actually know that the history of v4 is a superset of the history of
v3; it thinks of them as two totally different histories, and merging
from one to the other will be completely disastrous.  So you have to
do a bit of setup first

   # manually make sure all your required patches from v3 are now in v4.
   # just do it the way you used to do it (the hard way)
   # now tell git that it's done:
   git checkout v4
   git merge -s ours v3

After that, future merges from v3 to v4 will be easy (the first set of
steps above) and include only the newer changes.

Note that merging fixes back from v4 to v3 is entirely different,
because you'll *never* want to take *all* the changes from v4 and put
them into v3.  The best thing to do is apply them to v3 first, then
merge them into v4, but of course that won't always be how developers
end up doing it.  In that case, you can backport them using 'git
cherry-pick' (see the git docs).

Note that using topic branches, as Thomas and Jakub mentioned, is
orthogonal to this method.  That is, your problem could be resolved by
doing that, or this, or both.  (Although if the histories really are
totally disjoint, you'll still need to do something like the '-s ours'
trick first.)  On my own projects, I do a bit of both methods; simple
bugfixes go straight to the earliest relevant release branch, but
bigger changes go on topic branches.

Have fun,

Avery

Re: Commiting changes onto more than one branch

[Mike Jarmy]

Jakub Narebski wrote:
> Take a look at "Resolving conflicts/dependencies between topic
> branches early" blog post by Junio C Hamano (git maintainer) at
> http://gitster.livejournal.com/27297.html
>
> In short the solution is to create separate topic branch for a bugfix,
> branching off earliest place where it would be relevant, then merge
> this bugfix branch into all development branches you need
> (e.g. maint-v3, maint-v4, maint-v5, master).

Thanks guys.  I think some cross between this and the cherry-picking
idea would work for us most of the time, if we go with the workflow
that I originally specified.  I like the branch-per-bugfix idea -- its
taking some time for me to free my mind to the point where I grok how
lightweight and flexible branching really is.  Just branch off from
the earliest affected release, naming the branch after the bug
('fix-frobulator--issue-1235').  When finished, merge/cherry-pick back
into the various branches.

Thomas's idea of using a more sophisticated workflow has definitely
got me thinking.  Back in the day, once we started working on a new
release, say v6, then the other branches to an extent become
mothballed except for bugfixes.  In that case, using a merge-base
approach would make sense.  More and more though, we have started to
build separate sub-projects on top of specific releases -- in other
words, we are simultaneously making a new product on top of v4, while
preparing to release v5, and getting ready to start work on v6 in
earnest, all the while fixing bugs across mulitple releases going all
the way back to v2.  Its getting very complicated :-).  And our
current VCS is starting to become an active hindrance in helping us
manage all this in a sane way.

What I'm going to do is set up a toy environment that mirrors what I
hope our final repository will look like.  Then I'll play with it for
a while and concoct corner-case scenarios and see how it holds up.
Once I have a workflow that I like, I can demonstrate it to my
colleagues in gitk, and we can think about how to make it better.

Thanks,
Mike Jarmy

Re: Commiting changes onto more than one branch

[Nicolas Pitre]

On Wed, 25 Nov 2009, Mike Jarmy wrote:

> My question is this:  How do I manage a checkin for a bugfix that
> affects, say, only branches v3, v4, and v5?
> 
> Suppose that I checkout the v3 branch, and fix the bug by editing
> several different files.  (Lets assume for now that the files in
> question have not diverged between any of the 3 branches, even though
> tons of other files have changed).  How do I commit the bugfix into
> all of v3, v4 and v5?  Clearly, merging the branches together would be
> bad.  So I think what I should do is perform 3 different commits, but
> I'm not quite sure how to juggle the git index (or stash or whatever)
> to accomplish this.  This may be a really obvious question, but I'm a
> confused git newbie.

Besides what other people have already suggested, you might simply elect 
to use 'git cherry-pick' on the v4 and v5 branches to copy the fix from 
the v3 branch.

> Also, even though I may need to do 3 commits, it would be nice if the
> commits were related together in some way, since in a sense they
> represent only one action (namely, fixing the bug).  Is there a way to
> do that, so that its clear in gitk that it was really one unified
> thing? 

You can use the -x argument with 'git cherry-pick'.  This won't create a 
formal history graph but at least the commit log will record a reference 
to the original fix.


Nicolas

Re: Commiting changes onto more than one branch

[Junio C Hamano]

Mike Jarmy <mjarmy@gmail.com> writes:

> I guess I didn't explain it too well, I made it sound like v3, v4 and
> v5 were still more-or-less the same.  What I'm thinking about here is
> a case where we have switched to git a while back, and then done a lot
> of work on the various different branches, so that v3, v4 and v5 have
> diverged very far from each other.  In that case, we would never want
> to merge them together.

I take it to mean that even though v[345] have diverged, the area that the
particular change you have in mind has to touch haven't changed since they
forked.  Otherwise you wouldn't be able to apply the same change to all of
them in the same form and instead would be making the logically same
change in physically different ways per branch, and you won't be asking
this question.

        o--o--o--o--o--o--o--o v4
       /
    --o--x--x--x--x--x--x--x v3

If you implemented your change at the tip of v3 branch and merge the
result to v4, you will pull _all_ of 'x' into v4 that may not be desirable
if the branches diverged a lot, of course.  

        o--o--o--o--o--o--o--o-------M v4
       /                            /
    --o--x--x--x--x--x--x--x       /
                            \     /
                             Y---Y your change(s) == v3

But nobody tells you to do this.

Instead, you can fork such a topic from the latest common.

        o--o--o--o--o--o--o--o v4
       /
    --o--x--x--x--x--x--x--x v3
       \
        Y---Y
            your change(s)

and merging this to v3 and v4 will have the desired effect.

        o--o--o--o--o--o--o--o---------M v4
       /                              /
    --o--x--x--x--x--x--x--x---M v3  /
       \                      /     /
        Y---Y----------------.-----.
            your change(s)

The merges will incorporate this particular change alone without dragging
anything else.  When you merge it to v4, none of the unrelated 'x' will be
merged into it.

In general you shouldn't fork a topic from the _tip_ of the oldest branch,
if the branches are not meant to be merged as a whole.

Of course, if this becomes cumbersome, you would adopt a better branch
management to keep the numbers of 'x' that shouldn't be in later branches
to the minimum.

Re: Commiting changes onto more than one branch

[Mike Jarmy]

Junio C Hamano wrote:
> I take it to mean that even though v[345] have diverged, the area that the
> particular change you have in mind has to touch haven't changed since they
> forked.

Correct.  Sometimes, there might be unrelated changes to a given file
or files, in which case conflict resolution will be done manually.
But generally speaking, bugfixes will tend to go on quite cleanly.

> Instead, you can fork such a topic from the latest common.
>
>        o--o--o--o--o--o--o--o v4
>       /
>    --o--x--x--x--x--x--x--x v3
>       \
>        Y---Y
>            your change(s)
>
> and merging this to v3 and v4 will have the desired effect.
>
>        o--o--o--o--o--o--o--o---------M v4
>       /                              /
>    --o--x--x--x--x--x--x--x---M v3  /
>       \                      /     /
>        Y---Y----------------.-----.
>            your change(s)
>
> The merges will incorporate this particular change alone without dragging
> anything else.  When you merge it to v4, none of the unrelated 'x' will be
> merged into it.

That sounds a whole lot like what I need.

Right now I'm thinking that the right approach is that once we have
released v3, and started working on v4 heavily, we will probably not
want to check any commits directly into v3.  Bugfixes will have their
own branch.   If there is a whole new project or whatever being done
on top of v3, it will have its own branch as well.

So once v3 is in beta or whatever and we declare a code freeze, we
could have a rule that all commits must be merge-commits coming from
dedicated branches with descriptive, intelligible names.  For each
dedicated branch, some thought will have to be given as to just where
off of v3 to branch it from (not just carelessly off the latest tip).
Meanwhile, new development will continue on v4, with lots of commits
going right into the branch (or into v4 sub-branches with occasional
merge-commits into v4).

Re: Commiting changes onto more than one branch

[Daniel Barkalow]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2869 bytes --]

On Wed, 25 Nov 2009, Mike Jarmy wrote:

> Junio C Hamano wrote:
> > I take it to mean that even though v[345] have diverged, the area that the
> > particular change you have in mind has to touch haven't changed since they
> > forked.
> 
> Correct.  Sometimes, there might be unrelated changes to a given file
> or files, in which case conflict resolution will be done manually.
> But generally speaking, bugfixes will tend to go on quite cleanly.
> 
> > Instead, you can fork such a topic from the latest common.
> >
> >        o--o--o--o--o--o--o--o v4
> >       /
> >    --o--x--x--x--x--x--x--x v3
> >       \
> >        Y---Y
> >            your change(s)
> >
> > and merging this to v3 and v4 will have the desired effect.
> >
> >        o--o--o--o--o--o--o--o---------M v4
> >       /                              /
> >    --o--x--x--x--x--x--x--x---M v3  /
> >       \                      /     /
> >        Y---Y----------------.-----.
> >            your change(s)
> >
> > The merges will incorporate this particular change alone without dragging
> > anything else.  When you merge it to v4, none of the unrelated 'x' will be
> > merged into it.
> 
> That sounds a whole lot like what I need.
> 
> Right now I'm thinking that the right approach is that once we have
> released v3, and started working on v4 heavily, we will probably not
> want to check any commits directly into v3.  Bugfixes will have their
> own branch.   If there is a whole new project or whatever being done
> on top of v3, it will have its own branch as well.
> 
> So once v3 is in beta or whatever and we declare a code freeze, we
> could have a rule that all commits must be merge-commits coming from
> dedicated branches with descriptive, intelligible names.  For each
> dedicated branch, some thought will have to be given as to just where
> off of v3 to branch it from (not just carelessly off the latest tip).
> Meanwhile, new development will continue on v4, with lots of commits
> going right into the branch (or into v4 sub-branches with occasional
> merge-commits into v4).

It's worth noting that you don't have to have any general visibility to 
the dedicated branch. If you look at Junio's graph, there are only two 
branches marked: v3 and v4. The stuff in the bottom line only appears as 
the second parent in the merge commits. Git's data model doesn't require a 
branch for a commit to be "on" like many other VCSes, so it's perfectly 
reasonable for a developer to make a temporary branch, create the fix in 
some private repository, merge it to each applicable official branch, and 
then discard the temporary branch. (The temporary branch is only needed at 
all so that the developer has some way to refer to the fix while merging 
it, and the name only matters in that humans will see it in the merge 
commit message.)

	-Daniel
*This .sig left intentionally blank*

Re: Commiting changes onto more than one branch

[Andreas Schwab]

Daniel Barkalow <barkalow@iabervon.org> writes:

> (The temporary branch is only needed at all so that the developer has
> some way to refer to the fix while merging it, and the name only
> matters in that humans will see it in the merge commit message.)

If you merge by commit-ish the branch name does not even appear in the
commit message.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."