[ANNOUNCE] git reintegrate v0.3; manager of integration branches

[ANNOUNCE] git reintegrate v0.3; manager of integration branches

[Felipe Contreras]

Hi,

git reintegrate is a helper tool to manage integration branches, it
has all the options of other known tools.

This is a rewrite of John Keeping's git-integration in Ruby, it has
essentially the same features and passes all the git-integration
tests, but it has more features.

One feature that is missing from git-integration is the ability to
parse existing integration branches.

To give a try you can do:

  git clone https://github.com/gitster/git/
  cd git
  git fetch -u origin 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*'
  git checkout pu
  git reintegrate --generate pu master

Which will generate the integration instructions for you:

  % git reintegrate --cat
  base master
  merge jl/submodule-mv

    Moving a regular file in a repository with a .gitmodules file was
    producing a warning 'Could not find section in .gitmodules where
    path=<filename>'.

  merge ap/remote-hg-unquote-cquote

    A fast-import stream expresses a pathname with funny characters by
    quoting them in C style; remote-hg remote helper forgot to unquote
    such a path.

  merge jk/for-each-ref-skip-parsing
  merge jk/pack-corruption-post-mortem
  merge jk/reset-p-current-head-fix

    "git reset -p HEAD" has codepath to special case it from resetting
    to contents of other commits, but recent change broke it.

  ...

It also has support for "evil merges", so it should be perfectly
usable for git.git maintenance.

You can edit the instructions with `git reintegrate --edit`.

The simplest way to begin an integration branch is with:

  git reintegrate --create pu master
  git reintegrate --add=branch1 --add=branch2 --add=branch3

To generate the integration branch run `git reintegrate --rebuild`, if
there are merge conflicts, solve them and continue with `git
reintegrate --continue`.

Despite having more features, the code is actually smaller thanks to
Ruby awesomeness.

Enjoy.

https://github.com/felipec/git-reintegrate

Changes since v0.1:

 * Add support for empty commits
 * Add support for pause command
 * Update manpage
 * Add bash completion

Felipe Contreras (26):
      Add copyright and license notices
      Fix EDITOR support with arguments
      Trivial style cleanup
      Improve command regex
      Add support for empty commits
      test: improve check_int()
      Add support for 'pause' command
      doc: rename manpage file
      doc: update options
      doc: add description
      doc: add missing instruction commands
      doc: cleanup . command
      Verify branches after parsing
      test: fix test names
      Remove unused statements
      Update README
      test: cleanup instruction sheets
      Update copyright notices
      Add bash completion
      build: add installation stuff
      readme: add installation instructions
      Add gitignore for documentation
      trvis: initial configuration
      travis: add verbosity
      test: add test-lib helper
      travis: remove Ruby 1.8

-- 
Felipe Contreras

Re: [ANNOUNCE] git reintegrate v0.3; manager of integration branches

[Junio C Hamano]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> ...
> Which will generate the integration instructions for you:
>
>   % git reintegrate --cat
>   base master
>   merge jl/submodule-mv
>
>     Moving a regular file in a repository with a .gitmodules file was
>     producing a warning 'Could not find section in .gitmodules where
>     path=<filename>'.
>
>   merge ap/remote-hg-unquote-cquote
>
> It also has support for "evil merges", so it should be perfectly
> usable for git.git maintenance.

Yeah, it sounds like it is almost there.

I think the infrastructure to maintain "What's cooking" could be
updated to use these comments after "merge" instructions if I wanted
to.

I build two branches on top of 'master', one is called 'jch' and has
a marker line somewhere that says '### match next' that is turned
into an empty commit, and 'pu' that is built on top of the tip of
'jch'.  The marker line is used to apply only an earlier part of the
instruction stream to build 'jch' on top of 'master' on top of
'next' (i.e. "base master" in the above example will not be applied
to hard-reset 'next' to match master) and stop there, and is meant
to be a way to sanity check 'next' (which is made by repeated
incremental merges on top of 'master' without rewinding) by
comparing the "### match next" commit between 'master' and 'jch'
(which is made afresh from 'master' by taking only the necessary
topics).  They must match or I caught a possible mismerge on 'next'.

I presume that the workflow can be mimicked by having another branch
'match-next' and building it on top of 'master', and then building
'jch' on top of it, and then building 'pu' on top of it.  Then you
should be able to play 'match-next' instruction on top of 'next'
(provided that there is a way to tell it to replay on HEAD and
ignore "base" and have "merge" instruction become a no-op when the
branch has already been merged).

Fun.

Re: [ANNOUNCE] git reintegrate v0.3; manager of integration branches

[Felipe Contreras]

Junio C Hamano wrote:
> Felipe Contreras <felipe.contreras@gmail.com> writes:
> 
> > ...
> > Which will generate the integration instructions for you:
> >
> >   % git reintegrate --cat
> >   base master
> >   merge jl/submodule-mv
> >
> >     Moving a regular file in a repository with a .gitmodules file was
> >     producing a warning 'Could not find section in .gitmodules where
> >     path=<filename>'.
> >
> >   merge ap/remote-hg-unquote-cquote
> >
> > It also has support for "evil merges", so it should be perfectly
> > usable for git.git maintenance.
> 
> Yeah, it sounds like it is almost there.
> 
> I think the infrastructure to maintain "What's cooking" could be
> updated to use these comments after "merge" instructions if I wanted
> to.
> 
> I build two branches on top of 'master', one is called 'jch' and has
> a marker line somewhere that says '### match next' that is turned
> into an empty commit, and 'pu' that is built on top of the tip of
> 'jch'.  The marker line is used to apply only an earlier part of the
> instruction stream to build 'jch' on top of 'master' on top of
> 'next' (i.e. "base master" in the above example will not be applied
> to hard-reset 'next' to match master) and stop there, and is meant
> to be a way to sanity check 'next' (which is made by repeated
> incremental merges on top of 'master' without rewinding) by
> comparing the "### match next" commit between 'master' and 'jch'
> (which is made afresh from 'master' by taking only the necessary
> topics).  They must match or I caught a possible mismerge on 'next'.
> 
> I presume that the workflow can be mimicked by having another branch
> 'match-next' and building it on top of 'master', and then building
> 'jch' on top of it, and then building 'pu' on top of it.  Then you
> should be able to play 'match-next' instruction on top of 'next'
> (provided that there is a way to tell it to replay on HEAD and
> ignore "base" and have "merge" instruction become a no-op when the
> branch has already been merged)

Or have an option to specify a dynamic instruction sheet, so you can cat
the instructions of 'match-next' and replace the base. However, I don't
see the point of re-applying the branches for 'next' if you already know
that 'next' and 'match-next' are the same.

I would have two branches: 1) 'match-next' (or 'integration-next') that
has the instructions to build 'next' on top of 'master', and 2) 'pu',
which has the instructions to build 'pu' on top of 'next'.

-- 
Felipe Contreras

Re: [ANNOUNCE] git reintegrate v0.3; manager of integration branches

[Junio C Hamano]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> Or have an option to specify a dynamic instruction sheet, so you can cat
> the instructions of 'match-next' and replace the base. However, I don't
> see the point of re-applying the branches for 'next' if you already know
> that 'next' and 'match-next' are the same.

The output from Reintegrate script (in 'todo') only lists the names
of topic branches (or something like "xx/topic~4" when the topic is
not fully merged yet), and a topic branch may acquire a follow-up
change (or two) on top (there is a machinery to see if xx/topic~4
is still valid in such a case and update the offset as needed).

Rebuilding 'jch' on top of 'master' with the same insn sheet will
then merge the updated topic branch in its entirety, and the new
commits on the topic branch somehow needs to go to 'next' for the
"match next" on 'jch' and 'next' to be in sync (in addition, updated
'master' must be merged to 'next', but that goes without saying).

In other words, I already know that 'next' and "match next" are not
the same, that they must become the same, and there needs a way to
make them so.

And that is done by re-running the insn sheet for 'jch' up to the
"match next" mark, merging the topic that are not fully merged to
'next' while ignoring the ones that already are fully in 'next'.

Re: [ANNOUNCE] git reintegrate v0.3; manager of integration branches

[Felipe Contreras]

Junio C Hamano wrote:
> Felipe Contreras <felipe.contreras@gmail.com> writes:
> 
> > Or have an option to specify a dynamic instruction sheet, so you can cat
> > the instructions of 'match-next' and replace the base. However, I don't
> > see the point of re-applying the branches for 'next' if you already know
> > that 'next' and 'match-next' are the same.
> 
> The output from Reintegrate script (in 'todo') only lists the names
> of topic branches (or something like "xx/topic~4" when the topic is
> not fully merged yet), and a topic branch may acquire a follow-up
> change (or two) on top (there is a machinery to see if xx/topic~4
> is still valid in such a case and update the offset as needed).
> 
> Rebuilding 'jch' on top of 'master' with the same insn sheet will
> then merge the updated topic branch in its entirety, and the new
> commits on the topic branch somehow needs to go to 'next' for the
> "match next" on 'jch' and 'next' to be in sync (in addition, updated
> 'master' must be merged to 'next', but that goes without saying).
> 
> In other words, I already know that 'next' and "match next" are not
> the same, that they must become the same, and there needs a way to
> make them so.
> 
> And that is done by re-running the insn sheet for 'jch' up to the
> "match next" mark, merging the topic that are not fully merged to
> 'next' while ignoring the ones that already are fully in 'next'.

There could be a new --merge-missing option that takes the instruction
sheet of an integration branch (e.g. 'match-next'), ignores the 'base'
applies them in 'HEAD' but only when the topic branch isn't already in
'HEAD'.

I'm not sure what would be the usefulness of using things like
'xx/topic~4'.

-- 
Felipe Contreras

Re: [ANNOUNCE] git reintegrate v0.3; manager of integration branches

[Junio C Hamano]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> I'm not sure what would be the usefulness of using things like
> 'xx/topic~4'.

As a notation it is not very pretty ;-).

Imagine that xx/topic is about a multistep introduction of a
backward incompatible feature.  The beginning part of the series up
to xx/topic~4 are the step to start warning (i.e. "will change---do
this to keep the old one or do that to live in the future"),
xx/topic~3..xx/topic~1 are the next step to flip the default and
still keep warning (i.e. "have changed---do this to keep the old one
or do that to live in the present"), and the final xx/topic is the
endgame where everybody lives with the new default with no warning,
without having to know what the old default was.

While the topic is being prepared for the next release, the insn
sheet for 'jch' would have xx/topic~4 before "match next" marker,
and then an entry for xx/topic~1 somewhere after it, and finally an
entry for xx/topic (i.e. the tip, final commit).  When the first
step cooked well enough in 'next', selected entries of 'jch' insn
sheet before "match next" ones are used to merge them to 'master'
and the part up to xx/topic~4 (but not later patches on the topic
branch) will be part of the upcoming release.

You could simulate that with multiple branches stacked on top of
each other, but there are times when keeping things together in a
single branch is more handy.

In restrospect, if I found xx/topic~4 too ugly, I might have instead
made "branches stacked on top of each other" easier to manage, but
having Reintegrate support "in the middle of a branch" was simpler.
They are both OK solutions to the same problem, and I didn't have to
solve it both ways ;-)

Re: [ANNOUNCE] git reintegrate v0.3; manager of integration branches

[Felipe Contreras]

On Tue, May 20, 2014 at 5:47 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Felipe Contreras <felipe.contreras@gmail.com> writes:
>
>> I'm not sure what would be the usefulness of using things like
>> 'xx/topic~4'.
>
> As a notation it is not very pretty ;-).
>
> Imagine that xx/topic is about a multistep introduction of a
> backward incompatible feature.  The beginning part of the series up
> to xx/topic~4 are the step to start warning (i.e. "will change---do
> this to keep the old one or do that to live in the future"),
> xx/topic~3..xx/topic~1 are the next step to flip the default and
> still keep warning (i.e. "have changed---do this to keep the old one
> or do that to live in the present"), and the final xx/topic is the
> endgame where everybody lives with the new default with no warning,
> without having to know what the old default was.
>
> While the topic is being prepared for the next release, the insn
> sheet for 'jch' would have xx/topic~4 before "match next" marker,
> and then an entry for xx/topic~1 somewhere after it, and finally an
> entry for xx/topic (i.e. the tip, final commit).  When the first
> step cooked well enough in 'next', selected entries of 'jch' insn
> sheet before "match next" ones are used to merge them to 'master'
> and the part up to xx/topic~4 (but not later patches on the topic
> branch) will be part of the upcoming release.
>
> You could simulate that with multiple branches stacked on top of
> each other, but there are times when keeping things together in a
> single branch is more handy.
>
> In restrospect, if I found xx/topic~4 too ugly, I might have instead
> made "branches stacked on top of each other" easier to manage, but
> having Reintegrate support "in the middle of a branch" was simpler.
> They are both OK solutions to the same problem, and I didn't have to
> solve it both ways ;-)

Yes, I see how xx/topic~4 would be useful in the instruction sheet, I
just didn't see it would be useful to generate that from an existing
integration branch. After the explanation above I see how it could be
useful to some people (though not all). I'll implement that.

-- 
Felipe Contreras

Re: [ANNOUNCE] git reintegrate v0.3; manager of integration branches

[Felipe Contreras]

Junio C Hamano wrote:
> I presume that the workflow can be mimicked by having another branch
> 'match-next' and building it on top of 'master', and then building
> 'jch' on top of it, and then building 'pu' on top of it.  Then you
> should be able to play 'match-next' instruction on top of 'next'
> (provided that there is a way to tell it to replay on HEAD and
> ignore "base" and have "merge" instruction become a no-op when the
> branch has already been merged).

Done.

% git checkout next
% git reintegrate --apply match-next

https://github.com/felipec/git-reintegrate/commit/036395b

-- 
Felipe Contreras

Re: [ANNOUNCE] git reintegrate v0.3; manager of integration branches

[Felipe Contreras]

Felipe Contreras wrote:
> Yes, I see how xx/topic~4 would be useful in the instruction sheet, I
> just didn't see it would be useful to generate that from an existing
> integration branch. After the explanation above I see how it could be
> useful to some people (though not all). I'll implement that.

Actually it was already supported.

https://github.com/felipec/git-reintegrate/commit/f3aa558

-- 
Felipe Contreras