'git rebase' silently drops changes?

'git rebase' silently drops changes?

[Sergey Organov]

Hello,

I recently ran into an annoying problem: 'git rebase' apparently
silently drops changes in non-conflicting paths of merge commits
(git version 1.9.3). 

Is it a bug or feature? Is there a way to flatten history using rebase,
yet preserve manual changes found in merge commits?

Here is simplified reproduction of what I've encountered:

<SCRIPT>
git init t
cd t
git config rerere.enabled true

echo "I" > a; git add a
echo "I" > b; git add b
git commit -am "I"

git checkout -b test

echo "B" >> b; git commit -m "B" -a

git checkout master

echo "A" >> a
git commit -am "A"

git merge --no-edit test

# Clean merge, but result didn't compile, so I fixed it and
# amended the merge:
echo "Precious!" >> a # [!] This is modification that gets lost
git commit --amend --no-edit -a
cat a

# Now rebase my work.
git rebase -f HEAD~1

# What? Where is my "Precious" change in "a"???
cat a
</SCRIPT>

I.e., the modification marked [!] was silently lost during rebase!

-- 
Sergey.

Re: 'git rebase' silently drops changes?

[Sebastian Schuberth]

On 06.02.2015 22:28, Sergey Organov wrote:

> # Now rebase my work.
> git rebase -f HEAD~1
>
> # What? Where is my "Precious" change in "a"???
> cat a
> </SCRIPT>
>
> I.e., the modification marked [!] was silently lost during rebase!

Just a wild guess: Maybe because you omitted "-p" / "--preserve-merges" 
from "git rebase"?

-- 
Sebastian Schuberth

Re: 'git rebase' silently drops changes?

[Johannes Sixt]

Am 07.02.2015 um 22:32 schrieb Sebastian Schuberth:
> On 06.02.2015 22:28, Sergey Organov wrote:
> 
>> # Now rebase my work.
>> git rebase -f HEAD~1
>>
>> # What? Where is my "Precious" change in "a"???
>> cat a
>> </SCRIPT>
>>
>> I.e., the modification marked [!] was silently lost during rebase!
> 
> Just a wild guess: Maybe because you omitted "-p" / "--preserve-merges"
> from "git rebase"?

No, that would not help. --preserve-merges repeats the merge, but does
not apply the amendment.

It's just how rebase works: It omits merge commits when it linearizes
history.

Sergey, it is impossible for git rebase to decide to which rebased
commit the amendement applies. It doesn't even try to guess. It's the
responsibility of the user to apply the amendment to the correct commit.

-- Hannes

Re: 'git rebase' silently drops changes?

[Sergey Organov]

Johannes Sixt <j6t@kdbg.org> writes:

> Am 07.02.2015 um 22:32 schrieb Sebastian Schuberth:
>> On 06.02.2015 22:28, Sergey Organov wrote:
>> 
>>> # Now rebase my work.
>>> git rebase -f HEAD~1
>>>
>>> # What? Where is my "Precious" change in "a"???
>>> cat a
>>> </SCRIPT>
>>>
>>> I.e., the modification marked [!] was silently lost during rebase!
>> 
>> Just a wild guess: Maybe because you omitted "-p" / "--preserve-merges"
>> from "git rebase"?
>
> No, that would not help. --preserve-merges repeats the merge, but does
> not apply the amendment.

Really? Why? Here the valid concern you gave below doesn't even apply!

Check... yes, git silently drops amend even with --preserve-merges
(script to reproduce at the end[1])! How comes?

> It's just how rebase works: It omits merge commits when it linearizes
> history.
>
> Sergey, it is impossible for git rebase to decide to which rebased
> commit the amendement applies. It doesn't even try to guess. It's the
> responsibility of the user to apply the amendment to the correct
> commit.

Yeah, this sounds reasonable, /except/ git even gives no warning when it
drops amendments. Shouldn't 'git rebase' rather consider merge amendment
a kind of conflict?


[1] To reproduce amend drop by "git rebase --preserve-merges":

<SCRIPT>
git init t
cd t
git config rerere.enabled false # doesn't actually matter either way.

echo "I" > a; git add a
echo "I" > b; git add b
git commit -aqm "I"
git tag start

git checkout -b test

echo "B" >> b; git commit -m "B" -a

git checkout master

echo "A" >> a
git commit -aqm "A"

git merge --no-edit test
git branch -d test

# Clean merge, but result didn't compile, so I fixed it and
# amended the merge:
echo "Precious!" >> a # [!] This is modification that gets lost
git commit --amend --no-edit -aq
cat a

# Make a change earlier in history, to rebase my work on top of it.
git co -q start
git co -b test
echo "C" > c; git add c
git commit -aqm "C"

# Now rebase my work.
git co master
git rebase --preserve-merges --no-fork-point test

# What? Where is my "Precious" change in "a"???
cat a

</SCRIPT>

-- Sergey.

Re: 'git rebase' silently drops changes?

[Johannes Sixt]

Am 09.02.2015 um 13:53 schrieb Sergey Organov:
> Johannes Sixt <j6t@kdbg.org> writes:
> 
>> Am 07.02.2015 um 22:32 schrieb Sebastian Schuberth:
>>> On 06.02.2015 22:28, Sergey Organov wrote:
>>>
>>>> # Now rebase my work.
>>>> git rebase -f HEAD~1
>>>>
>>>> # What? Where is my "Precious" change in "a"???
>>>> cat a
>>>> </SCRIPT>
>>>>
>>>> I.e., the modification marked [!] was silently lost during rebase!
>>>
>>> Just a wild guess: Maybe because you omitted "-p" / "--preserve-merges"
>>> from "git rebase"?
>>
>> No, that would not help. --preserve-merges repeats the merge, but does
>> not apply the amendment.
> 
> Really? Why? Here the valid concern you gave below doesn't even apply!

--preserve-merges was bolted on to git-rebase. The first implementation
just re-computed the merge, and rebase would be interrupted if the merge
was not clean. This was good enough for many.

Later --preserve-merges was abused to replay not only integration
branches, but branchy history in general. At that time, the feature was
deemed wide spread enough that changing its behavior was a no-go.

There you have it.

If you want a version of --preserve-merges that does what *you* need,
consider this commit:

  git://repo.or.cz/git/mingw/j6t.git rebase-p-first-parent

Use it like this:

  git rebase -i -p --first-parent ...

Beware, its implementation is incomplete: if the rebase is interrupted,
then 'git rebase --continue' behaves as if --first-parent were not given.

>> it is impossible for git rebase to decide to which rebased
>> commit the amendement applies. It doesn't even try to guess. It's the
>> responsibility of the user to apply the amendment to the correct
>> commit.
> 
> Yeah, this sounds reasonable, /except/ git even gives no warning when it
> drops amendments. Shouldn't 'git rebase' rather consider merge amendment
> a kind of conflict?

There is work in progress where a merge is computed entirely in-memory
(without relying on files in the worktree). It could be used to detect
whether there are any changes beyond the automatic merge results, and
they could be warned about.

-- Hannes

Re: 'git rebase' silently drops changes?

[Sergey Organov]

Johannes Sixt <j6t@kdbg.org> writes:

> Am 09.02.2015 um 13:53 schrieb Sergey Organov:

[...]

> If you want a version of --preserve-merges that does what *you* need,
> consider this commit:
>
>   git://repo.or.cz/git/mingw/j6t.git rebase-p-first-parent
>
> Use it like this:
>
>   git rebase -i -p --first-parent ...

Thanks a lot, this sounds promising! I've read the message for this
commit and it mentions no drawbacks. Are you aware of any?

> Beware, its implementation is incomplete: if the rebase is interrupted,
> then 'git rebase --continue' behaves as if --first-parent were not
> given.

Just never did get round to it, or something more fundamental?

To be useful for me, it also needs a support for 'git pull' to pass this
flag to 'git rebase', but that I think I can easily fix myself.

>>> it is impossible for git rebase to decide to which rebased
>>> commit the amendement applies. It doesn't even try to guess. It's the
>>> responsibility of the user to apply the amendment to the correct
>>> commit.
>> 
>> Yeah, this sounds reasonable, /except/ git even gives no warning when it
>> drops amendments. Shouldn't 'git rebase' rather consider merge amendment
>> a kind of conflict?
>
> There is work in progress where a merge is computed entirely in-memory
> (without relying on files in the worktree). It could be used to detect
> whether there are any changes beyond the automatic merge results, and
> they could be warned about.

Nice to hear there are chances to improve this in the future.

Thanks again!

-- Sergey.

Re: 'git rebase' silently drops changes?

[Johannes Sixt]

Am 10.02.2015 um 12:46 schrieb Sergey Organov:
> Johannes Sixt <j6t@kdbg.org> writes:
> 
>> Am 09.02.2015 um 13:53 schrieb Sergey Organov:
> 
> [...]
> 
>> If you want a version of --preserve-merges that does what *you* need,
>> consider this commit:
>>
>>   git://repo.or.cz/git/mingw/j6t.git rebase-p-first-parent
>>
>> Use it like this:
>>
>>   git rebase -i -p --first-parent ...
> 
> Thanks a lot, this sounds promising! I've read the message for this
> commit and it mentions no drawbacks. Are you aware of any?

Except for this...

>> Beware, its implementation is incomplete: if the rebase is interrupted,
>> then 'git rebase --continue' behaves as if --first-parent were not
>> given.

... not anything that I would care about. Of course, you can't rebase
branchy history when this option is specified, but that's the whole point.

> Just never did get round to it, or something more fundamental?

Nothing fundamental. Just has to store the option in the rebase state
like all the other options.

-- Hannes