[PATCH] Add a reminder test case for a merge with F/D transition

[PATCH] Add a reminder test case for a merge with F/D transition

[Alex Riesen]

[-- Attachment #1: Type: text/plain, Size: 1547 bytes --]

The problem is that if a file was replaced with a directory containing
another file with the same content and mode, an attempt to merge it
with a branch descended from a commit before this F->D transition will
cause merge-recursive to break. It breaks even if there were no
conflicting changes on that other branch.

Originally reported by Anders Melchiorsen.

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---

2009/5/11 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
>
> Maybe you can turn this into a patch adding a test (with
> test_expect_failure to mark it as a bug)?  This would make debugging a lot
> easier, as a non-installed Git could be tested.

Here.

 t/t6020-merge-df.sh |   23 +++++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/t/t6020-merge-df.sh b/t/t6020-merge-df.sh
index a19d49d..b62b52a 100755
--- a/t/t6020-merge-df.sh
+++ b/t/t6020-merge-df.sh
@@ -22,4 +22,27 @@ git commit -m "File: dir"'

 test_expect_code 1 'Merge with d/f conflicts' 'git merge "merge msg" B master'

+test_expect_failure 'F/D conflict' '
+	git reset --hard &&
+	git checkout master &&
+	rm .git/index &&
+
+	mkdir before &&
+	echo FILE >before/one &&
+	echo FILE >after &&
+	git add . &&
+	git commit -mfirst &&
+
+	rm -f after &&
+	git mv before after &&
+	git commit -mmove &&
+
+	git checkout -b para HEAD^ &&
+	echo COMPLETELY ANOTHER FILE >another &&
+	git add . &&
+	git commit -mpara &&
+
+	git merge master
+'
+
 test_done
-- 
1.6.3.28.ga852b

[-- Attachment #2: 0001-Add-a-reminder-test-case-for-a-merge-with-F-D-transi.txt --]
[-- Type: text/plain, Size: 1459 bytes --]

From 05433a6f896c40e4a039815d86613d7e351ef0da Mon Sep 17 00:00:00 2001
From: Alex Riesen <raa.lkml@gmail.com>
Date: Mon, 11 May 2009 11:31:42 +0200
Subject: [PATCH] Add a reminder test case for a merge with F/D transition

The problem is that if a file was replaced with a directory containing
another file with the same content and mode, an attempt to merge it
with a branch descended from a commit before this F->D transition will
cause merge-recursive to break. It breaks even if there were no
conflicting changes on that other branch.

Originally reported by Anders Melchiorsen.

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---
 t/t6020-merge-df.sh |   23 +++++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/t/t6020-merge-df.sh b/t/t6020-merge-df.sh
index a19d49d..b62b52a 100755
--- a/t/t6020-merge-df.sh
+++ b/t/t6020-merge-df.sh
@@ -22,4 +22,27 @@ git commit -m "File: dir"'
 
 test_expect_code 1 'Merge with d/f conflicts' 'git merge "merge msg" B master'
 
+test_expect_failure 'F/D conflict' '
+	git reset --hard &&
+	git checkout master &&
+	rm .git/index &&
+
+	mkdir before &&
+	echo FILE >before/one &&
+	echo FILE >after &&
+	git add . &&
+	git commit -mfirst &&
+
+	rm -f after &&
+	git mv before after &&
+	git commit -mmove &&
+
+	git checkout -b para HEAD^ &&
+	echo COMPLETELY ANOTHER FILE >another &&
+	git add . &&
+	git commit -mpara &&
+
+	git merge master
+'
+
 test_done
-- 
1.6.3.28.ga852b

[PATCH] Fix for a merge where a branch has an F->D transition

[Alex Riesen]

Some path names which transitioned from file to a directory were not
updated in the final part of the merge (loop around unmerged entries in
merge_trees), because the branch in process_renames which filtered out
updates for the files with the same content ("merged same as existing")
has left the rename entry in processed state. In this case, the
processing cannot be finished at the process_renames phase (because
the old file still blocks creation of directory where new files should
appear), and must be postponed until the update_entry phase.

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
---

Alex Riesen, Mon, May 11, 2009 11:42:17 +0200:
> The problem is that if a file was replaced with a directory containing
> another file with the same content and mode, an attempt to merge it
> with a branch descended from a commit before this F->D transition will
> cause merge-recursive to break. It breaks even if there were no
> conflicting changes on that other branch.
> 
> 2009/5/11 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
> >
> > Maybe you can turn this into a patch adding a test (with
> > test_expect_failure to mark it as a bug)?  This would make debugging a lot
> > easier, as a non-installed Git could be tested.
> 
> Here.
> 
>  t/t6020-merge-df.sh |   23 +++++++++++++++++++++++
>  1 files changed, 23 insertions(+), 0 deletions(-)
> 

Frankly, I'm not really sure. The solution came largely ... empirical
way. IOW, I tried more or less random things which looked like they
should fix the problem. So a review is very much appreciated. Please.

 merge-recursive.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index a3721ef..3c5420b 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -980,14 +980,15 @@ static int process_renames(struct merge_options *o,
 
 				if (mfi.clean &&
 				    sha_eq(mfi.sha, ren1->pair->two->sha1) &&
-				    mfi.mode == ren1->pair->two->mode)
+				    mfi.mode == ren1->pair->two->mode) {
 					/*
 					 * This messaged is part of
 					 * t6022 test. If you change
 					 * it update the test too.
 					 */
 					output(o, 3, "Skipped %s (merged same as existing)", ren1_dst);
-				else {
+					ren1->dst_entry->processed = 0;
+				} else {
 					if (mfi.merge || !mfi.clean)
 						output(o, 1, "Renaming %s => %s", ren1_src, ren1_dst);
 					if (mfi.merge)
-- 
1.6.3.28.ga852b

Re: [PATCH] Fix for a merge where a branch has an F->D transition

[Junio C Hamano]

Alex Riesen <raa.lkml@gmail.com> writes:

> Frankly, I'm not really sure. The solution came largely ... empirical
> way. IOW, I tried more or less random things which looked like they
> should fix the problem. So a review is very much appreciated. Please.

I've always thought that D/F conflict logic in merge-recursive is placed
at the wrong processing phase.  IIRC, it enumerates potential D/F
conflicting paths before even attempting to process renames, and it is
easy to miss a path that was previously file going away as the result of a
clean merge (in which case it is ok to have a directory there as the
result of a merge for other paths).  This breakage could be a small
example of it.

Regardless, I think your patch is a reasonable fix to go to the
maintenance track.  Thanks for looking into it.

>  merge-recursive.c |    5 +++--
>  1 files changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index a3721ef..3c5420b 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -980,14 +980,15 @@ static int process_renames(struct merge_options *o,
>  
>  				if (mfi.clean &&
>  				    sha_eq(mfi.sha, ren1->pair->two->sha1) &&
> -				    mfi.mode == ren1->pair->two->mode)
> +				    mfi.mode == ren1->pair->two->mode) {
>  					/*
>  					 * This messaged is part of
>  					 * t6022 test. If you change
>  					 * it update the test too.
>  					 */
>  					output(o, 3, "Skipped %s (merged same as existing)", ren1_dst);
> -				else {
> +					ren1->dst_entry->processed = 0;
> +				} else {
>  					if (mfi.merge || !mfi.clean)
>  						output(o, 1, "Renaming %s => %s", ren1_src, ren1_dst);
>  					if (mfi.merge)
> -- 
> 1.6.3.28.ga852b

Re: [PATCH] Fix for a merge where a branch has an F->D transition

[Alex Riesen]

2009/5/13 Junio C Hamano <gitster@pobox.com>:
> Alex Riesen <raa.lkml@gmail.com> writes:
>> Frankly, I'm not really sure. The solution came largely ... empirical
>> way. IOW, I tried more or less random things which looked like they
>> should fix the problem. So a review is very much appreciated. Please.
>
> I've always thought that D/F conflict logic in merge-recursive is placed
> at the wrong processing phase.  IIRC, it enumerates potential D/F
> conflicting paths before even attempting to process renames, and it is
> easy to miss a path that was previously file going away as the result of a
> clean merge (in which case it is ok to have a directory there as the
> result of a merge for other paths).  This breakage could be a small
> example of it.
>
> Regardless, I think your patch is a reasonable fix to go to the
> maintenance track.  Thanks for looking into it.

I'm afraid the fix is not that simple: the "if" branch where the change
is placed supposed to prevent updating files in the working tree
which already have the same content as the merge's output.
My change may force them to be updated regardless. I think...
Johannes, you know this area the best, could you please have
a look?

Re: [PATCH] Fix for a merge where a branch has an F->D transition

[Johannes Schindelin]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1346 bytes --]

Hi,

On Wed, 13 May 2009, Alex Riesen wrote:

> 2009/5/13 Junio C Hamano <gitster@pobox.com>:
> > Alex Riesen <raa.lkml@gmail.com> writes:
> >> Frankly, I'm not really sure. The solution came largely ... empirical
> >> way. IOW, I tried more or less random things which looked like they
> >> should fix the problem. So a review is very much appreciated. Please.
> >
> > I've always thought that D/F conflict logic in merge-recursive is placed
> > at the wrong processing phase.  IIRC, it enumerates potential D/F
> > conflicting paths before even attempting to process renames, and it is
> > easy to miss a path that was previously file going away as the result of a
> > clean merge (in which case it is ok to have a directory there as the
> > result of a merge for other paths).  This breakage could be a small
> > example of it.
> >
> > Regardless, I think your patch is a reasonable fix to go to the
> > maintenance track.  Thanks for looking into it.
> 
> I'm afraid the fix is not that simple: the "if" branch where the change
> is placed supposed to prevent updating files in the working tree
> which already have the same content as the merge's output.
> My change may force them to be updated regardless. I think...
> Johannes, you know this area the best, could you please have
> a look?

Sorry, no time at the moment...

Ciao,
Dscho

Re: [PATCH] Fix for a merge where a branch has an F->D transition

[Alex Riesen]

2009/5/13 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
> On Wed, 13 May 2009, Alex Riesen wrote:
>> 2009/5/13 Junio C Hamano <gitster@pobox.com>:
>> > Alex Riesen <raa.lkml@gmail.com> writes:
>> >> Frankly, I'm not really sure. The solution came largely ... empirical
>> >> way. IOW, I tried more or less random things which looked like they
>> >> should fix the problem. So a review is very much appreciated. Please.
>> >
>> > I've always thought that D/F conflict logic in merge-recursive is placed
>> > at the wrong processing phase.  IIRC, it enumerates potential D/F
>> > conflicting paths before even attempting to process renames, and it is
>> > easy to miss a path that was previously file going away as the result of a
>> > clean merge (in which case it is ok to have a directory there as the
>> > result of a merge for other paths).  This breakage could be a small
>> > example of it.
>> >
>> > Regardless, I think your patch is a reasonable fix to go to the
>> > maintenance track.  Thanks for looking into it.
>>
>> I'm afraid the fix is not that simple: the "if" branch where the change
>> is placed supposed to prevent updating files in the working tree
>> which already have the same content as the merge's output.
>> My change may force them to be updated regardless. I think...
>> Johannes, you know this area the best, could you please have
>> a look?
>
> Sorry, no time at the moment...
>

Maybe the patch could live on master or next for while? Many people
track master, so a performance degradation, if it happens, will be noticed.
And with master (or even better - next) we have an auditory which is at
least capable to complain.

Re: [PATCH] Fix for a merge where a branch has an F->D transition

[Johannes Schindelin]

Hi,

On Mon, 11 May 2009, Alex Riesen wrote:

> Some path names which transitioned from file to a directory were not
> updated in the final part of the merge (loop around unmerged entries in
> merge_trees), because the branch in process_renames which filtered out
> updates for the files with the same content ("merged same as existing")
> has left the rename entry in processed state. In this case, the
> processing cannot be finished at the process_renames phase (because
> the old file still blocks creation of directory where new files should
> appear), and must be postponed until the update_entry phase.

I know that as a German, I am supposed to like long sentences and crowded 
paragraphs.  Maybe I am not that German after all.

> Frankly, I'm not really sure. The solution came largely ... empirical 
> way. IOW, I tried more or less random things which looked like they 
> should fix the problem.

This does not give me the cozy feeling I need to review a patch.  After 
all, I do not want to do all the work that should be done by the patch 
author, but I just want to put in my knowledge to verify that the patch 
is correct.

But as you asked me explicitely...

> diff --git a/merge-recursive.c b/merge-recursive.c
> index a3721ef..3c5420b 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -980,14 +980,15 @@ static int process_renames(struct merge_options *o,
>  
>  				if (mfi.clean &&
>  				    sha_eq(mfi.sha, ren1->pair->two->sha1) &&
> -				    mfi.mode == ren1->pair->two->mode)
> +				    mfi.mode == ren1->pair->two->mode) {
>  					/*
>  					 * This messaged is part of
>  					 * t6022 test. If you change
>  					 * it update the test too.
>  					 */
>  					output(o, 3, "Skipped %s (merged same as existing)", ren1_dst);
> -				else {
> +					ren1->dst_entry->processed = 0;
> +				} else {

So basically, you say that a dst_entry has not been processed, when it 
_has_ been?  That cannot be correct...

Ciao,
Dscho

Re: [PATCH] Fix for a merge where a branch has an F->D transition

[Alex Riesen]

2009/5/20 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
> On Mon, 11 May 2009, Alex Riesen wrote:
>
>> Some path names which transitioned from file to a directory were not
>> updated in the final part of the merge (loop around unmerged entries in
>> merge_trees), because the branch in process_renames which filtered out
>> updates for the files with the same content ("merged same as existing")
>> has left the rename entry in processed state. In this case, the
>> processing cannot be finished at the process_renames phase (because
>> the old file still blocks creation of directory where new files should
>> appear), and must be postponed until the update_entry phase.
>
> I know that as a German, I am supposed to like long sentences and crowded
> paragraphs.  Maybe I am not that German after all.
>

Will be shortened.

>> diff --git a/merge-recursive.c b/merge-recursive.c
>> index a3721ef..3c5420b 100644
>> --- a/merge-recursive.c
>> +++ b/merge-recursive.c
>> @@ -980,14 +980,15 @@ static int process_renames(struct merge_options *o,
>>
>>                               if (mfi.clean &&
>>                                   sha_eq(mfi.sha, ren1->pair->two->sha1) &&
>> -                                 mfi.mode == ren1->pair->two->mode)
>> +                                 mfi.mode == ren1->pair->two->mode) {
>>                                       /*
>>                                        * This messaged is part of
>>                                        * t6022 test. If you change
>>                                        * it update the test too.
>>                                        */
>>                                       output(o, 3, "Skipped %s (merged same as existing)", ren1_dst);
>> -                             else {
>> +                                     ren1->dst_entry->processed = 0;
>> +                             } else {
>
> So basically, you say that a dst_entry has not been processed, when it
> _has_ been?  That cannot be correct...
>

My feeling exactly. I'm afraid I just zeroed the effect of the branch
which should avoid rewriting of files with known same content.
It's just that the code has grown a little confusing...
I'm still thinking about how to avoid unneeded rewrites of the files
with same content under same name.