[BUG] merge-recursive triggered "BUG"

[BUG] merge-recursive triggered "BUG"

[Junio C Hamano]

As a part of my today's merge, I used 'next' that contains b2c8c0a
(merge-recursive: When we detect we can skip an update, actually skip it,
2011-02-28) to merge 'maint' into 'master' to propagate older releases up.

It triggered a "BUG" per merged path, and I bisected this breakage down to
the said commit. Luckily 'master' is not contaminated with the breakage,
so I used it to finish today's work.

When I push out the result from today, you can reproduce it with

    git checkout 0631623 ;# master to acquire changes from maint
    git merge [-s recursive] fbcda3c

I suspect that the new codepath introduced by b2c8c0a needs to pay
attention to the merge depth (for example, does it make any sense at all
to run lstat() when you are doing recursive common parent synthesis?), but
I didn't dig into it.

The command fails with this output:

error: addinfo_cache failed for path 'builtin/add.c'
error: addinfo_cache failed for path 'builtin/apply.c'
error: addinfo_cache failed for path 'builtin/branch.c'
error: addinfo_cache failed for path 'builtin/checkout.c'
error: addinfo_cache failed for path 'builtin/commit.c'
error: addinfo_cache failed for path 'builtin/config.c'
error: addinfo_cache failed for path 'builtin/diff-files.c'
error: addinfo_cache failed for path 'builtin/diff.c'
error: addinfo_cache failed for path 'builtin/fast-export.c'
error: addinfo_cache failed for path 'builtin/grep.c'
error: addinfo_cache failed for path 'builtin/hash-object.c'
error: addinfo_cache failed for path 'builtin/init-db.c'
error: addinfo_cache failed for path 'builtin/log.c'
error: addinfo_cache failed for path 'builtin/merge.c'
error: addinfo_cache failed for path 'builtin/push.c'
error: addinfo_cache failed for path 'builtin/rerere.c'
error: addinfo_cache failed for path 'builtin/update-index.c'
error: addinfo_cache failed for path 't/t7810-grep.sh'
BUG: There are unmerged index entries:
BUG: 1 builtin/add.cBUG: 2 builtin/add.cBUG: 3 builtin/add.cBUG: 1 builtin/apply.cBUG: 2 builtin/apply.cBUG: 3 builtin/apply.cBUG: 1 builtin/branch.cBUG: 2 builtin/branch.cBUG: 3 builtin/branch.cBUG: 1 builtin/checkout.cBUG: 2 builtin/checkout.cBUG: 3 builtin/checkout.cBUG: 1 builtin/commit.cBUG: 2 builtin/commit.cBUG: 3 builtin/commit.cBUG: 1 builtin/config.cBUG: 2 builtin/config.cBUG: 3 builtin/config.cBUG: 1 builtin/diff-files.cBUG: 2 builtin/diff-files.cBUG: 3 builtin/diff-files.cBUG: 1 builtin/diff.cBUG: 2 builtin/diff.cBUG: 3 builtin/diff.cBUG: 1 builtin/fast-export.cBUG: 2 builtin/fast-export.cBUG: 3 builtin/fast-export.cBUG: 1 builtin/grep.cBUG: 2 builtin/grep.cBUG: 3 builtin/grep.cBUG: 1 builtin/hash-object.cBUG: 2 builtin/hash-object.cBUG: 3 builtin/hash-object.cBUG: 1 builtin/in
 it-db.cBUG: 2 builtin/init-db.cBUG: 3 builtin/init-db.cBUG: 1 builtin/log.cBUG: 2 builtin/log.cBUG: 3 builtin/log.cBUG: 1 builtin/merge.cBUG: 2 builtin/merge.cBUG: 3 builtin/merge.cBUG: 1 builtin/push.cBUG: 2 builtin/push.cBUG: 3 builtin/push.cBUG: 1 builtin/rerere.cBUG: 2 builtin/rerere.cBUG: 3 builtin/rerere.cBUG: 1 builtin/update-index.cBUG: 2 builtin/update-index.cBUG: 3 builtin/update-index.cBUG: 1 t/t7810-grep.shBUG: 2 t/t7810-grep.shBUG: 3 t/t7810-grep.shfatal: Bug in merge-recursive.c


Thanks.

Re: [BUG] merge-recursive triggered "BUG"

[Junio C Hamano]

To illustrate the issue a bit better, with this patch applied on top of
the en/merge-recursive topic, we get the same errors:

    Merging:
    31734dd Renamed and modified
    virtual merge-branch-1
    found 1 common ancestor(s):
    13277ae Common commmit
    Skipped rename (merged same as existing)
    error: addinfo_cache failed for path 'rename'
    rename: unmerged (f00c965d8307308469e537302baa73048488f162)
    rename: unmerged (3bb459b831ea471b9cd1cbb7c6d54a74251a711b)
    rename: unmerged (f00c965d8307308469e537302baa73048488f162)
    fatal: git write-tree failed to write a tree

In the test case, the merge machinery should notice that the result of the
merge structurally place the merge result in the path "rename", and the
content of the resulting blob matches what our side already have, so we
should end up with a clean merge in the index (the index has the same blob
as the HEAD at path "rename"), keeping the updated contents in the working
tree.

 t/t6022-merge-rename.sh |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/t/t6022-merge-rename.sh b/t/t6022-merge-rename.sh
index 7d955c1..94b9c00 100755
--- a/t/t6022-merge-rename.sh
+++ b/t/t6022-merge-rename.sh
@@ -730,6 +730,7 @@ test_expect_success 'setup avoid unnecessary update, normal rename' '
 
 test_expect_success 'avoid unnecessary update, normal rename' '
 	git checkout -q avoid-unnecessary-update-1^0 &&
+	echo modified >>rename &&
 	test-chmtime =1000000000 rename &&
 	test-chmtime -v +0 rename >expect &&
 	git merge merge-branch-1 &&

[PATCH] merge-recursive: tweak magic band-aid

[Junio C Hamano]

Running checks against working tree (e.g. lstat()) and causing
changes to working tree (e.g. unlink()) while building a virtual
ancestor merge does not make any sense. Avoid doing so.

This is not a real fix; it is another magic band-aid on top of
another band-aid we placed earlier.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---

 * This does not fix the "even though we have local change in the working
   tree, we do have a clean index entry for that path that happens to be
   the one we renamed, and they didn't" case this message responds to, it
   does seem to fix the real-life breakage I saw when I merged 'maint' to
   'master' yesterday, admittedly in a clean working tree.

   merge-recursive is riddled with places that touch/inspect working tree
   when it shouldn't, and it is beyond salvage without a major refactoring
   in its current shape, so this magic band-aid should do for now.

   Generally speaking, the only valid kinds of accesses a merge strategy
   is allowed are:

   (1) to compare the working tree file with our original index entry to
       see if it has local changes. This should be done only for paths
       that the index entry of the merge result is different from the
       current one and the working tree file needs updating.  We need this
       check because the merge must not lose such local changes when we
       checkout the merge result. This check could use lstat(2) as part of
       ce_match_stat() from read-cache.c; or

   (2) to make sure there is no file that was not tracked in our original
       index in the working tree at the path that the result of the merge
       needs to create a file or a directory. We need this check because
       the merge must not lose such an untracked file when we checkout the
       merge result. This check would use lstat(2), as part of
       checkout_entry() from entry.c; or

   (3) when the result of the merge needs to create a file at a path and
       the working tree has a directory at the path, to make sure that the
       contents of the directory does not have any locally modified files
       relative to our original index, or untracked-and-unignored files.
       We need this check because the merge must be able to "rm -r" such a
       directory safely in order to checkout the merge result.

   As a special case, a file missing from the working tree is considered
   unmodified for the purpose of (1). IOW, removal of a path from the
   working tree without touching the index is considered a local change
   that we are willing to lose during a merge. This is to support a
   workflow (or a Porcelain script) that merges commits in a fresh
   temporary directory, e.g. this sequence

	cd_to_toplevel
	mkdir tmp_merge
        cd tmp_merge
        export GIT_DIR=../.git
        git read-tree pu ;# the branch may not be related to the current one
        git merge-resolve $(git merge-base pu topic) -- pu topic

   would start from an empty temporary merge directory, check out only the
   paths that were involved in the merge, potentially leaving conflict
   markers in them, and allows the user to resolve them, without touching
   the real working tree or the current branch. The resulting index can be
   written to a tree object to record the result of the merge.

 merge-recursive.c |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 847bc84..59482ff 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -370,6 +370,13 @@ static void make_room_for_directories_of_df_conflicts(struct merge_options *o,
 	struct stage_data *last_e;
 	int i;
 
+	/*
+	 * Do not do any of this crazyness during the recursive; we don't
+	 * even write anything to the working tree!
+	 */
+	if (o->call_depth)
+		return;
+
 	for (i = 0; i < entries->nr; i++) {
 		const char *path = entries->items[i].string;
 		int len = strlen(path);
@@ -1274,7 +1281,7 @@ static int merge_content(struct merge_options *o,
 
 	if (mfi.clean && !df_conflict_remains &&
 	    sha_eq(mfi.sha, a_sha) && mfi.mode == a.mode &&
-	    lstat(path, &st) == 0) {
+	    !o->call_depth && !lstat(path, &st)) {
 		output(o, 3, "Skipped %s (merged same as existing)", path);
 		add_cacheinfo(mfi.mode, mfi.sha, path,
 			      0 /*stage*/, 1 /*refresh*/, 0 /*options*/);
-- 
1.7.4.1.494.g5ddab

Re: [PATCH] merge-recursive: tweak magic band-aid

[Elijah Newren]

Hi,

On Fri, Mar 18, 2011 at 12:07 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Running checks against working tree (e.g. lstat()) and causing
> changes to working tree (e.g. unlink()) while building a virtual
> ancestor merge does not make any sense. Avoid doing so.
>
> This is not a real fix; it is another magic band-aid on top of
> another band-aid we placed earlier.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Sorry for not responding sooner; thanks for fixing this up, though.
Also, I have to apologize for having left a bunch of changes on my
harddrive for several months; one of those changes includes the first
half of your patch (which I should have remembered and not made the
problem worse with my more recent patch submission.)  I just never got
around to cleaning up those changes and getting them submitted.  Maybe
I should just send what I do have for now, and then send the cleaned
up version when I get a good chunk of time in another month or two.

>   merge-recursive is riddled with places that touch/inspect working tree
>   when it shouldn't, and it is beyond salvage without a major refactoring
>   in its current shape, so this magic band-aid should do for now.

Yeah, I agree this needs to be fixed up.  If no one else takes a look,
I'll try to when I get back to my other changes.

Re: [BUG] merge-recursive triggered "BUG"

[Jay Soffian]

On Wed, Mar 16, 2011 at 8:39 PM, Junio C Hamano <gitster@pobox.com> wrote:
> As a part of my today's merge, I used 'next' that contains b2c8c0a
> (merge-recursive: When we detect we can skip an update, actually skip it,
> 2011-02-28) to merge 'maint' into 'master' to propagate older releases up.
>
> It triggered a "BUG" per merged path, and I bisected this breakage down to
> the said commit. Luckily 'master' is not contaminated with the breakage,
> so I used it to finish today's work.

I just ran into this. It's not in a repo I can share however. But, why
did b2c8c0a make it into master with this known issue?

j.

> When I push out the result from today, you can reproduce it with
>
>    git checkout 0631623 ;# master to acquire changes from maint
>    git merge [-s recursive] fbcda3c
>
> I suspect that the new codepath introduced by b2c8c0a needs to pay
> attention to the merge depth (for example, does it make any sense at all
> to run lstat() when you are doing recursive common parent synthesis?), but
> I didn't dig into it.
>
> The command fails with this output:
>
> error: addinfo_cache failed for path 'builtin/add.c'
> error: addinfo_cache failed for path 'builtin/apply.c'
> error: addinfo_cache failed for path 'builtin/branch.c'
> error: addinfo_cache failed for path 'builtin/checkout.c'
> error: addinfo_cache failed for path 'builtin/commit.c'
> error: addinfo_cache failed for path 'builtin/config.c'
> error: addinfo_cache failed for path 'builtin/diff-files.c'
> error: addinfo_cache failed for path 'builtin/diff.c'
> error: addinfo_cache failed for path 'builtin/fast-export.c'
> error: addinfo_cache failed for path 'builtin/grep.c'
> error: addinfo_cache failed for path 'builtin/hash-object.c'
> error: addinfo_cache failed for path 'builtin/init-db.c'
> error: addinfo_cache failed for path 'builtin/log.c'
> error: addinfo_cache failed for path 'builtin/merge.c'
> error: addinfo_cache failed for path 'builtin/push.c'
> error: addinfo_cache failed for path 'builtin/rerere.c'
> error: addinfo_cache failed for path 'builtin/update-index.c'
> error: addinfo_cache failed for path 't/t7810-grep.sh'
> BUG: There are unmerged index entries:
> BUG: 1 builtin/add.cBUG: 2 builtin/add.cBUG: 3 builtin/add.cBUG: 1 builtin/apply.cBUG: 2 builtin/apply.cBUG: 3 builtin/apply.cBUG: 1 builtin/branch.cBUG: 2 builtin/branch.cBUG: 3 builtin/branch.cBUG: 1 builtin/checkout.cBUG: 2 builtin/checkout.cBUG: 3 builtin/checkout.cBUG: 1 builtin/commit.cBUG: 2 builtin/commit.cBUG: 3 builtin/commit.cBUG: 1 builtin/config.cBUG: 2 builtin/config.cBUG: 3 builtin/config.cBUG: 1 builtin/diff-files.cBUG: 2 builtin/diff-files.cBUG: 3 builtin/diff-files.cBUG: 1 builtin/diff.cBUG: 2 builtin/diff.cBUG: 3 builtin/diff.cBUG: 1 builtin/fast-export.cBUG: 2 builtin/fast-export.cBUG: 3 builtin/fast-export.cBUG: 1 builtin/grep.cBUG: 2 builtin/grep.cBUG: 3 builtin/grep.cBUG: 1 builtin/hash-object.cBUG: 2 builtin/hash-object.cBUG: 3 builtin/hash-object.cBUG: 1 builtin/init-db.cBUG: 2 builtin/init-db.cBUG: 3 builtin/init-db.cBUG: 1 builtin/log.cBUG: 2 builtin/log.cBUG: 3 builtin/log.cBUG: 1 builtin/merge.cBUG: 2 builtin/merge.cBUG: 3 builtin/merge.cBUG: 1 builtin/push.cBUG: 2 builtin/push.cBUG: 3 builtin/push.cBUG: 1 builtin/rerere.cBUG: 2 builtin/rerere.cBUG: 3 builtin/rerere.cBUG: 1 builtin/update-index.cBUG: 2 builtin/update-index.cBUG: 3 builtin/update-index.cBUG: 1 t/t7810-grep.shBUG: 2 t/t7810-grep.shBUG: 3 t/t7810-grep.shfatal: Bug in merge-recursive.c
>
>
> Thanks.
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [BUG] merge-recursive triggered "BUG"

[Jay Soffian]

On Thu, May 19, 2011 at 9:14 PM, Jay Soffian <jaysoffian@gmail.com> wrote:
> I just ran into this. It's not in a repo I can share however. But, why
> did b2c8c0a make it into master with this known issue?

Sorry, let me clarify. I got the:

  error: addinfo_cache failed for path '...'

line, but not the BUG. I can try to bisect git if this is a different issue.

j.

Re: [BUG] merge-recursive triggered "BUG"

[Junio C Hamano]

Jay Soffian <jaysoffian@gmail.com> writes:

> On Wed, Mar 16, 2011 at 8:39 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> As a part of my today's merge, I used 'next' that contains b2c8c0a
>> (merge-recursive: When we detect we can skip an update, actually skip it,
>> 2011-02-28) to merge 'maint' into 'master' to propagate older releases up.
>>
>> It triggered a "BUG" per merged path, and I bisected this breakage down to
>> the said commit. Luckily 'master' is not contaminated with the breakage,
>> so I used it to finish today's work.
>
> I just ran into this. It's not in a repo I can share however. But, why
> did b2c8c0a make it into master with this known issue?

Because it was patched by another band-aid, and apparently it was not
enough?

You are the second person to report the same regression, so let's revert
the merge of the entire topic, ac9666f (Merge branch 'en/merge-recursive',
2011-04-28) from master for now.

Thanks.

Re: [BUG] merge-recursive triggered "BUG"

[Jay Soffian]

On Thu, May 19, 2011 at 11:21 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> I just ran into this. It's not in a repo I can share however. But, why
>> did b2c8c0a make it into master with this known issue?
>
> Because it was patched by another band-aid, and apparently it was not
> enough?

Okay, I didn't see the band-aid, but I didn't look very hard.

> You are the second person to report the same regression, so let's revert
> the merge of the entire topic, ac9666f (Merge branch 'en/merge-recursive',
> 2011-04-28) from master for now.

I went to confirm the cause, did a merge with v1.7.4-rc0~102
(106e3afa6f) and did not see the "addinfo_cache failed" message. But
now here's the strange part, I then switched back to master
(11bc3e92bf), tried the merge again, and I'm still not seeing the
"addinfo_cache failed" message.

So now I'm trying to figure out what v1.7.4-rc0~102 altered about the
state of my repo that I'm no longer seeing the message.

Hmfph.

j.

Re: [BUG] merge-recursive triggered "BUG"

[Jay Soffian]

On Fri, May 20, 2011 at 8:29 AM, Jay Soffian <jaysoffian@gmail.com> wrote:
> On Thu, May 19, 2011 at 11:21 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>> I just ran into this. It's not in a repo I can share however. But, why
>>> did b2c8c0a make it into master with this known issue?
>>
>> Because it was patched by another band-aid, and apparently it was not
>> enough?
>
> Okay, I didn't see the band-aid, but I didn't look very hard.
>
>> You are the second person to report the same regression, so let's revert
>> the merge of the entire topic, ac9666f (Merge branch 'en/merge-recursive',
>> 2011-04-28) from master for now.
>
> I went to confirm the cause, did a merge with v1.7.4-rc0~102
> (106e3afa6f) and did not see the "addinfo_cache failed" message. But
> now here's the strange part, I then switched back to master
> (11bc3e92bf), tried the merge again, and I'm still not seeing the
> "addinfo_cache failed" message.
>
> So now I'm trying to figure out what v1.7.4-rc0~102 altered about the
> state of my repo that I'm no longer seeing the message.

Sorry, it's definitely that topic. I didn't realize you'd already
reverted it from master. The bug occurs between when the topic was
merged and when it was reverted.

Thanks,

j.