Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

[Orgad Shaneh]

I can't reproduce this with a minimal example, but it happens in my project.

What I tried to do for reproducing is:
rm -rf super sub
mkdir sub; cd sub; git init
git commit --allow-empty -m 'Initial commit'
mkdir ../super; cd ../super
git init
git submodule add ../sub
touch foo; git add foo sub
git commit -m 'Initial commit'
touch a; git add a; git commit -m 'a'
touch b; git add b; git commit -m 'b'
cd sub; git commit --allow-empty -m 'New commit'; cd ..
git rebase -i HEAD^^

Then drop a.

In my project I get:
error: cannot rebase: You have unstaged changes.

This works fine with 2.14.3.

- Orgad

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

[Stefan Beller]

On Thu, Nov 2, 2017 at 1:30 AM, Orgad Shaneh <orgads@gmail.com> wrote:
> I can't reproduce this with a minimal example, but it happens in my project.
>
> What I tried to do for reproducing is:
> rm -rf super sub
> mkdir sub; cd sub; git init
> git commit --allow-empty -m 'Initial commit'
> mkdir ../super; cd ../super
> git init
> git submodule add ../sub
> touch foo; git add foo sub
> git commit -m 'Initial commit'
> touch a; git add a; git commit -m 'a'
> touch b; git add b; git commit -m 'b'
> cd sub; git commit --allow-empty -m 'New commit'; cd ..
> git rebase -i HEAD^^
>
> Then drop a.
>
> In my project I get:
> error: cannot rebase: You have unstaged changes.
>
> This works fine with 2.14.3.

  git log --oneline v2.14.3..v2.15.0 -- submodule.c
doesn't give any promising hints (i.e. I don't think one of a
submodule related series introduced this either by chance or
on purpose)

"rebase -i" was rewritten into C in 570676e011, though
that series was extensively tested by DScho, so I wouldn't
want to point fingers here quickly.

Would you be willing to bisect this behavior?

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

[Orgad Shaneh]

On Thu, Nov 2, 2017 at 8:34 PM, Stefan Beller <sbeller@google.com> wrote:
> On Thu, Nov 2, 2017 at 1:30 AM, Orgad Shaneh <orgads@gmail.com> wrote:
>> I can't reproduce this with a minimal example, but it happens in my project.
>>
>> What I tried to do for reproducing is:
>> rm -rf super sub
>> mkdir sub; cd sub; git init
>> git commit --allow-empty -m 'Initial commit'
>> mkdir ../super; cd ../super
>> git init
>> git submodule add ../sub
>> touch foo; git add foo sub
>> git commit -m 'Initial commit'
>> touch a; git add a; git commit -m 'a'
>> touch b; git add b; git commit -m 'b'
>> cd sub; git commit --allow-empty -m 'New commit'; cd ..
>> git rebase -i HEAD^^
>>
>> Then drop a.
>>
>> In my project I get:
>> error: cannot rebase: You have unstaged changes.
>>
>> This works fine with 2.14.3.
>
>   git log --oneline v2.14.3..v2.15.0 -- submodule.c
> doesn't give any promising hints (i.e. I don't think one of a
> submodule related series introduced this either by chance or
> on purpose)
>
> "rebase -i" was rewritten into C in 570676e011, though
> that series was extensively tested by DScho, so I wouldn't
> want to point fingers here quickly.
>
> Would you be willing to bisect this behavior?

Bisected to ff6f1f564c48def1f8e1852826bab58af5044b06:
submodule-config: lazy-load a repository's .gitmodules file

- Orgad

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

[Johannes Schindelin]

Hi Orgad,

On Thu, 2 Nov 2017, Orgad Shaneh wrote:

> I can't reproduce this with a minimal example, but it happens in my project.
> 
> What I tried to do for reproducing is:
> rm -rf super sub
> mkdir sub; cd sub; git init
> git commit --allow-empty -m 'Initial commit'
> mkdir ../super; cd ../super
> git init
> git submodule add ../sub
> touch foo; git add foo sub
> git commit -m 'Initial commit'
> touch a; git add a; git commit -m 'a'
> touch b; git add b; git commit -m 'b'
> cd sub; git commit --allow-empty -m 'New commit'; cd ..
> git rebase -i HEAD^^
> 
> Then drop a.
> 
> In my project I get:
> error: cannot rebase: You have unstaged changes.
> 
> This works fine with 2.14.3.

I tried to turn this into a regression test, but I cannot make it fail:

-- snip --
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index ebf4f5e4b2c..55aebe53191 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -20,7 +20,7 @@ git_rebase () {
 	git rebase "$1"
 }
 
-test_submodule_switch "git_rebase"
+#test_submodule_switch "git_rebase"
 
 git_rebase_interactive () {
 	git status -su >expect &&
@@ -38,6 +38,27 @@ git_rebase_interactive () {
 	git rebase -i "$1"
 }
 
-test_submodule_switch "git_rebase_interactive"
+#test_submodule_switch "git_rebase_interactive"
+
+test_expect_success '123' '
+	git init sub &&
+	test_commit -C sub init-submodule &&
+	git init super &&
+	git -C super submodule add ../sub &&
+	(
+		cd super &&
+		test_tick &&
+		touch foo &&
+		git add sub foo &&
+		git commit -m initial &&
+		test_commit a &&
+		test_commit b &&
+		test_commit -C sub new &&
+		set_fake_editor &&
+		FAKE_LINES=2 &&
+		export FAKE_LINES &&
+		git rebase -i HEAD^^
+	)
+'
 
 test_done
-- snap --

Can you help me spot what I did wrong?

Ciao,
Dscho

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

[Johannes Schindelin]

Hi Orgad,

On Fri, 3 Nov 2017, Johannes Schindelin wrote:

> On Thu, 2 Nov 2017, Orgad Shaneh wrote:
> 
> > I can't reproduce this with a minimal example, but it happens in my project.

Whoa, I somehow overlooked the "can't". Sorry.

> > What I tried to do for reproducing is:
> > rm -rf super sub
> > mkdir sub; cd sub; git init
> > git commit --allow-empty -m 'Initial commit'
> > mkdir ../super; cd ../super
> > git init
> > git submodule add ../sub
> > touch foo; git add foo sub
> > git commit -m 'Initial commit'
> > touch a; git add a; git commit -m 'a'
> > touch b; git add b; git commit -m 'b'
> > cd sub; git commit --allow-empty -m 'New commit'; cd ..
> > git rebase -i HEAD^^
> > 
> > Then drop a.
> > 
> > In my project I get:
> > error: cannot rebase: You have unstaged changes.
> > 
> > This works fine with 2.14.3.
> 
> I tried to turn this into a regression test, but I cannot make it fail:
> 
> -- snip --
> diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
> index ebf4f5e4b2c..55aebe53191 100755
> --- a/t/t3426-rebase-submodule.sh
> +++ b/t/t3426-rebase-submodule.sh
> @@ -20,7 +20,7 @@ git_rebase () {
>  	git rebase "$1"
>  }
>  
> -test_submodule_switch "git_rebase"
> +#test_submodule_switch "git_rebase"
>  
>  git_rebase_interactive () {
>  	git status -su >expect &&
> @@ -38,6 +38,27 @@ git_rebase_interactive () {
>  	git rebase -i "$1"
>  }
>  
> -test_submodule_switch "git_rebase_interactive"
> +#test_submodule_switch "git_rebase_interactive"
> +
> +test_expect_success '123' '
> +	git init sub &&
> +	test_commit -C sub init-submodule &&
> +	git init super &&
> +	git -C super submodule add ../sub &&
> +	(
> +		cd super &&
> +		test_tick &&
> +		touch foo &&
> +		git add sub foo &&
> +		git commit -m initial &&
> +		test_commit a &&
> +		test_commit b &&
> +		test_commit -C sub new &&
> +		set_fake_editor &&
> +		FAKE_LINES=2 &&
> +		export FAKE_LINES &&

I inserted a `git diff-files` here, and it printed exactly what I
expected:

++ git diff-files
:160000 160000 62cab94c8d8cf047bbb60c12def559339300efa4 0000000000000000000000000000000000000000 M      sub

> +		git rebase -i HEAD^^
> +	)
> +'

There must be something else going wrong that we did not replicate here.
Maybe the `error: cannot rebase: You have unstaged changes.` message was
caused not by a change in the submodule? Could you run `git diff-files`
before the rebase?

This does *not* refresh the index, but maybe that is what is going wrong;
you could call `git update-index --refresh` before the rebase and see
whether that works around the issue?

Ciao,
Dscho

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

[Orgad Shaneh]

On Fri, Nov 3, 2017 at 6:20 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Orgad,
>
> On Fri, 3 Nov 2017, Johannes Schindelin wrote:
>
>> On Thu, 2 Nov 2017, Orgad Shaneh wrote:
>>
>> > I can't reproduce this with a minimal example, but it happens in my project.
>
> Whoa, I somehow overlooked the "can't". Sorry.
>
> I inserted a `git diff-files` here, and it printed exactly what I
> expected:
>
> ++ git diff-files
> :160000 160000 62cab94c8d8cf047bbb60c12def559339300efa4 0000000000000000000000000000000000000000 M      sub
>
>> +             git rebase -i HEAD^^
>> +     )
>> +'
>
> There must be something else going wrong that we did not replicate here.
> Maybe the `error: cannot rebase: You have unstaged changes.` message was
> caused not by a change in the submodule? Could you run `git diff-files`
> before the rebase?

It's the same before and during the rebase:
$ git diff-files
:160000 160000 c840225a7cf6bb2ec64da9d35d2c29210bc5e5e8
0000000000000000000000000000000000000000 M      sub


>
> This does *not* refresh the index, but maybe that is what is going wrong;
> you could call `git update-index --refresh` before the rebase and see
> whether that works around the issue?

Nope.

If I run git submodule update, then rebase --continue works fine, so
it's definitely somehow caused by the submodule.

- Orgad

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

[Orgad Shaneh]

On Sat, Nov 4, 2017 at 8:04 PM, Orgad Shaneh <orgads@gmail.com> wrote:
> On Fri, Nov 3, 2017 at 6:20 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
>> Hi Orgad,
>>
>> On Fri, 3 Nov 2017, Johannes Schindelin wrote:
>>
>>> On Thu, 2 Nov 2017, Orgad Shaneh wrote:
>>>
>>> > I can't reproduce this with a minimal example, but it happens in my project.
>>
>> Whoa, I somehow overlooked the "can't". Sorry.
>>
>> I inserted a `git diff-files` here, and it printed exactly what I
>> expected:
>>
>> ++ git diff-files
>> :160000 160000 62cab94c8d8cf047bbb60c12def559339300efa4 0000000000000000000000000000000000000000 M      sub
>>
>>> +             git rebase -i HEAD^^
>>> +     )
>>> +'
>>
>> There must be something else going wrong that we did not replicate here.
>> Maybe the `error: cannot rebase: You have unstaged changes.` message was
>> caused not by a change in the submodule? Could you run `git diff-files`
>> before the rebase?
>
> It's the same before and during the rebase:
> $ git diff-files
> :160000 160000 c840225a7cf6bb2ec64da9d35d2c29210bc5e5e8
> 0000000000000000000000000000000000000000 M      sub
>
>
>>
>> This does *not* refresh the index, but maybe that is what is going wrong;
>> you could call `git update-index --refresh` before the rebase and see
>> whether that works around the issue?
>
> Nope.
>
> If I run git submodule update, then rebase --continue works fine, so
> it's definitely somehow caused by the submodule.

I just checked out v2.15.0.windows.1 and reverted ff6f1f564c - it
solves the problem. I still have no idea how to minimally reproduce
(in my project it's easily reproducible) :)

- Orgad

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

[Brandon Williams]

On 11/05, Orgad Shaneh wrote:
> On Sat, Nov 4, 2017 at 8:04 PM, Orgad Shaneh <orgads@gmail.com> wrote:
> > On Fri, Nov 3, 2017 at 6:20 PM, Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> >> Hi Orgad,
> >>
> >> On Fri, 3 Nov 2017, Johannes Schindelin wrote:
> >>
> >>> On Thu, 2 Nov 2017, Orgad Shaneh wrote:
> >>>
> >>> > I can't reproduce this with a minimal example, but it happens in my project.
> >>
> >> Whoa, I somehow overlooked the "can't". Sorry.
> >>
> >> I inserted a `git diff-files` here, and it printed exactly what I
> >> expected:
> >>
> >> ++ git diff-files
> >> :160000 160000 62cab94c8d8cf047bbb60c12def559339300efa4 0000000000000000000000000000000000000000 M      sub
> >>
> >>> +             git rebase -i HEAD^^
> >>> +     )
> >>> +'
> >>
> >> There must be something else going wrong that we did not replicate here.
> >> Maybe the `error: cannot rebase: You have unstaged changes.` message was
> >> caused not by a change in the submodule? Could you run `git diff-files`
> >> before the rebase?
> >
> > It's the same before and during the rebase:
> > $ git diff-files
> > :160000 160000 c840225a7cf6bb2ec64da9d35d2c29210bc5e5e8
> > 0000000000000000000000000000000000000000 M      sub
> >
> >
> >>
> >> This does *not* refresh the index, but maybe that is what is going wrong;
> >> you could call `git update-index --refresh` before the rebase and see
> >> whether that works around the issue?
> >
> > Nope.
> >
> > If I run git submodule update, then rebase --continue works fine, so
> > it's definitely somehow caused by the submodule.
> 
> I just checked out v2.15.0.windows.1 and reverted ff6f1f564c - it
> solves the problem. I still have no idea how to minimally reproduce
> (in my project it's easily reproducible) :)
> 

After reading your bug report and the fact that you weren't able to
reproduce it outside of your project I think i figured out what is
happening.  Before ff6f1f564c the gitmodules file wasn't being loaded
unless a codepath explicitly wanted to work with submodules.  Now they
are being lazy-loaded so if you call into the submodule config subsystem
it'll work without having to have initialized it before.  I suspect
that the submodule which is causing the failure has a
"submodule.<name>.ignore" entry in the .gitmodules file or somewhere in
your repositories config (I actually suspect the latter based on the
code path).

When rebase calls into the diff machinery to see if there are unstaged
changes it explicitly requests that submodule's be ignored, but this
desired gets overridden by your repository's config, clearing the
ignored flag and making rebase actually pay attention to the fact that
the submodule has changes in it.

I don't have a patch available to for you to test just yet (but I'll
have some time later today to write one up) but could you verify that
(1) you have an ignore entry for the submodule in question in your
config and (2) removing it from your config avoids the failure?  If
that's the case then we would be able to put together a reproducible
recipe for this failure.

-- 
Brandon Williams

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

[Orgad Shaneh]

On Mon, Nov 6, 2017 at 8:37 PM, Brandon Williams <bmwill@google.com> wrote:

> After reading your bug report and the fact that you weren't able to
> reproduce it outside of your project I think i figured out what is
> happening.  Before ff6f1f564c the gitmodules file wasn't being loaded
> unless a codepath explicitly wanted to work with submodules.  Now they
> are being lazy-loaded so if you call into the submodule config subsystem
> it'll work without having to have initialized it before.  I suspect
> that the submodule which is causing the failure has a
> "submodule.<name>.ignore" entry in the .gitmodules file or somewhere in
> your repositories config (I actually suspect the latter based on the
> code path).
>
> When rebase calls into the diff machinery to see if there are unstaged
> changes it explicitly requests that submodule's be ignored, but this
> desired gets overridden by your repository's config, clearing the
> ignored flag and making rebase actually pay attention to the fact that
> the submodule has changes in it.
>
> I don't have a patch available to for you to test just yet (but I'll
> have some time later today to write one up) but could you verify that
> (1) you have an ignore entry for the submodule in question in your
> config and (2) removing it from your config avoids the failure?  If
> that's the case then we would be able to put together a reproducible
> recipe for this failure.

You're right. Thanks for the info. I have ignore = dirty, and removing
it from the config solves the problem indeed.

The following script reproduces the bug (it's the same as before, only
added git config):

rm -rf super sub
mkdir sub; cd sub; git init
git commit --allow-empty -m 'Initial commit'
mkdir ../super; cd ../super
git init
git submodule add ../sub
git config submodule.sub.ignore dirty
touch foo; git add foo sub
git commit -m 'Initial commit'
touch a; git add a; git commit -m 'a'
touch b; git add b; git commit -m 'b'
cd sub; git commit --allow-empty -m 'New commit'; cd ..
git rebase -i HEAD^^

- Orgad