Stage, test, and commit only some changes, then repeat

Stage, test, and commit only some changes, then repeat

[Géry Ogam]

Hello,

I would like to stage, test, and commit only *some* changes of the working tree, and then repeat this process with the remaining changes.

My current solution (published at https://stackoverflow.com/a/70914962/2326961):

1. Stage some changes:
   ```
   git add -p file
   ```
2. Save away the remaining changes:
   ```
   git diff >patch
   git stash push -k
   ```
3. Test the staged changes.
4. Commit the staged changes:
   ```
   git commit
   ```
5. Restore the remaining changes:
   ```
   git apply patch
   ```
6. Go to step 1.

It is not ideal because a) it uses a patch file for saving the remaining changes; b) it uses the stash only for setting the working tree to the index state.

It would be ideal if I could save *only* the remaining changes in the stash instead of resorting to a patch file. How to do it?

Best,

Géry Ogam


[Info système]
version git ::
git version 2.35.1
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname: Darwin 20.6.0 Darwin Kernel Version 20.6.0: Wed Nov 10 22:23:07 PST 2021; root:xnu-7195.141.14~1/RELEASE_X86_64 x86_64
info compilateur : clang: 13.0.0 (clang-1300.0.29.3)
info libc : aucune information de libc disponible
$SHELL (typically, interactive shell): /bin/zsh


[Crochets activés]

Re: Stage, test, and commit only some changes, then repeat

[Johannes Sixt]

Am 30.01.22 um 21:17 schrieb Géry Ogam:
> Hello,
> 
> I would like to stage, test, and commit only *some* changes of the working tree, and then repeat this process with the remaining changes.
> 
> My current solution (published at https://stackoverflow.com/a/70914962/2326961):
> 
> 1. Stage some changes:
>    ```
>    git add -p file
>    ```
> 2. Save away the remaining changes:
>    ```
>    git diff >patch
>    git stash push -k
>    ```
> 3. Test the staged changes.
> 4. Commit the staged changes:
>    ```
>    git commit
>    ```
> 5. Restore the remaining changes:
>    ```
>    git apply patch
>    ```
> 6. Go to step 1.>
> It is not ideal because a) it uses a patch file for saving the
> remaining changes; b) it uses the stash only for setting the working
> tree to the index state.
> 
> It would be ideal if I could save *only* the remaining changes in the
> stash instead of resorting to a patch file. How to do it?
For example:

1. Stage some changes
2. Commit
3. git stash
4. Test
4a. git commit --amend if tests show that the commit is not yet perfect
5. git stash pop
6. Goto 1.

Be prepared for conflicts in 5 if 4a was necessary.

-- Hannes

Re: Stage, test, and commit only some changes, then repeat

[Sergey Organov]

Géry Ogam <gery.ogam@gmail.com> writes:

> Hello,
>
> I would like to stage, test, and commit only *some* changes of the
> working tree, and then repeat this process with the remaining changes.
>
> My current solution (published at
> https://stackoverflow.com/a/70914962/2326961):
>
> 1. Stage some changes:
>    ```
>    git add -p file
>    ```
> 2. Save away the remaining changes:
>    ```
>    git diff >patch
>    git stash push -k
>    ```
> 3. Test the staged changes.
> 4. Commit the staged changes:
>    ```
>    git commit
>    ```
> 5. Restore the remaining changes:
>    ```
>    git apply patch
>    ```
> 6. Go to step 1.
>
> It is not ideal because a) it uses a patch file for saving the
> remaining changes; b) it uses the stash only for setting the working
> tree to the index state.
>
> It would be ideal if I could save *only* the remaining changes in the
> stash instead of resorting to a patch file. How to do it?

It looks like you don't need patch file for this workflow. What's wrong with:

git add...
git stash push --keep-index
... check, git add fixes
git commit
git stash apply

???

-- Sergey Organov

Re: Stage, test, and commit only some changes, then repeat

[Géry Ogam]

> Le 31 janv. 2022 à 17:27, Sergey Organov <sorganov@gmail.com> a écrit :
> 
> Géry Ogam <gery.ogam@gmail.com> writes:
> 
>> Hello,
>> 
>> I would like to stage, test, and commit only *some* changes of the
>> working tree, and then repeat this process with the remaining changes.
>> 
>> My current solution (published at
>> https://stackoverflow.com/a/70914962/2326961):
>> 
>> 1. Stage some changes:
>>   ```
>>   git add -p file
>>   ```
>> 2. Save away the remaining changes:
>>   ```
>>   git diff >patch
>>   git stash push -k
>>   ```
>> 3. Test the staged changes.
>> 4. Commit the staged changes:
>>   ```
>>   git commit
>>   ```
>> 5. Restore the remaining changes:
>>   ```
>>   git apply patch
>>   ```
>> 6. Go to step 1.
>> 
>> It is not ideal because a) it uses a patch file for saving the
>> remaining changes; b) it uses the stash only for setting the working
>> tree to the index state.
>> 
>> It would be ideal if I could save *only* the remaining changes in the
>> stash instead of resorting to a patch file. How to do it?
> 
> It looks like you don't need patch file for this workflow. What's wrong with:
> 
> git add...
> git stash push --keep-index
> ... check, git add fixes
> git commit
> git stash apply
> 
> ???
> 
> -- Sergey Organov

Hello Sergey,

`git stash` saves the transition from the HEAD state to the working tree state. It also sets the working tree to the *HEAD* state.

`git stash --keep-index` saves the transition from the HEAD state to the working tree state. It also sets the working tree to the *index* state.

`git stash pop` applies the last saved transition. So if the working tree was not in HEAD state (like after `git stash --keep-index`), there will be a conflict.

Best,

Géry Ogam

Re: Stage, test, and commit only some changes, then repeat

[Sergey Organov]

Géry Ogam <gery.ogam@gmail.com> writes:

>> Le 31 janv. 2022 à 17:27, Sergey Organov <sorganov@gmail.com> a écrit :
>> 
>> Géry Ogam <gery.ogam@gmail.com> writes:
>> 
>>> Hello,
>>> 
>>> I would like to stage, test, and commit only *some* changes of the
>>> working tree, and then repeat this process with the remaining changes.
>>> 
>>> My current solution (published at
>>> https://stackoverflow.com/a/70914962/2326961):
>>> 
>>> 1. Stage some changes:
>>>   ```
>>>   git add -p file
>>>   ```
>>> 2. Save away the remaining changes:
>>>   ```
>>>   git diff >patch
>>>   git stash push -k
>>>   ```
>>> 3. Test the staged changes.
>>> 4. Commit the staged changes:
>>>   ```
>>>   git commit
>>>   ```
>>> 5. Restore the remaining changes:
>>>   ```
>>>   git apply patch
>>>   ```
>>> 6. Go to step 1.
>>> 
>>> It is not ideal because a) it uses a patch file for saving the
>>> remaining changes; b) it uses the stash only for setting the working
>>> tree to the index state.
>>> 
>>> It would be ideal if I could save *only* the remaining changes in the
>>> stash instead of resorting to a patch file. How to do it?
>> 
>> It looks like you don't need patch file for this workflow. What's wrong with:
>> 
>> git add...
>> git stash push --keep-index
>> ... check, git add fixes
>> git commit
>> git stash apply
>> 
>> ???
>> 
>> -- Sergey Organov
>
> Hello Sergey,
>
> `git stash` saves the transition from the HEAD state to the working
> tree state. It also sets the working tree to the *HEAD* state.
>
> `git stash --keep-index` saves the transition from the HEAD state to
> the working tree state. It also sets the working tree to the *index*
> state.
>
> `git stash pop` applies the last saved transition. So if the working
> tree was not in HEAD state (like after `git stash --keep-index`),
> there will be a conflict.

Did you actually try it and got conflict? I doubt there will be any if
you don't modify anything after "git stash --keep-index" during testing,
and if you do, than any method might bring conflicts.

In fact I just re-tested this to make sure, and got no conflicts.

-- Sergey Organov

Re: Stage, test, and commit only some changes, then repeat

[Géry Ogam]

> Le 30 janv. 2022 à 22:13, Johannes Sixt <j6t@kdbg.org> a écrit :
> 
> Am 30.01.22 um 21:17 schrieb Géry Ogam:
>> Hello,
>> 
>> I would like to stage, test, and commit only *some* changes of the working tree, and then repeat this process with the remaining changes.
>> 
>> My current solution (published at https://stackoverflow.com/a/70914962/2326961):
>> 
>> 1. Stage some changes:
>>   ```
>>   git add -p file
>>   ```
>> 2. Save away the remaining changes:
>>   ```
>>   git diff >patch
>>   git stash push -k
>>   ```
>> 3. Test the staged changes.
>> 4. Commit the staged changes:
>>   ```
>>   git commit
>>   ```
>> 5. Restore the remaining changes:
>>   ```
>>   git apply patch
>>   ```
>> 6. Go to step 1.>
>> It is not ideal because a) it uses a patch file for saving the
>> remaining changes; b) it uses the stash only for setting the working
>> tree to the index state.
>> 
>> It would be ideal if I could save *only* the remaining changes in the
>> stash instead of resorting to a patch file. How to do it?
> For example:
> 
> 1. Stage some changes
> 2. Commit
> 3. git stash
> 4. Test
> 4a. git commit --amend if tests show that the commit is not yet perfect
> 5. git stash pop
> 6. Goto 1.
> 
> Be prepared for conflicts in 5 if 4a was necessary.
> 
> -- Hannes


Hello Johannes,

Thanks for the solution! I think you can avoid conflicts in step 5 by popping off the stash right before step 4a and pushing on the stash right after, like this (the three columns represent the state of the working tree, index, and HEAD):


edit
X X X
Y
Z

git add Y
X X X
Y Y
Z

git commit
X X X
Y Y Y
Z

git stash push (saves X Y +Z)
X X X
Y Y Y

while test fails {

git stash pop (restores X Y +Z)
X X X
Y Y Y
Z

edit
X X X
Y'Y Y
Z

git add Y'
X X X
Y'Y'Y
Z

git commit --amend
X X X
Y'Y'Y'
Z

git stash push (saves X Y' +Z)
X X X
Y'Y'Y'

}

git stash pop (restores X Y +Z)
X X X
Y Y Y
Z


But the problem with your solution is that you commit before testing and then amend in case of test failure.

The option --keep-index of the command git stash push is supposed to handle this, like specified in the reference documentation:
https://git-scm.com/docs/git-stash#Documentation/git-stash.txt-Testingpartialcommits


Testing partial commits

You can use git stash push --keep-index when you want to make two or more commits out of the changes in the work tree, and you want to test each change before committing:

# ... hack hack hack ...
$ git add --patch foo            # add just first part to the index
$ git stash push --keep-index    # save all other changes to the stash
$ edit/build/test first part
$ git commit -m 'First part'     # commit fully tested change
$ git stash pop                  # prepare to work on all other changes
# ... repeat above five steps until one commit remains ...
$ edit/build/test remaining parts
$ git commit foo -m 'Remaining parts'


But --keep-index is fundamentally flawed because though it sets the working tree to the index state (instead of the HEAD state like bare git stash push), it still saves the changes between the HEAD state and the working tree state (i.e. staged changes and unstaged changes). So when you git stash pop, you get conflicts since the working tree is not in the HEAD state like expected, but in the index state.

People have been complaining about this conflict issue on Stack Overflow many times:
https://stackoverflow.com/q/7650797/2326961
https://stackoverflow.com/q/20028507/2326961
https://stackoverflow.com/q/49301304/2326961
https://stackoverflow.com/q/25620844/2326961
https://stackoverflow.com/q/20681448/2326961
https://stackoverflow.com/q/57524906/2326961

FEATURE REQUEST
So could we add a new option -U|--unstaged (by symmetry with the existing option -S|--staged) to the command git stash push to do what the option --keep-index was trying to achieve? It would:
— save the changes between the index state and the working tree state, i.e. only unstaged changes;
— set the working tree in the index state.

So the difference with --keep-index is the source: the changes would be computed from the index instead of from HEAD.

With that new option -U|--unstaged, we could solve the ‘testing partial changes before committing’ problem elegantly (called ‘Testing partial commits’ in the reference documentation, which is a bit misleading), that is
— without conflicts contrary to the solution in the reference documentation:
 git stash push --keep-index
 test
 git commit
 git stash pop
— without temporary commits contrary to your suggested solution:
 git commit
 git stash push
 test
 git commit --amend
 git stash pop
— without temporary files contrary to my solution:
 git diff >unstaged
 git restore .
 test
 git commit
 git apply unstaged
 rm unstaged

Like this:


edit
X X X
Y
Z

git add Y
X X X
Y Y
Z

git stash push --unstaged (saves X Y +Z, not X +Y +Z like --keep-index does)
X X X
Y Y

while test fails {

git stash pop (restores X Y +Z)
X X X
Y Y
Z

edit
X X X
Y'Y
Z

git add Y'
X X X
Y'Y'
Z

git stash push --unstaged (saves X Y' +Z)
X X X
Y'Y'

}

git commit
X X X
Y Y Y

git stash pop (restores X Y +Z)
X X X
Y Y Y
Z


Best,

Géry Ogam

Re: Stage, test, and commit only some changes, then repeat

[Géry Ogam]

> Le 31 janv. 2022 à 22:56, Sergey Organov <sorganov@gmail.com> a écrit :
> 
> Géry Ogam <gery.ogam@gmail.com> writes:
> 
>>> Le 31 janv. 2022 à 17:27, Sergey Organov <sorganov@gmail.com> a écrit :
>>> 
>>> Géry Ogam <gery.ogam@gmail.com> writes:
>>> 
>>>> Hello,
>>>> 
>>>> I would like to stage, test, and commit only *some* changes of the
>>>> working tree, and then repeat this process with the remaining changes.
>>>> 
>>>> My current solution (published at
>>>> https://stackoverflow.com/a/70914962/2326961):
>>>> 
>>>> 1. Stage some changes:
>>>>  ```
>>>>  git add -p file
>>>>  ```
>>>> 2. Save away the remaining changes:
>>>>  ```
>>>>  git diff >patch
>>>>  git stash push -k
>>>>  ```
>>>> 3. Test the staged changes.
>>>> 4. Commit the staged changes:
>>>>  ```
>>>>  git commit
>>>>  ```
>>>> 5. Restore the remaining changes:
>>>>  ```
>>>>  git apply patch
>>>>  ```
>>>> 6. Go to step 1.
>>>> 
>>>> It is not ideal because a) it uses a patch file for saving the
>>>> remaining changes; b) it uses the stash only for setting the working
>>>> tree to the index state.
>>>> 
>>>> It would be ideal if I could save *only* the remaining changes in the
>>>> stash instead of resorting to a patch file. How to do it?
>>> 
>>> It looks like you don't need patch file for this workflow. What's wrong with:
>>> 
>>> git add...
>>> git stash push --keep-index
>>> ... check, git add fixes
>>> git commit
>>> git stash apply
>>> 
>>> ???
>>> 
>>> -- Sergey Organov
>> 
>> Hello Sergey,
>> 
>> `git stash` saves the transition from the HEAD state to the working
>> tree state. It also sets the working tree to the *HEAD* state.
>> 
>> `git stash --keep-index` saves the transition from the HEAD state to
>> the working tree state. It also sets the working tree to the *index*
>> state.
>> 
>> `git stash pop` applies the last saved transition. So if the working
>> tree was not in HEAD state (like after `git stash --keep-index`),
>> there will be a conflict.
> 
> Did you actually try it and got conflict? I doubt there will be any if
> you don't modify anything after "git stash --keep-index" during testing,
> and if you do, than any method might bring conflicts.
> 
> In fact I just re-tested this to make sure, and got no conflicts.
> 
> -- Sergey Organov

git init
touch file
git add file
git commit
echo one >>file
git add file
echo two >>file
git stash push --keep-index
git stash pop

Géry Ogam

Re: Stage, test, and commit only some changes, then repeat

[Sergey Organov]

Géry Ogam <gery.ogam@gmail.com> writes:

>> Le 31 janv. 2022 à 22:56, Sergey Organov <sorganov@gmail.com> a écrit :
>> 
>> Géry Ogam <gery.ogam@gmail.com> writes:
>> 
>>>> Le 31 janv. 2022 à 17:27, Sergey Organov <sorganov@gmail.com> a écrit :
>>>> 
>>>> Géry Ogam <gery.ogam@gmail.com> writes:
>>>> 
>>>>> Hello,
>>>>> 
>>>>> I would like to stage, test, and commit only *some* changes of the
>>>>> working tree, and then repeat this process with the remaining changes.
>>>>> 
>>>>> My current solution (published at
>>>>> https://stackoverflow.com/a/70914962/2326961):
>>>>> 
>>>>> 1. Stage some changes:
>>>>>  ```
>>>>>  git add -p file
>>>>>  ```
>>>>> 2. Save away the remaining changes:
>>>>>  ```
>>>>>  git diff >patch
>>>>>  git stash push -k
>>>>>  ```
>>>>> 3. Test the staged changes.
>>>>> 4. Commit the staged changes:
>>>>>  ```
>>>>>  git commit
>>>>>  ```
>>>>> 5. Restore the remaining changes:
>>>>>  ```
>>>>>  git apply patch
>>>>>  ```
>>>>> 6. Go to step 1.
>>>>> 
>>>>> It is not ideal because a) it uses a patch file for saving the
>>>>> remaining changes; b) it uses the stash only for setting the working
>>>>> tree to the index state.
>>>>> 
>>>>> It would be ideal if I could save *only* the remaining changes in the
>>>>> stash instead of resorting to a patch file. How to do it?
>>>> 
>>>> It looks like you don't need patch file for this workflow. What's
>>>> wrong with:
>>>> 
>>>> git add...
>>>> git stash push --keep-index
>>>> ... check, git add fixes
>>>> git commit
>>>> git stash apply
>>>> 
>>>> ???
>>>> 
>>>> -- Sergey Organov
>>> 
>>> Hello Sergey,
>>> 
>>> `git stash` saves the transition from the HEAD state to the working
>>> tree state. It also sets the working tree to the *HEAD* state.
>>> 
>>> `git stash --keep-index` saves the transition from the HEAD state to
>>> the working tree state. It also sets the working tree to the *index*
>>> state.
>>> 
>>> `git stash pop` applies the last saved transition. So if the working
>>> tree was not in HEAD state (like after `git stash --keep-index`),
>>> there will be a conflict.
>> 
>> Did you actually try it and got conflict? I doubt there will be any if
>> you don't modify anything after "git stash --keep-index" during testing,
>> and if you do, than any method might bring conflicts.
>> 
>> In fact I just re-tested this to make sure, and got no conflicts.
>> 
>> -- Sergey Organov
>
> git init
> touch file
> git add file
> git commit
> echo one >>file
> git add file
> echo two >>file
> git stash push --keep-index
> git stash pop

Yep, if you have overlapping changes in work-tree and in the index, it
will happen indeed. I've rather tested handling of non-overlapping
changes in the same file that occurs much more often in practice, at
least for me.

BTW, for reference, Emacs's magit, that is essentially alternate Git
porcelain, has support for all 4 possible modes of stashing:

z Save                  Z Snapshot          
i Save index            I Snapshot index    
w Save worktree         W Snapshot worktree 
x Save keeping index    r Snapshot to wipref

For Git itself I've recently added --staged/-S, so the only mode that is
still missing is --worktree indeed.

-- Sergey Organov