git reset --merge random behavior

git reset --merge random behavior

[Maximilian Reichel]

[-- Attachment #1: Type: text/plain, Size: 3137 bytes --]

Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)
Running the following script (test.sh) multiple times:

 ```
 #!/bin/bash
GIT_TRACE=true
export GIT_TRACE

mkdir gitDir0
cd gitDir0
env git -c init.defaultBranch=master init 
echo bar > bar.txt
env git add -v bar.txt 
env git -c user.name="P" -c user.email="e@mail.com" commit -v -m first 

mkdir ../gitDir2
cd ../gitDir2
env git -c init.defaultBranch=master init 
env git pull -v --set-upstream ../gitDir0 
echo foo > foo.txt
env git add -v foo.txt 
env git mv -v -f -- foo.txt bar.txt 
env git reset --merge HEAD
```

What did you expect to happen? (Expected behavior)
I expect that the script shows the same behavior on every execution.

What happened instead? (Actual behavior)
I encountered two different behaviors (random):
1. Most of the time the git reset command succeeds and the file gitDir2/bar.txt contains the string "bar" after running the script.
2. Sometimes the git reset command fails and the file gitDir2/bar.txt contains the string "foo" after running the script.
In this case, the reset command produces the following output:
```
error: Entry 'bar.txt' not uptodate. Cannot merge.
fatal: Could not reset index file to revision 'HEAD'.
```
The full console output of both cases is attached to this mail, but they only really differ in this error message.

Anything else you want to add:
I could observe this behavior on git 2.35.1 and 2.34.1.
The behavior is not affected by whether GIT_TRACE is set.

I wrote a script that executes this test script repeatedly and keeps track of the possible outcomes of the gitDir2/bar.txt file.
It will print a line each time the content differs from the previous execution.
This script is attached to this mail (runner.sh).
Sample output of this script:
```
execution #0 	 hash: c157a79031e1c40f85931829bc5fc552 	 date: Thu Mar 24 13:22:17 UTC 2022
execution #203 	 hash: d3b07384d113edec49eaa6238ad5ff00 	 date: Thu Mar 24 13:22:29 UTC 2022
execution #204 	 hash: c157a79031e1c40f85931829bc5fc552 	 date: Thu Mar 24 13:22:29 UTC 2022
execution #264 	 hash: d3b07384d113edec49eaa6238ad5ff00 	 date: Thu Mar 24 13:22:32 UTC 2022
execution #265 	 hash: c157a79031e1c40f85931829bc5fc552 	 date: Thu Mar 24 13:22:32 UTC 2022
execution #432 	 hash: d3b07384d113edec49eaa6238ad5ff00 	 date: Thu Mar 24 13:22:40 UTC 2022
execution #433 	 hash: c157a79031e1c40f85931829bc5fc552 	 date: Thu Mar 24 13:22:40 UTC 2022
execution #489 	 hash: d3b07384d113edec49eaa6238ad5ff00 	 date: Thu Mar 24 13:22:43 UTC 2022
```


[System Info]
git version:
git version 2.35.1
cpu: x86_64
built from commit: 4c53a8c20f8984adb226293a3ffd7b88c3f4ac1a
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname: Linux 5.10.25-linuxkit #1 SMP Tue Mar 23 09:27:39 UTC 2021 x86_64
compiler info: gnuc: 10.2
libc info: glibc: 2.31
$SHELL (typically, interactive shell): <unset>


[Enabled Hooks]
not run from a git repository - no hooks to show

[-- Attachment #2: runner.sh --]
[-- Type: application/octet-stream, Size: 1024 bytes --]

#!/bin/bash
outputs_dir=outputs
script=test.sh
work_dir=workDir
observed_file=gitDir2/bar.txt

mkdir -p "$work_dir"
mkdir -p "$outputs_dir"
i=0
last_hash=
trap "exit 0" INT
while true; do
    (   # prepare work dir
        rm -rf $work_dir && 
        mkdir $work_dir && 
        cp $script $work_dir/$script &&
        cd $work_dir &&
        ./$script # execute script
    ) > consoleOut.txt 2>&1
    hash=$(md5sum <${work_dir}/$observed_file | tr -cd '[:alnum:]') # use md5 instead of md5sum on macOS
    if [ "$hash" != "$last_hash" ]; then
        # report when file content has changed
        printf  "execution #%d \t hash: %s \t date: %s\n" $i "$hash" "$(date)"
        if [ ! -d "${outputs_dir}/${hash}/" ]; then
            # copy working dir and console output to output dir
            cp -r ${work_dir} "${outputs_dir}/${hash}"
            cp ${work_dir}/$observed_file "${outputs_dir}/${hash}.txt"
            cp consoleOut.txt "${outputs_dir}/${hash}"
        fi
    fi
    last_hash=$hash
    ((i=i+1))
done

[-- Attachment #3: case2ConsoleOut.txt --]
[-- Type: text/plain, Size: 2988 bytes --]

13:05:42.974156 git.c:458               trace: built-in: git init
warning: templates not found in /usr/local/share/git-core/templates
Initialized empty Git repository in /test2/workDir/gitDir0/.git/
13:05:42.976995 git.c:458               trace: built-in: git add -v bar.txt
add 'bar.txt'
13:05:42.979187 git.c:458               trace: built-in: git commit -v -m first
13:05:42.980431 run-command.c:654       trace: run_command: git maintenance run --auto --no-quiet
13:05:42.981375 git.c:458               trace: built-in: git maintenance run --auto --no-quiet
[master (root-commit) 4165aea] first
 1 file changed, 1 insertion(+)
 create mode 100644 bar.txt
13:05:42.984917 git.c:458               trace: built-in: git init
warning: templates not found in /usr/local/share/git-core/templates
Initialized empty Git repository in /test2/workDir/gitDir2/.git/
13:05:42.987933 git.c:458               trace: built-in: git pull -v --set-upstream ../gitDir0
13:05:42.988098 run-command.c:654       trace: run_command: git fetch --update-head-ok -v --set-upstream ../gitDir0
13:05:42.989104 git.c:458               trace: built-in: git fetch --update-head-ok -v --set-upstream ../gitDir0
13:05:42.989301 run-command.c:654       trace: run_command: unset GIT_PREFIX; GIT_PROTOCOL=version=2 'git-upload-pack '\''../gitDir0'\'''
13:05:42.990794 git.c:458               trace: built-in: git upload-pack ../gitDir0
13:05:42.991782 run-command.c:654       trace: run_command: git pack-objects --revs --thin --stdout --delta-base-offset
remote: 13:05:42.992962 git.c:458               trace: built-in: git pack-objects --revs --thin --stdout --delta-base-offset        
13:05:42.993799 run-command.c:654       trace: run_command: git unpack-objects -q --pack_header=2,3
13:05:42.995006 git.c:458               trace: built-in: git unpack-objects -q --pack_header=2,3
13:05:42.995872 run-command.c:654       trace: run_command: git rev-list --objects --stdin --not --all --quiet --alternate-refs
13:05:42.996906 git.c:458               trace: built-in: git rev-list --objects --stdin --not --all --quiet --alternate-refs
From ../gitDir0
 * branch            HEAD       -> FETCH_HEAD
13:05:42.998066 run-command.c:1582      run_processes_parallel: preparing to run up to 1 tasks
13:05:42.998076 run-command.c:1614      run_processes_parallel: done
13:05:42.998082 run-command.c:654       trace: run_command: git maintenance run --auto --no-quiet
13:05:42.999099 git.c:458               trace: built-in: git maintenance run --auto --no-quiet
13:05:43.002592 git.c:458               trace: built-in: git add -v foo.txt
add 'foo.txt'
13:05:43.004882 git.c:458               trace: built-in: git mv -v -f -- foo.txt bar.txt
warning: overwriting 'bar.txt'
Renaming foo.txt to bar.txt
13:05:43.007057 git.c:458               trace: built-in: git reset --merge HEAD
error: Entry 'bar.txt' not uptodate. Cannot merge.
fatal: Could not reset index file to revision 'HEAD'.

[-- Attachment #4: case1ConsoleOut.txt --]
[-- Type: text/plain, Size: 2881 bytes --]

13:05:36.550286 git.c:458               trace: built-in: git init
warning: templates not found in /usr/local/share/git-core/templates
Initialized empty Git repository in /test2/workDir/gitDir0/.git/
13:05:36.553483 git.c:458               trace: built-in: git add -v bar.txt
add 'bar.txt'
13:05:36.555923 git.c:458               trace: built-in: git commit -v -m first
13:05:36.557265 run-command.c:654       trace: run_command: git maintenance run --auto --no-quiet
13:05:36.558315 git.c:458               trace: built-in: git maintenance run --auto --no-quiet
[master (root-commit) aea16b2] first
 1 file changed, 1 insertion(+)
 create mode 100644 bar.txt
13:05:36.561756 git.c:458               trace: built-in: git init
warning: templates not found in /usr/local/share/git-core/templates
Initialized empty Git repository in /test2/workDir/gitDir2/.git/
13:05:36.564497 git.c:458               trace: built-in: git pull -v --set-upstream ../gitDir0
13:05:36.564679 run-command.c:654       trace: run_command: git fetch --update-head-ok -v --set-upstream ../gitDir0
13:05:36.565640 git.c:458               trace: built-in: git fetch --update-head-ok -v --set-upstream ../gitDir0
13:05:36.565830 run-command.c:654       trace: run_command: unset GIT_PREFIX; GIT_PROTOCOL=version=2 'git-upload-pack '\''../gitDir0'\'''
13:05:36.567311 git.c:458               trace: built-in: git upload-pack ../gitDir0
13:05:36.568164 run-command.c:654       trace: run_command: git pack-objects --revs --thin --stdout --delta-base-offset
remote: 13:05:36.569156 git.c:458               trace: built-in: git pack-objects --revs --thin --stdout --delta-base-offset        
13:05:36.570130 run-command.c:654       trace: run_command: git unpack-objects -q --pack_header=2,3
13:05:36.571443 git.c:458               trace: built-in: git unpack-objects -q --pack_header=2,3
13:05:36.572322 run-command.c:654       trace: run_command: git rev-list --objects --stdin --not --all --quiet --alternate-refs
13:05:36.573357 git.c:458               trace: built-in: git rev-list --objects --stdin --not --all --quiet --alternate-refs
From ../gitDir0
 * branch            HEAD       -> FETCH_HEAD
13:05:36.574654 run-command.c:1582      run_processes_parallel: preparing to run up to 1 tasks
13:05:36.574663 run-command.c:1614      run_processes_parallel: done
13:05:36.574669 run-command.c:654       trace: run_command: git maintenance run --auto --no-quiet
13:05:36.575573 git.c:458               trace: built-in: git maintenance run --auto --no-quiet
13:05:36.578926 git.c:458               trace: built-in: git add -v foo.txt
add 'foo.txt'
13:05:36.581235 git.c:458               trace: built-in: git mv -v -f -- foo.txt bar.txt
warning: overwriting 'bar.txt'
Renaming foo.txt to bar.txt
13:05:36.583892 git.c:458               trace: built-in: git reset --merge HEAD

Re: git reset --merge random behavior

[Victoria Dye]

Maximilian Reichel wrote:
> Thank you for filling out a Git bug report!
> Please answer the following questions to help us understand your issue.
> 
> What did you do before the bug happened? (Steps to reproduce your issue)
> Running the following script (test.sh) multiple times:
> 
>  ```
>  #!/bin/bash
> GIT_TRACE=true
> export GIT_TRACE
> 
> mkdir gitDir0
> cd gitDir0
> env git -c init.defaultBranch=master init 
> echo bar > bar.txt
> env git add -v bar.txt 
> env git -c user.name="P" -c user.email="e@mail.com" commit -v -m first 
> 
> mkdir ../gitDir2
> cd ../gitDir2
> env git -c init.defaultBranch=master init 
> env git pull -v --set-upstream ../gitDir0 
> echo foo > foo.txt
> env git add -v foo.txt 
> env git mv -v -f -- foo.txt bar.txt 
> env git reset --merge HEAD
> ```
> 
> What did you expect to happen? (Expected behavior)
> I expect that the script shows the same behavior on every execution.
> 
> What happened instead? (Actual behavior)
> I encountered two different behaviors (random):
> 1. Most of the time the git reset command succeeds and the file gitDir2/bar.txt contains the string "bar" after running the script.
> 2. Sometimes the git reset command fails and the file gitDir2/bar.txt contains the string "foo" after running the script.
> In this case, the reset command produces the following output:
> ```
> error: Entry 'bar.txt' not uptodate. Cannot merge.
> fatal: Could not reset index file to revision 'HEAD'.
> ```
> The full console output of both cases is attached to this mail, but they only really differ in this error message.
> 

I was able to reliably reproduce both the success and failure scenarios with
a slightly simplified script:

# Setup
$ git init test-repo
$ cd test-repo
$ echo bar >bar.txt
$ git commit -am first

# Succeeds
$ echo foo >foo.txt
$ git add foo.txt
$ git mv -f foo.txt bar.txt
$ git update-index --refresh
$ git reset --merge HEAD

# Fails
$ echo foo >foo.txt
$ git mv -f foo.txt bar.txt
$ git reset --merge HEAD

It looks like the error happens because 'mv' doesn't refresh the
"up-to-date" flag on 'bar.txt', so the subsequent 'reset --merge' thinks
there's a conflict that doesn't actually exist. 

I've submitted a patch [1] to fix this permanently, but if you need a
workaround in the meantime, you can call 'git update-index --refresh' after
'git mv' to avoid the error.

[1] https://lore.kernel.org/git/pull.1187.git.1648173419533.gitgitgadget@gmail.com/

> Anything else you want to add:
> I could observe this behavior on git 2.35.1 and 2.34.1.
> The behavior is not affected by whether GIT_TRACE is set.
> 
> I wrote a script that executes this test script repeatedly and keeps track of the possible outcomes of the gitDir2/bar.txt file.
> It will print a line each time the content differs from the previous execution.
> This script is attached to this mail (runner.sh).
> Sample output of this script:
> ```
> execution #0 	 hash: c157a79031e1c40f85931829bc5fc552 	 date: Thu Mar 24 13:22:17 UTC 2022
> execution #203 	 hash: d3b07384d113edec49eaa6238ad5ff00 	 date: Thu Mar 24 13:22:29 UTC 2022
> execution #204 	 hash: c157a79031e1c40f85931829bc5fc552 	 date: Thu Mar 24 13:22:29 UTC 2022
> execution #264 	 hash: d3b07384d113edec49eaa6238ad5ff00 	 date: Thu Mar 24 13:22:32 UTC 2022
> execution #265 	 hash: c157a79031e1c40f85931829bc5fc552 	 date: Thu Mar 24 13:22:32 UTC 2022
> execution #432 	 hash: d3b07384d113edec49eaa6238ad5ff00 	 date: Thu Mar 24 13:22:40 UTC 2022
> execution #433 	 hash: c157a79031e1c40f85931829bc5fc552 	 date: Thu Mar 24 13:22:40 UTC 2022
> execution #489 	 hash: d3b07384d113edec49eaa6238ad5ff00 	 date: Thu Mar 24 13:22:43 UTC 2022
> ```
> 
> 
> [System Info]
> git version:
> git version 2.35.1
> cpu: x86_64
> built from commit: 4c53a8c20f8984adb226293a3ffd7b88c3f4ac1a
> sizeof-long: 8
> sizeof-size_t: 8
> shell-path: /bin/sh
> uname: Linux 5.10.25-linuxkit #1 SMP Tue Mar 23 09:27:39 UTC 2021 x86_64
> compiler info: gnuc: 10.2
> libc info: glibc: 2.31
> $SHELL (typically, interactive shell): <unset>
> 
> 
> [Enabled Hooks]
> not run from a git repository - no hooks to show