* Noticed big performance issue when fetching multiple refs with depth 1 enabled
@ 2022-03-18 14:44 Ricard Valverde
2022-03-18 20:21 ` brian m. carlson
0 siblings, 1 reply; 2+ messages in thread
From: Ricard Valverde @ 2022-03-18 14:44 UTC (permalink / raw)
To: git
[-- Attachment #1: Type: text/plain, Size: 1441 bytes --]
Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.
What did you do before the bug happened? (Steps to reproduce your issue)
Do a git fetch --depth=1 , with no remote changes, having a lot of
remote branches makes the issue more visible
What did you expect to happen? (Expected behavior)
Shallow fetch should be faster or as fast as a full fetch
What happened instead? (Actual behavior)
Shallow fetch is about 10x slower than full fetch, consistently
What's different between what you expected and what actually happened?
I expected the shallow fetch to be faster than a normal fetch, or as
fast. Did not expect it being 10x slower.
Anything else you want to add:
Tested in a repository with ~10000 remote branches, a single Github
hosted remote.
Also tested with a local remote, and Git 2.35.1 version, with
comparable results.
Please review the rest of the bug report below.
You can delete any lines you don't wish to share.
[System Info]
git version:
git version 2.32.0 (Apple Git-132)
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname: Darwin 20.6.0 Darwin Kernel Version 20.6.0: Tue Feb 22 21:10:41
PST 2022; root:xnu-7195.141.26~1/RELEASE_X86_64 x86_64
compiler info: clang: 13.0.0 (clang-1300.0.29.30)
libc info: no libc information available
$SHELL (typically, interactive shell): /bin/bash
[Enabled Hooks]
[-- Attachment #2: fetch_github_with_depth.log --]
[-- Type: application/octet-stream, Size: 10945 bytes --]
15:00:02.181493 git.c:458 trace: built-in: git fetch --verbose --progress --prune --depth 1 -- git@github.com:socialpoint/sp-mobile-NewDragonCity.git '+refs/pull/10*:refs/remotes/origin/pr/10*'\r15:00:02.184520 run-command.c:654 trace: run_command: unset GIT_PREFIX; GIT_PROTOCOL=version=2 ssh -o SendEnv=GIT_PROTOCOL git@github.com 'git-upload-pack '\''socialpoint/sp-mobile-NewDragonCity.git'\'''\rremote: Enumerating objects: 27974, done. \rremote: Counting objects: 0% (1/25371) \r...\rremote: Counting objects: 100% (25371/25371), done. \rremote: Compressing objects: 0% (1/17591) \r... \rremote: Compressing objects: 100% (17591/17591), done. \r15:00:14.500533 run-command.c:654 trace: run_command: git --shallow-file /Users/ricardo/GitProjects/sp-mobile-NewDragonCity/.git/shallow.lock index-pack --stdin -v --fix-thin '--keep=fetch-pack 73254 on SPTM-RVALVERDE' --pack_header=2,24884\r15:00:14.509521 git.c:458 trace: built-in: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 73254 on SPTM-RVALVERDE' --pack_header=2,24884\rReceiving objects: 0% (1/24884)\r...\rReceiving objects: 99% (24636/24884), 101.34 MiB | 16.84 MiB/s\rremote: Total 24884 (delta 10981), reused 17977 (delta 6867), pack-reused 0 \rReceiving objects: 100% (24884/24884), 101.34 MiB | 16.84 MiB/s\rReceiving objects: 100% (24884/24884), 108.73 MiB | 15.34 MiB/s, done.\rResolving deltas: 0% (0/10981)\r...\rResolving deltas: 100% (10981/10981), completed with 130 local objects.\r15:00:22.753191 run-command.c:654 trace: run_command: git --shallow-file /Users/ricardo/GitProjects/sp-mobile-NewDragonCity/.git/shallow.lock rev-list --objects --stdin --quiet --alternate-refs\r15:00:22.765005 git.c:458 trace: built-in: git rev-list --objects --stdin --quiet --alternate-refs\rFrom github.com:socialpoint/sp-mobile-NewDragonCity\r = [up to date] refs/pull/10/head -> origin/pr/10/head\r = [up to date] refs/pull/100/head -> origin/pr/100/head\r = [up to date] refs/pull/1000/head -> origin/pr/1000/head\r = [up to date] refs/pull/1001/head -> origin/pr/1001/head\r = [up to date] refs/pull/1002/head -> origin/pr/1002/head\r = [up to date] refs/pull/1003/head -> origin/pr/1003/head\r = [up to date] refs/pull/1004/head -> origin/pr/1004/head\r = [up to date] refs/pull/1005/head -> origin/pr/1005/head\r = [up to date] refs/pull/1006/head -> origin/pr/1006/head\r = [up to date] refs/pull/1007/head -> origin/pr/1007/head\r = [up to date] refs/pull/1008/head -> origin/pr/1008/head\r = [up to date] refs/pull/1009/head -> origin/pr/1009/head\r = [up to date] refs/pull/101/head -> origin/pr/101/head\r = [up to date] refs/pull/1010/head -> origin/pr/1010/head\r = [up to date] refs/pull/1011/head -> origin/pr/1011/head\r = [up to date] refs/pull/1012/head -> origin/pr/1012/head\r = [up to date] refs/pull/1013/head -> origin/pr/1013/head\r = [up to date] refs/pull/1014/head -> origin/pr/1014/head\r = [up to date] refs/pull/1015/head -> origin/pr/1015/head\r = [up to date] refs/pull/1016/head -> origin/pr/1016/head\r = [up to date] refs/pull/1017/head -> origin/pr/1017/head\r = [up to date] refs/pull/1018/head -> origin/pr/1018/head\r = [up to date] refs/pull/1019/head -> origin/pr/1019/head\r = [up to date] refs/pull/102/head -> origin/pr/102/head\r = [up to date] refs/pull/1020/head -> origin/pr/1020/head\r = [up to date] refs/pull/1021/head -> origin/pr/1021/head\r = [up to date] refs/pull/1022/head -> origin/pr/1022/head\r = [up to date] refs/pull/1023/head -> origin/pr/1023/head\r = [up to date] refs/pull/1024/head -> origin/pr/1024/head\r = [up to date] refs/pull/1025/head -> origin/pr/1025/head\r = [up to date] refs/pull/1026/head -> origin/pr/1026/head\r = [up to date] refs/pull/1027/head -> origin/pr/1027/head\r = [up to date] refs/pull/1027/merge -> origin/pr/1027/merge\r = [up to date] refs/pull/1028/head -> origin/pr/1028/head\r = [up to date] refs/pull/1029/head -> origin/pr/1029/head\r = [up to date] refs/pull/103/head -> origin/pr/103/head\r = [up to date] refs/pull/1030/head -> origin/pr/1030/head\r = [up to date] refs/pull/1031/head -> origin/pr/1031/head\r = [up to date] refs/pull/1032/head -> origin/pr/1032/head\r = [up to date] refs/pull/1033/head -> origin/pr/1033/head\r = [up to date] refs/pull/1034/head -> origin/pr/1034/head\r = [up to date] refs/pull/1035/head -> origin/pr/1035/head\r = [up to date] refs/pull/1036/head -> origin/pr/1036/head\r = [up to date] refs/pull/1037/head -> origin/pr/1037/head\r = [up to date] refs/pull/1038/head -> origin/pr/1038/head\r = [up to date] refs/pull/1039/head -> origin/pr/1039/head\r = [up to date] refs/pull/104/head -> origin/pr/104/head\r = [up to date] refs/pull/104/merge -> origin/pr/104/merge\r = [up to date] refs/pull/1040/head -> origin/pr/1040/head\r = [up to date] refs/pull/1040/merge -> origin/pr/1040/merge\r = [up to date] refs/pull/1041/head -> origin/pr/1041/head\r = [up to date] refs/pull/1041/merge -> origin/pr/1041/merge\r = [up to date] refs/pull/1042/head -> origin/pr/1042/head\r = [up to date] refs/pull/1042/merge -> origin/pr/1042/merge\r = [up to date] refs/pull/1043/head -> origin/pr/1043/head\r = [up to date] refs/pull/1044/head -> origin/pr/1044/head\r = [up to date] refs/pull/1045/head -> origin/pr/1045/head\r = [up to date] refs/pull/1046/head -> origin/pr/1046/head\r = [up to date] refs/pull/1047/head -> origin/pr/1047/head\r = [up to date] refs/pull/1048/head -> origin/pr/1048/head\r = [up to date] refs/pull/1049/head -> origin/pr/1049/head\r = [up to date] refs/pull/105/head -> origin/pr/105/head\r = [up to date] refs/pull/1050/head -> origin/pr/1050/head\r = [up to date] refs/pull/1051/head -> origin/pr/1051/head\r = [up to date] refs/pull/1052/head -> origin/pr/1052/head\r = [up to date] refs/pull/1053/head -> origin/pr/1053/head\r = [up to date] refs/pull/1054/head -> origin/pr/1054/head\r = [up to date] refs/pull/1055/head -> origin/pr/1055/head\r = [up to date] refs/pull/1056/head -> origin/pr/1056/head\r = [up to date] refs/pull/1057/head -> origin/pr/1057/head\r = [up to date] refs/pull/1058/head -> origin/pr/1058/head\r = [up to date] refs/pull/1059/head -> origin/pr/1059/head\r = [up to date] refs/pull/106/head -> origin/pr/106/head\r = [up to date] refs/pull/1060/head -> origin/pr/1060/head\r = [up to date] refs/pull/1061/head -> origin/pr/1061/head\r = [up to date] refs/pull/1062/head -> origin/pr/1062/head\r = [up to date] refs/pull/1063/head -> origin/pr/1063/head\r = [up to date] refs/pull/1063/merge -> origin/pr/1063/merge\r = [up to date] refs/pull/1064/head -> origin/pr/1064/head\r = [up to date] refs/pull/1065/head -> origin/pr/1065/head\r = [up to date] refs/pull/1066/head -> origin/pr/1066/head\r = [up to date] refs/pull/1067/head -> origin/pr/1067/head\r = [up to date] refs/pull/1068/head -> origin/pr/1068/head\r = [up to date] refs/pull/1069/head -> origin/pr/1069/head\r = [up to date] refs/pull/107/head -> origin/pr/107/head\r = [up to date] refs/pull/1070/head -> origin/pr/1070/head\r = [up to date] refs/pull/1071/head -> origin/pr/1071/head\r = [up to date] refs/pull/1071/merge -> origin/pr/1071/merge\r = [up to date] refs/pull/1072/head -> origin/pr/1072/head\r = [up to date] refs/pull/1072/merge -> origin/pr/1072/merge\r = [up to date] refs/pull/1073/head -> origin/pr/1073/head\r = [up to date] refs/pull/1074/head -> origin/pr/1074/head\r = [up to date] refs/pull/1075/head -> origin/pr/1075/head\r = [up to date] refs/pull/1076/head -> origin/pr/1076/head\r = [up to date] refs/pull/1077/head -> origin/pr/1077/head\r = [up to date] refs/pull/1078/head -> origin/pr/1078/head\r = [up to date] refs/pull/1079/head -> origin/pr/1079/head\r = [up to date] refs/pull/108/head -> origin/pr/108/head\r = [up to date] refs/pull/1080/head -> origin/pr/1080/head\r = [up to date] refs/pull/1081/head -> origin/pr/1081/head\r = [up to date] refs/pull/1082/head -> origin/pr/1082/head\r = [up to date] refs/pull/1083/head -> origin/pr/1083/head\r = [up to date] refs/pull/1084/head -> origin/pr/1084/head\r = [up to date] refs/pull/1085/head -> origin/pr/1085/head\r = [up to date] refs/pull/1086/head -> origin/pr/1086/head\r = [up to date] refs/pull/1087/head -> origin/pr/1087/head\r = [up to date] refs/pull/1088/head -> origin/pr/1088/head\r = [up to date] refs/pull/1089/head -> origin/pr/1089/head\r = [up to date] refs/pull/109/head -> origin/pr/109/head\r = [up to date] refs/pull/1090/head -> origin/pr/1090/head\r = [up to date] refs/pull/1091/head -> origin/pr/1091/head\r = [up to date] refs/pull/1092/head -> origin/pr/1092/head\r = [up to date] refs/pull/1093/head -> origin/pr/1093/head\r = [up to date] refs/pull/1094/head -> origin/pr/1094/head\r = [up to date] refs/pull/1095/head -> origin/pr/1095/head\r = [up to date] refs/pull/1096/head -> origin/pr/1096/head\r = [up to date] refs/pull/1097/head -> origin/pr/1097/head\r = [up to date] refs/pull/1098/head -> origin/pr/1098/head\r = [up to date] refs/pull/1099/head -> origin/pr/1099/head\r15:00:22.866790 run-command.c:1582 run_processes_parallel: preparing to run up to 1 tasks\r15:00:22.866999 run-command.c:1614 run_processes_parallel: done\r15:00:22.867025 run-command.c:654 trace: run_command: git maintenance run --auto --no-quiet\r15:00:22.875049 git.c:458 trace: built-in: git maintenance run --auto --no-quiet\r
[-- Attachment #3: fetch_github_without_depth.log --]
[-- Type: application/octet-stream, Size: 9627 bytes --]
14:59:33.669234 git.c:458 trace: built-in: git fetch --verbose --progress --prune -- git@github.com:socialpoint/sp-mobile-NewDragonCity.git '+refs/pull/10*:refs/remotes/origin/pr/10*'
14:59:33.672063 run-command.c:654 trace: run_command: unset GIT_PREFIX; GIT_PROTOCOL=version=2 ssh -o SendEnv=GIT_PROTOCOL git@github.com 'git-upload-pack '\''socialpoint/sp-mobile-NewDragonCity.git'\'''
14:59:40.828277 run-command.c:654 trace: run_command: git rev-list --objects --stdin --not --all --quiet --alternate-refs
From github.com:socialpoint/sp-mobile-NewDragonCity
= [up to date] refs/pull/10/head -> origin/pr/10/head
= [up to date] refs/pull/100/head -> origin/pr/100/head
= [up to date] refs/pull/1000/head -> origin/pr/1000/head
= [up to date] refs/pull/1001/head -> origin/pr/1001/head
= [up to date] refs/pull/1002/head -> origin/pr/1002/head
= [up to date] refs/pull/1003/head -> origin/pr/1003/head
= [up to date] refs/pull/1004/head -> origin/pr/1004/head
= [up to date] refs/pull/1005/head -> origin/pr/1005/head
= [up to date] refs/pull/1006/head -> origin/pr/1006/head
= [up to date] refs/pull/1007/head -> origin/pr/1007/head
= [up to date] refs/pull/1008/head -> origin/pr/1008/head
= [up to date] refs/pull/1009/head -> origin/pr/1009/head
= [up to date] refs/pull/101/head -> origin/pr/101/head
= [up to date] refs/pull/1010/head -> origin/pr/1010/head
= [up to date] refs/pull/1011/head -> origin/pr/1011/head
= [up to date] refs/pull/1012/head -> origin/pr/1012/head
= [up to date] refs/pull/1013/head -> origin/pr/1013/head
= [up to date] refs/pull/1014/head -> origin/pr/1014/head
= [up to date] refs/pull/1015/head -> origin/pr/1015/head
= [up to date] refs/pull/1016/head -> origin/pr/1016/head
= [up to date] refs/pull/1017/head -> origin/pr/1017/head
= [up to date] refs/pull/1018/head -> origin/pr/1018/head
= [up to date] refs/pull/1019/head -> origin/pr/1019/head
= [up to date] refs/pull/102/head -> origin/pr/102/head
= [up to date] refs/pull/1020/head -> origin/pr/1020/head
= [up to date] refs/pull/1021/head -> origin/pr/1021/head
= [up to date] refs/pull/1022/head -> origin/pr/1022/head
= [up to date] refs/pull/1023/head -> origin/pr/1023/head
= [up to date] refs/pull/1024/head -> origin/pr/1024/head
= [up to date] refs/pull/1025/head -> origin/pr/1025/head
= [up to date] refs/pull/1026/head -> origin/pr/1026/head
= [up to date] refs/pull/1027/head -> origin/pr/1027/head
= [up to date] refs/pull/1027/merge -> origin/pr/1027/merge
= [up to date] refs/pull/1028/head -> origin/pr/1028/head
= [up to date] refs/pull/1029/head -> origin/pr/1029/head
= [up to date] refs/pull/103/head -> origin/pr/103/head
= [up to date] refs/pull/1030/head -> origin/pr/1030/head
= [up to date] refs/pull/1031/head -> origin/pr/1031/head
= [up to date] refs/pull/1032/head -> origin/pr/1032/head
= [up to date] refs/pull/1033/head -> origin/pr/1033/head
= [up to date] refs/pull/1034/head -> origin/pr/1034/head
= [up to date] refs/pull/1035/head -> origin/pr/1035/head
= [up to date] refs/pull/1036/head -> origin/pr/1036/head
= [up to date] refs/pull/1037/head -> origin/pr/1037/head
= [up to date] refs/pull/1038/head -> origin/pr/1038/head
= [up to date] refs/pull/1039/head -> origin/pr/1039/head
= [up to date] refs/pull/104/head -> origin/pr/104/head
= [up to date] refs/pull/104/merge -> origin/pr/104/merge
= [up to date] refs/pull/1040/head -> origin/pr/1040/head
= [up to date] refs/pull/1040/merge -> origin/pr/1040/merge
= [up to date] refs/pull/1041/head -> origin/pr/1041/head
= [up to date] refs/pull/1041/merge -> origin/pr/1041/merge
= [up to date] refs/pull/1042/head -> origin/pr/1042/head
= [up to date] refs/pull/1042/merge -> origin/pr/1042/merge
= [up to date] refs/pull/1043/head -> origin/pr/1043/head
= [up to date] refs/pull/1044/head -> origin/pr/1044/head
= [up to date] refs/pull/1045/head -> origin/pr/1045/head
= [up to date] refs/pull/1046/head -> origin/pr/1046/head
= [up to date] refs/pull/1047/head -> origin/pr/1047/head
= [up to date] refs/pull/1048/head -> origin/pr/1048/head
= [up to date] refs/pull/1049/head -> origin/pr/1049/head
= [up to date] refs/pull/105/head -> origin/pr/105/head
= [up to date] refs/pull/1050/head -> origin/pr/1050/head
= [up to date] refs/pull/1051/head -> origin/pr/1051/head
= [up to date] refs/pull/1052/head -> origin/pr/1052/head
= [up to date] refs/pull/1053/head -> origin/pr/1053/head
= [up to date] refs/pull/1054/head -> origin/pr/1054/head
= [up to date] refs/pull/1055/head -> origin/pr/1055/head
= [up to date] refs/pull/1056/head -> origin/pr/1056/head
= [up to date] refs/pull/1057/head -> origin/pr/1057/head
= [up to date] refs/pull/1058/head -> origin/pr/1058/head
= [up to date] refs/pull/1059/head -> origin/pr/1059/head
= [up to date] refs/pull/106/head -> origin/pr/106/head
= [up to date] refs/pull/1060/head -> origin/pr/1060/head
= [up to date] refs/pull/1061/head -> origin/pr/1061/head
= [up to date] refs/pull/1062/head -> origin/pr/1062/head
= [up to date] refs/pull/1063/head -> origin/pr/1063/head
= [up to date] refs/pull/1063/merge -> origin/pr/1063/merge
= [up to date] refs/pull/1064/head -> origin/pr/1064/head
= [up to date] refs/pull/1065/head -> origin/pr/1065/head
= [up to date] refs/pull/1066/head -> origin/pr/1066/head
= [up to date] refs/pull/1067/head -> origin/pr/1067/head
= [up to date] refs/pull/1068/head -> origin/pr/1068/head
= [up to date] refs/pull/1069/head -> origin/pr/1069/head
= [up to date] refs/pull/107/head -> origin/pr/107/head
= [up to date] refs/pull/1070/head -> origin/pr/1070/head
= [up to date] refs/pull/1071/head -> origin/pr/1071/head
= [up to date] refs/pull/1071/merge -> origin/pr/1071/merge
= [up to date] refs/pull/1072/head -> origin/pr/1072/head
= [up to date] refs/pull/1072/merge -> origin/pr/1072/merge
= [up to date] refs/pull/1073/head -> origin/pr/1073/head
= [up to date] refs/pull/1074/head -> origin/pr/1074/head
= [up to date] refs/pull/1075/head -> origin/pr/1075/head
= [up to date] refs/pull/1076/head -> origin/pr/1076/head
= [up to date] refs/pull/1077/head -> origin/pr/1077/head
= [up to date] refs/pull/1078/head -> origin/pr/1078/head
= [up to date] refs/pull/1079/head -> origin/pr/1079/head
= [up to date] refs/pull/108/head -> origin/pr/108/head
= [up to date] refs/pull/1080/head -> origin/pr/1080/head
= [up to date] refs/pull/1081/head -> origin/pr/1081/head
= [up to date] refs/pull/1082/head -> origin/pr/1082/head
= [up to date] refs/pull/1083/head -> origin/pr/1083/head
= [up to date] refs/pull/1084/head -> origin/pr/1084/head
= [up to date] refs/pull/1085/head -> origin/pr/1085/head
= [up to date] refs/pull/1086/head -> origin/pr/1086/head
= [up to date] refs/pull/1087/head -> origin/pr/1087/head
= [up to date] refs/pull/1088/head -> origin/pr/1088/head
= [up to date] refs/pull/1089/head -> origin/pr/1089/head
= [up to date] refs/pull/109/head -> origin/pr/109/head
= [up to date] refs/pull/1090/head -> origin/pr/1090/head
= [up to date] refs/pull/1091/head -> origin/pr/1091/head
= [up to date] refs/pull/1092/head -> origin/pr/1092/head
= [up to date] refs/pull/1093/head -> origin/pr/1093/head
= [up to date] refs/pull/1094/head -> origin/pr/1094/head
= [up to date] refs/pull/1095/head -> origin/pr/1095/head
= [up to date] refs/pull/1096/head -> origin/pr/1096/head
= [up to date] refs/pull/1097/head -> origin/pr/1097/head
= [up to date] refs/pull/1098/head -> origin/pr/1098/head
= [up to date] refs/pull/1099/head -> origin/pr/1099/head
14:59:46.238505 run-command.c:1582 run_processes_parallel: preparing to run up to 1 tasks
14:59:46.238698 run-command.c:1614 run_processes_parallel: done
14:59:46.238720 run-command.c:654 trace: run_command: git maintenance run --auto --no-quiet
14:59:46.245989 git.c:458 trace: built-in: git maintenance run --auto --no-quiet
[-- Attachment #4: fetch_local_with_depth.log --]
[-- Type: application/octet-stream, Size: 11064 bytes --]
14:57:55.217888 git.c:458 trace: built-in: git fetch --verbose --progress --prune --depth 1 -- file:///Users/ricardo/GitProjects/sp-mobile-NewDragonCity/ '+refs/remotes/origin/pr/10*:refs/remotes/origin/pr/10*'\r14:57:55.220384 run-command.c:654 trace: run_command: unset GIT_PREFIX; GIT_PROTOCOL=version=2 'git-upload-pack '\''/Users/ricardo/GitProjects/sp-mobile-NewDragonCity/'\'''\r14:57:55.229509 git.c:458 trace: built-in: git upload-pack /Users/ricardo/GitProjects/sp-mobile-NewDragonCity/\r14:58:00.738001 run-command.c:654 trace: run_command: git --shallow-file '' pack-objects --revs --thin --stdout --shallow --progress --delta-base-offset --include-tag\rremote: 14:58:00.744151 git.c:458 trace: built-in: git pack-objects --revs --thin --stdout --shallow --progress --delta-base-offset --include-tag \rremote: Enumerating objects: 27974, done. \rremote: Counting objects: 0% (1/25369) \r... \rremote: Counting objects: 100% (25369/25369), done. \rremote: Compressing objects: 0% (1/17569) \r... \rremote: Compressing objects: 100% (17569/17569), done. \r14:58:02.863151 run-command.c:654 trace: run_command: git --shallow-file /tmp/sp-mobile-NewDragonCity/.git/shallow.lock index-pack --stdin -v --fix-thin '--keep=fetch-pack 73162 on SPTM-RVALVERDE' --pack_header=2,24884\r14:58:02.872236 git.c:458 trace: built-in: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 73162 on SPTM-RVALVERDE' --pack_header=2,24884\rReceiving objects: 0% (1/24884)\r...\rremote: Total 24884 (delta 10974), reused 18000 (delta 6889), pack-reused 0 \rReceiving objects: 100% (24884/24884), 108.69 MiB | 35.71 MiB/s\rReceiving objects: 100% (24884/24884), 108.73 MiB | 35.66 MiB/s, done.\rResolving deltas: 0% (0/10974)\r...\rResolving deltas: 100% (10974/10974), completed with 130 local objects.\r14:58:07.091848 run-command.c:654 trace: run_command: git --shallow-file /tmp/sp-mobile-NewDragonCity/.git/shallow.lock rev-list --objects --stdin --quiet --alternate-refs\r14:58:07.099855 git.c:458 trace: built-in: git rev-list --objects --stdin --quiet --alternate-refs\rFrom file:///Users/ricardo/GitProjects/sp-mobile-NewDragonCity\r = [up to date] origin/pr/10/head -> origin/pr/10/head\r = [up to date] origin/pr/100/head -> origin/pr/100/head\r = [up to date] origin/pr/1000/head -> origin/pr/1000/head\r = [up to date] origin/pr/1001/head -> origin/pr/1001/head\r = [up to date] origin/pr/1002/head -> origin/pr/1002/head\r = [up to date] origin/pr/1003/head -> origin/pr/1003/head\r = [up to date] origin/pr/1004/head -> origin/pr/1004/head\r = [up to date] origin/pr/1005/head -> origin/pr/1005/head\r = [up to date] origin/pr/1006/head -> origin/pr/1006/head\r = [up to date] origin/pr/1007/head -> origin/pr/1007/head\r = [up to date] origin/pr/1008/head -> origin/pr/1008/head\r = [up to date] origin/pr/1009/head -> origin/pr/1009/head\r = [up to date] origin/pr/101/head -> origin/pr/101/head\r = [up to date] origin/pr/1010/head -> origin/pr/1010/head\r = [up to date] origin/pr/1011/head -> origin/pr/1011/head\r = [up to date] origin/pr/1012/head -> origin/pr/1012/head\r = [up to date] origin/pr/1013/head -> origin/pr/1013/head\r = [up to date] origin/pr/1014/head -> origin/pr/1014/head\r = [up to date] origin/pr/1015/head -> origin/pr/1015/head\r = [up to date] origin/pr/1016/head -> origin/pr/1016/head\r = [up to date] origin/pr/1017/head -> origin/pr/1017/head\r = [up to date] origin/pr/1018/head -> origin/pr/1018/head\r = [up to date] origin/pr/1019/head -> origin/pr/1019/head\r = [up to date] origin/pr/102/head -> origin/pr/102/head\r = [up to date] origin/pr/1020/head -> origin/pr/1020/head\r = [up to date] origin/pr/1021/head -> origin/pr/1021/head\r = [up to date] origin/pr/1022/head -> origin/pr/1022/head\r = [up to date] origin/pr/1023/head -> origin/pr/1023/head\r = [up to date] origin/pr/1024/head -> origin/pr/1024/head\r = [up to date] origin/pr/1025/head -> origin/pr/1025/head\r = [up to date] origin/pr/1026/head -> origin/pr/1026/head\r = [up to date] origin/pr/1027/head -> origin/pr/1027/head\r = [up to date] origin/pr/1027/merge -> origin/pr/1027/merge\r = [up to date] origin/pr/1028/head -> origin/pr/1028/head\r = [up to date] origin/pr/1029/head -> origin/pr/1029/head\r = [up to date] origin/pr/103/head -> origin/pr/103/head\r = [up to date] origin/pr/1030/head -> origin/pr/1030/head\r = [up to date] origin/pr/1031/head -> origin/pr/1031/head\r = [up to date] origin/pr/1032/head -> origin/pr/1032/head\r = [up to date] origin/pr/1033/head -> origin/pr/1033/head\r = [up to date] origin/pr/1034/head -> origin/pr/1034/head\r = [up to date] origin/pr/1035/head -> origin/pr/1035/head\r = [up to date] origin/pr/1036/head -> origin/pr/1036/head\r = [up to date] origin/pr/1037/head -> origin/pr/1037/head\r = [up to date] origin/pr/1038/head -> origin/pr/1038/head\r = [up to date] origin/pr/1039/head -> origin/pr/1039/head\r = [up to date] origin/pr/104/head -> origin/pr/104/head\r = [up to date] origin/pr/104/merge -> origin/pr/104/merge\r = [up to date] origin/pr/1040/head -> origin/pr/1040/head\r = [up to date] origin/pr/1040/merge -> origin/pr/1040/merge\r = [up to date] origin/pr/1041/head -> origin/pr/1041/head\r = [up to date] origin/pr/1041/merge -> origin/pr/1041/merge\r = [up to date] origin/pr/1042/head -> origin/pr/1042/head\r = [up to date] origin/pr/1042/merge -> origin/pr/1042/merge\r = [up to date] origin/pr/1043/head -> origin/pr/1043/head\r = [up to date] origin/pr/1044/head -> origin/pr/1044/head\r = [up to date] origin/pr/1045/head -> origin/pr/1045/head\r = [up to date] origin/pr/1046/head -> origin/pr/1046/head\r = [up to date] origin/pr/1047/head -> origin/pr/1047/head\r = [up to date] origin/pr/1048/head -> origin/pr/1048/head\r = [up to date] origin/pr/1049/head -> origin/pr/1049/head\r = [up to date] origin/pr/105/head -> origin/pr/105/head\r = [up to date] origin/pr/1050/head -> origin/pr/1050/head\r = [up to date] origin/pr/1051/head -> origin/pr/1051/head\r = [up to date] origin/pr/1052/head -> origin/pr/1052/head\r = [up to date] origin/pr/1053/head -> origin/pr/1053/head\r = [up to date] origin/pr/1054/head -> origin/pr/1054/head\r = [up to date] origin/pr/1055/head -> origin/pr/1055/head\r = [up to date] origin/pr/1056/head -> origin/pr/1056/head\r = [up to date] origin/pr/1057/head -> origin/pr/1057/head\r = [up to date] origin/pr/1058/head -> origin/pr/1058/head\r = [up to date] origin/pr/1059/head -> origin/pr/1059/head\r = [up to date] origin/pr/106/head -> origin/pr/106/head\r = [up to date] origin/pr/1060/head -> origin/pr/1060/head\r = [up to date] origin/pr/1061/head -> origin/pr/1061/head\r = [up to date] origin/pr/1062/head -> origin/pr/1062/head\r = [up to date] origin/pr/1063/head -> origin/pr/1063/head\r = [up to date] origin/pr/1063/merge -> origin/pr/1063/merge\r = [up to date] origin/pr/1064/head -> origin/pr/1064/head\r = [up to date] origin/pr/1065/head -> origin/pr/1065/head\r = [up to date] origin/pr/1066/head -> origin/pr/1066/head\r = [up to date] origin/pr/1067/head -> origin/pr/1067/head\r = [up to date] origin/pr/1068/head -> origin/pr/1068/head\r = [up to date] origin/pr/1069/head -> origin/pr/1069/head\r = [up to date] origin/pr/107/head -> origin/pr/107/head\r = [up to date] origin/pr/1070/head -> origin/pr/1070/head\r = [up to date] origin/pr/1071/head -> origin/pr/1071/head\r = [up to date] origin/pr/1071/merge -> origin/pr/1071/merge\r = [up to date] origin/pr/1072/head -> origin/pr/1072/head\r = [up to date] origin/pr/1072/merge -> origin/pr/1072/merge\r = [up to date] origin/pr/1073/head -> origin/pr/1073/head\r = [up to date] origin/pr/1074/head -> origin/pr/1074/head\r = [up to date] origin/pr/1075/head -> origin/pr/1075/head\r = [up to date] origin/pr/1076/head -> origin/pr/1076/head\r = [up to date] origin/pr/1077/head -> origin/pr/1077/head\r = [up to date] origin/pr/1078/head -> origin/pr/1078/head\r = [up to date] origin/pr/1079/head -> origin/pr/1079/head\r = [up to date] origin/pr/108/head -> origin/pr/108/head\r = [up to date] origin/pr/1080/head -> origin/pr/1080/head\r = [up to date] origin/pr/1081/head -> origin/pr/1081/head\r = [up to date] origin/pr/1082/head -> origin/pr/1082/head\r = [up to date] origin/pr/1083/head -> origin/pr/1083/head\r = [up to date] origin/pr/1084/head -> origin/pr/1084/head\r = [up to date] origin/pr/1085/head -> origin/pr/1085/head\r = [up to date] origin/pr/1086/head -> origin/pr/1086/head\r = [up to date] origin/pr/1087/head -> origin/pr/1087/head\r = [up to date] origin/pr/1088/head -> origin/pr/1088/head\r = [up to date] origin/pr/1089/head -> origin/pr/1089/head\r = [up to date] origin/pr/109/head -> origin/pr/109/head\r = [up to date] origin/pr/1090/head -> origin/pr/1090/head\r = [up to date] origin/pr/1091/head -> origin/pr/1091/head\r = [up to date] origin/pr/1092/head -> origin/pr/1092/head\r = [up to date] origin/pr/1093/head -> origin/pr/1093/head\r = [up to date] origin/pr/1094/head -> origin/pr/1094/head\r = [up to date] origin/pr/1095/head -> origin/pr/1095/head\r = [up to date] origin/pr/1096/head -> origin/pr/1096/head\r = [up to date] origin/pr/1097/head -> origin/pr/1097/head\r = [up to date] origin/pr/1098/head -> origin/pr/1098/head\r = [up to date] origin/pr/1099/head -> origin/pr/1099/head\r14:58:07.162347 run-command.c:1582 run_processes_parallel: preparing to run up to 1 tasks\r14:58:07.162523 run-command.c:1614 run_processes_parallel: done\r14:58:07.162544 run-command.c:654 trace: run_command: git maintenance run --auto --no-quiet\r14:58:07.170903 git.c:458 trace: built-in: git maintenance run --auto --no-quiet\r
[-- Attachment #5: fetch_local_without_depth.log --]
[-- Type: application/octet-stream, Size: 9511 bytes --]
14:57:11.834253 git.c:458 trace: built-in: git fetch --verbose --progress --prune -- file:///Users/ricardo/GitProjects/sp-mobile-NewDragonCity/ '+refs/remotes/origin/pr/10*:refs/remotes/origin/pr/10*'
14:57:11.836665 run-command.c:654 trace: run_command: unset GIT_PREFIX; GIT_PROTOCOL=version=2 'git-upload-pack '\''/Users/ricardo/GitProjects/sp-mobile-NewDragonCity/'\'''
14:57:11.846700 git.c:458 trace: built-in: git upload-pack /Users/ricardo/GitProjects/sp-mobile-NewDragonCity/
14:57:16.609838 run-command.c:654 trace: run_command: git rev-list --objects --stdin --not --all --quiet --alternate-refs
From file:///Users/ricardo/GitProjects/sp-mobile-NewDragonCity
= [up to date] origin/pr/10/head -> origin/pr/10/head
= [up to date] origin/pr/100/head -> origin/pr/100/head
= [up to date] origin/pr/1000/head -> origin/pr/1000/head
= [up to date] origin/pr/1001/head -> origin/pr/1001/head
= [up to date] origin/pr/1002/head -> origin/pr/1002/head
= [up to date] origin/pr/1003/head -> origin/pr/1003/head
= [up to date] origin/pr/1004/head -> origin/pr/1004/head
= [up to date] origin/pr/1005/head -> origin/pr/1005/head
= [up to date] origin/pr/1006/head -> origin/pr/1006/head
= [up to date] origin/pr/1007/head -> origin/pr/1007/head
= [up to date] origin/pr/1008/head -> origin/pr/1008/head
= [up to date] origin/pr/1009/head -> origin/pr/1009/head
= [up to date] origin/pr/101/head -> origin/pr/101/head
= [up to date] origin/pr/1010/head -> origin/pr/1010/head
= [up to date] origin/pr/1011/head -> origin/pr/1011/head
= [up to date] origin/pr/1012/head -> origin/pr/1012/head
= [up to date] origin/pr/1013/head -> origin/pr/1013/head
= [up to date] origin/pr/1014/head -> origin/pr/1014/head
= [up to date] origin/pr/1015/head -> origin/pr/1015/head
= [up to date] origin/pr/1016/head -> origin/pr/1016/head
= [up to date] origin/pr/1017/head -> origin/pr/1017/head
= [up to date] origin/pr/1018/head -> origin/pr/1018/head
= [up to date] origin/pr/1019/head -> origin/pr/1019/head
= [up to date] origin/pr/102/head -> origin/pr/102/head
= [up to date] origin/pr/1020/head -> origin/pr/1020/head
= [up to date] origin/pr/1021/head -> origin/pr/1021/head
= [up to date] origin/pr/1022/head -> origin/pr/1022/head
= [up to date] origin/pr/1023/head -> origin/pr/1023/head
= [up to date] origin/pr/1024/head -> origin/pr/1024/head
= [up to date] origin/pr/1025/head -> origin/pr/1025/head
= [up to date] origin/pr/1026/head -> origin/pr/1026/head
= [up to date] origin/pr/1027/head -> origin/pr/1027/head
= [up to date] origin/pr/1027/merge -> origin/pr/1027/merge
= [up to date] origin/pr/1028/head -> origin/pr/1028/head
= [up to date] origin/pr/1029/head -> origin/pr/1029/head
= [up to date] origin/pr/103/head -> origin/pr/103/head
= [up to date] origin/pr/1030/head -> origin/pr/1030/head
= [up to date] origin/pr/1031/head -> origin/pr/1031/head
= [up to date] origin/pr/1032/head -> origin/pr/1032/head
= [up to date] origin/pr/1033/head -> origin/pr/1033/head
= [up to date] origin/pr/1034/head -> origin/pr/1034/head
= [up to date] origin/pr/1035/head -> origin/pr/1035/head
= [up to date] origin/pr/1036/head -> origin/pr/1036/head
= [up to date] origin/pr/1037/head -> origin/pr/1037/head
= [up to date] origin/pr/1038/head -> origin/pr/1038/head
= [up to date] origin/pr/1039/head -> origin/pr/1039/head
= [up to date] origin/pr/104/head -> origin/pr/104/head
= [up to date] origin/pr/104/merge -> origin/pr/104/merge
= [up to date] origin/pr/1040/head -> origin/pr/1040/head
= [up to date] origin/pr/1040/merge -> origin/pr/1040/merge
= [up to date] origin/pr/1041/head -> origin/pr/1041/head
= [up to date] origin/pr/1041/merge -> origin/pr/1041/merge
= [up to date] origin/pr/1042/head -> origin/pr/1042/head
= [up to date] origin/pr/1042/merge -> origin/pr/1042/merge
= [up to date] origin/pr/1043/head -> origin/pr/1043/head
= [up to date] origin/pr/1044/head -> origin/pr/1044/head
= [up to date] origin/pr/1045/head -> origin/pr/1045/head
= [up to date] origin/pr/1046/head -> origin/pr/1046/head
= [up to date] origin/pr/1047/head -> origin/pr/1047/head
= [up to date] origin/pr/1048/head -> origin/pr/1048/head
= [up to date] origin/pr/1049/head -> origin/pr/1049/head
= [up to date] origin/pr/105/head -> origin/pr/105/head
= [up to date] origin/pr/1050/head -> origin/pr/1050/head
= [up to date] origin/pr/1051/head -> origin/pr/1051/head
= [up to date] origin/pr/1052/head -> origin/pr/1052/head
= [up to date] origin/pr/1053/head -> origin/pr/1053/head
= [up to date] origin/pr/1054/head -> origin/pr/1054/head
= [up to date] origin/pr/1055/head -> origin/pr/1055/head
= [up to date] origin/pr/1056/head -> origin/pr/1056/head
= [up to date] origin/pr/1057/head -> origin/pr/1057/head
= [up to date] origin/pr/1058/head -> origin/pr/1058/head
= [up to date] origin/pr/1059/head -> origin/pr/1059/head
= [up to date] origin/pr/106/head -> origin/pr/106/head
= [up to date] origin/pr/1060/head -> origin/pr/1060/head
= [up to date] origin/pr/1061/head -> origin/pr/1061/head
= [up to date] origin/pr/1062/head -> origin/pr/1062/head
= [up to date] origin/pr/1063/head -> origin/pr/1063/head
= [up to date] origin/pr/1063/merge -> origin/pr/1063/merge
= [up to date] origin/pr/1064/head -> origin/pr/1064/head
= [up to date] origin/pr/1065/head -> origin/pr/1065/head
= [up to date] origin/pr/1066/head -> origin/pr/1066/head
= [up to date] origin/pr/1067/head -> origin/pr/1067/head
= [up to date] origin/pr/1068/head -> origin/pr/1068/head
= [up to date] origin/pr/1069/head -> origin/pr/1069/head
= [up to date] origin/pr/107/head -> origin/pr/107/head
= [up to date] origin/pr/1070/head -> origin/pr/1070/head
= [up to date] origin/pr/1071/head -> origin/pr/1071/head
= [up to date] origin/pr/1071/merge -> origin/pr/1071/merge
= [up to date] origin/pr/1072/head -> origin/pr/1072/head
= [up to date] origin/pr/1072/merge -> origin/pr/1072/merge
= [up to date] origin/pr/1073/head -> origin/pr/1073/head
= [up to date] origin/pr/1074/head -> origin/pr/1074/head
= [up to date] origin/pr/1075/head -> origin/pr/1075/head
= [up to date] origin/pr/1076/head -> origin/pr/1076/head
= [up to date] origin/pr/1077/head -> origin/pr/1077/head
= [up to date] origin/pr/1078/head -> origin/pr/1078/head
= [up to date] origin/pr/1079/head -> origin/pr/1079/head
= [up to date] origin/pr/108/head -> origin/pr/108/head
= [up to date] origin/pr/1080/head -> origin/pr/1080/head
= [up to date] origin/pr/1081/head -> origin/pr/1081/head
= [up to date] origin/pr/1082/head -> origin/pr/1082/head
= [up to date] origin/pr/1083/head -> origin/pr/1083/head
= [up to date] origin/pr/1084/head -> origin/pr/1084/head
= [up to date] origin/pr/1085/head -> origin/pr/1085/head
= [up to date] origin/pr/1086/head -> origin/pr/1086/head
= [up to date] origin/pr/1087/head -> origin/pr/1087/head
= [up to date] origin/pr/1088/head -> origin/pr/1088/head
= [up to date] origin/pr/1089/head -> origin/pr/1089/head
= [up to date] origin/pr/109/head -> origin/pr/109/head
= [up to date] origin/pr/1090/head -> origin/pr/1090/head
= [up to date] origin/pr/1091/head -> origin/pr/1091/head
= [up to date] origin/pr/1092/head -> origin/pr/1092/head
= [up to date] origin/pr/1093/head -> origin/pr/1093/head
= [up to date] origin/pr/1094/head -> origin/pr/1094/head
= [up to date] origin/pr/1095/head -> origin/pr/1095/head
= [up to date] origin/pr/1096/head -> origin/pr/1096/head
= [up to date] origin/pr/1097/head -> origin/pr/1097/head
= [up to date] origin/pr/1098/head -> origin/pr/1098/head
= [up to date] origin/pr/1099/head -> origin/pr/1099/head
14:57:21.849492 run-command.c:1582 run_processes_parallel: preparing to run up to 1 tasks
14:57:21.849727 run-command.c:1614 run_processes_parallel: done
14:57:21.849775 run-command.c:654 trace: run_command: git maintenance run --auto --no-quiet
14:57:21.857975 git.c:458 trace: built-in: git maintenance run --auto --no-quiet
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: Noticed big performance issue when fetching multiple refs with depth 1 enabled
2022-03-18 14:44 Noticed big performance issue when fetching multiple refs with depth 1 enabled Ricard Valverde
@ 2022-03-18 20:21 ` brian m. carlson
0 siblings, 0 replies; 2+ messages in thread
From: brian m. carlson @ 2022-03-18 20:21 UTC (permalink / raw)
To: Ricard Valverde; +Cc: git
[-- Attachment #1: Type: text/plain, Size: 2018 bytes --]
On 2022-03-18 at 14:44:53, Ricard Valverde wrote:
> Thank you for filling out a Git bug report!
> Please answer the following questions to help us understand your issue.
>
> What did you do before the bug happened? (Steps to reproduce your issue)
> Do a git fetch --depth=1 , with no remote changes, having a lot of
> remote branches makes the issue more visible
> What did you expect to happen? (Expected behavior)
> Shallow fetch should be faster or as fast as a full fetch
> What happened instead? (Actual behavior)
> Shallow fetch is about 10x slower than full fetch, consistently
> What's different between what you expected and what actually happened?
> I expected the shallow fetch to be faster than a normal fetch, or as
> fast. Did not expect it being 10x slower.
> Anything else you want to add:
> Tested in a repository with ~10000 remote branches, a single Github
> hosted remote.
> Also tested with a local remote, and Git 2.35.1 version, with
> comparable results.
The reason you're seeing this is that fetching into an existing shallow
remote is extremely expensive on the server side. With a normal fetch,
we know that the user only needs objects which exist between the old and
new heads, and that they have every object which is reachable from the
old heads.
However, with a shallow fetch, we can't assume that, since by definition
the client side doesn't have all of those objects. A substantial amount
more work must be done to determine what the client already has. This
is made worse by the fact that you have 10,000 branches.
If you're working in a CI system or such, you should use a fresh shallow
clone each time, which will be much faster and easier on the server.
Otherwise, you may find that reducing the number of refs significantly
can also help performance. If you can guarantee that you'll be online
when working with this project, a partial clone may also meet your
needs.
--
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2022-03-18 20:22 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-18 14:44 Noticed big performance issue when fetching multiple refs with depth 1 enabled Ricard Valverde
2022-03-18 20:21 ` brian m. carlson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).