* Fetching 24 Linux commits = 1.2 GiB @ 2020-04-15 8:01 Jiri Slaby 2020-04-15 8:11 ` Andreas Schwab ` (2 more replies) 0 siblings, 3 replies; 20+ messages in thread From: Jiri Slaby @ 2020-04-15 8:01 UTC (permalink / raw) To: git Hi, I was at 8f3d9f354286 of: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git I did git remote update today and it fetched: Receiving objects: 100% (7330823/7330823), 1.20 GiB It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits. One colleague of mine fetched 1324 commits: Receiving objects: 100% (6820/6820), 4.21 MiB | 6.70 MiB/s, done. Resolving deltas: 100% (5114/5114), completed with 1035 local objects. From git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux 7e63420847ae..8632e9b5645b master -> origin/master Another colleague fetched the same what I and: Receiving objects: 100% (7330823/7330823), 1.20 GiB too. I did git gc --prune && git prune now and I am at 1.7G back from 3.5 G. Is that a bug? What info should I provide? thanks, -- js suse labs ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-04-15 8:01 Fetching 24 Linux commits = 1.2 GiB Jiri Slaby @ 2020-04-15 8:11 ` Andreas Schwab 2020-04-15 8:16 ` Jiri Slaby 2020-04-15 10:52 ` Kevin Daudt 2020-04-15 13:56 ` Konstantin Ryabitsev 2 siblings, 1 reply; 20+ messages in thread From: Andreas Schwab @ 2020-04-15 8:11 UTC (permalink / raw) To: Jiri Slaby; +Cc: git On Apr 15 2020, Jiri Slaby wrote: > I was at 8f3d9f354286 of: > git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > > I did git remote update today and it fetched: > Receiving objects: 100% (7330823/7330823), 1.20 GiB > It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits. What's your git version? I just did exactly the same update with git 2.26.1, and it only fetched 144 objects: Receiving objects: 100% (144/144), 50.50 KiB | 1.87 MiB/s, done. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-04-15 8:11 ` Andreas Schwab @ 2020-04-15 8:16 ` Jiri Slaby 2020-04-15 8:23 ` Andreas Schwab 0 siblings, 1 reply; 20+ messages in thread From: Jiri Slaby @ 2020-04-15 8:16 UTC (permalink / raw) To: Andreas Schwab; +Cc: git On 15. 04. 20, 10:11, Andreas Schwab wrote: > On Apr 15 2020, Jiri Slaby wrote: > >> I was at 8f3d9f354286 of: >> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git >> >> I did git remote update today and it fetched: >> Receiving objects: 100% (7330823/7330823), 1.20 GiB >> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits. > > What's your git version? I just did exactly the same update with git > 2.26.1, and it only fetched 144 objects: > > Receiving objects: 100% (144/144), 50.50 KiB | 1.87 MiB/s, done. $ git --version git version 2.26.1 The same as you have. -- js suse labs ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-04-15 8:16 ` Jiri Slaby @ 2020-04-15 8:23 ` Andreas Schwab 2020-04-15 8:27 ` Jiri Slaby 0 siblings, 1 reply; 20+ messages in thread From: Andreas Schwab @ 2020-04-15 8:23 UTC (permalink / raw) To: Jiri Slaby; +Cc: git On Apr 15 2020, Jiri Slaby wrote: > On 15. 04. 20, 10:11, Andreas Schwab wrote: >> On Apr 15 2020, Jiri Slaby wrote: >> >>> I was at 8f3d9f354286 of: >>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git >>> >>> I did git remote update today and it fetched: >>> Receiving objects: 100% (7330823/7330823), 1.20 GiB >>> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits. >> >> What's your git version? I just did exactly the same update with git >> 2.26.1, and it only fetched 144 objects: >> >> Receiving objects: 100% (144/144), 50.50 KiB | 1.87 MiB/s, done. > > $ git --version > git version 2.26.1 Was this the first time you used git >= 2.26.0? Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-04-15 8:23 ` Andreas Schwab @ 2020-04-15 8:27 ` Jiri Slaby 2020-04-15 8:34 ` Andreas Schwab 0 siblings, 1 reply; 20+ messages in thread From: Jiri Slaby @ 2020-04-15 8:27 UTC (permalink / raw) To: Andreas Schwab; +Cc: git On 15. 04. 20, 10:23, Andreas Schwab wrote: > On Apr 15 2020, Jiri Slaby wrote: > >> On 15. 04. 20, 10:11, Andreas Schwab wrote: >>> On Apr 15 2020, Jiri Slaby wrote: >>> >>>> I was at 8f3d9f354286 of: >>>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git >>>> >>>> I did git remote update today and it fetched: >>>> Receiving objects: 100% (7330823/7330823), 1.20 GiB >>>> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits. >>> >>> What's your git version? I just did exactly the same update with git >>> 2.26.1, and it only fetched 144 objects: >>> >>> Receiving objects: 100% (144/144), 50.50 KiB | 1.87 MiB/s, done. >> >> $ git --version >> git version 2.26.1 > > Was this the first time you used git >= 2.26.0? I doubt that: $ grep git-core /var/log/zypp/history|sed 's@x86_64.*@@' 2020-03-23 16:14:24|install|git-core|2.25.2-467.2| 2020-03-24 17:19:37|install|git-core|2.26.0-468.1| 2020-03-29 09:38:34|install|git-core|2.26.0-471.1| 2020-04-06 06:34:36|install|git-core|2.26.0-473.1| 2020-04-15 08:21:37|install|git-core|2.26.1-474.1| This is the first time I used 2.26.1, though. thanks, -- js suse labs ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-04-15 8:27 ` Jiri Slaby @ 2020-04-15 8:34 ` Andreas Schwab 0 siblings, 0 replies; 20+ messages in thread From: Andreas Schwab @ 2020-04-15 8:34 UTC (permalink / raw) To: Jiri Slaby; +Cc: git On Apr 15 2020, Jiri Slaby wrote: > This is the first time I used 2.26.1, though. Same for me. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-04-15 8:01 Fetching 24 Linux commits = 1.2 GiB Jiri Slaby 2020-04-15 8:11 ` Andreas Schwab @ 2020-04-15 10:52 ` Kevin Daudt 2020-04-15 13:56 ` Konstantin Ryabitsev 2 siblings, 0 replies; 20+ messages in thread From: Kevin Daudt @ 2020-04-15 10:52 UTC (permalink / raw) To: Jiri Slaby; +Cc: git On Wed, Apr 15, 2020 at 10:01:46AM +0200, Jiri Slaby wrote: > Hi, > > I was at 8f3d9f354286 of: > git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > > I did git remote update today and it fetched: > Receiving objects: 100% (7330823/7330823), 1.20 GiB > It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits. > > One colleague of mine fetched 1324 commits: > Receiving objects: 100% (6820/6820), 4.21 MiB | 6.70 MiB/s, done. > Resolving deltas: 100% (5114/5114), completed with 1035 local objects. > From git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux > 7e63420847ae..8632e9b5645b master -> origin/master > > Another colleague fetched the same what I and: > Receiving objects: 100% (7330823/7330823), 1.20 GiB > too. > > I did git gc --prune && git prune now and I am at 1.7G back from 3.5 G. > > Is that a bug? What info should I provide? > > thanks, > -- > js > suse labs Not as big as you report, but I recall a user on IRC (guardian) was wondering as well why a packfile was 240MB while they claimed that the objects were committed months ago. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-04-15 8:01 Fetching 24 Linux commits = 1.2 GiB Jiri Slaby 2020-04-15 8:11 ` Andreas Schwab 2020-04-15 10:52 ` Kevin Daudt @ 2020-04-15 13:56 ` Konstantin Ryabitsev 2020-04-15 15:08 ` Junio C Hamano 2020-04-16 6:31 ` Jiri Slaby 2 siblings, 2 replies; 20+ messages in thread From: Konstantin Ryabitsev @ 2020-04-15 13:56 UTC (permalink / raw) To: Jiri Slaby; +Cc: git On Wed, Apr 15, 2020 at 10:01:46AM +0200, Jiri Slaby wrote: > Hi, > > I was at 8f3d9f354286 of: > git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > > I did git remote update today and it fetched: > Receiving objects: 100% (7330823/7330823), 1.20 GiB > It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits. > > One colleague of mine fetched 1324 commits: > Receiving objects: 100% (6820/6820), 4.21 MiB | 6.70 MiB/s, done. > Resolving deltas: 100% (5114/5114), completed with 1035 local objects. > From git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux > 7e63420847ae..8632e9b5645b master -> origin/master > > Another colleague fetched the same what I and: > Receiving objects: 100% (7330823/7330823), 1.20 GiB > too. > > I did git gc --prune && git prune now and I am at 1.7G back from 3.5 G. > > Is that a bug? What info should I provide? I've helped sfr troubleshoot the same issue last week -- it's most likely due to 2.26 turning on protocol version=2 by default. Unfortunately, reproducing this has been tricky, so if you can reliably make this happen again, then providing a full copy of your local tree as well as the remote you're trying to fetch may greatly help narrow it down. With sfr (for whom fetching 1.2G from .au is a bit of a big deal), we solved it by forcing protocol.version=1. -K ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-04-15 13:56 ` Konstantin Ryabitsev @ 2020-04-15 15:08 ` Junio C Hamano 2020-04-15 15:16 ` Jeff King ` (2 more replies) 2020-04-16 6:31 ` Jiri Slaby 1 sibling, 3 replies; 20+ messages in thread From: Junio C Hamano @ 2020-04-15 15:08 UTC (permalink / raw) To: Jonathan Nieder, Jonathan Tan; +Cc: git Do these (and I think we saw other reports) make us rethink the status of protocol v2 as the default? Are all of these fallouts we saw so far easy-to-fix bugs, or are there more fundamental issues in the v2 protocol design? Thanks. Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes: > On Wed, Apr 15, 2020 at 10:01:46AM +0200, Jiri Slaby wrote: >> Hi, >> >> I was at 8f3d9f354286 of: >> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git >> >> I did git remote update today and it fetched: >> Receiving objects: 100% (7330823/7330823), 1.20 GiB >> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits. >> >> One colleague of mine fetched 1324 commits: >> Receiving objects: 100% (6820/6820), 4.21 MiB | 6.70 MiB/s, done. >> Resolving deltas: 100% (5114/5114), completed with 1035 local objects. >> From git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux >> 7e63420847ae..8632e9b5645b master -> origin/master >> >> Another colleague fetched the same what I and: >> Receiving objects: 100% (7330823/7330823), 1.20 GiB >> too. >> >> I did git gc --prune && git prune now and I am at 1.7G back from 3.5 G. >> >> Is that a bug? What info should I provide? > > I've helped sfr troubleshoot the same issue last week -- it's most > likely due to 2.26 turning on protocol version=2 by default. > Unfortunately, reproducing this has been tricky, so if you can reliably > make this happen again, then providing a full copy of your local tree as > well as the remote you're trying to fetch may greatly help narrow it > down. > > With sfr (for whom fetching 1.2G from .au is a bit of a big deal), we > solved it by forcing protocol.version=1. > > -K ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-04-15 15:08 ` Junio C Hamano @ 2020-04-15 15:16 ` Jeff King 2020-04-15 18:52 ` Jonathan Nieder 2020-05-20 8:53 ` Andreas Schwab 2 siblings, 0 replies; 20+ messages in thread From: Jeff King @ 2020-04-15 15:16 UTC (permalink / raw) To: Junio C Hamano; +Cc: Jonathan Nieder, Jonathan Tan, git On Wed, Apr 15, 2020 at 08:08:17AM -0700, Junio C Hamano wrote: > Do these (and I think we saw other reports) make us rethink the > status of protocol v2 as the default? Are all of these fallouts > we saw so far easy-to-fix bugs, or are there more fundamental issues > in the v2 protocol design? I don't think we know yet. I agree with Konstantin that the v2 switch is the likely culprit for these issues, but without having been able to reproduce, I don't think we know exactly what the problem is yet. It could be a protocol design issue, or it could be a minor implementation bug. Note that there is one other issue that's turned up, that I discussed here: https://lore.kernel.org/git/20200328154936.GA1217052@coredump.intra.peff.net/ That's more fundamental to the v2 design, but: - it only happens when one side drops the connection, so it's not impacting normal operation (it does turn an error case into a hang, though, which can be rather annoying) - it's not in the network protocol itself, but rather the protocol between Git and the remote helper. So we could solve it purely as a client-side fix. -Peff ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-04-15 15:08 ` Junio C Hamano 2020-04-15 15:16 ` Jeff King @ 2020-04-15 18:52 ` Jonathan Nieder 2020-05-20 8:53 ` Andreas Schwab 2 siblings, 0 replies; 20+ messages in thread From: Jonathan Nieder @ 2020-04-15 18:52 UTC (permalink / raw) To: Junio C Hamano; +Cc: Jonathan Tan, git, Jeff King, Konstantin Ryabitsev Hi, >> On Wed, Apr 15, 2020 at 10:01:46AM +0200, Jiri Slaby wrote: >>> I was at 8f3d9f354286 of: >>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git >>> >>> I did git remote update today and it fetched: >>> Receiving objects: 100% (7330823/7330823), 1.20 GiB >>> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits. I suspect this is due to negotiation differing in protocol v2: in protocols that do not maintain server side state, we have to record previously matched "have"s at each round and the number of additional "have"s sent on top leads the server to have insufficient information about what the client has. In other words, I suspect that with https:// in protocol v0 you would experience the same thing. Does git config --global fetch.negotiationStrategy skipping help? Thanks, Jonathan ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-04-15 15:08 ` Junio C Hamano 2020-04-15 15:16 ` Jeff King 2020-04-15 18:52 ` Jonathan Nieder @ 2020-05-20 8:53 ` Andreas Schwab 2020-05-20 8:57 ` Andreas Schwab ` (2 more replies) 2 siblings, 3 replies; 20+ messages in thread From: Andreas Schwab @ 2020-05-20 8:53 UTC (permalink / raw) To: Junio C Hamano; +Cc: Jonathan Nieder, Jonathan Tan, git On Apr 15 2020, Junio C Hamano wrote: > Do these (and I think we saw other reports) make us rethink the > status of protocol v2 as the default? Are all of these fallouts > we saw so far easy-to-fix bugs, or are there more fundamental issues > in the v2 protocol design? I'm now seeing the issue myself, and can provide a backup of the offending repository. $ git count-objects -v count: 17 size: 76 in-pack: 387240 packs: 37 size-pack: 203738 prune-packable: 0 garbage: 0 size-garbage: 0 alternate: /home/andreas/src/linux/git/torvalds/linux.git/objects alternate: /home/andreas/src/linux/git/stable/linux-stable.git/objects $ GIT_TRACE=1 git fetch 10:40:32.829450 git.c:439 trace: built-in: git fetch 10:40:33.133448 run-command.c:663 trace: run_command: unset GIT_DIR GIT_IMPLICIT_WORK_TREE GIT_PREFIX; git --git-dir=/daten/src/linux/git/torvalds/linux.git for-each-ref '--format=%(objectname)' 10:40:33.135756 git.c:439 trace: built-in: git for-each-ref '--format=%(objectname)' 10:40:33.143936 run-command.c:663 trace: run_command: unset GIT_DIR GIT_IMPLICIT_WORK_TREE GIT_PREFIX; git --git-dir=/daten/src/linux/git/stable/linux-stable.git for-each-ref '--format=%(objectname)' 10:40:33.146087 git.c:439 trace: built-in: git for-each-ref '--format=%(objectname)' remote: Enumerating objects: 30796, done. remote: Counting objects: 100% (30796/30796), done. remote: Compressing objects: 100% (6965/6965), done. 10:40:40.102198 run-command.c:663 trace: run_command: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 12342 on igel.home' --pack_header=2,7350969 10:40:40.104872 git.c:439 trace: built-in: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 12342 on igel.home' --pack_header=2,7350969 ^Cceiving objects: 0% (3092/7350969), 1.32 MiB | 891.00 KiB/s Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-05-20 8:53 ` Andreas Schwab @ 2020-05-20 8:57 ` Andreas Schwab 2020-05-20 10:05 ` Michal Suchánek 2020-05-20 19:40 ` Jeff King 2 siblings, 0 replies; 20+ messages in thread From: Andreas Schwab @ 2020-05-20 8:57 UTC (permalink / raw) To: Junio C Hamano; +Cc: Jonathan Nieder, Jonathan Tan, git On Mai 20 2020, Andreas Schwab wrote: > On Apr 15 2020, Junio C Hamano wrote: > >> Do these (and I think we saw other reports) make us rethink the >> status of protocol v2 as the default? Are all of these fallouts >> we saw so far easy-to-fix bugs, or are there more fundamental issues >> in the v2 protocol design? > > I'm now seeing the issue myself, and can provide a backup of the > offending repository. > > $ git count-objects -v > count: 17 > size: 76 > in-pack: 387240 > packs: 37 > size-pack: 203738 > prune-packable: 0 > garbage: 0 > size-garbage: 0 > alternate: /home/andreas/src/linux/git/torvalds/linux.git/objects > alternate: /home/andreas/src/linux/git/stable/linux-stable.git/objects > $ GIT_TRACE=1 git fetch > 10:40:32.829450 git.c:439 trace: built-in: git fetch > 10:40:33.133448 run-command.c:663 trace: run_command: unset GIT_DIR GIT_IMPLICIT_WORK_TREE GIT_PREFIX; git --git-dir=/daten/src/linux/git/torvalds/linux.git for-each-ref '--format=%(objectname)' > 10:40:33.135756 git.c:439 trace: built-in: git for-each-ref '--format=%(objectname)' > 10:40:33.143936 run-command.c:663 trace: run_command: unset GIT_DIR GIT_IMPLICIT_WORK_TREE GIT_PREFIX; git --git-dir=/daten/src/linux/git/stable/linux-stable.git for-each-ref '--format=%(objectname)' > 10:40:33.146087 git.c:439 trace: built-in: git for-each-ref '--format=%(objectname)' > remote: Enumerating objects: 30796, done. > remote: Counting objects: 100% (30796/30796), done. > remote: Compressing objects: 100% (6965/6965), done. > 10:40:40.102198 run-command.c:663 trace: run_command: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 12342 on igel.home' --pack_header=2,7350969 > 10:40:40.104872 git.c:439 trace: built-in: git index-pack --stdin -v --fix-thin '--keep=fetch-pack 12342 on igel.home' --pack_header=2,7350969 > ^Cceiving objects: 0% (3092/7350969), 1.32 MiB | 891.00 KiB/s $ git -c protocol.version=1 fetch remote: Enumerating objects: 13226, done. remote: Counting objects: 100% (10881/10881), done. remote: Compressing objects: 100% (2322/2322), done. remote: Total 10276 (delta 7860), reused 10215 (delta 7818), pack-reused 0 Receiving objects: 100% (10276/10276), 4.13 MiB | 615.00 KiB/s, done. Resolving deltas: 100% (7860/7860), completed with 192 local objects. From git://git.kernel.org/pub/scm/linux/kernel/git/geert/linux-m68k + c7630f6386a3...4684d38e4d47 m68k-queue -> m68k-queue (forced update) 39d91d3c3b47..e886fc082483 master -> master Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-05-20 8:53 ` Andreas Schwab 2020-05-20 8:57 ` Andreas Schwab @ 2020-05-20 10:05 ` Michal Suchánek 2020-05-20 10:34 ` Andreas Schwab 2020-05-20 19:40 ` Jeff King 2 siblings, 1 reply; 20+ messages in thread From: Michal Suchánek @ 2020-05-20 10:05 UTC (permalink / raw) To: Andreas Schwab; +Cc: Junio C Hamano, Jonathan Nieder, Jonathan Tan, git On Wed, May 20, 2020 at 10:53:36AM +0200, Andreas Schwab wrote: > On Apr 15 2020, Junio C Hamano wrote: > > > Do these (and I think we saw other reports) make us rethink the > > status of protocol v2 as the default? Are all of these fallouts > > we saw so far easy-to-fix bugs, or are there more fundamental issues > > in the v2 protocol design? > > I'm now seeing the issue myself, and can provide a backup of the > offending repository. What git version? Should be fixed in git 2.26.2 in SUSE and git 2.26.3 upstream. Thanks Michal ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-05-20 10:05 ` Michal Suchánek @ 2020-05-20 10:34 ` Andreas Schwab 0 siblings, 0 replies; 20+ messages in thread From: Andreas Schwab @ 2020-05-20 10:34 UTC (permalink / raw) To: Michal Suchánek; +Cc: Junio C Hamano, Jonathan Nieder, Jonathan Tan, git On Mai 20 2020, Michal Suchánek wrote: > What git version? 2.26.2 > git 2.26.3 I don't see that anywhere. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-05-20 8:53 ` Andreas Schwab 2020-05-20 8:57 ` Andreas Schwab 2020-05-20 10:05 ` Michal Suchánek @ 2020-05-20 19:40 ` Jeff King 2020-05-25 18:22 ` Lukas Wunner 2 siblings, 1 reply; 20+ messages in thread From: Jeff King @ 2020-05-20 19:40 UTC (permalink / raw) To: Andreas Schwab; +Cc: Junio C Hamano, Jonathan Nieder, Jonathan Tan, git On Wed, May 20, 2020 at 10:53:36AM +0200, Andreas Schwab wrote: > On Apr 15 2020, Junio C Hamano wrote: > > > Do these (and I think we saw other reports) make us rethink the > > status of protocol v2 as the default? Are all of these fallouts > > we saw so far easy-to-fix bugs, or are there more fundamental issues > > in the v2 protocol design? > > I'm now seeing the issue myself, and can provide a backup of the > offending repository. The "too big fetch" issue has since been fixed in "master", as well as reverting the switch to the v2 protocol (which I think is just belt-and-suspenders; AFAIK there are no known issues after the fix). Both will be in v2.27. I don't see anything on "maint", but they _could_ be part of an eventual v2.26.3. The fix was merged in 0b07eecf6e (Merge branch 'jt/v2-fetch-nego-fix', 2020-05-01) for reference. -Peff ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-05-20 19:40 ` Jeff King @ 2020-05-25 18:22 ` Lukas Wunner 2020-05-25 18:37 ` Junio C Hamano 0 siblings, 1 reply; 20+ messages in thread From: Lukas Wunner @ 2020-05-25 18:22 UTC (permalink / raw) To: Jeff King Cc: Andreas Schwab, Junio C Hamano, Jonathan Nieder, Jonathan Tan, git On Wed, May 20, 2020 at 03:40:19PM -0400, Jeff King wrote: > The "too big fetch" issue has since been fixed in "master", as well as > reverting the switch to the v2 protocol (which I think is just > belt-and-suspenders; AFAIK there are no known issues after the fix). > Both will be in v2.27. I don't see anything on "maint", but they _could_ > be part of an eventual v2.26.3. > > The fix was merged in 0b07eecf6e (Merge branch 'jt/v2-fetch-nego-fix', > 2020-05-01) for reference. Please consider cutting a v2.26.3 release with this fix at your earliest convenience. The waste of bandwidth is mind-boggling. (> 1 GByte whenever fetching from a kernel remote.) Thanks, Lukas ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-05-25 18:22 ` Lukas Wunner @ 2020-05-25 18:37 ` Junio C Hamano 0 siblings, 0 replies; 20+ messages in thread From: Junio C Hamano @ 2020-05-25 18:37 UTC (permalink / raw) To: Lukas Wunner Cc: Jeff King, Andreas Schwab, Jonathan Nieder, Jonathan Tan, git Lukas Wunner <lukas@wunner.de> writes: > On Wed, May 20, 2020 at 03:40:19PM -0400, Jeff King wrote: >> The "too big fetch" issue has since been fixed in "master", as well as >> reverting the switch to the v2 protocol (which I think is just >> belt-and-suspenders; AFAIK there are no known issues after the fix). >> Both will be in v2.27. I don't see anything on "maint", but they _could_ >> be part of an eventual v2.26.3. >> >> The fix was merged in 0b07eecf6e (Merge branch 'jt/v2-fetch-nego-fix', >> 2020-05-01) for reference. > > Please consider cutting a v2.26.3 release with this fix at your > earliest convenience. The waste of bandwidth is mind-boggling. > (> 1 GByte whenever fetching from a kernel remote.) In the meantime, v2.27.0 rc2 will be out tomorrow. Please consider helping us polish it by testing that version. Thanks. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-04-15 13:56 ` Konstantin Ryabitsev 2020-04-15 15:08 ` Junio C Hamano @ 2020-04-16 6:31 ` Jiri Slaby 2020-04-21 10:02 ` Martin Wilck 1 sibling, 1 reply; 20+ messages in thread From: Jiri Slaby @ 2020-04-16 6:31 UTC (permalink / raw) To: git On 15. 04. 20, 15:56, Konstantin Ryabitsev wrote: > On Wed, Apr 15, 2020 at 10:01:46AM +0200, Jiri Slaby wrote: >> Hi, >> >> I was at 8f3d9f354286 of: >> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git >> >> I did git remote update today and it fetched: >> Receiving objects: 100% (7330823/7330823), 1.20 GiB >> It updated master: 8f3d9f354286..8632e9b5645b, that is 24 small commits. >> >> One colleague of mine fetched 1324 commits: >> Receiving objects: 100% (6820/6820), 4.21 MiB | 6.70 MiB/s, done. >> Resolving deltas: 100% (5114/5114), completed with 1035 local objects. >> From git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux >> 7e63420847ae..8632e9b5645b master -> origin/master >> >> Another colleague fetched the same what I and: >> Receiving objects: 100% (7330823/7330823), 1.20 GiB >> too. >> >> I did git gc --prune && git prune now and I am at 1.7G back from 3.5 G. >> >> Is that a bug? What info should I provide? > > I've helped sfr troubleshoot the same issue last week -- it's most > likely due to 2.26 turning on protocol version=2 by default. > Unfortunately, reproducing this has been tricky, so if you can reliably > make this happen again, then providing a full copy of your local tree as > well as the remote you're trying to fetch may greatly help narrow it > down. I tried hard, but cannot reproduce. I noticed a difference between 2.25.1 and 2.25.1+protocol.version=2, though: $ git config protocol.version 1 # the default in 2.25 $ git fetch git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 8f3d9f354286745c751374f5f1fcafee6b3f3136 error: Server does not allow request for unadvertised object 8f3d9f354286745c751374f5f1fcafee6b3f3136 $ git config protocol.version 2 $ git fetch git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 8f3d9f354286745c751374f5f1fcafee6b3f3136 remote: Enumerating objects: 1433262, done. ... Doing fetch v5.7-rc1 (which is 8f3d9f3 above) with proto 1 works. So the server obviously advertises different set of objects with proto 1 and 2. thanks, -- js suse labs ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Fetching 24 Linux commits = 1.2 GiB 2020-04-16 6:31 ` Jiri Slaby @ 2020-04-21 10:02 ` Martin Wilck 0 siblings, 0 replies; 20+ messages in thread From: Martin Wilck @ 2020-04-21 10:02 UTC (permalink / raw) To: Jiri Slaby, git; +Cc: konstantin On Thu, 2020-04-16 at 08:31 +0200, Jiri Slaby wrote: > On 15. 04. 20, 15:56, Konstantin Ryabitsev wrote: > > > I tried hard, but cannot reproduce. I noticed a difference between > 2.25.1 and 2.25.1+protocol.version=2, though: > > $ git config protocol.version 1 # the default in 2.25 > $ git fetch > git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > 8f3d9f354286745c751374f5f1fcafee6b3f3136 > error: Server does not allow request for unadvertised object > 8f3d9f354286745c751374f5f1fcafee6b3f3136 > $ git config protocol.version 2 > $ git fetch > git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > 8f3d9f354286745c751374f5f1fcafee6b3f3136 > remote: Enumerating objects: 1433262, done. > ... > > Doing fetch v5.7-rc1 (which is 8f3d9f3 above) with proto 1 works. So > the > server obviously advertises different set of objects with proto 1 and > 2. I just had the same issue with several kernel repos. The affected repo is a bare clone of Linus' kernel repo, plus "stable" and various maintainer repos as additional remotes. Interestingly, a single "git fetch --dry-run" with protol version 1 "fixed" the issue. I hadn't expected that to happen, so unfortunately I hadn't made a backup of the repo before. As this is a 50GB bare repo, I'd appreciate instructions on how exactly to archive/upload it for debugging if this should happen again. mwilck@apollon:linux.git[BARE:master]> git fetch --dry-run -v net Looking up git.kernel.org ... done. Connecting to git.kernel.org (port 9418) ... 136.144.49.103 done. remote: Enumerating objects: 7331255, done. remote: Counting objects: 100% (7331255/7331255), done. remote: Compressing objects: 100% (1113751/1113751), done. ^C mwilck@apollon:linux.git[BARE:master]> git config --get protocol.version mwilck@apollon:linux.git[BARE:master]> git config protocol.version 1 mwilck@apollon:linux.git[BARE:master]> git fetch --dry-run -v net Looking up git.kernel.org ... done. Connecting to git.kernel.org (port 9418) ... 136.144.49.103 done. remote: Enumerating objects: 71, done. remote: Counting objects: 100% (71/71), done. remote: Compressing objects: 100% (48/48), done. remote: Total 53 (delta 42), reused 0 (delta 0) Unpacking objects: 100% (53/53), 8.06 KiB | 5.00 KiB/s, done. ^C (this fixed the issue despite "--dry-run" and myself interrupting the process). mwilck@apollon:linux.git[BARE:master]> git config protocol.version 2 mwilck@apollon:linux.git[BARE:master]> git fetch --dry-run -v net Looking up git.kernel.org ... done. Connecting to git.kernel.org (port 9418) ... 136.144.49.103 done. From git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 9bacd25..a460fc5 master -> net/master = [up to date] v2.6.11 -> v2.6.11 ... Regards Martin ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2020-05-25 18:37 UTC | newest] Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-04-15 8:01 Fetching 24 Linux commits = 1.2 GiB Jiri Slaby 2020-04-15 8:11 ` Andreas Schwab 2020-04-15 8:16 ` Jiri Slaby 2020-04-15 8:23 ` Andreas Schwab 2020-04-15 8:27 ` Jiri Slaby 2020-04-15 8:34 ` Andreas Schwab 2020-04-15 10:52 ` Kevin Daudt 2020-04-15 13:56 ` Konstantin Ryabitsev 2020-04-15 15:08 ` Junio C Hamano 2020-04-15 15:16 ` Jeff King 2020-04-15 18:52 ` Jonathan Nieder 2020-05-20 8:53 ` Andreas Schwab 2020-05-20 8:57 ` Andreas Schwab 2020-05-20 10:05 ` Michal Suchánek 2020-05-20 10:34 ` Andreas Schwab 2020-05-20 19:40 ` Jeff King 2020-05-25 18:22 ` Lukas Wunner 2020-05-25 18:37 ` Junio C Hamano 2020-04-16 6:31 ` Jiri Slaby 2020-04-21 10:02 ` Martin Wilck
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).