GIT BUG? GIT occasionally redownloads its entire data set

GIT BUG? GIT occasionally redownloads its entire data set

[David Howells]

Hi,

I'm seeing a possible bug in which GIT, in which I issue a git-pull command on
a tree that was okay the day before, GIT reports that there are "no common
commits" and then redownloads the entire tree (as far as I can tell).

This happened just now when I did I git-pull on a GIT repository that I think
went from Monday morning's head:

	http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=59fcbddaff6f862cc1584b488866d9c4a5579085

to v2.6.29.  I've attached the command output for reference.

I've seen this a number of times.  It has been suggested this usually happens
when a new version tag is committed, but I wouldn't normally see that as the
git-pull to my base repository is normally run by crond.

This bug is annoying as it can eat a big chunk out of the download quota set by
my ISP.

David
---
warthog>git-pull
remote: Counting objects: 447, done.
remote: Compressing objects: 100% (57/57), done.
remote: Total 266 (delta 216), reused 254 (delta 207)
Receiving objects: 100% (266/266), 37.94 KiB, done.
Resolving deltas: 100% (216/216), completed with 105 local objects.
warning: no common commits
remote: Counting objects: 1075436, done.
remote: Compressing objects: 100% (176743/176743), done.
remote: Total 1075436 (delta 896164), reused 1072243 (delta 893708)
Receiving objects: 100% (1075436/1075436), 260.71 MiB | 307 KiB/s, done.
Resolving deltas: 100% (896164/896164), done.
From git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6
 * [new tag]         v2.6.29    -> v2.6.29
Updating 59fcbdd..8e0ee43
Fast forward
 Documentation/filesystems/ext2.txt        |    5 +-
 Documentation/filesystems/proc.txt        |    7 +++
 Makefile                                  |    7 ++-
 arch/powerpc/kernel/head_32.S             |    9 ++++
 arch/sparc/include/asm/pil.h              |    1 +
 arch/sparc/kernel/kgdb_64.c               |    2 +-
 arch/sparc/kernel/pci_common.c            |    2 +-
 arch/sparc/kernel/ttable.S                |    7 +++-
 arch/sparc/mm/ultra.S                     |   24 +----------
 drivers/dca/dca-sysfs.c                   |   21 ++++++++++
 drivers/isdn/gigaset/bas-gigaset.c        |   16 ++++++-
 drivers/net/Kconfig                       |    2 +-
 drivers/net/benet/be.h                    |    1 +
 drivers/net/benet/be_main.c               |   60 ++++++++++++++++------------
 drivers/net/bnx2.c                        |   12 +++---
 drivers/net/bonding/bond_main.c           |   25 ++++++++++--
 drivers/net/dm9000.c                      |    6 ++-
 drivers/net/dnet.c                        |    6 +-
 drivers/net/sh_eth.c                      |   20 ++++++++-
 drivers/net/sh_eth.h                      |    4 +-
 drivers/net/smsc911x.c                    |    4 ++
 drivers/net/sungem.c                      |    9 ++--
 drivers/net/tulip/tulip_core.c            |   45 ++++++++++++++--------
 drivers/net/ucc_geth.c                    |   34 +++++-----------
 drivers/net/ucc_geth.h                    |    3 +-
 drivers/net/virtio_net.c                  |    3 +
 drivers/net/wireless/ath9k/ath9k.h        |    4 +-
 drivers/net/wireless/ath9k/core.h         |   33 ++++++++++++++++
 drivers/net/wireless/ath9k/hw.c           |   22 ++++++++++-
 drivers/net/wireless/ath9k/main.c         |    1 +
 drivers/net/wireless/zd1211rw/zd_mac.c    |    8 +++-
 drivers/sbus/char/openprom.c              |    1 +
 include/linux/dca.h                       |   20 ++++++++++
 include/net/netfilter/nf_conntrack_core.h |    3 +-
 kernel/signal.c                           |    8 ++++
 net/core/dev.c                            |    7 ++-
 net/ipv4/ip_fragment.c                    |    3 +-
 net/ipv6/netfilter/nf_conntrack_reasm.c   |    8 ++--
 net/ipv6/reassembly.c                     |    7 +--
 net/ipv6/sit.c                            |    2 +-
 net/mac80211/tx.c                         |    2 +
 net/netfilter/nf_conntrack_core.c         |    2 +-
 net/netfilter/nf_conntrack_netlink.c      |    1 +
 net/netfilter/nf_conntrack_proto_tcp.c    |    4 +-
 net/sctp/endpointola.c                    |    3 +-
 net/wireless/Kconfig                      |   10 +++++
 net/wireless/lib80211_crypt_ccmp.c        |    2 +
 net/wireless/lib80211_crypt_tkip.c        |    4 ++
 net/wireless/nl80211.c                    |   11 +++++
 49 files changed, 351 insertions(+), 150 deletions(-)
warthog>

Re: GIT BUG? GIT occasionally redownloads its entire data set

[Junio C Hamano]

David Howells <dhowells@redhat.com> writes:

> to v2.6.29.  I've attached the command output for reference.

Version of git used, and the .git/config ([remote "origin"] section)?

> warthog>git-pull
> remote: Counting objects: 447, done.
> remote: Compressing objects: 100% (57/57), done.
> remote: Total 266 (delta 216), reused 254 (delta 207)
> Receiving objects: 100% (266/266), 37.94 KiB, done.
> Resolving deltas: 100% (216/216), completed with 105 local objects.
> warning: no common commits
> remote: Counting objects: 1075436, done.
> remote: Compressing objects: 100% (176743/176743), done.
> remote: Total 1075436 (delta 896164), reused 1072243 (delta 893708)
> Receiving objects: 100% (1075436/1075436), 260.71 MiB | 307 KiB/s, done.
> Resolving deltas: 100% (896164/896164), done.
>>From git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6

This is intereseting because it shows that two separate transfer happened
back to back, which I do not think is what recent git does.

We used to first grab the objects reachable from primary refs (presumably
[remote "origin"] fetch = refs/heads/*:refs/remotes/origin/*) and then
turned around to ask for tags that possibly point at commits we freshly
obtained during the first transfer (i.e. the ones reachable from the tip
of "master").  In such an ancient version I vaguely recall some bugs fixed
before we stopped doing this two-stage transfer altogether, but I do not
recall offhand if a bug that loses track of what it just downloaded (which
is what your description suggests) was among them.

Rings a bell, anybody?

Re: GIT BUG? GIT occasionally redownloads its entire data set

[Dmitry Potapov]

On Tue, Mar 24, 2009 at 5:17 AM, Junio C Hamano <gitster@pobox.com> wrote:
>
> Rings a bell, anybody?

Maybe this:
http://article.gmane.org/gmane.comp.version-control.git/75201

It was fixed in Git 1.5.5 if I am not mistaken.


Dmitry

Re: GIT BUG? GIT occasionally redownloads its entire data set

[Junio C Hamano]

Dmitry Potapov <dpotapov@gmail.com> writes:

> On Tue, Mar 24, 2009 at 5:17 AM, Junio C Hamano <gitster@pobox.com> wrote:
>>
>> Rings a bell, anybody?
>
> Maybe this:
> http://article.gmane.org/gmane.comp.version-control.git/75201
>
> It was fixed in Git 1.5.5 if I am not mistaken.

Thanks.  Distributed memory works very well ;-)

Re: GIT BUG? GIT occasionally redownloads its entire data set

[David Howells]

Dmitry Potapov <dpotapov@gmail.com> wrote:

> It was fixed in Git 1.5.5 if I am not mistaken.

warthog>rpm -q git
git-1.5.4.3-3.fc8

It looks like I'm using a version of GIT that's just a little bit too old.

David