Re: Error converting from 1.4.4.1 to 1.5.0?

From: Junio C Hamano <junkio@cox.net>
To: Bill Lear <rael@zopyra.com>
Cc: git@vger.kernel.org
Subject: Re: Error converting from 1.4.4.1 to 1.5.0?
Date: Wed, 14 Feb 2007 13:12:22 -0800	[thread overview]
Message-ID: <7vmz3gmojt.fsf@assigned-by-dhcp.cox.net> (raw)
In-Reply-To: <17875.30187.289679.417079@lisa.zopyra.com> (Bill Lear's message of "Wed, 14 Feb 2007 14:49:47 -0600")

Bill Lear <rael@zopyra.com> writes:

>>How big is this pack, what platform are you working on and whose
>>SHA-1 implementation do you use?
>
> In order:
>
> % cd .git/objects/pack
> % ls -l
> -r--r--r--  1 rael software    77360 Feb 13 10:18 pack-23d1a9af78b4b78d...
> -r--r--r--  1 rael software 87874337 Feb 14 10:00 pack-23d1a9af78b4b78d...
>
> [output of ls trimmed to width]
>
> % uname -a
> Linux lisa.zopyra.com 2.6.9-34.0.2.ELsmp #1 SMP Fri Jul 7 18:22:55 CDT 2006 x86_64 x86_64 x86_64 GNU/Linux
>
> I don't know which SHA-1 implementation I use --- I just installed git
> and off I went.  I do see this:
>
> % which sha1sum
> /usr/bin/sha1sum
> % sha1sum --version
> shasum (coreutils) 5.2.1
> Written by Ulrich Drepper and Scott Miller.
> [...]
>
> But I'm not sure which library is in use --- how do I know?

"ldd ~/git-master/bin/git" tells me that it links with libcrypto.so,
so I am using OpenSSL's SHA-1 implementation.  I do not know
what your distro uses (or you hand built git yourself?).

I asked this question, because... 

>>Also, do you have a huge blob in the repository?  I do not know
>>if it is related but the write_sha1_file_prepare() function on
>>the codepath to write loose objects out would trigger the same
>>bug...
>
> I don't know what "huge" is, but the pack file seems to be the largest
> and then one of the objects is listed at 28,604,986 bytes, but nothing
> else is very large.

... we had a problem that some SHA-1 implementation gave bogus
results when we did this:

	SHA_CTX ctx;
	unsigned char sha1[20];

	SHA1_Init(&ctx);
        /* hash len bytes starting from buf */
	SHA1_Update(&ctx, buf, len);
	SHA1_Final(sha1, &ctx);

and asked to hash a large buffer in one go.  One of the SHA1
implementations we ship ourselves (I think it was handcrafted
PPC one) used to have problems.  But I do not think 27MB is
large enough to trigger such a library bug (the bug was integer
wraparound of a bit counter, I think).

So it looks more and more like a bit decay as Linus suspected...