Re: Performance issue: initial git clone causes massive repack

[Jon Smirl <jonsmirl@gmail.com>]

On Mon, Apr 6, 2009 at 1:15 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Nicolas Pitre <nico@cam.org> writes:
>
>> What git-pack-objects does in this case is not a full repack.  It
>> instead _reuse_ as much of the existing packs as possible, and only does
>> the heavy packing processing for loose objects and/or inter pack
>> boundaryes when gluing everything together for streaming over the net.
>> If for example you have a single pack because your repo is already fully
>> packed, then the "packing operation" involved during a clone should
>> merely copy the existing pack over with no further attempt at delta
>> compression.
>
> One possibile scenario that you still need to spend memory and cycle is if
> the cloned repository was packed to an excessive depth to cause many of
> its objects to be in deltified form on insanely deep chains, while cloning
> send-pack uses a depth that is more reasonable.  Then pack-objects invoked
> by send-pack is not allowed to reuse most of the objects and would end up
> redoing the delta on them.

That seems broken. You went through all of the trouble to make the
pack file smaller to reduce transmission time, and then clone undoes
the work.

What about making a very simple special case for an initial clone?
First thing an initial clone does is copy all of the pack files from
the server to the client without even looking at them. Some of these
packs will probably be marked 'keep' because they are old history and
have been densely packed. Once the packs are down, start over and do a
fetch taking these packs into account.

-- 
Jon Smirl
jonsmirl@gmail.com