Re: Performance issue: initial git clone causes massive repack

[Nicolas Pitre <nico@cam.org>]

On Sun, 5 Apr 2009, david@lang.hm wrote:

> On Mon, 6 Apr 2009, Nicolas Sebrecht wrote:
> 
> > On Sun, Apr 05, 2009 at 02:28:35PM -0700, david@lang.hm wrote:
> > 
> > > guys, back off a little on telling the gentoo people to change.
> > 
> > Don't blame Git people, please. I currently am the only one here to
> > discuss that way and see a painful work coming at Gentoo.
> > Git people didn't discuss around thoses issues.
> > 
> > >                                                                 the
> > > kernel developers don't split th kernel into 'core' 'drivers' etc pieces
> > > just because some people only work on one area.
> > 
> > And you might notice that they don't provide a CVS access and actually
> > don't work around an unique shared repo. Also, you might notice that
> > keeping the history clean to assure the work on the kernel easier is not
> > an elementary issue.
> 
> these issues are completely seperate from the issue that the initial poster
> asked about, which is that when someone tries to do a clone of the repository
> the system wastes a lot of time creating a new pack.

And this shouldn't be, by design.  Especially if your repo serving clone 
requests is already well packed.

What git-pack-objects does in this case is not a full repack.  It 
instead _reuse_ as much of the existing packs as possible, and only does 
the heavy packing processing for loose objects and/or inter pack 
boundaryes when gluing everything together for streaming over the net.  
If for example you have a single pack because your repo is already fully 
packed, then the "packing operation" involved during a clone should 
merely copy the existing pack over with no further attempt at delta 
compression.

> don't worry about this part of things, worry about why the server wastes so
> many resources.

Indeed.  And since a significant amount of code involved happens to be 
mine, I do wonder.


Nicolas