Re: Performance issue: initial git clone causes massive repack

[Jakub Narebski <jnareb@gmail.com>]

Björn Steinbrink <B.Steinbrink@gmx.de> writes:
> On 2009.04.05 23:24:27 -0400, Nicolas Pitre wrote:
> > On Sun, 5 Apr 2009, Sverre Rabbelier wrote:
> > > 
> > > I agree here, we should either say "look, we don't really support big
> > > repositories because [explanation here], unless you [workarounds
> > > here]" OR we should work to improve the support we do have. Of course,
> > > the latter option does not magically create developer time to work on
> > > that, but if we do go that way we should at least tell people that we
> > > are aware of the problems and that it's on the global TODO list (not
> > > necessarily on anyone's personal TODO list though).
> > 
> > For the record... I at least am aware of the problem and it is indeed on 
> > my personal git todo list.  Not that I have a clear solution yet (I've 
> > been pondering on some git packing issues for almost 4 years now).
> > 
> > Still, in this particular case, the problem appears to be unclear to me, 
> > like "this shouldn't be so bad".
> 
> It's not primarily pack-objects, I think. It's the rev-list that's run
> by upload-pack.  Running "git rev-list --objects --all" on that repo
> eats about 2G RSS, easily killing the system's cache on a small box,
> leading to swapping and a painful time reading the packfile contents
> afterwards to send them to the client.

Than I think that "packfile caching" GSoC project (which is IIRC
"object enumeration caching", or at least includes it) should help
here.  You would, from what I understand, run "git rev-list -objects
--all --not <tops of cache>" + sequential read of object enumeration
cache...

-- 
Jakub Narebski
Poland
ShadeHawk on #git