From: Thomas Lucas <toml@zuken.co.uk>
To: git@vger.kernel.org
Subject: Git Garbage Collect Error.
Date: Wed, 13 Jun 2012 11:27:04 +0100 [thread overview]
Message-ID: <4FD86AF8.1050100@zuken.co.uk> (raw)
Hi,
Hopefully this is the right place to send bug reports... The community page
"http://git-scm.com/community" suggests that it is.
Introduction
I am creating a large GIT repository fetching from a large SVN repository, as an
experiment initially. I usually use GIT repositories interfacing to parts of the
SVN repository.
Defect
During garbage collection (git gc) it encountered the following error:
git gc | git gc --prune :
Counting objects: 856758, done.
Delta compression using up to 2 threads.
fatal: Out of memory, malloc failed (tried to allocate 303237121 bytes)
error: failed to run repack
git gc --aggressive:
Counting objects: 856758, done.
Delta compression using up to 2 threads.
fatal: Out of memory, malloc failed (tried to allocate 291942401 bytes)
error: failed to run repack
At the moment the bare repository is about 4Gb in size and about 2/3rds the way
through fetching.
The compression gets over 90% of the way through before this error occurs, but I
don't think any compression results are kept, because when you repeat it has the
same amount of work to do.
Initially this happen during an automatic gc during the fetch process. This
aborted the fetch.
My system is XP64 2 core with 4Gb of memory and plenty of virtual memory.
Comments
If this a genuine limitation due to the size of an object and memory handling
limitations, then perhaps the error could be caught and the successful results
kept. Ie. do a partial compression. That way the process could continue.
Background
My requirement is to have GIT repositories of a source directory with all SVN
branches included so that I can more easily merge and compare branches using
GIT. However for even small source directories it takes weeks to fetch from the
SVN respository (including all tags and branches), whereas fetching just the
trunk takes a few hours. The SVN repository has over 90000 revisions. I am aware
that I can fetch a sub-set of revisions (I don't want to at the moment), but
I've found no way to fetch a sub-set of branches.
My config is as follows:
[svn-remote "svn"]
url = svn://svn
fetch = trunk:refs/remotes/svn/trunk
branches = branches/*:refs/remotes/svn/*
tags = tags/*:refs/remotes/svn/tags/*
I set this up using:
git svn init --prefix=svn/ --stdlayout --no-minimize-url svn://svn
To do this for individual directories I have to do the following:
git svn init --prefix=svn/ --stdlayout --no-minimize-url
svn://svn/trunk/source/<dir>
and then edit the config manually so that:
[svn-remote "svn"]
url = svn://svn
fetch = trunk/source/<dir>:refs/remotes/svn/trunk
branches = branches/*/source/<dir>:refs/remotes/svn/*
tags = tags/*/source/<dir>:refs/remotes/svn/tags/*
This works ok but I couldn't get this result by using "git svn init" directly.
Maybe I've missed something.
Regards,
Tom.
next reply other threads:[~2012-06-13 10:27 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-06-13 10:27 Thomas Lucas [this message]
2012-07-12 9:32 ` Git Garbage Collect Error Jeff King
2012-07-14 3:36 ` sascha-ml
2012-07-12 12:34 ` Philippe Vaucher
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4FD86AF8.1050100@zuken.co.uk \
--to=toml@zuken.co.uk \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).