* Why is git-clone --reference so slow?
@ 2007-02-05 11:46 David Woodhouse
2007-02-05 15:31 ` Johannes Schindelin
2007-02-07 10:03 ` Junio C Hamano
0 siblings, 2 replies; 4+ messages in thread
From: David Woodhouse @ 2007-02-05 11:46 UTC (permalink / raw)
To: git
My DSL line sucks; I know this. But why is git-clone so bad at using it?
This is slow and seems to be downloading a lot of stuff it doesn't
need...
pmac /pmac/git $ git-clone --reference /pmac/git/linux-2.6 git://git.kernel.org/pub/scm/linux/kernel/git/geoff/ps3-linux.git
remote: Generating pack...
remote: Done counting 14428 objects.
remote: Deltifying 14428 objects.
remote: 100% (14428/14428) done
Indexing 14428 objects.
25% (3662/14428) done
At this point I got bored and hit ^C, then logged in to hera and did it
there, then rsync'd the result...
hera /home/dwmw2/x $ git-clone -n --bare --reference /pub/scm/linux/kernel/git/torvalds/linux-2.6.git /pub/scm/linux/kernel/git/geoff/ps3-linux.git
remote: Generating pack...
remote: Done counting 14490 objects.
remote: Deltifying 14490 objects.
remote: 100% (14490/14490) done
Indexing 14490 objects.
remote: Total 14490, written 14490 (delta 579), reused 5598 (delta 367)
100% (14490/14490) done
Resolving 579 deltas.
100% (579/579) done
pmac /pmac/git/ps3-linux $ rsync -avz master.kernel.org:/home/dwmw2/x/ps3-linux/ .git
pmac /pmac/git/ps3-linux $ echo /pmac/git/linux-2.6/.git/objects > .git/objects/info/alternates
That was a _lot_ faster, and abused my poor DSL line a lot less. Why's
git so bad at it?
--
dwmw2
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Why is git-clone --reference so slow?
2007-02-05 11:46 Why is git-clone --reference so slow? David Woodhouse
@ 2007-02-05 15:31 ` Johannes Schindelin
2007-02-05 15:35 ` David Woodhouse
2007-02-07 10:03 ` Junio C Hamano
1 sibling, 1 reply; 4+ messages in thread
From: Johannes Schindelin @ 2007-02-05 15:31 UTC (permalink / raw)
To: David Woodhouse; +Cc: git
Hi,
On Mon, 5 Feb 2007, David Woodhouse wrote:
> My DSL line sucks; I know this. But why is git-clone so bad at using it?
>
> This is slow and seems to be downloading a lot of stuff it doesn't
> need...
>
> pmac /pmac/git $ git-clone --reference /pmac/git/linux-2.6 git://git.kernel.org/pub/scm/linux/kernel/git/geoff/ps3-linux.git
Is this a Mac? Is it Mac OS X? Which git version do you use?
We recently had a case like this, and it was fixed: apparently, mmap() on
Mac OS X is less than fast...
Hth,
Dscho
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Why is git-clone --reference so slow?
2007-02-05 15:31 ` Johannes Schindelin
@ 2007-02-05 15:35 ` David Woodhouse
0 siblings, 0 replies; 4+ messages in thread
From: David Woodhouse @ 2007-02-05 15:35 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git
On Mon, 2007-02-05 at 16:31 +0100, Johannes Schindelin wrote:
> Is this a Mac? Is it Mac OS X? Which git version do you use?
>
> We recently had a case like this, and it was fixed: apparently, mmap()
> on Mac OS X is less than fast...
Yes, it's a Mac G5. No, it's not OSX; it's Fedora Core 6. As far as I
can tell it's downloading far more than it needs before discarding
objects which are already in the local repository used in --reference.
--
dwmw2
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Why is git-clone --reference so slow?
2007-02-05 11:46 Why is git-clone --reference so slow? David Woodhouse
2007-02-05 15:31 ` Johannes Schindelin
@ 2007-02-07 10:03 ` Junio C Hamano
1 sibling, 0 replies; 4+ messages in thread
From: Junio C Hamano @ 2007-02-07 10:03 UTC (permalink / raw)
To: David Woodhouse; +Cc: git
David Woodhouse <dwmw2@infradead.org> writes:
> My DSL line sucks; I know this. But why is git-clone so bad at using it?
I did the same and it initially got 45MB pack (14.5k objects).
For my case, the reason was because the repository I used as the
reference was pruned and packed its refs, and git-clone was not
fully using the refs from the reference repository.
The following patch made it to be almost instantaneous. The
resulting clone has 600kB pack (0.5k objects).
diff --git a/git-clone.sh b/git-clone.sh
index 1710996..1bd54de 100755
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -178,46 +178,32 @@ esac && export GIT_DIR && git-init ${template+"$template"} || usage
if test -n "$reference"
then
+ ref_git=
if test -d "$reference"
then
if test -d "$reference/.git/objects"
then
- reference="$reference/.git"
+ ref_git="$reference/.git"
+ elif test -d "$reference/objects"
+ then
+ ref_git="$reference"
fi
- reference=$(cd "$reference" && pwd)
- echo "$reference/objects" >"$GIT_DIR/objects/info/alternates"
- (cd "$reference" && tar cf - refs) |
- (cd "$GIT_DIR/refs" &&
- mkdir reference-tmp &&
- cd reference-tmp &&
- tar xf - &&
- find refs ! -type d -print |
- while read ref
- do
- if test -h "$ref"
- then
- # Old-style symbolic link ref. Not likely
- # to appear under refs/ but we might as well
- # deal with them.
- :
- elif test -f "$ref"
- then
- point=$(cat "$ref") &&
- case "$point" in
- 'ref: '*) ;;
- *) continue ;;
- esac
- fi
- # The above makes true ref to 'continue' and
- # we will come here when we are looking at
- # symbolic link ref or a textual symref (or
- # garbage, like fifo).
- # The true ref pointed at by it is enough to
- # ensure that we do not fetch objects reachable
- # from it.
- rm -f "$ref"
- done
- )
+ fi
+ if test -n "$ref_git"
+ then
+ ref_git=$(cd "$ref_git" && pwd)
+ echo "$ref_git/objects" >"$GIT_DIR/objects/info/alternates"
+ (
+ GIT_DIR="$ref_git" git for-each-ref \
+ --format='%(objectname) %(*objectname)'
+ ) |
+ while read a b
+ do
+ test -z "$a" ||
+ git update-ref "refs/reference-tmp/$a" "$a"
+ test -z "$b" ||
+ git update-ref "refs/reference-tmp/$b" "$b"
+ done
else
die "reference repository '$reference' is not a local directory."
fi
^ permalink raw reply related [flat|nested] 4+ messages in thread
end of thread, other threads:[~2007-02-07 10:03 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-02-05 11:46 Why is git-clone --reference so slow? David Woodhouse
2007-02-05 15:31 ` Johannes Schindelin
2007-02-05 15:35 ` David Woodhouse
2007-02-07 10:03 ` Junio C Hamano
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.