* [PATCH 0/4] Fetching mass of objects at once
@ 2006-07-27 21:53 Petr Baudis
2006-07-27 21:56 ` [PATCH 1/4] Make pull() take some implicit data as explicit arguments Petr Baudis
` (4 more replies)
0 siblings, 5 replies; 7+ messages in thread
From: Petr Baudis @ 2006-07-27 21:53 UTC (permalink / raw)
To: git; +Cc: jonsmirl
Hi,
this series implements support for fetching many objects at once with
git-local-fetch and git-http-fetch, greatly helping Cogito's performance. It
would be of course nice to have this in Git 1.4.2, but it's probably too late
for that.
I didn't bother to convert git-ssh-fetch since I think noone uses that
anymore anyway (why would they...), or only some legacy users do.
I will followup with a patch for Cogito to take advantage of this. It's
now roughly on par with Git in cloning speed.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/4] Make pull() take some implicit data as explicit arguments
2006-07-27 21:53 [PATCH 0/4] Fetching mass of objects at once Petr Baudis
@ 2006-07-27 21:56 ` Petr Baudis
2006-07-27 21:56 ` [PATCH 2/4] Make pull() support fetching multiple targets at once Petr Baudis
` (3 subsequent siblings)
4 siblings, 0 replies; 7+ messages in thread
From: Petr Baudis @ 2006-07-27 21:56 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
Currently it's a bit weird that pull() takes a single argument
describing the commit but takes the write_ref from a global variable.
This makes it take that as a parameter as well, which might be nicer
for the libification in the future, but especially it will make for
nicer code when we implement pull()ing multiple commits at once.
Signed-off-by: Petr Baudis <pasky@suse.cz>
---
fetch.c | 6 ++----
fetch.h | 11 ++++-------
http-fetch.c | 4 ++--
local-fetch.c | 4 ++--
ssh-fetch.c | 4 ++--
5 files changed, 12 insertions(+), 17 deletions(-)
diff --git a/fetch.c b/fetch.c
index 989d7a4..3255cc6 100644
--- a/fetch.c
+++ b/fetch.c
@@ -8,9 +8,6 @@ #include "tag.h"
#include "blob.h"
#include "refs.h"
-const char *write_ref = NULL;
-const char *write_ref_log_details = NULL;
-
int get_tree = 0;
int get_history = 0;
int get_all = 0;
@@ -213,7 +210,8 @@ static int mark_complete(const char *pat
return 0;
}
-int pull(char *target)
+int pull(char *target, const char *write_ref,
+ const char *write_ref_log_details)
{
struct ref_lock *lock = NULL;
unsigned char sha1[20];
diff --git a/fetch.h b/fetch.h
index 841bb1a..7bda355 100644
--- a/fetch.h
+++ b/fetch.h
@@ -22,12 +22,6 @@ extern void prefetch(unsigned char *sha1
*/
extern int fetch_ref(char *ref, unsigned char *sha1);
-/* If set, the ref filename to write the target value to. */
-extern const char *write_ref;
-
-/* If set additional text will appear in the ref log. */
-extern const char *write_ref_log_details;
-
/* Set to fetch the target tree. */
extern int get_tree;
@@ -46,6 +40,9 @@ extern int get_recover;
/* Report what we got under get_verbosely */
extern void pull_say(const char *, const char *);
-extern int pull(char *target);
+/* If write_ref is set, the ref filename to write the target value to. */
+/* If write_ref_log_details is set, additional text will appear in the ref log. */
+extern int pull(char *target, const char *write_ref,
+ const char *write_ref_log_details);
#endif /* PULL_H */
diff --git a/http-fetch.c b/http-fetch.c
index dc286b7..963d439 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -1216,6 +1216,7 @@ int fetch_ref(char *ref, unsigned char *
int main(int argc, char **argv)
{
+ const char *write_ref = NULL;
char *commit_id;
char *url;
char *path;
@@ -1250,7 +1251,6 @@ int main(int argc, char **argv)
}
commit_id = argv[arg];
url = argv[arg + 1];
- write_ref_log_details = url;
http_init();
@@ -1268,7 +1268,7 @@ int main(int argc, char **argv)
alt->path_len = strlen(path);
}
- if (pull(commit_id))
+ if (pull(commit_id, write_ref, url))
rc = 1;
http_cleanup();
diff --git a/local-fetch.c b/local-fetch.c
index 65a803a..308ed00 100644
--- a/local-fetch.c
+++ b/local-fetch.c
@@ -204,6 +204,7 @@ static const char local_pull_usage[] =
*/
int main(int argc, char **argv)
{
+ const char *write_ref = NULL;
char *commit_id;
int arg = 1;
@@ -240,9 +241,8 @@ int main(int argc, char **argv)
usage(local_pull_usage);
commit_id = argv[arg];
path = argv[arg + 1];
- write_ref_log_details = path;
- if (pull(commit_id))
+ if (pull(commit_id, write_ref, path))
return 1;
return 0;
diff --git a/ssh-fetch.c b/ssh-fetch.c
index a8a6cfb..aef3aa4 100644
--- a/ssh-fetch.c
+++ b/ssh-fetch.c
@@ -123,6 +123,7 @@ static const char ssh_fetch_usage[] =
" [-c] [-t] [-a] [-v] [--recover] [-w ref] commit-id url";
int main(int argc, char **argv)
{
+ const char *write_ref = NULL;
char *commit_id;
char *url;
int arg = 1;
@@ -159,7 +160,6 @@ int main(int argc, char **argv)
}
commit_id = argv[arg];
url = argv[arg + 1];
- write_ref_log_details = url;
if (setup_connection(&fd_in, &fd_out, prog, url, arg, argv + 1))
return 1;
@@ -167,7 +167,7 @@ int main(int argc, char **argv)
if (get_version())
return 1;
- if (pull(commit_id))
+ if (pull(commit_id, write_ref, url))
return 1;
return 0;
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/4] Make pull() support fetching multiple targets at once
2006-07-27 21:53 [PATCH 0/4] Fetching mass of objects at once Petr Baudis
2006-07-27 21:56 ` [PATCH 1/4] Make pull() take some implicit data as explicit arguments Petr Baudis
@ 2006-07-27 21:56 ` Petr Baudis
2006-07-27 21:56 ` [PATCH 3/4] Teach git-local-fetch the --stdin switch Petr Baudis
` (2 subsequent siblings)
4 siblings, 0 replies; 7+ messages in thread
From: Petr Baudis @ 2006-07-27 21:56 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
pull() now takes an array of arguments instead of just one of each kind.
Currently, no users use the new capability, but that'll change.
Signed-off-by: Petr Baudis <pasky@suse.cz>
---
fetch.c | 78 +++++++++++++++++++++++++++++++++------------------------
fetch.h | 2 +
http-fetch.c | 2 +
local-fetch.c | 2 +
ssh-fetch.c | 2 +
5 files changed, 49 insertions(+), 37 deletions(-)
diff --git a/fetch.c b/fetch.c
index 3255cc6..281df61 100644
--- a/fetch.c
+++ b/fetch.c
@@ -210,55 +210,67 @@ static int mark_complete(const char *pat
return 0;
}
-int pull(char *target, const char *write_ref,
+int pull(int targets, char **target, const char **write_ref,
const char *write_ref_log_details)
{
- struct ref_lock *lock = NULL;
- unsigned char sha1[20];
+ struct ref_lock **lock = xcalloc(targets, sizeof(struct ref_lock *));
+ unsigned char *sha1 = xmalloc(targets * 20);
char *msg;
int ret;
+ int i;
save_commit_buffer = 0;
track_object_refs = 0;
- if (write_ref) {
- lock = lock_ref_sha1(write_ref, NULL, 0);
- if (!lock) {
- error("Can't lock ref %s", write_ref);
- return -1;
+
+ for (i = 0; i < targets; i++) {
+ if (!write_ref[i])
+ continue;
+
+ lock[i] = lock_ref_sha1(write_ref[i], NULL, 0);
+ if (!lock[i]) {
+ error("Can't lock ref %s", write_ref[i]);
+ goto unlock_and_fail;
}
}
if (!get_recover)
for_each_ref(mark_complete);
- if (interpret_target(target, sha1)) {
- error("Could not interpret %s as something to pull", target);
- if (lock)
- unlock_ref(lock);
- return -1;
+ for (i = 0; i < targets; i++) {
+ if (interpret_target(target[i], &sha1[20 * i])) {
+ error("Could not interpret %s as something to pull", target[i]);
+ goto unlock_and_fail;
+ }
+ if (process(lookup_unknown_object(&sha1[20 * i])))
+ goto unlock_and_fail;
}
- if (process(lookup_unknown_object(sha1))) {
- if (lock)
- unlock_ref(lock);
- return -1;
+
+ if (loop())
+ goto unlock_and_fail;
+
+ if (write_ref_log_details) {
+ msg = xmalloc(strlen(write_ref_log_details) + 12);
+ sprintf(msg, "fetch from %s", write_ref_log_details);
+ } else {
+ msg = NULL;
}
- if (loop()) {
- if (lock)
- unlock_ref(lock);
- return -1;
+ for (i = 0; i < targets; i++) {
+ if (!write_ref[i])
+ continue;
+ ret = write_ref_sha1(lock[i], &sha1[20 * i], msg ? msg : "fetch (unknown)");
+ lock[i] = NULL;
+ if (ret)
+ goto unlock_and_fail;
}
+ if (msg)
+ free(msg);
- if (write_ref) {
- if (write_ref_log_details) {
- msg = xmalloc(strlen(write_ref_log_details) + 12);
- sprintf(msg, "fetch from %s", write_ref_log_details);
- }
- else
- msg = NULL;
- ret = write_ref_sha1(lock, sha1, msg ? msg : "fetch (unknown)");
- if (msg)
- free(msg);
- return ret;
- }
return 0;
+
+
+unlock_and_fail:
+ for (i = 0; i < targets; i++)
+ if (lock[i])
+ unlock_ref(lock[i]);
+ return -1;
}
diff --git a/fetch.h b/fetch.h
index 7bda355..75e48af 100644
--- a/fetch.h
+++ b/fetch.h
@@ -42,7 +42,7 @@ extern void pull_say(const char *, const
/* If write_ref is set, the ref filename to write the target value to. */
/* If write_ref_log_details is set, additional text will appear in the ref log. */
-extern int pull(char *target, const char *write_ref,
+extern int pull(int targets, char **target, const char **write_ref,
const char *write_ref_log_details);
#endif /* PULL_H */
diff --git a/http-fetch.c b/http-fetch.c
index 963d439..bc67db1 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -1268,7 +1268,7 @@ int main(int argc, char **argv)
alt->path_len = strlen(path);
}
- if (pull(commit_id, write_ref, url))
+ if (pull(1, &commit_id, &write_ref, url))
rc = 1;
http_cleanup();
diff --git a/local-fetch.c b/local-fetch.c
index 308ed00..eb19f1a 100644
--- a/local-fetch.c
+++ b/local-fetch.c
@@ -242,7 +242,7 @@ int main(int argc, char **argv)
commit_id = argv[arg];
path = argv[arg + 1];
- if (pull(commit_id, write_ref, path))
+ if (pull(1, &commit_id, &write_ref, path))
return 1;
return 0;
diff --git a/ssh-fetch.c b/ssh-fetch.c
index aef3aa4..6e16568 100644
--- a/ssh-fetch.c
+++ b/ssh-fetch.c
@@ -167,7 +167,7 @@ int main(int argc, char **argv)
if (get_version())
return 1;
- if (pull(commit_id, write_ref, url))
+ if (pull(1, &commit_id, &write_ref, url))
return 1;
return 0;
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 3/4] Teach git-local-fetch the --stdin switch
2006-07-27 21:53 [PATCH 0/4] Fetching mass of objects at once Petr Baudis
2006-07-27 21:56 ` [PATCH 1/4] Make pull() take some implicit data as explicit arguments Petr Baudis
2006-07-27 21:56 ` [PATCH 2/4] Make pull() support fetching multiple targets at once Petr Baudis
@ 2006-07-27 21:56 ` Petr Baudis
2006-07-28 1:57 ` Petr Baudis
2006-07-27 21:56 ` [PATCH 4/4] Teach git-http-fetch " Petr Baudis
2006-07-27 21:57 ` [PATCH 0/4] Fetching mass of objects at once Petr Baudis
4 siblings, 1 reply; 7+ messages in thread
From: Petr Baudis @ 2006-07-27 21:56 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
This makes it possible to fetch many commits (refs) at once, greatly
speeding up cg-clone.
Signed-off-by: Petr Baudis <pasky@suse.cz>
---
Documentation/git-local-fetch.txt | 6 ++++++
fetch.c | 40 +++++++++++++++++++++++++++++++++++++
fetch.h | 6 ++++++
local-fetch.c | 32 ++++++++++++++++++++----------
4 files changed, 74 insertions(+), 10 deletions(-)
diff --git a/Documentation/git-local-fetch.txt b/Documentation/git-local-fetch.txt
index 87abec1..bd19820 100644
--- a/Documentation/git-local-fetch.txt
+++ b/Documentation/git-local-fetch.txt
@@ -29,6 +29,12 @@ OPTIONS
Writes the commit-id into the filename under $GIT_DIR/refs/<filename> on
the local end after the transfer is complete.
+--stdin::
+ Instead of a commit id on the commandline (which is not expected in this
+ case), 'git-local-fetch' excepts lines on stdin in the format
+
+ <commit-id>['\t'<filename-as-in--w>]
+
Author
------
Written by Junio C Hamano <junkio@cox.net>
diff --git a/fetch.c b/fetch.c
index 281df61..2151c7b 100644
--- a/fetch.c
+++ b/fetch.c
@@ -7,6 +7,7 @@ #include "tree-walk.h"
#include "tag.h"
#include "blob.h"
#include "refs.h"
+#include "strbuf.h"
int get_tree = 0;
int get_history = 0;
@@ -210,6 +211,45 @@ static int mark_complete(const char *pat
return 0;
}
+int pull_targets_stdin(char ***target, const char ***write_ref)
+{
+ int targets = 0, targets_alloc = 0;
+ struct strbuf buf;
+ *target = NULL; *write_ref = NULL;
+ strbuf_init(&buf);
+ while (1) {
+ char *rf_one = NULL;
+ char *tg_one;
+
+ read_line(&buf, stdin, '\n');
+ if (buf.eof)
+ break;
+ tg_one = buf.buf;
+ rf_one = strchr(tg_one, '\t');
+ if (rf_one)
+ *rf_one++ = 0;
+
+ if (targets >= targets_alloc) {
+ targets_alloc = targets_alloc ? targets_alloc * 2 : 64;
+ *target = xrealloc(*target, targets_alloc * sizeof(**target));
+ *write_ref = xrealloc(*write_ref, targets_alloc * sizeof(**write_ref));
+ }
+ (*target)[targets] = strdup(tg_one);
+ (*write_ref)[targets] = rf_one ? strdup(rf_one) : NULL;
+ targets++;
+ }
+ return targets;
+}
+
+void pull_targets_free(int targets, char **target, const char **write_ref)
+{
+ while (targets--) {
+ free(target[targets]);
+ if (write_ref[targets])
+ free((char *) write_ref[targets]);
+ }
+}
+
int pull(int targets, char **target, const char **write_ref,
const char *write_ref_log_details)
{
diff --git a/fetch.h b/fetch.h
index 75e48af..be48c6f 100644
--- a/fetch.h
+++ b/fetch.h
@@ -40,6 +40,12 @@ extern int get_recover;
/* Report what we got under get_verbosely */
extern void pull_say(const char *, const char *);
+/* Load pull targets from stdin */
+extern int pull_targets_stdin(char ***target, const char ***write_ref);
+
+/* Free up loaded targets */
+extern void pull_targets_free(int targets, char **target, const char **write_ref);
+
/* If write_ref is set, the ref filename to write the target value to. */
/* If write_ref_log_details is set, additional text will appear in the ref log. */
extern int pull(int targets, char **target, const char **write_ref,
diff --git a/local-fetch.c b/local-fetch.c
index eb19f1a..84b68a6 100644
--- a/local-fetch.c
+++ b/local-fetch.c
@@ -8,8 +8,9 @@ #include "fetch.h"
static int use_link = 0;
static int use_symlink = 0;
static int use_filecopy = 1;
+static int commits_on_stdin = 0;
-static char *path; /* "Remote" git repository */
+static const char *path; /* "Remote" git repository */
void prefetch(unsigned char *sha1)
{
@@ -194,7 +195,7 @@ int fetch_ref(char *ref, unsigned char *
}
static const char local_pull_usage[] =
-"git-local-fetch [-c] [-t] [-a] [-v] [-w filename] [--recover] [-l] [-s] [-n] commit-id path";
+"git-local-fetch [-c] [-t] [-a] [-v] [-w filename] [--recover] [-l] [-s] [-n] [--stdin] commit-id path";
/*
* By default we only use file copy.
@@ -202,10 +203,11 @@ static const char local_pull_usage[] =
* If -s is specified, then a symlink is attempted.
* If -n is _not_ specified, then a regular file-to-file copy is done.
*/
-int main(int argc, char **argv)
+int main(int argc, const char **argv)
{
- const char *write_ref = NULL;
- char *commit_id;
+ int commits;
+ const char **write_ref = NULL;
+ char **commit_id;
int arg = 1;
setup_git_directory();
@@ -230,20 +232,30 @@ int main(int argc, char **argv)
else if (argv[arg][1] == 'v')
get_verbosely = 1;
else if (argv[arg][1] == 'w')
- write_ref = argv[++arg];
+ write_ref = &argv[++arg];
else if (!strcmp(argv[arg], "--recover"))
get_recover = 1;
+ else if (!strcmp(argv[arg], "--stdin"))
+ commits_on_stdin = 1;
else
usage(local_pull_usage);
arg++;
}
- if (argc < arg + 2)
+ if (argc < arg + 2 - commits_on_stdin)
usage(local_pull_usage);
- commit_id = argv[arg];
- path = argv[arg + 1];
+ if (commits_on_stdin) {
+ commits = pull_targets_stdin(&commit_id, &write_ref);
+ } else {
+ commit_id = (char **) &argv[arg++];
+ commits = 1;
+ }
+ path = argv[arg];
- if (pull(1, &commit_id, &write_ref, path))
+ if (pull(commits, commit_id, write_ref, path))
return 1;
+ if (commits_on_stdin)
+ pull_targets_free(commits, commit_id, write_ref);
+
return 0;
}
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 4/4] Teach git-http-fetch the --stdin switch
2006-07-27 21:53 [PATCH 0/4] Fetching mass of objects at once Petr Baudis
` (2 preceding siblings ...)
2006-07-27 21:56 ` [PATCH 3/4] Teach git-local-fetch the --stdin switch Petr Baudis
@ 2006-07-27 21:56 ` Petr Baudis
2006-07-27 21:57 ` [PATCH 0/4] Fetching mass of objects at once Petr Baudis
4 siblings, 0 replies; 7+ messages in thread
From: Petr Baudis @ 2006-07-27 21:56 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
Speeds up things quite a lot when fetching tags with Cogito.
Signed-off-by: Petr Baudis <pasky@suse.cz>
---
Documentation/git-http-fetch.txt | 8 ++++++-
http-fetch.c | 45 ++++++++++++++++++++++++--------------
2 files changed, 36 insertions(+), 17 deletions(-)
diff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt
index bc1a132..bea522e 100644
--- a/Documentation/git-http-fetch.txt
+++ b/Documentation/git-http-fetch.txt
@@ -8,7 +8,7 @@ git-http-fetch - downloads a remote git
SYNOPSIS
--------
-'git-http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] <commit> <url>
+'git-http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] [--stdin] <commit> <url>
DESCRIPTION
-----------
@@ -32,6 +32,12 @@ commit-id::
-w <filename>::
Writes the commit-id into the filename under $GIT_DIR/refs/<filename> on
the local end after the transfer is complete.
+
+--stdin::
+ Instead of a commit id on the commandline (which is not expected in this
+ case), 'git-http-fetch' excepts lines on stdin in the format
+
+ <commit-id>['\t'<filename-as-in--w>]
Author
------
diff --git a/http-fetch.c b/http-fetch.c
index bc67db1..1aad39b 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -36,6 +36,8 @@ #endif
#define PREV_BUF_SIZE 4096
#define RANGE_HEADER_SIZE 30
+static int commits_on_stdin = 0;
+
static int got_alternates = -1;
static int corrupt_object_found = 0;
@@ -43,7 +45,7 @@ static struct curl_slist *no_pragma_head
struct alt_base
{
- char *base;
+ const char *base;
int path_len;
int got_indices;
struct packed_git *packs;
@@ -81,7 +83,7 @@ struct object_request
};
struct alternates_request {
- char *base;
+ const char *base;
char *url;
struct buffer *buffer;
struct active_request_slot *slot;
@@ -142,7 +144,7 @@ static size_t fwrite_sha1_file(void *ptr
return size;
}
-static void fetch_alternates(char *base);
+static void fetch_alternates(const char *base);
static void process_object_response(void *callback_data);
@@ -507,7 +509,7 @@ static void process_alternates_response(
(struct alternates_request *)callback_data;
struct active_request_slot *slot = alt_req->slot;
struct alt_base *tail = alt;
- char *base = alt_req->base;
+ const char *base = alt_req->base;
static const char null_byte = '\0';
char *data;
int i = 0;
@@ -612,7 +614,7 @@ static void process_alternates_response(
got_alternates = 1;
}
-static void fetch_alternates(char *base)
+static void fetch_alternates(const char *base)
{
struct buffer buffer;
char *url;
@@ -1185,7 +1187,7 @@ int fetch_ref(char *ref, unsigned char *
char *url;
char hex[42];
struct buffer buffer;
- char *base = alt->base;
+ const char *base = alt->base;
struct active_request_slot *slot;
struct slot_results results;
buffer.size = 41;
@@ -1214,11 +1216,12 @@ int fetch_ref(char *ref, unsigned char *
return 0;
}
-int main(int argc, char **argv)
+int main(int argc, const char **argv)
{
- const char *write_ref = NULL;
- char *commit_id;
- char *url;
+ int commits;
+ const char **write_ref = NULL;
+ char **commit_id;
+ const char *url;
char *path;
int arg = 1;
int rc = 0;
@@ -1238,19 +1241,26 @@ int main(int argc, char **argv)
} else if (argv[arg][1] == 'v') {
get_verbosely = 1;
} else if (argv[arg][1] == 'w') {
- write_ref = argv[arg + 1];
+ write_ref = &argv[arg + 1];
arg++;
} else if (!strcmp(argv[arg], "--recover")) {
get_recover = 1;
+ } else if (!strcmp(argv[arg], "--stdin")) {
+ commits_on_stdin = 1;
}
arg++;
}
- if (argc < arg + 2) {
- usage("git-http-fetch [-c] [-t] [-a] [-v] [--recover] [-w ref] commit-id url");
+ if (argc < arg + 2 - commits_on_stdin) {
+ usage("git-http-fetch [-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url");
return 1;
}
- commit_id = argv[arg];
- url = argv[arg + 1];
+ if (commits_on_stdin) {
+ commits = pull_targets_stdin(&commit_id, &write_ref);
+ } else {
+ commit_id = (char **) &argv[arg++];
+ commits = 1;
+ }
+ url = argv[arg];
http_init();
@@ -1268,13 +1278,16 @@ int main(int argc, char **argv)
alt->path_len = strlen(path);
}
- if (pull(1, &commit_id, &write_ref, url))
+ if (pull(commits, commit_id, write_ref, url))
rc = 1;
http_cleanup();
curl_slist_free_all(no_pragma_header);
+ if (commits_on_stdin)
+ pull_targets_free(commits, commit_id, write_ref);
+
if (corrupt_object_found) {
fprintf(stderr,
"Some loose object were found to be corrupt, but they might be just\n"
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 0/4] Fetching mass of objects at once
2006-07-27 21:53 [PATCH 0/4] Fetching mass of objects at once Petr Baudis
` (3 preceding siblings ...)
2006-07-27 21:56 ` [PATCH 4/4] Teach git-http-fetch " Petr Baudis
@ 2006-07-27 21:57 ` Petr Baudis
4 siblings, 0 replies; 7+ messages in thread
From: Petr Baudis @ 2006-07-27 21:57 UTC (permalink / raw)
To: git; +Cc: jonsmirl
Dear diary, on Thu, Jul 27, 2006 at 11:53:26PM CEST, I got a letter
where Petr Baudis <pasky@suse.cz> said that...
> I will followup with a patch for Cogito to take advantage of this. It's
> now roughly on par with Git in cloning speed.
diff --git a/cg-Xfetchprogress b/cg-Xfetchprogress
index 53bcbd3..4a272db 100755
--- a/cg-Xfetchprogress
+++ b/cg-Xfetchprogress
@@ -67,7 +67,7 @@ sub getline {
if (m#^(link|symlink|copy|got) ([a-f0-9]{2})([a-f0-9]{38})#) {
$object = "$2/$3";
- } elsif (m#^(walk) ([a-f0-9]{2})([a-f0-9]{38})#) {
+ } elsif (m#^(ref|walk) ([a-f0-9]{2})([a-f0-9]{38})#) {
return 1; # redundant information
# rsync
diff --git a/cg-fetch b/cg-fetch
index a6e6959..9277f99 100755
--- a/cg-fetch
+++ b/cg-fetch
@@ -124,16 +124,10 @@ get_rsync()
return ${PIPESTATUS[0]}
}
-fetch_rsync()
+fetch_rsync_save()
{
- if [ $verbose -ge 2 ]; then
- # We must not pipe to prevent buffered I/O
- get_rsync -s -d "$2/objects" "$_git_objects"
- else
- get_rsync -s -d "$2/objects" "$_git_objects" | fetch_progress
- fi
ret=${PIPESTATUS[0]}
- if [ "$3" ] && [ "$ret" -eq "0" ]; then
+ if [ "$1" ] && [ "$ret" -eq "0" ]; then
if [ "$orig_head" ]; then
git-rev-list --objects $new_head ^$orig_head |
while read obj type; do
@@ -141,12 +135,29 @@ fetch_rsync()
done ||
die "rsync fetch incomplete, some objects missing"
fi
- cat "$_git/refs/${3%/*}/.${3##*/}-fetching" > "$_git/refs/$3"
+ cat "$_git/refs/${3%/*}/.${3##*/}-fetching" > "$_git/refs/$1"
fi
return $ret
}
+fetch_rsync()
+{
+ if [ $verbose -ge 2 ]; then
+ # We must not pipe to prevent buffered I/O
+ get_rsync -s -d "$2/objects" "$_git_objects"
+ else
+ get_rsync -s -d "$2/objects" "$_git_objects" | fetch_progress
+ fi
+ if [ x"$1" = x"--stdin" ]; then
+ while read c w; do
+ echo "$c" >"$_git/refs/$w"
+ done
+ else
+ fetch_rsync_save "$3"
+ fi
+}
+
get_http()
{
[ "$1" = "-b" ] && shift
@@ -229,21 +240,13 @@ fetch_tags()
# if so, fetch the tag -- which should be
# a cheap operation -- to complete the chain.
- echo -n "Missing tag ${tagname#tags/}... "
- if $fetch "$tagname" "$uri" "$tagname" 2>/dev/null >&2; then
- echo "retrieved"
- else
- # 17 is code from packed transport, which
- # will grab all of them en masse later
- if [ "$?" -ne "17" ]; then
- echo "unable to retrieve"
- else
- echo ""
- fi
- fi
- done
- [ "${PIPESTATUS[0]}" -eq "0" ] ||
- echo "unable to get tags list (non-fatal)" >&2
+ echo "Missing tag ${tagname#tags/}..." >&2
+ echo -e "$tagname"\\t"$tagname"
+ done |
+ sort | uniq | $fetch --stdin "$uri"
+ if [ "${PIPESTATUS[0]}" -ne 0 -o "$?" -ne 0 ]; then
+ echo "unable to fetch tags (non-fatal)" >&2
+ fi
return 0
}
@@ -364,21 +367,15 @@ if [ "$packed_transport" ]; then
fetch_pack_recorder "refs/heads/$name" "fetching pack failed" ||
exit
- export _cg_taglist="$(mktemp -t gitfetch.XXXXXX)"
record_tags_to_fetch () {
- echo "refs/$1" >>"$_cg_taglist"
- return 17
- }
- fetch=record_tags_to_fetch
- fetch_tags
- if [ -s "$_cg_taglist" ]; then
- ( cat "$_cg_taglist" | tr '\n' '\0' |
+ ( cut -f 1 | tr '\n' '\0' |
xargs -0 git-fetch-pack $cloneorfetch "$uri" ||
echo "failed" "$rembranch" ) |
fetch_pack_recorder "" "unable to retrieve tags (non-fatal)"
- fi
- rm "$_cg_taglist"
+ }
+ fetch=record_tags_to_fetch
+ fetch_tags
rm "$dirtyfile"
show_changes_summary "$orig_head" "$(cg-object-id "$name")"
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Snow falling on Perl. White noise covering line noise.
Hides all the bugs too. -- J. Putnam
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 3/4] Teach git-local-fetch the --stdin switch
2006-07-27 21:56 ` [PATCH 3/4] Teach git-local-fetch the --stdin switch Petr Baudis
@ 2006-07-28 1:57 ` Petr Baudis
0 siblings, 0 replies; 7+ messages in thread
From: Petr Baudis @ 2006-07-28 1:57 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
Dear diary, on Thu, Jul 27, 2006 at 11:56:19PM CEST, I got a letter
where Petr Baudis <pasky@suse.cz> said that...
> +--stdin::
> + Instead of a commit id on the commandline (which is not expected in this
> + case), 'git-local-fetch' excepts lines on stdin in the format
^^^^^^^
Oops, should read "expects" in both patches - thanks, alp!
I blame all those tiny bugs lured into the room by light and then, well,
bugging you. (Real-world bugs, but they can transform.)
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Snow falling on Perl. White noise covering line noise.
Hides all the bugs too. -- J. Putnam
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2006-07-28 1:57 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2006-07-27 21:53 [PATCH 0/4] Fetching mass of objects at once Petr Baudis
2006-07-27 21:56 ` [PATCH 1/4] Make pull() take some implicit data as explicit arguments Petr Baudis
2006-07-27 21:56 ` [PATCH 2/4] Make pull() support fetching multiple targets at once Petr Baudis
2006-07-27 21:56 ` [PATCH 3/4] Teach git-local-fetch the --stdin switch Petr Baudis
2006-07-28 1:57 ` Petr Baudis
2006-07-27 21:56 ` [PATCH 4/4] Teach git-http-fetch " Petr Baudis
2006-07-27 21:57 ` [PATCH 0/4] Fetching mass of objects at once Petr Baudis
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).