All of lore.kernel.org
 help / color / mirror / Atom feed
From: Michael Haggerty <mhagger@alum.mit.edu>
To: Junio C Hamano <gitster@pobox.com>
Cc: Stefan Beller <sbeller@google.com>, Jeff King <peff@peff.net>,
	git@vger.kernel.org, Michael Haggerty <mhagger@alum.mit.edu>
Subject: [PATCH v3 08/19] prune_refs(): use delete_refs()
Date: Mon, 22 Jun 2015 16:02:59 +0200	[thread overview]
Message-ID: <a338a63f4c3225c7faa001a5bdde92c7690eba79.1434980615.git.mhagger@alum.mit.edu> (raw)
In-Reply-To: <cover.1434980615.git.mhagger@alum.mit.edu>

The old version just looped over the references to delete, calling
delete_ref() on each one. But that has quadratic behavior, because
each call to delete_ref() might have to rewrite the packed-refs file.
This can be very expensive in a repository with a large number of
references. In some (admittedly extreme) repositories, we've seen
cases where the ref-pruning part of fetch takes multiple tens of
minutes.

Instead call delete_refs(), which (aside from being less code) has the
optimization that it only rewrites the packed-refs file a single time.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 builtin/fetch.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 8d5b2db..34b6f5e 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -790,20 +790,29 @@ static int prune_refs(struct refspec *refs, int ref_count, struct ref *ref_map,
 	if (4 < i && !strncmp(".git", url + i - 3, 4))
 		url_len = i - 3;
 
-	for (ref = stale_refs; ref; ref = ref->next) {
-		if (!dry_run)
-			result |= delete_ref(ref->name, NULL, 0);
-		if (verbosity >= 0 && !shown_url) {
-			fprintf(stderr, _("From %.*s\n"), url_len, url);
-			shown_url = 1;
-		}
-		if (verbosity >= 0) {
+	if (!dry_run) {
+		struct string_list refnames = STRING_LIST_INIT_NODUP;
+
+		for (ref = stale_refs; ref; ref = ref->next)
+			string_list_append(&refnames, ref->name);
+
+		result = delete_refs(&refnames);
+		string_list_clear(&refnames, 0);
+	}
+
+	if (verbosity >= 0) {
+		for (ref = stale_refs; ref; ref = ref->next) {
+			if (!shown_url) {
+				fprintf(stderr, _("From %.*s\n"), url_len, url);
+				shown_url = 1;
+			}
 			fprintf(stderr, " x %-*s %-*s -> %s\n",
 				TRANSPORT_SUMMARY(_("[deleted]")),
 				REFCOL_WIDTH, _("(none)"), prettify_refname(ref->name));
 			warn_dangling_symref(stderr, dangling_msg, ref->name);
 		}
 	}
+
 	free(url);
 	free_refs(stale_refs);
 	return result;
-- 
2.1.4

  parent reply	other threads:[~2015-06-22 14:04 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-06-22 14:02 [PATCH v3 00/19] Improve "refs" encapsulation and speed up deletes Michael Haggerty
2015-06-22 14:02 ` [PATCH v3 01/19] delete_ref(): move declaration to refs.h Michael Haggerty
2015-06-22 14:02 ` [PATCH v3 02/19] remove_branches(): remove temporary Michael Haggerty
2015-06-22 14:02 ` [PATCH v3 03/19] delete_ref(): handle special case more explicitly Michael Haggerty
2015-06-22 14:02 ` [PATCH v3 04/19] delete_refs(): new function for the refs API Michael Haggerty
2015-06-22 14:02 ` [PATCH v3 05/19] delete_refs(): make error message more generic Michael Haggerty
2015-06-22 14:02 ` [PATCH v3 06/19] delete_refs(): bail early if the packed-refs file cannot be rewritten Michael Haggerty
2015-06-22 14:02 ` [PATCH v3 07/19] prune_remote(): use delete_refs() Michael Haggerty
2015-06-22 14:02 ` Michael Haggerty [this message]
2015-06-22 14:03 ` [PATCH v3 09/19] repack_without_refs(): make function private Michael Haggerty
2015-06-22 14:03 ` [PATCH v3 10/19] initial_ref_transaction_commit(): function for initial ref creation Michael Haggerty
2015-06-22 14:03 ` [PATCH v3 11/19] refs: remove some functions from the module's public interface Michael Haggerty
2015-06-22 14:03 ` [PATCH v3 12/19] initial_ref_transaction_commit(): check for duplicate refs Michael Haggerty
2015-06-22 21:06   ` Junio C Hamano
2015-06-23  7:11     ` Michael Haggerty
2015-06-23 17:44       ` Junio C Hamano
2015-06-22 14:03 ` [PATCH v3 13/19] initial_ref_transaction_commit(): check for ref D/F conflicts Michael Haggerty
2015-06-22 14:03 ` [PATCH v3 14/19] refs: move the remaining ref module declarations to refs.h Michael Haggerty
2015-06-22 14:03 ` [PATCH v3 15/19] refs.h: add some parameter names to function declarations Michael Haggerty
2015-06-22 14:03 ` [PATCH v3 16/19] check_branch_commit(): make first parameter const Michael Haggerty
2015-06-22 14:03 ` [PATCH v3 17/19] update_ref(): don't read old reference value before delete Michael Haggerty
2015-06-22 14:03 ` [PATCH v3 18/19] cmd_update_ref(): make logic more straightforward Michael Haggerty
2015-06-22 14:03 ` [PATCH v3 19/19] delete_ref(): use the usual convention for old_sha1 Michael Haggerty
2015-06-22 21:10   ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a338a63f4c3225c7faa001a5bdde92c7690eba79.1434980615.git.mhagger@alum.mit.edu \
    --to=mhagger@alum.mit.edu \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=peff@peff.net \
    --cc=sbeller@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.