All of lore.kernel.org
 help / color / mirror / Atom feed
From: Xin Li <delphij@google.com>
To: git@vger.kernel.org
Cc: Xin Li <delphij@google.com>
Subject: [PATCH v2 1/1] fetch: allow adding a filter after initial clone.
Date: Wed, 27 May 2020 19:54:00 -0700	[thread overview]
Message-ID: <20200528025359.20931-2-delphij@google.com> (raw)
In-Reply-To: <20200513234435.GG6605@camp.crustytoothpaste.net>

Retroactively adding filter can be useful for existing shallow clones as
they allow users to see earlier change histories without downloading all
git objects in a regular --unshallow fetch.

Previously this is possible by manually amending the repository
configuration to make git think there is an existing promisor. Because
the code already does most of the hard work, it's safer for git to
just perform the configuration change automatically instead.

Instead of bailing out immediately when no promisor is available, make
the code check more specific issue (extension became special in
repository version 1, while it can have any value in version 0, so
upgrade should not happen if the repository have an unsupported
configuration that would render it invalid if we upgraded).

Signed-off-by: Xin Li <delphij@google.com>
---
 builtin/fetch.c               |  3 ---
 list-objects-filter-options.c |  3 ++-
 repository.h                  |  6 ++++++
 setup.c                       | 30 ++++++++++++++++++++++++++++++
 t/t0410-partial-clone.sh      | 21 +++++++++++++++++++++
 5 files changed, 59 insertions(+), 4 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index b5788c16bf..3347d578ea 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1790,9 +1790,6 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 	if (depth || deepen_since || deepen_not.nr)
 		deepen = 1;
 
-	if (filter_options.choice && !has_promisor_remote())
-		die("--filter can only be used when extensions.partialClone is set");
-
 	if (all) {
 		if (argc == 1)
 			die(_("fetch --all does not take a repository argument"));
diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c
index 256bcfbdfe..6d62b60eac 100644
--- a/list-objects-filter-options.c
+++ b/list-objects-filter-options.c
@@ -326,7 +326,8 @@ void partial_clone_register(
 
 	/* Check if it is already registered */
 	if (!promisor_remote_find(remote)) {
-		git_config_set("core.repositoryformatversion", "1");
+		if (upgrade_repository_format(the_repository, 1) < 0)
+			die(_("Unable to upgrade repository format to support partial clone"));
 
 		/* Add promisor config for the remote */
 		cfg_name = xstrfmt("remote.%s.promisor", remote);
diff --git a/repository.h b/repository.h
index 6534fbb7b3..f301f6f456 100644
--- a/repository.h
+++ b/repository.h
@@ -196,4 +196,10 @@ void repo_update_index_if_able(struct repository *, struct lock_file *);
 
 void prepare_repo_settings(struct repository *r);
 
+/*
+ * Return 1 if upgrade repository format to target_version succeeded,
+ * 0 if no upgrade is necessary.
+ */
+int upgrade_repository_format(struct repository *r, int target_version);
+
 #endif /* REPOSITORY_H */
diff --git a/setup.c b/setup.c
index 65fe5ecefb..84da976e07 100644
--- a/setup.c
+++ b/setup.c
@@ -538,6 +538,36 @@ static int check_repository_format_gently(const char *gitdir, struct repository_
 	return 0;
 }
 
+int upgrade_repository_format(struct repository *r, int target_version)
+{
+	const char *gitdir = get_git_dir();
+	struct strbuf sb = STRBUF_INIT;
+	struct strbuf err = STRBUF_INIT;
+	struct strbuf repo_version = STRBUF_INIT;
+	struct repository_format repo_fmt = REPOSITORY_FORMAT_INIT;
+
+	strbuf_git_common_path(&sb, r, "/config");
+	read_repository_format(&repo_fmt, sb.buf);
+	strbuf_release(&sb);
+
+	if (repo_fmt.version >= target_version)
+		return 0;
+
+	repo_fmt.version = target_version;
+
+	if (verify_repository_format(&repo_fmt, &err) < 0) {
+		warning("Unable to upgrade repository format to %d: %s",
+		    target_version, err.buf);
+		strbuf_release(&err);
+		return -1;
+	}
+
+	strbuf_addf(&repo_version, "%d", target_version);
+	git_config_set("core.repositoryformatversion", repo_version.buf);
+	strbuf_release(&repo_version);
+	return 1;
+}
+
 static void init_repository_format(struct repository_format *format)
 {
 	const struct repository_format fresh = REPOSITORY_FORMAT_INIT;
diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh
index a3988bd4b8..71270d3a53 100755
--- a/t/t0410-partial-clone.sh
+++ b/t/t0410-partial-clone.sh
@@ -30,6 +30,27 @@ test_expect_success 'extensions.partialclone without filter' '
 	git -C client fetch origin
 '
 
+test_expect_success 'convert shallow clone to partial clone' '
+	rm -fr server client &&
+	test_create_repo server &&
+	test_commit -C server my_commit 1 &&
+	test_commit -C server my_commit2 1 &&
+	git clone --depth=1 "file://$(pwd)/server" client &&
+	git -C client fetch --unshallow --filter="blob:none" &&
+	test_cmp_config -C client true remote.origin.promisor &&
+	test_cmp_config -C client blob:none remote.origin.partialclonefilter &&
+	test_cmp_config -C client 1 core.repositoryformatversion
+'
+test_expect_success 'convert shallow clone to partial clone must fail with invalid extension' '
+	rm -fr server client &&
+	test_create_repo server &&
+	test_commit -C server my_commit 1 &&
+	test_commit -C server my_commit2 1 &&
+	git clone --depth=1 "file://$(pwd)/server" client &&
+	git -C client config extensions.sandwidth true &&
+	test_must_fail git -C client fetch --unshallow --filter="blob:none"
+'
+
 test_expect_success 'missing reflog object, but promised by a commit, passes fsck' '
 	rm -rf repo &&
 	test_create_repo repo &&
-- 
2.27.0.rc0.183.gde8f92d652-goog


  parent reply	other threads:[~2020-05-28  2:57 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-13 20:00 [PATCH] fetch: allow adding a filter after initial clone Xin Li
2020-05-13 20:43 ` Junio C Hamano
2020-05-13 21:41   ` Xin Li
2020-05-13 22:07     ` Junio C Hamano
2020-05-13 22:18     ` Junio C Hamano
2020-05-13 23:44 ` brian m. carlson
2020-05-28  2:53   ` [PATCH v2 0/1] " Xin Li
2020-05-28  2:54   ` Xin Li [this message]
2020-05-28  3:28     ` [PATCH v2 1/1] " Jonathan Nieder
2020-05-28  4:08       ` [PATCH v3] " Xin Li
2020-05-28 15:04     ` [PATCH v2 1/1] " Junio C Hamano
2020-05-28 17:19       ` Jonathan Nieder
2020-05-28 19:12         ` Xin Li
2020-05-28 19:17           ` Jonathan Nieder
2020-05-29  0:04             ` [PATCH v4] " Xin Li
2020-05-29  0:41               ` Junio C Hamano
2020-05-29 18:00                 ` Junio C Hamano
2020-05-29  1:01               ` Jonathan Nieder
2020-05-29  6:44                 ` [PATCH v5] " Xin Li
2020-05-29  6:54                 ` [PATCH v4] " Xin Li
2020-05-29 18:06                 ` Junio C Hamano
2020-06-05  9:10                   ` [PATCH v6 0/4] " Xin Li
2020-06-05  9:10                     ` [PATCH v6 1/4] repository: add a helper function to perform repository format upgrade Xin Li
2020-06-05 19:12                       ` Junio C Hamano
2020-06-05  9:10                     ` [PATCH v6 2/4] fetch: allow adding a filter after initial clone Xin Li
2020-06-05 19:15                       ` Junio C Hamano
2020-06-05  9:10                     ` [PATCH v6 3/4] sparse-checkout: upgrade repository to version 1 when enabling extension Xin Li
2020-06-05 19:21                       ` Junio C Hamano
2020-06-05  9:10                     ` [PATCH v6 4/4] check_repository_format_gently(): refuse extensions for old repositories Xin Li
2020-06-08 16:59                       ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200528025359.20931-2-delphij@google.com \
    --to=delphij@google.com \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.