All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: stolee@gmail.com, avarab@gmail.com, gitster@pobox.com,
	zhiyou.jx@alibaba-inc.com, jonathantanmy@google.com,
	Jeff Hostetler <git@jeffhostetler.com>,
	Derrick Stolee <derrickstolee@github.com>,
	Derrick Stolee <derrickstolee@github.com>
Subject: [PATCH v4 09/13] bundle: parse filter capability
Date: Wed, 09 Mar 2022 16:01:39 +0000	[thread overview]
Message-ID: <faf7a38b0e595e6fdb89b62563dcff855466c796.1646841704.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.1159.v4.git.1646841703.gitgitgadget@gmail.com>

From: Derrick Stolee <derrickstolee@github.com>

The v3 bundle format has capabilities, allowing newer versions of Git to
create bundles with newer features. Older versions that do not
understand these new capabilities will fail with a helpful warning.

Create a new capability allowing Git to understand that the contained
pack-file is filtered according to some object filter. Typically, this
filter will be "blob:none" for a blobless partial clone.

This change teaches Git to parse this capability, place its value in the
bundle header, and demonstrate this understanding by adding a message to
'git bundle verify'.

Since we will use gently_parse_list_objects_filter() outside of
list-objects-filter-options.c, make it an external method and move its
API documentation to before its declaration.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 Documentation/git-bundle.txt              |  7 +++++--
 Documentation/technical/bundle-format.txt | 11 ++++++++---
 bundle.c                                  | 15 ++++++++++++++-
 bundle.h                                  |  2 ++
 list-objects-filter-options.c             | 17 +----------------
 list-objects-filter-options.h             | 20 ++++++++++++++++++++
 6 files changed, 50 insertions(+), 22 deletions(-)

diff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt
index 72ab8139052..ac4c4352aae 100644
--- a/Documentation/git-bundle.txt
+++ b/Documentation/git-bundle.txt
@@ -75,8 +75,11 @@ verify <file>::
 	cleanly to the current repository.  This includes checks on the
 	bundle format itself as well as checking that the prerequisite
 	commits exist and are fully linked in the current repository.
-	'git bundle' prints a list of missing commits, if any, and exits
-	with a non-zero status.
+	Information about additional capabilities, such as "object filter",
+	is printed. See "Capabilities" in link:technical/bundle-format.html
+	for more information. Finally, 'git bundle' prints a list of
+	missing commits, if any. The exit code is zero for success, but
+	will be nonzero if the bundle file is invalid.
 
 list-heads <file>::
 	Lists the references defined in the bundle.  If followed by a
diff --git a/Documentation/technical/bundle-format.txt b/Documentation/technical/bundle-format.txt
index bac558d049a..b9be8644cf5 100644
--- a/Documentation/technical/bundle-format.txt
+++ b/Documentation/technical/bundle-format.txt
@@ -71,6 +71,11 @@ and the Git bundle v2 format cannot represent a shallow clone repository.
 == Capabilities
 
 Because there is no opportunity for negotiation, unknown capabilities cause 'git
-bundle' to abort.  The only known capability is `object-format`, which specifies
-the hash algorithm in use, and can take the same values as the
-`extensions.objectFormat` configuration value.
+bundle' to abort.
+
+* `object-format` specifies the hash algorithm in use, and can take the same
+  values as the `extensions.objectFormat` configuration value.
+
+* `filter` specifies an object filter as in the `--filter` option in
+  linkgit:git-rev-list[1]. The resulting pack-file must be marked as a
+  `.promisor` pack-file after it is unbundled.
diff --git a/bundle.c b/bundle.c
index 7ba60a573d7..41e75efab9a 100644
--- a/bundle.c
+++ b/bundle.c
@@ -11,7 +11,7 @@
 #include "run-command.h"
 #include "refs.h"
 #include "strvec.h"
-
+#include "list-objects-filter-options.h"
 
 static const char v2_bundle_signature[] = "# v2 git bundle\n";
 static const char v3_bundle_signature[] = "# v3 git bundle\n";
@@ -33,6 +33,7 @@ void bundle_header_release(struct bundle_header *header)
 {
 	string_list_clear(&header->prerequisites, 1);
 	string_list_clear(&header->references, 1);
+	list_objects_filter_release(&header->filter);
 }
 
 static int parse_capability(struct bundle_header *header, const char *capability)
@@ -45,6 +46,10 @@ static int parse_capability(struct bundle_header *header, const char *capability
 		header->hash_algo = &hash_algos[algo];
 		return 0;
 	}
+	if (skip_prefix(capability, "filter=", &arg)) {
+		parse_list_objects_filter(&header->filter, arg);
+		return 0;
+	}
 	return error(_("unknown capability '%s'"), capability);
 }
 
@@ -220,6 +225,8 @@ int verify_bundle(struct repository *r,
 	req_nr = revs.pending.nr;
 	setup_revisions(2, argv, &revs, NULL);
 
+	list_objects_filter_copy(&revs.filter, &header->filter);
+
 	if (prepare_revision_walk(&revs))
 		die(_("revision walk setup failed"));
 
@@ -259,6 +266,12 @@ int verify_bundle(struct repository *r,
 			     r->nr),
 			  r->nr);
 		list_refs(r, 0, NULL);
+
+		if (header->filter.choice) {
+			printf_ln("The bundle uses this filter: %s",
+				  list_objects_filter_spec(&header->filter));
+		}
+
 		r = &header->prerequisites;
 		if (!r->nr) {
 			printf_ln(_("The bundle records a complete history."));
diff --git a/bundle.h b/bundle.h
index 06009fe6b1f..7fef2108f43 100644
--- a/bundle.h
+++ b/bundle.h
@@ -4,12 +4,14 @@
 #include "strvec.h"
 #include "cache.h"
 #include "string-list.h"
+#include "list-objects-filter-options.h"
 
 struct bundle_header {
 	unsigned version;
 	struct string_list prerequisites;
 	struct string_list references;
 	const struct git_hash_algo *hash_algo;
+	struct list_objects_filter_options filter;
 };
 
 #define BUNDLE_HEADER_INIT \
diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c
index 449d53af69f..f02d8df1422 100644
--- a/list-objects-filter-options.c
+++ b/list-objects-filter-options.c
@@ -40,22 +40,7 @@ const char *list_object_filter_config_name(enum list_objects_filter_choice c)
 	BUG("list_object_filter_config_name: invalid argument '%d'", c);
 }
 
-/*
- * Parse value of the argument to the "filter" keyword.
- * On the command line this looks like:
- *       --filter=<arg>
- * and in the pack protocol as:
- *       "filter" SP <arg>
- *
- * The filter keyword will be used by many commands.
- * See Documentation/rev-list-options.txt for allowed values for <arg>.
- *
- * Capture the given arg as the "filter_spec".  This can be forwarded to
- * subordinate commands when necessary (although it's better to pass it through
- * expand_list_objects_filter_spec() first).  We also "intern" the arg for the
- * convenience of the current command.
- */
-static int gently_parse_list_objects_filter(
+int gently_parse_list_objects_filter(
 	struct list_objects_filter_options *filter_options,
 	const char *arg,
 	struct strbuf *errbuf)
diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h
index 425c38cae9d..2eb6c983949 100644
--- a/list-objects-filter-options.h
+++ b/list-objects-filter-options.h
@@ -72,6 +72,26 @@ struct list_objects_filter_options {
 /* Normalized command line arguments */
 #define CL_ARG__FILTER "filter"
 
+/*
+ * Parse value of the argument to the "filter" keyword.
+ * On the command line this looks like:
+ *       --filter=<arg>
+ * and in the pack protocol as:
+ *       "filter" SP <arg>
+ *
+ * The filter keyword will be used by many commands.
+ * See Documentation/rev-list-options.txt for allowed values for <arg>.
+ *
+ * Capture the given arg as the "filter_spec".  This can be forwarded to
+ * subordinate commands when necessary (although it's better to pass it through
+ * expand_list_objects_filter_spec() first).  We also "intern" the arg for the
+ * convenience of the current command.
+ */
+int gently_parse_list_objects_filter(
+	struct list_objects_filter_options *filter_options,
+	const char *arg,
+	struct strbuf *errbuf);
+
 void list_objects_filter_die_if_populated(
 	struct list_objects_filter_options *filter_options);
 
-- 
gitgitgadget


  parent reply	other threads:[~2022-03-09 16:02 UTC|newest]

Thread overview: 114+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-02-23 17:55 [PATCH 00/11] Partial bundles Derrick Stolee via GitGitGadget
2022-02-23 17:55 ` [PATCH 01/11] index-pack: document and test the --promisor option Derrick Stolee via GitGitGadget
2022-02-23 17:55 ` [PATCH 02/11] revision: put object filter into struct rev_info Derrick Stolee via GitGitGadget
2022-03-04 22:15   ` Junio C Hamano
2022-03-07 13:59     ` Derrick Stolee
2022-03-07 16:46       ` Junio C Hamano
2022-02-23 17:55 ` [PATCH 03/11] pack-objects: use rev.filter when possible Derrick Stolee via GitGitGadget
2022-03-04 22:25   ` Junio C Hamano
2022-02-23 17:55 ` [PATCH 04/11] pack-bitmap: drop filter in prepare_bitmap_walk() Derrick Stolee via GitGitGadget
2022-03-04 22:26   ` Junio C Hamano
2022-02-23 17:55 ` [PATCH 05/11] list-objects: consolidate traverse_commit_list[_filtered] Derrick Stolee via GitGitGadget
2022-03-04 22:30   ` Junio C Hamano
2022-02-23 17:55 ` [PATCH 06/11] MyFirstObjectWalk: update recommended usage Derrick Stolee via GitGitGadget
2022-03-04 22:33   ` Junio C Hamano
2022-03-07 14:05     ` Derrick Stolee
2022-03-07 16:47       ` Junio C Hamano
2022-02-23 17:55 ` [PATCH 07/11] bundle: safely handle --objects option Derrick Stolee via GitGitGadget
2022-02-28 16:00   ` Jeff Hostetler
2022-03-04 22:58     ` Junio C Hamano
2022-03-07 14:09       ` Derrick Stolee
2022-03-04 22:57   ` Junio C Hamano
2022-03-07 15:35   ` Ævar Arnfjörð Bjarmason
2022-02-23 17:55 ` [PATCH 08/11] bundle: parse filter capability Derrick Stolee via GitGitGadget
2022-03-07 15:38   ` Ævar Arnfjörð Bjarmason
2022-03-07 16:14     ` Derrick Stolee
2022-03-07 16:22       ` Ævar Arnfjörð Bjarmason
2022-03-07 16:29         ` Derrick Stolee
2022-03-07 15:55   ` Ævar Arnfjörð Bjarmason
2022-02-23 17:55 ` [PATCH 09/11] rev-list: move --filter parsing into revision.c Derrick Stolee via GitGitGadget
2022-02-23 17:55 ` [PATCH 10/11] bundle: create filtered bundles Derrick Stolee via GitGitGadget
2022-03-04 23:35   ` Junio C Hamano
2022-03-07 14:14     ` Derrick Stolee
2022-03-07 16:49       ` Junio C Hamano
2022-03-07 15:44   ` Ævar Arnfjörð Bjarmason
2022-02-23 17:55 ` [PATCH 11/11] bundle: unbundle promisor packs Derrick Stolee via GitGitGadget
2022-03-04 23:43   ` Junio C Hamano
2022-03-07 14:48     ` Derrick Stolee
2022-03-07 16:56       ` Junio C Hamano
2022-03-07 18:57         ` Derrick Stolee
2022-03-07 19:40           ` Junio C Hamano
2022-03-07 19:49             ` Derrick Stolee
2022-03-07 19:54               ` Junio C Hamano
2022-03-07 20:20                 ` Derrick Stolee
2022-03-07 21:35                   ` Junio C Hamano
2022-03-07 15:47   ` Ævar Arnfjörð Bjarmason
2022-03-07 16:10     ` Derrick Stolee
2022-02-28 17:00 ` [PATCH 00/11] Partial bundles Jeff Hostetler
2022-02-28 17:54   ` Derrick Stolee
2022-03-01 18:03     ` Jeff Hostetler
2022-03-04 19:19 ` Derrick Stolee
2022-03-07 14:55 ` Ævar Arnfjörð Bjarmason
2022-03-07 14:59   ` Derrick Stolee
2022-03-07 21:50 ` [PATCH v2 00/12] " Derrick Stolee via GitGitGadget
2022-03-07 21:50   ` [PATCH v2 01/12] index-pack: document and test the --promisor option Derrick Stolee via GitGitGadget
2022-03-07 21:50   ` [PATCH v2 02/12] revision: put object filter into struct rev_info Derrick Stolee via GitGitGadget
2022-03-07 21:50   ` [PATCH v2 03/12] pack-objects: use rev.filter when possible Derrick Stolee via GitGitGadget
2022-03-07 21:50   ` [PATCH v2 04/12] pack-bitmap: drop filter in prepare_bitmap_walk() Derrick Stolee via GitGitGadget
2022-03-07 21:50   ` [PATCH v2 05/12] list-objects: consolidate traverse_commit_list[_filtered] Derrick Stolee via GitGitGadget
2022-03-07 21:50   ` [PATCH v2 06/12] MyFirstObjectWalk: update recommended usage Derrick Stolee via GitGitGadget
2022-03-07 21:50   ` [PATCH v2 07/12] bundle: safely handle --objects option Derrick Stolee via GitGitGadget
2022-03-08  9:37     ` Ævar Arnfjörð Bjarmason
2022-03-08 13:45       ` Derrick Stolee
2022-03-08 13:53         ` Ævar Arnfjörð Bjarmason
2022-03-07 21:50   ` [PATCH v2 08/12] bundle: parse filter capability Derrick Stolee via GitGitGadget
2022-03-08  9:25     ` Ævar Arnfjörð Bjarmason
2022-03-08 13:43       ` Derrick Stolee
2022-03-07 21:50   ` [PATCH v2 09/12] rev-list: move --filter parsing into revision.c Derrick Stolee via GitGitGadget
2022-03-07 21:50   ` [PATCH v2 10/12] bundle: create filtered bundles Derrick Stolee via GitGitGadget
2022-03-07 21:50   ` [PATCH v2 11/12] bundle: unbundle promisor packs Derrick Stolee via GitGitGadget
2022-03-07 21:50   ` [PATCH v2 12/12] clone: fail gracefully when cloning filtered bundle Derrick Stolee via GitGitGadget
2022-03-07 22:11   ` [PATCH v2 00/12] Partial bundles Junio C Hamano
2022-03-08 14:39   ` [PATCH v3 " Derrick Stolee via GitGitGadget
2022-03-08 14:39     ` [PATCH v3 01/12] index-pack: document and test the --promisor option Derrick Stolee via GitGitGadget
2022-03-08 14:39     ` [PATCH v3 02/12] revision: put object filter into struct rev_info Derrick Stolee via GitGitGadget
2022-03-08 14:39     ` [PATCH v3 03/12] pack-objects: use rev.filter when possible Derrick Stolee via GitGitGadget
2022-03-08 14:39     ` [PATCH v3 04/12] pack-bitmap: drop filter in prepare_bitmap_walk() Derrick Stolee via GitGitGadget
2022-03-08 14:39     ` [PATCH v3 05/12] list-objects: consolidate traverse_commit_list[_filtered] Derrick Stolee via GitGitGadget
2022-03-09 13:24       ` Ævar Arnfjörð Bjarmason
2022-03-08 14:39     ` [PATCH v3 06/12] MyFirstObjectWalk: update recommended usage Derrick Stolee via GitGitGadget
2022-03-08 14:39     ` [PATCH v3 07/12] list-objects: handle NULL function pointers Ævar Arnfjörð Bjarmason via GitGitGadget
2022-03-08 17:26       ` Junio C Hamano
2022-03-09 13:40         ` Ævar Arnfjörð Bjarmason
2022-03-09 14:16           ` Derrick Stolee
2022-03-09 18:32           ` Junio C Hamano
2022-03-08 14:39     ` [PATCH v3 08/12] bundle: parse filter capability Derrick Stolee via GitGitGadget
2022-03-08 17:29       ` Junio C Hamano
2022-03-09 14:35         ` Derrick Stolee
2022-03-09 13:30       ` Ævar Arnfjörð Bjarmason
2022-03-08 14:39     ` [PATCH v3 09/12] rev-list: move --filter parsing into revision.c Derrick Stolee via GitGitGadget
2022-03-08 14:39     ` [PATCH v3 10/12] bundle: create filtered bundles Derrick Stolee via GitGitGadget
2022-03-08 14:39     ` [PATCH v3 11/12] bundle: unbundle promisor packs Derrick Stolee via GitGitGadget
2022-03-08 14:39     ` [PATCH v3 12/12] clone: fail gracefully when cloning filtered bundle Derrick Stolee via GitGitGadget
2022-03-08 16:10       ` Derrick Stolee
2022-03-08 17:19         ` Junio C Hamano
2022-03-09 16:01     ` [PATCH v4 00/13] Partial bundles Derrick Stolee via GitGitGadget
2022-03-09 16:01       ` [PATCH v4 01/13] index-pack: document and test the --promisor option Derrick Stolee via GitGitGadget
2022-03-09 16:01       ` [PATCH v4 02/13] list-objects-filter-options: create copy helper Derrick Stolee via GitGitGadget
2022-03-09 16:01       ` [PATCH v4 03/13] revision: put object filter into struct rev_info Derrick Stolee via GitGitGadget
2022-03-09 18:48         ` Junio C Hamano
2022-03-09 16:01       ` [PATCH v4 04/13] pack-objects: use rev.filter when possible Derrick Stolee via GitGitGadget
2022-03-10 13:11         ` Ævar Arnfjörð Bjarmason
2022-03-10 13:33           ` Derrick Stolee
2022-03-10 14:24             ` Ævar Arnfjörð Bjarmason
2022-03-09 16:01       ` [PATCH v4 05/13] pack-bitmap: drop filter in prepare_bitmap_walk() Derrick Stolee via GitGitGadget
2022-03-09 16:01       ` [PATCH v4 06/13] list-objects: consolidate traverse_commit_list[_filtered] Derrick Stolee via GitGitGadget
2022-03-09 16:01       ` [PATCH v4 07/13] MyFirstObjectWalk: update recommended usage Derrick Stolee via GitGitGadget
2022-03-09 16:01       ` [PATCH v4 08/13] list-objects: handle NULL function pointers Ævar Arnfjörð Bjarmason via GitGitGadget
2022-03-09 16:01       ` Derrick Stolee via GitGitGadget [this message]
2022-03-09 18:41         ` [PATCH v4 09/13] bundle: parse filter capability Junio C Hamano
2022-03-09 18:55           ` Derrick Stolee
2022-03-09 16:01       ` [PATCH v4 10/13] rev-list: move --filter parsing into revision.c Derrick Stolee via GitGitGadget
2022-03-09 16:01       ` [PATCH v4 11/13] bundle: create filtered bundles Derrick Stolee via GitGitGadget
2022-03-09 16:01       ` [PATCH v4 12/13] bundle: unbundle promisor packs Derrick Stolee via GitGitGadget
2022-03-09 16:01       ` [PATCH v4 13/13] clone: fail gracefully when cloning filtered bundle Derrick Stolee via GitGitGadget

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=faf7a38b0e595e6fdb89b62563dcff855466c796.1646841704.git.gitgitgadget@gmail.com \
    --to=gitgitgadget@gmail.com \
    --cc=avarab@gmail.com \
    --cc=derrickstolee@github.com \
    --cc=git@jeffhostetler.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jonathantanmy@google.com \
    --cc=stolee@gmail.com \
    --cc=zhiyou.jx@alibaba-inc.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.