All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: Patrick Steinhardt <ps@pks.im>, Dirk Gouders <dirk@gouders.net>,
	Johannes Schindelin <johannes.schindelin@gmx.de>,
	Johannes Schindelin <johannes.schindelin@gmx.de>
Subject: [PATCH v4 04/11] commit-reach(paint_down_to_common): prepare for handling shallow commits
Date: Wed, 28 Feb 2024 09:44:10 +0000	[thread overview]
Message-ID: <837aa5a89c640427a5de064da93f1de4934d8212.1709113458.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.1657.v4.git.1709113457.gitgitgadget@gmail.com>

From: Johannes Schindelin <johannes.schindelin@gmx.de>

When `git fetch --update-shallow` needs to test for commit ancestry, it
can naturally run into a missing object (e.g. if it is a parent of a
shallow commit). For the purpose of `--update-shallow`, this needs to be
treated as if the child commit did not even have that parent, i.e. the
commit history needs to be clamped.

For all other scenarios, clamping the commit history is actually a bug,
as it would hide repository corruption (for an analysis regarding
shallow and partial clones, see the analysis further down).

Add a flag to optionally ask the function to ignore missing commits, as
`--update-shallow` needs it to, while detecting missing objects as a
repository corruption error by default.

This flag is needed, and cannot replaced by `is_repository_shallow()` to
indicate that situation, because that function would return 0 in the
`--update-shallow` scenario: There is not actually a `shallow` file in
that scenario, as demonstrated e.g. by t5537.10 ("add new shallow root
with receive.updateshallow on") and t5538.4 ("add new shallow root with
receive.updateshallow on").

Note: shallow commits' parents are set to `NULL` internally already,
therefore there is no need to special-case shallow repositories here, as
the merge-base logic will not try to access parent commits of shallow
commits.

Likewise, partial clones aren't an issue either: If a commit is missing
during the revision walk in the merge-base logic, it is fetched via
`promisor_remote_get_direct()`. And not only the single missing commit
object: Due to the way the "promised" objects are fetched (in
`fetch_objects()` in `promisor-remote.c`, using `fetch
--filter=blob:none`), there is no actual way to fetch a single commit
object, as the remote side will pass that commit OID to `pack-objects
--revs [...]` which in turn passes it to `rev-list` which interprets
this as a commit _range_ instead of a single object. Therefore, in
partial clones (unless they are shallow in addition), all commits
reachable from a commit that is in the local object database are also
present in that local database.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 commit-reach.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/commit-reach.c b/commit-reach.c
index 5ff71d72d51..7112b10eeea 100644
--- a/commit-reach.c
+++ b/commit-reach.c
@@ -53,7 +53,8 @@ static int queue_has_nonstale(struct prio_queue *queue)
 static struct commit_list *paint_down_to_common(struct repository *r,
 						struct commit *one, int n,
 						struct commit **twos,
-						timestamp_t min_generation)
+						timestamp_t min_generation,
+						int ignore_missing_commits)
 {
 	struct prio_queue queue = { compare_commits_by_gen_then_commit_date };
 	struct commit_list *result = NULL;
@@ -108,6 +109,13 @@ static struct commit_list *paint_down_to_common(struct repository *r,
 			if (repo_parse_commit(r, p)) {
 				clear_prio_queue(&queue);
 				free_commit_list(result);
+				/*
+				 * At this stage, we know that the commit is
+				 * missing: `repo_parse_commit()` uses
+				 * `OBJECT_INFO_DIE_IF_CORRUPT` and therefore
+				 * corrupt commits would already have been
+				 * dispatched with a `die()`.
+				 */
 				return NULL;
 			}
 			p->object.flags |= flags;
@@ -143,7 +151,7 @@ static struct commit_list *merge_bases_many(struct repository *r,
 			return NULL;
 	}
 
-	list = paint_down_to_common(r, one, n, twos, 0);
+	list = paint_down_to_common(r, one, n, twos, 0, 0);
 
 	while (list) {
 		struct commit *commit = pop_commit(&list);
@@ -214,7 +222,7 @@ static int remove_redundant_no_gen(struct repository *r,
 				min_generation = curr_generation;
 		}
 		common = paint_down_to_common(r, array[i], filled,
-					      work, min_generation);
+					      work, min_generation, 0);
 		if (array[i]->object.flags & PARENT2)
 			redundant[i] = 1;
 		for (j = 0; j < filled; j++)
@@ -504,7 +512,7 @@ int repo_in_merge_bases_many(struct repository *r, struct commit *commit,
 
 	bases = paint_down_to_common(r, commit,
 				     nr_reference, reference,
-				     generation);
+				     generation, ignore_missing_commits);
 	if (commit->object.flags & PARENT2)
 		ret = 1;
 	clear_commit_marks(commit, all_flags);
-- 
gitgitgadget


  parent reply	other threads:[~2024-02-28  9:44 UTC|newest]

Thread overview: 70+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-02-13  8:41 [PATCH 00/12] The merge-base logic vs missing commit objects Johannes Schindelin via GitGitGadget
2024-02-13  8:41 ` [PATCH 01/12] paint_down_to_common: plug a memory leak Johannes Schindelin via GitGitGadget
2024-02-13  8:41 ` [PATCH 02/12] Let `repo_in_merge_bases()` report missing commits Johannes Schindelin via GitGitGadget
2024-02-15  9:33   ` Patrick Steinhardt
2024-02-13  8:41 ` [PATCH 03/12] Prepare `repo_in_merge_bases_many()` to optionally expect " Johannes Schindelin via GitGitGadget
2024-02-13  8:41 ` [PATCH 04/12] Prepare `paint_down_to_common()` for handling shallow commits Johannes Schindelin via GitGitGadget
2024-02-13  8:41 ` [PATCH 05/12] commit-reach: start reporting errors in `paint_down_to_common()` Johannes Schindelin via GitGitGadget
2024-02-13  8:41 ` [PATCH 06/12] merge_bases_many(): pass on errors from `paint_down_to_common()` Johannes Schindelin via GitGitGadget
2024-02-13  8:41 ` [PATCH 07/12] get_merge_bases_many_0(): pass on errors from `merge_bases_many()` Johannes Schindelin via GitGitGadget
2024-02-13  8:41 ` [PATCH 08/12] repo_get_merge_bases(): " Johannes Schindelin via GitGitGadget
2024-02-13  8:41 ` [PATCH 09/12] get_octopus_merge_bases(): " Johannes Schindelin via GitGitGadget
2024-02-13  8:41 ` [PATCH 10/12] repo_get_merge_bases_many(): " Johannes Schindelin via GitGitGadget
2024-02-13  8:41 ` [PATCH 11/12] repo_get_merge_bases_many_dirty(): " Johannes Schindelin via GitGitGadget
2024-02-13  8:41 ` [PATCH 12/12] paint_down_to_common(): special-case shallow/partial clones Johannes Schindelin via GitGitGadget
2024-02-13 18:37 ` [PATCH 00/12] The merge-base logic vs missing commit objects Junio C Hamano
2024-02-22 13:21 ` [PATCH v2 00/11] " Johannes Schindelin via GitGitGadget
2024-02-22 13:21   ` [PATCH v2 01/11] paint_down_to_common: plug two memory leaks Johannes Schindelin via GitGitGadget
2024-02-22 13:21   ` [PATCH v2 02/11] Prepare `repo_in_merge_bases_many()` to optionally expect missing commits Johannes Schindelin via GitGitGadget
2024-02-22 13:21   ` [PATCH v2 03/11] Start reporting missing commits in `repo_in_merge_bases_many()` Johannes Schindelin via GitGitGadget
2024-02-24  0:33     ` Junio C Hamano
2024-02-26  9:34       ` Johannes Schindelin
2024-02-26 20:01         ` Junio C Hamano
2024-03-01  6:56     ` Jeff King
2024-02-22 13:21   ` [PATCH v2 04/11] Prepare `paint_down_to_common()` for handling shallow commits Johannes Schindelin via GitGitGadget
2024-02-22 13:21   ` [PATCH v2 05/11] commit-reach: start reporting errors in `paint_down_to_common()` Johannes Schindelin via GitGitGadget
2024-02-22 13:21   ` [PATCH v2 06/11] merge_bases_many(): pass on errors from `paint_down_to_common()` Johannes Schindelin via GitGitGadget
2024-02-22 13:21   ` [PATCH v2 07/11] get_merge_bases_many_0(): pass on errors from `merge_bases_many()` Johannes Schindelin via GitGitGadget
2024-02-22 13:21   ` [PATCH v2 08/11] repo_get_merge_bases(): " Johannes Schindelin via GitGitGadget
2024-02-22 13:21   ` [PATCH v2 09/11] get_octopus_merge_bases(): " Johannes Schindelin via GitGitGadget
2024-02-22 13:21   ` [PATCH v2 10/11] repo_get_merge_bases_many(): " Johannes Schindelin via GitGitGadget
2024-02-22 13:21   ` [PATCH v2 11/11] repo_get_merge_bases_many_dirty(): " Johannes Schindelin via GitGitGadget
2024-02-27 13:28   ` [PATCH v3 00/11] The merge-base logic vs missing commit objects Johannes Schindelin via GitGitGadget
2024-02-27 13:28     ` [PATCH v3 01/11] paint_down_to_common: plug two memory leaks Johannes Schindelin via GitGitGadget
2024-02-27 13:28     ` [PATCH v3 02/11] Prepare `repo_in_merge_bases_many()` to optionally expect missing commits Johannes Schindelin via GitGitGadget
2024-02-27 13:28     ` [PATCH v3 03/11] Start reporting missing commits in `repo_in_merge_bases_many()` Johannes Schindelin via GitGitGadget
2024-02-27 13:28     ` [PATCH v3 04/11] Prepare `paint_down_to_common()` for handling shallow commits Johannes Schindelin via GitGitGadget
2024-02-27 14:46       ` Dirk Gouders
2024-02-27 15:00         ` Johannes Schindelin
2024-02-27 18:08           ` Junio C Hamano
2024-02-27 18:10             ` Junio C Hamano
2024-02-27 19:07             ` Dirk Gouders
2024-02-27 13:28     ` [PATCH v3 05/11] commit-reach: start reporting errors in `paint_down_to_common()` Johannes Schindelin via GitGitGadget
2024-02-27 14:56       ` Dirk Gouders
2024-02-27 15:08         ` Johannes Schindelin
2024-02-27 18:24           ` Junio C Hamano
2024-02-27 13:28     ` [PATCH v3 06/11] merge_bases_many(): pass on errors from `paint_down_to_common()` Johannes Schindelin via GitGitGadget
2024-02-27 18:29       ` Junio C Hamano
2024-02-27 13:28     ` [PATCH v3 07/11] get_merge_bases_many_0(): pass on errors from `merge_bases_many()` Johannes Schindelin via GitGitGadget
2024-02-27 13:28     ` [PATCH v3 08/11] repo_get_merge_bases(): " Johannes Schindelin via GitGitGadget
2024-02-27 13:28     ` [PATCH v3 09/11] get_octopus_merge_bases(): " Johannes Schindelin via GitGitGadget
2024-02-27 13:28     ` [PATCH v3 10/11] repo_get_merge_bases_many(): " Johannes Schindelin via GitGitGadget
2024-02-27 13:28     ` [PATCH v3 11/11] repo_get_merge_bases_many_dirty(): " Johannes Schindelin via GitGitGadget
2024-02-28  9:44     ` [PATCH v4 00/11] The merge-base logic vs missing commit objects Johannes Schindelin via GitGitGadget
2024-02-28  9:44       ` [PATCH v4 01/11] commit-reach(paint_down_to_common): plug two memory leaks Johannes Schindelin via GitGitGadget
2024-02-28  9:44       ` [PATCH v4 02/11] commit-reach(repo_in_merge_bases_many): optionally expect missing commits Johannes Schindelin via GitGitGadget
2024-02-28  9:44       ` [PATCH v4 03/11] commit-reach(repo_in_merge_bases_many): report " Johannes Schindelin via GitGitGadget
2024-03-01  6:58         ` Jeff King
2024-03-01 16:18           ` Junio C Hamano
2024-02-28  9:44       ` Johannes Schindelin via GitGitGadget [this message]
2024-02-28 20:24         ` [PATCH v4 04/11] commit-reach(paint_down_to_common): prepare for handling shallow commits Junio C Hamano
2024-02-28 20:59           ` Dirk Gouders
2024-02-29  9:54             ` Johannes Schindelin
2024-02-29  9:53           ` Johannes Schindelin
2024-02-28  9:44       ` [PATCH v4 05/11] commit-reach(paint_down_to_common): start reporting errors Johannes Schindelin via GitGitGadget
2024-02-28  9:44       ` [PATCH v4 06/11] commit-reach(merge_bases_many): pass on "missing commits" errors Johannes Schindelin via GitGitGadget
2024-02-28  9:44       ` [PATCH v4 07/11] commit-reach(get_merge_bases_many_0): " Johannes Schindelin via GitGitGadget
2024-02-28  9:44       ` [PATCH v4 08/11] commit-reach(repo_get_merge_bases): " Johannes Schindelin via GitGitGadget
2024-02-28  9:44       ` [PATCH v4 09/11] commit-reach(get_octopus_merge_bases): " Johannes Schindelin via GitGitGadget
2024-02-28  9:44       ` [PATCH v4 10/11] commit-reach(repo_get_merge_bases_many): " Johannes Schindelin via GitGitGadget
2024-02-28  9:44       ` [PATCH v4 11/11] commit-reach(repo_get_merge_bases_many_dirty): pass on errors Johannes Schindelin via GitGitGadget

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=837aa5a89c640427a5de064da93f1de4934d8212.1709113458.git.gitgitgadget@gmail.com \
    --to=gitgitgadget@gmail.com \
    --cc=dirk@gouders.net \
    --cc=git@vger.kernel.org \
    --cc=johannes.schindelin@gmx.de \
    --cc=ps@pks.im \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.