[PATCH 0/3] More add_submodule_odb() cleanup in merge code

[PATCH 0/3] More add_submodule_odb() cleanup in merge code

[Jonathan Tan]

(CC-ing Elijah in case he has insight into the merge-ort part.)

While continuing the effort into removing add_submodule_odb() (as part
of the submodule partial clone effort) I came across this part of the
merge code that specifies the repository being operated on in two ways:
one as a struct repository pointer and the other as a path. This patch
set unifies the two.

I normally would not send refactoring patches unless I also have a
feature patch that uses the results of said refactoring, but in this
case, I think that these patches are worth having since they clarify a
potentially unclear part of the API.

Note that these patches mean that the merging code no longer supports
submodules that have their .git dirs in the worktree, but from what I
can tell, this seems to be the direction we're going in
(repo_submodule_init() does not support such submodules).

Patch 3 is included to show how I'm verifying some things. Including
something like that in the master branch would probably require
conditional compilation (to exclude the additional field in struct
object used for checking, among other things), so I'm just including it
here for informational purposes.

All these patches work under GIT_TEST_MERGE_ALGORITHM=recursive and
GIT_TEST_MERGE_ALGORITHM=ort (and when that envvar is unset, for good
measure).

Jonathan Tan (3):
  t6437: run absorbgitdirs on repos
  revision: remove "submodule" from opt struct
  DO NOT SUBMIT commit-reach,revision: verify non-mixing

 alloc.c                    |  2 +
 commit-reach.c             | 60 +++++++++++++++++-------
 merge-ort.c                | 55 +++++++++++++++-------
 merge-recursive.c          | 51 +++++++++++++-------
 object.h                   |  1 +
 revision.c                 | 96 ++++++++++++++++++++++----------------
 revision.h                 |  1 -
 t/t6437-submodule-merge.sh |  9 ++--
 8 files changed, 179 insertions(+), 96 deletions(-)

-- 
2.33.0.309.g3052b89438-goog

[PATCH 1/3] t6437: run absorbgitdirs on repos

[Jonathan Tan]

The submodule merge code is being transitioned away from
add_submodule_odb() to repo_submodule_init(), and the latter does not
support submodules that have their .git directories in the worktree
(instead of in .git/modules). Migrate the test code by calling
absorbgitdirs wherever necessary to place the .git directories of
submodules in .git/modules of the superproject.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 t/t6437-submodule-merge.sh | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/t/t6437-submodule-merge.sh b/t/t6437-submodule-merge.sh
index e5e89c2045..8efce86b42 100755
--- a/t/t6437-submodule-merge.sh
+++ b/t/t6437-submodule-merge.sh
@@ -27,7 +27,8 @@ test_expect_success setup '
 	 git add file &&
 	 test_tick &&
 	 git commit -m sub-root) &&
-	git add sub &&
+	git submodule add ./sub sub &&
+	git submodule absorbgitdirs &&
 	test_tick &&
 	git commit -m root &&
 
@@ -82,7 +83,8 @@ test_expect_success 'setup for merge search' '
 	 git branch sub-a) &&
 	git commit --allow-empty -m init &&
 	git branch init &&
-	git add sub &&
+	git submodule add ./sub sub &&
+	git submodule absorbgitdirs &&
 	git commit -m "a" &&
 	git branch a &&
 
@@ -112,7 +114,8 @@ test_expect_success 'setup for merge search' '
 	git checkout -b g init &&
 	(cd sub &&
 	 git checkout -b sub-g sub-c) &&
-	git add sub &&
+	git submodule add ./sub sub &&
+	git submodule absorbgitdirs &&
 	git commit -a -m "g")
 '
 
-- 
2.33.0.309.g3052b89438-goog

[PATCH 2/3] revision: remove "submodule" from opt struct

[Jonathan Tan]

Clean up a TODO in revision.h by removing the "submodule" field from
struct setup_revision_opt. This field is only used to specify the ref
store to use, so use rev_info->repo to determine the ref store instead.

The only users of this field are merge-ort.c and merge-recursive.c.
However, both these files specify the superproject as rev_info->repo and
the submodule as setup_revision_opt->submodule. In order to be able to
pass the submodule as rev_info->repo, all commits must be parsed with
the submodule explicitly specified; this patch does that as well. (An
incremental solution in which only some commits are parsed with explicit
submodule will not work, because if the same commit is parsed twice in
different repositories, there will be 2 heap-allocated object structs
corresponding to that commit, and any flag set by the revision walking
mechanism on one of them will not be reflected onto the other.)

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 merge-ort.c       | 55 +++++++++++++++++++++++++++++++----------------
 merge-recursive.c | 51 ++++++++++++++++++++++++++++---------------
 revision.c        | 16 +++++---------
 revision.h        |  1 -
 4 files changed, 76 insertions(+), 47 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 515dc39b7f..419ac0f396 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -32,6 +32,7 @@
 #include "promisor-remote.h"
 #include "revision.h"
 #include "strmap.h"
+#include "submodule-config.h"
 #include "submodule.h"
 #include "tree.h"
 #include "unpack-trees.h"
@@ -1511,7 +1512,6 @@ static int find_first_merges(struct repository *repo,
 	xsnprintf(merged_revision, sizeof(merged_revision), "^%s",
 		  oid_to_hex(&a->object.oid));
 	repo_init_revisions(repo, &revs, NULL);
-	rev_opts.submodule = path;
 	/* FIXME: can't handle linked worktrees in submodules yet */
 	revs.single_worktree = path != NULL;
 	setup_revisions(ARRAY_SIZE(rev_args)-1, rev_args, &revs, &rev_opts);
@@ -1521,7 +1521,7 @@ static int find_first_merges(struct repository *repo,
 		die("revision walk setup failed");
 	while ((commit = get_revision(&revs)) != NULL) {
 		struct object *o = &(commit->object);
-		if (in_merge_bases(b, commit))
+		if (repo_in_merge_bases(repo, b, commit))
 			add_object_array(o, NULL, &merges);
 	}
 	reset_revision_walk();
@@ -1536,7 +1536,7 @@ static int find_first_merges(struct repository *repo,
 		contains_another = 0;
 		for (j = 0; j < merges.nr; j++) {
 			struct commit *m2 = (struct commit *) merges.objects[j].item;
-			if (i != j && in_merge_bases(m2, m1)) {
+			if (i != j && repo_in_merge_bases(repo, m2, m1)) {
 				contains_another = 1;
 				break;
 			}
@@ -1557,10 +1557,13 @@ static int merge_submodule(struct merge_options *opt,
 			   const struct object_id *b,
 			   struct object_id *result)
 {
+	const struct submodule *submodule;
+	struct repository subrepo;
+	struct strbuf sb = STRBUF_INIT;
+	int ret = 0;
 	struct commit *commit_o, *commit_a, *commit_b;
 	int parent_count;
 	struct object_array merges;
-	struct strbuf sb = STRBUF_INIT;
 
 	int i;
 	int search = !opt->priv->call_depth;
@@ -1576,6 +1579,10 @@ static int merge_submodule(struct merge_options *opt,
 	if (is_null_oid(b))
 		return 0;
 
+	/*
+	 * NEEDSWORK: Remove this when all submodule object accesses are
+	 * through explicitly specified repositores.
+	 */
 	if (add_submodule_odb(path)) {
 		path_msg(opt, path, 0,
 			 _("Failed to merge submodule %s (not checked out)"),
@@ -1583,39 +1590,49 @@ static int merge_submodule(struct merge_options *opt,
 		return 0;
 	}
 
-	if (!(commit_o = lookup_commit_reference(opt->repo, o)) ||
-	    !(commit_a = lookup_commit_reference(opt->repo, a)) ||
-	    !(commit_b = lookup_commit_reference(opt->repo, b))) {
+	submodule = submodule_from_path(opt->repo, null_oid(), path);
+	if (repo_submodule_init(&subrepo, opt->repo, submodule)) {
+		path_msg(opt, path, 0,
+				_("Failed to merge submodule %s (not checked out)"),
+				path);
+		return 0;
+	}
+
+	if (!(commit_o = lookup_commit_reference(&subrepo, o)) ||
+	    !(commit_a = lookup_commit_reference(&subrepo, a)) ||
+	    !(commit_b = lookup_commit_reference(&subrepo, b))) {
 		path_msg(opt, path, 0,
 			 _("Failed to merge submodule %s (commits not present)"),
 			 path);
-		return 0;
+		goto cleanup;
 	}
 
 	/* check whether both changes are forward */
-	if (!in_merge_bases(commit_o, commit_a) ||
-	    !in_merge_bases(commit_o, commit_b)) {
+	if (!repo_in_merge_bases(&subrepo, commit_o, commit_a) ||
+	    !repo_in_merge_bases(&subrepo, commit_o, commit_b)) {
 		path_msg(opt, path, 0,
 			 _("Failed to merge submodule %s "
 			   "(commits don't follow merge-base)"),
 			 path);
-		return 0;
+		goto cleanup;
 	}
 
 	/* Case #1: a is contained in b or vice versa */
-	if (in_merge_bases(commit_a, commit_b)) {
+	if (repo_in_merge_bases(&subrepo, commit_a, commit_b)) {
 		oidcpy(result, b);
 		path_msg(opt, path, 1,
 			 _("Note: Fast-forwarding submodule %s to %s"),
 			 path, oid_to_hex(b));
-		return 1;
+		ret = 1;
+		goto cleanup;
 	}
-	if (in_merge_bases(commit_b, commit_a)) {
+	if (repo_in_merge_bases(&subrepo, commit_b, commit_a)) {
 		oidcpy(result, a);
 		path_msg(opt, path, 1,
 			 _("Note: Fast-forwarding submodule %s to %s"),
 			 path, oid_to_hex(a));
-		return 1;
+		ret = 1;
+		goto cleanup;
 	}
 
 	/*
@@ -1627,10 +1644,10 @@ static int merge_submodule(struct merge_options *opt,
 
 	/* Skip the search if makes no sense to the calling context.  */
 	if (!search)
-		return 0;
+		goto cleanup;
 
 	/* find commit which merges them */
-	parent_count = find_first_merges(opt->repo, path, commit_a, commit_b,
+	parent_count = find_first_merges(&subrepo, path, commit_a, commit_b,
 					 &merges);
 	switch (parent_count) {
 	case 0:
@@ -1664,7 +1681,9 @@ static int merge_submodule(struct merge_options *opt,
 	}
 
 	object_array_clear(&merges);
-	return 0;
+cleanup:
+	repo_clear(&subrepo);
+	return ret;
 }
 
 static void initialize_attr_index(struct merge_options *opt)
diff --git a/merge-recursive.c b/merge-recursive.c
index 3355d50e8a..bc5b34d56e 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -24,6 +24,7 @@
 #include "repository.h"
 #include "revision.h"
 #include "string-list.h"
+#include "submodule-config.h"
 #include "submodule.h"
 #include "tag.h"
 #include "tree-walk.h"
@@ -1113,7 +1114,6 @@ static int find_first_merges(struct repository *repo,
 	xsnprintf(merged_revision, sizeof(merged_revision), "^%s",
 		  oid_to_hex(&a->object.oid));
 	repo_init_revisions(repo, &revs, NULL);
-	rev_opts.submodule = path;
 	/* FIXME: can't handle linked worktrees in submodules yet */
 	revs.single_worktree = path != NULL;
 	setup_revisions(ARRAY_SIZE(rev_args)-1, rev_args, &revs, &rev_opts);
@@ -1123,7 +1123,7 @@ static int find_first_merges(struct repository *repo,
 		die("revision walk setup failed");
 	while ((commit = get_revision(&revs)) != NULL) {
 		struct object *o = &(commit->object);
-		if (in_merge_bases(b, commit))
+		if (repo_in_merge_bases(repo, b, commit))
 			add_object_array(o, NULL, &merges);
 	}
 	reset_revision_walk();
@@ -1138,7 +1138,7 @@ static int find_first_merges(struct repository *repo,
 		contains_another = 0;
 		for (j = 0; j < merges.nr; j++) {
 			struct commit *m2 = (struct commit *) merges.objects[j].item;
-			if (i != j && in_merge_bases(m2, m1)) {
+			if (i != j && repo_in_merge_bases(repo, m2, m1)) {
 				contains_another = 1;
 				break;
 			}
@@ -1174,6 +1174,9 @@ static int merge_submodule(struct merge_options *opt,
 			   const struct object_id *base, const struct object_id *a,
 			   const struct object_id *b)
 {
+	const struct submodule *submodule;
+	struct repository subrepo;
+	int ret = 0;
 	struct commit *commit_base, *commit_a, *commit_b;
 	int parent_count;
 	struct object_array merges;
@@ -1197,27 +1200,37 @@ static int merge_submodule(struct merge_options *opt,
 	if (is_null_oid(b))
 		return 0;
 
+	/*
+	 * NEEDSWORK: Remove this when all submodule object accesses are
+	 * through explicitly specified repositores.
+	 */
 	if (add_submodule_odb(path)) {
 		output(opt, 1, _("Failed to merge submodule %s (not checked out)"), path);
 		return 0;
 	}
 
-	if (!(commit_base = lookup_commit_reference(opt->repo, base)) ||
-	    !(commit_a = lookup_commit_reference(opt->repo, a)) ||
-	    !(commit_b = lookup_commit_reference(opt->repo, b))) {
-		output(opt, 1, _("Failed to merge submodule %s (commits not present)"), path);
+	submodule = submodule_from_path(opt->repo, null_oid(), path);
+	if (repo_submodule_init(&subrepo, opt->repo, submodule)) {
+		output(opt, 1, _("Failed to merge submodule %s (not checked out)"), path);
 		return 0;
 	}
 
+	if (!(commit_base = lookup_commit_reference(&subrepo, base)) ||
+	    !(commit_a = lookup_commit_reference(&subrepo, a)) ||
+	    !(commit_b = lookup_commit_reference(&subrepo, b))) {
+		output(opt, 1, _("Failed to merge submodule %s (commits not present)"), path);
+		goto cleanup;
+	}
+
 	/* check whether both changes are forward */
-	if (!in_merge_bases(commit_base, commit_a) ||
-	    !in_merge_bases(commit_base, commit_b)) {
+	if (!repo_in_merge_bases(&subrepo, commit_base, commit_a) ||
+	    !repo_in_merge_bases(&subrepo, commit_base, commit_b)) {
 		output(opt, 1, _("Failed to merge submodule %s (commits don't follow merge-base)"), path);
-		return 0;
+		goto cleanup;
 	}
 
 	/* Case #1: a is contained in b or vice versa */
-	if (in_merge_bases(commit_a, commit_b)) {
+	if (repo_in_merge_bases(&subrepo, commit_a, commit_b)) {
 		oidcpy(result, b);
 		if (show(opt, 3)) {
 			output(opt, 3, _("Fast-forwarding submodule %s to the following commit:"), path);
@@ -1227,9 +1240,10 @@ static int merge_submodule(struct merge_options *opt,
 		else
 			; /* no output */
 
-		return 1;
+		ret = 1;
+		goto cleanup;
 	}
-	if (in_merge_bases(commit_b, commit_a)) {
+	if (repo_in_merge_bases(&subrepo, commit_b, commit_a)) {
 		oidcpy(result, a);
 		if (show(opt, 3)) {
 			output(opt, 3, _("Fast-forwarding submodule %s to the following commit:"), path);
@@ -1239,7 +1253,8 @@ static int merge_submodule(struct merge_options *opt,
 		else
 			; /* no output */
 
-		return 1;
+		ret = 1;
+		goto cleanup;
 	}
 
 	/*
@@ -1251,10 +1266,10 @@ static int merge_submodule(struct merge_options *opt,
 
 	/* Skip the search if makes no sense to the calling context.  */
 	if (!search)
-		return 0;
+		goto cleanup;
 
 	/* find commit which merges them */
-	parent_count = find_first_merges(opt->repo, &merges, path,
+	parent_count = find_first_merges(&subrepo, &merges, path,
 					 commit_a, commit_b);
 	switch (parent_count) {
 	case 0:
@@ -1281,7 +1296,9 @@ static int merge_submodule(struct merge_options *opt,
 	}
 
 	object_array_clear(&merges);
-	return 0;
+cleanup:
+	repo_clear(&subrepo);
+	return ret;
 }
 
 static int merge_mode_and_contents(struct merge_options *opt,
diff --git a/revision.c b/revision.c
index 0dabb5a0bc..7e7c41fedd 100644
--- a/revision.c
+++ b/revision.c
@@ -2563,8 +2563,7 @@ static int for_each_good_bisect_ref(struct ref_store *refs, each_ref_fn fn, void
 	return for_each_bisect_ref(refs, fn, cb_data, term_good);
 }
 
-static int handle_revision_pseudo_opt(const char *submodule,
-				      struct rev_info *revs,
+static int handle_revision_pseudo_opt(struct rev_info *revs,
 				      const char **argv, int *flags)
 {
 	const char *arg = argv[0];
@@ -2572,7 +2571,7 @@ static int handle_revision_pseudo_opt(const char *submodule,
 	struct ref_store *refs;
 	int argcount;
 
-	if (submodule) {
+	if (revs->repo != the_repository) {
 		/*
 		 * We need some something like get_submodule_worktrees()
 		 * before we can go through all worktrees of a submodule,
@@ -2581,9 +2580,8 @@ static int handle_revision_pseudo_opt(const char *submodule,
 		 */
 		if (!revs->single_worktree)
 			BUG("--single-worktree cannot be used together with submodule");
-		refs = get_submodule_ref_store(submodule);
-	} else
-		refs = get_main_ref_store(revs->repo);
+	}
+	refs = get_main_ref_store(revs->repo);
 
 	/*
 	 * NOTE!
@@ -2707,12 +2705,8 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s
 {
 	int i, flags, left, seen_dashdash, revarg_opt;
 	struct strvec prune_data = STRVEC_INIT;
-	const char *submodule = NULL;
 	int seen_end_of_options = 0;
 
-	if (opt)
-		submodule = opt->submodule;
-
 	/* First, search for "--" */
 	if (opt && opt->assume_dashdash) {
 		seen_dashdash = 1;
@@ -2741,7 +2735,7 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s
 		if (!seen_end_of_options && *arg == '-') {
 			int opts;
 
-			opts = handle_revision_pseudo_opt(submodule,
+			opts = handle_revision_pseudo_opt(
 						revs, argv + i,
 						&flags);
 			if (opts > 0) {
diff --git a/revision.h b/revision.h
index 0c65a760ee..5578bb4720 100644
--- a/revision.h
+++ b/revision.h
@@ -336,7 +336,6 @@ extern volatile show_early_output_fn_t show_early_output;
 struct setup_revision_opt {
 	const char *def;
 	void (*tweak)(struct rev_info *, struct setup_revision_opt *);
-	const char *submodule;	/* TODO: drop this and use rev_info->repo */
 	unsigned int	assume_dashdash:1,
 			allow_exclude_promisor_objects:1;
 	unsigned revarg_opt;
-- 
2.33.0.309.g3052b89438-goog

[PATCH 3/3] DO NOT SUBMIT commit-reach,revision: verify non-mixing

[Jonathan Tan]

This patch is provided to show reviewers how I'm verifying that there is
no mixing of object structs in the revision walking mechanisms. I've
intercepted any creation of commit nodes in alloc.c (whether through
alloc_commit_node() directly or through alloc_object_node()) to add the
repository in which that node is allocated, and checked all additions of
commits to data structures (as far as I can tell) in commit-reach.c and
revision.c to make sure that the repository in the added objects is what
we expect. This ensures that any flags set are set on the correct nodes
(for example, it won't be set on an object allocated in the_repository
and then checked from an object allocated in a submodule).

I'll leave out this patch in subsequent revisions of this patch set.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 alloc.c        |  2 ++
 commit-reach.c | 60 +++++++++++++++++++++++++------------
 object.h       |  1 +
 revision.c     | 80 ++++++++++++++++++++++++++++++++------------------
 4 files changed, 97 insertions(+), 46 deletions(-)

diff --git a/alloc.c b/alloc.c
index 957a0af362..98b8c06834 100644
--- a/alloc.c
+++ b/alloc.c
@@ -96,6 +96,7 @@ void *alloc_object_node(struct repository *r)
 {
 	struct object *obj = alloc_node(r->parsed_objects->object_state, sizeof(union any_object));
 	obj->type = OBJ_NONE;
+	obj->repo = r;
 	return obj;
 }
 
@@ -120,6 +121,7 @@ void *alloc_commit_node(struct repository *r)
 {
 	struct commit *c = alloc_node(r->parsed_objects->commit_state, sizeof(struct commit));
 	init_commit_node(c);
+	c->object.repo = r;
 	return c;
 }
 
diff --git a/commit-reach.c b/commit-reach.c
index c226ee3da4..eb868c3837 100644
--- a/commit-reach.c
+++ b/commit-reach.c
@@ -9,6 +9,30 @@
 #include "tag.h"
 #include "commit-reach.h"
 
+static void checked_put(struct repository *repo, struct prio_queue *queue, struct commit *commit)
+{
+	if (repo != commit->object.repo)
+		*(int *)0=1;
+	prio_queue_put(queue, commit);
+}
+#define prio_queue_put DO_NOT_USE
+
+static struct commit_list *checked_insert(struct repository *repo, struct commit *item, struct commit_list **list)
+{
+	if (repo != item->object.repo)
+		*(int *)0=1;
+	return commit_list_insert(item, list);
+}
+#define commit_list_insert DO_NOT_USE
+
+static struct commit_list *checked_insert_by_date(struct repository *repo, struct commit *item, struct commit_list **list)
+{
+	if (repo != item->object.repo)
+		*(int *)0=1;
+	return commit_list_insert_by_date(item, list);
+}
+#define commit_list_insert_by_date DO_NOT_USE
+
 /* Remember to update object flag allocation in object.h */
 #define PARENT1		(1u<<16)
 #define PARENT2		(1u<<17)
@@ -66,11 +90,11 @@ static struct commit_list *paint_down_to_common(struct repository *r,
 		commit_list_append(one, &result);
 		return result;
 	}
-	prio_queue_put(&queue, one);
+	checked_put(r, &queue, one);
 
 	for (i = 0; i < n; i++) {
 		twos[i]->object.flags |= PARENT2;
-		prio_queue_put(&queue, twos[i]);
+		checked_put(r, &queue, twos[i]);
 	}
 
 	while (queue_has_nonstale(&queue)) {
@@ -92,7 +116,7 @@ static struct commit_list *paint_down_to_common(struct repository *r,
 		if (flags == (PARENT1 | PARENT2)) {
 			if (!(commit->object.flags & RESULT)) {
 				commit->object.flags |= RESULT;
-				commit_list_insert_by_date(commit, &result);
+				checked_insert_by_date(r, commit, &result);
 			}
 			/* Mark parents of a found merge stale */
 			flags |= STALE;
@@ -106,7 +130,7 @@ static struct commit_list *paint_down_to_common(struct repository *r,
 			if (repo_parse_commit(r, p))
 				return NULL;
 			p->object.flags |= flags;
-			prio_queue_put(&queue, p);
+			checked_put(r, &queue, p);
 		}
 	}
 
@@ -128,7 +152,7 @@ static struct commit_list *merge_bases_many(struct repository *r,
 			 * We do not mark this even with RESULT so we do not
 			 * have to clean it up.
 			 */
-			return commit_list_insert(one, &result);
+			return checked_insert(r, one, &result);
 	}
 
 	if (repo_parse_commit(r, one))
@@ -143,7 +167,7 @@ static struct commit_list *merge_bases_many(struct repository *r,
 	while (list) {
 		struct commit *commit = pop_commit(&list);
 		if (!(commit->object.flags & STALE))
-			commit_list_insert_by_date(commit, &result);
+			checked_insert_by_date(r, commit, &result);
 	}
 	return result;
 }
@@ -155,7 +179,7 @@ struct commit_list *get_octopus_merge_bases(struct commit_list *in)
 	if (!in)
 		return ret;
 
-	commit_list_insert(in->item, &ret);
+	checked_insert(the_repository, in->item, &ret);
 
 	for (i = in->next; i; i = i->next) {
 		struct commit_list *new_commits = NULL, *end = NULL;
@@ -287,7 +311,7 @@ static int remove_redundant_with_gen(struct repository *r,
 		/* push the STALE bits up to min generation */
 		struct commit_list *stack = NULL;
 
-		commit_list_insert(walk_start[i], &stack);
+		checked_insert(r, walk_start[i], &stack);
 		walk_start[i]->object.flags |= STALE;
 
 		while (stack) {
@@ -317,7 +341,7 @@ static int remove_redundant_with_gen(struct repository *r,
 			while (parents) {
 				if (!(parents->item->object.flags & STALE)) {
 					parents->item->object.flags |= STALE;
-					commit_list_insert(parents->item, &stack);
+					checked_insert(r, parents->item, &stack);
 					break;
 				}
 				parents = parents->next;
@@ -410,7 +434,7 @@ static struct commit_list *get_merge_bases_many_0(struct repository *r,
 	cnt = remove_redundant(r, rslt, cnt);
 	result = NULL;
 	for (i = 0; i < cnt; i++)
-		commit_list_insert_by_date(rslt[i], &result);
+		checked_insert_by_date(r, rslt[i], &result);
 	free(rslt);
 	return result;
 }
@@ -451,7 +475,7 @@ int repo_is_descendant_of(struct repository *r,
 	if (generation_numbers_enabled(the_repository)) {
 		struct commit_list *from_list = NULL;
 		int result;
-		commit_list_insert(commit, &from_list);
+		checked_insert(r, commit, &from_list);
 		result = can_all_from_reach(from_list, with_commit, 0);
 		free_commit_list(from_list);
 		return result;
@@ -550,7 +574,7 @@ struct commit_list *reduce_heads(struct commit_list *heads)
 	}
 	num_head = remove_redundant(the_repository, array, num_head);
 	for (i = 0; i < num_head; i++)
-		tail = &commit_list_insert(array[i], tail)->next;
+		tail = &checked_insert(the_repository, array[i], tail)->next;
 	free(array);
 	return result;
 }
@@ -588,7 +612,7 @@ int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid)
 	if (parse_commit(new_commit) < 0)
 		return 0;
 
-	commit_list_insert(old_commit, &old_commit_list);
+	checked_insert(the_repository, old_commit, &old_commit_list);
 	ret = repo_is_descendant_of(the_repository,
 				    new_commit, old_commit_list);
 	free_commit_list(old_commit_list);
@@ -765,7 +789,7 @@ int can_all_from_reach_with_flag(struct object_array *from,
 		struct commit_list *stack = NULL;
 
 		list[i]->object.flags |= assign_flag;
-		commit_list_insert(list[i], &stack);
+		checked_insert(the_repository, list[i], &stack);
 
 		while (stack) {
 			struct commit_list *parent;
@@ -789,7 +813,7 @@ int can_all_from_reach_with_flag(struct object_array *from,
 					    commit_graph_generation(parent->item) < min_generation)
 						continue;
 
-					commit_list_insert(parent->item, &stack);
+					checked_insert(the_repository, parent->item, &stack);
 					break;
 				}
 			}
@@ -907,7 +931,7 @@ struct commit_list *get_reachable_subset(struct commit **from, int nr_from,
 			c->object.flags |= PARENT2;
 			parse_commit(c);
 
-			prio_queue_put(&queue, *item);
+			checked_put(the_repository, &queue, *item);
 		}
 	}
 
@@ -917,7 +941,7 @@ struct commit_list *get_reachable_subset(struct commit **from, int nr_from,
 		if (current->object.flags & PARENT1) {
 			current->object.flags &= ~PARENT1;
 			current->object.flags |= reachable_flag;
-			commit_list_insert(current, &found_commits);
+			checked_insert(the_repository, current, &found_commits);
 			num_to_find--;
 		}
 
@@ -933,7 +957,7 @@ struct commit_list *get_reachable_subset(struct commit **from, int nr_from,
 				continue;
 
 			p->object.flags |= PARENT2;
-			prio_queue_put(&queue, p);
+			checked_put(the_repository, &queue, p);
 		}
 	}
 
diff --git a/object.h b/object.h
index 3b38c9cc98..22e9d2a52a 100644
--- a/object.h
+++ b/object.h
@@ -90,6 +90,7 @@ struct object {
 	unsigned type : TYPE_BITS;
 	unsigned flags : FLAG_BITS;
 	struct object_id oid;
+	struct repository *repo;
 };
 
 const char *type_name(unsigned int type);
diff --git a/revision.c b/revision.c
index 7e7c41fedd..ba30e58921 100644
--- a/revision.c
+++ b/revision.c
@@ -33,6 +33,30 @@
 #include "bloom.h"
 #include "json-writer.h"
 
+static void checked_put(struct repository *repo, struct prio_queue *queue, struct commit *commit)
+{
+	if (repo != commit->object.repo)
+		*(int *)0=1;
+	prio_queue_put(queue, commit);
+}
+#define prio_queue_put DO_NOT_USE
+
+static struct commit_list *checked_insert(struct repository *repo, struct commit *item, struct commit_list **list)
+{
+	if (repo != item->object.repo)
+		*(int *)0=1;
+	return commit_list_insert(item, list);
+}
+#define commit_list_insert DO_NOT_USE
+
+static struct commit_list *checked_insert_by_date(struct repository *repo, struct commit *item, struct commit_list **list)
+{
+	if (repo != item->object.repo)
+		*(int *)0=1;
+	return commit_list_insert_by_date(item, list);
+}
+#define commit_list_insert_by_date DO_NOT_USE
+
 volatile show_early_output_fn_t show_early_output;
 
 static const char *term_bad;
@@ -1124,9 +1148,9 @@ static int process_parents(struct rev_info *revs, struct commit *commit,
 				continue;
 			p->object.flags |= (SEEN | NOT_USER_GIVEN);
 			if (list)
-				commit_list_insert_by_date(p, list);
+				checked_insert_by_date(revs->repo, p, list);
 			if (queue)
-				prio_queue_put(queue, p);
+				checked_put(revs->repo, queue, p);
 		}
 		return 0;
 	}
@@ -1166,9 +1190,9 @@ static int process_parents(struct rev_info *revs, struct commit *commit,
 		if (!(p->object.flags & SEEN)) {
 			p->object.flags |= (SEEN | NOT_USER_GIVEN);
 			if (list)
-				commit_list_insert_by_date(p, list);
+				checked_insert_by_date(revs->repo, p, list);
 			if (queue)
-				prio_queue_put(queue, p);
+				checked_put(revs->repo, queue, p);
 		}
 		if (revs->first_parent_only)
 			break;
@@ -1291,7 +1315,7 @@ static int still_interesting(struct commit_list *src, timestamp_t date, int slop
  * the result of "A..B" without --ancestry-path, and limits the latter
  * further to the ones that can reach one of the commits in "bottom".
  */
-static void limit_to_ancestry(struct commit_list *bottom, struct commit_list *list)
+static void limit_to_ancestry(struct repository *repo, struct commit_list *bottom, struct commit_list *list)
 {
 	struct commit_list *p;
 	struct commit_list *rlist = NULL;
@@ -1302,7 +1326,7 @@ static void limit_to_ancestry(struct commit_list *bottom, struct commit_list *li
 	 * process parents before children.
 	 */
 	for (p = list; p; p = p->next)
-		commit_list_insert(p->item, &rlist);
+		checked_insert(repo, p->item, &rlist);
 
 	for (p = bottom; p; p = p->next)
 		p->item->object.flags |= TMP_MARK;
@@ -1362,12 +1386,12 @@ static void limit_to_ancestry(struct commit_list *bottom, struct commit_list *li
  * to filter the result of "A..B" further to the ones that can actually
  * reach A.
  */
-static struct commit_list *collect_bottom_commits(struct commit_list *list)
+static struct commit_list *collect_bottom_commits(struct repository *repo, struct commit_list *list)
 {
 	struct commit_list *elem, *bottom = NULL;
 	for (elem = list; elem; elem = elem->next)
 		if (elem->item->object.flags & BOTTOM)
-			commit_list_insert(elem->item, &bottom);
+			checked_insert(repo, elem->item, &bottom);
 	return bottom;
 }
 
@@ -1399,7 +1423,7 @@ static int limit_list(struct rev_info *revs)
 	struct commit *interesting_cache = NULL;
 
 	if (revs->ancestry_path) {
-		bottom = collect_bottom_commits(original_list);
+		bottom = collect_bottom_commits(revs->repo, original_list);
 		if (!bottom)
 			die("--ancestry-path given but there are no bottom commits");
 	}
@@ -1427,7 +1451,7 @@ static int limit_list(struct rev_info *revs)
 		    !revs->line_level_traverse)
 			continue;
 		date = commit->date;
-		p = &commit_list_insert(commit, p)->next;
+		p = &checked_insert(revs->repo, commit, p)->next;
 
 		show = show_early_output;
 		if (!show)
@@ -1443,7 +1467,7 @@ static int limit_list(struct rev_info *revs)
 		limit_left_right(newlist, revs);
 
 	if (bottom) {
-		limit_to_ancestry(bottom, newlist);
+		limit_to_ancestry(revs->repo, bottom, newlist);
 		free_commit_list(bottom);
 	}
 
@@ -3133,14 +3157,14 @@ static struct commit_list **simplify_one(struct rev_info *revs, struct commit *c
 	for (cnt = 0, p = commit->parents; p; p = p->next) {
 		pst = locate_simplify_state(revs, p->item);
 		if (!pst->simplified) {
-			tail = &commit_list_insert(p->item, tail)->next;
+			tail = &checked_insert(revs->repo, p->item, tail)->next;
 			cnt++;
 		}
 		if (revs->first_parent_only)
 			break;
 	}
 	if (cnt) {
-		tail = &commit_list_insert(commit, tail)->next;
+		tail = &checked_insert(revs->repo, commit, tail)->next;
 		return tail;
 	}
 
@@ -3230,7 +3254,7 @@ static void simplify_merges(struct rev_info *revs)
 		 * Do not free(list) here yet; the original list
 		 * is used later in this function.
 		 */
-		commit_list_insert(commit, &yet_to_do);
+		checked_insert(revs->repo, commit, &yet_to_do);
 	}
 	while (yet_to_do) {
 		list = yet_to_do;
@@ -3252,7 +3276,7 @@ static void simplify_merges(struct rev_info *revs)
 		commit = pop_commit(&list);
 		st = locate_simplify_state(revs, commit);
 		if (st->simplified == commit)
-			tail = &commit_list_insert(commit, tail)->next;
+			tail = &checked_insert(revs->repo, commit, tail)->next;
 	}
 }
 
@@ -3316,13 +3340,13 @@ static void trace2_topo_walk_statistics_atexit(void)
 	jw_release(&jw);
 }
 
-static inline void test_flag_and_insert(struct prio_queue *q, struct commit *c, int flag)
+static inline void test_flag_and_insert(struct repository *repo, struct prio_queue *q, struct commit *c, int flag)
 {
 	if (c->object.flags & flag)
 		return;
 
 	c->object.flags |= flag;
-	prio_queue_put(q, c);
+	checked_put(repo, q, c);
 }
 
 static void explore_walk_step(struct rev_info *revs)
@@ -3352,7 +3376,7 @@ static void explore_walk_step(struct rev_info *revs)
 		mark_parents_uninteresting(c);
 
 	for (p = c->parents; p; p = p->next)
-		test_flag_and_insert(&info->explore_queue, p->item, TOPO_WALK_EXPLORED);
+		test_flag_and_insert(revs->repo, &info->explore_queue, p->item, TOPO_WALK_EXPLORED);
 }
 
 static void explore_to_depth(struct rev_info *revs,
@@ -3393,7 +3417,7 @@ static void indegree_walk_step(struct rev_info *revs)
 		else
 			*pi = 2;
 
-		test_flag_and_insert(&info->indegree_queue, parent, TOPO_WALK_INDEGREE);
+		test_flag_and_insert(revs->repo, &info->indegree_queue, parent, TOPO_WALK_INDEGREE);
 
 		if (revs->first_parent_only)
 			return;
@@ -3464,8 +3488,8 @@ static void init_topo_walk(struct rev_info *revs)
 		if (repo_parse_commit_gently(revs->repo, c, 1))
 			continue;
 
-		test_flag_and_insert(&info->explore_queue, c, TOPO_WALK_EXPLORED);
-		test_flag_and_insert(&info->indegree_queue, c, TOPO_WALK_INDEGREE);
+		test_flag_and_insert(revs->repo, &info->explore_queue, c, TOPO_WALK_EXPLORED);
+		test_flag_and_insert(revs->repo, &info->indegree_queue, c, TOPO_WALK_INDEGREE);
 
 		generation = commit_graph_generation(c);
 		if (generation < info->min_generation)
@@ -3482,7 +3506,7 @@ static void init_topo_walk(struct rev_info *revs)
 		struct commit *c = list->item;
 
 		if (*(indegree_slab_at(&info->indegree, c)) == 1)
-			prio_queue_put(&info->topo_queue, c);
+			checked_put(revs->repo, &info->topo_queue, c);
 	}
 
 	/*
@@ -3545,7 +3569,7 @@ static void expand_topo_walk(struct rev_info *revs, struct commit *commit)
 
 		(*pi)--;
 		if (*pi == 1)
-			prio_queue_put(&info->topo_queue, parent);
+			checked_put(revs->repo, &info->topo_queue, parent);
 
 		if (revs->first_parent_only)
 			return;
@@ -3635,7 +3659,7 @@ static enum rewrite_result rewrite_one_1(struct rev_info *revs,
 	}
 }
 
-static void merge_queue_into_list(struct prio_queue *q, struct commit_list **list)
+static void merge_queue_into_list(struct repository *repo, struct prio_queue *q, struct commit_list **list)
 {
 	while (q->nr) {
 		struct commit *item = prio_queue_peek(q);
@@ -3644,7 +3668,7 @@ static void merge_queue_into_list(struct prio_queue *q, struct commit_list **lis
 		if (p && p->item->date >= item->date)
 			list = &p->next;
 		else {
-			p = commit_list_insert(item, list);
+			p = checked_insert(repo, item, list);
 			list = &p->next; /* skip newly added item */
 			prio_queue_get(q); /* pop item */
 		}
@@ -3655,7 +3679,7 @@ static enum rewrite_result rewrite_one(struct rev_info *revs, struct commit **pp
 {
 	struct prio_queue queue = { compare_commits_by_commit_date };
 	enum rewrite_result ret = rewrite_one_1(revs, pp, &queue);
-	merge_queue_into_list(&queue, &revs->commits);
+	merge_queue_into_list(revs->repo, &queue, &revs->commits);
 	clear_prio_queue(&queue);
 	return ret;
 }
@@ -4075,7 +4099,7 @@ static void create_boundary_commit_list(struct rev_info *revs)
 		if (c->object.flags & (SHOWN | BOUNDARY))
 			continue;
 		c->object.flags |= BOUNDARY;
-		commit_list_insert(c, &revs->commits);
+		checked_insert(revs->repo, c, &revs->commits);
 	}
 
 	/*
@@ -4179,7 +4203,7 @@ struct commit *get_revision(struct rev_info *revs)
 	if (revs->reverse) {
 		reversed = NULL;
 		while ((c = get_revision_internal(revs)))
-			commit_list_insert(c, &reversed);
+			checked_insert(revs->repo, c, &reversed);
 		revs->commits = reversed;
 		revs->reverse = 0;
 		revs->reverse_output_stage = 1;
-- 
2.33.0.309.g3052b89438-goog

Re: [PATCH 1/3] t6437: run absorbgitdirs on repos

[Jonathan Tan]

> The submodule merge code is being transitioned away from
> add_submodule_odb() to repo_submodule_init(), and the latter does not
> support submodules that have their .git directories in the worktree
> (instead of in .git/modules). Migrate the test code by calling
> absorbgitdirs wherever necessary to place the .git directories of
> submodules in .git/modules of the superproject.

Upon further work, there seems to be quite a lot of test code to
migrate, and it might be easier instead to add support for unabsorbed
submodules to repo_submodule_init() (say, following what
open_submodule() in submodule.c does). Feel free to comment on this or
other aspects of the patch set, and in the meantime, I'll investigate
that.

Re: [PATCH 1/3] t6437: run absorbgitdirs on repos

[Junio C Hamano]

Jonathan Tan <jonathantanmy@google.com> writes:

> The submodule merge code is being transitioned away from
> add_submodule_odb() to repo_submodule_init(), and the latter does not
> support submodules that have their .git directories in the worktree
> (instead of in .git/modules). Migrate the test code by calling
> absorbgitdirs wherever necessary to place the .git directories of
> submodules in .git/modules of the superproject.

It is wonderful that we can work with the "absorbed" form of the
submodule repositories, and having all the submodules in "absorbed"
form may be ideal in the longer term, but I fail to convince myself
that this particular change to test script is a good idea.

After all, if you are not cloning from somebody else but creating
a new submodule in a superproject you have (whether that was locally
created or cloned from elsewhere), wouldn't it more natural to have
a submodule that is not "absorbed" there?  Is it easy to create an
"absorbed" submodule locally?  

The change to tests in this patch may make things work with the
repo_submodule_init() that is incapable of work with submodules with
in-worktree .git repository.  In theory, the users can do the same,
i.e. have an extra "git submodule absorbgitdirs" call, when adding a
new submodule, to make things work with repo_submodule_init(), too.

But that sounds like a workaround, not a feature.

Quite honestly, if repo_submodule_init() can only work with the
"absorbed" form, isn't it simply a bug?

Two independent (i.e. we can do either or both) things may improve
the situation:

 - allow repo_submodule_init() to work on both forms (in place---no
   cheating by calling "absorb" behind user's back).

 - teach "git submodule add" to transition an old-style submodule
   into the "absorbed" form (either with an option, or do so by
   default with an escape hatch to disable).

The latter indirectly attacks the "repo_submodule_init() can only
work with absorbed form" issue by making it harder to create a
non-absorbed submodule to begin with.

Hmm?

Re: [PATCH 0/3] More add_submodule_odb() cleanup in merge code

[Elijah Newren]

On Wed, Sep 8, 2021 at 11:18 AM Jonathan Tan <jonathantanmy@google.com> wrote:
>
> (CC-ing Elijah in case he has insight into the merge-ort part.)

All the submodule merging related functions were lifted from
merge-recursive.c and minimally modified to fit the new structure.
The only substantive change I made was to fix the merge result for
virtual merge bases, but that's like one or two lines of code.  In
particular, everything relating to submodule objects was totally
untouched...and I think that's reflected in the fact that your PATCH 2
basically is the same patch twice, once for merge-recursive and once
for merge-ort.

I read over PATCH 2 and I didn't find anything that looked
problematic, but I'm not up-to-speed on the add_submodule_odb and repo
handling bits of the codebase so I'm not sure I would catch anything.
But I am encouraged by the fact that it looks like you did the same
stuff to merge-recursive and merge-ort; I'd be worried you missed
something if that weren't the case.


As a sidenote, though...

This does remind me that I noticed that the following functions from
object-store.h do not take an explicit repository:

write_object_file()
hash_object_file()
hash_object_file_literally()
force_object_loose()

I have a patch sitting around somewhere (possibly only still
referenced in my 'temp' branch) to make repo_*() variants of the above
functions, and turn the above into simple wrappers of the repo_*()
variants which just pass the_repository (much like someone else did
with read_object_file() and repo_read_object_file()).  It also updates
merge-ort to use the new repo_*() functions.  However, I ended up
excluding it from my merge-ort submissions since it wasn't necessary.
Would this be of interest in your submodule work, though?  I guess
it'd only matter if we started doing real merges of submodules as part
of a parent repo merge.  (As opposed to the fast-forward-only merges
that merge-recursive/merge-ort do right now for submodules.)

> While continuing the effort into removing add_submodule_odb() (as part
> of the submodule partial clone effort) I came across this part of the
> merge code that specifies the repository being operated on in two ways:
> one as a struct repository pointer and the other as a path. This patch
> set unifies the two.
>
> I normally would not send refactoring patches unless I also have a
> feature patch that uses the results of said refactoring, but in this
> case, I think that these patches are worth having since they clarify a
> potentially unclear part of the API.
>
> Note that these patches mean that the merging code no longer supports
> submodules that have their .git dirs in the worktree, but from what I
> can tell, this seems to be the direction we're going in
> (repo_submodule_init() does not support such submodules).
>
> Patch 3 is included to show how I'm verifying some things. Including
> something like that in the master branch would probably require
> conditional compilation (to exclude the additional field in struct
> object used for checking, among other things), so I'm just including it
> here for informational purposes.
>
> All these patches work under GIT_TEST_MERGE_ALGORITHM=recursive and
> GIT_TEST_MERGE_ALGORITHM=ort (and when that envvar is unset, for good
> measure).
>
> Jonathan Tan (3):
>   t6437: run absorbgitdirs on repos
>   revision: remove "submodule" from opt struct
>   DO NOT SUBMIT commit-reach,revision: verify non-mixing
>
>  alloc.c                    |  2 +
>  commit-reach.c             | 60 +++++++++++++++++-------
>  merge-ort.c                | 55 +++++++++++++++-------
>  merge-recursive.c          | 51 +++++++++++++-------
>  object.h                   |  1 +
>  revision.c                 | 96 ++++++++++++++++++++++----------------
>  revision.h                 |  1 -
>  t/t6437-submodule-merge.sh |  9 ++--
>  8 files changed, 179 insertions(+), 96 deletions(-)
>
> --
> 2.33.0.309.g3052b89438-goog
>

Re: [PATCH 1/3] t6437: run absorbgitdirs on repos

[Jonathan Tan]

> Quite honestly, if repo_submodule_init() can only work with the
> "absorbed" form, isn't it simply a bug?
> 
> Two independent (i.e. we can do either or both) things may improve
> the situation:
> 
>  - allow repo_submodule_init() to work on both forms (in place---no
>    cheating by calling "absorb" behind user's back).
> 
>  - teach "git submodule add" to transition an old-style submodule
>    into the "absorbed" form (either with an option, or do so by
>    default with an escape hatch to disable).
> 
> The latter indirectly attacks the "repo_submodule_init() can only
> work with absorbed form" issue by making it harder to create a
> non-absorbed submodule to begin with.

Thanks for taking a look.

I took a further look and it doesn't look too difficult to teach
repo_submodule_init() to work on both forms (without calling
"absorbgitdirs") - I'll send an updated patch set soon. As for "git
submodule add" transitioning it into the absorbed form, I'll leave that
for someone else to do - although if repo_submodule_init() supports the
unabsorbed form now, that is not so urgent to do, I think.

Re: [PATCH 0/3] More add_submodule_odb() cleanup in merge code

[Jonathan Tan]

> On Wed, Sep 8, 2021 at 11:18 AM Jonathan Tan <jonathantanmy@google.com> wrote:
> >
> > (CC-ing Elijah in case he has insight into the merge-ort part.)
> 
> All the submodule merging related functions were lifted from
> merge-recursive.c and minimally modified to fit the new structure.
> The only substantive change I made was to fix the merge result for
> virtual merge bases, but that's like one or two lines of code.  In
> particular, everything relating to submodule objects was totally
> untouched...and I think that's reflected in the fact that your PATCH 2
> basically is the same patch twice, once for merge-recursive and once
> for merge-ort.
> 
> I read over PATCH 2 and I didn't find anything that looked
> problematic, but I'm not up-to-speed on the add_submodule_odb and repo
> handling bits of the codebase so I'm not sure I would catch anything.
> But I am encouraged by the fact that it looks like you did the same
> stuff to merge-recursive and merge-ort; I'd be worried you missed
> something if that weren't the case.

Thanks for taking a look.

> As a sidenote, though...
> 
> This does remind me that I noticed that the following functions from
> object-store.h do not take an explicit repository:
> 
> write_object_file()
> hash_object_file()
> hash_object_file_literally()
> force_object_loose()
> 
> I have a patch sitting around somewhere (possibly only still
> referenced in my 'temp' branch) to make repo_*() variants of the above
> functions, and turn the above into simple wrappers of the repo_*()
> variants which just pass the_repository (much like someone else did
> with read_object_file() and repo_read_object_file()).  It also updates
> merge-ort to use the new repo_*() functions.  However, I ended up
> excluding it from my merge-ort submissions since it wasn't necessary.
> Would this be of interest in your submodule work, though?  I guess
> it'd only matter if we started doing real merges of submodules as part
> of a parent repo merge.  (As opposed to the fast-forward-only merges
> that merge-recursive/merge-ort do right now for submodules.)

I think that these functions would be useful if we were to write into
submodules (for example, in a real merge of submodules, as you said).
Right now my submodule work is just making it support partial clones, so
I'm not planning to introduce any new submodule writes. (Well, besides
the fact that a lazy-fetch in a submodule would end up with new objects,
but that's already handled because we do the fetch in a separate process
in the submodule's gitdir.)

[PATCH v2 0/3] More add_submodule_odb() cleanup in merge code

[Jonathan Tan]

This is on jt/grep-wo-submodule-odb-as-alternate (unlike version 1)
because that branch introduces another usage of repo_submodule_init(),
and I need to update it.

The main change from version 1 is that I have introduced a patch (patch
2) that teaches repo_submodule_init() to support submodules with
unabsorbed gitdirs, which means that I no longer need the original patch
1 that updated t6437 to absorb all git dirs.

Once again, all these patches work under
GIT_TEST_MERGE_ALGORITHM=recursive and GIT_TEST_MERGE_ALGORITHM=ort (and
when that envvar is unset, for good measure).

I have also tested it with the "DO NOT SUBMIT" patch 3 from version 1.

Jonathan Tan (3):
  submodule: remove unnecessary unabsorbed fallback
  repository: support unabsorbed in repo_submodule_init
  revision: remove "submodule" from opt struct

 builtin/grep.c                               |  5 +-
 builtin/ls-files.c                           |  4 +-
 builtin/submodule--helper.c                  |  7 +--
 merge-ort.c                                  | 53 +++++++++++++-------
 merge-recursive.c                            | 49 +++++++++++-------
 repository.c                                 | 21 ++++----
 repository.h                                 | 15 +++---
 revision.c                                   | 16 ++----
 revision.h                                   |  1 -
 submodule.c                                  | 24 ++-------
 t/helper/test-submodule-nested-repo-config.c |  4 +-
 11 files changed, 102 insertions(+), 97 deletions(-)

-- 
2.33.0.309.g3052b89438-goog

[PATCH v2 1/3] submodule: remove unnecessary unabsorbed fallback

[Jonathan Tan]

In get_submodule_repo_for(), there is a fallback code path for the case
in which a submodule has an unabsorbed gitdir. (See the documentation
for "git submodule absorbgitdirs" for more information about absorbed
and unabsorbed gitdirs.) However, this code path is unnecessary, because
such submodules are already handled: when the fetch_task is created in
fetch_task_create(), it will create its own struct submodule with a path
and name, and repo_submodule_init() can handle such a struct.

This fallback was introduced in 26f80ccfc1 ("submodule: migrate
get_next_submodule to use repository structs", 2018-12-05). It was
unnecessary even then, but perhaps it escaped notice because its parent
commit d5498e0871 ("repository: repo_submodule_init to take a submodule
struct", 2018-12-05) was the one that taught repo_submodule_init() to
handle such created structs. Before, it took a path and always checked
.gitmodules, so it truly would have failed if there were no entry in
.gitmodules.

(Note to reviewers: in 26f80ccfc1, the "own struct submodule" I
mentioned is in get_next_submodule(), not fetch_task_create().)

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 submodule.c | 15 ++-------------
 1 file changed, 2 insertions(+), 13 deletions(-)

diff --git a/submodule.c b/submodule.c
index 8de1aecaeb..3af3da5b5e 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1409,19 +1409,8 @@ static struct repository *get_submodule_repo_for(struct repository *r,
 	struct repository *ret = xmalloc(sizeof(*ret));
 
 	if (repo_submodule_init(ret, r, sub)) {
-		/*
-		 * No entry in .gitmodules? Technically not a submodule,
-		 * but historically we supported repositories that happen to be
-		 * in-place where a gitlink is. Keep supporting them.
-		 */
-		struct strbuf gitdir = STRBUF_INIT;
-		strbuf_repo_worktree_path(&gitdir, r, "%s/.git", sub->path);
-		if (repo_init(ret, gitdir.buf, NULL)) {
-			strbuf_release(&gitdir);
-			free(ret);
-			return NULL;
-		}
-		strbuf_release(&gitdir);
+		free(ret);
+		return NULL;
 	}
 
 	return ret;
-- 
2.33.0.309.g3052b89438-goog

[PATCH v2 2/3] repository: support unabsorbed in repo_submodule_init

[Jonathan Tan]

In preparation for a subsequent commit that migrates code using
add_submodule_odb() to repo_submodule_init(), teach
repo_submodule_init() to support submodules with unabsorbed gitdirs.
(See the documentation for "git submodule absorbgitdirs" for more
information about absorbed and unabsorbed gitdirs.)

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 builtin/grep.c                               |  5 +----
 builtin/ls-files.c                           |  4 +---
 builtin/submodule--helper.c                  |  7 +------
 repository.c                                 | 21 +++++++++++---------
 repository.h                                 | 15 ++++++++------
 submodule.c                                  |  9 +++------
 t/helper/test-submodule-nested-repo-config.c |  4 +---
 7 files changed, 28 insertions(+), 37 deletions(-)

diff --git a/builtin/grep.c b/builtin/grep.c
index 51278b01fa..8af5249a7b 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -433,17 +433,14 @@ static int grep_submodule(struct grep_opt *opt,
 {
 	struct repository *subrepo;
 	struct repository *superproject = opt->repo;
-	const struct submodule *sub;
 	struct grep_opt subopt;
 	int hit = 0;
 
-	sub = submodule_from_path(superproject, null_oid(), path);
-
 	if (!is_submodule_active(superproject, path))
 		return 0;
 
 	subrepo = xmalloc(sizeof(*subrepo));
-	if (repo_submodule_init(subrepo, superproject, sub)) {
+	if (repo_submodule_init(subrepo, superproject, path, null_oid())) {
 		free(subrepo);
 		return 0;
 	}
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 29a26ad8ae..ec19bf54b2 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -209,10 +209,8 @@ static void show_submodule(struct repository *superproject,
 			   struct dir_struct *dir, const char *path)
 {
 	struct repository subrepo;
-	const struct submodule *sub = submodule_from_path(superproject,
-							  null_oid(), path);
 
-	if (repo_submodule_init(&subrepo, superproject, sub))
+	if (repo_submodule_init(&subrepo, superproject, path, null_oid()))
 		return;
 
 	if (repo_read_index(&subrepo) < 0)
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index ef2776a9e4..6718f202db 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2540,7 +2540,6 @@ static int push_check(int argc, const char **argv, const char *prefix)
 
 static int ensure_core_worktree(int argc, const char **argv, const char *prefix)
 {
-	const struct submodule *sub;
 	const char *path;
 	const char *cw;
 	struct repository subrepo;
@@ -2550,11 +2549,7 @@ static int ensure_core_worktree(int argc, const char **argv, const char *prefix)
 
 	path = argv[1];
 
-	sub = submodule_from_path(the_repository, null_oid(), path);
-	if (!sub)
-		BUG("We could get the submodule handle before?");
-
-	if (repo_submodule_init(&subrepo, the_repository, sub))
+	if (repo_submodule_init(&subrepo, the_repository, path, null_oid()))
 		die(_("could not get a repository handle for submodule '%s'"), path);
 
 	if (!repo_config_get_string_tmp(&subrepo, "core.worktree", &cw)) {
diff --git a/repository.c b/repository.c
index b2bf44c6fa..e4a1afb0ac 100644
--- a/repository.c
+++ b/repository.c
@@ -190,19 +190,15 @@ int repo_init(struct repository *repo,
 
 int repo_submodule_init(struct repository *subrepo,
 			struct repository *superproject,
-			const struct submodule *sub)
+			const char *path,
+			const struct object_id *treeish_name)
 {
 	struct strbuf gitdir = STRBUF_INIT;
 	struct strbuf worktree = STRBUF_INIT;
 	int ret = 0;
 
-	if (!sub) {
-		ret = -1;
-		goto out;
-	}
-
-	strbuf_repo_worktree_path(&gitdir, superproject, "%s/.git", sub->path);
-	strbuf_repo_worktree_path(&worktree, superproject, "%s", sub->path);
+	strbuf_repo_worktree_path(&gitdir, superproject, "%s/.git", path);
+	strbuf_repo_worktree_path(&worktree, superproject, "%s", path);
 
 	if (repo_init(subrepo, gitdir.buf, worktree.buf)) {
 		/*
@@ -212,6 +208,13 @@ int repo_submodule_init(struct repository *subrepo,
 		 * in the superproject's 'modules' directory.  In this case the
 		 * submodule would not have a worktree.
 		 */
+		const struct submodule *sub =
+			submodule_from_path(superproject, treeish_name, path);
+		if (!sub) {
+			ret = -1;
+			goto out;
+		}
+
 		strbuf_reset(&gitdir);
 		strbuf_repo_git_path(&gitdir, superproject,
 				     "modules/%s", sub->name);
@@ -225,7 +228,7 @@ int repo_submodule_init(struct repository *subrepo,
 	subrepo->submodule_prefix = xstrfmt("%s%s/",
 					    superproject->submodule_prefix ?
 					    superproject->submodule_prefix :
-					    "", sub->path);
+					    "", path);
 
 out:
 	strbuf_release(&gitdir);
diff --git a/repository.h b/repository.h
index 3740c93bc0..c24e177c7e 100644
--- a/repository.h
+++ b/repository.h
@@ -172,15 +172,18 @@ void initialize_the_repository(void);
 int repo_init(struct repository *r, const char *gitdir, const char *worktree);
 
 /*
- * Initialize the repository 'subrepo' as the submodule given by the
- * struct submodule 'sub' in parent repository 'superproject'.
- * Return 0 upon success and a non-zero value upon failure, which may happen
- * if the submodule is not found, or 'sub' is NULL.
+ * Initialize the repository 'subrepo' as the submodule at the given path. If
+ * the submodule's gitdir cannot be found at <path>/.git, this function calls
+ * submodule_from_path() to try to find it. treeish_name is only used if
+ * submodule_from_path() needs to be called; see its documentation for more
+ * information.
+ * Return 0 upon success and a non-zero value upon failure.
  */
-struct submodule;
+struct object_id;
 int repo_submodule_init(struct repository *subrepo,
 			struct repository *superproject,
-			const struct submodule *sub);
+			const char *path,
+			const struct object_id *treeish_name);
 void repo_clear(struct repository *repo);
 
 /*
diff --git a/submodule.c b/submodule.c
index 3af3da5b5e..ecda0229af 100644
--- a/submodule.c
+++ b/submodule.c
@@ -520,9 +520,6 @@ static void prepare_submodule_repo_env_in_gitdir(struct strvec *out)
 /*
  * Initialize a repository struct for a submodule based on the provided 'path'.
  *
- * Unlike repo_submodule_init, this tolerates submodules not present
- * in .gitmodules. This function exists only to preserve historical behavior,
- *
  * Returns the repository struct on success,
  * NULL when the submodule is not present.
  */
@@ -1404,11 +1401,11 @@ static void fetch_task_release(struct fetch_task *p)
 }
 
 static struct repository *get_submodule_repo_for(struct repository *r,
-						 const struct submodule *sub)
+						 const char *path)
 {
 	struct repository *ret = xmalloc(sizeof(*ret));
 
-	if (repo_submodule_init(ret, r, sub)) {
+	if (repo_submodule_init(ret, r, path, null_oid())) {
 		free(ret);
 		return NULL;
 	}
@@ -1452,7 +1449,7 @@ static int get_next_submodule(struct child_process *cp,
 			continue;
 		}
 
-		task->repo = get_submodule_repo_for(spf->r, task->sub);
+		task->repo = get_submodule_repo_for(spf->r, task->sub->path);
 		if (task->repo) {
 			struct strbuf submodule_prefix = STRBUF_INIT;
 			child_process_init(cp);
diff --git a/t/helper/test-submodule-nested-repo-config.c b/t/helper/test-submodule-nested-repo-config.c
index e3f11ff5a7..dc1c14bde3 100644
--- a/t/helper/test-submodule-nested-repo-config.c
+++ b/t/helper/test-submodule-nested-repo-config.c
@@ -11,15 +11,13 @@ static void die_usage(const char **argv, const char *msg)
 int cmd__submodule_nested_repo_config(int argc, const char **argv)
 {
 	struct repository subrepo;
-	const struct submodule *sub;
 
 	if (argc < 3)
 		die_usage(argv, "Wrong number of arguments.");
 
 	setup_git_directory();
 
-	sub = submodule_from_path(the_repository, null_oid(), argv[1]);
-	if (repo_submodule_init(&subrepo, the_repository, sub)) {
+	if (repo_submodule_init(&subrepo, the_repository, argv[1], null_oid())) {
 		die_usage(argv, "Submodule not found.");
 	}
 
-- 
2.33.0.309.g3052b89438-goog

[PATCH v2 3/3] revision: remove "submodule" from opt struct

[Jonathan Tan]

Clean up a TODO in revision.h by removing the "submodule" field from
struct setup_revision_opt. This field is only used to specify the ref
store to use, so use rev_info->repo to determine the ref store instead.

The only users of this field are merge-ort.c and merge-recursive.c.
However, both these files specify the superproject as rev_info->repo and
the submodule as setup_revision_opt->submodule. In order to be able to
pass the submodule as rev_info->repo, all commits must be parsed with
the submodule explicitly specified; this patch does that as well. (An
incremental solution in which only some commits are parsed with explicit
submodule will not work, because if the same commit is parsed twice in
different repositories, there will be 2 heap-allocated object structs
corresponding to that commit, and any flag set by the revision walking
mechanism on one of them will not be reflected onto the other.)

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 merge-ort.c       | 53 +++++++++++++++++++++++++++++++----------------
 merge-recursive.c | 49 ++++++++++++++++++++++++++++---------------
 revision.c        | 16 +++++---------
 revision.h        |  1 -
 4 files changed, 72 insertions(+), 47 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 6eb910d6f0..b8efaee8e0 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -32,6 +32,7 @@
 #include "promisor-remote.h"
 #include "revision.h"
 #include "strmap.h"
+#include "submodule-config.h"
 #include "submodule.h"
 #include "tree.h"
 #include "unpack-trees.h"
@@ -1499,7 +1500,6 @@ static int find_first_merges(struct repository *repo,
 	xsnprintf(merged_revision, sizeof(merged_revision), "^%s",
 		  oid_to_hex(&a->object.oid));
 	repo_init_revisions(repo, &revs, NULL);
-	rev_opts.submodule = path;
 	/* FIXME: can't handle linked worktrees in submodules yet */
 	revs.single_worktree = path != NULL;
 	setup_revisions(ARRAY_SIZE(rev_args)-1, rev_args, &revs, &rev_opts);
@@ -1509,7 +1509,7 @@ static int find_first_merges(struct repository *repo,
 		die("revision walk setup failed");
 	while ((commit = get_revision(&revs)) != NULL) {
 		struct object *o = &(commit->object);
-		if (in_merge_bases(b, commit))
+		if (repo_in_merge_bases(repo, b, commit))
 			add_object_array(o, NULL, &merges);
 	}
 	reset_revision_walk();
@@ -1524,7 +1524,7 @@ static int find_first_merges(struct repository *repo,
 		contains_another = 0;
 		for (j = 0; j < merges.nr; j++) {
 			struct commit *m2 = (struct commit *) merges.objects[j].item;
-			if (i != j && in_merge_bases(m2, m1)) {
+			if (i != j && repo_in_merge_bases(repo, m2, m1)) {
 				contains_another = 1;
 				break;
 			}
@@ -1545,10 +1545,12 @@ static int merge_submodule(struct merge_options *opt,
 			   const struct object_id *b,
 			   struct object_id *result)
 {
+	struct repository subrepo;
+	struct strbuf sb = STRBUF_INIT;
+	int ret = 0;
 	struct commit *commit_o, *commit_a, *commit_b;
 	int parent_count;
 	struct object_array merges;
-	struct strbuf sb = STRBUF_INIT;
 
 	int i;
 	int search = !opt->priv->call_depth;
@@ -1564,6 +1566,10 @@ static int merge_submodule(struct merge_options *opt,
 	if (is_null_oid(b))
 		return 0;
 
+	/*
+	 * NEEDSWORK: Remove this when all submodule object accesses are
+	 * through explicitly specified repositores.
+	 */
 	if (add_submodule_odb(path)) {
 		path_msg(opt, path, 0,
 			 _("Failed to merge submodule %s (not checked out)"),
@@ -1571,39 +1577,48 @@ static int merge_submodule(struct merge_options *opt,
 		return 0;
 	}
 
-	if (!(commit_o = lookup_commit_reference(opt->repo, o)) ||
-	    !(commit_a = lookup_commit_reference(opt->repo, a)) ||
-	    !(commit_b = lookup_commit_reference(opt->repo, b))) {
+	if (repo_submodule_init(&subrepo, opt->repo, path, null_oid())) {
+		path_msg(opt, path, 0,
+				_("Failed to merge submodule %s (not checked out)"),
+				path);
+		return 0;
+	}
+
+	if (!(commit_o = lookup_commit_reference(&subrepo, o)) ||
+	    !(commit_a = lookup_commit_reference(&subrepo, a)) ||
+	    !(commit_b = lookup_commit_reference(&subrepo, b))) {
 		path_msg(opt, path, 0,
 			 _("Failed to merge submodule %s (commits not present)"),
 			 path);
-		return 0;
+		goto cleanup;
 	}
 
 	/* check whether both changes are forward */
-	if (!in_merge_bases(commit_o, commit_a) ||
-	    !in_merge_bases(commit_o, commit_b)) {
+	if (!repo_in_merge_bases(&subrepo, commit_o, commit_a) ||
+	    !repo_in_merge_bases(&subrepo, commit_o, commit_b)) {
 		path_msg(opt, path, 0,
 			 _("Failed to merge submodule %s "
 			   "(commits don't follow merge-base)"),
 			 path);
-		return 0;
+		goto cleanup;
 	}
 
 	/* Case #1: a is contained in b or vice versa */
-	if (in_merge_bases(commit_a, commit_b)) {
+	if (repo_in_merge_bases(&subrepo, commit_a, commit_b)) {
 		oidcpy(result, b);
 		path_msg(opt, path, 1,
 			 _("Note: Fast-forwarding submodule %s to %s"),
 			 path, oid_to_hex(b));
-		return 1;
+		ret = 1;
+		goto cleanup;
 	}
-	if (in_merge_bases(commit_b, commit_a)) {
+	if (repo_in_merge_bases(&subrepo, commit_b, commit_a)) {
 		oidcpy(result, a);
 		path_msg(opt, path, 1,
 			 _("Note: Fast-forwarding submodule %s to %s"),
 			 path, oid_to_hex(a));
-		return 1;
+		ret = 1;
+		goto cleanup;
 	}
 
 	/*
@@ -1615,10 +1630,10 @@ static int merge_submodule(struct merge_options *opt,
 
 	/* Skip the search if makes no sense to the calling context.  */
 	if (!search)
-		return 0;
+		goto cleanup;
 
 	/* find commit which merges them */
-	parent_count = find_first_merges(opt->repo, path, commit_a, commit_b,
+	parent_count = find_first_merges(&subrepo, path, commit_a, commit_b,
 					 &merges);
 	switch (parent_count) {
 	case 0:
@@ -1652,7 +1667,9 @@ static int merge_submodule(struct merge_options *opt,
 	}
 
 	object_array_clear(&merges);
-	return 0;
+cleanup:
+	repo_clear(&subrepo);
+	return ret;
 }
 
 static void initialize_attr_index(struct merge_options *opt)
diff --git a/merge-recursive.c b/merge-recursive.c
index 3355d50e8a..fc8ac39d8c 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -24,6 +24,7 @@
 #include "repository.h"
 #include "revision.h"
 #include "string-list.h"
+#include "submodule-config.h"
 #include "submodule.h"
 #include "tag.h"
 #include "tree-walk.h"
@@ -1113,7 +1114,6 @@ static int find_first_merges(struct repository *repo,
 	xsnprintf(merged_revision, sizeof(merged_revision), "^%s",
 		  oid_to_hex(&a->object.oid));
 	repo_init_revisions(repo, &revs, NULL);
-	rev_opts.submodule = path;
 	/* FIXME: can't handle linked worktrees in submodules yet */
 	revs.single_worktree = path != NULL;
 	setup_revisions(ARRAY_SIZE(rev_args)-1, rev_args, &revs, &rev_opts);
@@ -1123,7 +1123,7 @@ static int find_first_merges(struct repository *repo,
 		die("revision walk setup failed");
 	while ((commit = get_revision(&revs)) != NULL) {
 		struct object *o = &(commit->object);
-		if (in_merge_bases(b, commit))
+		if (repo_in_merge_bases(repo, b, commit))
 			add_object_array(o, NULL, &merges);
 	}
 	reset_revision_walk();
@@ -1138,7 +1138,7 @@ static int find_first_merges(struct repository *repo,
 		contains_another = 0;
 		for (j = 0; j < merges.nr; j++) {
 			struct commit *m2 = (struct commit *) merges.objects[j].item;
-			if (i != j && in_merge_bases(m2, m1)) {
+			if (i != j && repo_in_merge_bases(repo, m2, m1)) {
 				contains_another = 1;
 				break;
 			}
@@ -1174,6 +1174,8 @@ static int merge_submodule(struct merge_options *opt,
 			   const struct object_id *base, const struct object_id *a,
 			   const struct object_id *b)
 {
+	struct repository subrepo;
+	int ret = 0;
 	struct commit *commit_base, *commit_a, *commit_b;
 	int parent_count;
 	struct object_array merges;
@@ -1197,27 +1199,36 @@ static int merge_submodule(struct merge_options *opt,
 	if (is_null_oid(b))
 		return 0;
 
+	/*
+	 * NEEDSWORK: Remove this when all submodule object accesses are
+	 * through explicitly specified repositores.
+	 */
 	if (add_submodule_odb(path)) {
 		output(opt, 1, _("Failed to merge submodule %s (not checked out)"), path);
 		return 0;
 	}
 
-	if (!(commit_base = lookup_commit_reference(opt->repo, base)) ||
-	    !(commit_a = lookup_commit_reference(opt->repo, a)) ||
-	    !(commit_b = lookup_commit_reference(opt->repo, b))) {
-		output(opt, 1, _("Failed to merge submodule %s (commits not present)"), path);
+	if (repo_submodule_init(&subrepo, opt->repo, path, null_oid())) {
+		output(opt, 1, _("Failed to merge submodule %s (not checked out)"), path);
 		return 0;
 	}
 
+	if (!(commit_base = lookup_commit_reference(&subrepo, base)) ||
+	    !(commit_a = lookup_commit_reference(&subrepo, a)) ||
+	    !(commit_b = lookup_commit_reference(&subrepo, b))) {
+		output(opt, 1, _("Failed to merge submodule %s (commits not present)"), path);
+		goto cleanup;
+	}
+
 	/* check whether both changes are forward */
-	if (!in_merge_bases(commit_base, commit_a) ||
-	    !in_merge_bases(commit_base, commit_b)) {
+	if (!repo_in_merge_bases(&subrepo, commit_base, commit_a) ||
+	    !repo_in_merge_bases(&subrepo, commit_base, commit_b)) {
 		output(opt, 1, _("Failed to merge submodule %s (commits don't follow merge-base)"), path);
-		return 0;
+		goto cleanup;
 	}
 
 	/* Case #1: a is contained in b or vice versa */
-	if (in_merge_bases(commit_a, commit_b)) {
+	if (repo_in_merge_bases(&subrepo, commit_a, commit_b)) {
 		oidcpy(result, b);
 		if (show(opt, 3)) {
 			output(opt, 3, _("Fast-forwarding submodule %s to the following commit:"), path);
@@ -1227,9 +1238,10 @@ static int merge_submodule(struct merge_options *opt,
 		else
 			; /* no output */
 
-		return 1;
+		ret = 1;
+		goto cleanup;
 	}
-	if (in_merge_bases(commit_b, commit_a)) {
+	if (repo_in_merge_bases(&subrepo, commit_b, commit_a)) {
 		oidcpy(result, a);
 		if (show(opt, 3)) {
 			output(opt, 3, _("Fast-forwarding submodule %s to the following commit:"), path);
@@ -1239,7 +1251,8 @@ static int merge_submodule(struct merge_options *opt,
 		else
 			; /* no output */
 
-		return 1;
+		ret = 1;
+		goto cleanup;
 	}
 
 	/*
@@ -1251,10 +1264,10 @@ static int merge_submodule(struct merge_options *opt,
 
 	/* Skip the search if makes no sense to the calling context.  */
 	if (!search)
-		return 0;
+		goto cleanup;
 
 	/* find commit which merges them */
-	parent_count = find_first_merges(opt->repo, &merges, path,
+	parent_count = find_first_merges(&subrepo, &merges, path,
 					 commit_a, commit_b);
 	switch (parent_count) {
 	case 0:
@@ -1281,7 +1294,9 @@ static int merge_submodule(struct merge_options *opt,
 	}
 
 	object_array_clear(&merges);
-	return 0;
+cleanup:
+	repo_clear(&subrepo);
+	return ret;
 }
 
 static int merge_mode_and_contents(struct merge_options *opt,
diff --git a/revision.c b/revision.c
index cddd0542a6..31fc1884d2 100644
--- a/revision.c
+++ b/revision.c
@@ -2561,8 +2561,7 @@ static int for_each_good_bisect_ref(struct ref_store *refs, each_ref_fn fn, void
 	return for_each_bisect_ref(refs, fn, cb_data, term_good);
 }
 
-static int handle_revision_pseudo_opt(const char *submodule,
-				      struct rev_info *revs,
+static int handle_revision_pseudo_opt(struct rev_info *revs,
 				      const char **argv, int *flags)
 {
 	const char *arg = argv[0];
@@ -2570,7 +2569,7 @@ static int handle_revision_pseudo_opt(const char *submodule,
 	struct ref_store *refs;
 	int argcount;
 
-	if (submodule) {
+	if (revs->repo != the_repository) {
 		/*
 		 * We need some something like get_submodule_worktrees()
 		 * before we can go through all worktrees of a submodule,
@@ -2579,9 +2578,8 @@ static int handle_revision_pseudo_opt(const char *submodule,
 		 */
 		if (!revs->single_worktree)
 			BUG("--single-worktree cannot be used together with submodule");
-		refs = get_submodule_ref_store(submodule);
-	} else
-		refs = get_main_ref_store(revs->repo);
+	}
+	refs = get_main_ref_store(revs->repo);
 
 	/*
 	 * NOTE!
@@ -2699,12 +2697,8 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s
 {
 	int i, flags, left, seen_dashdash, revarg_opt;
 	struct strvec prune_data = STRVEC_INIT;
-	const char *submodule = NULL;
 	int seen_end_of_options = 0;
 
-	if (opt)
-		submodule = opt->submodule;
-
 	/* First, search for "--" */
 	if (opt && opt->assume_dashdash) {
 		seen_dashdash = 1;
@@ -2733,7 +2727,7 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s
 		if (!seen_end_of_options && *arg == '-') {
 			int opts;
 
-			opts = handle_revision_pseudo_opt(submodule,
+			opts = handle_revision_pseudo_opt(
 						revs, argv + i,
 						&flags);
 			if (opts > 0) {
diff --git a/revision.h b/revision.h
index fbb068da9f..d8e0e93026 100644
--- a/revision.h
+++ b/revision.h
@@ -339,7 +339,6 @@ extern volatile show_early_output_fn_t show_early_output;
 struct setup_revision_opt {
 	const char *def;
 	void (*tweak)(struct rev_info *, struct setup_revision_opt *);
-	const char *submodule;	/* TODO: drop this and use rev_info->repo */
 	unsigned int	assume_dashdash:1,
 			allow_exclude_promisor_objects:1;
 	unsigned revarg_opt;
-- 
2.33.0.309.g3052b89438-goog

Re: [PATCH v2 0/3] More add_submodule_odb() cleanup in merge code

[Elijah Newren]

On Thu, Sep 9, 2021 at 11:50 AM Jonathan Tan <jonathantanmy@google.com> wrote:
>
> This is on jt/grep-wo-submodule-odb-as-alternate (unlike version 1)
> because that branch introduces another usage of repo_submodule_init(),
> and I need to update it.
>
> The main change from version 1 is that I have introduced a patch (patch
> 2) that teaches repo_submodule_init() to support submodules with
> unabsorbed gitdirs, which means that I no longer need the original patch
> 1 that updated t6437 to absorb all git dirs.
>
> Once again, all these patches work under
> GIT_TEST_MERGE_ALGORITHM=recursive and GIT_TEST_MERGE_ALGORITHM=ort (and
> when that envvar is unset, for good measure).
>
> I have also tested it with the "DO NOT SUBMIT" patch 3 from version 1.

While I don't yet know the details of repositories and submodules, I
know the merge-recursive and merge-ort side of things and those
changes all look reasonable to me.  So here's an Ack for that side of
patch 3.

Despite not being familiar with the repository and submodule storage
parts of the code, I took a look over the remainder of the 3 patches
to look for anything that might stand out and look amiss.  I didn't
spot anything; it all looks reasonable to me.