git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/6] clone, submodule update: check out submodule branches
@ 2022-08-29 20:54 Glen Choo via GitGitGadget
  2022-08-29 20:54 ` [PATCH 1/6] clone: teach --detach option Glen Choo via GitGitGadget
                   ` (6 more replies)
  0 siblings, 7 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-08-29 20:54 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Glen Choo

= Description

This series teaches "git clone --recurse-submodules" and "git submodule
update" to understand "submodule.propagateBranches" (see Further Reading for
context), i.e. if the superproject has a branch checked out and a submodule
is cloned, the submodule will have the same branch checked out.

To do this, "git submodule update" checks if "submodule.propagateBranches"
is true, and if so, gets the current superproject branch and updates the
submodule by running "git checkout -b my-superproject-branch". Since "git
clone --recurse-submodules" is implemented using "git submodule update", it
also learns this trick.

The main challenges with this approach are:

 * If the remote HEAD points to a branch, "git clone" always creates that
   branch in the clone. But with "submodule.propagateBranches", we want
   submodules to use the branch names of their superproject, not their
   upstream.
   
   This is solved by adding a new flag to "git clone", "--detach", which
   detaches the clone's HEAD at the branch and does not create it.

 * When "git submodule update" recurses into submodules, the parent process
   has to propagate the value of "submodule.propagateBranches" to child
   processes, otherwise the behavior will be inconsistent if the submodule
   has the config unset.
   
   This is solved by adding an internal GIT_* environment variable and
   passing it down via prepare_submodule_repo_env(). This is cleaner than
   passing "-c submodule.propagateBranches=true", but an even cleaner
   solution would be for submodules to read "submodule.propagateBranches"
   from their superproject config. This would also be useful for
   "submodule.alternateLocation" and "submodule.alternateErrorStrategy", as
   we wouldn't have to set those values in newly-cloned submodules. This
   requires teaching Git to treat submodules differently, which was the
   subject of some WIP in [1]. That topic has stalled, but I don't mind
   restarting it if others prefer that.

[1]
https://lore.kernel.org/git/20220310004423.2627181-1-emilyshaffer@google.com/

= Patch organization

 * Patch 1/6 adds "--detach" to "git clone"
 * Patch 2/6 creates the environment variable and repository setting for
   "submodule.propagateBranches"
 * Patches 3-5/6 are prep work, and 6/6 adds the actual
   "submodule.propagateBranches" behavior

= Future work

 * Patch 4, which refactors resolve_gitlink_ref(), notes that a better
   interface would be to return the refname instead of using an "out"
   parameter, but we use an "out" parameter so that any new callers trying
   to use the old function signature will get stopped by the compiler. The
   refactor can be finished at a later time.

= Further reading

Submodule branching RFC:
https://lore.kernel.org/git/kl6lv912uvjv.fsf@chooglen-macbookpro.roam.corp.google.com/

Original Submodule UX RFC/Discussion:
https://lore.kernel.org/git/YHofmWcIAidkvJiD@google.com/

Contributor Summit submodules Notes:
https://lore.kernel.org/git/nycvar.QRO.7.76.6.2110211148060.56@tvgsbejvaqbjf.bet/

Submodule UX overhaul updates:
https://lore.kernel.org/git/?q=Submodule+UX+overhaul+update

"git branch --recurse-submodules":
https://lore.kernel.org/git/20220129000446.99261-1-chooglen@google.com/

Glen Choo (6):
  clone: teach --detach option
  repo-settings: add submodule_propagate_branches
  t5617: drop references to remote-tracking branches
  submodule: return target of submodule symref
  submodule--helper: refactor up-to-date criterion
  clone, submodule update: check out branches

 Documentation/git-clone.txt                   |  7 +++-
 builtin/branch.c                              | 11 ++----
 builtin/clone.c                               | 12 ++++--
 builtin/submodule--helper.c                   | 37 +++++++++++++++++--
 builtin/update-index.c                        |  4 +-
 cache.h                                       |  1 +
 combine-diff.c                                |  3 +-
 diff-lib.c                                    |  2 +-
 dir.c                                         |  2 +-
 object-file.c                                 |  2 +-
 read-cache.c                                  |  4 +-
 refs.c                                        | 10 +++--
 refs.h                                        |  5 ++-
 repo-settings.c                               | 11 +++---
 repository.h                                  |  1 +
 submodule.c                                   |  2 +
 t/t5601-clone.sh                              | 22 +++++++++++
 ...es-remote.sh => t5617-clone-submodules.sh} | 36 +++++++++++++++++-
 t/t7406-submodule-update.sh                   | 22 +++++++++++
 unpack-trees.c                                |  3 +-
 20 files changed, 161 insertions(+), 36 deletions(-)
 rename t/{t5617-clone-submodules-remote.sh => t5617-clone-submodules.sh} (72%)


base-commit: 9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1321%2Fchooglen%2Fsubmodule%2Fclone-recursive-with-branch-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1321/chooglen/submodule/clone-recursive-with-branch-v1
Pull-Request: https://github.com/git/git/pull/1321
-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 56+ messages in thread

* [PATCH 1/6] clone: teach --detach option
  2022-08-29 20:54 [PATCH 0/6] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
@ 2022-08-29 20:54 ` Glen Choo via GitGitGadget
  2022-08-30  4:02   ` Philippe Blain
  2022-08-29 20:54 ` [PATCH 2/6] repo-settings: add submodule_propagate_branches Glen Choo via GitGitGadget
                   ` (5 subsequent siblings)
  6 siblings, 1 reply; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-08-29 20:54 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

Teach "git clone" the "--detach" option, which leaves the cloned repo in
detached HEAD (like "git checkout --detach"). If the clone is not bare,
the remote's HEAD branch is also not created (bare clones always copy
all remote branches directly to local branches, so the branch is still
created in the bare case).

This is especially useful in the "submodule.propagateBranches" workflow,
where the submodule branch names match the superproject's branch names,
so it makes no sense to name the branches after the submodule's remote's
branches.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 Documentation/git-clone.txt |  7 ++++++-
 builtin/clone.c             | 12 +++++++++---
 t/t5601-clone.sh            | 22 ++++++++++++++++++++++
 3 files changed, 37 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 632bd1348ea..a3af90824b6 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -16,7 +16,7 @@ SYNOPSIS
 	  [--depth <depth>] [--[no-]single-branch] [--no-tags]
 	  [--recurse-submodules[=<pathspec>]] [--[no-]shallow-submodules]
 	  [--[no-]remote-submodules] [--jobs <n>] [--sparse] [--[no-]reject-shallow]
-	  [--filter=<filter> [--also-filter-submodules]] [--] <repository>
+	  [--filter=<filter> [--also-filter-submodules] [--detach]] [--] <repository>
 	  [<directory>]
 
 DESCRIPTION
@@ -210,6 +210,11 @@ objects from the source repository into a pack in the cloned repository.
 	`--branch` can also take tags and detaches the HEAD at that commit
 	in the resulting repository.
 
+--detach::
+	If the cloned repository's HEAD points to a branch, point the newly
+	created HEAD to the branch's commit instead of the branch itself. In a
+	non-bare repository, the branch will not be created.
+
 -u <upload-pack>::
 --upload-pack <upload-pack>::
 	When given, and the repository to clone from is accessed
diff --git a/builtin/clone.c b/builtin/clone.c
index c4ff4643ecd..1bc1807360e 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -77,6 +77,7 @@ static int option_filter_submodules = -1;    /* unspecified */
 static int config_filter_submodules = -1;    /* unspecified */
 static struct string_list server_options = STRING_LIST_INIT_NODUP;
 static int option_remote_submodules;
+static int option_detach;
 
 static int recurse_submodules_cb(const struct option *opt,
 				 const char *arg, int unset)
@@ -160,6 +161,8 @@ static struct option builtin_clone_options[] = {
 		    N_("any cloned submodules will use their remote-tracking branch")),
 	OPT_BOOL(0, "sparse", &option_sparse_checkout,
 		    N_("initialize sparse-checkout file to include only files at root")),
+	OPT_BOOL(0, "detach", &option_detach,
+		 N_("detach HEAD and don't create branch")),
 	OPT_END()
 };
 
@@ -607,10 +610,12 @@ static void update_remote_refs(const struct ref *refs,
 }
 
 static void update_head(const struct ref *our, const struct ref *remote,
-			const char *unborn, const char *msg)
+			const char *unborn, int should_detach,
+			const char *msg)
 {
 	const char *head;
-	if (our && skip_prefix(our->name, "refs/heads/", &head)) {
+	if (our && !should_detach &&
+	    skip_prefix(our->name, "refs/heads/", &head)) {
 		/* Local default branch link */
 		if (create_symref("HEAD", our->name, NULL) < 0)
 			die(_("unable to update HEAD"));
@@ -1339,7 +1344,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 			   branch_top.buf, reflog_msg.buf, transport,
 			   !is_local);
 
-	update_head(our_head_points_at, remote_head, unborn_head, reflog_msg.buf);
+	update_head(our_head_points_at, remote_head, unborn_head,
+		    option_detach, reflog_msg.buf);
 
 	/*
 	 * We want to show progress for recursive submodule clones iff
diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh
index cf3be0584f4..1e7e5143a76 100755
--- a/t/t5601-clone.sh
+++ b/t/t5601-clone.sh
@@ -333,6 +333,28 @@ test_expect_success 'clone checking out a tag' '
 	test_cmp fetch.expected fetch.actual
 '
 
+test_expect_success '--detach detaches and does not create branch' '
+	test_when_finished "rm -fr dst" &&
+	git clone --detach src dst &&
+	(
+		cd dst &&
+		test_must_fail git rev-parse main &&
+		test_must_fail git symbolic-ref HEAD &&
+		test_cmp_rev HEAD refs/remotes/origin/HEAD
+	)
+'
+
+test_expect_success '--detach with --bare detaches but creates branch' '
+	test_when_finished "rm -fr dst" &&
+	git clone --bare --detach src dst &&
+	(
+		cd dst &&
+		git rev-parse main &&
+		test_must_fail git symbolic-ref HEAD &&
+		test_cmp_rev HEAD refs/heads/main
+	)
+'
+
 test_expect_success 'set up ssh wrapper' '
 	cp "$GIT_BUILD_DIR/t/helper/test-fake-ssh$X" \
 		"$TRASH_DIRECTORY/ssh$X" &&
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH 2/6] repo-settings: add submodule_propagate_branches
  2022-08-29 20:54 [PATCH 0/6] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
  2022-08-29 20:54 ` [PATCH 1/6] clone: teach --detach option Glen Choo via GitGitGadget
@ 2022-08-29 20:54 ` Glen Choo via GitGitGadget
  2022-08-30  4:02   ` Philippe Blain
  2022-08-29 20:54 ` [PATCH 3/6] t5617: drop references to remote-tracking branches Glen Choo via GitGitGadget
                   ` (4 subsequent siblings)
  6 siblings, 1 reply; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-08-29 20:54 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

When processes recurse into submodules, the child processes have to
use the same value of "submodule.propagateBranches" as the parent
process regardless of whether the process is spawned in the superproject
or submodule, otherwise the behavior may be inconsistent if the
repositories don't agree on the config.

We haven't needed a way to propagate the config because because the only
command that reads "submodule.propagateBranches" is "git branch", which
only has one mode of operation with "--recurse-submodules". However, a
future commit will teach "submodule.propagateBranches" to "git submodule
update", making this necessary.

Propagate "submodule.propagateBranches" to child processes by adding a
corresponding GIT_INTERNAL_* environment variable and repository
setting, and setting the environment variable inside
prepare_submodule_repo_env(). Then, refactor builtin/branch.c to read
the repository setting.

Using an internal environment variable is a potentially leaky
abstraction because environment variables can come from sources besides
the parent process. A more robust solution would be to teach Git that
the repository is a submodule and to only read
"submodule.propagateBranches" from the superproject config. There is WIP
for this on the ML [1].

Another alternative would be to pass "-c submodule.propagateBranches" to
all child processes. This is error-prone because many different
processes are invoked directly or indirectly by "git submodule update"
(e.g. "git submodule--helper clone", "git clone", "git checkout"). With
an environment variable, we can avoid this work because
prepare_submodule_repo_env() is already called for submodule child
processes.

[1] https://lore.kernel.org/git/20220310004423.2627181-1-emilyshaffer@google.com/

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/branch.c | 11 +++--------
 cache.h          |  1 +
 repo-settings.c  | 11 ++++++-----
 repository.h     |  1 +
 submodule.c      |  2 ++
 5 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index 55cd9a6e998..1201e41ef8c 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -40,7 +40,6 @@ static const char * const builtin_branch_usage[] = {
 static const char *head;
 static struct object_id head_oid;
 static int recurse_submodules = 0;
-static int submodule_propagate_branches = 0;
 
 static int branch_use_color = -1;
 static char branch_colors[][COLOR_MAXLEN] = {
@@ -106,10 +105,6 @@ static int git_branch_config(const char *var, const char *value, void *cb)
 		recurse_submodules = git_config_bool(var, value);
 		return 0;
 	}
-	if (!strcasecmp(var, "submodule.propagateBranches")) {
-		submodule_propagate_branches = git_config_bool(var, value);
-		return 0;
-	}
 
 	return git_color_default_config(var, value, cb);
 }
@@ -714,7 +709,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, builtin_branch_usage,
 			     0);
-
+	prepare_repo_settings(the_repository);
 	if (!delete && !rename && !copy && !edit_description && !new_upstream &&
 	    !show_current && !unset_upstream && argc == 0)
 		list = 1;
@@ -730,7 +725,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 		usage_with_options(builtin_branch_usage, options);
 
 	if (recurse_submodules_explicit) {
-		if (!submodule_propagate_branches)
+		if (!the_repository->settings.submodule_propagate_branches)
 			die(_("branch with --recurse-submodules can only be used if submodule.propagateBranches is enabled"));
 		if (noncreate_actions)
 			die(_("--recurse-submodules can only be used to create branches"));
@@ -738,7 +733,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 
 	recurse_submodules =
 		(recurse_submodules || recurse_submodules_explicit) &&
-		submodule_propagate_branches;
+		the_repository->settings.submodule_propagate_branches;
 
 	if (filter.abbrev == -1)
 		filter.abbrev = DEFAULT_ABBREV;
diff --git a/cache.h b/cache.h
index 302810b353a..023f8feba42 100644
--- a/cache.h
+++ b/cache.h
@@ -505,6 +505,7 @@ static inline enum object_type object_type(unsigned int mode)
 #define GIT_WORK_TREE_ENVIRONMENT "GIT_WORK_TREE"
 #define GIT_PREFIX_ENVIRONMENT "GIT_PREFIX"
 #define GIT_SUPER_PREFIX_ENVIRONMENT "GIT_INTERNAL_SUPER_PREFIX"
+#define GIT_SUBMODULE_PROPAGATE_BRANCHES_ENVIRONMENT "GIT_INTERNAL_SUBMODULE_PROPAGATE_BRANCHES"
 #define DEFAULT_GIT_DIR_ENVIRONMENT ".git"
 #define DB_ENVIRONMENT "GIT_OBJECT_DIRECTORY"
 #define INDEX_ENVIRONMENT "GIT_INDEX_FILE"
diff --git a/repo-settings.c b/repo-settings.c
index 43bc881bfc9..2fde9bcdbf6 100644
--- a/repo-settings.c
+++ b/repo-settings.c
@@ -60,16 +60,17 @@ void prepare_repo_settings(struct repository *r)
 	repo_cfg_bool(r, "pack.usesparse", &r->settings.pack_use_sparse, 1);
 	repo_cfg_bool(r, "core.multipackindex", &r->settings.core_multi_pack_index, 1);
 	repo_cfg_bool(r, "index.sparse", &r->settings.sparse_index, 0);
+	repo_cfg_bool(r, "submodule.propagateBranches", &r->settings.submodule_propagate_branches, 0);
 
 	/*
-	 * The GIT_TEST_MULTI_PACK_INDEX variable is special in that
-	 * either it *or* the config sets
-	 * r->settings.core_multi_pack_index if true. We don't take
-	 * the environment variable if it exists (even if false) over
-	 * any config, as in most other cases.
+	 * Boolean settings from config and environment variables. Only
+	 * take the environment variable if it is true, otherwise, use
+	 * the config.
 	 */
 	if (git_env_bool(GIT_TEST_MULTI_PACK_INDEX, 0))
 		r->settings.core_multi_pack_index = 1;
+	if (git_env_bool(GIT_SUBMODULE_PROPAGATE_BRANCHES_ENVIRONMENT, 0))
+		r->settings.submodule_propagate_branches = 1;
 
 	/*
 	 * Non-boolean config
diff --git a/repository.h b/repository.h
index 797f471cce9..bf36b9ee6d7 100644
--- a/repository.h
+++ b/repository.h
@@ -36,6 +36,7 @@ struct repo_settings {
 	int fetch_write_commit_graph;
 	int command_requires_full_index;
 	int sparse_index;
+	int submodule_propagate_branches;
 
 	struct fsmonitor_settings *fsmonitor; /* lazily loaded */
 
diff --git a/submodule.c b/submodule.c
index 3fa5db3ecdf..510c1b04a76 100644
--- a/submodule.c
+++ b/submodule.c
@@ -504,6 +504,8 @@ static void print_submodule_diff_summary(struct repository *r, struct rev_info *
 
 void prepare_submodule_repo_env(struct strvec *out)
 {
+	if (the_repository->settings.submodule_propagate_branches)
+		strvec_pushf(out, "%s=1", GIT_SUBMODULE_PROPAGATE_BRANCHES_ENVIRONMENT);
 	prepare_other_repo_env(out, DEFAULT_GIT_DIR_ENVIRONMENT);
 }
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH 3/6] t5617: drop references to remote-tracking branches
  2022-08-29 20:54 [PATCH 0/6] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
  2022-08-29 20:54 ` [PATCH 1/6] clone: teach --detach option Glen Choo via GitGitGadget
  2022-08-29 20:54 ` [PATCH 2/6] repo-settings: add submodule_propagate_branches Glen Choo via GitGitGadget
@ 2022-08-29 20:54 ` Glen Choo via GitGitGadget
  2022-08-30  4:03   ` Philippe Blain
  2022-08-29 20:54 ` [PATCH 4/6] submodule: return target of submodule symref Glen Choo via GitGitGadget
                   ` (3 subsequent siblings)
  6 siblings, 1 reply; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-08-29 20:54 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

It has included submodule cloning tests without remote-tracking branches
tests since f05da2b48b (clone, submodule: pass partial clone filters to
submodules, 2022-02-04) at least. Rename it accordingly so that we can
put future submodule cloning tests there.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 ...617-clone-submodules-remote.sh => t5617-clone-submodules.sh} | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 rename t/{t5617-clone-submodules-remote.sh => t5617-clone-submodules.sh} (97%)

diff --git a/t/t5617-clone-submodules-remote.sh b/t/t5617-clone-submodules.sh
similarity index 97%
rename from t/t5617-clone-submodules-remote.sh
rename to t/t5617-clone-submodules.sh
index ca8f80083a2..b5c66cb18cb 100755
--- a/t/t5617-clone-submodules-remote.sh
+++ b/t/t5617-clone-submodules.sh
@@ -1,6 +1,6 @@
 #!/bin/sh
 
-test_description='Test cloning repos with submodules using remote-tracking branches'
+test_description='Test cloning repos with submodules'
 
 GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
 export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH 4/6] submodule: return target of submodule symref
  2022-08-29 20:54 [PATCH 0/6] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
                   ` (2 preceding siblings ...)
  2022-08-29 20:54 ` [PATCH 3/6] t5617: drop references to remote-tracking branches Glen Choo via GitGitGadget
@ 2022-08-29 20:54 ` Glen Choo via GitGitGadget
  2022-09-01 20:01   ` Jonathan Tan
  2022-08-29 20:54 ` [PATCH 5/6] submodule--helper: refactor up-to-date criterion Glen Choo via GitGitGadget
                   ` (2 subsequent siblings)
  6 siblings, 1 reply; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-08-29 20:54 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

resolve_gitlink_ref() can tell us which oid the submodule ref is
pointing to, but in a future commit, we would also like to know the
symbolic ref target if we are checking a symbolic ref. Teach
resolve_gitlink_ref() to "return" the symbolic ref's target via an "out"
parameter.

This changes resolve_gitlink_ref()'s signature so that new callers
trying to use the old signature will be stopped by the compiler. If we
returned the target instead (just like refs_resolve_ref_unsafe()), we
would be more consistent with refs_resolve_ref_unsafe(), but callers
expecting the old signature will get the opposite return value from what
they expect (since exit code 0 means success, but NULL pointer means
failure). We should do this refactor once we think that nobody will try
to use the old signature.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/submodule--helper.c |  8 +++++---
 builtin/update-index.c      |  4 ++--
 combine-diff.c              |  3 ++-
 diff-lib.c                  |  2 +-
 dir.c                       |  2 +-
 object-file.c               |  2 +-
 read-cache.c                |  4 ++--
 refs.c                      | 10 ++++++----
 refs.h                      |  5 ++++-
 unpack-trees.c              |  3 ++-
 10 files changed, 26 insertions(+), 17 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index b63f420ecef..dd00a0db522 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2466,7 +2466,8 @@ static int update_submodule(struct update_data *update_data)
 
 	if (update_data->just_cloned)
 		oidcpy(&update_data->suboid, null_oid());
-	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD", &update_data->suboid))
+	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
+				     &update_data->suboid, NULL))
 		die(_("Unable to find current revision in submodule path '%s'"),
 			update_data->displaypath);
 
@@ -2482,7 +2483,8 @@ static int update_submodule(struct update_data *update_data)
 				    update_data->sm_path);
 		}
 
-		if (resolve_gitlink_ref(update_data->sm_path, remote_ref, &update_data->oid))
+		if (resolve_gitlink_ref(update_data->sm_path, remote_ref,
+					&update_data->oid, NULL))
 			die(_("Unable to find %s revision in submodule path '%s'"),
 			    remote_ref, update_data->sm_path);
 
@@ -3231,7 +3233,7 @@ static void die_on_repo_without_commits(const char *path)
 	strbuf_addstr(&sb, path);
 	if (is_nonbare_repository_dir(&sb)) {
 		struct object_id oid;
-		if (resolve_gitlink_ref(path, "HEAD", &oid) < 0)
+		if (resolve_gitlink_ref(path, "HEAD", &oid, NULL) < 0)
 			die(_("'%s' does not have a commit checked out"), path);
 	}
 	strbuf_release(&sb);
diff --git a/builtin/update-index.c b/builtin/update-index.c
index b62249905f1..19a21a4586c 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -339,7 +339,7 @@ static int process_directory(const char *path, int len, struct stat *st)
 		if (S_ISGITLINK(ce->ce_mode)) {
 
 			/* Do nothing to the index if there is no HEAD! */
-			if (resolve_gitlink_ref(path, "HEAD", &oid) < 0)
+			if (resolve_gitlink_ref(path, "HEAD", &oid, NULL) < 0)
 				return 0;
 
 			return add_one_path(ce, path, len, st);
@@ -365,7 +365,7 @@ static int process_directory(const char *path, int len, struct stat *st)
 	}
 
 	/* No match - should we add it as a gitlink? */
-	if (!resolve_gitlink_ref(path, "HEAD", &oid))
+	if (!resolve_gitlink_ref(path, "HEAD", &oid, NULL))
 		return add_one_path(NULL, path, len, st);
 
 	/* Error out. */
diff --git a/combine-diff.c b/combine-diff.c
index b0ece954808..88efcaeefa7 100644
--- a/combine-diff.c
+++ b/combine-diff.c
@@ -1060,7 +1060,8 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,
 			elem->mode = canon_mode(st.st_mode);
 		} else if (S_ISDIR(st.st_mode)) {
 			struct object_id oid;
-			if (resolve_gitlink_ref(elem->path, "HEAD", &oid) < 0)
+			if (resolve_gitlink_ref(elem->path, "HEAD", &oid,
+						NULL) < 0)
 				result = grab_blob(opt->repo, &elem->oid,
 						   elem->mode, &result_size,
 						   NULL, NULL);
diff --git a/diff-lib.c b/diff-lib.c
index 7eb66a417aa..0c1f76d5fe9 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -53,7 +53,7 @@ static int check_removed(const struct index_state *istate, const struct cache_en
 		 * a directory --- the blob was removed!
 		 */
 		if (!S_ISGITLINK(ce->ce_mode) &&
-		    resolve_gitlink_ref(ce->name, "HEAD", &sub))
+		    resolve_gitlink_ref(ce->name, "HEAD", &sub, NULL))
 			return 1;
 	}
 	return 0;
diff --git a/dir.c b/dir.c
index d7cfb08e441..33368b399e2 100644
--- a/dir.c
+++ b/dir.c
@@ -3255,7 +3255,7 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	struct object_id submodule_head;
 
 	if ((flag & REMOVE_DIR_KEEP_NESTED_GIT) &&
-	    !resolve_gitlink_ref(path->buf, "HEAD", &submodule_head)) {
+	    !resolve_gitlink_ref(path->buf, "HEAD", &submodule_head, NULL)) {
 		/* Do not descend and nuke a nested git work tree. */
 		if (kept_up)
 			*kept_up = 1;
diff --git a/object-file.c b/object-file.c
index 5b270f046dd..a10a49d7a2a 100644
--- a/object-file.c
+++ b/object-file.c
@@ -2526,7 +2526,7 @@ int index_path(struct index_state *istate, struct object_id *oid,
 		strbuf_release(&sb);
 		break;
 	case S_IFDIR:
-		return resolve_gitlink_ref(path, "HEAD", oid);
+		return resolve_gitlink_ref(path, "HEAD", oid, NULL);
 	default:
 		return error(_("%s: unsupported file type"), path);
 	}
diff --git a/read-cache.c b/read-cache.c
index 4de207752dc..4b68bd0c9a6 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -285,7 +285,7 @@ static int ce_compare_gitlink(const struct cache_entry *ce)
 	 *
 	 * If so, we consider it always to match.
 	 */
-	if (resolve_gitlink_ref(ce->name, "HEAD", &oid) < 0)
+	if (resolve_gitlink_ref(ce->name, "HEAD", &oid, NULL) < 0)
 		return 0;
 	return !oideq(&oid, &ce->oid);
 }
@@ -776,7 +776,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,
 
 	namelen = strlen(path);
 	if (S_ISDIR(st_mode)) {
-		if (resolve_gitlink_ref(path, "HEAD", &oid) < 0)
+		if (resolve_gitlink_ref(path, "HEAD", &oid, NULL) < 0)
 			return error(_("'%s' does not have a commit checked out"), path);
 		while (namelen && path[namelen-1] == '/')
 			namelen--;
diff --git a/refs.c b/refs.c
index 90bcb271687..d72015c95e9 100644
--- a/refs.c
+++ b/refs.c
@@ -1784,19 +1784,21 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
 }
 
 int resolve_gitlink_ref(const char *submodule, const char *refname,
-			struct object_id *oid)
+			struct object_id *oid, const char **referent_out)
 {
 	struct ref_store *refs;
 	int flags;
+	const char *referent;
 
 	refs = get_submodule_ref_store(submodule);
 
 	if (!refs)
 		return -1;
-
-	if (!refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags) ||
-	    is_null_oid(oid))
+	referent = refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags);
+	if (!referent || is_null_oid(oid))
 		return -1;
+	if (referent_out)
+		*referent_out = referent;
 	return 0;
 }
 
diff --git a/refs.h b/refs.h
index 47cb9edbaa8..c5ae76654c5 100644
--- a/refs.h
+++ b/refs.h
@@ -136,9 +136,12 @@ int peel_iterated_oid(const struct object_id *base, struct object_id *peeled);
  * submodule (which must be non-NULL). If the resolution is
  * successful, return 0 and set oid to the name of the object;
  * otherwise, return a non-zero value.
+ *
+ * FIXME: Return "referent" just like refs_resolve_ref_unsafe(). This will be
+ * safe to do once we merge resolve_gitlink_ref() into master.
  */
 int resolve_gitlink_ref(const char *submodule, const char *refname,
-			struct object_id *oid);
+			struct object_id *oid, const char **referent);
 
 /*
  * Return true iff abbrev_name is a possible abbreviation for
diff --git a/unpack-trees.c b/unpack-trees.c
index 8a454e03bff..43a1a056a19 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -2202,7 +2202,8 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 
 	if (S_ISGITLINK(ce->ce_mode)) {
 		struct object_id oid;
-		int sub_head = resolve_gitlink_ref(ce->name, "HEAD", &oid);
+		int sub_head =
+			resolve_gitlink_ref(ce->name, "HEAD", &oid, NULL);
 		/*
 		 * If we are not going to update the submodule, then
 		 * we don't care.
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH 5/6] submodule--helper: refactor up-to-date criterion
  2022-08-29 20:54 [PATCH 0/6] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
                   ` (3 preceding siblings ...)
  2022-08-29 20:54 ` [PATCH 4/6] submodule: return target of submodule symref Glen Choo via GitGitGadget
@ 2022-08-29 20:54 ` Glen Choo via GitGitGadget
  2022-08-29 20:54 ` [PATCH 6/6] clone, submodule update: check out branches Glen Choo via GitGitGadget
  2022-10-20 20:20 ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
  6 siblings, 0 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-08-29 20:54 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

Refactor builtin/submodule--helper.c:update_submodule() to check if the
submodule is up to date using a variable instead of checking the oids
directly. In a subsequent commit, we will expand the definition of "up
to date" to include checked out branches.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/submodule--helper.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index dd00a0db522..cbf6bda4850 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2455,6 +2455,8 @@ static void update_data_to_args(struct update_data *update_data, struct strvec *
 
 static int update_submodule(struct update_data *update_data)
 {
+	int submodule_up_to_date;
+
 	ensure_core_worktree(update_data->sm_path);
 
 	update_data->displaypath = get_submodule_displaypath(
@@ -2491,7 +2493,8 @@ static int update_submodule(struct update_data *update_data)
 		free(remote_ref);
 	}
 
-	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
+	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
+	if (!submodule_up_to_date || update_data->force)
 		if (run_update_procedure(update_data))
 			return 1;
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH 6/6] clone, submodule update: check out branches
  2022-08-29 20:54 [PATCH 0/6] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
                   ` (4 preceding siblings ...)
  2022-08-29 20:54 ` [PATCH 5/6] submodule--helper: refactor up-to-date criterion Glen Choo via GitGitGadget
@ 2022-08-29 20:54 ` Glen Choo via GitGitGadget
  2022-08-30  4:03   ` Philippe Blain
  2022-09-01 20:00   ` Jonathan Tan
  2022-10-20 20:20 ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
  6 siblings, 2 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-08-29 20:54 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

Teach "git submodule update" to update submodules by creating and
checking out the current superproject branch when
"submodule.propagateBranches=true". "git clone --recurse-submodules"
also learns this trick because it is implemented with "git submodule
update --recursive".

With "submodule.propagateBranches=true", submodules are cloned with
"--detach" so that they do not contain branches from their upstream.
This prevents conflicts between branch names from the superproject and
the branch names from the submodule's upstream. Arguably, "--detach"
should also be the default for "submodule.propagateBranches=false"
since it doesn't make sense to create a submodule branch when the
submodule is always expected to be in detached HEAD. But, to be
conservative, this commit does not change the behavior of
"submodule.propagateBranches=false".

"git submodule update" tries to create the branch as long as it is not
currently checked out, thus it will fail if the submodule has the
branch, but it is not checked out. This is fine because the main purpose
of "git submodule update" is to clone new submodules (which have no
branches, and will never have this problem). "git checkout" with
"submodule.propagateBranches" will cover the use case of recursively
checking out an existing branch.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/submodule--helper.c | 28 ++++++++++++++++++++++++++--
 t/t5617-clone-submodules.sh | 34 ++++++++++++++++++++++++++++++++++
 t/t7406-submodule-update.sh | 22 ++++++++++++++++++++++
 3 files changed, 82 insertions(+), 2 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index cbf6bda4850..7eb2c45900e 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1695,6 +1695,9 @@ static int clone_submodule(struct module_clone_data *clone_data)
 			strvec_push(&cp.args, clone_data->single_branch ?
 				    "--single-branch" :
 				    "--no-single-branch");
+		if (the_repository->settings.submodule_propagate_branches)
+			strvec_push(&cp.args, "--detach");
+
 
 		strvec_push(&cp.args, "--");
 		strvec_push(&cp.args, clone_data->url);
@@ -1733,6 +1736,9 @@ static int clone_submodule(struct module_clone_data *clone_data)
 	if (error_strategy)
 		git_config_set_in_file(p, "submodule.alternateErrorStrategy",
 				       error_strategy);
+	if (the_repository->settings.submodule_propagate_branches)
+		git_config_set_in_file(p, "submodule.propagateBranches",
+				       "true");
 
 	free(sm_alternate);
 	free(error_strategy);
@@ -1792,6 +1798,7 @@ static int module_clone(int argc, const char **argv, const char *prefix)
 	memset(&filter_options, 0, sizeof(filter_options));
 	argc = parse_options(argc, argv, prefix, module_clone_options,
 			     git_submodule_helper_usage, 0);
+	prepare_repo_settings(the_repository);
 
 	clone_data.dissociate = !!dissociate;
 	clone_data.quiet = !!quiet;
@@ -1872,6 +1879,7 @@ struct submodule_update_clone {
 struct update_data {
 	const char *prefix;
 	const char *displaypath;
+	const char *super_branch;
 	enum submodule_update_type update_default;
 	struct object_id suboid;
 	struct string_list references;
@@ -2206,6 +2214,8 @@ static int run_update_command(struct update_data *ud, int subforce)
 		strvec_pushl(&cp.args, "checkout", "-q", NULL);
 		if (subforce)
 			strvec_push(&cp.args, "-f");
+		if (ud->super_branch)
+			strvec_pushl(&cp.args, "-b", ud->super_branch, NULL);
 		break;
 	case SM_UPDATE_REBASE:
 		cp.git_cmd = 1;
@@ -2456,6 +2466,7 @@ static void update_data_to_args(struct update_data *update_data, struct strvec *
 static int update_submodule(struct update_data *update_data)
 {
 	int submodule_up_to_date;
+	const char *submodule_head = NULL;
 
 	ensure_core_worktree(update_data->sm_path);
 
@@ -2469,7 +2480,7 @@ static int update_submodule(struct update_data *update_data)
 	if (update_data->just_cloned)
 		oidcpy(&update_data->suboid, null_oid());
 	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
-				     &update_data->suboid, NULL))
+				     &update_data->suboid, &submodule_head))
 		die(_("Unable to find current revision in submodule path '%s'"),
 			update_data->displaypath);
 
@@ -2493,7 +2504,13 @@ static int update_submodule(struct update_data *update_data)
 		free(remote_ref);
 	}
 
-	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
+	if (update_data->super_branch &&
+	    submodule_head &&
+	    !skip_prefix(submodule_head, "refs/heads/", &submodule_head))
+		submodule_up_to_date = !strcmp(update_data->super_branch, submodule_head);
+	else
+		submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
+
 	if (!submodule_up_to_date || update_data->force)
 		if (run_update_procedure(update_data))
 			return 1;
@@ -2551,6 +2568,12 @@ static int update_submodules(struct update_data *update_data)
 		goto cleanup;
 	}
 
+	if (the_repository->settings.submodule_propagate_branches) {
+		struct branch *current_branch = branch_get(NULL);
+		if (current_branch)
+			update_data->super_branch = current_branch->name;
+	}
+
 	for (i = 0; i < suc.update_clone_nr; i++) {
 		struct update_clone_data ucd = suc.update_clone[i];
 
@@ -2634,6 +2657,7 @@ static int module_update(int argc, const char **argv, const char *prefix)
 	memset(&filter_options, 0, sizeof(filter_options));
 	argc = parse_options(argc, argv, prefix, module_update_options,
 			     git_submodule_helper_usage, 0);
+	prepare_repo_settings(the_repository);
 
 	if (opt.require_init)
 		opt.init = 1;
diff --git a/t/t5617-clone-submodules.sh b/t/t5617-clone-submodules.sh
index b5c66cb18cb..215fb02e9fb 100755
--- a/t/t5617-clone-submodules.sh
+++ b/t/t5617-clone-submodules.sh
@@ -12,10 +12,17 @@ pwd=$(pwd)
 test_expect_success 'setup' '
 	git checkout -b main &&
 	test_commit commit1 &&
+	mkdir subsub &&
+	(
+		cd subsub &&
+		git init &&
+		test_commit subsubcommit1
+	) &&
 	mkdir sub &&
 	(
 		cd sub &&
 		git init &&
+		git submodule add "file://$pwd/subsub" subsub &&
 		test_commit subcommit1 &&
 		git tag sub_when_added_to_super &&
 		git branch other
@@ -106,4 +113,31 @@ test_expect_success '--no-also-filter-submodules overrides clone.filterSubmodule
 	test_cmp_config -C super_clone3/sub false --default false remote.origin.promisor
 '
 
+test_expect_success 'submodule.propagateBranches checks out branches at correct commits' '
+	git -C sub checkout -b not-main &&
+	git -C subsub checkout -b not-main &&
+	git clone --recurse-submodules \
+		-c submodule.propagateBranches=true \
+		"file://$pwd/." super_clone4 &&
+
+	# Assert that each repo is pointing to "main"
+	for REPO in "super_clone4" "super_clone4/sub" "super_clone4/sub/subsub"
+	do
+	    HEAD_BRANCH=$(git -C $REPO symbolic-ref HEAD) &&
+	    test $HEAD_BRANCH = "refs/heads/main" || return 1
+	done &&
+
+	# Assert that the submodule branches are pointing to the right revs
+	EXPECT_SUB_OID="$(git -C super_clone4 rev-parse :sub)" &&
+	ACTUAL_SUB_OID="$(git -C super_clone4/sub rev-parse refs/heads/main)" &&
+	test $EXPECT_SUB_OID = $ACTUAL_SUB_OID &&
+	EXPECT_SUBSUB_OID="$(git -C super_clone4/sub rev-parse :subsub)" &&
+	ACTUAL_SUBSUB_OID="$(git -C super_clone4/sub/subsub rev-parse refs/heads/main)" &&
+	test $EXPECT_SUBSUB_OID = $ACTUAL_SUBSUB_OID &&
+
+	# Assert that the submodules do not have branches from their upstream
+	test_must_fail git -C super_clone4/sub rev-parse not-main &&
+	test_must_fail git -C super_clone4/sub/subsub rev-parse not-main
+'
+
 test_done
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 6cc07460dd2..00a6fec8912 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -1178,4 +1178,26 @@ test_expect_success 'submodule update --recursive skip submodules with strategy=
 	test_cmp expect.err actual.err
 '
 
+test_expect_success 'submodule update with submodule.propagateBranches checks out branches' '
+	test_when_finished "rm -fr top-cloned" &&
+	cp -r top-clean top-cloned &&
+
+	# Create a new upstream submodule
+	git init middle2 &&
+	test_commit -C middle2 "middle2" &&
+	git -C top submodule add ../middle2 middle2 &&
+	git -C top commit -m "add middle2" &&
+
+	git -C top-cloned checkout -b "new-branch" &&
+	git -C top-cloned pull origin main &&
+	test_config -C top-cloned submodule.propagateBranches true &&
+	git -C top-cloned submodule update --recursive &&
+
+	for REPO in "top-cloned/middle2" "top-cloned/middle" "top-cloned/middle/bottom"
+	do
+	    HEAD_BRANCH=$(git -C $REPO symbolic-ref HEAD) &&
+	    test $HEAD_BRANCH = "refs/heads/new-branch" || return 1
+	done
+'
+
 test_done
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 56+ messages in thread

* Re: [PATCH 1/6] clone: teach --detach option
  2022-08-29 20:54 ` [PATCH 1/6] clone: teach --detach option Glen Choo via GitGitGadget
@ 2022-08-30  4:02   ` Philippe Blain
  0 siblings, 0 replies; 56+ messages in thread
From: Philippe Blain @ 2022-08-30  4:02 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget, git; +Cc: Glen Choo

Hi Glen,

Le 2022-08-29 à 16:54, Glen Choo via GitGitGadget a écrit :
> From: Glen Choo <chooglen@google.com>
> 
> Teach "git clone" the "--detach" option, which leaves the cloned repo in
> detached HEAD (like "git checkout --detach"). If the clone is not bare,
> the remote's HEAD branch is also not created (bare clones always copy
> all remote branches directly to local branches, so the branch is still
> created in the bare case).

At first reading I thought you meant the 'origin/HEAD' symref, which is 
not the case here. I think something like this would maybe be clearer:

If the clone is not bare, skip the creation of a local branch corresponding to 
the branch pointed to by the remote's HEAD symref (bare clones...
to local branches, so that branch ...

(OK this is very verbose but in my opinion it's clearer.)

> This is especially useful in the "submodule.propagateBranches" workflow,
> where the submodule branch names match the superproject's branch names,
> so it makes no sense to name the branches after the submodule's remote's
> branches.

We are just skipping the creation of a single branch here, so it's unclear 
to me which other branches are being talked about in this last paragraph.
All remote-tracking branches are unaffected by this flag, no?

> Signed-off-by: Glen Choo <chooglen@google.com>
> ---
>  Documentation/git-clone.txt |  7 ++++++-
>  builtin/clone.c             | 12 +++++++++---
>  t/t5601-clone.sh            | 22 ++++++++++++++++++++++
>  3 files changed, 37 insertions(+), 4 deletions(-)
> 
> diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
> index 632bd1348ea..a3af90824b6 100644
> --- a/Documentation/git-clone.txt
> +++ b/Documentation/git-clone.txt
> @@ -16,7 +16,7 @@ SYNOPSIS
>  	  [--depth <depth>] [--[no-]single-branch] [--no-tags]
>  	  [--recurse-submodules[=<pathspec>]] [--[no-]shallow-submodules]
>  	  [--[no-]remote-submodules] [--jobs <n>] [--sparse] [--[no-]reject-shallow]
> -	  [--filter=<filter> [--also-filter-submodules]] [--] <repository>
> +	  [--filter=<filter> [--also-filter-submodules] [--detach]] [--] <repository>
>  	  [<directory>]
>  
>  DESCRIPTION
> @@ -210,6 +210,11 @@ objects from the source repository into a pack in the cloned repository.
>  	`--branch` can also take tags and detaches the HEAD at that commit
>  	in the resulting repository.
>  
> +--detach::
> +	If the cloned repository's HEAD points to a branch, point the newly
> +	created HEAD to the branch's commit instead of the branch itself. In a
> +	non-bare repository, the branch will not be created.

Again, I think the wording could be improved, maybe something along those lines:

If the cloned repository's HEAD points to a branch, detach the newly created HEAD
at the commit at the tip of that branch. Additionnally, in a non-bare repository,
skip creating a corresponding local branch.

> +
>  -u <upload-pack>::
>  --upload-pack <upload-pack>::
>  	When given, and the repository to clone from is accessed
> diff --git a/builtin/clone.c b/builtin/clone.c
> index c4ff4643ecd..1bc1807360e 100644
> --- a/builtin/clone.c
> +++ b/builtin/clone.c
> @@ -77,6 +77,7 @@ static int option_filter_submodules = -1;    /* unspecified */
>  static int config_filter_submodules = -1;    /* unspecified */
>  static struct string_list server_options = STRING_LIST_INIT_NODUP;
>  static int option_remote_submodules;
> +static int option_detach;
>  
>  static int recurse_submodules_cb(const struct option *opt,
>  				 const char *arg, int unset)
> @@ -160,6 +161,8 @@ static struct option builtin_clone_options[] = {
>  		    N_("any cloned submodules will use their remote-tracking branch")),
>  	OPT_BOOL(0, "sparse", &option_sparse_checkout,
>  		    N_("initialize sparse-checkout file to include only files at root")),
> +	OPT_BOOL(0, "detach", &option_detach,
> +		 N_("detach HEAD and don't create branch")),

maybe "don't create any local branch" ?

>  	OPT_END()
>  };
>  
> @@ -607,10 +610,12 @@ static void update_remote_refs(const struct ref *refs,
>  }
>  
>  static void update_head(const struct ref *our, const struct ref *remote,
> -			const char *unborn, const char *msg)
> +			const char *unborn, int should_detach,
> +			const char *msg)
>  {
>  	const char *head;
> -	if (our && skip_prefix(our->name, "refs/heads/", &head)) {
> +	if (our && !should_detach &&
> +	    skip_prefix(our->name, "refs/heads/", &head)) {
>  		/* Local default branch link */
>  		if (create_symref("HEAD", our->name, NULL) < 0)
>  			die(_("unable to update HEAD"));

OK, so the addition of that condition means that if --detach was given, we now
go into the 'else if (our)' branch, as long as 'our' is non-null, which means
that the remote's HEAD points to a branch or we gave --branch. This makes sense.
If the remote's HEAD does not point to a branch and we did not give --branch,
then we go into 'else if (remote)', as before. 

> @@ -1339,7 +1344,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
>  			   branch_top.buf, reflog_msg.buf, transport,
>  			   !is_local);
>  
> -	update_head(our_head_points_at, remote_head, unborn_head, reflog_msg.buf);
> +	update_head(our_head_points_at, remote_head, unborn_head,
> +		    option_detach, reflog_msg.buf);
>  
>  	/*
>  	 * We want to show progress for recursive submodule clones iff
> diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh
> index cf3be0584f4..1e7e5143a76 100755
> --- a/t/t5601-clone.sh
> +++ b/t/t5601-clone.sh
> @@ -333,6 +333,28 @@ test_expect_success 'clone checking out a tag' '
>  	test_cmp fetch.expected fetch.actual
>  '
>  
> +test_expect_success '--detach detaches and does not create branch' '
> +	test_when_finished "rm -fr dst" &&
> +	git clone --detach src dst &&
> +	(
> +		cd dst &&
> +		test_must_fail git rev-parse main &&
> +		test_must_fail git symbolic-ref HEAD &&
> +		test_cmp_rev HEAD refs/remotes/origin/HEAD
> +	)
> +'
> +
> +test_expect_success '--detach with --bare detaches but creates branch' '
> +	test_when_finished "rm -fr dst" &&
> +	git clone --bare --detach src dst &&
> +	(
> +		cd dst &&
> +		git rev-parse main &&
> +		test_must_fail git symbolic-ref HEAD &&
> +		test_cmp_rev HEAD refs/heads/main
> +	)
> +'
> +

Tests look good.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH 2/6] repo-settings: add submodule_propagate_branches
  2022-08-29 20:54 ` [PATCH 2/6] repo-settings: add submodule_propagate_branches Glen Choo via GitGitGadget
@ 2022-08-30  4:02   ` Philippe Blain
  0 siblings, 0 replies; 56+ messages in thread
From: Philippe Blain @ 2022-08-30  4:02 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget, git; +Cc: Glen Choo



Le 2022-08-29 à 16:54, Glen Choo via GitGitGadget a écrit :
> From: Glen Choo <chooglen@google.com>
> 
> When processes recurse into submodules, the child processes have to
> use the same value of "submodule.propagateBranches" as the parent
> process regardless of whether the process is spawned in the superproject
> or submodule, otherwise the behavior may be inconsistent if the
> repositories don't agree on the config.
> 
> We haven't needed a way to propagate the config because because the only

"because because"

> command that reads "submodule.propagateBranches" is "git branch", which
> only has one mode of operation with "--recurse-submodules". However, a
> future commit will teach "submodule.propagateBranches" to "git submodule
> update", making this necessary.
> 
> Propagate "submodule.propagateBranches" to child processes by adding a
> corresponding GIT_INTERNAL_* environment variable and repository
> setting, and setting the environment variable inside
> prepare_submodule_repo_env(). Then, refactor builtin/branch.c to read
> the repository setting.
> 
> Using an internal environment variable is a potentially leaky
> abstraction because environment variables can come from sources besides
> the parent process. A more robust solution would be to teach Git that
> the repository is a submodule and to only read
> "submodule.propagateBranches" from the superproject config. There is WIP
> for this on the ML [1].
> 
> Another alternative would be to pass "-c submodule.propagateBranches" to
> all child processes. This is error-prone because many different
> processes are invoked directly or indirectly by "git submodule update"
> (e.g. "git submodule--helper clone", "git clone", "git checkout"). With
> an environment variable, we can avoid this work because
> prepare_submodule_repo_env() is already called for submodule child
> processes.

I think this is a good justification. I agree adding '-c' everywhere would be
error-prone.


^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH 3/6] t5617: drop references to remote-tracking branches
  2022-08-29 20:54 ` [PATCH 3/6] t5617: drop references to remote-tracking branches Glen Choo via GitGitGadget
@ 2022-08-30  4:03   ` Philippe Blain
  0 siblings, 0 replies; 56+ messages in thread
From: Philippe Blain @ 2022-08-30  4:03 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget, git; +Cc: Glen Choo

Hi Glen,

Le 2022-08-29 à 16:54, Glen Choo via GitGitGadget a écrit :
> From: Glen Choo <chooglen@google.com>
> 
> It has included submodule cloning tests without remote-tracking branches
> tests since f05da2b48b (clone, submodule: pass partial clone filters to
> submodules, 2022-02-04) at least. Rename it accordingly so that we can
> put future submodule cloning tests there.

I think it was named "*-remote" because it was introduced when 
'git clone --remote-submodules' was added in 4c6910163a 
(clone: add `--remote-submodules` flag, 2019-05-19).

In any case, nice clean-up!

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH 6/6] clone, submodule update: check out branches
  2022-08-29 20:54 ` [PATCH 6/6] clone, submodule update: check out branches Glen Choo via GitGitGadget
@ 2022-08-30  4:03   ` Philippe Blain
  2022-08-30 22:54     ` Glen Choo
  2022-09-01 20:00   ` Jonathan Tan
  1 sibling, 1 reply; 56+ messages in thread
From: Philippe Blain @ 2022-08-30  4:03 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget, git; +Cc: Glen Choo

Hi Glen,

Le 2022-08-29 à 16:54, Glen Choo via GitGitGadget a écrit :
> From: Glen Choo <chooglen@google.com>
> 
> Teach "git submodule update" to update submodules by creating and
> checking out the current superproject branch when
> "submodule.propagateBranches=true". "git clone --recurse-submodules"
> also learns this trick because it is implemented with "git submodule
> update --recursive".

OK. Maybe a more descriptive title would then be:

clone, submodule update: create and check out submodule branches

?

Another thing, 'git pull --recurse-submodules' is also implemented using
'git submodule update --recursive'. But I don't think we want 'git pull'
to start creating new branches in submodules, even with submodule.propagateBranches=true
(though I haven't thought about it very hard). So maybe adding a word about
that would be nice.

> 
> With "submodule.propagateBranches=true", submodules are cloned with
> "--detach" so that they do not contain branches from their upstream.

We usually use the present tense to talk about the current state of the code base,
and then the imperative to order to codebase to improve itself;
here you already used the imperative "teach" in the previous paragraph,
so I'm assuming you are now talking about the new state of the code.
Maybe just adding "now" i.e. "submodules are now cloned" would help
readers ?

> This prevents conflicts between branch names from the superproject and
> the branch names from the submodule's upstream. Arguably, "--detach"
> should also be the default for "submodule.propagateBranches=false"
> since it doesn't make sense to create a submodule branch when the
> submodule is always expected to be in detached HEAD. But, to be
> conservative, this commit does not change the behavior of
> "submodule.propagateBranches=false".

I agree that it would be "cleaner" to make the change also for
"submodule.propagateBranches=false" eventually, but... let's not
change things just to change things :)

> "git submodule update" tries to create the branch as long as it is not
> currently checked out, thus it will fail if the submodule has the
> branch, but it is not checked out. This is fine because the main purpose
> of "git submodule update" is to clone new submodules (which have no
> branches, and will never have this problem). "git checkout" with
> "submodule.propagateBranches" will cover the use case of recursively
> checking out an existing branch.

I guess you mean "in a future series" for the last sentence ? FWIW I still have
your RFC from last Febryary about that [1] in my "unread Git mailing list" folder,
I always seem to lack the time to sit down and read it through, sorry!
Incidentally, I notice you did not link to it in the cover letter, 
any reasoon why?

[1] https://lore.kernel.org/git/20220209065236.36494-1-chooglen@google.com/

Stepping back a bit, you write "thus it will fail if the submodule has the
branch, but it is not checked out." If I read your patch correctly, this is
implicit in that 'git checkout -b super-branch' that is ran by 'run_update_command'
will error out if the branch already exists, right ? 

Is there anything more we should do in that case ? 
Should we remind the user, something like
"you have submodule.propagateBranches set, but the branch 'super-branch' already
exists in submodule 'that-sub'" ? 

I'm trying to think of a scenario in which this could happen...

Say a user:
1. clones a superproject with --recurse-submodules, but without 'submodule.propagateBranches'
2. runs 'git checkout -b topic' in the superproject
3. runs 'git branch topic' in the submodule
4. runs 'git submodule update' with 'submodule.propagateBranches' in the superproject

This fails:

fatal: a branch named 'topic' already exists
fatal: Unable to checkout 'deadbeef' in submodule path 'sub'

Do we need a more specific message ? I'm not sure.

> 
> Signed-off-by: Glen Choo <chooglen@google.com>
> ---
>  builtin/submodule--helper.c | 28 ++++++++++++++++++++++++++--
>  t/t5617-clone-submodules.sh | 34 ++++++++++++++++++++++++++++++++++
>  t/t7406-submodule-update.sh | 22 ++++++++++++++++++++++
>  3 files changed, 82 insertions(+), 2 deletions(-)
> 
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index cbf6bda4850..7eb2c45900e 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -1695,6 +1695,9 @@ static int clone_submodule(struct module_clone_data *clone_data)
>  			strvec_push(&cp.args, clone_data->single_branch ?
>  				    "--single-branch" :
>  				    "--no-single-branch");
> +		if (the_repository->settings.submodule_propagate_branches)
> +			strvec_push(&cp.args, "--detach");
> +
>  
>  		strvec_push(&cp.args, "--");
>  		strvec_push(&cp.args, clone_data->url);
> @@ -1733,6 +1736,9 @@ static int clone_submodule(struct module_clone_data *clone_data)
>  	if (error_strategy)
>  		git_config_set_in_file(p, "submodule.alternateErrorStrategy",
>  				       error_strategy);
> +	if (the_repository->settings.submodule_propagate_branches)
> +		git_config_set_in_file(p, "submodule.propagateBranches",
> +				       "true");

Why do we need to set that in the config of the submodule ? I'm guessing this 
is so that the new code also works for nested submodules, right ? 

I'm thinking about a user that would alternate between 'submodule.propagateBranches=true' and 'false'.
Maybe they sometimes have to work on the superproject and the submodule(s), sometimes 
only in the superproject. If they want to deactivate submodule.propagateBranches, would they have to
remember to also deactivate it in all submodules, in case of nested submodules ?...  if so,
this is a little unfortunate. But I _think_ they wouldn't have to, because as long as 
it's false in the superproject config, then we won't get into the new code at all when running
in the top level superproject...

>  	free(sm_alternate);
>  	free(error_strategy);
> @@ -1792,6 +1798,7 @@ static int module_clone(int argc, const char **argv, const char *prefix)
>  	memset(&filter_options, 0, sizeof(filter_options));
>  	argc = parse_options(argc, argv, prefix, module_clone_options,
>  			     git_submodule_helper_usage, 0);
> +	prepare_repo_settings(the_repository);
>  
>  	clone_data.dissociate = !!dissociate;
>  	clone_data.quiet = !!quiet;
> @@ -1872,6 +1879,7 @@ struct submodule_update_clone {
>  struct update_data {
>  	const char *prefix;
>  	const char *displaypath;
> +	const char *super_branch;
>  	enum submodule_update_type update_default;
>  	struct object_id suboid;
>  	struct string_list references;
> @@ -2206,6 +2214,8 @@ static int run_update_command(struct update_data *ud, int subforce)
>  		strvec_pushl(&cp.args, "checkout", "-q", NULL);
>  		if (subforce)
>  			strvec_push(&cp.args, "-f");
> +		if (ud->super_branch)
> +			strvec_pushl(&cp.args, "-b", ud->super_branch, NULL);
>  		break;
>  	case SM_UPDATE_REBASE:
>  		cp.git_cmd = 1;
> @@ -2456,6 +2466,7 @@ static void update_data_to_args(struct update_data *update_data, struct strvec *
>  static int update_submodule(struct update_data *update_data)
>  {
>  	int submodule_up_to_date;
> +	const char *submodule_head = NULL;
>  
>  	ensure_core_worktree(update_data->sm_path);
>  
> @@ -2469,7 +2480,7 @@ static int update_submodule(struct update_data *update_data)
>  	if (update_data->just_cloned)
>  		oidcpy(&update_data->suboid, null_oid());
>  	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
> -				     &update_data->suboid, NULL))
> +				     &update_data->suboid, &submodule_head))
>  		die(_("Unable to find current revision in submodule path '%s'"),
>  			update_data->displaypath);
>  
> @@ -2493,7 +2504,13 @@ static int update_submodule(struct update_data *update_data)
>  		free(remote_ref);
>  	}
>  
> -	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
> +	if (update_data->super_branch &&
> +	    submodule_head &&
> +	    !skip_prefix(submodule_head, "refs/heads/", &submodule_head))
> +		submodule_up_to_date = !strcmp(update_data->super_branch, submodule_head);

I'm not sure I understand this logic. We want to change the 'submodule_up_to_date' boolean,
so that we compare branch names instead of oid's, and we do that only if:

1. we are running with 'propagateBranches=true' (so update_data->super_branch will be set to the superproject's branch)
2. a ref is checked out in the submodule (so submodule_head will hold its name)
3. it's not a branch (so skip_prefix will return 0, and !skip_prefix will be 1). 
   In that case it must be simply "HEAD", i.e. the submodule's HEAD is detached.

Why do we need (2. + 3.) ? 

If branch 'foo' is currently checked out in the superproject, and
branch 'bar' is currently checked out in the submodule, and someone
runs 'git -c propagateBranches=true submodule update', wouldn't they expect
that 'bar' be checked out in the submodule ? Maybe not, but the commit message
and the tests should be more explicit about the expected behaviour in this case, I think.

And thinking about it more, won't this:

    submodule_up_to_date = !strcmp(update_data->super_branch, submodule_head);

always be false, since we already know that submodule_head is "HEAD" ?... 
Unless I'm confused...
 
> +	else
> +		submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
> +
>  	if (!submodule_up_to_date || update_data->force)
>  		if (run_update_procedure(update_data))
>  			return 1;
> @@ -2551,6 +2568,12 @@ static int update_submodules(struct update_data *update_data)
>  		goto cleanup;
>  	}
>  
> +	if (the_repository->settings.submodule_propagate_branches) {
> +		struct branch *current_branch = branch_get(NULL);
> +		if (current_branch)
> +			update_data->super_branch = current_branch->name;

OK, so this condition means that super_branch won't get set if we are not
currently on a branch, i.e. we are in detached HEAD. This makes sense as there
would be no branch to propagate. Do we need a test for this ? maybe a case where
we clone with '--recurse-submodules --branch some-tag' ?

> +	}
> +
>  	for (i = 0; i < suc.update_clone_nr; i++) {
>  		struct update_clone_data ucd = suc.update_clone[i];
>  
> @@ -2634,6 +2657,7 @@ static int module_update(int argc, const char **argv, const char *prefix)
>  	memset(&filter_options, 0, sizeof(filter_options));
>  	argc = parse_options(argc, argv, prefix, module_update_options,
>  			     git_submodule_helper_usage, 0);
> +	prepare_repo_settings(the_repository);
>  
>  	if (opt.require_init)
>  		opt.init = 1;
> diff --git a/t/t5617-clone-submodules.sh b/t/t5617-clone-submodules.sh
> index b5c66cb18cb..215fb02e9fb 100755
> --- a/t/t5617-clone-submodules.sh
> +++ b/t/t5617-clone-submodules.sh
> @@ -12,10 +12,17 @@ pwd=$(pwd)
>  test_expect_success 'setup' '
>  	git checkout -b main &&
>  	test_commit commit1 &&
> +	mkdir subsub &&
> +	(
> +		cd subsub &&
> +		git init &&
> +		test_commit subsubcommit1
> +	) &&
>  	mkdir sub &&
>  	(
>  		cd sub &&
>  		git init &&
> +		git submodule add "file://$pwd/subsub" subsub &&
>  		test_commit subcommit1 &&
>  		git tag sub_when_added_to_super &&
>  		git branch other
> @@ -106,4 +113,31 @@ test_expect_success '--no-also-filter-submodules overrides clone.filterSubmodule
>  	test_cmp_config -C super_clone3/sub false --default false remote.origin.promisor
>  '
>  
> +test_expect_success 'submodule.propagateBranches checks out branches at correct commits' '
> +	git -C sub checkout -b not-main &&
> +	git -C subsub checkout -b not-main &&
> +	git clone --recurse-submodules \
> +		-c submodule.propagateBranches=true \
> +		"file://$pwd/." super_clone4 &&
> +
> +	# Assert that each repo is pointing to "main"
> +	for REPO in "super_clone4" "super_clone4/sub" "super_clone4/sub/subsub"
> +	do
> +	    HEAD_BRANCH=$(git -C $REPO symbolic-ref HEAD) &&
> +	    test $HEAD_BRANCH = "refs/heads/main" || return 1
> +	done &&
> +
> +	# Assert that the submodule branches are pointing to the right revs
> +	EXPECT_SUB_OID="$(git -C super_clone4 rev-parse :sub)" &&
> +	ACTUAL_SUB_OID="$(git -C super_clone4/sub rev-parse refs/heads/main)" &&
> +	test $EXPECT_SUB_OID = $ACTUAL_SUB_OID &&
> +	EXPECT_SUBSUB_OID="$(git -C super_clone4/sub rev-parse :subsub)" &&
> +	ACTUAL_SUBSUB_OID="$(git -C super_clone4/sub/subsub rev-parse refs/heads/main)" &&
> +	test $EXPECT_SUBSUB_OID = $ACTUAL_SUBSUB_OID &&
> +
> +	# Assert that the submodules do not have branches from their upstream
> +	test_must_fail git -C super_clone4/sub rev-parse not-main &&
> +	test_must_fail git -C super_clone4/sub/subsub rev-parse not-main
> +'
> +
>  test_done
> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
> index 6cc07460dd2..00a6fec8912 100755
> --- a/t/t7406-submodule-update.sh
> +++ b/t/t7406-submodule-update.sh
> @@ -1178,4 +1178,26 @@ test_expect_success 'submodule update --recursive skip submodules with strategy=
>  	test_cmp expect.err actual.err
>  '
>  
> +test_expect_success 'submodule update with submodule.propagateBranches checks out branches' '
> +	test_when_finished "rm -fr top-cloned" &&
> +	cp -r top-clean top-cloned &&
> +
> +	# Create a new upstream submodule
> +	git init middle2 &&
> +	test_commit -C middle2 "middle2" &&
> +	git -C top submodule add ../middle2 middle2 &&
> +	git -C top commit -m "add middle2" &&
> +
> +	git -C top-cloned checkout -b "new-branch" &&
> +	git -C top-cloned pull origin main &&
> +	test_config -C top-cloned submodule.propagateBranches true &&
> +	git -C top-cloned submodule update --recursive &&
> +
> +	for REPO in "top-cloned/middle2" "top-cloned/middle" "top-cloned/middle/bottom"
> +	do
> +	    HEAD_BRANCH=$(git -C $REPO symbolic-ref HEAD) &&
> +	    test $HEAD_BRANCH = "refs/heads/new-branch" || return 1
> +	done
> +'
> +
>  test_done
> 

These tests look good, but maybe more tests would be needed in 
the light of my comments above... 

Thanks again for working on improving submodules!

Cheers,

Philippe.


^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH 6/6] clone, submodule update: check out branches
  2022-08-30  4:03   ` Philippe Blain
@ 2022-08-30 22:54     ` Glen Choo
  0 siblings, 0 replies; 56+ messages in thread
From: Glen Choo @ 2022-08-30 22:54 UTC (permalink / raw)
  To: Philippe Blain, Glen Choo via GitGitGadget, git

Philippe Blain <levraiphilippeblain@gmail.com> writes:

> Hi Glen,
>
> Le 2022-08-29 à 16:54, Glen Choo via GitGitGadget a écrit :
>> From: Glen Choo <chooglen@google.com>
>> 
>> Teach "git submodule update" to update submodules by creating and
>> checking out the current superproject branch when
>> "submodule.propagateBranches=true". "git clone --recurse-submodules"
>> also learns this trick because it is implemented with "git submodule
>> update --recursive".
>
> OK. Maybe a more descriptive title would then be:
>
> clone, submodule update: create and check out submodule branches
>
> ?

Ah, thanks. Your other wording suggestions upthread are also very
helpful.

>
> Another thing, 'git pull --recurse-submodules' is also implemented using
> 'git submodule update --recursive'. But I don't think we want 'git pull'
> to start creating new branches in submodules, even with submodule.propagateBranches=true
> (though I haven't thought about it very hard). So maybe adding a word about
> that would be nice.

Good point. I thought that `git pull --recurse-submodules` used the
`--merge` strategy (in which case, it wouldn't matter), but looks like
it uses the `--checkout` strategy.

I'm quite certain we want to replace this `git pull
--recurse-submodules` implementation, aka non recursive `git merge` +
`git submodule update`, with a recursive `git merge` (and possibly
updating the worktrees with `git checkout --recurse-submodules`). Since
this flag is still experimental and incomplete, I think we have the
freedom to say that we won't care about this for now, but either way
I'll mention this somewhere.

>> "git submodule update" tries to create the branch as long as it is not
>> currently checked out, thus it will fail if the submodule has the
>> branch, but it is not checked out. This is fine because the main purpose
>> of "git submodule update" is to clone new submodules (which have no
>> branches, and will never have this problem). "git checkout" with
>> "submodule.propagateBranches" will cover the use case of recursively
>> checking out an existing branch.
>
> I guess you mean "in a future series" for the last sentence ? FWIW I still have
> your RFC from last Febryary about that [1] in my "unread Git mailing list" folder,
> I always seem to lack the time to sit down and read it through, sorry!
> Incidentally, I notice you did not link to it in the cover letter, 
> any reasoon why?
>
> [1] https://lore.kernel.org/git/20220209065236.36494-1-chooglen@google.com/

Ah, yes, I meant "in a future series". I didn't think that the RFC would
be very useful to reviewers since the non-RFC version is likely change a
lot (I've done a lot of tinkering between then and now), and it didn't
gain much traction in the first place anyway.

>
> Stepping back a bit, you write "thus it will fail if the submodule has the
> branch, but it is not checked out." If I read your patch correctly, this is
> implicit in that 'git checkout -b super-branch' that is ran by 'run_update_command'
> will error out if the branch already exists, right ? 
>
> Is there anything more we should do in that case ? 
> Should we remind the user, something like
> "you have submodule.propagateBranches set, but the branch 'super-branch' already
> exists in submodule 'that-sub'" ? 
>
> I'm trying to think of a scenario in which this could happen...
>
> Say a user:
> 1. clones a superproject with --recurse-submodules, but without 'submodule.propagateBranches'
> 2. runs 'git checkout -b topic' in the superproject
> 3. runs 'git branch topic' in the submodule
> 4. runs 'git submodule update' with 'submodule.propagateBranches' in the superproject
>
> This fails:
>
> fatal: a branch named 'topic' already exists
> fatal: Unable to checkout 'deadbeef' in submodule path 'sub'
>
> Do we need a more specific message ? I'm not sure.

Hm, you're right, this does seem quite opaque to end users; this means
nothing if they don't know that `git submodule update` uses `git checkout
-b` under the hood, which they obviously shouldn't need to know.

The main simplifying assumption behind `submodule.propagateBranches` (or
at least, this early version of it) is that users won't interact with
branches on the submodules directly outside of very specific scenarios,
e.g. setting submodule-specific tracking info. So maybe the more
comprehensive solution would be to block users from creating branches if
the submodule's superproject uses `submodule.propagateBranches`.

>> 
>> Signed-off-by: Glen Choo <chooglen@google.com>
>> ---
>>  builtin/submodule--helper.c | 28 ++++++++++++++++++++++++++--
>>  t/t5617-clone-submodules.sh | 34 ++++++++++++++++++++++++++++++++++
>>  t/t7406-submodule-update.sh | 22 ++++++++++++++++++++++
>>  3 files changed, 82 insertions(+), 2 deletions(-)
>> 
>> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
>> index cbf6bda4850..7eb2c45900e 100644
>> --- a/builtin/submodule--helper.c
>> +++ b/builtin/submodule--helper.c
>> @@ -1695,6 +1695,9 @@ static int clone_submodule(struct module_clone_data *clone_data)
>>  			strvec_push(&cp.args, clone_data->single_branch ?
>>  				    "--single-branch" :
>>  				    "--no-single-branch");
>> +		if (the_repository->settings.submodule_propagate_branches)
>> +			strvec_push(&cp.args, "--detach");
>> +
>>  
>>  		strvec_push(&cp.args, "--");
>>  		strvec_push(&cp.args, clone_data->url);
>> @@ -1733,6 +1736,9 @@ static int clone_submodule(struct module_clone_data *clone_data)
>>  	if (error_strategy)
>>  		git_config_set_in_file(p, "submodule.alternateErrorStrategy",
>>  				       error_strategy);
>> +	if (the_repository->settings.submodule_propagate_branches)
>> +		git_config_set_in_file(p, "submodule.propagateBranches",
>> +				       "true");
>
> Why do we need to set that in the config of the submodule ? I'm guessing this 
> is so that the new code also works for nested submodules, right ? 

As long as the value is set in the superproject, the new code still
works. This is meant as a way of setting the user's preferred value in
the submodule. Although.. `git clone` doesn't automatically set this
value in the superproject - it would have to be read off
system/global/cli config, so maybe it's more coherent to acknowledge
that the user's preferred value probably isn't in the repo anyway, and
maybe I should just drop this.

>
> I'm thinking about a user that would alternate between 'submodule.propagateBranches=true' and 'false'.
> Maybe they sometimes have to work on the superproject and the submodule(s), sometimes 
> only in the superproject. If they want to deactivate submodule.propagateBranches, would they have to
> remember to also deactivate it in all submodules, in case of nested submodules ?...  if so,
> this is a little unfortunate. But I _think_ they wouldn't have to, because as long as 
> it's false in the superproject config, then we won't get into the new code at all when running
> in the top level superproject...

Hm, would a user want to alternate in the first place? Maybe? e.g. with
`git checkout topic`, "true" would check out the submodule worktree at
the branch (including any WIP you have) but "false" would give you the
worktree specified by the superproject. Both are useful.

The way it's written now, "submodule.propagateBranches" is only passed
to submodule processes if it is "true", so it can be overrided by
submodule config if superproject says "false" but submodule says "true".
I should fix that..

>>  	free(sm_alternate);
>>  	free(error_strategy);
>> @@ -1792,6 +1798,7 @@ static int module_clone(int argc, const char **argv, const char *prefix)
>>  	memset(&filter_options, 0, sizeof(filter_options));
>>  	argc = parse_options(argc, argv, prefix, module_clone_options,
>>  			     git_submodule_helper_usage, 0);
>> +	prepare_repo_settings(the_repository);
>>  
>>  	clone_data.dissociate = !!dissociate;
>>  	clone_data.quiet = !!quiet;
>> @@ -1872,6 +1879,7 @@ struct submodule_update_clone {
>>  struct update_data {
>>  	const char *prefix;
>>  	const char *displaypath;
>> +	const char *super_branch;
>>  	enum submodule_update_type update_default;
>>  	struct object_id suboid;
>>  	struct string_list references;
>> @@ -2206,6 +2214,8 @@ static int run_update_command(struct update_data *ud, int subforce)
>>  		strvec_pushl(&cp.args, "checkout", "-q", NULL);
>>  		if (subforce)
>>  			strvec_push(&cp.args, "-f");
>> +		if (ud->super_branch)
>> +			strvec_pushl(&cp.args, "-b", ud->super_branch, NULL);
>>  		break;
>>  	case SM_UPDATE_REBASE:
>>  		cp.git_cmd = 1;
>> @@ -2456,6 +2466,7 @@ static void update_data_to_args(struct update_data *update_data, struct strvec *
>>  static int update_submodule(struct update_data *update_data)
>>  {
>>  	int submodule_up_to_date;
>> +	const char *submodule_head = NULL;
>>  
>>  	ensure_core_worktree(update_data->sm_path);
>>  
>> @@ -2469,7 +2480,7 @@ static int update_submodule(struct update_data *update_data)
>>  	if (update_data->just_cloned)
>>  		oidcpy(&update_data->suboid, null_oid());
>>  	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
>> -				     &update_data->suboid, NULL))
>> +				     &update_data->suboid, &submodule_head))
>>  		die(_("Unable to find current revision in submodule path '%s'"),
>>  			update_data->displaypath);
>>  
>> @@ -2493,7 +2504,13 @@ static int update_submodule(struct update_data *update_data)
>>  		free(remote_ref);
>>  	}
>>  
>> -	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
>> +	if (update_data->super_branch &&
>> +	    submodule_head &&
>> +	    !skip_prefix(submodule_head, "refs/heads/", &submodule_head))
>> +		submodule_up_to_date = !strcmp(update_data->super_branch, submodule_head);
>
> I'm not sure I understand this logic. We want to change the 'submodule_up_to_date' boolean,
> so that we compare branch names instead of oid's, and we do that only if:
>
> 1. we are running with 'propagateBranches=true' (so update_data->super_branch will be set to the superproject's branch)
> 2. a ref is checked out in the submodule (so submodule_head will hold its name)
> 3. it's not a branch (so skip_prefix will return 0, and !skip_prefix will be 1). 
>    In that case it must be simply "HEAD", i.e. the submodule's HEAD is detached.
>
> Why do we need (2. + 3.) ? 

Oops I got skip_prefix() backwards, 3. should read "if a branch is
checked out". I'll add a test case for this (I could've sworn I had one
at some point).

>
> If branch 'foo' is currently checked out in the superproject, and
> branch 'bar' is currently checked out in the submodule, and someone
> runs 'git -c propagateBranches=true submodule update', wouldn't they expect
> that 'bar' be checked out in the submodule ? Maybe not, but the commit message
> and the tests should be more explicit about the expected behaviour in this case, I think.

Yeah, I'll call it out. I think this case is better addressed by having
`git checkout topic --recurse-submodules` automatically create "topic"
in the submodules that don't have it. This is one of the reasons why the
`git checkout` RFC isn't so relevant any more ;).

>> +	else
>> +		submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
>> +
>>  	if (!submodule_up_to_date || update_data->force)
>>  		if (run_update_procedure(update_data))
>>  			return 1;
>> @@ -2551,6 +2568,12 @@ static int update_submodules(struct update_data *update_data)
>>  		goto cleanup;
>>  	}
>>  
>> +	if (the_repository->settings.submodule_propagate_branches) {
>> +		struct branch *current_branch = branch_get(NULL);
>> +		if (current_branch)
>> +			update_data->super_branch = current_branch->name;
>
> OK, so this condition means that super_branch won't get set if we are not
> currently on a branch, i.e. we are in detached HEAD. This makes sense as there
> would be no branch to propagate. Do we need a test for this ? maybe a case where
> we clone with '--recurse-submodules --branch some-tag' ?

Good point, I'll add a test for this.

>> +	}
>> +
>>  	for (i = 0; i < suc.update_clone_nr; i++) {
>>  		struct update_clone_data ucd = suc.update_clone[i];
>>  
>> @@ -2634,6 +2657,7 @@ static int module_update(int argc, const char **argv, const char *prefix)
>>  	memset(&filter_options, 0, sizeof(filter_options));
>>  	argc = parse_options(argc, argv, prefix, module_update_options,
>>  			     git_submodule_helper_usage, 0);
>> +	prepare_repo_settings(the_repository);
>>  
>>  	if (opt.require_init)
>>  		opt.init = 1;
>> diff --git a/t/t5617-clone-submodules.sh b/t/t5617-clone-submodules.sh
>> index b5c66cb18cb..215fb02e9fb 100755
>> --- a/t/t5617-clone-submodules.sh
>> +++ b/t/t5617-clone-submodules.sh
>> @@ -12,10 +12,17 @@ pwd=$(pwd)
>>  test_expect_success 'setup' '
>>  	git checkout -b main &&
>>  	test_commit commit1 &&
>> +	mkdir subsub &&
>> +	(
>> +		cd subsub &&
>> +		git init &&
>> +		test_commit subsubcommit1
>> +	) &&
>>  	mkdir sub &&
>>  	(
>>  		cd sub &&
>>  		git init &&
>> +		git submodule add "file://$pwd/subsub" subsub &&
>>  		test_commit subcommit1 &&
>>  		git tag sub_when_added_to_super &&
>>  		git branch other
>> @@ -106,4 +113,31 @@ test_expect_success '--no-also-filter-submodules overrides clone.filterSubmodule
>>  	test_cmp_config -C super_clone3/sub false --default false remote.origin.promisor
>>  '
>>  
>> +test_expect_success 'submodule.propagateBranches checks out branches at correct commits' '
>> +	git -C sub checkout -b not-main &&
>> +	git -C subsub checkout -b not-main &&
>> +	git clone --recurse-submodules \
>> +		-c submodule.propagateBranches=true \
>> +		"file://$pwd/." super_clone4 &&
>> +
>> +	# Assert that each repo is pointing to "main"
>> +	for REPO in "super_clone4" "super_clone4/sub" "super_clone4/sub/subsub"
>> +	do
>> +	    HEAD_BRANCH=$(git -C $REPO symbolic-ref HEAD) &&
>> +	    test $HEAD_BRANCH = "refs/heads/main" || return 1
>> +	done &&
>> +
>> +	# Assert that the submodule branches are pointing to the right revs
>> +	EXPECT_SUB_OID="$(git -C super_clone4 rev-parse :sub)" &&
>> +	ACTUAL_SUB_OID="$(git -C super_clone4/sub rev-parse refs/heads/main)" &&
>> +	test $EXPECT_SUB_OID = $ACTUAL_SUB_OID &&
>> +	EXPECT_SUBSUB_OID="$(git -C super_clone4/sub rev-parse :subsub)" &&
>> +	ACTUAL_SUBSUB_OID="$(git -C super_clone4/sub/subsub rev-parse refs/heads/main)" &&
>> +	test $EXPECT_SUBSUB_OID = $ACTUAL_SUBSUB_OID &&
>> +
>> +	# Assert that the submodules do not have branches from their upstream
>> +	test_must_fail git -C super_clone4/sub rev-parse not-main &&
>> +	test_must_fail git -C super_clone4/sub/subsub rev-parse not-main
>> +'
>> +
>>  test_done
>> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
>> index 6cc07460dd2..00a6fec8912 100755
>> --- a/t/t7406-submodule-update.sh
>> +++ b/t/t7406-submodule-update.sh
>> @@ -1178,4 +1178,26 @@ test_expect_success 'submodule update --recursive skip submodules with strategy=
>>  	test_cmp expect.err actual.err
>>  '
>>  
>> +test_expect_success 'submodule update with submodule.propagateBranches checks out branches' '
>> +	test_when_finished "rm -fr top-cloned" &&
>> +	cp -r top-clean top-cloned &&
>> +
>> +	# Create a new upstream submodule
>> +	git init middle2 &&
>> +	test_commit -C middle2 "middle2" &&
>> +	git -C top submodule add ../middle2 middle2 &&
>> +	git -C top commit -m "add middle2" &&
>> +
>> +	git -C top-cloned checkout -b "new-branch" &&
>> +	git -C top-cloned pull origin main &&
>> +	test_config -C top-cloned submodule.propagateBranches true &&
>> +	git -C top-cloned submodule update --recursive &&
>> +
>> +	for REPO in "top-cloned/middle2" "top-cloned/middle" "top-cloned/middle/bottom"
>> +	do
>> +	    HEAD_BRANCH=$(git -C $REPO symbolic-ref HEAD) &&
>> +	    test $HEAD_BRANCH = "refs/heads/new-branch" || return 1
>> +	done
>> +'
>> +
>>  test_done
>> 
>
> These tests look good, but maybe more tests would be needed in 
> the light of my comments above... 
>
> Thanks again for working on improving submodules!

Thanks for lending your time and attention :)
>
> Cheers,
>
> Philippe.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH 6/6] clone, submodule update: check out branches
  2022-08-29 20:54 ` [PATCH 6/6] clone, submodule update: check out branches Glen Choo via GitGitGadget
  2022-08-30  4:03   ` Philippe Blain
@ 2022-09-01 20:00   ` Jonathan Tan
  1 sibling, 0 replies; 56+ messages in thread
From: Jonathan Tan @ 2022-09-01 20:00 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget; +Cc: Jonathan Tan, git, Philippe Blain, Glen Choo

"Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:
> From: Glen Choo <chooglen@google.com>
> 
> Teach "git submodule update" to update submodules by creating and
> checking out the current superproject branch when
> "submodule.propagateBranches=true".

"git submodule update" already knows how to update submodules; probably
better to say:

  Teach "git submodule update" to create and check out a branch of the
  same name as the current superproject branch when updating a submodule
  if "submodule.propagateBranches=true" is set on the superproject.

> With "submodule.propagateBranches=true", submodules are cloned with
> "--detach" so that they do not contain branches from their upstream.
> This prevents conflicts between branch names from the superproject and
> the branch names from the submodule's upstream. Arguably, "--detach"
> should also be the default for "submodule.propagateBranches=false"
> since it doesn't make sense to create a submodule branch when the
> submodule is always expected to be in detached HEAD.

This paragraph made me think of the use case in which we cloned a
submodule-using repo, made a commit in a submodule (thus advancing a
branch) without a corresponding commit in a superproject, and then
recloned our clone, hoping that the state will persist. It would not
persist, but as stated here, the existing behavior is already that
branches in submodules are not cloned, so retaining this existing
behavior is not a problem.

> "git submodule update" tries to create the branch as long as it is not
> currently checked out, thus it will fail if the submodule has the
> branch, but it is not checked out. This is fine because the main purpose
> of "git submodule update" is to clone new submodules (which have no
> branches, and will never have this problem). "git checkout" with
> "submodule.propagateBranches" will cover the use case of recursively
> checking out an existing branch.

In regular usage, the user will, as you say, run "git checkout". So when
"git submodule update" is run, a submodule will either have no branches
(because it was just cloned or because we have never switched to that
branch before in the superproject) or it will have the correct branch
already checked out, so it would already be considered up to date (no
matter whether the commit matches with the superproject's gitlink: only
the name of the branch matters).

I'm concerned about the case in which the user, say, has created a
branch in a submodule for some reason. E.g.:

  (cd sub; git branch my-branch)
  git checkout my-branch

so this would fail because we wouldn't be able to create "my-branch" in
the "sub" submodule. We might need a message explaining what can be done
to fix this situation, but for now, maybe a NEEDSWORK will suffice.

> @@ -2206,6 +2214,8 @@ static int run_update_command(struct update_data *ud, int subforce)
>  		strvec_pushl(&cp.args, "checkout", "-q", NULL);
>  		if (subforce)
>  			strvec_push(&cp.args, "-f");
> +		if (ud->super_branch)
> +			strvec_pushl(&cp.args, "-b", ud->super_branch, NULL);

Here is where the NEEDSWORK would go.

> @@ -106,4 +113,31 @@ test_expect_success '--no-also-filter-submodules overrides clone.filterSubmodule
>  	test_cmp_config -C super_clone3/sub false --default false remote.origin.promisor
>  '
>  
> +test_expect_success 'submodule.propagateBranches checks out branches at correct commits' '
> +	git -C sub checkout -b not-main &&
> +	git -C subsub checkout -b not-main &&
> +	git clone --recurse-submodules \
> +		-c submodule.propagateBranches=true \
> +		"file://$pwd/." super_clone4 &&
> +
> +	# Assert that each repo is pointing to "main"
> +	for REPO in "super_clone4" "super_clone4/sub" "super_clone4/sub/subsub"
> +	do
> +	    HEAD_BRANCH=$(git -C $REPO symbolic-ref HEAD) &&
> +	    test $HEAD_BRANCH = "refs/heads/main" || return 1
> +	done &&
> +
> +	# Assert that the submodule branches are pointing to the right revs
> +	EXPECT_SUB_OID="$(git -C super_clone4 rev-parse :sub)" &&
> +	ACTUAL_SUB_OID="$(git -C super_clone4/sub rev-parse refs/heads/main)" &&
> +	test $EXPECT_SUB_OID = $ACTUAL_SUB_OID &&
> +	EXPECT_SUBSUB_OID="$(git -C super_clone4/sub rev-parse :subsub)" &&
> +	ACTUAL_SUBSUB_OID="$(git -C super_clone4/sub/subsub rev-parse refs/heads/main)" &&
> +	test $EXPECT_SUBSUB_OID = $ACTUAL_SUBSUB_OID &&
> +
> +	# Assert that the submodules do not have branches from their upstream
> +	test_must_fail git -C super_clone4/sub rev-parse not-main &&
> +	test_must_fail git -C super_clone4/sub/subsub rev-parse not-main
> +'

Instead of reusing "main", can we use a branch name that exists in the
superproject but not the submodule? Here, we cannot tell the difference
between git reusing the referent of submodule's "main" versus git using
the gitlink in superproject's "main".

I'll write some more comments on the other patches, but overall this
patch set makes sense to me.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH 4/6] submodule: return target of submodule symref
  2022-08-29 20:54 ` [PATCH 4/6] submodule: return target of submodule symref Glen Choo via GitGitGadget
@ 2022-09-01 20:01   ` Jonathan Tan
  2022-09-01 20:46     ` Glen Choo
  0 siblings, 1 reply; 56+ messages in thread
From: Jonathan Tan @ 2022-09-01 20:01 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget; +Cc: Jonathan Tan, git, Philippe Blain, Glen Choo

"Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:
> diff --git a/refs.c b/refs.c
> index 90bcb271687..d72015c95e9 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -1784,19 +1784,21 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
>  }
>  
>  int resolve_gitlink_ref(const char *submodule, const char *refname,
> -			struct object_id *oid)
> +			struct object_id *oid, const char **referent_out)

s/referent/target/ throughout this patch, I think.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH 4/6] submodule: return target of submodule symref
  2022-09-01 20:01   ` Jonathan Tan
@ 2022-09-01 20:46     ` Glen Choo
  0 siblings, 0 replies; 56+ messages in thread
From: Glen Choo @ 2022-09-01 20:46 UTC (permalink / raw)
  To: Jonathan Tan, Glen Choo via GitGitGadget
  Cc: Jonathan Tan, git, Philippe Blain

Jonathan Tan <jonathantanmy@google.com> writes:

> "Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:
>> diff --git a/refs.c b/refs.c
>> index 90bcb271687..d72015c95e9 100644
>> --- a/refs.c
>> +++ b/refs.c
>> @@ -1784,19 +1784,21 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
>>  }
>>  
>>  int resolve_gitlink_ref(const char *submodule, const char *refname,
>> -			struct object_id *oid)
>> +			struct object_id *oid, const char **referent_out)
>
> s/referent/target/ throughout this patch, I think.

I prefer the word "target", but this is a break from existing
conventions, e.g.

  int refs_read_symbolic_ref(struct ref_store *ref_store, const char *refname,
          struct strbuf *referent);

We can do this change, but I think we should also change this everywhere
if we do.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* [PATCH v2 0/7] clone, submodule update: check out submodule branches
  2022-08-29 20:54 [PATCH 0/6] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
                   ` (5 preceding siblings ...)
  2022-08-29 20:54 ` [PATCH 6/6] clone, submodule update: check out branches Glen Choo via GitGitGadget
@ 2022-10-20 20:20 ` Glen Choo via GitGitGadget
  2022-10-20 20:20   ` [PATCH v2 1/7] clone: teach --detach option Glen Choo via GitGitGadget
                     ` (8 more replies)
  6 siblings, 9 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-20 20:20 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo

Thanks Philippe and Jonathan for your review on the previous version :)
Sorry for the long delay between iterations, I got wrapped up in other,
unrelated work.

The biggest change in this version is that "git submodule update" now
creates the branch when the submodule is cloned, not when the submodule's
worktree is updated. So when the worktree is updated, we use "git checkout "
instead of "git checkout -b ". This is more robust to branches created
directly in the submodule (which was pointed out by both reviewers), but
also allows us to add matching behavior to "git submodule add ", which also
clones + checks out submodules.

During the v1 discussion, I realize that my idea of the new submodule UX has
already diverged from what was initially communicated to the list. I plan to
check in a technical document describing the plans for new submodule UX,
which should hopefully make these discussions smoother (e.g. the commit
message in patch 7 can make reference to the doc).

= Description

This series teaches "git clone --recurse-submodules" and "git submodule
update" to understand "submodule.propagateBranches" (see Further Reading for
context), i.e. if the superproject has a branch checked out and a submodule
is cloned, the submodule will have the same branch checked out.

To do this, "git submodule update" checks if "submodule.propagateBranches"
is true, and if so, gets the current superproject branch and updates the
submodule by running "git checkout -b my-superproject-branch". Since "git
clone --recurse-submodules" is implemented using "git submodule update", it
also learns this trick.

The main challenges with this approach are:

 * If the remote HEAD points to a branch, "git clone" always creates that
   branch in the clone. But with "submodule.propagateBranches", we want
   submodules to use the branch names of their superproject, not their
   upstream.
   
   This is solved by adding a new flag to "git clone", "--detach", which
   detaches the clone's HEAD at the branch and does not create it.

 * When "git submodule update" recurses into submodules, the parent process
   has to propagate the value of "submodule.propagateBranches" to child
   processes, otherwise the behavior will be inconsistent if the submodule
   has the config unset.
   
   This is solved by adding an internal GIT_* environment variable and
   passing it down via prepare_submodule_repo_env(). This is cleaner than
   passing "-c submodule.propagateBranches=true", but an even cleaner
   solution would be for submodules to read "submodule.propagateBranches"
   from their superproject config. This would also be useful for
   "submodule.alternateLocation" and "submodule.alternateErrorStrategy", as
   we wouldn't have to set those values in newly-cloned submodules. This
   requires teaching Git to treat submodules differently, which was the
   subject of some WIP in [1]. That topic has stalled, but I don't mind
   restarting it if others prefer that.

[1]
https://lore.kernel.org/git/20220310004423.2627181-1-emilyshaffer@google.com/

= Patch organization

 * Patch 1/7 adds "--detach" to "git clone"
 * Patch 2/7 creates the environment variable and repository setting for
   "submodule.propagateBranches"
 * Patch 3/7 adds a new "--branch" option to "git submodule clone", which
   makes it create a named branch.
 * Patches 4-6/7 are prep work, and 7/7 adds the actual
   "submodule.propagateBranches" behavior

= Series history

Changes in v2:

 * The superproject's "submodule.propagateBranches" value is always used,
   even if false.
 * Branches are now created at clone time (by adding a new flag to "git
   submodule clone"), instead of at update time.
 * Rebase onto newer master. This got adjusted slightly to incorporate
   ab/submodule-helper-leakfix.
 * Add more tests to demonstrate edge case behavior.
 * Assorted commit message and doc improvements.

= Future work

 * Patch 5, which refactors resolve_gitlink_ref(), notes that a better
   interface would be to return the refname instead of using an "out"
   parameter, but we use an "out" parameter so that any new callers trying
   to use the old function signature will get stopped by the compiler. The
   refactor can be finished at a later time.

 * Patch 5 uses the name "target" when we are talking about what a symref
   points to, instead of "referent" like the other functions. "target" is
   the better choice, since "referent" could also apply to non-symbolic
   refs, but that cleanup is quite big.

= Further reading

Submodule branching RFC:
https://lore.kernel.org/git/kl6lv912uvjv.fsf@chooglen-macbookpro.roam.corp.google.com/

Original Submodule UX RFC/Discussion:
https://lore.kernel.org/git/YHofmWcIAidkvJiD@google.com/

Contributor Summit submodules Notes:
https://lore.kernel.org/git/nycvar.QRO.7.76.6.2110211148060.56@tvgsbejvaqbjf.bet/

Submodule UX overhaul updates:
https://lore.kernel.org/git/?q=Submodule+UX+overhaul+update

"git branch --recurse-submodules":
https://lore.kernel.org/git/20220129000446.99261-1-chooglen@google.com/

Glen Choo (7):
  clone: teach --detach option
  repo-settings: add submodule_propagate_branches
  submodule--helper clone: create named branch
  t5617: drop references to remote-tracking branches
  submodule: return target of submodule symref
  submodule update: refactor update targets
  clone, submodule update: create and check out branches

 Documentation/git-clone.txt                   |  8 +-
 builtin/branch.c                              | 11 +--
 builtin/clone.c                               | 12 ++-
 builtin/submodule--helper.c                   | 91 ++++++++++++++++---
 builtin/update-index.c                        |  4 +-
 cache.h                                       |  1 +
 combine-diff.c                                |  3 +-
 diff-lib.c                                    |  2 +-
 dir.c                                         |  2 +-
 object-file.c                                 |  2 +-
 read-cache.c                                  |  4 +-
 refs.c                                        | 10 +-
 refs.h                                        |  5 +-
 repo-settings.c                               | 10 ++
 repository.h                                  |  1 +
 submodule.c                                   |  2 +
 t/t5601-clone.sh                              | 22 +++++
 ...es-remote.sh => t5617-clone-submodules.sh} | 36 +++++++-
 t/t7406-submodule-update.sh                   | 64 +++++++++++++
 unpack-trees.c                                |  3 +-
 20 files changed, 251 insertions(+), 42 deletions(-)
 rename t/{t5617-clone-submodules-remote.sh => t5617-clone-submodules.sh} (72%)


base-commit: 45c9f05c44b1cb6bd2d6cb95a22cf5e3d21d5b63
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1321%2Fchooglen%2Fsubmodule%2Fclone-recursive-with-branch-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1321/chooglen/submodule/clone-recursive-with-branch-v2
Pull-Request: https://github.com/git/git/pull/1321

Range-diff vs v1:

 1:  936e237c716 ! 1:  432bc7cb3a4 clone: teach --detach option
     @@ Commit message
          clone: teach --detach option
      
          Teach "git clone" the "--detach" option, which leaves the cloned repo in
     -    detached HEAD (like "git checkout --detach"). If the clone is not bare,
     -    the remote's HEAD branch is also not created (bare clones always copy
     -    all remote branches directly to local branches, so the branch is still
     -    created in the bare case).
     +    detached HEAD (like "git checkout --detach"). In addition, if the clone
     +    is not bare, do not create the local branch pointed to by the remote's
     +    HEAD symref (bare clones always copy all remote branches directly to
     +    local branches, so the branch is still created in the bare case).
      
          This is especially useful in the "submodule.propagateBranches" workflow,
     -    where the submodule branch names match the superproject's branch names,
     -    so it makes no sense to name the branches after the submodule's remote's
     -    branches.
     +    where local submodule branches are named after the superproject's
     +    branches, so it makes no sense to create a local branch named after the
     +    submodule's remote's branch.
      
          Signed-off-by: Glen Choo <chooglen@google.com>
      
     @@ Documentation/git-clone.txt: objects from the source repository into a pack in t
       
      +--detach::
      +	If the cloned repository's HEAD points to a branch, point the newly
     -+	created HEAD to the branch's commit instead of the branch itself. In a
     -+	non-bare repository, the branch will not be created.
     ++	created HEAD to the branch's commit instead of the branch itself.
     ++	Additionally, in a non-bare repository, the corresponding local branch
     ++	will not be created.
      +
       -u <upload-pack>::
       --upload-pack <upload-pack>::
     @@ builtin/clone.c: static int option_filter_submodules = -1;    /* unspecified */
       static struct string_list server_options = STRING_LIST_INIT_NODUP;
       static int option_remote_submodules;
      +static int option_detach;
     + static const char *bundle_uri;
       
       static int recurse_submodules_cb(const struct option *opt,
     - 				 const char *arg, int unset)
      @@ builtin/clone.c: static struct option builtin_clone_options[] = {
       		    N_("any cloned submodules will use their remote-tracking branch")),
       	OPT_BOOL(0, "sparse", &option_sparse_checkout,
       		    N_("initialize sparse-checkout file to include only files at root")),
      +	OPT_BOOL(0, "detach", &option_detach,
     -+		 N_("detach HEAD and don't create branch")),
     ++		 N_("detach HEAD and don't create a local branch")),
     + 	OPT_STRING(0, "bundle-uri", &bundle_uri,
     + 		   N_("uri"), N_("a URI for downloading bundles before fetching from origin remote")),
       	OPT_END()
     - };
     - 
      @@ builtin/clone.c: static void update_remote_refs(const struct ref *refs,
       }
       
 2:  35f068eb031 ! 2:  20499c62065 repo-settings: add submodule_propagate_branches
     @@ Commit message
          or submodule, otherwise the behavior may be inconsistent if the
          repositories don't agree on the config.
      
     -    We haven't needed a way to propagate the config because because the only
     -    command that reads "submodule.propagateBranches" is "git branch", which
     -    only has one mode of operation with "--recurse-submodules". However, a
     -    future commit will teach "submodule.propagateBranches" to "git submodule
     +    We haven't needed a way to propagate the config because the only command
     +    that reads "submodule.propagateBranches" is "git branch", which only has
     +    one mode of operation with "--recurse-submodules". However, a future
     +    commit will teach "submodule.propagateBranches" to "git submodule
          update", making this necessary.
      
          Propagate "submodule.propagateBranches" to child processes by adding a
     @@ cache.h: static inline enum object_type object_type(unsigned int mode)
      
       ## repo-settings.c ##
      @@ repo-settings.c: void prepare_repo_settings(struct repository *r)
     - 	repo_cfg_bool(r, "pack.usesparse", &r->settings.pack_use_sparse, 1);
     - 	repo_cfg_bool(r, "core.multipackindex", &r->settings.core_multi_pack_index, 1);
     - 	repo_cfg_bool(r, "index.sparse", &r->settings.sparse_index, 0);
     -+	repo_cfg_bool(r, "submodule.propagateBranches", &r->settings.submodule_propagate_branches, 0);
     - 
     - 	/*
     --	 * The GIT_TEST_MULTI_PACK_INDEX variable is special in that
     --	 * either it *or* the config sets
     --	 * r->settings.core_multi_pack_index if true. We don't take
     --	 * the environment variable if it exists (even if false) over
     --	 * any config, as in most other cases.
     -+	 * Boolean settings from config and environment variables. Only
     -+	 * take the environment variable if it is true, otherwise, use
     -+	 * the config.
     - 	 */
       	if (git_env_bool(GIT_TEST_MULTI_PACK_INDEX, 0))
       		r->settings.core_multi_pack_index = 1;
     -+	if (git_env_bool(GIT_SUBMODULE_PROPAGATE_BRANCHES_ENVIRONMENT, 0))
     -+		r->settings.submodule_propagate_branches = 1;
       
     ++	/*
     ++	 * If the environment variable is set, assume that it came from the
     ++	 * superproject and ignore the config.
     ++	 */
     ++	r->settings.submodule_propagate_branches =
     ++		git_env_bool(GIT_SUBMODULE_PROPAGATE_BRANCHES_ENVIRONMENT, -1);
     ++	if (r->settings.submodule_propagate_branches == -1)
     ++		repo_cfg_bool(r, "submodule.propagateBranches",
     ++			      &r->settings.submodule_propagate_branches, 0);
     ++
       	/*
       	 * Non-boolean config
     + 	 */
      
       ## repository.h ##
      @@ repository.h: struct repo_settings {
 -:  ----------- > 3:  a4056e200ed submodule--helper clone: create named branch
 3:  d8a2faf9ce7 = 4:  affd0e24e1d t5617: drop references to remote-tracking branches
 4:  eab40dcb296 ! 5:  6f769cb80ad submodule: return target of submodule symref
     @@ builtin/submodule--helper.c: static int update_submodule(struct update_data *upd
      -	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD", &update_data->suboid))
      +	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
      +				     &update_data->suboid, NULL))
     - 		die(_("Unable to find current revision in submodule path '%s'"),
     - 			update_data->displaypath);
     + 		return die_message(_("Unable to find current revision in submodule path '%s'"),
     + 				   update_data->displaypath);
       
      @@ builtin/submodule--helper.c: static int update_submodule(struct update_data *update_data)
     - 				    update_data->sm_path);
     + 						   update_data->sm_path);
       		}
       
      -		if (resolve_gitlink_ref(update_data->sm_path, remote_ref, &update_data->oid))
      +		if (resolve_gitlink_ref(update_data->sm_path, remote_ref,
      +					&update_data->oid, NULL))
     - 			die(_("Unable to find %s revision in submodule path '%s'"),
     - 			    remote_ref, update_data->sm_path);
     + 			return die_message(_("Unable to find %s revision in submodule path '%s'"),
     + 					   remote_ref, update_data->sm_path);
       
      @@ builtin/submodule--helper.c: static void die_on_repo_without_commits(const char *path)
       	strbuf_addstr(&sb, path);
     @@ refs.c: const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
       
       int resolve_gitlink_ref(const char *submodule, const char *refname,
      -			struct object_id *oid)
     -+			struct object_id *oid, const char **referent_out)
     ++			struct object_id *oid, const char **target_out)
       {
       	struct ref_store *refs;
       	int flags;
     -+	const char *referent;
     ++	const char *target;
       
       	refs = get_submodule_ref_store(submodule);
       
     @@ refs.c: const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
      -
      -	if (!refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags) ||
      -	    is_null_oid(oid))
     -+	referent = refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags);
     -+	if (!referent || is_null_oid(oid))
     ++	target = refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags);
     ++	if (!target || is_null_oid(oid))
       		return -1;
     -+	if (referent_out)
     -+		*referent_out = referent;
     ++	if (target_out)
     ++		*target_out = target;
       	return 0;
       }
       
     @@ refs.h: int peel_iterated_oid(const struct object_id *base, struct object_id *pe
        * successful, return 0 and set oid to the name of the object;
        * otherwise, return a non-zero value.
      + *
     -+ * FIXME: Return "referent" just like refs_resolve_ref_unsafe(). This will be
     ++ * FIXME: Return "target" just like refs_resolve_ref_unsafe(). This will be
      + * safe to do once we merge resolve_gitlink_ref() into master.
        */
       int resolve_gitlink_ref(const char *submodule, const char *refname,
      -			struct object_id *oid);
     -+			struct object_id *oid, const char **referent);
     ++			struct object_id *oid, const char **target);
       
       /*
        * Return true iff abbrev_name is a possible abbreviation for
 5:  513d18e56d3 ! 6:  abdfa888ff5 submodule--helper: refactor up-to-date criterion
     @@ Metadata
      Author: Glen Choo <chooglen@google.com>
      
       ## Commit message ##
     -    submodule--helper: refactor up-to-date criterion
     +    submodule update: refactor update targets
      
     -    Refactor builtin/submodule--helper.c:update_submodule() to check if the
     -    submodule is up to date using a variable instead of checking the oids
     -    directly. In a subsequent commit, we will expand the definition of "up
     -    to date" to include checked out branches.
     +    Refactor two "git submodule update" code locations so that they no
     +    longer refer to oids directly. This shrinks the next commit's diff,
     +    where this code will need to handle branches.
      
          Signed-off-by: Glen Choo <chooglen@google.com>
      
       ## builtin/submodule--helper.c ##
     -@@ builtin/submodule--helper.c: static void update_data_to_args(struct update_data *update_data, struct strvec *
     +@@ builtin/submodule--helper.c: static int fetch_in_submodule(const char *module_path, int depth, int quiet,
     + static int run_update_command(const struct update_data *ud, int subforce)
     + {
     + 	struct child_process cp = CHILD_PROCESS_INIT;
     +-	char *oid = oid_to_hex(&ud->oid);
     ++	const char *update_target = oid_to_hex(&ud->oid);;
     + 	int ret;
     + 
     + 	switch (ud->update_strategy.type) {
     +@@ builtin/submodule--helper.c: static int run_update_command(const struct update_data *ud, int subforce)
     + 		BUG("unexpected update strategy type: %d",
     + 		    ud->update_strategy.type);
     + 	}
     +-	strvec_push(&cp.args, oid);
     ++	strvec_push(&cp.args, update_target);
     + 
     + 	cp.dir = ud->sm_path;
     + 	prepare_submodule_repo_env(&cp.env);
     +@@ builtin/submodule--helper.c: static int run_update_command(const struct update_data *ud, int subforce)
     + 		switch (ud->update_strategy.type) {
     + 		case SM_UPDATE_CHECKOUT:
     + 			die_message(_("Unable to checkout '%s' in submodule path '%s'"),
     +-				    oid, ud->displaypath);
     ++				    update_target, ud->displaypath);
     + 			/* No "ret" assignment, use "git checkout"'s */
     + 			break;
     + 		case SM_UPDATE_REBASE:
     + 			ret = die_message(_("Unable to rebase '%s' in submodule path '%s'"),
     +-					  oid, ud->displaypath);
     ++					  update_target, ud->displaypath);
     + 			break;
     + 		case SM_UPDATE_MERGE:
     + 			ret = die_message(_("Unable to merge '%s' in submodule path '%s'"),
     +-					  oid, ud->displaypath);
     ++					  update_target, ud->displaypath);
     + 			break;
     + 		case SM_UPDATE_COMMAND:
     + 			ret = die_message(_("Execution of '%s %s' failed in submodule path '%s'"),
     +-					  ud->update_strategy.command, oid, ud->displaypath);
     ++					  ud->update_strategy.command, update_target, ud->displaypath);
     + 			break;
     + 		default:
     + 			BUG("unexpected update strategy type: %d",
     +@@ builtin/submodule--helper.c: static int run_update_command(const struct update_data *ud, int subforce)
     + 	switch (ud->update_strategy.type) {
     + 	case SM_UPDATE_CHECKOUT:
     + 		printf(_("Submodule path '%s': checked out '%s'\n"),
     +-		       ud->displaypath, oid);
     ++		       ud->displaypath, update_target);
     + 		break;
     + 	case SM_UPDATE_REBASE:
     + 		printf(_("Submodule path '%s': rebased into '%s'\n"),
     +-		       ud->displaypath, oid);
     ++		       ud->displaypath, update_target);
     + 		break;
     + 	case SM_UPDATE_MERGE:
     + 		printf(_("Submodule path '%s': merged in '%s'\n"),
     +-		       ud->displaypath, oid);
     ++		       ud->displaypath, update_target);
     + 		break;
     + 	case SM_UPDATE_COMMAND:
     + 		printf(_("Submodule path '%s': '%s %s'\n"),
     +-		       ud->displaypath, ud->update_strategy.command, oid);
     ++		       ud->displaypath, ud->update_strategy.command, update_target);
     + 		break;
     + 	default:
     + 		BUG("unexpected update strategy type: %d",
     +@@ builtin/submodule--helper.c: static void update_data_to_args(const struct update_data *update_data,
       
       static int update_submodule(struct update_data *update_data)
       {
      +	int submodule_up_to_date;
     -+
     - 	ensure_core_worktree(update_data->sm_path);
     + 	int ret;
       
     - 	update_data->displaypath = get_submodule_displaypath(
     + 	ret = determine_submodule_update_strategy(the_repository,
      @@ builtin/submodule--helper.c: static int update_submodule(struct update_data *update_data)
       		free(remote_ref);
       	}
       
     --	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
     +-	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force) {
      +	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
     -+	if (!submodule_up_to_date || update_data->force)
     - 		if (run_update_procedure(update_data))
     - 			return 1;
     - 
     ++	if (!submodule_up_to_date || update_data->force) {
     + 		ret = run_update_procedure(update_data);
     + 		if (ret)
     + 			return ret;
 6:  6f7f2f9a3f1 ! 7:  3f98b0d1739 clone, submodule update: check out branches
     @@ Metadata
      Author: Glen Choo <chooglen@google.com>
      
       ## Commit message ##
     -    clone, submodule update: check out branches
     +    clone, submodule update: create and check out branches
      
     -    Teach "git submodule update" to update submodules by creating and
     -    checking out the current superproject branch when
     -    "submodule.propagateBranches=true". "git clone --recurse-submodules"
     -    also learns this trick because it is implemented with "git submodule
     -    update --recursive".
     +    Teach "git submodule update" to:
      
     -    With "submodule.propagateBranches=true", submodules are cloned with
     -    "--detach" so that they do not contain branches from their upstream.
     -    This prevents conflicts between branch names from the superproject and
     -    the branch names from the submodule's upstream. Arguably, "--detach"
     -    should also be the default for "submodule.propagateBranches=false"
     -    since it doesn't make sense to create a submodule branch when the
     -    submodule is always expected to be in detached HEAD. But, to be
     -    conservative, this commit does not change the behavior of
     -    "submodule.propagateBranches=false".
     +    - create the branch with the same name as the current superproject
     +      branch when cloning a submodule
     +    - check out that branch (instead of the commit OID) when updating
     +      the submodule worktree
      
     -    "git submodule update" tries to create the branch as long as it is not
     -    currently checked out, thus it will fail if the submodule has the
     -    branch, but it is not checked out. This is fine because the main purpose
     -    of "git submodule update" is to clone new submodules (which have no
     -    branches, and will never have this problem). "git checkout" with
     -    "submodule.propagateBranches" will cover the use case of recursively
     -    checking out an existing branch.
     +    when submodule branching is enabled (submodule.propagateBranches = true)
     +    on the superproject and a branch is checked out. "git clone
     +    --recurse-submodules" also learns this trick because it is implemented
     +    with "git submodule update --recursive".
     +
     +    This approach of checking out the branch will not result in a dirty
     +    worktree for freshly cloned submodules because we can ensure that the
     +    submodule branch points to the superproject gitlink. In other cases, it
     +    does not work as well, but we can handle them incrementally:
     +
     +    - "git pull --recurse-submodules" merges the superproject tree (without
     +      updating the submodule branches), and runs "git submodule update" to
     +      update the worktrees, so it is almost guaranteed to result in a dirty
     +      worktree.
     +
     +      The implementation of "git pull --recurse-submodules" is likely to
     +      change drastically as submodule.propagateBranches work progresses
     +      (e.g. "git merge" learns to recurse in to submodules), and we may be
     +      able to replace the "git submodule update" invocation, or teach it new
     +      tricks that make the update behave well.
     +
     +    - The user might make changes to the submodule branch without committing
     +      them back to superproject. This is primarily affects "git checkout
     +      --recurse-submodules", since that is the primary way of switching away
     +      from a branch and leaving behind WIP (as opposed to "git submodule
     +      update", which is run post-checkout).
     +
     +      In a future series, "git checkout --recurse-submodules" will learn to
     +      consider submodule branches. We can introduce appropriate guardrails
     +      then, e.g. requiring that the superproject working tree is not dirty
     +      before switching away.
      
          Signed-off-by: Glen Choo <chooglen@google.com>
      
       ## builtin/submodule--helper.c ##
     -@@ builtin/submodule--helper.c: static int clone_submodule(struct module_clone_data *clone_data)
     - 			strvec_push(&cp.args, clone_data->single_branch ?
     - 				    "--single-branch" :
     - 				    "--no-single-branch");
     -+		if (the_repository->settings.submodule_propagate_branches)
     -+			strvec_push(&cp.args, "--detach");
     -+
     - 
     - 		strvec_push(&cp.args, "--");
     - 		strvec_push(&cp.args, clone_data->url);
     -@@ builtin/submodule--helper.c: static int clone_submodule(struct module_clone_data *clone_data)
     - 	if (error_strategy)
     - 		git_config_set_in_file(p, "submodule.alternateErrorStrategy",
     - 				       error_strategy);
     -+	if (the_repository->settings.submodule_propagate_branches)
     -+		git_config_set_in_file(p, "submodule.propagateBranches",
     -+				       "true");
     - 
     - 	free(sm_alternate);
     - 	free(error_strategy);
     -@@ builtin/submodule--helper.c: static int module_clone(int argc, const char **argv, const char *prefix)
     - 	memset(&filter_options, 0, sizeof(filter_options));
     - 	argc = parse_options(argc, argv, prefix, module_clone_options,
     - 			     git_submodule_helper_usage, 0);
     -+	prepare_repo_settings(the_repository);
     - 
     - 	clone_data.dissociate = !!dissociate;
     - 	clone_data.quiet = !!quiet;
     -@@ builtin/submodule--helper.c: struct submodule_update_clone {
     +@@ builtin/submodule--helper.c: static void submodule_update_clone_release(struct submodule_update_clone *suc)
       struct update_data {
       	const char *prefix;
     - 	const char *displaypath;
     + 	char *displaypath;
      +	const char *super_branch;
       	enum submodule_update_type update_default;
       	struct object_id suboid;
       	struct string_list references;
     -@@ builtin/submodule--helper.c: static int run_update_command(struct update_data *ud, int subforce)
     - 		strvec_pushl(&cp.args, "checkout", "-q", NULL);
     - 		if (subforce)
     - 			strvec_push(&cp.args, "-f");
     -+		if (ud->super_branch)
     -+			strvec_pushl(&cp.args, "-b", ud->super_branch, NULL);
     - 		break;
     - 	case SM_UPDATE_REBASE:
     +@@ builtin/submodule--helper.c: static int prepare_to_clone_next_submodule(const struct cache_entry *ce,
     + 		strvec_push(&child->args, suc->update_data->single_branch ?
     + 					      "--single-branch" :
     + 					      "--no-single-branch");
     ++	if (ud->super_branch) {
     ++		strvec_pushf(&child->args, "--branch=%s", ud->super_branch);
     ++		strvec_pushf(&child->args, "--branch-oid=%s", oid_to_hex(&ce->oid));
     ++	}
     + 
     + cleanup:
     + 	free(displaypath);
     +@@ builtin/submodule--helper.c: static int fetch_in_submodule(const char *module_path, int depth, int quiet,
     + static int run_update_command(const struct update_data *ud, int subforce)
     + {
     + 	struct child_process cp = CHILD_PROCESS_INIT;
     +-	const char *update_target = oid_to_hex(&ud->oid);;
     ++	const char *update_target;
     + 	int ret;
     + 
     ++	if (ud->update_strategy.type == SM_UPDATE_CHECKOUT &&
     ++	    ud->super_branch)
     ++		update_target = ud->super_branch;
     ++	else
     ++		update_target = oid_to_hex(&ud->oid);
     ++
     + 	switch (ud->update_strategy.type) {
     + 	case SM_UPDATE_CHECKOUT:
       		cp.git_cmd = 1;
     -@@ builtin/submodule--helper.c: static void update_data_to_args(struct update_data *update_data, struct strvec *
     - static int update_submodule(struct update_data *update_data)
     +@@ builtin/submodule--helper.c: static int update_submodule(struct update_data *update_data)
       {
       	int submodule_up_to_date;
     -+	const char *submodule_head = NULL;
     - 
     - 	ensure_core_worktree(update_data->sm_path);
     + 	int ret;
     ++	const char *submodule_head = "HEAD";
       
     + 	ret = determine_submodule_update_strategy(the_repository,
     + 						  update_data->just_cloned,
      @@ builtin/submodule--helper.c: static int update_submodule(struct update_data *update_data)
       	if (update_data->just_cloned)
       		oidcpy(&update_data->suboid, null_oid());
       	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
      -				     &update_data->suboid, NULL))
      +				     &update_data->suboid, &submodule_head))
     - 		die(_("Unable to find current revision in submodule path '%s'"),
     - 			update_data->displaypath);
     + 		return die_message(_("Unable to find current revision in submodule path '%s'"),
     + 				   update_data->displaypath);
       
      @@ builtin/submodule--helper.c: static int update_submodule(struct update_data *update_data)
       		free(remote_ref);
       	}
       
      -	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
     -+	if (update_data->super_branch &&
     -+	    submodule_head &&
     -+	    !skip_prefix(submodule_head, "refs/heads/", &submodule_head))
     ++	if (!update_data->super_branch)
     ++		submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
     ++	else if (skip_prefix(submodule_head, "refs/heads/", &submodule_head))
      +		submodule_up_to_date = !strcmp(update_data->super_branch, submodule_head);
     ++	/* submodule_branch is "HEAD"; the submodule is in detached HEAD */
      +	else
     -+		submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
     ++		submodule_up_to_date = 0;
      +
     - 	if (!submodule_up_to_date || update_data->force)
     - 		if (run_update_procedure(update_data))
     - 			return 1;
     + 	if (!submodule_up_to_date || update_data->force) {
     + 		ret = run_update_procedure(update_data);
     + 		if (ret)
      @@ builtin/submodule--helper.c: static int update_submodules(struct update_data *update_data)
     - 		goto cleanup;
     - 	}
     + 	int i, ret = 0;
     + 	struct submodule_update_clone suc = SUBMODULE_UPDATE_CLONE_INIT;
       
      +	if (the_repository->settings.submodule_propagate_branches) {
      +		struct branch *current_branch = branch_get(NULL);
     @@ builtin/submodule--helper.c: static int update_submodules(struct update_data *up
      +			update_data->super_branch = current_branch->name;
      +	}
      +
     - 	for (i = 0; i < suc.update_clone_nr; i++) {
     - 		struct update_clone_data ucd = suc.update_clone[i];
     - 
     + 	suc.update_data = update_data;
     + 	run_processes_parallel_tr2(suc.update_data->max_jobs, update_clone_get_next_task,
     + 				   update_clone_start_failure,
      @@ builtin/submodule--helper.c: static int module_update(int argc, const char **argv, const char *prefix)
     - 	memset(&filter_options, 0, sizeof(filter_options));
     + 
       	argc = parse_options(argc, argv, prefix, module_update_options,
       			     git_submodule_helper_usage, 0);
      +	prepare_repo_settings(the_repository);
     @@ builtin/submodule--helper.c: static int module_update(int argc, const char **arg
       		opt.init = 1;
      
       ## t/t5617-clone-submodules.sh ##
     -@@ t/t5617-clone-submodules.sh: pwd=$(pwd)
     - test_expect_success 'setup' '
     +@@ t/t5617-clone-submodules.sh: test_expect_success 'setup' '
     + 	git config --global protocol.file.allow always &&
       	git checkout -b main &&
       	test_commit commit1 &&
      +	mkdir subsub &&
     @@ t/t7406-submodule-update.sh: test_expect_success 'submodule update --recursive s
       	test_cmp expect.err actual.err
       '
       
     -+test_expect_success 'submodule update with submodule.propagateBranches checks out branches' '
     -+	test_when_finished "rm -fr top-cloned" &&
     -+	cp -r top-clean top-cloned &&
     -+
     -+	# Create a new upstream submodule
     -+	git init middle2 &&
     -+	test_commit -C middle2 "middle2" &&
     -+	git -C top submodule add ../middle2 middle2 &&
     -+	git -C top commit -m "add middle2" &&
     -+
     -+	git -C top-cloned checkout -b "new-branch" &&
     -+	git -C top-cloned pull origin main &&
     -+	test_config -C top-cloned submodule.propagateBranches true &&
     -+	git -C top-cloned submodule update --recursive &&
     -+
     -+	for REPO in "top-cloned/middle2" "top-cloned/middle" "top-cloned/middle/bottom"
     -+	do
     -+	    HEAD_BRANCH=$(git -C $REPO symbolic-ref HEAD) &&
     -+	    test $HEAD_BRANCH = "refs/heads/new-branch" || return 1
     -+	done
     ++test_expect_success 'setup superproject with submodule.propagateBranches' '
     ++	git init sub1 &&
     ++	test_commit -C sub1 "sub1" &&
     ++	git init branch-super &&
     ++	git -C branch-super submodule add ../sub1 sub1 &&
     ++	git -C branch-super commit -m "super" &&
     ++
     ++	# Clone into a clean repo that we can cp around
     ++	git clone --recurse-submodules \
     ++		-c submodule.propagateBranches=true \
     ++		branch-super branch-super-clean &&
     ++	git -C branch-super-clean config submodule.propagateBranches true &&
     ++
     ++	# Create an upstream submodule not in the clone
     ++	git init sub2 &&
     ++	test_commit -C sub2 "sub2" &&
     ++	git -C branch-super submodule add ../sub2 sub2 &&
     ++	git -C branch-super commit -m "add sub2"
     ++'
     ++
     ++test_expect_success 'submodule.propagateBranches - detached HEAD' '
     ++	test_when_finished "rm -fr branch-super-cloned" &&
     ++	cp -r branch-super-clean branch-super-cloned &&
     ++
     ++	git -C branch-super-cloned checkout --detach &&
     ++	git -C branch-super-cloned pull origin main &&
     ++	git -C branch-super-cloned submodule update &&
     ++
     ++	# sub2 should be in detached HEAD
     ++	git -C branch-super-cloned/sub2 rev-parse --verify HEAD &&
     ++	test_must_fail git -C branch-super-cloned/sub2 symbolic-ref HEAD
     ++'
     ++
     ++test_expect_success 'submodule.propagateBranches - branch checked out' '
     ++	test_when_finished "rm -fr branch-super-cloned" &&
     ++	cp -r branch-super-clean branch-super-cloned &&
     ++
     ++	git -C branch-super-cloned branch --recurse-submodules new-branch &&
     ++	git -C branch-super-cloned checkout --recurse-submodules new-branch &&
     ++	git -C branch-super-cloned pull origin main &&
     ++	git -C branch-super-cloned submodule update &&
     ++
     ++	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
     ++	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
     ++	HEAD_BRANCH2=$(git -C branch-super-cloned/sub2 symbolic-ref HEAD) &&
     ++	test $HEAD_BRANCH2 = "refs/heads/new-branch"
     ++'
     ++
     ++test_expect_success 'submodule.propagateBranches - other branch checked out' '
     ++	test_when_finished "rm -fr branch-super-cloned" &&
     ++	cp -r branch-super-clean branch-super-cloned &&
     ++
     ++	git -C branch-super-cloned branch --recurse-submodules new-branch &&
     ++	git -C branch-super-cloned checkout --recurse-submodules new-branch &&
     ++	git -C branch-super-cloned/sub1 checkout -b other-branch &&
     ++	git -C branch-super-cloned pull origin main &&
     ++	git -C branch-super-cloned submodule update &&
     ++
     ++	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
     ++	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
     ++	HEAD_BRANCH2=$(git -C branch-super-cloned/sub2 symbolic-ref HEAD) &&
     ++	test $HEAD_BRANCH2 = "refs/heads/new-branch"
      +'
      +
       test_done

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 56+ messages in thread

* [PATCH v2 1/7] clone: teach --detach option
  2022-10-20 20:20 ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
@ 2022-10-20 20:20   ` Glen Choo via GitGitGadget
  2022-10-20 20:20   ` [PATCH v2 2/7] repo-settings: add submodule_propagate_branches Glen Choo via GitGitGadget
                     ` (7 subsequent siblings)
  8 siblings, 0 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-20 20:20 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

Teach "git clone" the "--detach" option, which leaves the cloned repo in
detached HEAD (like "git checkout --detach"). In addition, if the clone
is not bare, do not create the local branch pointed to by the remote's
HEAD symref (bare clones always copy all remote branches directly to
local branches, so the branch is still created in the bare case).

This is especially useful in the "submodule.propagateBranches" workflow,
where local submodule branches are named after the superproject's
branches, so it makes no sense to create a local branch named after the
submodule's remote's branch.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 Documentation/git-clone.txt |  8 +++++++-
 builtin/clone.c             | 12 +++++++++---
 t/t5601-clone.sh            | 22 ++++++++++++++++++++++
 3 files changed, 38 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index d6434d262d6..6a4e5d31b46 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -16,7 +16,7 @@ SYNOPSIS
 	  [--depth <depth>] [--[no-]single-branch] [--no-tags]
 	  [--recurse-submodules[=<pathspec>]] [--[no-]shallow-submodules]
 	  [--[no-]remote-submodules] [--jobs <n>] [--sparse] [--[no-]reject-shallow]
-	  [--filter=<filter> [--also-filter-submodules]] [--] <repository>
+	  [--filter=<filter> [--also-filter-submodules] [--detach]] [--] <repository>
 	  [<directory>]
 
 DESCRIPTION
@@ -210,6 +210,12 @@ objects from the source repository into a pack in the cloned repository.
 	`--branch` can also take tags and detaches the HEAD at that commit
 	in the resulting repository.
 
+--detach::
+	If the cloned repository's HEAD points to a branch, point the newly
+	created HEAD to the branch's commit instead of the branch itself.
+	Additionally, in a non-bare repository, the corresponding local branch
+	will not be created.
+
 -u <upload-pack>::
 --upload-pack <upload-pack>::
 	When given, and the repository to clone from is accessed
diff --git a/builtin/clone.c b/builtin/clone.c
index 547d6464b3c..e624d3f49a2 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -78,6 +78,7 @@ static int option_filter_submodules = -1;    /* unspecified */
 static int config_filter_submodules = -1;    /* unspecified */
 static struct string_list server_options = STRING_LIST_INIT_NODUP;
 static int option_remote_submodules;
+static int option_detach;
 static const char *bundle_uri;
 
 static int recurse_submodules_cb(const struct option *opt,
@@ -162,6 +163,8 @@ static struct option builtin_clone_options[] = {
 		    N_("any cloned submodules will use their remote-tracking branch")),
 	OPT_BOOL(0, "sparse", &option_sparse_checkout,
 		    N_("initialize sparse-checkout file to include only files at root")),
+	OPT_BOOL(0, "detach", &option_detach,
+		 N_("detach HEAD and don't create a local branch")),
 	OPT_STRING(0, "bundle-uri", &bundle_uri,
 		   N_("uri"), N_("a URI for downloading bundles before fetching from origin remote")),
 	OPT_END()
@@ -613,10 +616,12 @@ static void update_remote_refs(const struct ref *refs,
 }
 
 static void update_head(const struct ref *our, const struct ref *remote,
-			const char *unborn, const char *msg)
+			const char *unborn, int should_detach,
+			const char *msg)
 {
 	const char *head;
-	if (our && skip_prefix(our->name, "refs/heads/", &head)) {
+	if (our && !should_detach &&
+	    skip_prefix(our->name, "refs/heads/", &head)) {
 		/* Local default branch link */
 		if (create_symref("HEAD", our->name, NULL) < 0)
 			die(_("unable to update HEAD"));
@@ -1357,7 +1362,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 			   branch_top.buf, reflog_msg.buf, transport,
 			   !is_local);
 
-	update_head(our_head_points_at, remote_head, unborn_head, reflog_msg.buf);
+	update_head(our_head_points_at, remote_head, unborn_head,
+		    option_detach, reflog_msg.buf);
 
 	/*
 	 * We want to show progress for recursive submodule clones iff
diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh
index 45f0803ed4d..418cfd54717 100755
--- a/t/t5601-clone.sh
+++ b/t/t5601-clone.sh
@@ -333,6 +333,28 @@ test_expect_success 'clone checking out a tag' '
 	test_cmp fetch.expected fetch.actual
 '
 
+test_expect_success '--detach detaches and does not create branch' '
+	test_when_finished "rm -fr dst" &&
+	git clone --detach src dst &&
+	(
+		cd dst &&
+		test_must_fail git rev-parse main &&
+		test_must_fail git symbolic-ref HEAD &&
+		test_cmp_rev HEAD refs/remotes/origin/HEAD
+	)
+'
+
+test_expect_success '--detach with --bare detaches but creates branch' '
+	test_when_finished "rm -fr dst" &&
+	git clone --bare --detach src dst &&
+	(
+		cd dst &&
+		git rev-parse main &&
+		test_must_fail git symbolic-ref HEAD &&
+		test_cmp_rev HEAD refs/heads/main
+	)
+'
+
 test_expect_success 'set up ssh wrapper' '
 	cp "$GIT_BUILD_DIR/t/helper/test-fake-ssh$X" \
 		"$TRASH_DIRECTORY/ssh$X" &&
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH v2 2/7] repo-settings: add submodule_propagate_branches
  2022-10-20 20:20 ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
  2022-10-20 20:20   ` [PATCH v2 1/7] clone: teach --detach option Glen Choo via GitGitGadget
@ 2022-10-20 20:20   ` Glen Choo via GitGitGadget
  2022-10-25 18:03     ` Jonathan Tan
  2022-10-20 20:20   ` [PATCH v2 3/7] submodule--helper clone: create named branch Glen Choo via GitGitGadget
                     ` (6 subsequent siblings)
  8 siblings, 1 reply; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-20 20:20 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

When processes recurse into submodules, the child processes have to
use the same value of "submodule.propagateBranches" as the parent
process regardless of whether the process is spawned in the superproject
or submodule, otherwise the behavior may be inconsistent if the
repositories don't agree on the config.

We haven't needed a way to propagate the config because the only command
that reads "submodule.propagateBranches" is "git branch", which only has
one mode of operation with "--recurse-submodules". However, a future
commit will teach "submodule.propagateBranches" to "git submodule
update", making this necessary.

Propagate "submodule.propagateBranches" to child processes by adding a
corresponding GIT_INTERNAL_* environment variable and repository
setting, and setting the environment variable inside
prepare_submodule_repo_env(). Then, refactor builtin/branch.c to read
the repository setting.

Using an internal environment variable is a potentially leaky
abstraction because environment variables can come from sources besides
the parent process. A more robust solution would be to teach Git that
the repository is a submodule and to only read
"submodule.propagateBranches" from the superproject config. There is WIP
for this on the ML [1].

Another alternative would be to pass "-c submodule.propagateBranches" to
all child processes. This is error-prone because many different
processes are invoked directly or indirectly by "git submodule update"
(e.g. "git submodule--helper clone", "git clone", "git checkout"). With
an environment variable, we can avoid this work because
prepare_submodule_repo_env() is already called for submodule child
processes.

[1] https://lore.kernel.org/git/20220310004423.2627181-1-emilyshaffer@google.com/

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/branch.c | 11 +++--------
 cache.h          |  1 +
 repo-settings.c  | 10 ++++++++++
 repository.h     |  1 +
 submodule.c      |  2 ++
 5 files changed, 17 insertions(+), 8 deletions(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index e0e0af43202..8279d4eeb15 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -40,7 +40,6 @@ static const char * const builtin_branch_usage[] = {
 static const char *head;
 static struct object_id head_oid;
 static int recurse_submodules = 0;
-static int submodule_propagate_branches = 0;
 
 static int branch_use_color = -1;
 static char branch_colors[][COLOR_MAXLEN] = {
@@ -106,10 +105,6 @@ static int git_branch_config(const char *var, const char *value, void *cb)
 		recurse_submodules = git_config_bool(var, value);
 		return 0;
 	}
-	if (!strcasecmp(var, "submodule.propagateBranches")) {
-		submodule_propagate_branches = git_config_bool(var, value);
-		return 0;
-	}
 
 	return git_color_default_config(var, value, cb);
 }
@@ -723,7 +718,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, builtin_branch_usage,
 			     0);
-
+	prepare_repo_settings(the_repository);
 	if (!delete && !rename && !copy && !edit_description && !new_upstream &&
 	    !show_current && !unset_upstream && argc == 0)
 		list = 1;
@@ -739,7 +734,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 		usage_with_options(builtin_branch_usage, options);
 
 	if (recurse_submodules_explicit) {
-		if (!submodule_propagate_branches)
+		if (!the_repository->settings.submodule_propagate_branches)
 			die(_("branch with --recurse-submodules can only be used if submodule.propagateBranches is enabled"));
 		if (noncreate_actions)
 			die(_("--recurse-submodules can only be used to create branches"));
@@ -747,7 +742,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 
 	recurse_submodules =
 		(recurse_submodules || recurse_submodules_explicit) &&
-		submodule_propagate_branches;
+		the_repository->settings.submodule_propagate_branches;
 
 	if (filter.abbrev == -1)
 		filter.abbrev = DEFAULT_ABBREV;
diff --git a/cache.h b/cache.h
index 26ed03bd6de..151e1d49e77 100644
--- a/cache.h
+++ b/cache.h
@@ -505,6 +505,7 @@ static inline enum object_type object_type(unsigned int mode)
 #define GIT_WORK_TREE_ENVIRONMENT "GIT_WORK_TREE"
 #define GIT_PREFIX_ENVIRONMENT "GIT_PREFIX"
 #define GIT_SUPER_PREFIX_ENVIRONMENT "GIT_INTERNAL_SUPER_PREFIX"
+#define GIT_SUBMODULE_PROPAGATE_BRANCHES_ENVIRONMENT "GIT_INTERNAL_SUBMODULE_PROPAGATE_BRANCHES"
 #define DEFAULT_GIT_DIR_ENVIRONMENT ".git"
 #define DB_ENVIRONMENT "GIT_OBJECT_DIRECTORY"
 #define INDEX_ENVIRONMENT "GIT_INDEX_FILE"
diff --git a/repo-settings.c b/repo-settings.c
index e8b58151bc4..180df14e45f 100644
--- a/repo-settings.c
+++ b/repo-settings.c
@@ -71,6 +71,16 @@ void prepare_repo_settings(struct repository *r)
 	if (git_env_bool(GIT_TEST_MULTI_PACK_INDEX, 0))
 		r->settings.core_multi_pack_index = 1;
 
+	/*
+	 * If the environment variable is set, assume that it came from the
+	 * superproject and ignore the config.
+	 */
+	r->settings.submodule_propagate_branches =
+		git_env_bool(GIT_SUBMODULE_PROPAGATE_BRANCHES_ENVIRONMENT, -1);
+	if (r->settings.submodule_propagate_branches == -1)
+		repo_cfg_bool(r, "submodule.propagateBranches",
+			      &r->settings.submodule_propagate_branches, 0);
+
 	/*
 	 * Non-boolean config
 	 */
diff --git a/repository.h b/repository.h
index 24316ac944e..3df6fdbdd5c 100644
--- a/repository.h
+++ b/repository.h
@@ -37,6 +37,7 @@ struct repo_settings {
 	int fetch_write_commit_graph;
 	int command_requires_full_index;
 	int sparse_index;
+	int submodule_propagate_branches;
 
 	struct fsmonitor_settings *fsmonitor; /* lazily loaded */
 
diff --git a/submodule.c b/submodule.c
index bf7a2c79183..624404957fa 100644
--- a/submodule.c
+++ b/submodule.c
@@ -503,6 +503,8 @@ static void print_submodule_diff_summary(struct repository *r, struct rev_info *
 
 void prepare_submodule_repo_env(struct strvec *out)
 {
+	if (the_repository->settings.submodule_propagate_branches)
+		strvec_pushf(out, "%s=1", GIT_SUBMODULE_PROPAGATE_BRANCHES_ENVIRONMENT);
 	prepare_other_repo_env(out, DEFAULT_GIT_DIR_ENVIRONMENT);
 }
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH v2 3/7] submodule--helper clone: create named branch
  2022-10-20 20:20 ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
  2022-10-20 20:20   ` [PATCH v2 1/7] clone: teach --detach option Glen Choo via GitGitGadget
  2022-10-20 20:20   ` [PATCH v2 2/7] repo-settings: add submodule_propagate_branches Glen Choo via GitGitGadget
@ 2022-10-20 20:20   ` Glen Choo via GitGitGadget
  2022-10-25 18:00     ` Jonathan Tan
  2022-10-20 20:20   ` [PATCH v2 4/7] t5617: drop references to remote-tracking branches Glen Choo via GitGitGadget
                     ` (5 subsequent siblings)
  8 siblings, 1 reply; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-20 20:20 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

When submodule branching is enabled (i.e. "submodule.propagateBranches =
true"), submodules are expected to have the same set of branches as the
superproject. To support this behavior in newly cloned submodules, teach
"git submodule--helper clone" to:

- clone with the "--detach" flag (so that the submodule doesn't create a
  branch corresponding to the remote's HEAD)
- create a branch when using the --branch and --branch-oid flags

The --branch and --branch-oid flags are only allowed when submodule
branching is enabled, otherwise the named branch might conflict with the
branch from the submodule remote's HEAD.

These flags will be used by `git submodule update` in a later commit.
"git submodule add" (which also invokes "git submodule--helper clone")
should also do something similar when submodule branching is enabled,
but this is left for a later series.

Arguably, "--detach" should also be the default for
"submodule.propagateBranches=false" since it doesn't make sense to
create a submodule branch when the submodule is always expected to be in
detached HEAD. But, to be conservative, this commit does not change the
behavior of "submodule.propagateBranches=false".

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/submodule--helper.c | 33 ++++++++++++++++++++++++++++++++-
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 0b4acb442b2..1ce3458a29c 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1503,6 +1503,8 @@ struct module_clone_data {
 	const char *name;
 	const char *url;
 	const char *depth;
+	const char *branch;
+	const char *branch_oid;
 	struct list_objects_filter_options *filter_options;
 	unsigned int quiet: 1;
 	unsigned int progress: 1;
@@ -1692,6 +1694,8 @@ static int clone_submodule(const struct module_clone_data *clone_data,
 			strvec_push(&cp.args, clone_data->single_branch ?
 				    "--single-branch" :
 				    "--no-single-branch");
+		if (the_repository->settings.submodule_propagate_branches)
+			strvec_push(&cp.args, "--detach");
 
 		strvec_push(&cp.args, "--");
 		strvec_push(&cp.args, clone_data->url);
@@ -1704,6 +1708,21 @@ static int clone_submodule(const struct module_clone_data *clone_data,
 		if(run_command(&cp))
 			die(_("clone of '%s' into submodule path '%s' failed"),
 			    clone_data->url, clone_data_path);
+
+		if (clone_data->branch) {
+			struct child_process branch_cp = CHILD_PROCESS_INIT;
+
+			branch_cp.git_cmd = 1;
+			prepare_other_repo_env(&branch_cp.env, sm_gitdir);
+
+			strvec_pushl(&branch_cp.args, "branch",
+				     clone_data->branch, clone_data->branch_oid,
+				     NULL);
+
+			if (run_command(&branch_cp))
+				die(_("could not create branch '%s' in submodule path '%s'"),
+				    clone_data->branch, clone_data_path);
+		}
 	} else {
 		char *path;
 
@@ -1778,6 +1797,12 @@ static int module_clone(int argc, const char **argv, const char *prefix)
 			   N_("disallow cloning into non-empty directory")),
 		OPT_BOOL(0, "single-branch", &clone_data.single_branch,
 			 N_("clone only one branch, HEAD or --branch")),
+		OPT_STRING(0, "branch", &clone_data.branch,
+			   N_("string"),
+			   N_("name of branch to be created")),
+		OPT_STRING(0, "branch-oid", &clone_data.branch_oid,
+			   N_("object-id"),
+			   N_("commit id for new branch")),
 		OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
 		OPT_END()
 	};
@@ -1785,12 +1810,14 @@ static int module_clone(int argc, const char **argv, const char *prefix)
 		N_("git submodule--helper clone [--prefix=<path>] [--quiet] "
 		   "[--reference <repository>] [--name <name>] [--depth <depth>] "
 		   "[--single-branch] [--filter <filter-spec>] "
+		   "[--branch <branch> --branch-oid <oid>]"
 		   "--url <url> --path <path>"),
 		NULL
 	};
 
 	argc = parse_options(argc, argv, prefix, module_clone_options,
 			     git_submodule_helper_usage, 0);
+	prepare_repo_settings(the_repository);
 
 	clone_data.dissociate = !!dissociate;
 	clone_data.quiet = !!quiet;
@@ -1798,9 +1825,13 @@ static int module_clone(int argc, const char **argv, const char *prefix)
 	clone_data.require_init = !!require_init;
 	clone_data.filter_options = &filter_options;
 
-	if (argc || !clone_data.url || !clone_data.path || !*(clone_data.path))
+	if (argc || !clone_data.url || !clone_data.path || !*(clone_data.path)
+	    || (!!clone_data.branch != !!clone_data.branch_oid))
 		usage_with_options(git_submodule_helper_usage,
 				   module_clone_options);
+	if ((clone_data.branch &&
+	     !the_repository->settings.submodule_propagate_branches))
+		BUG("--branch is only expected with submodule.propagateBranches");
 
 	clone_submodule(&clone_data, &reference);
 	list_objects_filter_release(&filter_options);
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH v2 4/7] t5617: drop references to remote-tracking branches
  2022-10-20 20:20 ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
                     ` (2 preceding siblings ...)
  2022-10-20 20:20   ` [PATCH v2 3/7] submodule--helper clone: create named branch Glen Choo via GitGitGadget
@ 2022-10-20 20:20   ` Glen Choo via GitGitGadget
  2022-10-20 20:20   ` [PATCH v2 5/7] submodule: return target of submodule symref Glen Choo via GitGitGadget
                     ` (4 subsequent siblings)
  8 siblings, 0 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-20 20:20 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

It has included submodule cloning tests without remote-tracking branches
tests since f05da2b48b (clone, submodule: pass partial clone filters to
submodules, 2022-02-04) at least. Rename it accordingly so that we can
put future submodule cloning tests there.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 ...617-clone-submodules-remote.sh => t5617-clone-submodules.sh} | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 rename t/{t5617-clone-submodules-remote.sh => t5617-clone-submodules.sh} (97%)

diff --git a/t/t5617-clone-submodules-remote.sh b/t/t5617-clone-submodules.sh
similarity index 97%
rename from t/t5617-clone-submodules-remote.sh
rename to t/t5617-clone-submodules.sh
index 68843382493..c43a5b26fab 100755
--- a/t/t5617-clone-submodules-remote.sh
+++ b/t/t5617-clone-submodules.sh
@@ -1,6 +1,6 @@
 #!/bin/sh
 
-test_description='Test cloning repos with submodules using remote-tracking branches'
+test_description='Test cloning repos with submodules'
 
 GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
 export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH v2 5/7] submodule: return target of submodule symref
  2022-10-20 20:20 ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
                     ` (3 preceding siblings ...)
  2022-10-20 20:20   ` [PATCH v2 4/7] t5617: drop references to remote-tracking branches Glen Choo via GitGitGadget
@ 2022-10-20 20:20   ` Glen Choo via GitGitGadget
  2022-10-20 20:20   ` [PATCH v2 6/7] submodule update: refactor update targets Glen Choo via GitGitGadget
                     ` (3 subsequent siblings)
  8 siblings, 0 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-20 20:20 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

resolve_gitlink_ref() can tell us which oid the submodule ref is
pointing to, but in a future commit, we would also like to know the
symbolic ref target if we are checking a symbolic ref. Teach
resolve_gitlink_ref() to "return" the symbolic ref's target via an "out"
parameter.

This changes resolve_gitlink_ref()'s signature so that new callers
trying to use the old signature will be stopped by the compiler. If we
returned the target instead (just like refs_resolve_ref_unsafe()), we
would be more consistent with refs_resolve_ref_unsafe(), but callers
expecting the old signature will get the opposite return value from what
they expect (since exit code 0 means success, but NULL pointer means
failure). We should do this refactor once we think that nobody will try
to use the old signature.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/submodule--helper.c |  8 +++++---
 builtin/update-index.c      |  4 ++--
 combine-diff.c              |  3 ++-
 diff-lib.c                  |  2 +-
 dir.c                       |  2 +-
 object-file.c               |  2 +-
 read-cache.c                |  4 ++--
 refs.c                      | 10 ++++++----
 refs.h                      |  5 ++++-
 unpack-trees.c              |  3 ++-
 10 files changed, 26 insertions(+), 17 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 1ce3458a29c..138b133790f 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2531,7 +2531,8 @@ static int update_submodule(struct update_data *update_data)
 
 	if (update_data->just_cloned)
 		oidcpy(&update_data->suboid, null_oid());
-	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD", &update_data->suboid))
+	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
+				     &update_data->suboid, NULL))
 		return die_message(_("Unable to find current revision in submodule path '%s'"),
 				   update_data->displaypath);
 
@@ -2558,7 +2559,8 @@ static int update_submodule(struct update_data *update_data)
 						   update_data->sm_path);
 		}
 
-		if (resolve_gitlink_ref(update_data->sm_path, remote_ref, &update_data->oid))
+		if (resolve_gitlink_ref(update_data->sm_path, remote_ref,
+					&update_data->oid, NULL))
 			return die_message(_("Unable to find %s revision in submodule path '%s'"),
 					   remote_ref, update_data->sm_path);
 
@@ -3303,7 +3305,7 @@ static void die_on_repo_without_commits(const char *path)
 	strbuf_addstr(&sb, path);
 	if (is_nonbare_repository_dir(&sb)) {
 		struct object_id oid;
-		if (resolve_gitlink_ref(path, "HEAD", &oid) < 0)
+		if (resolve_gitlink_ref(path, "HEAD", &oid, NULL) < 0)
 			die(_("'%s' does not have a commit checked out"), path);
 	}
 	strbuf_release(&sb);
diff --git a/builtin/update-index.c b/builtin/update-index.c
index b62249905f1..19a21a4586c 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -339,7 +339,7 @@ static int process_directory(const char *path, int len, struct stat *st)
 		if (S_ISGITLINK(ce->ce_mode)) {
 
 			/* Do nothing to the index if there is no HEAD! */
-			if (resolve_gitlink_ref(path, "HEAD", &oid) < 0)
+			if (resolve_gitlink_ref(path, "HEAD", &oid, NULL) < 0)
 				return 0;
 
 			return add_one_path(ce, path, len, st);
@@ -365,7 +365,7 @@ static int process_directory(const char *path, int len, struct stat *st)
 	}
 
 	/* No match - should we add it as a gitlink? */
-	if (!resolve_gitlink_ref(path, "HEAD", &oid))
+	if (!resolve_gitlink_ref(path, "HEAD", &oid, NULL))
 		return add_one_path(NULL, path, len, st);
 
 	/* Error out. */
diff --git a/combine-diff.c b/combine-diff.c
index b0ece954808..88efcaeefa7 100644
--- a/combine-diff.c
+++ b/combine-diff.c
@@ -1060,7 +1060,8 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,
 			elem->mode = canon_mode(st.st_mode);
 		} else if (S_ISDIR(st.st_mode)) {
 			struct object_id oid;
-			if (resolve_gitlink_ref(elem->path, "HEAD", &oid) < 0)
+			if (resolve_gitlink_ref(elem->path, "HEAD", &oid,
+						NULL) < 0)
 				result = grab_blob(opt->repo, &elem->oid,
 						   elem->mode, &result_size,
 						   NULL, NULL);
diff --git a/diff-lib.c b/diff-lib.c
index 2edea41a234..ac94b6234ca 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -53,7 +53,7 @@ static int check_removed(const struct index_state *istate, const struct cache_en
 		 * a directory --- the blob was removed!
 		 */
 		if (!S_ISGITLINK(ce->ce_mode) &&
-		    resolve_gitlink_ref(ce->name, "HEAD", &sub))
+		    resolve_gitlink_ref(ce->name, "HEAD", &sub, NULL))
 			return 1;
 	}
 	return 0;
diff --git a/dir.c b/dir.c
index d604d1bab98..81d232424c1 100644
--- a/dir.c
+++ b/dir.c
@@ -3251,7 +3251,7 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	struct object_id submodule_head;
 
 	if ((flag & REMOVE_DIR_KEEP_NESTED_GIT) &&
-	    !resolve_gitlink_ref(path->buf, "HEAD", &submodule_head)) {
+	    !resolve_gitlink_ref(path->buf, "HEAD", &submodule_head, NULL)) {
 		/* Do not descend and nuke a nested git work tree. */
 		if (kept_up)
 			*kept_up = 1;
diff --git a/object-file.c b/object-file.c
index 5e309602346..a3c6580c16e 100644
--- a/object-file.c
+++ b/object-file.c
@@ -2522,7 +2522,7 @@ int index_path(struct index_state *istate, struct object_id *oid,
 		strbuf_release(&sb);
 		break;
 	case S_IFDIR:
-		return resolve_gitlink_ref(path, "HEAD", oid);
+		return resolve_gitlink_ref(path, "HEAD", oid, NULL);
 	default:
 		return error(_("%s: unsupported file type"), path);
 	}
diff --git a/read-cache.c b/read-cache.c
index 32024029274..4c1bf33ef48 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -285,7 +285,7 @@ static int ce_compare_gitlink(const struct cache_entry *ce)
 	 *
 	 * If so, we consider it always to match.
 	 */
-	if (resolve_gitlink_ref(ce->name, "HEAD", &oid) < 0)
+	if (resolve_gitlink_ref(ce->name, "HEAD", &oid, NULL) < 0)
 		return 0;
 	return !oideq(&oid, &ce->oid);
 }
@@ -781,7 +781,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,
 
 	namelen = strlen(path);
 	if (S_ISDIR(st_mode)) {
-		if (resolve_gitlink_ref(path, "HEAD", &oid) < 0)
+		if (resolve_gitlink_ref(path, "HEAD", &oid, NULL) < 0)
 			return error(_("'%s' does not have a commit checked out"), path);
 		while (namelen && path[namelen-1] == '/')
 			namelen--;
diff --git a/refs.c b/refs.c
index 1491ae937eb..a32a25ccb69 100644
--- a/refs.c
+++ b/refs.c
@@ -1904,19 +1904,21 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
 }
 
 int resolve_gitlink_ref(const char *submodule, const char *refname,
-			struct object_id *oid)
+			struct object_id *oid, const char **target_out)
 {
 	struct ref_store *refs;
 	int flags;
+	const char *target;
 
 	refs = get_submodule_ref_store(submodule);
 
 	if (!refs)
 		return -1;
-
-	if (!refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags) ||
-	    is_null_oid(oid))
+	target = refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags);
+	if (!target || is_null_oid(oid))
 		return -1;
+	if (target_out)
+		*target_out = target;
 	return 0;
 }
 
diff --git a/refs.h b/refs.h
index 8958717a17d..d5c32fac000 100644
--- a/refs.h
+++ b/refs.h
@@ -137,9 +137,12 @@ int peel_iterated_oid(const struct object_id *base, struct object_id *peeled);
  * submodule (which must be non-NULL). If the resolution is
  * successful, return 0 and set oid to the name of the object;
  * otherwise, return a non-zero value.
+ *
+ * FIXME: Return "target" just like refs_resolve_ref_unsafe(). This will be
+ * safe to do once we merge resolve_gitlink_ref() into master.
  */
 int resolve_gitlink_ref(const char *submodule, const char *refname,
-			struct object_id *oid);
+			struct object_id *oid, const char **target);
 
 /*
  * Return true iff abbrev_name is a possible abbreviation for
diff --git a/unpack-trees.c b/unpack-trees.c
index bae812156c4..db12bfcaffd 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -2288,7 +2288,8 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 
 	if (S_ISGITLINK(ce->ce_mode)) {
 		struct object_id oid;
-		int sub_head = resolve_gitlink_ref(ce->name, "HEAD", &oid);
+		int sub_head =
+			resolve_gitlink_ref(ce->name, "HEAD", &oid, NULL);
 		/*
 		 * If we are not going to update the submodule, then
 		 * we don't care.
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH v2 6/7] submodule update: refactor update targets
  2022-10-20 20:20 ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
                     ` (4 preceding siblings ...)
  2022-10-20 20:20   ` [PATCH v2 5/7] submodule: return target of submodule symref Glen Choo via GitGitGadget
@ 2022-10-20 20:20   ` Glen Choo via GitGitGadget
  2022-10-20 20:20   ` [PATCH v2 7/7] clone, submodule update: create and check out branches Glen Choo via GitGitGadget
                     ` (2 subsequent siblings)
  8 siblings, 0 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-20 20:20 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

Refactor two "git submodule update" code locations so that they no
longer refer to oids directly. This shrinks the next commit's diff,
where this code will need to handle branches.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/submodule--helper.c | 24 +++++++++++++-----------
 1 file changed, 13 insertions(+), 11 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 138b133790f..990adeb2e19 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2253,7 +2253,7 @@ static int fetch_in_submodule(const char *module_path, int depth, int quiet,
 static int run_update_command(const struct update_data *ud, int subforce)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
-	char *oid = oid_to_hex(&ud->oid);
+	const char *update_target = oid_to_hex(&ud->oid);;
 	int ret;
 
 	switch (ud->update_strategy.type) {
@@ -2283,7 +2283,7 @@ static int run_update_command(const struct update_data *ud, int subforce)
 		BUG("unexpected update strategy type: %d",
 		    ud->update_strategy.type);
 	}
-	strvec_push(&cp.args, oid);
+	strvec_push(&cp.args, update_target);
 
 	cp.dir = ud->sm_path;
 	prepare_submodule_repo_env(&cp.env);
@@ -2291,20 +2291,20 @@ static int run_update_command(const struct update_data *ud, int subforce)
 		switch (ud->update_strategy.type) {
 		case SM_UPDATE_CHECKOUT:
 			die_message(_("Unable to checkout '%s' in submodule path '%s'"),
-				    oid, ud->displaypath);
+				    update_target, ud->displaypath);
 			/* No "ret" assignment, use "git checkout"'s */
 			break;
 		case SM_UPDATE_REBASE:
 			ret = die_message(_("Unable to rebase '%s' in submodule path '%s'"),
-					  oid, ud->displaypath);
+					  update_target, ud->displaypath);
 			break;
 		case SM_UPDATE_MERGE:
 			ret = die_message(_("Unable to merge '%s' in submodule path '%s'"),
-					  oid, ud->displaypath);
+					  update_target, ud->displaypath);
 			break;
 		case SM_UPDATE_COMMAND:
 			ret = die_message(_("Execution of '%s %s' failed in submodule path '%s'"),
-					  ud->update_strategy.command, oid, ud->displaypath);
+					  ud->update_strategy.command, update_target, ud->displaypath);
 			break;
 		default:
 			BUG("unexpected update strategy type: %d",
@@ -2320,19 +2320,19 @@ static int run_update_command(const struct update_data *ud, int subforce)
 	switch (ud->update_strategy.type) {
 	case SM_UPDATE_CHECKOUT:
 		printf(_("Submodule path '%s': checked out '%s'\n"),
-		       ud->displaypath, oid);
+		       ud->displaypath, update_target);
 		break;
 	case SM_UPDATE_REBASE:
 		printf(_("Submodule path '%s': rebased into '%s'\n"),
-		       ud->displaypath, oid);
+		       ud->displaypath, update_target);
 		break;
 	case SM_UPDATE_MERGE:
 		printf(_("Submodule path '%s': merged in '%s'\n"),
-		       ud->displaypath, oid);
+		       ud->displaypath, update_target);
 		break;
 	case SM_UPDATE_COMMAND:
 		printf(_("Submodule path '%s': '%s %s'\n"),
-		       ud->displaypath, ud->update_strategy.command, oid);
+		       ud->displaypath, ud->update_strategy.command, update_target);
 		break;
 	default:
 		BUG("unexpected update strategy type: %d",
@@ -2519,6 +2519,7 @@ static void update_data_to_args(const struct update_data *update_data,
 
 static int update_submodule(struct update_data *update_data)
 {
+	int submodule_up_to_date;
 	int ret;
 
 	ret = determine_submodule_update_strategy(the_repository,
@@ -2567,7 +2568,8 @@ static int update_submodule(struct update_data *update_data)
 		free(remote_ref);
 	}
 
-	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force) {
+	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
+	if (!submodule_up_to_date || update_data->force) {
 		ret = run_update_procedure(update_data);
 		if (ret)
 			return ret;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH v2 7/7] clone, submodule update: create and check out branches
  2022-10-20 20:20 ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
                     ` (5 preceding siblings ...)
  2022-10-20 20:20   ` [PATCH v2 6/7] submodule update: refactor update targets Glen Choo via GitGitGadget
@ 2022-10-20 20:20   ` Glen Choo via GitGitGadget
  2022-10-25 17:56     ` Jonathan Tan
  2022-10-20 22:40   ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Junio C Hamano
  2022-10-28 20:14   ` [PATCH v3 0/8] " Glen Choo via GitGitGadget
  8 siblings, 1 reply; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-20 20:20 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

Teach "git submodule update" to:

- create the branch with the same name as the current superproject
  branch when cloning a submodule
- check out that branch (instead of the commit OID) when updating
  the submodule worktree

when submodule branching is enabled (submodule.propagateBranches = true)
on the superproject and a branch is checked out. "git clone
--recurse-submodules" also learns this trick because it is implemented
with "git submodule update --recursive".

This approach of checking out the branch will not result in a dirty
worktree for freshly cloned submodules because we can ensure that the
submodule branch points to the superproject gitlink. In other cases, it
does not work as well, but we can handle them incrementally:

- "git pull --recurse-submodules" merges the superproject tree (without
  updating the submodule branches), and runs "git submodule update" to
  update the worktrees, so it is almost guaranteed to result in a dirty
  worktree.

  The implementation of "git pull --recurse-submodules" is likely to
  change drastically as submodule.propagateBranches work progresses
  (e.g. "git merge" learns to recurse in to submodules), and we may be
  able to replace the "git submodule update" invocation, or teach it new
  tricks that make the update behave well.

- The user might make changes to the submodule branch without committing
  them back to superproject. This is primarily affects "git checkout
  --recurse-submodules", since that is the primary way of switching away
  from a branch and leaving behind WIP (as opposed to "git submodule
  update", which is run post-checkout).

  In a future series, "git checkout --recurse-submodules" will learn to
  consider submodule branches. We can introduce appropriate guardrails
  then, e.g. requiring that the superproject working tree is not dirty
  before switching away.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/submodule--helper.c | 32 +++++++++++++++++--
 t/t5617-clone-submodules.sh | 34 ++++++++++++++++++++
 t/t7406-submodule-update.sh | 64 +++++++++++++++++++++++++++++++++++++
 3 files changed, 127 insertions(+), 3 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 990adeb2e19..4576ba22544 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1915,6 +1915,7 @@ static void submodule_update_clone_release(struct submodule_update_clone *suc)
 struct update_data {
 	const char *prefix;
 	char *displaypath;
+	const char *super_branch;
 	enum submodule_update_type update_default;
 	struct object_id suboid;
 	struct string_list references;
@@ -2090,6 +2091,10 @@ static int prepare_to_clone_next_submodule(const struct cache_entry *ce,
 		strvec_push(&child->args, suc->update_data->single_branch ?
 					      "--single-branch" :
 					      "--no-single-branch");
+	if (ud->super_branch) {
+		strvec_pushf(&child->args, "--branch=%s", ud->super_branch);
+		strvec_pushf(&child->args, "--branch-oid=%s", oid_to_hex(&ce->oid));
+	}
 
 cleanup:
 	free(displaypath);
@@ -2253,9 +2258,15 @@ static int fetch_in_submodule(const char *module_path, int depth, int quiet,
 static int run_update_command(const struct update_data *ud, int subforce)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *update_target = oid_to_hex(&ud->oid);;
+	const char *update_target;
 	int ret;
 
+	if (ud->update_strategy.type == SM_UPDATE_CHECKOUT &&
+	    ud->super_branch)
+		update_target = ud->super_branch;
+	else
+		update_target = oid_to_hex(&ud->oid);
+
 	switch (ud->update_strategy.type) {
 	case SM_UPDATE_CHECKOUT:
 		cp.git_cmd = 1;
@@ -2521,6 +2532,7 @@ static int update_submodule(struct update_data *update_data)
 {
 	int submodule_up_to_date;
 	int ret;
+	const char *submodule_head = "HEAD";
 
 	ret = determine_submodule_update_strategy(the_repository,
 						  update_data->just_cloned,
@@ -2533,7 +2545,7 @@ static int update_submodule(struct update_data *update_data)
 	if (update_data->just_cloned)
 		oidcpy(&update_data->suboid, null_oid());
 	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
-				     &update_data->suboid, NULL))
+				     &update_data->suboid, &submodule_head))
 		return die_message(_("Unable to find current revision in submodule path '%s'"),
 				   update_data->displaypath);
 
@@ -2568,7 +2580,14 @@ static int update_submodule(struct update_data *update_data)
 		free(remote_ref);
 	}
 
-	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
+	if (!update_data->super_branch)
+		submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
+	else if (skip_prefix(submodule_head, "refs/heads/", &submodule_head))
+		submodule_up_to_date = !strcmp(update_data->super_branch, submodule_head);
+	/* submodule_branch is "HEAD"; the submodule is in detached HEAD */
+	else
+		submodule_up_to_date = 0;
+
 	if (!submodule_up_to_date || update_data->force) {
 		ret = run_update_procedure(update_data);
 		if (ret)
@@ -2603,6 +2622,12 @@ static int update_submodules(struct update_data *update_data)
 	int i, ret = 0;
 	struct submodule_update_clone suc = SUBMODULE_UPDATE_CLONE_INIT;
 
+	if (the_repository->settings.submodule_propagate_branches) {
+		struct branch *current_branch = branch_get(NULL);
+		if (current_branch)
+			update_data->super_branch = current_branch->name;
+	}
+
 	suc.update_data = update_data;
 	run_processes_parallel_tr2(suc.update_data->max_jobs, update_clone_get_next_task,
 				   update_clone_start_failure,
@@ -2718,6 +2743,7 @@ static int module_update(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, module_update_options,
 			     git_submodule_helper_usage, 0);
+	prepare_repo_settings(the_repository);
 
 	if (opt.require_init)
 		opt.init = 1;
diff --git a/t/t5617-clone-submodules.sh b/t/t5617-clone-submodules.sh
index c43a5b26fab..51593376ce4 100755
--- a/t/t5617-clone-submodules.sh
+++ b/t/t5617-clone-submodules.sh
@@ -13,10 +13,17 @@ test_expect_success 'setup' '
 	git config --global protocol.file.allow always &&
 	git checkout -b main &&
 	test_commit commit1 &&
+	mkdir subsub &&
+	(
+		cd subsub &&
+		git init &&
+		test_commit subsubcommit1
+	) &&
 	mkdir sub &&
 	(
 		cd sub &&
 		git init &&
+		git submodule add "file://$pwd/subsub" subsub &&
 		test_commit subcommit1 &&
 		git tag sub_when_added_to_super &&
 		git branch other
@@ -107,4 +114,31 @@ test_expect_success '--no-also-filter-submodules overrides clone.filterSubmodule
 	test_cmp_config -C super_clone3/sub false --default false remote.origin.promisor
 '
 
+test_expect_success 'submodule.propagateBranches checks out branches at correct commits' '
+	git -C sub checkout -b not-main &&
+	git -C subsub checkout -b not-main &&
+	git clone --recurse-submodules \
+		-c submodule.propagateBranches=true \
+		"file://$pwd/." super_clone4 &&
+
+	# Assert that each repo is pointing to "main"
+	for REPO in "super_clone4" "super_clone4/sub" "super_clone4/sub/subsub"
+	do
+	    HEAD_BRANCH=$(git -C $REPO symbolic-ref HEAD) &&
+	    test $HEAD_BRANCH = "refs/heads/main" || return 1
+	done &&
+
+	# Assert that the submodule branches are pointing to the right revs
+	EXPECT_SUB_OID="$(git -C super_clone4 rev-parse :sub)" &&
+	ACTUAL_SUB_OID="$(git -C super_clone4/sub rev-parse refs/heads/main)" &&
+	test $EXPECT_SUB_OID = $ACTUAL_SUB_OID &&
+	EXPECT_SUBSUB_OID="$(git -C super_clone4/sub rev-parse :subsub)" &&
+	ACTUAL_SUBSUB_OID="$(git -C super_clone4/sub/subsub rev-parse refs/heads/main)" &&
+	test $EXPECT_SUBSUB_OID = $ACTUAL_SUBSUB_OID &&
+
+	# Assert that the submodules do not have branches from their upstream
+	test_must_fail git -C super_clone4/sub rev-parse not-main &&
+	test_must_fail git -C super_clone4/sub/subsub rev-parse not-main
+'
+
 test_done
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index f094e3d7f36..54aa8c5cb54 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -1179,4 +1179,68 @@ test_expect_success 'submodule update --recursive skip submodules with strategy=
 	test_cmp expect.err actual.err
 '
 
+test_expect_success 'setup superproject with submodule.propagateBranches' '
+	git init sub1 &&
+	test_commit -C sub1 "sub1" &&
+	git init branch-super &&
+	git -C branch-super submodule add ../sub1 sub1 &&
+	git -C branch-super commit -m "super" &&
+
+	# Clone into a clean repo that we can cp around
+	git clone --recurse-submodules \
+		-c submodule.propagateBranches=true \
+		branch-super branch-super-clean &&
+	git -C branch-super-clean config submodule.propagateBranches true &&
+
+	# Create an upstream submodule not in the clone
+	git init sub2 &&
+	test_commit -C sub2 "sub2" &&
+	git -C branch-super submodule add ../sub2 sub2 &&
+	git -C branch-super commit -m "add sub2"
+'
+
+test_expect_success 'submodule.propagateBranches - detached HEAD' '
+	test_when_finished "rm -fr branch-super-cloned" &&
+	cp -r branch-super-clean branch-super-cloned &&
+
+	git -C branch-super-cloned checkout --detach &&
+	git -C branch-super-cloned pull origin main &&
+	git -C branch-super-cloned submodule update &&
+
+	# sub2 should be in detached HEAD
+	git -C branch-super-cloned/sub2 rev-parse --verify HEAD &&
+	test_must_fail git -C branch-super-cloned/sub2 symbolic-ref HEAD
+'
+
+test_expect_success 'submodule.propagateBranches - branch checked out' '
+	test_when_finished "rm -fr branch-super-cloned" &&
+	cp -r branch-super-clean branch-super-cloned &&
+
+	git -C branch-super-cloned branch --recurse-submodules new-branch &&
+	git -C branch-super-cloned checkout --recurse-submodules new-branch &&
+	git -C branch-super-cloned pull origin main &&
+	git -C branch-super-cloned submodule update &&
+
+	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
+	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
+	HEAD_BRANCH2=$(git -C branch-super-cloned/sub2 symbolic-ref HEAD) &&
+	test $HEAD_BRANCH2 = "refs/heads/new-branch"
+'
+
+test_expect_success 'submodule.propagateBranches - other branch checked out' '
+	test_when_finished "rm -fr branch-super-cloned" &&
+	cp -r branch-super-clean branch-super-cloned &&
+
+	git -C branch-super-cloned branch --recurse-submodules new-branch &&
+	git -C branch-super-cloned checkout --recurse-submodules new-branch &&
+	git -C branch-super-cloned/sub1 checkout -b other-branch &&
+	git -C branch-super-cloned pull origin main &&
+	git -C branch-super-cloned submodule update &&
+
+	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
+	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
+	HEAD_BRANCH2=$(git -C branch-super-cloned/sub2 symbolic-ref HEAD) &&
+	test $HEAD_BRANCH2 = "refs/heads/new-branch"
+'
+
 test_done
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 56+ messages in thread

* Re: [PATCH v2 0/7] clone, submodule update: check out submodule branches
  2022-10-20 20:20 ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
                     ` (6 preceding siblings ...)
  2022-10-20 20:20   ` [PATCH v2 7/7] clone, submodule update: create and check out branches Glen Choo via GitGitGadget
@ 2022-10-20 22:40   ` Junio C Hamano
  2022-10-20 23:53     ` Glen Choo
  2022-10-28 20:14   ` [PATCH v3 0/8] " Glen Choo via GitGitGadget
  8 siblings, 1 reply; 56+ messages in thread
From: Junio C Hamano @ 2022-10-20 22:40 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget; +Cc: git, Philippe Blain, Jonathan Tan, Glen Choo

"Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:

> During the v1 discussion, I realize that my idea of the new submodule UX has
> already diverged from what was initially communicated to the list. I plan to
> check in a technical document describing the plans for new submodule UX,
> which should hopefully make these discussions smoother (e.g. the commit
> message in patch 7 can make reference to the doc).

Good.

> = Description
>
> This series teaches "git clone --recurse-submodules" and "git submodule
> update" to understand "submodule.propagateBranches" (see Further Reading for
> context), i.e. if the superproject has a branch checked out and a submodule
> is cloned, the submodule will have the same branch checked out.
>
> To do this, "git submodule update" checks if ...
> = Series history
>
> Changes in v2:
>
>  * The superproject's "submodule.propagateBranches" value is always used,
>    even if false.
>  * Branches are now created at clone time (by adding a new flag to "git
>    submodule clone"), instead of at update time.
>  * Rebase onto newer master. This got adjusted slightly to incorporate
>    ab/submodule-helper-leakfix.
>  * Add more tests to demonstrate edge case behavior.
>  * Assorted commit message and doc improvements.

As the previous round was more than a month old and is clearly not a
bugfix but is adding a new feature, I do not mind updating to the
newer base after a new feature release was made.  There isn't much
to be gained, other than that we can easily sanity check by running
"git diff @{1} @{0}" on the branch to compare the iterations, by
keeping the same base.  We are not going to merge this topic down to
maintenance tracks after it graduates to 'master' anyway.

But I got curious and tried to adjust these patches back on the
previous base 07ee72db (Sync with 'maint', 2022-08-26).  It turns
out that the conflicts needed to be resolved were fairly trivial.

Merging the topic that was recreated on top of the same old base
into today's 'master' of course needed the same conflict resolution
but that is something we've been doing every time we rebuild 'seen'
(read: at least twice a day, but often more).  Applying these
patches directly on today's 'master' of course produced the
identical tree as the tree of this trial merge.

> = Future work
>
>  * Patch 5, which refactors resolve_gitlink_ref(), notes that a better
>    interface would be to return the refname instead of using an "out"
>    parameter, but we use an "out" parameter so that any new callers trying
>    to use the old function signature will get stopped by the compiler. The
>    refactor can be finished at a later time.

OK.

>  * Patch 5 uses the name "target" when we are talking about what a symref
>    points to, instead of "referent" like the other functions. "target" is
>    the better choice, since "referent" could also apply to non-symbolic
>    refs, but that cleanup is quite big.

I do not see a huge difference between the two.  "target" can be
used in contexts that are not about symbolic refs, and "referent"
can be used in contexts that are not about symbolic refs, too.  As
long as they are unified in one way or another, it would be fine.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v2 0/7] clone, submodule update: check out submodule branches
  2022-10-20 22:40   ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Junio C Hamano
@ 2022-10-20 23:53     ` Glen Choo
  2022-10-21  0:01       ` Junio C Hamano
  0 siblings, 1 reply; 56+ messages in thread
From: Glen Choo @ 2022-10-20 23:53 UTC (permalink / raw)
  To: Junio C Hamano, Glen Choo via GitGitGadget
  Cc: git, Philippe Blain, Jonathan Tan

Junio C Hamano <gitster@pobox.com> writes:

> "Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:
>> = Description
>>
>> This series teaches "git clone --recurse-submodules" and "git submodule
>> update" to understand "submodule.propagateBranches" (see Further Reading for
>> context), i.e. if the superproject has a branch checked out and a submodule
>> is cloned, the submodule will have the same branch checked out.
>>
>> To do this, "git submodule update" checks if ...
>> = Series history
>>
>> Changes in v2:
>>
>>  * The superproject's "submodule.propagateBranches" value is always used,
>>    even if false.
>>  * Branches are now created at clone time (by adding a new flag to "git
>>    submodule clone"), instead of at update time.
>>  * Rebase onto newer master. This got adjusted slightly to incorporate
>>    ab/submodule-helper-leakfix.
>>  * Add more tests to demonstrate edge case behavior.
>>  * Assorted commit message and doc improvements.
>
> As the previous round was more than a month old and is clearly not a
> bugfix but is adding a new feature, I do not mind updating to the
> newer base after a new feature release was made.  There isn't much
> to be gained, other than that we can easily sanity check by running
> "git diff @{1} @{0}" on the branch to compare the iterations, by
> keeping the same base.  We are not going to merge this topic down to
> maintenance tracks after it graduates to 'master' anyway.
>
> But I got curious and tried to adjust these patches back on the
> previous base 07ee72db (Sync with 'maint', 2022-08-26).  It turns
> out that the conflicts needed to be resolved were fairly trivial.
>
> Merging the topic that was recreated on top of the same old base
> into today's 'master' of course needed the same conflict resolution
> but that is something we've been doing every time we rebuild 'seen'
> (read: at least twice a day, but often more).  Applying these
> patches directly on today's 'master' of course produced the
> identical tree as the tree of this trial merge.

Thanks for your patience. For future reference, do you have a preference
either way? I suppose choosing a later base might make it easier for
reviewers who don't have the bandwidth to remember what "master" used to
look like, but it's just churn to you, since you're already rebuilding
"seen".

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v2 0/7] clone, submodule update: check out submodule branches
  2022-10-20 23:53     ` Glen Choo
@ 2022-10-21  0:01       ` Junio C Hamano
  0 siblings, 0 replies; 56+ messages in thread
From: Junio C Hamano @ 2022-10-21  0:01 UTC (permalink / raw)
  To: Glen Choo; +Cc: Glen Choo via GitGitGadget, git, Philippe Blain, Jonathan Tan

Glen Choo <chooglen@google.com> writes:

> Thanks for your patience. For future reference, do you have a preference
> either way? I suppose choosing a later base might make it easier for
> reviewers who don't have the bandwidth to remember what "master" used to
> look like, ...

That cuts both ways.  For brand-new reviewers who starts from v2
without ever seeing v1, and when the two iterations are far apart in
time, it may be true.

But reviewers who helped you with earlier rounds hopefully know what
they saw and commented on, and keeping the same base would help them
to see what is different in the updated iteration, without having to
see distracting changes in the surrounding area brought in by using
the newer base.

> but it's just churn to you, since you're already rebuilding
> "seen".

To me, it does not make too much of a difference (unless it is
clearly a fix for a grave issue that should eventually merge down to
older maintenance tracks, and this one is clearly not).  If you
rebase, I would double check your rebase by rebasing the new
interation back to the old base and then merging the new base in to
see how the result compares, like I just did this time, before
replacing the topic with the application of patches on the updated
base, so it is a one-time extra cost to me, but other than that,
what I do would not change all that much and it hopefully will make
it easier to queue later iterations.

Thanks.


^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v2 7/7] clone, submodule update: create and check out branches
  2022-10-20 20:20   ` [PATCH v2 7/7] clone, submodule update: create and check out branches Glen Choo via GitGitGadget
@ 2022-10-25 17:56     ` Jonathan Tan
  2022-10-25 21:49       ` Glen Choo
  0 siblings, 1 reply; 56+ messages in thread
From: Jonathan Tan @ 2022-10-25 17:56 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget; +Cc: Jonathan Tan, git, Philippe Blain, Glen Choo

"Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:
> Teach "git submodule update" to:
> 
> - create the branch with the same name as the current superproject
>   branch when cloning a submodule

Ah, this is when "git submodule update" needs to clone a submodule (as 
opposed to doing something with a submodule that is already cloned). 

> - check out that branch (instead of the commit OID) when updating
>   the submodule worktree

So whenever we run "git submodule update" and a submodule already has a 
branch of the same name as what's currently checked out in the 
superproject, we check out the submodule's branch, ignoring the gitlink 
in the superproject? 

> when submodule branching is enabled (submodule.propagateBranches = true)
> on the superproject and a branch is checked out. 

OK.

> "git clone
> --recurse-submodules" also learns this trick because it is implemented
> with "git submodule update --recursive".

Is this sentence redundant now that you're specifically calling out the 
cloning part above?

> This approach of checking out the branch will not result in a dirty
> worktree for freshly cloned submodules because we can ensure that the
> submodule branch points to the superproject gitlink. 

Makes sense.

> In other cases, it
> does not work as well, but we can handle them incrementally:
> 
> - "git pull --recurse-submodules" merges the superproject tree (without
>   updating the submodule branches), and runs "git submodule update" to
>   update the worktrees, so it is almost guaranteed to result in a dirty
>   worktree.

Is this because when we "git pull", the superproject likely now has a 
different gitlink, but the branch in the submodule hasn't changed? 

>   The implementation of "git pull --recurse-submodules" is likely to
>   change drastically as submodule.propagateBranches work progresses
>   (e.g. "git merge" learns to recurse in to submodules), and we may be
>   able to replace the "git submodule update" invocation, or teach it new
>   tricks that make the update behave well.
> 
> - The user might make changes to the submodule branch without committing
>   them back to superproject. This is primarily affects "git checkout
>   --recurse-submodules", since that is the primary way of switching away
>   from a branch and leaving behind WIP (as opposed to "git submodule
>   update", which is run post-checkout).

Makes sense - so in summary, there are (at least) two ways of the 
superproject's gitlink and the submodule's branch can go out of sync: 
the user changing the submodule branch (here) or the user changing the 
superproject's gitlink (above). 
 
>   In a future series, "git checkout --recurse-submodules" will learn to
>   consider submodule branches. We can introduce appropriate guardrails
>   then, e.g. requiring that the superproject working tree is not dirty
>   before switching away.
> 
> Signed-off-by: Glen Choo <chooglen@google.com>

[snip]

> @@ -2521,6 +2532,7 @@ static int update_submodule(struct update_data *update_data)
>  {
>  	int submodule_up_to_date;
>  	int ret;
> +	const char *submodule_head = "HEAD";

I think it's clearer if this is initialized to NULL. I don't think the 
submodule head is detached in all code paths when this is HEAD.  

>  	ret = determine_submodule_update_strategy(the_repository,
>  						  update_data->just_cloned,
> @@ -2533,7 +2545,7 @@ static int update_submodule(struct update_data *update_data)
>  	if (update_data->just_cloned)
>  		oidcpy(&update_data->suboid, null_oid());
>  	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
> -				     &update_data->suboid, NULL))
> +				     &update_data->suboid, &submodule_head))
>  		return die_message(_("Unable to find current revision in submodule path '%s'"),
>  				   update_data->displaypath);
>  
> @@ -2568,7 +2580,14 @@ static int update_submodule(struct update_data *update_data)
>  		free(remote_ref);
>  	}
>  
> -	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
> +	if (!update_data->super_branch)
> +		submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
> +	else if (skip_prefix(submodule_head, "refs/heads/", &submodule_head))
> +		submodule_up_to_date = !strcmp(update_data->super_branch, submodule_head);
> +	/* submodule_branch is "HEAD"; the submodule is in detached HEAD */
> +	else
> +		submodule_up_to_date = 0;

I think there needs to be a better comment here. It doesn't matter that 
the submodule is in detached HEAD; what matters is that the submodule 
doesn't have the superproject's branch checked out. So maybe something 
like: 

  if (update_data->super_branch) {
    /* (format this appropriately) We also need to check that the 
       submodule's HEAD points to super_branch. */ 
    const char *submodule_head;
    submodule_up_to_date = skip_prefix(...) && !strcmp(...)
  } else {
    submodule_up_to_date = oideq(...)
  }

> diff --git a/t/t5617-clone-submodules.sh b/t/t5617-clone-submodules.sh

[snip]

> @@ -107,4 +114,31 @@ test_expect_success '--no-also-filter-submodules overrides clone.filterSubmodule
>  	test_cmp_config -C super_clone3/sub false --default false remote.origin.promisor
>  '
>  
> +test_expect_success 'submodule.propagateBranches checks out branches at correct commits' '
> +	git -C sub checkout -b not-main &&
> +	git -C subsub checkout -b not-main &&
> +	git clone --recurse-submodules \
> +		-c submodule.propagateBranches=true \
> +		"file://$pwd/." super_clone4 &&
> +
> +	# Assert that each repo is pointing to "main"
> +	for REPO in "super_clone4" "super_clone4/sub" "super_clone4/sub/subsub"
> +	do
> +	    HEAD_BRANCH=$(git -C $REPO symbolic-ref HEAD) &&
> +	    test $HEAD_BRANCH = "refs/heads/main" || return 1
> +	done &&

As I said in my earlier review [1], could we use a branch name that is 
not  "main"? That way, we also check that the clone *creates* the 
branches,  not just reuses something already there. 

[1] https://lore.kernel.org/git/20220901200047.515294-1-jonathantanmy@google.com/

> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh

[snip]

> +test_expect_success 'submodule.propagateBranches - detached HEAD' '
> +	test_when_finished "rm -fr branch-super-cloned" &&
> +	cp -r branch-super-clean branch-super-cloned &&
> +
> +	git -C branch-super-cloned checkout --detach &&
> +	git -C branch-super-cloned pull origin main &&

If the behavior of "pull" is going to change soon (as stated in the 
commit message), can we avoid using it in tests here? 

> +	git -C branch-super-cloned submodule update &&
> +
> +	# sub2 should be in detached HEAD
> +	git -C branch-super-cloned/sub2 rev-parse --verify HEAD &&
> +	test_must_fail git -C branch-super-cloned/sub2 symbolic-ref HEAD
> +'
> +
> +test_expect_success 'submodule.propagateBranches - branch checked out' '
> +	test_when_finished "rm -fr branch-super-cloned" &&
> +	cp -r branch-super-clean branch-super-cloned &&
> +
> +	git -C branch-super-cloned branch --recurse-submodules new-branch &&
> +	git -C branch-super-cloned checkout --recurse-submodules new-branch &&
> +	git -C branch-super-cloned pull origin main &&
> +	git -C branch-super-cloned submodule update &&
> +
> +	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
> +	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
> +	HEAD_BRANCH2=$(git -C branch-super-cloned/sub2 symbolic-ref HEAD) &&
> +	test $HEAD_BRANCH2 = "refs/heads/new-branch"

I'm not sure of the behavior of "pull" here, so I didn't look too  
closely into what these tests test, but I think that you should cover 
these situations: 
 - submodule has correct OID and correct branch checked out
 - submodule has correct OID, but wrong branch checked out, and correct 
   branch doesn't exist 
 - submodule has correct OID, but wrong branch checked out, and correct 
   branch exists at correct OID 
 - submodule has correct OID, but wrong branch checked out, and correct 
   branch exists at wrong OID 
 - submodule has incorrect OID but correct branch checked out (if this 
   currently doesn't work, maybe add a test_must_fail with a NEEDSWORK) 

There are other combinations but I think these are the most important 
ones. 

Also, if there is a situation where you ignore the superproject's 
gitlink in favor of the submodule's branch, you should check the OIDs 
in all cases too, not just what the HEAD symbolic ref points to. 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v2 3/7] submodule--helper clone: create named branch
  2022-10-20 20:20   ` [PATCH v2 3/7] submodule--helper clone: create named branch Glen Choo via GitGitGadget
@ 2022-10-25 18:00     ` Jonathan Tan
  0 siblings, 0 replies; 56+ messages in thread
From: Jonathan Tan @ 2022-10-25 18:00 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget; +Cc: Jonathan Tan, git, Philippe Blain, Glen Choo

"Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:
> Signed-off-by: Glen Choo <chooglen@google.com>

Add a note in the commit message that this will be used and tested in a 
subsequent patch. 

> -	if (argc || !clone_data.url || !clone_data.path || !*(clone_data.path))
> +	if (argc || !clone_data.url || !clone_data.path || !*(clone_data.path)
> +	    || (!!clone_data.branch != !!clone_data.branch_oid))
>  		usage_with_options(git_submodule_helper_usage,
>  				   module_clone_options);

I know that this is just internal code, but could we have a better 
diagnostic? You can leave the existing check alone, and then do the 
!!clone_data.branch != !!clone_data.branch_oid with a BUG() if the 
result is not what you expect. 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v2 2/7] repo-settings: add submodule_propagate_branches
  2022-10-20 20:20   ` [PATCH v2 2/7] repo-settings: add submodule_propagate_branches Glen Choo via GitGitGadget
@ 2022-10-25 18:03     ` Jonathan Tan
  0 siblings, 0 replies; 56+ messages in thread
From: Jonathan Tan @ 2022-10-25 18:03 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget; +Cc: Jonathan Tan, git, Philippe Blain, Glen Choo

"Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:
> +	/*
> +	 * If the environment variable is set, assume that it came from the
> +	 * superproject and ignore the config.
> +	 */

Might be clearer to say:

  If the environment variable is set, assume that it was set by an 
  invocation of git running in a superproject with 
  submodule.propagateBranches set and that is recursing into this repo as 
  a submodule. Therefore, we should ignore whatever is set in this  
  repo's config. 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v2 7/7] clone, submodule update: create and check out branches
  2022-10-25 17:56     ` Jonathan Tan
@ 2022-10-25 21:49       ` Glen Choo
  0 siblings, 0 replies; 56+ messages in thread
From: Glen Choo @ 2022-10-25 21:49 UTC (permalink / raw)
  To: Jonathan Tan, Glen Choo via GitGitGadget
  Cc: Jonathan Tan, git, Philippe Blain

Jonathan Tan <jonathantanmy@google.com> writes:

> "Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:
>> Teach "git submodule update" to:
>> 
>> - create the branch with the same name as the current superproject
>>   branch when cloning a submodule

[...]
>> - check out that branch (instead of the commit OID) when updating
>>   the submodule worktree

[...]
>> when submodule branching is enabled (submodule.propagateBranches = true)
>> on the superproject and a branch is checked out. 
>> "git clone
>> --recurse-submodules" also learns this trick because it is implemented
>> with "git submodule update --recursive".
>
> Is this sentence redundant now that you're specifically calling out the 
> cloning part above?

Not quite. The "trick" I'm referring to is "clone, create the
named branch and check it out", which is implemented by "git submodule
update" (and its constituent subprocesses, e.g. "git submodule clone"),
so it is pertinent that "git clone" is implemented by "git submodule
update".

>> In other cases, it
>> does not work as well, but we can handle them incrementally:
>> 
>> - "git pull --recurse-submodules" merges the superproject tree (without
>>   updating the submodule branches), and runs "git submodule update" to
>>   update the worktrees, so it is almost guaranteed to result in a dirty
>>   worktree.
>
> Is this because when we "git pull", the superproject likely now has a 
> different gitlink, but the branch in the submodule hasn't changed? 

Yes. That sounds like a clarification I should make in v3.

>>   The implementation of "git pull --recurse-submodules" is likely to
>>   change drastically as submodule.propagateBranches work progresses
>>   (e.g. "git merge" learns to recurse in to submodules), and we may be
>>   able to replace the "git submodule update" invocation, or teach it new
>>   tricks that make the update behave well.
>> 
>> - The user might make changes to the submodule branch without committing
>>   them back to superproject. This is primarily affects "git checkout
>>   --recurse-submodules", since that is the primary way of switching away
>>   from a branch and leaving behind WIP (as opposed to "git submodule
>>   update", which is run post-checkout).
>
> Makes sense - so in summary, there are (at least) two ways of the 
> superproject's gitlink and the submodule's branch can go out of sync: 
> the user changing the submodule branch (here) or the user changing the 
> superproject's gitlink (above). 

Yes. I didn't mention modifying the superproject's gitlink (outside of
"git pull"), but I imagine that it would use the same guardrail I
mentioned here. I could add mention of it if that seems useful.

>  
>>   In a future series, "git checkout --recurse-submodules" will learn to
>>   consider submodule branches. We can introduce appropriate guardrails
>>   then, e.g. requiring that the superproject working tree is not dirty
>>   before switching away.
>> 
>> Signed-off-by: Glen Choo <chooglen@google.com>
>
> [snip]
>
>> @@ -2521,6 +2532,7 @@ static int update_submodule(struct update_data *update_data)
>>  {
>>  	int submodule_up_to_date;
>>  	int ret;
>> +	const char *submodule_head = "HEAD";
>
> I think it's clearer if this is initialized to NULL. I don't think the 
> submodule head is detached in all code paths when this is HEAD.  

Ok, I'll initialize to NULL since refs_resolve_ref_unsafe() can return
NULL. I'm quite convinced that it only returns NULL in the error case
though (and we need to handle that explicitly), otherwise detached HEAD
always results in HEAD.

>
>>  	ret = determine_submodule_update_strategy(the_repository,
>>  						  update_data->just_cloned,
>> @@ -2533,7 +2545,7 @@ static int update_submodule(struct update_data *update_data)
>>  	if (update_data->just_cloned)
>>  		oidcpy(&update_data->suboid, null_oid());
>>  	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
>> -				     &update_data->suboid, NULL))
>> +				     &update_data->suboid, &submodule_head))
>>  		return die_message(_("Unable to find current revision in submodule path '%s'"),
>>  				   update_data->displaypath);
>>  
>> @@ -2568,7 +2580,14 @@ static int update_submodule(struct update_data *update_data)
>>  		free(remote_ref);
>>  	}
>>  
>> -	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
>> +	if (!update_data->super_branch)
>> +		submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
>> +	else if (skip_prefix(submodule_head, "refs/heads/", &submodule_head))
>> +		submodule_up_to_date = !strcmp(update_data->super_branch, submodule_head);
>> +	/* submodule_branch is "HEAD"; the submodule is in detached HEAD */
>> +	else
>> +		submodule_up_to_date = 0;
>
> I think there needs to be a better comment here. It doesn't matter that 
> the submodule is in detached HEAD; what matters is that the submodule 
> doesn't have the superproject's branch checked out. So maybe something 
> like: 
>
>   if (update_data->super_branch) {
>     /* (format this appropriately) We also need to check that the 
>        submodule's HEAD points to super_branch. */ 
>     const char *submodule_head;
>     submodule_up_to_date = skip_prefix(...) && !strcmp(...)
>   } else {
>     submodule_up_to_date = oideq(...)
>   }
>

I see, thanks for the suggestion.

>> diff --git a/t/t5617-clone-submodules.sh b/t/t5617-clone-submodules.sh
>
> [snip]
>
>> @@ -107,4 +114,31 @@ test_expect_success '--no-also-filter-submodules overrides clone.filterSubmodule
>>  	test_cmp_config -C super_clone3/sub false --default false remote.origin.promisor
>>  '
>>  
>> +test_expect_success 'submodule.propagateBranches checks out branches at correct commits' '
>> +	git -C sub checkout -b not-main &&
>> +	git -C subsub checkout -b not-main &&
>> +	git clone --recurse-submodules \
>> +		-c submodule.propagateBranches=true \
>> +		"file://$pwd/." super_clone4 &&
>> +
>> +	# Assert that each repo is pointing to "main"
>> +	for REPO in "super_clone4" "super_clone4/sub" "super_clone4/sub/subsub"
>> +	do
>> +	    HEAD_BRANCH=$(git -C $REPO symbolic-ref HEAD) &&
>> +	    test $HEAD_BRANCH = "refs/heads/main" || return 1
>> +	done &&
>
> As I said in my earlier review [1], could we use a branch name that is 
> not  "main"? That way, we also check that the clone *creates* the 
> branches,  not just reuses something already there. 
>
> [1] https://lore.kernel.org/git/20220901200047.515294-1-jonathantanmy@google.com/

Ah I missed that, thanks.

>
>> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
>
> [snip]
>
>> +test_expect_success 'submodule.propagateBranches - detached HEAD' '
>> +	test_when_finished "rm -fr branch-super-cloned" &&
>> +	cp -r branch-super-clean branch-super-cloned &&
>> +
>> +	git -C branch-super-cloned checkout --detach &&
>> +	git -C branch-super-cloned pull origin main &&
>
> If the behavior of "pull" is going to change soon (as stated in the 
> commit message), can we avoid using it in tests here? 

Here I've omitted "--recurse-submodules", so this is a shorthand for
"update the superproject's HEAD to the remote's main". This behavior
isn't going to change, but perhaps I shouldn't rely on such subtleties
in test code (where readability is paramount).

>
>> +	git -C branch-super-cloned submodule update &&
>> +
>> +	# sub2 should be in detached HEAD
>> +	git -C branch-super-cloned/sub2 rev-parse --verify HEAD &&
>> +	test_must_fail git -C branch-super-cloned/sub2 symbolic-ref HEAD
>> +'
>> +
>> +test_expect_success 'submodule.propagateBranches - branch checked out' '
>> +	test_when_finished "rm -fr branch-super-cloned" &&
>> +	cp -r branch-super-clean branch-super-cloned &&
>> +
>> +	git -C branch-super-cloned branch --recurse-submodules new-branch &&
>> +	git -C branch-super-cloned checkout --recurse-submodules new-branch &&
>> +	git -C branch-super-cloned pull origin main &&
>> +	git -C branch-super-cloned submodule update &&
>> +
>> +	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
>> +	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
>> +	HEAD_BRANCH2=$(git -C branch-super-cloned/sub2 symbolic-ref HEAD) &&
>> +	test $HEAD_BRANCH2 = "refs/heads/new-branch"
>
> I'm not sure of the behavior of "pull" here, so I didn't look too  
> closely into what these tests test, but I think that you should cover 
> these situations: 
>  - submodule has correct OID and correct branch checked out
>  - submodule has correct OID, but wrong branch checked out, and correct 
>    branch doesn't exist 
>  - submodule has correct OID, but wrong branch checked out, and correct 
>    branch exists at correct OID 
>  - submodule has correct OID, but wrong branch checked out, and correct 
>    branch exists at wrong OID 
>  - submodule has incorrect OID but correct branch checked out (if this 
>    currently doesn't work, maybe add a test_must_fail with a NEEDSWORK) 
>
> There are other combinations but I think these are the most important 
> ones. 
>
> Also, if there is a situation where you ignore the superproject's 
> gitlink in favor of the submodule's branch, you should check the OIDs 
> in all cases too, not just what the HEAD symbolic ref points to. 

Makes sense. Thanks for the thorough review :)

^ permalink raw reply	[flat|nested] 56+ messages in thread

* [PATCH v3 0/8] clone, submodule update: check out submodule branches
  2022-10-20 20:20 ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
                     ` (7 preceding siblings ...)
  2022-10-20 22:40   ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Junio C Hamano
@ 2022-10-28 20:14   ` Glen Choo via GitGitGadget
  2022-10-28 20:14     ` [PATCH v3 1/8] clone: teach --detach option Glen Choo via GitGitGadget
                       ` (9 more replies)
  8 siblings, 10 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-28 20:14 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo

This version has relatively few changes, and should address all of
Jonathan's comments (thanks!).

In the final patch, submodule_head is now initialized to NULL [1], which
means that in the "just_cloned" case, it remains NULL and we have to be
careful not to call skip_prefix() on it. A new patch (7/8), prepares for
this and also cleans up "struct update_data" a little bit.

= Description

This series teaches "git clone --recurse-submodules" and "git submodule
update" to understand "submodule.propagateBranches" (see Further Reading for
context), i.e. if the superproject has a branch checked out and a submodule
is cloned, the submodule will have the same branch checked out.

To do this, "git submodule update" checks if "submodule.propagateBranches"
is true. If so, and if the superproject has the branch 'topic' checked out,
then:

 * Submodules are cloned without their upstream branches
 * The 'topic' branch is created in the submodule
 * The submodule is updated via "git checkout topic" instead of checking out
   the gitlink's OID.

Since "git clone --recurse-submodules" is implemented using "git submodule
update", it also learns to create and check out the branch in submodules.

The main challenges with this approach are:

 * If the remote HEAD points to a branch, "git clone" always creates that
   branch in the clone. But with "submodule.propagateBranches", we want
   submodules to use the branch names of their superproject, not their
   upstream.
   
   This is solved by adding a new flag to "git clone", "--detach", which
   detaches the clone's HEAD at the branch and does not create it.

 * When "git submodule update" recurses into submodules, the parent process
   has to propagate the value of "submodule.propagateBranches" to child
   processes, otherwise the behavior will be inconsistent if the submodule
   has the config unset.
   
   This is solved by adding an internal GIT_* environment variable and
   passing it down via prepare_submodule_repo_env(). This is cleaner than
   passing "-c submodule.propagateBranches=true", but an even cleaner
   solution would be for submodules to read "submodule.propagateBranches"
   from their superproject config. This would also be useful for
   "submodule.alternateLocation" and "submodule.alternateErrorStrategy", as
   we wouldn't have to set those values in newly-cloned submodules. This
   requires teaching Git to treat submodules differently, which was the
   subject of some WIP in [2]. That topic has stalled, but I don't mind
   restarting it if others prefer that.

[1]
https://lore.kernel.org/git/20221025175628.913542-1-jonathantanmy@google.com
[2]
https://lore.kernel.org/git/20220310004423.2627181-1-emilyshaffer@google.com/

= Patch organization

 * Patch 1/8 adds "--detach" to "git clone"
 * Patch 2/8 creates the environment variable and repository setting for
   "submodule.propagateBranches"
 * Patch 3/8 adds a new "--branch" option to "git submodule clone", which
   makes it create a named branch.
 * Patches 4-7/8 are prep work, and 8/8 adds the actual
   "submodule.propagateBranches" behavior

= Series history

Changes in v3:

 * Patch 3: Improve error messages for "git submodule--helper clone
   --branch"
 * Patch 7 (new): Remove update_data.suboid to make it easier to initialize
   submodule_head to NULL in the next patch.
 * Patch 8: Split tests into two parts - "newly cloned" and "already
   cloned", and test various conditions in already cloned submodules.
 * Assorted commit message and comment changes.
 * Reformat with "make style" (sorry for not doing this sooner..)

Changes in v2:

 * The superproject's "submodule.propagateBranches" value is always used,
   even if false.
 * Branches are now created at clone time (by adding a new flag to "git
   submodule clone"), instead of at update time.
 * Rebase onto newer master. This got adjusted slightly to incorporate
   ab/submodule-helper-leakfix.
 * Add more tests to demonstrate edge case behavior.
 * Assorted commit message and doc improvements.

= Future work

 * Patch 5, which refactors resolve_gitlink_ref(), notes that a better
   interface would be to return the refname instead of using an "out"
   parameter, but we use an "out" parameter so that any new callers trying
   to use the old function signature will get stopped by the compiler. The
   refactor can be finished at a later time.

 * Patch 5 uses the name "target" when we are talking about what a symref
   points to, instead of "referent" like the other functions. "target" is
   the better choice, since "referent" could also apply to non-symbolic
   refs, but that cleanup is quite big.

 * Patch 8 notes that for already cloned submodules, the branch may not
   point to the same OID as the superproject gitlink, and it may not even
   exist. This will be addressed in a more comprehensive manner when we add
   support for checking out branches with "git checkout
   --recurse-submodules".

= Further reading

Submodule branching RFC:
https://lore.kernel.org/git/kl6lv912uvjv.fsf@chooglen-macbookpro.roam.corp.google.com/

Original Submodule UX RFC/Discussion:
https://lore.kernel.org/git/YHofmWcIAidkvJiD@google.com/

Contributor Summit submodules Notes:
https://lore.kernel.org/git/nycvar.QRO.7.76.6.2110211148060.56@tvgsbejvaqbjf.bet/

Submodule UX overhaul updates:
https://lore.kernel.org/git/?q=Submodule+UX+overhaul+update

"git branch --recurse-submodules":
https://lore.kernel.org/git/20220129000446.99261-1-chooglen@google.com/

Glen Choo (8):
  clone: teach --detach option
  repo-settings: add submodule_propagate_branches
  submodule--helper clone: create named branch
  t5617: drop references to remote-tracking branches
  submodule: return target of submodule symref
  submodule update: refactor update targets
  submodule--helper: remove update_data.suboid
  clone, submodule update: create and check out branches

 Documentation/git-clone.txt                   |   8 +-
 builtin/branch.c                              |  11 +-
 builtin/clone.c                               |  12 +-
 builtin/submodule--helper.c                   | 102 +++++++++---
 builtin/update-index.c                        |   4 +-
 cache.h                                       |   1 +
 combine-diff.c                                |   3 +-
 diff-lib.c                                    |   2 +-
 dir.c                                         |   2 +-
 object-file.c                                 |   2 +-
 read-cache.c                                  |   4 +-
 refs.c                                        |  10 +-
 refs.h                                        |   5 +-
 repo-settings.c                               |  13 ++
 repository.h                                  |   1 +
 submodule.c                                   |   2 +
 t/t5601-clone.sh                              |  22 +++
 ...es-remote.sh => t5617-clone-submodules.sh} |  40 ++++-
 t/t7406-submodule-update.sh                   | 156 ++++++++++++++++++
 unpack-trees.c                                |   3 +-
 20 files changed, 357 insertions(+), 46 deletions(-)
 rename t/{t5617-clone-submodules-remote.sh => t5617-clone-submodules.sh} (70%)


base-commit: 45c9f05c44b1cb6bd2d6cb95a22cf5e3d21d5b63
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1321%2Fchooglen%2Fsubmodule%2Fclone-recursive-with-branch-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1321/chooglen/submodule/clone-recursive-with-branch-v3
Pull-Request: https://github.com/git/git/pull/1321

Range-diff vs v2:

 1:  432bc7cb3a4 = 1:  432bc7cb3a4 clone: teach --detach option
 2:  20499c62065 ! 2:  06d7e15e830 repo-settings: add submodule_propagate_branches
     @@ repo-settings.c: void prepare_repo_settings(struct repository *r)
       		r->settings.core_multi_pack_index = 1;
       
      +	/*
     -+	 * If the environment variable is set, assume that it came from the
     -+	 * superproject and ignore the config.
     ++	 * If the environment variable is set, assume that it was set by an
     ++	 * invocation of git running in a superproject with
     ++	 * submodule.propagateBranches set and that is recursing into this repo
     ++	 * as a submodule. Therefore, we should ignore whatever is set in this
     ++	 * repo's config.
      +	 */
      +	r->settings.submodule_propagate_branches =
      +		git_env_bool(GIT_SUBMODULE_PROPAGATE_BRANCHES_ENVIRONMENT, -1);
 3:  a4056e200ed ! 3:  5a24d7e9255 submodule--helper clone: create named branch
     @@ Commit message
          branching is enabled, otherwise the named branch might conflict with the
          branch from the submodule remote's HEAD.
      
     -    These flags will be used by `git submodule update` in a later commit.
     -    "git submodule add" (which also invokes "git submodule--helper clone")
     -    should also do something similar when submodule branching is enabled,
     -    but this is left for a later series.
     +    This functionality will be tested in a later commit where "git submodule
     +    update" uses it to create and check out the correct branch when
     +    submodule branching is enabled. "git submodule add" (which also invokes
     +    "git submodule--helper clone") should also do something similar when
     +    submodule branching is enabled, but this is left for a later series.
      
          Arguably, "--detach" should also be the default for
          "submodule.propagateBranches=false" since it doesn't make sense to
     @@ builtin/submodule--helper.c: static int module_clone(int argc, const char **argv
       	clone_data.dissociate = !!dissociate;
       	clone_data.quiet = !!quiet;
      @@ builtin/submodule--helper.c: static int module_clone(int argc, const char **argv, const char *prefix)
     - 	clone_data.require_init = !!require_init;
     - 	clone_data.filter_options = &filter_options;
     - 
     --	if (argc || !clone_data.url || !clone_data.path || !*(clone_data.path))
     -+	if (argc || !clone_data.url || !clone_data.path || !*(clone_data.path)
     -+	    || (!!clone_data.branch != !!clone_data.branch_oid))
       		usage_with_options(git_submodule_helper_usage,
       				   module_clone_options);
     + 
     ++	if (!!clone_data.branch != !!clone_data.branch_oid)
     ++		BUG("--branch and --branch-oid must be set/unset together");
      +	if ((clone_data.branch &&
      +	     !the_repository->settings.submodule_propagate_branches))
      +		BUG("--branch is only expected with submodule.propagateBranches");
     - 
     ++
       	clone_submodule(&clone_data, &reference);
       	list_objects_filter_release(&filter_options);
     + 	string_list_clear(&reference, 1);
 4:  affd0e24e1d = 4:  3a08f2ab776 t5617: drop references to remote-tracking branches
 5:  6f769cb80ad = 5:  bd8ffd7cde2 submodule: return target of submodule symref
 6:  abdfa888ff5 ! 6:  df1f7225f49 submodule update: refactor update targets
     @@ Commit message
          submodule update: refactor update targets
      
          Refactor two "git submodule update" code locations so that they no
     -    longer refer to oids directly. This shrinks the next commit's diff,
     +    longer refer to oids directly. This shrinks a subsequent commit's diff,
          where this code will need to handle branches.
      
          Signed-off-by: Glen Choo <chooglen@google.com>
 -:  ----------- > 7:  4e402b67145 submodule--helper: remove update_data.suboid
 7:  3f98b0d1739 ! 8:  7cdd6c4184d clone, submodule update: create and check out branches
     @@ Commit message
          submodule branch points to the superproject gitlink. In other cases, it
          does not work as well, but we can handle them incrementally:
      
     -    - "git pull --recurse-submodules" merges the superproject tree (without
     -      updating the submodule branches), and runs "git submodule update" to
     -      update the worktrees, so it is almost guaranteed to result in a dirty
     -      worktree.
     +    - "git pull --recurse-submodules" merges the superproject tree,
     +      (changing the gitlink without updating the submodule branches), and
     +      runs "git submodule update" to update the worktrees, so it is almost
     +      guaranteed to result in a dirty worktree.
      
            The implementation of "git pull --recurse-submodules" is likely to
            change drastically as submodule.propagateBranches work progresses
     @@ builtin/submodule--helper.c: static void submodule_update_clone_release(struct s
       	char *displaypath;
      +	const char *super_branch;
       	enum submodule_update_type update_default;
     - 	struct object_id suboid;
       	struct string_list references;
     + 	struct submodule_update_strategy update_strategy;
      @@ builtin/submodule--helper.c: static int prepare_to_clone_next_submodule(const struct cache_entry *ce,
       		strvec_push(&child->args, suc->update_data->single_branch ?
       					      "--single-branch" :
       					      "--no-single-branch");
      +	if (ud->super_branch) {
      +		strvec_pushf(&child->args, "--branch=%s", ud->super_branch);
     -+		strvec_pushf(&child->args, "--branch-oid=%s", oid_to_hex(&ce->oid));
     ++		strvec_pushf(&child->args, "--branch-oid=%s",
     ++			     oid_to_hex(&ce->oid));
      +	}
       
       cleanup:
     @@ builtin/submodule--helper.c: static int fetch_in_submodule(const char *module_pa
      +	const char *update_target;
       	int ret;
       
     -+	if (ud->update_strategy.type == SM_UPDATE_CHECKOUT &&
     -+	    ud->super_branch)
     ++	if (ud->update_strategy.type == SM_UPDATE_CHECKOUT && ud->super_branch)
      +		update_target = ud->super_branch;
      +	else
      +		update_target = oid_to_hex(&ud->oid);
     @@ builtin/submodule--helper.c: static int fetch_in_submodule(const char *module_pa
       	case SM_UPDATE_CHECKOUT:
       		cp.git_cmd = 1;
      @@ builtin/submodule--helper.c: static int update_submodule(struct update_data *update_data)
     - {
       	int submodule_up_to_date;
       	int ret;
     -+	const char *submodule_head = "HEAD";
     + 	struct object_id suboid;
     ++	const char *submodule_head = NULL;
       
       	ret = determine_submodule_update_strategy(the_repository,
       						  update_data->just_cloned,
      @@ builtin/submodule--helper.c: static int update_submodule(struct update_data *update_data)
     - 	if (update_data->just_cloned)
     - 		oidcpy(&update_data->suboid, null_oid());
     - 	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
     --				     &update_data->suboid, NULL))
     -+				     &update_data->suboid, &submodule_head))
     + 		return ret;
     + 
     + 	if (!update_data->just_cloned &&
     +-	    resolve_gitlink_ref(update_data->sm_path, "HEAD", &suboid, NULL))
     ++	    resolve_gitlink_ref(update_data->sm_path, "HEAD", &suboid,
     ++				&submodule_head))
       		return die_message(_("Unable to find current revision in submodule path '%s'"),
       				   update_data->displaypath);
       
     @@ builtin/submodule--helper.c: static int update_submodule(struct update_data *upd
       		free(remote_ref);
       	}
       
     --	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
     -+	if (!update_data->super_branch)
     -+		submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
     -+	else if (skip_prefix(submodule_head, "refs/heads/", &submodule_head))
     -+		submodule_up_to_date = !strcmp(update_data->super_branch, submodule_head);
     -+	/* submodule_branch is "HEAD"; the submodule is in detached HEAD */
     -+	else
     +-	submodule_up_to_date = !update_data->just_cloned &&
     +-		oideq(&update_data->oid, &suboid);
     ++	if (update_data->just_cloned)
      +		submodule_up_to_date = 0;
     ++	else if (update_data->super_branch)
     ++		/* Check that the submodule's HEAD points to super_branch. */
     ++		submodule_up_to_date =
     ++			skip_prefix(submodule_head, "refs/heads/",
     ++				    &submodule_head) &&
     ++			!strcmp(update_data->super_branch, submodule_head);
     ++	else
     ++		submodule_up_to_date = oideq(&update_data->oid, &suboid);
      +
       	if (!submodule_up_to_date || update_data->force) {
       		ret = run_update_procedure(update_data);
     @@ t/t5617-clone-submodules.sh: test_expect_success '--no-also-filter-submodules ov
       '
       
      +test_expect_success 'submodule.propagateBranches checks out branches at correct commits' '
     -+	git -C sub checkout -b not-main &&
     -+	git -C subsub checkout -b not-main &&
     ++	test_when_finished "git checkout main" &&
     ++
     ++	git checkout -b checked-out &&
     ++	git -C sub checkout -b not-in-clone &&
     ++	git -C subsub checkout -b not-in-clone &&
      +	git clone --recurse-submodules \
     ++		--branch checked-out \
      +		-c submodule.propagateBranches=true \
      +		"file://$pwd/." super_clone4 &&
      +
     -+	# Assert that each repo is pointing to "main"
     ++	# Assert that each repo is pointing to "checked-out"
      +	for REPO in "super_clone4" "super_clone4/sub" "super_clone4/sub/subsub"
      +	do
      +	    HEAD_BRANCH=$(git -C $REPO symbolic-ref HEAD) &&
     -+	    test $HEAD_BRANCH = "refs/heads/main" || return 1
     ++	    test $HEAD_BRANCH = "refs/heads/checked-out" || return 1
      +	done &&
      +
      +	# Assert that the submodule branches are pointing to the right revs
      +	EXPECT_SUB_OID="$(git -C super_clone4 rev-parse :sub)" &&
     -+	ACTUAL_SUB_OID="$(git -C super_clone4/sub rev-parse refs/heads/main)" &&
     ++	ACTUAL_SUB_OID="$(git -C super_clone4/sub rev-parse refs/heads/checked-out)" &&
      +	test $EXPECT_SUB_OID = $ACTUAL_SUB_OID &&
      +	EXPECT_SUBSUB_OID="$(git -C super_clone4/sub rev-parse :subsub)" &&
     -+	ACTUAL_SUBSUB_OID="$(git -C super_clone4/sub/subsub rev-parse refs/heads/main)" &&
     ++	ACTUAL_SUBSUB_OID="$(git -C super_clone4/sub/subsub rev-parse refs/heads/checked-out)" &&
      +	test $EXPECT_SUBSUB_OID = $ACTUAL_SUBSUB_OID &&
      +
      +	# Assert that the submodules do not have branches from their upstream
     -+	test_must_fail git -C super_clone4/sub rev-parse not-main &&
     -+	test_must_fail git -C super_clone4/sub/subsub rev-parse not-main
     ++	test_must_fail git -C super_clone4/sub rev-parse not-in-clone &&
     ++	test_must_fail git -C super_clone4/sub/subsub rev-parse not-in-clone
      +'
      +
       test_done
     @@ t/t7406-submodule-update.sh: test_expect_success 'submodule update --recursive s
      +		branch-super branch-super-clean &&
      +	git -C branch-super-clean config submodule.propagateBranches true &&
      +
     -+	# Create an upstream submodule not in the clone
     ++	# sub2 will not be in the clone. We will fetch the containing
     ++	# superproject commit and clone sub2 with "git submodule update".
      +	git init sub2 &&
      +	test_commit -C sub2 "sub2" &&
      +	git -C branch-super submodule add ../sub2 sub2 &&
      +	git -C branch-super commit -m "add sub2"
      +'
      +
     -+test_expect_success 'submodule.propagateBranches - detached HEAD' '
     ++test_clean_submodule ()
     ++{
     ++	local negate super_dir sub_dir expect_oid actual_oid &&
     ++	if test "$1" = "!"
     ++	then
     ++		negate=t
     ++		shift
     ++	fi
     ++	super_dir="$1" &&
     ++	sub_dir="$2" &&
     ++	expect_oid="$(git -C "$super_dir" rev-parse ":$sub_dir")" &&
     ++	actual_oid="$(git -C "$super_dir/$sub_dir" rev-parse HEAD)" &&
     ++	if test -n "$negate"
     ++	then
     ++		! test "$expect_oid" = "$actual_oid"
     ++	else
     ++		test "$expect_oid" = "$actual_oid"
     ++	fi
     ++}
     ++
     ++# Test the behavior of a newly cloned submodule
     ++test_expect_success 'branches - newly-cloned submodule, detached HEAD' '
      +	test_when_finished "rm -fr branch-super-cloned" &&
      +	cp -r branch-super-clean branch-super-cloned &&
      +
     -+	git -C branch-super-cloned checkout --detach &&
     -+	git -C branch-super-cloned pull origin main &&
     ++	git -C branch-super-cloned fetch origin main &&
     ++	git -C branch-super-cloned checkout FETCH_HEAD &&
     ++	git -C branch-super-cloned/sub1 checkout --detach &&
      +	git -C branch-super-cloned submodule update &&
      +
     -+	# sub2 should be in detached HEAD
     ++	# sub1 and sub2 should be in detached HEAD
     ++	git -C branch-super-cloned/sub1 rev-parse --verify HEAD &&
     ++	test_must_fail git -C branch-super-cloned/sub1 symbolic-ref HEAD &&
     ++	test_clean_submodule branch-super-cloned sub1 &&
      +	git -C branch-super-cloned/sub2 rev-parse --verify HEAD &&
     -+	test_must_fail git -C branch-super-cloned/sub2 symbolic-ref HEAD
     ++	test_must_fail git -C branch-super-cloned/sub2 symbolic-ref HEAD &&
     ++	test_clean_submodule branch-super-cloned sub2
     ++'
     ++
     ++test_expect_success 'branches - newly-cloned submodule, branch checked out' '
     ++	test_when_finished "rm -fr branch-super-cloned" &&
     ++	cp -r branch-super-clean branch-super-cloned &&
     ++
     ++	git -C branch-super-cloned fetch origin main &&
     ++	git -C branch-super-cloned checkout FETCH_HEAD &&
     ++	git -C branch-super-cloned branch new-branch &&
     ++	git -C branch-super-cloned checkout new-branch &&
     ++	git -C branch-super-cloned/sub1 branch new-branch &&
     ++	git -C branch-super-cloned submodule update &&
     ++
     ++	# Ignore sub1, we will test it later.
     ++	# sub2 should check out the branch
     ++	HEAD_BRANCH2=$(git -C branch-super-cloned/sub2 symbolic-ref HEAD) &&
     ++	test $HEAD_BRANCH2 = "refs/heads/new-branch" &&
     ++	test_clean_submodule branch-super-cloned sub2
      +'
      +
     -+test_expect_success 'submodule.propagateBranches - branch checked out' '
     ++# Test the behavior of an already-cloned submodule.
     ++# NEEDSWORK When updating with branches, we always use the branch instead of the
     ++# gitlink's OID. This results in some imperfect behavior:
     ++#
     ++# - If the gitlink's OID disagrees with the branch OID, updating with branches
     ++#   may result in a dirty worktree
     ++# - If the branch does not exist, the update fails.
     ++#
     ++# We will reevaluate when "git checkout --recurse-submodules" supports branches
     ++# For now, just test for this imperfect behavior.
     ++test_expect_success 'branches - correct branch checked out, OIDs agree' '
      +	test_when_finished "rm -fr branch-super-cloned" &&
      +	cp -r branch-super-clean branch-super-cloned &&
      +
      +	git -C branch-super-cloned branch --recurse-submodules new-branch &&
     -+	git -C branch-super-cloned checkout --recurse-submodules new-branch &&
     -+	git -C branch-super-cloned pull origin main &&
     ++	git -C branch-super-cloned checkout new-branch &&
     ++	git -C branch-super-cloned/sub1 checkout new-branch &&
      +	git -C branch-super-cloned submodule update &&
      +
      +	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
      +	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
     -+	HEAD_BRANCH2=$(git -C branch-super-cloned/sub2 symbolic-ref HEAD) &&
     -+	test $HEAD_BRANCH2 = "refs/heads/new-branch"
     ++	test_clean_submodule branch-super-cloned sub1
      +'
      +
     -+test_expect_success 'submodule.propagateBranches - other branch checked out' '
     ++test_expect_success 'branches - correct branch checked out, OIDs disagree' '
      +	test_when_finished "rm -fr branch-super-cloned" &&
      +	cp -r branch-super-clean branch-super-cloned &&
      +
      +	git -C branch-super-cloned branch --recurse-submodules new-branch &&
     -+	git -C branch-super-cloned checkout --recurse-submodules new-branch &&
     -+	git -C branch-super-cloned/sub1 checkout -b other-branch &&
     -+	git -C branch-super-cloned pull origin main &&
     ++	git -C branch-super-cloned checkout new-branch &&
     ++	git -C branch-super-cloned/sub1 checkout new-branch &&
     ++	test_commit -C branch-super-cloned/sub1 new-commit &&
      +	git -C branch-super-cloned submodule update &&
      +
      +	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
      +	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
     -+	HEAD_BRANCH2=$(git -C branch-super-cloned/sub2 symbolic-ref HEAD) &&
     -+	test $HEAD_BRANCH2 = "refs/heads/new-branch"
     ++	test_clean_submodule ! branch-super-cloned sub1
     ++'
     ++
     ++test_expect_success 'branches - other branch checked out, correct branch exists, OIDs agree' '
     ++	test_when_finished "rm -fr branch-super-cloned" &&
     ++	cp -r branch-super-clean branch-super-cloned &&
     ++
     ++	git -C branch-super-cloned branch --recurse-submodules new-branch &&
     ++	git -C branch-super-cloned checkout new-branch &&
     ++	git -C branch-super-cloned/sub1 checkout main &&
     ++	git -C branch-super-cloned submodule update &&
     ++
     ++	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
     ++	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
     ++	test_clean_submodule branch-super-cloned sub1
     ++'
     ++
     ++test_expect_success 'branches - other branch checked out, correct branch exists, OIDs disagree' '
     ++	test_when_finished "rm -fr branch-super-cloned" &&
     ++	cp -r branch-super-clean branch-super-cloned &&
     ++
     ++	git -C branch-super-cloned branch --recurse-submodules new-branch &&
     ++	git -C branch-super-cloned checkout new-branch &&
     ++	git -C branch-super-cloned/sub1 checkout new-branch &&
     ++	test_commit -C branch-super-cloned/sub1 new-commit &&
     ++	git -C branch-super-cloned/sub1 checkout main &&
     ++	git -C branch-super-cloned submodule update &&
     ++
     ++	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
     ++	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
     ++	test_clean_submodule ! branch-super-cloned sub1
     ++'
     ++
     ++test_expect_success 'branches - other branch checked out, correct branch does not exist' '
     ++	test_when_finished "rm -fr branch-super-cloned" &&
     ++	cp -r branch-super-clean branch-super-cloned &&
     ++
     ++	git -C branch-super-cloned branch new-branch &&
     ++	git -C branch-super-cloned checkout new-branch &&
     ++	test_must_fail git -C branch-super-cloned submodule update
      +'
      +
       test_done

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 56+ messages in thread

* [PATCH v3 1/8] clone: teach --detach option
  2022-10-28 20:14   ` [PATCH v3 0/8] " Glen Choo via GitGitGadget
@ 2022-10-28 20:14     ` Glen Choo via GitGitGadget
  2022-10-28 21:40       ` Junio C Hamano
  2022-11-08 13:32       ` Philippe Blain
  2022-10-28 20:14     ` [PATCH v3 2/8] repo-settings: add submodule_propagate_branches Glen Choo via GitGitGadget
                       ` (8 subsequent siblings)
  9 siblings, 2 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-28 20:14 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

Teach "git clone" the "--detach" option, which leaves the cloned repo in
detached HEAD (like "git checkout --detach"). In addition, if the clone
is not bare, do not create the local branch pointed to by the remote's
HEAD symref (bare clones always copy all remote branches directly to
local branches, so the branch is still created in the bare case).

This is especially useful in the "submodule.propagateBranches" workflow,
where local submodule branches are named after the superproject's
branches, so it makes no sense to create a local branch named after the
submodule's remote's branch.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 Documentation/git-clone.txt |  8 +++++++-
 builtin/clone.c             | 12 +++++++++---
 t/t5601-clone.sh            | 22 ++++++++++++++++++++++
 3 files changed, 38 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index d6434d262d6..6a4e5d31b46 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -16,7 +16,7 @@ SYNOPSIS
 	  [--depth <depth>] [--[no-]single-branch] [--no-tags]
 	  [--recurse-submodules[=<pathspec>]] [--[no-]shallow-submodules]
 	  [--[no-]remote-submodules] [--jobs <n>] [--sparse] [--[no-]reject-shallow]
-	  [--filter=<filter> [--also-filter-submodules]] [--] <repository>
+	  [--filter=<filter> [--also-filter-submodules] [--detach]] [--] <repository>
 	  [<directory>]
 
 DESCRIPTION
@@ -210,6 +210,12 @@ objects from the source repository into a pack in the cloned repository.
 	`--branch` can also take tags and detaches the HEAD at that commit
 	in the resulting repository.
 
+--detach::
+	If the cloned repository's HEAD points to a branch, point the newly
+	created HEAD to the branch's commit instead of the branch itself.
+	Additionally, in a non-bare repository, the corresponding local branch
+	will not be created.
+
 -u <upload-pack>::
 --upload-pack <upload-pack>::
 	When given, and the repository to clone from is accessed
diff --git a/builtin/clone.c b/builtin/clone.c
index 547d6464b3c..e624d3f49a2 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -78,6 +78,7 @@ static int option_filter_submodules = -1;    /* unspecified */
 static int config_filter_submodules = -1;    /* unspecified */
 static struct string_list server_options = STRING_LIST_INIT_NODUP;
 static int option_remote_submodules;
+static int option_detach;
 static const char *bundle_uri;
 
 static int recurse_submodules_cb(const struct option *opt,
@@ -162,6 +163,8 @@ static struct option builtin_clone_options[] = {
 		    N_("any cloned submodules will use their remote-tracking branch")),
 	OPT_BOOL(0, "sparse", &option_sparse_checkout,
 		    N_("initialize sparse-checkout file to include only files at root")),
+	OPT_BOOL(0, "detach", &option_detach,
+		 N_("detach HEAD and don't create a local branch")),
 	OPT_STRING(0, "bundle-uri", &bundle_uri,
 		   N_("uri"), N_("a URI for downloading bundles before fetching from origin remote")),
 	OPT_END()
@@ -613,10 +616,12 @@ static void update_remote_refs(const struct ref *refs,
 }
 
 static void update_head(const struct ref *our, const struct ref *remote,
-			const char *unborn, const char *msg)
+			const char *unborn, int should_detach,
+			const char *msg)
 {
 	const char *head;
-	if (our && skip_prefix(our->name, "refs/heads/", &head)) {
+	if (our && !should_detach &&
+	    skip_prefix(our->name, "refs/heads/", &head)) {
 		/* Local default branch link */
 		if (create_symref("HEAD", our->name, NULL) < 0)
 			die(_("unable to update HEAD"));
@@ -1357,7 +1362,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 			   branch_top.buf, reflog_msg.buf, transport,
 			   !is_local);
 
-	update_head(our_head_points_at, remote_head, unborn_head, reflog_msg.buf);
+	update_head(our_head_points_at, remote_head, unborn_head,
+		    option_detach, reflog_msg.buf);
 
 	/*
 	 * We want to show progress for recursive submodule clones iff
diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh
index 45f0803ed4d..418cfd54717 100755
--- a/t/t5601-clone.sh
+++ b/t/t5601-clone.sh
@@ -333,6 +333,28 @@ test_expect_success 'clone checking out a tag' '
 	test_cmp fetch.expected fetch.actual
 '
 
+test_expect_success '--detach detaches and does not create branch' '
+	test_when_finished "rm -fr dst" &&
+	git clone --detach src dst &&
+	(
+		cd dst &&
+		test_must_fail git rev-parse main &&
+		test_must_fail git symbolic-ref HEAD &&
+		test_cmp_rev HEAD refs/remotes/origin/HEAD
+	)
+'
+
+test_expect_success '--detach with --bare detaches but creates branch' '
+	test_when_finished "rm -fr dst" &&
+	git clone --bare --detach src dst &&
+	(
+		cd dst &&
+		git rev-parse main &&
+		test_must_fail git symbolic-ref HEAD &&
+		test_cmp_rev HEAD refs/heads/main
+	)
+'
+
 test_expect_success 'set up ssh wrapper' '
 	cp "$GIT_BUILD_DIR/t/helper/test-fake-ssh$X" \
 		"$TRASH_DIRECTORY/ssh$X" &&
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH v3 2/8] repo-settings: add submodule_propagate_branches
  2022-10-28 20:14   ` [PATCH v3 0/8] " Glen Choo via GitGitGadget
  2022-10-28 20:14     ` [PATCH v3 1/8] clone: teach --detach option Glen Choo via GitGitGadget
@ 2022-10-28 20:14     ` Glen Choo via GitGitGadget
  2022-10-28 20:14     ` [PATCH v3 3/8] submodule--helper clone: create named branch Glen Choo via GitGitGadget
                       ` (7 subsequent siblings)
  9 siblings, 0 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-28 20:14 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

When processes recurse into submodules, the child processes have to
use the same value of "submodule.propagateBranches" as the parent
process regardless of whether the process is spawned in the superproject
or submodule, otherwise the behavior may be inconsistent if the
repositories don't agree on the config.

We haven't needed a way to propagate the config because the only command
that reads "submodule.propagateBranches" is "git branch", which only has
one mode of operation with "--recurse-submodules". However, a future
commit will teach "submodule.propagateBranches" to "git submodule
update", making this necessary.

Propagate "submodule.propagateBranches" to child processes by adding a
corresponding GIT_INTERNAL_* environment variable and repository
setting, and setting the environment variable inside
prepare_submodule_repo_env(). Then, refactor builtin/branch.c to read
the repository setting.

Using an internal environment variable is a potentially leaky
abstraction because environment variables can come from sources besides
the parent process. A more robust solution would be to teach Git that
the repository is a submodule and to only read
"submodule.propagateBranches" from the superproject config. There is WIP
for this on the ML [1].

Another alternative would be to pass "-c submodule.propagateBranches" to
all child processes. This is error-prone because many different
processes are invoked directly or indirectly by "git submodule update"
(e.g. "git submodule--helper clone", "git clone", "git checkout"). With
an environment variable, we can avoid this work because
prepare_submodule_repo_env() is already called for submodule child
processes.

[1] https://lore.kernel.org/git/20220310004423.2627181-1-emilyshaffer@google.com/

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/branch.c | 11 +++--------
 cache.h          |  1 +
 repo-settings.c  | 13 +++++++++++++
 repository.h     |  1 +
 submodule.c      |  2 ++
 5 files changed, 20 insertions(+), 8 deletions(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index e0e0af43202..8279d4eeb15 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -40,7 +40,6 @@ static const char * const builtin_branch_usage[] = {
 static const char *head;
 static struct object_id head_oid;
 static int recurse_submodules = 0;
-static int submodule_propagate_branches = 0;
 
 static int branch_use_color = -1;
 static char branch_colors[][COLOR_MAXLEN] = {
@@ -106,10 +105,6 @@ static int git_branch_config(const char *var, const char *value, void *cb)
 		recurse_submodules = git_config_bool(var, value);
 		return 0;
 	}
-	if (!strcasecmp(var, "submodule.propagateBranches")) {
-		submodule_propagate_branches = git_config_bool(var, value);
-		return 0;
-	}
 
 	return git_color_default_config(var, value, cb);
 }
@@ -723,7 +718,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, builtin_branch_usage,
 			     0);
-
+	prepare_repo_settings(the_repository);
 	if (!delete && !rename && !copy && !edit_description && !new_upstream &&
 	    !show_current && !unset_upstream && argc == 0)
 		list = 1;
@@ -739,7 +734,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 		usage_with_options(builtin_branch_usage, options);
 
 	if (recurse_submodules_explicit) {
-		if (!submodule_propagate_branches)
+		if (!the_repository->settings.submodule_propagate_branches)
 			die(_("branch with --recurse-submodules can only be used if submodule.propagateBranches is enabled"));
 		if (noncreate_actions)
 			die(_("--recurse-submodules can only be used to create branches"));
@@ -747,7 +742,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
 
 	recurse_submodules =
 		(recurse_submodules || recurse_submodules_explicit) &&
-		submodule_propagate_branches;
+		the_repository->settings.submodule_propagate_branches;
 
 	if (filter.abbrev == -1)
 		filter.abbrev = DEFAULT_ABBREV;
diff --git a/cache.h b/cache.h
index 26ed03bd6de..151e1d49e77 100644
--- a/cache.h
+++ b/cache.h
@@ -505,6 +505,7 @@ static inline enum object_type object_type(unsigned int mode)
 #define GIT_WORK_TREE_ENVIRONMENT "GIT_WORK_TREE"
 #define GIT_PREFIX_ENVIRONMENT "GIT_PREFIX"
 #define GIT_SUPER_PREFIX_ENVIRONMENT "GIT_INTERNAL_SUPER_PREFIX"
+#define GIT_SUBMODULE_PROPAGATE_BRANCHES_ENVIRONMENT "GIT_INTERNAL_SUBMODULE_PROPAGATE_BRANCHES"
 #define DEFAULT_GIT_DIR_ENVIRONMENT ".git"
 #define DB_ENVIRONMENT "GIT_OBJECT_DIRECTORY"
 #define INDEX_ENVIRONMENT "GIT_INDEX_FILE"
diff --git a/repo-settings.c b/repo-settings.c
index e8b58151bc4..7ef2da9178d 100644
--- a/repo-settings.c
+++ b/repo-settings.c
@@ -71,6 +71,19 @@ void prepare_repo_settings(struct repository *r)
 	if (git_env_bool(GIT_TEST_MULTI_PACK_INDEX, 0))
 		r->settings.core_multi_pack_index = 1;
 
+	/*
+	 * If the environment variable is set, assume that it was set by an
+	 * invocation of git running in a superproject with
+	 * submodule.propagateBranches set and that is recursing into this repo
+	 * as a submodule. Therefore, we should ignore whatever is set in this
+	 * repo's config.
+	 */
+	r->settings.submodule_propagate_branches =
+		git_env_bool(GIT_SUBMODULE_PROPAGATE_BRANCHES_ENVIRONMENT, -1);
+	if (r->settings.submodule_propagate_branches == -1)
+		repo_cfg_bool(r, "submodule.propagateBranches",
+			      &r->settings.submodule_propagate_branches, 0);
+
 	/*
 	 * Non-boolean config
 	 */
diff --git a/repository.h b/repository.h
index 24316ac944e..3df6fdbdd5c 100644
--- a/repository.h
+++ b/repository.h
@@ -37,6 +37,7 @@ struct repo_settings {
 	int fetch_write_commit_graph;
 	int command_requires_full_index;
 	int sparse_index;
+	int submodule_propagate_branches;
 
 	struct fsmonitor_settings *fsmonitor; /* lazily loaded */
 
diff --git a/submodule.c b/submodule.c
index bf7a2c79183..624404957fa 100644
--- a/submodule.c
+++ b/submodule.c
@@ -503,6 +503,8 @@ static void print_submodule_diff_summary(struct repository *r, struct rev_info *
 
 void prepare_submodule_repo_env(struct strvec *out)
 {
+	if (the_repository->settings.submodule_propagate_branches)
+		strvec_pushf(out, "%s=1", GIT_SUBMODULE_PROPAGATE_BRANCHES_ENVIRONMENT);
 	prepare_other_repo_env(out, DEFAULT_GIT_DIR_ENVIRONMENT);
 }
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH v3 3/8] submodule--helper clone: create named branch
  2022-10-28 20:14   ` [PATCH v3 0/8] " Glen Choo via GitGitGadget
  2022-10-28 20:14     ` [PATCH v3 1/8] clone: teach --detach option Glen Choo via GitGitGadget
  2022-10-28 20:14     ` [PATCH v3 2/8] repo-settings: add submodule_propagate_branches Glen Choo via GitGitGadget
@ 2022-10-28 20:14     ` Glen Choo via GitGitGadget
  2022-10-28 20:14     ` [PATCH v3 4/8] t5617: drop references to remote-tracking branches Glen Choo via GitGitGadget
                       ` (6 subsequent siblings)
  9 siblings, 0 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-28 20:14 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

When submodule branching is enabled (i.e. "submodule.propagateBranches =
true"), submodules are expected to have the same set of branches as the
superproject. To support this behavior in newly cloned submodules, teach
"git submodule--helper clone" to:

- clone with the "--detach" flag (so that the submodule doesn't create a
  branch corresponding to the remote's HEAD)
- create a branch when using the --branch and --branch-oid flags

The --branch and --branch-oid flags are only allowed when submodule
branching is enabled, otherwise the named branch might conflict with the
branch from the submodule remote's HEAD.

This functionality will be tested in a later commit where "git submodule
update" uses it to create and check out the correct branch when
submodule branching is enabled. "git submodule add" (which also invokes
"git submodule--helper clone") should also do something similar when
submodule branching is enabled, but this is left for a later series.

Arguably, "--detach" should also be the default for
"submodule.propagateBranches=false" since it doesn't make sense to
create a submodule branch when the submodule is always expected to be in
detached HEAD. But, to be conservative, this commit does not change the
behavior of "submodule.propagateBranches=false".

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/submodule--helper.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 0b4acb442b2..c974206cad4 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1503,6 +1503,8 @@ struct module_clone_data {
 	const char *name;
 	const char *url;
 	const char *depth;
+	const char *branch;
+	const char *branch_oid;
 	struct list_objects_filter_options *filter_options;
 	unsigned int quiet: 1;
 	unsigned int progress: 1;
@@ -1692,6 +1694,8 @@ static int clone_submodule(const struct module_clone_data *clone_data,
 			strvec_push(&cp.args, clone_data->single_branch ?
 				    "--single-branch" :
 				    "--no-single-branch");
+		if (the_repository->settings.submodule_propagate_branches)
+			strvec_push(&cp.args, "--detach");
 
 		strvec_push(&cp.args, "--");
 		strvec_push(&cp.args, clone_data->url);
@@ -1704,6 +1708,21 @@ static int clone_submodule(const struct module_clone_data *clone_data,
 		if(run_command(&cp))
 			die(_("clone of '%s' into submodule path '%s' failed"),
 			    clone_data->url, clone_data_path);
+
+		if (clone_data->branch) {
+			struct child_process branch_cp = CHILD_PROCESS_INIT;
+
+			branch_cp.git_cmd = 1;
+			prepare_other_repo_env(&branch_cp.env, sm_gitdir);
+
+			strvec_pushl(&branch_cp.args, "branch",
+				     clone_data->branch, clone_data->branch_oid,
+				     NULL);
+
+			if (run_command(&branch_cp))
+				die(_("could not create branch '%s' in submodule path '%s'"),
+				    clone_data->branch, clone_data_path);
+		}
 	} else {
 		char *path;
 
@@ -1778,6 +1797,12 @@ static int module_clone(int argc, const char **argv, const char *prefix)
 			   N_("disallow cloning into non-empty directory")),
 		OPT_BOOL(0, "single-branch", &clone_data.single_branch,
 			 N_("clone only one branch, HEAD or --branch")),
+		OPT_STRING(0, "branch", &clone_data.branch,
+			   N_("string"),
+			   N_("name of branch to be created")),
+		OPT_STRING(0, "branch-oid", &clone_data.branch_oid,
+			   N_("object-id"),
+			   N_("commit id for new branch")),
 		OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
 		OPT_END()
 	};
@@ -1785,12 +1810,14 @@ static int module_clone(int argc, const char **argv, const char *prefix)
 		N_("git submodule--helper clone [--prefix=<path>] [--quiet] "
 		   "[--reference <repository>] [--name <name>] [--depth <depth>] "
 		   "[--single-branch] [--filter <filter-spec>] "
+		   "[--branch <branch> --branch-oid <oid>]"
 		   "--url <url> --path <path>"),
 		NULL
 	};
 
 	argc = parse_options(argc, argv, prefix, module_clone_options,
 			     git_submodule_helper_usage, 0);
+	prepare_repo_settings(the_repository);
 
 	clone_data.dissociate = !!dissociate;
 	clone_data.quiet = !!quiet;
@@ -1802,6 +1829,12 @@ static int module_clone(int argc, const char **argv, const char *prefix)
 		usage_with_options(git_submodule_helper_usage,
 				   module_clone_options);
 
+	if (!!clone_data.branch != !!clone_data.branch_oid)
+		BUG("--branch and --branch-oid must be set/unset together");
+	if ((clone_data.branch &&
+	     !the_repository->settings.submodule_propagate_branches))
+		BUG("--branch is only expected with submodule.propagateBranches");
+
 	clone_submodule(&clone_data, &reference);
 	list_objects_filter_release(&filter_options);
 	string_list_clear(&reference, 1);
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH v3 4/8] t5617: drop references to remote-tracking branches
  2022-10-28 20:14   ` [PATCH v3 0/8] " Glen Choo via GitGitGadget
                       ` (2 preceding siblings ...)
  2022-10-28 20:14     ` [PATCH v3 3/8] submodule--helper clone: create named branch Glen Choo via GitGitGadget
@ 2022-10-28 20:14     ` Glen Choo via GitGitGadget
  2022-10-28 20:14     ` [PATCH v3 5/8] submodule: return target of submodule symref Glen Choo via GitGitGadget
                       ` (5 subsequent siblings)
  9 siblings, 0 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-28 20:14 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

It has included submodule cloning tests without remote-tracking branches
tests since f05da2b48b (clone, submodule: pass partial clone filters to
submodules, 2022-02-04) at least. Rename it accordingly so that we can
put future submodule cloning tests there.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 ...617-clone-submodules-remote.sh => t5617-clone-submodules.sh} | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 rename t/{t5617-clone-submodules-remote.sh => t5617-clone-submodules.sh} (97%)

diff --git a/t/t5617-clone-submodules-remote.sh b/t/t5617-clone-submodules.sh
similarity index 97%
rename from t/t5617-clone-submodules-remote.sh
rename to t/t5617-clone-submodules.sh
index 68843382493..c43a5b26fab 100755
--- a/t/t5617-clone-submodules-remote.sh
+++ b/t/t5617-clone-submodules.sh
@@ -1,6 +1,6 @@
 #!/bin/sh
 
-test_description='Test cloning repos with submodules using remote-tracking branches'
+test_description='Test cloning repos with submodules'
 
 GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
 export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH v3 5/8] submodule: return target of submodule symref
  2022-10-28 20:14   ` [PATCH v3 0/8] " Glen Choo via GitGitGadget
                       ` (3 preceding siblings ...)
  2022-10-28 20:14     ` [PATCH v3 4/8] t5617: drop references to remote-tracking branches Glen Choo via GitGitGadget
@ 2022-10-28 20:14     ` Glen Choo via GitGitGadget
  2022-10-28 21:49       ` Junio C Hamano
  2022-10-28 20:14     ` [PATCH v3 6/8] submodule update: refactor update targets Glen Choo via GitGitGadget
                       ` (4 subsequent siblings)
  9 siblings, 1 reply; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-28 20:14 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

resolve_gitlink_ref() can tell us which oid the submodule ref is
pointing to, but in a future commit, we would also like to know the
symbolic ref target if we are checking a symbolic ref. Teach
resolve_gitlink_ref() to "return" the symbolic ref's target via an "out"
parameter.

This changes resolve_gitlink_ref()'s signature so that new callers
trying to use the old signature will be stopped by the compiler. If we
returned the target instead (just like refs_resolve_ref_unsafe()), we
would be more consistent with refs_resolve_ref_unsafe(), but callers
expecting the old signature will get the opposite return value from what
they expect (since exit code 0 means success, but NULL pointer means
failure). We should do this refactor once we think that nobody will try
to use the old signature.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/submodule--helper.c |  8 +++++---
 builtin/update-index.c      |  4 ++--
 combine-diff.c              |  3 ++-
 diff-lib.c                  |  2 +-
 dir.c                       |  2 +-
 object-file.c               |  2 +-
 read-cache.c                |  4 ++--
 refs.c                      | 10 ++++++----
 refs.h                      |  5 ++++-
 unpack-trees.c              |  3 ++-
 10 files changed, 26 insertions(+), 17 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index c974206cad4..9ca138374b7 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2533,7 +2533,8 @@ static int update_submodule(struct update_data *update_data)
 
 	if (update_data->just_cloned)
 		oidcpy(&update_data->suboid, null_oid());
-	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD", &update_data->suboid))
+	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
+				     &update_data->suboid, NULL))
 		return die_message(_("Unable to find current revision in submodule path '%s'"),
 				   update_data->displaypath);
 
@@ -2560,7 +2561,8 @@ static int update_submodule(struct update_data *update_data)
 						   update_data->sm_path);
 		}
 
-		if (resolve_gitlink_ref(update_data->sm_path, remote_ref, &update_data->oid))
+		if (resolve_gitlink_ref(update_data->sm_path, remote_ref,
+					&update_data->oid, NULL))
 			return die_message(_("Unable to find %s revision in submodule path '%s'"),
 					   remote_ref, update_data->sm_path);
 
@@ -3305,7 +3307,7 @@ static void die_on_repo_without_commits(const char *path)
 	strbuf_addstr(&sb, path);
 	if (is_nonbare_repository_dir(&sb)) {
 		struct object_id oid;
-		if (resolve_gitlink_ref(path, "HEAD", &oid) < 0)
+		if (resolve_gitlink_ref(path, "HEAD", &oid, NULL) < 0)
 			die(_("'%s' does not have a commit checked out"), path);
 	}
 	strbuf_release(&sb);
diff --git a/builtin/update-index.c b/builtin/update-index.c
index b62249905f1..19a21a4586c 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -339,7 +339,7 @@ static int process_directory(const char *path, int len, struct stat *st)
 		if (S_ISGITLINK(ce->ce_mode)) {
 
 			/* Do nothing to the index if there is no HEAD! */
-			if (resolve_gitlink_ref(path, "HEAD", &oid) < 0)
+			if (resolve_gitlink_ref(path, "HEAD", &oid, NULL) < 0)
 				return 0;
 
 			return add_one_path(ce, path, len, st);
@@ -365,7 +365,7 @@ static int process_directory(const char *path, int len, struct stat *st)
 	}
 
 	/* No match - should we add it as a gitlink? */
-	if (!resolve_gitlink_ref(path, "HEAD", &oid))
+	if (!resolve_gitlink_ref(path, "HEAD", &oid, NULL))
 		return add_one_path(NULL, path, len, st);
 
 	/* Error out. */
diff --git a/combine-diff.c b/combine-diff.c
index b0ece954808..88efcaeefa7 100644
--- a/combine-diff.c
+++ b/combine-diff.c
@@ -1060,7 +1060,8 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,
 			elem->mode = canon_mode(st.st_mode);
 		} else if (S_ISDIR(st.st_mode)) {
 			struct object_id oid;
-			if (resolve_gitlink_ref(elem->path, "HEAD", &oid) < 0)
+			if (resolve_gitlink_ref(elem->path, "HEAD", &oid,
+						NULL) < 0)
 				result = grab_blob(opt->repo, &elem->oid,
 						   elem->mode, &result_size,
 						   NULL, NULL);
diff --git a/diff-lib.c b/diff-lib.c
index 2edea41a234..ac94b6234ca 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -53,7 +53,7 @@ static int check_removed(const struct index_state *istate, const struct cache_en
 		 * a directory --- the blob was removed!
 		 */
 		if (!S_ISGITLINK(ce->ce_mode) &&
-		    resolve_gitlink_ref(ce->name, "HEAD", &sub))
+		    resolve_gitlink_ref(ce->name, "HEAD", &sub, NULL))
 			return 1;
 	}
 	return 0;
diff --git a/dir.c b/dir.c
index d604d1bab98..81d232424c1 100644
--- a/dir.c
+++ b/dir.c
@@ -3251,7 +3251,7 @@ static int remove_dir_recurse(struct strbuf *path, int flag, int *kept_up)
 	struct object_id submodule_head;
 
 	if ((flag & REMOVE_DIR_KEEP_NESTED_GIT) &&
-	    !resolve_gitlink_ref(path->buf, "HEAD", &submodule_head)) {
+	    !resolve_gitlink_ref(path->buf, "HEAD", &submodule_head, NULL)) {
 		/* Do not descend and nuke a nested git work tree. */
 		if (kept_up)
 			*kept_up = 1;
diff --git a/object-file.c b/object-file.c
index 5e309602346..a3c6580c16e 100644
--- a/object-file.c
+++ b/object-file.c
@@ -2522,7 +2522,7 @@ int index_path(struct index_state *istate, struct object_id *oid,
 		strbuf_release(&sb);
 		break;
 	case S_IFDIR:
-		return resolve_gitlink_ref(path, "HEAD", oid);
+		return resolve_gitlink_ref(path, "HEAD", oid, NULL);
 	default:
 		return error(_("%s: unsupported file type"), path);
 	}
diff --git a/read-cache.c b/read-cache.c
index 32024029274..4c1bf33ef48 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -285,7 +285,7 @@ static int ce_compare_gitlink(const struct cache_entry *ce)
 	 *
 	 * If so, we consider it always to match.
 	 */
-	if (resolve_gitlink_ref(ce->name, "HEAD", &oid) < 0)
+	if (resolve_gitlink_ref(ce->name, "HEAD", &oid, NULL) < 0)
 		return 0;
 	return !oideq(&oid, &ce->oid);
 }
@@ -781,7 +781,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,
 
 	namelen = strlen(path);
 	if (S_ISDIR(st_mode)) {
-		if (resolve_gitlink_ref(path, "HEAD", &oid) < 0)
+		if (resolve_gitlink_ref(path, "HEAD", &oid, NULL) < 0)
 			return error(_("'%s' does not have a commit checked out"), path);
 		while (namelen && path[namelen-1] == '/')
 			namelen--;
diff --git a/refs.c b/refs.c
index 1491ae937eb..a32a25ccb69 100644
--- a/refs.c
+++ b/refs.c
@@ -1904,19 +1904,21 @@ const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
 }
 
 int resolve_gitlink_ref(const char *submodule, const char *refname,
-			struct object_id *oid)
+			struct object_id *oid, const char **target_out)
 {
 	struct ref_store *refs;
 	int flags;
+	const char *target;
 
 	refs = get_submodule_ref_store(submodule);
 
 	if (!refs)
 		return -1;
-
-	if (!refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags) ||
-	    is_null_oid(oid))
+	target = refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags);
+	if (!target || is_null_oid(oid))
 		return -1;
+	if (target_out)
+		*target_out = target;
 	return 0;
 }
 
diff --git a/refs.h b/refs.h
index 8958717a17d..d5c32fac000 100644
--- a/refs.h
+++ b/refs.h
@@ -137,9 +137,12 @@ int peel_iterated_oid(const struct object_id *base, struct object_id *peeled);
  * submodule (which must be non-NULL). If the resolution is
  * successful, return 0 and set oid to the name of the object;
  * otherwise, return a non-zero value.
+ *
+ * FIXME: Return "target" just like refs_resolve_ref_unsafe(). This will be
+ * safe to do once we merge resolve_gitlink_ref() into master.
  */
 int resolve_gitlink_ref(const char *submodule, const char *refname,
-			struct object_id *oid);
+			struct object_id *oid, const char **target);
 
 /*
  * Return true iff abbrev_name is a possible abbreviation for
diff --git a/unpack-trees.c b/unpack-trees.c
index bae812156c4..db12bfcaffd 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -2288,7 +2288,8 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
 
 	if (S_ISGITLINK(ce->ce_mode)) {
 		struct object_id oid;
-		int sub_head = resolve_gitlink_ref(ce->name, "HEAD", &oid);
+		int sub_head =
+			resolve_gitlink_ref(ce->name, "HEAD", &oid, NULL);
 		/*
 		 * If we are not going to update the submodule, then
 		 * we don't care.
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH v3 6/8] submodule update: refactor update targets
  2022-10-28 20:14   ` [PATCH v3 0/8] " Glen Choo via GitGitGadget
                       ` (4 preceding siblings ...)
  2022-10-28 20:14     ` [PATCH v3 5/8] submodule: return target of submodule symref Glen Choo via GitGitGadget
@ 2022-10-28 20:14     ` Glen Choo via GitGitGadget
  2022-10-28 20:14     ` [PATCH v3 7/8] submodule--helper: remove update_data.suboid Glen Choo via GitGitGadget
                       ` (3 subsequent siblings)
  9 siblings, 0 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-28 20:14 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

Refactor two "git submodule update" code locations so that they no
longer refer to oids directly. This shrinks a subsequent commit's diff,
where this code will need to handle branches.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/submodule--helper.c | 24 +++++++++++++-----------
 1 file changed, 13 insertions(+), 11 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 9ca138374b7..894be133b3f 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2255,7 +2255,7 @@ static int fetch_in_submodule(const char *module_path, int depth, int quiet,
 static int run_update_command(const struct update_data *ud, int subforce)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
-	char *oid = oid_to_hex(&ud->oid);
+	const char *update_target = oid_to_hex(&ud->oid);;
 	int ret;
 
 	switch (ud->update_strategy.type) {
@@ -2285,7 +2285,7 @@ static int run_update_command(const struct update_data *ud, int subforce)
 		BUG("unexpected update strategy type: %d",
 		    ud->update_strategy.type);
 	}
-	strvec_push(&cp.args, oid);
+	strvec_push(&cp.args, update_target);
 
 	cp.dir = ud->sm_path;
 	prepare_submodule_repo_env(&cp.env);
@@ -2293,20 +2293,20 @@ static int run_update_command(const struct update_data *ud, int subforce)
 		switch (ud->update_strategy.type) {
 		case SM_UPDATE_CHECKOUT:
 			die_message(_("Unable to checkout '%s' in submodule path '%s'"),
-				    oid, ud->displaypath);
+				    update_target, ud->displaypath);
 			/* No "ret" assignment, use "git checkout"'s */
 			break;
 		case SM_UPDATE_REBASE:
 			ret = die_message(_("Unable to rebase '%s' in submodule path '%s'"),
-					  oid, ud->displaypath);
+					  update_target, ud->displaypath);
 			break;
 		case SM_UPDATE_MERGE:
 			ret = die_message(_("Unable to merge '%s' in submodule path '%s'"),
-					  oid, ud->displaypath);
+					  update_target, ud->displaypath);
 			break;
 		case SM_UPDATE_COMMAND:
 			ret = die_message(_("Execution of '%s %s' failed in submodule path '%s'"),
-					  ud->update_strategy.command, oid, ud->displaypath);
+					  ud->update_strategy.command, update_target, ud->displaypath);
 			break;
 		default:
 			BUG("unexpected update strategy type: %d",
@@ -2322,19 +2322,19 @@ static int run_update_command(const struct update_data *ud, int subforce)
 	switch (ud->update_strategy.type) {
 	case SM_UPDATE_CHECKOUT:
 		printf(_("Submodule path '%s': checked out '%s'\n"),
-		       ud->displaypath, oid);
+		       ud->displaypath, update_target);
 		break;
 	case SM_UPDATE_REBASE:
 		printf(_("Submodule path '%s': rebased into '%s'\n"),
-		       ud->displaypath, oid);
+		       ud->displaypath, update_target);
 		break;
 	case SM_UPDATE_MERGE:
 		printf(_("Submodule path '%s': merged in '%s'\n"),
-		       ud->displaypath, oid);
+		       ud->displaypath, update_target);
 		break;
 	case SM_UPDATE_COMMAND:
 		printf(_("Submodule path '%s': '%s %s'\n"),
-		       ud->displaypath, ud->update_strategy.command, oid);
+		       ud->displaypath, ud->update_strategy.command, update_target);
 		break;
 	default:
 		BUG("unexpected update strategy type: %d",
@@ -2521,6 +2521,7 @@ static void update_data_to_args(const struct update_data *update_data,
 
 static int update_submodule(struct update_data *update_data)
 {
+	int submodule_up_to_date;
 	int ret;
 
 	ret = determine_submodule_update_strategy(the_repository,
@@ -2569,7 +2570,8 @@ static int update_submodule(struct update_data *update_data)
 		free(remote_ref);
 	}
 
-	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force) {
+	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
+	if (!submodule_up_to_date || update_data->force) {
 		ret = run_update_procedure(update_data);
 		if (ret)
 			return ret;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH v3 7/8] submodule--helper: remove update_data.suboid
  2022-10-28 20:14   ` [PATCH v3 0/8] " Glen Choo via GitGitGadget
                       ` (5 preceding siblings ...)
  2022-10-28 20:14     ` [PATCH v3 6/8] submodule update: refactor update targets Glen Choo via GitGitGadget
@ 2022-10-28 20:14     ` Glen Choo via GitGitGadget
  2022-11-14 23:45       ` Jonathan Tan
  2022-10-28 20:14     ` [PATCH v3 8/8] clone, submodule update: create and check out branches Glen Choo via GitGitGadget
                       ` (2 subsequent siblings)
  9 siblings, 1 reply; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-28 20:14 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

update_data.suboid's value is only used in update_submodule()'s call
chain, where it represents the OID of the submodule's HEAD. If the
submodule is newly cloned, it is set to null_oid().

Instead of checking for the null OID, just check if the submodule is
newly cloned. This makes update_submodule() the only function where
update_data.suboid is used, so replace it with a local variable.

As a result, the submodule_up_to_date check is more explicit, which
makes the next commit slightly easier to reason about.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/submodule--helper.c | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 894be133b3f..ef76a111c7f 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1918,7 +1918,6 @@ struct update_data {
 	const char *prefix;
 	char *displaypath;
 	enum submodule_update_type update_default;
-	struct object_id suboid;
 	struct string_list references;
 	struct submodule_update_strategy update_strategy;
 	struct list_objects_filter_options *filter_options;
@@ -2346,7 +2345,7 @@ static int run_update_command(const struct update_data *ud, int subforce)
 
 static int run_update_procedure(const struct update_data *ud)
 {
-	int subforce = is_null_oid(&ud->suboid) || ud->force;
+	int subforce = ud->just_cloned || ud->force;
 
 	if (!ud->nofetch) {
 		/*
@@ -2523,6 +2522,7 @@ static int update_submodule(struct update_data *update_data)
 {
 	int submodule_up_to_date;
 	int ret;
+	struct object_id suboid;
 
 	ret = determine_submodule_update_strategy(the_repository,
 						  update_data->just_cloned,
@@ -2532,10 +2532,8 @@ static int update_submodule(struct update_data *update_data)
 	if (ret)
 		return ret;
 
-	if (update_data->just_cloned)
-		oidcpy(&update_data->suboid, null_oid());
-	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
-				     &update_data->suboid, NULL))
+	if (!update_data->just_cloned &&
+	    resolve_gitlink_ref(update_data->sm_path, "HEAD", &suboid, NULL))
 		return die_message(_("Unable to find current revision in submodule path '%s'"),
 				   update_data->displaypath);
 
@@ -2570,7 +2568,8 @@ static int update_submodule(struct update_data *update_data)
 		free(remote_ref);
 	}
 
-	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
+	submodule_up_to_date = !update_data->just_cloned &&
+		oideq(&update_data->oid, &suboid);
 	if (!submodule_up_to_date || update_data->force) {
 		ret = run_update_procedure(update_data);
 		if (ret)
@@ -2583,7 +2582,6 @@ static int update_submodule(struct update_data *update_data)
 
 		next.prefix = NULL;
 		oidcpy(&next.oid, null_oid());
-		oidcpy(&next.suboid, null_oid());
 
 		cp.dir = update_data->sm_path;
 		cp.git_cmd = 1;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [PATCH v3 8/8] clone, submodule update: create and check out branches
  2022-10-28 20:14   ` [PATCH v3 0/8] " Glen Choo via GitGitGadget
                       ` (6 preceding siblings ...)
  2022-10-28 20:14     ` [PATCH v3 7/8] submodule--helper: remove update_data.suboid Glen Choo via GitGitGadget
@ 2022-10-28 20:14     ` Glen Choo via GitGitGadget
  2022-11-08 13:53       ` Philippe Blain
  2022-11-15 18:15       ` Jonathan Tan
  2022-10-30 18:19     ` [PATCH v3 0/8] clone, submodule update: check out submodule branches Taylor Blau
  2022-11-08 14:23     ` Philippe Blain
  9 siblings, 2 replies; 56+ messages in thread
From: Glen Choo via GitGitGadget @ 2022-10-28 20:14 UTC (permalink / raw)
  To: git; +Cc: Philippe Blain, Jonathan Tan, Glen Choo, Glen Choo

From: Glen Choo <chooglen@google.com>

Teach "git submodule update" to:

- create the branch with the same name as the current superproject
  branch when cloning a submodule
- check out that branch (instead of the commit OID) when updating
  the submodule worktree

when submodule branching is enabled (submodule.propagateBranches = true)
on the superproject and a branch is checked out. "git clone
--recurse-submodules" also learns this trick because it is implemented
with "git submodule update --recursive".

This approach of checking out the branch will not result in a dirty
worktree for freshly cloned submodules because we can ensure that the
submodule branch points to the superproject gitlink. In other cases, it
does not work as well, but we can handle them incrementally:

- "git pull --recurse-submodules" merges the superproject tree,
  (changing the gitlink without updating the submodule branches), and
  runs "git submodule update" to update the worktrees, so it is almost
  guaranteed to result in a dirty worktree.

  The implementation of "git pull --recurse-submodules" is likely to
  change drastically as submodule.propagateBranches work progresses
  (e.g. "git merge" learns to recurse in to submodules), and we may be
  able to replace the "git submodule update" invocation, or teach it new
  tricks that make the update behave well.

- The user might make changes to the submodule branch without committing
  them back to superproject. This is primarily affects "git checkout
  --recurse-submodules", since that is the primary way of switching away
  from a branch and leaving behind WIP (as opposed to "git submodule
  update", which is run post-checkout).

  In a future series, "git checkout --recurse-submodules" will learn to
  consider submodule branches. We can introduce appropriate guardrails
  then, e.g. requiring that the superproject working tree is not dirty
  before switching away.

Signed-off-by: Glen Choo <chooglen@google.com>
---
 builtin/submodule--helper.c |  37 ++++++++-
 t/t5617-clone-submodules.sh |  38 +++++++++
 t/t7406-submodule-update.sh | 156 ++++++++++++++++++++++++++++++++++++
 3 files changed, 227 insertions(+), 4 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index ef76a111c7f..767a0c81cde 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1917,6 +1917,7 @@ static void submodule_update_clone_release(struct submodule_update_clone *suc)
 struct update_data {
 	const char *prefix;
 	char *displaypath;
+	const char *super_branch;
 	enum submodule_update_type update_default;
 	struct string_list references;
 	struct submodule_update_strategy update_strategy;
@@ -2091,6 +2092,11 @@ static int prepare_to_clone_next_submodule(const struct cache_entry *ce,
 		strvec_push(&child->args, suc->update_data->single_branch ?
 					      "--single-branch" :
 					      "--no-single-branch");
+	if (ud->super_branch) {
+		strvec_pushf(&child->args, "--branch=%s", ud->super_branch);
+		strvec_pushf(&child->args, "--branch-oid=%s",
+			     oid_to_hex(&ce->oid));
+	}
 
 cleanup:
 	free(displaypath);
@@ -2254,9 +2260,14 @@ static int fetch_in_submodule(const char *module_path, int depth, int quiet,
 static int run_update_command(const struct update_data *ud, int subforce)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *update_target = oid_to_hex(&ud->oid);;
+	const char *update_target;
 	int ret;
 
+	if (ud->update_strategy.type == SM_UPDATE_CHECKOUT && ud->super_branch)
+		update_target = ud->super_branch;
+	else
+		update_target = oid_to_hex(&ud->oid);
+
 	switch (ud->update_strategy.type) {
 	case SM_UPDATE_CHECKOUT:
 		cp.git_cmd = 1;
@@ -2523,6 +2534,7 @@ static int update_submodule(struct update_data *update_data)
 	int submodule_up_to_date;
 	int ret;
 	struct object_id suboid;
+	const char *submodule_head = NULL;
 
 	ret = determine_submodule_update_strategy(the_repository,
 						  update_data->just_cloned,
@@ -2533,7 +2545,8 @@ static int update_submodule(struct update_data *update_data)
 		return ret;
 
 	if (!update_data->just_cloned &&
-	    resolve_gitlink_ref(update_data->sm_path, "HEAD", &suboid, NULL))
+	    resolve_gitlink_ref(update_data->sm_path, "HEAD", &suboid,
+				&submodule_head))
 		return die_message(_("Unable to find current revision in submodule path '%s'"),
 				   update_data->displaypath);
 
@@ -2568,8 +2581,17 @@ static int update_submodule(struct update_data *update_data)
 		free(remote_ref);
 	}
 
-	submodule_up_to_date = !update_data->just_cloned &&
-		oideq(&update_data->oid, &suboid);
+	if (update_data->just_cloned)
+		submodule_up_to_date = 0;
+	else if (update_data->super_branch)
+		/* Check that the submodule's HEAD points to super_branch. */
+		submodule_up_to_date =
+			skip_prefix(submodule_head, "refs/heads/",
+				    &submodule_head) &&
+			!strcmp(update_data->super_branch, submodule_head);
+	else
+		submodule_up_to_date = oideq(&update_data->oid, &suboid);
+
 	if (!submodule_up_to_date || update_data->force) {
 		ret = run_update_procedure(update_data);
 		if (ret)
@@ -2603,6 +2625,12 @@ static int update_submodules(struct update_data *update_data)
 	int i, ret = 0;
 	struct submodule_update_clone suc = SUBMODULE_UPDATE_CLONE_INIT;
 
+	if (the_repository->settings.submodule_propagate_branches) {
+		struct branch *current_branch = branch_get(NULL);
+		if (current_branch)
+			update_data->super_branch = current_branch->name;
+	}
+
 	suc.update_data = update_data;
 	run_processes_parallel_tr2(suc.update_data->max_jobs, update_clone_get_next_task,
 				   update_clone_start_failure,
@@ -2718,6 +2746,7 @@ static int module_update(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, module_update_options,
 			     git_submodule_helper_usage, 0);
+	prepare_repo_settings(the_repository);
 
 	if (opt.require_init)
 		opt.init = 1;
diff --git a/t/t5617-clone-submodules.sh b/t/t5617-clone-submodules.sh
index c43a5b26fab..43f9b52bd44 100755
--- a/t/t5617-clone-submodules.sh
+++ b/t/t5617-clone-submodules.sh
@@ -13,10 +13,17 @@ test_expect_success 'setup' '
 	git config --global protocol.file.allow always &&
 	git checkout -b main &&
 	test_commit commit1 &&
+	mkdir subsub &&
+	(
+		cd subsub &&
+		git init &&
+		test_commit subsubcommit1
+	) &&
 	mkdir sub &&
 	(
 		cd sub &&
 		git init &&
+		git submodule add "file://$pwd/subsub" subsub &&
 		test_commit subcommit1 &&
 		git tag sub_when_added_to_super &&
 		git branch other
@@ -107,4 +114,35 @@ test_expect_success '--no-also-filter-submodules overrides clone.filterSubmodule
 	test_cmp_config -C super_clone3/sub false --default false remote.origin.promisor
 '
 
+test_expect_success 'submodule.propagateBranches checks out branches at correct commits' '
+	test_when_finished "git checkout main" &&
+
+	git checkout -b checked-out &&
+	git -C sub checkout -b not-in-clone &&
+	git -C subsub checkout -b not-in-clone &&
+	git clone --recurse-submodules \
+		--branch checked-out \
+		-c submodule.propagateBranches=true \
+		"file://$pwd/." super_clone4 &&
+
+	# Assert that each repo is pointing to "checked-out"
+	for REPO in "super_clone4" "super_clone4/sub" "super_clone4/sub/subsub"
+	do
+	    HEAD_BRANCH=$(git -C $REPO symbolic-ref HEAD) &&
+	    test $HEAD_BRANCH = "refs/heads/checked-out" || return 1
+	done &&
+
+	# Assert that the submodule branches are pointing to the right revs
+	EXPECT_SUB_OID="$(git -C super_clone4 rev-parse :sub)" &&
+	ACTUAL_SUB_OID="$(git -C super_clone4/sub rev-parse refs/heads/checked-out)" &&
+	test $EXPECT_SUB_OID = $ACTUAL_SUB_OID &&
+	EXPECT_SUBSUB_OID="$(git -C super_clone4/sub rev-parse :subsub)" &&
+	ACTUAL_SUBSUB_OID="$(git -C super_clone4/sub/subsub rev-parse refs/heads/checked-out)" &&
+	test $EXPECT_SUBSUB_OID = $ACTUAL_SUBSUB_OID &&
+
+	# Assert that the submodules do not have branches from their upstream
+	test_must_fail git -C super_clone4/sub rev-parse not-in-clone &&
+	test_must_fail git -C super_clone4/sub/subsub rev-parse not-in-clone
+'
+
 test_done
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index f094e3d7f36..b749d35f784 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -1179,4 +1179,160 @@ test_expect_success 'submodule update --recursive skip submodules with strategy=
 	test_cmp expect.err actual.err
 '
 
+test_expect_success 'setup superproject with submodule.propagateBranches' '
+	git init sub1 &&
+	test_commit -C sub1 "sub1" &&
+	git init branch-super &&
+	git -C branch-super submodule add ../sub1 sub1 &&
+	git -C branch-super commit -m "super" &&
+
+	# Clone into a clean repo that we can cp around
+	git clone --recurse-submodules \
+		-c submodule.propagateBranches=true \
+		branch-super branch-super-clean &&
+	git -C branch-super-clean config submodule.propagateBranches true &&
+
+	# sub2 will not be in the clone. We will fetch the containing
+	# superproject commit and clone sub2 with "git submodule update".
+	git init sub2 &&
+	test_commit -C sub2 "sub2" &&
+	git -C branch-super submodule add ../sub2 sub2 &&
+	git -C branch-super commit -m "add sub2"
+'
+
+test_clean_submodule ()
+{
+	local negate super_dir sub_dir expect_oid actual_oid &&
+	if test "$1" = "!"
+	then
+		negate=t
+		shift
+	fi
+	super_dir="$1" &&
+	sub_dir="$2" &&
+	expect_oid="$(git -C "$super_dir" rev-parse ":$sub_dir")" &&
+	actual_oid="$(git -C "$super_dir/$sub_dir" rev-parse HEAD)" &&
+	if test -n "$negate"
+	then
+		! test "$expect_oid" = "$actual_oid"
+	else
+		test "$expect_oid" = "$actual_oid"
+	fi
+}
+
+# Test the behavior of a newly cloned submodule
+test_expect_success 'branches - newly-cloned submodule, detached HEAD' '
+	test_when_finished "rm -fr branch-super-cloned" &&
+	cp -r branch-super-clean branch-super-cloned &&
+
+	git -C branch-super-cloned fetch origin main &&
+	git -C branch-super-cloned checkout FETCH_HEAD &&
+	git -C branch-super-cloned/sub1 checkout --detach &&
+	git -C branch-super-cloned submodule update &&
+
+	# sub1 and sub2 should be in detached HEAD
+	git -C branch-super-cloned/sub1 rev-parse --verify HEAD &&
+	test_must_fail git -C branch-super-cloned/sub1 symbolic-ref HEAD &&
+	test_clean_submodule branch-super-cloned sub1 &&
+	git -C branch-super-cloned/sub2 rev-parse --verify HEAD &&
+	test_must_fail git -C branch-super-cloned/sub2 symbolic-ref HEAD &&
+	test_clean_submodule branch-super-cloned sub2
+'
+
+test_expect_success 'branches - newly-cloned submodule, branch checked out' '
+	test_when_finished "rm -fr branch-super-cloned" &&
+	cp -r branch-super-clean branch-super-cloned &&
+
+	git -C branch-super-cloned fetch origin main &&
+	git -C branch-super-cloned checkout FETCH_HEAD &&
+	git -C branch-super-cloned branch new-branch &&
+	git -C branch-super-cloned checkout new-branch &&
+	git -C branch-super-cloned/sub1 branch new-branch &&
+	git -C branch-super-cloned submodule update &&
+
+	# Ignore sub1, we will test it later.
+	# sub2 should check out the branch
+	HEAD_BRANCH2=$(git -C branch-super-cloned/sub2 symbolic-ref HEAD) &&
+	test $HEAD_BRANCH2 = "refs/heads/new-branch" &&
+	test_clean_submodule branch-super-cloned sub2
+'
+
+# Test the behavior of an already-cloned submodule.
+# NEEDSWORK When updating with branches, we always use the branch instead of the
+# gitlink's OID. This results in some imperfect behavior:
+#
+# - If the gitlink's OID disagrees with the branch OID, updating with branches
+#   may result in a dirty worktree
+# - If the branch does not exist, the update fails.
+#
+# We will reevaluate when "git checkout --recurse-submodules" supports branches
+# For now, just test for this imperfect behavior.
+test_expect_success 'branches - correct branch checked out, OIDs agree' '
+	test_when_finished "rm -fr branch-super-cloned" &&
+	cp -r branch-super-clean branch-super-cloned &&
+
+	git -C branch-super-cloned branch --recurse-submodules new-branch &&
+	git -C branch-super-cloned checkout new-branch &&
+	git -C branch-super-cloned/sub1 checkout new-branch &&
+	git -C branch-super-cloned submodule update &&
+
+	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
+	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
+	test_clean_submodule branch-super-cloned sub1
+'
+
+test_expect_success 'branches - correct branch checked out, OIDs disagree' '
+	test_when_finished "rm -fr branch-super-cloned" &&
+	cp -r branch-super-clean branch-super-cloned &&
+
+	git -C branch-super-cloned branch --recurse-submodules new-branch &&
+	git -C branch-super-cloned checkout new-branch &&
+	git -C branch-super-cloned/sub1 checkout new-branch &&
+	test_commit -C branch-super-cloned/sub1 new-commit &&
+	git -C branch-super-cloned submodule update &&
+
+	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
+	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
+	test_clean_submodule ! branch-super-cloned sub1
+'
+
+test_expect_success 'branches - other branch checked out, correct branch exists, OIDs agree' '
+	test_when_finished "rm -fr branch-super-cloned" &&
+	cp -r branch-super-clean branch-super-cloned &&
+
+	git -C branch-super-cloned branch --recurse-submodules new-branch &&
+	git -C branch-super-cloned checkout new-branch &&
+	git -C branch-super-cloned/sub1 checkout main &&
+	git -C branch-super-cloned submodule update &&
+
+	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
+	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
+	test_clean_submodule branch-super-cloned sub1
+'
+
+test_expect_success 'branches - other branch checked out, correct branch exists, OIDs disagree' '
+	test_when_finished "rm -fr branch-super-cloned" &&
+	cp -r branch-super-clean branch-super-cloned &&
+
+	git -C branch-super-cloned branch --recurse-submodules new-branch &&
+	git -C branch-super-cloned checkout new-branch &&
+	git -C branch-super-cloned/sub1 checkout new-branch &&
+	test_commit -C branch-super-cloned/sub1 new-commit &&
+	git -C branch-super-cloned/sub1 checkout main &&
+	git -C branch-super-cloned submodule update &&
+
+	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
+	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
+	test_clean_submodule ! branch-super-cloned sub1
+'
+
+test_expect_success 'branches - other branch checked out, correct branch does not exist' '
+	test_when_finished "rm -fr branch-super-cloned" &&
+	cp -r branch-super-clean branch-super-cloned &&
+
+	git -C branch-super-cloned branch new-branch &&
+	git -C branch-super-cloned checkout new-branch &&
+	test_must_fail git -C branch-super-cloned submodule update
+'
+
 test_done
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 1/8] clone: teach --detach option
  2022-10-28 20:14     ` [PATCH v3 1/8] clone: teach --detach option Glen Choo via GitGitGadget
@ 2022-10-28 21:40       ` Junio C Hamano
  2022-10-28 21:54         ` Junio C Hamano
  2022-11-08 13:32       ` Philippe Blain
  1 sibling, 1 reply; 56+ messages in thread
From: Junio C Hamano @ 2022-10-28 21:40 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget; +Cc: git, Philippe Blain, Jonathan Tan, Glen Choo

"Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Glen Choo <chooglen@google.com>
>
> Teach "git clone" the "--detach" option, which leaves the cloned repo in
> detached HEAD (like "git checkout --detach"). In addition, if the clone
> is not bare, do not create the local branch pointed to by the remote's
> HEAD symref (bare clones always copy all remote branches directly to
> local branches, so the branch is still created in the bare case).
>
> This is especially useful in the "submodule.propagateBranches" workflow,
> where local submodule branches are named after the superproject's
> branches, so it makes no sense to create a local branch named after the
> submodule's remote's branch.

Wouldn't it the same thing to do "git clone -n && git checkout
--detach"?  If this is a pure implementation detail of another
command and will never be used directly by end users, I am not
sure if we should add a new option to do this.

Especially because it is probably not hard to perform internally an
equivalent of "checkout --detach" without forking these days,
judging from the fact that merges and rebases need to do so.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 5/8] submodule: return target of submodule symref
  2022-10-28 20:14     ` [PATCH v3 5/8] submodule: return target of submodule symref Glen Choo via GitGitGadget
@ 2022-10-28 21:49       ` Junio C Hamano
  2022-10-28 23:11         ` Glen Choo
  0 siblings, 1 reply; 56+ messages in thread
From: Junio C Hamano @ 2022-10-28 21:49 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget; +Cc: git, Philippe Blain, Jonathan Tan, Glen Choo

"Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:

>  int resolve_gitlink_ref(const char *submodule, const char *refname,
> -			struct object_id *oid)
> +			struct object_id *oid, const char **target_out)
>  {
>  	struct ref_store *refs;
>  	int flags;
> +	const char *target;
>  
>  	refs = get_submodule_ref_store(submodule);
>  
>  	if (!refs)
>  		return -1;
> -
> -	if (!refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags) ||
> -	    is_null_oid(oid))
> +	target = refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags);
> +	if (!target || is_null_oid(oid))
>  		return -1;
> +	if (target_out)
> +		*target_out = target;
>  	return 0;
>  }

Please remind me why we call this underlying helper _unsafe()?

Isn't it because we return a temporary buffer  (static strbuf),
whose contents is not permanent?

I am wondering if we should force the callers who care enough to
pass the optional **target_out to xstrdup() the result.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 1/8] clone: teach --detach option
  2022-10-28 21:40       ` Junio C Hamano
@ 2022-10-28 21:54         ` Junio C Hamano
  2022-10-28 22:55           ` Glen Choo
  0 siblings, 1 reply; 56+ messages in thread
From: Junio C Hamano @ 2022-10-28 21:54 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget; +Cc: git, Philippe Blain, Jonathan Tan, Glen Choo

Junio C Hamano <gitster@pobox.com> writes:

> Wouldn't it the same thing to do "git clone -n && git checkout
> --detach"?

Not exactly.  It still creates the initial branch and points HEAD at
it.  It is so close, which is a bit disappointing, though.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 1/8] clone: teach --detach option
  2022-10-28 21:54         ` Junio C Hamano
@ 2022-10-28 22:55           ` Glen Choo
  2022-10-30 18:14             ` Taylor Blau
  0 siblings, 1 reply; 56+ messages in thread
From: Glen Choo @ 2022-10-28 22:55 UTC (permalink / raw)
  To: Junio C Hamano, Glen Choo via GitGitGadget
  Cc: git, Philippe Blain, Jonathan Tan

Junio C Hamano <gitster@pobox.com> writes:

> Junio C Hamano <gitster@pobox.com> writes:
>
>> Wouldn't it the same thing to do "git clone -n && git checkout
>> --detach"?
>
> Not exactly.  It still creates the initial branch and points HEAD at
> it.  It is so close, which is a bit disappointing, though.

Yes. I was quite hopeful that "git clone -n" would just do the right
thing too :/

I'm guessing that changing the behavior of "git clone -n" is a
non-option, since there may be users relying on it.

So a better way forward is to add the new flag, which I imagine might be
useful to certain end users.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 5/8] submodule: return target of submodule symref
  2022-10-28 21:49       ` Junio C Hamano
@ 2022-10-28 23:11         ` Glen Choo
  0 siblings, 0 replies; 56+ messages in thread
From: Glen Choo @ 2022-10-28 23:11 UTC (permalink / raw)
  To: Junio C Hamano, Glen Choo via GitGitGadget
  Cc: git, Philippe Blain, Jonathan Tan

Junio C Hamano <gitster@pobox.com> writes:

> "Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>>  int resolve_gitlink_ref(const char *submodule, const char *refname,
>> -			struct object_id *oid)
>> +			struct object_id *oid, const char **target_out)
>>  {
>>  	struct ref_store *refs;
>>  	int flags;
>> +	const char *target;
>>  
>>  	refs = get_submodule_ref_store(submodule);
>>  
>>  	if (!refs)
>>  		return -1;
>> -
>> -	if (!refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags) ||
>> -	    is_null_oid(oid))
>> +	target = refs_resolve_ref_unsafe(refs, refname, 0, oid, &flags);
>> +	if (!target || is_null_oid(oid))
>>  		return -1;
>> +	if (target_out)
>> +		*target_out = target;
>>  	return 0;
>>  }
>
> Please remind me why we call this underlying helper _unsafe()?
>
> Isn't it because we return a temporary buffer  (static strbuf),
> whose contents is not permanent?

8cad4744ee (Rename resolve_ref() to resolve_ref_unsafe(), 2011-12-12)
seems to suggest so.

(For some reason, I thought it was *_unsafe() because it would die(),
but it obviously doesn't.)

> I am wondering if we should force the callers who care enough to
> pass the optional **target_out to xstrdup() the result.

Yes, makes sense. We even have the *_refdup() helper for that.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 1/8] clone: teach --detach option
  2022-10-28 22:55           ` Glen Choo
@ 2022-10-30 18:14             ` Taylor Blau
  2022-10-31 17:07               ` Glen Choo
  0 siblings, 1 reply; 56+ messages in thread
From: Taylor Blau @ 2022-10-30 18:14 UTC (permalink / raw)
  To: Glen Choo
  Cc: Junio C Hamano, Glen Choo via GitGitGadget, git, Philippe Blain,
	Jonathan Tan

On Fri, Oct 28, 2022 at 03:55:25PM -0700, Glen Choo wrote:
> So a better way forward is to add the new flag, which I imagine might
> be useful to certain end users.

Disappointing, though I understand why such a new flag was needed. Do we
really care about whether or not the branch exists so long as we are
detached from it, though?

Thanks,
Taylor

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 0/8] clone, submodule update: check out submodule branches
  2022-10-28 20:14   ` [PATCH v3 0/8] " Glen Choo via GitGitGadget
                       ` (7 preceding siblings ...)
  2022-10-28 20:14     ` [PATCH v3 8/8] clone, submodule update: create and check out branches Glen Choo via GitGitGadget
@ 2022-10-30 18:19     ` Taylor Blau
  2022-11-08 14:23     ` Philippe Blain
  9 siblings, 0 replies; 56+ messages in thread
From: Taylor Blau @ 2022-10-30 18:19 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget; +Cc: git, Philippe Blain, Jonathan Tan, Glen Choo

On Fri, Oct 28, 2022 at 08:14:48PM +0000, Glen Choo via GitGitGadget wrote:
> This version has relatively few changes, and should address all of
> Jonathan's comments (thanks!).

Updated, thanks. I haven't read the topic closely, but am curious
whether other reviewers are happy with this round so far.

Thanks,
Taylor

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 1/8] clone: teach --detach option
  2022-10-30 18:14             ` Taylor Blau
@ 2022-10-31 17:07               ` Glen Choo
  0 siblings, 0 replies; 56+ messages in thread
From: Glen Choo @ 2022-10-31 17:07 UTC (permalink / raw)
  To: Taylor Blau
  Cc: Junio C Hamano, Glen Choo via GitGitGadget, git, Philippe Blain,
	Jonathan Tan

Taylor Blau <me@ttaylorr.com> writes:

> On Fri, Oct 28, 2022 at 03:55:25PM -0700, Glen Choo wrote:
>> So a better way forward is to add the new flag, which I imagine might
>> be useful to certain end users.
>
> Disappointing, though I understand why such a new flag was needed. Do we
> really care about whether or not the branch exists so long as we are
> detached from it, though?

Yes.

- With submodule branching, the "main" branch should correspond to the
  gitlink of the superproject's "main" branch. So when we clone, we
  can't _already_ have a "main" branch coming from the submodule's
  remote.
- Without submodule branching, submodules are always in detached HEAD
  (e.g. when updating the worktree recursively) and no submodule
  recursing functions create branches, _except_ "git clone
  --recurse-submodules" (which as we've seen, may create the branch
  corresponding to the submodule's remote). This just looks like an
  oversight IMO, which is why I noted that even without branching, "git
  clone --recurse-submodules" should probably also use "--detach" [1].
- Outside of submodules, I can imagine there's at least one person who's
  performed a clone and then "git branch -D master" (maybe followed by
  "git checkout -b main"), and "git clone --detach" lets them skip the
  branch deletion.

[1] https://lore.kernel.org/git/5a24d7e9255de407e343ce8bd60edb63293505bb.1666988096.git.gitgitgadget@gmail.com

>
> Thanks,
> Taylor

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 1/8] clone: teach --detach option
  2022-10-28 20:14     ` [PATCH v3 1/8] clone: teach --detach option Glen Choo via GitGitGadget
  2022-10-28 21:40       ` Junio C Hamano
@ 2022-11-08 13:32       ` Philippe Blain
  1 sibling, 0 replies; 56+ messages in thread
From: Philippe Blain @ 2022-11-08 13:32 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget, git; +Cc: Jonathan Tan, Glen Choo

Hi Glen,

Le 2022-10-28 à 16:14, Glen Choo via GitGitGadget a écrit :
> From: Glen Choo <chooglen@google.com>
> 
> Teach "git clone" the "--detach" option, which leaves the cloned repo in
> detached HEAD (like "git checkout --detach"). In addition, if the clone
> is not bare, do not create the local branch pointed to by the remote's
> HEAD symref (bare clones always copy all remote branches directly to
> local branches, so the branch is still created in the bare case).
> 
> This is especially useful in the "submodule.propagateBranches" workflow,
> where local submodule branches are named after the superproject's
> branches, so it makes no sense to create a local branch named after the
> submodule's remote's branch.
> 
> Signed-off-by: Glen Choo <chooglen@google.com>
> ---
>  Documentation/git-clone.txt |  8 +++++++-
>  builtin/clone.c             | 12 +++++++++---
>  t/t5601-clone.sh            | 22 ++++++++++++++++++++++
>  3 files changed, 38 insertions(+), 4 deletions(-)
> 
> diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
> index d6434d262d6..6a4e5d31b46 100644
> --- a/Documentation/git-clone.txt
> +++ b/Documentation/git-clone.txt
> @@ -16,7 +16,7 @@ SYNOPSIS
>  	  [--depth <depth>] [--[no-]single-branch] [--no-tags]
>  	  [--recurse-submodules[=<pathspec>]] [--[no-]shallow-submodules]
>  	  [--[no-]remote-submodules] [--jobs <n>] [--sparse] [--[no-]reject-shallow]
> -	  [--filter=<filter> [--also-filter-submodules]] [--] <repository>
> +	  [--filter=<filter> [--also-filter-submodules] [--detach]] [--] <repository>
>  	  [<directory>]
>  
>  DESCRIPTION
> @@ -210,6 +210,12 @@ objects from the source repository into a pack in the cloned repository.
>  	`--branch` can also take tags and detaches the HEAD at that commit
>  	in the resulting repository.
>  
> +--detach::
> +	If the cloned repository's HEAD points to a branch, point the newly
> +	created HEAD to the branch's commit instead of the branch itself.
> +	Additionally, in a non-bare repository, the corresponding local branch
> +	will not be created.
> +


"point the newly created HEAD to the branch's tip commit"
would be slightly clearer, I think.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 8/8] clone, submodule update: create and check out branches
  2022-10-28 20:14     ` [PATCH v3 8/8] clone, submodule update: create and check out branches Glen Choo via GitGitGadget
@ 2022-11-08 13:53       ` Philippe Blain
  2022-11-15 18:15       ` Jonathan Tan
  1 sibling, 0 replies; 56+ messages in thread
From: Philippe Blain @ 2022-11-08 13:53 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget, git; +Cc: Jonathan Tan, Glen Choo

Hi Glen,

Le 2022-10-28 à 16:14, Glen Choo via GitGitGadget a écrit :
> From: Glen Choo <chooglen@google.com>
> 
> Teach "git submodule update" to:
> 
> - create the branch with the same name as the current superproject
>   branch when cloning a submodule
> - check out that branch (instead of the commit OID) when updating
>   the submodule worktree
> 
> when submodule branching is enabled (submodule.propagateBranches = true)
> on the superproject and a branch is checked out. "git clone
> --recurse-submodules" also learns this trick because it is implemented
> with "git submodule update --recursive".
> 
> This approach of checking out the branch will not result in a dirty
> worktree for freshly cloned submodules because we can ensure that the
> submodule branch points to the superproject gitlink. In other cases, it
> does not work as well, but we can handle them incrementally:
> 
> - "git pull --recurse-submodules" merges the superproject tree,
>   (changing the gitlink without updating the submodule branches), and
>   runs "git submodule update" to update the worktrees, so it is almost
>   guaranteed to result in a dirty worktree.

OK, here you mean that 'git -c submodule.propagateBranches=true submodule update'
would just checkout the submodule branch (or do nothing if it's already checked out),
but that branch would most likely not be up to date with the newly updated gitlink
in the superproject, resulting in a dirty worktree.

> 
>   The implementation of "git pull --recurse-submodules" is likely to
>   change drastically as submodule.propagateBranches work progresses
>   (e.g. "git merge" learns to recurse in to submodules), and we may be
>   able to replace the "git submodule update" invocation, or teach it new
>   tricks that make the update behave well.
> 
> - The user might make changes to the submodule branch without committing
>   them back to superproject. This is primarily affects "git checkout
>   --recurse-submodules", since that is the primary way of switching away
>   from a branch and leaving behind WIP (as opposed to "git submodule
>   update", which is run post-checkout).
> 
>   In a future series, "git checkout --recurse-submodules" will learn to
>   consider submodule branches. We can introduce appropriate guardrails
>   then, e.g. requiring that the superproject working tree is not dirty
>   before switching away.
> 
> Signed-off-by: Glen Choo <chooglen@google.com>
> ---
>  builtin/submodule--helper.c |  37 ++++++++-
>  t/t5617-clone-submodules.sh |  38 +++++++++
>  t/t7406-submodule-update.sh | 156 ++++++++++++++++++++++++++++++++++++
>  3 files changed, 227 insertions(+), 4 deletions(-)
> 

--8<--

> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
> index f094e3d7f36..b749d35f784 100755
> --- a/t/t7406-submodule-update.sh
> +++ b/t/t7406-submodule-update.sh

--8<--

> +# Test the behavior of an already-cloned submodule.
> +# NEEDSWORK When updating with branches, we always use the branch instead of the
> +# gitlink's OID. This results in some imperfect behavior:
> +#
> +# - If the gitlink's OID disagrees with the branch OID, updating with branches
> +#   may result in a dirty worktree
> +# - If the branch does not exist, the update fails.
> +#
> +# We will reevaluate when "git checkout --recurse-submodules" supports branches
> +# For now, just test for this imperfect behavior.

OK. Maybe it would be a good idea to explicitely flag which tests below have
an imperfect behaviour ? This way it's easier to spot where the NEEDSWORK applies...

--8<--

This following test show the imperfect behaviour:

> +test_expect_success 'branches - correct branch checked out, OIDs disagree' '
> +	test_when_finished "rm -fr branch-super-cloned" &&
> +	cp -r branch-super-clean branch-super-cloned &&
> +
> +	git -C branch-super-cloned branch --recurse-submodules new-branch &&
> +	git -C branch-super-cloned checkout new-branch &&
> +	git -C branch-super-cloned/sub1 checkout new-branch &&
> +	test_commit -C branch-super-cloned/sub1 new-commit &&
> +	git -C branch-super-cloned submodule update &&
> +
> +	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
> +	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
> +	test_clean_submodule ! branch-super-cloned sub1
> +'

--8<--

This one also:

> +
> +test_expect_success 'branches - other branch checked out, correct branch exists, OIDs disagree' '
> +	test_when_finished "rm -fr branch-super-cloned" &&
> +	cp -r branch-super-clean branch-super-cloned &&
> +
> +	git -C branch-super-cloned branch --recurse-submodules new-branch &&
> +	git -C branch-super-cloned checkout new-branch &&
> +	git -C branch-super-cloned/sub1 checkout new-branch &&
> +	test_commit -C branch-super-cloned/sub1 new-commit &&
> +	git -C branch-super-cloned/sub1 checkout main &&
> +	git -C branch-super-cloned submodule update &&
> +
> +	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
> +	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
> +	test_clean_submodule ! branch-super-cloned sub1
> +'
> +

This one I'm not sure about: with this series 'git submodule update' does not create
submodule branches at other times than at clone time, so I think this behviour would not change.
I think this one is really an edge case where the user mixes submodule branching on/off.

> +test_expect_success 'branches - other branch checked out, correct branch does not exist' '
> +	test_when_finished "rm -fr branch-super-cloned" &&
> +	cp -r branch-super-clean branch-super-cloned &&
> +
> +	git -C branch-super-cloned branch new-branch &&
> +	git -C branch-super-cloned checkout new-branch &&
> +	test_must_fail git -C branch-super-cloned submodule update
> +'
> +
>  test_done

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 0/8] clone, submodule update: check out submodule branches
  2022-10-28 20:14   ` [PATCH v3 0/8] " Glen Choo via GitGitGadget
                       ` (8 preceding siblings ...)
  2022-10-30 18:19     ` [PATCH v3 0/8] clone, submodule update: check out submodule branches Taylor Blau
@ 2022-11-08 14:23     ` Philippe Blain
  2022-11-08 20:43       ` Glen Choo
  9 siblings, 1 reply; 56+ messages in thread
From: Philippe Blain @ 2022-11-08 14:23 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget, git; +Cc: Jonathan Tan, Glen Choo

Hi Glen,

Le 2022-10-28 à 16:14, Glen Choo via GitGitGadget a écrit :
> This version has relatively few changes, and should address all of
> Jonathan's comments (thanks!).
> 

I was not able to take a look before now, but I think the suggestions by Jonathan
on v2 make a lot of sense, especially adding more tests in the last patch.
Thanks for these additional tests.

I have a few comments/questions on the overall design, which I'll write up 
at the end of this reply since they are more general.
> = Description
> 
> This series teaches "git clone --recurse-submodules" and "git submodule
> update" to understand "submodule.propagateBranches" (see Further Reading for
> context), i.e. if the superproject has a branch checked out and a submodule
> is cloned, the submodule will have the same branch checked out.
> 
> To do this, "git submodule update" checks if "submodule.propagateBranches"
> is true. If so, and if the superproject has the branch 'topic' checked out,
> then:
> 
>  * Submodules are cloned without their upstream branches
>  * The 'topic' branch is created in the submodule
>  * The submodule is updated via "git checkout topic" instead of checking out
>    the gitlink's OID.
> 

Currently, the description of submodule.propagateBranches is:

    [EXPERIMENTAL] A boolean that enables branching support when 
    using --recurse-submodules or submodule.recurse=true. Enabling this 
    will allow certain commands to accept --recurse-submodules and certain 
    commands that already accept --recurse-submodules will now consider branches. 
    Defaults to false.

I think with this series that description must be tweaked, because "git submodule update"
does not qualify as a command that "now accepts --recurse-submodules", neither does
it "already accept --recurse-submodules" but now changes behaviour to consider branches.

It does change behaviour to "now consider branches", but never had anything to do with
"--recurse-submodules".

--8<--

> 
> = Future work
> 
>  * Patch 5, which refactors resolve_gitlink_ref(), notes that a better
>    interface would be to return the refname instead of using an "out"
>    parameter, but we use an "out" parameter so that any new callers trying
>    to use the old function signature will get stopped by the compiler. The
>    refactor can be finished at a later time.
> 
>  * Patch 5 uses the name "target" when we are talking about what a symref
>    points to, instead of "referent" like the other functions. "target" is
>    the better choice, since "referent" could also apply to non-symbolic
>    refs, but that cleanup is quite big.
> 
>  * Patch 8 notes that for already cloned submodules, the branch may not
>    point to the same OID as the superproject gitlink, and it may not even
>    exist. This will be addressed in a more comprehensive manner when we add
>    support for checking out branches with "git checkout
>    --recurse-submodules".


A few points I thought about while reading this version:

1. There is always a possibility that the branch name in the superproject already exists
in the submodule remote, but is a completely different topic (think of a branch named "refactor"
for example). With this series (and propagateBranches=true), this would mean that 
the initial submodule clone would create a local branch "refactor" that points to the gitlink
in the superproject, and a remote-tracking branch "origin/refactor" that points to the unrelated
submodule topic branch from the submodule remote. This could be confusing... but I don't
really know what Git could do about it !

In patch 3/8 'git branch' is used to create the submodule branch from an oid, and so 
it does not track any branch, and is not affected by 'branch.autoSetupMerge' as far as I 
could test. But maybe this should be explicitely mentioned somewhere? 

2. The new "git submodule update" behaviour seems to only make sense with "--checkout", 
which is the default "git submodue update" mode. But what if one uses "git submodule
update --merge", or "--rebase", or has submodule.<name>.update set to a custom command
or "none" ? Is the idea that these modes are incompatible with submodule branching ?
I can understand that this gets really complex and might change when 'git merge' and 
'git rebase' themselves are taught to update submodule worktrees (and probably also submodule
branches from what you refer to as future work), but in any case I think we should at 
least test the new code when these options are used (if only to error out outright as
incompatible)...

Thanks and cheers,

Philippe.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 0/8] clone, submodule update: check out submodule branches
  2022-11-08 14:23     ` Philippe Blain
@ 2022-11-08 20:43       ` Glen Choo
  0 siblings, 0 replies; 56+ messages in thread
From: Glen Choo @ 2022-11-08 20:43 UTC (permalink / raw)
  To: Philippe Blain, Glen Choo via GitGitGadget, git; +Cc: Jonathan Tan

Hi Philippe.

Thanks for spending time on the high level ideas (and not just the
code); it really helps to keep this logical and consistent.

Philippe Blain <levraiphilippeblain@gmail.com> writes:

>> = Description
>> 
>> This series teaches "git clone --recurse-submodules" and "git submodule
>> update" to understand "submodule.propagateBranches" (see Further Reading for
>> context), i.e. if the superproject has a branch checked out and a submodule
>> is cloned, the submodule will have the same branch checked out.
>> 
>> To do this, "git submodule update" checks if "submodule.propagateBranches"
>> is true. If so, and if the superproject has the branch 'topic' checked out,
>> then:
>> 
>>  * Submodules are cloned without their upstream branches
>>  * The 'topic' branch is created in the submodule
>>  * The submodule is updated via "git checkout topic" instead of checking out
>>    the gitlink's OID.
>> 
>
> Currently, the description of submodule.propagateBranches is:
>
>     [EXPERIMENTAL] A boolean that enables branching support when 
>     using --recurse-submodules or submodule.recurse=true. Enabling this 
>     will allow certain commands to accept --recurse-submodules and certain 
>     commands that already accept --recurse-submodules will now consider branches. 
>     Defaults to false.
>
> I think with this series that description must be tweaked, because "git submodule update"
> does not qualify as a command that "now accepts --recurse-submodules", neither does
> it "already accept --recurse-submodules" but now changes behaviour to consider branches.
>
> It does change behaviour to "now consider branches", but never had anything to do with
> "--recurse-submodules".

Yes that's a good point, thanks.

>
> --8<--
>
>> 
>> = Future work
>> 
>>  * Patch 5, which refactors resolve_gitlink_ref(), notes that a better
>>    interface would be to return the refname instead of using an "out"
>>    parameter, but we use an "out" parameter so that any new callers trying
>>    to use the old function signature will get stopped by the compiler. The
>>    refactor can be finished at a later time.
>> 
>>  * Patch 5 uses the name "target" when we are talking about what a symref
>>    points to, instead of "referent" like the other functions. "target" is
>>    the better choice, since "referent" could also apply to non-symbolic
>>    refs, but that cleanup is quite big.
>> 
>>  * Patch 8 notes that for already cloned submodules, the branch may not
>>    point to the same OID as the superproject gitlink, and it may not even
>>    exist. This will be addressed in a more comprehensive manner when we add
>>    support for checking out branches with "git checkout
>>    --recurse-submodules".
>
>
> A few points I thought about while reading this version:

This is a good reminder for me to check in that doc I promised that
describes how branches in submodules would work. This is great feedback
for that work, though :)

>
> 1. There is always a possibility that the branch name in the superproject already exists
> in the submodule remote, but is a completely different topic (think of a branch named "refactor"
> for example). With this series (and propagateBranches=true), this would mean that 
> the initial submodule clone would create a local branch "refactor" that points to the gitlink
> in the superproject, and a remote-tracking branch "origin/refactor" that points to the unrelated
> submodule topic branch from the submodule remote. This could be confusing... but I don't
> really know what Git could do about it !

Yes.. I think submodule.propagateBranches requires a change in mental
model where submodule branch names have nothing to do with the
submodule's remote's branch names. It _might_ not be so confusing once
users have a grasp on that, but I recognize that that's quite optimistic
of me.

The biggest sources of confusion I see comes from remote-tracking,
either implicitly (e.g. "git checkout topic" when "topic" doesn't exist,
but "origin/topic" does) or manually (e.g. "git branch
--set-upstream-to"), because they'll see similarly-named remote and
local branches that have nothing to do with each other.

I can see some ways to address this, though they're not perfect:

- Remove the need for users to set up manual remote-tracking and disable
  implicit remote-tracking (possibly by using .gitmodules). 
- Block or warn on possibly confusing operations (e.g. don't allow users
  to create branches in the submodule directly)

> In patch 3/8 'git branch' is used to create the submodule branch from an oid, and so 
> it does not track any branch, and is not affected by 'branch.autoSetupMerge' as far as I 
> could test. But maybe this should be explicitely mentioned somewhere? 

I could mention it. For now, I don't see any reasonable way to respect
branch.autoSetupMerge, since there's no good way to map from 'submodule
remote branch' -> 'local submodule branch that's part of a superproject
branch'.

Between this and the previous section, perhaps we should think of
submodule branches as being fundamentally different from "regular"
branches..

> 2. The new "git submodule update" behaviour seems to only make sense with "--checkout", 
> which is the default "git submodue update" mode. But what if one uses "git submodule
> update --merge", or "--rebase", or has submodule.<name>.update set to a custom command
> or "none" ? Is the idea that these modes are incompatible with submodule branching ?
> I can understand that this gets really complex and might change when 'git merge' and 
> 'git rebase' themselves are taught to update submodule worktrees (and probably also submodule
> branches from what you refer to as future work), but in any case I think we should at 
> least test the new code when these options are used (if only to error out outright as
> incompatible)...

Ah, I should have been more explicit. Yes, I intend for the other modes
to be incompatible, so this should error out outright.

"git submodule update" is overdue for some refactoring IMO (at least for
internal use cases). It is quite badly overloaded - it peforms 3
different actions (clone missing submodules, fetch missing commits,
"update" worktree) and supports 4 different update strategies. In this
series, I was concerned mainly with "git clone --recurse-submodules",
and teaching "git submodule update --checkout" turned out to be a nice
side-effect, but because the "submodule update" API surface is so large,
we end up with this extra noise of how this new setting will interact
with the rest of API.

One goal of the Submodule UX (branches aside) is to remove the need for
manual submodule lifecycle management via "git submodule". For "git
submodule update", this would be:

- Teach fetch to clone missing submodules
- Teach fetch to fetch missing commits (I've done this already with my
  work on fetching in out-of-tree submodules)
- Teach checkout/reset to update working trees as we expect

I'm still not sure what to think of the merge/rebase/custom command
update strategies since they aren't simply "updating" the working tree.
My suspicion is that merge/rebase are more like partial implementations
of merge/rebase --recurse-submodules. The custom command is perhaps more
ofa way for users to orchestrate nonnative Git workflows on top of
submodules, so I don't feel bad about not supporting that at least.

> Thanks and cheers,
>
> Philippe.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 7/8] submodule--helper: remove update_data.suboid
  2022-10-28 20:14     ` [PATCH v3 7/8] submodule--helper: remove update_data.suboid Glen Choo via GitGitGadget
@ 2022-11-14 23:45       ` Jonathan Tan
  0 siblings, 0 replies; 56+ messages in thread
From: Jonathan Tan @ 2022-11-14 23:45 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget; +Cc: Jonathan Tan, git, Philippe Blain, Glen Choo

Rearranging the diffs...

"Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:
> @@ -2532,10 +2532,8 @@ static int update_submodule(struct update_data *update_data)
>  	if (ret)
>  		return ret;
>  
> -	if (update_data->just_cloned)
> -		oidcpy(&update_data->suboid, null_oid());
> -	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD",
> -				     &update_data->suboid, NULL))
> +	if (!update_data->just_cloned &&
> +	    resolve_gitlink_ref(update_data->sm_path, "HEAD", &suboid, NULL))
>  		return die_message(_("Unable to find current revision in submodule path '%s'"),
>  				   update_data->displaypath);
>  

Here, we only set suboid if !update_data->just_cloned...

> @@ -2570,7 +2568,8 @@ static int update_submodule(struct update_data *update_data)
>  		free(remote_ref);
>  	}
>  
> -	submodule_up_to_date = oideq(&update_data->oid, &update_data->suboid);
> +	submodule_up_to_date = !update_data->just_cloned &&
> +		oideq(&update_data->oid, &suboid);
>  	if (!submodule_up_to_date || update_data->force) {
>  		ret = run_update_procedure(update_data);
>  		if (ret)

...and here, we read suboid if !update_data->just_cloned.

> @@ -2523,6 +2522,7 @@ static int update_submodule(struct update_data *update_data)
>  {
>  	int submodule_up_to_date;
>  	int ret;
> +	struct object_id suboid;

And here we declare it with no initializer. This is safe for now since we only
read it when !update_data->just_cloned, which is also the condition for setting
it, but may be error-prone in the future. Probably best to initialize it to
something.
 
Other than that everything looks good up to and including this patch. This is
definitely an improvement over using a null OID to signal something, so thanks.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 8/8] clone, submodule update: create and check out branches
  2022-10-28 20:14     ` [PATCH v3 8/8] clone, submodule update: create and check out branches Glen Choo via GitGitGadget
  2022-11-08 13:53       ` Philippe Blain
@ 2022-11-15 18:15       ` Jonathan Tan
  2022-11-22 18:44         ` Glen Choo
  1 sibling, 1 reply; 56+ messages in thread
From: Jonathan Tan @ 2022-11-15 18:15 UTC (permalink / raw)
  To: Glen Choo via GitGitGadget; +Cc: Jonathan Tan, git, Philippe Blain, Glen Choo

"Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:
> +# Test the behavior of an already-cloned submodule.
> +# NEEDSWORK When updating with branches, we always use the branch instead of the
> +# gitlink's OID. This results in some imperfect behavior:
> +#
> +# - If the gitlink's OID disagrees with the branch OID, updating with branches
> +#   may result in a dirty worktree
> +# - If the branch does not exist, the update fails.
> +#
> +# We will reevaluate when "git checkout --recurse-submodules" supports branches
> +# For now, just test for this imperfect behavior.

I think the rationale for this behavior is as follows:

We want a world in which submodules have branches and Git commands use them
wherever possible. There are a few options for "git submodule update" when the
superproject has a branch checked out:

1. Checkout the branch, ignoring OID (as in this patch).
2. Checkout the branch, erroring out if the OID is wrong.
3. 1 + creating the branch if it does not exist.
4. 2 + creating the branch if it does not exist.
5. Always forcibly create the branch at the gitlink's OID and then checking
   it out.

At this point in the discussion, for a low-level command like "git submodule
update", doing as little as possible makes sense to me, which is 1.

But since we do not automatically create the branch if it does not exist, this
means that we have to do it when we clone the submodule. Our options are:

A. Create only the branch that is checked out in the superproject (as in this
   patch).
B. Create all branches that are present in the superproject.
C. Go back on our previous decision, switching to 3.

My instinct is that we want to maintain, as much as possible, the invariant
that for each branch in the superproject, if the branch tip has a gitlink
pointing to a submodule, that submodule has a branch of the same name. And I
think that this invariant can only be maintained by "git submodule update" if
we use B or C.

> +test_expect_success 'branches - other branch checked out, correct branch exists, OIDs disagree' '
> +	test_when_finished "rm -fr branch-super-cloned" &&
> +	cp -r branch-super-clean branch-super-cloned &&
> +
> +	git -C branch-super-cloned branch --recurse-submodules new-branch &&
> +	git -C branch-super-cloned checkout new-branch &&
> +	git -C branch-super-cloned/sub1 checkout new-branch &&
> +	test_commit -C branch-super-cloned/sub1 new-commit &&
> +	git -C branch-super-cloned/sub1 checkout main &&
> +	git -C branch-super-cloned submodule update &&
> +
> +	HEAD_BRANCH1=$(git -C branch-super-cloned/sub1 symbolic-ref HEAD) &&
> +	test $HEAD_BRANCH1 = "refs/heads/new-branch" &&
> +	test_clean_submodule ! branch-super-cloned sub1
> +'
> +
> +test_expect_success 'branches - other branch checked out, correct branch does not exist' '
> +	test_when_finished "rm -fr branch-super-cloned" &&
> +	cp -r branch-super-clean branch-super-cloned &&
> +
> +	git -C branch-super-cloned branch new-branch &&
> +	git -C branch-super-cloned checkout new-branch &&
> +	test_must_fail git -C branch-super-cloned submodule update

Can we also check what error message is being printed?
 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 8/8] clone, submodule update: create and check out branches
  2022-11-15 18:15       ` Jonathan Tan
@ 2022-11-22 18:44         ` Glen Choo
  2022-11-23  1:33           ` Jonathan Tan
  0 siblings, 1 reply; 56+ messages in thread
From: Glen Choo @ 2022-11-22 18:44 UTC (permalink / raw)
  To: Jonathan Tan, Glen Choo via GitGitGadget
  Cc: Jonathan Tan, git, Philippe Blain

Thanks for the thoughtful response, Jonathan :)

Jonathan Tan <jonathantanmy@google.com> writes:

> "Glen Choo via GitGitGadget" <gitgitgadget@gmail.com> writes:
>> +# Test the behavior of an already-cloned submodule.
>> +# NEEDSWORK When updating with branches, we always use the branch instead of the
>> +# gitlink's OID. This results in some imperfect behavior:
>> +#
>> +# - If the gitlink's OID disagrees with the branch OID, updating with branches
>> +#   may result in a dirty worktree
>> +# - If the branch does not exist, the update fails.
>> +#
>> +# We will reevaluate when "git checkout --recurse-submodules" supports branches
>> +# For now, just test for this imperfect behavior.
>
> I think the rationale for this behavior is as follows:
>
> We want a world in which submodules have branches and Git commands use them
> wherever possible. There are a few options for "git submodule update" when the
> superproject has a branch checked out:
>
> 1. Checkout the branch, ignoring OID (as in this patch).
> 2. Checkout the branch, erroring out if the OID is wrong.
> 3. 1 + creating the branch if it does not exist.
> 4. 2 + creating the branch if it does not exist.
> 5. Always forcibly create the branch at the gitlink's OID and then checking
>    it out.
>
> At this point in the discussion, for a low-level command like "git submodule
> update", doing as little as possible makes sense to me, which is 1.
>
> But since we do not automatically create the branch if it does not exist, this
> means that we have to do it when we clone the submodule. Our options are:
>
> A. Create only the branch that is checked out in the superproject (as in this
>    patch).
> B. Create all branches that are present in the superproject.
> C. Go back on our previous decision, switching to 3.
>
> My instinct is that we want to maintain, as much as possible, the invariant
> that for each branch in the superproject, if the branch tip has a gitlink
> pointing to a submodule, that submodule has a branch of the same name. And I
> think that this invariant can only be maintained by "git submodule update" if
> we use B or C.

I think C is good to have in this series, though for slightly different
reasons.

I agree that the invariant should be preserved when we check out
branches both in the initial clone and in subsequent checkouts. However,
I don't think that we necessarily need to have all superproject branches
after the initial clone. Even if the submodule only has a single
superproject branch, that's enough to have an ephemeral clone for
writing small changes. We could defer the "all superproject branches"
problem til after we worry about subsequent checkouts (i.e. "git
checkout" with branches).

We can handle "initial clone" and "subsequent checkout" as smaller, more
digestible series as long as the work for "initial clone" doesn't get in
the way of "subsequent checkout". My plan (as of v2) was:

- For the intial clone, create only the checked out superproject branch
  at clone time and check it out (aka A)
- For subsequent checkouts, check out the superproject branch, creating
  it if it does not exist (aka C)

But it doesn't make sense to mix both A _and_ C, since C would already
give us the same result as A, so it probably makes sense to go straight
to C in this series (i.e. only for the initial clone, not subsequent
checkouts). I'll do that in v3.

I prefer C in the long run, since both A and B require that the list of
submodule branches never get out of sync with the superproject, which is
hard to enforce, e.g.:

- The user could create a branch in the superproject without recursing
  in to submodules.
- The user could delete the branch in the submodule.
- (Worst yet) The process that creates branches in the submodule _after_
  creating the branch in the superproject could exit unexpectedly (e.g.
  SIGINT). There is no atomic way to create branches in both repos.

We could create a command that would repair broken branch states ("git
submodule repair"?), but C can self-repair, which avoids this problem
entirely.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 8/8] clone, submodule update: create and check out branches
  2022-11-22 18:44         ` Glen Choo
@ 2022-11-23  1:33           ` Jonathan Tan
  2022-11-23  4:00             ` Junio C Hamano
  0 siblings, 1 reply; 56+ messages in thread
From: Jonathan Tan @ 2022-11-23  1:33 UTC (permalink / raw)
  To: Glen Choo; +Cc: Jonathan Tan, Glen Choo via GitGitGadget, git, Philippe Blain

Glen Choo <chooglen@google.com> writes:
> > 1. Checkout the branch, ignoring OID (as in this patch).
> > 2. Checkout the branch, erroring out if the OID is wrong.
> > 3. 1 + creating the branch if it does not exist.
> > 4. 2 + creating the branch if it does not exist.
> > 5. Always forcibly create the branch at the gitlink's OID and then checking
> >    it out.

[snip]

> > A. Create only the branch that is checked out in the superproject (as in this
> >    patch).
> > B. Create all branches that are present in the superproject.
> > C. Go back on our previous decision, switching to 3.

[snip]

> But it doesn't make sense to mix both A _and_ C, since C would already
> give us the same result as A, so it probably makes sense to go straight
> to C in this series (i.e. only for the initial clone, not subsequent
> checkouts). I'll do that in v3.
> 
> I prefer C in the long run, since both A and B require that the list of
> submodule branches never get out of sync with the superproject, which is
> hard to enforce, e.g.:
 
I discussed this with Glen in-office and Glen pointed out that A is actually
not necessarily redundant with respect to C, since a "git submodule add" may
clone a submodule, but it would not run "git submodule update" (so 3 + C would
mean that no branch is created in the submodule, which is not what we want). So
we still need A.

As for 1 vs. 3, we will still need 3 in the future for the reasons I described
in my previous e-mail, but I think that that can be done incrementally. My
concern is to avoid doing something in a patch set that we will later need to
undo; I think that we are indeed avoiding it here (we're doing A but we will
still need it in the future, so there is no undoing of A needed).

So overall, after this discussion, this patch set looks good to me, except for
the minor points that I have commented on in my previous emails.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [PATCH v3 8/8] clone, submodule update: create and check out branches
  2022-11-23  1:33           ` Jonathan Tan
@ 2022-11-23  4:00             ` Junio C Hamano
  0 siblings, 0 replies; 56+ messages in thread
From: Junio C Hamano @ 2022-11-23  4:00 UTC (permalink / raw)
  To: Jonathan Tan; +Cc: Glen Choo, Glen Choo via GitGitGadget, git, Philippe Blain

Jonathan Tan <jonathantanmy@google.com> writes:

> ... My
> concern is to avoid doing something in a patch set that we will later need to
> undo; I think that we are indeed avoiding it here (we're doing A but we will
> still need it in the future, so there is no undoing of A needed).
>
> So overall, after this discussion, this patch set looks good to me, except for
> the minor points that I have commented on in my previous emails.

Thanks for a summary.

^ permalink raw reply	[flat|nested] 56+ messages in thread

end of thread, other threads:[~2022-11-23  4:00 UTC | newest]

Thread overview: 56+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-29 20:54 [PATCH 0/6] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
2022-08-29 20:54 ` [PATCH 1/6] clone: teach --detach option Glen Choo via GitGitGadget
2022-08-30  4:02   ` Philippe Blain
2022-08-29 20:54 ` [PATCH 2/6] repo-settings: add submodule_propagate_branches Glen Choo via GitGitGadget
2022-08-30  4:02   ` Philippe Blain
2022-08-29 20:54 ` [PATCH 3/6] t5617: drop references to remote-tracking branches Glen Choo via GitGitGadget
2022-08-30  4:03   ` Philippe Blain
2022-08-29 20:54 ` [PATCH 4/6] submodule: return target of submodule symref Glen Choo via GitGitGadget
2022-09-01 20:01   ` Jonathan Tan
2022-09-01 20:46     ` Glen Choo
2022-08-29 20:54 ` [PATCH 5/6] submodule--helper: refactor up-to-date criterion Glen Choo via GitGitGadget
2022-08-29 20:54 ` [PATCH 6/6] clone, submodule update: check out branches Glen Choo via GitGitGadget
2022-08-30  4:03   ` Philippe Blain
2022-08-30 22:54     ` Glen Choo
2022-09-01 20:00   ` Jonathan Tan
2022-10-20 20:20 ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Glen Choo via GitGitGadget
2022-10-20 20:20   ` [PATCH v2 1/7] clone: teach --detach option Glen Choo via GitGitGadget
2022-10-20 20:20   ` [PATCH v2 2/7] repo-settings: add submodule_propagate_branches Glen Choo via GitGitGadget
2022-10-25 18:03     ` Jonathan Tan
2022-10-20 20:20   ` [PATCH v2 3/7] submodule--helper clone: create named branch Glen Choo via GitGitGadget
2022-10-25 18:00     ` Jonathan Tan
2022-10-20 20:20   ` [PATCH v2 4/7] t5617: drop references to remote-tracking branches Glen Choo via GitGitGadget
2022-10-20 20:20   ` [PATCH v2 5/7] submodule: return target of submodule symref Glen Choo via GitGitGadget
2022-10-20 20:20   ` [PATCH v2 6/7] submodule update: refactor update targets Glen Choo via GitGitGadget
2022-10-20 20:20   ` [PATCH v2 7/7] clone, submodule update: create and check out branches Glen Choo via GitGitGadget
2022-10-25 17:56     ` Jonathan Tan
2022-10-25 21:49       ` Glen Choo
2022-10-20 22:40   ` [PATCH v2 0/7] clone, submodule update: check out submodule branches Junio C Hamano
2022-10-20 23:53     ` Glen Choo
2022-10-21  0:01       ` Junio C Hamano
2022-10-28 20:14   ` [PATCH v3 0/8] " Glen Choo via GitGitGadget
2022-10-28 20:14     ` [PATCH v3 1/8] clone: teach --detach option Glen Choo via GitGitGadget
2022-10-28 21:40       ` Junio C Hamano
2022-10-28 21:54         ` Junio C Hamano
2022-10-28 22:55           ` Glen Choo
2022-10-30 18:14             ` Taylor Blau
2022-10-31 17:07               ` Glen Choo
2022-11-08 13:32       ` Philippe Blain
2022-10-28 20:14     ` [PATCH v3 2/8] repo-settings: add submodule_propagate_branches Glen Choo via GitGitGadget
2022-10-28 20:14     ` [PATCH v3 3/8] submodule--helper clone: create named branch Glen Choo via GitGitGadget
2022-10-28 20:14     ` [PATCH v3 4/8] t5617: drop references to remote-tracking branches Glen Choo via GitGitGadget
2022-10-28 20:14     ` [PATCH v3 5/8] submodule: return target of submodule symref Glen Choo via GitGitGadget
2022-10-28 21:49       ` Junio C Hamano
2022-10-28 23:11         ` Glen Choo
2022-10-28 20:14     ` [PATCH v3 6/8] submodule update: refactor update targets Glen Choo via GitGitGadget
2022-10-28 20:14     ` [PATCH v3 7/8] submodule--helper: remove update_data.suboid Glen Choo via GitGitGadget
2022-11-14 23:45       ` Jonathan Tan
2022-10-28 20:14     ` [PATCH v3 8/8] clone, submodule update: create and check out branches Glen Choo via GitGitGadget
2022-11-08 13:53       ` Philippe Blain
2022-11-15 18:15       ` Jonathan Tan
2022-11-22 18:44         ` Glen Choo
2022-11-23  1:33           ` Jonathan Tan
2022-11-23  4:00             ` Junio C Hamano
2022-10-30 18:19     ` [PATCH v3 0/8] clone, submodule update: check out submodule branches Taylor Blau
2022-11-08 14:23     ` Philippe Blain
2022-11-08 20:43       ` Glen Choo

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).