[RFC/PATCH 0/4] submodule remotes

[RFC/PATCH 0/4] submodule remotes

[Patrick Steinhardt]

The following patch series implements a new feature on top of the
submodule command that allows for configuring multiple remotes for
a given submodule. Next to new subcommands `git submodule remote
(add|rm|set-url|show)` that allow to show and modify remotes for
a given submodule, `git remote sync` has been extended to apply
settings to the repositories.

The commands are implemented in such a way that they write the
remote configuration into .gitmodules according to the following
example:

    [submodule-remote "submodule-name.remote-name"]
    url = http://example.com/remote.git
    pushurl = git@example.com:remote.git

where "submodule-name" is the submodule's name and "remote-name"
is the name of the remote when it will be synchronized into the
submodule repository. The section-name is definitly up for
discussion and I don't really know if there might be issues with
the format "submodule-name.remote-name", but as far as I know
there is no possibility of having sub-subsections inside config
files.

There are some issues that I am currently aware of:

    - If we specify a remote "origin" for a submodule, `git
      submodule sync` will happily overwrite
      submodule.${submodule-name}.url. We certainly don't want to
      drop the old way of specifying a single URL but I am not
      sure what to do when a new "origin" has been specified.
      Perhaps a warning or user confirmation would suffice?

    - If the user specifies his own remotes and afterwards syncs
      the submodule's remotes, his settings will be overwritten.
      Maybe remotes should only be synced when a switch is
      specified (e.g. `git submodule sync --remotes` or `git
      submodule remotes sync`)?

This patch series is not intended to be included as-is as there
are no tests yet and the implementation has not been tested that
much. It should only evaluate if there is any interest and
hopefully spark some discussion as to if this feature is
something that is regarded as useful to others.

iveqy in IRC told me that there has been a discussion on
something similar, I wasn't able to find that though.

Regards
Patrick


Patrick Steinhardt (4):
  submodules: implement synchronizing of remotes.
  submodules: implement remote commands.
  submodules: update docs to reflect remotes.
  submodules: add bash completion for remotes.

 Documentation/git-submodule.txt        |  23 +++
 contrib/completion/git-completion.bash |   2 +-
 git-submodule.sh                       | 252 ++++++++++++++++++++++++++++++++-
 3 files changed, 274 insertions(+), 3 deletions(-)

-- 
2.3.5

[RFC/PATCH 1/4] submodules: implement synchronizing of remotes.

[Patrick Steinhardt]

Previously it was not possible to specify custom remotes for
submodules. This feature has now been implemented and can be
accessed by setting the keys 'submodule-remote.$name.$remote.url'
and 'submodule-remote.$name.$remote.push-url', respectively.

When issuing a `git submodule sync` we will test if submodules
have one or more remotes specified and if so those will be either
added if nonexistent or their URLs will be adjusted to match the
specified URLs.
---
 git-submodule.sh | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/git-submodule.sh b/git-submodule.sh
index 36797c3..599a847 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1268,6 +1268,7 @@ cmd_status()
 		fi
 	done
 }
+
 #
 # Sync remote urls for submodules
 # This makes the value for remote.$remote.url match the value
@@ -1347,6 +1348,32 @@ cmd_sync()
 			)
 			fi
 		fi
+
+		git config -f .gitmodules --get-regexp "submodule-remote\.$name\..*\.url" 2>/dev/null |
+		while read key url
+		do
+			remote=$(echo "$key" | sed "s/submodule-remote\.$name\.\(.*\)\.url/\1/")
+			pushurl=$(git config -f .gitmodules --get "submodule-remote.$name.$remote.pushurl")
+
+			(
+				cd "$sm_path"
+
+				if ! git remote | grep "^$remote$" >/dev/null 2>/dev/null
+				then
+					say "$(eval_gettext "Adding remote '$remote' for submodule '$prefix$sm_path'")"
+					git remote add "$remote" "$url"
+				else
+					say "$(eval_gettext "Setting URL for remote '$remote' in submodule '$prefix$sm_path'")"
+					git remote set-url "$remote" "$url"
+				fi
+
+				if test ! -z "$pushurl"
+				then
+					say "$(eval_gettext "Setting push URL for remote '$remote' in submodule '$prefix$sm_path'")"
+					git remote set-url --push "$remote" "$pushurl"
+				fi
+			)
+		done
 	done
 }
 
-- 
2.3.5

[RFC/PATCH 2/4] submodules: implement remote commands.

[Patrick Steinhardt]

Add commands to modify a submodule's remote configuration. There
are commands to add and remove submodule remotes as well as to
modify the URL of a submodule remote.
---
 git-submodule.sh | 225 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 223 insertions(+), 2 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 599a847..6904f29 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -12,7 +12,11 @@ USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <re
    or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase] [--reference <repository>] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
    or: $dashless [--quiet] foreach [--recursive] <command>
-   or: $dashless [--quiet] sync [--recursive] [--] [<path>...]"
+   or: $dashless [--quiet] sync [--recursive] [--] [<path>...]
+   or: $dashless [--quiet] remote add <path> <name> <url>
+   or: $dashless [--quiet] remote rm <path> <name>
+   or: $dashless [--quiet] remote show <path>
+   or: $dashless [--quiet] remote set-url [--push] <path> <name> <url>"
 OPTIONS_SPEC=
 SUBDIRECTORY_OK=Yes
 . git-sh-setup
@@ -1270,6 +1274,223 @@ cmd_status()
 }
 
 #
+# Modify remote configuration in .gitmodules
+#
+cmd_remote()
+{
+	while test $# -ne 0
+	do
+		case "$1" in
+			-q|--quiet)
+				GIT_QUIET=1
+				shift
+				;;
+			add|rm|show)
+				subcommand=$1
+				shift
+				;;
+			set-url)
+				subcommand=set_url
+				shift
+				;;
+			*)
+				break;;
+		esac
+	done
+
+	if test -z "$subcommand"
+	then
+		usage
+	fi
+
+	"cmd_remote_$subcommand" "$@"
+}
+
+#
+# Show remote configuration for a gitmodule
+#
+cmd_remote_show()
+{
+	while test $# -ne 0
+	do
+		case "$1" in
+			-v|--verbose)
+				verbose=1
+				shift
+				;;
+			*)
+				sm_path="$1"
+				shift
+				break;;
+		esac
+	done
+
+	if test $# -ne 0
+	then
+		usage
+	fi
+
+	if test -z "$sm_path"
+	then
+		die "$(gettext "No submodule path specified")"
+	fi
+
+	sm_name=$(module_name "$sm_path") || exit
+
+	cd_to_toplevel
+
+	git config -f .gitmodules --get-regexp "submodule-remote\.$sm_name\..*\.url" 2>/dev/null |
+	while read key url
+	do
+		remote=$(echo "$key" | sed "s/submodule-remote\.$sm_name\.\(.*\)\.url/\1/")
+		section="submodule-remote.$sm_name.$remote"
+
+		if test -z "$verbose"
+		then
+			echo "$remote"
+		else
+			url=$(git config -f .gitmodules "$section.url" 2>/dev/null)
+			pushurl=$(git config -f .gitmodules "$section.pushurl" 2>/dev/null)
+
+			if test -z "$pushurl"
+			then
+				pushurl="$url"
+			fi
+
+			echo -e "$remote\t$url (fetch)"
+			echo -e "$remote\t$pushurl (push)"
+		fi
+	done
+}
+
+#
+# Add remote configuration to .gitmodules
+# This adds a new remote with the key
+# submodule-remote.$name.$remote.url set to the specified value
+# to .gitmodules.
+#
+cmd_remote_add()
+{
+	if test $# -ne 3
+	then
+		usage
+	fi
+
+	sm_path="$1"
+	remote_name="$2"
+	remote_url="$3"
+
+	sm_name=$(module_name "$sm_path") || exit
+	displaypath=$(relative_path "$sm_path")
+	key="submodule-remote.$sm_name.$remote_name.url"
+
+	if test -z "$remote_name"
+	then
+		die "$(eval_gettext "Empty remote name not allowed")"
+	fi
+
+	cd_to_toplevel
+
+	if git config -f .gitmodules "$key" >/dev/null 2>/dev/null
+	then
+		die "$(eval_gettext "Remote '\$remote_name' for submodule '\$sm_name' already present")"
+	fi
+
+	if git config -f .gitmodules "submodule-remote.$sm_name.$remote_name.url" "$remote_url"
+	then
+		say "$(eval_gettext "Remote '\$remote_name' added for path '\$displaypath'")"
+	else
+		die "$(eval_gettext "Remote '\$remote_name' could not be added for path '\$displaypath'")"
+	fi
+}
+
+#
+# Remove remote configuration from .gitmodules
+# This removes the remote for the specified submodule and remote
+# name.
+#
+cmd_remote_rm()
+{
+	if test $# -ne 2
+	then
+		usage
+	fi
+
+	sm_path="$1"
+	remote_name="$2"
+
+	sm_name=$(module_name "$sm_path") || exit
+	displaypath=$(relative_path "$sm_path")
+	section="submodule-remote.$sm_name.$remote_name"
+
+	if test -z "$remote_name"
+	then
+		die "$(eval_gettext "Empty remote name not allowed")"
+	fi
+
+	if ! git config -f .gitmodules "$section.url" >/dev/null 2>/dev/null
+	then
+		die "$(eval_gettext "No remote '\$remote_name' present for path '\$displaypath'")"
+	fi
+
+	if git config -f .gitmodules --remove-section "$section" >/dev/null 2>/dev/null
+	then
+		say "$(eval_gettext "Remote '\$remote_name' removed for path '\$displaypath'")"
+	else
+		die "$(eval_gettext "Remote '\$remote_name' could not be removed for path '\$displaypath'")"
+	fi
+}
+
+#
+# Change remote URL configuration in .gitmodules
+# This sets the values submodule-remote.$name.$remote.url and
+# submodule-remote.$name.$remote.pushurl in .gitmodules.
+#
+cmd_remote_set_url()
+{
+	if test $# -lt 3
+	then
+		usage
+	fi
+
+	if test "$1" = "--push"
+	then
+		push=1
+		shift
+	fi
+
+	sm_path="$1"
+	remote_name="$2"
+	url="$3"
+
+	sm_name=$(module_name "$sm_path") || exit
+	displaypath=$(relative_path "$sm_path")
+
+	if test -z "$remote_name"
+	then
+		die "$(eval_gettext "Empty remote name not allowed")"
+	fi
+
+	section="submodule-remote.$sm_name.$remote_name"
+	if test -z $push
+	then
+		key="$section.url"
+	else
+		key="$section.pushurl"
+	fi
+
+	if ! git config -f .gitmodules "$section.url" >/dev/null 2>/dev/null
+	then
+		die "$(eval_gettext "No remote '\$remote_name' specified for path '\$displaypath'")"
+	fi
+
+	if ! git config -f .gitmodules "$key" "$url"
+	then
+		die "$(eval_gettext "could not set URL for '\$displaypath'")"
+	fi
+}
+
+#
 # Sync remote urls for submodules
 # This makes the value for remote.$remote.url match the value
 # specified in .gitmodules.
@@ -1386,7 +1607,7 @@ cmd_sync()
 while test $# != 0 && test -z "$command"
 do
 	case "$1" in
-	add | foreach | init | deinit | update | status | summary | sync)
+	add | foreach | init | deinit | update | status | summary | sync | remote)
 		command=$1
 		;;
 	-q|--quiet)
-- 
2.3.5

[RFC/PATCH 3/4] submodules: update docs to reflect remotes.

[Patrick Steinhardt]

---
 Documentation/git-submodule.txt | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 2c25916..a49a2ad 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -21,6 +21,9 @@ SYNOPSIS
 	      [commit] [--] [<path>...]
 'git submodule' [--quiet] foreach [--recursive] <command>
 'git submodule' [--quiet] sync [--recursive] [--] [<path>...]
+'git submodule' [--quiet] remote add <path> <name> <url>
+'git submodule' [--quiet] remote rm <path> <name>
+'git submodule' [--quiet] remote set-url [--push] <path> <name> <url>"
 
 
 DESCRIPTION
@@ -233,6 +236,22 @@ As an example, +git submodule foreach \'echo $path {backtick}git
 rev-parse HEAD{backtick}'+ will show the path and currently checked out
 commit for each submodule.
 
+remote::
+	Modify a submodule's remote configuration. The command has subcommands that
+	mirror the commands of `git remote`. The change will be reflected inside
+	of the .gitmodules file the submodule is specified in. Changes will be
+	synchronized with the submodule by running `git submodule sync`.
++
+	`git submodule remote add <sm_path> <remote> <url>`;;
+		add a new remote with the URL specified to the submodule
+	`git submodule remote rm <sm_path> <remote>`;;
+		remove a remote with the given name for the specified submodule
+	`git submodule remote show [-v|--verbose] <sm_path>`;;
+		show configured remotes for the submodule. If `--verbose` is specified,
+		also print URLs.
+	`git submodule remote set-url [--push] <sm_path> <remote> <url>`;;
+		set the (push) URL for the given remote name and submodule.
+
 sync::
 	Synchronizes submodules' remote URL configuration setting
 	to the value specified in .gitmodules. It will only affect those
@@ -240,6 +259,10 @@ sync::
 	case when they are initialized or freshly added). This is useful when
 	submodule URLs change upstream and you need to update your local
 	repositories accordingly.
+
+	Also synchronizes all remotes that have been configured in .gitmodules.
+	Missing remotes will be added to the submodule while existing ones will be
+	updated according to the configured fetch or push URLs.
 +
 "git submodule sync" synchronizes all submodules while
 "git submodule sync \-- A" synchronizes submodule "A" only.
-- 
2.3.5

[RFC/PATCH 4/4] submodules: add bash completion for remotes.

[Patrick Steinhardt]

---
 contrib/completion/git-completion.bash | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index fbe5972..9d52bf0 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2411,7 +2411,7 @@ _git_submodule ()
 {
 	__git_has_doubledash && return
 
-	local subcommands="add status init deinit update summary foreach sync"
+	local subcommands="add status init deinit update summary foreach sync remote"
 	if [ -z "$(__git_find_on_cmdline "$subcommands")" ]; then
 		case "$cur" in
 		--*)
-- 
2.3.5

Re: [RFC/PATCH 1/4] submodules: implement synchronizing of remotes.

[Junio C Hamano]

On Wed, Apr 8, 2015 at 3:58 AM, Patrick Steinhardt <ps@pks.im> wrote:
> Previously it was not possible to specify custom remotes for
> submodules. This feature has now been implemented and can be

I am not going to say whether it makes sense to add this feature or not,
but I'll just react to "Previously".

Let's stop saying "Previously we couldn't do X, now we can".

Instead, let's consistently say "We don't do X. Being able to do X is a
good thing for such and such reasons. Make us capable of doing X by
doing this and that."

Some people even say "Currently we cannot do X. Teach us to do so",
which is equally bad but that is primarily because some people say
"Previously" and they feel the need to clarify which reality they are
talking about. Once we stop saying "Previously", they will stop saying
"Currently", and the world would be a better place ;-).

Thanks.

Re: [RFC/PATCH 1/4] submodules: implement synchronizing of remotes.

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1105 bytes --]

On Wed, Apr 08, 2015 at 08:46:28AM -0700, Junio C Hamano wrote:
> On Wed, Apr 8, 2015 at 3:58 AM, Patrick Steinhardt <ps@pks.im> wrote:
> > Previously it was not possible to specify custom remotes for
> > submodules. This feature has now been implemented and can be
> 
> I am not going to say whether it makes sense to add this feature or not,
> but I'll just react to "Previously".
> 
> Let's stop saying "Previously we couldn't do X, now we can".
> 
> Instead, let's consistently say "We don't do X. Being able to do X is a
> good thing for such and such reasons. Make us capable of doing X by
> doing this and that."
> 
> Some people even say "Currently we cannot do X. Teach us to do so",
> which is equally bad but that is primarily because some people say
> "Previously" and they feel the need to clarify which reality they are
> talking about. Once we stop saying "Previously", they will stop saying
> "Currently", and the world would be a better place ;-).
> 
> Thanks.

Points taken, I'll reword the message if there is any interest in
the proposed feature. ;)

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]