[PATCH 0/2] Advice upon clone --recurse-submodules --reference

[PATCH 0/2] Advice upon clone --recurse-submodules --reference

[Jonathan Tan]

This patch set came about because of a situation in $DAY_JOB described
in the commit message of patch 2. I found the existing documentation and
error messages unclear, so I have improved them.

Jonathan Tan (2):
  Doc: explain submodule.alternateErrorStrategy
  submodule--helper: advise on fatal alternate error

 Documentation/config/advice.txt    |  3 +++
 Documentation/config/submodule.txt |  4 +++-
 advice.c                           |  2 ++
 advice.h                           |  1 +
 builtin/submodule--helper.c        | 10 ++++++++++
 5 files changed, 19 insertions(+), 1 deletion(-)

-- 
2.24.0.432.g9d3f5f5b63-goog

[PATCH 1/2] Doc: explain submodule.alternateErrorStrategy

[Jonathan Tan]

Commit 31224cbdc7 ("clone: recursive and reference option triggers
submodule alternates", 2016-08-17) taught Git to support the
configuration options "submodule.alternateLocation" and
"submodule.alternateErrorStrategy" on a superproject.

If "submodule.alternateLocation" is configured to "superproject" on a
superproject, whenever a submodule of that superproject is cloned, it
instead computes the analogous alternate path for that submodule from
$GIT_DIR/objects/info/alternates of the superproject, and references it.

The "submodule.alternateErrorStrategy" option determines what happens
if that alternate cannot be referenced. However, it is not clear that
the clone proceeds as if no alternate was specified when that option is
not set to "die" (as can be seen in the tests in 31224cbdc7). Therefore,
document it accordingly.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
When I said "not clear" above, I mean that it is not clear *to me*, and
I assume that others will feel the same way. Feel free to drop this from
the patch set if the existing documentation is clear to most people.
---
 Documentation/config/submodule.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Documentation/config/submodule.txt b/Documentation/config/submodule.txt
index 0a1293b051..b33177151c 100644
--- a/Documentation/config/submodule.txt
+++ b/Documentation/config/submodule.txt
@@ -79,4 +79,6 @@ submodule.alternateLocation::
 submodule.alternateErrorStrategy::
 	Specifies how to treat errors with the alternates for a submodule
 	as computed via `submodule.alternateLocation`. Possible values are
-	`ignore`, `info`, `die`. Default is `die`.
+	`ignore`, `info`, `die`. Default is `die`. Note that if set to `ignore`
+	or `info`, and if there is an error with the computed alternate, the
+	clone proceeds as if no alternate was specified.
-- 
2.24.0.432.g9d3f5f5b63-goog

[PATCH 2/2] submodule--helper: advise on fatal alternate error

[Jonathan Tan]

When recursively cloning a superproject with some shallow modules
defined in its .gitmodules, then recloning with "--reference=<path>", an
error occurs. For example:

  git clone --recurse-submodules --branch=master -j8 \
    https://android.googlesource.com/platform/superproject \
    master
  git clone --recurse-submodules --branch=master -j8 \
    https://android.googlesource.com/platform/superproject \
    --reference master master2

fails with:

  fatal: submodule '<snip>' cannot add alternate: reference repository
  '<snip>' is shallow

When a alternate computed from the superproject's alternate cannot be
added, whether in this case or another, advise about configuring the
"submodule.alternateErrorStrategy" configuration option and using
"--reference-if-able" instead of "--reference" when cloning.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 Documentation/config/advice.txt |  3 +++
 advice.c                        |  2 ++
 advice.h                        |  1 +
 builtin/submodule--helper.c     | 10 ++++++++++
 4 files changed, 16 insertions(+)

diff --git a/Documentation/config/advice.txt b/Documentation/config/advice.txt
index 6aaa360202..d4e698cd3f 100644
--- a/Documentation/config/advice.txt
+++ b/Documentation/config/advice.txt
@@ -107,4 +107,7 @@ advice.*::
 		editor input from the user.
 	nestedTag::
 		Advice shown if a user attempts to recursively tag a tag object.
+	submoduleAlternateErrorStrategyDie:
+		Advice shown when a submodule.alternateErrorStrategy option
+		configured to "die" causes a fatal error.
 --
diff --git a/advice.c b/advice.c
index 3ee0ee2d8f..249c60dcf3 100644
--- a/advice.c
+++ b/advice.c
@@ -30,6 +30,7 @@ int advice_waiting_for_editor = 1;
 int advice_graft_file_deprecated = 1;
 int advice_checkout_ambiguous_remote_branch_name = 1;
 int advice_nested_tag = 1;
+int advice_submodule_alternate_error_strategy_die = 1;
 
 static int advice_use_color = -1;
 static char advice_colors[][COLOR_MAXLEN] = {
@@ -89,6 +90,7 @@ static struct {
 	{ "graftFileDeprecated", &advice_graft_file_deprecated },
 	{ "checkoutAmbiguousRemoteBranchName", &advice_checkout_ambiguous_remote_branch_name },
 	{ "nestedTag", &advice_nested_tag },
+	{ "submoduleAlternateErrorStrategyDie", &advice_submodule_alternate_error_strategy_die },
 
 	/* make this an alias for backward compatibility */
 	{ "pushNonFastForward", &advice_push_update_rejected }
diff --git a/advice.h b/advice.h
index d015404843..b706780614 100644
--- a/advice.h
+++ b/advice.h
@@ -30,6 +30,7 @@ extern int advice_waiting_for_editor;
 extern int advice_graft_file_deprecated;
 extern int advice_checkout_ambiguous_remote_branch_name;
 extern int advice_nested_tag;
+extern int advice_submodule_alternate_error_strategy_die;
 
 int git_default_advice_config(const char *var, const char *value);
 __attribute__((format (printf, 1, 2)))
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2c2395a620..bdfc46fdea 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -19,6 +19,7 @@
 #include "diffcore.h"
 #include "diff.h"
 #include "object-store.h"
+#include "advice.h"
 
 #define OPT_QUIET (1 << 0)
 #define OPT_CACHED (1 << 1)
@@ -1268,6 +1269,14 @@ struct submodule_alternate_setup {
 #define SUBMODULE_ALTERNATE_SETUP_INIT { NULL, \
 	SUBMODULE_ALTERNATE_ERROR_IGNORE, NULL }
 
+static void advise_submodule_alternate_error_die(void)
+{
+	if (!advice_submodule_alternate_error_strategy_die)
+		return;
+	advise(_("An alternate computed from a superproject's alternate is invalid."));
+	advise(_("To allow Git to clone without an alternate in such a case, set submodule.alternateErrorStrategy to 'info' or, equivalently, clone with '--reference-if-able' instead of '--reference'."));
+}
+
 static int add_possible_reference_from_superproject(
 		struct object_directory *odb, void *sas_cb)
 {
@@ -1299,6 +1308,7 @@ static int add_possible_reference_from_superproject(
 		} else {
 			switch (sas->error_mode) {
 			case SUBMODULE_ALTERNATE_ERROR_DIE:
+				advise_submodule_alternate_error_die();
 				die(_("submodule '%s' cannot add alternate: %s"),
 				    sas->submodule_name, err.buf);
 			case SUBMODULE_ALTERNATE_ERROR_INFO:
-- 
2.24.0.432.g9d3f5f5b63-goog

Re: [PATCH 1/2] Doc: explain submodule.alternateErrorStrategy

[Jeff King]

On Mon, Nov 25, 2019 at 05:30:59PM -0800, Jonathan Tan wrote:

> Commit 31224cbdc7 ("clone: recursive and reference option triggers
> submodule alternates", 2016-08-17) taught Git to support the
> configuration options "submodule.alternateLocation" and
> "submodule.alternateErrorStrategy" on a superproject.
> 
> If "submodule.alternateLocation" is configured to "superproject" on a
> superproject, whenever a submodule of that superproject is cloned, it
> instead computes the analogous alternate path for that submodule from
> $GIT_DIR/objects/info/alternates of the superproject, and references it.
> 
> The "submodule.alternateErrorStrategy" option determines what happens
> if that alternate cannot be referenced. However, it is not clear that
> the clone proceeds as if no alternate was specified when that option is
> not set to "die" (as can be seen in the tests in 31224cbdc7). Therefore,
> document it accordingly.
> 
> Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
> ---
> When I said "not clear" above, I mean that it is not clear *to me*, and
> I assume that others will feel the same way. Feel free to drop this from
> the patch set if the existing documentation is clear to most people.

FWIW, I learned about these options for the first time from your email,
and I think the suggested change makes the behavior much more clear.

-Peff

Re: [PATCH 2/2] submodule--helper: advise on fatal alternate error

[Jeff King]

On Mon, Nov 25, 2019 at 05:31:00PM -0800, Jonathan Tan wrote:

> When recursively cloning a superproject with some shallow modules
> defined in its .gitmodules, then recloning with "--reference=<path>", an
> error occurs. For example:
> 
>   git clone --recurse-submodules --branch=master -j8 \
>     https://android.googlesource.com/platform/superproject \
>     master
>   git clone --recurse-submodules --branch=master -j8 \
>     https://android.googlesource.com/platform/superproject \
>     --reference master master2
> 
> fails with:
> 
>   fatal: submodule '<snip>' cannot add alternate: reference repository
>   '<snip>' is shallow
> 
> When a alternate computed from the superproject's alternate cannot be
> added, whether in this case or another, advise about configuring the
> "submodule.alternateErrorStrategy" configuration option and using
> "--reference-if-able" instead of "--reference" when cloning.

It sounds like that advice sends people in the right direction, which is
a good thing.

I kind of wonder if the default for alternateErrorStrategy should be
more lenient, but I don't really know much about the feature in the
first place. I'll let people who are more clueful ponder that, but
certainly your patch is an improvement in the meantime.

One minor suggestion:

> +static void advise_submodule_alternate_error_die(void)
> +{
> +	if (!advice_submodule_alternate_error_strategy_die)
> +		return;
> +	advise(_("An alternate computed from a superproject's alternate is invalid."));
> +	advise(_("To allow Git to clone without an alternate in such a case, set submodule.alternateErrorStrategy to 'info' or, equivalently, clone with '--reference-if-able' instead of '--reference'."));
> +}

The advise() function handles newlines gracefully (putting a "hint:" at
the start of each line). So you can put both in a single call, and wrap
the second long line, which also lets translators see the whole message
as a unit. Many of the existing calls define the message outside of any
function (e.g., embedded_advice), which makes it easier to see how long
the lines will be when displayed.

So maybe:

  static const alternate_error_advice[] = N_(
  "An alternate computed from a superproject's alternate is invalid.\n"
  "To allow Git to clone without an alternate in such a case, set\n"
  "submodule.alternateErrorStrategy to 'info' or, equivalently, clone with\n"
  "'--reference-if-able' instead of '--reference'."
  );

  ...
	switch (sas->error_mode) {
	case SUBMODULE_ALTERNATE_ERROR_DIE:
		if (advice_submodule_alternate_error_strategy_die)
			advise(_(alternate_error_advice));
		die(...);

?

-Peff

Re: [PATCH 1/2] Doc: explain submodule.alternateErrorStrategy

[Junio C Hamano]

Jonathan Tan <jonathantanmy@google.com> writes:

> Commit 31224cbdc7 ("clone: recursive and reference option triggers
> submodule alternates", 2016-08-17) taught Git to support the
> configuration options "submodule.alternateLocation" and
> "submodule.alternateErrorStrategy" on a superproject.
> ...
> The "submodule.alternateErrorStrategy" option determines what happens
> if that alternate cannot be referenced. However, it is not clear that
> the clone proceeds as if no alternate was specified when that option is
> not set to "die" (as can be seen in the tests in 31224cbdc7). Therefore,
> document it accordingly.

Given that for everyday use (cf. sha1-file.c::link_alt_odb_entry())
of an alternate is best-effort basis, I have a feeling that it was a
design mistake to have the "error strategy" configuration option in
the first place, and "clone --reference-if-able" was the result of
the same design mistake.  We would have been better off if we made
these the best-effort features as well.

But the ship has sailed long ago---so I think these two are the best
we can do at this point.  Perhaps flipping the default to warn may
be a longer term improvement, too.

[PATCH v2 0/2] Advice upon clone --recurse-submodules --reference

[Jonathan Tan]

Thanks everyone! Here's an updated v2 following Peff's suggestions on
how to write the advice part.

Making submodule.alternateErrorStrategy default to 'info' is reasonable,
but I don't know how much it matters in practice - here,
submodule.alternateLocation was set because of "clone
--recurse-submodules --reference", which also sets
submodule.alternateErrorStrategy. I don't know when
submodule.alternateLocation would ever be set alone, but then again, I'm
not very familiar with this part. I've left that alone for now.

Jonathan Tan (2):
  Doc: explain submodule.alternateErrorStrategy
  submodule--helper: advise on fatal alternate error

 Documentation/config/advice.txt    |  3 +++
 Documentation/config/submodule.txt |  4 +++-
 advice.c                           |  2 ++
 advice.h                           |  1 +
 builtin/submodule--helper.c        | 10 ++++++++++
 5 files changed, 19 insertions(+), 1 deletion(-)

-- 
2.24.0.393.g34dc348eaf-goog

[PATCH v2 1/2] Doc: explain submodule.alternateErrorStrategy

[Jonathan Tan]

Commit 31224cbdc7 ("clone: recursive and reference option triggers
submodule alternates", 2016-08-17) taught Git to support the
configuration options "submodule.alternateLocation" and
"submodule.alternateErrorStrategy" on a superproject.

If "submodule.alternateLocation" is configured to "superproject" on a
superproject, whenever a submodule of that superproject is cloned, it
instead computes the analogous alternate path for that submodule from
$GIT_DIR/objects/info/alternates of the superproject, and references it.

The "submodule.alternateErrorStrategy" option determines what happens
if that alternate cannot be referenced. However, it is not clear that
the clone proceeds as if no alternate was specified when that option is
not set to "die" (as can be seen in the tests in 31224cbdc7). Therefore,
document it accordingly.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 Documentation/config/submodule.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Documentation/config/submodule.txt b/Documentation/config/submodule.txt
index 0a1293b051..b33177151c 100644
--- a/Documentation/config/submodule.txt
+++ b/Documentation/config/submodule.txt
@@ -79,4 +79,6 @@ submodule.alternateLocation::
 submodule.alternateErrorStrategy::
 	Specifies how to treat errors with the alternates for a submodule
 	as computed via `submodule.alternateLocation`. Possible values are
-	`ignore`, `info`, `die`. Default is `die`.
+	`ignore`, `info`, `die`. Default is `die`. Note that if set to `ignore`
+	or `info`, and if there is an error with the computed alternate, the
+	clone proceeds as if no alternate was specified.
-- 
2.24.0.393.g34dc348eaf-goog

[PATCH v2 2/2] submodule--helper: advise on fatal alternate error

[Jonathan Tan]

When recursively cloning a superproject with some shallow modules
defined in its .gitmodules, then recloning with "--reference=<path>", an
error occurs. For example:

  git clone --recurse-submodules --branch=master -j8 \
    https://android.googlesource.com/platform/superproject \
    master
  git clone --recurse-submodules --branch=master -j8 \
    https://android.googlesource.com/platform/superproject \
    --reference master master2

fails with:

  fatal: submodule '<snip>' cannot add alternate: reference repository
  '<snip>' is shallow

When a alternate computed from the superproject's alternate cannot be
added, whether in this case or another, advise about configuring the
"submodule.alternateErrorStrategy" configuration option and using
"--reference-if-able" instead of "--reference" when cloning.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 Documentation/config/advice.txt |  3 +++
 advice.c                        |  2 ++
 advice.h                        |  1 +
 builtin/submodule--helper.c     | 10 ++++++++++
 4 files changed, 16 insertions(+)

diff --git a/Documentation/config/advice.txt b/Documentation/config/advice.txt
index 6aaa360202..d4e698cd3f 100644
--- a/Documentation/config/advice.txt
+++ b/Documentation/config/advice.txt
@@ -107,4 +107,7 @@ advice.*::
 		editor input from the user.
 	nestedTag::
 		Advice shown if a user attempts to recursively tag a tag object.
+	submoduleAlternateErrorStrategyDie:
+		Advice shown when a submodule.alternateErrorStrategy option
+		configured to "die" causes a fatal error.
 --
diff --git a/advice.c b/advice.c
index 3ee0ee2d8f..249c60dcf3 100644
--- a/advice.c
+++ b/advice.c
@@ -30,6 +30,7 @@ int advice_waiting_for_editor = 1;
 int advice_graft_file_deprecated = 1;
 int advice_checkout_ambiguous_remote_branch_name = 1;
 int advice_nested_tag = 1;
+int advice_submodule_alternate_error_strategy_die = 1;
 
 static int advice_use_color = -1;
 static char advice_colors[][COLOR_MAXLEN] = {
@@ -89,6 +90,7 @@ static struct {
 	{ "graftFileDeprecated", &advice_graft_file_deprecated },
 	{ "checkoutAmbiguousRemoteBranchName", &advice_checkout_ambiguous_remote_branch_name },
 	{ "nestedTag", &advice_nested_tag },
+	{ "submoduleAlternateErrorStrategyDie", &advice_submodule_alternate_error_strategy_die },
 
 	/* make this an alias for backward compatibility */
 	{ "pushNonFastForward", &advice_push_update_rejected }
diff --git a/advice.h b/advice.h
index d015404843..b706780614 100644
--- a/advice.h
+++ b/advice.h
@@ -30,6 +30,7 @@ extern int advice_waiting_for_editor;
 extern int advice_graft_file_deprecated;
 extern int advice_checkout_ambiguous_remote_branch_name;
 extern int advice_nested_tag;
+extern int advice_submodule_alternate_error_strategy_die;
 
 int git_default_advice_config(const char *var, const char *value);
 __attribute__((format (printf, 1, 2)))
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 2c2395a620..12d546dfbb 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -19,6 +19,7 @@
 #include "diffcore.h"
 #include "diff.h"
 #include "object-store.h"
+#include "advice.h"
 
 #define OPT_QUIET (1 << 0)
 #define OPT_CACHED (1 << 1)
@@ -1268,6 +1269,13 @@ struct submodule_alternate_setup {
 #define SUBMODULE_ALTERNATE_SETUP_INIT { NULL, \
 	SUBMODULE_ALTERNATE_ERROR_IGNORE, NULL }
 
+static const char alternate_error_advice[] = N_(
+"An alternate computed from a superproject's alternate is invalid.\n"
+"To allow Git to clone without an alternate in such a case, set\n"
+"submodule.alternateErrorStrategy to 'info' or, equivalently, clone with\n"
+"'--reference-if-able' instead of '--reference'."
+);
+
 static int add_possible_reference_from_superproject(
 		struct object_directory *odb, void *sas_cb)
 {
@@ -1299,6 +1307,8 @@ static int add_possible_reference_from_superproject(
 		} else {
 			switch (sas->error_mode) {
 			case SUBMODULE_ALTERNATE_ERROR_DIE:
+				if (advice_submodule_alternate_error_strategy_die)
+					advise(_(alternate_error_advice));
 				die(_("submodule '%s' cannot add alternate: %s"),
 				    sas->submodule_name, err.buf);
 			case SUBMODULE_ALTERNATE_ERROR_INFO:
-- 
2.24.0.393.g34dc348eaf-goog

Re: [PATCH v2 0/2] Advice upon clone --recurse-submodules --reference

[Jeff King]

On Mon, Dec 02, 2019 at 11:57:50AM -0800, Jonathan Tan wrote:

> Thanks everyone! Here's an updated v2 following Peff's suggestions on
> how to write the advice part.

Thanks, this looks good to me.

I think Junio and I both wondered if the default should be flipped to
"info", but I don't feel strongly about it, and this seems like an
improvement in the meantime.

-Peff

Re: [PATCH v2 0/2] Advice upon clone --recurse-submodules --reference

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Mon, Dec 02, 2019 at 11:57:50AM -0800, Jonathan Tan wrote:
>
>> Thanks everyone! Here's an updated v2 following Peff's suggestions on
>> how to write the advice part.
>
> Thanks, this looks good to me.
>
> I think Junio and I both wondered if the default should be flipped to
> "info", but I don't feel strongly about it, and this seems like an
> improvement in the meantime.

We indeed did, and I agree this is a good step regardless.

Thanks, both.