[PATCH 0/2] credential: handle partial URLs in config settings again (master)

[PATCH 0/2] credential: handle partial URLs in config settings again (master)

[Johannes Schindelin via GitGitGadget]

This is a companion patch to https://github.com/gitgitgadget/git/pull/615,
but targets maint instead of maint-2.17: There is not really a good way to
get those patches into maint by merging because the final patch looks quite
a bit different by necessity (after 46fd7b390034 (credential: allow wildcard
patterns when matching config, 2020-02-20) the urlmatch machinery is
supposed to handle credential.<url>.* config parsing).

IMHO the easiest strategy would be to apply these patches on top of maint,
then merge the other patches on top of maint-2.17 with -s ours.

Note: typically I do not send GitGitGadget PRs before the builds finished,
just in case that I did something really stupid that breaks plenty of tests.
However, in this case, I figure that y'all want to see them patches early,
and I can't wait for the build to finish because I need to get some rest,
like, right now.

Johannes Schindelin (2):
  credential: optionally allow partial URLs in
    credential_from_url_gently()
  credential: handle `credential.<partial-URL>.<key>` again

 credential.c           | 69 ++++++++++++++++++++++++++++++++++++++----
 t/t0300-credentials.sh | 38 +++++++++++++++++++++++
 urlmatch.c             | 10 ++++--
 urlmatch.h             |  5 +++
 4 files changed, 113 insertions(+), 9 deletions(-)


base-commit: af6b65d45ef179ed52087e80cb089f6b2349f4ec
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-620%2Fdscho%2Fcredential-config-partial-url-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-620/dscho/credential-config-partial-url-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/620
-- 
gitgitgadget

[PATCH 1/2] credential: optionally allow partial URLs in credential_from_url_gently()

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Prior to the fixes for CVE-2020-11008, we were _very_ lenient in what we
required from a URL in order to parse it into a `struct credential`.
That led to serious vulnerabilities.

There was one call site, though, that really needed that leniency: when
parsing config settings a la `credential.dev.azure.com.useHTTPPath`.
Settings like this might be desired when users want to use, say, a given
user name on a given host, regardless of the protocol to be used.

In preparation for fixing that bug, let's refactor the code to
optionally allow for partial URLs. For the moment, this functionality is
only exposed via the now-renamed function `credential_from_url_1()`, but
it is not used. The intention is to make it easier to verify that this
commit does not change the existing behavior unless explicitly allowing
for partial URLs.

Please note that this patch does more than just reinstating a way to
imitate the behavior before those CVE-2020-11008 fixes: Before that, we
would simply ignore URLs without a protocol. In other words,
misleadingly, the following setting would be applied to _all_ URLs:

	[credential "example.com"]
		username = that-me

The obvious intention is to match the host name only. With this patch,
we allow precisely that: when parsing the URL with non-zero
`allow_partial_url`, we do not simply return success if there was no
protocol, but we simply leave the protocol unset and continue parsing
the URL.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 credential.c | 42 ++++++++++++++++++++++++++++++++++++------
 1 file changed, 36 insertions(+), 6 deletions(-)

diff --git a/credential.c b/credential.c
index 108d9e183a5..b8f693fa288 100644
--- a/credential.c
+++ b/credential.c
@@ -377,8 +377,31 @@ static int check_url_component(const char *url, int quiet,
 	return -1;
 }
 
-int credential_from_url_gently(struct credential *c, const char *url,
-			       int quiet)
+/*
+ * Potentially-partial URLs can, but do not have to, contain
+ *
+ * - a protocol (or scheme) of the form "<protocol>://"
+ *
+ * - a host name (the part after the protocol and before the first slash after
+ *   that, if any)
+ *
+ * - a user name and potentially a password (as "<user>[:<password>]@" part of
+ *   the host name)
+ *
+ * - a path (the part after the host name, if any, starting with the slash)
+ *
+ * Missing parts will be left unset in `struct credential`. Thus, `https://`
+ * will have only the `protocol` set, `example.com` only the host name, and
+ * `/git` only the path.
+ *
+ * Note that an empty host name in an otherwise fully-qualified URL (e.g.
+ * `cert:///path/to/cert.pem`) will be treated as unset if we expect the URL to
+ * be potentially partial, and only then (otherwise, the empty string is used).
+ *
+ * The credential_from_url() function does not allow partial URLs.
+ */
+static int credential_from_url_1(struct credential *c, const char *url,
+				 int allow_partial_url, int quiet)
 {
 	const char *at, *colon, *cp, *slash, *host, *proto_end;
 
@@ -391,12 +414,12 @@ int credential_from_url_gently(struct credential *c, const char *url,
 	 *   (3) proto://<user>:<pass>@<host>/...
 	 */
 	proto_end = strstr(url, "://");
-	if (!proto_end || proto_end == url) {
+	if (!allow_partial_url && (!proto_end || proto_end == url)) {
 		if (!quiet)
 			warning(_("url has no scheme: %s"), url);
 		return -1;
 	}
-	cp = proto_end + 3;
+	cp = proto_end ? proto_end + 3 : url;
 	at = strchr(cp, '@');
 	colon = strchr(cp, ':');
 	slash = strchrnul(cp, '/');
@@ -420,8 +443,10 @@ int credential_from_url_gently(struct credential *c, const char *url,
 		host = at + 1;
 	}
 
-	c->protocol = xmemdupz(url, proto_end - url);
-	c->host = url_decode_mem(host, slash - host);
+	if (proto_end && proto_end - url > 0)
+		c->protocol = xmemdupz(url, proto_end - url);
+	if (!allow_partial_url || slash - host > 0)
+		c->host = url_decode_mem(host, slash - host);
 	/* Trim leading and trailing slashes from path */
 	while (*slash == '/')
 		slash++;
@@ -443,6 +468,11 @@ int credential_from_url_gently(struct credential *c, const char *url,
 	return 0;
 }
 
+int credential_from_url_gently(struct credential *c, const char *url, int quiet)
+{
+	return credential_from_url_1(c, url, 0, quiet);
+}
+
 void credential_from_url(struct credential *c, const char *url)
 {
 	if (credential_from_url_gently(c, url, 0) < 0)
-- 
gitgitgadget

[PATCH 2/2] credential: handle `credential.<partial-URL>.<key>` again

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

In the patches for CVE-2020-11008, the ability to specify credential
settings in the config for partial URLs got lost. For example, it used
to be possible to specify a credential helper for a specific protocol:

	[credential "https://"]
		helper = my-https-helper

Likewise, it used to be possible to configure settings for a specific
host, e.g.:

	[credential "dev.azure.com"]
		useHTTPPath = true

Let's reinstate this behavior.

While at it, increase the test coverage to document and verify the
behavior with a couple other categories of partial URLs.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 credential.c           | 27 +++++++++++++++++++++++++++
 t/t0300-credentials.sh | 38 ++++++++++++++++++++++++++++++++++++++
 urlmatch.c             | 10 +++++++---
 urlmatch.h             |  5 +++++
 4 files changed, 77 insertions(+), 3 deletions(-)

diff --git a/credential.c b/credential.c
index b8f693fa288..4e7197d97a9 100644
--- a/credential.c
+++ b/credential.c
@@ -37,6 +37,10 @@ int credential_match(const struct credential *want,
 #undef CHECK
 }
 
+
+static int credential_from_potentially_partial_url(struct credential *c,
+						   const char *url);
+
 static int credential_config_callback(const char *var, const char *value,
 				      void *data)
 {
@@ -82,6 +86,22 @@ static int select_all(const struct urlmatch_item *a,
 	return 0;
 }
 
+static int match_partial_url(const char *url, void *cb)
+{
+	struct credential *c = cb;
+	struct credential want = CREDENTIAL_INIT;
+	int matches = 0;
+
+	if (credential_from_potentially_partial_url(&want, url) < 0)
+		warning(_("skipping credential lookup for key: credential.%s"),
+			url);
+	else
+		matches = credential_match(&want, c);
+	credential_clear(&want);
+
+	return matches;
+}
+
 static void credential_apply_config(struct credential *c)
 {
 	char *normalized_url;
@@ -101,6 +121,7 @@ static void credential_apply_config(struct credential *c)
 	config.collect_fn = credential_config_callback;
 	config.cascade_fn = NULL;
 	config.select_fn = select_all;
+	config.fallback_match_fn = match_partial_url;
 	config.cb = c;
 
 	credential_format(c, &url);
@@ -468,6 +489,12 @@ static int credential_from_url_1(struct credential *c, const char *url,
 	return 0;
 }
 
+static int credential_from_potentially_partial_url(struct credential *c,
+						   const char *url)
+{
+	return credential_from_url_1(c, url, 1, 0);
+}
+
 int credential_from_url_gently(struct credential *c, const char *url, int quiet)
 {
 	return credential_from_url_1(c, url, 0, quiet);
diff --git a/t/t0300-credentials.sh b/t/t0300-credentials.sh
index 5555a1524f1..c6643288ed6 100755
--- a/t/t0300-credentials.sh
+++ b/t/t0300-credentials.sh
@@ -575,4 +575,42 @@ test_expect_success 'credential system refuses to work with missing protocol' '
 	test_i18ncmp expect stderr
 '
 
+test_expect_success 'credential config with partial URLs' '
+	echo "echo password=yep" | write_script git-credential-yep &&
+	test_write_lines url=https://user@example.com/repo.git >stdin &&
+	for partial in \
+		example.com \
+		user@example.com \
+		https:// \
+		https://example.com \
+		https://example.com/ \
+		https://user@example.com \
+		https://user@example.com/ \
+		https://example.com/repo.git \
+		https://user@example.com/repo.git \
+		/repo.git
+	do
+		git -c credential.$partial.helper=yep \
+			credential fill <stdin >stdout &&
+		grep yep stdout ||
+		return 1
+	done &&
+
+	for partial in \
+		dont.use.this \
+		http:// \
+		/repo
+	do
+		git -c credential.$partial.helper=yep \
+			credential fill <stdin >stdout &&
+		! grep yep stdout ||
+		return 1
+	done &&
+
+	git -c credential.$partial.helper=yep \
+		-c credential.with%0anewline.username=uh-oh \
+		credential fill <stdin >stdout 2>stderr &&
+	test_i18ngrep "skipping credential lookup for key" stderr
+'
+
 test_done
diff --git a/urlmatch.c b/urlmatch.c
index 29272a5c4f4..33a2ccd306b 100644
--- a/urlmatch.c
+++ b/urlmatch.c
@@ -572,10 +572,14 @@ int urlmatch_config_entry(const char *var, const char *value, void *cb)
 
 		config_url = xmemdupz(key, dot - key);
 		norm_url = url_normalize_1(config_url, &norm_info, 1);
+		if (norm_url)
+			retval = match_urls(url, &norm_info, &matched);
+		else if (collect->fallback_match_fn)
+			retval = collect->fallback_match_fn(config_url,
+							    collect->cb);
+		else
+			retval = 0;
 		free(config_url);
-		if (!norm_url)
-			return 0;
-		retval = match_urls(url, &norm_info, &matched);
 		free(norm_url);
 		if (!retval)
 			return 0;
diff --git a/urlmatch.h b/urlmatch.h
index 2407520731f..6ff42f81b0c 100644
--- a/urlmatch.h
+++ b/urlmatch.h
@@ -59,6 +59,11 @@ struct urlmatch_config {
 	 * specificity rules) than existing.
 	 */
 	int (*select_fn)(const struct urlmatch_item *found, const struct urlmatch_item *existing);
+	/*
+	 * An optional callback to allow e.g. for partial URLs; it shall
+	 * return 1 or 0 depending whether `url` matches or not.
+	 */
+	int (*fallback_match_fn)(const char *url, void *cb);
 };
 
 int urlmatch_config_entry(const char *var, const char *value, void *cb);
-- 
gitgitgadget

Re: [PATCH 0/2] credential: handle partial URLs in config settings again (master)

[Junio C Hamano]

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:

> This is a companion patch to https://github.com/gitgitgadget/git/pull/615,
> but targets maint instead of maint-2.17: There is not really a good way to
> get those patches into maint by merging because the final patch looks quite
> a bit different by necessity (after 46fd7b390034 (credential: allow wildcard
> patterns when matching config, 2020-02-20) the urlmatch machinery is
> supposed to handle credential.<url>.* config parsing).
>
> IMHO the easiest strategy would be to apply these patches on top of maint,
> then merge the other patches on top of maint-2.17 with -s ours.

Thanks for a quick help.