git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 0/2] config: allow specifying config entries via envvar pairs
@ 2020-11-24 10:50 Patrick Steinhardt
  2020-11-24 10:50 ` [PATCH v2 1/2] config: extract function to parse config pairs Patrick Steinhardt
                   ` (8 more replies)
  0 siblings, 9 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-11-24 10:50 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1775 bytes --]

Hi,

this is the second version of my patch series which aims to implement a
way to pass config entries via the environment while avoiding any
requirements to perform shell quoting on the user's side.

There's been quite some feedback on the first version, which I tried to
include in this version. Changes include:

    - I reworked how git detects which variables it should process.
      Instead of iterating from GIT_CONFIG_KEY_0 to $n until we find the
      first gap, this now uses a third environment variable
      GIT_CONFIG_COUNT which specifies show many environment config
      pairs should be processed. I've added this variable to the local
      environment variables, so that it's properly unset when moving
      between repos and printed by `git rev-parse --local-env-vars`.

    - Missing GIT_CONFIG_VALUE_$n keys for a given key are now treated
      as an error. The same is true for any environment value which
      should exist based on the value of GIT_CONFIG_COUNT.

    - I've changed priorities. The envvars are treated as command-level
      and as such override all values configured in files. But any
      explicit `git -c key=value` will now override these envvars.

    - I've improved test coverage to also nail down priorities.

Patrick

Patrick Steinhardt (2):
  config: extract function to parse config pairs
  config: allow specifying config entries via envvar pairs

 Documentation/git-config.txt |   9 +++
 cache.h                      |   1 +
 config.c                     |  96 +++++++++++++++++++++++++-------
 environment.c                |   1 +
 t/t1300-config.sh            | 105 ++++++++++++++++++++++++++++++++++-
 5 files changed, 190 insertions(+), 22 deletions(-)

-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v2 1/2] config: extract function to parse config pairs
  2020-11-24 10:50 [PATCH v2 0/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
@ 2020-11-24 10:50 ` Patrick Steinhardt
  2020-11-24 10:50 ` [PATCH v2 2/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
                   ` (7 subsequent siblings)
  8 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-11-24 10:50 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1819 bytes --]

The function `git_config_parse_parameter` is responsible for parsing a
`foo.bar=baz`-formatted configuration key, sanitizing the key and then
processing it via the given callback function. Given that we're about to
add a second user which is going to process keys in such which already
has keys and values separated, this commit extracts a function
`config_parse_pair` which only does the sanitization and processing
part.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 config.c | 24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/config.c b/config.c
index 2bdff4457b..3281b1374e 100644
--- a/config.c
+++ b/config.c
@@ -437,11 +437,26 @@ int git_config_key_is_valid(const char *key)
 	return !git_config_parse_key_1(key, NULL, NULL, 1);
 }
 
+static int config_parse_pair(const char *key, const char *value,
+			  config_fn_t fn, void *data)
+{
+	char *canonical_name;
+	int ret;
+
+	if (!strlen(key))
+		return error(_("empty config key"));
+	if (git_config_parse_key(key, &canonical_name, NULL))
+		return -1;
+
+	ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
+	free(canonical_name);
+	return ret;
+}
+
 int git_config_parse_parameter(const char *text,
 			       config_fn_t fn, void *data)
 {
 	const char *value;
-	char *canonical_name;
 	struct strbuf **pair;
 	int ret;
 
@@ -462,12 +477,7 @@ int git_config_parse_parameter(const char *text,
 		return error(_("bogus config parameter: %s"), text);
 	}
 
-	if (git_config_parse_key(pair[0]->buf, &canonical_name, NULL)) {
-		ret = -1;
-	} else {
-		ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
-		free(canonical_name);
-	}
+	ret = config_parse_pair(pair[0]->buf, value, fn, data);
 	strbuf_list_free(pair);
 	return ret;
 }
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v2 2/2] config: allow specifying config entries via envvar pairs
  2020-11-24 10:50 [PATCH v2 0/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
  2020-11-24 10:50 ` [PATCH v2 1/2] config: extract function to parse config pairs Patrick Steinhardt
@ 2020-11-24 10:50 ` Patrick Steinhardt
  2020-11-25  3:39   ` Junio C Hamano
                     ` (2 more replies)
  2020-11-25 10:41 ` [PATCH v2 0/2] " Jeff King
                   ` (6 subsequent siblings)
  8 siblings, 3 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-11-24 10:50 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 10235 bytes --]

While we currently have the `GIT_CONFIG_PARAMETERS` environment variable
which can be used to pass runtime configuration data to git processes,
it's an internal implementation detail and not supposed to be used by
end users.

Next to being for internal use only, this way of passing config entries
has a major downside: the config keys need to be parsed as they contain
both key and value in a single variable. As such, it is left to the user
to escape any potentially harmful characters in the value, which is
quite hard to do if values are controlled by a third party.

This commit thus adds a new way of adding config entries via the
environment which gets rid of this shortcoming. If the user passes the
`GIT_CONFIG_COUNT=$n` environment variable, Git will parse environment
variable pairs `GIT_CONFIG_KEY_$i` and `GIT_CONFIG_VALUE_$i` for each
`i` in `[0,n)`.

While the same can be achieved with `git -c <name>=<value>`, one may
wish to not do so for potentially sensitive information. E.g. if one
wants to set `http.extraHeader` to contain an authentication token,
doing so via `-c` would trivially leak those credentials via e.g. ps(1),
which typically also shows command arguments.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git-config.txt |   9 +++
 cache.h                      |   1 +
 config.c                     |  72 +++++++++++++++++++-----
 environment.c                |   1 +
 t/t1300-config.sh            | 105 ++++++++++++++++++++++++++++++++++-
 5 files changed, 173 insertions(+), 15 deletions(-)

diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt
index 7573160f21..84ae9b51a3 100644
--- a/Documentation/git-config.txt
+++ b/Documentation/git-config.txt
@@ -335,6 +335,15 @@ GIT_CONFIG_NOSYSTEM::
 	Whether to skip reading settings from the system-wide
 	$(prefix)/etc/gitconfig file. See linkgit:git[1] for details.
 
+GIT_CONFIG_COUNT,GIT_CONFIG_KEY_<n>,GIT_CONFIG_VALUE_<n>::
+	If GIT_CONFIG_COUNT is set to a positive number, all environment pairs
+	GIT_CONFIG_KEY_<n> and GIT_CONFIG_VALUE_<n> up to that number will be
+	added to the process's runtime configuration. The config pairs are
+	zero-indexed. Any missing key or value is treated as an error. An empty
+	GIT_CONFIG_COUNT is treated the same as GIT_CONFIG_COUNT=0, namely no
+	pairs are processed. Config entries set this way have command scope,
+	but will be overridden by any explicit options passed via `git -c`.
+
 See also <<FILES>>.
 
 
diff --git a/cache.h b/cache.h
index c0072d43b1..8a36146337 100644
--- a/cache.h
+++ b/cache.h
@@ -472,6 +472,7 @@ static inline enum object_type object_type(unsigned int mode)
 #define TEMPLATE_DIR_ENVIRONMENT "GIT_TEMPLATE_DIR"
 #define CONFIG_ENVIRONMENT "GIT_CONFIG"
 #define CONFIG_DATA_ENVIRONMENT "GIT_CONFIG_PARAMETERS"
+#define CONFIG_COUNT_ENVIRONMENT "GIT_CONFIG_COUNT"
 #define EXEC_PATH_ENVIRONMENT "GIT_EXEC_PATH"
 #define CEILING_DIRECTORIES_ENVIRONMENT "GIT_CEILING_DIRECTORIES"
 #define NO_REPLACE_OBJECTS_ENVIRONMENT "GIT_NO_REPLACE_OBJECTS"
diff --git a/config.c b/config.c
index 3281b1374e..5da1ae16c9 100644
--- a/config.c
+++ b/config.c
@@ -484,38 +484,82 @@ int git_config_parse_parameter(const char *text,
 
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
-	const char *env = getenv(CONFIG_DATA_ENVIRONMENT);
+	const char *env;
+	struct strbuf envvar = STRBUF_INIT;
 	int ret = 0;
-	char *envw;
+	char *envw = NULL;
 	const char **argv = NULL;
-	int nr = 0, alloc = 0;
 	int i;
 	struct config_source source;
 
-	if (!env)
-		return 0;
-
 	memset(&source, 0, sizeof(source));
 	source.prev = cf;
 	source.origin_type = CONFIG_ORIGIN_CMDLINE;
 	cf = &source;
 
-	/* sq_dequote will write over it */
-	envw = xstrdup(env);
+	env = getenv(CONFIG_COUNT_ENVIRONMENT);
+	if (env) {
+		unsigned long count;
+		char *endp;
 
-	if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
-		ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
-		goto out;
+		count = strtoul(env, &endp, 10);
+		if (*endp) {
+			ret = error(_("bogus count in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+		if (count > INT_MAX) {
+			ret = error(_("too many entries in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+
+		for (i = 0; i < count; i++) {
+			const char *key, *value;
+
+			strbuf_addf(&envvar, "GIT_CONFIG_KEY_%d", i);
+			key = getenv(envvar.buf);
+			if (!key) {
+				ret = error(_("missing config key %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			strbuf_addf(&envvar, "GIT_CONFIG_VALUE_%d", i);
+			value = getenv(envvar.buf);
+			if (!value) {
+				ret = error(_("missing config value %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			if (config_parse_pair(key, value, fn, data) < 0) {
+				ret = -1;
+				goto out;
+			}
+		}
 	}
 
-	for (i = 0; i < nr; i++) {
-		if (git_config_parse_parameter(argv[i], fn, data) < 0) {
-			ret = -1;
+	env = getenv(CONFIG_DATA_ENVIRONMENT);
+	if (env) {
+		int nr = 0, alloc = 0;
+
+		/* sq_dequote will write over it */
+		envw = xstrdup(env);
+
+		if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
+			ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
 			goto out;
 		}
+
+		for (i = 0; i < nr; i++) {
+			if (git_config_parse_parameter(argv[i], fn, data) < 0) {
+				ret = -1;
+				goto out;
+			}
+		}
 	}
 
 out:
+	strbuf_release(&envvar);
 	free(argv);
 	free(envw);
 	cf = source.prev;
diff --git a/environment.c b/environment.c
index bb518c61cd..e94eca92f3 100644
--- a/environment.c
+++ b/environment.c
@@ -116,6 +116,7 @@ const char * const local_repo_env[] = {
 	ALTERNATE_DB_ENVIRONMENT,
 	CONFIG_ENVIRONMENT,
 	CONFIG_DATA_ENVIRONMENT,
+	CONFIG_COUNT_ENVIRONMENT,
 	DB_ENVIRONMENT,
 	GIT_DIR_ENVIRONMENT,
 	GIT_WORK_TREE_ENVIRONMENT,
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 825d9a184f..8c90cca79d 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1316,6 +1316,107 @@ test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
 		git config --get-regexp "env.*"
 '
 
+test_expect_success 'git config handles environment config pairs' '
+	GIT_CONFIG_COUNT=2 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="foo" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="bar" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one foo
+	pair.two bar
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs without count' '
+	test_must_fail env GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one 2>error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one
+'
+
+test_expect_success 'git config ignores pairs exceeding count' '
+	GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="value" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with empty count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT= GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config fails with invalid count' '
+	test_must_fail env GIT_CONFIG_COUNT=10a git config --list 2>error &&
+	test_i18ngrep "bogus count" error &&
+	test_must_fail env GIT_CONFIG_COUNT=9999999999999999 git config --list 2>error &&
+	test_i18ngrep "too many entries" error
+'
+
+test_expect_success 'git config fails with missing config key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_VALUE_0="value" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config key" error
+'
+
+test_expect_success 'git config fails with missing config value' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0="pair.one" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config value" error
+'
+
+test_expect_success 'git config fails with invalid config pair key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0= GIT_CONFIG_VALUE_0=value \
+		git config --list &&
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0=missing-section GIT_CONFIG_VALUE_0=value \
+		git config --list
+'
+
+test_expect_success 'environment overrides config file' '
+	test_when_finished "rm -f .git/config" &&
+	cat >.git/config <<-EOF &&
+	[pair]
+	one = value
+	EOF
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=override \
+		git config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'command line overrides environment config' '
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
+		git -c pair.one=override config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
@@ -1661,9 +1762,11 @@ test_expect_success '--show-origin with --list' '
 	file:.git/config	user.override=local
 	file:.git/config	include.path=../include/relative.include
 	file:.git/../include/relative.include	user.relative=include
+	command line:	user.environ=true
 	command line:	user.cmdline=true
 	EOF
-	git -c user.cmdline=true config --list --show-origin >output &&
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=user.environ GIT_CONFIG_VALUE_0=true\
+		git -c user.cmdline=true config --list --show-origin >output &&
 	test_cmp expect output
 '
 
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 2/2] config: allow specifying config entries via envvar pairs
  2020-11-24 10:50 ` [PATCH v2 2/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
@ 2020-11-25  3:39   ` Junio C Hamano
  2020-11-25  7:06     ` Patrick Steinhardt
  2020-11-25  8:47   ` Ævar Arnfjörð Bjarmason
  2020-11-25  9:00   ` Ævar Arnfjörð Bjarmason
  2 siblings, 1 reply; 116+ messages in thread
From: Junio C Hamano @ 2020-11-25  3:39 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Jeff King,
	brian m. carlson, Philip Oakley

Patrick Steinhardt <ps@pks.im> writes:

> +GIT_CONFIG_COUNT,GIT_CONFIG_KEY_<n>,GIT_CONFIG_VALUE_<n>::

I think we write a header with multiple/related items like this
instead:

    GIT_CONFIG_COUNT::
    GIT_CONFIG_KEY_<n>::
    GIT_CONFIG_VALUE_<n>::

See how -f/--file is marked up in an earlier part of the same file.

> +	If GIT_CONFIG_COUNT is set to a positive number, all environment pairs
> +	GIT_CONFIG_KEY_<n> and GIT_CONFIG_VALUE_<n> up to that number will be
> +	added to the process's runtime configuration. The config pairs are
> +	zero-indexed. Any missing key or value is treated as an error. An empty
> +	GIT_CONFIG_COUNT is treated the same as GIT_CONFIG_COUNT=0, namely no
> +	pairs are processed. Config entries set this way have command scope,
> +	but will be overridden by any explicit options passed via `git -c`.
> +
>  See also <<FILES>>.

Doesn't this <<FILES>> refer to GIT_CONFIG and GIT_CONFIG_NOSYSTEM
that are described earlier?  It certainly looks out of place to see
it after the KEY/VALUE thing.

> +		for (i = 0; i < count; i++) {
> +			const char *key, *value;
> +
> +			strbuf_addf(&envvar, "GIT_CONFIG_KEY_%d", i);
> +			key = getenv(envvar.buf);
> +			if (!key) {
> +				ret = error(_("missing config key %s"), envvar.buf);
> +				goto out;
> +			}
> +			strbuf_reset(&envvar);
> +
> +			strbuf_addf(&envvar, "GIT_CONFIG_VALUE_%d", i);
> +			value = getenv(envvar.buf);
> +			if (!value) {
> +				ret = error(_("missing config value %s"), envvar.buf);
> +				goto out;
> +			}
> +			strbuf_reset(&envvar);

Didn't we got bitten by number of times that the string returned by
getenv() are not necessarily nonvolatile depending on platforms?  I
think the result of getenv() would need to be xstrdup'ed.

cf. 6776a84d (diff: ensure correct lifetime of external_diff_cmd,
2019-01-11)

> +			if (config_parse_pair(key, value, fn, data) < 0) {
> +				ret = -1;
> +				goto out;
> +			}
> +		}
>  	}
>  
> -	for (i = 0; i < nr; i++) {
> -		if (git_config_parse_parameter(argv[i], fn, data) < 0) {
> -			ret = -1;
> +	env = getenv(CONFIG_DATA_ENVIRONMENT);

> +	if (env) {
> +		int nr = 0, alloc = 0;
> +
> +		/* sq_dequote will write over it */
> +		envw = xstrdup(env);
> +
> +		if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
> +			ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
>  			goto out;
>  		}
> +
> +		for (i = 0; i < nr; i++) {
> +			if (git_config_parse_parameter(argv[i], fn, data) < 0) {
> +				ret = -1;
> +				goto out;
> +			}
> +		}
>  	}
>  
>  out:
> +	strbuf_release(&envvar);
>  	free(argv);
>  	free(envw);
>  	cf = source.prev;

With re-indentation this patch does, it is a bit hard to see the
correspondence between common lines in preimage and postimage, but I
think the patch adds the support of the new style environments
before the existing support of the GIT_CONFIG_DATA, but when there
is no compelling reason not to, new code should be added near the
bottom, not before the existing code, in the function.

Otherwise, this part of the patch looks OK to me.

Thanks.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 2/2] config: allow specifying config entries via envvar pairs
  2020-11-25  3:39   ` Junio C Hamano
@ 2020-11-25  7:06     ` Patrick Steinhardt
  2020-11-25  7:41       ` Junio C Hamano
  0 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2020-11-25  7:06 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Ævar Arnfjörð Bjarmason, Jeff King,
	brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 4178 bytes --]

On Tue, Nov 24, 2020 at 07:39:48PM -0800, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > +GIT_CONFIG_COUNT,GIT_CONFIG_KEY_<n>,GIT_CONFIG_VALUE_<n>::
> 
> I think we write a header with multiple/related items like this
> instead:
> 
>     GIT_CONFIG_COUNT::
>     GIT_CONFIG_KEY_<n>::
>     GIT_CONFIG_VALUE_<n>::
> 
> See how -f/--file is marked up in an earlier part of the same file.

Ah, thanks. I wondered how to format these but didn't spot other
examples.

> > +	If GIT_CONFIG_COUNT is set to a positive number, all environment pairs
> > +	GIT_CONFIG_KEY_<n> and GIT_CONFIG_VALUE_<n> up to that number will be
> > +	added to the process's runtime configuration. The config pairs are
> > +	zero-indexed. Any missing key or value is treated as an error. An empty
> > +	GIT_CONFIG_COUNT is treated the same as GIT_CONFIG_COUNT=0, namely no
> > +	pairs are processed. Config entries set this way have command scope,
> > +	but will be overridden by any explicit options passed via `git -c`.
> > +
> >  See also <<FILES>>.
> 
> Doesn't this <<FILES>> refer to GIT_CONFIG and GIT_CONFIG_NOSYSTEM
> that are described earlier?  It certainly looks out of place to see
> it after the KEY/VALUE thing.

Right, my fault.

> > +		for (i = 0; i < count; i++) {
> > +			const char *key, *value;
> > +
> > +			strbuf_addf(&envvar, "GIT_CONFIG_KEY_%d", i);
> > +			key = getenv(envvar.buf);
> > +			if (!key) {
> > +				ret = error(_("missing config key %s"), envvar.buf);
> > +				goto out;
> > +			}
> > +			strbuf_reset(&envvar);
> > +
> > +			strbuf_addf(&envvar, "GIT_CONFIG_VALUE_%d", i);
> > +			value = getenv(envvar.buf);
> > +			if (!value) {
> > +				ret = error(_("missing config value %s"), envvar.buf);
> > +				goto out;
> > +			}
> > +			strbuf_reset(&envvar);
> 
> Didn't we got bitten by number of times that the string returned by
> getenv() are not necessarily nonvolatile depending on platforms?  I
> think the result of getenv() would need to be xstrdup'ed.
> 
> cf. 6776a84d (diff: ensure correct lifetime of external_diff_cmd,
> 2019-01-11)

We did, but do we have to in this case? There is no interleaving calls
to getenv(3P), so we don't depend on at least $n getenv(3P) calls
succeeding without clobbering old values. It's true that it could be
that any other caller in the callchain clobbers the value, but as far as
I can see none does.

Anyway, I'm not opposed to changing this if you think it to be
necessary.

> > +			if (config_parse_pair(key, value, fn, data) < 0) {
> > +				ret = -1;
> > +				goto out;
> > +			}
> > +		}
> >  	}
> >  
> > -	for (i = 0; i < nr; i++) {
> > -		if (git_config_parse_parameter(argv[i], fn, data) < 0) {
> > -			ret = -1;
> > +	env = getenv(CONFIG_DATA_ENVIRONMENT);
> 
> > +	if (env) {
> > +		int nr = 0, alloc = 0;
> > +
> > +		/* sq_dequote will write over it */
> > +		envw = xstrdup(env);
> > +
> > +		if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
> > +			ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
> >  			goto out;
> >  		}
> > +
> > +		for (i = 0; i < nr; i++) {
> > +			if (git_config_parse_parameter(argv[i], fn, data) < 0) {
> > +				ret = -1;
> > +				goto out;
> > +			}
> > +		}
> >  	}
> >  
> >  out:
> > +	strbuf_release(&envvar);
> >  	free(argv);
> >  	free(envw);
> >  	cf = source.prev;
> 
> With re-indentation this patch does, it is a bit hard to see the
> correspondence between common lines in preimage and postimage, but I
> think the patch adds the support of the new style environments
> before the existing support of the GIT_CONFIG_DATA, but when there
> is no compelling reason not to, new code should be added near the
> bottom, not before the existing code, in the function.
> 
> Otherwise, this part of the patch looks OK to me.
> 
> Thanks.

It is required as this is what sets precedence of GIT_CONFIG_PARAMETERS
and thus `git -c` over GIT_CONFIG_COUNT. It's easy enough to split this
into two patches though, with a first refactoring which does the
indentation and a second one which adds the new code.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 2/2] config: allow specifying config entries via envvar pairs
  2020-11-25  7:06     ` Patrick Steinhardt
@ 2020-11-25  7:41       ` Junio C Hamano
  2020-11-25  7:57         ` Patrick Steinhardt
  0 siblings, 1 reply; 116+ messages in thread
From: Junio C Hamano @ 2020-11-25  7:41 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Jeff King,
	brian m. carlson, Philip Oakley

Patrick Steinhardt <ps@pks.im> writes:

>> > +		for (i = 0; i < count; i++) {
>> > +			const char *key, *value;
>> > +
>> > +			strbuf_addf(&envvar, "GIT_CONFIG_KEY_%d", i);
>> > +			key = getenv(envvar.buf);
>> > +			if (!key) {
>> > +				ret = error(_("missing config key %s"), envvar.buf);
>> > +				goto out;
>> > +			}
>> > +			strbuf_reset(&envvar);
>> > +
>> > +			strbuf_addf(&envvar, "GIT_CONFIG_VALUE_%d", i);
>> > +			value = getenv(envvar.buf);
>> > +			if (!value) {
>> > +				ret = error(_("missing config value %s"), envvar.buf);
>> > +				goto out;
>> > +			}
>> > +			strbuf_reset(&envvar);
>> 
>> Didn't we got bitten by number of times that the string returned by
>> getenv() are not necessarily nonvolatile depending on platforms?  I
>> think the result of getenv() would need to be xstrdup'ed.
>> 
>> cf. 6776a84d (diff: ensure correct lifetime of external_diff_cmd,
>> 2019-01-11)
>
> We did, but do we have to in this case? There is no interleaving calls
> to getenv(3P), so we don't depend on at least $n getenv(3P) calls
> succeeding without clobbering old values. It's true that it could be
> that any other caller in the callchain clobbers the value, but as far as
> I can see none does.

Doesn't the code expect "key" will stay valid even after another
call to getenv() grabs "value"?

> It is required as this is what sets precedence of GIT_CONFIG_PARAMETERS
> and thus `git -c` over GIT_CONFIG_COUNT.

OK, that is what the "will be overridden by any explicit options"
was about.  Perhaps that deserves an in-code comment, something like

	/*
	 * process GIT_CONFIG_KEY_N/GIT_CONFIG_VALUE_N pairs
	 * first, to be overridden by GIT_CONFIG_PARAMETERS
	 * inherited from parent Git processes' "git -c var=val"
	 * later
	 */

before we check GIT_CONFIG_COUNT and loop over the new style
environment variables.

Thanks.


^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 2/2] config: allow specifying config entries via envvar pairs
  2020-11-25  7:41       ` Junio C Hamano
@ 2020-11-25  7:57         ` Patrick Steinhardt
  0 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-11-25  7:57 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Ævar Arnfjörð Bjarmason, Jeff King,
	brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 2147 bytes --]

On Tue, Nov 24, 2020 at 11:41:34PM -0800, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> >> > +		for (i = 0; i < count; i++) {
> >> > +			const char *key, *value;
> >> > +
> >> > +			strbuf_addf(&envvar, "GIT_CONFIG_KEY_%d", i);
> >> > +			key = getenv(envvar.buf);
> >> > +			if (!key) {
> >> > +				ret = error(_("missing config key %s"), envvar.buf);
> >> > +				goto out;
> >> > +			}
> >> > +			strbuf_reset(&envvar);
> >> > +
> >> > +			strbuf_addf(&envvar, "GIT_CONFIG_VALUE_%d", i);
> >> > +			value = getenv(envvar.buf);
> >> > +			if (!value) {
> >> > +				ret = error(_("missing config value %s"), envvar.buf);
> >> > +				goto out;
> >> > +			}
> >> > +			strbuf_reset(&envvar);
> >> 
> >> Didn't we got bitten by number of times that the string returned by
> >> getenv() are not necessarily nonvolatile depending on platforms?  I
> >> think the result of getenv() would need to be xstrdup'ed.
> >> 
> >> cf. 6776a84d (diff: ensure correct lifetime of external_diff_cmd,
> >> 2019-01-11)
> >
> > We did, but do we have to in this case? There is no interleaving calls
> > to getenv(3P), so we don't depend on at least $n getenv(3P) calls
> > succeeding without clobbering old values. It's true that it could be
> > that any other caller in the callchain clobbers the value, but as far as
> > I can see none does.
> 
> Doesn't the code expect "key" will stay valid even after another
> call to getenv() grabs "value"?

Oh, right. No idea what I was thinking there.

> > It is required as this is what sets precedence of GIT_CONFIG_PARAMETERS
> > and thus `git -c` over GIT_CONFIG_COUNT.
> 
> OK, that is what the "will be overridden by any explicit options"
> was about.  Perhaps that deserves an in-code comment, something like
> 
> 	/*
> 	 * process GIT_CONFIG_KEY_N/GIT_CONFIG_VALUE_N pairs
> 	 * first, to be overridden by GIT_CONFIG_PARAMETERS
> 	 * inherited from parent Git processes' "git -c var=val"
> 	 * later
> 	 */
> 
> before we check GIT_CONFIG_COUNT and loop over the new style
> environment variables.
> 
> Thanks.

Will do, thanks!

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 2/2] config: allow specifying config entries via envvar pairs
  2020-11-24 10:50 ` [PATCH v2 2/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
  2020-11-25  3:39   ` Junio C Hamano
@ 2020-11-25  8:47   ` Ævar Arnfjörð Bjarmason
  2020-11-25  9:00   ` Ævar Arnfjörð Bjarmason
  2 siblings, 0 replies; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2020-11-25  8:47 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley


On Tue, Nov 24 2020, Patrick Steinhardt wrote:

> +test_expect_success 'command line overrides environment config' '
> +	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
> +		git -c pair.one=override config pair.one >actual &&
> +	cat >expect <<-EOF &&
> +	override
> +	EOF
> +	test_cmp expect actual
> +'
> +

Maybe a test to see which one of this new-style key-value thing
v.s. GIT_CONFIG_PARAMETERS wins? Helps if/when we ever refactor this to
at least see the behavior of the purely internal thing changed.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 2/2] config: allow specifying config entries via envvar pairs
  2020-11-24 10:50 ` [PATCH v2 2/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
  2020-11-25  3:39   ` Junio C Hamano
  2020-11-25  8:47   ` Ævar Arnfjörð Bjarmason
@ 2020-11-25  9:00   ` Ævar Arnfjörð Bjarmason
  2020-11-25 19:50     ` Junio C Hamano
  2 siblings, 1 reply; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2020-11-25  9:00 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley


On Tue, Nov 24 2020, Patrick Steinhardt wrote:

...some more feedback.

> +GIT_CONFIG_COUNT,GIT_CONFIG_KEY_<n>,GIT_CONFIG_VALUE_<n>::
> +	If GIT_CONFIG_COUNT is set to a positive number, all environment pairs
> +	GIT_CONFIG_KEY_<n> and GIT_CONFIG_VALUE_<n> up to that number will be
> +	added to the process's runtime configuration. The config pairs are
> +	zero-indexed. Any missing key or value is treated as an error. An empty
> +	GIT_CONFIG_COUNT is treated the same as GIT_CONFIG_COUNT=0, namely no
> +	pairs are processed. Config entries set this way have command scope,
> +	but will be overridden by any explicit options passed via `git -c`.

Perhaps work in some/all of some version of these:

 - There's also a GIT_CONFIG_PARAMETERS variable, which is considered
   internal to Git itself. Users are expected to set these.

   --> I.e. even if we're not going to support some format for
   --> GIT_CONFIG_PARAMETERS document what it is.

 - This is analogous to the pre-receive `GIT_PUSH_OPTION_*` variables
   (see linkgit:githooks[5]), but unlike those the `-c` option to
   linkgit:git(1) does not set `GIT_CONFIG_*`.

 - Saying "command scope" here I think is wrong/misleading. If I didn't
   know how this worked I'd expect the first git process to see it to
   delete it from the env, so e.g. the "fetch" command would see it, but
   not the "gc" it spawned (different commands). Maybe just say "the
   scope of these is as with other GIT_* environment variables, they'll
   be inherited by subprocesses".

> diff --git a/cache.h b/cache.h
> index c0072d43b1..8a36146337 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -472,6 +472,7 @@ static inline enum object_type object_type(unsigned int mode)
>  #define TEMPLATE_DIR_ENVIRONMENT "GIT_TEMPLATE_DIR"
>  #define CONFIG_ENVIRONMENT "GIT_CONFIG"
>  #define CONFIG_DATA_ENVIRONMENT "GIT_CONFIG_PARAMETERS"
> +#define CONFIG_COUNT_ENVIRONMENT "GIT_CONFIG_COUNT"

I was wondering if this shouldn't be "GIT_CONFIG_KEY_COUNT" to be
consistent with the push options environment, but on a closer look we
have:

 - GIT_CONFIG_COUNT
 - GIT_CONFIG_KEY_N
 - GIT_CONFIG_VALUE_N
 - GIT_PUSH_OPTION_COUNT
 - GIT_PUSH_OPTION_N

So I guess that makes sense & is consistent since we'd like to split the
key-value here to save the user the effort of figuring out which "="
they should split on.

> -	if (!env)
> -		return 0;
> -

Re the indent question to make the diff more readable question Junio
had: could set some "do we have this or that" variables here to not
reindent the existing code, but maybe not worth the effort...

> -	if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
> -		ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
> -		goto out;
> +		count = strtoul(env, &endp, 10);
> +		if (*endp) {
> +			ret = error(_("bogus count in %s"), CONFIG_COUNT_ENVIRONMENT);
> +			goto out;
> +		}
> +		if (count > INT_MAX) {
> +			ret = error(_("too many entries in %s"), CONFIG_COUNT_ENVIRONMENT);
> +			goto out;
> +		}
> +
> +		for (i = 0; i < count; i++) {
> +			const char *key, *value;
> +
> +			strbuf_addf(&envvar, "GIT_CONFIG_KEY_%d", i);
> +			key = getenv(envvar.buf);
> +			if (!key) {
> +				ret = error(_("missing config key %s"), envvar.buf);
> +				goto out;
> +			}
> +			strbuf_reset(&envvar);
> +
> +			strbuf_addf(&envvar, "GIT_CONFIG_VALUE_%d", i);
> +			value = getenv(envvar.buf);
> +			if (!value) {
> +				ret = error(_("missing config value %s"), envvar.buf);
> +				goto out;
> +			}
> +			strbuf_reset(&envvar);
> +
> +			if (config_parse_pair(key, value, fn, data) < 0) {
> +				ret = -1;
> +				goto out;
> +			}
> +		}
>  	}
>  
> -	for (i = 0; i < nr; i++) {
> -		if (git_config_parse_parameter(argv[i], fn, data) < 0) {
> -			ret = -1;
> +	env = getenv(CONFIG_DATA_ENVIRONMENT);
> +	if (env) {
> +		int nr = 0, alloc = 0;
> +
> +		/* sq_dequote will write over it */
> +		envw = xstrdup(env);
> +
> +		if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
> +			ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
>  			goto out;
>  		}
> +
> +		for (i = 0; i < nr; i++) {
> +			if (git_config_parse_parameter(argv[i], fn, data) < 0) {
> +				ret = -1;
> +				goto out;
> +			}
> +		}
>  	}
>  
>  out:
> +	strbuf_release(&envvar);
>  	free(argv);
>  	free(envw);
>  	cf = source.prev;
> diff --git a/environment.c b/environment.c
> index bb518c61cd..e94eca92f3 100644
> --- a/environment.c
> +++ b/environment.c
> @@ -116,6 +116,7 @@ const char * const local_repo_env[] = {
>  	ALTERNATE_DB_ENVIRONMENT,
>  	CONFIG_ENVIRONMENT,
>  	CONFIG_DATA_ENVIRONMENT,
> +	CONFIG_COUNT_ENVIRONMENT,
>  	DB_ENVIRONMENT,
>  	GIT_DIR_ENVIRONMENT,
>  	GIT_WORK_TREE_ENVIRONMENT,
> diff --git a/t/t1300-config.sh b/t/t1300-config.sh
> index 825d9a184f..8c90cca79d 100755
> --- a/t/t1300-config.sh
> +++ b/t/t1300-config.sh
> @@ -1316,6 +1316,107 @@ test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
>  		git config --get-regexp "env.*"
>  '
>  
> +test_expect_success 'git config handles environment config pairs' '

I was wondering if the patch would keep the current
GIT_CONFIG_PARAMETERS or replace it entirely with the new facility.

On the one hand it would make sense to just replace
GIT_CONFIG_PARAMETERS, we could make this code loop over the new values.

On the other hand, and this is an edge case I hadn't considered before,
any change to the semantics of GIT_CONFIG_PARAMETERS means that e.g. a
fetch->gc spawning would break in the face of a concurrent OS update to
/usr/bin/git, since "fetch" and "gc" might be of differing versions

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 0/2] config: allow specifying config entries via envvar pairs
  2020-11-24 10:50 [PATCH v2 0/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
  2020-11-24 10:50 ` [PATCH v2 1/2] config: extract function to parse config pairs Patrick Steinhardt
  2020-11-24 10:50 ` [PATCH v2 2/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
@ 2020-11-25 10:41 ` Jeff King
  2020-11-25 13:16   ` Patrick Steinhardt
                     ` (3 more replies)
  2020-12-01  9:55 ` [PATCH v3 0/4] " Patrick Steinhardt
                   ` (5 subsequent siblings)
  8 siblings, 4 replies; 116+ messages in thread
From: Jeff King @ 2020-11-25 10:41 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Junio C Hamano,
	brian m. carlson, Philip Oakley

On Tue, Nov 24, 2020 at 11:50:46AM +0100, Patrick Steinhardt wrote:

>     - I've changed priorities. The envvars are treated as command-level
>       and as such override all values configured in files. But any
>       explicit `git -c key=value` will now override these envvars.

That ordering makes sense. Those get passed through the environment,
too, but at some point there is a process where your new ones are in the
environment and the "-c" ones are on the command-line.

I do still think that a "--config-env" option solves your problem in a
much simpler way (especially in terms of interface we expose to users
that we'll be locked into forever). I sketched out the solution below if
it's of interest (and I'd be happy to polish it up, or hand it off to
you if so). But if you're unconvinced, I'll stop mentioning it.

diff --git a/config.c b/config.c
index 8f324ed3a6..d8cf6a5d6b 100644
--- a/config.c
+++ b/config.c
@@ -345,6 +345,27 @@ void git_config_push_parameter(const char *text)
 	strbuf_release(&env);
 }
 
+void git_config_push_env(const char *spec)
+{
+	struct strbuf buf = STRBUF_INIT;
+	const char *env_name;
+	const char *env_value;
+
+	env_name = strchr(spec, '=');
+	if (!env_name)
+		return; /* die or warn? */
+	env_name++;
+
+	env_value = getenv(env_name);
+	if (!env_value)
+		return; /* die or warn? */
+
+	strbuf_add(&buf, spec, env_name - spec);
+	strbuf_addstr(&buf, env_value);
+	git_config_push_parameter(buf.buf);
+	strbuf_release(&buf);
+}
+
 static inline int iskeychar(int c)
 {
 	return isalnum(c) || c == '-';
diff --git a/config.h b/config.h
index 91cdfbfb41..d05651c96c 100644
--- a/config.h
+++ b/config.h
@@ -138,6 +138,7 @@ int git_config_from_mem(config_fn_t fn,
 int git_config_from_blob_oid(config_fn_t fn, const char *name,
 			     const struct object_id *oid, void *data);
 void git_config_push_parameter(const char *text);
+void git_config_push_env(const char *spec);
 int git_config_from_parameters(config_fn_t fn, void *data);
 void read_early_config(config_fn_t cb, void *data);
 void read_very_early_config(config_fn_t cb, void *data);
diff --git a/git.c b/git.c
index 4b7bd77b80..342f2fb0c9 100644
--- a/git.c
+++ b/git.c
@@ -254,6 +254,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			git_config_push_parameter((*argv)[1]);
 			(*argv)++;
 			(*argc)--;
+		} else if (skip_prefix(cmd, "--config-env=", &cmd)) {
+			git_config_push_env(cmd);
 		} else if (!strcmp(cmd, "--literal-pathspecs")) {
 			setenv(GIT_LITERAL_PATHSPECS_ENVIRONMENT, "1", 1);
 			if (envchanged)

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 0/2] config: allow specifying config entries via envvar pairs
  2020-11-25 10:41 ` [PATCH v2 0/2] " Jeff King
@ 2020-11-25 13:16   ` Patrick Steinhardt
  2020-11-26  0:36     ` Jeff King
  2020-11-25 20:28   ` Junio C Hamano
                     ` (2 subsequent siblings)
  3 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2020-11-25 13:16 UTC (permalink / raw)
  To: Jeff King
  Cc: git, Ævar Arnfjörð Bjarmason, Junio C Hamano,
	brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 2087 bytes --]

On Wed, Nov 25, 2020 at 05:41:14AM -0500, Jeff King wrote:
> On Tue, Nov 24, 2020 at 11:50:46AM +0100, Patrick Steinhardt wrote:
> 
> >     - I've changed priorities. The envvars are treated as command-level
> >       and as such override all values configured in files. But any
> >       explicit `git -c key=value` will now override these envvars.
> 
> That ordering makes sense. Those get passed through the environment,
> too, but at some point there is a process where your new ones are in the
> environment and the "-c" ones are on the command-line.
> 
> I do still think that a "--config-env" option solves your problem in a
> much simpler way (especially in terms of interface we expose to users
> that we'll be locked into forever). I sketched out the solution below if
> it's of interest (and I'd be happy to polish it up, or hand it off to
> you if so). But if you're unconvinced, I'll stop mentioning it.

The thing I like more about using envvars only is that you only need to
modify a single part, while with `--config-env` there's two moving
parts. E.g. assume you have a script and want certain configuration to
apply to all git commands in that script. It's trivial in the envvar
case, while for `--config-env` you'll also have to modify each single
git call. You could get around that by using a wrapper, but it's still a
tad more involved. A second thing I briefly wondered about is the
maximum command line length, which may be easier to hit in case you want
to pass a lot of config entries.

None of these complaints apply to my original usecase, where
`--config-env` would work equally well. But I do think that for
scripting use, which is going to be most of all cases where my patch
series is useful, GIT_CONFIG_COUNT is easier to use.

There probably are good arguments for `--config-env`, for example that
it's easier to spot when executing a git command. I stil lean towards my
current implementation, but I'm obviously biased. So if there is
consensus that we should use `--config-env` instead, I'm not opposed.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 2/2] config: allow specifying config entries via envvar pairs
  2020-11-25  9:00   ` Ævar Arnfjörð Bjarmason
@ 2020-11-25 19:50     ` Junio C Hamano
  0 siblings, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2020-11-25 19:50 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Patrick Steinhardt, git, Jeff King, brian m. carlson, Philip Oakley

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Tue, Nov 24 2020, Patrick Steinhardt wrote:
>
> ...some more feedback.
>
>> +GIT_CONFIG_COUNT,GIT_CONFIG_KEY_<n>,GIT_CONFIG_VALUE_<n>::
>> +	If GIT_CONFIG_COUNT is set to a positive number, all environment pairs
>> +	GIT_CONFIG_KEY_<n> and GIT_CONFIG_VALUE_<n> up to that number will be
>> +	added to the process's runtime configuration. The config pairs are
>> +	zero-indexed. Any missing key or value is treated as an error. An empty
>> +	GIT_CONFIG_COUNT is treated the same as GIT_CONFIG_COUNT=0, namely no
>> +	pairs are processed. Config entries set this way have command scope,
>> +	but will be overridden by any explicit options passed via `git -c`.
>
> Perhaps work in some/all of some version of these:
>
>  - There's also a GIT_CONFIG_PARAMETERS variable, which is considered
>    internal to Git itself. Users are expected to set these.

"are", or "are not"?  I think it is the latter, and if so I agree
that it is a good thing to say here or somewhere nearby.

>    --> I.e. even if we're not going to support some format for
>    --> GIT_CONFIG_PARAMETERS document what it is.

My preference is to keep it an implementation detail, especially if
we were to be adding this new thing as a documented feature, so
documenting it beyond its existence and nature is counterproductive.

>  - This is analogous to the pre-receive `GIT_PUSH_OPTION_*` variables
>    (see linkgit:githooks[5]), but unlike those the `-c` option to
>    linkgit:git(1) does not set `GIT_CONFIG_*`.

I am slightly negative about this.  It would be an irrelevant noise
to readers who are interested in environment variables that affect
how "git config" works (which is what this section is about).  Also
for those who want to learn about GIT_PUSH_OPTION variable, I do not
think they would look for it in "git config" documentation and check
its ENVIRONMENT section.  It would be much more likely for them to
look for them in the documentation for receive-pack or push (and then
redirected to githooks doc).

>  - Saying "command scope" here I think is wrong/misleading. If I didn't
>    know how this worked I'd expect the first git process to see it to
>    delete it from the env, so e.g. the "fetch" command would see it, but
>    not the "gc" it spawned (different commands). Maybe just say "the
>    scope of these is as with other GIT_* environment variables, they'll
>    be inherited by subprocesses".

OK.

> Re the indent question to make the diff more readable question Junio
> had: could set some "do we have this or that" variables here to not
> reindent the existing code, but maybe not worth the effort...

I was leaving a clue for those who want to futz with "diff"
algorithm that this change can be a good test case for their
improvement.

I didn't mean that as a suggestion to help "diff" produce a better
result by twisting code.  We should not tweak our code to please
"git show" output.  Tweaking code to please "cat/less $file" output
is very much welcome, though.

Thanks.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 0/2] config: allow specifying config entries via envvar pairs
  2020-11-25 10:41 ` [PATCH v2 0/2] " Jeff King
  2020-11-25 13:16   ` Patrick Steinhardt
@ 2020-11-25 20:28   ` Junio C Hamano
  2020-11-25 22:47   ` brian m. carlson
  2020-12-01  9:47   ` Patrick Steinhardt
  3 siblings, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2020-11-25 20:28 UTC (permalink / raw)
  To: Jeff King
  Cc: Patrick Steinhardt, git, Ævar Arnfjörð Bjarmason,
	brian m. carlson, Philip Oakley

Jeff King <peff@peff.net> writes:

> I do still think that a "--config-env" option solves your problem in a
> much simpler way (especially in terms of interface we expose to users
> that we'll be locked into forever).

As a mechanism to allow a custom configuration for a single
invocation of a command, I tend to agree.  For a mechansim to affect
multiple commands in a sequence (read: scripts), I am not so sure.

The simplicity of the implementation we see below is also very
attractive.

> I sketched out the solution below if
> it's of interest (and I'd be happy to polish it up, or hand it off to
> you if so). But if you're unconvinced, I'll stop mentioning it.
>
> diff --git a/config.c b/config.c
> index 8f324ed3a6..d8cf6a5d6b 100644
> --- a/config.c
> +++ b/config.c
> @@ -345,6 +345,27 @@ void git_config_push_parameter(const char *text)
>  	strbuf_release(&env);
>  }
>  
> +void git_config_push_env(const char *spec)
> +{
> +	struct strbuf buf = STRBUF_INIT;
> +	const char *env_name;
> +	const char *env_value;
> +
> +	env_name = strchr(spec, '=');
> +	if (!env_name)
> +		return; /* die or warn? */
> +	env_name++;
> +
> +	env_value = getenv(env_name);
> +	if (!env_value)
> +		return; /* die or warn? */
> +
> +	strbuf_add(&buf, spec, env_name - spec);
> +	strbuf_addstr(&buf, env_value);
> +	git_config_push_parameter(buf.buf);
> +	strbuf_release(&buf);
> +}
> +
>  static inline int iskeychar(int c)
>  {
>  	return isalnum(c) || c == '-';
> diff --git a/config.h b/config.h
> index 91cdfbfb41..d05651c96c 100644
> --- a/config.h
> +++ b/config.h
> @@ -138,6 +138,7 @@ int git_config_from_mem(config_fn_t fn,
>  int git_config_from_blob_oid(config_fn_t fn, const char *name,
>  			     const struct object_id *oid, void *data);
>  void git_config_push_parameter(const char *text);
> +void git_config_push_env(const char *spec);
>  int git_config_from_parameters(config_fn_t fn, void *data);
>  void read_early_config(config_fn_t cb, void *data);
>  void read_very_early_config(config_fn_t cb, void *data);
> diff --git a/git.c b/git.c
> index 4b7bd77b80..342f2fb0c9 100644
> --- a/git.c
> +++ b/git.c
> @@ -254,6 +254,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
>  			git_config_push_parameter((*argv)[1]);
>  			(*argv)++;
>  			(*argc)--;
> +		} else if (skip_prefix(cmd, "--config-env=", &cmd)) {
> +			git_config_push_env(cmd);
>  		} else if (!strcmp(cmd, "--literal-pathspecs")) {
>  			setenv(GIT_LITERAL_PATHSPECS_ENVIRONMENT, "1", 1);
>  			if (envchanged)

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 0/2] config: allow specifying config entries via envvar pairs
  2020-11-25 10:41 ` [PATCH v2 0/2] " Jeff King
  2020-11-25 13:16   ` Patrick Steinhardt
  2020-11-25 20:28   ` Junio C Hamano
@ 2020-11-25 22:47   ` brian m. carlson
  2020-11-26  6:31     ` Patrick Steinhardt
  2020-12-01  9:47   ` Patrick Steinhardt
  3 siblings, 1 reply; 116+ messages in thread
From: brian m. carlson @ 2020-11-25 22:47 UTC (permalink / raw)
  To: Jeff King
  Cc: Patrick Steinhardt, git, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1525 bytes --]

On 2020-11-25 at 10:41:14, Jeff King wrote:
> On Tue, Nov 24, 2020 at 11:50:46AM +0100, Patrick Steinhardt wrote:
> 
> >     - I've changed priorities. The envvars are treated as command-level
> >       and as such override all values configured in files. But any
> >       explicit `git -c key=value` will now override these envvars.
> 
> That ordering makes sense. Those get passed through the environment,
> too, but at some point there is a process where your new ones are in the
> environment and the "-c" ones are on the command-line.

Yeah, I agree this would be the right way to go.

> I do still think that a "--config-env" option solves your problem in a
> much simpler way (especially in terms of interface we expose to users
> that we'll be locked into forever). I sketched out the solution below if
> it's of interest (and I'd be happy to polish it up, or hand it off to
> you if so). But if you're unconvinced, I'll stop mentioning it.

I do rather prefer this approach over the multiple key-value pairs.  I
think the use case of scripts could probably be easily solved with an
additional environment variable like so:

  args="--config-env abc.def=GHI --config-env jkl.mno=PQR"

This isn't necessarily super elegant, but I like it more than needing
to handle many key-value pairs.

But while I do have a moderately strong preference, I'm not going to
argue for blocking the series if you still want to go this way.
-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 0/2] config: allow specifying config entries via envvar pairs
  2020-11-25 13:16   ` Patrick Steinhardt
@ 2020-11-26  0:36     ` Jeff King
  0 siblings, 0 replies; 116+ messages in thread
From: Jeff King @ 2020-11-26  0:36 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Junio C Hamano,
	brian m. carlson, Philip Oakley

On Wed, Nov 25, 2020 at 02:16:38PM +0100, Patrick Steinhardt wrote:

> > I do still think that a "--config-env" option solves your problem in a
> > much simpler way (especially in terms of interface we expose to users
> > that we'll be locked into forever). I sketched out the solution below if
> > it's of interest (and I'd be happy to polish it up, or hand it off to
> > you if so). But if you're unconvinced, I'll stop mentioning it.
> 
> The thing I like more about using envvars only is that you only need to
> modify a single part, while with `--config-env` there's two moving
> parts. E.g. assume you have a script and want certain configuration to
> apply to all git commands in that script. It's trivial in the envvar
> case, while for `--config-env` you'll also have to modify each single
> git call. You could get around that by using a wrapper, but it's still a
> tad more involved. A second thing I briefly wondered about is the
> maximum command line length, which may be easier to hit in case you want
> to pass a lot of config entries.

Yeah, that's true. I haven't typically run across this myself because
usually such a script ends up invoked by git itself. I.e., it is
git-foo, and then I do:

  git -c some.var=value foo

which puts everything in the environment, but it's done by Git itself,
so the exact environment format remains opaque.

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 0/2] config: allow specifying config entries via envvar pairs
  2020-11-25 22:47   ` brian m. carlson
@ 2020-11-26  6:31     ` Patrick Steinhardt
  0 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-11-26  6:31 UTC (permalink / raw)
  To: brian m. carlson, Jeff King, git,
	Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1483 bytes --]

On Wed, Nov 25, 2020 at 10:47:37PM +0000, brian m. carlson wrote:
> On 2020-11-25 at 10:41:14, Jeff King wrote:
> > On Tue, Nov 24, 2020 at 11:50:46AM +0100, Patrick Steinhardt wrote:
> > I do still think that a "--config-env" option solves your problem in a
> > much simpler way (especially in terms of interface we expose to users
> > that we'll be locked into forever). I sketched out the solution below if
> > it's of interest (and I'd be happy to polish it up, or hand it off to
> > you if so). But if you're unconvinced, I'll stop mentioning it.
> 
> I do rather prefer this approach over the multiple key-value pairs.  I
> think the use case of scripts could probably be easily solved with an
> additional environment variable like so:
> 
>   args="--config-env abc.def=GHI --config-env jkl.mno=PQR"
> 
> This isn't necessarily super elegant, but I like it more than needing
> to handle many key-value pairs.
> 
> But while I do have a moderately strong preference, I'm not going to
> argue for blocking the series if you still want to go this way.

In the end, it probably boils down to taste. Both work to solve the
problem at hand while there are tradeoffs for other usecases for both.

Ultimately, those two ways are not mutually exclusive and we could even
implement both. So I might as well include Peffs patch in this series,
even though I'm not sure whether adding two new ways of doing things at
the same time would be welcome.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 0/2] config: allow specifying config entries via envvar pairs
  2020-11-25 10:41 ` [PATCH v2 0/2] " Jeff King
                     ` (2 preceding siblings ...)
  2020-11-25 22:47   ` brian m. carlson
@ 2020-12-01  9:47   ` Patrick Steinhardt
  2020-12-01 11:30     ` Jeff King
  3 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-01  9:47 UTC (permalink / raw)
  To: Jeff King
  Cc: git, Ævar Arnfjörð Bjarmason, Junio C Hamano,
	brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 3260 bytes --]

On Wed, Nov 25, 2020 at 05:41:14AM -0500, Jeff King wrote:
> On Tue, Nov 24, 2020 at 11:50:46AM +0100, Patrick Steinhardt wrote:
> 
> >     - I've changed priorities. The envvars are treated as command-level
> >       and as such override all values configured in files. But any
> >       explicit `git -c key=value` will now override these envvars.
> 
> That ordering makes sense. Those get passed through the environment,
> too, but at some point there is a process where your new ones are in the
> environment and the "-c" ones are on the command-line.
> 
> I do still think that a "--config-env" option solves your problem in a
> much simpler way (especially in terms of interface we expose to users
> that we'll be locked into forever). I sketched out the solution below if
> it's of interest (and I'd be happy to polish it up, or hand it off to
> you if so). But if you're unconvinced, I'll stop mentioning it.
> 
> diff --git a/config.c b/config.c
> index 8f324ed3a6..d8cf6a5d6b 100644
> --- a/config.c
> +++ b/config.c
> @@ -345,6 +345,27 @@ void git_config_push_parameter(const char *text)
>  	strbuf_release(&env);
>  }
>  
> +void git_config_push_env(const char *spec)
> +{
> +	struct strbuf buf = STRBUF_INIT;
> +	const char *env_name;
> +	const char *env_value;
> +
> +	env_name = strchr(spec, '=');
> +	if (!env_name)
> +		return; /* die or warn? */
> +	env_name++;
> +
> +	env_value = getenv(env_name);
> +	if (!env_value)
> +		return; /* die or warn? */
> +
> +	strbuf_add(&buf, spec, env_name - spec);
> +	strbuf_addstr(&buf, env_value);
> +	git_config_push_parameter(buf.buf);
> +	strbuf_release(&buf);
> +}

I realize that you say it's yet unpolished, but doesn't this have
parsing issues? The first strchr(3P) probably needs to be a strrchr(3P)
to correctly parse `includeIf./home/foo/=repo.path=MY_PATH_ENV`. But
we'd also have to handle shell quoting for the user, don't we?

Anyway, I'd be happy to adopt is as part of the series if we care
enough. For now I'll send out the current state I have though.

Patrick

>  static inline int iskeychar(int c)
>  {
>  	return isalnum(c) || c == '-';
> diff --git a/config.h b/config.h
> index 91cdfbfb41..d05651c96c 100644
> --- a/config.h
> +++ b/config.h
> @@ -138,6 +138,7 @@ int git_config_from_mem(config_fn_t fn,
>  int git_config_from_blob_oid(config_fn_t fn, const char *name,
>  			     const struct object_id *oid, void *data);
>  void git_config_push_parameter(const char *text);
> +void git_config_push_env(const char *spec);
>  int git_config_from_parameters(config_fn_t fn, void *data);
>  void read_early_config(config_fn_t cb, void *data);
>  void read_very_early_config(config_fn_t cb, void *data);
> diff --git a/git.c b/git.c
> index 4b7bd77b80..342f2fb0c9 100644
> --- a/git.c
> +++ b/git.c
> @@ -254,6 +254,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
>  			git_config_push_parameter((*argv)[1]);
>  			(*argv)++;
>  			(*argc)--;
> +		} else if (skip_prefix(cmd, "--config-env=", &cmd)) {
> +			git_config_push_env(cmd);
>  		} else if (!strcmp(cmd, "--literal-pathspecs")) {
>  			setenv(GIT_LITERAL_PATHSPECS_ENVIRONMENT, "1", 1);
>  			if (envchanged)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v3 0/4] config: allow specifying config entries via envvar pairs
  2020-11-24 10:50 [PATCH v2 0/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
                   ` (2 preceding siblings ...)
  2020-11-25 10:41 ` [PATCH v2 0/2] " Jeff King
@ 2020-12-01  9:55 ` Patrick Steinhardt
  2020-12-01  9:55   ` [PATCH v3 1/4] environment: make `getenv_safe()` non-static Patrick Steinhardt
                     ` (3 more replies)
  2020-12-09 11:52 ` [PATCH v4 0/6] config: allow specifying config entries via env Patrick Steinhardt
                   ` (4 subsequent siblings)
  8 siblings, 4 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-01  9:55 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1562 bytes --]

Hi,

this is the third version of my patch series which aims to implement a
way to pass config entries via the enviroment while avoiding any
requirements to perform shell quoting on the user's side.

Major changes include:

- Another test case to test interaction of GIT_CONFIG_PARAMETERS.

- I've exposed `getenv_safe` and now use that to retrieve envvars to
  avoid platform-specific lifetime issues of returned envvar values.

- I've split out a patch which performs reindentation of preexisting
  code to make the actual code change easier to review.

Still missing is the `--config-env` way of doing things. I'd be happy to
live with both ways of doing things and adopt it as part of this series,
but I wasn't sure whether this would be welcome or not. Too many ways to
do the same thing may be confusing in the end, even though their target
audience is probably different.

Patrick

Patrick Steinhardt (4):
  environment: make `getenv_safe()` non-static
  config: extract function to parse config pairs
  config: refactor parsing of GIT_CONFIG_PARAMETERS
  config: allow specifying config entries via envvar pairs

 Documentation/git-config.txt |  12 ++++
 cache.h                      |   1 +
 config.c                     |  99 +++++++++++++++++++++++-------
 environment.c                |   8 +--
 environment.h                |  12 ++++
 t/t1300-config.sh            | 115 ++++++++++++++++++++++++++++++++++-
 6 files changed, 220 insertions(+), 27 deletions(-)
 create mode 100644 environment.h

-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v3 1/4] environment: make `getenv_safe()` non-static
  2020-12-01  9:55 ` [PATCH v3 0/4] " Patrick Steinhardt
@ 2020-12-01  9:55   ` Patrick Steinhardt
  2020-12-01  9:56   ` [PATCH v3 2/4] config: extract function to parse config pairs Patrick Steinhardt
                     ` (2 subsequent siblings)
  3 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-01  9:55 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1661 bytes --]

The `getenv_safe()` helper function helps to safely retrieve multiple
environment values without the need to depend on platform-specific
behaviour for the return value's lifetime. We'll make use of this
function in a following patch, so let's make it available by making it
non-static and adding a declaration.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 environment.c |  7 ++-----
 environment.h | 12 ++++++++++++
 2 files changed, 14 insertions(+), 5 deletions(-)
 create mode 100644 environment.h

diff --git a/environment.c b/environment.c
index bb518c61cd..2234af462c 100644
--- a/environment.c
+++ b/environment.c
@@ -9,6 +9,7 @@
  */
 #include "cache.h"
 #include "branch.h"
+#include "environment.h"
 #include "repository.h"
 #include "config.h"
 #include "refs.h"
@@ -152,11 +153,7 @@ static char *expand_namespace(const char *raw_namespace)
 	return strbuf_detach(&buf, NULL);
 }
 
-/*
- * Wrapper of getenv() that returns a strdup value. This value is kept
- * in argv to be freed later.
- */
-static const char *getenv_safe(struct strvec *argv, const char *name)
+const char *getenv_safe(struct strvec *argv, const char *name)
 {
 	const char *value = getenv(name);
 
diff --git a/environment.h b/environment.h
new file mode 100644
index 0000000000..d438b5c8f3
--- /dev/null
+++ b/environment.h
@@ -0,0 +1,12 @@
+#ifndef ENVIRONMENT_H
+#define ENVIRONMENT_H
+
+#include "strvec.h"
+
+/*
+ * Wrapper of getenv() that returns a strdup value. This value is kept
+ * in argv to be freed later.
+ */
+const char *getenv_safe(struct strvec *argv, const char *name);
+
+#endif
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v3 2/4] config: extract function to parse config pairs
  2020-12-01  9:55 ` [PATCH v3 0/4] " Patrick Steinhardt
  2020-12-01  9:55   ` [PATCH v3 1/4] environment: make `getenv_safe()` non-static Patrick Steinhardt
@ 2020-12-01  9:56   ` Patrick Steinhardt
  2020-12-01  9:56   ` [PATCH v3 3/4] config: refactor parsing of GIT_CONFIG_PARAMETERS Patrick Steinhardt
  2020-12-01  9:56   ` [PATCH v3 4/4] config: allow specifying config entries via envvar pairs Patrick Steinhardt
  3 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-01  9:56 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1819 bytes --]

The function `git_config_parse_parameter` is responsible for parsing a
`foo.bar=baz`-formatted configuration key, sanitizing the key and then
processing it via the given callback function. Given that we're about to
add a second user which is going to process keys in such which already
has keys and values separated, this commit extracts a function
`config_parse_pair` which only does the sanitization and processing
part.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 config.c | 24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/config.c b/config.c
index 8f324ed3a6..4ae3711d3d 100644
--- a/config.c
+++ b/config.c
@@ -437,11 +437,26 @@ int git_config_key_is_valid(const char *key)
 	return !git_config_parse_key_1(key, NULL, NULL, 1);
 }
 
+static int config_parse_pair(const char *key, const char *value,
+			  config_fn_t fn, void *data)
+{
+	char *canonical_name;
+	int ret;
+
+	if (!strlen(key))
+		return error(_("empty config key"));
+	if (git_config_parse_key(key, &canonical_name, NULL))
+		return -1;
+
+	ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
+	free(canonical_name);
+	return ret;
+}
+
 int git_config_parse_parameter(const char *text,
 			       config_fn_t fn, void *data)
 {
 	const char *value;
-	char *canonical_name;
 	struct strbuf **pair;
 	int ret;
 
@@ -462,12 +477,7 @@ int git_config_parse_parameter(const char *text,
 		return error(_("bogus config parameter: %s"), text);
 	}
 
-	if (git_config_parse_key(pair[0]->buf, &canonical_name, NULL)) {
-		ret = -1;
-	} else {
-		ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
-		free(canonical_name);
-	}
+	ret = config_parse_pair(pair[0]->buf, value, fn, data);
 	strbuf_list_free(pair);
 	return ret;
 }
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v3 3/4] config: refactor parsing of GIT_CONFIG_PARAMETERS
  2020-12-01  9:55 ` [PATCH v3 0/4] " Patrick Steinhardt
  2020-12-01  9:55   ` [PATCH v3 1/4] environment: make `getenv_safe()` non-static Patrick Steinhardt
  2020-12-01  9:56   ` [PATCH v3 2/4] config: extract function to parse config pairs Patrick Steinhardt
@ 2020-12-01  9:56   ` Patrick Steinhardt
  2020-12-01  9:56   ` [PATCH v3 4/4] config: allow specifying config entries via envvar pairs Patrick Steinhardt
  3 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-01  9:56 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 2106 bytes --]

We're about to introduce a new way of passing parameters via environment
variables to git, which will require us to change the way we parse
config entries from parameters. Currently, `git_config_from_parameters`
is written in a way which makes it rather hard to extend.

Refactor the function to make it ready for the new logic as a
preparatory step in order to avoid reindenting code and adding new logic
in the same step, which would be much harder to reason about. This
refactoring is not expected to change any behaviour.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 config.c | 31 ++++++++++++++++---------------
 1 file changed, 16 insertions(+), 15 deletions(-)

diff --git a/config.c b/config.c
index 4ae3711d3d..ba67706854 100644
--- a/config.c
+++ b/config.c
@@ -484,35 +484,36 @@ int git_config_parse_parameter(const char *text,
 
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
-	const char *env = getenv(CONFIG_DATA_ENVIRONMENT);
+	const char *env;
 	int ret = 0;
-	char *envw;
+	char *envw = NULL;
 	const char **argv = NULL;
-	int nr = 0, alloc = 0;
 	int i;
 	struct config_source source;
 
-	if (!env)
-		return 0;
-
 	memset(&source, 0, sizeof(source));
 	source.prev = cf;
 	source.origin_type = CONFIG_ORIGIN_CMDLINE;
 	cf = &source;
 
-	/* sq_dequote will write over it */
-	envw = xstrdup(env);
+	env = getenv(CONFIG_DATA_ENVIRONMENT);
+	if (env) {
+		int nr = 0, alloc = 0;
 
-	if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
-		ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
-		goto out;
-	}
+		/* sq_dequote will write over it */
+		envw = xstrdup(env);
 
-	for (i = 0; i < nr; i++) {
-		if (git_config_parse_parameter(argv[i], fn, data) < 0) {
-			ret = -1;
+		if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
+			ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
 			goto out;
 		}
+
+		for (i = 0; i < nr; i++) {
+			if (git_config_parse_parameter(argv[i], fn, data) < 0) {
+				ret = -1;
+				goto out;
+			}
+		}
 	}
 
 out:
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v3 4/4] config: allow specifying config entries via envvar pairs
  2020-12-01  9:55 ` [PATCH v3 0/4] " Patrick Steinhardt
                     ` (2 preceding siblings ...)
  2020-12-01  9:56   ` [PATCH v3 3/4] config: refactor parsing of GIT_CONFIG_PARAMETERS Patrick Steinhardt
@ 2020-12-01  9:56   ` Patrick Steinhardt
  3 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-01  9:56 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 10022 bytes --]

While we currently have the `GIT_CONFIG_PARAMETERS` environment variable
which can be used to pass runtime configuration data to git processes,
it's an internal implementation detail and not supposed to be used by
end users.

Next to being for internal use only, this way of passing config entries
has a major downside: the config keys need to be parsed as they contain
both key and value in a single variable. As such, it is left to the user
to escape any potentially harmful characters in the value, which is
quite hard to do if values are controlled by a third party.

This commit thus adds a new way of adding config entries via the
environment which gets rid of this shortcoming. If the user passes the
`GIT_CONFIG_COUNT=$n` environment variable, Git will parse environment
variable pairs `GIT_CONFIG_KEY_$i` and `GIT_CONFIG_VALUE_$i` for each
`i` in `[0,n)`.

While the same can be achieved with `git -c <name>=<value>`, one may
wish to not do so for potentially sensitive information. E.g. if one
wants to set `http.extraHeader` to contain an authentication token,
doing so via `-c` would trivially leak those credentials via e.g. ps(1),
which typically also shows command arguments.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git-config.txt |  12 ++++
 cache.h                      |   1 +
 config.c                     |  46 ++++++++++++++
 environment.c                |   1 +
 t/t1300-config.sh            | 115 ++++++++++++++++++++++++++++++++++-
 5 files changed, 174 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt
index 7573160f21..073fb5229a 100644
--- a/Documentation/git-config.txt
+++ b/Documentation/git-config.txt
@@ -337,6 +337,18 @@ GIT_CONFIG_NOSYSTEM::
 
 See also <<FILES>>.
 
+GIT_CONFIG_COUNT::
+GIT_CONFIG_KEY_<n>::
+GIT_CONFIG_VALUE_<n>::
+	If GIT_CONFIG_COUNT is set to a positive number, all environment pairs
+	GIT_CONFIG_KEY_<n> and GIT_CONFIG_VALUE_<n> up to that number will be
+	added to the process's runtime configuration. The config pairs are
+	zero-indexed. Any missing key or value is treated as an error. An empty
+	GIT_CONFIG_COUNT is treated the same as GIT_CONFIG_COUNT=0, namely no
+	pairs are processed. These environment variables will override values
+	in configuration files, but will be overridden by any explicit options
+	passed via `git -c`.
+
 
 [[EXAMPLES]]
 EXAMPLES
diff --git a/cache.h b/cache.h
index c0072d43b1..8a36146337 100644
--- a/cache.h
+++ b/cache.h
@@ -472,6 +472,7 @@ static inline enum object_type object_type(unsigned int mode)
 #define TEMPLATE_DIR_ENVIRONMENT "GIT_TEMPLATE_DIR"
 #define CONFIG_ENVIRONMENT "GIT_CONFIG"
 #define CONFIG_DATA_ENVIRONMENT "GIT_CONFIG_PARAMETERS"
+#define CONFIG_COUNT_ENVIRONMENT "GIT_CONFIG_COUNT"
 #define EXEC_PATH_ENVIRONMENT "GIT_EXEC_PATH"
 #define CEILING_DIRECTORIES_ENVIRONMENT "GIT_CEILING_DIRECTORIES"
 #define NO_REPLACE_OBJECTS_ENVIRONMENT "GIT_NO_REPLACE_OBJECTS"
diff --git a/config.c b/config.c
index ba67706854..0b4d0b45d1 100644
--- a/config.c
+++ b/config.c
@@ -8,6 +8,7 @@
 #include "cache.h"
 #include "branch.h"
 #include "config.h"
+#include "environment.h"
 #include "repository.h"
 #include "lockfile.h"
 #include "exec-cmd.h"
@@ -485,6 +486,8 @@ int git_config_parse_parameter(const char *text,
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
 	const char *env;
+	struct strbuf envvar = STRBUF_INIT;
+	struct strvec to_free = STRVEC_INIT;
 	int ret = 0;
 	char *envw = NULL;
 	const char **argv = NULL;
@@ -496,6 +499,47 @@ int git_config_from_parameters(config_fn_t fn, void *data)
 	source.origin_type = CONFIG_ORIGIN_CMDLINE;
 	cf = &source;
 
+	env = getenv(CONFIG_COUNT_ENVIRONMENT);
+	if (env) {
+		unsigned long count;
+		char *endp;
+
+		count = strtoul(env, &endp, 10);
+		if (*endp) {
+			ret = error(_("bogus count in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+		if (count > INT_MAX) {
+			ret = error(_("too many entries in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+
+		for (i = 0; i < count; i++) {
+			const char *key, *value;
+
+			strbuf_addf(&envvar, "GIT_CONFIG_KEY_%d", i);
+			key = getenv_safe(&to_free, envvar.buf);
+			if (!key) {
+				ret = error(_("missing config key %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			strbuf_addf(&envvar, "GIT_CONFIG_VALUE_%d", i);
+			value = getenv_safe(&to_free, envvar.buf);
+			if (!value) {
+				ret = error(_("missing config value %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			if (config_parse_pair(key, value, fn, data) < 0) {
+				ret = -1;
+				goto out;
+			}
+		}
+	}
+
 	env = getenv(CONFIG_DATA_ENVIRONMENT);
 	if (env) {
 		int nr = 0, alloc = 0;
@@ -517,6 +561,8 @@ int git_config_from_parameters(config_fn_t fn, void *data)
 	}
 
 out:
+	strbuf_release(&envvar);
+	strvec_clear(&to_free);
 	free(argv);
 	free(envw);
 	cf = source.prev;
diff --git a/environment.c b/environment.c
index 2234af462c..2f27008424 100644
--- a/environment.c
+++ b/environment.c
@@ -117,6 +117,7 @@ const char * const local_repo_env[] = {
 	ALTERNATE_DB_ENVIRONMENT,
 	CONFIG_ENVIRONMENT,
 	CONFIG_DATA_ENVIRONMENT,
+	CONFIG_COUNT_ENVIRONMENT,
 	DB_ENVIRONMENT,
 	GIT_DIR_ENVIRONMENT,
 	GIT_WORK_TREE_ENVIRONMENT,
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 825d9a184f..756536067b 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1316,6 +1316,117 @@ test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
 		git config --get-regexp "env.*"
 '
 
+test_expect_success 'git config handles environment config pairs' '
+	GIT_CONFIG_COUNT=2 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="foo" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="bar" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one foo
+	pair.two bar
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs without count' '
+	test_must_fail env GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one 2>error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one
+'
+
+test_expect_success 'git config ignores pairs exceeding count' '
+	GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="value" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with empty count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT= GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config fails with invalid count' '
+	test_must_fail env GIT_CONFIG_COUNT=10a git config --list 2>error &&
+	test_i18ngrep "bogus count" error &&
+	test_must_fail env GIT_CONFIG_COUNT=9999999999999999 git config --list 2>error &&
+	test_i18ngrep "too many entries" error
+'
+
+test_expect_success 'git config fails with missing config key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_VALUE_0="value" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config key" error
+'
+
+test_expect_success 'git config fails with missing config value' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0="pair.one" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config value" error
+'
+
+test_expect_success 'git config fails with invalid config pair key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0= GIT_CONFIG_VALUE_0=value \
+		git config --list &&
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0=missing-section GIT_CONFIG_VALUE_0=value \
+		git config --list
+'
+
+test_expect_success 'environment overrides config file' '
+	test_when_finished "rm -f .git/config" &&
+	cat >.git/config <<-EOF &&
+	[pair]
+	one = value
+	EOF
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=override \
+		git config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'GIT_CONFIG_PARAMETERS overrides environment config' '
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
+		GIT_CONFIG_PARAMETERS="${SQ}pair.one=override${SQ}" \
+		git config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'command line overrides environment config' '
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
+		git -c pair.one=override config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
@@ -1661,9 +1772,11 @@ test_expect_success '--show-origin with --list' '
 	file:.git/config	user.override=local
 	file:.git/config	include.path=../include/relative.include
 	file:.git/../include/relative.include	user.relative=include
+	command line:	user.environ=true
 	command line:	user.cmdline=true
 	EOF
-	git -c user.cmdline=true config --list --show-origin >output &&
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=user.environ GIT_CONFIG_VALUE_0=true\
+		git -c user.cmdline=true config --list --show-origin >output &&
 	test_cmp expect output
 '
 
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v2 0/2] config: allow specifying config entries via envvar pairs
  2020-12-01  9:47   ` Patrick Steinhardt
@ 2020-12-01 11:30     ` Jeff King
  0 siblings, 0 replies; 116+ messages in thread
From: Jeff King @ 2020-12-01 11:30 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Junio C Hamano,
	brian m. carlson, Philip Oakley

On Tue, Dec 01, 2020 at 10:47:56AM +0100, Patrick Steinhardt wrote:

> > +void git_config_push_env(const char *spec)
> > +{
> > +	struct strbuf buf = STRBUF_INIT;
> > +	const char *env_name;
> > +	const char *env_value;
> > +
> > +	env_name = strchr(spec, '=');
> > +	if (!env_name)
> > +		return; /* die or warn? */
> > +	env_name++;
> > +
> > +	env_value = getenv(env_name);
> > +	if (!env_value)
> > +		return; /* die or warn? */
> > +
> > +	strbuf_add(&buf, spec, env_name - spec);
> > +	strbuf_addstr(&buf, env_value);
> > +	git_config_push_parameter(buf.buf);
> > +	strbuf_release(&buf);
> > +}
> 
> I realize that you say it's yet unpolished, but doesn't this have
> parsing issues? The first strchr(3P) probably needs to be a strrchr(3P)
> to correctly parse `includeIf./home/foo/=repo.path=MY_PATH_ENV`.

Without further changes to $GIT_CONFIG_PARAMETERS, there'd be little
point. The value we put in there has the same parsing issue when read
out of the environment (which we resolve by disallowing "=" in the
subsection, just as here).

I don't think it's actually that big of a deal in practice (it _could_
be an injection source, but it seems somewhat implausible that somebody
is generating complex config keys based on untrusted input). But if we
care, then we could pretty easily change the reading side to separately
quote the key/value in this case:

  'foo.subsection=with=equals.bar'='value=with=equals'

And then doing strrchr() would make sense, with the explicitly
documented rule that the environment variable name cannot contain an
equals sign. (Doing a raw "git -c" wouldn't work unless we introduce
another option that lets you specify the key and value separately; that
might be worthwhile, too).

> But we'd also have to handle shell quoting for the user, don't we?

I'm not sure exactly what you mean here. We wouldn't typically see any
shell quoting from the user, since the shell would dequote it and give
us a NUL-terminated argv. Or if you meant we'd have to adjust the shell
quoting in $GIT_CONFIG_PARAMETERS, then see above.

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v4 0/6] config: allow specifying config entries via env
  2020-11-24 10:50 [PATCH v2 0/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
                   ` (3 preceding siblings ...)
  2020-12-01  9:55 ` [PATCH v3 0/4] " Patrick Steinhardt
@ 2020-12-09 11:52 ` Patrick Steinhardt
  2020-12-09 11:52   ` [PATCH v4 1/6] git: add `--super-prefix` to usage string Patrick Steinhardt
                     ` (6 more replies)
  2020-12-16  7:52 ` [PATCH v5 0/8] " Patrick Steinhardt
                   ` (3 subsequent siblings)
  8 siblings, 7 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-09 11:52 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1851 bytes --]

Hi,

this is the fourth version of my patch series which aims to implement a
way to pass config entries via the environment while avoiding any
requirements to perform shell quoting on the user's side.

Given that the What's Cooking report notes that my third version is
about to be dropped dropped because the `--config-env` way of doing
things is preferred, I've now adopted that approach. I've taken the
patch which Peff posted originally (with one change strchr->strrchr) and
added documentation and tests to it.

This patch series still includes my old proposal as it would actually be
a better fit for our usecase at GitLab I have in mind, which is to put
all configuration which applies to all git commands into the commands
instead of using a config file for this. I have structured the series in
such a way though that those patches come last -- so if you continue to
think this approach shouldn't make it in, please feel free to drop
patches 3-6.

Patrick

Patrick Steinhardt (6):
  git: add `--super-prefix` to usage string
  config: add new way to pass config via `--config-env`
  environment: make `getenv_safe()` non-static
  config: extract function to parse config pairs
  config: refactor parsing of GIT_CONFIG_PARAMETERS
  config: allow specifying config entries via envvar pairs

 Documentation/git-config.txt |  12 +++
 Documentation/git.txt        |  11 ++-
 cache.h                      |   1 +
 config.c                     | 120 +++++++++++++++++++++-----
 config.h                     |   1 +
 environment.c                |   8 +-
 environment.h                |  12 +++
 git.c                        |   3 +
 t/t1300-config.sh            | 160 ++++++++++++++++++++++++++++++++++-
 9 files changed, 300 insertions(+), 28 deletions(-)
 create mode 100644 environment.h

-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v4 1/6] git: add `--super-prefix` to usage string
  2020-12-09 11:52 ` [PATCH v4 0/6] config: allow specifying config entries via env Patrick Steinhardt
@ 2020-12-09 11:52   ` Patrick Steinhardt
  2020-12-09 11:52   ` [PATCH v4 2/6] config: add new way to pass config via `--config-env` Patrick Steinhardt
                     ` (5 subsequent siblings)
  6 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-09 11:52 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1057 bytes --]

When the `--super-prefix` option was implmented in 74866d7579 (git: make
super-prefix option, 2016-10-07), its existence was only documented in
the manpage but not in the command's own usage string. Given that the
commit message didn't mention that this was done intentionally and given
that it's documented in the manpage, this seems like an oversight.

Add it to the usage string to fix the inconsistency.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 git.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/git.c b/git.c
index a00a0a4d94..5a8ff12f87 100644
--- a/git.c
+++ b/git.c
@@ -29,6 +29,7 @@ const char git_usage_string[] =
 	   "           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]\n"
 	   "           [-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]\n"
 	   "           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]\n"
+	   "           [--super-prefix=<path>]\n"
 	   "           <command> [<args>]");
 
 const char git_more_info_string[] =
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v4 2/6] config: add new way to pass config via `--config-env`
  2020-12-09 11:52 ` [PATCH v4 0/6] config: allow specifying config entries via env Patrick Steinhardt
  2020-12-09 11:52   ` [PATCH v4 1/6] git: add `--super-prefix` to usage string Patrick Steinhardt
@ 2020-12-09 11:52   ` Patrick Steinhardt
  2020-12-09 14:40     ` Ævar Arnfjörð Bjarmason
  2020-12-09 16:10     ` Jeff King
  2020-12-09 11:52   ` [PATCH v4 3/6] environment: make `getenv_safe()` non-static Patrick Steinhardt
                     ` (4 subsequent siblings)
  6 siblings, 2 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-09 11:52 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 6499 bytes --]

While it's already possible to pass runtime configuration via `git -c
<key>=<value>`, it may be undesirable to use when the value contains
sensitive information. E.g. if one wants to set `http.extraHeader` to
contain an authentication token, doing so via `-c` would trivially leak
those credentials via e.g. ps(1), which typically also shows command
arguments.

To enable this usecase without leaking credentials, this commit
introduces a new switch `--config-env=<key>=<envvar>`. Instead of
directly passing a value for the given key, it instead allows the user
to specify the name of an environment variable. The value of that
variable will then be used as value of the key.

Co-authored-by: Jeff King <peff@peff.net>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git.txt | 11 ++++++++++-
 config.c              | 21 ++++++++++++++++++++
 config.h              |  1 +
 git.c                 |  4 +++-
 t/t1300-config.sh     | 45 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 80 insertions(+), 2 deletions(-)

diff --git a/Documentation/git.txt b/Documentation/git.txt
index c463b937a8..9061c54792 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -13,7 +13,7 @@ SYNOPSIS
     [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
     [-p|--paginate|-P|--no-pager] [--no-replace-objects] [--bare]
     [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
-    [--super-prefix=<path>]
+    [--super-prefix=<path>] [--config-env <name>=<envvar>]
     <command> [<args>]
 
 DESCRIPTION
@@ -80,6 +80,15 @@ config file). Including the equals but with an empty value (like `git -c
 foo.bar= ...`) sets `foo.bar` to the empty string which `git config
 --type=bool` will convert to `false`.
 
+--config-env=<name>=<envvar>::
+	Pass a configuration parameter to the command. The <envvar>
+	given will be replaced with the contents of the environment
+	variable of that name. In contrast to `-c`, an envvar must
+	always be given and exist in the environment. Passing an
+	environment variable with empty value will set <name> to the
+	empty string which `git config --type=bool` will convert to
+	`false`.
+
 --exec-path[=<path>]::
 	Path to wherever your core Git programs are installed.
 	This can also be controlled by setting the GIT_EXEC_PATH
diff --git a/config.c b/config.c
index 1137bd73af..cde3511110 100644
--- a/config.c
+++ b/config.c
@@ -345,6 +345,27 @@ void git_config_push_parameter(const char *text)
 	strbuf_release(&env);
 }
 
+void git_config_push_env(const char *spec)
+{
+	struct strbuf buf = STRBUF_INIT;
+	const char *env_name;
+	const char *env_value;
+
+	env_name = strrchr(spec, '=');
+	if (!env_name)
+		die("invalid config format: %s", spec);
+	env_name++;
+
+	env_value = getenv(env_name);
+	if (!env_value)
+		die("config variable missing for '%s'", env_name);
+
+	strbuf_add(&buf, spec, env_name - spec);
+	strbuf_addstr(&buf, env_value);
+	git_config_push_parameter(buf.buf);
+	strbuf_release(&buf);
+}
+
 static inline int iskeychar(int c)
 {
 	return isalnum(c) || c == '-';
diff --git a/config.h b/config.h
index c1449bb790..19a9adbaa9 100644
--- a/config.h
+++ b/config.h
@@ -138,6 +138,7 @@ int git_config_from_mem(config_fn_t fn,
 int git_config_from_blob_oid(config_fn_t fn, const char *name,
 			     const struct object_id *oid, void *data);
 void git_config_push_parameter(const char *text);
+void git_config_push_env(const char *spec);
 int git_config_from_parameters(config_fn_t fn, void *data);
 void read_early_config(config_fn_t cb, void *data);
 void read_very_early_config(config_fn_t cb, void *data);
diff --git a/git.c b/git.c
index 5a8ff12f87..b5f63d346b 100644
--- a/git.c
+++ b/git.c
@@ -29,7 +29,7 @@ const char git_usage_string[] =
 	   "           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]\n"
 	   "           [-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]\n"
 	   "           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]\n"
-	   "           [--super-prefix=<path>]\n"
+	   "           [--super-prefix=<path>] [--config-env=<name>=<envvar>]\n"
 	   "           <command> [<args>]");
 
 const char git_more_info_string[] =
@@ -255,6 +255,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			git_config_push_parameter((*argv)[1]);
 			(*argv)++;
 			(*argc)--;
+		} else if (skip_prefix(cmd, "--config-env=", &cmd)) {
+			git_config_push_env(cmd);
 		} else if (!strcmp(cmd, "--literal-pathspecs")) {
 			setenv(GIT_LITERAL_PATHSPECS_ENVIRONMENT, "1", 1);
 			if (envchanged)
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 97a04c6cc2..46a94814d5 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1316,6 +1316,51 @@ test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
 		git config --get-regexp "env.*"
 '
 
+test_expect_success 'git --config-env=key=envvar support' '
+	cat >expect <<-\EOF &&
+	value
+	value
+	false
+	EOF
+	{
+		env ENVVAR=value git --config-env=core.name=ENVVAR config core.name &&
+		env ENVVAR=value git --config-env=foo.CamelCase=ENVVAR config foo.camelcase &&
+		env ENVVAR= git --config-env=foo.flag=ENVVAR config --bool foo.flag
+	} >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git --config-env fails with invalid parameters' '
+	test_must_fail git --config-env=foo.flag config --bool foo.flag 2>error &&
+	test_i18ngrep "invalid config format" error &&
+	test_must_fail git --config-env=foo.flag=NONEXISTENT config --bool foo.flag 2>error &&
+	test_i18ngrep "config variable missing" error
+'
+
+test_expect_success 'git -c and --config-env work together' '
+	cat >expect <<-\EOF &&
+	bar.cmd cmd-value
+	bar.env env-value
+	EOF
+	env ENVVAR=env-value git \
+		-c bar.cmd=cmd-value \
+		--config-env=bar.env=ENVVAR \
+		config --get-regexp "^bar.*" >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git -c and --config-env override each other' '
+	cat >expect <<-\EOF &&
+	env
+	cmd
+	EOF
+	{
+		env ENVVAR=env git -c bar.bar=cmd --config-env=bar.bar=ENVVAR config bar.bar &&
+		env ENVVAR=env git --config-env=bar.bar=ENVVAR -c bar.bar=cmd config bar.bar
+	} >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v4 3/6] environment: make `getenv_safe()` non-static
  2020-12-09 11:52 ` [PATCH v4 0/6] config: allow specifying config entries via env Patrick Steinhardt
  2020-12-09 11:52   ` [PATCH v4 1/6] git: add `--super-prefix` to usage string Patrick Steinhardt
  2020-12-09 11:52   ` [PATCH v4 2/6] config: add new way to pass config via `--config-env` Patrick Steinhardt
@ 2020-12-09 11:52   ` Patrick Steinhardt
  2020-12-09 11:52   ` [PATCH v4 4/6] config: extract function to parse config pairs Patrick Steinhardt
                     ` (3 subsequent siblings)
  6 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-09 11:52 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1661 bytes --]

The `getenv_safe()` helper function helps to safely retrieve multiple
environment values without the need to depend on platform-specific
behaviour for the return value's lifetime. We'll make use of this
function in a following patch, so let's make it available by making it
non-static and adding a declaration.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 environment.c |  7 ++-----
 environment.h | 12 ++++++++++++
 2 files changed, 14 insertions(+), 5 deletions(-)
 create mode 100644 environment.h

diff --git a/environment.c b/environment.c
index bb518c61cd..2234af462c 100644
--- a/environment.c
+++ b/environment.c
@@ -9,6 +9,7 @@
  */
 #include "cache.h"
 #include "branch.h"
+#include "environment.h"
 #include "repository.h"
 #include "config.h"
 #include "refs.h"
@@ -152,11 +153,7 @@ static char *expand_namespace(const char *raw_namespace)
 	return strbuf_detach(&buf, NULL);
 }
 
-/*
- * Wrapper of getenv() that returns a strdup value. This value is kept
- * in argv to be freed later.
- */
-static const char *getenv_safe(struct strvec *argv, const char *name)
+const char *getenv_safe(struct strvec *argv, const char *name)
 {
 	const char *value = getenv(name);
 
diff --git a/environment.h b/environment.h
new file mode 100644
index 0000000000..d438b5c8f3
--- /dev/null
+++ b/environment.h
@@ -0,0 +1,12 @@
+#ifndef ENVIRONMENT_H
+#define ENVIRONMENT_H
+
+#include "strvec.h"
+
+/*
+ * Wrapper of getenv() that returns a strdup value. This value is kept
+ * in argv to be freed later.
+ */
+const char *getenv_safe(struct strvec *argv, const char *name);
+
+#endif
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v4 4/6] config: extract function to parse config pairs
  2020-12-09 11:52 ` [PATCH v4 0/6] config: allow specifying config entries via env Patrick Steinhardt
                     ` (2 preceding siblings ...)
  2020-12-09 11:52   ` [PATCH v4 3/6] environment: make `getenv_safe()` non-static Patrick Steinhardt
@ 2020-12-09 11:52   ` Patrick Steinhardt
  2020-12-09 13:12     ` Ævar Arnfjörð Bjarmason
  2020-12-09 11:52   ` [PATCH v4 5/6] config: refactor parsing of GIT_CONFIG_PARAMETERS Patrick Steinhardt
                     ` (2 subsequent siblings)
  6 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-09 11:52 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1819 bytes --]

The function `git_config_parse_parameter` is responsible for parsing a
`foo.bar=baz`-formatted configuration key, sanitizing the key and then
processing it via the given callback function. Given that we're about to
add a second user which is going to process keys in such which already
has keys and values separated, this commit extracts a function
`config_parse_pair` which only does the sanitization and processing
part.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 config.c | 24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/config.c b/config.c
index cde3511110..151980e5c9 100644
--- a/config.c
+++ b/config.c
@@ -458,11 +458,26 @@ int git_config_key_is_valid(const char *key)
 	return !git_config_parse_key_1(key, NULL, NULL, 1);
 }
 
+static int config_parse_pair(const char *key, const char *value,
+			  config_fn_t fn, void *data)
+{
+	char *canonical_name;
+	int ret;
+
+	if (!strlen(key))
+		return error(_("empty config key"));
+	if (git_config_parse_key(key, &canonical_name, NULL))
+		return -1;
+
+	ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
+	free(canonical_name);
+	return ret;
+}
+
 int git_config_parse_parameter(const char *text,
 			       config_fn_t fn, void *data)
 {
 	const char *value;
-	char *canonical_name;
 	struct strbuf **pair;
 	int ret;
 
@@ -483,12 +498,7 @@ int git_config_parse_parameter(const char *text,
 		return error(_("bogus config parameter: %s"), text);
 	}
 
-	if (git_config_parse_key(pair[0]->buf, &canonical_name, NULL)) {
-		ret = -1;
-	} else {
-		ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
-		free(canonical_name);
-	}
+	ret = config_parse_pair(pair[0]->buf, value, fn, data);
 	strbuf_list_free(pair);
 	return ret;
 }
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v4 5/6] config: refactor parsing of GIT_CONFIG_PARAMETERS
  2020-12-09 11:52 ` [PATCH v4 0/6] config: allow specifying config entries via env Patrick Steinhardt
                     ` (3 preceding siblings ...)
  2020-12-09 11:52   ` [PATCH v4 4/6] config: extract function to parse config pairs Patrick Steinhardt
@ 2020-12-09 11:52   ` Patrick Steinhardt
  2020-12-09 11:52   ` [PATCH v4 6/6] config: allow specifying config entries via envvar pairs Patrick Steinhardt
  2020-12-09 15:29   ` [PATCH v4 0/6] config: allow specifying config entries via env Ævar Arnfjörð Bjarmason
  6 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-09 11:52 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 2106 bytes --]

We're about to introduce a new way of passing parameters via environment
variables to git, which will require us to change the way we parse
config entries from parameters. Currently, `git_config_from_parameters`
is written in a way which makes it rather hard to extend.

Refactor the function to make it ready for the new logic as a
preparatory step in order to avoid reindenting code and adding new logic
in the same step, which would be much harder to reason about. This
refactoring is not expected to change any behaviour.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 config.c | 31 ++++++++++++++++---------------
 1 file changed, 16 insertions(+), 15 deletions(-)

diff --git a/config.c b/config.c
index 151980e5c9..8162f3cec8 100644
--- a/config.c
+++ b/config.c
@@ -505,35 +505,36 @@ int git_config_parse_parameter(const char *text,
 
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
-	const char *env = getenv(CONFIG_DATA_ENVIRONMENT);
+	const char *env;
 	int ret = 0;
-	char *envw;
+	char *envw = NULL;
 	const char **argv = NULL;
-	int nr = 0, alloc = 0;
 	int i;
 	struct config_source source;
 
-	if (!env)
-		return 0;
-
 	memset(&source, 0, sizeof(source));
 	source.prev = cf;
 	source.origin_type = CONFIG_ORIGIN_CMDLINE;
 	cf = &source;
 
-	/* sq_dequote will write over it */
-	envw = xstrdup(env);
+	env = getenv(CONFIG_DATA_ENVIRONMENT);
+	if (env) {
+		int nr = 0, alloc = 0;
 
-	if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
-		ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
-		goto out;
-	}
+		/* sq_dequote will write over it */
+		envw = xstrdup(env);
 
-	for (i = 0; i < nr; i++) {
-		if (git_config_parse_parameter(argv[i], fn, data) < 0) {
-			ret = -1;
+		if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
+			ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
 			goto out;
 		}
+
+		for (i = 0; i < nr; i++) {
+			if (git_config_parse_parameter(argv[i], fn, data) < 0) {
+				ret = -1;
+				goto out;
+			}
+		}
 	}
 
 out:
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v4 6/6] config: allow specifying config entries via envvar pairs
  2020-12-09 11:52 ` [PATCH v4 0/6] config: allow specifying config entries via env Patrick Steinhardt
                     ` (4 preceding siblings ...)
  2020-12-09 11:52   ` [PATCH v4 5/6] config: refactor parsing of GIT_CONFIG_PARAMETERS Patrick Steinhardt
@ 2020-12-09 11:52   ` Patrick Steinhardt
  2020-12-09 15:29   ` [PATCH v4 0/6] config: allow specifying config entries via env Ævar Arnfjörð Bjarmason
  6 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-09 11:52 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 10021 bytes --]

While we currently have the `GIT_CONFIG_PARAMETERS` environment variable
which can be used to pass runtime configuration data to git processes,
it's an internal implementation detail and not supposed to be used by
end users.

Next to being for internal use only, this way of passing config entries
has a major downside: the config keys need to be parsed as they contain
both key and value in a single variable. As such, it is left to the user
to escape any potentially harmful characters in the value, which is
quite hard to do if values are controlled by a third party.

This commit thus adds a new way of adding config entries via the
environment which gets rid of this shortcoming. If the user passes the
`GIT_CONFIG_COUNT=$n` environment variable, Git will parse environment
variable pairs `GIT_CONFIG_KEY_$i` and `GIT_CONFIG_VALUE_$i` for each
`i` in `[0,n)`.

While the same can be achieved with `git -c <name>=<value>`, one may
wish to not do so for potentially sensitive information. E.g. if one
wants to set `http.extraHeader` to contain an authentication token,
doing so via `-c` would trivially leak those credentials via e.g. ps(1),
which typically also shows command arguments.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git-config.txt |  12 ++++
 cache.h                      |   1 +
 config.c                     |  46 ++++++++++++++
 environment.c                |   1 +
 t/t1300-config.sh            | 115 ++++++++++++++++++++++++++++++++++-
 5 files changed, 174 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt
index 0e9351d3cb..54c994aea1 100644
--- a/Documentation/git-config.txt
+++ b/Documentation/git-config.txt
@@ -346,6 +346,18 @@ GIT_CONFIG_NOSYSTEM::
 
 See also <<FILES>>.
 
+GIT_CONFIG_COUNT::
+GIT_CONFIG_KEY_<n>::
+GIT_CONFIG_VALUE_<n>::
+	If GIT_CONFIG_COUNT is set to a positive number, all environment pairs
+	GIT_CONFIG_KEY_<n> and GIT_CONFIG_VALUE_<n> up to that number will be
+	added to the process's runtime configuration. The config pairs are
+	zero-indexed. Any missing key or value is treated as an error. An empty
+	GIT_CONFIG_COUNT is treated the same as GIT_CONFIG_COUNT=0, namely no
+	pairs are processed. These environment variables will override values
+	in configuration files, but will be overridden by any explicit options
+	passed via `git -c`.
+
 
 [[EXAMPLES]]
 EXAMPLES
diff --git a/cache.h b/cache.h
index 8d279bc110..294841fca7 100644
--- a/cache.h
+++ b/cache.h
@@ -472,6 +472,7 @@ static inline enum object_type object_type(unsigned int mode)
 #define TEMPLATE_DIR_ENVIRONMENT "GIT_TEMPLATE_DIR"
 #define CONFIG_ENVIRONMENT "GIT_CONFIG"
 #define CONFIG_DATA_ENVIRONMENT "GIT_CONFIG_PARAMETERS"
+#define CONFIG_COUNT_ENVIRONMENT "GIT_CONFIG_COUNT"
 #define EXEC_PATH_ENVIRONMENT "GIT_EXEC_PATH"
 #define CEILING_DIRECTORIES_ENVIRONMENT "GIT_CEILING_DIRECTORIES"
 #define NO_REPLACE_OBJECTS_ENVIRONMENT "GIT_NO_REPLACE_OBJECTS"
diff --git a/config.c b/config.c
index 8162f3cec8..779487bc2d 100644
--- a/config.c
+++ b/config.c
@@ -8,6 +8,7 @@
 #include "cache.h"
 #include "branch.h"
 #include "config.h"
+#include "environment.h"
 #include "repository.h"
 #include "lockfile.h"
 #include "exec-cmd.h"
@@ -506,6 +507,8 @@ int git_config_parse_parameter(const char *text,
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
 	const char *env;
+	struct strbuf envvar = STRBUF_INIT;
+	struct strvec to_free = STRVEC_INIT;
 	int ret = 0;
 	char *envw = NULL;
 	const char **argv = NULL;
@@ -517,6 +520,47 @@ int git_config_from_parameters(config_fn_t fn, void *data)
 	source.origin_type = CONFIG_ORIGIN_CMDLINE;
 	cf = &source;
 
+	env = getenv(CONFIG_COUNT_ENVIRONMENT);
+	if (env) {
+		unsigned long count;
+		char *endp;
+
+		count = strtoul(env, &endp, 10);
+		if (*endp) {
+			ret = error(_("bogus count in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+		if (count > INT_MAX) {
+			ret = error(_("too many entries in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+
+		for (i = 0; i < count; i++) {
+			const char *key, *value;
+
+			strbuf_addf(&envvar, "GIT_CONFIG_KEY_%d", i);
+			key = getenv_safe(&to_free, envvar.buf);
+			if (!key) {
+				ret = error(_("missing config key %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			strbuf_addf(&envvar, "GIT_CONFIG_VALUE_%d", i);
+			value = getenv_safe(&to_free, envvar.buf);
+			if (!value) {
+				ret = error(_("missing config value %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			if (config_parse_pair(key, value, fn, data) < 0) {
+				ret = -1;
+				goto out;
+			}
+		}
+	}
+
 	env = getenv(CONFIG_DATA_ENVIRONMENT);
 	if (env) {
 		int nr = 0, alloc = 0;
@@ -538,6 +582,8 @@ int git_config_from_parameters(config_fn_t fn, void *data)
 	}
 
 out:
+	strbuf_release(&envvar);
+	strvec_clear(&to_free);
 	free(argv);
 	free(envw);
 	cf = source.prev;
diff --git a/environment.c b/environment.c
index 2234af462c..2f27008424 100644
--- a/environment.c
+++ b/environment.c
@@ -117,6 +117,7 @@ const char * const local_repo_env[] = {
 	ALTERNATE_DB_ENVIRONMENT,
 	CONFIG_ENVIRONMENT,
 	CONFIG_DATA_ENVIRONMENT,
+	CONFIG_COUNT_ENVIRONMENT,
 	DB_ENVIRONMENT,
 	GIT_DIR_ENVIRONMENT,
 	GIT_WORK_TREE_ENVIRONMENT,
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 46a94814d5..f157cd217e 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1361,6 +1361,117 @@ test_expect_success 'git -c and --config-env override each other' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git config handles environment config pairs' '
+	GIT_CONFIG_COUNT=2 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="foo" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="bar" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one foo
+	pair.two bar
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs without count' '
+	test_must_fail env GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one 2>error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one
+'
+
+test_expect_success 'git config ignores pairs exceeding count' '
+	GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="value" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with empty count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT= GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config fails with invalid count' '
+	test_must_fail env GIT_CONFIG_COUNT=10a git config --list 2>error &&
+	test_i18ngrep "bogus count" error &&
+	test_must_fail env GIT_CONFIG_COUNT=9999999999999999 git config --list 2>error &&
+	test_i18ngrep "too many entries" error
+'
+
+test_expect_success 'git config fails with missing config key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_VALUE_0="value" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config key" error
+'
+
+test_expect_success 'git config fails with missing config value' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0="pair.one" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config value" error
+'
+
+test_expect_success 'git config fails with invalid config pair key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0= GIT_CONFIG_VALUE_0=value \
+		git config --list &&
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0=missing-section GIT_CONFIG_VALUE_0=value \
+		git config --list
+'
+
+test_expect_success 'environment overrides config file' '
+	test_when_finished "rm -f .git/config" &&
+	cat >.git/config <<-EOF &&
+	[pair]
+	one = value
+	EOF
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=override \
+		git config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'GIT_CONFIG_PARAMETERS overrides environment config' '
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
+		GIT_CONFIG_PARAMETERS="${SQ}pair.one=override${SQ}" \
+		git config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'command line overrides environment config' '
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
+		git -c pair.one=override config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
@@ -1706,9 +1817,11 @@ test_expect_success '--show-origin with --list' '
 	file:.git/config	user.override=local
 	file:.git/config	include.path=../include/relative.include
 	file:.git/../include/relative.include	user.relative=include
+	command line:	user.environ=true
 	command line:	user.cmdline=true
 	EOF
-	git -c user.cmdline=true config --list --show-origin >output &&
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=user.environ GIT_CONFIG_VALUE_0=true\
+		git -c user.cmdline=true config --list --show-origin >output &&
 	test_cmp expect output
 '
 
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 4/6] config: extract function to parse config pairs
  2020-12-09 11:52   ` [PATCH v4 4/6] config: extract function to parse config pairs Patrick Steinhardt
@ 2020-12-09 13:12     ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2020-12-09 13:12 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley


On Wed, Dec 09 2020, Patrick Steinhardt wrote:

> +static int config_parse_pair(const char *key, const char *value,
> +			  config_fn_t fn, void *data)
> +{
> +	char *canonical_name;
> +	int ret;
> +
> +	if (!strlen(key))
> +		return error(_("empty config key"));

We just did this check (just before the context of the second hunk in
this patch) before calling this function:

    if (!pair[0]->len) {
        strbuf_list_free(pair);
        return error(_("bogus config parameter: %s"), text);

I think just removing this is best, for such a closely coupled static
function we can just rely on the sanity of the caller, but it should at
least be:

    if (!strlen(key))
        BUG("clear that it's unreachable, and a translator won't bother with it, like _()...");

Aside: It's more C-idiom-y in this case to write:

    if (!*key)


> +	if (git_config_parse_key(key, &canonical_name, NULL))
> +		return -1;
> +
> +	ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
> +	free(canonical_name);
> +	return ret;
> +}
> +
>  int git_config_parse_parameter(const char *text,
>  			       config_fn_t fn, void *data)
>  {
>  	const char *value;
> -	char *canonical_name;
>  	struct strbuf **pair;
>  	int ret;
>  
> @@ -483,12 +498,7 @@ int git_config_parse_parameter(const char *text,
>  		return error(_("bogus config parameter: %s"), text);
>  	}
>  
> -	if (git_config_parse_key(pair[0]->buf, &canonical_name, NULL)) {
> -		ret = -1;
> -	} else {
> -		ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
> -		free(canonical_name);
> -	}
> +	ret = config_parse_pair(pair[0]->buf, value, fn, data);
>  	strbuf_list_free(pair);
>  	return ret;
>  }


^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 2/6] config: add new way to pass config via `--config-env`
  2020-12-09 11:52   ` [PATCH v4 2/6] config: add new way to pass config via `--config-env` Patrick Steinhardt
@ 2020-12-09 14:40     ` Ævar Arnfjörð Bjarmason
  2020-12-09 16:24       ` Jeff King
  2020-12-09 16:10     ` Jeff King
  1 sibling, 1 reply; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2020-12-09 14:40 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley


On Wed, Dec 09 2020, Patrick Steinhardt wrote:

> While it's already possible to pass runtime configuration via `git -c
> <key>=<value>`, it may be undesirable to use when the value contains
> sensitive information. E.g. if one wants to set `http.extraHeader` to
> contain an authentication token, doing so via `-c` would trivially leak
> those credentials via e.g. ps(1), which typically also shows command
> arguments.
>
> To enable this usecase without leaking credentials, this commit
> introduces a new switch `--config-env=<key>=<envvar>`. Instead of
> directly passing a value for the given key, it instead allows the user
> to specify the name of an environment variable. The value of that
> variable will then be used as value of the key.
> [...]
> +--config-env=<name>=<envvar>::
> +	Pass a configuration parameter to the command. The <envvar>
> +	given will be replaced with the contents of the environment
> +	variable of that name. In contrast to `-c`, an envvar must
> +	always be given and exist in the environment. Passing an
> +	environment variable with empty value will set <name> to the
> +	empty string which `git config --type=bool` will convert to
> +	`false`.

Okey, because "-c foo.bar" (true) "-c foo.bar=" is the empty string, but
that doesn't make sene with "--config-env". Also the whole part about
--type=bool is just confusing, because it's referring to `-c`'s magic
behavior when it comes to `bool` which we don't have here.

I think it's also worth describing what this is for & what the
limitations are. Maybe:

    --config-env=<name>=<envvar>

        Like `-c <name>=<var>` except the value is the name of an
        environment variable from which to retrieve the value. Unlike
        `-c` there is no shortcut for directly setting the value to an
        empty string, instead the environment variable itself must be
        set to the empty strin. Errors if the `<envvar>` does not exist
        in the environment.

        This is useful for cases where you want to pass transitory
        configuration options to git, but are doing so on OS's where
        other processes might be able to read your cmdline
        (e.g. `/proc/self/cmdline`), but not your environ
        (e.g. `/proc/self/environ`). That behavior is the default on
        Linux, but may not be on your system.

	Note that this might add security for variables such as
	`http.extraHeader` where the sensitive information is part of
	the value, but not e.g. `url.<base.insteadOf` where the
	sensitive information can be part of the key.

> +void git_config_push_env(const char *spec)
> +{
> +	struct strbuf buf = STRBUF_INIT;
> +	const char *env_name;
> +	const char *env_value;
> +
> +	env_name = strrchr(spec, '=');
> +	if (!env_name)
> +		die("invalid config format: %s", spec);
> +	env_name++;

Not something new, and maybe not something for this series, but I wish
-c and --config-env would document this limitation that we support "="
in keys in config, but not via those parameters.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 0/6] config: allow specifying config entries via env
  2020-12-09 11:52 ` [PATCH v4 0/6] config: allow specifying config entries via env Patrick Steinhardt
                     ` (5 preceding siblings ...)
  2020-12-09 11:52   ` [PATCH v4 6/6] config: allow specifying config entries via envvar pairs Patrick Steinhardt
@ 2020-12-09 15:29   ` Ævar Arnfjörð Bjarmason
  2020-12-11 13:35     ` Patrick Steinhardt
  6 siblings, 1 reply; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2020-12-09 15:29 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley


On Wed, Dec 09 2020, Patrick Steinhardt wrote:

> this is the fourth version of my patch series which aims to implement a
> way to pass config entries via the environment while avoiding any
> requirements to perform shell quoting on the user's side.
>
> Given that the What's Cooking report notes that my third version is
> about to be dropped dropped because the `--config-env` way of doing
> things is preferred, I've now adopted that approach. I've taken the
> patch which Peff posted originally (with one change strchr->strrchr) and
> added documentation and tests to it.
>
> This patch series still includes my old proposal as it would actually be
> a better fit for our usecase at GitLab I have in mind, which is to put
> all configuration which applies to all git commands into the commands
> instead of using a config file for this. I have structured the series in
> such a way though that those patches come last -- so if you continue to
> think this approach shouldn't make it in, please feel free to drop
> patches 3-6.

To add even more to your headaches (sorry!) I hadn't really fully looked
at that --config-env proposal.

As noted in my per-patch reply in [1] it will still expose the key part
of the key=value, and in at least one place (url.<base>.insteadOf) the
key is where we'll pass the user/password on the command-line still with
that.

I'd much prefer either your 6/6 over --config-env for that reason & that
--config-env makes it impossible to pass a key with "=" in. For "-c" I
don't think that's much of an issue, but e.g. with
"url.<base>.insteadOf" needing to take arbitrary passwords + us
implicitly/explicitly advertising this as a "here's how you can pass the
password" feature not being able to have "=" is more painful.

I mildly prefer Jeff's suggestion of just getting GIT_CONFIG_PARAMETERS
to the point where we could document it [2][3] to both of those, but
that's mostly an asthetic concern of dealing with N values. It won't
matter for the security aspect (but I think you (but haven't tested)
that you still can't pass a "=", but your 6/6 does allow that).

I still can't quite shake the bad spidey-sense feeling that any of these
are bad in some way we haven't thought of, just from the perspective
that no other tool I can think of that accepts a password has this
mechanism for passing in a user/password or other sensitive data.

E.g. openssh explicitly has refused to add anything of the sort (a
--password parameter, but maybe they didn't consider
--password=ENV_VAR). E.g. curl has a mode where you can have a password
on the command-line, but they then make you use -netrc-file to grab it
from a file. From searching around I see concerns about shell histories
being part of the security model, maybe that's why it's not a common
pattern.

So I still wonder if some version of what I tried with /dev/fd/321 in
[4] would be best, i.e. something that combines transitory+no extra
command invocation+not adding things to shell history. We support that
pattern in general, just not in fetch.c/remote.c for no particular good
reason AFAICT.

I do that that whatever we go for this series would be much better if
the commit messages / added docs explained why we're doing particular
things, and to users why they'd use one method but not the other.

E.g. IIRC this whole series is because it's a hassle to invoke
core.askpass in some stateful program where you'd like to just provide a
transitory password. I think some brief cross-linking or explanation
somewhere of these various ways to pass sensitive values around would be
relly helpful.

1. https://lore.kernel.org/git/871rfzxctq.fsf@evledraar.gmail.com/
2. https://lore.kernel.org/git/20201117023454.GA34754@coredump.intra.peff.net/
3. https://lore.kernel.org/git/20201118015907.GD650959@coredump.intra.peff.net/
4. https://lore.kernel.org/git/87k0upflk4.fsf@evledraar.gmail.com/


^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 2/6] config: add new way to pass config via `--config-env`
  2020-12-09 11:52   ` [PATCH v4 2/6] config: add new way to pass config via `--config-env` Patrick Steinhardt
  2020-12-09 14:40     ` Ævar Arnfjörð Bjarmason
@ 2020-12-09 16:10     ` Jeff King
  2020-12-09 16:12       ` [PATCH 1/3] quote: make sq_dequote_step() a public function Jeff King
                         ` (4 more replies)
  1 sibling, 5 replies; 116+ messages in thread
From: Jeff King @ 2020-12-09 16:10 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Junio C Hamano,
	brian m. carlson, Philip Oakley

On Wed, Dec 09, 2020 at 12:52:26PM +0100, Patrick Steinhardt wrote:

> Co-authored-by: Jeff King <peff@peff.net>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>

In case we want it, this is also:

  Signed-off-by: Jeff King <peff@peff.net>

> +--config-env=<name>=<envvar>::
> +	Pass a configuration parameter to the command. The <envvar>
> +	given will be replaced with the contents of the environment
> +	variable of that name. In contrast to `-c`, an envvar must
> +	always be given and exist in the environment. Passing an
> +	environment variable with empty value will set <name> to the
> +	empty string which `git config --type=bool` will convert to
> +	`false`.

I agree with Ævar that we probably should keep an empty variable as the
empty string. I think some options use an empty string to clear a list
(e.g., push.pushOption), and I'm not sure how they'd react to a bool
instead. It would be nice to also have a way to do the implicit-bool
thing, but I don't think it's strictly necessary (it's always correct to
put the string "true" into the variable instead).

I think we should also document that <envvar> can't contain an "=" sign.
Of course using strrchr() here doesn't help much with just this patch,
because we flatten the string before stuffing it into
$GIT_CONFIG_PARAMETERS, so the reading side would mis-parse it.

But here's a fix for that. I built it on top of your whole series, since
you touched some of the related functions, but it could easily be
rebased onto just this part.

  [1/3]: quote: make sq_dequote_step() a public function
  [2/3]: config: parse more robust format in GIT_CONFIG_PARAMETERS
  [3/3]: config: store "git -c" variables using more robust format

 config.c          | 118 +++++++++++++++++++++++++++++++++++++---------
 quote.c           |  15 ++++--
 quote.h           |  18 ++++++-
 t/t1300-config.sh |  60 +++++++++++++++++++++++
 4 files changed, 183 insertions(+), 28 deletions(-)

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH 1/3] quote: make sq_dequote_step() a public function
  2020-12-09 16:10     ` Jeff King
@ 2020-12-09 16:12       ` Jeff King
  2020-12-09 16:17       ` [PATCH 2/3] config: parse more robust format in GIT_CONFIG_PARAMETERS Jeff King
                         ` (3 subsequent siblings)
  4 siblings, 0 replies; 116+ messages in thread
From: Jeff King @ 2020-12-09 16:12 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Junio C Hamano,
	brian m. carlson, Philip Oakley

We provide a function for dequoting an entire string, as well as one for
handling a space-separated list of quoted strings. But there's no way
for a caller to parse a string like 'foo'='bar', even though it is easy
to generate one using sq_quote_buf() or similar.

Let's make the single-step function available to callers outside of
quote.c. Note that we do need to adjust its implementation slightly: it
insists on seeing whitespace between items, and we'd like to be more
flexible than that. Since it only has a single caller, we can move that
check (and slurping up any extra whitespace) into that caller.

Signed-off-by: Jeff King <peff@peff.net>
---
 quote.c | 15 ++++++++++-----
 quote.h | 18 ++++++++++++++++--
 2 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/quote.c b/quote.c
index 69f4ca45da..8a3a5e39eb 100644
--- a/quote.c
+++ b/quote.c
@@ -116,7 +116,7 @@ void sq_append_quote_argv_pretty(struct strbuf *dst, const char **argv)
 	}
 }
 
-static char *sq_dequote_step(char *arg, char **next)
+char *sq_dequote_step(char *arg, char **next)
 {
 	char *dst = arg;
 	char *src = arg;
@@ -153,11 +153,8 @@ static char *sq_dequote_step(char *arg, char **next)
 			}
 		/* Fallthrough */
 		default:
-			if (!next || !isspace(*src))
+			if (!next)
 				return NULL;
-			do {
-				c = *++src;
-			} while (isspace(c));
 			*dst = 0;
 			*next = src;
 			return arg;
@@ -182,6 +179,14 @@ static int sq_dequote_to_argv_internal(char *arg,
 		char *dequoted = sq_dequote_step(next, &next);
 		if (!dequoted)
 			return -1;
+		if (next) {
+			char c;
+			if (!isspace(*next))
+				return -1;
+			do {
+				c = *++next;
+			} while (isspace(c));
+		}
 		if (argv) {
 			ALLOC_GROW(*argv, *nr + 1, *alloc);
 			(*argv)[(*nr)++] = dequoted;
diff --git a/quote.h b/quote.h
index 4b72a583cf..768cc6338e 100644
--- a/quote.h
+++ b/quote.h
@@ -42,12 +42,26 @@ void sq_quote_buf_pretty(struct strbuf *, const char *src);
 void sq_quote_argv_pretty(struct strbuf *, const char **argv);
 void sq_append_quote_argv_pretty(struct strbuf *dst, const char **argv);
 
-/* This unwraps what sq_quote() produces in place, but returns
+/*
+ * This unwraps what sq_quote() produces in place, but returns
  * NULL if the input does not look like what sq_quote would have
- * produced.
+ * produced (the full string must be a single quoted item).
  */
 char *sq_dequote(char *);
 
+/*
+ * Like sq_dequote(), but dequote a single item, and leave "next" pointing to
+ * the next character. E.g., in the string:
+ *
+ *   'one' 'two' 'three'
+ *
+ * after the first call, the return value would be the unquoted string "one",
+ * with "next" pointing to the space between "one" and "two"). The caller is
+ * responsible for advancing the pointer to the start of the next item before
+ * calling sq_dequote_step() again.
+ */
+char *sq_dequote_step(char *src, char **next);
+
 /*
  * Same as the above, but can be used to unwrap many arguments in the
  * same string separated by space. Like sq_quote, it works in place,
-- 
2.29.2.1019.g5c4255ecd5


^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH 2/3] config: parse more robust format in GIT_CONFIG_PARAMETERS
  2020-12-09 16:10     ` Jeff King
  2020-12-09 16:12       ` [PATCH 1/3] quote: make sq_dequote_step() a public function Jeff King
@ 2020-12-09 16:17       ` Jeff King
  2020-12-09 16:20       ` [PATCH 3/3] config: store "git -c" variables using more robust format Jeff King
                         ` (2 subsequent siblings)
  4 siblings, 0 replies; 116+ messages in thread
From: Jeff King @ 2020-12-09 16:17 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Junio C Hamano,
	brian m. carlson, Philip Oakley

When we stuff config options into GIT_CONFIG_PARAMETERS, we shell-quote
each one as a single unit, like:

  'section.one=value1' 'section.two=value2'

On the reading side, we de-quote to get the individual strings, and then
parse them by splitting on the first "=" we find. This format is
ambiguous, because an "=" may appear in a subsection. So the config
represented in a file by both:

  [section "subsection=with=equals"]
  key = value

and:

  [section]
  subsection = with=equals.key=value

ends up in this flattened format like:

  'section.subsection=with=equals.key=value'

and we can't tell which was desired. We have traditionally resolved this
by taking the first "=" we see starting from the left, meaning that we
allowed arbitrary content in the value, but not in the subsection.

Let's make our environment format a bit more robust by separately
quoting the key and value. That turns those examples into:

  'section.subsection=with=equals.key'='value'

and:

  'section.subsection'='with=equals.key=value'

respectively, and we can tell the difference between them. We can detect
which format is in use for any given element of the list based on the
presence of the unquoted "=". That means we can continue to allow the
old format to work to support any callers which manually used the old
format, and we can even intermingle the two formats. The old format
wasn't documented, and nobody was supposed to be using it. But it's
likely that such callers exist in the wild, so it's nice if we can avoid
breaking them. Likewise, it may be possible to trigger an older version
of "git -c" that runs a script that calls into a newer version of "git
-c"; that new version would see the intermingled format.

This does create one complication, which is that the obvious format in
the new scheme for

  [section]
  some-bool

is:

  'section.some-bool'

with no equals. We'd mistake that for an old-style variable. And it even
has the same meaning in the old style, but:

  [section "with=equals"]
  some-bool

does not. It would be:

  'section.with=equals=some-bool'

which we'd take to mean:

  [section]
  with = equals=some-bool

in the old, ambiguous style. Likewise, we can't use:

  'section.some-bool'=''

because that's ambiguous with an actual empty string. Instead, we'll
again use the shell-quoting to give us a hint, and use:

  'section.some-bool'=

to show that we have no value.

Note that this commit just expands the reading side. We'll start writing
the new format via "git -c" in a future patch. In the meantime, the
existing "git -c" tests will make sure we didn't break reading the old
format. But we'll also add some explicit coverage of the two formats to
make sure we continue to handle the old one after we move the writing
side over.

And one final note: since we're now using the shell-quoting as a
semantically meaningful hint, this closes the door to us ever allowing
arbitrary shell quoting, like:

  'a'shell'would'be'ok'with'this'.key=value

But we have never supported that (only what sq_quote() would produce),
and we are probably better off keeping things simple, robust, and
backwards-compatible, than trying to make it easier for humans. We'll
continue not to advertise the format of the variable to users, and
instead keep "git -c" as the recommended mechanism for setting config
(even if we are trying to be kind not to break users who may be relying
on the current undocumented format).

Signed-off-by: Jeff King <peff@peff.net>
---
 config.c          | 66 +++++++++++++++++++++++++++++++++++++----------
 t/t1300-config.sh | 52 +++++++++++++++++++++++++++++++++++++
 2 files changed, 104 insertions(+), 14 deletions(-)

diff --git a/config.c b/config.c
index 779487bc2d..fb160c33d2 100644
--- a/config.c
+++ b/config.c
@@ -504,6 +504,57 @@ int git_config_parse_parameter(const char *text,
 	return ret;
 }
 
+static int parse_config_env_list(char *env, config_fn_t fn, void *data)
+{
+	char *cur = env;
+	while (cur && *cur) {
+		const char *key = sq_dequote_step(cur, &cur);
+		if (!key)
+			return error(_("bogus format in %s"),
+				     CONFIG_DATA_ENVIRONMENT);
+
+		if (!cur || isspace(*cur)) {
+			/* old-style 'key=value' */
+			if (git_config_parse_parameter(key, fn, data) < 0)
+				return -1;
+		}
+		else if (*cur == '=') {
+			/* new-style 'key'='value' */
+			const char *value;
+
+			cur++;
+			if (*cur == '\'') {
+				/* quoted value */
+				value = sq_dequote_step(cur, &cur);
+				if (!value || (cur && !isspace(*cur))) {
+					return error(_("bogus format in %s"),
+						     CONFIG_DATA_ENVIRONMENT);
+				}
+			} else if (!*cur || isspace(*cur)) {
+				/* implicit bool: 'key'= */
+				value = NULL;
+			} else {
+				return error(_("bogus format in %s"),
+					     CONFIG_DATA_ENVIRONMENT);
+			}
+
+			if (config_parse_pair(key, value, fn, data) < 0)
+				return -1;
+		}
+		else {
+			/* unknown format */
+			return error(_("bogus format in %s"),
+				     CONFIG_DATA_ENVIRONMENT);
+		}
+
+		if (cur) {
+			while (isspace(*cur))
+				cur++;
+		}
+	}
+	return 0;
+}
+
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
 	const char *env;
@@ -563,22 +614,9 @@ int git_config_from_parameters(config_fn_t fn, void *data)
 
 	env = getenv(CONFIG_DATA_ENVIRONMENT);
 	if (env) {
-		int nr = 0, alloc = 0;
-
 		/* sq_dequote will write over it */
 		envw = xstrdup(env);
-
-		if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
-			ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
-			goto out;
-		}
-
-		for (i = 0; i < nr; i++) {
-			if (git_config_parse_parameter(argv[i], fn, data) < 0) {
-				ret = -1;
-				goto out;
-			}
-		}
+		ret = parse_config_env_list(envw, fn, data);
 	}
 
 out:
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index f157cd217e..bd602e7720 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1294,6 +1294,58 @@ test_expect_success 'git -c is not confused by empty environment' '
 	GIT_CONFIG_PARAMETERS="" git -c x.one=1 config --list
 '
 
+test_expect_success 'GIT_CONFIG_PARAMETERS handles old-style entries' '
+	v="${SQ}key.one=foo${SQ}" &&
+	v="$v  ${SQ}key.two=bar${SQ}" &&
+	v="$v ${SQ}key.ambiguous=section.whatever=value${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.one foo
+	key.two bar
+	key.ambiguous section.whatever=value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'GIT_CONFIG_PARAMETERS handles new-style entries' '
+	v="${SQ}key.one${SQ}=${SQ}foo${SQ}" &&
+	v="$v  ${SQ}key.two${SQ}=${SQ}bar${SQ}" &&
+	v="$v ${SQ}key.ambiguous=section.whatever${SQ}=${SQ}value${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.one foo
+	key.two bar
+	key.ambiguous=section.whatever value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'old and new-style entries can mix' '
+	v="${SQ}key.oldone=oldfoo${SQ}" &&
+	v="$v ${SQ}key.newone${SQ}=${SQ}newfoo${SQ}" &&
+	v="$v ${SQ}key.oldtwo=oldbar${SQ}" &&
+	v="$v ${SQ}key.newtwo${SQ}=${SQ}newbar${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.oldone oldfoo
+	key.newone newfoo
+	key.oldtwo oldbar
+	key.newtwo newbar
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'old and new bools with ambiguous subsection' '
+	v="${SQ}key.with=equals.oldbool${SQ}" &&
+	v="$v ${SQ}key.with=equals.newbool${SQ}=" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.with equals.oldbool
+	key.with=equals.newbool
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
 	cat >expect <<-\EOF &&
 	env.one one
-- 
2.29.2.1019.g5c4255ecd5


^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH 3/3] config: store "git -c" variables using more robust format
  2020-12-09 16:10     ` Jeff King
  2020-12-09 16:12       ` [PATCH 1/3] quote: make sq_dequote_step() a public function Jeff King
  2020-12-09 16:17       ` [PATCH 2/3] config: parse more robust format in GIT_CONFIG_PARAMETERS Jeff King
@ 2020-12-09 16:20       ` Jeff King
  2020-12-09 16:34         ` Jeff King
  2020-12-10 20:55         ` Ævar Arnfjörð Bjarmason
  2020-12-10  0:00       ` [PATCH v4 2/6] config: add new way to pass config via `--config-env` Junio C Hamano
  2020-12-11 13:24       ` Patrick Steinhardt
  4 siblings, 2 replies; 116+ messages in thread
From: Jeff King @ 2020-12-09 16:20 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Junio C Hamano,
	brian m. carlson, Philip Oakley

The previous commit added a new format for $GIT_CONFIG_PARAMETERS which
is able to robustly handle subsections with "=" in them. Let's start
writing the new format. Unfortunately, this does much less than you'd
hope, because "git -c" itself has the same ambiguity problem! But it's
still worth doing:

  - we've now pushed the problem from the inter-process communication
    into the "-c" command-line parser. This would free us up to later
    add an unambiguous format there (e.g., separate arguments like "git
    --config key value", etc).

  - for --config-env, the parser already disallows "=" in the
    environment variable name. So:

      git --config-env section.with=equals.key=ENVVAR

    will robustly set section.with=equals.key to the contents of
    $ENVVAR.

The new test shows the improvement for --config-env.

Signed-off-by: Jeff King <peff@peff.net>
---
One other side effect I just noticed is that we're very aggressive about
trimming leading and trailing whitespace in the old-style format, but
the new one will store values verbatim. IMHO that's better overall, but
we might consider a preparatory patch to remove that trimming
explicitly.

 config.c          | 52 ++++++++++++++++++++++++++++++++++++++++-------
 t/t1300-config.sh |  8 ++++++++
 2 files changed, 53 insertions(+), 7 deletions(-)

diff --git a/config.c b/config.c
index fb160c33d2..04029e45dc 100644
--- a/config.c
+++ b/config.c
@@ -333,38 +333,76 @@ int git_config_include(const char *var, const char *value, void *data)
 	return ret;
 }
 
-void git_config_push_parameter(const char *text)
+static void git_config_push_split_parameter(const char *key, const char *value)
 {
 	struct strbuf env = STRBUF_INIT;
 	const char *old = getenv(CONFIG_DATA_ENVIRONMENT);
 	if (old && *old) {
 		strbuf_addstr(&env, old);
 		strbuf_addch(&env, ' ');
 	}
-	sq_quote_buf(&env, text);
+	sq_quote_buf(&env, key);
+	strbuf_addch(&env, '=');
+	if (value)
+		sq_quote_buf(&env, value);
 	setenv(CONFIG_DATA_ENVIRONMENT, env.buf, 1);
 	strbuf_release(&env);
 }
 
+void git_config_push_parameter(const char *text)
+{
+	const char *value;
+
+	/*
+	 * When we see:
+	 *
+	 *   section.subsection=with=equals.key=value
+	 *
+	 * we cannot tell if it means:
+	 *
+	 *   [section "subsection=with=equals"]
+	 *   key = value
+	 *
+	 * or:
+	 *
+	 *   [section]
+	 *   subsection = with=equals.key=value
+	 *
+	 * We parse left-to-right for the first "=", meaning we'll prefer to
+	 * keep the value intact over the subsection. This is historical, but
+	 * also sensible since values are more likely to contain odd or
+	 * untrusted input than a section name.
+	 *
+	 * A missing equals is explicitly allowed (as a bool-only entry).
+	 */
+	value = strchr(text, '=');
+	if (value) {
+		char *key = xmemdupz(text, value - text);
+		git_config_push_split_parameter(key, value + 1);
+		free(key);
+	} else {
+		git_config_push_split_parameter(text, NULL);
+	}
+}
+
 void git_config_push_env(const char *spec)
 {
-	struct strbuf buf = STRBUF_INIT;
+	char *key;
 	const char *env_name;
 	const char *env_value;
 
 	env_name = strrchr(spec, '=');
 	if (!env_name)
 		die("invalid config format: %s", spec);
+	key = xmemdupz(spec, env_name - spec);
 	env_name++;
 
 	env_value = getenv(env_name);
 	if (!env_value)
 		die("config variable missing for '%s'", env_name);
 
-	strbuf_add(&buf, spec, env_name - spec);
-	strbuf_addstr(&buf, env_value);
-	git_config_push_parameter(buf.buf);
-	strbuf_release(&buf);
+	git_config_push_split_parameter(key, env_value);
+	free(key);
 }
 
 static inline int iskeychar(int c)
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index bd602e7720..e06961767f 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1413,6 +1413,14 @@ test_expect_success 'git -c and --config-env override each other' '
 	test_cmp expect actual
 '
 
+test_expect_success '--config-env handles keys with equals' '
+	echo value=with=equals >expect &&
+	ENVVAR=value=with=equals git \
+		--config-env=section.subsection=with=equals.key=ENVVAR \
+		config section.subsection=with=equals.key >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'git config handles environment config pairs' '
 	GIT_CONFIG_COUNT=2 \
 		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="foo" \
-- 
2.29.2.1019.g5c4255ecd5

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 2/6] config: add new way to pass config via `--config-env`
  2020-12-09 14:40     ` Ævar Arnfjörð Bjarmason
@ 2020-12-09 16:24       ` Jeff King
  2020-12-11 13:24         ` Patrick Steinhardt
  0 siblings, 1 reply; 116+ messages in thread
From: Jeff King @ 2020-12-09 16:24 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Patrick Steinhardt, git, Junio C Hamano, brian m. carlson, Philip Oakley

On Wed, Dec 09, 2020 at 03:40:17PM +0100, Ævar Arnfjörð Bjarmason wrote:

> > +--config-env=<name>=<envvar>::
> > +	Pass a configuration parameter to the command. The <envvar>
> > +	given will be replaced with the contents of the environment
> > +	variable of that name. In contrast to `-c`, an envvar must
> > +	always be given and exist in the environment. Passing an
> > +	environment variable with empty value will set <name> to the
> > +	empty string which `git config --type=bool` will convert to
> > +	`false`.
> 
> Okey, because "-c foo.bar" (true) "-c foo.bar=" is the empty string, but
> that doesn't make sene with "--config-env". Also the whole part about
> --type=bool is just confusing, because it's referring to `-c`'s magic
> behavior when it comes to `bool` which we don't have here.

Yeah, I agree.

> I think it's also worth describing what this is for & what the
> limitations are. Maybe:

Agreed, and the text you gave looks reasonable. Another reason to use it
is that it will (if we add the patches I just sent on top) avoid the
key/value ambiguity with equals in the section name.

> Not something new, and maybe not something for this series, but I wish
> -c and --config-env would document this limitation that we support "="
> in keys in config, but not via those parameters.

Yeah. If we add in my patches, then the limitation is gone here (but we
should mention the limitation on the environment variable name).

I stopped short of adding a variant of "-c" that avoids the ambiguity.
I'm certainly not opposed to one if somebody wants to do it, but I think
documenting the current limitation makes sense in the meantime (and we
should do it in this series while we're thinking about it).

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 3/3] config: store "git -c" variables using more robust format
  2020-12-09 16:20       ` [PATCH 3/3] config: store "git -c" variables using more robust format Jeff King
@ 2020-12-09 16:34         ` Jeff King
  2020-12-10 20:55         ` Ævar Arnfjörð Bjarmason
  1 sibling, 0 replies; 116+ messages in thread
From: Jeff King @ 2020-12-09 16:34 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Junio C Hamano,
	brian m. carlson, Philip Oakley

On Wed, Dec 09, 2020 at 11:20:26AM -0500, Jeff King wrote:

> One other side effect I just noticed is that we're very aggressive about
> trimming leading and trailing whitespace in the old-style format, but
> the new one will store values verbatim. IMHO that's better overall, but
> we might consider a preparatory patch to remove that trimming
> explicitly.

Actually, it looks like we just trim either side of the key. Which
is...weird. We've never generated any, and I wouldn't expect people to
write:

  git -c '  some.key = value'

And even if they did, then "value" would have extra whitespace. So I
don't think this is really changing anything important, though I'm still
tempted to do something like the patch below to clean up the reading
side (and as a bonus, it gets rid of a strbuf_split call, which is a
terrible and awkward interface).

diff --git a/config.c b/config.c
index 04029e45dc..ede33cf3d0 100644
--- a/config.c
+++ b/config.c
@@ -516,29 +516,21 @@ static int config_parse_pair(const char *key, const char *value,
 int git_config_parse_parameter(const char *text,
 			       config_fn_t fn, void *data)
 {
-	const char *value;
-	struct strbuf **pair;
+	char *to_free = NULL;
+	const char *key, *value;
 	int ret;
 
-	pair = strbuf_split_str(text, '=', 2);
-	if (!pair[0])
-		return error(_("bogus config parameter: %s"), text);
-
-	if (pair[0]->len && pair[0]->buf[pair[0]->len - 1] == '=') {
-		strbuf_setlen(pair[0], pair[0]->len - 1);
-		value = pair[1] ? pair[1]->buf : "";
+	value = strchr(text, '=');
+	if (value) {
+		key = to_free = xmemdupz(text, value - text);
+		value++;
 	} else {
-		value = NULL;
+		key = text;
 	}
 
-	strbuf_trim(pair[0]);
-	if (!pair[0]->len) {
-		strbuf_list_free(pair);
-		return error(_("bogus config parameter: %s"), text);
-	}
+	ret = config_parse_pair(key, value, fn, data);
 
-	ret = config_parse_pair(pair[0]->buf, value, fn, data);
-	strbuf_list_free(pair);
+	free(to_free);
 	return ret;
 }
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 2/6] config: add new way to pass config via `--config-env`
  2020-12-09 16:10     ` Jeff King
                         ` (2 preceding siblings ...)
  2020-12-09 16:20       ` [PATCH 3/3] config: store "git -c" variables using more robust format Jeff King
@ 2020-12-10  0:00       ` Junio C Hamano
  2020-12-10  0:09         ` Jeff King
  2020-12-11 13:24       ` Patrick Steinhardt
  4 siblings, 1 reply; 116+ messages in thread
From: Junio C Hamano @ 2020-12-10  0:00 UTC (permalink / raw)
  To: Jeff King
  Cc: Patrick Steinhardt, git, Ævar Arnfjörð Bjarmason,
	brian m. carlson, Philip Oakley

Jeff King <peff@peff.net> writes:

> On Wed, Dec 09, 2020 at 12:52:26PM +0100, Patrick Steinhardt wrote:
>
>> Co-authored-by: Jeff King <peff@peff.net>
>> Signed-off-by: Patrick Steinhardt <ps@pks.im>
>
> In case we want it, this is also:
>
>   Signed-off-by: Jeff King <peff@peff.net>
>
>> +--config-env=<name>=<envvar>::
>> +	Pass a configuration parameter to the command. The <envvar>
>> +	given will be replaced with the contents of the environment
>> +	variable of that name. In contrast to `-c`, an envvar must
>> +	always be given and exist in the environment. Passing an
>> +	environment variable with empty value will set <name> to the
>> +	empty string which `git config --type=bool` will convert to
>> +	`false`.
>
> I agree with Ævar that we probably should keep an empty variable as the
> empty string. I think some options use an empty string to clear a list
> (e.g., push.pushOption), and I'm not sure how they'd react to a bool
> instead. It would be nice to also have a way to do the implicit-bool
> thing, but I don't think it's strictly necessary (it's always correct to
> put the string "true" into the variable instead).
>
> I think we should also document that <envvar> can't contain an "=" sign.
> Of course using strrchr() here doesn't help much with just this patch,
> because we flatten the string before stuffing it into
> $GIT_CONFIG_PARAMETERS, so the reading side would mis-parse it.
>
> But here's a fix for that. I built it on top of your whole series, since
> you touched some of the related functions, but it could easily be
> rebased onto just this part.

Hmph, so 

 (1) Patrick's 1 & 2 are about adding --config-env,

 (2) These three patches can come on top to make it more robust to
     pass key=value with GIT_CONFIG_PARAMETERS (including what is
     added via the --config-env=name=envvar), and

 (3) The remainder of Patrick's 6-patch series is to further add the
     pairs of environment variables to pass keys and values?

I am still not sure if we want the last part, but whether we take
(1) or (3) or neither or both, (2) sounds like a good thing to do.
And (2) would not shine without (1).  In the traditional use of -c,
we do not know which = from the end user separates key and value,
but when (1) places a = to separate the <name> and the value in the
environment variable, we know where that = is and can quote
accordingly.

But these three patches are done on top of (1) and (3), at least for
now.

The above is my understanding of the state of these patches.  Am I
getting it right?

Thanks.

>   [1/3]: quote: make sq_dequote_step() a public function
>   [2/3]: config: parse more robust format in GIT_CONFIG_PARAMETERS
>   [3/3]: config: store "git -c" variables using more robust format
>
>  config.c          | 118 +++++++++++++++++++++++++++++++++++++---------
>  quote.c           |  15 ++++--
>  quote.h           |  18 ++++++-
>  t/t1300-config.sh |  60 +++++++++++++++++++++++
>  4 files changed, 183 insertions(+), 28 deletions(-)
>
> -Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 2/6] config: add new way to pass config via `--config-env`
  2020-12-10  0:00       ` [PATCH v4 2/6] config: add new way to pass config via `--config-env` Junio C Hamano
@ 2020-12-10  0:09         ` Jeff King
  2020-12-10  0:57           ` Junio C Hamano
  0 siblings, 1 reply; 116+ messages in thread
From: Jeff King @ 2020-12-10  0:09 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Patrick Steinhardt, git, Ævar Arnfjörð Bjarmason,
	brian m. carlson, Philip Oakley

On Wed, Dec 09, 2020 at 04:00:08PM -0800, Junio C Hamano wrote:

> > I think we should also document that <envvar> can't contain an "=" sign.
> > Of course using strrchr() here doesn't help much with just this patch,
> > because we flatten the string before stuffing it into
> > $GIT_CONFIG_PARAMETERS, so the reading side would mis-parse it.
> >
> > But here's a fix for that. I built it on top of your whole series, since
> > you touched some of the related functions, but it could easily be
> > rebased onto just this part.
> 
> Hmph, so 
> 
>  (1) Patrick's 1 & 2 are about adding --config-env,
> 
>  (2) These three patches can come on top to make it more robust to
>      pass key=value with GIT_CONFIG_PARAMETERS (including what is
>      added via the --config-env=name=envvar), and

Yep, exactly.

>  (3) The remainder of Patrick's 6-patch series is to further add the
>      pairs of environment variables to pass keys and values?

More or less. I did use the config_parse_pair() helper from patch 4, and
there's some textual dependency on his patch 5 (but it would be easy to
rebase).

> I am still not sure if we want the last part, but whether we take
> (1) or (3) or neither or both, (2) sounds like a good thing to do.
> And (2) would not shine without (1).  In the traditional use of -c,
> we do not know which = from the end user separates key and value,
> but when (1) places a = to separate the <name> and the value in the
> environment variable, we know where that = is and can quote
> accordingly.

Exactly. Without (1), then (2) is not nearly as exciting. We could also
implement a non-ambiguous version of "-c", like:

  git --config key.with=equals.foo some-value ...

which would also benefit from (2). But I think Patrick's main goal was
to get secret values off the command-line, so he'd want either (1) or
(3) for that.

> But these three patches are done on top of (1) and (3), at least for
> now.

Yes. I'd be happy to rebase them if we're not going to do the
GIT_CONFIG_{KEY,VALUE}_<n> parts.

> The above is my understanding of the state of these patches.  Am I
> getting it right?

Yep.

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 2/6] config: add new way to pass config via `--config-env`
  2020-12-10  0:09         ` Jeff King
@ 2020-12-10  0:57           ` Junio C Hamano
  0 siblings, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2020-12-10  0:57 UTC (permalink / raw)
  To: Jeff King
  Cc: Patrick Steinhardt, git, Ævar Arnfjörð Bjarmason,
	brian m. carlson, Philip Oakley

Jeff King <peff@peff.net> writes:

> Yes. I'd be happy to rebase them if we're not going to do the
> GIT_CONFIG_{KEY,VALUE}_<n> parts.
>
>> The above is my understanding of the state of these patches.  Am I
>> getting it right?
>
> Yep.

Thanks.  

I have no strong feeling for or against the env-pair feature, so the
way these three parts are structured (i.e. more robust parsing of
'=' comes at the end) is fine by me.



^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 3/3] config: store "git -c" variables using more robust format
  2020-12-09 16:20       ` [PATCH 3/3] config: store "git -c" variables using more robust format Jeff King
  2020-12-09 16:34         ` Jeff King
@ 2020-12-10 20:55         ` Ævar Arnfjörð Bjarmason
  2020-12-10 21:49           ` Junio C Hamano
  2020-12-11 13:21           ` Jeff King
  1 sibling, 2 replies; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2020-12-10 20:55 UTC (permalink / raw)
  To: Jeff King
  Cc: Patrick Steinhardt, git, Junio C Hamano, brian m. carlson, Philip Oakley


On Wed, Dec 09 2020, Jeff King wrote:

> The previous commit added a new format for $GIT_CONFIG_PARAMETERS which
> is able to robustly handle subsections with "=" in them. Let's start
> writing the new format. Unfortunately, this does much less than you'd
> hope, because "git -c" itself has the same ambiguity problem! But it's
> still worth doing:
>
>   - we've now pushed the problem from the inter-process communication
>     into the "-c" command-line parser. This would free us up to later
>     add an unambiguous format there (e.g., separate arguments like "git
>     --config key value", etc).
>
>   - for --config-env, the parser already disallows "=" in the
>     environment variable name. So:
>
>       git --config-env section.with=equals.key=ENVVAR
>
>     will robustly set section.with=equals.key to the contents of
>     $ENVVAR.
>
> The new test shows the improvement for --config-env.
>
> Signed-off-by: Jeff King <peff@peff.net>
> ---
> One other side effect I just noticed is that we're very aggressive about
> trimming leading and trailing whitespace in the old-style format, but
> the new one will store values verbatim. IMHO that's better overall, but
> we might consider a preparatory patch to remove that trimming
> explicitly.
>
>  config.c          | 52 ++++++++++++++++++++++++++++++++++++++++-------
>  t/t1300-config.sh |  8 ++++++++
>  2 files changed, 53 insertions(+), 7 deletions(-)
>
> diff --git a/config.c b/config.c
> index fb160c33d2..04029e45dc 100644
> --- a/config.c
> +++ b/config.c
> @@ -333,38 +333,76 @@ int git_config_include(const char *var, const char *value, void *data)
>  	return ret;
>  }
>  
> -void git_config_push_parameter(const char *text)
> +static void git_config_push_split_parameter(const char *key, const char *value)
>  {
>  	struct strbuf env = STRBUF_INIT;
>  	const char *old = getenv(CONFIG_DATA_ENVIRONMENT);
>  	if (old && *old) {
>  		strbuf_addstr(&env, old);
>  		strbuf_addch(&env, ' ');
>  	}
> -	sq_quote_buf(&env, text);
> +	sq_quote_buf(&env, key);
> +	strbuf_addch(&env, '=');
> +	if (value)
> +		sq_quote_buf(&env, value);
>  	setenv(CONFIG_DATA_ENVIRONMENT, env.buf, 1);
>  	strbuf_release(&env);
>  }
>  
> +void git_config_push_parameter(const char *text)
> +{
> +	const char *value;
> +
> +	/*
> +	 * When we see:
> +	 *
> +	 *   section.subsection=with=equals.key=value
> +	 *
> +	 * we cannot tell if it means:
> +	 *
> +	 *   [section "subsection=with=equals"]
> +	 *   key = value
> +	 *
> +	 * or:
> +	 *
> +	 *   [section]
> +	 *   subsection = with=equals.key=value
> +	 *
> +	 * We parse left-to-right for the first "=", meaning we'll prefer to
> +	 * keep the value intact over the subsection. This is historical, but
> +	 * also sensible since values are more likely to contain odd or
> +	 * untrusted input than a section name.
> +	 *
> +	 * A missing equals is explicitly allowed (as a bool-only entry).
> +	 */
> +	value = strchr(text, '=');
> +	if (value) {
> +		char *key = xmemdupz(text, value - text);
> +		git_config_push_split_parameter(key, value + 1);
> +		free(key);
> +	} else {
> +		git_config_push_split_parameter(text, NULL);
> +	}
> +}
> +
>  void git_config_push_env(const char *spec)
>  {
> -	struct strbuf buf = STRBUF_INIT;
> +	char *key;
>  	const char *env_name;
>  	const char *env_value;
>  
>  	env_name = strrchr(spec, '=');
>  	if (!env_name)
>  		die("invalid config format: %s", spec);
> +	key = xmemdupz(spec, env_name - spec);
>  	env_name++;
>  
>  	env_value = getenv(env_name);
>  	if (!env_value)
>  		die("config variable missing for '%s'", env_name);
>  
> -	strbuf_add(&buf, spec, env_name - spec);
> -	strbuf_addstr(&buf, env_value);
> -	git_config_push_parameter(buf.buf);
> -	strbuf_release(&buf);
> +	git_config_push_split_parameter(key, env_value);
> +	free(key);
>  }
>  
>  static inline int iskeychar(int c)
> diff --git a/t/t1300-config.sh b/t/t1300-config.sh
> index bd602e7720..e06961767f 100755
> --- a/t/t1300-config.sh
> +++ b/t/t1300-config.sh
> @@ -1413,6 +1413,14 @@ test_expect_success 'git -c and --config-env override each other' '
>  	test_cmp expect actual
>  '
>  
> +test_expect_success '--config-env handles keys with equals' '
> +	echo value=with=equals >expect &&
> +	ENVVAR=value=with=equals git \
> +		--config-env=section.subsection=with=equals.key=ENVVAR \
> +		config section.subsection=with=equals.key >actual &&
> +	test_cmp expect actual
> +'
> +

Maybe worth adding a test for the strrchr() semantics here with:

    perl -we '$ENV{"Y=Z"}="why and zed"; system "Z=zed git --config-env=X=Y=Z ..."'

Which would show that we can't look up "Y=Z", but will always get "Z".

I think that's fine b.t.w., 1/2 of the minor objection I had to
--config-env in
https://lore.kernel.org/git/87y2i7vvz4.fsf@evledraar.gmail.com/ was
mainly about being unable to e.g. support odd token usernames with the
"insteadOf" feature.

But aside from having a feature meant to improve security being able to
be combined with a config variable we have in a way that leaks the
password in ps(1) I think these improved semantics make sense.

I.e. I can't imagine someone wants an env var with "=" in it, even
though POSIX makes such a thing possible (you just can't do it in a
shellscript).

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 3/3] config: store "git -c" variables using more robust format
  2020-12-10 20:55         ` Ævar Arnfjörð Bjarmason
@ 2020-12-10 21:49           ` Junio C Hamano
  2020-12-11 13:21           ` Jeff King
  1 sibling, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2020-12-10 21:49 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Jeff King, Patrick Steinhardt, git, brian m. carlson, Philip Oakley

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Wed, Dec 09 2020, Jeff King wrote:
> ...
>> +test_expect_success '--config-env handles keys with equals' '
>> +	echo value=with=equals >expect &&
>> +	ENVVAR=value=with=equals git \
>> +		--config-env=section.subsection=with=equals.key=ENVVAR \
>> +		config section.subsection=with=equals.key >actual &&
>> +	test_cmp expect actual
>> +'
>> +
>
> Maybe worth adding a test for the strrchr() semantics here with:
>
>     perl -we '$ENV{"Y=Z"}="why and zed"; system "Z=zed git --config-env=X=Y=Z ..."'
>
> Which would show that we can't look up "Y=Z", but will always get "Z".

Yes, that was explained in the cover letter of these three patches
in <X9D23LQv34A5Q5DC@coredump.intra.peff.net>.  

We really should document that <envvar> can't contain an "=" sign,
but I do not see much point in casting that limitation in stone with
a test.  As long as we know things work correctly with environment
variables without '=' in their names, we should be happy.



^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 3/3] config: store "git -c" variables using more robust format
  2020-12-10 20:55         ` Ævar Arnfjörð Bjarmason
  2020-12-10 21:49           ` Junio C Hamano
@ 2020-12-11 13:21           ` Jeff King
  1 sibling, 0 replies; 116+ messages in thread
From: Jeff King @ 2020-12-11 13:21 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Patrick Steinhardt, git, Junio C Hamano, brian m. carlson, Philip Oakley

On Thu, Dec 10, 2020 at 09:55:18PM +0100, Ævar Arnfjörð Bjarmason wrote:

> > diff --git a/t/t1300-config.sh b/t/t1300-config.sh
> > index bd602e7720..e06961767f 100755
> > --- a/t/t1300-config.sh
> > +++ b/t/t1300-config.sh
> > @@ -1413,6 +1413,14 @@ test_expect_success 'git -c and --config-env override each other' '
> >  	test_cmp expect actual
> >  '
> >  
> > +test_expect_success '--config-env handles keys with equals' '
> > +	echo value=with=equals >expect &&
> > +	ENVVAR=value=with=equals git \
> > +		--config-env=section.subsection=with=equals.key=ENVVAR \
> > +		config section.subsection=with=equals.key >actual &&
> > +	test_cmp expect actual
> > +'
> > +
> 
> Maybe worth adding a test for the strrchr() semantics here with:
> 
>     perl -we '$ENV{"Y=Z"}="why and zed"; system "Z=zed git --config-env=X=Y=Z ..."'
> 
> Which would show that we can't look up "Y=Z", but will always get "Z".

We're already testing those here, though. If we used strchr(), then we'd
end up setting section.subsection in the above test.

I.e., your X=Y=Z can be tested in either of two ways:

  - check that X=Y is set to $Z

  - check that X is not set to Y=Z

It's sufficient to test only one, because success in one implies the
other. And of the two, I think testing the first is much more
interesting (because testing "this expected thing happened" is much more
robust than "this unexpected thing didn't happen").

> I.e. I can't imagine someone wants an env var with "=" in it, even
> though POSIX makes such a thing possible (you just can't do it in a
> shellscript).

Yeah, I'm definitely OK with that limitation, but we should document it.

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 2/6] config: add new way to pass config via `--config-env`
  2020-12-09 16:24       ` Jeff King
@ 2020-12-11 13:24         ` Patrick Steinhardt
  2020-12-11 14:21           ` Jeff King
  0 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-11 13:24 UTC (permalink / raw)
  To: Jeff King
  Cc: Ævar Arnfjörð Bjarmason, git, Junio C Hamano,
	brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 2069 bytes --]

On Wed, Dec 09, 2020 at 11:24:03AM -0500, Jeff King wrote:
> On Wed, Dec 09, 2020 at 03:40:17PM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> > > +--config-env=<name>=<envvar>::
> > > +	Pass a configuration parameter to the command. The <envvar>
> > > +	given will be replaced with the contents of the environment
> > > +	variable of that name. In contrast to `-c`, an envvar must
> > > +	always be given and exist in the environment. Passing an
> > > +	environment variable with empty value will set <name> to the
> > > +	empty string which `git config --type=bool` will convert to
> > > +	`false`.
> > 
> > Okey, because "-c foo.bar" (true) "-c foo.bar=" is the empty string, but
> > that doesn't make sene with "--config-env". Also the whole part about
> > --type=bool is just confusing, because it's referring to `-c`'s magic
> > behavior when it comes to `bool` which we don't have here.
> 
> Yeah, I agree.
> 
> > I think it's also worth describing what this is for & what the
> > limitations are. Maybe:
> 
> Agreed, and the text you gave looks reasonable. Another reason to use it
> is that it will (if we add the patches I just sent on top) avoid the
> key/value ambiguity with equals in the section name.

Yeah, I'll pick up the explanation by Ævar, it's a lot better compared
to what I had. Thanks!

> > Not something new, and maybe not something for this series, but I wish
> > -c and --config-env would document this limitation that we support "="
> > in keys in config, but not via those parameters.
> 
> Yeah. If we add in my patches, then the limitation is gone here (but we
> should mention the limitation on the environment variable name).
> 
> I stopped short of adding a variant of "-c" that avoids the ambiguity.
> I'm certainly not opposed to one if somebody wants to do it, but I think
> documenting the current limitation makes sense in the meantime (and we
> should do it in this series while we're thinking about it).
> 
> -Peff

Do you want me to adopt your patches as part of this series?

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 2/6] config: add new way to pass config via `--config-env`
  2020-12-09 16:10     ` Jeff King
                         ` (3 preceding siblings ...)
  2020-12-10  0:00       ` [PATCH v4 2/6] config: add new way to pass config via `--config-env` Junio C Hamano
@ 2020-12-11 13:24       ` Patrick Steinhardt
  2020-12-11 14:20         ` Jeff King
  4 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-11 13:24 UTC (permalink / raw)
  To: Jeff King
  Cc: git, Ævar Arnfjörð Bjarmason, Junio C Hamano,
	brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 2415 bytes --]

On Wed, Dec 09, 2020 at 11:10:04AM -0500, Jeff King wrote:
> On Wed, Dec 09, 2020 at 12:52:26PM +0100, Patrick Steinhardt wrote:
> 
> > Co-authored-by: Jeff King <peff@peff.net>
> > Signed-off-by: Patrick Steinhardt <ps@pks.im>
> 
> In case we want it, this is also:
> 
>   Signed-off-by: Jeff King <peff@peff.net>
> 
> > +--config-env=<name>=<envvar>::
> > +	Pass a configuration parameter to the command. The <envvar>
> > +	given will be replaced with the contents of the environment
> > +	variable of that name. In contrast to `-c`, an envvar must
> > +	always be given and exist in the environment. Passing an
> > +	environment variable with empty value will set <name> to the
> > +	empty string which `git config --type=bool` will convert to
> > +	`false`.
> 
> I agree with Ævar that we probably should keep an empty variable as the
> empty string. I think some options use an empty string to clear a list
> (e.g., push.pushOption), and I'm not sure how they'd react to a bool
> instead. It would be nice to also have a way to do the implicit-bool
> thing, but I don't think it's strictly necessary (it's always correct to
> put the string "true" into the variable instead).

I think this is just weirdly worded in the `-c` case, which I mostly
copied. We _do_ keep the empty string, which effectively means that `git
config --type=bool` will return `false`.

Or do you mean that we should allow `--config-env=foo.bar=`?

> I think we should also document that <envvar> can't contain an "=" sign.
> Of course using strrchr() here doesn't help much with just this patch,
> because we flatten the string before stuffing it into
> $GIT_CONFIG_PARAMETERS, so the reading side would mis-parse it.

Makes sense.

Patrick

> But here's a fix for that. I built it on top of your whole series, since
> you touched some of the related functions, but it could easily be
> rebased onto just this part.
> 
>   [1/3]: quote: make sq_dequote_step() a public function
>   [2/3]: config: parse more robust format in GIT_CONFIG_PARAMETERS
>   [3/3]: config: store "git -c" variables using more robust format
> 
>  config.c          | 118 +++++++++++++++++++++++++++++++++++++---------
>  quote.c           |  15 ++++--
>  quote.h           |  18 ++++++-
>  t/t1300-config.sh |  60 +++++++++++++++++++++++
>  4 files changed, 183 insertions(+), 28 deletions(-)
> 
> -Peff

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 0/6] config: allow specifying config entries via env
  2020-12-09 15:29   ` [PATCH v4 0/6] config: allow specifying config entries via env Ævar Arnfjörð Bjarmason
@ 2020-12-11 13:35     ` Patrick Steinhardt
  2020-12-11 14:27       ` Jeff King
  0 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-11 13:35 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 4252 bytes --]

On Wed, Dec 09, 2020 at 04:29:35PM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> On Wed, Dec 09 2020, Patrick Steinhardt wrote:
> 
> > this is the fourth version of my patch series which aims to implement a
> > way to pass config entries via the environment while avoiding any
> > requirements to perform shell quoting on the user's side.
> >
> > Given that the What's Cooking report notes that my third version is
> > about to be dropped dropped because the `--config-env` way of doing
> > things is preferred, I've now adopted that approach. I've taken the
> > patch which Peff posted originally (with one change strchr->strrchr) and
> > added documentation and tests to it.
> >
> > This patch series still includes my old proposal as it would actually be
> > a better fit for our usecase at GitLab I have in mind, which is to put
> > all configuration which applies to all git commands into the commands
> > instead of using a config file for this. I have structured the series in
> > such a way though that those patches come last -- so if you continue to
> > think this approach shouldn't make it in, please feel free to drop
> > patches 3-6.
> 
> To add even more to your headaches (sorry!) I hadn't really fully looked
> at that --config-env proposal.
> 
> As noted in my per-patch reply in [1] it will still expose the key part
> of the key=value, and in at least one place (url.<base>.insteadOf) the
> key is where we'll pass the user/password on the command-line still with
> that.

True, that's one of the things I don't quite like about `--config-env`.

> I'd much prefer either your 6/6 over --config-env for that reason & that
> --config-env makes it impossible to pass a key with "=" in. For "-c" I
> don't think that's much of an issue, but e.g. with
> "url.<base>.insteadOf" needing to take arbitrary passwords + us
> implicitly/explicitly advertising this as a "here's how you can pass the
> password" feature not being able to have "=" is more painful.
> 
> I mildly prefer Jeff's suggestion of just getting GIT_CONFIG_PARAMETERS
> to the point where we could document it [2][3] to both of those, but
> that's mostly an asthetic concern of dealing with N values. It won't
> matter for the security aspect (but I think you (but haven't tested)
> that you still can't pass a "=", but your 6/6 does allow that).

Documenting the format would be interesting, but I'm still not quite
sure how it'd be used then. Using a separate `git shell-quote` binary
just to correctly convert the strings to what git would expect doesn't
seem ideal to me, also because it would mean a separate process for each
git invocation which wants to use GIT_CONFIG_PARAMETERS. On the other
hand, reimplementing the shellquoting functionality wherever you want to
use it doesn't sound ideal either.

[snip]

> I do that that whatever we go for this series would be much better if
> the commit messages / added docs explained why we're doing particular
> things, and to users why they'd use one method but not the other.

Makes sense. The commit messages do mention it, but docs don't. I plan
to take your explanation anyway as it's a lot better compared to what I
had, and it does explain why one would want to use `--config-env`.

> E.g. IIRC this whole series is because it's a hassle to invoke
> core.askpass in some stateful program where you'd like to just provide a
> transitory password. I think some brief cross-linking or explanation
> somewhere of these various ways to pass sensitive values around would be
> relly helpful.

It had been the original intention, yes. And it still is, but in fact
the usecase has broadened to also use it to get rid of our global git
config in Gitaly. Which is a little bit awkward to do with
`--config-env` or `-c`, as now a ps(1) would first show a dozen of
configuration values only to have the real command buried somewhere at
the back. It would have been easy to implement though with the
GIT_CONFIG_ envvars.

Granted, we could still do the same by just using GIT_CONFIG_PAREMETERS.
But I'm kind of hesitant to reimplement the shell-quoting procedures in
Gitaly, especially considering that we'd put untrusted data in there.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 2/6] config: add new way to pass config via `--config-env`
  2020-12-11 13:24       ` Patrick Steinhardt
@ 2020-12-11 14:20         ` Jeff King
  0 siblings, 0 replies; 116+ messages in thread
From: Jeff King @ 2020-12-11 14:20 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Junio C Hamano,
	brian m. carlson, Philip Oakley

On Fri, Dec 11, 2020 at 02:24:35PM +0100, Patrick Steinhardt wrote:

> > > +--config-env=<name>=<envvar>::
> > > +	Pass a configuration parameter to the command. The <envvar>
> > > +	given will be replaced with the contents of the environment
> > > +	variable of that name. In contrast to `-c`, an envvar must
> > > +	always be given and exist in the environment. Passing an
> > > +	environment variable with empty value will set <name> to the
> > > +	empty string which `git config --type=bool` will convert to
> > > +	`false`.
> > 
> > I agree with Ævar that we probably should keep an empty variable as the
> > empty string. I think some options use an empty string to clear a list
> > (e.g., push.pushOption), and I'm not sure how they'd react to a bool
> > instead. It would be nice to also have a way to do the implicit-bool
> > thing, but I don't think it's strictly necessary (it's always correct to
> > put the string "true" into the variable instead).
> 
> I think this is just weirdly worded in the `-c` case, which I mostly
> copied. We _do_ keep the empty string, which effectively means that `git
> config --type=bool` will return `false`.

Oh indeed, I misread what you wrote in the documentation. I think it is
doing the right thing, then. IMHO it is not worth even calling out
specially, since there is no matching implicit-bool form. I.e., I'd
probably just cut the final sentence.

> Or do you mean that we should allow `--config-env=foo.bar=`?

Hmm, yeah, that would work as an "implicit bool". But it's sufficiently
ugly and non-intuitive (and weirdly overlapping with "-c") that I'm not
sure it is worth supporting.

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 2/6] config: add new way to pass config via `--config-env`
  2020-12-11 13:24         ` Patrick Steinhardt
@ 2020-12-11 14:21           ` Jeff King
  2020-12-11 14:54             ` Patrick Steinhardt
  0 siblings, 1 reply; 116+ messages in thread
From: Jeff King @ 2020-12-11 14:21 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: Ævar Arnfjörð Bjarmason, git, Junio C Hamano,
	brian m. carlson, Philip Oakley

On Fri, Dec 11, 2020 at 02:24:33PM +0100, Patrick Steinhardt wrote:

> Do you want me to adopt your patches as part of this series?

Yeah, if you're willing to. I don't mind spinning it off into its own
series if you don't want to (the tricky part is that we're touching a
couple of the same spots, though, so if you're willing to pick them up,
I think that makes coordination easier).

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 0/6] config: allow specifying config entries via env
  2020-12-11 13:35     ` Patrick Steinhardt
@ 2020-12-11 14:27       ` Jeff King
  2020-12-11 14:42         ` Jeff King
  2020-12-11 14:47         ` Patrick Steinhardt
  0 siblings, 2 replies; 116+ messages in thread
From: Jeff King @ 2020-12-11 14:27 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: Ævar Arnfjörð Bjarmason, git, Junio C Hamano,
	brian m. carlson, Philip Oakley

On Fri, Dec 11, 2020 at 02:35:01PM +0100, Patrick Steinhardt wrote:

> > E.g. IIRC this whole series is because it's a hassle to invoke
> > core.askpass in some stateful program where you'd like to just provide a
> > transitory password. I think some brief cross-linking or explanation
> > somewhere of these various ways to pass sensitive values around would be
> > relly helpful.
> 
> It had been the original intention, yes. And it still is, but in fact
> the usecase has broadened to also use it to get rid of our global git
> config in Gitaly. Which is a little bit awkward to do with
> `--config-env` or `-c`, as now a ps(1) would first show a dozen of
> configuration values only to have the real command buried somewhere at
> the back. It would have been easy to implement though with the
> GIT_CONFIG_ envvars.

I don't know what kinds of variables you want to set exactly, but
another possible option here is some mechanism to point Git to an extra
config file. This would work if you are setting a bunch of options in
some static way, but not if you're setting them to custom values for
each command invocation (because then you'd be dealing with a temp file,
which is annoying and error-prone).

I'm thinking something like a $GIT_CONFIG_ENV_FILE that is parsed after
repo config but before $GIT_CONFIG_PARAMETERS.

Or alternatively, add an includeIf directive that lets you do something
like:

  [includeIf "env:FOO"]
  path = foo.gitconfig

which triggers if $FOO is set. But again, that's only useful if you have
certain "profiles" of config you're trying to set, and not custom
values.

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 0/6] config: allow specifying config entries via env
  2020-12-11 14:27       ` Jeff King
@ 2020-12-11 14:42         ` Jeff King
  2020-12-11 14:58           ` Patrick Steinhardt
  2020-12-11 14:47         ` Patrick Steinhardt
  1 sibling, 1 reply; 116+ messages in thread
From: Jeff King @ 2020-12-11 14:42 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: Ævar Arnfjörð Bjarmason, git, Junio C Hamano,
	brian m. carlson, Philip Oakley

On Fri, Dec 11, 2020 at 09:27:57AM -0500, Jeff King wrote:

> I don't know what kinds of variables you want to set exactly, but
> another possible option here is some mechanism to point Git to an extra
> config file. This would work if you are setting a bunch of options in
> some static way, but not if you're setting them to custom values for
> each command invocation (because then you'd be dealing with a temp file,
> which is annoying and error-prone).
> 
> I'm thinking something like a $GIT_CONFIG_ENV_FILE that is parsed after
> repo config but before $GIT_CONFIG_PARAMETERS.

One more (probably insane) idea, that you are free to ignore unless it
sparks your interest.

$GIT_CONFIG_ENV could contain an actual config-file snippet itself.
I.e.:

  GIT_CONFIG_ENV='
	[foo]
	bar = value
	[another "section"]
	key = "more complicated value"
  '

In fact, we could have implemented $GIT_CONFIG_PARAMETERS that way from
the very beginning. I'd be hesitant to change it now, though.

It doesn't really make your quoting problem go away, in that you'd now
have to generate a valid and correct config file, which is even more
complicated than shell-quoting. :) But it is at least a well-documented
format whose generator might be used for other things, too.

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 0/6] config: allow specifying config entries via env
  2020-12-11 14:27       ` Jeff King
  2020-12-11 14:42         ` Jeff King
@ 2020-12-11 14:47         ` Patrick Steinhardt
  2020-12-11 15:21           ` Ævar Arnfjörð Bjarmason
  2020-12-11 16:02           ` Jeff King
  1 sibling, 2 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-11 14:47 UTC (permalink / raw)
  To: Jeff King
  Cc: Ævar Arnfjörð Bjarmason, git, Junio C Hamano,
	brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 3319 bytes --]

On Fri, Dec 11, 2020 at 09:27:57AM -0500, Jeff King wrote:
> On Fri, Dec 11, 2020 at 02:35:01PM +0100, Patrick Steinhardt wrote:
> 
> > > E.g. IIRC this whole series is because it's a hassle to invoke
> > > core.askpass in some stateful program where you'd like to just provide a
> > > transitory password. I think some brief cross-linking or explanation
> > > somewhere of these various ways to pass sensitive values around would be
> > > relly helpful.
> > 
> > It had been the original intention, yes. And it still is, but in fact
> > the usecase has broadened to also use it to get rid of our global git
> > config in Gitaly. Which is a little bit awkward to do with
> > `--config-env` or `-c`, as now a ps(1) would first show a dozen of
> > configuration values only to have the real command buried somewhere at
> > the back. It would have been easy to implement though with the
> > GIT_CONFIG_ envvars.
> 
> I don't know what kinds of variables you want to set exactly, but
> another possible option here is some mechanism to point Git to an extra
> config file. This would work if you are setting a bunch of options in
> some static way, but not if you're setting them to custom values for
> each command invocation (because then you'd be dealing with a temp file,
> which is annoying and error-prone).
> 
> I'm thinking something like a $GIT_CONFIG_ENV_FILE that is parsed after
> repo config but before $GIT_CONFIG_PARAMETERS.
> 
> Or alternatively, add an includeIf directive that lets you do something
> like:
> 
>   [includeIf "env:FOO"]
>   path = foo.gitconfig
> 
> which triggers if $FOO is set. But again, that's only useful if you have
> certain "profiles" of config you're trying to set, and not custom
> values.
> 
> -Peff

The issue we have is that the config file isn't necessarily under our
control. It is in most cases, like e.g. when Gitaly gets deployed via
Omnibus. But we also allow for source-based installations, where the
user configures most things manually. And in that case, we have to ask
the user to "Please set config variables A, B and C". Naturally, this is
easy to forget, will drift apart in future releases and so on.

To fix this, the plan is to move all required configuration items into
Gitaly itself, which GIT_CONFIG_COUNT would've allowd to do quite
nicely. Something like Ævar's proposal to allow reading the config from
a file descriptor would also work, and just putting the whole
configuration into an environment variable (similar to your
GIT_CONFIG_ENV_FILE, but containing contents instead of a path). And
finally, using `-c` would also work, with the downside of making it
harder to see what's going on with all the git processes.

With regards to what we require from the config, you can have a look
e.g. at [1]. It doesn't contain much, but we expect the following ones
to be set:

    - core.autocrlf=input
    - gc.auto=0
    - repack.writeBitmaps=true
    - receive.advertisePushOptions=true
    - core.fsyncObjectFiles=true

Anyway, this is all rather specific to Gitaly and may thus not be too
interesting for other. So in the end, we'll just live with the tradeoffs
of whatever solution we end up with.

Patrick

[1]: https://docs.gitlab.com/ee/install/installation.html#configure-it

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 2/6] config: add new way to pass config via `--config-env`
  2020-12-11 14:21           ` Jeff King
@ 2020-12-11 14:54             ` Patrick Steinhardt
  2020-12-11 16:10               ` Jeff King
  0 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-11 14:54 UTC (permalink / raw)
  To: Jeff King
  Cc: Ævar Arnfjörð Bjarmason, git, Junio C Hamano,
	brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1011 bytes --]

On Fri, Dec 11, 2020 at 09:21:38AM -0500, Jeff King wrote:
> On Fri, Dec 11, 2020 at 02:24:33PM +0100, Patrick Steinhardt wrote:
> 
> > Do you want me to adopt your patches as part of this series?
> 
> Yeah, if you're willing to. I don't mind spinning it off into its own
> series if you don't want to (the tricky part is that we're touching a
> couple of the same spots, though, so if you're willing to pick them up,
> I think that makes coordination easier).
> 
> -Peff

I can do so. The only question that I have is whether I should rebase it
on top of 6/6 or on top of 2/6. It's hard for me to gauge whether 6/6 is
going to make it in or not due to the conflicting opinions on it. It
currently seems to me like we tend towards a "no", which is also what
the "What's cooking" report said. But there were also some opinions in
favor of it, which made me wonder. If this is a definitive "no", then
I'm happy to stop bothering with them to make the patch series easier to
manage.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 0/6] config: allow specifying config entries via env
  2020-12-11 14:42         ` Jeff King
@ 2020-12-11 14:58           ` Patrick Steinhardt
  0 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-11 14:58 UTC (permalink / raw)
  To: Jeff King
  Cc: Ævar Arnfjörð Bjarmason, git, Junio C Hamano,
	brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1813 bytes --]

On Fri, Dec 11, 2020 at 09:42:14AM -0500, Jeff King wrote:
> On Fri, Dec 11, 2020 at 09:27:57AM -0500, Jeff King wrote:
> 
> > I don't know what kinds of variables you want to set exactly, but
> > another possible option here is some mechanism to point Git to an extra
> > config file. This would work if you are setting a bunch of options in
> > some static way, but not if you're setting them to custom values for
> > each command invocation (because then you'd be dealing with a temp file,
> > which is annoying and error-prone).
> > 
> > I'm thinking something like a $GIT_CONFIG_ENV_FILE that is parsed after
> > repo config but before $GIT_CONFIG_PARAMETERS.
> 
> One more (probably insane) idea, that you are free to ignore unless it
> sparks your interest.
> 
> $GIT_CONFIG_ENV could contain an actual config-file snippet itself.
> I.e.:
> 
>   GIT_CONFIG_ENV='
> 	[foo]
> 	bar = value
> 	[another "section"]
> 	key = "more complicated value"
>   '
> 
> In fact, we could have implemented $GIT_CONFIG_PARAMETERS that way from
> the very beginning. I'd be hesitant to change it now, though.
> 
> It doesn't really make your quoting problem go away, in that you'd now
> have to generate a valid and correct config file, which is even more
> complicated than shell-quoting. :) But it is at least a well-documented
> format whose generator might be used for other things, too.
> 
> -Peff

Our mails crossed, but I did have the same idea. I don't even think it
that insane -- the format is well-documented, it solves some of the
issues I have and implementing it shouldn't be hard considering that all
infra for it exists already. True, it won't fix the quoting issue. But
it would neatly fix our "global config" problem without cluttering
output of ps(1).

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 0/6] config: allow specifying config entries via env
  2020-12-11 14:47         ` Patrick Steinhardt
@ 2020-12-11 15:21           ` Ævar Arnfjörð Bjarmason
  2020-12-11 16:02           ` Jeff King
  1 sibling, 0 replies; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2020-12-11 15:21 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: Jeff King, git, Junio C Hamano, brian m. carlson, Philip Oakley


On Fri, Dec 11 2020, Patrick Steinhardt wrote:

> On Fri, Dec 11, 2020 at 09:27:57AM -0500, Jeff King wrote:
>> On Fri, Dec 11, 2020 at 02:35:01PM +0100, Patrick Steinhardt wrote:
>> 
>> > > E.g. IIRC this whole series is because it's a hassle to invoke
>> > > core.askpass in some stateful program where you'd like to just provide a
>> > > transitory password. I think some brief cross-linking or explanation
>> > > somewhere of these various ways to pass sensitive values around would be
>> > > relly helpful.
>> > 
>> > It had been the original intention, yes. And it still is, but in fact
>> > the usecase has broadened to also use it to get rid of our global git
>> > config in Gitaly. Which is a little bit awkward to do with
>> > `--config-env` or `-c`, as now a ps(1) would first show a dozen of
>> > configuration values only to have the real command buried somewhere at
>> > the back. It would have been easy to implement though with the
>> > GIT_CONFIG_ envvars.
>> 
>> I don't know what kinds of variables you want to set exactly, but
>> another possible option here is some mechanism to point Git to an extra
>> config file. This would work if you are setting a bunch of options in
>> some static way, but not if you're setting them to custom values for
>> each command invocation (because then you'd be dealing with a temp file,
>> which is annoying and error-prone).
>> 
>> I'm thinking something like a $GIT_CONFIG_ENV_FILE that is parsed after
>> repo config but before $GIT_CONFIG_PARAMETERS.
>> 
>> Or alternatively, add an includeIf directive that lets you do something
>> like:
>> 
>>   [includeIf "env:FOO"]
>>   path = foo.gitconfig
>> 
>> which triggers if $FOO is set. But again, that's only useful if you have
>> certain "profiles" of config you're trying to set, and not custom
>> values.
>> 
>> -Peff
>
> The issue we have is that the config file isn't necessarily under our
> control. It is in most cases, like e.g. when Gitaly gets deployed via
> Omnibus. But we also allow for source-based installations, where the
> user configures most things manually. And in that case, we have to ask
> the user to "Please set config variables A, B and C". Naturally, this is
> easy to forget, will drift apart in future releases and so on.
>
> To fix this, the plan is to move all required configuration items into
> Gitaly itself, which GIT_CONFIG_COUNT would've allowd to do quite
> nicely. Something like Ævar's proposal to allow reading the config from
> a file descriptor would also work, and just putting the whole
> configuration into an environment variable (similar to your
> GIT_CONFIG_ENV_FILE, but containing contents instead of a path). And
> finally, using `-c` would also work, with the downside of making it
> harder to see what's going on with all the git processes.

Aside from other stuff mentioned in this thread a trick I've used for a
while to make things "git-y" is:

    [alias]
    sh = !sh

Then you can just:

    git -c foo.bar=baz sh -c 'git config --get foo.bar'

Or, with a symlink from "git-aly" to "gitaly" in $PATH:

    git -c foo.bar=baz aly [...]

Although that's more a hack, and may go away depending on what happens
to dashed builtins (I don't know what Johannes was planning there).

Of course this only works for global config and "I want to run this
script doing a bunch of git stuff, and using this config", not
e.g. dynamically setting a password for one request.

> With regards to what we require from the config, you can have a look
> e.g. at [1]. It doesn't contain much, but we expect the following ones
> to be set:
>
>     - core.autocrlf=input
>     - gc.auto=0
>     - repack.writeBitmaps=true
>     - receive.advertisePushOptions=true
>     - core.fsyncObjectFiles=true
>
> Anyway, this is all rather specific to Gitaly and may thus not be too
> interesting for other. So in the end, we'll just live with the tradeoffs
> of whatever solution we end up with.
>
> Patrick
>
> [1]: https://docs.gitlab.com/ee/install/installation.html#configure-it


^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 0/6] config: allow specifying config entries via env
  2020-12-11 14:47         ` Patrick Steinhardt
  2020-12-11 15:21           ` Ævar Arnfjörð Bjarmason
@ 2020-12-11 16:02           ` Jeff King
  1 sibling, 0 replies; 116+ messages in thread
From: Jeff King @ 2020-12-11 16:02 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: Ævar Arnfjörð Bjarmason, git, Junio C Hamano,
	brian m. carlson, Philip Oakley

On Fri, Dec 11, 2020 at 03:47:38PM +0100, Patrick Steinhardt wrote:

> The issue we have is that the config file isn't necessarily under our
> control. It is in most cases, like e.g. when Gitaly gets deployed via
> Omnibus. But we also allow for source-based installations, where the
> user configures most things manually. And in that case, we have to ask
> the user to "Please set config variables A, B and C". Naturally, this is
> easy to forget, will drift apart in future releases and so on.

For GitHub, we ship a VM appliance, so we just put what we want into
/etc/gitconfig. I think in the worst case you could simplify everything
down to "put [include]path=/usr/share/gitlab/gitconfig into your system
config.  Though I'm a little surprised that you wouldn't just ship your
own version of Git that is used on the backend (so you know you have the
right version, plus any custom patches you'd want), and then point its
system config to /usr/share/gitlab/ or whatever.

But I'm probably just showing my ignorance of your setup / install
procedures, and there are a dozen reasons why that wouldn't work. ;)

> To fix this, the plan is to move all required configuration items into
> Gitaly itself, which GIT_CONFIG_COUNT would've allowd to do quite
> nicely. Something like Ævar's proposal to allow reading the config from
> a file descriptor would also work, and just putting the whole
> configuration into an environment variable (similar to your
> GIT_CONFIG_ENV_FILE, but containing contents instead of a path). And
> finally, using `-c` would also work, with the downside of making it
> harder to see what's going on with all the git processes.

We do have a couple scripts (like our git-repack wrapper) that make
heavy use of "git -c" to tweak things that don't have a command-line
option, or for which using it is awkward. It does clutter up "ps" a bit,
but it's sometimes nice to see the extra values, too (just as you'd see
command-line options).

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v4 2/6] config: add new way to pass config via `--config-env`
  2020-12-11 14:54             ` Patrick Steinhardt
@ 2020-12-11 16:10               ` Jeff King
  0 siblings, 0 replies; 116+ messages in thread
From: Jeff King @ 2020-12-11 16:10 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: Ævar Arnfjörð Bjarmason, git, Junio C Hamano,
	brian m. carlson, Philip Oakley

On Fri, Dec 11, 2020 at 03:54:03PM +0100, Patrick Steinhardt wrote:

> > Yeah, if you're willing to. I don't mind spinning it off into its own
> > series if you don't want to (the tricky part is that we're touching a
> > couple of the same spots, though, so if you're willing to pick them up,
> > I think that makes coordination easier).
> > 
> 
> I can do so. The only question that I have is whether I should rebase it
> on top of 6/6 or on top of 2/6. It's hard for me to gauge whether 6/6 is
> going to make it in or not due to the conflicting opinions on it. It
> currently seems to me like we tend towards a "no", which is also what
> the "What's cooking" report said. But there were also some opinions in
> favor of it, which made me wonder. If this is a definitive "no", then
> I'm happy to stop bothering with them to make the patch series easier to
> manage.

I'd probably do it on top of 2/6 (well, perhaps shuffling 4/6 forward is
needed, then, I think). And then that punts the decision.

As for the general idea of 6/6, I think I'm a soft "no" there. Normally
my opinion for things I wouldn't use myself is "hey, go to town, if
you're willing to write the patch and it won't hurt anybody else". My
only reservation is that it's a public-facing interface, so we'll have
to support it forever. And I don't love the interface.

That's not a reflection on how you did the series, btw. I think you've
done a very good job of trying to address everyone's concerns, and
balance them with having a way to get data through the environment that
doesn't require error-prone quoting and parsing. But at the end, I think
we are left with a fundamental tradeoff: an interface that is clunky
because of the counted variables, or one if that is clunky because of
the quoting.

(And by "soft no", I just mean that I wouldn't pursue it further in your
shoes, but I'm not going to strenuously object if you and others want to
go forward).

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v5 0/8] config: allow specifying config entries via env
  2020-11-24 10:50 [PATCH v2 0/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
                   ` (4 preceding siblings ...)
  2020-12-09 11:52 ` [PATCH v4 0/6] config: allow specifying config entries via env Patrick Steinhardt
@ 2020-12-16  7:52 ` Patrick Steinhardt
  2020-12-16  7:52   ` [PATCH v5 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
                     ` (7 more replies)
  2021-01-07  6:36 ` [PATCH v6 0/8] config: allow specifying config entries via env Patrick Steinhardt
                   ` (2 subsequent siblings)
  8 siblings, 8 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-16  7:52 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 2241 bytes --]

Hi,

this is the fifth version of my patch series which aims to implement a
way to pass config entries via the environment while avoiding any
requirements to perform shell quoting on the user's side.

Changes in this version include:

    - I've adopted Jeff's patches to make GIT_CONFIG_PARAMETERS more
      robust by using quoting for both key and value of the config
      entry. This allows to store entries which for example have an
      equals sign in their key.

    - I've replaced the documentation of `git --config-env` by Ævar's,
      which was much better.

    - I've amended the documentation of GIT_CONFIG_COUNT to document the
      intended usecase.

The series is structured as following:

    - Patch 1 is a while-at-it patch for the `--super-prefix` usage
      string which was missing in `git --help`.

    - Patch 2 implements `git --config-env`.

    - Patch 3-6 implement robust handling of GIT_CONFIG_PARAMETERS.

    - Patch 7-8 implement GIT_CONFIG_COUNT handling.

As before, if the GIT_CONFIG_COUNT code is unwanted, please feel free to
cut off after the 6th patch.

Patrick

Jeff King (3):
  quote: make sq_dequote_step() a public function
  config: store "git -c" variables using more robust format
  config: parse more robust format in GIT_CONFIG_PARAMETERS

Patrick Steinhardt (5):
  git: add `--super-prefix` to usage string
  config: add new way to pass config via `--config-env`
  config: extract function to parse config pairs
  environment: make `getenv_safe()` a public function
  config: allow specifying config entries via envvar pairs

 Documentation/git-config.txt |  16 +++
 Documentation/git.txt        |  23 +++-
 cache.h                      |   1 +
 config.c                     | 205 ++++++++++++++++++++++++++++----
 config.h                     |   1 +
 environment.c                |   8 +-
 environment.h                |  12 ++
 git.c                        |   3 +
 quote.c                      |  15 ++-
 quote.h                      |  18 ++-
 t/t1300-config.sh            | 220 ++++++++++++++++++++++++++++++++++-
 11 files changed, 483 insertions(+), 39 deletions(-)
 create mode 100644 environment.h

-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v5 1/8] git: add `--super-prefix` to usage string
  2020-12-16  7:52 ` [PATCH v5 0/8] " Patrick Steinhardt
@ 2020-12-16  7:52   ` Patrick Steinhardt
  2020-12-16  7:52   ` [PATCH v5 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
                     ` (6 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-16  7:52 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1057 bytes --]

When the `--super-prefix` option was implmented in 74866d7579 (git: make
super-prefix option, 2016-10-07), its existence was only documented in
the manpage but not in the command's own usage string. Given that the
commit message didn't mention that this was done intentionally and given
that it's documented in the manpage, this seems like an oversight.

Add it to the usage string to fix the inconsistency.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 git.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/git.c b/git.c
index a00a0a4d94..5a8ff12f87 100644
--- a/git.c
+++ b/git.c
@@ -29,6 +29,7 @@ const char git_usage_string[] =
 	   "           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]\n"
 	   "           [-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]\n"
 	   "           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]\n"
+	   "           [--super-prefix=<path>]\n"
 	   "           <command> [<args>]");
 
 const char git_more_info_string[] =
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 2/8] config: add new way to pass config via `--config-env`
  2020-12-16  7:52 ` [PATCH v5 0/8] " Patrick Steinhardt
  2020-12-16  7:52   ` [PATCH v5 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
@ 2020-12-16  7:52   ` Patrick Steinhardt
  2020-12-23 21:35     ` Junio C Hamano
  2020-12-16  7:54   ` [PATCH v5 4/8] config: extract function to parse config pairs Patrick Steinhardt
                     ` (5 subsequent siblings)
  7 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-16  7:52 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 7144 bytes --]

While it's already possible to pass runtime configuration via `git -c
<key>=<value>`, it may be undesirable to use when the value contains
sensitive information. E.g. if one wants to set `http.extraHeader` to
contain an authentication token, doing so via `-c` would trivially leak
those credentials via e.g. ps(1), which typically also shows command
arguments.

To enable this usecase without leaking credentials, this commit
introduces a new switch `--config-env=<key>=<envvar>`. Instead of
directly passing a value for the given key, it instead allows the user
to specify the name of an environment variable. The value of that
variable will then be used as value of the key.

Co-authored-by: Jeff King <peff@peff.net>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git.txt | 23 +++++++++++++++++++++-
 config.c              | 21 ++++++++++++++++++++
 config.h              |  1 +
 git.c                 |  4 +++-
 t/t1300-config.sh     | 45 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 92 insertions(+), 2 deletions(-)

diff --git a/Documentation/git.txt b/Documentation/git.txt
index c463b937a8..80fb8fab11 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -13,7 +13,7 @@ SYNOPSIS
     [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
     [-p|--paginate|-P|--no-pager] [--no-replace-objects] [--bare]
     [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
-    [--super-prefix=<path>]
+    [--super-prefix=<path>] [--config-env <name>=<envvar>]
     <command> [<args>]
 
 DESCRIPTION
@@ -80,6 +80,27 @@ config file). Including the equals but with an empty value (like `git -c
 foo.bar= ...`) sets `foo.bar` to the empty string which `git config
 --type=bool` will convert to `false`.
 
+--config-env=<name>=<envvar>::
+	Like `-c <name>=<var>` except the value is the name of an
+	environment variable from which to retrieve the value. Unlike
+	`-c` there is no shortcut for directly setting the value to an
+	empty string, instead the environment variable itself must be
+	set to the empty strin. Errors if the `<envvar>` does not exist
+	in the environment. `<envvar>` may not contain an equals sign
+	to avoid ambiguity with `<name>`s which contain one.
+
+	This is useful for cases where you want to pass transitory
+	configuration options to git, but are doing so on OS's where
+	other processes might be able to read your cmdline
+	(e.g. `/proc/self/cmdline`), but not your environ
+	(e.g. `/proc/self/environ`). That behavior is the default on
+	Linux, but may not be on your system.
+
+	Note that this might add security for variables such as
+	`http.extraHeader` where the sensitive information is part of
+	the value, but not e.g. `url.<base.insteadOf` where the
+	sensitive information can be part of the key.
+
 --exec-path[=<path>]::
 	Path to wherever your core Git programs are installed.
 	This can also be controlled by setting the GIT_EXEC_PATH
diff --git a/config.c b/config.c
index 1137bd73af..cde3511110 100644
--- a/config.c
+++ b/config.c
@@ -345,6 +345,27 @@ void git_config_push_parameter(const char *text)
 	strbuf_release(&env);
 }
 
+void git_config_push_env(const char *spec)
+{
+	struct strbuf buf = STRBUF_INIT;
+	const char *env_name;
+	const char *env_value;
+
+	env_name = strrchr(spec, '=');
+	if (!env_name)
+		die("invalid config format: %s", spec);
+	env_name++;
+
+	env_value = getenv(env_name);
+	if (!env_value)
+		die("config variable missing for '%s'", env_name);
+
+	strbuf_add(&buf, spec, env_name - spec);
+	strbuf_addstr(&buf, env_value);
+	git_config_push_parameter(buf.buf);
+	strbuf_release(&buf);
+}
+
 static inline int iskeychar(int c)
 {
 	return isalnum(c) || c == '-';
diff --git a/config.h b/config.h
index c1449bb790..19a9adbaa9 100644
--- a/config.h
+++ b/config.h
@@ -138,6 +138,7 @@ int git_config_from_mem(config_fn_t fn,
 int git_config_from_blob_oid(config_fn_t fn, const char *name,
 			     const struct object_id *oid, void *data);
 void git_config_push_parameter(const char *text);
+void git_config_push_env(const char *spec);
 int git_config_from_parameters(config_fn_t fn, void *data);
 void read_early_config(config_fn_t cb, void *data);
 void read_very_early_config(config_fn_t cb, void *data);
diff --git a/git.c b/git.c
index 5a8ff12f87..b5f63d346b 100644
--- a/git.c
+++ b/git.c
@@ -29,7 +29,7 @@ const char git_usage_string[] =
 	   "           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]\n"
 	   "           [-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]\n"
 	   "           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]\n"
-	   "           [--super-prefix=<path>]\n"
+	   "           [--super-prefix=<path>] [--config-env=<name>=<envvar>]\n"
 	   "           <command> [<args>]");
 
 const char git_more_info_string[] =
@@ -255,6 +255,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			git_config_push_parameter((*argv)[1]);
 			(*argv)++;
 			(*argc)--;
+		} else if (skip_prefix(cmd, "--config-env=", &cmd)) {
+			git_config_push_env(cmd);
 		} else if (!strcmp(cmd, "--literal-pathspecs")) {
 			setenv(GIT_LITERAL_PATHSPECS_ENVIRONMENT, "1", 1);
 			if (envchanged)
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 97a04c6cc2..46a94814d5 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1316,6 +1316,51 @@ test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
 		git config --get-regexp "env.*"
 '
 
+test_expect_success 'git --config-env=key=envvar support' '
+	cat >expect <<-\EOF &&
+	value
+	value
+	false
+	EOF
+	{
+		env ENVVAR=value git --config-env=core.name=ENVVAR config core.name &&
+		env ENVVAR=value git --config-env=foo.CamelCase=ENVVAR config foo.camelcase &&
+		env ENVVAR= git --config-env=foo.flag=ENVVAR config --bool foo.flag
+	} >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git --config-env fails with invalid parameters' '
+	test_must_fail git --config-env=foo.flag config --bool foo.flag 2>error &&
+	test_i18ngrep "invalid config format" error &&
+	test_must_fail git --config-env=foo.flag=NONEXISTENT config --bool foo.flag 2>error &&
+	test_i18ngrep "config variable missing" error
+'
+
+test_expect_success 'git -c and --config-env work together' '
+	cat >expect <<-\EOF &&
+	bar.cmd cmd-value
+	bar.env env-value
+	EOF
+	env ENVVAR=env-value git \
+		-c bar.cmd=cmd-value \
+		--config-env=bar.env=ENVVAR \
+		config --get-regexp "^bar.*" >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git -c and --config-env override each other' '
+	cat >expect <<-\EOF &&
+	env
+	cmd
+	EOF
+	{
+		env ENVVAR=env git -c bar.bar=cmd --config-env=bar.bar=ENVVAR config bar.bar &&
+		env ENVVAR=env git --config-env=bar.bar=ENVVAR -c bar.bar=cmd config bar.bar
+	} >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 4/8] config: extract function to parse config pairs
  2020-12-16  7:52 ` [PATCH v5 0/8] " Patrick Steinhardt
  2020-12-16  7:52   ` [PATCH v5 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
  2020-12-16  7:52   ` [PATCH v5 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
@ 2020-12-16  7:54   ` Patrick Steinhardt
  2020-12-16  7:54   ` [PATCH v5 7/8] environment: make `getenv_safe()` a public function Patrick Steinhardt
                     ` (4 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-16  7:54 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1833 bytes --]

The function `git_config_parse_parameter` is responsible for parsing a
`foo.bar=baz`-formatted configuration key, sanitizing the key and then
processing it via the given callback function. Given that we're about to
add a second user which is going to process keys which already has keys
and values separated, this commit extracts a function
`config_parse_pair` which only does the sanitization and processing
part as a preparatory step.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 config.c | 24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/config.c b/config.c
index cde3511110..151980e5c9 100644
--- a/config.c
+++ b/config.c
@@ -458,11 +458,26 @@ int git_config_key_is_valid(const char *key)
 	return !git_config_parse_key_1(key, NULL, NULL, 1);
 }
 
+static int config_parse_pair(const char *key, const char *value,
+			  config_fn_t fn, void *data)
+{
+	char *canonical_name;
+	int ret;
+
+	if (!strlen(key))
+		return error(_("empty config key"));
+	if (git_config_parse_key(key, &canonical_name, NULL))
+		return -1;
+
+	ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
+	free(canonical_name);
+	return ret;
+}
+
 int git_config_parse_parameter(const char *text,
 			       config_fn_t fn, void *data)
 {
 	const char *value;
-	char *canonical_name;
 	struct strbuf **pair;
 	int ret;
 
@@ -483,12 +498,7 @@ int git_config_parse_parameter(const char *text,
 		return error(_("bogus config parameter: %s"), text);
 	}
 
-	if (git_config_parse_key(pair[0]->buf, &canonical_name, NULL)) {
-		ret = -1;
-	} else {
-		ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
-		free(canonical_name);
-	}
+	ret = config_parse_pair(pair[0]->buf, value, fn, data);
 	strbuf_list_free(pair);
 	return ret;
 }
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 7/8] environment: make `getenv_safe()` a public function
  2020-12-16  7:52 ` [PATCH v5 0/8] " Patrick Steinhardt
                     ` (2 preceding siblings ...)
  2020-12-16  7:54   ` [PATCH v5 4/8] config: extract function to parse config pairs Patrick Steinhardt
@ 2020-12-16  7:54   ` Patrick Steinhardt
  2020-12-16  7:54   ` [PATCH v5 8/8] config: allow specifying config entries via envvar pairs Patrick Steinhardt
                     ` (3 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-16  7:54 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1661 bytes --]

The `getenv_safe()` helper function helps to safely retrieve multiple
environment values without the need to depend on platform-specific
behaviour for the return value's lifetime. We'll make use of this
function in a following patch, so let's make it available by making it
non-static and adding a declaration.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 environment.c |  7 ++-----
 environment.h | 12 ++++++++++++
 2 files changed, 14 insertions(+), 5 deletions(-)
 create mode 100644 environment.h

diff --git a/environment.c b/environment.c
index bb518c61cd..2234af462c 100644
--- a/environment.c
+++ b/environment.c
@@ -9,6 +9,7 @@
  */
 #include "cache.h"
 #include "branch.h"
+#include "environment.h"
 #include "repository.h"
 #include "config.h"
 #include "refs.h"
@@ -152,11 +153,7 @@ static char *expand_namespace(const char *raw_namespace)
 	return strbuf_detach(&buf, NULL);
 }
 
-/*
- * Wrapper of getenv() that returns a strdup value. This value is kept
- * in argv to be freed later.
- */
-static const char *getenv_safe(struct strvec *argv, const char *name)
+const char *getenv_safe(struct strvec *argv, const char *name)
 {
 	const char *value = getenv(name);
 
diff --git a/environment.h b/environment.h
new file mode 100644
index 0000000000..d438b5c8f3
--- /dev/null
+++ b/environment.h
@@ -0,0 +1,12 @@
+#ifndef ENVIRONMENT_H
+#define ENVIRONMENT_H
+
+#include "strvec.h"
+
+/*
+ * Wrapper of getenv() that returns a strdup value. This value is kept
+ * in argv to be freed later.
+ */
+const char *getenv_safe(struct strvec *argv, const char *name);
+
+#endif
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 8/8] config: allow specifying config entries via envvar pairs
  2020-12-16  7:52 ` [PATCH v5 0/8] " Patrick Steinhardt
                     ` (3 preceding siblings ...)
  2020-12-16  7:54   ` [PATCH v5 7/8] environment: make `getenv_safe()` a public function Patrick Steinhardt
@ 2020-12-16  7:54   ` Patrick Steinhardt
  2020-12-23 21:14     ` Junio C Hamano
  2020-12-16  7:56   ` [PATCH v5 3/8] quote: make sq_dequote_step() a public function Patrick Steinhardt
                     ` (2 subsequent siblings)
  7 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-16  7:54 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 10492 bytes --]

While we currently have the `GIT_CONFIG_PARAMETERS` environment variable
which can be used to pass runtime configuration data to git processes,
it's an internal implementation detail and not supposed to be used by
end users.

Next to being for internal use only, this way of passing config entries
has a major downside: the config keys need to be parsed as they contain
both key and value in a single variable. As such, it is left to the user
to escape any potentially harmful characters in the value, which is
quite hard to do if values are controlled by a third party.

This commit thus adds a new way of adding config entries via the
environment which gets rid of this shortcoming. If the user passes the
`GIT_CONFIG_COUNT=$n` environment variable, Git will parse environment
variable pairs `GIT_CONFIG_KEY_$i` and `GIT_CONFIG_VALUE_$i` for each
`i` in `[0,n)`.

While the same can be achieved with `git -c <name>=<value>`, one may
wish to not do so for potentially sensitive information. E.g. if one
wants to set `http.extraHeader` to contain an authentication token,
doing so via `-c` would trivially leak those credentials via e.g. ps(1),
which typically also shows command arguments.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git-config.txt |  16 +++++
 cache.h                      |   1 +
 config.c                     |  67 +++++++++++++++++---
 environment.c                |   1 +
 t/t1300-config.sh            | 115 ++++++++++++++++++++++++++++++++++-
 5 files changed, 191 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt
index 0e9351d3cb..72ccea4419 100644
--- a/Documentation/git-config.txt
+++ b/Documentation/git-config.txt
@@ -346,6 +346,22 @@ GIT_CONFIG_NOSYSTEM::
 
 See also <<FILES>>.
 
+GIT_CONFIG_COUNT::
+GIT_CONFIG_KEY_<n>::
+GIT_CONFIG_VALUE_<n>::
+	If GIT_CONFIG_COUNT is set to a positive number, all environment pairs
+	GIT_CONFIG_KEY_<n> and GIT_CONFIG_VALUE_<n> up to that number will be
+	added to the process's runtime configuration. The config pairs are
+	zero-indexed. Any missing key or value is treated as an error. An empty
+	GIT_CONFIG_COUNT is treated the same as GIT_CONFIG_COUNT=0, namely no
+	pairs are processed. These environment variables will override values
+	in configuration files, but will be overridden by any explicit options
+	passed via `git -c`.
+
+	This is useful for cases where you want to spawn multiple git commands
+	with a common configuration but cannot depend on a configuration file,
+	for example when writing scripts.
+
 
 [[EXAMPLES]]
 EXAMPLES
diff --git a/cache.h b/cache.h
index 8d279bc110..294841fca7 100644
--- a/cache.h
+++ b/cache.h
@@ -472,6 +472,7 @@ static inline enum object_type object_type(unsigned int mode)
 #define TEMPLATE_DIR_ENVIRONMENT "GIT_TEMPLATE_DIR"
 #define CONFIG_ENVIRONMENT "GIT_CONFIG"
 #define CONFIG_DATA_ENVIRONMENT "GIT_CONFIG_PARAMETERS"
+#define CONFIG_COUNT_ENVIRONMENT "GIT_CONFIG_COUNT"
 #define EXEC_PATH_ENVIRONMENT "GIT_EXEC_PATH"
 #define CEILING_DIRECTORIES_ENVIRONMENT "GIT_CEILING_DIRECTORIES"
 #define NO_REPLACE_OBJECTS_ENVIRONMENT "GIT_NO_REPLACE_OBJECTS"
diff --git a/config.c b/config.c
index 60a7261807..1742aefa3e 100644
--- a/config.c
+++ b/config.c
@@ -8,6 +8,7 @@
 #include "cache.h"
 #include "branch.h"
 #include "config.h"
+#include "environment.h"
 #include "repository.h"
 #include "lockfile.h"
 #include "exec-cmd.h"
@@ -594,23 +595,73 @@ static int parse_config_env_list(char *env, config_fn_t fn, void *data)
 
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
-	const char *env = getenv(CONFIG_DATA_ENVIRONMENT);
+	const char *env;
+	struct strbuf envvar = STRBUF_INIT;
+	struct strvec to_free = STRVEC_INIT;
 	int ret = 0;
-	char *envw;
+	char *envw = NULL;
 	struct config_source source;
 
-	if (!env)
-		return 0;
-
 	memset(&source, 0, sizeof(source));
 	source.prev = cf;
 	source.origin_type = CONFIG_ORIGIN_CMDLINE;
 	cf = &source;
 
-	/* sq_dequote will write over it */
-	envw = xstrdup(env);
-	ret = parse_config_env_list(envw, fn, data);
+	env = getenv(CONFIG_COUNT_ENVIRONMENT);
+	if (env) {
+		unsigned long count;
+		char *endp;
+		int i;
 
+		count = strtoul(env, &endp, 10);
+		if (*endp) {
+			ret = error(_("bogus count in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+		if (count > INT_MAX) {
+			ret = error(_("too many entries in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+
+		for (i = 0; i < count; i++) {
+			const char *key, *value;
+
+			strbuf_addf(&envvar, "GIT_CONFIG_KEY_%d", i);
+			key = getenv_safe(&to_free, envvar.buf);
+			if (!key) {
+				ret = error(_("missing config key %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			strbuf_addf(&envvar, "GIT_CONFIG_VALUE_%d", i);
+			value = getenv_safe(&to_free, envvar.buf);
+			if (!value) {
+				ret = error(_("missing config value %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			if (config_parse_pair(key, value, fn, data) < 0) {
+				ret = -1;
+				goto out;
+			}
+		}
+	}
+
+	env = getenv(CONFIG_DATA_ENVIRONMENT);
+	if (env) {
+		/* sq_dequote will write over it */
+		envw = xstrdup(env);
+		if (parse_config_env_list(envw, fn, data) < 0) {
+			ret = -1;
+			goto out;
+		}
+	}
+
+out:
+	strbuf_release(&envvar);
+	strvec_clear(&to_free);
 	free(envw);
 	cf = source.prev;
 	return ret;
diff --git a/environment.c b/environment.c
index 2234af462c..2f27008424 100644
--- a/environment.c
+++ b/environment.c
@@ -117,6 +117,7 @@ const char * const local_repo_env[] = {
 	ALTERNATE_DB_ENVIRONMENT,
 	CONFIG_ENVIRONMENT,
 	CONFIG_DATA_ENVIRONMENT,
+	CONFIG_COUNT_ENVIRONMENT,
 	DB_ENVIRONMENT,
 	GIT_DIR_ENVIRONMENT,
 	GIT_WORK_TREE_ENVIRONMENT,
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 35a1a6e8b1..e06961767f 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1421,6 +1421,117 @@ test_expect_success '--config-env handles keys with equals' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git config handles environment config pairs' '
+	GIT_CONFIG_COUNT=2 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="foo" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="bar" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one foo
+	pair.two bar
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs without count' '
+	test_must_fail env GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one 2>error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one
+'
+
+test_expect_success 'git config ignores pairs exceeding count' '
+	GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="value" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with empty count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT= GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config fails with invalid count' '
+	test_must_fail env GIT_CONFIG_COUNT=10a git config --list 2>error &&
+	test_i18ngrep "bogus count" error &&
+	test_must_fail env GIT_CONFIG_COUNT=9999999999999999 git config --list 2>error &&
+	test_i18ngrep "too many entries" error
+'
+
+test_expect_success 'git config fails with missing config key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_VALUE_0="value" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config key" error
+'
+
+test_expect_success 'git config fails with missing config value' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0="pair.one" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config value" error
+'
+
+test_expect_success 'git config fails with invalid config pair key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0= GIT_CONFIG_VALUE_0=value \
+		git config --list &&
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0=missing-section GIT_CONFIG_VALUE_0=value \
+		git config --list
+'
+
+test_expect_success 'environment overrides config file' '
+	test_when_finished "rm -f .git/config" &&
+	cat >.git/config <<-EOF &&
+	[pair]
+	one = value
+	EOF
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=override \
+		git config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'GIT_CONFIG_PARAMETERS overrides environment config' '
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
+		GIT_CONFIG_PARAMETERS="${SQ}pair.one=override${SQ}" \
+		git config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'command line overrides environment config' '
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
+		git -c pair.one=override config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
@@ -1766,9 +1877,11 @@ test_expect_success '--show-origin with --list' '
 	file:.git/config	user.override=local
 	file:.git/config	include.path=../include/relative.include
 	file:.git/../include/relative.include	user.relative=include
+	command line:	user.environ=true
 	command line:	user.cmdline=true
 	EOF
-	git -c user.cmdline=true config --list --show-origin >output &&
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=user.environ GIT_CONFIG_VALUE_0=true\
+		git -c user.cmdline=true config --list --show-origin >output &&
 	test_cmp expect output
 '
 
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 3/8] quote: make sq_dequote_step() a public function
  2020-12-16  7:52 ` [PATCH v5 0/8] " Patrick Steinhardt
                     ` (4 preceding siblings ...)
  2020-12-16  7:54   ` [PATCH v5 8/8] config: allow specifying config entries via envvar pairs Patrick Steinhardt
@ 2020-12-16  7:56   ` Patrick Steinhardt
  2020-12-16  7:56   ` [PATCH v5 5/8] config: store "git -c" variables using more robust format Patrick Steinhardt
  2020-12-16  7:57   ` [PATCH v5 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS Patrick Steinhardt
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-16  7:56 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 3152 bytes --]

From: Jeff King <peff@peff.net>

We provide a function for dequoting an entire string, as well as one for
handling a space-separated list of quoted strings. But there's no way
for a caller to parse a string like 'foo'='bar', even though it is easy
to generate one using sq_quote_buf() or similar.

Let's make the single-step function available to callers outside of
quote.c. Note that we do need to adjust its implementation slightly: it
insists on seeing whitespace between items, and we'd like to be more
flexible than that. Since it only has a single caller, we can move that
check (and slurping up any extra whitespace) into that caller.

Signed-off-by: Jeff King <peff@peff.net>
---
 quote.c | 15 ++++++++++-----
 quote.h | 18 ++++++++++++++++--
 2 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/quote.c b/quote.c
index 69f4ca45da..8a3a5e39eb 100644
--- a/quote.c
+++ b/quote.c
@@ -116,7 +116,7 @@ void sq_append_quote_argv_pretty(struct strbuf *dst, const char **argv)
 	}
 }
 
-static char *sq_dequote_step(char *arg, char **next)
+char *sq_dequote_step(char *arg, char **next)
 {
 	char *dst = arg;
 	char *src = arg;
@@ -153,11 +153,8 @@ static char *sq_dequote_step(char *arg, char **next)
 			}
 		/* Fallthrough */
 		default:
-			if (!next || !isspace(*src))
+			if (!next)
 				return NULL;
-			do {
-				c = *++src;
-			} while (isspace(c));
 			*dst = 0;
 			*next = src;
 			return arg;
@@ -182,6 +179,14 @@ static int sq_dequote_to_argv_internal(char *arg,
 		char *dequoted = sq_dequote_step(next, &next);
 		if (!dequoted)
 			return -1;
+		if (next) {
+			char c;
+			if (!isspace(*next))
+				return -1;
+			do {
+				c = *++next;
+			} while (isspace(c));
+		}
 		if (argv) {
 			ALLOC_GROW(*argv, *nr + 1, *alloc);
 			(*argv)[(*nr)++] = dequoted;
diff --git a/quote.h b/quote.h
index 4b72a583cf..768cc6338e 100644
--- a/quote.h
+++ b/quote.h
@@ -42,12 +42,26 @@ void sq_quote_buf_pretty(struct strbuf *, const char *src);
 void sq_quote_argv_pretty(struct strbuf *, const char **argv);
 void sq_append_quote_argv_pretty(struct strbuf *dst, const char **argv);
 
-/* This unwraps what sq_quote() produces in place, but returns
+/*
+ * This unwraps what sq_quote() produces in place, but returns
  * NULL if the input does not look like what sq_quote would have
- * produced.
+ * produced (the full string must be a single quoted item).
  */
 char *sq_dequote(char *);
 
+/*
+ * Like sq_dequote(), but dequote a single item, and leave "next" pointing to
+ * the next character. E.g., in the string:
+ *
+ *   'one' 'two' 'three'
+ *
+ * after the first call, the return value would be the unquoted string "one",
+ * with "next" pointing to the space between "one" and "two"). The caller is
+ * responsible for advancing the pointer to the start of the next item before
+ * calling sq_dequote_step() again.
+ */
+char *sq_dequote_step(char *src, char **next);
+
 /*
  * Same as the above, but can be used to unwrap many arguments in the
  * same string separated by space. Like sq_quote, it works in place,
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 5/8] config: store "git -c" variables using more robust format
  2020-12-16  7:52 ` [PATCH v5 0/8] " Patrick Steinhardt
                     ` (5 preceding siblings ...)
  2020-12-16  7:56   ` [PATCH v5 3/8] quote: make sq_dequote_step() a public function Patrick Steinhardt
@ 2020-12-16  7:56   ` Patrick Steinhardt
  2020-12-16  7:57   ` [PATCH v5 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS Patrick Steinhardt
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-16  7:56 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 4173 bytes --]

From: Jeff King <peff@peff.net>

The previous commit added a new format for $GIT_CONFIG_PARAMETERS which
is able to robustly handle subsections with "=" in them. Let's start
writing the new format. Unfortunately, this does much less than you'd
hope, because "git -c" itself has the same ambiguity problem! But it's
still worth doing:

  - we've now pushed the problem from the inter-process communication
    into the "-c" command-line parser. This would free us up to later
    add an unambiguous format there (e.g., separate arguments like "git
    --config key value", etc).

  - for --config-env, the parser already disallows "=" in the
    environment variable name. So:

      git --config-env section.with=equals.key=ENVVAR

    will robustly set section.with=equals.key to the contents of
    $ENVVAR.

The new test shows the improvement for --config-env.

Signed-off-by: Jeff King <peff@peff.net>
---
 config.c          | 52 ++++++++++++++++++++++++++++++++++++++++-------
 t/t1300-config.sh |  8 ++++++++
 2 files changed, 53 insertions(+), 7 deletions(-)

diff --git a/config.c b/config.c
index 151980e5c9..53ed048689 100644
--- a/config.c
+++ b/config.c
@@ -332,7 +332,7 @@ int git_config_include(const char *var, const char *value, void *data)
 	return ret;
 }
 
-void git_config_push_parameter(const char *text)
+static void git_config_push_split_parameter(const char *key, const char *value)
 {
 	struct strbuf env = STRBUF_INIT;
 	const char *old = getenv(CONFIG_DATA_ENVIRONMENT);
@@ -340,30 +340,68 @@ void git_config_push_parameter(const char *text)
 		strbuf_addstr(&env, old);
 		strbuf_addch(&env, ' ');
 	}
-	sq_quote_buf(&env, text);
+	sq_quote_buf(&env, key);
+	strbuf_addch(&env, '=');
+	if (value)
+		sq_quote_buf(&env, value);
 	setenv(CONFIG_DATA_ENVIRONMENT, env.buf, 1);
 	strbuf_release(&env);
 }
 
+void git_config_push_parameter(const char *text)
+{
+	const char *value;
+
+	/*
+	 * When we see:
+	 *
+	 *   section.subsection=with=equals.key=value
+	 *
+	 * we cannot tell if it means:
+	 *
+	 *   [section "subsection=with=equals"]
+	 *   key = value
+	 *
+	 * or:
+	 *
+	 *   [section]
+	 *   subsection = with=equals.key=value
+	 *
+	 * We parse left-to-right for the first "=", meaning we'll prefer to
+	 * keep the value intact over the subsection. This is historical, but
+	 * also sensible since values are more likely to contain odd or
+	 * untrusted input than a section name.
+	 *
+	 * A missing equals is explicitly allowed (as a bool-only entry).
+	 */
+	value = strchr(text, '=');
+	if (value) {
+		char *key = xmemdupz(text, value - text);
+		git_config_push_split_parameter(key, value + 1);
+		free(key);
+	} else {
+		git_config_push_split_parameter(text, NULL);
+	}
+}
+
 void git_config_push_env(const char *spec)
 {
-	struct strbuf buf = STRBUF_INIT;
+	char *key;
 	const char *env_name;
 	const char *env_value;
 
 	env_name = strrchr(spec, '=');
 	if (!env_name)
 		die("invalid config format: %s", spec);
+	key = xmemdupz(spec, env_name - spec);
 	env_name++;
 
 	env_value = getenv(env_name);
 	if (!env_value)
 		die("config variable missing for '%s'", env_name);
 
-	strbuf_add(&buf, spec, env_name - spec);
-	strbuf_addstr(&buf, env_value);
-	git_config_push_parameter(buf.buf);
-	strbuf_release(&buf);
+	git_config_push_split_parameter(key, env_value);
+	free(key);
 }
 
 static inline int iskeychar(int c)
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 46a94814d5..36a60879f6 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1361,6 +1361,14 @@ test_expect_success 'git -c and --config-env override each other' '
 	test_cmp expect actual
 '
 
+test_expect_success '--config-env handles keys with equals' '
+	echo value=with=equals >expect &&
+	ENVVAR=value=with=equals git \
+		--config-env=section.subsection=with=equals.key=ENVVAR \
+		config section.subsection=with=equals.key >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS
  2020-12-16  7:52 ` [PATCH v5 0/8] " Patrick Steinhardt
                     ` (6 preceding siblings ...)
  2020-12-16  7:56   ` [PATCH v5 5/8] config: store "git -c" variables using more robust format Patrick Steinhardt
@ 2020-12-16  7:57   ` Patrick Steinhardt
  2020-12-16 20:01     ` Phillip Wood
  7 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2020-12-16  7:57 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 8135 bytes --]

From: Jeff King <peff@peff.net>

When we stuff config options into GIT_CONFIG_PARAMETERS, we shell-quote
each one as a single unit, like:

  'section.one=value1' 'section.two=value2'

On the reading side, we de-quote to get the individual strings, and then
parse them by splitting on the first "=" we find. This format is
ambiguous, because an "=" may appear in a subsection. So the config
represented in a file by both:

  [section "subsection=with=equals"]
  key = value

and:

  [section]
  subsection = with=equals.key=value

ends up in this flattened format like:

  'section.subsection=with=equals.key=value'

and we can't tell which was desired. We have traditionally resolved this
by taking the first "=" we see starting from the left, meaning that we
allowed arbitrary content in the value, but not in the subsection.

Let's make our environment format a bit more robust by separately
quoting the key and value. That turns those examples into:

  'section.subsection=with=equals.key'='value'

and:

  'section.subsection'='with=equals.key=value'

respectively, and we can tell the difference between them. We can detect
which format is in use for any given element of the list based on the
presence of the unquoted "=". That means we can continue to allow the
old format to work to support any callers which manually used the old
format, and we can even intermingle the two formats. The old format
wasn't documented, and nobody was supposed to be using it. But it's
likely that such callers exist in the wild, so it's nice if we can avoid
breaking them. Likewise, it may be possible to trigger an older version
of "git -c" that runs a script that calls into a newer version of "git
-c"; that new version would see the intermingled format.

This does create one complication, which is that the obvious format in
the new scheme for

  [section]
  some-bool

is:

  'section.some-bool'

with no equals. We'd mistake that for an old-style variable. And it even
has the same meaning in the old style, but:

  [section "with=equals"]
  some-bool

does not. It would be:

  'section.with=equals=some-bool'

which we'd take to mean:

  [section]
  with = equals=some-bool

in the old, ambiguous style. Likewise, we can't use:

  'section.some-bool'=''

because that's ambiguous with an actual empty string. Instead, we'll
again use the shell-quoting to give us a hint, and use:

  'section.some-bool'=

to show that we have no value.

Note that this commit just expands the reading side. We'll start writing
the new format via "git -c" in a future patch. In the meantime, the
existing "git -c" tests will make sure we didn't break reading the old
format. But we'll also add some explicit coverage of the two formats to
make sure we continue to handle the old one after we move the writing
side over.

And one final note: since we're now using the shell-quoting as a
semantically meaningful hint, this closes the door to us ever allowing
arbitrary shell quoting, like:

  'a'shell'would'be'ok'with'this'.key=value

But we have never supported that (only what sq_quote() would produce),
and we are probably better off keeping things simple, robust, and
backwards-compatible, than trying to make it easier for humans. We'll
continue not to advertise the format of the variable to users, and
instead keep "git -c" as the recommended mechanism for setting config
(even if we are trying to be kind not to break users who may be relying
on the current undocumented format).

Signed-off-by: Jeff King <peff@peff.net>
---
 config.c          | 69 +++++++++++++++++++++++++++++++++++------------
 t/t1300-config.sh | 52 +++++++++++++++++++++++++++++++++++
 2 files changed, 104 insertions(+), 17 deletions(-)

diff --git a/config.c b/config.c
index 53ed048689..60a7261807 100644
--- a/config.c
+++ b/config.c
@@ -541,14 +541,62 @@ int git_config_parse_parameter(const char *text,
 	return ret;
 }
 
+static int parse_config_env_list(char *env, config_fn_t fn, void *data)
+{
+	char *cur = env;
+	while (cur && *cur) {
+		const char *key = sq_dequote_step(cur, &cur);
+		if (!key)
+			return error(_("bogus format in %s"),
+				     CONFIG_DATA_ENVIRONMENT);
+
+		if (!cur || isspace(*cur)) {
+			/* old-style 'key=value' */
+			if (git_config_parse_parameter(key, fn, data) < 0)
+				return -1;
+		}
+		else if (*cur == '=') {
+			/* new-style 'key'='value' */
+			const char *value;
+
+			cur++;
+			if (*cur == '\'') {
+				/* quoted value */
+				value = sq_dequote_step(cur, &cur);
+				if (!value || (cur && !isspace(*cur))) {
+					return error(_("bogus format in %s"),
+						     CONFIG_DATA_ENVIRONMENT);
+				}
+			} else if (!*cur || isspace(*cur)) {
+				/* implicit bool: 'key'= */
+				value = NULL;
+			} else {
+				return error(_("bogus format in %s"),
+					     CONFIG_DATA_ENVIRONMENT);
+			}
+
+			if (config_parse_pair(key, value, fn, data) < 0)
+				return -1;
+		}
+		else {
+			/* unknown format */
+			return error(_("bogus format in %s"),
+				     CONFIG_DATA_ENVIRONMENT);
+		}
+
+		if (cur) {
+			while (isspace(*cur))
+				cur++;
+		}
+	}
+	return 0;
+}
+
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
 	const char *env = getenv(CONFIG_DATA_ENVIRONMENT);
 	int ret = 0;
 	char *envw;
-	const char **argv = NULL;
-	int nr = 0, alloc = 0;
-	int i;
 	struct config_source source;
 
 	if (!env)
@@ -561,21 +609,8 @@ int git_config_from_parameters(config_fn_t fn, void *data)
 
 	/* sq_dequote will write over it */
 	envw = xstrdup(env);
+	ret = parse_config_env_list(envw, fn, data);
 
-	if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
-		ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
-		goto out;
-	}
-
-	for (i = 0; i < nr; i++) {
-		if (git_config_parse_parameter(argv[i], fn, data) < 0) {
-			ret = -1;
-			goto out;
-		}
-	}
-
-out:
-	free(argv);
 	free(envw);
 	cf = source.prev;
 	return ret;
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 36a60879f6..35a1a6e8b1 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1294,6 +1294,58 @@ test_expect_success 'git -c is not confused by empty environment' '
 	GIT_CONFIG_PARAMETERS="" git -c x.one=1 config --list
 '
 
+test_expect_success 'GIT_CONFIG_PARAMETERS handles old-style entries' '
+	v="${SQ}key.one=foo${SQ}" &&
+	v="$v  ${SQ}key.two=bar${SQ}" &&
+	v="$v ${SQ}key.ambiguous=section.whatever=value${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.one foo
+	key.two bar
+	key.ambiguous section.whatever=value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'GIT_CONFIG_PARAMETERS handles new-style entries' '
+	v="${SQ}key.one${SQ}=${SQ}foo${SQ}" &&
+	v="$v  ${SQ}key.two${SQ}=${SQ}bar${SQ}" &&
+	v="$v ${SQ}key.ambiguous=section.whatever${SQ}=${SQ}value${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.one foo
+	key.two bar
+	key.ambiguous=section.whatever value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'old and new-style entries can mix' '
+	v="${SQ}key.oldone=oldfoo${SQ}" &&
+	v="$v ${SQ}key.newone${SQ}=${SQ}newfoo${SQ}" &&
+	v="$v ${SQ}key.oldtwo=oldbar${SQ}" &&
+	v="$v ${SQ}key.newtwo${SQ}=${SQ}newbar${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.oldone oldfoo
+	key.newone newfoo
+	key.oldtwo oldbar
+	key.newtwo newbar
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'old and new bools with ambiguous subsection' '
+	v="${SQ}key.with=equals.oldbool${SQ}" &&
+	v="$v ${SQ}key.with=equals.newbool${SQ}=" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.with equals.oldbool
+	key.with=equals.newbool
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
 	cat >expect <<-\EOF &&
 	env.one one
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v5 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS
  2020-12-16  7:57   ` [PATCH v5 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS Patrick Steinhardt
@ 2020-12-16 20:01     ` Phillip Wood
  0 siblings, 0 replies; 116+ messages in thread
From: Phillip Wood @ 2020-12-16 20:01 UTC (permalink / raw)
  To: Patrick Steinhardt, git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

Hi Patrick/Peff

On 16/12/2020 07:57, Patrick Steinhardt wrote:
> From: Jeff King <peff@peff.net>
> 
> When we stuff config options into GIT_CONFIG_PARAMETERS, we shell-quote
> each one as a single unit, like:
> 
>    'section.one=value1' 'section.two=value2'
> 
> On the reading side, we de-quote to get the individual strings, and then
> parse them by splitting on the first "=" we find. This format is
> ambiguous, because an "=" may appear in a subsection. So the config
> represented in a file by both:
> 
>    [section "subsection=with=equals"]
>    key = value
> 
> and:
> 
>    [section]
>    subsection = with=equals.key=value
> 
> ends up in this flattened format like:
> 
>    'section.subsection=with=equals.key=value'
> 
> and we can't tell which was desired. We have traditionally resolved this
> by taking the first "=" we see starting from the left, meaning that we
> allowed arbitrary content in the value, but not in the subsection.

I was just wondering what happens if a subsection name contains a single 
quote - can we handle that now and how is it affected by this change?

Best Wishes

Phillip

> Let's make our environment format a bit more robust by separately
> quoting the key and value. That turns those examples into:
> 
>    'section.subsection=with=equals.key'='value'
> 
> and:
> 
>    'section.subsection'='with=equals.key=value'
> 
> respectively, and we can tell the difference between them. We can detect
> which format is in use for any given element of the list based on the
> presence of the unquoted "=". That means we can continue to allow the
> old format to work to support any callers which manually used the old
> format, and we can even intermingle the two formats. The old format
> wasn't documented, and nobody was supposed to be using it. But it's
> likely that such callers exist in the wild, so it's nice if we can avoid
> breaking them. Likewise, it may be possible to trigger an older version
> of "git -c" that runs a script that calls into a newer version of "git
> -c"; that new version would see the intermingled format.
> 
> This does create one complication, which is that the obvious format in
> the new scheme for
> 
>    [section]
>    some-bool
> 
> is:
> 
>    'section.some-bool'
> 
> with no equals. We'd mistake that for an old-style variable. And it even
> has the same meaning in the old style, but:
> 
>    [section "with=equals"]
>    some-bool
> 
> does not. It would be:
> 
>    'section.with=equals=some-bool'
> 
> which we'd take to mean:
> 
>    [section]
>    with = equals=some-bool
> 
> in the old, ambiguous style. Likewise, we can't use:
> 
>    'section.some-bool'=''
> 
> because that's ambiguous with an actual empty string. Instead, we'll
> again use the shell-quoting to give us a hint, and use:
> 
>    'section.some-bool'=
> 
> to show that we have no value.
> 
> Note that this commit just expands the reading side. We'll start writing
> the new format via "git -c" in a future patch. In the meantime, the
> existing "git -c" tests will make sure we didn't break reading the old
> format. But we'll also add some explicit coverage of the two formats to
> make sure we continue to handle the old one after we move the writing
> side over.
> 
> And one final note: since we're now using the shell-quoting as a
> semantically meaningful hint, this closes the door to us ever allowing
> arbitrary shell quoting, like:
> 
>    'a'shell'would'be'ok'with'this'.key=value
> 
> But we have never supported that (only what sq_quote() would produce),
> and we are probably better off keeping things simple, robust, and
> backwards-compatible, than trying to make it easier for humans. We'll
> continue not to advertise the format of the variable to users, and
> instead keep "git -c" as the recommended mechanism for setting config
> (even if we are trying to be kind not to break users who may be relying
> on the current undocumented format).
> 
> Signed-off-by: Jeff King <peff@peff.net>
> ---
>   config.c          | 69 +++++++++++++++++++++++++++++++++++------------
>   t/t1300-config.sh | 52 +++++++++++++++++++++++++++++++++++
>   2 files changed, 104 insertions(+), 17 deletions(-)
> 
> diff --git a/config.c b/config.c
> index 53ed048689..60a7261807 100644
> --- a/config.c
> +++ b/config.c
> @@ -541,14 +541,62 @@ int git_config_parse_parameter(const char *text,
>   	return ret;
>   }
>   
> +static int parse_config_env_list(char *env, config_fn_t fn, void *data)
> +{
> +	char *cur = env;
> +	while (cur && *cur) {
> +		const char *key = sq_dequote_step(cur, &cur);
> +		if (!key)
> +			return error(_("bogus format in %s"),
> +				     CONFIG_DATA_ENVIRONMENT);
> +
> +		if (!cur || isspace(*cur)) {
> +			/* old-style 'key=value' */
> +			if (git_config_parse_parameter(key, fn, data) < 0)
> +				return -1;
> +		}
> +		else if (*cur == '=') {
> +			/* new-style 'key'='value' */
> +			const char *value;
> +
> +			cur++;
> +			if (*cur == '\'') {
> +				/* quoted value */
> +				value = sq_dequote_step(cur, &cur);
> +				if (!value || (cur && !isspace(*cur))) {
> +					return error(_("bogus format in %s"),
> +						     CONFIG_DATA_ENVIRONMENT);
> +				}
> +			} else if (!*cur || isspace(*cur)) {
> +				/* implicit bool: 'key'= */
> +				value = NULL;
> +			} else {
> +				return error(_("bogus format in %s"),
> +					     CONFIG_DATA_ENVIRONMENT);
> +			}
> +
> +			if (config_parse_pair(key, value, fn, data) < 0)
> +				return -1;
> +		}
> +		else {
> +			/* unknown format */
> +			return error(_("bogus format in %s"),
> +				     CONFIG_DATA_ENVIRONMENT);
> +		}
> +
> +		if (cur) {
> +			while (isspace(*cur))
> +				cur++;
> +		}
> +	}
> +	return 0;
> +}
> +
>   int git_config_from_parameters(config_fn_t fn, void *data)
>   {
>   	const char *env = getenv(CONFIG_DATA_ENVIRONMENT);
>   	int ret = 0;
>   	char *envw;
> -	const char **argv = NULL;
> -	int nr = 0, alloc = 0;
> -	int i;
>   	struct config_source source;
>   
>   	if (!env)
> @@ -561,21 +609,8 @@ int git_config_from_parameters(config_fn_t fn, void *data)
>   
>   	/* sq_dequote will write over it */
>   	envw = xstrdup(env);
> +	ret = parse_config_env_list(envw, fn, data);
>   
> -	if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
> -		ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
> -		goto out;
> -	}
> -
> -	for (i = 0; i < nr; i++) {
> -		if (git_config_parse_parameter(argv[i], fn, data) < 0) {
> -			ret = -1;
> -			goto out;
> -		}
> -	}
> -
> -out:
> -	free(argv);
>   	free(envw);
>   	cf = source.prev;
>   	return ret;
> diff --git a/t/t1300-config.sh b/t/t1300-config.sh
> index 36a60879f6..35a1a6e8b1 100755
> --- a/t/t1300-config.sh
> +++ b/t/t1300-config.sh
> @@ -1294,6 +1294,58 @@ test_expect_success 'git -c is not confused by empty environment' '
>   	GIT_CONFIG_PARAMETERS="" git -c x.one=1 config --list
>   '
>   
> +test_expect_success 'GIT_CONFIG_PARAMETERS handles old-style entries' '
> +	v="${SQ}key.one=foo${SQ}" &&
> +	v="$v  ${SQ}key.two=bar${SQ}" &&
> +	v="$v ${SQ}key.ambiguous=section.whatever=value${SQ}" &&
> +	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
> +	cat >expect <<-EOF &&
> +	key.one foo
> +	key.two bar
> +	key.ambiguous section.whatever=value
> +	EOF
> +	test_cmp expect actual
> +'
> +
> +test_expect_success 'GIT_CONFIG_PARAMETERS handles new-style entries' '
> +	v="${SQ}key.one${SQ}=${SQ}foo${SQ}" &&
> +	v="$v  ${SQ}key.two${SQ}=${SQ}bar${SQ}" &&
> +	v="$v ${SQ}key.ambiguous=section.whatever${SQ}=${SQ}value${SQ}" &&
> +	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
> +	cat >expect <<-EOF &&
> +	key.one foo
> +	key.two bar
> +	key.ambiguous=section.whatever value
> +	EOF
> +	test_cmp expect actual
> +'
> +
> +test_expect_success 'old and new-style entries can mix' '
> +	v="${SQ}key.oldone=oldfoo${SQ}" &&
> +	v="$v ${SQ}key.newone${SQ}=${SQ}newfoo${SQ}" &&
> +	v="$v ${SQ}key.oldtwo=oldbar${SQ}" &&
> +	v="$v ${SQ}key.newtwo${SQ}=${SQ}newbar${SQ}" &&
> +	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
> +	cat >expect <<-EOF &&
> +	key.oldone oldfoo
> +	key.newone newfoo
> +	key.oldtwo oldbar
> +	key.newtwo newbar
> +	EOF
> +	test_cmp expect actual
> +'
> +
> +test_expect_success 'old and new bools with ambiguous subsection' '
> +	v="${SQ}key.with=equals.oldbool${SQ}" &&
> +	v="$v ${SQ}key.with=equals.newbool${SQ}=" &&
> +	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
> +	cat >expect <<-EOF &&
> +	key.with equals.oldbool
> +	key.with=equals.newbool
> +	EOF
> +	test_cmp expect actual
> +'
> +
>   test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
>   	cat >expect <<-\EOF &&
>   	env.one one
> 

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v5 8/8] config: allow specifying config entries via envvar pairs
  2020-12-16  7:54   ` [PATCH v5 8/8] config: allow specifying config entries via envvar pairs Patrick Steinhardt
@ 2020-12-23 21:14     ` Junio C Hamano
  2020-12-23 21:55       ` Junio C Hamano
  0 siblings, 1 reply; 116+ messages in thread
From: Junio C Hamano @ 2020-12-23 21:14 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Jeff King,
	brian m. carlson, Philip Oakley

Patrick Steinhardt <ps@pks.im> writes:

> While we currently have the `GIT_CONFIG_PARAMETERS` environment variable
> which can be used to pass runtime configuration data to git processes,
> it's an internal implementation detail and not supposed to be used by
> end users.
>
> Next to being for internal use only, this way of passing config entries
> has a major downside: the config keys need to be parsed as they contain
> both key and value in a single variable. As such, it is left to the user
> to escape any potentially harmful characters in the value, which is
> quite hard to do if values are controlled by a third party.
>
> This commit thus adds a new way of adding config entries via the
> environment which gets rid of this shortcoming. If the user passes the
> `GIT_CONFIG_COUNT=$n` environment variable, Git will parse environment
> variable pairs `GIT_CONFIG_KEY_$i` and `GIT_CONFIG_VALUE_$i` for each
> `i` in `[0,n)`.
>
> While the same can be achieved with `git -c <name>=<value>`, one may
> wish to not do so for potentially sensitive information. E.g. if one
> wants to set `http.extraHeader` to contain an authentication token,
> doing so via `-c` would trivially leak those credentials via e.g. ps(1),
> which typically also shows command arguments.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  Documentation/git-config.txt |  16 +++++
>  cache.h                      |   1 +
>  config.c                     |  67 +++++++++++++++++---
>  environment.c                |   1 +
>  t/t1300-config.sh            | 115 ++++++++++++++++++++++++++++++++++-
>  5 files changed, 191 insertions(+), 9 deletions(-)
>
> diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt
> index 0e9351d3cb..72ccea4419 100644
> --- a/Documentation/git-config.txt
> +++ b/Documentation/git-config.txt
> @@ -346,6 +346,22 @@ GIT_CONFIG_NOSYSTEM::
>  
>  See also <<FILES>>.
>  
> +GIT_CONFIG_COUNT::
> +GIT_CONFIG_KEY_<n>::
> +GIT_CONFIG_VALUE_<n>::
> +	If GIT_CONFIG_COUNT is set to a positive number, all environment pairs
> +	GIT_CONFIG_KEY_<n> and GIT_CONFIG_VALUE_<n> up to that number will be
> +	added to the process's runtime configuration. The config pairs are
> +	zero-indexed. Any missing key or value is treated as an error. An empty
> +	GIT_CONFIG_COUNT is treated the same as GIT_CONFIG_COUNT=0, namely no
> +	pairs are processed. These environment variables will override values
> +	in configuration files, but will be overridden by any explicit options
> +	passed via `git -c`.
> +
> +	This is useful for cases where you want to spawn multiple git commands
> +	with a common configuration but cannot depend on a configuration file,
> +	for example when writing scripts.

Dedent these three lines, and replace the blank lines before it with
a line with a single '+' on it (an example is found in the paragraph
that describes the "--get-color" option; look for "type=color" in
the same file).  Otherwise these subsequent paragraphs are treated
differently from the first paragraph.

The same problem may exist in new paragraphs in git.txt that
describes the "--config-env" stuff.

Thanks.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v5 2/8] config: add new way to pass config via `--config-env`
  2020-12-16  7:52   ` [PATCH v5 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
@ 2020-12-23 21:35     ` Junio C Hamano
  0 siblings, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2020-12-23 21:35 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Jeff King,
	brian m. carlson, Philip Oakley

Patrick Steinhardt <ps@pks.im> writes:

> +--config-env=<name>=<envvar>::
> +	Like `-c <name>=<var>` except the value is the name of an
> +	environment variable from which to retrieve the value. Unlike

Let's avoid overusing the word "value", as it can refer to
<name>=<envvar> as the whole (which is the value given to
--config-env), or <envvar> itself (which may appear to the value
given to <name>), or the value in the environment veraible.

	Like `-c <name>=<value>`, give configuration variable
	'<name>' a value, where <envvar> is the name of an
	environment variable from which to retrieve the
	value.

or something along that line.

> +	`-c` there is no shortcut for directly setting the value to an
> +	empty string, instead the environment variable itself must be
> +	set to the empty strin. Errors if the `<envvar>` does not exist

	set to the empty string.  It is an error if the ...

> +	in the environment. `<envvar>` may not contain an equals sign
> +	to avoid ambiguity with `<name>`s which contain one.

	which may contain one.

> +	This is useful for cases where you want to pass transitory
> +	configuration options to git, but are doing so on OS's where
> +	other processes might be able to read your cmdline
> +	(e.g. `/proc/self/cmdline`), but not your environ
> +	(e.g. `/proc/self/environ`). That behavior is the default on
> +	Linux, but may not be on your system.
> +
> +	Note that this might add security for variables such as
> +	`http.extraHeader` where the sensitive information is part of
> +	the value, but not e.g. `url.<base.insteadOf` where the

"url.<base>.insteadOf"

> +	sensitive information can be part of the key.

When writing multi-paragraph description, the second and later
paragraphs need to be dedented and the paragraph breaks are denoted
not by a blank line but by a line with only a single '+' on it.

I didn't look at the implementation or tests, as I think it hasn't
changed since the last round, and the last round was looked at by
Peff already.

Thanks.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v5 8/8] config: allow specifying config entries via envvar pairs
  2020-12-23 21:14     ` Junio C Hamano
@ 2020-12-23 21:55       ` Junio C Hamano
  2021-01-06 10:28         ` Patrick Steinhardt
  0 siblings, 1 reply; 116+ messages in thread
From: Junio C Hamano @ 2020-12-23 21:55 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Jeff King,
	brian m. carlson, Philip Oakley

Junio C Hamano <gitster@pobox.com> writes:

> The same problem may exist in new paragraphs in git.txt that
> describes the "--config-env" stuff.

Here is what I tentatively queued on top of these 8 patches as a fixup.

Thanks.


 Documentation/git-config.txt |  8 ++++----
 Documentation/git.txt        | 29 +++++++++++++++--------------
 2 files changed, 19 insertions(+), 18 deletions(-)

diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt
index b71c1ac7b8..67eb40f506 100644
--- a/Documentation/git-config.txt
+++ b/Documentation/git-config.txt
@@ -348,10 +348,10 @@ GIT_CONFIG_VALUE_<n>::
 	pairs are processed. These environment variables will override values
 	in configuration files, but will be overridden by any explicit options
 	passed via `git -c`.
-
-	This is useful for cases where you want to spawn multiple git commands
-	with a common configuration but cannot depend on a configuration file,
-	for example when writing scripts.
++
+This is useful for cases where you want to spawn multiple git commands
+with a common configuration but cannot depend on a configuration file,
+for example when writing scripts.
 
 
 [[EXAMPLES]]
diff --git a/Documentation/git.txt b/Documentation/git.txt
index 80fb8fab11..3b0f87a71b 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -81,25 +81,26 @@ foo.bar= ...`) sets `foo.bar` to the empty string which `git config
 --type=bool` will convert to `false`.
 
 --config-env=<name>=<envvar>::
-	Like `-c <name>=<var>` except the value is the name of an
+	Like `-c <name>=<value>`, give configuration variable
+	'<name>' a value, where <envvar> is the name of an
 	environment variable from which to retrieve the value. Unlike
 	`-c` there is no shortcut for directly setting the value to an
 	empty string, instead the environment variable itself must be
-	set to the empty strin. Errors if the `<envvar>` does not exist
+	set to the empty string.  It is an error if the `<envvar>` does not exist
 	in the environment. `<envvar>` may not contain an equals sign
 	to avoid ambiguity with `<name>`s which contain one.
-
-	This is useful for cases where you want to pass transitory
-	configuration options to git, but are doing so on OS's where
-	other processes might be able to read your cmdline
-	(e.g. `/proc/self/cmdline`), but not your environ
-	(e.g. `/proc/self/environ`). That behavior is the default on
-	Linux, but may not be on your system.
-
-	Note that this might add security for variables such as
-	`http.extraHeader` where the sensitive information is part of
-	the value, but not e.g. `url.<base.insteadOf` where the
-	sensitive information can be part of the key.
++
+This is useful for cases where you want to pass transitory
+configuration options to git, but are doing so on OS's where
+other processes might be able to read your cmdline
+(e.g. `/proc/self/cmdline`), but not your environ
+(e.g. `/proc/self/environ`). That behavior is the default on
+Linux, but may not be on your system.
++
+Note that this might add security for variables such as
+`http.extraHeader` where the sensitive information is part of
+the value, but not e.g. `url.<base>.insteadOf` where the
+sensitive information can be part of the key.
 
 --exec-path[=<path>]::
 	Path to wherever your core Git programs are installed.
-- 
2.30.0-rc1-197-ga312a798fc


^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v5 8/8] config: allow specifying config entries via envvar pairs
  2020-12-23 21:55       ` Junio C Hamano
@ 2021-01-06 10:28         ` Patrick Steinhardt
  2021-01-06 21:07           ` Junio C Hamano
  0 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-06 10:28 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Ævar Arnfjörð Bjarmason, Jeff King,
	brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 3696 bytes --]

On Wed, Dec 23, 2020 at 01:55:08PM -0800, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > The same problem may exist in new paragraphs in git.txt that
> > describes the "--config-env" stuff.
> 
> Here is what I tentatively queued on top of these 8 patches as a fixup.
> 
> Thanks.

Your changes look good to me, thanks!

Patrick

> 
>  Documentation/git-config.txt |  8 ++++----
>  Documentation/git.txt        | 29 +++++++++++++++--------------
>  2 files changed, 19 insertions(+), 18 deletions(-)
> 
> diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt
> index b71c1ac7b8..67eb40f506 100644
> --- a/Documentation/git-config.txt
> +++ b/Documentation/git-config.txt
> @@ -348,10 +348,10 @@ GIT_CONFIG_VALUE_<n>::
>  	pairs are processed. These environment variables will override values
>  	in configuration files, but will be overridden by any explicit options
>  	passed via `git -c`.
> -
> -	This is useful for cases where you want to spawn multiple git commands
> -	with a common configuration but cannot depend on a configuration file,
> -	for example when writing scripts.
> ++
> +This is useful for cases where you want to spawn multiple git commands
> +with a common configuration but cannot depend on a configuration file,
> +for example when writing scripts.
>  
>  
>  [[EXAMPLES]]
> diff --git a/Documentation/git.txt b/Documentation/git.txt
> index 80fb8fab11..3b0f87a71b 100644
> --- a/Documentation/git.txt
> +++ b/Documentation/git.txt
> @@ -81,25 +81,26 @@ foo.bar= ...`) sets `foo.bar` to the empty string which `git config
>  --type=bool` will convert to `false`.
>  
>  --config-env=<name>=<envvar>::
> -	Like `-c <name>=<var>` except the value is the name of an
> +	Like `-c <name>=<value>`, give configuration variable
> +	'<name>' a value, where <envvar> is the name of an
>  	environment variable from which to retrieve the value. Unlike
>  	`-c` there is no shortcut for directly setting the value to an
>  	empty string, instead the environment variable itself must be
> -	set to the empty strin. Errors if the `<envvar>` does not exist
> +	set to the empty string.  It is an error if the `<envvar>` does not exist
>  	in the environment. `<envvar>` may not contain an equals sign
>  	to avoid ambiguity with `<name>`s which contain one.
> -
> -	This is useful for cases where you want to pass transitory
> -	configuration options to git, but are doing so on OS's where
> -	other processes might be able to read your cmdline
> -	(e.g. `/proc/self/cmdline`), but not your environ
> -	(e.g. `/proc/self/environ`). That behavior is the default on
> -	Linux, but may not be on your system.
> -
> -	Note that this might add security for variables such as
> -	`http.extraHeader` where the sensitive information is part of
> -	the value, but not e.g. `url.<base.insteadOf` where the
> -	sensitive information can be part of the key.
> ++
> +This is useful for cases where you want to pass transitory
> +configuration options to git, but are doing so on OS's where
> +other processes might be able to read your cmdline
> +(e.g. `/proc/self/cmdline`), but not your environ
> +(e.g. `/proc/self/environ`). That behavior is the default on
> +Linux, but may not be on your system.
> ++
> +Note that this might add security for variables such as
> +`http.extraHeader` where the sensitive information is part of
> +the value, but not e.g. `url.<base>.insteadOf` where the
> +sensitive information can be part of the key.
>  
>  --exec-path[=<path>]::
>  	Path to wherever your core Git programs are installed.
> -- 
> 2.30.0-rc1-197-ga312a798fc
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v5 8/8] config: allow specifying config entries via envvar pairs
  2021-01-06 10:28         ` Patrick Steinhardt
@ 2021-01-06 21:07           ` Junio C Hamano
  0 siblings, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2021-01-06 21:07 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Ævar Arnfjörð Bjarmason, Jeff King,
	brian m. carlson, Philip Oakley

Patrick Steinhardt <ps@pks.im> writes:

> On Wed, Dec 23, 2020 at 01:55:08PM -0800, Junio C Hamano wrote:
>> Junio C Hamano <gitster@pobox.com> writes:
>> 
>> > The same problem may exist in new paragraphs in git.txt that
>> > describes the "--config-env" stuff.
>> 
>> Here is what I tentatively queued on top of these 8 patches as a fixup.
>> 
>> Thanks.
>
> Your changes look good to me, thanks!

You're welcome.  Looking forward to seeing a new round with these
minor fixes squashed in, so that we do not have a series with known
breakages in early parts that are fixed in later steps.

Thanks.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v6 0/8] config: allow specifying config entries via env
  2020-11-24 10:50 [PATCH v2 0/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
                   ` (5 preceding siblings ...)
  2020-12-16  7:52 ` [PATCH v5 0/8] " Patrick Steinhardt
@ 2021-01-07  6:36 ` Patrick Steinhardt
  2021-01-07  6:36   ` [PATCH v6 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
                     ` (7 more replies)
  2021-01-11  8:36 ` [PATCH v7 0/8] " Patrick Steinhardt
  2021-01-12 12:26 ` [PATCH v8 0/8] " Patrick Steinhardt
  8 siblings, 8 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-07  6:36 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 5095 bytes --]

Hi,

this is the sixth version of my patch series which aims to implement a
way to pass config entries via the environment while avoiding any
requirements to perform shell quoting on the user's side.

The only change in this version is improved formatting and wording of
the documentation as proposed by Junio. Please see the attached
range-diff.

Patrick

Jeff King (3):
  quote: make sq_dequote_step() a public function
  config: store "git -c" variables using more robust format
  config: parse more robust format in GIT_CONFIG_PARAMETERS

Patrick Steinhardt (5):
  git: add `--super-prefix` to usage string
  config: add new way to pass config via `--config-env`
  config: extract function to parse config pairs
  environment: make `getenv_safe()` a public function
  config: allow specifying config entries via envvar pairs

 Documentation/git-config.txt |  16 +++
 Documentation/git.txt        |  24 +++-
 cache.h                      |   1 +
 config.c                     | 205 ++++++++++++++++++++++++++++----
 config.h                     |   1 +
 environment.c                |   8 +-
 environment.h                |  12 ++
 git.c                        |   3 +
 quote.c                      |  15 ++-
 quote.h                      |  18 ++-
 t/t1300-config.sh            | 220 ++++++++++++++++++++++++++++++++++-
 11 files changed, 484 insertions(+), 39 deletions(-)
 create mode 100644 environment.h

Range-diff against v5:
1:  470521e728 = 1:  cd3de0743a git: add `--super-prefix` to usage string
2:  56c9221c4c ! 2:  9b8461010e config: add new way to pass config via `--config-env`
    @@ Documentation/git.txt: config file). Including the equals but with an empty valu
      --type=bool` will convert to `false`.
      
     +--config-env=<name>=<envvar>::
    -+	Like `-c <name>=<var>` except the value is the name of an
    ++	Like `-c <name>=<value>`, give configuration variable
    ++	'<name>' a value, where <envvar> is the name of an
     +	environment variable from which to retrieve the value. Unlike
     +	`-c` there is no shortcut for directly setting the value to an
     +	empty string, instead the environment variable itself must be
    -+	set to the empty strin. Errors if the `<envvar>` does not exist
    ++	set to the empty string.  It is an error if the `<envvar>` does not exist
     +	in the environment. `<envvar>` may not contain an equals sign
     +	to avoid ambiguity with `<name>`s which contain one.
    -+
    -+	This is useful for cases where you want to pass transitory
    -+	configuration options to git, but are doing so on OS's where
    -+	other processes might be able to read your cmdline
    -+	(e.g. `/proc/self/cmdline`), but not your environ
    -+	(e.g. `/proc/self/environ`). That behavior is the default on
    -+	Linux, but may not be on your system.
    -+
    -+	Note that this might add security for variables such as
    -+	`http.extraHeader` where the sensitive information is part of
    -+	the value, but not e.g. `url.<base.insteadOf` where the
    -+	sensitive information can be part of the key.
    +++
    ++This is useful for cases where you want to pass transitory
    ++configuration options to git, but are doing so on OS's where
    ++other processes might be able to read your cmdline
    ++(e.g. `/proc/self/cmdline`), but not your environ
    ++(e.g. `/proc/self/environ`). That behavior is the default on
    ++Linux, but may not be on your system.
    +++
    ++Note that this might add security for variables such as
    ++`http.extraHeader` where the sensitive information is part of
    ++the value, but not e.g. `url.<base>.insteadOf` where the
    ++sensitive information can be part of the key.
     +
      --exec-path[=<path>]::
      	Path to wherever your core Git programs are installed.
3:  5729f5d406 = 3:  9d4c8d7be9 quote: make sq_dequote_step() a public function
4:  8c6cdd57a0 = 4:  0a9b085fe5 config: extract function to parse config pairs
5:  ff96e59e79 = 5:  b96686c9cd config: store "git -c" variables using more robust format
6:  d832f3dedf = 6:  6597700ffb config: parse more robust format in GIT_CONFIG_PARAMETERS
7:  2f51a0c5fc = 7:  cade8fb12f environment: make `getenv_safe()` a public function
8:  dfceffd8d4 ! 8:  4e3f208d13 config: allow specifying config entries via envvar pairs
    @@ Documentation/git-config.txt: GIT_CONFIG_NOSYSTEM::
     +	pairs are processed. These environment variables will override values
     +	in configuration files, but will be overridden by any explicit options
     +	passed via `git -c`.
    -+
    -+	This is useful for cases where you want to spawn multiple git commands
    -+	with a common configuration but cannot depend on a configuration file,
    -+	for example when writing scripts.
    +++
    ++This is useful for cases where you want to spawn multiple git commands
    ++with a common configuration but cannot depend on a configuration file,
    ++for example when writing scripts.
     +
      
      [[EXAMPLES]]
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v6 1/8] git: add `--super-prefix` to usage string
  2021-01-07  6:36 ` [PATCH v6 0/8] config: allow specifying config entries via env Patrick Steinhardt
@ 2021-01-07  6:36   ` Patrick Steinhardt
  2021-01-07  6:36   ` [PATCH v6 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
                     ` (6 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-07  6:36 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1057 bytes --]

When the `--super-prefix` option was implmented in 74866d7579 (git: make
super-prefix option, 2016-10-07), its existence was only documented in
the manpage but not in the command's own usage string. Given that the
commit message didn't mention that this was done intentionally and given
that it's documented in the manpage, this seems like an oversight.

Add it to the usage string to fix the inconsistency.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 git.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/git.c b/git.c
index a00a0a4d94..5a8ff12f87 100644
--- a/git.c
+++ b/git.c
@@ -29,6 +29,7 @@ const char git_usage_string[] =
 	   "           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]\n"
 	   "           [-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]\n"
 	   "           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]\n"
+	   "           [--super-prefix=<path>]\n"
 	   "           <command> [<args>]");
 
 const char git_more_info_string[] =
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 2/8] config: add new way to pass config via `--config-env`
  2021-01-07  6:36 ` [PATCH v6 0/8] config: allow specifying config entries via env Patrick Steinhardt
  2021-01-07  6:36   ` [PATCH v6 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
@ 2021-01-07  6:36   ` Patrick Steinhardt
  2021-01-10 20:29     ` Simon Ruderich
  2021-01-07  6:36   ` [PATCH v6 3/8] quote: make sq_dequote_step() a public function Patrick Steinhardt
                     ` (5 subsequent siblings)
  7 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-07  6:36 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 7198 bytes --]

While it's already possible to pass runtime configuration via `git -c
<key>=<value>`, it may be undesirable to use when the value contains
sensitive information. E.g. if one wants to set `http.extraHeader` to
contain an authentication token, doing so via `-c` would trivially leak
those credentials via e.g. ps(1), which typically also shows command
arguments.

To enable this usecase without leaking credentials, this commit
introduces a new switch `--config-env=<key>=<envvar>`. Instead of
directly passing a value for the given key, it instead allows the user
to specify the name of an environment variable. The value of that
variable will then be used as value of the key.

Co-authored-by: Jeff King <peff@peff.net>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git.txt | 24 ++++++++++++++++++++++-
 config.c              | 21 ++++++++++++++++++++
 config.h              |  1 +
 git.c                 |  4 +++-
 t/t1300-config.sh     | 45 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 93 insertions(+), 2 deletions(-)

diff --git a/Documentation/git.txt b/Documentation/git.txt
index a6d4ad0818..d36e6fd482 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -13,7 +13,7 @@ SYNOPSIS
     [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
     [-p|--paginate|-P|--no-pager] [--no-replace-objects] [--bare]
     [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
-    [--super-prefix=<path>]
+    [--super-prefix=<path>] [--config-env <name>=<envvar>]
     <command> [<args>]
 
 DESCRIPTION
@@ -80,6 +80,28 @@ config file). Including the equals but with an empty value (like `git -c
 foo.bar= ...`) sets `foo.bar` to the empty string which `git config
 --type=bool` will convert to `false`.
 
+--config-env=<name>=<envvar>::
+	Like `-c <name>=<value>`, give configuration variable
+	'<name>' a value, where <envvar> is the name of an
+	environment variable from which to retrieve the value. Unlike
+	`-c` there is no shortcut for directly setting the value to an
+	empty string, instead the environment variable itself must be
+	set to the empty string.  It is an error if the `<envvar>` does not exist
+	in the environment. `<envvar>` may not contain an equals sign
+	to avoid ambiguity with `<name>`s which contain one.
++
+This is useful for cases where you want to pass transitory
+configuration options to git, but are doing so on OS's where
+other processes might be able to read your cmdline
+(e.g. `/proc/self/cmdline`), but not your environ
+(e.g. `/proc/self/environ`). That behavior is the default on
+Linux, but may not be on your system.
++
+Note that this might add security for variables such as
+`http.extraHeader` where the sensitive information is part of
+the value, but not e.g. `url.<base>.insteadOf` where the
+sensitive information can be part of the key.
+
 --exec-path[=<path>]::
 	Path to wherever your core Git programs are installed.
 	This can also be controlled by setting the GIT_EXEC_PATH
diff --git a/config.c b/config.c
index 1137bd73af..cde3511110 100644
--- a/config.c
+++ b/config.c
@@ -345,6 +345,27 @@ void git_config_push_parameter(const char *text)
 	strbuf_release(&env);
 }
 
+void git_config_push_env(const char *spec)
+{
+	struct strbuf buf = STRBUF_INIT;
+	const char *env_name;
+	const char *env_value;
+
+	env_name = strrchr(spec, '=');
+	if (!env_name)
+		die("invalid config format: %s", spec);
+	env_name++;
+
+	env_value = getenv(env_name);
+	if (!env_value)
+		die("config variable missing for '%s'", env_name);
+
+	strbuf_add(&buf, spec, env_name - spec);
+	strbuf_addstr(&buf, env_value);
+	git_config_push_parameter(buf.buf);
+	strbuf_release(&buf);
+}
+
 static inline int iskeychar(int c)
 {
 	return isalnum(c) || c == '-';
diff --git a/config.h b/config.h
index c1449bb790..19a9adbaa9 100644
--- a/config.h
+++ b/config.h
@@ -138,6 +138,7 @@ int git_config_from_mem(config_fn_t fn,
 int git_config_from_blob_oid(config_fn_t fn, const char *name,
 			     const struct object_id *oid, void *data);
 void git_config_push_parameter(const char *text);
+void git_config_push_env(const char *spec);
 int git_config_from_parameters(config_fn_t fn, void *data);
 void read_early_config(config_fn_t cb, void *data);
 void read_very_early_config(config_fn_t cb, void *data);
diff --git a/git.c b/git.c
index 5a8ff12f87..b5f63d346b 100644
--- a/git.c
+++ b/git.c
@@ -29,7 +29,7 @@ const char git_usage_string[] =
 	   "           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]\n"
 	   "           [-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]\n"
 	   "           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]\n"
-	   "           [--super-prefix=<path>]\n"
+	   "           [--super-prefix=<path>] [--config-env=<name>=<envvar>]\n"
 	   "           <command> [<args>]");
 
 const char git_more_info_string[] =
@@ -255,6 +255,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			git_config_push_parameter((*argv)[1]);
 			(*argv)++;
 			(*argc)--;
+		} else if (skip_prefix(cmd, "--config-env=", &cmd)) {
+			git_config_push_env(cmd);
 		} else if (!strcmp(cmd, "--literal-pathspecs")) {
 			setenv(GIT_LITERAL_PATHSPECS_ENVIRONMENT, "1", 1);
 			if (envchanged)
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 97a04c6cc2..46a94814d5 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1316,6 +1316,51 @@ test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
 		git config --get-regexp "env.*"
 '
 
+test_expect_success 'git --config-env=key=envvar support' '
+	cat >expect <<-\EOF &&
+	value
+	value
+	false
+	EOF
+	{
+		env ENVVAR=value git --config-env=core.name=ENVVAR config core.name &&
+		env ENVVAR=value git --config-env=foo.CamelCase=ENVVAR config foo.camelcase &&
+		env ENVVAR= git --config-env=foo.flag=ENVVAR config --bool foo.flag
+	} >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git --config-env fails with invalid parameters' '
+	test_must_fail git --config-env=foo.flag config --bool foo.flag 2>error &&
+	test_i18ngrep "invalid config format" error &&
+	test_must_fail git --config-env=foo.flag=NONEXISTENT config --bool foo.flag 2>error &&
+	test_i18ngrep "config variable missing" error
+'
+
+test_expect_success 'git -c and --config-env work together' '
+	cat >expect <<-\EOF &&
+	bar.cmd cmd-value
+	bar.env env-value
+	EOF
+	env ENVVAR=env-value git \
+		-c bar.cmd=cmd-value \
+		--config-env=bar.env=ENVVAR \
+		config --get-regexp "^bar.*" >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git -c and --config-env override each other' '
+	cat >expect <<-\EOF &&
+	env
+	cmd
+	EOF
+	{
+		env ENVVAR=env git -c bar.bar=cmd --config-env=bar.bar=ENVVAR config bar.bar &&
+		env ENVVAR=env git --config-env=bar.bar=ENVVAR -c bar.bar=cmd config bar.bar
+	} >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 3/8] quote: make sq_dequote_step() a public function
  2021-01-07  6:36 ` [PATCH v6 0/8] config: allow specifying config entries via env Patrick Steinhardt
  2021-01-07  6:36   ` [PATCH v6 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
  2021-01-07  6:36   ` [PATCH v6 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
@ 2021-01-07  6:36   ` Patrick Steinhardt
  2021-01-07  6:37   ` [PATCH v6 4/8] config: extract function to parse config pairs Patrick Steinhardt
                     ` (4 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-07  6:36 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 3152 bytes --]

From: Jeff King <peff@peff.net>

We provide a function for dequoting an entire string, as well as one for
handling a space-separated list of quoted strings. But there's no way
for a caller to parse a string like 'foo'='bar', even though it is easy
to generate one using sq_quote_buf() or similar.

Let's make the single-step function available to callers outside of
quote.c. Note that we do need to adjust its implementation slightly: it
insists on seeing whitespace between items, and we'd like to be more
flexible than that. Since it only has a single caller, we can move that
check (and slurping up any extra whitespace) into that caller.

Signed-off-by: Jeff King <peff@peff.net>
---
 quote.c | 15 ++++++++++-----
 quote.h | 18 ++++++++++++++++--
 2 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/quote.c b/quote.c
index 69f4ca45da..8a3a5e39eb 100644
--- a/quote.c
+++ b/quote.c
@@ -116,7 +116,7 @@ void sq_append_quote_argv_pretty(struct strbuf *dst, const char **argv)
 	}
 }
 
-static char *sq_dequote_step(char *arg, char **next)
+char *sq_dequote_step(char *arg, char **next)
 {
 	char *dst = arg;
 	char *src = arg;
@@ -153,11 +153,8 @@ static char *sq_dequote_step(char *arg, char **next)
 			}
 		/* Fallthrough */
 		default:
-			if (!next || !isspace(*src))
+			if (!next)
 				return NULL;
-			do {
-				c = *++src;
-			} while (isspace(c));
 			*dst = 0;
 			*next = src;
 			return arg;
@@ -182,6 +179,14 @@ static int sq_dequote_to_argv_internal(char *arg,
 		char *dequoted = sq_dequote_step(next, &next);
 		if (!dequoted)
 			return -1;
+		if (next) {
+			char c;
+			if (!isspace(*next))
+				return -1;
+			do {
+				c = *++next;
+			} while (isspace(c));
+		}
 		if (argv) {
 			ALLOC_GROW(*argv, *nr + 1, *alloc);
 			(*argv)[(*nr)++] = dequoted;
diff --git a/quote.h b/quote.h
index 4b72a583cf..768cc6338e 100644
--- a/quote.h
+++ b/quote.h
@@ -42,12 +42,26 @@ void sq_quote_buf_pretty(struct strbuf *, const char *src);
 void sq_quote_argv_pretty(struct strbuf *, const char **argv);
 void sq_append_quote_argv_pretty(struct strbuf *dst, const char **argv);
 
-/* This unwraps what sq_quote() produces in place, but returns
+/*
+ * This unwraps what sq_quote() produces in place, but returns
  * NULL if the input does not look like what sq_quote would have
- * produced.
+ * produced (the full string must be a single quoted item).
  */
 char *sq_dequote(char *);
 
+/*
+ * Like sq_dequote(), but dequote a single item, and leave "next" pointing to
+ * the next character. E.g., in the string:
+ *
+ *   'one' 'two' 'three'
+ *
+ * after the first call, the return value would be the unquoted string "one",
+ * with "next" pointing to the space between "one" and "two"). The caller is
+ * responsible for advancing the pointer to the start of the next item before
+ * calling sq_dequote_step() again.
+ */
+char *sq_dequote_step(char *src, char **next);
+
 /*
  * Same as the above, but can be used to unwrap many arguments in the
  * same string separated by space. Like sq_quote, it works in place,
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 4/8] config: extract function to parse config pairs
  2021-01-07  6:36 ` [PATCH v6 0/8] config: allow specifying config entries via env Patrick Steinhardt
                     ` (2 preceding siblings ...)
  2021-01-07  6:36   ` [PATCH v6 3/8] quote: make sq_dequote_step() a public function Patrick Steinhardt
@ 2021-01-07  6:37   ` Patrick Steinhardt
  2021-01-07  6:37   ` [PATCH v6 5/8] config: store "git -c" variables using more robust format Patrick Steinhardt
                     ` (3 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-07  6:37 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1833 bytes --]

The function `git_config_parse_parameter` is responsible for parsing a
`foo.bar=baz`-formatted configuration key, sanitizing the key and then
processing it via the given callback function. Given that we're about to
add a second user which is going to process keys which already has keys
and values separated, this commit extracts a function
`config_parse_pair` which only does the sanitization and processing
part as a preparatory step.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 config.c | 24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/config.c b/config.c
index cde3511110..151980e5c9 100644
--- a/config.c
+++ b/config.c
@@ -458,11 +458,26 @@ int git_config_key_is_valid(const char *key)
 	return !git_config_parse_key_1(key, NULL, NULL, 1);
 }
 
+static int config_parse_pair(const char *key, const char *value,
+			  config_fn_t fn, void *data)
+{
+	char *canonical_name;
+	int ret;
+
+	if (!strlen(key))
+		return error(_("empty config key"));
+	if (git_config_parse_key(key, &canonical_name, NULL))
+		return -1;
+
+	ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
+	free(canonical_name);
+	return ret;
+}
+
 int git_config_parse_parameter(const char *text,
 			       config_fn_t fn, void *data)
 {
 	const char *value;
-	char *canonical_name;
 	struct strbuf **pair;
 	int ret;
 
@@ -483,12 +498,7 @@ int git_config_parse_parameter(const char *text,
 		return error(_("bogus config parameter: %s"), text);
 	}
 
-	if (git_config_parse_key(pair[0]->buf, &canonical_name, NULL)) {
-		ret = -1;
-	} else {
-		ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
-		free(canonical_name);
-	}
+	ret = config_parse_pair(pair[0]->buf, value, fn, data);
 	strbuf_list_free(pair);
 	return ret;
 }
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 5/8] config: store "git -c" variables using more robust format
  2021-01-07  6:36 ` [PATCH v6 0/8] config: allow specifying config entries via env Patrick Steinhardt
                     ` (3 preceding siblings ...)
  2021-01-07  6:37   ` [PATCH v6 4/8] config: extract function to parse config pairs Patrick Steinhardt
@ 2021-01-07  6:37   ` Patrick Steinhardt
  2021-01-07  6:37   ` [PATCH v6 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS Patrick Steinhardt
                     ` (2 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-07  6:37 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 4173 bytes --]

From: Jeff King <peff@peff.net>

The previous commit added a new format for $GIT_CONFIG_PARAMETERS which
is able to robustly handle subsections with "=" in them. Let's start
writing the new format. Unfortunately, this does much less than you'd
hope, because "git -c" itself has the same ambiguity problem! But it's
still worth doing:

  - we've now pushed the problem from the inter-process communication
    into the "-c" command-line parser. This would free us up to later
    add an unambiguous format there (e.g., separate arguments like "git
    --config key value", etc).

  - for --config-env, the parser already disallows "=" in the
    environment variable name. So:

      git --config-env section.with=equals.key=ENVVAR

    will robustly set section.with=equals.key to the contents of
    $ENVVAR.

The new test shows the improvement for --config-env.

Signed-off-by: Jeff King <peff@peff.net>
---
 config.c          | 52 ++++++++++++++++++++++++++++++++++++++++-------
 t/t1300-config.sh |  8 ++++++++
 2 files changed, 53 insertions(+), 7 deletions(-)

diff --git a/config.c b/config.c
index 151980e5c9..53ed048689 100644
--- a/config.c
+++ b/config.c
@@ -332,7 +332,7 @@ int git_config_include(const char *var, const char *value, void *data)
 	return ret;
 }
 
-void git_config_push_parameter(const char *text)
+static void git_config_push_split_parameter(const char *key, const char *value)
 {
 	struct strbuf env = STRBUF_INIT;
 	const char *old = getenv(CONFIG_DATA_ENVIRONMENT);
@@ -340,30 +340,68 @@ void git_config_push_parameter(const char *text)
 		strbuf_addstr(&env, old);
 		strbuf_addch(&env, ' ');
 	}
-	sq_quote_buf(&env, text);
+	sq_quote_buf(&env, key);
+	strbuf_addch(&env, '=');
+	if (value)
+		sq_quote_buf(&env, value);
 	setenv(CONFIG_DATA_ENVIRONMENT, env.buf, 1);
 	strbuf_release(&env);
 }
 
+void git_config_push_parameter(const char *text)
+{
+	const char *value;
+
+	/*
+	 * When we see:
+	 *
+	 *   section.subsection=with=equals.key=value
+	 *
+	 * we cannot tell if it means:
+	 *
+	 *   [section "subsection=with=equals"]
+	 *   key = value
+	 *
+	 * or:
+	 *
+	 *   [section]
+	 *   subsection = with=equals.key=value
+	 *
+	 * We parse left-to-right for the first "=", meaning we'll prefer to
+	 * keep the value intact over the subsection. This is historical, but
+	 * also sensible since values are more likely to contain odd or
+	 * untrusted input than a section name.
+	 *
+	 * A missing equals is explicitly allowed (as a bool-only entry).
+	 */
+	value = strchr(text, '=');
+	if (value) {
+		char *key = xmemdupz(text, value - text);
+		git_config_push_split_parameter(key, value + 1);
+		free(key);
+	} else {
+		git_config_push_split_parameter(text, NULL);
+	}
+}
+
 void git_config_push_env(const char *spec)
 {
-	struct strbuf buf = STRBUF_INIT;
+	char *key;
 	const char *env_name;
 	const char *env_value;
 
 	env_name = strrchr(spec, '=');
 	if (!env_name)
 		die("invalid config format: %s", spec);
+	key = xmemdupz(spec, env_name - spec);
 	env_name++;
 
 	env_value = getenv(env_name);
 	if (!env_value)
 		die("config variable missing for '%s'", env_name);
 
-	strbuf_add(&buf, spec, env_name - spec);
-	strbuf_addstr(&buf, env_value);
-	git_config_push_parameter(buf.buf);
-	strbuf_release(&buf);
+	git_config_push_split_parameter(key, env_value);
+	free(key);
 }
 
 static inline int iskeychar(int c)
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 46a94814d5..36a60879f6 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1361,6 +1361,14 @@ test_expect_success 'git -c and --config-env override each other' '
 	test_cmp expect actual
 '
 
+test_expect_success '--config-env handles keys with equals' '
+	echo value=with=equals >expect &&
+	ENVVAR=value=with=equals git \
+		--config-env=section.subsection=with=equals.key=ENVVAR \
+		config section.subsection=with=equals.key >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS
  2021-01-07  6:36 ` [PATCH v6 0/8] config: allow specifying config entries via env Patrick Steinhardt
                     ` (4 preceding siblings ...)
  2021-01-07  6:37   ` [PATCH v6 5/8] config: store "git -c" variables using more robust format Patrick Steinhardt
@ 2021-01-07  6:37   ` Patrick Steinhardt
  2021-01-07  6:37   ` [PATCH v6 7/8] environment: make `getenv_safe()` a public function Patrick Steinhardt
  2021-01-07  6:37   ` [PATCH v6 8/8] config: allow specifying config entries via envvar pairs Patrick Steinhardt
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-07  6:37 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 8135 bytes --]

From: Jeff King <peff@peff.net>

When we stuff config options into GIT_CONFIG_PARAMETERS, we shell-quote
each one as a single unit, like:

  'section.one=value1' 'section.two=value2'

On the reading side, we de-quote to get the individual strings, and then
parse them by splitting on the first "=" we find. This format is
ambiguous, because an "=" may appear in a subsection. So the config
represented in a file by both:

  [section "subsection=with=equals"]
  key = value

and:

  [section]
  subsection = with=equals.key=value

ends up in this flattened format like:

  'section.subsection=with=equals.key=value'

and we can't tell which was desired. We have traditionally resolved this
by taking the first "=" we see starting from the left, meaning that we
allowed arbitrary content in the value, but not in the subsection.

Let's make our environment format a bit more robust by separately
quoting the key and value. That turns those examples into:

  'section.subsection=with=equals.key'='value'

and:

  'section.subsection'='with=equals.key=value'

respectively, and we can tell the difference between them. We can detect
which format is in use for any given element of the list based on the
presence of the unquoted "=". That means we can continue to allow the
old format to work to support any callers which manually used the old
format, and we can even intermingle the two formats. The old format
wasn't documented, and nobody was supposed to be using it. But it's
likely that such callers exist in the wild, so it's nice if we can avoid
breaking them. Likewise, it may be possible to trigger an older version
of "git -c" that runs a script that calls into a newer version of "git
-c"; that new version would see the intermingled format.

This does create one complication, which is that the obvious format in
the new scheme for

  [section]
  some-bool

is:

  'section.some-bool'

with no equals. We'd mistake that for an old-style variable. And it even
has the same meaning in the old style, but:

  [section "with=equals"]
  some-bool

does not. It would be:

  'section.with=equals=some-bool'

which we'd take to mean:

  [section]
  with = equals=some-bool

in the old, ambiguous style. Likewise, we can't use:

  'section.some-bool'=''

because that's ambiguous with an actual empty string. Instead, we'll
again use the shell-quoting to give us a hint, and use:

  'section.some-bool'=

to show that we have no value.

Note that this commit just expands the reading side. We'll start writing
the new format via "git -c" in a future patch. In the meantime, the
existing "git -c" tests will make sure we didn't break reading the old
format. But we'll also add some explicit coverage of the two formats to
make sure we continue to handle the old one after we move the writing
side over.

And one final note: since we're now using the shell-quoting as a
semantically meaningful hint, this closes the door to us ever allowing
arbitrary shell quoting, like:

  'a'shell'would'be'ok'with'this'.key=value

But we have never supported that (only what sq_quote() would produce),
and we are probably better off keeping things simple, robust, and
backwards-compatible, than trying to make it easier for humans. We'll
continue not to advertise the format of the variable to users, and
instead keep "git -c" as the recommended mechanism for setting config
(even if we are trying to be kind not to break users who may be relying
on the current undocumented format).

Signed-off-by: Jeff King <peff@peff.net>
---
 config.c          | 69 +++++++++++++++++++++++++++++++++++------------
 t/t1300-config.sh | 52 +++++++++++++++++++++++++++++++++++
 2 files changed, 104 insertions(+), 17 deletions(-)

diff --git a/config.c b/config.c
index 53ed048689..60a7261807 100644
--- a/config.c
+++ b/config.c
@@ -541,14 +541,62 @@ int git_config_parse_parameter(const char *text,
 	return ret;
 }
 
+static int parse_config_env_list(char *env, config_fn_t fn, void *data)
+{
+	char *cur = env;
+	while (cur && *cur) {
+		const char *key = sq_dequote_step(cur, &cur);
+		if (!key)
+			return error(_("bogus format in %s"),
+				     CONFIG_DATA_ENVIRONMENT);
+
+		if (!cur || isspace(*cur)) {
+			/* old-style 'key=value' */
+			if (git_config_parse_parameter(key, fn, data) < 0)
+				return -1;
+		}
+		else if (*cur == '=') {
+			/* new-style 'key'='value' */
+			const char *value;
+
+			cur++;
+			if (*cur == '\'') {
+				/* quoted value */
+				value = sq_dequote_step(cur, &cur);
+				if (!value || (cur && !isspace(*cur))) {
+					return error(_("bogus format in %s"),
+						     CONFIG_DATA_ENVIRONMENT);
+				}
+			} else if (!*cur || isspace(*cur)) {
+				/* implicit bool: 'key'= */
+				value = NULL;
+			} else {
+				return error(_("bogus format in %s"),
+					     CONFIG_DATA_ENVIRONMENT);
+			}
+
+			if (config_parse_pair(key, value, fn, data) < 0)
+				return -1;
+		}
+		else {
+			/* unknown format */
+			return error(_("bogus format in %s"),
+				     CONFIG_DATA_ENVIRONMENT);
+		}
+
+		if (cur) {
+			while (isspace(*cur))
+				cur++;
+		}
+	}
+	return 0;
+}
+
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
 	const char *env = getenv(CONFIG_DATA_ENVIRONMENT);
 	int ret = 0;
 	char *envw;
-	const char **argv = NULL;
-	int nr = 0, alloc = 0;
-	int i;
 	struct config_source source;
 
 	if (!env)
@@ -561,21 +609,8 @@ int git_config_from_parameters(config_fn_t fn, void *data)
 
 	/* sq_dequote will write over it */
 	envw = xstrdup(env);
+	ret = parse_config_env_list(envw, fn, data);
 
-	if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
-		ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
-		goto out;
-	}
-
-	for (i = 0; i < nr; i++) {
-		if (git_config_parse_parameter(argv[i], fn, data) < 0) {
-			ret = -1;
-			goto out;
-		}
-	}
-
-out:
-	free(argv);
 	free(envw);
 	cf = source.prev;
 	return ret;
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 36a60879f6..35a1a6e8b1 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1294,6 +1294,58 @@ test_expect_success 'git -c is not confused by empty environment' '
 	GIT_CONFIG_PARAMETERS="" git -c x.one=1 config --list
 '
 
+test_expect_success 'GIT_CONFIG_PARAMETERS handles old-style entries' '
+	v="${SQ}key.one=foo${SQ}" &&
+	v="$v  ${SQ}key.two=bar${SQ}" &&
+	v="$v ${SQ}key.ambiguous=section.whatever=value${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.one foo
+	key.two bar
+	key.ambiguous section.whatever=value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'GIT_CONFIG_PARAMETERS handles new-style entries' '
+	v="${SQ}key.one${SQ}=${SQ}foo${SQ}" &&
+	v="$v  ${SQ}key.two${SQ}=${SQ}bar${SQ}" &&
+	v="$v ${SQ}key.ambiguous=section.whatever${SQ}=${SQ}value${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.one foo
+	key.two bar
+	key.ambiguous=section.whatever value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'old and new-style entries can mix' '
+	v="${SQ}key.oldone=oldfoo${SQ}" &&
+	v="$v ${SQ}key.newone${SQ}=${SQ}newfoo${SQ}" &&
+	v="$v ${SQ}key.oldtwo=oldbar${SQ}" &&
+	v="$v ${SQ}key.newtwo${SQ}=${SQ}newbar${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.oldone oldfoo
+	key.newone newfoo
+	key.oldtwo oldbar
+	key.newtwo newbar
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'old and new bools with ambiguous subsection' '
+	v="${SQ}key.with=equals.oldbool${SQ}" &&
+	v="$v ${SQ}key.with=equals.newbool${SQ}=" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.with equals.oldbool
+	key.with=equals.newbool
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
 	cat >expect <<-\EOF &&
 	env.one one
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 7/8] environment: make `getenv_safe()` a public function
  2021-01-07  6:36 ` [PATCH v6 0/8] config: allow specifying config entries via env Patrick Steinhardt
                     ` (5 preceding siblings ...)
  2021-01-07  6:37   ` [PATCH v6 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS Patrick Steinhardt
@ 2021-01-07  6:37   ` Patrick Steinhardt
  2021-01-07  6:37   ` [PATCH v6 8/8] config: allow specifying config entries via envvar pairs Patrick Steinhardt
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-07  6:37 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1661 bytes --]

The `getenv_safe()` helper function helps to safely retrieve multiple
environment values without the need to depend on platform-specific
behaviour for the return value's lifetime. We'll make use of this
function in a following patch, so let's make it available by making it
non-static and adding a declaration.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 environment.c |  7 ++-----
 environment.h | 12 ++++++++++++
 2 files changed, 14 insertions(+), 5 deletions(-)
 create mode 100644 environment.h

diff --git a/environment.c b/environment.c
index bb518c61cd..2234af462c 100644
--- a/environment.c
+++ b/environment.c
@@ -9,6 +9,7 @@
  */
 #include "cache.h"
 #include "branch.h"
+#include "environment.h"
 #include "repository.h"
 #include "config.h"
 #include "refs.h"
@@ -152,11 +153,7 @@ static char *expand_namespace(const char *raw_namespace)
 	return strbuf_detach(&buf, NULL);
 }
 
-/*
- * Wrapper of getenv() that returns a strdup value. This value is kept
- * in argv to be freed later.
- */
-static const char *getenv_safe(struct strvec *argv, const char *name)
+const char *getenv_safe(struct strvec *argv, const char *name)
 {
 	const char *value = getenv(name);
 
diff --git a/environment.h b/environment.h
new file mode 100644
index 0000000000..d438b5c8f3
--- /dev/null
+++ b/environment.h
@@ -0,0 +1,12 @@
+#ifndef ENVIRONMENT_H
+#define ENVIRONMENT_H
+
+#include "strvec.h"
+
+/*
+ * Wrapper of getenv() that returns a strdup value. This value is kept
+ * in argv to be freed later.
+ */
+const char *getenv_safe(struct strvec *argv, const char *name);
+
+#endif
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 8/8] config: allow specifying config entries via envvar pairs
  2021-01-07  6:36 ` [PATCH v6 0/8] config: allow specifying config entries via env Patrick Steinhardt
                     ` (6 preceding siblings ...)
  2021-01-07  6:37   ` [PATCH v6 7/8] environment: make `getenv_safe()` a public function Patrick Steinhardt
@ 2021-01-07  6:37   ` Patrick Steinhardt
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-07  6:37 UTC (permalink / raw)
  To: git
  Cc: Ævar Arnfjörð Bjarmason, Junio C Hamano,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 10490 bytes --]

While we currently have the `GIT_CONFIG_PARAMETERS` environment variable
which can be used to pass runtime configuration data to git processes,
it's an internal implementation detail and not supposed to be used by
end users.

Next to being for internal use only, this way of passing config entries
has a major downside: the config keys need to be parsed as they contain
both key and value in a single variable. As such, it is left to the user
to escape any potentially harmful characters in the value, which is
quite hard to do if values are controlled by a third party.

This commit thus adds a new way of adding config entries via the
environment which gets rid of this shortcoming. If the user passes the
`GIT_CONFIG_COUNT=$n` environment variable, Git will parse environment
variable pairs `GIT_CONFIG_KEY_$i` and `GIT_CONFIG_VALUE_$i` for each
`i` in `[0,n)`.

While the same can be achieved with `git -c <name>=<value>`, one may
wish to not do so for potentially sensitive information. E.g. if one
wants to set `http.extraHeader` to contain an authentication token,
doing so via `-c` would trivially leak those credentials via e.g. ps(1),
which typically also shows command arguments.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git-config.txt |  16 +++++
 cache.h                      |   1 +
 config.c                     |  67 +++++++++++++++++---
 environment.c                |   1 +
 t/t1300-config.sh            | 115 ++++++++++++++++++++++++++++++++++-
 5 files changed, 191 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt
index 0e9351d3cb..4b4cc5c5e8 100644
--- a/Documentation/git-config.txt
+++ b/Documentation/git-config.txt
@@ -346,6 +346,22 @@ GIT_CONFIG_NOSYSTEM::
 
 See also <<FILES>>.
 
+GIT_CONFIG_COUNT::
+GIT_CONFIG_KEY_<n>::
+GIT_CONFIG_VALUE_<n>::
+	If GIT_CONFIG_COUNT is set to a positive number, all environment pairs
+	GIT_CONFIG_KEY_<n> and GIT_CONFIG_VALUE_<n> up to that number will be
+	added to the process's runtime configuration. The config pairs are
+	zero-indexed. Any missing key or value is treated as an error. An empty
+	GIT_CONFIG_COUNT is treated the same as GIT_CONFIG_COUNT=0, namely no
+	pairs are processed. These environment variables will override values
+	in configuration files, but will be overridden by any explicit options
+	passed via `git -c`.
++
+This is useful for cases where you want to spawn multiple git commands
+with a common configuration but cannot depend on a configuration file,
+for example when writing scripts.
+
 
 [[EXAMPLES]]
 EXAMPLES
diff --git a/cache.h b/cache.h
index 7109765748..a2e318c62b 100644
--- a/cache.h
+++ b/cache.h
@@ -472,6 +472,7 @@ static inline enum object_type object_type(unsigned int mode)
 #define TEMPLATE_DIR_ENVIRONMENT "GIT_TEMPLATE_DIR"
 #define CONFIG_ENVIRONMENT "GIT_CONFIG"
 #define CONFIG_DATA_ENVIRONMENT "GIT_CONFIG_PARAMETERS"
+#define CONFIG_COUNT_ENVIRONMENT "GIT_CONFIG_COUNT"
 #define EXEC_PATH_ENVIRONMENT "GIT_EXEC_PATH"
 #define CEILING_DIRECTORIES_ENVIRONMENT "GIT_CEILING_DIRECTORIES"
 #define NO_REPLACE_OBJECTS_ENVIRONMENT "GIT_NO_REPLACE_OBJECTS"
diff --git a/config.c b/config.c
index 60a7261807..1742aefa3e 100644
--- a/config.c
+++ b/config.c
@@ -8,6 +8,7 @@
 #include "cache.h"
 #include "branch.h"
 #include "config.h"
+#include "environment.h"
 #include "repository.h"
 #include "lockfile.h"
 #include "exec-cmd.h"
@@ -594,23 +595,73 @@ static int parse_config_env_list(char *env, config_fn_t fn, void *data)
 
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
-	const char *env = getenv(CONFIG_DATA_ENVIRONMENT);
+	const char *env;
+	struct strbuf envvar = STRBUF_INIT;
+	struct strvec to_free = STRVEC_INIT;
 	int ret = 0;
-	char *envw;
+	char *envw = NULL;
 	struct config_source source;
 
-	if (!env)
-		return 0;
-
 	memset(&source, 0, sizeof(source));
 	source.prev = cf;
 	source.origin_type = CONFIG_ORIGIN_CMDLINE;
 	cf = &source;
 
-	/* sq_dequote will write over it */
-	envw = xstrdup(env);
-	ret = parse_config_env_list(envw, fn, data);
+	env = getenv(CONFIG_COUNT_ENVIRONMENT);
+	if (env) {
+		unsigned long count;
+		char *endp;
+		int i;
 
+		count = strtoul(env, &endp, 10);
+		if (*endp) {
+			ret = error(_("bogus count in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+		if (count > INT_MAX) {
+			ret = error(_("too many entries in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+
+		for (i = 0; i < count; i++) {
+			const char *key, *value;
+
+			strbuf_addf(&envvar, "GIT_CONFIG_KEY_%d", i);
+			key = getenv_safe(&to_free, envvar.buf);
+			if (!key) {
+				ret = error(_("missing config key %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			strbuf_addf(&envvar, "GIT_CONFIG_VALUE_%d", i);
+			value = getenv_safe(&to_free, envvar.buf);
+			if (!value) {
+				ret = error(_("missing config value %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			if (config_parse_pair(key, value, fn, data) < 0) {
+				ret = -1;
+				goto out;
+			}
+		}
+	}
+
+	env = getenv(CONFIG_DATA_ENVIRONMENT);
+	if (env) {
+		/* sq_dequote will write over it */
+		envw = xstrdup(env);
+		if (parse_config_env_list(envw, fn, data) < 0) {
+			ret = -1;
+			goto out;
+		}
+	}
+
+out:
+	strbuf_release(&envvar);
+	strvec_clear(&to_free);
 	free(envw);
 	cf = source.prev;
 	return ret;
diff --git a/environment.c b/environment.c
index 2234af462c..2f27008424 100644
--- a/environment.c
+++ b/environment.c
@@ -117,6 +117,7 @@ const char * const local_repo_env[] = {
 	ALTERNATE_DB_ENVIRONMENT,
 	CONFIG_ENVIRONMENT,
 	CONFIG_DATA_ENVIRONMENT,
+	CONFIG_COUNT_ENVIRONMENT,
 	DB_ENVIRONMENT,
 	GIT_DIR_ENVIRONMENT,
 	GIT_WORK_TREE_ENVIRONMENT,
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 35a1a6e8b1..e06961767f 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1421,6 +1421,117 @@ test_expect_success '--config-env handles keys with equals' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git config handles environment config pairs' '
+	GIT_CONFIG_COUNT=2 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="foo" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="bar" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one foo
+	pair.two bar
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs without count' '
+	test_must_fail env GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one 2>error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one
+'
+
+test_expect_success 'git config ignores pairs exceeding count' '
+	GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="value" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with empty count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT= GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config fails with invalid count' '
+	test_must_fail env GIT_CONFIG_COUNT=10a git config --list 2>error &&
+	test_i18ngrep "bogus count" error &&
+	test_must_fail env GIT_CONFIG_COUNT=9999999999999999 git config --list 2>error &&
+	test_i18ngrep "too many entries" error
+'
+
+test_expect_success 'git config fails with missing config key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_VALUE_0="value" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config key" error
+'
+
+test_expect_success 'git config fails with missing config value' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0="pair.one" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config value" error
+'
+
+test_expect_success 'git config fails with invalid config pair key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0= GIT_CONFIG_VALUE_0=value \
+		git config --list &&
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0=missing-section GIT_CONFIG_VALUE_0=value \
+		git config --list
+'
+
+test_expect_success 'environment overrides config file' '
+	test_when_finished "rm -f .git/config" &&
+	cat >.git/config <<-EOF &&
+	[pair]
+	one = value
+	EOF
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=override \
+		git config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'GIT_CONFIG_PARAMETERS overrides environment config' '
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
+		GIT_CONFIG_PARAMETERS="${SQ}pair.one=override${SQ}" \
+		git config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'command line overrides environment config' '
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
+		git -c pair.one=override config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
@@ -1766,9 +1877,11 @@ test_expect_success '--show-origin with --list' '
 	file:.git/config	user.override=local
 	file:.git/config	include.path=../include/relative.include
 	file:.git/../include/relative.include	user.relative=include
+	command line:	user.environ=true
 	command line:	user.cmdline=true
 	EOF
-	git -c user.cmdline=true config --list --show-origin >output &&
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=user.environ GIT_CONFIG_VALUE_0=true\
+		git -c user.cmdline=true config --list --show-origin >output &&
 	test_cmp expect output
 '
 
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v6 2/8] config: add new way to pass config via `--config-env`
  2021-01-07  6:36   ` [PATCH v6 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
@ 2021-01-10 20:29     ` Simon Ruderich
  2021-01-11  0:29       ` Junio C Hamano
  0 siblings, 1 reply; 116+ messages in thread
From: Simon Ruderich @ 2021-01-10 20:29 UTC (permalink / raw)
  To: git
  Cc: Patrick Steinhardt, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 802 bytes --]

On Thu, Jan 07, 2021 at 07:36:52AM +0100, Patrick Steinhardt wrote:
> [snip]
>
> +void git_config_push_env(const char *spec)
> +{
> +	struct strbuf buf = STRBUF_INIT;
> +	const char *env_name;
> +	const char *env_value;
> +
> +	env_name = strrchr(spec, '=');
> +	if (!env_name)
> +		die("invalid config format: %s", spec);
> +	env_name++;
> +
> +	env_value = getenv(env_name);
> +	if (!env_value)
> +		die("config variable missing for '%s'", env_name);

I think "environment variable" should be mentioned in the error
message to make it clear what kind of "variable" is missing.

Btw. shouldn't these strings get translated (or does die() do
that automatically)?

Regards
Simon
-- 
+ privacy is necessary
+ using gnupg http://gnupg.org
+ public key id: 0x92FEFDB7E44C32F9

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v6 2/8] config: add new way to pass config via `--config-env`
  2021-01-10 20:29     ` Simon Ruderich
@ 2021-01-11  0:29       ` Junio C Hamano
  2021-01-11  8:24         ` Patrick Steinhardt
  0 siblings, 1 reply; 116+ messages in thread
From: Junio C Hamano @ 2021-01-11  0:29 UTC (permalink / raw)
  To: Simon Ruderich
  Cc: git, Patrick Steinhardt, Ævar Arnfjörð Bjarmason,
	Jeff King, brian m. carlson, Philip Oakley

Simon Ruderich <simon@ruderich.org> writes:

> On Thu, Jan 07, 2021 at 07:36:52AM +0100, Patrick Steinhardt wrote:
>> [snip]
>>
>> +void git_config_push_env(const char *spec)
>> +{
>> +	struct strbuf buf = STRBUF_INIT;
>> +	const char *env_name;
>> +	const char *env_value;
>> +
>> +	env_name = strrchr(spec, '=');
>> +	if (!env_name)
>> +		die("invalid config format: %s", spec);
>> +	env_name++;
>> +
>> +	env_value = getenv(env_name);
>> +	if (!env_value)
>> +		die("config variable missing for '%s'", env_name);
>
> I think "environment variable" should be mentioned in the error
> message to make it clear what kind of "variable" is missing.

Good observation.  This parses foo=bar and complains about bar
missing in the environment; It is not a "config variable" that is
missing.

It is "'bar', which is supposed to be there whose value is going to
be used as the value of configuration variable 'foo', is missing."

I wonder if we should also talk about 'foo' at the same time as a
hint for what went wrong?  E.g.

	die(_("missing environment variable '%s' for configuration '%.*s'"),
            env_name, (int)((env_name-1) - spec), spec);

I don't offhand know if that is too much info that may not be all
that useful, though.

> Btw. shouldn't these strings get translated (or does die() do
> that automatically)?

The format string given to die/error/warn should be marked with _().

Thanks.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v6 2/8] config: add new way to pass config via `--config-env`
  2021-01-11  0:29       ` Junio C Hamano
@ 2021-01-11  8:24         ` Patrick Steinhardt
  0 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-11  8:24 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Simon Ruderich, git, Ævar Arnfjörð Bjarmason,
	Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1785 bytes --]

On Sun, Jan 10, 2021 at 04:29:01PM -0800, Junio C Hamano wrote:
> Simon Ruderich <simon@ruderich.org> writes:
> 
> > On Thu, Jan 07, 2021 at 07:36:52AM +0100, Patrick Steinhardt wrote:
> >> [snip]
> >>
> >> +void git_config_push_env(const char *spec)
> >> +{
> >> +	struct strbuf buf = STRBUF_INIT;
> >> +	const char *env_name;
> >> +	const char *env_value;
> >> +
> >> +	env_name = strrchr(spec, '=');
> >> +	if (!env_name)
> >> +		die("invalid config format: %s", spec);
> >> +	env_name++;
> >> +
> >> +	env_value = getenv(env_name);
> >> +	if (!env_value)
> >> +		die("config variable missing for '%s'", env_name);
> >
> > I think "environment variable" should be mentioned in the error
> > message to make it clear what kind of "variable" is missing.
> 
> Good observation.  This parses foo=bar and complains about bar
> missing in the environment; It is not a "config variable" that is
> missing.
> 
> It is "'bar', which is supposed to be there whose value is going to
> be used as the value of configuration variable 'foo', is missing."

Indeed.

> I wonder if we should also talk about 'foo' at the same time as a
> hint for what went wrong?  E.g.
> 
> 	die(_("missing environment variable '%s' for configuration '%.*s'"),
>             env_name, (int)((env_name-1) - spec), spec);
> 
> I don't offhand know if that is too much info that may not be all
> that useful, though.

No, I think that this error message is quite useful as it points out
both what's missing and why we expect it to exist in the first place.

> > Btw. shouldn't these strings get translated (or does die() do
> > that automatically)?
> 
> The format string given to die/error/warn should be marked with _().
> 
> Thanks.

Right, will fix.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v7 0/8] config: allow specifying config entries via envvar pairs
  2020-11-24 10:50 [PATCH v2 0/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
                   ` (6 preceding siblings ...)
  2021-01-07  6:36 ` [PATCH v6 0/8] config: allow specifying config entries via env Patrick Steinhardt
@ 2021-01-11  8:36 ` Patrick Steinhardt
  2021-01-11  8:36   ` [PATCH v7 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
                     ` (7 more replies)
  2021-01-12 12:26 ` [PATCH v8 0/8] " Patrick Steinhardt
  8 siblings, 8 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-11  8:36 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 5038 bytes --]

Hi,

this is the seventh version of my patch series which aims to implement a
way to pass config entries via the environment while avoiding any
requirements to perform shell quoting on the user's side.

The only change in this version is improved error handling for the
`--config-env` switch:

    - Error messages are now correctly marked for translation.

    - A separate error message is given if no value is passed to
      `--config-env`. Previously, we would've tried to look up the
      empty environment variable (`getenv("")`).

    - The error message when the environment variable is missing was
      improved.

Please see the attached ranged-diff for further details.

Patrick

Jeff King (2):
  quote: make sq_dequote_step() a public function
  config: parse more robust format in GIT_CONFIG_PARAMETERS

Patrick Steinhardt (6):
  git: add `--super-prefix` to usage string
  config: add new way to pass config via `--config-env`
  config: extract function to parse config pairs
  config: store "git -c" variables using more robust format
  environment: make `getenv_safe()` a public function
  config: allow specifying config entries via envvar pairs

 Documentation/git-config.txt |  16 +++
 Documentation/git.txt        |  24 +++-
 cache.h                      |   1 +
 config.c                     | 208 ++++++++++++++++++++++++++++----
 config.h                     |   1 +
 environment.c                |   8 +-
 environment.h                |  12 ++
 git.c                        |   3 +
 quote.c                      |  15 ++-
 quote.h                      |  18 ++-
 t/t1300-config.sh            | 222 ++++++++++++++++++++++++++++++++++-
 11 files changed, 489 insertions(+), 39 deletions(-)
 create mode 100644 environment.h

Range-diff against v6:
1:  cd3de0743a = 1:  55fa4d0d11 git: add `--super-prefix` to usage string
2:  9b8461010e ! 2:  b9cf47afe8 config: add new way to pass config via `--config-env`
    @@ config.c: void git_config_push_parameter(const char *text)
     +
     +	env_name = strrchr(spec, '=');
     +	if (!env_name)
    -+		die("invalid config format: %s", spec);
    ++		die(_("invalid config format: %s"), spec);
     +	env_name++;
    ++	if (!*env_name)
    ++		die(_("missing value for --config-env"));
     +
     +	env_value = getenv(env_name);
     +	if (!env_value)
    -+		die("config variable missing for '%s'", env_name);
    ++		die(_("missing environment variable '%s' for configuration '%.*s'"),
    ++		    env_name, (int)(env_name - spec - 1), spec);
     +
     +	strbuf_add(&buf, spec, env_name - spec);
     +	strbuf_addstr(&buf, env_value);
    @@ t/t1300-config.sh: test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
     +test_expect_success 'git --config-env fails with invalid parameters' '
     +	test_must_fail git --config-env=foo.flag config --bool foo.flag 2>error &&
     +	test_i18ngrep "invalid config format" error &&
    ++	test_must_fail git --config-env=foo.flag= config --bool foo.flag 2>error &&
    ++	test_i18ngrep "missing value for --config-env" error &&
     +	test_must_fail git --config-env=foo.flag=NONEXISTENT config --bool foo.flag 2>error &&
    -+	test_i18ngrep "config variable missing" error
    ++	test_i18ngrep "missing environment variable ${SQ}NONEXISTENT${SQ} for configuration ${SQ}foo.flag${SQ}" error
     +'
     +
     +test_expect_success 'git -c and --config-env work together' '
3:  9d4c8d7be9 = 3:  1b47f0db98 quote: make sq_dequote_step() a public function
4:  0a9b085fe5 = 4:  b9565a050e config: extract function to parse config pairs
5:  b96686c9cd ! 5:  8f998ac81a config: store "git -c" variables using more robust format
    @@
      ## Metadata ##
    -Author: Jeff King <peff@peff.net>
    +Author: Patrick Steinhardt <ps@pks.im>
     
      ## Commit message ##
         config: store "git -c" variables using more robust format
    @@ config.c: void git_config_push_parameter(const char *text)
      
      	env_name = strrchr(spec, '=');
      	if (!env_name)
    - 		die("invalid config format: %s", spec);
    + 		die(_("invalid config format: %s"), spec);
     +	key = xmemdupz(spec, env_name - spec);
      	env_name++;
    - 
    - 	env_value = getenv(env_name);
    - 	if (!env_value)
    - 		die("config variable missing for '%s'", env_name);
    + 	if (!*env_name)
    + 		die(_("missing value for --config-env"));
    +@@ config.c: void git_config_push_env(const char *spec)
    + 		die(_("missing environment variable '%s' for configuration '%.*s'"),
    + 		    env_name, (int)(env_name - spec - 1), spec);
      
     -	strbuf_add(&buf, spec, env_name - spec);
     -	strbuf_addstr(&buf, env_value);
6:  6597700ffb = 6:  e7b073c9dc config: parse more robust format in GIT_CONFIG_PARAMETERS
7:  cade8fb12f = 7:  6c1800a18f environment: make `getenv_safe()` a public function
8:  4e3f208d13 = 8:  ac9e778704 config: allow specifying config entries via envvar pairs
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v7 1/8] git: add `--super-prefix` to usage string
  2021-01-11  8:36 ` [PATCH v7 0/8] " Patrick Steinhardt
@ 2021-01-11  8:36   ` Patrick Steinhardt
  2021-01-11  8:36   ` [PATCH v7 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
                     ` (6 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-11  8:36 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1057 bytes --]

When the `--super-prefix` option was implmented in 74866d7579 (git: make
super-prefix option, 2016-10-07), its existence was only documented in
the manpage but not in the command's own usage string. Given that the
commit message didn't mention that this was done intentionally and given
that it's documented in the manpage, this seems like an oversight.

Add it to the usage string to fix the inconsistency.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 git.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/git.c b/git.c
index a00a0a4d94..5a8ff12f87 100644
--- a/git.c
+++ b/git.c
@@ -29,6 +29,7 @@ const char git_usage_string[] =
 	   "           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]\n"
 	   "           [-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]\n"
 	   "           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]\n"
+	   "           [--super-prefix=<path>]\n"
 	   "           <command> [<args>]");
 
 const char git_more_info_string[] =
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v7 2/8] config: add new way to pass config via `--config-env`
  2021-01-11  8:36 ` [PATCH v7 0/8] " Patrick Steinhardt
  2021-01-11  8:36   ` [PATCH v7 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
@ 2021-01-11  8:36   ` Patrick Steinhardt
  2021-01-11 22:34     ` Junio C Hamano
  2021-01-11  8:36   ` [PATCH v7 3/8] quote: make sq_dequote_step() a public function Patrick Steinhardt
                     ` (5 subsequent siblings)
  7 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-11  8:36 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 7540 bytes --]

While it's already possible to pass runtime configuration via `git -c
<key>=<value>`, it may be undesirable to use when the value contains
sensitive information. E.g. if one wants to set `http.extraHeader` to
contain an authentication token, doing so via `-c` would trivially leak
those credentials via e.g. ps(1), which typically also shows command
arguments.

To enable this usecase without leaking credentials, this commit
introduces a new switch `--config-env=<key>=<envvar>`. Instead of
directly passing a value for the given key, it instead allows the user
to specify the name of an environment variable. The value of that
variable will then be used as value of the key.

Co-authored-by: Jeff King <peff@peff.net>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git.txt | 24 +++++++++++++++++++++-
 config.c              | 24 ++++++++++++++++++++++
 config.h              |  1 +
 git.c                 |  4 +++-
 t/t1300-config.sh     | 47 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 98 insertions(+), 2 deletions(-)

diff --git a/Documentation/git.txt b/Documentation/git.txt
index a6d4ad0818..d36e6fd482 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -13,7 +13,7 @@ SYNOPSIS
     [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
     [-p|--paginate|-P|--no-pager] [--no-replace-objects] [--bare]
     [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
-    [--super-prefix=<path>]
+    [--super-prefix=<path>] [--config-env <name>=<envvar>]
     <command> [<args>]
 
 DESCRIPTION
@@ -80,6 +80,28 @@ config file). Including the equals but with an empty value (like `git -c
 foo.bar= ...`) sets `foo.bar` to the empty string which `git config
 --type=bool` will convert to `false`.
 
+--config-env=<name>=<envvar>::
+	Like `-c <name>=<value>`, give configuration variable
+	'<name>' a value, where <envvar> is the name of an
+	environment variable from which to retrieve the value. Unlike
+	`-c` there is no shortcut for directly setting the value to an
+	empty string, instead the environment variable itself must be
+	set to the empty string.  It is an error if the `<envvar>` does not exist
+	in the environment. `<envvar>` may not contain an equals sign
+	to avoid ambiguity with `<name>`s which contain one.
++
+This is useful for cases where you want to pass transitory
+configuration options to git, but are doing so on OS's where
+other processes might be able to read your cmdline
+(e.g. `/proc/self/cmdline`), but not your environ
+(e.g. `/proc/self/environ`). That behavior is the default on
+Linux, but may not be on your system.
++
+Note that this might add security for variables such as
+`http.extraHeader` where the sensitive information is part of
+the value, but not e.g. `url.<base>.insteadOf` where the
+sensitive information can be part of the key.
+
 --exec-path[=<path>]::
 	Path to wherever your core Git programs are installed.
 	This can also be controlled by setting the GIT_EXEC_PATH
diff --git a/config.c b/config.c
index 1137bd73af..6484d13c46 100644
--- a/config.c
+++ b/config.c
@@ -345,6 +345,30 @@ void git_config_push_parameter(const char *text)
 	strbuf_release(&env);
 }
 
+void git_config_push_env(const char *spec)
+{
+	struct strbuf buf = STRBUF_INIT;
+	const char *env_name;
+	const char *env_value;
+
+	env_name = strrchr(spec, '=');
+	if (!env_name)
+		die(_("invalid config format: %s"), spec);
+	env_name++;
+	if (!*env_name)
+		die(_("missing value for --config-env"));
+
+	env_value = getenv(env_name);
+	if (!env_value)
+		die(_("missing environment variable '%s' for configuration '%.*s'"),
+		    env_name, (int)(env_name - spec - 1), spec);
+
+	strbuf_add(&buf, spec, env_name - spec);
+	strbuf_addstr(&buf, env_value);
+	git_config_push_parameter(buf.buf);
+	strbuf_release(&buf);
+}
+
 static inline int iskeychar(int c)
 {
 	return isalnum(c) || c == '-';
diff --git a/config.h b/config.h
index c1449bb790..19a9adbaa9 100644
--- a/config.h
+++ b/config.h
@@ -138,6 +138,7 @@ int git_config_from_mem(config_fn_t fn,
 int git_config_from_blob_oid(config_fn_t fn, const char *name,
 			     const struct object_id *oid, void *data);
 void git_config_push_parameter(const char *text);
+void git_config_push_env(const char *spec);
 int git_config_from_parameters(config_fn_t fn, void *data);
 void read_early_config(config_fn_t cb, void *data);
 void read_very_early_config(config_fn_t cb, void *data);
diff --git a/git.c b/git.c
index 5a8ff12f87..b5f63d346b 100644
--- a/git.c
+++ b/git.c
@@ -29,7 +29,7 @@ const char git_usage_string[] =
 	   "           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]\n"
 	   "           [-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]\n"
 	   "           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]\n"
-	   "           [--super-prefix=<path>]\n"
+	   "           [--super-prefix=<path>] [--config-env=<name>=<envvar>]\n"
 	   "           <command> [<args>]");
 
 const char git_more_info_string[] =
@@ -255,6 +255,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			git_config_push_parameter((*argv)[1]);
 			(*argv)++;
 			(*argc)--;
+		} else if (skip_prefix(cmd, "--config-env=", &cmd)) {
+			git_config_push_env(cmd);
 		} else if (!strcmp(cmd, "--literal-pathspecs")) {
 			setenv(GIT_LITERAL_PATHSPECS_ENVIRONMENT, "1", 1);
 			if (envchanged)
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 97a04c6cc2..1e23eb8213 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1316,6 +1316,53 @@ test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
 		git config --get-regexp "env.*"
 '
 
+test_expect_success 'git --config-env=key=envvar support' '
+	cat >expect <<-\EOF &&
+	value
+	value
+	false
+	EOF
+	{
+		env ENVVAR=value git --config-env=core.name=ENVVAR config core.name &&
+		env ENVVAR=value git --config-env=foo.CamelCase=ENVVAR config foo.camelcase &&
+		env ENVVAR= git --config-env=foo.flag=ENVVAR config --bool foo.flag
+	} >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git --config-env fails with invalid parameters' '
+	test_must_fail git --config-env=foo.flag config --bool foo.flag 2>error &&
+	test_i18ngrep "invalid config format" error &&
+	test_must_fail git --config-env=foo.flag= config --bool foo.flag 2>error &&
+	test_i18ngrep "missing value for --config-env" error &&
+	test_must_fail git --config-env=foo.flag=NONEXISTENT config --bool foo.flag 2>error &&
+	test_i18ngrep "missing environment variable ${SQ}NONEXISTENT${SQ} for configuration ${SQ}foo.flag${SQ}" error
+'
+
+test_expect_success 'git -c and --config-env work together' '
+	cat >expect <<-\EOF &&
+	bar.cmd cmd-value
+	bar.env env-value
+	EOF
+	env ENVVAR=env-value git \
+		-c bar.cmd=cmd-value \
+		--config-env=bar.env=ENVVAR \
+		config --get-regexp "^bar.*" >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git -c and --config-env override each other' '
+	cat >expect <<-\EOF &&
+	env
+	cmd
+	EOF
+	{
+		env ENVVAR=env git -c bar.bar=cmd --config-env=bar.bar=ENVVAR config bar.bar &&
+		env ENVVAR=env git --config-env=bar.bar=ENVVAR -c bar.bar=cmd config bar.bar
+	} >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v7 3/8] quote: make sq_dequote_step() a public function
  2021-01-11  8:36 ` [PATCH v7 0/8] " Patrick Steinhardt
  2021-01-11  8:36   ` [PATCH v7 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
  2021-01-11  8:36   ` [PATCH v7 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
@ 2021-01-11  8:36   ` Patrick Steinhardt
  2021-01-11  8:36   ` [PATCH v7 4/8] config: extract function to parse config pairs Patrick Steinhardt
                     ` (4 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-11  8:36 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 3152 bytes --]

From: Jeff King <peff@peff.net>

We provide a function for dequoting an entire string, as well as one for
handling a space-separated list of quoted strings. But there's no way
for a caller to parse a string like 'foo'='bar', even though it is easy
to generate one using sq_quote_buf() or similar.

Let's make the single-step function available to callers outside of
quote.c. Note that we do need to adjust its implementation slightly: it
insists on seeing whitespace between items, and we'd like to be more
flexible than that. Since it only has a single caller, we can move that
check (and slurping up any extra whitespace) into that caller.

Signed-off-by: Jeff King <peff@peff.net>
---
 quote.c | 15 ++++++++++-----
 quote.h | 18 ++++++++++++++++--
 2 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/quote.c b/quote.c
index 69f4ca45da..8a3a5e39eb 100644
--- a/quote.c
+++ b/quote.c
@@ -116,7 +116,7 @@ void sq_append_quote_argv_pretty(struct strbuf *dst, const char **argv)
 	}
 }
 
-static char *sq_dequote_step(char *arg, char **next)
+char *sq_dequote_step(char *arg, char **next)
 {
 	char *dst = arg;
 	char *src = arg;
@@ -153,11 +153,8 @@ static char *sq_dequote_step(char *arg, char **next)
 			}
 		/* Fallthrough */
 		default:
-			if (!next || !isspace(*src))
+			if (!next)
 				return NULL;
-			do {
-				c = *++src;
-			} while (isspace(c));
 			*dst = 0;
 			*next = src;
 			return arg;
@@ -182,6 +179,14 @@ static int sq_dequote_to_argv_internal(char *arg,
 		char *dequoted = sq_dequote_step(next, &next);
 		if (!dequoted)
 			return -1;
+		if (next) {
+			char c;
+			if (!isspace(*next))
+				return -1;
+			do {
+				c = *++next;
+			} while (isspace(c));
+		}
 		if (argv) {
 			ALLOC_GROW(*argv, *nr + 1, *alloc);
 			(*argv)[(*nr)++] = dequoted;
diff --git a/quote.h b/quote.h
index 4b72a583cf..768cc6338e 100644
--- a/quote.h
+++ b/quote.h
@@ -42,12 +42,26 @@ void sq_quote_buf_pretty(struct strbuf *, const char *src);
 void sq_quote_argv_pretty(struct strbuf *, const char **argv);
 void sq_append_quote_argv_pretty(struct strbuf *dst, const char **argv);
 
-/* This unwraps what sq_quote() produces in place, but returns
+/*
+ * This unwraps what sq_quote() produces in place, but returns
  * NULL if the input does not look like what sq_quote would have
- * produced.
+ * produced (the full string must be a single quoted item).
  */
 char *sq_dequote(char *);
 
+/*
+ * Like sq_dequote(), but dequote a single item, and leave "next" pointing to
+ * the next character. E.g., in the string:
+ *
+ *   'one' 'two' 'three'
+ *
+ * after the first call, the return value would be the unquoted string "one",
+ * with "next" pointing to the space between "one" and "two"). The caller is
+ * responsible for advancing the pointer to the start of the next item before
+ * calling sq_dequote_step() again.
+ */
+char *sq_dequote_step(char *src, char **next);
+
 /*
  * Same as the above, but can be used to unwrap many arguments in the
  * same string separated by space. Like sq_quote, it works in place,
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v7 4/8] config: extract function to parse config pairs
  2021-01-11  8:36 ` [PATCH v7 0/8] " Patrick Steinhardt
                     ` (2 preceding siblings ...)
  2021-01-11  8:36   ` [PATCH v7 3/8] quote: make sq_dequote_step() a public function Patrick Steinhardt
@ 2021-01-11  8:36   ` Patrick Steinhardt
  2021-01-11  8:37   ` [PATCH v7 5/8] config: store "git -c" variables using more robust format Patrick Steinhardt
                     ` (3 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-11  8:36 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1833 bytes --]

The function `git_config_parse_parameter` is responsible for parsing a
`foo.bar=baz`-formatted configuration key, sanitizing the key and then
processing it via the given callback function. Given that we're about to
add a second user which is going to process keys which already has keys
and values separated, this commit extracts a function
`config_parse_pair` which only does the sanitization and processing
part as a preparatory step.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 config.c | 24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/config.c b/config.c
index 6484d13c46..aeffe4f4bc 100644
--- a/config.c
+++ b/config.c
@@ -461,11 +461,26 @@ int git_config_key_is_valid(const char *key)
 	return !git_config_parse_key_1(key, NULL, NULL, 1);
 }
 
+static int config_parse_pair(const char *key, const char *value,
+			  config_fn_t fn, void *data)
+{
+	char *canonical_name;
+	int ret;
+
+	if (!strlen(key))
+		return error(_("empty config key"));
+	if (git_config_parse_key(key, &canonical_name, NULL))
+		return -1;
+
+	ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
+	free(canonical_name);
+	return ret;
+}
+
 int git_config_parse_parameter(const char *text,
 			       config_fn_t fn, void *data)
 {
 	const char *value;
-	char *canonical_name;
 	struct strbuf **pair;
 	int ret;
 
@@ -486,12 +501,7 @@ int git_config_parse_parameter(const char *text,
 		return error(_("bogus config parameter: %s"), text);
 	}
 
-	if (git_config_parse_key(pair[0]->buf, &canonical_name, NULL)) {
-		ret = -1;
-	} else {
-		ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
-		free(canonical_name);
-	}
+	ret = config_parse_pair(pair[0]->buf, value, fn, data);
 	strbuf_list_free(pair);
 	return ret;
 }
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v7 5/8] config: store "git -c" variables using more robust format
  2021-01-11  8:36 ` [PATCH v7 0/8] " Patrick Steinhardt
                     ` (3 preceding siblings ...)
  2021-01-11  8:36   ` [PATCH v7 4/8] config: extract function to parse config pairs Patrick Steinhardt
@ 2021-01-11  8:37   ` Patrick Steinhardt
  2021-01-11  8:37   ` [PATCH v7 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS Patrick Steinhardt
                     ` (2 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-11  8:37 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 4287 bytes --]

The previous commit added a new format for $GIT_CONFIG_PARAMETERS which
is able to robustly handle subsections with "=" in them. Let's start
writing the new format. Unfortunately, this does much less than you'd
hope, because "git -c" itself has the same ambiguity problem! But it's
still worth doing:

  - we've now pushed the problem from the inter-process communication
    into the "-c" command-line parser. This would free us up to later
    add an unambiguous format there (e.g., separate arguments like "git
    --config key value", etc).

  - for --config-env, the parser already disallows "=" in the
    environment variable name. So:

      git --config-env section.with=equals.key=ENVVAR

    will robustly set section.with=equals.key to the contents of
    $ENVVAR.

The new test shows the improvement for --config-env.

Signed-off-by: Jeff King <peff@peff.net>
---
 config.c          | 52 ++++++++++++++++++++++++++++++++++++++++-------
 t/t1300-config.sh |  8 ++++++++
 2 files changed, 53 insertions(+), 7 deletions(-)

diff --git a/config.c b/config.c
index aeffe4f4bc..b5475c28aa 100644
--- a/config.c
+++ b/config.c
@@ -332,7 +332,7 @@ int git_config_include(const char *var, const char *value, void *data)
 	return ret;
 }
 
-void git_config_push_parameter(const char *text)
+static void git_config_push_split_parameter(const char *key, const char *value)
 {
 	struct strbuf env = STRBUF_INIT;
 	const char *old = getenv(CONFIG_DATA_ENVIRONMENT);
@@ -340,20 +340,60 @@ void git_config_push_parameter(const char *text)
 		strbuf_addstr(&env, old);
 		strbuf_addch(&env, ' ');
 	}
-	sq_quote_buf(&env, text);
+	sq_quote_buf(&env, key);
+	strbuf_addch(&env, '=');
+	if (value)
+		sq_quote_buf(&env, value);
 	setenv(CONFIG_DATA_ENVIRONMENT, env.buf, 1);
 	strbuf_release(&env);
 }
 
+void git_config_push_parameter(const char *text)
+{
+	const char *value;
+
+	/*
+	 * When we see:
+	 *
+	 *   section.subsection=with=equals.key=value
+	 *
+	 * we cannot tell if it means:
+	 *
+	 *   [section "subsection=with=equals"]
+	 *   key = value
+	 *
+	 * or:
+	 *
+	 *   [section]
+	 *   subsection = with=equals.key=value
+	 *
+	 * We parse left-to-right for the first "=", meaning we'll prefer to
+	 * keep the value intact over the subsection. This is historical, but
+	 * also sensible since values are more likely to contain odd or
+	 * untrusted input than a section name.
+	 *
+	 * A missing equals is explicitly allowed (as a bool-only entry).
+	 */
+	value = strchr(text, '=');
+	if (value) {
+		char *key = xmemdupz(text, value - text);
+		git_config_push_split_parameter(key, value + 1);
+		free(key);
+	} else {
+		git_config_push_split_parameter(text, NULL);
+	}
+}
+
 void git_config_push_env(const char *spec)
 {
-	struct strbuf buf = STRBUF_INIT;
+	char *key;
 	const char *env_name;
 	const char *env_value;
 
 	env_name = strrchr(spec, '=');
 	if (!env_name)
 		die(_("invalid config format: %s"), spec);
+	key = xmemdupz(spec, env_name - spec);
 	env_name++;
 	if (!*env_name)
 		die(_("missing value for --config-env"));
@@ -363,10 +403,8 @@ void git_config_push_env(const char *spec)
 		die(_("missing environment variable '%s' for configuration '%.*s'"),
 		    env_name, (int)(env_name - spec - 1), spec);
 
-	strbuf_add(&buf, spec, env_name - spec);
-	strbuf_addstr(&buf, env_value);
-	git_config_push_parameter(buf.buf);
-	strbuf_release(&buf);
+	git_config_push_split_parameter(key, env_value);
+	free(key);
 }
 
 static inline int iskeychar(int c)
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 1e23eb8213..24919d5445 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1363,6 +1363,14 @@ test_expect_success 'git -c and --config-env override each other' '
 	test_cmp expect actual
 '
 
+test_expect_success '--config-env handles keys with equals' '
+	echo value=with=equals >expect &&
+	ENVVAR=value=with=equals git \
+		--config-env=section.subsection=with=equals.key=ENVVAR \
+		config section.subsection=with=equals.key >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v7 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS
  2021-01-11  8:36 ` [PATCH v7 0/8] " Patrick Steinhardt
                     ` (4 preceding siblings ...)
  2021-01-11  8:37   ` [PATCH v7 5/8] config: store "git -c" variables using more robust format Patrick Steinhardt
@ 2021-01-11  8:37   ` Patrick Steinhardt
  2021-01-11  8:37   ` [PATCH v7 7/8] environment: make `getenv_safe()` a public function Patrick Steinhardt
  2021-01-11  8:37   ` [PATCH v7 8/8] config: allow specifying config entries via envvar pairs Patrick Steinhardt
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-11  8:37 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 8135 bytes --]

From: Jeff King <peff@peff.net>

When we stuff config options into GIT_CONFIG_PARAMETERS, we shell-quote
each one as a single unit, like:

  'section.one=value1' 'section.two=value2'

On the reading side, we de-quote to get the individual strings, and then
parse them by splitting on the first "=" we find. This format is
ambiguous, because an "=" may appear in a subsection. So the config
represented in a file by both:

  [section "subsection=with=equals"]
  key = value

and:

  [section]
  subsection = with=equals.key=value

ends up in this flattened format like:

  'section.subsection=with=equals.key=value'

and we can't tell which was desired. We have traditionally resolved this
by taking the first "=" we see starting from the left, meaning that we
allowed arbitrary content in the value, but not in the subsection.

Let's make our environment format a bit more robust by separately
quoting the key and value. That turns those examples into:

  'section.subsection=with=equals.key'='value'

and:

  'section.subsection'='with=equals.key=value'

respectively, and we can tell the difference between them. We can detect
which format is in use for any given element of the list based on the
presence of the unquoted "=". That means we can continue to allow the
old format to work to support any callers which manually used the old
format, and we can even intermingle the two formats. The old format
wasn't documented, and nobody was supposed to be using it. But it's
likely that such callers exist in the wild, so it's nice if we can avoid
breaking them. Likewise, it may be possible to trigger an older version
of "git -c" that runs a script that calls into a newer version of "git
-c"; that new version would see the intermingled format.

This does create one complication, which is that the obvious format in
the new scheme for

  [section]
  some-bool

is:

  'section.some-bool'

with no equals. We'd mistake that for an old-style variable. And it even
has the same meaning in the old style, but:

  [section "with=equals"]
  some-bool

does not. It would be:

  'section.with=equals=some-bool'

which we'd take to mean:

  [section]
  with = equals=some-bool

in the old, ambiguous style. Likewise, we can't use:

  'section.some-bool'=''

because that's ambiguous with an actual empty string. Instead, we'll
again use the shell-quoting to give us a hint, and use:

  'section.some-bool'=

to show that we have no value.

Note that this commit just expands the reading side. We'll start writing
the new format via "git -c" in a future patch. In the meantime, the
existing "git -c" tests will make sure we didn't break reading the old
format. But we'll also add some explicit coverage of the two formats to
make sure we continue to handle the old one after we move the writing
side over.

And one final note: since we're now using the shell-quoting as a
semantically meaningful hint, this closes the door to us ever allowing
arbitrary shell quoting, like:

  'a'shell'would'be'ok'with'this'.key=value

But we have never supported that (only what sq_quote() would produce),
and we are probably better off keeping things simple, robust, and
backwards-compatible, than trying to make it easier for humans. We'll
continue not to advertise the format of the variable to users, and
instead keep "git -c" as the recommended mechanism for setting config
(even if we are trying to be kind not to break users who may be relying
on the current undocumented format).

Signed-off-by: Jeff King <peff@peff.net>
---
 config.c          | 69 +++++++++++++++++++++++++++++++++++------------
 t/t1300-config.sh | 52 +++++++++++++++++++++++++++++++++++
 2 files changed, 104 insertions(+), 17 deletions(-)

diff --git a/config.c b/config.c
index b5475c28aa..33099a3b0d 100644
--- a/config.c
+++ b/config.c
@@ -544,14 +544,62 @@ int git_config_parse_parameter(const char *text,
 	return ret;
 }
 
+static int parse_config_env_list(char *env, config_fn_t fn, void *data)
+{
+	char *cur = env;
+	while (cur && *cur) {
+		const char *key = sq_dequote_step(cur, &cur);
+		if (!key)
+			return error(_("bogus format in %s"),
+				     CONFIG_DATA_ENVIRONMENT);
+
+		if (!cur || isspace(*cur)) {
+			/* old-style 'key=value' */
+			if (git_config_parse_parameter(key, fn, data) < 0)
+				return -1;
+		}
+		else if (*cur == '=') {
+			/* new-style 'key'='value' */
+			const char *value;
+
+			cur++;
+			if (*cur == '\'') {
+				/* quoted value */
+				value = sq_dequote_step(cur, &cur);
+				if (!value || (cur && !isspace(*cur))) {
+					return error(_("bogus format in %s"),
+						     CONFIG_DATA_ENVIRONMENT);
+				}
+			} else if (!*cur || isspace(*cur)) {
+				/* implicit bool: 'key'= */
+				value = NULL;
+			} else {
+				return error(_("bogus format in %s"),
+					     CONFIG_DATA_ENVIRONMENT);
+			}
+
+			if (config_parse_pair(key, value, fn, data) < 0)
+				return -1;
+		}
+		else {
+			/* unknown format */
+			return error(_("bogus format in %s"),
+				     CONFIG_DATA_ENVIRONMENT);
+		}
+
+		if (cur) {
+			while (isspace(*cur))
+				cur++;
+		}
+	}
+	return 0;
+}
+
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
 	const char *env = getenv(CONFIG_DATA_ENVIRONMENT);
 	int ret = 0;
 	char *envw;
-	const char **argv = NULL;
-	int nr = 0, alloc = 0;
-	int i;
 	struct config_source source;
 
 	if (!env)
@@ -564,21 +612,8 @@ int git_config_from_parameters(config_fn_t fn, void *data)
 
 	/* sq_dequote will write over it */
 	envw = xstrdup(env);
+	ret = parse_config_env_list(envw, fn, data);
 
-	if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
-		ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
-		goto out;
-	}
-
-	for (i = 0; i < nr; i++) {
-		if (git_config_parse_parameter(argv[i], fn, data) < 0) {
-			ret = -1;
-			goto out;
-		}
-	}
-
-out:
-	free(argv);
 	free(envw);
 	cf = source.prev;
 	return ret;
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 24919d5445..0063e9f059 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1294,6 +1294,58 @@ test_expect_success 'git -c is not confused by empty environment' '
 	GIT_CONFIG_PARAMETERS="" git -c x.one=1 config --list
 '
 
+test_expect_success 'GIT_CONFIG_PARAMETERS handles old-style entries' '
+	v="${SQ}key.one=foo${SQ}" &&
+	v="$v  ${SQ}key.two=bar${SQ}" &&
+	v="$v ${SQ}key.ambiguous=section.whatever=value${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.one foo
+	key.two bar
+	key.ambiguous section.whatever=value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'GIT_CONFIG_PARAMETERS handles new-style entries' '
+	v="${SQ}key.one${SQ}=${SQ}foo${SQ}" &&
+	v="$v  ${SQ}key.two${SQ}=${SQ}bar${SQ}" &&
+	v="$v ${SQ}key.ambiguous=section.whatever${SQ}=${SQ}value${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.one foo
+	key.two bar
+	key.ambiguous=section.whatever value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'old and new-style entries can mix' '
+	v="${SQ}key.oldone=oldfoo${SQ}" &&
+	v="$v ${SQ}key.newone${SQ}=${SQ}newfoo${SQ}" &&
+	v="$v ${SQ}key.oldtwo=oldbar${SQ}" &&
+	v="$v ${SQ}key.newtwo${SQ}=${SQ}newbar${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.oldone oldfoo
+	key.newone newfoo
+	key.oldtwo oldbar
+	key.newtwo newbar
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'old and new bools with ambiguous subsection' '
+	v="${SQ}key.with=equals.oldbool${SQ}" &&
+	v="$v ${SQ}key.with=equals.newbool${SQ}=" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.with equals.oldbool
+	key.with=equals.newbool
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
 	cat >expect <<-\EOF &&
 	env.one one
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v7 7/8] environment: make `getenv_safe()` a public function
  2021-01-11  8:36 ` [PATCH v7 0/8] " Patrick Steinhardt
                     ` (5 preceding siblings ...)
  2021-01-11  8:37   ` [PATCH v7 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS Patrick Steinhardt
@ 2021-01-11  8:37   ` Patrick Steinhardt
  2021-01-11  8:37   ` [PATCH v7 8/8] config: allow specifying config entries via envvar pairs Patrick Steinhardt
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-11  8:37 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1661 bytes --]

The `getenv_safe()` helper function helps to safely retrieve multiple
environment values without the need to depend on platform-specific
behaviour for the return value's lifetime. We'll make use of this
function in a following patch, so let's make it available by making it
non-static and adding a declaration.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 environment.c |  7 ++-----
 environment.h | 12 ++++++++++++
 2 files changed, 14 insertions(+), 5 deletions(-)
 create mode 100644 environment.h

diff --git a/environment.c b/environment.c
index bb518c61cd..2234af462c 100644
--- a/environment.c
+++ b/environment.c
@@ -9,6 +9,7 @@
  */
 #include "cache.h"
 #include "branch.h"
+#include "environment.h"
 #include "repository.h"
 #include "config.h"
 #include "refs.h"
@@ -152,11 +153,7 @@ static char *expand_namespace(const char *raw_namespace)
 	return strbuf_detach(&buf, NULL);
 }
 
-/*
- * Wrapper of getenv() that returns a strdup value. This value is kept
- * in argv to be freed later.
- */
-static const char *getenv_safe(struct strvec *argv, const char *name)
+const char *getenv_safe(struct strvec *argv, const char *name)
 {
 	const char *value = getenv(name);
 
diff --git a/environment.h b/environment.h
new file mode 100644
index 0000000000..d438b5c8f3
--- /dev/null
+++ b/environment.h
@@ -0,0 +1,12 @@
+#ifndef ENVIRONMENT_H
+#define ENVIRONMENT_H
+
+#include "strvec.h"
+
+/*
+ * Wrapper of getenv() that returns a strdup value. This value is kept
+ * in argv to be freed later.
+ */
+const char *getenv_safe(struct strvec *argv, const char *name);
+
+#endif
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v7 8/8] config: allow specifying config entries via envvar pairs
  2021-01-11  8:36 ` [PATCH v7 0/8] " Patrick Steinhardt
                     ` (6 preceding siblings ...)
  2021-01-11  8:37   ` [PATCH v7 7/8] environment: make `getenv_safe()` a public function Patrick Steinhardt
@ 2021-01-11  8:37   ` Patrick Steinhardt
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-11  8:37 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 10490 bytes --]

While we currently have the `GIT_CONFIG_PARAMETERS` environment variable
which can be used to pass runtime configuration data to git processes,
it's an internal implementation detail and not supposed to be used by
end users.

Next to being for internal use only, this way of passing config entries
has a major downside: the config keys need to be parsed as they contain
both key and value in a single variable. As such, it is left to the user
to escape any potentially harmful characters in the value, which is
quite hard to do if values are controlled by a third party.

This commit thus adds a new way of adding config entries via the
environment which gets rid of this shortcoming. If the user passes the
`GIT_CONFIG_COUNT=$n` environment variable, Git will parse environment
variable pairs `GIT_CONFIG_KEY_$i` and `GIT_CONFIG_VALUE_$i` for each
`i` in `[0,n)`.

While the same can be achieved with `git -c <name>=<value>`, one may
wish to not do so for potentially sensitive information. E.g. if one
wants to set `http.extraHeader` to contain an authentication token,
doing so via `-c` would trivially leak those credentials via e.g. ps(1),
which typically also shows command arguments.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git-config.txt |  16 +++++
 cache.h                      |   1 +
 config.c                     |  67 +++++++++++++++++---
 environment.c                |   1 +
 t/t1300-config.sh            | 115 ++++++++++++++++++++++++++++++++++-
 5 files changed, 191 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt
index 0e9351d3cb..4b4cc5c5e8 100644
--- a/Documentation/git-config.txt
+++ b/Documentation/git-config.txt
@@ -346,6 +346,22 @@ GIT_CONFIG_NOSYSTEM::
 
 See also <<FILES>>.
 
+GIT_CONFIG_COUNT::
+GIT_CONFIG_KEY_<n>::
+GIT_CONFIG_VALUE_<n>::
+	If GIT_CONFIG_COUNT is set to a positive number, all environment pairs
+	GIT_CONFIG_KEY_<n> and GIT_CONFIG_VALUE_<n> up to that number will be
+	added to the process's runtime configuration. The config pairs are
+	zero-indexed. Any missing key or value is treated as an error. An empty
+	GIT_CONFIG_COUNT is treated the same as GIT_CONFIG_COUNT=0, namely no
+	pairs are processed. These environment variables will override values
+	in configuration files, but will be overridden by any explicit options
+	passed via `git -c`.
++
+This is useful for cases where you want to spawn multiple git commands
+with a common configuration but cannot depend on a configuration file,
+for example when writing scripts.
+
 
 [[EXAMPLES]]
 EXAMPLES
diff --git a/cache.h b/cache.h
index 7109765748..a2e318c62b 100644
--- a/cache.h
+++ b/cache.h
@@ -472,6 +472,7 @@ static inline enum object_type object_type(unsigned int mode)
 #define TEMPLATE_DIR_ENVIRONMENT "GIT_TEMPLATE_DIR"
 #define CONFIG_ENVIRONMENT "GIT_CONFIG"
 #define CONFIG_DATA_ENVIRONMENT "GIT_CONFIG_PARAMETERS"
+#define CONFIG_COUNT_ENVIRONMENT "GIT_CONFIG_COUNT"
 #define EXEC_PATH_ENVIRONMENT "GIT_EXEC_PATH"
 #define CEILING_DIRECTORIES_ENVIRONMENT "GIT_CEILING_DIRECTORIES"
 #define NO_REPLACE_OBJECTS_ENVIRONMENT "GIT_NO_REPLACE_OBJECTS"
diff --git a/config.c b/config.c
index 33099a3b0d..2627a05e91 100644
--- a/config.c
+++ b/config.c
@@ -8,6 +8,7 @@
 #include "cache.h"
 #include "branch.h"
 #include "config.h"
+#include "environment.h"
 #include "repository.h"
 #include "lockfile.h"
 #include "exec-cmd.h"
@@ -597,23 +598,73 @@ static int parse_config_env_list(char *env, config_fn_t fn, void *data)
 
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
-	const char *env = getenv(CONFIG_DATA_ENVIRONMENT);
+	const char *env;
+	struct strbuf envvar = STRBUF_INIT;
+	struct strvec to_free = STRVEC_INIT;
 	int ret = 0;
-	char *envw;
+	char *envw = NULL;
 	struct config_source source;
 
-	if (!env)
-		return 0;
-
 	memset(&source, 0, sizeof(source));
 	source.prev = cf;
 	source.origin_type = CONFIG_ORIGIN_CMDLINE;
 	cf = &source;
 
-	/* sq_dequote will write over it */
-	envw = xstrdup(env);
-	ret = parse_config_env_list(envw, fn, data);
+	env = getenv(CONFIG_COUNT_ENVIRONMENT);
+	if (env) {
+		unsigned long count;
+		char *endp;
+		int i;
 
+		count = strtoul(env, &endp, 10);
+		if (*endp) {
+			ret = error(_("bogus count in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+		if (count > INT_MAX) {
+			ret = error(_("too many entries in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+
+		for (i = 0; i < count; i++) {
+			const char *key, *value;
+
+			strbuf_addf(&envvar, "GIT_CONFIG_KEY_%d", i);
+			key = getenv_safe(&to_free, envvar.buf);
+			if (!key) {
+				ret = error(_("missing config key %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			strbuf_addf(&envvar, "GIT_CONFIG_VALUE_%d", i);
+			value = getenv_safe(&to_free, envvar.buf);
+			if (!value) {
+				ret = error(_("missing config value %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			if (config_parse_pair(key, value, fn, data) < 0) {
+				ret = -1;
+				goto out;
+			}
+		}
+	}
+
+	env = getenv(CONFIG_DATA_ENVIRONMENT);
+	if (env) {
+		/* sq_dequote will write over it */
+		envw = xstrdup(env);
+		if (parse_config_env_list(envw, fn, data) < 0) {
+			ret = -1;
+			goto out;
+		}
+	}
+
+out:
+	strbuf_release(&envvar);
+	strvec_clear(&to_free);
 	free(envw);
 	cf = source.prev;
 	return ret;
diff --git a/environment.c b/environment.c
index 2234af462c..2f27008424 100644
--- a/environment.c
+++ b/environment.c
@@ -117,6 +117,7 @@ const char * const local_repo_env[] = {
 	ALTERNATE_DB_ENVIRONMENT,
 	CONFIG_ENVIRONMENT,
 	CONFIG_DATA_ENVIRONMENT,
+	CONFIG_COUNT_ENVIRONMENT,
 	DB_ENVIRONMENT,
 	GIT_DIR_ENVIRONMENT,
 	GIT_WORK_TREE_ENVIRONMENT,
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 0063e9f059..6ecf2c11a7 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1423,6 +1423,117 @@ test_expect_success '--config-env handles keys with equals' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git config handles environment config pairs' '
+	GIT_CONFIG_COUNT=2 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="foo" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="bar" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one foo
+	pair.two bar
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs without count' '
+	test_must_fail env GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one 2>error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one
+'
+
+test_expect_success 'git config ignores pairs exceeding count' '
+	GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="value" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with empty count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT= GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config fails with invalid count' '
+	test_must_fail env GIT_CONFIG_COUNT=10a git config --list 2>error &&
+	test_i18ngrep "bogus count" error &&
+	test_must_fail env GIT_CONFIG_COUNT=9999999999999999 git config --list 2>error &&
+	test_i18ngrep "too many entries" error
+'
+
+test_expect_success 'git config fails with missing config key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_VALUE_0="value" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config key" error
+'
+
+test_expect_success 'git config fails with missing config value' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0="pair.one" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config value" error
+'
+
+test_expect_success 'git config fails with invalid config pair key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0= GIT_CONFIG_VALUE_0=value \
+		git config --list &&
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0=missing-section GIT_CONFIG_VALUE_0=value \
+		git config --list
+'
+
+test_expect_success 'environment overrides config file' '
+	test_when_finished "rm -f .git/config" &&
+	cat >.git/config <<-EOF &&
+	[pair]
+	one = value
+	EOF
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=override \
+		git config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'GIT_CONFIG_PARAMETERS overrides environment config' '
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
+		GIT_CONFIG_PARAMETERS="${SQ}pair.one=override${SQ}" \
+		git config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'command line overrides environment config' '
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
+		git -c pair.one=override config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
@@ -1768,9 +1879,11 @@ test_expect_success '--show-origin with --list' '
 	file:.git/config	user.override=local
 	file:.git/config	include.path=../include/relative.include
 	file:.git/../include/relative.include	user.relative=include
+	command line:	user.environ=true
 	command line:	user.cmdline=true
 	EOF
-	git -c user.cmdline=true config --list --show-origin >output &&
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=user.environ GIT_CONFIG_VALUE_0=true\
+		git -c user.cmdline=true config --list --show-origin >output &&
 	test_cmp expect output
 '
 
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v7 2/8] config: add new way to pass config via `--config-env`
  2021-01-11  8:36   ` [PATCH v7 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
@ 2021-01-11 22:34     ` Junio C Hamano
  0 siblings, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2021-01-11 22:34 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Jeff King, brian m. carlson, Philip Oakley

Patrick Steinhardt <ps@pks.im> writes:

> +void git_config_push_env(const char *spec)
> +{
> +	struct strbuf buf = STRBUF_INIT;
> +	const char *env_name;
> +	const char *env_value;
> +
> +	env_name = strrchr(spec, '=');
> +	if (!env_name)
> +		die(_("invalid config format: %s"), spec);
> +	env_name++;
> +	if (!*env_name)
> +		die(_("missing value for --config-env"));

If reporting the name of the configuration variable, for which we
checked an environment variable, is worth doing in the !env_value
case below, shouldn't we be doing the same here, too?  I.e.

		die(_("missing environment variable name in %s", spec));;

or something to complain against "git --config-env foo="?

> +	env_value = getenv(env_name);
> +	if (!env_value)
> +		die(_("missing environment variable '%s' for configuration '%.*s'"),
> +		    env_name, (int)(env_name - spec - 1), spec);

> +test_expect_success 'git --config-env=key=envvar support' '
> +	cat >expect <<-\EOF &&
> +	value
> +	value
> +	false
> +	EOF
> +	{
> +		env ENVVAR=value git --config-env=core.name=ENVVAR config core.name &&
> +		env ENVVAR=value git --config-env=foo.CamelCase=ENVVAR config foo.camelcase &&
> +		env ENVVAR= git --config-env=foo.flag=ENVVAR config --bool foo.flag

These "env " prefixes are not wrong per-se but are unnecessary.  The
same for the rest of this patch.

> +	} >actual &&
> +	test_cmp expect actual
> +'
> +
> +test_expect_success 'git --config-env fails with invalid parameters' '
> +	test_must_fail git --config-env=foo.flag config --bool foo.flag 2>error &&
> +	test_i18ngrep "invalid config format" error &&
> +	test_must_fail git --config-env=foo.flag= config --bool foo.flag 2>error &&
> +	test_i18ngrep "missing value for --config-env" error &&
> +	test_must_fail git --config-env=foo.flag=NONEXISTENT config --bool foo.flag 2>error &&

How are we making sure 

	$ NONEXISTENT=True make test

is not what the end-user is running?

	sane_unset X &&
	test_must_fail git --config-env foo.flag=X config --bool foo.flag

or something along that line, perhaps?

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v8 0/8] config: allow specifying config entries via envvar pairs
  2020-11-24 10:50 [PATCH v2 0/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
                   ` (7 preceding siblings ...)
  2021-01-11  8:36 ` [PATCH v7 0/8] " Patrick Steinhardt
@ 2021-01-12 12:26 ` Patrick Steinhardt
  2021-01-12 12:26   ` [PATCH v8 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
                     ` (7 more replies)
  8 siblings, 8 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-12 12:26 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 5672 bytes --]

Hi,

this is the eighth version of my patch series which aims to implement a
way to pass config entries via the environment while avoiding any
requirements to perform shell quoting on the user's side.

Changes were all proposed by Junio and are solely in patch 2/8:

    - Improved the error message to include the the config key when the
      environment variable name is missing.

    - Dropped needless use of env(1) in added tests.

    - The NONEXISTENT envvar is being unset now to fix the case where
      tests may be run with that variable being set by the user.

Please see the attached range-diff for more details.

Patrick


Jeff King (2):
  quote: make sq_dequote_step() a public function
  config: parse more robust format in GIT_CONFIG_PARAMETERS

Patrick Steinhardt (6):
  git: add `--super-prefix` to usage string
  config: add new way to pass config via `--config-env`
  config: extract function to parse config pairs
  config: store "git -c" variables using more robust format
  environment: make `getenv_safe()` a public function
  config: allow specifying config entries via envvar pairs

 Documentation/git-config.txt |  16 +++
 Documentation/git.txt        |  24 +++-
 cache.h                      |   1 +
 config.c                     | 209 ++++++++++++++++++++++++++++----
 config.h                     |   1 +
 environment.c                |   8 +-
 environment.h                |  12 ++
 git.c                        |   3 +
 quote.c                      |  15 ++-
 quote.h                      |  18 ++-
 t/t1300-config.sh            | 223 ++++++++++++++++++++++++++++++++++-
 11 files changed, 491 insertions(+), 39 deletions(-)
 create mode 100644 environment.h

Range-diff against v7:
1:  55fa4d0d11 = 1:  55fa4d0d11 git: add `--super-prefix` to usage string
2:  b9cf47afe8 ! 2:  470396d36f config: add new way to pass config via `--config-env`
    @@ config.c: void git_config_push_parameter(const char *text)
     +		die(_("invalid config format: %s"), spec);
     +	env_name++;
     +	if (!*env_name)
    -+		die(_("missing value for --config-env"));
    ++		die(_("missing environment variable name for configuration '%.*s'"),
    ++		    (int)(env_name - spec - 1), spec);
     +
     +	env_value = getenv(env_name);
     +	if (!env_value)
    @@ t/t1300-config.sh: test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
     +	false
     +	EOF
     +	{
    -+		env ENVVAR=value git --config-env=core.name=ENVVAR config core.name &&
    -+		env ENVVAR=value git --config-env=foo.CamelCase=ENVVAR config foo.camelcase &&
    -+		env ENVVAR= git --config-env=foo.flag=ENVVAR config --bool foo.flag
    ++		ENVVAR=value git --config-env=core.name=ENVVAR config core.name &&
    ++		ENVVAR=value git --config-env=foo.CamelCase=ENVVAR config foo.camelcase &&
    ++		ENVVAR= git --config-env=foo.flag=ENVVAR config --bool foo.flag
     +	} >actual &&
     +	test_cmp expect actual
     +'
     +
     +test_expect_success 'git --config-env fails with invalid parameters' '
     +	test_must_fail git --config-env=foo.flag config --bool foo.flag 2>error &&
    -+	test_i18ngrep "invalid config format" error &&
    ++	test_i18ngrep "invalid config format: foo.flag" error &&
     +	test_must_fail git --config-env=foo.flag= config --bool foo.flag 2>error &&
    -+	test_i18ngrep "missing value for --config-env" error &&
    ++	test_i18ngrep "missing environment variable name for configuration ${SQ}foo.flag${SQ}" error &&
    ++	sane_unset NONEXISTENT &&
     +	test_must_fail git --config-env=foo.flag=NONEXISTENT config --bool foo.flag 2>error &&
     +	test_i18ngrep "missing environment variable ${SQ}NONEXISTENT${SQ} for configuration ${SQ}foo.flag${SQ}" error
     +'
    @@ t/t1300-config.sh: test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
     +	bar.cmd cmd-value
     +	bar.env env-value
     +	EOF
    -+	env ENVVAR=env-value git \
    ++	ENVVAR=env-value git \
     +		-c bar.cmd=cmd-value \
     +		--config-env=bar.env=ENVVAR \
     +		config --get-regexp "^bar.*" >actual &&
    @@ t/t1300-config.sh: test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
     +	cmd
     +	EOF
     +	{
    -+		env ENVVAR=env git -c bar.bar=cmd --config-env=bar.bar=ENVVAR config bar.bar &&
    -+		env ENVVAR=env git --config-env=bar.bar=ENVVAR -c bar.bar=cmd config bar.bar
    ++		ENVVAR=env git -c bar.bar=cmd --config-env=bar.bar=ENVVAR config bar.bar &&
    ++		ENVVAR=env git --config-env=bar.bar=ENVVAR -c bar.bar=cmd config bar.bar
     +	} >actual &&
     +	test_cmp expect actual
     +'
3:  1b47f0db98 = 3:  7a7a4ae234 quote: make sq_dequote_step() a public function
4:  b9565a050e = 4:  39552eb8b9 config: extract function to parse config pairs
5:  8f998ac81a ! 5:  36c2a51b13 config: store "git -c" variables using more robust format
    @@ config.c: void git_config_push_parameter(const char *text)
     +	key = xmemdupz(spec, env_name - spec);
      	env_name++;
      	if (!*env_name)
    - 		die(_("missing value for --config-env"));
    + 		die(_("missing environment variable name for configuration '%.*s'"),
     @@ config.c: void git_config_push_env(const char *spec)
      		die(_("missing environment variable '%s' for configuration '%.*s'"),
      		    env_name, (int)(env_name - spec - 1), spec);
6:  e7b073c9dc = 6:  d67a3c0f9f config: parse more robust format in GIT_CONFIG_PARAMETERS
7:  6c1800a18f = 7:  28cc229ade environment: make `getenv_safe()` a public function
8:  ac9e778704 = 8:  07697b0c21 config: allow specifying config entries via envvar pairs
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v8 1/8] git: add `--super-prefix` to usage string
  2021-01-12 12:26 ` [PATCH v8 0/8] " Patrick Steinhardt
@ 2021-01-12 12:26   ` Patrick Steinhardt
  2021-01-12 12:26   ` [PATCH v8 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
                     ` (6 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-12 12:26 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1057 bytes --]

When the `--super-prefix` option was implmented in 74866d7579 (git: make
super-prefix option, 2016-10-07), its existence was only documented in
the manpage but not in the command's own usage string. Given that the
commit message didn't mention that this was done intentionally and given
that it's documented in the manpage, this seems like an oversight.

Add it to the usage string to fix the inconsistency.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 git.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/git.c b/git.c
index a00a0a4d94..5a8ff12f87 100644
--- a/git.c
+++ b/git.c
@@ -29,6 +29,7 @@ const char git_usage_string[] =
 	   "           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]\n"
 	   "           [-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]\n"
 	   "           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]\n"
+	   "           [--super-prefix=<path>]\n"
 	   "           <command> [<args>]");
 
 const char git_more_info_string[] =
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v8 2/8] config: add new way to pass config via `--config-env`
  2021-01-12 12:26 ` [PATCH v8 0/8] " Patrick Steinhardt
  2021-01-12 12:26   ` [PATCH v8 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
@ 2021-01-12 12:26   ` Patrick Steinhardt
  2021-04-16 15:40     ` Ævar Arnfjörð Bjarmason
  2021-01-12 12:26   ` [PATCH v8 3/8] quote: make sq_dequote_step() a public function Patrick Steinhardt
                     ` (5 subsequent siblings)
  7 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-12 12:26 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 7666 bytes --]

While it's already possible to pass runtime configuration via `git -c
<key>=<value>`, it may be undesirable to use when the value contains
sensitive information. E.g. if one wants to set `http.extraHeader` to
contain an authentication token, doing so via `-c` would trivially leak
those credentials via e.g. ps(1), which typically also shows command
arguments.

To enable this usecase without leaking credentials, this commit
introduces a new switch `--config-env=<key>=<envvar>`. Instead of
directly passing a value for the given key, it instead allows the user
to specify the name of an environment variable. The value of that
variable will then be used as value of the key.

Co-authored-by: Jeff King <peff@peff.net>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git.txt | 24 +++++++++++++++++++++-
 config.c              | 25 ++++++++++++++++++++++
 config.h              |  1 +
 git.c                 |  4 +++-
 t/t1300-config.sh     | 48 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 100 insertions(+), 2 deletions(-)

diff --git a/Documentation/git.txt b/Documentation/git.txt
index a6d4ad0818..d36e6fd482 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -13,7 +13,7 @@ SYNOPSIS
     [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
     [-p|--paginate|-P|--no-pager] [--no-replace-objects] [--bare]
     [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
-    [--super-prefix=<path>]
+    [--super-prefix=<path>] [--config-env <name>=<envvar>]
     <command> [<args>]
 
 DESCRIPTION
@@ -80,6 +80,28 @@ config file). Including the equals but with an empty value (like `git -c
 foo.bar= ...`) sets `foo.bar` to the empty string which `git config
 --type=bool` will convert to `false`.
 
+--config-env=<name>=<envvar>::
+	Like `-c <name>=<value>`, give configuration variable
+	'<name>' a value, where <envvar> is the name of an
+	environment variable from which to retrieve the value. Unlike
+	`-c` there is no shortcut for directly setting the value to an
+	empty string, instead the environment variable itself must be
+	set to the empty string.  It is an error if the `<envvar>` does not exist
+	in the environment. `<envvar>` may not contain an equals sign
+	to avoid ambiguity with `<name>`s which contain one.
++
+This is useful for cases where you want to pass transitory
+configuration options to git, but are doing so on OS's where
+other processes might be able to read your cmdline
+(e.g. `/proc/self/cmdline`), but not your environ
+(e.g. `/proc/self/environ`). That behavior is the default on
+Linux, but may not be on your system.
++
+Note that this might add security for variables such as
+`http.extraHeader` where the sensitive information is part of
+the value, but not e.g. `url.<base>.insteadOf` where the
+sensitive information can be part of the key.
+
 --exec-path[=<path>]::
 	Path to wherever your core Git programs are installed.
 	This can also be controlled by setting the GIT_EXEC_PATH
diff --git a/config.c b/config.c
index 1137bd73af..fd8c0c4dfc 100644
--- a/config.c
+++ b/config.c
@@ -345,6 +345,31 @@ void git_config_push_parameter(const char *text)
 	strbuf_release(&env);
 }
 
+void git_config_push_env(const char *spec)
+{
+	struct strbuf buf = STRBUF_INIT;
+	const char *env_name;
+	const char *env_value;
+
+	env_name = strrchr(spec, '=');
+	if (!env_name)
+		die(_("invalid config format: %s"), spec);
+	env_name++;
+	if (!*env_name)
+		die(_("missing environment variable name for configuration '%.*s'"),
+		    (int)(env_name - spec - 1), spec);
+
+	env_value = getenv(env_name);
+	if (!env_value)
+		die(_("missing environment variable '%s' for configuration '%.*s'"),
+		    env_name, (int)(env_name - spec - 1), spec);
+
+	strbuf_add(&buf, spec, env_name - spec);
+	strbuf_addstr(&buf, env_value);
+	git_config_push_parameter(buf.buf);
+	strbuf_release(&buf);
+}
+
 static inline int iskeychar(int c)
 {
 	return isalnum(c) || c == '-';
diff --git a/config.h b/config.h
index c1449bb790..19a9adbaa9 100644
--- a/config.h
+++ b/config.h
@@ -138,6 +138,7 @@ int git_config_from_mem(config_fn_t fn,
 int git_config_from_blob_oid(config_fn_t fn, const char *name,
 			     const struct object_id *oid, void *data);
 void git_config_push_parameter(const char *text);
+void git_config_push_env(const char *spec);
 int git_config_from_parameters(config_fn_t fn, void *data);
 void read_early_config(config_fn_t cb, void *data);
 void read_very_early_config(config_fn_t cb, void *data);
diff --git a/git.c b/git.c
index 5a8ff12f87..b5f63d346b 100644
--- a/git.c
+++ b/git.c
@@ -29,7 +29,7 @@ const char git_usage_string[] =
 	   "           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]\n"
 	   "           [-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]\n"
 	   "           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]\n"
-	   "           [--super-prefix=<path>]\n"
+	   "           [--super-prefix=<path>] [--config-env=<name>=<envvar>]\n"
 	   "           <command> [<args>]");
 
 const char git_more_info_string[] =
@@ -255,6 +255,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			git_config_push_parameter((*argv)[1]);
 			(*argv)++;
 			(*argc)--;
+		} else if (skip_prefix(cmd, "--config-env=", &cmd)) {
+			git_config_push_env(cmd);
 		} else if (!strcmp(cmd, "--literal-pathspecs")) {
 			setenv(GIT_LITERAL_PATHSPECS_ENVIRONMENT, "1", 1);
 			if (envchanged)
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 97a04c6cc2..853f2509c5 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1316,6 +1316,54 @@ test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
 		git config --get-regexp "env.*"
 '
 
+test_expect_success 'git --config-env=key=envvar support' '
+	cat >expect <<-\EOF &&
+	value
+	value
+	false
+	EOF
+	{
+		ENVVAR=value git --config-env=core.name=ENVVAR config core.name &&
+		ENVVAR=value git --config-env=foo.CamelCase=ENVVAR config foo.camelcase &&
+		ENVVAR= git --config-env=foo.flag=ENVVAR config --bool foo.flag
+	} >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git --config-env fails with invalid parameters' '
+	test_must_fail git --config-env=foo.flag config --bool foo.flag 2>error &&
+	test_i18ngrep "invalid config format: foo.flag" error &&
+	test_must_fail git --config-env=foo.flag= config --bool foo.flag 2>error &&
+	test_i18ngrep "missing environment variable name for configuration ${SQ}foo.flag${SQ}" error &&
+	sane_unset NONEXISTENT &&
+	test_must_fail git --config-env=foo.flag=NONEXISTENT config --bool foo.flag 2>error &&
+	test_i18ngrep "missing environment variable ${SQ}NONEXISTENT${SQ} for configuration ${SQ}foo.flag${SQ}" error
+'
+
+test_expect_success 'git -c and --config-env work together' '
+	cat >expect <<-\EOF &&
+	bar.cmd cmd-value
+	bar.env env-value
+	EOF
+	ENVVAR=env-value git \
+		-c bar.cmd=cmd-value \
+		--config-env=bar.env=ENVVAR \
+		config --get-regexp "^bar.*" >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git -c and --config-env override each other' '
+	cat >expect <<-\EOF &&
+	env
+	cmd
+	EOF
+	{
+		ENVVAR=env git -c bar.bar=cmd --config-env=bar.bar=ENVVAR config bar.bar &&
+		ENVVAR=env git --config-env=bar.bar=ENVVAR -c bar.bar=cmd config bar.bar
+	} >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v8 3/8] quote: make sq_dequote_step() a public function
  2021-01-12 12:26 ` [PATCH v8 0/8] " Patrick Steinhardt
  2021-01-12 12:26   ` [PATCH v8 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
  2021-01-12 12:26   ` [PATCH v8 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
@ 2021-01-12 12:26   ` Patrick Steinhardt
  2021-01-12 12:26   ` [PATCH v8 4/8] config: extract function to parse config pairs Patrick Steinhardt
                     ` (4 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-12 12:26 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 3152 bytes --]

From: Jeff King <peff@peff.net>

We provide a function for dequoting an entire string, as well as one for
handling a space-separated list of quoted strings. But there's no way
for a caller to parse a string like 'foo'='bar', even though it is easy
to generate one using sq_quote_buf() or similar.

Let's make the single-step function available to callers outside of
quote.c. Note that we do need to adjust its implementation slightly: it
insists on seeing whitespace between items, and we'd like to be more
flexible than that. Since it only has a single caller, we can move that
check (and slurping up any extra whitespace) into that caller.

Signed-off-by: Jeff King <peff@peff.net>
---
 quote.c | 15 ++++++++++-----
 quote.h | 18 ++++++++++++++++--
 2 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/quote.c b/quote.c
index 69f4ca45da..8a3a5e39eb 100644
--- a/quote.c
+++ b/quote.c
@@ -116,7 +116,7 @@ void sq_append_quote_argv_pretty(struct strbuf *dst, const char **argv)
 	}
 }
 
-static char *sq_dequote_step(char *arg, char **next)
+char *sq_dequote_step(char *arg, char **next)
 {
 	char *dst = arg;
 	char *src = arg;
@@ -153,11 +153,8 @@ static char *sq_dequote_step(char *arg, char **next)
 			}
 		/* Fallthrough */
 		default:
-			if (!next || !isspace(*src))
+			if (!next)
 				return NULL;
-			do {
-				c = *++src;
-			} while (isspace(c));
 			*dst = 0;
 			*next = src;
 			return arg;
@@ -182,6 +179,14 @@ static int sq_dequote_to_argv_internal(char *arg,
 		char *dequoted = sq_dequote_step(next, &next);
 		if (!dequoted)
 			return -1;
+		if (next) {
+			char c;
+			if (!isspace(*next))
+				return -1;
+			do {
+				c = *++next;
+			} while (isspace(c));
+		}
 		if (argv) {
 			ALLOC_GROW(*argv, *nr + 1, *alloc);
 			(*argv)[(*nr)++] = dequoted;
diff --git a/quote.h b/quote.h
index 4b72a583cf..768cc6338e 100644
--- a/quote.h
+++ b/quote.h
@@ -42,12 +42,26 @@ void sq_quote_buf_pretty(struct strbuf *, const char *src);
 void sq_quote_argv_pretty(struct strbuf *, const char **argv);
 void sq_append_quote_argv_pretty(struct strbuf *dst, const char **argv);
 
-/* This unwraps what sq_quote() produces in place, but returns
+/*
+ * This unwraps what sq_quote() produces in place, but returns
  * NULL if the input does not look like what sq_quote would have
- * produced.
+ * produced (the full string must be a single quoted item).
  */
 char *sq_dequote(char *);
 
+/*
+ * Like sq_dequote(), but dequote a single item, and leave "next" pointing to
+ * the next character. E.g., in the string:
+ *
+ *   'one' 'two' 'three'
+ *
+ * after the first call, the return value would be the unquoted string "one",
+ * with "next" pointing to the space between "one" and "two"). The caller is
+ * responsible for advancing the pointer to the start of the next item before
+ * calling sq_dequote_step() again.
+ */
+char *sq_dequote_step(char *src, char **next);
+
 /*
  * Same as the above, but can be used to unwrap many arguments in the
  * same string separated by space. Like sq_quote, it works in place,
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v8 4/8] config: extract function to parse config pairs
  2021-01-12 12:26 ` [PATCH v8 0/8] " Patrick Steinhardt
                     ` (2 preceding siblings ...)
  2021-01-12 12:26   ` [PATCH v8 3/8] quote: make sq_dequote_step() a public function Patrick Steinhardt
@ 2021-01-12 12:26   ` Patrick Steinhardt
  2021-01-12 12:27   ` [PATCH v8 5/8] config: store "git -c" variables using more robust format Patrick Steinhardt
                     ` (3 subsequent siblings)
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-12 12:26 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1833 bytes --]

The function `git_config_parse_parameter` is responsible for parsing a
`foo.bar=baz`-formatted configuration key, sanitizing the key and then
processing it via the given callback function. Given that we're about to
add a second user which is going to process keys which already has keys
and values separated, this commit extracts a function
`config_parse_pair` which only does the sanitization and processing
part as a preparatory step.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 config.c | 24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/config.c b/config.c
index fd8c0c4dfc..b7a8129f6c 100644
--- a/config.c
+++ b/config.c
@@ -462,11 +462,26 @@ int git_config_key_is_valid(const char *key)
 	return !git_config_parse_key_1(key, NULL, NULL, 1);
 }
 
+static int config_parse_pair(const char *key, const char *value,
+			  config_fn_t fn, void *data)
+{
+	char *canonical_name;
+	int ret;
+
+	if (!strlen(key))
+		return error(_("empty config key"));
+	if (git_config_parse_key(key, &canonical_name, NULL))
+		return -1;
+
+	ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
+	free(canonical_name);
+	return ret;
+}
+
 int git_config_parse_parameter(const char *text,
 			       config_fn_t fn, void *data)
 {
 	const char *value;
-	char *canonical_name;
 	struct strbuf **pair;
 	int ret;
 
@@ -487,12 +502,7 @@ int git_config_parse_parameter(const char *text,
 		return error(_("bogus config parameter: %s"), text);
 	}
 
-	if (git_config_parse_key(pair[0]->buf, &canonical_name, NULL)) {
-		ret = -1;
-	} else {
-		ret = (fn(canonical_name, value, data) < 0) ? -1 : 0;
-		free(canonical_name);
-	}
+	ret = config_parse_pair(pair[0]->buf, value, fn, data);
 	strbuf_list_free(pair);
 	return ret;
 }
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v8 5/8] config: store "git -c" variables using more robust format
  2021-01-12 12:26 ` [PATCH v8 0/8] " Patrick Steinhardt
                     ` (3 preceding siblings ...)
  2021-01-12 12:26   ` [PATCH v8 4/8] config: extract function to parse config pairs Patrick Steinhardt
@ 2021-01-12 12:27   ` Patrick Steinhardt
  2021-01-15 19:16     ` Jeff King
  2021-01-12 12:27   ` [PATCH v8 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS Patrick Steinhardt
                     ` (2 subsequent siblings)
  7 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-12 12:27 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 4314 bytes --]

The previous commit added a new format for $GIT_CONFIG_PARAMETERS which
is able to robustly handle subsections with "=" in them. Let's start
writing the new format. Unfortunately, this does much less than you'd
hope, because "git -c" itself has the same ambiguity problem! But it's
still worth doing:

  - we've now pushed the problem from the inter-process communication
    into the "-c" command-line parser. This would free us up to later
    add an unambiguous format there (e.g., separate arguments like "git
    --config key value", etc).

  - for --config-env, the parser already disallows "=" in the
    environment variable name. So:

      git --config-env section.with=equals.key=ENVVAR

    will robustly set section.with=equals.key to the contents of
    $ENVVAR.

The new test shows the improvement for --config-env.

Signed-off-by: Jeff King <peff@peff.net>
---
 config.c          | 52 ++++++++++++++++++++++++++++++++++++++++-------
 t/t1300-config.sh |  8 ++++++++
 2 files changed, 53 insertions(+), 7 deletions(-)

diff --git a/config.c b/config.c
index b7a8129f6c..7f7da60574 100644
--- a/config.c
+++ b/config.c
@@ -332,7 +332,7 @@ int git_config_include(const char *var, const char *value, void *data)
 	return ret;
 }
 
-void git_config_push_parameter(const char *text)
+static void git_config_push_split_parameter(const char *key, const char *value)
 {
 	struct strbuf env = STRBUF_INIT;
 	const char *old = getenv(CONFIG_DATA_ENVIRONMENT);
@@ -340,20 +340,60 @@ void git_config_push_parameter(const char *text)
 		strbuf_addstr(&env, old);
 		strbuf_addch(&env, ' ');
 	}
-	sq_quote_buf(&env, text);
+	sq_quote_buf(&env, key);
+	strbuf_addch(&env, '=');
+	if (value)
+		sq_quote_buf(&env, value);
 	setenv(CONFIG_DATA_ENVIRONMENT, env.buf, 1);
 	strbuf_release(&env);
 }
 
+void git_config_push_parameter(const char *text)
+{
+	const char *value;
+
+	/*
+	 * When we see:
+	 *
+	 *   section.subsection=with=equals.key=value
+	 *
+	 * we cannot tell if it means:
+	 *
+	 *   [section "subsection=with=equals"]
+	 *   key = value
+	 *
+	 * or:
+	 *
+	 *   [section]
+	 *   subsection = with=equals.key=value
+	 *
+	 * We parse left-to-right for the first "=", meaning we'll prefer to
+	 * keep the value intact over the subsection. This is historical, but
+	 * also sensible since values are more likely to contain odd or
+	 * untrusted input than a section name.
+	 *
+	 * A missing equals is explicitly allowed (as a bool-only entry).
+	 */
+	value = strchr(text, '=');
+	if (value) {
+		char *key = xmemdupz(text, value - text);
+		git_config_push_split_parameter(key, value + 1);
+		free(key);
+	} else {
+		git_config_push_split_parameter(text, NULL);
+	}
+}
+
 void git_config_push_env(const char *spec)
 {
-	struct strbuf buf = STRBUF_INIT;
+	char *key;
 	const char *env_name;
 	const char *env_value;
 
 	env_name = strrchr(spec, '=');
 	if (!env_name)
 		die(_("invalid config format: %s"), spec);
+	key = xmemdupz(spec, env_name - spec);
 	env_name++;
 	if (!*env_name)
 		die(_("missing environment variable name for configuration '%.*s'"),
@@ -364,10 +404,8 @@ void git_config_push_env(const char *spec)
 		die(_("missing environment variable '%s' for configuration '%.*s'"),
 		    env_name, (int)(env_name - spec - 1), spec);
 
-	strbuf_add(&buf, spec, env_name - spec);
-	strbuf_addstr(&buf, env_value);
-	git_config_push_parameter(buf.buf);
-	strbuf_release(&buf);
+	git_config_push_split_parameter(key, env_value);
+	free(key);
 }
 
 static inline int iskeychar(int c)
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 853f2509c5..25437324c1 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1364,6 +1364,14 @@ test_expect_success 'git -c and --config-env override each other' '
 	test_cmp expect actual
 '
 
+test_expect_success '--config-env handles keys with equals' '
+	echo value=with=equals >expect &&
+	ENVVAR=value=with=equals git \
+		--config-env=section.subsection=with=equals.key=ENVVAR \
+		config section.subsection=with=equals.key >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v8 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS
  2021-01-12 12:26 ` [PATCH v8 0/8] " Patrick Steinhardt
                     ` (4 preceding siblings ...)
  2021-01-12 12:27   ` [PATCH v8 5/8] config: store "git -c" variables using more robust format Patrick Steinhardt
@ 2021-01-12 12:27   ` Patrick Steinhardt
  2021-01-12 12:27   ` [PATCH v8 7/8] environment: make `getenv_safe()` a public function Patrick Steinhardt
  2021-01-12 12:27   ` [PATCH v8 8/8] config: allow specifying config entries via envvar pairs Patrick Steinhardt
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-12 12:27 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 8135 bytes --]

From: Jeff King <peff@peff.net>

When we stuff config options into GIT_CONFIG_PARAMETERS, we shell-quote
each one as a single unit, like:

  'section.one=value1' 'section.two=value2'

On the reading side, we de-quote to get the individual strings, and then
parse them by splitting on the first "=" we find. This format is
ambiguous, because an "=" may appear in a subsection. So the config
represented in a file by both:

  [section "subsection=with=equals"]
  key = value

and:

  [section]
  subsection = with=equals.key=value

ends up in this flattened format like:

  'section.subsection=with=equals.key=value'

and we can't tell which was desired. We have traditionally resolved this
by taking the first "=" we see starting from the left, meaning that we
allowed arbitrary content in the value, but not in the subsection.

Let's make our environment format a bit more robust by separately
quoting the key and value. That turns those examples into:

  'section.subsection=with=equals.key'='value'

and:

  'section.subsection'='with=equals.key=value'

respectively, and we can tell the difference between them. We can detect
which format is in use for any given element of the list based on the
presence of the unquoted "=". That means we can continue to allow the
old format to work to support any callers which manually used the old
format, and we can even intermingle the two formats. The old format
wasn't documented, and nobody was supposed to be using it. But it's
likely that such callers exist in the wild, so it's nice if we can avoid
breaking them. Likewise, it may be possible to trigger an older version
of "git -c" that runs a script that calls into a newer version of "git
-c"; that new version would see the intermingled format.

This does create one complication, which is that the obvious format in
the new scheme for

  [section]
  some-bool

is:

  'section.some-bool'

with no equals. We'd mistake that for an old-style variable. And it even
has the same meaning in the old style, but:

  [section "with=equals"]
  some-bool

does not. It would be:

  'section.with=equals=some-bool'

which we'd take to mean:

  [section]
  with = equals=some-bool

in the old, ambiguous style. Likewise, we can't use:

  'section.some-bool'=''

because that's ambiguous with an actual empty string. Instead, we'll
again use the shell-quoting to give us a hint, and use:

  'section.some-bool'=

to show that we have no value.

Note that this commit just expands the reading side. We'll start writing
the new format via "git -c" in a future patch. In the meantime, the
existing "git -c" tests will make sure we didn't break reading the old
format. But we'll also add some explicit coverage of the two formats to
make sure we continue to handle the old one after we move the writing
side over.

And one final note: since we're now using the shell-quoting as a
semantically meaningful hint, this closes the door to us ever allowing
arbitrary shell quoting, like:

  'a'shell'would'be'ok'with'this'.key=value

But we have never supported that (only what sq_quote() would produce),
and we are probably better off keeping things simple, robust, and
backwards-compatible, than trying to make it easier for humans. We'll
continue not to advertise the format of the variable to users, and
instead keep "git -c" as the recommended mechanism for setting config
(even if we are trying to be kind not to break users who may be relying
on the current undocumented format).

Signed-off-by: Jeff King <peff@peff.net>
---
 config.c          | 69 +++++++++++++++++++++++++++++++++++------------
 t/t1300-config.sh | 52 +++++++++++++++++++++++++++++++++++
 2 files changed, 104 insertions(+), 17 deletions(-)

diff --git a/config.c b/config.c
index 7f7da60574..99062915d7 100644
--- a/config.c
+++ b/config.c
@@ -545,14 +545,62 @@ int git_config_parse_parameter(const char *text,
 	return ret;
 }
 
+static int parse_config_env_list(char *env, config_fn_t fn, void *data)
+{
+	char *cur = env;
+	while (cur && *cur) {
+		const char *key = sq_dequote_step(cur, &cur);
+		if (!key)
+			return error(_("bogus format in %s"),
+				     CONFIG_DATA_ENVIRONMENT);
+
+		if (!cur || isspace(*cur)) {
+			/* old-style 'key=value' */
+			if (git_config_parse_parameter(key, fn, data) < 0)
+				return -1;
+		}
+		else if (*cur == '=') {
+			/* new-style 'key'='value' */
+			const char *value;
+
+			cur++;
+			if (*cur == '\'') {
+				/* quoted value */
+				value = sq_dequote_step(cur, &cur);
+				if (!value || (cur && !isspace(*cur))) {
+					return error(_("bogus format in %s"),
+						     CONFIG_DATA_ENVIRONMENT);
+				}
+			} else if (!*cur || isspace(*cur)) {
+				/* implicit bool: 'key'= */
+				value = NULL;
+			} else {
+				return error(_("bogus format in %s"),
+					     CONFIG_DATA_ENVIRONMENT);
+			}
+
+			if (config_parse_pair(key, value, fn, data) < 0)
+				return -1;
+		}
+		else {
+			/* unknown format */
+			return error(_("bogus format in %s"),
+				     CONFIG_DATA_ENVIRONMENT);
+		}
+
+		if (cur) {
+			while (isspace(*cur))
+				cur++;
+		}
+	}
+	return 0;
+}
+
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
 	const char *env = getenv(CONFIG_DATA_ENVIRONMENT);
 	int ret = 0;
 	char *envw;
-	const char **argv = NULL;
-	int nr = 0, alloc = 0;
-	int i;
 	struct config_source source;
 
 	if (!env)
@@ -565,21 +613,8 @@ int git_config_from_parameters(config_fn_t fn, void *data)
 
 	/* sq_dequote will write over it */
 	envw = xstrdup(env);
+	ret = parse_config_env_list(envw, fn, data);
 
-	if (sq_dequote_to_argv(envw, &argv, &nr, &alloc) < 0) {
-		ret = error(_("bogus format in %s"), CONFIG_DATA_ENVIRONMENT);
-		goto out;
-	}
-
-	for (i = 0; i < nr; i++) {
-		if (git_config_parse_parameter(argv[i], fn, data) < 0) {
-			ret = -1;
-			goto out;
-		}
-	}
-
-out:
-	free(argv);
 	free(envw);
 	cf = source.prev;
 	return ret;
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 25437324c1..3f6778d474 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1294,6 +1294,58 @@ test_expect_success 'git -c is not confused by empty environment' '
 	GIT_CONFIG_PARAMETERS="" git -c x.one=1 config --list
 '
 
+test_expect_success 'GIT_CONFIG_PARAMETERS handles old-style entries' '
+	v="${SQ}key.one=foo${SQ}" &&
+	v="$v  ${SQ}key.two=bar${SQ}" &&
+	v="$v ${SQ}key.ambiguous=section.whatever=value${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.one foo
+	key.two bar
+	key.ambiguous section.whatever=value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'GIT_CONFIG_PARAMETERS handles new-style entries' '
+	v="${SQ}key.one${SQ}=${SQ}foo${SQ}" &&
+	v="$v  ${SQ}key.two${SQ}=${SQ}bar${SQ}" &&
+	v="$v ${SQ}key.ambiguous=section.whatever${SQ}=${SQ}value${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.one foo
+	key.two bar
+	key.ambiguous=section.whatever value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'old and new-style entries can mix' '
+	v="${SQ}key.oldone=oldfoo${SQ}" &&
+	v="$v ${SQ}key.newone${SQ}=${SQ}newfoo${SQ}" &&
+	v="$v ${SQ}key.oldtwo=oldbar${SQ}" &&
+	v="$v ${SQ}key.newtwo${SQ}=${SQ}newbar${SQ}" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.oldone oldfoo
+	key.newone newfoo
+	key.oldtwo oldbar
+	key.newtwo newbar
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'old and new bools with ambiguous subsection' '
+	v="${SQ}key.with=equals.oldbool${SQ}" &&
+	v="$v ${SQ}key.with=equals.newbool${SQ}=" &&
+	GIT_CONFIG_PARAMETERS=$v git config --get-regexp "key.*" >actual &&
+	cat >expect <<-EOF &&
+	key.with equals.oldbool
+	key.with=equals.newbool
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'detect bogus GIT_CONFIG_PARAMETERS' '
 	cat >expect <<-\EOF &&
 	env.one one
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v8 7/8] environment: make `getenv_safe()` a public function
  2021-01-12 12:26 ` [PATCH v8 0/8] " Patrick Steinhardt
                     ` (5 preceding siblings ...)
  2021-01-12 12:27   ` [PATCH v8 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS Patrick Steinhardt
@ 2021-01-12 12:27   ` Patrick Steinhardt
  2021-01-12 12:27   ` [PATCH v8 8/8] config: allow specifying config entries via envvar pairs Patrick Steinhardt
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-12 12:27 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1661 bytes --]

The `getenv_safe()` helper function helps to safely retrieve multiple
environment values without the need to depend on platform-specific
behaviour for the return value's lifetime. We'll make use of this
function in a following patch, so let's make it available by making it
non-static and adding a declaration.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 environment.c |  7 ++-----
 environment.h | 12 ++++++++++++
 2 files changed, 14 insertions(+), 5 deletions(-)
 create mode 100644 environment.h

diff --git a/environment.c b/environment.c
index bb518c61cd..2234af462c 100644
--- a/environment.c
+++ b/environment.c
@@ -9,6 +9,7 @@
  */
 #include "cache.h"
 #include "branch.h"
+#include "environment.h"
 #include "repository.h"
 #include "config.h"
 #include "refs.h"
@@ -152,11 +153,7 @@ static char *expand_namespace(const char *raw_namespace)
 	return strbuf_detach(&buf, NULL);
 }
 
-/*
- * Wrapper of getenv() that returns a strdup value. This value is kept
- * in argv to be freed later.
- */
-static const char *getenv_safe(struct strvec *argv, const char *name)
+const char *getenv_safe(struct strvec *argv, const char *name)
 {
 	const char *value = getenv(name);
 
diff --git a/environment.h b/environment.h
new file mode 100644
index 0000000000..d438b5c8f3
--- /dev/null
+++ b/environment.h
@@ -0,0 +1,12 @@
+#ifndef ENVIRONMENT_H
+#define ENVIRONMENT_H
+
+#include "strvec.h"
+
+/*
+ * Wrapper of getenv() that returns a strdup value. This value is kept
+ * in argv to be freed later.
+ */
+const char *getenv_safe(struct strvec *argv, const char *name);
+
+#endif
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v8 8/8] config: allow specifying config entries via envvar pairs
  2021-01-12 12:26 ` [PATCH v8 0/8] " Patrick Steinhardt
                     ` (6 preceding siblings ...)
  2021-01-12 12:27   ` [PATCH v8 7/8] environment: make `getenv_safe()` a public function Patrick Steinhardt
@ 2021-01-12 12:27   ` Patrick Steinhardt
  7 siblings, 0 replies; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-12 12:27 UTC (permalink / raw)
  To: git
  Cc: Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, Jeff King, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 10490 bytes --]

While we currently have the `GIT_CONFIG_PARAMETERS` environment variable
which can be used to pass runtime configuration data to git processes,
it's an internal implementation detail and not supposed to be used by
end users.

Next to being for internal use only, this way of passing config entries
has a major downside: the config keys need to be parsed as they contain
both key and value in a single variable. As such, it is left to the user
to escape any potentially harmful characters in the value, which is
quite hard to do if values are controlled by a third party.

This commit thus adds a new way of adding config entries via the
environment which gets rid of this shortcoming. If the user passes the
`GIT_CONFIG_COUNT=$n` environment variable, Git will parse environment
variable pairs `GIT_CONFIG_KEY_$i` and `GIT_CONFIG_VALUE_$i` for each
`i` in `[0,n)`.

While the same can be achieved with `git -c <name>=<value>`, one may
wish to not do so for potentially sensitive information. E.g. if one
wants to set `http.extraHeader` to contain an authentication token,
doing so via `-c` would trivially leak those credentials via e.g. ps(1),
which typically also shows command arguments.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git-config.txt |  16 +++++
 cache.h                      |   1 +
 config.c                     |  67 +++++++++++++++++---
 environment.c                |   1 +
 t/t1300-config.sh            | 115 ++++++++++++++++++++++++++++++++++-
 5 files changed, 191 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt
index 0e9351d3cb..4b4cc5c5e8 100644
--- a/Documentation/git-config.txt
+++ b/Documentation/git-config.txt
@@ -346,6 +346,22 @@ GIT_CONFIG_NOSYSTEM::
 
 See also <<FILES>>.
 
+GIT_CONFIG_COUNT::
+GIT_CONFIG_KEY_<n>::
+GIT_CONFIG_VALUE_<n>::
+	If GIT_CONFIG_COUNT is set to a positive number, all environment pairs
+	GIT_CONFIG_KEY_<n> and GIT_CONFIG_VALUE_<n> up to that number will be
+	added to the process's runtime configuration. The config pairs are
+	zero-indexed. Any missing key or value is treated as an error. An empty
+	GIT_CONFIG_COUNT is treated the same as GIT_CONFIG_COUNT=0, namely no
+	pairs are processed. These environment variables will override values
+	in configuration files, but will be overridden by any explicit options
+	passed via `git -c`.
++
+This is useful for cases where you want to spawn multiple git commands
+with a common configuration but cannot depend on a configuration file,
+for example when writing scripts.
+
 
 [[EXAMPLES]]
 EXAMPLES
diff --git a/cache.h b/cache.h
index 7109765748..a2e318c62b 100644
--- a/cache.h
+++ b/cache.h
@@ -472,6 +472,7 @@ static inline enum object_type object_type(unsigned int mode)
 #define TEMPLATE_DIR_ENVIRONMENT "GIT_TEMPLATE_DIR"
 #define CONFIG_ENVIRONMENT "GIT_CONFIG"
 #define CONFIG_DATA_ENVIRONMENT "GIT_CONFIG_PARAMETERS"
+#define CONFIG_COUNT_ENVIRONMENT "GIT_CONFIG_COUNT"
 #define EXEC_PATH_ENVIRONMENT "GIT_EXEC_PATH"
 #define CEILING_DIRECTORIES_ENVIRONMENT "GIT_CEILING_DIRECTORIES"
 #define NO_REPLACE_OBJECTS_ENVIRONMENT "GIT_NO_REPLACE_OBJECTS"
diff --git a/config.c b/config.c
index 99062915d7..a32569438a 100644
--- a/config.c
+++ b/config.c
@@ -8,6 +8,7 @@
 #include "cache.h"
 #include "branch.h"
 #include "config.h"
+#include "environment.h"
 #include "repository.h"
 #include "lockfile.h"
 #include "exec-cmd.h"
@@ -598,23 +599,73 @@ static int parse_config_env_list(char *env, config_fn_t fn, void *data)
 
 int git_config_from_parameters(config_fn_t fn, void *data)
 {
-	const char *env = getenv(CONFIG_DATA_ENVIRONMENT);
+	const char *env;
+	struct strbuf envvar = STRBUF_INIT;
+	struct strvec to_free = STRVEC_INIT;
 	int ret = 0;
-	char *envw;
+	char *envw = NULL;
 	struct config_source source;
 
-	if (!env)
-		return 0;
-
 	memset(&source, 0, sizeof(source));
 	source.prev = cf;
 	source.origin_type = CONFIG_ORIGIN_CMDLINE;
 	cf = &source;
 
-	/* sq_dequote will write over it */
-	envw = xstrdup(env);
-	ret = parse_config_env_list(envw, fn, data);
+	env = getenv(CONFIG_COUNT_ENVIRONMENT);
+	if (env) {
+		unsigned long count;
+		char *endp;
+		int i;
 
+		count = strtoul(env, &endp, 10);
+		if (*endp) {
+			ret = error(_("bogus count in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+		if (count > INT_MAX) {
+			ret = error(_("too many entries in %s"), CONFIG_COUNT_ENVIRONMENT);
+			goto out;
+		}
+
+		for (i = 0; i < count; i++) {
+			const char *key, *value;
+
+			strbuf_addf(&envvar, "GIT_CONFIG_KEY_%d", i);
+			key = getenv_safe(&to_free, envvar.buf);
+			if (!key) {
+				ret = error(_("missing config key %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			strbuf_addf(&envvar, "GIT_CONFIG_VALUE_%d", i);
+			value = getenv_safe(&to_free, envvar.buf);
+			if (!value) {
+				ret = error(_("missing config value %s"), envvar.buf);
+				goto out;
+			}
+			strbuf_reset(&envvar);
+
+			if (config_parse_pair(key, value, fn, data) < 0) {
+				ret = -1;
+				goto out;
+			}
+		}
+	}
+
+	env = getenv(CONFIG_DATA_ENVIRONMENT);
+	if (env) {
+		/* sq_dequote will write over it */
+		envw = xstrdup(env);
+		if (parse_config_env_list(envw, fn, data) < 0) {
+			ret = -1;
+			goto out;
+		}
+	}
+
+out:
+	strbuf_release(&envvar);
+	strvec_clear(&to_free);
 	free(envw);
 	cf = source.prev;
 	return ret;
diff --git a/environment.c b/environment.c
index 2234af462c..2f27008424 100644
--- a/environment.c
+++ b/environment.c
@@ -117,6 +117,7 @@ const char * const local_repo_env[] = {
 	ALTERNATE_DB_ENVIRONMENT,
 	CONFIG_ENVIRONMENT,
 	CONFIG_DATA_ENVIRONMENT,
+	CONFIG_COUNT_ENVIRONMENT,
 	DB_ENVIRONMENT,
 	GIT_DIR_ENVIRONMENT,
 	GIT_WORK_TREE_ENVIRONMENT,
diff --git a/t/t1300-config.sh b/t/t1300-config.sh
index 3f6778d474..89b47bf5bd 100755
--- a/t/t1300-config.sh
+++ b/t/t1300-config.sh
@@ -1424,6 +1424,117 @@ test_expect_success '--config-env handles keys with equals' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git config handles environment config pairs' '
+	GIT_CONFIG_COUNT=2 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="foo" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="bar" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one foo
+	pair.two bar
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs without count' '
+	test_must_fail env GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one 2>error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one
+'
+
+test_expect_success 'git config ignores pairs exceeding count' '
+	GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		GIT_CONFIG_KEY_1="pair.two" GIT_CONFIG_VALUE_1="value" \
+		git config --get-regexp "pair.*" >actual &&
+	cat >expect <<-EOF &&
+	pair.one value
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'git config ignores pairs with zero count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT=0 GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config ignores pairs with empty count' '
+	test_must_fail env \
+		GIT_CONFIG_COUNT= GIT_CONFIG_KEY_0="pair.one" GIT_CONFIG_VALUE_0="value" \
+		git config pair.one >error &&
+	test_must_be_empty error
+'
+
+test_expect_success 'git config fails with invalid count' '
+	test_must_fail env GIT_CONFIG_COUNT=10a git config --list 2>error &&
+	test_i18ngrep "bogus count" error &&
+	test_must_fail env GIT_CONFIG_COUNT=9999999999999999 git config --list 2>error &&
+	test_i18ngrep "too many entries" error
+'
+
+test_expect_success 'git config fails with missing config key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_VALUE_0="value" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config key" error
+'
+
+test_expect_success 'git config fails with missing config value' '
+	test_must_fail env GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0="pair.one" \
+		git config --list 2>error &&
+	test_i18ngrep "missing config value" error
+'
+
+test_expect_success 'git config fails with invalid config pair key' '
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0= GIT_CONFIG_VALUE_0=value \
+		git config --list &&
+	test_must_fail env GIT_CONFIG_COUNT=1 \
+		GIT_CONFIG_KEY_0=missing-section GIT_CONFIG_VALUE_0=value \
+		git config --list
+'
+
+test_expect_success 'environment overrides config file' '
+	test_when_finished "rm -f .git/config" &&
+	cat >.git/config <<-EOF &&
+	[pair]
+	one = value
+	EOF
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=override \
+		git config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'GIT_CONFIG_PARAMETERS overrides environment config' '
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
+		GIT_CONFIG_PARAMETERS="${SQ}pair.one=override${SQ}" \
+		git config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success 'command line overrides environment config' '
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=pair.one GIT_CONFIG_VALUE_0=value \
+		git -c pair.one=override config pair.one >actual &&
+	cat >expect <<-EOF &&
+	override
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'git config --edit works' '
 	git config -f tmp test.value no &&
 	echo test.value=yes >expect &&
@@ -1769,9 +1880,11 @@ test_expect_success '--show-origin with --list' '
 	file:.git/config	user.override=local
 	file:.git/config	include.path=../include/relative.include
 	file:.git/../include/relative.include	user.relative=include
+	command line:	user.environ=true
 	command line:	user.cmdline=true
 	EOF
-	git -c user.cmdline=true config --list --show-origin >output &&
+	GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=user.environ GIT_CONFIG_VALUE_0=true\
+		git -c user.cmdline=true config --list --show-origin >output &&
 	test_cmp expect output
 '
 
-- 
2.30.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v8 5/8] config: store "git -c" variables using more robust format
  2021-01-12 12:27   ` [PATCH v8 5/8] config: store "git -c" variables using more robust format Patrick Steinhardt
@ 2021-01-15 19:16     ` Jeff King
  2021-01-20  6:29       ` Patrick Steinhardt
  0 siblings, 1 reply; 116+ messages in thread
From: Jeff King @ 2021-01-15 19:16 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, brian m. carlson, Philip Oakley

On Tue, Jan 12, 2021 at 01:27:01PM +0100, Patrick Steinhardt wrote:

> The previous commit added a new format for $GIT_CONFIG_PARAMETERS which
> is able to robustly handle subsections with "=" in them. Let's start

It looks like this commit and 6 got flipped from the original ordering
(it's the "previous commit" talked about here). And indeed, running the
tests on the individual commits in this series shows that we fail at
this step (because we are writing the new format, but the reader is too
strict to accept it).

That doesn't matter to the end result, of course, but it hurts later
bisecting. Just flipping patches 5 and 6 makes it all work.

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v8 5/8] config: store "git -c" variables using more robust format
  2021-01-15 19:16     ` Jeff King
@ 2021-01-20  6:29       ` Patrick Steinhardt
  2021-01-20  6:55         ` Junio C Hamano
  0 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-20  6:29 UTC (permalink / raw)
  To: Jeff King
  Cc: git, Simon Ruderich, Ævar Arnfjörð Bjarmason,
	Junio C Hamano, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 976 bytes --]

On Fri, Jan 15, 2021 at 02:16:46PM -0500, Jeff King wrote:
> On Tue, Jan 12, 2021 at 01:27:01PM +0100, Patrick Steinhardt wrote:
> 
> > The previous commit added a new format for $GIT_CONFIG_PARAMETERS which
> > is able to robustly handle subsections with "=" in them. Let's start
> 
> It looks like this commit and 6 got flipped from the original ordering
> (it's the "previous commit" talked about here). And indeed, running the
> tests on the individual commits in this series shows that we fail at
> this step (because we are writing the new format, but the reader is too
> strict to accept it).
> 
> That doesn't matter to the end result, of course, but it hurts later
> bisecting. Just flipping patches 5 and 6 makes it all work.
> 
> -Peff

Oops, yes. That always happens to me when I start using git-am(1). I see
that the patch series has been applied to "next" already, so does it
make any sense to resend with patches 5 and 6 flipped?

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v8 5/8] config: store "git -c" variables using more robust format
  2021-01-20  6:29       ` Patrick Steinhardt
@ 2021-01-20  6:55         ` Junio C Hamano
  2021-01-20  7:42           ` Patrick Steinhardt
  0 siblings, 1 reply; 116+ messages in thread
From: Junio C Hamano @ 2021-01-20  6:55 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: Jeff King, git, Simon Ruderich,
	Ævar Arnfjörð Bjarmason, brian m. carlson,
	Philip Oakley

Patrick Steinhardt <ps@pks.im> writes:

> On Fri, Jan 15, 2021 at 02:16:46PM -0500, Jeff King wrote:
>> On Tue, Jan 12, 2021 at 01:27:01PM +0100, Patrick Steinhardt wrote:
>> 
>> > The previous commit added a new format for $GIT_CONFIG_PARAMETERS which
>> > is able to robustly handle subsections with "=" in them. Let's start
>> 
>> It looks like this commit and 6 got flipped from the original ordering
>> (it's the "previous commit" talked about here). And indeed, running the
>> tests on the individual commits in this series shows that we fail at
>> this step (because we are writing the new format, but the reader is too
>> strict to accept it).
>> 
>> That doesn't matter to the end result, of course, but it hurts later
>> bisecting. Just flipping patches 5 and 6 makes it all work.
>> 
>> -Peff
>
> Oops, yes. That always happens to me when I start using git-am(1). I see
> that the patch series has been applied to "next" already, so does it
> make any sense to resend with patches 5 and 6 flipped?

I recall saying that I'd "rebase -i" before merging it to "next".
Did I forget to do so?

Disecting 4ed03412 (Merge branch 'ps/config-env-pairs' into next,
2021-01-15), we see:

$ git log --oneline --reverse master..4ed03412^2 | cat -n
     1	b0812b6ac0 git: add `--super-prefix` to usage string
     2	ce81b1da23 config: add new way to pass config via `--config-env`
     3	13c44953fb quote: make sq_dequote_step() a public function
     4	b342ae61b3 config: extract function to parse config pairs
     5	f9dbb64fad config: parse more robust format in GIT_CONFIG_PARAMETERS
     6	1ff21c05ba config: store "git -c" variables using more robust format
     7	b9d147fb15 environment: make `getenv_safe()` a public function
     8	d8d77153ea config: allow specifying config entries via envvar pairs

The 5/8 that needs to come after 6/8 has title "store ... using more
rebust format" and that is the 6th patch in the series merged to
'next'.  The 6/8 that needs to come before that one was called
"parse more robust format" and it now appears as the 5th patch.

So it seems all is well?

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v8 5/8] config: store "git -c" variables using more robust format
  2021-01-20  6:55         ` Junio C Hamano
@ 2021-01-20  7:42           ` Patrick Steinhardt
  2021-01-20 22:28             ` Junio C Hamano
  0 siblings, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2021-01-20  7:42 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Jeff King, git, Simon Ruderich,
	Ævar Arnfjörð Bjarmason, brian m. carlson,
	Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 2376 bytes --]

On Tue, Jan 19, 2021 at 10:55:48PM -0800, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > On Fri, Jan 15, 2021 at 02:16:46PM -0500, Jeff King wrote:
> >> On Tue, Jan 12, 2021 at 01:27:01PM +0100, Patrick Steinhardt wrote:
> >> 
> >> > The previous commit added a new format for $GIT_CONFIG_PARAMETERS which
> >> > is able to robustly handle subsections with "=" in them. Let's start
> >> 
> >> It looks like this commit and 6 got flipped from the original ordering
> >> (it's the "previous commit" talked about here). And indeed, running the
> >> tests on the individual commits in this series shows that we fail at
> >> this step (because we are writing the new format, but the reader is too
> >> strict to accept it).
> >> 
> >> That doesn't matter to the end result, of course, but it hurts later
> >> bisecting. Just flipping patches 5 and 6 makes it all work.
> >> 
> >> -Peff
> >
> > Oops, yes. That always happens to me when I start using git-am(1). I see
> > that the patch series has been applied to "next" already, so does it
> > make any sense to resend with patches 5 and 6 flipped?
> 
> I recall saying that I'd "rebase -i" before merging it to "next".
> Did I forget to do so?
> 
> Disecting 4ed03412 (Merge branch 'ps/config-env-pairs' into next,
> 2021-01-15), we see:
> 
> $ git log --oneline --reverse master..4ed03412^2 | cat -n
>      1	b0812b6ac0 git: add `--super-prefix` to usage string
>      2	ce81b1da23 config: add new way to pass config via `--config-env`
>      3	13c44953fb quote: make sq_dequote_step() a public function
>      4	b342ae61b3 config: extract function to parse config pairs
>      5	f9dbb64fad config: parse more robust format in GIT_CONFIG_PARAMETERS
>      6	1ff21c05ba config: store "git -c" variables using more robust format
>      7	b9d147fb15 environment: make `getenv_safe()` a public function
>      8	d8d77153ea config: allow specifying config entries via envvar pairs
> 
> The 5/8 that needs to come after 6/8 has title "store ... using more
> rebust format" and that is the 6th patch in the series merged to
> 'next'.  The 6/8 that needs to come before that one was called
> "parse more robust format" and it now appears as the 5th patch.
> 
> So it seems all is well?

Indeed, I missed your message about the interactive rebase. Thanks!

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v8 5/8] config: store "git -c" variables using more robust format
  2021-01-20  7:42           ` Patrick Steinhardt
@ 2021-01-20 22:28             ` Junio C Hamano
  0 siblings, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2021-01-20 22:28 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: Jeff King, git, Simon Ruderich,
	Ævar Arnfjörð Bjarmason, brian m. carlson,
	Philip Oakley

Patrick Steinhardt <ps@pks.im> writes:

>> The 5/8 that needs to come after 6/8 has title "store ... using more
>> rebust format" and that is the 6th patch in the series merged to
>> 'next'.  The 6/8 that needs to come before that one was called
>> "parse more robust format" and it now appears as the 5th patch.
>> 
>> So it seems all is well?
>
> Indeed, I missed your message about the interactive rebase. Thanks!

Thanks for contributing in the first place, and thanks for double
checking.  Very much appreciated.


^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v8 2/8] config: add new way to pass config via `--config-env`
  2021-01-12 12:26   ` [PATCH v8 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
@ 2021-04-16 15:40     ` Ævar Arnfjörð Bjarmason
  2021-04-17  8:38       ` Jeff King
  0 siblings, 1 reply; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-16 15:40 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: git, Simon Ruderich, Junio C Hamano, Jeff King, brian m. carlson,
	Philip Oakley


On Tue, Jan 12 2021, Patrick Steinhardt wrote:

A minor doc bug that wasn't spotted before landing. Here we say
"--config-env foo=bar" will work:

> diff --git a/Documentation/git.txt b/Documentation/git.txt
> index a6d4ad0818..d36e6fd482 100644
> --- a/Documentation/git.txt
> +++ b/Documentation/git.txt
> @@ -13,7 +13,7 @@ SYNOPSIS
>      [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
>      [-p|--paginate|-P|--no-pager] [--no-replace-objects] [--bare]
>      [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
> -    [--super-prefix=<path>]
> +    [--super-prefix=<path>] [--config-env <name>=<envvar>]
>      <command> [<args>]
>  
>  DESCRIPTION
> @@ -80,6 +80,28 @@ config file). Including the equals but with an empty value (like `git -c
>  foo.bar= ...`) sets `foo.bar` to the empty string which `git config
>  --type=bool` will convert to `false`.

But not here, we ask for "--config-env=" (note the "="):

> +--config-env=<name>=<envvar>::
> +	Like `-c <name>=<value>`, give configuration variable
> +	'<name>' a value, where <envvar> is the name of an
> +	environment variable from which to retrieve the value. Unlike
> +	`-c` there is no shortcut for directly setting the value to an
> +	empty string, instead the environment variable itself must be
> +	set to the empty string.  It is an error if the `<envvar>` does not exist
> +	in the environment. `<envvar>` may not contain an equals sign
> +	to avoid ambiguity with `<name>`s which contain one.
> ++
> [...]
> +		} else if (skip_prefix(cmd, "--config-env=", &cmd)) {
> +			git_config_push_env(cmd);

But as this...

> +test_expect_success 'git --config-env=key=envvar support' '
> +	cat >expect <<-\EOF &&
> +	value
> +	value
> +	false
> +	EOF
> +	{
> +		ENVVAR=value git --config-env=core.name=ENVVAR config core.name &&
> +		ENVVAR=value git --config-env=foo.CamelCase=ENVVAR config foo.camelcase &&
> +		ENVVAR= git --config-env=foo.flag=ENVVAR config --bool foo.flag
> +	} >actual &&
> +	test_cmp expect actual
> +'
> +
> +test_expect_success 'git --config-env fails with invalid parameters' '
> +	test_must_fail git --config-env=foo.flag config --bool foo.flag 2>error &&
> +	test_i18ngrep "invalid config format: foo.flag" error &&
> +	test_must_fail git --config-env=foo.flag= config --bool foo.flag 2>error &&
> +	test_i18ngrep "missing environment variable name for configuration ${SQ}foo.flag${SQ}" error &&
> +	sane_unset NONEXISTENT &&
> +	test_must_fail git --config-env=foo.flag=NONEXISTENT config --bool foo.flag 2>error &&
> +	test_i18ngrep "missing environment variable ${SQ}NONEXISTENT${SQ} for configuration ${SQ}foo.flag${SQ}" error
> +'
> +
> +test_expect_success 'git -c and --config-env work together' '
> +	cat >expect <<-\EOF &&
> +	bar.cmd cmd-value
> +	bar.env env-value
> +	EOF
> +	ENVVAR=env-value git \
> +		-c bar.cmd=cmd-value \
> +		--config-env=bar.env=ENVVAR \
> +		config --get-regexp "^bar.*" >actual &&
> +	test_cmp expect actual
> +'
> +
> +test_expect_success 'git -c and --config-env override each other' '
> +	cat >expect <<-\EOF &&
> +	env
> +	cmd
> +	EOF
> +	{
> +		ENVVAR=env git -c bar.bar=cmd --config-env=bar.bar=ENVVAR config bar.bar &&
> +		ENVVAR=env git --config-env=bar.bar=ENVVAR -c bar.bar=cmd config bar.bar
> +	} >actual &&
> +	test_cmp expect actual
> +'
> +
>  test_expect_success 'git config --edit works' '
>  	git config -f tmp test.value no &&
>  	echo test.value=yes >expect &&

...and the tests show we just support the --opt=foo=bar form, not --opt
foo=bar.

Bonus points to anyone sorting out some of the existing inconsistencies
when fixing this, i.e. --exec-path supports either the "=" form, or not,
but various other skip_prefix() in the same function don't, seemingly
(but I have not tested) for no good reason.

It seems to me that having a skip_prefix_opt() or something would be a
good fix for this, i.e. a "maybe trim the last '='" version of
skip_prefix. Then we could just consistently use that.

Or maybe there's some reason we don't want to be as lax as --exec-path
with any other option...


^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v8 2/8] config: add new way to pass config via `--config-env`
  2021-04-16 15:40     ` Ævar Arnfjörð Bjarmason
@ 2021-04-17  8:38       ` Jeff King
  2021-04-19 15:28         ` Patrick Steinhardt
  2021-04-20 10:59         ` Ævar Arnfjörð Bjarmason
  0 siblings, 2 replies; 116+ messages in thread
From: Jeff King @ 2021-04-17  8:38 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Patrick Steinhardt, git, Simon Ruderich, Junio C Hamano,
	brian m. carlson, Philip Oakley

On Fri, Apr 16, 2021 at 05:40:36PM +0200, Ævar Arnfjörð Bjarmason wrote:

> Bonus points to anyone sorting out some of the existing inconsistencies
> when fixing this, i.e. --exec-path supports either the "=" form, or not,
> but various other skip_prefix() in the same function don't, seemingly
> (but I have not tested) for no good reason.

I suspect just because it's more (per-option) work to support both
types, and nobody really cared enough to do so.

> It seems to me that having a skip_prefix_opt() or something would be a
> good fix for this, i.e. a "maybe trim the last '='" version of
> skip_prefix. Then we could just consistently use that.

There's a similar situation in the revision parser (which does not use
our regular parse-options). There we have a parse_long_opt() helper
which does the right thing. We could use that more widely.

I also wouldn't be surprised if we could leverage one of the
sub-functions of parse-options, but it might turn into a rabbit hole.
Converting the whole thing to the usual parse_options() might get
awkward, since many of the options operate at time-of-parse, not after
we've seen everything (I suspect many of them don't care either way, but
you're always risking subtle regressions there).

> Or maybe there's some reason we don't want to be as lax as --exec-path
> with any other option...

I can't think of one.

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v8 2/8] config: add new way to pass config via `--config-env`
  2021-04-17  8:38       ` Jeff King
@ 2021-04-19 15:28         ` Patrick Steinhardt
  2021-04-20 11:01           ` Ævar Arnfjörð Bjarmason
  2021-04-20 10:59         ` Ævar Arnfjörð Bjarmason
  1 sibling, 1 reply; 116+ messages in thread
From: Patrick Steinhardt @ 2021-04-19 15:28 UTC (permalink / raw)
  To: Jeff King
  Cc: Ævar Arnfjörð Bjarmason, git, Simon Ruderich,
	Junio C Hamano, brian m. carlson, Philip Oakley

[-- Attachment #1: Type: text/plain, Size: 1769 bytes --]

On Sat, Apr 17, 2021 at 04:38:31AM -0400, Jeff King wrote:
> On Fri, Apr 16, 2021 at 05:40:36PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> > Bonus points to anyone sorting out some of the existing inconsistencies
> > when fixing this, i.e. --exec-path supports either the "=" form, or not,
> > but various other skip_prefix() in the same function don't, seemingly
> > (but I have not tested) for no good reason.
> 
> I suspect just because it's more (per-option) work to support both
> types, and nobody really cared enough to do so.
> 
> > It seems to me that having a skip_prefix_opt() or something would be a
> > good fix for this, i.e. a "maybe trim the last '='" version of
> > skip_prefix. Then we could just consistently use that.
> 
> There's a similar situation in the revision parser (which does not use
> our regular parse-options). There we have a parse_long_opt() helper
> which does the right thing. We could use that more widely.
> 
> I also wouldn't be surprised if we could leverage one of the
> sub-functions of parse-options, but it might turn into a rabbit hole.
> Converting the whole thing to the usual parse_options() might get
> awkward, since many of the options operate at time-of-parse, not after
> we've seen everything (I suspect many of them don't care either way, but
> you're always risking subtle regressions there).
> 
> > Or maybe there's some reason we don't want to be as lax as --exec-path
> > with any other option...
> 
> I can't think of one.
> 
> -Peff

`--exec-path` does two different things based on whether you pass a "="
or not: either you tell git where its binaries are, or you ask it where
it thinks they're. It's still true for some (most?) of the other options
though.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v8 2/8] config: add new way to pass config via `--config-env`
  2021-04-17  8:38       ` Jeff King
  2021-04-19 15:28         ` Patrick Steinhardt
@ 2021-04-20 10:59         ` Ævar Arnfjörð Bjarmason
  2021-04-23 10:05           ` Jeff King
  1 sibling, 1 reply; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-20 10:59 UTC (permalink / raw)
  To: Jeff King
  Cc: Patrick Steinhardt, git, Simon Ruderich, Junio C Hamano,
	brian m. carlson, Philip Oakley


On Sat, Apr 17 2021, Jeff King wrote:

> On Fri, Apr 16, 2021 at 05:40:36PM +0200, Ævar Arnfjörð Bjarmason wrote:
>
>> Bonus points to anyone sorting out some of the existing inconsistencies
>> when fixing this, i.e. --exec-path supports either the "=" form, or not,
>> but various other skip_prefix() in the same function don't, seemingly
>> (but I have not tested) for no good reason.
>
> I suspect just because it's more (per-option) work to support both
> types, and nobody really cared enough to do so.
>
>> It seems to me that having a skip_prefix_opt() or something would be a
>> good fix for this, i.e. a "maybe trim the last '='" version of
>> skip_prefix. Then we could just consistently use that.
>
> There's a similar situation in the revision parser (which does not use
> our regular parse-options). There we have a parse_long_opt() helper
> which does the right thing. We could use that more widely.
>
> I also wouldn't be surprised if we could leverage one of the
> sub-functions of parse-options, but it might turn into a rabbit hole.
> Converting the whole thing to the usual parse_options() might get
> awkward, since many of the options operate at time-of-parse, not after
> we've seen everything (I suspect many of them don't care either way, but
> you're always risking subtle regressions there).

So we could use parse_options() and guarantee the existing behavior if
they were all OPT_CALLBACK?

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v8 2/8] config: add new way to pass config via `--config-env`
  2021-04-19 15:28         ` Patrick Steinhardt
@ 2021-04-20 11:01           ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-20 11:01 UTC (permalink / raw)
  To: Patrick Steinhardt
  Cc: Jeff King, git, Simon Ruderich, Junio C Hamano, brian m. carlson,
	Philip Oakley


On Mon, Apr 19 2021, Patrick Steinhardt wrote:

> On Sat, Apr 17, 2021 at 04:38:31AM -0400, Jeff King wrote:
>> On Fri, Apr 16, 2021 at 05:40:36PM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> > Bonus points to anyone sorting out some of the existing inconsistencies
>> > when fixing this, i.e. --exec-path supports either the "=" form, or not,
>> > but various other skip_prefix() in the same function don't, seemingly
>> > (but I have not tested) for no good reason.
>> 
>> I suspect just because it's more (per-option) work to support both
>> types, and nobody really cared enough to do so.
>> 
>> > It seems to me that having a skip_prefix_opt() or something would be a
>> > good fix for this, i.e. a "maybe trim the last '='" version of
>> > skip_prefix. Then we could just consistently use that.
>> 
>> There's a similar situation in the revision parser (which does not use
>> our regular parse-options). There we have a parse_long_opt() helper
>> which does the right thing. We could use that more widely.
>> 
>> I also wouldn't be surprised if we could leverage one of the
>> sub-functions of parse-options, but it might turn into a rabbit hole.
>> Converting the whole thing to the usual parse_options() might get
>> awkward, since many of the options operate at time-of-parse, not after
>> we've seen everything (I suspect many of them don't care either way, but
>> you're always risking subtle regressions there).
>> 
>> > Or maybe there's some reason we don't want to be as lax as --exec-path
>> > with any other option...
>> 
>> I can't think of one.
>> 
>> -Peff
>
> `--exec-path` does two different things based on whether you pass a "="
> or not: either you tell git where its binaries are, or you ask it where
> it thinks they're. It's still true for some (most?) of the other options
> though.

I don't know how I got those conflated, but FWIW I meant (or at least
think I did) the likes of --namespace[=], --git-dir[=] and
--work-tree[=] where the "=" is really optional, but otherwise the
option behaves the same.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v8 2/8] config: add new way to pass config via `--config-env`
  2021-04-20 10:59         ` Ævar Arnfjörð Bjarmason
@ 2021-04-23 10:05           ` Jeff King
  2021-05-19 11:36             ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 116+ messages in thread
From: Jeff King @ 2021-04-23 10:05 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Patrick Steinhardt, git, Simon Ruderich, Junio C Hamano,
	brian m. carlson, Philip Oakley

On Tue, Apr 20, 2021 at 12:59:16PM +0200, Ævar Arnfjörð Bjarmason wrote:

> >> It seems to me that having a skip_prefix_opt() or something would be a
> >> good fix for this, i.e. a "maybe trim the last '='" version of
> >> skip_prefix. Then we could just consistently use that.
> >
> > There's a similar situation in the revision parser (which does not use
> > our regular parse-options). There we have a parse_long_opt() helper
> > which does the right thing. We could use that more widely.
> >
> > I also wouldn't be surprised if we could leverage one of the
> > sub-functions of parse-options, but it might turn into a rabbit hole.
> > Converting the whole thing to the usual parse_options() might get
> > awkward, since many of the options operate at time-of-parse, not after
> > we've seen everything (I suspect many of them don't care either way, but
> > you're always risking subtle regressions there).
> 
> So we could use parse_options() and guarantee the existing behavior if
> they were all OPT_CALLBACK?

I _think_ so, but the result might be quite hard to read (the logic
would be scattered all over a bunch of tiny callbacks). But it might not
be too bad. Especially if you figure out which ones actually need the
time-of-parse logic and use more vanilla OPT_* for the others (that's
the rabbit hole I alluded to).

I think things like the "--exec-path" behavior that Patrick mentioned
would still work (I think it's just a stock OPTARG).

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v8 2/8] config: add new way to pass config via `--config-env`
  2021-04-23 10:05           ` Jeff King
@ 2021-05-19 11:36             ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-19 11:36 UTC (permalink / raw)
  To: Jeff King
  Cc: Patrick Steinhardt, git, Simon Ruderich, Junio C Hamano,
	brian m. carlson, Philip Oakley


On Fri, Apr 23 2021, Jeff King wrote:

> On Tue, Apr 20, 2021 at 12:59:16PM +0200, Ævar Arnfjörð Bjarmason wrote:
>
>> >> It seems to me that having a skip_prefix_opt() or something would be a
>> >> good fix for this, i.e. a "maybe trim the last '='" version of
>> >> skip_prefix. Then we could just consistently use that.
>> >
>> > There's a similar situation in the revision parser (which does not use
>> > our regular parse-options). There we have a parse_long_opt() helper
>> > which does the right thing. We could use that more widely.
>> >
>> > I also wouldn't be surprised if we could leverage one of the
>> > sub-functions of parse-options, but it might turn into a rabbit hole.
>> > Converting the whole thing to the usual parse_options() might get
>> > awkward, since many of the options operate at time-of-parse, not after
>> > we've seen everything (I suspect many of them don't care either way, but
>> > you're always risking subtle regressions there).
>> 
>> So we could use parse_options() and guarantee the existing behavior if
>> they were all OPT_CALLBACK?
>
> I _think_ so, but the result might be quite hard to read (the logic
> would be scattered all over a bunch of tiny callbacks). But it might not
> be too bad. Especially if you figure out which ones actually need the
> time-of-parse logic and use more vanilla OPT_* for the others (that's
> the rabbit hole I alluded to).
>
> I think things like the "--exec-path" behavior that Patrick mentioned
> would still work (I think it's just a stock OPTARG).

[Mostly for my own future reference]: There's also the
parse_options_step() API to process options one at a time, which AFAICT
could be used in this case.

But having glanced at it again (but not come up with a patch) I think it
could be handled with OPT_CALLBACK + not caring about the order,
mostly. The "envchanged" could be passed as a custom flag probably...

^ permalink raw reply	[flat|nested] 116+ messages in thread

end of thread, other threads:[~2021-05-19 11:39 UTC | newest]

Thread overview: 116+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-24 10:50 [PATCH v2 0/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
2020-11-24 10:50 ` [PATCH v2 1/2] config: extract function to parse config pairs Patrick Steinhardt
2020-11-24 10:50 ` [PATCH v2 2/2] config: allow specifying config entries via envvar pairs Patrick Steinhardt
2020-11-25  3:39   ` Junio C Hamano
2020-11-25  7:06     ` Patrick Steinhardt
2020-11-25  7:41       ` Junio C Hamano
2020-11-25  7:57         ` Patrick Steinhardt
2020-11-25  8:47   ` Ævar Arnfjörð Bjarmason
2020-11-25  9:00   ` Ævar Arnfjörð Bjarmason
2020-11-25 19:50     ` Junio C Hamano
2020-11-25 10:41 ` [PATCH v2 0/2] " Jeff King
2020-11-25 13:16   ` Patrick Steinhardt
2020-11-26  0:36     ` Jeff King
2020-11-25 20:28   ` Junio C Hamano
2020-11-25 22:47   ` brian m. carlson
2020-11-26  6:31     ` Patrick Steinhardt
2020-12-01  9:47   ` Patrick Steinhardt
2020-12-01 11:30     ` Jeff King
2020-12-01  9:55 ` [PATCH v3 0/4] " Patrick Steinhardt
2020-12-01  9:55   ` [PATCH v3 1/4] environment: make `getenv_safe()` non-static Patrick Steinhardt
2020-12-01  9:56   ` [PATCH v3 2/4] config: extract function to parse config pairs Patrick Steinhardt
2020-12-01  9:56   ` [PATCH v3 3/4] config: refactor parsing of GIT_CONFIG_PARAMETERS Patrick Steinhardt
2020-12-01  9:56   ` [PATCH v3 4/4] config: allow specifying config entries via envvar pairs Patrick Steinhardt
2020-12-09 11:52 ` [PATCH v4 0/6] config: allow specifying config entries via env Patrick Steinhardt
2020-12-09 11:52   ` [PATCH v4 1/6] git: add `--super-prefix` to usage string Patrick Steinhardt
2020-12-09 11:52   ` [PATCH v4 2/6] config: add new way to pass config via `--config-env` Patrick Steinhardt
2020-12-09 14:40     ` Ævar Arnfjörð Bjarmason
2020-12-09 16:24       ` Jeff King
2020-12-11 13:24         ` Patrick Steinhardt
2020-12-11 14:21           ` Jeff King
2020-12-11 14:54             ` Patrick Steinhardt
2020-12-11 16:10               ` Jeff King
2020-12-09 16:10     ` Jeff King
2020-12-09 16:12       ` [PATCH 1/3] quote: make sq_dequote_step() a public function Jeff King
2020-12-09 16:17       ` [PATCH 2/3] config: parse more robust format in GIT_CONFIG_PARAMETERS Jeff King
2020-12-09 16:20       ` [PATCH 3/3] config: store "git -c" variables using more robust format Jeff King
2020-12-09 16:34         ` Jeff King
2020-12-10 20:55         ` Ævar Arnfjörð Bjarmason
2020-12-10 21:49           ` Junio C Hamano
2020-12-11 13:21           ` Jeff King
2020-12-10  0:00       ` [PATCH v4 2/6] config: add new way to pass config via `--config-env` Junio C Hamano
2020-12-10  0:09         ` Jeff King
2020-12-10  0:57           ` Junio C Hamano
2020-12-11 13:24       ` Patrick Steinhardt
2020-12-11 14:20         ` Jeff King
2020-12-09 11:52   ` [PATCH v4 3/6] environment: make `getenv_safe()` non-static Patrick Steinhardt
2020-12-09 11:52   ` [PATCH v4 4/6] config: extract function to parse config pairs Patrick Steinhardt
2020-12-09 13:12     ` Ævar Arnfjörð Bjarmason
2020-12-09 11:52   ` [PATCH v4 5/6] config: refactor parsing of GIT_CONFIG_PARAMETERS Patrick Steinhardt
2020-12-09 11:52   ` [PATCH v4 6/6] config: allow specifying config entries via envvar pairs Patrick Steinhardt
2020-12-09 15:29   ` [PATCH v4 0/6] config: allow specifying config entries via env Ævar Arnfjörð Bjarmason
2020-12-11 13:35     ` Patrick Steinhardt
2020-12-11 14:27       ` Jeff King
2020-12-11 14:42         ` Jeff King
2020-12-11 14:58           ` Patrick Steinhardt
2020-12-11 14:47         ` Patrick Steinhardt
2020-12-11 15:21           ` Ævar Arnfjörð Bjarmason
2020-12-11 16:02           ` Jeff King
2020-12-16  7:52 ` [PATCH v5 0/8] " Patrick Steinhardt
2020-12-16  7:52   ` [PATCH v5 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
2020-12-16  7:52   ` [PATCH v5 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
2020-12-23 21:35     ` Junio C Hamano
2020-12-16  7:54   ` [PATCH v5 4/8] config: extract function to parse config pairs Patrick Steinhardt
2020-12-16  7:54   ` [PATCH v5 7/8] environment: make `getenv_safe()` a public function Patrick Steinhardt
2020-12-16  7:54   ` [PATCH v5 8/8] config: allow specifying config entries via envvar pairs Patrick Steinhardt
2020-12-23 21:14     ` Junio C Hamano
2020-12-23 21:55       ` Junio C Hamano
2021-01-06 10:28         ` Patrick Steinhardt
2021-01-06 21:07           ` Junio C Hamano
2020-12-16  7:56   ` [PATCH v5 3/8] quote: make sq_dequote_step() a public function Patrick Steinhardt
2020-12-16  7:56   ` [PATCH v5 5/8] config: store "git -c" variables using more robust format Patrick Steinhardt
2020-12-16  7:57   ` [PATCH v5 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS Patrick Steinhardt
2020-12-16 20:01     ` Phillip Wood
2021-01-07  6:36 ` [PATCH v6 0/8] config: allow specifying config entries via env Patrick Steinhardt
2021-01-07  6:36   ` [PATCH v6 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
2021-01-07  6:36   ` [PATCH v6 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
2021-01-10 20:29     ` Simon Ruderich
2021-01-11  0:29       ` Junio C Hamano
2021-01-11  8:24         ` Patrick Steinhardt
2021-01-07  6:36   ` [PATCH v6 3/8] quote: make sq_dequote_step() a public function Patrick Steinhardt
2021-01-07  6:37   ` [PATCH v6 4/8] config: extract function to parse config pairs Patrick Steinhardt
2021-01-07  6:37   ` [PATCH v6 5/8] config: store "git -c" variables using more robust format Patrick Steinhardt
2021-01-07  6:37   ` [PATCH v6 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS Patrick Steinhardt
2021-01-07  6:37   ` [PATCH v6 7/8] environment: make `getenv_safe()` a public function Patrick Steinhardt
2021-01-07  6:37   ` [PATCH v6 8/8] config: allow specifying config entries via envvar pairs Patrick Steinhardt
2021-01-11  8:36 ` [PATCH v7 0/8] " Patrick Steinhardt
2021-01-11  8:36   ` [PATCH v7 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
2021-01-11  8:36   ` [PATCH v7 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
2021-01-11 22:34     ` Junio C Hamano
2021-01-11  8:36   ` [PATCH v7 3/8] quote: make sq_dequote_step() a public function Patrick Steinhardt
2021-01-11  8:36   ` [PATCH v7 4/8] config: extract function to parse config pairs Patrick Steinhardt
2021-01-11  8:37   ` [PATCH v7 5/8] config: store "git -c" variables using more robust format Patrick Steinhardt
2021-01-11  8:37   ` [PATCH v7 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS Patrick Steinhardt
2021-01-11  8:37   ` [PATCH v7 7/8] environment: make `getenv_safe()` a public function Patrick Steinhardt
2021-01-11  8:37   ` [PATCH v7 8/8] config: allow specifying config entries via envvar pairs Patrick Steinhardt
2021-01-12 12:26 ` [PATCH v8 0/8] " Patrick Steinhardt
2021-01-12 12:26   ` [PATCH v8 1/8] git: add `--super-prefix` to usage string Patrick Steinhardt
2021-01-12 12:26   ` [PATCH v8 2/8] config: add new way to pass config via `--config-env` Patrick Steinhardt
2021-04-16 15:40     ` Ævar Arnfjörð Bjarmason
2021-04-17  8:38       ` Jeff King
2021-04-19 15:28         ` Patrick Steinhardt
2021-04-20 11:01           ` Ævar Arnfjörð Bjarmason
2021-04-20 10:59         ` Ævar Arnfjörð Bjarmason
2021-04-23 10:05           ` Jeff King
2021-05-19 11:36             ` Ævar Arnfjörð Bjarmason
2021-01-12 12:26   ` [PATCH v8 3/8] quote: make sq_dequote_step() a public function Patrick Steinhardt
2021-01-12 12:26   ` [PATCH v8 4/8] config: extract function to parse config pairs Patrick Steinhardt
2021-01-12 12:27   ` [PATCH v8 5/8] config: store "git -c" variables using more robust format Patrick Steinhardt
2021-01-15 19:16     ` Jeff King
2021-01-20  6:29       ` Patrick Steinhardt
2021-01-20  6:55         ` Junio C Hamano
2021-01-20  7:42           ` Patrick Steinhardt
2021-01-20 22:28             ` Junio C Hamano
2021-01-12 12:27   ` [PATCH v8 6/8] config: parse more robust format in GIT_CONFIG_PARAMETERS Patrick Steinhardt
2021-01-12 12:27   ` [PATCH v8 7/8] environment: make `getenv_safe()` a public function Patrick Steinhardt
2021-01-12 12:27   ` [PATCH v8 8/8] config: allow specifying config entries via envvar pairs Patrick Steinhardt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).