All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/1] remote-curl: unbreak http.extraHeader with custom allocators
@ 2019-11-05 21:59 Johannes Schindelin via GitGitGadget
  2019-11-05 21:59 ` [PATCH 1/1] " Johannes Schindelin via GitGitGadget
  2019-11-06 10:04 ` [PATCH v2 0/1] " Johannes Schindelin via GitGitGadget
  0 siblings, 2 replies; 17+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2019-11-05 21:59 UTC (permalink / raw)
  To: git; +Cc: Carlo Marcelo Arenas Belón, Johannes Schindelin, Junio C Hamano

This is one of those bugs that can cost entire days of work.

The symptom of this bug is that git -c http.extraheader push ... will fail
frequently in Git for Windows v2.24.0, but not always. When it fails,
though, it will cause a segmentation fault in git remote-https while calling 
http_cleanup() and leave no visual indication of that problem, the only
indication of a problem will be the exit code of the caller (in this
instance, git push will fail with exit code 1).

In my tests during the pre-release period, I pushed many a time, it probably
failed a lot, in the way indicated above, and due to the absence of any
error message, I failed to realize that there was a problem in the first
place.

Fun side note: this bug haunted me for the best part of yesterday, when I
tried to get Git for Windows v2.24.0 out the door. Large parts of Git for
Windows' release management are scripted, and that script failed, claiming
to have been unsuccessful in pushing the tag v2.24.0.windows.1, just after
printing a message to the extent that the tag is already up to date (except
in the first attempt, when it reported to have been successful in pushing
the tag). My attempts to fix the release script were doomed to fail because
the root cause was not a bug in the script, but the bug fixed in this patch.

Johannes Schindelin (1):
  remote-curl: unbreak http.extraHeader with custom allocators

 http.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)


base-commit: 93b980e58f5624ee4e3b2dc0d0babaa97ef66d19
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-453%2Fdscho%2Ffix-curl-xmalloc-regression-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-453/dscho/fix-curl-xmalloc-regression-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/453
-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-05 21:59 [PATCH 0/1] remote-curl: unbreak http.extraHeader with custom allocators Johannes Schindelin via GitGitGadget
@ 2019-11-05 21:59 ` Johannes Schindelin via GitGitGadget
  2019-11-06  4:16   ` Jeff King
  2019-11-06 10:04 ` [PATCH v2 0/1] " Johannes Schindelin via GitGitGadget
  1 sibling, 1 reply; 17+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2019-11-05 21:59 UTC (permalink / raw)
  To: git
  Cc: Carlo Marcelo Arenas Belón, Johannes Schindelin,
	Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to
ask cURL to use `xmalloc()`, and if compiled with nedmalloc, that means
implicitly a different allocator than the system one.

Which means that all of cURL's allocations and releases now _need_ to
use that allocator.

However, the `http_options()` function used `slist_append()` to add any
configured extra HTTP header(s) _before_ asking cURL to use `xmalloc()`,
and `http_cleanup()` would release them _afterwards_, i.e. in the
presence of custom allocators, cURL would attempt to use the wrong
allocator to release the memory.

Let's fix this by moving the initialization _before_ the
`http_options()` function is called.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 http.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/http.c b/http.c
index 27aa0a3192..13f50ba158 100644
--- a/http.c
+++ b/http.c
@@ -1062,6 +1062,9 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)
 	char *normalized_url;
 	struct urlmatch_config config = { STRING_LIST_INIT_DUP };
 
+	if (curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK)
+		die("curl_global_init failed");
+
 	config.section = "http";
 	config.key = NULL;
 	config.collect_fn = http_options;
@@ -1101,9 +1104,6 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)
 	}
 #endif
 
-	if (curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK)
-		die("curl_global_init failed");
-
 	http_proactive_auth = proactive_auth;
 
 	if (remote && remote->http_proxy)
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* Re: [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-05 21:59 ` [PATCH 1/1] " Johannes Schindelin via GitGitGadget
@ 2019-11-06  4:16   ` Jeff King
  2019-11-06  9:14     ` Johannes Schindelin
  0 siblings, 1 reply; 17+ messages in thread
From: Jeff King @ 2019-11-06  4:16 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget
  Cc: git, Carlo Marcelo Arenas Belón, Johannes Schindelin,
	Junio C Hamano

On Tue, Nov 05, 2019 at 09:59:18PM +0000, Johannes Schindelin via GitGitGadget wrote:

> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> 
> In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to
> ask cURL to use `xmalloc()`, and if compiled with nedmalloc, that means
> implicitly a different allocator than the system one.
> 
> Which means that all of cURL's allocations and releases now _need_ to
> use that allocator.
> 
> However, the `http_options()` function used `slist_append()` to add any
> configured extra HTTP header(s) _before_ asking cURL to use `xmalloc()`,
> and `http_cleanup()` would release them _afterwards_, i.e. in the
> presence of custom allocators, cURL would attempt to use the wrong
> allocator to release the memory.
> 
> Let's fix this by moving the initialization _before_ the
> `http_options()` function is called.

Nicely explained.

Another option would be to separate our config mechanism from curl
entirely by putting the list of headers into a string_list, and then
transforming it later into a curl_slist. I don't think that really buys
us much, though. This is all inside http.c, so it's fairly contained.
It's not like other random parts of Git are using curl's slist before
calling http_init().

I did briefly grep around for other slist users, but they're all what
you'd expect: code in http-push.c and remote-curl.c creating header
lists while working with an active http request (so well after
http_init() has been called).

> ---
>  http.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)

The patch itself looks good.

-Peff

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-06  4:16   ` Jeff King
@ 2019-11-06  9:14     ` Johannes Schindelin
  2019-11-06  9:49       ` Junio C Hamano
  0 siblings, 1 reply; 17+ messages in thread
From: Johannes Schindelin @ 2019-11-06  9:14 UTC (permalink / raw)
  To: Jeff King
  Cc: Johannes Schindelin via GitGitGadget, git,
	Carlo Marcelo Arenas Belón, Junio C Hamano

Hi Peff,

On Tue, 5 Nov 2019, Jeff King wrote:

> On Tue, Nov 05, 2019 at 09:59:18PM +0000, Johannes Schindelin via GitGitGadget wrote:
>
> > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> >
> > In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to
> > ask cURL to use `xmalloc()`, and if compiled with nedmalloc, that means
> > implicitly a different allocator than the system one.
> >
> > Which means that all of cURL's allocations and releases now _need_ to
> > use that allocator.
> >
> > However, the `http_options()` function used `slist_append()` to add any
> > configured extra HTTP header(s) _before_ asking cURL to use `xmalloc()`,
> > and `http_cleanup()` would release them _afterwards_, i.e. in the
> > presence of custom allocators, cURL would attempt to use the wrong
> > allocator to release the memory.
> >
> > Let's fix this by moving the initialization _before_ the
> > `http_options()` function is called.
>
> Nicely explained.
>
> Another option would be to separate our config mechanism from curl
> entirely by putting the list of headers into a string_list, and then
> transforming it later into a curl_slist. I don't think that really buys
> us much, though.

Alas, it _does_ buy us a lot, as I *just* found out (can you imagine how
glad I am not to have rushed out another Git for Windows release?). It
buys us one less bug: my patch introduces the bug where
`http.sslbackend` can no longer be used, because `curl_global_sslset()`
needs to be called _before_ `curl_global_init()`, but my patch breaks
that because we _need_ to parse the config before we can ask cURL for a
specific backend.

> This is all inside http.c, so it's fairly contained.  It's not like
> other random parts of Git are using curl's slist before calling
> http_init().

Indeed. We cannot use cURL's slist anywhere outside of the
cURL-dependent code because we want to keep `NO_CURL=Yep` working.

> I did briefly grep around for other slist users, but they're all what
> you'd expect: code in http-push.c and remote-curl.c creating header
> lists while working with an active http request (so well after
> http_init() has been called).

Indeed, I came to the same conclusion that Carlo's patch only broke
support for `http.extraheaders` (and only with custom allocators),
nothing else.

I will change the patch to avoid using `slist` early and send another
iteration.

Thanks,
Dscho

> > ---
> >  http.c | 6 +++---
> >  1 file changed, 3 insertions(+), 3 deletions(-)
>
> The patch itself looks good.
>
> -Peff
>

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-06  9:14     ` Johannes Schindelin
@ 2019-11-06  9:49       ` Junio C Hamano
  2019-11-06 19:38         ` Johannes Schindelin
  0 siblings, 1 reply; 17+ messages in thread
From: Junio C Hamano @ 2019-11-06  9:49 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Jeff King, Johannes Schindelin via GitGitGadget, git,
	Carlo Marcelo Arenas Belón

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Tue, 5 Nov 2019, Jeff King wrote:
>
>> ... transforming it later into a curl_slist. I don't think that really buys
>> us much, though.
>
> Alas, it _does_ buy us a lot, as I *just* found out (can you imagine how
> glad I am not to have rushed out another Git for Windows release?).

Thanks both, and especially thanks Dscho for both for a fix and for
restraining yourself from the urge to pull trigger too soon ;-)

> I will change the patch to avoid using `slist` early and send another
> iteration.

Will look forward to seeing it.

Thanks.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH v2 0/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-05 21:59 [PATCH 0/1] remote-curl: unbreak http.extraHeader with custom allocators Johannes Schindelin via GitGitGadget
  2019-11-05 21:59 ` [PATCH 1/1] " Johannes Schindelin via GitGitGadget
@ 2019-11-06 10:04 ` Johannes Schindelin via GitGitGadget
  2019-11-06 10:04   ` [PATCH v2 1/1] " Johannes Schindelin via GitGitGadget
  2019-11-06 12:15   ` [PATCH v2 0/1] " Junio C Hamano
  1 sibling, 2 replies; 17+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2019-11-06 10:04 UTC (permalink / raw)
  To: git; +Cc: Carlo Marcelo Arenas Belón, Johannes Schindelin, Junio C Hamano

This is one of those bugs that can cost entire days of work.

The symptom of this bug is that git -c http.extraheader push ... will fail
frequently in Git for Windows v2.24.0, but not always. When it fails,
though, it will cause a segmentation fault in git remote-https while calling 
http_cleanup() and leave no visual indication of that problem, the only
indication of a problem will be the exit code of the caller (in this
instance, git push will fail with exit code 1).

In my tests during the pre-release period, I pushed many a time, it probably
failed a lot, in the way indicated above, and due to the absence of any
error message, I failed to realize that there was a problem in the first
place.

Fun side note: this bug haunted me for the best part of this past Monday,
when I tried to get Git for Windows v2.24.0 out the door. Large parts of Git
for Windows' release management are scripted, and that script failed,
claiming to have been unsuccessful in pushing the tag v2.24.0.windows.1,
just after printing a message to the extent that the tag is already up to
date (except in the first attempt, when it reported to have been successful
in pushing the tag). My attempts to fix the release script were doomed to
fail because the root cause was not a bug in the script, but the bug fixed
in this patch.

Changes since v1:

 * The patch was completely redone: instead of moving the call to 
   curl_global_init() (which would have broken support for http.sslbackend),
   this iteration instead replaces the usage of cURL's slist in the config
   handling by using Git's own string_list.

Johannes Schindelin (1):
  remote-curl: unbreak http.extraHeader with custom allocators

 http.c | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)


base-commit: 93b980e58f5624ee4e3b2dc0d0babaa97ef66d19
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-453%2Fdscho%2Ffix-curl-xmalloc-regression-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-453/dscho/fix-curl-xmalloc-regression-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/453

Range-diff vs v1:

 1:  d47a2aa594 ! 1:  3168ba2c9e remote-curl: unbreak http.extraHeader with custom allocators
     @@ -15,8 +15,18 @@
          presence of custom allocators, cURL would attempt to use the wrong
          allocator to release the memory.
      
     -    Let's fix this by moving the initialization _before_ the
     -    `http_options()` function is called.
     +    A naïve attempt at fixing this would move the call to
     +    `curl_global_init()` _before_ the config is parsed (i.e. before that
     +    call to `slist_append()`).
     +
     +    However, that does work, as we _also_ parse the config setting
     +    `http.sslbackend` and if found, call `curl_global_sslset()` which *must*
     +    be called before `curl_global_init()`, for details see:
     +    https://curl.haxx.se/libcurl/c/curl_global_sslset.html
     +
     +    So let's instead make the config parsing entirely independent from
     +    cURL's data structures. Incidentally, this deletes two more lines than
     +    it introduces, which is nice.
      
          Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     @@ -24,22 +34,50 @@
       --- a/http.c
       +++ b/http.c
      @@
     - 	char *normalized_url;
     - 	struct urlmatch_config config = { STRING_LIST_INIT_DUP };
     - 
     -+	if (curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK)
     -+		die("curl_global_init failed");
     -+
     - 	config.section = "http";
     - 	config.key = NULL;
     - 	config.collect_fn = http_options;
     + 
     + static struct curl_slist *pragma_header;
     + static struct curl_slist *no_pragma_header;
     +-static struct curl_slist *extra_http_headers;
     ++static struct string_list extra_http_headers = STRING_LIST_INIT_DUP;
     + 
     + static struct active_request_slot *active_queue_head;
     + 
      @@
     + 		if (!value) {
     + 			return config_error_nonbool(var);
     + 		} else if (!*value) {
     +-			curl_slist_free_all(extra_http_headers);
     +-			extra_http_headers = NULL;
     ++			string_list_clear(&extra_http_headers, 0);
     + 		} else {
     +-			extra_http_headers =
     +-				curl_slist_append(extra_http_headers, value);
     ++			string_list_append(&extra_http_headers, value);
     + 		}
     + 		return 0;
       	}
     +@@
       #endif
     + 	curl_global_cleanup();
     + 
     +-	curl_slist_free_all(extra_http_headers);
     +-	extra_http_headers = NULL;
     ++	string_list_clear(&extra_http_headers, 0);
     + 
     + 	curl_slist_free_all(pragma_header);
     + 	pragma_header = NULL;
     +@@
     + 
     + struct curl_slist *http_copy_default_headers(void)
     + {
     +-	struct curl_slist *headers = NULL, *h;
     ++	struct curl_slist *headers = NULL;
     ++	const struct string_list_item *item;
       
     --	if (curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK)
     --		die("curl_global_init failed");
     --
     - 	http_proactive_auth = proactive_auth;
     +-	for (h = extra_http_headers; h; h = h->next)
     +-		headers = curl_slist_append(headers, h->data);
     ++	for_each_string_list_item(item, &extra_http_headers)
     ++		headers = curl_slist_append(headers, item->string);
       
     - 	if (remote && remote->http_proxy)
     + 	return headers;
     + }

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-06 10:04 ` [PATCH v2 0/1] " Johannes Schindelin via GitGitGadget
@ 2019-11-06 10:04   ` Johannes Schindelin via GitGitGadget
  2019-11-06 11:29     ` Jeff King
  2019-11-06 12:12     ` Junio C Hamano
  2019-11-06 12:15   ` [PATCH v2 0/1] " Junio C Hamano
  1 sibling, 2 replies; 17+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2019-11-06 10:04 UTC (permalink / raw)
  To: git
  Cc: Carlo Marcelo Arenas Belón, Johannes Schindelin,
	Junio C Hamano, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to
ask cURL to use `xmalloc()`, and if compiled with nedmalloc, that means
implicitly a different allocator than the system one.

Which means that all of cURL's allocations and releases now _need_ to
use that allocator.

However, the `http_options()` function used `slist_append()` to add any
configured extra HTTP header(s) _before_ asking cURL to use `xmalloc()`,
and `http_cleanup()` would release them _afterwards_, i.e. in the
presence of custom allocators, cURL would attempt to use the wrong
allocator to release the memory.

A naïve attempt at fixing this would move the call to
`curl_global_init()` _before_ the config is parsed (i.e. before that
call to `slist_append()`).

However, that does work, as we _also_ parse the config setting
`http.sslbackend` and if found, call `curl_global_sslset()` which *must*
be called before `curl_global_init()`, for details see:
https://curl.haxx.se/libcurl/c/curl_global_sslset.html

So let's instead make the config parsing entirely independent from
cURL's data structures. Incidentally, this deletes two more lines than
it introduces, which is nice.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 http.c | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/http.c b/http.c
index 27aa0a3192..82f493c7fd 100644
--- a/http.c
+++ b/http.c
@@ -150,7 +150,7 @@ static unsigned long empty_auth_useless =
 
 static struct curl_slist *pragma_header;
 static struct curl_slist *no_pragma_header;
-static struct curl_slist *extra_http_headers;
+static struct string_list extra_http_headers = STRING_LIST_INIT_DUP;
 
 static struct active_request_slot *active_queue_head;
 
@@ -414,11 +414,9 @@ static int http_options(const char *var, const char *value, void *cb)
 		if (!value) {
 			return config_error_nonbool(var);
 		} else if (!*value) {
-			curl_slist_free_all(extra_http_headers);
-			extra_http_headers = NULL;
+			string_list_clear(&extra_http_headers, 0);
 		} else {
-			extra_http_headers =
-				curl_slist_append(extra_http_headers, value);
+			string_list_append(&extra_http_headers, value);
 		}
 		return 0;
 	}
@@ -1199,8 +1197,7 @@ void http_cleanup(void)
 #endif
 	curl_global_cleanup();
 
-	curl_slist_free_all(extra_http_headers);
-	extra_http_headers = NULL;
+	string_list_clear(&extra_http_headers, 0);
 
 	curl_slist_free_all(pragma_header);
 	pragma_header = NULL;
@@ -1624,10 +1621,11 @@ int run_one_slot(struct active_request_slot *slot,
 
 struct curl_slist *http_copy_default_headers(void)
 {
-	struct curl_slist *headers = NULL, *h;
+	struct curl_slist *headers = NULL;
+	const struct string_list_item *item;
 
-	for (h = extra_http_headers; h; h = h->next)
-		headers = curl_slist_append(headers, h->data);
+	for_each_string_list_item(item, &extra_http_headers)
+		headers = curl_slist_append(headers, item->string);
 
 	return headers;
 }
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-06 10:04   ` [PATCH v2 1/1] " Johannes Schindelin via GitGitGadget
@ 2019-11-06 11:29     ` Jeff King
  2019-11-06 12:12     ` Junio C Hamano
  1 sibling, 0 replies; 17+ messages in thread
From: Jeff King @ 2019-11-06 11:29 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget
  Cc: git, Carlo Marcelo Arenas Belón, Johannes Schindelin,
	Junio C Hamano

On Wed, Nov 06, 2019 at 10:04:55AM +0000, Johannes Schindelin via GitGitGadget wrote:

> A naïve attempt at fixing this would move the call to
> `curl_global_init()` _before_ the config is parsed (i.e. before that
> call to `slist_append()`).
> 
> However, that does work, as we _also_ parse the config setting
> `http.sslbackend` and if found, call `curl_global_sslset()` which *must*
> be called before `curl_global_init()`, for details see:
> https://curl.haxx.se/libcurl/c/curl_global_sslset.html

Yikes, good catch. It didn't even occur to me that there might be curl
things we had to do _before_ calling curl_global_init().

> So let's instead make the config parsing entirely independent from
> cURL's data structures. Incidentally, this deletes two more lines than
> it introduces, which is nice.

Yes, I actually find the resulting code easier to read. I had feared
having to add an extra step to initialize the slist, but it's all
handled quite neatly in http_copy_default_headers().

>  http.c | 18 ++++++++----------
>  1 file changed, 8 insertions(+), 10 deletions(-)

The patch itself looks good to me.

-Peff

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-06 10:04   ` [PATCH v2 1/1] " Johannes Schindelin via GitGitGadget
  2019-11-06 11:29     ` Jeff King
@ 2019-11-06 12:12     ` Junio C Hamano
  2019-11-06 19:34       ` Johannes Schindelin
  1 sibling, 1 reply; 17+ messages in thread
From: Junio C Hamano @ 2019-11-06 12:12 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget
  Cc: git, Carlo Marcelo Arenas Belón, Johannes Schindelin

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:

> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to
> ask cURL to use `xmalloc()`, and if compiled with nedmalloc, that means
> implicitly a different allocator than the system one.
>
> Which means that all of cURL's allocations and releases now _need_ to
> use that allocator.
>
> However, the `http_options()` function used `slist_append()` to add any
> configured extra HTTP header(s) _before_ asking cURL to use `xmalloc()`,
> and `http_cleanup()` would release them _afterwards_, i.e. in the
> presence of custom allocators, cURL would attempt to use the wrong
> allocator to release the memory.

s/allocator/de&/; perhaps, even though it is clear enough from the
context, so it is probably OK as is.

> A naïve attempt at fixing this would move the call to
> `curl_global_init()` _before_ the config is parsed (i.e. before that
> call to `slist_append()`).
>
> However, that does work, as we _also_ parse the config setting

s/does work/does not work/; presumably?

> `http.sslbackend` and if found, call `curl_global_sslset()` which *must*
> be called before `curl_global_init()`, for details see:
> https://curl.haxx.se/libcurl/c/curl_global_sslset.html
>
> So let's instead make the config parsing entirely independent from
> cURL's data structures. Incidentally, this deletes two more lines than
> it introduces, which is nice.

Yeah, string_list_clear() is more concise than curl_slist_free_all(),
and we have already been copying one list to another anyway, so we
lucked out ;-)

The patch looked good to me, too.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 0/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-06 10:04 ` [PATCH v2 0/1] " Johannes Schindelin via GitGitGadget
  2019-11-06 10:04   ` [PATCH v2 1/1] " Johannes Schindelin via GitGitGadget
@ 2019-11-06 12:15   ` Junio C Hamano
  1 sibling, 0 replies; 17+ messages in thread
From: Junio C Hamano @ 2019-11-06 12:15 UTC (permalink / raw)
  To: Johannes Schindelin via GitGitGadget
  Cc: git, Carlo Marcelo Arenas Belón, Johannes Schindelin

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:

> Johannes Schindelin (1):
>   remote-curl: unbreak http.extraHeader with custom allocators
>
>  http.c | 18 ++++++++----------
>  1 file changed, 8 insertions(+), 10 deletions(-)
>
> base-commit: 93b980e58f5624ee4e3b2dc0d0babaa97ef66d19

This fixes by immediately building on top of the patch that caused
the issue, so I should do the same so that any integration branch
(either mine or somebody else's) that has the breakage can merge the
fix in.  Good.

Thanks.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-06 12:12     ` Junio C Hamano
@ 2019-11-06 19:34       ` Johannes Schindelin
  2019-11-07  5:39         ` Junio C Hamano
  0 siblings, 1 reply; 17+ messages in thread
From: Johannes Schindelin @ 2019-11-06 19:34 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Johannes Schindelin via GitGitGadget, git,
	Carlo Marcelo Arenas Belón

[-- Attachment #1: Type: text/plain, Size: 1981 bytes --]

Hi Junio,

On Wed, 6 Nov 2019, Junio C Hamano wrote:

> "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
> writes:
>
> > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> >
> > In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to
> > ask cURL to use `xmalloc()`, and if compiled with nedmalloc, that means
> > implicitly a different allocator than the system one.
> >
> > Which means that all of cURL's allocations and releases now _need_ to
> > use that allocator.
> >
> > However, the `http_options()` function used `slist_append()` to add any
> > configured extra HTTP header(s) _before_ asking cURL to use `xmalloc()`,
> > and `http_cleanup()` would release them _afterwards_, i.e. in the
> > presence of custom allocators, cURL would attempt to use the wrong
> > allocator to release the memory.
>
> s/allocator/de&/; perhaps, even though it is clear enough from the
> context, so it is probably OK as is.
>
> > A naïve attempt at fixing this would move the call to
> > `curl_global_init()` _before_ the config is parsed (i.e. before that
> > call to `slist_append()`).
> >
> > However, that does work, as we _also_ parse the config setting
>
> s/does work/does not work/; presumably?

Indeed. Could I ask you to fix up locally, or do you want me to send a
new revision of the patch?

Ciao,
Dscho

>
> > `http.sslbackend` and if found, call `curl_global_sslset()` which *must*
> > be called before `curl_global_init()`, for details see:
> > https://curl.haxx.se/libcurl/c/curl_global_sslset.html
> >
> > So let's instead make the config parsing entirely independent from
> > cURL's data structures. Incidentally, this deletes two more lines than
> > it introduces, which is nice.
>
> Yeah, string_list_clear() is more concise than curl_slist_free_all(),
> and we have already been copying one list to another anyway, so we
> lucked out ;-)
>
> The patch looked good to me, too.
>

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-06  9:49       ` Junio C Hamano
@ 2019-11-06 19:38         ` Johannes Schindelin
  2019-11-08  8:34           ` Junio C Hamano
  0 siblings, 1 reply; 17+ messages in thread
From: Johannes Schindelin @ 2019-11-06 19:38 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Jeff King, Johannes Schindelin via GitGitGadget, git,
	Carlo Marcelo Arenas Belón

Hi Junio,

On Wed, 6 Nov 2019, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > On Tue, 5 Nov 2019, Jeff King wrote:
> >
> >> ... transforming it later into a curl_slist. I don't think that really buys
> >> us much, though.
> >
> > Alas, it _does_ buy us a lot, as I *just* found out (can you imagine how
> > glad I am not to have rushed out another Git for Windows release?).
>
> Thanks both, and especially thanks Dscho for both for a fix and for
> restraining yourself from the urge to pull trigger too soon ;-)

Well, I _will_ trigger the pipeline publishing Git for Windows
v2.24.0(2) in a moment, with this fix, but more importantly with a fix
for the installer that tries to install some Cygwin-style symlinks,
which was broken in non-US locales (most notably in a Japanese setup;
apparently there _still_ are developers who did not yet move away from a
Japanese locale... at least on Windows).

Ciao,
Dscho

>
> > I will change the patch to avoid using `slist` early and send another
> > iteration.
>
> Will look forward to seeing it.
>
> Thanks.
>

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-06 19:34       ` Johannes Schindelin
@ 2019-11-07  5:39         ` Junio C Hamano
  2019-11-07 12:40           ` Johannes Schindelin
  0 siblings, 1 reply; 17+ messages in thread
From: Junio C Hamano @ 2019-11-07  5:39 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Johannes Schindelin via GitGitGadget, git,
	Carlo Marcelo Arenas Belón

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Wed, 6 Nov 2019, Junio C Hamano wrote:
>
>> > presence of custom allocators, cURL would attempt to use the wrong
>> > allocator to release the memory.
>>
>> s/allocator/de&/; perhaps, even though it is clear enough from the
>> context, so it is probably OK as is.
>>
>> > A naïve attempt at fixing this would move the call to
>> > `curl_global_init()` _before_ the config is parsed (i.e. before that
>> > call to `slist_append()`).
>> >
>> > However, that does work, as we _also_ parse the config setting
>>
>> s/does work/does not work/; presumably?
>
> Indeed. Could I ask you to fix up locally, or do you want me to send a
> new revision of the patch?

You can certainly tell me to locally tweak, but you'd need to be
more specific when some of my suggestions were "perhaps" (aka "you
could do it this way, which may be better, but I do not care too
deeply---I am OK with whichever you chooose after you (re)think
about it, but I am not choosing for you").

For now, I'll do the "does not work" thing only.

Thanks.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-07  5:39         ` Junio C Hamano
@ 2019-11-07 12:40           ` Johannes Schindelin
  2019-11-08  3:03             ` Junio C Hamano
  0 siblings, 1 reply; 17+ messages in thread
From: Johannes Schindelin @ 2019-11-07 12:40 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Johannes Schindelin via GitGitGadget, git,
	Carlo Marcelo Arenas Belón

[-- Attachment #1: Type: text/plain, Size: 1496 bytes --]

Hi Junio,

On Thu, 7 Nov 2019, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > On Wed, 6 Nov 2019, Junio C Hamano wrote:
> >
> >> > presence of custom allocators, cURL would attempt to use the wrong
> >> > allocator to release the memory.
> >>
> >> s/allocator/de&/; perhaps, even though it is clear enough from the
> >> context, so it is probably OK as is.
> >>
> >> > A naïve attempt at fixing this would move the call to
> >> > `curl_global_init()` _before_ the config is parsed (i.e. before that
> >> > call to `slist_append()`).
> >> >
> >> > However, that does work, as we _also_ parse the config setting
> >>
> >> s/does work/does not work/; presumably?
> >
> > Indeed. Could I ask you to fix up locally, or do you want me to send a
> > new revision of the patch?
>
> You can certainly tell me to locally tweak, but you'd need to be
> more specific when some of my suggestions were "perhaps" (aka "you
> could do it this way, which may be better, but I do not care too
> deeply---I am OK with whichever you chooose after you (re)think
> about it, but I am not choosing for you").

Right, I should have been more specific.

> For now, I'll do the "does not work" thing only.

Thanks!

FWIW I would like to stick with "custom allocator" because even
releasing the memory is the duty of an "allocator", even if that
allocator happens to "deallocate" at that point what it has allocated
before.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-07 12:40           ` Johannes Schindelin
@ 2019-11-08  3:03             ` Junio C Hamano
  0 siblings, 0 replies; 17+ messages in thread
From: Junio C Hamano @ 2019-11-08  3:03 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Johannes Schindelin via GitGitGadget, git,
	Carlo Marcelo Arenas Belón

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> FWIW I would like to stick with "custom allocator" because even
> releasing the memory is the duty of an "allocator", even if that
> allocator happens to "deallocate" at that point what it has allocated
> before.

My preference is to think of the set of functions we feed
curl-global-init-mem as a (allocator, deallocator, reallocator, ...)
tuple.

But you are calling that set itself as an allocator that has service
functions like (alloc, free, realloc), and as long as that (i.e. how
you defined the word "allocator") is clear to the readers that is
perfectly OK.  

To me as one reader, it was not clear and that was where my comment
came from.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-06 19:38         ` Johannes Schindelin
@ 2019-11-08  8:34           ` Junio C Hamano
  2019-11-08 13:44             ` Johannes Schindelin
  0 siblings, 1 reply; 17+ messages in thread
From: Junio C Hamano @ 2019-11-08  8:34 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Jeff King, Johannes Schindelin via GitGitGadget, git,
	Carlo Marcelo Arenas Belón

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> ... which was broken in non-US locales (most notably in a Japanese setup;
> apparently there _still_ are developers who did not yet move away from a
> Japanese locale... at least on Windows).

I think the above was a reference to a tangential comment I made in
a response to Peff.

Oh, I do not expect Japanese Windows users would "move away from"
Japanese locale, ever.

I do however know that an app that supports only legacy encoding is
frowned upon these days.  They still use and will keep using
Japanese menus and messages, and they still write their documents in
Japanese and not in US English.

But they store their Japenese documents encoded in UTF-8 on their
system that is in Japanese locale.

There are two models of gadgets I am interested in getting, between
which one of them that is slightly older supports UTF-8 and MS-Kanji
(aka Shift-JIS) while the latest model only supports MS-Kanji.  The
list price of them are comparable (actually, the latest one lists a
bit more), but the latest model is deeply discounted while the other
one with UTF-8 is not as much.  At shopping sites, user reviews
often mention "I've migrated my text to UTF-8 already and going back
to Shift JIS in this year is too cumbersome, so I'll skip this
latest model".

I am hoping a software update might happen, and will pull the
trigger once the latest one starts supporting UTF-8 ;-)

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators
  2019-11-08  8:34           ` Junio C Hamano
@ 2019-11-08 13:44             ` Johannes Schindelin
  0 siblings, 0 replies; 17+ messages in thread
From: Johannes Schindelin @ 2019-11-08 13:44 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Jeff King, Johannes Schindelin via GitGitGadget, git,
	Carlo Marcelo Arenas Belón

Hi Junio,

On Fri, 8 Nov 2019, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > ... which was broken in non-US locales (most notably in a Japanese setup;
> > apparently there _still_ are developers who did not yet move away from a
> > Japanese locale... at least on Windows).
>
> I think the above was a reference to a tangential comment I made in
> a response to Peff.
>
> Oh, I do not expect Japanese Windows users would "move away from"
> Japanese locale, ever.
>
> I do however know that an app that supports only legacy encoding is
> frowned upon these days.  They still use and will keep using
> Japanese menus and messages, and they still write their documents in
> Japanese and not in US English.
>
> But they store their Japenese documents encoded in UTF-8 on their
> system that is in Japanese locale.
>
> There are two models of gadgets I am interested in getting, between
> which one of them that is slightly older supports UTF-8 and MS-Kanji
> (aka Shift-JIS) while the latest model only supports MS-Kanji.  The
> list price of them are comparable (actually, the latest one lists a
> bit more), but the latest model is deeply discounted while the other
> one with UTF-8 is not as much.  At shopping sites, user reviews
> often mention "I've migrated my text to UTF-8 already and going back
> to Shift JIS in this year is too cumbersome, so I'll skip this
> latest model".
>
> I am hoping a software update might happen, and will pull the
> trigger once the latest one starts supporting UTF-8 ;-)

Thank you for that interesting note.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2019-11-08 13:44 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-05 21:59 [PATCH 0/1] remote-curl: unbreak http.extraHeader with custom allocators Johannes Schindelin via GitGitGadget
2019-11-05 21:59 ` [PATCH 1/1] " Johannes Schindelin via GitGitGadget
2019-11-06  4:16   ` Jeff King
2019-11-06  9:14     ` Johannes Schindelin
2019-11-06  9:49       ` Junio C Hamano
2019-11-06 19:38         ` Johannes Schindelin
2019-11-08  8:34           ` Junio C Hamano
2019-11-08 13:44             ` Johannes Schindelin
2019-11-06 10:04 ` [PATCH v2 0/1] " Johannes Schindelin via GitGitGadget
2019-11-06 10:04   ` [PATCH v2 1/1] " Johannes Schindelin via GitGitGadget
2019-11-06 11:29     ` Jeff King
2019-11-06 12:12     ` Junio C Hamano
2019-11-06 19:34       ` Johannes Schindelin
2019-11-07  5:39         ` Junio C Hamano
2019-11-07 12:40           ` Johannes Schindelin
2019-11-08  3:03             ` Junio C Hamano
2019-11-06 12:15   ` [PATCH v2 0/1] " Junio C Hamano

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.