* [PATCH 0/1] remote-curl: unbreak http.extraHeader with custom allocators @ 2019-11-05 21:59 Johannes Schindelin via GitGitGadget 2019-11-05 21:59 ` [PATCH 1/1] " Johannes Schindelin via GitGitGadget 2019-11-06 10:04 ` [PATCH v2 0/1] " Johannes Schindelin via GitGitGadget 0 siblings, 2 replies; 17+ messages in thread From: Johannes Schindelin via GitGitGadget @ 2019-11-05 21:59 UTC (permalink / raw) To: git; +Cc: Carlo Marcelo Arenas Belón, Johannes Schindelin, Junio C Hamano This is one of those bugs that can cost entire days of work. The symptom of this bug is that git -c http.extraheader push ... will fail frequently in Git for Windows v2.24.0, but not always. When it fails, though, it will cause a segmentation fault in git remote-https while calling http_cleanup() and leave no visual indication of that problem, the only indication of a problem will be the exit code of the caller (in this instance, git push will fail with exit code 1). In my tests during the pre-release period, I pushed many a time, it probably failed a lot, in the way indicated above, and due to the absence of any error message, I failed to realize that there was a problem in the first place. Fun side note: this bug haunted me for the best part of yesterday, when I tried to get Git for Windows v2.24.0 out the door. Large parts of Git for Windows' release management are scripted, and that script failed, claiming to have been unsuccessful in pushing the tag v2.24.0.windows.1, just after printing a message to the extent that the tag is already up to date (except in the first attempt, when it reported to have been successful in pushing the tag). My attempts to fix the release script were doomed to fail because the root cause was not a bug in the script, but the bug fixed in this patch. Johannes Schindelin (1): remote-curl: unbreak http.extraHeader with custom allocators http.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) base-commit: 93b980e58f5624ee4e3b2dc0d0babaa97ef66d19 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-453%2Fdscho%2Ffix-curl-xmalloc-regression-v1 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-453/dscho/fix-curl-xmalloc-regression-v1 Pull-Request: https://github.com/gitgitgadget/git/pull/453 -- gitgitgadget ^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-05 21:59 [PATCH 0/1] remote-curl: unbreak http.extraHeader with custom allocators Johannes Schindelin via GitGitGadget @ 2019-11-05 21:59 ` Johannes Schindelin via GitGitGadget 2019-11-06 4:16 ` Jeff King 2019-11-06 10:04 ` [PATCH v2 0/1] " Johannes Schindelin via GitGitGadget 1 sibling, 1 reply; 17+ messages in thread From: Johannes Schindelin via GitGitGadget @ 2019-11-05 21:59 UTC (permalink / raw) To: git Cc: Carlo Marcelo Arenas Belón, Johannes Schindelin, Junio C Hamano, Johannes Schindelin From: Johannes Schindelin <johannes.schindelin@gmx.de> In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to ask cURL to use `xmalloc()`, and if compiled with nedmalloc, that means implicitly a different allocator than the system one. Which means that all of cURL's allocations and releases now _need_ to use that allocator. However, the `http_options()` function used `slist_append()` to add any configured extra HTTP header(s) _before_ asking cURL to use `xmalloc()`, and `http_cleanup()` would release them _afterwards_, i.e. in the presence of custom allocators, cURL would attempt to use the wrong allocator to release the memory. Let's fix this by moving the initialization _before_ the `http_options()` function is called. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> --- http.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/http.c b/http.c index 27aa0a3192..13f50ba158 100644 --- a/http.c +++ b/http.c @@ -1062,6 +1062,9 @@ void http_init(struct remote *remote, const char *url, int proactive_auth) char *normalized_url; struct urlmatch_config config = { STRING_LIST_INIT_DUP }; + if (curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK) + die("curl_global_init failed"); + config.section = "http"; config.key = NULL; config.collect_fn = http_options; @@ -1101,9 +1104,6 @@ void http_init(struct remote *remote, const char *url, int proactive_auth) } #endif - if (curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK) - die("curl_global_init failed"); - http_proactive_auth = proactive_auth; if (remote && remote->http_proxy) -- gitgitgadget ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-05 21:59 ` [PATCH 1/1] " Johannes Schindelin via GitGitGadget @ 2019-11-06 4:16 ` Jeff King 2019-11-06 9:14 ` Johannes Schindelin 0 siblings, 1 reply; 17+ messages in thread From: Jeff King @ 2019-11-06 4:16 UTC (permalink / raw) To: Johannes Schindelin via GitGitGadget Cc: git, Carlo Marcelo Arenas Belón, Johannes Schindelin, Junio C Hamano On Tue, Nov 05, 2019 at 09:59:18PM +0000, Johannes Schindelin via GitGitGadget wrote: > From: Johannes Schindelin <johannes.schindelin@gmx.de> > > In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to > ask cURL to use `xmalloc()`, and if compiled with nedmalloc, that means > implicitly a different allocator than the system one. > > Which means that all of cURL's allocations and releases now _need_ to > use that allocator. > > However, the `http_options()` function used `slist_append()` to add any > configured extra HTTP header(s) _before_ asking cURL to use `xmalloc()`, > and `http_cleanup()` would release them _afterwards_, i.e. in the > presence of custom allocators, cURL would attempt to use the wrong > allocator to release the memory. > > Let's fix this by moving the initialization _before_ the > `http_options()` function is called. Nicely explained. Another option would be to separate our config mechanism from curl entirely by putting the list of headers into a string_list, and then transforming it later into a curl_slist. I don't think that really buys us much, though. This is all inside http.c, so it's fairly contained. It's not like other random parts of Git are using curl's slist before calling http_init(). I did briefly grep around for other slist users, but they're all what you'd expect: code in http-push.c and remote-curl.c creating header lists while working with an active http request (so well after http_init() has been called). > --- > http.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) The patch itself looks good. -Peff ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-06 4:16 ` Jeff King @ 2019-11-06 9:14 ` Johannes Schindelin 2019-11-06 9:49 ` Junio C Hamano 0 siblings, 1 reply; 17+ messages in thread From: Johannes Schindelin @ 2019-11-06 9:14 UTC (permalink / raw) To: Jeff King Cc: Johannes Schindelin via GitGitGadget, git, Carlo Marcelo Arenas Belón, Junio C Hamano Hi Peff, On Tue, 5 Nov 2019, Jeff King wrote: > On Tue, Nov 05, 2019 at 09:59:18PM +0000, Johannes Schindelin via GitGitGadget wrote: > > > From: Johannes Schindelin <johannes.schindelin@gmx.de> > > > > In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to > > ask cURL to use `xmalloc()`, and if compiled with nedmalloc, that means > > implicitly a different allocator than the system one. > > > > Which means that all of cURL's allocations and releases now _need_ to > > use that allocator. > > > > However, the `http_options()` function used `slist_append()` to add any > > configured extra HTTP header(s) _before_ asking cURL to use `xmalloc()`, > > and `http_cleanup()` would release them _afterwards_, i.e. in the > > presence of custom allocators, cURL would attempt to use the wrong > > allocator to release the memory. > > > > Let's fix this by moving the initialization _before_ the > > `http_options()` function is called. > > Nicely explained. > > Another option would be to separate our config mechanism from curl > entirely by putting the list of headers into a string_list, and then > transforming it later into a curl_slist. I don't think that really buys > us much, though. Alas, it _does_ buy us a lot, as I *just* found out (can you imagine how glad I am not to have rushed out another Git for Windows release?). It buys us one less bug: my patch introduces the bug where `http.sslbackend` can no longer be used, because `curl_global_sslset()` needs to be called _before_ `curl_global_init()`, but my patch breaks that because we _need_ to parse the config before we can ask cURL for a specific backend. > This is all inside http.c, so it's fairly contained. It's not like > other random parts of Git are using curl's slist before calling > http_init(). Indeed. We cannot use cURL's slist anywhere outside of the cURL-dependent code because we want to keep `NO_CURL=Yep` working. > I did briefly grep around for other slist users, but they're all what > you'd expect: code in http-push.c and remote-curl.c creating header > lists while working with an active http request (so well after > http_init() has been called). Indeed, I came to the same conclusion that Carlo's patch only broke support for `http.extraheaders` (and only with custom allocators), nothing else. I will change the patch to avoid using `slist` early and send another iteration. Thanks, Dscho > > --- > > http.c | 6 +++--- > > 1 file changed, 3 insertions(+), 3 deletions(-) > > The patch itself looks good. > > -Peff > ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-06 9:14 ` Johannes Schindelin @ 2019-11-06 9:49 ` Junio C Hamano 2019-11-06 19:38 ` Johannes Schindelin 0 siblings, 1 reply; 17+ messages in thread From: Junio C Hamano @ 2019-11-06 9:49 UTC (permalink / raw) To: Johannes Schindelin Cc: Jeff King, Johannes Schindelin via GitGitGadget, git, Carlo Marcelo Arenas Belón Johannes Schindelin <Johannes.Schindelin@gmx.de> writes: > On Tue, 5 Nov 2019, Jeff King wrote: > >> ... transforming it later into a curl_slist. I don't think that really buys >> us much, though. > > Alas, it _does_ buy us a lot, as I *just* found out (can you imagine how > glad I am not to have rushed out another Git for Windows release?). Thanks both, and especially thanks Dscho for both for a fix and for restraining yourself from the urge to pull trigger too soon ;-) > I will change the patch to avoid using `slist` early and send another > iteration. Will look forward to seeing it. Thanks. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-06 9:49 ` Junio C Hamano @ 2019-11-06 19:38 ` Johannes Schindelin 2019-11-08 8:34 ` Junio C Hamano 0 siblings, 1 reply; 17+ messages in thread From: Johannes Schindelin @ 2019-11-06 19:38 UTC (permalink / raw) To: Junio C Hamano Cc: Jeff King, Johannes Schindelin via GitGitGadget, git, Carlo Marcelo Arenas Belón Hi Junio, On Wed, 6 Nov 2019, Junio C Hamano wrote: > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes: > > > On Tue, 5 Nov 2019, Jeff King wrote: > > > >> ... transforming it later into a curl_slist. I don't think that really buys > >> us much, though. > > > > Alas, it _does_ buy us a lot, as I *just* found out (can you imagine how > > glad I am not to have rushed out another Git for Windows release?). > > Thanks both, and especially thanks Dscho for both for a fix and for > restraining yourself from the urge to pull trigger too soon ;-) Well, I _will_ trigger the pipeline publishing Git for Windows v2.24.0(2) in a moment, with this fix, but more importantly with a fix for the installer that tries to install some Cygwin-style symlinks, which was broken in non-US locales (most notably in a Japanese setup; apparently there _still_ are developers who did not yet move away from a Japanese locale... at least on Windows). Ciao, Dscho > > > I will change the patch to avoid using `slist` early and send another > > iteration. > > Will look forward to seeing it. > > Thanks. > ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-06 19:38 ` Johannes Schindelin @ 2019-11-08 8:34 ` Junio C Hamano 2019-11-08 13:44 ` Johannes Schindelin 0 siblings, 1 reply; 17+ messages in thread From: Junio C Hamano @ 2019-11-08 8:34 UTC (permalink / raw) To: Johannes Schindelin Cc: Jeff King, Johannes Schindelin via GitGitGadget, git, Carlo Marcelo Arenas Belón Johannes Schindelin <Johannes.Schindelin@gmx.de> writes: > ... which was broken in non-US locales (most notably in a Japanese setup; > apparently there _still_ are developers who did not yet move away from a > Japanese locale... at least on Windows). I think the above was a reference to a tangential comment I made in a response to Peff. Oh, I do not expect Japanese Windows users would "move away from" Japanese locale, ever. I do however know that an app that supports only legacy encoding is frowned upon these days. They still use and will keep using Japanese menus and messages, and they still write their documents in Japanese and not in US English. But they store their Japenese documents encoded in UTF-8 on their system that is in Japanese locale. There are two models of gadgets I am interested in getting, between which one of them that is slightly older supports UTF-8 and MS-Kanji (aka Shift-JIS) while the latest model only supports MS-Kanji. The list price of them are comparable (actually, the latest one lists a bit more), but the latest model is deeply discounted while the other one with UTF-8 is not as much. At shopping sites, user reviews often mention "I've migrated my text to UTF-8 already and going back to Shift JIS in this year is too cumbersome, so I'll skip this latest model". I am hoping a software update might happen, and will pull the trigger once the latest one starts supporting UTF-8 ;-) ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-08 8:34 ` Junio C Hamano @ 2019-11-08 13:44 ` Johannes Schindelin 0 siblings, 0 replies; 17+ messages in thread From: Johannes Schindelin @ 2019-11-08 13:44 UTC (permalink / raw) To: Junio C Hamano Cc: Jeff King, Johannes Schindelin via GitGitGadget, git, Carlo Marcelo Arenas Belón Hi Junio, On Fri, 8 Nov 2019, Junio C Hamano wrote: > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes: > > > ... which was broken in non-US locales (most notably in a Japanese setup; > > apparently there _still_ are developers who did not yet move away from a > > Japanese locale... at least on Windows). > > I think the above was a reference to a tangential comment I made in > a response to Peff. > > Oh, I do not expect Japanese Windows users would "move away from" > Japanese locale, ever. > > I do however know that an app that supports only legacy encoding is > frowned upon these days. They still use and will keep using > Japanese menus and messages, and they still write their documents in > Japanese and not in US English. > > But they store their Japenese documents encoded in UTF-8 on their > system that is in Japanese locale. > > There are two models of gadgets I am interested in getting, between > which one of them that is slightly older supports UTF-8 and MS-Kanji > (aka Shift-JIS) while the latest model only supports MS-Kanji. The > list price of them are comparable (actually, the latest one lists a > bit more), but the latest model is deeply discounted while the other > one with UTF-8 is not as much. At shopping sites, user reviews > often mention "I've migrated my text to UTF-8 already and going back > to Shift JIS in this year is too cumbersome, so I'll skip this > latest model". > > I am hoping a software update might happen, and will pull the > trigger once the latest one starts supporting UTF-8 ;-) Thank you for that interesting note. Ciao, Dscho ^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v2 0/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-05 21:59 [PATCH 0/1] remote-curl: unbreak http.extraHeader with custom allocators Johannes Schindelin via GitGitGadget 2019-11-05 21:59 ` [PATCH 1/1] " Johannes Schindelin via GitGitGadget @ 2019-11-06 10:04 ` Johannes Schindelin via GitGitGadget 2019-11-06 10:04 ` [PATCH v2 1/1] " Johannes Schindelin via GitGitGadget 2019-11-06 12:15 ` [PATCH v2 0/1] " Junio C Hamano 1 sibling, 2 replies; 17+ messages in thread From: Johannes Schindelin via GitGitGadget @ 2019-11-06 10:04 UTC (permalink / raw) To: git; +Cc: Carlo Marcelo Arenas Belón, Johannes Schindelin, Junio C Hamano This is one of those bugs that can cost entire days of work. The symptom of this bug is that git -c http.extraheader push ... will fail frequently in Git for Windows v2.24.0, but not always. When it fails, though, it will cause a segmentation fault in git remote-https while calling http_cleanup() and leave no visual indication of that problem, the only indication of a problem will be the exit code of the caller (in this instance, git push will fail with exit code 1). In my tests during the pre-release period, I pushed many a time, it probably failed a lot, in the way indicated above, and due to the absence of any error message, I failed to realize that there was a problem in the first place. Fun side note: this bug haunted me for the best part of this past Monday, when I tried to get Git for Windows v2.24.0 out the door. Large parts of Git for Windows' release management are scripted, and that script failed, claiming to have been unsuccessful in pushing the tag v2.24.0.windows.1, just after printing a message to the extent that the tag is already up to date (except in the first attempt, when it reported to have been successful in pushing the tag). My attempts to fix the release script were doomed to fail because the root cause was not a bug in the script, but the bug fixed in this patch. Changes since v1: * The patch was completely redone: instead of moving the call to curl_global_init() (which would have broken support for http.sslbackend), this iteration instead replaces the usage of cURL's slist in the config handling by using Git's own string_list. Johannes Schindelin (1): remote-curl: unbreak http.extraHeader with custom allocators http.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) base-commit: 93b980e58f5624ee4e3b2dc0d0babaa97ef66d19 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-453%2Fdscho%2Ffix-curl-xmalloc-regression-v2 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-453/dscho/fix-curl-xmalloc-regression-v2 Pull-Request: https://github.com/gitgitgadget/git/pull/453 Range-diff vs v1: 1: d47a2aa594 ! 1: 3168ba2c9e remote-curl: unbreak http.extraHeader with custom allocators @@ -15,8 +15,18 @@ presence of custom allocators, cURL would attempt to use the wrong allocator to release the memory. - Let's fix this by moving the initialization _before_ the - `http_options()` function is called. + A naïve attempt at fixing this would move the call to + `curl_global_init()` _before_ the config is parsed (i.e. before that + call to `slist_append()`). + + However, that does work, as we _also_ parse the config setting + `http.sslbackend` and if found, call `curl_global_sslset()` which *must* + be called before `curl_global_init()`, for details see: + https://curl.haxx.se/libcurl/c/curl_global_sslset.html + + So let's instead make the config parsing entirely independent from + cURL's data structures. Incidentally, this deletes two more lines than + it introduces, which is nice. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> @@ -24,22 +34,50 @@ --- a/http.c +++ b/http.c @@ - char *normalized_url; - struct urlmatch_config config = { STRING_LIST_INIT_DUP }; - -+ if (curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK) -+ die("curl_global_init failed"); -+ - config.section = "http"; - config.key = NULL; - config.collect_fn = http_options; + + static struct curl_slist *pragma_header; + static struct curl_slist *no_pragma_header; +-static struct curl_slist *extra_http_headers; ++static struct string_list extra_http_headers = STRING_LIST_INIT_DUP; + + static struct active_request_slot *active_queue_head; + @@ + if (!value) { + return config_error_nonbool(var); + } else if (!*value) { +- curl_slist_free_all(extra_http_headers); +- extra_http_headers = NULL; ++ string_list_clear(&extra_http_headers, 0); + } else { +- extra_http_headers = +- curl_slist_append(extra_http_headers, value); ++ string_list_append(&extra_http_headers, value); + } + return 0; } +@@ #endif + curl_global_cleanup(); + +- curl_slist_free_all(extra_http_headers); +- extra_http_headers = NULL; ++ string_list_clear(&extra_http_headers, 0); + + curl_slist_free_all(pragma_header); + pragma_header = NULL; +@@ + + struct curl_slist *http_copy_default_headers(void) + { +- struct curl_slist *headers = NULL, *h; ++ struct curl_slist *headers = NULL; ++ const struct string_list_item *item; -- if (curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK) -- die("curl_global_init failed"); -- - http_proactive_auth = proactive_auth; +- for (h = extra_http_headers; h; h = h->next) +- headers = curl_slist_append(headers, h->data); ++ for_each_string_list_item(item, &extra_http_headers) ++ headers = curl_slist_append(headers, item->string); - if (remote && remote->http_proxy) + return headers; + } -- gitgitgadget ^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-06 10:04 ` [PATCH v2 0/1] " Johannes Schindelin via GitGitGadget @ 2019-11-06 10:04 ` Johannes Schindelin via GitGitGadget 2019-11-06 11:29 ` Jeff King 2019-11-06 12:12 ` Junio C Hamano 2019-11-06 12:15 ` [PATCH v2 0/1] " Junio C Hamano 1 sibling, 2 replies; 17+ messages in thread From: Johannes Schindelin via GitGitGadget @ 2019-11-06 10:04 UTC (permalink / raw) To: git Cc: Carlo Marcelo Arenas Belón, Johannes Schindelin, Junio C Hamano, Johannes Schindelin From: Johannes Schindelin <johannes.schindelin@gmx.de> In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to ask cURL to use `xmalloc()`, and if compiled with nedmalloc, that means implicitly a different allocator than the system one. Which means that all of cURL's allocations and releases now _need_ to use that allocator. However, the `http_options()` function used `slist_append()` to add any configured extra HTTP header(s) _before_ asking cURL to use `xmalloc()`, and `http_cleanup()` would release them _afterwards_, i.e. in the presence of custom allocators, cURL would attempt to use the wrong allocator to release the memory. A naïve attempt at fixing this would move the call to `curl_global_init()` _before_ the config is parsed (i.e. before that call to `slist_append()`). However, that does work, as we _also_ parse the config setting `http.sslbackend` and if found, call `curl_global_sslset()` which *must* be called before `curl_global_init()`, for details see: https://curl.haxx.se/libcurl/c/curl_global_sslset.html So let's instead make the config parsing entirely independent from cURL's data structures. Incidentally, this deletes two more lines than it introduces, which is nice. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> --- http.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/http.c b/http.c index 27aa0a3192..82f493c7fd 100644 --- a/http.c +++ b/http.c @@ -150,7 +150,7 @@ static unsigned long empty_auth_useless = static struct curl_slist *pragma_header; static struct curl_slist *no_pragma_header; -static struct curl_slist *extra_http_headers; +static struct string_list extra_http_headers = STRING_LIST_INIT_DUP; static struct active_request_slot *active_queue_head; @@ -414,11 +414,9 @@ static int http_options(const char *var, const char *value, void *cb) if (!value) { return config_error_nonbool(var); } else if (!*value) { - curl_slist_free_all(extra_http_headers); - extra_http_headers = NULL; + string_list_clear(&extra_http_headers, 0); } else { - extra_http_headers = - curl_slist_append(extra_http_headers, value); + string_list_append(&extra_http_headers, value); } return 0; } @@ -1199,8 +1197,7 @@ void http_cleanup(void) #endif curl_global_cleanup(); - curl_slist_free_all(extra_http_headers); - extra_http_headers = NULL; + string_list_clear(&extra_http_headers, 0); curl_slist_free_all(pragma_header); pragma_header = NULL; @@ -1624,10 +1621,11 @@ int run_one_slot(struct active_request_slot *slot, struct curl_slist *http_copy_default_headers(void) { - struct curl_slist *headers = NULL, *h; + struct curl_slist *headers = NULL; + const struct string_list_item *item; - for (h = extra_http_headers; h; h = h->next) - headers = curl_slist_append(headers, h->data); + for_each_string_list_item(item, &extra_http_headers) + headers = curl_slist_append(headers, item->string); return headers; } -- gitgitgadget ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-06 10:04 ` [PATCH v2 1/1] " Johannes Schindelin via GitGitGadget @ 2019-11-06 11:29 ` Jeff King 2019-11-06 12:12 ` Junio C Hamano 1 sibling, 0 replies; 17+ messages in thread From: Jeff King @ 2019-11-06 11:29 UTC (permalink / raw) To: Johannes Schindelin via GitGitGadget Cc: git, Carlo Marcelo Arenas Belón, Johannes Schindelin, Junio C Hamano On Wed, Nov 06, 2019 at 10:04:55AM +0000, Johannes Schindelin via GitGitGadget wrote: > A naïve attempt at fixing this would move the call to > `curl_global_init()` _before_ the config is parsed (i.e. before that > call to `slist_append()`). > > However, that does work, as we _also_ parse the config setting > `http.sslbackend` and if found, call `curl_global_sslset()` which *must* > be called before `curl_global_init()`, for details see: > https://curl.haxx.se/libcurl/c/curl_global_sslset.html Yikes, good catch. It didn't even occur to me that there might be curl things we had to do _before_ calling curl_global_init(). > So let's instead make the config parsing entirely independent from > cURL's data structures. Incidentally, this deletes two more lines than > it introduces, which is nice. Yes, I actually find the resulting code easier to read. I had feared having to add an extra step to initialize the slist, but it's all handled quite neatly in http_copy_default_headers(). > http.c | 18 ++++++++---------- > 1 file changed, 8 insertions(+), 10 deletions(-) The patch itself looks good to me. -Peff ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-06 10:04 ` [PATCH v2 1/1] " Johannes Schindelin via GitGitGadget 2019-11-06 11:29 ` Jeff King @ 2019-11-06 12:12 ` Junio C Hamano 2019-11-06 19:34 ` Johannes Schindelin 1 sibling, 1 reply; 17+ messages in thread From: Junio C Hamano @ 2019-11-06 12:12 UTC (permalink / raw) To: Johannes Schindelin via GitGitGadget Cc: git, Carlo Marcelo Arenas Belón, Johannes Schindelin "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Johannes Schindelin <johannes.schindelin@gmx.de> > > In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to > ask cURL to use `xmalloc()`, and if compiled with nedmalloc, that means > implicitly a different allocator than the system one. > > Which means that all of cURL's allocations and releases now _need_ to > use that allocator. > > However, the `http_options()` function used `slist_append()` to add any > configured extra HTTP header(s) _before_ asking cURL to use `xmalloc()`, > and `http_cleanup()` would release them _afterwards_, i.e. in the > presence of custom allocators, cURL would attempt to use the wrong > allocator to release the memory. s/allocator/de&/; perhaps, even though it is clear enough from the context, so it is probably OK as is. > A naïve attempt at fixing this would move the call to > `curl_global_init()` _before_ the config is parsed (i.e. before that > call to `slist_append()`). > > However, that does work, as we _also_ parse the config setting s/does work/does not work/; presumably? > `http.sslbackend` and if found, call `curl_global_sslset()` which *must* > be called before `curl_global_init()`, for details see: > https://curl.haxx.se/libcurl/c/curl_global_sslset.html > > So let's instead make the config parsing entirely independent from > cURL's data structures. Incidentally, this deletes two more lines than > it introduces, which is nice. Yeah, string_list_clear() is more concise than curl_slist_free_all(), and we have already been copying one list to another anyway, so we lucked out ;-) The patch looked good to me, too. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-06 12:12 ` Junio C Hamano @ 2019-11-06 19:34 ` Johannes Schindelin 2019-11-07 5:39 ` Junio C Hamano 0 siblings, 1 reply; 17+ messages in thread From: Johannes Schindelin @ 2019-11-06 19:34 UTC (permalink / raw) To: Junio C Hamano Cc: Johannes Schindelin via GitGitGadget, git, Carlo Marcelo Arenas Belón [-- Attachment #1: Type: text/plain, Size: 1981 bytes --] Hi Junio, On Wed, 6 Nov 2019, Junio C Hamano wrote: > "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com> > writes: > > > From: Johannes Schindelin <johannes.schindelin@gmx.de> > > > > In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to > > ask cURL to use `xmalloc()`, and if compiled with nedmalloc, that means > > implicitly a different allocator than the system one. > > > > Which means that all of cURL's allocations and releases now _need_ to > > use that allocator. > > > > However, the `http_options()` function used `slist_append()` to add any > > configured extra HTTP header(s) _before_ asking cURL to use `xmalloc()`, > > and `http_cleanup()` would release them _afterwards_, i.e. in the > > presence of custom allocators, cURL would attempt to use the wrong > > allocator to release the memory. > > s/allocator/de&/; perhaps, even though it is clear enough from the > context, so it is probably OK as is. > > > A naïve attempt at fixing this would move the call to > > `curl_global_init()` _before_ the config is parsed (i.e. before that > > call to `slist_append()`). > > > > However, that does work, as we _also_ parse the config setting > > s/does work/does not work/; presumably? Indeed. Could I ask you to fix up locally, or do you want me to send a new revision of the patch? Ciao, Dscho > > > `http.sslbackend` and if found, call `curl_global_sslset()` which *must* > > be called before `curl_global_init()`, for details see: > > https://curl.haxx.se/libcurl/c/curl_global_sslset.html > > > > So let's instead make the config parsing entirely independent from > > cURL's data structures. Incidentally, this deletes two more lines than > > it introduces, which is nice. > > Yeah, string_list_clear() is more concise than curl_slist_free_all(), > and we have already been copying one list to another anyway, so we > lucked out ;-) > > The patch looked good to me, too. > ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-06 19:34 ` Johannes Schindelin @ 2019-11-07 5:39 ` Junio C Hamano 2019-11-07 12:40 ` Johannes Schindelin 0 siblings, 1 reply; 17+ messages in thread From: Junio C Hamano @ 2019-11-07 5:39 UTC (permalink / raw) To: Johannes Schindelin Cc: Johannes Schindelin via GitGitGadget, git, Carlo Marcelo Arenas Belón Johannes Schindelin <Johannes.Schindelin@gmx.de> writes: > On Wed, 6 Nov 2019, Junio C Hamano wrote: > >> > presence of custom allocators, cURL would attempt to use the wrong >> > allocator to release the memory. >> >> s/allocator/de&/; perhaps, even though it is clear enough from the >> context, so it is probably OK as is. >> >> > A naïve attempt at fixing this would move the call to >> > `curl_global_init()` _before_ the config is parsed (i.e. before that >> > call to `slist_append()`). >> > >> > However, that does work, as we _also_ parse the config setting >> >> s/does work/does not work/; presumably? > > Indeed. Could I ask you to fix up locally, or do you want me to send a > new revision of the patch? You can certainly tell me to locally tweak, but you'd need to be more specific when some of my suggestions were "perhaps" (aka "you could do it this way, which may be better, but I do not care too deeply---I am OK with whichever you chooose after you (re)think about it, but I am not choosing for you"). For now, I'll do the "does not work" thing only. Thanks. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-07 5:39 ` Junio C Hamano @ 2019-11-07 12:40 ` Johannes Schindelin 2019-11-08 3:03 ` Junio C Hamano 0 siblings, 1 reply; 17+ messages in thread From: Johannes Schindelin @ 2019-11-07 12:40 UTC (permalink / raw) To: Junio C Hamano Cc: Johannes Schindelin via GitGitGadget, git, Carlo Marcelo Arenas Belón [-- Attachment #1: Type: text/plain, Size: 1496 bytes --] Hi Junio, On Thu, 7 Nov 2019, Junio C Hamano wrote: > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes: > > > On Wed, 6 Nov 2019, Junio C Hamano wrote: > > > >> > presence of custom allocators, cURL would attempt to use the wrong > >> > allocator to release the memory. > >> > >> s/allocator/de&/; perhaps, even though it is clear enough from the > >> context, so it is probably OK as is. > >> > >> > A naïve attempt at fixing this would move the call to > >> > `curl_global_init()` _before_ the config is parsed (i.e. before that > >> > call to `slist_append()`). > >> > > >> > However, that does work, as we _also_ parse the config setting > >> > >> s/does work/does not work/; presumably? > > > > Indeed. Could I ask you to fix up locally, or do you want me to send a > > new revision of the patch? > > You can certainly tell me to locally tweak, but you'd need to be > more specific when some of my suggestions were "perhaps" (aka "you > could do it this way, which may be better, but I do not care too > deeply---I am OK with whichever you chooose after you (re)think > about it, but I am not choosing for you"). Right, I should have been more specific. > For now, I'll do the "does not work" thing only. Thanks! FWIW I would like to stick with "custom allocator" because even releasing the memory is the duty of an "allocator", even if that allocator happens to "deallocate" at that point what it has allocated before. Ciao, Dscho ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 1/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-07 12:40 ` Johannes Schindelin @ 2019-11-08 3:03 ` Junio C Hamano 0 siblings, 0 replies; 17+ messages in thread From: Junio C Hamano @ 2019-11-08 3:03 UTC (permalink / raw) To: Johannes Schindelin Cc: Johannes Schindelin via GitGitGadget, git, Carlo Marcelo Arenas Belón Johannes Schindelin <Johannes.Schindelin@gmx.de> writes: > FWIW I would like to stick with "custom allocator" because even > releasing the memory is the duty of an "allocator", even if that > allocator happens to "deallocate" at that point what it has allocated > before. My preference is to think of the set of functions we feed curl-global-init-mem as a (allocator, deallocator, reallocator, ...) tuple. But you are calling that set itself as an allocator that has service functions like (alloc, free, realloc), and as long as that (i.e. how you defined the word "allocator") is clear to the readers that is perfectly OK. To me as one reader, it was not clear and that was where my comment came from. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 0/1] remote-curl: unbreak http.extraHeader with custom allocators 2019-11-06 10:04 ` [PATCH v2 0/1] " Johannes Schindelin via GitGitGadget 2019-11-06 10:04 ` [PATCH v2 1/1] " Johannes Schindelin via GitGitGadget @ 2019-11-06 12:15 ` Junio C Hamano 1 sibling, 0 replies; 17+ messages in thread From: Junio C Hamano @ 2019-11-06 12:15 UTC (permalink / raw) To: Johannes Schindelin via GitGitGadget Cc: git, Carlo Marcelo Arenas Belón, Johannes Schindelin "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com> writes: > Johannes Schindelin (1): > remote-curl: unbreak http.extraHeader with custom allocators > > http.c | 18 ++++++++---------- > 1 file changed, 8 insertions(+), 10 deletions(-) > > base-commit: 93b980e58f5624ee4e3b2dc0d0babaa97ef66d19 This fixes by immediately building on top of the patch that caused the issue, so I should do the same so that any integration branch (either mine or somebody else's) that has the breakage can merge the fix in. Good. Thanks. ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2019-11-08 13:44 UTC | newest] Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-11-05 21:59 [PATCH 0/1] remote-curl: unbreak http.extraHeader with custom allocators Johannes Schindelin via GitGitGadget 2019-11-05 21:59 ` [PATCH 1/1] " Johannes Schindelin via GitGitGadget 2019-11-06 4:16 ` Jeff King 2019-11-06 9:14 ` Johannes Schindelin 2019-11-06 9:49 ` Junio C Hamano 2019-11-06 19:38 ` Johannes Schindelin 2019-11-08 8:34 ` Junio C Hamano 2019-11-08 13:44 ` Johannes Schindelin 2019-11-06 10:04 ` [PATCH v2 0/1] " Johannes Schindelin via GitGitGadget 2019-11-06 10:04 ` [PATCH v2 1/1] " Johannes Schindelin via GitGitGadget 2019-11-06 11:29 ` Jeff King 2019-11-06 12:12 ` Junio C Hamano 2019-11-06 19:34 ` Johannes Schindelin 2019-11-07 5:39 ` Junio C Hamano 2019-11-07 12:40 ` Johannes Schindelin 2019-11-08 3:03 ` Junio C Hamano 2019-11-06 12:15 ` [PATCH v2 0/1] " Junio C Hamano
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.