All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jeff King <peff@peff.net>
To: David Turner <dturner@twosigma.com>
Cc: git@vger.kernel.org
Subject: Re: [PATCH v3] http.postbuffer: allow full range of ssize_t values
Date: Sat, 1 Apr 2017 02:01:16 -0400	[thread overview]
Message-ID: <20170401060116.b2v7tyoi7fcxwbvo@sigill.intra.peff.net> (raw)
In-Reply-To: <20170331172631.12024-1-dturner@twosigma.com>

On Fri, Mar 31, 2017 at 01:26:31PM -0400, David Turner wrote:

> Unfortunately, in order to push some large repos, the http postbuffer
> must sometimes exceed two gigabytes.  On a 64-bit system, this is OK:
> we just malloc a larger buffer.

I'm still not sure why a 2GB post-buffer is necessary. It sounds like
something is broken in your setup. Large pushes should be sent chunked.

I know broken setups are a fact of life, but this feels like a really
hacky work-around.

> diff --git a/cache.h b/cache.h
> index fbdf7a815a..5e6747dbb4 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -1900,6 +1900,7 @@ extern int git_parse_maybe_bool(const char *);
>  extern int git_config_int(const char *, const char *);
>  extern int64_t git_config_int64(const char *, const char *);
>  extern unsigned long git_config_ulong(const char *, const char *);
> +extern ssize_t git_config_ssize_t(const char *, const char *);

For most of our other "big" values we use git_config_ulong(). E.g.,
core.bigfilethreshold. I suspect that would be fine for your purposes
here, though using size_t is more correct (on Windows "unsigned long" is
still only 32 bits, even on 64-bit systems).

The ultimate fate of this number, though, is to be handed to:

  curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDSIZE, rpc->len);

where the final argument is interpreted as a long. So I suspect that on
64-bit Windows, setting http.postbuffer to "3G" would cause some kind of
weird error (either a truncated post or some internal curl error due to
the negative size, depending on how curl handles it). And in that sense
it's worse than the status quo, which rejects this at the config level.

I think a "git_config_long()" would probably do everything correctly.

> +static int git_parse_ssize_t(const char *value, ssize_t *ret)
> +{
> +	ssize_t tmp;
> +	if (!git_parse_signed(value, &tmp, maximum_signed_value_of_type(ssize_t)))
> +		return 0;
> +	*ret = tmp;
> +	return 1;
> +}

I saw the earlier iteration used a size_t, but you switched it after the
compiler (rightfully) complained about the signedness. But I'm not sure
why we would want ssize_t here instead of just using git_parse_unsigned().

All of that's moot if we switch to parsing it as a "long" anyway,
though.

-Peff

  parent reply	other threads:[~2017-04-01  6:01 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-03-31 17:26 David Turner
2017-03-31 19:51 ` Junio C Hamano
2017-04-01  6:01 ` Jeff King [this message]
2017-04-01 18:09   ` Junio C Hamano
2017-04-03 17:03   ` David Turner
2017-04-04  2:01     ` Jeff King
2017-04-04 18:42       ` David Turner
2017-04-04 20:40         ` Jeff King
2017-04-06 17:24           ` Christian Couder
2017-04-07  4:48             ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170401060116.b2v7tyoi7fcxwbvo@sigill.intra.peff.net \
    --to=peff@peff.net \
    --cc=dturner@twosigma.com \
    --cc=git@vger.kernel.org \
    --subject='Re: [PATCH v3] http.postbuffer: allow full range of ssize_t values' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.