From: Jeff King <peff@peff.net>
To: Florian Manschwetus <manschwetus@cs-software-gmbh.de>
Cc: Chris Packham <judge.packham@gmail.com>,
Konstantin Khomoutov <kostix+git@007spb.ru>,
"git@vger.kernel.org" <git@vger.kernel.org>
Subject: Re: [PATCH] Fix http-backend reading till EOF, ignoring CONTENT_LENGTH, violating rfc3875 -- WAS: Problem with git-http-backend.exe as iis cgi
Date: Tue, 29 Mar 2016 16:13:49 -0400 [thread overview]
Message-ID: <20160329201349.GB9527@sigill.intra.peff.net> (raw)
In-Reply-To: <F0F5A56A22F20D4CB4A03BB8D6658797E261A3D6@SERVER2011.CS-SOFTWARE.local>
On Tue, Mar 29, 2016 at 10:38:23AM +0000, Florian Manschwetus wrote:
> > | A request-body is supplied with the request if the CONTENT_LENGTH is
> > | not NULL. The server MUST make at least that many bytes available
> > | for the script to read. The server MAY signal an end-of-file
> > | condition after CONTENT_LENGTH bytes have been read or it MAY supply
> > | extension data. Therefore, the script MUST NOT attempt to read more
> > | than CONTENT_LENGTH bytes, even if more data is available. However,
> > | it is not obliged to read any of the data.
> >
> > So yes, if Git currently reads until EOF, it's an error.
> > The correct way would be:
> >
> > 1) Check to see if the CONTENT_LENGTH variable is available in the
> > environment. If no, read nothing.
> >
> > 2) Otherwise read as many bytes it specifies, and no more.
> >
> > 1. https://www.ietf.org/rfc/rfc3875
I don't think the second part of (1) will work very well if the client
sends a chunked transfer-encoding (which git will do if the input is large). In
such a case the server would either have to buffer the entire input to
find its length, or stream the data to the CGI without setting
$CONTENT_LENGTH. At least some servers choose the latter (including
Apache).
> diff --git a/http-backend.c b/http-backend.c
> index 8870a26..94976df 100644
> --- a/http-backend.c
> +++ b/http-backend.c
> @@ -277,16 +277,32 @@ static struct rpc_service *select_service(const char *name)
> */
> static ssize_t read_request(int fd, unsigned char **out)
> {
> - size_t len = 0, alloc = 8192;
> - unsigned char *buf = xmalloc(alloc);
> + unsigned char *buf = null;
> + size_t len = 0;
> + /* get request size */
> + size_t req_len = git_env_ulong("CONTENT_LENGTH",
> + 0);
> +
> + /* check request size */
> + if (max_request_buffer < req_len) {
> + die("request was larger than our maximum size (%lu);"
> + " try setting GIT_HTTP_MAX_REQUEST_BUFFER",
> + max_request_buffer);
> + }
> +
> + if (req_len <= 0) {
> + *out = null;
> + return 0;
> + }
git-am complained that your patch did not apply, but after writing
something similar locally, I found that t5551.25 hangs indefinitely.
Which is not surprising. Most tests are doing very limited ref
negotiation, so the content that hits read_request() here is small, and
we send it in a single write with a content-length header. But t5551.25
uses a much bigger workload, which causes the client to use a chunked
transfer-encoding, and this code to refuse to read anything (and then
the protocol stalls, as we are waiting for the client to say something).
So I think you'd want to take a missing CONTENT_LENGTH as a hint to read
until EOF.
That also raises another issue: what happens in the paths that don't hit
read_request()? We may also process input via:
- inflate_request(), if the client gzipped it; for well-formed input,
I think we'll stop reading when the gzip stream tells us there is no
more data, but a malformed one would have us reading until EOF,
regardless of what $CONTENT_LENGTH says.
- for input which we expect to be large (like incoming packfiles for a
push), buffer_input will be unset, and we will pass the descriptor
directly to a sub-program like git-index-pack. Again, for
well-formed input it would read just the packfile, but it may
actually continue to EOF.
So I don't think your patch is covering all cases.
-Peff
next prev parent reply other threads:[~2016-03-29 20:14 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-03-29 10:38 [PATCH] Fix http-backend reading till EOF, ignoring CONTENT_LENGTH, violating rfc3875 -- WAS: Problem with git-http-backend.exe as iis cgi Florian Manschwetus
2016-03-29 20:13 ` Jeff King [this message]
2016-03-30 9:08 ` AW: " Florian Manschwetus
2016-04-01 23:55 ` Jeff King
2017-11-23 23:45 ` [PATCH] http-backend: respect CONTENT_LENGTH as specified by rfc3875 Max Kirillov
2017-11-24 1:30 ` Eric Sunshine
2017-11-25 21:47 ` Max Kirillov
2017-11-26 0:38 ` Eric Sunshine
2017-11-26 0:43 ` Max Kirillov
2017-11-24 5:54 ` Junio C Hamano
2017-11-24 8:30 ` AW: " Florian Manschwetus
2017-11-26 1:50 ` Max Kirillov
2017-11-26 1:47 ` [PATCH v4 0/2] " Max Kirillov
2017-11-26 1:47 ` [PATCH v4 1/2] " Max Kirillov
2017-11-26 1:47 ` [PATCH v4 2/2] t5560-http-backend-noserver.sh: add CONTENT_LENGTH cases Max Kirillov
2017-11-26 1:54 ` [PATCH v5 0/2] http-backend: respect CONTENT_LENGTH as specified by rfc3875 Max Kirillov
2017-11-26 1:54 ` [PATCH v5 1/2] " Max Kirillov
2017-11-26 3:46 ` Junio C Hamano
2017-11-26 8:13 ` Max Kirillov
2017-11-26 9:38 ` Junio C Hamano
2017-11-26 19:39 ` Max Kirillov
2017-11-26 1:54 ` [PATCH v5 2/2] t5560-http-backend-noserver.sh: add CONTENT_LENGTH cases Max Kirillov
2017-11-26 19:38 ` [PATCH v6 0/2] http-backend: respect CONTENT_LENGTH as specified by rfc3875 Max Kirillov
2017-11-26 19:38 ` [PATCH v6 1/2] " Max Kirillov
2017-11-26 22:08 ` Eric Sunshine
2017-11-29 3:22 ` Jeff King
2017-12-03 1:02 ` Junio C Hamano
2017-12-03 2:49 ` Jeff King
2017-12-03 6:07 ` Junio C Hamano
2017-12-04 7:18 ` AW: " Florian Manschwetus
2017-12-04 17:13 ` Jeff King
2017-11-26 19:38 ` [PATCH v6 2/2] t5560-http-backend-noserver.sh: add CONTENT_LENGTH cases Max Kirillov
2017-11-26 22:18 ` Eric Sunshine
2017-11-26 22:40 ` Max Kirillov
2017-11-29 3:26 ` Jeff King
2017-11-29 5:19 ` Max Kirillov
2017-12-03 0:46 ` Junio C Hamano
2017-11-27 0:29 ` Junio C Hamano
2017-11-27 4:02 ` [PATCH v6 0/2] http-backend: respect CONTENT_LENGTH as specified by rfc3875 Junio C Hamano
2017-11-29 5:07 ` Max Kirillov
2017-12-03 0:48 ` Junio C Hamano
2017-12-12 16:17 ` Need to add test artifacts to .gitignore Dan Jacques
2017-12-12 19:00 ` [RFC PATCH] t/helper: Move sources to t/helper-src; gitignore any files in t/helper Stefan Beller
2017-12-12 19:59 ` Junio C Hamano
2017-12-12 20:56 ` [PATCH] t/helper: ignore everything but sources Stefan Beller
2017-12-12 21:06 ` Junio C Hamano
2017-12-13 20:12 ` Stefan Beller
2017-12-12 21:06 ` Todd Zullinger
2017-12-19 22:13 ` [PATCH v6 0/2] http-backend: respect CONTENT_LENGTH as specified by rfc3875 Junio C Hamano
2017-12-20 4:30 ` Max Kirillov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160329201349.GB9527@sigill.intra.peff.net \
--to=peff@peff.net \
--cc=git@vger.kernel.org \
--cc=judge.packham@gmail.com \
--cc=kostix+git@007spb.ru \
--cc=manschwetus@cs-software-gmbh.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).