From: Jeff King <peff@peff.net>
To: Denton Liu <liu.denton@gmail.com>
Cc: Git Mailing List <git@vger.kernel.org>,
Eric Sunshine <sunshine@sunshineco.com>,
Force Charlie <charlieio@outlook.com>
Subject: Re: [PATCH v2 7/7] stateless-connect: send response end packet
Date: Mon, 18 May 2020 13:26:05 -0400 [thread overview]
Message-ID: <20200518172605.GA52384@coredump.intra.peff.net> (raw)
In-Reply-To: <20200518171249.GA2462058@generichostname>
On Mon, May 18, 2020 at 01:12:49PM -0400, Denton Liu wrote:
> > I just wonder if there is a better place to put this logic that would be
> > more certain to catch every place we'd expect to read to the end of a
> > response. But I suppose not. We could push it down into process_acks(),
> > but it would have the same READY logic that's here. I'll admit part of
> > my complaint is that the existing do_fetch_pack_v2 state-machine switch
> > is kind of hard to follow, but that's not your fault.
>
> I debated between the current implementation and doing something like
>
> int first_iteration = 1;
>
> ...
>
> while (state != FETCH_DONE) {
> switch (...) {
> ...
> }
>
> if (args->stateless_rpc && !first_iteration && (state == FETCH_SEND_REQUEST || state == FETCH_DONE)) {
> if (packet_reader_read(&reader) != PACKET_READ_RESPONSE_END)
> die(_("git fetch-pack: expected response end packet"));
> if (packet_reader_read(&reader) != PACKET_READ_FLUSH)
> die(_("git fetch-pack: expected flush packet"));
> }
> first_iteration = 0;
> }
>
> I think that this catches _all_ the cases without fiddling with any of
> the state machine logic.
I think you're right that it does, but TBH I find it harder to follow,
because now we're even further away from the actual response reads. What
I most want to verify is:
- there is no response-reading code path that does not check the
delimiters
- there is no code path where we check delimiters but did not actually
read a response
So to me, putting the check as close to the response-reads as possible
makes sense. I think the patch below is equivalent and easier to verify,
but I admit it's somewhat subjective:
diff --git a/fetch-pack.c b/fetch-pack.c
index 5325649eda..0bf1e5760c 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1443,6 +1443,17 @@ static void receive_wanted_refs(struct packet_reader *reader,
die(_("error processing wanted refs: %d"), reader->status);
}
+static void check_stateless_delimiter(struct fetch_pack_args *args,
+ struct packet_reader *reader)
+{
+ if (!args->stateless_rpc)
+ return; /* not in stateless mode, no delimiter expected */
+ if (packet_reader_read(reader) != PACKET_READ_RESPONSE_END)
+ die(_("git fetch-pack: expected response end packet"));
+ if (packet_reader_read(reader) != PACKET_READ_FLUSH)
+ die(_("git fetch-pack: expected flush packet"));
+}
+
enum fetch_state {
FETCH_CHECK_LOCAL = 0,
FETCH_SEND_REQUEST,
@@ -1469,7 +1480,6 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
struct fetch_negotiator negotiator_alloc;
struct fetch_negotiator *negotiator;
int seen_ack = 0;
- int check_http_delimiter;
if (args->no_dependents) {
negotiator = NULL;
@@ -1488,8 +1498,6 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
}
while (state != FETCH_DONE) {
- check_http_delimiter = 0;
-
switch (state) {
case FETCH_CHECK_LOCAL:
sort_ref_list(&ref, ref_compare_name);
@@ -1538,15 +1546,19 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
/* Process ACKs/NAKs */
switch (process_acks(negotiator, &reader, &common)) {
case READY:
+ /*
+ * Don't check for response delimiter; get_pack() will
+ * read the rest of this response.
+ */
state = FETCH_GET_PACK;
break;
case COMMON_FOUND:
in_vain = 0;
seen_ack = 1;
/* fallthrough */
case NO_COMMON_FOUND:
state = FETCH_SEND_REQUEST;
- check_http_delimiter = 1;
+ check_stateless_delimiter(args, &reader);
break;
}
break;
@@ -1565,20 +1577,13 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
process_section_header(&reader, "packfile", 0);
if (get_pack(args, fd, pack_lockfile, sought, nr_sought))
die(_("git fetch-pack: fetch failed."));
+ check_stateless_delimiter(args, &reader);
state = FETCH_DONE;
- check_http_delimiter = 1;
break;
case FETCH_DONE:
continue;
}
-
- if (args->stateless_rpc && check_http_delimiter) {
- if (packet_reader_read(&reader) != PACKET_READ_RESPONSE_END)
- die(_("git fetch-pack: expected response end packet"));
- if (packet_reader_read(&reader) != PACKET_READ_FLUSH)
- die(_("git fetch-pack: expected flush packet"));
- }
}
if (negotiator)
> > Speaking of which: this is a change to the remote-helper protocol, since
> > we're now expecting stateless-connect helpers to produce these delim
> > packets (and failing if they don't). I kind of doubt that anybody but
> > remote-curl has implemented v2 stateless-connect, but should we be
> > marking this with an extra capability to be on the safe side?
>
> I think that we're probably safe from breaking anything external.
> According to the gitremote-helpers documentation,
>
> 'stateless-connect'::
> Experimental; for internal use only.
>
> so I think that gives us a bit of leeway in terms of the changes we can
> make to the stateless-connect protocol. They've been warned ;)
OK, I didn't realize we had explicitly marked it as experimental. I
agree we're probably fine here, then (now that v2 is starting to settle
a bit more, we might think about lifting that warning, but this patch
series shows we're not quite there yet :) ).
-Peff
next prev parent reply other threads:[~2020-05-18 17:26 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-13 18:04 [PATCH 0/6] remote-curl: partial fix for a deadlock with stateless rpc Denton Liu
2020-05-13 18:04 ` [PATCH 1/6] remote-curl: fix typo Denton Liu
2020-05-13 18:04 ` [PATCH 2/6] remote-curl: remove label indentation Denton Liu
2020-05-13 18:04 ` [PATCH 3/6] transport: combine common cases with a fallthrough Denton Liu
2020-05-13 23:14 ` Eric Sunshine
2020-05-18 9:18 ` Denton Liu
2020-05-18 17:43 ` Eric Sunshine
2020-05-13 18:04 ` [PATCH 4/6] pkt-line: extern packet_length() Denton Liu
2020-05-13 23:23 ` Eric Sunshine
2020-05-15 20:56 ` Jeff King
2020-05-15 20:57 ` Jeff King
2020-05-13 18:04 ` [PATCH 5/6] remote-curl: error on incomplete packet Denton Liu
2020-05-15 21:38 ` Jeff King
2020-05-18 9:08 ` Denton Liu
2020-05-18 15:49 ` Jeff King
2020-05-13 18:04 ` [PATCH 6/6] remote-curl: ensure last packet is a flush Denton Liu
2020-05-15 21:02 ` Denton Liu
2020-05-15 21:41 ` Jeff King
2020-05-18 16:34 ` Junio C Hamano
2020-05-18 16:52 ` Jeff King
2020-05-18 21:00 ` Jeff King
2020-05-18 15:47 ` [PATCH v2 0/7] remote-curl: fix deadlocks when remote server disconnects Denton Liu
2020-05-18 15:47 ` [PATCH v2 1/7] remote-curl: fix typo Denton Liu
2020-05-18 15:47 ` [PATCH v2 2/7] remote-curl: remove label indentation Denton Liu
2020-05-18 18:37 ` Junio C Hamano
2020-05-18 15:47 ` [PATCH v2 3/7] transport: extract common fetch_pack() call Denton Liu
2020-05-18 18:40 ` Junio C Hamano
2020-05-18 15:47 ` [PATCH v2 4/7] pkt-line: extern packet_length() Denton Liu
2020-05-18 16:04 ` Jeff King
2020-05-18 17:50 ` Eric Sunshine
2020-05-18 20:08 ` Jeff King
2020-05-18 18:44 ` Junio C Hamano
2020-05-18 15:47 ` [PATCH v2 5/7] remote-curl: error on incomplete packet Denton Liu
2020-05-18 16:22 ` Jeff King
2020-05-18 16:51 ` Denton Liu
2020-05-18 15:47 ` [PATCH v2 6/7] pkt-line: PACKET_READ_RESPONSE_END Denton Liu
2020-05-18 15:47 ` [PATCH v2 7/7] stateless-connect: send response end packet Denton Liu
2020-05-18 16:43 ` Jeff King
2020-05-18 17:12 ` Denton Liu
2020-05-18 17:26 ` Jeff King [this message]
2020-05-18 16:50 ` [PATCH v2 0/7] remote-curl: fix deadlocks when remote server disconnects Jeff King
2020-05-18 17:36 ` Denton Liu
2020-05-18 20:58 ` Jeff King
2020-05-18 22:52 ` Junio C Hamano
2020-05-19 2:38 ` Jeff King
2020-05-18 19:36 ` Junio C Hamano
2020-05-19 10:53 ` [PATCH v3 " Denton Liu
2020-05-19 10:53 ` [PATCH v3 1/7] remote-curl: fix typo Denton Liu
2020-05-19 10:53 ` [PATCH v3 2/7] remote-curl: remove label indentation Denton Liu
2020-05-19 10:53 ` [PATCH v3 3/7] transport: extract common fetch_pack() call Denton Liu
2020-05-19 10:53 ` [PATCH v3 4/7] pkt-line: extern packet_length() Denton Liu
2020-05-19 16:23 ` Eric Sunshine
2020-05-19 10:53 ` [PATCH v3 5/7] remote-curl: error on incomplete packet Denton Liu
2020-05-19 10:53 ` [PATCH v3 6/7] pkt-line: define PACKET_READ_RESPONSE_END Denton Liu
2020-05-19 10:54 ` [PATCH v3 7/7] stateless-connect: send response end packet Denton Liu
2020-05-19 18:40 ` [PATCH v3 0/7] remote-curl: fix deadlocks when remote server disconnects Jeff King
2020-05-19 21:14 ` Denton Liu
2020-05-19 20:51 ` [PATCH v3 8/7] fixup! pkt-line: extern packet_length() Denton Liu
2020-05-22 13:33 ` [PATCH v3 9/9] fixup! remote-curl: error on incomplete packet Denton Liu
2020-05-22 15:54 ` Jeff King
2020-05-22 16:05 ` Denton Liu
2020-05-22 16:31 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200518172605.GA52384@coredump.intra.peff.net \
--to=peff@peff.net \
--cc=charlieio@outlook.com \
--cc=git@vger.kernel.org \
--cc=liu.denton@gmail.com \
--cc=sunshine@sunshineco.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).