netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Gil Pedersen <kanongil@gmail.com>
To: Neal Cardwell <ncardwell@google.com>
Cc: David Miller <davem@davemloft.net>,
	Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>,
	dsahern@kernel.org, Netdev <netdev@vger.kernel.org>,
	Yuchung Cheng <ycheng@google.com>,
	Eric Dumazet <edumazet@google.com>
Subject: Re: TCP stall issue
Date: Wed, 24 Feb 2021 16:36:14 +0100	[thread overview]
Message-ID: <93A31D2F-1CDE-4042-9D00-A7E1E49A99A9@gmail.com> (raw)
In-Reply-To: <CADVnQynP40vvvTV3VY0fvYwEcSGQ=Y=F53FU8sEc-Bc=mzij5g@mail.gmail.com>


> On 24 Feb 2021, at 15.55, Neal Cardwell <ncardwell@google.com> wrote:
> 
> On Wed, Feb 24, 2021 at 5:03 AM Gil Pedersen <kanongil@gmail.com> wrote:
>> Sure, I attached a trace from the server that should illustrate the issue.
>> 
>> The trace is cut from a longer flow with the server at 188.120.85.11 and a client window scaling factor of 256.
>> 
>> Packet 78 is a TLP, followed by a delayed DUPACK with a SACK from the client.
>> The SACK triggers a single segment fast re-transmit with an ignored?? D-SACK in packet 81.
>> The first RTO happens at packet 82.
> 
> Thanks for the trace! That is very helpful. I have attached a plot and
> my notes on the trace, for discussion.
> 
> AFAICT the client appears to be badly misbehaving, and misrepresenting
> what has happened.  At each point where the client sends a DSACK,
> there is an apparent contradiction. Either the client has received
> that data before, or it hasn't. If the client *has* already received
> that data, then it should have already cumulatively ACKed it. If the
> client has *not* already received that data, then it shouldn't send a
> DSACK for it.
> 
> Given that, from the server's perspective, the client is
> misbehaving/lying, it's not clear what inferences the server can
> safely make. Though I agree it's probably possible to do much better
> than the current server behavior.
> 
> A few questions.
> 
> (a) is there a middlebox (firewall, NAT, etc) in the path?
> 
> (b) is it possible to capture a client-side trace, to help
> disambiguate whether there is a client-side Linux bug or a middlebox
> bug?

Yes, this sounds like a sound analysis, and matches my observation. The client is confused about whether it has the data or not.

Unfortunately I only have that (un-rooted) device available, so I can't do traces on it. The connection path is Client -> Wi-Fi -> NAT -> NAT -> Internet -> Server (which has a basic UFW firewall).
I will try to do a trace on the first NAT router.

My first priority is to make the server behave better in this case, but I understand that you would like to investigate the client / connection issue as well? From the server POV, this is clearly an edge case, but a fast re-transmit does seem more appropriate.

Btw. the "client SACKs TLP retransmit" note is not correct. This is an old ACK, which can be seen from the ecr value.

/Gil

  reply	other threads:[~2021-02-24 16:02 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-23 10:09 TCP stall issue Gil Pedersen
2021-02-23 15:41 ` Neal Cardwell
2021-02-24 10:03   ` Gil Pedersen
2021-02-24 14:55     ` Neal Cardwell
2021-02-24 15:36       ` Gil Pedersen [this message]
2021-02-25 15:05         ` Neal Cardwell
2021-02-26 14:39           ` David Laight
2021-02-26 16:26             ` Gil Pedersen
2021-02-26 17:50               ` Neal Cardwell
2021-02-26 21:59                 ` Maciej Żenczykowski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=93A31D2F-1CDE-4042-9D00-A7E1E49A99A9@gmail.com \
    --to=kanongil@gmail.com \
    --cc=davem@davemloft.net \
    --cc=dsahern@kernel.org \
    --cc=edumazet@google.com \
    --cc=ncardwell@google.com \
    --cc=netdev@vger.kernel.org \
    --cc=ycheng@google.com \
    --cc=yoshfuji@linux-ipv6.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).