Re: TCP stall issue

From: "Maciej Żenczykowski" <maze@google.com>
To: Neal Cardwell <ncardwell@google.com>
Cc: Gil Pedersen <kanongil@gmail.com>,
	David Laight <David.Laight@aculab.com>,
	David Miller <davem@davemloft.net>,
	Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>,
	"dsahern@kernel.org" <dsahern@kernel.org>,
	Netdev <netdev@vger.kernel.org>,
	Yuchung Cheng <ycheng@google.com>,
	Eric Dumazet <edumazet@google.com>
Subject: Re: TCP stall issue
Date: Fri, 26 Feb 2021 13:59:26 -0800	[thread overview]
Message-ID: <CANP3RGfTy089zgbzhLiUApSuMBZtn71szC6MWFkKdDQ-PN0XSA@mail.gmail.com> (raw)
In-Reply-To: <CADVnQyk+hZX46gVogJpqMQrpQEPdPZRd=hr2zcYxTTtMZubY+g@mail.gmail.com>

On Fri, Feb 26, 2021 at 9:50 AM Neal Cardwell <ncardwell@google.com> wrote:
>
> On Fri, Feb 26, 2021 at 11:26 AM Gil Pedersen <kanongil@gmail.com> wrote:
> >
> >
> > > On 26 Feb 2021, at 15.39, David Laight <David.Laight@ACULAB.COM> wrote:
> > >
> > > Some thoughts...
> > >
> > > Does a non-android linux system behave correctly through the same NAT gateways?
> > > Particularly with a similar kernel version.
> > >
> > > If you have a USB OTG cable and USB ethernet dongle you may be able to get
> > > android to use a wired ethernet connection - excluding any WiFi issues.
> > > (OTG usually works for keyboard and mouse, dunno if ethernet support is there.)
> > >
> > > Does you android device work on any other networks?
> >
> > I have done some further tests. I managed to find another Android device (kernel 4.9.113), which thankfully does _not_ send the weird D-SACKs and quickly recovers, so the problem appears to be on the original device.
> >
> > Additionally, I have managed to do a trace on the WLAN AP, where I can confirm that all packets seem to be transferred without unnecessary modifications or re-ordering. Ie. all segments sent from the server make it to the device and any loss will be device local. As such this points to a driver-level issue?
> >
> > I don't have an ethernet dongle ready. I tried to connect using cellular and was unable to replicate the issue, so this further points at a driver-level issue.
> >
> > Given that it now seems relevant, the device is an Android P20 Lite, running a variant of Android 9.1 with an update from this year (kernel was built jan. 05 2021).
>
> Thanks for the details. Agreed, it does sound as if a wifi
> hardware/firmare/driver issue on that particular Android device is the
> most likely cause of those symptoms.
>
> The only sequence I can think of that would cause these  symptoms
> would be if the wifi hardware/firmer/driver on that device is somehow
> both:
>
> (1) duplicating each of the retransmit packets that it passes up the
> network stack, and
> (2) dropping the first ACK packet generated by the first of the two
> copies of the retransmit
>
> Though that sounds so unlikely that perhaps there is a different explanation...
>
> neal

I got pulled into this thread without any real background... so I'm
guessing a fair bit.

I'm not at all sure what 'P20 Lite' refers to, but assuming this is
talking about a Pixel 2020 (perhaps the Pixel 4a 5G) then they were
never released with Android 9 (Pie), but rather with Android 11 (R) --
there were I think some early dev versions on Android 10 (Q), but
definitely nothing on P.  Plus the kernel is 4.19 based, not 4.9.

Based on the above, I'd assume you're not talking about a pixel
phone... but rather some other P20 Lite - perhaps from Huawei.
For non-Google/Pixel devices, all I can really say is talk to the
chipset vendor / oem (manufacturer) and possibly the carrier...
There's usually tons of 'value add' added by them (chipset vendor /
oem) or because of them (carrier) - in particular in the kernel...
Maybe ask for your device's kernel GPL source...

Additionally 4.9 LTS is at 4.9.258, so the kernel on the device is
simply very old (though that might be the revision on a different
device?).

Cheers,
Maciej