From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3D07C4361B for ; Mon, 7 Dec 2020 15:26:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C5E2423770 for ; Mon, 7 Dec 2020 15:26:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726119AbgLGP0K (ORCPT ); Mon, 7 Dec 2020 10:26:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48954 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725973AbgLGP0K (ORCPT ); Mon, 7 Dec 2020 10:26:10 -0500 Received: from mail-il1-x143.google.com (mail-il1-x143.google.com [IPv6:2607:f8b0:4864:20::143]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A7C4FC061794 for ; Mon, 7 Dec 2020 07:25:30 -0800 (PST) Received: by mail-il1-x143.google.com with SMTP id p5so12501086iln.8 for ; Mon, 07 Dec 2020 07:25:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=Uet7jOSWGSMmrkCLu1M1tIB1UFY2TuovlvGBTfJXDos=; b=bXcPFPj7b/p9PTgaxnfklSaCtv5D3FXr72XlilxCgoH7Te/G8fvRzkps5xq3jTygKs opwQZ2WxxEGGf4k58wF6Wu/xIfv6cZLBpeb6ESpHNstDcNN2atlqhrKjMgj51TsUUgRe emMjGHc/GbQFLJMdsJgDkPjYKoNC033DRHtJEGoCcNioebRlKta55Z6VPTXbCPLfCVvb kdweR6OSjziRs6CF3ezTWYBJCix21XaWPf04SkxxXXdTxOlmX6Qtbykp38423ggEa6Qa I4HrQe85AZ6EjF0x2cVGQN2Z2tniU2F2/9cvcU1Ith70ER8LDyW/L6CCZrYYCXR+zpBQ ZB/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=Uet7jOSWGSMmrkCLu1M1tIB1UFY2TuovlvGBTfJXDos=; b=qvTatKwVUMdD5ctJIA/udrpu81da42/RfT04gg4qv4/3evSRlXPKlw/6BUF3d3Sqg7 kGtsjeSqAzf/mGpuQ85ZIdPHb+aBKcOT+cqyr1TE7eOCqVc8FbhW7Sl2pywvbG4Wxsgg fVNMYm8V9qMh7szALw0exCTnLLZ/jPiZmH2XA4jcd8kVSHpMSVTUyz3siXX60lwQZwpH bRkSWh3RTbQZdFv1hrcKE3DvMSgwdg3vs9J2xmQKbwQ2U2DSWTwBVyU2T495dDUlaQjF s0T2c3RhRsggYTGYjcve6V752lsQLsB8e6nYRQKxHGsJxKKLtIby4qypVM0+sCq29bLq 3KJg== X-Gm-Message-State: AOAM532dHZ0hwYJtgh713Yj+I0iYjJ3xBSpQHurypbIKZ4H7yfpFF8f8 xWsVo5lzmcxPeVUTQIs1ZZraK9L1b/qvg/d9tAigOQ== X-Google-Smtp-Source: ABdhPJwUyBnHZ0e8dS+madvprroq9nMIZZ/fa0+cNzOgpCjMXZ64sb7bWyd1XqnAk7TkvsgPDmLX8vLpTqHE2TGSz2I= X-Received: by 2002:a92:d0ca:: with SMTP id y10mr13211293ila.68.1607354729664; Mon, 07 Dec 2020 07:25:29 -0800 (PST) MIME-Version: 1.0 References: <20201204180622.14285-1-abuehaze@amazon.com> <44E3AA29-F033-4B8E-A1BC-E38824B5B1E3@amazon.com> <3F02FF08-EDA6-4DFD-8D93-479A5B05E25A@amazon.com> In-Reply-To: <3F02FF08-EDA6-4DFD-8D93-479A5B05E25A@amazon.com> From: Eric Dumazet Date: Mon, 7 Dec 2020 16:25:17 +0100 Message-ID: Subject: Re: [PATCH net-next] tcp: optimise receiver buffer autotuning initialisation for high latency connections To: "Mohamed Abuelfotoh, Hazem" Cc: "netdev@vger.kernel.org" , "stable@vger.kernel.org" , "ycheng@google.com" , "ncardwell@google.com" , "weiwan@google.com" , "Strohman, Andy" , "Herrenschmidt, Benjamin" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org On Sat, Dec 5, 2020 at 1:03 PM Mohamed Abuelfotoh, Hazem wrote: > > Unfortunately few things are missing in this report. > > What is the RTT between hosts in your test ? > >>>>>RTT in my test is 162 msec, but I am able to reproduce it with = lower RTTs for example I could see the issue downloading from google endp= oint with RTT of 16.7 msec, as mentioned in my previous e-mail the issue is= reproducible whenever RTT exceeded 12msec given that the sender is usin= g bbr. > > RTT between hosts where I run the iperf test. > # ping 54.199.163.187 > PING 54.199.163.187 (54.199.163.187) 56(84) bytes of data. > 64 bytes from 54.199.163.187: icmp_seq=3D1 ttl=3D33 time=3D162 ms > 64 bytes from 54.199.163.187: icmp_seq=3D2 ttl=3D33 time=3D162 ms > 64 bytes from 54.199.163.187: icmp_seq=3D3 ttl=3D33 time=3D162 ms > 64 bytes from 54.199.163.187: icmp_seq=3D4 ttl=3D33 time=3D162 ms > > RTT between my EC2 instances and google endpoint. > # ping 172.217.4.240 > PING 172.217.4.240 (172.217.4.240) 56(84) bytes of data. > 64 bytes from 172.217.4.240: icmp_seq=3D1 ttl=3D101 time=3D16.7 m= s > 64 bytes from 172.217.4.240: icmp_seq=3D2 ttl=3D101 time=3D16.7 m= s > 64 bytes from 172.217.4.240: icmp_seq=3D3 ttl=3D101 time=3D16.7 m= s > 64 bytes from 172.217.4.240: icmp_seq=3D4 ttl=3D101 time=3D16.7 m= s > > What driver is used at the receiving side ? > >>>>>>I am using ENA driver version version: 2.2.10g on the receive= r with scatter gathering enabled. > > # ethtool -k eth0 | grep scatter-gather > scatter-gather: on > tx-scatter-gather: on > tx-scatter-gather-fraglist: off [fixed] This ethtool output refers to TX scatter gather, which is not relevant for this bug. I see ENA driver might use 16 KB per incoming packet (if ENA_PAGE_SIZE is 1= 6 KB) Since I can not reproduce this problem with another NIC on x86, I really wonder if this is not an issue with ENA driver on PowerPC perhaps ?