From: Kai-Heng Feng <kai.heng.feng@canonical.com>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: Heiner Kallweit <hkallweit1@gmail.com>,
nic_swsd <nic_swsd@realtek.com>,
Bjorn Helgaas <bhelgaas@google.com>,
David Miller <davem@davemloft.net>,
Jakub Kicinski <kuba@kernel.org>,
Anthony Wong <anthony.wong@canonical.com>,
Linux Netdev List <netdev@vger.kernel.org>,
Linux PCI <linux-pci@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: [RFC] [PATCH net-next v5 0/3] r8169: Implement dynamic ASPM mechanism for recent 1.0/2.5Gbps Realtek NICs
Date: Fri, 1 Oct 2021 12:17:26 +0800 [thread overview]
Message-ID: <CAAd53p409uhbor1ArZ=kfiMK2JRHVGVyYukDSSyDvFsVSs=ErQ@mail.gmail.com> (raw)
In-Reply-To: <20210917220942.GA1748301@bjorn-Precision-5520>
On Sat, Sep 18, 2021 at 6:09 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> On Thu, Sep 16, 2021 at 11:44:14PM +0800, Kai-Heng Feng wrote:
> > The purpose of the series is to get comments and reviews so we can merge
> > and test the series in downstream kernel.
> >
> > The latest Realtek vendor driver and its Windows driver implements a
> > feature called "dynamic ASPM" which can improve performance on it's
> > ethernet NICs.
> >
> > Heiner Kallweit pointed out the potential root cause can be that the
> > buffer is too small for its ASPM exit latency.
>
> I looked at the lspci data in your bugzilla
> (https://bugzilla.kernel.org/show_bug.cgi?id=214307).
>
> L1.2 is enabled, which requires the Latency Tolerance Reporting
> capability, which helps determine when the Link will be put in L1.2.
> IIUC, these are analogous to the DevCap "Acceptable Latency" values.
> Zero latency values indicate the device will be impacted by any delay
> (PCIe r5.0, sec 6.18).
>
> Linux does not currently program those values, so the values there
> must have been set by the BIOS. On the working AMD system, they're
> set to 1048576ns, while on the broken Intel system, they're set to
> 3145728ns.
>
> I don't really understand how these values should be computed, and I
> think they depend on some electrical characteristics of the Link, so
> I'm not sure it's *necessarily* a problem that they are different.
> But a 3X difference does seem pretty large.
>
> So I'm curious whether this is related to the problem. Here are some
> things we could try on the broken Intel system:
Original network speed, tested via iperf3:
TX: ~255 Mbps
RX: ~490 Mbps
>
> - What happens if you disable ASPM L1.2 using
> /sys/devices/pci*/.../link/l1_2_aspm?
TX: ~670 Mbps
RX: ~670 Mbps
>
> - If that doesn't work, what happens if you also disable PCI-PM L1.2
> using /sys/devices/pci*/.../link/l1_2_pcipm?
Same as only disables l1_2_aspm.
>
> - If either of the above makes things work, then at least we know
> the problem is sensitive to L1.2.
Right now the downstream kernel disables ASPM L1.2 as workaround.
>
> - Then what happens if you use setpci to set the LTR Latency
> registers to 0, then re-enable ASPM L1.2 and PCI-PM L1.2? This
> should mean the Realtek device wants the best possible service and
> the Link probably won't spend much time in L1.2.
# setpci -s 01:00.0 ECAP_LTR+4.w=0x0
# setpci -s 01:00.0 ECAP_LTR+6.w=0x0
Then re-enable ASPM L1.2, the issue persists - the network speed is
still very slow.
>
> - What happens if you set the LTR Latency registers to 0x1001
> (should be the same as on the AMD system)?
Same slow speed here.
Kai-Heng
next prev parent reply other threads:[~2021-10-01 4:17 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-16 15:44 [RFC] [PATCH net-next v5 0/3] r8169: Implement dynamic ASPM mechanism for recent 1.0/2.5Gbps Realtek NICs Kai-Heng Feng
2021-09-16 15:44 ` [RFC] [PATCH net-next v5 1/3] PCI/ASPM: Introduce a new helper to report ASPM capability Kai-Heng Feng
2021-09-16 15:44 ` [RFC] [PATCH net-next v5 2/3] r8169: Use PCIe ASPM status for NIC ASPM enablement Kai-Heng Feng
2021-09-16 17:07 ` Bjorn Helgaas
2021-09-17 4:09 ` Kai-Heng Feng
2021-09-17 15:26 ` Bjorn Helgaas
2021-10-01 4:32 ` Kai-Heng Feng
2021-09-16 15:44 ` [RFC] [PATCH net-next v5 3/3] r8169: Implement dynamic ASPM mechanism Kai-Heng Feng
2021-09-16 17:12 ` Bjorn Helgaas
2021-09-17 4:19 ` Kai-Heng Feng
2021-09-17 22:09 ` [RFC] [PATCH net-next v5 0/3] r8169: Implement dynamic ASPM mechanism for recent 1.0/2.5Gbps Realtek NICs Bjorn Helgaas
2021-10-01 4:17 ` Kai-Heng Feng [this message]
2021-10-07 19:10 ` Bjorn Helgaas
2021-10-08 13:56 ` Bjorn Helgaas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAAd53p409uhbor1ArZ=kfiMK2JRHVGVyYukDSSyDvFsVSs=ErQ@mail.gmail.com' \
--to=kai.heng.feng@canonical.com \
--cc=anthony.wong@canonical.com \
--cc=bhelgaas@google.com \
--cc=davem@davemloft.net \
--cc=helgaas@kernel.org \
--cc=hkallweit1@gmail.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=nic_swsd@realtek.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).