From: Bjorn Helgaas <helgaas@kernel.org> To: Ian Kumlien <ian.kumlien@gmail.com> Cc: Kai-Heng Feng <kai.heng.feng@canonical.com>, linux-pci <linux-pci@vger.kernel.org>, Alexander Duyck <alexander.duyck@gmail.com>, "Saheed O. Bolarinwa" <refactormyself@gmail.com>, Puranjay Mohan <puranjay12@gmail.com>, Jesse Brandeburg <jesse.brandeburg@intel.com>, Tony Nguyen <anthony.l.nguyen@intel.com>, "David S. Miller" <davem@davemloft.net>, Jakub Kicinski <kuba@kernel.org>, Heiner Kallweit <hkallweit1@gmail.com>, intel-wired-lan <intel-wired-lan@lists.osuosl.org>, Linux Kernel Network Developers <netdev@vger.kernel.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org> Subject: Re: [PATCH 1/3] PCI/ASPM: Use the path max in L1 ASPM latency check Date: Mon, 14 Dec 2020 18:40:04 -0600 [thread overview] Message-ID: <20201215004004.GA280628@bjorn-Precision-5520> (raw) In-Reply-To: <CAA85sZs8Li7+8BQWj0e+Qrxes1VF6K_Ukqrqgs1E3hHmaXqsbQ@mail.gmail.com> On Mon, Dec 14, 2020 at 11:56:31PM +0100, Ian Kumlien wrote: > On Mon, Dec 14, 2020 at 8:19 PM Bjorn Helgaas <helgaas@kernel.org> wrote: > > If you're interested, you could probably unload the Realtek drivers, > > remove the devices, and set the PCI_EXP_LNKCTL_LD (Link Disable) bit > > in 02:04.0, e.g., > > > > # RT=/sys/devices/pci0000:00/0000:00:01.2/0000:01:00.0/0000:02:04.0 > > # echo 1 > $RT/0000:04:00.0/remove > > # echo 1 > $RT/0000:04:00.1/remove > > # echo 1 > $RT/0000:04:00.2/remove > > # echo 1 > $RT/0000:04:00.4/remove > > # echo 1 > $RT/0000:04:00.7/remove > > # setpci -s02:04.0 CAP_EXP+0x10.w=0x0010 > > > > That should take 04:00.x out of the picture. > > Didn't actually change the behaviour, I'm suspecting an errata for AMD pcie... > > So did this, with unpatched kernel: > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 4.56 MBytes 38.2 Mbits/sec 0 67.9 KBytes > [ 5] 1.00-2.00 sec 4.47 MBytes 37.5 Mbits/sec 0 96.2 KBytes > [ 5] 2.00-3.00 sec 4.85 MBytes 40.7 Mbits/sec 0 50.9 KBytes > [ 5] 3.00-4.00 sec 4.23 MBytes 35.4 Mbits/sec 0 70.7 KBytes > [ 5] 4.00-5.00 sec 4.23 MBytes 35.4 Mbits/sec 0 48.1 KBytes > [ 5] 5.00-6.00 sec 4.23 MBytes 35.4 Mbits/sec 0 45.2 KBytes > [ 5] 6.00-7.00 sec 4.23 MBytes 35.4 Mbits/sec 0 36.8 KBytes > [ 5] 7.00-8.00 sec 3.98 MBytes 33.4 Mbits/sec 0 36.8 KBytes > [ 5] 8.00-9.00 sec 4.23 MBytes 35.4 Mbits/sec 0 36.8 KBytes > [ 5] 9.00-10.00 sec 4.23 MBytes 35.4 Mbits/sec 0 48.1 KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 43.2 MBytes 36.2 Mbits/sec 0 sender > [ 5] 0.00-10.00 sec 42.7 MBytes 35.8 Mbits/sec receiver > > and: > echo 0 > /sys/devices/pci0000:00/0000:00:01.2/0000:01:00.0/link/l1_aspm BTW, thanks a lot for testing out the "l1_aspm" sysfs file. I'm very pleased that it seems to be working as intended. > and: > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 113 MBytes 951 Mbits/sec 153 772 KBytes > [ 5] 1.00-2.00 sec 109 MBytes 912 Mbits/sec 276 550 KBytes > [ 5] 2.00-3.00 sec 111 MBytes 933 Mbits/sec 123 625 KBytes > [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 31 687 KBytes > [ 5] 4.00-5.00 sec 110 MBytes 923 Mbits/sec 0 679 KBytes > [ 5] 5.00-6.00 sec 110 MBytes 923 Mbits/sec 136 577 KBytes > [ 5] 6.00-7.00 sec 110 MBytes 923 Mbits/sec 214 645 KBytes > [ 5] 7.00-8.00 sec 110 MBytes 923 Mbits/sec 32 628 KBytes > [ 5] 8.00-9.00 sec 110 MBytes 923 Mbits/sec 81 537 KBytes > [ 5] 9.00-10.00 sec 110 MBytes 923 Mbits/sec 10 577 KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 1.08 GBytes 927 Mbits/sec 1056 sender > [ 5] 0.00-10.00 sec 1.07 GBytes 923 Mbits/sec receiver > > But this only confirms that the fix i experience is a side effect. > > The original code is still wrong :) What exactly is this machine? Brand, model, config? Maybe you could add this and a dmesg log to the buzilla? It seems like other people should be seeing the same problem, so I'm hoping to grub around on the web to see if there are similar reports involving these devices. https://bugzilla.kernel.org/show_bug.cgi?id=209725 Here's one that is superficially similar: https://linux-hardware.org/index.php?probe=e5f24075e5&log=lspci_all in that it has a RP -- switch -- I211 path. Interestingly, the switch here advertises <64us L1 exit latency instead of the <32us latency your switch advertises. Of course, I can't tell if it's exactly the same switch. Bjorn
WARNING: multiple messages have this Message-ID (diff)
From: Bjorn Helgaas <helgaas@kernel.org> To: intel-wired-lan@osuosl.org Subject: [Intel-wired-lan] [PATCH 1/3] PCI/ASPM: Use the path max in L1 ASPM latency check Date: Mon, 14 Dec 2020 18:40:04 -0600 [thread overview] Message-ID: <20201215004004.GA280628@bjorn-Precision-5520> (raw) In-Reply-To: <CAA85sZs8Li7+8BQWj0e+Qrxes1VF6K_Ukqrqgs1E3hHmaXqsbQ@mail.gmail.com> On Mon, Dec 14, 2020 at 11:56:31PM +0100, Ian Kumlien wrote: > On Mon, Dec 14, 2020 at 8:19 PM Bjorn Helgaas <helgaas@kernel.org> wrote: > > If you're interested, you could probably unload the Realtek drivers, > > remove the devices, and set the PCI_EXP_LNKCTL_LD (Link Disable) bit > > in 02:04.0, e.g., > > > > # RT=/sys/devices/pci0000:00/0000:00:01.2/0000:01:00.0/0000:02:04.0 > > # echo 1 > $RT/0000:04:00.0/remove > > # echo 1 > $RT/0000:04:00.1/remove > > # echo 1 > $RT/0000:04:00.2/remove > > # echo 1 > $RT/0000:04:00.4/remove > > # echo 1 > $RT/0000:04:00.7/remove > > # setpci -s02:04.0 CAP_EXP+0x10.w=0x0010 > > > > That should take 04:00.x out of the picture. > > Didn't actually change the behaviour, I'm suspecting an errata for AMD pcie... > > So did this, with unpatched kernel: > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 4.56 MBytes 38.2 Mbits/sec 0 67.9 KBytes > [ 5] 1.00-2.00 sec 4.47 MBytes 37.5 Mbits/sec 0 96.2 KBytes > [ 5] 2.00-3.00 sec 4.85 MBytes 40.7 Mbits/sec 0 50.9 KBytes > [ 5] 3.00-4.00 sec 4.23 MBytes 35.4 Mbits/sec 0 70.7 KBytes > [ 5] 4.00-5.00 sec 4.23 MBytes 35.4 Mbits/sec 0 48.1 KBytes > [ 5] 5.00-6.00 sec 4.23 MBytes 35.4 Mbits/sec 0 45.2 KBytes > [ 5] 6.00-7.00 sec 4.23 MBytes 35.4 Mbits/sec 0 36.8 KBytes > [ 5] 7.00-8.00 sec 3.98 MBytes 33.4 Mbits/sec 0 36.8 KBytes > [ 5] 8.00-9.00 sec 4.23 MBytes 35.4 Mbits/sec 0 36.8 KBytes > [ 5] 9.00-10.00 sec 4.23 MBytes 35.4 Mbits/sec 0 48.1 KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 43.2 MBytes 36.2 Mbits/sec 0 sender > [ 5] 0.00-10.00 sec 42.7 MBytes 35.8 Mbits/sec receiver > > and: > echo 0 > /sys/devices/pci0000:00/0000:00:01.2/0000:01:00.0/link/l1_aspm BTW, thanks a lot for testing out the "l1_aspm" sysfs file. I'm very pleased that it seems to be working as intended. > and: > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 113 MBytes 951 Mbits/sec 153 772 KBytes > [ 5] 1.00-2.00 sec 109 MBytes 912 Mbits/sec 276 550 KBytes > [ 5] 2.00-3.00 sec 111 MBytes 933 Mbits/sec 123 625 KBytes > [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 31 687 KBytes > [ 5] 4.00-5.00 sec 110 MBytes 923 Mbits/sec 0 679 KBytes > [ 5] 5.00-6.00 sec 110 MBytes 923 Mbits/sec 136 577 KBytes > [ 5] 6.00-7.00 sec 110 MBytes 923 Mbits/sec 214 645 KBytes > [ 5] 7.00-8.00 sec 110 MBytes 923 Mbits/sec 32 628 KBytes > [ 5] 8.00-9.00 sec 110 MBytes 923 Mbits/sec 81 537 KBytes > [ 5] 9.00-10.00 sec 110 MBytes 923 Mbits/sec 10 577 KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 1.08 GBytes 927 Mbits/sec 1056 sender > [ 5] 0.00-10.00 sec 1.07 GBytes 923 Mbits/sec receiver > > But this only confirms that the fix i experience is a side effect. > > The original code is still wrong :) What exactly is this machine? Brand, model, config? Maybe you could add this and a dmesg log to the buzilla? It seems like other people should be seeing the same problem, so I'm hoping to grub around on the web to see if there are similar reports involving these devices. https://bugzilla.kernel.org/show_bug.cgi?id=209725 Here's one that is superficially similar: https://linux-hardware.org/index.php?probe=e5f24075e5&log=lspci_all in that it has a RP -- switch -- I211 path. Interestingly, the switch here advertises <64us L1 exit latency instead of the <32us latency your switch advertises. Of course, I can't tell if it's exactly the same switch. Bjorn
next prev parent reply other threads:[~2020-12-15 0:40 UTC|newest] Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-10-07 13:28 [PATCH] Use maximum latency when determining L1 ASPM Ian Kumlien 2020-10-08 4:20 ` Kai-Heng Feng 2020-10-08 16:13 ` Bjorn Helgaas 2020-10-12 10:20 ` Ian Kumlien 2020-10-14 8:34 ` Kai-Heng Feng 2020-10-14 13:33 ` Ian Kumlien 2020-10-14 14:36 ` Bjorn Helgaas 2020-10-14 15:39 ` Ian Kumlien 2020-10-16 14:53 ` Ian Kumlien 2020-10-16 21:28 ` Bjorn Helgaas 2020-10-16 22:41 ` Ian Kumlien 2020-10-18 11:35 ` Ian Kumlien 2020-10-22 15:37 ` Bjorn Helgaas 2020-10-22 15:41 ` Ian Kumlien 2020-10-22 18:30 ` Bjorn Helgaas 2020-10-24 20:55 ` [PATCH 1/3] PCI/ASPM: Use the path max in L1 ASPM latency check Ian Kumlien 2020-10-24 20:55 ` [PATCH 2/3] PCI/ASPM: Fix L0s max " Ian Kumlien 2020-11-15 21:49 ` Ian Kumlien 2020-10-24 20:55 ` [PATCH 3/3] [RFC] PCI/ASPM: Print L1/L0s latency messages per endpoint Ian Kumlien 2020-11-15 21:49 ` [PATCH 1/3] PCI/ASPM: Use the path max in L1 ASPM latency check Ian Kumlien 2020-12-07 11:04 ` Ian Kumlien 2020-12-12 23:47 ` Bjorn Helgaas 2020-12-13 21:39 ` Ian Kumlien 2020-12-14 5:44 ` Bjorn Helgaas 2020-12-14 5:44 ` [Intel-wired-lan] " Bjorn Helgaas 2020-12-14 9:14 ` Ian Kumlien 2020-12-14 9:14 ` [Intel-wired-lan] " Ian Kumlien 2020-12-14 14:02 ` Bjorn Helgaas 2020-12-14 14:02 ` [Intel-wired-lan] " Bjorn Helgaas 2020-12-14 15:47 ` Ian Kumlien 2020-12-14 15:47 ` [Intel-wired-lan] " Ian Kumlien 2020-12-14 19:19 ` Bjorn Helgaas 2020-12-14 19:19 ` [Intel-wired-lan] " Bjorn Helgaas 2020-12-14 22:56 ` Ian Kumlien 2020-12-14 22:56 ` [Intel-wired-lan] " Ian Kumlien 2020-12-15 0:40 ` Bjorn Helgaas [this message] 2020-12-15 0:40 ` Bjorn Helgaas 2020-12-15 13:09 ` Ian Kumlien 2020-12-15 13:09 ` [Intel-wired-lan] " Ian Kumlien 2020-12-16 0:08 ` Bjorn Helgaas 2020-12-16 0:08 ` [Intel-wired-lan] " Bjorn Helgaas 2020-12-16 11:20 ` Ian Kumlien 2020-12-16 11:20 ` [Intel-wired-lan] " Ian Kumlien 2020-12-16 23:21 ` Bjorn Helgaas 2020-12-16 23:21 ` [Intel-wired-lan] " Bjorn Helgaas 2020-12-17 23:37 ` Ian Kumlien 2020-12-17 23:37 ` [Intel-wired-lan] " Ian Kumlien 2021-01-12 20:42 ` Bjorn Helgaas 2021-01-28 12:41 ` Ian Kumlien 2021-02-24 22:19 ` Ian Kumlien 2021-02-25 22:03 ` Bjorn Helgaas 2021-04-26 14:36 ` Ian Kumlien 2021-04-28 21:15 ` Bjorn Helgaas 2021-05-15 11:52 ` Ian Kumlien
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20201215004004.GA280628@bjorn-Precision-5520 \ --to=helgaas@kernel.org \ --cc=alexander.duyck@gmail.com \ --cc=anthony.l.nguyen@intel.com \ --cc=davem@davemloft.net \ --cc=hkallweit1@gmail.com \ --cc=ian.kumlien@gmail.com \ --cc=intel-wired-lan@lists.osuosl.org \ --cc=jesse.brandeburg@intel.com \ --cc=kai.heng.feng@canonical.com \ --cc=kuba@kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-pci@vger.kernel.org \ --cc=netdev@vger.kernel.org \ --cc=puranjay12@gmail.com \ --cc=refactormyself@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.