* [PATCH 0/2] Revert ASPM L1 Substates updates
@ 2023-02-03 22:48 Bjorn Helgaas
2023-02-03 22:48 ` [PATCH 1/2] Revert "PCI/ASPM: Save L1 PM Substates Capability for suspend/resume" Bjorn Helgaas
2023-02-03 22:48 ` [PATCH 2/2] Revert "PCI/ASPM: Refactor L1 PM Substates Control Register programming" Bjorn Helgaas
0 siblings, 2 replies; 5+ messages in thread
From: Bjorn Helgaas @ 2023-02-03 22:48 UTC (permalink / raw)
To: linux-pci; +Cc: Thomas Witt, Vidya Sagar, linux-kernel, Bjorn Helgaas
From: Bjorn Helgaas <bhelgaas@google.com>
Thomas Witt reported that 5e85eba6f50d ("PCI/ASPM: Refactor L1 PM
Substates Control Register programming") broke suspend/resume on a
Tuxedo Infinitybook S 14 v5, which seems to use a Clevo L140CU Mainboard.
See https://bugzilla.kernel.org/show_bug.cgi?id=216877
Since we haven't figured out a root cause, revert the relevant patches for
now.
Note that reverting "Save L1 PM Substates Capability for suspend/resume"
means machines will use more power after suspend/resume because the L1
Substates configuration is lost, but they should still work correctly.
Bjorn Helgaas (2):
Revert "PCI/ASPM: Save L1 PM Substates Capability for suspend/resume"
Revert "PCI/ASPM: Refactor L1 PM Substates Control Register
programming"
drivers/pci/pci.c | 7 ---
drivers/pci/pci.h | 4 --
drivers/pci/pcie/aspm.c | 109 ++++++++++++----------------------------
3 files changed, 33 insertions(+), 87 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 1/2] Revert "PCI/ASPM: Save L1 PM Substates Capability for suspend/resume"
2023-02-03 22:48 [PATCH 0/2] Revert ASPM L1 Substates updates Bjorn Helgaas
@ 2023-02-03 22:48 ` Bjorn Helgaas
2023-02-03 22:48 ` [PATCH 2/2] Revert "PCI/ASPM: Refactor L1 PM Substates Control Register programming" Bjorn Helgaas
1 sibling, 0 replies; 5+ messages in thread
From: Bjorn Helgaas @ 2023-02-03 22:48 UTC (permalink / raw)
To: linux-pci; +Cc: Thomas Witt, Vidya Sagar, linux-kernel, Bjorn Helgaas
From: Bjorn Helgaas <bhelgaas@google.com>
This reverts commit 4ff116d0d5fd8a025604b0802d93a2d5f4e465d1.
Thomas Witt reported that 5e85eba6f50d ("PCI/ASPM: Refactor L1 PM Substates
Control Register programming") broke suspend/resume.
4ff116d0d5fd ("PCI/ASPM: Save L1 PM Substates Capability for
suspend/resume") uses a function added by 5e85eba6f50d, so revert it first.
Note that reverting 4ff116d0d5fd means L1 Substates config will be lost on
suspend/resume. As far as we know the box will use more power but will
still *work* correctly.
Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=216877
Reported-by: Thomas Witt <kernel@witt.link>
Tested-by: Thomas Witt <kernel@witt.link>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Vidya Sagar <vidyas@nvidia.com>
---
drivers/pci/pci.c | 7 -------
drivers/pci/pci.h | 4 ----
drivers/pci/pcie/aspm.c | 37 -------------------------------------
3 files changed, 48 deletions(-)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index fba95486caaf..5641786bd020 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -1665,7 +1665,6 @@ int pci_save_state(struct pci_dev *dev)
return i;
pci_save_ltr_state(dev);
- pci_save_aspm_l1ss_state(dev);
pci_save_dpc_state(dev);
pci_save_aer_state(dev);
pci_save_ptm_state(dev);
@@ -1772,7 +1771,6 @@ void pci_restore_state(struct pci_dev *dev)
* LTR itself (in the PCIe capability).
*/
pci_restore_ltr_state(dev);
- pci_restore_aspm_l1ss_state(dev);
pci_restore_pcie_state(dev);
pci_restore_pasid_state(dev);
@@ -3465,11 +3463,6 @@ void pci_allocate_cap_save_buffers(struct pci_dev *dev)
if (error)
pci_err(dev, "unable to allocate suspend buffer for LTR\n");
- error = pci_add_ext_cap_save_buffer(dev, PCI_EXT_CAP_ID_L1SS,
- 2 * sizeof(u32));
- if (error)
- pci_err(dev, "unable to allocate suspend buffer for ASPM-L1SS\n");
-
pci_allocate_vc_save_buffers(dev);
}
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 9ed3b5550043..9049d07d3aae 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -566,14 +566,10 @@ bool pcie_wait_for_link(struct pci_dev *pdev, bool active);
void pcie_aspm_init_link_state(struct pci_dev *pdev);
void pcie_aspm_exit_link_state(struct pci_dev *pdev);
void pcie_aspm_powersave_config_link(struct pci_dev *pdev);
-void pci_save_aspm_l1ss_state(struct pci_dev *dev);
-void pci_restore_aspm_l1ss_state(struct pci_dev *dev);
#else
static inline void pcie_aspm_init_link_state(struct pci_dev *pdev) { }
static inline void pcie_aspm_exit_link_state(struct pci_dev *pdev) { }
static inline void pcie_aspm_powersave_config_link(struct pci_dev *pdev) { }
-static inline void pci_save_aspm_l1ss_state(struct pci_dev *dev) { }
-static inline void pci_restore_aspm_l1ss_state(struct pci_dev *dev) { }
#endif
#ifdef CONFIG_PCIE_ECRC
diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
index 53a1fa306e1e..915cbd939dd9 100644
--- a/drivers/pci/pcie/aspm.c
+++ b/drivers/pci/pcie/aspm.c
@@ -757,43 +757,6 @@ static void pcie_config_aspm_l1ss(struct pcie_link_state *link, u32 state)
PCI_L1SS_CTL1_L1SS_MASK, val);
}
-void pci_save_aspm_l1ss_state(struct pci_dev *dev)
-{
- struct pci_cap_saved_state *save_state;
- u16 l1ss = dev->l1ss;
- u32 *cap;
-
- if (!l1ss)
- return;
-
- save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_L1SS);
- if (!save_state)
- return;
-
- cap = (u32 *)&save_state->cap.data[0];
- pci_read_config_dword(dev, l1ss + PCI_L1SS_CTL2, cap++);
- pci_read_config_dword(dev, l1ss + PCI_L1SS_CTL1, cap++);
-}
-
-void pci_restore_aspm_l1ss_state(struct pci_dev *dev)
-{
- struct pci_cap_saved_state *save_state;
- u32 *cap, ctl1, ctl2;
- u16 l1ss = dev->l1ss;
-
- if (!l1ss)
- return;
-
- save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_L1SS);
- if (!save_state)
- return;
-
- cap = (u32 *)&save_state->cap.data[0];
- ctl2 = *cap++;
- ctl1 = *cap;
- aspm_program_l1ss(dev, ctl1, ctl2);
-}
-
static void pcie_config_aspm_dev(struct pci_dev *pdev, u32 val)
{
pcie_capability_clear_and_set_word(pdev, PCI_EXP_LNKCTL,
--
2.25.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 2/2] Revert "PCI/ASPM: Refactor L1 PM Substates Control Register programming"
2023-02-03 22:48 [PATCH 0/2] Revert ASPM L1 Substates updates Bjorn Helgaas
2023-02-03 22:48 ` [PATCH 1/2] Revert "PCI/ASPM: Save L1 PM Substates Capability for suspend/resume" Bjorn Helgaas
@ 2023-02-03 22:48 ` Bjorn Helgaas
2023-02-04 17:45 ` Lukas Wunner
1 sibling, 1 reply; 5+ messages in thread
From: Bjorn Helgaas @ 2023-02-03 22:48 UTC (permalink / raw)
To: linux-pci; +Cc: Thomas Witt, Vidya Sagar, linux-kernel, Bjorn Helgaas
From: Bjorn Helgaas <bhelgaas@google.com>
This reverts commit 5e85eba6f50dc288c22083a7e213152bcc4b8208.
Thomas Witt reported that 5e85eba6f50d ("PCI/ASPM: Refactor L1 PM Substates
Control Register programming") broke suspend/resume on a Tuxedo
Infinitybook S 14 v5, which seems to use a Clevo L140CU Mainboard.
The main symptom is:
iwlwifi 0000:02:00.0: Unable to change power state from D3hot to D0, device inaccessible
nvme 0000:03:00.0: Unable to change power state from D3hot to D0, device inaccessible
and the machine is only partially usable after resume. It can't run dmesg
and can't do a clean reboot. This happens on every suspend/resume cycle.
Revert 5e85eba6f50d until we can figure out the root cause.
Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=216877
Reported-by: Thomas Witt <kernel@witt.link>
Tested-by: Thomas Witt <kernel@witt.link>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Vidya Sagar <vidyas@nvidia.com>
---
drivers/pci/pcie/aspm.c | 72 +++++++++++++++++++----------------------
1 file changed, 33 insertions(+), 39 deletions(-)
diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
index 915cbd939dd9..4b4184563a92 100644
--- a/drivers/pci/pcie/aspm.c
+++ b/drivers/pci/pcie/aspm.c
@@ -470,31 +470,6 @@ static void pci_clear_and_set_dword(struct pci_dev *pdev, int pos,
pci_write_config_dword(pdev, pos, val);
}
-static void aspm_program_l1ss(struct pci_dev *dev, u32 ctl1, u32 ctl2)
-{
- u16 l1ss = dev->l1ss;
- u32 l1_2_enable;
-
- /*
- * Per PCIe r6.0, sec 5.5.4, T_POWER_ON in PCI_L1SS_CTL2 must be
- * programmed prior to setting the L1.2 enable bits in PCI_L1SS_CTL1.
- */
- pci_write_config_dword(dev, l1ss + PCI_L1SS_CTL2, ctl2);
-
- /*
- * In addition, Common_Mode_Restore_Time and LTR_L1.2_THRESHOLD in
- * PCI_L1SS_CTL1 must be programmed *before* setting the L1.2
- * enable bits, even though they're all in PCI_L1SS_CTL1.
- */
- l1_2_enable = ctl1 & PCI_L1SS_CTL1_L1_2_MASK;
- ctl1 &= ~PCI_L1SS_CTL1_L1_2_MASK;
-
- pci_write_config_dword(dev, l1ss + PCI_L1SS_CTL1, ctl1);
- if (l1_2_enable)
- pci_write_config_dword(dev, l1ss + PCI_L1SS_CTL1,
- ctl1 | l1_2_enable);
-}
-
/* Calculate L1.2 PM substate timing parameters */
static void aspm_calc_l1ss_info(struct pcie_link_state *link,
u32 parent_l1ss_cap, u32 child_l1ss_cap)
@@ -504,6 +479,7 @@ static void aspm_calc_l1ss_info(struct pcie_link_state *link,
u32 t_common_mode, t_power_on, l1_2_threshold, scale, value;
u32 ctl1 = 0, ctl2 = 0;
u32 pctl1, pctl2, cctl1, cctl2;
+ u32 pl1_2_enables, cl1_2_enables;
if (!(link->aspm_support & ASPM_STATE_L1_2_MASK))
return;
@@ -552,21 +528,39 @@ static void aspm_calc_l1ss_info(struct pcie_link_state *link,
ctl2 == pctl2 && ctl2 == cctl2)
return;
- pctl1 &= ~(PCI_L1SS_CTL1_CM_RESTORE_TIME |
- PCI_L1SS_CTL1_LTR_L12_TH_VALUE |
- PCI_L1SS_CTL1_LTR_L12_TH_SCALE);
- pctl1 |= (ctl1 & (PCI_L1SS_CTL1_CM_RESTORE_TIME |
- PCI_L1SS_CTL1_LTR_L12_TH_VALUE |
- PCI_L1SS_CTL1_LTR_L12_TH_SCALE));
- aspm_program_l1ss(parent, pctl1, ctl2);
+ /* Disable L1.2 while updating. See PCIe r5.0, sec 5.5.4, 7.8.3.3 */
+ pl1_2_enables = pctl1 & PCI_L1SS_CTL1_L1_2_MASK;
+ cl1_2_enables = cctl1 & PCI_L1SS_CTL1_L1_2_MASK;
- cctl1 &= ~(PCI_L1SS_CTL1_CM_RESTORE_TIME |
- PCI_L1SS_CTL1_LTR_L12_TH_VALUE |
- PCI_L1SS_CTL1_LTR_L12_TH_SCALE);
- cctl1 |= (ctl1 & (PCI_L1SS_CTL1_CM_RESTORE_TIME |
- PCI_L1SS_CTL1_LTR_L12_TH_VALUE |
- PCI_L1SS_CTL1_LTR_L12_TH_SCALE));
- aspm_program_l1ss(child, cctl1, ctl2);
+ if (pl1_2_enables || cl1_2_enables) {
+ pci_clear_and_set_dword(child, child->l1ss + PCI_L1SS_CTL1,
+ PCI_L1SS_CTL1_L1_2_MASK, 0);
+ pci_clear_and_set_dword(parent, parent->l1ss + PCI_L1SS_CTL1,
+ PCI_L1SS_CTL1_L1_2_MASK, 0);
+ }
+
+ /* Program T_POWER_ON times in both ports */
+ pci_write_config_dword(parent, parent->l1ss + PCI_L1SS_CTL2, ctl2);
+ pci_write_config_dword(child, child->l1ss + PCI_L1SS_CTL2, ctl2);
+
+ /* Program Common_Mode_Restore_Time in upstream device */
+ pci_clear_and_set_dword(parent, parent->l1ss + PCI_L1SS_CTL1,
+ PCI_L1SS_CTL1_CM_RESTORE_TIME, ctl1);
+
+ /* Program LTR_L1.2_THRESHOLD time in both ports */
+ pci_clear_and_set_dword(parent, parent->l1ss + PCI_L1SS_CTL1,
+ PCI_L1SS_CTL1_LTR_L12_TH_VALUE |
+ PCI_L1SS_CTL1_LTR_L12_TH_SCALE, ctl1);
+ pci_clear_and_set_dword(child, child->l1ss + PCI_L1SS_CTL1,
+ PCI_L1SS_CTL1_LTR_L12_TH_VALUE |
+ PCI_L1SS_CTL1_LTR_L12_TH_SCALE, ctl1);
+
+ if (pl1_2_enables || cl1_2_enables) {
+ pci_clear_and_set_dword(parent, parent->l1ss + PCI_L1SS_CTL1, 0,
+ pl1_2_enables);
+ pci_clear_and_set_dword(child, child->l1ss + PCI_L1SS_CTL1, 0,
+ cl1_2_enables);
+ }
}
static void aspm_l1ss_init(struct pcie_link_state *link)
--
2.25.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 2/2] Revert "PCI/ASPM: Refactor L1 PM Substates Control Register programming"
2023-02-03 22:48 ` [PATCH 2/2] Revert "PCI/ASPM: Refactor L1 PM Substates Control Register programming" Bjorn Helgaas
@ 2023-02-04 17:45 ` Lukas Wunner
2023-02-04 20:48 ` Bjorn Helgaas
0 siblings, 1 reply; 5+ messages in thread
From: Lukas Wunner @ 2023-02-04 17:45 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: linux-pci, Thomas Witt, Vidya Sagar, linux-kernel, Bjorn Helgaas
On Fri, Feb 03, 2023 at 04:48:20PM -0600, Bjorn Helgaas wrote:
> Thomas Witt reported that 5e85eba6f50d ("PCI/ASPM: Refactor L1 PM Substates
> Control Register programming") broke suspend/resume on a Tuxedo
> Infinitybook S 14 v5, which seems to use a Clevo L140CU Mainboard.
>
> The main symptom is:
>
> iwlwifi 0000:02:00.0: Unable to change power state from D3hot to D0, device inaccessible
> nvme 0000:03:00.0: Unable to change power state from D3hot to D0, device inaccessible
>
> and the machine is only partially usable after resume. It can't run dmesg
> and can't do a clean reboot. This happens on every suspend/resume cycle.
>
> Revert 5e85eba6f50d until we can figure out the root cause.
>
> Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=216877
Link: https://bugzilla.kernel.org/show_bug.cgi?id=216877
Fixes: 5e85eba6f50d ("PCI/ASPM: Refactor L1 PM Substates Control Register programming")
Cc: stable@vger.kernel.org # v6.1+
> Reported-by: Thomas Witt <kernel@witt.link>
> Tested-by: Thomas Witt <kernel@witt.link>
> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Vidya Sagar <vidyas@nvidia.com>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 2/2] Revert "PCI/ASPM: Refactor L1 PM Substates Control Register programming"
2023-02-04 17:45 ` Lukas Wunner
@ 2023-02-04 20:48 ` Bjorn Helgaas
0 siblings, 0 replies; 5+ messages in thread
From: Bjorn Helgaas @ 2023-02-04 20:48 UTC (permalink / raw)
To: Lukas Wunner
Cc: linux-pci, Thomas Witt, Vidya Sagar, linux-kernel, Bjorn Helgaas
On Sat, Feb 04, 2023 at 06:45:25PM +0100, Lukas Wunner wrote:
> On Fri, Feb 03, 2023 at 04:48:20PM -0600, Bjorn Helgaas wrote:
> > Thomas Witt reported that 5e85eba6f50d ("PCI/ASPM: Refactor L1 PM Substates
> > Control Register programming") broke suspend/resume on a Tuxedo
> > Infinitybook S 14 v5, which seems to use a Clevo L140CU Mainboard.
> >
> > The main symptom is:
> >
> > iwlwifi 0000:02:00.0: Unable to change power state from D3hot to D0, device inaccessible
> > nvme 0000:03:00.0: Unable to change power state from D3hot to D0, device inaccessible
> >
> > and the machine is only partially usable after resume. It can't run dmesg
> > and can't do a clean reboot. This happens on every suspend/resume cycle.
> >
> > Revert 5e85eba6f50d until we can figure out the root cause.
> >
> > Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=216877
>
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=216877
> Fixes: 5e85eba6f50d ("PCI/ASPM: Refactor L1 PM Substates Control Register programming")
> Cc: stable@vger.kernel.org # v6.1+
It's a pattern ;) Thanks again!
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-02-04 20:48 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-03 22:48 [PATCH 0/2] Revert ASPM L1 Substates updates Bjorn Helgaas
2023-02-03 22:48 ` [PATCH 1/2] Revert "PCI/ASPM: Save L1 PM Substates Capability for suspend/resume" Bjorn Helgaas
2023-02-03 22:48 ` [PATCH 2/2] Revert "PCI/ASPM: Refactor L1 PM Substates Control Register programming" Bjorn Helgaas
2023-02-04 17:45 ` Lukas Wunner
2023-02-04 20:48 ` Bjorn Helgaas
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).