From: Mingchuang Qiao <mingchuang.qiao@mediatek.com>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: Mika Westerberg <mika.westerberg@linux.intel.com>,
Bjorn Helgaas <bhelgaas@google.com>,
"Rafael J. Wysocki" <rjw@rjwysocki.net>,
"Utkarsh H Patel" <utkarsh.h.patel@intel.com>,
<linux-pci@vger.kernel.org>, <matthias.bgg@gmail.com>,
<lambert.wang@mediatek.com>, <linux-mediatek@lists.infradead.org>,
<haijun.liu@mediatek.com>, <linux-arm-kernel@lists.infradead.org>,
<linux-kernel@vger.kernel.org>,
"Alex Williamson" <alex.williamson@redhat.com>
Subject: Re: [PATCH v2] PCI: Re-enable downstream port LTR if it was previously enabled
Date: Fri, 22 Jan 2021 15:03:11 +0800 [thread overview]
Message-ID: <1611298991.5980.42.camel@mcddlt001> (raw)
In-Reply-To: <20210121223139.GA2698934@bjorn-Precision-5520>
On Thu, 2021-01-21 at 16:31 -0600, Bjorn Helgaas wrote:
> [+cc Alex and Mingchuang et al from
> https://lore.kernel.org/r/20210112072739.31624-1-mingchuang.qiao@mediatek.com]
>
> On Tue, Jan 19, 2021 at 04:14:10PM +0300, Mika Westerberg wrote:
> > PCIe r5.0, sec 7.5.3.16 says that the downstream ports must reset the
> > LTR enable bit if the link goes down (port goes DL_Down status). Now, if
> > we had LTR previously enabled and the PCIe endpoint gets hot-removed and
> > then hot-added back the ->ltr_path of the downstream port is still set
> > but the port now does not have the LTR enable bit set anymore.
> >
> > For this reason check if the bridge upstream had LTR enabled previously
> > and re-enable it before enabling LTR for the endpoint.
> >
> > Reported-by: Utkarsh H Patel <utkarsh.h.patel@intel.com>
> > Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
>
> I think this and Mingchuang's patch, which is essentially identical,
> are right and solves the problem for hot-remove/hot-add. In that
> scenario we call pci_configure_ltr() on the hot-added device, and
> with this patch, we'll re-enable LTR on the bridge leading to the new
> device before enabling LTR on the new device itself.
>
> But don't we have a similar problem if we simply do a Fundamental
> Reset on a device? I think the reset path will restore the device's
> state, including PCI_EXP_DEVCTL2, but it doesn't do anything with the
> upstream bridge, does it?
>
Yes. I think the same problem exists under such scenario, and that’s the
issue my patch intends to resolve.
I also prepared a v2 patch for review(update the patch description).
Shall I submit the v2 patch for review?
> So if a bridge and a device below it both have LTR enabled, can't we
> have the following:
>
> - bridge LTR enabled
> - device LTR enabled
> - reset device, e.g., via Secondary Bus Reset
> - link goes down, bridge disables LTR
> - link comes back up, LTR disabled in both bridge and device
> - restore device state, including LTR enable
> - device sends LTR message
> - bridge reports Unsupported Request
>
> > ---
> > Previous version can be found here:
> >
> > https://lore.kernel.org/linux-pci/20210114134724.79511-1-mika.westerberg@linux.intel.com/
> >
> > Changes from the previous version:
> >
> > * Corrected typos in the commit message
> > * No need to call pcie_downstream_port()
> >
> > drivers/pci/probe.c | 17 ++++++++++++++++-
> > 1 file changed, 16 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> > index 0eb68b47354f..a4a8c0305fb9 100644
> > --- a/drivers/pci/probe.c
> > +++ b/drivers/pci/probe.c
> > @@ -2153,7 +2153,7 @@ static void pci_configure_ltr(struct pci_dev *dev)
> > {
> > #ifdef CONFIG_PCIEASPM
> > struct pci_host_bridge *host = pci_find_host_bridge(dev->bus);
> > - struct pci_dev *bridge;
> > + struct pci_dev *bridge = NULL;
> > u32 cap, ctl;
> >
> > if (!pci_is_pcie(dev))
> > @@ -2191,6 +2191,21 @@ static void pci_configure_ltr(struct pci_dev *dev)
> > if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT ||
> > ((bridge = pci_upstream_bridge(dev)) &&
> > bridge->ltr_path)) {
> > + /*
> > + * Downstream ports reset the LTR enable bit when the
> > + * link goes down (e.g on hot-remove) so re-enable the
> > + * bit here if not set anymore.
> > + * PCIe r5.0, sec 7.5.3.16.
> > + */
> > + if (bridge) {
> > + pcie_capability_read_dword(bridge, PCI_EXP_DEVCTL2, &ctl);
> > + if (!(ctl & PCI_EXP_DEVCTL2_LTR_EN)) {
> > + pci_dbg(bridge, "re-enabling LTR\n");
> > + pcie_capability_set_word(bridge, PCI_EXP_DEVCTL2,
> > + PCI_EXP_DEVCTL2_LTR_EN);
> > + }
> > + }
> > + pci_dbg(dev, "enabling LTR\n");
> > pcie_capability_set_word(dev, PCI_EXP_DEVCTL2,
> > PCI_EXP_DEVCTL2_LTR_EN);
> > dev->ltr_path = 1;
> > --
> > 2.29.2
> >
next prev parent reply other threads:[~2021-01-22 7:13 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-19 13:14 [PATCH v2] PCI: Re-enable downstream port LTR if it was previously enabled Mika Westerberg
2021-01-21 22:31 ` Bjorn Helgaas
2021-01-22 7:03 ` Mingchuang Qiao [this message]
2021-01-22 10:05 ` Mika Westerberg
2021-01-22 13:20 ` Bjorn Helgaas
2021-01-25 10:14 ` Mingchuang Qiao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1611298991.5980.42.camel@mcddlt001 \
--to=mingchuang.qiao@mediatek.com \
--cc=alex.williamson@redhat.com \
--cc=bhelgaas@google.com \
--cc=haijun.liu@mediatek.com \
--cc=helgaas@kernel.org \
--cc=lambert.wang@mediatek.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mediatek@lists.infradead.org \
--cc=linux-pci@vger.kernel.org \
--cc=matthias.bgg@gmail.com \
--cc=mika.westerberg@linux.intel.com \
--cc=rjw@rjwysocki.net \
--cc=utkarsh.h.patel@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).