From: helgaas@kernel.org (Bjorn Helgaas)
To: linux-arm-kernel@lists.infradead.org
Subject: [PATCH v9 2/4] PCI: Disable PCIe Relaxed Ordering if unsupported
Date: Tue, 8 Aug 2017 22:25:03 -0500 [thread overview]
Message-ID: <20170809032503.GB7191@bhelgaas-glaptop.roam.corp.google.com> (raw)
In-Reply-To: <20170809022239.GP16580@bhelgaas-glaptop.roam.corp.google.com>
On Tue, Aug 08, 2017 at 09:22:39PM -0500, Bjorn Helgaas wrote:
> On Sat, Aug 05, 2017 at 03:15:11PM +0800, Ding Tianhong wrote:
> > When bit4 is set in the PCIe Device Control register, it indicates
> > whether the device is permitted to use relaxed ordering.
> > On some platforms using relaxed ordering can have performance issues or
> > due to erratum can cause data-corruption. In such cases devices must avoid
> > using relaxed ordering.
> >
> > This patch checks if there is any node in the hierarchy that indicates that
> > using relaxed ordering is not safe.
...
> > +EXPORT_SYMBOL(pcie_relaxed_ordering_supported);
>
> This is misnamed. This doesn't tell us anything about whether the
> device *supports* relaxed ordering. It only tells us whether the
> device is *permitted* to use it.
>
> When a device initiates a transaction, the hardware should set the RO
> bit in the TLP with logic something like this:
>
> RO = <this Function supports relaxed ordering> &&
> <this transaction doesn't require strong write ordering> &&
> <PCI_EXP_DEVCTL_RELAX_EN is set>
>
> The issue you're fixing is that some Completers don't handle RO
> correctly. The determining factor is not the Requester, but the
> Completer (for this series, a Root Port). So I think this should be
> something like:
>
> int pcie_relaxed_ordering_broken(struct pci_dev *completer)
> {
> if (!completer)
> return 0;
>
> return completer->dev_flags & PCI_DEV_FLAGS_NO_RELAXED_ORDERING;
> }
>
> and the caller should do something like this:
>
> if (pcie_relaxed_ordering_broken(pci_find_pcie_root_port(pdev)))
> adapter->flags |= ROOT_NO_RELAXED_ORDERING;
>
> That way it's obvious where the issue is, and it's obvious that the
> answer might be different for peer-to-peer transactions than it is for
> transactions to the root port, i.e., to coherent memory.
After looking at the driver, I wonder if it would be simpler like
this:
int pcie_relaxed_ordering_enabled(struct pci_dev *dev)
{
u16 ctl;
pcie_capability_read_word(dev, PCI_EXP_DEVCTL, &ctl);
return ctl & PCI_EXP_DEVCTL_RELAX_EN;
}
EXPORT_SYMBOL(pcie_relaxed_ordering_enabled);
static void pci_configure_relaxed_ordering(struct pci_dev *dev)
{
struct pci_dev *root;
if (dev->is_virtfn)
return; /* PCI_EXP_DEVCTL_RELAX_EN is RsvdP in VFs */
if (!pcie_relaxed_ordering_enabled(dev))
return;
/*
* For now, we only deal with Relaxed Ordering issues with Root
* Ports. Peer-to-peer DMA is another can of worms.
*/
root = pci_find_pcie_root_port(dev);
if (!root)
return;
if (root->relaxed_ordering_broken)
pcie_capability_clear_word(dev, PCI_EXP_DEVCTL,
PCI_EXP_DEVCTL_RELAX_EN);
}
This doesn't check every intervening switch, but I don't think we know
about any issues except with root ports.
And the driver could do:
if (!pcie_relaxed_ordering_enabled(pdev))
adapter->flags |= ROOT_NO_RELAXED_ORDERING;
The driver code wouldn't show anything about coherent memory vs.
peer-to-peer, but we really don't have a clue about how to handle that
yet anyway.
I guess this is back to exactly what you proposed, except that I
changed the name of pcie_relaxed_ordering_supported() to
pcie_relaxed_ordering_enabled(), which I think is slightly more
specific from the device's point of view.
Bjorn
next prev parent reply other threads:[~2017-08-09 3:25 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-08-05 7:15 [PATCH v9 0/4] Add new PCI_DEV_FLAGS_NO_RELAXED_ORDERING flag Ding Tianhong
2017-08-05 7:15 ` [PATCH v9 1/4] PCI: Add new PCIe Fabric End Node flag, PCI_DEV_FLAGS_NO_RELAXED_ORDERING Ding Tianhong
2017-08-08 23:22 ` Bjorn Helgaas
2017-08-09 1:40 ` Casey Leedom
2017-08-09 3:02 ` Bjorn Helgaas
2017-08-09 12:17 ` Ding Tianhong
2017-08-09 16:36 ` Casey Leedom
2017-08-09 15:58 ` Raj, Ashok
2017-08-09 16:46 ` Casey Leedom
2017-08-09 18:00 ` Raj, Ashok
2017-08-09 20:11 ` Casey Leedom
2017-08-05 7:15 ` [PATCH v9 2/4] PCI: Disable PCIe Relaxed Ordering if unsupported Ding Tianhong
2017-08-09 2:22 ` Bjorn Helgaas
2017-08-09 3:25 ` Bjorn Helgaas [this message]
2017-08-09 13:42 ` Ding Tianhong
2017-08-09 12:33 ` Casey Leedom
2017-08-09 13:23 ` Ding Tianhong
2017-08-05 7:15 ` [PATCH v9 3/4] net/cxgb4: Use new PCI_DEV_FLAGS_NO_RELAXED_ORDERING flag Ding Tianhong
2017-08-05 7:15 ` [PATCH v9 4/4] net/cxgb4vf: " Ding Tianhong
2017-08-07 3:47 ` [PATCH v9 0/4] Add " David Miller
2017-08-07 4:13 ` Ding Tianhong
2017-08-07 21:14 ` David Miller
2017-08-08 1:56 ` Bjorn Helgaas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170809032503.GB7191@bhelgaas-glaptop.roam.corp.google.com \
--to=helgaas@kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).