Patch "PCI: Set Read Completion Boundary to 128 iff Root Port supports it (_HPX)" has been added to the 4.8-stable tree

* Patch "PCI: Set Read Completion Boundary to 128 iff Root Port supports it (_HPX)" has been added to the 4.8-stable tree
@ 2016-12-04 11:45 gregkh
  2016-12-05  1:37 ` Stefan Lippers-Hollmann
  0 siblings, 1 reply; 5+ messages in thread
From: gregkh @ 2016-12-04 11:45 UTC (permalink / raw)
  To: jthumshirn; +Cc: stable, stable-commits


This is a note to let you know that I've just added the patch titled

    PCI: Set Read Completion Boundary to 128 iff Root Port supports it (_HPX)

to the 4.8-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     pci-set-read-completion-boundary-to-128-iff-root-port-supports-it-_hpx.patch
and it can be found in the queue-4.8 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@vger.kernel.org> know about it.


>From e42010d8207f9d15a605ceb8e321bcd9648071b0 Mon Sep 17 00:00:00 2001
From: Johannes Thumshirn <jthumshirn@suse.de>
Date: Wed, 23 Nov 2016 10:56:28 -0600
Subject: PCI: Set Read Completion Boundary to 128 iff Root Port supports it (_HPX)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Johannes Thumshirn <jthumshirn@suse.de>

commit e42010d8207f9d15a605ceb8e321bcd9648071b0 upstream.

Per PCIe spec r3.0, sec 2.3.1.1, the Read Completion Boundary (RCB)
determines the naturally aligned address boundaries on which a Read Request
may be serviced with multiple Completions:

  - For a Root Complex, RCB is 64 bytes or 128 bytes
    This value is reported in the Link Control Register

    Note: Bridges and Endpoints may implement a corresponding command bit
    which may be set by system software to indicate the RCB value for the
    Root Complex, allowing the Bridge/Endpoint to optimize its behavior
    when the Root Complex’s RCB is 128 bytes.

  - For all other system elements, RCB is 128 bytes

Per sec 7.8.7, if a Root Port only supports a 64-byte RCB, the RCB of all
downstream devices must be clear, indicating an RCB of 64 bytes.  If the
Root Port supports a 128-byte RCB, we may optionally set the RCB of
downstream devices so they know they can generate larger Completions.

Some BIOSes supply an _HPX that tells us to set RCB, even though the Root
Port doesn't have RCB set, which may lead to Malformed TLP errors if the
Endpoint generates completions larger than the Root Port can handle.

The IBM x3850 X6 with BIOS version -[A8E120CUS-1.30]- 08/22/2016 supplies
such an _HPX and a Mellanox MT27500 ConnectX-3 device fails to initialize:

  mlx4_core 0000:41:00.0: command 0xfff timed out (go bit not cleared)
  mlx4_core 0000:41:00.0: device is going to be reset
  mlx4_core 0000:41:00.0: Failed to obtain HW semaphore, aborting
  mlx4_core 0000:41:00.0: Fail to reset HCA
  ------------[ cut here ]------------
  kernel BUG at drivers/net/ethernet/mellanox/mlx4/catas.c:193!

After 6cd33649fa83 ("PCI: Add pci_configure_device() during enumeration")
and 7a1562d4f2d0 ("PCI: Apply _HPX Link Control settings to all devices
with a link"), we apply _HPX settings to *all* devices, not just those
hot-added after boot.

Before 7a1562d4f2d0, we didn't touch the Mellanox RCB, and the device
worked.  After 7a1562d4f2d0, we set its RCB to 128, and it failed.

Set the RCB to 128 iff the Root Port supports a 128-byte RCB.  Otherwise,
set RCB to 64 bytes.  This effectively ignores what _HPX tells us about
RCB.

Note that this change only affects _HPX handling.  If we have no _HPX, this
does nothing with RCB.

[bhelgaas: changelog, clear RCB if not set for Root Port]
Fixes: 6cd33649fa83 ("PCI: Add pci_configure_device() during enumeration")
Fixes: 7a1562d4f2d0 ("PCI: Apply _HPX Link Control settings to all devices with a link")
Link: https://bugzilla.kernel.org/show_bug.cgi?id=187781
Tested-by: Frank Danapfel <fdanapfe@redhat.com>
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Myron Stowe <myron.stowe@redhat.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 drivers/pci/probe.c |   28 +++++++++++++++++++++++++++-
 1 file changed, 27 insertions(+), 1 deletion(-)

--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1439,6 +1439,21 @@ static void program_hpp_type1(struct pci
 		dev_warn(&dev->dev, "PCI-X settings not supported\n");
 }
 
+static bool pcie_root_rcb_set(struct pci_dev *dev)
+{
+	struct pci_dev *rp = pcie_find_root_port(dev);
+	u16 lnkctl;
+
+	if (!rp)
+		return false;
+
+	pcie_capability_read_word(rp, PCI_EXP_LNKCTL, &lnkctl);
+	if (lnkctl & PCI_EXP_LNKCTL_RCB)
+		return true;
+
+	return false;
+}
+
 static void program_hpp_type2(struct pci_dev *dev, struct hpp_type2 *hpp)
 {
 	int pos;
@@ -1468,9 +1483,20 @@ static void program_hpp_type2(struct pci
 			~hpp->pci_exp_devctl_and, hpp->pci_exp_devctl_or);
 
 	/* Initialize Link Control Register */
-	if (pcie_cap_has_lnkctl(dev))
+	if (pcie_cap_has_lnkctl(dev)) {
+
+		/*
+		 * If the Root Port supports Read Completion Boundary of
+		 * 128, set RCB to 128.  Otherwise, clear it.
+		 */
+		hpp->pci_exp_lnkctl_and |= PCI_EXP_LNKCTL_RCB;
+		hpp->pci_exp_lnkctl_or &= ~PCI_EXP_LNKCTL_RCB;
+		if (pcie_root_rcb_set(dev))
+			hpp->pci_exp_lnkctl_or |= PCI_EXP_LNKCTL_RCB;
+
 		pcie_capability_clear_and_set_word(dev, PCI_EXP_LNKCTL,
 			~hpp->pci_exp_lnkctl_and, hpp->pci_exp_lnkctl_or);
+	}
 
 	/* Find Advanced Error Reporting Enhanced Capability */
 	pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ERR);


Patches currently in stable-queue which might be from jthumshirn@suse.de are

queue-4.8/scsi-libfc-fix-seconds_since_last_reset-miscalculation.patch
queue-4.8/pci-set-read-completion-boundary-to-128-iff-root-port-supports-it-_hpx.patch

^ permalink raw reply	[flat|nested] 5+ messages in thread