[PATCH v2 0/2] ACPI / PCI: Fix _PRT lookup for ARI enabled devices

[PATCH v2 0/2] ACPI / PCI: Fix _PRT lookup for ARI enabled devices

[Alex Williamson]

v2: don't modify entry->id.device

In most cases we only use ARI with SR-IOV VFs, which do not support
INTx and therefore never hit this problem.  However, some non-SR-IOV
implementations create multiple PFs, extending beyond the standard
3-bit function numbers with ARI, and do support INTx for those
additional functions.  This can happen with Solarflare SFC9120
adapters.  The host driver typically doesn't use INTx, so we also
haven't noticed this problem on bare metal, but when we attempt to
assign the device to a VM using vfio-pci, we fail trying to setup
default INTx signaling.  Thanks,

Alex

---

Alex Williamson (2):
      PCI: Move pci_ari_enabled() to global header
      ACPI / PCI: Account for ARI in _PRT lookups


 drivers/acpi/pci_irq.c |    2 +-
 drivers/pci/pci.h      |   11 -----------
 include/linux/pci.h    |   11 +++++++++++
 3 files changed, 12 insertions(+), 12 deletions(-)

[PATCH v2 1/2] PCI: Move pci_ari_enabled() to global header

[Alex Williamson]

This is useful outside of drivers/pci, particularly for deriving INTx
routing via ACPI _PRT.  Also convert to bool return.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 drivers/pci/pci.h   |   11 -----------
 include/linux/pci.h |   11 +++++++++++
 2 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 9bd762c2..c1b2a43 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -216,17 +216,6 @@ void __pci_bus_assign_resources(const struct pci_bus *bus,
 				struct list_head *fail_head);
 bool pci_bus_clip_resource(struct pci_dev *dev, int idx);
 
-/**
- * pci_ari_enabled - query ARI forwarding status
- * @bus: the PCI bus
- *
- * Returns 1 if ARI forwarding is enabled, or 0 if not enabled;
- */
-static inline int pci_ari_enabled(struct pci_bus *bus)
-{
-	return bus->self && bus->self->ari_enabled;
-}
-
 void pci_reassigndev_resource_alignment(struct pci_dev *dev);
 void pci_disable_bridge_window(struct pci_dev *dev);
 
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 353db8d..2925561 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1905,4 +1905,15 @@ static inline bool pci_is_dev_assigned(struct pci_dev *pdev)
 {
 	return (pdev->dev_flags & PCI_DEV_FLAGS_ASSIGNED) == PCI_DEV_FLAGS_ASSIGNED;
 }
+
+/**
+ * pci_ari_enabled - query ARI forwarding status
+ * @bus: the PCI bus
+ *
+ * Returns true if ARI forwarding is enabled.
+ */
+static inline bool pci_ari_enabled(struct pci_bus *bus)
+{
+	return bus->self && bus->self->ari_enabled;
+}
 #endif /* LINUX_PCI_H */

[PATCH v2 2/2] ACPI / PCI: Account for ARI in _PRT lookups

[Alex Williamson]

The PCIe specification, rev 3.0, section 2.2.8.1, contains the
following implementation note:

    Virtual Wire Mapping for INTx Interrupts From ARI Devices

    The implied Device Number for an ARI Device is 0. When ARI-aware
    software (including BIOS and operating system) enables ARI
    Forwarding in the Downstream Port immediately above an ARI Device
    in order to access its Extended Functions, software must
    comprehend that the Downstream Port will use Device Number 0 for
    the virtual wire mappings of INTx interrupts coming from all
    Functions of the ARI Device. If non-ARI-aware software attempts
    to determine the virtual wire mappings for Extended Functions, it
    can come up with incorrect mappings by examining the traditional
    Device Number field and finding it to be non-0.

We account for this in pci_swizzle_interrupt_pin(), but it looks like
we miss it here, looking for a _PRT entry with a slot matching the
ARI device slot number.  This can cause errors like:

pcieport 0000:80:03.0: can't derive routing for PCI INT B
sfc 0000:82:01.1: PCI INT B: no GSI

pci_dev.irq is then invalid, resulting in errors for drivers that
attempt to enable INTx on the device.  Fix by using slot 0 for ARI
enabled devices.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 drivers/acpi/pci_irq.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/acpi/pci_irq.c b/drivers/acpi/pci_irq.c
index b1def41..4db10b1 100644
--- a/drivers/acpi/pci_irq.c
+++ b/drivers/acpi/pci_irq.c
@@ -163,7 +163,7 @@ static int acpi_pci_irq_check_entry(acpi_handle handle, struct pci_dev *dev,
 {
 	int segment = pci_domain_nr(dev->bus);
 	int bus = dev->bus->number;
-	int device = PCI_SLOT(dev->devfn);
+	int device = pci_ari_enabled(dev->bus) ? 0 : PCI_SLOT(dev->devfn);
 	struct acpi_prt_entry *entry;
 
 	if (((prt->address >> 16) & 0xffff) != device ||

Re: [PATCH v2 0/2] ACPI / PCI: Fix _PRT lookup for ARI enabled devices

[Don Dutile]

On 05/26/2015 05:11 PM, Alex Williamson wrote:
> v2: don't modify entry->id.device
>
> In most cases we only use ARI with SR-IOV VFs, which do not support
> INTx and therefore never hit this problem.  However, some non-SR-IOV
> implementations create multiple PFs, extending beyond the standard
> 3-bit function numbers with ARI, and do support INTx for those
> additional functions.  This can happen with Solarflare SFC9120
> adapters.  The host driver typically doesn't use INTx, so we also
> haven't noticed this problem on bare metal, but when we attempt to
> assign the device to a VM using vfio-pci, we fail trying to setup
> default INTx signaling.  Thanks,
>
> Alex
>
> ---
>
> Alex Williamson (2):
>        PCI: Move pci_ari_enabled() to global header
>        ACPI / PCI: Account for ARI in _PRT lookups
>
>
>   drivers/acpi/pci_irq.c |    2 +-
>   drivers/pci/pci.h      |   11 -----------
>   include/linux/pci.h    |   11 +++++++++++
>   3 files changed, 12 insertions(+), 12 deletions(-)
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
Feel free to add my Reviewed-by / Acked-by

-dd

Re: [PATCH v2 0/2] ACPI / PCI: Fix _PRT lookup for ARI enabled devices

[Bjorn Helgaas]

On Tue, May 26, 2015 at 03:11:38PM -0600, Alex Williamson wrote:
> v2: don't modify entry->id.device
> 
> In most cases we only use ARI with SR-IOV VFs, which do not support
> INTx and therefore never hit this problem.  However, some non-SR-IOV
> implementations create multiple PFs, extending beyond the standard
> 3-bit function numbers with ARI, and do support INTx for those
> additional functions.  This can happen with Solarflare SFC9120
> adapters.  The host driver typically doesn't use INTx, so we also
> haven't noticed this problem on bare metal, but when we attempt to
> assign the device to a VM using vfio-pci, we fail trying to setup
> default INTx signaling.  Thanks,
> 
> Alex
> 
> ---
> 
> Alex Williamson (2):
>       PCI: Move pci_ari_enabled() to global header
>       ACPI / PCI: Account for ARI in _PRT lookups

Rafael, do you want this or should I take it?

I put it on my pci/virtualization branch for v4.2.  If you want it, let me
know and I'll drop it and you can add my:

Acked-by: Bjorn Helgaas <bhelgaas@google.com>

Re: [PATCH v2 0/2] ACPI / PCI: Fix _PRT lookup for ARI enabled devices

[Rafael J. Wysocki]

On Friday, May 29, 2015 05:18:48 PM Bjorn Helgaas wrote:
> On Tue, May 26, 2015 at 03:11:38PM -0600, Alex Williamson wrote:
> > v2: don't modify entry->id.device
> > 
> > In most cases we only use ARI with SR-IOV VFs, which do not support
> > INTx and therefore never hit this problem.  However, some non-SR-IOV
> > implementations create multiple PFs, extending beyond the standard
> > 3-bit function numbers with ARI, and do support INTx for those
> > additional functions.  This can happen with Solarflare SFC9120
> > adapters.  The host driver typically doesn't use INTx, so we also
> > haven't noticed this problem on bare metal, but when we attempt to
> > assign the device to a VM using vfio-pci, we fail trying to setup
> > default INTx signaling.  Thanks,
> > 
> > Alex
> > 
> > ---
> > 
> > Alex Williamson (2):
> >       PCI: Move pci_ari_enabled() to global header
> >       ACPI / PCI: Account for ARI in _PRT lookups
> 
> Rafael, do you want this or should I take it?
> 
> I put it on my pci/virtualization branch for v4.2.

That's fine.  Please feel free to add my ACKs to those if that helps.

Rafael