[PATCH v1 0/3] PCI: Avoid touching non-compliant BARs

[PATCH v1 0/3] PCI: Avoid touching non-compliant BARs

[Bjorn Helgaas]

Hi guys,

Here's a proposal for dealing with these devices that have non-BAR
registers where BARs are supposed to be.  The idea is to:

  - have an early quirk mark these devices,
  - disable IO & MEM decoding so the devices don't consume address space
    we don't know about, and
  - skip BAR sizing (so the struct resources stay zeroed out)

This is basically a modification of what Andi originally proposed.  The
difference here is that we never touch the BAR at all, and we don't have to
add any struct resource flags, so we don't have to worry about changing any
consumers of the struct resources.

Let me know what you think.

---

Bjorn Helgaas (3):
      PCI: Disable IO/MEM decoding for devices with non-compliant BARs
      x86/PCI: Mark Broadwell-EP Home Agent & PCU as having non-compliant BARs
      PCI: Mark Broadcom Vulcan bridges as having non-compliant BARs


 arch/x86/pci/fixup.c |    7 +++++++
 drivers/pci/probe.c  |   14 ++++++++++++++
 drivers/pci/quirks.c |   14 ++++++++++++++
 include/linux/pci.h  |    1 +
 4 files changed, 36 insertions(+)

[PATCH v1 0/3] PCI: Avoid touching non-compliant BARs

[Bjorn Helgaas]

Hi guys,

Here's a proposal for dealing with these devices that have non-BAR
registers where BARs are supposed to be.  The idea is to:

  - have an early quirk mark these devices,
  - disable IO & MEM decoding so the devices don't consume address space
    we don't know about, and
  - skip BAR sizing (so the struct resources stay zeroed out)

This is basically a modification of what Andi originally proposed.  The
difference here is that we never touch the BAR at all, and we don't have to
add any struct resource flags, so we don't have to worry about changing any
consumers of the struct resources.

Let me know what you think.

---

Bjorn Helgaas (3):
      PCI: Disable IO/MEM decoding for devices with non-compliant BARs
      x86/PCI: Mark Broadwell-EP Home Agent & PCU as having non-compliant BARs
      PCI: Mark Broadcom Vulcan bridges as having non-compliant BARs


 arch/x86/pci/fixup.c |    7 +++++++
 drivers/pci/probe.c  |   14 ++++++++++++++
 drivers/pci/quirks.c |   14 ++++++++++++++
 include/linux/pci.h  |    1 +
 4 files changed, 36 insertions(+)

[PATCH v1 1/3] PCI: Disable IO/MEM decoding for devices with non-compliant BARs

[Bjorn Helgaas]

The PCI config header (first 64 bytes of each device's config space) is
defined by the PCI spec so generic software can identify the device and
manage its usage of I/O, memory, and IRQ resources.

Some non-spec-compliant devices put registers other than BARs where the
BARs should be.  When the PCI core sizes these "BARs", the reads and writes
it does may have unwanted side effects, and the "BAR" may appear to
describe non-sensical address space.

Add a flag bit to mark non-compliant devices so we don't touch their BARs.
Turn off IO/MEM decoding to prevent the devices from consuming address
space, since we can't read the BARs to find out what that address space
would be.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/pci/probe.c |   14 ++++++++++++++
 include/linux/pci.h |    1 +
 2 files changed, 15 insertions(+)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 6d7ab9b..6b0056e 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -179,6 +179,9 @@ int __pci_read_base(struct pci_dev *dev, enum pci_bar_type type,
 	u16 orig_cmd;
 	struct pci_bus_region region, inverted_region;
 
+	if (dev->non_compliant_bars)
+		return 0;
+
 	mask = type ? PCI_ROM_ADDRESS_MASK : ~0;
 
 	/* No printks while decoding is disabled! */
@@ -1171,6 +1174,7 @@ static void pci_msi_setup_pci_dev(struct pci_dev *dev)
 int pci_setup_device(struct pci_dev *dev)
 {
 	u32 class;
+	u16 cmd;
 	u8 hdr_type;
 	int pos = 0;
 	struct pci_bus_region region;
@@ -1214,6 +1218,16 @@ int pci_setup_device(struct pci_dev *dev)
 	/* device class may be changed after fixup */
 	class = dev->class >> 8;
 
+	if (dev->non_compliant_bars) {
+		pci_read_config_word(dev, PCI_COMMAND, &cmd);
+		if (cmd & (PCI_COMMAND_IO | PCI_COMMAND_MEMORY)) {
+			dev_info(&dev->dev, "device has non-compliant BARs; disabling IO/MEM decoding\n");
+			cmd &= ~PCI_COMMAND_IO;
+			cmd &= ~PCI_COMMAND_MEMORY;
+			pci_write_config_word(dev, PCI_COMMAND, cmd);
+		}
+	}
+
 	switch (dev->hdr_type) {		    /* header type */
 	case PCI_HEADER_TYPE_NORMAL:		    /* standard header */
 		if (class == PCI_CLASS_BRIDGE_PCI)
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 27df4a6..5f80661 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -359,6 +359,7 @@ struct pci_dev {
 	unsigned int	io_window_1k:1;	/* Intel P2P bridge 1K I/O windows */
 	unsigned int	irq_managed:1;
 	unsigned int	has_secondary_link:1;
+	unsigned int	non_compliant_bars:1;	/* broken BARs; ignore them */
 	pci_dev_flags_t dev_flags;
 	atomic_t	enable_cnt;	/* pci_enable_device has been called */
 

[PATCH v1 1/3] PCI: Disable IO/MEM decoding for devices with non-compliant BARs

[Bjorn Helgaas]

The PCI config header (first 64 bytes of each device's config space) is
defined by the PCI spec so generic software can identify the device and
manage its usage of I/O, memory, and IRQ resources.

Some non-spec-compliant devices put registers other than BARs where the
BARs should be.  When the PCI core sizes these "BARs", the reads and writes
it does may have unwanted side effects, and the "BAR" may appear to
describe non-sensical address space.

Add a flag bit to mark non-compliant devices so we don't touch their BARs.
Turn off IO/MEM decoding to prevent the devices from consuming address
space, since we can't read the BARs to find out what that address space
would be.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/pci/probe.c |   14 ++++++++++++++
 include/linux/pci.h |    1 +
 2 files changed, 15 insertions(+)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 6d7ab9b..6b0056e 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -179,6 +179,9 @@ int __pci_read_base(struct pci_dev *dev, enum pci_bar_type type,
 	u16 orig_cmd;
 	struct pci_bus_region region, inverted_region;
 
+	if (dev->non_compliant_bars)
+		return 0;
+
 	mask = type ? PCI_ROM_ADDRESS_MASK : ~0;
 
 	/* No printks while decoding is disabled! */
@@ -1171,6 +1174,7 @@ static void pci_msi_setup_pci_dev(struct pci_dev *dev)
 int pci_setup_device(struct pci_dev *dev)
 {
 	u32 class;
+	u16 cmd;
 	u8 hdr_type;
 	int pos = 0;
 	struct pci_bus_region region;
@@ -1214,6 +1218,16 @@ int pci_setup_device(struct pci_dev *dev)
 	/* device class may be changed after fixup */
 	class = dev->class >> 8;
 
+	if (dev->non_compliant_bars) {
+		pci_read_config_word(dev, PCI_COMMAND, &cmd);
+		if (cmd & (PCI_COMMAND_IO | PCI_COMMAND_MEMORY)) {
+			dev_info(&dev->dev, "device has non-compliant BARs; disabling IO/MEM decoding\n");
+			cmd &= ~PCI_COMMAND_IO;
+			cmd &= ~PCI_COMMAND_MEMORY;
+			pci_write_config_word(dev, PCI_COMMAND, cmd);
+		}
+	}
+
 	switch (dev->hdr_type) {		    /* header type */
 	case PCI_HEADER_TYPE_NORMAL:		    /* standard header */
 		if (class == PCI_CLASS_BRIDGE_PCI)
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 27df4a6..5f80661 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -359,6 +359,7 @@ struct pci_dev {
 	unsigned int	io_window_1k:1;	/* Intel P2P bridge 1K I/O windows */
 	unsigned int	irq_managed:1;
 	unsigned int	has_secondary_link:1;
+	unsigned int	non_compliant_bars:1;	/* broken BARs; ignore them */
 	pci_dev_flags_t dev_flags;
 	atomic_t	enable_cnt;	/* pci_enable_device has been called */
 

[PATCH v1 2/3] x86/PCI: Mark Broadwell-EP Home Agent & PCU as having non-compliant BARs

[Bjorn Helgaas]

The Home Agent and PCU PCI devices in Broadwell-EP have a non-BAR register
where a BAR should be.  We don't know what the side effects of sizing the
"BAR" would be, and we don't know what address space the "BAR" might appear
to describe.

Mark these devices as having non-compliant BARs so the PCI core doesn't
touch them.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 arch/x86/pci/fixup.c |    7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c
index e585655..0ae7e9f 100644
--- a/arch/x86/pci/fixup.c
+++ b/arch/x86/pci/fixup.c
@@ -540,3 +540,10 @@ static void twinhead_reserve_killing_zone(struct pci_dev *dev)
         }
 }
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x27B9, twinhead_reserve_killing_zone);
+
+static void pci_bdwep_bar(struct pci_dev *dev)
+{
+	dev->non_compliant_bars = 1;
+}
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fa0, pci_bdwep_bar);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fc0, pci_bdwep_bar);

[PATCH v1 2/3] x86/PCI: Mark Broadwell-EP Home Agent & PCU as having non-compliant BARs

[Bjorn Helgaas]

The Home Agent and PCU PCI devices in Broadwell-EP have a non-BAR register
where a BAR should be.  We don't know what the side effects of sizing the
"BAR" would be, and we don't know what address space the "BAR" might appear
to describe.

Mark these devices as having non-compliant BARs so the PCI core doesn't
touch them.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 arch/x86/pci/fixup.c |    7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c
index e585655..0ae7e9f 100644
--- a/arch/x86/pci/fixup.c
+++ b/arch/x86/pci/fixup.c
@@ -540,3 +540,10 @@ static void twinhead_reserve_killing_zone(struct pci_dev *dev)
         }
 }
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x27B9, twinhead_reserve_killing_zone);
+
+static void pci_bdwep_bar(struct pci_dev *dev)
+{
+	dev->non_compliant_bars = 1;
+}
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fa0, pci_bdwep_bar);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fc0, pci_bdwep_bar);

[PATCH v1 3/3] PCI: Mark Broadcom Vulcan bridges as having non-compliant BARs

[Bjorn Helgaas]

Broadcom Vulcan bridges have a BAR 0 for internal debug registers.  But
writing certain addresses to the BAR can trigger a hardware problem in the
device.  Linux doesn't need any BARs on these devices, so we can ignore
them.

Mark these bridges as having non-compliant BARs so the PCI core doesn't
touch them.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/pci/quirks.c |   14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 0575a1e..978c00c 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -3705,6 +3705,20 @@ DECLARE_PCI_FIXUP_HEADER(0x1283, 0x8892, quirk_use_pcie_bridge_dma_alias);
 DECLARE_PCI_FIXUP_HEADER(0x8086, 0x244e, quirk_use_pcie_bridge_dma_alias);
 
 /*
+ * Writing to BAR 0 on Broadcom Vulcan bridges can trigger a hardware
+ * problem in the device.  Linux doesn't need any BARs on these devices, so
+ * mark the device as having non-compliant BARs so we will ignore them.
+ */
+static void quirk_bridge_brcm_vulcan_internal(struct pci_dev *pdev)
+{
+	pdev->non_compliant_bars = 1;
+}
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_BROADCOM, 0x9000,
+			quirk_bridge_brcm_vulcan_internal);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_BROADCOM, 0x9039,
+			quirk_bridge_brcm_vulcan_internal);
+
+/*
  * Intersil/Techwell TW686[4589]-based video capture cards have an empty (zero)
  * class code.  Fix it.
  */

[PATCH v1 3/3] PCI: Mark Broadcom Vulcan bridges as having non-compliant BARs

[Bjorn Helgaas]

Broadcom Vulcan bridges have a BAR 0 for internal debug registers.  But
writing certain addresses to the BAR can trigger a hardware problem in the
device.  Linux doesn't need any BARs on these devices, so we can ignore
them.

Mark these bridges as having non-compliant BARs so the PCI core doesn't
touch them.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/pci/quirks.c |   14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 0575a1e..978c00c 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -3705,6 +3705,20 @@ DECLARE_PCI_FIXUP_HEADER(0x1283, 0x8892, quirk_use_pcie_bridge_dma_alias);
 DECLARE_PCI_FIXUP_HEADER(0x8086, 0x244e, quirk_use_pcie_bridge_dma_alias);
 
 /*
+ * Writing to BAR 0 on Broadcom Vulcan bridges can trigger a hardware
+ * problem in the device.  Linux doesn't need any BARs on these devices, so
+ * mark the device as having non-compliant BARs so we will ignore them.
+ */
+static void quirk_bridge_brcm_vulcan_internal(struct pci_dev *pdev)
+{
+	pdev->non_compliant_bars = 1;
+}
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_BROADCOM, 0x9000,
+			quirk_bridge_brcm_vulcan_internal);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_BROADCOM, 0x9039,
+			quirk_bridge_brcm_vulcan_internal);
+
+/*
  * Intersil/Techwell TW686[4589]-based video capture cards have an empty (zero)
  * class code.  Fix it.
  */

Re: [PATCH v1 0/3] PCI: Avoid touching non-compliant BARs

[Andi Kleen]

On Fri, Feb 26, 2016 at 09:58:42AM -0600, Bjorn Helgaas wrote:
> Hi guys,
> 
> Here's a proposal for dealing with these devices that have non-BAR
> registers where BARs are supposed to be.  The idea is to:
> 
>   - have an early quirk mark these devices,
>   - disable IO & MEM decoding so the devices don't consume address space
>     we don't know about, and
>   - skip BAR sizing (so the struct resources stay zeroed out)
> 
> This is basically a modification of what Andi originally proposed.  The
> difference here is that we never touch the BAR at all, and we don't have to
> add any struct resource flags, so we don't have to worry about changing any
> consumers of the struct resources.
> 
> Let me know what you think.

I tested the patch on Broadwell and it avoids the Firmware Bug messages.

Tested-by: Andi Kleen <ak@linux.intel.com>

-Andi

Re: [PATCH v1 0/3] PCI: Avoid touching non-compliant BARs

[Andi Kleen]

On Fri, Feb 26, 2016 at 09:58:42AM -0600, Bjorn Helgaas wrote:
> Hi guys,
> 
> Here's a proposal for dealing with these devices that have non-BAR
> registers where BARs are supposed to be.  The idea is to:
> 
>   - have an early quirk mark these devices,
>   - disable IO & MEM decoding so the devices don't consume address space
>     we don't know about, and
>   - skip BAR sizing (so the struct resources stay zeroed out)
> 
> This is basically a modification of what Andi originally proposed.  The
> difference here is that we never touch the BAR at all, and we don't have to
> add any struct resource flags, so we don't have to worry about changing any
> consumers of the struct resources.
> 
> Let me know what you think.

I tested the patch on Broadwell and it avoids the Firmware Bug messages.

Tested-by: Andi Kleen <ak@linux.intel.com>

-Andi

Re: [PATCH v1 0/3] PCI: Avoid touching non-compliant BARs

[Bjorn Helgaas]

On Fri, Feb 26, 2016 at 09:58:42AM -0600, Bjorn Helgaas wrote:
> Hi guys,
> 
> Here's a proposal for dealing with these devices that have non-BAR
> registers where BARs are supposed to be.  The idea is to:
> 
>   - have an early quirk mark these devices,
>   - disable IO & MEM decoding so the devices don't consume address space
>     we don't know about, and
>   - skip BAR sizing (so the struct resources stay zeroed out)
> 
> This is basically a modification of what Andi originally proposed.  The
> difference here is that we never touch the BAR at all, and we don't have to
> add any struct resource flags, so we don't have to worry about changing any
> consumers of the struct resources.
> 
> Let me know what you think.
> 
> ---
> 
> Bjorn Helgaas (3):
>       PCI: Disable IO/MEM decoding for devices with non-compliant BARs
>       x86/PCI: Mark Broadwell-EP Home Agent & PCU as having non-compliant BARs
>       PCI: Mark Broadcom Vulcan bridges as having non-compliant BARs
> 
> 
>  arch/x86/pci/fixup.c |    7 +++++++
>  drivers/pci/probe.c  |   14 ++++++++++++++
>  drivers/pci/quirks.c |   14 ++++++++++++++
>  include/linux/pci.h  |    1 +
>  4 files changed, 36 insertions(+)

I applied the first two patches, with Andi's Tested-by, to
pci/resource for v4.6.

Per Jayachandran, the Broadcom patch turned out not to be necessary,
so I dropped that one.

Bjorn

Re: [PATCH v1 0/3] PCI: Avoid touching non-compliant BARs

[Bjorn Helgaas]

On Fri, Feb 26, 2016 at 09:58:42AM -0600, Bjorn Helgaas wrote:
> Hi guys,
> 
> Here's a proposal for dealing with these devices that have non-BAR
> registers where BARs are supposed to be.  The idea is to:
> 
>   - have an early quirk mark these devices,
>   - disable IO & MEM decoding so the devices don't consume address space
>     we don't know about, and
>   - skip BAR sizing (so the struct resources stay zeroed out)
> 
> This is basically a modification of what Andi originally proposed.  The
> difference here is that we never touch the BAR at all, and we don't have to
> add any struct resource flags, so we don't have to worry about changing any
> consumers of the struct resources.
> 
> Let me know what you think.
> 
> ---
> 
> Bjorn Helgaas (3):
>       PCI: Disable IO/MEM decoding for devices with non-compliant BARs
>       x86/PCI: Mark Broadwell-EP Home Agent & PCU as having non-compliant BARs
>       PCI: Mark Broadcom Vulcan bridges as having non-compliant BARs
> 
> 
>  arch/x86/pci/fixup.c |    7 +++++++
>  drivers/pci/probe.c  |   14 ++++++++++++++
>  drivers/pci/quirks.c |   14 ++++++++++++++
>  include/linux/pci.h  |    1 +
>  4 files changed, 36 insertions(+)

I applied the first two patches, with Andi's Tested-by, to
pci/resource for v4.6.

Per Jayachandran, the Broadcom patch turned out not to be necessary,
so I dropped that one.

Bjorn

Re: [PATCH v1 0/3] PCI: Avoid touching non-compliant BARs

[Andi Kleen]

> I applied the first two patches, with Andi's Tested-by, to
> pci/resource for v4.6.
> 
> Per Jayachandran, the Broadcom patch turned out not to be necessary,
> so I dropped that one.

Thanks. Can the patches please also be marked Cc: stable
for earlier kernels.

-Andi

Re: [PATCH v1 0/3] PCI: Avoid touching non-compliant BARs

[Andi Kleen]

> I applied the first two patches, with Andi's Tested-by, to
> pci/resource for v4.6.
> 
> Per Jayachandran, the Broadcom patch turned out not to be necessary,
> so I dropped that one.

Thanks. Can the patches please also be marked Cc: stable
for earlier kernels.

-Andi

Re: [PATCH v1 0/3] PCI: Avoid touching non-compliant BARs

[Bjorn Helgaas]

On Tue, Mar 08, 2016 at 10:05:20AM -0800, Andi Kleen wrote:
> > I applied the first two patches, with Andi's Tested-by, to
> > pci/resource for v4.6.
> > 
> > Per Jayachandran, the Broadcom patch turned out not to be necessary,
> > so I dropped that one.
> 
> Thanks. Can the patches please also be marked Cc: stable
> for earlier kernels.

Sure, done.  I updated the pci/resource and next branches.

Re: [PATCH v1 0/3] PCI: Avoid touching non-compliant BARs

[Bjorn Helgaas]

On Tue, Mar 08, 2016 at 10:05:20AM -0800, Andi Kleen wrote:
> > I applied the first two patches, with Andi's Tested-by, to
> > pci/resource for v4.6.
> > 
> > Per Jayachandran, the Broadcom patch turned out not to be necessary,
> > so I dropped that one.
> 
> Thanks. Can the patches please also be marked Cc: stable
> for earlier kernels.

Sure, done.  I updated the pci/resource and next branches.