[PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets

[PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets

[Sean V Kelley]

Changes since v1 [1]:

- Correct Documentation section title for 6300ESB chipset.
(Jonathan Derrick)

- Use consistent abbreviations in comments for IO-APIC and Core IO.
(Andy Shevchenko)

- Retained Reviewed-by tag due to no technical changes.

[1]: https://lore.kernel.org/lkml/20200214213313.66622-1-sean.v.kelley@linux.intel.com/

Bjorn, I'm open for it to go to stable as well.

--

When IRQ lines on secondary or higher IO-APICs are masked (e.g.,
Real-Time threaded interrupts), many chipsets redirect IRQs on
this line to the legacy PCH and in turn the base IO-APIC in the
system. The unhandled interrupts on the base IO-APIC will be
identified by the Linux kernel as Spurious Interrupts and can
lead to disabled IRQ lines.

Disabling this legacy PCI interrupt routing is chipset-specific and
varies in mechanism between chipset vendors and across generations.
In some cases the mechanism is exposed to BIOS but not all BIOS
vendors chose to pick it up. With the increasing usage of RT as it
marches towards mainline, additional issues have been raised with
more recent Xeon chipsets.

This patchset disables the boot interrupt on these Xeon chipsets where
this is possible with an additional mechanism. In addition, this
patchset includes documentation covering the background of this quirk.


Sean V Kelley (2):
  pci: Add boot interrupt quirk mechanism for Xeon chipsets
  Documentation:PCI: Add background on Boot Interrupts

 Documentation/PCI/boot-interrupts.rst | 153 ++++++++++++++++++++++++++
 Documentation/PCI/index.rst           |   1 +
 drivers/pci/quirks.c                  |  80 ++++++++++++--
 3 files changed, 227 insertions(+), 7 deletions(-)
 create mode 100644 Documentation/PCI/boot-interrupts.rst

--
2.25.1

[PATCH v2 1/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets

[Sean V Kelley]

The following was observed by Kar Hin Ong with RT patchset:
Backtrace:
irq 19: nobody cared (try booting with the "irqpoll" option)
CPU: 0 PID: 3329 Comm: irq/34-nipalk Tainted:4.14.87-rt49 #1
Hardware name: National Instruments NI PXIe-8880/NI PXIe-8880,
         BIOS 2.1.5f1 01/09/2020
Call Trace:
<IRQ>
  ? dump_stack+0x46/0x5e
  ? __report_bad_irq+0x2e/0xb0
  ? note_interrupt+0x242/0x290
  ? nNIKAL100_memoryRead16+0x8/0x10 [nikal]
  ? handle_irq_event_percpu+0x55/0x70
  ? handle_irq_event+0x4f/0x80
  ? handle_fasteoi_irq+0x81/0x180
  ? handle_irq+0x1c/0x30
  ? do_IRQ+0x41/0xd0
  ? common_interrupt+0x84/0x84
</IRQ>
...
handlers:
[<ffffffffb3297200>] irq_default_primary_handler threaded
[<ffffffffb3669180>] usb_hcd_irq
Disabling IRQ #19

The problem being that this device is triggering boot interrupts
due to threaded interrupt handling and masking of the IO-APIC. These
boot interrupts are then forwarded on to the legacy PCH's PIRQ lines
where there is no handler present for the device.

Whenever a PCI device is firing interrupt (INTx) to Pin 20 of IOAPIC 2
(GSI 44); the kernel will receives 2 interrupts:
   1. Interrupt from Pin 20 of IOAPIC 2  -> Expected
   2. Interrupt from Pin 19 of IOAPIC 1  -> UNEXPECTED

Quirks for disabling boot interrupts (preferred) or rerouting the handler
exist but do not address these Xeon chipsets' mechanism:
https://lore.kernel.org/lkml/12131949181903-git-send-email-sassmann@suse.de/

This patch adds a new mechanism via PCI CFG for those chipsets supporting
CIPINTRC register's dis_intx_rout2ich bit.

Reported-by: Kar Hin Ong <kar.hin.ong@ni.com>
Tested-by: Kar Hin Ong <kar.hin.ong@ni.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Sean V Kelley <sean.v.kelley@linux.intel.com>
---
 drivers/pci/quirks.c | 80 ++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 73 insertions(+), 7 deletions(-)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 29f473ebf20f..b7347bc6a24d 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -1970,26 +1970,92 @@ DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,	PCI_DEVICE_ID_INTEL_80332_1,	quirk
 /*
  * IO-APIC1 on 6300ESB generates boot interrupts, see Intel order no
  * 300641-004US, section 5.7.3.
+ *
+ * Core IO on Xeon E5 1600/2600/4600, see Intel order no 326509-003.
+ * Core IO on Xeon E5 v2, see Intel order no 329188-003.
+ * Core IO on Xeon E7 v2, see Intel order no 329595-002.
+ * Core IO on Xeon E5 v3, see Intel order no 330784-003.
+ * Core IO on Xeon E7 v3, see Intel order no 332315-001US.
+ * Core IO on Xeon E5 v4, see Intel order no 333810-002US.
+ * Core IO on Xeon E7 v4, see Intel order no 332315-001US.
+ * Core IO on Xeon D-1500, see Intel order no 332051-001.
+ * Core IO on Xeon Scalable, see Intel order no 610950.
  */
-#define INTEL_6300_IOAPIC_ABAR		0x40
+#define INTEL_6300_IOAPIC_ABAR		0x40	/* Bus 0, Dev 29, Func 5 */
 #define INTEL_6300_DISABLE_BOOT_IRQ	(1<<14)
 
+#define INTEL_CIPINTRC_CFG_OFFSET	0x14C	/* Bus 0, Dev 5, Func 0 */
+#define INTEL_CIPINTRC_DIS_INTX_ICH	(1<<25)
+
 static void quirk_disable_intel_boot_interrupt(struct pci_dev *dev)
 {
 	u16 pci_config_word;
+	u32 pci_config_dword;
 
 	if (noioapicquirk)
 		return;
 
-	pci_read_config_word(dev, INTEL_6300_IOAPIC_ABAR, &pci_config_word);
-	pci_config_word |= INTEL_6300_DISABLE_BOOT_IRQ;
-	pci_write_config_word(dev, INTEL_6300_IOAPIC_ABAR, pci_config_word);
-
+	switch (dev->device) {
+	case PCI_DEVICE_ID_INTEL_ESB_10:
+		pci_read_config_word(dev, INTEL_6300_IOAPIC_ABAR,
+				     &pci_config_word);
+		pci_config_word |= INTEL_6300_DISABLE_BOOT_IRQ;
+		pci_write_config_word(dev, INTEL_6300_IOAPIC_ABAR,
+				      pci_config_word);
+		break;
+	case 0x3c28:	/* Xeon E5 1600/2600/4600	*/
+	case 0x0e28:	/* Xeon E5/E7 V2		*/
+	case 0x2f28:	/* Xeon E5/E7 V3,V4		*/
+	case 0x6f28:	/* Xeon D-1500			*/
+	case 0x2034:	/* Xeon Scalable Family		*/
+		pci_read_config_dword(dev, INTEL_CIPINTRC_CFG_OFFSET,
+				      &pci_config_dword);
+		pci_config_dword |= INTEL_CIPINTRC_DIS_INTX_ICH;
+		pci_write_config_dword(dev, INTEL_CIPINTRC_CFG_OFFSET,
+				       pci_config_dword);
+		break;
+	default:
+		return;
+	}
 	pci_info(dev, "disabled boot interrupts on device [%04x:%04x]\n",
 		 dev->vendor, dev->device);
 }
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,   PCI_DEVICE_ID_INTEL_ESB_10,	quirk_disable_intel_boot_interrupt);
-DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,   PCI_DEVICE_ID_INTEL_ESB_10,	quirk_disable_intel_boot_interrupt);
+/*
+ * Device 29 Func 5 Device IDs of IO-APIC
+ * containing ABAR—APIC1 Alternate Base Address Register
+ */
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,	PCI_DEVICE_ID_INTEL_ESB_10,
+		quirk_disable_intel_boot_interrupt);
+DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,	PCI_DEVICE_ID_INTEL_ESB_10,
+		quirk_disable_intel_boot_interrupt);
+
+/*
+ * Device 5 Func 0 Device IDs of Core IO modules/hubs
+ * containing Coherent Interface Protocol Interrupt Control
+ *
+ * Device IDs obtained from volume 2 datasheets of commented
+ * families above.
+ */
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,	0x3c28,
+		quirk_disable_intel_boot_interrupt);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,	0x0e28,
+		quirk_disable_intel_boot_interrupt);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,	0x2f28,
+		quirk_disable_intel_boot_interrupt);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,	0x6f28,
+		quirk_disable_intel_boot_interrupt);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,	0x2034,
+		quirk_disable_intel_boot_interrupt);
+DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,	0x3c28,
+		quirk_disable_intel_boot_interrupt);
+DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,	0x0e28,
+		quirk_disable_intel_boot_interrupt);
+DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,	0x2f28,
+		quirk_disable_intel_boot_interrupt);
+DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,	0x6f28,
+		quirk_disable_intel_boot_interrupt);
+DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL,	0x2034,
+		quirk_disable_intel_boot_interrupt);
 
 /* Disable boot interrupts on HT-1000 */
 #define BC_HT1000_FEATURE_REG		0x64
-- 
2.25.1

[PATCH v2 2/2] Documentation:PCI: Add background on Boot Interrupts

[Sean V Kelley]

Improve understanding of the PCI quirks for this legacy PCI interrupt
behavior to the benefit of developers and users alike.

Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Sean V Kelley <sean.v.kelley@linux.intel.com>
---
 Documentation/PCI/boot-interrupts.rst | 153 ++++++++++++++++++++++++++
 Documentation/PCI/index.rst           |   1 +
 2 files changed, 154 insertions(+)
 create mode 100644 Documentation/PCI/boot-interrupts.rst

diff --git a/Documentation/PCI/boot-interrupts.rst b/Documentation/PCI/boot-interrupts.rst
new file mode 100644
index 000000000000..b4d42481fd7f
--- /dev/null
+++ b/Documentation/PCI/boot-interrupts.rst
@@ -0,0 +1,153 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+===============
+Boot Interrupts
+===============
+
+:Author: - Sean V Kelley <sean.v.kelley@linux.intel.com>
+
+Overview
+========
+
+On PCI Express, interrupts are represented with either MSI or inbound interrupt
+messages (Assert_INTx/Deassert_INTx). The integrated IO-APIC in a given Core
+IO converts the legacy interrupt messages from PCI Express to MSI interrupts.
+If the IO-APIC is disabled (via the mask bits in the IO-APIC table entries),
+the messages are routed to the legacy PCH. This in-band interrupt mechanism was
+traditionally necessary for systems that did not support the IO-APIC and for
+boot. Intel in the past has used the term "boot interrupts" to describe this
+mechanism. Further, the PCI Express protocol describes this in-band legacy
+wire-interrupt INTx mechanism for I/O devices to signal PCI-style level
+interrupts. The subsequent paragraphs describe problems with the Core IO
+handling of INTx message routing to the PCH and mitigation within BIOS and
+the OS.
+
+
+Problem
+=======
+
+When in-band legacy INTx messages are forwarded to the PCH, they in turn
+trigger a new interrupt for which the OS likely lacks a handler. When an
+interrupt goes unhandled over time, they are tracked by the Linux kernel
+as Spurious Interrupts. The IRQ will be disabled by the Linux kernel after
+it reaches a specific count with the error "nobody cared". This disabled
+IRQ now prevents valid usage by an existing interrupt which may happen to
+share the IRQ line.
+
+irq 19: nobody cared (try booting with the "irqpoll" option)
+CPU: 0 PID: 2988 Comm: irq/34-nipalk Tainted: 4.14.87-rt49-02410-g4a640ec-dirty #1
+Hardware name: National Instruments NI PXIe-8880/NI PXIe-8880, BIOS 2.1.5f1 01/09/2020
+Call Trace:
+<IRQ>
+ ? dump_stack+0x46/0x5e
+ ? __report_bad_irq+0x2e/0xb0
+ ? note_interrupt+0x242/0x290
+ ? nNIKAL100_memoryRead16+0x8/0x10 [nikal]
+ ? handle_irq_event_percpu+0x55/0x70
+ ? handle_irq_event+0x4f/0x80
+ ? handle_fasteoi_irq+0x81/0x180
+ ? handle_irq+0x1c/0x30
+ ? do_IRQ+0x41/0xd0
+ ? common_interrupt+0x84/0x84
+</IRQ>
+
+handlers:
+irq_default_primary_handler threaded usb_hcd_irq
+Disabling IRQ #19
+
+
+Conditions
+==========
+
+The use of threaded interrupts is the most likely condition to trigger this
+problem today. Threaded interrupts may not be reenabled after the IRQ handler
+wakes. These "one shot" conditions mean that the threaded interrupt needs to
+keep the interrupt line masked until the threaded handler has run. Especially
+when dealing with high data rate interrupts, the thread needs to run to completion
+otherwise some handlers will end up in stack overflows since the interrupt
+of the issuing device is still active.
+
+Affected Chipsets
+=================
+
+The legacy interrupt forwarding mechansim exists today in a number of devices
+including but not limited to chipsets from AMD/ATI, Broadcom, and Intel. Changes
+made through the mitigations below have been applied to drivers/pci/quirks.c
+
+Starting with ICX there are no longer any IO-APICs in the Core IO's devices.
+IO-APIC is only in the PCH.  Devices connected to the Core IO's PCIE Root Ports
+will use native MSI/MSI-X mechanisms.
+
+Mitigations
+===========
+
+The mitigations take the form of PCI quirks. The preference has been to first
+identify and make use of a means to disable the routing to the PCH. In such a
+case a quirk to disable boot interrupt generation can be added.[1]
+
+Intel® 6300ESB I/O Controller Hub
+Alternate Base Address Register:
+ BIE: Boot Interrupt Enable
+	0 = Boot interrupt is enabled.
+	1 = Boot interrupt is disabled.
+
+Intel® Sandy Bridge through Sky Lake based Xeon servers:
+Coherent Interface Protocol Interrupt Control
+ dis_intx_route2pch/dis_intx_route2ich/dis_intx_route2dmi2:
+	When this bit is set. Local INTx messages received from the
+	Intel® Quick Data DMA/PCI Express ports are not routed to legacy
+	PCH - they are either converted into MSI via the integrated IO-APIC
+	(if the IO-APIC mask bit is clear in the appropriate entries)
+	or cause no further action (when mask bit is set)
+
+In the absence of a way to directly disable the routing, another approach
+has been to make use of PCI Interrupt pin to INTx routing tables for purposes
+of redirecting the interrupt handler to the rerouted interrupt line by default.
+Therefore, on chipsets where this INTx routing cannot be disabled, the
+Linux kernel will reroute the valid interrupt to its legacy interrupt. This
+redirection of the handler will prevent the occurrence of the spurious
+interrupt detection which would ordinarily disable the IRQ line due to
+excessive unhandled counts.[2]
+
+The config option X86_REROUTE_FOR_BROKEN_BOOT_IRQS exists to enable
+(or disable) the redirection of the interrupt handler to the PCH interrupt
+line. The option can be overridden by either pci=ioapicreroute or
+pci=noioapicreroute.[3]
+
+
+More Documentation
+==================
+
+There is an overview of the legacy interrupt handling mentioned in several
+datasheets (6300ESB and 6700PXH below). While largely the same, it provides
+insight into the evolution of its handling with chipsets.
+
+Example of disabling of the boot interrupt
+------------------------------------------
+
+Intel® 6300ESB I/O Controller Hub (Document # 300641-004US)
+	5.7.3 Boot Interrupt
+	https://www.intel.com/content/dam/doc/datasheet/6300esb-io-controller-hub-datasheet.pdf
+
+Intel® Xeon® Processor E5-1600/2400/2600/4600 v3 Product Families
+Datasheet - Volume 2: Registers (Dcument # 330784-003)
+	6.6.41 cipintrc Coherent Interface Protocol Interrupt Control
+	https://www.intel.com/content/dam/www/public/us/en/documents/datasheets/xeon-e5-v3-datasheet-vol-2.pdf
+
+Example of handler rerouting
+----------------------------
+
+Intel® 6700PXH 64-bit PCI Hub (Document # 302628)
+	2.15.2 PCI Express Legacy INTx Support and Boot Interrupt
+	https://www.intel.com/content/dam/doc/datasheet/6700pxh-64-bit-pci-hub-datasheet.pdf
+
+
+If you have any legacy PCI interrupt questions that aren't answered, email me.
+
+Cheers,
+    Sean V Kelley
+    sean.v.kelley@linux.intel.com
+
+[1] https://lore.kernel.org/lkml/12131949181903-git-send-email-sassmann@suse.de/
+[2] https://lore.kernel.org/lkml/12131949182094-git-send-email-sassmann@suse.de/
+[3] https://lore.kernel.org/lkml/487C8EA7.6020205@suse.de/
diff --git a/Documentation/PCI/index.rst b/Documentation/PCI/index.rst
index 6768305e4c26..8f66feaafd4f 100644
--- a/Documentation/PCI/index.rst
+++ b/Documentation/PCI/index.rst
@@ -16,3 +16,4 @@ Linux PCI Bus Subsystem
    pci-error-recovery
    pcieaer-howto
    endpoint/index
+   boot-interrupts
-- 
2.25.1

Re: [PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets

[Bjorn Helgaas]

On Thu, Feb 20, 2020 at 11:29:28AM -0800, Sean V Kelley wrote:
> Changes since v1 [1]:
> 
> - Correct Documentation section title for 6300ESB chipset.
> (Jonathan Derrick)
> 
> - Use consistent abbreviations in comments for IO-APIC and Core IO.
> (Andy Shevchenko)
> 
> - Retained Reviewed-by tag due to no technical changes.
> 
> [1]: https://lore.kernel.org/lkml/20200214213313.66622-1-sean.v.kelley@linux.intel.com/
> 
> Bjorn, I'm open for it to go to stable as well.
> 
> --
> 
> When IRQ lines on secondary or higher IO-APICs are masked (e.g.,
> Real-Time threaded interrupts), many chipsets redirect IRQs on
> this line to the legacy PCH and in turn the base IO-APIC in the
> system. The unhandled interrupts on the base IO-APIC will be
> identified by the Linux kernel as Spurious Interrupts and can
> lead to disabled IRQ lines.
> 
> Disabling this legacy PCI interrupt routing is chipset-specific and
> varies in mechanism between chipset vendors and across generations.
> In some cases the mechanism is exposed to BIOS but not all BIOS
> vendors chose to pick it up. With the increasing usage of RT as it
> marches towards mainline, additional issues have been raised with
> more recent Xeon chipsets.
> 
> This patchset disables the boot interrupt on these Xeon chipsets where
> this is possible with an additional mechanism. In addition, this
> patchset includes documentation covering the background of this quirk.
> 
> 
> Sean V Kelley (2):
>   pci: Add boot interrupt quirk mechanism for Xeon chipsets
>   Documentation:PCI: Add background on Boot Interrupts
> 
>  Documentation/PCI/boot-interrupts.rst | 153 ++++++++++++++++++++++++++
>  Documentation/PCI/index.rst           |   1 +
>  drivers/pci/quirks.c                  |  80 ++++++++++++--
>  3 files changed, 227 insertions(+), 7 deletions(-)
>  create mode 100644 Documentation/PCI/boot-interrupts.rst

Applied to pci/interrupts for v5.7.  I added a stable tag.

Thanks a lot; this is really a nice piece of work!

Re: boot interrupt quirk (also in 4.19.y) breaks serial ports (was: [PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets)

[Thomas Gleixner]

Stefan,

On Wed, Sep 16 2020 at 12:12, Stefan Bühler wrote:

sorry for the delay. This fell through the cracks.

> this quirk breaks our serial ports PCIe card (i.e. we don't see any 
> output from the connected devices; no idea whether anything we send 
> reaches them):
>
> 05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa)
> 06:00.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
> 06:00.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
> 06:01.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
> 06:01.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART)
> function 0 (Disabled)

Can you please provide the output of:

 for ID in 05:00.0 06:00.0 06:00.1 06:01.0 06:01.1; do lspci -s $ID -vvv; done

Thanks,

        tglx

Re: boot interrupt quirk (also in 4.19.y) breaks serial ports (was: [PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets)

[Stefan Bühler]

[-- Attachment #1: Type: text/plain, Size: 1128 bytes --]

Hi tglx,

On 11/25/20 12:54 PM, Thomas Gleixner wrote:
> Stefan,
> 
> On Wed, Sep 16 2020 at 12:12, Stefan Bühler wrote:
> 
> sorry for the delay. This fell through the cracks.
> 
>> this quirk breaks our serial ports PCIe card (i.e. we don't see any 
>> output from the connected devices; no idea whether anything we send 
>> reaches them):
>>
>> 05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa)
>> 06:00.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
>> 06:00.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
>> 06:01.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
>> 06:01.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART)
>> function 0 (Disabled)
> 
> Can you please provide the output of:
> 
>  for ID in 05:00.0 06:00.0 06:00.1 06:01.0 06:01.1; do lspci -s $ID -vvv; done
> 

See attachment.

Also I boot the affected systems now with "pci=noioapicquirk", which
"solves" the issue too (instead of patching the kernel).

cheers,
Stefan

[-- Attachment #2: oxford-serial-lspci.txt --]
[-- Type: text/plain, Size: 3721 bytes --]

05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
	Physical Slot: 1
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 32 bytes
	Interrupt: pin A routed to IRQ 16
	NUMA node: 0
	Bus: primary=05, secondary=06, subordinate=06, sec-latency=64
	I/O behind bridge: 0000e000-0000efff
	Memory behind bridge: fb400000-fb4fffff
	Prefetchable memory behind bridge: fff00000-000fffff
	Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: <access denied>

06:00.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart) (prog-if 06 [16950])
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 16
	NUMA node: 0
	Region 0: I/O ports at e0e0 [size=32]
	Region 1: Memory at fb407000 (32-bit, non-prefetchable) [size=4K]
	Region 2: I/O ports at e0c0 [size=32]
	Region 3: Memory at fb406000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: <access denied>
	Kernel driver in use: serial

06:00.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	NUMA node: 0
	Region 0: I/O ports at e0a0 [disabled] [size=32]
	Region 1: Memory at fb405000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Region 2: I/O ports at e080 [disabled] [size=32]
	Region 3: Memory at fb404000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Capabilities: <access denied>

06:01.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart) (prog-if 06 [16950])
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 17
	NUMA node: 0
	Region 0: I/O ports at e060 [size=32]
	Region 1: Memory at fb403000 (32-bit, non-prefetchable) [size=4K]
	Region 2: I/O ports at e040 [size=32]
	Region 3: Memory at fb402000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: <access denied>
	Kernel driver in use: serial

06:01.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	NUMA node: 0
	Region 0: I/O ports at e020 [disabled] [size=32]
	Region 1: Memory at fb401000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Region 2: I/O ports at e000 [disabled] [size=32]
	Region 3: Memory at fb400000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Capabilities: <access denied>

Re: boot interrupt quirk (also in 4.19.y) breaks serial ports (was: [PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets)

[Thomas Gleixner]

Stefan,

On Wed, Nov 25 2020 at 14:41, Stefan Bühler wrote:
> On 11/25/20 12:54 PM, Thomas Gleixner wrote:
>> On Wed, Sep 16 2020 at 12:12, Stefan Bühler wrote:
>> Can you please provide the output of:
>> 
>>  for ID in 05:00.0 06:00.0 06:00.1 06:01.0 06:01.1; do lspci -s $ID -vvv; done
>
> See attachment.
>
> Also I boot the affected systems now with "pci=noioapicquirk", which
> "solves" the issue too (instead of patching the kernel).

Yes, it skips the quirks.

> 05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
> 	Physical Slot: 1
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> 	Latency: 0, Cache Line Size: 32 bytes
> 	Interrupt: pin A routed to IRQ 16
> 	NUMA node: 0
> 	Bus: primary=05, secondary=06, subordinate=06, sec-latency=64
> 	I/O behind bridge: 0000e000-0000efff
> 	Memory behind bridge: fb400000-fb4fffff
> 	Prefetchable memory behind bridge: fff00000-000fffff
> 	Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
> 	BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
> 		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> 	Capabilities: <access denied>

Can you please run this as root so the Capabilities are accessible?

Thanks,

        tglx

Re: boot interrupt quirk (also in 4.19.y) breaks serial ports (was: [PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets)

[Stefan Bühler]

[-- Attachment #1: Type: text/plain, Size: 3146 bytes --]

Hi tglx,

On 11/27/20 12:45 AM, Thomas Gleixner wrote:
> Stefan,
> 
> On Wed, Nov 25 2020 at 14:41, Stefan Bühler wrote:
>> On 11/25/20 12:54 PM, Thomas Gleixner wrote:
>>> On Wed, Sep 16 2020 at 12:12, Stefan Bühler wrote:
>>> Can you please provide the output of:
>>>
>>>  for ID in 05:00.0 06:00.0 06:00.1 06:01.0 06:01.1; do lspci -s $ID -vvv; done
>>
>> 05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
>>      ...
>> 	Capabilities: <access denied>
> 
> Can you please run this as root so the Capabilities are accessible?

My bad, sorry. I did intend to run it as root, but should have checked
the output.  Again see attached file.


While we're at it: the EEPROM for the PEX8112 is:

00000000  5a 03 3c 00 10 00 00 00  00 00 00 00 b5 10 12 81  |Z.<.............|
00000010  64 00 20 00 00 00 00 01  04 00 01 00 0c 10 00 fe  |d. .............|
00000020  fe 03 20 10 f0 10 00 00  00 10 33 00 00 00 70 00  |.. .......3...p.|
00000030  00 00 11 00 48 00 00 00  00 00 34 00 50 00 00 00  |....H.....4.P...|
00000040  04 00 55 66 77 88                                 |..Ufw.|
00000046

(This is what the firmware tool provided to me writes, although I think 
the cards usually came pre-flashed with it.  They gave me the tool 
because on some cards the second function on OX16PCI954 was sometimes 
uninitialized, came up with device id 0x9511 "8-bit bus" 
(PCI_DEVICE_ID_OXSEMI_16PCI95N) and the kernel tries to treat it as UART 
too.)

I think some time ago I found a PDF to decode this here:
https://www.broadcom.com/products/pcie-switches-bridges/pcie-bridges/pex8112#documentation

But the broadcom site is completely broken right now (at least for me; 
there own search for "PEX 8112" links it, but then it says "not found").

Anyway, back then I decoded this to:

- `0x5A 0x03`: Magic Header, contains register and shared memory settings
- `0x003C` = 60 bytes for configs (10 registers):
  - `@0x0010`: `0x00000000` -- BAR0: Locate anywhere in 32-bit
  - `@0x0000`: `0x811210B5` -- Vendor `10B5`, Device `8112` (default)
  - `@0x0064`: `0x00000020` -- Device Capability: Enable "Support 8-bit Tag" field
  - `@0x0100`: `0x00010004` -- Power Budget Enhanced Capability Header (default)
  - `@0x100C`: `0x03FEFE00` -- PCI Control:
    - PCI-To-PCI Express Retry Count set to 0xFE (default: `0x80`)
    - PCI Express-to-PCI Retry Count set to 0xFE (default: `0x00`)
  - `@0x1020`: `0x000010F0` -- GPIO Control
    - GPIO[1-3] Output enable (GPIO[0] is Output enabled by default)
    - GPIO Diagnostic Select: `10b` (default: `01b`)
  - `@0x1000`: `0x00000033` -- Device Initialization (default)
  - `@0x0070`: `0x00110000` -- Link control: default
  - `@0x0048`: `0x00000000` -- Device-Specific Control (default 0)
  - `@0x0034`: `0x00000050` -- PCI Capability pointer `0x50` (default: `0x40`)
    - Skips (disables) Power Management Capability
    - Remaining: MSI and PCI Express
- `0x0004` bytes for shared memory:
  - `0x55`, `0x66`, `0x77`, `0x88`


TLDR: the most notable part probably being "disabling Power Management 
Capability" by the EEPROM.

cheers,
Stefan

[-- Attachment #2: oxford-serial-lspci.txt --]
[-- Type: text/plain, Size: 5194 bytes --]

05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
	Physical Slot: 1
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 32 bytes
	Interrupt: pin A routed to IRQ 16
	NUMA node: 0
	Bus: primary=05, secondary=06, subordinate=06, sec-latency=64
	I/O behind bridge: 0000e000-0000efff
	Memory behind bridge: fb400000-fb4fffff
	Prefetchable memory behind bridge: fff00000-000fffff
	Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [60] Express (v1) PCI-Express to PCI/PCI-X Bridge, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- SlotPowerLimit 26.000W
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
	Capabilities: [100 v1] Power Budgeting <?>

06:00.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart) (prog-if 06 [16950])
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 16
	NUMA node: 0
	Region 0: I/O ports at e0e0 [size=32]
	Region 1: Memory at fb407000 (32-bit, non-prefetchable) [size=4K]
	Region 2: I/O ports at e0c0 [size=32]
	Region 3: Memory at fb406000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [40] Power Management version 1
		Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0+,D1-,D2+,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: serial

06:00.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	NUMA node: 0
	Region 0: I/O ports at e0a0 [disabled] [size=32]
	Region 1: Memory at fb405000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Region 2: I/O ports at e080 [disabled] [size=32]
	Region 3: Memory at fb404000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Capabilities: [40] Power Management version 1
		Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0+,D1-,D2+,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

06:01.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart) (prog-if 06 [16950])
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 17
	NUMA node: 0
	Region 0: I/O ports at e060 [size=32]
	Region 1: Memory at fb403000 (32-bit, non-prefetchable) [size=4K]
	Region 2: I/O ports at e040 [size=32]
	Region 3: Memory at fb402000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [40] Power Management version 1
		Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0+,D1-,D2+,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: serial

06:01.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	NUMA node: 0
	Region 0: I/O ports at e020 [disabled] [size=32]
	Region 1: Memory at fb401000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Region 2: I/O ports at e000 [disabled] [size=32]
	Region 3: Memory at fb400000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Capabilities: [40] Power Management version 1
		Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0+,D1-,D2+,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

Re: boot interrupt quirk (also in 4.19.y) breaks serial ports (was: [PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets)

[Thomas Gleixner]

Stefan,

On Fri, Nov 27 2020 at 10:17, Stefan Bühler wrote:
> On 11/27/20 12:45 AM, Thomas Gleixner wrote:
>> Can you please run this as root so the Capabilities are accessible?
>
> My bad, sorry. I did intend to run it as root, but should have checked
> the output.  Again see attached file.

No problem.

> 05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
> 	Physical Slot: 1
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> 	Latency: 0, Cache Line Size: 32 bytes
> 	Interrupt: pin A routed to IRQ 16
> 	NUMA node: 0
> 	Bus: primary=05, secondary=06, subordinate=06, sec-latency=64
> 	I/O behind bridge: 0000e000-0000efff
> 	Memory behind bridge: fb400000-fb4fffff
> 	Prefetchable memory behind bridge: fff00000-000fffff
> 	Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
> 	BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
> 		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> 	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
> 		Address: 0000000000000000  Data: 0000

So the bridge would support MSI, but obviously the devices on the PCI
side do not.

> 06:00.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart) (prog-if 06 [16950])
> 	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
> 	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> 	Interrupt: pin A routed to IRQ 16
> 	NUMA node: 0
> 	Region 0: I/O ports at e0e0 [size=32]
> 	Region 1: Memory at fb407000 (32-bit, non-prefetchable) [size=4K]
> 	Region 2: I/O ports at e0c0 [size=32]
> 	Region 3: Memory at fb406000 (32-bit, non-prefetchable) [size=4K]
> 	Capabilities: [40] Power Management version 1
> 		Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0+,D1-,D2+,D3hot+,D3cold-)
> 		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

But that should still work because the boot interrupt quirk should not
affect interrupts which are routed through the IOAPIC.

Sean, any idea what's going on here?

Thanks,

        tglx

Re: boot interrupt quirk (also in 4.19.y) breaks serial ports (was: [PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets)

[Grzegorz Halat]

Hi,

On Wed, Sep 16 2020 at 12:12, Stefan Bühler wrote:
> this quirk breaks our serial ports PCIe card (i.e. we don't see any
> output from the connected devices; no idea whether anything we send
> reaches them):

I have the same problem, also with a PCI serial adapter from Oxford Semiconductor.
I've bisected the kernel and it was introduced in b88bf6c3b6ff.
When the system is booted with "pci=noioapicquirk" then the PCI card works fine.
The CPU is Intel Xeon E5-2680 v3 @ 2.50GHz.

Sean, do you have any news about this issue?

Grzegorz

Re: boot interrupt quirk (also in 4.19.y) breaks serial ports (was: [PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets)

[Bjorn Helgaas]

On Fri, Sep 23, 2022 at 09:20:30PM +0200, Grzegorz Halat wrote:
> On Wed, Sep 16 2020 at 12:12, Stefan Bühler wrote:
> > this quirk breaks our serial ports PCIe card (i.e. we don't see any
> > output from the connected devices; no idea whether anything we send
> > reaches them):
> 
> I have the same problem, also with a PCI serial adapter from Oxford
> Semiconductor.  I've bisected the kernel and it was introduced in
> b88bf6c3b6ff.  When the system is booted with "pci=noioapicquirk"
> then the PCI card works fine.  The CPU is Intel Xeon E5-2680 v3 @
> 2.50GHz.
> 
> Sean, do you have any news about this issue?

I think Sean has moved on and isn't available to work this issue.

To help whoever *can* work on it, would you mind attaching the
complete dmesg log and "sudo lspci -vv" output for your system?  Then
we can potentially tweak the quirk to not break your system.

Bjorn

Re: boot interrupt quirk (also in 4.19.y) breaks serial ports (was: [PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets) #forregzbot

[Thorsten Leemhuis]

TWIMC: this mail is primarily send for documentation purposes and for
regzbot, my Linux kernel regression tracking bot. These mails usually
contain '#forregzbot' in the subject, to make them easy to spot and filter.

[TLDR: I'm adding this regression report to the list of tracked
regressions; all text from me you find below is based on a few templates
paragraphs you might have encountered already already in similar form.]

Hi, this is your Linux kernel regression tracker.

On 23.09.22 21:20, Grzegorz Halat wrote:
> Hi,
> 
> On Wed, Sep 16 2020 at 12:12, Stefan Bühler wrote:
>> this quirk breaks our serial ports PCIe card (i.e. we don't see any
>> output from the connected devices; no idea whether anything we send
>> reaches them):
> 
> I have the same problem, also with a PCI serial adapter from Oxford Semiconductor.
> I've bisected the kernel and it was introduced in b88bf6c3b6ff.
> When the system is booted with "pci=noioapicquirk" then the PCI card works fine.
> The CPU is Intel Xeon E5-2680 v3 @ 2.50GHz.
> 
> Sean, do you have any news about this issue?

Thanks for the report. To be sure below issue doesn't fall through the
cracks unnoticed, I'm adding it to regzbot, my Linux kernel regression
tracking bot:

#regzbot ^introduced b88bf6c3b6ff77948
#regzbot title boot interrupt quirk (also in 4.19.y) breaks serial ports
#regzbot ignore-activity
#regzbot backburner: known since 2020, original developer likely moved on

This isn't a regression? This issue or a fix for it are already
discussed somewhere else? It was fixed already? You want to clarify when
the regression started to happen? Or point out I got the title or
something else totally wrong? Then just reply -- ideally with also
telling regzbot about it, as explained here:
https://linux-regtracking.leemhuis.info/tracked-regression/

Reminder for developers: When fixing the issue, add 'Link:' tags
pointing to the report (the mail this one replies to), as explained for
in the Linux kernel's documentation; above webpage explains why this is
important for tracked regressions.

Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)

P.S.: As the Linux kernel's regression tracker I deal with a lot of
reports and sometimes miss something important when writing mails like
this. If that's the case here, don't hesitate to tell me in a public
reply, it's in everyone's interest to set the public record straight.