archive mirror
 help / color / mirror / Atom feed
From: Sean V Kelley <>
	Sean V Kelley <>
Subject: [PATCH v2 2/2] Documentation:PCI: Add background on Boot Interrupts
Date: Thu, 20 Feb 2020 11:29:30 -0800	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

Improve understanding of the PCI quirks for this legacy PCI interrupt
behavior to the benefit of developers and users alike.

Reviewed-by: Thomas Gleixner <>
Signed-off-by: Sean V Kelley <>
 Documentation/PCI/boot-interrupts.rst | 153 ++++++++++++++++++++++++++
 Documentation/PCI/index.rst           |   1 +
 2 files changed, 154 insertions(+)
 create mode 100644 Documentation/PCI/boot-interrupts.rst

diff --git a/Documentation/PCI/boot-interrupts.rst b/Documentation/PCI/boot-interrupts.rst
new file mode 100644
index 000000000000..b4d42481fd7f
--- /dev/null
+++ b/Documentation/PCI/boot-interrupts.rst
@@ -0,0 +1,153 @@
+.. SPDX-License-Identifier: GPL-2.0
+Boot Interrupts
+:Author: - Sean V Kelley <>
+On PCI Express, interrupts are represented with either MSI or inbound interrupt
+messages (Assert_INTx/Deassert_INTx). The integrated IO-APIC in a given Core
+IO converts the legacy interrupt messages from PCI Express to MSI interrupts.
+If the IO-APIC is disabled (via the mask bits in the IO-APIC table entries),
+the messages are routed to the legacy PCH. This in-band interrupt mechanism was
+traditionally necessary for systems that did not support the IO-APIC and for
+boot. Intel in the past has used the term "boot interrupts" to describe this
+mechanism. Further, the PCI Express protocol describes this in-band legacy
+wire-interrupt INTx mechanism for I/O devices to signal PCI-style level
+interrupts. The subsequent paragraphs describe problems with the Core IO
+handling of INTx message routing to the PCH and mitigation within BIOS and
+the OS.
+When in-band legacy INTx messages are forwarded to the PCH, they in turn
+trigger a new interrupt for which the OS likely lacks a handler. When an
+interrupt goes unhandled over time, they are tracked by the Linux kernel
+as Spurious Interrupts. The IRQ will be disabled by the Linux kernel after
+it reaches a specific count with the error "nobody cared". This disabled
+IRQ now prevents valid usage by an existing interrupt which may happen to
+share the IRQ line.
+irq 19: nobody cared (try booting with the "irqpoll" option)
+CPU: 0 PID: 2988 Comm: irq/34-nipalk Tainted: 4.14.87-rt49-02410-g4a640ec-dirty #1
+Hardware name: National Instruments NI PXIe-8880/NI PXIe-8880, BIOS 2.1.5f1 01/09/2020
+Call Trace:
+ ? dump_stack+0x46/0x5e
+ ? __report_bad_irq+0x2e/0xb0
+ ? note_interrupt+0x242/0x290
+ ? nNIKAL100_memoryRead16+0x8/0x10 [nikal]
+ ? handle_irq_event_percpu+0x55/0x70
+ ? handle_irq_event+0x4f/0x80
+ ? handle_fasteoi_irq+0x81/0x180
+ ? handle_irq+0x1c/0x30
+ ? do_IRQ+0x41/0xd0
+ ? common_interrupt+0x84/0x84
+irq_default_primary_handler threaded usb_hcd_irq
+Disabling IRQ #19
+The use of threaded interrupts is the most likely condition to trigger this
+problem today. Threaded interrupts may not be reenabled after the IRQ handler
+wakes. These "one shot" conditions mean that the threaded interrupt needs to
+keep the interrupt line masked until the threaded handler has run. Especially
+when dealing with high data rate interrupts, the thread needs to run to completion
+otherwise some handlers will end up in stack overflows since the interrupt
+of the issuing device is still active.
+Affected Chipsets
+The legacy interrupt forwarding mechansim exists today in a number of devices
+including but not limited to chipsets from AMD/ATI, Broadcom, and Intel. Changes
+made through the mitigations below have been applied to drivers/pci/quirks.c
+Starting with ICX there are no longer any IO-APICs in the Core IO's devices.
+IO-APIC is only in the PCH.  Devices connected to the Core IO's PCIE Root Ports
+will use native MSI/MSI-X mechanisms.
+The mitigations take the form of PCI quirks. The preference has been to first
+identify and make use of a means to disable the routing to the PCH. In such a
+case a quirk to disable boot interrupt generation can be added.[1]
+Intel® 6300ESB I/O Controller Hub
+Alternate Base Address Register:
+ BIE: Boot Interrupt Enable
+	0 = Boot interrupt is enabled.
+	1 = Boot interrupt is disabled.
+Intel® Sandy Bridge through Sky Lake based Xeon servers:
+Coherent Interface Protocol Interrupt Control
+ dis_intx_route2pch/dis_intx_route2ich/dis_intx_route2dmi2:
+	When this bit is set. Local INTx messages received from the
+	Intel® Quick Data DMA/PCI Express ports are not routed to legacy
+	PCH - they are either converted into MSI via the integrated IO-APIC
+	(if the IO-APIC mask bit is clear in the appropriate entries)
+	or cause no further action (when mask bit is set)
+In the absence of a way to directly disable the routing, another approach
+has been to make use of PCI Interrupt pin to INTx routing tables for purposes
+of redirecting the interrupt handler to the rerouted interrupt line by default.
+Therefore, on chipsets where this INTx routing cannot be disabled, the
+Linux kernel will reroute the valid interrupt to its legacy interrupt. This
+redirection of the handler will prevent the occurrence of the spurious
+interrupt detection which would ordinarily disable the IRQ line due to
+excessive unhandled counts.[2]
+The config option X86_REROUTE_FOR_BROKEN_BOOT_IRQS exists to enable
+(or disable) the redirection of the interrupt handler to the PCH interrupt
+line. The option can be overridden by either pci=ioapicreroute or
+More Documentation
+There is an overview of the legacy interrupt handling mentioned in several
+datasheets (6300ESB and 6700PXH below). While largely the same, it provides
+insight into the evolution of its handling with chipsets.
+Example of disabling of the boot interrupt
+Intel® 6300ESB I/O Controller Hub (Document # 300641-004US)
+	5.7.3 Boot Interrupt
+Intel® Xeon® Processor E5-1600/2400/2600/4600 v3 Product Families
+Datasheet - Volume 2: Registers (Dcument # 330784-003)
+	6.6.41 cipintrc Coherent Interface Protocol Interrupt Control
+Example of handler rerouting
+Intel® 6700PXH 64-bit PCI Hub (Document # 302628)
+	2.15.2 PCI Express Legacy INTx Support and Boot Interrupt
+If you have any legacy PCI interrupt questions that aren't answered, email me.
+    Sean V Kelley
diff --git a/Documentation/PCI/index.rst b/Documentation/PCI/index.rst
index 6768305e4c26..8f66feaafd4f 100644
--- a/Documentation/PCI/index.rst
+++ b/Documentation/PCI/index.rst
@@ -16,3 +16,4 @@ Linux PCI Bus Subsystem
+   boot-interrupts

  parent reply	other threads:[~2020-02-20 19:29 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-20 19:29 [PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets Sean V Kelley
2020-02-20 19:29 ` [PATCH v2 1/2] " Sean V Kelley
2020-02-20 19:29 ` Sean V Kelley [this message]
2020-02-27 22:49 ` [PATCH v2 0/2] " Bjorn Helgaas
     [not found] ` <>
2020-11-25 11:54   ` boot interrupt quirk (also in 4.19.y) breaks serial ports (was: [PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets) Thomas Gleixner
2020-11-25 13:41     ` Stefan Bühler
2020-11-26 23:45       ` Thomas Gleixner
2020-11-27  9:17         ` Stefan Bühler
2020-11-30 10:48           ` Thomas Gleixner
2022-09-23 19:20   ` Grzegorz Halat
2022-09-26 21:17     ` Bjorn Helgaas
2022-09-28  8:34     ` boot interrupt quirk (also in 4.19.y) breaks serial ports (was: [PATCH v2 0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets) #forregzbot Thorsten Leemhuis

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).