* [PATCH 1/1] PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU
@ 2021-09-01 12:40 Adrian Huang
2021-09-01 15:17 ` Jon Derrick
2021-10-12 13:54 ` Lorenzo Pieralisi
0 siblings, 2 replies; 4+ messages in thread
From: Adrian Huang @ 2021-09-01 12:40 UTC (permalink / raw)
To: linux-pci
Cc: Lorenzo Pieralisi, Rob Herring, kw, Bjorn Helgaas, Adrian Huang,
Jon Derrick, Nirmal Patel, Joerg Roedel, Adrian Huang
From: Adrian Huang <ahuang12@lenovo.com>
When enabling VMD in BIOS setup (Ice Lake Processor: Whitley platform),
the host OS cannot boot successfully with the following error message:
nvme nvme0: I/O 12 QID 0 timeout, completion polled
nvme nvme0: Shutdown timeout set to 6 seconds
DMAR: DRHD: handling fault status reg 2
DMAR: [INTR-REMAP] Request device [0x00:0x00.5] fault index 0xa00 [fault reason 0x25] Blocked a compatibility format interrupt request
The request device is the VMD controller:
# lspci -s 0000:00.5 -nn
0000:00:00.5 RAID bus controller [0104]: Intel Corporation Volume
Management Device NVMe RAID Controller [8086:28c0] (rev 04)
`git bisect` points to this offending commit ee81ee84f873 ("PCI:
vmd: Disable MSI-X remapping when possible"), which disables VMD MSI
remapping. The IOMMU hardware blocks the compatibility format
interrupt request because Interrupt Remapping Enable Status (IRES) and
Extended Interrupt Mode Enable (EIME) are enabled. Please refer to
section "5.1.4 Interrupt-Remapping Hardware Operation" in Intel VT-d
spec.
To fix the issue, VMD driver still enables the interrupt remapping
irrespective of VMD_FEAT_CAN_BYPASS_MSI_REMAP if the IOMMU subsystem
enables the interrupt remapping.
Test configuration is shown as follows:
* Two VMD controllers
1. 8086:28c0 (Whitley's VMD)
2. 8086:201d (Purley's VMD: The issue does not appear in this
controller. Just make sure if any side effect occurs.)
* w/wo intremap=off
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=214219
Cc: Jon Derrick <jonathan.derrick@intel.com>
Cc: Nirmal Patel <nirmal.patel@linux.intel.com>
Cc: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Adrian Huang <ahuang12@lenovo.com>
---
drivers/pci/controller/vmd.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index e3fcdfec58b3..db72932d049f 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -6,6 +6,7 @@
#include <linux/device.h>
#include <linux/interrupt.h>
+#include <linux/iommu.h>
#include <linux/irq.h>
#include <linux/kernel.h>
#include <linux/module.h>
@@ -710,7 +711,8 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
* acceptable because the guest is usually CPU-limited and MSI
* remapping doesn't become a performance bottleneck.
*/
- if (!(features & VMD_FEAT_CAN_BYPASS_MSI_REMAP) ||
+ if (iommu_capable(vmd->dev->dev.bus, IOMMU_CAP_INTR_REMAP) ||
+ !(features & VMD_FEAT_CAN_BYPASS_MSI_REMAP) ||
offset[0] || offset[1]) {
ret = vmd_alloc_irqs(vmd);
if (ret)
--
2.27.0
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU
2021-09-01 12:40 [PATCH 1/1] PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU Adrian Huang
@ 2021-09-01 15:17 ` Jon Derrick
2021-10-01 7:51 ` Huang Adrian
2021-10-12 13:54 ` Lorenzo Pieralisi
1 sibling, 1 reply; 4+ messages in thread
From: Jon Derrick @ 2021-09-01 15:17 UTC (permalink / raw)
To: Adrian Huang, linux-pci
Cc: Lorenzo Pieralisi, Rob Herring, kw, Bjorn Helgaas, Nirmal Patel,
Joerg Roedel, Adrian Huang
Thank you Adrian
On 9/1/21 6:40 AM, Adrian Huang wrote:
> From: Adrian Huang <ahuang12@lenovo.com>
>
> When enabling VMD in BIOS setup (Ice Lake Processor: Whitley platform),
> the host OS cannot boot successfully with the following error message:
>
> nvme nvme0: I/O 12 QID 0 timeout, completion polled
> nvme nvme0: Shutdown timeout set to 6 seconds
> DMAR: DRHD: handling fault status reg 2
> DMAR: [INTR-REMAP] Request device [0x00:0x00.5] fault index 0xa00 [fault reason 0x25] Blocked a compatibility format interrupt request
I know we'd really prefer to support interrupt remapping with the VMD feature,
and I'm not certain how EIME differs from the interrupt remapping modes that
were tested while developing the VMD feature.
I think this will have to be acceptable for now.
Reviewed-by: Jon Derrick <jonathan.derrick@intel.com>
>
> The request device is the VMD controller:
> # lspci -s 0000:00.5 -nn
> 0000:00:00.5 RAID bus controller [0104]: Intel Corporation Volume
> Management Device NVMe RAID Controller [8086:28c0] (rev 04)
>
> `git bisect` points to this offending commit ee81ee84f873 ("PCI:
> vmd: Disable MSI-X remapping when possible"), which disables VMD MSI
> remapping. The IOMMU hardware blocks the compatibility format
> interrupt request because Interrupt Remapping Enable Status (IRES) and
> Extended Interrupt Mode Enable (EIME) are enabled. Please refer to
> section "5.1.4 Interrupt-Remapping Hardware Operation" in Intel VT-d
> spec.
>
> To fix the issue, VMD driver still enables the interrupt remapping
> irrespective of VMD_FEAT_CAN_BYPASS_MSI_REMAP if the IOMMU subsystem
> enables the interrupt remapping.
>
> Test configuration is shown as follows:
> * Two VMD controllers
> 1. 8086:28c0 (Whitley's VMD)
> 2. 8086:201d (Purley's VMD: The issue does not appear in this
> controller. Just make sure if any side effect occurs.)
> * w/wo intremap=off
>
> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=214219
> Cc: Jon Derrick <jonathan.derrick@intel.com>
> Cc: Nirmal Patel <nirmal.patel@linux.intel.com>
> Cc: Joerg Roedel <jroedel@suse.de>
> Signed-off-by: Adrian Huang <ahuang12@lenovo.com>
> ---
> drivers/pci/controller/vmd.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
> index e3fcdfec58b3..db72932d049f 100644
> --- a/drivers/pci/controller/vmd.c
> +++ b/drivers/pci/controller/vmd.c
> @@ -6,6 +6,7 @@
>
> #include <linux/device.h>
> #include <linux/interrupt.h>
> +#include <linux/iommu.h>
> #include <linux/irq.h>
> #include <linux/kernel.h>
> #include <linux/module.h>
> @@ -710,7 +711,8 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
> * acceptable because the guest is usually CPU-limited and MSI
> * remapping doesn't become a performance bottleneck.
> */
> - if (!(features & VMD_FEAT_CAN_BYPASS_MSI_REMAP) ||
> + if (iommu_capable(vmd->dev->dev.bus, IOMMU_CAP_INTR_REMAP) ||
> + !(features & VMD_FEAT_CAN_BYPASS_MSI_REMAP) ||
> offset[0] || offset[1]) {
> ret = vmd_alloc_irqs(vmd);
> if (ret)
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU
2021-09-01 15:17 ` Jon Derrick
@ 2021-10-01 7:51 ` Huang Adrian
0 siblings, 0 replies; 4+ messages in thread
From: Huang Adrian @ 2021-10-01 7:51 UTC (permalink / raw)
To: Jon Derrick
Cc: linux-pci, Lorenzo Pieralisi, Rob Herring, kw, Bjorn Helgaas,
Nirmal Patel, Joerg Roedel, Adrian Huang
Hi,
On Wed, Sep 1, 2021 at 11:18 PM Jon Derrick <jonathan.derrick@intel.com> wrote:
>
> Thank you Adrian
>
> On 9/1/21 6:40 AM, Adrian Huang wrote:
> > From: Adrian Huang <ahuang12@lenovo.com>
> >
> > When enabling VMD in BIOS setup (Ice Lake Processor: Whitley platform),
> > the host OS cannot boot successfully with the following error message:
> >
> > nvme nvme0: I/O 12 QID 0 timeout, completion polled
> > nvme nvme0: Shutdown timeout set to 6 seconds
> > DMAR: DRHD: handling fault status reg 2
> > DMAR: [INTR-REMAP] Request device [0x00:0x00.5] fault index 0xa00 [fault reason 0x25] Blocked a compatibility format interrupt request
>
> I know we'd really prefer to support interrupt remapping with the VMD feature,
> and I'm not certain how EIME differs from the interrupt remapping modes that
> were tested while developing the VMD feature.
>
> I think this will have to be acceptable for now.
>
> Reviewed-by: Jon Derrick <jonathan.derrick@intel.com>
>
Gentle ping. Any comments about this patch (with Jon's Reviewed-by)?
-- Adrian
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU
2021-09-01 12:40 [PATCH 1/1] PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU Adrian Huang
2021-09-01 15:17 ` Jon Derrick
@ 2021-10-12 13:54 ` Lorenzo Pieralisi
1 sibling, 0 replies; 4+ messages in thread
From: Lorenzo Pieralisi @ 2021-10-12 13:54 UTC (permalink / raw)
To: Adrian Huang, linux-pci
Cc: Lorenzo Pieralisi, Rob Herring, Adrian Huang, Bjorn Helgaas,
Joerg Roedel, Nirmal Patel, kw, Jon Derrick
On Wed, 1 Sep 2021 20:40:47 +0800, Adrian Huang wrote:
> From: Adrian Huang <ahuang12@lenovo.com>
>
> When enabling VMD in BIOS setup (Ice Lake Processor: Whitley platform),
> the host OS cannot boot successfully with the following error message:
>
> nvme nvme0: I/O 12 QID 0 timeout, completion polled
> nvme nvme0: Shutdown timeout set to 6 seconds
> DMAR: DRHD: handling fault status reg 2
> DMAR: [INTR-REMAP] Request device [0x00:0x00.5] fault index 0xa00 [fault reason 0x25] Blocked a compatibility format interrupt request
>
> [...]
Applied to pci/vmd, thanks!
[1/1] PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU
https://git.kernel.org/lpieralisi/pci/c/2565e5b69c
Thanks,
Lorenzo
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-10-12 13:54 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-01 12:40 [PATCH 1/1] PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU Adrian Huang
2021-09-01 15:17 ` Jon Derrick
2021-10-01 7:51 ` Huang Adrian
2021-10-12 13:54 ` Lorenzo Pieralisi
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.