From: Adrian Huang <adrianhuang0701@gmail.com>
To: Joerg Roedel <joro@8bytes.org>
Cc: iommu@lists.linux-foundation.org, Adrian Huang <ahuang12@lenovo.com>
Subject: [PATCH 1/1] iommu/amd: Fix the overwritten exclusion range with multiple IVMDs
Date: Mon, 4 Nov 2019 13:58:52 +0800 [thread overview]
Message-ID: <20191104055852.24395-1-ahuang12@lenovo.com> (raw)
When attaching two Broadcom RAID controllers to a server, the first one
reports the failure during booting (the disks connecting to the RAID
controller cannot be detected):
megaraid_sas 0000:42:00.0: Init cmd return status FAILED for SCSI host 0
megaraid_sas 0000:42:00.0: Failed from megasas_init_fw 6376
Root-cause of the issue:
1) Those two RAID controllers define their own IVMDs with the
valid exclusion range, and they are associated with the same
IOMMU hardware:
Subtable Type : 10 [Hardware Definition Block]
Flags : B0
Length : 0028
DeviceId : 4002
Capability Offset : 0040
Base Address : 00000000B4100000
PCI Segment Group : 0000
Virtualization Info : 0000
Reserved : 80048F6F
Entry Type : 03
Device ID : 4008
Data Setting : 00
Entry Type : 04
Device ID : 7FFE
Data Setting : 00
Subtable Type : 21 [Memory Definition Block]
Flags : 08
Length : 0020
DeviceId : 4200
Auxiliary Data : 0000
Reserved : 0000000000000000
Start Address : 000000009F58D000
Memory Length : 0000000008040000
Subtable Type : 21 [Memory Definition Block]
Flags : 08
Length : 0020
DeviceId : 4300
Auxiliary Data : 0000
Reserved : 0000000000000000
Start Address : 000000009754D000
Memory Length : 0000000008040000
2) When set_device_exclusion_range() parses the IVMD of devce id
'4200', the exclusion range of the amd_iommu struct becomes:
iommu->exclusion_start = 0x9F58D000;
iommu->exclusion_length = 0x8040000;
3) When parsing the second IVMD (device id '4300') in
set_device_exclusion_range(), the exclusion range of the
amd_iommu struct becomes:
iommu->exclusion_start = 0x9754D000;
iommu->exclusion_length = 0x8040000;
This overwrites the first IVMD configuration, which leads to
the failure of the first RAID controller.
This patch fixes the issue by using unity map for multiple IVMDs if
those IVMDs define the valid exclusion range (different exclusion range)
and they are associated with the same IOMMU hardware. Note that the first
IVMD still uses the exclusion range.
Signed-off-by: Adrian Huang <ahuang12@lenovo.com>
---
drivers/iommu/amd_iommu_init.c | 33 +++++++++++++++++++++++++++++++--
1 file changed, 31 insertions(+), 2 deletions(-)
diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c
index 568c52317757..d65b548a42f5 100644
--- a/drivers/iommu/amd_iommu_init.c
+++ b/drivers/iommu/amd_iommu_init.c
@@ -71,6 +71,8 @@
#define IVHD_FLAG_ISOC_EN_MASK 0x08
#define IVMD_FLAG_EXCL_RANGE 0x08
+#define IVMD_FLAG_IW 0x04
+#define IVMD_FLAG_IR 0x02
#define IVMD_FLAG_UNITY_MAP 0x01
#define ACPI_DEVFLAG_INITPASS 0x01
@@ -1110,6 +1112,32 @@ static int __init add_early_maps(void)
return 0;
}
+static int __init exclusion_range_has_configured(struct amd_iommu *iommu,
+ struct ivmd_header *m)
+{
+ /* Not configure yet. */
+ if (!iommu->exclusion_start) {
+ iommu->exclusion_start = m->range_start;
+ iommu->exclusion_length = m->range_length;
+
+ return 0;
+ }
+
+ if (iommu->exclusion_start == m->range_start &&
+ iommu->exclusion_length == m->range_length)
+ return 0;
+
+ /*
+ * The exclusion range of the iommu has been configured
+ * by the other IVMD, so we need to use unity map for this
+ * IVMD to avoid the overwritten exclusion range members of the
+ * amd_iommu structure.
+ */
+ m->flags = (IVMD_FLAG_IW | IVMD_FLAG_IR | IVMD_FLAG_UNITY_MAP);
+
+ return 1;
+}
+
/*
* Reads the device exclusion range from ACPI and initializes the IOMMU with
* it
@@ -1122,14 +1150,15 @@ static void __init set_device_exclusion_range(u16 devid, struct ivmd_header *m)
return;
if (iommu) {
+ if (exclusion_range_has_configured(iommu, m))
+ return;
+
/*
* We only can configure exclusion ranges per IOMMU, not
* per device. But we can enable the exclusion range per
* device. This is done here
*/
set_dev_entry_bit(devid, DEV_ENTRY_EX);
- iommu->exclusion_start = m->range_start;
- iommu->exclusion_length = m->range_length;
}
}
--
2.17.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
next reply other threads:[~2019-11-04 16:28 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-04 5:58 Adrian Huang [this message]
2019-11-11 15:22 ` [PATCH 1/1] iommu/amd: Fix the overwritten exclusion range with multiple IVMDs Joerg Roedel
2019-11-12 9:32 ` Huang Adrian
2019-11-12 16:05 ` Joerg Roedel
2019-11-13 1:05 ` Huang Adrian
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191104055852.24395-1-ahuang12@lenovo.com \
--to=adrianhuang0701@gmail.com \
--cc=ahuang12@lenovo.com \
--cc=iommu@lists.linux-foundation.org \
--cc=joro@8bytes.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).