All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-08-05  8:07 ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Hi,

The series adds support to IORT RMR nodes specified in IORT
Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
memory ranges that are used by endpoints and require a unity
mapping in SMMU.

We have faced issues with 3408iMR RAID controller cards which
fail to boot when SMMU is enabled. This is because these
controllers make use of host memory for various caching related
purposes and when SMMU is enabled the iMR firmware fails to 
access these memory regions as there is no mapping for them.
IORT RMR provides a way for UEFI to describe and report these
memory regions so that the kernel can make a unity mapping for
these in SMMU.

Change History: 

v6 --> v7

The only change from v6 is the fix pointed out by Steve to
the SMMUv2 SMR bypass install in patch #8.

Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
yet as the series still needs more review[1].

Feedback and tests on this series is very much appreciated.

v5 --> v6
- Addressed comments from Robin & Lorenzo.
  : Moved iort_parse_rmr() to acpi_iort_init() from
    iort_init_platform_devices().
  : Removed use of struct iort_rmr_entry during the initial
    parse. Using struct iommu_resv_region instead.
  : Report RMR address alignment and overlap errors, but continue.
  : Reworked arm_smmu_init_bypass_stes() (patch # 6).
- Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
- Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
  on Type of RMR region. Suggested by Jon N.

Thanks,
Shameer
[0] https://developer.arm.com/documentation/den0049/latest/
[1] https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.kolothum.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
------
v4 --> v5
 -Added a fw_data union to struct iommu_resv_region and removed
  struct iommu_rmr (Based on comments from Joerg/Robin).
 -Added iommu_put_rmrs() to release mem.
 -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
  yet because of the above changes.

v3 -->v4
-Included the SMMUv2 SMR bypass install changes suggested by
 Steve(patch #7)
-As per Robin's comments, RMR reserve implementation is now
 more generic  (patch #8) and dropped v3 patches 8 and 10.
-Rebase to 5.13-rc1

RFC v2 --> v3
 -Dropped RFC tag as the ACPICA header changes are now ready to be
  part of 5.13[0]. But this series still has a dependency on that patch.
 -Added IORT E.b related changes(node flags, _DSM function 5 checks for
  PCIe).
 -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
  discussion here[1].
 -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
------

Jon Nettleton (1):
  iommu/arm-smmu: Get associated RMR info and install bypass SMR

Shameer Kolothum (8):
  iommu: Introduce a union to struct iommu_resv_region
  ACPI/IORT: Add support for RMR node parsing
  iommu/dma: Introduce generic helper to retrieve RMR info
  ACPI/IORT: Add a helper to retrieve RMR memory regions
  iommu/arm-smmu-v3: Introduce strtab init helper
  iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
    bypass
  iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
  iommu/dma: Reserve any RMR regions associated with a dev

 drivers/acpi/arm64/iort.c                   | 172 +++++++++++++++++++-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
 drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
 drivers/iommu/dma-iommu.c                   |  89 +++++++++-
 include/linux/acpi_iort.h                   |   7 +
 include/linux/dma-iommu.h                   |  13 ++
 include/linux/iommu.h                       |  11 ++
 7 files changed, 393 insertions(+), 23 deletions(-)

-- 
2.17.1


^ permalink raw reply	[flat|nested] 150+ messages in thread

* [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-08-05  8:07 ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: robin.murphy, jon, linuxarm, steven.price, guohanjun, yangyicong,
	Sami.Mujawar, will, wanghuiqiang

Hi,

The series adds support to IORT RMR nodes specified in IORT
Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
memory ranges that are used by endpoints and require a unity
mapping in SMMU.

We have faced issues with 3408iMR RAID controller cards which
fail to boot when SMMU is enabled. This is because these
controllers make use of host memory for various caching related
purposes and when SMMU is enabled the iMR firmware fails to 
access these memory regions as there is no mapping for them.
IORT RMR provides a way for UEFI to describe and report these
memory regions so that the kernel can make a unity mapping for
these in SMMU.

Change History: 

v6 --> v7

The only change from v6 is the fix pointed out by Steve to
the SMMUv2 SMR bypass install in patch #8.

Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
yet as the series still needs more review[1].

Feedback and tests on this series is very much appreciated.

v5 --> v6
- Addressed comments from Robin & Lorenzo.
  : Moved iort_parse_rmr() to acpi_iort_init() from
    iort_init_platform_devices().
  : Removed use of struct iort_rmr_entry during the initial
    parse. Using struct iommu_resv_region instead.
  : Report RMR address alignment and overlap errors, but continue.
  : Reworked arm_smmu_init_bypass_stes() (patch # 6).
- Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
- Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
  on Type of RMR region. Suggested by Jon N.

Thanks,
Shameer
[0] https://developer.arm.com/documentation/den0049/latest/
[1] https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.kolothum.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
------
v4 --> v5
 -Added a fw_data union to struct iommu_resv_region and removed
  struct iommu_rmr (Based on comments from Joerg/Robin).
 -Added iommu_put_rmrs() to release mem.
 -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
  yet because of the above changes.

v3 -->v4
-Included the SMMUv2 SMR bypass install changes suggested by
 Steve(patch #7)
-As per Robin's comments, RMR reserve implementation is now
 more generic  (patch #8) and dropped v3 patches 8 and 10.
-Rebase to 5.13-rc1

RFC v2 --> v3
 -Dropped RFC tag as the ACPICA header changes are now ready to be
  part of 5.13[0]. But this series still has a dependency on that patch.
 -Added IORT E.b related changes(node flags, _DSM function 5 checks for
  PCIe).
 -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
  discussion here[1].
 -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
------

Jon Nettleton (1):
  iommu/arm-smmu: Get associated RMR info and install bypass SMR

Shameer Kolothum (8):
  iommu: Introduce a union to struct iommu_resv_region
  ACPI/IORT: Add support for RMR node parsing
  iommu/dma: Introduce generic helper to retrieve RMR info
  ACPI/IORT: Add a helper to retrieve RMR memory regions
  iommu/arm-smmu-v3: Introduce strtab init helper
  iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
    bypass
  iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
  iommu/dma: Reserve any RMR regions associated with a dev

 drivers/acpi/arm64/iort.c                   | 172 +++++++++++++++++++-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
 drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
 drivers/iommu/dma-iommu.c                   |  89 +++++++++-
 include/linux/acpi_iort.h                   |   7 +
 include/linux/dma-iommu.h                   |  13 ++
 include/linux/iommu.h                       |  11 ++
 7 files changed, 393 insertions(+), 23 deletions(-)

-- 
2.17.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-08-05  8:07 ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Hi,

The series adds support to IORT RMR nodes specified in IORT
Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
memory ranges that are used by endpoints and require a unity
mapping in SMMU.

We have faced issues with 3408iMR RAID controller cards which
fail to boot when SMMU is enabled. This is because these
controllers make use of host memory for various caching related
purposes and when SMMU is enabled the iMR firmware fails to 
access these memory regions as there is no mapping for them.
IORT RMR provides a way for UEFI to describe and report these
memory regions so that the kernel can make a unity mapping for
these in SMMU.

Change History: 

v6 --> v7

The only change from v6 is the fix pointed out by Steve to
the SMMUv2 SMR bypass install in patch #8.

Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
yet as the series still needs more review[1].

Feedback and tests on this series is very much appreciated.

v5 --> v6
- Addressed comments from Robin & Lorenzo.
  : Moved iort_parse_rmr() to acpi_iort_init() from
    iort_init_platform_devices().
  : Removed use of struct iort_rmr_entry during the initial
    parse. Using struct iommu_resv_region instead.
  : Report RMR address alignment and overlap errors, but continue.
  : Reworked arm_smmu_init_bypass_stes() (patch # 6).
- Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
- Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
  on Type of RMR region. Suggested by Jon N.

Thanks,
Shameer
[0] https://developer.arm.com/documentation/den0049/latest/
[1] https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.kolothum.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
------
v4 --> v5
 -Added a fw_data union to struct iommu_resv_region and removed
  struct iommu_rmr (Based on comments from Joerg/Robin).
 -Added iommu_put_rmrs() to release mem.
 -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
  yet because of the above changes.

v3 -->v4
-Included the SMMUv2 SMR bypass install changes suggested by
 Steve(patch #7)
-As per Robin's comments, RMR reserve implementation is now
 more generic  (patch #8) and dropped v3 patches 8 and 10.
-Rebase to 5.13-rc1

RFC v2 --> v3
 -Dropped RFC tag as the ACPICA header changes are now ready to be
  part of 5.13[0]. But this series still has a dependency on that patch.
 -Added IORT E.b related changes(node flags, _DSM function 5 checks for
  PCIe).
 -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
  discussion here[1].
 -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
------

Jon Nettleton (1):
  iommu/arm-smmu: Get associated RMR info and install bypass SMR

Shameer Kolothum (8):
  iommu: Introduce a union to struct iommu_resv_region
  ACPI/IORT: Add support for RMR node parsing
  iommu/dma: Introduce generic helper to retrieve RMR info
  ACPI/IORT: Add a helper to retrieve RMR memory regions
  iommu/arm-smmu-v3: Introduce strtab init helper
  iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
    bypass
  iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
  iommu/dma: Reserve any RMR regions associated with a dev

 drivers/acpi/arm64/iort.c                   | 172 +++++++++++++++++++-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
 drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
 drivers/iommu/dma-iommu.c                   |  89 +++++++++-
 include/linux/acpi_iort.h                   |   7 +
 include/linux/dma-iommu.h                   |  13 ++
 include/linux/iommu.h                       |  11 ++
 7 files changed, 393 insertions(+), 23 deletions(-)

-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
  2021-08-05  8:07 ` Shameer Kolothum
  (?)
@ 2021-08-05  8:07   ` Shameer Kolothum
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

A union is introduced to struct iommu_resv_region to hold
any firmware specific data. This is in preparation to add
support for IORT RMR reserve regions and the union now holds
the RMR specific information.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 include/linux/iommu.h | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 32d448050bf7..bd0e4641c569 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -114,6 +114,13 @@ enum iommu_resv_type {
 	IOMMU_RESV_SW_MSI,
 };
 
+struct iommu_iort_rmr_data {
+#define IOMMU_RMR_REMAP_PERMITTED	(1 << 0)
+	u32 flags;
+	u32 sid;	/* Stream Id associated with RMR entry */
+	void *smmu;	/* Associated IORT SMMU node pointer */
+};
+
 /**
  * struct iommu_resv_region - descriptor for a reserved memory region
  * @list: Linked list pointers
@@ -121,6 +128,7 @@ enum iommu_resv_type {
  * @length: Length of the region in bytes
  * @prot: IOMMU Protection flags (READ/WRITE/...)
  * @type: Type of the reserved region
+ * @rmr: ACPI IORT RMR specific data
  */
 struct iommu_resv_region {
 	struct list_head	list;
@@ -128,6 +136,9 @@ struct iommu_resv_region {
 	size_t			length;
 	int			prot;
 	enum iommu_resv_type	type;
+	union {
+		struct iommu_iort_rmr_data rmr;
+	} fw_data;
 };
 
 /**
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: robin.murphy, jon, linuxarm, steven.price, guohanjun, yangyicong,
	Sami.Mujawar, will, wanghuiqiang

A union is introduced to struct iommu_resv_region to hold
any firmware specific data. This is in preparation to add
support for IORT RMR reserve regions and the union now holds
the RMR specific information.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 include/linux/iommu.h | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 32d448050bf7..bd0e4641c569 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -114,6 +114,13 @@ enum iommu_resv_type {
 	IOMMU_RESV_SW_MSI,
 };
 
+struct iommu_iort_rmr_data {
+#define IOMMU_RMR_REMAP_PERMITTED	(1 << 0)
+	u32 flags;
+	u32 sid;	/* Stream Id associated with RMR entry */
+	void *smmu;	/* Associated IORT SMMU node pointer */
+};
+
 /**
  * struct iommu_resv_region - descriptor for a reserved memory region
  * @list: Linked list pointers
@@ -121,6 +128,7 @@ enum iommu_resv_type {
  * @length: Length of the region in bytes
  * @prot: IOMMU Protection flags (READ/WRITE/...)
  * @type: Type of the reserved region
+ * @rmr: ACPI IORT RMR specific data
  */
 struct iommu_resv_region {
 	struct list_head	list;
@@ -128,6 +136,9 @@ struct iommu_resv_region {
 	size_t			length;
 	int			prot;
 	enum iommu_resv_type	type;
+	union {
+		struct iommu_iort_rmr_data rmr;
+	} fw_data;
 };
 
 /**
-- 
2.17.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

A union is introduced to struct iommu_resv_region to hold
any firmware specific data. This is in preparation to add
support for IORT RMR reserve regions and the union now holds
the RMR specific information.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 include/linux/iommu.h | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 32d448050bf7..bd0e4641c569 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -114,6 +114,13 @@ enum iommu_resv_type {
 	IOMMU_RESV_SW_MSI,
 };
 
+struct iommu_iort_rmr_data {
+#define IOMMU_RMR_REMAP_PERMITTED	(1 << 0)
+	u32 flags;
+	u32 sid;	/* Stream Id associated with RMR entry */
+	void *smmu;	/* Associated IORT SMMU node pointer */
+};
+
 /**
  * struct iommu_resv_region - descriptor for a reserved memory region
  * @list: Linked list pointers
@@ -121,6 +128,7 @@ enum iommu_resv_type {
  * @length: Length of the region in bytes
  * @prot: IOMMU Protection flags (READ/WRITE/...)
  * @type: Type of the reserved region
+ * @rmr: ACPI IORT RMR specific data
  */
 struct iommu_resv_region {
 	struct list_head	list;
@@ -128,6 +136,9 @@ struct iommu_resv_region {
 	size_t			length;
 	int			prot;
 	enum iommu_resv_type	type;
+	union {
+		struct iommu_iort_rmr_data rmr;
+	} fw_data;
 };
 
 /**
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-08-05  8:07 ` Shameer Kolothum
  (?)
@ 2021-08-05  8:07   ` Shameer Kolothum
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Add support for parsing RMR node information from ACPI.

Find the associated streamid and smmu node info from the
RMR node and populate a linked list with RMR memory
descriptors.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/acpi/arm64/iort.c | 134 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 133 insertions(+), 1 deletion(-)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index 3b23fb775ac4..d76ba46ebe67 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -40,6 +40,8 @@ struct iort_fwnode {
 static LIST_HEAD(iort_fwnode_list);
 static DEFINE_SPINLOCK(iort_fwnode_lock);
 
+static LIST_HEAD(iort_rmr_list);	/* list of RMR regions from ACPI */
+
 /**
  * iort_set_fwnode() - Create iort_fwnode and use it to register
  *		       iommu data in the iort_fwnode_list
@@ -393,7 +395,8 @@ static struct acpi_iort_node *iort_node_get_id(struct acpi_iort_node *node,
 		if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
 		    node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
 		    node->type == ACPI_IORT_NODE_SMMU_V3 ||
-		    node->type == ACPI_IORT_NODE_PMCG) {
+		    node->type == ACPI_IORT_NODE_PMCG ||
+		    node->type == ACPI_IORT_NODE_RMR) {
 			*id_out = map->output_base;
 			return parent;
 		}
@@ -1566,6 +1569,134 @@ static void __init iort_enable_acs(struct acpi_iort_node *iort_node)
 #else
 static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
 #endif
+static void iort_rmr_desc_check_overlap(struct acpi_iort_rmr_desc *desc, u32 count)
+{
+	int i, j;
+
+	for (i = 0; i < count; i++) {
+		u64 end, start = desc[i].base_address, length = desc[i].length;
+
+		end = start + length - 1;
+
+		/* Check for address overlap */
+		for (j = i + 1; j < count; j++) {
+			u64 e_start = desc[j].base_address;
+			u64 e_end = e_start + desc[j].length - 1;
+
+			if (start <= e_end && end >= e_start)
+				pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] overlaps, continue anyway\n",
+				       start, end);
+		}
+	}
+}
+
+static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
+{
+	struct acpi_iort_node *smmu;
+	struct acpi_iort_rmr *rmr;
+	struct acpi_iort_rmr_desc *rmr_desc;
+	u32 map_count = iort_node->mapping_count;
+	u32 sid;
+	int i;
+
+	if (!iort_node->mapping_offset || map_count != 1) {
+		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
+		       iort_node);
+		return;
+	}
+
+	/* Retrieve associated smmu and stream id */
+	smmu = iort_node_get_id(iort_node, &sid, 0);
+	if (!smmu) {
+		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
+		       iort_node);
+		return;
+	}
+
+	/* Retrieve RMR data */
+	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
+	if (!rmr->rmr_offset || !rmr->rmr_count) {
+		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
+		       iort_node);
+		return;
+	}
+
+	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
+				rmr->rmr_offset);
+
+	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
+
+	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
+		struct iommu_resv_region *region;
+		enum iommu_resv_type type;
+		int prot = IOMMU_READ | IOMMU_WRITE;
+		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
+
+		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
+			/* PAGE align base addr and size */
+			addr &= PAGE_MASK;
+			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
+
+			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
+			       rmr_desc->base_address,
+			       rmr_desc->base_address + rmr_desc->length - 1,
+			       addr, addr + size - 1);
+		}
+		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
+			type = IOMMU_RESV_DIRECT_RELAXABLE;
+			/*
+			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
+			 * normally used for allocated system memory that is
+			 * then used for device specific reserved regions.
+			 */
+			prot |= IOMMU_CACHE;
+		} else {
+			type = IOMMU_RESV_DIRECT;
+			/*
+			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
+			 * for device memory like MSI doorbell.
+			 */
+			prot |= IOMMU_MMIO;
+		}
+
+		region = iommu_alloc_resv_region(addr, size, prot, type);
+		if (region) {
+			region->fw_data.rmr.flags = rmr->flags;
+			region->fw_data.rmr.sid = sid;
+			region->fw_data.rmr.smmu = smmu;
+			list_add_tail(&region->list, &iort_rmr_list);
+		}
+	}
+}
+
+static void __init iort_parse_rmr(void)
+{
+	struct acpi_iort_node *iort_node, *iort_end;
+	struct acpi_table_iort *iort;
+	int i;
+
+	if (iort_table->revision < 3)
+		return;
+
+	iort = (struct acpi_table_iort *)iort_table;
+
+	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
+				 iort->node_offset);
+	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
+				iort_table->length);
+
+	for (i = 0; i < iort->node_count; i++) {
+		if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
+			       "IORT node pointer overflows, bad table!\n"))
+			return;
+
+		if (iort_node->type == ACPI_IORT_NODE_RMR)
+			iort_node_get_rmr_info(iort_node);
+
+		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
+					 iort_node->length);
+	}
+}
 
 static void __init iort_init_platform_devices(void)
 {
@@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
 	}
 
 	iort_init_platform_devices();
+	iort_parse_rmr();
 }
 
 #ifdef CONFIG_ZONE_DMA
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: robin.murphy, jon, linuxarm, steven.price, guohanjun, yangyicong,
	Sami.Mujawar, will, wanghuiqiang

Add support for parsing RMR node information from ACPI.

Find the associated streamid and smmu node info from the
RMR node and populate a linked list with RMR memory
descriptors.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/acpi/arm64/iort.c | 134 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 133 insertions(+), 1 deletion(-)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index 3b23fb775ac4..d76ba46ebe67 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -40,6 +40,8 @@ struct iort_fwnode {
 static LIST_HEAD(iort_fwnode_list);
 static DEFINE_SPINLOCK(iort_fwnode_lock);
 
+static LIST_HEAD(iort_rmr_list);	/* list of RMR regions from ACPI */
+
 /**
  * iort_set_fwnode() - Create iort_fwnode and use it to register
  *		       iommu data in the iort_fwnode_list
@@ -393,7 +395,8 @@ static struct acpi_iort_node *iort_node_get_id(struct acpi_iort_node *node,
 		if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
 		    node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
 		    node->type == ACPI_IORT_NODE_SMMU_V3 ||
-		    node->type == ACPI_IORT_NODE_PMCG) {
+		    node->type == ACPI_IORT_NODE_PMCG ||
+		    node->type == ACPI_IORT_NODE_RMR) {
 			*id_out = map->output_base;
 			return parent;
 		}
@@ -1566,6 +1569,134 @@ static void __init iort_enable_acs(struct acpi_iort_node *iort_node)
 #else
 static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
 #endif
+static void iort_rmr_desc_check_overlap(struct acpi_iort_rmr_desc *desc, u32 count)
+{
+	int i, j;
+
+	for (i = 0; i < count; i++) {
+		u64 end, start = desc[i].base_address, length = desc[i].length;
+
+		end = start + length - 1;
+
+		/* Check for address overlap */
+		for (j = i + 1; j < count; j++) {
+			u64 e_start = desc[j].base_address;
+			u64 e_end = e_start + desc[j].length - 1;
+
+			if (start <= e_end && end >= e_start)
+				pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] overlaps, continue anyway\n",
+				       start, end);
+		}
+	}
+}
+
+static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
+{
+	struct acpi_iort_node *smmu;
+	struct acpi_iort_rmr *rmr;
+	struct acpi_iort_rmr_desc *rmr_desc;
+	u32 map_count = iort_node->mapping_count;
+	u32 sid;
+	int i;
+
+	if (!iort_node->mapping_offset || map_count != 1) {
+		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
+		       iort_node);
+		return;
+	}
+
+	/* Retrieve associated smmu and stream id */
+	smmu = iort_node_get_id(iort_node, &sid, 0);
+	if (!smmu) {
+		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
+		       iort_node);
+		return;
+	}
+
+	/* Retrieve RMR data */
+	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
+	if (!rmr->rmr_offset || !rmr->rmr_count) {
+		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
+		       iort_node);
+		return;
+	}
+
+	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
+				rmr->rmr_offset);
+
+	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
+
+	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
+		struct iommu_resv_region *region;
+		enum iommu_resv_type type;
+		int prot = IOMMU_READ | IOMMU_WRITE;
+		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
+
+		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
+			/* PAGE align base addr and size */
+			addr &= PAGE_MASK;
+			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
+
+			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
+			       rmr_desc->base_address,
+			       rmr_desc->base_address + rmr_desc->length - 1,
+			       addr, addr + size - 1);
+		}
+		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
+			type = IOMMU_RESV_DIRECT_RELAXABLE;
+			/*
+			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
+			 * normally used for allocated system memory that is
+			 * then used for device specific reserved regions.
+			 */
+			prot |= IOMMU_CACHE;
+		} else {
+			type = IOMMU_RESV_DIRECT;
+			/*
+			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
+			 * for device memory like MSI doorbell.
+			 */
+			prot |= IOMMU_MMIO;
+		}
+
+		region = iommu_alloc_resv_region(addr, size, prot, type);
+		if (region) {
+			region->fw_data.rmr.flags = rmr->flags;
+			region->fw_data.rmr.sid = sid;
+			region->fw_data.rmr.smmu = smmu;
+			list_add_tail(&region->list, &iort_rmr_list);
+		}
+	}
+}
+
+static void __init iort_parse_rmr(void)
+{
+	struct acpi_iort_node *iort_node, *iort_end;
+	struct acpi_table_iort *iort;
+	int i;
+
+	if (iort_table->revision < 3)
+		return;
+
+	iort = (struct acpi_table_iort *)iort_table;
+
+	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
+				 iort->node_offset);
+	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
+				iort_table->length);
+
+	for (i = 0; i < iort->node_count; i++) {
+		if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
+			       "IORT node pointer overflows, bad table!\n"))
+			return;
+
+		if (iort_node->type == ACPI_IORT_NODE_RMR)
+			iort_node_get_rmr_info(iort_node);
+
+		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
+					 iort_node->length);
+	}
+}
 
 static void __init iort_init_platform_devices(void)
 {
@@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
 	}
 
 	iort_init_platform_devices();
+	iort_parse_rmr();
 }
 
 #ifdef CONFIG_ZONE_DMA
-- 
2.17.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Add support for parsing RMR node information from ACPI.

Find the associated streamid and smmu node info from the
RMR node and populate a linked list with RMR memory
descriptors.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/acpi/arm64/iort.c | 134 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 133 insertions(+), 1 deletion(-)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index 3b23fb775ac4..d76ba46ebe67 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -40,6 +40,8 @@ struct iort_fwnode {
 static LIST_HEAD(iort_fwnode_list);
 static DEFINE_SPINLOCK(iort_fwnode_lock);
 
+static LIST_HEAD(iort_rmr_list);	/* list of RMR regions from ACPI */
+
 /**
  * iort_set_fwnode() - Create iort_fwnode and use it to register
  *		       iommu data in the iort_fwnode_list
@@ -393,7 +395,8 @@ static struct acpi_iort_node *iort_node_get_id(struct acpi_iort_node *node,
 		if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
 		    node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
 		    node->type == ACPI_IORT_NODE_SMMU_V3 ||
-		    node->type == ACPI_IORT_NODE_PMCG) {
+		    node->type == ACPI_IORT_NODE_PMCG ||
+		    node->type == ACPI_IORT_NODE_RMR) {
 			*id_out = map->output_base;
 			return parent;
 		}
@@ -1566,6 +1569,134 @@ static void __init iort_enable_acs(struct acpi_iort_node *iort_node)
 #else
 static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
 #endif
+static void iort_rmr_desc_check_overlap(struct acpi_iort_rmr_desc *desc, u32 count)
+{
+	int i, j;
+
+	for (i = 0; i < count; i++) {
+		u64 end, start = desc[i].base_address, length = desc[i].length;
+
+		end = start + length - 1;
+
+		/* Check for address overlap */
+		for (j = i + 1; j < count; j++) {
+			u64 e_start = desc[j].base_address;
+			u64 e_end = e_start + desc[j].length - 1;
+
+			if (start <= e_end && end >= e_start)
+				pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] overlaps, continue anyway\n",
+				       start, end);
+		}
+	}
+}
+
+static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
+{
+	struct acpi_iort_node *smmu;
+	struct acpi_iort_rmr *rmr;
+	struct acpi_iort_rmr_desc *rmr_desc;
+	u32 map_count = iort_node->mapping_count;
+	u32 sid;
+	int i;
+
+	if (!iort_node->mapping_offset || map_count != 1) {
+		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
+		       iort_node);
+		return;
+	}
+
+	/* Retrieve associated smmu and stream id */
+	smmu = iort_node_get_id(iort_node, &sid, 0);
+	if (!smmu) {
+		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
+		       iort_node);
+		return;
+	}
+
+	/* Retrieve RMR data */
+	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
+	if (!rmr->rmr_offset || !rmr->rmr_count) {
+		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
+		       iort_node);
+		return;
+	}
+
+	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
+				rmr->rmr_offset);
+
+	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
+
+	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
+		struct iommu_resv_region *region;
+		enum iommu_resv_type type;
+		int prot = IOMMU_READ | IOMMU_WRITE;
+		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
+
+		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
+			/* PAGE align base addr and size */
+			addr &= PAGE_MASK;
+			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
+
+			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
+			       rmr_desc->base_address,
+			       rmr_desc->base_address + rmr_desc->length - 1,
+			       addr, addr + size - 1);
+		}
+		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
+			type = IOMMU_RESV_DIRECT_RELAXABLE;
+			/*
+			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
+			 * normally used for allocated system memory that is
+			 * then used for device specific reserved regions.
+			 */
+			prot |= IOMMU_CACHE;
+		} else {
+			type = IOMMU_RESV_DIRECT;
+			/*
+			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
+			 * for device memory like MSI doorbell.
+			 */
+			prot |= IOMMU_MMIO;
+		}
+
+		region = iommu_alloc_resv_region(addr, size, prot, type);
+		if (region) {
+			region->fw_data.rmr.flags = rmr->flags;
+			region->fw_data.rmr.sid = sid;
+			region->fw_data.rmr.smmu = smmu;
+			list_add_tail(&region->list, &iort_rmr_list);
+		}
+	}
+}
+
+static void __init iort_parse_rmr(void)
+{
+	struct acpi_iort_node *iort_node, *iort_end;
+	struct acpi_table_iort *iort;
+	int i;
+
+	if (iort_table->revision < 3)
+		return;
+
+	iort = (struct acpi_table_iort *)iort_table;
+
+	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
+				 iort->node_offset);
+	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
+				iort_table->length);
+
+	for (i = 0; i < iort->node_count; i++) {
+		if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
+			       "IORT node pointer overflows, bad table!\n"))
+			return;
+
+		if (iort_node->type == ACPI_IORT_NODE_RMR)
+			iort_node_get_rmr_info(iort_node);
+
+		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
+					 iort_node->length);
+	}
+}
 
 static void __init iort_init_platform_devices(void)
 {
@@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
 	}
 
 	iort_init_platform_devices();
+	iort_parse_rmr();
 }
 
 #ifdef CONFIG_ZONE_DMA
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 3/9] iommu/dma: Introduce generic helper to retrieve RMR info
  2021-08-05  8:07 ` Shameer Kolothum
  (?)
@ 2021-08-05  8:07   ` Shameer Kolothum
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Reserved Memory Regions(RMR) associated with an IOMMU can be
described through ACPI IORT tables in systems with devices
that require a unity mapping or bypass for those
regions.

Introduce a generic interface so that IOMMU drivers can retrieve
and set up necessary mappings.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/dma-iommu.c | 29 +++++++++++++++++++++++++++++
 include/linux/dma-iommu.h | 13 +++++++++++++
 2 files changed, 42 insertions(+)

diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 98ba927aee1a..2fa2445e9070 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -174,6 +174,35 @@ void iommu_put_dma_cookie(struct iommu_domain *domain)
 }
 EXPORT_SYMBOL(iommu_put_dma_cookie);
 
+/**
+ *
+ * iommu_dma_get_rmrs - Retrieve Reserved Memory Regions(RMRs) associated
+ *                      with a given IOMMU
+ * @iommu_fwnode: fwnode associated with IOMMU
+ * @list: RMR list to be populated
+ *
+ */
+int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
+		       struct list_head *list)
+{
+	return -EINVAL;
+}
+EXPORT_SYMBOL(iommu_dma_get_rmrs);
+
+/**
+ *
+ * iommu_dma_put_rmrs - Release Reserved Memory Regions(RMRs) associated
+ *                      with a given IOMMU
+ * @iommu_fwnode: fwnode associated with IOMMU
+ * @list: RMR list
+ *
+ */
+void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *list)
+{
+}
+EXPORT_SYMBOL(iommu_dma_put_rmrs);
+
 /**
  * iommu_dma_get_resv_regions - Reserved region driver helper
  * @dev: Device from iommu_get_resv_regions()
diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
index 758ca4694257..3b7b2d096c6e 100644
--- a/include/linux/dma-iommu.h
+++ b/include/linux/dma-iommu.h
@@ -42,12 +42,16 @@ void iommu_dma_free_cpu_cached_iovas(unsigned int cpu,
 
 extern bool iommu_dma_forcedac;
 
+int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list);
+void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head *list);
+
 #else /* CONFIG_IOMMU_DMA */
 
 struct iommu_domain;
 struct msi_desc;
 struct msi_msg;
 struct device;
+struct fwnode_handle;
 
 static inline void iommu_setup_dma_ops(struct device *dev, u64 dma_base,
 				       u64 dma_limit)
@@ -83,5 +87,14 @@ static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_he
 {
 }
 
+static int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list)
+{
+	return -ENODEV;
+}
+
+static void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head *list)
+{
+}
+
 #endif	/* CONFIG_IOMMU_DMA */
 #endif	/* __DMA_IOMMU_H */
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 3/9] iommu/dma: Introduce generic helper to retrieve RMR info
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: robin.murphy, jon, linuxarm, steven.price, guohanjun, yangyicong,
	Sami.Mujawar, will, wanghuiqiang

Reserved Memory Regions(RMR) associated with an IOMMU can be
described through ACPI IORT tables in systems with devices
that require a unity mapping or bypass for those
regions.

Introduce a generic interface so that IOMMU drivers can retrieve
and set up necessary mappings.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/dma-iommu.c | 29 +++++++++++++++++++++++++++++
 include/linux/dma-iommu.h | 13 +++++++++++++
 2 files changed, 42 insertions(+)

diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 98ba927aee1a..2fa2445e9070 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -174,6 +174,35 @@ void iommu_put_dma_cookie(struct iommu_domain *domain)
 }
 EXPORT_SYMBOL(iommu_put_dma_cookie);
 
+/**
+ *
+ * iommu_dma_get_rmrs - Retrieve Reserved Memory Regions(RMRs) associated
+ *                      with a given IOMMU
+ * @iommu_fwnode: fwnode associated with IOMMU
+ * @list: RMR list to be populated
+ *
+ */
+int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
+		       struct list_head *list)
+{
+	return -EINVAL;
+}
+EXPORT_SYMBOL(iommu_dma_get_rmrs);
+
+/**
+ *
+ * iommu_dma_put_rmrs - Release Reserved Memory Regions(RMRs) associated
+ *                      with a given IOMMU
+ * @iommu_fwnode: fwnode associated with IOMMU
+ * @list: RMR list
+ *
+ */
+void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *list)
+{
+}
+EXPORT_SYMBOL(iommu_dma_put_rmrs);
+
 /**
  * iommu_dma_get_resv_regions - Reserved region driver helper
  * @dev: Device from iommu_get_resv_regions()
diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
index 758ca4694257..3b7b2d096c6e 100644
--- a/include/linux/dma-iommu.h
+++ b/include/linux/dma-iommu.h
@@ -42,12 +42,16 @@ void iommu_dma_free_cpu_cached_iovas(unsigned int cpu,
 
 extern bool iommu_dma_forcedac;
 
+int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list);
+void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head *list);
+
 #else /* CONFIG_IOMMU_DMA */
 
 struct iommu_domain;
 struct msi_desc;
 struct msi_msg;
 struct device;
+struct fwnode_handle;
 
 static inline void iommu_setup_dma_ops(struct device *dev, u64 dma_base,
 				       u64 dma_limit)
@@ -83,5 +87,14 @@ static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_he
 {
 }
 
+static int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list)
+{
+	return -ENODEV;
+}
+
+static void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head *list)
+{
+}
+
 #endif	/* CONFIG_IOMMU_DMA */
 #endif	/* __DMA_IOMMU_H */
-- 
2.17.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 3/9] iommu/dma: Introduce generic helper to retrieve RMR info
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Reserved Memory Regions(RMR) associated with an IOMMU can be
described through ACPI IORT tables in systems with devices
that require a unity mapping or bypass for those
regions.

Introduce a generic interface so that IOMMU drivers can retrieve
and set up necessary mappings.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/dma-iommu.c | 29 +++++++++++++++++++++++++++++
 include/linux/dma-iommu.h | 13 +++++++++++++
 2 files changed, 42 insertions(+)

diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 98ba927aee1a..2fa2445e9070 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -174,6 +174,35 @@ void iommu_put_dma_cookie(struct iommu_domain *domain)
 }
 EXPORT_SYMBOL(iommu_put_dma_cookie);
 
+/**
+ *
+ * iommu_dma_get_rmrs - Retrieve Reserved Memory Regions(RMRs) associated
+ *                      with a given IOMMU
+ * @iommu_fwnode: fwnode associated with IOMMU
+ * @list: RMR list to be populated
+ *
+ */
+int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
+		       struct list_head *list)
+{
+	return -EINVAL;
+}
+EXPORT_SYMBOL(iommu_dma_get_rmrs);
+
+/**
+ *
+ * iommu_dma_put_rmrs - Release Reserved Memory Regions(RMRs) associated
+ *                      with a given IOMMU
+ * @iommu_fwnode: fwnode associated with IOMMU
+ * @list: RMR list
+ *
+ */
+void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *list)
+{
+}
+EXPORT_SYMBOL(iommu_dma_put_rmrs);
+
 /**
  * iommu_dma_get_resv_regions - Reserved region driver helper
  * @dev: Device from iommu_get_resv_regions()
diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
index 758ca4694257..3b7b2d096c6e 100644
--- a/include/linux/dma-iommu.h
+++ b/include/linux/dma-iommu.h
@@ -42,12 +42,16 @@ void iommu_dma_free_cpu_cached_iovas(unsigned int cpu,
 
 extern bool iommu_dma_forcedac;
 
+int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list);
+void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head *list);
+
 #else /* CONFIG_IOMMU_DMA */
 
 struct iommu_domain;
 struct msi_desc;
 struct msi_msg;
 struct device;
+struct fwnode_handle;
 
 static inline void iommu_setup_dma_ops(struct device *dev, u64 dma_base,
 				       u64 dma_limit)
@@ -83,5 +87,14 @@ static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_he
 {
 }
 
+static int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list)
+{
+	return -ENODEV;
+}
+
+static void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head *list)
+{
+}
+
 #endif	/* CONFIG_IOMMU_DMA */
 #endif	/* __DMA_IOMMU_H */
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 4/9] ACPI/IORT: Add a helper to retrieve RMR memory regions
  2021-08-05  8:07 ` Shameer Kolothum
  (?)
@ 2021-08-05  8:07   ` Shameer Kolothum
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Add a helper function (iort_iommu_get_rmrs()) that retrieves RMR
memory descriptors associated with a given IOMMU. This will be used
by IOMMU drivers to setup necessary mappings.

Invoke it from the generic helper iommu_dma_get_rmrs().

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/acpi/arm64/iort.c | 38 ++++++++++++++++++++++++++++++++++++++
 drivers/iommu/dma-iommu.c |  4 ++++
 include/linux/acpi_iort.h |  7 +++++++
 3 files changed, 49 insertions(+)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index d76ba46ebe67..3c32d62e63b6 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -809,6 +809,42 @@ static struct acpi_iort_node *iort_get_msi_resv_iommu(struct device *dev)
 	return NULL;
 }
 
+/**
+ * iort_iommu_get_rmrs() - Helper to retrieve RMR info associated with IOMMU
+ * @iommu_fwnode: fwnode for the IOMMU
+ * @head: RMR list head to be populated
+ *
+ * Returns: 0 on success, <0 failure. Please note, we will keep the already
+ *          allocated RMR reserve regions in case of a kmemdup()
+ *          failure.
+ */
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *head)
+{
+	struct iommu_resv_region *e;
+	struct acpi_iort_node *iommu;
+	int rmrs = 0;
+
+	iommu = iort_get_iort_node(iommu_fwnode);
+	if (!iommu || list_empty(&iort_rmr_list))
+		return -ENODEV;
+
+	list_for_each_entry(e, &iort_rmr_list, list) {
+		struct iommu_resv_region *region;
+
+		if (e->fw_data.rmr.smmu != iommu)
+			continue;
+
+		region = kmemdup(e, sizeof(*region), GFP_KERNEL);
+		if (region) {
+			list_add_tail(&region->list, head);
+			rmrs++;
+		}
+	}
+
+	return (rmrs == 0) ? -ENODEV : 0;
+}
+
 /**
  * iort_iommu_msi_get_resv_regions - Reserved region driver helper
  * @dev: Device from iommu_get_resv_regions()
@@ -1041,6 +1077,8 @@ int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
 { return 0; }
 int iort_iommu_configure_id(struct device *dev, const u32 *input_id)
 { return -ENODEV; }
+int iort_iommu_get_rmrs(struct fwnode_handle *fwnode, struct list_head *head)
+{ return -ENODEV; }
 #endif
 
 static int nc_dma_get_range(struct device *dev, u64 *size)
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 2fa2445e9070..1b6e27475279 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -185,6 +185,9 @@ EXPORT_SYMBOL(iommu_put_dma_cookie);
 int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
 		       struct list_head *list)
 {
+	if (!is_of_node(iommu_fwnode))
+		return iort_iommu_get_rmrs(iommu_fwnode, list);
+
 	return -EINVAL;
 }
 EXPORT_SYMBOL(iommu_dma_get_rmrs);
@@ -200,6 +203,7 @@ EXPORT_SYMBOL(iommu_dma_get_rmrs);
 void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
 			struct list_head *list)
 {
+	generic_iommu_put_resv_regions(iommu_fwnode->dev, list);
 }
 EXPORT_SYMBOL(iommu_dma_put_rmrs);
 
diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
index f1f0842a2cb2..d8c030c103f5 100644
--- a/include/linux/acpi_iort.h
+++ b/include/linux/acpi_iort.h
@@ -38,6 +38,8 @@ int iort_dma_get_ranges(struct device *dev, u64 *size);
 int iort_iommu_configure_id(struct device *dev, const u32 *id_in);
 int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head);
 phys_addr_t acpi_iort_dma_get_max_cpu_address(void);
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *list);
 #else
 static inline void acpi_iort_init(void) { }
 static inline u32 iort_msi_map_id(struct device *dev, u32 id)
@@ -57,6 +59,11 @@ int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
 
 static inline phys_addr_t acpi_iort_dma_get_max_cpu_address(void)
 { return PHYS_ADDR_MAX; }
+
+static inline
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *list)
+{ return -ENODEV; }
 #endif
 
 #endif /* __ACPI_IORT_H__ */
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 4/9] ACPI/IORT: Add a helper to retrieve RMR memory regions
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: robin.murphy, jon, linuxarm, steven.price, guohanjun, yangyicong,
	Sami.Mujawar, will, wanghuiqiang

Add a helper function (iort_iommu_get_rmrs()) that retrieves RMR
memory descriptors associated with a given IOMMU. This will be used
by IOMMU drivers to setup necessary mappings.

Invoke it from the generic helper iommu_dma_get_rmrs().

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/acpi/arm64/iort.c | 38 ++++++++++++++++++++++++++++++++++++++
 drivers/iommu/dma-iommu.c |  4 ++++
 include/linux/acpi_iort.h |  7 +++++++
 3 files changed, 49 insertions(+)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index d76ba46ebe67..3c32d62e63b6 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -809,6 +809,42 @@ static struct acpi_iort_node *iort_get_msi_resv_iommu(struct device *dev)
 	return NULL;
 }
 
+/**
+ * iort_iommu_get_rmrs() - Helper to retrieve RMR info associated with IOMMU
+ * @iommu_fwnode: fwnode for the IOMMU
+ * @head: RMR list head to be populated
+ *
+ * Returns: 0 on success, <0 failure. Please note, we will keep the already
+ *          allocated RMR reserve regions in case of a kmemdup()
+ *          failure.
+ */
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *head)
+{
+	struct iommu_resv_region *e;
+	struct acpi_iort_node *iommu;
+	int rmrs = 0;
+
+	iommu = iort_get_iort_node(iommu_fwnode);
+	if (!iommu || list_empty(&iort_rmr_list))
+		return -ENODEV;
+
+	list_for_each_entry(e, &iort_rmr_list, list) {
+		struct iommu_resv_region *region;
+
+		if (e->fw_data.rmr.smmu != iommu)
+			continue;
+
+		region = kmemdup(e, sizeof(*region), GFP_KERNEL);
+		if (region) {
+			list_add_tail(&region->list, head);
+			rmrs++;
+		}
+	}
+
+	return (rmrs == 0) ? -ENODEV : 0;
+}
+
 /**
  * iort_iommu_msi_get_resv_regions - Reserved region driver helper
  * @dev: Device from iommu_get_resv_regions()
@@ -1041,6 +1077,8 @@ int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
 { return 0; }
 int iort_iommu_configure_id(struct device *dev, const u32 *input_id)
 { return -ENODEV; }
+int iort_iommu_get_rmrs(struct fwnode_handle *fwnode, struct list_head *head)
+{ return -ENODEV; }
 #endif
 
 static int nc_dma_get_range(struct device *dev, u64 *size)
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 2fa2445e9070..1b6e27475279 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -185,6 +185,9 @@ EXPORT_SYMBOL(iommu_put_dma_cookie);
 int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
 		       struct list_head *list)
 {
+	if (!is_of_node(iommu_fwnode))
+		return iort_iommu_get_rmrs(iommu_fwnode, list);
+
 	return -EINVAL;
 }
 EXPORT_SYMBOL(iommu_dma_get_rmrs);
@@ -200,6 +203,7 @@ EXPORT_SYMBOL(iommu_dma_get_rmrs);
 void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
 			struct list_head *list)
 {
+	generic_iommu_put_resv_regions(iommu_fwnode->dev, list);
 }
 EXPORT_SYMBOL(iommu_dma_put_rmrs);
 
diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
index f1f0842a2cb2..d8c030c103f5 100644
--- a/include/linux/acpi_iort.h
+++ b/include/linux/acpi_iort.h
@@ -38,6 +38,8 @@ int iort_dma_get_ranges(struct device *dev, u64 *size);
 int iort_iommu_configure_id(struct device *dev, const u32 *id_in);
 int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head);
 phys_addr_t acpi_iort_dma_get_max_cpu_address(void);
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *list);
 #else
 static inline void acpi_iort_init(void) { }
 static inline u32 iort_msi_map_id(struct device *dev, u32 id)
@@ -57,6 +59,11 @@ int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
 
 static inline phys_addr_t acpi_iort_dma_get_max_cpu_address(void)
 { return PHYS_ADDR_MAX; }
+
+static inline
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *list)
+{ return -ENODEV; }
 #endif
 
 #endif /* __ACPI_IORT_H__ */
-- 
2.17.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 4/9] ACPI/IORT: Add a helper to retrieve RMR memory regions
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Add a helper function (iort_iommu_get_rmrs()) that retrieves RMR
memory descriptors associated with a given IOMMU. This will be used
by IOMMU drivers to setup necessary mappings.

Invoke it from the generic helper iommu_dma_get_rmrs().

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/acpi/arm64/iort.c | 38 ++++++++++++++++++++++++++++++++++++++
 drivers/iommu/dma-iommu.c |  4 ++++
 include/linux/acpi_iort.h |  7 +++++++
 3 files changed, 49 insertions(+)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index d76ba46ebe67..3c32d62e63b6 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -809,6 +809,42 @@ static struct acpi_iort_node *iort_get_msi_resv_iommu(struct device *dev)
 	return NULL;
 }
 
+/**
+ * iort_iommu_get_rmrs() - Helper to retrieve RMR info associated with IOMMU
+ * @iommu_fwnode: fwnode for the IOMMU
+ * @head: RMR list head to be populated
+ *
+ * Returns: 0 on success, <0 failure. Please note, we will keep the already
+ *          allocated RMR reserve regions in case of a kmemdup()
+ *          failure.
+ */
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *head)
+{
+	struct iommu_resv_region *e;
+	struct acpi_iort_node *iommu;
+	int rmrs = 0;
+
+	iommu = iort_get_iort_node(iommu_fwnode);
+	if (!iommu || list_empty(&iort_rmr_list))
+		return -ENODEV;
+
+	list_for_each_entry(e, &iort_rmr_list, list) {
+		struct iommu_resv_region *region;
+
+		if (e->fw_data.rmr.smmu != iommu)
+			continue;
+
+		region = kmemdup(e, sizeof(*region), GFP_KERNEL);
+		if (region) {
+			list_add_tail(&region->list, head);
+			rmrs++;
+		}
+	}
+
+	return (rmrs == 0) ? -ENODEV : 0;
+}
+
 /**
  * iort_iommu_msi_get_resv_regions - Reserved region driver helper
  * @dev: Device from iommu_get_resv_regions()
@@ -1041,6 +1077,8 @@ int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
 { return 0; }
 int iort_iommu_configure_id(struct device *dev, const u32 *input_id)
 { return -ENODEV; }
+int iort_iommu_get_rmrs(struct fwnode_handle *fwnode, struct list_head *head)
+{ return -ENODEV; }
 #endif
 
 static int nc_dma_get_range(struct device *dev, u64 *size)
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 2fa2445e9070..1b6e27475279 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -185,6 +185,9 @@ EXPORT_SYMBOL(iommu_put_dma_cookie);
 int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
 		       struct list_head *list)
 {
+	if (!is_of_node(iommu_fwnode))
+		return iort_iommu_get_rmrs(iommu_fwnode, list);
+
 	return -EINVAL;
 }
 EXPORT_SYMBOL(iommu_dma_get_rmrs);
@@ -200,6 +203,7 @@ EXPORT_SYMBOL(iommu_dma_get_rmrs);
 void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
 			struct list_head *list)
 {
+	generic_iommu_put_resv_regions(iommu_fwnode->dev, list);
 }
 EXPORT_SYMBOL(iommu_dma_put_rmrs);
 
diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
index f1f0842a2cb2..d8c030c103f5 100644
--- a/include/linux/acpi_iort.h
+++ b/include/linux/acpi_iort.h
@@ -38,6 +38,8 @@ int iort_dma_get_ranges(struct device *dev, u64 *size);
 int iort_iommu_configure_id(struct device *dev, const u32 *id_in);
 int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head);
 phys_addr_t acpi_iort_dma_get_max_cpu_address(void);
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *list);
 #else
 static inline void acpi_iort_init(void) { }
 static inline u32 iort_msi_map_id(struct device *dev, u32 id)
@@ -57,6 +59,11 @@ int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
 
 static inline phys_addr_t acpi_iort_dma_get_max_cpu_address(void)
 { return PHYS_ADDR_MAX; }
+
+static inline
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *list)
+{ return -ENODEV; }
 #endif
 
 #endif /* __ACPI_IORT_H__ */
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 5/9] iommu/arm-smmu-v3: Introduce strtab init helper
  2021-08-05  8:07 ` Shameer Kolothum
  (?)
@ 2021-08-05  8:07   ` Shameer Kolothum
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Introduce a helper to check the sid range and to init the l2 strtab
entries(bypass). This will be useful when we have to initialize the
l2 strtab with bypass for RMR SIDs.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 +++++++++++----------
 1 file changed, 15 insertions(+), 13 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 235f9bdaeaf2..31940e53c675 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2518,6 +2518,19 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid)
 	return sid < limit;
 }
 
+static int arm_smmu_init_sid_strtab(struct arm_smmu_device *smmu, u32 sid)
+{
+	/* Check the SIDs are in range of the SMMU and our stream table */
+	if (!arm_smmu_sid_in_range(smmu, sid))
+		return -ERANGE;
+
+	/* Ensure l2 strtab is initialised */
+	if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB)
+		return arm_smmu_init_l2_strtab(smmu, sid);
+
+	return 0;
+}
+
 static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 				  struct arm_smmu_master *master)
 {
@@ -2541,20 +2554,9 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 		new_stream->id = sid;
 		new_stream->master = master;
 
-		/*
-		 * Check the SIDs are in range of the SMMU and our stream table
-		 */
-		if (!arm_smmu_sid_in_range(smmu, sid)) {
-			ret = -ERANGE;
+		ret = arm_smmu_init_sid_strtab(smmu, sid);
+		if (ret)
 			break;
-		}
-
-		/* Ensure l2 strtab is initialised */
-		if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) {
-			ret = arm_smmu_init_l2_strtab(smmu, sid);
-			if (ret)
-				break;
-		}
 
 		/* Insert into SID tree */
 		new_node = &(smmu->streams.rb_node);
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 5/9] iommu/arm-smmu-v3: Introduce strtab init helper
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: robin.murphy, jon, linuxarm, steven.price, guohanjun, yangyicong,
	Sami.Mujawar, will, wanghuiqiang

Introduce a helper to check the sid range and to init the l2 strtab
entries(bypass). This will be useful when we have to initialize the
l2 strtab with bypass for RMR SIDs.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 +++++++++++----------
 1 file changed, 15 insertions(+), 13 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 235f9bdaeaf2..31940e53c675 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2518,6 +2518,19 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid)
 	return sid < limit;
 }
 
+static int arm_smmu_init_sid_strtab(struct arm_smmu_device *smmu, u32 sid)
+{
+	/* Check the SIDs are in range of the SMMU and our stream table */
+	if (!arm_smmu_sid_in_range(smmu, sid))
+		return -ERANGE;
+
+	/* Ensure l2 strtab is initialised */
+	if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB)
+		return arm_smmu_init_l2_strtab(smmu, sid);
+
+	return 0;
+}
+
 static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 				  struct arm_smmu_master *master)
 {
@@ -2541,20 +2554,9 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 		new_stream->id = sid;
 		new_stream->master = master;
 
-		/*
-		 * Check the SIDs are in range of the SMMU and our stream table
-		 */
-		if (!arm_smmu_sid_in_range(smmu, sid)) {
-			ret = -ERANGE;
+		ret = arm_smmu_init_sid_strtab(smmu, sid);
+		if (ret)
 			break;
-		}
-
-		/* Ensure l2 strtab is initialised */
-		if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) {
-			ret = arm_smmu_init_l2_strtab(smmu, sid);
-			if (ret)
-				break;
-		}
 
 		/* Insert into SID tree */
 		new_node = &(smmu->streams.rb_node);
-- 
2.17.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 5/9] iommu/arm-smmu-v3: Introduce strtab init helper
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Introduce a helper to check the sid range and to init the l2 strtab
entries(bypass). This will be useful when we have to initialize the
l2 strtab with bypass for RMR SIDs.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 +++++++++++----------
 1 file changed, 15 insertions(+), 13 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 235f9bdaeaf2..31940e53c675 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2518,6 +2518,19 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid)
 	return sid < limit;
 }
 
+static int arm_smmu_init_sid_strtab(struct arm_smmu_device *smmu, u32 sid)
+{
+	/* Check the SIDs are in range of the SMMU and our stream table */
+	if (!arm_smmu_sid_in_range(smmu, sid))
+		return -ERANGE;
+
+	/* Ensure l2 strtab is initialised */
+	if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB)
+		return arm_smmu_init_l2_strtab(smmu, sid);
+
+	return 0;
+}
+
 static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 				  struct arm_smmu_master *master)
 {
@@ -2541,20 +2554,9 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
 		new_stream->id = sid;
 		new_stream->master = master;
 
-		/*
-		 * Check the SIDs are in range of the SMMU and our stream table
-		 */
-		if (!arm_smmu_sid_in_range(smmu, sid)) {
-			ret = -ERANGE;
+		ret = arm_smmu_init_sid_strtab(smmu, sid);
+		if (ret)
 			break;
-		}
-
-		/* Ensure l2 strtab is initialised */
-		if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) {
-			ret = arm_smmu_init_l2_strtab(smmu, sid);
-			if (ret)
-				break;
-		}
 
 		/* Insert into SID tree */
 		new_node = &(smmu->streams.rb_node);
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 6/9] iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force bypass
  2021-08-05  8:07 ` Shameer Kolothum
  (?)
@ 2021-08-05  8:07   ` Shameer Kolothum
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

By default, disable_bypass flag is set and any dev without
an iommu domain installs STE with CFG_ABORT during
arm_smmu_init_bypass_stes(). Introduce a "force" flag and
move the STE update logic to arm_smmu_init_bypass_stes()
so that we can force it to install CFG_BYPASS STE for specific
SIDs.

This will be useful in follow-up patch to install bypass
for IORT RMR SIDs.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 31940e53c675..85f6f1925a36 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1357,12 +1357,21 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
 		arm_smmu_cmdq_issue_cmd(smmu, &prefetch_cmd);
 }
 
-static void arm_smmu_init_bypass_stes(__le64 *strtab, unsigned int nent)
+static void arm_smmu_init_bypass_stes(__le64 *strtab, unsigned int nent, bool force)
 {
 	unsigned int i;
+	u64 val = STRTAB_STE_0_V;
+
+	if (disable_bypass && !force)
+		val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT);
+	else
+		val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS);
 
 	for (i = 0; i < nent; ++i) {
-		arm_smmu_write_strtab_ent(NULL, -1, strtab);
+		strtab[0] = cpu_to_le64(val);
+		strtab[1] = cpu_to_le64(FIELD_PREP(STRTAB_STE_1_SHCFG,
+						   STRTAB_STE_1_SHCFG_INCOMING));
+		strtab[2] = 0;
 		strtab += STRTAB_STE_DWORDS;
 	}
 }
@@ -1390,7 +1399,7 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)
 		return -ENOMEM;
 	}
 
-	arm_smmu_init_bypass_stes(desc->l2ptr, 1 << STRTAB_SPLIT);
+	arm_smmu_init_bypass_stes(desc->l2ptr, 1 << STRTAB_SPLIT, false);
 	arm_smmu_write_strtab_l1_desc(strtab, desc);
 	return 0;
 }
@@ -3042,7 +3051,7 @@ static int arm_smmu_init_strtab_linear(struct arm_smmu_device *smmu)
 	reg |= FIELD_PREP(STRTAB_BASE_CFG_LOG2SIZE, smmu->sid_bits);
 	cfg->strtab_base_cfg = reg;
 
-	arm_smmu_init_bypass_stes(strtab, cfg->num_l1_ents);
+	arm_smmu_init_bypass_stes(strtab, cfg->num_l1_ents, false);
 	return 0;
 }
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 6/9] iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force bypass
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: robin.murphy, jon, linuxarm, steven.price, guohanjun, yangyicong,
	Sami.Mujawar, will, wanghuiqiang

By default, disable_bypass flag is set and any dev without
an iommu domain installs STE with CFG_ABORT during
arm_smmu_init_bypass_stes(). Introduce a "force" flag and
move the STE update logic to arm_smmu_init_bypass_stes()
so that we can force it to install CFG_BYPASS STE for specific
SIDs.

This will be useful in follow-up patch to install bypass
for IORT RMR SIDs.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 31940e53c675..85f6f1925a36 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1357,12 +1357,21 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
 		arm_smmu_cmdq_issue_cmd(smmu, &prefetch_cmd);
 }
 
-static void arm_smmu_init_bypass_stes(__le64 *strtab, unsigned int nent)
+static void arm_smmu_init_bypass_stes(__le64 *strtab, unsigned int nent, bool force)
 {
 	unsigned int i;
+	u64 val = STRTAB_STE_0_V;
+
+	if (disable_bypass && !force)
+		val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT);
+	else
+		val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS);
 
 	for (i = 0; i < nent; ++i) {
-		arm_smmu_write_strtab_ent(NULL, -1, strtab);
+		strtab[0] = cpu_to_le64(val);
+		strtab[1] = cpu_to_le64(FIELD_PREP(STRTAB_STE_1_SHCFG,
+						   STRTAB_STE_1_SHCFG_INCOMING));
+		strtab[2] = 0;
 		strtab += STRTAB_STE_DWORDS;
 	}
 }
@@ -1390,7 +1399,7 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)
 		return -ENOMEM;
 	}
 
-	arm_smmu_init_bypass_stes(desc->l2ptr, 1 << STRTAB_SPLIT);
+	arm_smmu_init_bypass_stes(desc->l2ptr, 1 << STRTAB_SPLIT, false);
 	arm_smmu_write_strtab_l1_desc(strtab, desc);
 	return 0;
 }
@@ -3042,7 +3051,7 @@ static int arm_smmu_init_strtab_linear(struct arm_smmu_device *smmu)
 	reg |= FIELD_PREP(STRTAB_BASE_CFG_LOG2SIZE, smmu->sid_bits);
 	cfg->strtab_base_cfg = reg;
 
-	arm_smmu_init_bypass_stes(strtab, cfg->num_l1_ents);
+	arm_smmu_init_bypass_stes(strtab, cfg->num_l1_ents, false);
 	return 0;
 }
 
-- 
2.17.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 6/9] iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force bypass
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

By default, disable_bypass flag is set and any dev without
an iommu domain installs STE with CFG_ABORT during
arm_smmu_init_bypass_stes(). Introduce a "force" flag and
move the STE update logic to arm_smmu_init_bypass_stes()
so that we can force it to install CFG_BYPASS STE for specific
SIDs.

This will be useful in follow-up patch to install bypass
for IORT RMR SIDs.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 31940e53c675..85f6f1925a36 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1357,12 +1357,21 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
 		arm_smmu_cmdq_issue_cmd(smmu, &prefetch_cmd);
 }
 
-static void arm_smmu_init_bypass_stes(__le64 *strtab, unsigned int nent)
+static void arm_smmu_init_bypass_stes(__le64 *strtab, unsigned int nent, bool force)
 {
 	unsigned int i;
+	u64 val = STRTAB_STE_0_V;
+
+	if (disable_bypass && !force)
+		val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT);
+	else
+		val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS);
 
 	for (i = 0; i < nent; ++i) {
-		arm_smmu_write_strtab_ent(NULL, -1, strtab);
+		strtab[0] = cpu_to_le64(val);
+		strtab[1] = cpu_to_le64(FIELD_PREP(STRTAB_STE_1_SHCFG,
+						   STRTAB_STE_1_SHCFG_INCOMING));
+		strtab[2] = 0;
 		strtab += STRTAB_STE_DWORDS;
 	}
 }
@@ -1390,7 +1399,7 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)
 		return -ENOMEM;
 	}
 
-	arm_smmu_init_bypass_stes(desc->l2ptr, 1 << STRTAB_SPLIT);
+	arm_smmu_init_bypass_stes(desc->l2ptr, 1 << STRTAB_SPLIT, false);
 	arm_smmu_write_strtab_l1_desc(strtab, desc);
 	return 0;
 }
@@ -3042,7 +3051,7 @@ static int arm_smmu_init_strtab_linear(struct arm_smmu_device *smmu)
 	reg |= FIELD_PREP(STRTAB_BASE_CFG_LOG2SIZE, smmu->sid_bits);
 	cfg->strtab_base_cfg = reg;
 
-	arm_smmu_init_bypass_stes(strtab, cfg->num_l1_ents);
+	arm_smmu_init_bypass_stes(strtab, cfg->num_l1_ents, false);
 	return 0;
 }
 
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 7/9] iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
  2021-08-05  8:07 ` Shameer Kolothum
  (?)
@ 2021-08-05  8:07   ` Shameer Kolothum
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Check if there is any RMR info associated with the devices behind
the SMMUv3 and if any, install bypass STEs for them. This is to
keep any ongoing traffic associated with these devices alive
when we enable/reset SMMUv3 during probe().

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 31 +++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 85f6f1925a36..1165605d6f7a 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -3760,6 +3760,34 @@ static void __iomem *arm_smmu_ioremap(struct device *dev, resource_size_t start,
 	return devm_ioremap_resource(dev, &res);
 }
 
+static void arm_smmu_rmr_install_bypass_ste(struct arm_smmu_device *smmu)
+{
+	struct list_head rmr_list;
+	struct iommu_resv_region *e;
+	int ret;
+
+	INIT_LIST_HEAD(&rmr_list);
+	if (iommu_dma_get_rmrs(dev_fwnode(smmu->dev), &rmr_list))
+		return;
+
+	list_for_each_entry(e, &rmr_list, list) {
+		__le64 *step;
+		u32 sid = e->fw_data.rmr.sid;
+
+		ret = arm_smmu_init_sid_strtab(smmu, sid);
+		if (ret) {
+			dev_err(smmu->dev, "RMR SID(0x%x) bypass failed\n",
+				sid);
+			continue;
+		}
+
+		step = arm_smmu_get_step_for_sid(smmu, sid);
+		arm_smmu_init_bypass_stes(step, 1, true);
+	}
+
+	iommu_dma_put_rmrs(dev_fwnode(smmu->dev), &rmr_list);
+}
+
 static int arm_smmu_device_probe(struct platform_device *pdev)
 {
 	int irq, ret;
@@ -3841,6 +3869,9 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
 	/* Record our private device structure */
 	platform_set_drvdata(pdev, smmu);
 
+	/* Check for RMRs and install bypass STEs if any */
+	arm_smmu_rmr_install_bypass_ste(smmu);
+
 	/* Reset the device */
 	ret = arm_smmu_device_reset(smmu, bypass);
 	if (ret)
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 7/9] iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: robin.murphy, jon, linuxarm, steven.price, guohanjun, yangyicong,
	Sami.Mujawar, will, wanghuiqiang

Check if there is any RMR info associated with the devices behind
the SMMUv3 and if any, install bypass STEs for them. This is to
keep any ongoing traffic associated with these devices alive
when we enable/reset SMMUv3 during probe().

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 31 +++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 85f6f1925a36..1165605d6f7a 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -3760,6 +3760,34 @@ static void __iomem *arm_smmu_ioremap(struct device *dev, resource_size_t start,
 	return devm_ioremap_resource(dev, &res);
 }
 
+static void arm_smmu_rmr_install_bypass_ste(struct arm_smmu_device *smmu)
+{
+	struct list_head rmr_list;
+	struct iommu_resv_region *e;
+	int ret;
+
+	INIT_LIST_HEAD(&rmr_list);
+	if (iommu_dma_get_rmrs(dev_fwnode(smmu->dev), &rmr_list))
+		return;
+
+	list_for_each_entry(e, &rmr_list, list) {
+		__le64 *step;
+		u32 sid = e->fw_data.rmr.sid;
+
+		ret = arm_smmu_init_sid_strtab(smmu, sid);
+		if (ret) {
+			dev_err(smmu->dev, "RMR SID(0x%x) bypass failed\n",
+				sid);
+			continue;
+		}
+
+		step = arm_smmu_get_step_for_sid(smmu, sid);
+		arm_smmu_init_bypass_stes(step, 1, true);
+	}
+
+	iommu_dma_put_rmrs(dev_fwnode(smmu->dev), &rmr_list);
+}
+
 static int arm_smmu_device_probe(struct platform_device *pdev)
 {
 	int irq, ret;
@@ -3841,6 +3869,9 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
 	/* Record our private device structure */
 	platform_set_drvdata(pdev, smmu);
 
+	/* Check for RMRs and install bypass STEs if any */
+	arm_smmu_rmr_install_bypass_ste(smmu);
+
 	/* Reset the device */
 	ret = arm_smmu_device_reset(smmu, bypass);
 	if (ret)
-- 
2.17.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 7/9] iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Check if there is any RMR info associated with the devices behind
the SMMUv3 and if any, install bypass STEs for them. This is to
keep any ongoing traffic associated with these devices alive
when we enable/reset SMMUv3 during probe().

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 31 +++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 85f6f1925a36..1165605d6f7a 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -3760,6 +3760,34 @@ static void __iomem *arm_smmu_ioremap(struct device *dev, resource_size_t start,
 	return devm_ioremap_resource(dev, &res);
 }
 
+static void arm_smmu_rmr_install_bypass_ste(struct arm_smmu_device *smmu)
+{
+	struct list_head rmr_list;
+	struct iommu_resv_region *e;
+	int ret;
+
+	INIT_LIST_HEAD(&rmr_list);
+	if (iommu_dma_get_rmrs(dev_fwnode(smmu->dev), &rmr_list))
+		return;
+
+	list_for_each_entry(e, &rmr_list, list) {
+		__le64 *step;
+		u32 sid = e->fw_data.rmr.sid;
+
+		ret = arm_smmu_init_sid_strtab(smmu, sid);
+		if (ret) {
+			dev_err(smmu->dev, "RMR SID(0x%x) bypass failed\n",
+				sid);
+			continue;
+		}
+
+		step = arm_smmu_get_step_for_sid(smmu, sid);
+		arm_smmu_init_bypass_stes(step, 1, true);
+	}
+
+	iommu_dma_put_rmrs(dev_fwnode(smmu->dev), &rmr_list);
+}
+
 static int arm_smmu_device_probe(struct platform_device *pdev)
 {
 	int irq, ret;
@@ -3841,6 +3869,9 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
 	/* Record our private device structure */
 	platform_set_drvdata(pdev, smmu);
 
+	/* Check for RMRs and install bypass STEs if any */
+	arm_smmu_rmr_install_bypass_ste(smmu);
+
 	/* Reset the device */
 	ret = arm_smmu_device_reset(smmu, bypass);
 	if (ret)
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 8/9] iommu/arm-smmu: Get associated RMR info and install bypass SMR
  2021-08-05  8:07 ` Shameer Kolothum
  (?)
@ 2021-08-05  8:07   ` Shameer Kolothum
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

From: Jon Nettleton <jon@solid-run.com>

Check if there is any RMR info associated with the devices behind
the SMMU and if any, install bypass SMRs for them. This is to
keep any ongoing traffic associated with these devices alive
when we enable/reset SMMU during probe().

Signed-off-by: Jon Nettleton <jon@solid-run.com>
Signed-off-by: Steven Price <steven.price@arm.com>
Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu/arm-smmu.c | 48 +++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c b/drivers/iommu/arm/arm-smmu/arm-smmu.c
index f22dbeb1e510..315feab9e85b 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu.c
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c
@@ -2063,6 +2063,50 @@ err_reset_platform_ops: __maybe_unused;
 	return err;
 }
 
+static void arm_smmu_rmr_install_bypass_smr(struct arm_smmu_device *smmu)
+{
+	struct list_head rmr_list;
+	struct iommu_resv_region *e;
+	int i, cnt = 0;
+	u32 reg;
+
+	INIT_LIST_HEAD(&rmr_list);
+	if (iommu_dma_get_rmrs(dev_fwnode(smmu->dev), &rmr_list))
+		return;
+
+	/*
+	 * Rather than trying to look at existing mappings that
+	 * are setup by the firmware and then invalidate the ones
+	 * that do no have matching RMR entries, just disable the
+	 * SMMU until it gets enabled again in the reset routine.
+	 */
+	reg = arm_smmu_gr0_read(smmu, ARM_SMMU_GR0_sCR0);
+	reg |= ARM_SMMU_sCR0_CLIENTPD;
+	arm_smmu_gr0_write(smmu, ARM_SMMU_GR0_sCR0, reg);
+
+	list_for_each_entry(e, &rmr_list, list) {
+		u32 sid = e->fw_data.rmr.sid;
+
+		i = arm_smmu_find_sme(smmu, sid, ~0);
+		if (i < 0)
+			continue;
+		if (smmu->s2crs[i].count == 0) {
+			smmu->smrs[i].id = sid;
+			smmu->smrs[i].mask = 0;
+			smmu->smrs[i].valid = true;
+		}
+		smmu->s2crs[i].count++;
+		smmu->s2crs[i].type = S2CR_TYPE_BYPASS;
+		smmu->s2crs[i].privcfg = S2CR_PRIVCFG_DEFAULT;
+
+		cnt++;
+	}
+
+	dev_notice(smmu->dev, "\tpreserved %d boot mapping%s\n", cnt,
+		   cnt == 1 ? "" : "s");
+	iommu_dma_put_rmrs(dev_fwnode(smmu->dev), &rmr_list);
+}
+
 static int arm_smmu_device_probe(struct platform_device *pdev)
 {
 	struct resource *res;
@@ -2189,6 +2233,10 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
 	}
 
 	platform_set_drvdata(pdev, smmu);
+
+	/* Check for RMRs and install bypass SMRs if any */
+	arm_smmu_rmr_install_bypass_smr(smmu);
+
 	arm_smmu_device_reset(smmu);
 	arm_smmu_test_smr_masks(smmu);
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 8/9] iommu/arm-smmu: Get associated RMR info and install bypass SMR
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: robin.murphy, jon, linuxarm, steven.price, guohanjun, yangyicong,
	Sami.Mujawar, will, wanghuiqiang

From: Jon Nettleton <jon@solid-run.com>

Check if there is any RMR info associated with the devices behind
the SMMU and if any, install bypass SMRs for them. This is to
keep any ongoing traffic associated with these devices alive
when we enable/reset SMMU during probe().

Signed-off-by: Jon Nettleton <jon@solid-run.com>
Signed-off-by: Steven Price <steven.price@arm.com>
Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu/arm-smmu.c | 48 +++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c b/drivers/iommu/arm/arm-smmu/arm-smmu.c
index f22dbeb1e510..315feab9e85b 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu.c
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c
@@ -2063,6 +2063,50 @@ err_reset_platform_ops: __maybe_unused;
 	return err;
 }
 
+static void arm_smmu_rmr_install_bypass_smr(struct arm_smmu_device *smmu)
+{
+	struct list_head rmr_list;
+	struct iommu_resv_region *e;
+	int i, cnt = 0;
+	u32 reg;
+
+	INIT_LIST_HEAD(&rmr_list);
+	if (iommu_dma_get_rmrs(dev_fwnode(smmu->dev), &rmr_list))
+		return;
+
+	/*
+	 * Rather than trying to look at existing mappings that
+	 * are setup by the firmware and then invalidate the ones
+	 * that do no have matching RMR entries, just disable the
+	 * SMMU until it gets enabled again in the reset routine.
+	 */
+	reg = arm_smmu_gr0_read(smmu, ARM_SMMU_GR0_sCR0);
+	reg |= ARM_SMMU_sCR0_CLIENTPD;
+	arm_smmu_gr0_write(smmu, ARM_SMMU_GR0_sCR0, reg);
+
+	list_for_each_entry(e, &rmr_list, list) {
+		u32 sid = e->fw_data.rmr.sid;
+
+		i = arm_smmu_find_sme(smmu, sid, ~0);
+		if (i < 0)
+			continue;
+		if (smmu->s2crs[i].count == 0) {
+			smmu->smrs[i].id = sid;
+			smmu->smrs[i].mask = 0;
+			smmu->smrs[i].valid = true;
+		}
+		smmu->s2crs[i].count++;
+		smmu->s2crs[i].type = S2CR_TYPE_BYPASS;
+		smmu->s2crs[i].privcfg = S2CR_PRIVCFG_DEFAULT;
+
+		cnt++;
+	}
+
+	dev_notice(smmu->dev, "\tpreserved %d boot mapping%s\n", cnt,
+		   cnt == 1 ? "" : "s");
+	iommu_dma_put_rmrs(dev_fwnode(smmu->dev), &rmr_list);
+}
+
 static int arm_smmu_device_probe(struct platform_device *pdev)
 {
 	struct resource *res;
@@ -2189,6 +2233,10 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
 	}
 
 	platform_set_drvdata(pdev, smmu);
+
+	/* Check for RMRs and install bypass SMRs if any */
+	arm_smmu_rmr_install_bypass_smr(smmu);
+
 	arm_smmu_device_reset(smmu);
 	arm_smmu_test_smr_masks(smmu);
 
-- 
2.17.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 8/9] iommu/arm-smmu: Get associated RMR info and install bypass SMR
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

From: Jon Nettleton <jon@solid-run.com>

Check if there is any RMR info associated with the devices behind
the SMMU and if any, install bypass SMRs for them. This is to
keep any ongoing traffic associated with these devices alive
when we enable/reset SMMU during probe().

Signed-off-by: Jon Nettleton <jon@solid-run.com>
Signed-off-by: Steven Price <steven.price@arm.com>
Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu/arm-smmu.c | 48 +++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c b/drivers/iommu/arm/arm-smmu/arm-smmu.c
index f22dbeb1e510..315feab9e85b 100644
--- a/drivers/iommu/arm/arm-smmu/arm-smmu.c
+++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c
@@ -2063,6 +2063,50 @@ err_reset_platform_ops: __maybe_unused;
 	return err;
 }
 
+static void arm_smmu_rmr_install_bypass_smr(struct arm_smmu_device *smmu)
+{
+	struct list_head rmr_list;
+	struct iommu_resv_region *e;
+	int i, cnt = 0;
+	u32 reg;
+
+	INIT_LIST_HEAD(&rmr_list);
+	if (iommu_dma_get_rmrs(dev_fwnode(smmu->dev), &rmr_list))
+		return;
+
+	/*
+	 * Rather than trying to look at existing mappings that
+	 * are setup by the firmware and then invalidate the ones
+	 * that do no have matching RMR entries, just disable the
+	 * SMMU until it gets enabled again in the reset routine.
+	 */
+	reg = arm_smmu_gr0_read(smmu, ARM_SMMU_GR0_sCR0);
+	reg |= ARM_SMMU_sCR0_CLIENTPD;
+	arm_smmu_gr0_write(smmu, ARM_SMMU_GR0_sCR0, reg);
+
+	list_for_each_entry(e, &rmr_list, list) {
+		u32 sid = e->fw_data.rmr.sid;
+
+		i = arm_smmu_find_sme(smmu, sid, ~0);
+		if (i < 0)
+			continue;
+		if (smmu->s2crs[i].count == 0) {
+			smmu->smrs[i].id = sid;
+			smmu->smrs[i].mask = 0;
+			smmu->smrs[i].valid = true;
+		}
+		smmu->s2crs[i].count++;
+		smmu->s2crs[i].type = S2CR_TYPE_BYPASS;
+		smmu->s2crs[i].privcfg = S2CR_PRIVCFG_DEFAULT;
+
+		cnt++;
+	}
+
+	dev_notice(smmu->dev, "\tpreserved %d boot mapping%s\n", cnt,
+		   cnt == 1 ? "" : "s");
+	iommu_dma_put_rmrs(dev_fwnode(smmu->dev), &rmr_list);
+}
+
 static int arm_smmu_device_probe(struct platform_device *pdev)
 {
 	struct resource *res;
@@ -2189,6 +2233,10 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
 	}
 
 	platform_set_drvdata(pdev, smmu);
+
+	/* Check for RMRs and install bypass SMRs if any */
+	arm_smmu_rmr_install_bypass_smr(smmu);
+
 	arm_smmu_device_reset(smmu);
 	arm_smmu_test_smr_masks(smmu);
 
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
  2021-08-05  8:07 ` Shameer Kolothum
  (?)
@ 2021-08-05  8:07   ` Shameer Kolothum
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Get ACPI IORT RMR regions associated with a dev reserved
so that there is a unity mapping for them in SMMU.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/dma-iommu.c | 56 +++++++++++++++++++++++++++++++++++----
 1 file changed, 51 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 1b6e27475279..c1ae0c3d4b33 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -207,22 +207,68 @@ void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
 }
 EXPORT_SYMBOL(iommu_dma_put_rmrs);
 
+static bool iommu_dma_dev_has_rmr(struct iommu_fwspec *fwspec,
+				  struct iommu_resv_region *e)
+{
+	int i;
+
+	for (i = 0; i < fwspec->num_ids; i++) {
+		if (e->fw_data.rmr.sid == fwspec->ids[i])
+			return true;
+	}
+
+	return false;
+}
+
+static void iommu_dma_get_rmr_resv_regions(struct device *dev,
+					   struct list_head *list)
+{
+	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+	struct list_head rmr_list;
+	struct iommu_resv_region *rmr, *tmp;
+
+	INIT_LIST_HEAD(&rmr_list);
+	if (iommu_dma_get_rmrs(fwspec->iommu_fwnode, &rmr_list))
+		return;
+
+	if (dev_is_pci(dev)) {
+		struct pci_dev *pdev = to_pci_dev(dev);
+		struct pci_host_bridge *host = pci_find_host_bridge(pdev->bus);
+
+		if (!host->preserve_config)
+			return;
+	}
+
+	list_for_each_entry_safe(rmr, tmp, &rmr_list, list) {
+		if (!iommu_dma_dev_has_rmr(fwspec, rmr))
+			continue;
+
+		/* Remove from iommu RMR list and add to dev resv_regions */
+		list_del_init(&rmr->list);
+		list_add_tail(&rmr->list, list);
+	}
+
+	iommu_dma_put_rmrs(fwspec->iommu_fwnode, &rmr_list);
+}
+
 /**
  * iommu_dma_get_resv_regions - Reserved region driver helper
  * @dev: Device from iommu_get_resv_regions()
  * @list: Reserved region list from iommu_get_resv_regions()
  *
  * IOMMU drivers can use this to implement their .get_resv_regions callback
- * for general non-IOMMU-specific reservations. Currently, this covers GICv3
- * ITS region reservation on ACPI based ARM platforms that may require HW MSI
- * reservation.
+ * for general non-IOMMU-specific reservations. Currently this covers,
+ *  -GICv3 ITS region reservation on ACPI based ARM platforms that may
+ *   require HW MSI reservation.
+ *  -Any ACPI IORT RMR memory range reservations (IORT spec rev E.b)
  */
 void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
 {
 
-	if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode))
+	if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode)) {
 		iort_iommu_msi_get_resv_regions(dev, list);
-
+		iommu_dma_get_rmr_resv_regions(dev, list);
+	}
 }
 EXPORT_SYMBOL(iommu_dma_get_resv_regions);
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: robin.murphy, jon, linuxarm, steven.price, guohanjun, yangyicong,
	Sami.Mujawar, will, wanghuiqiang

Get ACPI IORT RMR regions associated with a dev reserved
so that there is a unity mapping for them in SMMU.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/dma-iommu.c | 56 +++++++++++++++++++++++++++++++++++----
 1 file changed, 51 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 1b6e27475279..c1ae0c3d4b33 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -207,22 +207,68 @@ void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
 }
 EXPORT_SYMBOL(iommu_dma_put_rmrs);
 
+static bool iommu_dma_dev_has_rmr(struct iommu_fwspec *fwspec,
+				  struct iommu_resv_region *e)
+{
+	int i;
+
+	for (i = 0; i < fwspec->num_ids; i++) {
+		if (e->fw_data.rmr.sid == fwspec->ids[i])
+			return true;
+	}
+
+	return false;
+}
+
+static void iommu_dma_get_rmr_resv_regions(struct device *dev,
+					   struct list_head *list)
+{
+	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+	struct list_head rmr_list;
+	struct iommu_resv_region *rmr, *tmp;
+
+	INIT_LIST_HEAD(&rmr_list);
+	if (iommu_dma_get_rmrs(fwspec->iommu_fwnode, &rmr_list))
+		return;
+
+	if (dev_is_pci(dev)) {
+		struct pci_dev *pdev = to_pci_dev(dev);
+		struct pci_host_bridge *host = pci_find_host_bridge(pdev->bus);
+
+		if (!host->preserve_config)
+			return;
+	}
+
+	list_for_each_entry_safe(rmr, tmp, &rmr_list, list) {
+		if (!iommu_dma_dev_has_rmr(fwspec, rmr))
+			continue;
+
+		/* Remove from iommu RMR list and add to dev resv_regions */
+		list_del_init(&rmr->list);
+		list_add_tail(&rmr->list, list);
+	}
+
+	iommu_dma_put_rmrs(fwspec->iommu_fwnode, &rmr_list);
+}
+
 /**
  * iommu_dma_get_resv_regions - Reserved region driver helper
  * @dev: Device from iommu_get_resv_regions()
  * @list: Reserved region list from iommu_get_resv_regions()
  *
  * IOMMU drivers can use this to implement their .get_resv_regions callback
- * for general non-IOMMU-specific reservations. Currently, this covers GICv3
- * ITS region reservation on ACPI based ARM platforms that may require HW MSI
- * reservation.
+ * for general non-IOMMU-specific reservations. Currently this covers,
+ *  -GICv3 ITS region reservation on ACPI based ARM platforms that may
+ *   require HW MSI reservation.
+ *  -Any ACPI IORT RMR memory range reservations (IORT spec rev E.b)
  */
 void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
 {
 
-	if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode))
+	if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode)) {
 		iort_iommu_msi_get_resv_regions(dev, list);
-
+		iommu_dma_get_rmr_resv_regions(dev, list);
+	}
 }
 EXPORT_SYMBOL(iommu_dma_get_resv_regions);
 
-- 
2.17.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
@ 2021-08-05  8:07   ` Shameer Kolothum
  0 siblings, 0 replies; 150+ messages in thread
From: Shameer Kolothum @ 2021-08-05  8:07 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	eric.auger, yangyicong

Get ACPI IORT RMR regions associated with a dev reserved
so that there is a unity mapping for them in SMMU.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/dma-iommu.c | 56 +++++++++++++++++++++++++++++++++++----
 1 file changed, 51 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 1b6e27475279..c1ae0c3d4b33 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -207,22 +207,68 @@ void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
 }
 EXPORT_SYMBOL(iommu_dma_put_rmrs);
 
+static bool iommu_dma_dev_has_rmr(struct iommu_fwspec *fwspec,
+				  struct iommu_resv_region *e)
+{
+	int i;
+
+	for (i = 0; i < fwspec->num_ids; i++) {
+		if (e->fw_data.rmr.sid == fwspec->ids[i])
+			return true;
+	}
+
+	return false;
+}
+
+static void iommu_dma_get_rmr_resv_regions(struct device *dev,
+					   struct list_head *list)
+{
+	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+	struct list_head rmr_list;
+	struct iommu_resv_region *rmr, *tmp;
+
+	INIT_LIST_HEAD(&rmr_list);
+	if (iommu_dma_get_rmrs(fwspec->iommu_fwnode, &rmr_list))
+		return;
+
+	if (dev_is_pci(dev)) {
+		struct pci_dev *pdev = to_pci_dev(dev);
+		struct pci_host_bridge *host = pci_find_host_bridge(pdev->bus);
+
+		if (!host->preserve_config)
+			return;
+	}
+
+	list_for_each_entry_safe(rmr, tmp, &rmr_list, list) {
+		if (!iommu_dma_dev_has_rmr(fwspec, rmr))
+			continue;
+
+		/* Remove from iommu RMR list and add to dev resv_regions */
+		list_del_init(&rmr->list);
+		list_add_tail(&rmr->list, list);
+	}
+
+	iommu_dma_put_rmrs(fwspec->iommu_fwnode, &rmr_list);
+}
+
 /**
  * iommu_dma_get_resv_regions - Reserved region driver helper
  * @dev: Device from iommu_get_resv_regions()
  * @list: Reserved region list from iommu_get_resv_regions()
  *
  * IOMMU drivers can use this to implement their .get_resv_regions callback
- * for general non-IOMMU-specific reservations. Currently, this covers GICv3
- * ITS region reservation on ACPI based ARM platforms that may require HW MSI
- * reservation.
+ * for general non-IOMMU-specific reservations. Currently this covers,
+ *  -GICv3 ITS region reservation on ACPI based ARM platforms that may
+ *   require HW MSI reservation.
+ *  -Any ACPI IORT RMR memory range reservations (IORT spec rev E.b)
  */
 void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
 {
 
-	if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode))
+	if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode)) {
 		iort_iommu_msi_get_resv_regions(dev, list);
-
+		iommu_dma_get_rmr_resv_regions(dev, list);
+	}
 }
 EXPORT_SYMBOL(iommu_dma_get_resv_regions);
 
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
  2021-08-05  8:07 ` Shameer Kolothum
  (?)
@ 2021-08-05 13:22   ` Ard Biesheuvel
  -1 siblings, 0 replies; 150+ messages in thread
From: Ard Biesheuvel @ 2021-08-05 13:22 UTC (permalink / raw)
  To: Shameer Kolothum
  Cc: Linux ARM, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Lorenzo Pieralisi, Joerg Roedel, Robin Murphy, Will Deacon,
	wanghuiqiang, Hanjun Guo, Steven Price, Sami Mujawar,
	Jon Nettleton, Eric Auger, yangyicong

On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum
<shameerali.kolothum.thodi@huawei.com> wrote:
>
> Hi,
>
> The series adds support to IORT RMR nodes specified in IORT
> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> memory ranges that are used by endpoints and require a unity
> mapping in SMMU.
>
> We have faced issues with 3408iMR RAID controller cards which
> fail to boot when SMMU is enabled. This is because these
> controllers make use of host memory for various caching related
> purposes and when SMMU is enabled the iMR firmware fails to
> access these memory regions as there is no mapping for them.
> IORT RMR provides a way for UEFI to describe and report these
> memory regions so that the kernel can make a unity mapping for
> these in SMMU.
>

Does this mean we are ignoring the RMR memory ranges, and exposing the
entire physical address space to devices using the stream IDs in
question?


> Change History:
>
> v6 --> v7
>
> The only change from v6 is the fix pointed out by Steve to
> the SMMUv2 SMR bypass install in patch #8.
>
> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> yet as the series still needs more review[1].
>
> Feedback and tests on this series is very much appreciated.
>
> v5 --> v6
> - Addressed comments from Robin & Lorenzo.
>   : Moved iort_parse_rmr() to acpi_iort_init() from
>     iort_init_platform_devices().
>   : Removed use of struct iort_rmr_entry during the initial
>     parse. Using struct iommu_resv_region instead.
>   : Report RMR address alignment and overlap errors, but continue.
>   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
>   on Type of RMR region. Suggested by Jon N.
>
> Thanks,
> Shameer
> [0] https://developer.arm.com/documentation/den0049/latest/
> [1] https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.kolothum.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> ------
> v4 --> v5
>  -Added a fw_data union to struct iommu_resv_region and removed
>   struct iommu_rmr (Based on comments from Joerg/Robin).
>  -Added iommu_put_rmrs() to release mem.
>  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
>   yet because of the above changes.
>
> v3 -->v4
> -Included the SMMUv2 SMR bypass install changes suggested by
>  Steve(patch #7)
> -As per Robin's comments, RMR reserve implementation is now
>  more generic  (patch #8) and dropped v3 patches 8 and 10.
> -Rebase to 5.13-rc1
>
> RFC v2 --> v3
>  -Dropped RFC tag as the ACPICA header changes are now ready to be
>   part of 5.13[0]. But this series still has a dependency on that patch.
>  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
>   PCIe).
>  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
>   discussion here[1].
>  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> ------
>
> Jon Nettleton (1):
>   iommu/arm-smmu: Get associated RMR info and install bypass SMR
>
> Shameer Kolothum (8):
>   iommu: Introduce a union to struct iommu_resv_region
>   ACPI/IORT: Add support for RMR node parsing
>   iommu/dma: Introduce generic helper to retrieve RMR info
>   ACPI/IORT: Add a helper to retrieve RMR memory regions
>   iommu/arm-smmu-v3: Introduce strtab init helper
>   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
>     bypass
>   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
>   iommu/dma: Reserve any RMR regions associated with a dev
>
>  drivers/acpi/arm64/iort.c                   | 172 +++++++++++++++++++-
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
>  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
>  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
>  include/linux/acpi_iort.h                   |   7 +
>  include/linux/dma-iommu.h                   |  13 ++
>  include/linux/iommu.h                       |  11 ++
>  7 files changed, 393 insertions(+), 23 deletions(-)
>
> --
> 2.17.1
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-08-05 13:22   ` Ard Biesheuvel
  0 siblings, 0 replies; 150+ messages in thread
From: Ard Biesheuvel @ 2021-08-05 13:22 UTC (permalink / raw)
  To: Shameer Kolothum
  Cc: Will Deacon, Jon Nettleton, Linuxarm, Steven Price,
	ACPI Devel Maling List, Linux IOMMU, wanghuiqiang, Hanjun Guo,
	yangyicong, Sami Mujawar, Robin Murphy, Linux ARM

On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum
<shameerali.kolothum.thodi@huawei.com> wrote:
>
> Hi,
>
> The series adds support to IORT RMR nodes specified in IORT
> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> memory ranges that are used by endpoints and require a unity
> mapping in SMMU.
>
> We have faced issues with 3408iMR RAID controller cards which
> fail to boot when SMMU is enabled. This is because these
> controllers make use of host memory for various caching related
> purposes and when SMMU is enabled the iMR firmware fails to
> access these memory regions as there is no mapping for them.
> IORT RMR provides a way for UEFI to describe and report these
> memory regions so that the kernel can make a unity mapping for
> these in SMMU.
>

Does this mean we are ignoring the RMR memory ranges, and exposing the
entire physical address space to devices using the stream IDs in
question?


> Change History:
>
> v6 --> v7
>
> The only change from v6 is the fix pointed out by Steve to
> the SMMUv2 SMR bypass install in patch #8.
>
> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> yet as the series still needs more review[1].
>
> Feedback and tests on this series is very much appreciated.
>
> v5 --> v6
> - Addressed comments from Robin & Lorenzo.
>   : Moved iort_parse_rmr() to acpi_iort_init() from
>     iort_init_platform_devices().
>   : Removed use of struct iort_rmr_entry during the initial
>     parse. Using struct iommu_resv_region instead.
>   : Report RMR address alignment and overlap errors, but continue.
>   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
>   on Type of RMR region. Suggested by Jon N.
>
> Thanks,
> Shameer
> [0] https://developer.arm.com/documentation/den0049/latest/
> [1] https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.kolothum.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> ------
> v4 --> v5
>  -Added a fw_data union to struct iommu_resv_region and removed
>   struct iommu_rmr (Based on comments from Joerg/Robin).
>  -Added iommu_put_rmrs() to release mem.
>  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
>   yet because of the above changes.
>
> v3 -->v4
> -Included the SMMUv2 SMR bypass install changes suggested by
>  Steve(patch #7)
> -As per Robin's comments, RMR reserve implementation is now
>  more generic  (patch #8) and dropped v3 patches 8 and 10.
> -Rebase to 5.13-rc1
>
> RFC v2 --> v3
>  -Dropped RFC tag as the ACPICA header changes are now ready to be
>   part of 5.13[0]. But this series still has a dependency on that patch.
>  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
>   PCIe).
>  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
>   discussion here[1].
>  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> ------
>
> Jon Nettleton (1):
>   iommu/arm-smmu: Get associated RMR info and install bypass SMR
>
> Shameer Kolothum (8):
>   iommu: Introduce a union to struct iommu_resv_region
>   ACPI/IORT: Add support for RMR node parsing
>   iommu/dma: Introduce generic helper to retrieve RMR info
>   ACPI/IORT: Add a helper to retrieve RMR memory regions
>   iommu/arm-smmu-v3: Introduce strtab init helper
>   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
>     bypass
>   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
>   iommu/dma: Reserve any RMR regions associated with a dev
>
>  drivers/acpi/arm64/iort.c                   | 172 +++++++++++++++++++-
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
>  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
>  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
>  include/linux/acpi_iort.h                   |   7 +
>  include/linux/dma-iommu.h                   |  13 ++
>  include/linux/iommu.h                       |  11 ++
>  7 files changed, 393 insertions(+), 23 deletions(-)
>
> --
> 2.17.1
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-08-05 13:22   ` Ard Biesheuvel
  0 siblings, 0 replies; 150+ messages in thread
From: Ard Biesheuvel @ 2021-08-05 13:22 UTC (permalink / raw)
  To: Shameer Kolothum
  Cc: Linux ARM, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Lorenzo Pieralisi, Joerg Roedel, Robin Murphy, Will Deacon,
	wanghuiqiang, Hanjun Guo, Steven Price, Sami Mujawar,
	Jon Nettleton, Eric Auger, yangyicong

On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum
<shameerali.kolothum.thodi@huawei.com> wrote:
>
> Hi,
>
> The series adds support to IORT RMR nodes specified in IORT
> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> memory ranges that are used by endpoints and require a unity
> mapping in SMMU.
>
> We have faced issues with 3408iMR RAID controller cards which
> fail to boot when SMMU is enabled. This is because these
> controllers make use of host memory for various caching related
> purposes and when SMMU is enabled the iMR firmware fails to
> access these memory regions as there is no mapping for them.
> IORT RMR provides a way for UEFI to describe and report these
> memory regions so that the kernel can make a unity mapping for
> these in SMMU.
>

Does this mean we are ignoring the RMR memory ranges, and exposing the
entire physical address space to devices using the stream IDs in
question?


> Change History:
>
> v6 --> v7
>
> The only change from v6 is the fix pointed out by Steve to
> the SMMUv2 SMR bypass install in patch #8.
>
> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> yet as the series still needs more review[1].
>
> Feedback and tests on this series is very much appreciated.
>
> v5 --> v6
> - Addressed comments from Robin & Lorenzo.
>   : Moved iort_parse_rmr() to acpi_iort_init() from
>     iort_init_platform_devices().
>   : Removed use of struct iort_rmr_entry during the initial
>     parse. Using struct iommu_resv_region instead.
>   : Report RMR address alignment and overlap errors, but continue.
>   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
>   on Type of RMR region. Suggested by Jon N.
>
> Thanks,
> Shameer
> [0] https://developer.arm.com/documentation/den0049/latest/
> [1] https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.kolothum.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> ------
> v4 --> v5
>  -Added a fw_data union to struct iommu_resv_region and removed
>   struct iommu_rmr (Based on comments from Joerg/Robin).
>  -Added iommu_put_rmrs() to release mem.
>  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
>   yet because of the above changes.
>
> v3 -->v4
> -Included the SMMUv2 SMR bypass install changes suggested by
>  Steve(patch #7)
> -As per Robin's comments, RMR reserve implementation is now
>  more generic  (patch #8) and dropped v3 patches 8 and 10.
> -Rebase to 5.13-rc1
>
> RFC v2 --> v3
>  -Dropped RFC tag as the ACPICA header changes are now ready to be
>   part of 5.13[0]. But this series still has a dependency on that patch.
>  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
>   PCIe).
>  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
>   discussion here[1].
>  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> ------
>
> Jon Nettleton (1):
>   iommu/arm-smmu: Get associated RMR info and install bypass SMR
>
> Shameer Kolothum (8):
>   iommu: Introduce a union to struct iommu_resv_region
>   ACPI/IORT: Add support for RMR node parsing
>   iommu/dma: Introduce generic helper to retrieve RMR info
>   ACPI/IORT: Add a helper to retrieve RMR memory regions
>   iommu/arm-smmu-v3: Introduce strtab init helper
>   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
>     bypass
>   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
>   iommu/dma: Reserve any RMR regions associated with a dev
>
>  drivers/acpi/arm64/iort.c                   | 172 +++++++++++++++++++-
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
>  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
>  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
>  include/linux/acpi_iort.h                   |   7 +
>  include/linux/dma-iommu.h                   |  13 ++
>  include/linux/iommu.h                       |  11 ++
>  7 files changed, 393 insertions(+), 23 deletions(-)
>
> --
> 2.17.1
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
  2021-08-05 13:22   ` Ard Biesheuvel
  (?)
@ 2021-08-05 13:35     ` Shameerali Kolothum Thodi
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-08-05 13:35 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Linux ARM, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Lorenzo Pieralisi, Joerg Roedel, Robin Murphy, Will Deacon,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Jon Nettleton, Eric Auger,
	yangyicong



> -----Original Message-----
> From: Ard Biesheuvel [mailto:ardb@kernel.org]
> Sent: 05 August 2021 14:23
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel
> <joro@8bytes.org>; Robin Murphy <robin.murphy@arm.com>; Will Deacon
> <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Jon
> Nettleton <jon@solid-run.com>; Eric Auger <eric.auger@redhat.com>;
> yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> 
> On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> > Hi,
> >
> > The series adds support to IORT RMR nodes specified in IORT
> > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> > memory ranges that are used by endpoints and require a unity
> > mapping in SMMU.
> >
> > We have faced issues with 3408iMR RAID controller cards which
> > fail to boot when SMMU is enabled. This is because these
> > controllers make use of host memory for various caching related
> > purposes and when SMMU is enabled the iMR firmware fails to
> > access these memory regions as there is no mapping for them.
> > IORT RMR provides a way for UEFI to describe and report these
> > memory regions so that the kernel can make a unity mapping for
> > these in SMMU.
> >
> 
> Does this mean we are ignoring the RMR memory ranges, and exposing the
> entire physical address space to devices using the stream IDs in
> question?

Nope. RMR node is used to describe the memory ranges used by end points
behind SMMU. And this information is used to create 1 : 1 mappings for those
ranges in SMMU. Anything outside those ranges will result in translation
fault(if there are no other dynamic DMA mappings).

Thanks,
Shameer

> 
> > Change History:
> >
> > v6 --> v7
> >
> > The only change from v6 is the fix pointed out by Steve to
> > the SMMUv2 SMR bypass install in patch #8.
> >
> > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> > yet as the series still needs more review[1].
> >
> > Feedback and tests on this series is very much appreciated.
> >
> > v5 --> v6
> > - Addressed comments from Robin & Lorenzo.
> >   : Moved iort_parse_rmr() to acpi_iort_init() from
> >     iort_init_platform_devices().
> >   : Removed use of struct iort_rmr_entry during the initial
> >     parse. Using struct iommu_resv_region instead.
> >   : Report RMR address alignment and overlap errors, but continue.
> >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
> >   on Type of RMR region. Suggested by Jon N.
> >
> > Thanks,
> > Shameer
> > [0] https://developer.arm.com/documentation/den0049/latest/
> > [1]
> https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth
> um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> > ------
> > v4 --> v5
> >  -Added a fw_data union to struct iommu_resv_region and removed
> >   struct iommu_rmr (Based on comments from Joerg/Robin).
> >  -Added iommu_put_rmrs() to release mem.
> >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
> >   yet because of the above changes.
> >
> > v3 -->v4
> > -Included the SMMUv2 SMR bypass install changes suggested by
> >  Steve(patch #7)
> > -As per Robin's comments, RMR reserve implementation is now
> >  more generic  (patch #8) and dropped v3 patches 8 and 10.
> > -Rebase to 5.13-rc1
> >
> > RFC v2 --> v3
> >  -Dropped RFC tag as the ACPICA header changes are now ready to be
> >   part of 5.13[0]. But this series still has a dependency on that patch.
> >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
> >   PCIe).
> >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
> >   discussion here[1].
> >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> > ------
> >
> > Jon Nettleton (1):
> >   iommu/arm-smmu: Get associated RMR info and install bypass SMR
> >
> > Shameer Kolothum (8):
> >   iommu: Introduce a union to struct iommu_resv_region
> >   ACPI/IORT: Add support for RMR node parsing
> >   iommu/dma: Introduce generic helper to retrieve RMR info
> >   ACPI/IORT: Add a helper to retrieve RMR memory regions
> >   iommu/arm-smmu-v3: Introduce strtab init helper
> >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
> >     bypass
> >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> >   iommu/dma: Reserve any RMR regions associated with a dev
> >
> >  drivers/acpi/arm64/iort.c                   | 172
> +++++++++++++++++++-
> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
> >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
> >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
> >  include/linux/acpi_iort.h                   |   7 +
> >  include/linux/dma-iommu.h                   |  13 ++
> >  include/linux/iommu.h                       |  11 ++
> >  7 files changed, 393 insertions(+), 23 deletions(-)
> >
> > --
> > 2.17.1
> >
> >
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-08-05 13:35     ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-08-05 13:35 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Will Deacon, Jon Nettleton, Linuxarm, Steven Price,
	ACPI Devel Maling List, Linux IOMMU, wanghuiqiang,
	Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Robin Murphy, Linux ARM



> -----Original Message-----
> From: Ard Biesheuvel [mailto:ardb@kernel.org]
> Sent: 05 August 2021 14:23
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel
> <joro@8bytes.org>; Robin Murphy <robin.murphy@arm.com>; Will Deacon
> <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Jon
> Nettleton <jon@solid-run.com>; Eric Auger <eric.auger@redhat.com>;
> yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> 
> On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> > Hi,
> >
> > The series adds support to IORT RMR nodes specified in IORT
> > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> > memory ranges that are used by endpoints and require a unity
> > mapping in SMMU.
> >
> > We have faced issues with 3408iMR RAID controller cards which
> > fail to boot when SMMU is enabled. This is because these
> > controllers make use of host memory for various caching related
> > purposes and when SMMU is enabled the iMR firmware fails to
> > access these memory regions as there is no mapping for them.
> > IORT RMR provides a way for UEFI to describe and report these
> > memory regions so that the kernel can make a unity mapping for
> > these in SMMU.
> >
> 
> Does this mean we are ignoring the RMR memory ranges, and exposing the
> entire physical address space to devices using the stream IDs in
> question?

Nope. RMR node is used to describe the memory ranges used by end points
behind SMMU. And this information is used to create 1 : 1 mappings for those
ranges in SMMU. Anything outside those ranges will result in translation
fault(if there are no other dynamic DMA mappings).

Thanks,
Shameer

> 
> > Change History:
> >
> > v6 --> v7
> >
> > The only change from v6 is the fix pointed out by Steve to
> > the SMMUv2 SMR bypass install in patch #8.
> >
> > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> > yet as the series still needs more review[1].
> >
> > Feedback and tests on this series is very much appreciated.
> >
> > v5 --> v6
> > - Addressed comments from Robin & Lorenzo.
> >   : Moved iort_parse_rmr() to acpi_iort_init() from
> >     iort_init_platform_devices().
> >   : Removed use of struct iort_rmr_entry during the initial
> >     parse. Using struct iommu_resv_region instead.
> >   : Report RMR address alignment and overlap errors, but continue.
> >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
> >   on Type of RMR region. Suggested by Jon N.
> >
> > Thanks,
> > Shameer
> > [0] https://developer.arm.com/documentation/den0049/latest/
> > [1]
> https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth
> um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> > ------
> > v4 --> v5
> >  -Added a fw_data union to struct iommu_resv_region and removed
> >   struct iommu_rmr (Based on comments from Joerg/Robin).
> >  -Added iommu_put_rmrs() to release mem.
> >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
> >   yet because of the above changes.
> >
> > v3 -->v4
> > -Included the SMMUv2 SMR bypass install changes suggested by
> >  Steve(patch #7)
> > -As per Robin's comments, RMR reserve implementation is now
> >  more generic  (patch #8) and dropped v3 patches 8 and 10.
> > -Rebase to 5.13-rc1
> >
> > RFC v2 --> v3
> >  -Dropped RFC tag as the ACPICA header changes are now ready to be
> >   part of 5.13[0]. But this series still has a dependency on that patch.
> >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
> >   PCIe).
> >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
> >   discussion here[1].
> >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> > ------
> >
> > Jon Nettleton (1):
> >   iommu/arm-smmu: Get associated RMR info and install bypass SMR
> >
> > Shameer Kolothum (8):
> >   iommu: Introduce a union to struct iommu_resv_region
> >   ACPI/IORT: Add support for RMR node parsing
> >   iommu/dma: Introduce generic helper to retrieve RMR info
> >   ACPI/IORT: Add a helper to retrieve RMR memory regions
> >   iommu/arm-smmu-v3: Introduce strtab init helper
> >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
> >     bypass
> >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> >   iommu/dma: Reserve any RMR regions associated with a dev
> >
> >  drivers/acpi/arm64/iort.c                   | 172
> +++++++++++++++++++-
> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
> >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
> >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
> >  include/linux/acpi_iort.h                   |   7 +
> >  include/linux/dma-iommu.h                   |  13 ++
> >  include/linux/iommu.h                       |  11 ++
> >  7 files changed, 393 insertions(+), 23 deletions(-)
> >
> > --
> > 2.17.1
> >
> >
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-08-05 13:35     ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-08-05 13:35 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Linux ARM, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Lorenzo Pieralisi, Joerg Roedel, Robin Murphy, Will Deacon,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Jon Nettleton, Eric Auger,
	yangyicong



> -----Original Message-----
> From: Ard Biesheuvel [mailto:ardb@kernel.org]
> Sent: 05 August 2021 14:23
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel
> <joro@8bytes.org>; Robin Murphy <robin.murphy@arm.com>; Will Deacon
> <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Jon
> Nettleton <jon@solid-run.com>; Eric Auger <eric.auger@redhat.com>;
> yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> 
> On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> > Hi,
> >
> > The series adds support to IORT RMR nodes specified in IORT
> > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> > memory ranges that are used by endpoints and require a unity
> > mapping in SMMU.
> >
> > We have faced issues with 3408iMR RAID controller cards which
> > fail to boot when SMMU is enabled. This is because these
> > controllers make use of host memory for various caching related
> > purposes and when SMMU is enabled the iMR firmware fails to
> > access these memory regions as there is no mapping for them.
> > IORT RMR provides a way for UEFI to describe and report these
> > memory regions so that the kernel can make a unity mapping for
> > these in SMMU.
> >
> 
> Does this mean we are ignoring the RMR memory ranges, and exposing the
> entire physical address space to devices using the stream IDs in
> question?

Nope. RMR node is used to describe the memory ranges used by end points
behind SMMU. And this information is used to create 1 : 1 mappings for those
ranges in SMMU. Anything outside those ranges will result in translation
fault(if there are no other dynamic DMA mappings).

Thanks,
Shameer

> 
> > Change History:
> >
> > v6 --> v7
> >
> > The only change from v6 is the fix pointed out by Steve to
> > the SMMUv2 SMR bypass install in patch #8.
> >
> > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> > yet as the series still needs more review[1].
> >
> > Feedback and tests on this series is very much appreciated.
> >
> > v5 --> v6
> > - Addressed comments from Robin & Lorenzo.
> >   : Moved iort_parse_rmr() to acpi_iort_init() from
> >     iort_init_platform_devices().
> >   : Removed use of struct iort_rmr_entry during the initial
> >     parse. Using struct iommu_resv_region instead.
> >   : Report RMR address alignment and overlap errors, but continue.
> >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
> >   on Type of RMR region. Suggested by Jon N.
> >
> > Thanks,
> > Shameer
> > [0] https://developer.arm.com/documentation/den0049/latest/
> > [1]
> https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth
> um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> > ------
> > v4 --> v5
> >  -Added a fw_data union to struct iommu_resv_region and removed
> >   struct iommu_rmr (Based on comments from Joerg/Robin).
> >  -Added iommu_put_rmrs() to release mem.
> >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
> >   yet because of the above changes.
> >
> > v3 -->v4
> > -Included the SMMUv2 SMR bypass install changes suggested by
> >  Steve(patch #7)
> > -As per Robin's comments, RMR reserve implementation is now
> >  more generic  (patch #8) and dropped v3 patches 8 and 10.
> > -Rebase to 5.13-rc1
> >
> > RFC v2 --> v3
> >  -Dropped RFC tag as the ACPICA header changes are now ready to be
> >   part of 5.13[0]. But this series still has a dependency on that patch.
> >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
> >   PCIe).
> >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
> >   discussion here[1].
> >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> > ------
> >
> > Jon Nettleton (1):
> >   iommu/arm-smmu: Get associated RMR info and install bypass SMR
> >
> > Shameer Kolothum (8):
> >   iommu: Introduce a union to struct iommu_resv_region
> >   ACPI/IORT: Add support for RMR node parsing
> >   iommu/dma: Introduce generic helper to retrieve RMR info
> >   ACPI/IORT: Add a helper to retrieve RMR memory regions
> >   iommu/arm-smmu-v3: Introduce strtab init helper
> >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
> >     bypass
> >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> >   iommu/dma: Reserve any RMR regions associated with a dev
> >
> >  drivers/acpi/arm64/iort.c                   | 172
> +++++++++++++++++++-
> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
> >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
> >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
> >  include/linux/acpi_iort.h                   |   7 +
> >  include/linux/dma-iommu.h                   |  13 ++
> >  include/linux/iommu.h                       |  11 ++
> >  7 files changed, 393 insertions(+), 23 deletions(-)
> >
> > --
> > 2.17.1
> >
> >
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
  2021-08-05 13:35     ` Shameerali Kolothum Thodi
  (?)
@ 2021-08-05 14:09       ` Ard Biesheuvel
  -1 siblings, 0 replies; 150+ messages in thread
From: Ard Biesheuvel @ 2021-08-05 14:09 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi
  Cc: Linux ARM, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Lorenzo Pieralisi, Joerg Roedel, Robin Murphy, Will Deacon,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Jon Nettleton, Eric Auger,
	yangyicong

On Thu, 5 Aug 2021 at 15:35, Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Ard Biesheuvel [mailto:ardb@kernel.org]
> > Sent: 05 August 2021 14:23
> > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> > Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel
> > <joro@8bytes.org>; Robin Murphy <robin.murphy@arm.com>; Will Deacon
> > <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> > (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> > <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Jon
> > Nettleton <jon@solid-run.com>; Eric Auger <eric.auger@redhat.com>;
> > yangyicong <yangyicong@huawei.com>
> > Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> >
> > On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum
> > <shameerali.kolothum.thodi@huawei.com> wrote:
> > >
> > > Hi,
> > >
> > > The series adds support to IORT RMR nodes specified in IORT
> > > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> > > memory ranges that are used by endpoints and require a unity
> > > mapping in SMMU.
> > >
> > > We have faced issues with 3408iMR RAID controller cards which
> > > fail to boot when SMMU is enabled. This is because these
> > > controllers make use of host memory for various caching related
> > > purposes and when SMMU is enabled the iMR firmware fails to
> > > access these memory regions as there is no mapping for them.
> > > IORT RMR provides a way for UEFI to describe and report these
> > > memory regions so that the kernel can make a unity mapping for
> > > these in SMMU.
> > >
> >
> > Does this mean we are ignoring the RMR memory ranges, and exposing the
> > entire physical address space to devices using the stream IDs in
> > question?
>
> Nope. RMR node is used to describe the memory ranges used by end points
> behind SMMU. And this information is used to create 1 : 1 mappings for those
> ranges in SMMU. Anything outside those ranges will result in translation
> fault(if there are no other dynamic DMA mappings).
>

Excellent! It was not obvious to me from looking at the patches, so I
had to ask.

Thanks,
Ard.

>
> >
> > > Change History:
> > >
> > > v6 --> v7
> > >
> > > The only change from v6 is the fix pointed out by Steve to
> > > the SMMUv2 SMR bypass install in patch #8.
> > >
> > > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> > > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> > > yet as the series still needs more review[1].
> > >
> > > Feedback and tests on this series is very much appreciated.
> > >
> > > v5 --> v6
> > > - Addressed comments from Robin & Lorenzo.
> > >   : Moved iort_parse_rmr() to acpi_iort_init() from
> > >     iort_init_platform_devices().
> > >   : Removed use of struct iort_rmr_entry during the initial
> > >     parse. Using struct iommu_resv_region instead.
> > >   : Report RMR address alignment and overlap errors, but continue.
> > >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> > > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> > > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
> > >   on Type of RMR region. Suggested by Jon N.
> > >
> > > Thanks,
> > > Shameer
> > > [0] https://developer.arm.com/documentation/den0049/latest/
> > > [1]
> > https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth
> > um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> > > ------
> > > v4 --> v5
> > >  -Added a fw_data union to struct iommu_resv_region and removed
> > >   struct iommu_rmr (Based on comments from Joerg/Robin).
> > >  -Added iommu_put_rmrs() to release mem.
> > >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
> > >   yet because of the above changes.
> > >
> > > v3 -->v4
> > > -Included the SMMUv2 SMR bypass install changes suggested by
> > >  Steve(patch #7)
> > > -As per Robin's comments, RMR reserve implementation is now
> > >  more generic  (patch #8) and dropped v3 patches 8 and 10.
> > > -Rebase to 5.13-rc1
> > >
> > > RFC v2 --> v3
> > >  -Dropped RFC tag as the ACPICA header changes are now ready to be
> > >   part of 5.13[0]. But this series still has a dependency on that patch.
> > >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
> > >   PCIe).
> > >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
> > >   discussion here[1].
> > >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> > > ------
> > >
> > > Jon Nettleton (1):
> > >   iommu/arm-smmu: Get associated RMR info and install bypass SMR
> > >
> > > Shameer Kolothum (8):
> > >   iommu: Introduce a union to struct iommu_resv_region
> > >   ACPI/IORT: Add support for RMR node parsing
> > >   iommu/dma: Introduce generic helper to retrieve RMR info
> > >   ACPI/IORT: Add a helper to retrieve RMR memory regions
> > >   iommu/arm-smmu-v3: Introduce strtab init helper
> > >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
> > >     bypass
> > >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> > >   iommu/dma: Reserve any RMR regions associated with a dev
> > >
> > >  drivers/acpi/arm64/iort.c                   | 172
> > +++++++++++++++++++-
> > >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
> > >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
> > >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
> > >  include/linux/acpi_iort.h                   |   7 +
> > >  include/linux/dma-iommu.h                   |  13 ++
> > >  include/linux/iommu.h                       |  11 ++
> > >  7 files changed, 393 insertions(+), 23 deletions(-)
> > >
> > > --
> > > 2.17.1
> > >
> > >
> > > _______________________________________________
> > > linux-arm-kernel mailing list
> > > linux-arm-kernel@lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-08-05 14:09       ` Ard Biesheuvel
  0 siblings, 0 replies; 150+ messages in thread
From: Ard Biesheuvel @ 2021-08-05 14:09 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi
  Cc: Will Deacon, Jon Nettleton, Linuxarm, Steven Price,
	ACPI Devel Maling List, Linux IOMMU, wanghuiqiang,
	Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Robin Murphy, Linux ARM

On Thu, 5 Aug 2021 at 15:35, Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Ard Biesheuvel [mailto:ardb@kernel.org]
> > Sent: 05 August 2021 14:23
> > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> > Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel
> > <joro@8bytes.org>; Robin Murphy <robin.murphy@arm.com>; Will Deacon
> > <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> > (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> > <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Jon
> > Nettleton <jon@solid-run.com>; Eric Auger <eric.auger@redhat.com>;
> > yangyicong <yangyicong@huawei.com>
> > Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> >
> > On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum
> > <shameerali.kolothum.thodi@huawei.com> wrote:
> > >
> > > Hi,
> > >
> > > The series adds support to IORT RMR nodes specified in IORT
> > > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> > > memory ranges that are used by endpoints and require a unity
> > > mapping in SMMU.
> > >
> > > We have faced issues with 3408iMR RAID controller cards which
> > > fail to boot when SMMU is enabled. This is because these
> > > controllers make use of host memory for various caching related
> > > purposes and when SMMU is enabled the iMR firmware fails to
> > > access these memory regions as there is no mapping for them.
> > > IORT RMR provides a way for UEFI to describe and report these
> > > memory regions so that the kernel can make a unity mapping for
> > > these in SMMU.
> > >
> >
> > Does this mean we are ignoring the RMR memory ranges, and exposing the
> > entire physical address space to devices using the stream IDs in
> > question?
>
> Nope. RMR node is used to describe the memory ranges used by end points
> behind SMMU. And this information is used to create 1 : 1 mappings for those
> ranges in SMMU. Anything outside those ranges will result in translation
> fault(if there are no other dynamic DMA mappings).
>

Excellent! It was not obvious to me from looking at the patches, so I
had to ask.

Thanks,
Ard.

>
> >
> > > Change History:
> > >
> > > v6 --> v7
> > >
> > > The only change from v6 is the fix pointed out by Steve to
> > > the SMMUv2 SMR bypass install in patch #8.
> > >
> > > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> > > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> > > yet as the series still needs more review[1].
> > >
> > > Feedback and tests on this series is very much appreciated.
> > >
> > > v5 --> v6
> > > - Addressed comments from Robin & Lorenzo.
> > >   : Moved iort_parse_rmr() to acpi_iort_init() from
> > >     iort_init_platform_devices().
> > >   : Removed use of struct iort_rmr_entry during the initial
> > >     parse. Using struct iommu_resv_region instead.
> > >   : Report RMR address alignment and overlap errors, but continue.
> > >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> > > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> > > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
> > >   on Type of RMR region. Suggested by Jon N.
> > >
> > > Thanks,
> > > Shameer
> > > [0] https://developer.arm.com/documentation/den0049/latest/
> > > [1]
> > https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth
> > um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> > > ------
> > > v4 --> v5
> > >  -Added a fw_data union to struct iommu_resv_region and removed
> > >   struct iommu_rmr (Based on comments from Joerg/Robin).
> > >  -Added iommu_put_rmrs() to release mem.
> > >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
> > >   yet because of the above changes.
> > >
> > > v3 -->v4
> > > -Included the SMMUv2 SMR bypass install changes suggested by
> > >  Steve(patch #7)
> > > -As per Robin's comments, RMR reserve implementation is now
> > >  more generic  (patch #8) and dropped v3 patches 8 and 10.
> > > -Rebase to 5.13-rc1
> > >
> > > RFC v2 --> v3
> > >  -Dropped RFC tag as the ACPICA header changes are now ready to be
> > >   part of 5.13[0]. But this series still has a dependency on that patch.
> > >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
> > >   PCIe).
> > >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
> > >   discussion here[1].
> > >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> > > ------
> > >
> > > Jon Nettleton (1):
> > >   iommu/arm-smmu: Get associated RMR info and install bypass SMR
> > >
> > > Shameer Kolothum (8):
> > >   iommu: Introduce a union to struct iommu_resv_region
> > >   ACPI/IORT: Add support for RMR node parsing
> > >   iommu/dma: Introduce generic helper to retrieve RMR info
> > >   ACPI/IORT: Add a helper to retrieve RMR memory regions
> > >   iommu/arm-smmu-v3: Introduce strtab init helper
> > >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
> > >     bypass
> > >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> > >   iommu/dma: Reserve any RMR regions associated with a dev
> > >
> > >  drivers/acpi/arm64/iort.c                   | 172
> > +++++++++++++++++++-
> > >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
> > >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
> > >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
> > >  include/linux/acpi_iort.h                   |   7 +
> > >  include/linux/dma-iommu.h                   |  13 ++
> > >  include/linux/iommu.h                       |  11 ++
> > >  7 files changed, 393 insertions(+), 23 deletions(-)
> > >
> > > --
> > > 2.17.1
> > >
> > >
> > > _______________________________________________
> > > linux-arm-kernel mailing list
> > > linux-arm-kernel@lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-08-05 14:09       ` Ard Biesheuvel
  0 siblings, 0 replies; 150+ messages in thread
From: Ard Biesheuvel @ 2021-08-05 14:09 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi
  Cc: Linux ARM, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Lorenzo Pieralisi, Joerg Roedel, Robin Murphy, Will Deacon,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Jon Nettleton, Eric Auger,
	yangyicong

On Thu, 5 Aug 2021 at 15:35, Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Ard Biesheuvel [mailto:ardb@kernel.org]
> > Sent: 05 August 2021 14:23
> > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> > Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel
> > <joro@8bytes.org>; Robin Murphy <robin.murphy@arm.com>; Will Deacon
> > <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> > (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> > <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Jon
> > Nettleton <jon@solid-run.com>; Eric Auger <eric.auger@redhat.com>;
> > yangyicong <yangyicong@huawei.com>
> > Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> >
> > On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum
> > <shameerali.kolothum.thodi@huawei.com> wrote:
> > >
> > > Hi,
> > >
> > > The series adds support to IORT RMR nodes specified in IORT
> > > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> > > memory ranges that are used by endpoints and require a unity
> > > mapping in SMMU.
> > >
> > > We have faced issues with 3408iMR RAID controller cards which
> > > fail to boot when SMMU is enabled. This is because these
> > > controllers make use of host memory for various caching related
> > > purposes and when SMMU is enabled the iMR firmware fails to
> > > access these memory regions as there is no mapping for them.
> > > IORT RMR provides a way for UEFI to describe and report these
> > > memory regions so that the kernel can make a unity mapping for
> > > these in SMMU.
> > >
> >
> > Does this mean we are ignoring the RMR memory ranges, and exposing the
> > entire physical address space to devices using the stream IDs in
> > question?
>
> Nope. RMR node is used to describe the memory ranges used by end points
> behind SMMU. And this information is used to create 1 : 1 mappings for those
> ranges in SMMU. Anything outside those ranges will result in translation
> fault(if there are no other dynamic DMA mappings).
>

Excellent! It was not obvious to me from looking at the patches, so I
had to ask.

Thanks,
Ard.

>
> >
> > > Change History:
> > >
> > > v6 --> v7
> > >
> > > The only change from v6 is the fix pointed out by Steve to
> > > the SMMUv2 SMR bypass install in patch #8.
> > >
> > > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> > > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> > > yet as the series still needs more review[1].
> > >
> > > Feedback and tests on this series is very much appreciated.
> > >
> > > v5 --> v6
> > > - Addressed comments from Robin & Lorenzo.
> > >   : Moved iort_parse_rmr() to acpi_iort_init() from
> > >     iort_init_platform_devices().
> > >   : Removed use of struct iort_rmr_entry during the initial
> > >     parse. Using struct iommu_resv_region instead.
> > >   : Report RMR address alignment and overlap errors, but continue.
> > >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> > > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> > > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
> > >   on Type of RMR region. Suggested by Jon N.
> > >
> > > Thanks,
> > > Shameer
> > > [0] https://developer.arm.com/documentation/den0049/latest/
> > > [1]
> > https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth
> > um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> > > ------
> > > v4 --> v5
> > >  -Added a fw_data union to struct iommu_resv_region and removed
> > >   struct iommu_rmr (Based on comments from Joerg/Robin).
> > >  -Added iommu_put_rmrs() to release mem.
> > >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
> > >   yet because of the above changes.
> > >
> > > v3 -->v4
> > > -Included the SMMUv2 SMR bypass install changes suggested by
> > >  Steve(patch #7)
> > > -As per Robin's comments, RMR reserve implementation is now
> > >  more generic  (patch #8) and dropped v3 patches 8 and 10.
> > > -Rebase to 5.13-rc1
> > >
> > > RFC v2 --> v3
> > >  -Dropped RFC tag as the ACPICA header changes are now ready to be
> > >   part of 5.13[0]. But this series still has a dependency on that patch.
> > >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
> > >   PCIe).
> > >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
> > >   discussion here[1].
> > >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> > > ------
> > >
> > > Jon Nettleton (1):
> > >   iommu/arm-smmu: Get associated RMR info and install bypass SMR
> > >
> > > Shameer Kolothum (8):
> > >   iommu: Introduce a union to struct iommu_resv_region
> > >   ACPI/IORT: Add support for RMR node parsing
> > >   iommu/dma: Introduce generic helper to retrieve RMR info
> > >   ACPI/IORT: Add a helper to retrieve RMR memory regions
> > >   iommu/arm-smmu-v3: Introduce strtab init helper
> > >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
> > >     bypass
> > >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> > >   iommu/dma: Reserve any RMR regions associated with a dev
> > >
> > >  drivers/acpi/arm64/iort.c                   | 172
> > +++++++++++++++++++-
> > >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
> > >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
> > >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
> > >  include/linux/acpi_iort.h                   |   7 +
> > >  include/linux/dma-iommu.h                   |  13 ++
> > >  include/linux/iommu.h                       |  11 ++
> > >  7 files changed, 393 insertions(+), 23 deletions(-)
> > >
> > > --
> > > 2.17.1
> > >
> > >
> > > _______________________________________________
> > > linux-arm-kernel mailing list
> > > linux-arm-kernel@lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 4/9] ACPI/IORT: Add a helper to retrieve RMR memory regions
  2021-08-05  8:07   ` Shameer Kolothum
  (?)
@ 2021-08-05 15:43     ` Lorenzo Pieralisi
  -1 siblings, 0 replies; 150+ messages in thread
From: Lorenzo Pieralisi @ 2021-08-05 15:43 UTC (permalink / raw)
  To: Shameer Kolothum
  Cc: linux-arm-kernel, linux-acpi, iommu, linuxarm, joro,
	robin.murphy, will, wanghuiqiang, guohanjun, steven.price,
	Sami.Mujawar, jon, eric.auger, yangyicong

On Thu, Aug 05, 2021 at 09:07:19AM +0100, Shameer Kolothum wrote:
> Add a helper function (iort_iommu_get_rmrs()) that retrieves RMR
> memory descriptors associated with a given IOMMU. This will be used
> by IOMMU drivers to setup necessary mappings.
> 
> Invoke it from the generic helper iommu_dma_get_rmrs().
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>  drivers/acpi/arm64/iort.c | 38 ++++++++++++++++++++++++++++++++++++++
>  drivers/iommu/dma-iommu.c |  4 ++++
>  include/linux/acpi_iort.h |  7 +++++++
>  3 files changed, 49 insertions(+)

Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index d76ba46ebe67..3c32d62e63b6 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -809,6 +809,42 @@ static struct acpi_iort_node *iort_get_msi_resv_iommu(struct device *dev)
>  	return NULL;
>  }
>  
> +/**
> + * iort_iommu_get_rmrs() - Helper to retrieve RMR info associated with IOMMU
> + * @iommu_fwnode: fwnode for the IOMMU
> + * @head: RMR list head to be populated
> + *
> + * Returns: 0 on success, <0 failure. Please note, we will keep the already
> + *          allocated RMR reserve regions in case of a kmemdup()
> + *          failure.
> + */
> +int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
> +			struct list_head *head)
> +{
> +	struct iommu_resv_region *e;
> +	struct acpi_iort_node *iommu;
> +	int rmrs = 0;
> +
> +	iommu = iort_get_iort_node(iommu_fwnode);
> +	if (!iommu || list_empty(&iort_rmr_list))
> +		return -ENODEV;
> +
> +	list_for_each_entry(e, &iort_rmr_list, list) {
> +		struct iommu_resv_region *region;
> +
> +		if (e->fw_data.rmr.smmu != iommu)
> +			continue;
> +
> +		region = kmemdup(e, sizeof(*region), GFP_KERNEL);
> +		if (region) {
> +			list_add_tail(&region->list, head);
> +			rmrs++;
> +		}
> +	}
> +
> +	return (rmrs == 0) ? -ENODEV : 0;
> +}
> +
>  /**
>   * iort_iommu_msi_get_resv_regions - Reserved region driver helper
>   * @dev: Device from iommu_get_resv_regions()
> @@ -1041,6 +1077,8 @@ int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
>  { return 0; }
>  int iort_iommu_configure_id(struct device *dev, const u32 *input_id)
>  { return -ENODEV; }
> +int iort_iommu_get_rmrs(struct fwnode_handle *fwnode, struct list_head *head)
> +{ return -ENODEV; }
>  #endif
>  
>  static int nc_dma_get_range(struct device *dev, u64 *size)
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 2fa2445e9070..1b6e27475279 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -185,6 +185,9 @@ EXPORT_SYMBOL(iommu_put_dma_cookie);
>  int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
>  		       struct list_head *list)
>  {
> +	if (!is_of_node(iommu_fwnode))
> +		return iort_iommu_get_rmrs(iommu_fwnode, list);
> +
>  	return -EINVAL;
>  }
>  EXPORT_SYMBOL(iommu_dma_get_rmrs);
> @@ -200,6 +203,7 @@ EXPORT_SYMBOL(iommu_dma_get_rmrs);
>  void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
>  			struct list_head *list)
>  {
> +	generic_iommu_put_resv_regions(iommu_fwnode->dev, list);
>  }
>  EXPORT_SYMBOL(iommu_dma_put_rmrs);
>  
> diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
> index f1f0842a2cb2..d8c030c103f5 100644
> --- a/include/linux/acpi_iort.h
> +++ b/include/linux/acpi_iort.h
> @@ -38,6 +38,8 @@ int iort_dma_get_ranges(struct device *dev, u64 *size);
>  int iort_iommu_configure_id(struct device *dev, const u32 *id_in);
>  int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head);
>  phys_addr_t acpi_iort_dma_get_max_cpu_address(void);
> +int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
> +			struct list_head *list);
>  #else
>  static inline void acpi_iort_init(void) { }
>  static inline u32 iort_msi_map_id(struct device *dev, u32 id)
> @@ -57,6 +59,11 @@ int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
>  
>  static inline phys_addr_t acpi_iort_dma_get_max_cpu_address(void)
>  { return PHYS_ADDR_MAX; }
> +
> +static inline
> +int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
> +			struct list_head *list)
> +{ return -ENODEV; }
>  #endif
>  
>  #endif /* __ACPI_IORT_H__ */
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 4/9] ACPI/IORT: Add a helper to retrieve RMR memory regions
@ 2021-08-05 15:43     ` Lorenzo Pieralisi
  0 siblings, 0 replies; 150+ messages in thread
From: Lorenzo Pieralisi @ 2021-08-05 15:43 UTC (permalink / raw)
  To: Shameer Kolothum
  Cc: will, jon, linuxarm, steven.price, linux-acpi, iommu,
	wanghuiqiang, guohanjun, yangyicong, Sami.Mujawar, robin.murphy,
	linux-arm-kernel

On Thu, Aug 05, 2021 at 09:07:19AM +0100, Shameer Kolothum wrote:
> Add a helper function (iort_iommu_get_rmrs()) that retrieves RMR
> memory descriptors associated with a given IOMMU. This will be used
> by IOMMU drivers to setup necessary mappings.
> 
> Invoke it from the generic helper iommu_dma_get_rmrs().
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>  drivers/acpi/arm64/iort.c | 38 ++++++++++++++++++++++++++++++++++++++
>  drivers/iommu/dma-iommu.c |  4 ++++
>  include/linux/acpi_iort.h |  7 +++++++
>  3 files changed, 49 insertions(+)

Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index d76ba46ebe67..3c32d62e63b6 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -809,6 +809,42 @@ static struct acpi_iort_node *iort_get_msi_resv_iommu(struct device *dev)
>  	return NULL;
>  }
>  
> +/**
> + * iort_iommu_get_rmrs() - Helper to retrieve RMR info associated with IOMMU
> + * @iommu_fwnode: fwnode for the IOMMU
> + * @head: RMR list head to be populated
> + *
> + * Returns: 0 on success, <0 failure. Please note, we will keep the already
> + *          allocated RMR reserve regions in case of a kmemdup()
> + *          failure.
> + */
> +int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
> +			struct list_head *head)
> +{
> +	struct iommu_resv_region *e;
> +	struct acpi_iort_node *iommu;
> +	int rmrs = 0;
> +
> +	iommu = iort_get_iort_node(iommu_fwnode);
> +	if (!iommu || list_empty(&iort_rmr_list))
> +		return -ENODEV;
> +
> +	list_for_each_entry(e, &iort_rmr_list, list) {
> +		struct iommu_resv_region *region;
> +
> +		if (e->fw_data.rmr.smmu != iommu)
> +			continue;
> +
> +		region = kmemdup(e, sizeof(*region), GFP_KERNEL);
> +		if (region) {
> +			list_add_tail(&region->list, head);
> +			rmrs++;
> +		}
> +	}
> +
> +	return (rmrs == 0) ? -ENODEV : 0;
> +}
> +
>  /**
>   * iort_iommu_msi_get_resv_regions - Reserved region driver helper
>   * @dev: Device from iommu_get_resv_regions()
> @@ -1041,6 +1077,8 @@ int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
>  { return 0; }
>  int iort_iommu_configure_id(struct device *dev, const u32 *input_id)
>  { return -ENODEV; }
> +int iort_iommu_get_rmrs(struct fwnode_handle *fwnode, struct list_head *head)
> +{ return -ENODEV; }
>  #endif
>  
>  static int nc_dma_get_range(struct device *dev, u64 *size)
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 2fa2445e9070..1b6e27475279 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -185,6 +185,9 @@ EXPORT_SYMBOL(iommu_put_dma_cookie);
>  int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
>  		       struct list_head *list)
>  {
> +	if (!is_of_node(iommu_fwnode))
> +		return iort_iommu_get_rmrs(iommu_fwnode, list);
> +
>  	return -EINVAL;
>  }
>  EXPORT_SYMBOL(iommu_dma_get_rmrs);
> @@ -200,6 +203,7 @@ EXPORT_SYMBOL(iommu_dma_get_rmrs);
>  void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
>  			struct list_head *list)
>  {
> +	generic_iommu_put_resv_regions(iommu_fwnode->dev, list);
>  }
>  EXPORT_SYMBOL(iommu_dma_put_rmrs);
>  
> diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
> index f1f0842a2cb2..d8c030c103f5 100644
> --- a/include/linux/acpi_iort.h
> +++ b/include/linux/acpi_iort.h
> @@ -38,6 +38,8 @@ int iort_dma_get_ranges(struct device *dev, u64 *size);
>  int iort_iommu_configure_id(struct device *dev, const u32 *id_in);
>  int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head);
>  phys_addr_t acpi_iort_dma_get_max_cpu_address(void);
> +int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
> +			struct list_head *list);
>  #else
>  static inline void acpi_iort_init(void) { }
>  static inline u32 iort_msi_map_id(struct device *dev, u32 id)
> @@ -57,6 +59,11 @@ int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
>  
>  static inline phys_addr_t acpi_iort_dma_get_max_cpu_address(void)
>  { return PHYS_ADDR_MAX; }
> +
> +static inline
> +int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
> +			struct list_head *list)
> +{ return -ENODEV; }
>  #endif
>  
>  #endif /* __ACPI_IORT_H__ */
> -- 
> 2.17.1
> 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 4/9] ACPI/IORT: Add a helper to retrieve RMR memory regions
@ 2021-08-05 15:43     ` Lorenzo Pieralisi
  0 siblings, 0 replies; 150+ messages in thread
From: Lorenzo Pieralisi @ 2021-08-05 15:43 UTC (permalink / raw)
  To: Shameer Kolothum
  Cc: linux-arm-kernel, linux-acpi, iommu, linuxarm, joro,
	robin.murphy, will, wanghuiqiang, guohanjun, steven.price,
	Sami.Mujawar, jon, eric.auger, yangyicong

On Thu, Aug 05, 2021 at 09:07:19AM +0100, Shameer Kolothum wrote:
> Add a helper function (iort_iommu_get_rmrs()) that retrieves RMR
> memory descriptors associated with a given IOMMU. This will be used
> by IOMMU drivers to setup necessary mappings.
> 
> Invoke it from the generic helper iommu_dma_get_rmrs().
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>  drivers/acpi/arm64/iort.c | 38 ++++++++++++++++++++++++++++++++++++++
>  drivers/iommu/dma-iommu.c |  4 ++++
>  include/linux/acpi_iort.h |  7 +++++++
>  3 files changed, 49 insertions(+)

Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index d76ba46ebe67..3c32d62e63b6 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -809,6 +809,42 @@ static struct acpi_iort_node *iort_get_msi_resv_iommu(struct device *dev)
>  	return NULL;
>  }
>  
> +/**
> + * iort_iommu_get_rmrs() - Helper to retrieve RMR info associated with IOMMU
> + * @iommu_fwnode: fwnode for the IOMMU
> + * @head: RMR list head to be populated
> + *
> + * Returns: 0 on success, <0 failure. Please note, we will keep the already
> + *          allocated RMR reserve regions in case of a kmemdup()
> + *          failure.
> + */
> +int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
> +			struct list_head *head)
> +{
> +	struct iommu_resv_region *e;
> +	struct acpi_iort_node *iommu;
> +	int rmrs = 0;
> +
> +	iommu = iort_get_iort_node(iommu_fwnode);
> +	if (!iommu || list_empty(&iort_rmr_list))
> +		return -ENODEV;
> +
> +	list_for_each_entry(e, &iort_rmr_list, list) {
> +		struct iommu_resv_region *region;
> +
> +		if (e->fw_data.rmr.smmu != iommu)
> +			continue;
> +
> +		region = kmemdup(e, sizeof(*region), GFP_KERNEL);
> +		if (region) {
> +			list_add_tail(&region->list, head);
> +			rmrs++;
> +		}
> +	}
> +
> +	return (rmrs == 0) ? -ENODEV : 0;
> +}
> +
>  /**
>   * iort_iommu_msi_get_resv_regions - Reserved region driver helper
>   * @dev: Device from iommu_get_resv_regions()
> @@ -1041,6 +1077,8 @@ int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
>  { return 0; }
>  int iort_iommu_configure_id(struct device *dev, const u32 *input_id)
>  { return -ENODEV; }
> +int iort_iommu_get_rmrs(struct fwnode_handle *fwnode, struct list_head *head)
> +{ return -ENODEV; }
>  #endif
>  
>  static int nc_dma_get_range(struct device *dev, u64 *size)
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 2fa2445e9070..1b6e27475279 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -185,6 +185,9 @@ EXPORT_SYMBOL(iommu_put_dma_cookie);
>  int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
>  		       struct list_head *list)
>  {
> +	if (!is_of_node(iommu_fwnode))
> +		return iort_iommu_get_rmrs(iommu_fwnode, list);
> +
>  	return -EINVAL;
>  }
>  EXPORT_SYMBOL(iommu_dma_get_rmrs);
> @@ -200,6 +203,7 @@ EXPORT_SYMBOL(iommu_dma_get_rmrs);
>  void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
>  			struct list_head *list)
>  {
> +	generic_iommu_put_resv_regions(iommu_fwnode->dev, list);
>  }
>  EXPORT_SYMBOL(iommu_dma_put_rmrs);
>  
> diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
> index f1f0842a2cb2..d8c030c103f5 100644
> --- a/include/linux/acpi_iort.h
> +++ b/include/linux/acpi_iort.h
> @@ -38,6 +38,8 @@ int iort_dma_get_ranges(struct device *dev, u64 *size);
>  int iort_iommu_configure_id(struct device *dev, const u32 *id_in);
>  int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head);
>  phys_addr_t acpi_iort_dma_get_max_cpu_address(void);
> +int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
> +			struct list_head *list);
>  #else
>  static inline void acpi_iort_init(void) { }
>  static inline u32 iort_msi_map_id(struct device *dev, u32 id)
> @@ -57,6 +59,11 @@ int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
>  
>  static inline phys_addr_t acpi_iort_dma_get_max_cpu_address(void)
>  { return PHYS_ADDR_MAX; }
> +
> +static inline
> +int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
> +			struct list_head *list)
> +{ return -ENODEV; }
>  #endif
>  
>  #endif /* __ACPI_IORT_H__ */
> -- 
> 2.17.1
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-08-05  8:07   ` Shameer Kolothum
  (?)
@ 2021-08-05 16:03     ` Lorenzo Pieralisi
  -1 siblings, 0 replies; 150+ messages in thread
From: Lorenzo Pieralisi @ 2021-08-05 16:03 UTC (permalink / raw)
  To: Shameer Kolothum
  Cc: linux-arm-kernel, linux-acpi, iommu, linuxarm, joro,
	robin.murphy, will, wanghuiqiang, guohanjun, steven.price,
	Sami.Mujawar, jon, eric.auger, yangyicong

On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:

[...]

> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> +{
> +	struct acpi_iort_node *smmu;
> +	struct acpi_iort_rmr *rmr;
> +	struct acpi_iort_rmr_desc *rmr_desc;
> +	u32 map_count = iort_node->mapping_count;
> +	u32 sid;
> +	int i;
> +
> +	if (!iort_node->mapping_offset || map_count != 1) {
> +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	/* Retrieve associated smmu and stream id */
> +	smmu = iort_node_get_id(iort_node, &sid, 0);
> +	if (!smmu) {
> +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	/* Retrieve RMR data */
> +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> +	if (!rmr->rmr_offset || !rmr->rmr_count) {
> +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> +				rmr->rmr_offset);
> +
> +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> +
> +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> +		struct iommu_resv_region *region;
> +		enum iommu_resv_type type;
> +		int prot = IOMMU_READ | IOMMU_WRITE;
> +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> +
> +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> +			/* PAGE align base addr and size */
> +			addr &= PAGE_MASK;
> +			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> +
> +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> +			       rmr_desc->base_address,
> +			       rmr_desc->base_address + rmr_desc->length - 1,
> +			       addr, addr + size - 1);
> +		}
> +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> +			type = IOMMU_RESV_DIRECT_RELAXABLE;
> +			/*
> +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> +			 * normally used for allocated system memory that is
> +			 * then used for device specific reserved regions.
> +			 */
> +			prot |= IOMMU_CACHE;
> +		} else {
> +			type = IOMMU_RESV_DIRECT;
> +			/*
> +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> +			 * for device memory like MSI doorbell.
> +			 */
> +			prot |= IOMMU_MMIO;
> +		}

On the prot value assignment based on the remapping flag, I'd like to
hear Robin/Joerg's opinion, I'd avoid being in a situation where
"normally" this would work but then we have to quirk it.

Is this a valid assumption _always_ ?

Thanks,
Lorenzo

> +
> +		region = iommu_alloc_resv_region(addr, size, prot, type);
> +		if (region) {
> +			region->fw_data.rmr.flags = rmr->flags;
> +			region->fw_data.rmr.sid = sid;
> +			region->fw_data.rmr.smmu = smmu;
> +			list_add_tail(&region->list, &iort_rmr_list);
> +		}
> +	}
> +}
> +
> +static void __init iort_parse_rmr(void)
> +{
> +	struct acpi_iort_node *iort_node, *iort_end;
> +	struct acpi_table_iort *iort;
> +	int i;
> +
> +	if (iort_table->revision < 3)
> +		return;
> +
> +	iort = (struct acpi_table_iort *)iort_table;
> +
> +	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> +				 iort->node_offset);
> +	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> +				iort_table->length);
> +
> +	for (i = 0; i < iort->node_count; i++) {
> +		if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
> +			       "IORT node pointer overflows, bad table!\n"))
> +			return;
> +
> +		if (iort_node->type == ACPI_IORT_NODE_RMR)
> +			iort_node_get_rmr_info(iort_node);
> +
> +		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> +					 iort_node->length);
> +	}
> +}
>  
>  static void __init iort_init_platform_devices(void)
>  {
> @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
>  	}
>  
>  	iort_init_platform_devices();
> +	iort_parse_rmr();
>  }
>  
>  #ifdef CONFIG_ZONE_DMA
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-08-05 16:03     ` Lorenzo Pieralisi
  0 siblings, 0 replies; 150+ messages in thread
From: Lorenzo Pieralisi @ 2021-08-05 16:03 UTC (permalink / raw)
  To: Shameer Kolothum
  Cc: will, jon, linuxarm, steven.price, linux-acpi, iommu,
	wanghuiqiang, guohanjun, yangyicong, Sami.Mujawar, robin.murphy,
	linux-arm-kernel

On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:

[...]

> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> +{
> +	struct acpi_iort_node *smmu;
> +	struct acpi_iort_rmr *rmr;
> +	struct acpi_iort_rmr_desc *rmr_desc;
> +	u32 map_count = iort_node->mapping_count;
> +	u32 sid;
> +	int i;
> +
> +	if (!iort_node->mapping_offset || map_count != 1) {
> +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	/* Retrieve associated smmu and stream id */
> +	smmu = iort_node_get_id(iort_node, &sid, 0);
> +	if (!smmu) {
> +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	/* Retrieve RMR data */
> +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> +	if (!rmr->rmr_offset || !rmr->rmr_count) {
> +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> +				rmr->rmr_offset);
> +
> +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> +
> +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> +		struct iommu_resv_region *region;
> +		enum iommu_resv_type type;
> +		int prot = IOMMU_READ | IOMMU_WRITE;
> +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> +
> +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> +			/* PAGE align base addr and size */
> +			addr &= PAGE_MASK;
> +			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> +
> +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> +			       rmr_desc->base_address,
> +			       rmr_desc->base_address + rmr_desc->length - 1,
> +			       addr, addr + size - 1);
> +		}
> +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> +			type = IOMMU_RESV_DIRECT_RELAXABLE;
> +			/*
> +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> +			 * normally used for allocated system memory that is
> +			 * then used for device specific reserved regions.
> +			 */
> +			prot |= IOMMU_CACHE;
> +		} else {
> +			type = IOMMU_RESV_DIRECT;
> +			/*
> +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> +			 * for device memory like MSI doorbell.
> +			 */
> +			prot |= IOMMU_MMIO;
> +		}

On the prot value assignment based on the remapping flag, I'd like to
hear Robin/Joerg's opinion, I'd avoid being in a situation where
"normally" this would work but then we have to quirk it.

Is this a valid assumption _always_ ?

Thanks,
Lorenzo

> +
> +		region = iommu_alloc_resv_region(addr, size, prot, type);
> +		if (region) {
> +			region->fw_data.rmr.flags = rmr->flags;
> +			region->fw_data.rmr.sid = sid;
> +			region->fw_data.rmr.smmu = smmu;
> +			list_add_tail(&region->list, &iort_rmr_list);
> +		}
> +	}
> +}
> +
> +static void __init iort_parse_rmr(void)
> +{
> +	struct acpi_iort_node *iort_node, *iort_end;
> +	struct acpi_table_iort *iort;
> +	int i;
> +
> +	if (iort_table->revision < 3)
> +		return;
> +
> +	iort = (struct acpi_table_iort *)iort_table;
> +
> +	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> +				 iort->node_offset);
> +	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> +				iort_table->length);
> +
> +	for (i = 0; i < iort->node_count; i++) {
> +		if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
> +			       "IORT node pointer overflows, bad table!\n"))
> +			return;
> +
> +		if (iort_node->type == ACPI_IORT_NODE_RMR)
> +			iort_node_get_rmr_info(iort_node);
> +
> +		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> +					 iort_node->length);
> +	}
> +}
>  
>  static void __init iort_init_platform_devices(void)
>  {
> @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
>  	}
>  
>  	iort_init_platform_devices();
> +	iort_parse_rmr();
>  }
>  
>  #ifdef CONFIG_ZONE_DMA
> -- 
> 2.17.1
> 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-08-05 16:03     ` Lorenzo Pieralisi
  0 siblings, 0 replies; 150+ messages in thread
From: Lorenzo Pieralisi @ 2021-08-05 16:03 UTC (permalink / raw)
  To: Shameer Kolothum
  Cc: linux-arm-kernel, linux-acpi, iommu, linuxarm, joro,
	robin.murphy, will, wanghuiqiang, guohanjun, steven.price,
	Sami.Mujawar, jon, eric.auger, yangyicong

On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:

[...]

> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> +{
> +	struct acpi_iort_node *smmu;
> +	struct acpi_iort_rmr *rmr;
> +	struct acpi_iort_rmr_desc *rmr_desc;
> +	u32 map_count = iort_node->mapping_count;
> +	u32 sid;
> +	int i;
> +
> +	if (!iort_node->mapping_offset || map_count != 1) {
> +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	/* Retrieve associated smmu and stream id */
> +	smmu = iort_node_get_id(iort_node, &sid, 0);
> +	if (!smmu) {
> +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	/* Retrieve RMR data */
> +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> +	if (!rmr->rmr_offset || !rmr->rmr_count) {
> +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> +				rmr->rmr_offset);
> +
> +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> +
> +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> +		struct iommu_resv_region *region;
> +		enum iommu_resv_type type;
> +		int prot = IOMMU_READ | IOMMU_WRITE;
> +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> +
> +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> +			/* PAGE align base addr and size */
> +			addr &= PAGE_MASK;
> +			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> +
> +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> +			       rmr_desc->base_address,
> +			       rmr_desc->base_address + rmr_desc->length - 1,
> +			       addr, addr + size - 1);
> +		}
> +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> +			type = IOMMU_RESV_DIRECT_RELAXABLE;
> +			/*
> +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> +			 * normally used for allocated system memory that is
> +			 * then used for device specific reserved regions.
> +			 */
> +			prot |= IOMMU_CACHE;
> +		} else {
> +			type = IOMMU_RESV_DIRECT;
> +			/*
> +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> +			 * for device memory like MSI doorbell.
> +			 */
> +			prot |= IOMMU_MMIO;
> +		}

On the prot value assignment based on the remapping flag, I'd like to
hear Robin/Joerg's opinion, I'd avoid being in a situation where
"normally" this would work but then we have to quirk it.

Is this a valid assumption _always_ ?

Thanks,
Lorenzo

> +
> +		region = iommu_alloc_resv_region(addr, size, prot, type);
> +		if (region) {
> +			region->fw_data.rmr.flags = rmr->flags;
> +			region->fw_data.rmr.sid = sid;
> +			region->fw_data.rmr.smmu = smmu;
> +			list_add_tail(&region->list, &iort_rmr_list);
> +		}
> +	}
> +}
> +
> +static void __init iort_parse_rmr(void)
> +{
> +	struct acpi_iort_node *iort_node, *iort_end;
> +	struct acpi_table_iort *iort;
> +	int i;
> +
> +	if (iort_table->revision < 3)
> +		return;
> +
> +	iort = (struct acpi_table_iort *)iort_table;
> +
> +	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> +				 iort->node_offset);
> +	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> +				iort_table->length);
> +
> +	for (i = 0; i < iort->node_count; i++) {
> +		if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
> +			       "IORT node pointer overflows, bad table!\n"))
> +			return;
> +
> +		if (iort_node->type == ACPI_IORT_NODE_RMR)
> +			iort_node_get_rmr_info(iort_node);
> +
> +		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> +					 iort_node->length);
> +	}
> +}
>  
>  static void __init iort_init_platform_devices(void)
>  {
> @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
>  	}
>  
>  	iort_init_platform_devices();
> +	iort_parse_rmr();
>  }
>  
>  #ifdef CONFIG_ZONE_DMA
> -- 
> 2.17.1
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-08-05 16:03     ` Lorenzo Pieralisi
  (?)
@ 2021-08-05 16:31       ` Jon Nettleton
  -1 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-08-05 16:31 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	iommu, Linuxarm, joro, Robin Murphy, Will Deacon, wanghuiqiang,
	Hanjun Guo, Steven Price, Sami Mujawar, eric.auger, yangyicong

On Thu, Aug 5, 2021 at 6:03 PM Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:
>
> [...]
>
> > +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> > +{
> > +     struct acpi_iort_node *smmu;
> > +     struct acpi_iort_rmr *rmr;
> > +     struct acpi_iort_rmr_desc *rmr_desc;
> > +     u32 map_count = iort_node->mapping_count;
> > +     u32 sid;
> > +     int i;
> > +
> > +     if (!iort_node->mapping_offset || map_count != 1) {
> > +             pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     /* Retrieve associated smmu and stream id */
> > +     smmu = iort_node_get_id(iort_node, &sid, 0);
> > +     if (!smmu) {
> > +             pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     /* Retrieve RMR data */
> > +     rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> > +     if (!rmr->rmr_offset || !rmr->rmr_count) {
> > +             pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> > +                             rmr->rmr_offset);
> > +
> > +     iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> > +
> > +     for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> > +             struct iommu_resv_region *region;
> > +             enum iommu_resv_type type;
> > +             int prot = IOMMU_READ | IOMMU_WRITE;
> > +             u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> > +
> > +             if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> > +                     /* PAGE align base addr and size */
> > +                     addr &= PAGE_MASK;
> > +                     size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> > +
> > +                     pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> > +                            rmr_desc->base_address,
> > +                            rmr_desc->base_address + rmr_desc->length - 1,
> > +                            addr, addr + size - 1);
> > +             }
> > +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> > +                     /*
> > +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > +                      * normally used for allocated system memory that is
> > +                      * then used for device specific reserved regions.
> > +                      */
> > +                     prot |= IOMMU_CACHE;
> > +             } else {
> > +                     type = IOMMU_RESV_DIRECT;
> > +                     /*
> > +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> > +                      * for device memory like MSI doorbell.
> > +                      */
> > +                     prot |= IOMMU_MMIO;
> > +             }
>
> On the prot value assignment based on the remapping flag, I'd like to
> hear Robin/Joerg's opinion, I'd avoid being in a situation where
> "normally" this would work but then we have to quirk it.
>
> Is this a valid assumption _always_ ?

These assumptions were made based on the historic use cases I could
find reading the history.  There aren't many known examples "in the wild"
because so far we haven't had a mechanism other than quirks based
around device-tree implementations.

Ultimately I believe the proper solution will need to be another flag
in the RMR table that specifies the type of memory an RMR Node
describes, not just the base and length.

-Jon

>
> Thanks,
> Lorenzo
>
> > +
> > +             region = iommu_alloc_resv_region(addr, size, prot, type);
> > +             if (region) {
> > +                     region->fw_data.rmr.flags = rmr->flags;
> > +                     region->fw_data.rmr.sid = sid;
> > +                     region->fw_data.rmr.smmu = smmu;
> > +                     list_add_tail(&region->list, &iort_rmr_list);
> > +             }
> > +     }
> > +}
> > +
> > +static void __init iort_parse_rmr(void)
> > +{
> > +     struct acpi_iort_node *iort_node, *iort_end;
> > +     struct acpi_table_iort *iort;
> > +     int i;
> > +
> > +     if (iort_table->revision < 3)
> > +             return;
> > +
> > +     iort = (struct acpi_table_iort *)iort_table;
> > +
> > +     iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +                              iort->node_offset);
> > +     iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +                             iort_table->length);
> > +
> > +     for (i = 0; i < iort->node_count; i++) {
> > +             if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
> > +                            "IORT node pointer overflows, bad table!\n"))
> > +                     return;
> > +
> > +             if (iort_node->type == ACPI_IORT_NODE_RMR)
> > +                     iort_node_get_rmr_info(iort_node);
> > +
> > +             iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> > +                                      iort_node->length);
> > +     }
> > +}
> >
> >  static void __init iort_init_platform_devices(void)
> >  {
> > @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
> >       }
> >
> >       iort_init_platform_devices();
> > +     iort_parse_rmr();
> >  }
> >
> >  #ifdef CONFIG_ZONE_DMA
> > --
> > 2.17.1
> >

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-08-05 16:31       ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-08-05 16:31 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Will Deacon, Linuxarm, Steven Price, ACPI Devel Maling List,
	iommu, wanghuiqiang, Hanjun Guo, yangyicong, Sami Mujawar,
	Robin Murphy, linux-arm-kernel

On Thu, Aug 5, 2021 at 6:03 PM Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:
>
> [...]
>
> > +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> > +{
> > +     struct acpi_iort_node *smmu;
> > +     struct acpi_iort_rmr *rmr;
> > +     struct acpi_iort_rmr_desc *rmr_desc;
> > +     u32 map_count = iort_node->mapping_count;
> > +     u32 sid;
> > +     int i;
> > +
> > +     if (!iort_node->mapping_offset || map_count != 1) {
> > +             pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     /* Retrieve associated smmu and stream id */
> > +     smmu = iort_node_get_id(iort_node, &sid, 0);
> > +     if (!smmu) {
> > +             pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     /* Retrieve RMR data */
> > +     rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> > +     if (!rmr->rmr_offset || !rmr->rmr_count) {
> > +             pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> > +                             rmr->rmr_offset);
> > +
> > +     iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> > +
> > +     for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> > +             struct iommu_resv_region *region;
> > +             enum iommu_resv_type type;
> > +             int prot = IOMMU_READ | IOMMU_WRITE;
> > +             u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> > +
> > +             if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> > +                     /* PAGE align base addr and size */
> > +                     addr &= PAGE_MASK;
> > +                     size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> > +
> > +                     pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> > +                            rmr_desc->base_address,
> > +                            rmr_desc->base_address + rmr_desc->length - 1,
> > +                            addr, addr + size - 1);
> > +             }
> > +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> > +                     /*
> > +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > +                      * normally used for allocated system memory that is
> > +                      * then used for device specific reserved regions.
> > +                      */
> > +                     prot |= IOMMU_CACHE;
> > +             } else {
> > +                     type = IOMMU_RESV_DIRECT;
> > +                     /*
> > +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> > +                      * for device memory like MSI doorbell.
> > +                      */
> > +                     prot |= IOMMU_MMIO;
> > +             }
>
> On the prot value assignment based on the remapping flag, I'd like to
> hear Robin/Joerg's opinion, I'd avoid being in a situation where
> "normally" this would work but then we have to quirk it.
>
> Is this a valid assumption _always_ ?

These assumptions were made based on the historic use cases I could
find reading the history.  There aren't many known examples "in the wild"
because so far we haven't had a mechanism other than quirks based
around device-tree implementations.

Ultimately I believe the proper solution will need to be another flag
in the RMR table that specifies the type of memory an RMR Node
describes, not just the base and length.

-Jon

>
> Thanks,
> Lorenzo
>
> > +
> > +             region = iommu_alloc_resv_region(addr, size, prot, type);
> > +             if (region) {
> > +                     region->fw_data.rmr.flags = rmr->flags;
> > +                     region->fw_data.rmr.sid = sid;
> > +                     region->fw_data.rmr.smmu = smmu;
> > +                     list_add_tail(&region->list, &iort_rmr_list);
> > +             }
> > +     }
> > +}
> > +
> > +static void __init iort_parse_rmr(void)
> > +{
> > +     struct acpi_iort_node *iort_node, *iort_end;
> > +     struct acpi_table_iort *iort;
> > +     int i;
> > +
> > +     if (iort_table->revision < 3)
> > +             return;
> > +
> > +     iort = (struct acpi_table_iort *)iort_table;
> > +
> > +     iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +                              iort->node_offset);
> > +     iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +                             iort_table->length);
> > +
> > +     for (i = 0; i < iort->node_count; i++) {
> > +             if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
> > +                            "IORT node pointer overflows, bad table!\n"))
> > +                     return;
> > +
> > +             if (iort_node->type == ACPI_IORT_NODE_RMR)
> > +                     iort_node_get_rmr_info(iort_node);
> > +
> > +             iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> > +                                      iort_node->length);
> > +     }
> > +}
> >
> >  static void __init iort_init_platform_devices(void)
> >  {
> > @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
> >       }
> >
> >       iort_init_platform_devices();
> > +     iort_parse_rmr();
> >  }
> >
> >  #ifdef CONFIG_ZONE_DMA
> > --
> > 2.17.1
> >
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-08-05 16:31       ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-08-05 16:31 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	iommu, Linuxarm, joro, Robin Murphy, Will Deacon, wanghuiqiang,
	Hanjun Guo, Steven Price, Sami Mujawar, eric.auger, yangyicong

On Thu, Aug 5, 2021 at 6:03 PM Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:
>
> [...]
>
> > +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> > +{
> > +     struct acpi_iort_node *smmu;
> > +     struct acpi_iort_rmr *rmr;
> > +     struct acpi_iort_rmr_desc *rmr_desc;
> > +     u32 map_count = iort_node->mapping_count;
> > +     u32 sid;
> > +     int i;
> > +
> > +     if (!iort_node->mapping_offset || map_count != 1) {
> > +             pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     /* Retrieve associated smmu and stream id */
> > +     smmu = iort_node_get_id(iort_node, &sid, 0);
> > +     if (!smmu) {
> > +             pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     /* Retrieve RMR data */
> > +     rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> > +     if (!rmr->rmr_offset || !rmr->rmr_count) {
> > +             pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> > +                             rmr->rmr_offset);
> > +
> > +     iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> > +
> > +     for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> > +             struct iommu_resv_region *region;
> > +             enum iommu_resv_type type;
> > +             int prot = IOMMU_READ | IOMMU_WRITE;
> > +             u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> > +
> > +             if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> > +                     /* PAGE align base addr and size */
> > +                     addr &= PAGE_MASK;
> > +                     size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> > +
> > +                     pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> > +                            rmr_desc->base_address,
> > +                            rmr_desc->base_address + rmr_desc->length - 1,
> > +                            addr, addr + size - 1);
> > +             }
> > +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> > +                     /*
> > +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > +                      * normally used for allocated system memory that is
> > +                      * then used for device specific reserved regions.
> > +                      */
> > +                     prot |= IOMMU_CACHE;
> > +             } else {
> > +                     type = IOMMU_RESV_DIRECT;
> > +                     /*
> > +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> > +                      * for device memory like MSI doorbell.
> > +                      */
> > +                     prot |= IOMMU_MMIO;
> > +             }
>
> On the prot value assignment based on the remapping flag, I'd like to
> hear Robin/Joerg's opinion, I'd avoid being in a situation where
> "normally" this would work but then we have to quirk it.
>
> Is this a valid assumption _always_ ?

These assumptions were made based on the historic use cases I could
find reading the history.  There aren't many known examples "in the wild"
because so far we haven't had a mechanism other than quirks based
around device-tree implementations.

Ultimately I believe the proper solution will need to be another flag
in the RMR table that specifies the type of memory an RMR Node
describes, not just the base and length.

-Jon

>
> Thanks,
> Lorenzo
>
> > +
> > +             region = iommu_alloc_resv_region(addr, size, prot, type);
> > +             if (region) {
> > +                     region->fw_data.rmr.flags = rmr->flags;
> > +                     region->fw_data.rmr.sid = sid;
> > +                     region->fw_data.rmr.smmu = smmu;
> > +                     list_add_tail(&region->list, &iort_rmr_list);
> > +             }
> > +     }
> > +}
> > +
> > +static void __init iort_parse_rmr(void)
> > +{
> > +     struct acpi_iort_node *iort_node, *iort_end;
> > +     struct acpi_table_iort *iort;
> > +     int i;
> > +
> > +     if (iort_table->revision < 3)
> > +             return;
> > +
> > +     iort = (struct acpi_table_iort *)iort_table;
> > +
> > +     iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +                              iort->node_offset);
> > +     iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +                             iort_table->length);
> > +
> > +     for (i = 0; i < iort->node_count; i++) {
> > +             if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
> > +                            "IORT node pointer overflows, bad table!\n"))
> > +                     return;
> > +
> > +             if (iort_node->type == ACPI_IORT_NODE_RMR)
> > +                     iort_node_get_rmr_info(iort_node);
> > +
> > +             iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> > +                                      iort_node->length);
> > +     }
> > +}
> >
> >  static void __init iort_init_platform_devices(void)
> >  {
> > @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
> >       }
> >
> >       iort_init_platform_devices();
> > +     iort_parse_rmr();
> >  }
> >
> >  #ifdef CONFIG_ZONE_DMA
> > --
> > 2.17.1
> >

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-08-05 16:03     ` Lorenzo Pieralisi
  (?)
@ 2021-08-05 18:37       ` Laurentiu Tudor
  -1 siblings, 0 replies; 150+ messages in thread
From: Laurentiu Tudor @ 2021-08-05 18:37 UTC (permalink / raw)
  To: Lorenzo Pieralisi, Shameer Kolothum
  Cc: will, jon, linuxarm, steven.price, linux-acpi, iommu,
	wanghuiqiang, guohanjun, yangyicong, Sami.Mujawar, robin.murphy,
	linux-arm-kernel



On 8/5/2021 7:03 PM, Lorenzo Pieralisi wrote:
> On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:
> 
> [...]
> 
>> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
>> +{
>> +	struct acpi_iort_node *smmu;
>> +	struct acpi_iort_rmr *rmr;
>> +	struct acpi_iort_rmr_desc *rmr_desc;
>> +	u32 map_count = iort_node->mapping_count;
>> +	u32 sid;
>> +	int i;
>> +
>> +	if (!iort_node->mapping_offset || map_count != 1) {
>> +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	/* Retrieve associated smmu and stream id */
>> +	smmu = iort_node_get_id(iort_node, &sid, 0);
>> +	if (!smmu) {
>> +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	/* Retrieve RMR data */
>> +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
>> +	if (!rmr->rmr_offset || !rmr->rmr_count) {
>> +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
>> +				rmr->rmr_offset);
>> +
>> +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
>> +
>> +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
>> +		struct iommu_resv_region *region;
>> +		enum iommu_resv_type type;
>> +		int prot = IOMMU_READ | IOMMU_WRITE;
>> +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
>> +
>> +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
>> +			/* PAGE align base addr and size */
>> +			addr &= PAGE_MASK;
>> +			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
>> +
>> +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
>> +			       rmr_desc->base_address,
>> +			       rmr_desc->base_address + rmr_desc->length - 1,
>> +			       addr, addr + size - 1);
>> +		}
>> +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
>> +			type = IOMMU_RESV_DIRECT_RELAXABLE;
>> +			/*
>> +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
>> +			 * normally used for allocated system memory that is
>> +			 * then used for device specific reserved regions.
>> +			 */
>> +			prot |= IOMMU_CACHE;
>> +		} else {
>> +			type = IOMMU_RESV_DIRECT;
>> +			/*
>> +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
>> +			 * for device memory like MSI doorbell.
>> +			 */
>> +			prot |= IOMMU_MMIO;
>> +		}
> 
> On the prot value assignment based on the remapping flag, I'd like to
> hear Robin/Joerg's opinion, I'd avoid being in a situation where
> "normally" this would work but then we have to quirk it.
> 
> Is this a valid assumption _always_ ?

I think we enable quite a bit of platforms with this assumption, so IMHO
it's a fair compromise for now. As per Jon's comment and oob
discussions, in the long run the spec should probably be updated to
include a way of explicitly specifying memory attributes.

---
Thanks & Best Regards, Laurentiu

> 
>> +
>> +		region = iommu_alloc_resv_region(addr, size, prot, type);
>> +		if (region) {
>> +			region->fw_data.rmr.flags = rmr->flags;
>> +			region->fw_data.rmr.sid = sid;
>> +			region->fw_data.rmr.smmu = smmu;
>> +			list_add_tail(&region->list, &iort_rmr_list);
>> +		}
>> +	}
>> +}
>> +
>> +static void __init iort_parse_rmr(void)
>> +{
>> +	struct acpi_iort_node *iort_node, *iort_end;
>> +	struct acpi_table_iort *iort;
>> +	int i;
>> +
>> +	if (iort_table->revision < 3)
>> +		return;
>> +
>> +	iort = (struct acpi_table_iort *)iort_table;
>> +
>> +	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
>> +				 iort->node_offset);
>> +	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
>> +				iort_table->length);
>> +
>> +	for (i = 0; i < iort->node_count; i++) {
>> +		if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
>> +			       "IORT node pointer overflows, bad table!\n"))
>> +			return;
>> +
>> +		if (iort_node->type == ACPI_IORT_NODE_RMR)
>> +			iort_node_get_rmr_info(iort_node);
>> +
>> +		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
>> +					 iort_node->length);
>> +	}
>> +}
>>  
>>  static void __init iort_init_platform_devices(void)
>>  {
>> @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
>>  	}
>>  
>>  	iort_init_platform_devices();
>> +	iort_parse_rmr();
>>  }
>>  
>>  #ifdef CONFIG_ZONE_DMA
>> -- 
>> 2.17.1
>>
> _______________________________________________
> iommu mailing list
> iommu@lists.linux-foundation.org
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.linuxfoundation.org%2Fmailman%2Flistinfo%2Fiommu&amp;data=04%7C01%7Claurentiu.tudor%40nxp.com%7Cb020e5093dee4374ee0b08d9582a9238%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637637762131278563%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=8Q%2Fu5qawL94YhbKLujOAlJjTVEWZircjviccWnnqPxs%3D&amp;reserved=0
> 

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-08-05 18:37       ` Laurentiu Tudor
  0 siblings, 0 replies; 150+ messages in thread
From: Laurentiu Tudor @ 2021-08-05 18:37 UTC (permalink / raw)
  To: Lorenzo Pieralisi, Shameer Kolothum
  Cc: robin.murphy, jon, linuxarm, steven.price, linux-acpi, iommu,
	linux-arm-kernel, guohanjun, yangyicong, Sami.Mujawar, will,
	wanghuiqiang



On 8/5/2021 7:03 PM, Lorenzo Pieralisi wrote:
> On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:
> 
> [...]
> 
>> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
>> +{
>> +	struct acpi_iort_node *smmu;
>> +	struct acpi_iort_rmr *rmr;
>> +	struct acpi_iort_rmr_desc *rmr_desc;
>> +	u32 map_count = iort_node->mapping_count;
>> +	u32 sid;
>> +	int i;
>> +
>> +	if (!iort_node->mapping_offset || map_count != 1) {
>> +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	/* Retrieve associated smmu and stream id */
>> +	smmu = iort_node_get_id(iort_node, &sid, 0);
>> +	if (!smmu) {
>> +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	/* Retrieve RMR data */
>> +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
>> +	if (!rmr->rmr_offset || !rmr->rmr_count) {
>> +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
>> +				rmr->rmr_offset);
>> +
>> +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
>> +
>> +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
>> +		struct iommu_resv_region *region;
>> +		enum iommu_resv_type type;
>> +		int prot = IOMMU_READ | IOMMU_WRITE;
>> +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
>> +
>> +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
>> +			/* PAGE align base addr and size */
>> +			addr &= PAGE_MASK;
>> +			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
>> +
>> +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
>> +			       rmr_desc->base_address,
>> +			       rmr_desc->base_address + rmr_desc->length - 1,
>> +			       addr, addr + size - 1);
>> +		}
>> +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
>> +			type = IOMMU_RESV_DIRECT_RELAXABLE;
>> +			/*
>> +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
>> +			 * normally used for allocated system memory that is
>> +			 * then used for device specific reserved regions.
>> +			 */
>> +			prot |= IOMMU_CACHE;
>> +		} else {
>> +			type = IOMMU_RESV_DIRECT;
>> +			/*
>> +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
>> +			 * for device memory like MSI doorbell.
>> +			 */
>> +			prot |= IOMMU_MMIO;
>> +		}
> 
> On the prot value assignment based on the remapping flag, I'd like to
> hear Robin/Joerg's opinion, I'd avoid being in a situation where
> "normally" this would work but then we have to quirk it.
> 
> Is this a valid assumption _always_ ?

I think we enable quite a bit of platforms with this assumption, so IMHO
it's a fair compromise for now. As per Jon's comment and oob
discussions, in the long run the spec should probably be updated to
include a way of explicitly specifying memory attributes.

---
Thanks & Best Regards, Laurentiu

> 
>> +
>> +		region = iommu_alloc_resv_region(addr, size, prot, type);
>> +		if (region) {
>> +			region->fw_data.rmr.flags = rmr->flags;
>> +			region->fw_data.rmr.sid = sid;
>> +			region->fw_data.rmr.smmu = smmu;
>> +			list_add_tail(&region->list, &iort_rmr_list);
>> +		}
>> +	}
>> +}
>> +
>> +static void __init iort_parse_rmr(void)
>> +{
>> +	struct acpi_iort_node *iort_node, *iort_end;
>> +	struct acpi_table_iort *iort;
>> +	int i;
>> +
>> +	if (iort_table->revision < 3)
>> +		return;
>> +
>> +	iort = (struct acpi_table_iort *)iort_table;
>> +
>> +	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
>> +				 iort->node_offset);
>> +	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
>> +				iort_table->length);
>> +
>> +	for (i = 0; i < iort->node_count; i++) {
>> +		if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
>> +			       "IORT node pointer overflows, bad table!\n"))
>> +			return;
>> +
>> +		if (iort_node->type == ACPI_IORT_NODE_RMR)
>> +			iort_node_get_rmr_info(iort_node);
>> +
>> +		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
>> +					 iort_node->length);
>> +	}
>> +}
>>  
>>  static void __init iort_init_platform_devices(void)
>>  {
>> @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
>>  	}
>>  
>>  	iort_init_platform_devices();
>> +	iort_parse_rmr();
>>  }
>>  
>>  #ifdef CONFIG_ZONE_DMA
>> -- 
>> 2.17.1
>>
> _______________________________________________
> iommu mailing list
> iommu@lists.linux-foundation.org
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.linuxfoundation.org%2Fmailman%2Flistinfo%2Fiommu&amp;data=04%7C01%7Claurentiu.tudor%40nxp.com%7Cb020e5093dee4374ee0b08d9582a9238%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637637762131278563%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=8Q%2Fu5qawL94YhbKLujOAlJjTVEWZircjviccWnnqPxs%3D&amp;reserved=0
> 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-08-05 18:37       ` Laurentiu Tudor
  0 siblings, 0 replies; 150+ messages in thread
From: Laurentiu Tudor @ 2021-08-05 18:37 UTC (permalink / raw)
  To: Lorenzo Pieralisi, Shameer Kolothum
  Cc: will, jon, linuxarm, steven.price, linux-acpi, iommu,
	wanghuiqiang, guohanjun, yangyicong, Sami.Mujawar, robin.murphy,
	linux-arm-kernel



On 8/5/2021 7:03 PM, Lorenzo Pieralisi wrote:
> On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:
> 
> [...]
> 
>> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
>> +{
>> +	struct acpi_iort_node *smmu;
>> +	struct acpi_iort_rmr *rmr;
>> +	struct acpi_iort_rmr_desc *rmr_desc;
>> +	u32 map_count = iort_node->mapping_count;
>> +	u32 sid;
>> +	int i;
>> +
>> +	if (!iort_node->mapping_offset || map_count != 1) {
>> +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	/* Retrieve associated smmu and stream id */
>> +	smmu = iort_node_get_id(iort_node, &sid, 0);
>> +	if (!smmu) {
>> +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	/* Retrieve RMR data */
>> +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
>> +	if (!rmr->rmr_offset || !rmr->rmr_count) {
>> +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
>> +				rmr->rmr_offset);
>> +
>> +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
>> +
>> +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
>> +		struct iommu_resv_region *region;
>> +		enum iommu_resv_type type;
>> +		int prot = IOMMU_READ | IOMMU_WRITE;
>> +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
>> +
>> +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
>> +			/* PAGE align base addr and size */
>> +			addr &= PAGE_MASK;
>> +			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
>> +
>> +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
>> +			       rmr_desc->base_address,
>> +			       rmr_desc->base_address + rmr_desc->length - 1,
>> +			       addr, addr + size - 1);
>> +		}
>> +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
>> +			type = IOMMU_RESV_DIRECT_RELAXABLE;
>> +			/*
>> +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
>> +			 * normally used for allocated system memory that is
>> +			 * then used for device specific reserved regions.
>> +			 */
>> +			prot |= IOMMU_CACHE;
>> +		} else {
>> +			type = IOMMU_RESV_DIRECT;
>> +			/*
>> +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
>> +			 * for device memory like MSI doorbell.
>> +			 */
>> +			prot |= IOMMU_MMIO;
>> +		}
> 
> On the prot value assignment based on the remapping flag, I'd like to
> hear Robin/Joerg's opinion, I'd avoid being in a situation where
> "normally" this would work but then we have to quirk it.
> 
> Is this a valid assumption _always_ ?

I think we enable quite a bit of platforms with this assumption, so IMHO
it's a fair compromise for now. As per Jon's comment and oob
discussions, in the long run the spec should probably be updated to
include a way of explicitly specifying memory attributes.

---
Thanks & Best Regards, Laurentiu

> 
>> +
>> +		region = iommu_alloc_resv_region(addr, size, prot, type);
>> +		if (region) {
>> +			region->fw_data.rmr.flags = rmr->flags;
>> +			region->fw_data.rmr.sid = sid;
>> +			region->fw_data.rmr.smmu = smmu;
>> +			list_add_tail(&region->list, &iort_rmr_list);
>> +		}
>> +	}
>> +}
>> +
>> +static void __init iort_parse_rmr(void)
>> +{
>> +	struct acpi_iort_node *iort_node, *iort_end;
>> +	struct acpi_table_iort *iort;
>> +	int i;
>> +
>> +	if (iort_table->revision < 3)
>> +		return;
>> +
>> +	iort = (struct acpi_table_iort *)iort_table;
>> +
>> +	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
>> +				 iort->node_offset);
>> +	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
>> +				iort_table->length);
>> +
>> +	for (i = 0; i < iort->node_count; i++) {
>> +		if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
>> +			       "IORT node pointer overflows, bad table!\n"))
>> +			return;
>> +
>> +		if (iort_node->type == ACPI_IORT_NODE_RMR)
>> +			iort_node_get_rmr_info(iort_node);
>> +
>> +		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
>> +					 iort_node->length);
>> +	}
>> +}
>>  
>>  static void __init iort_init_platform_devices(void)
>>  {
>> @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
>>  	}
>>  
>>  	iort_init_platform_devices();
>> +	iort_parse_rmr();
>>  }
>>  
>>  #ifdef CONFIG_ZONE_DMA
>> -- 
>> 2.17.1
>>
> _______________________________________________
> iommu mailing list
> iommu@lists.linux-foundation.org
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.linuxfoundation.org%2Fmailman%2Flistinfo%2Fiommu&amp;data=04%7C01%7Claurentiu.tudor%40nxp.com%7Cb020e5093dee4374ee0b08d9582a9238%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637637762131278563%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=8Q%2Fu5qawL94YhbKLujOAlJjTVEWZircjviccWnnqPxs%3D&amp;reserved=0
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
  2021-08-05  8:07   ` Shameer Kolothum
  (?)
@ 2021-08-20 10:22     ` Steven Price
  -1 siblings, 0 replies; 150+ messages in thread
From: Steven Price @ 2021-08-20 10:22 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, Sami.Mujawar, jon, eric.auger,
	yangyicong

On 05/08/2021 09:07, Shameer Kolothum wrote:
> A union is introduced to struct iommu_resv_region to hold
> any firmware specific data. This is in preparation to add
> support for IORT RMR reserve regions and the union now holds
> the RMR specific information.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>  include/linux/iommu.h | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> index 32d448050bf7..bd0e4641c569 100644
> --- a/include/linux/iommu.h
> +++ b/include/linux/iommu.h
> @@ -114,6 +114,13 @@ enum iommu_resv_type {
>  	IOMMU_RESV_SW_MSI,
>  };
>  
> +struct iommu_iort_rmr_data {
> +#define IOMMU_RMR_REMAP_PERMITTED	(1 << 0)
> +	u32 flags;
> +	u32 sid;	/* Stream Id associated with RMR entry */
> +	void *smmu;	/* Associated IORT SMMU node pointer */
> +};
> +
>  /**
>   * struct iommu_resv_region - descriptor for a reserved memory region
>   * @list: Linked list pointers
> @@ -121,6 +128,7 @@ enum iommu_resv_type {
>   * @length: Length of the region in bytes
>   * @prot: IOMMU Protection flags (READ/WRITE/...)
>   * @type: Type of the reserved region
> + * @rmr: ACPI IORT RMR specific data

NIT: This will provoke a kernel-doc warning as the field name in the
structure is 'fw_data' not 'rmr' ('rmr being a field of the anonymous
union).

I've also retested this series on my Juno setup, so feel free to add:

Tested-by: Steven Price <steven.price@arm.com>

Thanks,

Steve

>   */
>  struct iommu_resv_region {
>  	struct list_head	list;
> @@ -128,6 +136,9 @@ struct iommu_resv_region {
>  	size_t			length;
>  	int			prot;
>  	enum iommu_resv_type	type;
> +	union {
> +		struct iommu_iort_rmr_data rmr;
> +	} fw_data;
>  };
>  
>  /**
> 


^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
@ 2021-08-20 10:22     ` Steven Price
  0 siblings, 0 replies; 150+ messages in thread
From: Steven Price @ 2021-08-20 10:22 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: robin.murphy, jon, linuxarm, guohanjun, yangyicong, Sami.Mujawar,
	will, wanghuiqiang

On 05/08/2021 09:07, Shameer Kolothum wrote:
> A union is introduced to struct iommu_resv_region to hold
> any firmware specific data. This is in preparation to add
> support for IORT RMR reserve regions and the union now holds
> the RMR specific information.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>  include/linux/iommu.h | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> index 32d448050bf7..bd0e4641c569 100644
> --- a/include/linux/iommu.h
> +++ b/include/linux/iommu.h
> @@ -114,6 +114,13 @@ enum iommu_resv_type {
>  	IOMMU_RESV_SW_MSI,
>  };
>  
> +struct iommu_iort_rmr_data {
> +#define IOMMU_RMR_REMAP_PERMITTED	(1 << 0)
> +	u32 flags;
> +	u32 sid;	/* Stream Id associated with RMR entry */
> +	void *smmu;	/* Associated IORT SMMU node pointer */
> +};
> +
>  /**
>   * struct iommu_resv_region - descriptor for a reserved memory region
>   * @list: Linked list pointers
> @@ -121,6 +128,7 @@ enum iommu_resv_type {
>   * @length: Length of the region in bytes
>   * @prot: IOMMU Protection flags (READ/WRITE/...)
>   * @type: Type of the reserved region
> + * @rmr: ACPI IORT RMR specific data

NIT: This will provoke a kernel-doc warning as the field name in the
structure is 'fw_data' not 'rmr' ('rmr being a field of the anonymous
union).

I've also retested this series on my Juno setup, so feel free to add:

Tested-by: Steven Price <steven.price@arm.com>

Thanks,

Steve

>   */
>  struct iommu_resv_region {
>  	struct list_head	list;
> @@ -128,6 +136,9 @@ struct iommu_resv_region {
>  	size_t			length;
>  	int			prot;
>  	enum iommu_resv_type	type;
> +	union {
> +		struct iommu_iort_rmr_data rmr;
> +	} fw_data;
>  };
>  
>  /**
> 

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
@ 2021-08-20 10:22     ` Steven Price
  0 siblings, 0 replies; 150+ messages in thread
From: Steven Price @ 2021-08-20 10:22 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, Sami.Mujawar, jon, eric.auger,
	yangyicong

On 05/08/2021 09:07, Shameer Kolothum wrote:
> A union is introduced to struct iommu_resv_region to hold
> any firmware specific data. This is in preparation to add
> support for IORT RMR reserve regions and the union now holds
> the RMR specific information.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>  include/linux/iommu.h | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> index 32d448050bf7..bd0e4641c569 100644
> --- a/include/linux/iommu.h
> +++ b/include/linux/iommu.h
> @@ -114,6 +114,13 @@ enum iommu_resv_type {
>  	IOMMU_RESV_SW_MSI,
>  };
>  
> +struct iommu_iort_rmr_data {
> +#define IOMMU_RMR_REMAP_PERMITTED	(1 << 0)
> +	u32 flags;
> +	u32 sid;	/* Stream Id associated with RMR entry */
> +	void *smmu;	/* Associated IORT SMMU node pointer */
> +};
> +
>  /**
>   * struct iommu_resv_region - descriptor for a reserved memory region
>   * @list: Linked list pointers
> @@ -121,6 +128,7 @@ enum iommu_resv_type {
>   * @length: Length of the region in bytes
>   * @prot: IOMMU Protection flags (READ/WRITE/...)
>   * @type: Type of the reserved region
> + * @rmr: ACPI IORT RMR specific data

NIT: This will provoke a kernel-doc warning as the field name in the
structure is 'fw_data' not 'rmr' ('rmr being a field of the anonymous
union).

I've also retested this series on my Juno setup, so feel free to add:

Tested-by: Steven Price <steven.price@arm.com>

Thanks,

Steve

>   */
>  struct iommu_resv_region {
>  	struct list_head	list;
> @@ -128,6 +136,9 @@ struct iommu_resv_region {
>  	size_t			length;
>  	int			prot;
>  	enum iommu_resv_type	type;
> +	union {
> +		struct iommu_iort_rmr_data rmr;
> +	} fw_data;
>  };
>  
>  /**
> 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
  2021-08-05 14:09       ` Ard Biesheuvel
  (?)
@ 2021-08-31  5:06         ` Jon Nettleton
  -1 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-08-31  5:06 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Shameerali Kolothum Thodi, Linux ARM, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Lorenzo Pieralisi, Joerg Roedel,
	Robin Murphy, Will Deacon, wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong

On Thu, Aug 5, 2021 at 4:09 PM Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Thu, 5 Aug 2021 at 15:35, Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Ard Biesheuvel [mailto:ardb@kernel.org]
> > > Sent: 05 August 2021 14:23
> > > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> > > Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > > Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel
> > > <joro@8bytes.org>; Robin Murphy <robin.murphy@arm.com>; Will Deacon
> > > <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> > > (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> > > <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Jon
> > > Nettleton <jon@solid-run.com>; Eric Auger <eric.auger@redhat.com>;
> > > yangyicong <yangyicong@huawei.com>
> > > Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> > >
> > > On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum
> > > <shameerali.kolothum.thodi@huawei.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > The series adds support to IORT RMR nodes specified in IORT
> > > > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> > > > memory ranges that are used by endpoints and require a unity
> > > > mapping in SMMU.
> > > >
> > > > We have faced issues with 3408iMR RAID controller cards which
> > > > fail to boot when SMMU is enabled. This is because these
> > > > controllers make use of host memory for various caching related
> > > > purposes and when SMMU is enabled the iMR firmware fails to
> > > > access these memory regions as there is no mapping for them.
> > > > IORT RMR provides a way for UEFI to describe and report these
> > > > memory regions so that the kernel can make a unity mapping for
> > > > these in SMMU.
> > > >
> > >
> > > Does this mean we are ignoring the RMR memory ranges, and exposing the
> > > entire physical address space to devices using the stream IDs in
> > > question?
> >
> > Nope. RMR node is used to describe the memory ranges used by end points
> > behind SMMU. And this information is used to create 1 : 1 mappings for those
> > ranges in SMMU. Anything outside those ranges will result in translation
> > fault(if there are no other dynamic DMA mappings).
> >
>
> Excellent! It was not obvious to me from looking at the patches, so I
> had to ask.
>
> Thanks,
> Ard.
>
> >
> > >
> > > > Change History:
> > > >
> > > > v6 --> v7
> > > >
> > > > The only change from v6 is the fix pointed out by Steve to
> > > > the SMMUv2 SMR bypass install in patch #8.
> > > >
> > > > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> > > > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> > > > yet as the series still needs more review[1].
> > > >
> > > > Feedback and tests on this series is very much appreciated.
> > > >
> > > > v5 --> v6
> > > > - Addressed comments from Robin & Lorenzo.
> > > >   : Moved iort_parse_rmr() to acpi_iort_init() from
> > > >     iort_init_platform_devices().
> > > >   : Removed use of struct iort_rmr_entry during the initial
> > > >     parse. Using struct iommu_resv_region instead.
> > > >   : Report RMR address alignment and overlap errors, but continue.
> > > >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> > > > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> > > > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
> > > >   on Type of RMR region. Suggested by Jon N.
> > > >
> > > > Thanks,
> > > > Shameer
> > > > [0] https://developer.arm.com/documentation/den0049/latest/
> > > > [1]
> > > https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth
> > > um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> > > > ------
> > > > v4 --> v5
> > > >  -Added a fw_data union to struct iommu_resv_region and removed
> > > >   struct iommu_rmr (Based on comments from Joerg/Robin).
> > > >  -Added iommu_put_rmrs() to release mem.
> > > >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
> > > >   yet because of the above changes.
> > > >
> > > > v3 -->v4
> > > > -Included the SMMUv2 SMR bypass install changes suggested by
> > > >  Steve(patch #7)
> > > > -As per Robin's comments, RMR reserve implementation is now
> > > >  more generic  (patch #8) and dropped v3 patches 8 and 10.
> > > > -Rebase to 5.13-rc1
> > > >
> > > > RFC v2 --> v3
> > > >  -Dropped RFC tag as the ACPICA header changes are now ready to be
> > > >   part of 5.13[0]. But this series still has a dependency on that patch.
> > > >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
> > > >   PCIe).
> > > >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
> > > >   discussion here[1].
> > > >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> > > > ------
> > > >
> > > > Jon Nettleton (1):
> > > >   iommu/arm-smmu: Get associated RMR info and install bypass SMR
> > > >
> > > > Shameer Kolothum (8):
> > > >   iommu: Introduce a union to struct iommu_resv_region
> > > >   ACPI/IORT: Add support for RMR node parsing
> > > >   iommu/dma: Introduce generic helper to retrieve RMR info
> > > >   ACPI/IORT: Add a helper to retrieve RMR memory regions
> > > >   iommu/arm-smmu-v3: Introduce strtab init helper
> > > >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
> > > >     bypass
> > > >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> > > >   iommu/dma: Reserve any RMR regions associated with a dev
> > > >
> > > >  drivers/acpi/arm64/iort.c                   | 172
> > > +++++++++++++++++++-
> > > >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
> > > >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
> > > >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
> > > >  include/linux/acpi_iort.h                   |   7 +
> > > >  include/linux/dma-iommu.h                   |  13 ++
> > > >  include/linux/iommu.h                       |  11 ++
> > > >  7 files changed, 393 insertions(+), 23 deletions(-)
> > > >
> > > > --
> > > > 2.17.1
> > > >
> > > >
> > > > _______________________________________________
> > > > linux-arm-kernel mailing list
> > > > linux-arm-kernel@lists.infradead.org
> > > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Ping,  Can we get some movement on this patchset?  The standard was
was ratified over a year ago, and there is real world hardware that is
using or needs this functionality.

Thanks,
-Jon

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-08-31  5:06         ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-08-31  5:06 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Will Deacon, Linuxarm, Steven Price, ACPI Devel Maling List,
	Linux IOMMU, wanghuiqiang, Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Robin Murphy, Linux ARM

On Thu, Aug 5, 2021 at 4:09 PM Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Thu, 5 Aug 2021 at 15:35, Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Ard Biesheuvel [mailto:ardb@kernel.org]
> > > Sent: 05 August 2021 14:23
> > > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> > > Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > > Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel
> > > <joro@8bytes.org>; Robin Murphy <robin.murphy@arm.com>; Will Deacon
> > > <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> > > (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> > > <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Jon
> > > Nettleton <jon@solid-run.com>; Eric Auger <eric.auger@redhat.com>;
> > > yangyicong <yangyicong@huawei.com>
> > > Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> > >
> > > On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum
> > > <shameerali.kolothum.thodi@huawei.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > The series adds support to IORT RMR nodes specified in IORT
> > > > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> > > > memory ranges that are used by endpoints and require a unity
> > > > mapping in SMMU.
> > > >
> > > > We have faced issues with 3408iMR RAID controller cards which
> > > > fail to boot when SMMU is enabled. This is because these
> > > > controllers make use of host memory for various caching related
> > > > purposes and when SMMU is enabled the iMR firmware fails to
> > > > access these memory regions as there is no mapping for them.
> > > > IORT RMR provides a way for UEFI to describe and report these
> > > > memory regions so that the kernel can make a unity mapping for
> > > > these in SMMU.
> > > >
> > >
> > > Does this mean we are ignoring the RMR memory ranges, and exposing the
> > > entire physical address space to devices using the stream IDs in
> > > question?
> >
> > Nope. RMR node is used to describe the memory ranges used by end points
> > behind SMMU. And this information is used to create 1 : 1 mappings for those
> > ranges in SMMU. Anything outside those ranges will result in translation
> > fault(if there are no other dynamic DMA mappings).
> >
>
> Excellent! It was not obvious to me from looking at the patches, so I
> had to ask.
>
> Thanks,
> Ard.
>
> >
> > >
> > > > Change History:
> > > >
> > > > v6 --> v7
> > > >
> > > > The only change from v6 is the fix pointed out by Steve to
> > > > the SMMUv2 SMR bypass install in patch #8.
> > > >
> > > > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> > > > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> > > > yet as the series still needs more review[1].
> > > >
> > > > Feedback and tests on this series is very much appreciated.
> > > >
> > > > v5 --> v6
> > > > - Addressed comments from Robin & Lorenzo.
> > > >   : Moved iort_parse_rmr() to acpi_iort_init() from
> > > >     iort_init_platform_devices().
> > > >   : Removed use of struct iort_rmr_entry during the initial
> > > >     parse. Using struct iommu_resv_region instead.
> > > >   : Report RMR address alignment and overlap errors, but continue.
> > > >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> > > > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> > > > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
> > > >   on Type of RMR region. Suggested by Jon N.
> > > >
> > > > Thanks,
> > > > Shameer
> > > > [0] https://developer.arm.com/documentation/den0049/latest/
> > > > [1]
> > > https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth
> > > um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> > > > ------
> > > > v4 --> v5
> > > >  -Added a fw_data union to struct iommu_resv_region and removed
> > > >   struct iommu_rmr (Based on comments from Joerg/Robin).
> > > >  -Added iommu_put_rmrs() to release mem.
> > > >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
> > > >   yet because of the above changes.
> > > >
> > > > v3 -->v4
> > > > -Included the SMMUv2 SMR bypass install changes suggested by
> > > >  Steve(patch #7)
> > > > -As per Robin's comments, RMR reserve implementation is now
> > > >  more generic  (patch #8) and dropped v3 patches 8 and 10.
> > > > -Rebase to 5.13-rc1
> > > >
> > > > RFC v2 --> v3
> > > >  -Dropped RFC tag as the ACPICA header changes are now ready to be
> > > >   part of 5.13[0]. But this series still has a dependency on that patch.
> > > >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
> > > >   PCIe).
> > > >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
> > > >   discussion here[1].
> > > >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> > > > ------
> > > >
> > > > Jon Nettleton (1):
> > > >   iommu/arm-smmu: Get associated RMR info and install bypass SMR
> > > >
> > > > Shameer Kolothum (8):
> > > >   iommu: Introduce a union to struct iommu_resv_region
> > > >   ACPI/IORT: Add support for RMR node parsing
> > > >   iommu/dma: Introduce generic helper to retrieve RMR info
> > > >   ACPI/IORT: Add a helper to retrieve RMR memory regions
> > > >   iommu/arm-smmu-v3: Introduce strtab init helper
> > > >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
> > > >     bypass
> > > >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> > > >   iommu/dma: Reserve any RMR regions associated with a dev
> > > >
> > > >  drivers/acpi/arm64/iort.c                   | 172
> > > +++++++++++++++++++-
> > > >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
> > > >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
> > > >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
> > > >  include/linux/acpi_iort.h                   |   7 +
> > > >  include/linux/dma-iommu.h                   |  13 ++
> > > >  include/linux/iommu.h                       |  11 ++
> > > >  7 files changed, 393 insertions(+), 23 deletions(-)
> > > >
> > > > --
> > > > 2.17.1
> > > >
> > > >
> > > > _______________________________________________
> > > > linux-arm-kernel mailing list
> > > > linux-arm-kernel@lists.infradead.org
> > > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Ping,  Can we get some movement on this patchset?  The standard was
was ratified over a year ago, and there is real world hardware that is
using or needs this functionality.

Thanks,
-Jon
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-08-31  5:06         ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-08-31  5:06 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Shameerali Kolothum Thodi, Linux ARM, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Lorenzo Pieralisi, Joerg Roedel,
	Robin Murphy, Will Deacon, wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong

On Thu, Aug 5, 2021 at 4:09 PM Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Thu, 5 Aug 2021 at 15:35, Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Ard Biesheuvel [mailto:ardb@kernel.org]
> > > Sent: 05 August 2021 14:23
> > > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> > > Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > > Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel
> > > <joro@8bytes.org>; Robin Murphy <robin.murphy@arm.com>; Will Deacon
> > > <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> > > (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> > > <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Jon
> > > Nettleton <jon@solid-run.com>; Eric Auger <eric.auger@redhat.com>;
> > > yangyicong <yangyicong@huawei.com>
> > > Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> > >
> > > On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum
> > > <shameerali.kolothum.thodi@huawei.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > The series adds support to IORT RMR nodes specified in IORT
> > > > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> > > > memory ranges that are used by endpoints and require a unity
> > > > mapping in SMMU.
> > > >
> > > > We have faced issues with 3408iMR RAID controller cards which
> > > > fail to boot when SMMU is enabled. This is because these
> > > > controllers make use of host memory for various caching related
> > > > purposes and when SMMU is enabled the iMR firmware fails to
> > > > access these memory regions as there is no mapping for them.
> > > > IORT RMR provides a way for UEFI to describe and report these
> > > > memory regions so that the kernel can make a unity mapping for
> > > > these in SMMU.
> > > >
> > >
> > > Does this mean we are ignoring the RMR memory ranges, and exposing the
> > > entire physical address space to devices using the stream IDs in
> > > question?
> >
> > Nope. RMR node is used to describe the memory ranges used by end points
> > behind SMMU. And this information is used to create 1 : 1 mappings for those
> > ranges in SMMU. Anything outside those ranges will result in translation
> > fault(if there are no other dynamic DMA mappings).
> >
>
> Excellent! It was not obvious to me from looking at the patches, so I
> had to ask.
>
> Thanks,
> Ard.
>
> >
> > >
> > > > Change History:
> > > >
> > > > v6 --> v7
> > > >
> > > > The only change from v6 is the fix pointed out by Steve to
> > > > the SMMUv2 SMR bypass install in patch #8.
> > > >
> > > > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> > > > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> > > > yet as the series still needs more review[1].
> > > >
> > > > Feedback and tests on this series is very much appreciated.
> > > >
> > > > v5 --> v6
> > > > - Addressed comments from Robin & Lorenzo.
> > > >   : Moved iort_parse_rmr() to acpi_iort_init() from
> > > >     iort_init_platform_devices().
> > > >   : Removed use of struct iort_rmr_entry during the initial
> > > >     parse. Using struct iommu_resv_region instead.
> > > >   : Report RMR address alignment and overlap errors, but continue.
> > > >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> > > > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> > > > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
> > > >   on Type of RMR region. Suggested by Jon N.
> > > >
> > > > Thanks,
> > > > Shameer
> > > > [0] https://developer.arm.com/documentation/den0049/latest/
> > > > [1]
> > > https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth
> > > um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> > > > ------
> > > > v4 --> v5
> > > >  -Added a fw_data union to struct iommu_resv_region and removed
> > > >   struct iommu_rmr (Based on comments from Joerg/Robin).
> > > >  -Added iommu_put_rmrs() to release mem.
> > > >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
> > > >   yet because of the above changes.
> > > >
> > > > v3 -->v4
> > > > -Included the SMMUv2 SMR bypass install changes suggested by
> > > >  Steve(patch #7)
> > > > -As per Robin's comments, RMR reserve implementation is now
> > > >  more generic  (patch #8) and dropped v3 patches 8 and 10.
> > > > -Rebase to 5.13-rc1
> > > >
> > > > RFC v2 --> v3
> > > >  -Dropped RFC tag as the ACPICA header changes are now ready to be
> > > >   part of 5.13[0]. But this series still has a dependency on that patch.
> > > >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
> > > >   PCIe).
> > > >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
> > > >   discussion here[1].
> > > >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> > > > ------
> > > >
> > > > Jon Nettleton (1):
> > > >   iommu/arm-smmu: Get associated RMR info and install bypass SMR
> > > >
> > > > Shameer Kolothum (8):
> > > >   iommu: Introduce a union to struct iommu_resv_region
> > > >   ACPI/IORT: Add support for RMR node parsing
> > > >   iommu/dma: Introduce generic helper to retrieve RMR info
> > > >   ACPI/IORT: Add a helper to retrieve RMR memory regions
> > > >   iommu/arm-smmu-v3: Introduce strtab init helper
> > > >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
> > > >     bypass
> > > >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> > > >   iommu/dma: Reserve any RMR regions associated with a dev
> > > >
> > > >  drivers/acpi/arm64/iort.c                   | 172
> > > +++++++++++++++++++-
> > > >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
> > > >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
> > > >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
> > > >  include/linux/acpi_iort.h                   |   7 +
> > > >  include/linux/dma-iommu.h                   |  13 ++
> > > >  include/linux/iommu.h                       |  11 ++
> > > >  7 files changed, 393 insertions(+), 23 deletions(-)
> > > >
> > > > --
> > > > 2.17.1
> > > >
> > > >
> > > > _______________________________________________
> > > > linux-arm-kernel mailing list
> > > > linux-arm-kernel@lists.infradead.org
> > > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Ping,  Can we get some movement on this patchset?  The standard was
was ratified over a year ago, and there is real world hardware that is
using or needs this functionality.

Thanks,
-Jon

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-08-05 16:03     ` Lorenzo Pieralisi
  (?)
@ 2021-09-06 17:44       ` Robin Murphy
  -1 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-09-06 17:44 UTC (permalink / raw)
  To: Lorenzo Pieralisi, Shameer Kolothum, jon, Laurentiu Tudor
  Cc: linux-arm-kernel, linux-acpi, iommu, linuxarm, joro, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, eric.auger,
	yangyicong

On 2021-08-05 17:03, Lorenzo Pieralisi wrote:
> On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:
> 
> [...]
> 
>> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
>> +{
>> +	struct acpi_iort_node *smmu;
>> +	struct acpi_iort_rmr *rmr;
>> +	struct acpi_iort_rmr_desc *rmr_desc;
>> +	u32 map_count = iort_node->mapping_count;
>> +	u32 sid;
>> +	int i;
>> +
>> +	if (!iort_node->mapping_offset || map_count != 1) {
>> +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	/* Retrieve associated smmu and stream id */
>> +	smmu = iort_node_get_id(iort_node, &sid, 0);
>> +	if (!smmu) {
>> +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	/* Retrieve RMR data */
>> +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
>> +	if (!rmr->rmr_offset || !rmr->rmr_count) {
>> +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
>> +				rmr->rmr_offset);
>> +
>> +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
>> +
>> +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
>> +		struct iommu_resv_region *region;
>> +		enum iommu_resv_type type;
>> +		int prot = IOMMU_READ | IOMMU_WRITE;
>> +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
>> +
>> +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
>> +			/* PAGE align base addr and size */
>> +			addr &= PAGE_MASK;
>> +			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
>> +
>> +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
>> +			       rmr_desc->base_address,
>> +			       rmr_desc->base_address + rmr_desc->length - 1,
>> +			       addr, addr + size - 1);
>> +		}
>> +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
>> +			type = IOMMU_RESV_DIRECT_RELAXABLE;
>> +			/*
>> +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
>> +			 * normally used for allocated system memory that is
>> +			 * then used for device specific reserved regions.
>> +			 */
>> +			prot |= IOMMU_CACHE;
>> +		} else {
>> +			type = IOMMU_RESV_DIRECT;
>> +			/*
>> +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
>> +			 * for device memory like MSI doorbell.
>> +			 */
>> +			prot |= IOMMU_MMIO;
>> +		}
> 
> On the prot value assignment based on the remapping flag, I'd like to
> hear Robin/Joerg's opinion, I'd avoid being in a situation where
> "normally" this would work but then we have to quirk it.
> 
> Is this a valid assumption _always_ ?

No. Certainly applying IOMMU_CACHE without reference to the device's 
_CCA attribute or how CPUs may be accessing a shared buffer could lead 
to a loss of coherency. At worst, applying IOMMU_MMIO to a 
device-private buffer *could* cause the device to lose coherency with 
itself if the memory underlying the RMR may have allocated into system 
caches. Note that the expected use for non-remappable RMRs is the device 
holding some sort of long-lived private data in system RAM - the MSI 
doorbell trick is far more of a niche hack really.

At the very least I think we need to refer to the device's memory access 
properties here.

Jon, Laurentiu - how do RMRs correspond to the EFI memory map on your 
firmware? I'm starting to think that as long as the underlying memory is 
described appropriately there then we should be able to infer correct 
attributes from the EFI memory type and flags.

Robin.

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-06 17:44       ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-09-06 17:44 UTC (permalink / raw)
  To: Lorenzo Pieralisi, Shameer Kolothum, jon, Laurentiu Tudor
  Cc: linuxarm, steven.price, linux-acpi, iommu, wanghuiqiang,
	guohanjun, yangyicong, Sami.Mujawar, will, linux-arm-kernel

On 2021-08-05 17:03, Lorenzo Pieralisi wrote:
> On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:
> 
> [...]
> 
>> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
>> +{
>> +	struct acpi_iort_node *smmu;
>> +	struct acpi_iort_rmr *rmr;
>> +	struct acpi_iort_rmr_desc *rmr_desc;
>> +	u32 map_count = iort_node->mapping_count;
>> +	u32 sid;
>> +	int i;
>> +
>> +	if (!iort_node->mapping_offset || map_count != 1) {
>> +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	/* Retrieve associated smmu and stream id */
>> +	smmu = iort_node_get_id(iort_node, &sid, 0);
>> +	if (!smmu) {
>> +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	/* Retrieve RMR data */
>> +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
>> +	if (!rmr->rmr_offset || !rmr->rmr_count) {
>> +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
>> +				rmr->rmr_offset);
>> +
>> +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
>> +
>> +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
>> +		struct iommu_resv_region *region;
>> +		enum iommu_resv_type type;
>> +		int prot = IOMMU_READ | IOMMU_WRITE;
>> +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
>> +
>> +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
>> +			/* PAGE align base addr and size */
>> +			addr &= PAGE_MASK;
>> +			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
>> +
>> +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
>> +			       rmr_desc->base_address,
>> +			       rmr_desc->base_address + rmr_desc->length - 1,
>> +			       addr, addr + size - 1);
>> +		}
>> +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
>> +			type = IOMMU_RESV_DIRECT_RELAXABLE;
>> +			/*
>> +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
>> +			 * normally used for allocated system memory that is
>> +			 * then used for device specific reserved regions.
>> +			 */
>> +			prot |= IOMMU_CACHE;
>> +		} else {
>> +			type = IOMMU_RESV_DIRECT;
>> +			/*
>> +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
>> +			 * for device memory like MSI doorbell.
>> +			 */
>> +			prot |= IOMMU_MMIO;
>> +		}
> 
> On the prot value assignment based on the remapping flag, I'd like to
> hear Robin/Joerg's opinion, I'd avoid being in a situation where
> "normally" this would work but then we have to quirk it.
> 
> Is this a valid assumption _always_ ?

No. Certainly applying IOMMU_CACHE without reference to the device's 
_CCA attribute or how CPUs may be accessing a shared buffer could lead 
to a loss of coherency. At worst, applying IOMMU_MMIO to a 
device-private buffer *could* cause the device to lose coherency with 
itself if the memory underlying the RMR may have allocated into system 
caches. Note that the expected use for non-remappable RMRs is the device 
holding some sort of long-lived private data in system RAM - the MSI 
doorbell trick is far more of a niche hack really.

At the very least I think we need to refer to the device's memory access 
properties here.

Jon, Laurentiu - how do RMRs correspond to the EFI memory map on your 
firmware? I'm starting to think that as long as the underlying memory is 
described appropriately there then we should be able to infer correct 
attributes from the EFI memory type and flags.

Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-06 17:44       ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-09-06 17:44 UTC (permalink / raw)
  To: Lorenzo Pieralisi, Shameer Kolothum, jon, Laurentiu Tudor
  Cc: linux-arm-kernel, linux-acpi, iommu, linuxarm, joro, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, eric.auger,
	yangyicong

On 2021-08-05 17:03, Lorenzo Pieralisi wrote:
> On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:
> 
> [...]
> 
>> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
>> +{
>> +	struct acpi_iort_node *smmu;
>> +	struct acpi_iort_rmr *rmr;
>> +	struct acpi_iort_rmr_desc *rmr_desc;
>> +	u32 map_count = iort_node->mapping_count;
>> +	u32 sid;
>> +	int i;
>> +
>> +	if (!iort_node->mapping_offset || map_count != 1) {
>> +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	/* Retrieve associated smmu and stream id */
>> +	smmu = iort_node_get_id(iort_node, &sid, 0);
>> +	if (!smmu) {
>> +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	/* Retrieve RMR data */
>> +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
>> +	if (!rmr->rmr_offset || !rmr->rmr_count) {
>> +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
>> +		       iort_node);
>> +		return;
>> +	}
>> +
>> +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
>> +				rmr->rmr_offset);
>> +
>> +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
>> +
>> +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
>> +		struct iommu_resv_region *region;
>> +		enum iommu_resv_type type;
>> +		int prot = IOMMU_READ | IOMMU_WRITE;
>> +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
>> +
>> +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
>> +			/* PAGE align base addr and size */
>> +			addr &= PAGE_MASK;
>> +			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
>> +
>> +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
>> +			       rmr_desc->base_address,
>> +			       rmr_desc->base_address + rmr_desc->length - 1,
>> +			       addr, addr + size - 1);
>> +		}
>> +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
>> +			type = IOMMU_RESV_DIRECT_RELAXABLE;
>> +			/*
>> +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
>> +			 * normally used for allocated system memory that is
>> +			 * then used for device specific reserved regions.
>> +			 */
>> +			prot |= IOMMU_CACHE;
>> +		} else {
>> +			type = IOMMU_RESV_DIRECT;
>> +			/*
>> +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
>> +			 * for device memory like MSI doorbell.
>> +			 */
>> +			prot |= IOMMU_MMIO;
>> +		}
> 
> On the prot value assignment based on the remapping flag, I'd like to
> hear Robin/Joerg's opinion, I'd avoid being in a situation where
> "normally" this would work but then we have to quirk it.
> 
> Is this a valid assumption _always_ ?

No. Certainly applying IOMMU_CACHE without reference to the device's 
_CCA attribute or how CPUs may be accessing a shared buffer could lead 
to a loss of coherency. At worst, applying IOMMU_MMIO to a 
device-private buffer *could* cause the device to lose coherency with 
itself if the memory underlying the RMR may have allocated into system 
caches. Note that the expected use for non-remappable RMRs is the device 
holding some sort of long-lived private data in system RAM - the MSI 
doorbell trick is far more of a niche hack really.

At the very least I think we need to refer to the device's memory access 
properties here.

Jon, Laurentiu - how do RMRs correspond to the EFI memory map on your 
firmware? I'm starting to think that as long as the underlying memory is 
described appropriately there then we should be able to infer correct 
attributes from the EFI memory type and flags.

Robin.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-09-06 17:44       ` Robin Murphy
  (?)
@ 2021-09-06 19:51         ` Jon Nettleton
  -1 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-09-06 19:51 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Lorenzo Pieralisi, Shameer Kolothum, Laurentiu Tudor,
	linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Joerg Roedel, Will Deacon, wanghuiqiang, Hanjun Guo,
	Steven Price, Sami Mujawar, Eric Auger, yangyicong

On Mon, Sep 6, 2021 at 7:44 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-08-05 17:03, Lorenzo Pieralisi wrote:
> > On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:
> >
> > [...]
> >
> >> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> >> +{
> >> +    struct acpi_iort_node *smmu;
> >> +    struct acpi_iort_rmr *rmr;
> >> +    struct acpi_iort_rmr_desc *rmr_desc;
> >> +    u32 map_count = iort_node->mapping_count;
> >> +    u32 sid;
> >> +    int i;
> >> +
> >> +    if (!iort_node->mapping_offset || map_count != 1) {
> >> +            pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> >> +                   iort_node);
> >> +            return;
> >> +    }
> >> +
> >> +    /* Retrieve associated smmu and stream id */
> >> +    smmu = iort_node_get_id(iort_node, &sid, 0);
> >> +    if (!smmu) {
> >> +            pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> >> +                   iort_node);
> >> +            return;
> >> +    }
> >> +
> >> +    /* Retrieve RMR data */
> >> +    rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> >> +    if (!rmr->rmr_offset || !rmr->rmr_count) {
> >> +            pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> >> +                   iort_node);
> >> +            return;
> >> +    }
> >> +
> >> +    rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> >> +                            rmr->rmr_offset);
> >> +
> >> +    iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> >> +
> >> +    for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> >> +            struct iommu_resv_region *region;
> >> +            enum iommu_resv_type type;
> >> +            int prot = IOMMU_READ | IOMMU_WRITE;
> >> +            u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> >> +
> >> +            if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> >> +                    /* PAGE align base addr and size */
> >> +                    addr &= PAGE_MASK;
> >> +                    size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> >> +
> >> +                    pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> >> +                           rmr_desc->base_address,
> >> +                           rmr_desc->base_address + rmr_desc->length - 1,
> >> +                           addr, addr + size - 1);
> >> +            }
> >> +            if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> >> +                    type = IOMMU_RESV_DIRECT_RELAXABLE;
> >> +                    /*
> >> +                     * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> >> +                     * normally used for allocated system memory that is
> >> +                     * then used for device specific reserved regions.
> >> +                     */
> >> +                    prot |= IOMMU_CACHE;
> >> +            } else {
> >> +                    type = IOMMU_RESV_DIRECT;
> >> +                    /*
> >> +                     * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> >> +                     * for device memory like MSI doorbell.
> >> +                     */
> >> +                    prot |= IOMMU_MMIO;
> >> +            }
> >
> > On the prot value assignment based on the remapping flag, I'd like to
> > hear Robin/Joerg's opinion, I'd avoid being in a situation where
> > "normally" this would work but then we have to quirk it.
> >
> > Is this a valid assumption _always_ ?
>
> No. Certainly applying IOMMU_CACHE without reference to the device's
> _CCA attribute or how CPUs may be accessing a shared buffer could lead
> to a loss of coherency. At worst, applying IOMMU_MMIO to a
> device-private buffer *could* cause the device to lose coherency with
> itself if the memory underlying the RMR may have allocated into system
> caches. Note that the expected use for non-remappable RMRs is the device
> holding some sort of long-lived private data in system RAM - the MSI
> doorbell trick is far more of a niche hack really.
>
> At the very least I think we need to refer to the device's memory access
> properties here.
>
> Jon, Laurentiu - how do RMRs correspond to the EFI memory map on your
> firmware? I'm starting to think that as long as the underlying memory is
> described appropriately there then we should be able to infer correct
> attributes from the EFI memory type and flags.

The devices are all cache coherent and marked as _CCA, 1.  The Memory
regions are in the virt table as ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.

The current chicken and egg problem we have is that during the fsl-mc-bus
initialization we call

error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
                                              &mc_stream_id);

which gets deferred because the SMMU has not been initialized yet. Then we
initialize the RMR tables but there is no device reference there to be able to
query device properties, only the stream id.  After the IORT tables are parsed
and the SMMU is setup, on the second device probe we associate everything
based on the stream id and the fsl-mc-bus device is able to claim its 1-1 DMA
mappings.

cat /sys/kernel/iommu_groups/0/reserved_regions
0x0000000001000000 0x0000000010ffffff direct-relaxable
0x0000000008000000 0x00000000080fffff msi
0x000000080c000000 0x000000081bffffff direct-relaxable
0x0000001c00000000 0x0000001c001fffff direct-relaxable
0x0000002080000000 0x000000209fffffff direct-relaxable

-Jon

>
> Robin.

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-06 19:51         ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-09-06 19:51 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Linuxarm, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon,
	linux-arm-kernel

On Mon, Sep 6, 2021 at 7:44 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-08-05 17:03, Lorenzo Pieralisi wrote:
> > On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:
> >
> > [...]
> >
> >> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> >> +{
> >> +    struct acpi_iort_node *smmu;
> >> +    struct acpi_iort_rmr *rmr;
> >> +    struct acpi_iort_rmr_desc *rmr_desc;
> >> +    u32 map_count = iort_node->mapping_count;
> >> +    u32 sid;
> >> +    int i;
> >> +
> >> +    if (!iort_node->mapping_offset || map_count != 1) {
> >> +            pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> >> +                   iort_node);
> >> +            return;
> >> +    }
> >> +
> >> +    /* Retrieve associated smmu and stream id */
> >> +    smmu = iort_node_get_id(iort_node, &sid, 0);
> >> +    if (!smmu) {
> >> +            pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> >> +                   iort_node);
> >> +            return;
> >> +    }
> >> +
> >> +    /* Retrieve RMR data */
> >> +    rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> >> +    if (!rmr->rmr_offset || !rmr->rmr_count) {
> >> +            pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> >> +                   iort_node);
> >> +            return;
> >> +    }
> >> +
> >> +    rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> >> +                            rmr->rmr_offset);
> >> +
> >> +    iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> >> +
> >> +    for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> >> +            struct iommu_resv_region *region;
> >> +            enum iommu_resv_type type;
> >> +            int prot = IOMMU_READ | IOMMU_WRITE;
> >> +            u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> >> +
> >> +            if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> >> +                    /* PAGE align base addr and size */
> >> +                    addr &= PAGE_MASK;
> >> +                    size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> >> +
> >> +                    pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> >> +                           rmr_desc->base_address,
> >> +                           rmr_desc->base_address + rmr_desc->length - 1,
> >> +                           addr, addr + size - 1);
> >> +            }
> >> +            if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> >> +                    type = IOMMU_RESV_DIRECT_RELAXABLE;
> >> +                    /*
> >> +                     * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> >> +                     * normally used for allocated system memory that is
> >> +                     * then used for device specific reserved regions.
> >> +                     */
> >> +                    prot |= IOMMU_CACHE;
> >> +            } else {
> >> +                    type = IOMMU_RESV_DIRECT;
> >> +                    /*
> >> +                     * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> >> +                     * for device memory like MSI doorbell.
> >> +                     */
> >> +                    prot |= IOMMU_MMIO;
> >> +            }
> >
> > On the prot value assignment based on the remapping flag, I'd like to
> > hear Robin/Joerg's opinion, I'd avoid being in a situation where
> > "normally" this would work but then we have to quirk it.
> >
> > Is this a valid assumption _always_ ?
>
> No. Certainly applying IOMMU_CACHE without reference to the device's
> _CCA attribute or how CPUs may be accessing a shared buffer could lead
> to a loss of coherency. At worst, applying IOMMU_MMIO to a
> device-private buffer *could* cause the device to lose coherency with
> itself if the memory underlying the RMR may have allocated into system
> caches. Note that the expected use for non-remappable RMRs is the device
> holding some sort of long-lived private data in system RAM - the MSI
> doorbell trick is far more of a niche hack really.
>
> At the very least I think we need to refer to the device's memory access
> properties here.
>
> Jon, Laurentiu - how do RMRs correspond to the EFI memory map on your
> firmware? I'm starting to think that as long as the underlying memory is
> described appropriately there then we should be able to infer correct
> attributes from the EFI memory type and flags.

The devices are all cache coherent and marked as _CCA, 1.  The Memory
regions are in the virt table as ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.

The current chicken and egg problem we have is that during the fsl-mc-bus
initialization we call

error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
                                              &mc_stream_id);

which gets deferred because the SMMU has not been initialized yet. Then we
initialize the RMR tables but there is no device reference there to be able to
query device properties, only the stream id.  After the IORT tables are parsed
and the SMMU is setup, on the second device probe we associate everything
based on the stream id and the fsl-mc-bus device is able to claim its 1-1 DMA
mappings.

cat /sys/kernel/iommu_groups/0/reserved_regions
0x0000000001000000 0x0000000010ffffff direct-relaxable
0x0000000008000000 0x00000000080fffff msi
0x000000080c000000 0x000000081bffffff direct-relaxable
0x0000001c00000000 0x0000001c001fffff direct-relaxable
0x0000002080000000 0x000000209fffffff direct-relaxable

-Jon

>
> Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-06 19:51         ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-09-06 19:51 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Lorenzo Pieralisi, Shameer Kolothum, Laurentiu Tudor,
	linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Joerg Roedel, Will Deacon, wanghuiqiang, Hanjun Guo,
	Steven Price, Sami Mujawar, Eric Auger, yangyicong

On Mon, Sep 6, 2021 at 7:44 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-08-05 17:03, Lorenzo Pieralisi wrote:
> > On Thu, Aug 05, 2021 at 09:07:17AM +0100, Shameer Kolothum wrote:
> >
> > [...]
> >
> >> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> >> +{
> >> +    struct acpi_iort_node *smmu;
> >> +    struct acpi_iort_rmr *rmr;
> >> +    struct acpi_iort_rmr_desc *rmr_desc;
> >> +    u32 map_count = iort_node->mapping_count;
> >> +    u32 sid;
> >> +    int i;
> >> +
> >> +    if (!iort_node->mapping_offset || map_count != 1) {
> >> +            pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> >> +                   iort_node);
> >> +            return;
> >> +    }
> >> +
> >> +    /* Retrieve associated smmu and stream id */
> >> +    smmu = iort_node_get_id(iort_node, &sid, 0);
> >> +    if (!smmu) {
> >> +            pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> >> +                   iort_node);
> >> +            return;
> >> +    }
> >> +
> >> +    /* Retrieve RMR data */
> >> +    rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> >> +    if (!rmr->rmr_offset || !rmr->rmr_count) {
> >> +            pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> >> +                   iort_node);
> >> +            return;
> >> +    }
> >> +
> >> +    rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> >> +                            rmr->rmr_offset);
> >> +
> >> +    iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> >> +
> >> +    for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> >> +            struct iommu_resv_region *region;
> >> +            enum iommu_resv_type type;
> >> +            int prot = IOMMU_READ | IOMMU_WRITE;
> >> +            u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> >> +
> >> +            if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> >> +                    /* PAGE align base addr and size */
> >> +                    addr &= PAGE_MASK;
> >> +                    size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> >> +
> >> +                    pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> >> +                           rmr_desc->base_address,
> >> +                           rmr_desc->base_address + rmr_desc->length - 1,
> >> +                           addr, addr + size - 1);
> >> +            }
> >> +            if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> >> +                    type = IOMMU_RESV_DIRECT_RELAXABLE;
> >> +                    /*
> >> +                     * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> >> +                     * normally used for allocated system memory that is
> >> +                     * then used for device specific reserved regions.
> >> +                     */
> >> +                    prot |= IOMMU_CACHE;
> >> +            } else {
> >> +                    type = IOMMU_RESV_DIRECT;
> >> +                    /*
> >> +                     * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> >> +                     * for device memory like MSI doorbell.
> >> +                     */
> >> +                    prot |= IOMMU_MMIO;
> >> +            }
> >
> > On the prot value assignment based on the remapping flag, I'd like to
> > hear Robin/Joerg's opinion, I'd avoid being in a situation where
> > "normally" this would work but then we have to quirk it.
> >
> > Is this a valid assumption _always_ ?
>
> No. Certainly applying IOMMU_CACHE without reference to the device's
> _CCA attribute or how CPUs may be accessing a shared buffer could lead
> to a loss of coherency. At worst, applying IOMMU_MMIO to a
> device-private buffer *could* cause the device to lose coherency with
> itself if the memory underlying the RMR may have allocated into system
> caches. Note that the expected use for non-remappable RMRs is the device
> holding some sort of long-lived private data in system RAM - the MSI
> doorbell trick is far more of a niche hack really.
>
> At the very least I think we need to refer to the device's memory access
> properties here.
>
> Jon, Laurentiu - how do RMRs correspond to the EFI memory map on your
> firmware? I'm starting to think that as long as the underlying memory is
> described appropriately there then we should be able to infer correct
> attributes from the EFI memory type and flags.

The devices are all cache coherent and marked as _CCA, 1.  The Memory
regions are in the virt table as ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.

The current chicken and egg problem we have is that during the fsl-mc-bus
initialization we call

error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
                                              &mc_stream_id);

which gets deferred because the SMMU has not been initialized yet. Then we
initialize the RMR tables but there is no device reference there to be able to
query device properties, only the stream id.  After the IORT tables are parsed
and the SMMU is setup, on the second device probe we associate everything
based on the stream id and the fsl-mc-bus device is able to claim its 1-1 DMA
mappings.

cat /sys/kernel/iommu_groups/0/reserved_regions
0x0000000001000000 0x0000000010ffffff direct-relaxable
0x0000000008000000 0x00000000080fffff msi
0x000000080c000000 0x000000081bffffff direct-relaxable
0x0000001c00000000 0x0000001c001fffff direct-relaxable
0x0000002080000000 0x000000209fffffff direct-relaxable

-Jon

>
> Robin.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-09-06 19:51         ` Jon Nettleton
  (?)
@ 2021-09-16  7:26           ` Shameerali Kolothum Thodi
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-09-16  7:26 UTC (permalink / raw)
  To: Jon Nettleton, Robin Murphy
  Cc: Lorenzo Pieralisi, Shameerali Kolothum Thodi, Laurentiu Tudor,
	linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU,
	Joerg Roedel, Will Deacon, wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong



> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 06 September 2021 20:51
> To: Robin Murphy <robin.murphy@arm.com>
> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>; Laurentiu Tudor
> <laurentiu.tudor@nxp.com>; linux-arm-kernel
> <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> Joerg Roedel <joro@8bytes.org>; Will Deacon <will@kernel.org>;
> wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> Mujawar <Sami.Mujawar@arm.com>; Eric Auger <eric.auger@redhat.com>;
> yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> 
[...]

> > >
> > > On the prot value assignment based on the remapping flag, I'd like
> > > to hear Robin/Joerg's opinion, I'd avoid being in a situation where
> > > "normally" this would work but then we have to quirk it.
> > >
> > > Is this a valid assumption _always_ ?
> >
> > No. Certainly applying IOMMU_CACHE without reference to the device's
> > _CCA attribute or how CPUs may be accessing a shared buffer could lead
> > to a loss of coherency. At worst, applying IOMMU_MMIO to a
> > device-private buffer *could* cause the device to lose coherency with
> > itself if the memory underlying the RMR may have allocated into system
> > caches. Note that the expected use for non-remappable RMRs is the
> > device holding some sort of long-lived private data in system RAM -
> > the MSI doorbell trick is far more of a niche hack really.
> >
> > At the very least I think we need to refer to the device's memory
> > access properties here.
> >
> > Jon, Laurentiu - how do RMRs correspond to the EFI memory map on your
> > firmware? I'm starting to think that as long as the underlying memory
> > is described appropriately there then we should be able to infer
> > correct attributes from the EFI memory type and flags.
> 
> The devices are all cache coherent and marked as _CCA, 1.  The Memory
> regions are in the virt table as ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> 
> The current chicken and egg problem we have is that during the fsl-mc-bus
> initialization we call
> 
> error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
>                                               &mc_stream_id);
> 
> which gets deferred because the SMMU has not been initialized yet. Then we
> initialize the RMR tables but there is no device reference there to be able to
> query device properties, only the stream id.  After the IORT tables are parsed
> and the SMMU is setup, on the second device probe we associate everything
> based on the stream id and the fsl-mc-bus device is able to claim its 1-1 DMA
> mappings.

Can we solve this order problem by delaying the iommu_alloc_resv_region()
to the iommu_dma_get_rmr_resv_regions(dev, list) ? We could invoke
device_get_dma_attr() from there which I believe will return the _CCA attribute.

Or is that still early to invoke that?

Thanks,
Shameer

> cat /sys/kernel/iommu_groups/0/reserved_regions
> 0x0000000001000000 0x0000000010ffffff direct-relaxable
> 0x0000000008000000 0x00000000080fffff msi
> 0x000000080c000000 0x000000081bffffff direct-relaxable
> 0x0000001c00000000 0x0000001c001fffff direct-relaxable
> 0x0000002080000000 0x000000209fffffff direct-relaxable
> 
> -Jon
> 
> >
> > Robin.

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-16  7:26           ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-09-16  7:26 UTC (permalink / raw)
  To: Jon Nettleton, Robin Murphy
  Cc: Steven Price, ACPI Devel Maling List, Linux IOMMU, wanghuiqiang,
	Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Will Deacon, linux-arm-kernel



> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 06 September 2021 20:51
> To: Robin Murphy <robin.murphy@arm.com>
> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>; Laurentiu Tudor
> <laurentiu.tudor@nxp.com>; linux-arm-kernel
> <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> Joerg Roedel <joro@8bytes.org>; Will Deacon <will@kernel.org>;
> wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> Mujawar <Sami.Mujawar@arm.com>; Eric Auger <eric.auger@redhat.com>;
> yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> 
[...]

> > >
> > > On the prot value assignment based on the remapping flag, I'd like
> > > to hear Robin/Joerg's opinion, I'd avoid being in a situation where
> > > "normally" this would work but then we have to quirk it.
> > >
> > > Is this a valid assumption _always_ ?
> >
> > No. Certainly applying IOMMU_CACHE without reference to the device's
> > _CCA attribute or how CPUs may be accessing a shared buffer could lead
> > to a loss of coherency. At worst, applying IOMMU_MMIO to a
> > device-private buffer *could* cause the device to lose coherency with
> > itself if the memory underlying the RMR may have allocated into system
> > caches. Note that the expected use for non-remappable RMRs is the
> > device holding some sort of long-lived private data in system RAM -
> > the MSI doorbell trick is far more of a niche hack really.
> >
> > At the very least I think we need to refer to the device's memory
> > access properties here.
> >
> > Jon, Laurentiu - how do RMRs correspond to the EFI memory map on your
> > firmware? I'm starting to think that as long as the underlying memory
> > is described appropriately there then we should be able to infer
> > correct attributes from the EFI memory type and flags.
> 
> The devices are all cache coherent and marked as _CCA, 1.  The Memory
> regions are in the virt table as ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> 
> The current chicken and egg problem we have is that during the fsl-mc-bus
> initialization we call
> 
> error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
>                                               &mc_stream_id);
> 
> which gets deferred because the SMMU has not been initialized yet. Then we
> initialize the RMR tables but there is no device reference there to be able to
> query device properties, only the stream id.  After the IORT tables are parsed
> and the SMMU is setup, on the second device probe we associate everything
> based on the stream id and the fsl-mc-bus device is able to claim its 1-1 DMA
> mappings.

Can we solve this order problem by delaying the iommu_alloc_resv_region()
to the iommu_dma_get_rmr_resv_regions(dev, list) ? We could invoke
device_get_dma_attr() from there which I believe will return the _CCA attribute.

Or is that still early to invoke that?

Thanks,
Shameer

> cat /sys/kernel/iommu_groups/0/reserved_regions
> 0x0000000001000000 0x0000000010ffffff direct-relaxable
> 0x0000000008000000 0x00000000080fffff msi
> 0x000000080c000000 0x000000081bffffff direct-relaxable
> 0x0000001c00000000 0x0000001c001fffff direct-relaxable
> 0x0000002080000000 0x000000209fffffff direct-relaxable
> 
> -Jon
> 
> >
> > Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-16  7:26           ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-09-16  7:26 UTC (permalink / raw)
  To: Jon Nettleton, Robin Murphy
  Cc: Lorenzo Pieralisi, Shameerali Kolothum Thodi, Laurentiu Tudor,
	linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU,
	Joerg Roedel, Will Deacon, wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong



> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 06 September 2021 20:51
> To: Robin Murphy <robin.murphy@arm.com>
> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>; Laurentiu Tudor
> <laurentiu.tudor@nxp.com>; linux-arm-kernel
> <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> Joerg Roedel <joro@8bytes.org>; Will Deacon <will@kernel.org>;
> wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> Mujawar <Sami.Mujawar@arm.com>; Eric Auger <eric.auger@redhat.com>;
> yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> 
[...]

> > >
> > > On the prot value assignment based on the remapping flag, I'd like
> > > to hear Robin/Joerg's opinion, I'd avoid being in a situation where
> > > "normally" this would work but then we have to quirk it.
> > >
> > > Is this a valid assumption _always_ ?
> >
> > No. Certainly applying IOMMU_CACHE without reference to the device's
> > _CCA attribute or how CPUs may be accessing a shared buffer could lead
> > to a loss of coherency. At worst, applying IOMMU_MMIO to a
> > device-private buffer *could* cause the device to lose coherency with
> > itself if the memory underlying the RMR may have allocated into system
> > caches. Note that the expected use for non-remappable RMRs is the
> > device holding some sort of long-lived private data in system RAM -
> > the MSI doorbell trick is far more of a niche hack really.
> >
> > At the very least I think we need to refer to the device's memory
> > access properties here.
> >
> > Jon, Laurentiu - how do RMRs correspond to the EFI memory map on your
> > firmware? I'm starting to think that as long as the underlying memory
> > is described appropriately there then we should be able to infer
> > correct attributes from the EFI memory type and flags.
> 
> The devices are all cache coherent and marked as _CCA, 1.  The Memory
> regions are in the virt table as ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> 
> The current chicken and egg problem we have is that during the fsl-mc-bus
> initialization we call
> 
> error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
>                                               &mc_stream_id);
> 
> which gets deferred because the SMMU has not been initialized yet. Then we
> initialize the RMR tables but there is no device reference there to be able to
> query device properties, only the stream id.  After the IORT tables are parsed
> and the SMMU is setup, on the second device probe we associate everything
> based on the stream id and the fsl-mc-bus device is able to claim its 1-1 DMA
> mappings.

Can we solve this order problem by delaying the iommu_alloc_resv_region()
to the iommu_dma_get_rmr_resv_regions(dev, list) ? We could invoke
device_get_dma_attr() from there which I believe will return the _CCA attribute.

Or is that still early to invoke that?

Thanks,
Shameer

> cat /sys/kernel/iommu_groups/0/reserved_regions
> 0x0000000001000000 0x0000000010ffffff direct-relaxable
> 0x0000000008000000 0x00000000080fffff msi
> 0x000000080c000000 0x000000081bffffff direct-relaxable
> 0x0000001c00000000 0x0000001c001fffff direct-relaxable
> 0x0000002080000000 0x000000209fffffff direct-relaxable
> 
> -Jon
> 
> >
> > Robin.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-09-16  7:26           ` Shameerali Kolothum Thodi
  (?)
@ 2021-09-16  7:52             ` Jon Nettleton
  -1 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-09-16  7:52 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi
  Cc: Robin Murphy, Lorenzo Pieralisi, Laurentiu Tudor,
	linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU,
	Joerg Roedel, Will Deacon, wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong

On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Jon Nettleton [mailto:jon@solid-run.com]
> > Sent: 06 September 2021 20:51
> > To: Robin Murphy <robin.murphy@arm.com>
> > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com>; Laurentiu Tudor
> > <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > Joerg Roedel <joro@8bytes.org>; Will Deacon <will@kernel.org>;
> > wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> > Mujawar <Sami.Mujawar@arm.com>; Eric Auger <eric.auger@redhat.com>;
> > yangyicong <yangyicong@huawei.com>
> > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> >
> [...]
>
> > > >
> > > > On the prot value assignment based on the remapping flag, I'd like
> > > > to hear Robin/Joerg's opinion, I'd avoid being in a situation where
> > > > "normally" this would work but then we have to quirk it.
> > > >
> > > > Is this a valid assumption _always_ ?
> > >
> > > No. Certainly applying IOMMU_CACHE without reference to the device's
> > > _CCA attribute or how CPUs may be accessing a shared buffer could lead
> > > to a loss of coherency. At worst, applying IOMMU_MMIO to a
> > > device-private buffer *could* cause the device to lose coherency with
> > > itself if the memory underlying the RMR may have allocated into system
> > > caches. Note that the expected use for non-remappable RMRs is the
> > > device holding some sort of long-lived private data in system RAM -
> > > the MSI doorbell trick is far more of a niche hack really.
> > >
> > > At the very least I think we need to refer to the device's memory
> > > access properties here.
> > >
> > > Jon, Laurentiu - how do RMRs correspond to the EFI memory map on your
> > > firmware? I'm starting to think that as long as the underlying memory
> > > is described appropriately there then we should be able to infer
> > > correct attributes from the EFI memory type and flags.
> >
> > The devices are all cache coherent and marked as _CCA, 1.  The Memory
> > regions are in the virt table as ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> >
> > The current chicken and egg problem we have is that during the fsl-mc-bus
> > initialization we call
> >
> > error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
> >                                               &mc_stream_id);
> >
> > which gets deferred because the SMMU has not been initialized yet. Then we
> > initialize the RMR tables but there is no device reference there to be able to
> > query device properties, only the stream id.  After the IORT tables are parsed
> > and the SMMU is setup, on the second device probe we associate everything
> > based on the stream id and the fsl-mc-bus device is able to claim its 1-1 DMA
> > mappings.
>
> Can we solve this order problem by delaying the iommu_alloc_resv_region()
> to the iommu_dma_get_rmr_resv_regions(dev, list) ? We could invoke
> device_get_dma_attr() from there which I believe will return the _CCA attribute.
>
> Or is that still early to invoke that?

That looks like it should work. Do we then also need to parse through the
VirtualMemoryTable matching the start and end addresses to determine the
other memory attributes like MMIO?

-Jon

>
> Thanks,
> Shameer
>
> > cat /sys/kernel/iommu_groups/0/reserved_regions
> > 0x0000000001000000 0x0000000010ffffff direct-relaxable
> > 0x0000000008000000 0x00000000080fffff msi
> > 0x000000080c000000 0x000000081bffffff direct-relaxable
> > 0x0000001c00000000 0x0000001c001fffff direct-relaxable
> > 0x0000002080000000 0x000000209fffffff direct-relaxable
> >
> > -Jon
> >
> > >
> > > Robin.

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-16  7:52             ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-09-16  7:52 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi
  Cc: Will Deacon, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Robin Murphy, linux-arm-kernel

On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Jon Nettleton [mailto:jon@solid-run.com]
> > Sent: 06 September 2021 20:51
> > To: Robin Murphy <robin.murphy@arm.com>
> > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com>; Laurentiu Tudor
> > <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > Joerg Roedel <joro@8bytes.org>; Will Deacon <will@kernel.org>;
> > wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> > Mujawar <Sami.Mujawar@arm.com>; Eric Auger <eric.auger@redhat.com>;
> > yangyicong <yangyicong@huawei.com>
> > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> >
> [...]
>
> > > >
> > > > On the prot value assignment based on the remapping flag, I'd like
> > > > to hear Robin/Joerg's opinion, I'd avoid being in a situation where
> > > > "normally" this would work but then we have to quirk it.
> > > >
> > > > Is this a valid assumption _always_ ?
> > >
> > > No. Certainly applying IOMMU_CACHE without reference to the device's
> > > _CCA attribute or how CPUs may be accessing a shared buffer could lead
> > > to a loss of coherency. At worst, applying IOMMU_MMIO to a
> > > device-private buffer *could* cause the device to lose coherency with
> > > itself if the memory underlying the RMR may have allocated into system
> > > caches. Note that the expected use for non-remappable RMRs is the
> > > device holding some sort of long-lived private data in system RAM -
> > > the MSI doorbell trick is far more of a niche hack really.
> > >
> > > At the very least I think we need to refer to the device's memory
> > > access properties here.
> > >
> > > Jon, Laurentiu - how do RMRs correspond to the EFI memory map on your
> > > firmware? I'm starting to think that as long as the underlying memory
> > > is described appropriately there then we should be able to infer
> > > correct attributes from the EFI memory type and flags.
> >
> > The devices are all cache coherent and marked as _CCA, 1.  The Memory
> > regions are in the virt table as ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> >
> > The current chicken and egg problem we have is that during the fsl-mc-bus
> > initialization we call
> >
> > error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
> >                                               &mc_stream_id);
> >
> > which gets deferred because the SMMU has not been initialized yet. Then we
> > initialize the RMR tables but there is no device reference there to be able to
> > query device properties, only the stream id.  After the IORT tables are parsed
> > and the SMMU is setup, on the second device probe we associate everything
> > based on the stream id and the fsl-mc-bus device is able to claim its 1-1 DMA
> > mappings.
>
> Can we solve this order problem by delaying the iommu_alloc_resv_region()
> to the iommu_dma_get_rmr_resv_regions(dev, list) ? We could invoke
> device_get_dma_attr() from there which I believe will return the _CCA attribute.
>
> Or is that still early to invoke that?

That looks like it should work. Do we then also need to parse through the
VirtualMemoryTable matching the start and end addresses to determine the
other memory attributes like MMIO?

-Jon

>
> Thanks,
> Shameer
>
> > cat /sys/kernel/iommu_groups/0/reserved_regions
> > 0x0000000001000000 0x0000000010ffffff direct-relaxable
> > 0x0000000008000000 0x00000000080fffff msi
> > 0x000000080c000000 0x000000081bffffff direct-relaxable
> > 0x0000001c00000000 0x0000001c001fffff direct-relaxable
> > 0x0000002080000000 0x000000209fffffff direct-relaxable
> >
> > -Jon
> >
> > >
> > > Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-16  7:52             ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-09-16  7:52 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi
  Cc: Robin Murphy, Lorenzo Pieralisi, Laurentiu Tudor,
	linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU,
	Joerg Roedel, Will Deacon, wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong

On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Jon Nettleton [mailto:jon@solid-run.com]
> > Sent: 06 September 2021 20:51
> > To: Robin Murphy <robin.murphy@arm.com>
> > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com>; Laurentiu Tudor
> > <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > Joerg Roedel <joro@8bytes.org>; Will Deacon <will@kernel.org>;
> > wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> > Mujawar <Sami.Mujawar@arm.com>; Eric Auger <eric.auger@redhat.com>;
> > yangyicong <yangyicong@huawei.com>
> > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> >
> [...]
>
> > > >
> > > > On the prot value assignment based on the remapping flag, I'd like
> > > > to hear Robin/Joerg's opinion, I'd avoid being in a situation where
> > > > "normally" this would work but then we have to quirk it.
> > > >
> > > > Is this a valid assumption _always_ ?
> > >
> > > No. Certainly applying IOMMU_CACHE without reference to the device's
> > > _CCA attribute or how CPUs may be accessing a shared buffer could lead
> > > to a loss of coherency. At worst, applying IOMMU_MMIO to a
> > > device-private buffer *could* cause the device to lose coherency with
> > > itself if the memory underlying the RMR may have allocated into system
> > > caches. Note that the expected use for non-remappable RMRs is the
> > > device holding some sort of long-lived private data in system RAM -
> > > the MSI doorbell trick is far more of a niche hack really.
> > >
> > > At the very least I think we need to refer to the device's memory
> > > access properties here.
> > >
> > > Jon, Laurentiu - how do RMRs correspond to the EFI memory map on your
> > > firmware? I'm starting to think that as long as the underlying memory
> > > is described appropriately there then we should be able to infer
> > > correct attributes from the EFI memory type and flags.
> >
> > The devices are all cache coherent and marked as _CCA, 1.  The Memory
> > regions are in the virt table as ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> >
> > The current chicken and egg problem we have is that during the fsl-mc-bus
> > initialization we call
> >
> > error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
> >                                               &mc_stream_id);
> >
> > which gets deferred because the SMMU has not been initialized yet. Then we
> > initialize the RMR tables but there is no device reference there to be able to
> > query device properties, only the stream id.  After the IORT tables are parsed
> > and the SMMU is setup, on the second device probe we associate everything
> > based on the stream id and the fsl-mc-bus device is able to claim its 1-1 DMA
> > mappings.
>
> Can we solve this order problem by delaying the iommu_alloc_resv_region()
> to the iommu_dma_get_rmr_resv_regions(dev, list) ? We could invoke
> device_get_dma_attr() from there which I believe will return the _CCA attribute.
>
> Or is that still early to invoke that?

That looks like it should work. Do we then also need to parse through the
VirtualMemoryTable matching the start and end addresses to determine the
other memory attributes like MMIO?

-Jon

>
> Thanks,
> Shameer
>
> > cat /sys/kernel/iommu_groups/0/reserved_regions
> > 0x0000000001000000 0x0000000010ffffff direct-relaxable
> > 0x0000000008000000 0x00000000080fffff msi
> > 0x000000080c000000 0x000000081bffffff direct-relaxable
> > 0x0000001c00000000 0x0000001c001fffff direct-relaxable
> > 0x0000002080000000 0x000000209fffffff direct-relaxable
> >
> > -Jon
> >
> > >
> > > Robin.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-09-16  7:52             ` Jon Nettleton
  (?)
@ 2021-09-16  8:26               ` Shameerali Kolothum Thodi
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-09-16  8:26 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Robin Murphy, Lorenzo Pieralisi, Laurentiu Tudor,
	linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU,
	Joerg Roedel, Will Deacon, wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong



> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 16 September 2021 08:52
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com>; Laurentiu Tudor <laurentiu.tudor@nxp.com>;
> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> List <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>; Will
> Deacon <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>;
> Guohanjun (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Eric
> Auger <eric.auger@redhat.com>; yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> 
> On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > Sent: 06 September 2021 20:51
> > > To: Robin Murphy <robin.murphy@arm.com>
> > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali
> > > Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; Laurentiu
> > > Tudor <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > > Joerg Roedel <joro@8bytes.org>; Will Deacon <will@kernel.org>;
> > > wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> > > Mujawar <Sami.Mujawar@arm.com>; Eric Auger
> <eric.auger@redhat.com>;
> > > yangyicong <yangyicong@huawei.com>
> > > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
> > > parsing
> > >
> > [...]
> >
> > > > >
> > > > > On the prot value assignment based on the remapping flag, I'd
> > > > > like to hear Robin/Joerg's opinion, I'd avoid being in a
> > > > > situation where "normally" this would work but then we have to quirk
> it.
> > > > >
> > > > > Is this a valid assumption _always_ ?
> > > >
> > > > No. Certainly applying IOMMU_CACHE without reference to the
> > > > device's _CCA attribute or how CPUs may be accessing a shared
> > > > buffer could lead to a loss of coherency. At worst, applying
> > > > IOMMU_MMIO to a device-private buffer *could* cause the device to
> > > > lose coherency with itself if the memory underlying the RMR may
> > > > have allocated into system caches. Note that the expected use for
> > > > non-remappable RMRs is the device holding some sort of long-lived
> > > > private data in system RAM - the MSI doorbell trick is far more of a niche
> hack really.
> > > >
> > > > At the very least I think we need to refer to the device's memory
> > > > access properties here.
> > > >
> > > > Jon, Laurentiu - how do RMRs correspond to the EFI memory map on
> > > > your firmware? I'm starting to think that as long as the
> > > > underlying memory is described appropriately there then we should
> > > > be able to infer correct attributes from the EFI memory type and flags.
> > >
> > > The devices are all cache coherent and marked as _CCA, 1.  The
> > > Memory regions are in the virt table as
> ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> > >
> > > The current chicken and egg problem we have is that during the
> > > fsl-mc-bus initialization we call
> > >
> > > error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
> > >                                               &mc_stream_id);
> > >
> > > which gets deferred because the SMMU has not been initialized yet.
> > > Then we initialize the RMR tables but there is no device reference
> > > there to be able to query device properties, only the stream id.
> > > After the IORT tables are parsed and the SMMU is setup, on the
> > > second device probe we associate everything based on the stream id
> > > and the fsl-mc-bus device is able to claim its 1-1 DMA mappings.
> >
> > Can we solve this order problem by delaying the
> > iommu_alloc_resv_region() to the iommu_dma_get_rmr_resv_regions(dev,
> > list) ? We could invoke
> > device_get_dma_attr() from there which I believe will return the _CCA
> attribute.
> >
> > Or is that still early to invoke that?
> 
> That looks like it should work. Do we then also need to parse through the
> VirtualMemoryTable matching the start and end addresses to determine the
> other memory attributes like MMIO?

Yes. But that looks tricky as I can't find that readily available on Arm, like the
efi_mem_attributes(). I will take a look.

Please let me know if there is one or any other easy way to retrieve it.

Thanks,
Shameer

> 
> -Jon
> 
> >
> > Thanks,
> > Shameer
> >
> > > cat /sys/kernel/iommu_groups/0/reserved_regions
> > > 0x0000000001000000 0x0000000010ffffff direct-relaxable
> > > 0x0000000008000000 0x00000000080fffff msi
> > > 0x000000080c000000 0x000000081bffffff direct-relaxable
> > > 0x0000001c00000000 0x0000001c001fffff direct-relaxable
> > > 0x0000002080000000 0x000000209fffffff direct-relaxable
> > >
> > > -Jon
> > >
> > > >
> > > > Robin.

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-16  8:26               ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-09-16  8:26 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Will Deacon, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Robin Murphy, linux-arm-kernel



> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 16 September 2021 08:52
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com>; Laurentiu Tudor <laurentiu.tudor@nxp.com>;
> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> List <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>; Will
> Deacon <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>;
> Guohanjun (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Eric
> Auger <eric.auger@redhat.com>; yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> 
> On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > Sent: 06 September 2021 20:51
> > > To: Robin Murphy <robin.murphy@arm.com>
> > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali
> > > Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; Laurentiu
> > > Tudor <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > > Joerg Roedel <joro@8bytes.org>; Will Deacon <will@kernel.org>;
> > > wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> > > Mujawar <Sami.Mujawar@arm.com>; Eric Auger
> <eric.auger@redhat.com>;
> > > yangyicong <yangyicong@huawei.com>
> > > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
> > > parsing
> > >
> > [...]
> >
> > > > >
> > > > > On the prot value assignment based on the remapping flag, I'd
> > > > > like to hear Robin/Joerg's opinion, I'd avoid being in a
> > > > > situation where "normally" this would work but then we have to quirk
> it.
> > > > >
> > > > > Is this a valid assumption _always_ ?
> > > >
> > > > No. Certainly applying IOMMU_CACHE without reference to the
> > > > device's _CCA attribute or how CPUs may be accessing a shared
> > > > buffer could lead to a loss of coherency. At worst, applying
> > > > IOMMU_MMIO to a device-private buffer *could* cause the device to
> > > > lose coherency with itself if the memory underlying the RMR may
> > > > have allocated into system caches. Note that the expected use for
> > > > non-remappable RMRs is the device holding some sort of long-lived
> > > > private data in system RAM - the MSI doorbell trick is far more of a niche
> hack really.
> > > >
> > > > At the very least I think we need to refer to the device's memory
> > > > access properties here.
> > > >
> > > > Jon, Laurentiu - how do RMRs correspond to the EFI memory map on
> > > > your firmware? I'm starting to think that as long as the
> > > > underlying memory is described appropriately there then we should
> > > > be able to infer correct attributes from the EFI memory type and flags.
> > >
> > > The devices are all cache coherent and marked as _CCA, 1.  The
> > > Memory regions are in the virt table as
> ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> > >
> > > The current chicken and egg problem we have is that during the
> > > fsl-mc-bus initialization we call
> > >
> > > error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
> > >                                               &mc_stream_id);
> > >
> > > which gets deferred because the SMMU has not been initialized yet.
> > > Then we initialize the RMR tables but there is no device reference
> > > there to be able to query device properties, only the stream id.
> > > After the IORT tables are parsed and the SMMU is setup, on the
> > > second device probe we associate everything based on the stream id
> > > and the fsl-mc-bus device is able to claim its 1-1 DMA mappings.
> >
> > Can we solve this order problem by delaying the
> > iommu_alloc_resv_region() to the iommu_dma_get_rmr_resv_regions(dev,
> > list) ? We could invoke
> > device_get_dma_attr() from there which I believe will return the _CCA
> attribute.
> >
> > Or is that still early to invoke that?
> 
> That looks like it should work. Do we then also need to parse through the
> VirtualMemoryTable matching the start and end addresses to determine the
> other memory attributes like MMIO?

Yes. But that looks tricky as I can't find that readily available on Arm, like the
efi_mem_attributes(). I will take a look.

Please let me know if there is one or any other easy way to retrieve it.

Thanks,
Shameer

> 
> -Jon
> 
> >
> > Thanks,
> > Shameer
> >
> > > cat /sys/kernel/iommu_groups/0/reserved_regions
> > > 0x0000000001000000 0x0000000010ffffff direct-relaxable
> > > 0x0000000008000000 0x00000000080fffff msi
> > > 0x000000080c000000 0x000000081bffffff direct-relaxable
> > > 0x0000001c00000000 0x0000001c001fffff direct-relaxable
> > > 0x0000002080000000 0x000000209fffffff direct-relaxable
> > >
> > > -Jon
> > >
> > > >
> > > > Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-16  8:26               ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-09-16  8:26 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Robin Murphy, Lorenzo Pieralisi, Laurentiu Tudor,
	linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU,
	Joerg Roedel, Will Deacon, wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong



> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 16 September 2021 08:52
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com>; Laurentiu Tudor <laurentiu.tudor@nxp.com>;
> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> List <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>; Will
> Deacon <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>;
> Guohanjun (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Eric
> Auger <eric.auger@redhat.com>; yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> 
> On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > Sent: 06 September 2021 20:51
> > > To: Robin Murphy <robin.murphy@arm.com>
> > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali
> > > Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; Laurentiu
> > > Tudor <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > > Joerg Roedel <joro@8bytes.org>; Will Deacon <will@kernel.org>;
> > > wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> > > Mujawar <Sami.Mujawar@arm.com>; Eric Auger
> <eric.auger@redhat.com>;
> > > yangyicong <yangyicong@huawei.com>
> > > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
> > > parsing
> > >
> > [...]
> >
> > > > >
> > > > > On the prot value assignment based on the remapping flag, I'd
> > > > > like to hear Robin/Joerg's opinion, I'd avoid being in a
> > > > > situation where "normally" this would work but then we have to quirk
> it.
> > > > >
> > > > > Is this a valid assumption _always_ ?
> > > >
> > > > No. Certainly applying IOMMU_CACHE without reference to the
> > > > device's _CCA attribute or how CPUs may be accessing a shared
> > > > buffer could lead to a loss of coherency. At worst, applying
> > > > IOMMU_MMIO to a device-private buffer *could* cause the device to
> > > > lose coherency with itself if the memory underlying the RMR may
> > > > have allocated into system caches. Note that the expected use for
> > > > non-remappable RMRs is the device holding some sort of long-lived
> > > > private data in system RAM - the MSI doorbell trick is far more of a niche
> hack really.
> > > >
> > > > At the very least I think we need to refer to the device's memory
> > > > access properties here.
> > > >
> > > > Jon, Laurentiu - how do RMRs correspond to the EFI memory map on
> > > > your firmware? I'm starting to think that as long as the
> > > > underlying memory is described appropriately there then we should
> > > > be able to infer correct attributes from the EFI memory type and flags.
> > >
> > > The devices are all cache coherent and marked as _CCA, 1.  The
> > > Memory regions are in the virt table as
> ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> > >
> > > The current chicken and egg problem we have is that during the
> > > fsl-mc-bus initialization we call
> > >
> > > error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
> > >                                               &mc_stream_id);
> > >
> > > which gets deferred because the SMMU has not been initialized yet.
> > > Then we initialize the RMR tables but there is no device reference
> > > there to be able to query device properties, only the stream id.
> > > After the IORT tables are parsed and the SMMU is setup, on the
> > > second device probe we associate everything based on the stream id
> > > and the fsl-mc-bus device is able to claim its 1-1 DMA mappings.
> >
> > Can we solve this order problem by delaying the
> > iommu_alloc_resv_region() to the iommu_dma_get_rmr_resv_regions(dev,
> > list) ? We could invoke
> > device_get_dma_attr() from there which I believe will return the _CCA
> attribute.
> >
> > Or is that still early to invoke that?
> 
> That looks like it should work. Do we then also need to parse through the
> VirtualMemoryTable matching the start and end addresses to determine the
> other memory attributes like MMIO?

Yes. But that looks tricky as I can't find that readily available on Arm, like the
efi_mem_attributes(). I will take a look.

Please let me know if there is one or any other easy way to retrieve it.

Thanks,
Shameer

> 
> -Jon
> 
> >
> > Thanks,
> > Shameer
> >
> > > cat /sys/kernel/iommu_groups/0/reserved_regions
> > > 0x0000000001000000 0x0000000010ffffff direct-relaxable
> > > 0x0000000008000000 0x00000000080fffff msi
> > > 0x000000080c000000 0x000000081bffffff direct-relaxable
> > > 0x0000001c00000000 0x0000001c001fffff direct-relaxable
> > > 0x0000002080000000 0x000000209fffffff direct-relaxable
> > >
> > > -Jon
> > >
> > > >
> > > > Robin.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-09-16  8:26               ` Shameerali Kolothum Thodi
  (?)
@ 2021-09-16 11:16                 ` Jon Nettleton
  -1 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-09-16 11:16 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi
  Cc: Will Deacon, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Robin Murphy, linux-arm-kernel

On Thu, Sep 16, 2021 at 10:26 AM Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Jon Nettleton [mailto:jon@solid-run.com]
> > Sent: 16 September 2021 08:52
> > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> > Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
> > <lorenzo.pieralisi@arm.com>; Laurentiu Tudor <laurentiu.tudor@nxp.com>;
> > linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> > List <linux-acpi@vger.kernel.org>; Linux IOMMU
> > <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>; Will
> > Deacon <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>;
> > Guohanjun (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> > <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Eric
> > Auger <eric.auger@redhat.com>; yangyicong <yangyicong@huawei.com>
> > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> >
> > On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > > Sent: 06 September 2021 20:51
> > > > To: Robin Murphy <robin.murphy@arm.com>
> > > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali
> > > > Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; Laurentiu
> > > > Tudor <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > > > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > > > Joerg Roedel <joro@8bytes.org>; Will Deacon <will@kernel.org>;
> > > > wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> > > > Mujawar <Sami.Mujawar@arm.com>; Eric Auger
> > <eric.auger@redhat.com>;
> > > > yangyicong <yangyicong@huawei.com>
> > > > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
> > > > parsing
> > > >
> > > [...]
> > >
> > > > > >
> > > > > > On the prot value assignment based on the remapping flag, I'd
> > > > > > like to hear Robin/Joerg's opinion, I'd avoid being in a
> > > > > > situation where "normally" this would work but then we have to quirk
> > it.
> > > > > >
> > > > > > Is this a valid assumption _always_ ?
> > > > >
> > > > > No. Certainly applying IOMMU_CACHE without reference to the
> > > > > device's _CCA attribute or how CPUs may be accessing a shared
> > > > > buffer could lead to a loss of coherency. At worst, applying
> > > > > IOMMU_MMIO to a device-private buffer *could* cause the device to
> > > > > lose coherency with itself if the memory underlying the RMR may
> > > > > have allocated into system caches. Note that the expected use for
> > > > > non-remappable RMRs is the device holding some sort of long-lived
> > > > > private data in system RAM - the MSI doorbell trick is far more of a niche
> > hack really.
> > > > >
> > > > > At the very least I think we need to refer to the device's memory
> > > > > access properties here.
> > > > >
> > > > > Jon, Laurentiu - how do RMRs correspond to the EFI memory map on
> > > > > your firmware? I'm starting to think that as long as the
> > > > > underlying memory is described appropriately there then we should
> > > > > be able to infer correct attributes from the EFI memory type and flags.
> > > >
> > > > The devices are all cache coherent and marked as _CCA, 1.  The
> > > > Memory regions are in the virt table as
> > ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> > > >
> > > > The current chicken and egg problem we have is that during the
> > > > fsl-mc-bus initialization we call
> > > >
> > > > error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
> > > >                                               &mc_stream_id);
> > > >
> > > > which gets deferred because the SMMU has not been initialized yet.
> > > > Then we initialize the RMR tables but there is no device reference
> > > > there to be able to query device properties, only the stream id.
> > > > After the IORT tables are parsed and the SMMU is setup, on the
> > > > second device probe we associate everything based on the stream id
> > > > and the fsl-mc-bus device is able to claim its 1-1 DMA mappings.
> > >
> > > Can we solve this order problem by delaying the
> > > iommu_alloc_resv_region() to the iommu_dma_get_rmr_resv_regions(dev,
> > > list) ? We could invoke
> > > device_get_dma_attr() from there which I believe will return the _CCA
> > attribute.
> > >
> > > Or is that still early to invoke that?
> >
> > That looks like it should work. Do we then also need to parse through the
> > VirtualMemoryTable matching the start and end addresses to determine the
> > other memory attributes like MMIO?
>
> Yes. But that looks tricky as I can't find that readily available on Arm, like the
> efi_mem_attributes(). I will take a look.
>
> Please let me know if there is one or any other easy way to retrieve it.

maybe we don't need to.  Maybe it is enough to just move
iommu_alloc_resv_regions and then set the IOMMU_CACHE flag
if type = IOMMU_RESV_DIRECT_RELAXABLE and _CCN=1?

-Jon

>
> Thanks,
> Shameer
>
> >
> > -Jon
> >
> > >
> > > Thanks,
> > > Shameer
> > >
> > > > cat /sys/kernel/iommu_groups/0/reserved_regions
> > > > 0x0000000001000000 0x0000000010ffffff direct-relaxable
> > > > 0x0000000008000000 0x00000000080fffff msi
> > > > 0x000000080c000000 0x000000081bffffff direct-relaxable
> > > > 0x0000001c00000000 0x0000001c001fffff direct-relaxable
> > > > 0x0000002080000000 0x000000209fffffff direct-relaxable
> > > >
> > > > -Jon
> > > >
> > > > >
> > > > > Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-16 11:16                 ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-09-16 11:16 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi
  Cc: Robin Murphy, Lorenzo Pieralisi, Laurentiu Tudor,
	linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU,
	Joerg Roedel, Will Deacon, wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong

On Thu, Sep 16, 2021 at 10:26 AM Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Jon Nettleton [mailto:jon@solid-run.com]
> > Sent: 16 September 2021 08:52
> > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> > Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
> > <lorenzo.pieralisi@arm.com>; Laurentiu Tudor <laurentiu.tudor@nxp.com>;
> > linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> > List <linux-acpi@vger.kernel.org>; Linux IOMMU
> > <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>; Will
> > Deacon <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>;
> > Guohanjun (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> > <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Eric
> > Auger <eric.auger@redhat.com>; yangyicong <yangyicong@huawei.com>
> > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> >
> > On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > > Sent: 06 September 2021 20:51
> > > > To: Robin Murphy <robin.murphy@arm.com>
> > > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali
> > > > Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; Laurentiu
> > > > Tudor <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > > > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > > > Joerg Roedel <joro@8bytes.org>; Will Deacon <will@kernel.org>;
> > > > wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> > > > Mujawar <Sami.Mujawar@arm.com>; Eric Auger
> > <eric.auger@redhat.com>;
> > > > yangyicong <yangyicong@huawei.com>
> > > > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
> > > > parsing
> > > >
> > > [...]
> > >
> > > > > >
> > > > > > On the prot value assignment based on the remapping flag, I'd
> > > > > > like to hear Robin/Joerg's opinion, I'd avoid being in a
> > > > > > situation where "normally" this would work but then we have to quirk
> > it.
> > > > > >
> > > > > > Is this a valid assumption _always_ ?
> > > > >
> > > > > No. Certainly applying IOMMU_CACHE without reference to the
> > > > > device's _CCA attribute or how CPUs may be accessing a shared
> > > > > buffer could lead to a loss of coherency. At worst, applying
> > > > > IOMMU_MMIO to a device-private buffer *could* cause the device to
> > > > > lose coherency with itself if the memory underlying the RMR may
> > > > > have allocated into system caches. Note that the expected use for
> > > > > non-remappable RMRs is the device holding some sort of long-lived
> > > > > private data in system RAM - the MSI doorbell trick is far more of a niche
> > hack really.
> > > > >
> > > > > At the very least I think we need to refer to the device's memory
> > > > > access properties here.
> > > > >
> > > > > Jon, Laurentiu - how do RMRs correspond to the EFI memory map on
> > > > > your firmware? I'm starting to think that as long as the
> > > > > underlying memory is described appropriately there then we should
> > > > > be able to infer correct attributes from the EFI memory type and flags.
> > > >
> > > > The devices are all cache coherent and marked as _CCA, 1.  The
> > > > Memory regions are in the virt table as
> > ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> > > >
> > > > The current chicken and egg problem we have is that during the
> > > > fsl-mc-bus initialization we call
> > > >
> > > > error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
> > > >                                               &mc_stream_id);
> > > >
> > > > which gets deferred because the SMMU has not been initialized yet.
> > > > Then we initialize the RMR tables but there is no device reference
> > > > there to be able to query device properties, only the stream id.
> > > > After the IORT tables are parsed and the SMMU is setup, on the
> > > > second device probe we associate everything based on the stream id
> > > > and the fsl-mc-bus device is able to claim its 1-1 DMA mappings.
> > >
> > > Can we solve this order problem by delaying the
> > > iommu_alloc_resv_region() to the iommu_dma_get_rmr_resv_regions(dev,
> > > list) ? We could invoke
> > > device_get_dma_attr() from there which I believe will return the _CCA
> > attribute.
> > >
> > > Or is that still early to invoke that?
> >
> > That looks like it should work. Do we then also need to parse through the
> > VirtualMemoryTable matching the start and end addresses to determine the
> > other memory attributes like MMIO?
>
> Yes. But that looks tricky as I can't find that readily available on Arm, like the
> efi_mem_attributes(). I will take a look.
>
> Please let me know if there is one or any other easy way to retrieve it.

maybe we don't need to.  Maybe it is enough to just move
iommu_alloc_resv_regions and then set the IOMMU_CACHE flag
if type = IOMMU_RESV_DIRECT_RELAXABLE and _CCN=1?

-Jon

>
> Thanks,
> Shameer
>
> >
> > -Jon
> >
> > >
> > > Thanks,
> > > Shameer
> > >
> > > > cat /sys/kernel/iommu_groups/0/reserved_regions
> > > > 0x0000000001000000 0x0000000010ffffff direct-relaxable
> > > > 0x0000000008000000 0x00000000080fffff msi
> > > > 0x000000080c000000 0x000000081bffffff direct-relaxable
> > > > 0x0000001c00000000 0x0000001c001fffff direct-relaxable
> > > > 0x0000002080000000 0x000000209fffffff direct-relaxable
> > > >
> > > > -Jon
> > > >
> > > > >
> > > > > Robin.

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-16 11:16                 ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-09-16 11:16 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi
  Cc: Robin Murphy, Lorenzo Pieralisi, Laurentiu Tudor,
	linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU,
	Joerg Roedel, Will Deacon, wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong

On Thu, Sep 16, 2021 at 10:26 AM Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Jon Nettleton [mailto:jon@solid-run.com]
> > Sent: 16 September 2021 08:52
> > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> > Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
> > <lorenzo.pieralisi@arm.com>; Laurentiu Tudor <laurentiu.tudor@nxp.com>;
> > linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> > List <linux-acpi@vger.kernel.org>; Linux IOMMU
> > <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>; Will
> > Deacon <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>;
> > Guohanjun (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> > <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Eric
> > Auger <eric.auger@redhat.com>; yangyicong <yangyicong@huawei.com>
> > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> >
> > On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > > Sent: 06 September 2021 20:51
> > > > To: Robin Murphy <robin.murphy@arm.com>
> > > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali
> > > > Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; Laurentiu
> > > > Tudor <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > > > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> > > > Joerg Roedel <joro@8bytes.org>; Will Deacon <will@kernel.org>;
> > > > wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> > > > Mujawar <Sami.Mujawar@arm.com>; Eric Auger
> > <eric.auger@redhat.com>;
> > > > yangyicong <yangyicong@huawei.com>
> > > > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
> > > > parsing
> > > >
> > > [...]
> > >
> > > > > >
> > > > > > On the prot value assignment based on the remapping flag, I'd
> > > > > > like to hear Robin/Joerg's opinion, I'd avoid being in a
> > > > > > situation where "normally" this would work but then we have to quirk
> > it.
> > > > > >
> > > > > > Is this a valid assumption _always_ ?
> > > > >
> > > > > No. Certainly applying IOMMU_CACHE without reference to the
> > > > > device's _CCA attribute or how CPUs may be accessing a shared
> > > > > buffer could lead to a loss of coherency. At worst, applying
> > > > > IOMMU_MMIO to a device-private buffer *could* cause the device to
> > > > > lose coherency with itself if the memory underlying the RMR may
> > > > > have allocated into system caches. Note that the expected use for
> > > > > non-remappable RMRs is the device holding some sort of long-lived
> > > > > private data in system RAM - the MSI doorbell trick is far more of a niche
> > hack really.
> > > > >
> > > > > At the very least I think we need to refer to the device's memory
> > > > > access properties here.
> > > > >
> > > > > Jon, Laurentiu - how do RMRs correspond to the EFI memory map on
> > > > > your firmware? I'm starting to think that as long as the
> > > > > underlying memory is described appropriately there then we should
> > > > > be able to infer correct attributes from the EFI memory type and flags.
> > > >
> > > > The devices are all cache coherent and marked as _CCA, 1.  The
> > > > Memory regions are in the virt table as
> > ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> > > >
> > > > The current chicken and egg problem we have is that during the
> > > > fsl-mc-bus initialization we call
> > > >
> > > > error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
> > > >                                               &mc_stream_id);
> > > >
> > > > which gets deferred because the SMMU has not been initialized yet.
> > > > Then we initialize the RMR tables but there is no device reference
> > > > there to be able to query device properties, only the stream id.
> > > > After the IORT tables are parsed and the SMMU is setup, on the
> > > > second device probe we associate everything based on the stream id
> > > > and the fsl-mc-bus device is able to claim its 1-1 DMA mappings.
> > >
> > > Can we solve this order problem by delaying the
> > > iommu_alloc_resv_region() to the iommu_dma_get_rmr_resv_regions(dev,
> > > list) ? We could invoke
> > > device_get_dma_attr() from there which I believe will return the _CCA
> > attribute.
> > >
> > > Or is that still early to invoke that?
> >
> > That looks like it should work. Do we then also need to parse through the
> > VirtualMemoryTable matching the start and end addresses to determine the
> > other memory attributes like MMIO?
>
> Yes. But that looks tricky as I can't find that readily available on Arm, like the
> efi_mem_attributes(). I will take a look.
>
> Please let me know if there is one or any other easy way to retrieve it.

maybe we don't need to.  Maybe it is enough to just move
iommu_alloc_resv_regions and then set the IOMMU_CACHE flag
if type = IOMMU_RESV_DIRECT_RELAXABLE and _CCN=1?

-Jon

>
> Thanks,
> Shameer
>
> >
> > -Jon
> >
> > >
> > > Thanks,
> > > Shameer
> > >
> > > > cat /sys/kernel/iommu_groups/0/reserved_regions
> > > > 0x0000000001000000 0x0000000010ffffff direct-relaxable
> > > > 0x0000000008000000 0x00000000080fffff msi
> > > > 0x000000080c000000 0x000000081bffffff direct-relaxable
> > > > 0x0000001c00000000 0x0000001c001fffff direct-relaxable
> > > > 0x0000002080000000 0x000000209fffffff direct-relaxable
> > > >
> > > > -Jon
> > > >
> > > > >
> > > > > Robin.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-09-16 11:16                 ` Jon Nettleton
  (?)
@ 2021-09-17 11:26                   ` Shameerali Kolothum Thodi
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-09-17 11:26 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Robin Murphy, Lorenzo Pieralisi, Laurentiu Tudor,
	linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU,
	Joerg Roedel, Will Deacon, wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong



> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 16 September 2021 12:17
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com>; Laurentiu Tudor <laurentiu.tudor@nxp.com>;
> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> List <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>; Will
> Deacon <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>;
> Guohanjun (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Eric
> Auger <eric.auger@redhat.com>; yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> 
> On Thu, Sep 16, 2021 at 10:26 AM Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > Sent: 16 September 2021 08:52
> > > To: Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>
> > > Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
> > > <lorenzo.pieralisi@arm.com>; Laurentiu Tudor
> > > <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>;
> > > Will Deacon <will@kernel.org>; wanghuiqiang
> > > <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> > > Mujawar <Sami.Mujawar@arm.com>; Eric Auger
> <eric.auger@redhat.com>;
> > > yangyicong <yangyicong@huawei.com>
> > > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
> > > parsing
> > >
> > > On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
> > > <shameerali.kolothum.thodi@huawei.com> wrote:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > > > Sent: 06 September 2021 20:51
> > > > > To: Robin Murphy <robin.murphy@arm.com>
> > > > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali
> > > > > Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; Laurentiu
> > > > > Tudor <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > > > > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > > > <iommu@lists.linux-foundation.org>; Linuxarm
> > > > > <linuxarm@huawei.com>; Joerg Roedel <joro@8bytes.org>; Will
> > > > > Deacon <will@kernel.org>; wanghuiqiang
> > > > > <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > > > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>;
> > > > > Sami Mujawar <Sami.Mujawar@arm.com>; Eric Auger
> > > <eric.auger@redhat.com>;
> > > > > yangyicong <yangyicong@huawei.com>
> > > > > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
> > > > > parsing
> > > > >
> > > > [...]
> > > >
> > > > > > >
> > > > > > > On the prot value assignment based on the remapping flag,
> > > > > > > I'd like to hear Robin/Joerg's opinion, I'd avoid being in a
> > > > > > > situation where "normally" this would work but then we have
> > > > > > > to quirk
> > > it.
> > > > > > >
> > > > > > > Is this a valid assumption _always_ ?
> > > > > >
> > > > > > No. Certainly applying IOMMU_CACHE without reference to the
> > > > > > device's _CCA attribute or how CPUs may be accessing a shared
> > > > > > buffer could lead to a loss of coherency. At worst, applying
> > > > > > IOMMU_MMIO to a device-private buffer *could* cause the device
> > > > > > to lose coherency with itself if the memory underlying the RMR
> > > > > > may have allocated into system caches. Note that the expected
> > > > > > use for non-remappable RMRs is the device holding some sort of
> > > > > > long-lived private data in system RAM - the MSI doorbell trick
> > > > > > is far more of a niche
> > > hack really.
> > > > > >
> > > > > > At the very least I think we need to refer to the device's
> > > > > > memory access properties here.
> > > > > >
> > > > > > Jon, Laurentiu - how do RMRs correspond to the EFI memory map
> > > > > > on your firmware? I'm starting to think that as long as the
> > > > > > underlying memory is described appropriately there then we
> > > > > > should be able to infer correct attributes from the EFI memory type
> and flags.
> > > > >
> > > > > The devices are all cache coherent and marked as _CCA, 1.  The
> > > > > Memory regions are in the virt table as
> > > ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> > > > >
> > > > > The current chicken and egg problem we have is that during the
> > > > > fsl-mc-bus initialization we call
> > > > >
> > > > > error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
> > > > >                                               &mc_stream_id);
> > > > >
> > > > > which gets deferred because the SMMU has not been initialized yet.
> > > > > Then we initialize the RMR tables but there is no device
> > > > > reference there to be able to query device properties, only the stream
> id.
> > > > > After the IORT tables are parsed and the SMMU is setup, on the
> > > > > second device probe we associate everything based on the stream
> > > > > id and the fsl-mc-bus device is able to claim its 1-1 DMA mappings.
> > > >
> > > > Can we solve this order problem by delaying the
> > > > iommu_alloc_resv_region() to the
> > > > iommu_dma_get_rmr_resv_regions(dev,
> > > > list) ? We could invoke
> > > > device_get_dma_attr() from there which I believe will return the
> > > > _CCA
> > > attribute.
> > > >
> > > > Or is that still early to invoke that?
> > >
> > > That looks like it should work. Do we then also need to parse
> > > through the VirtualMemoryTable matching the start and end addresses
> > > to determine the other memory attributes like MMIO?
> >
> > Yes. But that looks tricky as I can't find that readily available on
> > Arm, like the efi_mem_attributes(). I will take a look.
> >
> > Please let me know if there is one or any other easy way to retrieve it.
> 
> maybe we don't need to.  Maybe it is enough to just move
> iommu_alloc_resv_regions and then set the IOMMU_CACHE flag if type =
> IOMMU_RESV_DIRECT_RELAXABLE and _CCN=1?

It looks like we could simply call efi_mem_type() and check for
EFI_MEMORY_MAPPED_IO. I have updated the code to set the
RMR prot value based on _CCA and EFI md type. Please see the 
last commit on this branch here(not tested),

https://github.com/hisilicon/kernel-dev/commits/private-v5.14-rc4-rmr-v7-ext

Please take a look and let me know if this is good enough to solve this problem.

Thanks,
Shameer


^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-17 11:26                   ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-09-17 11:26 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Will Deacon, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Robin Murphy, linux-arm-kernel



> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 16 September 2021 12:17
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com>; Laurentiu Tudor <laurentiu.tudor@nxp.com>;
> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> List <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>; Will
> Deacon <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>;
> Guohanjun (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Eric
> Auger <eric.auger@redhat.com>; yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> 
> On Thu, Sep 16, 2021 at 10:26 AM Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > Sent: 16 September 2021 08:52
> > > To: Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>
> > > Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
> > > <lorenzo.pieralisi@arm.com>; Laurentiu Tudor
> > > <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>;
> > > Will Deacon <will@kernel.org>; wanghuiqiang
> > > <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> > > Mujawar <Sami.Mujawar@arm.com>; Eric Auger
> <eric.auger@redhat.com>;
> > > yangyicong <yangyicong@huawei.com>
> > > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
> > > parsing
> > >
> > > On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
> > > <shameerali.kolothum.thodi@huawei.com> wrote:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > > > Sent: 06 September 2021 20:51
> > > > > To: Robin Murphy <robin.murphy@arm.com>
> > > > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali
> > > > > Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; Laurentiu
> > > > > Tudor <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > > > > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > > > <iommu@lists.linux-foundation.org>; Linuxarm
> > > > > <linuxarm@huawei.com>; Joerg Roedel <joro@8bytes.org>; Will
> > > > > Deacon <will@kernel.org>; wanghuiqiang
> > > > > <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > > > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>;
> > > > > Sami Mujawar <Sami.Mujawar@arm.com>; Eric Auger
> > > <eric.auger@redhat.com>;
> > > > > yangyicong <yangyicong@huawei.com>
> > > > > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
> > > > > parsing
> > > > >
> > > > [...]
> > > >
> > > > > > >
> > > > > > > On the prot value assignment based on the remapping flag,
> > > > > > > I'd like to hear Robin/Joerg's opinion, I'd avoid being in a
> > > > > > > situation where "normally" this would work but then we have
> > > > > > > to quirk
> > > it.
> > > > > > >
> > > > > > > Is this a valid assumption _always_ ?
> > > > > >
> > > > > > No. Certainly applying IOMMU_CACHE without reference to the
> > > > > > device's _CCA attribute or how CPUs may be accessing a shared
> > > > > > buffer could lead to a loss of coherency. At worst, applying
> > > > > > IOMMU_MMIO to a device-private buffer *could* cause the device
> > > > > > to lose coherency with itself if the memory underlying the RMR
> > > > > > may have allocated into system caches. Note that the expected
> > > > > > use for non-remappable RMRs is the device holding some sort of
> > > > > > long-lived private data in system RAM - the MSI doorbell trick
> > > > > > is far more of a niche
> > > hack really.
> > > > > >
> > > > > > At the very least I think we need to refer to the device's
> > > > > > memory access properties here.
> > > > > >
> > > > > > Jon, Laurentiu - how do RMRs correspond to the EFI memory map
> > > > > > on your firmware? I'm starting to think that as long as the
> > > > > > underlying memory is described appropriately there then we
> > > > > > should be able to infer correct attributes from the EFI memory type
> and flags.
> > > > >
> > > > > The devices are all cache coherent and marked as _CCA, 1.  The
> > > > > Memory regions are in the virt table as
> > > ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> > > > >
> > > > > The current chicken and egg problem we have is that during the
> > > > > fsl-mc-bus initialization we call
> > > > >
> > > > > error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
> > > > >                                               &mc_stream_id);
> > > > >
> > > > > which gets deferred because the SMMU has not been initialized yet.
> > > > > Then we initialize the RMR tables but there is no device
> > > > > reference there to be able to query device properties, only the stream
> id.
> > > > > After the IORT tables are parsed and the SMMU is setup, on the
> > > > > second device probe we associate everything based on the stream
> > > > > id and the fsl-mc-bus device is able to claim its 1-1 DMA mappings.
> > > >
> > > > Can we solve this order problem by delaying the
> > > > iommu_alloc_resv_region() to the
> > > > iommu_dma_get_rmr_resv_regions(dev,
> > > > list) ? We could invoke
> > > > device_get_dma_attr() from there which I believe will return the
> > > > _CCA
> > > attribute.
> > > >
> > > > Or is that still early to invoke that?
> > >
> > > That looks like it should work. Do we then also need to parse
> > > through the VirtualMemoryTable matching the start and end addresses
> > > to determine the other memory attributes like MMIO?
> >
> > Yes. But that looks tricky as I can't find that readily available on
> > Arm, like the efi_mem_attributes(). I will take a look.
> >
> > Please let me know if there is one or any other easy way to retrieve it.
> 
> maybe we don't need to.  Maybe it is enough to just move
> iommu_alloc_resv_regions and then set the IOMMU_CACHE flag if type =
> IOMMU_RESV_DIRECT_RELAXABLE and _CCN=1?

It looks like we could simply call efi_mem_type() and check for
EFI_MEMORY_MAPPED_IO. I have updated the code to set the
RMR prot value based on _CCA and EFI md type. Please see the 
last commit on this branch here(not tested),

https://github.com/hisilicon/kernel-dev/commits/private-v5.14-rc4-rmr-v7-ext

Please take a look and let me know if this is good enough to solve this problem.

Thanks,
Shameer

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-09-17 11:26                   ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-09-17 11:26 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Robin Murphy, Lorenzo Pieralisi, Laurentiu Tudor,
	linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU,
	Joerg Roedel, Will Deacon, wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong



> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 16 September 2021 12:17
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com>; Laurentiu Tudor <laurentiu.tudor@nxp.com>;
> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> List <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>; Will
> Deacon <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>;
> Guohanjun (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
> <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Eric
> Auger <eric.auger@redhat.com>; yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> 
> On Thu, Sep 16, 2021 at 10:26 AM Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > Sent: 16 September 2021 08:52
> > > To: Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>
> > > Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
> > > <lorenzo.pieralisi@arm.com>; Laurentiu Tudor
> > > <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>;
> > > Will Deacon <will@kernel.org>; wanghuiqiang
> > > <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> > > Mujawar <Sami.Mujawar@arm.com>; Eric Auger
> <eric.auger@redhat.com>;
> > > yangyicong <yangyicong@huawei.com>
> > > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
> > > parsing
> > >
> > > On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
> > > <shameerali.kolothum.thodi@huawei.com> wrote:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > > > Sent: 06 September 2021 20:51
> > > > > To: Robin Murphy <robin.murphy@arm.com>
> > > > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali
> > > > > Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; Laurentiu
> > > > > Tudor <laurentiu.tudor@nxp.com>; linux-arm-kernel
> > > > > <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
> > > > > <linux-acpi@vger.kernel.org>; Linux IOMMU
> > > > > <iommu@lists.linux-foundation.org>; Linuxarm
> > > > > <linuxarm@huawei.com>; Joerg Roedel <joro@8bytes.org>; Will
> > > > > Deacon <will@kernel.org>; wanghuiqiang
> > > > > <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > > > <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>;
> > > > > Sami Mujawar <Sami.Mujawar@arm.com>; Eric Auger
> > > <eric.auger@redhat.com>;
> > > > > yangyicong <yangyicong@huawei.com>
> > > > > Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
> > > > > parsing
> > > > >
> > > > [...]
> > > >
> > > > > > >
> > > > > > > On the prot value assignment based on the remapping flag,
> > > > > > > I'd like to hear Robin/Joerg's opinion, I'd avoid being in a
> > > > > > > situation where "normally" this would work but then we have
> > > > > > > to quirk
> > > it.
> > > > > > >
> > > > > > > Is this a valid assumption _always_ ?
> > > > > >
> > > > > > No. Certainly applying IOMMU_CACHE without reference to the
> > > > > > device's _CCA attribute or how CPUs may be accessing a shared
> > > > > > buffer could lead to a loss of coherency. At worst, applying
> > > > > > IOMMU_MMIO to a device-private buffer *could* cause the device
> > > > > > to lose coherency with itself if the memory underlying the RMR
> > > > > > may have allocated into system caches. Note that the expected
> > > > > > use for non-remappable RMRs is the device holding some sort of
> > > > > > long-lived private data in system RAM - the MSI doorbell trick
> > > > > > is far more of a niche
> > > hack really.
> > > > > >
> > > > > > At the very least I think we need to refer to the device's
> > > > > > memory access properties here.
> > > > > >
> > > > > > Jon, Laurentiu - how do RMRs correspond to the EFI memory map
> > > > > > on your firmware? I'm starting to think that as long as the
> > > > > > underlying memory is described appropriately there then we
> > > > > > should be able to infer correct attributes from the EFI memory type
> and flags.
> > > > >
> > > > > The devices are all cache coherent and marked as _CCA, 1.  The
> > > > > Memory regions are in the virt table as
> > > ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
> > > > >
> > > > > The current chicken and egg problem we have is that during the
> > > > > fsl-mc-bus initialization we call
> > > > >
> > > > > error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
> > > > >                                               &mc_stream_id);
> > > > >
> > > > > which gets deferred because the SMMU has not been initialized yet.
> > > > > Then we initialize the RMR tables but there is no device
> > > > > reference there to be able to query device properties, only the stream
> id.
> > > > > After the IORT tables are parsed and the SMMU is setup, on the
> > > > > second device probe we associate everything based on the stream
> > > > > id and the fsl-mc-bus device is able to claim its 1-1 DMA mappings.
> > > >
> > > > Can we solve this order problem by delaying the
> > > > iommu_alloc_resv_region() to the
> > > > iommu_dma_get_rmr_resv_regions(dev,
> > > > list) ? We could invoke
> > > > device_get_dma_attr() from there which I believe will return the
> > > > _CCA
> > > attribute.
> > > >
> > > > Or is that still early to invoke that?
> > >
> > > That looks like it should work. Do we then also need to parse
> > > through the VirtualMemoryTable matching the start and end addresses
> > > to determine the other memory attributes like MMIO?
> >
> > Yes. But that looks tricky as I can't find that readily available on
> > Arm, like the efi_mem_attributes(). I will take a look.
> >
> > Please let me know if there is one or any other easy way to retrieve it.
> 
> maybe we don't need to.  Maybe it is enough to just move
> iommu_alloc_resv_regions and then set the IOMMU_CACHE flag if type =
> IOMMU_RESV_DIRECT_RELAXABLE and _CCN=1?

It looks like we could simply call efi_mem_type() and check for
EFI_MEMORY_MAPPED_IO. I have updated the code to set the
RMR prot value based on _CCA and EFI md type. Please see the 
last commit on this branch here(not tested),

https://github.com/hisilicon/kernel-dev/commits/private-v5.14-rc4-rmr-v7-ext

Please take a look and let me know if this is good enough to solve this problem.

Thanks,
Shameer

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
  2021-08-05  8:07 ` Shameer Kolothum
  (?)
@ 2021-09-30  9:47   ` Eric Auger
  -1 siblings, 0 replies; 150+ messages in thread
From: Eric Auger @ 2021-09-30  9:47 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu,
	Jean-Philippe Brucker
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	yangyicong

Hi Shameer,

On 8/5/21 10:07 AM, Shameer Kolothum wrote:
> Hi,
>
> The series adds support to IORT RMR nodes specified in IORT
> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> memory ranges that are used by endpoints and require a unity
> mapping in SMMU.

I used your series and RMRs to force a guest iommu (vSMMUv3 nested stage
use case) to have a flat mapping for IOVAs within [0x8000000, 0x8100000]
(matching MSI_IOVA_BASE and MSI_IOVA_LENGTH) used by the host to map MSI
physical doorbells.

That way when an assigned device protected by a vSMMUv3 implemented upon
nested stage issues an MSI transaction, let's say using IOVA=0x8000000,
we would get:
                    S1 (guest)           S2 (host)
0x8000000            0x8000000            Physical DB

This method was suggested by Jean-Philippe (added in CC) and it
simplifies the nested stage integration because we don't have to care
about nested stage MSI bindings.

However if I understand correctly we cannot define a range of SIDs using
the same RMR (due to the single mapping bit which must be set, Table 5
flags format). This is a spec restriction and not an issue with your series.

As VFIO devices can be hot-plugged we thus need to create as many RMR
nodes as potential BDFs, leading to 256 * 6 = 1536 RMR nodes if you have
5 pcie root ports as it is usual in VMs. Then this causes some trouble
at qemu level for instance, wrt migration. See [RFC]
hw/arm/virt-acpi-build: Add IORT RMR regions to handle MSI nested binding.

Do you know if there is a plan to remove the single mapping limitation
in the spec?

Thanks

Eric
>
> We have faced issues with 3408iMR RAID controller cards which
> fail to boot when SMMU is enabled. This is because these
> controllers make use of host memory for various caching related
> purposes and when SMMU is enabled the iMR firmware fails to 
> access these memory regions as there is no mapping for them.
> IORT RMR provides a way for UEFI to describe and report these
> memory regions so that the kernel can make a unity mapping for
> these in SMMU.
>
> Change History: 
>
> v6 --> v7
>
> The only change from v6 is the fix pointed out by Steve to
> the SMMUv2 SMR bypass install in patch #8.
>
> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> yet as the series still needs more review[1].
>
> Feedback and tests on this series is very much appreciated.
>
> v5 --> v6
> - Addressed comments from Robin & Lorenzo.
>   : Moved iort_parse_rmr() to acpi_iort_init() from
>     iort_init_platform_devices().
>   : Removed use of struct iort_rmr_entry during the initial
>     parse. Using struct iommu_resv_region instead.
>   : Report RMR address alignment and overlap errors, but continue.
>   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
>   on Type of RMR region. Suggested by Jon N.
>
> Thanks,
> Shameer
> [0] https://developer.arm.com/documentation/den0049/latest/
> [1] https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.kolothum.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> ------
> v4 --> v5
>  -Added a fw_data union to struct iommu_resv_region and removed
>   struct iommu_rmr (Based on comments from Joerg/Robin).
>  -Added iommu_put_rmrs() to release mem.
>  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
>   yet because of the above changes.
>
> v3 -->v4
> -Included the SMMUv2 SMR bypass install changes suggested by
>  Steve(patch #7)
> -As per Robin's comments, RMR reserve implementation is now
>  more generic  (patch #8) and dropped v3 patches 8 and 10.
> -Rebase to 5.13-rc1
>
> RFC v2 --> v3
>  -Dropped RFC tag as the ACPICA header changes are now ready to be
>   part of 5.13[0]. But this series still has a dependency on that patch.
>  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
>   PCIe).
>  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
>   discussion here[1].
>  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> ------
>
> Jon Nettleton (1):
>   iommu/arm-smmu: Get associated RMR info and install bypass SMR
>
> Shameer Kolothum (8):
>   iommu: Introduce a union to struct iommu_resv_region
>   ACPI/IORT: Add support for RMR node parsing
>   iommu/dma: Introduce generic helper to retrieve RMR info
>   ACPI/IORT: Add a helper to retrieve RMR memory regions
>   iommu/arm-smmu-v3: Introduce strtab init helper
>   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
>     bypass
>   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
>   iommu/dma: Reserve any RMR regions associated with a dev
>
>  drivers/acpi/arm64/iort.c                   | 172 +++++++++++++++++++-
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
>  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
>  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
>  include/linux/acpi_iort.h                   |   7 +
>  include/linux/dma-iommu.h                   |  13 ++
>  include/linux/iommu.h                       |  11 ++
>  7 files changed, 393 insertions(+), 23 deletions(-)
>


^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-09-30  9:47   ` Eric Auger
  0 siblings, 0 replies; 150+ messages in thread
From: Eric Auger @ 2021-09-30  9:47 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu,
	Jean-Philippe Brucker
  Cc: robin.murphy, jon, linuxarm, steven.price, guohanjun, yangyicong,
	Sami.Mujawar, will, wanghuiqiang

Hi Shameer,

On 8/5/21 10:07 AM, Shameer Kolothum wrote:
> Hi,
>
> The series adds support to IORT RMR nodes specified in IORT
> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> memory ranges that are used by endpoints and require a unity
> mapping in SMMU.

I used your series and RMRs to force a guest iommu (vSMMUv3 nested stage
use case) to have a flat mapping for IOVAs within [0x8000000, 0x8100000]
(matching MSI_IOVA_BASE and MSI_IOVA_LENGTH) used by the host to map MSI
physical doorbells.

That way when an assigned device protected by a vSMMUv3 implemented upon
nested stage issues an MSI transaction, let's say using IOVA=0x8000000,
we would get:
                    S1 (guest)           S2 (host)
0x8000000            0x8000000            Physical DB

This method was suggested by Jean-Philippe (added in CC) and it
simplifies the nested stage integration because we don't have to care
about nested stage MSI bindings.

However if I understand correctly we cannot define a range of SIDs using
the same RMR (due to the single mapping bit which must be set, Table 5
flags format). This is a spec restriction and not an issue with your series.

As VFIO devices can be hot-plugged we thus need to create as many RMR
nodes as potential BDFs, leading to 256 * 6 = 1536 RMR nodes if you have
5 pcie root ports as it is usual in VMs. Then this causes some trouble
at qemu level for instance, wrt migration. See [RFC]
hw/arm/virt-acpi-build: Add IORT RMR regions to handle MSI nested binding.

Do you know if there is a plan to remove the single mapping limitation
in the spec?

Thanks

Eric
>
> We have faced issues with 3408iMR RAID controller cards which
> fail to boot when SMMU is enabled. This is because these
> controllers make use of host memory for various caching related
> purposes and when SMMU is enabled the iMR firmware fails to 
> access these memory regions as there is no mapping for them.
> IORT RMR provides a way for UEFI to describe and report these
> memory regions so that the kernel can make a unity mapping for
> these in SMMU.
>
> Change History: 
>
> v6 --> v7
>
> The only change from v6 is the fix pointed out by Steve to
> the SMMUv2 SMR bypass install in patch #8.
>
> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> yet as the series still needs more review[1].
>
> Feedback and tests on this series is very much appreciated.
>
> v5 --> v6
> - Addressed comments from Robin & Lorenzo.
>   : Moved iort_parse_rmr() to acpi_iort_init() from
>     iort_init_platform_devices().
>   : Removed use of struct iort_rmr_entry during the initial
>     parse. Using struct iommu_resv_region instead.
>   : Report RMR address alignment and overlap errors, but continue.
>   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
>   on Type of RMR region. Suggested by Jon N.
>
> Thanks,
> Shameer
> [0] https://developer.arm.com/documentation/den0049/latest/
> [1] https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.kolothum.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> ------
> v4 --> v5
>  -Added a fw_data union to struct iommu_resv_region and removed
>   struct iommu_rmr (Based on comments from Joerg/Robin).
>  -Added iommu_put_rmrs() to release mem.
>  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
>   yet because of the above changes.
>
> v3 -->v4
> -Included the SMMUv2 SMR bypass install changes suggested by
>  Steve(patch #7)
> -As per Robin's comments, RMR reserve implementation is now
>  more generic  (patch #8) and dropped v3 patches 8 and 10.
> -Rebase to 5.13-rc1
>
> RFC v2 --> v3
>  -Dropped RFC tag as the ACPICA header changes are now ready to be
>   part of 5.13[0]. But this series still has a dependency on that patch.
>  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
>   PCIe).
>  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
>   discussion here[1].
>  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> ------
>
> Jon Nettleton (1):
>   iommu/arm-smmu: Get associated RMR info and install bypass SMR
>
> Shameer Kolothum (8):
>   iommu: Introduce a union to struct iommu_resv_region
>   ACPI/IORT: Add support for RMR node parsing
>   iommu/dma: Introduce generic helper to retrieve RMR info
>   ACPI/IORT: Add a helper to retrieve RMR memory regions
>   iommu/arm-smmu-v3: Introduce strtab init helper
>   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
>     bypass
>   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
>   iommu/dma: Reserve any RMR regions associated with a dev
>
>  drivers/acpi/arm64/iort.c                   | 172 +++++++++++++++++++-
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
>  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
>  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
>  include/linux/acpi_iort.h                   |   7 +
>  include/linux/dma-iommu.h                   |  13 ++
>  include/linux/iommu.h                       |  11 ++
>  7 files changed, 393 insertions(+), 23 deletions(-)
>

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-09-30  9:47   ` Eric Auger
  0 siblings, 0 replies; 150+ messages in thread
From: Eric Auger @ 2021-09-30  9:47 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu,
	Jean-Philippe Brucker
  Cc: linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, guohanjun, steven.price, Sami.Mujawar, jon,
	yangyicong

Hi Shameer,

On 8/5/21 10:07 AM, Shameer Kolothum wrote:
> Hi,
>
> The series adds support to IORT RMR nodes specified in IORT
> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> memory ranges that are used by endpoints and require a unity
> mapping in SMMU.

I used your series and RMRs to force a guest iommu (vSMMUv3 nested stage
use case) to have a flat mapping for IOVAs within [0x8000000, 0x8100000]
(matching MSI_IOVA_BASE and MSI_IOVA_LENGTH) used by the host to map MSI
physical doorbells.

That way when an assigned device protected by a vSMMUv3 implemented upon
nested stage issues an MSI transaction, let's say using IOVA=0x8000000,
we would get:
                    S1 (guest)           S2 (host)
0x8000000            0x8000000            Physical DB

This method was suggested by Jean-Philippe (added in CC) and it
simplifies the nested stage integration because we don't have to care
about nested stage MSI bindings.

However if I understand correctly we cannot define a range of SIDs using
the same RMR (due to the single mapping bit which must be set, Table 5
flags format). This is a spec restriction and not an issue with your series.

As VFIO devices can be hot-plugged we thus need to create as many RMR
nodes as potential BDFs, leading to 256 * 6 = 1536 RMR nodes if you have
5 pcie root ports as it is usual in VMs. Then this causes some trouble
at qemu level for instance, wrt migration. See [RFC]
hw/arm/virt-acpi-build: Add IORT RMR regions to handle MSI nested binding.

Do you know if there is a plan to remove the single mapping limitation
in the spec?

Thanks

Eric
>
> We have faced issues with 3408iMR RAID controller cards which
> fail to boot when SMMU is enabled. This is because these
> controllers make use of host memory for various caching related
> purposes and when SMMU is enabled the iMR firmware fails to 
> access these memory regions as there is no mapping for them.
> IORT RMR provides a way for UEFI to describe and report these
> memory regions so that the kernel can make a unity mapping for
> these in SMMU.
>
> Change History: 
>
> v6 --> v7
>
> The only change from v6 is the fix pointed out by Steve to
> the SMMUv2 SMR bypass install in patch #8.
>
> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> yet as the series still needs more review[1].
>
> Feedback and tests on this series is very much appreciated.
>
> v5 --> v6
> - Addressed comments from Robin & Lorenzo.
>   : Moved iort_parse_rmr() to acpi_iort_init() from
>     iort_init_platform_devices().
>   : Removed use of struct iort_rmr_entry during the initial
>     parse. Using struct iommu_resv_region instead.
>   : Report RMR address alignment and overlap errors, but continue.
>   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
>   on Type of RMR region. Suggested by Jon N.
>
> Thanks,
> Shameer
> [0] https://developer.arm.com/documentation/den0049/latest/
> [1] https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.kolothum.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> ------
> v4 --> v5
>  -Added a fw_data union to struct iommu_resv_region and removed
>   struct iommu_rmr (Based on comments from Joerg/Robin).
>  -Added iommu_put_rmrs() to release mem.
>  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
>   yet because of the above changes.
>
> v3 -->v4
> -Included the SMMUv2 SMR bypass install changes suggested by
>  Steve(patch #7)
> -As per Robin's comments, RMR reserve implementation is now
>  more generic  (patch #8) and dropped v3 patches 8 and 10.
> -Rebase to 5.13-rc1
>
> RFC v2 --> v3
>  -Dropped RFC tag as the ACPICA header changes are now ready to be
>   part of 5.13[0]. But this series still has a dependency on that patch.
>  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
>   PCIe).
>  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
>   discussion here[1].
>  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> ------
>
> Jon Nettleton (1):
>   iommu/arm-smmu: Get associated RMR info and install bypass SMR
>
> Shameer Kolothum (8):
>   iommu: Introduce a union to struct iommu_resv_region
>   ACPI/IORT: Add support for RMR node parsing
>   iommu/dma: Introduce generic helper to retrieve RMR info
>   ACPI/IORT: Add a helper to retrieve RMR memory regions
>   iommu/arm-smmu-v3: Introduce strtab init helper
>   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
>     bypass
>   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
>   iommu/dma: Reserve any RMR regions associated with a dev
>
>  drivers/acpi/arm64/iort.c                   | 172 +++++++++++++++++++-
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
>  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
>  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
>  include/linux/acpi_iort.h                   |   7 +
>  include/linux/dma-iommu.h                   |  13 ++
>  include/linux/iommu.h                       |  11 ++
>  7 files changed, 393 insertions(+), 23 deletions(-)
>


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
  2021-09-30  9:47   ` Eric Auger
  (?)
@ 2021-09-30 10:50     ` Shameerali Kolothum Thodi
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-09-30 10:50 UTC (permalink / raw)
  To: eric.auger, linux-arm-kernel, linux-acpi, iommu, Jean-Philippe Brucker
  Cc: Linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	steven.price, Sami.Mujawar, jon, yangyicong

Hi Eric,

> -----Original Message-----
> From: Eric Auger [mailto:eric.auger@redhat.com]
> Sent: 30 September 2021 10:48
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org; Jean-Philippe Brucker
> <Jean-Philippe.Brucker@arm.com>
> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> joro@8bytes.org; robin.murphy@arm.com; will@kernel.org; wanghuiqiang
> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; steven.price@arm.com; Sami.Mujawar@arm.com;
> jon@solid-run.com; yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> 
> Hi Shameer,
> 
> On 8/5/21 10:07 AM, Shameer Kolothum wrote:
> > Hi,
> >
> > The series adds support to IORT RMR nodes specified in IORT
> > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> > memory ranges that are used by endpoints and require a unity
> > mapping in SMMU.
> 
> I used your series and RMRs to force a guest iommu (vSMMUv3 nested stage
> use case) to have a flat mapping for IOVAs within [0x8000000, 0x8100000]
> (matching MSI_IOVA_BASE and MSI_IOVA_LENGTH) used by the host to map
> MSI
> physical doorbells.
> 
> That way when an assigned device protected by a vSMMUv3 implemented
> upon
> nested stage issues an MSI transaction, let's say using IOVA=0x8000000,
> we would get:
>                     S1 (guest)           S2 (host)
> 0x8000000            0x8000000            Physical DB
> 
> This method was suggested by Jean-Philippe (added in CC) and it
> simplifies the nested stage integration because we don't have to care
> about nested stage MSI bindings.
> 
> However if I understand correctly we cannot define a range of SIDs using
> the same RMR (due to the single mapping bit which must be set, Table 5
> flags format). This is a spec restriction and not an issue with your series.

Yes. The spec currently mandates single mapping bit to be set.

> 
> As VFIO devices can be hot-plugged we thus need to create as many RMR
> nodes as potential BDFs, leading to 256 * 6 = 1536 RMR nodes if you have
> 5 pcie root ports as it is usual in VMs. Then this causes some trouble
> at qemu level for instance, wrt migration. See [RFC]
> hw/arm/virt-acpi-build: Add IORT RMR regions to handle MSI nested binding.
> 
> Do you know if there is a plan to remove the single mapping limitation
> in the spec?

I would imagine so. In an earlier comment[1], Robin did mention about possible
relaxing of this in future spec revision.

Thanks,
Shameer
1. https://lore.kernel.org/linux-arm-kernel/15c7fac0-11a8-0cdb-aac3-b5d8feb8f066@arm.com/

> Thanks
> 
> Eric
> >
> > We have faced issues with 3408iMR RAID controller cards which
> > fail to boot when SMMU is enabled. This is because these
> > controllers make use of host memory for various caching related
> > purposes and when SMMU is enabled the iMR firmware fails to
> > access these memory regions as there is no mapping for them.
> > IORT RMR provides a way for UEFI to describe and report these
> > memory regions so that the kernel can make a unity mapping for
> > these in SMMU.
> >
> > Change History:
> >
> > v6 --> v7
> >
> > The only change from v6 is the fix pointed out by Steve to
> > the SMMUv2 SMR bypass install in patch #8.
> >
> > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> > yet as the series still needs more review[1].
> >
> > Feedback and tests on this series is very much appreciated.
> >
> > v5 --> v6
> > - Addressed comments from Robin & Lorenzo.
> >   : Moved iort_parse_rmr() to acpi_iort_init() from
> >     iort_init_platform_devices().
> >   : Removed use of struct iort_rmr_entry during the initial
> >     parse. Using struct iommu_resv_region instead.
> >   : Report RMR address alignment and overlap errors, but continue.
> >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
> >   on Type of RMR region. Suggested by Jon N.
> >
> > Thanks,
> > Shameer
> > [0] https://developer.arm.com/documentation/den0049/latest/
> > [1]
> https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth
> um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> > ------
> > v4 --> v5
> >  -Added a fw_data union to struct iommu_resv_region and removed
> >   struct iommu_rmr (Based on comments from Joerg/Robin).
> >  -Added iommu_put_rmrs() to release mem.
> >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
> >   yet because of the above changes.
> >
> > v3 -->v4
> > -Included the SMMUv2 SMR bypass install changes suggested by
> >  Steve(patch #7)
> > -As per Robin's comments, RMR reserve implementation is now
> >  more generic  (patch #8) and dropped v3 patches 8 and 10.
> > -Rebase to 5.13-rc1
> >
> > RFC v2 --> v3
> >  -Dropped RFC tag as the ACPICA header changes are now ready to be
> >   part of 5.13[0]. But this series still has a dependency on that patch.
> >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
> >   PCIe).
> >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
> >   discussion here[1].
> >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> > ------
> >
> > Jon Nettleton (1):
> >   iommu/arm-smmu: Get associated RMR info and install bypass SMR
> >
> > Shameer Kolothum (8):
> >   iommu: Introduce a union to struct iommu_resv_region
> >   ACPI/IORT: Add support for RMR node parsing
> >   iommu/dma: Introduce generic helper to retrieve RMR info
> >   ACPI/IORT: Add a helper to retrieve RMR memory regions
> >   iommu/arm-smmu-v3: Introduce strtab init helper
> >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
> >     bypass
> >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> >   iommu/dma: Reserve any RMR regions associated with a dev
> >
> >  drivers/acpi/arm64/iort.c                   | 172
> +++++++++++++++++++-
> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
> >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
> >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
> >  include/linux/acpi_iort.h                   |   7 +
> >  include/linux/dma-iommu.h                   |  13 ++
> >  include/linux/iommu.h                       |  11 ++
> >  7 files changed, 393 insertions(+), 23 deletions(-)
> >


^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-09-30 10:50     ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-09-30 10:50 UTC (permalink / raw)
  To: eric.auger, linux-arm-kernel, linux-acpi, iommu, Jean-Philippe Brucker
  Cc: robin.murphy, jon, Linuxarm, steven.price, Guohanjun (Hanjun Guo),
	yangyicong, Sami.Mujawar, will, wanghuiqiang

Hi Eric,

> -----Original Message-----
> From: Eric Auger [mailto:eric.auger@redhat.com]
> Sent: 30 September 2021 10:48
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org; Jean-Philippe Brucker
> <Jean-Philippe.Brucker@arm.com>
> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> joro@8bytes.org; robin.murphy@arm.com; will@kernel.org; wanghuiqiang
> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; steven.price@arm.com; Sami.Mujawar@arm.com;
> jon@solid-run.com; yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> 
> Hi Shameer,
> 
> On 8/5/21 10:07 AM, Shameer Kolothum wrote:
> > Hi,
> >
> > The series adds support to IORT RMR nodes specified in IORT
> > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> > memory ranges that are used by endpoints and require a unity
> > mapping in SMMU.
> 
> I used your series and RMRs to force a guest iommu (vSMMUv3 nested stage
> use case) to have a flat mapping for IOVAs within [0x8000000, 0x8100000]
> (matching MSI_IOVA_BASE and MSI_IOVA_LENGTH) used by the host to map
> MSI
> physical doorbells.
> 
> That way when an assigned device protected by a vSMMUv3 implemented
> upon
> nested stage issues an MSI transaction, let's say using IOVA=0x8000000,
> we would get:
>                     S1 (guest)           S2 (host)
> 0x8000000            0x8000000            Physical DB
> 
> This method was suggested by Jean-Philippe (added in CC) and it
> simplifies the nested stage integration because we don't have to care
> about nested stage MSI bindings.
> 
> However if I understand correctly we cannot define a range of SIDs using
> the same RMR (due to the single mapping bit which must be set, Table 5
> flags format). This is a spec restriction and not an issue with your series.

Yes. The spec currently mandates single mapping bit to be set.

> 
> As VFIO devices can be hot-plugged we thus need to create as many RMR
> nodes as potential BDFs, leading to 256 * 6 = 1536 RMR nodes if you have
> 5 pcie root ports as it is usual in VMs. Then this causes some trouble
> at qemu level for instance, wrt migration. See [RFC]
> hw/arm/virt-acpi-build: Add IORT RMR regions to handle MSI nested binding.
> 
> Do you know if there is a plan to remove the single mapping limitation
> in the spec?

I would imagine so. In an earlier comment[1], Robin did mention about possible
relaxing of this in future spec revision.

Thanks,
Shameer
1. https://lore.kernel.org/linux-arm-kernel/15c7fac0-11a8-0cdb-aac3-b5d8feb8f066@arm.com/

> Thanks
> 
> Eric
> >
> > We have faced issues with 3408iMR RAID controller cards which
> > fail to boot when SMMU is enabled. This is because these
> > controllers make use of host memory for various caching related
> > purposes and when SMMU is enabled the iMR firmware fails to
> > access these memory regions as there is no mapping for them.
> > IORT RMR provides a way for UEFI to describe and report these
> > memory regions so that the kernel can make a unity mapping for
> > these in SMMU.
> >
> > Change History:
> >
> > v6 --> v7
> >
> > The only change from v6 is the fix pointed out by Steve to
> > the SMMUv2 SMR bypass install in patch #8.
> >
> > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> > yet as the series still needs more review[1].
> >
> > Feedback and tests on this series is very much appreciated.
> >
> > v5 --> v6
> > - Addressed comments from Robin & Lorenzo.
> >   : Moved iort_parse_rmr() to acpi_iort_init() from
> >     iort_init_platform_devices().
> >   : Removed use of struct iort_rmr_entry during the initial
> >     parse. Using struct iommu_resv_region instead.
> >   : Report RMR address alignment and overlap errors, but continue.
> >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
> >   on Type of RMR region. Suggested by Jon N.
> >
> > Thanks,
> > Shameer
> > [0] https://developer.arm.com/documentation/den0049/latest/
> > [1]
> https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth
> um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> > ------
> > v4 --> v5
> >  -Added a fw_data union to struct iommu_resv_region and removed
> >   struct iommu_rmr (Based on comments from Joerg/Robin).
> >  -Added iommu_put_rmrs() to release mem.
> >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
> >   yet because of the above changes.
> >
> > v3 -->v4
> > -Included the SMMUv2 SMR bypass install changes suggested by
> >  Steve(patch #7)
> > -As per Robin's comments, RMR reserve implementation is now
> >  more generic  (patch #8) and dropped v3 patches 8 and 10.
> > -Rebase to 5.13-rc1
> >
> > RFC v2 --> v3
> >  -Dropped RFC tag as the ACPICA header changes are now ready to be
> >   part of 5.13[0]. But this series still has a dependency on that patch.
> >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
> >   PCIe).
> >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
> >   discussion here[1].
> >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> > ------
> >
> > Jon Nettleton (1):
> >   iommu/arm-smmu: Get associated RMR info and install bypass SMR
> >
> > Shameer Kolothum (8):
> >   iommu: Introduce a union to struct iommu_resv_region
> >   ACPI/IORT: Add support for RMR node parsing
> >   iommu/dma: Introduce generic helper to retrieve RMR info
> >   ACPI/IORT: Add a helper to retrieve RMR memory regions
> >   iommu/arm-smmu-v3: Introduce strtab init helper
> >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
> >     bypass
> >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> >   iommu/dma: Reserve any RMR regions associated with a dev
> >
> >  drivers/acpi/arm64/iort.c                   | 172
> +++++++++++++++++++-
> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
> >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
> >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
> >  include/linux/acpi_iort.h                   |   7 +
> >  include/linux/dma-iommu.h                   |  13 ++
> >  include/linux/iommu.h                       |  11 ++
> >  7 files changed, 393 insertions(+), 23 deletions(-)
> >

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2021-09-30 10:50     ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-09-30 10:50 UTC (permalink / raw)
  To: eric.auger, linux-arm-kernel, linux-acpi, iommu, Jean-Philippe Brucker
  Cc: Linuxarm, lorenzo.pieralisi, joro, robin.murphy, will,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	steven.price, Sami.Mujawar, jon, yangyicong

Hi Eric,

> -----Original Message-----
> From: Eric Auger [mailto:eric.auger@redhat.com]
> Sent: 30 September 2021 10:48
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org; Jean-Philippe Brucker
> <Jean-Philippe.Brucker@arm.com>
> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> joro@8bytes.org; robin.murphy@arm.com; will@kernel.org; wanghuiqiang
> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; steven.price@arm.com; Sami.Mujawar@arm.com;
> jon@solid-run.com; yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> 
> Hi Shameer,
> 
> On 8/5/21 10:07 AM, Shameer Kolothum wrote:
> > Hi,
> >
> > The series adds support to IORT RMR nodes specified in IORT
> > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> > memory ranges that are used by endpoints and require a unity
> > mapping in SMMU.
> 
> I used your series and RMRs to force a guest iommu (vSMMUv3 nested stage
> use case) to have a flat mapping for IOVAs within [0x8000000, 0x8100000]
> (matching MSI_IOVA_BASE and MSI_IOVA_LENGTH) used by the host to map
> MSI
> physical doorbells.
> 
> That way when an assigned device protected by a vSMMUv3 implemented
> upon
> nested stage issues an MSI transaction, let's say using IOVA=0x8000000,
> we would get:
>                     S1 (guest)           S2 (host)
> 0x8000000            0x8000000            Physical DB
> 
> This method was suggested by Jean-Philippe (added in CC) and it
> simplifies the nested stage integration because we don't have to care
> about nested stage MSI bindings.
> 
> However if I understand correctly we cannot define a range of SIDs using
> the same RMR (due to the single mapping bit which must be set, Table 5
> flags format). This is a spec restriction and not an issue with your series.

Yes. The spec currently mandates single mapping bit to be set.

> 
> As VFIO devices can be hot-plugged we thus need to create as many RMR
> nodes as potential BDFs, leading to 256 * 6 = 1536 RMR nodes if you have
> 5 pcie root ports as it is usual in VMs. Then this causes some trouble
> at qemu level for instance, wrt migration. See [RFC]
> hw/arm/virt-acpi-build: Add IORT RMR regions to handle MSI nested binding.
> 
> Do you know if there is a plan to remove the single mapping limitation
> in the spec?

I would imagine so. In an earlier comment[1], Robin did mention about possible
relaxing of this in future spec revision.

Thanks,
Shameer
1. https://lore.kernel.org/linux-arm-kernel/15c7fac0-11a8-0cdb-aac3-b5d8feb8f066@arm.com/

> Thanks
> 
> Eric
> >
> > We have faced issues with 3408iMR RAID controller cards which
> > fail to boot when SMMU is enabled. This is because these
> > controllers make use of host memory for various caching related
> > purposes and when SMMU is enabled the iMR firmware fails to
> > access these memory regions as there is no mapping for them.
> > IORT RMR provides a way for UEFI to describe and report these
> > memory regions so that the kernel can make a unity mapping for
> > these in SMMU.
> >
> > Change History:
> >
> > v6 --> v7
> >
> > The only change from v6 is the fix pointed out by Steve to
> > the SMMUv2 SMR bypass install in patch #8.
> >
> > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> > yet as the series still needs more review[1].
> >
> > Feedback and tests on this series is very much appreciated.
> >
> > v5 --> v6
> > - Addressed comments from Robin & Lorenzo.
> >   : Moved iort_parse_rmr() to acpi_iort_init() from
> >     iort_init_platform_devices().
> >   : Removed use of struct iort_rmr_entry during the initial
> >     parse. Using struct iommu_resv_region instead.
> >   : Report RMR address alignment and overlap errors, but continue.
> >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).
> > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
> > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
> >   on Type of RMR region. Suggested by Jon N.
> >
> > Thanks,
> > Shameer
> > [0] https://developer.arm.com/documentation/den0049/latest/
> > [1]
> https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth
> um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
> > ------
> > v4 --> v5
> >  -Added a fw_data union to struct iommu_resv_region and removed
> >   struct iommu_rmr (Based on comments from Joerg/Robin).
> >  -Added iommu_put_rmrs() to release mem.
> >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
> >   yet because of the above changes.
> >
> > v3 -->v4
> > -Included the SMMUv2 SMR bypass install changes suggested by
> >  Steve(patch #7)
> > -As per Robin's comments, RMR reserve implementation is now
> >  more generic  (patch #8) and dropped v3 patches 8 and 10.
> > -Rebase to 5.13-rc1
> >
> > RFC v2 --> v3
> >  -Dropped RFC tag as the ACPICA header changes are now ready to be
> >   part of 5.13[0]. But this series still has a dependency on that patch.
> >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for
> >   PCIe).
> >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
> >   discussion here[1].
> >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
> > ------
> >
> > Jon Nettleton (1):
> >   iommu/arm-smmu: Get associated RMR info and install bypass SMR
> >
> > Shameer Kolothum (8):
> >   iommu: Introduce a union to struct iommu_resv_region
> >   ACPI/IORT: Add support for RMR node parsing
> >   iommu/dma: Introduce generic helper to retrieve RMR info
> >   ACPI/IORT: Add a helper to retrieve RMR memory regions
> >   iommu/arm-smmu-v3: Introduce strtab init helper
> >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
> >     bypass
> >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> >   iommu/dma: Reserve any RMR regions associated with a dev
> >
> >  drivers/acpi/arm64/iort.c                   | 172
> +++++++++++++++++++-
> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
> >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
> >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-
> >  include/linux/acpi_iort.h                   |   7 +
> >  include/linux/dma-iommu.h                   |  13 ++
> >  include/linux/iommu.h                       |  11 ++
> >  7 files changed, 393 insertions(+), 23 deletions(-)
> >

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-09-17 11:26                   ` Shameerali Kolothum Thodi
  (?)
@ 2021-10-05 10:53                     ` Laurentiu Tudor
  -1 siblings, 0 replies; 150+ messages in thread
From: Laurentiu Tudor @ 2021-10-05 10:53 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi, Jon Nettleton
  Cc: Robin Murphy, Lorenzo Pieralisi, linux-arm-kernel,
	ACPI Devel Maling List, Linux IOMMU, Joerg Roedel, Will Deacon,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong



On 9/17/2021 2:26 PM, Shameerali Kolothum Thodi wrote:
> 
> 
>> -----Original Message-----
>> From: Jon Nettleton [mailto:jon@solid-run.com]
>> Sent: 16 September 2021 12:17
>> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
>> Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
>> <lorenzo.pieralisi@arm.com>; Laurentiu Tudor <laurentiu.tudor@nxp.com>;
>> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
>> List <linux-acpi@vger.kernel.org>; Linux IOMMU
>> <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>; Will
>> Deacon <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>;
>> Guohanjun (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
>> <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Eric
>> Auger <eric.auger@redhat.com>; yangyicong <yangyicong@huawei.com>
>> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
>>
>> On Thu, Sep 16, 2021 at 10:26 AM Shameerali Kolothum Thodi
>> <shameerali.kolothum.thodi@huawei.com> wrote:
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Jon Nettleton [mailto:jon@solid-run.com]
>>>> Sent: 16 September 2021 08:52
>>>> To: Shameerali Kolothum Thodi
>> <shameerali.kolothum.thodi@huawei.com>
>>>> Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
>>>> <lorenzo.pieralisi@arm.com>; Laurentiu Tudor
>>>> <laurentiu.tudor@nxp.com>; linux-arm-kernel
>>>> <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
>>>> <linux-acpi@vger.kernel.org>; Linux IOMMU
>>>> <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>;
>>>> Will Deacon <will@kernel.org>; wanghuiqiang
>>>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
>>>> <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
>>>> Mujawar <Sami.Mujawar@arm.com>; Eric Auger
>> <eric.auger@redhat.com>;
>>>> yangyicong <yangyicong@huawei.com>
>>>> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
>>>> parsing
>>>>
>>>> On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
>>>> <shameerali.kolothum.thodi@huawei.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Jon Nettleton [mailto:jon@solid-run.com]
>>>>>> Sent: 06 September 2021 20:51
>>>>>> To: Robin Murphy <robin.murphy@arm.com>
>>>>>> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali
>>>>>> Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; Laurentiu
>>>>>> Tudor <laurentiu.tudor@nxp.com>; linux-arm-kernel
>>>>>> <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
>>>>>> <linux-acpi@vger.kernel.org>; Linux IOMMU
>>>>>> <iommu@lists.linux-foundation.org>; Linuxarm
>>>>>> <linuxarm@huawei.com>; Joerg Roedel <joro@8bytes.org>; Will
>>>>>> Deacon <will@kernel.org>; wanghuiqiang
>>>>>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
>>>>>> <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>;
>>>>>> Sami Mujawar <Sami.Mujawar@arm.com>; Eric Auger
>>>> <eric.auger@redhat.com>;
>>>>>> yangyicong <yangyicong@huawei.com>
>>>>>> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
>>>>>> parsing
>>>>>>
>>>>> [...]
>>>>>
>>>>>>>>
>>>>>>>> On the prot value assignment based on the remapping flag,
>>>>>>>> I'd like to hear Robin/Joerg's opinion, I'd avoid being in a
>>>>>>>> situation where "normally" this would work but then we have
>>>>>>>> to quirk
>>>> it.
>>>>>>>>
>>>>>>>> Is this a valid assumption _always_ ?
>>>>>>>
>>>>>>> No. Certainly applying IOMMU_CACHE without reference to the
>>>>>>> device's _CCA attribute or how CPUs may be accessing a shared
>>>>>>> buffer could lead to a loss of coherency. At worst, applying
>>>>>>> IOMMU_MMIO to a device-private buffer *could* cause the device
>>>>>>> to lose coherency with itself if the memory underlying the RMR
>>>>>>> may have allocated into system caches. Note that the expected
>>>>>>> use for non-remappable RMRs is the device holding some sort of
>>>>>>> long-lived private data in system RAM - the MSI doorbell trick
>>>>>>> is far more of a niche
>>>> hack really.
>>>>>>>
>>>>>>> At the very least I think we need to refer to the device's
>>>>>>> memory access properties here.
>>>>>>>
>>>>>>> Jon, Laurentiu - how do RMRs correspond to the EFI memory map
>>>>>>> on your firmware? I'm starting to think that as long as the
>>>>>>> underlying memory is described appropriately there then we
>>>>>>> should be able to infer correct attributes from the EFI memory type
>> and flags.
>>>>>>
>>>>>> The devices are all cache coherent and marked as _CCA, 1.  The
>>>>>> Memory regions are in the virt table as
>>>> ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
>>>>>>
>>>>>> The current chicken and egg problem we have is that during the
>>>>>> fsl-mc-bus initialization we call
>>>>>>
>>>>>> error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
>>>>>>                                               &mc_stream_id);
>>>>>>
>>>>>> which gets deferred because the SMMU has not been initialized yet.
>>>>>> Then we initialize the RMR tables but there is no device
>>>>>> reference there to be able to query device properties, only the stream
>> id.
>>>>>> After the IORT tables are parsed and the SMMU is setup, on the
>>>>>> second device probe we associate everything based on the stream
>>>>>> id and the fsl-mc-bus device is able to claim its 1-1 DMA mappings.
>>>>>
>>>>> Can we solve this order problem by delaying the
>>>>> iommu_alloc_resv_region() to the
>>>>> iommu_dma_get_rmr_resv_regions(dev,
>>>>> list) ? We could invoke
>>>>> device_get_dma_attr() from there which I believe will return the
>>>>> _CCA
>>>> attribute.
>>>>>
>>>>> Or is that still early to invoke that?
>>>>
>>>> That looks like it should work. Do we then also need to parse
>>>> through the VirtualMemoryTable matching the start and end addresses
>>>> to determine the other memory attributes like MMIO?
>>>
>>> Yes. But that looks tricky as I can't find that readily available on
>>> Arm, like the efi_mem_attributes(). I will take a look.
>>>
>>> Please let me know if there is one or any other easy way to retrieve it.
>>
>> maybe we don't need to.  Maybe it is enough to just move
>> iommu_alloc_resv_regions and then set the IOMMU_CACHE flag if type =
>> IOMMU_RESV_DIRECT_RELAXABLE and _CCN=1?
> 
> It looks like we could simply call efi_mem_type() and check for
> EFI_MEMORY_MAPPED_IO. I have updated the code to set the
> RMR prot value based on _CCA and EFI md type. Please see the 
> last commit on this branch here(not tested),
> 
> https://github.com/hisilicon/kernel-dev/commits/private-v5.14-rc4-rmr-v7-ext
> 
> Please take a look and let me know if this is good enough to solve this problem.
> 

Sorry for the delay, I managed to test on a NXP LX2160A and things look
fine, so:

Tested-by: Laurentiu Tudor <laurentiu.tudor@nxp.com>

---
Best Regards, Laurentiu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-10-05 10:53                     ` Laurentiu Tudor
  0 siblings, 0 replies; 150+ messages in thread
From: Laurentiu Tudor @ 2021-10-05 10:53 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi, Jon Nettleton
  Cc: Will Deacon, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Robin Murphy, linux-arm-kernel



On 9/17/2021 2:26 PM, Shameerali Kolothum Thodi wrote:
> 
> 
>> -----Original Message-----
>> From: Jon Nettleton [mailto:jon@solid-run.com]
>> Sent: 16 September 2021 12:17
>> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
>> Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
>> <lorenzo.pieralisi@arm.com>; Laurentiu Tudor <laurentiu.tudor@nxp.com>;
>> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
>> List <linux-acpi@vger.kernel.org>; Linux IOMMU
>> <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>; Will
>> Deacon <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>;
>> Guohanjun (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
>> <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Eric
>> Auger <eric.auger@redhat.com>; yangyicong <yangyicong@huawei.com>
>> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
>>
>> On Thu, Sep 16, 2021 at 10:26 AM Shameerali Kolothum Thodi
>> <shameerali.kolothum.thodi@huawei.com> wrote:
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Jon Nettleton [mailto:jon@solid-run.com]
>>>> Sent: 16 September 2021 08:52
>>>> To: Shameerali Kolothum Thodi
>> <shameerali.kolothum.thodi@huawei.com>
>>>> Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
>>>> <lorenzo.pieralisi@arm.com>; Laurentiu Tudor
>>>> <laurentiu.tudor@nxp.com>; linux-arm-kernel
>>>> <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
>>>> <linux-acpi@vger.kernel.org>; Linux IOMMU
>>>> <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>;
>>>> Will Deacon <will@kernel.org>; wanghuiqiang
>>>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
>>>> <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
>>>> Mujawar <Sami.Mujawar@arm.com>; Eric Auger
>> <eric.auger@redhat.com>;
>>>> yangyicong <yangyicong@huawei.com>
>>>> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
>>>> parsing
>>>>
>>>> On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
>>>> <shameerali.kolothum.thodi@huawei.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Jon Nettleton [mailto:jon@solid-run.com]
>>>>>> Sent: 06 September 2021 20:51
>>>>>> To: Robin Murphy <robin.murphy@arm.com>
>>>>>> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali
>>>>>> Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; Laurentiu
>>>>>> Tudor <laurentiu.tudor@nxp.com>; linux-arm-kernel
>>>>>> <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
>>>>>> <linux-acpi@vger.kernel.org>; Linux IOMMU
>>>>>> <iommu@lists.linux-foundation.org>; Linuxarm
>>>>>> <linuxarm@huawei.com>; Joerg Roedel <joro@8bytes.org>; Will
>>>>>> Deacon <will@kernel.org>; wanghuiqiang
>>>>>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
>>>>>> <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>;
>>>>>> Sami Mujawar <Sami.Mujawar@arm.com>; Eric Auger
>>>> <eric.auger@redhat.com>;
>>>>>> yangyicong <yangyicong@huawei.com>
>>>>>> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
>>>>>> parsing
>>>>>>
>>>>> [...]
>>>>>
>>>>>>>>
>>>>>>>> On the prot value assignment based on the remapping flag,
>>>>>>>> I'd like to hear Robin/Joerg's opinion, I'd avoid being in a
>>>>>>>> situation where "normally" this would work but then we have
>>>>>>>> to quirk
>>>> it.
>>>>>>>>
>>>>>>>> Is this a valid assumption _always_ ?
>>>>>>>
>>>>>>> No. Certainly applying IOMMU_CACHE without reference to the
>>>>>>> device's _CCA attribute or how CPUs may be accessing a shared
>>>>>>> buffer could lead to a loss of coherency. At worst, applying
>>>>>>> IOMMU_MMIO to a device-private buffer *could* cause the device
>>>>>>> to lose coherency with itself if the memory underlying the RMR
>>>>>>> may have allocated into system caches. Note that the expected
>>>>>>> use for non-remappable RMRs is the device holding some sort of
>>>>>>> long-lived private data in system RAM - the MSI doorbell trick
>>>>>>> is far more of a niche
>>>> hack really.
>>>>>>>
>>>>>>> At the very least I think we need to refer to the device's
>>>>>>> memory access properties here.
>>>>>>>
>>>>>>> Jon, Laurentiu - how do RMRs correspond to the EFI memory map
>>>>>>> on your firmware? I'm starting to think that as long as the
>>>>>>> underlying memory is described appropriately there then we
>>>>>>> should be able to infer correct attributes from the EFI memory type
>> and flags.
>>>>>>
>>>>>> The devices are all cache coherent and marked as _CCA, 1.  The
>>>>>> Memory regions are in the virt table as
>>>> ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
>>>>>>
>>>>>> The current chicken and egg problem we have is that during the
>>>>>> fsl-mc-bus initialization we call
>>>>>>
>>>>>> error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
>>>>>>                                               &mc_stream_id);
>>>>>>
>>>>>> which gets deferred because the SMMU has not been initialized yet.
>>>>>> Then we initialize the RMR tables but there is no device
>>>>>> reference there to be able to query device properties, only the stream
>> id.
>>>>>> After the IORT tables are parsed and the SMMU is setup, on the
>>>>>> second device probe we associate everything based on the stream
>>>>>> id and the fsl-mc-bus device is able to claim its 1-1 DMA mappings.
>>>>>
>>>>> Can we solve this order problem by delaying the
>>>>> iommu_alloc_resv_region() to the
>>>>> iommu_dma_get_rmr_resv_regions(dev,
>>>>> list) ? We could invoke
>>>>> device_get_dma_attr() from there which I believe will return the
>>>>> _CCA
>>>> attribute.
>>>>>
>>>>> Or is that still early to invoke that?
>>>>
>>>> That looks like it should work. Do we then also need to parse
>>>> through the VirtualMemoryTable matching the start and end addresses
>>>> to determine the other memory attributes like MMIO?
>>>
>>> Yes. But that looks tricky as I can't find that readily available on
>>> Arm, like the efi_mem_attributes(). I will take a look.
>>>
>>> Please let me know if there is one or any other easy way to retrieve it.
>>
>> maybe we don't need to.  Maybe it is enough to just move
>> iommu_alloc_resv_regions and then set the IOMMU_CACHE flag if type =
>> IOMMU_RESV_DIRECT_RELAXABLE and _CCN=1?
> 
> It looks like we could simply call efi_mem_type() and check for
> EFI_MEMORY_MAPPED_IO. I have updated the code to set the
> RMR prot value based on _CCA and EFI md type. Please see the 
> last commit on this branch here(not tested),
> 
> https://github.com/hisilicon/kernel-dev/commits/private-v5.14-rc4-rmr-v7-ext
> 
> Please take a look and let me know if this is good enough to solve this problem.
> 

Sorry for the delay, I managed to test on a NXP LX2160A and things look
fine, so:

Tested-by: Laurentiu Tudor <laurentiu.tudor@nxp.com>

---
Best Regards, Laurentiu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-10-05 10:53                     ` Laurentiu Tudor
  0 siblings, 0 replies; 150+ messages in thread
From: Laurentiu Tudor @ 2021-10-05 10:53 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi, Jon Nettleton
  Cc: Robin Murphy, Lorenzo Pieralisi, linux-arm-kernel,
	ACPI Devel Maling List, Linux IOMMU, Joerg Roedel, Will Deacon,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong



On 9/17/2021 2:26 PM, Shameerali Kolothum Thodi wrote:
> 
> 
>> -----Original Message-----
>> From: Jon Nettleton [mailto:jon@solid-run.com]
>> Sent: 16 September 2021 12:17
>> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
>> Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
>> <lorenzo.pieralisi@arm.com>; Laurentiu Tudor <laurentiu.tudor@nxp.com>;
>> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
>> List <linux-acpi@vger.kernel.org>; Linux IOMMU
>> <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>; Will
>> Deacon <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>;
>> Guohanjun (Hanjun Guo) <guohanjun@huawei.com>; Steven Price
>> <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Eric
>> Auger <eric.auger@redhat.com>; yangyicong <yangyicong@huawei.com>
>> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
>>
>> On Thu, Sep 16, 2021 at 10:26 AM Shameerali Kolothum Thodi
>> <shameerali.kolothum.thodi@huawei.com> wrote:
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Jon Nettleton [mailto:jon@solid-run.com]
>>>> Sent: 16 September 2021 08:52
>>>> To: Shameerali Kolothum Thodi
>> <shameerali.kolothum.thodi@huawei.com>
>>>> Cc: Robin Murphy <robin.murphy@arm.com>; Lorenzo Pieralisi
>>>> <lorenzo.pieralisi@arm.com>; Laurentiu Tudor
>>>> <laurentiu.tudor@nxp.com>; linux-arm-kernel
>>>> <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
>>>> <linux-acpi@vger.kernel.org>; Linux IOMMU
>>>> <iommu@lists.linux-foundation.org>; Joerg Roedel <joro@8bytes.org>;
>>>> Will Deacon <will@kernel.org>; wanghuiqiang
>>>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
>>>> <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
>>>> Mujawar <Sami.Mujawar@arm.com>; Eric Auger
>> <eric.auger@redhat.com>;
>>>> yangyicong <yangyicong@huawei.com>
>>>> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
>>>> parsing
>>>>
>>>> On Thu, Sep 16, 2021 at 9:26 AM Shameerali Kolothum Thodi
>>>> <shameerali.kolothum.thodi@huawei.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Jon Nettleton [mailto:jon@solid-run.com]
>>>>>> Sent: 06 September 2021 20:51
>>>>>> To: Robin Murphy <robin.murphy@arm.com>
>>>>>> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Shameerali
>>>>>> Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; Laurentiu
>>>>>> Tudor <laurentiu.tudor@nxp.com>; linux-arm-kernel
>>>>>> <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List
>>>>>> <linux-acpi@vger.kernel.org>; Linux IOMMU
>>>>>> <iommu@lists.linux-foundation.org>; Linuxarm
>>>>>> <linuxarm@huawei.com>; Joerg Roedel <joro@8bytes.org>; Will
>>>>>> Deacon <will@kernel.org>; wanghuiqiang
>>>>>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
>>>>>> <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>;
>>>>>> Sami Mujawar <Sami.Mujawar@arm.com>; Eric Auger
>>>> <eric.auger@redhat.com>;
>>>>>> yangyicong <yangyicong@huawei.com>
>>>>>> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node
>>>>>> parsing
>>>>>>
>>>>> [...]
>>>>>
>>>>>>>>
>>>>>>>> On the prot value assignment based on the remapping flag,
>>>>>>>> I'd like to hear Robin/Joerg's opinion, I'd avoid being in a
>>>>>>>> situation where "normally" this would work but then we have
>>>>>>>> to quirk
>>>> it.
>>>>>>>>
>>>>>>>> Is this a valid assumption _always_ ?
>>>>>>>
>>>>>>> No. Certainly applying IOMMU_CACHE without reference to the
>>>>>>> device's _CCA attribute or how CPUs may be accessing a shared
>>>>>>> buffer could lead to a loss of coherency. At worst, applying
>>>>>>> IOMMU_MMIO to a device-private buffer *could* cause the device
>>>>>>> to lose coherency with itself if the memory underlying the RMR
>>>>>>> may have allocated into system caches. Note that the expected
>>>>>>> use for non-remappable RMRs is the device holding some sort of
>>>>>>> long-lived private data in system RAM - the MSI doorbell trick
>>>>>>> is far more of a niche
>>>> hack really.
>>>>>>>
>>>>>>> At the very least I think we need to refer to the device's
>>>>>>> memory access properties here.
>>>>>>>
>>>>>>> Jon, Laurentiu - how do RMRs correspond to the EFI memory map
>>>>>>> on your firmware? I'm starting to think that as long as the
>>>>>>> underlying memory is described appropriately there then we
>>>>>>> should be able to infer correct attributes from the EFI memory type
>> and flags.
>>>>>>
>>>>>> The devices are all cache coherent and marked as _CCA, 1.  The
>>>>>> Memory regions are in the virt table as
>>>> ARM_MEMORY_REGION_ATTRIBUTE_DEVICE.
>>>>>>
>>>>>> The current chicken and egg problem we have is that during the
>>>>>> fsl-mc-bus initialization we call
>>>>>>
>>>>>> error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
>>>>>>                                               &mc_stream_id);
>>>>>>
>>>>>> which gets deferred because the SMMU has not been initialized yet.
>>>>>> Then we initialize the RMR tables but there is no device
>>>>>> reference there to be able to query device properties, only the stream
>> id.
>>>>>> After the IORT tables are parsed and the SMMU is setup, on the
>>>>>> second device probe we associate everything based on the stream
>>>>>> id and the fsl-mc-bus device is able to claim its 1-1 DMA mappings.
>>>>>
>>>>> Can we solve this order problem by delaying the
>>>>> iommu_alloc_resv_region() to the
>>>>> iommu_dma_get_rmr_resv_regions(dev,
>>>>> list) ? We could invoke
>>>>> device_get_dma_attr() from there which I believe will return the
>>>>> _CCA
>>>> attribute.
>>>>>
>>>>> Or is that still early to invoke that?
>>>>
>>>> That looks like it should work. Do we then also need to parse
>>>> through the VirtualMemoryTable matching the start and end addresses
>>>> to determine the other memory attributes like MMIO?
>>>
>>> Yes. But that looks tricky as I can't find that readily available on
>>> Arm, like the efi_mem_attributes(). I will take a look.
>>>
>>> Please let me know if there is one or any other easy way to retrieve it.
>>
>> maybe we don't need to.  Maybe it is enough to just move
>> iommu_alloc_resv_regions and then set the IOMMU_CACHE flag if type =
>> IOMMU_RESV_DIRECT_RELAXABLE and _CCN=1?
> 
> It looks like we could simply call efi_mem_type() and check for
> EFI_MEMORY_MAPPED_IO. I have updated the code to set the
> RMR prot value based on _CCA and EFI md type. Please see the 
> last commit on this branch here(not tested),
> 
> https://github.com/hisilicon/kernel-dev/commits/private-v5.14-rc4-rmr-v7-ext
> 
> Please take a look and let me know if this is good enough to solve this problem.
> 

Sorry for the delay, I managed to test on a NXP LX2160A and things look
fine, so:

Tested-by: Laurentiu Tudor <laurentiu.tudor@nxp.com>

---
Best Regards, Laurentiu

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
  2021-08-05  8:07   ` Shameer Kolothum
  (?)
@ 2021-10-08 12:14     ` Robin Murphy
  -1 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-08 12:14 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: jon, linuxarm, steven.price, guohanjun, yangyicong, Sami.Mujawar,
	will, wanghuiqiang

On 2021-08-05 09:07, Shameer Kolothum wrote:
> A union is introduced to struct iommu_resv_region to hold
> any firmware specific data. This is in preparation to add
> support for IORT RMR reserve regions and the union now holds
> the RMR specific information.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>   include/linux/iommu.h | 11 +++++++++++
>   1 file changed, 11 insertions(+)
> 
> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> index 32d448050bf7..bd0e4641c569 100644
> --- a/include/linux/iommu.h
> +++ b/include/linux/iommu.h
> @@ -114,6 +114,13 @@ enum iommu_resv_type {
>   	IOMMU_RESV_SW_MSI,
>   };
>   
> +struct iommu_iort_rmr_data {
> +#define IOMMU_RMR_REMAP_PERMITTED	(1 << 0)
> +	u32 flags;
> +	u32 sid;	/* Stream Id associated with RMR entry */
> +	void *smmu;	/* Associated IORT SMMU node pointer */
> +};

Do we really need to duplicate all this data? AFAICS we could just save 
the acpi_iort_rmr pointer in the iommu_resv_region (with a forward 
declaration here if necessary) and defer parsing its actual mappings 
until the point where we can directly consume the results.

Robin.

> +
>   /**
>    * struct iommu_resv_region - descriptor for a reserved memory region
>    * @list: Linked list pointers
> @@ -121,6 +128,7 @@ enum iommu_resv_type {
>    * @length: Length of the region in bytes
>    * @prot: IOMMU Protection flags (READ/WRITE/...)
>    * @type: Type of the reserved region
> + * @rmr: ACPI IORT RMR specific data
>    */
>   struct iommu_resv_region {
>   	struct list_head	list;
> @@ -128,6 +136,9 @@ struct iommu_resv_region {
>   	size_t			length;
>   	int			prot;
>   	enum iommu_resv_type	type;
> +	union {
> +		struct iommu_iort_rmr_data rmr;
> +	} fw_data;
>   };
>   
>   /**
> 

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
@ 2021-10-08 12:14     ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-08 12:14 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: jon, linuxarm, steven.price, guohanjun, yangyicong, Sami.Mujawar,
	will, wanghuiqiang

On 2021-08-05 09:07, Shameer Kolothum wrote:
> A union is introduced to struct iommu_resv_region to hold
> any firmware specific data. This is in preparation to add
> support for IORT RMR reserve regions and the union now holds
> the RMR specific information.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>   include/linux/iommu.h | 11 +++++++++++
>   1 file changed, 11 insertions(+)
> 
> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> index 32d448050bf7..bd0e4641c569 100644
> --- a/include/linux/iommu.h
> +++ b/include/linux/iommu.h
> @@ -114,6 +114,13 @@ enum iommu_resv_type {
>   	IOMMU_RESV_SW_MSI,
>   };
>   
> +struct iommu_iort_rmr_data {
> +#define IOMMU_RMR_REMAP_PERMITTED	(1 << 0)
> +	u32 flags;
> +	u32 sid;	/* Stream Id associated with RMR entry */
> +	void *smmu;	/* Associated IORT SMMU node pointer */
> +};

Do we really need to duplicate all this data? AFAICS we could just save 
the acpi_iort_rmr pointer in the iommu_resv_region (with a forward 
declaration here if necessary) and defer parsing its actual mappings 
until the point where we can directly consume the results.

Robin.

> +
>   /**
>    * struct iommu_resv_region - descriptor for a reserved memory region
>    * @list: Linked list pointers
> @@ -121,6 +128,7 @@ enum iommu_resv_type {
>    * @length: Length of the region in bytes
>    * @prot: IOMMU Protection flags (READ/WRITE/...)
>    * @type: Type of the reserved region
> + * @rmr: ACPI IORT RMR specific data
>    */
>   struct iommu_resv_region {
>   	struct list_head	list;
> @@ -128,6 +136,9 @@ struct iommu_resv_region {
>   	size_t			length;
>   	int			prot;
>   	enum iommu_resv_type	type;
> +	union {
> +		struct iommu_iort_rmr_data rmr;
> +	} fw_data;
>   };
>   
>   /**
> 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
@ 2021-10-08 12:14     ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-08 12:14 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: jon, linuxarm, steven.price, guohanjun, yangyicong, Sami.Mujawar,
	will, wanghuiqiang

On 2021-08-05 09:07, Shameer Kolothum wrote:
> A union is introduced to struct iommu_resv_region to hold
> any firmware specific data. This is in preparation to add
> support for IORT RMR reserve regions and the union now holds
> the RMR specific information.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>   include/linux/iommu.h | 11 +++++++++++
>   1 file changed, 11 insertions(+)
> 
> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> index 32d448050bf7..bd0e4641c569 100644
> --- a/include/linux/iommu.h
> +++ b/include/linux/iommu.h
> @@ -114,6 +114,13 @@ enum iommu_resv_type {
>   	IOMMU_RESV_SW_MSI,
>   };
>   
> +struct iommu_iort_rmr_data {
> +#define IOMMU_RMR_REMAP_PERMITTED	(1 << 0)
> +	u32 flags;
> +	u32 sid;	/* Stream Id associated with RMR entry */
> +	void *smmu;	/* Associated IORT SMMU node pointer */
> +};

Do we really need to duplicate all this data? AFAICS we could just save 
the acpi_iort_rmr pointer in the iommu_resv_region (with a forward 
declaration here if necessary) and defer parsing its actual mappings 
until the point where we can directly consume the results.

Robin.

> +
>   /**
>    * struct iommu_resv_region - descriptor for a reserved memory region
>    * @list: Linked list pointers
> @@ -121,6 +128,7 @@ enum iommu_resv_type {
>    * @length: Length of the region in bytes
>    * @prot: IOMMU Protection flags (READ/WRITE/...)
>    * @type: Type of the reserved region
> + * @rmr: ACPI IORT RMR specific data
>    */
>   struct iommu_resv_region {
>   	struct list_head	list;
> @@ -128,6 +136,9 @@ struct iommu_resv_region {
>   	size_t			length;
>   	int			prot;
>   	enum iommu_resv_type	type;
> +	union {
> +		struct iommu_iort_rmr_data rmr;
> +	} fw_data;
>   };
>   
>   /**
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-08-05  8:07   ` Shameer Kolothum
  (?)
@ 2021-10-08 12:48     ` Robin Murphy
  -1 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-08 12:48 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: jon, linuxarm, steven.price, guohanjun, yangyicong, Sami.Mujawar,
	will, wanghuiqiang

On 2021-08-05 09:07, Shameer Kolothum wrote:
> Add support for parsing RMR node information from ACPI.
> 
> Find the associated streamid and smmu node info from the
> RMR node and populate a linked list with RMR memory
> descriptors.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>   drivers/acpi/arm64/iort.c | 134 +++++++++++++++++++++++++++++++++++++-
>   1 file changed, 133 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index 3b23fb775ac4..d76ba46ebe67 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -40,6 +40,8 @@ struct iort_fwnode {
>   static LIST_HEAD(iort_fwnode_list);
>   static DEFINE_SPINLOCK(iort_fwnode_lock);
>   
> +static LIST_HEAD(iort_rmr_list);	/* list of RMR regions from ACPI */
> +
>   /**
>    * iort_set_fwnode() - Create iort_fwnode and use it to register
>    *		       iommu data in the iort_fwnode_list
> @@ -393,7 +395,8 @@ static struct acpi_iort_node *iort_node_get_id(struct acpi_iort_node *node,
>   		if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
>   		    node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
>   		    node->type == ACPI_IORT_NODE_SMMU_V3 ||
> -		    node->type == ACPI_IORT_NODE_PMCG) {
> +		    node->type == ACPI_IORT_NODE_PMCG ||
> +		    node->type == ACPI_IORT_NODE_RMR) {
>   			*id_out = map->output_base;
>   			return parent;
>   		}
> @@ -1566,6 +1569,134 @@ static void __init iort_enable_acs(struct acpi_iort_node *iort_node)
>   #else
>   static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
>   #endif
> +static void iort_rmr_desc_check_overlap(struct acpi_iort_rmr_desc *desc, u32 count)
> +{
> +	int i, j;
> +
> +	for (i = 0; i < count; i++) {
> +		u64 end, start = desc[i].base_address, length = desc[i].length;
> +
> +		end = start + length - 1;
> +
> +		/* Check for address overlap */
> +		for (j = i + 1; j < count; j++) {
> +			u64 e_start = desc[j].base_address;
> +			u64 e_end = e_start + desc[j].length - 1;
> +
> +			if (start <= e_end && end >= e_start)
> +				pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] overlaps, continue anyway\n",
> +				       start, end);
> +		}
> +	}
> +}
> +
> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> +{
> +	struct acpi_iort_node *smmu;
> +	struct acpi_iort_rmr *rmr;
> +	struct acpi_iort_rmr_desc *rmr_desc;
> +	u32 map_count = iort_node->mapping_count;
> +	u32 sid;
> +	int i;
> +
> +	if (!iort_node->mapping_offset || map_count != 1) {
> +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	/* Retrieve associated smmu and stream id */
> +	smmu = iort_node_get_id(iort_node, &sid, 0);
> +	if (!smmu) {
> +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	/* Retrieve RMR data */
> +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> +	if (!rmr->rmr_offset || !rmr->rmr_count) {
> +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> +				rmr->rmr_offset);
> +
> +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> +
> +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> +		struct iommu_resv_region *region;
> +		enum iommu_resv_type type;
> +		int prot = IOMMU_READ | IOMMU_WRITE;
> +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> +
> +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> +			/* PAGE align base addr and size */
> +			addr &= PAGE_MASK;
> +			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> +
> +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> +			       rmr_desc->base_address,
> +			       rmr_desc->base_address + rmr_desc->length - 1,
> +			       addr, addr + size - 1);
> +		}
> +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> +			type = IOMMU_RESV_DIRECT_RELAXABLE;
> +			/*
> +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> +			 * normally used for allocated system memory that is
> +			 * then used for device specific reserved regions.
> +			 */
> +			prot |= IOMMU_CACHE;
> +		} else {
> +			type = IOMMU_RESV_DIRECT;
> +			/*
> +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> +			 * for device memory like MSI doorbell.
> +			 */
> +			prot |= IOMMU_MMIO;
> +		}

I'm not sure we ever got a definitive answer to this - does DPAA2 
actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to 
make the fewest possible assumptions, since at this point it's basically 
just a stop-gap until we can fix the spec. It's become clear that we 
can't reliably rely on guessing attributes, so I'm not too fussed about 
theoretical cases that currently don't work (due to complete lack of RMR 
support) continuing to not work for the moment, as long as we can make 
the real-world cases we actually have work at all. Anything which only 
affects performance I'd rather leave until firmware can tell us what to do.

> +		region = iommu_alloc_resv_region(addr, size, prot, type);
> +		if (region) {
> +			region->fw_data.rmr.flags = rmr->flags;
> +			region->fw_data.rmr.sid = sid;
> +			region->fw_data.rmr.smmu = smmu;
> +			list_add_tail(&region->list, &iort_rmr_list);
> +		}
> +	}
> +}
> +
> +static void __init iort_parse_rmr(void)
> +{
> +	struct acpi_iort_node *iort_node, *iort_end;
> +	struct acpi_table_iort *iort;
> +	int i;
> +
> +	if (iort_table->revision < 3)
> +		return;
> +
> +	iort = (struct acpi_table_iort *)iort_table;
> +
> +	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> +				 iort->node_offset);
> +	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> +				iort_table->length);
> +
> +	for (i = 0; i < iort->node_count; i++) {
> +		if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
> +			       "IORT node pointer overflows, bad table!\n"))
> +			return;
> +
> +		if (iort_node->type == ACPI_IORT_NODE_RMR)
> +			iort_node_get_rmr_info(iort_node);
> +
> +		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> +					 iort_node->length);
> +	}
> +}
>   
>   static void __init iort_init_platform_devices(void)
>   {
> @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
>   	}
>   
>   	iort_init_platform_devices();
> +	iort_parse_rmr();

I guess initcall ordering vs. driver registration probably covers it up, 
but for the sake of cleanliness I'd rather make sure the RMRs are fully 
discovered *before* we create the SMMU devices that we expect to start 
consuming them.

Robin.

>   }
>   
>   #ifdef CONFIG_ZONE_DMA
> 

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-10-08 12:48     ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-08 12:48 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: jon, linuxarm, steven.price, guohanjun, yangyicong, Sami.Mujawar,
	will, wanghuiqiang

On 2021-08-05 09:07, Shameer Kolothum wrote:
> Add support for parsing RMR node information from ACPI.
> 
> Find the associated streamid and smmu node info from the
> RMR node and populate a linked list with RMR memory
> descriptors.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>   drivers/acpi/arm64/iort.c | 134 +++++++++++++++++++++++++++++++++++++-
>   1 file changed, 133 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index 3b23fb775ac4..d76ba46ebe67 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -40,6 +40,8 @@ struct iort_fwnode {
>   static LIST_HEAD(iort_fwnode_list);
>   static DEFINE_SPINLOCK(iort_fwnode_lock);
>   
> +static LIST_HEAD(iort_rmr_list);	/* list of RMR regions from ACPI */
> +
>   /**
>    * iort_set_fwnode() - Create iort_fwnode and use it to register
>    *		       iommu data in the iort_fwnode_list
> @@ -393,7 +395,8 @@ static struct acpi_iort_node *iort_node_get_id(struct acpi_iort_node *node,
>   		if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
>   		    node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
>   		    node->type == ACPI_IORT_NODE_SMMU_V3 ||
> -		    node->type == ACPI_IORT_NODE_PMCG) {
> +		    node->type == ACPI_IORT_NODE_PMCG ||
> +		    node->type == ACPI_IORT_NODE_RMR) {
>   			*id_out = map->output_base;
>   			return parent;
>   		}
> @@ -1566,6 +1569,134 @@ static void __init iort_enable_acs(struct acpi_iort_node *iort_node)
>   #else
>   static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
>   #endif
> +static void iort_rmr_desc_check_overlap(struct acpi_iort_rmr_desc *desc, u32 count)
> +{
> +	int i, j;
> +
> +	for (i = 0; i < count; i++) {
> +		u64 end, start = desc[i].base_address, length = desc[i].length;
> +
> +		end = start + length - 1;
> +
> +		/* Check for address overlap */
> +		for (j = i + 1; j < count; j++) {
> +			u64 e_start = desc[j].base_address;
> +			u64 e_end = e_start + desc[j].length - 1;
> +
> +			if (start <= e_end && end >= e_start)
> +				pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] overlaps, continue anyway\n",
> +				       start, end);
> +		}
> +	}
> +}
> +
> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> +{
> +	struct acpi_iort_node *smmu;
> +	struct acpi_iort_rmr *rmr;
> +	struct acpi_iort_rmr_desc *rmr_desc;
> +	u32 map_count = iort_node->mapping_count;
> +	u32 sid;
> +	int i;
> +
> +	if (!iort_node->mapping_offset || map_count != 1) {
> +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	/* Retrieve associated smmu and stream id */
> +	smmu = iort_node_get_id(iort_node, &sid, 0);
> +	if (!smmu) {
> +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	/* Retrieve RMR data */
> +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> +	if (!rmr->rmr_offset || !rmr->rmr_count) {
> +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> +				rmr->rmr_offset);
> +
> +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> +
> +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> +		struct iommu_resv_region *region;
> +		enum iommu_resv_type type;
> +		int prot = IOMMU_READ | IOMMU_WRITE;
> +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> +
> +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> +			/* PAGE align base addr and size */
> +			addr &= PAGE_MASK;
> +			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> +
> +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> +			       rmr_desc->base_address,
> +			       rmr_desc->base_address + rmr_desc->length - 1,
> +			       addr, addr + size - 1);
> +		}
> +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> +			type = IOMMU_RESV_DIRECT_RELAXABLE;
> +			/*
> +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> +			 * normally used for allocated system memory that is
> +			 * then used for device specific reserved regions.
> +			 */
> +			prot |= IOMMU_CACHE;
> +		} else {
> +			type = IOMMU_RESV_DIRECT;
> +			/*
> +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> +			 * for device memory like MSI doorbell.
> +			 */
> +			prot |= IOMMU_MMIO;
> +		}

I'm not sure we ever got a definitive answer to this - does DPAA2 
actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to 
make the fewest possible assumptions, since at this point it's basically 
just a stop-gap until we can fix the spec. It's become clear that we 
can't reliably rely on guessing attributes, so I'm not too fussed about 
theoretical cases that currently don't work (due to complete lack of RMR 
support) continuing to not work for the moment, as long as we can make 
the real-world cases we actually have work at all. Anything which only 
affects performance I'd rather leave until firmware can tell us what to do.

> +		region = iommu_alloc_resv_region(addr, size, prot, type);
> +		if (region) {
> +			region->fw_data.rmr.flags = rmr->flags;
> +			region->fw_data.rmr.sid = sid;
> +			region->fw_data.rmr.smmu = smmu;
> +			list_add_tail(&region->list, &iort_rmr_list);
> +		}
> +	}
> +}
> +
> +static void __init iort_parse_rmr(void)
> +{
> +	struct acpi_iort_node *iort_node, *iort_end;
> +	struct acpi_table_iort *iort;
> +	int i;
> +
> +	if (iort_table->revision < 3)
> +		return;
> +
> +	iort = (struct acpi_table_iort *)iort_table;
> +
> +	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> +				 iort->node_offset);
> +	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> +				iort_table->length);
> +
> +	for (i = 0; i < iort->node_count; i++) {
> +		if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
> +			       "IORT node pointer overflows, bad table!\n"))
> +			return;
> +
> +		if (iort_node->type == ACPI_IORT_NODE_RMR)
> +			iort_node_get_rmr_info(iort_node);
> +
> +		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> +					 iort_node->length);
> +	}
> +}
>   
>   static void __init iort_init_platform_devices(void)
>   {
> @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
>   	}
>   
>   	iort_init_platform_devices();
> +	iort_parse_rmr();

I guess initcall ordering vs. driver registration probably covers it up, 
but for the sake of cleanliness I'd rather make sure the RMRs are fully 
discovered *before* we create the SMMU devices that we expect to start 
consuming them.

Robin.

>   }
>   
>   #ifdef CONFIG_ZONE_DMA
> 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-10-08 12:48     ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-08 12:48 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: jon, linuxarm, steven.price, guohanjun, yangyicong, Sami.Mujawar,
	will, wanghuiqiang

On 2021-08-05 09:07, Shameer Kolothum wrote:
> Add support for parsing RMR node information from ACPI.
> 
> Find the associated streamid and smmu node info from the
> RMR node and populate a linked list with RMR memory
> descriptors.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>   drivers/acpi/arm64/iort.c | 134 +++++++++++++++++++++++++++++++++++++-
>   1 file changed, 133 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index 3b23fb775ac4..d76ba46ebe67 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -40,6 +40,8 @@ struct iort_fwnode {
>   static LIST_HEAD(iort_fwnode_list);
>   static DEFINE_SPINLOCK(iort_fwnode_lock);
>   
> +static LIST_HEAD(iort_rmr_list);	/* list of RMR regions from ACPI */
> +
>   /**
>    * iort_set_fwnode() - Create iort_fwnode and use it to register
>    *		       iommu data in the iort_fwnode_list
> @@ -393,7 +395,8 @@ static struct acpi_iort_node *iort_node_get_id(struct acpi_iort_node *node,
>   		if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
>   		    node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
>   		    node->type == ACPI_IORT_NODE_SMMU_V3 ||
> -		    node->type == ACPI_IORT_NODE_PMCG) {
> +		    node->type == ACPI_IORT_NODE_PMCG ||
> +		    node->type == ACPI_IORT_NODE_RMR) {
>   			*id_out = map->output_base;
>   			return parent;
>   		}
> @@ -1566,6 +1569,134 @@ static void __init iort_enable_acs(struct acpi_iort_node *iort_node)
>   #else
>   static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
>   #endif
> +static void iort_rmr_desc_check_overlap(struct acpi_iort_rmr_desc *desc, u32 count)
> +{
> +	int i, j;
> +
> +	for (i = 0; i < count; i++) {
> +		u64 end, start = desc[i].base_address, length = desc[i].length;
> +
> +		end = start + length - 1;
> +
> +		/* Check for address overlap */
> +		for (j = i + 1; j < count; j++) {
> +			u64 e_start = desc[j].base_address;
> +			u64 e_end = e_start + desc[j].length - 1;
> +
> +			if (start <= e_end && end >= e_start)
> +				pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] overlaps, continue anyway\n",
> +				       start, end);
> +		}
> +	}
> +}
> +
> +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> +{
> +	struct acpi_iort_node *smmu;
> +	struct acpi_iort_rmr *rmr;
> +	struct acpi_iort_rmr_desc *rmr_desc;
> +	u32 map_count = iort_node->mapping_count;
> +	u32 sid;
> +	int i;
> +
> +	if (!iort_node->mapping_offset || map_count != 1) {
> +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	/* Retrieve associated smmu and stream id */
> +	smmu = iort_node_get_id(iort_node, &sid, 0);
> +	if (!smmu) {
> +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	/* Retrieve RMR data */
> +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> +	if (!rmr->rmr_offset || !rmr->rmr_count) {
> +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> +		       iort_node);
> +		return;
> +	}
> +
> +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> +				rmr->rmr_offset);
> +
> +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> +
> +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> +		struct iommu_resv_region *region;
> +		enum iommu_resv_type type;
> +		int prot = IOMMU_READ | IOMMU_WRITE;
> +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> +
> +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> +			/* PAGE align base addr and size */
> +			addr &= PAGE_MASK;
> +			size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> +
> +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> +			       rmr_desc->base_address,
> +			       rmr_desc->base_address + rmr_desc->length - 1,
> +			       addr, addr + size - 1);
> +		}
> +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> +			type = IOMMU_RESV_DIRECT_RELAXABLE;
> +			/*
> +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> +			 * normally used for allocated system memory that is
> +			 * then used for device specific reserved regions.
> +			 */
> +			prot |= IOMMU_CACHE;
> +		} else {
> +			type = IOMMU_RESV_DIRECT;
> +			/*
> +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> +			 * for device memory like MSI doorbell.
> +			 */
> +			prot |= IOMMU_MMIO;
> +		}

I'm not sure we ever got a definitive answer to this - does DPAA2 
actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to 
make the fewest possible assumptions, since at this point it's basically 
just a stop-gap until we can fix the spec. It's become clear that we 
can't reliably rely on guessing attributes, so I'm not too fussed about 
theoretical cases that currently don't work (due to complete lack of RMR 
support) continuing to not work for the moment, as long as we can make 
the real-world cases we actually have work at all. Anything which only 
affects performance I'd rather leave until firmware can tell us what to do.

> +		region = iommu_alloc_resv_region(addr, size, prot, type);
> +		if (region) {
> +			region->fw_data.rmr.flags = rmr->flags;
> +			region->fw_data.rmr.sid = sid;
> +			region->fw_data.rmr.smmu = smmu;
> +			list_add_tail(&region->list, &iort_rmr_list);
> +		}
> +	}
> +}
> +
> +static void __init iort_parse_rmr(void)
> +{
> +	struct acpi_iort_node *iort_node, *iort_end;
> +	struct acpi_table_iort *iort;
> +	int i;
> +
> +	if (iort_table->revision < 3)
> +		return;
> +
> +	iort = (struct acpi_table_iort *)iort_table;
> +
> +	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> +				 iort->node_offset);
> +	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> +				iort_table->length);
> +
> +	for (i = 0; i < iort->node_count; i++) {
> +		if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
> +			       "IORT node pointer overflows, bad table!\n"))
> +			return;
> +
> +		if (iort_node->type == ACPI_IORT_NODE_RMR)
> +			iort_node_get_rmr_info(iort_node);
> +
> +		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> +					 iort_node->length);
> +	}
> +}
>   
>   static void __init iort_init_platform_devices(void)
>   {
> @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
>   	}
>   
>   	iort_init_platform_devices();
> +	iort_parse_rmr();

I guess initcall ordering vs. driver registration probably covers it up, 
but for the sake of cleanliness I'd rather make sure the RMRs are fully 
discovered *before* we create the SMMU devices that we expect to start 
consuming them.

Robin.

>   }
>   
>   #ifdef CONFIG_ZONE_DMA
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 3/9] iommu/dma: Introduce generic helper to retrieve RMR info
  2021-08-05  8:07   ` Shameer Kolothum
  (?)
@ 2021-10-08 13:03     ` Robin Murphy
  -1 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-08 13:03 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, will, wanghuiqiang, guohanjun,
	steven.price, Sami.Mujawar, jon, eric.auger, yangyicong

On 2021-08-05 09:07, Shameer Kolothum wrote:
> Reserved Memory Regions(RMR) associated with an IOMMU can be
> described through ACPI IORT tables in systems with devices
> that require a unity mapping or bypass for those
> regions.
> 
> Introduce a generic interface so that IOMMU drivers can retrieve
> and set up necessary mappings.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>   drivers/iommu/dma-iommu.c | 29 +++++++++++++++++++++++++++++
>   include/linux/dma-iommu.h | 13 +++++++++++++
>   2 files changed, 42 insertions(+)
> 
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 98ba927aee1a..2fa2445e9070 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -174,6 +174,35 @@ void iommu_put_dma_cookie(struct iommu_domain *domain)
>   }
>   EXPORT_SYMBOL(iommu_put_dma_cookie);
>   
> +/**
> + *
> + * iommu_dma_get_rmrs - Retrieve Reserved Memory Regions(RMRs) associated
> + *                      with a given IOMMU
> + * @iommu_fwnode: fwnode associated with IOMMU
> + * @list: RMR list to be populated
> + *
> + */
> +int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
> +		       struct list_head *list)
> +{
> +	return -EINVAL;
> +}
> +EXPORT_SYMBOL(iommu_dma_get_rmrs);
> +
> +/**
> + *
> + * iommu_dma_put_rmrs - Release Reserved Memory Regions(RMRs) associated
> + *                      with a given IOMMU
> + * @iommu_fwnode: fwnode associated with IOMMU
> + * @list: RMR list
> + *
> + */
> +void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
> +			struct list_head *list)
> +{
> +}
> +EXPORT_SYMBOL(iommu_dma_put_rmrs);

Unless there's something special you expect to need to do here, can we 
just uphold the prevailing expectation that resv_regions are kmalloc()ed 
and can be freed directly by the generic function?

> +
>   /**
>    * iommu_dma_get_resv_regions - Reserved region driver helper
>    * @dev: Device from iommu_get_resv_regions()
> diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
> index 758ca4694257..3b7b2d096c6e 100644
> --- a/include/linux/dma-iommu.h
> +++ b/include/linux/dma-iommu.h
> @@ -42,12 +42,16 @@ void iommu_dma_free_cpu_cached_iovas(unsigned int cpu,
>   
>   extern bool iommu_dma_forcedac;
>   
> +int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list);
> +void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head *list);
> +
>   #else /* CONFIG_IOMMU_DMA */
>   
>   struct iommu_domain;
>   struct msi_desc;
>   struct msi_msg;
>   struct device;
> +struct fwnode_handle;
>   
>   static inline void iommu_setup_dma_ops(struct device *dev, u64 dma_base,
>   				       u64 dma_limit)
> @@ -83,5 +87,14 @@ static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_he
>   {
>   }
>   
> +static int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list)
> +{
> +	return -ENODEV;

Hmm, if this needs to be stubbed at all then returning an error seems 
like probably the wrong thing to do. I guess it's for 32-bit builds of 
arm-smmu? It is not an error if the firmware describes no RMRs because 
there are no RMRs, so it hardly deserves to be an error if there are no 
RMRs simply because the firmware isn't ACPI.

Robin.

> +}
> +
> +static void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head *list)
> +{
> +}
> +
>   #endif	/* CONFIG_IOMMU_DMA */
>   #endif	/* __DMA_IOMMU_H */
> 

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 3/9] iommu/dma: Introduce generic helper to retrieve RMR info
@ 2021-10-08 13:03     ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-08 13:03 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: jon, linuxarm, steven.price, guohanjun, yangyicong, Sami.Mujawar,
	will, wanghuiqiang

On 2021-08-05 09:07, Shameer Kolothum wrote:
> Reserved Memory Regions(RMR) associated with an IOMMU can be
> described through ACPI IORT tables in systems with devices
> that require a unity mapping or bypass for those
> regions.
> 
> Introduce a generic interface so that IOMMU drivers can retrieve
> and set up necessary mappings.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>   drivers/iommu/dma-iommu.c | 29 +++++++++++++++++++++++++++++
>   include/linux/dma-iommu.h | 13 +++++++++++++
>   2 files changed, 42 insertions(+)
> 
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 98ba927aee1a..2fa2445e9070 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -174,6 +174,35 @@ void iommu_put_dma_cookie(struct iommu_domain *domain)
>   }
>   EXPORT_SYMBOL(iommu_put_dma_cookie);
>   
> +/**
> + *
> + * iommu_dma_get_rmrs - Retrieve Reserved Memory Regions(RMRs) associated
> + *                      with a given IOMMU
> + * @iommu_fwnode: fwnode associated with IOMMU
> + * @list: RMR list to be populated
> + *
> + */
> +int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
> +		       struct list_head *list)
> +{
> +	return -EINVAL;
> +}
> +EXPORT_SYMBOL(iommu_dma_get_rmrs);
> +
> +/**
> + *
> + * iommu_dma_put_rmrs - Release Reserved Memory Regions(RMRs) associated
> + *                      with a given IOMMU
> + * @iommu_fwnode: fwnode associated with IOMMU
> + * @list: RMR list
> + *
> + */
> +void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
> +			struct list_head *list)
> +{
> +}
> +EXPORT_SYMBOL(iommu_dma_put_rmrs);

Unless there's something special you expect to need to do here, can we 
just uphold the prevailing expectation that resv_regions are kmalloc()ed 
and can be freed directly by the generic function?

> +
>   /**
>    * iommu_dma_get_resv_regions - Reserved region driver helper
>    * @dev: Device from iommu_get_resv_regions()
> diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
> index 758ca4694257..3b7b2d096c6e 100644
> --- a/include/linux/dma-iommu.h
> +++ b/include/linux/dma-iommu.h
> @@ -42,12 +42,16 @@ void iommu_dma_free_cpu_cached_iovas(unsigned int cpu,
>   
>   extern bool iommu_dma_forcedac;
>   
> +int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list);
> +void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head *list);
> +
>   #else /* CONFIG_IOMMU_DMA */
>   
>   struct iommu_domain;
>   struct msi_desc;
>   struct msi_msg;
>   struct device;
> +struct fwnode_handle;
>   
>   static inline void iommu_setup_dma_ops(struct device *dev, u64 dma_base,
>   				       u64 dma_limit)
> @@ -83,5 +87,14 @@ static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_he
>   {
>   }
>   
> +static int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list)
> +{
> +	return -ENODEV;

Hmm, if this needs to be stubbed at all then returning an error seems 
like probably the wrong thing to do. I guess it's for 32-bit builds of 
arm-smmu? It is not an error if the firmware describes no RMRs because 
there are no RMRs, so it hardly deserves to be an error if there are no 
RMRs simply because the firmware isn't ACPI.

Robin.

> +}
> +
> +static void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head *list)
> +{
> +}
> +
>   #endif	/* CONFIG_IOMMU_DMA */
>   #endif	/* __DMA_IOMMU_H */
> 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 3/9] iommu/dma: Introduce generic helper to retrieve RMR info
@ 2021-10-08 13:03     ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-08 13:03 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, will, wanghuiqiang, guohanjun,
	steven.price, Sami.Mujawar, jon, eric.auger, yangyicong

On 2021-08-05 09:07, Shameer Kolothum wrote:
> Reserved Memory Regions(RMR) associated with an IOMMU can be
> described through ACPI IORT tables in systems with devices
> that require a unity mapping or bypass for those
> regions.
> 
> Introduce a generic interface so that IOMMU drivers can retrieve
> and set up necessary mappings.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>   drivers/iommu/dma-iommu.c | 29 +++++++++++++++++++++++++++++
>   include/linux/dma-iommu.h | 13 +++++++++++++
>   2 files changed, 42 insertions(+)
> 
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 98ba927aee1a..2fa2445e9070 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -174,6 +174,35 @@ void iommu_put_dma_cookie(struct iommu_domain *domain)
>   }
>   EXPORT_SYMBOL(iommu_put_dma_cookie);
>   
> +/**
> + *
> + * iommu_dma_get_rmrs - Retrieve Reserved Memory Regions(RMRs) associated
> + *                      with a given IOMMU
> + * @iommu_fwnode: fwnode associated with IOMMU
> + * @list: RMR list to be populated
> + *
> + */
> +int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
> +		       struct list_head *list)
> +{
> +	return -EINVAL;
> +}
> +EXPORT_SYMBOL(iommu_dma_get_rmrs);
> +
> +/**
> + *
> + * iommu_dma_put_rmrs - Release Reserved Memory Regions(RMRs) associated
> + *                      with a given IOMMU
> + * @iommu_fwnode: fwnode associated with IOMMU
> + * @list: RMR list
> + *
> + */
> +void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
> +			struct list_head *list)
> +{
> +}
> +EXPORT_SYMBOL(iommu_dma_put_rmrs);

Unless there's something special you expect to need to do here, can we 
just uphold the prevailing expectation that resv_regions are kmalloc()ed 
and can be freed directly by the generic function?

> +
>   /**
>    * iommu_dma_get_resv_regions - Reserved region driver helper
>    * @dev: Device from iommu_get_resv_regions()
> diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
> index 758ca4694257..3b7b2d096c6e 100644
> --- a/include/linux/dma-iommu.h
> +++ b/include/linux/dma-iommu.h
> @@ -42,12 +42,16 @@ void iommu_dma_free_cpu_cached_iovas(unsigned int cpu,
>   
>   extern bool iommu_dma_forcedac;
>   
> +int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list);
> +void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head *list);
> +
>   #else /* CONFIG_IOMMU_DMA */
>   
>   struct iommu_domain;
>   struct msi_desc;
>   struct msi_msg;
>   struct device;
> +struct fwnode_handle;
>   
>   static inline void iommu_setup_dma_ops(struct device *dev, u64 dma_base,
>   				       u64 dma_limit)
> @@ -83,5 +87,14 @@ static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_he
>   {
>   }
>   
> +static int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list)
> +{
> +	return -ENODEV;

Hmm, if this needs to be stubbed at all then returning an error seems 
like probably the wrong thing to do. I guess it's for 32-bit builds of 
arm-smmu? It is not an error if the firmware describes no RMRs because 
there are no RMRs, so it hardly deserves to be an error if there are no 
RMRs simply because the firmware isn't ACPI.

Robin.

> +}
> +
> +static void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head *list)
> +{
> +}
> +
>   #endif	/* CONFIG_IOMMU_DMA */
>   #endif	/* __DMA_IOMMU_H */
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
  2021-08-05  8:07   ` Shameer Kolothum
  (?)
@ 2021-10-08 13:09     ` Robin Murphy
  -1 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-08 13:09 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, will, wanghuiqiang, guohanjun,
	steven.price, Sami.Mujawar, jon, eric.auger, yangyicong

On 2021-08-05 09:07, Shameer Kolothum wrote:
> Get ACPI IORT RMR regions associated with a dev reserved
> so that there is a unity mapping for them in SMMU.

This feels like most of it belongs in the IORT code rather than 
iommu-dma (which should save the temporary list copy as well).

Robin.

> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>   drivers/iommu/dma-iommu.c | 56 +++++++++++++++++++++++++++++++++++----
>   1 file changed, 51 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 1b6e27475279..c1ae0c3d4b33 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -207,22 +207,68 @@ void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
>   }
>   EXPORT_SYMBOL(iommu_dma_put_rmrs);
>   
> +static bool iommu_dma_dev_has_rmr(struct iommu_fwspec *fwspec,
> +				  struct iommu_resv_region *e)
> +{
> +	int i;
> +
> +	for (i = 0; i < fwspec->num_ids; i++) {
> +		if (e->fw_data.rmr.sid == fwspec->ids[i])
> +			return true;
> +	}
> +
> +	return false;
> +}
> +
> +static void iommu_dma_get_rmr_resv_regions(struct device *dev,
> +					   struct list_head *list)
> +{
> +	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
> +	struct list_head rmr_list;
> +	struct iommu_resv_region *rmr, *tmp;
> +
> +	INIT_LIST_HEAD(&rmr_list);
> +	if (iommu_dma_get_rmrs(fwspec->iommu_fwnode, &rmr_list))
> +		return;
> +
> +	if (dev_is_pci(dev)) {
> +		struct pci_dev *pdev = to_pci_dev(dev);
> +		struct pci_host_bridge *host = pci_find_host_bridge(pdev->bus);
> +
> +		if (!host->preserve_config)
> +			return;
> +	}
> +
> +	list_for_each_entry_safe(rmr, tmp, &rmr_list, list) {
> +		if (!iommu_dma_dev_has_rmr(fwspec, rmr))
> +			continue;
> +
> +		/* Remove from iommu RMR list and add to dev resv_regions */
> +		list_del_init(&rmr->list);
> +		list_add_tail(&rmr->list, list);
> +	}
> +
> +	iommu_dma_put_rmrs(fwspec->iommu_fwnode, &rmr_list);
> +}
> +
>   /**
>    * iommu_dma_get_resv_regions - Reserved region driver helper
>    * @dev: Device from iommu_get_resv_regions()
>    * @list: Reserved region list from iommu_get_resv_regions()
>    *
>    * IOMMU drivers can use this to implement their .get_resv_regions callback
> - * for general non-IOMMU-specific reservations. Currently, this covers GICv3
> - * ITS region reservation on ACPI based ARM platforms that may require HW MSI
> - * reservation.
> + * for general non-IOMMU-specific reservations. Currently this covers,
> + *  -GICv3 ITS region reservation on ACPI based ARM platforms that may
> + *   require HW MSI reservation.
> + *  -Any ACPI IORT RMR memory range reservations (IORT spec rev E.b)
>    */
>   void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
>   {
>   
> -	if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode))
> +	if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode)) {
>   		iort_iommu_msi_get_resv_regions(dev, list);
> -
> +		iommu_dma_get_rmr_resv_regions(dev, list);
> +	}
>   }
>   EXPORT_SYMBOL(iommu_dma_get_resv_regions);
>   
> 

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
@ 2021-10-08 13:09     ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-08 13:09 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: jon, linuxarm, steven.price, guohanjun, yangyicong, Sami.Mujawar,
	will, wanghuiqiang

On 2021-08-05 09:07, Shameer Kolothum wrote:
> Get ACPI IORT RMR regions associated with a dev reserved
> so that there is a unity mapping for them in SMMU.

This feels like most of it belongs in the IORT code rather than 
iommu-dma (which should save the temporary list copy as well).

Robin.

> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>   drivers/iommu/dma-iommu.c | 56 +++++++++++++++++++++++++++++++++++----
>   1 file changed, 51 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 1b6e27475279..c1ae0c3d4b33 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -207,22 +207,68 @@ void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
>   }
>   EXPORT_SYMBOL(iommu_dma_put_rmrs);
>   
> +static bool iommu_dma_dev_has_rmr(struct iommu_fwspec *fwspec,
> +				  struct iommu_resv_region *e)
> +{
> +	int i;
> +
> +	for (i = 0; i < fwspec->num_ids; i++) {
> +		if (e->fw_data.rmr.sid == fwspec->ids[i])
> +			return true;
> +	}
> +
> +	return false;
> +}
> +
> +static void iommu_dma_get_rmr_resv_regions(struct device *dev,
> +					   struct list_head *list)
> +{
> +	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
> +	struct list_head rmr_list;
> +	struct iommu_resv_region *rmr, *tmp;
> +
> +	INIT_LIST_HEAD(&rmr_list);
> +	if (iommu_dma_get_rmrs(fwspec->iommu_fwnode, &rmr_list))
> +		return;
> +
> +	if (dev_is_pci(dev)) {
> +		struct pci_dev *pdev = to_pci_dev(dev);
> +		struct pci_host_bridge *host = pci_find_host_bridge(pdev->bus);
> +
> +		if (!host->preserve_config)
> +			return;
> +	}
> +
> +	list_for_each_entry_safe(rmr, tmp, &rmr_list, list) {
> +		if (!iommu_dma_dev_has_rmr(fwspec, rmr))
> +			continue;
> +
> +		/* Remove from iommu RMR list and add to dev resv_regions */
> +		list_del_init(&rmr->list);
> +		list_add_tail(&rmr->list, list);
> +	}
> +
> +	iommu_dma_put_rmrs(fwspec->iommu_fwnode, &rmr_list);
> +}
> +
>   /**
>    * iommu_dma_get_resv_regions - Reserved region driver helper
>    * @dev: Device from iommu_get_resv_regions()
>    * @list: Reserved region list from iommu_get_resv_regions()
>    *
>    * IOMMU drivers can use this to implement their .get_resv_regions callback
> - * for general non-IOMMU-specific reservations. Currently, this covers GICv3
> - * ITS region reservation on ACPI based ARM platforms that may require HW MSI
> - * reservation.
> + * for general non-IOMMU-specific reservations. Currently this covers,
> + *  -GICv3 ITS region reservation on ACPI based ARM platforms that may
> + *   require HW MSI reservation.
> + *  -Any ACPI IORT RMR memory range reservations (IORT spec rev E.b)
>    */
>   void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
>   {
>   
> -	if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode))
> +	if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode)) {
>   		iort_iommu_msi_get_resv_regions(dev, list);
> -
> +		iommu_dma_get_rmr_resv_regions(dev, list);
> +	}
>   }
>   EXPORT_SYMBOL(iommu_dma_get_resv_regions);
>   
> 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
@ 2021-10-08 13:09     ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-08 13:09 UTC (permalink / raw)
  To: Shameer Kolothum, linux-arm-kernel, linux-acpi, iommu
  Cc: linuxarm, lorenzo.pieralisi, joro, will, wanghuiqiang, guohanjun,
	steven.price, Sami.Mujawar, jon, eric.auger, yangyicong

On 2021-08-05 09:07, Shameer Kolothum wrote:
> Get ACPI IORT RMR regions associated with a dev reserved
> so that there is a unity mapping for them in SMMU.

This feels like most of it belongs in the IORT code rather than 
iommu-dma (which should save the temporary list copy as well).

Robin.

> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>   drivers/iommu/dma-iommu.c | 56 +++++++++++++++++++++++++++++++++++----
>   1 file changed, 51 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 1b6e27475279..c1ae0c3d4b33 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -207,22 +207,68 @@ void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
>   }
>   EXPORT_SYMBOL(iommu_dma_put_rmrs);
>   
> +static bool iommu_dma_dev_has_rmr(struct iommu_fwspec *fwspec,
> +				  struct iommu_resv_region *e)
> +{
> +	int i;
> +
> +	for (i = 0; i < fwspec->num_ids; i++) {
> +		if (e->fw_data.rmr.sid == fwspec->ids[i])
> +			return true;
> +	}
> +
> +	return false;
> +}
> +
> +static void iommu_dma_get_rmr_resv_regions(struct device *dev,
> +					   struct list_head *list)
> +{
> +	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
> +	struct list_head rmr_list;
> +	struct iommu_resv_region *rmr, *tmp;
> +
> +	INIT_LIST_HEAD(&rmr_list);
> +	if (iommu_dma_get_rmrs(fwspec->iommu_fwnode, &rmr_list))
> +		return;
> +
> +	if (dev_is_pci(dev)) {
> +		struct pci_dev *pdev = to_pci_dev(dev);
> +		struct pci_host_bridge *host = pci_find_host_bridge(pdev->bus);
> +
> +		if (!host->preserve_config)
> +			return;
> +	}
> +
> +	list_for_each_entry_safe(rmr, tmp, &rmr_list, list) {
> +		if (!iommu_dma_dev_has_rmr(fwspec, rmr))
> +			continue;
> +
> +		/* Remove from iommu RMR list and add to dev resv_regions */
> +		list_del_init(&rmr->list);
> +		list_add_tail(&rmr->list, list);
> +	}
> +
> +	iommu_dma_put_rmrs(fwspec->iommu_fwnode, &rmr_list);
> +}
> +
>   /**
>    * iommu_dma_get_resv_regions - Reserved region driver helper
>    * @dev: Device from iommu_get_resv_regions()
>    * @list: Reserved region list from iommu_get_resv_regions()
>    *
>    * IOMMU drivers can use this to implement their .get_resv_regions callback
> - * for general non-IOMMU-specific reservations. Currently, this covers GICv3
> - * ITS region reservation on ACPI based ARM platforms that may require HW MSI
> - * reservation.
> + * for general non-IOMMU-specific reservations. Currently this covers,
> + *  -GICv3 ITS region reservation on ACPI based ARM platforms that may
> + *   require HW MSI reservation.
> + *  -Any ACPI IORT RMR memory range reservations (IORT spec rev E.b)
>    */
>   void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
>   {
>   
> -	if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode))
> +	if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode)) {
>   		iort_iommu_msi_get_resv_regions(dev, list);
> -
> +		iommu_dma_get_rmr_resv_regions(dev, list);
> +	}
>   }
>   EXPORT_SYMBOL(iommu_dma_get_resv_regions);
>   
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
  2021-10-08 12:14     ` Robin Murphy
  (?)
@ 2021-10-09  6:57       ` Jon Nettleton
  -1 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-10-09  6:57 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Steven Price, Hanjun Guo, yangyicong,
	Sami Mujawar, Will Deacon, wanghuiqiang

On Fri, Oct 8, 2021 at 2:14 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > A union is introduced to struct iommu_resv_region to hold
> > any firmware specific data. This is in preparation to add
> > support for IORT RMR reserve regions and the union now holds
> > the RMR specific information.
> >
> > Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   include/linux/iommu.h | 11 +++++++++++
> >   1 file changed, 11 insertions(+)
> >
> > diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> > index 32d448050bf7..bd0e4641c569 100644
> > --- a/include/linux/iommu.h
> > +++ b/include/linux/iommu.h
> > @@ -114,6 +114,13 @@ enum iommu_resv_type {
> >       IOMMU_RESV_SW_MSI,
> >   };
> >
> > +struct iommu_iort_rmr_data {
> > +#define IOMMU_RMR_REMAP_PERMITTED    (1 << 0)
> > +     u32 flags;
> > +     u32 sid;        /* Stream Id associated with RMR entry */
> > +     void *smmu;     /* Associated IORT SMMU node pointer */
> > +};
>
> Do we really need to duplicate all this data? AFAICS we could just save
> the acpi_iort_rmr pointer in the iommu_resv_region (with a forward
> declaration here if necessary) and defer parsing its actual mappings
> until the point where we can directly consume the results.

From earlier discussions on this patchset, the original goal was also for
device-tree mechanisms to be able to hook into this code to support
similar RMR's and SMMU initialization, just not through the ACPI / IORT
path.

>
> Robin.
>
> > +
> >   /**
> >    * struct iommu_resv_region - descriptor for a reserved memory region
> >    * @list: Linked list pointers
> > @@ -121,6 +128,7 @@ enum iommu_resv_type {
> >    * @length: Length of the region in bytes
> >    * @prot: IOMMU Protection flags (READ/WRITE/...)
> >    * @type: Type of the reserved region
> > + * @rmr: ACPI IORT RMR specific data
> >    */
> >   struct iommu_resv_region {
> >       struct list_head        list;
> > @@ -128,6 +136,9 @@ struct iommu_resv_region {
> >       size_t                  length;
> >       int                     prot;
> >       enum iommu_resv_type    type;
> > +     union {
> > +             struct iommu_iort_rmr_data rmr;
> > +     } fw_data;
> >   };
> >
> >   /**
> >

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
@ 2021-10-09  6:57       ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-10-09  6:57 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Linuxarm, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon,
	linux-arm-kernel

On Fri, Oct 8, 2021 at 2:14 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > A union is introduced to struct iommu_resv_region to hold
> > any firmware specific data. This is in preparation to add
> > support for IORT RMR reserve regions and the union now holds
> > the RMR specific information.
> >
> > Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   include/linux/iommu.h | 11 +++++++++++
> >   1 file changed, 11 insertions(+)
> >
> > diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> > index 32d448050bf7..bd0e4641c569 100644
> > --- a/include/linux/iommu.h
> > +++ b/include/linux/iommu.h
> > @@ -114,6 +114,13 @@ enum iommu_resv_type {
> >       IOMMU_RESV_SW_MSI,
> >   };
> >
> > +struct iommu_iort_rmr_data {
> > +#define IOMMU_RMR_REMAP_PERMITTED    (1 << 0)
> > +     u32 flags;
> > +     u32 sid;        /* Stream Id associated with RMR entry */
> > +     void *smmu;     /* Associated IORT SMMU node pointer */
> > +};
>
> Do we really need to duplicate all this data? AFAICS we could just save
> the acpi_iort_rmr pointer in the iommu_resv_region (with a forward
> declaration here if necessary) and defer parsing its actual mappings
> until the point where we can directly consume the results.

From earlier discussions on this patchset, the original goal was also for
device-tree mechanisms to be able to hook into this code to support
similar RMR's and SMMU initialization, just not through the ACPI / IORT
path.

>
> Robin.
>
> > +
> >   /**
> >    * struct iommu_resv_region - descriptor for a reserved memory region
> >    * @list: Linked list pointers
> > @@ -121,6 +128,7 @@ enum iommu_resv_type {
> >    * @length: Length of the region in bytes
> >    * @prot: IOMMU Protection flags (READ/WRITE/...)
> >    * @type: Type of the reserved region
> > + * @rmr: ACPI IORT RMR specific data
> >    */
> >   struct iommu_resv_region {
> >       struct list_head        list;
> > @@ -128,6 +136,9 @@ struct iommu_resv_region {
> >       size_t                  length;
> >       int                     prot;
> >       enum iommu_resv_type    type;
> > +     union {
> > +             struct iommu_iort_rmr_data rmr;
> > +     } fw_data;
> >   };
> >
> >   /**
> >
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
@ 2021-10-09  6:57       ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-10-09  6:57 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Steven Price, Hanjun Guo, yangyicong,
	Sami Mujawar, Will Deacon, wanghuiqiang

On Fri, Oct 8, 2021 at 2:14 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > A union is introduced to struct iommu_resv_region to hold
> > any firmware specific data. This is in preparation to add
> > support for IORT RMR reserve regions and the union now holds
> > the RMR specific information.
> >
> > Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   include/linux/iommu.h | 11 +++++++++++
> >   1 file changed, 11 insertions(+)
> >
> > diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> > index 32d448050bf7..bd0e4641c569 100644
> > --- a/include/linux/iommu.h
> > +++ b/include/linux/iommu.h
> > @@ -114,6 +114,13 @@ enum iommu_resv_type {
> >       IOMMU_RESV_SW_MSI,
> >   };
> >
> > +struct iommu_iort_rmr_data {
> > +#define IOMMU_RMR_REMAP_PERMITTED    (1 << 0)
> > +     u32 flags;
> > +     u32 sid;        /* Stream Id associated with RMR entry */
> > +     void *smmu;     /* Associated IORT SMMU node pointer */
> > +};
>
> Do we really need to duplicate all this data? AFAICS we could just save
> the acpi_iort_rmr pointer in the iommu_resv_region (with a forward
> declaration here if necessary) and defer parsing its actual mappings
> until the point where we can directly consume the results.

From earlier discussions on this patchset, the original goal was also for
device-tree mechanisms to be able to hook into this code to support
similar RMR's and SMMU initialization, just not through the ACPI / IORT
path.

>
> Robin.
>
> > +
> >   /**
> >    * struct iommu_resv_region - descriptor for a reserved memory region
> >    * @list: Linked list pointers
> > @@ -121,6 +128,7 @@ enum iommu_resv_type {
> >    * @length: Length of the region in bytes
> >    * @prot: IOMMU Protection flags (READ/WRITE/...)
> >    * @type: Type of the reserved region
> > + * @rmr: ACPI IORT RMR specific data
> >    */
> >   struct iommu_resv_region {
> >       struct list_head        list;
> > @@ -128,6 +136,9 @@ struct iommu_resv_region {
> >       size_t                  length;
> >       int                     prot;
> >       enum iommu_resv_type    type;
> > +     union {
> > +             struct iommu_iort_rmr_data rmr;
> > +     } fw_data;
> >   };
> >
> >   /**
> >

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-10-08 12:48     ` Robin Murphy
  (?)
@ 2021-10-09  7:06       ` Jon Nettleton
  -1 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-10-09  7:06 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Steven Price, Hanjun Guo, yangyicong,
	Sami Mujawar, Will Deacon, wanghuiqiang

On Fri, Oct 8, 2021 at 2:49 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > Add support for parsing RMR node information from ACPI.
> >
> > Find the associated streamid and smmu node info from the
> > RMR node and populate a linked list with RMR memory
> > descriptors.
> >
> > Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   drivers/acpi/arm64/iort.c | 134 +++++++++++++++++++++++++++++++++++++-
> >   1 file changed, 133 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> > index 3b23fb775ac4..d76ba46ebe67 100644
> > --- a/drivers/acpi/arm64/iort.c
> > +++ b/drivers/acpi/arm64/iort.c
> > @@ -40,6 +40,8 @@ struct iort_fwnode {
> >   static LIST_HEAD(iort_fwnode_list);
> >   static DEFINE_SPINLOCK(iort_fwnode_lock);
> >
> > +static LIST_HEAD(iort_rmr_list);     /* list of RMR regions from ACPI */
> > +
> >   /**
> >    * iort_set_fwnode() - Create iort_fwnode and use it to register
> >    *                 iommu data in the iort_fwnode_list
> > @@ -393,7 +395,8 @@ static struct acpi_iort_node *iort_node_get_id(struct acpi_iort_node *node,
> >               if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
> >                   node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
> >                   node->type == ACPI_IORT_NODE_SMMU_V3 ||
> > -                 node->type == ACPI_IORT_NODE_PMCG) {
> > +                 node->type == ACPI_IORT_NODE_PMCG ||
> > +                 node->type == ACPI_IORT_NODE_RMR) {
> >                       *id_out = map->output_base;
> >                       return parent;
> >               }
> > @@ -1566,6 +1569,134 @@ static void __init iort_enable_acs(struct acpi_iort_node *iort_node)
> >   #else
> >   static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
> >   #endif
> > +static void iort_rmr_desc_check_overlap(struct acpi_iort_rmr_desc *desc, u32 count)
> > +{
> > +     int i, j;
> > +
> > +     for (i = 0; i < count; i++) {
> > +             u64 end, start = desc[i].base_address, length = desc[i].length;
> > +
> > +             end = start + length - 1;
> > +
> > +             /* Check for address overlap */
> > +             for (j = i + 1; j < count; j++) {
> > +                     u64 e_start = desc[j].base_address;
> > +                     u64 e_end = e_start + desc[j].length - 1;
> > +
> > +                     if (start <= e_end && end >= e_start)
> > +                             pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] overlaps, continue anyway\n",
> > +                                    start, end);
> > +             }
> > +     }
> > +}
> > +
> > +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> > +{
> > +     struct acpi_iort_node *smmu;
> > +     struct acpi_iort_rmr *rmr;
> > +     struct acpi_iort_rmr_desc *rmr_desc;
> > +     u32 map_count = iort_node->mapping_count;
> > +     u32 sid;
> > +     int i;
> > +
> > +     if (!iort_node->mapping_offset || map_count != 1) {
> > +             pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     /* Retrieve associated smmu and stream id */
> > +     smmu = iort_node_get_id(iort_node, &sid, 0);
> > +     if (!smmu) {
> > +             pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     /* Retrieve RMR data */
> > +     rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> > +     if (!rmr->rmr_offset || !rmr->rmr_count) {
> > +             pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> > +                             rmr->rmr_offset);
> > +
> > +     iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> > +
> > +     for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> > +             struct iommu_resv_region *region;
> > +             enum iommu_resv_type type;
> > +             int prot = IOMMU_READ | IOMMU_WRITE;
> > +             u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> > +
> > +             if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> > +                     /* PAGE align base addr and size */
> > +                     addr &= PAGE_MASK;
> > +                     size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> > +
> > +                     pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> > +                            rmr_desc->base_address,
> > +                            rmr_desc->base_address + rmr_desc->length - 1,
> > +                            addr, addr + size - 1);
> > +             }
> > +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> > +                     /*
> > +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > +                      * normally used for allocated system memory that is
> > +                      * then used for device specific reserved regions.
> > +                      */
> > +                     prot |= IOMMU_CACHE;
> > +             } else {
> > +                     type = IOMMU_RESV_DIRECT;
> > +                     /*
> > +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> > +                      * for device memory like MSI doorbell.
> > +                      */
> > +                     prot |= IOMMU_MMIO;
> > +             }
>
> I'm not sure we ever got a definitive answer to this - does DPAA2
> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> make the fewest possible assumptions, since at this point it's basically
> just a stop-gap until we can fix the spec. It's become clear that we
> can't reliably rely on guessing attributes, so I'm not too fussed about
> theoretical cases that currently don't work (due to complete lack of RMR
> support) continuing to not work for the moment, as long as we can make
> the real-world cases we actually have work at all. Anything which only
> affects performance I'd rather leave until firmware can tell us what to do.

Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
mappings.  DPAA2 is just one connected device.

-Jon

>
> > +             region = iommu_alloc_resv_region(addr, size, prot, type);
> > +             if (region) {
> > +                     region->fw_data.rmr.flags = rmr->flags;
> > +                     region->fw_data.rmr.sid = sid;
> > +                     region->fw_data.rmr.smmu = smmu;
> > +                     list_add_tail(&region->list, &iort_rmr_list);
> > +             }
> > +     }
> > +}
> > +
> > +static void __init iort_parse_rmr(void)
> > +{
> > +     struct acpi_iort_node *iort_node, *iort_end;
> > +     struct acpi_table_iort *iort;
> > +     int i;
> > +
> > +     if (iort_table->revision < 3)
> > +             return;
> > +
> > +     iort = (struct acpi_table_iort *)iort_table;
> > +
> > +     iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +                              iort->node_offset);
> > +     iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +                             iort_table->length);
> > +
> > +     for (i = 0; i < iort->node_count; i++) {
> > +             if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
> > +                            "IORT node pointer overflows, bad table!\n"))
> > +                     return;
> > +
> > +             if (iort_node->type == ACPI_IORT_NODE_RMR)
> > +                     iort_node_get_rmr_info(iort_node);
> > +
> > +             iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> > +                                      iort_node->length);
> > +     }
> > +}
> >
> >   static void __init iort_init_platform_devices(void)
> >   {
> > @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
> >       }
> >
> >       iort_init_platform_devices();
> > +     iort_parse_rmr();
>
> I guess initcall ordering vs. driver registration probably covers it up,
> but for the sake of cleanliness I'd rather make sure the RMRs are fully
> discovered *before* we create the SMMU devices that we expect to start
> consuming them.
>
> Robin.
>
> >   }
> >
> >   #ifdef CONFIG_ZONE_DMA
> >

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-10-09  7:06       ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-10-09  7:06 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Linuxarm, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon,
	linux-arm-kernel

On Fri, Oct 8, 2021 at 2:49 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > Add support for parsing RMR node information from ACPI.
> >
> > Find the associated streamid and smmu node info from the
> > RMR node and populate a linked list with RMR memory
> > descriptors.
> >
> > Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   drivers/acpi/arm64/iort.c | 134 +++++++++++++++++++++++++++++++++++++-
> >   1 file changed, 133 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> > index 3b23fb775ac4..d76ba46ebe67 100644
> > --- a/drivers/acpi/arm64/iort.c
> > +++ b/drivers/acpi/arm64/iort.c
> > @@ -40,6 +40,8 @@ struct iort_fwnode {
> >   static LIST_HEAD(iort_fwnode_list);
> >   static DEFINE_SPINLOCK(iort_fwnode_lock);
> >
> > +static LIST_HEAD(iort_rmr_list);     /* list of RMR regions from ACPI */
> > +
> >   /**
> >    * iort_set_fwnode() - Create iort_fwnode and use it to register
> >    *                 iommu data in the iort_fwnode_list
> > @@ -393,7 +395,8 @@ static struct acpi_iort_node *iort_node_get_id(struct acpi_iort_node *node,
> >               if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
> >                   node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
> >                   node->type == ACPI_IORT_NODE_SMMU_V3 ||
> > -                 node->type == ACPI_IORT_NODE_PMCG) {
> > +                 node->type == ACPI_IORT_NODE_PMCG ||
> > +                 node->type == ACPI_IORT_NODE_RMR) {
> >                       *id_out = map->output_base;
> >                       return parent;
> >               }
> > @@ -1566,6 +1569,134 @@ static void __init iort_enable_acs(struct acpi_iort_node *iort_node)
> >   #else
> >   static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
> >   #endif
> > +static void iort_rmr_desc_check_overlap(struct acpi_iort_rmr_desc *desc, u32 count)
> > +{
> > +     int i, j;
> > +
> > +     for (i = 0; i < count; i++) {
> > +             u64 end, start = desc[i].base_address, length = desc[i].length;
> > +
> > +             end = start + length - 1;
> > +
> > +             /* Check for address overlap */
> > +             for (j = i + 1; j < count; j++) {
> > +                     u64 e_start = desc[j].base_address;
> > +                     u64 e_end = e_start + desc[j].length - 1;
> > +
> > +                     if (start <= e_end && end >= e_start)
> > +                             pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] overlaps, continue anyway\n",
> > +                                    start, end);
> > +             }
> > +     }
> > +}
> > +
> > +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> > +{
> > +     struct acpi_iort_node *smmu;
> > +     struct acpi_iort_rmr *rmr;
> > +     struct acpi_iort_rmr_desc *rmr_desc;
> > +     u32 map_count = iort_node->mapping_count;
> > +     u32 sid;
> > +     int i;
> > +
> > +     if (!iort_node->mapping_offset || map_count != 1) {
> > +             pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     /* Retrieve associated smmu and stream id */
> > +     smmu = iort_node_get_id(iort_node, &sid, 0);
> > +     if (!smmu) {
> > +             pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     /* Retrieve RMR data */
> > +     rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> > +     if (!rmr->rmr_offset || !rmr->rmr_count) {
> > +             pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> > +                             rmr->rmr_offset);
> > +
> > +     iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> > +
> > +     for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> > +             struct iommu_resv_region *region;
> > +             enum iommu_resv_type type;
> > +             int prot = IOMMU_READ | IOMMU_WRITE;
> > +             u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> > +
> > +             if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> > +                     /* PAGE align base addr and size */
> > +                     addr &= PAGE_MASK;
> > +                     size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> > +
> > +                     pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> > +                            rmr_desc->base_address,
> > +                            rmr_desc->base_address + rmr_desc->length - 1,
> > +                            addr, addr + size - 1);
> > +             }
> > +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> > +                     /*
> > +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > +                      * normally used for allocated system memory that is
> > +                      * then used for device specific reserved regions.
> > +                      */
> > +                     prot |= IOMMU_CACHE;
> > +             } else {
> > +                     type = IOMMU_RESV_DIRECT;
> > +                     /*
> > +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> > +                      * for device memory like MSI doorbell.
> > +                      */
> > +                     prot |= IOMMU_MMIO;
> > +             }
>
> I'm not sure we ever got a definitive answer to this - does DPAA2
> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> make the fewest possible assumptions, since at this point it's basically
> just a stop-gap until we can fix the spec. It's become clear that we
> can't reliably rely on guessing attributes, so I'm not too fussed about
> theoretical cases that currently don't work (due to complete lack of RMR
> support) continuing to not work for the moment, as long as we can make
> the real-world cases we actually have work at all. Anything which only
> affects performance I'd rather leave until firmware can tell us what to do.

Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
mappings.  DPAA2 is just one connected device.

-Jon

>
> > +             region = iommu_alloc_resv_region(addr, size, prot, type);
> > +             if (region) {
> > +                     region->fw_data.rmr.flags = rmr->flags;
> > +                     region->fw_data.rmr.sid = sid;
> > +                     region->fw_data.rmr.smmu = smmu;
> > +                     list_add_tail(&region->list, &iort_rmr_list);
> > +             }
> > +     }
> > +}
> > +
> > +static void __init iort_parse_rmr(void)
> > +{
> > +     struct acpi_iort_node *iort_node, *iort_end;
> > +     struct acpi_table_iort *iort;
> > +     int i;
> > +
> > +     if (iort_table->revision < 3)
> > +             return;
> > +
> > +     iort = (struct acpi_table_iort *)iort_table;
> > +
> > +     iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +                              iort->node_offset);
> > +     iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +                             iort_table->length);
> > +
> > +     for (i = 0; i < iort->node_count; i++) {
> > +             if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
> > +                            "IORT node pointer overflows, bad table!\n"))
> > +                     return;
> > +
> > +             if (iort_node->type == ACPI_IORT_NODE_RMR)
> > +                     iort_node_get_rmr_info(iort_node);
> > +
> > +             iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> > +                                      iort_node->length);
> > +     }
> > +}
> >
> >   static void __init iort_init_platform_devices(void)
> >   {
> > @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
> >       }
> >
> >       iort_init_platform_devices();
> > +     iort_parse_rmr();
>
> I guess initcall ordering vs. driver registration probably covers it up,
> but for the sake of cleanliness I'd rather make sure the RMRs are fully
> discovered *before* we create the SMMU devices that we expect to start
> consuming them.
>
> Robin.
>
> >   }
> >
> >   #ifdef CONFIG_ZONE_DMA
> >
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-10-09  7:06       ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-10-09  7:06 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Steven Price, Hanjun Guo, yangyicong,
	Sami Mujawar, Will Deacon, wanghuiqiang

On Fri, Oct 8, 2021 at 2:49 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > Add support for parsing RMR node information from ACPI.
> >
> > Find the associated streamid and smmu node info from the
> > RMR node and populate a linked list with RMR memory
> > descriptors.
> >
> > Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   drivers/acpi/arm64/iort.c | 134 +++++++++++++++++++++++++++++++++++++-
> >   1 file changed, 133 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> > index 3b23fb775ac4..d76ba46ebe67 100644
> > --- a/drivers/acpi/arm64/iort.c
> > +++ b/drivers/acpi/arm64/iort.c
> > @@ -40,6 +40,8 @@ struct iort_fwnode {
> >   static LIST_HEAD(iort_fwnode_list);
> >   static DEFINE_SPINLOCK(iort_fwnode_lock);
> >
> > +static LIST_HEAD(iort_rmr_list);     /* list of RMR regions from ACPI */
> > +
> >   /**
> >    * iort_set_fwnode() - Create iort_fwnode and use it to register
> >    *                 iommu data in the iort_fwnode_list
> > @@ -393,7 +395,8 @@ static struct acpi_iort_node *iort_node_get_id(struct acpi_iort_node *node,
> >               if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
> >                   node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
> >                   node->type == ACPI_IORT_NODE_SMMU_V3 ||
> > -                 node->type == ACPI_IORT_NODE_PMCG) {
> > +                 node->type == ACPI_IORT_NODE_PMCG ||
> > +                 node->type == ACPI_IORT_NODE_RMR) {
> >                       *id_out = map->output_base;
> >                       return parent;
> >               }
> > @@ -1566,6 +1569,134 @@ static void __init iort_enable_acs(struct acpi_iort_node *iort_node)
> >   #else
> >   static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
> >   #endif
> > +static void iort_rmr_desc_check_overlap(struct acpi_iort_rmr_desc *desc, u32 count)
> > +{
> > +     int i, j;
> > +
> > +     for (i = 0; i < count; i++) {
> > +             u64 end, start = desc[i].base_address, length = desc[i].length;
> > +
> > +             end = start + length - 1;
> > +
> > +             /* Check for address overlap */
> > +             for (j = i + 1; j < count; j++) {
> > +                     u64 e_start = desc[j].base_address;
> > +                     u64 e_end = e_start + desc[j].length - 1;
> > +
> > +                     if (start <= e_end && end >= e_start)
> > +                             pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] overlaps, continue anyway\n",
> > +                                    start, end);
> > +             }
> > +     }
> > +}
> > +
> > +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> > +{
> > +     struct acpi_iort_node *smmu;
> > +     struct acpi_iort_rmr *rmr;
> > +     struct acpi_iort_rmr_desc *rmr_desc;
> > +     u32 map_count = iort_node->mapping_count;
> > +     u32 sid;
> > +     int i;
> > +
> > +     if (!iort_node->mapping_offset || map_count != 1) {
> > +             pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     /* Retrieve associated smmu and stream id */
> > +     smmu = iort_node_get_id(iort_node, &sid, 0);
> > +     if (!smmu) {
> > +             pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     /* Retrieve RMR data */
> > +     rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> > +     if (!rmr->rmr_offset || !rmr->rmr_count) {
> > +             pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> > +                    iort_node);
> > +             return;
> > +     }
> > +
> > +     rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> > +                             rmr->rmr_offset);
> > +
> > +     iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> > +
> > +     for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> > +             struct iommu_resv_region *region;
> > +             enum iommu_resv_type type;
> > +             int prot = IOMMU_READ | IOMMU_WRITE;
> > +             u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> > +
> > +             if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> > +                     /* PAGE align base addr and size */
> > +                     addr &= PAGE_MASK;
> > +                     size = PAGE_ALIGN(size + offset_in_page(rmr_desc->base_address));
> > +
> > +                     pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to 64K, continue with [0x%llx - 0x%llx]\n",
> > +                            rmr_desc->base_address,
> > +                            rmr_desc->base_address + rmr_desc->length - 1,
> > +                            addr, addr + size - 1);
> > +             }
> > +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> > +                     /*
> > +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > +                      * normally used for allocated system memory that is
> > +                      * then used for device specific reserved regions.
> > +                      */
> > +                     prot |= IOMMU_CACHE;
> > +             } else {
> > +                     type = IOMMU_RESV_DIRECT;
> > +                     /*
> > +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> > +                      * for device memory like MSI doorbell.
> > +                      */
> > +                     prot |= IOMMU_MMIO;
> > +             }
>
> I'm not sure we ever got a definitive answer to this - does DPAA2
> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> make the fewest possible assumptions, since at this point it's basically
> just a stop-gap until we can fix the spec. It's become clear that we
> can't reliably rely on guessing attributes, so I'm not too fussed about
> theoretical cases that currently don't work (due to complete lack of RMR
> support) continuing to not work for the moment, as long as we can make
> the real-world cases we actually have work at all. Anything which only
> affects performance I'd rather leave until firmware can tell us what to do.

Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
mappings.  DPAA2 is just one connected device.

-Jon

>
> > +             region = iommu_alloc_resv_region(addr, size, prot, type);
> > +             if (region) {
> > +                     region->fw_data.rmr.flags = rmr->flags;
> > +                     region->fw_data.rmr.sid = sid;
> > +                     region->fw_data.rmr.smmu = smmu;
> > +                     list_add_tail(&region->list, &iort_rmr_list);
> > +             }
> > +     }
> > +}
> > +
> > +static void __init iort_parse_rmr(void)
> > +{
> > +     struct acpi_iort_node *iort_node, *iort_end;
> > +     struct acpi_table_iort *iort;
> > +     int i;
> > +
> > +     if (iort_table->revision < 3)
> > +             return;
> > +
> > +     iort = (struct acpi_table_iort *)iort_table;
> > +
> > +     iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +                              iort->node_offset);
> > +     iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +                             iort_table->length);
> > +
> > +     for (i = 0; i < iort->node_count; i++) {
> > +             if (WARN_TAINT(iort_node >= iort_end, TAINT_FIRMWARE_WORKAROUND,
> > +                            "IORT node pointer overflows, bad table!\n"))
> > +                     return;
> > +
> > +             if (iort_node->type == ACPI_IORT_NODE_RMR)
> > +                     iort_node_get_rmr_info(iort_node);
> > +
> > +             iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> > +                                      iort_node->length);
> > +     }
> > +}
> >
> >   static void __init iort_init_platform_devices(void)
> >   {
> > @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
> >       }
> >
> >       iort_init_platform_devices();
> > +     iort_parse_rmr();
>
> I guess initcall ordering vs. driver registration probably covers it up,
> but for the sake of cleanliness I'd rather make sure the RMRs are fully
> discovered *before* we create the SMMU devices that we expect to start
> consuming them.
>
> Robin.
>
> >   }
> >
> >   #ifdef CONFIG_ZONE_DMA
> >

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
  2021-10-08 13:09     ` Robin Murphy
  (?)
@ 2021-10-09  7:07       ` Jon Nettleton
  -1 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-10-09  7:07 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Lorenzo Pieralisi, Joerg Roedel,
	Will Deacon, wanghuiqiang, Hanjun Guo, Steven Price,
	Sami Mujawar, Eric Auger, yangyicong

On Fri, Oct 8, 2021 at 3:10 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > Get ACPI IORT RMR regions associated with a dev reserved
> > so that there is a unity mapping for them in SMMU.
>
> This feels like most of it belongs in the IORT code rather than
> iommu-dma (which should save the temporary list copy as well).

See previous comment.  The original intent was for device-tree to also
be able to use these mechanisms to create RMR's and support them
in the SMMU.

-Jon

>
> Robin.
>
> > Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   drivers/iommu/dma-iommu.c | 56 +++++++++++++++++++++++++++++++++++----
> >   1 file changed, 51 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> > index 1b6e27475279..c1ae0c3d4b33 100644
> > --- a/drivers/iommu/dma-iommu.c
> > +++ b/drivers/iommu/dma-iommu.c
> > @@ -207,22 +207,68 @@ void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
> >   }
> >   EXPORT_SYMBOL(iommu_dma_put_rmrs);
> >
> > +static bool iommu_dma_dev_has_rmr(struct iommu_fwspec *fwspec,
> > +                               struct iommu_resv_region *e)
> > +{
> > +     int i;
> > +
> > +     for (i = 0; i < fwspec->num_ids; i++) {
> > +             if (e->fw_data.rmr.sid == fwspec->ids[i])
> > +                     return true;
> > +     }
> > +
> > +     return false;
> > +}
> > +
> > +static void iommu_dma_get_rmr_resv_regions(struct device *dev,
> > +                                        struct list_head *list)
> > +{
> > +     struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
> > +     struct list_head rmr_list;
> > +     struct iommu_resv_region *rmr, *tmp;
> > +
> > +     INIT_LIST_HEAD(&rmr_list);
> > +     if (iommu_dma_get_rmrs(fwspec->iommu_fwnode, &rmr_list))
> > +             return;
> > +
> > +     if (dev_is_pci(dev)) {
> > +             struct pci_dev *pdev = to_pci_dev(dev);
> > +             struct pci_host_bridge *host = pci_find_host_bridge(pdev->bus);
> > +
> > +             if (!host->preserve_config)
> > +                     return;
> > +     }
> > +
> > +     list_for_each_entry_safe(rmr, tmp, &rmr_list, list) {
> > +             if (!iommu_dma_dev_has_rmr(fwspec, rmr))
> > +                     continue;
> > +
> > +             /* Remove from iommu RMR list and add to dev resv_regions */
> > +             list_del_init(&rmr->list);
> > +             list_add_tail(&rmr->list, list);
> > +     }
> > +
> > +     iommu_dma_put_rmrs(fwspec->iommu_fwnode, &rmr_list);
> > +}
> > +
> >   /**
> >    * iommu_dma_get_resv_regions - Reserved region driver helper
> >    * @dev: Device from iommu_get_resv_regions()
> >    * @list: Reserved region list from iommu_get_resv_regions()
> >    *
> >    * IOMMU drivers can use this to implement their .get_resv_regions callback
> > - * for general non-IOMMU-specific reservations. Currently, this covers GICv3
> > - * ITS region reservation on ACPI based ARM platforms that may require HW MSI
> > - * reservation.
> > + * for general non-IOMMU-specific reservations. Currently this covers,
> > + *  -GICv3 ITS region reservation on ACPI based ARM platforms that may
> > + *   require HW MSI reservation.
> > + *  -Any ACPI IORT RMR memory range reservations (IORT spec rev E.b)
> >    */
> >   void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
> >   {
> >
> > -     if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode))
> > +     if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode)) {
> >               iort_iommu_msi_get_resv_regions(dev, list);
> > -
> > +             iommu_dma_get_rmr_resv_regions(dev, list);
> > +     }
> >   }
> >   EXPORT_SYMBOL(iommu_dma_get_resv_regions);
> >
> >

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
@ 2021-10-09  7:07       ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-10-09  7:07 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Linuxarm, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon,
	linux-arm-kernel

On Fri, Oct 8, 2021 at 3:10 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > Get ACPI IORT RMR regions associated with a dev reserved
> > so that there is a unity mapping for them in SMMU.
>
> This feels like most of it belongs in the IORT code rather than
> iommu-dma (which should save the temporary list copy as well).

See previous comment.  The original intent was for device-tree to also
be able to use these mechanisms to create RMR's and support them
in the SMMU.

-Jon

>
> Robin.
>
> > Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   drivers/iommu/dma-iommu.c | 56 +++++++++++++++++++++++++++++++++++----
> >   1 file changed, 51 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> > index 1b6e27475279..c1ae0c3d4b33 100644
> > --- a/drivers/iommu/dma-iommu.c
> > +++ b/drivers/iommu/dma-iommu.c
> > @@ -207,22 +207,68 @@ void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
> >   }
> >   EXPORT_SYMBOL(iommu_dma_put_rmrs);
> >
> > +static bool iommu_dma_dev_has_rmr(struct iommu_fwspec *fwspec,
> > +                               struct iommu_resv_region *e)
> > +{
> > +     int i;
> > +
> > +     for (i = 0; i < fwspec->num_ids; i++) {
> > +             if (e->fw_data.rmr.sid == fwspec->ids[i])
> > +                     return true;
> > +     }
> > +
> > +     return false;
> > +}
> > +
> > +static void iommu_dma_get_rmr_resv_regions(struct device *dev,
> > +                                        struct list_head *list)
> > +{
> > +     struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
> > +     struct list_head rmr_list;
> > +     struct iommu_resv_region *rmr, *tmp;
> > +
> > +     INIT_LIST_HEAD(&rmr_list);
> > +     if (iommu_dma_get_rmrs(fwspec->iommu_fwnode, &rmr_list))
> > +             return;
> > +
> > +     if (dev_is_pci(dev)) {
> > +             struct pci_dev *pdev = to_pci_dev(dev);
> > +             struct pci_host_bridge *host = pci_find_host_bridge(pdev->bus);
> > +
> > +             if (!host->preserve_config)
> > +                     return;
> > +     }
> > +
> > +     list_for_each_entry_safe(rmr, tmp, &rmr_list, list) {
> > +             if (!iommu_dma_dev_has_rmr(fwspec, rmr))
> > +                     continue;
> > +
> > +             /* Remove from iommu RMR list and add to dev resv_regions */
> > +             list_del_init(&rmr->list);
> > +             list_add_tail(&rmr->list, list);
> > +     }
> > +
> > +     iommu_dma_put_rmrs(fwspec->iommu_fwnode, &rmr_list);
> > +}
> > +
> >   /**
> >    * iommu_dma_get_resv_regions - Reserved region driver helper
> >    * @dev: Device from iommu_get_resv_regions()
> >    * @list: Reserved region list from iommu_get_resv_regions()
> >    *
> >    * IOMMU drivers can use this to implement their .get_resv_regions callback
> > - * for general non-IOMMU-specific reservations. Currently, this covers GICv3
> > - * ITS region reservation on ACPI based ARM platforms that may require HW MSI
> > - * reservation.
> > + * for general non-IOMMU-specific reservations. Currently this covers,
> > + *  -GICv3 ITS region reservation on ACPI based ARM platforms that may
> > + *   require HW MSI reservation.
> > + *  -Any ACPI IORT RMR memory range reservations (IORT spec rev E.b)
> >    */
> >   void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
> >   {
> >
> > -     if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode))
> > +     if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode)) {
> >               iort_iommu_msi_get_resv_regions(dev, list);
> > -
> > +             iommu_dma_get_rmr_resv_regions(dev, list);
> > +     }
> >   }
> >   EXPORT_SYMBOL(iommu_dma_get_resv_regions);
> >
> >
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
@ 2021-10-09  7:07       ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-10-09  7:07 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Lorenzo Pieralisi, Joerg Roedel,
	Will Deacon, wanghuiqiang, Hanjun Guo, Steven Price,
	Sami Mujawar, Eric Auger, yangyicong

On Fri, Oct 8, 2021 at 3:10 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > Get ACPI IORT RMR regions associated with a dev reserved
> > so that there is a unity mapping for them in SMMU.
>
> This feels like most of it belongs in the IORT code rather than
> iommu-dma (which should save the temporary list copy as well).

See previous comment.  The original intent was for device-tree to also
be able to use these mechanisms to create RMR's and support them
in the SMMU.

-Jon

>
> Robin.
>
> > Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   drivers/iommu/dma-iommu.c | 56 +++++++++++++++++++++++++++++++++++----
> >   1 file changed, 51 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> > index 1b6e27475279..c1ae0c3d4b33 100644
> > --- a/drivers/iommu/dma-iommu.c
> > +++ b/drivers/iommu/dma-iommu.c
> > @@ -207,22 +207,68 @@ void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
> >   }
> >   EXPORT_SYMBOL(iommu_dma_put_rmrs);
> >
> > +static bool iommu_dma_dev_has_rmr(struct iommu_fwspec *fwspec,
> > +                               struct iommu_resv_region *e)
> > +{
> > +     int i;
> > +
> > +     for (i = 0; i < fwspec->num_ids; i++) {
> > +             if (e->fw_data.rmr.sid == fwspec->ids[i])
> > +                     return true;
> > +     }
> > +
> > +     return false;
> > +}
> > +
> > +static void iommu_dma_get_rmr_resv_regions(struct device *dev,
> > +                                        struct list_head *list)
> > +{
> > +     struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
> > +     struct list_head rmr_list;
> > +     struct iommu_resv_region *rmr, *tmp;
> > +
> > +     INIT_LIST_HEAD(&rmr_list);
> > +     if (iommu_dma_get_rmrs(fwspec->iommu_fwnode, &rmr_list))
> > +             return;
> > +
> > +     if (dev_is_pci(dev)) {
> > +             struct pci_dev *pdev = to_pci_dev(dev);
> > +             struct pci_host_bridge *host = pci_find_host_bridge(pdev->bus);
> > +
> > +             if (!host->preserve_config)
> > +                     return;
> > +     }
> > +
> > +     list_for_each_entry_safe(rmr, tmp, &rmr_list, list) {
> > +             if (!iommu_dma_dev_has_rmr(fwspec, rmr))
> > +                     continue;
> > +
> > +             /* Remove from iommu RMR list and add to dev resv_regions */
> > +             list_del_init(&rmr->list);
> > +             list_add_tail(&rmr->list, list);
> > +     }
> > +
> > +     iommu_dma_put_rmrs(fwspec->iommu_fwnode, &rmr_list);
> > +}
> > +
> >   /**
> >    * iommu_dma_get_resv_regions - Reserved region driver helper
> >    * @dev: Device from iommu_get_resv_regions()
> >    * @list: Reserved region list from iommu_get_resv_regions()
> >    *
> >    * IOMMU drivers can use this to implement their .get_resv_regions callback
> > - * for general non-IOMMU-specific reservations. Currently, this covers GICv3
> > - * ITS region reservation on ACPI based ARM platforms that may require HW MSI
> > - * reservation.
> > + * for general non-IOMMU-specific reservations. Currently this covers,
> > + *  -GICv3 ITS region reservation on ACPI based ARM platforms that may
> > + *   require HW MSI reservation.
> > + *  -Any ACPI IORT RMR memory range reservations (IORT spec rev E.b)
> >    */
> >   void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
> >   {
> >
> > -     if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode))
> > +     if (!is_of_node(dev_iommu_fwspec_get(dev)->iommu_fwnode)) {
> >               iort_iommu_msi_get_resv_regions(dev, list);
> > -
> > +             iommu_dma_get_rmr_resv_regions(dev, list);
> > +     }
> >   }
> >   EXPORT_SYMBOL(iommu_dma_get_resv_regions);
> >
> >

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
  2021-10-09  6:57       ` Jon Nettleton
  (?)
@ 2021-10-11  5:47         ` Shameerali Kolothum Thodi
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-10-11  5:47 UTC (permalink / raw)
  To: Jon Nettleton, Robin Murphy
  Cc: linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Steven Price, Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Will Deacon, wanghuiqiang



> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 09 October 2021 07:58
> To: Robin Murphy <robin.murphy@arm.com>
> Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> List <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> Steven Price <steven.price@arm.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>; Sami
> Mujawar <Sami.Mujawar@arm.com>; Will Deacon <will@kernel.org>;
> wanghuiqiang <wanghuiqiang@huawei.com>
> Subject: Re: [PATCH v7 1/9] iommu: Introduce a union to struct
> iommu_resv_region
> 
> On Fri, Oct 8, 2021 at 2:14 PM Robin Murphy <robin.murphy@arm.com>
> wrote:
> >
> > On 2021-08-05 09:07, Shameer Kolothum wrote:
> > > A union is introduced to struct iommu_resv_region to hold any
> > > firmware specific data. This is in preparation to add support for
> > > IORT RMR reserve regions and the union now holds the RMR specific
> > > information.
> > >
> > > Signed-off-by: Shameer Kolothum
> > > <shameerali.kolothum.thodi@huawei.com>
> > > ---
> > >   include/linux/iommu.h | 11 +++++++++++
> > >   1 file changed, 11 insertions(+)
> > >
> > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h index
> > > 32d448050bf7..bd0e4641c569 100644
> > > --- a/include/linux/iommu.h
> > > +++ b/include/linux/iommu.h
> > > @@ -114,6 +114,13 @@ enum iommu_resv_type {
> > >       IOMMU_RESV_SW_MSI,
> > >   };
> > >
> > > +struct iommu_iort_rmr_data {
> > > +#define IOMMU_RMR_REMAP_PERMITTED    (1 << 0)
> > > +     u32 flags;
> > > +     u32 sid;        /* Stream Id associated with RMR entry */
> > > +     void *smmu;     /* Associated IORT SMMU node pointer */
> > > +};
> >
> > Do we really need to duplicate all this data? AFAICS we could just
> > save the acpi_iort_rmr pointer in the iommu_resv_region (with a
> > forward declaration here if necessary) and defer parsing its actual
> > mappings until the point where we can directly consume the results.
> 
> From earlier discussions on this patchset, the original goal was also for
> device-tree mechanisms to be able to hook into this code to support similar
> RMR's and SMMU initialization, just not through the ACPI / IORT path.

Yes. IIRC, there were some earlier attempts to have DT support for reserved regions
and there was a suggestion to provide generic interfaces so that when DT solution
comes up it is easier to add the support.

Thanks,
Shameer

> >
> > Robin.
> >
> > > +
> > >   /**
> > >    * struct iommu_resv_region - descriptor for a reserved memory region
> > >    * @list: Linked list pointers
> > > @@ -121,6 +128,7 @@ enum iommu_resv_type {
> > >    * @length: Length of the region in bytes
> > >    * @prot: IOMMU Protection flags (READ/WRITE/...)
> > >    * @type: Type of the reserved region
> > > + * @rmr: ACPI IORT RMR specific data
> > >    */
> > >   struct iommu_resv_region {
> > >       struct list_head        list;
> > > @@ -128,6 +136,9 @@ struct iommu_resv_region {
> > >       size_t                  length;
> > >       int                     prot;
> > >       enum iommu_resv_type    type;
> > > +     union {
> > > +             struct iommu_iort_rmr_data rmr;
> > > +     } fw_data;
> > >   };
> > >
> > >   /**
> > >

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
@ 2021-10-11  5:47         ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-10-11  5:47 UTC (permalink / raw)
  To: Jon Nettleton, Robin Murphy
  Cc: Linuxarm, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Will Deacon, linux-arm-kernel



> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 09 October 2021 07:58
> To: Robin Murphy <robin.murphy@arm.com>
> Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> List <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> Steven Price <steven.price@arm.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>; Sami
> Mujawar <Sami.Mujawar@arm.com>; Will Deacon <will@kernel.org>;
> wanghuiqiang <wanghuiqiang@huawei.com>
> Subject: Re: [PATCH v7 1/9] iommu: Introduce a union to struct
> iommu_resv_region
> 
> On Fri, Oct 8, 2021 at 2:14 PM Robin Murphy <robin.murphy@arm.com>
> wrote:
> >
> > On 2021-08-05 09:07, Shameer Kolothum wrote:
> > > A union is introduced to struct iommu_resv_region to hold any
> > > firmware specific data. This is in preparation to add support for
> > > IORT RMR reserve regions and the union now holds the RMR specific
> > > information.
> > >
> > > Signed-off-by: Shameer Kolothum
> > > <shameerali.kolothum.thodi@huawei.com>
> > > ---
> > >   include/linux/iommu.h | 11 +++++++++++
> > >   1 file changed, 11 insertions(+)
> > >
> > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h index
> > > 32d448050bf7..bd0e4641c569 100644
> > > --- a/include/linux/iommu.h
> > > +++ b/include/linux/iommu.h
> > > @@ -114,6 +114,13 @@ enum iommu_resv_type {
> > >       IOMMU_RESV_SW_MSI,
> > >   };
> > >
> > > +struct iommu_iort_rmr_data {
> > > +#define IOMMU_RMR_REMAP_PERMITTED    (1 << 0)
> > > +     u32 flags;
> > > +     u32 sid;        /* Stream Id associated with RMR entry */
> > > +     void *smmu;     /* Associated IORT SMMU node pointer */
> > > +};
> >
> > Do we really need to duplicate all this data? AFAICS we could just
> > save the acpi_iort_rmr pointer in the iommu_resv_region (with a
> > forward declaration here if necessary) and defer parsing its actual
> > mappings until the point where we can directly consume the results.
> 
> From earlier discussions on this patchset, the original goal was also for
> device-tree mechanisms to be able to hook into this code to support similar
> RMR's and SMMU initialization, just not through the ACPI / IORT path.

Yes. IIRC, there were some earlier attempts to have DT support for reserved regions
and there was a suggestion to provide generic interfaces so that when DT solution
comes up it is easier to add the support.

Thanks,
Shameer

> >
> > Robin.
> >
> > > +
> > >   /**
> > >    * struct iommu_resv_region - descriptor for a reserved memory region
> > >    * @list: Linked list pointers
> > > @@ -121,6 +128,7 @@ enum iommu_resv_type {
> > >    * @length: Length of the region in bytes
> > >    * @prot: IOMMU Protection flags (READ/WRITE/...)
> > >    * @type: Type of the reserved region
> > > + * @rmr: ACPI IORT RMR specific data
> > >    */
> > >   struct iommu_resv_region {
> > >       struct list_head        list;
> > > @@ -128,6 +136,9 @@ struct iommu_resv_region {
> > >       size_t                  length;
> > >       int                     prot;
> > >       enum iommu_resv_type    type;
> > > +     union {
> > > +             struct iommu_iort_rmr_data rmr;
> > > +     } fw_data;
> > >   };
> > >
> > >   /**
> > >
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
@ 2021-10-11  5:47         ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-10-11  5:47 UTC (permalink / raw)
  To: Jon Nettleton, Robin Murphy
  Cc: linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Steven Price, Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Will Deacon, wanghuiqiang



> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 09 October 2021 07:58
> To: Robin Murphy <robin.murphy@arm.com>
> Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> List <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> Steven Price <steven.price@arm.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>; Sami
> Mujawar <Sami.Mujawar@arm.com>; Will Deacon <will@kernel.org>;
> wanghuiqiang <wanghuiqiang@huawei.com>
> Subject: Re: [PATCH v7 1/9] iommu: Introduce a union to struct
> iommu_resv_region
> 
> On Fri, Oct 8, 2021 at 2:14 PM Robin Murphy <robin.murphy@arm.com>
> wrote:
> >
> > On 2021-08-05 09:07, Shameer Kolothum wrote:
> > > A union is introduced to struct iommu_resv_region to hold any
> > > firmware specific data. This is in preparation to add support for
> > > IORT RMR reserve regions and the union now holds the RMR specific
> > > information.
> > >
> > > Signed-off-by: Shameer Kolothum
> > > <shameerali.kolothum.thodi@huawei.com>
> > > ---
> > >   include/linux/iommu.h | 11 +++++++++++
> > >   1 file changed, 11 insertions(+)
> > >
> > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h index
> > > 32d448050bf7..bd0e4641c569 100644
> > > --- a/include/linux/iommu.h
> > > +++ b/include/linux/iommu.h
> > > @@ -114,6 +114,13 @@ enum iommu_resv_type {
> > >       IOMMU_RESV_SW_MSI,
> > >   };
> > >
> > > +struct iommu_iort_rmr_data {
> > > +#define IOMMU_RMR_REMAP_PERMITTED    (1 << 0)
> > > +     u32 flags;
> > > +     u32 sid;        /* Stream Id associated with RMR entry */
> > > +     void *smmu;     /* Associated IORT SMMU node pointer */
> > > +};
> >
> > Do we really need to duplicate all this data? AFAICS we could just
> > save the acpi_iort_rmr pointer in the iommu_resv_region (with a
> > forward declaration here if necessary) and defer parsing its actual
> > mappings until the point where we can directly consume the results.
> 
> From earlier discussions on this patchset, the original goal was also for
> device-tree mechanisms to be able to hook into this code to support similar
> RMR's and SMMU initialization, just not through the ACPI / IORT path.

Yes. IIRC, there were some earlier attempts to have DT support for reserved regions
and there was a suggestion to provide generic interfaces so that when DT solution
comes up it is easier to add the support.

Thanks,
Shameer

> >
> > Robin.
> >
> > > +
> > >   /**
> > >    * struct iommu_resv_region - descriptor for a reserved memory region
> > >    * @list: Linked list pointers
> > > @@ -121,6 +128,7 @@ enum iommu_resv_type {
> > >    * @length: Length of the region in bytes
> > >    * @prot: IOMMU Protection flags (READ/WRITE/...)
> > >    * @type: Type of the reserved region
> > > + * @rmr: ACPI IORT RMR specific data
> > >    */
> > >   struct iommu_resv_region {
> > >       struct list_head        list;
> > > @@ -128,6 +136,9 @@ struct iommu_resv_region {
> > >       size_t                  length;
> > >       int                     prot;
> > >       enum iommu_resv_type    type;
> > > +     union {
> > > +             struct iommu_iort_rmr_data rmr;
> > > +     } fw_data;
> > >   };
> > >
> > >   /**
> > >
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 3/9] iommu/dma: Introduce generic helper to retrieve RMR info
  2021-10-08 13:03     ` Robin Murphy
  (?)
@ 2021-10-11  5:51       ` Shameerali Kolothum Thodi
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-10-11  5:51 UTC (permalink / raw)
  To: Robin Murphy, linux-arm-kernel, linux-acpi, iommu
  Cc: Linuxarm, lorenzo.pieralisi, joro, will, wanghuiqiang,
	Guohanjun (Hanjun Guo),
	steven.price, Sami.Mujawar, jon, eric.auger, yangyicong



> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy@arm.com]
> Sent: 08 October 2021 14:04
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org
> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> joro@8bytes.org; will@kernel.org; wanghuiqiang
> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; steven.price@arm.com; Sami.Mujawar@arm.com;
> jon@solid-run.com; eric.auger@redhat.com; yangyicong
> <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 3/9] iommu/dma: Introduce generic helper to retrieve
> RMR info
> 
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > Reserved Memory Regions(RMR) associated with an IOMMU can be
> > described through ACPI IORT tables in systems with devices
> > that require a unity mapping or bypass for those
> > regions.
> >
> > Introduce a generic interface so that IOMMU drivers can retrieve
> > and set up necessary mappings.
> >
> > Signed-off-by: Shameer Kolothum
> <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   drivers/iommu/dma-iommu.c | 29 +++++++++++++++++++++++++++++
> >   include/linux/dma-iommu.h | 13 +++++++++++++
> >   2 files changed, 42 insertions(+)
> >
> > diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> > index 98ba927aee1a..2fa2445e9070 100644
> > --- a/drivers/iommu/dma-iommu.c
> > +++ b/drivers/iommu/dma-iommu.c
> > @@ -174,6 +174,35 @@ void iommu_put_dma_cookie(struct
> iommu_domain *domain)
> >   }
> >   EXPORT_SYMBOL(iommu_put_dma_cookie);
> >
> > +/**
> > + *
> > + * iommu_dma_get_rmrs - Retrieve Reserved Memory Regions(RMRs)
> associated
> > + *                      with a given IOMMU
> > + * @iommu_fwnode: fwnode associated with IOMMU
> > + * @list: RMR list to be populated
> > + *
> > + */
> > +int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
> > +		       struct list_head *list)
> > +{
> > +	return -EINVAL;
> > +}
> > +EXPORT_SYMBOL(iommu_dma_get_rmrs);
> > +
> > +/**
> > + *
> > + * iommu_dma_put_rmrs - Release Reserved Memory Regions(RMRs)
> associated
> > + *                      with a given IOMMU
> > + * @iommu_fwnode: fwnode associated with IOMMU
> > + * @list: RMR list
> > + *
> > + */
> > +void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
> > +			struct list_head *list)
> > +{
> > +}
> > +EXPORT_SYMBOL(iommu_dma_put_rmrs);
> 
> Unless there's something special you expect to need to do here, can we
> just uphold the prevailing expectation that resv_regions are kmalloc()ed
> and can be freed directly by the generic function?

Right. I think we can do that. 

> 
> > +
> >   /**
> >    * iommu_dma_get_resv_regions - Reserved region driver helper
> >    * @dev: Device from iommu_get_resv_regions()
> > diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
> > index 758ca4694257..3b7b2d096c6e 100644
> > --- a/include/linux/dma-iommu.h
> > +++ b/include/linux/dma-iommu.h
> > @@ -42,12 +42,16 @@ void iommu_dma_free_cpu_cached_iovas(unsigned
> int cpu,
> >
> >   extern bool iommu_dma_forcedac;
> >
> > +int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head
> *list);
> > +void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head
> *list);
> > +
> >   #else /* CONFIG_IOMMU_DMA */
> >
> >   struct iommu_domain;
> >   struct msi_desc;
> >   struct msi_msg;
> >   struct device;
> > +struct fwnode_handle;
> >
> >   static inline void iommu_setup_dma_ops(struct device *dev, u64
> dma_base,
> >   				       u64 dma_limit)
> > @@ -83,5 +87,14 @@ static inline void
> iommu_dma_get_resv_regions(struct device *dev, struct list_he
> >   {
> >   }
> >
> > +static int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct
> list_head *list)
> > +{
> > +	return -ENODEV;
> 
> Hmm, if this needs to be stubbed at all then returning an error seems
> like probably the wrong thing to do. I guess it's for 32-bit builds of
> arm-smmu? It is not an error if the firmware describes no RMRs because
> there are no RMRs, so it hardly deserves to be an error if there are no
> RMRs simply because the firmware isn't ACPI.

Yes, definitely not an error return. I will change that.

Thanks,
Shameer

> 
> Robin.
> 
> > +}
> > +
> > +static void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct
> list_head *list)
> > +{
> > +}
> > +
> >   #endif	/* CONFIG_IOMMU_DMA */
> >   #endif	/* __DMA_IOMMU_H */
> >

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 3/9] iommu/dma: Introduce generic helper to retrieve RMR info
@ 2021-10-11  5:51       ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-10-11  5:51 UTC (permalink / raw)
  To: Robin Murphy, linux-arm-kernel, linux-acpi, iommu
  Cc: jon, Linuxarm, steven.price, Guohanjun (Hanjun Guo),
	yangyicong, Sami.Mujawar, will, wanghuiqiang



> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy@arm.com]
> Sent: 08 October 2021 14:04
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org
> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> joro@8bytes.org; will@kernel.org; wanghuiqiang
> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; steven.price@arm.com; Sami.Mujawar@arm.com;
> jon@solid-run.com; eric.auger@redhat.com; yangyicong
> <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 3/9] iommu/dma: Introduce generic helper to retrieve
> RMR info
> 
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > Reserved Memory Regions(RMR) associated with an IOMMU can be
> > described through ACPI IORT tables in systems with devices
> > that require a unity mapping or bypass for those
> > regions.
> >
> > Introduce a generic interface so that IOMMU drivers can retrieve
> > and set up necessary mappings.
> >
> > Signed-off-by: Shameer Kolothum
> <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   drivers/iommu/dma-iommu.c | 29 +++++++++++++++++++++++++++++
> >   include/linux/dma-iommu.h | 13 +++++++++++++
> >   2 files changed, 42 insertions(+)
> >
> > diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> > index 98ba927aee1a..2fa2445e9070 100644
> > --- a/drivers/iommu/dma-iommu.c
> > +++ b/drivers/iommu/dma-iommu.c
> > @@ -174,6 +174,35 @@ void iommu_put_dma_cookie(struct
> iommu_domain *domain)
> >   }
> >   EXPORT_SYMBOL(iommu_put_dma_cookie);
> >
> > +/**
> > + *
> > + * iommu_dma_get_rmrs - Retrieve Reserved Memory Regions(RMRs)
> associated
> > + *                      with a given IOMMU
> > + * @iommu_fwnode: fwnode associated with IOMMU
> > + * @list: RMR list to be populated
> > + *
> > + */
> > +int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
> > +		       struct list_head *list)
> > +{
> > +	return -EINVAL;
> > +}
> > +EXPORT_SYMBOL(iommu_dma_get_rmrs);
> > +
> > +/**
> > + *
> > + * iommu_dma_put_rmrs - Release Reserved Memory Regions(RMRs)
> associated
> > + *                      with a given IOMMU
> > + * @iommu_fwnode: fwnode associated with IOMMU
> > + * @list: RMR list
> > + *
> > + */
> > +void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
> > +			struct list_head *list)
> > +{
> > +}
> > +EXPORT_SYMBOL(iommu_dma_put_rmrs);
> 
> Unless there's something special you expect to need to do here, can we
> just uphold the prevailing expectation that resv_regions are kmalloc()ed
> and can be freed directly by the generic function?

Right. I think we can do that. 

> 
> > +
> >   /**
> >    * iommu_dma_get_resv_regions - Reserved region driver helper
> >    * @dev: Device from iommu_get_resv_regions()
> > diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
> > index 758ca4694257..3b7b2d096c6e 100644
> > --- a/include/linux/dma-iommu.h
> > +++ b/include/linux/dma-iommu.h
> > @@ -42,12 +42,16 @@ void iommu_dma_free_cpu_cached_iovas(unsigned
> int cpu,
> >
> >   extern bool iommu_dma_forcedac;
> >
> > +int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head
> *list);
> > +void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head
> *list);
> > +
> >   #else /* CONFIG_IOMMU_DMA */
> >
> >   struct iommu_domain;
> >   struct msi_desc;
> >   struct msi_msg;
> >   struct device;
> > +struct fwnode_handle;
> >
> >   static inline void iommu_setup_dma_ops(struct device *dev, u64
> dma_base,
> >   				       u64 dma_limit)
> > @@ -83,5 +87,14 @@ static inline void
> iommu_dma_get_resv_regions(struct device *dev, struct list_he
> >   {
> >   }
> >
> > +static int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct
> list_head *list)
> > +{
> > +	return -ENODEV;
> 
> Hmm, if this needs to be stubbed at all then returning an error seems
> like probably the wrong thing to do. I guess it's for 32-bit builds of
> arm-smmu? It is not an error if the firmware describes no RMRs because
> there are no RMRs, so it hardly deserves to be an error if there are no
> RMRs simply because the firmware isn't ACPI.

Yes, definitely not an error return. I will change that.

Thanks,
Shameer

> 
> Robin.
> 
> > +}
> > +
> > +static void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct
> list_head *list)
> > +{
> > +}
> > +
> >   #endif	/* CONFIG_IOMMU_DMA */
> >   #endif	/* __DMA_IOMMU_H */
> >
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 3/9] iommu/dma: Introduce generic helper to retrieve RMR info
@ 2021-10-11  5:51       ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-10-11  5:51 UTC (permalink / raw)
  To: Robin Murphy, linux-arm-kernel, linux-acpi, iommu
  Cc: Linuxarm, lorenzo.pieralisi, joro, will, wanghuiqiang,
	Guohanjun (Hanjun Guo),
	steven.price, Sami.Mujawar, jon, eric.auger, yangyicong



> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy@arm.com]
> Sent: 08 October 2021 14:04
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org
> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> joro@8bytes.org; will@kernel.org; wanghuiqiang
> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; steven.price@arm.com; Sami.Mujawar@arm.com;
> jon@solid-run.com; eric.auger@redhat.com; yangyicong
> <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 3/9] iommu/dma: Introduce generic helper to retrieve
> RMR info
> 
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > Reserved Memory Regions(RMR) associated with an IOMMU can be
> > described through ACPI IORT tables in systems with devices
> > that require a unity mapping or bypass for those
> > regions.
> >
> > Introduce a generic interface so that IOMMU drivers can retrieve
> > and set up necessary mappings.
> >
> > Signed-off-by: Shameer Kolothum
> <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   drivers/iommu/dma-iommu.c | 29 +++++++++++++++++++++++++++++
> >   include/linux/dma-iommu.h | 13 +++++++++++++
> >   2 files changed, 42 insertions(+)
> >
> > diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> > index 98ba927aee1a..2fa2445e9070 100644
> > --- a/drivers/iommu/dma-iommu.c
> > +++ b/drivers/iommu/dma-iommu.c
> > @@ -174,6 +174,35 @@ void iommu_put_dma_cookie(struct
> iommu_domain *domain)
> >   }
> >   EXPORT_SYMBOL(iommu_put_dma_cookie);
> >
> > +/**
> > + *
> > + * iommu_dma_get_rmrs - Retrieve Reserved Memory Regions(RMRs)
> associated
> > + *                      with a given IOMMU
> > + * @iommu_fwnode: fwnode associated with IOMMU
> > + * @list: RMR list to be populated
> > + *
> > + */
> > +int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
> > +		       struct list_head *list)
> > +{
> > +	return -EINVAL;
> > +}
> > +EXPORT_SYMBOL(iommu_dma_get_rmrs);
> > +
> > +/**
> > + *
> > + * iommu_dma_put_rmrs - Release Reserved Memory Regions(RMRs)
> associated
> > + *                      with a given IOMMU
> > + * @iommu_fwnode: fwnode associated with IOMMU
> > + * @list: RMR list
> > + *
> > + */
> > +void iommu_dma_put_rmrs(struct fwnode_handle *iommu_fwnode,
> > +			struct list_head *list)
> > +{
> > +}
> > +EXPORT_SYMBOL(iommu_dma_put_rmrs);
> 
> Unless there's something special you expect to need to do here, can we
> just uphold the prevailing expectation that resv_regions are kmalloc()ed
> and can be freed directly by the generic function?

Right. I think we can do that. 

> 
> > +
> >   /**
> >    * iommu_dma_get_resv_regions - Reserved region driver helper
> >    * @dev: Device from iommu_get_resv_regions()
> > diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
> > index 758ca4694257..3b7b2d096c6e 100644
> > --- a/include/linux/dma-iommu.h
> > +++ b/include/linux/dma-iommu.h
> > @@ -42,12 +42,16 @@ void iommu_dma_free_cpu_cached_iovas(unsigned
> int cpu,
> >
> >   extern bool iommu_dma_forcedac;
> >
> > +int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head
> *list);
> > +void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct list_head
> *list);
> > +
> >   #else /* CONFIG_IOMMU_DMA */
> >
> >   struct iommu_domain;
> >   struct msi_desc;
> >   struct msi_msg;
> >   struct device;
> > +struct fwnode_handle;
> >
> >   static inline void iommu_setup_dma_ops(struct device *dev, u64
> dma_base,
> >   				       u64 dma_limit)
> > @@ -83,5 +87,14 @@ static inline void
> iommu_dma_get_resv_regions(struct device *dev, struct list_he
> >   {
> >   }
> >
> > +static int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct
> list_head *list)
> > +{
> > +	return -ENODEV;
> 
> Hmm, if this needs to be stubbed at all then returning an error seems
> like probably the wrong thing to do. I guess it's for 32-bit builds of
> arm-smmu? It is not an error if the firmware describes no RMRs because
> there are no RMRs, so it hardly deserves to be an error if there are no
> RMRs simply because the firmware isn't ACPI.

Yes, definitely not an error return. I will change that.

Thanks,
Shameer

> 
> Robin.
> 
> > +}
> > +
> > +static void iommu_dma_put_rmrs(struct fwnode_handle *iommu, struct
> list_head *list)
> > +{
> > +}
> > +
> >   #endif	/* CONFIG_IOMMU_DMA */
> >   #endif	/* __DMA_IOMMU_H */
> >
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-10-08 12:48     ` Robin Murphy
  (?)
@ 2021-10-11  5:59       ` Shameerali Kolothum Thodi
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-10-11  5:59 UTC (permalink / raw)
  To: Robin Murphy, linux-arm-kernel, linux-acpi, iommu
  Cc: jon, Linuxarm, steven.price, Guohanjun (Hanjun Guo),
	yangyicong, Sami.Mujawar, will, wanghuiqiang



> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy@arm.com]
> Sent: 08 October 2021 13:49
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org
> Cc: jon@solid-run.com; Linuxarm <linuxarm@huawei.com>;
> steven.price@arm.com; Guohanjun (Hanjun Guo) <guohanjun@huawei.com>;
> yangyicong <yangyicong@huawei.com>; Sami.Mujawar@arm.com;
> will@kernel.org; wanghuiqiang <wanghuiqiang@huawei.com>
> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> 
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > Add support for parsing RMR node information from ACPI.
> >
> > Find the associated streamid and smmu node info from the
> > RMR node and populate a linked list with RMR memory
> > descriptors.
> >
> > Signed-off-by: Shameer Kolothum
> <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   drivers/acpi/arm64/iort.c | 134
> +++++++++++++++++++++++++++++++++++++-
> >   1 file changed, 133 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> > index 3b23fb775ac4..d76ba46ebe67 100644
> > --- a/drivers/acpi/arm64/iort.c
> > +++ b/drivers/acpi/arm64/iort.c
> > @@ -40,6 +40,8 @@ struct iort_fwnode {
> >   static LIST_HEAD(iort_fwnode_list);
> >   static DEFINE_SPINLOCK(iort_fwnode_lock);
> >
> > +static LIST_HEAD(iort_rmr_list);	/* list of RMR regions from ACPI */
> > +
> >   /**
> >    * iort_set_fwnode() - Create iort_fwnode and use it to register
> >    *		       iommu data in the iort_fwnode_list
> > @@ -393,7 +395,8 @@ static struct acpi_iort_node
> *iort_node_get_id(struct acpi_iort_node *node,
> >   		if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
> >   		    node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
> >   		    node->type == ACPI_IORT_NODE_SMMU_V3 ||
> > -		    node->type == ACPI_IORT_NODE_PMCG) {
> > +		    node->type == ACPI_IORT_NODE_PMCG ||
> > +		    node->type == ACPI_IORT_NODE_RMR) {
> >   			*id_out = map->output_base;
> >   			return parent;
> >   		}
> > @@ -1566,6 +1569,134 @@ static void __init iort_enable_acs(struct
> acpi_iort_node *iort_node)
> >   #else
> >   static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
> >   #endif
> > +static void iort_rmr_desc_check_overlap(struct acpi_iort_rmr_desc *desc,
> u32 count)
> > +{
> > +	int i, j;
> > +
> > +	for (i = 0; i < count; i++) {
> > +		u64 end, start = desc[i].base_address, length = desc[i].length;
> > +
> > +		end = start + length - 1;
> > +
> > +		/* Check for address overlap */
> > +		for (j = i + 1; j < count; j++) {
> > +			u64 e_start = desc[j].base_address;
> > +			u64 e_end = e_start + desc[j].length - 1;
> > +
> > +			if (start <= e_end && end >= e_start)
> > +				pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] overlaps,
> continue anyway\n",
> > +				       start, end);
> > +		}
> > +	}
> > +}
> > +
> > +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> > +{
> > +	struct acpi_iort_node *smmu;
> > +	struct acpi_iort_rmr *rmr;
> > +	struct acpi_iort_rmr_desc *rmr_desc;
> > +	u32 map_count = iort_node->mapping_count;
> > +	u32 sid;
> > +	int i;
> > +
> > +	if (!iort_node->mapping_offset || map_count != 1) {
> > +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> > +		       iort_node);
> > +		return;
> > +	}
> > +
> > +	/* Retrieve associated smmu and stream id */
> > +	smmu = iort_node_get_id(iort_node, &sid, 0);
> > +	if (!smmu) {
> > +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR
> node %p\n",
> > +		       iort_node);
> > +		return;
> > +	}
> > +
> > +	/* Retrieve RMR data */
> > +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> > +	if (!rmr->rmr_offset || !rmr->rmr_count) {
> > +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR
> node %p\n",
> > +		       iort_node);
> > +		return;
> > +	}
> > +
> > +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> > +				rmr->rmr_offset);
> > +
> > +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> > +
> > +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> > +		struct iommu_resv_region *region;
> > +		enum iommu_resv_type type;
> > +		int prot = IOMMU_READ | IOMMU_WRITE;
> > +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> > +
> > +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> > +			/* PAGE align base addr and size */
> > +			addr &= PAGE_MASK;
> > +			size = PAGE_ALIGN(size +
> offset_in_page(rmr_desc->base_address));
> > +
> > +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to
> 64K, continue with [0x%llx - 0x%llx]\n",
> > +			       rmr_desc->base_address,
> > +			       rmr_desc->base_address + rmr_desc->length - 1,
> > +			       addr, addr + size - 1);
> > +		}
> > +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > +			type = IOMMU_RESV_DIRECT_RELAXABLE;
> > +			/*
> > +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > +			 * normally used for allocated system memory that is
> > +			 * then used for device specific reserved regions.
> > +			 */
> > +			prot |= IOMMU_CACHE;
> > +		} else {
> > +			type = IOMMU_RESV_DIRECT;
> > +			/*
> > +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally
> used
> > +			 * for device memory like MSI doorbell.
> > +			 */
> > +			prot |= IOMMU_MMIO;
> > +		}
> 
> I'm not sure we ever got a definitive answer to this - does DPAA2
> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> make the fewest possible assumptions, since at this point it's basically
> just a stop-gap until we can fix the spec. It's become clear that we
> can't reliably rely on guessing attributes, so I'm not too fussed about
> theoretical cases that currently don't work (due to complete lack of RMR
> support) continuing to not work for the moment, as long as we can make
> the real-world cases we actually have work at all. Anything which only
> affects performance I'd rather leave until firmware can tell us what to do.

Just to report back, we have done some basic sanity tests with IOMMU_MMIO
set as default and it works for us. But I see that it doesn't for Jon's case.
So not sure what the stop-gap can be.. Can we use the _CCA + EFI approach
and override it later when the spec gets updated?

Thanks,
Shameer
 
> 
> > +		region = iommu_alloc_resv_region(addr, size, prot, type);
> > +		if (region) {
> > +			region->fw_data.rmr.flags = rmr->flags;
> > +			region->fw_data.rmr.sid = sid;
> > +			region->fw_data.rmr.smmu = smmu;
> > +			list_add_tail(&region->list, &iort_rmr_list);
> > +		}
> > +	}
> > +}
> > +
> > +static void __init iort_parse_rmr(void)
> > +{
> > +	struct acpi_iort_node *iort_node, *iort_end;
> > +	struct acpi_table_iort *iort;
> > +	int i;
> > +
> > +	if (iort_table->revision < 3)
> > +		return;
> > +
> > +	iort = (struct acpi_table_iort *)iort_table;
> > +
> > +	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +				 iort->node_offset);
> > +	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +				iort_table->length);
> > +
> > +	for (i = 0; i < iort->node_count; i++) {
> > +		if (WARN_TAINT(iort_node >= iort_end,
> TAINT_FIRMWARE_WORKAROUND,
> > +			       "IORT node pointer overflows, bad table!\n"))
> > +			return;
> > +
> > +		if (iort_node->type == ACPI_IORT_NODE_RMR)
> > +			iort_node_get_rmr_info(iort_node);
> > +
> > +		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> > +					 iort_node->length);
> > +	}
> > +}
> >
> >   static void __init iort_init_platform_devices(void)
> >   {
> > @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
> >   	}
> >
> >   	iort_init_platform_devices();
> > +	iort_parse_rmr();
> 
> I guess initcall ordering vs. driver registration probably covers it up,
> but for the sake of cleanliness I'd rather make sure the RMRs are fully
> discovered *before* we create the SMMU devices that we expect to start
> consuming them.
> 
> Robin.
> 
> >   }
> >
> >   #ifdef CONFIG_ZONE_DMA
> >

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-10-11  5:59       ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-10-11  5:59 UTC (permalink / raw)
  To: Robin Murphy, linux-arm-kernel, linux-acpi, iommu
  Cc: jon, Linuxarm, steven.price, Guohanjun (Hanjun Guo),
	yangyicong, Sami.Mujawar, will, wanghuiqiang



> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy@arm.com]
> Sent: 08 October 2021 13:49
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org
> Cc: jon@solid-run.com; Linuxarm <linuxarm@huawei.com>;
> steven.price@arm.com; Guohanjun (Hanjun Guo) <guohanjun@huawei.com>;
> yangyicong <yangyicong@huawei.com>; Sami.Mujawar@arm.com;
> will@kernel.org; wanghuiqiang <wanghuiqiang@huawei.com>
> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> 
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > Add support for parsing RMR node information from ACPI.
> >
> > Find the associated streamid and smmu node info from the
> > RMR node and populate a linked list with RMR memory
> > descriptors.
> >
> > Signed-off-by: Shameer Kolothum
> <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   drivers/acpi/arm64/iort.c | 134
> +++++++++++++++++++++++++++++++++++++-
> >   1 file changed, 133 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> > index 3b23fb775ac4..d76ba46ebe67 100644
> > --- a/drivers/acpi/arm64/iort.c
> > +++ b/drivers/acpi/arm64/iort.c
> > @@ -40,6 +40,8 @@ struct iort_fwnode {
> >   static LIST_HEAD(iort_fwnode_list);
> >   static DEFINE_SPINLOCK(iort_fwnode_lock);
> >
> > +static LIST_HEAD(iort_rmr_list);	/* list of RMR regions from ACPI */
> > +
> >   /**
> >    * iort_set_fwnode() - Create iort_fwnode and use it to register
> >    *		       iommu data in the iort_fwnode_list
> > @@ -393,7 +395,8 @@ static struct acpi_iort_node
> *iort_node_get_id(struct acpi_iort_node *node,
> >   		if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
> >   		    node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
> >   		    node->type == ACPI_IORT_NODE_SMMU_V3 ||
> > -		    node->type == ACPI_IORT_NODE_PMCG) {
> > +		    node->type == ACPI_IORT_NODE_PMCG ||
> > +		    node->type == ACPI_IORT_NODE_RMR) {
> >   			*id_out = map->output_base;
> >   			return parent;
> >   		}
> > @@ -1566,6 +1569,134 @@ static void __init iort_enable_acs(struct
> acpi_iort_node *iort_node)
> >   #else
> >   static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
> >   #endif
> > +static void iort_rmr_desc_check_overlap(struct acpi_iort_rmr_desc *desc,
> u32 count)
> > +{
> > +	int i, j;
> > +
> > +	for (i = 0; i < count; i++) {
> > +		u64 end, start = desc[i].base_address, length = desc[i].length;
> > +
> > +		end = start + length - 1;
> > +
> > +		/* Check for address overlap */
> > +		for (j = i + 1; j < count; j++) {
> > +			u64 e_start = desc[j].base_address;
> > +			u64 e_end = e_start + desc[j].length - 1;
> > +
> > +			if (start <= e_end && end >= e_start)
> > +				pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] overlaps,
> continue anyway\n",
> > +				       start, end);
> > +		}
> > +	}
> > +}
> > +
> > +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> > +{
> > +	struct acpi_iort_node *smmu;
> > +	struct acpi_iort_rmr *rmr;
> > +	struct acpi_iort_rmr_desc *rmr_desc;
> > +	u32 map_count = iort_node->mapping_count;
> > +	u32 sid;
> > +	int i;
> > +
> > +	if (!iort_node->mapping_offset || map_count != 1) {
> > +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> > +		       iort_node);
> > +		return;
> > +	}
> > +
> > +	/* Retrieve associated smmu and stream id */
> > +	smmu = iort_node_get_id(iort_node, &sid, 0);
> > +	if (!smmu) {
> > +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR
> node %p\n",
> > +		       iort_node);
> > +		return;
> > +	}
> > +
> > +	/* Retrieve RMR data */
> > +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> > +	if (!rmr->rmr_offset || !rmr->rmr_count) {
> > +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR
> node %p\n",
> > +		       iort_node);
> > +		return;
> > +	}
> > +
> > +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> > +				rmr->rmr_offset);
> > +
> > +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> > +
> > +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> > +		struct iommu_resv_region *region;
> > +		enum iommu_resv_type type;
> > +		int prot = IOMMU_READ | IOMMU_WRITE;
> > +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> > +
> > +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> > +			/* PAGE align base addr and size */
> > +			addr &= PAGE_MASK;
> > +			size = PAGE_ALIGN(size +
> offset_in_page(rmr_desc->base_address));
> > +
> > +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to
> 64K, continue with [0x%llx - 0x%llx]\n",
> > +			       rmr_desc->base_address,
> > +			       rmr_desc->base_address + rmr_desc->length - 1,
> > +			       addr, addr + size - 1);
> > +		}
> > +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > +			type = IOMMU_RESV_DIRECT_RELAXABLE;
> > +			/*
> > +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > +			 * normally used for allocated system memory that is
> > +			 * then used for device specific reserved regions.
> > +			 */
> > +			prot |= IOMMU_CACHE;
> > +		} else {
> > +			type = IOMMU_RESV_DIRECT;
> > +			/*
> > +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally
> used
> > +			 * for device memory like MSI doorbell.
> > +			 */
> > +			prot |= IOMMU_MMIO;
> > +		}
> 
> I'm not sure we ever got a definitive answer to this - does DPAA2
> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> make the fewest possible assumptions, since at this point it's basically
> just a stop-gap until we can fix the spec. It's become clear that we
> can't reliably rely on guessing attributes, so I'm not too fussed about
> theoretical cases that currently don't work (due to complete lack of RMR
> support) continuing to not work for the moment, as long as we can make
> the real-world cases we actually have work at all. Anything which only
> affects performance I'd rather leave until firmware can tell us what to do.

Just to report back, we have done some basic sanity tests with IOMMU_MMIO
set as default and it works for us. But I see that it doesn't for Jon's case.
So not sure what the stop-gap can be.. Can we use the _CCA + EFI approach
and override it later when the spec gets updated?

Thanks,
Shameer
 
> 
> > +		region = iommu_alloc_resv_region(addr, size, prot, type);
> > +		if (region) {
> > +			region->fw_data.rmr.flags = rmr->flags;
> > +			region->fw_data.rmr.sid = sid;
> > +			region->fw_data.rmr.smmu = smmu;
> > +			list_add_tail(&region->list, &iort_rmr_list);
> > +		}
> > +	}
> > +}
> > +
> > +static void __init iort_parse_rmr(void)
> > +{
> > +	struct acpi_iort_node *iort_node, *iort_end;
> > +	struct acpi_table_iort *iort;
> > +	int i;
> > +
> > +	if (iort_table->revision < 3)
> > +		return;
> > +
> > +	iort = (struct acpi_table_iort *)iort_table;
> > +
> > +	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +				 iort->node_offset);
> > +	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +				iort_table->length);
> > +
> > +	for (i = 0; i < iort->node_count; i++) {
> > +		if (WARN_TAINT(iort_node >= iort_end,
> TAINT_FIRMWARE_WORKAROUND,
> > +			       "IORT node pointer overflows, bad table!\n"))
> > +			return;
> > +
> > +		if (iort_node->type == ACPI_IORT_NODE_RMR)
> > +			iort_node_get_rmr_info(iort_node);
> > +
> > +		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> > +					 iort_node->length);
> > +	}
> > +}
> >
> >   static void __init iort_init_platform_devices(void)
> >   {
> > @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
> >   	}
> >
> >   	iort_init_platform_devices();
> > +	iort_parse_rmr();
> 
> I guess initcall ordering vs. driver registration probably covers it up,
> but for the sake of cleanliness I'd rather make sure the RMRs are fully
> discovered *before* we create the SMMU devices that we expect to start
> consuming them.
> 
> Robin.
> 
> >   }
> >
> >   #ifdef CONFIG_ZONE_DMA
> >
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-10-11  5:59       ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-10-11  5:59 UTC (permalink / raw)
  To: Robin Murphy, linux-arm-kernel, linux-acpi, iommu
  Cc: jon, Linuxarm, steven.price, Guohanjun (Hanjun Guo),
	yangyicong, Sami.Mujawar, will, wanghuiqiang



> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy@arm.com]
> Sent: 08 October 2021 13:49
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org
> Cc: jon@solid-run.com; Linuxarm <linuxarm@huawei.com>;
> steven.price@arm.com; Guohanjun (Hanjun Guo) <guohanjun@huawei.com>;
> yangyicong <yangyicong@huawei.com>; Sami.Mujawar@arm.com;
> will@kernel.org; wanghuiqiang <wanghuiqiang@huawei.com>
> Subject: Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
> 
> On 2021-08-05 09:07, Shameer Kolothum wrote:
> > Add support for parsing RMR node information from ACPI.
> >
> > Find the associated streamid and smmu node info from the
> > RMR node and populate a linked list with RMR memory
> > descriptors.
> >
> > Signed-off-by: Shameer Kolothum
> <shameerali.kolothum.thodi@huawei.com>
> > ---
> >   drivers/acpi/arm64/iort.c | 134
> +++++++++++++++++++++++++++++++++++++-
> >   1 file changed, 133 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> > index 3b23fb775ac4..d76ba46ebe67 100644
> > --- a/drivers/acpi/arm64/iort.c
> > +++ b/drivers/acpi/arm64/iort.c
> > @@ -40,6 +40,8 @@ struct iort_fwnode {
> >   static LIST_HEAD(iort_fwnode_list);
> >   static DEFINE_SPINLOCK(iort_fwnode_lock);
> >
> > +static LIST_HEAD(iort_rmr_list);	/* list of RMR regions from ACPI */
> > +
> >   /**
> >    * iort_set_fwnode() - Create iort_fwnode and use it to register
> >    *		       iommu data in the iort_fwnode_list
> > @@ -393,7 +395,8 @@ static struct acpi_iort_node
> *iort_node_get_id(struct acpi_iort_node *node,
> >   		if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
> >   		    node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
> >   		    node->type == ACPI_IORT_NODE_SMMU_V3 ||
> > -		    node->type == ACPI_IORT_NODE_PMCG) {
> > +		    node->type == ACPI_IORT_NODE_PMCG ||
> > +		    node->type == ACPI_IORT_NODE_RMR) {
> >   			*id_out = map->output_base;
> >   			return parent;
> >   		}
> > @@ -1566,6 +1569,134 @@ static void __init iort_enable_acs(struct
> acpi_iort_node *iort_node)
> >   #else
> >   static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
> >   #endif
> > +static void iort_rmr_desc_check_overlap(struct acpi_iort_rmr_desc *desc,
> u32 count)
> > +{
> > +	int i, j;
> > +
> > +	for (i = 0; i < count; i++) {
> > +		u64 end, start = desc[i].base_address, length = desc[i].length;
> > +
> > +		end = start + length - 1;
> > +
> > +		/* Check for address overlap */
> > +		for (j = i + 1; j < count; j++) {
> > +			u64 e_start = desc[j].base_address;
> > +			u64 e_end = e_start + desc[j].length - 1;
> > +
> > +			if (start <= e_end && end >= e_start)
> > +				pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] overlaps,
> continue anyway\n",
> > +				       start, end);
> > +		}
> > +	}
> > +}
> > +
> > +static void __init iort_node_get_rmr_info(struct acpi_iort_node *iort_node)
> > +{
> > +	struct acpi_iort_node *smmu;
> > +	struct acpi_iort_rmr *rmr;
> > +	struct acpi_iort_rmr_desc *rmr_desc;
> > +	u32 map_count = iort_node->mapping_count;
> > +	u32 sid;
> > +	int i;
> > +
> > +	if (!iort_node->mapping_offset || map_count != 1) {
> > +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> > +		       iort_node);
> > +		return;
> > +	}
> > +
> > +	/* Retrieve associated smmu and stream id */
> > +	smmu = iort_node_get_id(iort_node, &sid, 0);
> > +	if (!smmu) {
> > +		pr_err(FW_BUG "Invalid SMMU reference, skipping RMR
> node %p\n",
> > +		       iort_node);
> > +		return;
> > +	}
> > +
> > +	/* Retrieve RMR data */
> > +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> > +	if (!rmr->rmr_offset || !rmr->rmr_count) {
> > +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR
> node %p\n",
> > +		       iort_node);
> > +		return;
> > +	}
> > +
> > +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> > +				rmr->rmr_offset);
> > +
> > +	iort_rmr_desc_check_overlap(rmr_desc, rmr->rmr_count);
> > +
> > +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> > +		struct iommu_resv_region *region;
> > +		enum iommu_resv_type type;
> > +		int prot = IOMMU_READ | IOMMU_WRITE;
> > +		u64 addr = rmr_desc->base_address, size = rmr_desc->length;
> > +
> > +		if (!IS_ALIGNED(addr, SZ_64K) || !IS_ALIGNED(size, SZ_64K)) {
> > +			/* PAGE align base addr and size */
> > +			addr &= PAGE_MASK;
> > +			size = PAGE_ALIGN(size +
> offset_in_page(rmr_desc->base_address));
> > +
> > +			pr_err(FW_BUG "RMR descriptor[0x%llx - 0x%llx] not aligned to
> 64K, continue with [0x%llx - 0x%llx]\n",
> > +			       rmr_desc->base_address,
> > +			       rmr_desc->base_address + rmr_desc->length - 1,
> > +			       addr, addr + size - 1);
> > +		}
> > +		if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > +			type = IOMMU_RESV_DIRECT_RELAXABLE;
> > +			/*
> > +			 * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > +			 * normally used for allocated system memory that is
> > +			 * then used for device specific reserved regions.
> > +			 */
> > +			prot |= IOMMU_CACHE;
> > +		} else {
> > +			type = IOMMU_RESV_DIRECT;
> > +			/*
> > +			 * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally
> used
> > +			 * for device memory like MSI doorbell.
> > +			 */
> > +			prot |= IOMMU_MMIO;
> > +		}
> 
> I'm not sure we ever got a definitive answer to this - does DPAA2
> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> make the fewest possible assumptions, since at this point it's basically
> just a stop-gap until we can fix the spec. It's become clear that we
> can't reliably rely on guessing attributes, so I'm not too fussed about
> theoretical cases that currently don't work (due to complete lack of RMR
> support) continuing to not work for the moment, as long as we can make
> the real-world cases we actually have work at all. Anything which only
> affects performance I'd rather leave until firmware can tell us what to do.

Just to report back, we have done some basic sanity tests with IOMMU_MMIO
set as default and it works for us. But I see that it doesn't for Jon's case.
So not sure what the stop-gap can be.. Can we use the _CCA + EFI approach
and override it later when the spec gets updated?

Thanks,
Shameer
 
> 
> > +		region = iommu_alloc_resv_region(addr, size, prot, type);
> > +		if (region) {
> > +			region->fw_data.rmr.flags = rmr->flags;
> > +			region->fw_data.rmr.sid = sid;
> > +			region->fw_data.rmr.smmu = smmu;
> > +			list_add_tail(&region->list, &iort_rmr_list);
> > +		}
> > +	}
> > +}
> > +
> > +static void __init iort_parse_rmr(void)
> > +{
> > +	struct acpi_iort_node *iort_node, *iort_end;
> > +	struct acpi_table_iort *iort;
> > +	int i;
> > +
> > +	if (iort_table->revision < 3)
> > +		return;
> > +
> > +	iort = (struct acpi_table_iort *)iort_table;
> > +
> > +	iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +				 iort->node_offset);
> > +	iort_end = ACPI_ADD_PTR(struct acpi_iort_node, iort,
> > +				iort_table->length);
> > +
> > +	for (i = 0; i < iort->node_count; i++) {
> > +		if (WARN_TAINT(iort_node >= iort_end,
> TAINT_FIRMWARE_WORKAROUND,
> > +			       "IORT node pointer overflows, bad table!\n"))
> > +			return;
> > +
> > +		if (iort_node->type == ACPI_IORT_NODE_RMR)
> > +			iort_node_get_rmr_info(iort_node);
> > +
> > +		iort_node = ACPI_ADD_PTR(struct acpi_iort_node, iort_node,
> > +					 iort_node->length);
> > +	}
> > +}
> >
> >   static void __init iort_init_platform_devices(void)
> >   {
> > @@ -1636,6 +1767,7 @@ void __init acpi_iort_init(void)
> >   	}
> >
> >   	iort_init_platform_devices();
> > +	iort_parse_rmr();
> 
> I guess initcall ordering vs. driver registration probably covers it up,
> but for the sake of cleanliness I'd rather make sure the RMRs are fully
> discovered *before* we create the SMMU devices that we expect to start
> consuming them.
> 
> Robin.
> 
> >   }
> >
> >   #ifdef CONFIG_ZONE_DMA
> >
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
  2021-10-11  5:47         ` Shameerali Kolothum Thodi
  (?)
@ 2021-10-11 13:47           ` Robin Murphy
  -1 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-11 13:47 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi, Jon Nettleton
  Cc: linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Steven Price, Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Will Deacon, wanghuiqiang

On 2021-10-11 06:47, Shameerali Kolothum Thodi wrote:
> 
> 
>> -----Original Message-----
>> From: Jon Nettleton [mailto:jon@solid-run.com]
>> Sent: 09 October 2021 07:58
>> To: Robin Murphy <robin.murphy@arm.com>
>> Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
>> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
>> List <linux-acpi@vger.kernel.org>; Linux IOMMU
>> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
>> Steven Price <steven.price@arm.com>; Guohanjun (Hanjun Guo)
>> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>; Sami
>> Mujawar <Sami.Mujawar@arm.com>; Will Deacon <will@kernel.org>;
>> wanghuiqiang <wanghuiqiang@huawei.com>
>> Subject: Re: [PATCH v7 1/9] iommu: Introduce a union to struct
>> iommu_resv_region
>>
>> On Fri, Oct 8, 2021 at 2:14 PM Robin Murphy <robin.murphy@arm.com>
>> wrote:
>>>
>>> On 2021-08-05 09:07, Shameer Kolothum wrote:
>>>> A union is introduced to struct iommu_resv_region to hold any
>>>> firmware specific data. This is in preparation to add support for
>>>> IORT RMR reserve regions and the union now holds the RMR specific
>>>> information.
>>>>
>>>> Signed-off-by: Shameer Kolothum
>>>> <shameerali.kolothum.thodi@huawei.com>
>>>> ---
>>>>    include/linux/iommu.h | 11 +++++++++++
>>>>    1 file changed, 11 insertions(+)
>>>>
>>>> diff --git a/include/linux/iommu.h b/include/linux/iommu.h index
>>>> 32d448050bf7..bd0e4641c569 100644
>>>> --- a/include/linux/iommu.h
>>>> +++ b/include/linux/iommu.h
>>>> @@ -114,6 +114,13 @@ enum iommu_resv_type {
>>>>        IOMMU_RESV_SW_MSI,
>>>>    };
>>>>
>>>> +struct iommu_iort_rmr_data {
>>>> +#define IOMMU_RMR_REMAP_PERMITTED    (1 << 0)
>>>> +     u32 flags;
>>>> +     u32 sid;        /* Stream Id associated with RMR entry */
>>>> +     void *smmu;     /* Associated IORT SMMU node pointer */
>>>> +};
>>>
>>> Do we really need to duplicate all this data? AFAICS we could just
>>> save the acpi_iort_rmr pointer in the iommu_resv_region (with a
>>> forward declaration here if necessary) and defer parsing its actual
>>> mappings until the point where we can directly consume the results.
>>
>>  From earlier discussions on this patchset, the original goal was also for
>> device-tree mechanisms to be able to hook into this code to support similar
>> RMR's and SMMU initialization, just not through the ACPI / IORT path.
> 
> Yes. IIRC, there were some earlier attempts to have DT support for reserved regions
> and there was a suggestion to provide generic interfaces so that when DT solution
> comes up it is easier to add the support.

OK, but in that case why is every single part of it IORT-specific in 
either name, description or function?

Regardless, s/acpi_iort_rmr/original firmware descriptor of whatever 
variety/ and my comment still stands. If a firmware-specific structure 
is still going to exist to begin with, then what do we gain from 
interpreting details earlier than needed and wasting memory storing 
copies of them? This isn't something we're looking up hundreds of times 
per second and need to cache in some more efficient format.

Furthermore, it seems unlikely that the eventual DT solution would end 
up being semantically identical to IORT RMRs, so there's every 
possibility that the One True Abstract Structure would need changing to 
work for another firmware implementation anyway. Heck, it might not even 
fit future IORT if it becomes permissible for multiple StreamIDs to 
share a single RMR descriptor.

Thanks,
Robin.

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
@ 2021-10-11 13:47           ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-11 13:47 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi, Jon Nettleton
  Cc: Linuxarm, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Will Deacon, linux-arm-kernel

On 2021-10-11 06:47, Shameerali Kolothum Thodi wrote:
> 
> 
>> -----Original Message-----
>> From: Jon Nettleton [mailto:jon@solid-run.com]
>> Sent: 09 October 2021 07:58
>> To: Robin Murphy <robin.murphy@arm.com>
>> Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
>> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
>> List <linux-acpi@vger.kernel.org>; Linux IOMMU
>> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
>> Steven Price <steven.price@arm.com>; Guohanjun (Hanjun Guo)
>> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>; Sami
>> Mujawar <Sami.Mujawar@arm.com>; Will Deacon <will@kernel.org>;
>> wanghuiqiang <wanghuiqiang@huawei.com>
>> Subject: Re: [PATCH v7 1/9] iommu: Introduce a union to struct
>> iommu_resv_region
>>
>> On Fri, Oct 8, 2021 at 2:14 PM Robin Murphy <robin.murphy@arm.com>
>> wrote:
>>>
>>> On 2021-08-05 09:07, Shameer Kolothum wrote:
>>>> A union is introduced to struct iommu_resv_region to hold any
>>>> firmware specific data. This is in preparation to add support for
>>>> IORT RMR reserve regions and the union now holds the RMR specific
>>>> information.
>>>>
>>>> Signed-off-by: Shameer Kolothum
>>>> <shameerali.kolothum.thodi@huawei.com>
>>>> ---
>>>>    include/linux/iommu.h | 11 +++++++++++
>>>>    1 file changed, 11 insertions(+)
>>>>
>>>> diff --git a/include/linux/iommu.h b/include/linux/iommu.h index
>>>> 32d448050bf7..bd0e4641c569 100644
>>>> --- a/include/linux/iommu.h
>>>> +++ b/include/linux/iommu.h
>>>> @@ -114,6 +114,13 @@ enum iommu_resv_type {
>>>>        IOMMU_RESV_SW_MSI,
>>>>    };
>>>>
>>>> +struct iommu_iort_rmr_data {
>>>> +#define IOMMU_RMR_REMAP_PERMITTED    (1 << 0)
>>>> +     u32 flags;
>>>> +     u32 sid;        /* Stream Id associated with RMR entry */
>>>> +     void *smmu;     /* Associated IORT SMMU node pointer */
>>>> +};
>>>
>>> Do we really need to duplicate all this data? AFAICS we could just
>>> save the acpi_iort_rmr pointer in the iommu_resv_region (with a
>>> forward declaration here if necessary) and defer parsing its actual
>>> mappings until the point where we can directly consume the results.
>>
>>  From earlier discussions on this patchset, the original goal was also for
>> device-tree mechanisms to be able to hook into this code to support similar
>> RMR's and SMMU initialization, just not through the ACPI / IORT path.
> 
> Yes. IIRC, there were some earlier attempts to have DT support for reserved regions
> and there was a suggestion to provide generic interfaces so that when DT solution
> comes up it is easier to add the support.

OK, but in that case why is every single part of it IORT-specific in 
either name, description or function?

Regardless, s/acpi_iort_rmr/original firmware descriptor of whatever 
variety/ and my comment still stands. If a firmware-specific structure 
is still going to exist to begin with, then what do we gain from 
interpreting details earlier than needed and wasting memory storing 
copies of them? This isn't something we're looking up hundreds of times 
per second and need to cache in some more efficient format.

Furthermore, it seems unlikely that the eventual DT solution would end 
up being semantically identical to IORT RMRs, so there's every 
possibility that the One True Abstract Structure would need changing to 
work for another firmware implementation anyway. Heck, it might not even 
fit future IORT if it becomes permissible for multiple StreamIDs to 
share a single RMR descriptor.

Thanks,
Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region
@ 2021-10-11 13:47           ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-11 13:47 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi, Jon Nettleton
  Cc: linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Steven Price, Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Will Deacon, wanghuiqiang

On 2021-10-11 06:47, Shameerali Kolothum Thodi wrote:
> 
> 
>> -----Original Message-----
>> From: Jon Nettleton [mailto:jon@solid-run.com]
>> Sent: 09 October 2021 07:58
>> To: Robin Murphy <robin.murphy@arm.com>
>> Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
>> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
>> List <linux-acpi@vger.kernel.org>; Linux IOMMU
>> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
>> Steven Price <steven.price@arm.com>; Guohanjun (Hanjun Guo)
>> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>; Sami
>> Mujawar <Sami.Mujawar@arm.com>; Will Deacon <will@kernel.org>;
>> wanghuiqiang <wanghuiqiang@huawei.com>
>> Subject: Re: [PATCH v7 1/9] iommu: Introduce a union to struct
>> iommu_resv_region
>>
>> On Fri, Oct 8, 2021 at 2:14 PM Robin Murphy <robin.murphy@arm.com>
>> wrote:
>>>
>>> On 2021-08-05 09:07, Shameer Kolothum wrote:
>>>> A union is introduced to struct iommu_resv_region to hold any
>>>> firmware specific data. This is in preparation to add support for
>>>> IORT RMR reserve regions and the union now holds the RMR specific
>>>> information.
>>>>
>>>> Signed-off-by: Shameer Kolothum
>>>> <shameerali.kolothum.thodi@huawei.com>
>>>> ---
>>>>    include/linux/iommu.h | 11 +++++++++++
>>>>    1 file changed, 11 insertions(+)
>>>>
>>>> diff --git a/include/linux/iommu.h b/include/linux/iommu.h index
>>>> 32d448050bf7..bd0e4641c569 100644
>>>> --- a/include/linux/iommu.h
>>>> +++ b/include/linux/iommu.h
>>>> @@ -114,6 +114,13 @@ enum iommu_resv_type {
>>>>        IOMMU_RESV_SW_MSI,
>>>>    };
>>>>
>>>> +struct iommu_iort_rmr_data {
>>>> +#define IOMMU_RMR_REMAP_PERMITTED    (1 << 0)
>>>> +     u32 flags;
>>>> +     u32 sid;        /* Stream Id associated with RMR entry */
>>>> +     void *smmu;     /* Associated IORT SMMU node pointer */
>>>> +};
>>>
>>> Do we really need to duplicate all this data? AFAICS we could just
>>> save the acpi_iort_rmr pointer in the iommu_resv_region (with a
>>> forward declaration here if necessary) and defer parsing its actual
>>> mappings until the point where we can directly consume the results.
>>
>>  From earlier discussions on this patchset, the original goal was also for
>> device-tree mechanisms to be able to hook into this code to support similar
>> RMR's and SMMU initialization, just not through the ACPI / IORT path.
> 
> Yes. IIRC, there were some earlier attempts to have DT support for reserved regions
> and there was a suggestion to provide generic interfaces so that when DT solution
> comes up it is easier to add the support.

OK, but in that case why is every single part of it IORT-specific in 
either name, description or function?

Regardless, s/acpi_iort_rmr/original firmware descriptor of whatever 
variety/ and my comment still stands. If a firmware-specific structure 
is still going to exist to begin with, then what do we gain from 
interpreting details earlier than needed and wasting memory storing 
copies of them? This isn't something we're looking up hundreds of times 
per second and need to cache in some more efficient format.

Furthermore, it seems unlikely that the eventual DT solution would end 
up being semantically identical to IORT RMRs, so there's every 
possibility that the One True Abstract Structure would need changing to 
work for another firmware implementation anyway. Heck, it might not even 
fit future IORT if it becomes permissible for multiple StreamIDs to 
share a single RMR descriptor.

Thanks,
Robin.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-10-09  7:06       ` Jon Nettleton
  (?)
@ 2021-10-11 14:04         ` Robin Murphy
  -1 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-11 14:04 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Linuxarm, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon,
	linux-arm-kernel

On 2021-10-09 08:06, Jon Nettleton wrote:
[...]
>>> +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
>>> +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
>>> +                     /*
>>> +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
>>> +                      * normally used for allocated system memory that is
>>> +                      * then used for device specific reserved regions.
>>> +                      */
>>> +                     prot |= IOMMU_CACHE;
>>> +             } else {
>>> +                     type = IOMMU_RESV_DIRECT;
>>> +                     /*
>>> +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
>>> +                      * for device memory like MSI doorbell.
>>> +                      */
>>> +                     prot |= IOMMU_MMIO;
>>> +             }
>>
>> I'm not sure we ever got a definitive answer to this - does DPAA2
>> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
>> make the fewest possible assumptions, since at this point it's basically
>> just a stop-gap until we can fix the spec. It's become clear that we
>> can't reliably rely on guessing attributes, so I'm not too fussed about
>> theoretical cases that currently don't work (due to complete lack of RMR
>> support) continuing to not work for the moment, as long as we can make
>> the real-world cases we actually have work at all. Anything which only
>> affects performance I'd rather leave until firmware can tell us what to do.
> 
> Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
> mappings.  DPAA2 is just one connected device.

Apologies if I'm being overly loose with terminology there - my point of 
reference for this hardware is documentation for the old LS2080A, where 
the "DPAA2 Reference Manual" gives a strong impression that the MC is a 
component belonging to the overall DPAA2 architecture. Either way it 
technically stands to reason that the other DPAA2 components would only 
be usable if the MC itself works (unless I've been holding a major 
misconception about that for years as well).

In the context of this discussion, please consider any reference I may 
make to bits of NXP's hardware to be shorthand for "the thing for which 
NXP have a vested interest in IORT RMRs".

Thanks,
Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-10-11 14:04         ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-11 14:04 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Steven Price, Hanjun Guo, yangyicong,
	Sami Mujawar, Will Deacon, wanghuiqiang

On 2021-10-09 08:06, Jon Nettleton wrote:
[...]
>>> +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
>>> +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
>>> +                     /*
>>> +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
>>> +                      * normally used for allocated system memory that is
>>> +                      * then used for device specific reserved regions.
>>> +                      */
>>> +                     prot |= IOMMU_CACHE;
>>> +             } else {
>>> +                     type = IOMMU_RESV_DIRECT;
>>> +                     /*
>>> +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
>>> +                      * for device memory like MSI doorbell.
>>> +                      */
>>> +                     prot |= IOMMU_MMIO;
>>> +             }
>>
>> I'm not sure we ever got a definitive answer to this - does DPAA2
>> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
>> make the fewest possible assumptions, since at this point it's basically
>> just a stop-gap until we can fix the spec. It's become clear that we
>> can't reliably rely on guessing attributes, so I'm not too fussed about
>> theoretical cases that currently don't work (due to complete lack of RMR
>> support) continuing to not work for the moment, as long as we can make
>> the real-world cases we actually have work at all. Anything which only
>> affects performance I'd rather leave until firmware can tell us what to do.
> 
> Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
> mappings.  DPAA2 is just one connected device.

Apologies if I'm being overly loose with terminology there - my point of 
reference for this hardware is documentation for the old LS2080A, where 
the "DPAA2 Reference Manual" gives a strong impression that the MC is a 
component belonging to the overall DPAA2 architecture. Either way it 
technically stands to reason that the other DPAA2 components would only 
be usable if the MC itself works (unless I've been holding a major 
misconception about that for years as well).

In the context of this discussion, please consider any reference I may 
make to bits of NXP's hardware to be shorthand for "the thing for which 
NXP have a vested interest in IORT RMRs".

Thanks,
Robin.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-10-11 14:04         ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-11 14:04 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Steven Price, Hanjun Guo, yangyicong,
	Sami Mujawar, Will Deacon, wanghuiqiang

On 2021-10-09 08:06, Jon Nettleton wrote:
[...]
>>> +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
>>> +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
>>> +                     /*
>>> +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
>>> +                      * normally used for allocated system memory that is
>>> +                      * then used for device specific reserved regions.
>>> +                      */
>>> +                     prot |= IOMMU_CACHE;
>>> +             } else {
>>> +                     type = IOMMU_RESV_DIRECT;
>>> +                     /*
>>> +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
>>> +                      * for device memory like MSI doorbell.
>>> +                      */
>>> +                     prot |= IOMMU_MMIO;
>>> +             }
>>
>> I'm not sure we ever got a definitive answer to this - does DPAA2
>> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
>> make the fewest possible assumptions, since at this point it's basically
>> just a stop-gap until we can fix the spec. It's become clear that we
>> can't reliably rely on guessing attributes, so I'm not too fussed about
>> theoretical cases that currently don't work (due to complete lack of RMR
>> support) continuing to not work for the moment, as long as we can make
>> the real-world cases we actually have work at all. Anything which only
>> affects performance I'd rather leave until firmware can tell us what to do.
> 
> Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
> mappings.  DPAA2 is just one connected device.

Apologies if I'm being overly loose with terminology there - my point of 
reference for this hardware is documentation for the old LS2080A, where 
the "DPAA2 Reference Manual" gives a strong impression that the MC is a 
component belonging to the overall DPAA2 architecture. Either way it 
technically stands to reason that the other DPAA2 components would only 
be usable if the MC itself works (unless I've been holding a major 
misconception about that for years as well).

In the context of this discussion, please consider any reference I may 
make to bits of NXP's hardware to be shorthand for "the thing for which 
NXP have a vested interest in IORT RMRs".

Thanks,
Robin.

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
  2021-10-09  7:07       ` Jon Nettleton
  (?)
@ 2021-10-11 15:00         ` Robin Murphy
  -1 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-11 15:00 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Lorenzo Pieralisi, Joerg Roedel,
	Will Deacon, wanghuiqiang, Hanjun Guo, Steven Price,
	Sami Mujawar, Eric Auger, yangyicong

On 2021-10-09 08:07, Jon Nettleton wrote:
> On Fri, Oct 8, 2021 at 3:10 PM Robin Murphy <robin.murphy@arm.com> wrote:
>>
>> On 2021-08-05 09:07, Shameer Kolothum wrote:
>>> Get ACPI IORT RMR regions associated with a dev reserved
>>> so that there is a unity mapping for them in SMMU.
>>
>> This feels like most of it belongs in the IORT code rather than
>> iommu-dma (which should save the temporary list copy as well).
> 
> See previous comment.  The original intent was for device-tree to also
> be able to use these mechanisms to create RMR's and support them
> in the SMMU.

Can you clarify how code behind an "if (!is_of_node(...))" check 
alongside other IORT-specific code is expected to be useful for DT?

Yes, iommu_dma_get_resv_regions() itself wants to end up serving as an 
abstraction layer, but that still doesn't mean it has to do much more 
than dispatch into firmware-specific backends as appropriate.

Robin.

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
@ 2021-10-11 15:00         ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-11 15:00 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Linuxarm, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon,
	linux-arm-kernel

On 2021-10-09 08:07, Jon Nettleton wrote:
> On Fri, Oct 8, 2021 at 3:10 PM Robin Murphy <robin.murphy@arm.com> wrote:
>>
>> On 2021-08-05 09:07, Shameer Kolothum wrote:
>>> Get ACPI IORT RMR regions associated with a dev reserved
>>> so that there is a unity mapping for them in SMMU.
>>
>> This feels like most of it belongs in the IORT code rather than
>> iommu-dma (which should save the temporary list copy as well).
> 
> See previous comment.  The original intent was for device-tree to also
> be able to use these mechanisms to create RMR's and support them
> in the SMMU.

Can you clarify how code behind an "if (!is_of_node(...))" check 
alongside other IORT-specific code is expected to be useful for DT?

Yes, iommu_dma_get_resv_regions() itself wants to end up serving as an 
abstraction layer, but that still doesn't mean it has to do much more 
than dispatch into firmware-specific backends as appropriate.

Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
@ 2021-10-11 15:00         ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-10-11 15:00 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Lorenzo Pieralisi, Joerg Roedel,
	Will Deacon, wanghuiqiang, Hanjun Guo, Steven Price,
	Sami Mujawar, Eric Auger, yangyicong

On 2021-10-09 08:07, Jon Nettleton wrote:
> On Fri, Oct 8, 2021 at 3:10 PM Robin Murphy <robin.murphy@arm.com> wrote:
>>
>> On 2021-08-05 09:07, Shameer Kolothum wrote:
>>> Get ACPI IORT RMR regions associated with a dev reserved
>>> so that there is a unity mapping for them in SMMU.
>>
>> This feels like most of it belongs in the IORT code rather than
>> iommu-dma (which should save the temporary list copy as well).
> 
> See previous comment.  The original intent was for device-tree to also
> be able to use these mechanisms to create RMR's and support them
> in the SMMU.

Can you clarify how code behind an "if (!is_of_node(...))" check 
alongside other IORT-specific code is expected to be useful for DT?

Yes, iommu_dma_get_resv_regions() itself wants to end up serving as an 
abstraction layer, but that still doesn't mean it has to do much more 
than dispatch into firmware-specific backends as appropriate.

Robin.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
  2021-10-11 15:00         ` Robin Murphy
  (?)
@ 2021-10-11 15:42           ` Shameerali Kolothum Thodi
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-10-11 15:42 UTC (permalink / raw)
  To: Robin Murphy, Jon Nettleton
  Cc: linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Lorenzo Pieralisi, Joerg Roedel, Will Deacon, wanghuiqiang,
	Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong



> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy@arm.com]
> Sent: 11 October 2021 16:01
> To: Jon Nettleton <jon@solid-run.com>
> Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> List <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel
> <joro@8bytes.org>; Will Deacon <will@kernel.org>; wanghuiqiang
> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> Mujawar <Sami.Mujawar@arm.com>; Eric Auger <eric.auger@redhat.com>;
> yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated
> with a dev
> 
> On 2021-10-09 08:07, Jon Nettleton wrote:
> > On Fri, Oct 8, 2021 at 3:10 PM Robin Murphy <robin.murphy@arm.com>
> wrote:
> >>
> >> On 2021-08-05 09:07, Shameer Kolothum wrote:
> >>> Get ACPI IORT RMR regions associated with a dev reserved
> >>> so that there is a unity mapping for them in SMMU.
> >>
> >> This feels like most of it belongs in the IORT code rather than
> >> iommu-dma (which should save the temporary list copy as well).
> >
> > See previous comment.  The original intent was for device-tree to also
> > be able to use these mechanisms to create RMR's and support them
> > in the SMMU.
> 
> Can you clarify how code behind an "if (!is_of_node(...))" check
> alongside other IORT-specific code is expected to be useful for DT?
> 
> Yes, iommu_dma_get_resv_regions() itself wants to end up serving as an
> abstraction layer, but that still doesn't mean it has to do much more
> than dispatch into firmware-specific backends as appropriate.

(Resending as I accidently replied earlier from our internal ML id. Sorry)

The way I thought about is as below,

1.  iommu_dma_get_resv_regions() will invoke the common iommu_dma_get_rmr_resv_regions().
    Yes, the if (!is_of_node(...)) is not required here.
2.  iommu_dma_get_rmr_resv_regions() calls iommu_dma_get_rmrs().
    iommu_dma_get_rmrs() has the  (!is_of_node(...)) check to call into IORT or DT specific functions
    to retrieve the RMR reserve regions associated with a given iommu_fwnode.
3.  The common iommu_dma_get_rmr_resv_regions() further checks for PCI host preserve_config
    and whether the returned RMR list actually has any dev specific region to reserve or not.

So the only firmware specific backend is handled inside the iommu_dma_get_rmrs() and that is also called
from the SMMU driver probe to install bypass SIDs.

Anyway, if the eventual DT implementation or further IORT spec changes makes this abstraction
irrelevant I am Ok to move this into the IORT code.

Thanks,
Shameer

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
@ 2021-10-11 15:42           ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-10-11 15:42 UTC (permalink / raw)
  To: Robin Murphy, Jon Nettleton
  Cc: Linuxarm, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Guohanjun (Hanjun Guo),
	yangyicong, Sami Mujawar, Will Deacon, linux-arm-kernel



> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy@arm.com]
> Sent: 11 October 2021 16:01
> To: Jon Nettleton <jon@solid-run.com>
> Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> List <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel
> <joro@8bytes.org>; Will Deacon <will@kernel.org>; wanghuiqiang
> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> Mujawar <Sami.Mujawar@arm.com>; Eric Auger <eric.auger@redhat.com>;
> yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated
> with a dev
> 
> On 2021-10-09 08:07, Jon Nettleton wrote:
> > On Fri, Oct 8, 2021 at 3:10 PM Robin Murphy <robin.murphy@arm.com>
> wrote:
> >>
> >> On 2021-08-05 09:07, Shameer Kolothum wrote:
> >>> Get ACPI IORT RMR regions associated with a dev reserved
> >>> so that there is a unity mapping for them in SMMU.
> >>
> >> This feels like most of it belongs in the IORT code rather than
> >> iommu-dma (which should save the temporary list copy as well).
> >
> > See previous comment.  The original intent was for device-tree to also
> > be able to use these mechanisms to create RMR's and support them
> > in the SMMU.
> 
> Can you clarify how code behind an "if (!is_of_node(...))" check
> alongside other IORT-specific code is expected to be useful for DT?
> 
> Yes, iommu_dma_get_resv_regions() itself wants to end up serving as an
> abstraction layer, but that still doesn't mean it has to do much more
> than dispatch into firmware-specific backends as appropriate.

(Resending as I accidently replied earlier from our internal ML id. Sorry)

The way I thought about is as below,

1.  iommu_dma_get_resv_regions() will invoke the common iommu_dma_get_rmr_resv_regions().
    Yes, the if (!is_of_node(...)) is not required here.
2.  iommu_dma_get_rmr_resv_regions() calls iommu_dma_get_rmrs().
    iommu_dma_get_rmrs() has the  (!is_of_node(...)) check to call into IORT or DT specific functions
    to retrieve the RMR reserve regions associated with a given iommu_fwnode.
3.  The common iommu_dma_get_rmr_resv_regions() further checks for PCI host preserve_config
    and whether the returned RMR list actually has any dev specific region to reserve or not.

So the only firmware specific backend is handled inside the iommu_dma_get_rmrs() and that is also called
from the SMMU driver probe to install bypass SIDs.

Anyway, if the eventual DT implementation or further IORT spec changes makes this abstraction
irrelevant I am Ok to move this into the IORT code.

Thanks,
Shameer
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev
@ 2021-10-11 15:42           ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2021-10-11 15:42 UTC (permalink / raw)
  To: Robin Murphy, Jon Nettleton
  Cc: linux-arm-kernel, ACPI Devel Maling List, Linux IOMMU, Linuxarm,
	Lorenzo Pieralisi, Joerg Roedel, Will Deacon, wanghuiqiang,
	Guohanjun (Hanjun Guo),
	Steven Price, Sami Mujawar, Eric Auger, yangyicong



> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy@arm.com]
> Sent: 11 October 2021 16:01
> To: Jon Nettleton <jon@solid-run.com>
> Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling
> List <linux-acpi@vger.kernel.org>; Linux IOMMU
> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;
> Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel
> <joro@8bytes.org>; Will Deacon <will@kernel.org>; wanghuiqiang
> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; Steven Price <steven.price@arm.com>; Sami
> Mujawar <Sami.Mujawar@arm.com>; Eric Auger <eric.auger@redhat.com>;
> yangyicong <yangyicong@huawei.com>
> Subject: Re: [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated
> with a dev
> 
> On 2021-10-09 08:07, Jon Nettleton wrote:
> > On Fri, Oct 8, 2021 at 3:10 PM Robin Murphy <robin.murphy@arm.com>
> wrote:
> >>
> >> On 2021-08-05 09:07, Shameer Kolothum wrote:
> >>> Get ACPI IORT RMR regions associated with a dev reserved
> >>> so that there is a unity mapping for them in SMMU.
> >>
> >> This feels like most of it belongs in the IORT code rather than
> >> iommu-dma (which should save the temporary list copy as well).
> >
> > See previous comment.  The original intent was for device-tree to also
> > be able to use these mechanisms to create RMR's and support them
> > in the SMMU.
> 
> Can you clarify how code behind an "if (!is_of_node(...))" check
> alongside other IORT-specific code is expected to be useful for DT?
> 
> Yes, iommu_dma_get_resv_regions() itself wants to end up serving as an
> abstraction layer, but that still doesn't mean it has to do much more
> than dispatch into firmware-specific backends as appropriate.

(Resending as I accidently replied earlier from our internal ML id. Sorry)

The way I thought about is as below,

1.  iommu_dma_get_resv_regions() will invoke the common iommu_dma_get_rmr_resv_regions().
    Yes, the if (!is_of_node(...)) is not required here.
2.  iommu_dma_get_rmr_resv_regions() calls iommu_dma_get_rmrs().
    iommu_dma_get_rmrs() has the  (!is_of_node(...)) check to call into IORT or DT specific functions
    to retrieve the RMR reserve regions associated with a given iommu_fwnode.
3.  The common iommu_dma_get_rmr_resv_regions() further checks for PCI host preserve_config
    and whether the returned RMR list actually has any dev specific region to reserve or not.

So the only firmware specific backend is handled inside the iommu_dma_get_rmrs() and that is also called
from the SMMU driver probe to install bypass SIDs.

Anyway, if the eventual DT implementation or further IORT spec changes makes this abstraction
irrelevant I am Ok to move this into the IORT code.

Thanks,
Shameer
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-10-11 14:04         ` Robin Murphy
  (?)
@ 2021-10-12  8:00           ` Jon Nettleton
  -1 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-10-12  8:00 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Linuxarm, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon,
	linux-arm-kernel

On Mon, Oct 11, 2021 at 4:04 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-10-09 08:06, Jon Nettleton wrote:
> [...]
> >>> +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> >>> +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> >>> +                     /*
> >>> +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> >>> +                      * normally used for allocated system memory that is
> >>> +                      * then used for device specific reserved regions.
> >>> +                      */
> >>> +                     prot |= IOMMU_CACHE;
> >>> +             } else {
> >>> +                     type = IOMMU_RESV_DIRECT;
> >>> +                     /*
> >>> +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> >>> +                      * for device memory like MSI doorbell.
> >>> +                      */
> >>> +                     prot |= IOMMU_MMIO;
> >>> +             }
> >>
> >> I'm not sure we ever got a definitive answer to this - does DPAA2
> >> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> >> make the fewest possible assumptions, since at this point it's basically
> >> just a stop-gap until we can fix the spec. It's become clear that we
> >> can't reliably rely on guessing attributes, so I'm not too fussed about
> >> theoretical cases that currently don't work (due to complete lack of RMR
> >> support) continuing to not work for the moment, as long as we can make
> >> the real-world cases we actually have work at all. Anything which only
> >> affects performance I'd rather leave until firmware can tell us what to do.
> >
> > Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
> > mappings.  DPAA2 is just one connected device.
>
> Apologies if I'm being overly loose with terminology there - my point of
> reference for this hardware is documentation for the old LS2080A, where
> the "DPAA2 Reference Manual" gives a strong impression that the MC is a
> component belonging to the overall DPAA2 architecture. Either way it
> technically stands to reason that the other DPAA2 components would only
> be usable if the MC itself works (unless I've been holding a major
> misconception about that for years as well).
>
> In the context of this discussion, please consider any reference I may
> make to bits of NXP's hardware to be shorthand for "the thing for which
> NXP have a vested interest in IORT RMRs".

Ultimately the spec doesn't mention what IOMMU properties the regions
should have.  Even marking them as IOMMU_READ/WRITE is as much
of an assumption as using IOMMU_MMIO or IOMMU_CACHE. It just seems
IOMMU_MMIO is the most popular since all the examples use it for MSI
doorbells in the documentation.

I am interested why this concern is only being brought up at this point
in a patchset that has been on the mailing list for 8+ months?  This is
based on a spec that has existed from Arm since 2020 with the most recent
revisions published in Feb 2021.  The lack of RMR support in the kernel
is affecting real world products, and the ability for SystemReady ES
certified systems from just fully working with recent distributions.  Even
worse, is that without this patchset customers are forced to jump through
hoops to purposefully re-enable smmu bypass making their systems less
secure.

How is this a good experience for customers of SystemReady hardware
when for any mainline distribution to work the first thing they have to do is
make their system less secure?

-Jon

>
> Thanks,
> Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-10-12  8:00           ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-10-12  8:00 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Steven Price, Hanjun Guo, yangyicong,
	Sami Mujawar, Will Deacon, wanghuiqiang

On Mon, Oct 11, 2021 at 4:04 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-10-09 08:06, Jon Nettleton wrote:
> [...]
> >>> +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> >>> +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> >>> +                     /*
> >>> +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> >>> +                      * normally used for allocated system memory that is
> >>> +                      * then used for device specific reserved regions.
> >>> +                      */
> >>> +                     prot |= IOMMU_CACHE;
> >>> +             } else {
> >>> +                     type = IOMMU_RESV_DIRECT;
> >>> +                     /*
> >>> +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> >>> +                      * for device memory like MSI doorbell.
> >>> +                      */
> >>> +                     prot |= IOMMU_MMIO;
> >>> +             }
> >>
> >> I'm not sure we ever got a definitive answer to this - does DPAA2
> >> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> >> make the fewest possible assumptions, since at this point it's basically
> >> just a stop-gap until we can fix the spec. It's become clear that we
> >> can't reliably rely on guessing attributes, so I'm not too fussed about
> >> theoretical cases that currently don't work (due to complete lack of RMR
> >> support) continuing to not work for the moment, as long as we can make
> >> the real-world cases we actually have work at all. Anything which only
> >> affects performance I'd rather leave until firmware can tell us what to do.
> >
> > Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
> > mappings.  DPAA2 is just one connected device.
>
> Apologies if I'm being overly loose with terminology there - my point of
> reference for this hardware is documentation for the old LS2080A, where
> the "DPAA2 Reference Manual" gives a strong impression that the MC is a
> component belonging to the overall DPAA2 architecture. Either way it
> technically stands to reason that the other DPAA2 components would only
> be usable if the MC itself works (unless I've been holding a major
> misconception about that for years as well).
>
> In the context of this discussion, please consider any reference I may
> make to bits of NXP's hardware to be shorthand for "the thing for which
> NXP have a vested interest in IORT RMRs".

Ultimately the spec doesn't mention what IOMMU properties the regions
should have.  Even marking them as IOMMU_READ/WRITE is as much
of an assumption as using IOMMU_MMIO or IOMMU_CACHE. It just seems
IOMMU_MMIO is the most popular since all the examples use it for MSI
doorbells in the documentation.

I am interested why this concern is only being brought up at this point
in a patchset that has been on the mailing list for 8+ months?  This is
based on a spec that has existed from Arm since 2020 with the most recent
revisions published in Feb 2021.  The lack of RMR support in the kernel
is affecting real world products, and the ability for SystemReady ES
certified systems from just fully working with recent distributions.  Even
worse, is that without this patchset customers are forced to jump through
hoops to purposefully re-enable smmu bypass making their systems less
secure.

How is this a good experience for customers of SystemReady hardware
when for any mainline distribution to work the first thing they have to do is
make their system less secure?

-Jon

>
> Thanks,
> Robin.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-10-12  8:00           ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-10-12  8:00 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Steven Price, Hanjun Guo, yangyicong,
	Sami Mujawar, Will Deacon, wanghuiqiang

On Mon, Oct 11, 2021 at 4:04 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> On 2021-10-09 08:06, Jon Nettleton wrote:
> [...]
> >>> +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> >>> +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> >>> +                     /*
> >>> +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> >>> +                      * normally used for allocated system memory that is
> >>> +                      * then used for device specific reserved regions.
> >>> +                      */
> >>> +                     prot |= IOMMU_CACHE;
> >>> +             } else {
> >>> +                     type = IOMMU_RESV_DIRECT;
> >>> +                     /*
> >>> +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> >>> +                      * for device memory like MSI doorbell.
> >>> +                      */
> >>> +                     prot |= IOMMU_MMIO;
> >>> +             }
> >>
> >> I'm not sure we ever got a definitive answer to this - does DPAA2
> >> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> >> make the fewest possible assumptions, since at this point it's basically
> >> just a stop-gap until we can fix the spec. It's become clear that we
> >> can't reliably rely on guessing attributes, so I'm not too fussed about
> >> theoretical cases that currently don't work (due to complete lack of RMR
> >> support) continuing to not work for the moment, as long as we can make
> >> the real-world cases we actually have work at all. Anything which only
> >> affects performance I'd rather leave until firmware can tell us what to do.
> >
> > Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
> > mappings.  DPAA2 is just one connected device.
>
> Apologies if I'm being overly loose with terminology there - my point of
> reference for this hardware is documentation for the old LS2080A, where
> the "DPAA2 Reference Manual" gives a strong impression that the MC is a
> component belonging to the overall DPAA2 architecture. Either way it
> technically stands to reason that the other DPAA2 components would only
> be usable if the MC itself works (unless I've been holding a major
> misconception about that for years as well).
>
> In the context of this discussion, please consider any reference I may
> make to bits of NXP's hardware to be shorthand for "the thing for which
> NXP have a vested interest in IORT RMRs".

Ultimately the spec doesn't mention what IOMMU properties the regions
should have.  Even marking them as IOMMU_READ/WRITE is as much
of an assumption as using IOMMU_MMIO or IOMMU_CACHE. It just seems
IOMMU_MMIO is the most popular since all the examples use it for MSI
doorbells in the documentation.

I am interested why this concern is only being brought up at this point
in a patchset that has been on the mailing list for 8+ months?  This is
based on a spec that has existed from Arm since 2020 with the most recent
revisions published in Feb 2021.  The lack of RMR support in the kernel
is affecting real world products, and the ability for SystemReady ES
certified systems from just fully working with recent distributions.  Even
worse, is that without this patchset customers are forced to jump through
hoops to purposefully re-enable smmu bypass making their systems less
secure.

How is this a good experience for customers of SystemReady hardware
when for any mainline distribution to work the first thing they have to do is
make their system less secure?

-Jon

>
> Thanks,
> Robin.

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-10-12  8:00           ` Jon Nettleton
  (?)
@ 2021-12-08 12:18             ` Lorenzo Pieralisi
  -1 siblings, 0 replies; 150+ messages in thread
From: Lorenzo Pieralisi @ 2021-12-08 12:18 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Robin Murphy, Shameer Kolothum, linux-arm-kernel,
	ACPI Devel Maling List, Linux IOMMU, Linuxarm, Steven Price,
	Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon, wanghuiqiang

On Tue, Oct 12, 2021 at 10:00:24AM +0200, Jon Nettleton wrote:
> On Mon, Oct 11, 2021 at 4:04 PM Robin Murphy <robin.murphy@arm.com> wrote:
> >
> > On 2021-10-09 08:06, Jon Nettleton wrote:
> > [...]
> > >>> +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > >>> +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> > >>> +                     /*
> > >>> +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > >>> +                      * normally used for allocated system memory that is
> > >>> +                      * then used for device specific reserved regions.
> > >>> +                      */
> > >>> +                     prot |= IOMMU_CACHE;
> > >>> +             } else {
> > >>> +                     type = IOMMU_RESV_DIRECT;
> > >>> +                     /*
> > >>> +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> > >>> +                      * for device memory like MSI doorbell.
> > >>> +                      */
> > >>> +                     prot |= IOMMU_MMIO;
> > >>> +             }
> > >>
> > >> I'm not sure we ever got a definitive answer to this - does DPAA2
> > >> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> > >> make the fewest possible assumptions, since at this point it's basically
> > >> just a stop-gap until we can fix the spec. It's become clear that we
> > >> can't reliably rely on guessing attributes, so I'm not too fussed about
> > >> theoretical cases that currently don't work (due to complete lack of RMR
> > >> support) continuing to not work for the moment, as long as we can make
> > >> the real-world cases we actually have work at all. Anything which only
> > >> affects performance I'd rather leave until firmware can tell us what to do.
> > >
> > > Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
> > > mappings.  DPAA2 is just one connected device.
> >
> > Apologies if I'm being overly loose with terminology there - my point of
> > reference for this hardware is documentation for the old LS2080A, where
> > the "DPAA2 Reference Manual" gives a strong impression that the MC is a
> > component belonging to the overall DPAA2 architecture. Either way it
> > technically stands to reason that the other DPAA2 components would only
> > be usable if the MC itself works (unless I've been holding a major
> > misconception about that for years as well).
> >
> > In the context of this discussion, please consider any reference I may
> > make to bits of NXP's hardware to be shorthand for "the thing for which
> > NXP have a vested interest in IORT RMRs".
> 
> Ultimately the spec doesn't mention what IOMMU properties the regions
> should have.

It will have to and that's what we are working on.

> Even marking them as IOMMU_READ/WRITE is as much of an assumption as
> using IOMMU_MMIO or IOMMU_CACHE. It just seems IOMMU_MMIO is the most
> popular since all the examples use it for MSI doorbells in the
> documentation.

We don't merge code based on assumptions that can easily break because
the specifications don't contemplate the details that are required.

> I am interested why this concern is only being brought up at this point
> on a patchset that has been on the mailing list for 8+ months? 

See above. We don't merge code that we know can break and is based on
assumptions, we need to update the IORT specifications to make them
cover all the use cases - in a predictable way - and that's what we are
working on.

> This is based on a spec that has existed from Arm since 2020 with the
> most recent revisions published in Feb 2021.  The lack of RMR support
> in the kernel is affecting real world products, and the ability for
> SystemReady ES certified systems from just fully working with recent
> distributions.

I answered above - if you have any questions please ask them, here,
as far as Linux code is concerned.

I understand this is taking a long time, it is also helping us
understand all the possible use cases and how to cover them in
a way that is maintainable in the long run.

Thanks,
Lorenzo

> Even worse, is that without this patchset customers are forced to jump
> through hoops to purposefully re-enable smmu bypass making their
> systems less secure.
> 
> How is this a good experience for customers of SystemReady hardware
> when for any mainline distribution to work the first thing they have
> to do is make their system less secure?
> 
> -Jon
> 
> >
> > Thanks,
> > Robin.
> 

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-12-08 12:18             ` Lorenzo Pieralisi
  0 siblings, 0 replies; 150+ messages in thread
From: Lorenzo Pieralisi @ 2021-12-08 12:18 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Will Deacon, Linuxarm, Steven Price, ACPI Devel Maling List,
	Linux IOMMU, wanghuiqiang, Hanjun Guo, yangyicong, Sami Mujawar,
	Robin Murphy, linux-arm-kernel

On Tue, Oct 12, 2021 at 10:00:24AM +0200, Jon Nettleton wrote:
> On Mon, Oct 11, 2021 at 4:04 PM Robin Murphy <robin.murphy@arm.com> wrote:
> >
> > On 2021-10-09 08:06, Jon Nettleton wrote:
> > [...]
> > >>> +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > >>> +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> > >>> +                     /*
> > >>> +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > >>> +                      * normally used for allocated system memory that is
> > >>> +                      * then used for device specific reserved regions.
> > >>> +                      */
> > >>> +                     prot |= IOMMU_CACHE;
> > >>> +             } else {
> > >>> +                     type = IOMMU_RESV_DIRECT;
> > >>> +                     /*
> > >>> +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> > >>> +                      * for device memory like MSI doorbell.
> > >>> +                      */
> > >>> +                     prot |= IOMMU_MMIO;
> > >>> +             }
> > >>
> > >> I'm not sure we ever got a definitive answer to this - does DPAA2
> > >> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> > >> make the fewest possible assumptions, since at this point it's basically
> > >> just a stop-gap until we can fix the spec. It's become clear that we
> > >> can't reliably rely on guessing attributes, so I'm not too fussed about
> > >> theoretical cases that currently don't work (due to complete lack of RMR
> > >> support) continuing to not work for the moment, as long as we can make
> > >> the real-world cases we actually have work at all. Anything which only
> > >> affects performance I'd rather leave until firmware can tell us what to do.
> > >
> > > Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
> > > mappings.  DPAA2 is just one connected device.
> >
> > Apologies if I'm being overly loose with terminology there - my point of
> > reference for this hardware is documentation for the old LS2080A, where
> > the "DPAA2 Reference Manual" gives a strong impression that the MC is a
> > component belonging to the overall DPAA2 architecture. Either way it
> > technically stands to reason that the other DPAA2 components would only
> > be usable if the MC itself works (unless I've been holding a major
> > misconception about that for years as well).
> >
> > In the context of this discussion, please consider any reference I may
> > make to bits of NXP's hardware to be shorthand for "the thing for which
> > NXP have a vested interest in IORT RMRs".
> 
> Ultimately the spec doesn't mention what IOMMU properties the regions
> should have.

It will have to and that's what we are working on.

> Even marking them as IOMMU_READ/WRITE is as much of an assumption as
> using IOMMU_MMIO or IOMMU_CACHE. It just seems IOMMU_MMIO is the most
> popular since all the examples use it for MSI doorbells in the
> documentation.

We don't merge code based on assumptions that can easily break because
the specifications don't contemplate the details that are required.

> I am interested why this concern is only being brought up at this point
> on a patchset that has been on the mailing list for 8+ months? 

See above. We don't merge code that we know can break and is based on
assumptions, we need to update the IORT specifications to make them
cover all the use cases - in a predictable way - and that's what we are
working on.

> This is based on a spec that has existed from Arm since 2020 with the
> most recent revisions published in Feb 2021.  The lack of RMR support
> in the kernel is affecting real world products, and the ability for
> SystemReady ES certified systems from just fully working with recent
> distributions.

I answered above - if you have any questions please ask them, here,
as far as Linux code is concerned.

I understand this is taking a long time, it is also helping us
understand all the possible use cases and how to cover them in
a way that is maintainable in the long run.

Thanks,
Lorenzo

> Even worse, is that without this patchset customers are forced to jump
> through hoops to purposefully re-enable smmu bypass making their
> systems less secure.
> 
> How is this a good experience for customers of SystemReady hardware
> when for any mainline distribution to work the first thing they have
> to do is make their system less secure?
> 
> -Jon
> 
> >
> > Thanks,
> > Robin.
> 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-12-08 12:18             ` Lorenzo Pieralisi
  0 siblings, 0 replies; 150+ messages in thread
From: Lorenzo Pieralisi @ 2021-12-08 12:18 UTC (permalink / raw)
  To: Jon Nettleton
  Cc: Robin Murphy, Shameer Kolothum, linux-arm-kernel,
	ACPI Devel Maling List, Linux IOMMU, Linuxarm, Steven Price,
	Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon, wanghuiqiang

On Tue, Oct 12, 2021 at 10:00:24AM +0200, Jon Nettleton wrote:
> On Mon, Oct 11, 2021 at 4:04 PM Robin Murphy <robin.murphy@arm.com> wrote:
> >
> > On 2021-10-09 08:06, Jon Nettleton wrote:
> > [...]
> > >>> +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > >>> +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> > >>> +                     /*
> > >>> +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > >>> +                      * normally used for allocated system memory that is
> > >>> +                      * then used for device specific reserved regions.
> > >>> +                      */
> > >>> +                     prot |= IOMMU_CACHE;
> > >>> +             } else {
> > >>> +                     type = IOMMU_RESV_DIRECT;
> > >>> +                     /*
> > >>> +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> > >>> +                      * for device memory like MSI doorbell.
> > >>> +                      */
> > >>> +                     prot |= IOMMU_MMIO;
> > >>> +             }
> > >>
> > >> I'm not sure we ever got a definitive answer to this - does DPAA2
> > >> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> > >> make the fewest possible assumptions, since at this point it's basically
> > >> just a stop-gap until we can fix the spec. It's become clear that we
> > >> can't reliably rely on guessing attributes, so I'm not too fussed about
> > >> theoretical cases that currently don't work (due to complete lack of RMR
> > >> support) continuing to not work for the moment, as long as we can make
> > >> the real-world cases we actually have work at all. Anything which only
> > >> affects performance I'd rather leave until firmware can tell us what to do.
> > >
> > > Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
> > > mappings.  DPAA2 is just one connected device.
> >
> > Apologies if I'm being overly loose with terminology there - my point of
> > reference for this hardware is documentation for the old LS2080A, where
> > the "DPAA2 Reference Manual" gives a strong impression that the MC is a
> > component belonging to the overall DPAA2 architecture. Either way it
> > technically stands to reason that the other DPAA2 components would only
> > be usable if the MC itself works (unless I've been holding a major
> > misconception about that for years as well).
> >
> > In the context of this discussion, please consider any reference I may
> > make to bits of NXP's hardware to be shorthand for "the thing for which
> > NXP have a vested interest in IORT RMRs".
> 
> Ultimately the spec doesn't mention what IOMMU properties the regions
> should have.

It will have to and that's what we are working on.

> Even marking them as IOMMU_READ/WRITE is as much of an assumption as
> using IOMMU_MMIO or IOMMU_CACHE. It just seems IOMMU_MMIO is the most
> popular since all the examples use it for MSI doorbells in the
> documentation.

We don't merge code based on assumptions that can easily break because
the specifications don't contemplate the details that are required.

> I am interested why this concern is only being brought up at this point
> on a patchset that has been on the mailing list for 8+ months? 

See above. We don't merge code that we know can break and is based on
assumptions, we need to update the IORT specifications to make them
cover all the use cases - in a predictable way - and that's what we are
working on.

> This is based on a spec that has existed from Arm since 2020 with the
> most recent revisions published in Feb 2021.  The lack of RMR support
> in the kernel is affecting real world products, and the ability for
> SystemReady ES certified systems from just fully working with recent
> distributions.

I answered above - if you have any questions please ask them, here,
as far as Linux code is concerned.

I understand this is taking a long time, it is also helping us
understand all the possible use cases and how to cover them in
a way that is maintainable in the long run.

Thanks,
Lorenzo

> Even worse, is that without this patchset customers are forced to jump
> through hoops to purposefully re-enable smmu bypass making their
> systems less secure.
> 
> How is this a good experience for customers of SystemReady hardware
> when for any mainline distribution to work the first thing they have
> to do is make their system less secure?
> 
> -Jon
> 
> >
> > Thanks,
> > Robin.
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-12-08 12:18             ` Lorenzo Pieralisi
  (?)
@ 2021-12-08 13:26               ` Jon Nettleton
  -1 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-12-08 13:26 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Robin Murphy, Shameer Kolothum, linux-arm-kernel,
	ACPI Devel Maling List, Linux IOMMU, Linuxarm, Steven Price,
	Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon, wanghuiqiang

On Wed, Dec 8, 2021 at 1:19 PM Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Tue, Oct 12, 2021 at 10:00:24AM +0200, Jon Nettleton wrote:
> > On Mon, Oct 11, 2021 at 4:04 PM Robin Murphy <robin.murphy@arm.com> wrote:
> > >
> > > On 2021-10-09 08:06, Jon Nettleton wrote:
> > > [...]
> > > >>> +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > > >>> +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> > > >>> +                     /*
> > > >>> +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > > >>> +                      * normally used for allocated system memory that is
> > > >>> +                      * then used for device specific reserved regions.
> > > >>> +                      */
> > > >>> +                     prot |= IOMMU_CACHE;
> > > >>> +             } else {
> > > >>> +                     type = IOMMU_RESV_DIRECT;
> > > >>> +                     /*
> > > >>> +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> > > >>> +                      * for device memory like MSI doorbell.
> > > >>> +                      */
> > > >>> +                     prot |= IOMMU_MMIO;
> > > >>> +             }
> > > >>
> > > >> I'm not sure we ever got a definitive answer to this - does DPAA2
> > > >> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> > > >> make the fewest possible assumptions, since at this point it's basically
> > > >> just a stop-gap until we can fix the spec. It's become clear that we
> > > >> can't reliably rely on guessing attributes, so I'm not too fussed about
> > > >> theoretical cases that currently don't work (due to complete lack of RMR
> > > >> support) continuing to not work for the moment, as long as we can make
> > > >> the real-world cases we actually have work at all. Anything which only
> > > >> affects performance I'd rather leave until firmware can tell us what to do.
> > > >
> > > > Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
> > > > mappings.  DPAA2 is just one connected device.
> > >
> > > Apologies if I'm being overly loose with terminology there - my point of
> > > reference for this hardware is documentation for the old LS2080A, where
> > > the "DPAA2 Reference Manual" gives a strong impression that the MC is a
> > > component belonging to the overall DPAA2 architecture. Either way it
> > > technically stands to reason that the other DPAA2 components would only
> > > be usable if the MC itself works (unless I've been holding a major
> > > misconception about that for years as well).
> > >
> > > In the context of this discussion, please consider any reference I may
> > > make to bits of NXP's hardware to be shorthand for "the thing for which
> > > NXP have a vested interest in IORT RMRs".
> >
> > Ultimately the spec doesn't mention what IOMMU properties the regions
> > should have.
>
> It will have to and that's what we are working on.

Where is this being worked on?  I see no open tickets for this.

>
> > Even marking them as IOMMU_READ/WRITE is as much of an assumption as
> > using IOMMU_MMIO or IOMMU_CACHE. It just seems IOMMU_MMIO is the most
> > popular since all the examples use it for MSI doorbells in the
> > documentation.
>
> We don't merge code based on assumptions that can easily break because
> the specifications don't contemplate the details that are required.
>
> > I am interested why this concern is only being brought up at this point
> > on a patchset that has been on the mailing list for 8+ months?
>
> See above. We don't merge code that we know can break and is based on
> assumptions, we need to update the IORT specifications to make them
> cover all the use cases - in a predictable way - and that's what we are
> working on.

This is not really an answer to the question.  The latest version of the
IORT RMR spec was published in Feb 2021. Why was this issue not
brought up with Rev 1 of this patchset? Instead you have wasted
10 months of developer and customer time. This could have easily been
turned into a code first spec change request, which is a valid option
for ACPI changes.

>
> > This is based on a spec that has existed from Arm since 2020 with the
> > most recent revisions published in Feb 2021.  The lack of RMR support
> > in the kernel is affecting real world products, and the ability for
> > SystemReady ES certified systems from just fully working with recent
> > distributions.
>
> I answered above - if you have any questions please ask them, here,
> as far as Linux code is concerned.
>
> I understand this is taking a long time, it is also helping us
> understand all the possible use cases and how to cover them in
> a way that is maintainable in the long run.

Every month that this patchset has sat being unattended by the
maintainers is another kernel dev cycle missed, it is another
another distribution release where users need to add hackish
kernel command-line options to disable security features that
were forced on by default. Not to mention Linux is just one
platform. What if other platforms have already adopted the
existing spec? These are Arm specs and Arm maintainers and
yet nobody seems to agree on anything and absolutely nothing
has been achieved except wasting the time of Shameer, myself,
our companies, and our customers.

-Jon

>
> Thanks,
> Lorenzo
>
> > Even worse, is that without this patchset customers are forced to jump
> > through hoops to purposefully re-enable smmu bypass making their
> > systems less secure.
> >
> > How is this a good experience for customers of SystemReady hardware
> > when for any mainline distribution to work the first thing they have
> > to do is make their system less secure?
> >
> > -Jon
> >
> > >
> > > Thanks,
> > > Robin.
> >

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-12-08 13:26               ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-12-08 13:26 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Will Deacon, Linuxarm, Steven Price, ACPI Devel Maling List,
	Linux IOMMU, wanghuiqiang, Hanjun Guo, yangyicong, Sami Mujawar,
	Robin Murphy, linux-arm-kernel

On Wed, Dec 8, 2021 at 1:19 PM Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Tue, Oct 12, 2021 at 10:00:24AM +0200, Jon Nettleton wrote:
> > On Mon, Oct 11, 2021 at 4:04 PM Robin Murphy <robin.murphy@arm.com> wrote:
> > >
> > > On 2021-10-09 08:06, Jon Nettleton wrote:
> > > [...]
> > > >>> +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > > >>> +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> > > >>> +                     /*
> > > >>> +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > > >>> +                      * normally used for allocated system memory that is
> > > >>> +                      * then used for device specific reserved regions.
> > > >>> +                      */
> > > >>> +                     prot |= IOMMU_CACHE;
> > > >>> +             } else {
> > > >>> +                     type = IOMMU_RESV_DIRECT;
> > > >>> +                     /*
> > > >>> +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> > > >>> +                      * for device memory like MSI doorbell.
> > > >>> +                      */
> > > >>> +                     prot |= IOMMU_MMIO;
> > > >>> +             }
> > > >>
> > > >> I'm not sure we ever got a definitive answer to this - does DPAA2
> > > >> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> > > >> make the fewest possible assumptions, since at this point it's basically
> > > >> just a stop-gap until we can fix the spec. It's become clear that we
> > > >> can't reliably rely on guessing attributes, so I'm not too fussed about
> > > >> theoretical cases that currently don't work (due to complete lack of RMR
> > > >> support) continuing to not work for the moment, as long as we can make
> > > >> the real-world cases we actually have work at all. Anything which only
> > > >> affects performance I'd rather leave until firmware can tell us what to do.
> > > >
> > > > Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
> > > > mappings.  DPAA2 is just one connected device.
> > >
> > > Apologies if I'm being overly loose with terminology there - my point of
> > > reference for this hardware is documentation for the old LS2080A, where
> > > the "DPAA2 Reference Manual" gives a strong impression that the MC is a
> > > component belonging to the overall DPAA2 architecture. Either way it
> > > technically stands to reason that the other DPAA2 components would only
> > > be usable if the MC itself works (unless I've been holding a major
> > > misconception about that for years as well).
> > >
> > > In the context of this discussion, please consider any reference I may
> > > make to bits of NXP's hardware to be shorthand for "the thing for which
> > > NXP have a vested interest in IORT RMRs".
> >
> > Ultimately the spec doesn't mention what IOMMU properties the regions
> > should have.
>
> It will have to and that's what we are working on.

Where is this being worked on?  I see no open tickets for this.

>
> > Even marking them as IOMMU_READ/WRITE is as much of an assumption as
> > using IOMMU_MMIO or IOMMU_CACHE. It just seems IOMMU_MMIO is the most
> > popular since all the examples use it for MSI doorbells in the
> > documentation.
>
> We don't merge code based on assumptions that can easily break because
> the specifications don't contemplate the details that are required.
>
> > I am interested why this concern is only being brought up at this point
> > on a patchset that has been on the mailing list for 8+ months?
>
> See above. We don't merge code that we know can break and is based on
> assumptions, we need to update the IORT specifications to make them
> cover all the use cases - in a predictable way - and that's what we are
> working on.

This is not really an answer to the question.  The latest version of the
IORT RMR spec was published in Feb 2021. Why was this issue not
brought up with Rev 1 of this patchset? Instead you have wasted
10 months of developer and customer time. This could have easily been
turned into a code first spec change request, which is a valid option
for ACPI changes.

>
> > This is based on a spec that has existed from Arm since 2020 with the
> > most recent revisions published in Feb 2021.  The lack of RMR support
> > in the kernel is affecting real world products, and the ability for
> > SystemReady ES certified systems from just fully working with recent
> > distributions.
>
> I answered above - if you have any questions please ask them, here,
> as far as Linux code is concerned.
>
> I understand this is taking a long time, it is also helping us
> understand all the possible use cases and how to cover them in
> a way that is maintainable in the long run.

Every month that this patchset has sat being unattended by the
maintainers is another kernel dev cycle missed, it is another
another distribution release where users need to add hackish
kernel command-line options to disable security features that
were forced on by default. Not to mention Linux is just one
platform. What if other platforms have already adopted the
existing spec? These are Arm specs and Arm maintainers and
yet nobody seems to agree on anything and absolutely nothing
has been achieved except wasting the time of Shameer, myself,
our companies, and our customers.

-Jon

>
> Thanks,
> Lorenzo
>
> > Even worse, is that without this patchset customers are forced to jump
> > through hoops to purposefully re-enable smmu bypass making their
> > systems less secure.
> >
> > How is this a good experience for customers of SystemReady hardware
> > when for any mainline distribution to work the first thing they have
> > to do is make their system less secure?
> >
> > -Jon
> >
> > >
> > > Thanks,
> > > Robin.
> >
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-12-08 13:26               ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-12-08 13:26 UTC (permalink / raw)
  To: Lorenzo Pieralisi
  Cc: Robin Murphy, Shameer Kolothum, linux-arm-kernel,
	ACPI Devel Maling List, Linux IOMMU, Linuxarm, Steven Price,
	Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon, wanghuiqiang

On Wed, Dec 8, 2021 at 1:19 PM Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Tue, Oct 12, 2021 at 10:00:24AM +0200, Jon Nettleton wrote:
> > On Mon, Oct 11, 2021 at 4:04 PM Robin Murphy <robin.murphy@arm.com> wrote:
> > >
> > > On 2021-10-09 08:06, Jon Nettleton wrote:
> > > [...]
> > > >>> +             if (rmr->flags & IOMMU_RMR_REMAP_PERMITTED) {
> > > >>> +                     type = IOMMU_RESV_DIRECT_RELAXABLE;
> > > >>> +                     /*
> > > >>> +                      * Set IOMMU_CACHE as IOMMU_RESV_DIRECT_RELAXABLE is
> > > >>> +                      * normally used for allocated system memory that is
> > > >>> +                      * then used for device specific reserved regions.
> > > >>> +                      */
> > > >>> +                     prot |= IOMMU_CACHE;
> > > >>> +             } else {
> > > >>> +                     type = IOMMU_RESV_DIRECT;
> > > >>> +                     /*
> > > >>> +                      * Set IOMMU_MMIO as IOMMU_RESV_DIRECT is normally used
> > > >>> +                      * for device memory like MSI doorbell.
> > > >>> +                      */
> > > >>> +                     prot |= IOMMU_MMIO;
> > > >>> +             }
> > > >>
> > > >> I'm not sure we ever got a definitive answer to this - does DPAA2
> > > >> actually go wrong if we use IOMMU_MMIO here? I'd still much prefer to
> > > >> make the fewest possible assumptions, since at this point it's basically
> > > >> just a stop-gap until we can fix the spec. It's become clear that we
> > > >> can't reliably rely on guessing attributes, so I'm not too fussed about
> > > >> theoretical cases that currently don't work (due to complete lack of RMR
> > > >> support) continuing to not work for the moment, as long as we can make
> > > >> the real-world cases we actually have work at all. Anything which only
> > > >> affects performance I'd rather leave until firmware can tell us what to do.
> > > >
> > > > Well it isn't DPAA2, it is FSL_MC_BUS that fails with IOMMU_MMIO
> > > > mappings.  DPAA2 is just one connected device.
> > >
> > > Apologies if I'm being overly loose with terminology there - my point of
> > > reference for this hardware is documentation for the old LS2080A, where
> > > the "DPAA2 Reference Manual" gives a strong impression that the MC is a
> > > component belonging to the overall DPAA2 architecture. Either way it
> > > technically stands to reason that the other DPAA2 components would only
> > > be usable if the MC itself works (unless I've been holding a major
> > > misconception about that for years as well).
> > >
> > > In the context of this discussion, please consider any reference I may
> > > make to bits of NXP's hardware to be shorthand for "the thing for which
> > > NXP have a vested interest in IORT RMRs".
> >
> > Ultimately the spec doesn't mention what IOMMU properties the regions
> > should have.
>
> It will have to and that's what we are working on.

Where is this being worked on?  I see no open tickets for this.

>
> > Even marking them as IOMMU_READ/WRITE is as much of an assumption as
> > using IOMMU_MMIO or IOMMU_CACHE. It just seems IOMMU_MMIO is the most
> > popular since all the examples use it for MSI doorbells in the
> > documentation.
>
> We don't merge code based on assumptions that can easily break because
> the specifications don't contemplate the details that are required.
>
> > I am interested why this concern is only being brought up at this point
> > on a patchset that has been on the mailing list for 8+ months?
>
> See above. We don't merge code that we know can break and is based on
> assumptions, we need to update the IORT specifications to make them
> cover all the use cases - in a predictable way - and that's what we are
> working on.

This is not really an answer to the question.  The latest version of the
IORT RMR spec was published in Feb 2021. Why was this issue not
brought up with Rev 1 of this patchset? Instead you have wasted
10 months of developer and customer time. This could have easily been
turned into a code first spec change request, which is a valid option
for ACPI changes.

>
> > This is based on a spec that has existed from Arm since 2020 with the
> > most recent revisions published in Feb 2021.  The lack of RMR support
> > in the kernel is affecting real world products, and the ability for
> > SystemReady ES certified systems from just fully working with recent
> > distributions.
>
> I answered above - if you have any questions please ask them, here,
> as far as Linux code is concerned.
>
> I understand this is taking a long time, it is also helping us
> understand all the possible use cases and how to cover them in
> a way that is maintainable in the long run.

Every month that this patchset has sat being unattended by the
maintainers is another kernel dev cycle missed, it is another
another distribution release where users need to add hackish
kernel command-line options to disable security features that
were forced on by default. Not to mention Linux is just one
platform. What if other platforms have already adopted the
existing spec? These are Arm specs and Arm maintainers and
yet nobody seems to agree on anything and absolutely nothing
has been achieved except wasting the time of Shameer, myself,
our companies, and our customers.

-Jon

>
> Thanks,
> Lorenzo
>
> > Even worse, is that without this patchset customers are forced to jump
> > through hoops to purposefully re-enable smmu bypass making their
> > systems less secure.
> >
> > How is this a good experience for customers of SystemReady hardware
> > when for any mainline distribution to work the first thing they have
> > to do is make their system less secure?
> >
> > -Jon
> >
> > >
> > > Thanks,
> > > Robin.
> >

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-12-08 13:26               ` Jon Nettleton
  (?)
@ 2021-12-08 14:37                 ` Robin Murphy
  -1 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-12-08 14:37 UTC (permalink / raw)
  To: Jon Nettleton, Lorenzo Pieralisi
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Steven Price, Hanjun Guo, yangyicong,
	Sami Mujawar, Will Deacon, wanghuiqiang

Jon,

On 2021-12-08 13:26, Jon Nettleton wrote:
[...]
>>> Even marking them as IOMMU_READ/WRITE is as much of an assumption as
>>> using IOMMU_MMIO or IOMMU_CACHE. It just seems IOMMU_MMIO is the most
>>> popular since all the examples use it for MSI doorbells in the
>>> documentation.
>>
>> We don't merge code based on assumptions that can easily break because
>> the specifications don't contemplate the details that are required.
>>
>>> I am interested why this concern is only being brought up at this point
>>> on a patchset that has been on the mailing list for 8+ months?
>>
>> See above. We don't merge code that we know can break and is based on
>> assumptions, we need to update the IORT specifications to make them
>> cover all the use cases - in a predictable way - and that's what we are
>> working on.
> 
> This is not really an answer to the question.  The latest version of the
> IORT RMR spec was published in Feb 2021. Why was this issue not
> brought up with Rev 1 of this patchset? Instead you have wasted
> 10 months of developer and customer time. This could have easily been
> turned into a code first spec change request, which is a valid option
> for ACPI changes.

It was only on v5 of the patchset - *six months* after the original RFC 
posting - that anyone even first started to question the initial 
assumptions made about attributes[1], and even then somebody familiar 
countered that it didn't appear to matter[2]. Sorry, but you don't get 
to U-turn and throw unjust shade at Arm for not being prescient.

Yes, when those of us within Arm set out the initial RMR spec, an 
assumption was made that it seemed reasonable for an OS to simply pick 
some default strong memory type (Device or Normal-NC) and full 
permissions if it did need to map RMRs at stage 1. That spec was 
reviewed and published externally and no interested parties came forth 
asking "hey, what about attributes?". Linux patches were written around 
that assumption and proceeded through many rounds of review until we 
eventually received sufficient feedback to demonstrate that the 
assumption did not in fact hold well enough in general and there seemed 
to be a genuine need for RMR attributes, and at that point we started 
work on revising the spec.

In the meantime, these patches have sat at v7 for four months - the 
*other* outstanding review comments have not been addressed; I still 
don't recall seeing an answer about whether LX2160 or anything else 
currently deployed actually *needs* cacheable mappings or whether it 
could muddle through with the IOMMU_MMIO assumption until proper "RMR 
v2" support arrived later; even if so, an interim workaround specific to 
LX2160 could have been proposed but hasn't. It is hardly reasonable to 
pretend that Arm or the upstream maintainers are responsible for a lack 
of development activity on the part of the submitters, no matter how 
much blatant misinformation is repeated on Twitter.

Regards,
Robin.

[1] 
https://lore.kernel.org/linux-iommu/13c2499e-cc0c-d395-0d60-6c3437f206ac@nxp.com/
[2] 
https://lore.kernel.org/linux-iommu/CABdtJHv2QBHNoWTyp51H-J_apc75imPj0FbrV70Tm8xuNjpiTA@mail.gmail.com/

>>
>>> This is based on a spec that has existed from Arm since 2020 with the
>>> most recent revisions published in Feb 2021.  The lack of RMR support
>>> in the kernel is affecting real world products, and the ability for
>>> SystemReady ES certified systems from just fully working with recent
>>> distributions.
>>
>> I answered above - if you have any questions please ask them, here,
>> as far as Linux code is concerned.
>>
>> I understand this is taking a long time, it is also helping us
>> understand all the possible use cases and how to cover them in
>> a way that is maintainable in the long run.
> 
> Every month that this patchset has sat being unattended by the
> maintainers is another kernel dev cycle missed, it is another
> another distribution release where users need to add hackish
> kernel command-line options to disable security features that
> were forced on by default. Not to mention Linux is just one
> platform. What if other platforms have already adopted the
> existing spec? These are Arm specs and Arm maintainers and
> yet nobody seems to agree on anything and absolutely nothing
> has been achieved except wasting the time of Shameer, myself,
> our companies, and our customers.
> 
> -Jon
> 
>>
>> Thanks,
>> Lorenzo
>>
>>> Even worse, is that without this patchset customers are forced to jump
>>> through hoops to purposefully re-enable smmu bypass making their
>>> systems less secure.
>>>
>>> How is this a good experience for customers of SystemReady hardware
>>> when for any mainline distribution to work the first thing they have
>>> to do is make their system less secure?
>>>
>>> -Jon
>>>
>>>>
>>>> Thanks,
>>>> Robin.
>>>

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-12-08 14:37                 ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-12-08 14:37 UTC (permalink / raw)
  To: Jon Nettleton, Lorenzo Pieralisi
  Cc: Linuxarm, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon,
	linux-arm-kernel

Jon,

On 2021-12-08 13:26, Jon Nettleton wrote:
[...]
>>> Even marking them as IOMMU_READ/WRITE is as much of an assumption as
>>> using IOMMU_MMIO or IOMMU_CACHE. It just seems IOMMU_MMIO is the most
>>> popular since all the examples use it for MSI doorbells in the
>>> documentation.
>>
>> We don't merge code based on assumptions that can easily break because
>> the specifications don't contemplate the details that are required.
>>
>>> I am interested why this concern is only being brought up at this point
>>> on a patchset that has been on the mailing list for 8+ months?
>>
>> See above. We don't merge code that we know can break and is based on
>> assumptions, we need to update the IORT specifications to make them
>> cover all the use cases - in a predictable way - and that's what we are
>> working on.
> 
> This is not really an answer to the question.  The latest version of the
> IORT RMR spec was published in Feb 2021. Why was this issue not
> brought up with Rev 1 of this patchset? Instead you have wasted
> 10 months of developer and customer time. This could have easily been
> turned into a code first spec change request, which is a valid option
> for ACPI changes.

It was only on v5 of the patchset - *six months* after the original RFC 
posting - that anyone even first started to question the initial 
assumptions made about attributes[1], and even then somebody familiar 
countered that it didn't appear to matter[2]. Sorry, but you don't get 
to U-turn and throw unjust shade at Arm for not being prescient.

Yes, when those of us within Arm set out the initial RMR spec, an 
assumption was made that it seemed reasonable for an OS to simply pick 
some default strong memory type (Device or Normal-NC) and full 
permissions if it did need to map RMRs at stage 1. That spec was 
reviewed and published externally and no interested parties came forth 
asking "hey, what about attributes?". Linux patches were written around 
that assumption and proceeded through many rounds of review until we 
eventually received sufficient feedback to demonstrate that the 
assumption did not in fact hold well enough in general and there seemed 
to be a genuine need for RMR attributes, and at that point we started 
work on revising the spec.

In the meantime, these patches have sat at v7 for four months - the 
*other* outstanding review comments have not been addressed; I still 
don't recall seeing an answer about whether LX2160 or anything else 
currently deployed actually *needs* cacheable mappings or whether it 
could muddle through with the IOMMU_MMIO assumption until proper "RMR 
v2" support arrived later; even if so, an interim workaround specific to 
LX2160 could have been proposed but hasn't. It is hardly reasonable to 
pretend that Arm or the upstream maintainers are responsible for a lack 
of development activity on the part of the submitters, no matter how 
much blatant misinformation is repeated on Twitter.

Regards,
Robin.

[1] 
https://lore.kernel.org/linux-iommu/13c2499e-cc0c-d395-0d60-6c3437f206ac@nxp.com/
[2] 
https://lore.kernel.org/linux-iommu/CABdtJHv2QBHNoWTyp51H-J_apc75imPj0FbrV70Tm8xuNjpiTA@mail.gmail.com/

>>
>>> This is based on a spec that has existed from Arm since 2020 with the
>>> most recent revisions published in Feb 2021.  The lack of RMR support
>>> in the kernel is affecting real world products, and the ability for
>>> SystemReady ES certified systems from just fully working with recent
>>> distributions.
>>
>> I answered above - if you have any questions please ask them, here,
>> as far as Linux code is concerned.
>>
>> I understand this is taking a long time, it is also helping us
>> understand all the possible use cases and how to cover them in
>> a way that is maintainable in the long run.
> 
> Every month that this patchset has sat being unattended by the
> maintainers is another kernel dev cycle missed, it is another
> another distribution release where users need to add hackish
> kernel command-line options to disable security features that
> were forced on by default. Not to mention Linux is just one
> platform. What if other platforms have already adopted the
> existing spec? These are Arm specs and Arm maintainers and
> yet nobody seems to agree on anything and absolutely nothing
> has been achieved except wasting the time of Shameer, myself,
> our companies, and our customers.
> 
> -Jon
> 
>>
>> Thanks,
>> Lorenzo
>>
>>> Even worse, is that without this patchset customers are forced to jump
>>> through hoops to purposefully re-enable smmu bypass making their
>>> systems less secure.
>>>
>>> How is this a good experience for customers of SystemReady hardware
>>> when for any mainline distribution to work the first thing they have
>>> to do is make their system less secure?
>>>
>>> -Jon
>>>
>>>>
>>>> Thanks,
>>>> Robin.
>>>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-12-08 14:37                 ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2021-12-08 14:37 UTC (permalink / raw)
  To: Jon Nettleton, Lorenzo Pieralisi
  Cc: Shameer Kolothum, linux-arm-kernel, ACPI Devel Maling List,
	Linux IOMMU, Linuxarm, Steven Price, Hanjun Guo, yangyicong,
	Sami Mujawar, Will Deacon, wanghuiqiang

Jon,

On 2021-12-08 13:26, Jon Nettleton wrote:
[...]
>>> Even marking them as IOMMU_READ/WRITE is as much of an assumption as
>>> using IOMMU_MMIO or IOMMU_CACHE. It just seems IOMMU_MMIO is the most
>>> popular since all the examples use it for MSI doorbells in the
>>> documentation.
>>
>> We don't merge code based on assumptions that can easily break because
>> the specifications don't contemplate the details that are required.
>>
>>> I am interested why this concern is only being brought up at this point
>>> on a patchset that has been on the mailing list for 8+ months?
>>
>> See above. We don't merge code that we know can break and is based on
>> assumptions, we need to update the IORT specifications to make them
>> cover all the use cases - in a predictable way - and that's what we are
>> working on.
> 
> This is not really an answer to the question.  The latest version of the
> IORT RMR spec was published in Feb 2021. Why was this issue not
> brought up with Rev 1 of this patchset? Instead you have wasted
> 10 months of developer and customer time. This could have easily been
> turned into a code first spec change request, which is a valid option
> for ACPI changes.

It was only on v5 of the patchset - *six months* after the original RFC 
posting - that anyone even first started to question the initial 
assumptions made about attributes[1], and even then somebody familiar 
countered that it didn't appear to matter[2]. Sorry, but you don't get 
to U-turn and throw unjust shade at Arm for not being prescient.

Yes, when those of us within Arm set out the initial RMR spec, an 
assumption was made that it seemed reasonable for an OS to simply pick 
some default strong memory type (Device or Normal-NC) and full 
permissions if it did need to map RMRs at stage 1. That spec was 
reviewed and published externally and no interested parties came forth 
asking "hey, what about attributes?". Linux patches were written around 
that assumption and proceeded through many rounds of review until we 
eventually received sufficient feedback to demonstrate that the 
assumption did not in fact hold well enough in general and there seemed 
to be a genuine need for RMR attributes, and at that point we started 
work on revising the spec.

In the meantime, these patches have sat at v7 for four months - the 
*other* outstanding review comments have not been addressed; I still 
don't recall seeing an answer about whether LX2160 or anything else 
currently deployed actually *needs* cacheable mappings or whether it 
could muddle through with the IOMMU_MMIO assumption until proper "RMR 
v2" support arrived later; even if so, an interim workaround specific to 
LX2160 could have been proposed but hasn't. It is hardly reasonable to 
pretend that Arm or the upstream maintainers are responsible for a lack 
of development activity on the part of the submitters, no matter how 
much blatant misinformation is repeated on Twitter.

Regards,
Robin.

[1] 
https://lore.kernel.org/linux-iommu/13c2499e-cc0c-d395-0d60-6c3437f206ac@nxp.com/
[2] 
https://lore.kernel.org/linux-iommu/CABdtJHv2QBHNoWTyp51H-J_apc75imPj0FbrV70Tm8xuNjpiTA@mail.gmail.com/

>>
>>> This is based on a spec that has existed from Arm since 2020 with the
>>> most recent revisions published in Feb 2021.  The lack of RMR support
>>> in the kernel is affecting real world products, and the ability for
>>> SystemReady ES certified systems from just fully working with recent
>>> distributions.
>>
>> I answered above - if you have any questions please ask them, here,
>> as far as Linux code is concerned.
>>
>> I understand this is taking a long time, it is also helping us
>> understand all the possible use cases and how to cover them in
>> a way that is maintainable in the long run.
> 
> Every month that this patchset has sat being unattended by the
> maintainers is another kernel dev cycle missed, it is another
> another distribution release where users need to add hackish
> kernel command-line options to disable security features that
> were forced on by default. Not to mention Linux is just one
> platform. What if other platforms have already adopted the
> existing spec? These are Arm specs and Arm maintainers and
> yet nobody seems to agree on anything and absolutely nothing
> has been achieved except wasting the time of Shameer, myself,
> our companies, and our customers.
> 
> -Jon
> 
>>
>> Thanks,
>> Lorenzo
>>
>>> Even worse, is that without this patchset customers are forced to jump
>>> through hoops to purposefully re-enable smmu bypass making their
>>> systems less secure.
>>>
>>> How is this a good experience for customers of SystemReady hardware
>>> when for any mainline distribution to work the first thing they have
>>> to do is make their system less secure?
>>>
>>> -Jon
>>>
>>>>
>>>> Thanks,
>>>> Robin.
>>>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
  2021-12-08 14:37                 ` Robin Murphy
  (?)
@ 2021-12-08 15:11                   ` Jon Nettleton
  -1 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-12-08 15:11 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Lorenzo Pieralisi, Shameer Kolothum, linux-arm-kernel,
	ACPI Devel Maling List, Linux IOMMU, Linuxarm, Steven Price,
	Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon, wanghuiqiang

On Wed, Dec 8, 2021 at 3:37 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> Jon,
>
> On 2021-12-08 13:26, Jon Nettleton wrote:
> [...]
> >>> Even marking them as IOMMU_READ/WRITE is as much of an assumption as
> >>> using IOMMU_MMIO or IOMMU_CACHE. It just seems IOMMU_MMIO is the most
> >>> popular since all the examples use it for MSI doorbells in the
> >>> documentation.
> >>
> >> We don't merge code based on assumptions that can easily break because
> >> the specifications don't contemplate the details that are required.
> >>
> >>> I am interested why this concern is only being brought up at this point
> >>> on a patchset that has been on the mailing list for 8+ months?
> >>
> >> See above. We don't merge code that we know can break and is based on
> >> assumptions, we need to update the IORT specifications to make them
> >> cover all the use cases - in a predictable way - and that's what we are
> >> working on.
> >
> > This is not really an answer to the question.  The latest version of the
> > IORT RMR spec was published in Feb 2021. Why was this issue not
> > brought up with Rev 1 of this patchset? Instead you have wasted
> > 10 months of developer and customer time. This could have easily been
> > turned into a code first spec change request, which is a valid option
> > for ACPI changes.
>
> It was only on v5 of the patchset - *six months* after the original RFC
> posting - that anyone even first started to question the initial
> assumptions made about attributes[1], and even then somebody familiar
> countered that it didn't appear to matter[2]. Sorry, but you don't get
> to U-turn and throw unjust shade at Arm for not being prescient.
>
> Yes, when those of us within Arm set out the initial RMR spec, an
> assumption was made that it seemed reasonable for an OS to simply pick
> some default strong memory type (Device or Normal-NC) and full
> permissions if it did need to map RMRs at stage 1. That spec was
> reviewed and published externally and no interested parties came forth
> asking "hey, what about attributes?". Linux patches were written around
> that assumption and proceeded through many rounds of review until we
> eventually received sufficient feedback to demonstrate that the
> assumption did not in fact hold well enough in general and there seemed
> to be a genuine need for RMR attributes, and at that point we started
> work on revising the spec.

Was it documented anywhere that the RMR spec mandated Device
or Normal-NC memory attributes? I have read through the spec
pretty thoroughly and not seen this requirement documented
anywhere. Also please feel free to point out where we can find information
regarding how the spec is being revised. I am on causeway and in
all the SC meetings and haven't seen this topic brought up at all.

>
> In the meantime, these patches have sat at v7 for four months - the
> *other* outstanding review comments have not been addressed; I still
> don't recall seeing an answer about whether LX2160 or anything else
> currently deployed actually *needs* cacheable mappings or whether it
> could muddle through with the IOMMU_MMIO assumption until proper "RMR
> v2" support arrived later; even if so, an interim workaround specific to
> LX2160 could have been proposed but hasn't. It is hardly reasonable to
> pretend that Arm or the upstream maintainers are responsible for a lack
> of development activity on the part of the submitters, no matter how
> much blatant misinformation is repeated on Twitter.

Oh the "other" comments where after 7 series of patches you decided
that the approach that was agreed upon on the mailing list was no longer
to your liking? Not to mention the month the patchset sat idle after initial
comments from Ard that were cleared up, when I pinged the thread
and it was ignored. If there was some sort of prompt response on the
threads by the maintainers with accurate information about why the
patches were being held up, or that they were working on a new
spec maybe developers would have bothered to push the patchset forward.
There is no misinformation on Twitter.  After 7 series on a patchset after
initial discussion that it would be designed so device-tree could leverage
the backend work, you just changed your mind, and basically sent
everything back to the start.  Meanwhile only now in this thread are we
finding out that the spec is getting re-worked again, which means that
we will need to update our firmware, and wait for someone to write patches
for the new spec, because guess what?... Arm didn't write the patches
for any of the initial specs.

Arm maintainers should be helping to find ways to get Arm "specifications",
integrated for SystemReady customers to use.  Instead we get delayed
responses, about right turns on the path of the patches, or just outright
ignored for a month at a time. Maybe if Arm developers had actually
written the patches when the spec was released there wouldn't have been
almost a year of wasted time by the hardware manufacturers actually
trying to build and deploy products.

-Jon

>
> Regards,
> Robin.
>
> [1]
> https://lore.kernel.org/linux-iommu/13c2499e-cc0c-d395-0d60-6c3437f206ac@nxp.com/
> [2]
> https://lore.kernel.org/linux-iommu/CABdtJHv2QBHNoWTyp51H-J_apc75imPj0FbrV70Tm8xuNjpiTA@mail.gmail.com/
>
> >>
> >>> This is based on a spec that has existed from Arm since 2020 with the
> >>> most recent revisions published in Feb 2021.  The lack of RMR support
> >>> in the kernel is affecting real world products, and the ability for
> >>> SystemReady ES certified systems from just fully working with recent
> >>> distributions.
> >>
> >> I answered above - if you have any questions please ask them, here,
> >> as far as Linux code is concerned.
> >>
> >> I understand this is taking a long time, it is also helping us
> >> understand all the possible use cases and how to cover them in
> >> a way that is maintainable in the long run.
> >
> > Every month that this patchset has sat being unattended by the
> > maintainers is another kernel dev cycle missed, it is another
> > another distribution release where users need to add hackish
> > kernel command-line options to disable security features that
> > were forced on by default. Not to mention Linux is just one
> > platform. What if other platforms have already adopted the
> > existing spec? These are Arm specs and Arm maintainers and
> > yet nobody seems to agree on anything and absolutely nothing
> > has been achieved except wasting the time of Shameer, myself,
> > our companies, and our customers.
> >
> > -Jon
> >
> >>
> >> Thanks,
> >> Lorenzo
> >>
> >>> Even worse, is that without this patchset customers are forced to jump
> >>> through hoops to purposefully re-enable smmu bypass making their
> >>> systems less secure.
> >>>
> >>> How is this a good experience for customers of SystemReady hardware
> >>> when for any mainline distribution to work the first thing they have
> >>> to do is make their system less secure?
> >>>
> >>> -Jon
> >>>
> >>>>
> >>>> Thanks,
> >>>> Robin.
> >>>

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-12-08 15:11                   ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-12-08 15:11 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Linuxarm, Steven Price, ACPI Devel Maling List, Linux IOMMU,
	wanghuiqiang, Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon,
	linux-arm-kernel

On Wed, Dec 8, 2021 at 3:37 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> Jon,
>
> On 2021-12-08 13:26, Jon Nettleton wrote:
> [...]
> >>> Even marking them as IOMMU_READ/WRITE is as much of an assumption as
> >>> using IOMMU_MMIO or IOMMU_CACHE. It just seems IOMMU_MMIO is the most
> >>> popular since all the examples use it for MSI doorbells in the
> >>> documentation.
> >>
> >> We don't merge code based on assumptions that can easily break because
> >> the specifications don't contemplate the details that are required.
> >>
> >>> I am interested why this concern is only being brought up at this point
> >>> on a patchset that has been on the mailing list for 8+ months?
> >>
> >> See above. We don't merge code that we know can break and is based on
> >> assumptions, we need to update the IORT specifications to make them
> >> cover all the use cases - in a predictable way - and that's what we are
> >> working on.
> >
> > This is not really an answer to the question.  The latest version of the
> > IORT RMR spec was published in Feb 2021. Why was this issue not
> > brought up with Rev 1 of this patchset? Instead you have wasted
> > 10 months of developer and customer time. This could have easily been
> > turned into a code first spec change request, which is a valid option
> > for ACPI changes.
>
> It was only on v5 of the patchset - *six months* after the original RFC
> posting - that anyone even first started to question the initial
> assumptions made about attributes[1], and even then somebody familiar
> countered that it didn't appear to matter[2]. Sorry, but you don't get
> to U-turn and throw unjust shade at Arm for not being prescient.
>
> Yes, when those of us within Arm set out the initial RMR spec, an
> assumption was made that it seemed reasonable for an OS to simply pick
> some default strong memory type (Device or Normal-NC) and full
> permissions if it did need to map RMRs at stage 1. That spec was
> reviewed and published externally and no interested parties came forth
> asking "hey, what about attributes?". Linux patches were written around
> that assumption and proceeded through many rounds of review until we
> eventually received sufficient feedback to demonstrate that the
> assumption did not in fact hold well enough in general and there seemed
> to be a genuine need for RMR attributes, and at that point we started
> work on revising the spec.

Was it documented anywhere that the RMR spec mandated Device
or Normal-NC memory attributes? I have read through the spec
pretty thoroughly and not seen this requirement documented
anywhere. Also please feel free to point out where we can find information
regarding how the spec is being revised. I am on causeway and in
all the SC meetings and haven't seen this topic brought up at all.

>
> In the meantime, these patches have sat at v7 for four months - the
> *other* outstanding review comments have not been addressed; I still
> don't recall seeing an answer about whether LX2160 or anything else
> currently deployed actually *needs* cacheable mappings or whether it
> could muddle through with the IOMMU_MMIO assumption until proper "RMR
> v2" support arrived later; even if so, an interim workaround specific to
> LX2160 could have been proposed but hasn't. It is hardly reasonable to
> pretend that Arm or the upstream maintainers are responsible for a lack
> of development activity on the part of the submitters, no matter how
> much blatant misinformation is repeated on Twitter.

Oh the "other" comments where after 7 series of patches you decided
that the approach that was agreed upon on the mailing list was no longer
to your liking? Not to mention the month the patchset sat idle after initial
comments from Ard that were cleared up, when I pinged the thread
and it was ignored. If there was some sort of prompt response on the
threads by the maintainers with accurate information about why the
patches were being held up, or that they were working on a new
spec maybe developers would have bothered to push the patchset forward.
There is no misinformation on Twitter.  After 7 series on a patchset after
initial discussion that it would be designed so device-tree could leverage
the backend work, you just changed your mind, and basically sent
everything back to the start.  Meanwhile only now in this thread are we
finding out that the spec is getting re-worked again, which means that
we will need to update our firmware, and wait for someone to write patches
for the new spec, because guess what?... Arm didn't write the patches
for any of the initial specs.

Arm maintainers should be helping to find ways to get Arm "specifications",
integrated for SystemReady customers to use.  Instead we get delayed
responses, about right turns on the path of the patches, or just outright
ignored for a month at a time. Maybe if Arm developers had actually
written the patches when the spec was released there wouldn't have been
almost a year of wasted time by the hardware manufacturers actually
trying to build and deploy products.

-Jon

>
> Regards,
> Robin.
>
> [1]
> https://lore.kernel.org/linux-iommu/13c2499e-cc0c-d395-0d60-6c3437f206ac@nxp.com/
> [2]
> https://lore.kernel.org/linux-iommu/CABdtJHv2QBHNoWTyp51H-J_apc75imPj0FbrV70Tm8xuNjpiTA@mail.gmail.com/
>
> >>
> >>> This is based on a spec that has existed from Arm since 2020 with the
> >>> most recent revisions published in Feb 2021.  The lack of RMR support
> >>> in the kernel is affecting real world products, and the ability for
> >>> SystemReady ES certified systems from just fully working with recent
> >>> distributions.
> >>
> >> I answered above - if you have any questions please ask them, here,
> >> as far as Linux code is concerned.
> >>
> >> I understand this is taking a long time, it is also helping us
> >> understand all the possible use cases and how to cover them in
> >> a way that is maintainable in the long run.
> >
> > Every month that this patchset has sat being unattended by the
> > maintainers is another kernel dev cycle missed, it is another
> > another distribution release where users need to add hackish
> > kernel command-line options to disable security features that
> > were forced on by default. Not to mention Linux is just one
> > platform. What if other platforms have already adopted the
> > existing spec? These are Arm specs and Arm maintainers and
> > yet nobody seems to agree on anything and absolutely nothing
> > has been achieved except wasting the time of Shameer, myself,
> > our companies, and our customers.
> >
> > -Jon
> >
> >>
> >> Thanks,
> >> Lorenzo
> >>
> >>> Even worse, is that without this patchset customers are forced to jump
> >>> through hoops to purposefully re-enable smmu bypass making their
> >>> systems less secure.
> >>>
> >>> How is this a good experience for customers of SystemReady hardware
> >>> when for any mainline distribution to work the first thing they have
> >>> to do is make their system less secure?
> >>>
> >>> -Jon
> >>>
> >>>>
> >>>> Thanks,
> >>>> Robin.
> >>>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing
@ 2021-12-08 15:11                   ` Jon Nettleton
  0 siblings, 0 replies; 150+ messages in thread
From: Jon Nettleton @ 2021-12-08 15:11 UTC (permalink / raw)
  To: Robin Murphy
  Cc: Lorenzo Pieralisi, Shameer Kolothum, linux-arm-kernel,
	ACPI Devel Maling List, Linux IOMMU, Linuxarm, Steven Price,
	Hanjun Guo, yangyicong, Sami Mujawar, Will Deacon, wanghuiqiang

On Wed, Dec 8, 2021 at 3:37 PM Robin Murphy <robin.murphy@arm.com> wrote:
>
> Jon,
>
> On 2021-12-08 13:26, Jon Nettleton wrote:
> [...]
> >>> Even marking them as IOMMU_READ/WRITE is as much of an assumption as
> >>> using IOMMU_MMIO or IOMMU_CACHE. It just seems IOMMU_MMIO is the most
> >>> popular since all the examples use it for MSI doorbells in the
> >>> documentation.
> >>
> >> We don't merge code based on assumptions that can easily break because
> >> the specifications don't contemplate the details that are required.
> >>
> >>> I am interested why this concern is only being brought up at this point
> >>> on a patchset that has been on the mailing list for 8+ months?
> >>
> >> See above. We don't merge code that we know can break and is based on
> >> assumptions, we need to update the IORT specifications to make them
> >> cover all the use cases - in a predictable way - and that's what we are
> >> working on.
> >
> > This is not really an answer to the question.  The latest version of the
> > IORT RMR spec was published in Feb 2021. Why was this issue not
> > brought up with Rev 1 of this patchset? Instead you have wasted
> > 10 months of developer and customer time. This could have easily been
> > turned into a code first spec change request, which is a valid option
> > for ACPI changes.
>
> It was only on v5 of the patchset - *six months* after the original RFC
> posting - that anyone even first started to question the initial
> assumptions made about attributes[1], and even then somebody familiar
> countered that it didn't appear to matter[2]. Sorry, but you don't get
> to U-turn and throw unjust shade at Arm for not being prescient.
>
> Yes, when those of us within Arm set out the initial RMR spec, an
> assumption was made that it seemed reasonable for an OS to simply pick
> some default strong memory type (Device or Normal-NC) and full
> permissions if it did need to map RMRs at stage 1. That spec was
> reviewed and published externally and no interested parties came forth
> asking "hey, what about attributes?". Linux patches were written around
> that assumption and proceeded through many rounds of review until we
> eventually received sufficient feedback to demonstrate that the
> assumption did not in fact hold well enough in general and there seemed
> to be a genuine need for RMR attributes, and at that point we started
> work on revising the spec.

Was it documented anywhere that the RMR spec mandated Device
or Normal-NC memory attributes? I have read through the spec
pretty thoroughly and not seen this requirement documented
anywhere. Also please feel free to point out where we can find information
regarding how the spec is being revised. I am on causeway and in
all the SC meetings and haven't seen this topic brought up at all.

>
> In the meantime, these patches have sat at v7 for four months - the
> *other* outstanding review comments have not been addressed; I still
> don't recall seeing an answer about whether LX2160 or anything else
> currently deployed actually *needs* cacheable mappings or whether it
> could muddle through with the IOMMU_MMIO assumption until proper "RMR
> v2" support arrived later; even if so, an interim workaround specific to
> LX2160 could have been proposed but hasn't. It is hardly reasonable to
> pretend that Arm or the upstream maintainers are responsible for a lack
> of development activity on the part of the submitters, no matter how
> much blatant misinformation is repeated on Twitter.

Oh the "other" comments where after 7 series of patches you decided
that the approach that was agreed upon on the mailing list was no longer
to your liking? Not to mention the month the patchset sat idle after initial
comments from Ard that were cleared up, when I pinged the thread
and it was ignored. If there was some sort of prompt response on the
threads by the maintainers with accurate information about why the
patches were being held up, or that they were working on a new
spec maybe developers would have bothered to push the patchset forward.
There is no misinformation on Twitter.  After 7 series on a patchset after
initial discussion that it would be designed so device-tree could leverage
the backend work, you just changed your mind, and basically sent
everything back to the start.  Meanwhile only now in this thread are we
finding out that the spec is getting re-worked again, which means that
we will need to update our firmware, and wait for someone to write patches
for the new spec, because guess what?... Arm didn't write the patches
for any of the initial specs.

Arm maintainers should be helping to find ways to get Arm "specifications",
integrated for SystemReady customers to use.  Instead we get delayed
responses, about right turns on the path of the patches, or just outright
ignored for a month at a time. Maybe if Arm developers had actually
written the patches when the spec was released there wouldn't have been
almost a year of wasted time by the hardware manufacturers actually
trying to build and deploy products.

-Jon

>
> Regards,
> Robin.
>
> [1]
> https://lore.kernel.org/linux-iommu/13c2499e-cc0c-d395-0d60-6c3437f206ac@nxp.com/
> [2]
> https://lore.kernel.org/linux-iommu/CABdtJHv2QBHNoWTyp51H-J_apc75imPj0FbrV70Tm8xuNjpiTA@mail.gmail.com/
>
> >>
> >>> This is based on a spec that has existed from Arm since 2020 with the
> >>> most recent revisions published in Feb 2021.  The lack of RMR support
> >>> in the kernel is affecting real world products, and the ability for
> >>> SystemReady ES certified systems from just fully working with recent
> >>> distributions.
> >>
> >> I answered above - if you have any questions please ask them, here,
> >> as far as Linux code is concerned.
> >>
> >> I understand this is taking a long time, it is also helping us
> >> understand all the possible use cases and how to cover them in
> >> a way that is maintainable in the long run.
> >
> > Every month that this patchset has sat being unattended by the
> > maintainers is another kernel dev cycle missed, it is another
> > another distribution release where users need to add hackish
> > kernel command-line options to disable security features that
> > were forced on by default. Not to mention Linux is just one
> > platform. What if other platforms have already adopted the
> > existing spec? These are Arm specs and Arm maintainers and
> > yet nobody seems to agree on anything and absolutely nothing
> > has been achieved except wasting the time of Shameer, myself,
> > our companies, and our customers.
> >
> > -Jon
> >
> >>
> >> Thanks,
> >> Lorenzo
> >>
> >>> Even worse, is that without this patchset customers are forced to jump
> >>> through hoops to purposefully re-enable smmu bypass making their
> >>> systems less secure.
> >>>
> >>> How is this a good experience for customers of SystemReady hardware
> >>> when for any mainline distribution to work the first thing they have
> >>> to do is make their system less secure?
> >>>
> >>> -Jon
> >>>
> >>>>
> >>>> Thanks,
> >>>> Robin.
> >>>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
  2021-08-05  8:07 ` Shameer Kolothum
  (?)
@ 2022-01-25 13:00   ` Shameerali Kolothum Thodi
  -1 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi via iommu @ 2022-01-25 13:00 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: will, jon, Linuxarm, steven.price, Guohanjun (Hanjun Guo),
	yangyicong, Sami.Mujawar, robin.murphy, wanghuiqiang

Hi Robin/Lorenzo,

> -----Original Message-----
> From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On Behalf
> Of Shameer Kolothum
> Sent: 05 August 2021 09:07
> To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org
> Cc: robin.murphy@arm.com; jon@solid-run.com; Linuxarm
> <linuxarm@huawei.com>; steven.price@arm.com; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>;
> Sami.Mujawar@arm.com; will@kernel.org; wanghuiqiang
> <wanghuiqiang@huawei.com>
> Subject: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> 
> Hi,
> 
> The series adds support to IORT RMR nodes specified in IORT
> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> memory ranges that are used by endpoints and require a unity
> mapping in SMMU.
> 
> We have faced issues with 3408iMR RAID controller cards which
> fail to boot when SMMU is enabled. This is because these
> controllers make use of host memory for various caching related
> purposes and when SMMU is enabled the iMR firmware fails to
> access these memory regions as there is no mapping for them.
> IORT RMR provides a way for UEFI to describe and report these
> memory regions so that the kernel can make a unity mapping for
> these in SMMU.
> 
> Change History:
> 
> v6 --> v7
> 
> The only change from v6 is the fix pointed out by Steve to
> the SMMUv2 SMR bypass install in patch #8.
> 
> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> yet as the series still needs more review[1].
> 
> Feedback and tests on this series is very much appreciated.

Since we have an update to IORT spec(E.c) now[1] and includes additional
attributes/flags for the RMR node, I am planning to respin this series soon.

Going through the new spec, I have a few queries,

The memory range attributes can now be described as one of the following,

0x00: Device-nGnRnE memory
0x01: Device-nGnRE memory
0x02: Device-nGRE memory
0x03: Device-GRE memory
0x04: Normal Inner Non-cacheable Outer Non-cacheable
0x05: Normal Inner Write-back Outer Write-back Inner Shareable

I am not sure how this needs to be captured and used in the kernel. Is there
any intention of using these fine-grained attributes in the kernel now
or a generic mapping of the above to the struct iommu_rev_region prot field
is enough? i.e., something like,

{
    ....
    prot = IOMMU_READ | IOMMU_WRITE;

    if (rmr_attr == normal_mem) // 0x05
        prot |= IOMMU_CACHE;
    
    if (rmr_attr == device_mem) { //0x00 - 0x03
        prot |= IOMMU_MMIO;
        prot |= IOMMU_NOEXEC;
    }
    ....
}

Similarly for the 'flags' field, the new 'Access Privilege' is intended to set the
IOMMU_PRIV ?
  
Please let me know.

Thanks,
Shameer

[1] https://developer.arm.com/documentation/den0049/ec/?lang=en

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2022-01-25 13:00   ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2022-01-25 13:00 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: robin.murphy, Lorenzo Pieralisi, jon, Linuxarm, steven.price,
	Guohanjun (Hanjun Guo),
	yangyicong, Sami.Mujawar, will, wanghuiqiang, eric.auger,
	Laurentiu Tudor

Hi Robin/Lorenzo,

> -----Original Message-----
> From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On Behalf
> Of Shameer Kolothum
> Sent: 05 August 2021 09:07
> To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org
> Cc: robin.murphy@arm.com; jon@solid-run.com; Linuxarm
> <linuxarm@huawei.com>; steven.price@arm.com; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>;
> Sami.Mujawar@arm.com; will@kernel.org; wanghuiqiang
> <wanghuiqiang@huawei.com>
> Subject: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> 
> Hi,
> 
> The series adds support to IORT RMR nodes specified in IORT
> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> memory ranges that are used by endpoints and require a unity
> mapping in SMMU.
> 
> We have faced issues with 3408iMR RAID controller cards which
> fail to boot when SMMU is enabled. This is because these
> controllers make use of host memory for various caching related
> purposes and when SMMU is enabled the iMR firmware fails to
> access these memory regions as there is no mapping for them.
> IORT RMR provides a way for UEFI to describe and report these
> memory regions so that the kernel can make a unity mapping for
> these in SMMU.
> 
> Change History:
> 
> v6 --> v7
> 
> The only change from v6 is the fix pointed out by Steve to
> the SMMUv2 SMR bypass install in patch #8.
> 
> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> yet as the series still needs more review[1].
> 
> Feedback and tests on this series is very much appreciated.

Since we have an update to IORT spec(E.c) now[1] and includes additional
attributes/flags for the RMR node, I am planning to respin this series soon.

Going through the new spec, I have a few queries,

The memory range attributes can now be described as one of the following,

0x00: Device-nGnRnE memory
0x01: Device-nGnRE memory
0x02: Device-nGRE memory
0x03: Device-GRE memory
0x04: Normal Inner Non-cacheable Outer Non-cacheable
0x05: Normal Inner Write-back Outer Write-back Inner Shareable

I am not sure how this needs to be captured and used in the kernel. Is there
any intention of using these fine-grained attributes in the kernel now
or a generic mapping of the above to the struct iommu_rev_region prot field
is enough? i.e., something like,

{
    ....
    prot = IOMMU_READ | IOMMU_WRITE;

    if (rmr_attr == normal_mem) // 0x05
        prot |= IOMMU_CACHE;
    
    if (rmr_attr == device_mem) { //0x00 - 0x03
        prot |= IOMMU_MMIO;
        prot |= IOMMU_NOEXEC;
    }
    ....
}

Similarly for the 'flags' field, the new 'Access Privilege' is intended to set the
IOMMU_PRIV ?
  
Please let me know.

Thanks,
Shameer

[1] https://developer.arm.com/documentation/den0049/ec/?lang=en

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

* RE: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2022-01-25 13:00   ` Shameerali Kolothum Thodi
  0 siblings, 0 replies; 150+ messages in thread
From: Shameerali Kolothum Thodi @ 2022-01-25 13:00 UTC (permalink / raw)
  To: linux-arm-kernel, linux-acpi, iommu
  Cc: robin.murphy, Lorenzo Pieralisi, jon, Linuxarm, steven.price,
	Guohanjun (Hanjun Guo),
	yangyicong, Sami.Mujawar, will, wanghuiqiang, eric.auger,
	Laurentiu Tudor

Hi Robin/Lorenzo,

> -----Original Message-----
> From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On Behalf
> Of Shameer Kolothum
> Sent: 05 August 2021 09:07
> To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org
> Cc: robin.murphy@arm.com; jon@solid-run.com; Linuxarm
> <linuxarm@huawei.com>; steven.price@arm.com; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>;
> Sami.Mujawar@arm.com; will@kernel.org; wanghuiqiang
> <wanghuiqiang@huawei.com>
> Subject: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> 
> Hi,
> 
> The series adds support to IORT RMR nodes specified in IORT
> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> memory ranges that are used by endpoints and require a unity
> mapping in SMMU.
> 
> We have faced issues with 3408iMR RAID controller cards which
> fail to boot when SMMU is enabled. This is because these
> controllers make use of host memory for various caching related
> purposes and when SMMU is enabled the iMR firmware fails to
> access these memory regions as there is no mapping for them.
> IORT RMR provides a way for UEFI to describe and report these
> memory regions so that the kernel can make a unity mapping for
> these in SMMU.
> 
> Change History:
> 
> v6 --> v7
> 
> The only change from v6 is the fix pointed out by Steve to
> the SMMUv2 SMR bypass install in patch #8.
> 
> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> yet as the series still needs more review[1].
> 
> Feedback and tests on this series is very much appreciated.

Since we have an update to IORT spec(E.c) now[1] and includes additional
attributes/flags for the RMR node, I am planning to respin this series soon.

Going through the new spec, I have a few queries,

The memory range attributes can now be described as one of the following,

0x00: Device-nGnRnE memory
0x01: Device-nGnRE memory
0x02: Device-nGRE memory
0x03: Device-GRE memory
0x04: Normal Inner Non-cacheable Outer Non-cacheable
0x05: Normal Inner Write-back Outer Write-back Inner Shareable

I am not sure how this needs to be captured and used in the kernel. Is there
any intention of using these fine-grained attributes in the kernel now
or a generic mapping of the above to the struct iommu_rev_region prot field
is enough? i.e., something like,

{
    ....
    prot = IOMMU_READ | IOMMU_WRITE;

    if (rmr_attr == normal_mem) // 0x05
        prot |= IOMMU_CACHE;
    
    if (rmr_attr == device_mem) { //0x00 - 0x03
        prot |= IOMMU_MMIO;
        prot |= IOMMU_NOEXEC;
    }
    ....
}

Similarly for the 'flags' field, the new 'Access Privilege' is intended to set the
IOMMU_PRIV ?
  
Please let me know.

Thanks,
Shameer

[1] https://developer.arm.com/documentation/den0049/ec/?lang=en


^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
  2022-01-25 13:00   ` Shameerali Kolothum Thodi
  (?)
@ 2022-01-25 19:30     ` Robin Murphy
  -1 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2022-01-25 19:30 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi, linux-arm-kernel, linux-acpi, iommu
  Cc: Lorenzo Pieralisi, jon, Linuxarm, steven.price,
	Guohanjun (Hanjun Guo),
	yangyicong, Sami.Mujawar, will, wanghuiqiang, eric.auger,
	Laurentiu Tudor

On 2022-01-25 13:00, Shameerali Kolothum Thodi wrote:
> Hi Robin/Lorenzo,
> 
>> -----Original Message-----
>> From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On Behalf
>> Of Shameer Kolothum
>> Sent: 05 August 2021 09:07
>> To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
>> iommu@lists.linux-foundation.org
>> Cc: robin.murphy@arm.com; jon@solid-run.com; Linuxarm
>> <linuxarm@huawei.com>; steven.price@arm.com; Guohanjun (Hanjun Guo)
>> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>;
>> Sami.Mujawar@arm.com; will@kernel.org; wanghuiqiang
>> <wanghuiqiang@huawei.com>
>> Subject: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
>>
>> Hi,
>>
>> The series adds support to IORT RMR nodes specified in IORT
>> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
>> memory ranges that are used by endpoints and require a unity
>> mapping in SMMU.
>>
>> We have faced issues with 3408iMR RAID controller cards which
>> fail to boot when SMMU is enabled. This is because these
>> controllers make use of host memory for various caching related
>> purposes and when SMMU is enabled the iMR firmware fails to
>> access these memory regions as there is no mapping for them.
>> IORT RMR provides a way for UEFI to describe and report these
>> memory regions so that the kernel can make a unity mapping for
>> these in SMMU.
>>
>> Change History:
>>
>> v6 --> v7
>>
>> The only change from v6 is the fix pointed out by Steve to
>> the SMMUv2 SMR bypass install in patch #8.
>>
>> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
>> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
>> yet as the series still needs more review[1].
>>
>> Feedback and tests on this series is very much appreciated.
> 
> Since we have an update to IORT spec(E.c) now[1] and includes additional
> attributes/flags for the RMR node, I am planning to respin this series soon.
> 
> Going through the new spec, I have a few queries,
> 
> The memory range attributes can now be described as one of the following,
> 
> 0x00: Device-nGnRnE memory
> 0x01: Device-nGnRE memory
> 0x02: Device-nGRE memory
> 0x03: Device-GRE memory
> 0x04: Normal Inner Non-cacheable Outer Non-cacheable
> 0x05: Normal Inner Write-back Outer Write-back Inner Shareable
> 
> I am not sure how this needs to be captured and used in the kernel. Is there
> any intention of using these fine-grained attributes in the kernel now
> or a generic mapping of the above to the struct iommu_rev_region prot field
> is enough? i.e., something like,
> 
> {
>      ....
>      prot = IOMMU_READ | IOMMU_WRITE;
> 
>      if (rmr_attr == normal_mem) // 0x05
>          prot |= IOMMU_CACHE;
>      
>      if (rmr_attr == device_mem) { //0x00 - 0x03
>          prot |= IOMMU_MMIO;
>          prot |= IOMMU_NOEXEC;
>      }
>      ....
> }

Yup, pretty much that, except don't bother with IOMMU_NOEXEC. We can't 
reliably infer it - e.g. on an AXI-based interconnect AxCACHE and AxPROT 
are entirely orthogonal, so a Device-type read with the "Instruction 
access" hint is perfectly legal - and in the common IORT code we're not 
in a position to second-guess what any given RMR might represent for 
whatever agent is accessing it.

All we can reasonably do here is map the Device types to IOMMU_MMIO and 
Write-back to IOMMU_CACHE, and if anyone ever does want to insist that 
that's not sufficient, then they're welcome to send patches to make the 
IOMMU API more expressive :)

> Similarly for the 'flags' field, the new 'Access Privilege' is intended to set the
> IOMMU_PRIV ?

Yes, exactly!

Cheers,
Robin.

>    
> Please let me know.
> 
> Thanks,
> Shameer
> 
> [1] https://developer.arm.com/documentation/den0049/ec/?lang=en
> 

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2022-01-25 19:30     ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2022-01-25 19:30 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi, linux-arm-kernel, linux-acpi, iommu
  Cc: jon, Linuxarm, steven.price, Guohanjun (Hanjun Guo),
	yangyicong, Sami.Mujawar, will, wanghuiqiang

On 2022-01-25 13:00, Shameerali Kolothum Thodi wrote:
> Hi Robin/Lorenzo,
> 
>> -----Original Message-----
>> From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On Behalf
>> Of Shameer Kolothum
>> Sent: 05 August 2021 09:07
>> To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
>> iommu@lists.linux-foundation.org
>> Cc: robin.murphy@arm.com; jon@solid-run.com; Linuxarm
>> <linuxarm@huawei.com>; steven.price@arm.com; Guohanjun (Hanjun Guo)
>> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>;
>> Sami.Mujawar@arm.com; will@kernel.org; wanghuiqiang
>> <wanghuiqiang@huawei.com>
>> Subject: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
>>
>> Hi,
>>
>> The series adds support to IORT RMR nodes specified in IORT
>> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
>> memory ranges that are used by endpoints and require a unity
>> mapping in SMMU.
>>
>> We have faced issues with 3408iMR RAID controller cards which
>> fail to boot when SMMU is enabled. This is because these
>> controllers make use of host memory for various caching related
>> purposes and when SMMU is enabled the iMR firmware fails to
>> access these memory regions as there is no mapping for them.
>> IORT RMR provides a way for UEFI to describe and report these
>> memory regions so that the kernel can make a unity mapping for
>> these in SMMU.
>>
>> Change History:
>>
>> v6 --> v7
>>
>> The only change from v6 is the fix pointed out by Steve to
>> the SMMUv2 SMR bypass install in patch #8.
>>
>> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
>> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
>> yet as the series still needs more review[1].
>>
>> Feedback and tests on this series is very much appreciated.
> 
> Since we have an update to IORT spec(E.c) now[1] and includes additional
> attributes/flags for the RMR node, I am planning to respin this series soon.
> 
> Going through the new spec, I have a few queries,
> 
> The memory range attributes can now be described as one of the following,
> 
> 0x00: Device-nGnRnE memory
> 0x01: Device-nGnRE memory
> 0x02: Device-nGRE memory
> 0x03: Device-GRE memory
> 0x04: Normal Inner Non-cacheable Outer Non-cacheable
> 0x05: Normal Inner Write-back Outer Write-back Inner Shareable
> 
> I am not sure how this needs to be captured and used in the kernel. Is there
> any intention of using these fine-grained attributes in the kernel now
> or a generic mapping of the above to the struct iommu_rev_region prot field
> is enough? i.e., something like,
> 
> {
>      ....
>      prot = IOMMU_READ | IOMMU_WRITE;
> 
>      if (rmr_attr == normal_mem) // 0x05
>          prot |= IOMMU_CACHE;
>      
>      if (rmr_attr == device_mem) { //0x00 - 0x03
>          prot |= IOMMU_MMIO;
>          prot |= IOMMU_NOEXEC;
>      }
>      ....
> }

Yup, pretty much that, except don't bother with IOMMU_NOEXEC. We can't 
reliably infer it - e.g. on an AXI-based interconnect AxCACHE and AxPROT 
are entirely orthogonal, so a Device-type read with the "Instruction 
access" hint is perfectly legal - and in the common IORT code we're not 
in a position to second-guess what any given RMR might represent for 
whatever agent is accessing it.

All we can reasonably do here is map the Device types to IOMMU_MMIO and 
Write-back to IOMMU_CACHE, and if anyone ever does want to insist that 
that's not sufficient, then they're welcome to send patches to make the 
IOMMU API more expressive :)

> Similarly for the 'flags' field, the new 'Access Privilege' is intended to set the
> IOMMU_PRIV ?

Yes, exactly!

Cheers,
Robin.

>    
> Please let me know.
> 
> Thanks,
> Shameer
> 
> [1] https://developer.arm.com/documentation/den0049/ec/?lang=en
> 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 150+ messages in thread

* Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
@ 2022-01-25 19:30     ` Robin Murphy
  0 siblings, 0 replies; 150+ messages in thread
From: Robin Murphy @ 2022-01-25 19:30 UTC (permalink / raw)
  To: Shameerali Kolothum Thodi, linux-arm-kernel, linux-acpi, iommu
  Cc: Lorenzo Pieralisi, jon, Linuxarm, steven.price,
	Guohanjun (Hanjun Guo),
	yangyicong, Sami.Mujawar, will, wanghuiqiang, eric.auger,
	Laurentiu Tudor

On 2022-01-25 13:00, Shameerali Kolothum Thodi wrote:
> Hi Robin/Lorenzo,
> 
>> -----Original Message-----
>> From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On Behalf
>> Of Shameer Kolothum
>> Sent: 05 August 2021 09:07
>> To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
>> iommu@lists.linux-foundation.org
>> Cc: robin.murphy@arm.com; jon@solid-run.com; Linuxarm
>> <linuxarm@huawei.com>; steven.price@arm.com; Guohanjun (Hanjun Guo)
>> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>;
>> Sami.Mujawar@arm.com; will@kernel.org; wanghuiqiang
>> <wanghuiqiang@huawei.com>
>> Subject: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
>>
>> Hi,
>>
>> The series adds support to IORT RMR nodes specified in IORT
>> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
>> memory ranges that are used by endpoints and require a unity
>> mapping in SMMU.
>>
>> We have faced issues with 3408iMR RAID controller cards which
>> fail to boot when SMMU is enabled. This is because these
>> controllers make use of host memory for various caching related
>> purposes and when SMMU is enabled the iMR firmware fails to
>> access these memory regions as there is no mapping for them.
>> IORT RMR provides a way for UEFI to describe and report these
>> memory regions so that the kernel can make a unity mapping for
>> these in SMMU.
>>
>> Change History:
>>
>> v6 --> v7
>>
>> The only change from v6 is the fix pointed out by Steve to
>> the SMMUv2 SMR bypass install in patch #8.
>>
>> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
>> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
>> yet as the series still needs more review[1].
>>
>> Feedback and tests on this series is very much appreciated.
> 
> Since we have an update to IORT spec(E.c) now[1] and includes additional
> attributes/flags for the RMR node, I am planning to respin this series soon.
> 
> Going through the new spec, I have a few queries,
> 
> The memory range attributes can now be described as one of the following,
> 
> 0x00: Device-nGnRnE memory
> 0x01: Device-nGnRE memory
> 0x02: Device-nGRE memory
> 0x03: Device-GRE memory
> 0x04: Normal Inner Non-cacheable Outer Non-cacheable
> 0x05: Normal Inner Write-back Outer Write-back Inner Shareable
> 
> I am not sure how this needs to be captured and used in the kernel. Is there
> any intention of using these fine-grained attributes in the kernel now
> or a generic mapping of the above to the struct iommu_rev_region prot field
> is enough? i.e., something like,
> 
> {
>      ....
>      prot = IOMMU_READ | IOMMU_WRITE;
> 
>      if (rmr_attr == normal_mem) // 0x05
>          prot |= IOMMU_CACHE;
>      
>      if (rmr_attr == device_mem) { //0x00 - 0x03
>          prot |= IOMMU_MMIO;
>          prot |= IOMMU_NOEXEC;
>      }
>      ....
> }

Yup, pretty much that, except don't bother with IOMMU_NOEXEC. We can't 
reliably infer it - e.g. on an AXI-based interconnect AxCACHE and AxPROT 
are entirely orthogonal, so a Device-type read with the "Instruction 
access" hint is perfectly legal - and in the common IORT code we're not 
in a position to second-guess what any given RMR might represent for 
whatever agent is accessing it.

All we can reasonably do here is map the Device types to IOMMU_MMIO and 
Write-back to IOMMU_CACHE, and if anyone ever does want to insist that 
that's not sufficient, then they're welcome to send patches to make the 
IOMMU API more expressive :)

> Similarly for the 'flags' field, the new 'Access Privilege' is intended to set the
> IOMMU_PRIV ?

Yes, exactly!

Cheers,
Robin.

>    
> Please let me know.
> 
> Thanks,
> Shameer
> 
> [1] https://developer.arm.com/documentation/den0049/ec/?lang=en
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 150+ messages in thread

end of thread, other threads:[~2022-01-25 19:32 UTC | newest]

Thread overview: 150+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-05  8:07 [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node Shameer Kolothum
2021-08-05  8:07 ` Shameer Kolothum
2021-08-05  8:07 ` Shameer Kolothum
2021-08-05  8:07 ` [PATCH v7 1/9] iommu: Introduce a union to struct iommu_resv_region Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-20 10:22   ` Steven Price
2021-08-20 10:22     ` Steven Price
2021-08-20 10:22     ` Steven Price
2021-10-08 12:14   ` Robin Murphy
2021-10-08 12:14     ` Robin Murphy
2021-10-08 12:14     ` Robin Murphy
2021-10-09  6:57     ` Jon Nettleton
2021-10-09  6:57       ` Jon Nettleton
2021-10-09  6:57       ` Jon Nettleton
2021-10-11  5:47       ` Shameerali Kolothum Thodi
2021-10-11  5:47         ` Shameerali Kolothum Thodi
2021-10-11  5:47         ` Shameerali Kolothum Thodi
2021-10-11 13:47         ` Robin Murphy
2021-10-11 13:47           ` Robin Murphy
2021-10-11 13:47           ` Robin Murphy
2021-08-05  8:07 ` [PATCH v7 2/9] ACPI/IORT: Add support for RMR node parsing Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05 16:03   ` Lorenzo Pieralisi
2021-08-05 16:03     ` Lorenzo Pieralisi
2021-08-05 16:03     ` Lorenzo Pieralisi
2021-08-05 16:31     ` Jon Nettleton
2021-08-05 16:31       ` Jon Nettleton
2021-08-05 16:31       ` Jon Nettleton
2021-08-05 18:37     ` Laurentiu Tudor
2021-08-05 18:37       ` Laurentiu Tudor
2021-08-05 18:37       ` Laurentiu Tudor
2021-09-06 17:44     ` Robin Murphy
2021-09-06 17:44       ` Robin Murphy
2021-09-06 17:44       ` Robin Murphy
2021-09-06 19:51       ` Jon Nettleton
2021-09-06 19:51         ` Jon Nettleton
2021-09-06 19:51         ` Jon Nettleton
2021-09-16  7:26         ` Shameerali Kolothum Thodi
2021-09-16  7:26           ` Shameerali Kolothum Thodi
2021-09-16  7:26           ` Shameerali Kolothum Thodi
2021-09-16  7:52           ` Jon Nettleton
2021-09-16  7:52             ` Jon Nettleton
2021-09-16  7:52             ` Jon Nettleton
2021-09-16  8:26             ` Shameerali Kolothum Thodi
2021-09-16  8:26               ` Shameerali Kolothum Thodi
2021-09-16  8:26               ` Shameerali Kolothum Thodi
2021-09-16 11:16               ` Jon Nettleton
2021-09-16 11:16                 ` Jon Nettleton
2021-09-16 11:16                 ` Jon Nettleton
2021-09-17 11:26                 ` Shameerali Kolothum Thodi
2021-09-17 11:26                   ` Shameerali Kolothum Thodi
2021-09-17 11:26                   ` Shameerali Kolothum Thodi
2021-10-05 10:53                   ` Laurentiu Tudor
2021-10-05 10:53                     ` Laurentiu Tudor
2021-10-05 10:53                     ` Laurentiu Tudor
2021-10-08 12:48   ` Robin Murphy
2021-10-08 12:48     ` Robin Murphy
2021-10-08 12:48     ` Robin Murphy
2021-10-09  7:06     ` Jon Nettleton
2021-10-09  7:06       ` Jon Nettleton
2021-10-09  7:06       ` Jon Nettleton
2021-10-11 14:04       ` Robin Murphy
2021-10-11 14:04         ` Robin Murphy
2021-10-11 14:04         ` Robin Murphy
2021-10-12  8:00         ` Jon Nettleton
2021-10-12  8:00           ` Jon Nettleton
2021-10-12  8:00           ` Jon Nettleton
2021-12-08 12:18           ` Lorenzo Pieralisi
2021-12-08 12:18             ` Lorenzo Pieralisi
2021-12-08 12:18             ` Lorenzo Pieralisi
2021-12-08 13:26             ` Jon Nettleton
2021-12-08 13:26               ` Jon Nettleton
2021-12-08 13:26               ` Jon Nettleton
2021-12-08 14:37               ` Robin Murphy
2021-12-08 14:37                 ` Robin Murphy
2021-12-08 14:37                 ` Robin Murphy
2021-12-08 15:11                 ` Jon Nettleton
2021-12-08 15:11                   ` Jon Nettleton
2021-12-08 15:11                   ` Jon Nettleton
2021-10-11  5:59     ` Shameerali Kolothum Thodi
2021-10-11  5:59       ` Shameerali Kolothum Thodi
2021-10-11  5:59       ` Shameerali Kolothum Thodi
2021-08-05  8:07 ` [PATCH v7 3/9] iommu/dma: Introduce generic helper to retrieve RMR info Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-10-08 13:03   ` Robin Murphy
2021-10-08 13:03     ` Robin Murphy
2021-10-08 13:03     ` Robin Murphy
2021-10-11  5:51     ` Shameerali Kolothum Thodi
2021-10-11  5:51       ` Shameerali Kolothum Thodi
2021-10-11  5:51       ` Shameerali Kolothum Thodi
2021-08-05  8:07 ` [PATCH v7 4/9] ACPI/IORT: Add a helper to retrieve RMR memory regions Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05 15:43   ` Lorenzo Pieralisi
2021-08-05 15:43     ` Lorenzo Pieralisi
2021-08-05 15:43     ` Lorenzo Pieralisi
2021-08-05  8:07 ` [PATCH v7 5/9] iommu/arm-smmu-v3: Introduce strtab init helper Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05  8:07 ` [PATCH v7 6/9] iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force bypass Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05  8:07 ` [PATCH v7 7/9] iommu/arm-smmu-v3: Get associated RMR info and install bypass STE Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05  8:07 ` [PATCH v7 8/9] iommu/arm-smmu: Get associated RMR info and install bypass SMR Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05  8:07 ` [PATCH v7 9/9] iommu/dma: Reserve any RMR regions associated with a dev Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-08-05  8:07   ` Shameer Kolothum
2021-10-08 13:09   ` Robin Murphy
2021-10-08 13:09     ` Robin Murphy
2021-10-08 13:09     ` Robin Murphy
2021-10-09  7:07     ` Jon Nettleton
2021-10-09  7:07       ` Jon Nettleton
2021-10-09  7:07       ` Jon Nettleton
2021-10-11 15:00       ` Robin Murphy
2021-10-11 15:00         ` Robin Murphy
2021-10-11 15:00         ` Robin Murphy
2021-10-11 15:42         ` Shameerali Kolothum Thodi
2021-10-11 15:42           ` Shameerali Kolothum Thodi
2021-10-11 15:42           ` Shameerali Kolothum Thodi
2021-08-05 13:22 ` [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node Ard Biesheuvel
2021-08-05 13:22   ` Ard Biesheuvel
2021-08-05 13:22   ` Ard Biesheuvel
2021-08-05 13:35   ` Shameerali Kolothum Thodi
2021-08-05 13:35     ` Shameerali Kolothum Thodi
2021-08-05 13:35     ` Shameerali Kolothum Thodi
2021-08-05 14:09     ` Ard Biesheuvel
2021-08-05 14:09       ` Ard Biesheuvel
2021-08-05 14:09       ` Ard Biesheuvel
2021-08-31  5:06       ` Jon Nettleton
2021-08-31  5:06         ` Jon Nettleton
2021-08-31  5:06         ` Jon Nettleton
2021-09-30  9:47 ` Eric Auger
2021-09-30  9:47   ` Eric Auger
2021-09-30  9:47   ` Eric Auger
2021-09-30 10:50   ` Shameerali Kolothum Thodi
2021-09-30 10:50     ` Shameerali Kolothum Thodi
2021-09-30 10:50     ` Shameerali Kolothum Thodi
2022-01-25 13:00 ` Shameerali Kolothum Thodi via iommu
2022-01-25 13:00   ` Shameerali Kolothum Thodi
2022-01-25 13:00   ` Shameerali Kolothum Thodi
2022-01-25 19:30   ` Robin Murphy
2022-01-25 19:30     ` Robin Murphy
2022-01-25 19:30     ` Robin Murphy

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.